it's a nice, clean re-usable solution. I like it!

@quba did you have a chance to test it and can confirm the feature?

@mattab What did you think about the second paragraph?

I think using API parameters might be limiting however. After thinking about it, it might be better to make it use regex groups. Then we can add two more options, --ignore-fields=userid, --w3c-field-regex=my-custom-field=myregex so users can import any field they want into a custom variable.

Yes, it could be useful to have this ability but maybe we wait for some use cases? AFAIK it would not invalidate this PR and new options so it could be done later
maybe create a separate issue?

It wouldn't invalidate it, but it would be redundant since you could use the other options to do what's in this use case.

What would be equivalent to --api-arg-to-visit-cvar="uid=User Name" with these options?

--regex-group-to-visit-cvar="userid=User Name"

ok looks nice! btw is there an option to list the regex groups available for a given log file?

It would be for a format, I was planning on adding the info to the help documentation.

ok looks nice!

I'll make the changes, please don't merge until I finish them.

@mattab I've modified the pull request, new options now include:

--regex-group-to-visit-cvar
--regex-group-to-page-cvar
--ignore-fields
--w3c-field-regex

First 3 are for @quba's use case. Last one can be combined w/ other ones to track any w3c log field that the importer doesn't natively recognize in a custom variable.

Let me know what you think.

@diosmosis how can I choose which custom variable will be used?

You mean which index? Or what the custom variable name to use? For name, --regex-group-to-visit-cvar="userid=User Name". I don't think index matters.

after a quick review, looks good to me! 👍

@diosmosis in the morning i'm thinking, maybe we could add a test case in python for these new parameters? they can be difficult to grasp and this would ensure we don't regress without noticing.

There are tests in ImportLogsTest.php, do you mean add more tests to log-analytics/tests/*?

do you mean add more tests to log-analytics/tests/*?

yes