Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Autodiscover tables for segments w/ complex segment expressions #13664

Merged
merged 6 commits into from Dec 10, 2018

Conversation

diosmosis
Copy link
Member

Currently if a segment has a custom $sqlSegment that includes functions/operations (like UNIX_TIMESTAMP(log_visit.visit_first_action_time) - log_visit.days_since_first_visit * 86400), the joins for segment expressions won't be discovered. So if we try to use this segment w/ log_conversion, it will skip the log_visit join and fail.

This PR fixes that by parsing out table/column names from the segment expression.

Also includes a new public method to return the ArchiveWriter instance in ArchiveProcessor so it's easier to create new ArchiveProcessor instances from within Archivers.

@diosmosis diosmosis added the Needs Review PRs that need a code review label Nov 2, 2018
@diosmosis diosmosis added this to the 3.8.0 milestone Nov 2, 2018
@diosmosis diosmosis changed the title Autodiscover segment expression for segments w/ complex segment expressions Autodiscover tables for segments w/ complex segment expressions Nov 2, 2018
@tsteur
Copy link
Member

tsteur commented Nov 4, 2018

Can you add some tests for this @diosmosis ?

@diosmosis
Copy link
Member Author

👍 will do

@diosmosis
Copy link
Member Author

Added tests.

*/
private function parseColumnsFromSqlExpr($field)
{
preg_match_all('/\b([a-zA-Z0-9_`]+\.[a-zA-Z0-9_`]+)\b/', $field, $matches);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@diosmosis not sure if on purpose... when field is eg "log_action.type" then

preg_match_all('/\b([a-zA-Z0-9_]+.[a-zA-Z0-9_]+)\b/', 'log_action.type', $matches);

returns this:
image
also see https://3v4l.org/0v32G
not sure if the field can be like this.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated to better handle backticks (it's not super intelligent, but should work given this is for plugin devs).

@tsteur
Copy link
Member

tsteur commented Dec 9, 2018

Tested and worked. Only thing I would have done be to maybe write some tests for parseColumnsFromSqlExpr($field) to make sure it works in various cases. I think the other tests don't quite cover this as extensively? not sure... maybe they do indirectly (I think so).

I used xdebug to see what the inputs are, how they are generated etc. to see about possible inputs etc... seeing maybe just a few examples in tests might help understand what is being down there but all good if not adding more tests and if it is indirectly covered by other tests.

@diosmosis
Copy link
Member Author

Only thing I would have done be to maybe write some tests for parseColumnsFromSqlExpr($field) to make sure it works in various cases.

All it should do is look for table.column expressions in strings. Ie, it doesn't parse the SQL expression into pieces at all, so the only real variants are having one vs. many & backticks vs. no backticks. Ie, the presence of operators, function calls shouldn't have an effect.

@tsteur
Copy link
Member

tsteur commented Dec 9, 2018

oki 👍

@diosmosis
Copy link
Member Author

Added a test as you suggested. Will merge if the build passes.

@diosmosis diosmosis merged commit 4d61d27 into 3.x-dev Dec 10, 2018
@diosmosis diosmosis deleted the autodiscover-segment-expr branch December 10, 2018 04:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Needs Review PRs that need a code review
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants