@tsteur opened this Issue on March 10th 2021 Member

When there is an archiving query for a segment, then we don't know which time period this query is selecting because the time period was already included in the query when the temporary table was created (https://github.com/matomo-org/matomo/blob/4.2.1/core/DataAccess/LogAggregator.php#L273)

image

When a segment is used, then we'd basically need to add another comment here including the date: https://github.com/matomo-org/matomo/blob/4.2.1/core/DataAccess/LogAggregator.php#L400

Eg if it queries the day for 2020-02-02 then there be a comment /* 2020-02-02 */. We wouldn't need to know the exact time range used but could do that if it's easier. We could also generally add this comment and adjust the query hint accordingly around https://github.com/matomo-org/matomo/blob/4.2.1/core/ArchiveProcessor/PluginsArchiver.php#L153

If it's easily possible, then we'd also want to add the segmentId (might be already the case)

While working on this could then maybe also quickly look at https://github.com/matomo-org/matomo/issues/17327

Technically this should be already the case but maybe this is not working? https://github.com/matomo-org/matomo/blob/4.2.1/core/DataAccess/LogAggregator.php#L406-L408

We would expect to see something like SELECT /* 2020-02-02,2020-02-03 sites 1,2,3 */ /* plugin name /* ...

This should only apply to archiver queries when you for example execute ./console core:archive and it is executing a query of some plugin archiver.

@flamisz commented on March 11th 2021 Contributor

I'm investigation this issue and I see something that can be the root of the issue (or at least that's why a couple of things should work but not working).

In this part of the code:
https://github.com/matomo-org/matomo/blob/3688fff9947853789ef748ec3e74d814a8214517/core/DataAccess/LogAggregator.php#L400-L411

we basically check if the the query starts with SELECT, but when we use the SegmentQueryDecorator here:
https://github.com/matomo-org/matomo/blob/3688fff9947853789ef748ec3e74d814a8214517/plugins/SegmentEditor/SegmentQueryDecorator.php#L44-L46

we prefix the SELECT query with /* trigger = CronArchive */ string (and we add new line as well), so we don't add the queryOriginHint and dates and other useful information for the query as comment in the LogAggregator class.

The first time I use this feature, so maybe I'm wrong, but this is the result of my first investigation of this issue.
Do you think it is the problem? Ping @tsteur @sgiehl

This Issue was closed on March 15th 2021
Powered by GitHub Issue Mirror