This fixes issue #13047
The check does not take into consideration
range period, where if you have two adjacent days, the difference is
47h59m59s and not a day.
Example: If I want to get archived data from
2018-06-11, I need to check everything between
2018-06-10 00:00:00 and
2018-06-11 23:59:59. The check disregards the time part of the date.
I think this bug was introduced when functions
Date::getDateEndUTC were deprecated. In the code they are replaced with
getTimestampUTC which doesn't return the same format as the other two. The tests in
ArchiveProcessorTest still use the deprecated functions though.
p.s. Apologies for not providing unit tests for this but I couldn't set up phpunit for this project. I also realize this is not the most elegant solution but I didn't want to change too many parts of the code.
The code here is part of the code that aggregates log data: it checks if the period is for a single day & site, and if so aggregates the log data. If it isn't, it means the period spans multiple days and the archiver will initiate archiving for each individual day and then aggregate the results for the whole period.
@KaiNissen's issue sounds like pre-archived data from pre-3.0 versions aren't being found for ranges that consist of two adjacent days, so I'm not sure what effect this will have. @KaiNissen are you able to confirm if this fixes the issue for you?
As for the change, I think it could be fixed a bit more generally by changing the
< (so if one day is
2018-06-10 and the end is
2018-06-11, it will know it's not a single day). That way it will also be fixed for non-range periods (plugins are able to add custom periods).
@diosmosis We didn't roll it out on our production system, but after applying this to a test system, I wasn't able to reproduce the issue anymore. So yes, this change seems to fix the problem.
By the way, it affects all data that is older than two weeks, not only what has been archived prior to the upgrade.
That's interesting, @KaiNissen, I'm not able to reproduce this on my own. Would you be able to check in your test system if changing the
< in https://github.com/matomo-org/matomo/blob/3.x-dev/core/ArchiveProcessor/Parameters.php#L214 also fixes the issue for you?
Hi @KaiNissen @tzhelyazkova could you please check @diosmosis answer in https://github.com/matomo-org/matomo/pull/13315#issuecomment-417787446
or @diosmosis maybe you have some ideas to could get to the bottom of this issue?
@mattab I'll create a PR w/ a test for the operator bug which we can merge in 3.6.1. If someone else reports the issue then we can debug again?
sounds good :+1:
@tzhelyazkova could you see what
@tzhelyazkova apologies for the late reply, just getting back to this. I'm not sure why this is happening, but I have a guess that it might be timezone related. Does your site in Matomo have a specific timezone?
@diosmosis do you maybe have a chance to look into this again?
Yes, I'll take a look today.
@tzhelyazkova Well, I'm still unable to reproduce this, but if
getRangeString() works while the actual code doesn't, then changing
getTimestamp() might fix the issue for you. Can you try this change: https://github.com/matomo-org/matomo/compare/parameters-date-timezone-check?expand=1?