Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix check if the difference between start and end date is a day #13315

Closed

Conversation

tzhelyazkova
Copy link

This fixes issue #13047

The check does not take into consideration range period, where if you have two adjacent days, the difference is 47h59m59s and not a day.
Example: If I want to get archived data from 2018-06-10 to 2018-06-11, I need to check everything between 2018-06-10 00:00:00 and 2018-06-11 23:59:59. The check disregards the time part of the date.
I think this bug was introduced when functions Date::getDateStartUTC and Date::getDateEndUTC were deprecated. In the code they are replaced with getTimestampUTC which doesn't return the same format as the other two. The tests in ArchiveProcessorTest still use the deprecated functions though.

p.s. Apologies for not providing unit tests for this but I couldn't set up phpunit for this project. I also realize this is not the most elegant solution but I didn't want to change too many parts of the code.

Issue: matomo-org#13047

The  check does not take into consideration range period, where if you have two adjusent days, it's 47h59m59s and not a day.
@KaiNissen
Copy link

Many thanks for digging into this, @tzhelyazkova!

@diosmosis
Copy link
Member

Thanks for the PR and research @tzhelyazkova, it's very much appreciated! I think you definitely found a bug here, but I'm not sure it's the cause of @KaiNissen's issue.

The code here is part of the code that aggregates log data: it checks if the period is for a single day & site, and if so aggregates the log data. If it isn't, it means the period spans multiple days and the archiver will initiate archiving for each individual day and then aggregate the results for the whole period.

@KaiNissen's issue sounds like pre-archived data from pre-3.0 versions aren't being found for ranges that consist of two adjacent days, so I'm not sure what effect this will have. @KaiNissen are you able to confirm if this fixes the issue for you?

As for the change, I think it could be fixed a bit more generally by changing the <= to < (so if one day is 2018-06-10 and the end is 2018-06-11, it will know it's not a single day). That way it will also be fixed for non-range periods (plugins are able to add custom periods).

@KaiNissen
Copy link

@diosmosis We didn't roll it out on our production system, but after applying this to a test system, I wasn't able to reproduce the issue anymore. So yes, this change seems to fix the problem.

By the way, it affects all data that is older than two weeks, not only what has been archived prior to the upgrade.

@diosmosis
Copy link
Member

That's interesting, @KaiNissen, I'm not able to reproduce this on my own. Would you be able to check in your test system if changing the <= to < in https://github.com/matomo-org/matomo/blob/3.x-dev/core/ArchiveProcessor/Parameters.php#L214 also fixes the issue for you?

@mattab mattab added the Needs Review PRs that need a code review label Sep 1, 2018
@mattab
Copy link
Member

mattab commented Oct 8, 2018

Hi @KaiNissen @tzhelyazkova could you please check @diosmosis answer in #13315 (comment)

or @diosmosis maybe you have some ideas to could get to the bottom of this issue?

@mattab mattab added this to the 3.7.0 milestone Oct 9, 2018
@diosmosis
Copy link
Member

@mattab I'll create a PR w/ a test for the operator bug which we can merge in 3.6.1. If someone else reports the issue then we can debug again?

@mattab
Copy link
Member

mattab commented Oct 10, 2018

sounds good 👍

@tzhelyazkova
Copy link
Author

Hey @mattab @diosmosis
Sorry for the late response. I tried replacing <= with < on my test environment, but it doesn't fix the issue. The matomo version is still 3.5.1.
Also when I print $secondsInPeriod it always seems to be int 0. 😕
Thanks for looking into the issue!

@diosmosis
Copy link
Member

@tzhelyazkova could you see what $period->getRangeString() is?

@tzhelyazkova
Copy link
Author

@diosmosis

@tzhelyazkova could you see what $period->getRangeString() is?

It prints a string with the right start and end date I choose as a range period,
e.g. string '2017-02-23,2017-02-24'

@diosmosis
Copy link
Member

@tzhelyazkova apologies for the late reply, just getting back to this. I'm not sure why this is happening, but I have a guess that it might be timezone related. Does your site in Matomo have a specific timezone?

@tsteur
Copy link
Member

tsteur commented Dec 19, 2018

@tzhelyazkova any chance you could maybe reply to @diosmosis question in the last comment?

@tzhelyazkova
Copy link
Author

@diosmosis @tsteur yes, our matomo is in the Europe/Berlin timezone.

@tsteur
Copy link
Member

tsteur commented Dec 20, 2018

@diosmosis do you maybe have a chance to look into this again?

@diosmosis
Copy link
Member

Yes, I'll take a look today.

@diosmosis
Copy link
Member

@tzhelyazkova Well, I'm still unable to reproduce this, but if getRangeString() works while the actual code doesn't, then changing getTimestampUTC() to getTimestamp() might fix the issue for you. Can you try this change: https://github.com/matomo-org/matomo/compare/parameters-date-timezone-check?expand=1?

@mattab
Copy link
Member

mattab commented Dec 24, 2018

Hi @tzhelyazkova
Are you still experiencing this issue with the latest version of Matomo?
If so please let us know the steps to reproduce so we have a chance to investigate and fix it.
Thanks,

@mattab mattab modified the milestones: 3.8.0, 3.9.0 Dec 24, 2018
@diosmosis
Copy link
Member

Closing since original issue is closed.

@diosmosis diosmosis closed this Mar 4, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Needs Review PRs that need a code review
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

5 participants