Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Always purge today & yesterday and add update for 3.13.5 #15832

Merged
merged 4 commits into from Apr 21, 2020

Conversation

diosmosis
Copy link
Member

Fix #15822

@diosmosis diosmosis added not-in-changelog For issues or pull requests that should not be included in our release changelog on matomo.org. Needs Review PRs that need a code review labels Apr 20, 2020
@diosmosis diosmosis added this to the 3.13.5 milestone Apr 20, 2020
@Findus23
Copy link
Member

Do I understand it correctly that the update script does something similar to this?

sudo -u www-data php7.3 console core:purge-old-archive-data all

Because that took over 10 minutes with my small DB and fast SSD, which means the update will time out for nearly all Matomo users (unless they do it from the cli, which nearly no one does).

@diosmosis
Copy link
Member Author

@Findus23 The update adds the dates to the list that is used during scheduled task execution. So the next scheduled task run will run the equivalent of that command (though I think the table optimization is in it's own task).

@tsteur
Copy link
Member

tsteur commented Apr 20, 2020

@diosmosis I reckon it be enough to add this in the update script only for the months since march 2020 since that's when it was an issue?

@diosmosis
Copy link
Member Author

@tsteur 👍 will make that change

@diosmosis
Copy link
Member Author

@tsteur updated

list($year, $month) = explode('_', $date);

$dateObj = Date::factory("$year-$month-01 00:00:00");
if ($dateObj->isEarlier($startOfProblem)) { // only add if the table is for march 2020 or above since that is when the problem appeared
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I haven't checked it yet... but this would only run it for march if it was executed eg on April 23rd 2020 right? Although I suppose that's fine since we'd add today anyway on every purge and April would be deleted that way 👍

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure what you mean, but it will run it if it finds an archive table for march or later. Though actually, this won't work because we also have to add the year table. I'll just add it if the year is 2020 or greater.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I meant "2020-04-01 00:00:00" would be earlier than 2020-04-23 and therefore it wouldn't do it for April but it's not important since it would do it automatically anyway when the task is executed next

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The dates used should always be -01 00:00:00 in this code, i'm not sure why the execution date of the update matters.

@tsteur tsteur merged commit 9aee02d into 3.x-dev Apr 21, 2020
@tsteur tsteur deleted the 15822-always-purge-today-yest branch April 21, 2020 02:33
jonasgrilleres pushed a commit to 1024pix/pix-analytics that referenced this pull request Sep 22, 2020
…15832)

* Always purge today/yesterday when running invalidated archive purge task and add update to add all tables to purge for next version.

* Add some comments.

* Only add dates if they are past 2020-03.

* Only add tables above 2020-01.
jbuget pushed a commit to 1024pix/pix-analytics that referenced this pull request Sep 26, 2020
…15832)

* Always purge today/yesterday when running invalidated archive purge task and add update to add all tables to purge for next version.

* Add some comments.

* Only add dates if they are past 2020-03.

* Only add tables above 2020-01.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Needs Review PRs that need a code review not-in-changelog For issues or pull requests that should not be included in our release changelog on matomo.org.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants