Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

remove deletion of old archives in CronArchive since we do this in ArchiveWriter, and it is buggy here #17314

Merged
merged 8 commits into from Mar 15, 2021

Conversation

diosmosis
Copy link
Member

@diosmosis diosmosis commented Mar 8, 2021

Description:

Prevent edge case when partial archive archives and latest archive is invalidated (should generally not happen since in that case it would be processed before the partial archive, but just in case), by removing the method. The removal is done in ArchiveWriter as well anyway, so it shouldn't be needed here too.

Also another small potential edge case I saw in applyScheduledReArchiving() if for some reason it is ever called w/o superuser access.

Review

  • Functional review done
  • Potential edge cases thought about (behavior of the code with strange input, with strange internal state or possible interactions with other Matomo subsystems)
  • Usability review done (is anything maybe unclear or think about anything that would cause people to reach out to support)
  • Security review done see checklist
  • Code review done
  • Tests were added if useful/possible
  • Reviewed for breaking changes
  • Developer changelog updated if needed
  • Documentation added if needed
  • Existing documentation updated if needed

… invalidated (should generally not happen since in that case it would be processed before the partial archive, but just in case)
@diosmosis diosmosis added this to the 4.3.0 milestone Mar 8, 2021
@diosmosis diosmosis marked this pull request as draft March 8, 2021 05:55
@diosmosis diosmosis added the Needs Review PRs that need a code review label Mar 14, 2021
@diosmosis diosmosis marked this pull request as ready for review March 14, 2021 18:00
@diosmosis diosmosis changed the title handle partial archive edge case if invalidated archive exists but partial archive is being processed remove deletion of old archives in CronArchive since we do this in ArchiveWriter, and it is buggy here Mar 14, 2021
@@ -576,7 +577,10 @@ public function applyScheduledReArchiving()
continue;
}

$idSites = Site::getIdSitesFromIdSitesString($entry['idSites']);
$idSites = Access::doAsSuperUser(function () use ($entry) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

would this invalidate things maybe for too many sites even where we shouldn't? Like if a user doesn't have access to these sites then we wouldn't want to invalidate them?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

in the context of a segment where idSites = 'all', then no matter what it means every site, correct? these are also in applyScheduledReArchiving where we are applying the saved scheduleRearchiving() calls, which means those could be called in different contexts and access levels, but will still save 'all'. if it's possible that 'all' might not mean every site installed in a rearchiving context (rearchiving segment report, rearchiving plugin reports, rearchiving plugin entity reports), then maybe we should rethink this a bit more.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm thinking idsites=all be mostly triggered by super users (eg activating a plugin or making a segment available for all sites).

If someone who is not super user was to create something for multiple sites, then it should not appear in other sites.

Not sure if this helps?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll just remove this, if it becomes a problem it can be fixed later.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

@@ -931,6 +931,42 @@ private function getInvalidationsInTable($includeInvalidated = false)
$sql = "SELECT idarchive, idsite, period, date1, date2, name, report$suffix FROM `$table`";
return Db::fetchAll($sql);
}

private function insertArchiveData($archivesToInsert)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm just wondering where are we using these 3 methods?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

these were for some tests in a previous commit i removed, i'll remove these before it gets merged

@tsteur
Copy link
Member

tsteur commented Mar 15, 2021

fyi @diosmosis few tests are failing, not sure if due to this PR

…alidating partial archives, they should not change the done flag to DONE_INVALIDATED)
@diosmosis
Copy link
Member Author

@tsteur found another issue while fixing tests: when invalidating partial archives we don't want to change the done value for DONE_PARTIAL archives, since that will also change the semantics. Ie, DONE_INVALIDATED archives are in some cases treated like DONE_OK ones, but DONE_PARTIAL ones should never be.

@diosmosis diosmosis merged commit 34bf9d8 into 4.x-dev Mar 15, 2021
@diosmosis diosmosis deleted the partial-archive-regression branch March 15, 2021 23:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Needs Review PRs that need a code review
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants