@diosmosis opened this Pull Request on November 11th 2020 Member

Description:

Refs #16689

Better detection of in progress invalidations: if an invalidation's status is set to in progress, but the invalidation was created over a day ago. (We also update the ts_invalidated time to now() when starting an archive).

CC @tsteur

Review

  • [ ] Functional review done
  • [ ] Usability review done (is anything maybe unclear or think about anything that would cause people to reach out to support)
  • [ ] Security review done see checklist
  • [ ] Code review done
  • [ ] Tests were added if useful/possible
  • [ ] Reviewed for breaking changes
  • [ ] Developer changelog updated if needed
  • [ ] Documentation added if needed
  • [ ] Existing documentation updated if needed
@diosmosis commented on November 11th 2020 Member

Note: working on a test

@diosmosis commented on November 11th 2020 Member

@tsteur updated

@diosmosis commented on November 11th 2020 Member

looks good to me. The only thing I'm scared of is that an archive run that takes longer than a day would be basically always reset to 0 and always run again but suppose one day shouldn't happen. I wouldn't be surprised if it happens though eg when a monthly archive is triggered, and then it notices some days are outdated, and it also rearchives several days and then weeks etc. Even though we triggered month, I think it might still archive individual days when needed not sure...

this shouldn't technically happen since we order the periods by day => month and don't allow simultaneously archiving intersecting periods. there might be some edge cases where it does however, like someone deletes an archive right as a month archive starts, but it would be rare I think. best way to avoid this would be to check the Lock before resetting an archive, not sure if that would be worth doing.

@tsteur commented on November 11th 2020 Member

this shouldn't technically happen since we order the periods by day => month and don't allow simultaneously archiving intersecting periods

I'm thinking this might still happen. When requesting a month period then it has to archive within the same request a missing day archive etc.

@diosmosis commented on November 12th 2020 Member

It might happen but not to the extent that it would end up taking so much longer, I think? Maybe something happens to cause some days to be deleted while archiving a month, but not the entire month? Maybe if there's a purge but then we shouldn't be archiving that data...

@diosmosis commented on November 12th 2020 Member

@tsteur updated

This Pull Request was closed on November 12th 2020
Powered by GitHub Issue Mirror