Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Ensure that the archive algorithm cannot be triggered multiple times for a same site/period/segment #8444

Closed
tsteur opened this issue Jul 28, 2015 · 10 comments
Labels
Bug For errors / faults / flaws / inconsistencies etc. Major Indicates the severity or impact or benefit of an issue is much higher than normal but not critical. Regression Indicates a feature used to work in a certain way but it no longer does even though it should.

Comments

@tsteur
Copy link
Member

tsteur commented Jul 28, 2015

refs #1938

I've seen this here in UI tests:

33131fd2-24a8-11e5-9644-e01aa4b831c3

As one can see the archive was created multiple times

@quba
Copy link
Contributor

quba commented Jul 28, 2015

But actually it should work so - e.g. for current day/week/month/year. Then temporary archives are purged once a day. Could you check the done flag?

Additionally sometimes it makes sense if there's some missing data. E.g. if one updates from old Piwik version (e.g. 1.6 -> latest) then it makes sense to rearchive some data (e.g. returning visitors because latest Piwik is using a segment to calculate it). But maybe it's a different case (new segment).

@tsteur
Copy link
Member Author

tsteur commented Jul 28, 2015

I do not have the database anymore. Should have made a dump :(

It happened when starting a UI test when inserting the fixture so there should be no reason for duplicated archives

@mattab
Copy link
Member

mattab commented Sep 18, 2015

@quba on the screenshots there are two done flags with different idarchive for the same idsite, same dates and period. This is a bug as Piwik should not archive twice the exact same archive.

(this probably regressed a long time ago but still a regression as it used to work eg. in 2.0 or earlier)

@mattab mattab added Bug For errors / faults / flaws / inconsistencies etc. Regression Indicates a feature used to work in a certain way but it no longer does even though it should. labels Sep 18, 2015
@quba
Copy link
Contributor

quba commented Sep 18, 2015

So it's not possible when e.g. you archive two times current day (so those will be only temporary archives)? Same works for current periods which are always temporary.

@mattab mattab added this to the Short term milestone Sep 20, 2015
@mattab
Copy link
Member

mattab commented Sep 20, 2015

So it's not possible when e.g. you archive two times current day (so those will be only temporary archives)?

Not sure what you meant, but what I mean is:

  • if one loads dashboard, and 10 widgets call the API at the same time (same second), we expect that Piwik will trigger archiving only once (in one thread) and other threads will wait for this process, then when archiving is finished all widgets loading will be unblocked and will read this same archive

@quba
Copy link
Contributor

quba commented Sep 21, 2015

Yes, that's one case. What I meant was the fact, that multiple archives for same date and period with different idarchive can be created at the same day (e.g. temporary archives created today for period=day,week,month,year). Is that correct? If yes, then the provided screenshot doesn't have to mean that there's a bug. On the other hand, those archives are for the year 2012 so it may be also connected with concurrency.

@mattab
Copy link
Member

mattab commented Sep 22, 2015

What I meant was the fact, that multiple archives for same date and period with different idarchive can be created at the same day

this works well as long as you don't make concurrent requests. Piwik will only create one archive when necessary, eg. temporary archives will be re-created if the last temporary is older than N seconds (as per general settings).

If yes, then the provided screenshot doesn't have to mean that there's a bug.

the screenshot is clearly a bug because:

  • the reports are for 2012 (not "today")
  • two archives for exactly same date/period/site have been created on the same second (<- should not happen if the concurrency lock didn't regress)

@mattab
Copy link
Member

mattab commented Mar 10, 2016

(not 100% sure but I suppose this bug report in forum: https://forum.piwik.org/t/sql-error-sqlstate-23000-integrity-constraint-violation-1062-duplicate-entry-archive-numeric-2014-09-for-key-primary/17863 is caused by this bug here. The error is visible in the dashboard SQLSTATE[23000]: Integrity constraint violation: 1062 Duplicate entry 'archive_numeric_2014_09' for key 'PRIMARY')

@mattab mattab modified the milestones: 3.0.0, Mid term Nov 30, 2016
@mattab
Copy link
Member

mattab commented Oct 13, 2017

we're again seen this error "Integrity constraint violation 1062 duplicate entry archive_numeric_*"

pasted_image_at_2017_10_13_12_34_pm

@tsteur
Copy link
Member Author

tsteur commented Nov 15, 2021

This should be solved by #18243

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Bug For errors / faults / flaws / inconsistencies etc. Major Indicates the severity or impact or benefit of an issue is much higher than normal but not critical. Regression Indicates a feature used to work in a certain way but it no longer does even though it should.
Projects
None yet
Development

No branches or pull requests

3 participants