@diosmosis can you have a look at the tests eg https://travis-ci.org/matomo-org/matomo/jobs/611159119?utm_medium=notification&utm_source=github_status ?

@diosmosis I'm not sure this works as expected. I've been looking through the code for quite a while and been testing a few times and it seems like it's not reprocessing the data for today now.

Like here... What I did was tracking a request... then archive the data (no force-idsite param)... then tracked another request... expected it to archive again but because it wasn't invalidated and isOldReportInvalidatedForWebsite() returns false it was not archiving by the looks. Or maybe I'm understanding it wrong?

Like here... What I did was tracking a request... then archive the data (no force-idsite param)... then tracked another request... expected it to archive again but because it wasn't invalidated and isOldReportInvalidatedForWebsite() returns false it was not archiving by the looks. Or maybe I'm understanding it wrong?

It was working for me when I tested it last... I'll try and test it again.

@tsteur I was able to get it to work after a while after realizing visitorgenerator sets forced datetimes. It was setting the value to maybe an hour or so in the past, but the last successful archive run was from ~10s ago when I ran it. When I removed the setForceVisitDateTime() method call in VisitorGenerator it started working.

After that I noticed one other issue, if you run multiple core:archives too quickly, it will skip archiving because not enough time has passed:

Notice it says tracking data found but also Skipped website id 1, already done 42s ago. I think these are probably unrelated issues.

@diosmosis I had tested it with a simple tracking request. Archive. Send a basic tracking request through a website using JS tracking. Triggered archiving again. I had set the interval between archives from 900s to 1s so that shouldn't have been an issue. Could you maybe double check?

@tsteur tested again, it works for me:

initial archive:

initial visit count in log_visit:

visit count after tracking:

core archive after setting archive TTL to 1 (w/ a return 1 in Rules):

@tsteur if you could tell me what the reason for the site being skipped is, that might help.

@diosmosis sorry it works now indeed. Not sure why it didn't before. LGTM 👍

@tsteur I hope it was just a failure to track or something that caused it to fail for you, because if it was the code at fault then it would probably be a timezone issue or something...