@diosmosis opened this Pull Request on November 13th 2019 Member

Quick fix for #15086 ... and fix for archive purging logic now that today dates are invalidated and temporary archives do not exist.

Fixes #15086
Fixes #10439

@tsteur commented on November 13th 2019 Member
@tsteur commented on November 15th 2019 Member

@diosmosis I'm not sure this works as expected. I've been looking through the code for quite a while and been testing a few times and it seems like it's not reprocessing the data for today now.

image

Like here... What I did was tracking a request... then archive the data (no force-idsite param)... then tracked another request... expected it to archive again but because it wasn't invalidated and isOldReportInvalidatedForWebsite() returns false it was not archiving by the looks. Or maybe I'm understanding it wrong?

@diosmosis commented on November 15th 2019 Member

Like here... What I did was tracking a request... then archive the data (no force-idsite param)... then tracked another request... expected it to archive again but because it wasn't invalidated and isOldReportInvalidatedForWebsite() returns false it was not archiving by the looks. Or maybe I'm understanding it wrong?

It was working for me when I tested it last... I'll try and test it again.

@diosmosis commented on November 15th 2019 Member

@tsteur I was able to get it to work after a while after realizing visitorgenerator sets forced datetimes. It was setting the value to maybe an hour or so in the past, but the last successful archive run was from ~10s ago when I ran it. When I removed the setForceVisitDateTime() method call in VisitorGenerator it started working.

After that I noticed one other issue, if you run multiple core:archives too quickly, it will skip archiving because not enough time has passed:

image

Notice it says tracking data found but also Skipped website id 1, already done 42s ago. I think these are probably unrelated issues.

@tsteur commented on November 17th 2019 Member

@diosmosis I had tested it with a simple tracking request. Archive. Send a basic tracking request through a website using JS tracking. Triggered archiving again. I had set the interval between archives from 900s to 1s so that shouldn't have been an issue. Could you maybe double check?

@diosmosis commented on November 18th 2019 Member

@tsteur tested again, it works for me:

initial archive:

Screen Shot 2019-11-17 at 9 06 53 PM

initial visit count in log_visit:

Screen Shot 2019-11-17 at 9 09 51 PM

visit count after tracking:

Screen Shot 2019-11-17 at 9 10 21 PM

core archive after setting archive TTL to 1 (w/ a return 1 in Rules):

Screen Shot 2019-11-17 at 9 12 04 PM
@diosmosis commented on November 18th 2019 Member

@tsteur if you could tell me what the reason for the site being skipped is, that might help.

@tsteur commented on November 18th 2019 Member

@diosmosis sorry it works now indeed. Not sure why it didn't before. LGTM 👍

@diosmosis commented on November 19th 2019 Member

@tsteur I hope it was just a failure to track or something that caused it to fail for you, because if it was the code at fault then it would probably be a timezone issue or something...

This Pull Request was closed on November 19th 2019
Powered by GitHub Issue Mirror