Matomo Issue #19675 - cURL cron auto archive terminates without reason

@JustinFinkelsteinKainos opened this Issue on August 25th 2022

Expected Behavior

We have an automated cURL job which calls https://analytics/misc/cron/archive.php?token_auth=X at 5 mins past the hour, every hour to run the auto-archive process and we log the response from the request.

Our script which executes this cURL looks for the strings "Success:ok" or "Finished archiving" to identify that the script has executed correctly; if these are missing or the string "Fatal error" is found, it alerts us.

Yesterday, our script triggered an alert because the cURL completed successfully, but didn't contain the expected "Success" strings we're used to finding.

Current Behavior

Log data from the cURL is returned as follows:

INFO CoreAdminHome[2022-08-24 17:05:00 UTC] [49486] ---------------------------
INFO CoreAdminHome[2022-08-24 17:05:00 UTC] [49486] INIT
INFO CoreAdminHome[2022-08-24 17:05:00 UTC] [49486] Running Matomo 4.10.1 as Super User
INFO CoreAdminHome[2022-08-24 17:05:00 UTC] [49486] ---------------------------
INFO CoreAdminHome[2022-08-24 17:05:00 UTC] [49486] NOTES
INFO CoreAdminHome[2022-08-24 17:05:00 UTC] [49486] - Async process archiving supported, using CliMulti.
INFO CoreAdminHome[2022-08-24 17:05:00 UTC] [49486] - Reports for today will be processed at most every 3600 seconds. You can change this value in Matomo UI > Settings > General Settings.
INFO CoreAdminHome[2022-08-24 17:05:00 UTC] [49486] - Archiving was last executed without error 1 hours 0 min ago.
INFO CoreAdminHome[2022-08-24 17:05:00 UTC] [49486] ---------------------------
INFO CoreAdminHome[2022-08-24 17:05:00 UTC] [49486] START
INFO CoreAdminHome[2022-08-24 17:05:00 UTC] [49486] Starting Matomo reports archiving...
INFO CoreAdminHome[2022-08-24 17:05:00 UTC] [49486] Start processing archives for site 1.
INFO CoreAdminHome[2022-08-24 17:05:00 UTC] [49486] Will invalidate archived reports for today in site ID = 1's timezone (2022-08-24 00:00:00).
INFO CoreAdminHome[2022-08-24 17:05:00 UTC] [49486] Will invalidate archived reports for yesterday in site ID = 1's timezone (2022-08-23 00:00:00).
INFO CoreAdminHome[2022-08-24 17:05:03 UTC] [49486] Archived website id 1, period = day, date = 2022-08-24, segment = '', 272 visits found. Time elapsed: 3.054s
INFO CoreAdminHome[2022-08-24 17:05:06 UTC] [49486] Archived website id 1, period = day, date = 2022-08-24, segment = 'deviceType==smartphone,deviceType==tablet', 120 visits found. Time elapsed: 2.714s
INFO CoreAdminHome[2022-08-24 17:05:06 UTC] [49486] Archived website id 1, period = day, date = 2022-08-24, segment = 'dimension1==None', 0 visits found. Time elapsed: 2.714s
INFO CoreAdminHome[2022-08-24 17:05:06 UTC] [49486] Archived website id 1, period = day, date = 2022-08-24, segment = 'referrerUrl=<a class='mention' href='https://github.com/https'>@https</a>%3A%2F%2Fwww.somesite.com%2F', 0 visits found. Time elapsed: 2.715s

Possible Solution

Can there be some clarification on when this script completes its execution successfully? This seems to have changed across versions without any clear documentation.

Steps to Reproduce (for Bugs)

This one's a bit tricky, but executing the same script should provide you some data

Context

See initial description. One other thing to note is that we've exceeded the licensing on some of our plugins and occasionally see the following show up in our logs:

[Wed Aug 24 17:05:04.736354 2022] [php:notice] [pid 69] [client 169.254.132.1:44239] [host] Error in Matomo (tracker): Request was meant for a plugin which is no longer activated. Request needs to be ignored., referer: https://oursite/

This looks like Matomo is throwing an (unrelated) error when it shouldn't be.

Your Environment