cURL cron auto archive terminates without reason #19675

Expected Behavior

We have an automated cURL job which calls https://analytics/misc/cron/archive.php?token_auth=X at 5 mins past the hour, every hour to run the auto-archive process and we log the response from the request.

Our script which executes this cURL looks for the strings "Success:ok" or "Finished archiving" to identify that the script has executed correctly; if these are missing or the string "Fatal error" is found, it alerts us.

Yesterday, our script triggered an alert because the cURL completed successfully, but didn't contain the expected "Success" strings we're used to finding.

Current Behavior

Log data from the cURL is returned as follows:

INFO CoreAdminHome[2022-08-24 17:05:00 UTC] [49486] ---------------------------
INFO CoreAdminHome[2022-08-24 17:05:00 UTC] [49486] INIT
INFO CoreAdminHome[2022-08-24 17:05:00 UTC] [49486] Running Matomo 4.10.1 as Super User
INFO CoreAdminHome[2022-08-24 17:05:00 UTC] [49486] ---------------------------
INFO CoreAdminHome[2022-08-24 17:05:00 UTC] [49486] NOTES
INFO CoreAdminHome[2022-08-24 17:05:00 UTC] [49486] - Async process archiving supported, using CliMulti.
INFO CoreAdminHome[2022-08-24 17:05:00 UTC] [49486] - Reports for today will be processed at most every 3600 seconds. You can change this value in Matomo UI > Settings > General Settings.
INFO CoreAdminHome[2022-08-24 17:05:00 UTC] [49486] - Archiving was last executed without error 1 hours 0 min ago.
INFO CoreAdminHome[2022-08-24 17:05:00 UTC] [49486] ---------------------------
INFO CoreAdminHome[2022-08-24 17:05:00 UTC] [49486] START
INFO CoreAdminHome[2022-08-24 17:05:00 UTC] [49486] Starting Matomo reports archiving...
INFO CoreAdminHome[2022-08-24 17:05:00 UTC] [49486] Start processing archives for site 1.
INFO CoreAdminHome[2022-08-24 17:05:00 UTC] [49486] Will invalidate archived reports for today in site ID = 1's timezone (2022-08-24 00:00:00).
INFO CoreAdminHome[2022-08-24 17:05:00 UTC] [49486] Will invalidate archived reports for yesterday in site ID = 1's timezone (2022-08-23 00:00:00).
INFO CoreAdminHome[2022-08-24 17:05:03 UTC] [49486] Archived website id 1, period = day, date = 2022-08-24, segment = '', 272 visits found. Time elapsed: 3.054s
INFO CoreAdminHome[2022-08-24 17:05:06 UTC] [49486] Archived website id 1, period = day, date = 2022-08-24, segment = 'deviceType==smartphone,deviceType==tablet', 120 visits found. Time elapsed: 2.714s
INFO CoreAdminHome[2022-08-24 17:05:06 UTC] [49486] Archived website id 1, period = day, date = 2022-08-24, segment = 'dimension1==None', 0 visits found. Time elapsed: 2.714s
INFO CoreAdminHome[2022-08-24 17:05:06 UTC] [49486] Archived website id 1, period = day, date = 2022-08-24, segment = 'referrerUrl=<a class='mention' href='https://github.com/https'>@https</a>%3A%2F%2Fwww.somesite.com%2F', 0 visits found. Time elapsed: 2.715s

Possible Solution

Can there be some clarification on when this script completes its execution successfully? This seems to have changed across versions without any clear documentation.

Steps to Reproduce (for Bugs)

This one's a bit tricky, but executing the same script should provide you some data

Context

See initial description. One other thing to note is that we've exceeded the licensing on some of our plugins and occasionally see the following show up in our logs:

[Wed Aug 24 17:05:04.736354 2022] [php:notice] [pid 69] [client 169.254.132.1:44239] [host] Error in Matomo (tracker): Request was meant for a plugin which is no longer activated. Request needs to be ignored., referer: https://oursite/

This looks like Matomo is throwing an (unrelated) error when it shouldn't be.

Your Environment

• We're using Matomo Docker
• Matomo Version: 4.10.1
• PHP Version: 8.0.22
• Server Operating System: docker standard
• Additionally installed plugins:
• Plugins Activated: API, Actions, Annotations, BulkTracking, Contents, CoreAdminHome, CoreConsole, CoreHome, CorePluginsAdmin, CoreUpdater, CoreVisualizations, CoreVue, CustomDimensions, CustomJsTracker, CustomReports 4.0.15, Dashboard, DevicePlugins, DevicesDetection, Diagnostics, Ecommerce, Events, Feedback, FormAnalytics 4.0.8, Funnels 4.0.11, GeoIp2, Goals, Heartbeat, ImageGraph, Insights, Installation, Intl, IntranetMeasurable, LanguagesManager, Live, Login, Marketplace, MobileMessaging, Monolog, Morpheus, MultiSites, Overlay, PagePerformance, PrivacyManager, ProfessionalServices, Proxy, Referrers, Resolution, RssWidget, SEO, ScheduledReports, SegmentEditor, SitesManager, Tour, Transitions, TwoFactorAuth, UserCountry, UserCountryMap, UserId, UserLanguage, UsersFlow 4.0.4, UsersManager, VisitFrequency, VisitTime, VisitorInterest, VisitsSummary, WebsiteMeasurable, Widgetize
• Browser: Not relevant
• Operating System: Not relevant

This issue has been mentioned on Matomo forums. There might be relevant details there:

https://forum.matomo.org/t/curl-based-cron-auto-archive-success-notification/47214/4

@JustinFinkelsteinKainos sorry for the late reply, I believe it should always print the keyword done at the end if the process success finished.

As @peterhashair mentioned: The archiving should in the end print an info Done archiving!, followed by a summary.
Depending on the parameters afterwards the scheduled tasks might also be executed.
This behavior did not change in the last versions. Maybe the archiver was interrupted for some other reason, and thus did not complete in your case.

@sgiehl As you mentioned, the archiving should complete with 'done' as its last output, but when it's failed, I've not seen anything in our logs to indicate a failure. Is there a "debug" flag that I could set if this fails again?

The summary should contain either no error or x errors. If the archiving finishes without printing the summary, there might some fatal error or similar. You can enable debug output with -v, but that is quite noisy.