Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cURL cron auto archive terminates without reason #19675

Closed
JustinFinkelsteinKainos opened this issue Aug 25, 2022 · 5 comments
Closed

cURL cron auto archive terminates without reason #19675

JustinFinkelsteinKainos opened this issue Aug 25, 2022 · 5 comments
Labels
answered For when a question was asked and we referred to forum or answered it.

Comments

@JustinFinkelsteinKainos

Expected Behavior

We have an automated cURL job which calls https://analytics/misc/cron/archive.php?token_auth=X at 5 mins past the hour, every hour to run the auto-archive process and we log the response from the request.

Our script which executes this cURL looks for the strings "Success:ok" or "Finished archiving" to identify that the script has executed correctly; if these are missing or the string "Fatal error" is found, it alerts us.

Yesterday, our script triggered an alert because the cURL completed successfully, but didn't contain the expected "Success" strings we're used to finding.

Current Behavior

Log data from the cURL is returned as follows:

INFO CoreAdminHome[2022-08-24 17:05:00 UTC] [49486] ---------------------------
INFO CoreAdminHome[2022-08-24 17:05:00 UTC] [49486] INIT
INFO CoreAdminHome[2022-08-24 17:05:00 UTC] [49486] Running Matomo 4.10.1 as Super User
INFO CoreAdminHome[2022-08-24 17:05:00 UTC] [49486] ---------------------------
INFO CoreAdminHome[2022-08-24 17:05:00 UTC] [49486] NOTES
INFO CoreAdminHome[2022-08-24 17:05:00 UTC] [49486] - Async process archiving supported, using CliMulti.
INFO CoreAdminHome[2022-08-24 17:05:00 UTC] [49486] - Reports for today will be processed at most every 3600 seconds. You can change this value in Matomo UI > Settings > General Settings.
INFO CoreAdminHome[2022-08-24 17:05:00 UTC] [49486] - Archiving was last executed without error 1 hours 0 min ago.
INFO CoreAdminHome[2022-08-24 17:05:00 UTC] [49486] ---------------------------
INFO CoreAdminHome[2022-08-24 17:05:00 UTC] [49486] START
INFO CoreAdminHome[2022-08-24 17:05:00 UTC] [49486] Starting Matomo reports archiving...
INFO CoreAdminHome[2022-08-24 17:05:00 UTC] [49486] Start processing archives for site 1.
INFO CoreAdminHome[2022-08-24 17:05:00 UTC] [49486] Will invalidate archived reports for today in site ID = 1's timezone (2022-08-24 00:00:00).
INFO CoreAdminHome[2022-08-24 17:05:00 UTC] [49486] Will invalidate archived reports for yesterday in site ID = 1's timezone (2022-08-23 00:00:00).
INFO CoreAdminHome[2022-08-24 17:05:03 UTC] [49486] Archived website id 1, period = day, date = 2022-08-24, segment = '', 272 visits found. Time elapsed: 3.054s
INFO CoreAdminHome[2022-08-24 17:05:06 UTC] [49486] Archived website id 1, period = day, date = 2022-08-24, segment = 'deviceType==smartphone,deviceType==tablet', 120 visits found. Time elapsed: 2.714s
INFO CoreAdminHome[2022-08-24 17:05:06 UTC] [49486] Archived website id 1, period = day, date = 2022-08-24, segment = 'dimension1==None', 0 visits found. Time elapsed: 2.714s
INFO CoreAdminHome[2022-08-24 17:05:06 UTC] [49486] Archived website id 1, period = day, date = 2022-08-24, segment = 'referrerUrl=@https%3A%2F%2Fwww.somesite.com%2F', 0 visits found. Time elapsed: 2.715s

Possible Solution

Can there be some clarification on when this script completes its execution successfully? This seems to have changed across versions without any clear documentation.

Steps to Reproduce (for Bugs)

This one's a bit tricky, but executing the same script should provide you some data

Context

See initial description. One other thing to note is that we've exceeded the licensing on some of our plugins and occasionally see the following show up in our logs:

[Wed Aug 24 17:05:04.736354 2022] [php:notice] [pid 69] [client 169.254.132.1:44239] [host] Error in Matomo (tracker): Request was meant for a plugin which is no longer activated. Request needs to be ignored., referer: https://oursite/

This looks like Matomo is throwing an (unrelated) error when it shouldn't be.

Your Environment

  • We're using Matomo Docker
  • Matomo Version: 4.10.1
  • PHP Version: 8.0.22
  • Server Operating System: docker standard
  • Additionally installed plugins:
  • Plugins Activated: API, Actions, Annotations, BulkTracking, Contents, CoreAdminHome, CoreConsole, CoreHome, CorePluginsAdmin, CoreUpdater, CoreVisualizations, CoreVue, CustomDimensions, CustomJsTracker, CustomReports 4.0.15, Dashboard, DevicePlugins, DevicesDetection, Diagnostics, Ecommerce, Events, Feedback, FormAnalytics 4.0.8, Funnels 4.0.11, GeoIp2, Goals, Heartbeat, ImageGraph, Insights, Installation, Intl, IntranetMeasurable, LanguagesManager, Live, Login, Marketplace, MobileMessaging, Monolog, Morpheus, MultiSites, Overlay, PagePerformance, PrivacyManager, ProfessionalServices, Proxy, Referrers, Resolution, RssWidget, SEO, ScheduledReports, SegmentEditor, SitesManager, Tour, Transitions, TwoFactorAuth, UserCountry, UserCountryMap, UserId, UserLanguage, UsersFlow 4.0.4, UsersManager, VisitFrequency, VisitTime, VisitorInterest, VisitsSummary, WebsiteMeasurable, Widgetize
  • Browser: Not relevant
  • Operating System: Not relevant
@JustinFinkelsteinKainos JustinFinkelsteinKainos added the Potential Bug Something that might be a bug, but needs validation and confirmation it can be reproduced. label Aug 25, 2022
@MatomoForumNotifications

This issue has been mentioned on Matomo forums. There might be relevant details there:

https://forum.matomo.org/t/curl-based-cron-auto-archive-success-notification/47214/4

@peterhashair
Copy link
Contributor

@JustinFinkelsteinKainos sorry for the late reply, I believe it should always print the keyword done at the end if the process success finished.

@peterhashair peterhashair added answered For when a question was asked and we referred to forum or answered it. and removed Potential Bug Something that might be a bug, but needs validation and confirmation it can be reproduced. labels Nov 9, 2022
@sgiehl
Copy link
Member

sgiehl commented Nov 21, 2022

As @peterhashair mentioned: The archiving should in the end print an info Done archiving!, followed by a summary.
Depending on the parameters afterwards the scheduled tasks might also be executed.
This behavior did not change in the last versions. Maybe the archiver was interrupted for some other reason, and thus did not complete in your case.

@sgiehl sgiehl closed this as completed Nov 21, 2022
@JustinFinkelsteinKainos
Copy link
Author

@sgiehl As you mentioned, the archiving should complete with 'done' as its last output, but when it's failed, I've not seen anything in our logs to indicate a failure. Is there a "debug" flag that I could set if this fails again?

@sgiehl
Copy link
Member

sgiehl commented Nov 22, 2022

The summary should contain either no error or x errors. If the archiving finishes without printing the summary, there might some fatal error or similar. You can enable debug output with -v, but that is quite noisy.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
answered For when a question was asked and we referred to forum or answered it.
Projects
None yet
Development

No branches or pull requests

4 participants