@mattkolb opened this Issue on August 10th 2021

We don't use yearly archives because they take too long to process. Because of this, we have the following settings in our config:

[General]
enabled_periods_UI = "day,week,month,range"
enabled_periods_API = "day,week,month,range"

After upgrading from 3.14.0 to 4.4.1, I'm unable to run the cron archive. The output is below.

console core:archive --url=... --force-periods=day --disable-scheduled-tasks

INFO [2021-08-10 18:58:48] 85257 ---------------------------
INFO [2021-08-10 18:58:48] 85257 INIT
INFO [2021-08-10 18:58:48] 85257 Running Matomo 4.4.1 as Super User
INFO [2021-08-10 18:58:48] 85257 ---------------------------
INFO [2021-08-10 18:58:48] 85257 NOTES
INFO [2021-08-10 18:58:48] 85257 - Async process archiving supported, using CliMulti.
INFO [2021-08-10 18:58:48] 85257 - Reports for today will be processed at most every 10 seconds. You can change this value in Matomo UI > Settings > General Settings.
INFO [2021-08-10 18:58:48] 85257 - Archiving was last executed without error 7s ago
INFO [2021-08-10 18:58:48] 85257 - Will only process the following periods: day (--force-periods)
INFO [2021-08-10 18:58:48] 85257 ---------------------------
INFO [2021-08-10 18:58:48] 85257 START
INFO [2021-08-10 18:58:48] 85257 Starting Matomo reports archiving...
INFO [2021-08-10 18:58:48] 85257 Start processing archives for site 1.
INFO [2021-08-10 18:58:48] 85257 Will invalidate archived reports for today in site ID = 1's timezone (2021-08-10 00:00:00).
ERROR [2021-08-10 18:58:48] 85257 Uncaught exception: /home/aatrkco/public_html/stats/core/Period/Factory.php(131): The period 'year' is not supported. Try any of the following instead: day, week, month, range [Query: , CLI mode: 1]

[Exception]
The period 'year' is not supported. Try any of the following instead: day, week, month, range

@mattkolb commented on August 10th 2021

Also, I believe I found the log of the very first time the archiving was attempted after the upgrade. The log is listed below.

INFO [2021-08-10 18:45:01] 111145 ---------------------------
INFO [2021-08-10 18:45:01] 111145 INIT
INFO [2021-08-10 18:45:01] 111145 Running Matomo 4.4.1 as Super User
INFO [2021-08-10 18:45:01] 111145 ---------------------------
INFO [2021-08-10 18:45:01] 111145 NOTES
INFO [2021-08-10 18:45:01] 111145 - Async process archiving supported, using CliMulti.
INFO [2021-08-10 18:45:01] 111145 - Reports for today will be processed at most every 10 seconds. You can change this value in Matomo UI > Settings > General Settings.
INFO [2021-08-10 18:45:01] 111145 - Archiving was last executed without error 4 hours 57 min ago
INFO [2021-08-10 18:45:01] 111145 - Will only process the following periods: day (--force-periods)
INFO [2021-08-10 18:45:01] 111145 ---------------------------
INFO [2021-08-10 18:45:01] 111145 START
INFO [2021-08-10 18:45:01] 111145 Starting Matomo reports archiving...
INFO [2021-08-10 18:45:01] 111145 Failed to create invalidations for report re-archiving (idSites = "all", pluginName = VisitFrequency, report = , startDate = 1606089600): Exception: The period 'year' is not supported. Try any of the following instead: day, week, month, range in /public_html/stats/core/Period/Factory.php:131
Stack trace:
#0 /public_html/stats/core/Period/Factory.php(118): Piwik\Period\Factory::throwExceptionInvalidPeriod('year')
#1 /public_html/stats/core/Period/Factory.php(74): Piwik\Period\Factory::checkPeriodIsEnabled('year')
#2 /public_html/stats/core/Archive/ArchiveInvalidator.php(401): Piwik\Period\Factory::build('year', Object(Piwik\Date))
#3 /public_html/stats/core/Archive/ArchiveInvalidator.php(403): Piwik\Archive\ArchiveInvalidator->addParentPeriodsByYearMonth(Array, Object(Piwik\Period\Month), Object(Piwik\Date))
#4 /public_html/stats/core/Archive/ArchiveInvalidator.php(403): Piwik\Archive\ArchiveInvalidator->addParentPeriodsByYearMonth(Array, Object(Piwik\Period\Week), Object(Piwik\Date))
#5 /public_html/stats/core/Archive/ArchiveInvalidator.php(360): Piwik\Archive\ArchiveInvalidator->addParentPeriodsByYearMonth(Array, Object(Piwik\Period\Day))
#6 /public_html/stats/core/Archive/ArchiveInvalidator.php(314): Piwik\Archive\ArchiveInvalidator->getAllPeriodsByYearMonth('day', Array, false)
#7 /public_html/stats/core/Archive/ArchiveInvalidator.php(494): Piwik\Archive\ArchiveInvalidator->markArchivesAsInvalidated(Array, Array, 'day', NULL, false, false, 'VisitFrequency')
#8 /public_html/stats/core/Archive/ArchiveInvalidator.php(576): Piwik\Archive\ArchiveInvalidator->reArchiveReport(Array, 'VisitFrequency', NULL, Object(Piwik\Date), NULL)
#9 /public_html/stats/core/CronArchive.php(348): Piwik\Archive\ArchiveInvalidator->applyScheduledReArchiving()
#10 /public_html/stats/core/Tracker/Cache.php(297): Piwik\CronArchive->Piwik{closure}()
#11 /public_html/stats/core/CronArchive.php(349): Piwik\Tracker\Cache::withDelegatedCacheClears(Object(Closure))
#12 /public_html/stats/core/CronArchive.php(263): Piwik\CronArchive->run()
#13 /public_html/stats/core/Access.php(664): Piwik\CronArchive->Piwik{closure}()
#14 /public_html/stats/core/CronArchive.php(269): Piwik\Access::doAsSuperUser(Object(Closure))
#15 /public_html/stats/plugins/CoreConsole/Commands/CoreArchiver.php(32): Piwik\CronArchive->main()
#16 /public_html/stats/vendor/symfony/console/Symfony/Component/Console/Command/Command.php(257): Piwik\Plugins\CoreConsole\Commands\CoreArchiver->execute(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#17 /public_html/stats/vendor/symfony/console/Symfony/Component/Console/Application.php(874): Symfony\Component\Console\Command\Command->run(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#18 /public_html/stats/vendor/symfony/console/Symfony/Component/Console/Application.php(195): Symfony\Component\Console\Application->doRunCommand(Object(Piwik\Plugins\CoreConsole\Commands\CoreArchiver), Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#19 [internal function]: Symfony\Component\Console\Application->doRun(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#20 /public_html/stats/core/Console.php(130): call_user_func(Array, Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#21 /public_html/stats/core/Access.php(673): Piwik\Console->Piwik{closure}()
#22 /public_html/stats/core/Console.php(131): Piwik\Access::doAsSuperUser(Object(Closure))
#23 /public_html/stats/core/Console.php(82): Piwik\Console->doRunImpl(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#24 /public_html/stats/vendor/symfony/console/Symfony/Component/Console/Application.php(126): Piwik\Console->doRun(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#25 /public_html/stats/console(32): Symfony\Component\Console\Application->run()
#26 {main}
INFO [2021-08-10 18:45:01] 111145 Start processing archives for site 1.
INFO [2021-08-10 18:45:01] 111145 Will invalidate archived reports for today in site ID = 1's timezone (2021-08-10 00:00:00).
ERROR [2021-08-10 18:45:02] 111145 Uncaught exception: /public_html/stats/core/Period/Factory.php(131): The period 'year' is not supported. Try any of the following instead: day, week, month, range [Query: , CLI mode: 1]

[Exception]
The period 'year' is not supported. Try any of the following instead: day, week, month, range

@diosmosis commented on August 10th 2021 Member

Hi @mattkolb can you run the core:archive command with -vvv and provide the error's stack trace (which should be in the output)?

EDIT: Thanks, good timing.

@diosmosis commented on August 10th 2021 Member

@mattkolb thanks for the stack trace, this looks like a regression, I'll look for a fix today.

@mattkolb commented on August 10th 2021

If it helps, here is the current stack trace:

INFO [2021-08-10 19:50:17] 23574 ---------------------------
INFO [2021-08-10 19:50:17] 23574 INIT
INFO [2021-08-10 19:50:17] 23574 Running Matomo 4.4.1 as Super User
INFO [2021-08-10 19:50:17] 23574 ---------------------------
INFO [2021-08-10 19:50:17] 23574 NOTES
INFO [2021-08-10 19:50:17] 23574 - Async process archiving supported, using CliMulti.
INFO [2021-08-10 19:50:17] 23574 - Reports for today will be processed at most every 10 seconds. You can change this value in Matomo UI > Settings > General Settings.
INFO [2021-08-10 19:50:17] 23574 - Archiving was last executed without error 51 min 36s ago
INFO [2021-08-10 19:50:17] 23574 - Continuing ongoing archiving run by pulling from shared idSite queue.
INFO [2021-08-10 19:50:17] 23574 - Will only process the following periods: day (--force-periods)
INFO [2021-08-10 19:50:17] 23574 ---------------------------
INFO [2021-08-10 19:50:17] 23574 START
INFO [2021-08-10 19:50:17] 23574 Starting Matomo reports archiving...
DEBUG [2021-08-10 19:50:17] 23574 Applying queued rearchiving...
INFO [2021-08-10 19:50:17] 23574 Start processing archives for site 4.
DEBUG [2021-08-10 19:50:17] 23574 Checking for queued invalidations...
DEBUG [2021-08-10 19:50:17] 23574 Today archive can be skipped due to no visits for idSite = 4, skipping invalidation...
DEBUG [2021-08-10 19:50:17] 23574 Yesterday archive can be skipped due to no visits for idSite = 4, skipping invalidation...
DEBUG [2021-08-10 19:50:17] 23574 Done invalidating
DEBUG [2021-08-10 19:50:17] 23574 No next invalidated archive.
INFO [2021-08-10 19:50:17] 23574 Finished archiving for site 4, 0 API requests, Time elapsed: 0.034s [1 / 31 done]
INFO [2021-08-10 19:50:18] 23574 Start processing archives for site 5.
DEBUG [2021-08-10 19:50:18] 23574 Checking for queued invalidations...
INFO [2021-08-10 19:50:18] 23574 Will invalidate archived reports for today in site ID = 5's timezone (2021-08-10 00:00:00).
ERROR [2021-08-10 19:50:18] 23574 Uncaught exception: /public_html/stats/core/Period/Factory.php(131): The period 'year' is not supported. Try any of the following instead: day, week, month, range [Query: , CLI mode: 1]
DEBUG [2021-08-10 19:50:18] 23574 Loaded plugins: CorePluginsAdmin, CoreAdminHome, CoreHome, WebsiteMeasurable, IntranetMeasurable, Diagnostics, CoreVisualizations, Proxy, API, Widgetize, Transitions, LanguagesManager, Actions, Dashboard, MultiSites, Referrers, UserLanguage, DevicesDetection, Goals, Ecommerce, SEO, Events, UserCountry, GeoIp2, VisitsSummary, VisitFrequency, VisitTime, VisitorInterest, Monolog, Login, TwoFactorAuth, UsersManager, SitesManager, Installation, CoreUpdater, CoreConsole, ScheduledReports, UserCountryMap, PrivacyManager, ImageGraph, Annotations, Overlay, SegmentEditor, Insights, Morpheus, Contents, BulkTracking, Resolution, DevicePlugins, Heartbeat, Intl, Marketplace, ProfessionalServices, UserId, Tour, CustomVariables, Provider, WebsiteGroups

[Exception]
The period 'year' is not supported. Try any of the following instead: day, week, month, range

Exception trace:
() at /public_html/stats/core/Period/Factory.php:131
Piwik\Period\Factory::throwExceptionInvalidPeriod() at /public_html/stats/core/Period/Factory.php:118
Piwik\Period\Factory::checkPeriodIsEnabled() at /public_html/stats/core/Period/Factory.php:74
Piwik\Period\Factory::build() at /public_html/stats/core/Archive/ArchiveInvalidator.php:401
Piwik\Archive\ArchiveInvalidator->addParentPeriodsByYearMonth() at /public_html/stats/core/Archive/ArchiveInvalidator.php:403
Piwik\Archive\ArchiveInvalidator->addParentPeriodsByYearMonth() at /public_html/stats/core/Archive/ArchiveInvalidator.php:403
Piwik\Archive\ArchiveInvalidator->addParentPeriodsByYearMonth() at /public_html/stats/core/Archive/ArchiveInvalidator.php:360
Piwik\Archive\ArchiveInvalidator->getAllPeriodsByYearMonth() at /public_html/stats/core/Archive/ArchiveInvalidator.php:314
Piwik\Archive\ArchiveInvalidator->markArchivesAsInvalidated() at /public_html/stats/plugins/CoreAdminHome/API.php:166
Piwik\Plugins\CoreAdminHome\API->invalidateArchivedReports() at /public_html/stats/core/CronArchive.php:888
Piwik\CronArchive->invalidateWithSegments() at /public_html/stats/core/CronArchive.php:864
Piwik\CronArchive->invalidateRecentDate() at /public_html/stats/core/CronArchive.php:814
Piwik\CronArchive->invalidateArchivedReportsForSitesThatNeedToBeArchivedAgainImpl() at /public_html/stats/core/CronArchive.php:768
Piwik\CronArchive->Piwik{closure}() at /public_html/stats/core/Tracker/Cache.php:297
Piwik\Tracker\Cache::withDelegatedCacheClears() at /public_html/stats/core/CronArchive.php:769
Piwik\CronArchive->invalidateArchivedReportsForSitesThatNeedToBeArchivedAgain() at /public_html/stats/core/CronArchive/QueueConsumer.php:164
Piwik\CronArchive\QueueConsumer->getNextArchivesToProcess() at /public_html/stats/core/CronArchive.php:375
Piwik\CronArchive->run() at /public_html/stats/core/CronArchive.php:263
Piwik\CronArchive->Piwik{closure}() at /public_html/stats/core/Access.php:664
Piwik\Access::doAsSuperUser() at /public_html/stats/core/CronArchive.php:269
Piwik\CronArchive->main() at /public_html/stats/plugins/CoreConsole/Commands/CoreArchiver.php:32
Piwik\Plugins\CoreConsole\Commands\CoreArchiver->execute() at /public_html/stats/vendor/symfony/console/Symfony/Component/Console/Command/Command.php:257
Symfony\Component\Console\Command\Command->run() at /public_html/stats/vendor/symfony/console/Symfony/Component/Console/Application.php:874
Symfony\Component\Console\Application->doRunCommand() at /public_html/stats/vendor/symfony/console/Symfony/Component/Console/Application.php:195
Symfony\Component\Console\Application->doRun() at n/a:n/a
call_user_func() at /public_html/stats/core/Console.php:130
Piwik\Console->Piwik{closure}() at /public_html/stats/core/Access.php:673
Piwik\Access::doAsSuperUser() at /public_html/stats/core/Console.php:131
Piwik\Console->doRunImpl() at /public_html/stats/core/Console.php:82
Piwik\Console->doRun() at /public_html/stats/vendor/symfony/console/Symfony/Component/Console/Application.php:126
Symfony\Component\Console\Application->run() at /public_html/stats/console:32

@diosmosis commented on August 10th 2021 Member

@mattkolb I left a PR for this here: https://github.com/matomo-org/matomo/pull/17874. Can you see if it fixes the issue for you?

@mattkolb commented on August 11th 2021

Still appears to have issues. Current output below.

INFO [2021-08-11 13:37:14] 29963 ---------------------------
INFO [2021-08-11 13:37:14] 29963 INIT
INFO [2021-08-11 13:37:14] 29963 Running Matomo 4.4.1 as Super User
INFO [2021-08-11 13:37:14] 29963 ---------------------------
INFO [2021-08-11 13:37:14] 29963 NOTES
INFO [2021-08-11 13:37:14] 29963 - Async process archiving supported, using CliMulti.
INFO [2021-08-11 13:37:14] 29963 - Reports for today will be processed at most every 10 seconds. You can change this value in Matomo UI > Settings > General Settings.
INFO [2021-08-11 13:37:14] 29963 - Archiving was last executed without error 18 hours 38 min ago
INFO [2021-08-11 13:37:14] 29963 - Continuing ongoing archiving run by pulling from shared idSite queue.
INFO [2021-08-11 13:37:14] 29963 - Will only process the following periods: day (--force-periods)
INFO [2021-08-11 13:37:14] 29963 ---------------------------
INFO [2021-08-11 13:37:14] 29963 START
INFO [2021-08-11 13:37:14] 29963 Starting Matomo reports archiving...
DEBUG [2021-08-11 13:37:15] 29963 Applying queued rearchiving...
INFO [2021-08-11 13:37:15] 29963 Start processing archives for site 36.
DEBUG [2021-08-11 13:37:15] 29963 Checking for queued invalidations...
INFO [2021-08-11 13:37:15] 29963 Will invalidate archived reports for today in site ID = 36's timezone (2021-08-11 00:00:00).
INFO [2021-08-11 13:37:15] 29963 Will invalidate archived reports for yesterday in site ID = 36's timezone (2021-08-10 00:00:00).
DEBUG [2021-08-11 13:37:15] 29963 Done invalidating
DEBUG [2021-08-11 13:37:15] 29963 No usable archive exists (ts_archived of existing = 2021-08-10 05:55:13, now = 2021-08-11 13:37:15).
DEBUG [2021-08-11 13:37:15] 29963 Processing invalidation: [idinvalidation = 59, idsite = 36, period = day(2021-08-11 - 2021-08-11), name = done, segment = ].
DEBUG [2021-08-11 13:37:15] 29963 No usable archive exists (ts_archived of existing = , now = 2021-08-11 13:37:15).
DEBUG [2021-08-11 13:37:15] 29963 Processing invalidation: [idinvalidation = 60, idsite = 36, period = day(2021-08-10 - 2021-08-10), name = done, segment = ].
DEBUG [2021-08-11 13:37:15] 29963 Found archive with intersecting period with others in concurrent batch, skipping until next batch: [idinvalidation = 50, idsite = 36, period = week(2021-08-09 - 2021-08-15), name = done, segment = ]
DEBUG [2021-08-11 13:37:15] 29963 Found archive with intersecting period with others in concurrent batch, skipping until next batch: [idinvalidation = 51, idsite = 36, period = month(2021-08-01 - 2021-08-31), name = done, segment = ]
ERROR [2021-08-11 13:37:15] 29963 Uncaught exception: /public_html/stats/core/Period/Factory.php(131): The period 'year' is not supported. Try any of the following instead: day, week, month, range [Query: , CLI mode: 1]
DEBUG [2021-08-11 13:37:15] 29963 Loaded plugins: CorePluginsAdmin, CoreAdminHome, CoreHome, WebsiteMeasurable, IntranetMeasurable, Diagnostics, CoreVisualizations, Proxy, API, Widgetize, Transitions, LanguagesManager, Actions, Dashboard, MultiSites, Referrers, UserLanguage, DevicesDetection, Goals, Ecommerce, SEO, Events, UserCountry, GeoIp2, VisitsSummary, VisitFrequency, VisitTime, VisitorInterest, Monolog, Login, TwoFactorAuth, UsersManager, SitesManager, Installation, CoreUpdater, CoreConsole, ScheduledReports, UserCountryMap, PrivacyManager, ImageGraph, Annotations, Overlay, SegmentEditor, Insights, Morpheus, Contents, BulkTracking, Resolution, DevicePlugins, Heartbeat, Intl, Marketplace, ProfessionalServices, UserId, Tour, CustomVariables, Provider, WebsiteGroups

[Exception]
The period 'year' is not supported. Try any of the following instead: day, week, month, range

Exception trace:
() at /public_html/stats/core/Period/Factory.php:131
Piwik\Period\Factory::throwExceptionInvalidPeriod() at /public_html/stats/core/Period/Factory.php:118
Piwik\Period\Factory::checkPeriodIsEnabled() at /public_html/stats/core/Period/Factory.php:74
Piwik\Period\Factory::build() at /public_html/stats/core/CronArchive/QueueConsumer.php:351
Piwik\CronArchive\QueueConsumer->getNextInvalidatedArchive() at /public_html/stats/core/CronArchive/QueueConsumer.php:180
Piwik\CronArchive\QueueConsumer->getNextArchivesToProcess() at /public_html/stats/core/CronArchive.php:375
Piwik\CronArchive->run() at /public_html/stats/core/CronArchive.php:263
Piwik\CronArchive->Piwik{closure}() at /public_html/stats/core/Access.php:664
Piwik\Access::doAsSuperUser() at /public_html/stats/core/CronArchive.php:269
Piwik\CronArchive->main() at /public_html/stats/plugins/CoreConsole/Commands/CoreArchiver.php:32
Piwik\Plugins\CoreConsole\Commands\CoreArchiver->execute() at /public_html/stats/vendor/symfony/console/Symfony/Component/Console/Command/Command.php:257
Symfony\Component\Console\Command\Command->run() at /public_html/stats/vendor/symfony/console/Symfony/Component/Console/Application.php:874
Symfony\Component\Console\Application->doRunCommand() at /public_html/stats/vendor/symfony/console/Symfony/Component/Console/Application.php:195
Symfony\Component\Console\Application->doRun() at n/a:n/a
call_user_func() at /public_html/stats/core/Console.php:130
Piwik\Console->Piwik{closure}() at /public_html/stats/core/Access.php:673
Piwik\Access::doAsSuperUser() at /public_html/stats/core/Console.php:131
Piwik\Console->doRunImpl() at /public_html/stats/core/Console.php:82
Piwik\Console->doRun() at /public_html/stats/vendor/symfony/console/Symfony/Component/Console/Application.php:126
Symfony\Component\Console\Application->run() at /public_html/stats/console:32

@diosmosis commented on August 12th 2021 Member

@mattkolb I fixed a couple more issues in the same pull request: https://github.com/matomo-org/matomo/pull/17874, can you check if it now fixes the issue you're experiencing?

@mattkolb commented on August 12th 2021

That appears to have fixed it. Thanks for the fast turnaround time. Am I correct that these changes will most likely be included in the 4.5 release?

@diosmosis commented on August 12th 2021 Member

@mattkolb yes, 4.5. Unlikely to be earlier.

This Issue was closed on August 12th 2021
Powered by GitHub Issue Mirror