Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Fix possible error when generating reports for custom dimensions #18608

Closed
tsteur opened this issue Jan 12, 2022 · 3 comments · Fixed by #18614
Closed

Fix possible error when generating reports for custom dimensions #18608

tsteur opened this issue Jan 12, 2022 · 3 comments · Fixed by #18614
Assignees
@sgiehl
Labels
Potential Bug Something that might be a bug, but needs validation and confirmation it can be reproduced. Regression Indicates a feature used to work in a certain way but it no longer does even though it should.
Milestone
4.7.0

Comments

@tsteur
Copy link
Member

tsteur commented Jan 12, 2022

This is on PHP 8.

Uncaught exception: TypeError: Unsupported operand types: string / int in /core/Archive/Chunk.php:33
Stack trace:
#0 /core/DataAccess/ArchiveSelector.php(293): Piwik\Archive\Chunk->getRecordNameForTableId('CustomDimension...', '')
#1 /core/Archive.php(528): Piwik\DataAccess\ArchiveSelector::getArchiveData(Array, Array, 'blob', '')
#2 /core/Archive.php(373): Piwik\Archive->get(Array, 'blob', '')
#3 /core/Archive.php(464): Piwik\Archive->getDataTable('CustomDimension...', '')
#4 /plugins/CustomDimensions/API.php(63): Piwik\Archive::createDataTableFromArchive('CustomDimension...', '3', 'day', '2022-01-11', false, false, false, '')
#5 [internal function]: Piwik\Plugins\CustomDimensions\API->getCustomDimension('1', '3', 'day', '2022-01-11', false, false, false, '')
#6 /core/API/Proxy.php(244): call_user_func_array(Array, Array)
#7 /core/Context.php(28): Piwik\API\Proxy->Piwik\API{closure}()
#8 /core/API/Proxy.php(335): Piwik\Context::executeWithQueryParameters(Array, Object(Closure))
#9 /core/API/Request.php(266): Piwik\API\Proxy->call('\Piwik\Plugins\...', 'getCustomDimens...', Array)
#10 /plugins/API/ProcessedReport.php(378): Piwik\API\Request->process()
#11 /plugins/API/API.php(287): Piwik\Plugins\API\ProcessedReport->getProcessedReport('3', 'day', '2022-01-11', 'CustomDimension...', 'getCustomDimens...', false, Array, false, 'de', '1', false, false, false, 'bc', '1')
#12 [internal function]: Piwik\Plugins\API\API->getProcessedReport('3', 'day', '2022-01-11', 'CustomDimension...', 'getCustomDimens...', false, false, false, 'de', '1', false, false, false, 'bc', '1')
#13 /core/API/Proxy.php(244): call_user_func_array(Array, Array)
#14 /core/Context.php(28): Piwik\API\Proxy->Piwik\API{closure}()
#15 /core/API/Proxy.php(335): Piwik\Context::executeWithQueryParameters(Array, Object(Closure))
#16 /core/API/Request.php(266): Piwik\API\Proxy->call('\Piwik\Plugins\...', 'getProcessedRep...', Array)
#17 /core/API/Request.php(559): Piwik\API\Request->process()
#18 /plugins/ImageGraph/API.php(379): Piwik\API\Request::processRequest('API.getProcesse...', Array)
#19 [internal function]: Piwik\Plugins\ImageGraph\API->get(3, 'day', '2022-01-11', 'CustomDimension...', 'getCustomDimens...', 'horizontalBar', '2', false, false, true, 760, 220, 9, 11, true, false, false, '222222', 'FFFFFF', 'CCCCCC', false, true, false, '1')
#20 /core/API/Proxy.php(244): call_user_func_array(Array, Array)
#21 /core/Context.php(28): Piwik\API\Proxy->Piwik\API{closure}()
#22 /core/API/Proxy.php(335): Piwik\Context::executeWithQueryParameters(Array, Object(Closure))
#23 /core/API/Request.php(266): Piwik\API\Proxy->call('\Piwik\Plugins\...', 'get', Array)
#24 /core/ReportRenderer.php(270): Piwik\API\Request->process()
#25 /core/ReportRenderer/Pdf.php(433): Piwik\ReportRenderer::getStaticGraph(Array, 760, 220, false, NULL)
#26 /core/ReportRenderer/Pdf.php(324): Piwik\ReportRenderer\Pdf->paintGraph()
#27 [internal function]: Piwik\ReportRenderer\Pdf->renderReport(Array, 48)
#28 /plugins/ScheduledReports/API.php(540): array_walk(Array, Array)
#29 /plugins/ScheduledReports/API.php(608): Piwik\Plugins\ScheduledReports\API->generateReport('4', '2022-01-11', 'de', 2, 'day')
#30 /core/Context.php(75): Piwik\Plugins\ScheduledReports\API->Piwik\Plugins\ScheduledReports{closure}()
#31 /plugins/ScheduledReports/API.php(679): Piwik\Context::changeIdSite('3', Object(Closure))
#32 [internal function]: Piwik\Plugins\ScheduledReports\API->sendReport('4')
#33 /core/Scheduler/Scheduler.php(276): call_user_func(Array, '4')
#34 /core/Scheduler/Scheduler.php(148): Piwik\Scheduler\Scheduler->executeTask(Object(Piwik\Scheduler\Task))
#35 /plugins/CoreAdminHome/API.php(69): Piwik\Scheduler\Scheduler->run()
#36 /core/CronArchive.php(627): Piwik\Plugins\CoreAdminHome\API->runScheduledTasks()
#37 /core/CronArchive.php(264): Piwik\CronArchive->runScheduledTasks()
#38 /core/Access.php(661): Piwik\CronArchive->Piwik{closure}()
#39 /core/CronArchive.php(269): Piwik\Access::doAsSuperUser(Object(Closure))
#40 /plugins/CoreConsole/Commands/CoreArchiver.php(32): Piwik\CronArchive->main()
#41 /vendor/symfony/console/Symfony/Component/Console/Command/Command.php(257): Piwik\Plugins\CoreConsole\Commands\CoreArchiver->execute(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#42 /vendor/symfony/console/Symfony/Component/Console/Application.php(874): Symfony\Component\Console\Command\Command->run(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#43 /vendor/symfony/console/Symfony/Component/Console/Application.php(195): Symfony\Component\Console\Application->doRunCommand(Object(Piwik\Plugins\CoreConsole\Commands\CoreArchiver), Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#44 [internal function]: Symfony\Component\Console\Application->doRun(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#45 /core/Console.php(130): call_user_func(Array, Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#46 /core/Access.php(670): Piwik\Console->Piwik{closure}()
#47 /core/Console.php(131): Piwik\Access::doAsSuperUser(Object(Closure))
#48 /core/Console.php(82): Piwik\Console->doRunImpl(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#49 /vendor/symfony/console/Symfony/Component/Console/Application.php(126): Piwik\Console->doRun(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#50 /console(32): Symfony\Component\Console\Application->run()
#51 {main}
@tsteur tsteur added the Potential Bug Something that might be a bug, but needs validation and confirmation it can be reproduced. label Jan 12, 2022
@tsteur tsteur added this to the 4.7.0 milestone Jan 12, 2022
@tsteur tsteur added the Regression Indicates a feature used to work in a certain way but it no longer does even though it should. label Jan 12, 2022
@tsteur
Copy link
Member Author

tsteur commented Jan 12, 2022

Marking this as a regression as it used to work on older PHP versions and now prevents us from sending scheduled reports.

@sgiehl
Copy link
Member

sgiehl commented Jan 12, 2022

Hm. I wonder if that might be an edge case maybe. I did run all tests on PHP 8 / 8.1 and fixed all errors / warnings that occurred. So either this case isn't handled by any test, or it's a special case. Are you able to look up the scheduled reports definition so I can try to reproduce that locally?

When looking at the code it might actually be a more complex issue. As you can see in the stack trace the API method CustomDimension.getCustomDimension is called with an empty string as $idSubtable. That is passed through to the Archive and ArchiveSelector class. As those classes only check if the value of idSubtable is null (or false), this might even cause the archive names to have an appended _, caused by

matomo/core/Archive.php

Lines 498 to 507 in 556aada

if ($idSubtable !== null
&& $idSubtable !== self::ID_SUBTABLE_LOAD_ALL_SUBTABLES
) {
// this is also done in ArchiveSelector. It should be actually only done in ArchiveSelector but DataCollection
// does require to have the subtableId appended. Needs to be changed in refactoring to have it only in one
// place.
$dataNames = array();
foreach ($archiveNames as $name) {
$dataNames[] = ArchiveSelector::appendIdsubtable($name, $idSubtable);
}

matomo/core/DataAccess/ArchiveSelector.php

Lines 371 to 374 in 88faa0a

public static function appendIdSubtable($recordName, $id)
{
return $recordName . "_" . $id;
}

Logically it would be correct to handle an empty string the same as null in that case. I have pushed an according change to see if that breaks any tests (https://github.com/matomo-org/matomo/compare/archivefix?expand=1).

Btw. are warnings disabled? That code should have produced a warning on PHP 7 already.

@tsteur
Copy link
Member Author

tsteur commented Jan 12, 2022

@sgiehl below the definition

INSERT INTO `report` (`idreport`, `idsite`, `login`, `description`, `idsegment`, `period`, `hour`, `type`, `format`, `reports`, `parameters`, `ts_created`, `ts_last_sent`, `deleted`, `evolution_graph_within_period`, `evolution_graph_period_n`, `period_param`)
VALUES
	(4, 3, 'example', 'test', NULL, 'day', 0, 'email', 'pdf', '[\"MultiSites_getAll\",\"VisitsSummary_get\",\"UserCountry_getRegion\",\"UserLanguage_getLanguage\",\"UserCountry_getCity\",\"UserLanguage_getLanguageCode\",\"DevicesDetection_getType\",\"DevicesDetection_getModel\",\"DevicesDetection_getBrand\",\"Resolution_getResolution\",\"DevicesDetection_getOsVersions\",\"DevicesDetection_getBrowsers\",\"DevicesDetection_getBrowserVersions\",\"Resolution_getConfiguration\",\"DevicesDetection_getOsFamilies\",\"DevicesDetection_getBrowserEngines\",\"DevicePlugins_getPlugin\",\"VisitTime_getVisitInformationPerLocalTime\",\"VisitTime_getVisitInformationPerServerTime\",\"VisitTime_getByDayOfWeek\",\"UserId_getUsers\",\"CustomVariables_getCustomVariables\",\"Actions_get\",\"UsersFlow_getUsersFlowPretty\",\"Actions_getPageUrls\",\"Actions_getEntryPageUrls\",\"Actions_getEntryPageTitles\",\"Actions_getExitPageUrls\",\"Actions_getExitPageTitles\",\"Actions_getPageTitles\",\"Actions_getSiteSearchKeywords\",\"Actions_getPageUrlsFollowingSiteSearch\",\"Actions_getSiteSearchNoResultKeywords\",\"Actions_getPageTitlesFollowingSiteSearch\",\"Actions_getSiteSearchCategories\",\"Actions_getOutlinks\",\"Actions_getDownloads\",\"Events_getCategory\",\"Events_getAction\",\"Events_getName\",\"Contents_getContentNames\",\"Contents_getContentPieces\",\"VisitorInterest_getNumberOfVisitsPerVisitDuration\",\"VisitorInterest_getNumberOfVisitsPerPage\",\"VisitorInterest_getNumberOfVisitsByVisitCount\",\"VisitorInterest_getNumberOfVisitsByDaysSinceLast\",\"VisitFrequency_get\",\"PagePerformance_get\",\"CustomDimensions_getCustomDimension_idDimension--1\",\"CustomDimensions_getCustomDimension_idDimension--3\",\"CustomDimensions_getCustomDimension_idDimension--2\",\"Referrers_get\",\"Referrers_getReferrerType\",\"Referrers_getAll\",\"Referrers_getKeywords\",\"Referrers_getSearchEngines\",\"Referrers_getWebsites\",\"Referrers_getSocials\",\"MarketingCampaignsReporting_getName\",\"MarketingCampaignsReporting_getKeyword\",\"MarketingCampaignsReporting_getSource\",\"MarketingCampaignsReporting_getMedium\",\"MarketingCampaignsReporting_getContent\",\"MarketingCampaignsReporting_getSourceMedium\",\"MarketingCampaignsReporting_getId\",\"MarketingCampaignsReporting_getGroup\",\"MarketingCampaignsReporting_getPlacement\",\"Goals_get\",\"Goals_getVisitsUntilConversion\",\"Goals_getDaysToConversion\",\"Goals_get_idGoal--0\",\"Goals_getVisitsUntilConversion_idGoal--0\",\"Goals_getDaysToConversion_idGoal--0\",\"Goals_get_idGoal--1\",\"Goals_getVisitsUntilConversion_idGoal--1\",\"Goals_getDaysToConversion_idGoal--1\",\"Goals_get_idGoal--2\",\"Goals_getVisitsUntilConversion_idGoal--2\",\"Goals_getDaysToConversion_idGoal--2\",\"Goals_get_idGoal--3\",\"Goals_getVisitsUntilConversion_idGoal--3\",\"Goals_getDaysToConversion_idGoal--3\",\"FormAnalytics_get\",\"FormAnalytics_get_idForm--5\",\"FormAnalytics_getPageUrls_idForm--5\",\"FormAnalytics_get_idForm--6\",\"FormAnalytics_getPageUrls_idForm--6\",\"MediaAnalytics_get\",\"MediaAnalytics_getVideoTitles\",\"MediaAnalytics_getVideoResources\",\"MediaAnalytics_getGroupedVideoResources\",\"MediaAnalytics_getVideoHours\",\"MediaAnalytics_getVideoResolutions\",\"MediaAnalytics_getAudioTitles\",\"MediaAnalytics_getAudioResources\",\"MediaAnalytics_getGroupedAudioResources\",\"MediaAnalytics_getAudioHours\",\"MediaAnalytics_getPlayers\",\"CustomReports_getCustomReport_idCustomReport--3\"]', '{\"displayFormat\":\"3\",\"emailMe\":true,\"evolutionGraph\":false}', '2021-09-16 11:31:00', '2022-01-10 01:39:33', 0, 0, 30, 'day');

I don't think it produced a warning before but it's possible. Usually AFAIK we get emails when there's any warning during archiving.

@sgiehl sgiehl self-assigned this Jan 13, 2022
@sgiehl sgiehl mentioned this issue Jan 13, 2022
Merged
11 tasks
@justinvelluppillai justinvelluppillai changed the title Uncaught exception: TypeError: Unsupported operand types: string / int in /core/Archive/Chunk.php:33 Fix possible error when generating reports for custom dimensions Feb 1, 2022
@tsteur tsteur mentioned this issue Mar 28, 2022
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@sgiehl sgiehl

Labels
Potential Bug Something that might be a bug, but needs validation and confirmation it can be reproduced. Regression Indicates a feature used to work in a certain way but it no longer does even though it should.
Projects
None yet
Milestone
4.7.0
Development

Successfully merging a pull request may close this issue.

Fix possible error when generating reports for custom dimensions matomo-org/matomo
2 participants
@tsteur @sgiehl