@nelhefni
opened this Issue on April 7th 2021
@nelhefni
Hello,
We think PR https://github.com/matomo-org/matomo/pull/17216 has introduced some regressions to the archiving process since Matomo 4.2.0
FYI, browser archiving is disabled:
enable_browser_archiving_triggering = 0
browser_archiving_disabled_enforce = 1
archiving_range_force_on_browser_request = 0For a given plugin, we are facing 2 issues since the update to Matomo 4.2.0,
- Issue#1: Some daily reports disappear from the UI, even though
delete_reports_enableis set to 0 - Issue#2: Some weekly reports have inaccurate data: gather the values of only some days of the week.
This seems to be related to the changes made with the handling of partial archives.
Indeed, we have a lot of partial archives in our DB, which doesn't contain the reports of our impacted plugins:
MariaDB [matomo]> select * from matomo_archive_numeric_2021_03 where idarchive = 46176;
| idarchive | name | idsite | date1 | date2 | period | ts_archived | value |
| 46176 | myPlugin | 1 | 2021-03-22 | 2021-03-22 | 1 | 2021-03-23 00:05:52 | 10 |
| 46176 | otherPlugin | 1 | 2021-03-22 | 2021-03-22 | 1 | 2021-03-23 00:05:52 | 20 |
| 46176 | done | 1 | 2021-03-22 | 2021-03-22 | 1 | 2021-03-23 00:05:52 | 1 |MariaDB [matomo]> select * from matomo_archive_numeric_2021_03 where idarchive = 56405;
| idarchive | name | idsite | date1 | date2 | period | ts_archived | value |
| 56405 | otherPlugin | 1 | 2021-03-22 | 2021-03-22 | 1 | 2021-04-05 18:52:03 | 20 |
| 56405 | done | 1 | 2021-03-22 | 2021-03-22 | 1 | 2021-04-05 18:52:03 | 5 |As you can see, myPlugin report is not part of partial archive 56405 (partial because the value of done row is 5), but it is nevertheless selected since Matomo 4.2.0, as ts_archived is more recent.
Since https://github.com/matomo-org/matomo/pull/17216, checkAuthorizedToArchive = false is passed to Rules::getSelectableDoneFlagValues during the retrival of archive to display in the UI
https://github.com/matomo-org/matomo/blob/2c30dc8d231af58cdf2516a413da2d86bf7d62c6/core/DataAccess/ArchiveSelector.php#L370
As a consequence, both ArchiveWriter::DONE_PARTIAL and ArchiveWriter::DONE_INVALIDATED are added to the list of accepted value. Which explains why archive 56405 is selected instead of 46176 in our case.
https://github.com/matomo-org/matomo/blob/2c30dc8d231af58cdf2516a413da2d86bf7d62c6/core/ArchiveProcessor/Rules.php#L321
Thus, as soon as archive 56405 was created, myPlugin report disappeared from the UI.
I don't think ArchiveWriter::DONE_PARTIAL should be part of the list of accepted values.
We have been able to fix this issue#1 with this PR https://github.com/matomo-org/matomo/pull/17425
Concerning issue#2, we have been able to fix it by reverting the changes done in the same PR to ArchiveSelector::findArchiveDataWithLatestTsArchived, and then invalidating faulty reports and re-running the cron job.
Your Environment
Matomo 4.2.1
@diosmosis
commented
on April 7th 2021
@diosmosis
Hi @nelhefni, thanks for the detailed report. Partial archives are meant to be included in the entire list of all idarchives to look through. We don't select only one archive, but the latest DONE_OK or DONE_INVALIDATED archive, then all DONE_PARTIAL archives that are later. We then look for data in all of those archives. Using this knowledge, can you take another look? Or if possible can you provide access so we can take a closer look?
@nelhefni
commented
on April 8th 2021
Hi @diosmosis, thanks for the fast answer.
I've been tracing the code, and ArchiveSelector::getArchiveIds returns only one idarchive
archiveIds =
Array
(
[0] => Array
(
[idsite] => 1
[date1] => 2021-03-22
[date2] => 2021-03-22
[archives] =>
56405|done|5,53884|done|5,49890|done.Funnels|5,46176|done|1
)
)As there is only one element in $idarchives array, we loop only once in https://github.com/matomo-org/matomo/blob/2c30dc8d231af58cdf2516a413da2d86bf7d62c6/core/DataAccess/ArchiveSelector.php#L202
$pairs =
Array
(
[0] => 56405|done|5
[1] => 53884|done|5
[2] => 49890|done.Funnels|5
[3] => 46176|done|1
)then we break, because idarchive 56405 contains done without any dot https://github.com/matomo-org/matomo/blob/2c30dc8d231af58cdf2516a413da2d86bf7d62c6/core/DataAccess/ArchiveSelector.php#L212
finally, we have only one idarchive returned https://github.com/matomo-org/matomo/blob/2c30dc8d231af58cdf2516a413da2d86bf7d62c6/core/Archive.php#L638
$idarchivesByReport =
Array
(
[done] => Array
(
[2021-03-22,2021-03-22] => Array
(
[0] => 56405
)
)
)
@diosmosis
commented
on April 8th 2021
@nelhefni thanks again for the detailed post. That code looks broken but it's actually working correctly, the problem is that archives like 56405|done|5 should never exist... partial archives have to be specific to a plugin, they should never have a done flag of just done.
Can you see what reports are in 56405 (eg, what names are in the blob table for 56405)?
I think we can put in some code making sure partial archives are never saved w/ that done flag, and modify the code you reference just in case this edge case happens in some other way, but I can't think of how this might happen.
@nelhefni
commented
on April 8th 2021
@diosmosis thanks for the input, the issue could then be a side effect of https://github.com/matomo-org/matomo/issues/17434
Here are the reports for idarchive 56405:
MariaDB [matomo]> select * from matomo_archive_numeric_2021_03 where idarchive = 56405;
| idarchive | name | idsite | date1 | date2 | period | ts_archived | value |
| 56405 | Actions_nb_outlinks | 1 | 2021-03-22 | 2021-03-22 | 1 | 2021-04-05 18:52:03 | 999 |
| 56405 | Actions_nb_pageviews | 1 | 2021-03-22 | 2021-03-22 | 1 | 2021-04-05 18:52:03 | 999 |
| 56405 | Actions_nb_uniq_outlinks | 1 | 2021-03-22 | 2021-03-22 | 1 | 2021-04-05 18:52:03 | 999 |
| 56405 | Actions_nb_uniq_pageviews | 1 | 2021-03-22 | 2021-03-22 | 1 | 2021-04-05 18:52:03 | 999 |
| 56405 | bounce_count | 1 | 2021-03-22 | 2021-03-22 | 1 | 2021-04-05 18:52:03 | 999 |
| 56405 | done | 1 | 2021-03-22 | 2021-03-22 | 1 | 2021-04-05 18:52:03 | 5 |
| 56405 | Goal_10_nb_conversions | 1 | 2021-03-22 | 2021-03-22 | 1 | 2021-04-05 18:52:03 | 999 |
| 56405 | Goal_10_nb_visits_converted | 1 | 2021-03-22 | 2021-03-22 | 1 | 2021-04-05 18:52:03 | 999 |
| 56405 | Goal_10_revenue | 1 | 2021-03-22 | 2021-03-22 | 1 | 2021-04-05 18:52:03 | 99.9 |
| 56405 | Goal_11_nb_conversions | 1 | 2021-03-22 | 2021-03-22 | 1 | 2021-04-05 18:52:03 | 999 |
| 56405 | Goal_11_nb_visits_converted | 1 | 2021-03-22 | 2021-03-22 | 1 | 2021-04-05 18:52:03 | 999 |
| 56405 | Goal_12_nb_conversions | 1 | 2021-03-22 | 2021-03-22 | 1 | 2021-04-05 18:52:03 | 999 |
| 56405 | Goal_12_nb_visits_converted | 1 | 2021-03-22 | 2021-03-22 | 1 | 2021-04-05 18:52:03 | 999 |
| 56405 | Goal_13_nb_conversions | 1 | 2021-03-22 | 2021-03-22 | 1 | 2021-04-05 18:52:03 | 999 |
| 56405 | Goal_13_nb_visits_converted | 1 | 2021-03-22 | 2021-03-22 | 1 | 2021-04-05 18:52:03 | 999 |
| 56405 | Goal_14_nb_conversions | 1 | 2021-03-22 | 2021-03-22 | 1 | 2021-04-05 18:52:03 | 999 |
| 56405 | Goal_14_nb_visits_converted | 1 | 2021-03-22 | 2021-03-22 | 1 | 2021-04-05 18:52:03 | 999 |
| 56405 | Goal_15_nb_conversions | 1 | 2021-03-22 | 2021-03-22 | 1 | 2021-04-05 18:52:03 | 999 |
| 56405 | Goal_15_nb_visits_converted | 1 | 2021-03-22 | 2021-03-22 | 1 | 2021-04-05 18:52:03 | 999 |
| 56405 | Goal_16_nb_conversions | 1 | 2021-03-22 | 2021-03-22 | 1 | 2021-04-05 18:52:14 | 999 |
| 56405 | Goal_16_nb_visits_converted | 1 | 2021-03-22 | 2021-03-22 | 1 | 2021-04-05 18:52:14 | 999 |
| 56405 | Goal_17_nb_conversions | 1 | 2021-03-22 | 2021-03-22 | 1 | 2021-04-05 18:52:14 | 999 |
| 56405 | Goal_17_nb_visits_converted | 1 | 2021-03-22 | 2021-03-22 | 1 | 2021-04-05 18:52:14 | 999 |
| 56405 | Goal_18_nb_conversions | 1 | 2021-03-22 | 2021-03-22 | 1 | 2021-04-05 18:52:14 | 999 |
| 56405 | Goal_18_nb_visits_converted | 1 | 2021-03-22 | 2021-03-22 | 1 | 2021-04-05 18:52:14 | 999 |
| 56405 | Goal_19_nb_conversions | 1 | 2021-03-22 | 2021-03-22 | 1 | 2021-04-05 18:52:14 | 999 |
| 56405 | Goal_19_nb_visits_converted | 1 | 2021-03-22 | 2021-03-22 | 1 | 2021-04-05 18:52:14 | 999 |
| 56405 | Goal_1_nb_conversions | 1 | 2021-03-22 | 2021-03-22 | 1 | 2021-04-05 18:52:03 | 999 |
| 56405 | Goal_1_nb_visits_converted | 1 | 2021-03-22 | 2021-03-22 | 1 | 2021-04-05 18:52:03 | 999 |
| 56405 | Goal_20_nb_conversions | 1 | 2021-03-22 | 2021-03-22 | 1 | 2021-04-05 18:52:14 | 999 |
| 56405 | Goal_20_nb_visits_converted | 1 | 2021-03-22 | 2021-03-22 | 1 | 2021-04-05 18:52:14 | 999 |
| 56405 | Goal_21_nb_conversions | 1 | 2021-03-22 | 2021-03-22 | 1 | 2021-04-05 18:52:14 | 999 |
| 56405 | Goal_21_nb_visits_converted | 1 | 2021-03-22 | 2021-03-22 | 1 | 2021-04-05 18:52:14 | 999 |
| 56405 | Goal_22_nb_conversions | 1 | 2021-03-22 | 2021-03-22 | 1 | 2021-04-05 18:52:14 | 999 |
| 56405 | Goal_22_nb_visits_converted | 1 | 2021-03-22 | 2021-03-22 | 1 | 2021-04-05 18:52:14 | 999 |
| 56405 | Goal_23_nb_conversions | 1 | 2021-03-22 | 2021-03-22 | 1 | 2021-04-05 18:52:14 | 999 |
| 56405 | Goal_23_nb_visits_converted | 1 | 2021-03-22 | 2021-03-22 | 1 | 2021-04-05 18:52:14 | 999 |
| 56405 | Goal_24_nb_conversions | 1 | 2021-03-22 | 2021-03-22 | 1 | 2021-04-05 18:52:14 | 999 |
| 56405 | Goal_24_nb_visits_converted | 1 | 2021-03-22 | 2021-03-22 | 1 | 2021-04-05 18:52:14 | 999 |
| 56405 | Goal_25_nb_conversions | 1 | 2021-03-22 | 2021-03-22 | 1 | 2021-04-05 18:52:14 | 999 |
| 56405 | Goal_25_nb_visits_converted | 1 | 2021-03-22 | 2021-03-22 | 1 | 2021-04-05 18:52:14 | 999 |
| 56405 | Goal_26_nb_conversions | 1 | 2021-03-22 | 2021-03-22 | 1 | 2021-04-05 18:52:14 | 999 |
| 56405 | Goal_26_nb_visits_converted | 1 | 2021-03-22 | 2021-03-22 | 1 | 2021-04-05 18:52:14 | 999 |
| 56405 | Goal_27_nb_conversions | 1 | 2021-03-22 | 2021-03-22 | 1 | 2021-04-05 18:52:14 | 999 |
| 56405 | Goal_27_nb_visits_converted | 1 | 2021-03-22 | 2021-03-22 | 1 | 2021-04-05 18:52:14 | 999 |
| 56405 | Goal_28_nb_conversions | 1 | 2021-03-22 | 2021-03-22 | 1 | 2021-04-05 18:52:14 | 999 |
| 56405 | Goal_28_nb_visits_converted | 1 | 2021-03-22 | 2021-03-22 | 1 | 2021-04-05 18:52:14 | 999 |
| 56405 | Goal_29_nb_conversions | 1 | 2021-03-22 | 2021-03-22 | 1 | 2021-04-05 18:52:14 | 999 |
| 56405 | Goal_29_nb_visits_converted | 1 | 2021-03-22 | 2021-03-22 | 1 | 2021-04-05 18:52:14 | 999 |
| 56405 | Goal_2_nb_conversions | 1 | 2021-03-22 | 2021-03-22 | 1 | 2021-04-05 18:52:03 | 999 |
| 56405 | Goal_2_nb_visits_converted | 1 | 2021-03-22 | 2021-03-22 | 1 | 2021-04-05 18:52:03 | 999 |
| 56405 | Goal_30_nb_conversions | 1 | 2021-03-22 | 2021-03-22 | 1 | 2021-04-05 18:52:14 | 999 |
| 56405 | Goal_30_nb_visits_converted | 1 | 2021-03-22 | 2021-03-22 | 1 | 2021-04-05 18:52:14 | 999 |
| 56405 | Goal_31_nb_conversions | 1 | 2021-03-22 | 2021-03-22 | 1 | 2021-04-05 18:52:14 | 999 |
| 56405 | Goal_31_nb_visits_converted | 1 | 2021-03-22 | 2021-03-22 | 1 | 2021-04-05 18:52:14 | 999 |
| 56405 | Goal_3_nb_conversions | 1 | 2021-03-22 | 2021-03-22 | 1 | 2021-04-05 18:52:03 | 999 |
| 56405 | Goal_3_nb_visits_converted | 1 | 2021-03-22 | 2021-03-22 | 1 | 2021-04-05 18:52:03 | 999 |
| 56405 | Goal_4_nb_conversions | 1 | 2021-03-22 | 2021-03-22 | 1 | 2021-04-05 18:52:03 | 999 |
| 56405 | Goal_4_nb_visits_converted | 1 | 2021-03-22 | 2021-03-22 | 1 | 2021-04-05 18:52:03 | 999 |
| 56405 | Goal_5_nb_conversions | 1 | 2021-03-22 | 2021-03-22 | 1 | 2021-04-05 18:52:03 | 999 |
| 56405 | Goal_5_nb_visits_converted | 1 | 2021-03-22 | 2021-03-22 | 1 | 2021-04-05 18:52:03 | 999 |
| 56405 | Goal_6_nb_conversions | 1 | 2021-03-22 | 2021-03-22 | 1 | 2021-04-05 18:52:03 | 999 |
| 56405 | Goal_6_nb_visits_converted | 1 | 2021-03-22 | 2021-03-22 | 1 | 2021-04-05 18:52:03 | 999 |
| 56405 | Goal_7_nb_conversions | 1 | 2021-03-22 | 2021-03-22 | 1 | 2021-04-05 18:52:03 | 999 |
| 56405 | Goal_7_nb_visits_converted | 1 | 2021-03-22 | 2021-03-22 | 1 | 2021-04-05 18:52:03 | 999 |
| 56405 | Goal_8_nb_conversions | 1 | 2021-03-22 | 2021-03-22 | 1 | 2021-04-05 18:52:03 | 999 |
| 56405 | Goal_8_nb_visits_converted | 1 | 2021-03-22 | 2021-03-22 | 1 | 2021-04-05 18:52:03 | 999 |
| 56405 | Goal_9_nb_conversions | 1 | 2021-03-22 | 2021-03-22 | 1 | 2021-04-05 18:52:03 | 999 |
| 56405 | Goal_9_nb_visits_converted | 1 | 2021-03-22 | 2021-03-22 | 1 | 2021-04-05 18:52:03 | 999 |
| 56405 | Goal_nb_conversions | 1 | 2021-03-22 | 2021-03-22 | 1 | 2021-04-05 18:52:14 | 999 |
| 56405 | Goal_nb_visits_converted | 1 | 2021-03-22 | 2021-03-22 | 1 | 2021-04-05 18:52:14 | 999 |
| 56405 | Goal_revenue | 1 | 2021-03-22 | 2021-03-22 | 1 | 2021-04-05 18:52:14 | 99.9 |
| 56405 | max_actions | 1 | 2021-03-22 | 2021-03-22 | 1 | 2021-04-05 18:52:03 | 999 |
| 56405 | nb_actions | 1 | 2021-03-22 | 2021-03-22 | 1 | 2021-04-05 18:52:03 | 999 |
| 56405 | nb_uniq_visitors | 1 | 2021-03-22 | 2021-03-22 | 1 | 2021-04-05 18:52:03 | 999 |
| 56405 | nb_users | 1 | 2021-03-22 | 2021-03-22 | 1 | 2021-04-05 18:52:03 | 999 |
| 56405 | nb_visits | 1 | 2021-03-22 | 2021-03-22 | 1 | 2021-04-05 18:52:03 | 999 |
| 56405 | nb_visits_converted | 1 | 2021-03-22 | 2021-03-22 | 1 | 2021-04-05 18:52:03 | 999 |
| 56405 | PagePerformance_domcompletion_hits | 1 | 2021-03-22 | 2021-03-22 | 1 | 2021-04-05 18:52:14 | 999 |
| 56405 | PagePerformance_domcompletion_time | 1 | 2021-03-22 | 2021-03-22 | 1 | 2021-04-05 18:52:14 | 999 |
| 56405 | PagePerformance_domprocessing_hits | 1 | 2021-03-22 | 2021-03-22 | 1 | 2021-04-05 18:52:14 | 999 |
| 56405 | PagePerformance_domprocessing_time | 1 | 2021-03-22 | 2021-03-22 | 1 | 2021-04-05 18:52:14 | 999 |
| 56405 | PagePerformance_network_hits | 1 | 2021-03-22 | 2021-03-22 | 1 | 2021-04-05 18:52:14 | 999 |
| 56405 | PagePerformance_network_time | 1 | 2021-03-22 | 2021-03-22 | 1 | 2021-04-05 18:52:14 | 999 |
| 56405 | PagePerformance_onload_hits | 1 | 2021-03-22 | 2021-03-22 | 1 | 2021-04-05 18:52:14 | 999 |
| 56405 | PagePerformance_onload_time | 1 | 2021-03-22 | 2021-03-22 | 1 | 2021-04-05 18:52:14 | 999 |
| 56405 | PagePerformance_pageload_hits | 1 | 2021-03-22 | 2021-03-22 | 1 | 2021-04-05 18:52:14 | 999 |
| 56405 | PagePerformance_pageload_time | 1 | 2021-03-22 | 2021-03-22 | 1 | 2021-04-05 18:52:14 | 999 |
| 56405 | PagePerformance_servery_time | 1 | 2021-03-22 | 2021-03-22 | 1 | 2021-04-05 18:52:14 | 999 |
| 56405 | PagePerformance_server_hits | 1 | 2021-03-22 | 2021-03-22 | 1 | 2021-04-05 18:52:14 | 999 |
| 56405 | PagePerformance_transfer_hits | 1 | 2021-03-22 | 2021-03-22 | 1 | 2021-04-05 18:52:14 | 999 |
| 56405 | PagePerformance_transfer_time | 1 | 2021-03-22 | 2021-03-22 | 1 | 2021-04-05 18:52:14 | 999 |
| 56405 | Referrers_distinctCampaigns | 1 | 2021-03-22 | 2021-03-22 | 1 | 2021-04-05 18:52:03 | 999 |
| 56405 | Referrers_distinctKeywords | 1 | 2021-03-22 | 2021-03-22 | 1 | 2021-04-05 18:52:03 | 999 |
| 56405 | Referrers_distinctSearchEngines | 1 | 2021-03-22 | 2021-03-22 | 1 | 2021-04-05 18:52:03 | 999 |
| 56405 | Referrers_distinctSocialNetworks | 1 | 2021-03-22 | 2021-03-22 | 1 | 2021-04-05 18:52:03 | 999 |
| 56405 | Referrers_distinctWebsites | 1 | 2021-03-22 | 2021-03-22 | 1 | 2021-04-05 18:52:03 | 999 |
| 56405 | Referrers_distinctWebsitesUrls | 1 | 2021-03-22 | 2021-03-22 | 1 | 2021-04-05 18:52:03 | 999 |
| 56405 | sum_visit_length | 1 | 2021-03-22 | 2021-03-22 | 1 | 2021-04-05 18:52:03 | 999 |
| 56405 | UserCountry_distinctCountries | 1 | 2021-03-22 | 2021-03-22 | 1 | 2021-04-05 18:52:14 | 1 |
Which doesn't include myPlugin
@diosmosis
commented
on April 8th 2021
@nelhefni the Goals plugin doesn't have any logic to rearchive in the past when it changes: https://github.com/matomo-org/matomo/blob/4.x-dev/plugins/Goals/API.php#L198-L232. The funnels plugin does, which is connected, but then I would expect to see Funnels reports in this table. Can you try editing a goal or creating a goal, then checking what gets added to archive_invalidations (then truncate the table or take note of the max idinvalidation and delete everything greater than it to avoid launching any archiving)?
Based on the archive contents above it seems like that archive should have a done value of DONE_OK (or 1). I guess it might be possible that because it gets set as value = 5, it thinks the archiving never happens and keeps trying... I'll have a look at the code when I can to see where it might be possible to set the value as 5 for an archive w/ name = 'done'.
@nelhefni
commented
on April 8th 2021
@diosmosis I did update the funnel attached to one of my goals.
ID of the funnel changed from 32 to 33.
After the update, nothing was added to the archive_invalidations table, but 2 rows were removed:
| idinvalidation | idarchive | name | idsite | date1 | date2 | period | ts_invalidated | ts_started | status | report |
| 1260359 | 457962 | done.Funnels | 1 | 2021-01-01 | 2021-12-31 | 4 | 2021-03-25 06:06:09 | 2021-03-26 20:04:28 | 0 | 32 |
| 1260644 | NULL | done8826e5af80dda71509c44b83f1853924.Funnels | 1 | 2021-01-01 | 2021-12-31 | 4 | 2021-03-25 06:06:49 | NULL | 0 | 32 |Then, when the cron job started, the invalidation process added a lot of new rows to the archive_invalidations table:
select * from matomo_archive_invalidations where name like '%unnel%' and report = 33;
| idinvalidation | idarchive | name | idsite | date1 | date2 | period | ts_invalidated | ts_started | status | report |
| 1429392 | 54863 | done.Funnels | 1 | 2021-03-01 | 2021-03-01 | 1 | 2021-04-08 05:28:02 | NULL | 0 | 33 |
| 1429393 | 54894 | done.Funnels | 1 | 2021-03-01 | 2021-03-07 | 2 | 2021-04-08 05:28:02 | NULL | 0 | 33 |
| 1429394 | 190726 | done.Funnels | 1 | 2021-03-01 | 2021-03-31 | 3 | 2021-04-08 05:28:02 | NULL | 0 | 33 |
| 1429395 | 54862 | done.Funnels | 1 | 2021-03-02 | 2021-03-02 | 1 | 2021-04-08 05:28:02 | NULL | 0 | 33 |
| 1429396 | 54858 | done.Funnels | 1 | 2021-03-03 | 2021-03-03 | 1 | 2021-04-08 05:28:02 | NULL | 0 | 33 |
| 1429397 | 54827 | done.Funnels | 1 | 2021-03-04 | 2021-03-04 | 1 | 2021-04-08 05:28:02 | NULL | 0 | 33 |
| 1429398 | 54825 | done.Funnels | 1 | 2021-03-05 | 2021-03-05 | 1 | 2021-04-08 05:28:02 | NULL | 0 | 33 |
| 1429399 | 54822 | done.Funnels | 1 | 2021-03-06 | 2021-03-06 | 1 | 2021-04-08 05:28:02 | NULL | 0 | 33 |
| 1429400 | 54447 | done.Funnels | 1 | 2021-03-07 | 2021-03-07 | 1 | 2021-04-08 05:28:02 | NULL | 0 | 33 |
| 1429401 | 54446 | done.Funnels | 1 | 2021-03-08 | 2021-03-08 | 1 | 2021-04-08 05:28:02 | NULL | 0 | 33 |
| 1429402 | 54472 | done.Funnels | 1 | 2021-03-08 | 2021-03-14 | 2 | 2021-04-08 05:28:02 | NULL | 0 | 33 |
| 1429403 | 54443 | done.Funnels | 1 | 2021-03-09 | 2021-03-09 | 1 | 2021-04-08 05:28:02 | NULL | 0 | 33 |
| 1429404 | 54411 | done.Funnels | 1 | 2021-03-10 | 2021-03-10 | 1 | 2021-04-08 05:28:02 | NULL | 0 | 33 |
| 1429405 | 54410 | done.Funnels | 1 | 2021-03-11 | 2021-03-11 | 1 | 2021-04-08 05:28:02 | NULL | 0 | 33 |
| 1429406 | 54407 | done.Funnels | 1 | 2021-03-12 | 2021-03-12 | 1 | 2021-04-08 05:28:02 | NULL | 0 | 33 |
| 1429407 | 54019 | done.Funnels | 1 | 2021-03-13 | 2021-03-13 | 1 | 2021-04-08 05:28:02 | NULL | 0 | 33 |
| 1429408 | 54015 | done.Funnels | 1 | 2021-03-14 | 2021-03-14 | 1 | 2021-04-08 05:28:02 | NULL | 0 | 33 |
| 1429409 | 53983 | done.Funnels | 1 | 2021-03-15 | 2021-03-15 | 1 | 2021-04-08 05:28:02 | NULL | 0 | 33 |
| 1429410 | 54017 | done.Funnels | 1 | 2021-03-15 | 2021-03-21 | 2 | 2021-04-08 05:28:02 | NULL | 0 | 33 |
| 1429411 | 53982 | done.Funnels | 1 | 2021-03-16 | 2021-03-16 | 1 | 2021-04-08 05:28:02 | NULL | 0 | 33 |
| 1429412 | 53978 | done.Funnels | 1 | 2021-03-17 | 2021-03-17 | 1 | 2021-04-08 05:28:02 | NULL | 0 | 33 |
| 1429413 | 53947 | done.Funnels | 1 | 2021-03-18 | 2021-03-18 | 1 | 2021-04-08 05:28:02 | NULL | 0 | 33 |
| 1429414 | 53946 | done.Funnels | 1 | 2021-03-19 | 2021-03-19 | 1 | 2021-04-08 05:28:02 | NULL | 0 | 33 |
| 1429415 | 53942 | done.Funnels | 1 | 2021-03-20 | 2021-03-20 | 1 | 2021-04-08 05:28:02 | NULL | 0 | 33 |
| 1429416 | 53894 | done.Funnels | 1 | 2021-03-21 | 2021-03-21 | 1 | 2021-04-08 05:28:02 | NULL | 0 | 33 |
| 1429417 | 53890 | done.Funnels | 1 | 2021-03-22 | 2021-03-22 | 1 | 2021-04-08 05:28:02 | NULL | 0 | 33 |
| 1429418 | 67174 | done.Funnels | 1 | 2021-03-22 | 2021-03-28 | 2 | 2021-04-08 05:28:02 | NULL | 0 | 33 |
| 1429419 | 53889 | done.Funnels | 1 | 2021-03-23 | 2021-03-23 | 1 | 2021-04-08 05:28:02 | NULL | 0 | 33 |
| 1429420 | NULL | done.Funnels | 1 | 2021-03-24 | 2021-03-24 | 1 | 2021-04-08 05:28:02 | NULL | 0 | 33 |
| 1429421 | NULL | done.Funnels | 1 | 2021-03-25 | 2021-03-25 | 1 | 2021-04-08 05:28:02 | NULL | 0 | 33 |
| 1429422 | NULL | done.Funnels | 1 | 2021-03-26 | 2021-03-26 | 1 | 2021-04-08 05:28:02 | NULL | 0 | 33 |
| 1429423 | NULL | done.Funnels | 1 | 2021-03-27 | 2021-03-27 | 1 | 2021-04-08 05:28:02 | NULL | 0 | 33 |
| 1429424 | NULL | done.Funnels | 1 | 2021-03-28 | 2021-03-28 | 1 | 2021-04-08 05:28:02 | NULL | 0 | 33 |
| 1429425 | NULL | done.Funnels | 1 | 2021-03-29 | 2021-03-29 | 1 | 2021-04-08 05:28:02 | NULL | 0 | 33 |
| 1429426 | NULL | done.Funnels | 1 | 2021-03-29 | 2021-04-04 | 2 | 2021-04-08 05:28:02 | NULL | 0 | 33 |
| 1429427 | NULL | done.Funnels | 1 | 2021-03-30 | 2021-03-30 | 1 | 2021-04-08 05:28:02 | NULL | 0 | 33 |
| 1429428 | NULL | done.Funnels | 1 | 2021-03-31 | 2021-03-31 | 1 | 2021-04-08 05:28:02 | NULL | 0 | 33 |
| 1429455 | 457962 | done.Funnels | 1 | 2021-01-01 | 2021-12-31 | 4 | 2021-04-08 05:28:02 | NULL | 0 | 33 |
| 1429456 | NULL | done.Funnels | 1 | 2021-04-01 | 2021-04-01 | 1 | 2021-04-08 05:28:02 | NULL | 0 | 33 |
| 1429457 | NULL | done.Funnels | 1 | 2021-04-01 | 2021-04-30 | 3 | 2021-04-08 05:28:02 | NULL | 0 | 33 |
| 1429458 | NULL | done.Funnels | 1 | 2021-04-02 | 2021-04-02 | 1 | 2021-04-08 05:28:02 | NULL | 0 | 33 |
| 1429459 | NULL | done.Funnels | 1 | 2021-04-03 | 2021-04-03 | 1 | 2021-04-08 05:28:02 | NULL | 0 | 33 |
| 1429460 | NULL | done.Funnels | 1 | 2021-04-04 | 2021-04-04 | 1 | 2021-04-08 05:28:02 | NULL | 0 | 33 |
| 1429461 | NULL | done.Funnels | 1 | 2021-04-05 | 2021-04-05 | 1 | 2021-04-08 05:28:02 | NULL | 0 | 33 |
| 1429462 | NULL | done.Funnels | 1 | 2021-04-05 | 2021-04-11 | 2 | 2021-04-08 05:28:02 | NULL | 0 | 33 |
| 1429463 | NULL | done.Funnels | 1 | 2021-04-06 | 2021-04-06 | 1 | 2021-04-08 05:28:02 | NULL | 0 | 33 |
| 1429471 | NULL | done252e705405118c93ee8ffcda1daff4ab.Funnels | 1 | 2021-03-01 | 2021-03-01 | 1 | 2021-04-08 05:28:07 | NULL | 0 | 33 |
| 1429472 | NULL | done252e705405118c93ee8ffcda1daff4ab.Funnels | 1 | 2021-03-01 | 2021-03-07 | 2 | 2021-04-08 05:28:07 | NULL | 0 | 33 |
| 1429473 | NULL | done252e705405118c93ee8ffcda1daff4ab.Funnels | 1 | 2021-03-01 | 2021-03-31 | 3 | 2021-04-08 05:28:07 | NULL | 0 | 33 |
| 1429474 | NULL | done252e705405118c93ee8ffcda1daff4ab.Funnels | 1 | 2021-03-02 | 2021-03-02 | 1 | 2021-04-08 05:28:07 | NULL | 0 | 33 |
| 1429475 | NULL | done252e705405118c93ee8ffcda1daff4ab.Funnels | 1 | 2021-03-03 | 2021-03-03 | 1 | 2021-04-08 05:28:07 | NULL | 0 | 33 |
| 1429476 | NULL | done252e705405118c93ee8ffcda1daff4ab.Funnels | 1 | 2021-03-04 | 2021-03-04 | 1 | 2021-04-08 05:28:07 | NULL | 0 | 33 |
| 1429477 | NULL | done252e705405118c93ee8ffcda1daff4ab.Funnels | 1 | 2021-03-05 | 2021-03-05 | 1 | 2021-04-08 05:28:07 | NULL | 0 | 33 |
| 1429478 | NULL | done252e705405118c93ee8ffcda1daff4ab.Funnels | 1 | 2021-03-06 | 2021-03-06 | 1 | 2021-04-08 05:28:07 | NULL | 0 | 33 |
| 1429479 | NULL | done252e705405118c93ee8ffcda1daff4ab.Funnels | 1 | 2021-03-07 | 2021-03-07 | 1 | 2021-04-08 05:28:07 | NULL | 0 | 33 |
| 1429480 | NULL | done252e705405118c93ee8ffcda1daff4ab.Funnels | 1 | 2021-03-08 | 2021-03-08 | 1 | 2021-04-08 05:28:07 | NULL | 0 | 33 |
| 1429481 | NULL | done252e705405118c93ee8ffcda1daff4ab.Funnels | 1 | 2021-03-08 | 2021-03-14 | 2 | 2021-04-08 05:28:07 | NULL | 0 | 33 |
| 1429482 | NULL | done252e705405118c93ee8ffcda1daff4ab.Funnels | 1 | 2021-03-09 | 2021-03-09 | 1 | 2021-04-08 05:28:07 | NULL | 0 | 33 |
| 1429483 | NULL | done252e705405118c93ee8ffcda1daff4ab.Funnels | 1 | 2021-03-10 | 2021-03-10 | 1 | 2021-04-08 05:28:07 | NULL | 0 | 33 |
| 1429484 | NULL | done252e705405118c93ee8ffcda1daff4ab.Funnels | 1 | 2021-03-11 | 2021-03-11 | 1 | 2021-04-08 05:28:07 | NULL | 0 | 33 |
| 1429485 | NULL | done252e705405118c93ee8ffcda1daff4ab.Funnels | 1 | 2021-03-12 | 2021-03-12 | 1 | 2021-04-08 05:28:07 | NULL | 0 | 33 |
| 1429486 | NULL | done252e705405118c93ee8ffcda1daff4ab.Funnels | 1 | 2021-03-13 | 2021-03-13 | 1 | 2021-04-08 05:28:07 | NULL | 0 | 33 |
| 1429487 | NULL | done252e705405118c93ee8ffcda1daff4ab.Funnels | 1 | 2021-03-14 | 2021-03-14 | 1 | 2021-04-08 05:28:07 | NULL | 0 | 33 |
| 1429488 | NULL | done252e705405118c93ee8ffcda1daff4ab.Funnels | 1 | 2021-03-15 | 2021-03-15 | 1 | 2021-04-08 05:28:07 | NULL | 0 | 33 |
| 1429489 | NULL | done252e705405118c93ee8ffcda1daff4ab.Funnels | 1 | 2021-03-15 | 2021-03-21 | 2 | 2021-04-08 05:28:07 | NULL | 0 | 33 |
| 1429490 | NULL | done252e705405118c93ee8ffcda1daff4ab.Funnels | 1 | 2021-03-16 | 2021-03-16 | 1 | 2021-04-08 05:28:07 | NULL | 0 | 33 |
| 1429491 | NULL | done252e705405118c93ee8ffcda1daff4ab.Funnels | 1 | 2021-03-17 | 2021-03-17 | 1 | 2021-04-08 05:28:07 | NULL | 0 | 33 |
| 1429492 | NULL | done252e705405118c93ee8ffcda1daff4ab.Funnels | 1 | 2021-03-18 | 2021-03-18 | 1 | 2021-04-08 05:28:07 | NULL | 0 | 33 |
| 1429493 | NULL | done252e705405118c93ee8ffcda1daff4ab.Funnels | 1 | 2021-03-19 | 2021-03-19 | 1 | 2021-04-08 05:28:07 | NULL | 0 | 33 |
| 1429494 | NULL | done252e705405118c93ee8ffcda1daff4ab.Funnels | 1 | 2021-03-20 | 2021-03-20 | 1 | 2021-04-08 05:28:07 | NULL | 0 | 33 |
| 1429495 | NULL | done252e705405118c93ee8ffcda1daff4ab.Funnels | 1 | 2021-03-21 | 2021-03-21 | 1 | 2021-04-08 05:28:07 | NULL | 0 | 33 |
| 1429496 | NULL | done252e705405118c93ee8ffcda1daff4ab.Funnels | 1 | 2021-03-22 | 2021-03-22 | 1 | 2021-04-08 05:28:07 | NULL | 0 | 33 |
| 1429497 | NULL | done252e705405118c93ee8ffcda1daff4ab.Funnels | 1 | 2021-03-22 | 2021-03-28 | 2 | 2021-04-08 05:28:07 | NULL | 0 | 33 |
| 1429498 | NULL | done252e705405118c93ee8ffcda1daff4ab.Funnels | 1 | 2021-03-23 | 2021-03-23 | 1 | 2021-04-08 05:28:07 | NULL | 0 | 33 |
| 1429499 | NULL | done252e705405118c93ee8ffcda1daff4ab.Funnels | 1 | 2021-03-24 | 2021-03-24 | 1 | 2021-04-08 05:28:07 | NULL | 0 | 33 |
| 1429500 | NULL | done252e705405118c93ee8ffcda1daff4ab.Funnels | 1 | 2021-03-25 | 2021-03-25 | 1 | 2021-04-08 05:28:07 | NULL | 0 | 33 |
| 1429501 | NULL | done252e705405118c93ee8ffcda1daff4ab.Funnels | 1 | 2021-03-26 | 2021-03-26 | 1 | 2021-04-08 05:28:07 | NULL | 0 | 33 |
| 1429502 | NULL | done252e705405118c93ee8ffcda1daff4ab.Funnels | 1 | 2021-03-27 | 2021-03-27 | 1 | 2021-04-08 05:28:07 | NULL | 0 | 33 |
| 1429503 | NULL | done252e705405118c93ee8ffcda1daff4ab.Funnels | 1 | 2021-03-28 | 2021-03-28 | 1 | 2021-04-08 05:28:07 | NULL | 0 | 33 |
| 1429504 | NULL | done252e705405118c93ee8ffcda1daff4ab.Funnels | 1 | 2021-03-29 | 2021-03-29 | 1 | 2021-04-08 05:28:07 | NULL | 0 | 33 |
| 1429505 | NULL | done252e705405118c93ee8ffcda1daff4ab.Funnels | 1 | 2021-03-29 | 2021-04-04 | 2 | 2021-04-08 05:28:07 | NULL | 0 | 33 |
| 1429506 | NULL | done252e705405118c93ee8ffcda1daff4ab.Funnels | 1 | 2021-03-30 | 2021-03-30 | 1 | 2021-04-08 05:28:07 | NULL | 0 | 33 |
| 1429507 | NULL | done252e705405118c93ee8ffcda1daff4ab.Funnels | 1 | 2021-03-31 | 2021-03-31 | 1 | 2021-04-08 05:28:07 | NULL | 0 | 33 |
| 1429534 | NULL | done252e705405118c93ee8ffcda1daff4ab.Funnels | 1 | 2021-01-01 | 2021-12-31 | 4 | 2021-04-08 05:28:07 | NULL | 0 | 33 |
| 1429535 | NULL | done252e705405118c93ee8ffcda1daff4ab.Funnels | 1 | 2021-04-01 | 2021-04-01 | 1 | 2021-04-08 05:28:07 | NULL | 0 | 33 |
| 1429536 | NULL | done252e705405118c93ee8ffcda1daff4ab.Funnels | 1 | 2021-04-01 | 2021-04-30 | 3 | 2021-04-08 05:28:07 | NULL | 0 | 33 |
| 1429537 | NULL | done252e705405118c93ee8ffcda1daff4ab.Funnels | 1 | 2021-04-02 | 2021-04-02 | 1 | 2021-04-08 05:28:07 | NULL | 0 | 33 |
| 1429538 | NULL | done252e705405118c93ee8ffcda1daff4ab.Funnels | 1 | 2021-04-03 | 2021-04-03 | 1 | 2021-04-08 05:28:07 | NULL | 0 | 33 |
| 1429539 | NULL | done252e705405118c93ee8ffcda1daff4ab.Funnels | 1 | 2021-04-04 | 2021-04-04 | 1 | 2021-04-08 05:28:07 | NULL | 0 | 33 |
| 1429540 | NULL | done252e705405118c93ee8ffcda1daff4ab.Funnels | 1 | 2021-04-05 | 2021-04-05 | 1 | 2021-04-08 05:28:07 | NULL | 0 | 33 |
| 1429541 | NULL | done252e705405118c93ee8ffcda1daff4ab.Funnels | 1 | 2021-04-05 | 2021-04-11 | 2 | 2021-04-08 05:28:07 | NULL | 0 | 33 |
| 1429542 | NULL | done252e705405118c93ee8ffcda1daff4ab.Funnels | 1 | 2021-04-06 | 2021-04-06 | 1 | 2021-04-08 05:28:07 | NULL | 0 | 33 |
| 1429550 | 54888 | done0454a4912949ddaecc8bf8e707b93f17.Funnels | 1 | 2021-03-01 | 2021-03-01 | 1 | 2021-04-08 05:28:12 | NULL | 0 | 33 |
| 1429551 | 55095 | done0454a4912949ddaecc8bf8e707b93f17.Funnels | 1 | 2021-03-01 | 2021-03-07 | 2 | 2021-04-08 05:28:12 | NULL | 0 | 33 |
| 1429552 | 96258 | done0454a4912949ddaecc8bf8e707b93f17.Funnels | 1 | 2021-03-01 | 2021-03-31 | 3 | 2021-04-08 05:28:12 | NULL | 0 | 33 |
| 1429553 | 54879 | done0454a4912949ddaecc8bf8e707b93f17.Funnels | 1 | 2021-03-02 | 2021-03-02 | 1 | 2021-04-08 05:28:12 | NULL | 0 | 33 |
| 1429554 | 54868 | done0454a4912949ddaecc8bf8e707b93f17.Funnels | 1 | 2021-03-03 | 2021-03-03 | 1 | 2021-04-08 05:28:12 | NULL | 0 | 33 |
| 1429555 | 54853 | done0454a4912949ddaecc8bf8e707b93f17.Funnels | 1 | 2021-03-04 | 2021-03-04 | 1 | 2021-04-08 05:28:12 | NULL | 0 | 33 |
| 1429556 | 54844 | done0454a4912949ddaecc8bf8e707b93f17.Funnels | 1 | 2021-03-05 | 2021-03-05 | 1 | 2021-04-08 05:28:12 | NULL | 0 | 33 |
| 1429557 | 54831 | done0454a4912949ddaecc8bf8e707b93f17.Funnels | 1 | 2021-03-06 | 2021-03-06 | 1 | 2021-04-08 05:28:12 | NULL | 0 | 33 |
| 1429558 | 54672 | done0454a4912949ddaecc8bf8e707b93f17.Funnels | 1 | 2021-03-07 | 2021-03-07 | 1 | 2021-04-08 05:28:12 | NULL | 0 | 33 |
| 1429559 | 54464 | done0454a4912949ddaecc8bf8e707b93f17.Funnels | 1 | 2021-03-08 | 2021-03-08 | 1 | 2021-04-08 05:28:12 | NULL | 0 | 33 |
| 1429560 | 54677 | done0454a4912949ddaecc8bf8e707b93f17.Funnels | 1 | 2021-03-08 | 2021-03-14 | 2 | 2021-04-08 05:28:12 | NULL | 0 | 33 |
| 1429561 | 54452 | done0454a4912949ddaecc8bf8e707b93f17.Funnels | 1 | 2021-03-09 | 2021-03-09 | 1 | 2021-04-08 05:28:12 | NULL | 0 | 33 |
| 1429562 | 54437 | done0454a4912949ddaecc8bf8e707b93f17.Funnels | 1 | 2021-03-10 | 2021-03-10 | 1 | 2021-04-08 05:28:12 | NULL | 0 | 33 |
| 1429563 | 54428 | done0454a4912949ddaecc8bf8e707b93f17.Funnels | 1 | 2021-03-11 | 2021-03-11 | 1 | 2021-04-08 05:28:12 | NULL | 0 | 33 |
| 1429564 | 54416 | done0454a4912949ddaecc8bf8e707b93f17.Funnels | 1 | 2021-03-12 | 2021-03-12 | 1 | 2021-04-08 05:28:12 | NULL | 0 | 33 |
| 1429565 | 54401 | done0454a4912949ddaecc8bf8e707b93f17.Funnels | 1 | 2021-03-13 | 2021-03-13 | 1 | 2021-04-08 05:28:12 | NULL | 0 | 33 |
| 1429566 | 54390 | done0454a4912949ddaecc8bf8e707b93f17.Funnels | 1 | 2021-03-14 | 2021-03-14 | 1 | 2021-04-08 05:28:12 | NULL | 0 | 33 |
| 1429567 | 54008 | done0454a4912949ddaecc8bf8e707b93f17.Funnels | 1 | 2021-03-15 | 2021-03-15 | 1 | 2021-04-08 05:28:12 | NULL | 0 | 33 |
| 1429568 | 54223 | done0454a4912949ddaecc8bf8e707b93f17.Funnels | 1 | 2021-03-15 | 2021-03-21 | 2 | 2021-04-08 05:28:12 | NULL | 0 | 33 |
| 1429569 | 54000 | done0454a4912949ddaecc8bf8e707b93f17.Funnels | 1 | 2021-03-16 | 2021-03-16 | 1 | 2021-04-08 05:28:12 | NULL | 0 | 33 |
| 1429570 | 53988 | done0454a4912949ddaecc8bf8e707b93f17.Funnels | 1 | 2021-03-17 | 2021-03-17 | 1 | 2021-04-08 05:28:12 | NULL | 0 | 33 |
| 1429571 | 53973 | done0454a4912949ddaecc8bf8e707b93f17.Funnels | 1 | 2021-03-18 | 2021-03-18 | 1 | 2021-04-08 05:28:12 | NULL | 0 | 33 |
| 1429572 | 53964 | done0454a4912949ddaecc8bf8e707b93f17.Funnels | 1 | 2021-03-19 | 2021-03-19 | 1 | 2021-04-08 05:28:12 | NULL | 0 | 33 |
| 1429573 | 53952 | done0454a4912949ddaecc8bf8e707b93f17.Funnels | 1 | 2021-03-20 | 2021-03-20 | 1 | 2021-04-08 05:28:12 | NULL | 0 | 33 |
| 1429574 | 53936 | done0454a4912949ddaecc8bf8e707b93f17.Funnels | 1 | 2021-03-21 | 2021-03-21 | 1 | 2021-04-08 05:28:12 | NULL | 0 | 33 |
| 1429575 | 53907 | done0454a4912949ddaecc8bf8e707b93f17.Funnels | 1 | 2021-03-22 | 2021-03-22 | 1 | 2021-04-08 05:28:12 | NULL | 0 | 33 |
| 1429576 | 67373 | done0454a4912949ddaecc8bf8e707b93f17.Funnels | 1 | 2021-03-22 | 2021-03-28 | 2 | 2021-04-08 05:28:12 | NULL | 0 | 33 |
| 1429577 | 53898 | done0454a4912949ddaecc8bf8e707b93f17.Funnels | 1 | 2021-03-23 | 2021-03-23 | 1 | 2021-04-08 05:28:12 | NULL | 0 | 33 |
| 1429578 | NULL | done0454a4912949ddaecc8bf8e707b93f17.Funnels | 1 | 2021-03-24 | 2021-03-24 | 1 | 2021-04-08 05:28:12 | NULL | 0 | 33 |
| 1429579 | NULL | done0454a4912949ddaecc8bf8e707b93f17.Funnels | 1 | 2021-03-25 | 2021-03-25 | 1 | 2021-04-08 05:28:12 | NULL | 0 | 33 |
| 1429580 | NULL | done0454a4912949ddaecc8bf8e707b93f17.Funnels | 1 | 2021-03-26 | 2021-03-26 | 1 | 2021-04-08 05:28:12 | NULL | 0 | 33 |
| 1429581 | NULL | done0454a4912949ddaecc8bf8e707b93f17.Funnels | 1 | 2021-03-27 | 2021-03-27 | 1 | 2021-04-08 05:28:12 | NULL | 0 | 33 |
| 1429582 | NULL | done0454a4912949ddaecc8bf8e707b93f17.Funnels | 1 | 2021-03-28 | 2021-03-28 | 1 | 2021-04-08 05:28:12 | NULL | 0 | 33 |
| 1429583 | NULL | done0454a4912949ddaecc8bf8e707b93f17.Funnels | 1 | 2021-03-29 | 2021-03-29 | 1 | 2021-04-08 05:28:12 | NULL | 0 | 33 |
| 1429584 | NULL | done0454a4912949ddaecc8bf8e707b93f17.Funnels | 1 | 2021-03-29 | 2021-04-04 | 2 | 2021-04-08 05:28:12 | NULL | 0 | 33 |
| 1429585 | NULL | done0454a4912949ddaecc8bf8e707b93f17.Funnels | 1 | 2021-03-30 | 2021-03-30 | 1 | 2021-04-08 05:28:12 | NULL | 0 | 33 |
| 1429586 | NULL | done0454a4912949ddaecc8bf8e707b93f17.Funnels | 1 | 2021-03-31 | 2021-03-31 | 1 | 2021-04-08 05:28:12 | NULL | 0 | 33 |
| 1429613 | 891194 | done0454a4912949ddaecc8bf8e707b93f17.Funnels | 1 | 2021-01-01 | 2021-12-31 | 4 | 2021-04-08 05:28:12 | NULL | 0 | 33 |
| 1429614 | NULL | done0454a4912949ddaecc8bf8e707b93f17.Funnels | 1 | 2021-04-01 | 2021-04-01 | 1 | 2021-04-08 05:28:12 | NULL | 0 | 33 |
| 1429615 | NULL | done0454a4912949ddaecc8bf8e707b93f17.Funnels | 1 | 2021-04-01 | 2021-04-30 | 3 | 2021-04-08 05:28:12 | NULL | 0 | 33 |
| 1429616 | NULL | done0454a4912949ddaecc8bf8e707b93f17.Funnels | 1 | 2021-04-02 | 2021-04-02 | 1 | 2021-04-08 05:28:12 | NULL | 0 | 33 |
| 1429617 | NULL | done0454a4912949ddaecc8bf8e707b93f17.Funnels | 1 | 2021-04-03 | 2021-04-03 | 1 | 2021-04-08 05:28:12 | NULL | 0 | 33 |
| 1429618 | NULL | done0454a4912949ddaecc8bf8e707b93f17.Funnels | 1 | 2021-04-04 | 2021-04-04 | 1 | 2021-04-08 05:28:12 | NULL | 0 | 33 |
| 1429619 | NULL | done0454a4912949ddaecc8bf8e707b93f17.Funnels | 1 | 2021-04-05 | 2021-04-05 | 1 | 2021-04-08 05:28:12 | NULL | 0 | 33 |
| 1429620 | NULL | done0454a4912949ddaecc8bf8e707b93f17.Funnels | 1 | 2021-04-05 | 2021-04-11 | 2 | 2021-04-08 05:28:12 | NULL | 0 | 33 |
| 1429621 | NULL | done0454a4912949ddaecc8bf8e707b93f17.Funnels | 1 | 2021-04-06 | 2021-04-06 | 1 | 2021-04-08 05:28:12 | NULL | 0 | 33 |
| 1429629 | 54887 | done8826e5af80dda71509c44b83f1853924.Funnels | 1 | 2021-03-01 | 2021-03-01 | 1 | 2021-04-08 05:28:17 | NULL | 0 | 33 |
| 1429630 | 55096 | done8826e5af80dda71509c44b83f1853924.Funnels | 1 | 2021-03-01 | 2021-03-07 | 2 | 2021-04-08 05:28:17 | NULL | 0 | 33 |
| 1429631 | 96257 | done8826e5af80dda71509c44b83f1853924.Funnels | 1 | 2021-03-01 | 2021-03-31 | 3 | 2021-04-08 05:28:17 | NULL | 0 | 33 |
| 1429632 | 54878 | done8826e5af80dda71509c44b83f1853924.Funnels | 1 | 2021-03-02 | 2021-03-02 | 1 | 2021-04-08 05:28:17 | NULL | 0 | 33 |
| 1429633 | 54867 | done8826e5af80dda71509c44b83f1853924.Funnels | 1 | 2021-03-03 | 2021-03-03 | 1 | 2021-04-08 05:28:17 | NULL | 0 | 33 |
| 1429634 | 54852 | done8826e5af80dda71509c44b83f1853924.Funnels | 1 | 2021-03-04 | 2021-03-04 | 1 | 2021-04-08 05:28:17 | NULL | 0 | 33 |
| 1429635 | 54843 | done8826e5af80dda71509c44b83f1853924.Funnels | 1 | 2021-03-05 | 2021-03-05 | 1 | 2021-04-08 05:28:17 | NULL | 0 | 33 |
| 1429636 | 54832 | done8826e5af80dda71509c44b83f1853924.Funnels | 1 | 2021-03-06 | 2021-03-06 | 1 | 2021-04-08 05:28:17 | NULL | 0 | 33 |
| 1429637 | 54475 | done8826e5af80dda71509c44b83f1853924.Funnels | 1 | 2021-03-07 | 2021-03-07 | 1 | 2021-04-08 05:28:17 | NULL | 0 | 33 |
| 1429638 | 54463 | done8826e5af80dda71509c44b83f1853924.Funnels | 1 | 2021-03-08 | 2021-03-08 | 1 | 2021-04-08 05:28:17 | NULL | 0 | 33 |
| 1429639 | 54678 | done8826e5af80dda71509c44b83f1853924.Funnels | 1 | 2021-03-08 | 2021-03-14 | 2 | 2021-04-08 05:28:17 | NULL | 0 | 33 |
| 1429640 | 54451 | done8826e5af80dda71509c44b83f1853924.Funnels | 1 | 2021-03-09 | 2021-03-09 | 1 | 2021-04-08 05:28:17 | NULL | 0 | 33 |
| 1429641 | 54436 | done8826e5af80dda71509c44b83f1853924.Funnels | 1 | 2021-03-10 | 2021-03-10 | 1 | 2021-04-08 05:28:17 | NULL | 0 | 33 |
| 1429642 | 54427 | done8826e5af80dda71509c44b83f1853924.Funnels | 1 | 2021-03-11 | 2021-03-11 | 1 | 2021-04-08 05:28:17 | NULL | 0 | 33 |
| 1429643 | 54415 | done8826e5af80dda71509c44b83f1853924.Funnels | 1 | 2021-03-12 | 2021-03-12 | 1 | 2021-04-08 05:28:17 | NULL | 0 | 33 |
| 1429644 | 54400 | done8826e5af80dda71509c44b83f1853924.Funnels | 1 | 2021-03-13 | 2021-03-13 | 1 | 2021-04-08 05:28:17 | NULL | 0 | 33 |
| 1429645 | 54389 | done8826e5af80dda71509c44b83f1853924.Funnels | 1 | 2021-03-14 | 2021-03-14 | 1 | 2021-04-08 05:28:17 | NULL | 0 | 33 |
| 1429646 | 54009 | done8826e5af80dda71509c44b83f1853924.Funnels | 1 | 2021-03-15 | 2021-03-15 | 1 | 2021-04-08 05:28:17 | NULL | 0 | 33 |
| 1429647 | 54222 | done8826e5af80dda71509c44b83f1853924.Funnels | 1 | 2021-03-15 | 2021-03-21 | 2 | 2021-04-08 05:28:17 | NULL | 0 | 33 |
| 1429648 | 53999 | done8826e5af80dda71509c44b83f1853924.Funnels | 1 | 2021-03-16 | 2021-03-16 | 1 | 2021-04-08 05:28:17 | NULL | 0 | 33 |
| 1429649 | 53987 | done8826e5af80dda71509c44b83f1853924.Funnels | 1 | 2021-03-17 | 2021-03-17 | 1 | 2021-04-08 05:28:17 | NULL | 0 | 33 |
| 1429650 | 53972 | done8826e5af80dda71509c44b83f1853924.Funnels | 1 | 2021-03-18 | 2021-03-18 | 1 | 2021-04-08 05:28:17 | NULL | 0 | 33 |
| 1429651 | 53963 | done8826e5af80dda71509c44b83f1853924.Funnels | 1 | 2021-03-19 | 2021-03-19 | 1 | 2021-04-08 05:28:17 | NULL | 0 | 33 |
| 1429652 | 53951 | done8826e5af80dda71509c44b83f1853924.Funnels | 1 | 2021-03-20 | 2021-03-20 | 1 | 2021-04-08 05:28:17 | NULL | 0 | 33 |
| 1429653 | 53937 | done8826e5af80dda71509c44b83f1853924.Funnels | 1 | 2021-03-21 | 2021-03-21 | 1 | 2021-04-08 05:28:17 | NULL | 0 | 33 |
| 1429654 | 53908 | done8826e5af80dda71509c44b83f1853924.Funnels | 1 | 2021-03-22 | 2021-03-22 | 1 | 2021-04-08 05:28:17 | NULL | 0 | 33 |
| 1429655 | 67372 | done8826e5af80dda71509c44b83f1853924.Funnels | 1 | 2021-03-22 | 2021-03-28 | 2 | 2021-04-08 05:28:17 | NULL | 0 | 33 |
| 1429656 | 53899 | done8826e5af80dda71509c44b83f1853924.Funnels | 1 | 2021-03-23 | 2021-03-23 | 1 | 2021-04-08 05:28:17 | NULL | 0 | 33 |
| 1429657 | NULL | done8826e5af80dda71509c44b83f1853924.Funnels | 1 | 2021-03-24 | 2021-03-24 | 1 | 2021-04-08 05:28:17 | NULL | 0 | 33 |
| 1429658 | NULL | done8826e5af80dda71509c44b83f1853924.Funnels | 1 | 2021-03-25 | 2021-03-25 | 1 | 2021-04-08 05:28:17 | NULL | 0 | 33 |
| 1429659 | NULL | done8826e5af80dda71509c44b83f1853924.Funnels | 1 | 2021-03-26 | 2021-03-26 | 1 | 2021-04-08 05:28:17 | NULL | 0 | 33 |
| 1429660 | NULL | done8826e5af80dda71509c44b83f1853924.Funnels | 1 | 2021-03-27 | 2021-03-27 | 1 | 2021-04-08 05:28:17 | NULL | 0 | 33 |
| 1429661 | NULL | done8826e5af80dda71509c44b83f1853924.Funnels | 1 | 2021-03-28 | 2021-03-28 | 1 | 2021-04-08 05:28:17 | NULL | 0 | 33 |
| 1429662 | NULL | done8826e5af80dda71509c44b83f1853924.Funnels | 1 | 2021-03-29 | 2021-03-29 | 1 | 2021-04-08 05:28:17 | NULL | 0 | 33 |
| 1429663 | NULL | done8826e5af80dda71509c44b83f1853924.Funnels | 1 | 2021-03-29 | 2021-04-04 | 2 | 2021-04-08 05:28:17 | NULL | 0 | 33 |
| 1429664 | NULL | done8826e5af80dda71509c44b83f1853924.Funnels | 1 | 2021-03-30 | 2021-03-30 | 1 | 2021-04-08 05:28:17 | NULL | 0 | 33 |
| 1429665 | NULL | done8826e5af80dda71509c44b83f1853924.Funnels | 1 | 2021-03-31 | 2021-03-31 | 1 | 2021-04-08 05:28:17 | NULL | 0 | 33 |
| 1429692 | NULL | done8826e5af80dda71509c44b83f1853924.Funnels | 1 | 2021-01-01 | 2021-12-31 | 4 | 2021-04-08 05:28:18 | NULL | 0 | 33 |
| 1429693 | NULL | done8826e5af80dda71509c44b83f1853924.Funnels | 1 | 2021-04-01 | 2021-04-01 | 1 | 2021-04-08 05:28:18 | NULL | 0 | 33 |
| 1429694 | NULL | done8826e5af80dda71509c44b83f1853924.Funnels | 1 | 2021-04-01 | 2021-04-30 | 3 | 2021-04-08 05:28:18 | NULL | 0 | 33 |
| 1429695 | NULL | done8826e5af80dda71509c44b83f1853924.Funnels | 1 | 2021-04-02 | 2021-04-02 | 1 | 2021-04-08 05:28:18 | NULL | 0 | 33 |
| 1429696 | NULL | done8826e5af80dda71509c44b83f1853924.Funnels | 1 | 2021-04-03 | 2021-04-03 | 1 | 2021-04-08 05:28:18 | NULL | 0 | 33 |
| 1429697 | NULL | done8826e5af80dda71509c44b83f1853924.Funnels | 1 | 2021-04-04 | 2021-04-04 | 1 | 2021-04-08 05:28:18 | NULL | 0 | 33 |
| 1429698 | NULL | done8826e5af80dda71509c44b83f1853924.Funnels | 1 | 2021-04-05 | 2021-04-05 | 1 | 2021-04-08 05:28:18 | NULL | 0 | 33 |
| 1429699 | NULL | done8826e5af80dda71509c44b83f1853924.Funnels | 1 | 2021-04-05 | 2021-04-11 | 2 | 2021-04-08 05:28:18 | NULL | 0 | 33 |
| 1429700 | NULL | done8826e5af80dda71509c44b83f1853924.Funnels | 1 | 2021-04-06 | 2021-04-06 | 1 | 2021-04-08 05:28:18 | NULL | 0 | 33 |
This seems to make sense, as we have 3 segments, and go back to March 1st because of the following setting:rearchive_reports_in_past_last_n_months = last1
Not too sure why there are a lot of idarchive set to NULL though.
Moreover, If I look for the corresponding idarchive in matomo_archive_numeric2021**, I find only one r0w:
select * from matomo_archive_numeric_2021_03 where idarchive = 54863;
| idarchive | name | idsite | date1 | date2 | period | ts_archived | value |
| 54863 | done.Funnels | 1 | 2021-03-01 | 2021-03-01 | 1 | 2021-03-25 06:35:35 | 4 |I'm not sure to understand the purpose of this done.Funnels, could you please explain the concept behind it?
Also, I deleted manually all the rows for period 3, as I want to avoid the issue described here: https://github.com/matomo-org/matomo/issues/17434,
delete from matomo_archive_invalidations where name like '%unnel%' and period = 3 and report = 33;
| idinvalidation | idarchive | name | idsite | date1 | date2 | period | ts_invalidated | ts_started | status | report |
| 1429394 | 190726 | done.Funnels | 1 | 2021-03-01 | 2021-03-31 | 3 | 2021-04-08 05:28:02 | NULL | 0 | 33 |
| 1429457 | NULL | done.Funnels | 1 | 2021-04-01 | 2021-04-30 | 3 | 2021-04-08 05:28:02 | NULL | 0 | 33 |
| 1429473 | NULL | done252e705405118c93ee8ffcda1daff4ab.Funnels | 1 | 2021-03-01 | 2021-03-31 | 3 | 2021-04-08 05:28:07 | NULL | 0 | 33 |
| 1429536 | NULL | done252e705405118c93ee8ffcda1daff4ab.Funnels | 1 | 2021-04-01 | 2021-04-30 | 3 | 2021-04-08 05:28:07 | NULL | 0 | 33 |
| 1429552 | 96258 | done0454a4912949ddaecc8bf8e707b93f17.Funnels | 1 | 2021-03-01 | 2021-03-31 | 3 | 2021-04-08 05:28:12 | NULL | 0 | 33 |
| 1429615 | NULL | done0454a4912949ddaecc8bf8e707b93f17.Funnels | 1 | 2021-04-01 | 2021-04-30 | 3 | 2021-04-08 05:28:12 | NULL | 0 | 33 |
| 1429631 | 96257 | done8826e5af80dda71509c44b83f1853924.Funnels | 1 | 2021-03-01 | 2021-03-31 | 3 | 2021-04-08 05:28:17 | NULL | 0 | 33 |
| 1429694 | NULL | done8826e5af80dda71509c44b83f1853924.Funnels | 1 | 2021-04-01 | 2021-04-30 | 3 | 2021-04-08 05:28:18 | NULL | 0 | 33 |but then some more got added during the archiving process:
| idinvalidation | idarchive | name | idsite | date1 | date2 | period | ts_invalidated | ts_started | status | report |
| 1430850 | NULL | done.Funnels | 1 | 2021-03-01 | 2021-03-31 | 3 | 2021-04-08 05:28:02 | NULL | 0 | 33 |
| 1430851 | NULL | done8826e5af80dda71509c44b83f1853924.Funnels | 1 | 2021-03-01 | 2021-03-31 | 3 | 2021-04-08 05:28:17 | NULL | 0 | 33 |
| 1430852 | NULL | done0454a4912949ddaecc8bf8e707b93f17.Funnels | 1 | 2021-03-01 | 2021-03-31 | 3 | 2021-04-08 05:28:12 | NULL | 0 | 33 |
| 1430853 | NULL | done252e705405118c93ee8ffcda1daff4ab.Funnels | 1 | 2021-03-01 | 2021-03-31 | 3 | 2021-04-08 05:28:07 | NULL | 0 | 33 |And those rows keep coming back. This is probably due to the invalidation cascading.
@diosmosis
commented
on April 8th 2021
Hi @nelhefni, I'll add context to your investigation below:
After the update, nothing was added to the archive_invalidations table, but 2 raws were removed:
This is expected, we remove pending invalidations for the old funnel ID.
Then, when the cron job started, the invalidation process added a lot of new raws to the archive_invalidations table:
So far, still expected: when we edit the funnel, we add the information to rearchive to list in the option table, core:archive will look at that list before archiving a site and create invalidations. We do this so the request to edit a funnel doesn't slow down due to inserting invalidations.
Not too sure why there are a lot of idarchive set to NULL though.
The idarchive is just a diagnostic, the code doesn't use it. It's set to the existing latest idarchive w/ the same parameters in case we want to know which previous idarchive the invalidation is meant to replace. If it's NULL, it would mean there wasn't one.
I'm not sure to understand the purpose of this done.Funnels, could you please explain the concept behind it?
So the archive in question is one of the old ones that are meant to be replaced. I'm not sure exactly what you're asking for, so I'll explain but feel free to ask if there's more information you want. The "done flag" of an archive tells us a couple things:
- having
donein the name tells us that this row is the status indicator for the archive - having
.Funnelsmeans the archive should only contain reports for the Funnels plugin. Archiving in matomo is the slowest and most intensive part, so we don't want to do it when we don't have to. If we know that we don't need to archive every other plugin, then we can save a lot of computation by only archiving the single one. If we set the value toDONE_PARTIAL(5) then we know it only has some or one report for the plugin that are meant to override older archives.
This was also a partial archive and should not have been set to DONE_INVALIDATED. This was an issue that I thought was fixed, but I don't see the change in 4.x-dev. I'll put up a PR for this, I suspect it will solve some problems.
And those raws keep coming back. This is probably due to the invalidation cascading.
That's odd, they shouldn't come back if you remove them from the invalidations table. Can you check the contents of the ReArchiveList option? (ie, SELECT * FROM option WHERE option_name = 'ReArchiveList')
@nelhefni
commented
on April 8th 2021
@diosmosis thanks for the explanation
Here is the content of the ReArchiveList option:
SELECT * FROM matomo_option WHERE option_name = 'ReArchiveList'
| option_name | option_value | autoload |
| ReArchiveList | a:2:{i:0;s:86:"{"idSites":"all","pluginName":"Cohorts","report":null,"startDate":null,"segment":null}";i:1;s:88:"{"idSites":"all","pluginName":"UsersFlow","report":null,"startDate":null,"segment":null}";} | 0 |
@diosmosis
commented
on April 9th 2021
Hi @nelhefni, thanks for going through the trouble of investigating this issue so thoroughly! I was able to track down the root causes of the two issues identified so far:
They will hopefully be ready to test or merge soon.
I'm still not sure about the Funnel invalidations reoccurring, though. If you continue to see this, it might help to add a warning log in ArchiveInvalidator::markArchivesInvalidated() when the parameters match the plugin/report/idsite combination and see where it's happening.
@nelhefni
commented
on April 9th 2021
thanks @diosmosis !
@diosmosis
commented
on April 12th 2021
@nelhefni both of the PRs have been merged, would you be able to see if they solve the problem for your?