@diosmosis opened this Pull Request on October 8th 2020 Member

Changes:

  • unset status on failed archiving jobs so they will be picked up again in the next core:archive run
  • make sure ArchiveSelector::getArchiveIds behaves consistently with ArchiveSelector::getArchiveIdAndVisits
  • make sure the 'done' all plugins archives aren't used w/ pluginOnly=1, otherwise we will see the 'done' archive and assume we can use it. pluginOnly=1 is used when an invalidation for a specific plugin/report is added (ie, w/ reArchiveReport). so if it's set, and we find a usable archive, we force archiving again.

    NOTE: this should only happen in core:archive when we see an invalidation w/ the plugin column set. I don't think this can currently happen outside of ArchiveInvalidator::reArchiveReport().

FYI @tsteur

@tsteur commented on October 8th 2020 Member

👍

@diosmosis commented on October 9th 2020 Member

@tsteur found another (potential) issue fixed in https://github.com/matomo-org/matomo/pull/16548/commits/a075a9bb936d2041e18852bac6fa42ff50f96edb

code's a bit confusing, but I don't think it can be made much simpler (we need to get the latest idarchives for each requested plugin, then ignore older ones, and ignore the all plugins archive if individual plugin archives are found). this should avoid some odd edge cases. will merge if the build passes, but could use a review after the fact.

@diosmosis commented on October 9th 2020 Member

@tsteur here's an example of the query, btw:

SELECT idsite, date1, date2,
        GROUP_CONCAT(CONCAT(idarchive,'|',`name`) ORDER BY idarchive DESC SEPARATOR ',') AS archives
   FROM matomo_archive_numeric_2014_05
  WHERE idsite IN (1)
    AND ((name IN ('done','done.ExamplePlugin')) AND (value IN (1,3,4,5)))
    AND ts_archived IS NOT NULL
    AND ((period = 1 AND date1 = '2020-10-03' AND date2 = '2020-10-03'))
GROUP BY idsite, date1, date2

the explain for it is:

--------+------------------------------------+
| id | select_type | table                          | partitions | type  | possible_keys                                   | key                       | key_len | ref  | rows | filtered | Extra                              |
+----+-------------+--------------------------------+------------+-------+-------------------------------------------------+---------------------------+---------+------+------+----------+------------------------------------+
|  1 | SIMPLE      | matomo_archive_numeric_2014_05 | NULL       | range | index_idsite_dates_period,index_period_archived | index_idsite_dates_period | 21      | NULL |    1 |     8.00 | Using index condition; Using where |
+----+-------------+--------------------------------+------------+-------+-------------------------------------------------+---------------------------+---------+------+------+----------+------------------------------------+
This Pull Request was closed on October 9th 2020
Powered by GitHub Issue Mirror