@MichaIng opened this Issue on May 24th 2021

Since Matomo v4.3.0 I think, the dashboard widgets "Insights Overview" and "Moves and Shakers" of the Insights plugin show only "Others +100%" and "Others -100%" where most changed page title views should be listed. It looks like page titles with less views/changes are accumulated as "Others", but all of them together change most, which doesn't seem like intended, at least makes it useless.

Expected Behavior

Two distinct most changed page titles should be shown in the widgets.

Current Behavior

PAGE TITLES UNIQUE PAGEVIEWS EVOLUTION
Others +5259 +100%
Others -5691 -100%

Possible Solution

No idea. Probably an underlying API has changed, when obtaining a list of page titles, where less important page titles are merged?

Steps to Reproduce (for Bugs)

  1. Enable Insights plugin
  2. Add both Insights widgets "Insights Overview" and "Moves and Shakers" to dashboard

Context

Your Environment

  • Matomo Version: 4.3.1-rc1 (issue was present with 4.3.0 already)
  • PHP Version: 8.0.5
  • Server Operating System: Debian Bullseye
  • Additionally installed plugins:
    API, Actions, Annotations, BotTracker 2.02, BulkTracking, CoreAdminHome, CoreConsole, CoreHome, CorePluginsAdmin, CoreUpdater, CoreVisualizations, DBStats, DarkTheme 1.1.6, Dashboard, DevicePlugins, DevicesDetection, Diagnostics, Goals, ImageGraph, Insights, Installation, Intl, LanguagesManager, Live, LogViewer 4.0.1, Login, Marketplace, Monolog, Morpheus, PagePerformance, PrivacyManager, Proxy, Referrers, Resolution, SEO, SegmentEditor, SitesManager, Transitions, UserLanguage, UsersManager, VisitFrequency, VisitTime, VisitorInterest, VisitsSummary, WebsiteMeasurable
  • Browser: Opera 78
  • Operating System: Windows 10 21H1
@diosmosis commented on May 25th 2021 Member

Hi @MichaIng, thanks for the report, we see this on our instances as well.

@flamisz commented on June 2nd 2021 Contributor

I investigated the issue. I haven't sold it yet, but it looks like it's happening because we modified the logic around the summary row for datatables.

https://github.com/matomo-org/matomo/blob/c3bf4c85315a7c853eb13fa7318ca669fcab88c6/core/DataTable.php#L841-L849

It adds an extra row and we don't filter that out.

@flamisz commented on June 3rd 2021 Contributor

The last working version is 4.3.0-rc1.
The diff between 4.3.0-rc1 and 4.3.0-rc2 is here: https://github.com/matomo-org/matomo/compare/4.3.0-rc1...4.3.0-rc2

It has a commit about summary rows. I check if it works without that commit.

@flamisz commented on June 3rd 2021 Contributor

@diosmosis I checked and this commit causes the issue here: https://github.com/matomo-org/matomo/commit/8366ad8ded026d8e6907a4738be44744c72a0312

Do you have an idea why it could happen? As I check the data I see we have an extra row now and don't filter that out during the process.

@diosmosis commented on June 3rd 2021 Member

@flamisz no, I'm not sure why that would happen. To figure out why, I might take the following approaches:

  • look in the Insights code for getRowFromLabel()/getRows() calls since those were the main methods that changed
  • look in the Insights code for uses of subtables & summary rows and see if the use could cause different results now
  • create an automated test that fails on 4.x-dev but passes when checking out 8366ad8^, and figure out what the difference in execution is
@flamisz commented on June 3rd 2021 Contributor

It looks like the problem is in the rebuildIndex() method.

https://github.com/matomo-org/matomo/blob/8b4b8ce017c15a534101fc10494a9f05cb824a58/core/DataTable.php#L740-L753

The diff is:

public function rebuildIndex()
{
    $this->rowsIndexByLabel = array();
    $this->rebuildIndexContinuously = true;
    foreach ($this->rows as $id => $row) {
        $label = $row->getColumn('label');
        if ($label !== false) {
            $this->rowsIndexByLabel[$label] = $id;
        }
    }

-    if ($this->summaryRow) {
-        $label = $this->summaryRow->getColumn('label');
-        if ($label !== false) {
-            $this->rowsIndexByLabel[$label] = DataTable::ID_SUMMARY_ROW;
-        }
-    }

    $this->indexNotUpToDate = false;
}

Is this make sense @diosmosis? Is this something that should be fixed in this method or something should be fixed in Insight plugin?

@diosmosis commented on June 4th 2021 Member

@flamisz we can't add it to the rowsIndexByLabel variable, since the summary row label is -1, but sometimes users track data as -1, so one of the two gets lost or overwritten. So it would probably have to be fixed in the Insight plugin.

This Issue was closed on June 13th 2021
Powered by GitHub Issue Mirror