Track ecommerce views as new dimensions instead of custom variables #15999

sgiehl · 2020-05-28T09:41:27Z

fixes #12035

…ew data

…fore 4.0

…lues

sgiehl · 2020-05-29T20:11:42Z

composer.json

@@ -35,7 +35,7 @@
        "matomo/cache": "~2.0",
        "matomo/decompress": "~2.0",
        "matomo/ini": "~2.0",
-        "matomo/matomo-php-tracker": "dev-4.x-dev",
+        "matomo/matomo-php-tracker": "dev-ecommerceview",


This needs to be reverted after matomo-org/matomo-php-tracker#68 has been merged

sgiehl · 2020-05-29T20:18:19Z

...est_ecommerceOrderWithItemsmultipleDates_andMultipleWebsites__Goals.getItemsCategory_day.xml

@@ -7,53 +7,89 @@
 				<revenue>2500</revenue>
 				<quantity>3</quantity>
 				<orders>2</orders>
+				<nb_visits>3</nb_visits>


The product view metrics were missing here before because of

matomo/plugins/Goals/API.php

Lines 792 to 796 in bc499de

// we do not enter the IF

// if case idSite=1,3 AND period=day&date=datefrom,dateto,

if ($customVariableTableForDate instanceof DataTable

&& $customVariableTableForDate->getMetadata(Archive\DataTableFactory::TABLE_METADATA_PERIOD_INDEX)

) {

That's no longer the case, as product views are directly archived...

sgiehl · 2020-05-29T20:26:19Z

tests/PHPUnit/System/expected/test_reportLimiting__Goals.getItemsCategory_day.xml

@@ -2,19 +2,46 @@
 <result>
 	<row>
 		<label>Books Cat #0</label>
+		<nb_visits>5</nb_visits>


The reason why we have more categories here as before is, that custom variables reports are limited by the report limiting in the test. The ecommerce reports do not have any limits when archived, so all categories are shown now. Not sure if that's something we should maybe change.

sgiehl · 2020-05-29T21:06:01Z

tests/javascript/index.php

-        deepEqual( tracker3.getCustomVariable(2, "page"), false, "No data Ecommerce price");
-        deepEqual( tracker3.getCustomVariable(3, "page"), false, "No data Ecommerce view SKU");
-        deepEqual( tracker3.getCustomVariable(4, "page"), false, "No data Ecommerce view Name");
-        deepEqual( tracker3.getCustomVariable(5, "page"), ["_pkc",""], "No data Ecommerce view Category");


it's no longer possible to receive the product view values, as they are no longer stored in custom variables. Not sure if we should add a getProductView method in order to be able to test that better...

tsteur

Left a few comments. Looks good overall. FYI @mattab this will basically add 8 new columns to log_link_visit_action

plugins/Ecommerce/Columns/ProductViewName.php

plugins/Ecommerce/Columns/ProductViewCategory.php

plugins/Ecommerce/Columns/ProductViewPrice.php

plugins/Ecommerce/Columns/ProductViewSku.php

plugins/Ecommerce/Columns/ProductViewPrice.php

tsteur · 2020-06-04T01:39:39Z

plugins/Goals/API.php

+        // Before Matomo 4.0.0 ecommerce views were tracked in custom variables
+        // So if Matomo was installed before try to fetch the views from custom variables and enrich the report
+        if (version_compare(DbHelper::getInstallVersion(),'4.0.0-b2', '<')) {
+            $this->enrichItemsTableWithViewMetrics($dataTable, $recordName, $idSite, $period, $date, $segment);


In the enrichItemsTableWithViewMetrics might need to check if the custom variables plugin is active since we will move it to the marketplace plus people might deactivate it?

Might not be fully understanding it... are we there basically merging the metrics from custom variables with the metrics from log_link_visit_action if Matomo was installed before 4.0?

I suppose this would usually be mostly for data on the day they upgrade to Matomo 4 cause all previous day will have data from custom variables and all other ones from log_link_visit_action.

That's basically for BC. If someone installs Matomo after 4 the variables will be directly tracked into the log_link_visit_action table, so there is no need to fallback to the old behavior.
If Matomo was installed before it might be possible that there were some ecommerce views tracked as custom variables. If those old data would be archived there wouldn't be any ecommerce views reported, so we try to get those data from the custom variables reports and merge that in. (That was the way it was done before).
Actually we should see in the UI tests that this behaviour works, as I didn't update the OmniFixture.sql, so it still includes ecommerce views as custom variables but is archived with the new logic before the tests run...

tsteur · 2020-06-04T01:41:56Z

plugins/Goals/API.php

-        $this->enrichItemsTableWithViewMetrics($dataTable, $recordName, $idSite, $period, $date, $segment);
+        // Before Matomo 4.0.0 ecommerce views were tracked in custom variables
+        // So if Matomo was installed before try to fetch the views from custom variables and enrich the report
+        if (version_compare(DbHelper::getInstallVersion(),'4.0.0-b2', '<')) {


btw getInstallVersion might not be set AFAIK as it was only added in 2018

That should be fine I guess. version_compare(null, '4.0.0-b2', '<') returns true

tsteur · 2020-06-04T01:43:31Z

plugins/CustomVariables/Archiver.php

-        if (in_array($slot, array(\MatomoTracker::CVAR_INDEX_ECOMMERCE_ITEM_SKU, \MatomoTracker::CVAR_INDEX_ECOMMERCE_ITEM_NAME, \MatomoTracker::CVAR_INDEX_ECOMMERCE_ITEM_CATEGORY))) {
+        // Before Matomo 4.0.0 ecommerce views were tracked in custom variables
+        // So if Matomo was installed before still try to archive it the old way, as old data might be archived
+        if (version_compare(DbHelper::getInstallVersion(),'4.0.0-b2', '<') && in_array($slot, array(3, 4, 5))) {


As mentioned below getInstallVersion might not return anything just fyi

sgiehl · 2020-06-05T11:36:47Z

@tsteur left some comments and applied the other feedback

tsteur · 2020-06-09T01:41:25Z

@sgiehl tracking etc worked. Not sure if I was supposed to see this in the visits log?

i tracked some data and didn't see it on hover but not sure if it maybe used to be shown? Eg in #12035 (comment) it looks like they used to be shown?

Otherwise worked nicely 👍 Could you maybe add it if needed to the visitor log, resolve merge conflicts and then merge?

sgiehl added 5 commits May 28, 2020 11:37

Adds new columns to log_link_visit_action table to track ecommerce vi…

5a7a014

…ew data

update piwik.js to track ecommerce view s the new way

3921ebc

Adjust archiving

0f1685a

update tests

ca06f57

[TEMP] use php-tracker branch for composer

ae91446

sgiehl added the Pull Request WIP Indicates the current pull request is still work in progress and not ready yet for a review. label May 28, 2020

sgiehl added this to the 4.0.0 milestone May 28, 2020

This was referenced May 28, 2020

Track ecommerce views not as custom variables matomo-org/matomo-php-tracker#68

Merged

Track ecommerce events with improved data matomo-org/plugin-VisitorGenerator#57

Merged

sgiehl and others added 12 commits May 28, 2020 12:00

Archive custom variables the old way, only if Matomo was installed be…

eecc00c

…fore 4.0

add fallback to custom variables to keep bc when replaying old logs

3d6ee1b

update some expected test files

689f3ac

fix jslint errors

18e093d

update javascript test

1aea36e

update dimension names

2988ae7

updates some expected test files

7cb1afc

add product view data to visitor details and add suggested segment va…

86cf609

…lues

updates some expected test files

7b053ea

rebuilt piwik.js

443305f

adjust checked version

bc499de

update expected ui files

1ad8480

sgiehl commented May 29, 2020

View reviewed changes

sgiehl added 2 commits May 29, 2020 22:42

updates some expected test files

6d3e603

submodule updates

2b51865

sgiehl commented May 29, 2020

View reviewed changes

sgiehl removed the not-in-changelog For issues or pull requests that should not be included in our release changelog on matomo.org. label May 30, 2020

sgiehl marked this pull request as ready for review May 30, 2020 14:40

tsteur requested changes Jun 4, 2020

View reviewed changes

sgiehl added 2 commits June 5, 2020 13:09

Merge branch '4.x-dev' into ecommerceview

4c5b384

apply some review feedback

d1c3756

tsteur mentioned this pull request Jun 9, 2020

Export of reports no longer working in Matomo 4 #16043

Closed

sgiehl added 3 commits June 9, 2020 08:50

Merge branch '4.x-dev' into ecommerceview

7bb4ed8

Show product views in action tooltips

0298ffd

updates expected UI files

e0678ae

sgiehl merged commit 8e25f1c into 4.x-dev Jun 9, 2020

sgiehl deleted the ecommerceview branch June 9, 2020 08:58

mattab added the Enhancement For new feature suggestions that enhance Matomo's capabilities or add a new report, new API etc. label Sep 28, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Track ecommerce views as new dimensions instead of custom variables #15999

Track ecommerce views as new dimensions instead of custom variables #15999

sgiehl commented May 28, 2020

sgiehl May 29, 2020

sgiehl May 29, 2020

sgiehl May 29, 2020

sgiehl May 29, 2020 •

edited

tsteur left a comment

tsteur Jun 4, 2020

tsteur Jun 4, 2020

sgiehl Jun 5, 2020

tsteur Jun 4, 2020

sgiehl Jun 5, 2020

tsteur Jun 4, 2020

sgiehl commented Jun 5, 2020

tsteur commented Jun 9, 2020

	// we do not enter the IF
	// if case idSite=1,3 AND period=day&date=datefrom,dateto,
	if ($customVariableTableForDate instanceof DataTable
	&& $customVariableTableForDate->getMetadata(Archive\DataTableFactory::TABLE_METADATA_PERIOD_INDEX)
	) {

Track ecommerce views as new dimensions instead of custom variables #15999

Track ecommerce views as new dimensions instead of custom variables #15999

Conversation

sgiehl commented May 28, 2020

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

sgiehl May 29, 2020 • edited

Choose a reason for hiding this comment

tsteur left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

sgiehl commented Jun 5, 2020

tsteur commented Jun 9, 2020

sgiehl May 29, 2020 •

edited