Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Data in Sales Report and Channel Grouping Segment differences #19590

Open
dave-anh-kpi opened this issue Aug 2, 2022 · 7 comments
Open

Data in Sales Report and Channel Grouping Segment differences #19590

dave-anh-kpi opened this issue Aug 2, 2022 · 7 comments
Assignees
@mattab
Labels
c: Attribution c: Data Integrity & Accuracy c: Segments Segmentation and Segment editor related improvements and fixes. Potential Bug Something that might be a bug, but needs validation and confirmation it can be reproduced.
Milestone
5.5.0

Comments

@dave-anh-kpi
Copy link

Expected Behavior

Normally, Matomo is based on a last-click attribution model by default. This means, if you look at your sales per channel in the Sales report, you should find a specific number per each channel. Let's say you are looking at your sales from "Direct" traffic.
If you look at your sales while using a segment based on "Direct Traffic", you should find the exact same number of sales in the "E-commerce > Overview" report.

Current Behavior

Well, this is not happening. The data is different, which mean I don't know how to assess the performance of my different traffic channels.

Here's an example of my case (find enclosed some screenshots Screenshot Matomo problème attrib.pdf):

In the Sales Report: I have 4 orders attributed to “Direct” channel (1).
When I am using a segment based on “Direct traffic”, Matomo shows me 10 orders (2, 3).
These 10 orders are also found in the User Log report of Direct Traffic" (4).
I also tried the Multi Attribution Report but the data shown is once again different… (5)

Steps to Reproduce (for Bugs)

I noticed that the issue is the same for all the dimensions you use to segment your traffic. Here's an example to reproduce the problem on https://demo.matomo.cloud/

  1. Select the following dates: From 2022-07-01 To 2022-07-31
  2. Go to "E-commerce > Sales"
  3. Click on "Country" in the section "Sales by User location"
  4. Search for New Zealand (you will find 13 e-commerce orders)
  5. Now, Go to E-commerce > Overview
  6. Apply the segment "NZ visitors" (you will find 10 e-commerce orders)

Context

I already talked to Matthieu about this issue, he told me to send an email to the support team, which I did. The support Team told me to post a message here.

Your Environment

I use Matomo Cloud
@dave-anh-kpi dave-anh-kpi added the Potential Bug Something that might be a bug, but needs validation and confirmation it can be reproduced. label Aug 2, 2022
@sgiehl
Copy link
Member

sgiehl commented Aug 2, 2022

Hi @dave-anh-kpi
Thanks for creating this issue. I guess we might need to dig a beet deeper into how the conversions are calculated in detail.
A first assumption would be that, the reports might differ as the dimension stored for the visit differs from the conversion.
The conversion reports should be calculated based on the data stored together with the conversion, while segments are mostly applied to the data stored with the visit. So if bother differs, that could cause some conversions not to show up in the segmented report.
I'll hand this over to our product team to plan the further investigation of this issue.

@sgiehl sgiehl added this to the For Prioritization milestone Aug 2, 2022
@dave-anh-kpi
Copy link
Author

Hello,
Do you have any update on the issue ?
We are talking about a major issue --> My client (+10K€ Matomo Subscription) is not able to understand which channel is converting the most ! This is a basic in Web Analytics...
Here more recent screenshots showing the huge difference between the ways of looking at the data :

image

image

image

Thank you for your help.

@jane-twizel jane-twizel modified the milestones: For Prioritization, 5.0.0 Sep 22, 2022
@jane-twizel
Copy link

@dave-anh-kpi I really appreciate you letting us know about this issue and also the great description. Our developers will investigate this more to work out next steps.

@mattab mattab added the 5.0.0 label Jan 4, 2023
@gwizcole
Copy link

Can confirm we are seeing a similar issue. In addition, we’re seeing large variances in how conversions are grouped into which channel based on the report we choose. There is no solid consistency when comparing metrics for the same dimension for the same date range between different sales reports that are intended to show essentially the same thing.

  • Channel grouping reports, sales/eComm reports, UTM reports, multi-attribution reports are not consistent.
  • We use the UTM add-on. Figures in UTM add-on reports are not matching Sales reports, and UTM data is not accurately grouped or included at all. We use the same UTM structure across all tracked campaigns. In the case of Google Ads, email, referring websites, and other channels, a portion of our campaigns get grouped and identified in Matomo, others do not, and this also changes depending on which report we view.
  • Viewing the live log, watch can watch in real time as some conversion journeys get split after the user lands on-site and visits a specific page, or the ID gets split before they enter, each showing different source attribution metrics.

An a related note. In the multi Attribution comparison tools, the last click model show different figures than Matomo’s default reports, which are stated as using last referrer.

Is there something we can do to remedy this before 5.0? We would love to support Matomo but at this stage we can’t make use of reports.

@mattab
Copy link
Member

mattab commented Jan 27, 2023

An a related note. In the multi Attribution comparison tools, the last click model show different figures than Matomo’s default reports, which are stated as using last referrer.

Thanks @gwizcole for the report. I've just checked and notice we're seeing a difference of 1-3% in one particular account, which is not actually expected.
Are you seeing a similar discrepancy?

Also would you maybe be able to post 2 screenshots showing the difference between

  1. Multi attribution (with days to look back set to 90 days) and Attribution model set to "Last Non-Direct" and Goal X selected
  2. and the Goals > Goal X > Channel Type report

Then this can help assess how big the problem is (i'll investigate with the team what could explain this. it might be the fact a 90 days window does not look back everyone in case they first visited more than 90 days ago).

@mattab mattab self-assigned this Jan 27, 2023
@gwizcole
Copy link

gwizcole commented Feb 2, 2023

Hi @mattab, appreciate the response!

Yes, we are seeing a similar discrepancy. It appears to be even higher between channel figures within different reports, but still seeing it with multi-attribution and with 90 day window selected. I've compiled a report with the screenshots you've requested showing the discrepancies between each.

Due to the sensitive nature of the information would it be okay to send the screenshots elsewhere — via email or something else? If you have an email address I can send these to we can continue the conversation here if it helps others, just hoping to keep exact figures private. Thanks!

@mattab mattab modified the milestones: 5.0.0, 5.3.0 Feb 22, 2023
@mattab mattab removed the 5.0.0 label Aug 4, 2023
@mattab mattab added c: Attribution c: Data Integrity & Accuracy c: Segments Segmentation and Segment editor related improvements and fixes. labels Dec 10, 2023
@mattab mattab modified the milestones: 5.3.0, 5.5.0 Dec 14, 2023
@9joshua
Copy link
Contributor

9joshua commented Dec 18, 2023

Reporting on another user who has an issue with Goals being attributed to the Search channel differently when a segment is applied...

All visits segment
Screenshot from 2023-12-19 09-27-37

Segment = Channel is Search
Screenshot from 2023-12-19 09-27-20

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@mattab mattab

Labels
c: Attribution c: Data Integrity & Accuracy c: Segments Segmentation and Segment editor related improvements and fixes. Potential Bug Something that might be a bug, but needs validation and confirmation it can be reproduced.
Projects
None yet
Milestone
5.5.0
Development

No branches or pull requests
6 participants
@mattab @sgiehl @9joshua @gwizcole @jane-twizel @dave-anh-kpi