Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Proportional evolution comparison for incomplete periods #18099

Merged
merged 33 commits into from Nov 15, 2021

Conversation

bx80
Copy link
Contributor

@bx80 bx80 commented Oct 5, 2021

Description:

Fixes #6784

This PR changes multi site evolution metrics to calculate proportionally to the percent of the current period which is complete.

For example, if there were 1,000 site visits last week, the current time is midnight on Monday and there have been 150 visits so far this week, then the All Websites visits evolution percent would have shown as a red -85% because 150 visits is a big drop from 1,000. (This Week Visits - Last Week Visits) / (Last Week Visits)

However this in an unfair comparison as the current week is incomplete. If the remaining days of the week continue to at 150 visits then the current week will be an improvement over the previous week.

Since only ~14% of the current week has elapsed a better comparison would be 14% of the previous week total visits vs the current visits in this week, this results in a green 5% evolution percent. (This Week Visits - (Last Week Visits * Percent of Period Complete) / (Last Week Visits * Percent of Period Complete)

Review

…rcent of the current period which is complete
@bx80 bx80 added not-in-changelog For issues or pull requests that should not be included in our release changelog on matomo.org. Needs Review PRs that need a code review labels Oct 5, 2021
@bx80 bx80 added this to the 4.6.0 milestone Oct 5, 2021
Copy link
Member

@tsteur tsteur left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@bx80 nice, I did a very basic check locally and this seems to have worked 🎉

Could we also add some kind of system tests for this to ensure this works (and keeps working in the future)?

Not sure if easy to do. Be good to add a tooltip on the evolution percentage when the currently viewed report includes today to then say something like that this percentage is based on comparing the time that has been past in the current period.
image

Or if I understand the code directly, that's not just for periods that include today but eg would apply when viewing a month and the previous month has more or less days.

Otherwise people will look at the previous month, and calculate the percentage themselves and be confused why they come up with different percentage.

@mattab can you confirm we also want to adjust the percentage when eg one month has 30 days and the other 31 and then adjust the percentage accordingly? Or should this happen only when the currently viewed period includes today?

plugins/CoreHome/Columns/Metrics/EvolutionMetric.php Outdated Show resolved Hide resolved
plugins/CoreHome/Columns/Metrics/EvolutionMetric.php Outdated Show resolved Hide resolved
@mattab
Copy link
Member

mattab commented Oct 6, 2021

@mattab can you confirm we also want to adjust the percentage when eg one month has 30 days and the other 31 and then adjust the percentage accordingly? Or should this happen only when the currently viewed period includes today?

This "proportional-adjustement" logic should happen only when currently viewed period includes today.

@bx80 bx80 self-assigned this Oct 6, 2021
@bx80
Copy link
Contributor Author

bx80 commented Oct 7, 2021

I've added a tooltip to explain when the comparison is proportional.
image

If the current period doesn't contain the current date/time then the proportional adjustment will not be applied (ratio = 1, do nothing)

Also added a unit test which feeds some fixed date empty datatables to the ratio calculation code to check that the expected ratio is returned.

@mattab
Copy link
Member

mattab commented Oct 7, 2021

Q: would make the tooltip maybe more descriptive if possible to remove any possible confusion in how we came up the with number, for example this below (or a better text):

The currently selected time period is 53% complete.
When the previous period was also 53% complete, there would have been an estimated 5,300 visits (out of a total of 10,000 visits in the previous period).
12,925 visits in today|this week|this month compared to 5,300 visits in the previous period (2021-09-07.) Evolution: +x%

(note: the last line is from the existing tooltip used in Acquisition > Overview report)

@tsteur
Copy link
Member

tsteur commented Oct 7, 2021

@mattab yes it would be, and thought about it, but considering 80/20 it be likely good enough to just have something generic

@bx80
Copy link
Contributor Author

bx80 commented Oct 7, 2021

It was quick to add some more detail to the tool tip, so I went ahead and did it, hope that's okay!

Copy link
Member

@tsteur tsteur left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@bx80 nice, that works already. Left few comments and seems there are some tests that need adjusting https://app.travis-ci.com/github/matomo-org/matomo/jobs/542051052 and https://app.travis-ci.com/github/matomo-org/matomo/jobs/542051049

plugins/CoreHome/Columns/Metrics/EvolutionMetric.php Outdated Show resolved Hide resolved
plugins/CoreHome/Columns/Metrics/EvolutionMetric.php Outdated Show resolved Hide resolved
plugins/CoreHome/Columns/Metrics/EvolutionMetric.php Outdated Show resolved Hide resolved
Copy link
Member

@tsteur tsteur left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice. I see you got the ts_archived working 👍

There are some tests failing in https://app.travis-ci.com/github/matomo-org/matomo/builds/239598140

Can we maybe remove this metadata from each row in the final API output? This way we reduce chances of tests failing anymore randomly as ts_archived might be sometimes bit different and reduce a lot of the output in the API response.

We can still use the ts_archived in our other filters.

@tsteur tsteur removed the Needs Review PRs that need a code review label Oct 13, 2021
@bx80 bx80 added the Needs Review PRs that need a code review label Oct 17, 2021
@github-actions
Copy link
Contributor

This issue is in "needs review" but there has been no activity for 7 days. ping @matomo-org/core-reviewers

@github-actions github-actions bot added the Stale The label used by the Close Stale Issues action label Oct 25, 2021
@bx80
Copy link
Contributor Author

bx80 commented Oct 26, 2021

@tsteur @sgiehl I've removed metadata from the API output and fixed the broken tests, this PR should now be ready for review again.

@tsteur
Copy link
Member

tsteur commented Oct 26, 2021

Awesome 👍 @sgiehl can you maybe have a last look?

@github-actions github-actions bot removed the Stale The label used by the Close Stale Issues action label Oct 27, 2021
…moved additional constructor param to end, null checks
@sgiehl sgiehl removed the Needs Review PRs that need a code review label Oct 28, 2021
@bx80 bx80 added the Needs Review PRs that need a code review label Oct 28, 2021
Copy link
Member

@sgiehl sgiehl left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Found some other issues while testing it locally. Guess the formatting issue might actually have already existed before 🙈

plugins/MultiSites/API.php Outdated Show resolved Hide resolved
Copy link
Member

@sgiehl sgiehl left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

With the fixes / test updates I have pushed, this PR should be ready to merge.
@bx80 maybe have a look at the additional changes and feel free to merge afterwards

@bx80 bx80 merged commit 556aada into 4.x-dev Nov 15, 2021
@bx80 bx80 deleted the m-6784-evolution-data-interpolation branch November 15, 2021 20:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Needs Review PRs that need a code review not-in-changelog For issues or pull requests that should not be included in our release changelog on matomo.org.
Projects
None yet
4 participants