Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Do not always extend visit for content impressions #15860

Closed
wants to merge 3 commits into from

Conversation

tsteur
Copy link
Member

@tsteur tsteur commented Apr 25, 2020

@mattab the idea here is that content impressions is not quite an interactive action and as such it should not extend the visit or maybe only less often (if user scrolls after a while on the page we could consider it maybe interactive). The reason is mostly performance related to have less concurrent log_visit updates and therefore less potential locks. Often people have say 10-30 content blocks and either all of them are sent at the same time (eg if trackAllContentBlocks) or many of them are sent in parallel (when trackVisibleBlocksOnly and many become visible while scrolling). In Matomo 4 we can start queueing this requests so they are not sent all at the same time and instead as bulk (maybe they are already not sure) but in general we want to avoid too many concurrent requests here.

The idea is so far we only extend visit if the last action was more than a minute ago. Could also completely remove it and never extend the visit (only content interactions would extend it). This can avoid many log_visit locks.

Haven't fully tested it yet, but wanted to get feedback on it as I reckon it can help.

For content impressions it will basically no longer update anything on log_visit which be good.

@tsteur tsteur added the Needs Review PRs that need a code review label Apr 25, 2020
@tsteur tsteur added this to the 3.13.6 milestone Apr 25, 2020
@tsteur tsteur added the c: Performance For when we could improve the performance / speed of Matomo. label Apr 25, 2020
@mattab
Copy link
Member

mattab commented Apr 26, 2020

Sounds great, change makes sense. If users want accurate visit time, heartbeat timer would be a better solution. It would be fine to also remove the 1 minute logic and never update the visit. Be good to mention in developer changelog too.

// it is only numeric when directly being called afterRequestProcessed() and not eg handleExistingVisit
// because the VisitLastActionTime dimension will overwrite the original value of the visitor.
// we want to make sure to work on the value from the DB
$lastActionTimeDate = Date::factory($lastActionTime)->addPeriod(1, 'minutes');
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should this be addPeriod(5 or is the comment below inaccurate?

{
if ($request->getMetadata('Actions', 'doNotExtendVisit')) {
unset($valuesToUpdate['visit_last_action_time']);
unset($valuesToUpdate['visit_total_time']);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just a suggestion, but doNotExtendVisit could be handled in Visit.php so any plugin could add behavior like this.

@tsteur
Copy link
Member Author

tsteur commented Apr 27, 2020

fyi it seems I can't make this work. It seems actually needed to update visit_last_action_time as otherwise the time_spent_ref_action is calculated wrong.

You can see here some content impressions and their time_spent_ref_action where then eg multiple content impressions get a time_spend_ref_action of 120, but only the first should get 120 and the remaining to 0 . The same for some other entries around 60s etc.

image

This means the data in the exported visitor log shows a wrong time_spent_ref_action

image

As a result, it also changes eg sum_time_spent on page where it aggregates the time_spent_ref_action and it results in an increased value as it tracks eg 120 multiple times instead of just once.

image

Technically, I noticed the page view report should maybe not include time_spent_ref_action of content impressions for a given page, but then we'd result in 0 sum_time_spent and it probably should be included.

So I basically couldn't figure out a way to make this work unfortunately.

FYI @mattab

@tsteur tsteur closed this Apr 27, 2020
@tsteur tsteur deleted the coimpresperfimpr branch April 27, 2020 22:57
@mattab
Copy link
Member

mattab commented Apr 30, 2020

@tsteur

You can see here some content impressions and their time_spent_ref_action where then eg multiple content impressions get a time_spend_ref_action of 120, but only the first should get 120 and the remaining to 0 . The same for some other entries around 60s etc.

i'm not sure i fully understand yet, but say if we "forced" content impressions' time_spent_ref_action values to be 0, would it mean we could reconsider the PR and performance improvements?

Technically, I noticed the page view report should maybe not include time_spent_ref_action of content impressions for a given page, but then we'd result in 0 sum_time_spent and it probably should be included.

it would be fine I think to have sum_time_spent as 0 in these cases, especially if it solves some major performance problem edge cases... The view would be that if people need accurate time on page, they would better use HeartBeat timer.

EDIT: there is also this issue but it's not wanted for now due to performance impacts #9748

@tsteur
Copy link
Member Author

tsteur commented Apr 30, 2020

Yeah we'd need to implement #9748 to make this work which in the end would cause more issues.

If we forced content impressions to have 0 values for time_spent_ref_action, then we could use this PR. However, anyone with content impressions wouldn't get an accurate time on page anymore (which may be fine as events seem to be breaking time on page feature already anyway if I see this right)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
c: Performance For when we could improve the performance / speed of Matomo. Needs Review PRs that need a code review
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants