This is correct and expected behaviour. You can see a bit more info here: #15049 (comment)

We'll need to update the docs that it does not extend the visit. You'd need some custom plugin in this case to get this behaviour.

@tsteur do you have suggestions on workaround for this usecase ?

Initial thought is to introduce ping=2. If set, it should update log_visit.visit_total_time and log_visit.visit_last_action_time.

Adding a regular _paq.push(['trackEvent', 'ping', 'pong', 'pung']) instead of heartbeats would increase the tablesize for log_link_visit_action, which is not a good idea.

ping=2 seems like, for us at least, a viable option. But if upstream would approve of such change is another story. Or another solutions is to fork the HearBeat plugin, and extend the log_visit table with e.g. last_heartbeat_time which holds last DATETIME for that visit.

In reality, this value is already present since log_visit.visit_last_action_time + log_visit. visit_total_time would equal to last_heartbeat_time

Happy to create an PR if any of the initial thoughts seems like a good idea.

I suppose you want to extend all visits?

You could do this in a custom plugin without changing the ping value. All you would need to do is put a file like this in Tracker/MyRequestProcessor.php in the plugin which does say

namespace ...
class MyRequestProcessor{

    public function onExistingVisit(&$valuesToUpdate, VisitProperties $visitProperties, Request $request)
    {
        $valuesToUpdate['visit_last_action_time'] = $request->getCurrentTimestamp()
    }
}

Haven't tested it but something like this should work.

Actually @toredash not sure if we have any public documentation that heart beat extends the visit? it looks like we can actually close the issue.

@tsteur For what I've seen regarding docs, it is for me difficult to say if the docs are explicit enough to answer that question.

https://matomo.org/faq/how-to/faq_21824/

By default, Matomo (Piwik) will accurately track the time spent on your pages for all pages, except the last page view of the visit. By default, on the last page view of any visit, Matomo counts the “Time spent on page” as 0 second. And when your visitor views only one page in the website, the visit duration will be set to a default of 10 seconds.

-> It is possible to configure Matomo to track the time spent on the page accurately, including the last page view of each visit. Learn more about Accurately measure the time spent on each page in the JavaScript Tracking code guide.

I'm not sure if the current implementation does this the way I interpret the docs. From the screenshot in my first post, the ping updates the "Time spent on page" for the last action, not the page itself.

Our applications are primarily single-page-applications, and in rare-cases we might have no actions sent from the client, so we are unable have an accurate report with CustomReports which informs us when the visit ended. This information is used to graph the number of concurrent users active on our service by the minute.

We are able to determine when they came (visit_first_action_time), how long they stayed (visit_total_time), but visit_last_action_time is not a good representation for when the visit ended, even if the source code indicates it:

If anything should be changed, it should be the definition of $segmentName = 'visitEndServer*';, since it is labeled as "End Of Visit", which is not correct, it is actually "Time of last action" (https://github.com/matomo-org/matomo/blob/1a76568aa7f85920e8f4b58e0aa2fd32f64de8ad/plugins/VisitTime/lang/en.json):

All the information is present in log_visit table, it could be computed as such:

SELECT *, (visit_last_action_time + visit_total_time) AS visit_last_heartbeat
FROM log_visit

I looked at the Segment docs (https://developer.matomo.org/api-reference/Piwik/Plugin/Segment#setsqlsegment) to see if there is a way create a new segment, but I think that performance would suffer it it needs to calculate a value based on two other values.

I suppose you want to extend all visits?

Yes, that is correct, I'll try the code you suggested, thanks!

You could do this in a custom plugin without changing the ping value. All you would need to do is put a file like this in Tracker/MyRequestProcessor.php in the plugin which does say

namespace ...
class MyRequestProcessor{

    public function onExistingVisit(&$valuesToUpdate, VisitProperties $visitProperties, Request $request)
    {
        $valuesToUpdate['visit_last_action_time'] = $request->getCurrentTimestamp()
    }
}

Haven't tested it but something like this should work.

If anything should be changed, it should be the definition of $segmentName = 'visitEndServer*';, since it is labeled as "End Of Visit", which is not correct, it is actually "Time of last action"

I reckon that makes sense 👍 Any thoughts @mattab ?

Sounds like a good fix to rename the dimension + fix the code comment @tsteur @toredash

I looked at the Segment docs (https://developer.matomo.org/api-reference/Piwik/Plugin/Segment#setsqlsegment) to see if there is a way create a new segment, but I think that performance would suffer it it needs to calculate a value based on two other values.

@toredash I reckon performance of using 2 fields VS 1 field shouldn't be an issue? if you can try this and add a new segment, that would be great... so we can cover all use cases including yours 👍

@mattab

Our use case is that we want to graph the number of unique visitors on a granularity of per minute. That is not possible with how Matomo works now, as the segment only considers when the first and last actions was done, not anything in between.

When using Custom Reports, we have two reports, one with "Server Time - Start of Visit (Hour)" and "Server Time - Start of Visit (Minute)". The other report contains "Server Time - End of Visit (Hour)" and "Server Time - End of Visit (Minute)". This does not give us the correct data, example:

User A join at UNIX time: 1000, and is active to time 2000. visit_first_action_time = 1000, visit_last_action_time = 2000.

User B join at UNIX time: 1500, and is active to time 2500. visit_first_action_time = 1500, visit_last_action_time = 2500.

User A would then be considered to start the visit at hour 00 and minute 16.
User B would then be considered to start the visit at hour 00 and minute 25.

User A would then be considered to end the visit at hour 00 and minute 33.
User B would then be considered to end the visit at hour 00 and minute 41.

When the use the start of visit-report and substract the corresponding values for the same time from the end of visit-report.
We then want to be able to get data that looks like so:

HH:MM - Uniq visitors
00:15 - 0
00:16 - 1
00:17 - 1
...
00:24 - 1
00:25 - 2
00:26 - 2
...
00:32 - 2
00:33 - 1
00:34 - 1
00:35 - 1
...
00:40 - 1
00:41 - 1
00:42 - 0
00:43 - 0

But that is not what we get, we get:

HH:MM - Uniq visitors
00:15 - null
00:16 - 1
00:17 - null
...
00:24 - null
00:25 - 1
00:26 - null
...
00:32 - null
00:33 - null
00:34 - null
00:35 - null
...
00:40 - null
00:41 - null
00:42 - null
00:43 - null

This stems from the fact that the segment only uses the visit_first_action_time and visit_last_action_time, it does not know of the values in between. So we know when the visit started, when it ended, but the reporting API / Custom Report is not able to provide data which states that the visit was active at time (HH:MM) 00:00, 00:01, 00:02, ... .

I know we could solve this with a recursive CTE in SQL, but it seems we are pushing matomo in a direction that is not meant for. Or, we are doing something wrong with CustomReports.

Here are the Start and End reports we are using:

@mattab We have an ongoing e-mail discussion with you about this in email, if you believe this should be possible in Matomo, should we continue the discussion there ?

Be good to follow up in the email discussion. @toredash

So in this issue it's only about renaming the segment/dimension.

Be good to follow up in the email discussion. @toredash

So in this issue it's only about renaming the segment/dimension.

Yes, that is correct.

@tsteur PR open #15518

fixed in #15993

Thanks for this @toredash