Maybe related to #14664 ?

Hi @heurteph-ei, segments that work on actions (including custom action dimensions), don't select only the actions that match the segment, but visits that have at least one action that matches the segment. Can you check whether the visits in the visitor log in this segment all have the action? If there are visits that don't have an action that matches, then there would be an issue.

Yes, I understand this point of view, BUT in this case, the segmented visit log:

should show this fact. Instead of, I got no result:

Note: you are right, the dimension value changed duting the single visit and took all values that are displayed in the widget, and no other values

Hi, @heurteph-ei can you provide the exact steps you got to display the segmented visitor log like that? Are you able to reproduce this on https://demo.matomo.cloud ? If not, can you check if the same issue occurs on the latest Matomo version?

Hi @diosmosis
Unfortunately, I can't test on a local latest version of Matomo. Also, I cannot add new segment to matomo.cloud, maybe you could do it for me?
For example, please add a segment on Page Author == 'Git-17388', then I'll be able to do a tracking on a page to reproduce the use case.

Also, I tried several times to be tracked on divezone.net in order to appear on https://demo.matomo.cloud, but it never works. I even tried on my own computer, and on several browsers (Chrome, Edge, Firefox).
Note: on Firefox, I cannot disable the HTTP parameter DNT:1

Hi @heurteph-ei, demo.matomo.cloud is a special instance that uses test data to showcase matomo features. It's not directly connected to divezone.net. I can add a segment for the page author today. Would you be able to post the tracking request(s) you'd use here in the meantime? It might help in reproducing locally.

@diosmosis okay, I have pushed some tracking on matomo.cloud...
But the Page Author behaviour seems to encounter some problems... I'll wait untill tomorrow in order to check it the server needs time to update its data...

OK, my data are well stored:

@diosmosis can you add the segment Page Author == 17388?

Anyway, on this sample, I enconter some strange behaviour:

Prerequisite

Behaviour > Page Author
Date: 31st of March, 2021 (day)
Segment: All visits

Step 1

Add segment comparison: Returning users
OK, the data seem good (as my tests were not done as returning users)

Step 2

Open the Segmented visits log of Page Author D2 for Segment Returning users. I expect getting no result, but this is not the case:

Also note that all these logs are not related to my "visit" at all, as I was just "tracked" via the HTTP API.

Apologies for the delay @heurteph-ei, I just added the segment: https://demo.matomo.cloud/index.php?module=CoreHome&action=index&idSite=1&period=day&date=yesterday#?idSite=1&period=day&date=yesterday&segment=dimension2%3D%3DGit-17388&category=Dashboard_Dashboard&subcategory=1

Thanks for providing the extra details! We'll take a look after the weekend.

No worries :-)

Visits log

The Visitors > Visits Log works as expected on the new segment: The log displays the whole session in which the Page Author == Git-17388.

• As soon as the user hits Page Author == Git-17388 during his session : I can see all events of the session.
• If a user does not hit Page Author == Git-17388 during his session : The session is not visible at all.

Page Author behaviour

Now I have to wait a few time for the new segment to be processed by the system for Page Author ;-)
I won't be available until April, the 6th. So I'll continue my test on Tuesday :-)

Enjoy your weekend!

Hello, I'm back
So the behaviour I encounter on premises the same as in the cloud.

Please have a look at: https://demo.matomo.cloud/index.php?module=CoreHome&action=index&idSite=1&period=day&date=yesterday#?idSite=1&period=day&date=2021-03-31&segment=dimension2%3D%3DGit-17388&category=General_Actions&subcategory=customdimension2

The table displays all the author values I encountered during the visit in which I saw the famous Page Author Git-17388. I don't know if it as an expected behaviour of Matomo, or if we expect to have only one single row: Git-17388.

But as the table displays 3 actions for Page Author Julien, when I click on the segmented visit log of Julien row, I should get some data instead of nothing:

Thanks @heurteph-ei for going through all the trouble of reproducing it on demo.matomo.cloud! It's certainly a bug. Seems to have to do with segments like dimension2==Git-17388;dimension2==another-author. I think it should be interpreted as "select visits that contain an action w/ dimension2 == Git-17388 AND an action w/ dimension2 == another-author", but it's clearly not doing anything.

Dear @diosmosis I invested several hours on this bug to find a solution. First of all I can confirm it is a bug.

It was introduced with 48c9f2c which was despite this bug an awesome work by @tsteur 👍 🥇

The LogAggregator is using a temporary table to speed up calculation. Whilst this is super efficient it does not respect action based dimensions.

This is what the LogAggregator does:

• retrieve all visit ids where there was at least one action matching the segment containing the action based dimension
• Filter to only these visits when querying the results from the log_link_visit_action table.
• The result contains not only actions matching the customDimension but instead contains ALL actions of these visits

Suggestion

While the use of a temporary table is very sufficient for visit based segments it might not be as sufficient for segments containing action based segments.
Filtering the log_link_visit_action table with custom_dimension_x = ? might reduce the resultset in the same way as log_link_visit_action.idvisit in tempTable.idvisit (with a join) does currenlty.
Do you agree with that @tsteur ?

Therefore I would suggest to not use a temporary table when the segment contains an action based dimension.

Proposed Code Change

Currently

Suggested Change:

    private function segmentContainsActionBasedDimensions()
    {
        if (strpos($this->segment->getString(), 'dimension') < 0) {
            // containing no dimension at all
            return false;
        }

        $segmentExpressionSql = $this->segment->getSegmentExpression()->getSql();
        // the sql will contain the log_link_visit_action table name if the dimension is action based
        return strpos($segmentExpressionSql['where'], 'log_link_visit_action.') >= 0;
    }

    public function generateQuery($select, $from, $where, $groupBy, $orderBy, $limit = 0, $offset = 0)
    {
        $segment = $this->segment;
        $bind = $this->getGeneralQueryBindParams();

        // if segment is based on a custom dimension which is action based do not use TMP table
        if (!$this->segment->isEmpty() && $this->isSegmentCacheEnabled() && !$this->segmentContainsActionBasedDimensions()) {

If anybody uses this code on her website: make sure to invalidate all reports, since the results get archived/cached.

I am willing to provide a PR, but was struggling to create a proper Unit Test.
My company would pay someone to get a fix into Matomo Core, so please leave a comment if you are interested

Thank you for your time reading this comment 😄 ❤️

@sgotre if I understand it right you are maybe after this feature: #15438

from my perspective it works correct, also for actions just like segmentations should. The behaviour you are maybe after are "Filters": #15438

Existing workarounds for previous behaviour would be to set [General] enable_segments_cache = 0 in the config file or to use our Custom Reports feature.

Let me know if that's not what you are after regarding the Filters feature.

@tsteur thanks for your super fast reply. You are right, we all have been trying to use segments as filters.

Disabling the segment cache has a huge performance impact, right? With about 100 Mio hits per month this might not be the best solution.

We will use custom reports instead.

For all wondering why @tsteur is right: Segments are by definition visit based. Filters are by definition action based.

Segment – For a segment, every visit is checked to see if it satisfies the conditions of the segment. For sessions that satisfy the condition, all rows are returned. For sessions that do not, no rows are returned.

Filter – For a filter all the rows for ALL visits are considered, then only the rows that satisfy the conditions of the filter will be returned.

Source

I think you can close this issue in favor of #15438
@tsteur this bug is not a duplicate, as it originally reported another bug as i was assuming. See this comment for root cause description of this reported UI issue

@sgotre I think this cannot be closed as the segment on 2 action dimensions doesn't work (I am not in the configuration of a filter)

@heurteph-ei you can set the global config [General] enable_segments_cache = 0 to make segments work like filters. The behaviour we are seeing here ist epected by design: **Segments are by definition visit based. **

@sgotre I don't want to use filter but segment.
See #17388 (comment)
If you segment 2 values on an action dimension, the segment doesn't work.
In the example, I wanted all the visits where users set dimension2 to Git-17388 during a part of its visit, and set dimension2 to Julien during another part of its visit...

@heurteph-ei You are right, segmentation works as you expected but its not possible to show the segmented visit logs. seems like the query here is using segments as filters. I'll check the sql to see where this error occurs.

@heurteph-ei this is NOT a SQL issue but an UI issue.

When requesting the segmented visit logs the URL is

https://demo.matomo.cloud/index.php?segment=dimension2==Git-17388;dimension2==D2;actionUrl=$divezone.net%2Fheurteph%2Ftest&date=2021-03-31&module=Live&action=indexVisitorLog&disableLink=1&small=1&enableAddNewSegment=1&period=day&idDimension=2&idSubtable=4&idSite=1

Please notice segment=dimension2==Git-17388;dimension2==D2;
=> resulting query is: "give me all visits having actions where action.dimension2 is Git-17388 AND action.dimension2 is D2'
=> result is of course empty

Query Executed:

SELECT log_visit.*
FROM matomo_log_visit AS log_visit
         LEFT JOIN matomo_log_link_visit_action AS log_link_visit_action
                   ON log_link_visit_action.idvisit = log_visit.idvisit
WHERE (log_visit.idsite in (?) AND log_visit.visit_last_action_time >= ? AND log_visit.visit_last_action_time <= ?)
  AND 
    (log_link_visit_action.custom_dimension_3 = ? 
      AND log_link_visit_action.custom_dimension_3 = ?  /** <---- this is not intended*/
    AND
       ((log_link_visit_action.idaction_url IN
         (SELECT idaction FROM matomo_log_action WHERE (name LIKE CONCAT('%', ?) AND type = 1))) OR
        (log_link_visit_action.idaction_url IN
         (SELECT idaction FROM matomo_log_action WHERE (name LIKE CONCAT('%', ?) AND type = 3))) OR
        (log_link_visit_action.idaction_url IN
         (SELECT idaction FROM matomo_log_action WHERE (name LIKE CONCAT('%', ?) AND type = 2))) OR
        (log_link_visit_action.idaction_url IN
         (SELECT idaction FROM matomo_log_action WHERE (name LIKE CONCAT('%', ?) AND type = 10)))))
GROUP BY log_visit.idvisit
ORDER BY log_visit.idsite DESC, log_visit.visit_last_action_time DESC
LIMIT 0, 12

This would be correct if segment where filters.

Correct URL

Instead the URL should contain this segment segment=dimension2==Git-17388;dimension2==Git-17388,dimension2==D2

=> resulting query: "give me all visits that had action.dimension2=Git-17388 AND (dimension2=Git 17388 OR dimension2==D2)

Query Executed:

SELECT log_visit.*
FROM matomo_log_visit AS log_visit
         LEFT JOIN matomo_log_link_visit_action AS log_link_visit_action
                   ON log_link_visit_action.idvisit = log_visit.idvisit
WHERE (log_visit.idsite in (?) AND log_visit.visit_last_action_time >= ? AND log_visit.visit_last_action_time <= ?)
  AND (log_link_visit_action.custom_dimension_3 = ? AND
       (log_link_visit_action.custom_dimension_3 = ? OR log_link_visit_action.custom_dimension_3 = ?) /** <-- thats it */
    AND
       ((log_link_visit_action.idaction_url IN
         (SELECT idaction FROM matomo_log_action WHERE (name LIKE CONCAT('%', ?) AND type = 1))) OR
        (log_link_visit_action.idaction_url IN
         (SELECT idaction FROM matomo_log_action WHERE (name LIKE CONCAT('%', ?) AND type = 3))) OR
        (log_link_visit_action.idaction_url IN
         (SELECT idaction FROM matomo_log_action WHERE (name LIKE CONCAT('%', ?) AND type = 2))) OR
        (log_link_visit_action.idaction_url IN
         (SELECT idaction FROM matomo_log_action WHERE (name LIKE CONCAT('%', ?) AND type = 10)))))
GROUP BY log_visit.idvisit
ORDER BY log_visit.idsite DESC, log_visit.visit_last_action_time DESC
LIMIT 0, 12;

When the global option enable_segments_cache is 1 the URL is correct (but it will never be called in this way since such a result cannot appear in a report)
If the segment cache is enabled the generated segment ist wrong and the segment part of this URL should be changed