Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

After upgrade all new segments are pre-processed #19153

Open
PeteTrombone opened this issue Apr 28, 2022 · 11 comments
Open

After upgrade all new segments are pre-processed #19153

PeteTrombone opened this issue Apr 28, 2022 · 11 comments
Labels
Enhancement For new feature suggestions that enhance Matomo's capabilities or add a new report, new API etc.

Comments

@PeteTrombone
Copy link

Since we upgraded to version 4.3.1 last July, all new segments are pre-processed. We have
no entry “enable_create_realtime_segments = 0” in config.ini.php.
According to the description the default should be processed in real time. But it is not.
An entry “enable_create_realtime_segments = 1” unfortunately changes nothing.
image

Our current version is 4.9.0

@PeteTrombone PeteTrombone added the Potential Bug Something that might be a bug, but needs validation and confirmation it can be reproduced. label Apr 28, 2022
@sgiehl
Copy link
Member

sgiehl commented Apr 28, 2022

Hi @PeteTrombone
Would you mind to explain what exactly your expected behavior would be?
Is it that the preselected value when creating a new segment is pre-processed even if real time segments are enabled?

@PeteTrombone
Copy link
Author

Before the upgrade, the default setting in the segment menu was "SEGMENTED REPORTS ARE PROCESSED IN REAL TIME".
After upgrade: "SEGMENTED REPORTS ARE PRE-PROCESSED".
Although we did'nt change anything in the settings as described above.

An entry in config.ini.php “enable_create_realtime_segments = 1” unfortunately changes nothing.

We don't want the default "SEGMENTED REPORTS ARE PRE-PROCESSED". However, it cannot be changed.

@sgiehl
Copy link
Member

sgiehl commented Apr 28, 2022

@PeteTrombone do you have browser archiving enabled?

@PeteTrombone
Copy link
Author

@sgiehl we have the following settings:
[General]
enable_segments_cache = 0
datatable_row_limits = "5,10,25,50,100,250,500,1000"
minimum_memory_limit_when_archiving = 9500
force_ssl = 1
scheduled_reports_truncate = 500
datatable_archiving_maximum_rows_actions = 1000
datatable_archiving_maximum_rows_subtable_actions = 500
assume_secure_protocol = 1
proxy_uri_header = 1

image

@sgiehl
Copy link
Member

sgiehl commented Apr 29, 2022

Ok. Looking at the code that seems clear now:

<option {% if isBrowserArchivingEnabled %}selected="1"{% endif%} value="0">{{ 'SegmentEditor_AutoArchiveRealTime'|translate }}</option>

The option is not pre-selected if browser archiving is disabled. I guess the reason for that is, that pre-processed segments should be preferred if an archiving cron is running.
The option to chose a real time segment should still be available, right?

What exactly is your problem with having segments preprocessed?

@PeteTrombone
Copy link
Author

@sgiehl
The option to chose a real time segment should still be available, right?
Yes, as default or as setting (enable_create_realtime_segments = 1)

What exactly is your problem with having segments preprocessed?
The preprocessing makes the archiving run very long. That takes hours.
We would like the option that we have an archiving cron but the segments are still processed in real time. As it was before version 4.3.1.

@sgiehl
Copy link
Member

sgiehl commented Jul 22, 2022

@PeteTrombone So would you need some sort of setting/config to fully disable pre-archived segments, or is it only the pre selection when creating segments that is bothering you?

@PeteTrombone
Copy link
Author

@sgiehl yes, we need a setting to fully disable pre-archived segments.
The pre selection when creating segments can only be made by admin role anyway. We do not assign this role to our customers.

@sgiehl sgiehl added this to the For Prioritization milestone Jul 25, 2022
@sgiehl
Copy link
Member

sgiehl commented Jul 25, 2022

You can simply skip archiving any segments by appending --skip-all-segment to the cron archive command.
But if new segments are created as pre-archived, that would prevent those segments from being archived at all.

@PeteTrombone
Copy link
Author

@sgiehl it doesn't seem clear yet what we want, so I'll describe it again.

  1. We use the archive cron job
  2. By default, all segments created by normal users (role view and write) should be processed in real time and not pre processed.
  3. The admins (only my team) should be able to create segments which are processed in real time or pre processed.

This is how it was before version 4.3.1
We track over 200 websites in one matomo instance and have over 500 segments.
I hope our requirement is clearer now.

We urgently need a solution here, as archiving takes too long if each segment is pre processed by default.

@sgiehl
Copy link
Member

sgiehl commented Jul 28, 2022

Thanks for the detailed description. I guess for now you won't have another possibility than adjusting the code yourself for now, so the default setting is changed.
Our product team will review this feature request and plan it in at some point. But I can't promise when that will be the case.

@sgiehl sgiehl added Enhancement For new feature suggestions that enhance Matomo's capabilities or add a new report, new API etc. and removed Potential Bug Something that might be a bug, but needs validation and confirmation it can be reproduced. labels Nov 13, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Enhancement For new feature suggestions that enhance Matomo's capabilities or add a new report, new API etc.
Projects
None yet
Development

No branches or pull requests

2 participants