Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add setting to customize the time-limited seed of config_id hash #19913

Closed
sandin78 opened this issue Oct 26, 2022 · 7 comments
Closed

Add setting to customize the time-limited seed of config_id hash #19913

sandin78 opened this issue Oct 26, 2022 · 7 comments
Labels
answered For when a question was asked and we referred to forum or answered it. Enhancement For new feature suggestions that enhance Matomo's capabilities or add a new report, new API etc.

Comments

@sandin78
Copy link

Summary

When user_id or cookies are not available Matomo falls back on using config_id to identify, as far as it can be done, unique browsers. When reports are archived, the parameter window_look_back_for_visitor limits how much of the history is searched to determine if the visitor is returning or new, but this setting can be modified to longer time periods.
However, the config_id is not persistent over long time.

From https://matomo.org/faq/general/how-is-the-visitor-config_id-processed/
the config_id is only valid for 24 hours maximum and is then rotated, meaning the same visitor will have a different config_id each day.

If the time period of the rotation could be set in config, or even turned off, the unique visitors would not be limited to daily reports.
This would be specially helpful for tracking returning visitors coming via iframes etc, or where there are technical reasons why cookies are unavailable.

@sandin78 sandin78 added the Enhancement For new feature suggestions that enhance Matomo's capabilities or add a new report, new API etc. label Oct 26, 2022
@bx80
Copy link
Contributor

bx80 commented Oct 26, 2022

Hi @sandin78, it would be nice to be able to look back further for config_id matches, but it seems like there may be issues with daily archiving accuracy if the config_id doesn't rotate every 24hrs.

Disabling the create_new_visit_after_midnight = 1 config setting would prevent the use of a 24hr finger print salt, but the config setting comes with the warning ONLY CHANGE THIS VALUE WHEN YOU DO NOT USE MATOMO ARCHIVING, SINCE THIS COULD CAUSE PARTIALLY MISSING ARCHIVE DATA so clearly isn't recommended.

@tsteur, @sgiehl Do you have any insight into why there is a 24hr maximum for the config_id?

@bx80 bx80 added this to the For Prioritization milestone Oct 26, 2022
@tsteur
Copy link
Member

tsteur commented Oct 27, 2022

It was restricted to 24 hours for privacy reasons. Usually, cookies or some similar technology can be used to track users over a long period and if they can't or shouldn't be used then we restrict to 24 hours as otherwise you could see the config_id as fingerprinting.

An option to change this behaviour could be possible though 👍

@sandin78
Copy link
Author

Yes. I understand the benefit of having this restriction and the selling point for the Matomo user is "i can't change it even if I wanted to". I guess there's a whole bunch of Matomo settings that helps the Matomo admin to comply with various privacy laws, but they can be configured in a non-compliant way. So it's up to the Matomo admin to comply, and I think it would be the same regarding this setting. Default should be 24h.

@sandin78
Copy link
Author

Disabling the create_new_visit_after_midnight = 1 config setting would prevent the use of a 24hr finger print salt, but the config setting comes with the warning ONLY CHANGE THIS VALUE WHEN YOU DO NOT USE MATOMO ARCHIVING, SINCE THIS COULD CAUSE PARTIALLY MISSING ARCHIVE DATA so clearly isn't recommended.

Hm, obviously I don't know why this could be, but it would be great to get some insight.
Is it because otherwise an ongoing visit would not be attributed to the new date? I mean if the visit started on 23:50 and continued over midnight.

If I were to test setting create_new_visit_after_midnight = 0 , this would mean that the limit to finding config_id has would be truly limited by the window_look_back_for_visitor which was what I originally had hoped.

@sandin78
Copy link
Author

Okay I dug a little and found https://github.com/matomo-org/matomo/issues/9156
In fact it seems that actions logged after midnight, from a visit that began before midnight, will be lost. I assume this is because the actions would belong to the visit but the report is already archived for the previous day and will not update.

I guess the way the archiving is designed, it runs each day separately. I mean of course there is no limit for how long visits could last, but it would be possible to regenerate day1 if finding actions on day2 belonging to day1.
But of course, this opens more problems, for instance if day2 is in a new week, or a new month.

I will test this. It will be interesting to see if the actions of each day add up to the weekly :).
Not sure if this setting is the way forward, but atleast I'm thankful that you opted in the end to have this as config, and not default for all users. I will have to do a test and then weigh the pros and cons.

But I think that this issue could be closed! There's no need to set a limit for this when the limit can already be removed and timelimited then by another setting.

@bx80
Copy link
Contributor

bx80 commented Oct 27, 2022

Thanks for posting your findings @sandin78, this information might be useful for other people wanting to try the same thing and can help inform any future improvements made to this part of Matomo 👍

@bx80 bx80 closed this as completed Oct 27, 2022
@MatomoForumNotifications

This issue has been mentioned on Matomo forums. There might be relevant details there:

https://forum.matomo.org/t/change-config-id-settings-possible/48338/2

@elabuwa elabuwa added the answered For when a question was asked and we referred to forum or answered it. label Nov 22, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
answered For when a question was asked and we referred to forum or answered it. Enhancement For new feature suggestions that enhance Matomo's capabilities or add a new report, new API etc.
Projects
None yet
Development

No branches or pull requests

5 participants