Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

GDPR: data subject search can take a long time (5min or more) #12837

Open
mattab opened this issue May 7, 2018 · 2 comments
Open

GDPR: data subject search can take a long time (5min or more) #12837

mattab opened this issue May 7, 2018 · 2 comments
Labels
c: Performance For when we could improve the performance / speed of Matomo. c: Privacy For issues that impact or improve the privacy.
Milestone
Backlog (Help wan...

Comments

@mattab
Copy link
Member

mattab commented May 7, 2018

When exporting data subjects' data, using for example the filter "where User ID is xyz", on a reasonnably sized Matomo instance, the data subject search can take 5min or more. This has several issues: the requests can time out, users think it does not work, etc.

-> How could we improve this situation?

(For example in some cases the data subject search may take 10min or 1hour. Other tools out there will "Schedule" the data subject search, rather than do it in real time like we do. )
@mattab mattab added c: Performance For when we could improve the performance / speed of Matomo. c: Privacy For issues that impact or improve the privacy. labels May 7, 2018
@mattab
Copy link
Member Author

mattab commented May 8, 2018

We currently have a system of logs of tasks when anonymising the historical raw data.
Maybe we could reuse the same system and have a queue of data subject exports and a link to the download the Json/html files?

log

@mattab mattab added this to the 3.6.0 milestone May 8, 2018
@tsteur tsteur changed the title GDPR: data subject export can take a long time (5min or more) GDPR: data subject search can take a long time (5min or more) May 18, 2018
@tsteur
Copy link
Member

tsteur commented May 18, 2018

Changed the title as export is not slow but the search.

One thing to do be to only show visit segments here as any action segment etc can make things VERY slow considering it goes on all data... Also to identify a subject, only visit segments should be interesting anyway.

@mattab mattab modified the milestones: 3.6.0, 3.7.0 Jun 28, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
c: Performance For when we could improve the performance / speed of Matomo. c: Privacy For issues that impact or improve the privacy.
Projects
None yet
Milestone
Backlog (Help wanted)
Development

No branches or pull requests
2 participants
@tsteur @mattab