Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

http archive request should pre-process listed segments in reasonnable fashion #3658

Closed
anonymous-matomo-user opened this issue Jan 10, 2013 · 4 comments
Labels
Task Indicates an issue is neither a feature nor a bug and it's purely a "technical" change. worksforme The issue cannot be reproduced and things work as intended.
Milestone

Comments

@anonymous-matomo-user
Copy link

In few words :
the archive.php script does implement // request for segment, but does not implement any queue mechanism which makes it unusable with many segments.

Long story : http://forum.piwik.org/read.php?3,97712
(with background cli task version being much much faster)

Current proposal http only :

"the logic used for queue is trivial : add x item to queue, then throw all requests and grab the result. This could be "optimized" to add x item to queue, then throw all queued requests and constantly check for result and add new requests as we remove them.
I quote optimize because I also think that this is debatable. Since the archiving server is likely to be the tracking server itself, your probably do not want to overload it. Current option goes in that direction since the average // request number will be lower than the maximum set and there will be tiny breaks to allow the rest of the server tasks to go on.
This of course only mean something with segment (the more segment, the more meaning) since otherwise queuing just does not occur at all (one single request at a time). "

@anonymous-matomo-user
Copy link
Author

Attachment:
http_only.diff

@anonymous-matomo-user
Copy link
Author

Unfortunately, using // resulted in italic font instead of a short hand for parallel,

Please read :
"the archive.php script does implement parallel requests for segment"
instead of :
"the archive.php script does implement // requests for segment"

@mattab
Copy link
Member

mattab commented Jan 21, 2013

Thanks for the patch, I'll investigate at some point!

@mattab
Copy link
Member

mattab commented Oct 25, 2013

It would be great if you could submit a pull request if this code is still relevant. Thank you so much for your support!

@anonymous-matomo-user anonymous-matomo-user added this to the 2.x - The Great Piwik 2.x Backlog milestone Jul 8, 2014
This issue was closed.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Task Indicates an issue is neither a feature nor a bug and it's purely a "technical" change. worksforme The issue cannot be reproduced and things work as intended.
Projects
None yet
Development

No branches or pull requests

2 participants