Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cache result of "isIpInRanges" for faster tracking in Bulk / Queue #7757

Merged
merged 1 commit into from Apr 24, 2015

Conversation

tsteur
Copy link
Member

@tsteur tsteur commented Apr 24, 2015

See screenshot of a profile of a ./console queuedtracking:process run. The time spent in $ip->isInRanges() is about 10% time of the total time. That's the same amount of time needed to recognize all visitors (and that queries the database etc).

Background: When using eg queuedtracking:process we sometimes insert 25k tracking requests at once or maybe even more. In the attached profile we processed 16.000 tracking requests so this method was called 16.000 times (it is used in VisitExcluded). It took 10% of total time or 27 seconds. When we process so many tracking requests we have very likely the same IP given multiple times which it makes it worth caching. All actions of the same visitor are inserted in the same queue therefore it is likely that the same IP is given multiple times. This means when there are eg 8k different visitors of those 16k tracking requests we gain 50% performance improvement. If there are 4k different visitors we gain a 75% percent improvement.

isipinrange

@tsteur tsteur added c: Performance For when we could improve the performance / speed of Matomo. Needs Review PRs that need a code review labels Apr 24, 2015
@tsteur tsteur added this to the Piwik 2.13.0 milestone Apr 24, 2015
mattab pushed a commit that referenced this pull request Apr 24, 2015
Cache result of "isIpInRanges" for faster tracking in Bulk / Queue
@mattab mattab merged commit c87806d into master Apr 24, 2015
@mnapoli mnapoli deleted the cached_iprange branch April 24, 2015 05:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
c: Performance For when we could improve the performance / speed of Matomo. Needs Review PRs that need a code review
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants