Maybe there are Mysql based solutions (percona ? other engines?) that make adding INDEXes faster on multi-Gb databases?

Source: mysql performance blog

These tools are out there for years, yet it appears that still many MySQL users are not aware of them. Let me just name here pt-online-schema-change from Percona Toolkit, oak-online-alter-table from Openark Kit or Facebook OSC.

Maybe it would be worth considering alternative storage means, i.e. something that MySQL or RDBMS altogether (for large setups only). Just like Redis for the tracker is optional and meant for large setups.

That would mean a huge work on the codebase I can imagine, but if we can (painfully) get a 50% increase on the max volumetry with MySQL will that be enough? Do we need to scale much bigger, like ×10?

This issue here is not really about scaling further but rather investigate existing issues at current scale (and these issues will get worse / more frequent as we grow and as time goes by). Alternative storage means are discussed in #1999 with focus on mysql engines for now because implementing any other solution (such as a nosql DB) would take us 1+ year of development.

Edit: maybe some of these challenges above cannot be solved with Mysql alone and then we can re-assess... but hopefully we can find some smart solutions to some/most of these :-)

very interesting blog post Avoiding MySQL ALTER table downtime

A lot of progress has been made (i've edited the issue to cross out issues that are now done). We are continuing to work on performance and will do this one next: scale the Website Manager when there are thousands of websites (#3838)

Hey Matt, et all, not sure if this is the right place to post this, but I was wondering if there are any procedures for removing features or blocks of features from Piwik setup/DB?

I am running analytics engine that takes use of Piwik's Event Tracking engine. We are operating on websites with 3M+ uniques and are looking to grow. Since we aren't using all of the features of PW, perhaps there's a way to "lighten" the whole thing? Like a race car :)

hi @dmitrisgit, the official way is to disable some plugins from Piwik > Administration > Plugins, this will speed up the Tracker, Archiving, and possibly the UI as well.

Will now close this issue as we did a lot of work over the last two years and have solved most obvious issues. Piwik 3 will go even further with BIGINT fields to let users track by default more than 4 billion actions & visits.

We'll discuss further performance issues/ suggestions in separate issues. Check the tag c: Performance: https://github.com/piwik/piwik/labels/c%3A%20Performance