Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Ability to easily change database engine #10583

Open
udomsak opened this issue Sep 27, 2016 · 4 comments
Open

Ability to easily change database engine #10583

udomsak opened this issue Sep 27, 2016 · 4 comments
Labels
c: Performance For when we could improve the performance / speed of Matomo.
Milestone
Backlog (Help wan...

Comments

@udomsak
Copy link

udomsak commented Sep 27, 2016

Problem

After few day on lab with how to archive high speed and volume with Piwik. I have many DB choice to run

  • MariaDB Cassandra
  • MariaDB Spider Storage
  • TokuDB

Both Cassandra and Spider Storage engine are problem with schema table that difficult to change although you can do it manually database shell. But when you running with Piwik can cause Piwik error and stop to work.

  • datetime data-type from Piwik not compatible with time data type of Cassandra.
  • When you change schema on Database can cause Piwik stop this issue are effect with both ( Cassandra and Spider storage database engine )

Speed

  • Cassandra work well.
  • Spider storage engine work great.

Goal

  • Handy and easy to modify Piwik database schema at lease change Table storage engine.
  • This can help when you deal with traffic over 100 million stat per week. or at lease 5Million per day.
  • HHVM does not help cause most work load ( 90% ) on 'read' operation such as report or statistic on time you select.

    Other sugestion

  • Please change database driver to support Asynchronous too.

From my mind. :)

  • Piwik is a great software and fast ( for small traffic ) with a lot of custom tracking feature but it hard to modify back-end as need.

Thank you for greate software building. 👍
@udomsak udomsak changed the title Vote I need database engine can do free from use with no issue. Vote I need database engine can change with no issue. Sep 27, 2016
@gaumondp
Copy link

Can you share your experience with Toku ? I've been interested in that one for a long time but never had time and resources to do speed testing. What kills the Piwik experience here is speed when doing "Date range" and "Segmentation based on few IP exclusions".

I only have 100 millions row (about 35 GB) for 3 years of data and it takes 30 minutes to get a report using date range+IP exclusion.

@udomsak
Copy link
Author

udomsak commented Sep 28, 2016

On testing @gaumondp :) I just put on staging last night. Will share you later, sure. :)

@mattab mattab added the c: Performance For when we could improve the performance / speed of Matomo. label Nov 11, 2016
@mattab mattab added this to the Mid term milestone Nov 11, 2016
@mattab mattab changed the title Vote I need database engine can change with no issue. Ability to easily change database engine Feb 20, 2017
@DeepakkumarArumugam
Copy link

Hi,
Do we have any update on supporting different database schema like maria db with Matamo?
Any help will be highly appreciated.
Thanks

@tsteur
Copy link
Member

tsteur commented Jan 31, 2018

Maria DB with Matomo should work and there is nothing needed to be changed 👍

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
c: Performance For when we could improve the performance / speed of Matomo.
Projects
None yet
Milestone
Backlog (Help wanted)
Development

No branches or pull requests
5 participants
@tsteur @udomsak @mattab @gaumondp @DeepakkumarArumugam