Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve process of adding new custom variables #8555

Closed
quba opened this issue Aug 12, 2015 · 3 comments
Closed

Improve process of adding new custom variables #8555

quba opened this issue Aug 12, 2015 · 3 comments
Assignees
Labels
Enhancement For new feature suggestions that enhance Matomo's capabilities or add a new report, new API etc. not-in-changelog For issues or pull requests that should not be included in our release changelog on matomo.org.
Milestone

Comments

@quba
Copy link
Contributor

quba commented Aug 12, 2015

Currently it works so that we have 2 commands - one to add/delete custom variables and one to check if everything is OK. My suggestion is to improve the process of adding new custom variables.

Use case
Currently adding new custom vars means more or less iterating log_* tables. If there are discrepancies between tables (in number of custom vars), Piwik interface displays errors and archiving is not working. The only good thing is fact that tracker still works.

When this can happen
When there's a really big database and it takes around 24 hours to alter log_link_visit_action table

Why this is important?
We want to assure zero downtime (archiving, interface). It is doable, especially given that there are HOT alters in MariaDB 10+ (so we don't have to disable DB replication to do this).

Proposed solution
Let's make Piwik a little bit more resistant:

  • let's confirm that the order of alters is proper (I'd start with the biggest table - usable especially when there's ability to run a "hot alter") - to be honest I don't know what's the current order,
  • let's change Piwik's logic so that Piwik will always take the minimum number of custom variables from all log tables (log_visit, log_link_visit_action, log_conversion). Therefore everything will work even if some alters are still in progress.

https://github.com/piwik/piwik/blob/master/plugins/CustomVariables/CustomVariables.php#L79
I'd start with 1 here (because that's the minimum value - one may just want to change the number of custom variables from 5 to 1) and then check what's the minimum number of columns in all 3 tables.

@MagicFab MagicFab added the Enhancement For new feature suggestions that enhance Matomo's capabilities or add a new report, new API etc. label Aug 13, 2015
@mattab
Copy link
Member

mattab commented Aug 13, 2015

Targetting to 2.15.0 as if we can improve this now, it will be awesome to have in LTS

@mattab mattab modified the milestones: 2.15.0, 2.15.1 Aug 13, 2015
@tsteur
Copy link
Member

tsteur commented Aug 29, 2015

current order is:

  • 'log_link_visit_action'
  • 'log_visit'
  • 'log_conversion'

@tsteur tsteur self-assigned this Aug 29, 2015
@tsteur tsteur modified the milestones: 2.15.0, 2.15.1 Aug 29, 2015
diosmosis added a commit that referenced this issue Aug 30, 2015
Fixes #8555, always use the minimum number of custom variables from all log tables instead of maximum so Piwik won't fail when tables are in inconsistent state.
@mattab
Copy link
Member

mattab commented Aug 31, 2015

Nice fix, and cheers @quba for reporting the issue

@mattab mattab added the not-in-changelog For issues or pull requests that should not be included in our release changelog on matomo.org. label Oct 13, 2015
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Enhancement For new feature suggestions that enhance Matomo's capabilities or add a new report, new API etc. not-in-changelog For issues or pull requests that should not be included in our release changelog on matomo.org.
Projects
None yet
Development

No branches or pull requests

4 participants