Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Monthly archive is huge #6115

Closed
ThaDafinser opened this issue Sep 3, 2014 · 6 comments
Closed

Monthly archive is huge #6115

ThaDafinser opened this issue Sep 3, 2014 · 6 comments
Labels
answered For when a question was asked and we referred to forum or answered it. Bug For errors / faults / flaws / inconsistencies etc. c: Performance For when we could improve the performance / speed of Matomo.

Comments

@ThaDafinser
Copy link
Contributor

Hello together,

i started with Piwik tracking about 2 months ago in my company.

But now the archive is exploding..._11GB blob archive for one month_?

Some notes:

  • _i'm having a huge numbers of actions_ > 100k (see screenshot)
  • i use a lot (different) custom visitor and action variables
  • i use segments

Some ideas?

archive_huge
segments

@ThaDafinser
Copy link
Contributor Author

If if see it correctly...all numbers are getting calculated on each action? That's why the archive is that huge?

So now i'm facing two problems:
I have on one side very dynamic page titles and partly dynamic urls (e.g. with ID).

I can replace the title or use something more generic this is an easy part. Since i only use the report for pages.

But for URI/URL/... is this also so easy?

_Is this for example the same action for piwik?_
/something/blubb?id=123
/something/blubb?id=456

_What is with this? Same page or different for piwik?_
/module/controller/action/id/123
/module/controller/action/id/456

Is it possible to adjust this when it's a "new" action?

@ThaDafinser
Copy link
Contributor Author

I found this: http://piwik.org/faq/how-to/#faq_81

But seems to be more difficult...so a custom solution seems to be handy.

E.g. adjust my tracker to exclude the dynamic parameters etc...

How can i reset all existing actions without clearing the database complete? Otherwise i'll have this size growing still in the long term?

@ThaDafinser
Copy link
Contributor Author

Here you see the count of each page.

4922 seems to be the "base" value (no click was tracked on that site)

countentries

@mattab mattab added the Bug label Sep 3, 2014
@mattab mattab added this to the Short term milestone Sep 3, 2014
@mattab
Copy link
Member

mattab commented Sep 3, 2014

Is it possible to adjust this when it's a "new" action?

See Javascript tracking guide - you can use setCustomUrl to override the URL, and you can use setDocumentTitle() to override the page title. Does it help?

@ThaDafinser
Copy link
Contributor Author

@mattab i hope so 😄

Since i only have 2 months of tracking until yet...do you think it would be better to start with a fresh database? I'm afraid that the already inserted >100k actions will make troubles in the future?

Doing now something like this at AJAX request tracking:

//use only everything before the "?"
var useUrl = completeUrlFromSomewhere;
useUrl = useUrl.split('?');
useUrl = useUrl[0];

//only use the first 3 parts
var parts = useUrl.split('/');
parts = parts.slice(1,4);
useUrl = '/' + parts.join('/');

_paq.push([ 'setCustomUrl', useUrl ]);
_paq.push([ 'setDocumentTitle', 'static title' ]);

@mattab mattab added the c: Performance For when we could improve the performance / speed of Matomo. label Sep 10, 2014
@ThaDafinser
Copy link
Contributor Author

@mattab i "reseted" now my database to start fresh out with tracking data.

What i did:

  • like posted above
    • remove the title to a generic
    • grouped a lot of actions together
  • removed segments down to 10

_my result_

  • actions are now down to 4k

only 10 days for now...
screen_piwik

_my final thoughts_
Like i already mentioned above, it should be easier to solved this issue. You have to know currently to much, to resolve such an issue and a normal user wont be able to do this.

@mattab mattab added the answered For when a question was asked and we referred to forum or answered it. label Oct 14, 2014
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
answered For when a question was asked and we referred to forum or answered it. Bug For errors / faults / flaws / inconsistencies etc. c: Performance For when we could improve the performance / speed of Matomo.
Projects
None yet
Development

No branches or pull requests

3 participants