Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

simplify archiving logic by implementing archiving layer as caching backend #7126

Closed
diosmosis opened this issue Feb 3, 2015 · 7 comments
Closed
Labels
Enhancement For new feature suggestions that enhance Matomo's capabilities or add a new report, new API etc. wontfix If you can reproduce this issue, please reopen the issue or create a new one describing it.
Milestone

Comments

@diosmosis
Copy link
Member

The logic that uses and manages archive tables should be simple. If an archive exists, use it. If it doesn't exist, get the report/metric data and cache it in an archive. This is what the logic should be like, but the reality is that the archiving code is a complex mess of object spaghetti.

Since the archive tables function essentially as a cache for report & metric data, we could simplify the code greatly by making the archive tables a Piwik Cache backend.

The resulting Archive.php code might look like:

function get($idSite, $period, $date, ...) {
    $cache = new ArchiveTableCache();
    if (!$cache->contains($idSite, $period, $date)) {
        $data = ...

        $cache->save($idSite, $period, $date, $data);
    }

    return $cache->fetch($idSite, $period, $date);
}

Possible problems:

  1. The caching layer would have to support fetching multiple pieces of data at once.
  2. The contains()/fetch() calls have to merged somehow. It would be inefficient to issue two almost identical queries for this.
@diosmosis diosmosis added the RFC Indicates the issue is a request for comments where the author is looking for feedback. label Feb 3, 2015
@tsteur
Copy link
Member

tsteur commented Feb 3, 2015

Everything that makes this part easier and understandable is greatly appreciated 👍

One more use case: If a cache entry is marked as to be invalidated it will be still valid until there is a new archive which shouldn't be a big problem to implement.

Challenge as you said performance but that sounds doable.

@mnapoli
Copy link
Contributor

mnapoli commented Feb 3, 2015

👍

@mattab
Copy link
Member

mattab commented Feb 9, 2015

seems most agree with this nice idea, maybe we can close RFC and create follow up issue for new cache backend etc?

@mattab mattab added this to the Piwik 2.11.0 milestone Feb 9, 2015
@diosmosis
Copy link
Member Author

I think it's a non-trivial, I doubt it can be done by 2.11.

@mattab mattab modified the milestones: Piwik 2.12.0, Piwik 2.11.0 Feb 9, 2015
@mattab
Copy link
Member

mattab commented Feb 9, 2015

Moved to 2.12.0, where we can close this RFC and create follow up issue eg. in Short term.

@diosmosis
Copy link
Member Author

Can't you just move this ticket to Short term and remove 'RFC'?

@mattab mattab added Enhancement For new feature suggestions that enhance Matomo's capabilities or add a new report, new API etc. and removed RFC Indicates the issue is a request for comments where the author is looking for feedback. labels Feb 10, 2015
@mattab mattab modified the milestones: Short term, Piwik 2.12.0 Feb 10, 2015
@mattab
Copy link
Member

mattab commented Feb 10, 2015

done

@mattab mattab modified the milestones: Short term, Mid term Apr 7, 2015
@mattab mattab closed this as completed Jul 8, 2016
@mattab mattab added the wontfix If you can reproduce this issue, please reopen the issue or create a new one describing it. label Jul 8, 2016
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Enhancement For new feature suggestions that enhance Matomo's capabilities or add a new report, new API etc. wontfix If you can reproduce this issue, please reopen the issue or create a new one describing it.
Projects
None yet
Development

No branches or pull requests

4 participants