Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

in multiple frontends setup, the cache file is not up to date on all frontends #6637

Closed
mattab opened this issue Nov 11, 2014 · 22 comments
Closed
Assignees
Labels
Bug For errors / faults / flaws / inconsistencies etc.
Milestone

Comments

@mattab
Copy link
Member

mattab commented Nov 11, 2014

The goal of this issue is that when piwik is served via several frontends and the file cache is invalidated (eg. when any of its values is updated or when piwik is upgraded), then the file cache should be deleted/refreshed on each of the frontend servers automatically.

For this maybe we can set a option flag in the db to store the minimum valid timestamp for file cache if file cache modified time is older we refresh the cache.

Thoughts?

@mattab mattab added the Bug For errors / faults / flaws / inconsistencies etc. label Nov 11, 2014
@mattab mattab added this to the Piwik 2.10.0 milestone Nov 11, 2014
@tsteur
Copy link
Member

tsteur commented Nov 11, 2014

Instead of file another backend should be used that is build for this. My number one on my wish todo list is to replace hard coded usage of file with https://github.com/doctrine/cache . Then people can just configure Redis or something else.

@mattab
Copy link
Member Author

mattab commented Nov 12, 2014

@tsteur can you link the doctrine/cache issue to this issue?

@tsteur
Copy link
Member

tsteur commented Nov 12, 2014

I don't think there is an issue for this. It is on my personal todo list that I want to see in Piwik. But I won't add an issue to use a technology when there is not a problem. So I was glad you mentioned a problem here :) I think so far we only talked about this probably 5 times in chats and meetups.

@tsteur
Copy link
Member

tsteur commented Nov 12, 2014

Config could look like this

[cache]
; redis, memcache, array, apc, ...
backend=file

Eg in development mode we would use an array (and in StaticCache) to prevent any actual caching which would remove some "hacks".

As one wants to use maybe multiple redis servers I am just not sure how this could be configured as I guess something like this is not possible in INI format:

[cache]
backend=redis
redis[] = { host: 54545, port: 3434}
redis[] = { host: 54545, port: 3434}
redis[] = { host: 54545, port: 3434}

Would have to be something like host:54545,port:3434 I guess. Many providers do not need a config at all.

But then we might later want to use eg Redis also for other stuff (eg queue #6075) so it could maybe look like this:

[cache]
backend=redis

[redis]
server[] = "host:54545,port:3434"
server[] = "host:54545,port:3434"

@mnapoli
Copy link
Contributor

mnapoli commented Nov 12, 2014

Or we could configure it using the container, that would be a perfect fit and that would be forward-compatible.

@tsteur
Copy link
Member

tsteur commented Nov 12, 2014

as long as people don't have to deal with any PHP code apart from maybe arrays yes

@tsteur tsteur assigned tsteur and unassigned tsteur Nov 13, 2014
@tsteur
Copy link
Member

tsteur commented Nov 16, 2014

Last week I was working on this for an hour or so and was afterwards thinking we should maybe only support Array (useful for development), File (same as current by default) and Redis (as it will be also used for #6075 and it would maybe also make sense to offer a Redis Session driver (easy doable)).

I was also preparing usage of Memcache (could be also used for Session) and APC as they are easy to integrate with a few lines but maybe using only Redis in the beginning should make things still simpler and easier to test.

BTW: Piwik-Dev-VM should already come with Redis and Memcache.

@mattab
Copy link
Member Author

mattab commented Nov 19, 2014

and APC as they are easy to integrate

Just a note regarding APC, I heard it will be deprecated in PHP 7 or so, so +1 for not implementing this one.

maybe using only Redis in the beginning should make things still simpler and easier to test.

sounds good!

@mnapoli
Copy link
Contributor

mnapoli commented Nov 30, 2014

Had a look at doctrine/cache, unfortunately this library doesn't have deleteAll() (or similar) method, which we need :(

Wonder how they did it in the ORM…

Had a look at other libraries, maybe https://github.com/illuminate/cache could do it. Or https://github.com/tedious/Stash (which is one of the guy working on the future cache PSR): this one has a more complex API though (like the future PSR), so would need an adapter in Piwik to simplify its usage. I guess until there's a PSR there would be an adapter anyway because we don't want to be coupled to another library.

@tsteur
Copy link
Member

tsteur commented Nov 30, 2014

@mnapoli
Copy link
Contributor

mnapoli commented Nov 30, 2014

Yep indeed, don't know how I missed that… x_x Too bad it's not in the interface but it doesn't matter for us.

@sgiehl
Copy link
Member

sgiehl commented Nov 30, 2014

Please ping me if we are going to use an external library for caching like doctrine-cache. I'm currently considering to use a library for DeviceDetector cache aswell, but haven't put any deeper thoughts on that by now.

@mattab
Copy link
Member Author

mattab commented Dec 1, 2014

@sgiehl yes we'd like to use doctrine-cache - we're waiting for @tsteur to get to it I think ;-)

@sgiehl
Copy link
Member

sgiehl commented Dec 1, 2014

Ok. I'll put that on my list for DD then aswell.

@tsteur tsteur self-assigned this Dec 5, 2014
tsteur added a commit that referenced this issue Dec 9, 2014
…sting many different changes so only want to take a snapshot of the current state.
@tsteur
Copy link
Member

tsteur commented Dec 10, 2014

Might use stash... Worked with doctrine cache for a while but the architecture is not really so nice and I'd kinda end up building a caching library. Laravel one requires PHP 5.4 so won't work.

From what I see so far Stash is a bit over engineered and might be a bit slow which is no good as the cache will be used VERY often during a normal request. Also we probably won't be able to allow devs to write their own drivers unless we wrap a lot of logic since we do not want to be dependent on this library. Will have a further look...

@mnapoli
Copy link
Contributor

mnapoli commented Dec 10, 2014

Hopefully in a few months the caching PSR will be finally released… ;) That would simplify things a bit.

@tsteur
Copy link
Member

tsteur commented Dec 15, 2014

tsteur added a commit that referenced this issue Dec 16, 2014
…sting many different changes so only want to take a snapshot of the current state.
@mattab
Copy link
Member Author

mattab commented Dec 16, 2014

Hey @tsteur How is this issue looking for 2.10.0 ?

@tsteur
Copy link
Member

tsteur commented Dec 16, 2014

Gonna do some final tests re speed and whether it actually works. Otherwise it is done. Not sure whether we should directly include it in Piwik 2.10.0 as caching was often not tested when a class used caching.

@mattab
Copy link
Member Author

mattab commented Dec 17, 2014

Let's include it as better release early and often. I'll deploy it on demo now.

@tsteur
Copy link
Member

tsteur commented Dec 17, 2014

I will close it probably tmrw then unless we directly identify an issue

@tsteur
Copy link
Member

tsteur commented Dec 17, 2014

Closing it now. The issue I expected was reported in #6873 :)

@tsteur tsteur closed this as completed Dec 17, 2014
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Bug For errors / faults / flaws / inconsistencies etc.
Projects
None yet
Development

No branches or pull requests

4 participants