@dev-101 opened this Issue on February 6th 2019

Well, let's try this once again (previously in #13327). Old issue now closed is still very much actual in my case.

I haven't investigated this much, but it is totally annoying and unreliable. It doesn't work, really.

Steps to "reproduce" this issue:

  1. run Chrome
  2. access website/matomo/
  3. enter log-in credentials, check 'remember me', submit login form
  4. once logged-in, close Chrome
  5. repeat from step (1) above - try to access website/matomo/ again, and ... log-in form is presented once again !?

I run Matomo on VPS and cPanel shared hosting plans, and it happens on both of them, and once-in-a-while Matomo decides to finally remember me, so it works for a while until I clear App Storage in Chrome or history + cookies.

What is happening here? I know you switched away from cookies, but sessions in:

/var/lib/php/sessions

files are all with 0 length.

Ok, so, Piwik/Matomo [maybe] stores sessions locally in its own folder (?):

website/matomo/tmp/sessions/

Ok, there we go. Now, here's the funny thing - if I delete those temp sessions in matomo's local tmp folder, I am still logged-in. So, they aren't used for that, huh??

Also, how / when are they being created in the first place? When I log-in, I don't see new session file stored in there. Also, php's default sessions files are not related with matomo's log-in, that makes sense already (read above).

So, it is stored in browser (where else could it possible be?)? Then, why this happens? Why Matomo keeps 'forgetting' me?

How can we fix this once and for all?

(please do not close this issue too early)

Thanks!

@Findus23 commented on February 6th 2019 Member

Hi,

I know you switched away from cookies

Sessions still need cookies, otherwise the server can't know which session belongs to whom.

So, they aren't used for that, huh??

Since Matomo 3.8.0 sessions are stored in the database by default. (session table)

Why Matomo keeps 'forgetting' me?

Just to make sure: You haven't configured Chrome to delete cookies on closing?

@dev-101 commented on February 6th 2019

Hi Findus23,

Thanks for clarifying at least some things.
No, as far as I know Chrome keeps a lot of cookies for me (I need them for other sites!).

@dev-101 commented on February 6th 2019

"Keep local data only until you quit your browser" option is OFF

@dev-101 commented on February 6th 2019

Since this installations are updates of older ones, could it be that something went wrong with database upgrade? How can I check this in database and make sure everything is all right?

@dev-101 commented on February 6th 2019

Also, I remember from the old days that Piwik/Matomo offered storing sessions in database, and that never worked properly for me (I think I even mentioned that in one of the older issues about Piwik's sessions here).

Is there something we should put in config.ini.php ?

@dev-101 commented on February 6th 2019

Ok, I accessed database, I assume it's the _session table.

I found 54 rows, deleted them all, obviously website/matomo/ log-in form was shown, I logged-in with 'Remember me' option checked... []and no session was stored in the database! I am confused, maybe it's different table.[] Sorry, I switched to _access table by accident and haven't noticed (I need to go, I'm already late and in hurry). Session is there.

@fdellwing commented on February 6th 2019 Contributor

Do you run Matomo behind a reverse proxy?

@dev-101 commented on February 6th 2019

Nope, straight apache on VPS, not sure about shared hosting but front server is LiteSpeed.

@dev-101 commented on February 6th 2019

OK, it is definitely related to cookies, and now I suspect the mechanism how it happens (still just a theory as I don't closely follow Matomo's source code).

First of all, it is definitely not Chrome-related issue, just tested on my ancient computer where I keep old Chrome just for this kind of issues. It happens there, too. And, it happens on all other PCs I have tried, including mobile browsers (but there works for the moment and I don't wish to experiment there, not touching too late, as I purged all sessions before for testing :( ) [ another post update: it seems that this bug does not affect Android Chrome the same way like Desktop Chrome, in another words, it is working fine on Android ??? ]

Now, here's what's weird: when I access matomo login page for the first time, I am assigned cookie with id (hash) value A, and A is stored in _sessions table. As soon as I hit the submit button, my cookie id is changed to value B, and B is now stored in _sessions, there are no traces of id "A" any longer. Is this how it's supposed to be working?

Now, when I restart Chrome, my MATOMO cookie id is set to new value (cookie "C"), in database is still present the one from the previous session (cookie "B"), but it no longer matches, so the log-in page is shown again. That part works, but is it possible that Matomo erroneously assigns new cookie id when it shouldn't? In fact, I am practically now 100% convinced that Matomo does this on its own (destroys / unsets old cookie and sets new one). Why/How? Because, my own plugin "piwik_ignore" cookie is always/still present after the browser restart, which means something else other than browser is doing it. For completeness, all chrome extensions were disabled just in case, no change in behavior.

Again, 3 different PCs, different generations, same issue. It cannot be a coincidence.

@fdellwing commented on February 6th 2019 Contributor

Could you show us the cookie details after login? Path, expires, etc?

I just tested this myself, the session cookie that gets set is missing the expire date, so it will be deleted of you close the browser. This should not happen if you set "remember me".

This is the exact set-cookie header that the browser gets after login in with remember me: MATOMO_SESSID=9uflvb5k0q8s8fd45d5322uvv1; path=/; secure; HttpOnly

/cc @mattab @diosmosis

@dev-101 commented on February 6th 2019

Nice catch! My DevTools columns have shrunk to the left while I was focused on the name, completely missed the expiration date column:

1969-12-31T23:59:59.000Z

related: https://stackoverflow.com/questions/51333255/mysterious-cookie-expiration-date-1969-12-31t235959-000z

@dev-101 commented on February 6th 2019

Bingo! After setting exp date manually to +30 days it works! :)

@Findus23 commented on February 6th 2019 Member

As you can see here:
https://github.com/matomo-org/matomo/blob/491ff0d8ecd7d2e9daeeb103a2a7fc099526dff9/plugins/Login/Login.php#L141-L145
Matomo tells the session module to set the remember-me-cookie expiration to the value of login_cookie_expire in the [General] setting of the config.

And as you can see here, this value is by default 14 days:
https://github.com/matomo-org/matomo/blob/433afeafcfe7cb29f7d4747f8fcf14a05da753f0/config/global.ini.php#L393-L395

Is it possible that you set the value to -1 which seems to be interpreted as the unix time -1 (as in one second before 1970).?

@dev-101 commented on February 6th 2019

Nope, this is in my config.ini.php:

[General]
login_cookie_expire = 2592000
@fdellwing commented on February 6th 2019 Contributor

I'm currently not at work, but I'm like 99,9% sure that I did not touched that setting.

@dev-101 commented on February 6th 2019

If you refer to latest Matomo releases (e.g. 3.8.0/3.8.1), I am having this issue for a lot longer than that.
Sorry, not sure what your comment means :)

Btw. I have set that 2592000 value manually few months ago (see related issue in first post).

@dev-101 commented on February 6th 2019

btw. shouldn't cookie exp date take current time into consideration?
e.g.:

time() + Config::getInstance()->General['login_cookie_expire']

@fdellwing commented on February 6th 2019 Contributor

No, that is handled by the browser.

@dev-101 commented on February 6th 2019

According to my simple tests with setcookie() that doesn't sound right. Without time() Chrome just fails to set it with plain login_cookie_expire value (or less).

Also, check Cookie class: https://github.com/matomo-org/matomo/blob/3.x-dev/core/Cookie.php
I see that there is some time/date calculation going on.

@dev-101 commented on February 6th 2019

fix: https://github.com/matomo-org/matomo/commit/160299d8a544163253a6fdc706000106f22340d5

This will set proper cookie expiration time if option 'Remember me' is checked.

@Findus23 commented on February 6th 2019 Member

@dev-101

At least according to the PHP docs the current value is correct:
https://secure.php.net/manual/en/session.configuration.php#ini.session.gc-maxlifetime

session.gc_maxlifetime specifies the number of seconds after which data will be seen as 'garbage' and potentially cleaned up.
default: "1440"

As the current value is working for many existing Matomo users, there has to be something different in your PHP config.

@dev-101 commented on February 6th 2019

But, you no longer use ordinary sessions, right? They are now in database?
And 3rd parameter in setcookie() is $expiration, which requires time().
Cookie is the problem here, not sessions.

@dev-101 commented on February 6th 2019

I have changed only 1-2 parameters in php.ini, file/fopen related, never touched anything related to sessions, iirc.

@dev-101 commented on February 6th 2019

Remember, I've tested it on several different servers and computers/devices, this is a consistent bug.

@fdellwing commented on February 6th 2019 Contributor

I'll look into this tomorrow. I never noticed this before, because I never close my browser at work.

Modifying the gc_maxlifetime seems not to be the correct solution for this, but I have to debug the actual code to see what i going wrong.

@dev-101 commented on February 6th 2019

Ok, maybe this needs to be fixed in Cookie class, that's why I haven't made a PR. I haven't traced entire code flow, one possibility why this works is because param for exp date is read from session value and stored into cookie. Can't be bothered with that right now, this is a temp fix that seems to be working.

Thanks

@Findus23 commented on February 6th 2019 Member

I'm pretty sure this has nothing to do with the cookie class. The session cookie is not set by Matomo, but by PHP itself.

If you follow
https://github.com/matomo-org/matomo/blob/491ff0d8ecd7d2e9daeeb103a2a7fc099526dff9/plugins/Login/Login.php#L141-L145
then you can see that this uses the Zend class
https://github.com/matomo-org/matomo/blob/433afeafcfe7cb29f7d4747f8fcf14a05da753f0/libs/Zend/Session.php#L330-L336
which again just calls session_set_cookie_params
https://github.com/matomo-org/matomo/blob/433afeafcfe7cb29f7d4747f8fcf14a05da753f0/libs/Zend/Session.php#L351-L376

session_set_cookie_params is just a function that tells PHP how (and how long) to set this session cookie.
And interestingly [there is a comment](https://secure.php.net/manual/en/function.session-set-cookie-params.php#100657) explaining how to make the session cookie renew itself every time one visits the page, which seems to be exactly what https://github.com/matomo-org/matomo/pull/13554 implements.

@dev-101 commented on February 6th 2019

Again, something I've noticed and reported in the previous issue still holds true (this is without above 'fix', original 3.8.1 code):

On the "fresh" log-in procedure (cleared all sessions @ database + cleared all cookies @ chrome), initial exp date value for cookie will be set to:

1969-12-31T23:59:59.000Z

However, if - during the very same session - I use sign-out from the menu, and than immediately log-in again, date will be set as per config.ini.php directive (30 days from now) or 14 days (default).

@fdellwing commented on February 7th 2019 Contributor

So, I think I found a (the) issue and I can not see this working for anyone @Findus23

We have these two functions in plugins/Login/Login.php, the error_logs got added by me:

public function beforeSessionStart()
    {
        if (!$this->shouldHandleRememberMe()) {
            return;
        }

        // if this is a login request & form_rememberme was set, change the session cookie expire time before starting the session
        $rememberMe = isset($_POST['form_rememberme']) ? $_POST['form_rememberme'] : null;
        error_log("rememberme :: " . $rememberMe);
        if ($rememberMe == '1') {
            Session::rememberMe(Config::getInstance()->General['login_cookie_expire']);
        }
    }

private function shouldHandleRememberMe()
    {
        $module = Common::getRequestVar('module', false);
        $action = Common::getRequestVar('action', false);
        error_log("-----debug-----");
        error_log($module);
        error_log($action);
        error_log(($module == 'Login' || $module == 'CoreHome') && (empty($action) || $action == 'index' || $action == 'login'));
        error_log(isset($_POST['form_rememberme']) ? $_POST['form_rememberme'] : null);
        return ($module == 'Login' || $module == 'CoreHome') && (empty($action) || $action == 'index' || $action == 'login');
    }

And this is the output if you login with rememberme tickt:

2019/02/07 08:52:54 [error] 6342<a href='/6342'>#6342</a>: *166837 FastCGI sent in stderr: "PHP message: -----debug-----
PHP message: 
PHP message: 
PHP message: 
PHP message: 1" while reading response header from upstream, client: X.X.X.X, server: stats.example.com, request: "POST / HTTP/1.1", upstream: "fastcgi://unix:/var/run/php/php7.0-fpm.sock:", host: "stats.example.com", referrer: "https://stats.example.com/"
2019/02/07 08:52:54 [error] 6340<a href='/6340'>#6340</a>: *166824 FastCGI sent in stderr: "PHP message: -----debug-----
PHP message: 
PHP message: 
PHP message: 
PHP message:" while reading response header from upstream, client: X.X.X.X, server: stats.example.com, request: "GET / HTTP/1.1", upstream: "fastcgi://unix:/var/run/php/php7.0-fpm.sock:", host: "stats.example.com", referrer: "https://stats.example.com/"
2019/02/07 08:52:55 [error] 6342<a href='/6342'>#6342</a>: *166837 FastCGI sent in stderr: "PHP message: -----debug-----
PHP message: CoreHome
PHP message: index
PHP message: 1
PHP message: 
PHP message: rememberme ::" while reading response header from upstream, client: X.X.X.X, server: stats.example.com, request: "GET /index.php?module=CoreHome&action=index&idSite=1&period=day&date=yesterday HTTP/1.1", upstream: "fastcgi://unix:/var/run/php/php7.0-fpm.sock:", host: "stats.example.com", referrer: "https://stats.example.com/"
2019/02/07 08:52:55 [error] 6342<a href='/6342'>#6342</a>: *166837 FastCGI sent in stderr: "PHP message: -----debug-----
PHP message: Proxy
PHP message: getCoreJs
PHP message: 
PHP message:" while reading response header from upstream, client: X.X.X.X, server: stats.example.com, request: "GET /index.php?module=Proxy&action=getCoreJs&cb=ef58613c1a79bcefb0959c846c3a9439 HTTP/1.1", upstream: "fastcgi://unix:/var/run/php/php7.0-fpm.sock:", host: "stats.example.com", referrer: "https://stats.example.com/index.php?module=CoreHome&action=index&idSite=1&period=day&date=yesterday"
2019/02/07 08:52:55 [error] 6340<a href='/6340'>#6340</a>: *166824 FastCGI sent in stderr: "PHP message: -----debug-----
PHP message: Proxy
PHP message: getCss
PHP message: 
PHP message:" while reading response header from upstream, client: X.X.X.X, server: stats.example.com, request: "GET /index.php?module=Proxy&action=getCss&cb=e253a28adad42dcc42771ef26cd549ef HTTP/1.1", upstream: "fastcgi://unix:/var/run/php/php7.0-fpm.sock:", host: "stats.example.com", referrer: "https://stats.example.com/index.php?module=CoreHome&action=index&idSite=1&period=day&date=yesterday"
2019/02/07 08:52:55 [error] 6341<a href='/6341'>#6341</a>: *166841 FastCGI sent in stderr: "PHP message: -----debug-----
PHP message: Proxy
PHP message: getNonCoreJs
PHP message: 
PHP message:" while reading response header from upstream, client: X.X.X.X, server: stats.example.com, request: "GET /index.php?module=Proxy&action=getNonCoreJs&cb=ef58613c1a79bcefb0959c846c3a9439 HTTP/1.1", upstream: "fastcgi://unix:/var/run/php/php7.0-fpm.sock:", host: "stats.example.com", referrer: "https://stats.example.com/index.php?module=CoreHome&action=index&idSite=1&period=day&date=yesterday"

So, as you can see the Session::rememberMe(Config::getInstance()->General['login_cookie_expire']); is never called on login. First function call has the form_rememberme but no module and action and third request has the module and action but no more form_rememberme.

Additional note: If you have a cookie with expires set, the time is correctly set to last access + 14 days as #13554 implements.

@fdellwing commented on February 7th 2019 Contributor

I found one more thing:

If you logout and than login, the above code works as expected (URL: https://stats.promato.de/index.php?module=CoreHome&action=&period=day&date=today). It only fails for me if I type in the URL of my instance and login (URL: https://stats.promato.de/).

As you can see, the URL after the logout contains both the module and the action that is required to hande rememberme and the manually typed does not.

@dev-101 commented on February 7th 2019

Yes, that confirms previous finding.

Btw. thanks for finally getting to the bottom of it. It sure shook my confidence for a moment :)

@diosmosis commented on February 7th 2019 Member

@fdellwing thanks for delving into this issue! I've created a PR here: https://github.com/matomo-org/matomo/pull/14081

@fdellwing @dev-101 can you two check whether the fix works for you?

@dev-101 commented on February 7th 2019

Thanks, quick tests show that it works!

This Issue was closed on February 11th 2019
Powered by GitHub Issue Mirror