@HugoHeneault opened this Issue on February 16th 2019

I have an issue blocking my archiving process:

INFO [2019-02-16 09:34:00] 24602  ---------------------------
INFO [2019-02-16 09:34:00] 24602  INIT
INFO [2019-02-16 09:34:00] 24602  Running Matomo 3.8.1 as Super User
INFO [2019-02-16 09:34:00] 24602  ---------------------------
INFO [2019-02-16 09:34:00] 24602  NOTES
INFO [2019-02-16 09:34:00] 24602  - Reports for today will be processed at most every 3600 seconds. You can change this value in Matomo UI > Settings > General Settings.
INFO [2019-02-16 09:34:00] 24602  - Reports for the current week/month/year will be requested at most every 3600 seconds.
INFO [2019-02-16 09:34:00] 24602  - Archiving was last executed without error 58 days 11 hours ago
INFO [2019-02-16 09:34:00] 24602  - Will pre-process 1 Segments for each website and each period: visitCount==1
INFO [2019-02-16 09:34:00] 24602  - Will process 56 other websites because the last time they were archived was on a different day (in the website's timezone) , IDs: 1, 3, 4, 5, 6, 7, 8, 9, 10, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 52, 53, 54, 57, 60, 61, 62, 63
INFO [2019-02-16 09:34:00] 24602  - Will process 36 other websites because some old data reports have been invalidated (eg. using the Log Import script or the InvalidateReports plugin) , IDs: 1, 3, 5, 6, 8, 14, 15, 18, 20, 21, 23, 24, 25, 27, 29, 30, 32, 33, 34, 35, 36, 37, 40, 41, 42, 44, 45, 46, 50, 52, 53, 57, 60, 61, 62, 63
INFO [2019-02-16 09:34:00] 24602  ---------------------------
INFO [2019-02-16 09:34:00] 24602  START
INFO [2019-02-16 09:34:00] 24602  Starting Matomo reports archiving...
INFO [2019-02-16 09:34:00] 24602  Day has finished for website id 1 since last run
INFO [2019-02-16 09:34:00] 24602  Will pre-process for website id = 1, period = day, date = last52
INFO [2019-02-16 09:34:00] 24602  - pre-processing all visits
ERROR [2019-02-16 09:34:01] 24602  Got invalid response from API request: ?module=API&method=API.get&idSite=1&period=day&date=last52&format=php&trigger=archivephp. Response was 'Got invalid response from API request: https://piwik.xxxxx.fr/?module=API&method=API.get&idSite=1&period=day&date=last52&format=php&trigger=archivephp&token_auth=removed. Response was 'curl_exec: HTTP/2 stream 0 was not closed cleanly: PROTOCOL_ERROR (err 1). Hostname requested was: piwik.xxxxx.fr''
ERROR [2019-02-16 09:34:01] 24602  Empty or invalid response '' for website id 1, Time elapsed: 1.397s, skipping
INFO [2019-02-16 09:34:01] 24602  Day has finished for website id 3 since last run
INFO [2019-02-16 09:34:01] 24602  Will pre-process for website id = 3, period = day, date = last52
INFO [2019-02-16 09:34:01] 24602  - pre-processing all visits

Lines we're looking at are:

ERROR [2019-02-16 09:34:01] 24602  Got invalid response from API request: ?module=API&method=API.get&idSite=1&period=day&date=last52&format=php&trigger=archivephp. Response was 'Got invalid response from API request: https://piwik.xxxxx.fr/?module=API&method=API.get&idSite=1&period=day&date=last52&format=php&trigger=archivephp&token_auth=removed. Response was 'curl_exec: HTTP/2 stream 0 was not closed cleanly: PROTOCOL_ERROR (err 1). Hostname requested was: piwik.xxxxx.fr''
ERROR [2019-02-16 09:34:01] 24602  Empty or invalid response '' for website id 1, Time elapsed: 1.397s, skipping

After digging a little without much findings, I got it fixed by adding in $curl_options of Http.php

                CURLOPT_HTTP_VERSION   => CURL_HTTP_VERSION_1_1

So it might be useful to add in settings a flag to choose which HTTP_VERSION should be used by curl for cases like this?

@fdellwing commented on February 16th 2019 Contributor

I think it would be better to find the reason why this error happens instead of forcing http 1.1, because a server might not support it in the future.

@Findus23 commented on February 16th 2019 Member

If your server instructs clients to use HTTP2 and then fails to respond to HTTP2 requests then there might also be a fraction of your website visitors for which this HTTP request fails and who therefore will never show up in Matomo.

Therefore I'd recommend you to troubleshoot your server first. (or check if you are using an outdated curl version in case this is a curl bug)

@tsteur commented on February 16th 2019 Member

I don't think this seems like an issue in Matomo? Can we close the issue here? And if needed discuss further in the forum maybe or continue discussing here but keep the issue closed unless there is a bug in Matomo?

@HugoHeneault commented on February 18th 2019

Here is my curl --version output:

libcurl/7.29.0 NSS/3.36 zlib/1.2.7 libidn/1.28 libssh2/1.4.3
Protocols: dict file ftp ftps gopher http https imap imaps ldap ldaps pop3 pop3s rtsp scp sftp smtp smtps telnet tftp
Features: AsynchDNS GSS-Negotiate IDN IPv6 Largefile NTLM NTLM_WB SSL libz unix-sockets

When I run the curl command from my server I got a normal answer

curl  "https://piwik.xxx.fr/?module=API&method=API.get&idSite=1&period=day&date=last52&format=php&trigger=archivephp&token_auth=xxxx"

But from PHP I got an issue. I'll try to reproduce it in a script for my hoster and ask him.

What more could I give you to track this issue? You can also close it as it's not really a Matomo issue if you prefer not to handle a protocol switch (which makes sense)

@fdellwing commented on February 18th 2019 Contributor

I'm pretty sure your cURL does not support http2:

curl 7.52.1 (x86_64-pc-linux-gnu) libcurl/7.52.1 OpenSSL/1.0.2q zlib/1.2.8 libidn2/0.16 libpsl/0.17.0 (+libidn2/0.16) libssh2/1.7.0 nghttp2/1.18.1 librtmp/2.3
Protocols: dict file ftp ftps gopher http https imap imaps ldap ldaps pop3 pop3s rtmp rtsp scp sftp smb smbs smtp smtps telnet tftp
Features: AsynchDNS IDN IPv6 Largefile GSS-API Kerberos SPNEGO NTLM NTLM_WB SSL libz TLS-SRP HTTP2 UnixSockets HTTPS-proxy PSL

To test this, run curl -v --http2 https://google.com.

@HugoHeneault commented on February 18th 2019

You're right.

$ curl -v --http2 https://google.com.
curl: option --http2: is unknown
curl: try 'curl --help' or 'curl --manual' for more information

But PHP seems to be running a different curl...

image

Might be something like https://framagit.org/hubzilla/core/issues/1291#note_345604 ?
There also are issues reported with CPanel : https://forums.cpanel.net/threads/http2-stream-1-was-not-closed-cleanly.636993/

@tsteur commented on February 18th 2019 Member

FYI I'll close this issue for now. Feel free to continue discussing and to reopen if there's an issue with Matomo and not the server itself.

This Issue was closed on February 18th 2019
Powered by GitHub Issue Mirror