Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

http2 issue with curl #14105

Closed
HugoHeneault opened this issue Feb 16, 2019 · 7 comments
Closed

http2 issue with curl #14105

HugoHeneault opened this issue Feb 16, 2019 · 7 comments
Labels
answered For when a question was asked and we referred to forum or answered it.

Comments

@HugoHeneault
Copy link

I have an issue blocking my archiving process:

INFO [2019-02-16 09:34:00] 24602  ---------------------------
INFO [2019-02-16 09:34:00] 24602  INIT
INFO [2019-02-16 09:34:00] 24602  Running Matomo 3.8.1 as Super User
INFO [2019-02-16 09:34:00] 24602  ---------------------------
INFO [2019-02-16 09:34:00] 24602  NOTES
INFO [2019-02-16 09:34:00] 24602  - Reports for today will be processed at most every 3600 seconds. You can change this value in Matomo UI > Settings > General Settings.
INFO [2019-02-16 09:34:00] 24602  - Reports for the current week/month/year will be requested at most every 3600 seconds.
INFO [2019-02-16 09:34:00] 24602  - Archiving was last executed without error 58 days 11 hours ago
INFO [2019-02-16 09:34:00] 24602  - Will pre-process 1 Segments for each website and each period: visitCount==1
INFO [2019-02-16 09:34:00] 24602  - Will process 56 other websites because the last time they were archived was on a different day (in the website's timezone) , IDs: 1, 3, 4, 5, 6, 7, 8, 9, 10, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 52, 53, 54, 57, 60, 61, 62, 63
INFO [2019-02-16 09:34:00] 24602  - Will process 36 other websites because some old data reports have been invalidated (eg. using the Log Import script or the InvalidateReports plugin) , IDs: 1, 3, 5, 6, 8, 14, 15, 18, 20, 21, 23, 24, 25, 27, 29, 30, 32, 33, 34, 35, 36, 37, 40, 41, 42, 44, 45, 46, 50, 52, 53, 57, 60, 61, 62, 63
INFO [2019-02-16 09:34:00] 24602  ---------------------------
INFO [2019-02-16 09:34:00] 24602  START
INFO [2019-02-16 09:34:00] 24602  Starting Matomo reports archiving...
INFO [2019-02-16 09:34:00] 24602  Day has finished for website id 1 since last run
INFO [2019-02-16 09:34:00] 24602  Will pre-process for website id = 1, period = day, date = last52
INFO [2019-02-16 09:34:00] 24602  - pre-processing all visits
ERROR [2019-02-16 09:34:01] 24602  Got invalid response from API request: ?module=API&method=API.get&idSite=1&period=day&date=last52&format=php&trigger=archivephp. Response was 'Got invalid response from API request: https://piwik.xxxxx.fr/?module=API&method=API.get&idSite=1&period=day&date=last52&format=php&trigger=archivephp&token_auth=removed. Response was 'curl_exec: HTTP/2 stream 0 was not closed cleanly: PROTOCOL_ERROR (err 1). Hostname requested was: piwik.xxxxx.fr''
ERROR [2019-02-16 09:34:01] 24602  Empty or invalid response '' for website id 1, Time elapsed: 1.397s, skipping
INFO [2019-02-16 09:34:01] 24602  Day has finished for website id 3 since last run
INFO [2019-02-16 09:34:01] 24602  Will pre-process for website id = 3, period = day, date = last52
INFO [2019-02-16 09:34:01] 24602  - pre-processing all visits

Lines we're looking at are:

ERROR [2019-02-16 09:34:01] 24602  Got invalid response from API request: ?module=API&method=API.get&idSite=1&period=day&date=last52&format=php&trigger=archivephp. Response was 'Got invalid response from API request: https://piwik.xxxxx.fr/?module=API&method=API.get&idSite=1&period=day&date=last52&format=php&trigger=archivephp&token_auth=removed. Response was 'curl_exec: HTTP/2 stream 0 was not closed cleanly: PROTOCOL_ERROR (err 1). Hostname requested was: piwik.xxxxx.fr''
ERROR [2019-02-16 09:34:01] 24602  Empty or invalid response '' for website id 1, Time elapsed: 1.397s, skipping

After digging a little without much findings, I got it fixed by adding in $curl_options of Http.php

                CURLOPT_HTTP_VERSION   => CURL_HTTP_VERSION_1_1

So it might be useful to add in settings a flag to choose which HTTP_VERSION should be used by curl for cases like this?

@fdellwing
Copy link
Contributor

I think it would be better to find the reason why this error happens instead of forcing http 1.1, because a server might not support it in the future.

@Findus23
Copy link
Member

If your server instructs clients to use HTTP2 and then fails to respond to HTTP2 requests then there might also be a fraction of your website visitors for which this HTTP request fails and who therefore will never show up in Matomo.

Therefore I'd recommend you to troubleshoot your server first. (or check if you are using an outdated curl version in case this is a curl bug)

@tsteur
Copy link
Member

tsteur commented Feb 16, 2019

I don't think this seems like an issue in Matomo? Can we close the issue here? And if needed discuss further in the forum maybe or continue discussing here but keep the issue closed unless there is a bug in Matomo?

@HugoHeneault
Copy link
Author

HugoHeneault commented Feb 18, 2019

Here is my curl --version output:

libcurl/7.29.0 NSS/3.36 zlib/1.2.7 libidn/1.28 libssh2/1.4.3
Protocols: dict file ftp ftps gopher http https imap imaps ldap ldaps pop3 pop3s rtsp scp sftp smtp smtps telnet tftp
Features: AsynchDNS GSS-Negotiate IDN IPv6 Largefile NTLM NTLM_WB SSL libz unix-sockets

When I run the curl command from my server I got a normal answer

curl  "https://piwik.xxx.fr/?module=API&method=API.get&idSite=1&period=day&date=last52&format=php&trigger=archivephp&token_auth=xxxx"

But from PHP I got an issue. I'll try to reproduce it in a script for my hoster and ask him.

What more could I give you to track this issue? You can also close it as it's not really a Matomo issue if you prefer not to handle a protocol switch (which makes sense)

@fdellwing
Copy link
Contributor

I'm pretty sure your cURL does not support http2:

curl 7.52.1 (x86_64-pc-linux-gnu) libcurl/7.52.1 OpenSSL/1.0.2q zlib/1.2.8 libidn2/0.16 libpsl/0.17.0 (+libidn2/0.16) libssh2/1.7.0 nghttp2/1.18.1 librtmp/2.3
Protocols: dict file ftp ftps gopher http https imap imaps ldap ldaps pop3 pop3s rtmp rtsp scp sftp smb smbs smtp smtps telnet tftp
Features: AsynchDNS IDN IPv6 Largefile GSS-API Kerberos SPNEGO NTLM NTLM_WB SSL libz TLS-SRP HTTP2 UnixSockets HTTPS-proxy PSL

To test this, run curl -v --http2 https://google.com.

@HugoHeneault
Copy link
Author

HugoHeneault commented Feb 18, 2019

You're right.

$ curl -v --http2 https://google.com.
curl: option --http2: is unknown
curl: try 'curl --help' or 'curl --manual' for more information

But PHP seems to be running a different curl...

image

Might be something like https://framagit.org/hubzilla/core/issues/1291#note_345604 ?
There also are issues reported with CPanel : https://forums.cpanel.net/threads/http2-stream-1-was-not-closed-cleanly.636993/

@tsteur
Copy link
Member

tsteur commented Feb 18, 2019

FYI I'll close this issue for now. Feel free to continue discussing and to reopen if there's an issue with Matomo and not the server itself.

@tsteur tsteur closed this as completed Feb 18, 2019
@tsteur tsteur added the answered For when a question was asked and we referred to forum or answered it. label Feb 18, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
answered For when a question was asked and we referred to forum or answered it.
Projects
None yet
Development

No branches or pull requests

4 participants