Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Reports with umlauts in the subject line #14313

Closed
mritzmann opened this issue Apr 4, 2019 · 10 comments
Closed

Reports with umlauts in the subject line #14313

mritzmann opened this issue Apr 4, 2019 · 10 comments
Labels
not-in-changelog For issues or pull requests that should not be included in our release changelog on matomo.org.

Comments

@mritzmann
Copy link

mritzmann commented Apr 4, 2019

  • We have websites names with umlauts (äüö)
  • The umlauts end up in the subject of e-mail reports
  • Sending is done with Postfix 2.11x (default on debian 8)
  • Webserver: nginx/1.14.2
  • PHP Version: 7.2.16

The problem is now: The e-mail server (here postfix) does not accept e-mail headers that are not correctly encoded. In this case, the subject.

Reason for bounce:

Reporting-MTA: dns; ****
X-Postfix-Queue-ID: 71E9F40059
X-Postfix-Sender: rfc822; postmaster@***
Arrival-Date: Mon, 18 Mar 2019 10:19:37 +0100 (CET)

Final-Recipient: rfc822; ***@***
Original-Recipient: rfc822;***@***
Action: failed
Status: 5.6.7
Diagnostic-Code: X-Postfix; SMTPUTF8 is required, but was not offered by host

(Note: The Postfix option smtputf8_enable is only available from Postfix 3.x on)

I can reproduce the problem with a newly installed version (Standard installation with ZIP from matomo.org). Interestingly, the problem only occurs if the interface is set to deutsch. If the interface is in English, it works without problems.

Step to reproduce:

  1. Use a Postfix Version without smtputf8_enable (default Debian 8)
  2. Setup Matomo and create a new website with äöü as website name
  3. Create a new report.
  4. Set interface language to english and trigger "Send Report now": E-mail delivery works
  5. Set interface language to deutsch and trigger "Bericht jetzt senden": E-mail delivery does not work
@Findus23
Copy link
Member

Findus23 commented Apr 4, 2019

This should be the perfect bug for me to reproduce:

  • I am using Debian 9 with the default postfix (3.1.9)
  • smtputf8_enable seems to be disabled (I never changed anything in the config)
➜  /etc/postfix grep -r "smtputf8_enable"
main.cf:smtputf8_enable = no
  • my Matomo is set to German
  • I have a website that has an "ö" in the title
  • I have just created a new report for this website and clicked on Bericht jetzt senden

But unlike in your case, for me the email is sent normally and I get it with the umlaut in the title of the mail.

As the only difference seems to be the the debian (and therefore postfix) version, I think that this isn't necessarily a bug in Matomo, but maybe rather something odd in your setup.

Are you using SMTP or phpmail in Matomo? (I quickly switched to SMTP, but the result stayed the same)

@fdellwing
Copy link
Contributor

fdellwing commented Apr 4, 2019

Could you both take a look at the subject header? To work everywhere it should look like this =?UTF-8?B?w6TDtsO8?= or =?UTF-8?Q?=C3=A4=C3=B6=C3=BC?=.

To do this in PHP is really simply:

$subject = "=?UTF-8?Q?" . quoted_printable_encode($subject) . "?=";

If this is not in Matomo, it should be added.

(There are a lot more complex methods to do this, see https://stackoverflow.com/a/27648245/7849268 for more info)

@mritzmann
Copy link
Author

mritzmann commented Apr 4, 2019

Could you both take a look at the subject header?

# first in german, second in english

$ tail -n 2 ~/log/php-mail.log 
[04-Apr-2019 13:15:18 UTC] mail() on [/home/matomo/www/libs/Zend/Mail/Transport/Sendmail.php:106]: To: markus@example.com -- Headers: From: Matomo Analytics <noreply@matomo.debugg.ops.opsserver.ch> Reply-To: admin <markus@example.com> Date: Thu, 04 Apr 2019 13:15:18 +0000 Content-Type: multipart/related;  boundary="=_e7389ca96f4c90ac0c36c7746f88819a" Content-Transfer-Encoding: quoted-printable Content-Disposition: inline MIME-Version: 1.0 -- Subject: =?utf-8?Q?Bericht=20B=C3=A4rentatze=20-=20Mittwoch,=203.=20April=202019?=
[04-Apr-2019 13:15:32 UTC] mail() on [/home/matomo/www/libs/Zend/Mail/Transport/Sendmail.php:106]: To: markus@example.com -- Headers: From: Matomo Analytics <noreply@matomo.debugg.ops.opsserver.ch> Reply-To: admin <markus@example.com> Date: Thu, 04 Apr 2019 13:15:32 +0000 Content-Type: multipart/related;  boundary="=_84ed37fcef806309a62f03d977cb2914" Content-Transfer-Encoding: quoted-printable Content-Disposition: inline MIME-Version: 1.0 -- Subject: =?utf-8?Q?Report=20B=C3=A4rentatze=20-=20Wednesday,=20April=203,=202019?=

Looks good to me.


Are you using SMTP or phpmail in Matomo?

phpmail, with sendmail:

$ grep -i sendmail /etc/php72/user/matomo/php.ini 
sendmail_path=/usr/sbin/sendmail -t -i

@mritzmann
Copy link
Author

I am using Debian 9 with the default postfix (3.1.9)

My setup:

+--------------------------------+                   +--------------------------------+
|                                |                   |                                |
| LEMP Stack                     |                   | Outgoing Mailrelay             |
| ==========                     |                   | ==================             |
|                                |                   |                                |
| OS: Debian 9                   |                   | OS: Debian 8                   |
| Webserver: Nginx               |          +--------+ Postfix: 2.11.3                |
| PHP: 7.2.16 (fpm)              |          |        |                                |
| Postfix: 3.1.9                 +----------+        |                                |
|                                |                   |                                |
+--------------------------------+                   +--------------------------------+

The e-mail is rejected by the outgoing mail relay. Seems as if the problem does not exist with newer Postfix versions. I will update the Mailrelay in the near future, then the problem has probably been solved.

Just found it strange that the problem only occurs when the interface is set to "Deutsch"... Hhm.

@mritzmann
Copy link
Author

Just found it strange that the problem only occurs when the interface is set to "Deutsch"... Hhm.

I mean: What is the difference between German and English when sending e-mails from Matomo?

@fdellwing
Copy link
Contributor

fdellwing commented Apr 4, 2019

With this header, the error should never appear. So this is strange.

Could you try sending a mail from your Matomo host via CLI using the correctly encoded header?

@mritzmann
Copy link
Author

mritzmann commented Apr 4, 2019

Could you try sending a mail from your Matomo host via CLI using the correctly encoded header?

This works:

$ mail -s "=?utf-8?Q?Bericht=20B=C3=A4rentatze=20-=20Mittwoch,=203.=20April=202019?=" markus@example.com < /dev/null
mail: Null message body; hope that's ok

Found out something interesting:

I saved the mail locally instead of sending it to see if I could see a difference.

$ grep -i sendmail /etc/php72/user/matomo/php.ini
sendmail_path=/usr/local/vzscripts/stdintotmp

$ cat /usr/local/vzscripts/stdintotmp
#!/bin/sh
cat < /dev/stdin > $(/bin/mktemp --tmpdir=${HOME}/tmp/ mail.$(date +%Y-%m-%d_%H-%M-%S).XXX.eml)

The Mail in English is stored as UTF-8, the German as ASCII only:

# english
$ file mail.2019-04-04_16-23-10.Hw9.eml
mail.2019-04-04_16-23-10.Hw9.eml: HTML document, UTF-8 Unicode text

# german
$ file mail.2019-04-04_16-23-42.Apl.eml
mail.2019-04-04_16-23-42.Apl.eml: HTML document, ASCII text

mail.2019-04-04_16-23-42.Apl.txt
mail.2019-04-04_16-23-10.Hw9.txt

@Findus23
Copy link
Member

Findus23 commented Apr 4, 2019

Hi everyone,

I have been moving this discussion to the forum as in my opinion it is far more useful for quick discussions and troubleshooting than GitHub where every response is sent to more than 400 people. If a specific bug in Matomo is found, a new GitHub issue with the description can be opened.


I want to try in the future to pay more attention (and add more documentation for users) on when to use the forum and when to open a github issue.
Personally I think GitHub is far more useful if every open issue is a bug (or feature request) that can be reproduced and someone can immediately start working on. That way we can maybe get GitHub more clear and avoid reaching 2000 open issues in the future.

@Findus23 Findus23 closed this as completed Apr 4, 2019
@Findus23 Findus23 added the not-in-changelog For issues or pull requests that should not be included in our release changelog on matomo.org. label Apr 4, 2019
@xenDE
Copy link

xenDE commented Apr 22, 2020

Hi,

22.04.2020 - i now have the same problem and i can not find a solution in this thread.

my local postfix becomes statusmails with utf8 chars in header and can not send the mails to extern mailservers:

Apr 22 15:26:00 servername postfix/smtp[21645]: AABBCCDDEEFF: to=<statusmail-rcpt@domain.de>, relay=mxtls.expurgate.net[123.123.123.123]:25, delay=0.13, delays=0.06/0.01/0.06/0, dsn=5.6.7, status=bounced (SMTPUTF8 is required, but was not offered by host mxtls.expurgate.net[123.123.123.123])

i think the problem starts after moving my webserver/matomo from an old debian to a new debian9 server.

How can i fix this?

  • its a matomo bug und still no patch for this?
  • needs change in postfix config to convert this utf8 headers in mails?

What can i do?

best regards,
daniel

@mritzmann
Copy link
Author

mritzmann commented Apr 22, 2020

@xenDE I see two solutions/workarounds:

  • Do not create reports and/or websites with umlauts.
  • Or set the following in your postfix configuration.
$ cat /etc/postfix/main.cf | grep smtputf8_enable
smtputf8_enable = no

The second one works for me.

For me it is not clear whether this is a configuration problem or a Matomo problem. On the one hand it can be fixed with a postfix configuration change, on the other hand the problem only occurs if Matomo is set to German.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
not-in-changelog For issues or pull requests that should not be included in our release changelog on matomo.org.
Projects
None yet
Development

No branches or pull requests

4 participants