Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Content Tracking: '&' in redirecturl gets replaced by '&' #15240

Closed
beamylake opened this issue Dec 5, 2019 · 3 comments
Closed

Content Tracking: '&' in redirecturl gets replaced by '&' #15240

beamylake opened this issue Dec 5, 2019 · 3 comments
Labels
answered For when a question was asked and we referred to forum or answered it.

Comments

@beamylake
Copy link

Hi,

we recently enabled content tracking and it works but with one major downside:

As soon as the URL of the page, where the user should be directed to, contains a '&' char, Matomo replaces it with "& amp;" (space is just needed as the autoformat would only display a single &)

Examples:

Original Link:
https://www.mydomain.com/AddBundleCS.action?searchFor=1&products=10018981-1_10024886-1_10027707-1_10022368-1_10018830-1_10034914-1_10024379-1_10022677-1_10018823-1_10044787-1

Link with content tracking enabled:
https://piwik.mydomain.com/piwik.php?redirecturl=https%3A%2F%2Fwww.mydomain.com%2FAddBundleCS.action%3FsearchFor%3D1%26products%3D10018981-1_10024886-1_10027707-1_10022368-1_10018830-1_10034914-1_10024379-1_10022677-1_10018823-1_10044787-1&c_i=click&c_n=Desktop_Article_Header-1_Shopthelook&c_p=Rezept%3A_Zimttorte_mit_Dattelkaramell&c_t=https%3A%2F%2Fwww.mydomain.com%2FAddBundleCS.action%3FsearchFor%3D1%26products%3D10018981-1_10024886-1_10027707-1_10022368-1_10018830-1_10034914-1_10024379-1_10022677-1_10018823-1_10044787-1&idsite=1&rec=1&r=272702&h=10&m=28&s=1&url=https%3A%2F%2Fwww.mydomain.com%2Fernaehrung%2Fbeitrag%2FZimttorte-mit-Dattelkaramell-mydomain-Online-Shop%2F&_id=cafa16456d5ba26f&_idts=1575468919&_idvc=3&_idn=0&_refts=0&_viewts=1575537505&send_image=0&cookie=1&res=3072x1728&gt_ms=676

When a user clicks the above link, he gets redirected by Matomo to this url:
https://www.mydomain.com/AddBundleCS.action?searchFor=1&products=10018981-1_10024886-1_10027707-1_10022368-1_10018830-1_10034914-1_10024379-1_10022677-1_10018823-1_10044787-1

and the "& amp;" (space again needed to fight autoformat) in the URL causes the application to serve a "Not found page" (although the HTTP status returned is 200)

Please see the attached screenshot taken from Firefox Dev Console / Network Analysis.

This error can be reproduced easily: Every link tagged with content-tracking attributes and containing a '&' character in the URL, will get this character replaced by "& amp;" (autoformat...) in the redirect served by Matomo.

If an adblocker is enabled client-side (preventing the Matomo JS from loading), the website will display the Original link mentioned above which works just fine.

This issue occured with the implementation of content tracking and it seems that its occurance is limited to links tagged with content tracking attributes.

Browser used to reproduce this problem: Firefox 70.0.1 and 71.0 (64Bit, Linux&Windows)

We did some research on earlier, similar issues which look similar in some way, but imho those two aren't directly connected to content tracking, but I could be wrong.

#3549
#8496

Our Matomo Details:

Version: 3.13 (Apache with mod_php)
PHP Version: 7.2

Content of config.ini.php:

; DO NOT REMOVE THIS LINE
; file automatically generated or modified by Matomo; you can manually override the default values in global.ini.php by redefining them in this file.
[database]
host = "[REDACTED]"
username = "[REDACTED]"
password = "[REDACTED]"
dbname = "[REDACTED]"
tables_prefix = "matomo_"

[log]
log_writers[] = "file"

[General]
noreply_email_address = "noreply@mydomain.com"
multi_server_environment = 1
salt = "[REDACTED]"
trusted_hosts[] = "piwik.mydomain.com"
trusted_hosts[] = "mat.mydomain.com"
enable_logging = 1

[Deletelogs]
delete_logs_enable = 1
delete_logs_older_than = 400

[Deletereports]
delete_reports_enable = 1

[mail]
transport = "smtp"
port = 587
host = "mail.mydomain.com"
type = "Login"
username = "user@mydomain.com"
password = "[REDACTED]"
encryption = "tls"
defaultHostnameifEmpty = "https://[REDACTED].com/"

[Plugins]
Plugins[] = "CorePluginsAdmin"
Plugins[] = "CoreAdminHome"
Plugins[] = "CoreHome"
Plugins[] = "WebsiteMeasurable"
Plugins[] = "IntranetMeasurable"
Plugins[] = "Diagnostics"
Plugins[] = "CoreVisualizations"
Plugins[] = "Proxy"
Plugins[] = "API"
Plugins[] = "Widgetize"
Plugins[] = "Transitions"
Plugins[] = "LanguagesManager"
Plugins[] = "Actions"
Plugins[] = "Dashboard"
Plugins[] = "MultiSites"
Plugins[] = "Referrers"
Plugins[] = "UserLanguage"
Plugins[] = "DevicesDetection"
Plugins[] = "Goals"
Plugins[] = "Ecommerce"
Plugins[] = "SEO"
Plugins[] = "Events"
Plugins[] = "UserCountry"
Plugins[] = "GeoIp2"
Plugins[] = "VisitsSummary"
Plugins[] = "VisitFrequency"
Plugins[] = "VisitTime"
Plugins[] = "VisitorInterest"
Plugins[] = "RssWidget"
Plugins[] = "Feedback"
Plugins[] = "Monolog"
Plugins[] = "Login"
Plugins[] = "TwoFactorAuth"
Plugins[] = "UsersManager"
Plugins[] = "SitesManager"
Plugins[] = "Installation"
Plugins[] = "CoreUpdater"
Plugins[] = "CoreConsole"
Plugins[] = "ScheduledReports"
Plugins[] = "UserCountryMap"
Plugins[] = "Live"
Plugins[] = "CustomVariables"
Plugins[] = "PrivacyManager"
Plugins[] = "ImageGraph"
Plugins[] = "Annotations"
Plugins[] = "MobileMessaging"
Plugins[] = "Overlay"
Plugins[] = "SegmentEditor"
Plugins[] = "Insights"
Plugins[] = "Morpheus"
Plugins[] = "Contents"
Plugins[] = "BulkTracking"
Plugins[] = "Resolution"
Plugins[] = "DevicePlugins"
Plugins[] = "Heartbeat"
Plugins[] = "Intl"
Plugins[] = "Marketplace"
Plugins[] = "UserId"
Plugins[] = "CustomPiwikJs"
Plugins[] = "Tour"
Plugins[] = "TagManager"
Plugins[] = "CustomDimensions"
Plugins[] = "HeatmapSessionRecording"
Plugins[] = "LoginFailLog"
Plugins[] = "MarketingCampaignsReporting"
Plugins[] = "SearchEngineKeywordsPerformance"
Plugins[] = "UsersFlow"
Plugins[] = "WhiteLabel"

[PluginsInstalled]
PluginsInstalled[] = "Diagnostics"
PluginsInstalled[] = "Login"
PluginsInstalled[] = "CoreAdminHome"
PluginsInstalled[] = "UsersManager"
PluginsInstalled[] = "SitesManager"
PluginsInstalled[] = "Installation"
PluginsInstalled[] = "Monolog"
PluginsInstalled[] = "Intl"
PluginsInstalled[] = "CorePluginsAdmin"
PluginsInstalled[] = "CoreHome"
PluginsInstalled[] = "WebsiteMeasurable"
PluginsInstalled[] = "IntranetMeasurable"
PluginsInstalled[] = "CoreVisualizations"
PluginsInstalled[] = "Proxy"
PluginsInstalled[] = "API"
PluginsInstalled[] = "ExamplePlugin"
PluginsInstalled[] = "Widgetize"
PluginsInstalled[] = "Transitions"
PluginsInstalled[] = "LanguagesManager"
PluginsInstalled[] = "Actions"
PluginsInstalled[] = "Dashboard"
PluginsInstalled[] = "MultiSites"
PluginsInstalled[] = "Referrers"
PluginsInstalled[] = "UserLanguage"
PluginsInstalled[] = "DevicesDetection"
PluginsInstalled[] = "Goals"
PluginsInstalled[] = "Ecommerce"
PluginsInstalled[] = "SEO"
PluginsInstalled[] = "Events"
PluginsInstalled[] = "UserCountry"
PluginsInstalled[] = "GeoIp2"
PluginsInstalled[] = "VisitsSummary"
PluginsInstalled[] = "VisitFrequency"
PluginsInstalled[] = "VisitTime"
PluginsInstalled[] = "VisitorInterest"
PluginsInstalled[] = "ExampleAPI"
PluginsInstalled[] = "RssWidget"
PluginsInstalled[] = "Feedback"
PluginsInstalled[] = "CoreUpdater"
PluginsInstalled[] = "CoreConsole"
PluginsInstalled[] = "ScheduledReports"
PluginsInstalled[] = "UserCountryMap"
PluginsInstalled[] = "Live"
PluginsInstalled[] = "CustomVariables"
PluginsInstalled[] = "PrivacyManager"
PluginsInstalled[] = "ImageGraph"
PluginsInstalled[] = "Annotations"
PluginsInstalled[] = "MobileMessaging"
PluginsInstalled[] = "Overlay"
PluginsInstalled[] = "SegmentEditor"
PluginsInstalled[] = "Insights"
PluginsInstalled[] = "Morpheus"
PluginsInstalled[] = "Contents"
PluginsInstalled[] = "BulkTracking"
PluginsInstalled[] = "Resolution"
PluginsInstalled[] = "DevicePlugins"
PluginsInstalled[] = "Heartbeat"
PluginsInstalled[] = "Marketplace"
PluginsInstalled[] = "ProfessionalServices"
PluginsInstalled[] = "UserId"
PluginsInstalled[] = "CustomPiwikJs"
PluginsInstalled[] = "DBStats"
PluginsInstalled[] = "LoginFailLog"
PluginsInstalled[] = "TwoFactorAuth"
PluginsInstalled[] = "TagManager"
PluginsInstalled[] = "MarketingCampaignsReporting"
PluginsInstalled[] = "UsersFlow"
PluginsInstalled[] = "CustomDimensions"
PluginsInstalled[] = "HeatmapSessionRecording"
PluginsInstalled[] = "WhiteLabel"
PluginsInstalled[] = "Tour"
PluginsInstalled[] = "SearchEngineKeywordsPerformance"

[UsersFlow]
UsersFlow_num_max_steps = 10
UsersFlow_num_max_rows_in_actions = 100
UsersFlow_num_max_links_per_interaction = 5000

[HeatmapSessionRecording]
add_tracking_code_only_when_needed = 1
session_recording_sample_limits = "50,100,250,500,1000,2000,5000"

If you need any further information, please let me know.

We appreciate any help, thanks in advance!

requests

@tsteur tsteur added the Bug For errors / faults / flaws / inconsistencies etc. label Dec 5, 2019
tsteur added a commit that referenced this issue Dec 5, 2019
@tsteur
Copy link
Member

tsteur commented Dec 5, 2019

FYI this will be fixed with #6265
Can you try this change? https://github.com/matomo-org/matomo/compare/3.x-dev...15240?quick_pull=1#diff-4e097e37559c4b44dddae3d3c61f97c6R124

We likely won't merge this change since I reckon we will remove the redirecturl feature and instead use tracking requests to track the click in the future to prevent these kind of things

@beamylake
Copy link
Author

Thanks a lot for your quick reply!

Your patch is already implemented in our test environment, I will let you know how it works out!

@beamylake
Copy link
Author

We've pushed your patch into production and can confirm that everything is working as expected.

Thanks again for your quick help!

@tsteur tsteur added answered For when a question was asked and we referred to forum or answered it. and removed Bug For errors / faults / flaws / inconsistencies etc. labels Dec 6, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
answered For when a question was asked and we referred to forum or answered it.
Projects
None yet
Development

No branches or pull requests

2 participants