Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Check downloaded referrer spam list for validity #19841

Closed
Starker3 opened this issue Oct 10, 2022 · 1 comment · Fixed by #19845
Closed

Check downloaded referrer spam list for validity #19841

Starker3 opened this issue Oct 10, 2022 · 1 comment · Fixed by #19845
Assignees
@sgiehl
Labels
Bug For errors / faults / flaws / inconsistencies etc.
Milestone
4.12.1

Comments

@Starker3
Copy link
Contributor

We have a user who's normal website traffic was being ignored as a result of the referrer spam blacklist check.
Upon further investigation it was found the the referrer_spam_blacklist row in the matomo_option table contained the HTML content from an internal firewall/content filter that blocked the outgoing request.

The option had the following value for example:

option_name: referrer_spam_blacklist

option_value: a:67:{i:0;s:12:"<HTML><HEAD>";i:1;s:0:"";i:2;s:28:"<TITLE>Access Denied</TITLE>";i:3;s:0:"";i:4;s:7:"</HEAD>";i:5;s:0:"";i:6;s:6:"<BODY>";i:7;s:0:"";i:8;s:236:"<div style="OVERFLOW: hidden; MAX-WIDTH: 700px; BORDER-TOP: #285ea6 1px solid; BORDER-RIGHT: #285ea6 1px solid; WIDTH: 95%; BORDER-BOTTOM: #285ea6 1px solid; MARGIN-LEFT: 10px; BORDER-LEFT: #285ea6 1px solid; BACKGROUND-COLOR: #eef2f7">";i:9;s:0:"";i:10;s:36:"<TABLE width="100%" border=0><TBODY>";i:11;s:0:"";i:12;s:4:"<TR>";i:13;s:0:"";i:14;s:283:"<TH align=left><IMG title="Logo xyz" style="FLOAT: left; PADDING-BOTTOM: 2px; PADDING-TOP: 2px; PADDING-LEFT: 0px; CLEAR: both; MARGIN: 0px 10px 6px 0px; PADDING-RIGHT: 0px" alt="Logo xyz" src=http://security-portal.xyz.org/xyzBanner.png></TH></TR></TBODY></TABLE>";i:15;s:0:"";i:16;s:104:"<noscript><h1 id="noscript-blockMessage">Content blocked or subject to user confirmation</h1></noscript>";i:17;s:0:"";i:18;s:209:"<div style="font-family: Arial, Helvetica, sans-serif; font-size: 12pt; padding: 0px; background: #EEF2F7 url('http://xyz.org/block.png') 5px 5px no-repeat; margin: 5px 7px;" id="pageBody">";i:19;s:0:"";i:20;s:130:"<H1 style="font-size: 1.2em; border-bottom: 1px solid #8c8c8c; padding-bottom: 6px; margin: 0px 0px 15px 30px;" id="blockMessage">";i:21;s:0:"";i:22;s:94:"Access Denied </H1><NOSCRIPT></NOSCRIPT><!-- This displays the reason the site was blocked -->";i:23;s:0:"";i:24;s:151:"<p style="font-size: 90%; margin-top: 0; margin-bottom: 10px; display: block; float: left; width: 100px; font-weight: bold; " class="label">Reason:</p>";i:25;s:0:"";i:26;s:301:"<p style="font-size: 90%; margin-top: 0; margin-bottom: 10px; padding-top: 1px; margin: 0px 5px 10px 100px; " class="first-option" id="reason-text">Your request was denied because of its content categorization: "Allow MacMini;Content Delivery Networks" (Access Denied - content_filter_denied) </p>";i:27;s:0:"";i:28;s:63:"</P><!-- This displays the URL the user attempted to access -->";i:29;s:0:"";i:30;s:148:"<P style="font-size: 90%; margin-top: 0; margin-bottom: 10px; display: block; float: left; width: 100px; font-weight: bold; " class="label">[url=URL:]URL:</P>";i:31;s:0:"";i:32;s:179:"<P style="font-size: 90%; margin-top: 0; margin-bottom: 10px; padding-top: 1px; margin: 0px 5px 10px 100px; " id="url-text">tcp://raw.githubusercontent.com:443/</P>;";i:33;s:0:"";i:34;s:37:"<DIV style="CLEAR: both"></DIV></div>";i:35;s:0:"";i:36;s:5:"<br/>";i:37;s:0:"";i:38;s:161:"<DIV style="width: 100%; border-top: 1px solid #D7E1EC; border-bottom: 1px solid #D7E1EC; background: #ffffff; padding: 6px 0px 6px 0px; clear: both;"id=options>";i:39;s:0:"";i:40;s:150:"<P style="font-size: 90%; margin-top: 0; margin-bottom: 10px; display: block; float: left; width: 100px; font-weight: bold; " class=label>Options:</P>";i:41;s:0:"";i:42;s:195:"<P style="font-size: 90%; margin-top: 0; margin-bottom: 10px; padding-top: 1px; margin: 0px 5px 10px 100px;" >Click <B>Go Back</B> or use the browser's Back button to return to the previous page.";i:43;s:0:"";i:44;s:116:"<br/><INPUT onClick="history.go(-1);return true;" type=button value=" Go Back " name=ws-back></P></FORM>";i:45;s:0:"";i:46;s:70:"<DIV style="OVERFLOW: hidden; HEIGHT: 1px; CLEAR: both"></DIV></DIV></";i:47;s:0:"";i:48;s:5:"<br/>";i:49;s:0:"";i:50;s:5:"<br/>";i:51;s:0:"";i:52;s:5:"<br/>";i:53;s:0:"";i:54;s:5:"<br/>";i:55;s:0:"";i:56;s:5:"<br/>";i:57;s:0:"";i:58;s:5:"<br/>";i:59;s:0:"";i:60;s:5:"<br/>";i:61;s:0:"";i:62;s:0:"";i:63;s:0:"";i:64;s:6:"</div>";i:65;s:0:"";i:66;s:14:"</BODY></HTML>";}

Since this effectively was a silent fail the user was not alerted to a failed request for referrer spam blacklist.
@Starker3 Starker3 added the Potential Bug Something that might be a bug, but needs validation and confirmation it can be reproduced. label Oct 10, 2022
@sgiehl
Copy link
Member

sgiehl commented Oct 11, 2022

That indeed could be checked. I'll prepare a small PR to fix that.

@sgiehl sgiehl self-assigned this Oct 11, 2022
@sgiehl sgiehl added this to the 4.12.1 milestone Oct 11, 2022
@sgiehl sgiehl added Bug For errors / faults / flaws / inconsistencies etc. and removed Potential Bug Something that might be a bug, but needs validation and confirmation it can be reproduced. labels Oct 11, 2022
@sgiehl sgiehl mentioned this issue Oct 11, 2022
Merged
11 tasks
@justinvelluppillai justinvelluppillai changed the title Referrer Spam Blacklist scheduled task does not validate returned content Check downloaded referrer spam list for validity Oct 18, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@sgiehl sgiehl

Labels
Bug For errors / faults / flaws / inconsistencies etc.
Projects
None yet
Milestone
4.12.1
Development

Successfully merging a pull request may close this issue.

Check downloaded referrer spam list for validity matomo-org/matomo
2 participants
@sgiehl @Starker3