@tsteur opened this Pull Request on September 28th 2018 Member

fix #12922

  • When a tracking request is invalid for any of the 2 reasons, we log the request as a failure
  • Any failed request older than 2 days ago is automatically deleted again
  • Weekly we check for tracking failures and email super users only (doing this for admins is bit tricky)
  • Admins and SuperUsers see a new widget on Admin Home screen plus it can be used in the dashboard:
    image
  • New diagnostics page that shows tracking failures in depth
    image

I was also thinking about showing a box in the reporting toolbar, eg on the right... but then thought it may not be needed for now and might be rather annoying. May be good to first see how many false reports maybe come in etc.

I reckon it is a good working solution v1 for this tool and see from there how/where it goes. Heaps could be done there.

Todo: We should write FAQ articles for those 2 particular errors and adjust the links in the code

@mattab commented on October 3rd 2018 Member

@tsteur just thought of another use case which occurs from time to time, if Matomo is in an unstable state, there may be columns missing, this happened to me regularly, such errors:


DEBUG: Exception: Error query: SQLSTATE[42S22]: Column not found: 1054 Unknown column 'last_idlink_va' in 'field list' In query: SELECT visit_last_action_time, visit_first_action_time, idvisitor, idvisit, user_id, visit_exit_idaction_url, visit_exit_idaction_name, visitor_returning, visitor_days_since_first, visitor_days_since_order, visitor_count_visits, visit_goal_buyer, location_country, location_region, location_city, location_latitude, location_longitude, referer_name, referer_keyword, referer_type, idsite, visit_entry_idaction_url, visit_total_actions, visit_total_interactions, visit_total_searches, config_device_brand, config_device_model, config_device_type, visit_total_events, visit_total_time, location_ip, location_browser_lang, campaign_content, campaign_id, campaign_keyword, campaign_medium, campaign_name, campaign_source, last_idlink_va, custom_var_k1, custom_var_v1, custom_var_k2, custom_var_v2, custom_var_k3, custom_var_v3, custom_var_k4, custom_var_v4, custom_var_k5, custom_var_v5   FROM log_visit WHERE visit_last_action_time >= ? AND visit_last_action_time <= ? AND idsite = ? AND config_id = ?
DEBUG:                 ORDER BY visit_last_action_time DESC
DEBUG:                 LIMIT 1 Parameters: array (
DEBUG:   0 => '2018-10-03 05:21:30',
DEBUG:   1 => '2018-10-03 06:21:30',
DEBUG:   2 => 1,
DEBUG:   3 => '/ƫK���',
DEBUG: )

So it would be really great if we could detect these problems and also report them in the page :+1:

@tsteur commented on October 3rd 2018 Member

Not really so much possible. Cause it was said it is not needed to support anything else and therefore it is not built to include metadata etc. If something like this is possible, it would be only possible to say a column was missing, but not which column or table etc.

@diosmosis commented on November 30th 2018 Member

Left some comments & need to resolve some conflicts. Haven't tested locally yet, will do so next review.

@tsteur commented on November 30th 2018 Member

@diosmosis applied the review feedback and resolved the conflicts. I might need to update some UI tests...

@diosmosis commented on December 1st 2018 Member

Left one further comment, but will probably merge this today. if my comment is important can be done in a new PR

@tsteur commented on December 1st 2018 Member

@diosmosis just realising a feature to log not auth request regressed due to another PR... because isVisitExcluded seems to no longer work basically when CIP is set... be good to not merge.

@tsteur commented on December 1st 2018 Member

@diosmosis just pushing a fix, and also fixing the failed system and ui tests with this commit

@diosmosis commented on December 1st 2018 Member

Looks like there are some other test failures now? the novisit tests are failing (and possibly a unit test). Maybe some others too.

@tsteur commented on December 1st 2018 Member

hoping to have fixed them @diosmosis will check the results tomorrow.

@tsteur commented on December 2nd 2018 Member

this was tricky... hopefully this time it fixes most or all of the tests :)

@mattab commented on December 5th 2018 Member

Will be very useful :+1:

Not really so much possible. Cause it was said it is not needed to support anything else and therefore it is not built to include metadata etc. If something like this is possible, it would be only possible to say a column was missing, but not which column or table etc.

Would it be possible to catch these sql error cases, and report them as failures in this new tool's UI? (currently they may go un-detected for a few hours or days until someone realises tracking does not anymore)

@tsteur commented on December 5th 2018 Member

No, at least not as part of 3.8 release.

@diosmosis commented on December 6th 2018 Member

Almost working, looks like BulkTracking & QueuedTracking have some failing integration tests

@tsteur commented on December 6th 2018 Member

@diosmosis fixed the bulk and queued tracking tests. Looks like MarketingCampaignsReporting isn't directly failing because of this change. Hopefully can be finally merged :)

This Pull Request was closed on December 6th 2018
Powered by GitHub Issue Mirror