Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Trying to fix randomly failing tests #15461

Merged
merged 9 commits into from Jan 30, 2020
Merged

Trying to fix randomly failing tests #15461

merged 9 commits into from Jan 30, 2020

Conversation

tsteur
Copy link
Member

@tsteur tsteur commented Jan 26, 2020

There are tests like https://builds-artifacts.matomo.org/matomo-org/matomo/4.x-dev/37972/UIIntegrationTest_dashboard2.png that are randomly failing since we execute the UI tests on PHP 7.

Not sure if this fixes it and whether it's the best way.

@tsteur tsteur added not-in-changelog For issues or pull requests that should not be included in our release changelog on matomo.org. Needs Review PRs that need a code review labels Jan 26, 2020
@tsteur tsteur added this to the 4.0.0 milestone Jan 26, 2020
if ($a[$field] === $b[$field]) {
return 0;
}
if ($a[$field] < $b[$field]) {
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It may be better to use strcmp for title etc but think it should not be needed

if ($a['idlink_va'] > $b['idlink_va']) {
return 1;
usort($actions, function ($a, $b) {
$fields = array('serverTimePretty', 'idlink_va', 'title', 'url', 'type', 'pageIdAction', 'goalId');
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

eg https://builds-artifacts.matomo.org/matomo-org/matomo/4.x-dev/37985/UIIntegrationTest_widgetize_visitor_log.png was still failing randomly cause there are 2 goal conversions with same time and idlinkva. Only the goal ID was different

INSERT INTO `log_conversion` (`idvisit`, `idsite`, `idvisitor`, `server_time`, `idaction_url`, `idlink_va`, `idgoal`, `buster`, `idorder`, `items`, `url`, `visitor_days_since_first`, `visitor_days_since_order`, `visitor_returning`, `visitor_count_visits`, `referer_keyword`, `referer_name`, `referer_type`, `config_device_brand`, `config_device_model`, `config_device_type`, `location_city`, `location_country`, `location_latitude`, `location_longitude`, `location_region`, `revenue`, `revenue_discount`, `revenue_shipping`, `revenue_subtotal`, `revenue_tax`, `custom_var_k1`, `custom_var_v1`, `custom_var_k2`, `custom_var_v2`, `custom_var_k3`, `custom_var_v3`, `custom_var_k4`, `custom_var_v4`, `custom_var_k5`, `custom_var_v5`, `custom_dimension_1`, `custom_dimension_2`, `custom_dimension_3`, `custom_dimension_4`, `custom_dimension_5`)
VALUES
	(57, 1, X'D0966C8CDB89610B', '2012-09-30 08:10:38', 1, 70, 1, 0, NULL, NULL, 'http://piwik.net/blog/category/meta/', 0, 0, 0, 1, NULL, NULL, 1, '', '', NULL, NULL, 'jp', 36.000000, 138.000000, NULL, 5, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL),
	(57, 1, X'D0966C8CDB89610B', '2012-09-30 08:10:38', 1, 70, 2, 0, NULL, NULL, 'http://piwik.net/blog/category/meta/', 0, 0, 0, 1, NULL, NULL, 1, '', '', NULL, NULL, 'jp', 36.000000, 138.000000, NULL, 5, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL);

@tsteur
Copy link
Member Author

tsteur commented Jan 27, 2020

Be good if someone could have a look at this. I haven't updated the UI tests again so it's easier to see the difference.

I've been looking into this for quite some time and it looks like there are sometimes manual goal conversions that have no idlink_va and they are now shown after the pageview instead of before the pageview. I reckon that makes more sense actually when the time is the same.

Eg see this visit info:

[{"type":"1","url":"example.org\/","url_prefix":"0","pageTitle":"home","pageIdAction":"2","idpageview":"81adad","idlink_va":"44","serverTimePretty":"2010-02-05 16:58:33","pageId":"44","custom_float":"251","interaction_position":"1","eventType":null,"eventCategory":null,"eventAction":null,"custom_var_k1":null,"custom_var_v1":null,"custom_var_k2":null,"custom_var_v2":null,"custom_var_k3":null,"custom_var_v3":null,"custom_var_k4":null,"custom_var_v4":null,"custom_var_k5":null,"custom_var_v5":null,"contentName":null,"contentPiece":null,"contentTarget":null,"contentInteraction":null,"bandwidth":"12053","custom_dimension_1":null,"custom_dimension_2":null,"custom_dimension_3":null,"custom_dimension_4":null,"custom_dimension_5":null,"timeSpent":"180","timeSpentPretty":"3 min 0s"},{"type":"13","url":"http:\/\/example.org\/","url_prefix":null,"pageTitle":null,"pageIdAction":"3","idpageview":"81adad","idlink_va":"45","serverTimePretty":"2010-02-05 16:58:33","timeSpentRef":"0","pageId":"45","custom_float":null,"interaction_position":"1","eventType":null,"eventCategory":null,"eventAction":null,"custom_var_k1":null,"custom_var_v1":null,"custom_var_k2":null,"custom_var_v2":null,"custom_var_k3":null,"custom_var_v3":null,"custom_var_k4":null,"custom_var_v4":null,"custom_var_k5":null,"custom_var_v5":null,"contentName":"product slider","contentPiece":"product_16.jpg","contentTarget":"http:\/\/example.org\/product16","contentInteraction":null,"bandwidth":"12053","custom_dimension_1":null,"custom_dimension_2":null,"custom_dimension_3":null,"custom_dimension_4":null,"custom_dimension_5":null},{"type":"10","url":"example.org\/","url_prefix":"0","pageTitle":null,"pageIdAction":"16","idpageview":"81adad","idlink_va":"46","serverTimePretty":"2010-02-05 17:01:33","pageId":"46","custom_float":null,"interaction_position":"1","eventType":"10","eventCategory":"product slider","eventAction":"next","custom_var_k1":null,"custom_var_v1":null,"custom_var_k2":null,"custom_var_v2":null,"custom_var_k3":null,"custom_var_v3":null,"custom_var_k4":null,"custom_var_v4":null,"custom_var_k5":null,"custom_var_v5":null,"contentName":null,"contentPiece":null,"contentTarget":null,"contentInteraction":null,"bandwidth":"12053","custom_dimension_1":null,"custom_dimension_2":null,"custom_dimension_3":null,"custom_dimension_4":null,"custom_dimension_5":null,"timeSpent":"72","timeSpentPretty":"1 min 12s"},{"type":"13","url":"http:\/\/example.org\/","url_prefix":null,"pageTitle":null,"pageIdAction":"3","idpageview":"81adad","idlink_va":"47","serverTimePretty":"2010-02-05 17:01:33","timeSpentRef":"0","pageId":"47","custom_float":null,"interaction_position":"1","eventType":null,"eventCategory":null,"eventAction":null,"custom_var_k1":null,"custom_var_v1":null,"custom_var_k2":null,"custom_var_v2":null,"custom_var_k3":null,"custom_var_v3":null,"custom_var_k4":null,"custom_var_v4":null,"custom_var_k5":null,"custom_var_v5":null,"contentName":"product slider","contentPiece":"product_42.jpg","contentTarget":"http:\/\/example.org\/product42","contentInteraction":null,"bandwidth":"12053","custom_dimension_1":null,"custom_dimension_2":null,"custom_dimension_3":null,"custom_dimension_4":null,"custom_dimension_5":null},{"type":"13","url":"http:\/\/example.org\/","url_prefix":null,"pageTitle":null,"pageIdAction":"3","idpageview":"81adad","idlink_va":"48","serverTimePretty":"2010-02-05 17:03:21","timeSpentRef":"108","pageId":"48","custom_float":null,"interaction_position":"1","eventType":null,"eventCategory":null,"eventAction":null,"custom_var_k1":null,"custom_var_v1":null,"custom_var_k2":null,"custom_var_v2":null,"custom_var_k3":null,"custom_var_v3":null,"custom_var_k4":null,"custom_var_v4":null,"custom_var_k5":null,"custom_var_v5":null,"contentName":"product slider","contentPiece":"product_42.jpg","contentTarget":"http:\/\/example.org\/product42","contentInteraction":"click","bandwidth":"12053","custom_dimension_1":null,"custom_dimension_2":null,"custom_dimension_3":null,"custom_dimension_4":null,"custom_dimension_5":null},{"type":"1","url":"example.org\/product42","url_prefix":"0","pageTitle":"product 42","pageIdAction":"23","idpageview":"41d466","idlink_va":"49","serverTimePretty":"2010-02-05 17:04:33","pageId":"49","custom_float":"83","interaction_position":"2","eventType":null,"eventCategory":null,"eventAction":null,"custom_var_k1":null,"custom_var_v1":null,"custom_var_k2":"_pkp","custom_var_v2":"60","custom_var_k3":"_pks","custom_var_v3":"P42X4D","custom_var_k4":"_pkn","custom_var_v4":"product 42","custom_var_k5":"_pkc","custom_var_v5":"software","contentName":null,"contentPiece":null,"contentTarget":null,"contentInteraction":null,"bandwidth":"36053","custom_dimension_1":"NZD","custom_dimension_2":null,"custom_dimension_3":null,"custom_dimension_4":null,"custom_dimension_5":null,"timeSpent":"144","timeSpentPretty":"2 min 24s"},{"type":"3","url":"http:\/\/example.org\/product42\/productsheet.pdf","url_prefix":null,"pageTitle":null,"pageIdAction":"24","idpageview":"41d466","idlink_va":"50","serverTimePretty":"2010-02-05 17:06:57","pageId":"50","custom_float":null,"interaction_position":"2","eventType":null,"eventCategory":null,"eventAction":null,"custom_var_k1":null,"custom_var_v1":null,"custom_var_k2":null,"custom_var_v2":null,"custom_var_k3":null,"custom_var_v3":null,"custom_var_k4":null,"custom_var_v4":null,"custom_var_k5":null,"custom_var_v5":null,"contentName":null,"contentPiece":null,"contentTarget":null,"contentInteraction":null,"bandwidth":"36053","custom_dimension_1":null,"custom_dimension_2":null,"custom_dimension_3":null,"custom_dimension_4":null,"custom_dimension_5":null,"timeSpent":"180","timeSpentPretty":"3 min 0s"},{"type":"ecommerceOrder","orderId":"X42FCY","revenue":140,"revenueSubTotal":120,"revenueTax":20,"revenueShipping":"6.90","revenueDiscount":"6.90","items":"2","serverTimePretty":"2010-02-05 17:10:33","idlink_va":null,"idpageview":null,"itemDetails":[{"itemSKU":"P42X4D","itemName":"product 42","itemCategory":"software","price":60,"quantity":"2","categories":["software"]}]},{"type":"1","url":"example.org\/search","url_prefix":"0","pageTitle":"product search","pageIdAction":"28","idpageview":"751bc7","idlink_va":"51","serverTimePretty":"2010-02-05 17:13:33","pageId":"51","custom_float":"318","interaction_position":"3","eventType":null,"eventCategory":null,"eventAction":null,"custom_var_k1":null,"custom_var_v1":null,"custom_var_k2":null,"custom_var_v2":null,"custom_var_k3":null,"custom_var_v3":null,"custom_var_k4":null,"custom_var_v4":null,"custom_var_k5":null,"custom_var_v5":null,"contentName":null,"contentPiece":null,"contentTarget":null,"contentInteraction":null,"bandwidth":"2583","custom_dimension_1":null,"custom_dimension_2":null,"custom_dimension_3":null,"custom_dimension_4":null,"custom_dimension_5":null,"timeSpent":"144","timeSpentPretty":"2 min 24s"},{"type":"8","url":null,"url_prefix":null,"pageTitle":"fancy product","pageIdAction":null,"idpageview":"751bc7","idlink_va":"52","serverTimePretty":"2010-02-05 17:15:57","pageId":"52","custom_float":"318","interaction_position":"4","eventType":null,"eventCategory":null,"eventAction":null,"custom_var_k1":null,"custom_var_v1":null,"custom_var_k2":null,"custom_var_v2":null,"custom_var_k3":null,"custom_var_v3":null,"custom_var_k4":null,"custom_var_v4":null,"custom_var_k5":"_pk_scount","custom_var_v5":"12560","contentName":null,"contentPiece":null,"contentTarget":null,"contentInteraction":null,"bandwidth":"2583","custom_dimension_1":null,"custom_dimension_2":null,"custom_dimension_3":null,"custom_dimension_4":null,"custom_dimension_5":null,"timeSpent":"108","timeSpentPretty":"1 min 48s"},{"type":"10","url":"example.org\/search","url_prefix":"0","pageTitle":"enormous category","pageIdAction":"30","idpageview":"751bc7","idlink_va":"53","serverTimePretty":"2010-02-05 17:17:45","pageId":"53","custom_float":null,"interaction_position":"4","eventType":"10","eventCategory":"search","eventAction":"filter","custom_var_k1":null,"custom_var_v1":null,"custom_var_k2":null,"custom_var_v2":null,"custom_var_k3":null,"custom_var_v3":null,"custom_var_k4":null,"custom_var_v4":null,"custom_var_k5":null,"custom_var_v5":null,"contentName":null,"contentPiece":null,"contentTarget":null,"contentInteraction":null,"bandwidth":"2583","custom_dimension_1":null,"custom_dimension_2":null,"custom_dimension_3":null,"custom_dimension_4":null,"custom_dimension_5":null,"timeSpent":"108","timeSpentPretty":"1 min 48s"},{"type":"8","url":null,"url_prefix":null,"pageTitle":"fancy product","pageIdAction":null,"idpageview":"751bc7","idlink_va":"54","serverTimePretty":"2010-02-05 17:19:33","pageId":"54","custom_float":"318","interaction_position":"5","eventType":null,"eventCategory":null,"eventAction":null,"custom_var_k1":null,"custom_var_v1":null,"custom_var_k2":null,"custom_var_v2":null,"custom_var_k3":null,"custom_var_v3":null,"custom_var_k4":"_pk_scat","custom_var_v4":"enormous category","custom_var_k5":"_pk_scount","custom_var_v5":"13","contentName":null,"contentPiece":null,"contentTarget":null,"contentInteraction":null,"bandwidth":"2583","custom_dimension_1":null,"custom_dimension_2":null,"custom_dimension_3":null,"custom_dimension_4":null,"custom_dimension_5":null,"timeSpent":"0","timeSpentPretty":"0s"},{"type":"goal","goalName":"Successfully used Search","goalId":"1","idpageview":null,"revenue":"0","idlink_va":null,"goalPageId":null,"serverTimePretty":"2010-02-05 17:21:21","url":"http:\/\/example.org\/search"},{"type":"1","url":"example.org\/fancyproduct","url_prefix":"0","pageTitle":"fancy product","pageIdAction":"35","idpageview":"a2d57a","idlink_va":"55","serverTimePretty":"2010-02-05 17:21:21","pageId":"55","custom_float":"151","interaction_position":"6","eventType":null,"eventCategory":null,"eventAction":null,"custom_var_k1":null,"custom_var_v1":null,"custom_var_k2":"_pkp","custom_var_v2":"40","custom_var_k3":"_pks","custom_var_v3":"F4NCYX","custom_var_k4":"_pkn","custom_var_v4":"fancy product","custom_var_k5":"_pkc","custom_var_v5":"software","contentName":null,"contentPiece":null,"contentTarget":null,"contentInteraction":null,"bandwidth":"68895","custom_dimension_1":null,"custom_dimension_2":null,"custom_dimension_3":null,"custom_dimension_4":null,"custom_dimension_5":null,"timeSpent":"0","timeSpentPretty":"0s"},{"type":"ecommerceAbandonedCart","revenue":140,"items":"2","serverTimePretty":"2010-02-05 17:22:33","idlink_va":null,"idpageview":null,"itemDetails":[{"itemSKU":"F4NCYX","itemName":"fancy product","itemCategory":"software","price":40,"quantity":"2","categories":["software"]}]},{"type":"1","url":"example.org\/cart","url_prefix":"0","pageTitle":"cart","pageIdAction":"37","idpageview":"01827a","idlink_va":"56","serverTimePretty":"2010-02-05 17:22:33","pageId":"56","custom_float":"33","interaction_position":"7","eventType":null,"eventCategory":null,"eventAction":null,"custom_var_k1":null,"custom_var_v1":null,"custom_var_k2":null,"custom_var_v2":null,"custom_var_k3":null,"custom_var_v3":null,"custom_var_k4":null,"custom_var_v4":null,"custom_var_k5":null,"custom_var_v5":null,"contentName":null,"contentPiece":null,"contentTarget":null,"contentInteraction":null,"bandwidth":"1590","custom_dimension_1":null,"custom_dimension_2":null,"custom_dimension_3":null,"custom_dimension_4":null,"custom_dimension_5":null,"timeSpent":"360","timeSpentPretty":"6 min 0s"},{"type":"2","url":"http:\/\/vendor.site","url_prefix":null,"pageTitle":null,"pageIdAction":"38","idpageview":"01827a","idlink_va":"57","serverTimePretty":"2010-02-05 17:28:33","pageId":"57","custom_float":null,"interaction_position":"7","eventType":null,"eventCategory":null,"eventAction":null,"custom_var_k1":null,"custom_var_v1":null,"custom_var_k2":null,"custom_var_v2":null,"custom_var_k3":null,"custom_var_v3":null,"custom_var_k4":null,"custom_var_v4":null,"custom_var_k5":null,"custom_var_v5":null,"contentName":null,"contentPiece":null,"contentTarget":null,"contentInteraction":null,"bandwidth":"1590","custom_dimension_1":null,"custom_dimension_2":null,"custom_dimension_3":null,"custom_dimension_4":null,"custom_dimension_5":null}]

image

You can see the difference in this test: https://builds-artifacts.matomo.org/matomo-org/matomo/4.x-dev/37998/Live_visitor_log_expand_pageview_actions.png

@tsteur tsteur requested a review from sgiehl January 27, 2020 02:59
@millnut
Copy link

millnut commented Jan 27, 2020

@tsteur what if we used uasort? and if idlink_va is null put before?

@tsteur
Copy link
Member Author

tsteur commented Jan 27, 2020

Cheers @sgiehl it looks like the screenshot tests are now passing.

@limitstudios would uasort work well for arrays with numeric indexes such as here:

image

In general here it doesn't seem needed as the tests hopefully pass now and for end users it wouldn't be too critical which one be listed first. Based on the listed properties they should now be mostly sorted.

@millnut
Copy link

millnut commented Jan 29, 2020

@tsteur I believe it should although haven't tested it. If this works I would say leave as is.

Slightly offtopic but I'm currently looking at the SystemTestsPlugins failures at the moment to get the tests green before I start some other changes as well as adding PHP 7.4 to the tests but I'll create a PR for that later

@tsteur tsteur merged commit 3b836fa into 4.x-dev Jan 30, 2020
@tsteur tsteur deleted the fixbuildsort branch January 30, 2020 20:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Needs Review PRs that need a code review not-in-changelog For issues or pull requests that should not be included in our release changelog on matomo.org.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants