This is related to #134, but is not duplicate. #134 is about bulk loading logs into Piwik, this ticket is a first step:
we can then use this API to build simple integration test.
For example, we can generate 2 visits with known keywords/resolution/website/countries, etc. Then call the API that will archive data and return reports. First time we would save the XML reports in some directory. Next times, the output would be compared to the saved XML files, and if there is a difference, test will fail. Developper can then manually check the diff between outputs and overwrite the 'expected' outputs if they are indeed expected (and so if the format/algorithms/data changed).
By using this technique, I hope that we can implement pretty easily a very good generic test coverage for Piwik, before the 1.0 release.
Attachment:
WrongidSite.png
Attachment:
Screenshot.png
(In [2430]) Refs #1446
in [2429]
Introducing documented piwik.php PHP Tracking client.
Also adding integration test suite in place.
* Adding new mechanism to trigger known tracking requests, then call all API methods get* and compare XML to the previous 'expected' XML.
* make sure overwritten config files are not read during tests. Use defaults from global.ini.php.
* piwik.php Tracker allow calling script to force Datetime and user IP for a specific request.
* Wrote 3 main tests
o wrong tracking requests should not return error and should not record anything
o standard one visitor, 2 visits, with page view, download, outlink and goal tracked
o Added integration test as an example in the ExampleAPI plugin (pretty simple to add testing code)
TODO
* Finish PiwikTracker and show it in UI
* Show how to use image based tracker in UI
* Add more tests (multi periods and multi sites) in Main.test.php
* Fix failing Config.test.php test (when ran via all_tests.php doesn't fail otherwise)
(In [2431]) Now testing various API exports for ExampleAPI plugins, for all API formats: xml, csv, tsv, json, etc.
Any breaking or modifying change in the renderer or data conversion should trigger a test fail.
Improved parameters in ExampleAPI API.php
Refs #1446
(In [2432]) Refs #1446 Fixing failing test due to Windows EOL and headers already sent error. Also not testing 'format=original'.
(In [2436]) Fixing typos in tests Refs #1446
(In [2441]) Refs #1446
(In [2442]) Fixing bug in Archiving found with this new test suite (max_actions was incorrectly summed during archiving, rather than max() operation)
Updating expected file as naming convention changed in previous commit
Refs #1446
(In [2453]) Fixes #1446
Regression: Getting wrong idsite in alternative tracking code for idsite > 1. Screenshot attached.
On a second view: Duplicate http:// showing in alternative tracking code.
(In [2457]) Refs #1446
Fixes idSite display and duplicate http://
(In [2472]) Refs #1446 Refs #818
Note: on my box, when I run all_tests.php, I don't see the green bar at the bottom of the page anymore. It simply displays the Time and Memory delta. I didn't see any commit that coudl have broken this so I'm a bit confused as to why the green/red bar doesn't display anymore?
installer is broken:
(In [2482]) Fixes #1446: translating step name + fixing smarty error
(In [2495]) Refs #1446
Integration tests were often different simply because the internal idsubdatatable in the response was different. However. this idsubtable is really just an internal ID and it is not expected that this ID is the same across several archiving processes. Therefore integration tests will now set a special flag to remove the ID from the response, to minimize noise.