Scheduled reports (PDF, HTML, SMS) are currently not part of the integration test suite.
Questions to address :
Feedback
Thanks for looking into it: it always made me uneasy that HTML/PDF reports were not tested. Now that we also have SMS reports, and that the 2 plugins are loosely coupled, it is very good improvement to our QA to add these tests!
which information should be removed, e.g: dates
Dates should stay since they won't/shouldn't change from run to run.
which period(s) should be tested
I'd say the period with the most data so that code coverage is maximum. For the test OneVisitorTwoVisits visits are on the same day so period=day is good enough.
(In [6849]) refs #3323 #3088 #2708 #71 #2318
(In [6883]) refs #3323
removing hard coded list of reports
TODO: expected integration files need to be updated, pending GD discrepancy issues
(In [6932]) refs #3323 - include images in scheduled reports only if system under test matches some technical characteristics of the Piwik QA Server
TODO
Is there a reason why you made setUpScheduledReports non-static and moved the call to a seperate test case? I guess it was more correct before, as reports should be setup before running the tests (it's more part of the setup than of a test case). Btw: now it won't be possible only to run the api tests
(In [6936]) refs #3323 follow best practices per comment:6:ticket:3323
Do tell me if it's better like that. The idea is to warn the developer that some tests are being skipped because of system incompatibilities.
(In [6946]) refs #3323 ignoring .html, .pdf and *.txt files in processed directory
Looks good like that. I'm wondering if we should really test for images at all now, since most developers won't have images generated, but it could be helpful.. Can the ticket be closed?
static png graphs will be tested on the integration server
this insures no regression will occur, especially since we are likely to refactor underlying libraries (pChart)
(In [7557]) refs #3323 updating qa server specs for inclusion of static images in integration testing
(In [7604]) Refs #3323 Scheduled Reports integration tests until they are generate in expected/ with the static images same features as Jenkins build
(In [7613]) refs #3323
(In [7675]) refs #3323 some ubuntu precise simply output ubuntu, lets try to widen the boxes that can run static images in tests
PS: all pdf tests on my box!
In f387a8914484fca089bd221bac660dd88909ec56: refs #3323 enable static png images in integration tests across additional php versions
In 824eceda83ccaaa96291301c81d90c2f6b2d6865: refs #3323 remove unused expected integration files
In 545c9998b75a38739e761b9157e988cf4aab77bd: refs #3323 enable static png images in integration tests for travis-ci
In efa5344a7f0618997d2a60edcb0bfb8c7d82c957: refs #3323 exclude static png graph tests in travis-ci until I can set-up travis-ci gd version on my box
thanks, travis is now green again :)
It will be good to use gd 2.0.34: I just realized most servers in productions are using this version.
Julien, after talking to Fabian, I think we should use phpenv on our dev boxes. Would you like to give it a try? I will also try to install it but if you do it first, thanks for posting here any instructions for ubuntu (which we can then add in the tests/README).
In 18974f8405c4462392677599ab7de24fe4deb4c2: refs #3323 doc
In b993d279014b941611eb3fd9abca51bc8cfe7fa5: refs #3323 test for one row evolution based png graph