Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Make UI tests work on all servers #6558

Open
tsteur opened this issue Oct 30, 2014 · 4 comments
Open

Make UI tests work on all servers #6558

tsteur opened this issue Oct 30, 2014 · 4 comments
Labels
c: Tests & QA For issues related to automated tests or making it easier to QA & test issues. Task Indicates an issue is neither a feature nor a bug and it's purely a "technical" change.
Milestone
Backlog (Help wan...

Comments

@tsteur
Copy link
Member

tsteur commented Oct 30, 2014

Currently, some screenshot tests containing the PIWIK_INCLUDE_PATH which makes the tests fail on some servers if for instance not the same linux user is used as on travis. Solution: See #6429 as suggested by @diosmosis we could grep the html output for PiwikIncludePath and replace it with '/' for instance.

Currently failing tests on AWS

Installation_system_check
UIIntegrationTest_admin_manage_tracking_code
UIIntegrationTest_admin_plugins Expected    
UIIntegrationTest_admin_security_info   
UIIntegrationTest_admin_visitor_generator
UIIntegrationTest_db_connect_error
Updater_main

System check and security infos both fail as well. Not sure how to make them work on all servers. Probably we'll have to "inject/mock" the security result in PHP layer to see if it is rendered well. For this we could use DI. Mocking it is fine as we are only interested in testing the Controller / template / CSS / JS layer anyway. The actual logic of the security check should be tested in Unit/Integration tests.
@tsteur tsteur added the Task Indicates an issue is neither a feature nor a bug and it's purely a "technical" change. label Oct 30, 2014
@diosmosis
Copy link
Member

If SystemCheck & SecurityInfo are migrated to use angularjs, we could use $httpBackend. Which would be closer to what I wanted for UI tests (ie, don't test PHP API too, just JS + UI related PHP code).

@mattab
Copy link
Member

mattab commented Oct 30, 2014

@diosmosis you won't easily convince me to skip testing PHP API in UI tests. IMO it's a major benefit of our screenshot tests :)

@diosmosis
Copy link
Member

Technically API should be tested by system + Integration tests (and maybe sometimes unit). Ideally, UI shouldn't. And since it currently loads data from a SQL dump that has been pre-archived, it doesn't really.

@tsteur
Copy link
Member Author

tsteur commented Oct 30, 2014

@diosmosis 👍 100% true. Such API's have to be covered by unit and integration tests and not by UI tests. UI tests should be only there to "test" Controller, Templates, CSS and partially JS . To be clear: If not, we waste a lot of time (and money) and end up having more bugs etc

@mattab mattab added this to the Short term milestone Nov 3, 2014
@mattab mattab added the c: Tests & QA For issues related to automated tests or making it easier to QA & test issues. label Nov 3, 2014
@mattab mattab modified the milestones: Short term, Mid term Apr 7, 2015
@mattab mattab modified the milestones: Long term, Mid term Dec 5, 2016
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
c: Tests & QA For issues related to automated tests or making it easier to QA & test issues. Task Indicates an issue is neither a feature nor a bug and it's purely a "technical" change.
Projects
None yet
Milestone
Backlog (Help wanted)
Development

No branches or pull requests
3 participants
@diosmosis @tsteur @mattab