@tsteur opened this Issue on November 11th 2014 Member

Currently, it is possible to execute tests by "testsuite", "group" and "file/dir":

./console tests:run --testsuite unit --file tests/PHPUnit/file.php groupName

The argument is currently interpreted as a group. I'd suggest to make this argument a bit smarter.

  • [x] If argument is unit, integration, system, plugins or core interpret it as "--testsuite". This allows us to run a testsuite like ./console tests:run unit
  • [x] If argument is a plugin name run all tests within this plugin eg ./console tests:run Insights. It would be nice if lower case was possible as well.
  • [ ] If arguments ends with .php search for a while having this name in tests folder eg ./console tests:run Widgets.php. To be defined if multiple files have this name. Can we specify multiple files in PHPUnit maybe? --> NO. Maybe instead let the user choose which one should be executed (eg enter number: [1] foo/bar.php, [2] bar/bar.php)
  • [x] If arguments starts with tests/PHPUnit, plugins/ or if file_exists interpret it as file / directory
  • [ ] If nothing matches do not execute anything? Or shall we interpret it as --group? I'd prefer the first one as it could be possible that there is eg a group having the given name but you thought you were executing plugin tests or whatever. Eg if there is a typo etc.
  • [x] It is to be defined how we handle multiple parameters. Eg would ./console tests:run unit Insights run all unit tests of the insights plugin? Could be nice
  • [ ] document internal change in Changelog or write in dev chat
  • [ ] remove no longer needed groups from tests (eg the ones having a plugin name, 'core', 'plugins', etc)

If someone specifies a --testsuite option and a argument named unit we should maybe interpret the argument as group? Same for --file etc.

This will allow us to remove some current groups. Executing groups is not so useful currently as they are not always set and very inconsistent. Eg not all tests in plugins have the group name set. This way we kinda gain "dynamical groups" that actually work.

As this adds some kind of magic and someone might still want to execute a group "unit" or "Insights" we should add another option --group so people can still avoid the magic and it will be more consistent with --testsuite and --file and with PHPUnit command line parameters.

@mnapoli commented on November 11th 2014 Contributor

Related, but I didn't want to derail the topic: #6634

Except that, +1 with your suggestion, running tests is a little challenge everytime ;)

@mattab commented on November 11th 2014 Member

All ideas are excellent - we will all appreciate!

document teal change in Changelog or write in dev chat

+1 for documenting in changelog maybe by pasting/reusing the help text of the command.

Powered by GitHub Issue Mirror