Migrations that have not been part of a previous release and have a version other than the current core version should cause the test build to fail. This should happen even if the migration was committed with the version of the current release, but the current release version subsequently changes.
Migrations added with a version other than the current release are executed by the current test setup and will pass all test builds, but after release will not be executed when users upgrade.
Users then end up with an updated version of Matomo without the required migration changes which usually breaks some or all application functionality. There is no easy way to correct this without releasing a new version or asking users to run database queries.
Create a github action that runs on every pull request.
The action would then simply fetch all already released versions of the repo and check if update scripts that have been added are newer than the last released version. If that isn’t the case, the action would fail.
1) A developer commits a migration labelled for a previous version. (eg.
4.12.1 when the next release target version will be
4.12.2). This is easily done as the future version is usually unknown at the time the migration is written and may potentially change multiple times during the PR lifecycle.
2) The PR test builds pass because the core version is still
4.12.1 in development, so the migration is applied. The PR is merged into core (or plugin via sub-module update merge)
3) At release time the core version is changed to
4.12.2 ready for release.
4) Any final test build done just before release will still pass because the test setup starts with a historic omnifixture (db snapshot) and then runs all migration scripts, regardless of version. So the incorrectly labelled migration is still run and the tests pass.
5) The version is released.
6) Users already running
4.12.1 update to
4.12.2, but the migration is not run as it is assigned to
4.12.1 and the user update process will only run migrations labelled for versions larger than the version being upgraded (