Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Prefix all our vendor libs #16905

Closed
tsteur opened this issue Dec 8, 2020 · 24 comments
Closed

Prefix all our vendor libs #16905

tsteur opened this issue Dec 8, 2020 · 24 comments
Labels
wontfix If you can reproduce this issue, please reopen the issue or create a new one describing it.

Comments

@tsteur
Copy link
Member

tsteur commented Dec 8, 2020

see matomo-org/matomo-for-wordpress#233

Need to do this in a major release. Should investigate if we can make this work. We'd only prefix vendor libraries. Could use Matomo or MatomoLib or something as prefix. eg Di\object then becomes Matomo\Di\object. To have it actually useful in development too we would put this into the composer json similar to google/site-kit-wp#612 see eg https://github.com/google/site-kit-wp/pull/730/files

More details in #233

@tsteur tsteur added the Task Indicates an issue is neither a feature nor a bug and it's purely a "technical" change. label Dec 8, 2020
@tsteur tsteur added this to the 5.0.0 milestone Dec 8, 2020
@tsteur
Copy link
Member Author

tsteur commented May 10, 2022

This could be a lot of work and cause a lot of breaking changes.

It will save a lot of time in the long run as Matomo for WordPress will be more stable and have less compatibility issues. The effort it takes to do this one, might not be worth it though. Depends though how much work it really is?

@jane-twizel
Copy link

Will check the level of work needed before confirming we will do it in 5.0.0

@tsteur
Copy link
Member Author

tsteur commented Jul 8, 2022

Using the proposed logic, is it possible to only prefix some vendor libraries to reduce the effort / amount of work on this issue so that we get a good ration on effort vs value? Or would above solution only be able to prefix all vendor libraries?

The libraries that cause the most issues are typically

  • Twig
  • Monolog
  • PHP-DI
  • Psr/Container
  • Psr/log

The biggest library that's causing issues is likely Twig.

The goal would be to keep breaking changes small so we need to change less code and reduce the amount of possible regressions etc. How much effort would it be to implement this feature? And if it is a lot of effort, is there a way to reduce the effort significantly while still getting a lot of the value?

@diosmosis
Copy link
Member

@tsteur I looked into this issue and I couldn't think of another way to accomplish it than using php-scoper (specifically I couldn't find a simple, dynamic solution). Since a dynamic, runtime solution isn't possible, I think the best alternative is automation, specifically:

  • A command that runs php-scoper to scope composer dependencies, either in core or in a plugin. Run as a composer post install command like in the site kit repo you linked.
  • A command that replaces unscoped use declarations for those dependencies to the correctly prefixed use statements (assuming php-scoper isn't capable of this itself). Also used on either core or a plugin.

And for BC:

  • An autoloader that can be turned on/off that checks for unprefixed uses (ie, Twig\) and loads the matomo prefixed version if encountered (ie, Matomo\Dependencies\Twig\). This would be turned on by default, and in matomo for wordpress would be turned off.

What do you think? Are these deliverables sound? If so I can start working on this.

@tsteur
Copy link
Member Author

tsteur commented Aug 29, 2022

A command that runs php-scoper to scope composer dependencies, either in core or in a plugin. Run as a composer post install command like in the site kit repo you linked.

👍

A command that replaces unscoped use declarations for those dependencies to the correctly prefixed use statements (assuming php-scoper isn't capable of this itself). Also used on either core or a plugin.

Sounds good. Although not sure how many usages are used and if it could be done manually. If this worked automatically that be awesome.

And for BC:

If we include this in Matomo 5, maybe we could skip the BC part? Although maybe this is needed during the update process when someone updates from an older Matomo On-Premise version when some files are already loaded and try to use the old naming and then can't find the file.

I'm mostly worried about this that plugins then work with On-Premise but not with Matomo for WordPress because in a plugin it was missed to update a use statement and it's hard to find these errors. Most developers etc wouldn't notice the problem but WordPress users then run into the problem. Not sure it's clear what I mean?

@sgiehl
Copy link
Member

sgiehl commented Aug 29, 2022

I really doubt that a script to automatically prefix any vendor class namespace is a solution we should go with. That will always have the risk that updating a vendor lib could break everything. There are a lot cases that might never be detectable using a script. Like dynamic instances of classes built from string variables or when class names are compared in the code and similar stuff.
This is for sure a lot easier to handle for small wordpress plugins that only have a couple of dependencies. But imho we have a way too many dependencies...
Note: Some of our dependencies are not even using namespaces, so you would need to prefix the class name there instead.

I'm really not sure if the benefits of doing that are higher than the pain we might have one day after doing that.

@diosmosis
Copy link
Member

@tsteur

If we include this in Matomo 5, maybe we could skip the BC part? Although maybe this is needed during the update process when someone updates from an older Matomo On-Premise version when some files are already loaded and try to use the old naming and then can't find the file.

This was mostly just for third party plugin developers that may not update their code. I suppose if it's not important those plugins just automatically work. (Though from @sgiehl's comment it seems this sort of autoloader could have another potential use, see below.)

@sgiehl

That will always have the risk that updating a vendor lib could break everything.

How often are composer libraries updated? Would tests fail if updating a vendor lib caused a problem?

Like dynamic instances of classes built from string variables or when class names are compared in the code and similar stuff.

This could certainly be an issue. From the php-scoper docs it says it has "limited support" for changing such code. In this case the BC autoloader would be useful for making sure things still work in that case (it would be disabled in tests).

Note: Some of our dependencies are not even using namespaces, so you would need to prefix the class name there instead.

I don't know how php-scoper exactly works, but couldn't it potentially change \MyClass to \Matomo\Dependencies\MyClass?

@sgiehl
Copy link
Member

sgiehl commented Aug 29, 2022

How often are composer libraries updated? Would tests fail if updating a vendor lib caused a problem?

Patch and in some cases minor releases are updated automatically when they are released. We have a action to run a composer update weekly. Everything that needs to be changed in composer.json is done manually when needed. But major version updates are most likely only done with major Matomo releases.

This could certainly be an issue. From the php-scoper docs it says it has "limited support" for changing such code. In this case the BC autoloader would be useful for making sure things still work in that case (it would be disabled in tests).

That could work for Matomo itself. But it might produce the same issues we are trying to solve for Matomo for Wordpress, as it might load the libs from another plugin if the un-prefixed names are used.

I don't know how php-scoper exactly works, but couldn't it potentially change \MyClass to \Matomo\Dependencies\MyClass?

That depends if php-scoper is able to handle that. But for e.g. the less lib we use, I guess that would fail, as they ship a custom autoloader and I doubt that can be easily rewritten. See https://github.com/wikimedia/less.php/blob/main/lib/Less/Autoloader.php

@tsteur
Copy link
Member Author

tsteur commented Aug 29, 2022

Note that I believe we'd get the biggest benefit by looking at the libs in #16905 (comment) and we don't need to replace all of them if that makes things easier. It wouldn't need to work for the less lib from my perspective as this library is used more rarely by other plugins. Even only changing the namespace for twig and/or Monolog could be a win.

Most incompatibilities seem to come from Twig and monolog see https://matomo.org/faq/wordpress/which-plugins-is-matomo-for-wordpress-known-to-be-not-compatible-with/ but we also have quite a few cases around PHP-DI.

PHP-Scoper should for example prefix a \Matomo namespace to all their classes. We would maybe need to test how well it does that? If it doesn't work right out of the box already now and tests fail to start then it's an easy decision not to do it?

I assume if we prefix for example php-di and other dependencies want to use PHP-DI without a prefix, then it could maybe cause an issue for example. So maybe it would only work for twig and monolog. If it doesn't replace all the namespaces correctly then of course this be a problem and it creates more problems than it solves. And it could be also a problem in the future if there's a change in a library that php-scoper can't replace automatically.

@diosmosis
Copy link
Member

@tsteur So is it worth trying now with twig/monolog? If so I'll get started.

@tsteur
Copy link
Member Author

tsteur commented Aug 29, 2022

be great to try it from my perspective 👍

@sgiehl
Copy link
Member

sgiehl commented Aug 30, 2022

If we limit this to only a couple of libs and don't prefix all our libs (as the title says) it might be feasible.

@diosmosis diosmosis self-assigned this Aug 30, 2022
@tsteur
Copy link
Member Author

tsteur commented Sep 5, 2022

FYI just now we got a report that WP-Rocket started using Monolog 2 and we're now incompatible with it see https://wordpress.org/support/topic/archive-error-with-wp-rocket/ .

WP-Rocket is the 7th most popular WordPress plugin and we're using it as well (https://en.wikipedia.org/wiki/WP_Rocket#Usage) and runs on millions of sites (source)

Updating our monolog to version 2 wouldn't help much either as we then become incompatible with many other popular plugins.

@sgiehl
Copy link
Member

sgiehl commented Sep 6, 2022

@tsteur Can't we make Matomo compatible with both monolog versions? I guess there are some changes we would need to apply to make Matomo compatible, but we could maybe try to detect the monolog version being used and use different code in Matomo then.

@tsteur
Copy link
Member Author

tsteur commented Sep 6, 2022

I don't know if we can and what that would mean. I guess that would then also involve scanning all WordPress plugins to see what monolog version which plugin uses?

@sgiehl
Copy link
Member

sgiehl commented Sep 7, 2022

I haven't had a look at the required changes to update Monolog to version 2 in our codebase, but my idea was, to simply check the monolog version using Monolog\Logger::API and use different code in our monolog plugin based on that. If you want, I can give it a try for a couple of hours to check if we could implement a quick fix for monolog 2 being used instead of monolog 1.

@diosmosis
Copy link
Member

@tsteur I do not want to complicate this thread any more than it is already, but in case it's worth knowing, I should have a proof of concep of a prefixing workflow (with twig) for evaluation by end of week at the latest.

@tsteur
Copy link
Member Author

tsteur commented Nov 12, 2022

@diosmosis we discussed it would be great to also prefix PHP-DI if any possible. Because namespaced functioned can't be aliased (like class_alias), we'll then need a wrapper like Matomo\DI\get() for these DI functions. In these wrapper classes we might need to do once a check if class_exists(\Matomo\Depencency\DI) to know if we need to call the prefixed function or when it's used via composer we may need to call the regular function. Does that make sense? Could you try to mark PHP-DI as prefixed as well and add the Matomo\DI class for the few functions we need to wrap?

We'll then test the composer install that things still work nicely when Matomo is installed via composer.
We'll also need to test an update from Matomo 2.X to Matomo 5.0. Back then in Matomo 2.x we hadn't yet done a redirect after the new files were installed and still a few methods were called after the files were replaced.

@mattab mattab added the 5.0.0 label Jan 4, 2023
@tsteur
Copy link
Member Author

tsteur commented Jan 23, 2023

Ping @diosmosis

@diosmosis
Copy link
Member

@tsteur this has been done for a while now

@tsteur
Copy link
Member Author

tsteur commented Jan 24, 2023

@mattab @sgiehl that means this one is ready for another review.

@sgiehl sgiehl removed this from the 5.0.0 milestone May 10, 2023
@sgiehl sgiehl removed the 5.0.0 label May 10, 2023
@schwarzpunkt
Copy link

Has the conflict between Matomo and WP Rocket been resolved with version 5.0?

@michalkleiner
Copy link
Contributor

It looks like it might be compatible now, @schwarzpunkt, but not because of Matomo 5. Best to check/keep an eye on the Matomo for Wordpress issue #1010.

@sgiehl
Copy link
Member

sgiehl commented Feb 6, 2024

Closing this issue here, as we won't prefix the vendor libs in Matomo.
Matomo for wordpress should meanwhile use a prefixed Matomo version, which hopefully should fix those incompatibilities. See matomo-org/matomo-for-wordpress#1022

@sgiehl sgiehl closed this as completed Feb 6, 2024
@sgiehl sgiehl added wontfix If you can reproduce this issue, please reopen the issue or create a new one describing it. and removed Task Indicates an issue is neither a feature nor a bug and it's purely a "technical" change. labels Feb 6, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
wontfix If you can reproduce this issue, please reopen the issue or create a new one describing it.
Projects
None yet
7 participants