Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Moby DIoC: Step 1 - Piwik environment encapsulation #7644

Merged
merged 67 commits into from May 7, 2015
Merged

Conversation

diosmosis
Copy link
Member

Call me Ishmael.

This pull requests provides logic to encapsulate the entire environment setup logic in a single class. As a proof of value, the Plugin\Manager has been successfully added to the DI container.

New Concepts

This PR introduces one new concept:

Kernel Objects

Currently in Piwik, most services are singletons, despite the presence of a DI container. Ideally, all of these singletons should be moved to the DI container, however, this is non-trivial since in order to create the container, some singletons are used (specifically Config & Plugin\Manager).

In order to decouple container creation from these singletons, I introduce the concept of Piwik's kernel, which is the core part of Piwik that cannot be overridden through DI (and thus by plugins). Currently, only two types of objects exist here: the PluginList & the GlobalSettingsProvider.

The PluginList currently returns the list of activated plugins, so the container can load plugin-specific config. The GlobalSettingsProvider provides config information used to create the container (and for the default implementation of PluginList).

These objects are created before the DI container is created, but they are placed in the container so other objects can access them w/o having to treat them as singletons.

Though these objects cannot be overridden through DI, it is my intent to allow this eventually, through Application encapsulation (see the next steps list).

List of changes

  1. Introduction of Environment class

    This class encapsulates the creation of kernel objects & then the DI container. It also encapsulates environment validation, which before, was in several places, including Config.php/FrontController.php/CronArchive.php

  2. Introduction of PluginList kernel object interface + IniPluginList implementation

    This class is used to create the container, and so cannot be created through DI.

    Rationale: I created a new interface for this functionality, as it could be useful later. When we modify plugins to be available through composer, we could create a new implementation to manage the activated plugin list using composer.json or another file.

  3. Introduction of GlobalSettingsProvider kernel object interface + IniSettingsProvider implementation

    This class is used to create the container, and so cannot be created through DI.

    Rationale: I created a new interface, because it might be useful to provide new implementations later. Eg, during the meetup, pro devs described needing to store most config values in the DB so propagating changes through a load balanced environment would be easier. Creating another implementation of this class (when it's possible to) could provide a possible solution.

  4. Moving all environment validation logic to EnvironmentValidator class (another kernel object).

    Currently in Piwik, Environment validation is done manually in Config, FrontController & CronArchive. This PR moves all that validation to a new class used by Environment. The EnvironmentValidator class will check if the global.ini.php/config.ini.php file is available, and based on the current execution environment (ie, tracker/cli/web), will display an error or start the installation process.

    There is also a system test that verifies this behaviour.

  5. Removed ability to change config files used by a Config instance

    In order to fix tests after this change and to allow INI files to be read before Config is created, I had to ensure that files used by Config couldn't change at any random point during test execution. Instead, one must create a new singleton instance. This is closer to overriding via DI, so should help in putting Config in DI.

    This change includes:

    • Deprecating init(), setTestEnvironment(), clear() and removing uses in core Piwik code.
    • Making reload() protected.
  6. Moved Config::encodeValues/Config::decodeValues to IniFileChain.

    Since the encoding/decoding logic is used to avoid some INI parsing bugs, it should be part of IniFileChain, not just the Piwik-specific Config class.

  7. Lots and lots of hacks to test code

    These changes are changes to TestingEnvironment.php, the introduction of TestConfig.php and changes to test case classes. These are all there just to make tests pass, and should be considered temporary. Eventually, I will clean up test code significantly.

  8. Tweaks to CronArchive.php environment setup

    CronArchive.php has some unnecessary code for environment setup. Now, since archiving is initiated through archive.php & through core:archive, the environment will already be setup, so there's no need to set the host and create the Config instance. I verified it works w/ --piwik-domain & w/ Host: ... HTTP header.

  9. Change to core\IP.php

    There's a small change here. For some reason, the error silencing on the statement @Config::getInstance()->General['...']; didn't work unless it was split into two statements.

  10. Plugin\Manager added to DI

    Not much to say about this. ::getInstance() is still there, as more objects are added to DI, we can perhaps remove it.

  11. Removed cli-script-bootstrap.php

    No longer used, so removed.

Next Steps

  1. Application Encapsulation.

    Involves encapsulating every entry point in a class that creates an Environment. For example, the tracker can be encapsulated in a TrackerApplication class that provides a public function track($params) method. The piwik.php file should then be just:

    require_once PIWIK_INCLUDE_PATH . '/core/Tracker/Application.php'; // handles autoload requiring + whatever else
    
    $tracker = new TrackerApplication();
    $tracker->track($_GET + $_POST, $_SERVER, ...);
    
  2. Moving Config class to DI & deprecating, merging with IniSettingsProvider

    If we merge the rest of Config logic w/ IniSettingsProvider, we will have one less singleton. Obviously, we can't get rid of Config because of it's pervasive use, but it would become just a view of IniSettingsProvider.

    Removing it completely will force cleaning up the test environment setup code (which is do-able, but I cannot think of a concrete solution w/o starting to work on it).

  3. Filling out PluginList description

    We can move other plugin list manipulation logic from Plugin\Manager to PluginList + IniPluginList to make it easier in the future to install plugins through Composer. We can potentially create an implementation that uses composer.json or another file.

Post-merge TODO

  • Get builds using CloudFixture to pass. I have a branch that does some of the work, but I haven't updated in a while. Waiting for this to be merged.
  • Add system test to make sure cron archiving (via cli + web) works when another piwik domain is used. Requires --dry-run parameter to CronArchive so tests don't become too slow.

diosmosis and others added 30 commits April 1, 2015 22:15
…ironment(). Replace last w/ mock config class, which is closer to use of DI.
The container now uses IniFileChain, which means it doesn't need the Config object anymore.
…Piwik kernel components. These objects are used to create the DI container and thus cannot be overidden through DI. Currently only used in DI, though both interfaces should be used by Plugin\Manager + Config.
…estCase base class + ability to override environment in UnitTestCase.
Conflicts:
	tests/UI/expected-ui-screenshots
…in is shared (again, regressed in a previous commit), and fixed many tests. Changed CacheTest from integration to unit.
…tic cache in Archive.php never being cleared, and the environment not existing in web archiving.
…thod since it is not required for this PR.
…nce and not reloaded w/ different files later.
…+ related file checking (both in code & in time (so done in one place in code & one time in execution)).
diosmosis added 11 commits April 28, 2015 00:18
…ecks if paths are supplied after singleton instance is created.
…construct to fix other tests & do not call assetManager::removeMergedAssets method in tearDown of test if it was not successfully created.
…create environment after setting custom config in setUp().
…Provider class, the class is the old IniSettingsProvider class.
@diosmosis
Copy link
Member Author

@mnapoli I merged IniPluginList w/ PluginList and IniSettingsProvider w/ GlobalSettingsProvider, do you think you could do another review? Not many changes from last time, though.

return;
}

Translate::loadAllTranslations();
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are you sure this is needed? I checked out the branch to give it a try and commenting this line doesn't seem to change anything, and it seems to make sense because the local (or global) config doesn't exist so… By the way IIRC core translations are always loaded (since the translator refactoring translations are loaded on demand)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe not, I'll have to check...

@mnapoli
Copy link
Contributor

mnapoli commented May 7, 2015

@diosmosis I reviewed the latest commits where you merged the interfaces/classes, I'm definitely +1 with that change. All this isn't public API anyway so we can always change it whenever we like later.

I've checked out the branch, had a look around. The diff is still huge, but I'm +1 for merging: the longer we wait the higher the probability is we never merge this. We have to merge it when we start 2.14 to be able to continue the work on this topic.

Just one thing: I don't remember if there was a reason to have IniPluginList and GlobalSettingsProvider as 2 separate classes. It seems (from the changes in the PR) that those 2 classes are actually really similar and that we need to break the encapsulation of GlobalsSettingsProvider to access IniPluginList which isn't the best. Would it make sense to merge them? We could also do it in a separate pull request to avoid adding more and more… (as always, it's not API, we can change it later)

Except that, needs a rebase (master is green for now which is great), green tests and then I think it's OK to merge.

@diosmosis
Copy link
Member Author

Re IniPluginList + GlobalSettingsProvider: This is mostly thinking ahead for composer. The list of plugins to load doesn't have to come from INI file forever. I don't know what you mean by 'break encapsulation', can you explain?

@mnapoli
Copy link
Contributor

mnapoli commented May 7, 2015

I don't know what you mean by 'break encapsulation', can you explain?

Code that uses GlobalSettingsProvider sometimes need to access the underlying IniPluginList object and does so by calling the getIniFileChain() getter: the globalsettingsprovider doesn't entirely encapsulate the inipluginlist object.

OK not a big deal, especially if it will be useful later. Let's merge asap.

@diosmosis
Copy link
Member Author

Code that uses GlobalSettingsProvider sometimes need to access the underlying IniPluginList object and does so by calling the getIniFileChain() getter: the globalsettingsprovider doesn't entirely encapsulate the inipluginlist object.

It's not supposed to encapsulate it. IniPluginList uses the GlobalSettingsProvider as its source, so IniPluginList depends on GlobalSettingsProvider which depends on the IniFileChain. At least that's how it's supposed to be. I haven't looked at the code in a while.

We can discuss this later though. I'll merge after the smaller changes you noted above.

@mnapoli
Copy link
Contributor

mnapoli commented May 7, 2015

Ahh sorry I meant IniFileChain instead of IniPluginList! ;)

@diosmosis
Copy link
Member Author

FYI, one other note: I can remove the singleton nature of IniFileChain after this step is finished.

@diosmosis
Copy link
Member Author

Only UI test failures are some Installation tests, and new screenshots look more accurate than existing. Will copy them over after merging.

diosmosis added a commit that referenced this pull request May 7, 2015
Moby DIoC: Step 1 - Piwik environment encapsulation. List of all changes:

* introduction of Environment class that creates and encapsulates Piwik DI container
* introduction of concept of 'kernel global'. a kernel global is an object that cannot be overridden via the DI container, since it is used to create it.
* introduces three kernel objects: PluginList (describes which plugins are activated), GlobalSettingsProvider (provides global settings), and EnvironmentValidator (validates the environment)
* all environment validation logic was moved to EnvironmentValidator (tests added as well)
* removed ability to change Config files used by existing Config instance
* Moved Config::encodeValues/::decodeValues to IniFileChain
* Lots of hacks to test code
* removal of some useless CronArchive.php setup code
* small change to core\IP.php (fix to error silencing)
* added Plugin\Manager to DI container
* Removed cli-script-bootstrap.php (no longer used)
@diosmosis diosmosis merged commit b27f01e into master May 7, 2015
@diosmosis diosmosis deleted the config_step_2 branch May 7, 2015 19:02
@diosmosis diosmosis removed the Needs Review PRs that need a code review label May 7, 2015
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants