@dessant opened this Issue on June 26th 2016

If config.ini.php is present, the installation sequence is skipped entirely.

When the required fields are present in the config file (database host, name, password etc), the installation could happen in a headless way, otherwise the manual istallation page would be loaded on the first visit, and the entered data would be stored in config without overwiting the existing contents of the config file.

This would make it easy to provide a custom config during provisioning and fully set up Piwik, without the need for hacks like these: https://github.com/MarvAmBass/docker-piwik/blob/master/startup-piwik.sh.

I am aware of headless installation being available in the Pro version, please note that this issue was opened for the free version of Piwik.

@tsteur commented on June 29th 2016 Member

The installation is skipped indeed if all needed fields are there in the config and if database tables exist. I'm not quite sure what the exact problem is? Did you want to install Piwik via command line by providing parameters like "--database-host"?

@dessant commented on June 29th 2016

Yeah, the main issue is that headless installation is not supported and one has to use the ui to provide the db connection and bootstrap the database. While this is ok for one-off installations, it becomes problematic when you're doing automated provisioning.

The config way was one idea, ideally the main outcome would be to have a way to install Piwik up to a state where the admin user is created, without requiring manual steps.

@tsteur commented on June 30th 2016 Member

got it. I thought I created a similar issue a few years ago when I built a vagrant box for Piwik but cannot find it anymore. Possibly I didn't actually create the issue. It would be a useful feature to have nowadays.

Basically we would only need to create the config file from the input and validate the entered data. For validation and creation of config file everything is already here so it shouldn't be too complex

@mattab commented on July 8th 2016 Member

Hi @dessant

we may in the future release such a tool as open source. If your company can support us in this area, please get in touch via https://piwik.org/contact/

@Toub commented on July 29th 2016

Let's say the automation can be run with a script named install-piwik.sh, we need 2 different ways to provide the configuration:

  • command lines args (e.g. install-piwik.sh --dbuser=myuser --dbpassword=pass ...) => drawback: the tool appears in the command history
  • install.ini file (running install-piwik.sh will then remove install.ini, so it can be more secure)
@tsteur commented on July 30th 2016 Member

If someone is keen to work on this I'm happy to assist. Be good to have this

@nebev commented on October 6th 2016

I had a stab at this yesterday. It's not particularly brilliant, but gets the job done for me.
I'm running docker containers, so it's not really an option for me to use the CURL method.
For anyone interested: https://github.com/nebev/piwik-cli-setup

@gaumondp commented on October 7th 2016

Thanks @nebev , I'm no programmer but I can think of some more features or maybe it should be a different script?

  1. Option to upgrade instead of installing
  2. Option to fetch the source from http://builds.piwik.org/latest.tar.gz (or any version) or a local directory.
  3. Test download against *.asc PGP signature.

I'm also wondering if such script should be maintain by Piwik CoreTeam once interest grow?

All in all, thanks anyway for sharing the code. :)

@lorengordon commented on November 10th 2016

I'm also very interested in an automated mechanism to complete the initial configuration and creation/update of the db tables. 👍

@mattab commented on February 18th 2017 Member

Please note we have this automation available but it's one of the rare things we haven't published as Open source. If you want to access this please contact InnoCraft here: https://www.innocraft.com/#contact

@pc-magas commented on February 18th 2017

As for me I wanted to have a cli installer in order to make a custom docker image.

@chriscroome commented on March 13th 2017

I have been looking at how to script Piwik installs and came across the Cauldron install script which adopts another approach to this problem from the examples above -- using a minimal MySQL dump to populate the database and then using the php console script to setup the config file.

@nebulade commented on March 14th 2017

Hi, I'm the author of that Cloudron package and I think it is a quite bad hack using the sql dump. Since I lack deeper knowledge of the piwik install process, this was just an option to get around the setup in our case, but I would not recommend that in the long run at all. The alternative solution I did first was to replay the requests to the setup webinterface via script, but that was even uglier.

@nebev commented on March 14th 2017

I agree. SQL dump is a bad idea as schemas change. Best to use API calls and existing functions where possible. I'm successfully running my CLI setup script in a docker image for brand new installs of Piwik 3.0.2.

@tomaluca95 commented on December 26th 2017

I'm interested if anyone has a good solution to automatically install (and maybe upgrade) to the last stable version.
I'm trying to create an ansible playbook to setup the full stack and piwik is part of the environment.

@jamietanna commented on December 26th 2017

I've been doing this via Chef, but it uses SQL dumps so as mentioned, isn't great long term. However, it means I can always install the same Piwik version correctly

@minac commented on July 24th 2018

Any news on this one? I have everything setup using docker and docker-composer including a customized config.ini.php file. It all works when I do a first manual run that creates the user and database tables, but if config.ini.php is present it fails (as was said above). Could we see matomo create/bootstrap the database when it's not there but the credentials are there?

Thanks!

@mikkeschiren commented on February 20th 2019

@minac I have been working on this the recent days, repo is over here: https://github.com/nodeone/extratools
The plugin adds some new console commands - and most important matomo:install .

The work is still work in progress - but I hope I have the most parts ready this week and will after that ask to have the plugin published at https://plugins.matomo.org/ if the Matomo people approve.

@mattab commented on February 22nd 2019 Member

Hi @mikkeschiren - it's great to see your project, giving everyone the ability to install Matomo in one command line... on our end in Matomo, we'd need to have the install command in core (and maybe can work together to get it merged into core later), and in the meantime the community will hopefully start using your project :+1:

@mikkeschiren commented on February 23rd 2019

I will gladly help to get this into core later @mattab. We (Digitalist Sweden) - mainly work with fully automated workflows and Matomo has become a very important part of our business the latest year, so we really want to contribute back.

@mikkeschiren commented on March 1st 2019

Still some minor details I need to solve before an official release, but should be just days from now,

@mikkeschiren commented on March 6th 2019

First alpha is out now: https://plugins.matomo.org/ExtraTools

@colans commented on June 18th 2019

I'm trying to create an ansible playbook to setup the full stack and piwik is part of the environment. I'm interested if anyone has a good solution to automatically install (and maybe upgrade) to the last stable version.

In researching the Ansible roles for Matomo that I've found, most of them are downloading the zip file, which seems sub-optimal.

I believe the following plan should do the trick (in an Ansible role):

  1. Install the Debian package like this example does (in its "Installing Piwik" section).
  2. Keep the VM up-to-date with Automatic Updates. This will ensure that the Debian package is kept up-to-date. (You could do this in your default-VM role.)
  3. Run through the manual Web configuration once, which generates config.ini.php. Save that file as a template, which gets injected into the installation, but replace hard-coded values with variables. Some of the roles I've seen are doing this, but others are injecting a database dump, which I feel is overkill.

Unless somebody beats me to it, I'll publish a role that does this.

Correct me if I'm wrong, but I believe the above solution solves the issue without the need for a console option. I'm assuming this will also work with Chef, Puppet, Salt, etc.

@minac commented on June 19th 2019

Hey @colans,

That approach seems sound. The only downside is the database upgrades. Matomo has 2 types of upgrades that usually go together: files updates and database updates. Your point #2 could solve the files updates, assuming that the package is being maintained properly, but for the database stuff, the only way that I know is to do it was manually through the interface (this may have changed with https://plugins.matomo.org/ExtraTools).

As for #3, you can inject the config.ini.php file into your ansible playbooks, no need to manually run through it. :)

My two cents.

@chriscroome commented on June 19th 2019

The Matomon Ansible role I have been working on and using is here: https://git.coop/webarch/matomo

I've been using it for upgrading quite a few sites and have recently also got it working for installs and adding users, the next step is adding sites, but that is currently held up by this issue.

@J0WI commented on June 20th 2019
@colans commented on July 31st 2019

for the database stuff, the only way that I know is to do [upgrades] was manually through the interface

This can now be done by the Debian package. I ended up using a DB dump template after all, but it doesn't matter if it gets stale. Once the Debian package gets installed (at the end), it'll upgrade the DB schema. And of course, this will happen whenever OS updates are run.

See my role for details, soon to be published on Galaxy.

So I didn't need any of these console commands yet, but I'm certainly glad folks have been working on this as I'm sure I'll find it useful later (e.g. for adding users, etc.). Please keep up the great work!

Edit: My role is now published at https://galaxy.ansible.com/consensus/matomo.

@AbcSxyZ commented on March 10th 2020

I saw https://github.com/matomo-org/matomo/issues/1586 and this issue. I was also intersting by this kind of feature, and I'm not sure if someone is working on, issue of this kind have years, checking current commit, seem having nothing...

I'm working on since 2 days and it's still in progress, but I'm trying to code within the Installation plugin to add an headless interface to be filled with variable.

At this day, I've been able to extract the installation logic from the plugin controller, and go through all installation step. All setup step are done in my InstallManager, just missing js tracker code generation for tonight : https://github.com/AbcSxyZ/matomo/blob/headless_init/plugins/Installation/InstallManager.php

At this day, I've no way to send my own data, but It seem I've got the API to do it. I've to care about error and probably some extra stuff, and think about the way to check if the install have to be done headless or not, how variable will be passed etc.

I'm thinking it in a way that could be isolated, to avoid breaking feature in Installation plugin, but in a way that installation controller can use it for upgrade, because there is a lot of change. It's possible easily because I had to isolate the installation entirely contained in the Controler (see https://github.com/AbcSxyZ/matomo/blob/headless_init/plugins/Installation/Controller.php, install is done through my InstallManager).

Any feedback is welcome, and if there is any tips to increase the php style because I'm not using it usually.

@chriscroome commented on March 10th 2020
@AbcSxyZ commented on March 10th 2020

I thought this tool was a Web Interface wrapper, because it wasn't added to the core, but it's using piwik functions to perform the install. Also https://github.com/nebev/piwik-cli-setup/blob/master/install.php have the same kind of setup. I'm wondering why they didn't added it to the matomo source code.

It look intersting for me to add this features in the Installation plugin.

EDIT: I also found https://github.com/matomo-org/matomo/pull/13676, probably some stuff to do with it.

@AbcSxyZ commented on March 10th 2020

I'm able to go through the admin installation and redirect to default index.php using an array, seem to have all table created. See welcome function of the Installation\Controller to get the array and my headlessInstall in my InstallManager for the installation.

@AbcSxyZ commented on March 12th 2020

UPDATE
I'm getting a mecanisme which work like that :

This will install the matomo server on first request, and show the login page of matomo. I had to change some core feature, like isPiwikInstalled and the EnvironnementValidator to plug the InstallManager. I'm not aware of the best possible place, I tried to find it, at least for testing.

I'm not sure if all is running properly (fore sure not), but I'm able to create a correct config file, fill database, login into the matomo homepage with specified password. I tried to activate and deactivate a plugin, it works. I've no way to test it with a Matomo normal usage, I've no website using it, and I'm still stuck at the installation !

Idea in mind

  • I'm thinking to move it as a core feature, and leave the installation process with GUI as a plugin. (see https://github.com/AbcSxyZ/matomo/blob/headless_init/core/MatomoInstaller.php)
  • Must remove some Installation Plugin utility (like install_in_progress settings) from the InstallManager, to have an independant process to perform the installation.
  • Thinking of a class who have some method overridable, like the current InstallManager::isHeadlessInstall, to define custom check to know if it's headless install + combinated with a method to override for retrieving data for the installation. In this way, it could allow any kind of installation, with config.ini.php, environnement variables, console, with post request or whatever you create.
@bluikko commented on January 15th 2021

As is evident from the amount of issues opened around this problem - any modern app these days really must support headless install.

The days of manually browsing to an install page are truly over.

@chriscroome commented on January 15th 2021

While the ExtraTools plugin doesn't work for Matomo 4 yet, so the console command matomo:install can't be used for automatic installs, I have found that automating via the web installer is not so hard, see for example these Ansible tasks which are based on these tasks.

However I do agree with @bluikko — it would be great to have the ability to properly interact with Matomo using a CLI.

Powered by GitHub Issue Mirror