@dessant opened this Issue on June 26th 2016

If config.ini.php is present, the installation sequence is skipped entirely.

When the required fields are present in the config file (database host, name, password etc), the installation could happen in a headless way, otherwise the manual istallation page would be loaded on the first visit, and the entered data would be stored in config without overwiting the existing contents of the config file.

This would make it easy to provide a custom config during provisioning and fully set up Piwik, without the need for hacks like these: https://github.com/MarvAmBass/docker-piwik/blob/master/startup-piwik.sh.

I am aware of headless installation being available in the Pro version, please note that this issue was opened for the free version of Piwik.

@tsteur commented on June 29th 2016 Member

The installation is skipped indeed if all needed fields are there in the config and if database tables exist. I'm not quite sure what the exact problem is? Did you want to install Piwik via command line by providing parameters like "--database-host"?

@dessant commented on June 29th 2016

Yeah, the main issue is that headless installation is not supported and one has to use the ui to provide the db connection and bootstrap the database. While this is ok for one-off installations, it becomes problematic when you're doing automated provisioning.

The config way was one idea, ideally the main outcome would be to have a way to install Piwik up to a state where the admin user is created, without requiring manual steps.

@tsteur commented on June 30th 2016 Member

got it. I thought I created a similar issue a few years ago when I built a vagrant box for Piwik but cannot find it anymore. Possibly I didn't actually create the issue. It would be a useful feature to have nowadays.

Basically we would only need to create the config file from the input and validate the entered data. For validation and creation of config file everything is already here so it shouldn't be too complex

@mattab commented on July 8th 2016 Member

Hi @dessant

we may in the future release such a tool as open source. If your company can support us in this area, please get in touch via https://piwik.org/contact/

@Toub commented on July 29th 2016

Let's say the automation can be run with a script named install-piwik.sh, we need 2 different ways to provide the configuration:

  • command lines args (e.g. install-piwik.sh --dbuser=myuser --dbpassword=pass ...) => drawback: the tool appears in the command history
  • install.ini file (running install-piwik.sh will then remove install.ini, so it can be more secure)
@tsteur commented on July 30th 2016 Member

If someone is keen to work on this I'm happy to assist. Be good to have this

@nebev commented on October 6th 2016

I had a stab at this yesterday. It's not particularly brilliant, but gets the job done for me.
I'm running docker containers, so it's not really an option for me to use the CURL method.
For anyone interested: https://github.com/nebev/piwik-cli-setup

@gaumondp commented on October 7th 2016

Thanks @nebev , I'm no programmer but I can think of some more features or maybe it should be a different script?

  1. Option to upgrade instead of installing
  2. Option to fetch the source from http://builds.piwik.org/latest.tar.gz (or any version) or a local directory.
  3. Test download against *.asc PGP signature.

I'm also wondering if such script should be maintain by Piwik CoreTeam once interest grow?

All in all, thanks anyway for sharing the code. :)

@lorengordon commented on November 10th 2016

I'm also very interested in an automated mechanism to complete the initial configuration and creation/update of the db tables. 👍

@mattab commented on February 18th 2017 Member

Please note we have this automation available but it's one of the rare things we haven't published as Open source. If you want to access this please contact InnoCraft here: https://www.innocraft.com/#contact

@pc-magas commented on February 18th 2017

As for me I wanted to have a cli installer in order to make a custom docker image.

@chriscroome commented on March 13th 2017

I have been looking at how to script Piwik installs and came across the Cauldron install script which adopts another approach to this problem from the examples above -- using a minimal MySQL dump to populate the database and then using the php console script to setup the config file.

@nebulade commented on March 14th 2017

Hi, I'm the author of that Cloudron package and I think it is a quite bad hack using the sql dump. Since I lack deeper knowledge of the piwik install process, this was just an option to get around the setup in our case, but I would not recommend that in the long run at all. The alternative solution I did first was to replay the requests to the setup webinterface via script, but that was even uglier.

@nebev commented on March 14th 2017

I agree. SQL dump is a bad idea as schemas change. Best to use API calls and existing functions where possible. I'm successfully running my CLI setup script in a docker image for brand new installs of Piwik 3.0.2.

@tomaluca95 commented on December 26th 2017

I'm interested if anyone has a good solution to automatically install (and maybe upgrade) to the last stable version.
I'm trying to create an ansible playbook to setup the full stack and piwik is part of the environment.

@jamietanna commented on December 26th 2017

I've been doing this via Chef, but it uses SQL dumps so as mentioned, isn't great long term. However, it means I can always install the same Piwik version correctly

@minac commented on July 24th 2018

Any news on this one? I have everything setup using docker and docker-composer including a customized config.ini.php file. It all works when I do a first manual run that creates the user and database tables, but if config.ini.php is present it fails (as was said above). Could we see matomo create/bootstrap the database when it's not there but the credentials are there?

Thanks!

@mikkeschiren commented on February 20th 2019 Contributor

@minac I have been working on this the recent days, repo is over here: https://github.com/nodeone/extratools
The plugin adds some new console commands - and most important matomo:install .

The work is still work in progress - but I hope I have the most parts ready this week and will after that ask to have the plugin published at https://plugins.matomo.org/ if the Matomo people approve.

@mattab commented on February 22nd 2019 Member

Hi @mikkeschiren - it's great to see your project, giving everyone the ability to install Matomo in one command line... on our end in Matomo, we'd need to have the install command in core (and maybe can work together to get it merged into core later), and in the meantime the community will hopefully start using your project :+1:

@mikkeschiren commented on February 23rd 2019 Contributor

I will gladly help to get this into core later @mattab. We (Digitalist Sweden) - mainly work with fully automated workflows and Matomo has become a very important part of our business the latest year, so we really want to contribute back.

@mikkeschiren commented on March 1st 2019 Contributor

Still some minor details I need to solve before an official release, but should be just days from now,

@mikkeschiren commented on March 6th 2019 Contributor

First alpha is out now: https://plugins.matomo.org/ExtraTools

@colans commented on June 18th 2019

I'm trying to create an ansible playbook to setup the full stack and piwik is part of the environment. I'm interested if anyone has a good solution to automatically install (and maybe upgrade) to the last stable version.

In researching the Ansible roles for Matomo that I've found, most of them are downloading the zip file, which seems sub-optimal.

I believe the following plan should do the trick (in an Ansible role):

  1. Install the Debian package like this example does (in its "Installing Piwik" section).
  2. Keep the VM up-to-date with Automatic Updates. This will ensure that the Debian package is kept up-to-date. (You could do this in your default-VM role.)
  3. Run through the manual Web configuration once, which generates config.ini.php. Save that file as a template, which gets injected into the installation, but replace hard-coded values with variables. Some of the roles I've seen are doing this, but others are injecting a database dump, which I feel is overkill.

Unless somebody beats me to it, I'll publish a role that does this.

Correct me if I'm wrong, but I believe the above solution solves the issue without the need for a console option. I'm assuming this will also work with Chef, Puppet, Salt, etc.

@minac commented on June 19th 2019

Hey @colans,

That approach seems sound. The only downside is the database upgrades. Matomo has 2 types of upgrades that usually go together: files updates and database updates. Your point #2 could solve the files updates, assuming that the package is being maintained properly, but for the database stuff, the only way that I know is to do it was manually through the interface (this may have changed with https://plugins.matomo.org/ExtraTools).

As for #3, you can inject the config.ini.php file into your ansible playbooks, no need to manually run through it. :)

My two cents.

@chriscroome commented on June 19th 2019

The Matomon Ansible role I have been working on and using is here: https://git.coop/webarch/matomo

I've been using it for upgrading quite a few sites and have recently also got it working for installs and adding users, the next step is adding sites, but that is currently held up by this issue.

@J0WI commented on June 20th 2019
@colans commented on July 31st 2019

for the database stuff, the only way that I know is to do [upgrades] was manually through the interface

This can now be done by the Debian package. I ended up using a DB dump template after all, but it doesn't matter if it gets stale. Once the Debian package gets installed (at the end), it'll upgrade the DB schema. And of course, this will happen whenever OS updates are run.

See my role for details, soon to be published on Galaxy.

So I didn't need any of these console commands yet, but I'm certainly glad folks have been working on this as I'm sure I'll find it useful later (e.g. for adding users, etc.). Please keep up the great work!

Edit: My role is now published at https://galaxy.ansible.com/consensus/matomo.

@AbcSxyZ commented on March 10th 2020

I saw https://github.com/matomo-org/matomo/issues/1586 and this issue. I was also intersting by this kind of feature, and I'm not sure if someone is working on, issue of this kind have years, checking current commit, seem having nothing...

I'm working on since 2 days and it's still in progress, but I'm trying to code within the Installation plugin to add an headless interface to be filled with variable.

At this day, I've been able to extract the installation logic from the plugin controller, and go through all installation step. All setup step are done in my InstallManager, just missing js tracker code generation for tonight : https://github.com/AbcSxyZ/matomo/blob/headless_init/plugins/Installation/InstallManager.php

At this day, I've no way to send my own data, but It seem I've got the API to do it. I've to care about error and probably some extra stuff, and think about the way to check if the install have to be done headless or not, how variable will be passed etc.

I'm thinking it in a way that could be isolated, to avoid breaking feature in Installation plugin, but in a way that installation controller can use it for upgrade, because there is a lot of change. It's possible easily because I had to isolate the installation entirely contained in the Controler (see https://github.com/AbcSxyZ/matomo/blob/headless_init/plugins/Installation/Controller.php, install is done through my InstallManager).

Any feedback is welcome, and if there is any tips to increase the php style because I'm not using it usually.

@chriscroome commented on March 10th 2020
@AbcSxyZ commented on March 10th 2020

I thought this tool was a Web Interface wrapper, because it wasn't added to the core, but it's using piwik functions to perform the install. Also https://github.com/nebev/piwik-cli-setup/blob/master/install.php have the same kind of setup. I'm wondering why they didn't added it to the matomo source code.

It look intersting for me to add this features in the Installation plugin.

EDIT: I also found https://github.com/matomo-org/matomo/pull/13676, probably some stuff to do with it.

@AbcSxyZ commented on March 10th 2020

I'm able to go through the admin installation and redirect to default index.php using an array, seem to have all table created. See welcome function of the Installation\Controller to get the array and my headlessInstall in my InstallManager for the installation.

@AbcSxyZ commented on March 12th 2020

UPDATE
I'm getting a mecanisme which work like that :

This will install the matomo server on first request, and show the login page of matomo. I had to change some core feature, like isPiwikInstalled and the EnvironnementValidator to plug the InstallManager. I'm not aware of the best possible place, I tried to find it, at least for testing.

I'm not sure if all is running properly (fore sure not), but I'm able to create a correct config file, fill database, login into the matomo homepage with specified password. I tried to activate and deactivate a plugin, it works. I've no way to test it with a Matomo normal usage, I've no website using it, and I'm still stuck at the installation !

Idea in mind

  • I'm thinking to move it as a core feature, and leave the installation process with GUI as a plugin. (see https://github.com/AbcSxyZ/matomo/blob/headless_init/core/MatomoInstaller.php)
  • Must remove some Installation Plugin utility (like install_in_progress settings) from the InstallManager, to have an independant process to perform the installation.
  • Thinking of a class who have some method overridable, like the current InstallManager::isHeadlessInstall, to define custom check to know if it's headless install + combinated with a method to override for retrieving data for the installation. In this way, it could allow any kind of installation, with config.ini.php, environnement variables, console, with post request or whatever you create.
@bluikko commented on January 15th 2021

As is evident from the amount of issues opened around this problem - any modern app these days really must support headless install.

The days of manually browsing to an install page are truly over.

@chriscroome commented on January 15th 2021

While the ExtraTools plugin doesn't work for Matomo 4 yet, so the console command matomo:install can't be used for automatic installs, I have found that automating via the web installer is not so hard, see for example these Ansible tasks which are based on these tasks.

However I do agree with @bluikko — it would be great to have the ability to properly interact with Matomo using a CLI.

@gitressa commented on January 20th 2022

Just chiming in here, and agreeing that having Matomo CLI installation would be awesome.

You can install Drupal with Drush from the command line with this:

drush site:install --db-url=mysql://drupal9:drupal9<a class='mention' href='https://github.com/database'>@database</a>/drupal9

It would be great if similarly you could download Matomo, instantiate whatever web server you like and install it:

matomo install \
--db-url=mysql://lamp:lamp<a class='mention' href='https://github.com/database'>@database</a>/lamp \
--option user=admin \
--option pw=admin \
--option email=admin<a class='mention' href='https://github.com/example'>@example</a>.org \
--option sitename='Example Org' \
--option siteurl=example.org
@arosenhagen commented on January 21st 2022

FYI: we managed to automate the installation in a docker environment without going through the web-wizard by
a) cloning the matomo github repo
b) cloning the ExtraTools-Plugin into the plugin directory
c) update composer dependencies (incl. sub-repositories)
d) build a custom docker image (especially requiring mysql-client / mysqladmin dependencies for ExtraTools)
e) running the parameterized install command from ExtraTools in the container

We also added several other plugins and install those properly by ExtraTools.

@vikenbauer commented on February 9th 2022

@arosenhagen , how do you create the database? I've tried to repeat the steps you tell above, but it doesn't allow to activate ExtraTools as there is no database created (and I've to activate ExtraTools for being able to run step e ).

FYI: we managed to automate the installation in a docker environment without going through the web-wizard by a) cloning the matomo github repo b) cloning the ExtraTools-Plugin into the plugin directory c) update composer dependencies (incl. sub-repositories) d) build a custom docker image (especially requiring mysql-client / mysqladmin dependencies for ExtraTools) e) running the parameterized install command from ExtraTools in the container

We also added several other plugins and install those properly by ExtraTools.

@bluikko commented on February 10th 2022

It does work but it is very, very painful.

The most stupidest thing of all (...I've ever seen?!) is that the database is not fully initialized until first HTTP GET to the website!!

@vikenbauer commented on February 14th 2022

Following @arosenhagen message https://github.com/matomo-org/matomo/issues/10257#issuecomment-1018498246 guidance, I've managed to get a full unattended installation, doing some tinkering with ExtraTools (https://plugins.matomo.org/ExtraTools). Those are the steps I've followed:

  1. Choose your favourite method for getting a uninitialized Matomo server (in my case, I was using a docker compose configuration like https://github.com/matomo-org/docker/blob/master/.examples/apache/docker-compose.yml). I've not tried installing from zip or git in a PHP server, but I guess it doesn't change so much.
  2. Install ExtraTools server dependencies:
    apt update && apt install -y unzip mariadb-client
    (Dependency is actually mysqladmin, but for mariaDB it comes into mariadb-client package).
  3. Install composer.
  4. If Matomo composer.json is missing, add it (i.e. from GitHub: https://raw.githubusercontent.com/matomo-org/matomo/4.7.1/composer.json ).
  5. Install ExtraTools composer dependencies: https://plugins.matomo.org/ExtraTools#documentation.
  6. Install ExtraTools: Download zip, uncompress and move it to plugins directory.
  7. Copy a config.ini.php in Matomo config dir, containing database connection parameters only.
  8. Copy plugins/ExtraTools/Commands/InstallMatomo.php to plugins/CoreConsole/Commands and change its namespace to Piwik\Plugins\CoreConsole\Commands: Here is where the magic is. ExtraTools is not going to work initially; there is no database created yet so ExtraTools activation fails. Doing this copy, the next step will work.
  9. Run matomo:install command: Installation parameters can be added as command arguments, but it's easier to create an install.json file (like in https://github.com/digitalist-se/extratools/blob/4.x-dev/docs/example.json):
    console matomo:install --install-file=install.json
  10. Delete temporary plugins/ExtraTools/Commands/InstallMatomo.php.
  11. Add an initial site (current ExtraTools version doesn't create any site in matomo:install).
  12. As I'm building a docker container, cleaning up (mariadb-client, ExtraTools, composer...).

All steps can be easily automated in a script or Dockerfile.

I hope it helps someone there. As I said is mostly tinkering, and I think this feature would have to be available in vanilla Matomo, as unattended containers or virtualized installations are common today.

@CrimsonScythe commented on September 27th 2022

Hey @vikenbauer I am trying to get this set up on a serverless system. Therefore, I am extending the Docker image for matomo:alpine and writing the steps you mentioned in that Dockerfile.
However, I seem to be getting this error when running matomo:install:
[Exception] Adapter 'mysql' is not valid. Maybe check that your Matomo configuration files in config/ *.ini.php are readable by the webserver.
I have given the folder read and write access to everything in config/ and also doublechedk with ls -l any ideas on what could be wrong?

Trying to activate the ExtraTools plugin leads to the following error:
[PDOException] SQLSTATE[42S02]: Base table or view not found: 1146 Table 'matomo.matomo_option' doesn't exist

@CrimsonScythe commented on September 27th 2022

I updated the confg.ini.php to only have db connection params and the plugin now activates however, still getting the same error on matomo:install.
On closer inspection, matomo:install seems to be changing the config.ini.php file to include adapter = "mysql" under [database]

@JMLX42 commented on October 19th 2022

All steps can be easily automated in a script or Dockerfile.

@vikenbauer do you mind providing a bash script to do all of that please?

Powered by GitHub Issue Mirror