config.ini.php is present, the installation sequence is skipped entirely.
When the required fields are present in the config file (database host, name, password etc), the installation could happen in a headless way, otherwise the manual istallation page would be loaded on the first visit, and the entered data would be stored in config without overwiting the existing contents of the config file.
This would make it easy to provide a custom config during provisioning and fully set up Piwik, without the need for hacks like these: https://github.com/MarvAmBass/docker-piwik/blob/master/startup-piwik.sh.
I am aware of headless installation being available in the Pro version, please note that this issue was opened for the free version of Piwik.
The installation is skipped indeed if all needed fields are there in the config and if database tables exist. I'm not quite sure what the exact problem is? Did you want to install Piwik via command line by providing parameters like "--database-host"?
Yeah, the main issue is that headless installation is not supported and one has to use the ui to provide the db connection and bootstrap the database. While this is ok for one-off installations, it becomes problematic when you're doing automated provisioning.
The config way was one idea, ideally the main outcome would be to have a way to install Piwik up to a state where the admin user is created, without requiring manual steps.
got it. I thought I created a similar issue a few years ago when I built a vagrant box for Piwik but cannot find it anymore. Possibly I didn't actually create the issue. It would be a useful feature to have nowadays.
Basically we would only need to create the config file from the input and validate the entered data. For validation and creation of config file everything is already here so it shouldn't be too complex
Let's say the automation can be run with a script named install-piwik.sh, we need 2 different ways to provide the configuration:
If someone is keen to work on this I'm happy to assist. Be good to have this
I had a stab at this yesterday. It's not particularly brilliant, but gets the job done for me.
I'm running docker containers, so it's not really an option for me to use the CURL method.
For anyone interested: https://github.com/nebev/piwik-cli-setup
Thanks @nebev , I'm no programmer but I can think of some more features or maybe it should be a different script?
I'm also wondering if such script should be maintain by Piwik CoreTeam once interest grow?
All in all, thanks anyway for sharing the code. :)
I'm also very interested in an automated mechanism to complete the initial configuration and creation/update of the db tables. 👍
Please note we have this automation available but it's one of the rare things we haven't published as Open source. If you want to access this please contact InnoCraft here: https://www.innocraft.com/#contact
As for me I wanted to have a cli installer in order to make a custom docker image.
I have been looking at how to script Piwik installs and came across the Cauldron install script which adopts another approach to this problem from the examples above -- using a minimal MySQL dump to populate the database and then using the php console script to setup the config file.
Hi, I'm the author of that Cloudron package and I think it is a quite bad hack using the sql dump. Since I lack deeper knowledge of the piwik install process, this was just an option to get around the setup in our case, but I would not recommend that in the long run at all. The alternative solution I did first was to replay the requests to the setup webinterface via script, but that was even uglier.
I agree. SQL dump is a bad idea as schemas change. Best to use API calls and existing functions where possible. I'm successfully running my CLI setup script in a docker image for brand new installs of Piwik 3.0.2.
I'm interested if anyone has a good solution to automatically install (and maybe upgrade) to the last stable version.
I'm trying to create an ansible playbook to setup the full stack and piwik is part of the environment.
I've been doing this via Chef, but it uses SQL dumps so as mentioned, isn't great long term. However, it means I can always install the same Piwik version correctly
Any news on this one? I have everything setup using docker and docker-composer including a customized config.ini.php file. It all works when I do a first manual run that creates the user and database tables, but if config.ini.php is present it fails (as was said above). Could we see matomo create/bootstrap the database when it's not there but the credentials are there?
The work is still work in progress - but I hope I have the most parts ready this week and will after that ask to have the plugin published at https://plugins.matomo.org/ if the Matomo people approve.
Hi @mikkeschiren - it's great to see your project, giving everyone the ability to install Matomo in one command line... on our end in Matomo, we'd need to have the
install command in core (and maybe can work together to get it merged into core later), and in the meantime the community will hopefully start using your project :+1:
I will gladly help to get this into core later @mattab. We (Digitalist Sweden) - mainly work with fully automated workflows and Matomo has become a very important part of our business the latest year, so we really want to contribute back.
Still some minor details I need to solve before an official release, but should be just days from now,
First alpha is out now: https://plugins.matomo.org/ExtraTools
I'm trying to create an ansible playbook to setup the full stack and piwik is part of the environment. I'm interested if anyone has a good solution to automatically install (and maybe upgrade) to the last stable version.
In researching the Ansible roles for Matomo that I've found, most of them are downloading the
zip file, which seems sub-optimal.
I believe the following plan should do the trick (in an Ansible role):
config.ini.php. Save that file as a template, which gets injected into the installation, but replace hard-coded values with variables. Some of the roles I've seen are doing this, but others are injecting a database dump, which I feel is overkill.
Unless somebody beats me to it, I'll publish a role that does this.
Correct me if I'm wrong, but I believe the above solution solves the issue without the need for a console option. I'm assuming this will also work with Chef, Puppet, Salt, etc.
That approach seems sound. The only downside is the database upgrades. Matomo has 2 types of upgrades that usually go together: files updates and database updates. Your point #2 could solve the files updates, assuming that the package is being maintained properly, but for the database stuff, the only way that I know is to do it was manually through the interface (this may have changed with https://plugins.matomo.org/ExtraTools).
As for #3, you can inject the config.ini.php file into your ansible playbooks, no need to manually run through it. :)
My two cents.
I tried this in https://github.com/J0WI/docker-piwik/commit/6015481e1b139d8f10fb10bba4234b9d6d1df8d2 for the Docker container. This would be very useful.
for the database stuff, the only way that I know is to do [upgrades] was manually through the interface
This can now be done by the Debian package. I ended up using a DB dump template after all, but it doesn't matter if it gets stale. Once the Debian package gets installed (at the end), it'll upgrade the DB schema. And of course, this will happen whenever OS updates are run.
So I didn't need any of these console commands yet, but I'm certainly glad folks have been working on this as I'm sure I'll find it useful later (e.g. for adding users, etc.). Please keep up the great work!
Edit: My role is now published at https://galaxy.ansible.com/consensus/matomo.
I saw https://github.com/matomo-org/matomo/issues/1586 and this issue. I was also intersting by this kind of feature, and I'm not sure if someone is working on, issue of this kind have years, checking current commit, seem having nothing...
I'm working on since 2 days and it's still in progress, but I'm trying to code within the Installation plugin to add an headless interface to be filled with variable.
At this day, I've been able to extract the installation logic from the plugin controller, and go through all installation step. All setup step are done in my
just missing js tracker code generation for tonight : https://github.com/AbcSxyZ/matomo/blob/headless_init/plugins/Installation/InstallManager.php
At this day, I've no way to send my own data, but It seem I've got the API to do it. I've to care about error and probably some extra stuff, and think about the way to check if the install have to be done headless or not, how variable will be passed etc.
I'm thinking it in a way that could be isolated, to avoid breaking feature in Installation plugin, but in a way that installation controller can use it for upgrade, because there is a lot of change. It's possible easily because I had to isolate the installation entirely contained in the Controler (see https://github.com/AbcSxyZ/matomo/blob/headless_init/plugins/Installation/Controller.php, install is done through my
Any feedback is welcome, and if there is any tips to increase the php style because I'm not using it usually.
@AbcSxyZ I have implemented automated installs both via git and the tgz download using Ansible and the Extra Tools matomo:install command, see this PHP file.
I thought this tool was a Web Interface wrapper, because it wasn't added to the core, but it's using piwik functions to perform the install. Also https://github.com/nebev/piwik-cli-setup/blob/master/install.php have the same kind of setup. I'm wondering why they didn't added it to the matomo source code.
It look intersting for me to add this features in the Installation plugin.
EDIT: I also found https://github.com/matomo-org/matomo/pull/13676, probably some stuff to do with it.
I'm able to go through the admin installation and redirect to default
index.php using an array, seem to have all table created. See
welcome function of the
Installation\Controller to get the array and my
headlessInstall in my
InstallManager for the installation.
I'm getting a mecanisme which work like that :
MatomoInstallsection, and filled like so: https://github.com/AbcSxyZ/matomo/blob/headless_init/config/config.ini.php.template
This will install the matomo server on first request, and show the login page of matomo. I had to change some core feature, like
isPiwikInstalled and the
EnvironnementValidator to plug the InstallManager. I'm not aware of the best possible place, I tried to find it, at least for testing.
I'm not sure if all is running properly (fore sure not), but I'm able to create a correct config file, fill database, login into the matomo homepage with specified password. I tried to activate and deactivate a plugin, it works. I've no way to test it with a Matomo normal usage, I've no website using it, and I'm still stuck at the installation !
Idea in mind
install_in_progresssettings) from the
InstallManager, to have an independant process to perform the installation
InstallManager::isHeadlessInstall, to define custom check to know if it's headless install + combinated with a method to override for retrieving data for the installation. In this way, it could allow any kind of installation, with
config.ini.php, environnement variables, console, with post request or whatever you create.
As is evident from the amount of issues opened around this problem - any modern app these days really must support headless install.
The days of manually browsing to an install page are truly over.
While the ExtraTools plugin doesn't work for Matomo 4 yet, so the console command
matomo:install can't be used for automatic installs, I have found that automating via the web installer is not so hard, see for example these Ansible tasks which are based on these tasks.
However I do agree with @bluikko — it would be great to have the ability to properly interact with Matomo using a CLI.
Just chiming in here, and agreeing that having Matomo CLI installation would be awesome.
You can install Drupal with Drush from the command line with this:
drush site:install --db-url=mysql://drupal9:drupal9<a class='mention' href='https://github.com/database'>@database</a>/drupal9
It would be great if similarly you could download Matomo, instantiate whatever web server you like and install it:
matomo install \ --db-url=mysql://lamp:lamp<a class='mention' href='https://github.com/database'>@database</a>/lamp \ --option user=admin \ --option pw=admin \ --option email=admin<a class='mention' href='https://github.com/example'>@example</a>.org \ --option sitename='Example Org' \ --option siteurl=example.org
FYI: we managed to automate the installation in a docker environment without going through the web-wizard by
a) cloning the matomo github repo
b) cloning the ExtraTools-Plugin into the plugin directory
c) update composer dependencies (incl. sub-repositories)
d) build a custom docker image (especially requiring mysql-client / mysqladmin dependencies for ExtraTools)
e) running the parameterized install command from ExtraTools in the container
We also added several other plugins and install those properly by ExtraTools.
@arosenhagen , how do you create the database? I've tried to repeat the steps you tell above, but it doesn't allow to activate ExtraTools as there is no database created (and I've to activate ExtraTools for being able to run step e ).
FYI: we managed to automate the installation in a docker environment without going through the web-wizard by a) cloning the matomo github repo b) cloning the ExtraTools-Plugin into the plugin directory c) update composer dependencies (incl. sub-repositories) d) build a custom docker image (especially requiring mysql-client / mysqladmin dependencies for ExtraTools) e) running the parameterized install command from ExtraTools in the container
We also added several other plugins and install those properly by ExtraTools.
It does work but it is very, very painful.
The most stupidest thing of all (...I've ever seen?!) is that the database is not fully initialized until first HTTP GET to the website!!
Following @arosenhagen message https://github.com/matomo-org/matomo/issues/10257#issuecomment-1018498246 guidance, I've managed to get a full unattended installation, doing some tinkering with ExtraTools (https://plugins.matomo.org/ExtraTools). Those are the steps I've followed:
apt update && apt install -y unzip mariadb-client
composer.jsonis missing, add it (i.e. from GitHub: https://raw.githubusercontent.com/matomo-org/matomo/4.7.1/composer.json ).
config.ini.phpin Matomo config dir, containing database connection parameters only.
plugins/CoreConsole/Commandsand change its namespace to
Piwik\Plugins\CoreConsole\Commands: Here is where the magic is. ExtraTools is not going to work initially; there is no database created yet so ExtraTools activation fails. Doing this copy, the next step will work.
matomo:installcommand: Installation parameters can be added as command arguments, but it's easier to create an
install.jsonfile (like in https://github.com/digitalist-se/extratools/blob/4.x-dev/docs/example.json):
console matomo:install --install-file=install.json
All steps can be easily automated in a script or Dockerfile.
I hope it helps someone there. As I said is mostly tinkering, and I think this feature would have to be available in vanilla Matomo, as unattended containers or virtualized installations are common today.