Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve filesystem detection for NFS #8734

Closed
nuxwin opened this issue Sep 8, 2015 · 13 comments
Closed

Improve filesystem detection for NFS #8734

nuxwin opened this issue Sep 8, 2015 · 13 comments
Assignees
Labels
Bug For errors / faults / flaws / inconsistencies etc.
Milestone

Comments

@nuxwin
Copy link
Contributor

nuxwin commented Sep 8, 2015

Hello ;

I installed Piwik on a test server which is managed by i-MSCP (A Linux control panel for shared hosting environments). When I check the system using Piwik, I get the following warning:

Your server is using an NFS filesystem.
This means Piwik will be extremely slow when using file based sessions. 

and therefore, the session handler is automatically set to dbtable.

While this behavior is expected for NFS, this should not be the case for bind mounts. Indeed, Piwik detects that I use NFS but this is totally wrong. i-MSCP automatically re-mounts the Web folder directories (bind mount) as shared subtree. More explaination can be found here: https://github.com/i-MSCP/imscp/blob/1.3.x/docs/1.3.x_errata.md#shared-subtree-homedir

In my case, the Web folder is re-mounted as follow:

/dev/sda1 on /var/www/virtual/test.tld type ext4 (rw,relatime,errors=remount-ro,data=ordered)

As you can see here, that is not an NFS filesystem. I don't know how the check is made by Piwik but it seem that the filesystem type is not really checked.

That bind mount is managed by i-MSCP:

# fstab-like configuration file - auto-generated by i-MSCP
#     DO NOT EDIT THIS FILE BY HAND -- YOUR CHANGES WILL BE OVERWRITTEN
/var/www/virtual/test.tld /var/www/virtual/test.tld none shared,bind
/var/log/apache2/test.tld /var/www/virtual/test.tld/logs/test.tld none bind
/var/log/apache2/piwik.test.tld /var/www/virtual/test.tld/logs/piwik.test.tld none bind

See https://github.com/i-MSCP/imscp/blob/1.3.x/docs/1.3.x_errata.md#imscp_mountall-service

Note: I'm the i-MSCP project director.

@nuxwin
Copy link
Contributor Author

nuxwin commented Sep 8, 2015

Re ;

I could provide a patch if you want (PR). It seem that the check is done in the core/Filesystem.php file. I must investigate but right now, when running the df command manually on my system, I get

root@jessie:/var/www/virtual/test.tld/piwik/htdocs/tmp/sessions# LANG=C df -T -t nfs  /var/www/virtual/test.tld/piwik/htdocs/tmp/sessions 2>&1
df: no file systems processed
root@jessie:/var/www/virtual/test.tld/piwik/htdocs/tmp/sessions# echo $?
1

which is expected. Thus, I presume that the check is wrong somewhere in the checkIfFileSystemIsNFS() method of the Piwik\Filesystem class.

My environment:

root@jessie:/var/www/virtual/test.tld/piwik/htdocs# lsb_release -a
No LSB modules are available.
Distributor ID: Debian
Description:    Debian GNU/Linux 8.2 (jessie)
Release:    8.2
Codename:   jessie

root@jessie:/var/www/virtual/test.tld/piwik/htdocs/tmp/sessions# apache2ctl -v
Server version: Apache/2.4.10 (Debian)
Server built:   Aug 28 2015 16:28:08

root@jessie:/var/www/virtual/test.tld/piwik/htdocs/tmp/sessions# php5-fpm -v
PHP 5.6.12-0+deb8u1 (fpm-fcgi) (built: Aug 16 2015 12:16:05)
Copyright (c) 1997-2015 The PHP Group
Zend Engine v2.6.0, Copyright (c) 1998-2015 Zend Technologies
    with Zend OPcache v7.0.6-dev, Copyright (c) 1999-2015, by Zend Technologies

For the record, when I run the following script (CLI):

<?php

function checkIfFileSystemIsNFS()
{
        $sessionsPath = '/var/www/virtual/test.tld/piwik/htdocs/tmp/sessions';

        // this command will display details for the filesystem that holds the $sessionsPath
        // path, but only if its type is NFS. if not NFS, df will return one or less lines
        // and the return code 1. if NFS, it will return 0 and at least 2 lines of text.
        $command = "df -T -t nfs \"$sessionsPath\" 2>&1";

        if (function_exists('exec')) {
            // use exec

            $output = $returnCode = null;
            @exec($command, $output, $returnCode);

            // check if filesystem is NFS
            if ($returnCode == 0
                && count($output) > 1
            ) {
                return true;
            }
        } elseif (function_exists('shell_exec')) {
            // use shell_exec

            $output = @shell_exec($command);
            if ($output) {
                $output = explode("\n", $output);
                if (count($output) > 1) {
                    // check if filesystem is NFS

                    return true;
                }
            }
        }

        return false; // not NFS, or we can't run a program to find out
}


if(checkIfFileSystemIsNFS()) {
        print "NFS filesystem has been detected\n";
} else {
        print "Not an NFS filesystem\n";
}

I get the expected result:

root@jessie:/var/www/virtual/test.tld/piwik/htdocs/tmp/sessions# php test.php 
Not an NFS filesystem

Thus, I must investigate more...

@nuxwin
Copy link
Contributor Author

nuxwin commented Sep 8, 2015

I find the cause:

        } elseif (function_exists('shell_exec')) {
            // use shell_exec

            $output = @shell_exec($command);
            if ($output) {
                $output = explode("\n", $output);
                if (count($output) > 1) {
                    // check if filesystem is NFS

                    return true;
                }
            }
        }

is wrong... You cannot rely on the output of that command AS THIS to detect filesystem.

In my case, output array (after explode) hold:

Array
(
    [0] => df: no file systems processed
    [1] => 
)

Then, the result is erroned.

You can fix that issue by applying trim() on the output before the explode:

$output = trim($output);

@mattab Do you want a PR?

@tsteur
Copy link
Member

tsteur commented Sep 8, 2015

PR would be awesome 👍

@nuxwin
Copy link
Contributor Author

nuxwin commented Sep 8, 2015

@tsteur

The PR must be made against master or do you have a specific branch for development purpose?

@tsteur
Copy link
Member

tsteur commented Sep 8, 2015

PR should be made against master 👍

@nuxwin
Copy link
Contributor Author

nuxwin commented Sep 8, 2015

@tsteur

Ok, then. I'll do that PR in few minutes ;)

nuxwin added a commit to nuxwin/piwik that referenced this issue Sep 8, 2015
@nuxwin
Copy link
Contributor Author

nuxwin commented Sep 8, 2015

@tsteur

Done. See #8737

BTW:

@mattab

The following check which was previously added in master:

$commandFailed = (false !== strpos($output, "no file systems processed"));

was not sufficient. Think about LANG environment variable which can be different.

tsteur added a commit that referenced this issue Sep 8, 2015
#8734 Bad filesystem detection (NFS)
@tsteur
Copy link
Member

tsteur commented Sep 8, 2015

I presume we can close this one then?

@nuxwin
Copy link
Contributor Author

nuxwin commented Sep 8, 2015

@tsteur
Sure since you merged the referenced PR ;)

@nuxwin
Copy link
Contributor Author

nuxwin commented Sep 8, 2015

@tsteur

I close it right now but I cannot add the BUG label on it ;)

@nuxwin nuxwin closed this as completed Sep 8, 2015
@tsteur tsteur added the Bug For errors / faults / flaws / inconsistencies etc. label Sep 8, 2015
@tsteur tsteur added this to the 2.15.0 milestone Sep 8, 2015
@tsteur tsteur self-assigned this Sep 8, 2015
@tsteur
Copy link
Member

tsteur commented Sep 8, 2015

Done, thx

@mattab mattab changed the title Bad filesystem detection (NFS) Improve filesystem detection for NFS Oct 13, 2015
@DohmenICT
Copy link

DohmenICT commented Feb 25, 2019

Hi!

I run Matomo on a Synology Diskstation locally, and this checkIfFileSystemIsNFS() still returns TRUE!

I did some testing and when I run the following command on my DS
df -T -t nfs "/volume1/web/piwik/tmp/sessions"

I get:

Filesystem     Type  1K-blocks       Used  Available Use% Mounted on
-              -    6039797760 3668011728 2371786032  61% /volume1/web

And the function returns true;

When I change the function so that it just returns False; Matomo works a lot quicker since it really isn't a NFS.

Can this be patched?

@tsteur
Copy link
Member

tsteur commented Feb 25, 2019

BTW: We don't use files for sessions anymore and should be pretty fast.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Bug For errors / faults / flaws / inconsistencies etc.
Projects
None yet
Development

No branches or pull requests

3 participants