@mikkeschiren opened this Issue on December 25th 2022 Contributor

With DevicesDetection installed and making a request for any reports, thousands of requests are made for cached items for DevicesDetection even when it's not required. We are using Redis for cache. Could be that this a general "thing", not just for DevicesDetection. And could also be expected behaviour, that I don't know.

Expected Behavior

Cached items should only be asked for when needed.

Current Behavior

Requests are done for cached items not needed.

Steps to Reproduce (for Bugs)

  1. Activate DevicesDetection
  2. Do a request for a report not displaying any info about devices.
  3. With a network sniffer, you could see requests done for DevicesDetection cached items.

Your Environment

  • Matomo Version: 4.12.3
  • PHP Version: 8..0.25
  • Server Operating System: Linux Alpine
bildbild
@mikkeschiren commented on December 25th 2022 Contributor

As an example, the cache for "carbrowser604-all" is requested 30 times for one request on the example report, which contains no devices at all. All the cache requests return the same thing.

@mikkeschiren commented on December 25th 2022 Contributor

My guess is that every day in the visitor overview, which is 30 days, makes requests for the cached items of devices.

@sanchezzzhak commented on December 28th 2022

This is the expected behavior for Device Detector, since one php instance is launched for each visitor.

in my projects, I use Device Detector like this:

namespace app/helpers 

class DeviceDetectorHelper 
{
    /**
     * <a class='mention' href='https://github.com/param'>@param</a> string $userAgent - user agent string
     * <a class='mention' href='https://github.com/param'>@param</a> array $headers - http headers
     * <a class='mention' href='https://github.com/return'>@return</a> DeviceDetector
     */
    public static function detect(string $userAgent, array $headers = []): DeviceDetector
    {
        static $detector;
        if ($detector === null) {
            $detector = new DeviceDetector();
            $detector->setCache(new Cache());
        }
        $clientHints = ClientHints::factory($headers);
        $detector->setUserAgent($userAgent);
        $detector->setClientHints($clientHints);
        $detector->parse();

        return $detector;
    }

}

I will share my experience, maybe give ideas for the future
I analyze data in the background, I write all the data to the buffer table as is.
in the console script, I analyze the data from the buffer table.

the script is looped through + cron * * * * *
```php

// mutex file lock (only single process)
if ($this->getMutex()->acquire('stat')) {
    return;
}
$repeat = 0;
while (true) {
   $data = $this->getBufferData($limit = 10000);
   $countRows = count(data);
   if ($countRows === 0 ) {
     $iterate++;
   } else {
     // prepare visitors save and clean tmp buffer data;
     $this->processBufferAndClean($data);
      countine;
   }
   // if there is no data for more than 3 attempts, we exit
   if ($iterate === 3) {
       break;
   }
   // timeout delay 5 second
   sleep(5);
}

with this approach, you lose the minimum amount of time.

@mikkeschiren commented on January 5th 2023 Contributor

Ok, so you are not using the plugin to get the devices?

@sgiehl commented on January 9th 2023 Member

@mikkeschiren He is using device detector for something else than Matomo.
Nevertheless your report does not have anything to do with the DevicesDetection plugin.
Matomo tries to identify the browser as user is visiting with in order to display a warning if it is not supported.
This should normally be at most done once per request and should also be cached.
Might be worth looking at that, as doing unneeded device detections might cost some useless time.

Powered by GitHub Issue Mirror