import_logs: gracefully handle deleted sites ID #3627
Labels
Enhancement
For new feature suggestions that enhance Matomo's capabilities or add a new report, new API etc.
Milestone
Use case: you have many sites (e.g. customers), and importing logs takes a lot of time. Obviously, you use the dynamic resolver.
The dynamic resolver will fetch the site ID for a specific host the first time it parses a log line with that specific host. The site ID is then stored in cache and will be reused for each subsequent log line with that host.
If this specific site is deleted during the importing process, import_logs will raise an error:
Which is quite normal, as the site ID has disappeared. Unfortunately, the whole importing process is stopped. I think such an error should be handled gracefully and the importing process should continue.
The problem is that the import script doesn't know if an error is fatal or not. When an error occurs, I think Piwik should return it in a JSON format. Instead of returning:
it should return something like:
which can then be parsed by import_logs to determine if it's a fatal error or not.
The problem can be simulated by running import_logs with a static resolved the lines between:
and
(in StaticResolver.init) and adding:
below. Then run import_logs as usual with --idsite=42 (42 must be an invalid site ID).
Keywords: import_logs
The text was updated successfully, but these errors were encountered: