Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

import_logs: gracefully handle deleted sites ID #3627

Open
cbay opened this issue Dec 17, 2012 · 0 comments
Open

import_logs: gracefully handle deleted sites ID #3627

cbay opened this issue Dec 17, 2012 · 0 comments
Labels
Enhancement For new feature suggestions that enhance Matomo's capabilities or add a new report, new API etc.

Comments

@cbay
Copy link
Contributor

cbay commented Dec 17, 2012

Use case: you have many sites (e.g. customers), and importing logs takes a lot of time. Obviously, you use the dynamic resolver.

The dynamic resolver will fetch the site ID for a specific host the first time it parses a log line with that specific host. The site ID is then stored in cache and will be reused for each subsequent log line with that host.

If this specific site is deleted during the importing process, import_logs will raise an error:

Fatal error: The requested website id = 42 (or its property main_url) couldn't be found

Which is quite normal, as the site ID has disappeared. Unfortunately, the whole importing process is stopped. I think such an error should be handled gracefully and the importing process should continue.

The problem is that the import script doesn't know if an error is fatal or not. When an error occurs, I think Piwik should return it in a JSON format. Instead of returning:

The requested website id = 42 (or its property main_url) couldn't be found 

it should return something like:

'{"message": "Fatal error: The requested website id = 42 (or its property main_url) couldn\'t be found", "code": 3, "severity": "low"}' 

which can then be parsed by import_logs to determine if it's a fatal error or not.

The problem can be simulated by running import_logs with a static resolved the lines between:

# Go get the main URL

and

self._main_url = site['main_url']

(in StaticResolver.init) and adding:

self._main_url = 'http://example.org'

below. Then run import_logs as usual with --idsite=42 (42 must be an invalid site ID).
Keywords: import_logs

@cbay cbay added this to the 2.x - The Great Piwik 2.x Backlog milestone Jul 8, 2014
@mattab mattab modified the milestones: Long term, Mid term Dec 5, 2016
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Enhancement For new feature suggestions that enhance Matomo's capabilities or add a new report, new API etc.
Projects
None yet
Development

No branches or pull requests

2 participants