Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add possibility to profile archiving #12935

Closed
tsteur opened this issue May 18, 2018 · 9 comments
Closed

Add possibility to profile archiving #12935

tsteur opened this issue May 18, 2018 · 9 comments
Assignees
@diosmosis
Labels
c: Performance For when we could improve the performance / speed of Matomo.
Milestone
3.6.0

Comments

@tsteur
Copy link
Member

tsteur commented May 18, 2018 •
edited

We already log some information in debug level. However, it is not practical in Matomo to lower the debug level because then those messages would be sent to the user.

It would be useful to have a config like archiving_profile_cli=0/1 or two configs like archiving_profile = 0/1 and archiving_profile_mode[]='cli' / 'web'.

The output of this should be visible in the cli archive log if easily possible otherwise only in logs.

We should log info:
Plugin Archiving: plugin: $PluginName, period: $period, segment: $segmentStringIfSet, siteId: $site, Duration $duration, Memory Peak: $memoryPeak

We don't need summary per query for now. It is only for an indication to see how long archiving takes in various places.
@tsteur tsteur added the c: Performance For when we could improve the performance / speed of Matomo. label May 18, 2018
@tsteur tsteur added this to the 3.6.0 milestone May 18, 2018
@diosmosis
Copy link
Member

Is it required to have two modes cli vs web? I can't tell what the difference in output/behavior would be.

@tsteur
Copy link
Member Author

tsteur commented Jul 24, 2018

The thing is that you want to profile the archiving but not have any logs visible in the UI and you need to ensure in "web" the profiling won't be activated.

@diosmosis
Copy link
Member

If the logs are made w/ INFO level, then they will appear in CLI output but not in UI output. Is that the effect you're looking for?

@tsteur
Copy link
Member Author

tsteur commented Jul 24, 2018

Here it is less about the log level as we don't want to have any other output with "info" level logged to the CLI output. It was really meant to only enable specific profiling messages but not any other log messages that may be suppressed because of the log level.

What I need is basically a possibility to see some profiling information in the CLI output of the archiver when enabled. But those messages should not be shown in the UI in any way. We already have enable_sql_profiler = 0 so would need something like enable_archive_profiler = 0/1. But the messages should never make it to the UI and only be printed in the CLI output when enabled.

Hope that makes it a bit more clear.

@diosmosis
Copy link
Member

I still don't see the need for archiving_profile_mode[] = 'web'/'cli'. If enable_archive_profiler=1 and INFO level logs are used, then it will behave as you expect, correct? Am I missing something?

@tsteur
Copy link
Member Author

tsteur commented Jul 24, 2018

The default log level is warn. We can't change our log level to info.

Those profiling messages could be logged as warning when it is enabled so they appear with the default log level. The mode cli/web may not be needed if those messages don't appear in the UI. If those messages would somehow end up in the UI, then a mode or something will be needed.

@diosmosis
Copy link
Member

The default log level is warn. We can't change our log level to info.

But that's to the UI, no? For the CLI the log level should be INFO.

If those messages would somehow end up in the UI, then a mode or something will be needed.

Ok, then to my understanding the requirement is to display these in the CLI output for core:archive when enabled, but never in the UI (correct me if I'm wrong).

@tsteur
Copy link
Member Author

tsteur commented Jul 24, 2018

But that's to the UI, no? For the CLI the log level should be INFO.

Not that I know but it might be. AFAIK there is only one log level configured in the config ini.

@diosmosis
Copy link
Member

I think it's changed in DI in the Monolog plugin (Monolog/config/cli.php).

@diosmosis diosmosis self-assigned this Jul 26, 2018
@diosmosis diosmosis mentioned this issue Jul 27, 2018
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@diosmosis diosmosis

Labels
c: Performance For when we could improve the performance / speed of Matomo.
Projects
None yet
Milestone
3.6.0
Development

No branches or pull requests
2 participants
@diosmosis @tsteur