Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Display a new row "Total" showing the sum of values for all DataTable columns #5267

Closed
mattab opened this issue May 30, 2014 · 8 comments
Closed
Assignees
Labels
c: Usability For issues that let users achieve a defined goal more effectively or efficiently. Enhancement For new feature suggestions that enhance Matomo's capabilities or add a new report, new API etc. Major Indicates the severity or impact or benefit of an issue is much higher than normal but not critical.
Milestone

Comments

@mattab
Copy link
Member

mattab commented May 30, 2014

Since #57 we process in Metadata Reports, a new <reportTotal>

For example in this metadata api output

    <reportTotal>
        <nb_visits>3</nb_visits>
        <nb_uniq_visitors>2</nb_uniq_visitors>
        <nb_actions>13</nb_actions>
        <nb_conversions>3</nb_conversions>
        <bounce_count>0</bounce_count>
        <revenue>3121</revenue>
    </reportTotal>

For Actions.getPageUrl here is the total (source code):

    <reportTotal>
        <nb_hits>4</nb_hits>
        <entry_nb_visits>2</entry_nb_visits>
    </reportTotal>

The goal of this issue is to:

  • Check/ensure that the <reportTotal> includes the total for all columns that can be summed
  • Add a new Cog icon option to show the Report Total in a new row in the report
  • By default we don't enable it currently, but we may enable it in the future

Feel free to add your feedback as a comment here!

@mattab mattab added this to the 2.x - The Great Piwik 2.x Backlog milestone Jul 8, 2014
@mattab mattab removed the P: normal label Aug 3, 2014
@rivadav
Copy link

rivadav commented Sep 25, 2014

+1 for disabling it by default
+1 for putting it into settings under Cog icon

Perhaps we can also make it a global setting in Settings/Manage/User settings for users convenience.

Another idea - after turning totals on for the first time for specific widget, we can ask user if he wants to display it for all tables (and where to disable it if he changes his mind).

What do you think?

@mattab
Copy link
Member Author

mattab commented Sep 26, 2014

@rivadav Good ideas and suggestions! I moved ticket to Short term.

@mattab mattab modified the milestones: Short term, Mid term Sep 26, 2014
@mattab mattab added c: Usability For issues that let users achieve a defined goal more effectively or efficiently. and removed c: UI - UX (AngularJS twig less) labels Oct 12, 2014
@mattab mattab added the Major Indicates the severity or impact or benefit of an issue is much higher than normal but not critical. label Apr 8, 2015
@mattab mattab modified the milestones: Mid term, Short term Apr 8, 2015
@mattab mattab added duplicate For issues that already existed in our issue tracker and were reported previously. and removed Major Indicates the severity or impact or benefit of an issue is much higher than normal but not critical. labels Dec 5, 2016
@mattab mattab modified the milestones: Long term, Mid term Dec 5, 2016
@mattab mattab removed the duplicate For issues that already existed in our issue tracker and were reported previously. label Dec 5, 2016
@mattab mattab modified the milestones: Backlog (Help wanted), 3.7.0 Jun 14, 2018
@mattab mattab added the Major Indicates the severity or impact or benefit of an issue is much higher than normal but not critical. label Jun 29, 2018
@tsteur
Copy link
Member

tsteur commented Oct 8, 2018

fyi: looking into report totals calculator this currently only works for a few defined metrics that can be easily summed. It doesn't work for avg, min, max, or any processed metric etc. This won't be too easy to develop.

You'd think using sumRow might help like this (as it supports min, max, and custom aggregators): https://github.com/matomo-org/matomo/compare/5267?expand=1

but even this won't work in all cases. Eg many filters and processors wouldn't be applied, depending on the report you may even need to apply the queued filters first or later after summing all rows etc. Also it wouldn't process any processed metrics. for many reports it would likely work just like that but not for all and not in all cases depending on which features are used.

There may be a way by not using reportTotalCalculator but it means changing quite a lot of core code and not only introducing a summary row but also a total row. We would probably need to fetch this information directly when fetching the original archive when we build the data table basically eg in Archive::createDataTableFromArchive. It may require a lot of code change and even plugins to consider it. The "total" row would need to be part of the generic data table that reports fetch, otherwise they cannot modify it, or react to it etc. It would be many many days of work I'd say.

I can create a simple PR for it and maybe someone can have a look to see if the data is mostly correct? Note: Because it calculates totals also for averages, it will show on hover percentage values for pretty much all columns now. This would be fixed and removed so they only appear on those columns as defined before. The grey percentage values can only be calculated for values that can be summed.

Todo: It would also need lots of extra code in the visualization to not activate things like Row evolution etc. If the solution was built properly by doing this in archive creation as suggested, things like Row evolution would work for this likely. But it would also turn it into MANY days of work.

Note: Would need to be tested how this affects performance.

@mattab
Copy link
Member Author

mattab commented Oct 8, 2018

There may be a way by not using reportTotalCalculator but it means changing quite a lot of core code and not only introducing a summary row but also a total row. We would probably need to fetch this information directly when fetching the original archive when we build the data table basically eg in Archive::createDataTableFromArchive. It may require a lot of code change and even plugins to consider it. The "total" row would need to be part of the generic data table that reports fetch, otherwise they cannot modify it, or react to it etc. It would be many many days of work I'd say.

This solution sounds good. Also for tables that have sub-tables, then the only way to get totals across all sub-tables is to process them at archiving times as you explain here?

So moving this out of 3.7.0 for now as it's a bit too complicated maybe?

@mattab mattab modified the milestones: 3.7.0, 3.8.0 Oct 8, 2018
@tsteur
Copy link
Member

tsteur commented Oct 8, 2018

Nope, I meant it needs to fetch the dataTable that is being fetched in ReportsTotalsCalculator directly during Archive:: createDataTableFromArchive maybe. There is a PR for the first solution and it might work #13555

@tsteur tsteur modified the milestones: 3.8.0, 3.7.0 Oct 8, 2018
@tsteur tsteur self-assigned this Oct 9, 2018
@tsteur
Copy link
Member

tsteur commented Oct 9, 2018

Implemented feature in #13555

FYI: I didn't add a global setting to enable/disable it and neither ask on activation whether to enable it for all reports. That would be a new generic feature. Also you would kind of need to ask when disabling it for one report, whether to disable it for all reports otherwise you need to manually disable it everywhere etc.

@diosmosis
Copy link
Member

@tsteur this issue is done right? Or was there a requirement not fulfilled by #13555?

@tsteur
Copy link
Member

tsteur commented Dec 17, 2018

Should be all done 👍

@tsteur tsteur closed this as completed Dec 17, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
c: Usability For issues that let users achieve a defined goal more effectively or efficiently. Enhancement For new feature suggestions that enhance Matomo's capabilities or add a new report, new API etc. Major Indicates the severity or impact or benefit of an issue is much higher than normal but not critical.
Projects
None yet
Development

No branches or pull requests

4 participants