Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adding new PivotByDimension DataTable filter that can pivot a report by (almost) any dimension. #6243

Merged
merged 30 commits into from Sep 21, 2014

Conversation

diosmosis
Copy link
Member

Pull request for #6078.

Adds a new DataTable filter that can be invoked via query parameters. Filter allows pivoting a report by another dimension.

In the UI, feature can be accessed through cog option 'Pivot by subtable'.

Includes non-pivot related changes:

  • change to CSV DataTable renderer so column names w/ commas & quotes can appear in text
  • change to XML DataTable renderer so column names w/ invalid XML characters can be rendered (bit of an iffy change, XML format needs an overhaul I think)

Known limitations:

  • in the UI can only pivot by subtable
  • pivoting by segment is slow, will hopefully be improved in the future
  • cannot fetch by segment for some reports, eg, UserCountry.getCity, since it would require setting other segments in addition to city=...
  • the new filter does not play nice w/ queued filters. this is due to nature of queued filters and the complexity they impose on required filter order. after I refactor processed metrics, won't be an issue though.

Notes on tests:

Includes unit & integration tests. I did only some manual tests, too tired right now to do more.

UI tests will be added later.

diosmosis and others added 13 commits September 17, 2014 20:40
…by (almost) any dimension. The filter can pivot reports by their subtable dimension and can also pivot by other dimensions (by using segments).

Notes:

- in the UI, only pivoting by subtable is supported
- change to CSV DataTable renderer so column names w/ commas & quotes can appear in text
- change to XML DataTable renderer so column names w/ invalid XML characters can be rendered (bit of an iffy change, XML format needs an overhaul I think)
- includes new config option 'pivot_by_filter_enable_fetch_by_segment'
- includes additions to component metadata classes (ie, Report/Dimension)
…when fetching intersected tables by segment.
…ure pivotBy works w/ Events plugins' secondaryDimension query parameter..
…tables of specific reports and use in Events to display nb_events. Also do not set columns_to_display in Events plugin if table is pivoted.
. "pivot by dimension = {$this->pivotByDimension->getId()}]");
}
} else {
$canFetchBySubtable = !empty($this->subtableDimension)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

you may reuse isPivotDimensionSubtable() here

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That won't provide as detailed exception messages; splitting the call up will be better for debugging.

@mattab
Copy link
Member

mattab commented Sep 20, 2014

  • Well done for this PR and the way you implemented Pivot! this is brilliant!
  • The tests are really well made and readable too!
  • Small note: instead of writing testXMLRendererSuccessfullyRendersWhenDataTableColumnsHaveInvalidXmlCharacters maybe write with _ eg testXMLRenderer_SuccessfullyRenders_WhenDataTableColumnsHaveInvalidXmlCharacters - kinda makes the test file easier to dig in as function names act as Index... maybe we can rename all existing test functions to comply at some point as I noticed old tests have old name schema

The other changes discussed:

  • Pre-pend the column id in each column name,
  • add a column 'Others' containing the sum of remaining values.
  • change pivot_by_filter_default_column_limit to show 10 columns by default.
  • etc.
    will make this feature simply amazing!

Looking forward to next iteration and feel free to merge once review items are covered!

Notes:

@diosmosis
Copy link
Member Author

Re test names: I don't mind changing the test names, but it will be less readable when used w/ --testdox if both underscores and camel case are used. Thomas and I were looking at codeception a while back as an alternative for PHPUnit for better looking tests, maybe we can start a discussion about switching.

diosmosis added 5 commits September 20, 2014 16:02
…ivotted and has no subtables (eg, when flattened).
…e calling disablePivotBySubtableIfTableHasNoSubtables.
…; (use decoded entity in PHP and do not select columns using jquery selector in dataTable.js).
…o save viewdatatable params when un-pivoting.
mattab pushed a commit that referenced this pull request Sep 21, 2014
Adding new PivotByDimension DataTable filter that can pivot a report by (almost) any dimension. refs #6078
@mattab mattab merged commit 2fc11ee into master Sep 21, 2014
@mattab
Copy link
Member

mattab commented Sep 21, 2014

Nice work @diosmosis 💯

@mattab mattab added the Enhancement For new feature suggestions that enhance Matomo's capabilities or add a new report, new API etc. label Sep 21, 2014
@mattab mattab added this to the Piwik 2.7.0 milestone Sep 21, 2014
@mattab mattab added the c: Platform For Matomo platform changes that aren't impacting any of our APIs but improve the core itself. label Sep 21, 2014
@diosmosis diosmosis deleted the 6078 branch September 21, 2014 06:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
c: Platform For Matomo platform changes that aren't impacting any of our APIs but improve the core itself. Enhancement For new feature suggestions that enhance Matomo's capabilities or add a new report, new API etc.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants