Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

make ordering of datatable filters that are always applied to DataTables correct/more flexible #5822

Closed
diosmosis opened this issue Jul 12, 2014 · 1 comment
Labels
c: Platform For Matomo platform changes that aren't impacting any of our APIs but improve the core itself. Enhancement For new feature suggestions that enhance Matomo's capabilities or add a new report, new API etc. wontfix If you can reproduce this issue, please reopen the issue or create a new one describing it.
Milestone

Comments

@diosmosis
Copy link
Member

Currently DataTable filter/manipulator ordering creates certain errors in API responses. For example, if filter_limit is supplied w/ flat=1, the limit will be applied to all loaded subtables creating potential inaccuracies. All of these bugs can be fixed by:

  1. turning all datatable manipulators into filters (there's no reason they have to be something else and manipulating in-place will be more efficient)
  2. merging DataTableGenericFilter w/ all other query parameter initiated filter/manipulators and clarifying the exact order of all filters (including 'queued' filters + 'priority queued' filters (see ViewDataTable))
@diosmosis
Copy link
Member Author

NOTES:

List of query parameter invoked filters & other filters (in order of execution):

  • Flattener (write, removes subtables)

    • merges subtables

    required pre-conditions:

    • none
  • ReportTotalsCalculator (read-only, requires INDEX values, but should not be necessary to use them)

    • calculates totals values

    required pre-conditions:

    • requested columns must exist
    • table hierarchy must be final and non-altered
  • priority queued filters (write, remove/add rows, remove/add/change columns, any sort of manipulation to non-data)

    required pre-conditions:

    • requires datatable to be in initial state. visualizations should not be required to know if the table has been flattened or not. this would mean, however, that they should run on subtables so if they are flattened the result is changed correctly.
  • ExcludeLowPopulation (write)

    • removes rows with column value lower than, etc.

    required pre-conditions:

    • dependent columns must be pre-prettified
  • SafeDecodeLabel (write, changes label values, not super necessary)

    • changes label values

    required pre-conditions:

    • table row count has to be considered final
    • has to be run before label columns considered 'final'
  • LabelFilter (write)

    • removes rows that do not match label query

    required pre-conditions:

    • labels must have FINAL values (table does not have to be final value)
  • Pattern(Recursive) (write)

    • removes rows that do not match pattern

    required pre-conditions:

    • dependent columns must be pre-prettified
    • table cannot truncated or limited
  • AddColumnsProcessedMetrics(Goal) (write)

    • adds lazy column values that calculate processed metrics

    required pre-conditions:

    • dependent columns must be pre-prettified
  • Sort (write)

    • re-orders rows

    required pre-conditions:

    • the column must exist and must have its pre-prettified value
  • Truncate (write)

    • cuts off after a certain amount

    required pre-conditions:

    • the table must be in the right ORDER
  • Limit (write)

    • takes subsection of whole table

    required pre-conditions:

    • the table must be in its FINAL state
  • presentation/queued filters

    required pre-conditions: datatable must be in final state

    • TODO: rename queueFilter() in DataTable to queuePresentationFilter()?
  • ColumnDelete (write)

    • removes column values

    required pre-conditions:

    • processed columns must be final

    TODO:

    • must modify so processed columns are set

NOTE: priority queued filters have to be able to run on pre-ReplaceColumnNames tables

Other notes:

  • concept of 'recursive' filter should be removed. all filters should be recursive, if it is not desired to run them on subtables, the subtables should not be loaded (ie, expanded=0)

@mattab mattab added this to the Short term milestone Aug 3, 2014
@mattab mattab added Enhancement and removed Bug labels Aug 3, 2014
@mattab mattab added the c: Platform For Matomo platform changes that aren't impacting any of our APIs but improve the core itself. label Sep 9, 2014
@mattab mattab removed the Bug For errors / faults / flaws / inconsistencies etc. label Dec 1, 2014
@mattab mattab modified the milestones: Mid term, Short term Dec 1, 2014
@mattab mattab removed the Major Indicates the severity or impact or benefit of an issue is much higher than normal but not critical. label Dec 1, 2014
@mattab mattab closed this as completed Jul 8, 2016
@mattab mattab added the wontfix If you can reproduce this issue, please reopen the issue or create a new one describing it. label Jul 8, 2016
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
c: Platform For Matomo platform changes that aren't impacting any of our APIs but improve the core itself. Enhancement For new feature suggestions that enhance Matomo's capabilities or add a new report, new API etc. wontfix If you can reproduce this issue, please reopen the issue or create a new one describing it.
Projects
None yet
Development

No branches or pull requests

2 participants