Matomo Issue #572 - Core: Provide custom date range feature

@mattab opened this Issue on February 27th 2009 Member

Piwik provides reports for days, weeks, month and year, but not for custom date ranges.

The mechanism used to process reports for weeks, month and year use a generic code that could be used to generate reports on any custom date range.

List of changes to make this happen:

the date range should be set in the "date" parameter and not in a new parameter date_range, and "period" parameter would be "range"
Archiving: when period=range, the Archiving code will test if an archive exists already. If it doesn't, it will look at the most efficient way to process the range: the minimum number of sub date ranges.
- For example, requesting data from Jan 1st to March 15th 2009:
- we can sum jan month + feb + sum of the first 15 days of july? which is 17 archives to select and sum
- but if we minimize the nb of date ranges in the requested range, by doing: jan + feb + day 1st of march + week from 2nd to 8th + week from 9th to 15th, we only select & sum 5 archives which will be a lot more efficient and better user experience.
  The archive sum of all sub archives will then be recorded and read as normal.
UI: period selector will change to use a calendar that allows for date range selector (example). We could add "Date range" after "day, month, year" buttons, when clicked would show 2 calendars.
UI: maybe it would be nice to show date ranges shortcut, "last 7 days", "last 30 days" somehow ?
QA: add new integration tests that test with the date range "date=2011-01-01,2011-03-22&period=range"
Note: none of the plugins code for Archiving, API querying etc. will need to be modified, since all the logic is hidden

Any ideas? Any volunteer to work on this interesting challenge? Please comment :)

@anonymous-matomo-user commented on April 7th 2009

Think this is a must as a core feature, and was the first thing one of my users asked :)

@yan- commented on July 6th 2010

I think this is a must-have. Amazing that there is so little request for this feauture.

@anonymous-matomo-user commented on August 3rd 2010

Agree, this is a must have feature.

@gka commented on August 5th 2010 Contributor

I would suggest a different spec than mentioned in the ticket desc:

Instead of introducing a new "custom" period, I would keep the periods day, week, month and year.
There are two modes: the first is the current behavior where there is a parameter "date" and the exact date range is selected automatically, (either the day itself or the week/month/year that contains this date). The second mode (the new one) uses the parameters fromDate and toDate instead of just date. This will define the custom date range.
The selected period now defines how to sum up the data, if the range is from Jan 1st until March 15th and the period is week, the weekly values are summed up.
If the user selects a date range that would require much computing time (eg. period: day and range: Jan 2009 until Jul 2010), Piwik will display a warning before the computation starts..

@mattab commented on August 29th 2010 Member

See also 'Week beginning setting' #1633

@mattab commented on November 16th 2010 Member

When this is implemented, we can improve user settings, and allow to set the default date to "Last 7 days", "Last 30 days", etc.

@anonymous-matomo-user commented on December 23rd 2010

I agree with @greg's suggestion of implementation. Has anyone begun on this? If not, I can look into it, but I don't want to step on someone else's half-completed implementation.

Thank you.

@mattab commented on December 23rd 2010 Member

mrisher, work hasn't started on this so feel free to pick it up. However, I think the proposed implementation should be followed (Piwik chooses which is the most efficient way of archiving custom date ranges, selecting weeks and/or months when this is faster).

@anonymous-matomo-user commented on January 3rd 2011

Hi, Matt:
Unfortunately, our project needs have shifted a bit and so I don't think that we'll be working on this feature in the short term.

I wanted to update the comment so nobody else was scared off.

Sorry,
--Mark

@anonymous-matomo-user commented on February 3rd 2011

I'm working on this preferring a last N Days range since I do not need a in between Dates range. Hope to come up with this soon.

Regards, Ron

@mattab commented on March 21st 2011 Member

(In [4159]) Refs #572

Implementing Custom Date range in Core, eg "period=range&date=2010-01-05,2010-02-03"
Only the requested plugin's report will be processed, ensuring fast responses
when a large period is requested, the code will try and select an optimal number of sub periods (see Range.test.php) - a mix of weeks, months and days inside the requested range
adding integration testing + unit test for optimal subperiods algorithm

TODO

we should also use Yearly archives whenever possible (when date range is very large)
if custom date range start date == website creation date or before, can use more optimal subperiods (using starting month / year / week)
Test/implement "last N days"
Test with timezones when the custom date range ends today
UI to allow selecting custom range (show 2 calendars)
Update API reference doc + add example in API page listing
Add "last 7 days", "last 30 days" in the user settings for "Default date to load"

NB: 3 tests fail in Main.test.php on my box, but I'm really not sure why... let's see if Jenkins fails as well

@mattab commented on March 23rd 2011 Member

(In [4170]) Refs #572 Fixing bug when the range goes in the future of current month, forcing to stop at the specified end date

@mattab commented on March 23rd 2011 Member

(In [4171]) Refs #572 Health checks

@mattab commented on March 23rd 2011 Member

Note: update FAQ: http://piwik.org/faq/how-to/#faq_33 when feature released

@mattab commented on March 25th 2011 Member

(In [4174]) Refs #572 Fixing known archiving issues when period=range