Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Static PNG Graph: plot several metrics & multiple row evolutions #3013

Open
mattab opened this issue Mar 6, 2012 · 32 comments
Open

Static PNG Graph: plot several metrics & multiple row evolutions #3013

mattab opened this issue Mar 6, 2012 · 32 comments
Labels
Enhancement For new feature suggestions that enhance Matomo's capabilities or add a new report, new API etc.

Comments

@mattab
Copy link
Member

mattab commented Mar 6, 2012

It would be very nice of the Static Image Graph API could plot several metrics at once, similarly to the JS graphs.

For example we could plot Visits and Pageviews on the same graphs with: index.php?module=API&method=ImageGraph.get&idSite=3&apiModule=VisitsSummary&apiAction=get&token_auth=anonymous&graphType=evolution&period=day&date=previous30&width=500&height=250&columns=nb_actions,nb_visits

  • For consistency we could rename parameter column to columns
  • Accept comma separated list of columns (consistent with other APIs)

The Static Image Graph API should also support a new parameter named &labels to plot multiple row evolutions.

  • Accept comma separated list of row labels
  • when $labels == false, plot the top N labels. N should be by default equal to 5 and can be overridden with the value of filter_limit
  • new metadata field : imageGraphEvolutionUrl
@mattab
Copy link
Member Author

mattab commented Mar 6, 2012

This feature would be very useful for:

@julienmoumne
Copy link
Member

This ticket seems to be more than plotting multiple metrics.

#3014 requires the ImageGraph plugin to also plot row(s) evolution.

The first question we need to answer is whether we :

  • add a new &labels parameter to ImageGraph.get, or
  • add a new API method ImageGraph.getRowEvolution

If the first option is selected, we need to define when should the Row Evolution API be solicited.

Here are the main two cases I could think of :

  • when at least one row label is provided via the &labels parameter
  • when &period &date correspond to multiple periods and the requested report has a dimension (such as Referrers, UserSettings, UserCountry and so on). Currently, ImageGraph.get generates an error see The graph cannot be drawn for this combination of 'date' and 'period' parameters.
  • should both cases above be required to solicitate the Row Evolution API or should one be sufficient ? For example, if no labels are supplied, maybe we could automatically plot ALL labels?

This evolution will also require to update the Metadata imageGraphUrl field.

The current behavior of the imageGraphUrl is as follow :

After updating the ImageGraph API to support row(s) evolution, we need to determine the value of the imageGraphUrl field for the first case (API.getMetadata Referers.get period=day date=last30).

Instead of rewriting &period &date, the imageGraphUrl would contain an API call which returns row evolution. However, we need to decide which label(s) should be drawn. Ideally, we could plot all labels. This gets complicated when the report does not have a constant number of labels (constantRowsCount!=1) or when the number of rows is too big.

@mattab
Copy link
Member Author

mattab commented Jun 19, 2012

Replying to JulienM:

This ticket seems to be more than plotting multiple metrics.

#3014 requires the ImageGraph plugin to also plot row(s) evolution.

Thank you for your great analysis :)

The first question we need to answer is whether we :

  • add a new &labels parameter to ImageGraph.get, or

I think this solution is nice indeed!

Here are the main two cases I could think of :

  • when at least one row label is provided via the &labels parameter
  • when &period &date correspond to multiple periods and the requested report has a dimension (such as Referrers, UserSettings, UserCountry and so on). Currently, ImageGraph.get generates an error see The graph cannot be drawn for this combination of 'date' and 'period' parameters.
  • should both cases above be required to solicitate the Row Evolution API or should one be sufficient ? For example, if no labels are supplied, maybe we could automatically plot ALL labels?

Agreed that when no labels are supplied, we could plot the evolution of the top 5 labels for example ?

After updating the ImageGraph API to support row(s) evolution, we need to determine the value of the imageGraphUrl field for the first case (API.getMetadata Referers.get period=day date=last30).

Instead of rewriting &period &date, the imageGraphUrl would contain an API call which returns row evolution. However, we need to decide which label(s) should be drawn. Ideally, we could plot all labels. This gets complicated when the report does not have a constant number of labels (constantRowsCount!=1) or when the number of rows is too big.

Plotting all labels it not so possible, because above 5 lines the graph becomes hard to read.

  • I propose that we plot the top 5 labels
  • the imageGraphUrl is rewritten with a new parameter &filter_limit=5 -- this will be used to limit the data from API but also the user could easily increase the value if he wishes to plot top 10 labels on a wider graph.
  • Even we could have fun with Thomas and for example plot top 10 lines when the graph is full screen on a tablet ;-)

--> &labels and &filter_limit support sounds like perfect API evolution for this feature!

There is also an open question about how to plot Ecommerce metrics, Goal metrics, on the graph. Maybe in a different ticket we could propose to have &enhanced=1 parameter that would return the goal/ecommerce metrics as well.

@julienmoumne
Copy link
Member

Attachment:
VisitFrequency with multiple metrics.png

@julienmoumne
Copy link
Member

@julienmoumne
Copy link
Member

@julienmoumne
Copy link
Member

Pending comment:3:ticket:3158.

If it can't be fixed, we can throw an exception when trying to plot row evolutions for Goal reports when Piwik_ImageGraph_API.get $idGoal is not empty.

We would also need to alter the imageGraphUrl construction logic to switch back to non-evolution graph.

@julienmoumne
Copy link
Member

The logic to retrieve the top 5 row evolutions should maybe be part of #3158, see comment:4:ticket:3158.

@julienmoumne
Copy link
Member

Attachment: &labels=Search Engines,Direct Entry,Websites&columns=nb_actions
Referers.getRefererType row evolution.png

@mattab
Copy link
Member Author

mattab commented Jun 24, 2012

Very nice graphs!!

If it can't be fixed, we can throw an exception when trying to plot row evolutions for Goal reports when Piwik_ImageGraph_API.get $idGoal is not empty.
You meant "when idGoal is empty" I suppose?

We would also need to alter the imageGraphUrl construction logic to switch back to non-evolution graph.
+1

  • As noted before, the row evolution should not require the &labels= parameter (for example to plot evolution of referrer types, &labels is not required)
  • This new ability to plot multiple values at once will be great when we implement "Compare the monthly report to the last month" refs #5192 Simplify / improve top menu  #291 and we could show, on the graph, the values for the current and previous month...

@julienmoumne
Copy link
Member

@julienmoumne
Copy link
Member

(In [6918]) refs #3013

  • plot multiple metrics using $columns (renamed from $column)
  • plot row evolutions using optional $labels, will plot top 5 labels when omitted
  • imageGraphEvolutionUrl added to report metadata (only for reports without constant row count and Referers.getRefererType)
  • $showMetricTitle renamed to $showLegend

refs #3014

  • new report parameter 'Show Historical Graphs for the top 5 values'
  • better looking evolution graph on landcape pages

refs #3158

  • $label is now optional, will return top N=filter_limit row evolutions sorted with filter_sort_column when omitted
  • support for $idGoal
  • integration tests updated

constantRowsCount set on

  • Goals_getDaysToConversion
  • Goals_getVisitsUntilConversion
  • Referers_getRefererType

merges

  • r6874

IDEA

  • in Piwik UI, add a link 'display top 10 row evolutions' using API.getRowEvolution &label=false&filter_sort_column=xyz&filter_limit=10

TODO

  • fix scheduled report integration tests
  • review @reviews

@mattab
Copy link
Member Author

mattab commented Sep 5, 2012

Great patch and improvements! :)

  • //@review without adding this, I can not set [...]

    That's expected, OK.

  • // The other way to solve this issue is to add the following exceptions in API.getRowEvolution: module=Goals AND (method=getVisitsUntilConversion OR getDaysToConversion)

--> This other way to solve would be better than the committed fix: sometimes it is important to delete the rows that have visits=0 but some conversions, because sometimes the "Keywords" or "websites" used in previous visits can be credited a "conversion" if they are the last referrer.

So not deleting the conversion==0 row in this datatable would result in the UI showing rows with 0 visits which was a bug we fixed with this code to delete rows :)

  • Regarding code:

    1135                            // retrieve top labels 
    1136                            $dataTableArray = $this->loadRowEvolutionDataFromAPI( 
    1137                                    $idSite, 
    1138                                    $period, 
    1139                                    $date, 
    1140                                    $apiModule, 
    1141                                    $apiAction, 
    1142                                    null, 
    1143                                    $segment, 
    1144                                    $idGoal 
    1145                            ); 
    1146     
    1147                            // get the last, ie. most recent, datatable 
    1148                            $dataTables = $dataTableArray->getArray(); 
    1149                            $mostRecentDataTable = end($dataTables); 
    1150     
    1151                            $labels = $mostRecentDataTable->getColumn('label'); 

Here it would be more efficient to directly call the API using Piwik_API_Request() and http_build_query() to build the request string.
The advantage will be better performance since you can only request the last period's worth of data.

You should be able to use the functions: Piwik_Archive::isMultiplePeriod and Piwik_Period_Range($dateString) to get the last date of the range.

let me know if there is any problem doing this change? it will speed up graphs.

  • I don't think this change is necessary? why adding the clause here?

1244    1269                    // add "processed metrics" like actions per visit or bounce rate 
1245                        if ($apiModule != 'Actions') 
    1270                    if ($apiModule != 'Actions' && $label) 

@julienmoumne
Copy link
Member

(In [6925])
refs #3158

  • adding more report exceptions from being filtered by AddColumnProcessedMetrics filter
  • I forgot to manage an edge case when $label=false and there are no data, what should the API return ? see @review
  • only retrieve the last period when looking for top n labels

refs #3013

  • better error message when specifying invalid metrics in $columns
  • same image height for all graph types in all page orientations for pdf reports

@mattab
Copy link
Member Author

mattab commented Sep 7, 2012

paste due to misclick on my new laptop ;)

@julienmoumne
Copy link
Member

(In [6947]) refs #1721 refs #3013

  • show label icons in evolution graph
  • smaller legend font size
  • draw colored shadows in legend
  • vertical legend for evolution graphs
  • enhanced label skipping for daily evolution graph

@julienmoumne
Copy link
Member

(In [6951]) refs #1721 refs #3013

  • better legend font sizes
  • legend shadow only when plotting multiple series
  • more shadow padding

@julienmoumne
Copy link
Member

(In [6969]) refs #3013

  • reduce shadow width to clear off series' icons
  • fix shadow on big font sizes
  • increased line icon width in evolution graphs

@julienmoumne
Copy link
Member

(In [6970]) refs #3013

  • forgot to update Piwik_ImageGraph_StaticGraph_Exception

@julienmoumne
Copy link
Member

(In [6982]) refs #3013

  • horizontal shadow padding
  • top & left margin added for vertical legends

@julienmoumne
Copy link
Member

(In [7009]) refs #3013

  • fix pie charts

@mattab
Copy link
Member Author

mattab commented Sep 18, 2012

This is a test comment but a real comment too, awesome work & can't wait to see the next improvements ;-)

@julienmoumne
Copy link
Member

(In [7026]) refs #3013

  • remove value markers on evolution graphs

@julienmoumne
Copy link
Member

(In [7077]) refs #3013

  • truncate legend labels in evolution graphs

@julienmoumne
Copy link
Member

(In [7078]) refs #3013

  • rounding top scale value to the next multiple of 10

@julienmoumne
Copy link
Member

(In [7079]) refs #3013

  • more labels displayed on vertical bars and evolution graphs

@julienmoumne
Copy link
Member

(In [7086])

refs #3158 refs #3013

  • remove metric names from metadata with new API parameter $legendAppendMetric == false (defaults to true)

refs #3158

@julienmoumne
Copy link
Member

(In [7088])
refs #3013

  • truncate vertical legend labels at 70% of total image width
  • reduce default legend font size

@julienmoumne
Copy link
Member

(In [7102])
refs #3013

  • fix label skipping algorithm when plotting data using period=day&date=previous|last7

@mattab
Copy link
Member Author

mattab commented Oct 15, 2012

(In [Refs #3013 I'm in a rush, but I had to add this code as the $periodsData$i) was NULL in my case, not sure why it hasn't been needed until now, so maybe there is an underlying problem in Site Search, or maybe this change is acceptable...?

@mattab
Copy link
Member Author

mattab commented Oct 24, 2012

(In [7303]) Fixes #3474

  • sometimes width or height are not defined, in particular when the API fails before the $width and $height are defined in line 216-217 of ImageGraph/API.php
    so we always check the error graph has a width/height set.

Ping Refs #3013

@mattab
Copy link
Member Author

mattab commented Mar 2, 2013

This ticket is getting long, What is the status of #3013?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Enhancement For new feature suggestions that enhance Matomo's capabilities or add a new report, new API etc.
Projects
None yet
Development

No branches or pull requests

2 participants