Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Dashboard for multiple sites (potentially hundreds of websites) #389

Closed
mattab opened this issue Oct 14, 2008 · 35 comments
Closed

Dashboard for multiple sites (potentially hundreds of websites) #389

mattab opened this issue Oct 14, 2008 · 35 comments
Assignees
Labels
Critical Indicates the severity of an issue is very critical and the issue has a very high priority. Enhancement For new feature suggestions that enhance Matomo's capabilities or add a new report, new API etc.
Milestone

Comments

@mattab
Copy link
Member

mattab commented Oct 14, 2008

Some Piwik users add a lot of websites in Piwik.
It is also common that users of a given Piwik installation can access more than 10 websites.
Currently they have to go to each of these websites to check their reports.

They need quick & easy way to get an overview of what’s happening in all their websites at once, and a quick access to more details.

Mockup

[[Image(http://issues.piwik.org/attachments/389/multi%20site%20report.PNG?format=raw)]]

Overview
- This page is space efficient but clear to display data about several websites
- gives an overview of what happened on all your websites, so you can choose which ones need investigation.
- display raw numbers
- and show the evolution of the metrics over the last days/weeks/month. via sparkline + red/green indicator
– this page links to each website report by a click on the website name
- possible to change the period of the dashboard: Day/Week/Month/Year
- each website is a simple row on the page
- for each website, print the
– website name
– nb of unique visitors
– nb of pages
– (keep space for nb of goal conversions – coming)
– a sparklines of the last 30 days/week/month/year for unique visitors
- when hovering the website name, the “drag icon” appears on the left of the website name. This icon makes draggable the list of websites so that each user can reorder their list of websites. By default website are ordered alphabetically. The website order in the list are saved for each user in the database (like the dashboard is), except for anonymous for whom the order is only stored in session (alternatively the feature can be disabled for anonymous user). The drag n drop feature would use Jquery UI draggable that we already use to reorder widgets in the dashboard.
- This dashboard can be accessed by a new entry “All websites” that always appears on the top of the “Website selector” on the top right of the page.

Speed
When loading 300 websites, we will lookup numerical values, for the period selected.

To show UU + visits + pages + conversions we need to load the 4 related numeric values for 300 websites. that’s 1 SELECT query to return 1200 values.

to show green/red evolution indicator we need to select the compared numeric value from yesterday/last week/last month/last year. that’s one SELECT to return 300 values.

loading 300 sparklines will take a loong time, as the browser will issue limited http request on the piwik domain. loading 300 sparklines will certainly take more than 20 seconds.
is that an issue?

Future releases
- adding sorting feature so that user can sort table by unique visitors, conversions, etc.
- adding search feature so that user can see only websites with names matching a string. the search box would have auto suggest feature.
- add possibility to choose the default page to arrive on – either the dashboard with all websites or the first available website in the list

Estimate

between 4 and 6 days of work
Keywords: multi-sites

@mattab
Copy link
Member Author

mattab commented Dec 3, 2008

Attachment:
[multi site report.PNG](http://issues.piwik.org/attachments/389/multi_site_report.PNG)

@zawadzinski
Copy link
Contributor

Current version in the attachment.

Now working on core to optimize Archive/Array/*.php

@anonymous-matomo-user
Copy link

I’ve installed and activated this plugin but there is no new entry in the “Website selector” on the top right of the page. Only my sites are listed, but there still isn’t an extra one for All websites. I installed and activated it on two different servers, and neither shows the new selection in the drop-down.

@anonymous-matomo-user
Copy link

Aha! I figured this one out. The documentation is incorrect. The plugin does work, just not as it states above. Rather than placing a new entry for All sites in the website selector in the top right of the page, it instead places an extra menu item in the main blue horizontal menu at the end after Dashboard, Visitors, Actions, Referrers. The new item is called Sites Summary.

@anonymous-matomo-user
Copy link

I originally reported this problem on the forums because trac was down for me, but now I’ve come here. Here’s a summary of the bug below, but a larger version of the error is at the forum thread located at http://forum.piwik.org/index.php?showtopic=831. Neither this report nor the one at the forum includes the entire error because it is incredibly huge.

Since I upgraded to version 0.2.33, the multisites plugin has produced an error – the FIRST time I run it each day. After the first time, it’s fine (until the next day). Here’s an edited version of the error. The real version was ENORMOUS. I’ve also edited my domain info to keep it generic.

Notice: Undefined property: Piwik_DataTable_Simple::$rows in /home/myfolder/public_html/piwik/core/DataTable.php on line 457 Backtrace —>#0 Piwik_ErrorHandler(8, Undefined property: Piwik_DataTable_Simple::$rows, /home/myfolder/public_html/piwik/core/DataTable.php, 457, Array ()) called at [/home/myfolder/public_html/piwik/core/DataTable.php:457]…………

@mattab
Copy link
Member Author

mattab commented Apr 30, 2009

see also: add a line with total for all websites: http://www.karlkopp.com/blog/2009/4/30/piwik-updated-multisitesplugin

@mattab
Copy link
Member Author

mattab commented May 13, 2009

Please update to the new MultiSitesPlugin.tar for compatibility with 0.2.35.

@anonymous-matomo-user
Copy link

This pops up with piwik 0.2.35 (using the updated version above):

Fatal error: Unsupported operand types in /var/www/apps/content/piwik/plugins/MultiSitesPlugin/Controller.php on line 204

@zawadzinski
Copy link
Contributor

Current version of MultiSitesPlugin. It still needs a lot of improvements and refactoring.

I am concerning rewriting the plugin to use directly piwik_archive_numeric_* tables in order to speed up the plugin (it is still very slow when even having "just" hundreds of sites).

If you got < 500 sites you can use this version (but remember to setup auto archiving http://piwik.org/docs/setup-auto-archiving/ - piwik won't be capable of archiving so many sites via web request!)

@zawadzinski
Copy link
Contributor

The Problem.
There is no way to use SQL query result limiting (LIMIT) and sorting (ORDER BY) using Piwik API. This is very important for MultiSitesPlugin, because when having thousands of sites we do not want to fetch all results, and sort & limit them in PHP.

Propsed solution.
We can build MultiSitesPlugin_API that will query directly SQL fetching data from piwik_archive_numeric_* tables. We do not want to check if data for specific site is archived, so we assume that all data is archived using cron auto-archiving (users will be noticed about it when installing MultiSitesPlugin).

This solution will significantly increase MultiSitesPlugin performance.

@zawadzinski
Copy link
Contributor

Attachment:
MultiSitesPlugin.zip

@mattab
Copy link
Member Author

mattab commented Jun 11, 2009

good stuff!
here is my feedback:

about the UI

  • the sparklines html includes could be loaded directly, no need to show the rolling image (ie. we know how the call looks like); currently it does one http request to request each image HTML but jquery could write directly the right html.
  • the numbers and text could be centered / aligned middle in the cell, and cells could be less high
  • 1 - 12 z 12 is in polish?
  • the CSS could reuse the existing data table CSS for UI consistency

thanks! it is getting real close to be included in core ;)

PS: we can include in core before the optimization happen on the php side, so please feel free to do the UI updates and then work on the php side if you wish to push this in core.

@anonymous-matomo-user
Copy link

Is this placement normal ?

[[Image(http://img229.imageshack.us/img229/4594/20062009083816.jpg)]]

If anyone know what to change in the main CSS file...

@zawadzinski
Copy link
Contributor

Attachment:
MultiSitesPlugin v0.2.zip

@zawadzinski
Copy link
Contributor

New version in the attachment.

We use now Piwik's CSS styles, and plugin Piwik coding standards are followed.

@anonymous-matomo-user
Copy link

I'm sorry to say, the "button" in the menu is still "misplaced" :

(Firefox 3.0.11)

[[Image(http://img526.imageshack.us/img526/3176/09072009100750.jpg)]]

@zawadzinski
Copy link
Contributor

Its caused by translation (that is too long and does not fit in the button), it is not this plugin related issue.

@anonymous-matomo-user
Copy link

Switching to english i still get this :

[[Image(http://img200.imageshack.us/img200/1931/10072009151203.jpg)]]

The submenu is always present no matter what language.

@mattab
Copy link
Member Author

mattab commented Jul 10, 2009

the location in the menu is temporary and will be fixed before submitting the plugin to the core - most likely it will appear as "All Websites" next to Dashboard, top left of screen.

@robocoder
Copy link
Contributor

  • For consistency, I'd suggest dropping the "Plugin" suffix from the plugin name.
  • Given the autoloader, the require_once's can be removed from Controller.php and MultiSitesPlugin.php.

@anonymous-matomo-user
Copy link

A few issues I have on this plugin. Firstly, it forces the php process to use 100% of the CPU for a long period of time (around 60 seconds - which would match the max process time in php.ini). It also throws a javascript error:

Webpage error details

User Agent: Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.1; Trident/4.0; SLCC2; .NET CLR 2.0.50727; .NET CLR 3.5.30729; .NET CLR 3.0.30729; Media Center PC 6.0; InfoPath.2; OfficeLiveConnector.1.3; OfficeLivePatch.0.0; .NET CLR 4.0.20506; Tablet PC 2.0;)
Timestamp: Wed, 15 Jul 2009 00:33:50 UTC

Message: Unknown runtime error
Line: 124348672
Char: 3
Code: 0
URI: http://example.com/piwik/index.php?module=CoreHome&action=index&idSite=2&period=day&date=2009-07-15

Message: Unknown runtime error
Line: 124348672
Char: 3
Code: 0
URI: http://example.com/piwik/index.php?module=CoreHome&action=index&idSite=2&period=day&date=2009-07-15

@anonymous-matomo-user
Copy link

this plugin can't work properly on my IE browser(can't list any website),on chrome and firefox it works well,is it a bug or I've done something wrong?

@mattab
Copy link
Member Author

mattab commented Nov 17, 2009

Once the few bugs listed here are fixed, we would like to commit it to trunk asap after Piwik 0.6. Maciej, would you mind taking a look at the reported issues? thanks!

@pebosi
Copy link
Contributor

pebosi commented Dec 4, 2009

Attachment:
MultiSites.zip

@pebosi
Copy link
Contributor

pebosi commented Dec 4, 2009

removed the suffix and the require_once calls. added a link to topbar (see included patch).

changed some stuff in index.tpl, but i was just testing, so there's a problem when changing date or period.

regards pebosi

@mattab
Copy link
Member Author

mattab commented Dec 9, 2009

Pebosi, can you please clarify what the issues are? What is missing for the plugin to be included in 0.5.1 in a stable state?
Thank you!

@pebosi
Copy link
Contributor

pebosi commented Dec 10, 2009

When changing the date the parameter "date" in url is not replace but it's appended to the url without an ampersand. so there's an error...
i tried using no ajax so i think there's a problem with the broadcast.

regards

@pebosi
Copy link
Contributor

pebosi commented Dec 10, 2009

Attachment:
MultiSites.2.zip

@robocoder
Copy link
Contributor

In [1678], refs #389, commit MultiSites plugin

@pebosi
Copy link
Contributor

pebosi commented Dec 14, 2009

Hi,

bugs in the last commit [1678]:

  • Site selector is not hidden
  • date change is not affected

regards pebosi

@mattab
Copy link
Member Author

mattab commented Dec 14, 2009

(In [1692]) refs #389 Fixing date selection, fixing column sorting, few styles modifications

@mattab
Copy link
Member Author

mattab commented Dec 14, 2009

(In [1696]) MultiSites plugin enabled by default refs #389

@mattab
Copy link
Member Author

mattab commented Dec 14, 2009

(In [1701]) refs #389 using the jquery magic so that IE works OK

@mattab
Copy link
Member Author

mattab commented Dec 14, 2009

Included and enabled by default in 0.5.3

Next steps are performance improvements, see #1077

@julienmoumne
Copy link
Member

(In [6478]) fixes #2708

refs #5491

  • PDFReports major refactoring. Any plugin can now add new kinds of reports. Required for Send reports via SMS #2708 and Report Publisher #3118.
  • test report functionality ($idReport == 0) dropped in Piwik_PDFReports_API->generateReport()
  • All Websites report shows 3 more metrics: Goal Conversions, eCommerce Conversions and eCommerce Revenue. Can be removed if asked to.
  • Piwik_PDFReports_API->sendEmailReport renamed to sendReport
  • All Piwik_PDFReports_API method signatures updated to support generic report parameters
    refs Dashboard for multiple sites (potentially hundreds of websites) #389
  • new API method to retrieve only one Piwik site : Piwik_MultiSites_API->getOne()
  • per Send reports via SMS #2708 description, Piwik_MultiSites_API methods now support a new parameter named enhanced. When activated, Goal Conversions, eCommerce Conversions and eCommerce Revenue along with their evolution will be included in the API output.
  • API metrics refactored in (@ignored)Piwik_MultiSites_API->getApiMetrics()
  • Metadata now returns 12 metrics : nb_visits, visits_evolution, nb_actions, actions_evolution, revenu, revenue_evolution, nb_conversions, nb_conversions_evolution, orders, orders_evolution, ecommerce_revenue, ecommerce_revenue_evolution
    refs Report Publisher #3118
  • ReportPublisher plugin could now easily be implemented
    commits merged
  • r6243
  • r6422 (Scheduled report by email with custom date range triggers error #3012)
    TODO
  • the MobileMessaging settings page may need some embellishment
  • @review annotations need some attention
  • test if the MultiSites API evolutions have some impact on Piwik Mobile and other client code

This issue was closed.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Critical Indicates the severity of an issue is very critical and the issue has a very high priority. Enhancement For new feature suggestions that enhance Matomo's capabilities or add a new report, new API etc.
Projects
None yet
Development

No branches or pull requests

6 participants