Not 100% sure what you mean by that but shouldn't there be an API for this? Probably via Report Metadata? Would also allow other apps such as Piwik Mobile to get this configuration/metadata

At some point later when moving to Angular ideally Piwik also only uses getReportMetadata and getProcessedReport as does Piwik Mobile. Not more. This makes sure to have consistent UI and features across apps and allows other developers to build similar apps. For instance for desktop or their own UI.

Using the API means creating AJAX requests every time report metadata is needed when the data is only changed during plugin enabling/disabling. Storing in tmp/assets and serving as merged JS is served will remove many AJAX calls. Other apps can use this data since it will essentially be JSON. Or they can go through the old metadata API. They are not relevant to this ticket.

To clarify, the issue isn't about replacing metadata classes w/ JSON files, just to store the data as an asset so it doesn't have to be loaded more than once by the UI.

Ah you mean we do not have to load the PHP Classes more than once? So it is about caching and performance? And it will be merged with all the other JavaScript files?

FYI: In Piwik Mobile I request report metadata only once to get all reports and then getProcessedReport for each individual. In theory we could maybe request report metadata as json and store this as JSON asset? It would be nice to use same API's etc for Piwik and Piwik Mobile so we don't have to maintain two solutions. Report metadata could be enriched for this which I need in Piwik Mobile anyway

Sorry @tsteur never got an email about this comment.

It's more about not loading the metadata in the UI more than once, and getting data for Dimensions/Segments and everything else, not just what Piwik provides in the currently provided API methods. Loading the data once in Piwik Mobile is fine, but in a browser it won't work since Piwik doesn't provide cache headers for normal API responses.

we could maybe request report metadata as json and store this as JSON asset?

This is the idea essentially, though I would want the metadata (enriched or new) to mirror the Report/Dimension/Segment classes as closely as possible. And to actually provide Dimension/Segment info. And maybe other components that may be added in the future like Metrics/ProcessedMetrics.

I think the getProcessedReport and other reports should eventually be deprecated; I don't think they will scale in terms of wealth of information. And they mix concerns (report data + report metadata).

EDIT:

And it will be merged with all the other JavaScript files?

I was considering creating a separate file that would just be JSON, like, tmp/assets/metadata.json (w/ the additional tmp/assets/metadata.json.deflate). Ie, it wouldn't be part of the merged JS, just stored & served like it.

The Metadata can change per-user I think, so if we cache it we likely should cache per-user / per-website

Do you remember how it can change per-user? I know it can change per website due to goals (though I want to change that somehow).

It does not currently change per-user I believe, but it would make sense that in the future it will change per user.

I know it can change per website due to goals (though I want to change that somehow).

do you have some idea how this could be done?

do you have some idea how this could be done?

Not at the moment, I haven't actually thought about what would be required. In fact, I don't think I could come up w/ ideas w/o experimenting.

But! I think it should be done. Metadata should not be dynamic.

Metadata should not be dynamic.

I'm not convinced of that yet... so let's discuss again when this becomes more important.

Hard to explain exactly what my issue is, but here's what I see is the problem:

In the Report classes, there's just one for the Goals.get (Reports/Get.php). In the API, there is just one Goals.get method. And yet in report metadata, there are numerous reports that call Goals.get. This is inconsistent.

In this particular case, I'd say idGoal is a secondary dimension for the report. Perhaps the main problem is that Reports aren't allowed to have multiple dimensions?

Perhaps the main problem is that Reports aren't allowed to have multiple dimensions?

yes maybe allowing reports to have multiple sub-reports would make sense.

Maybe you can create issue for this idea, and close this one?

This issue which is about allowing all report metadata to be cached as an asset and loaded in the UI is still useful I think.

Ok good point, so we could simply make our response dynamically serve 304 headers when content hasn't changed so it's cached in browser but still allows dynamic change of metadata .

BTW: In theory plugins can already limit reports etc for different users etc. I would not be surprised if such plugins are out there

@tsteur That seems ok since the asset would have to be regenerated on plugin enable/disable? Anyway, this all sounds like it would be better w/ a POC.

I reckon it means it is about caching/creating compiled asset.js based on site and user. In theory metadata could be also based on any other parameters. Depending what plugins do... So we would have to create something like a hash per generated metadata file and save different compiled asset.js or so (if we ship metadata.js with asset.js). Don't know you are more into this topic ;)

Like I said above, I would prefer if metadata weren't able to change based on any parameters, only on what exists in the code (ie, API methods). This would require separating filtering concerns (ie, what user has permissions to what reports) from metadata concerns (what is the name of what report) which I think is only tenable for the web UI (in other words, for code we control). But then, that's what I'm actually interested in ;) Specifically for converting to angularjs.

Anyway, if I ever find the time I'll post POC and maybe the idea will be more fleshed out.

I think I know what you have in mind and can imagine an implementation :) Still, the platform should be not limited in such a case in my opinion. That's actually a big advantage of Piwik. There might be still a solution with changing based on any params. Hope you'll find some time ;)

This will be done as part of the UI & Client side improvements project, where we slowly move logic from server side controllers towards client side controllers (AngularJS), and improving our HTTP APIs so that clients (eg. browser) can do everything with such APIs. Some work is already done in Piwik 3.0 branch, for more info see this issue #7131