Will importing history from GA cause duplication in Piwik? e.g. For a site which has been using GA for a year, but only 3 months in Piwik, we might want to import the previous 9 months to get 1 year of history.
But not the overlapping recent 3 months.
Do you plan on deduping or providing a data range filter?
Thanks

What's the overall status of this? It seems that there's no activity at all in both issues. Do you have this in your short term roadmap?

There is no update and while it is currently in the Short term it may not be done in short term due to the complexity of the task.

This feature would be great!

This is great feature!

We are really looking forward to this feature in our company. We cannot ask our clients to use our central Piwik installation until this has been released, since they have been relying on Google Analytics for too long and wish to keep their historical data.

+1

What about importing raw google analytics data? I have a copy of every hit saved, can these be imported into piwik?

+1

@mattab cool, thanks! I wish I could help, will just watch for an update for now.

Really interested in this.

@mattab I can look at getting some of my guys involved in this later in February there is no problem. We are in the process of developing additional implantation with our CMS engine and Matamo. We did this previously with the Google api. So when we are finished this could dig much deeper into this.

that's really exciting news @alexgogan !

@alexgogan be good to keep us up to date and let us know if we can be of any help. We're keen on investigating this as well.

Also in general: It's obviously not the same but may be sometimes better than nothing to use Log Analytics to import old data if the logs still exist.

I've done some research into the feasibility of importing log data from Google Analytics into Matomo. I've come up with two possible solutions, one which probably isn't workable, and another whose feasibility won't become apparent until a PoC is built. Both solutions allow the visitor log to function w/ the imported data. If we just import reports, this won't be the case.

First, here are the limitations with Google Analytics API:

• Clients are limited to a total of 50,000 API resuests per day.
• There are only two APIs, one to get aggregated reports and one to get an individual user's activity.
  • The first API imposes a limit on the number of dimensions you can use, at most 7. So we can't aggregate by every possible dimension and determine the properties for indivdual actions (at least not w/ ease).
  • The second API only provides the activity for a single user, so it doesn't provide a lot of data, and the data will be out of order w/ the rest of the user data.
  • There is no way to get a list of user IDs (or client IDs which are like Matomo config IDs). You can see them in the UI, but there is explicitly no API (and no user id/client id dimension).

Here is the solution that probably isn't workable, but would be easier to understand:

1. For each day in the time range, use the userActivity.search API method (https://developers.google.com/analytics/devguides/reporting/core/v4/rest/v4/userActivity/search?authuser=2) to query actions for all users in a site.
2. Re-order them by action time.
3. Map them to Matomo tracker requests and track them like normal requests.

Pros

• It's simple to understand and straightforward to implement.

Cons

• There's no way to get a list of user IDs/client IDs through the API (AFAIK). Which makes this solution unworkable.
• The API only allows querying actions for a single user, which means the number of requests depends on the number of users. A site with 50,000 users will require 50,000 API requests per day, which means we'll hit the API limit immediately.
• Since we are getting actions for a user and not for a day, we'll have to incorporate different user's actions before tracking because we cannot track them out of order. This means we'll have to get a potentially large amount of actions, re-order them and then track them.

The other solution uses the reporting API to make multiple requests that are combined into a result we can use:

1. For each day in the time range, make multiple report queries, one for every possible pair of dimensions we're interested in. We query just for hits.
2. The report queries will return a list of values for the respective dimensions. We combine these to get the hits for every possible dimension value. The combination is done progressively and is explained below.
3. When we have the number of hits (ie, actions) for every dimension value combination, we can recreate actions and re-track them.

Explanation for combining reports to get a report for every dimension:

Ideally, we want to get the number of hits for every dimension value (eg, browser name, action time, referrer name, etc.). Since there's a limit of 7 dimensions, we can't do this. Instead we have to find some way to combine queries for less dimensions. To illustrate how we do this, let's limit the number of dimensions and dimension values. We'll say there are 3 dimensions total (DA, DB, DC), 1 dimension value for each (A, B, C), and GA only allows selecting 2 dimensions.

So w/ these limitations, we have to find out what the report for dimensions = [DA, DB, DC] when we can only query two at a time. We'll do this by first querying every possible combination of two dimensions:

(where N(...) is just a placeholder for the hits)

The thing to understand here is that the hits for dimensions = [DA, DB] is the cardinality of the subset of all actions where DA = A & DB = B. We want to find the hits for dimensions = [DA, DB, DC], which is the cardinality of the subset of all actions where DA = A, DB = B & DC = C.

This is the intersection of sets (DA + DB), (DA + DC), (DB + DC), which means we can calculate N(A, B, C) by getting the minimum value of the cardinality of all these sets, ie: |DA + DB + DC| = min(|DA + DB|, |DA + DC|, |DB + DC|).

The idea is to use this property and apply to all the dimensions we're interested in and all their queried values. This will give us essentially the properties for every action.

Pros

• It is very friendly to the GA API rate limit. The number of queries made will be N * (N - 1) * D where N is the number of dimensions we need & D is the number of days. It will take some time to hit the query limit, no matter what the scale of the site is.
• While computing every possible combination will still be memory intensive, it will likely be far less memory intensive than the previous solution on a high trafic site.
• It might actually work.

Cons

• It may be hard to understand and hard to code.
• It's possible I'm looking at the solution incorrectly, and a PoC that is time consuming to create will turn out to not actually work.

EDIT: One way to get user IDs/client IDs would be to issue requests like those that are made in the UI. I'm not sure how well that would work, and I'm guessing the page size will be smaller so it would take more requests. Also I am not sure if there are any limits to doing that, since that's outside of the API.

I would say raw data may not be as important? Of course be great to have it for segmentation, but maybe not needed eg if there's a problem with the 50K req/day limit?

@diosmosis thanks for the report. Sounds like importing RAW data is tricky and not easily do-able at this time.

So we probably want to import Aggregated reports only. And forget about RAW data.

-> As a next step, would be interesting to understand what it would take to build a Google Analytics Report data importer?

@mattab We would map matomo dimensions to GA dimensions, query for report data, then map GA values to Matomo values (ie, dimensions). It's pretty straightforward (though value mapping might be difficult). We could also allow users to supply segments to make available (we can add segments to the report fetching API method).

In V1 we would just want to keep it simple and import "All visits" segment (ie. not import any segment and segment data from GA)

Status update:

• created proof of concept importer that successfully imports entities including site, goals & custom dimensions and successfully imports some reports including referrers reports, actions reports & custom dimension reports (though there are issues w/ each)
• created automated testing setup for the importer that uses GA

Next things todo:

• finish code to import the remaining reports and entities
• to our test GA site add missing data so we can test every part of the import
• manually test w/ a larger GA site
• fix any import accuracy issues

Hi Everyone! good news, we are almost done with the Google Analytics to Matomo Importer implementation. We'll start the beta release in a few days. Get ready for testing to import your Google Analytics data in Matomo 😎

@mattab . Thanks for this great work, Please I have migrated from 3.11 to 3.12 but I don't see the google analytics import section.
Thank you,

Hi @kimoudev the plugin hasn't been released yet, there are still a couple import accuracy bugs to work through before it's ready for a public beta. Should be soon though.

Hi @diosmosis . Ok, Thank you so much.

Hi Everyone,

The Google Analytics Importer for Matomo is now released and available to all! 🎉

• Get the plugin from: https://plugins.matomo.org/GoogleAnalyticsImporter (see also: How do I install a plugin in Matomo?)
• Read the user guide at: https://matomo.org/docs/google-analytics-importer/
• Report your feedback in the forum: https://forum.matomo.org/c/google-analytics-importer
• if you find a bug please create a new issue in the plugin's issue tracker at: https://github.com/matomo-org/plugin-GoogleAnalyticsImporter/issues

We're looking forward to hearing how it goes for you to import your GA data into Matomo.
Kuddos @diosmosis for building this tool 💪
We're happy to help you migrate your GA data into Matomo and stay in control! 🚀
Thanks to everyone who will help test this tool and report their feedback.

To use the Google Analytics Importer, you first need to upgrade to the latest Matomo 3.12.0 beta version. This can be done in 3 clicks, see instructions here

Hope you enjoy the Google Analytics importer tool. You can post your feedback in the issue tracker: https://github.com/matomo-org/plugin-GoogleAnalyticsImporter/issues

Will close it now as we consider it done. Exciting! 🎉