Let users import Google Analytics data in Piwik #6095

The goal of this ticket is to create an easy to use tool that will let anyone import in Piwik their Google Analytics data.

Features expected in this tool

• Import one or several websites
• Import data for all periods, or only import for a custom date range
• For each website, we import the report data: Top pages, top events, top browsers, etc. (for the 50 reports available in Piwik or so)
• Note: will not import the raw log visitor data since it is not available via Google Analytics API
• Create a simple user interface, which uses a console command

  Tasks

• Proposed name: Google Analytics Importer
• Create new project at piwik/importer-google-analytics
• This is a Piwik plugin published on the Marketplace
• Let user connect via oAuth to the Google APIs, fetch automatically the profiles, websites available.
• Run import via the UI or directly let user execute the console command
• Test the importer on a few websites,
  • including a website with medium traffic to check that we work around Google API limits
• Release the tool to public:
  • update this blog post, this FAQ
  • write a new blog post about the tool

This issue requires #6094 (Create an API to let users import historical report data in Piwik)

Will importing history from GA cause duplication in Piwik? e.g. For a site which has been using GA for a year, but only 3 months in Piwik, we might want to import the previous 9 months to get 1 year of history.
But not the overlapping recent 3 months.
Do you plan on deduping or providing a data range filter?
Thanks

What's the overall status of this? It seems that there's no activity at all in both issues. Do you have this in your short term roadmap?

There is no update and while it is currently in the Short term it may not be done in short term due to the complexity of the task. If you need this and have a budget, please contact Piwik PRO who may be able to help!

This feature would be great!

This is great feature!

We are really looking forward to this feature in our company. We cannot ask our clients to use our central Piwik installation until this has been released, since they have been relying on Google Analytics for too long and wish to keep their historical data.

+1

+1

+1

+1

What about importing raw google analytics data? I have a copy of every hit saved, can these be imported into piwik?

+1

+1

+1

+1

+1

+1

+1

+1

I'm in! What task need to be done first? :)

Come on guys, we need this!!!

+1

+1

+1

Hi, can you tell me the status for this issue?

Not sure if anyone is working on that at all :disappointed:

+1

We won't be able to work on this, unless someone can sponsor us to work on it. Please contact us via https://piwik.org/development/ if you're able to sponsor the team!

Why do not we/you create a crowdfounding campaign? I think it would be of great success...

I'd certainly give a few dollars :)

+10 (+10$ at least in a crowdfunding)

Possible already-available solution:

Is there a tool that allows us to export from Google Analytics to Apache-log similar access.log files?

Then it would be easy to re-import to Piwik, with import_logs.py...

What do you think?

+1

@mattab how much money do you need to make the programming? I am very interested to get it.

Do you have some estimate price/hours to develop this feature?

+1
I am absolutely willing to pay for such a plugin/script.

+1 would pay.

+1 I would pay. @mattab can you create a kickstarter or similar?

Same for me. I'd be happy to contribute financially to help this feature / tool occur!

I have years of Google Analytics data logged before I learnt about Piwik, and I'm sure I'm not the only one. There'd be great value in implementing a tool like this if you guys can pull it off.

We are also willing to chip in for this feature.

Bueller?

+1

We could use patreon.
Take a look at this or this.

I would like to offer a development of such functionalities (that includes the API and the plugin to import GA data). A monthly financial contribution is crucial. Do you want me to set up a Patreon (or similar)?
In case of questions, please ask on sd@szymondukla.co.uk

We'd also like to work on this, but so far the problem is how would it technically work? @SzymonDukla do you have some thoughts on this? we haven't found a GA API that we can use to import all the data safely and accurately yet...

I have a project behind me now that involves Analytics data gathering through GA API. It's nearly (as close as we can really get) to raw data. I have a nice and flexible API wrapper written in PHP I'm going to use.
I've set up a Patreon: https://www.patreon.com/ga_for_piwik
I'll be answering all the questions on there.

Hi @SzymonDukla what's your idea of getting as close as you can get raw data? would you call the GA API with many dimensions to get raw data or so?

+1 i'd pay too. Please,please, help us to opt out of GA. I guess it would be a great functionnality of piwik!

I can see chats are going on on this subject, nice.

regards.
f.

Sorry for the delay, last two weeks have been crazy...
Getting back on this project, that's correct @mattab - I have a side project that involves getting and storing all the possible data from GA accounts and then generating monthly/weekly/yearly SEO reports based on that so I think I will have the same approach which is getting as many metrics in as many configurations (as some metrics are now working with other ones) exported into a JSON or similar file that then will be imported via Piwik plugin, processed and merged into Piwik itself.

@fabrice-regnier, thank you very much for your interest! I do that because I have the same problem - years worth of data on Google I have no way to export.

As mentioned, I've set up a Patreon (https://www.patreon.com/ga_for_piwik), let's move all further discussion on this project there for those interested :)

will have the same approach which is getting as many metrics in as many configurations (as some metrics are now working with other ones) exported into a JSON or similar file that then will be imported via Piwik plugin, processed and merged into Piwik itself.

Did you mean 'metrics' or rather 'dimensions'? Our previous version of the GA importer (since deprecated) used this trick, the problem was that back in the days there were not enough precise dimensions, so we couldn't get useful "Per visitor" information and import all visitors.
Could you show us an example of how you'll export the GA data to then import it in Piwik?

@SzymonDukla

As mentioned, I've set up a Patreon (https://www.patreon.com/ga_for_piwik), let's move all further discussion on this project there for those interested :)

I was really interested in this and went to back your patreon and then I noticed that you are asking for over $2000/month to create the plugin, and then expect most of us to purchase it on top of that. I have a great appreciation for the amount of work involved and needing to be able to divert your attention from other projects, but I really prefer a game to have either free to play, pay to upgrade or pay to play, free to upgrade models of pricing, and REALLY hate when they stick it to me on both ends, which is exactly what this feels like.

It looks like @SzymonDukla is no longer supporting this issue and the patreon is not running. Can someone update the issue, outline who is responsible for creating or outsourcing this piece of work?

Hey @mattab, any updates on this?

No update @suzigrishpul

we want to build this feature as open source, it's really something we would like to develop.

But so far we are not sure of the technical feasability of a GA->Matomo migration. If anyone is expert in GA API and has some idea how we could migrate data over to Matomo, we'd love to hear. We are pretty much keen to implement it in the next few months if it's technically possible.

(We likely will spend a few days investigating the feasability, later this year, if nobody comments here with points.)

@mattab cool, thanks! I wish I could help, will just watch for an update for now.

Really interested in this.