Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merge data from two sites into one #7650

Open
Kornexl opened this issue Apr 9, 2015 · 35 comments
Open

Merge data from two sites into one #7650

Kornexl opened this issue Apr 9, 2015 · 35 comments
Labels
Enhancement For new feature suggestions that enhance Matomo's capabilities or add a new report, new API etc.

Comments

@Kornexl
Copy link

Kornexl commented Apr 9, 2015

I would like to merge the data from two sites into one of these
(id 5 and id 15 to 15)

@daandres
Copy link

daandres commented May 1, 2015

I support this request.
My scenario is:
I have a Wordpress site and use wp-piwik. It sneds data automatically to piwik site with id 1.
After migrating my site to use only ssl, wp-piwik detects this as a new site (id=2).
I have seen that this behaviour is fixed in wp-piwik, but too late for me.

Therefore I want to merge the sites with id 1 and 2 in one single side with only one id.

Thanks

@Creat1veM1nd
Copy link

I do support this request also.

@mattab mattab added the Enhancement For new feature suggestions that enhance Matomo's capabilities or add a new report, new API etc. label Jul 14, 2015
@mattab mattab added this to the Mid term milestone Jul 14, 2015
@lioman
Copy link

lioman commented Jul 19, 2015

This would be useful. I accidently created a new website in Piwik and now the data of half a year is in the wrong website.

A simple SQL script to merge the data together would be usefull too.

@mnapoli mnapoli changed the title merge the data from two siteids to one of these Merge data from two sites into one Jul 19, 2015
@Breighton
Copy link

I also support this request. We went from a "beta" site to a "production" site but our marketing folks need data that goes back through the "beta" site's data, and the merging of the two sites would do exactly that for us.

@Undergrounder
Copy link

I am extending the SiteMigration plugin to allow doing this. I hope to get this feature this week done.

@benji1000
Copy link

Oh, that would be very useful!

I just ran into the situation where I moved my websites from one server to another, initiated the tracking on the new server, and made the mistake to migrate the data from the old server afterwards.

Although the two configurations were quite identical (same domains, same ID), migrating data of one of the two websites with the SiteMigration plugin created a third website on the new server, instead of merging with the corresponding ID.

Thank you Undergrounder!

@monty241
Copy link

Great change! SiteMigration plugin is quite slow with large websites and creates a new idsite for every date-from/date-to range being uploaded. This would allow multiple uploads to run in parallel.

@davidak
Copy link

davidak commented Jul 16, 2016

@Undergrounder is that now supported? How would i do it?

@Undergrounder
Copy link

No, sorry.

My old company switched to Google analytics and the implementation was't yet complete.

@mattab mattab modified the milestones: Long term, Mid term Dec 5, 2016
@christianhennen
Copy link

Are there any updates on this? We at @ZIMK would greatly appreciate such a feature.

@saschafoerster
Copy link

Just have the same problem, that the plugin wp-piwik created a new ID. So another +1 for this year!

@krrishh-org
Copy link

+1

@D37R4C7
Copy link

D37R4C7 commented Sep 18, 2018

+1

@Kornexl Kornexl closed this as completed Feb 28, 2019
@b3n4kh
Copy link

b3n4kh commented Mar 8, 2019

@Kornexl could you elaborate on why this feature is unwanted?

@Creat1veM1nd
Copy link

Would like to know the reason for closing also. Thanks.

@mattab
Copy link
Member

mattab commented Mar 18, 2019

Sorry it was closed by mistake. Let's re-open.

Btw we have released a new slightly related plugin which lets you migrate one website from one instance to another: http://plugins.matomo.org/Migration
But it doesn't work for "merging" two websites together. Maybe it can be useful in some cases.

@mattab mattab reopened this Mar 18, 2019
@prathers
Copy link

prathers commented Dec 1, 2019

This issue has become more important since the launch of the Google Analytics Importer plugin. If you choose to import older data from Google after you already has the same website enabled in Matomo, you end up with two set of data.

@Kornexl Kornexl closed this as completed Jan 21, 2020
@Kornexl Kornexl reopened this Jan 21, 2020
@MaghSamana
Copy link

Once you set up and use Google Analytics Importer Plugin, you end up with two sites. The one you have set up with Matomo (its tracker includes the website ID), and the other that's created from Google Analytics continued import (different website ID).

In case you continue to get analytics from both Google and Matomo, merging them would have to deal with the issue of double records - same visitors tracking from both. It needs some scripting for this. In addition some metrics differ, so how would these be handled?

Adding the past data from GA to Matomo's website seems a better option if you want to have a single website for tracking, and want to fill in the past gaps (e.g. if you've been using GA for years and switched to Matomo at some point). I think the migration plugin, mentioned earlier may be useful here.

@mattab
Copy link
Member

mattab commented Mar 4, 2020

@simardcasanova
Copy link

+1 for merging two IDs together

@xavez
Copy link

xavez commented Apr 10, 2020

Just wanted to weigh in here that I also have the same problem.

@davidak
Copy link

davidak commented Apr 24, 2020

@xavez @simardcasanova please only add 👍 to the initial issue! This way the maintainers can easily see that the issue affects many people. Writing a comment with "+1" don't add any useful information and everyone in this thread get's an e-mail about it. That wastes peoples time and it will take longer until issues get fixed.

@UttaraRC
Copy link

I would like to merge the data from two sites into one of these
(id 5 and id 15 to 15)

I have same issue. My both websites on same installation of matomo.
can anyone having solution for this

@jwest75674
Copy link

Can anyone comment as to if there is a manual method for copying historical data from one site to another? I am not averse to copying the historical and deleting the then redundant site if there is a way?

@tsteur
Copy link
Member

tsteur commented Dec 23, 2020

@jwest75674 sorry I'm not aware of any manual way so far.

@simardcasanova
Copy link

simardcasanova commented Jan 12, 2021

@xavez @simardcasanova please only add 👍 to the initial issue! This way the maintainers can easily see that the issue affects many people. Writing a comment with "+1" don't add any useful information and everyone in this thread get's an e-mail about it. That wastes peoples time and it will take longer until issues get fixed.

I'm sorry @davidak but I couldn't find any mention anywhere about not doing this. You cannot expect people to comply with rules that are never specified.

So speaking about "useful information", I would then suggest to update the README to explain more clearly how people are expected to +1 feature requests 🙂

@dl-lim
Copy link

dl-lim commented Jan 14, 2021

As @jwest75674 mentioned for a manual method for copying, I second this with perhaps a simpler suggestion for implementation while we iron out the many kinks of undertaking such an effort.

What if we try just overwriting overlapping data for the same dates first, since there could be some users (myself included) who only ran the two datasets in parallel for a short time. One simple way would be to set a cut-off date. Of course a huge CAVEAT in red text's gotta be shown before confirming the merging and losing partial data :P

This way, we can alpha test the feature before fully implementing it by handling overlaps, goals and custom dimensions. Also saves us time and effort from figuring out all the technical details while having something functional in the meantime.

@UttaraRC
Copy link

UttaraRC commented Jan 14, 2021 via email

@Amunak
Copy link

Amunak commented Jan 13, 2022

What if we try just overwriting overlapping data for the same dates first, since there could be some users (myself included) who only ran the two datasets in parallel for a short time. One simple way would be to set a cut-off date.

Another really simple and yet effective algorithm would be to look at which site has a higher amount of data (say, higher number of visits) and use that dataset for that day.

@n-vilmart
Copy link

n-vilmart commented Jan 13, 2022

Hi,
Still no solution for merging 2 site id ?
+1 then

@Zenithcoder
Copy link

Hi, is there a way to copy a date range of data into existing matomo website

@haet
Copy link

haet commented May 31, 2022

In our use case we have one site (ID 13) containing data from 2014 until now (imported from Analytics) and another site (ID 9) containing current Matomo tracking data starting in May 2022. I decided to move data until April from site 13 to site 9 to avoid overlapping data and simplify the transfer.

I have successfully merged my sites with this SQL statement:

USE matomodb;

delimiter //

drop procedure if exists merge_sites //
create procedure merge_sites(
    IN  id_old INT,
    IN  id_new INT,
    IN  month_start VARCHAR(7),
    IN  month_end VARCHAR(7)
)
begin
    DECLARE done BOOL default false;
    DECLARE tablename CHAR(255);

    DECLARE cur1 cursor for 
        SELECT TABLE_NAME 
        FROM INFORMATION_SCHEMA.TABLES 
        WHERE (
            TABLE_NAME >= CONCAT("matomo_archive_numeric_", REPLACE( month_start, '-', '_' ) )
            AND TABLE_NAME <= CONCAT("matomo_archive_numeric_", REPLACE( month_end, '-', '_' ) )
        ) OR (
            TABLE_NAME >= CONCAT("matomo_archive_blob_", REPLACE( month_start, '-', '_' ) )
            AND TABLE_NAME <= CONCAT("matomo_archive_blob_", REPLACE( month_end, '-', '_' ) )
        ) 
        ;
    DECLARE CONTINUE HANDLER FOR NOT FOUND SET done = TRUE;
    open cur1;

    myloop: loop
        fetch cur1 into tablename;
        if done then
            leave myloop;
        end if;
        -- SELECT tablename;
        
        set @sql = CONCAT('UPDATE ', tablename, ' SET idsite=', id_new, ' WHERE idsite=', id_old );
        -- SELECT @sql;
        prepare stmt from @sql;
        execute stmt;
        drop prepare stmt;
    
    end loop;

    close cur1;

    UPDATE matomo_site 
    SET ts_created=TIMESTAMP( CONCAT( month_start, '-01 00:00:00.000' ) )
    WHERE idsite = id_new ;
end //

delimiter ;

call merge_sites( 
    13, 		-- source site id
    9, 			-- target site id
    '2019-01',	-- start month
    '2022-04'	-- end month
);

Attention!! Backup your database before using this!

@fvettore
Copy link

After updating ids in archive_numeric and archive_blob don't forget to edit ts_created field in site table: stats older than the date reported in this field are not shown

@MatomoForumNotifications

This issue has been mentioned on Matomo forums. There might be relevant details there:

https://forum.matomo.org/t/restart-google-analytics-import-problem/46851/8

@MatomoForumNotifications

This issue has been mentioned on Matomo forums. There might be relevant details there:

https://forum.matomo.org/t/tutorial-for-merge-2-site-ids-data-working-method/50027/4

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Enhancement For new feature suggestions that enhance Matomo's capabilities or add a new report, new API etc.
Projects
None yet
Development

No branches or pull requests