@mattab opened this Issue on June 5th 2014 Member

Follows up #5273 where we migrated Piwik issues from Trac to Github issues. This is following up our previous migration from SVN to git.

The possibly only downside of moving to Github issues is that we lose the control over the tickets. This has many up-sides (github is faster, more reliable, less frustrating to manage) but we also lose traffic from Search Engine, and possibly Github could go down.

Goal: We wish to build a Mirror of all our issues content.

Note:

@mattab commented on June 10th 2014 Member

From Github, a bit more info:

Thanks for your feedback. Currently, as you noticed, the API allows you to make a backup of all the issues and comments in your repositories. The rate limits are in place to keep the API fast and reliable for everyone, which has always been a priority for as, as is making this data available to you.

At the moment, we don't offer a way to download all issues and comments as a ZIP archive. I'm happy to pass the suggestion to the team, but I can't promise an ETA for when this might be available.

In the meantime, using caching and conditional requests should allow you to re-fetch issues and comments for each archive cycle without draining your API quota:

https://developer.github.com/v3/#conditional-requests

@mattab commented on July 9th 2014 Member

In our latest blog post I wrote: At Piwik we care a lot about Data ownership. For this reason we need to have an up to date copy of all our tickets and comments out of github.com servers. Our next step will be to create and release as open source a tool to let anyone create a Mirror of their Github issues.

@tsteur commented on July 11th 2014 Member
@tsteur commented on July 11th 2014 Member

A first version is running on http://issues.piwik.org/ . A cronjob is setup to sync issues once per day (can take up to an hour or even more). We're getting an email to developer@piwik.org in case something goes wrong during the cronjob.

It's a bit "slow" meaning it takes about 400ms on the server side, sometimes up to 1second. Can you enable APC on this server? It takes not even 100ms on my server...

Need to see how stable the importer is in the next days and whether everything works...

@tsteur commented on July 18th 2014 Member

Seems to work and google is already indexing. Will close it for now. Future issues should be created in https://github.com/piwik/github-issues-mirror/issues

@mattab commented on August 3rd 2014 Member

Amazing Thomas! well done on this ticket.

This Issue was closed on July 18th 2014
Powered by GitHub Issue Mirror