Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Discontinue git.piwik.org service #6676

Closed
mattab opened this issue Nov 15, 2014 · 3 comments
Closed

Discontinue git.piwik.org service #6676

mattab opened this issue Nov 15, 2014 · 3 comments
Labels
answered For when a question was asked and we referred to forum or answered it. c: Security For issues that make Matomo more secure. Please report issues through HackerOne and not in Github. c: Website matomo.org For issues related to our matomo.org website. Task Indicates an issue is neither a feature nor a bug and it's purely a "technical" change.
Milestone
Backlog (Help wan...

Comments

@mattab
Copy link
Member

mattab commented Nov 15, 2014

The goal of this issue is to develop automatic mirroring of our public git repositories in both organisations Piwik and PiwikPro into git.piwik.org subdomain. This will involve using latest version of gitlist and develop sync script to mirror repos from github orgs automatically when we create new ones.

Why is it important? our git mirror service at git.piwik.org got cracked and the server was seen sending spam requests to other servers. Someone got backdoor access. We guess that it's caused by Gitlist which was the only tool running there.

Proposed steps

  • (@mattab) get the current code and crontab scripts from the infected server
  • install gitlist latest on repos.piwik.org
  • create a crontab script that calls the github API to list the public repositories and then adds the repository not yet into the gitlist
    • when we add repositories lateer for new projects, plugins, etc. they will be automatically synched
  • we will want to mirror all repositories in organisations github.com/piwik and github.com/piwikpro
  • create crontab to sync the git mirror (maybe hourly?)
  • put the code and crontab documentation into a new project at piwik/github-gitlist or so
    • the little script that we develop could be reused happily by other projects that need the same thing as us: easily mirror of github repos
  • once repos.piwik.org works we can ask root admin to switch it to git.piwik.org
@mattab mattab added Task Indicates an issue is neither a feature nor a bug and it's purely a "technical" change. c: Website matomo.org For issues related to our matomo.org website. c: Security For issues that make Matomo more secure. Please report issues through HackerOne and not in Github. labels Nov 15, 2014
@mattab mattab added this to the Piwik 2.11.0 milestone Nov 15, 2014
@mattab
Copy link
Member Author

mattab commented Nov 15, 2014

Note that the service git.piwik.org also currently services the git-diff mailing list which is an important feature for the community! For example some core developers use the git diff emails (even to browse the subject names) to keep an eye on the overall progress of the Piwik platform.

Note about piwk-git mailing list:

I've pasted the useful scripts from the current git.piwik.org in here: https://github.com/mattab/gitlist-gitdiff - there is here the script to add a new repository to the gitlist, and a script to set the postreceive hook on the git repos. I'm still waiting for the crontab scripts and couple other info and will update this comment later.

Update: here is the current crontab -l

piwik-git@piwik:~$ crontab -l
*/4 * * * * /home/piwik-git/git/send-git-push-mails.sh >> /home/piwik-git/git-commit-notifier-log/cronjob.log 2>&1
0 0 * * 0 echo "" > /home/piwik-git/git-commit-notifier-log/cronjob.log
0 0 * * 0 echo "" > /home/piwik-git/git-commit-notifier-log/sendmail.log

@mattab
Copy link
Member Author

mattab commented Nov 15, 2014

This follows up #3721 and #5299 and #5380 where we start building useful re-usable toolkits to "Liberate Github data".

@mattab mattab modified the milestones: Piwik 2.11.0, Short term Dec 10, 2014
@mattab
Copy link
Member Author

mattab commented Mar 19, 2015

I've completely removed git.piwik.org service - while useful as a github mirror service, the software we used to run it was not ideal and created 200G of content in www/ directory. We have SSD disk so we can't afford 200G for this service.

I've also discontinued the piwik-git mailing list. Issue closed for now.

@mattab mattab closed this as completed Mar 19, 2015
@mattab mattab changed the title Upgrade Gitlist on git.piwik.org (this service got cracked) Discontinue git.piwik.org service Mar 19, 2015
@mattab mattab mentioned this issue May 22, 2015
Closed
@mattab mattab added the answered For when a question was asked and we referred to forum or answered it. label Oct 5, 2016
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
answered For when a question was asked and we referred to forum or answered it. c: Security For issues that make Matomo more secure. Please report issues through HackerOne and not in Github. c: Website matomo.org For issues related to our matomo.org website. Task Indicates an issue is neither a feature nor a bug and it's purely a "technical" change.
Projects
None yet
Milestone
Backlog (Help wanted)
Development

No branches or pull requests
1 participant
@mattab