Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Pull requests by community members: build artifacts should upload to our artifacts server #9141

Closed
tsteur opened this issue Nov 1, 2015 · 15 comments
Labels
c: Tests & QA For issues related to automated tests or making it easier to QA & test issues.
Milestone

Comments

@tsteur
Copy link
Member

tsteur commented Nov 1, 2015

I think when there is a PR like #9137 (not from piwik/piwik) the artifacts upload fails because of invalid authentication key see https://travis-ci.org/piwik/piwik/jobs/88599319#L1349 or maybe it is because of something else?

For reviews it would be good if we could see the uploaded artifacts. Do we actually need a key for piwik/piwik? I think we used to not have a key and we added it afterwards possibly to not allow to upload anything. Maybe there's a way we could allow uploading articfacts?

@mattab
Copy link
Member

mattab commented Nov 2, 2015

I believe it's due to the encryption of github tokens that only work when the PR is done by someone who is member of the Piwik ORG. @diosmosis would there be a way to have artifacts uploaded for all PRs including external contributors?

@diosmosis
Copy link
Member

Not for external contributors w/o making the artifacts password public (ie, unencrypted).

@tsteur
Copy link
Member Author

tsteur commented Nov 5, 2015

It would be helpful to have a solution for this. Maybe we can make the password public and instead add some security on the uploads server to make sure only expected things are uploaded (images etc)?
I just had a look at a PR with failing UI tests and had to run UI tests manually. Maybe there are other ideas?

@diosmosis
Copy link
Member

instead add some security on the uploads server to make sure only expected things are uploaded (images etc)?

Should already be in place. Issue is w/ people potentially uploading lots of images or DDoS-ing. If there is some way to verify the request comes from travis-ci, then we don't need the password.

@tsteur
Copy link
Member Author

tsteur commented Nov 5, 2015

Not sure if that's possible as possibly pretty much anything can be changed/faked if someone wants to

@diosmosis
Copy link
Member

Just thought of something: try removing the environment variables from .travis.yml and setting them here: https://travis-ci.org/piwik/piwik/settings . Maybe these will be used by PRs.

@tsteur
Copy link
Member Author

tsteur commented Nov 5, 2015

That's cool, haven't seen it before. I presume it's exactly for such cases maybe.

@diosmosis
Copy link
Member

Actually in that case should make sure it's only the artifacts password that's used there. The autoupdate token shouldn't be forwarded to other repos.

@mattab
Copy link
Member

mattab commented Nov 6, 2015

Adding to 2.15.1 as it would be awesome to get UI tests artifacts for third party contributed Pull requests 👍

@mattab mattab added this to the 2.15.1 milestone Nov 6, 2015
@mattab mattab added the c: Tests & QA For issues related to automated tests or making it easier to QA & test issues. label Nov 6, 2015
@tsteur
Copy link
Member Author

tsteur commented Nov 10, 2015

I will try it now

@tsteur
Copy link
Member Author

tsteur commented Nov 10, 2015

Actually, I just read this:

By default, the value of these new environment variables is hidden from the export line in the logs. This corresponds to the behavior of encrypted variables in your .travis.yml.

Similarly, we do not provide these values to untrusted builds, triggered by pull requests from another repository.

So it won't work as pull requests could read the secret token otherwise. It should work though if we made the token visible. In this case it would as well work to define it in .travis.yml in plain text. But then anyone could ddos again and we cannot verify request is made from travis

@mattab mattab removed this from the 2.15.1 milestone Dec 4, 2015
@mattab mattab added this to the 2.16.0 milestone Dec 23, 2015
@diosmosis
Copy link
Member

Got screenshots to upload for a PR (build: https://travis-ci.org/piwik/piwik/jobs/99020321, screens: http://builds-artifacts.piwik.org/piwik/piwik/master/17369/).

@diosmosis
Copy link
Member

Works for plugins as well: (build: https://travis-ci.org/piwik/plugin-TasksTimetable/builds/99023364, screens: http://builds-artifacts.piwik.org/piwik/plugin-TasksTimetable/master/72/).

See ui-tests-viewer for relevant changes.

Closing this issue.

@tsteur
Copy link
Member Author

tsteur commented Jan 10, 2016

👍 Awesome

@mattab mattab changed the title Upload artifacts for pull requests fails? Pull requests by community members: build artifacts should upload to our artifacts server Jan 29, 2016
@tsteur
Copy link
Member Author

tsteur commented Feb 3, 2016

It doesn't seem to work for System tests. I've created a new issue: #9685

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
c: Tests & QA For issues related to automated tests or making it easier to QA & test issues.
Projects
None yet
Development

No branches or pull requests

3 participants