Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[RFE] Cross repo pass/fail reporting #529

Open
NickLaMuro opened this issue Nov 6, 2020 · 10 comments
Open

[RFE] Cross repo pass/fail reporting #529

NickLaMuro opened this issue Nov 6, 2020 · 10 comments

Comments

@NickLaMuro
Copy link
Member

NickLaMuro commented Nov 6, 2020

A comment from @lpichler

ManageIQ/manageiq#20776 (comment)

Tests are 🟢 ManageIQ/manageiq-cross_repo-tests#220

Got me thinking: "This would be excellent if the bot could do this for us!"

Approach 1

On new commits to a cross_repo build, we check on the status of a cross repo build and report back when it has completed (pass/fail), and delay said job if it hasn't completed.

This is probably the simplest solution, but also the biggest hack.

Approach 2

Add some form of web hook to the travis build to inform the associated PR of a build pass/failure. Not sure of the possibility of this, so this would require more research. At a minimum, we probably could be able to send a web request to the bot, or enable a direct status post from the cross_repo travis run on the success of the run.

@Fryguy
Copy link
Member

Fryguy commented Nov 6, 2020

Sounds kind of similar to #451 but specific to cross repo?

@NickLaMuro
Copy link
Member Author

NickLaMuro commented Nov 6, 2020

@Fryguy Bit different.

The idea for that one is that it is notifying gitter that there is a failure in one of the main branches, and takes a guess at maybe what PR caused it.

In this case, we are looking a particular cross_repo pull request ("run"), and reporting back to the PR that might have issued it. Similar code might be able to be shared, but the target we are watching and receiver of the notification are much more focused.


This is more for a reviewer of a particular PR knows that the cross repo tests that were run have completed as expected.

While mostly saving a quick jumping through a link, there is a scenario where this makes it much more useful:

Say, a given PR is open and has a cross-repo build associated with it, and the cross repo build happens and the bot pings on the calling PR that it passes. After some code review, the PR is rebased (or even just some extra fixes are added), but said changes would have actually broken the cross_repo build, a reviewer would at least be able to see that the confirmation of a working happened before the new changes were introduced.

A bit contrived, but I don't think far fetched either.

@Fryguy
Copy link
Member

Fryguy commented Nov 6, 2020

Oh ok...I was thinking more the tech needed to "monitor" would be the same as they both would watch Travis, though one is watching Travis runs for PRs, and the other for primary branches.

@agrare
Copy link
Member

agrare commented Nov 6, 2020

Maybe dreaming...but what about a Github Checks entry for the cross-repo test run instead of just a comment?

@NickLaMuro
Copy link
Member Author

@agrare kinda what I lumped into “webhook” above, though I am admittedly unsure of the capabilities of that system. Specifically, unsure if you can dynamically add checks to a PR, since I am not sure we want to make every PR require a cross repo test.

@agrare
Copy link
Member

agrare commented Nov 7, 2020

👍 to not require for every PR, I'm assuming miq_bot could post to some API to add a check after starting the cross-repo-test but you're right that is an assumption

@NickLaMuro
Copy link
Member Author

@agrare yeah, and it might be something where this is a feature that really might only need to be coded on the manageiq-cross_repo end, and there is nothing that the bot actually has to do to support this. But for now, figured I would at least open the idea up here to see what is possible.

Probably deserves a help wanted tag, though, since I don't plan on addressing it myself at he moment.

@miq-bot add_label help wanted

@Fryguy
Copy link
Member

Fryguy commented Nov 9, 2020

Now that we have our bot "public", we have a few more of these options for callbacks...we could even do webhooks instead of notifications for other things ;)

@kbrock
Copy link
Member

kbrock commented Nov 10, 2020

whoa - we have a travis status and a code climate status - are you suggesting we could have a cross repo status?

@NickLaMuro
Copy link
Member Author

@kbrock suggesting as a possibility, yes.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

6 participants