From 195eda2e0a336100910051d0cb08ac3ba6c1e810 Mon Sep 17 00:00:00 2001 From: airghy Date: Wed, 2 Oct 2024 23:07:46 +0200 Subject: [PATCH 1/2] added github-events to cdevents rfc Signed-off-by: airghy --- rfc/github-cdevents.md | 47 ++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 47 insertions(+) create mode 100644 rfc/github-cdevents.md diff --git a/rfc/github-cdevents.md b/rfc/github-cdevents.md new file mode 100644 index 0000000..ffcddbe --- /dev/null +++ b/rfc/github-cdevents.md @@ -0,0 +1,47 @@ +## Github-CDEvents Integration + +### Overview +The integration of CDEvents with Github is made by converting Github events into CDEvents. This effort uses Github Webhook generated events and translate them into CDEvents. + +Github utilizes two types of events: the github-event and the issue-event. These events are to be converted into [CDEvents-Source Code Control Events](https://github.com/cdevents/spec/blob/v0.3.0/source-code-version-control.md) using a custom github-event to CDEvent translator. + +### Github webhook +On github a webhook has access to repository, organization, Github Marketplace account and Github sponsor account or Github App where it is installed. In the first place we will focus on the webhook installed on a simgle repository. + +#### Create a repository webhook +We have a [public project](https://github.com/rjtch/nodejs-webhook-server) on my github-account that is a simple nodejs-server capable to receive webhook-event. +As webhook server we use [Hookdeck](https://hookdeck.com/webhooks/platforms/tutorial-github-webhooks). After everything i well configured we can start the webhook server together with our node-js-server. + +### Configure webhook +Everything concerning the webhook's configuration is explained in the Hookdeck quickstart tutorial. + ```config +[remote "github-events"] + Dashboard = https://console.hookdeck.com + Source: https://hkdk.events/75yvzpqic09vvw + Connections: github -> github_to_cli-github forwarding to /github-webhooks-endpoint + Secret: 1234ABCD +``` +for additional informations you can refer to the [Hookdeck](https://hookdeck.com/webhooks/platforms/tutorial-github-webhooks) configuration page and the [github-webhook](https://docs.github.com/en/webhooks/using-webhooks/creating-webhooks) page. + +### CDEvent github-webhook-tranlator + +A new REST endpoint `/github-events` will be implemented to receive all github-events and translate them to CDEvents. +The below list of [CDEvents-Source Code Control Events](https://github.com/cdevents/spec/blob/v0.3.0/source-code-version-control.md) will be mapped with corresponding Github event types to translate. + + +| CDEvent Type | Github Event Type | Comments | +|:---------------------------------|:----------------------------------------|:----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| +| dev.cdevents.repository.created | ForkEvent | A user forks a repository. | +| dev.cdevents.repository.modified | GollumEvent, MemberEvent, PublicEvent, WatchEvent | When a user is added to a repository or a private repo is made public or a wiki page is added to a repository, all this events modify the repository. | | +| dev.cdevents.repository.deleted | ref-updated | To delete repository in Gerrit Hide the Project State under Repository Options
Event JSON {"submitter":{"name":"Administrator","email":"admin@example.com","username":"admin"},"refUpdate":{"oldRev":"2d351d6d3bd1bec64f6b165a843c9dd18ac4d2cd","newRev":"5a2347177cdc707c6b42444c505b974d6499047d","refName":"refs/meta/config","project":"TestRepo3"},"type":"ref-updated","eventCreatedOn":1700221454}
| +| dev.cdevents.change.created | ref-updated |
Event JSON {"submitter":{"name":"Administrator","email":"admin@example.com","username":"admin"},"refUpdate":{"oldRev":"0000000000000000000000000000000000000000","newRev":"0c8c548ca93b44dd501c000ebb79ecc8f9aa4bd6","refName":"refs/heads/test_branch","project":"TestRepo"},"type":"ref-updated","eventCreatedOn":1700147670}
| +| dev.cdevents.change.modified | ref-updated |
Event JSON {"submitter":{"name":"Administrator","email":"admin@example.com","username":"admin"},"refUpdate":{"oldRev":"0000000000000000000000000000000000000000","newRev":"0c8c548ca93b44dd501c000ebb79ecc8f9aa4bd6","refName":"refs/heads/test_branch","project":"TestRepo"},"type":"ref-updated","eventCreatedOn":1700147670}
| +| dev.cdevents.change.updated | ref-updated |
Event JSON {"submitter":{"name":"Administrator","email":"admin@example.com","username":"admin"},"refUpdate":{"oldRev":"0000000000000000000000000000000000000000","newRev":"0c8c548ca93b44dd501c000ebb79ecc8f9aa4bd6","refName":"refs/heads/test_branch","project":"TestRepo"},"type":"ref-updated","eventCreatedOn":1700147670}
| +| dev.cdevents.branch.deleted | DeleteEvent | A Git branch or tag is deleted | | | +| dev.cdevents.branch.created | CreateEvent | A Git branch or tag is created. | +| dev.cdevents.branch.reviewed | CommitCommentEvent | A commit comment is created | +| dev.cdevents.change.merged | change-merged |
Event JSON {"submitter":{"name":"Administrator","email":"admin@example.com","username":"admin"},"newRev":"4c688da535ea3352536e3b320880aa353bd404b2","patchSet":{"number":2,"revision":"ba8c3584916c93b3e17a1ab63072b0ebd0d3ed84","parents":["0c8c548ca93b44dd501c000ebb79ecc8f9aa4bd6"],"ref":"refs/changes/01/1/2","uploader":{"name":"Administrator","email":"admin@example.com","username":"admin"},"createdOn":1700230164,"author":{"name":"Administrator","email":"admin@example.com","username":"admin"},"kind":"NO_CODE_CHANGE","sizeInsertions":9,"sizeDeletions":0},"change":{"project":"TestRepo","branch":"test_branch","id":"I55862204ef71f69bc88c79fe2259f7cb8365699a","number":1,"subject":"updates to the test branch 2 Change-Id: I55862204ef71f69bc88c79fe2259f7cb8365699a","owner":{"name":"Administrator","email":"admin@example.com","username":"admin"},"url":"http://959be7129610/c/TestRepo/+/1","commitMessage":"updates to the test branch 2\nChange-Id: I55862204ef71f69bc88c79fe2259f7cb8365699a\n","createdOn":1700229687,"status":"MERGED"},"project":{"name":"TestRepo"},"refName":"refs/heads/test_branch","changeKey":{"key":"I55862204ef71f69bc88c79fe2259f7cb8365699a"},"type":"change-merged","eventCreatedOn":1700231698}

Followed by ref-updated and eventCreatedOn the same time, that can be ignored
Event JSON {"submitter":{"name":"Administrator","email":"admin@example.com","username":"admin"},"refUpdate":{"oldRev":"ba876fbc1bbe1c3c1670163275bceca061eb98e8","newRev":"2ae17bf7464a6a4871d16d3d506df75476ea87e4","refName":"refs/changes/01/1/meta","project":"TestRepo"},"type":"ref-updated","eventCreatedOn":1700231698}
| +| dev.cdevents.branch.deleted | change-abandoned | A Git branch or tag is deleted. | +| dev.cdevents.branch.updated | patchset-created |
Event JSON {"uploader":{"name":"Administrator","email":"admin@example.com","username":"admin"},"patchSet":{"number":2,"revision":"66e097f59116e761f5473b66b351f7eba63aed50","parents":["b8112960b06810405b3e8ce6c8560ee93694700f"],"ref":"refs/changes/02/2/2","uploader":{"name":"Administrator","email":"admin@example.com","username":"admin"},"createdOn":1700479995,"author":{"name":"Administrator","email":"admin@example.com","username":"admin"},"kind":"REWORK","sizeInsertions":11,"sizeDeletions":0},"change":{"project":"TestRepo1","branch":"patch_create","id":"Ie10fd901201c53e9fe3b9766343060763ac81dc6","number":2,"subject":"update commit as per review","owner":{"name":"Administrator","email":"admin@example.com","username":"admin"},"url":"http://959be7129610/c/TestRepo1/+/2","commitMessage":"update commit as per review\n\nChange-Id: Ie10fd901201c53e9fe3b9766343060763ac81dc6\n","createdOn":1700478731,"status":"NEW"},"project":{"name":"TestRepo1"},"refName":"refs/heads/patch_create","changeKey":{"key":"Ie10fd901201c53e9fe3b9766343060763ac81dc6"},"type":"patchset-created","eventCreatedOn":1700479996}

Followed by ref-updated and eventCreatedOn the same time, that can be ignored
Event JSON {"submitter":{"name":"Administrator","email":"admin@example.com","username":"admin"},"refUpdate":{"oldRev":"97de9dd4a272dffe8cf881057108713d01e03930","newRev":"06932a8928e6e0beff7f9b65f04d9aa0f59b10d1","refName":"refs/changes/02/2/meta","project":"TestRepo1"},"type":"ref-updated","eventCreatedOn":1700479996}
| + +Once mapped the CDEvent will be created using CDEvents SDK(Java/Go) and send to the configured `Events Broker URL` From 464fd6a04b395ac00720c8cd568401fae5a02c60 Mon Sep 17 00:00:00 2001 From: airghy Date: Fri, 4 Oct 2024 16:26:18 +0200 Subject: [PATCH 2/2] added more specific webhook events Signed-off-by: airghy --- rfc/github-cdevents.md | 57 +++++++++++++++++++++++++----------------- 1 file changed, 34 insertions(+), 23 deletions(-) diff --git a/rfc/github-cdevents.md b/rfc/github-cdevents.md index ffcddbe..90d0441 100644 --- a/rfc/github-cdevents.md +++ b/rfc/github-cdevents.md @@ -3,17 +3,24 @@ ### Overview The integration of CDEvents with Github is made by converting Github events into CDEvents. This effort uses Github Webhook generated events and translate them into CDEvents. -Github utilizes two types of events: the github-event and the issue-event. These events are to be converted into [CDEvents-Source Code Control Events](https://github.com/cdevents/spec/blob/v0.3.0/source-code-version-control.md) using a custom github-event to CDEvent translator. +Github utilizes two types of events: the github-event and the issue-event. These events are to be converted into [CDEvents-Source Code Control Events](https://github.com/cdevents/spec/blob/v0.3.0/source-code-version-control.md) +using a custom github-event to CDEvent translator. ### Github webhook -On github a webhook has access to repository, organization, Github Marketplace account and Github sponsor account or Github App where it is installed. In the first place we will focus on the webhook installed on a simgle repository. +On github, there are different types of webhooks: organization, repository, marketplace, sponsors and app webhook. +- Organization webhooks can subscribe to events that happen in all repositories owned by the organization. They can also subscribe to events that happen at the organization level that are outside of any particular repository. +- Marketplace webhook is the one that subscribes to events relating to an app that you published in GitHub Marketplace. +- Sponsors webhook is the kind of webhook that subscribe to events related to GitHub Sponsors. +- App webhook is a more generic webhook that can be configured to receive specific events that occur in a repository or organization that the app has been granted access to. + +In this effort, we will focus on repository webhook because we want to be able to process events happening inside a repository. #### Create a repository webhook -We have a [public project](https://github.com/rjtch/nodejs-webhook-server) on my github-account that is a simple nodejs-server capable to receive webhook-event. -As webhook server we use [Hookdeck](https://hookdeck.com/webhooks/platforms/tutorial-github-webhooks). After everything i well configured we can start the webhook server together with our node-js-server. +We have a [public project](https://github.com/rjtch/nodejs-webhook-server) on our github-account that is a simple nodejs-server capable to receive webhook-event (But any other server can be used for this end). +As webhook server we use [Hookdeck](https://hookdeck.com/webhooks/platforms/tutorial-github-webhooks). After everything is well configured we can start the webhook server together with our node-js-server. ### Configure webhook -Everything concerning the webhook's configuration is explained in the Hookdeck quickstart tutorial. +Everything concerning the webhook's configuration is explained in the Hookdeck [quickstart](https://hookdeck.com/webhooks/platforms/tutorial-github-webhooks) tutorial. ```config [remote "github-events"] Dashboard = https://console.hookdeck.com @@ -21,27 +28,31 @@ Everything concerning the webhook's configuration is explained in the Hookdeck q Connections: github -> github_to_cli-github forwarding to /github-webhooks-endpoint Secret: 1234ABCD ``` -for additional informations you can refer to the [Hookdeck](https://hookdeck.com/webhooks/platforms/tutorial-github-webhooks) configuration page and the [github-webhook](https://docs.github.com/en/webhooks/using-webhooks/creating-webhooks) page. +for additional information you can refer to the [Hookdeck](https://hookdeck.com/webhooks/platforms/tutorial-github-webhooks) configuration page and the +[github-webhook](https://docs.github.com/en/webhooks/using-webhooks/creating-webhooks) page. ### CDEvent github-webhook-tranlator A new REST endpoint `/github-events` will be implemented to receive all github-events and translate them to CDEvents. -The below list of [CDEvents-Source Code Control Events](https://github.com/cdevents/spec/blob/v0.3.0/source-code-version-control.md) will be mapped with corresponding Github event types to translate. - - -| CDEvent Type | Github Event Type | Comments | -|:---------------------------------|:----------------------------------------|:----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| -| dev.cdevents.repository.created | ForkEvent | A user forks a repository. | -| dev.cdevents.repository.modified | GollumEvent, MemberEvent, PublicEvent, WatchEvent | When a user is added to a repository or a private repo is made public or a wiki page is added to a repository, all this events modify the repository. | | -| dev.cdevents.repository.deleted | ref-updated | To delete repository in Gerrit Hide the Project State under Repository Options
Event JSON {"submitter":{"name":"Administrator","email":"admin@example.com","username":"admin"},"refUpdate":{"oldRev":"2d351d6d3bd1bec64f6b165a843c9dd18ac4d2cd","newRev":"5a2347177cdc707c6b42444c505b974d6499047d","refName":"refs/meta/config","project":"TestRepo3"},"type":"ref-updated","eventCreatedOn":1700221454}
| -| dev.cdevents.change.created | ref-updated |
Event JSON {"submitter":{"name":"Administrator","email":"admin@example.com","username":"admin"},"refUpdate":{"oldRev":"0000000000000000000000000000000000000000","newRev":"0c8c548ca93b44dd501c000ebb79ecc8f9aa4bd6","refName":"refs/heads/test_branch","project":"TestRepo"},"type":"ref-updated","eventCreatedOn":1700147670}
| -| dev.cdevents.change.modified | ref-updated |
Event JSON {"submitter":{"name":"Administrator","email":"admin@example.com","username":"admin"},"refUpdate":{"oldRev":"0000000000000000000000000000000000000000","newRev":"0c8c548ca93b44dd501c000ebb79ecc8f9aa4bd6","refName":"refs/heads/test_branch","project":"TestRepo"},"type":"ref-updated","eventCreatedOn":1700147670}
| -| dev.cdevents.change.updated | ref-updated |
Event JSON {"submitter":{"name":"Administrator","email":"admin@example.com","username":"admin"},"refUpdate":{"oldRev":"0000000000000000000000000000000000000000","newRev":"0c8c548ca93b44dd501c000ebb79ecc8f9aa4bd6","refName":"refs/heads/test_branch","project":"TestRepo"},"type":"ref-updated","eventCreatedOn":1700147670}
| -| dev.cdevents.branch.deleted | DeleteEvent | A Git branch or tag is deleted | | | -| dev.cdevents.branch.created | CreateEvent | A Git branch or tag is created. | -| dev.cdevents.branch.reviewed | CommitCommentEvent | A commit comment is created | -| dev.cdevents.change.merged | change-merged |
Event JSON {"submitter":{"name":"Administrator","email":"admin@example.com","username":"admin"},"newRev":"4c688da535ea3352536e3b320880aa353bd404b2","patchSet":{"number":2,"revision":"ba8c3584916c93b3e17a1ab63072b0ebd0d3ed84","parents":["0c8c548ca93b44dd501c000ebb79ecc8f9aa4bd6"],"ref":"refs/changes/01/1/2","uploader":{"name":"Administrator","email":"admin@example.com","username":"admin"},"createdOn":1700230164,"author":{"name":"Administrator","email":"admin@example.com","username":"admin"},"kind":"NO_CODE_CHANGE","sizeInsertions":9,"sizeDeletions":0},"change":{"project":"TestRepo","branch":"test_branch","id":"I55862204ef71f69bc88c79fe2259f7cb8365699a","number":1,"subject":"updates to the test branch 2 Change-Id: I55862204ef71f69bc88c79fe2259f7cb8365699a","owner":{"name":"Administrator","email":"admin@example.com","username":"admin"},"url":"http://959be7129610/c/TestRepo/+/1","commitMessage":"updates to the test branch 2\nChange-Id: I55862204ef71f69bc88c79fe2259f7cb8365699a\n","createdOn":1700229687,"status":"MERGED"},"project":{"name":"TestRepo"},"refName":"refs/heads/test_branch","changeKey":{"key":"I55862204ef71f69bc88c79fe2259f7cb8365699a"},"type":"change-merged","eventCreatedOn":1700231698}

Followed by ref-updated and eventCreatedOn the same time, that can be ignored
Event JSON {"submitter":{"name":"Administrator","email":"admin@example.com","username":"admin"},"refUpdate":{"oldRev":"ba876fbc1bbe1c3c1670163275bceca061eb98e8","newRev":"2ae17bf7464a6a4871d16d3d506df75476ea87e4","refName":"refs/changes/01/1/meta","project":"TestRepo"},"type":"ref-updated","eventCreatedOn":1700231698}
| -| dev.cdevents.branch.deleted | change-abandoned | A Git branch or tag is deleted. | -| dev.cdevents.branch.updated | patchset-created |
Event JSON {"uploader":{"name":"Administrator","email":"admin@example.com","username":"admin"},"patchSet":{"number":2,"revision":"66e097f59116e761f5473b66b351f7eba63aed50","parents":["b8112960b06810405b3e8ce6c8560ee93694700f"],"ref":"refs/changes/02/2/2","uploader":{"name":"Administrator","email":"admin@example.com","username":"admin"},"createdOn":1700479995,"author":{"name":"Administrator","email":"admin@example.com","username":"admin"},"kind":"REWORK","sizeInsertions":11,"sizeDeletions":0},"change":{"project":"TestRepo1","branch":"patch_create","id":"Ie10fd901201c53e9fe3b9766343060763ac81dc6","number":2,"subject":"update commit as per review","owner":{"name":"Administrator","email":"admin@example.com","username":"admin"},"url":"http://959be7129610/c/TestRepo1/+/2","commitMessage":"update commit as per review\n\nChange-Id: Ie10fd901201c53e9fe3b9766343060763ac81dc6\n","createdOn":1700478731,"status":"NEW"},"project":{"name":"TestRepo1"},"refName":"refs/heads/patch_create","changeKey":{"key":"Ie10fd901201c53e9fe3b9766343060763ac81dc6"},"type":"patchset-created","eventCreatedOn":1700479996}

Followed by ref-updated and eventCreatedOn the same time, that can be ignored
Event JSON {"submitter":{"name":"Administrator","email":"admin@example.com","username":"admin"},"refUpdate":{"oldRev":"97de9dd4a272dffe8cf881057108713d01e03930","newRev":"06932a8928e6e0beff7f9b65f04d9aa0f59b10d1","refName":"refs/changes/02/2/meta","project":"TestRepo1"},"type":"ref-updated","eventCreatedOn":1700479996}
| +The below list of [CDEvents-Source Code Control Events](https://github.com/cdevents/spec/blob/v0.3.0/source-code-version-control.md) will be mapped with corresponding Github-Webhook event types to translate. +Since this effort is to process [webhook events](https://docs.github.com/en/webhooks/webhook-events-and-payloads), we consider processing the webhook-events +related to branch, repository, deployment and artifact. The [CDEvent-Spec](https://github.com/cdevents/spec/blob/v0.3.0/spec.md) does not have events for tags or release yet. + + +| CDEvent Type | Webhook/Github Event Type | Comments | +|:---------------------------------|:----------------------------|:------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| +| dev.cdevents.repository.forked | fork | A user forks a repository. (should be defined in the spec) | +| dev.cdevents.repository.imported | repository_import | when a new repository is imported to Github. (should be defined in the spec) | +| dev.cdevents.repository.* | repository | This event occurs when there is activity relating to repositories.
Actions {"Action type":{"created", "modified", "deleted", "edited", "privatized", "publicized", "renamed", "transfered", "unarchived", "archived"}
| | +| dev.cdevents.repository.created | push | A Git repository has been created from a template. | +| dev.cdevents.branch.deleted | delete | A Git branch or tag is deleted | | | +| dev.cdevents.branch.created | create | A Git branch or tag is created. | +| dev.cdevents.branch.deleted | push | A Git branch has been deleted. But to subscribe to only branch and tag deletions, use the delete webhook event. | +| dev.cdevents.change.updated | push | A commit/ commit tag is pushed. | +| dev.cdevents.change.* | pull_request | This event occurs when there is activity on a pull request.
Actions {"Action type":{"assigned", "auto_merge_disabled", "auto_merge_enabled", "closed", "converted_to_draft", "demilestoned", "dequeued", "edited", "enqueued",
"archived", "labeled", "locked", "milestoned", "opened", "ready_for_review","reopened", "review_request_removed", "review_requested", "synchronized", "unassigned", "unlabeled", "unlocked"}
| +| dev.cdevents.change.* | pull_request_review_comment | This event occurs when there is activity on a pull request review comment.
Actions {"Action type":{"created", "reviewed", "commented"} | +| dev.cdevents.change.* | pull_request_review | This event occurs when there is activity relating to a pull request review.
Actions {"Action type":{"edited", "dismissed", "submitted"}
| +| dev.cdevents.change.* | pull_request_review_thread | This event occurs when there is activity relating to a comment thread on a pull request.
Actions {"Action type":{"resolved", "unresolved"}
| +| dev.cdevents.artifact.* | package/registry_package | This event occurs when there is activity relating to GitHub Packages/registry.
Actions {"Action type":{"published", "updated"}
| Once mapped the CDEvent will be created using CDEvents SDK(Java/Go) and send to the configured `Events Broker URL`