Skip to content

Conversation

dscho
Copy link
Member

@dscho dscho commented Sep 2, 2025

After gitgitgadget/gitgitgadget#609 has been addressed, the architecture is noticeably different. Here is an attempt to visualize the new lay of the land:

gitgitgadget-architecture

@dscho dscho requested review from mjcheetham and webstech September 2, 2025 09:54
@dscho dscho self-assigned this Sep 2, 2025
@dscho dscho force-pushed the update-architecture branch from 9fef636 to bbad667 Compare September 3, 2025 15:27
@dscho dscho requested a review from rimrul September 3, 2025 16:38
Copy link
Contributor

@rimrul rimrul left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't love the positioning of the arrow labels overlapping multiple arrows, but there seems to be very little we can do about that.

@dscho dscho force-pushed the update-architecture branch from bbad667 to bc6fc86 Compare September 4, 2025 20:26
It is a good idea to stay up to date, anyway, but in this instance the
motivation is to get across the breaking changes in
https://github.com/gohugoio/hugo/releases/tag/v0.148.0 that affect how
layouts work (something that I am about to use when I introduce support
for Graphviz diagrams).

Signed-off-by: Johannes Schindelin <[email protected]>
GitGitGadget deploys an Azure Function to act on webhook events. The
documentation still points to the old location, but as of
gitgitgadget/gitgitgadget@dbe71cb7 (Remove the
code of the Azure Function, 2023-08-17), that location is no longer
valid, because the respective source code moved to
https://github.com/gitgitgadget/gitgitgadget-github-app.

Signed-off-by: Johannes Schindelin <[email protected]>
As of gitgitgadget/gitgitgadget#609,
GitGitGadget no longer runs on a single Azure Pipelines runner, but
instead on GitHub-hosted GitHub Actions runners.

Let's adjust the part that described the Azure Pipelines' names and
purposes, talking about the GitHub workflows and Actions instead.

Signed-off-by: Johannes Schindelin <[email protected]>
We no longer use Azure Pipelines (having traded their flexibility for
the increased transparency of GitHub workflows). In other words, the
exact steps can be inspected by everyone now, in
https://github.com/gitgitgadget-workflows/gitgitgadget-workflows.

Signed-off-by: Johannes Schindelin <[email protected]>
We now use GitHub workflows for synchronizing.

While in the area, also mentioned the current Git GUI maintainer (this
was somehow missed in a5db6b8 (Update git-gui references, 2025-08-21)).

Signed-off-by: Johannes Schindelin <[email protected]>
@dscho dscho force-pushed the update-architecture branch from a36b6ca to a2aad48 Compare September 5, 2025 07:36
@dscho
Copy link
Member Author

dscho commented Sep 5, 2025

I force-pushed some changes.

Range-diff
  • 9: bc6fc86 ! 1: 9fc0bb6 Upgrade Hugo

    @@ Metadata
      ## Commit message ##
         Upgrade Hugo
     
    +    It is a good idea to stay up to date, anyway, but in this instance the
    +    motivation is to get across the breaking changes in
    +    https://github.com/gohugoio/hugo/releases/tag/v0.148.0 that affect how
    +    layouts work (something that I am about to use when I introduce support
    +    for Graphviz diagrams).
    +
         Signed-off-by: Johannes Schindelin <[email protected]>
     
      ## hugo.yml ##
  • 1: 8e1e093 = 2: c5403ef architecture: fix the link to GitGitGadget's Azure Function definition

  • 2: 1aaef27 = 3: 33df62b architecture: adjust after the migration to GitHub Actions

  • 3: 7a94922 = 4: 458dcd4 architecture: stop describing Azure Pipelines' individual tasks

  • 4: a471c8e = 5: 51264ff architecture: adjust the Git GUI section

  • 5: 69b8750 = 6: 2182f7d Add support for rendering Graphviz diagrams

  • 6: af226bb ! 7: 5370634 architecture: preface it with some diagrams

    @@ content/architecture.md
     +
     +  GitGitGadget [label="GitGitGadget", fillcolor="#ffffff"];
     +
    -+  user -> pr_repo [taillabel=" opens PR"];
    ++  user -> pr_repo [label=" opens PR"];
     +
     +  pr_repo -> GitGitGadget [label="slash commands"];
     +
    @@ content/architecture.md
     +```graphviz
     +digraph GitGitGadgetFlow {
     +  layout="neato";
    -+  rankdir=TB;
    ++  rankdir="TB";
     +  splines=true;
     +  overlap=false;
    -+  fontname="Arial";
     +  fontsize=10;
     +  sep="+4.5";
     +
    -+  node [shape=box, style=filled];
    -+  edge [fontsize=7];
    ++  node [fontname="Arial", shape=box, style=filled];
    ++  edge [fontname="Arial", fontsize=7];
     +
     +  // Node styles
     +  user [label="user (Git contributor)", fillcolor="#7777ff"];
  • 7: 266bccc = 8: f84301a Optionally pre-render the Graphviz diagrams

  • 8: a2170ee = 9: a2aad48 deploy/pr: pre-render the Graphviz diagrams

  • 10: a36b6ca < -: ------- Update content/architecture.md

@dscho
Copy link
Member Author

dscho commented Sep 5, 2025

Updates can be adored here: https://dscho.github.io/gitgitgadget.github.io/architecture

@rimrul
Copy link
Contributor

rimrul commented Sep 5, 2025

I did some fiddling with overlapping edges, unlabled edges, label placement and readability. What do you think of

Improved layout of the complicated diagram

digraph GitGitGadgetFlow {
  layout="neato";
  rankdir="TB";
  splines=true;
  overlap=false;
  fontsize=10;
  sep="+4.5";
  node [fontname="Arial", shape=box, style=filled];
  edge [fontname="Arial", fontsize=7];
  // Node styles
  user [label="user (Git contributor)", fillcolor="#7777ff"];
  pr_repo [label="pr-repo", fillcolor="#eaa666", penwidth=2];
  upstream_repo [label="upstream-repo", fillcolor="#eaa666", penwidth=2];
  test_repo [label="test-repo", fillcolor="#eaa666", penwidth=2];
  GitGitGadget [label="GitGitGadget", fillcolor="#ffffff"];
  gitgitgadget_github_app_repo [label="gitgitgadget-github-app-repo", fillcolor="#fb7", penwidth=2];
  azure_function [label="azure-function", fillcolor="#ffffff"];
  gitgitgadget_workflows_repo [label="gitgitgadget-workflows-repo", fillcolor="#fb7", penwidth=2];
  gitgitgadget_repo [label="gitgitgadget-repo", fillcolor="#fb7", penwidth=2];
  mailing_list [label="mailing-list", fillcolor="#ffffff"];
  mailing_list_repo [label="mailing-list-repo", fillcolor="#fb7", penwidth=2];
  mailing_list_repo_mirror [label="mailing-list-repo-mirror", fillcolor="#fb7", penwidth=2];
  user -> pr_repo:nw [taillabel="     opens PR"];
  user -> upstream_repo [taillabel="opens PR"];
  user -> test_repo [taillabel=" opens PR"];
  upstream_repo -> pr_repo:se [label="syncs\nbranches"];
  pr_repo -> GitGitGadget [headlabel="slash commands"];
  upstream_repo -> GitGitGadget [label="slash commands\n(App)"];
  test_repo -> GitGitGadget [label="slash commands  \n(owner only)"];
  GitGitGadget -> mailing_list [label="sends patch series  "];
  GitGitGadget -> gitgitgadget_workflows_repo [label="runs in"];
  gitgitgadget_workflows_repo -> gitgitgadget_repo [label="uses Actions  "];
  pr_repo -> azure_function [label="webhook  "];
  upstream_repo -> azure_function [label="webhook  "];
  test_repo -> azure_function [label="webhook"];
  mailing_list_repo_mirror:ne -> azure_function:w [taillabel="       webhook"];
  gitgitgadget_github_app_repo -> azure_function [label="deploys to  "];
  azure_function -> GitGitGadget [headlabel="triggers            "];
  mailing_list:sw -> mailing_list_repo:se [label="public-inbox"];
  mailing_list_repo -> mailing_list_repo_mirror [label="syncs to  "];
  mailing_list_repo_mirror:se -> pr_repo [label="mirrors\nreplies"];
  mailing_list_repo_mirror -> upstream_repo  [label="mirrors replies       "];
  mailing_list_repo_mirror:n -> test_repo  [label="mirrors replies"];
}

@dscho
Copy link
Member Author

dscho commented Sep 5, 2025

What do you think

I love it! And I integrated your changes as b81468e. See it in action here: https://dscho.github.io/gitgitgadget.github.io/architecture

I tried to add some Mermaid diagrams, but they simply do not look good.
Graphviz offers much better control over the layout.

Unfortunately, unlike Mermaid (which is a JavaScript library), Graphviz
is a command-line program, and therefore not available in the browser.

Fortunately, some helpful people maintain a WebAssembly version of
Graphviz, which _is_ a JavaScript library:
https://github.com/mdaines/viz-js

Unfortunately, unlike Mermaid (which weighs ~25kB), the GraphViz
WebAssembly weighs 1.4MB. But that's not quite correct, as Mermaid then
loads another half megabyte, whereas `viz-js` does not load anything.

The `viz-global.js` file was downloaded via:

  npm pack @viz-js/[email protected]
  tar Oxvf viz-js-viz-3.17.0.tgz package/dist/viz-global.js \
    >static/js/viz-global.js

Signed-off-by: Johannes Schindelin <[email protected]>
There are some issues when adding SVGs inside <svg> HTML elements, e.g.
that positioning can be harder than with <img> ones, as pointed out at
git/git-scm.com#2052 (comment).

Let's use <img> elements instead, passing the SVG via data URLs.

There are more benefits to that: For example, most modern browsers allow
copying an image into the clipboard that is specified as an `<img>`
element, but not `<svg>` ones. Likewise, the "Open Image in New Tab"
functionality typically only works with the former but not with the
latter.

Signed-off-by: Johannes Schindelin <[email protected]>
No need to set a background color, is there?

Signed-off-by: Johannes Schindelin <[email protected]>
This can be done like so:

  ```graphviz {.engine="neato"}
  digraph diagram {
    A -> B
  }
  ```

Signed-off-by: Johannes Schindelin <[email protected]>
Since the WebAssembly version of Graphviz is not exactly light (1.4MB at
the time of writing), and since it is a waste to let everybody re-render
the SVG client-side over and over again even though the diagram's
definition hasn't changed, let's add a script that performs that
rendering "on the server side" (more precisely: during deployment).

This script takes no arguments and post-processes the output of Hugo in
`public/`.

For performance reasons, it requires the list of files that contain
Graphviz diagrams. With this here site, it might not matter much because
there are only a handful images here. However, I want to reuse the same
method on git-scm.com where there _are_ thousands of files that do not
contain any Graphviz diagrams, and therefore it is a necessary
optimization to process only the files that _do_ contain Graphviz
diagrams.

To get that list, a new layout and page are added that Hugo processes,
identifyng said list of files and writing the result to
`public/diagram-list.html`.

Since this script runs via Node.JS and therefore lacks the convenient
built-in HTML parser of browser-based JavaScript engines, a prerequisite
is now to run `npm install` so that the `node-html-parser` package is
available.

Signed-off-by: Johannes Schindelin <[email protected]>
Since we're generating `<img>` elements anyway, might just as well keep
that accessibility support.

Signed-off-by: Johannes Schindelin <[email protected]>
This avoids the 1.4MB download on the client-side, in favor of inlined,
small SVG elements.

Signed-off-by: Johannes Schindelin <[email protected]>
@dscho dscho force-pushed the update-architecture branch from b81468e to 24de833 Compare September 18, 2025 21:34
@dscho
Copy link
Member Author

dscho commented Sep 18, 2025

Range-diff
  • 1: 2182f7d ! 1: 17ef958 Add support for rendering Graphviz diagrams

    @@ layouts/_default/baseof.html
     +{{- if .Store.Get "hasGraphviz" }}
     +  <script src="{{ relURL "js/viz-global.js" }}"> </script>
     +  <script type="text/javascript">
    ++    let vizInstance
     +    window.addEventListener("DOMContentLoaded", async () => {
    -+      for (const x of [...document.querySelectorAll("pre[class=graphviz]")]) {
    ++      [...document.querySelectorAll("pre[class=graphviz]")].forEach(async (x) => {
    ++        if (!vizInstance) vizInstance = await Viz.instance()
     +        if (x.style.display === 'none') return
    -+        const engine = x.getAttribute('engine') || 'dot'
     +        const svg = (await Viz.instance()).renderSVGElement(x.innerText, { engine })
     +        x.parentNode.insertBefore(svg, x)
     +        x.style.display = 'none'
    -+      }
    ++      })
     +    })
     +  </script>
     +{{- end }}
  • -: ------- > 2: 3e9e79d graphviz: use <img> elements instead of ones

  • -: ------- > 3: 6e89300 graphviz: make the diagrams' background transparent by default

  • -: ------- > 4: 37e69c9 graphviz: allow overriding the engine

  • 3: f84301a ! 5: 9a5e2a2 Optionally pre-render the Graphviz diagrams

    @@ Commit message
      ## content/diagram-list.html (new) ##
     @@
     +---
    -+layout: diagram-list
    ++outputs:
    ++  - json
     +---
     
    - ## layouts/diagram-list.html (new) ##
    + ## hugo.yml ##
    +@@ hugo.yml: markup:
    +   goldmark:
    +     renderer:
    +       unsafe: true
    ++mediaTypes:
    ++  application/json:
    ++    suffixes:
    ++    - json
    + module:
    +   mounts:
    +   - source: content
    +
    + ## layouts/_default/single.json.json (new) ##
     @@
    -+<pre>
    -+{{- range .Site.RegularPages -}}
    -+  {{- if in .RawContent "```graphviz" -}}
    -+    public{{ .Path }}.html
    -+  {{- end }}
    -+{{ end -}}
    -+</pre>
    ++{{- $diagram_list := where .Site.Pages "RawContent" "like" "```graphviz" -}}
    ++{{- $diagram_list | jsonify -}}
     
      ## package-lock.json (new) ##
     @@
    @@ script/graphviz-ssr.js (new)
     + * via `viz-js`.
     + */
     +;(async () => {
    -+  const opts = { format: "svg" }
    -+  for (const path of readFileSync("public/diagram-list.html", "utf-8")
    -+    .split("\n")
    -+    .filter((x) => x.startsWith("public/"))) {
    ++  for (const { Path: pathInPublic } of JSON.parse(readFileSync("public/diagram-list.json", "utf-8"))) {
    ++    const path = `public${pathInPublic}.html`
     +    const contents = readFileSync(path, "utf-8")
     +    const html = parse(contents)
     +    const vizImport = html.querySelector('script[src$="viz-global.js"]')
    @@ script/graphviz-ssr.js (new)
     +    vizImport.remove() // remove the import
     +
     +    for (const pre of html.querySelectorAll("pre.graphviz")) {
    -+      const svg = (await Viz.instance()).renderString(pre.textContent, opts)
    -+      pre.replaceWith(svg.substring(svg.indexOf("<svg")))
    ++      const engine = pre.getAttribute("engine") || "dot"
    ++      const svg = (await Viz.instance()).renderString(pre.textContent, {
    ++        format: "svg",
    ++        graphAttributes: {
    ++          bgcolor: "transparent",
    ++        },
    ++        engine,
    ++      })
    ++      const dataURL = `data:image/svg+xml;base64,${Buffer.from(svg).toString("base64")}`
    ++      pre.replaceWith(`<img src="${dataURL}" />`)
     +    }
     +    console.log(`Rewriting ${path}`)
     +    writeFileSync(`${path}`, html.toString())
  • -: ------- > 6: 12ac91b graphviz: support the alt attribute

  • 4: a2aad48 = 7: 5ceac84 deploy/pr: pre-render the Graphviz diagrams

  • 2: 5370634 ! 8: 3a064ae architecture: preface it with some diagrams

    @@ content/architecture.md
     +  fontsize=10;
     +  sep="+4.5";
     +
    -+  node [fontname="Arial", shape=box, style=filled];
    ++  node [fontname="Arial", shape=box, style="filled,rounded"];
     +  edge [fontname="Arial", fontsize=7];
     +
     +  // Node styles
    @@ content/architecture.md
     +  fontsize=10;
     +  sep="+4.5";
     +
    -+  node [fontname="Arial", shape=box, style=filled];
    ++  node [fontname="Arial", shape=box, style="filled,rounded"];
     +  edge [fontname="Arial", fontsize=7];
     +
     +  // Node styles
  • 5: b81468e = 9: 24de833 architecture: improve the diagram layout

@dscho
Copy link
Member Author

dscho commented Sep 18, 2025

@rimrul as before, I pushed to my fork so that it can be seen in action: https://dscho.github.io/gitgitgadget.github.io/architecture

dscho and others added 2 commits September 18, 2025 23:43
A picture says more than a thousand words, they say.

Note: I originally intended to add two Mermaid diagrams. It turned out,
though, that there is too little control over the layout using that
method, and I turned to a much more complex yet equally more satisfying
solution: Graphviz.

Signed-off-by: Johannes Schindelin <[email protected]>
I did some fiddling with overlapping edges, unlabled edges, label
placement and readability.

Signed-off-by: Matthias Aßhauer <[email protected]>
Signed-off-by: Johannes Schindelin <[email protected]>
@dscho dscho force-pushed the update-architecture branch from 24de833 to 9325a7c Compare September 18, 2025 21:43
@rimrul
Copy link
Contributor

rimrul commented Sep 19, 2025

LGTM

@dscho dscho merged commit 77eb329 into main Sep 19, 2025
1 check passed
@dscho dscho deleted the update-architecture branch September 19, 2025 11:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants