Add customizers for the module canvas feature

[ajax-semenov-y]

Hello. The Spring Modulith Documentation module offers a great feature that allows us to create documentation with essential insights about application modules:

Documenter(applicationModules)
    .writeModuleCanvases()

However, there is no option to customize (add or remove) information from this documentation. In our case, we do not use Spring annotations like @Repository or @Controller. Instead, we use the @Component annotation for all beans. Additionally, we do not use the default Spring events system - we use our proprietary pub/sub framework, so "Published events" and "Events listened to" are not applicable to us.

It would be good to have the ability to override the default toModuleCanvas template located in Documenter#toModuleCanvas, allowing us to customize it and include only the categories we are interested in.

To provide more context, we use Kafka and Nats (nats.io) for external communication and our internal framework for pub/sub communication between modules. I would like to generate documentation with the following sections: "Kafka listeners," "Nats listeners," and "Internal listeners." So to implement what we want to have, we need the ability to 1) filter application module classes that implement specific interfaces, 2) override default canvas template to add custom categories.

Please let me know if this approach is acceptable so I can propose my implementation vision.

[ajax-semenov-y]

It seems that CanvasOptions#groupers is what I was looking for. It solves a significant part of our problem. However, the ability to override detection for "categories" would still be beneficial. For instance, for the category "Published events," only classes matching this predicate are filtered:

private List<EventType> findPublishedEvents() {

    DescribedPredicate<JavaClass> isEvent = implement(JMoleculesTypes.DOMAIN_EVENT)
        .or(isAnnotatedWith(JMoleculesTypes.AT_DOMAIN_EVENT));

    return classes.that(isEvent).stream()
        .map(EventType::new).toList();
}

In our case, we use a custom framework, so it would be good to able to override this predicate. As a workaround, we could add the JMolecules annotation to each published event and create an ArchUnit test to ensure all published events have this annotation. However, a much simpler solution would be to override the predicate.

[odrotbohm]

You bring up a very interesting topic that's already worked on to some degree. Not directly in Spring Modulith, but jMolecules in particular. But let me start with what we can do with the former.

The canvas currently groups Spring beans separately to convey that the components listed are candidates for dependency injection. As you already discovered, the standard grouping facilities should work for those. For the events, we currently use the term “published” for events crossing module boundaries, not necessarily ones that are externalized into message brokers. To make things flexible, I think we could add API to register a CanvasCustomizer, that can produce additional rows, that you can freely populate. That would require a few additional methods on ApplicationModule to select types based on certain criteria, and likely also expose a dedicated API to generate Asciidoctor. This should be doable for the upcoming 1.4 release. Can you elaborate on what canvas building blocks you'd like to see? Is the Javadoc extraction of any interest?

For reference, in jMolecules, we're working on an extensible stereotype metamodel and the corresponding inspection APIs. Once that is in place, you'd be able to define custom stereotype annotations and interfaces, that we could create rows for automatically.

[ajax-semenov-y]

I apologize for the delayed response. I've reviewed the stereotype metamodel design document and believe it't a good idea that definitely enhance the flexibility and extensibility of module structure discovery. However, some cases remain unaddressed. Let me share some of my real-world use cases. We have a number of microservices that communicate using proto files via Kafka. Each service publishes a JAR artifact containing proto files and the Kafka topics it exposes as its API. Other services can add this artifact as a dependency and start using this API by subscribing to the provided topics and converting bytes from them to protobuf. Services are also broken down into modules using Spring Modulith. I want to create documentation with Spring Modulith that includes a section like "API Dependencies," listing the Kafka topics that each module depends on.

To achieve this, I need to:

1. Parse the classpath to build a map of dependencies from current services to other services. We use custom annotations, which I can filter to detect the API dependencies we have on other services. (For this part, I don't need any changes in Modulith)
2. Access ApplicationModule#allClasses (which is unfortunately private) to determine whether a module uses an API and the specific topics it uses. (For this part, I need changes in Modulith).
3. Afterward, add to the canvas a building block that looks like this. (It seems that I also need to make changes in Modulith for this part.)

API dependencies

service-a

• topic-1
• topic-2

service-b

• topic-6
• topic-7

[ajax-semenov-y]

Is the Javadoc extraction of any interest?

It would also be great. I believe there are many use cases for this. For example, we currently have custom code that uses the ApplicationModule class and based on it, we calculate the module's root directory and have custom validator interfaces that allow us to ensure the module contains a README.md file with a structure that adheres to our conventions. It would be helpful to be able to parse the "Overview" section from the README and integrate it into the modulith-generated documentation.

Additionally, we check that our module includes a catalog-info.yaml file, which is required by https://backstage.io/, (a tool we use in our organization). By parsing this file and the classpath, we can verify that the catalog-info contains the correct "dependsOn" entries (https://backstage.io/docs/features/software-catalog/descriptor-format/#specdependson-optional), among other things. This could also be a significant improvement to modulith.

[ajax-semenov-y]

Is the Javadoc extraction of any interest?

Within the context of the stereotype metamodel, it would be good to parse the Javadoc of annotated classes and incorporate this documentation into the generated documentation. This approach would allow us to include not only class names but also documentation in the "Exposed API" section.I believe this strategy could turn the generated documentation into a single source for all module-related information that developers need to know before delving into the module.

It might be worthwhile to make this feature configurable, allowing you to enable or disable Javadoc parsing for certain classes.

[ajax-semenov-y]

@odrotbohm what are your thoughts on this?

[ajax-semenov-y]

@odrotbohm hi. I'm continuing to implement modulith into my services and I have a new idea. Currently, the structure of the aggregated documentation (all-docs.adoc) is hardcoded, which prevents me from adding additional information to my modules. It would be nice to define a template for module documentation to make it extensible.

For example, include a service resources file like module-template.adoc with content such as:

== {{module.displayName}}
plantuml::module-{{module.id}}.puml[,,format=svg]
include::module-{{module.id}}.adoc[]

This way, I can modify the template and add my custom adoc files to the aggregated documentation.

[odrotbohm]

I don't think we want to extend on the customization of all-docs.adoc. It's merely considered a helpful default aggregator to easily display the generated partials. That said, real-world applications include the individual snippets into Asciidoctor documents that contain prose (see an example here).

[ajax-semenov-y]

Thank you. I also took this approach by generating the aggregated document myself. It's good that PlantUML and canvases are saved as separate adoc files, so they can be easily included in other documents. Thank you for the clarification.

[ajax-semenov-y]

@odrotbohm, we aim to integrate these documents into our internal developer portal (we use Backstage). However, Backstage does not support AsciiDoc, only Markdown. Therefore, we have decided to generate documentation in Markdown. Could you please make Documenter#CanvasOptions#groupBeans, Documenter#CanvasOptions#Groupings, public so that we can reuse this logic to generate the same content in Markdown format?

[odrotbohm]

Hm, ideally, I'd like to keep the public API of CanvasOptions focussed on the configuration aspect. Would you be interested in contributing the Markdown code generation? I guess we could have an alternative implementation to Asciidoctor and expose a CanvasOptions.format(…) method that would flip to that alternate implementation?