Boundary for docs

[LostKobrakai]

I'm wondering if boundary could generate ex_doc module groups per boundary and maybe documentation metadata for if a module is exported or not. The latter could be used by ex_doc if available (and commonly used).

[sasa1977]

I really like both ideas. I'd definitely like to see boundary integrating with the wider ecosystem, and enriching the generated docs is a great first step! However I'm not completely sure what is currently possible.

Module groups have to be declared in mix.exs, which is evaluated before the code is compiled and boundary data is gathered, so I don't think we can do anything here ATM. However, I'm not super familiar with ins and outs of ex_doc, so maybe I'm missing something. Do you see some way how this could be done today? If not, I guess we should reach out to ex_doc team and discuss possible extensions to ex_doc interface first.

Documentation metadata might be doable today, but this requires some experimenting. Essentially, we should be able to change the @moduledoc tag (if it exists), by adding some metadata. Perhaps it's as easy as adding another @moduledoc tag using before_compile, or otherwise we could change the semantics of the @ directive (which IIRC can be done, and I think that norm does something like that).

However, ATM my capacity for working on this is limited, so someone else should do the initial research. This doesn't even need to be done with boundary. Basically, we need a basic mix project with the module Foo which, when used by the module Bar adds custom metadata to Bar's module doc. This should be a basic proof of concept, so no need for any checks, error handling, options, and whatnot. Once we figure out the technical approach, we can discuss the exact interface and the proper implementation.

If you're willing to work on this, feel free to assign yourself to the issue, and let me know if you need any help. Note that I'm currently on vacation, so I can't respond immediately, but I'll try to address any question as soon as I can :-)

[LostKobrakai]

Module groups have to be declared in mix.exs, which is evaluated before the code is compiled and boundary data is gathered, so I don't think we can do anything here ATM.

I‘m not sure this is too much a problem. Boundary could generate/update something like a boundary.exs (e.g. as a mix task), which can be evaled by mix.exs on subsequent ˋmix docsˋ calls.

[sasa1977]

Boundary could generate/update something like a boundary.exs (e.g. as a mix task), which can be evaled by mix.exs on subsequent ˋmix docsˋ calls.

This is an interesting idea. One drawback is that a user has to remember to manually invoke the boundary task before docs. This might be tackled with aliased task, but I'm not completely sure it would work consistently, because boundary.exs would have to be evaled after the code compilation but before the docs task reads the info. I also wonder how would it work with hex.publish which IIRC implicitly invokes the docs task.

To be clear, I'm not against this approach, but someone needs to make a proof of concept to check how is it behaving when we invoke mix docs which will lead to code being recompiled and boundary.exs regenerated. Again, such experiment can be done on a small independent mix project.

[LostKobrakai]

Before compile does work without problems to add metadata to the current module doc:

https://gist.github.com/LostKobrakai/19c326cf1fec6273ee64152bb640d335

[LostKobrakai]

In regards to the mix task for module groups: hex does document aliasing as the way to do more than the default mix docs (even for using other generators then ex_doc) and I just tried out a small testcase, where I compiled the app, then manually changed an external docs.exs and it was picked up by the following mix docs without issue.

[sasa1977]

Before compile does work without problems to add metadata to the current module doc:

This looks cool, thanks for trying it out! I think we can go ahead with specifying how it would look like from the user's perspective. Are you willing to do this? It doesn't have to be anything formal, just a brief description in this issue.

I just tried out a small testcase, where I compiled the app, then manually changed an external docs.exs and it was picked up by the following mix docs without issue

The issue I worry about is the following:

1. Invoke mix docs.
2. Change boundary settings in some module. Don't invoke mix compile.
3. Invoke mix docs.

My hunch is that in step 3, mix might eagerly load the older version of boundary.exs before the code is recompiled.

[LostKobrakai]

In the docs I'd imagine something like the following:

---

Boundary

This module is part of the boundary Some.Prefix.*.

The root module for the boundary is: SomeModuleName | "This module"
The functions for this module are exported for other boundaries to be used: true | false
Modules in the current boundary: 14 (show all)

Exports:

• …

Dependencies: [

• …

---

Export and Dependency (and maybe current boundary module) listings might be enough when shown on the root module.

[sasa1977]

Sorry for the late reply, my vacation got a bit more active :-)

I don't think we need to define the exact shape of the documentation upfront. This can be done as we go along, and will probably be tweaked over time.

When I said

we can go ahead with specifying how it would look like from the user's perspective

I was mostly concerned with how the client developers (those using Boundary, such as lib & project authors) will enable this option (or these options). Do they need to configure something in mix.exs?

Re module groups, I don't think that my concern stated in this comment is answered.

That aside, the documentation meta looks good, but I'm not sure if we should add it by default. What's your opinion on this?

[LostKobrakai]

So the documentation metadata I feel good including by default. Unless it's MB of additional filesize, which e.g. for nerves people can be problematic. In such a case I guess a global config to disable it would be enough to handle that concern.

For the module groups boundary would need a mix task to generate the file and people would need to integrate the aliases for mix docs based on documentation within their own mix.exs files. Not much we can do automatically there.

Re module groups, I don't think that my concern stated in this comment is answered.

A mix task can ensure compile is triggered, so boundary would always generate a most recent boundary.exs before the real mix docs does include it in the documentation.

[sasa1977]

So the documentation metadata I feel good including by default.

The thing I'm worried about here is that it might add an unwanted noise to the docs. If I understand correctly, if we add metadata, they will be always shown in docs generated by ex_doc, right? If that is the case, then I'd definitely like to make it an opt-in, which would probably mean we need a :boundary section in mix.exs project config.

[LostKobrakai]

I feel the other way round is actually better. Always include the metadata (unless there are valid concerns around filesize) and leave the option of enabling/disabling output to the consumer of the metadata (a.k.a. ex_docs). This allows the metadata not only to be used by ex_docs, but potentially also other tooling, where it's not a global choice if those can be enabled or disabled.

[sasa1977]

I'm not worried about the file size but about automatically adding unwanted noise in docs. Suppose I'm writing a lib and I'm using boundary internally. I might not want this meta to be included in the published hex docs.

but potentially also other tooling, where it's not a global choice if those can be enabled or disabled

What other tooling? We're discussing docs here, nothing else. If you want to build other kind of tooling on top of boundary, the library already exposes some functions for retrieving boundary definitions (example), and we can of course expand this as needed.

[LostKobrakai]

I can see your point, but there's not just ex_doc consuming the documentation stored in beam files. Afaik elixir-ls uses data in there, iex's h module, …. Those might not yet use boundary metadata, but might at some point. That's why I feel the decision to show or not show certain data should be on the consumer side not (only) on the side of boundary.

[sasa1977]

Yeah, I consider these tools as different views of the docs. My point remains the same: the fact that I'm using boundary is by default an internal detail until I decide to expose it. I'm definitely open to changing my mind in the future, but I'd prefer to start conservative, rather than unconditionally adding some extra noise to every doc of every client project :-)

In other words, I suggest we start with adding an opt-in support for docs enrichment, using something like

# mix.exs
def project do
  [
    boundary: [docs_meta: true],
    # ...
  ]
end

Assuming we can agree on this, I think that the work on meta can be started. Regarding module groups, I'm still somewhat uncertain about the issue I mentioned. I'll try to find the time to prove or disprove this issue myself. Assuming that there's no problem, I think we should first define a sketch of the usage proposal (what should the client developer do in mix.exs to make it work).

But anyway, I think we can tackle these two things separately, and meta seems like a simpler of the two things, so let's start with that.

There's one other thing worth mentioning. Some extensive work has been done for the upcoming version in this branch. IIRC, this work is basically complete and ready to be merged and released, but I need to double check it, b/c I haven't been working on this in the past few months. In any case, we should do the future work on top of that branch, since IIRC there have been some changes in the internals.

[LostKobrakai]

I created two PRs for the two parts of the issue: #21 #20

[sasa1977]

Thanks! My plan is to release next version this weekend, and then I'll take a look at these PRs (I'll try to do that this weekend too).

[LostKobrakai]

The mix task for module groups is now merged. Having additional documentation by ex_doc for boundary data seems to be more problematic as expect within boundary itself (if not impossible) and the ex_doc team doesn't have the bandwidth at the moment for a more direct integration, so that part of this issue is something to be solved at a later time.

[Nezteb]

Since #21 was closed, I assume this isn't implemented? What work would need to be done to support @moduledoc metadata? 😄

[sasa1977]

I think it's pretty much described in the PR discussion itself (see in particular this comment). Adding metadata to the root module (the one that has use Boundary) should be trivial. Adding this meta to all other modules in the boundary is currently not possible, and it looks like it would require changing ex_doc.

This means we need to think about how ex_doc should be changed to support this feature, preferably in a boundary-agnostic way. Then, we should propose the change to the ex_doc team, and, assuming the change is approved, make a PR. Personally, I don't have the capacity for this.