Structured diagnostics for the compiler #13766

Octachron · 2025-01-31T16:00:57Z

See also ocaml/RFCs#45 and https://icfp24.sigplan.org/details/ocaml-2024-papers/16/Structured-diagnostics-for-the-OCaml-compiler for previous discussions .

A limitation of the current compiler diagnostics (error messages, warnings, time profile information and other compiler developer debugging output) is that it cannot be parsed easily and reliably by developer tools. This creates an impedance mismatch between the compiler and those tools that tend to resort to trying to parse manually compiler error message.

This PR proposes to fix this problem by adding to the compiler a machinery for building structured compiler diagnostics before displaying them in various formats.

This PR is probably best reviewed commit by commit:

the first eight commits slowly introduce the new machinery part by part
commit 9 to 14 converts the current compiler diagnostics to this new machinery (going through the error messages, the debugging outputs, then the toplevel, and finally the compiler configuration)
commit 15 adds a new tools for instrospecting the compiler diagnostics, and the following commit uses this new tools
to export json-schema and adds the corresponding tests in the testsuite
the last commits 4 commits update the manual and the man pages.

Octachron · 2025-01-31T18:33:20Z

Note that I have tested some quick prototypes for deserializers and a bridge between the proposed compiler diagnostic and the grace library at https://github.com/Octachron/sdiag-prototypes .

johnyob · 2025-01-31T20:06:15Z

Note that I have tested some quick prototypes for deserializers and a bridge between the proposed compiler diagnostic and the grace library at https://github.com/Octachron/sdiag-prototypes .

Its lovely to see grace being used here :) I look forward to playing with this

Octachron · 2025-02-18T15:48:06Z

I have pushed a commit that hide the change of API to the toplevel library beyond a compatibility layer.

gasche

I did a first round of looking at the code, rather shallow as a review.

I did not look at the manual yet, as I read in commit order; this was a mistake and I'll start there on my next round.
I did not look at the new modules (diagnostics, diagnostics_history) in depth, only quickly at the interface (assessment: too hard to follow for me, I need to read the manual first). It's a lot of non-critical code and I plan to declare that it is reasonable and go by the tests mostly, instead of trying to review everything line-by-line.
I looked at the code that adapts the compiler codebase. This is as I expected, invasive but reasonable.

gasche · 2025-05-11T15:19:23Z

utils/diagnostic.mli

+
+  | List: 'a typ -> 'a list typ (** Combinators *)
+  | Pair: 'a typ * 'b typ -> ('a * 'b) typ
+  (** Specialize (2,3,4)-tuples to avoid defining tuples as heterogeneous


Nitpick: clearly(?!) this should be {2,3,4}-tuples rather than (2,3,4)-tuples.

gasche · 2025-05-11T15:22:52Z

utils/diagnostic.mli

+      record at update [u]. The field is optional if [opt] is [Some true] *)
+
+  val new_field_opt: vl update  -> string -> 'a typ -> 'a field
+  (** [new_field_opt] is a short-hand for [new_field ~opt:true] *)


I would expect either two functions or an optional argument, do we really want to offer both?

gasche · 2025-05-11T15:25:56Z

utils/diagnostic_history.mli

+  type point =
+    | Inception
+     (** constructor only: derived constructor are incepted before being
+         published.*)


I'm not sure what "inception" mean.

(Also: derived constructors are incepted ...)

gasche · 2025-05-11T15:28:44Z

utils/diagnostic_history.mli

+(** Last version present in the history *)
+val current_version: 'a t -> version
+
+(** An ['id update] is a version registered in the history ['id History.t].*)


['id Diagnostic_history.t], I suppose?

gasche · 2025-05-11T15:32:25Z

utils/diagnostic_history.mli

+(** An ['id update] is a version registered in the history ['id History.t].*)
+type 'a update
+val new_version: 'a t -> version -> 'a update
+(** [new_version h v] unconditionnally creates an update but register an error


gasche · 2025-05-13T07:03:31Z

toplevel/topcommon.ml

  in
-  fprintf ppf
-    "@.Hint: @[\
+    fprintf ppf


the extra spacing of fprintf looks unintentional.

gasche · 2025-05-13T07:14:41Z

utils/config_diagnostic.ml

+
+let print log =
+  log_variables log;
+  Log.flush log


I tested the new -config outputs on my machine and I observe a bug:

ocamlc -config works exactly as before

ocamlc -log-format {json,sexp} -config work as expected (nice!)
(nitpick: the sexp format will sometimes fit several fields on the same line, I would rather have a vertical printing with at most one field per line)

ocamlc -log-format stdout -config prints only the values and not the keys/variables, so that output is incorrect

@Octachron points out that in fact stdout is not supposed to be the same as not setting anything, currently there is stdout and stdout-with-keys and the latter is not exposed. It would be nice if both options were exposed, and maybe they could be named stdout-light and stdout-full, or maybe stdout-values and just stdout.

gasche · 2025-05-13T07:46:08Z

tools/diaginfo.mli

+(*   the GNU Lesser General Public License version 2.1, with the          *)
+(*   special exception on linking described in the file LICENSE.          *)
+(*                                                                        *)
+(**************************************************************************)


The .mli would be an okay place to include some documentation of how to use diaginfo.

Help output

The documentation of -schema-format is weird.

-schema-format {json|adt} <name> print the schema <name> in <name> format

I don't understand what it means. (Does this command expect two parameters, or just one that is either json or adt? I assume the latter.)

How to get a list of schemas?

On a first try using the tool, I failed to understand how to get a list of schemas. I think that there should be an explicit option, for example --list, to list all schemas. When I printed the --help output to write the comment about I noticed the part about *, but this does not seem to work easily on my machine:

$ ./runtime/ocamlrun ./tools/ocamldiaginfo -schema * Unknown schema name: aclocal.m4 $ ./runtime/ocamlrun ./tools/ocamldiaginfo -schema "*"

The first comes from the fact that my shell interpolates the *, and the second produces empty output.

History output

A trailing newline seems to be missing.

$ ./runtime/ocamlrun ./tools/ocamldiaginfo -history | tail -n 5 New label backtrace, ?l structured_text New label compiler, ?compiler New label errors, ?l l structured_text New label trace, ?l l structured_text Seal[gasche@framawork github-trunk (structured_diagnostics_full)]$

gasche · 2025-05-13T07:48:57Z

diagnostic_schemes/config_scheme.json

+  "properties" :
+    {
+      "metadata" : { "$ref" : "#/$defs/metadata"},
+      "linear_magic_number" : { "type" : "string"},


Nitpick: it's slightly weird that there is a space before the key ({ "type") but not after the value ("string"}).

gasche · 2025-05-13T07:53:12Z

toplevel/topdirs.ml

+  Fun.protect (fun () -> f log x)
+  ~finally:(fun () -> Log.flush log)
+
+  let dir_load = with_log V2.dir_load


The indentation here is confusing

Octachron force-pushed the structured_diagnostics_full branch from 8edb2d8 to 8e07bd5 Compare February 3, 2025 12:17

gasche self-assigned this Feb 5, 2025

Octachron force-pushed the structured_diagnostics_full branch from a3ede5d to 355143d Compare February 18, 2025 15:46

johnyob mentioned this pull request Apr 15, 2025

More multi-location error messages #12633

Open

gasche mentioned this pull request May 11, 2025

[minor] use a record in Compenv.process_deferred_actions #14026

Merged

gasche reviewed May 13, 2025

View reviewed changes

Octachron force-pushed the structured_diagnostics_full branch from 355143d to b0ea420 Compare June 30, 2025 15:40

Octachron added 3 commits June 30, 2025 18:01

structured diagnostic: definitions

e0b6d19

structured diagnostics: validation

88ebaeb

structured diagnostics: log

16a4f5d

Octachron force-pushed the structured_diagnostics_full branch from b0ea420 to 31e8135 Compare June 30, 2025 16:07

Octachron added 16 commits July 2, 2025 13:27

structured diagnostic: color handling refactorization

e6154cd

Format doc: tree representation

c44172b

structured diagnostic: compiler diagnostic definition

f183e57

structured diagnostics: json and s-expression backends

c2875c7

fully switch otype printer to Format_doc

ed1c626

structured diagnostics: switch to error compiler to log

1ef9421

structure diagnostics: timing profile

ab23a72

structured diagnostics: compiler debugging output

df35fae

structured diagnostic: toplevel response

b885d63

structured diagnostics: config

37af581

structured diagnostics: diaginfo tool

6cb4c80

structured diagnostic: testsuite

4f07f47

structured diagnostics: manual

5a73220

diagnostic: add description to fields and constructors

820e7b4

Update man pages

abb426c

Manual, version 2

23f3f44

Octachron added 10 commits July 2, 2025 13:28

Rename Config_diagnostic to avoid the config prefix

fa70089

toplevel: backward compatibility for toplevel API

ceb40dd

backward compatibility: keep Location.report_exception

ff26a10

structured diagnostic: split stdout format in two

a5fab69

SQUASH_ME: fix text_in_prefix

194ec86

review: improve diaginfo arguments

11f4341

review: rename inception to preview

453b13f

review: split optional field for documentation purpose

d9809a2

review: typo in diagnostic history

2696e2b

review: profile

0487e62

Octachron force-pushed the structured_diagnostics_full branch from 862634e to 0487e62 Compare July 8, 2025 15:26

review: improve formatting

4cecf90

OlivierNicole self-requested a review November 26, 2025 15:53

Structured diagnostics for the compiler #13766

Are you sure you want to change the base?

Structured diagnostics for the compiler #13766

Uh oh!

Conversation

Octachron commented Jan 31, 2025

Uh oh!

Octachron commented Jan 31, 2025

Uh oh!

johnyob commented Jan 31, 2025

Uh oh!

Octachron commented Feb 18, 2025

Uh oh!

gasche left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Help output

How to get a list of schemas?

History output

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants