SciMLLogging Integration #2895

jClugstor · 2025-10-22T15:47:25Z

Checklist

Appropriate tests were added
Any code changes were done in a way that does not break public API
All documentation related to code changes were updated
The new code follows the
contributor guidelines, in particular the SciML Style Guide and
COLPRAC.
Any new documentation only uses public API

Additional context

Add any other context about the problem here.

jClugstor · 2025-10-22T19:47:16Z

will need SciML/SciMLBase.jl#1159 to fully work

termi-official · 2025-10-24T12:24:31Z

Since you are already working on the logging system, is there any chance that we can have additional logging messages for the time integration loop at trace level to debug issues in the timestepping itself? Or is that too costly?

ChrisRackauckas · 2025-10-24T12:39:10Z

We are splitting that to a different step. First let's get the infrastructure together, then add new messages.

Or is that too costly?

Never! If it's super costly, just put it in the All preset only. Semi? Detailed. Since there's individual switching and tons of levels, the goal is to just add tons and tons of default prints and if they are costly just don't have them on by default, but if you switch to All then you just get a lot of information.

jClugstor · 2025-10-24T17:00:02Z

At this point anything that was behind a verbose is now under a @SciMLMessage and is controlled by an ODEVerbosity.

Currently, the linear_verbosity and nonlinear_verbosity fields are not used because those get set in alg_cache or build_nlsolver which do not have access to the integrator or the DEOptions where the verbosity object is. So setting those in ODEVerbosity will not do anything.

For the Linear solves and Nonlinear solves I set verbose to Minimal() to prevent unnecessary printing from those during an ODE solve.

ChrisRackauckas · 2025-10-24T17:30:49Z

is this ready to review?

jClugstor · 2025-10-24T17:57:19Z

Yep, ready to be looked at.

ChrisRackauckas · 2025-10-25T07:31:46Z

Lots of test failures

jClugstor · 2025-10-27T14:52:05Z

Yes, because I'm using Minimal() for verbose for the Nonlinear and Linear solves, and they can't handle that until
SciML/NonlinearSolve.jl#647 and
SciML/LinearSolve.jl#756

jClugstor · 2025-11-04T23:26:06Z

There are a lot of

ERROR: LoadError: Unsatisfiable requirements detected for package OrdinaryDiffEqCore [bbf590c4]:
 OrdinaryDiffEqCore [bbf590c4] log:
 ├─possible versions are: 1.0.0-1.36.0 or uninstalled
 └─restricted to versions 1.37.0-1 by OrdinaryDiffEq [1dea7af3] — no versions left
   └─OrdinaryDiffEq [1dea7af3] log:
     ├─possible versions are: 6.103.0 or uninstalled
     └─OrdinaryDiffEq [1dea7af3] is fixed to version 6.103.0

which is weird because I bumped OrdinaryDiffEq to 1.37 in this PR, is CI not using the versions in this PR?

Adds @static if compatibility to support both old and new versions of OrdinaryDiffEqCore before and after PR SciML/OrdinaryDiffEq.jl#2895. The PR adds typeof(verbose) as a type parameter to DEOptions. This change makes DelayDiffEq.jl robust to this API change by detecting at compile time which version of DEOptions is available and using the appropriate constructor. Implementation: - Adds _count_deoptions_typeparams() to count type parameters - Uses DEOPTIONS_HAS_VERBOSE_TYPEPARAM constant for compile-time detection - Old version (OrdinaryDiffEqCore ≤ 1.36): 20 type parameters - New version (OrdinaryDiffEqCore ≥ 1.37): 21 type parameters Benefits: - Zero runtime overhead (@static if is compile-time) - Automatic version detection - Backward and forward compatible - No version number checking needed 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <[email protected]>

ChrisRackauckas · 2025-11-05T06:29:47Z

LTS ignores sources

ChrisRackauckas · 2025-11-05T06:31:11Z

docs/src/verbosity.md

@@ -0,0 +1,9 @@
+# Controlling Solver Verbosity


DiffEqDocs needs a PR that explains it in more detail how it's a solver-package specific thing.

Actually it makes it a bit rough for it to be solver-specific... since then you cannot do things in benchmarks and optimization more generally.

lib/OrdinaryDiffEqBDF/src/OrdinaryDiffEqBDF.jl

ChrisRackauckas · 2025-11-05T06:32:37Z

lib/OrdinaryDiffEqBDF/Project.toml

 TruncatedStacktraces = "1.4"
 SciMLBase = "2.99"
-OrdinaryDiffEqCore = "1.29.0"
+OrdinaryDiffEqCore = "1.37.0"


why is this required?

ChrisRackauckas · 2025-11-05T06:35:04Z

lib/OrdinaryDiffEqCore/src/verbosity.jl

+
+# Get all options in a group
+function group_options(verbosity::ODEVerbosity, group::Symbol)
+    if group === :error_control


This is very type unstable... is it supposed to be a generated function?

This isn't fixed so it's clear why you have boxing.

That's not used anywhere in the constructors, or anywhere else for that matter.

Then just cut it. It's dead code.

ChrisRackauckas · 2025-11-05T06:36:24Z

lib/OrdinaryDiffEqCore/src/verbosity.jl

+        )
+    elseif verbose isa Standard
+        # Standard: Everything from Minimal + non-fatal warnings
+        ODEVerbosity()


linear_verbosity should be Minimal still, so should nonlinear_verbosity.

Yes, that's handled by the default constructor

ChrisRackauckas · 2025-11-05T06:45:51Z

lib/OrdinaryDiffEqFIRK/src/firk_caches.jl

    linsolve = init(
        linprob, alg.linsolve, alias = LinearAliasSpecifier(alias_A = true, alias_b = true),
-        assumptions = LinearSolve.OperatorAssumptions(true))
+        assumptions = LinearSolve.OperatorAssumptions(true), verbose = Minimal())


this should be passing it on from the verbose.

all the linear and nonlinear verbosity specifiers are now coming from the ODEVerbosity.
I had to add an argument to alg_cache in order to pass the verbosity object on to the initialization of the linear and nonlinear caches

lib/OrdinaryDiffEqRosenbrock/src/rosenbrock_caches.jl

lib/OrdinaryDiffEqNonlinearSolve/src/utils.jl

jClugstor · 2025-11-05T21:12:00Z

Is there anything I can do about errors like this?

Failed to precompile OrdinaryDiffEqAdamsBashforthMoulton [89bda076-bce5-4f1c-845f-551c83cdda9a] to "/home/chrisrackauckas/.julia/compiled/v1.12/OrdinaryDiffEqAdamsBashforthMoulton/jl_QB6FLI".
ERROR: LoadError: Error opening package file /home/chrisrackauckas/.julia/compiled/v1.12/OrdinaryDiffEqCore/SoYJU_894Ui.so: /home/chrisrackauckas/.julia/compiled/v1.12/OrdinaryDiffEqCore/SoYJU_894Ui.so: cannot open shared object file: No such file or directory

jClugstor · 2025-11-06T04:49:26Z

@ChrisRackauckas I think this is ready, but it's hard to tell with all of the test failures. Most of them are because LTS doesn't use sources, some are from JET, and some are failures to open .so files. But I went through as many of the legitimate failures that I could find and fixed them. I also bumped all of minor versions of every subpackage to reflect that.

The biggest issue here turned out to be that in order to get the Linear and Nonlinear verbosity objects to the linear and nonlinear caches I had to add an argument to every alg_cache function and call, as well as build_nlsolver. This messes up DelayDiffEq and StochasticDiffEq, since DelayDiffEq uses OrdinaryDiffEq.alg_cache and StochasticDiffEq uses build_nlsolver. The solution for that is probably to just update those to SciMLLogging? But that will take a day or two probably.

But yeah not sure if it's mergable in this state.

ChrisRackauckas · 2025-11-06T10:42:45Z

The biggest issue here turned out to be that in order to get the Linear and Nonlinear verbosity objects to the linear and nonlinear caches I had to add an argument to every alg_cache function and call, as well as build_nlsolver. This messes up DelayDiffEq and StochasticDiffEq, since DelayDiffEq uses OrdinaryDiffEq.alg_cache and StochasticDiffEq uses build_nlsolver. The solution for that is probably to just update those to SciMLLogging? But that will take a day or two probably.

Yes, make the downstream PRs. But this should be easy to make backwards compatible with just the standard tricks right? Just make verbose be the last argument and make a dispatch that has a default verbose=ODEVerbosity(), so that it just adds the arg and passes down. Label it in a deprecated.jl file for future removal. Do this on alg_cache and build_nlsolver. Then downstream will work, just ignoring the verbosity argument until updated.

jClugstor marked this pull request as ready for review October 24, 2025 16:45

jClugstor force-pushed the use_scimlverbosity branch 2 times, most recently from ca97de9 to 2566850 Compare November 4, 2025 23:01

ChrisRackauckas-Claude mentioned this pull request Nov 5, 2025

Add compatibility layer for OrdinaryDiffEqCore DEOptions API change SciML/DelayDiffEq.jl#337

Merged

ChrisRackauckas reviewed Nov 5, 2025

View reviewed changes

lib/OrdinaryDiffEqBDF/src/OrdinaryDiffEqBDF.jl Outdated Show resolved Hide resolved

ChrisRackauckas reviewed Nov 5, 2025

View reviewed changes

lib/OrdinaryDiffEqRosenbrock/src/rosenbrock_caches.jl Outdated Show resolved Hide resolved

ChrisRackauckas reviewed Nov 5, 2025

View reviewed changes

lib/OrdinaryDiffEqNonlinearSolve/src/utils.jl Outdated Show resolved Hide resolved

jClugstor force-pushed the use_scimlverbosity branch from 2b9767b to 9ee7c80 Compare November 6, 2025 03:34

jClugstor mentioned this pull request Nov 7, 2025

Add fallback for Bool verbose, and two-arg SciMLMessage Macro SciML/SciMLLogging.jl#38

Merged

5 tasks

jClugstor added 29 commits November 14, 2025 10:55

fix test source for OrdinaryDiffEqRosenbrock for ODELinear

5b2906b

fix source of OrdinaryDiffEqSDIRK for OrdinaryDiffEqNonlinearSolve

a84745c

fix source of OrdinaryDiffEqTsit5 for OrdinaryDiffEqSymplecticRK

5163b43

fix source of SDIRK for ImplicitDiscreteSolve

5b5bae4

remove unused @SciMLMessage in OrdinaryDiffEqBDF

e890349

fix source of OrdinaryDiffEqRKN for OrdinaryDiffEqBDF

cfa5c5b

add several verbosity toggles

723757b

add several verbosity toggles

067835c

fix the None constructor

4547271

add stiff detection toggle

6206ade

add instability toggle back for SciMLBase

627fe0d

use lazy strings for interpolation

d8d1447

fix step rejected message

a818ea4

bump all library minor versions

f0542e1

remove unused SciMLMessages

fe0afcd

fix generic rosenbrock caches

c6e8f89

back compat for DelayDiffEq

f105fc3

use nonlinear verbosity in _initialize_dae!

56e821d

remove bad backwards compat

14b6447

backwards compat for stochasticdiffeq and delaydiffeq

17afac9

move deprecated alg_cache and build_nlsolver to deprecated files

f9ecd33

remove message in loopheader so Delay and StochasticDiffEq work

1985af0

add another dep path for oop

5b78108

put loopheader messages back

0d6e662

fix alg_cache for CG4a

96ba527

inferrable default path, make grouping more inferrable

8653c49

use multiple dispatch for handling verbose kwarg

939dbf9

add generated path for ODEVerbosity constructor

f277e96

fix up the verbosity tests

24b600d

jClugstor force-pushed the use_scimlverbosity branch from ee9fb5f to 24b600d Compare November 14, 2025 15:57

Uh oh!

SciMLLogging Integration #2895

Are you sure you want to change the base?

SciMLLogging Integration #2895

Conversation

jClugstor commented Oct 22, 2025

Checklist

Additional context

Uh oh!

jClugstor commented Oct 22, 2025

Uh oh!

termi-official commented Oct 24, 2025

Uh oh!

ChrisRackauckas commented Oct 24, 2025

Uh oh!

jClugstor commented Oct 24, 2025

Uh oh!

ChrisRackauckas commented Oct 24, 2025

Uh oh!

jClugstor commented Oct 24, 2025

Uh oh!

ChrisRackauckas commented Oct 25, 2025

Uh oh!

jClugstor commented Oct 27, 2025

Uh oh!

jClugstor commented Nov 4, 2025

Uh oh!

ChrisRackauckas commented Nov 5, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

jClugstor commented Nov 5, 2025

Uh oh!

jClugstor commented Nov 6, 2025

Uh oh!

ChrisRackauckas commented Nov 6, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants