Feat: Flexible configuration mechanism

[tomAgnicio]

Is your feature request related to a problem? Please describe.
The config mechanism is rather rigid, for this topic I'm focussing on ConstraintProvider. You can only pass in Class<? extends ConstraintProvider>

Describe the solution you'd like
An option to pass in an instance of a ConstrainProvider , so I can take full power.

I want to be able to load my constraints from a database or any other storage and define scores based on some runtime context.

Describe alternatives you've considered
There are some override mechanisms but its all seems limited and clumsy.

I took a checkout of the solver core to implement an alternative mechanism but the class is being used all over the place and instantiation happens late. I hoped to easily provide an alternative solution by implementing some interfaces.

So before even attempting this I want to get other people's opinion on this.

One other option is to make a really weird setup where I make a different class for each possible use case and have that delegate to everything else. But this feels weird and ugly.

[triceo]

Hello @tomAgnicio, and thanks for the feedback. To properly consider the best solution to your problem, would you mind explaining what exactly you're trying to accomplish?

• Why exactly is it important that constraints be loaded from a database?
• Why is the idea of having code in a database preferable to editing that same code in an IDE that is actually designed to do that?
• Why is it important for a constraint provider to be mutable, and therefore for you to be able to provide your own instances?

About constraint weights: To set constraint weights at runtime, we do have the constraint weight override feature, as you mentioned. Those constraint weights belong together with your planning solution, because only together do they define the problem you're solving. With different weights, you will get substantially different solutions even to the same data set - and therefore the dataset and the weights have to be considered as a single unit.

About the constraint provider class: the solver config needs to be serializable to a file, and therefore we can not accept arguments to it which do not serialize. For that reason, we only accept FQNs of classes, and not instances of those classes. Also, constraint providers are designed to be immutable and stateless, and therefore it should not matter that we don't let you create their instances.

We are aware of limitations of the constraint provider API though - it doesn't allow for easy extensibility and composability. Those are things we want to change, and we may even have some changes already in the works. But we are unlikely to ever make it easy to load in constraints from the database, that is simply not the development model we want to encourage. Constraints are code, and should be treated as such - including test coverage, CI etc.

[tomAgnicio]

Hi @triceo
Sorry for the delay; I was on vacation last week.

Regarding my request, the database aspect was the first thing that came to mind. In general, it would be nice to build the configuration myself using, for example, the builder pattern.

What I'm trying to accomplish:
1. Why exactly is it important that constraints be loaded from a database?

We are building a SAAS application. I envision providing a variety of constraints, like a ConstraintCatalog.
Not every tenant might want or even allow the same constraints. This is especially true for soft constraints. So, there is the concept of a ConstraintSet, which represents enabled constraints together with weights.

I also want to delegate testing and evaluation of constraints away from development. Users (analysts) can configure different ConstraintSets by adjusting weights. They can then run data scenarios against those different ConstraintSets to see how they perform, compare them, and optimize them.

2. Why is the idea of having code in a database preferable to editing that same code in an IDE that is actually designed to do that?
We are not putting the code in the database. Every constraint in the ConstraintCatalog is backed by a piece of code. It is the configuration of that constraint (the ConstraintSet) that is stored in the database.

3. Why is it important for a constraint provider to be mutable, and therefore for you to be able to provide your own instances?
Not sure if the word "mutable" is correct here; I'd call it "configurable." Let me give an example.

Note that the serializable requirement is new to me, I don't think this is stated in the documentation and it's not reflected in any of the quickstart examples.

Is this for the enterprise version? I'm doing experiments in the open-source version and since the constraintProvider is created through reflection I could technically pass in any instance by changing some code. It's just not easy to do, that's how I ended up here.

package com.agnicio.solver.constraints;

import ai.timefold.solver.core.api.score.stream.Constraint;
import ai.timefold.solver.core.api.score.stream.ConstraintFactory;
import ai.timefold.solver.core.api.score.stream.ConstraintProvider;
import org.jspecify.annotations.NonNull;

import java.io.Serializable;

public class MyCustomConstraintProvider  implements ConstraintProvider, Serializable {
    private final boolean isTodayANiceDay;

    public MyCustomConstraintProvider(boolean isTodayANiceDay) {
        this.isTodayANiceDay = isTodayANiceDay;
    }

    @Override
    public Constraint @NonNull [] defineConstraints(@NonNull ConstraintFactory constraintFactory) {
        if (isTodayANiceDay) {
            return new Constraint[0];
        } else {
            return new Constraint[0];
        }
    }
}

Yes, I could create two constraintProviders and put the if statement on the outside, but that's beside the question. In the ConstraintSet example I gave above, I have essentially n constraints, n if-statements, and 2^n constraint providers.

This class is still serializable. But I have more control over how to return those constraints.

As I said, I didn't know about the serializable requirement. So as far as I was concerned I would pass in spring beans, doing whatever I want, depending on whatever I want, and just adhere to the ConstraintProvider interface.

Pretty much any configuration mechanism would come in handy. The interface could for example accept an extra serializable argument.

    @Override
    public Constraint @NonNull [] defineConstraints(@NonNull ConstraintFactory constraintFactory, String constraintFactoryContext) {
        if (constraintFactoryContext.contains("....")) {
            return new Constraint[0];
        } else {
            return new Constraint[0];
        }
    }

At this moment the only way I could think of to do anything configurable would be through static fields, but besides the fact that you essentially have a singleton, those fields are not serializable anyway.

[tomAgnicio]

Hi @triceo,
did you have time to read my response?

[triceo]

Hello @tomAgnicio, thanks for bringing this up. The notification got lost among others, sorry for that.

First of all, I want to clear up one miscommunication - the constraint provider doesn't need to be serializable. The way we refer to it needs to be - I cannot put new MyConstraintProvider() into solver config files, those do not understand Java code; I can put MyConstraintProvider as a string in there though. This has nothing to do with `java.io.Serializable' - poor original explanation on my side.

This has a side benefit of forcing constraint providers to be immutable. Constraints are fundamentally static definitions and it is imperative that people do not use any state in their constraint providers. Not providing access to parameterized constructors helps with that, and it is very intentional. Unfortunately, nothing is ever perfect - people have been able to work around this by misusing static fields, or other means too obscure to mention.

Your idea of ConstraintSet seems to map exactly to ConstraintWeightOverrides - which are a set of enabled constraints and the weights that should be used. By overriding a constraint's weight to zero, the constraint will be entirely disabled, as if it never existed. It is trivial to build this object from any records kept in a database (or anywhere else), as ConstraintWeightOverrides.of(...) accepts a simple java.util.Map. This is intended to be your primary way of controlling which constraints are used with which solutions, and for the time being we have no intention of providing any other.

That said, user creativity appears to be limitless and the "crimes" committed in the name of parameterizing constraints are serious enough for us to have to reconsider this particular aspect. For one reason or another, people just really want to have dynamic constraint providers. Eventually, we are likely to provide an improvement to the API to help with this. It is not a priority right now though, and we generally don't give out estimates on community features.

[tomAgnicio]

@triceo

Thanks for your reply, ill dive into ConstraintWeightOverrides