Skip to content

Fully support choice only models #881

@AlexanderFengler

Description

@AlexanderFengler

We are currently working through a refactor that will introduce a HSSMBase class which both the current HSSM and the new HSSMRL classes are supposed to inherit from.

Especially for the HSSMRL class we already worked through the definition of composable model configs that combine a learning_process with a decision_process.

A long-standing "special case" of what we are doing are models that concern themselves only with p(choice|theta, X) and ignore rts.

The time is ripe now to address this and benefit from the emerging rl capabilities in one go.

Proposal:

We need the following steps,

  1. basic implementation of a softmax likelihood in pytensor/jax (very easy) with associated model config (include proposal on how to signify a likelihood as choice only)
  2. create new or just generalize current likelihood constructors to accept data that has only (theta_1, ..., theta_n, choice) inputs (this is a question of adjusting some of the dimensionality expectations. Note, we already have capability for choice probability networks, definitely take a look at that, it should be very reusable and informative.
  3. inherit from base class, and make a HSSCM class that deals with choice only models. Fiddle around with it until the basic softmax model works. The key will be some adjustments to the data validators and requesting the right underlying likelihood constructors. Otherwise it should be relatively straight forward.
  4. incorporate utilization of choice probability networks where they are available for a given model (this infrastructure will need to be fleshed out by adding more huggingface models). So if the user request e.g. angle model to the HSSCM class, we check if there is a "choice only" version of the likelihood available, if not throw error.
  5. Expand to HSSCMRL class, which connects choice only models to RL processes. This should be quite straight forward, at that point, because we already figured out all key pieces.

@krishnbera @digicosmos86 @cpaniaguam

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions