-
Notifications
You must be signed in to change notification settings - Fork 18
Open
Description
We are currently working through a refactor that will introduce a HSSMBase class which both the current HSSM and the new HSSMRL classes are supposed to inherit from.
Especially for the HSSMRL class we already worked through the definition of composable model configs that combine a learning_process with a decision_process.
A long-standing "special case" of what we are doing are models that concern themselves only with p(choice|theta, X) and ignore rts.
The time is ripe now to address this and benefit from the emerging rl capabilities in one go.
Proposal:
We need the following steps,
- basic implementation of a
softmaxlikelihood in pytensor/jax (very easy) with associated model config (include proposal on how to signify a likelihood as choice only) - create new or just generalize current likelihood constructors to accept data that has only
(theta_1, ..., theta_n, choice)inputs (this is a question of adjusting some of the dimensionality expectations. Note, we already have capability for choice probability networks, definitely take a look at that, it should be very reusable and informative. - inherit from base class, and make a
HSSCMclass that deals with choice only models. Fiddle around with it until the basicsoftmaxmodel works. The key will be some adjustments to the data validators and requesting the right underlying likelihood constructors. Otherwise it should be relatively straight forward. - incorporate utilization of choice probability networks where they are available for a given model (this infrastructure will need to be fleshed out by adding more huggingface models). So if the user request e.g.
anglemodel to theHSSCMclass, we check if there is a "choice only" version of the likelihood available, if not throw error. - Expand to
HSSCMRLclass, which connects choice only models to RL processes. This should be quite straight forward, at that point, because we already figured out all key pieces.
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
No labels