Improvements looking for your help!

[fedebotu]

Would you like to contribute to RL4CO?

This section is about improvements to the library that are not currently worked on by someone and are up for grabs. Feel free to comment if you would like to take on some task!

TODOs

• Logging hyperparameters with Wandb
  Details

  
  The logger on Wandb does not properly "unpack" keyword arguments, as can be seen here:
  
  Normally, we should see some nested tab structure (like optimizer_kwargs: 1) ... 2)... ), which is also easier to search
  We suspect it should not be hard to do - probably, the answer is somewhere in here!

• MDAM beam search; refactoring with latest structure

• Pointer Network refactoring with latest structure

• Fix Google Colab notebook badges, e.g. here

Done

• Hydra documentation (thanks @LTluttmann!)
  -

  Details

  
  Currently, the Hydra documentation is a bit... ehehm ... lacking. Indeed, it is not easy to navigate the first time, no matter how much you will love it after. Would be great to have some documentation section (or even, say, some notebook) explaining how configurations are handled, along with tips and tricks
  For example:
  add new element to configuration with "+" (in this case, the trainer.devices argument): python run.py experiment=routing/am +trainer.devices='[1]'

Tip

Do you have some feature you would like to add, such as a new policy, RL algorithm, problem and more? Just get in touch ;)

[LTluttmann]

Maybe this does not directly belong here, but regarding hydra I struggled when using the debug configuration, as this disables the wandb logger. This in turn leads to a failure in the hydra.run.dir, as this (by default) expects the wand logger. Maybe a better approach is to use:

hydra:
  run:
    dir: ${paths.log_dir}/${mode}/runs/${now:%Y-%m-%d}_${now:%H-%M-%S}
  sweep:
    dir: ${paths.log_dir}/${mode}/multiruns/${now:%Y-%m-%d}_${now:%H-%M-%S}
    subdir: ${hydra.job.num}

as default and override this in the wand.yaml config as follows:

# @package _global_

# https://wandb.ai
logger:
  wandb:
    _target_: lightning.pytorch.loggers.wandb.WandbLogger
    # name: "" # name of the run (normally generated by wandb)
    save_dir: "${paths.output_dir}"
    offline: False
    id: null # pass correct id to resume experiment!
    anonymous: null # enable anonymous logging
    project: "rl4co"
    log_model: False # upload lightning ckpts
    prefix: "" # a string to put at the beginning of metric keys
    # entity: "" # set to name of your wandb team
    group: ""
    tags: []
    job_type: ""

hydra:
  run:
    dir: ${paths.log_dir}/${mode}/runs/${logger.wandb.group}/${logger.wandb.name}/${now:%Y-%m-%d}_${now:%H-%M-%S}
  sweep:
    dir: ${paths.log_dir}/${mode}/multiruns/${logger.wandb.group}/${logger.wandb.name}/${now:%Y-%m-%d}_${now:%H-%M-%S}
    subdir: ${hydra.job.num}

What do you think? I could make a PR for this, maybe including some examples for how to use hydra properly.

[fedebotu]

Totally agree! Since we almost always use Wandb, we have not encountered this problem so far. But it's best to modify for clarity!
Also about the documentation, right now it is not that clear so it is best to clarify it 👍