Unable to CMORize data for CMIP7 (tos)

Hi! I am trying to cmorize data for FESOM for CMIP7 but for some reason I don't understand it is not working. It seems to complain about missing the tables, but I have cloned the submodules, so I don't know why it is not finding them.

Pycmor branch: `prep-release`
Pycmor commit: `b6107d110face3c7156c6681092d5aca8ac8f3c7`
Data path: `/work/bb1469/a270089/runtime/awiesm3-develop/TL255L91-CORE2_fesom27_1m_spinup_output_005/outdata/fesom`
Work directory: `/work/ab0995/a270152/pycmor_workspace/awicm3_cmip7` containing the scripts, yamls and logs
Pycmor dir: `/home/a/a270152/pycmor`

## Installation

I installed pycmor doing the following:

```bash
git clone https://github.com/esm-tools/pycmor.git
cd pycmor
git checkout prep-release
git submodule update --init --recursive # To get all the tables and CVs
mamba create -n pycmor python=3.10
conda activate pycmor
pip install -e .[dev,fesom]
```

## Error

I am running the sbatch command (slurm script and yaml at the bottom of the issue.

```bash
sbatch pycmor.slurm
```

The output I get in the main slurm job is:
```bash
[12/09/25 15:45:19] WARNING  CMIP7 Data Request API not    cmip7_interface.py:59
                             available: No module named
                             'data_request_api'. Install
                             with: pip install
                             CMIP7-data-request-api
[12/09/25 15:45:19] WARNING  CMIP7 Data Request API not    cmip7_interface.py:59
                             available: No module named
                             'data_request_api'. Install
                             with: pip install
                             CMIP7-data-request-api
WARNING: LocalCluster(96c5a11b, 'tcp://127.0.0.1:45409', workers=16, threads=256, memory=235.00 GiB) does not support adaptive scaling!
Cluster can be found at: self._cluster=LocalCluster(96c5a11b, 'tcp://127.0.0.1:45409', workers=16, threads=256, memory=235.00 GiB)
Dashboard http://127.0.0.1:8787
To see the dashboards run the following command in your computer's terminal:
        pycmor ssh-tunnel --username a270152 --compute-node l50292.lvt.dkrz.de
Importing Dask Extras...
...flox...
...done! Imported 1 libraries.
Attempting to download cmip7-tables from git...
WARNING: Failed to download cmip7-tables from git
Using packaged cmip7-tables: /home/a/a270152/pycmor/src/pycmor/data/cmip7
Attempting to download cmip7-tables from git...
WARNING: Failed to download cmip7-tables from git
Using packaged cmip7-tables: /home/a/a270152/pycmor/src/pycmor/data/cmip7
WARNING: CMIP7 Data Request API not available. Install with: pip install CMIP7-data-request-api
Attempting to download cmip7-tables from git...
WARNING: Failed to download cmip7-tables from git
Using packaged cmip7-tables: /home/a/a270152/pycmor/src/pycmor/data/cmip7
15:45:29.763 | INFO    | prefect - Starting temporary server on http://127.0.0.1:8156
See https://docs.prefect.io/v3/concepts/server#how-to-guides for more information on running a dedicated Prefect server.
15:45:38.016 | INFO    | Flow run 'amaranth-wildcat' - Beginning flow run 'amaranth-wildcat' for flow 'CMORizer Process'
15:45:38.043 | INFO    | Flow run 'amaranth-wildcat' - Finished in state Completed()
15:45:38.068 | INFO    | prefect - Stopping temporary server on http://127.0.0.1:8156
```

If I grep for errors in the logs folder I get:

```bash
$ grep -ir error logs/
logs/pycmor-process_1.log:2025-12-09 15:02:23.201 | ERROR    | pycmor.core.resource_locator:_download_from_git:573 - export_dreq_lists_json command not found. Install with: pip install git+ht
tps://github.com/WCRP-CMIP/CMIP7_DReq_Software
logs/pycmor-process_1.log:2025-12-09 15:02:23.202 | ERROR    | pycmor.core.resource_locator:locate:198 - Could not locate cmip7_metadata from any source. Tried: user path, cache, remote git,
packaged resources, vendored submodules.
logs/pycmor-process_2.log:2025-12-09 15:07:32.172 | ERROR    | pycmor.core.resource_locator:_download_from_git:573 - export_dreq_lists_json command not found. Install with: pip install git+ht
tps://github.com/WCRP-CMIP/CMIP7_DReq_Software
logs/pycmor-process_2.log:2025-12-09 15:07:32.172 | ERROR    | pycmor.core.resource_locator:locate:198 - Could not locate cmip7_metadata from any source. Tried: user path, cache, remote git, packaged resources, vendored submodules.
```

## Scripts and files used

awicm3_cmip7.yaml
```yaml
general:
  name: "AWI-ESM-3"
  description: "CMOR configuration for AWIESM 3.4"
  maintainer: "Miguel"
  email: "email"
  cmor_version: "CMIP7"
  #CMIP_Tables_Dir: /home/a/a270152/pycmor/CMIP7_DReq_Software
  #CV_dir: /home/a/a270152/pycmor/CMIP7-CVs
pycmor:
  # parallel: True
  warn_on_no_rule: False
  use_flox: True
  dask_cluster: "slurm"
  dask_cluster_scaling_mode: fixed
  fixed_jobs: 1
  # minimum_jobs: 8
  # maximum_jobs: 30
  # You can add your own path to the dimensionless mapping table
  # If nothing is specified here, it will use the built-in one.

inherit:
  # Common attributes shared by all rules
  activity_id: CMIP
  institution_id: AWI
  source_id: AWI-ESM-3
  variant_label: r1i1p1f1
  experiment_id: spinup
  grid_label: gn
  output_directory: .
  grid_file: /work/ab0246/a270092/input/fesom2/core2/core2_griddes_nodes.nc
  mesh_path: /work/ab0246/a270092/input/fesom2/core2/

rules:
  - name: Sea Surface Temperature
    description: "not important for now"
    inputs:
      - path: "/work/bb1469/a270089/runtime/awiesm3-develop/TL255L91-CORE2_fesom27_1m_spinup_output_005/outdata/fesom/"
        pattern: sst.fesom.*.nc_*
    cmor_variable: silly
    compound_name: ocean.tos.tavg-u-hxy-sea.mon.GLB # CMIP7 compound name
    model_component: ocean
    model_variable: "sst"
    #model_unit: ""

distributed:
  worker:
    memory:
      target: 0.6 # Target 60% of worker memory usage
      spill: 0.7 # Spill to disk when 70% of memory is used
      pause: 0.8 # Pause workers if memory usage exceeds 80%
      terminate: 0.95 # Terminate workers at 95% memory usage
    resources:
      CPU: 4 # Assign 4 CPUs per worker
    death-timeout: 600 # Worker timeout if no heartbeat (seconds): Keep workers alive for 5 minutes
# SLURM-specific settings for launching workers
jobqueue:
  slurm:
    name: pycmor-worker
    queue: compute # SLURM queue/partition to submit jobs
    account: ab0246 # SLURM project/account name
    cores: 4 # Number of cores per worker
    memory: 128GB # Memory per worker
    walltime: '00:30:00' # Maximum walltime per job
    interface: ib0 # Network interface for communication
    job-extra-directives: # Additional SLURM job options
      - '--exclusive' # Run on exclusive nodes
      - '--nodes=1'
    # Worker template
    worker-extra:
      - "--nthreads"
      - 4
      - "--memory-limit"
      - "128GB"
      - "--lifetime"
      - "25m"
      - "--lifetime-stagger"
      - "4m"
    # How to launch workers and scheduler
    job-cpu: 128
    job-mem: 256GB
    # worker-command: dask-worker
    processes: 4 # Limited by memory per worker!
    # scheduler-command: dask-scheduler
```

Slurm script:
```bash
#!/bin/bash
#SBATCH --job-name=pycmor-controller  # <<< This is the main job, it will launch subjobs if you have Dask enabled.
#SBATCH --account=ab0995                # <<< Adapt this to your computing account!
#SBATCH --partition=compute
#SBATCH --nodes=1
#SBATCH --time=00:30:00                 # <<< You may need more time, adapt as needed!
export PREFECT_SERVER_ALLOW_EPHEMERAL_MODE=True
export PREFECT_SERVER_API_HOST=0.0.0.0
# For more info about Prefect caching, see:
# https://docs-3.prefect.io/v3/develop/settings-ref#local-storage-path
export PREFECT_RESULTS_LOCAL_STORAGE_PATH=/scratch/a/${USER}/prefect

module load python3
eval "$(conda shell.bash hook)"
conda activate pycmor
prefect server start -b
time pycmor process awicm3_cmip7.yaml
prefect server stop
```

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Unable to CMORize data for CMIP7 (tos) #250

Installation

Error

Scripts and files used

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Unable to CMORize data for CMIP7 (tos) #250

Description

Installation

Error

Scripts and files used

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions