Potential regression on symlink feature for FileTransform

[djieff]

Hi everyone,

When using ocio 1.0.9 FileTransform operations would allow us the usage of extension-less symlinks. Depending on the shot context, this symlinks could point to different files, of different formats.
While this was not an official feature, this was very convenient for example, the way our Digital Intermediate Space (dis), and their corresponding luts works:

Colorspace from config:

• !
  name: default_dis
  family: "Core"
  equalitygroup: ""
  bitdepth: 32f
  description: |
  DI Space
  isdata: false
  allocation: lg2
  allocationvars: [-15, 6]
  from_reference: ! {src: "default_dis", interpolation: linear}

• !
  name: tv1_dis
  family: "Core"
  equalitygroup: ""
  bitdepth: 32f
  description: |
  DI Space
  isdata: false
  allocation: lg2
  allocationvars: [-15, 6]
  from_reference: ! {src: "src_slog3.cube", interpolation: linear}

• !
  name: tv2_dis
  family: "Core"
  equalitygroup: ""
  bitdepth: 32f
  description: |
  DI Space
  isdata: false
  allocation: lg2
  allocationvars: [-15, 6]
  from_reference: ! {src: "src_slog3.cube", interpolation: linear}

• !
  name: tv3_dis
  family: "Core"
  equalitygroup: ""
  bitdepth: 32f
  description: |
  DI Space
  isdata: false
  allocation: lg2
  allocationvars: [-15, 6]
  from_reference: ! {src: "alexav3logc.spi1d", interpolation: linear}

• !
  name: default_display_LUT
  family: "Core"
  equalitygroup: ""
  bitdepth: 32f
  description: |
  Client display LUT
  isdata: false
  allocation: uniform
  allocationvars: [0, 1]
  from_reference: ! {src: "default_display_LUT", interpolation: linear}

• !
  name: tv1_display_LUT
  family: "Core"
  equalitygroup: ""
  bitdepth: 32f
  description: |
  Client display LUT
  isdata: false
  allocation: uniform
  allocationvars: [0, 1]
  from_reference: ! {src: "D65-VMod.cube", interpolation: linear}

• !
  name: tv2_display_LUT
  family: "Core"
  equalitygroup: ""
  bitdepth: 32f
  description: |
  Client display LUT
  isdata: false
  allocation: uniform
  allocationvars: [0, 1]
  from_reference: ! {src: "D065-mod_forSlog3input.cube", interpolation: linear}

• !
  name: tv3_display_LUT
  family: "Core"
  equalitygroup: ""
  bitdepth: 32f
  description: |
  Client display LUT
  isdata: false
  allocation: uniform
  allocationvars: [0, 1]
  from_reference: ! {src: "AlexaV3_K1S1_LogC2Video_Rec709_EE_nuke3d.cube", interpolation: linear}

Main lut directory files:
100 -rwxrwxr-x 1 user artists 94699 Nov 4 08:55 alexalogc.spi1d
7248 -rwxrwxr-x 1 user artists 7414986 Jan 8 08:55 AlexaV3_K1S1_LogC2Video_Rec709_EE_nuke3d.cube
952 -rwxrwxr-x 1 user artists 970348 Sep 27 12:33 D065-mod_forSlog3input.cube
636 -rwxrwxr-x 1 user artists 644435 Sep 27 12:33 D65-VMod.cube
4 lrwxrwxrwx 1 user artists 7 Nov 4 10:24 default_dis -> tv2_dis
4 lrwxrwxrwx 1 user artists 15 Nov 4 10:24 default_display_LUT -> tv2_display_LUT
4 lrwxrwxrwx 1 user artists 14 Nov 4 10:24 tv1_dis -> src_slog3.cube
4 lrwxrwxrwx 1 user artists 13 Nov 4 10:24 tv1_display_LUT -> D65-VMod.cube
4 lrwxrwxrwx 1 user artists 14 Nov 4 10:24 tv2_dis -> src_slog3.cube
4 lrwxrwxrwx 1 user artists 27 Nov 4 10:24 tv2_display_LUT -> D065-mod_forSlog3input.cube
4 lrwxrwxrwx 1 user artists 14 Nov 4 10:24 tv3_dis -> alexav3logc.spi1d
4 lrwxrwxrwx 1 user artists 13 Nov 4 10:24 tv3_display_LUT -> AlexaV3_K1S1_LogC2Video_Rec709_EE_nuke3d.cube

files for shot 201011_010:
4 lrwxrwxrwx 1 user artists 32 Oct 30 15:50 default_dis -> ../../_ocio/current/luts/tv1_dis
4 lrwxrwxrwx 1 user artists 40 Oct 30 15:50 default_display_LUT -> ../../_ocio/current/luts/tv1_display_LUT

files for shot 201012_020:
4 lrwxrwxrwx 1 user artists 32 Oct 30 15:50 default_dis -> ../../_ocio/current/luts/tv2_dis
4 lrwxrwxrwx 1 user artists 40 Oct 30 15:50 default_display_LUT -> ../../_ocio/current/luts/tv2_display_LUT

files for shot 201020_100:
4 lrwxrwxrwx 1 user artists 32 Oct 30 15:50 default_dis -> ../../_ocio/current/luts/tv3_dis
4 lrwxrwxrwx 1 user artists 40 Oct 30 15:50 default_display_LUT -> ../../_ocio/current/luts/tv3_display_LUT

However, when using ocio 1.1, all the symlinks used by FileTransform operations needs to have a file extension to work. Having to set an extension removes some flexibility we had in the past.
Here is the above example, adapted for ocio 1.1

• !
  name: default_dis
  family: "Core"
  equalitygroup: ""
  bitdepth: 32f
  description: |
  DI Space
  isdata: false
  allocation: lg2
  allocationvars: [-15, 6]
  from_reference: ! {src: "default_dis", interpolation: linear}

• !
  name: tv1_dis
  family: "Core"
  equalitygroup: ""
  bitdepth: 32f
  description: |
  DI Space
  isdata: false
  allocation: lg2
  allocationvars: [-15, 6]
  from_reference: ! {src: "tv1.dis.cube", interpolation: linear}

• !
  name: tv2_dis
  family: "Core"
  equalitygroup: ""
  bitdepth: 32f
  description: |
  DI Space
  isdata: false
  allocation: lg2
  allocationvars: [-15, 6]
  from_reference: ! {src: "tv2.dis.cube", interpolation: linear}

• !
  name: tv3_dis
  family: "Core"
  equalitygroup: ""
  bitdepth: 32f
  description: |
  DI Space
  isdata: false
  allocation: lg2
  allocationvars: [-15, 6]
  from_reference: ! {src: "tv3_dis.spi1d", interpolation: linear}

• !
  name: default_display_LUT
  family: "Core"
  equalitygroup: ""
  bitdepth: 32f
  description: |
  Client display LUT
  isdata: false
  allocation: uniform
  allocationvars: [0, 1]
  from_reference: ! {src: "default_display_LUT", interpolation: linear}

• !
  name: tv1_display_LUT
  family: "Core"
  equalitygroup: ""
  bitdepth: 32f
  description: |
  Client display LUT
  isdata: false
  allocation: uniform
  allocationvars: [0, 1]
  from_reference: ! {src: "tv1_display_LUT.cube", interpolation: linear}

• !
  name: tv2_display_LUT
  family: "Core"
  equalitygroup: ""
  bitdepth: 32f
  description: |
  Client display LUT
  isdata: false
  allocation: uniform
  allocationvars: [0, 1]
  from_reference: ! {src: "tv2_display_LUT.cube", interpolation: linear}

• !
  name: tv3_display_LUT
  family: "Core"
  equalitygroup: ""
  bitdepth: 32f
  description: |
  Client display LUT
  isdata: false
  allocation: uniform
  allocationvars: [0, 1]
  from_reference: ! {src: "tv3_display_LUT.cube", interpolation: linear}

Main lut directory files:
100 -rwxrwxr-x 1 user artists 94699 Nov 4 08:55 alexalogc.spi1d
7248 -rwxrwxr-x 1 user artists 7414986 Jan 8 08:55 AlexaV3_K1S1_LogC2Video_Rec709_EE_nuke3d.cube
952 -rwxrwxr-x 1 user artists 970348 Sep 27 12:33 D065-mod_forSlog3input.cube
636 -rwxrwxr-x 1 user artists 644435 Sep 27 12:33 D65-VMod.cube
4 lrwxrwxrwx 1 user artists 7 Nov 4 10:24 default_dis.cube -> tv2_dis.cube
4 lrwxrwxrwx 1 user artists 15 Nov 4 10:24 default_display_LUT.cube -> tv2_display_LUT.cube
4 lrwxrwxrwx 1 user artists 14 Nov 4 10:24 tv1_dis.cube -> src_slog3.cube
4 lrwxrwxrwx 1 user artists 13 Nov 4 10:24 tv1_display_LUT.cube -> D65-VMod.cube
4 lrwxrwxrwx 1 user artists 14 Nov 4 10:24 tv2_dis.cube -> src_slog3.cube
4 lrwxrwxrwx 1 user artists 27 Nov 4 10:24 tv2_display_LUT.cube -> D065-mod_forSlog3input.cube
4 lrwxrwxrwx 1 user artists 14 Nov 4 10:24 tv3_dis.spi1d -> alexav3logc.spi1d
4 lrwxrwxrwx 1 user artists 13 Nov 4 10:24 tv3_display_LUT.cube -> AlexaV3_K1S1_LogC2Video_Rec709_EE_nuke3d.cube

files for shot 201011_010:
4 lrwxrwxrwx 1 user artists 32 Oct 30 15:50 default_dis.cube -> ../../_ocio/current/luts/tv1_dis.cube
4 lrwxrwxrwx 1 user artists 40 Oct 30 15:50 default_display_LUT.cube -> ../../_ocio/current/luts/tv1_display_LUT.cube

files for shot 201012_020:
4 lrwxrwxrwx 1 user artists 32 Oct 30 15:50 default_dis.cube -> ../../_ocio/current/luts/tv2_dis.cube
4 lrwxrwxrwx 1 user artists 40 Oct 30 15:50 default_display_LUT.cube -> ../../_ocio/current/luts/tv2_display_LUT.cube

files for shot 201020_100:
4 lrwxrwxrwx 1 user artists 32 Oct 30 15:50 default_dis.cube -> ../../_ocio/current/luts/tv3_dis.spi1d (extension doesn't match, this file won't be resolved by ocio)
4 lrwxrwxrwx 1 user artists 40 Oct 30 15:50 default_display_LUT.cube -> ../../_ocio/current/luts/tv3_display_LUT.cube

In the above example, the tv3_dis colorspace would fail to load properly, since it wouldn't link to a .cube file, but a .spi1d file. We would have to convert the 1d lut to a 3d lut.

The result of this is a greater puzzle than before, considering we need to have matching extensions and transforms counts, to be able to have dynamic looks per shot.
per instance, from another config, here is one of our colorspace (every colorspace prefixed by 'cam', needs to be dynamically resolved per shot/camera):

• !
  name: "Client look"
  family: Looks
  equalitygroup: ""
  bitdepth: 32f
  description: |
  Client look
  isdata: false
  allocation: uniform
  allocationvars: [0, 1]
  from_reference: !
  children:
  - ! {src: linear, dst: white_balance, direction: inverse}
  - ! {src: linear, dst: cam_matrix1}
  - ! {src: linear, dst: cam_matrix2}
  - ! {src: linear, dst: cam_gamut_matrix}
  - ! {src: linear, dst: cam_linToLog}
  - ! {src: linear, dst: cam_gamut_LUT}
  - ! {src: linear, dst: client_grade}
  - ! {src: linear, dst: cam_client_LUT}

Was there an official reason to remove this mechanism, could it be an official feature?

If not, what would you recommend to do to precise colorspaces set per shots?

Thanks,
JF

[hodoulp]

Hi @djieff

Thanks for your feedback and sorry for the late answer.
Please feel free to contribute (if you have some time) so we can all improve the library overall quality. Any kind of contributions are always welcome.

Patrick