multi-game support and flesh out cli describe command for more formats

[GompDS]

This pull-request does a couple things:

1. Update the cli, dvdbnd, and msb files to have multi-game support
2. Flesh out the cli describe command to support formats inside nested bnds, like matbin, flver, and msb
3. Tweaked the output of the cli extract command so output files would follow the folder structure of the dvdbnd

In regards to multi-game support:

• fstools should now support reading files from Elden Ring: Nightreign and Elden Ring: Shadow of the Erdtree
• The '--game-type' arg, which specifies the target game, is now required for the cli. Valid types include: er-pc and nr-pc
• As a result, pem keys for a specific game are now required to be in a subfolder of the keys dir which has the name of the chosen game type
• Following suit, the extract command will extract files to a subfolder with the game type name.
• The format I've tweaked the most by far is msb. The structure has changed quite a bit so that the param enums are segmented into different modules for each game.

In regards to describing formats inside nested bnds:

• A new option has been added to the describe command called '-n'. This takes any number of paths, separated by commas, which represent a chain of bnds leading to the format within. The first path should be a full dvdbnd path to the top level bnd and any subsequent paths should only be file names of nested bnds.

One more thing:
Since this is my first pr to the repo, and I'm new to rust programming, please let me know if there's anything I should adjust or what not. Thanks.

[garyttierney]

For multi-game support this looks like an okay first-pass, but I expect the design to change quite a bit. For types that exist across several games we'll probably want some sort of proc macro that generates variants of that type:

#[versioned(by = Game)]
#[repr(C, packed)]
pub struct SomeType {
   #[added(EldenRing)]
   #[removed(Nightreign)]
   some_value: U32,
}

which would generate SomeTypeEldenring and SomeTypeNightreign, and some trait impls for accessing data. This starts to get complicated when we think about "sub-types" (i.e. MSB param types) so it's not quite as simple as the contrived example above.

We can see how it impacts downstream API consumers already:

                    if let PartData::EldenRing(parts::elden_ring::PartData::DummyAsset(_)) = part.part {
                        return None;
                    }

For this example I think the direction towards the ideal API is something like:

// Generated types from something like the macro above:
pub trait DummyAssetPart {}
pub struct EldenRingDummyAssetPart {}
pub struct NightreignDumymAssetPart {}

let Ok(data) = part.data::<DummyAssetPart>(); else { panic!("not a dummy asset part!"); };
let some_value : Option<u32> = data.some_value();
let some_value_raw = data.downcast::<EldenRingDummyAssetPart>();

[garyttierney]

Within the formats crate we try to avoid allocating unless absolutely necessary. Can we make this return an iterator instead of a collection?

[garyttierney]

You can use regular Rust comments to write the help message for options, i.e.

        /// Chain of nested bnd names. Required to describe a file therein.
        ///
        /// Examples:
        ///    Describe a tae inside the anibnd of an objbnd: -n obj\o000100.objbnd.dcx
        /// ...
        #[arg(short, long, required = false, value_delimiter = ',')]
        nested_bnd_names: Vec<String>,

[garyttierney]

FWIW, to fix the bulk of these CI complaints automatically:

> $ cargo clippy --fix --all [--allow-staged --allow-dirty]
> $ cargo fix --all --allow-staged --allow-dirty
> $ cargo +nightly fmt

[GompDS]

For multi-game support this looks like an okay first-pass, but I expect the design to change quite a bit. For types that exist across several games we'll probably want some sort of proc macro that generates variants of that type:

#[versioned(by = Game)]
#[repr(C, packed)]
pub struct SomeType {
   #[added(EldenRing)]
   #[removed(Nightreign)]
   some_value: U32,
}

which would generate SomeTypeEldenring and SomeTypeNightreign, and some trait impls for accessing data. This starts to get complicated when we think about "sub-types" (i.e. MSB param types) so it's not quite as simple as the contrived example above.

We can see how it impacts downstream API consumers already:

                    if let PartData::EldenRing(parts::elden_ring::PartData::DummyAsset(_)) = part.part {
                        return None;
                    }

For this example I think the direction towards the ideal API is something like:

// Generated types from something like the macro above:
pub trait DummyAssetPart {}
pub struct EldenRingDummyAssetPart {}
pub struct NightreignDumymAssetPart {}

let Ok(data) = part.data::<DummyAssetPart>(); else { panic!("not a dummy asset part!"); };
let some_value : Option<u32> = data.some_value();
let some_value_raw = data.downcast::<EldenRingDummyAssetPart>();

Thanks for the feedback. It's an approach I hadn't considered as a new rust programmer and I definitely agree that my current approach isn't ideal. I'll have to look into how macros work in more detail.

[garyttierney]

@GompDS unless you've already started working on something for this (we don't need to tackle that as part of this PR btw, it's going to be a pretty involved piece of work with a lot of design decisions) I'm going to pull this branch and see what I can do to bring in some form of abstraction.

[GompDS]

I agree it would be good if we could separate the updates to describe ive done and the multi game support stuff. It seems like two different things to tackle.

[garyttierney]

Alright, here's an initial sketch of how this is working:

#[versioned(versions = [eldenring, nightreign])]
pub enum PointData<'a> {
   Other,
   GroupDefeatReward {
      unk0: I32<LE>,
      unk4: I32<LE>,
      unk8: I32<LE>,
      unkc: I32<LE>,
      unk10: I32<LE>,
      unk14: [I32<LE>; 8],
      unk34: I32<LE>,
      unk38: I32<LE>,
      unk3c: I32<LE>,
      unk40: I32<LE>,
      unk44: I32<LE>,
      unk48: I32<LE>,
      unk4c: I32<LE>,
      unk50: I32<LE>,
      unk54: I32<LE>,
      unk58: I32<LE>,
      unk5c: I32<LE>,
   },
   #[added(nightreign)]
   UserEdgeEliminationExterior {
      // we want the structs inline, we'll generate separate structs based on the version permutations
   },
   // ...
}

This generates quite a few things.

We get shared structs where all versions share a data structure:

   pub struct GroupDefeatReward {
      unk0: I32<LE>,
      unk4: I32<LE>,
      unk8: I32<LE>,
      unkc: I32<LE>,
      unk10: I32<LE>,
      unk14: [I32<LE>; 8],
      unk34: I32<LE>,
      unk38: I32<LE>,
      unk3c: I32<LE>,
      unk40: I32<LE>,
      unk44: I32<LE>,
      unk48: I32<LE>,
      unk4c: I32<LE>,
      unk50: I32<LE>,
      unk54: I32<LE>,
      unk58: I32<LE>,
      unk5c: I32<LE>,
   }

We generate a struct for the contents of enum variants (see why below):

mod nightreign {
   pub struct UserEdgeEliminationExterior {
      
   }
}

We generate traits to expose a common API across multiple versions:

pub trait UserEdgeEliminationExterior {
    fn some_field(&self) -> ...;
}

impl UserEdgeEliminationExterior for nightreign::UserEdgeEliminationExterior {}

I'll split out the describe/versioning changes from this PR when I get to opening one on top of this.

[GompDS]

If I'm understanding this correctly, would it mean that there is also an eldenring::GroupDefeatReward and a nightreign::GroupDefeatReward? or are those variants only implemented when an added or removed macro is used?

[garyttierney]

or are those variants only implemented when an added or removed macro is used?

This is correct, the generated enum would look something like:

pub enum NightreignPointParam {
   GroupDefeatReward(common::GroupDefeatReward /* actual struct with the fields */),
   UserEdgeEliminationExterior(nightreign::UserEdgeEliminationExterior)
}

And we'd also have a GroupDefeatReward trait (naming TBD, repeating this much gets confusing) for forward compatibility. E.g., a new title we support adds a field to GroupDefeatReward:

   GroupDefeatReward {
      unk0: I32<LE>,
      unk4: I32<LE>,
      unk8: I32<LE>,
      unkc: I32<LE>,
      unk10: I32<LE>,
      unk14: [I32<LE>; 8],
      unk34: I32<LE>,
      unk38: I32<LE>,
      unk3c: I32<LE>,
      unk40: I32<LE>,
      unk44: I32<LE>,
      unk48: I32<LE>,
      unk4c: I32<LE>,
      unk50: I32<LE>,
      unk54: I32<LE>,
      unk58: I32<LE>,
      unk5c: I32<LE>,
      #[added(version = "bloodborne2")]
      oh_no: I32<LE>,
   },

This forces us to split the once common struct into at least 2 versions and is a breaking change if users are consuming this directly. Instead, we'd have a trait we can program against and that new field results in an accessor that returns an optional value (note we'll probably want to make it something other than Option to discriminate from truly optional values)

pub trait GroupDefeatReward {
    // ...
    fn oh_no(&self) -> Option<i32>;

[GompDS]

Ok cool. My main concern here was that common structs could vary between games, so I'm glad to see you've considered that. Seems like a good approach so far.

[garyttierney]

@GompDS we have a new #[derive(Describe)] in #50 we can leverage for this and better handling of keys for multi-game support. I'll pull your describe changes into that and create a PR for the versioned derive macro after.

[GompDS]

Great, that sounds good. I think it's best we separate that and the multi game support now anyways.