[1/x] Account WIT interface generation

[greenhat]

Close #720

Corresponding template repo PR that switches new project templates to the WIT generation: 0xMiden/rust-templates#30

The notes below are put as doc and module comments as well.

How to use it

1. Add #[component] on you impl MyAccountType {.
2. Add #[export_type] on every defined type that is used in the public(exported) method signature.

Example:

#[export_type]
pub struct StructA {
    pub foo: Word,
    pub asset: Asset,
}

#[export_type]
pub struct StructB {
    pub bar: Felt,
    pub baz: Felt,
}

#[component]
struct MyAccount;

#[component]
impl MyAccount {
    pub fn (&self, a: StructA) -> StructB {
        ...
    }
}

Foreseeing the case where a user forgets to put #[export_type] on their type the friendly error message with a hint is displayed.

How it works

1. #[export_type] collects the type descriptions.
2. #[component] on impl generates a WIT interface with all public methods, calls wit_bindgen::generate! and implements the Guest trait.
3. The user-defined types are mapped to the WIT types so in the generated bindings the user-defined types are used instead of the WIT-derived.

The WIT file is written in a subfolder under the target folder.

Escape hatch

In a small fraction of the cases where the WIT generation is not possible (think a type defined only in an external WIT file) or not desirable, the WIT generation can be disabled:

To disable WIT interface generation:

• Don't use the #[component] attribute macro in the impl MyAccountType section;

To use the manually crafted WIT interface:

• Put the WIT file in the wit folder;
• call miden::generate!(); and bindings::export!(MyAccountType);
• implement impl Guest for MyAccountType;

These steps are described in the doc comments to the #[component] macro.
See the storage-example using this escape hatch and opting out of the WIT generation.

Next steps

1. Implement proper enum and no Vec and Option support since there is no support for them in the frontend yet. In this PR I implemented enum without the data constructors, and it works fine. I'm pretty sure at this point that Vec and Option are also doable.

Tests

The new test module for macros-related tests is created at rust_sdk::macros with a test for various shapes of the user-defined types and an account <> note script execution test.

[greenhat]

@bitwalker @otrho @bobbinth I think I cracked the WIT generation for an account. Here is a brief description of how it works. Please check it out if I'm missing something. Overall, I think it'd work and the DX is quite nice.

The user writes only the Rust code like - https://github.com/0xMiden/compiler/blob/681d52ab02f52c1536cc4d30ff89e9d5dfe8c344/examples/basic-wallet/src/lib.rs?plain=1
Then, the #[component] macro on the impl MyAccount generates the WIT for the pub methods (see here) and also impl Guest for MyAccount (see below, notice the self argument in the user-written methods). The user-defined types that are used in the pub methods (see MyAccount::test_custom_types) have to be marked with another macro. I called it #[export_type] for now.
Of course, references and arbitrary generic types are not supported (no WIT equivalent) with appropriate error messages.

impl Guest for MyAccount from macro expansion:

impl self::bindings::exports::miden::basic_wallet::basic_wallet::Guest for MyAccount {
    #[doc = " Adds an asset to the account."]
    #[doc = ""]
    #[doc = " This function adds the specified asset to the account\'s asset list."]
    #[doc = ""]
    #[doc = " # Arguments"]
    #[doc = " * `asset` - The asset to be added to the account"]
    fn receive_asset(asset: Asset) {
        let __component_instance = MyAccount::default();
        __component_instance.receive_asset(asset);
    }
    #[doc = " Moves an asset from the account to a note."]
    #[doc = ""]
    #[doc = " This function removes the specified asset from the account and adds it to"]
    #[doc = " the note identified by the given index."]
    #[doc = ""]
    #[doc = " # Arguments"]
    #[doc = " * `asset` - The asset to move from the account to the note"]
    #[doc = " * `note_idx` - The index of the note to receive the asset"]
    fn move_asset_to_note(asset: Asset, note_idx: NoteIdx) {
        let __component_instance = MyAccount::default();
        __component_instance.move_asset_to_note(asset, note_idx);
    }
    fn test_custom_types(
        a: self::bindings::exports::miden::basic_wallet::basic_wallet::StructA,
        _b: self::bindings::exports::miden::basic_wallet::basic_wallet::EnumA,
    ) -> self::bindings::exports::miden::basic_wallet::basic_wallet::StructB {
        let __component_instance = MyAccount::default();
        let a: StructA = (a).into();
        let _b: EnumA = (_b).into();
        let result = __component_instance.test_custom_types(a, _b);
        (result).into()
    }
}

EDIT: Please ignore the red CI. It's PoC. The one test that matters (tests basic-wallet and P2ID note compilation) is green.

[bobbinth]

Very cool! This looks great!

Regarding the self argument: the example doesn't really showcase how to use it, but I'm assuming we'll be able to use it to access storage, right? (e.g., let x = self.foo.read(); or something like that?)

Also, I wonder if at some point in the future, we could also use self for common account methods so that we could do something like:

pub fn receive_asset(&self, asset: Asset) {
    self.add_asset(asset);
}

The assumption here is that add_asset() method could be auto-implemented on MyAccount struct by the #[component] macro.

Do we need the ability to define WIT manually?

Assuming we the bindings generator @bitwalker is working on can generate WIT based on type signatures in MASM, I don't think we'll need to write WIT manually.

[greenhat]

Regarding the self argument: the example doesn't really showcase how to use it, but I'm assuming we'll be able to use it to access storage, right? (e.g., let x = self.foo.read(); or something like that?)

Yes, exactly! Like in the storage example -

compiler/examples/storage-example/src/lib.rs

Lines 47 to 60 in 681d52a

	impl foo::Guest for MyAccount {
	fn set_asset_qty(pub_key: Word, asset: Asset, qty: Felt) {
	let my_account = MyAccount::default();
	let owner_key: Word = my_account.owner_public_key.read();
	if pub_key == owner_key {
	my_account.asset_qty_map.set(asset, qty);
	}
	}
	fn get_asset_qty(asset: Asset) -> Felt {
	let my_account = MyAccount::default();
	my_account.asset_qty_map.get(&asset)
	}
	}

but via self instead of my_account instance.

Also, I wonder if at some point in the future, we could also use self for common account methods so that we could do something like:

pub fn receive_asset(&self, asset: Asset) {
    self.add_asset(asset);
}

The assumption here is that add_asset() method could be auto-implemented on MyAccount struct by the #[component] macro.

We can generate it in the macro if we can define what tx kernel functions we want to wrap this way. I don't think asking the user is a suitable option. The DX problem I see here is how to convey to the user what functions the macros add.

Alternatively, we can define these wrappers in the BaseWallet trait in the SDK:

trait BaseWallet {
    fn add_asset(&mut self, a: Asset) {
        account::add_asset(a);
    }
}

And then a user would simply add impl BaseWallet for MyAccount {} to access them. This way the user has a clearer picture, what functions are added and why. I'd prefer this option vs. macro magic.

[greenhat]

Following up on the discussion elsewhere we need to have an option to disable WIT generation and a way to provide the WIT file(s) to have an escape hatch for edge cases (bring a WIT-defined type and use it in Rust code, etc.) that we cannot support in our WIT generation.

[bobbinth]

We can generate it in the macro if we can define what tx kernel functions we want to wrap this way. I don't think asking the user is a suitable option. The DX problem I see here is how to convey to the user what functions the macros add.

I think these would be all functions that are exposed for the "active account" - like getting the account ID, reading account vault, adding assets to the vault etc. (we won't need to function for storage because storage would be accessed through the fields).

We could have something like Account trait - e.g., something like:

pub trait Account {
    fn id(&self) -> AccountId {
        miden::account::get_id()
    }

    ...
}

And then have the user implement it manually - e.g.:

#[component]
struct MyAccount;

impl Account for MyAccount { }

But I think it may be better if the #[component] macro implemented it automatically.

Alternatively, we can define these wrappers in the BaseWallet trait in the SDK:

I think this could work too - but I was trying to address a different issue that is not related to the basic wallet: every account component "inherits" functions provided by the transaction kernel, and so, when we are inside the functions of the account component, it would be more natural to say something like self.id() (or maybe self.account_id()) rather than miden::account::get_id().

[greenhat]

We can generate it in the macro if we can define what tx kernel functions we want to wrap this way. I don't think asking the user is a suitable option. The DX problem I see here is how to convey to the user what functions the macros add.

I think these would be all functions that are exposed for the "active account" - like getting the account ID, reading account vault, adding assets to the vault etc. (we won't need to function for storage because storage would be accessed through the fields).

We could have something like Account trait - e.g., something like:

pub trait Account {
    fn id(&self) -> AccountId {
        miden::account::get_id()
    }

    ...
}

And then have the user implement it manually - e.g.:

#[component]
struct MyAccount;

impl Account for MyAccount { }

But I think it may be better if the #[component] macro implemented it automatically.

Works for me. The #[component] macro can add impl Account for MyAccount { }. With the Account trait defined in the SDK.

Alternatively, we can define these wrappers in the BaseWallet trait in the SDK:

I think this could work too - but I was trying to address a different issue that is not related to the basic wallet: every account component "inherits" functions provided by the transaction kernel, and so, when we are inside the functions of the account component, it would be more natural to say something like self.id() (or maybe self.account_id()) rather than miden::account::get_id().

Oh, I meant exactly what you described above with the Account trait. I just picked the wrong trait name.

[bitwalker]

LGTM, I didn't go crazy deep into the macro code, but this stuff is still in flux and I'm comfortable with what I've seen so far. Nice work!