Build guide: validation #520

pdaoust · 2025-02-04T00:16:41Z

Closes #475 .

This PR depends on #516 and #517 . Don't review until they're merged.

So while this PR reflects my attempts to simplify the work I'm doing on the Build Guide, it doesn't incorporate the feedback I got today to simplify even more. So it does do some explanatory stuff that may or may not be necessary. Feel free to offer feedback on whether it's too much information, with the caveat that I'll be using that feedback in future PRs and will probably leave this one as it is, because it's already been written and it's probably useful information. (That said, if you feel it gets in the way, please explicitly say that you think it should be removed.)

Covers:

validate callback
genesis_self_check callback
must_get_*
a few design considerations for validation

ordering

…ucture-zome-functions

…tiny edits

…e/app-structure-dnas

…cture-happs

ThetaSinner · 2025-02-21T18:30:56Z

Testing is a big subject that we probably ought to add to the build guide. Would a simple mention to "test test test -- we'll write more about this later but for now, look at the stub tests that the scaffold tool generated for you" be enough, do you think?

I'd like to just say something like I did. We suggest testing all your validation rules with a single agent, then also testing them with at least two agents and making sure the DHT syncs. For sure refer them to the generated tryorama tests for how to actually do that. I'm pretty sure it comes with multi-agent tests.

ThetaSinner · 2025-02-21T18:31:09Z

Sorry for the close/open - keyboard shortcuts while trying to enter code

pdaoust · 2025-02-21T22:55:50Z

@ThetaSinner

some code
This builds fine inside an integrity zome but is a really bad idea

Oh my goodness. That is an eldritch horror. Okay, I'll add something to the docs.

pdaoust · 2025-02-24T18:55:14Z

@ThetaSinner this commit b19615c warns about non-deterministic stuff.

pdaoust · 2025-02-21T22:53:31Z

src/pages/build/validation.md

+* Data may have dependencies that affect validation outcomes, but those dependencies must be [addressable](/build/identifiers/), they must be retrievable from the same DHT, and their addresses must be constructable from data in the operation being validated. (Note that because an action references the action preceding it on an agent's source chain, this is a dependency you don't need to build into your data.) If dependencies can't be retrieved at validation time, the `validate` callback terminates early with an [indeterminate result](/build/validate-callback/#validation-outcomes), which will cause Holochain to try again later.
+* When multiple actions are written in a same [atomic transaction](/build/zome-functions/#atomic-transactional-commits), actions' ops can only have dependencies on prior actions committed in the transaction, not later actions.
+* You don't need to validate your data manually before committing --- Holochain [validates it after the zome function that writes it is finished](/build/zome-functions/#validate-dht-operations), and returns any validation failure to the caller.
+* **Test, test, test.** Validation is the gate that accepts or rejects all DHT data, so make sure you write thorough test coverage for your validation functions. If the data being validated has no dependencies on DHT data or DNA/zome info, we recommend writing Rust unit tests for the [validation function stubs](/build/validate-callback/#create-boilerplate-code-with-the-scaffolding-tool) that the scaffolding tool generates. We also recommend testing your validation code by writing single- and multiple-agent [Tryorama](https://github.com/holochain/tryorama/) test scenarios for zome functions that write data (we'll write about Tryorama soon; in the meantime, you can check the Tryorama GitHub readme and the scaffolded tests in a project's `tests/src/` folder for guidance).<!-- TODO: link to Tryorama page -->


@ThetaSinner here's a note about testing. Does it look okay? (The scaffolding tool creates nice stub functions that lend themselves to unit testing, as long as your data doesn't rely on any host functions to validate.)

actions' ops can only have dependencies on prior actions committed in the transaction, not later actions.

Technically this is a should not rather than a cannot but probably best to document it that way :D Structural references that sys validation will check must be in the right direction. There's nothing stopping you mixing things up in app validation but it will slow down validation and if you create a circular dependency then it will stall validation.

we recommend writing Rust unit tests for the

I personally wouldn't do this. Constructing the Holochain types as test data is not straight forward. You actually need to know quite a lot about how to construct them for this to work and even then, you have to know the inputs to validation well enough to construct good test cases. It can be done, I wouldn't bother.

src/pages/build/callbacks-and-lifecycle-hooks.md

src/pages/build/dht-operations.md

pdaoust · 2025-02-24T17:51:21Z

src/pages/build/dht-operations.md

+    * [`RegisterAgentActivity`](https://docs.rs/holochain_integrity_types/latest/holochain_integrity_types/op/enum.Op.html#variant.RegisterAgentActivity)
+        * Basis address: author's public key
+        * Contents: action (and optionally entry, if applicable)
+        * Effect: Store the action. Because one agent's actions will all go to the same set of authorities, their job is to check for [source chain forks](/resources/glossary/#fork-source-chain), which is a system-level rule that you don't have to implement.


I think I'd like to discuss the scope of this story then -- to me DHT ops seem like quite an advanced topic, at least if you're planning to do something different from what the scaffolded validate callback does. The basic division of labour it sets up seems like a sensible set of defaults to me, probably as much as 80% of hApp devs need.

Moving a discussion of DHT ops to a separate page is something that came out of guidance from @mattyg who advised me that we should focus on the action-based stub functions -- in fact I don't think I should've written this page at all, but it's here now, so... :)

@ThetaSinner What on this page seems like essential, non-advanced things everyone should know about ops and validation? I'd like to move anything like that off this page and onto the validation page.

pdaoust · 2025-02-24T17:53:57Z

src/pages/build/validate-callback.md

+    // `UnresolvedDependencies` outcome for you.
+    let director_entry =  must_get_entry(movie.director_hash)?;
+    // Try to turn it into an entry of the right type.
+    let _director = crate::Director::try_from(director_entry)?;


fixed in 15dae35#diff-6292d0d84462fb21a3681bc06fadd4f6cc525f5bc93015e51e25adf0e0c2b622R92-R94

pdaoust · 2025-02-24T19:00:26Z

src/pages/build/genesis-self-check-callback.md

+
+## Membrane proof: a per-agent joining code for a network
+
+If your network needs to enforce membership control, your [`validate` callback](/build/validate-callback/) can check the contents of the [`AgentValidationPkg`](https://docs.rs/holochain_integrity_types/latest/holochain_integrity_types/action/enum.Action.html#variant.AgentValidationPkg) record. This [**genesis record**](/resources/glossary/#genesis-records) contains data that's entered by the user on app installation. If validators find an invalid proof, they can ban the unauthorized agent from joining the network.


15dae35#diff-c14599996cdb7d4e9167c9e2177448afca23015bcda3b5358cf1cf23560425b2R9

15dae35#diff-c14599996cdb7d4e9167c9e2177448afca23015bcda3b5358cf1cf23560425b2R16

15dae35#diff-c14599996cdb7d4e9167c9e2177448afca23015bcda3b5358cf1cf23560425b2R21

pdaoust · 2025-02-24T19:01:04Z

src/pages/build/genesis-self-check-callback.md

+
+If your network needs to enforce membership control, your [`validate` callback](/build/validate-callback/) can check the contents of the [`AgentValidationPkg`](https://docs.rs/holochain_integrity_types/latest/holochain_integrity_types/action/enum.Action.html#variant.AgentValidationPkg) record. This [**genesis record**](/resources/glossary/#genesis-records) contains data that's entered by the user on app installation. If validators find an invalid proof, they can ban the unauthorized agent from joining the network.
+
+A membrane proof can be basic, like per-agent invite codes, or it can be something complex like a [JSON Web Token (JWT)](https://jwt.io/) signed by an agent that has authority to admit members.


simplified in 15dae35#diff-c14599996cdb7d4e9167c9e2177448afca23015bcda3b5358cf1cf23560425b2R46-L92

pdaoust · 2025-02-24T19:01:25Z

src/pages/build/genesis-self-check-callback.md

+#[derive(Serialize, Deserialize)]
+pub struct MembraneProofJwtPayload {
+    pub iss: AgentPubKeyB64,
+    pub sub: AgentPubKeyB64,


15dae35#diff-c14599996cdb7d4e9167c9e2177448afca23015bcda3b5358cf1cf23560425b2R46-L92

pdaoust · 2025-02-24T19:01:50Z

src/pages/build/validate-callback.md

+2. `ValidateCallbackResult::Invalid(String)` carries information about the validation failure for debugging.
+3. <code>ValidateCallbackResult::UnresolvedDependencies([UnresolvedDependencies](https://docs.rs/holochain_integrity_types/latest/holochain_integrity_types/validate/enum.UnresolvedDependencies.html))</code> means that validation couldn't finish because the required dependencies couldn't be fetched from the DHT.
+
+Outcomes 1 and 2 are definitive and the op won't be validated again. Outcome 3, which happens automatically when a call any of the [`must_get_*` functions](https://docs.rs/hdi/latest/hdi/entry/index.html#functions) yields no data, asks Holochain to try running the op through validation later in the hope that it can retrieve the required dependencies. (It'll give up retrying after some time.)


15dae35#diff-6292d0d84462fb21a3681bc06fadd4f6cc525f5bc93015e51e25adf0e0c2b622R19

pdaoust · 2025-02-24T19:02:26Z

src/pages/build/validate-callback.md

+
+## Create boilerplate code with the scaffolding tool
+
+Even after flattening, there are a lot of variants of an op, which makes for a lot of match arms in your `validate` callback. And a lot of the arms do similar but slightly different things. It's a great idea to let the scaffolding tool generate the callback for you. We won't paste a sample here --- you can scaffold an integrity zome and a few entry and link types and see what it looks like.


15dae35#diff-6292d0d84462fb21a3681bc06fadd4f6cc525f5bc93015e51e25adf0e0c2b622R47-R49 , also includes some removal of redundant prose

pdaoust · 2025-02-24T19:58:11Z

src/pages/build/dht-operations.md

@@ -13,17 +13,21 @@ An agent receives **DHT operations**, which are requests for them to transform [

 ## What is a DHT operation?

-When an agent writes an action, its data gets added to the shared database by being transformed into DHT operations which are sent to various peers in the network. Each operation represents a request to add the action data in a
+When an agent writes an [action](/build/working-with-data/#entries-actions-and-records-primary-data), its data gets added to the shared database by being transformed into multiple DHT operations which the author sends to various [**authorities**](/resources/glossary/#validation-authority) --- agents in the network who have taken responsibility to validate, store, and serve all data that belongs to a range of [**basis addresses**](/resources/glossary/#basis-address). Each operation represents a request to add the action data to the primary content or metadata stored at that address.


missed that this paragraph never got completed.

pdaoust · 2025-02-24T20:01:46Z

src/pages/build/dht-operations.md

+
+### Choosing who should validate what
+
+In practice, it's usually okay to have all groups of authorities validate all the data in an action. However, if your validation logic is highly complex and computationally costly, it can sometimes be useful to choose different validation tasks for the different DHT operations. For instance, a `RegisterAgentActivity` authority may not need to validate entry data; it could just focus on the action in the context of the entire source chain. Or a `StoreEntry` authority could focus on entry contents and leave checking of create or update privileges to `StoreRecord` authorities.


4de1af4 is a partial attempt to respond to the concern, as well as an attempt to untangle some text that seemed unclear.

mattyg · 2025-02-24T23:44:37Z

src/pages/build/validate-callback.md

+
+In this function you can write rules for the contents of your entries and their dependencies. You can also write rules for the actions that write them, which means you can create rules for write privileges. (Note that it's called for both kinds of [`EntryCreationAction`](https://docs.rs/holochain_integrity_types/latest/holochain_integrity_types/op/enum.EntryCreationAction.html) --- `Create` and `Update`.)
+
+This example checks that a movie is within [sensible bounds](https://en.wikipedia.org/wiki/Roundhay_Garden_Scene).


Suggested change

This example checks that a movie is within [sensible bounds](https://en.wikipedia.org/wiki/Roundhay_Garden_Scene).

This example checks that a movie's release date is within sensible bounds.

mattyg · 2025-02-24T23:46:17Z

src/pages/build/validate-callback.md

+) -> ExternResult<ValidateCallbackResult> {
+    // Just call the function, and Holochain will handle the
+    // `UnresolvedDependencies` outcome for you.
+    let director_entry =  must_get_entry(movie.director_hash)?;


Suggested change

let director_entry = must_get_entry(movie.director_hash)?;

let director_entry = must_get_entry(movie.director_hash)?;

mattyg · 2025-02-25T00:06:48Z

src/pages/build/validate-callback.md

+
+Use this function to validate the [**membrane proof**](/build/genesis-self-check-callback/#membrane-proof-a-joining-code-for-a-network). Note that this is different from `genesis_self_check`, in that it's called from the `validate` function so it can access DHT data.
+
+This example implements a simple invite code for a network that people can invite their friends to join. All that's required is the presence of an 'invite' action on the DHT, whose hash becomes the invite code. (It's not a very secure pattern; please don't duplicate this in high-security hApps.) As a bonus, it shows a good pattern where the code for basic pre-validation is shared with `genesis_self_check`.


Suggested change

This example implements a simple invite code for a network that people can invite their friends to join. All that's required is the presence of an 'invite' action on the DHT, whose hash becomes the invite code. (It's not a very secure pattern; please don't duplicate this in high-security hApps.) As a bonus, it shows a good pattern where the code for basic pre-validation is shared with `genesis_self_check`.

This example implements a simple invite code for a network that people can invite their friends to join. All that's required is the presence of an 'invite' action on the DHT, whose hash becomes the invite code. (It's not a very secure pattern; please don't duplicate this in high-security hApps.) Some of the business logic is shared with `genesis_self_check`.

mattyg · 2025-02-25T00:08:38Z

src/pages/build/validate-callback.md

+
+Use this function to validate the [**membrane proof**](/build/genesis-self-check-callback/#membrane-proof-a-joining-code-for-a-network). Note that this is different from `genesis_self_check`, in that it's called from the `validate` function so it can access DHT data.
+
+This example implements a simple invite code for a network that people can invite their friends to join. All that's required is the presence of an 'invite' action on the DHT, whose hash becomes the invite code. (It's not a very secure pattern; please don't duplicate this in high-security hApps.) As a bonus, it shows a good pattern where the code for basic pre-validation is shared with `genesis_self_check`.


Suggested change

This example implements a simple invite code for a network that people can invite their friends to join. All that's required is the presence of an 'invite' action on the DHT, whose hash becomes the invite code. (It's not a very secure pattern; please don't duplicate this in high-security hApps.) As a bonus, it shows a good pattern where the code for basic pre-validation is shared with `genesis_self_check`.

This example implements a simple invite code for a network that people can invite their friends to join. All that's required is the presence of an 'invite' action on the DHT, whose hash becomes the invite code. As a bonus, it shows a good pattern where the code for basic pre-validation is shared with `genesis_self_check`.

Why not just validate that the invitation includes the recipient agent? Would it be secure then, or is there something else missing?

mattyg · 2025-02-25T00:13:49Z

src/pages/build/validate-callback.md

+use hdi::prelude::*;
+use base64::prelude::*;
+
+pub fn validate_agent_joining(


Why does this use functions validate_invite_code_format and decode_invite_code that haven't been defined anywhere?

Also it doesn't look like any of the business logic has been shared with genesis_self_check? Is it supposed to be or am I catching this PR while changes are in-progress?

mattyg · 2025-02-25T00:17:03Z

src/pages/build/validate-callback.md

+* [`holochain_integrity_types::op::Op`](https://docs.rs/holochain_integrity_types/latest/holochain_integrity_types/op/enum.Op.html)
+* [`holochain_integrity_types::action::AgentValidationPkg`](https://docs.rs/holochain_integrity_types/latest/holochain_integrity_types/action/enum.Action.html#variant.AgentValidationPkg)
+* [`holochain_integrity_types::genesis::GenesisSelfCheckData`](https://docs.rs/holochain_integrity_types/latest/holochain_integrity_types/genesis/type.GenesisSelfCheckData.html)
+* [`holochain_integrity_types::validate::ValidateCallbackResult`](https://docs.rs/holochain_integrity_types/latest/holochain_integrity_types/genesis/type.GenesisSelfCheckData.html)


Suggested change

* [`holochain_integrity_types::validate::ValidateCallbackResult`](https://docs.rs/holochain_integrity_types/latest/holochain_integrity_types/genesis/type.GenesisSelfCheckData.html)

* [`holochain_integrity_types::validate::ValidateCallbackResult`](https://docs.rs/holochain_integrity_types/latest/holochain_integrity_types/validate/enum.ValidateCallbackResult.html)

mattyg · 2025-02-25T00:36:01Z

src/pages/build/validation.md

+Validation gives shape to your [DNA](/build/dnas/)'s data model. It defines the 'rules of the game' for a network --- who can create, modify, or delete data, and what that data should and shouldn't look like. It's also the basis for Holochain's peer-auditing security model.
+:::
+
+You implement your validation logic in your application's [integrity zomes](/build/zomes/#integrity). While [entry and link types](/build/working-with-data/) enumerate the kinds of data the integrity zome defines, the data is just bytes until your validation logic gives it some shape and purpose.


Suggested change

You implement your validation logic in your application's [integrity zomes](/build/zomes/#integrity). While [entry and link types](/build/working-with-data/) enumerate the kinds of data the integrity zome defines, the data is just bytes until your validation logic gives it some shape and purpose.

You implement your validation logic in your application's [integrity zomes](/build/zomes/#integrity).

IMO this is more confusing than clarifying.

mattyg · 2025-02-25T00:37:15Z

src/pages/build/validation.md

+1. By an author of data, to protect them from publishing invalid data, and
+2. By an agent that's received data to store and serve, to equip them to detect invalid data and take action against the author.
+
+Because every peer has the DNA's validation logic on their own machine and is expected to check the data they author before they publish it, it can be assumed that invalid data is (almost) always an intentionally malicious act.


Suggested change

Because every peer has the DNA's validation logic on their own machine and is expected to check the data they author before they publish it, it can be assumed that invalid data is (almost) always an intentionally malicious act.

Because every peer has the DNA's validation logic on their own machine and is expected to check the data they author before they publish it, it can be assumed that invalid data is always an intentionally malicious act.

Is there a case where a user can write invalid data unintentionally?

mattyg · 2025-02-25T00:41:51Z

src/pages/build/validation.md

+There are two callbacks that implement validation logic:
+
+* `validate` is the core of the zome's validation logic. It receives a [**DHT operation**](/build/dht-operations/) and returns a success/failure/indeterminate result.
+* [`genesis_self_check`](/build/genesis-self-check-callback/) 'pre-validates' an agent's [**membrane proof**](/concepts/3_source_chain/#source-chain-your-own-data-store) before trying to connect to peers in the network.


Suggested change

* [`genesis_self_check`](/build/genesis-self-check-callback/) 'pre-validates' an agent's [**membrane proof**](/concepts/3_source_chain/#source-chain-your-own-data-store) before trying to connect to peers in the network.

* [`genesis_self_check`](/build/genesis-self-check-callback/) 'pre-validates' your own agent's [**membrane proof**](/concepts/3_source_chain/#source-chain-your-own-data-store) before trying to connect to peers in the network.

Is that clearer?

mattyg · 2025-02-25T00:43:10Z

src/pages/build/validation.md

+
+* The structure of the `Op` type that a `validate` callback receives is complex and deeply nested, and it's best to let the [scaffolding tool](/get-started/3-forum-app-tutorial/) generate the callback for you. It generates stub functions that let you think in terms of [actions](/build/working-with-data/#entries-actions-and-records-primary-data) rather than operations, which is more natural and good enough for most needs. [Read all about DHT operations](/build/dht-operations/) if you want deep detail.
+* While an entry or link can be thought of as 'things', the actions that create, update, or delete them are verbs. Validating a whole action lets you not just check the content and structure of your things, but also enforce write privileges and even throttle an agent's frequency of writes by looking at the action's place in their source chain.
+* Validation rules should **always yield the same true/false outcome** for a given operation regardless of who is validating them and when. Don't use any source of non-determinism, such as instantiating and comparing two `std::time::Instant`s. In fact Holochain prevents your validation callbacks from calling any non-deterministic host functions. Read more about the [available host functions](#available-host-functions).


Suggested change

* Validation rules should **always yield the same true/false outcome** for a given operation regardless of who is validating them and when. Don't use any source of non-determinism, such as instantiating and comparing two `std::time::Instant`s. In fact Holochain prevents your validation callbacks from calling any non-deterministic host functions. Read more about the [available host functions](#available-host-functions).

* Validation rules must **always yield the same true/false outcome** for a given operation regardless of who is validating them and when. Don't use any source of non-determinism, such as instantiating and comparing two `std::time::Instant`s. In fact Holochain prevents your validation callbacks from calling any non-deterministic host functions. Read more about the [available host functions](#available-host-functions).

mattyg · 2025-02-25T00:44:34Z

src/pages/build/validation.md

+* The structure of the `Op` type that a `validate` callback receives is complex and deeply nested, and it's best to let the [scaffolding tool](/get-started/3-forum-app-tutorial/) generate the callback for you. It generates stub functions that let you think in terms of [actions](/build/working-with-data/#entries-actions-and-records-primary-data) rather than operations, which is more natural and good enough for most needs. [Read all about DHT operations](/build/dht-operations/) if you want deep detail.
+* While an entry or link can be thought of as 'things', the actions that create, update, or delete them are verbs. Validating a whole action lets you not just check the content and structure of your things, but also enforce write privileges and even throttle an agent's frequency of writes by looking at the action's place in their source chain.
+* Validation rules should **always yield the same true/false outcome** for a given operation regardless of who is validating them and when. Don't use any source of non-determinism, such as instantiating and comparing two `std::time::Instant`s. In fact Holochain prevents your validation callbacks from calling any non-deterministic host functions. Read more about the [available host functions](#available-host-functions).
+* Data may have dependencies that affect validation outcomes, but those dependencies must be [addressable](/build/identifiers/), they must be retrievable from the same DHT, and their addresses must be constructable from data in the operation being validated. (Note that because an action references the action preceding it on an agent's source chain, this is a dependency you don't need to build into your data.) If dependencies can't be retrieved at validation time, the `validate` callback terminates early with an [indeterminate result](/build/validate-callback/#validation-outcomes), which will cause Holochain to try again later.


Suggested change

* Data may have dependencies that affect validation outcomes, but those dependencies must be [addressable](/build/identifiers/), they must be retrievable from the same DHT, and their addresses must be constructable from data in the operation being validated. (Note that because an action references the action preceding it on an agent's source chain, this is a dependency you don't need to build into your data.) If dependencies can't be retrieved at validation time, the `validate` callback terminates early with an [indeterminate result](/build/validate-callback/#validation-outcomes), which will cause Holochain to try again later.

* Data may have dependencies that affect validation outcomes, but those dependencies must be [addressable](/build/identifiers/), they must be retrievable from the same DHT, and their addresses must be known. If a dependency can't be retrieved at validation time, the `validate` callback terminates early with an [indeterminate result](/build/validate-callback/#validation-outcomes), which will cause Holochain to try again later.

If you want to note that actions include the hash of the prior action, I might put it in another bullet point, or at the end of this one.

mattyg · 2025-02-25T00:52:13Z

src/pages/build/validation.md

+* While an entry or link can be thought of as 'things', the actions that create, update, or delete them are verbs. Validating a whole action lets you not just check the content and structure of your things, but also enforce write privileges and even throttle an agent's frequency of writes by looking at the action's place in their source chain.
+* Validation rules should **always yield the same true/false outcome** for a given operation regardless of who is validating them and when. Don't use any source of non-determinism, such as instantiating and comparing two `std::time::Instant`s. In fact Holochain prevents your validation callbacks from calling any non-deterministic host functions. Read more about the [available host functions](#available-host-functions).
+* Data may have dependencies that affect validation outcomes, but those dependencies must be [addressable](/build/identifiers/), they must be retrievable from the same DHT, and their addresses must be constructable from data in the operation being validated. (Note that because an action references the action preceding it on an agent's source chain, this is a dependency you don't need to build into your data.) If dependencies can't be retrieved at validation time, the `validate` callback terminates early with an [indeterminate result](/build/validate-callback/#validation-outcomes), which will cause Holochain to try again later.
+* When multiple actions are written in a same [atomic transaction](/build/zome-functions/#atomic-transactional-commits), actions' ops can only have dependencies on prior actions committed in the transaction, not later actions.


Suggested change

* When multiple actions are written in a same [atomic transaction](/build/zome-functions/#atomic-transactional-commits), actions' ops can only have dependencies on prior actions committed in the transaction, not later actions.

* Despite that multiple actions can be written within an [atomic transaction](/build/zome-functions/#atomic-transactional-commits), they are *not* validated together as an atomic transaction. An action can only have dependencies on prior actions, not subsequent actions.

mattyg · 2025-02-25T00:53:40Z

src/pages/build/validation.md

+* Data may have dependencies that affect validation outcomes, but those dependencies must be [addressable](/build/identifiers/), they must be retrievable from the same DHT, and their addresses must be constructable from data in the operation being validated. (Note that because an action references the action preceding it on an agent's source chain, this is a dependency you don't need to build into your data.) If dependencies can't be retrieved at validation time, the `validate` callback terminates early with an [indeterminate result](/build/validate-callback/#validation-outcomes), which will cause Holochain to try again later.
+* When multiple actions are written in a same [atomic transaction](/build/zome-functions/#atomic-transactional-commits), actions' ops can only have dependencies on prior actions committed in the transaction, not later actions.
+* You don't need to validate your data manually before committing --- Holochain [validates it after the zome function that writes it is finished](/build/zome-functions/#validate-dht-operations), and returns any validation failure to the caller.
+* **Test, test, test.** Validation is the gate that accepts or rejects all DHT data, so make sure you write thorough test coverage for your validation functions. If the data being validated has no dependencies on DHT data or DNA/zome info, we recommend writing Rust unit tests for the [validation function stubs](/build/validate-callback/#create-boilerplate-code-with-the-scaffolding-tool) that the scaffolding tool generates. We also recommend testing your validation code by writing single- and multiple-agent [Tryorama](https://github.com/holochain/tryorama/) test scenarios for zome functions that write data (we'll write about Tryorama soon; in the meantime, you can check the Tryorama GitHub readme and the scaffolded tests in a project's `tests/src/` folder for guidance).<!-- TODO: link to Tryorama page -->


Suggested change

* **Test, test, test.** Validation is the gate that accepts or rejects all DHT data, so make sure you write thorough test coverage for your validation functions. If the data being validated has no dependencies on DHT data or DNA/zome info, we recommend writing Rust unit tests for the [validation function stubs](/build/validate-callback/#create-boilerplate-code-with-the-scaffolding-tool) that the scaffolding tool generates. We also recommend testing your validation code by writing single- and multiple-agent [Tryorama](https://github.com/holochain/tryorama/) test scenarios for zome functions that write data (we'll write about Tryorama soon; in the meantime, you can check the Tryorama GitHub readme and the scaffolded tests in a project's `tests/src/` folder for guidance).

* **Test, test, test.** Validation is the gate that accepts or rejects all DHT data, so make sure you write thorough test coverage for your validation functions. We recommend testing your validation code by writing [Tryorama](https://github.com/holochain/tryorama/) test scenarios for zome functions that write data (we'll write about Tryorama soon; in the meantime, you can check the Tryorama GitHub readme and the scaffolded tests in a project's `tests/src/` folder for guidance).

mattyg · 2025-02-25T01:02:59Z

src/pages/resources/glossary.md

+#### Genesis
+
+The time period in the lifecycle of a [cell](#cell) in which [genesis records](#genesis-records) are being written. This happens before the cell has access to its [DNA](#dna)'s network and [DHT](#distributed-hash-table-dht) data. A [genesis self-check callback](#genesis-self-check-callback) can be written to pre-validate the [membrane proof](#membrane-proof) during this period.
+
 #### Genesis records

 The four records at the beginning of an [agent's](#agent) [source chain](#source-chain), consisting of:


Suggested change

The four records at the beginning of an [agent's](#agent) [source chain](#source-chain), consisting of:

The three records at the beginning of an [agent's](#agent) [source chain](#source-chain), consisting of:

Or am I missing something?

ThetaSinner · 2025-02-25T11:29:30Z

src/pages/build/validation.md

+* Data may have dependencies that affect validation outcomes, but those dependencies must be [addressable](/build/identifiers/), they must be retrievable from the same DHT, and their addresses must be constructable from data in the operation being validated. (Note that because an action references the action preceding it on an agent's source chain, this is a dependency you don't need to build into your data.) If dependencies can't be retrieved at validation time, the `validate` callback terminates early with an [indeterminate result](/build/validate-callback/#validation-outcomes), which will cause Holochain to try again later.
+* When multiple actions are written in a same [atomic transaction](/build/zome-functions/#atomic-transactional-commits), actions' ops can only have dependencies on prior actions committed in the transaction, not later actions.
+* You don't need to validate your data manually before committing --- Holochain [validates it after the zome function that writes it is finished](/build/zome-functions/#validate-dht-operations), and returns any validation failure to the caller.
+* **Test, test, test.** Validation is the gate that accepts or rejects all DHT data, so make sure you write thorough test coverage for your validation functions. If the data being validated has no dependencies on DHT data or DNA/zome info, we recommend writing Rust unit tests for the [validation function stubs](/build/validate-callback/#create-boilerplate-code-with-the-scaffolding-tool) that the scaffolding tool generates. We also recommend testing your validation code by writing single- and multiple-agent [Tryorama](https://github.com/holochain/tryorama/) test scenarios for zome functions that write data (we'll write about Tryorama soon; in the meantime, you can check the Tryorama GitHub readme and the scaffolded tests in a project's `tests/src/` folder for guidance).<!-- TODO: link to Tryorama page -->


Suggested change

* **Test, test, test.** Validation is the gate that accepts or rejects all DHT data, so make sure you write thorough test coverage for your validation functions. If the data being validated has no dependencies on DHT data or DNA/zome info, we recommend writing Rust unit tests for the [validation function stubs](/build/validate-callback/#create-boilerplate-code-with-the-scaffolding-tool) that the scaffolding tool generates. We also recommend testing your validation code by writing single- and multiple-agent [Tryorama](https://github.com/holochain/tryorama/) test scenarios for zome functions that write data (we'll write about Tryorama soon; in the meantime, you can check the Tryorama GitHub readme and the scaffolded tests in a project's `tests/src/` folder for guidance).

* **Test, test, test.** Validation is the gate that accepts or rejects all DHT data, so make sure you write thorough test coverage for your validation functions. If the data being validated has no dependencies on DHT data or DNA/zome info, we recommend writing Rust unit tests for the [validation function stubs](/build/validate-callback/#create-boilerplate-code-with-the-scaffolding-tool) that the scaffolding tool generates. We also recommend testing your validation code by writing single- and multiple-agent [Tryorama](https://github.com/holochain/tryorama/) test scenarios for zome functions that write data. This lets you check that your validation rules pass both when authoring data and checking data authored by other agents. (we'll write about Tryorama soon; in the meantime, you can check the Tryorama GitHub readme and the scaffolded tests in a project's `tests/src/` folder for guidance).

Keen to include the why there. We can skip going into detail about how

pdaoust added 30 commits January 15, 2025 13:33

lifecycle callbacks and zome functions pages

f154488

add new pages to navs

8fc90cd

link up all references to two new pages

ae585c0

remove redundant descriptions of lifecycles, add examples for relaxed

e20390e

ordering

add under-the-hood for CreateLink and DeleteLink

f16a4a5

add snapshotted to dict

cd3eee3

fix broken JSON

2ccb02d

fix broken links

5cb51e1

fix hdk_entry_defs, whoops

60cf346

add example for post_commit

060a7d8

simplify/elaborate post_commit example

1010a44

Merge branch 'feat/guide/app-structure-zomes' into feat/guide/app-str…

39813c0

…ucture-zome-functions

improve language around callbacks and lifecycle hooks, plus a couple …

5103988

…tiny edits

little bit more of the same

0cc2fca

test/fix all code samples in callbacks page

a1c3af3

small edits to callbacks page

26b63dd

link from identifiers to post_commit page (plus a typo fix)

e2c3f7b

reference/further reading for zomes and callbacks pages

2a74d82

Merge branch 'feat/guide/app-structure-zome-functions' into feat/guid…

f7d3638

…e/app-structure-dnas

DNAs page

05fbd84

add DNAs page to nav and other references

445e799

mention using a Cargo workspace

f618460

try to fix JSON syntax error which doesn't appear to exist

a8e63fb

fix broken URLs and a typo

994c997

WIP

12c7985

add section to DNAs on remote calling

854c45b

Merge branch 'feat/guide/app-structure-dnas' into feat/guide/app-stru…

7ecf6d5

…cture-happs

add DNAs and hApps to build guide overview

cf2afe5

hApps page draft

b948adb

change note about clone_limit

45d90ba

pdaoust added 2 commits February 21, 2025 14:36

edit: typo

6856d88

edit: mention testing in the context of validation

4b2fb15

pdaoust force-pushed the feat/guide/validation branch from 91e9a4c to e6e4ecc Compare February 21, 2025 22:51

edit: talk more about Tryorama testing

0d0adac

pdaoust force-pushed the feat/guide/validation branch from e6e4ecc to 0d0adac Compare February 21, 2025 22:53

edit: warn people against doing non-deterministic things in validation

b19615c

pdaoust commented Feb 24, 2025

View reviewed changes

pdaoust added 2 commits February 24, 2025 11:04

edit: use UNIX timestamps rather than date strings

afe9fe1

edit: try to improve wording re: who validates what

4de1af4

pdaoust commented Feb 24, 2025

View reviewed changes

edit: fix/improve glossary entry on basis address s/base/basis/

e04adec

mattyg reviewed Feb 24, 2025

View reviewed changes

mattyg reviewed Feb 25, 2025

View reviewed changes

ThetaSinner reviewed Feb 25, 2025

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Build guide: validation #520

Build guide: validation #520

pdaoust commented Feb 4, 2025 •

edited

Loading

ThetaSinner commented Feb 21, 2025

ThetaSinner commented Feb 21, 2025

pdaoust commented Feb 21, 2025

pdaoust commented Feb 24, 2025

pdaoust Feb 21, 2025

ThetaSinner Feb 25, 2025

pdaoust Feb 24, 2025

pdaoust Feb 24, 2025

pdaoust Feb 24, 2025

pdaoust Feb 24, 2025

pdaoust Feb 24, 2025

pdaoust Feb 24, 2025

pdaoust Feb 24, 2025

pdaoust Feb 24, 2025

pdaoust Feb 24, 2025

mattyg Feb 24, 2025 •

edited

Loading

mattyg Feb 24, 2025

mattyg Feb 25, 2025

mattyg Feb 25, 2025

mattyg Feb 25, 2025 •

edited

Loading

mattyg Feb 25, 2025

mattyg Feb 25, 2025

mattyg Feb 25, 2025

mattyg Feb 25, 2025

mattyg Feb 25, 2025

mattyg Feb 25, 2025 •

edited

Loading

mattyg Feb 25, 2025

mattyg Feb 25, 2025

mattyg Feb 25, 2025

ThetaSinner Feb 25, 2025

ThetaSinner Feb 25, 2025


		## Membrane proof: a per-agent joining code for a network

		If your network needs to enforce membership control, your [`validate` callback](/build/validate-callback/) can check the contents of the [`AgentValidationPkg`](https://docs.rs/holochain_integrity_types/latest/holochain_integrity_types/action/enum.Action.html#variant.AgentValidationPkg) record. This [genesis record](/resources/glossary/#genesis-records) contains data that's entered by the user on app installation. If validators find an invalid proof, they can ban the unauthorized agent from joining the network.


		If your network needs to enforce membership control, your [`validate` callback](/build/validate-callback/) can check the contents of the [`AgentValidationPkg`](https://docs.rs/holochain_integrity_types/latest/holochain_integrity_types/action/enum.Action.html#variant.AgentValidationPkg) record. This [genesis record](/resources/glossary/#genesis-records) contains data that's entered by the user on app installation. If validators find an invalid proof, they can ban the unauthorized agent from joining the network.

		A membrane proof can be basic, like per-agent invite codes, or it can be something complex like a [JSON Web Token (JWT)](https://jwt.io/) signed by an agent that has authority to admit members.


		## Create boilerplate code with the scaffolding tool

		Even after flattening, there are a lot of variants of an op, which makes for a lot of match arms in your `validate` callback. And a lot of the arms do similar but slightly different things. It's a great idea to let the scaffolding tool generate the callback for you. We won't paste a sample here --- you can scaffold an integrity zome and a few entry and link types and see what it looks like.


		### Choosing who should validate what

		In practice, it's usually okay to have all groups of authorities validate all the data in an action. However, if your validation logic is highly complex and computationally costly, it can sometimes be useful to choose different validation tasks for the different DHT operations. For instance, a `RegisterAgentActivity` authority may not need to validate entry data; it could just focus on the action in the context of the entire source chain. Or a `StoreEntry` authority could focus on entry contents and leave checking of create or update privileges to `StoreRecord` authorities.


		In this function you can write rules for the contents of your entries and their dependencies. You can also write rules for the actions that write them, which means you can create rules for write privileges. (Note that it's called for both kinds of [`EntryCreationAction`](https://docs.rs/holochain_integrity_types/latest/holochain_integrity_types/op/enum.EntryCreationAction.html) --- `Create` and `Update`.)

		This example checks that a movie is within [sensible bounds](https://en.wikipedia.org/wiki/Roundhay_Garden_Scene).

	This example checks that a movie is within [sensible bounds](https://en.wikipedia.org/wiki/Roundhay_Garden_Scene).
	This example checks that a movie's release date is within sensible bounds.

	let director_entry = must_get_entry(movie.director_hash)?;
	let director_entry = must_get_entry(movie.director_hash)?;


		Use this function to validate the [membrane proof](/build/genesis-self-check-callback/#membrane-proof-a-joining-code-for-a-network). Note that this is different from `genesis_self_check`, in that it's called from the `validate` function so it can access DHT data.

		This example implements a simple invite code for a network that people can invite their friends to join. All that's required is the presence of an 'invite' action on the DHT, whose hash becomes the invite code. (It's not a very secure pattern; please don't duplicate this in high-security hApps.) As a bonus, it shows a good pattern where the code for basic pre-validation is shared with `genesis_self_check`.

	* [`holochain_integrity_types::validate::ValidateCallbackResult`](https://docs.rs/holochain_integrity_types/latest/holochain_integrity_types/genesis/type.GenesisSelfCheckData.html)
	* [`holochain_integrity_types::validate::ValidateCallbackResult`](https://docs.rs/holochain_integrity_types/latest/holochain_integrity_types/validate/enum.ValidateCallbackResult.html)

	You implement your validation logic in your application's [integrity zomes](/build/zomes/#integrity). While [entry and link types](/build/working-with-data/) enumerate the kinds of data the integrity zome defines, the data is just bytes until your validation logic gives it some shape and purpose.
	You implement your validation logic in your application's [integrity zomes](/build/zomes/#integrity).

	Because every peer has the DNA's validation logic on their own machine and is expected to check the data they author before they publish it, it can be assumed that invalid data is (almost) always an intentionally malicious act.
	Because every peer has the DNA's validation logic on their own machine and is expected to check the data they author before they publish it, it can be assumed that invalid data is always an intentionally malicious act.

	* [`genesis_self_check`](/build/genesis-self-check-callback/) 'pre-validates' an agent's [membrane proof](/concepts/3_source_chain/#source-chain-your-own-data-store) before trying to connect to peers in the network.
	* [`genesis_self_check`](/build/genesis-self-check-callback/) 'pre-validates' your own agent's [membrane proof](/concepts/3_source_chain/#source-chain-your-own-data-store) before trying to connect to peers in the network.

	* When multiple actions are written in a same [atomic transaction](/build/zome-functions/#atomic-transactional-commits), actions' ops can only have dependencies on prior actions committed in the transaction, not later actions.
	* Despite that multiple actions can be written within an [atomic transaction](/build/zome-functions/#atomic-transactional-commits), they are not validated together as an atomic transaction. An action can only have dependencies on prior actions, not subsequent actions.

	The four records at the beginning of an [agent's](#agent) [source chain](#source-chain), consisting of:
	The three records at the beginning of an [agent's](#agent) [source chain](#source-chain), consisting of:

Build guide: validation #520

Are you sure you want to change the base?

Build guide: validation #520

Conversation

pdaoust commented Feb 4, 2025 • edited Loading

ThetaSinner commented Feb 21, 2025

ThetaSinner commented Feb 21, 2025

pdaoust commented Feb 21, 2025

pdaoust commented Feb 24, 2025

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mattyg Feb 24, 2025 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mattyg Feb 25, 2025 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mattyg Feb 25, 2025 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

pdaoust commented Feb 4, 2025 •

edited

Loading

mattyg Feb 24, 2025 •

edited

Loading

mattyg Feb 25, 2025 •

edited

Loading

mattyg Feb 25, 2025 •

edited

Loading