Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add new check description to F/C test format #19

Merged
merged 7 commits into from
Jul 2, 2024
Merged
Show file tree
Hide file tree
Changes from 5 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions tests/formats/fork_choice/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -177,6 +177,10 @@ finalized_checkpoint: {
root: string, -- Encoded 32-byte value from store.finalized_checkpoint.root
}
proposer_boost_root: string -- Encoded 32-byte value from store.proposer_boost_root
viable_for_head_roots_and_weights: {
root: string, -- Encoded 32-byte value of filtered_block_tree leaf blocks
weight: int -- Integer value from get_weight(store, viable_block_root)
}
ericsson49 marked this conversation as resolved.
Show resolved Hide resolved
```

Additionally, these fields if `get_proposer_head` and `should_override_forkchoice_update` features are implemented:
Expand Down
42 changes: 42 additions & 0 deletions tests/generators/fork_choice_generated/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,3 +7,45 @@ Implementation of the approach described in the [Fork Choice compliance testing
Preliminary research has been also performed in this [repo](https://github.com/txrx-research/fork_choice_test_generation/tree/main).

To simplfy adoption of the tests, we follow the test format described in the [fork choice test formats documentation](../../formats/fork_choice/README.md), with a minor exception (new check added).

# Pre-requisites

Install requirements (preferrably, in a dedicated Python environment)

```
> pip install -r requirements.txt
```

In order to run tests, install `tqdm` additionally.
```
> pip install tqdm
```

# Generating tests

```
> python test_gen.py -o ${test_dir} --fc-gen-config ${config_dir}/test_gen.yaml
```

There are three configurations in the repo: [tiny](tiny/), [small](small/) and [standard](standard/).

# Running tests

Install `tqdm` library (to show progress)
```
> pip install tqdm
```

and then
```
> python test_run.py -i ${test_dir}
```

# Generating configurations

Files in [tiny](tiny/), [small](small/) and [standard](standard/) are generated with [generate_test_instances.py](generate_test_instances.py), e.g.
```
> python generate_test_instances.py
```

But one normally doesn't need to generate them.
32 changes: 16 additions & 16 deletions tests/generators/fork_choice_generated/generate_test_instances.py
Original file line number Diff line number Diff line change
Expand Up @@ -115,26 +115,26 @@ def generate_block_cover(params):

gen_params = {
###################
# small instances #
# tiny instances #
###################

'block_tree_tree_small': {
'out_path': 'small/block_tree_tree.yaml',
'block_tree_tree_tiny': {
'out_path': 'tiny/block_tree_tree.yaml',
'models': ['sm_link', 'block_tree'],
'params': [
({'anchor_epoch': 0, 'number_of_epochs': 4, 'number_of_links': 3}, {'number_of_blocks': 8, 'max_children': 2, 'number_of_solutions': 3}),
([{'sm_links': [[0, 1], [0, 2], [2, 3], [3, 4]]}], {'number_of_blocks': 16, 'max_children': 2, 'number_of_solutions': 3}),
]
},
'block_tree_other_small': {
'out_path': 'small/block_tree_other.yaml',
'block_tree_other_tiny': {
'out_path': 'tiny/block_tree_other.yaml',
'models': ['sm_link', 'block_tree'],
'params': [
([{'sm_links': [[0, 1], [0, 2], [2, 3], [3, 4]]}], {'number_of_blocks': 12, 'max_children': 2, 'number_of_solutions': 3}),
]
},
'block_cover_small': {
'out_path': 'small/block_cover.yaml',
'block_cover_tiny': {
'out_path': 'tiny/block_cover.yaml',
'models': ['block_cover'],
'params': [
({'anchor_epoch': 0, 'number_of_solutions': 1},),
Expand All @@ -143,36 +143,36 @@ def generate_block_cover(params):
},

###################
# smoke instances #
# small instances #
###################

'block_tree_tree_smoke': {
'out_path': 'smoke/block_tree_tree.yaml',
'block_tree_tree_small': {
'out_path': 'small/block_tree_tree.yaml',
'models': ['sm_link', 'block_tree'],
'params': [
({'anchor_epoch': 0, 'number_of_epochs': 5, 'number_of_links': 3}, {'number_of_blocks': 16, 'max_children': 2, 'number_of_solutions': 2}),
([{'sm_links': [[0, 1], [0, 2], [2, 3], [3, 4]]}], {'number_of_blocks': 5, 'max_children': 3, 'number_of_solutions': None}),
([{'sm_links': [[0, 1], [0, 2], [2, 3], [3, 4]]}], {'number_of_blocks': 12, 'max_children': 2, 'number_of_solutions': 73}),
]
},
'block_tree_tree_smoke_2': {
'out_path': 'smoke/block_tree_tree_2.yaml',
'block_tree_tree_small_2': {
'out_path': 'small/block_tree_tree_2.yaml',
'models': ['sm_link', 'block_tree'],
'params': [
({'anchor_epoch': 0, 'number_of_epochs': 6, 'number_of_links': 4}, {'number_of_blocks': 16, 'max_children': 2, 'number_of_solutions': 2}),
([{'sm_links': [[0, 1], [0, 2], [2, 3], [3, 4]]}], {'number_of_blocks': 6, 'max_children': 4, 'number_of_solutions': None}),
([{'sm_links': [[0, 1], [0, 2], [2, 3], [3, 4]]}], {'number_of_blocks': 12, 'max_children': 2, 'number_of_solutions': 283}),
]
},
'block_tree_other_smoke': {
'out_path': 'smoke/block_tree_other.yaml',
'block_tree_other_small': {
'out_path': 'small/block_tree_other.yaml',
'models': ['sm_link', 'block_tree'],
'params': [
([{'sm_links': [[0, 1], [0, 2], [2, 3], [3, 4]]}], {'number_of_blocks': 12, 'max_children': 2, 'number_of_solutions': 4}),
]
},
'block_cover_smoke': {
'out_path': 'smoke/block_cover.yaml',
'block_cover_small': {
'out_path': 'small/block_cover.yaml',
'models': ['block_cover'],
'params': [
({'anchor_epoch': 0, 'number_of_solutions': 2},),
Expand Down
Loading