Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update tutorials #344

Merged
merged 35 commits into from
Feb 24, 2025
Merged
Changes from 1 commit
Commits
Show all changes
35 commits
Select commit Hold shift + click to select a range
e5070fc
Rename to archer2 tutorial so we can add others
tkoskela Sep 20, 2024
fd9a277
Add raw markdown from durham tutorial
tkoskela Sep 20, 2024
50466d7
Add reframe tutorial to navigation
tkoskela Sep 20, 2024
402833c
Remove hackmd formatting
tkoskela Sep 20, 2024
83f650e
Update tutorial links
tkoskela Sep 20, 2024
db85a8a
Add some comments
tkoskela Sep 20, 2024
edf276d
Fix headings
tkoskela Sep 20, 2024
e730c23
It's more logical to start with the ReFrame tutorial
tkoskela Sep 20, 2024
7f961b8
Update docs/tutorial/durham_reframe_tutorial.md
tkoskela Sep 23, 2024
8ed358e
Apply suggestions from code review
tkoskela Sep 23, 2024
6cf5053
Address review commits
tkoskela Sep 23, 2024
2362c96
Merge branch 'tk/add-cosma-tutorial' of github.com:ukri-excalibur/exc…
tkoskela Sep 23, 2024
99b2400
Try adding another space
tkoskela Sep 23, 2024
869f8e8
One more space
tkoskela Sep 23, 2024
d669be9
Remove duplicate and add note about platform specific stuff
tkoskela Sep 23, 2024
4ef37cc
Rewrite part on perflogs, add link to excalibur-tests.
tkoskela Sep 23, 2024
0fd3565
Add note on how to run
tkoskela Sep 23, 2024
88a52a1
Merge branch 'main' into tk/add-cosma-tutorial
tkoskela Dec 16, 2024
1b6d738
Test tabs
tkoskela Dec 16, 2024
e86c207
Update with ARCHER2 version
tkoskela Dec 16, 2024
2d939b7
WIP, renders tabs
tkoskela Dec 16, 2024
b02ceab
Update python
tkoskela Dec 16, 2024
ca9b889
Add more verbose text, tabs for system specifics
tkoskela Dec 17, 2024
a84a01d
Rename tutorials with more descriptive names
tkoskela Dec 17, 2024
e9bf04a
Fix warnings
tkoskela Dec 17, 2024
185861f
Fix indentation
tkoskela Dec 17, 2024
ab5ebac
Restructure tutorials
tkoskela Dec 18, 2024
b63883d
Some refactoring of duplicate content
tkoskela Dec 18, 2024
25df451
Add dependency
tkoskela Dec 18, 2024
4775a74
Add profiling tutorial
tkoskela Dec 18, 2024
fcc121d
Add tabs to getting started page
tkoskela Dec 18, 2024
6f43d06
Add tabs to MFA section
tkoskela Dec 18, 2024
78fbeae
Address review comments
tkoskela Jan 31, 2025
ce213f9
Add section on spack cache and local data paths
tkoskela Feb 10, 2025
9a2ed42
Merge branch 'main' into tk/add-cosma-tutorial
tkoskela Feb 11, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
WIP, renders tabs
  • Loading branch information
tkoskela committed Dec 16, 2024

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature.
commit 2d939b7ac34574e5c70d086bbc947fd7423f5ce5
110 changes: 62 additions & 48 deletions docs/tutorial/durham_reframe_tutorial.md
Original file line number Diff line number Diff line change
@@ -27,29 +27,27 @@ You can customise the behaviour of each stage or add a hook before or after each

---

## Set up environment
## Set up python environment

=== "Cosma"

This tutorial was originally run on the [Cosma](https://cosma.readthedocs.io/en/latest/) supercomputer.
This tutorial is run on the [Cosma](https://cosma.readthedocs.io/en/latest/) supercomputer.
It should be straightforward to run on a different platform, the requirements are `gcc`, `git` and `python3`. (for the later parts you also need `make`, `autotools`, `cmake` and `spack`).
Before proceeding to install ReFrame, we recommend creating a python virtual environment to avoid clashes with other installed python packages.
First load a newer python module.

```bash
module swap python/3.10.12
```
Before proceeding to install ReFrame, we recommend creating a python virtual environment to avoid clashes with other installed python packages.
First load a newer python module.
```bash
module swap python/3.10.12
```

=== "ARCHER2"

This tutorial is run on ARCHER2, you should have signed up for a training account before starting.
It can be ran on other HPC systems with a batch scheduler but will require making some changes to the config.
Before proceeding to install ReFrame, we recommend creating a python virtual environment to avoid clashes with other installed python packages.
First load the system python module.

```bash
module load cray-python
```
This tutorial is run on ARCHER2, you should have signed up for a training account before starting.
It can be ran on other HPC systems with a batch scheduler but will require making some changes to the config.
Before proceeding to install ReFrame, we recommend creating a python virtual environment to avoid clashes with other installed python packages.
First load the system python module.
```bash
module load cray-python
```

Then create an environment and activate it with

@@ -173,30 +171,21 @@ compiling a C++ or Fortran code will fail
## Configuring ReFrame for HPC systems
> In ReFrame, all the details of the various interactions of a test with the system environment are handled transparently and are set up in its [configuration file](https://reframe-hpc.readthedocs.io/en/v4.5.2/tutorial_basics.html#more-of-hello-world).

=== "Cosma"

To configure ReFrame for Cosma
- Create a system with a name and a description
- Set the module system
- Create a compute node partition
- Set a scheduler and a MPI launcher to run on compute nodes
- On Cosma, the scheduler rejects jobs that don't set a time limit. Add `time_limit = 1m` to ReFrame tests to run on Cosma or set from command line with `-S time_limit='1m'`
- Set accounting parameters with `'access': ['--partition=bluefield1', '--account=do009'],`
- Create at least one programming environment to set compilers

=== "ARCHER2"
For the minimum configuration to run jobs on the system we need to

To configure ReFrame for ARCHER2
- Create a system with a name and a description
- Set the module system
- Create a compute node partition
- Set a scheduler and a MPI launcher to run on compute nodes
- Set accounting parameters with `'access': ['--partition=standard', '--qos=short'],`
- Create at least one programming environment to set compilers
=== "Cosma"

----
- Create a system with a name and a description
- Set the module system for accessing centrally installed modules
- Create a compute node partition
- Set a scheduler and a MPI launcher to run on compute nodes
- On Cosma, the scheduler rejects jobs that don't set a time limit. Add `time_limit = 1m` to ReFrame tests to run on Cosma or set from command line with `-S time_limit='1m'`
- Set access options
```
'access': ['--partition=bluefield1', '--account=do009'],
```
- Create at least one programming environment to set compilers

=== "Cosma"
```python
site_configuration = {
'systems' : [
@@ -226,10 +215,20 @@ compiling a C++ or Fortran code will fail
},
]
}
```
```

=== "ARCHER2"

- Create a system with a name and a description
- Set the module system for accessing centrally installed modules
- Create a compute node partition
- Set a scheduler and a MPI launcher to run on compute nodes
- Set access options with
```
'access': ['--partition=standard', '--qos=short'],
```
- Create at least one programming environment to set compilers

```python
site_configuration = {
'systems' : [
@@ -257,7 +256,7 @@ compiling a C++ or Fortran code will fail
},
]
}
```
```

---

@@ -269,17 +268,32 @@ Performance tests capture data in performance variables. For simplicity, we use

### Boilerplate

Same as before. We can now specify valid systems and prog environments on Cosma. You can adapt these to your platform, or use `'*'` to run on any platform.
Same as before. We can now specify valid systems and programming environments to run on the system we just configured.
You can adapt these to your system, or keep using `'*'` to run on any platform.

```python
import reframe as rfm
import reframe.utility.sanity as sn

@rfm.simple_test
class StreamTest(rfm.RegressionTest):
valid_systems = ['cosma']
valid_prog_environs = ['gnu', 'intel']
```
=== "Cosma"
```python
import reframe as rfm
import reframe.utility.sanity as sn

@rfm.simple_test
class StreamTest(rfm.RegressionTest):
valid_systems = ['cosma']
valid_prog_environs = ['gnu']
```

=== "ARCHER2"

```python
import reframe as rfm
import reframe.utility.sanity as sn

@rfm.simple_test
class StreamTest(rfm.RegressionTest):
valid_systems = ['archer2']
valid_prog_environs = ['cray']
```

----

3 changes: 2 additions & 1 deletion mkdocs.yml
Original file line number Diff line number Diff line change
@@ -70,4 +70,5 @@ markdown_extensions:
- pymdownx.inlinehilite
- pymdownx.snippets
- pymdownx.superfences
- pymdownx.tabbed
- pymdownx.tabbed:
alternate_style: true
Loading