|
| 1 | +LSST Rust Development Guide |
| 2 | +=========================== |
| 3 | + |
| 4 | +This document outlines the guidelines for developing Rust code within the LSST (Large Synoptic Survey Telescope) DM (Data Management) Stack. It’s designed to complement existing Python and C++ development guides and ensure consistency across the project. |
| 5 | +This guide is a living document and will be updated as needed. Please provide feedback and suggestions to help improve it. |
| 6 | + |
| 7 | +1. Introduction |
| 8 | +--------------- |
| 9 | + |
| 10 | +This guide details the standards and best practices for writing Rust code within the LSST DM Stack and includes basic knowledge of the package. Rust is being adopted to provide performance-critical components and leverage its memory safety features. This document assumes a basic understanding of Rust and the LSST DM Stack. |
| 11 | + |
| 12 | +2. Rust Version |
| 13 | +--------------- |
| 14 | + |
| 15 | +All LSST Rust code must be compatible with the standard Rust toolchain supported by the LSST DM Stack as provided in the conda environment. Specific versions will be maintained and announced through the standard LSST communication channels. |
| 16 | + |
| 17 | +3. Resources |
| 18 | +------------ |
| 19 | + |
| 20 | +* The `Rust Book <https://doc.rust-lang.org/book/>`_ is a good resource to find out information on the rust language. |
| 21 | +* `crates.io <https://crates.io/>`_ is where third party crates (libraries) are stored and distributed. This contains links to the documentation on each crate. |
| 22 | +* While knowledge of this is not needed by most developers, the `cargo book <https://doc.rust-lang.org/cargo/>`_ may be useful for any obscure packaging issues. |
| 23 | +* Likewise the python/rust build system can be found at `maturin <https://www.maturin.rs/>`_ |
| 24 | +* Documentation for the python binding engine PyO3 can be found `here <https://docs.rs/pyo3/latest/pyo3/>`_ |
| 25 | + |
| 26 | +4. Code Organization |
| 27 | +-------------------- |
| 28 | + |
| 29 | +Unlike the package-centric organization often seen in Python, Rust code within the LSST DM Stack should be organized by functionality. This promotes code reuse and maintainability. |
| 30 | +* Top-Level Module: All Rust code will be bound to a single top-level module. This module will serve as the entry point for Python interaction. If for some reason independent repos are needed they must first be proposed in an RFC and will adhear otherwise to what is in this guide. |
| 31 | +* Functional Modules: Within this top-level module, create sub-modules representing distinct functionalities (e.g., image processing, coordinate transformations, data structures). |
| 32 | +* Avoid Package-Specific Structure: Do not mirror the Python package structure in your Rust organization. Focus on logical groupings of functionality. |
| 33 | + |
| 34 | +5. Python Interoperability |
| 35 | +-------------------------- |
| 36 | + |
| 37 | +All Rust code intended for use within the LSST DM Stack must be exposed to Python via a well-defined API. This is achieved using the `pyO3`_ bindings. |
| 38 | +* `pyO3`_ is Mandatory: `pyO3`_ is the only supported mechanism for exposing Rust functionality to Python. |
| 39 | +* API Design: Carefully design your Python API to be intuitive and consistent with existing LSST Python interfaces. |
| 40 | +* Documentation: Thoroughly document your Python API using docstrings. Docstrings are written as doc comments in the rust and are automatically translated to python docstrings by pyO3. Doc strings should be written in the same numpydoc format as specified in the python section of the devguide. |
| 41 | + |
| 42 | +6. Dependencies |
| 43 | +--------------- |
| 44 | + |
| 45 | +Managing dependencies is crucial for maintaining a stable and reproducible build environment. |
| 46 | +* RFC Process: All new Rust dependencies added to the Cargo.toml file must be approved via the LSST RFC (Request for Comments) process. This ensures that dependencies are vetted for licensing, security, and long-term maintainability. |
| 47 | +* Dependency Versions: Pin dependency versions in Cargo.toml to ensure reproducible builds. |
| 48 | +* Minimal Dependencies: Strive to minimize the number of dependencies to reduce build times and potential conflicts. |
| 49 | + |
| 50 | +7. Testing |
| 51 | +---------- |
| 52 | + |
| 53 | +Robust testing is essential for ensuring the quality and reliability of the LSST DM Stack. |
| 54 | +Python Unit Tests: All Rust code must be tested via Python unit tests. This provides a consistent testing framework and leverages the existing LSST testing infrastructure. |
| 55 | +pytest: Use pytest as the testing framework. |
| 56 | +Import Rust Modules: Python tests should import the Rust modules (exposed via `pyO3`_) and exercise their functionality. |
| 57 | +Comprehensive Coverage: Strive for high test coverage to ensure that all critical code paths are tested. |
| 58 | +Integration Tests: In addition to unit tests, consider integration tests to verify the interaction between Rust components and other parts of the LSST DM Stack. |
| 59 | + |
| 60 | +8. Code Style and Formatting |
| 61 | +---------------------------- |
| 62 | + |
| 63 | +Consistent code style improves readability and maintainability. |
| 64 | + |
| 65 | +* `rustfmt`_: Use rustfmt to automatically format your Rust code. |
| 66 | +* `Clippy`_: Use Clippy to lint your Rust code and identify potential issues. |
| 67 | +* Documentation Comments: Write clear and concise documentation comments for all public functions and data structures. |
| 68 | +* To the extent possible, make apis that will be public to python feel exactly as if they are written in python. |
| 69 | + |
| 70 | +9. Testing |
| 71 | +---------- |
| 72 | + |
| 73 | +Test code in the most appropriate way possible. |
| 74 | + |
| 75 | +* If possible write tests against the public python api using pytest. These tests should be placed in the **tests** top level directory. |
| 76 | +* For functionality that does not have a public python api, or is not well covered by a python api, or is difficult to appropriately test with a python unit test rust unit tests may be written using the standard rust unit test infrastructure. Genreally avoid a rust until test on any code that is wrapped with pyo3. |
| 77 | + |
| 78 | +10. Logging |
| 79 | +----------- |
| 80 | + |
| 81 | +Logging should be done using the log crate, and accompanying macros. The standard rubinoxide package initializes the pyo3_log crate to forward all logs through to the python log handler to the approprate log level. |
| 82 | + |
| 83 | +11. Don'ts |
| 84 | +---------- |
| 85 | + |
| 86 | +* Do not use implicit multithreading in rust |
| 87 | +* Do not introduce any cross package rust bindings that transit through python aka how we use c++ now. |
| 88 | +* As mentioned above, do not arange module structure according to lsst packages. Write modules by related functionality. |
| 89 | + |
| 90 | +12. Build and management system integration |
| 91 | +------------------------------------------- |
| 92 | + |
| 93 | +* Rust packages should be built using maturin, which manages the complexities of compiling and bundling cargo products. |
| 94 | +* Cargo additionally is to be used manage dependencies and run rust level tests. |
| 95 | +* pip is used as the mechanism to locally depoly the wheels created by maturin |
| 96 | +* pytest is used to run python level unit tests |
| 97 | +* coordinating these scripts for the developer is a Makefile |
| 98 | + |
| 99 | +.. _rustfmt: https://github.com/rust-lang/rustfmt |
| 100 | +.. _Clippy: https://github.com/rust-lang/rust-clippy |
| 101 | +.. _pyO3: https://pyo3.rs/ |
0 commit comments