[RFC] tools: Integrate DTSh, a shell-like interface to Devicetree, as an optional module project (v2)

[dottspina]

Introduction

Note

This RFC initially proposed moving DTSh development to Zephyr’s devicetree tooling, sibling to the python-devicetree library.

The conversation showed that such close integration was questionable, especially:

• There are users interested in such a tool, but it does not fit everyone's work-flow, and should preferably be opt-in (disabled by default).
• The size of the PR (about a hundred new files) discouraged its reviewers and raised serious concerns about its future maintenance.

Acceptable approaches were eventually selected during an Architecture WG meeting on February, 2025 (minutes).

Among these, it seems there is now some consensus to integrate DTSh as an optional project (@henrikbrixandersen , @decsny): this second version of the RFC follows this design. Thanks for your feedback.

Problem description

When I was first getting to know Zephyr's use of Devicetree, I wished I had a quick and easy tool at hand to explore the DTS files generated at build time:

• visualize the devicetree as, well, a tree
• find nodes based on supported bus protocols, generated IRQs, or keywords like "BME680" or "PWM"
• jump from the DTS to the other relevant files, e.g. bindings or SVD
• export simple text summary or figures to illustrate my note-taking

Others had already experienced a comparable lack, still without an obvious answer, see e.g. #29295, October 2020:

Is your enhancement proposal related to a problem? Please describe.

Devicetree is widely used by Zephyr, but it lacks convenient tooling
for development.

Describe the solution you'd like

A graphical devicetree viewer and editor, like 'guiconfig' for DTS.

Proposed Change (Summary)

DTSh is an interactive DTS files viewer with a shell-like command line interface: the devicetree is presented as a hierarchical file system, navigated and queried with familiar commands like cd, ls or find. Commands output redirection with e.g. > permits to export custom reproducible views of the devicetree to text, HTML, or SVG files.
The dtsh shell has a rich user interface (contextual auto-completion, commands history, semantic highlighting, pager integration, user profiles). It can also execute commands or scripts non-interactively.
Please refer to the DTSh manual for details.

The proposed change is a new West command, provided by an optional project, that opens DTS files in dtsh:

$ west build
$ west dtsh
dtsh (0.2.2): Shell-like interface with Devicetree
How to exit: q, or quit, or exit, or press Ctrl-D

/
> cd &flash_controller

/soc/flash-controller@4001e000
> find -E --also-known-as (image|storage).* --format NKd -T
                             Also Known As               Description
                             ───────────────────────────────────────────────────────────────────────────────────
flash-controller@4001e000    flash_controller            Nordic NVMC (Non-Volatile Memory Controller)
└── flash@0                  flash0                      Flash node
    └── partitions                                       This binding is used to describe fixed partitions…
        ├── partition@c000   image-0, slot0_partition    Each child node of the fixed-partitions node represents…
        ├── partition@82000  image-1, slot1_partition    Each child node of the fixed-partitions node represents…
        └── partition@f8000  storage, storage_partition  Each child node of the fixed-partitions node represents…

where:

• west dtsh: called without argument, the command will open the DTS file build/zephyr/zephyr.dts, and retrieve the bindings that Zephyr used at build-time from the CMake cache file build/CMakeCache.txt
• cd &flash_controller changes the current working branch from the devicetree's root to the node at /soc/flash-controller@4001e000, using its DTS label flash_controller
• find is with no surprise a shell command that will search for devices, bindings, buses, interrupts, etc
• here -E --also-known-as (image|storage).* will match nodes with a label or alias starting with image or storage; predicates like --with-reg-size >4k are also supported
• --format NKd: set the node output format to the columns "node Name,", "Also Known As" (all labels and aliases), and "description" (D is the "Depends-on" column)
• the descriptions, e.g. "Flash node", are actually hyperlinks that, when clicked, will open the corresponding binding files
• -T: list found nodes in tree-like format
• appending > doc/partitions.svg to the last command line would save the partitions tree to the file doc/partitions.svg, in SVG format

Detailed RFC

This RFC proposes integrating DTSh as an optional module project.

Proposed Change (Detailed)

With this approach, changes to the zephyrproject-rtos/zephyr repository are minimal:

• add optional project to submanifests/optional.yaml
• add a brief entry to zephyr-cmds.rst to introduce the West command

Optional project

Add optional project zephyrproject-rtos/dtsh to submanifests/optional.yaml:

    - name: dtsh
      description: Devicetree shell 
      revision: zephyr
      path: modules/tools/dtsh
      remote: upstream
      west-commands: scripts/west-commands.yml
      groups:
        - optional

The proposed project layout is given bellow:

zephyrproject-rtos/dtsh/
├── lib
│   └── dtsh
├── LICENSE
├── mypy.ini
├── pyproject.toml
├── README.md
├── ruff.toml
├── scripts
│   ├── dtsh_cmd.py
│   └── west-commands.yml
├── tests
└── zephyr
    ├── module.yml
    └── requirements.txt

where:

• lib/dtsh contains DTSh implementation
• scripts contains the West extension added by the project
• tests contains DTSh unit tests
• zephyr contains the Zephyr module definition, mainly to benefit from the west packages commodity

Documentation

Although the main documentation should belong to zephyrproject-rtos/dtsh, adding a brief entry to zephyr-cmds.rst might improve discoverability, especially since west --help won't show the dtsh command.

Dependencies

DTSh is implemented in Python, with few dependencies:

• the python-devicetree library, developed and distributed by the Zephyr project
• a couple of external Python requirements available on PyPI

External dependencies

These are external Python libraries needed to run west dtsh that have not already been installed with Zephyr requirements:

• Textualize's rich, Python library for rich text and beautiful formatting in the terminal
• gnureadline, macOS only

These dependencies will be installed as module requirements with:

west packages -m dtsh pip install

Note

For auto-completion, command history and key bindings, DTSh relies on the readline module of the Python standard library.

On POSIX-like systems, the underlying Readline library API may be implemented by the libedit library instead of GNU readline which DTSh expects. On macOS, where libedit will likely be the default for most Python distributions, DTSh will substitute the shipped readline module with gnureadline, which is statically linked to GNU Readline.

On MS Windows, the readline module has a rather eventful history, e.g.:

• There's no readline module on Windows Python
• python's readline module not available for windows?
• Support GNU readline-like keybinds in the Windows REPL

In a nutshell, although the situation is improving (Python 3.13 for MS Windows includes an almost complete readline module, and Python 3.14 should be fine), Readline support will very likely be disabled on MS Windows, resulting in a degraded user experience.
This kind of small differences in the non-essential tools available across platforms is deemed acceptable.

python-devicetree

DTSh relies on Zephyr’s python-devicetree library for loading DTS and binding files into Devicetree models.

We agreed (see comment) that west dtsh should always import the devicetree package from $ZEPHYR_BASE/scripts/dts/python-devicetree/src (in-tree): it is therefore not a requirement of the module.

DTSh is however developed out-of-tree, and may be distributed and used outside of Zephyr: in these contexts, we should not depend on any ZEPHYR_BASE. For this, we agreed that regular updates to the python-devicetree release available on PyPI would definitely be a plus, and @mbolivar seems quite willing to take it on.

Note

Although not a priority, compatibility with other users of the Devicetree specifications, such as the Linux kernel, was part of Zephyr Design goals.

DTSh shares this hope hand in hand with edtlib, even if not in a foreseeable future.

Concerns and Unresolved Questions

Repositories and policies

The development of DTSh continues in the repository dottspina/dtsh.

Integration with Zephyr is carried out within the optional project zephyrproject-rtos/dtsh (to be created). Also being a module, its default branch should be named zephyr.
It is not considered a direct downstream of dottspina/dtsh, and may not preserve its whole folder structure.

Bellow are a some policies that I think are reasonable, and largely match the Module Repositories guidelines:

• changes to the module repository should go through pull requests, that might need the approval of a couple of Zephyr maintainers
• these pull requests should trigger CI actions for compliance and integration tests (à la .github/workflows/)
• changes that are not specific to DTSh integration with Zephyr should first appear in dottspina/dtsh (subject to exceptions)
• users of west dtsh should first report issues to the module repository, they will be upstreamed to dottspina/dtsh if appropriate
• I obviously volunteer to maintain this repository (dtsh updates, be a default assignee, CI configuration)

These rules are probably not detailed enough: to be supplemented with the help of reviewers.

Note

For example, the PR fails compliance tests at:

ERROR   : Test ModulesMaintainers failed: 
Missing MAINTAINERS.yml entry for: "West project: dtsh"

CI shows a similar error:

Pull request is labeled as "DNM (manifest)".
label: manifest-dtsh
This workflow fails so that the pull request cannot be merged.

Should I propose adding myself somewhere in the MAINTAINERS file, or ask for a volunteer ?

Kconfig

From Build system integration:

To satisfy compliance checking when building Zephyr without the module present, it’s recommended for the module to have default definitions for these symbols in its respective Kconfig file under modules/ in the Zephyr main tree.

Although the DTSh module does not actually integrate with Zephyr build system, should we include a minimal Kconfig file to at least define ZEPHYR_DTSH_MODULE ?

Security

The proposed change is not that hazardous:

• dtsh does not evaluate (eval()) any part of the user input
• dtsh does not pipe any part of the user input to the system interpreter (e.g. os.system())
• Devicetree source and binding files, CMake cache files, etc, are all opened read-only
• by default dtsh will not override user files, excepted when appending a redirected command output to an existing file
• the writable user specific files (command history, custom configuration and theme files) are written to (and read from) a per-user application data directory located under the appropriate platform-dependent root (e.g. $XDG_CONFIG_HOME or %LOCALAPPDATA%)

West command completion

I didn't figure out how to integrate with West commands completion (we can't add ourselves to the scripts/west_commands/completion/west-completion.* files as Zephyr extensions usually do).

Personal biases

Why a command line application ? Well, when you've done west build, you're already at the command line, and I think continuing from there with west dtsh, getting a different prompt but the same user interface paradigms and even key bindings (the base ones used by Zsh, GNU Bash, Emacs, GDB), is more ditrsaction-free than opening a GUI.

Beyond this personal biases, things like PyGObject or Qt for Python have nonetheless been considered, but:

• these would be huge requirements to add to Zephyr, and to the users' workspaces
• I wanted something simple and fast, easy to extend and to prototype with
• at the end of the day, I'm not sure a full GUI will be more user-friendly or informative than what is possible with a CLI

Alternatives Considered

There doesn't seem to be many DTS file viewers, and even fewer that support Zephyr's bindings.

Nordic Semiconductor's distributes the nRF DeviceTree extension for VS Code, which looks interesting but does not address the initial problem (as @hongshui3000 also pointed out):

• it's not Open Source, and seems to assume you're using the nRF Connect SDK, not Zephyr
• VS Code with the nRF Connect Extension Pack is not really a simple tool everyone is supposed to have on hand (or to work with)

Work on the more ambitious RFC - DTV (Device Tree Visualiser) appears to have stopped in 2020.

[hongshui3000]

I think VS Code extension: nRF DeviceTree is a good graphical tool. But it's not open source. Not particularly friendly to non-nRF devices

[mbolivar-ampere]

Thanks for this. I think it's a great idea and I especially appreciate the attention to detail you put into the RFC as well as volunteering maintenance. Consider me onboard with this idea. I'll give it some testing and review.

[mbolivar-ampere]

Some unboxing comments:

Python provided by Homebrew or Fink: should also work OOTB

I can't get tab completion working as I expect using the homebrew python 3.11. From the prompt:

/
❭ 

I press TAB and expect commands to be completed. But instead I see a tab character is inserted. Is that expected? Here is my python info (using a virtual environment):

(devel) mbolivar@PDX-FY74HK3HFV zephyr % python -V
Python 3.11.4
(devel) mbolivar@PDX-FY74HK3HFV zephyr % echo $(readlink $(which python))
/opt/homebrew/opt/[email protected]/bin/python3.11

I can import readline from the python REPL with no error. I did a pip install -r zp/zephyr/scripts/dts/dtsh/requirements.txt before running west dtsh.

The command parser also doesn't seem to be stripping leading whitespace:

/
❭     q
dtsh: command not found: q

/
❭ q
bye.

this isn't like what you'd expect from a shell in my opinion -- I would expect both of those to quit.

[dottspina]

Assuming that it is up to me to decide among acceptable approaches, I can propose:

• we integrate DTSh as an optional module that defines and implements the new West extension, hosted in a dedicated repository (preferably a new repository under the zephyrproject-rtos GitHub organization, compared to adding a new branch to the existing DTSh repository)
• only the module integration bits (e.g. in submanifests/optional.yaml) go to Zephyr main tree

If we agree on this base, I will prepare the corresponding new PRs.

Thanks.

[henrikbrixandersen]

Assuming that it is up to me to decide among acceptable approaches, I can propose:

• we integrate DTSh as an optional module that defines and implements the new West extension, hosted in a dedicated repository (preferably a new repository under the zephyrproject-rtos GitHub organization, compared to adding a new branch to the existing DTSh repository)
• only the module integration bits (e.g. in submanifests/optional.yaml) go to Zephyr main tree

If we agree on this base, I will prepare the corresponding new PRs.

So this would be solution number 1 listed in the outcome of the Architecture WG meeting on February 4th, 2025 (#59863 (comment)). I think that's a good solution. CC: @carlescufi

[github-actions]

This pull request has been marked as stale because it has been open (more than) 60 days with no activity. Remove the stale label or add a comment saying that you would like to have the label removed otherwise this pull request will automatically be closed in 14 days. Note, that you can always re-open a closed pull request at any time.

[github-actions]

This pull request has been marked as stale because it has been open (more than) 60 days with no activity. Remove the stale label or add a comment saying that you would like to have the label removed otherwise this pull request will automatically be closed in 14 days. Note, that you can always re-open a closed pull request at any time.

[github-actions]

The following west manifest projects have changed revision in this Pull Request:

Name	Old Revision	New Revision	Diff
dtsh 🆕	None (Added)	zephyr	N/A

⛔ DNM label due to: 1 added project and 1 unreachable repo

Note: This message is automatically posted and updated by the Manifest GitHub Action.

[henrikbrixandersen]

Please list this in https://docs.zephyrproject.org/latest/develop/manifest/index.html#external-projects-modules instead. This is inline with this RFC from @nashif: #91061

[dottspina]

Thanks, I wasn't aware of this issue and related changes.
Can we still keep a brief entry in zephyr-cmds.rst for better visibility of the West command?

[henrikbrixandersen]

I think it should go into the documentation for the module (i.e. https://docs.zephyrproject.org/latest/develop/manifest/external/dtsh.html).

[dottspina]

OK, this will be more consistent with the other external modules (AFAICT they don't add a West extension, though).

[sonarqubecloud]

Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud

[dottspina]

So this would be solution number 1 listed in the outcome of the Architecture WG meeting on February 4th, 2025 (#59863 (comment)). I think that's a good solution. CC: @carlescufi

I have largely rewritten the RFC description to focus on this approach (an optional/external module project): this should clarify where this long and sometimes confusing conversation has taken us, what needs review, and what the remaining issues are.

I also updated the PR accordingly: with this approach, changes to the zephyrproject-rtos/zephyr repository are now minimal, and the diffstat should be much less scary (2 files changed).

With Zephyr 4.3 reaching Fearure Freeze in the next few days, I think we should try (one last time) to postpone DTSh integration (to milestone 4.4).

Hope this helps.

Thanks.

[henrikbrixandersen]

With Zephyr 4.3 reaching Fearure Freeze in the next few days, I think we should try (one last time) to postpone DTSh integration (to milestone 4.4).

If this ends up just being a documentation change, it can even go in after the feature freeze.