Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[docs] Refreshes development.md #3932

Open
wants to merge 34 commits into
base: main
Choose a base branch
from

Conversation

bjacobgordon
Copy link
Contributor

@bjacobgordon bjacobgordon commented Dec 23, 2024

First PR, super stoked! 🥳

Context

As part of Turbine Camp, I was referred to this doc to build Torch-MLIR on my Linux VM. It has a lot of potential, and I decided to nudge it in a productive direction a bunch of times as I read and used the doc.

Approach

I tried to make the commits here "atomic", i.e. each one is a singular unit of work such that the file "works better"/"makes more sense" post-commit than pre-commit.

The majority of the commits in this PR is the story of how I "refactored" the doc step by step to look the way it does. I wrote them with Future Us™ in mind! Each commit should have useful information of the changes made that IDEs like VSCode should be able to surface, so please don't squash haha.

Ready when you are, @marbre!

@bjacobgordon bjacobgordon marked this pull request as draft December 23, 2024 23:16
@bjacobgordon
Copy link
Contributor Author

@marbre The only thing I'm unsure about is removing the libtorch flags

@stellaraccident
Copy link
Collaborator

Thanks! Marius/Jacob, please review and land as you see fit. I'll have a closer look when I'm back and can contribute any adjustments then.

@bjacobgordon bjacobgordon force-pushed the docs-refreshes-development-md branch from bd16189 to 0927acd Compare January 2, 2025 14:48
@marbre marbre self-requested a review January 2, 2025 18:12
@bjacobgordon bjacobgordon marked this pull request as ready for review January 3, 2025 18:08
@bjacobgordon bjacobgordon changed the title Refreshes docs/development.md [docs] Refreshes development.md Jan 3, 2025
@bjacobgordon bjacobgordon force-pushed the docs-refreshes-development-md branch 4 times, most recently from 304b07f to a7275ff Compare January 13, 2025 15:21
@bjacobgordon bjacobgordon force-pushed the docs-refreshes-development-md branch 2 times, most recently from 05ea09d to 7f23d36 Compare January 20, 2025 14:47
Copy link
Member

@marbre marbre left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for working through the documentation. I certainly understand that this needs improvement and should be restructured in some way. However, I am not sure if that is the structure we should aim for. Maybe we should split out some things, like the formatting, and land this separately. Next we could iterate on the docs, maybe in smaller chunks, before restructuring to much at once. Just thinking loud and other approaches have their benefits as well.

docs/development.md Outdated Show resolved Hide resolved
docs/development.md Outdated Show resolved Hide resolved
docs/development.md Outdated Show resolved Hide resolved
docs/development.md Outdated Show resolved Hide resolved
docs/development.md Outdated Show resolved Hide resolved
@bjacobgordon bjacobgordon force-pushed the docs-refreshes-development-md branch from 7f23d36 to d4b0153 Compare January 21, 2025 17:47
docs/development.md Outdated Show resolved Hide resolved
docs/development.md Outdated Show resolved Hide resolved
docs/development.md Outdated Show resolved Hide resolved
docs/development.md Outdated Show resolved Hide resolved
docs/development.md Outdated Show resolved Hide resolved
docs/development.md Outdated Show resolved Hide resolved
@bjacobgordon bjacobgordon force-pushed the docs-refreshes-development-md branch from d4b0153 to 1c5d64e Compare January 21, 2025 23:05
bjacobgordon and others added 3 commits January 28, 2025 14:44
… by blank lines

- enforced by markdown linter
- splits CLI commands so copy button can be used individually
- nests details for step 3 accordingly
@bjacobgordon bjacobgordon force-pushed the docs-refreshes-development-md branch 2 times, most recently from 2daa32c to ef8e61b Compare January 28, 2025 20:29
@bjacobgordon bjacobgordon requested a review from marbre January 28, 2025 20:33
@bjacobgordon
Copy link
Contributor Author

Okay, @marbre, I think we're almost there! How's it look now?

Copy link
Member

@marbre marbre left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just a quick review, traveling, but leaving a few comments.


Two setups are possible to build: in-tree and out-of-tree. The in-tree setup is the most straightforward, as it will build LLVM dependencies as well.
- Some older pip installs may not be able to handle the recent PyTorch deps
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
- Some older pip installs may not be able to handle the recent PyTorch deps
Some older pip installs may not be able to handle the recent PyTorch deps

I don't see the need for a bullet point here. Same at some places below.

Copy link
Contributor Author

@bjacobgordon bjacobgordon Jan 30, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This phrase was originally a comment in the block above the shell command, very subtle. How do we communicate "you can gloss over this line"? Maybe prepending "NOTE:" instead of having the bullet point?

-DPython3_FIND_VIRTUALENV=ONLY \
-DMLIR_ENABLE_BINDINGS_PYTHON=ON \
-DLLVM_TARGETS_TO_BUILD=host \
\
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
\

The additional line looks odd to me.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree. Wanting to somehow communicate at-a-glance "this chunk of options is the key difference between the two otherwise code blocks". What's another way we could do that?

cmake --build build --target check-torch-mlir
```shell
cmake -GNinja -Bbuild \
`# Enables "--debug" and "--debug-only" flags for the "torch-mlir-opt" tool` \
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm guessing this is not what you had in mind haha. I'll need a little more direction here, which parts seem odd?

-DPython3_FIND_VIRTUALENV=ONLY \
-DMLIR_ENABLE_BINDINGS_PYTHON=ON \
-DLLVM_TARGETS_TO_BUILD=host \
\
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
\

docs/development.md Show resolved Hide resolved
```

- Be sure to have built LLVM with `-DLLVM_ENABLE_PROJECTS=mlir`.
- Be aware that the installed version of LLVM needs in general to match the committed version in `externals/llvm-project`. Using a different version may or may not work.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's rather the MLIR commit than LLVM.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is how it's worded on main, I think all I did was move it and add a bullet point.

Could we defer rephrasing this to a subsequent PR?

1. **If you anticipate needing to frequently rebuild LLVM**, append:

```shell
\
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is what you had meant, right? All you need to get going is copying, pasting and running one of the "base" commands. Tacking this on before running is optional, right?

1. **If you need to enable local end-to-end tests**, append:

```shell
\
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
\

docs/development.md Outdated Show resolved Hide resolved
docs/development.md Outdated Show resolved Hide resolved
- Before, they were easy to miss and required you to back track a bit
…ails

- Makes the "in-tree" and "out-of-tree" variants adjacent to each other
- emphasizes the common options between "in-tree extended", "in-of-tree base", and "out-of-tree"
- tweaks existing headers to communicate command construction rather than immediate execution
- adds new header to indicate when the command should actually be executed
- the use-case for this doc leans more toward "development" rather than "release"
- leverages numbering and indentation to reduce cognitive load
CMake issued a warning upon fresh build:

> Manually-specified variables were not used by the project:

>   LIBTORCH_CACHE

>   LIBTORCH_SRC_BUILD

>   LIBTORCH_VARIANT
…arate code blocks

- makes them individually copyable from GitHub
… Build"

- makes it so that the GitHub copy button only grabs the code snippet itself
- less repetitive
- emphasizes that the targets can be chained
@bjacobgordon bjacobgordon force-pushed the docs-refreshes-development-md branch from ef8e61b to 7825aa4 Compare January 30, 2025 21:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants