Skip to content
This repository was archived by the owner on Aug 27, 2024. It is now read-only.

Updates to installation guide #772

Merged
merged 9 commits into from
Nov 18, 2020
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
145 changes: 51 additions & 94 deletions docs/knowledgebase/getting-started/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,41 +3,42 @@ title: Installation
---

This page will guide you through the steps needed to prepare a computer for Substrate development.
As Substrate is built with [the Rust programming language](https://www.rust-lang.org/), the first
thing you will need to do is prepare the development enviroment, these steps will vary based
on the computer's operating system. You can utilize helpful utilities from the Rust toolchain
to configure the Rust development environment.
Since Substrate is built with [the Rust programming language](https://www.rust-lang.org/), the first
thing you will need to do is prepare the computer for Rust development - these steps will vary based
on the computer's operating system. Once Rust is configured, you will use its toolchains to interact
with Rust projects; the commands for Rust's toolchains will be the same for all supported,
Unix-based operating systems.

### Unix-Based Operating Systems
## Unix-Based Operating Systems

Substrate development is optimized for Unix-based operating systems like macOS or Linux. The examples
Substrate development is easiest on Unix-based operating systems like macOS or Linux. The examples
in the Substrate [Tutorials](../../../../tutorials) and [Recipes](https://substrate.dev/recipes/)
use the terminal to demonstrate how to interact with Substrate from the command line.
use Unix-style terminals to demonstrate how to interact with Substrate from the command line.

#### macOS
### macOS

Open the Terminal application and execute the following commands:

```bash
# Install Homebrew if necessary https://brew.sh/
/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install.sh)"

# Make sure Homebrew is up-to-date and install some dependencies
# Make sure Homebrew is up-to-date, install openssl and cmake
brew update
brew install openssl cmake
```

#### Ubuntu/Debian
### Ubuntu/Debian

Use your favorite terminal shell to execute the following commands:
Use a terminal shell to execute the following commands:

```bash
sudo apt update
# May prompt for location information
sudo apt install -y cmake pkg-config libssl-dev git build-essential clang libclang-dev curl
```

#### Arch Linux
### Arch Linux

Run these commands from a terminal:

Expand All @@ -47,7 +48,7 @@ export OPENSSL_LIB_DIR="/usr/lib/openssl-1.0"
export OPENSSL_INCLUDE_DIR="/usr/include/openssl-1.0"
```

### Windows
## Windows

Please refer to the separate [guide for Windows users](windows-users.md).

Expand All @@ -59,8 +60,7 @@ configure `rustup`:
```bash
# Install
curl https://sh.rustup.rs -sSf | sh
# Add the rust compiler and other tools to your PATH.
# Make sure to add this to your shell startup script, too.
# Configure
Comment on lines -62 to +63
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I do not think it's necessary to dive into the details of what this does. Furthermore, I don't believe it's necessary to add this to the startup script.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You source that file manually in every terminal you want to use rust?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry I just realized I never responded to this comment. I'm not sure if the installation scripts have been updated or something but it's no longer necessary to manually add the command to the shell startup script. I have not added such a command, nor do I need to source the file manually in ever terminal. I reviewed all the installation instructions for Rustup that I could find and I was not able to see this listed anywhere as a requirement https://rust-lang.github.io/rustup/installation/index.html

source ~/.cargo/env
```

Expand All @@ -74,119 +74,76 @@ rustup default stable

Substrate uses [WebAssembly](https://webassembly.org/) (Wasm) to produce portable blockchain
runtimes. You will need to configure your Rust compiler to use
[`nightly` builds](https://doc.rust-lang.org/book/appendix-07-nightly-rust.html) to allow
compiled substrate compatible runtimes to Wasm.
[`nightly` builds](https://doc.rust-lang.org/book/appendix-07-nightly-rust.html) to allow you to
compile Substrate runtime code to the Wasm target.

#### Rust Nightly Toolchain

Because the nightly toolchain is a moving target and receives daily changes the chance
that some of them break the substrate build from time to time is non-negligible.

Therefore it is advised to use a fixed nightly version rather than the latest one to
build the runtime. You can install a specific version using this command:
Developers building with Substrate should use a specific Rust nightly version that is known to be
compatible with the version of Substrate they are using; this version will vary from project to
project and different projects may use different mechanisms to communicate this version to
developers. For instance, the Polkadot client specifies this information in its
[release notes](https://github.com/paritytech/polkadot/releases). The Substrate Node Template uses
an
[init script](https://github.com/substrate-developer-hub/substrate-node-template/blob/master/scripts/init.sh)
and
[Makefile](https://github.com/substrate-developer-hub/substrate-node-template/blob/master/Makefile)
to specify the Rust nightly version and encapsulate the following steps. Use Rustup to install the
correct nightly:

```bash
rustup install nightly-<yyyy-MM-dd>
Copy link
Contributor

@wheresaddie wheresaddie Nov 17, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

at the time of writing 2020-11-17, the current default nightly which compiles is 2020-10-05

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't want to add this because it creates the question: "the current default nightly that compiles with what?"

Copy link
Contributor

@riusricardo riusricardo Nov 18, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It may be good idea to mention how to find the correct nightly for any case. You mention how to do it in Pokadot but why not mention that a way to find out is to take a look to the date of the release and use that date in the nightly-<date of release>

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

From my POV that gets too deeply into supporting other projects. We need to be very careful about what we are documenting here. We are trying to teach people the best practices for working with Substrate code. I think that at this time it is very important to use a well-known nightly version that corresponds with whatever project you are using. I do not want to encourage people to release Substrate-based code without specifying an exact nightly to use, nor do I want to encourage our users to try to "guess" which nightly they should use. This is definitely an area where I am deciding not to document every possible technical detail in order to encourage people to follow what I believe we should set up as best practice.

Copy link
Contributor

@riusricardo riusricardo Nov 18, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree with almost everything except.

encourage our users to try to "guess" which nightly they should use

I think that this a good to have as a way to troubleshoot. In the back it happens all the time, your machine doesn't has the same nightly as mine because we update at different time. My point is that it is a rule of thumb to get the date of the release and use it for nightly. I'm not talking about other projects I referring to every time we have a Substrate release.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nope, it could go there but it isn't covered. It would be suggesting them to use a very specific date which will work in 99.9999% of the cases. Substrate 2.0 was release in September 22 so using nightly-2020-09-22 will work for them.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No I'm sorry that is just not something I am comfortable supporting at this time.

Copy link
Contributor

@riusricardo riusricardo Nov 18, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't get why not. There is no reason for not doing it. It is correct to do it and will prevent us explaining the same thing every time or adding a lot of patches as you have been doing. Please give a good justification for not doing it.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just saying that you are not comfortable with something that doesn't requires support isn't a valid justification.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would say I provided that justification in my previous comment #772 (comment) I don't feel comfortable setting a precedent that the Knowledge Base provides general support for any or all Substrate-based projects. I don't even really feel comfortable with the Knowledge Base having these instructions at all. I really believe this information belongs with the Node Template. I do not think it is good practice for a Substrate-based project to make a release without specifying the nightly it builds with, so therefore I do not feel comfortable providing any type of support or advice for these cases. It is "undefined behavior". That is my justification for not feeling comfortable supporting this.

```

---
**NOTE**
Due to a regression in the rust compiler, using the newest rust nightly for compiling
the runtime will result in compilation errors. Therefore, it is advised to use the
following version until this issue is resolved:

```bash
rustup install nightly-2020-10-06
```

---

#### Wasm Toolchain

Now, configure the choosen nightly version to work with the Wasm compilation target:
Now, configure the nightly version to work with the Wasm compilation target:

```bash
rustup target add wasm32-unknown-unknown --toolchain nightly-<yyyy-MM-dd>
```

#### Specifying Nightly Version

When a Substrate based project builds its included runtime it picks the latest
installed nightly version by default. If the nightly version is incompatible
you can override that decision by setting the `WASM_BUILD_TOOLCHAIN` environment variable
when building the project by using the following command:
Use the `WASM_BUILD_TOOLCHAIN` environment variable to specify the Rust nightly version a Substrate
project should use for Wasm compilation:

```bash
WASM_BUILD_TOOLCHAIN=nightly-<yyyy-MM-dd> cargo build
WASM_BUILD_TOOLCHAIN=nightly-<yyyy-MM-dd> cargo build --release
```

Note that this builds only the runtime with the specified toolchain. The rest of project will
be compiled with your default toolchain, which is usually the latest installed stable toolchain.
Comment on lines -123 to -124
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why remove this, it seems like helpful info for this context?

Note that this only builds _the runtime_ with the specified nightly. The rest of project will be
compiled with the default toolchain, i.e. the latest installed stable toolchain.

#### Latest Nightly
#### Latest Nightly for Substrate `master`

If you want to build the runtime with the latest nightly compiler which should **generally** be
possible you can install the unspecific `nightly` toolchain:
Developers that are building Substrate _itself_ should always use the latest bug-free versions of
Rust stable and nightly. This is because the Substrate codebase follows the tip of Rust nightly,
which means that changes in Substrate often depend on upstream changes in the Rust nightly compiler.
To ensure your Rust compiler is always up to date, you should run:

```bash
rustup install nightly
rustup update
rustup update nightly
rustup target add wasm32-unknown-unknown --toolchain nightly
```

This toolchain is not tied to a specific version and will be updated just as the
`stable` toolchain:
**It may be necessary to occasionally rerun `rustup update`** if a change in the upstream Substrate
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Id remove this comment as again it creates more questions than answers and by the time people need to rerun rustup update I hope they are far enough into the community that this is a known req at times

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this is something that the core developers have asked that we keep in the docs in order to help with some common questions they receive. I hear what you're saying for sure and I do think it's important we don't get too into the weeds in this doc, but given the above and the fact that this statement seems like it logically dovetails with the preceding statement I think I'm going to leave it at this time.

codebase depends on a new feature of the Rust compiler.

```bash
rustup update
```
#### Downgrading Rust Nightly

## Tips
If your computer is configured to use the latest Rust nightly and you would like to downgrade to a
specific nightly version, follow these steps:

**It may be necessary to occasionally rerun `rustup update`** if a change in the upstream Substrate
codebase depends on the most recent version of the Rust compiler.
```sh
rustup uninstall nightly
rustup install nightly-<yyyy-MM-dd>
rustup target add wasm32-unknown-unknown --toolchain nightly-<yyyy-MM-dd>
```

## Test Your Set-Up

The best way to ensure that you have successfully prepared a computer for Substrate development is
to follow the steps in our first tutorial,
[Create Your First Substrate Chain](../../tutorials/create-your-first-substrate-chain/).

## Build with Nix shell

Nix is a package manager for many different types of projects, including those based on Rust. Using Nix makes developers confident that the compilation output of their substrate-based projects is reliable and reproducible. Find more benefits on the [Nix website] (https://nixos.org/).

### Steps to use Nix shell

#### Install Nix

Run the following command on a Unix based operating system:
```shell
sh <(curl -L https://nixos.org/nix/install) --daemon
```

#### Verify that Nix was properly installed

```shell
nix-env --version
```

You should see an output like: `nix-env (Nix) 2.3.7`.

#### How to use

If you are using `substrate` or `substrate-node-template`, the `shell.nix` file should already be in the project's root directory.

```shell
cd substrate-node-template # or substrate

nix-shell # start the nix shell environment by using dependencies in file `shell.nix`, it may take some time if this's your first time.
```

#### Run cargo commands in the nix shell


```shell
cargo build --release # build the project

./target/release/node-template # run the compiled binary
```
2 changes: 1 addition & 1 deletion docs/tutorials/add-a-pallet/import-a-pallet.md
Original file line number Diff line number Diff line change
Expand Up @@ -170,5 +170,5 @@ error[E0425]: cannot find function `memory_new` in module `sandbox`
Before moving on, check that the new dependencies resolve correctly by running:

```bash
cargo check -p node-template-runtime
SKIP_WASM_BUILD=1 cargo check -p node-template-runtime
```
2 changes: 1 addition & 1 deletion docs/tutorials/add-a-pallet/use-a-pallet.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ Now you are ready to compile and run your node that has been enhanced with nickn
from the Nicks pallet. Compile the node in release mode with:

```bash
cargo build --release
WASM_BUILD_TOOLCHAIN=nightly-2020-10-05 cargo build --release
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All these tutorials use 2.0.0?

You should use a nightly that is around the release of 2.0.0.

And we should also mention which version should be installed for the tutorials.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, all our tutorials are on v2.0.0 🎉 Furthermore, each tutorial (including this one) mentions the specific version of Substrate/the Node Template on which it is built ✔️

Can you recommend a specific nightly? I was under the impression that 2020-10-05 was "the right" one to use w/ v2.0.0. Is this incorrect? Is there a compelling reason to move to another nightly? I have been specifying 2020-10-05 for a while now and I would prefer not to change things if it's not necessary.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If that works it is okay, but there was some other nightly that broke the compilation in between as well and I think it was before october.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I believe we have constantly been referencing 2020-10-05 with success and no one reporting issues afaik @danforbes @bkchr

```

If the build fails, go back to the previous section and make sure you followed all the steps
Expand Down
8 changes: 4 additions & 4 deletions docs/tutorials/add-contracts-pallet.md
Original file line number Diff line number Diff line change
Expand Up @@ -168,7 +168,7 @@ std = [
Now is a good time to check that everything compiles correctly so far with:

```bash
cargo check
SKIP_WASM_BUILD=1 cargo check
```

## Adding the Contracts Pallet
Expand Down Expand Up @@ -286,7 +286,7 @@ should compile, the entire node will not (yet). So we will use this command to c
runtime.

```bash
cargo check -p node-template-runtime
SKIP_WASM_BUILD=1 cargo check -p node-template-runtime
```

### Exposing The Contracts API
Expand Down Expand Up @@ -381,7 +381,7 @@ impl_runtime_apis! {
This is another good time to check that your runtime compiles correctly so far.

```bash
cargo check -p node-template-runtime
SKIP_WASM_BUILD=1 cargo check -p node-template-runtime
```

## Updating the Outer Node
Expand Down Expand Up @@ -499,7 +499,7 @@ Now you are ready to compile and run your contract-capable node. Compile the nod
with

```bash
cargo build --release
WASM_BUILD_TOOLCHAIN=nightly-2020-10-05 cargo build --release
```

Now launch the executable you just built by running this command
Expand Down
2 changes: 1 addition & 1 deletion docs/tutorials/build-a-dapp/pallet.md
Original file line number Diff line number Diff line change
Expand Up @@ -300,7 +300,7 @@ file, you should be able to recompile your node without warning or error. Run th
root directory of the `substrate-node-template` repository to build and run the node:

```bash
cargo run -- --dev --tmp
WASM_BUILD_TOOLCHAIN=nightly-2020-10-05 cargo run --release -- --dev --tmp
```

Now it is time to interact with our new Proof of Existence pallet!
2 changes: 1 addition & 1 deletion docs/tutorials/build-permission-network/launch-network.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ title: Launch our permissioned network
Before even starting a node, let's make sure everything is compiled for our project.

```shell
cargo build --release
WASM_BUILD_TOOLCHAIN=nightly-2020-10-05 cargo build --release
```

In this part, we'll launch 4 nodes, include 3 well known nodes and 1 normal node. Here are the node keys and peer ids that are used in this tutorial.
Expand Down
6 changes: 3 additions & 3 deletions docs/tutorials/create-a-pallet.md
Original file line number Diff line number Diff line change
Expand Up @@ -83,7 +83,7 @@ You should be able to successfully check the Substrate pallet template with:

```bash
cd test-pallet
cargo check
SKIP_WASM_BUILD=1 cargo check
```

### Your Pallet's `std` Feature
Expand Down Expand Up @@ -148,7 +148,7 @@ sp-core = { default-features = false, version = '2.0.0' }
You can confirm that the tests in the Substrate pallet template pass with:

```bash
cargo test
SKIP_WASM_BUILD=1 cargo test
```

When updating this pallet to include your own custom logic, you will likely add dependencies of your
Expand Down Expand Up @@ -216,7 +216,7 @@ Make sure you're back in the node template's root directory, then compile the no
development mode with the following command:

```bash
cargo run -- --dev --tmp
WASM_BUILD_TOOLCHAIN=nightly-2020-10-05 cargo run --release -- --dev --tmp
```

Now, start the
Expand Down
49 changes: 7 additions & 42 deletions docs/tutorials/create-your-first-substrate-chain/setup.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,36 +14,11 @@ You will probably need to do some set-up to prepare your computer for Substrate

### Substrate Development

If you are using a Unix-based machine (Linux, MacOS), we have created a simple one-liner to help you
set up your computer:

```bash
curl https://getsubstrate.io -sSf | bash -s -- --fast
```

> If you did not have Rust installed prior to running this script, make sure to restart your
> terminal before continuing.

<details>
<summary>Learn what this script does.</summary>

> You can view the source code for this script by visiting
> [https://getsubstrate.io](https://getsubstrate.io) in your browser.

It will automatically install:

- [CMake](https://cmake.org/install/)
- [pkg-config](https://www.freedesktop.org/wiki/Software/pkg-config/)
- [OpenSSL](https://www.openssl.org/)
- [Git](https://git-scm.com/downloads)
- [Rust](https://www.rust-lang.org/tools/install)

</details>

If you are using Windows and do not have the
[Windows Subsystem for Linux](https://docs.microsoft.com/en-us/windows/wsl/install-win10), the
process is a bit harder, but well documented
[here](../../knowledgebase/getting-started/windows-users).
Refer to the [official Installation guide](../../knowledgebase/getting-started/). Since this
tutorial uses the Node Template, which includes configuration files and scripts for initializing and
interacting with it, it's not necessary to complete all the steps from the Installation guide. After
`rustup` has been installed and configured, and you've configured the Rust toolchain to default to
the latest stable version you can return to these steps.

## Compiling Substrate

Expand All @@ -59,23 +34,13 @@ Template, which serves as a good starting point for building on Substrate.
2. Initialize your WebAssembly build environment

```bash
# Load settings into the current shell script if you can't use rustup command
# If you've run this before, you don't need to run it again. But doing so is harmless.
source ~/.cargo/env

# Update Rust
rustup update nightly
rustup update stable

# Add Wasm target
rustup target add wasm32-unknown-unknown --toolchain nightly
make init
```

3. Compile the Node Template

```bash
cd substrate-node-template/
cargo build --release
make build
```

The time required for the compilation step depends on the hardware you're using. Don't wait before
Expand Down
Loading