Skip to content

We are so back #12

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
wants to merge 3 commits into
base: main
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 3 additions & 1 deletion .gitignore
Original file line number Diff line number Diff line change
@@ -1,2 +1,4 @@
node_modules/
build/
build/
bun.lockb
.DS_Store
6 changes: 5 additions & 1 deletion package.json
Original file line number Diff line number Diff line change
@@ -1,4 +1,8 @@
{
{
"author": "bytebone",
"homepage": "https://how2host.it",
"name": "how2host",
"displayName": "How2Host",
"scripts": {
"start": "retype start",
"build": "retype build"
Expand Down
149 changes: 149 additions & 0 deletions src/first-steps/2-docker-setup/docker-concepts.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,149 @@
---
# author: bytebone
---

[!ref Official Docs](https://docs.docker.com/engine/install/ubuntu/#install-using-the-repository)

# Docker Concepts

Docker is a containerization software that allows you to run many different programs and servers in isolated environments, where they will always have all their required dependencies installed without you needing them on your host system. Different apps are cut off from each other, unless you manually connect them.

## Images

A *Docker Image* is the actual software image, stored on your hard drive, that will be used to run the app you're setting up. It contains all the compiled code, as well as the OS requirements that the image developer included. The size of Docker images can range from a couple of megabytes to over a gigabyte.

### Important commands

- `docker pull <image>` will download the image from the repository
- `docker image ls` will list all images that you have stored on your drive
- `docker image rm <image>` will delete the specified image from your storage
- this command will fail if a container is still using that image, or if another image depends on that image

## Containers

*Containers* are the result of running an app from its image. This container usually has a startup command that is executed whenever the container is started, for example to run a webserver or serve an app on a specific port. Data in a running container is **ephemeral**, meaning that any changes inside it will be lost once the container is deleted. To persist data in a container, Docker makes use of [Volumes](#volumes).

### Important commands

- `docker run <image>` starts a container based on the specified image
- `docker stop <container>` stops the specified container
- `docker container ls` lists all running containers
- add `-a` to list running and stopped containers
- `docker container rm <container>` deletes the specified container

## Volumes

Since all data in a container is deleted with the container itself, *Images* are used to persist data between separate instances of an image. This is important for any data you don't want to lose when you update images or perform maintenance on an app. There are two ways to use volumes: bind mounts and *Docker volumes*.

### Bind mount

A bind mount takes a folder on your storage and directly mounts it into the container. This way, you can add specific system files or folders into a container, or store the appdata right beside your configuration files. If you're using bind mnounts, it is your job to ensure that the folders you're mounting are accessible by the app that is supposed to read from them. This usually involves researching what user the app runs as and a bit of `chown` and `chmod` to grant the app access to your host folder.

### Docker volume

Docker volumes are an abstraction of regular mounted folders. They are also stored on your storage, specifically at `/var/lib/docker/volumes`, but Docker automatically manages all permissions for them, as well as keeping track which app made which volume. This means that you can very quickly set up apps and never have to worry about permissions, but it can make extracting the data and moving it to a new server more work, since Docker tracks changes in volumes and will be confused about volumes that it has not created itself.

!!!
Using Volumes is recommended for anyone who is just getting started with Docker. They are fast and easy to use, and do not require maintenance. Once you're more comfortable with the inner workings of Docker, you can always manually move the data into bind mounts.
!!!

### Important commands

- `docker volume create <name>` creates a new volume
- `docker volume ls` will list all volumes
- `docker volume rm <name>` will delete the specified volume

## Networks

*Networks* are crucial to understand and use when running larger applications that split their work across multiple smaller images. They allow you set up walled gardens per app, limiting which containers can communicate with one another.

Many apps include every required service in their base image, and do not need companion services to function. If this is the case for the app you're setting up, you usually don't need to worry about networking.

As soon as an app requires more than one container, you should create a separate network for these services to communicate within.

Docker containers are, by default, able to network through your host, allowing them to communicate with other devices in your network. This is not as relevant when using a VPS, which is already isolated, but becomes more important when hosting in your home.
If you want to completely block off containers from external networking, you can create special *internal networks*. Any image in this network is cut off from the host and therefor the external internet, and can only communicate with other containers via Docker networks.

### Important commands

- `docker network create <name>` creates a new regular network
- use `create --internal` to create an internal network instead
- `docker network ls` lists all networks
- `docker network rm <name>` deletes the specified network
- this command will fail if there are still containers in that network

## Docker compose

<!-- add a link here -->
[!ref Docker compose specification]()

Docker compose is a plugin for Docker that allows you to use *compose files* to define one or multiple containers in one file. Compared to using the `docker run` command, this allows you to store all the information required to run your images in a simple, human-readable script, which makes it a lot easier to manage your containers in the long run.

Any guide on this site makes use of Docker compose. The files are usually called `compose.yml`, and are written in a simple YAML syntax. Any argument that exists for `docker run` has a counterpart in docker compose. And if an app you want to run only offers a `docker run` command, there are great web tools that translate this into a compose file.

### Typical compose file layout

!!!
This example does not make realistic sense, since Caddy does not require a database and the paths for volumes are imaginary. Use this only to understand the structure of a compose file, not to actually run the services
!!!

```yml compose.yml
service:
app: # this is the docker internal service name, it can be whatever you deem practical
image: caddy/caddy:latest # image specification, made up of <owner>/<image-name>:<version>
container_name: service-app # if you dont name the container here, it will be named automatically
restart: unless-stopped
volumes:
# volumes are always specified as <location on host>:<location in container>
- ./folder:/data # bind mount with a host path relative to the location of the compose file
- /usr/bin/ffmpeg:/usr/bin/ffmpeg # bind mount with an absolute host path
- data:/other-data # volume mount. note that this volume has to be defined in the volume section below
networks:
- reverse-proxy
- internal

db:
image: postgres:15 # some very important images do not have a maintainer
container_name: service-db
restart: unless-stopped
volumes:
- other-volume: /var/lib/postgresql/
networks:
# since the database should only be accessibly by its accompanied app,
# we create a network only for internal communications of this service
- internal

volumes:
# you only need to define volumes, not bind mounts
data:
# youre able to mount volumes that have not been created by this compose file,
# but they need to be marked as external
other-volume:
external: true

networks:
# same thing as with volumes: all networks you want to use in this compose file need to be listed here.
internal:
# this makes the network fully blocked-off from external networking
# any container that's only in internal networks will not be able to access the internet at all
internal: true
# networks not managed by this compose file need to be marked as external.
reverse-proxy:
external: true
```

Every compose file has a *service name*. This name is usually taken from the name of the folder containing your `compose.yml`. When you create networks and volumes in your compose file, their names get prefixed with the service name. An example:
- You're setting up Vaultwarden. You're `compose.yml` is in `/home/docker/vaultwarden/`
- Taken from the folder name, Docker assumes the service name as `vaultwarden`
- If you create a network `internal` in this compose file, it will actually be called `vaultwarden-internal`
- Same goes for volumes: a volume called `data` in your compose file will actually be called `vaultwarden-data`

Take this into account when defining names __in your compose files__. You don't need to prefix them in there, and can use the same generic names for each of your apps. The more you use patterns for naming your components, the easier it will be to maintain a lot of services at once.

## Docker networking

Lastly, if you're running an app with multiple services, how do you actually make them talk to one another? Docker makes this incredibly easy by allowing you to use the container and service names **instead of IP addresses**. Docker also allows apps to connect on specific ports **without having to expose them on the host**.

To connect an app to its other services, you usually use configuration files or environmental variables. This depends on the app and is usually specified in its documentation. In general, you can always use **the container name** you specified in your compose file as the address: `http://vaultwarden-db:5432`.

Never use the IP addresses assigned to containers by Docker. By default, they are not static and will change when recreating the container, forcing you to reconfigure your apps. There is no benefit in using IPs over your container names.
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,10 @@ order: -2

[!ref Official Docs](https://docs.docker.com/engine/install/ubuntu/#install-using-the-repository)

!!!
If you've never before used Docker and are not familiar with concepts like images, containers, volumes and networks, please give the [quick intro to docker concepts](docker-concepts) a go.
!!!

## Setting up Docker and Docker Compose

This step is fairly simple. If you're not already, connect to your server via SSH. We're now going to run a couple of commands:
Expand Down
10 changes: 7 additions & 3 deletions src/first-steps/3-reverse-proxy/nginx.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,10 @@ order: 1 #remove this once the other reverse proxy guides are done

## Installation with Docker Compose

!!!
If you're not familiar with Docker, it is recommended to read the [Docker Core Concepts page](../2-docker-setup/docker-concepts.md) before continuing here
!!!

Nginx is a reverse proxy, which we will use to direct traffic on your domain name to the different services on your server. Since the classic Nginx is entirely configured in text files, we will be using an extension called "Nginx Proxy Manager" or "NPM" for short. It automates the creation of your config files with an easy-to-use web interface and allows for a fast and easy overview and management of your hosts.

To get started with the Nginx setup, create a new folder at any location you please. As an example, we're going to use `mkdir -p /home/docker/nginx`, followed by `cd /home/docker/nginx`. Now, run `nano compose.yml` and paste the following code:
Expand All @@ -32,7 +36,7 @@ services:
- data:/data
- data:/etc/letsencrypt
networks:
- default
- proxy
- internal
depends_on:
- db
Expand All @@ -53,8 +57,8 @@ services:
- MYSQL_USER=npm

networks:
default:
internal:
proxy:
internal:

volumes:
db:
Expand Down
26 changes: 9 additions & 17 deletions src/first-steps/3-reverse-proxy/readme.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,11 +7,11 @@ order: -3

A reverse proxy connects an app running locally on your server to a web address through subdomains (`my-app.example.com`) or subdirectories (`example.com/my-app`). This keeps your links clean and connections secure, since you no longer need to open server ports for accessing apps.

If you wanted to skip the reverse proxy, you would access your services through links with portnumbers, like `example.com:8000`. This approach would also be very cumbersome, if not impossible to setup when using Cloudflare.
If you skipped the reverse proxy, you would access your services through links with portnumbers, like `example.com:8000`. This approach would also be very cumbersome, if not impossible to setup when using Cloudflare.

## Common Mistakes

⚠ **Exposing ports of applications in your Docker Compose files:** Many official setup guides for apps include a section in their compose file that would open external ports for app access. These sections should **always be removed**, unless you know exactly what you're doing.
**Exposing ports of applications in your Docker Compose files:** Many official setup guides for apps include a section in their compose file that would open external ports for app access. These sections are rarely written with reverse proxies in mind and should **always be removed**, unless you know exactly what you're doing.

⚠ **Not using proper encryption between your server and Cloudflare:** When using the Cloudflare proxy, all your traffic is sent from your server to Cloudflare, who pass it on to your users. To make sure that no one can intercept, read or manipulate the datastream between your and Cloudflares server, the data should be encrypted using a CF-generated certificate, and Cloudflare should be set to refuse any connection that is not encrypted with this certificate.

Expand All @@ -34,32 +34,24 @@ Many guides are currently written with this proxy in mind. While any of the apps

[!ref](./nginx.md)

### <a href="https://doc.traefik.io/traefik/" target="_blank">Traefik</a>
A proxy designed for servers with many apps, controlled through code right in your Docker Compose files.

| Pro | Con |
|---|---|
| <ul><li>Built from the ground up for this use-case</li><li>Configuration of your apps and the proxy is in one place - the Compose file</li> </ul> | <ul> <li>The configuration syntax is not easy to remember</li> </ul> |

[!ref Traefik **(External Link)**](./traefik.md)

### <a href="https://caddyserver.com/docs/quick-starts/reverse-proxy" target="_blank">Caddy</a>

Once just a webserver, Caddy has evolved into a fast proxy that has an incredibly easy configuration syntax and uses ridiculously little ressources.

| Pro | Con |
|---|---|
| <ul><li>Easy configuration with Caddyfiles and easy syntax</li><li>No-Fuss proxy with high speeds and performance</li> </ul> | <ul> <li>Doesn't scale well when hosting many services</li> </ul> |
| <ul><li>Easy configuration with Caddyfiles and easy syntax</li><li>No-Fuss proxy with high speeds and performance</li><li>Fast and fully automatic SSL certificate generation - useful when not using the Cloudflare proxy</li> </ul> | <ul> <li>No Web UI, configured entirely via configuration filels</li> </ul> |

[!ref Caddy **(External Link)**](./caddy.md)

!!!success Interested in some tinkering?
Caddy isn't ideal for a server with many apps running in parallel, but there are community efforts to transform it into something that works more like Nginx - **a singular instance controlling all traffic and internally redirecting to the correct target**.
### <a href="https://doc.traefik.io/traefik/" target="_blank">Traefik</a>
A proxy designed for docker servers, configured through labels right in your Docker Compose files.

This project combines the performance and easy configuration of Caddy with the "configuration-in-compose" approach from Traefik. While the How2Host authors have not yet tested this, it may be promising to the adventurous.
| Pro | Con |
|---|---|
| <ul><li>Built from the ground up for this use-case</li><li>Configuration of your apps and the proxy is in one place - the Compose file</li> </ul> | <ul> <li>The configuration syntax is not easy to remember</li> </ul> |

[!button target="blank" icon="link-external" text="Check it out"](https://github.com/lucaslorentz/caddy-docker-proxy)
!!!
[!ref Traefik **(External Link)**](./traefik.md)

### <a href="https://developers.cloudflare.com/cloudflare-one/connections/connect-networks/" target="_blank">Cloudflare Tunnels</a>

Expand Down