feat: add support for other asset stores than S3 (#59)

Co-authored-by: Hong Minhee (洪 民憙) <[email protected]>

.env.sample

@@ -1,5 +1,4 @@
 DATABASE_URL=postgresql://user:password@localhost:5432/dbname
-REDIS_URL=redis://localhost/0
 HOME_URL=https://example.com/  # optional; if present, the home page will redirect to this URL
 SECRET_KEY=secret_key  # generate a secret key with `openssl rand -base64 32`
 LOG_LEVEL=info
@@ -12,10 +11,12 @@ LISTEN_PORT=3000
 # is implemented.
 ALLOW_PRIVATE_ADDRESS=false
 REMOTE_ACTOR_FETCH_POSTS=10
+DRIVE_DISK=
+ASSET_URL_BASE=
+FS_ASSET_PATH=
 AWS_ACCESS_KEY_ID=
 AWS_SECRET_ACCESS_KEY=
 S3_REGION=
 S3_ENDPOINT_URL=
 S3_BUCKET=
-S3_URL_BASE=
 S3_FORCE_PATH_STYLE=false

.gitignore

@@ -1,3 +1,4 @@
 .env
+assets/
 fedify-hollo-*.tgz
 node_modules/

CHANGES.md

@@ -6,6 +6,18 @@ Version 0.3.0
 
 To be released.
 
+ -  Added support for local filesystem storage for media files.
+    You can now configure `DRIVE_DISK=fs` and `FS_ASSET_PATH` to store media
+    files in the local filesystem.  [[#59]]
+
+     -  Added `DRIVE_DISK` environment variable.
+     -  Added `FS_ASSET_PATH` environment variable.
+     -  Added `ASSET_URL_BASE` environment variable to replace `S3_URL_BASE`.
+     -  Deprecated `S3_URL_BASE` environment variable in favor of
+        `ASSET_URL_BASE`.
+
+[#59]: https://github.com/dahlia/hollo/pull/59
+
 
 Version 0.2.1
 -------------

compose-fs.yaml

@@ -0,0 +1,32 @@
+services:
+  hollo:
+    image: ghcr.io/dahlia/hollo:canary
+    ports:
+      - "3000:3000"
+    environment:
+      DATABASE_URL: "postgres://user:password@postgres:5432/database"
+      SECRET_KEY: "${SECRET_KEY}"
+      LOG_LEVEL: "${LOG_LEVEL}"
+      BEHIND_PROXY: "${BEHIND_PROXY}"
+      DRIVE_DISK: fs
+      ASSET_URL_BASE: http://localhost:3000/assets/
+      FS_ASSET_PATH: /var/lib/hollo
+    depends_on:
+      - postgres
+    volumes:
+      - assets_data:/var/lib/hollo
+    restart: unless-stopped
+
+  postgres:
+    image: postgres:17
+    environment:
+      POSTGRES_USER: user
+      POSTGRES_PASSWORD: password
+      POSTGRES_DB: database
+    volumes:
+      - postgres_data:/var/lib/postgresql/data
+    restart: unless-stopped
+
+volumes:
+  postgres_data:
+  assets_data:

compose.yaml

@@ -8,9 +8,10 @@ services:
       SECRET_KEY: "${SECRET_KEY}"
       LOG_LEVEL: "${LOG_LEVEL}"
       BEHIND_PROXY: "${BEHIND_PROXY}"
+      DRIVE_DISK: s3
+      ASSET_URL_BASE: http://localhost:9000/hollo/
       S3_REGION: us-east-1
       S3_BUCKET: hollo
-      S3_URL_BASE: http://localhost:9000/hollo/
       S3_ENDPOINT_URL: http://minio:9000
       S3_FORCE_PATH_STYLE: "true"
       AWS_ACCESS_KEY_ID: minioadmin

docs/src/content/docs/install/env.mdx

@@ -9,6 +9,10 @@ Hollo is configured using environment variables.  You can set them in an *.env*
 file in the root directory of the project, or you can set them using Docker's
 `-e`/`--env` option or Railway's environment variables.
 
+
+Basic settings
+--------------
+
 ### `PORT` <Badge text="Optional" /> <Badge text="Unused in Railway" variant="tip" />
 
 The port number to listen on.  3000 by default.
@@ -18,11 +22,6 @@ The port number to listen on.  3000 by default.
 The URL of the PostgreSQL database, e.g.,
 `postgresql://hollo:password@localhost/hollo`.
 
-###  `HOME_URL` <Badge text="Optional" />
-
-If present, the home page will redirect to this URL.  If not set, the home page
-will show the list of accounts on the instance.
-
 ### `SECRET_KEY` <Badge text="Unused in Railway" variant="tip" />
 
 The secret key for securing the session.  You can generate a random secret key
@@ -32,13 +31,52 @@ using the following command:
 openssl rand -hex 32
 ~~~~
 
+### `BEHIND_PROXY` <Badge text="Optional" /> <Badge text="Unused in Railway" variant="tip" />
+
+Set this to `true` if Hollo is behind a reverse proxy.  If you place the Hollo
+behind an L7 load balancer (you usually should do this), turn this on.
+
+Turned off by default.
+
+<Aside>
+  With this option, Hollo will trust the `X-Forwarded-For`, `X-Forwarded-Proto`,
+  and `X-Forwarded-Host` headers from the reverse proxy.  This is important for
+  security reasons.
+</Aside>
+
+### `ALLOW_PRIVATE_ADDRESS` <Badge text="Optional" />
+
+Setting this to `true` disables SSRF (Server-Side Request Forgery) protection.
+
+Turn on to test in local network.
+
+Turned off by default.
+
+<Aside>
+  Turning on this option is dangerous security-wise.  Only use this option in
+  a trusted environment and never in production.
+</Aside>
+
+
+Additional features
+-------------------
+
+###  `HOME_URL` <Badge text="Optional" />
+
+If present, the home page will redirect to this URL.  If not set, the home page
+will show the list of accounts on the instance.
+
 ### `REMOTE_ACTOR_FETCH_POSTS` <Badge text="Optional" />
 
 The number of recent public posts to fetch from remote actors when they are
 encountered first time.
 
 `10` by default.
 
+
+Logging
+-------
+
 ### `LOG_LEVEL` <Badge text="Optional" />
 
 The log level for the application.  `debug`, `info`, `warning`, `error`, and
@@ -52,61 +90,96 @@ Set this to `true` to log SQL queries.
 
 Turned off by default.
 
-### `BEHIND_PROXY` <Badge text="Optional" /> <Badge text="Unused in Railway" variant="tip" />
 
-Set this to `true` if Hollo is behind a reverse proxy.  If you place the Hollo
-behind an L7 load balancer (you usually should do this), turn this on.
+Asset storage
+-------------
 
-Turned off by default.
+### `DRIVE_DISK`
 
-<Aside>
-  With this option, Hollo will trust the `X-Forwarded-For`, `X-Forwarded-Proto`,
-  and `X-Forwarded-Host` headers from the reverse proxy.  This is important for
-  security reasons.
+The disk driver used by Hollo to store blobs such as avatars, custom emojis,
+and other media.
+
+Valid values are `fs` (local filesystem) and `s3` (S3-compatible object storage).
+
+Defaults to `s3` for backward compatibility, but it is recommended to explicitly
+configure the driver as the default value will be removed in the future.
+
+See the [FlyDrive docs] for details about the drivers.
+
+<Aside type="caution">
+  If you change the `DRIVE_DISK` and related settings of a running Hollo during
+  production, existing files are not moved; only new files are stored according
+  to the new settings.
+
+  Also, keep in mind that existing files will still be served according to the
+  old settings, so if you change `S3_BUCKET`, for example, they will still be
+  served from the old bucket and should not be deleted.
 </Aside>
 
-### `ALLOW_PRIVATE_ADDRESS` <Badge text="Optional" />
+[FlyDrive docs]: https://flydrive.dev/docs/drive_manager
 
-Setting this to `true` disables SSRF (Server-Side Request Forgery) protection.
+### Local filesystem settings
 
-Turn on to test in local network.
+#### `FS_ASSET_PATH` <Badge text="Required with FS driver" variant="note" />
 
-Turned off by default.
+The path in the local filesystem where blob assets are stored, e.g.,
+`/var/lib/hollo`.
 
 <Aside>
-  Turning on this option is dangerous security-wise.  Only use this option in
-  a trusted environment and never in production.
+  - Directory must exist and be writable by the Hollo process.
+  - Recommended permissions: 755 for directories, 644 for files.
+  - Ensure sufficient storage capacity for your use case.
+  - Regular backups are strongly recommended.
+  - When using Docker, ensure the path is properly mounted.
 </Aside>
 
-### `S3_REGION` <Badge text="Optional" />
+### S3-compatible object storage settings
 
-The region of the S3-compatible object storage, e.g., `us-east-1`.  On some
-non-S3 services, this can be omitted.  `auto` by default.
+<Aside>
+  These settings are only required when `DRIVE_DISK=s3`.
 
-### `S3_BUCKET`
+  Ensure your AWS IAM policy grants the following permissions:
 
-The bucket name of the S3-compatible object storage, e.g., `hollo`.
+  - `s3:PutObject`
+  - `s3:GetObject`
+  - `s3:DeleteObject`
+  - `s3:ListBucket`
+</Aside>
 
-### `S3_URL_BASE`
+#### `ASSET_URL_BASE` <Badge text="Required with S3 driver" variant="caution" />
 
-The public URL base of the S3-compatible object storage, e.g.,
+The public URL base of the asset storage, e.g.,
 `https://hollo.s3.us-east-1.amazonaws.com`.
 
-### `S3_ENDPOINT_URL`
+<Aside type="caution">
+  - HTTPS is required in production environments.
+  - Must be publicly accessible for federation to work correctly.
+</Aside>
+
+#### `S3_REGION` <Badge text="Optional" />
+
+The region of the S3-compatible object storage, e.g., `us-east-1`.  On some
+non-S3 services, this can be omitted.  `auto` by default.
+
+#### `S3_BUCKET` <Badge text="Required with S3 driver" variant="caution" />
+
+The bucket name of the S3-compatible object storage, e.g., `hollo`.
+
+#### `S3_ENDPOINT_URL` <Badge text="Required with S3 driver" variant="caution" />
 
 The endpoint URL for S3-compatible object storage, e.g.,
 `https://s3.us-east-1.amazonaws.com`.
 
-### `S3_FORCE_PATH_STYLE`
+#### `S3_FORCE_PATH_STYLE` <Badge text="Optional" />
 
 Whether to force path-style URLs for S3-compatible object storage.  `true` to
 turn on, `false` to turn off.  Useful for non-AWS S3-compatible services.
 Turned off by default.
 
-### `AWS_ACCESS_KEY_ID`
+#### `AWS_ACCESS_KEY_ID` <Badge text="Required with S3 driver" variant="caution" />
 
 The access key for S3-compatible object storage.
 
-### `AWS_SECRET_ACCESS_KEY`
+#### `AWS_SECRET_ACCESS_KEY` <Badge text="Required with S3 driver" variant="caution" />
 
-The secret key for S3-compatible object storage
+The secret key for S3-compatible object storage.