Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
118 changes: 118 additions & 0 deletions sql-migrate/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,118 @@
---
title: sql-migrate
homepage: https://github.com/therootcompany/golib/tree/main/cmd/sql-migrate
tagline: |
sql-migrate: A lightweight, feature-branch-friendly SQL migrator.
---

To update or switch versions, run `webi sql-migrate@stable` (or `@v2`, `@beta`,
etc).

### Files

These are the files that are created and/or modified with this installer:

```text
~/.config/envman/PATH.env
~/.local/bin/sql-migrate
~/.local/opt/sql-migrate-VERSION/bin/sql-migrate

<PROJECT-DIR>/migrations.log
<PROJECT-DIR>/sql/migrations/<yyyy-mm-dd>-<number>_<name>.<up|down>.sql
```

## Cheat Sheet

> `sql-migrate` is a lightweight migration tool that gets out of your way - it
> works with your existing SQL tools and allows working with distinct sets of
> migrations, such as is typical in feature branches.

### How to use sql-migrate

Migration commands output a POSIX shell script which should be run with `sh`.

`init`, `sync`, `up`, `down`, `list`, `status`

```sh
# Initialize a migration directory
sql-migrate -d ./sql/migrations/ init --sql-command psql

# Export ENVs for your database
export PG_URL='postgres://user:pass@example.com/dbname?sslmode=require&sslnegotiation=direct'

# Create a new migration (auto-generates up/down pair with incrementing number)
sql-migrate -d ./sql/migrations/ create add-users-table

# SELECT and log existing migrations
sql-migrate -d ./sql/migrations/ sync | sh

# Run all pending migrations up
sql-migrate -d ./sql/migrations/ up | sh

# Run 3 migrations up
sql-migrate -d ./sql/migrations/ up 3 | sh

# Run 1 migration down (roll back)
sql-migrate -d ./sql/migrations/ down | sh

# Run 2 migrations down
sql-migrate -d ./sql/migrations/ down 2 | sh

# List all migrations
sql-migrate -d ./sql/migrations/ list

# See which migrations have been applied
sql-migrate -d ./sql/migrations/ status
```

### Migration directory layout

Migrations follow the naming format
`<yyyy-mm-dd>-<number>_<name>.<up|down>.sql`:

```text
sql/
├── migrations.log # transaction log (auto-managed)
└── migrations/
├── 0001-01-01-001000_init-migrations.up.sql # generated by 'init' (has config vars)
├── 2021-02-03-001000_init-app.up.sql
├── 2021-02-03-001000_init-app.down.sql
├── 2021-02-03-002000_add-products.up.sql
├── 2021-02-03-002000_add-products.down.sql
└── 2021-02-03-003000_add-customers.up.sql
```

The initial `0001-01-01-001000_init-migrations.up.sql` migration contains
configuration variables:

```sql
-- migrations_log: ./sql/migrations.log
-- sql_command: psql "$PG_URL" -v ON_ERROR_STOP=on --no-align --tuples-only --file %s
```

Environment variables by database:

- PostgreSQL: `PG_URL` (auth url), `PGOPTIONS` (to set `schema` and other
specific options)
- SQLite3: `SQLITE_PATH`
- SQL Server: `SQLCMDSERVER`, `SQLCMDDATABASE`, `SQLCMDUSER`, `SQLCMDPASSWORD`
- MySQL / MariaDB: `MY_CNF` (path to `my.cnf`, containing credentials)

### Database client compatibility

The `--sql-command` flag tells sql-migrate how to talk to your database:

```sh
sql-migrate -d ./sql/migrations/ init --sql-command psql
```

The following clients are known and will have the correct options applied:

- psql (PostgreSQL)
- sqlite3
- sqlcmd (mssql / Microsoft SQL Server)
- mariadb / mysql

Since the migrations run via shell commands, you can make `sql-migrate`
compatible with any SQL client by setting `sql_command` in
`migrations/0001-01-01-001000_init-migrations.up.sql`.
78 changes: 78 additions & 0 deletions sql-migrate/SKILL.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,78 @@
---
name: sql-migrate
description:
Manage SQL database migrations as plain .sql files with a transaction log. Use
when asked about sql-migrate, database migrations, or how to set up migration
tooling.
---

# sql-migrate (CLI)

The agent skill is embedded in the help output.

Run `sql-migrate --help` use its output to guide usage decisions.

The `up`, `down`, and `sync` subcommands produce POSIX shell scripts - pipe them
to `sh` to run.

## Migration layout

Migrations follow the naming format
`<yyyy-mm-dd>-<number>_<name>.<up|down>.sql`:

```
sql/
├── migrations.log # transaction log (auto-managed)
└── migrations/
├── 0001-01-01-001000_init-migrations.up.sql # generated by 'init'
├── 2021-02-03-001000_init-app.up.sql
├── 2021-02-03-001000_init-app.down.sql
└── ...
```

The initial migration file contains configuration variables:

```sql
-- migrations_log: ./sql/migrations.log
-- sql_command: psql "$PG_URL" -v ON_ERROR_STOP=on --no-align --tuples-only --file %s
```

## Migration file structure

The migration files contain their own management, including a randomly-generated
id:

```sql
-- change_me (up)
SELECT 'place your UP migration here';

-- leave this as the last line
INSERT INTO _migrations (name, id) VALUES ('2026-05-27-002000_change_me', 'e22295e5');
```

## Environment variables by database

- PostgreSQL: `PG_URL` (auth URL), `PGOPTIONS` (set `schema` and other options)
- SQLite3: `SQLITE_PATH`
- SQL Server: `SQLCMDSERVER`, `SQLCMDDATABASE`, `SQLCMDUSER`, `SQLCMDPASSWORD`
- MySQL / MariaDB: `MY_CNF` (path to `my.cnf` containing credentials)

## SQL client extensibility

The `--sql-command` flag tells sql-migrate how to talk to your database. Known
clients (`psql`, `sqlite3`, `sqlcmd`, `mariadb`/`mysql`) have correct options
applied automatically. Since migrations run via shell commands, you can make
sql-migrate compatible with any SQL client by setting `sql_command` in the init
migration file.

# sqlmigrate (Go module)

See `go doc` for each independent module (golib is a monorepo):

- `github.com/therootcompany/golib/database/sqlmigrate/v2`
- `github.com/therootcompany/golib/database/sqlmigrate/pgmigrate`
- `github.com/therootcompany/golib/database/sqlmigrate/litemigrate`
- `github.com/therootcompany/golib/database/sqlmigrate/msmigrate`
- `github.com/therootcompany/golib/database/sqlmigrate/mymigrate`

DO NOT search the parent golib module.
47 changes: 47 additions & 0 deletions sql-migrate/install.ps1
Original file line number Diff line number Diff line change
@@ -0,0 +1,47 @@
#!/usr/bin/env pwsh

######################
# Install sql-migrate #
######################

$pkg_cmd_name = "sql-migrate"

$pkg_dst_cmd = "$Env:USERPROFILE\.local\bin\sql-migrate.exe"
$pkg_dst = "$pkg_dst_cmd"

$pkg_src_cmd = "$Env:USERPROFILE\.local\opt\sql-migrate-v$Env:WEBI_VERSION\bin\sql-migrate.exe"
$pkg_src_bin = "$Env:USERPROFILE\.local\opt\sql-migrate-v$Env:WEBI_VERSION\bin"
$pkg_src_dir = "$Env:USERPROFILE\.local\opt\sql-migrate-v$Env:WEBI_VERSION"
$pkg_src = "$pkg_src_cmd"

New-Item "$Env:USERPROFILE\Downloads\webi" -ItemType Directory -Force | Out-Null
$pkg_download = "$Env:USERPROFILE\Downloads\webi\$Env:WEBI_PKG_FILE"

# Fetch archive
if (!(Test-Path -Path "$Env:USERPROFILE\Downloads\webi\$Env:WEBI_PKG_FILE")) {
Write-Output "Downloading sql-migrate from $Env:WEBI_PKG_URL to $pkg_download"
& curl.exe -A "$Env:WEBI_UA" -fsSL "$Env:WEBI_PKG_URL" -o "$pkg_download.part"
& Move-Item "$pkg_download.part" "$pkg_download"
}

if (!(Test-Path -Path "$pkg_src_cmd")) {
Write-Output "Installing sql-migrate"

Push-Location .local\tmp

Remove-Item -Path ".\sql-migrate-v*" -Recurse -ErrorAction Ignore
Remove-Item -Path ".\sql-migrate.exe" -Recurse -ErrorAction Ignore

Write-Output "Unpacking $pkg_download"
& tar xf "$pkg_download"

Write-Output "Install Location: $pkg_src_cmd"
New-Item "$pkg_src_bin" -ItemType Directory -Force | Out-Null
Move-Item -Path ".\sql-migrate.exe" -Destination "$pkg_src_bin"

Pop-Location
}

Write-Output "Copying into '$pkg_dst_cmd' from '$pkg_src_cmd'"
Remove-Item -Path "$pkg_dst_cmd" -Recurse -ErrorAction Ignore | Out-Null
Copy-Item -Path "$pkg_src" -Destination "$pkg_dst" -Recurse
33 changes: 33 additions & 0 deletions sql-migrate/install.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,33 @@
#!/bin/sh
# shellcheck disable=SC2034

set -e
set -u

__init_sql_migrate() {
pkg_cmd_name="sql-migrate"

pkg_dst_cmd="${HOME}/.local/bin/sql-migrate"
pkg_dst="${pkg_dst_cmd}"

pkg_src_cmd="${HOME}/.local/opt/sql-migrate-v${WEBI_VERSION}/bin/sql-migrate"
pkg_src_dir="${HOME}/.local/opt/sql-migrate-v${WEBI_VERSION}"
pkg_src="${pkg_src_cmd}"

pkg_install() {
pkg_src_bin=$(dirname "${pkg_src_cmd}")
mkdir -p "${pkg_src_bin}"
mv ./sql-migrate "${pkg_src_cmd}"
}

# pkg_get_current_version is recommended, but (soon) not required
pkg_get_current_version() {
# 'sql-migrate version' has output in this format:
# sql-migrate v0.0.0-dev 0000000 (0001-01-01)
# This trims it down to just the version number:
# v0.0.0-dev
sql-migrate version 2> /dev/null | head -n 1 | cut -d ' ' -f 2
}
}

__init_sql_migrate
2 changes: 2 additions & 0 deletions sql-migrate/releases.conf
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
github_releases = therootcompany/golib
tag_prefix = cmd/sql-migrate/v
Loading