Skip to content

Commit 4d94abf

Browse files
proofrockproofrock
committed
Finish v0.15.0
2 parents e61384a + d62af1f commit 4d94abf

File tree

13 files changed

+226
-90
lines changed

13 files changed

+226
-90
lines changed

‎BUILDING.md‎

Lines changed: 51 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,51 @@
1+
# How to build ws4sqlite
2+
3+
The build system uses `make`. There are two kinds of targets:
4+
5+
- "direct" builds, that use go(lang) tooling to build a statically or dinamically linked binary or set of binaries;
6+
- docker-based builds, that build binaries or containers under a docker environment, using the "official" golang docker
7+
image as a base.
8+
9+
All binaries generated for distribution are statically linked.
10+
11+
## Direct targets
12+
13+
#### make build
14+
15+
Builds a statically linked binary under the current architecture, in the `bin/` folder.
16+
17+
#### make build-nostatic
18+
19+
Builds a dinamically linked binary under the current architecture, in the `bin/` folder.
20+
21+
#### make zbuild-all
22+
23+
Builds statically linked binaries for all the supported OSs/architectures, also creating the distribution archives in
24+
the `bin/` folder.
25+
26+
## Docker targets
27+
28+
In general, docker images are built in the debian-based official docker image; the generated binary is then distributed
29+
in a `distroless/static-debian11` image. The final size is about 20Mb.
30+
31+
*NB: buildx must be installed/enabled*
32+
*NB2: the **current** sources will be copied to the docker context.*
33+
34+
#### make docker-test-and-zbuild-all
35+
36+
Builds the distribution archives for the supported OSs/architectures (see `zbuild-all`) under a docker environment, and
37+
copies them in the `bin/` folder.
38+
39+
#### make docker
40+
41+
Builds a docker image (called `local_ws4sqlite:latest`) in the current architecture.
42+
43+
#### make docker-multiarch
44+
45+
Builds docker images for AMD64, ARMv7 and ARM64v8. The images are named like the official ones, i.e.
46+
`germanorizzo/ws4sqlite:v0.xx.xx-xxx`.
47+
48+
#### make docker-multiarch
49+
#### make docker-devel
50+
51+
Reserved, for publishing in Docker Hub.

‎CHANGELOG.md‎

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,12 @@
1+
## v 0.15.0
2+
*2023-05-07, Windhoek*
3+
4+
- Issue #29: Allow different path for db companion YAML file
5+
- Issue #28: Rework build system
6+
- Issue #27: Add support for linux/S390x
7+
- Documentation regarding #33 (Dockerfile: allow specifying UID/GID with which ws4sqlite starts)
8+
- Library updates
9+
110
## v0.14.2
211
*2023-4-21, Busan*
312

‎Dockerfile.binaries‎

Lines changed: 17 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,17 @@
1+
# Used by make docker-test-and-zbuild-all. See BUILDING.md
2+
3+
FROM golang:latest as build
4+
5+
RUN apt-get update
6+
RUN apt-get full-upgrade -y
7+
RUN apt-get install -y zip
8+
9+
WORKDIR /go/src/app
10+
COPY . .
11+
12+
RUN make test
13+
RUN make zbuild-all
14+
15+
# Now copy it into our base image.
16+
FROM scratch AS export
17+
COPY --from=build /go/src/app/bin/* .

‎Dockerfile‎ renamed to ‎Dockerfile.containers‎

Lines changed: 5 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1,15 +1,17 @@
1-
# docker build -t ws4sqlite .
1+
# See BUILDING.md
22

33
FROM golang:latest as build
44

55
WORKDIR /go/src/app
66
COPY . .
77

8-
# RUN CGO_ENABLED=0 go build -o /go/bin/ws4sqlite -trimpath
9-
RUN make build-nostatic
8+
RUN make build
109

1110
# Now copy it into our base image.
1211
FROM gcr.io/distroless/static-debian11
1312
COPY --from=build /go/src/app/bin/ws4sqlite /
1413

14+
EXPOSE 12321
15+
VOLUME /data
16+
1517
ENTRYPOINT ["/ws4sqlite"]

‎Makefile‎

Lines changed: 46 additions & 39 deletions
Original file line numberDiff line numberDiff line change
@@ -8,79 +8,86 @@ cleanup:
88
- rm -r bin
99
- rm src/ws4sqlite
1010

11+
upd-libraries:
12+
cd src; go get -u
13+
cd src; go mod tidy
14+
1115
build:
1216
make build-prepare
13-
cd src; CGO_ENABLED=0 go build -a -tags netgo,osusergo -ldflags '-w -extldflags "-static"' -o ws4sqlite -trimpath
17+
cd src; CGO_ENABLED=0 go build -trimpath
1418
mv src/ws4sqlite bin/
1519

16-
zbuild:
17-
make build
18-
cd bin; 7zr a -mx9 -t7z ws4sqlite-v0.14.2-`uname -s|tr '[:upper:]' '[:lower:]'`-`uname -m`.7z ws4sqlite
19-
2020
build-nostatic:
2121
make build-prepare
22-
cd src; CGO_ENABLED=0 go build -o ws4sqlite -trimpath
22+
cd src; go build -trimpath
2323
mv src/ws4sqlite bin/
2424

25-
zbuild-nostatic:
26-
make build-nostatic
27-
cd bin; 7zr a -mx9 -t7z ws4sqlite-v0.14.2-`uname -s|tr '[:upper:]' '[:lower:]'`-`uname -m`.7z ws4sqlite
28-
2925
zbuild-all:
3026
make build-prepare
31-
cd src; CGO_ENABLED=0 GOOS=linux GOARCH=amd64 go build -a -tags netgo,osusergo -ldflags '-w -extldflags "-static"' -trimpath
32-
cd src; tar czf ../bin/ws4sqlite-v0.14.2-linux-amd64.tar.gz ws4sqlite
27+
cd src; CGO_ENABLED=0 GOOS=linux GOARCH=amd64 go build -trimpath
28+
cd src; tar czf ../bin/ws4sqlite-v0.15.0-linux-amd64.tar.gz ws4sqlite
29+
rm src/ws4sqlite
30+
cd src; CGO_ENABLED=0 GOOS=linux GOARCH=arm go build -trimpath
31+
cd src; tar czf ../bin/ws4sqlite-v0.15.0-linux-arm.tar.gz ws4sqlite
3332
rm src/ws4sqlite
34-
cd src; CGO_ENABLED=0 GOOS=linux GOARCH=arm go build -a -tags netgo,osusergo -ldflags '-w -extldflags "-static"' -trimpath
35-
cd src; tar czf ../bin/ws4sqlite-v0.14.2-linux-arm.tar.gz ws4sqlite
33+
cd src; CGO_ENABLED=0 GOOS=linux GOARCH=arm64 go build -trimpath
34+
cd src; tar czf ../bin/ws4sqlite-v0.15.0-linux-arm64.tar.gz ws4sqlite
3635
rm src/ws4sqlite
37-
cd src; CGO_ENABLED=0 GOOS=linux GOARCH=arm64 go build -a -tags netgo,osusergo -ldflags '-w -extldflags "-static"' -trimpath
38-
cd src; tar czf ../bin/ws4sqlite-v0.14.2-linux-arm64.tar.gz ws4sqlite
36+
cd src; CGO_ENABLED=0 GOOS=linux GOARCH=riscv64 go build -trimpath
37+
cd src; tar czf ../bin/ws4sqlite-v0.15.0-linux-riscv64.tar.gz ws4sqlite
3938
rm src/ws4sqlite
40-
cd src; CGO_ENABLED=0 GOOS=linux GOARCH=riscv64 go build -a -tags netgo,osusergo -ldflags '-w -extldflags "-static"' -trimpath
41-
cd src; tar czf ../bin/ws4sqlite-v0.14.2-linux-riscv64.tar.gz ws4sqlite
39+
cd src; CGO_ENABLED=0 GOOS=linux GOARCH=s390x go build -trimpath
40+
cd src; tar czf ../bin/ws4sqlite-v0.15.0-linux-s390x.tar.gz ws4sqlite
4241
rm src/ws4sqlite
4342
cd src; CGO_ENABLED=0 GOOS=darwin GOARCH=amd64 go build -trimpath
44-
cd src; zip -9 ../bin/ws4sqlite-v0.14.2-darwin-amd64.zip ws4sqlite
43+
cd src; zip -9 ../bin/ws4sqlite-v0.15.0-darwin-amd64.zip ws4sqlite
4544
rm src/ws4sqlite
4645
cd src; CGO_ENABLED=0 GOOS=darwin GOARCH=arm64 go build -trimpath
47-
cd src; zip -9 ../bin/ws4sqlite-v0.14.2-darwin-arm64.zip ws4sqlite
46+
cd src; zip -9 ../bin/ws4sqlite-v0.15.0-darwin-arm64.zip ws4sqlite
4847
rm src/ws4sqlite
4948
cd src; CGO_ENABLED=0 GOOS=windows GOARCH=amd64 go build -trimpath
50-
cd src; zip -9 ../bin/ws4sqlite-v0.14.2-win-amd64.zip ws4sqlite.exe
49+
cd src; zip -9 ../bin/ws4sqlite-v0.15.0-win-amd64.zip ws4sqlite.exe
5150
rm src/ws4sqlite.exe
5251
cd src; CGO_ENABLED=0 GOOS=windows GOARCH=arm64 go build -trimpath
53-
cd src; zip -9 ../bin/ws4sqlite-v0.14.2-win-arm64.zip ws4sqlite.exe
52+
cd src; zip -9 ../bin/ws4sqlite-v0.15.0-win-arm64.zip ws4sqlite.exe
5453
rm src/ws4sqlite.exe
5554
cd src; CGO_ENABLED=0 GOOS=freebsd GOARCH=amd64 go build -trimpath
56-
cd src; tar czf ../bin/ws4sqlite-v0.14.2-freebsd-amd64.tar.gz ws4sqlite
55+
cd src; tar czf ../bin/ws4sqlite-v0.15.0-freebsd-amd64.tar.gz ws4sqlite
5756
rm src/ws4sqlite
5857
cd src; CGO_ENABLED=0 GOOS=freebsd GOARCH=arm64 go build -trimpath
59-
cd src; tar czf ../bin/ws4sqlite-v0.14.2-freebsd-arm64.tar.gz ws4sqlite
58+
cd src; tar czf ../bin/ws4sqlite-v0.15.0-freebsd-arm64.tar.gz ws4sqlite
6059
rm src/ws4sqlite
6160

6261
test:
6362
cd src; go test -v -timeout 6m
6463

6564
docker:
66-
sudo docker buildx build --no-cache -t local_ws4sqlite:latest .
65+
sudo docker buildx build -f Dockerfile.containers --no-cache -t local_ws4sqlite:latest .
66+
67+
docker-multiarch:
68+
sudo docker run --privileged --rm tonistiigi/binfmt --install arm64,arm
69+
sudo docker buildx build -f Dockerfile.containers --no-cache -t germanorizzo/ws4sqlite:v0.15.0-amd64 .
70+
sudo docker buildx build -f Dockerfile.containers --no-cache --platform linux/arm/v7 -t germanorizzo/ws4sqlite:v0.15.0-arm .
71+
sudo docker buildx build -f Dockerfile.containers --no-cache --platform linux/arm64/v8 -t germanorizzo/ws4sqlite:v0.15.0-arm64 .
6772

6873
docker-publish:
69-
## Prepare system with:
70-
## (verify which is latest at https://hub.docker.com/r/docker/binfmt/tags)
71-
sudo docker run --privileged --rm docker/binfmt:a7996909642ee92942dcd6cff44b9b95f08dad64
72-
sudo docker buildx build --no-cache -t germanorizzo/ws4sqlite:v0.14.2-amd64 .
73-
sudo docker buildx build --no-cache --platform linux/arm/v7 -t germanorizzo/ws4sqlite:v0.14.2-arm .
74-
sudo docker buildx build --no-cache --platform linux/arm64/v8 -t germanorizzo/ws4sqlite:v0.14.2-arm64 .
75-
sudo docker push germanorizzo/ws4sqlite:v0.14.2-amd64
76-
sudo docker push germanorizzo/ws4sqlite:v0.14.2-arm
77-
sudo docker push germanorizzo/ws4sqlite:v0.14.2-arm64
78-
sudo docker manifest create -a germanorizzo/ws4sqlite:v0.14.2 germanorizzo/ws4sqlite:v0.14.2-amd64 germanorizzo/ws4sqlite:v0.14.2-arm germanorizzo/ws4sqlite:v0.14.2-arm64
79-
sudo docker manifest push germanorizzo/ws4sqlite:v0.14.2
74+
make docker-multiarch
75+
sudo docker push germanorizzo/ws4sqlite:v0.15.0-amd64
76+
sudo docker push germanorizzo/ws4sqlite:v0.15.0-arm
77+
sudo docker push germanorizzo/ws4sqlite:v0.15.0-arm64
78+
sudo docker manifest create -a germanorizzo/ws4sqlite:v0.15.0 germanorizzo/ws4sqlite:v0.15.0-amd64 germanorizzo/ws4sqlite:v0.15.0-arm germanorizzo/ws4sqlite:v0.15.0-arm64
79+
sudo docker manifest push germanorizzo/ws4sqlite:v0.15.0
8080
- sudo docker manifest rm germanorizzo/ws4sqlite:latest
81-
sudo docker manifest create germanorizzo/ws4sqlite:latest germanorizzo/ws4sqlite:v0.14.2-amd64 germanorizzo/ws4sqlite:v0.14.2-arm germanorizzo/ws4sqlite:v0.14.2-arm64
81+
sudo docker manifest create germanorizzo/ws4sqlite:latest germanorizzo/ws4sqlite:v0.15.0-amd64 germanorizzo/ws4sqlite:v0.15.0-arm germanorizzo/ws4sqlite:v0.15.0-arm64
8282
sudo docker manifest push germanorizzo/ws4sqlite:latest
8383

8484
docker-devel:
85-
sudo docker buildx build --no-cache -t germanorizzo/ws4sqlite:edge .
86-
sudo docker push germanorizzo/ws4sqlite:edge
85+
sudo docker buildx build -f Dockerfile.containers --no-cache -t germanorizzo/ws4sqlite:edge .
86+
sudo docker push germanorizzo/ws4sqlite:edge
87+
88+
docker-test-and-zbuild-all:
89+
sudo docker buildx build -f Dockerfile.binaries --target export -t tmp_binaries_build . --output bin
90+
91+
docker-cleanup:
92+
sudo docker builder prune -af
93+
sudo docker image prune -af

‎README.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -72,7 +72,7 @@ Obtaining an answer of
7272
- [**Embedded web server**](https://germ.gitbook.io/ws4sqlite/documentation/web-server) to directly serve web pages that can access ws4sqlite without CORS;
7373
- Compact codebase;
7474
- Comprehensive test suite (`make test`);
75-
- 10 os's/arch's directly supported;
75+
- 11 os's/arch's directly supported;
7676
- [**Docker images**](https://germ.gitbook.io/ws4sqlite/documentation/installation/docker), for amd64, arm and arm64.
7777

7878
# Security Features

‎src/cli.go‎

Lines changed: 26 additions & 24 deletions
Original file line numberDiff line numberDiff line change
@@ -22,7 +22,6 @@ import (
2222
"path/filepath"
2323
"strings"
2424

25-
"github.com/mitchellh/go-homedir"
2625
mllog "github.com/proofrock/go-mylittlelogger"
2726
"gopkg.in/yaml.v2"
2827
)
@@ -70,30 +69,33 @@ func parseCLI() config {
7069
}
7170

7271
var ret config
73-
var err error
7472

7573
// Fail fast
7674
if len(dbFiles)+len(memDb) == 0 && *serveDir == "" {
7775
mllog.Fatal("no database and no dir to serve specified")
7876
}
7977

8078
for i := range dbFiles {
79+
// if there's no ":", second is empty ("")
80+
dbFile, yamlFile := splitOnColon(dbFiles[i])
81+
8182
// resolves '~'
82-
dbFiles[i], err = homedir.Expand(dbFiles[i])
83-
if err != nil {
84-
mllog.Fatal("in expanding database file path: ", err.Error())
85-
}
83+
dbFile = expandHomeDir(dbFile, "database file")
8684

87-
dir := filepath.Dir(dbFiles[i])
85+
dir := filepath.Dir(dbFile)
8886

8987
//strips the extension from the file name (see issue #11)
90-
id := strings.TrimSuffix(filepath.Base(dbFiles[i]), filepath.Ext(dbFiles[i]))
88+
id := strings.TrimSuffix(filepath.Base(dbFile), filepath.Ext(dbFile))
9189

9290
if len(id) == 0 {
9391
mllog.Fatal("base filename cannot be empty")
9492
}
9593

96-
yamlFile := filepath.Join(dir, id+".yaml")
94+
if yamlFile == "" {
95+
yamlFile = filepath.Join(dir, id+".yaml")
96+
} else {
97+
yamlFile = expandHomeDir(yamlFile, "companion file")
98+
}
9799

98100
var dbConfig db
99101
if fileExists(yamlFile) {
@@ -105,27 +107,24 @@ func parseCLI() config {
105107
if err = yaml.Unmarshal(cfgData, &dbConfig); err != nil {
106108
mllog.Fatal("in parsing config file: ", err.Error())
107109
}
108-
109-
dbConfig.HasConfigFile = true
110+
} else {
111+
yamlFile = ""
110112
}
111113

112114
dbConfig.Id = id
113-
dbConfig.Path = dbFiles[i]
115+
dbConfig.Path = dbFile
116+
dbConfig.CompanionFilePath = yamlFile
114117
ret.Databases = append(ret.Databases, dbConfig)
115118
}
116119

117120
for i := range memDb {
118-
components := append(strings.SplitN(memDb[i], ":", 2), "")
119-
// if there's no ":", second is empty
120-
id, yamlFile := components[0], components[1]
121+
// if there's no ":", second is empty ("")
122+
id, yamlFile := splitOnColon(memDb[i])
121123

122124
var dbConfig db
123125
if yamlFile != "" {
124126
// resolves '~'
125-
yamlFile, err = homedir.Expand(yamlFile)
126-
if err != nil {
127-
mllog.Fatal("in expanding mem-db yaml file path: ", err.Error())
128-
}
127+
yamlFile = expandHomeDir(yamlFile, "mem-db yaml file")
129128

130129
if !fileExists(yamlFile) {
131130
mllog.Fatal("mem-db yaml file does not exist")
@@ -139,20 +138,23 @@ func parseCLI() config {
139138
if err = yaml.Unmarshal(cfgData, &dbConfig); err != nil {
140139
mllog.Fatal("in parsing config file: ", err.Error())
141140
}
142-
143-
dbConfig.HasConfigFile = true
144141
}
145142

146143
dbConfig.Id = id
147144
dbConfig.Path = ":memory:"
145+
dbConfig.CompanionFilePath = yamlFile
148146
ret.Databases = append(ret.Databases, dbConfig)
149147
}
150148

151149
if *serveDir != "" {
152-
if !dirExists(*serveDir) {
153-
mllog.Fatalf("directory to serve as HTTP does not exist: %s", *serveDir)
150+
sd := *serveDir
151+
// resolves '~'
152+
sd = expandHomeDir(sd, "directory to serve")
153+
154+
if !dirExists(sd) {
155+
mllog.Fatalf("directory to serve does not exist: %s", *serveDir)
154156
}
155-
ret.ServeDir = serveDir
157+
ret.ServeDir = &sd
156158
}
157159

158160
// embed the cli parameters in the config

‎src/cli_test.go‎

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -122,7 +122,7 @@ func TestConfigs(t *testing.T) {
122122

123123
assert(t, cfg.Databases[0].Id == "test1", "the db has a wrong id")
124124
assert(t, cfg.Databases[0].Path == "../test/test1.db", "the db has not the correct Path")
125-
assert(t, cfg.Databases[0].HasConfigFile, "the db is not correctly marked regarding having config file")
125+
assert(t, cfg.Databases[0].CompanionFilePath != "", "the db is not correctly marked regarding having config file")
126126
assert(t, cfg.Databases[0].Auth.Mode == authModeHttp, "the db has not the correct Auth Mode")
127127
assert(t, cfg.Databases[0].Auth.ByQuery == "", "the db has ByQuery with a value")
128128
assert(t, len(cfg.Databases[0].Auth.ByCredentials) == 2, "the db has not the correct number of credentials")
@@ -146,7 +146,7 @@ func TestConfigs(t *testing.T) {
146146

147147
assert(t, cfg.Databases[1].Id == "test2", "the db has a wrong id")
148148
assert(t, cfg.Databases[1].Path == "../test/test2.db", "the db has not the correct Path")
149-
assert(t, !cfg.Databases[1].HasConfigFile, "the db is not correctly marked regarding having config file")
149+
assert(t, cfg.Databases[1].CompanionFilePath == "", "the db is not correctly marked regarding having config file")
150150
assert(t, cfg.Databases[1].Auth == nil, "the db has not the correct Auth Mode")
151151
assert(t, !cfg.Databases[1].DisableWALMode, "the db has not the correct WAL mode")
152152
assert(t, !cfg.Databases[1].ReadOnly, "the db has not the correct ReadOnly value")
@@ -158,7 +158,7 @@ func TestConfigs(t *testing.T) {
158158

159159
assert(t, cfg.Databases[2].Id == "mem1", "the db has a wrong id")
160160
assert(t, cfg.Databases[2].Path == ":memory:", "the db is not on memory")
161-
assert(t, cfg.Databases[2].HasConfigFile, "the db is not correctly marked regarding having config file")
161+
assert(t, cfg.Databases[2].CompanionFilePath != "", "the db is not correctly marked regarding having config file")
162162
assert(t, cfg.Databases[2].Auth.Mode == authModeInline, "the db has not the correct Auth Mode")
163163
assert(t, cfg.Databases[2].Auth.ByQuery != "", "the db has ByQuery without a value")
164164
assert(t, len(cfg.Databases[2].Auth.ByCredentials) == 0, "the db has not the correct number of credentials")
@@ -172,7 +172,7 @@ func TestConfigs(t *testing.T) {
172172

173173
assert(t, cfg.Databases[3].Id == "mem2", "the db has a wrong id")
174174
assert(t, cfg.Databases[3].Path == ":memory:", "the db is not on memory")
175-
assert(t, !cfg.Databases[3].HasConfigFile, "the db is not correctly marked regarding having config file")
175+
assert(t, cfg.Databases[3].CompanionFilePath == "", "the db is not correctly marked regarding having config file")
176176
assert(t, cfg.Databases[3].Auth == nil, "the db has not the correct Auth Mode")
177177
assert(t, !cfg.Databases[3].DisableWALMode, "the db has not the correct WAL mode")
178178
assert(t, !cfg.Databases[3].ReadOnly, "the db has not the correct ReadOnly value")

0 commit comments

Comments
 (0)