Skip to content

README.md

Latest commit

 

History

History
361 lines (289 loc) · 14.2 KB

README.md

File metadata and controls

361 lines (289 loc) · 14.2 KB
Raw

traust-sdk (Go)

go get github.com/traust-security/traust-sdk/go@latest

Consumer setup is three handles, with no traust tables or types of your own:

Need Package You supply
Store and read traust artifacts v1/storage *sql.DB (+ optional storage.Resolver for reads); Init bootstraps the contracts DDL in traust_storage
Dispositions, events, countersign v1/ledger ledger URL + token
Run skills v1/skills a skills.Provider

Keep consumer tables for consumer data only and reference traust rows by binding_id / layer_id. Overview: ../README.md → Purpose.

Reader normalization review candidate

The unreleased v1/enums.Normalize reader interprets generated registry metadata, not constants. Placement and affected-contract review remain pending; this does not activate ledger reads or writers.

view, err := enums.Normalize("severity", "high")
if err != nil {
    return err
}
for i := 0; i < view.Len(); i++ {
    pair := view.At(i)
    fmt.Println(pair.Enum, pair.Value)
}

Import github.com/traust-security/traust-sdk/go/v1/enums. Use a registry document's name, such as remediation_effort, not its hyphenated filename. Replacements preserve declared order across rename/merge/split cases; retired keys work without a current constant. A one-way drop returns the original pair and view.Dropped=true. Unknown values in a known enum retain their exact spelling; an unknown enum returns an error. At returns a value, not shared mutable registry data, and known-enum calls allocate nothing. Out-of-range indices panic like slice indexing.

make generate produces normalization_gen.go from the same verified published contracts pin as the existing assets. That pin currently has no deprecations; no legacy effort-to-size relationship is inferred. Neutral shared cases exercise replacement metadata in isolated candidate generation, not the published assets. No runtime files, sibling checkout, event/identifier rewriting, free-text mapping or cross-field policy are introduced.

Skills SDK

Import a skill, plug in your provider, call it — typed input in, typed result out:

import "github.com/traust-security/traust-sdk/go/v1/skills"

provider := myK8sProvider(...)  // you implement skills.Provider

artifact, err := skills.Scan.Run(ctx, provider, skills.ScanInput{
    Repo: "https://github.com/org/repo",
    Ref:  "main",
})
report, err := artifact.Value()
if err != nil {
    return err
}
for _, finding := range report.Findings {
    fmt.Println(finding.Id, finding.Severity, finding.Title)
}

skills.Provider is the only interface you implement — it runs a skill somewhere (K8s Job, Docker, local subprocess, ...) and returns raw bytes:

type Provider interface {
    Execute(ctx context.Context, skill SkillMeta, input []byte) ([]byte, error)
}

The SDK validates the response and retains its exact bytes with the generated Go type. Artifact.Value decodes typed data; Artifact.Payload returns owned, byte-identical evidence suitable for storage or export.

Calling multiple skills

Bind the provider once with a Client:

client := skills.NewClient(provider)
report, _ := client.Scan(ctx, skills.ScanInput{...})
triage, _ := client.Triage(ctx, skills.TriageInput{...})

Testing

import "github.com/traust-security/traust-sdk/go/v1/skills/skillstest"

provider := skillstest.NewStaticProvider().
    WithScanResult(skillstest.FixtureReport())

report, _ := skills.Scan.Run(ctx, provider, skills.ScanInput{
    Repo: "https://github.com/org/repo",
    Ref:  "main",
})

Ledger SDK

Submit reports and human events to the ledger service:

import "github.com/traust-security/traust-sdk/go/v1/ledger"

client := ledger.NewHTTPClient("https://ledger.example.com",
    ledger.WithBearerToken(os.Getenv("LEDGER_TOKEN")),
)

// Machine lane — triage report (derives disposition events)
resp, err := client.SubmitTriageReport(ctx, ledger.TriageReportInput{
    ReportMeta: ledger.ReportMeta{
        LayerID:    "repo-a",
        SourceRef:  "findings/repo-a/triage.json",
        RecordedAt: time.Now().UTC().Format(time.RFC3339),
    },
    Report: triageReport,
})

// Human lane — countersign
resp, err = client.SubmitCountersign(ctx, ledger.CountersignInput{
    EventMeta: ledger.EventMeta{
        LayerID:    "repo-a",
        RecordedAt: time.Now().UTC().Format(time.RFC3339),
    },
    FindingRef:    "FIND-001",
    Verdict:       "true_positive",
    Justification: "Reviewed source and confirmed exploit path.",
})

// Sign a layer's merkle tree
signResp, err := client.SignLayer(ctx, "repo-a", ledger.SignOpts{})

Layers are created explicitly, each under the storage product_repo it belongs to, and found again by that product_repo:

_, err = client.InitializeLayer(ctx, "repo-a", ledger.InitializeInput{
    ProductRepoID: &productRepoID,
    Layer:         shell, // complete types.Layer with metadata and no events
})
ref, found, err := client.FindLayer(ctx, productRepoID) // found=false on 404
layers, err := client.ListLayers(ctx)                   // []LayerRef{LayerID, ProductRepoID}

Machine-lane report kinds: triage, validation. Human-lane event kinds: countersign, severity. Additional operations: BatchSubmit, ResolveReviewItem, ComputeFingerprints, StampEventIdentities, and SignLayer.

The built-in HTTP transport handles envelope formatting, lane routing, and auth. For custom transports, implement ledger.Provider and use ledger.NewClient(provider).

Testing

import "github.com/traust-security/traust-sdk/go/v1/ledger/ingesttest"

provider := ingesttest.NewStaticProvider().
    WithTriageResponse(ingesttest.FixtureTriageResponse())
client := ledger.NewClient(provider)

Query SDK

Read layers, findings, and verification results from the ledger service:

import "github.com/traust-security/traust-sdk/go/v1/ledger"

client := ledger.NewHTTPClient("https://ledger.example.com",
    ledger.WithBearerToken(os.Getenv("LEDGER_TOKEN")),
)

// Read resolved findings for a layer
findings, err := client.GetFindings(ctx, "repo-a")
// Paginated bulk findings
page, err := client.ListFindings(ctx, ledger.ListFindingsOpts{Limit: 50})
// Verify layer integrity
result, err := client.VerifyLayer(ctx, "repo-a", ledger.VerifyOpts{CheckSignatures: true})
// Resolve the authenticated actor without recording an event
actor, err := client.Whoami(ctx)

The built-in HTTP transport handles auth and path construction. For custom transports, implement ledger.Provider and use ledger.NewClient(provider).

Testing

import "github.com/traust-security/traust-sdk/go/v1/ledger/querytest"

provider := querytest.NewStaticProvider().
    WithFindingsResponse("repo-a", querytest.FixtureFindingsResponse())
client := ledger.NewClient(provider)

Storage SDK

Storage retains exact artifact evidence and its canonical relational projection in one SQLite or PostgreSQL transaction. PostgreSQL relations use the fixed traust_storage schema so application tables and search_path cannot redirect storage operations. SQLite uses the caller-selected database file as its physical namespace; a dedicated file is recommended. The caller owns the driver, DSN, pool settings, health checks, and database closure:

import (
    "database/sql"

    _ "github.com/jackc/pgx/v5/stdlib"
    "github.com/traust-security/traust-sdk/go/v1/storage"
)

db, err := sql.Open("pgx", dsn)
if err != nil {
    return err
}
defer db.Close()

// resolver implements storage.Resolver: it fetches bytes from the locations the
// producer (the harness) already wrote to. Register-only loaders omit it.
// storagetest.MemoryResolver stands in for tests.
client, err := storage.NewClient(ctx, db, storage.WithResolver(resolver))
if err != nil {
    return err
}
if err = client.Init(ctx); err != nil {
    return err
}
subjectID := "myapp:subject:" + subjectKey
runID := "myapp:run:" + runKey
result, err := client.SaveVulnFindings(ctx, storage.SaveVulnFindingsInput{
    Binding: storage.Binding{
        ScopeID:   "local",
        SubjectID: &subjectID,
        RunID:     &runID,
    },
    Artifact:   artifact,
    References: []string{"s3://sci-reports/scans/42/7/report.json"},
})

Every schema has named typed save and read operations. A named save validates the source bytes, computes their SHA-256 digest, records globally deduplicated artifact_evidence (digest and byte size), and creates a context-specific artifact_binding and projection in one SQL transaction.

Storage never writes artifact bytes. The producer that wrote them (the harness, into the deployment's locations.analysis_results or a results bucket) is the only writer. A save registers where they are: References are opaque strings recorded against the binding (artifact_location), never fetched or parsed at save time. A retry may add references; none is removed. A reference must keep resolving to these exact bytes: pin it (a digest key from storage.ObjectKey(prefix, digest), a commit, or an object version) rather than pointing at a path the next rescan overwrites.

Typed reads and GetPayload(ctx, bindingID) fetch through the configured storage.Resolver, trying references in registration order, and return the first bytes that match the recorded size and SHA-256. Overwritten locations fail with storage.ErrEvidenceCorrupt, missing ones with storage.ErrNotFound, and a client built without a resolver with storage.ErrNoResolver. GetBinding returns the role, references and ByteSize without fetching. SaveResult.AlreadyBound reports only whether that binding existed; evidence-level deduplication remains private.

The registry anchors artifacts to what a product ships. Register products, repos and the product_repo pairing (ref "" is the default branch); each call is idempotent on its natural key and returns the stored ID. FindProductRepo reads without creating, so loaders can reject what the registry does not know:

productID, err := client.RegisterProduct(ctx, "volsync", nil)
repoID, err := client.RegisterRepo(ctx, "https://github.com/backube/volsync")
productRepoID, err := client.RegisterProductRepo(ctx, storage.ProductRepo{
    ProductID: productID, RepoID: repoID, Ref: "release-0.14",
})
id, found, err := client.FindProductRepo(ctx, "volsync", "https://github.com/backube/volsync", "release-0.14")

RegisterProductRepoVersion and RegisterRepoOwner record inventory attributes and refresh on re-registration. Binding.ProductRepoID attaches an artifact to a registered product_repo (a foreign key) and Binding.CommitSHA records the commit it describes. Neither is part of the binding ID; a re-save must repeat both, and a supersession must keep the product_repo while the commit may change.

Binding.Role separates bindings that are otherwise identical, such as a report that is the baseline audit and one that is its cumulative findings-current restatement. Allowed roles come from the contracts profile (storage.ErrRoleNotAllowed otherwise); a role is part of the binding ID and must match across a supersession. Bindings without a role keep their IDs.

The exact-evidence boundary is the named Save call. Callers may perform optional processing, such as asking Ledger to stamp finding fingerprints, before creating the artifact passed to storage. Storage records the digest of those accepted bytes but does not compute fingerprints or treat them as storage-owned identity. When Ledger is present, consumers join disposition data by the binding's layer_id and the artifact-relative finding ID; fingerprints remain report evidence or Ledger-owned state rather than storage join keys.

Every artifact contract has a schema-specific SQL projection. Root fields become typed columns while nested values remain JSON until a query justifies child tables. Scoped views exercise cross-artifact binding relationships; all projections retain generated save and smoke-test coverage. Run-bound saves require caller-supplied subject and run IDs; layer saves require a Ledger layer ID. Identifiers are opaque UTF-8 strings, and omitted scope defaults to local.

Typed reads use BindingID to guard the stored schema interpretation. Use GetPayload(ctx, bindingID) only when raw bytes without a type claim are intended. Scoped view reads require an explicit scope list.

Data-only usage

For consumers who just need types or validation without invoking skills:

Package Contents
v1/types Generated Go structs for every schema in schemas/v1
v1/enums Generated shared vocabularies (severity, validity, verdicts, ...)
v1/validate Validate raw JSON bytes against an embedded v1 schema

Regenerating types/enums/validate/storage

make generate     # verified pinned contracts commit → generated Go outputs
make check-drift  # fails if generated output is stale

No sibling checkout is needed. make generate obtains the full immutable CONTRACTS_REF from CONTRACTS_REPO and validates that the checkout matches it exactly; it never generates from latest main. The checkout is reused from CONTRACTS_CACHE (by default the user cache) at CONTRACTS_SOURCE. All four variables are caller-overridable for fork, CI-cache, and pinned-ref workflows. Normal make test uses only repository fixtures and requires no network.

Run make test-integration for the storage suite against SQLite and PostgreSQL. The PostgreSQL suite uses one fixed local test DSN, recreates traust_storage for each test, and verifies that same-named application tables remain untouched. Start the database with:

podman run --name traust-postgres --rm -d -e POSTGRES_USER=traust -e POSTGRES_PASSWORD=traust-test-only -e POSTGRES_DB=traust_test -p 127.0.0.1:5432:5432 -v traust-postgres-data:/var/lib/postgresql/data docker.io/library/postgres:16

These are fixed local test-only credentials, not production/deployed secrets, and no DSN override is supported. make test-integration disables the Go test cache so PostgreSQL availability is checked on every run. PostgreSQL absence, connectivity failure, or authentication failure produces a visible skip while SQLite still executes. Once PostgreSQL accepts a connection, all version, initialization, and protocol failures fail.