Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
138 changes: 138 additions & 0 deletions AGENTS.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,138 @@
# PROJECT KNOWLEDGE BASE

**Generated:** 2026-02-12
**Commit:** 661961a5
**Branch:** main

## OVERVIEW

Official Go SDK for Stellar blockchain. Provides transaction building (`txnbuild`), API clients (`horizonclient`, `rpcclient`), ledger ingestion (`ingest`), and XDR type definitions. Previously a monorepo—services moved to separate repos (Oct 2025).

## STRUCTURE

```
go-stellar-sdk/
├── txnbuild/ # Transaction building API (83 Go files) - START HERE for tx construction
├── clients/ # Network clients
│ ├── horizonclient/ # Horizon REST API client (primary)
│ ├── rpcclient/ # Stellar RPC client
│ ├── stellarcore/ # Direct stellar-core client
│ └── stellartoml/ # stellar.toml parser
├── ingest/ # Ledger data ingestion library
│ └── ledgerbackend/ # Captive Core, RPC backend, buffered storage
├── xdr/ # XDR types (MOSTLY GENERATED - see xdr/AGENTS.md)
├── support/ # Internal utilities (db, http, config, errors, logging)
├── historyarchive/ # History archive access
├── protocols/ # Protocol response types (horizon, rpc, stellarcore)
├── processors/ # ETL-style data processors
├── strkey/ # Stellar key encoding (G*, S*, M*, P*, C* addresses)
├── keypair/ # Cryptographic key pair operations
├── tools/ # Standalone CLI utilities (not library code)
└── exp/ # DEPRECATED experimental packages
```

## WHERE TO LOOK

| Task | Location | Notes |
|------|----------|-------|
| Build transactions | `txnbuild/` | Start with `NewTransaction()` |
| Submit to Horizon | `clients/horizonclient/` | Use with txnbuild |
| Submit to RPC | `clients/rpcclient/` | Soroban smart contracts |
| Parse ledger data | `ingest/` | LedgerTransactionReader, CheckpointChangeReader |
| Run Captive Core | `ingest/ledgerbackend/` | CaptiveCoreConfig |
| XDR type helpers | `xdr/` | Hand-written helpers in non-generated files |
| Key encoding | `strkey/` | Decode/Encode/CheckValid |
| Sign transactions | `keypair/` | Full.Sign(), ParseAddress() |
| Network passphrases | `network/` | TestNetworkPassphrase, PublicNetworkPassphrase |
| Database utilities | `support/db/` | Session, batch insert builders |
| HTTP utilities | `support/http/` | Middleware, signed requests |

## CODE MAP

**Primary Public Interfaces:**

| Package | Key Types/Funcs | Role |
|---------|-----------------|------|
| `txnbuild` | `Transaction`, `NewTransaction()`, `Operation` | Construct and sign transactions |
| `horizonclient` | `Client`, `DefaultTestNetClient` | Query Horizon, submit transactions |
| `rpcclient` | `Client` | Interact with Stellar RPC |
| `ingest` | `LedgerTransactionReader`, `CheckpointChangeReader` | Parse ledger data |
| `ledgerbackend` | `CaptiveCoreConfig`, `NewCaptive()` | Run embedded stellar-core |
| `xdr` | All Stellar protocol types | Network data structures |
| `keypair` | `Full`, `FromAddress`, `Parse()`, `Random()` | Key management |
| `strkey` | `Decode()`, `Encode()`, `CheckValid()` | Address encoding |

## CONVENTIONS

**Deviations from standard Go:**
- No top-level `cmd/` or `pkg/`—CLIs under `tools/`, libs at root
- Nested `internal/` dirs (e.g., `strkey/internal/`) instead of single `/internal`
- Many `main.go` files are NOT executables—check `package` declaration
- Line length limit: 140 chars (relaxed)
- Function length: up to 100 lines allowed

**Error handling:**
- Mixed stdlib `fmt.Errorf("%w", err)` and `github.com/pkg/errors.Wrap()`
- Prefer `%w` for new code

**Formatting:**
- Run `goimports -w .` before PR (preferred over plain gofmt)
- CI enforces via `./gofmt.sh`

## ANTI-PATTERNS (THIS PROJECT)

| Don't | Do Instead |
|-------|------------|
| Edit `xdr/xdr_generated.go` | Modify `.x` files, run `make xdr` |
| Edit `*.pb.go` files | Modify `.proto` files, run `make generate-proto` |
| Use `xdr.UnmarshalBinary()` on MarshalBinaryBase64 output | Use matching decode method |
| Wrap `io.EOF` in xdr streaming | Return bare `io.EOF` |
| Use deprecated `PathPayment` | Use `PathPaymentStrictReceive` |
| Use deprecated `AllowTrust` | Use `SetTrustLineFlags` |
| Use `horizon` client | Use `horizonclient` |

**Generated code locations:**
- `xdr/xdr_generated.go` — XDR types (DO NOT EDIT)
- `gxdr/xdr_generated.go` — Alternative XDR (DO NOT EDIT)
- `*.pb.go` files — Protobuf (DO NOT EDIT)

## COMMANDS

```bash
# Format and lint (run before PR)
./gofmt.sh && ./gomod.sh && ./govet.sh && ./staticcheck.sh

# Run tests
go test ./... # All tests (some need postgres)
go test ./txnbuild # Just txnbuild (safe)
go test -race -cover ./... # CI command

# Regenerate XDR types
make xdr # Requires Docker

# Regenerate proto
make generate-proto

# Build all
go build ./...

# Run specific tool
go run ./tools/<tool>
```

## NOTES

**Test dependencies:**
- Some tests require PostgreSQL 12+ (set `PGHOST`, `PGUSER`, etc.)
- `ingest/ledgerbackend` tests may need stellar-core binary
- Fuzz tests under `xdr/fuzz/`

**Module path:** `github.com/stellar/go-stellar-sdk`

**Go version:** 1.24+ (tests against 1.24 and 1.25)

**Subdirectory AGENTS.md files exist for:**
- `txnbuild/AGENTS.md` — Transaction building details
- `xdr/AGENTS.md` — Generated vs manual code guidance
- `ingest/AGENTS.md` — Ingestion library patterns
- `clients/horizonclient/AGENTS.md` — Horizon client usage
89 changes: 89 additions & 0 deletions clients/horizonclient/AGENTS.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,89 @@
# horizonclient

HTTP client for Stellar Horizon API. Query network state, submit transactions.

## OVERVIEW

Primary client for interacting with Horizon servers. Use with `txnbuild` for complete transaction workflow.

## KEY TYPES

| Type | Purpose |
|------|---------|
| `Client` | Main client; set `HorizonURL` |
| `DefaultTestNetClient` | Pre-configured testnet client |
| `DefaultPublicNetClient` | Pre-configured pubnet client |
| `AccountRequest` | Query account details |
| `TransactionRequest` | Query transactions |
| `Error` | Horizon error with problem details |

## COMMON PATTERNS

```go
// Use default testnet client
client := horizonclient.DefaultTestNetClient

// Or configure custom
client := &horizonclient.Client{HorizonURL: "https://horizon.stellar.org"}

// Get account
account, err := client.AccountDetail(horizonclient.AccountRequest{
AccountID: "GABC...",
})

// Submit transaction (built with txnbuild)
resp, err := client.SubmitTransaction(tx)
// or from XDR
resp, err := client.SubmitTransactionXDR(base64XDR)

// Stream ledgers
client.StreamLedgers(ctx, horizonclient.LedgerRequest{}, func(ledger horizon.Ledger) {
// handle ledger
})
```

## REQUEST TYPES

| Request | Endpoint |
|---------|----------|
| `AccountRequest` | `/accounts/{id}` |
| `AccountsRequest` | `/accounts` (list) |
| `TransactionRequest` | `/transactions` |
| `OperationRequest` | `/operations` |
| `EffectRequest` | `/effects` |
| `LedgerRequest` | `/ledgers` |
| `OfferRequest` | `/offers` |
| `TradeRequest` | `/trades` |
| `PathsRequest` | `/paths` |
| `OrderBookRequest` | `/order_book` |
| `AssetRequest` | `/assets` |
| `ClaimableBalanceRequest` | `/claimable_balances` |

## ERROR HANDLING

```go
resp, err := client.SubmitTransaction(tx)
if err != nil {
if hzErr, ok := err.(*horizonclient.Error); ok {
// Access Horizon problem details
fmt.Println(hzErr.Problem.Title)
fmt.Println(hzErr.Problem.Extras["result_codes"])
}
}
```

## ANTI-PATTERNS

| Don't | Do Instead |
|-------|------------|
| Use deprecated `horizon` package | Use `horizonclient` |
| Ignore `*horizonclient.Error` type | Type assert for detailed error info |
| Hardcode Horizon URLs | Use `DefaultTestNetClient` / `DefaultPublicNetClient` |

## TESTING

```bash
go test ./clients/horizonclient # Unit tests (mocked)
```

Mock available: `horizonclient.MockClient` for testing without network.
100 changes: 100 additions & 0 deletions ingest/AGENTS.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,100 @@
# ingest

Ledger data ingestion library. Parse raw ledger data from Captive Core, RPC, or history archives.

## OVERVIEW

Build custom ingestion pipelines. Read transactions, state changes, and ledger metadata. Used by Horizon and custom analytics tools.

## ARCHITECTURE

```
[ Your Code ]
|
[ Readers ]
/ \
[ChangeReader] [TransactionReader]
| |
CheckpointChange LedgerTransaction
Reader Reader
|
[ LedgerBackend ]
|
|---------|---------|
Captive Buffered RPC
Core Storage Backend
```

## KEY TYPES

| Type | Purpose |
|------|---------|
| `LedgerTransactionReader` | Read transactions from a ledger |
| `CheckpointChangeReader` | Read state changes at checkpoints |
| `LedgerChangeReader` | Read changes within a single ledger |
| `Change` | Represents a ledger entry change (created/updated/removed) |
| `LedgerTransaction` | Transaction with result and metadata |

## BACKENDS (`ledgerbackend/`)

| Backend | Use Case |
|---------|----------|
| `CaptiveCoreBackend` | Run embedded stellar-core (most common) |
| `BufferedStorageBackend` | Read from pre-exported data lake |
| `RPCBackend` | Read from Stellar RPC |

## COMMON PATTERNS

```go
// Create Captive Core backend
backend, err := ledgerbackend.NewCaptive(ledgerbackend.CaptiveCoreConfig{
BinaryPath: "/usr/bin/stellar-core",
NetworkPassphrase: network.TestNetworkPassphrase,
HistoryArchiveURLs: network.TestNetworkhistoryArchiveURLs,
Toml: captiveCoreToml,
})
defer backend.Close()

// Prepare ledger range
err = backend.PrepareRange(ctx, ledgerbackend.BoundedRange(1000, 2000))

// Read transactions
reader, err := ingest.NewLedgerTransactionReader(ctx, backend, passphrase, seq)
for {
tx, err := reader.Read()
if err == io.EOF { break }
// Process tx
}
```

## WHERE TO LOOK

| Task | File |
|------|------|
| Read transactions | `ledger_transaction_reader.go` |
| Read state changes | `checkpoint_change_reader.go`, `ledger_change_reader.go` |
| Configure Captive Core | `ledgerbackend/captive_core_backend.go` |
| TOML config generation | `ledgerbackend/toml.go` |
| Buffered/lake backend | `ledgerbackend/buffered_storage_backend.go` |

## ANTI-PATTERNS

| Don't | Do Instead |
|-------|------------|
| Use `FeeChanges` directly | Use helper methods on `LedgerTransaction` |
| Ignore `PrepareRange` | Always prepare range before `GetLedger` |
| Skip `Close()` on backend | Defer close to avoid resource leaks |

## TESTING

```bash
go test ./ingest # Unit tests
go test ./ingest/ledgerbackend # Backend tests (may need stellar-core)
```

**Note:** Some tests require stellar-core binary or external services.

## TUTORIALS

Working examples in `ingest/tutorial/`:
- `expiring-sac-balances/` — Track expiring Soroban balances
68 changes: 68 additions & 0 deletions txnbuild/AGENTS.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,68 @@
# txnbuild

Transaction building API for Stellar network. Construct, sign, and serialize transactions.

## OVERVIEW

Primary SDK entry point for building transactions. Use `NewTransaction()` with operations, sign with keypairs, submit via `horizonclient`.

## KEY TYPES

| Type | Purpose |
|------|---------|
| `Transaction` | Immutable signed transaction; call `Sign()` to add signatures |
| `TransactionParams` | Input to `NewTransaction()`: source, ops, fee, preconditions |
| `FeeBumpTransaction` | Wrapper to increase fee on existing transaction |
| `Operation` | Interface implemented by all 20+ operation types |
| `Account` | Interface for source account (sequence number management) |
| `Preconditions` | TimeBounds, LedgerBounds, MinSequence, etc. |

## COMMON PATTERNS

```go
// Build transaction
tx, err := txnbuild.NewTransaction(txnbuild.TransactionParams{
SourceAccount: &account,
IncrementSequenceNum: true,
Operations: []txnbuild.Operation{&op},
BaseFee: txnbuild.MinBaseFee,
Preconditions: txnbuild.Preconditions{TimeBounds: txnbuild.NewTimeout(300)},
})

// Sign (returns NEW Transaction - immutable)
tx, err = tx.Sign(network.TestNetworkPassphrase, keypair)

// Serialize for submission
xdrBase64, err := tx.Base64()
```

## WHERE TO LOOK

| Task | File |
|------|------|
| Create transaction | `transaction.go` — `NewTransaction()` |
| Payment operation | `payment.go` |
| Path payments | `path_payment_strict_send.go`, `path_payment.go` |
| Account creation | `create_account.go` |
| Trust lines | `change_trust.go`, `set_trust_line_flags.go` |
| Offers/DEX | `manage_offer.go`, `manage_buy_offer.go` |
| Soroban invoke | `invoke_host_function.go` |
| Fee bumps | `transaction.go` — `NewFeeBumpTransaction()` |

## ANTI-PATTERNS

| Don't | Do Instead |
|-------|------------|
| Use `PathPayment` | Use `PathPaymentStrictReceive` (renamed) |
| Use `AllowTrust` | Use `SetTrustLineFlags` |
| Mutate Transaction after Sign | Transaction is immutable; Sign returns new instance |
| Forget `IncrementSequenceNum` | Set `true` unless managing sequence manually |

## TESTING

```bash
go test ./txnbuild # Unit tests (no external deps)
go test ./txnbuild -v # Verbose
```

Demo available: `go run ./txnbuild/cmd/demo`
Loading
Loading