Skip to content

docs: add local development guide#210

Closed
malon64 wants to merge 1 commit intomainfrom
agent/po-floe-20260306-1251-T1
Closed

docs: add local development guide#210
malon64 wants to merge 1 commit intomainfrom
agent/po-floe-20260306-1251-T1

Conversation

@malon64
Copy link
Owner

@malon64 malon64 commented Mar 6, 2026

Automated changes by agent.\n\nInstruction:\n# DEV TASK PROMPT (FLOE)

Project context

Repo: malon64/floe
Language: Rust (main), plus orchestrator packages in Python.

Floe is a config-driven data ingestion and technical validation tool.
It is primarily used by data engineers / analytics engineers who need to:

  • ingest files (CSV/JSON/Parquet/Excel and cloud-backed sources),
  • enforce schema and data-quality checks,
  • route accepted/rejected data outputs,
  • produce deterministic run reports for operations and monitoring,
  • integrate execution with orchestrators (e.g., Airflow/Dagster).

Core product goals:

  • predictable, reproducible ingestion runs,
  • explicit validation behavior driven by YAML config,
  • clear observability (logs + reports),
  • production-grade CLI ergonomics for local and CI usage.

When implementing tasks, preserve backwards compatibility of existing config semantics unless the task explicitly requests a breaking change.
Goal: produce clean, idiomatic, performant code consistent with existing architecture.

Task

Objective

Create docs/local-dev.md for Rust/Cargo local development that enables a first-time contributor to run Floe locally.

Scope (where to change)

  • docs/local-dev.md (new file only)

Non-goals (what NOT to touch)

  • Do not edit README.md in this task.
  • Do not change Rust application logic/code paths.
  • Do not modify CI/workflow configuration.

Implementation notes / constraints

  • Add sections: Prerequisites, Setup, Run, Test, Quality checks, Troubleshooting.
  • Commands should be copy-paste ready and aligned with this repo’s workspace:
    • cargo build
    • cargo run -p floe-cli -- --help
    • cargo run -p floe-cli -- run -c example/config.yml
    • cargo run -p floe-cli -- validate -c example/config.yml
    • cargo test
    • cargo fmt -- --check (or cargo fmt fallback if needed)
    • cargo clippy --workspace --all-targets -- -D warnings
  • Keep guidance concise and beginner-friendly.
  • Troubleshooting must include at least 3 actionable items.
  • Ensure all paths/commands are consistent with current repository layout.

Speed policy:

  • Keep discovery lean and proportional to task size.
  • For docs-only tasks (docs/**, README.md), do not run heavy runtime validation (cargo build, cargo run, cargo clippy, cargo test) unless explicitly requested in the task.
  • For docs-only tasks, prefer: quick path existence checks + markdown consistency checks + focused edit + commit.

Tests (mandatory)

What to add/update

  • Unit tests: put them under crates/floe-core/tests/unit/** (or the relevant crate’s tests dir).

  • Integration tests (only if needed for the change): crates/floe-core/tests/integration/**.

  • Docs-only task: no new Rust test files required.

  • Validate command snippets for syntactic correctness and repository consistency.

Local self-validation (must run before finishing)

Run and fix until green, but scope checks to impacted areas only:

  • Always run formatting/lint for touched language(s):
    • Rust touched: cargo fmt and cargo clippy --workspace --all-targets -- -D warnings
    • Python orchestrator touched: relevant formatter/linter only for touched paths
  • Rust tests policy (mirror CI selective strategy in .github/workflows/ci.yml):
    • Code changes in Rust modules: run only the relevant filtered test targets (for example cargo test -p floe-core --test unit <module_path>:: or cargo test -p floe-core --test integration when applicable).
    • Reuse compiled artifacts across commands in the same run (avoid clean rebuild loops; do not run cargo clean unless strictly needed).
    • Docs-only changes (e.g., README.md, docs/**): do not run Rust tests.
    • Avoid full workspace/crate test runs unless explicitly required by the task.
  • Python tests policy:
    • Run only relevant pytest path(s) when Python code is touched.
    • Docs-only changes: no pytest required.

Git / PR requirements (mandatory)

  • Start from main (worktree branch created from origin/main).
  • Make focused commits.
  • Ensure changes are staged: git add -A
  • Commit with a clear message.
  • Push branch to origin.
  • Ensure a PR can be created (there must be at least one commit).

Acceptance criteria

  • docs/local-dev.md exists and documents complete local Rust workflow.
  • Includes the required sections (Prerequisites, Setup, Run, Test, Quality checks, Troubleshooting).
  • Commands are accurate for this repository and runnable by a first-time contributor.
  • Troubleshooting includes at least 3 practical fixes.
  • At least one commit exists on the task branch so PR creation succeeds.

Output requirement

When done, provide:

  • Summary of changes
  • Commands run + results
  • What tests were added/updated and where

@malon64 malon64 closed this Mar 6, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant