Generate Fully Standalone Python Scripts by Bundling Required Operations

[kaushikharsh99]

Summary

FlowWeaver currently generates Python scripts that depend on the flowweaver package:

from flowweaver.std.io import import_dataset
from flowweaver.std.text import lowercase

While this works for FlowWeaver users, the generated scripts are not truly standalone and require the FlowWeaver SDK to be installed.

The compiler should instead generate completely self-contained Python scripts by automatically bundling only the operations and helper functions required by the compiled pipeline.

The generated script should become the final executable artifact.

---

Vision

A user should be able to:

1. Build a preprocessing pipeline visually.
2. Click Compile.
3. Receive a single pipeline.py.
4. Copy the file to any machine.
5. Run:

python pipeline.py

without installing FlowWeaver.

FlowWeaver should behave like a compiler and linker rather than a runtime.

---

Proposed Architecture

Pipeline JSON
        │
        ▼
Compiler
        │
        ▼
Dependency Resolver
        │
        ▼
Operation Linker
        │
        ▼
Import Deduplication
        │
        ▼
Standalone Python Generator
        │
        ▼
pipeline.py

---

Design Goals

• No flowweaver.* imports in generated scripts.
• Bundle only the operations used by the pipeline.
• Bundle required helper functions automatically.
• Deduplicate imports.
• Deduplicate helper functions.
• Produce clean, readable Python.
• Keep generated scripts deterministic.

---

Compiler Responsibilities

The compiler should:

• Analyze every operation used by the pipeline.
• Resolve operation dependencies.
• Resolve helper function dependencies.
• Resolve required imports.
• Merge imports.
• Inline only the required source code.
• Generate a single standalone script.

---

Example

Current output:

from flowweaver.std.io import import_dataset
from flowweaver.std.dedup import dedup_exact
from flowweaver.std.io import export_csv

raw_dataset = import_dataset(...)
clean_dataset = dedup_exact(raw_dataset)
export_csv(clean_dataset)

Desired output:

import csv
import json
import pathlib
import logging

def import_dataset(...):
    ...

def dedup_exact(...):
    ...

def export_csv(...):
    ...

def main():
    ...

No FlowWeaver dependency should remain.

---

Dependency Resolution

The compiler should automatically include:

• Operation source
• Internal helper functions
• Required constants
• Required imports
• Shared utility functions

Unused code must not be included.

---

Linking Strategy

Treat flowweaver.std as compiler source rather than a runtime library.

flowweaver.std

↓

Operation Parser

↓

Dependency Graph

↓

Reachability Analysis

↓

Dead Code Elimination

↓

Standalone Script

---

Scope

This issue includes:

• Source collection
• Dependency graph generation
• Import deduplication
• Helper function bundling
• Operation bundling
• Standalone code generation

This issue does not include:

• UI changes
• New preprocessing nodes
• Compiler optimizations
• Pipeline execution changes

---

Acceptance Criteria

• Generated scripts contain no flowweaver.* imports.
• Generated scripts run on any machine with a compatible Python version.
• Only required operations are bundled.
• Helper functions are bundled automatically.
• Imports are deduplicated.
• Generated code remains readable and well formatted.
• Existing compiler tests continue to pass.
• New tests verify standalone execution.

---

Future Enhancements

• AST-based source extraction.
• Automatic dead code elimination.
• Shared utility deduplication.
• Optional generation of multi-file standalone projects.
• Support for optional external dependency bundling.

---

Motivation

FlowWeaver's generated Python should be the final product—not a wrapper around the FlowWeaver SDK.

The generated script should be portable, reproducible, shareable, version-controllable, and executable on any machine without requiring FlowWeaver itself.

This architectural change transforms FlowWeaver from a workflow application into a true visual compiler for AI data preprocessing.