Automation Generator

🤖 Transform your Standard Operating Procedures into intelligent automation workflows with AI assistance

This application allows users to upload SOP-based PDF documents and automatically create comprehensive automation solutions. The system features advanced semantic search powered by vector embeddings, professional flowcharts with proper symbols, executable scripts, and detailed prerequisites using Google's Gemini AI.

🌐 Live Application: Automation Generator

🚀 Version 2.1 - Agentic Workflow Ready!

✅ Agentic Generation Flow: A new orchestration layer coordinates summary, flowchart, script, tests, and prerequisites generation
✅ Self-Healing Validation: Mermaid and Python outputs are validated, retried, and safely downgraded to fallbacks when needed
✅ Real-Time Agent Console: Streamlit now shows safe progress events, validation results, retries, and completion status
✅ Semantic Vector Search: Find solutions by meaning, not just keywords
✅ Professional UI: Beautiful interface with real-time status indicators
✅ Secure Admin: Password-protected admin interface for advanced features
✅ Smart Fallbacks: Always working search with multiple fallback mechanisms
✅ Credential-Safe Startup: The app can start in degraded mode when Gemini or MongoDB secrets are missing

🔍 Advanced Search Capabilities

Semantic Search (New!)

Find by Meaning: Search "automate reports" and find "report generation automation"
AI-Powered Relevance: Vector embeddings using Google's Gemini for better results
Smart Fallbacks: Vector → Hybrid → Text → Regex search chain ensures results
Real-time Status: Visual indicators show search type (Semantic/Keyword)

Search Flow Architecture

User Query → 🔍 Vector Search → 🔄 Hybrid Search → 📝 Text Search → 🔍 Regex Search

✨ Core Features

Document Processing & AI Generation

PDF Processing: Upload and analyze SOP-based PDF documents
Agentic Automation Generation: A single orchestrator coordinates summary, diagram, script, tests, and prerequisites
AI-Powered PDF Summarization: Intelligent content extraction and summarization
Real-Time Agent Console: Displays safe operational events for generation steps, validation results, retries, and completion
Self-Healing Output Validation:
- Mermaid diagrams are checked, corrected, and replaced with template fallbacks when needed
- Python scripts are checked with syntax and safety validation before display
- Unit tests are cleaned and parsed before being included in the final package
Professional Flowchart Generation: Creates Mermaid diagrams with proper flowchart symbols:
- Ovals for Start/End points
- Diamonds for Decision points
- Rectangles for Process steps
- Parallelograms for Input/Output operations
Script Generation: AI-powered automation script creation
Unit Test Generation: Produces Python tests alongside generated scripts
Prerequisites Analysis: Comprehensive requirement identification

Enhanced User Experience

Interactive Workflow: Generate, validate, review, download, and save complete automation packages
Multiple Complexity Levels: Simple, moderate, and complex flowchart generation
Theme Support: Light and dark mode compatibility
Real-time Search Stats: Vector search readiness and performance metrics
Visual Status Indicators: 🟢/🔴 status for all system components
Degraded Mode Warnings: Sidebar warnings make local storage mode and disabled generation explicit

Security & Administration

Password-Protected Admin: Secure access to migration and admin tools
Secrets Management: All sensitive data stored in secure configuration
Restricted Operations: Admin functions require authentication
Comprehensive Monitoring: Real-time statistics and performance tracking

💻 Technical Requirements

Core Dependencies

Python 3.8+ (Required for modern AI libraries)
Streamlit (Web interface and UI components)
PyPDF2 (PDF document processing)
google.generativeai (Gemini AI integration for embeddings and generation)
google-antigravity (Planned SDK dependency; current implementation uses a compatible local orchestration boundary when SDK imports are unavailable)
pymongo (MongoDB Atlas integration for vector search)
tenacity (Retry handling for Gemini generation calls)

Additional Libraries

base64 (File encoding)
logging (Application logging)
hashlib (Diagram identification)
datetime (Timestamp management)
typing (Type annotations)

External Services

Google Gemini API (AI generation and vector embeddings)
MongoDB Atlas (Database with vector search capabilities)
Streamlit Cloud (Recommended hosting platform)

🚀 Quick Start Installation

1. Clone Repository

git clone https://github.com/Shishir420-GIT/Automation-Generator.git
cd Automation-Generator

2. Install Dependencies

pip install -r requirements.txt

3. Verify Installation

python3 -c "import streamlit, PyPDF2, google.generativeai, pymongo; print('✅ All dependencies installed successfully')"

4. Run Tests

python3 -m unittest discover tests

Configuration

Set up Gemini and MongoDB secrets:

Create a .streamlit/secrets.toml file in the project directory

Add your API key and secrets:

API_KEY = "your_api_key_id"
mongoDB_URI = "your_mongoDB_connection_uri"
ADMIN_PASSWORD = "your_secure_admin_password"

Do not commit this file to version control

You can also copy .streamlit/.example.secrets.toml as a starting point if present.

Ensure you have project created in Google Cloud with valid billing.
Optional degraded modes:

If mongoDB_URI is missing or unreachable, the app runs in local JSON fallback mode at .streamlit/local_db.json.
If API_KEY is missing, the app still starts, but generation controls are disabled until the key is configured.

Usage

Start the Application:
```
streamlit run app.py
```
Access the Web Interface: Open the provided URL in your web browser (usually http://localhost:8501)
Configure Your Automation:
- Enter the domain/industry context
- Add any additional information for better AI understanding
Upload PDF Document: Upload your SOP-based PDF file for AI-powered analysis and summarization
Generate Automation: Click "Generate Complete Automation" to create:
- Document Summary: Generated by the agent workflow
- Professional Flowchart: Interactive Mermaid diagram with proper symbols
- Automation Script: Executable code tailored to your requirements
- Unit Tests: Test scaffolding for the generated script
- Prerequisites List: Comprehensive setup and dependency requirements
- Validation Report: Mermaid, script, and test validation status
Review and Refine:
- Watch the Agent Console for progress and validation events
- Examine the generated summary, flowchart, script, tests, and prerequisites
- Download individual artifacts or the combined automation package
Interactive Features:
- Switch between different complexity levels
- Apply different themes (light/dark)
- Export diagrams and scripts

🏗️ Architecture & Components

The application follows a modular structure with advanced vector search capabilities:

Automation-Generator/
├── app.py                      # Main Streamlit application with enhanced UI
├── utils/
│   ├── agent_service.py        # Agent-style orchestration, validation, retries, and event streaming
│   ├── GeminiFunctions.py      # AI integration and content generation
│   ├── mermaid_renderer.py     # Professional flowchart generation with proper symbols
│   ├── MongoDBFunctions.py     # Enhanced database operations with vector search
│   ├── embedding_service.py    # Gemini embedding integration (768-dimensional vectors)
│   ├── vector_admin.py         # Admin interface for vector search management
│   └── validators.py           # Input validation utilities
├── tests/
│   ├── test_agent_workflow.py  # Agent orchestration and validation tests
│   ├── test_local_fallback.py  # Local JSON fallback tests
│   ├── test_vector_search.py   # Vector search testing utilities
│   └── test_secrets_password.py # Secrets configuration testing
└── README.md

🔧 Key Components:

Vector Search System: MongoDB Atlas Vector Search with Gemini embeddings
AutomationAgentService: Coordinates generation steps, emits safe progress events, validates outputs, and retries failed components
GeminiEmbeddingService: 768-dimensional vector generation using gemini-embedding-001
Smart Search Fallbacks: Vector → Hybrid → Text → Regex search chain
MermaidRenderer: Enhanced flowchart generation with proper symbols
Local JSON Fallback: Saves and searches generated solutions locally when MongoDB is unavailable
Secure Admin Interface: Password-protected admin tools and migration controls
Hybrid Search: Combines semantic vector search with traditional text search
Migration System: Batch processing for existing document embedding generation

🚀 Deployment Guide

Production Deployment (Streamlit Cloud - Recommended)

Fork this repository to your GitHub account
Connect your GitHub repository to Streamlit Cloud

Configure MongoDB Atlas Vector Search Index:

{
  "name": "vector_search_index",
  "definition": {
    "fields": [
      {
        "type": "vector",
        "path": "embedding",
        "numDimensions": 768,
        "similarity": "cosine"
      }
    ]
  }
}

Add your secrets in Streamlit Cloud dashboard under "Secrets":

API_KEY = "your_gemini_api_key"
mongoDB_URI = "your_mongoDB_connection_uri"
ADMIN_PASSWORD = "your_secure_admin_password"

Deploy automatically from your repository
Run Initial Migration: Use admin interface to migrate existing documents

Local Development

Ensure all dependencies are installed
Set up .streamlit/secrets.toml with your API key
Create MongoDB vector search index if using Atlas search
Run streamlit run app.py
Access admin interface for testing and migration
Run python3 -m unittest discover tests before committing changes

MongoDB Atlas Requirements

Cluster: M10+ recommended for vector search
Vector Index: vector_search_index (768 dimensions, cosine similarity)
Text Index: summary_search_index (for fallback search)
Connection: MongoDB Atlas URI with appropriate permissions

Performance Considerations

Search Response: < 2 seconds for most queries
Migration Speed: 5-10 documents per minute (API rate limited)
API Costs: ~$10-50/month for moderate usage
Uptime: 99.9% with fallback mechanisms

⚠️ Important Considerations

💰 Cost Management

Gemini API: ~$0.0001 per 1K characters for embeddings
Monthly Estimate: $10-50 for moderate usage
Optimization: Smart fallbacks reduce unnecessary API calls
Monitoring: Track usage through Google Cloud Console
Review: Gemini pricing before extended use

🔐 Security Best Practices

Never commit API keys or secrets to version control
Use secure secrets management (.streamlit/secrets.toml)
Admin Access: Password-protected admin interface
Strong Passwords: Use 12+ character passwords with mixed case, numbers, symbols
Billing Setup: Ensure Google Cloud project has proper billing configuration

📊 Performance Expectations

PDF Processing: Optimized for standard SOP documents
Vector Search: < 2 seconds response time
Flowchart Rendering: Supports complex diagrams with multiple decision points
AI Generation: 10-30 seconds depending on complexity
Agent Validation: Adds retries only when generated Mermaid, scripts, or tests fail validation
Search Accuracy: 80%+ improvement in semantic relevance
Migration: Batch processing with rate limiting

🔍 Search Performance

Semantic Search: Finds content by meaning, not just keywords
Smart Fallbacks: Always returns results through 4-tier search system
Real-time Status: Visual indicators show current search method
Hybrid Results: Combines vector similarity with keyword matching

📈 Version History & Updates

🎉 Version 2.1 - Agentic Workflow Ready! (Current)

✅ Agent Service Boundary: Added utils/agent_service.py with request/result/event models
✅ Agent Console: Streamlit generation flow now streams safe operational events
✅ Self-Healing Validation: Mermaid and Python outputs are validated and retried before display
✅ Credential-Safe Startup: Missing Gemini or MongoDB secrets no longer hard-stop the whole app
✅ Local Fallback Improvements: Local saves now include unit tests, UUIDs, and broader field search
✅ Focused Tests: Added unit tests for agent orchestration and local fallback behavior

🎉 Version 2.0 - Vector Search Production Ready

✅ Vector Search System: MongoDB Atlas Vector Search with Gemini embeddings
✅ Semantic Search: Find solutions by meaning, not just keywords
✅ Smart Fallbacks: 4-tier search system ensures results (Vector → Hybrid → Text → Regex)
✅ Enhanced UI: Vector search status prominently displayed with real-time indicators
✅ Secure Admin: Password-protected admin interface for advanced features
✅ Migration Tools: Batch migration system for existing documents
✅ Professional Flowcharts: Proper symbols with enhanced styling

🔧 Technical Improvements

✅ Enhanced Flowchart Symbols: Updated to use proper flowchart conventions

Ovals for Start/End points
Diamonds for Decision points
Rectangles for Process steps
Parallelograms for Input/Output operations

✅ Advanced Search Architecture: Multi-tier search system with intelligent fallbacks
✅ Embedding Integration: 768-dimensional vectors using gemini-embedding-001
✅ Security Enhancements: Secrets management and password-protected admin access
✅ Performance Optimization: Smart caching and rate limiting
✅ Modular Design: Organized code structure in utils/ folder for maintainability

🚀 Coming Next

Confirm and wire the real Google Antigravity SDK API when available in the runtime
Query result caching for cost optimization
Advanced analytics and search usage patterns
Enhanced bulk operations and management tools
REST API endpoints for external integrations
Role-based access control
Performance tuning and query optimization

📞 Support & Documentation

📋 Additional Resources

VECTOR_SEARCH_DEPLOYMENT.md - Complete deployment guide
implementation_plan_revised.md - Current agentic implementation plan
utils/agent_service.py - Agent orchestration implementation
tests/test_agent_workflow.py - Agent workflow tests
tests/test_local_fallback.py - Local fallback tests
tests/test_vector_search.py - Testing utilities and examples
tests/test_secrets_password.py - Secrets configuration validation

🛠️ Troubleshooting

Generation Disabled: Add API_KEY to .streamlit/secrets.toml
Local Storage Mode: Add or fix mongoDB_URI to use MongoDB Atlas instead of .streamlit/local_db.json
Agent SDK Unavailable: The app currently uses the local agent orchestration boundary when google-antigravity is not importable
Vector Search Issues: Check MongoDB Atlas index configuration
API Errors: Verify Gemini API key and billing setup
Admin Access: Ensure correct password in secrets.toml
Performance: Monitor search response times and API usage

📊 Monitoring

Vector search success rates
API usage and costs
Search result quality
Migration progress
System uptime and performance

🎊 Status: Agentic Workflow Ready | Vector Search: Implemented | Security: Admin Protected

Name		Name	Last commit message	Last commit date
Latest commit History 26 Commits
.devcontainer		.devcontainer
.streamlit		.streamlit
assets/images		assets/images
deprecated-versions		deprecated-versions
docs		docs
tests		tests
utils		utils
.DS_Store		.DS_Store
.gitignore		.gitignore
LICENSE.txt		LICENSE.txt
README.md		README.md
app.py		app.py
requirements.txt		requirements.txt

Folders and files

Latest commit

History

Repository files navigation

Automation Generator

🚀 Version 2.1 - Agentic Workflow Ready!

🔍 Advanced Search Capabilities

Semantic Search (New!)

Search Flow Architecture

✨ Core Features

Document Processing & AI Generation

Enhanced User Experience

Security & Administration

💻 Technical Requirements

Core Dependencies

Additional Libraries

External Services

🚀 Quick Start Installation

1. Clone Repository

2. Install Dependencies

3. Verify Installation

4. Run Tests

Configuration

Usage

🏗️ Architecture & Components

🔧 Key Components:

🚀 Deployment Guide

Production Deployment (Streamlit Cloud - Recommended)

Local Development

MongoDB Atlas Requirements

Performance Considerations

⚠️ Important Considerations

💰 Cost Management

🔐 Security Best Practices

📊 Performance Expectations

🔍 Search Performance

📈 Version History & Updates

🎉 Version 2.1 - Agentic Workflow Ready! (Current)

🎉 Version 2.0 - Vector Search Production Ready

🔧 Technical Improvements

🚀 Coming Next

📞 Support & Documentation

📋 Additional Resources

🛠️ Troubleshooting

📊 Monitoring

About

Topics

Resources

License

Uh oh!

Stars

Watchers

Forks

Releases 3

Packages 0

Uh oh!

Uh oh!

Contributors

Uh oh!

Languages

Packages