AdalFlow Documentation Guide

Content Overview

• AdalFlow Documentation Guide
  • Content Overview
  • Introduction
    • How the Documentation Works
  • Prerequisites
  • Setup
    • 1. Clone the Repository
    • 2. Install Dependencies
    • 3. Verify Setup
  • File Structure
    • conf.py
    • index.rst
    • Existing Sections
  • Editing and Updating Documentation
    • Updating Docstrings in Source Code
    • Adding New Code and Docstrings
  • Building the Documentation
    • Steps to Build
    • Quick Start
    • Viewing the Documentation
  • Testing the Documentation
  • Contributing
    • Git Workflow
    • Directory Structure
  • Optional Tools
    • Convert Markdown to reStructuredText

Introduction

AdalFlow uses Sphinx and reStructuredText for generating project documentation. Sphinx reads configurations from a Python script (conf.py), pulls documentation from comments in the code (via the autodoc extension), and organizes content through .rst files.

How the Documentation Works

• Configuration: Managed via conf.py.
• Automatic Documentation: Generated from docstrings using autodoc.
• Content Organization: Structured through .rst files and the table of contents defined in index.rst.

This guide will walk you through setting up, building, and contributing to the AdalFlow documentation.

Prerequisites

Before you begin, ensure you have the following installed:

• Poetry: For dependency management. Install it with:

  pip install poetry

• Optional Tools:

  • pandoc for converting Markdown to reStructuredText.
  • A modern browser for viewing documentation.

Setup

1. Clone the Repository

Clone the AdalFlow GitHub repository:

git clone https://github.com/SylphAI-Inc/AdalFlow.git
cd AdalFlow

2. Install Dependencies

Install all necessary dependencies using poetry:

poetry install

If you encounter issues with poetry, ensure it is up-to-date by reinstalling:

pip install --upgrade poetry

Note: Be sure to run these commands from the root level of the project.

3. Verify Setup

Ensure all dependencies are installed correctly:

poetry check

File Structure

conf.py

The docs/source/conf.py controls the configurations used by Sphinx to build the documentation, including the project-related information, sphinx extensions, templates configuration, HTML theme, patterns to exclude, language configuration, project path setup, etc.

index.rst

The docs/source/index.rst is the root document for Sphinx-generated documentation ("homepage" for the documentation site). It includes the toctree that defines the documentation hierarchical structure (sections/chapters). It also links to other .rst files that users can navigate through.

For example, in the index.rst, the :caption: Get Started corresponds to the section name of the documentation site. installation and introduction are the detailed pages.

.. toctree::
   :glob:
   :maxdepth: 1
   :caption: Get Started

   get_started/installation
   get_started/introduction

Existing Sections

Existing sections include:

• get_started/: Includes installation and AdalFlow in 10 minutes
• tutorials/: Includes our main tutorials
• use_cases/: Includes the use cases of AdalFlow that will be added in the future and which accepts community contributions
• apis/: All the source-code-related documents will be included in this directory

Editing and Updating Documentation

Updating Docstrings in Source Code

Most of the documentation is automatically generated from code comments using the autodoc extension. To update:

1. Edit the docstrings in your source code.
2. Rebuild the documentation (see Building the Documentation).

Note: Ensure that your docstrings follow the reStructuredText format for proper rendering.

Adding New Code and Docstrings

When you add new modules or code to the project, it's essential to generate corresponding .rst files to include them in the documentation. Here's how to do it:

1. Ensure Proper Structure:

   • If your new module is a folder, it should contain an __init__.py file to be recognized as a package.
   • Write comprehensive docstrings for your new code to leverage the autodoc extension effectively.

2. Generate .rst Files Using sphinx-apidoc:

   Use the sphinx-apidoc command to automatically generate .rst files for your new modules:

   sphinx-apidoc [OPTIONS] -o <OUTPUT_PATH> <MODULE_PATH> [EXCLUDE_PATTERN …]

   Example:

   While located in the root directory of your project, run the following command to generate .rst files for the use_cases module, excluding any files or directories with "test" in their names:

   sphinx-apidoc --force -o docs/source/tutorials ./use_cases "*test*"
   • --force: Overwrites any existing .rst files in the output directory. This is safe as long as you haven't made manual edits to these .rst files.
   • -o docs/source/tutorials: Specifies the directory where the .rst files will be generated.
   • ./use_cases: Path to the module or package to document.
   • "*test*": Excludes any files or directories with "test" in their names.

   Caution: The --force flag will overwrite existing .rst files in the specified output directory. Ensure that you haven't made manual edits to these files that you wish to preserve. If you have custom .rst files, consider backing them up or omitting the --force flag.

3. Update index.rst to Include New Sections:

   After generating the .rst files, update the index.rst to include the new documentation sections. For example:

   .. toctree::
      :glob:
      :maxdepth: 1
      :caption: Tutorials
   
      tutorials/use_cases

   This ensures the new module is linked in the documentation's table of contents.

4. Rebuild the Documentation:

   After adding new docstrings and generating .rst files, rebuild the documentation to see the updates

   Summary of Steps When Adding New Code with Docstrings:

   1. Add the new code/module with proper docstrings.
   2. Generate .rst files using sphinx-apidoc.
   3. Update index.rst to include new sections.
   4. Rebuild the documentation to reflect changes.

Building the Documentation

Steps to Build

1. Navigate to the docs Directory:

   cd docs

2. Clean Previous Builds:

   Remove previous build artifacts to ensure a fresh build.

   make clean

3. Build the Documentation:

   Generate the HTML documentation.

   make html

4. (Optional) Build with Verbose Output:

   For more detailed build logs, useful for debugging.

   sphinx-build -b html source build -v

Quick Start

For users who prefer straightforward commands without additional options:

cd docs
make html

After the build completes, open the documentation:

• macOS:

  open build/html/index.html

• Linux:

  xdg-open build/html/index.html

• Windows:

  Manually open the index.html file in your preferred browser.

Viewing the Documentation

After building, the HTML files will be available in docs/build/html. Open index.html in your browser to view the documentation.

Handling Browser Restrictions on Local Resources:

Some browsers restrict loading local resources like CSS for security reasons. In such cases, use a local server to serve the files:

cd docs/build/html
python -m http.server

Visit http://127.0.0.1:8000/ in your browser to view the documentation.

Testing the Documentation

To ensure the documentation builds correctly without errors or warnings, run:

sphinx-build -n -W --keep-going source build

• -n: Runs Sphinx in nit-picky mode to catch missing references.
• -W: Treats warnings as errors, causing the build to fail if any warnings are present.
• --keep-going: Continues the build as much as possible after encountering errors.

Recommendation: Fix any issues reported by this command before committing your changes to maintain documentation quality.

Contributing

Git Workflow

To maintain a clean repository, exclude unnecessary files from commits. Specifically, avoid committing the docs/build directory, as documentation builds are dynamic and can be regenerated locally.

Example .gitignore:

# Ignore build files
docs/build/
*.pyc
__pycache__/

• Commit Only Necessary Files:
  • Source files (.rst, .py, etc.)
  • Configuration files (conf.py, Makefile, pyproject.toml, etc.)
• Exclude:
  • docs/build/: Generated HTML files.
  • Compiled Python files (*.pyc, __pycache__/).

Directory Structure

Ensure the project follows this structure:

AdalFlow/
├── docs/
│   ├── apis/
│   │   ├── core/
│   │   │   ├── core.module1.rst
│   │   │   ├── core.module2.rst
│   │   ├── components/
│   │   │   ├── components.module1.rst
│   │   │   ├── components.module2.rst
│   ├── build/
│   │   ├── html/
│   │   │   ├── _static/
│   │   │   ├── _templates/
│   │   │   ├── index.html
│   │   │   ├── core/
│   │   │   │   ├── core.module1.html
│   │   │   │   ├── core.module2.html
│   │   │   ├── components/
│   │   │   │   ├── components.module1.html
│   │   │   │   ├── components.module2.html
│   ├── _static/
│   ├── _templates/
│   ├── conf.py
│   ├── index.rst
│   ├── Makefile
│   ├── pyproject.toml
│   ├── poetry.lock

Note: The build/ directory contains generated files and should not be manually edited or committed.

Optional Tools

If you want to add any written files such as README.md to the documentation, there is an easy way to transform the files to .rst files using Pandoc.

Convert Markdown to reStructuredText

To convert .md files (e.g., README.md) to .rst:

1. Install pandoc:

   brew install pandoc

   For non-Homebrew users, refer to Pandoc's installation guide.

2. Run the Conversion:

   Conversion Syntax:

   pandoc -s <input.md> -o <path/to/target.rst>

   Example:

   This command will take content from README.md and create an introduction.rst file in the specified directory.

   pandoc -s README.md -o docs/source/get_started/introduction.rst

3. Rebuild the Documentation:

   After converting, rebuild the documentation to include the new .rst files.

   cd docs
   make clean
   make html

Note: Ensure that the converted .rst files are correctly linked in your index.rst or appropriate parent .rst files.