Speech-to-Text with Chrome's Web Speech API

This project provides a lightweight speech-to-text solution by leveraging Chrome's built-in Web Speech API through Selenium WebDriver. It bypasses the need for external APIs like Google Cloud Speech-to-Text, making it a cost-effective and accessible tool for converting spoken input into text using a browser-based interface.

Features

• Headless Browser Automation: Utilizes Selenium with Chrome in headless mode for seamless speech recognition without a visible browser window.
• Configurable Language Support: Allows setting the input language via a .env file for multilingual speech recognition.
• Query Processing: Automatically formats transcribed text as questions or statements based on detected keywords (e.g., "what," "how").
• No External API Costs: Relies on Chrome’s native Web Speech API, eliminating dependency on paid cloud services.
• Customizable Timeout: Stops speech recognition after 10 seconds of silence, with adjustable polling for responsiveness.
• File-Based Status Tracking: Stores assistant status in a temporary file for integration with other applications.

Potential Use Cases

• Voice-Activated Assistants: Integrate with chatbot or virtual assistant projects for voice command processing.
• Accessibility Tools: Create applications for users with mobility or typing challenges by enabling voice input.
• Prototyping Voice Interfaces: Test speech-to-text functionality for applications without investing in premium APIs.
• Educational Tools: Build language learning apps that transcribe and analyze spoken input in various languages.

Prerequisites

• Python: Version 3.8 or higher.
• Google Chrome: Latest version installed.
• Microphone: A working microphone for speech input.
• Python Packages:
  • selenium: For browser automation.
  • webdriver_manager: To automatically manage ChromeDriver.
  • python-dotenv: For loading environment variables.

Install dependencies using:

pip install selenium webdriver_manager python-dotenv

Setup Instructions

1. Clone the Repository:

   git clone https://github.com/your-username/speech-to-text-chrome.git
   cd speech-to-text-chrome

2. Set Up Environment Variables: Create a .env file in the project root with the following content:

   InputLanguage=en-US

   Replace en-US with a supported language code (e.g., es-ES for Spanish, fr-FR for French). A list of supported languages can be found in the Web Speech API documentation.

3. Directory Structure: Ensure the project has the following structure:

   speech-to-text-chrome/
   ├── Backend/
   │   └── Speech_to_Text.html
   ├── Frontend/
   │   └── Files/
   │       └── Status.data (auto-generated)
   ├── Speech_to_Text.py
   └── .env
   

   • Backend/Speech_to_Text.html: Contains the HTML and JavaScript for the Web Speech API interface.
   • Frontend/Files/Status.data: Temporary file for storing assistant status.
   • Speech_to_Text.py: Main script for running the speech recognition.
   • .env: Configuration file for the input language.

4. Verify Chrome Installation: Ensure Google Chrome is installed and up to date. The script uses webdriver_manager to automatically download the compatible ChromeDriver version.

Usage

1. Run the Script:

   python Speech_to_Text.py

   The script will:

   • Load the Speech_to_Text.html file in a headless Chrome browser.
   • Update the HTML file with the specified language from the .env file.
   • Start speech recognition automatically when the "Start" button is triggered.
   • Display transcribed text in the console, formatted as a question (e.g., "What is the weather?") or statement (e.g., "Open the door.").
   • Stop recognition after 10 seconds of silence or when manually interrupted.

2. Speak Clearly: Use a microphone and speak clearly. The Web Speech API processes the audio and outputs text to the console.

3. Stop the Script: Press Ctrl+C to terminate the script. The WebDriver will close automatically, ensuring no lingering processes.

How It Works

Components

• Speech_to_Text.html:

  • A simple webpage with JavaScript leveraging the Web Speech API (SpeechRecognition interface).
  • Contains "Start" and "Stop" buttons (controlled via Selenium) and an output element for transcribed text.
  • Dynamically updated with the language code specified in the .env file.

• Speech_to_Text.py:

  • Environment Setup: Loads the language from the .env file and updates the HTML file accordingly.
  • Selenium Configuration: Runs Chrome in headless mode with flags to enable microphone access (--use-fake-ui-for-media-stream) and suppress unnecessary logs.
  • Speech Recognition Loop: Polls the HTML output every 0.3 seconds for new transcribed text. If no new text is detected for 10 seconds, recognition stops, and the text is processed.
  • Query Processing: The QueryModifier function formats the text by:
    • Converting to lowercase and stripping whitespace.
    • Adding a question mark (?) if the text starts with question words (e.g., "how," "what") or a period (.) otherwise.
  • Status Tracking: Writes the assistant’s status to Status.data for potential integration with other systems.

Technical Details

• Web Speech API: Chrome’s built-in API for speech recognition, supporting multiple languages and continuous transcription.
• Headless Chrome: Runs without a visible UI, reducing resource usage and enabling server-side deployment.
• Selenium WebDriver: Automates browser interactions, such as clicking buttons and retrieving text from the webpage.
• Polling Mechanism: Checks for new transcribed text every 0.3 seconds to balance responsiveness and performance.
• Error Handling: Includes basic logging and exception handling for WebDriver initialization and text retrieval.

Limitations

• Browser Dependency: Requires Google Chrome, as the Web Speech API is not fully supported in other browsers.
• Recognition Accuracy: Depends on the Web Speech API’s capabilities, which may vary by language and audio quality.
• Headless Mode Constraints: Microphone access in headless mode requires fake media stream flags, which may not work in all environments.
• Single-Threaded Execution: The script processes one speech input at a time and may need optimization for concurrent use.
• Security: Not hardened for production use; additional measures are needed to secure the Chrome profile and file handling.

Extending the Project

• Add Real-Time Processing: Modify the script to stream transcribed text to another application or API.
• Support Multiple Languages: Allow dynamic language switching during runtime.
• Improve Error Handling: Add retries for WebDriver failures or network issues.
• Integrate with NLP: Combine with natural language processing tools to analyze or respond to transcribed text.
• GUI Interface: Replace the headless browser with a visible interface for manual testing.

Troubleshooting

• WebDriver Fails to Start:
  • Ensure Chrome is installed and updated.
  • Verify webdriver_manager is downloading the correct ChromeDriver version.
  • Check for conflicting Chrome processes and terminate them.
• No Speech Recognition:
  • Confirm your microphone is working and accessible.
  • Ensure the --use-fake-ui-for-media-stream and --use-fake-device-for-media-stream flags are set.
  • Test with a non-headless browser to verify API functionality (chrome_options.add_argument("--headless=new") can be removed temporarily).
• Incorrect Language:
  • Validate the InputLanguage in the .env file against supported Web Speech API language codes.
• Performance Issues:
  • Adjust the polling interval (time.sleep(0.3)) to balance responsiveness and CPU usage.
  • Ensure sufficient system resources for headless Chrome.

Contributing

We welcome contributions to enhance functionality or fix issues. To contribute:

1. Fork the repository.
2. Create a feature branch (git checkout -b feature/your-feature).
3. Commit your changes (git commit -m 'Add your feature').
4. Push to the branch (git push origin feature/your-feature).
5. Open a pull request with a clear description of your changes.

Please include tests and documentation updates where applicable.

License

This project is licensed under the MIT License. See the LICENSE file for details.

Acknowledgments

• Built using the Web Speech API provided by Google Chrome.
• Powered by Selenium WebDriver and webdriver_manager for browser automation.
• Inspired by the need for a free, lightweight alternative to cloud-based speech-to-text APIs.