Implementing Audio Receiver Pipelines

For general information about receiver pipelines refer to the Receiver Pipelines section section.

Audio Processing Pipeline Architecture

This document describes an audio processing pipeline where audio streams are processed through synchronous or asynchronous slots, depending on the characteristics of each component’s workload. The ssdk::audio::AudioDispatcher manages distribution to ssdk::audio::AudioInput components, and data is then routed through a sequence of ssdk::util::PipelineSlot components for processing before reaching the final output interface.

Core Components

Network Transport and AudioDispatcher

Audio data is obtained from the ssdk::transport_common::ClientTransport interface and passed to the ssdk::audio::AudioDispatcher object via the transport_common::ClientTransport::AudioReceiverCallback. Upon receiving audio buffers, ssdk::audio::AudioDispatcher forwards them to instances of the ssdk::audio::AudioInput class. Each AudioInput object:

• Decodes Compressed Audio: If compressed, it includes a decoder to handle decompression.
• Forwards Uncompressed Audio: For uncompressed audio, it directly passes the data to the processing pipeline.

AudioReceiverPipeline

After decoding or pass-through, audio data is submitted to the processing pipeline via the AudioReceiverPipeline::SubmitInput() method. This pipeline consists of slots based on ssdk::util::PipelineSlot, each containing a pointer to an amf::AMFComponent object, which defines the specific processing behavior within the slot.

Synchronous and Asynchronous Slots

There are two types of PipelineSlot classes, each suited to different processing patterns:

1. Synchronous Slots (ssdk::util::SynchronousSlot): These slots process data in a single thread, calling amf::AMFComponent::SubmitInput and amf::AMFComponent::QueryOutput sequentially. Synchronous slots are suited for components where the input-output ratio is predictable, as synchronous processing can simplify thread management and reduce context-switching overhead.

2. Asynchronous Slots (ssdk::util::AsynchronousSlot): These slots operate across multiple threads, allowing SubmitInput and QueryOutput to be called from different threads. This setup is beneficial for components with dynamic input-output behavior (i.e., where the output count may differ from the input count), allowing for more responsive handling of variable workloads and minimizing bottlenecks.

Class Hierarchy

• ssdk::audio::AudioInput: Responsible for decoding and format monitoring, calls the pipeline when the format changes.
• ssdk::util::PipelineSlot: The base class for all pipeline slots, each containing a pointer to an amf::AMFComponent.
  • ssdk::util::SynchronousSlot: Calls SubmitInput and QueryOutput in the same thread.
  • ssdk::util::ASynchronousSlot: Calls SubmitInput and QueryOutput across different threads.

Pipeline Data Flow

The audio processing pipeline sits between AudioInput and the final output interface. Audio samples are sequentially processed by each slot, following this structure:

1. Format Matching: AudioInput submits data through the pipeline. Each slot matches the output format with the input expectations of the next slot.
2. Output: Processed data reaches the final output interface, ready for playback.

Recommendations for Component Evaluation

To optimize pipeline performance, it is advisable to experiment with each amf::AMFComponent added to the pipeline:

1. Evaluate Synchronization Requirements: If a component has a stable 1:1 input-output ratio, try a SynchronousSlot for efficiency.
2. Consider Asynchronous Processing for Variable Outputs: Components that may produce variable outputs should be placed in ASynchronousSlot to allow parallelized processing and avoid potential bottlenecks.

This approach provides a structured yet flexible framework, enabling each component to function optimally within the processing pipeline.