Eight voices. Three ESP32-S3 processors. One instrument.
RTAL_ANIMUSEA_SOUND_DEMO.mp4
Project: RTAL-EAI-009-Amusea-Synthesizer
Developer: RealTimeAudioLab (RTAL)
Platform: Up to 3 × ESP32-S3
Display: Guition JC4827W543
Status: Early Development / Technology Preview
AMUSEA is an experimental 8-voice polyphonic synthesizer built around multiple ESP32-S3 processors, combining analog-inspired synthesis, extensive modulation, touchscreen operation, sequencing and a dedicated stereo effects processor.
AMUSEA is inspired by the architecture and sonic philosophy of powerful modern polyphonic analog synthesizers, with the Moog Muse serving as an important technical and sonic reference.
The goal, however, is not simply to reproduce an existing instrument.
AMUSEA explores how far a distributed ESP32-S3 architecture can be pushed toward a powerful, expressive and characterful polyphonic synthesizer while adding capabilities of its own.
Modern microcontrollers have reached a point where surprisingly complex real-time synthesis systems can be implemented without a conventional desktop-class CPU.
AMUSEA takes this idea further by distributing the workload across up to three ESP32-S3 processors.
RTAL AMUSEA
|
+---------------+---------------+
| | |
ESP32-S3 #1 ESP32-S3 #2 ESP32-S3 #3
MASTER VOICES EFX
| | |
Voices 1-4 Voices 5-8 Stereo FX
| | |
Display | Reverb
Touch | Delay
MIDI | Chorus
SD | Flanger
Sequencer | Phaser
| | |
+------- Digital Audio ----------+
|
I2S Stereo DAC
|
Stereo Output
Instead of forcing the graphical user interface, eight synthesis voices, storage, MIDI, sequencing and stereo effects onto a single processor, AMUSEA separates these tasks into specialized processing domains.
This provides considerably more DSP headroom and makes more ambitious synthesis algorithms possible.
The first ESP32-S3 is integrated into the Guition JC4827W543 touchscreen module.
It acts as the central controller and also calculates the first four synthesis voices.
Responsibilities include:
- Voices 1-4
- LVGL 9 graphical user interface
- GT911 capacitive touchscreen
- DIN MIDI
- Native USB MIDI
- MIDI Clock
- Transport control
- Global voice allocation
- Arpeggiator
- Polyphonic sequencer
- Modulation control
- Preset management
- SD card access
- Communication with the additional DSP processors
The GUI and control subsystem are deliberately separated from the time-critical audio processing.
The second ESP32-S3 is dedicated primarily to synthesis.
It calculates:
Voices 5-8
This processor does not need to run a complex graphical interface or perform SD-card operations.
Its available processing power can therefore be concentrated on:
- Oscillators
- Modulation oscillator
- Ring modulation
- Mixer
- Filters
- Envelopes
- Modulation
- VCA
- Stereo voice processing
Together, ESP32-S3 #1 and ESP32-S3 #2 form the complete eight-voice synthesis engine.
The third ESP32-S3 acts as a dedicated stereo effects processor.
Planned effects include:
- Stereo Delay
- Ping-Pong Delay
- Multi-Tap Delay
- Diffusion Delay
- Chorus
- Ensemble
- Flanger
- Phaser
- Reverb
- Room
- Plate
- Hall
- Saturation
- Stereo processing
The AMUSEA effects architecture is deliberately intended to extend beyond the effects capabilities of the instrument that inspired the project.
AMUSEA is designed around eight independent synthesis voices.
Each voice is planned to contain:
OSCILLATOR 1 --------+
|
OSCILLATOR 2 --------+
|
MOD OSCILLATOR ------+
|
RING MOD ------------+
|
NOISE ---------------+
|
v
ANALOG-STYLE MIXER
|
DRIVE
|
v
DUAL LADDER FILTER
|
v
VOICE VCA
|
v
STEREO PAN
The intention is to preserve as much processing as practical per voice, rather than relying on global filtering or modulation stages.
Each AMUSEA voice is planned around:
- Oscillator 1
- Oscillator 2
- Dedicated modulation oscillator
Planned oscillator capabilities include:
- Continuously variable waveforms
- Sawtooth
- Triangle
- Pulse
- Variable pulse width
- PWM
- Oscillator sync
- Frequency modulation
- Audio-rate modulation
- Oscillator detuning
- Octave selection
- Fine tuning
- Oscillator drift
- Per-voice variation
The intention is not simply to generate mathematically perfect waveforms.
The oscillator system will be developed and tuned as part of the complete signal path.
Oscillators can interact through ring modulation.
This allows AMUSEA to move beyond conventional subtractive synthesis into:
- Metallic timbres
- Bell-like sounds
- Complex harmonics
- Evolving textures
- Aggressive modulation sounds
- Experimental sound design
A major focus of AMUSEA is the signal path before the filters.
The mixer is planned as more than a simple digital summing stage.
OSC 1 -----+
OSC 2 -----+
MOD OSC ---+
RING ------+
NOISE -----+
|
v
SUMMING STAGE
|
ASYMMETRIC DRIVE
|
OVERLOAD
|
v
FILTERS
The design is inspired by the nonlinear behaviour associated with classic analog Moog mixer stages.
The interaction between oscillator level, mixer saturation and filter input level will form an important part of AMUSEA's sonic character.
Filtering will be one of the central development areas of the project.
Each voice is planned to contain two ladder-style filters.
INPUT
|
v
FILTER 1
|
v
FILTER 2
|
v
OUTPUT
INPUT
|
v
FILTER 2
|
v
FILTER 1
|
v
OUTPUT
+-- FILTER 1 --+
| |
INPUT --------+ +---- OUTPUT
| |
+-- FILTER 2 --+
Stereo-oriented configurations are also planned.
Parameters will include:
- Cutoff 1
- Cutoff 2
- Resonance 1
- Resonance 2
- Filter Spacing
- Keyboard Tracking
- Envelope Amount
- Velocity Modulation
- Filter Drive
- Modulation Matrix Control
Filter 1 is intended to support both low-pass and high-pass operation, while Filter 2 provides an additional ladder low-pass stage.
A significant part of the project will be dedicated to nonlinear filter modelling and the interaction between resonance, drive and oscillator level.
AMUSEA is intended to provide a deep modulation system.
Planned modulation sources include:
- LFO 1
- LFO 2
- Pitch LFO
- Per-voice Mod Oscillator
- Filter Envelope
- Amplifier Envelope
- Optional third envelope
- Velocity
- Release Velocity
- Note Number
- Keyboard Tracking
- Pitch Bend
- Mod Wheel
- Aftertouch
- Random per Note
- Continuous Random
- Sequencer Modulation
- MIDI CC
- Internal Macros
A flexible modulation matrix will connect these sources to a large number of synthesis destinations.
The initial target is:
Up to 32 modulation assignments per sound.
At least two complete envelopes are planned for every voice:
FILTER ENVELOPE
Attack
Decay
Sustain
Release
and:
AMPLIFIER ENVELOPE
Attack
Decay
Sustain
Release
Envelope parameters will be modulatable and velocity-sensitive.
An additional modulation envelope may be introduced as development progresses.
The planned LFO architecture includes:
- LFO 1
- LFO 2
- Pitch LFO
- Per-voice modulation oscillator
LFOs may operate:
- Globally
- Per voice
- Free-running
- Retriggered
- Synchronized to MIDI Clock
This allows both traditional modulation and complex polyphonic movement.
Stereo begins inside the synthesis engine, not merely inside the effects processor.
Every voice can have its own stereo position.
This makes possible:
- Alternating voice pan
- Random pan
- Modulation-controlled pan
- Stereo unison
- Wide polyphonic pads
- Animated stereo distributions
The dedicated effects processor can subsequently expand this stereo image further.
The master ESP32-S3 controls all eight voices.
VOICE 1 ---- Local
VOICE 2 ---- Local
VOICE 3 ---- Local
VOICE 4 ---- Local
VOICE 5 ---- Remote DSP
VOICE 6 ---- Remote DSP
VOICE 7 ---- Remote DSP
VOICE 8 ---- Remote DSP
Planned operating modes include:
- Poly
- Mono
- Legato
- Unison 2
- Unison 4
- Unison 8
- Chord
- Split
- Stack
Voice stealing and allocation algorithms will be optimized during development.
AMUSEA is being designed with two timbres in mind.
TIMBRE A
+
TIMBRE B
Possible configurations:
SINGLE
SPLIT
STACK
This architecture is being considered from the beginning even if early firmware versions initially operate as a single-timbre synthesizer.
The instrument is built around the Guition JC4827W543.
The display is not intended to act merely as a menu system.
It is the primary synthesizer control surface.
The interface uses:
- LVGL 9
- Capacitive GT911 touch
- Real-time sliders
- Buttons
- Graphical envelopes
- Modulation displays
- Sequencer editing
- Preset browsing
- System configuration
Planned main pages include:
HOME
OSC 1
OSC 2
MOD OSC
MIXER
FILTER
FILTER ENV
AMP ENV
LFO 1
LFO 2
PITCH LFO
MOD MATRIX
VOICE
UNISON
ARP
SEQUENCER
TIMBRE
FX
PRESET
MIDI
SYSTEM
Touch controls change synthesis parameters in real time.
One important architectural principle is that GUI, MIDI, presets and DSP do not maintain separate parameter implementations.
Instead, AMUSEA uses a central parameter model.
TOUCH
|
DIN MIDI -----------+
|
USB MIDI -----------+
|
v
PARAMETER REGISTRY
|
+---------+---------+
| | |
DSP PRESET MOD
|
+---------------> ESP32-S3 #2
|
+---------------> ESP32-S3 #3
A parameter therefore has one authoritative identity regardless of how it is changed.
This architecture is intended to keep touchscreen, MIDI, preset recall and remote processors synchronized.
AMUSEA supports both traditional hardware MIDI and native USB MIDI.
Serial DIN MIDI uses the Arduino MIDI library.
Native ESP32-S3 USB MIDI is used simultaneously.
The USB device identifies itself as:
USBMIDI USBMidi("RTAL MIDI");Both interfaces can control the synthesizer concurrently.
The intention is to expose essentially the complete synthesizer parameter set through MIDI CC wherever practical.
This includes:
- Oscillators
- Mixer
- Filters
- Envelopes
- LFOs
- Modulation
- Voice parameters
- Arpeggiator
- Sequencer
- Effects
- Performance parameters
MIDI and touchscreen editing share the same internal parameter registry.
MIDI Clock can be received through both:
- DIN MIDI
- USB MIDI
The two interfaces are monitored independently.
DIN F8 ----> DIN Clock Counter
USB F8 ----> USB Clock Counter
This is important because both interfaces may be connected simultaneously.
Transport commands are also supported.
FA START -> RUN
FB CONTINUE -> RUN
FC STOP -> STOP
Planned clock source modes:
INTERNAL
DIN
USB
AUTO
AUTO mode will select a valid external clock source while preventing simultaneous DIN and USB clock streams from accidentally doubling the sequencer tempo.
A comprehensive arpeggiator is planned.
Possible modes include:
- Up
- Down
- Up/Down
- Down/Up
- Order
- Random
- Chord
- Pattern
Planned features include:
- MIDI Clock synchronization
- Internal clock
- Octave range
- Gate length
- Swing
- Probability
- Latch/Hold
- Velocity processing
- Pattern variations
AMUSEA is also planned to include a graphical polyphonic sequencer.
The touchscreen makes it possible to edit notes and modulation directly.
Potential features include:
- Polyphonic note recording
- Step editing
- Real-time recording
- Velocity
- Gate
- Probability
- Ratchets
- Ties
- Parameter automation
- Modulation lanes
- MIDI Clock synchronization
The third ESP32-S3 receives digital audio from the synthesis processors.
SYNTH ENGINE
|
+-----------+-----------+
| | |
DRY MOD FX SEND DELAY SEND
| | |
| | REVERB SEND
| | |
+-----------+-----------+
|
v
ESP32-S3 #3
|
+-----------+-----------+
| | |
MOD FX DELAY REVERB
| | |
+-----------+-----------+
|
MASTER DSP
|
v
I2S DAC
|
v
STEREO OUT
The exact inter-processor audio protocol will be optimized during development.
TDM digital audio is one of the primary candidates.
AMUSEA is intended to go considerably beyond a simple chorus-and-delay architecture.
- Chorus
- Ensemble
- Flanger
- Phaser
- Vibrato
- Digital Delay
- Analog-Style Delay
- Tape-Style Delay
- Ping-Pong Delay
- Multi-Tap Delay
- Diffusion Delay
- Room
- Plate
- Hall
- Dark Hall
- Diffusion Reverb
Additional processing may include:
- Saturation
- Stereo Width
- HP/LP Feedback Filtering
- Tempo Synchronization
- Freeze
- Ducking
- Cross Feedback
- Modulated Delay Lines
Sounds are stored on SD card.
The planned directory structure is:
/RTAL_AMUSEA/
PRESETS/
PERFORMANCES/
SYSTEM/
Preset files will be versioned rather than being simple memory dumps.
This allows future firmware versions to evolve without unnecessarily invalidating older sounds.
Planned preset metadata includes:
- Preset Name
- Category
- Author
- Format Version
- Firmware Version
- Synthesis Parameters
- Modulation Matrix
- Effects Settings
- Performance Settings
- Checksum
Real-time audio always has priority over the graphical interface.
ESP32-S3 #1
CORE 1
|
+-- AudioTask
|
+-- Voices 1-4
+-- Audio Processing
+-- Digital Audio Transport
CORE 0
|
+-- MIDI
+-- USB MIDI
+-- Touch
+-- LVGL
+-- SD
+-- Sequencer
+-- Arpeggiator
+-- Inter-Processor Control
The second and third processors follow the same philosophy:
Core 1 for time-critical DSP.
Core 0 for communication and control.
AMUSEA is not intended to be a collection of computationally expensive textbook DSP algorithms.
The implementation is optimized specifically for ESP32-S3.
Where appropriate, the audio hot path will avoid expensive operations such as:
powf()
tanhf()
sinf()
cosf()
sqrtf()
Instead, techniques such as:
- Lookup tables
- Polynomial approximations
- Precomputed coefficients
- Control-rate calculations
- Block-rate parameter updates
- Parameter smoothing
- Carefully selected oversampling
will be used.
Processing power should be spent where it produces an audible improvement.
Arduino IDE 2.3.10
Arduino-ESP32 3.3.11
MCU ESP32-S3
Display Guition JC4827W543
GUI LVGL 9
Touch GT911
MIDI DIN + Native USB MIDI
Storage SD Card
Audio I2S / TDM
M001
|
+-- Platform Bring-Up
+-- LVGL / Touch
+-- DIN MIDI / USB MIDI
+-- MIDI Clock
+-- SD
+-- I2S
+-- 4-Voice Test Engine
|
v
M002
|
+-- Complete AMUSEA GUI
+-- Parameter Architecture
+-- Clock Source Arbitration
+-- Inter-Processor Framework
|
v
M003
|
+-- Single High-Quality AMUSEA Voice
|
v
M004
|
+-- Oscillator Architecture
+-- Mod Oscillator
+-- Ring Modulation
+-- Analog-Style Mixer
|
v
M005
|
+-- Dual Nonlinear Ladder Filter
|
v
M006
|
+-- Envelopes
+-- LFOs
+-- Modulation Matrix
|
v
M007
|
+-- Four Local Production Voices
|
v
M008
|
+-- Global 8-Voice Allocator
|
v
M009
|
+-- ESP32-S3 #2
+-- Voices 5-8
+-- Digital Audio Link
|
v
M010
|
+-- ESP32-S3 #3
+-- Stereo Multi-FX
|
v
M011
|
+-- Arpeggiator
+-- Sequencer
+-- MIDI Clock
|
v
M012
|
+-- Timbre A/B
+-- Split
+-- Stack
|
v
M013
|
+-- Preset System
+-- Performance System
|
v
M014+
|
+-- Sound Character Optimization
+-- Performance Profiling
+-- Stability Testing
The number of features is not the primary objective.
The central question is:
How close can a carefully optimized multi-ESP32-S3 architecture get to the behaviour and musical character of a sophisticated polyphonic analog synthesizer?
Particular attention will therefore be given to the complete signal chain:
OSCILLATORS
|
v
MIXER NONLINEARITY
|
v
FILTER INPUT DRIVE
|
v
DUAL LADDER FILTER
|
v
RESONANCE INTERACTION
|
v
VCA
|
v
VOICE STEREO POSITION
|
v
STEREO EFFECTS
The individual blocks will not be evaluated only in isolation.
Their interaction is what ultimately determines the character of the instrument.
Although the Moog Muse is an important source of technical and sonic inspiration for AMUSEA, the project is deliberately free to move beyond that architecture.
Potential AMUSEA-specific extensions include:
- Larger modulation matrix
- Additional modulation sources
- Expanded stereo architecture
- Extensive MIDI control
- Additional envelopes
- Deeper sequencer modulation
- Multi-effects
- Multiple delay algorithms
- Multiple reverbs
- Chorus
- Ensemble
- Flanger
- Phaser
- User macros
- Experimental voice modes
AMUSEA is therefore best understood as an independent ESP32-S3 synthesizer inspired by a remarkable analog instrument rather than a literal hardware replica.
The first platform milestone, AMUSEA M001, establishes the technical foundation.
Development is now moving toward the complete user interface and, most importantly, the first high-quality AMUSEA synthesis voice.
The project is under active development.
Expect experiments.
Expect measurements.
Expect discarded algorithms.
And hopefully, eventually:
A very large analog-inspired sound coming from a surprisingly small collection of ESP32-S3 processors.
RTAL-EAI-009-Amusea-Synthesizer
AMUSEA is part of the RealTimeAudioLab Experimental Audio Instruments series.
RTAL-EAI-009
AMUSEA
RealTimeAudioLab (RTAL)
Experimental audio electronics, embedded DSP, synthesizers, samplers and MIDI systems.
AMUSEA is an independent experimental open-source development project by RealTimeAudioLab.
It is not affiliated with, endorsed by, sponsored by, or associated with Moog Music.
The names Moog and Muse are referenced only to identify and describe the historical, technical and sonic inspiration for this independent experimental project.
All trademarks and product names belong to their respective owners.
WORK IN PROGRESS
Hardware architecture, DSP algorithms, user interface and specifications may change substantially during development.
Follow the repository to watch AMUSEA evolve from the first ESP32-S3 test voice into a complete eight-voice instrument.
RealTimeAudioLab - RTAL-EAI-009