audio-enabled navigation labels (TOC headings)

[danielweck]

Work in progress, demo

See:
w3c/epub-specs#647 (comment)

CC @becka11y

[danielweck]

Dear all,

Yesterday we discussed structured-audio / audio-only talking books
based on EPUB3 Media Overlays, and more specifically: support for
pre-recorded audio in speech-enabled navigation headings (conceptually
similar to the TOC audio labels in DAISY NCX).
To demonstrate the feasibility of this new reading system feature, I
implemented a Readium proof-of-concept based on the online "cloud
reader" (and the offline Chrome extension, as they both share the same
code).

Here is a link to the online prototype:
http://readium-smilnav.surge.sh/?epub=moby-dick-mo-TOC

(note that in this demo, only the TOC labels for chapter one and
two have SMIL audio, so most navigation headings are in fact silent)

Usage instructions:

1. load above link in web browser, wait for publication to load (may
   take some time depending on network latency)
2. press keyboard key t, or click "table of contents" icon (this
   should open navigation pane on left hand side)
3. hit arrow down key five times in order to reach "chapter 1"
   heading, wait for audio label to start playing automatically, wait for
   end of playback (note that HTML5 audio innacurracies may result from
   slow network)
4. hit arrow down key once to listen to "chapter 2" phrase, hit
   arrow up key once to interrupt current playback, and wait to hear
   audio for previous heading
5. keep arrow down or arrow up pressed to quickly skim list of
   headings, when key released, playback should start for currently
   focused heading (if this heading contains audio, naturally)
6. go to chapter 1, and start playing regular Media Overlays (hit key
   m or use the "play" icon buttons)
7. re-focus keyboard on audio-enabled TOC headings, and see how MO
   playback pauses to avoid overlapping audio
8. change MO playback rate from 1 to faster (e.g. 3), and listen
   to TOC audio again (the speed should be identical)

Explanation: there is nothing special about the SMIL or any other
parts in this EPUB3 example. I just used the IDPF Moby Dick sample
that has two chapters with Media Overlays, and I hand-crafted the SMIL
for the navigation document, linked below:
http://readium-smilnav.surge.sh/moby-dick-mo-TOC/OPS/nav.smil
Note that I did not re-create the audio file, I simply copied the clip
being+end timing values that are used for the main HTML content
documents (which contain the same heading texts as the nav-doc, so the
audio narration matches too).
The NavDoc does not need to be in the spine for this feature to
work. All that's needed is the usual association between the HTML
manifest item and its SMIL counterpart, in the OPF package definition.

The code change necessary to implement this new reading system feature
is not complicated, and definitely much simpler than to support SMIL
synchronisation arcs and event-based timing:
develop...feature/smilToc

Regards, Daniel

[danielweck]

Navigable audio-only EPUB3 Guidelines
Guidelines for Navigable Audio Only EPUB 3 Publications
Version: 1.0
Release date: April 24, 2015

http://www.daisy.org/ties/navigable-audio-only-epub3-guidelines

[danielweck]

updated specification link: https://daisy.org/info-help/guidance-training/standards/navigable-audio-only-epub3-guidelines/