Skip to content

Open apps by name with voice commands - #392

Open
ziyadmohamed wants to merge 4 commits into
trypsynth:masterfrom
ziyadmohamed:voice-command-apps-and-visible-controls
Open

ziyadmohamed wants to merge 4 commits into
trypsynth:masterfrom
ziyadmohamed:voice-command-apps-and-visible-controls

Conversation

@ziyadmohamed

@ziyadmohamed ziyadmohamed commented Oct 31, 2026 •

Copy link
Copy Markdown

This adds an "Open " command to Backtalk's existing voice commands, addressing #244. Instead of opening the app drawer and finding an icon, you can say something like "Open WhatsApp".

The "Open apps by voice" switch is under Controls in Backtalk settings. It is off by default and hidden on watches and TVs. Matching uses the full launcher or application label and requires at least 85% similarity. The closest match wins; equal matches belonging to different apps are rejected with the usual "command not recognized" message. Multiple launcher entries for the same app are allowed. The "Open" prefix is currently English.

App names are queried for each request, so installations, removals and disabled apps are picked up without restarting Backtalk. The lookup runs on a worker thread, and a newer command, disabling the setting or stopping the service cancels a pending launch. With the switch enabled, recognition waits for a final speech result so "Open YouTube" cannot launch halfway through "Open YouTube Music". A recognition error or timeout cannot turn an unfinished phrase into an app launch. Starting a call cancels a pending app launch, and opening an app clears an unanswered call confirmation. Other phrases use the existing command handler, and leaving the switch off preserves the existing recognition timing.

I previously tested opening apps on a Samsung Galaxy A24 running Android 16, configured with Multilingual TTS 1.82 and Google's recognition service. The setting, phone/watch/TV filtering and interaction with call confirmations are covered by automated tests; I have not retested these changes on the phone. The documentation is now in the Voice commands section of differences.md.

The full testPhoneDebugUnitTest suite passed all 1,191 tests, including 18 app-command checks. :lintPhoneDebug and :lintWearDebug also passed. The tests cover fuzzy matching, ambiguous names and launcher aliases, failed or canceled launches, the default setting, phone/watch/TV filtering, interaction with call confirmations, and final speech results, errors and timeouts on Android 8 and Android 14.

The separate change for clicking visible controls with confirmation is in #394.

@trypsynth trypsynth left a comment

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks! Opening apps by name answers #244 well. It reuses the existing voice command path, it's off by default, and the strings and docs are done properly. Some things first:

  1. Please split it into two PRs: opening an app by name, and clicking a visible control by its label. The HandlerCompat.hasCallbacks change in SpeechRecognizerPerformer is a real fix for Android 8 and 9 on its own, and I'm fixing that on master separately, so you can drop it.
  2. Clicking by label is risky. Saying a visible label clicks it straight away with no confirmation, including "Delete", "Send", "Pay", "Uninstall" and keyboard keys. VisibleVoiceTarget.windows() (VisibleVoiceTarget.kt:35-60) doesn't filter window types, so the keyboard, System UI and Backtalk's own windows are searched. And labels are matched before built-in commands, so a visible "Home", "Back" or "Settings" item takes over that command whenever the switch is on. Please match built-in commands first, and leave out the keyboard and Backtalk's own windows.
  3. The new switch needs a TalkBackPreferenceFilter entry with HIDDEN_ON_WATCH | HIDDEN_ON_TV, like the other voice command settings, because voice commands aren't on watches.
  4. Minor: the differences.md section is under Backtalk menu, but it belongs with voice commands.

@ziyadmohamed ziyadmohamed changed the title Open apps and activate visible controls by voice Open apps by name with voice commands Oct 31, 2026
@amanKG777 amanKG777 mentioned this pull request Oct 31, 2026
@trypsynth

Copy link
Copy Markdown
Owner

Thanks, this conflicts with master now. Could you rebase it, and I'll review it again?

@ziyadmohamed
ziyadmohamed force-pushed the voice-command-apps-and-visible-controls branch from a12051a to 77d15f3 Compare November 2, 2026 11:43
@ziyadmohamed

Copy link
Copy Markdown
Author

Hi @trypsynth, the rebase is done and GitHub shows no merge conflicts. Validation on November 2 passed all 1,191 unit tests and both phone and watch lint checks. It's ready for another look when you have time. Thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants