Skip to content
This repository was archived by the owner on Aug 30, 2020. It is now read-only.
This repository was archived by the owner on Aug 30, 2020. It is now read-only.

Clarification on intent handling / Remote Server needed #217

@patrickjane

Description

@patrickjane

I want to use rhasspy to build a voice assistant which (for now) contains the following functionality:

  • control lights via home assistant
  • control heater via home assistant
  • get train departure information (from the internet)
  • get weather info (from the internet)

As you can see, not all of those tasks are handled by home assistant. In fact, I have existing implementations (in python) for Snips.ai for all those tasks.
To reuse them, I am developing some kind of plugin-based skill server, where I can hook on my existing code from snips (with, of course, slight modifications).

Now, regarding rhasspy, I understand that it can have several endpoints for intent handling:

  • home assistant (not viable for me as I need to handle non-HA-related intents)
  • node-red (not compatible with my existing skill implementations)
  • remote server <<< this is what I am trying to use

From the documentation I understand that rhasspy will HTTP POST any intent which was detected to my server. This works. I can see the intent JSON coming in at my skill server. However, it is unclear to me what kind of HTTP response is expected. From the documentation I can see it must be JSON, however I fail to find a detailed description of how this JSON should look like.

If I return an empty JSON, rhasspy complains: TypeError: e.data.intent is undefined (I'll get this error as popup in the rhasspy browser, not in the rhasspy log files)
The log looks like:

May  1 11:32:56 calypso rhasspy[547]: DEBUG:InboxActor: -> stopped
May  1 11:32:56 calypso rhasspy[547]: DEBUG:__main__:{"intent": {"name": "GetTemperature", "confidence": 1.0}, "entities": [], "text": "wie warm ist es gerade", "raw_text": "wie warm ist es gerade", "recognize_seconds": 0.0030146099998091813, "tokens": ["wie", "warm", "ist", "es", "gerade"], "raw_tokens": ["wie", "warm", "ist", "es", "gerade"], "wav_seconds": 0.0, "transcribe_seconds": 0.0, "speech_confidence": 1, "slots": {}, "wakeId": "", "siteId": "default", "time_sec": 0.014754056930541992}
May  1 11:32:56 calypso rhasspy[547]: DEBUG:urllib3.connectionpool:Starting new HTTP connection (1): 10.0.60.40:8888
May  1 11:32:56 calypso rhasspy[547]: DEBUG:urllib3.connectionpool:http://10.0.60.40:8888 "POST /intents HTTP/1.1" 200 2
May  1 11:32:56 calypso rhasspy[547]: DEBUG:RemoteIntentHandler:{}
May  1 11:32:56 calypso rhasspy[547]: DEBUG:InboxActor: -> stopped
May  1 11:32:56 calypso rhasspy[547]: [2020-05-01 11:32:56,159] 10.0.60.40:61297 POST /api/text-to-intent 1.1 200 2 110961
May  1 11:32:56 calypso rhasspy[547]: INFO:quart.serving:10.0.60.40:61297 POST /api/text-to-intent 1.1 200 2 110961

If I return { "intent": {}}, rhasspy complains: TypeError: e.data.time_sec is undefined (I'll get this error as popup in the rhasspy browser, not in the rhasspy log files)
The log looks like:

May  1 11:35:38 calypso rhasspy[547]: DEBUG:InboxActor: -> stopped
May  1 11:35:38 calypso rhasspy[547]: DEBUG:__main__:{"intent": {"name": "GetTemperature", "confidence": 1.0}, "entities": [], "text": "wie warm ist es gerade", "raw_text": "wie warm ist es gerade", "recognize_seconds": 0.0030003929987287847, "tokens": ["wie", "warm", "ist", "es", "gerade"], "raw_tokens": ["wie", "warm", "ist", "es", "gerade"], "wav_seconds": 0.0, "transcribe_seconds": 0.0, "speech_confidence": 1, "slots": {}, "wakeId": "", "siteId": "default", "time_sec": 0.01453399658203125}
May  1 11:35:38 calypso rhasspy[547]: DEBUG:urllib3.connectionpool:Starting new HTTP connection (1): 10.0.60.40:8888
May  1 11:35:38 calypso rhasspy[547]: DEBUG:urllib3.connectionpool:http://10.0.60.40:8888 "POST /intents HTTP/1.1" 200 14
May  1 11:35:38 calypso rhasspy[547]: DEBUG:RemoteIntentHandler:{'intent': {}}
May  1 11:35:38 calypso rhasspy[547]: DEBUG:InboxActor: -> stopped
May  1 11:35:38 calypso rhasspy[547]: [2020-05-01 11:35:38,979] 10.0.60.40:61320 POST /api/text-to-intent 1.1 200 13 120977
May  1 11:35:38 calypso rhasspy[547]: INFO:quart.serving:10.0.60.40:61320 POST /api/text-to-intent 1.1 200 13 120977

From the documentation I can see that in case of outputting speech, this should be given:

{
  ...
  "speech": {
    "text": "Some text to speak."
  }
}

and in case of forwarding something (what?) to home assistant, this should be given:

{
  // rest of input JSON
  // ...
  "hass_event": {
    "event_type": "...",
    "event_data": {
      "key": "value",
      // ...
    }
  }
}

(and in this case: what is 'rest of input JSON'?)

So, long story short, what do I need to send back to rhasspy after my remote server has successfully handled some intent which was detected by rhasspy and send to my remote server?

And what is the idea of the "forward to home assistant" feature? I mean if my remote server shall handle the intent, why forward anything else to home assistant? Is this meant to be some kind of light-wrapper for the HA-API in order to enable the remote server to easily generate HA events in addition to its very own intent handling?

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions