Question about context being passed to chatlab functions.

[smeggingsmegger]

Question

When using chatlab inside of FastAPI, I may have authentication data that I would need to be passed to a chatlab function that's been registered. For instance, let's say I had an auth-token that I resolve to a user object, and I want to send the user_id to the chatlab function, how could I make that happen? Is that even possible right now?

I feel like I must be missing something very obvious here but I am unsure.

[rgbkrk]

I realize you've closed this already but this is a really good question. For any readers that get here, the chatlab.Chat class is intended for interactive use (particularly in notebooks). However, chatlab.FunctionRegistry is the foundation that works for servers, cli tools, etc., forming the base of a typical workflow:

• Register functions on demand
• Pull the schema to pass to OpenAI (or functionary)
• Send the schema to OpenAI along with the rest of the chat context
• Call the function, possibly augmented with a user parameter or context

At the base layer, the function you have to expose for the LLM has to already have the credentials bound in some way or for our calling of the function to inject the extra parameter. My preferred way to do this would be to have a class you can instantiate with the user parameter.

from typing import List

class User:
    pass

def books_loved_by_user(user: User, filter: List[str]):
    return ["Database Reliability Engineering"]

class API:
    def __init__(self, user: User):
        self.user = user
    def favorite_books(self, filter: List[str]):
        """Get the user's favorite books"""
        return books_by_user(self.user, filter)

from chatlab import FunctionRegistry

user = User()
api = API(user)

fr = FunctionRegistry()
fr.register(api.favorite_books)
print(fr.api_manifest())

{'functions': [{'name': 'favorite_books',
   'parameters': {'properties': {'filter': {'items': {'type': 'string'},
      'type': 'array'}},
    'required': ['filter'],
    'type': 'object'},
   'description': "Get the user's favorite books"}],
 'function_call': 'auto'}

This keeps the user in scope for the call while the function the model sees is without the self.

[smeggingsmegger]

Thank you @rgbkrk for that explanation! You are right that I figured that out on my own, felt bad for asking, and stealth-deleted my question. My intent was to add something to the documentation via pull request shortly. Thanks for the excellent library! It's very cool.

[rgbkrk]

Totally cool to ask! The docs definitely need it. I welcome the contribution. Feel free to use any, all, or none of what I wrote above if it helps.