-
Notifications
You must be signed in to change notification settings - Fork 6
Open
Labels
bountyWe will pay to get this done. Needs to be clean code!We will pay to get this done. Needs to be clean code!
Description
billing speed up and reliability:
- directly hit db for billing
- use a similar "lazy" mechanism for hitting the db as we do for the stats (check on first connect, cache info, queue updates to db, auto-transactions). abstract this mechanism so we have a single db update thread for billing and stats updates!
- track account credits internally in ram for recent users (LRU cache), so we don't have to check billing at all for inference continuation (better to err on allowing small overdraft too)
- remove endpoint from vercel / reliance on vercel
Metadata
Metadata
Assignees
Labels
bountyWe will pay to get this done. Needs to be clean code!We will pay to get this done. Needs to be clean code!