Discussions

• 1 You must be logged in to vote

  🙏

  cublas can run now but max token size is greatly reduced.

  hiqsociety asked Sep 21, 2023 in Q&A · Closed · Unanswered

  

  

  4
• 1 You must be logged in to vote

  🙏

  how to disable f16memory and keep the memory within the llama.cpp settings? i'm oom ed with go-llama as below

  hiqsociety asked Sep 25, 2023 in Q&A · Unanswered

  

  

  0
• 1 You must be logged in to vote

  🙏

  inference from the same model for multiple concurrent users

  alph4b3th asked Sep 19, 2023 in Q&A · Unanswered

  

  

  0