[Roadmap] sLM for WelsonJS project

[gnh1201]

Summary

I am considering integrating sLM into the WelsonJS project. The leading candidate currently under review is Phi-3.5-mini.

If there are any updates on this matter, I will comment them in this issue.

Related links

• https://huggingface.co/microsoft/Phi-3.5-mini-instruct
• https://huggingface.co/SicariusSicariiStuff/Phi-3.5-mini-instruct_Uncensored
• https://github.com/microsoft/Phi-3CookBook/blob/main/code/04.Finetuning/Phi_3_Inference_Finetuning.ipynb

[km19809]

What runtime are you going to use?
I found these options, but is one applicable to WelonJS?
(Of course, I need to modify some codes.)

어떤 런타임을 사용하여 sLM을 실행하실 건지 궁금합니다.
아래 목록 중 WelonJS에 적용가능한 게 있을까요?
(당연히, 코드를 좀 수정해야 할 것입니다.)

• https://github.com/withcatai/node-llama-cpp Node.js binding of llama.cpp
• https://github.com/LlamaEdge/LlamaEdge WASI binding of llama.cpp

[gnh1201]

There is no decision yet on which runtime will be used.

However, there has been a recent requirement to use LLMs to control Windows-based systems, so we are looking for ways to address this.

The most likely option is to provide support using a separate server program. I have plans to provide support through integration with WelsonJS, which I'm currently working on in another (aka. Caterpillar) project.

It will likely be supported via JSON-RPC 2.0 based communication.

어떤 런타임을 사용할지 현재 정해진 것은 없습니다.

다만, 최근에 LLM을 Windows 기반 시스템을 제어하는데 사용하고자 하는 요건이 있어 대응할 방안을 찾고 있습니다.

현재 유력한 것은 별도의 서버 프로그램을 이용한 지원입니다. 현재 제가 진행중인 다른 프로젝트 (송충이 프로젝트)를 통해 WelsonJS와 연동하여 지원 계획이 있습니다.

JSON-RPC 2.0 기반 통신으로 지원하지 않을까 합니다.

• https://github.com/gnh1201/caterpillar (Caterpillar Project)