A web application illustrating tokenization methods when selecting certain LLMs.
Start the project by opening it in VS and running command in terminal: dotnet run
Start the script of tokenization by opening it in PyCharm and running command in terminal: uvicorn main:app --reload