A Webapp that uses Retrieval Augmented Generation (RAG) and Large Language Models to interact with a PDF directly in the browser.
There are three main pillars this tool is based on:
Ask my PDF uses PDF.js to process a PDF locally and split the content up into separate lines.
Individual paragraphs and then sentences are extracted from the document and will be mapped to a multi-dimensional dense vector space using all-MiniLM-L6-v2 with TransformersJS. Those entries are then stored in an in-memory VectorDB directly in the browser.
As soon as a query is submitted, it is also vectorized and the cosine similarity search is used to find the most similar text sections, together with the lines surrounding them, so as not to lose the context.
The text sections found this way together with the query and a few instructions are then used as the input prompt to the Gemma2-2B or Gemma2-9B, compiled to WebAssembly and WebGPU using MLC LLM, which will then generate a response.
If available, also the Prompt API can be used to generate text. The Prompt API is an experimental project by Google Chrome, so it is only available in a few browsers yet.