feat: add support for local LLMs

Adds Ollama support (Gemma 3 models) to the web-codegen-scorer.

Author: hawkgs

README.md

@@ -40,6 +40,13 @@ export OPENAI_API_KEY="YOUR_API_KEY_HERE" # If you're using OpenAI models
 export ANTHROPIC_API_KEY="YOUR_API_KEY_HERE" # If you're using Anthropic models
 ```
 
+> [!NOTE]
+> Web Codegen Scorer supports locals models via Ollama as well. In order to use them, you must have a running Ollama server with the respective model(s) installed. By default, the tool is listening on port `11434` for the server. However, you can change that port by setting the `OLLAMA_PORT` environment variable.
+>
+> Be aware that using local models might sometimes lead to execution errors due to the output not conforming to our desired format. Unfortunately, this is a present-day limitation of these models. That being said, you can treat the feature as experimental.
+>
+> Currently supported models: `gemma3:4b`, `gemma3:12b`, `codegemma:7b`
+
 3. **Run an eval:**
 
    You can run your first eval using our Angular example with the following command:

package.json

@@ -70,6 +70,7 @@
     "file-type": "^21.0.0",
     "genkit": "^1.19.1",
     "genkitx-anthropic": "0.23.1",
+    "genkitx-ollama": "^1.19.2",
     "gpt-tokenizer": "^3.0.1",
     "handlebars": "^4.7.8",
     "limiter": "^3.0.0",
@@ -78,8 +79,8 @@
     "p-queue": "^8.1.0",
     "puppeteer": "^24.10.1",
     "sass": "^1.89.2",
-    "stylelint": "^16.21.1",
     "strict-csp": "^1.1.1",
+    "stylelint": "^16.21.1",
     "stylelint-config-recommended-scss": "^16.0.0",
     "tinyglobby": "^0.2.14",
     "tsx": "^4.20.3",