|
2 | 2 |
|
3 | 3 | ## Overview
|
4 | 4 |
|
5 |
| -Single page demonstration on running machine learning NLP models in the browser with transformers.js. |
| 5 | +This repository contains several demonstrations of running machine learning NLP models directly in the browser using transformers.js. |
6 | 6 |
|
7 |
| -### 1. Comparison Matrix |
| 7 | +### 1. Embeddings |
8 | 8 |
|
9 |
| -Visualizes the cosine similarity between sentences and p5.js for rendering a matrix. |
| 9 | +Embeds sentences using `Xenova/all-MiniLM-L6-v2` and visualizes the results in two different contexts: |
10 | 10 |
|
11 |
| -- Feature Extraction (embeddings): Xenova/all-MiniLM-L6-v2 |
| 11 | +- **Comparison Matrix**: Visualizes the cosine similarity between different sentences using p5.js to render a matrix. |
| 12 | +- **Clustering**: Plots sentence embeddings in a two-dimensional space with umap-js for dimensionality reduction. |
12 | 13 |
|
13 |
| -### 2. Clustering |
| 14 | +- Feature Extraction (embeddings): `Xenova/all-MiniLM-L6-v2` |
14 | 15 |
|
15 |
| -Embeds sentences using `Xenova/all-MiniLM-L6-v2` and plots them in a two-dimensional space with umap-js dimensionality reduction. |
| 16 | +### 2. Language Models |
16 | 17 |
|
17 |
| -- Feature Extraction (embeddings): Xenova/all-MiniLM-L6-v2 |
| 18 | +A set of demos using language models to generate or complete text interactively in the browser. |
18 | 19 |
|
19 |
| -### 3. Other Model Demos |
| 20 | +- **Chatbot Demo**: Implements a simple chatbot using `onnx-community/Qwen2.5-0.5B-Instruct`. |
| 21 | +- **Text Generation**: Generate text using various models including `onnx-community/Llama-3.2-1B-Instruct-q4f16` and `HuggingFaceTB/SmolLM-135M`. |
20 | 22 |
|
21 |
| -A few demos using other models from transformers.js. |
| 23 | +### 3. Vision Models |
22 | 24 |
|
23 |
| -- Summarization: Xenova/distilbart-cnn-6-6 |
24 |
| -- Text Generation: Xenova/LaMini-Flan-T5-783M |
25 |
| -- Text Classification: Xenova/toxic-bert |
| 25 | +- **Depth Estimation**: Estimates depth from an image using `onnx-community/depth-anything-v2-small` and visualizes it using p5.js. |
| 26 | +- **Object Detection**: Detects objects in images and video in real-time using `Xenova/detr-resnet-50` with p5.js for visualization. |
| 27 | + |
| 28 | +### 4. Whisper |
| 29 | + |
| 30 | +Implements a real-time audio transcription system using `Xenova/whisper-tiny.en`. |
26 | 31 |
|
27 | 32 | ### Example Code
|
28 | 33 |
|
@@ -57,11 +62,22 @@ For further information and a comprehensive list of supported tasks and models,
|
57 | 62 |
|
58 | 63 | The [Transformers.js Model Hub](https://huggingface.co/models?library=transformers.js) lists models compatible with the library, searchable by tasks.
|
59 | 64 |
|
60 |
| -## p5.js Integration and Instance Mode |
| 65 | +## p5.js Integration and Usage |
| 66 | + |
| 67 | +To integrate transformers.js with p5.js, instance mode can be used to enable compatibility with ES6 import statements required by transformers.js. However, dynamic import statements can also be used directly with p5.js as follows. |
61 | 68 |
|
62 |
| -To integrate transformers.js with p5.js, instance mode is required to enable compatibility with ES6 import statements required by transformers.js. |
| 69 | +### Example of Loading Transformers.js |
63 | 70 |
|
64 |
| -[Learn more about p5.js instance mode in this wiki](https://github.com/processing/p5.js/wiki/Global-and-instance-mode) or [instance mode video tutorial](https://youtu.be/Su792jEauZg). |
| 71 | +```javascript |
| 72 | +// Function to load transformers.js dynamically |
| 73 | +async function loadTransformers() { |
| 74 | + console.log('Loading transformers.js...'); |
| 75 | + const module = await import('https://cdn.jsdelivr.net/npm/@huggingface/transformers'); |
| 76 | + const { pipeline } = module; |
| 77 | + console.log('Transformers.js loaded successfully.'); |
| 78 | + return pipeline; |
| 79 | +} |
| 80 | +``` |
65 | 81 |
|
66 | 82 | ## What is UMAP?
|
67 | 83 |
|
|
0 commit comments