Error during chat completion: TypeError: Cannot read properties of undefined (reading 'dims')

[jayasooryantm]

System Info

transformerjs@latest
os: ubuntu
browser: chrome

Environment/Platform

• Website/web-app
• Browser extension
• Server-side (e.g., Node.js, Deno, Bun)
• Desktop app (e.g., Electron)
• Other (e.g., VSCode extension)

Description

I have fine-tuned Xenova/distilgpt2 with corporate dataset (100 lines of corporate conversation). successfully fine-tuned and output was good in kaggle. I have used transformerjs convert python file to convert it to onnx format. with quantization, the model loaded in the browser but thrown an error while predicting: Error during chat completion: TypeError: Cannot read properties of undefined (reading 'dims'), what did I do wrong? below is the command I ran to convert the model:
python3 convert.py --quantize --model_id path/to/distilgpt2-corporate --output_parent_dir output/path/to/distilgpt2-corporate/models --task text-generation

Reproduction

jayasooryansd/distilgpt-corporate

[jayasooryantm]

So basically what I understood from 2-3 days of grinding on the transformerjs converter library, it is just a placeholder to show that users can convert the model to onnx, but in fact it doesn't work. I have tried various models, the models that are in the supported list is not working. so the answer is is @xenova converted the model you can use it in the browser, or else sound of crickets

[xenova]

Hi there 👋 Is the model on the HF hub? I can't seem to find it at https://huggingface.co/jayasooryansd/distilgpt-corporate

Also, can you provide the sample code you used to run it?

[jayasooryantm]

Hi there,

first of all transformerjs is a great library, because I have tried myself to implement ml model in browser and it was a nightmare. but the issue I have faced is bit specific I believe.
I am building a next-word prediction feature (just like the gmail feature), since server hosted models have latency more than 50ms product manager decided to go with browser based model. I have used various models like tiny-llama, gpt2, distilgpt, smollm, etc. i liked the Xenova/distilgpt2 which is fast and good result. but the problem with distilgpt2 results, are casual language and not for professional setup (since we don't have the option of context, language tone is very important). so I decided to fine-tune it, it worked in python. but when I convert it using the transformerjs converter module, the model is not working (throw this error: Cannot read properties of undefined (reading 'dims')) so i tried other models as well, same issue, and tried with optimum, and onnx converter itself, not able to do it. don't know what is the issue. I tried converting the model directly from HF, it has the same error, so maybe I am doing the conversion wrong or I am missing something. I will post the sample code you asked here, currently I don't have the access to code, it's in my office laptop. If you think there is another solution please suggest, or could you please show me how to convert the fine-tuned distilgpt2 model to run in transformerjs. btw the link(repo) i posted in HF is not available anymore, I have uploaded various model version and ended up deleting the entire repo, I will add the converted model too.
cheers

[xenova]

Sounds good 👍 Let me know when you can share the model id/link on the HF hub & code -- as it would make debugging a lot easier!

[jayasooryantm]

Hey,

here is the model: jayasooryansd/distilgpt-corporate
here is the code: https://www.kaggle.com/code/jayasooryantm/next-word-pred-fine-tune-distilgpt

i have included the training code, conversion command i used, sample code of running the model in browser.

[xenova]

Thanks for the additional information. It looks like you are facing this issue because you do not have a chat template in tokenizer_config.json, but you are trying to run it using a list of messages (objects with role & content fields).

This will look something like this:

"chat_template": "{% for message in messages %}{% if loop.first and messages[0]['role'] != 'system' %}{{ '<\|im_start\|>system\nYou are a helpful AI assistant named SmolLM, trained by Hugging Face<\|im_end\|>\n' }}{% endif %}{{'<\|im_start\|>' + message['role'] + '\n' + message['content'] + '<\|im_end\|>' + '\n'}}{% endfor %}{% if add_generation_prompt %}{{ '<\|im_start\|>assistant\n' }}{% endif %}",

Make sure to clear cache after updating the config on HF!