New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

有无计划发布量化后的版本 #25

Open

yonghanzhuce opened this issue Nov 10, 2023 · 2 comments

yonghanzhuce commented Nov 10, 2023

想问一下有无计划发布int4，或int8量化后的模型

Collaborator

guoday commented Nov 22, 2023

已有社区开源了量化后的模型：https://huggingface.co/TheBloke/deepseek-coder-33B-instruct-GPTQ

shuaizai88 commented Jan 17, 2024

我的p40 显卡只支持fp32和int8 。。量化版本能支持用int8跑吗?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment