Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Panic on large inputs? #21

Open
corani opened this issue Sep 21, 2023 · 1 comment
Open

Panic on large inputs? #21

corani opened this issue Sep 21, 2023 · 1 comment

Comments

@corani
Copy link

corani commented Sep 21, 2023

Is the example supposed to panic if the input is too large? With ~2400 characters I get:

ggml_new_tensor_impl: not enough space in the context's memory pool (needed 271388624, available 260703040)
SIGSEGV: segmentation violation
PC=0x4d2645 m=0 sigcode=1
signal arrived during cgo execution
@corani
Copy link
Author

corani commented Sep 22, 2023

As a workaround I've added an N *= 2; in bert_embeddings, with that things work fine: if the input it too large, I get an Too many tokens, maximum is 512 printed to the output.

Obviously this isn't the right solution, but it seems somewhere a calculation is off.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant