Skip to content

Conversation

yatarkan
Copy link
Contributor

@yatarkan yatarkan commented Aug 15, 2025

Some models (e.g. openbmb/MiniCPM3-4B) have macro in their chat templates and execute them with call blocks.

This PR:

  • Enables call / endcall for calling macro (w/ macro arguments and caller() syntax) and passing a macro to another macro
  • Adds tests for call block syntax
  • Adds openbmb/MiniCPM3-4B to test models

Fixes #73

Copy link
Contributor

@ochafik ochafik left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is awesome, thanks @yatarkan !

prithivMLmods/Qwen2.5-7B-DeepSeek-R1-1M
prithivMLmods/QwQ-Math-IO-500M
prithivMLmods/Triangulum-v2-10B
qingy2024/Falcon3-2x10B-MoE-Instruct
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this removal accidental?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

EXPECT_THAT([]() { render("{% if 1 %}{% else %}{% elif 1 %}{% endif %}", {}, {}); }, ThrowsWithSubstr("Unterminated if"));
EXPECT_THAT([]() { render("{% filter trim %}", {}, {}); }, ThrowsWithSubstr("Unterminated filter"));
EXPECT_THAT([]() { render("{# ", {}, {}); }, ThrowsWithSubstr("Missing end of comment tag"));

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you please also add an unterminated call test:

EXPECT_THAT([]() { render("{%- call test -%}", {}, {}); }, ThrowsWithSubstr("Missing end of call tag"));

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tests for call and macro added

@yatarkan yatarkan requested a review from ochafik September 2, 2025 08:03
Copy link
Contributor

@ochafik ochafik left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks again @yatarkan !

I've just lost write access to this repo (left Google over 3 months ago 🤪), @matthewchan-g @Ferev could one of you please merge this one?

@matthewchan-g
Copy link
Collaborator

@ochafik I'm happy to help with merging the PR. There are currently 8 failing checks. Should the PR still be merged or should these failing checks be fixed first?

@CISC
Copy link

CISC commented Sep 9, 2025

It looks like the tests are failing because they are attempting to get gated files from HF.

@matthewchan-g matthewchan-g merged commit 3e4c61c into google:main Sep 9, 2025
3 of 11 checks passed
@CISC
Copy link

CISC commented Oct 10, 2025

Seems this introduces a memory leak:
ggml-org/llama.cpp#16500 (comment)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add support for call blocks

4 participants