Fixed the issue where using built-in functions in PromptTemplate caused validation failures, and added a method:skipValidate() to skip validation for PromptTemplate.

[sunyuhan1998]

As mentioned in #2456, when using PromptTemplate, if the template contains certain built-in functions, it fails validation during render(). I made the following two fixes and optimizations:

1. By comparing the function names in org.stringtemplate.v4.compiler.Compiler#funcs, I skip the validation of these built-in functions.
2. Added a skipValidate() method to allow users to actively skip this validation.

[ThomasVitale]

@sunyuhan1998 Thanks for your contribution! #2780 introduced a new TemplateRenderer API to decouple PromptTemplate from the specific template engine used. The ST logic has been moved to StTemplateRenderer which now allows configuring the validation behaviour (throw, warn, none). This new structure also allows providing a fully custom TemplateRenderer to use ST in a different way than the default implementation and get full flexibility.

Would that address the case where built-in functions are used and the validation needs to be skipped?

[sunyuhan1998]

@sunyuhan1998 Thanks for your contribution! #2780 introduced a new TemplateRenderer API to decouple PromptTemplate from the specific template engine used. The ST logic has been moved to StTemplateRenderer which now allows configuring the validation behaviour (throw, warn, none). This new structure also allows providing a fully custom TemplateRenderer to use ST in a different way than the default implementation and get full flexibility.

Would that address the case where built-in functions are used and the validation needs to be skipped?

Hi, @ThomasVitale!
I've reviewed this PR, and I think decoupling the specific template engine implementation from PromptTemplate is a wise choice. The StTemplateRenderer is also more user-friendly now that it supports different validation behaviors. However, regarding the built-in functions, do you think we should still skip their validation? Consider this scenario: I'm using the new StTemplateRenderer, and I want an exception to be thrown during the render() call if I forget to replace certain variables in the prompt. So, I set validationMode to THROW. But since I'm using built-in functions, even though I've replaced all the variables, it seems I would still receive an IllegalStateException?

[markpollack]

This change is still good, but needs to be updated to reflect the changes to switching to a TemplateRenderer interface. The StringTemplate implementation is in StTemplateRenderer which does allow to disable validation completely, but the work you did here @sunyuhan1998 is still a nice addition for when validation is enabled and it skips built in functions. Would you be able to submit a new PR with the same functionality but on the latest code base please?

[sunyuhan1998]

This change is still good, but needs to be updated to reflect the changes to switching to a TemplateRenderer interface. The StringTemplate implementation is in StTemplateRenderer which does allow to disable validation completely, but the work you did here @sunyuhan1998 is still a nice addition for when validation is enabled and it skips built in functions. Would you be able to submit a new PR with the same functionality but on the latest code base please?

Hi! @markpollack
No problem at all. I will submit a PR shortly to implement the skip built-in functions functionality on StTemplateRenderer. Once I do, I'll reply to you here.

[sunyuhan1998]

This change is still good, but needs to be updated to reflect the changes to switching to a TemplateRenderer interface. The StringTemplate implementation is in StTemplateRenderer which does allow to disable validation completely, but the work you did here @sunyuhan1998 is still a nice addition for when validation is enabled and it skips built in functions. Would you be able to submit a new PR with the same functionality but on the latest code base please?

Hi! @markpollack
I've submitted a new PR(#2956) to allow users to choose whether to skip validation of built-in functions in StTemplateRenderer. Could you please review it when you have time? Thanks!

[markpollack]

forgot to close. Thanks @sunyuhan1998 additional changes in 0e15197