Skip to content

feat: update rust to pip install code block requirements #752

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

starpit
Copy link
Member

@starpit starpit commented Mar 11, 2025

  • adds a requirements attribute to CodeBlock
  • does not update the pdl_interpreter logic to deal with it
  • instead, updates the rust logic to scan the program and pip install those requirements
  • for now, those requirements are installed into the main venv used by the interpreter

TODO

The thought process here:

Logic: code-generate a new program such that each code block with a requirements attribute is rewritten to:

  1. remove the requirements attribute as it is no longer needed
  2. write the python code to a temp file, call it /tmp/logic.py for this discussion
  3. update the code attribute to look like
command: /path/to/venv/bin/python /tmp/logic.py

This should allow us to use the pdl_interpreter, where it is none the wiser of our venv caching.

@@ -462,6 +462,9 @@ class CodeBlock(LeafBlock):
]
"""Programming language of the code.
"""
requirements: Optional[str] = None
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should be only enabled for lang: python. Should we check for it?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nice catch. do you know how to express this in pydantic?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this issue seems to indicate that we could do it, but only with a custom model validator pydantic/pydantic#10252

Copy link
Member Author

@starpit starpit Mar 11, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was going to suggest Pydantic validator as well. Also wondering if requirements should actually be Optional[str | List[str]?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

very good! done

@starpit starpit force-pushed the pip-install-reqs branch 5 times, most recently from badb6ca to 447dc05 Compare March 11, 2025 23:15
@starpit starpit force-pushed the pip-install-reqs branch 6 times, most recently from 852b315 to c5ed197 Compare March 17, 2025 17:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants