Skip to content

DataTableStore so that RewriteRpc does not need to push ExecutionContext or DataTable over the wire. #5260

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

jkschneider
Copy link
Member

@jkschneider jkschneider commented Apr 6, 2025

What's changed?

ExecutionContext should refer to DataTableStore rather than directly storing rows on itself in memory. By default, Java-only recipe execution can still use an InMemoryDataTableStore that essentially behaves as it does now.

However, this will enable us to inject a different implementation into recipe runs that use RewriteRpc so that each process can push rows to a single store without necessarily passing these rows over the wire as control passes between the processes.

This will also ultimately help solve large data table downloading in the SaaS.

DataTableStore has an acceptRows method on it that can be used by Preconditions implementations to essentially shut off the flow of rows into the store by recipes being used as preconditions.

The implementation of openrewrite/rewrite-maven-plugin#751 would have to change accordingly.

@shanman190
Copy link
Contributor

shanman190 commented Apr 6, 2025

It's linked from the Maven issue, but figured that I'd link it here directly too. Here's the parallel Gradle PR for exporting data tables that would need to be adjusted as well.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Status: In Progress
Development

Successfully merging this pull request may close these issues.

2 participants