[pull] master from apache:master - #511
Merged
Merged
Conversation
…#39168) Bumps [github.com/nats-io/nats-server/v2](https://github.com/nats-io/nats-server) from 2.14.2 to 2.14.3. - [Release notes](https://github.com/nats-io/nats-server/releases) - [Changelog](https://github.com/nats-io/nats-server/blob/main/RELEASES.md) - [Commits](nats-io/nats-server@v2.14.2...v2.14.3) --- updated-dependencies: - dependency-name: github.com/nats-io/nats-server/v2 dependency-version: 2.14.3 dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Bumps [github.com/aws/aws-sdk-go-v2/config](https://github.com/aws/aws-sdk-go-v2) from 1.32.25 to 1.32.26. - [Release notes](https://github.com/aws/aws-sdk-go-v2/releases) - [Commits](aws/aws-sdk-go-v2@config/v1.32.25...config/v1.32.26) --- updated-dependencies: - dependency-name: github.com/aws/aws-sdk-go-v2/config dependency-version: 1.32.26 dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Bumps [js-yaml](https://github.com/nodeca/js-yaml) from 3.14.2 to 3.15.0. - [Changelog](https://github.com/nodeca/js-yaml/blob/master/CHANGELOG.md) - [Commits](nodeca/js-yaml@3.14.2...3.15.0) --- updated-dependencies: - dependency-name: js-yaml dependency-version: 3.15.0 dependency-type: indirect ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Bumps com.gradle.common-custom-user-data-gradle-plugin from 2.6.0 to 2.7.0. --- updated-dependencies: - dependency-name: com.gradle.common-custom-user-data-gradle-plugin dependency-version: 2.7.0 dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Bumps [github.com/aws/aws-sdk-go-v2/feature/s3/manager](https://github.com/aws/aws-sdk-go-v2) from 1.22.28 to 1.22.29. - [Release notes](https://github.com/aws/aws-sdk-go-v2/releases) - [Commits](aws/aws-sdk-go-v2@feature/s3/manager/v1.22.28...feature/s3/manager/v1.22.29) --- updated-dependencies: - dependency-name: github.com/aws/aws-sdk-go-v2/feature/s3/manager dependency-version: 1.22.29 dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Derrick Williams <derrickaw@google.com>
* Add embedded NVIDIA Dynamo support to vLLM ModelHandler VLLMCompletionsModelHandler and VLLMChatModelHandler gain two keyword-only parameters, use_dynamo (default False) and dynamo_frontend_kwargs. When use_dynamo=True, the handler launches a dynamo.frontend process as the OpenAI-compatible local endpoint plus a separate dynamo.vllm worker, instead of vllm.entrypoints.openai.api_server. The existing native-vLLM path is unchanged when the flag is absent. The example pipeline vllm_text_completion.py gains --use_dynamo and --max_tokens flags. validate_inference_args is now a no-op on both handlers so OpenAI-style request kwargs (e.g. max_tokens) can be passed through RunInference. A new unit-test module covers process-launch behaviour for both paths. This supersedes #36966 (now closed) and rebases the embedded-Dynamo approach onto current master, preserving the recent batching-kwargs additions to the ModelHandler base. Co-authored-by: Danny McCormick <damccorm@google.com> * Harden _VLLMModelServer process lifecycle per code review Apply five robustness fixes flagged on PR #38701: - Track the temporary etcd data dir as self._etcd_data_dir and shutil.rmtree(..., ignore_errors=True) it in _stop_processes so worker restarts don't leak /tmp directories. - Wrap process.terminate() / process.wait() / process.kill() in a single try/except OSError to absorb the ProcessLookupError race when a process exits between poll() and the signal call. - Switch the ETCD_ENDPOINTS removal from `del os.environ[...]` to `os.environ.pop(..., None)` to be idempotent. - Wrap __del__ in try/except Exception so cleanup never raises during interpreter shutdown. - Add the embedded etcd process to the check_connectivity() poll loop so an etcd death fails fast instead of waiting out the 10-minute timeout. * Enable Dataflow IT for embedded Dynamo on T4 Bump vllm.dockerfile.old to apache-beam[gcp]==2.71.0 (and the COPY-from beam_python3.12_sdk image to 2.71.0), install ai-dynamo[vllm], and add the etcd binary required by embedded Dynamo's runtime discovery. Uncomment the Dynamo IT block in common.gradle. Drop the unused machine_type override so it inherits n1-standard-4 from argMap, and switch nvidia-l4 -> nvidia-tesla-t4 to match the existing native vLLM ITs and the local Dataflow validation (per @damccorm review). Validated end-to-end on Dataflow with Qwen/Qwen3-0.6B; the nvext.timing field present on every PredictionResult confirms the Dynamo frontend served the requests. * Trigger Python PostCommit for Dynamo IT Bump the beam_PostCommit_Python trigger file so the postcommit suite (inferencePostCommitITPy312 -> vllmTests) runs the embedded Dynamo IT against the rebuilt apache-beam-testing vLLM image. * fix: run Dynamo vLLM IT separately in py312 PostCommit Split vllmDynamoTests from vllmTests so py312 validates Dynamo without blocking on the pre-existing native opt-125m hang in apache-beam-testing. * Restore full vLLM postcommit suite * fix: fold Dynamo IT back into vllmTests for py312 PostCommit Revert the Option A split now that the native opt-125m vLLM hang is fixed (3.12 PostCommit passed in ~2.5h). vllmTests again runs completion -> chat -> Dynamo as a single suite; the separate vllmDynamoTests task is removed. Bump PostCommit trigger to re-run. --------- Co-authored-by: Danny McCormick <damccorm@google.com>
* Fix DirectRunner PubSub subscriber client cleanup * Avoid global lock during PubSub subscription creation
* Retry starting SubprocessServer on failure Wraps the subprocess startup and connection sequence in a retry loop with a limit of 3 attempts. This mitigates "address already in use" errors caused by race conditions during dynamic port allocation. * Formatting * Apply suggestion Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> --------- Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
See Commits and Changes for more details.
Created by
pull[bot] (v2.0.0-alpha.4)
Can you help keep this open source service alive? 💖 Please sponsor : )