Add Metal backend core ETMetal runtime. #15020
Open
+1,143
−0
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
🔗 Helpful Links🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/15020
Note: Links to docs will display an error until the docs builds have been completed. ❗ 1 Active SEVsThere are 1 currently active SEVs. If your PR is affected, please view them below: ❌ 3 New Failures, 1 Pending, 1 Unrelated FailureAs of commit 7e971b0 with merge base 896178e ( NEW FAILURES - The following jobs have failed:
FLAKY - The following job failed but was likely due to flakiness present on trunk:
This comment was automatically generated by Dr. CI and updates every 15 minutes. |
meta-cla
bot
added
the
CLA Signed
This was referenced Oct 10, 2025
manuelcandales
requested review from
larryliu0820 and
mergennachin
and removed request for
cccclai and
shoumikhin
| // Commit buffer and allow immediate reuse for better performance | ||
| [commandBuffer_ commit]; | ||
| ET_LOG(Debug, "ETMetalStream::commitAndContinue: Committed buffer %p with continue", commandBuffer_); | ||
|
|
There was a problem hiding this comment.
Don't you need to release the buffer after commit?
[commandBuffer_ release];
commandBuffer_ = nil;
Comment on lines
+274
to
+275
| if (cps_) [cps_ retain]; | ||
| if (func_) [func_ retain]; |
There was a problem hiding this comment.
Why do you need these retain lines? Aren't these already owned by the class?
Comment on lines
+280
to
+291
| // Don't release encoder_ here - the stream owns it | ||
| // Only clean up our own references | ||
| if (cps_) { | ||
| [cps_ release]; | ||
| cps_ = nil; | ||
| } | ||
| if (func_) { | ||
| [func_ release]; | ||
| func_ = nil; | ||
| } | ||
|
|
||
| encoder_ = nil; // Clear reference without releasing |
There was a problem hiding this comment.
Just this?
cps_ = nil;
func_ = nil;
encoder_ =nil;
| resultsDictionary:results | ||
| executionDescriptor:nil]; | ||
|
|
||
| //synchronize(syncType); |
| }; | ||
|
|
||
| // ======================= | ||
| // ETMetalShaderLibrary - ExecuTorch Metal shader library management |
There was a problem hiding this comment.
Add a bit more comments. ETMetalShaderLibrary
- Compiles Metal Shading Language (MSL) source to MTLLibrary
- Caches compiled pipeline states for reuse
- Creates ETMetalKernelFunction lazily
- Lifetime: Persistent (application lifetime)
| }; | ||
|
|
||
| // ======================= | ||
| // ETMetalKernelFunction - ExecuTorch Metal kernel function execution |
There was a problem hiding this comment.
- Encodes kernel dispatches into command buffers
- Lifetime: Per-kernel-invocation
| }; | ||
|
|
||
| // ======================= | ||
| // ETMetalStream - Metal command buffer and synchronization management |
There was a problem hiding this comment.
ETMetalStream
- Manages Metal device, command queue, and command buffers
- Provides synchronization primitives (5 modes)
- Implements kernel coalescing for batch optimization
- Executes MPSGraph operations for AOTI-compiled models
| extern "C" { | ||
| #endif | ||
|
|
||
| // Memory management functions for Metal |
There was a problem hiding this comment.
Docblock about the memory management aspect of this design, such as buffer lifecycle, thread safety etc.
|
|
||
| // C++ only - expose the Metal buffer mapping | ||
| #ifdef __OBJC__ | ||
| extern std::unordered_map<void*, MTLBuffer_t> ptr_to_mtl_buffer; |
There was a problem hiding this comment.
do you need a lock and thread safety to access ptr_to_mtl_buffer?
Comment on lines
+744
to
+745
| void ETMetalStream::copy(id<MTLBuffer> srcBuffer, id<MTLBuffer> dstBuffer, size_t length, | ||
| size_t srcOffset, size_t dstOffset, SyncType syncType) { |
There was a problem hiding this comment.
before copying check that offsets are within the buffer bounds
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This commit introduces the foundational Metal backend runtime.
Key features:
This provides the necessary runtime primitives for executing compute shaders and MPSGraph workloads.