[ports] Unpack ports atomically to improve concurrency robustness (#27161)

When building ports in parallel, different processes can race on
checking
the port's up-to-date status. Currently, we unpack directly into the
target
directory and then write the marker file. This creates a window of time
where the directory exists but is incomplete and lacks the marker.

If another process checks the port status during this window, it will
conclude the port is invalid/incomplete, and aggressively delete the
directory and its cached builds to re-unpack it. This can disrupt other
processes currently compiling from that directory or lead to
EM_CACHE_IS_LOCKED errors when child processes try to rebuild deleted
libraries.

By unpacking to a temporary directory and atomically renaming it to the
final destination, we ensure that the target directory either does not
exist at all or is 100% complete with the marker. This eliminates the
partially unpacked state and leverages the atomicity of directory
rename (which is highly coherent in OS VFS layers) to reduce the race
window to zero.

This should help resolve intermittent bot failures on macOS and Windows,
where VFS metadata latency can cause processes to see inconsistent
directory
states.

Hopefully this will address #27160, although time will tell.

Author: sbc100

tools/ports/__init__.py

@@ -373,9 +373,22 @@ def retrieve():
 
     def unpack():
       logger.info(f'unpacking port: {name}')
-      utils.safe_ensure_dirs(fullname)
-      shutil.unpack_archive(filename=fullpath, extract_dir=fullname)
-      utils.write_file(marker, url + '\n')
+      unpack_dir = fullname + '.tmp'
+      # We unpack to a temporary directory and then atomically rename it to the
+      # final destination. This ensures that the destination directory either
+      # does not exist or is 100% complete, avoiding races where other processes
+      # might see a partially unpacked directory (lacking the marker) and
+      # incorrectly assume it is invalid or needs to be cleared.
+      utils.delete_dir(unpack_dir)
+      utils.safe_ensure_dirs(unpack_dir)
+
+      shutil.unpack_archive(filename=fullpath, extract_dir=unpack_dir)
+      tmp_marker = os.path.join(unpack_dir, '.emscripten_url')
+      utils.write_file(tmp_marker, url + '\n')
+
+      # Atomically replace the target directory
+      utils.delete_dir(fullname)
+      os.replace(unpack_dir, fullname)
 
     def up_to_date():
       return os.path.exists(marker) and utils.read_file(marker).strip() == url