Keep torch.Tensor outputs for calls without tensor inputs

The announced rule said that a call without tensor inputs, such as
get_attribute(), a bundled program check, or a method that takes no
inputs, would later return torch.Tensor only if torch was already
imported. So its type would depend on whether unrelated code imported
torch first, and the warning told users to rely on that import order.

Such calls now keep returning torch.Tensor outputs whenever torch can be
imported, now and in the future release, which is what they do today.
They no longer warn, since their outputs will not change. Calls whose
tensor inputs are not torch.Tensor still warn, as before. The warning,
the README, the API reference and the Method.execute docstring describe
the new rule.

Author: shoumikhin

docs/source/runtime-python-api-reference.rst

@@ -48,7 +48,7 @@ Output types
 
 Importing ``executorch.runtime`` does not import PyTorch, unless the setting below asks for ``torch``. When PyTorch is installed, ``Method.execute`` returns ``torch.Tensor`` outputs, and the first call that returns one imports ``torch``. Without PyTorch, or when importing it raises an ``ImportError``, it returns ``ExecuTorchResult`` outputs. Any other error from importing PyTorch is raised by every call that returns a tensor; set the setting below to ``executorch`` to run without PyTorch.
 
-A future release will make the outputs follow the tensor inputs of each call. A call with any ``torch.Tensor`` input will return ``torch.Tensor`` outputs. A call whose tensor inputs are all NumPy arrays, other buffers, DLPack objects, or ``ExecuTorchResult`` values will return ``ExecuTorchResult`` outputs, even when PyTorch is installed. A call with no tensor inputs will return ``torch.Tensor`` outputs only if ``torch`` is already imported. Each call whose outputs will change warns with a ``FutureWarning``.
+A future release will make the outputs follow the tensor inputs of each call. A call with any ``torch.Tensor`` input will return ``torch.Tensor`` outputs. A call whose tensor inputs are all NumPy arrays, other buffers, DLPack objects, or ``ExecuTorchResult`` values will return ``ExecuTorchResult`` outputs, even when PyTorch is installed. A call with no tensor inputs keeps returning ``torch.Tensor`` outputs whenever ``torch`` can be imported. Each call whose outputs will change warns with a ``FutureWarning``.
 
 To choose the output type now, set ``EXECUTORCH_PYBINDINGS_TENSOR_OUTPUT`` before importing ExecuTorch. With ``executorch``, every call returns ``ExecuTorchResult`` outputs and ExecuTorch never imports PyTorch. With ``torch``, every call returns ``torch.Tensor`` outputs, and importing ExecuTorch imports PyTorch.
 

extension/pybindings/README.md

@@ -100,7 +100,7 @@ call, including inputs kept from an earlier call. A call with any torch tensor
 input will return torch tensors. A call whose tensor inputs are all NumPy
 arrays, other buffers, DLPack objects, or `ExecuTorchResult` values will return
 `ExecuTorchResult` values, even when PyTorch is installed. A call with no tensor
-inputs will return torch tensors only if `torch` is already imported. Each call
+inputs keeps returning torch tensors whenever `torch` can be imported. Each call
 whose outputs will change warns with a `FutureWarning`. To choose the output
 type now, set `EXECUTORCH_PYBINDINGS_TENSOR_OUTPUT` to `executorch` or `torch`
 before importing the bindings. The setting is fixed when the extension loads,

extension/pybindings/pybindings.cpp

@@ -737,12 +737,11 @@ constexpr char kTensorOutputEnvironmentVariable[] =
     "EXECUTORCH_PYBINDINGS_TENSOR_OUTPUT";
 
 constexpr char kTensorOutputWarning[] =
-    "ExecuTorch returns torch.Tensor outputs for a call without torch.Tensor "
-    "inputs. A future release will return ExecuTorchResult outputs for such a "
-    "call, unless it has no tensor inputs and torch is already imported. To "
-    "keep torch.Tensor outputs, pass torch.Tensor inputs, or import torch "
-    "before a call without tensor inputs. To choose the type for every call, "
-    "set EXECUTORCH_PYBINDINGS_TENSOR_OUTPUT to torch or executorch before "
+    "ExecuTorch returns torch.Tensor outputs for a call whose tensor inputs "
+    "are not torch.Tensor. A future release will return ExecuTorchResult "
+    "outputs for such a call. To keep torch.Tensor outputs, pass torch.Tensor "
+    "inputs. To choose the type for every call, set "
+    "EXECUTORCH_PYBINDINGS_TENSOR_OUTPUT to torch or executorch before "
     "importing ExecuTorch. With executorch, PyTorch is not loaded.";
 
 // Set from EXECUTORCH_PYBINDINGS_TENSOR_OUTPUT when the extension loads.
@@ -785,10 +784,8 @@ int caller_stacklevel() {
   return level;
 }
 
-// Until a later release makes outputs follow inputs, a call that would then
-// return ExecuTorchResult outputs keeps returning torch.Tensor outputs
-// whenever torch can be imported, as before, and warns.
-bool keep_torch_outputs() {
+// Whether torch is imported, importing it if it can be.
+bool torch_available() {
   // Read and written with the GIL held. An import inside a function-local
   // static's initializer could deadlock, since the import lets other threads
   // run and one of them could then wait for that static while holding the GIL.
@@ -820,6 +817,16 @@ bool keep_torch_outputs() {
       return false;
     }
   }
+  return true;
+}
+
+// Until a later release makes outputs follow inputs, a call that would then
+// return ExecuTorchResult outputs keeps returning torch.Tensor outputs
+// whenever torch can be imported, as before, and warns.
+bool keep_torch_outputs() {
+  if (!torch_available()) {
+    return false;
+  }
   // Every such call warns, so Python's warning filters decide what shows. A
   // warning hidden by a filter at one call still shows at a later one.
   if (PyErr_WarnEx(
@@ -834,10 +841,11 @@ bool keep_torch_outputs() {
 
 // Picks the type of a call's tensor outputs. A later release will follow the
 // call's tensor inputs, including inputs kept from earlier calls: torch.Tensor
-// if any of them is one, ExecuTorchResult if all are other tensors, and with
-// no tensor inputs, torch.Tensor only if torch is imported. Until then,
-// keep_torch_outputs() keeps torch.Tensor where that rule would change it. A
-// call that returns no tensor never imports torch.
+// if any of them is one, and ExecuTorchResult if all are other tensors. Until
+// then, keep_torch_outputs() keeps torch.Tensor where that rule would change
+// it. A call with no tensor inputs returns torch.Tensor whenever torch can be
+// imported, now and later, so its type never depends on what imported torch
+// first. A call that returns no tensor never imports torch.
 PortableTensorOutput portable_tensor_output(
     bool returns_tensor,
     bool torch_input,
@@ -849,9 +857,13 @@ PortableTensorOutput portable_tensor_output(
       (!other_tensor_input && loaded_torch_module() != nullptr)) {
     return PortableTensorOutput::Torch;
   }
-  return returns_tensor && keep_torch_outputs()
-      ? PortableTensorOutput::Torch
-      : PortableTensorOutput::ExecuTorch;
+  if (!returns_tensor) {
+    return PortableTensorOutput::ExecuTorch;
+  }
+  const bool torch_output =
+      other_tensor_input ? keep_torch_outputs() : torch_available();
+  return torch_output ? PortableTensorOutput::Torch
+                      : PortableTensorOutput::ExecuTorch;
 }
 
 // From the declared outputs, so the type is picked before the method runs.

extension/pybindings/test/test_pybindings.py

@@ -485,17 +485,21 @@ def run(index):
                 """,
             ),
         }
-        # Each of these calls returns torch.Tensor outputs now and will return
-        # ExecuTorchResult outputs later, so it warns: calls without tensor
-        # inputs before torch is imported, and calls with tensor inputs of
-        # other kinds after it is.
-        for call, torch_imported in (
-            ("module.run_method('tensor_constant')", False),
-            ("[method.get_attribute('state')]", False),
-            ("bundled.verify_result_with_bundled_expected_output('forward', 0)", False),
-            ("module.forward([x, x])", True),
-            ("module.forward(capsules)", True),
-            ("method(capsules)", True),
+        # Calls with tensor inputs of other kinds return torch.Tensor outputs
+        # now and will return ExecuTorchResult outputs later, so they warn,
+        # even after torch is imported. Calls without tensor inputs keep
+        # torch.Tensor outputs, so they do not, even before it is.
+        for call, torch_imported, warnings_expected in (
+            ("module.run_method('tensor_constant')", False, 0),
+            ("[method.get_attribute('state')]", False, 0),
+            (
+                "bundled.verify_result_with_bundled_expected_output('forward', 0)",
+                False,
+                0,
+            ),
+            ("module.forward([x, x])", True, 1),
+            ("module.forward(capsules)", True, 1),
+            ("method(capsules)", True, 1),
         ):
             scenarios[f"{call}, torch imported: {torch_imported}"] = (
                 None,
@@ -508,8 +512,8 @@ def run(index):
                 with open(bundled_path, "rb") as source:
                     bundled = portable_lib._load_bundled_program_from_buffer(source.read())
                 capsules = [x.__dlpack__(), x.__dlpack__()]
-                (warning,), line = check(lambda: {call}, ["torch"], 1), sys._getframe().f_lineno
-                assert warning.lineno == line, (warning.lineno, line)
+                caught, line = check(lambda: {call}, ["torch"], {warnings_expected}), sys._getframe().f_lineno
+                assert all(warning.lineno == line for warning in caught), caught
                 """,
             )
         with tempfile.TemporaryDirectory() as directory:

runtime/__init__.py

@@ -179,11 +179,11 @@ def execute(
         .. warning::
 
             A future release will make tensor outputs follow the tensor inputs:
-            torch.Tensor objects if any input is one, ExecuTorchResult objects
-            if all are other tensors, even when PyTorch is installed, and with
-            no tensor inputs, torch.Tensor objects only if torch is already
-            imported. Each call whose outputs will change warns with a
-            FutureWarning. Set ``EXECUTORCH_PYBINDINGS_TENSOR_OUTPUT`` to
+            torch.Tensor objects if any input is one, and ExecuTorchResult
+            objects if all are other tensors, even when PyTorch is installed.
+            With no tensor inputs, outputs stay torch.Tensor objects whenever
+            torch can be imported. Each call whose outputs will change warns
+            with a FutureWarning. Set ``EXECUTORCH_PYBINDINGS_TENSOR_OUTPUT`` to
             ``torch`` or ``executorch`` before importing ExecuTorch to choose
             the type now.
         """