From ff90402faa27bf04779af61607e4f0602a045b9b Mon Sep 17 00:00:00 2001 From: Kurt Mohler Date: Wed, 22 Jun 2022 15:08:08 -0500 Subject: [PATCH 1/6] Create RFC for a logging system and list some required features --- RFC-0026-logging-system.md | 103 +++++++++++++++++++++++++++++++++++++ 1 file changed, 103 insertions(+) create mode 100644 RFC-0026-logging-system.md diff --git a/RFC-0026-logging-system.md b/RFC-0026-logging-system.md new file mode 100644 index 00000000..24b691ae --- /dev/null +++ b/RFC-0026-logging-system.md @@ -0,0 +1,103 @@ +# PyTorch Logging System + +## **Summary** +Create a message logging system for PyTorch with the following requirements: + +* All errors, warnings, and other messages generated by PyTorch should be + emitted using the the logging system API + +* The APIs for emitting messages and changing settings should all be consistent + between C++ and Python + +* Offer different message severity levels, including at least the following: + + - **Info**: Emits a message without creating a warning or error. By default, + this gets printed to stdout + + - **Warning**: Emits a message as a warning. By default, this will turn into + a Python warning + + - **Error**: Emits a message as an error. By default, this will turn into + a Python error + + - TODO: Should we also have a **Fatal** severity for integration with + Meta's internal logging system? A fatal message terminates the program + +* Offer different classes of messages, including at least the following: + + - **Default**: A catch-all message class + + - **Nondeterministic**: Emitted when `torch.use_deterministic_algorithms(True)` + is set and a nondeterministic operation is called + + - **Deprecated**: Emitted when a deprecated function is called + + - **Beta**: Emitted when a beta feature is called. See + [PyTorch feature classifications](https://pytorch.org/blog/pytorch-feature-classification-changes/) + + - **Prototype**: Emitted when a prototype feature is called. See + [PyTorch feature classifications](https://pytorch.org/blog/pytorch-feature-classification-changes/) + +* Continue using warning/error APIs that currently exist in PyTorch wherever + possible. For instance, `TORCH_CHECK`, `TORCH_WARN`, and `TORCH_WARN_ONCE` + should continue to be used in C++ + + - NOTE: These existing APIs don't currently have a concept of message classes, + so that will need to be added + +* Creating new message classes and severity levels should be easy + +* Settings to turn warnings for a specific message class into errors + +* Settings to disable specific message classes and severity levels + + - TODO: However, most errors should not be disableable, right? Perhaps only + some message classes should allow disabling or downgrading errors. For + instance, currently in PyTorch, we can downgrade a nondeterministic error + to a warning, but we wouldn't want to downgrade an error from invalid + arguments given to an operation. + +* Settings to avoid emitting duplicate messages generated by multiple + `torch.distribted` ranks (related to issue + [#68768](https://github.com/pytorch/pytorch/issues/68768)) + +* Ability to make a particular warning only warn once + + - NOTE: Currently `TORCH_WARN_ONCE` does this in C++, but there is no Python + equivalent + + - TODO: Should there be a setting to turn a warn-always into a warn-once for + a given message class and vice versa? + + - TODO: Should warn-once be its own separate severity level? + +* Settings can be changed from Python, C++, or environment variables + + - Filtering warnings with Python command line arguments should + remain possible. For instance, the following turns a `DeprecationWarning` + into an error: `python -W error::DeprecationWarning your_script.py` + +* Should integrate with Meta's internal logging system, which is + [glog](https://github.com/google/glog) + + - TODO: What are all the requirements that define "integrating with glog" + +* Must be OSS-friendly, so it shouldn't require libraries (like glog) which may + cause incompatibility issues for projects that use PyTorch + +* TODO: Determine the requirements for the following concepts: + + - Log files (default behavior and any settings) + + +## **Motivation** +Original issue: [link](https://github.com/pytorch/pytorch/issues/72948) + +Currently, it is challenging for PyTorch developers to provide messages that +act consistently between Python and C++. + +It is also challenging for PyTorch users to manage the messages that PyTorch +emits. For instance, if a PyTorch user happens to be calling PyTorch functions +that emit lots of warnings, it can be difficult for them to filter out those +warnings so that their project's users don't get bombarded with warnings that +they don't need to see. From 72797e2534c39d3c53c6500f8ca01880ec45eeee Mon Sep 17 00:00:00 2001 From: Kurt Mohler Date: Tue, 5 Jul 2022 15:40:10 -0500 Subject: [PATCH 2/6] Update requirements from discussion --- .RFC-0000-template.md.swp | Bin 0 -> 16384 bytes RFC-0026-logging-system.md | 37 +++++++++++++++++++++++++++---------- 2 files changed, 27 insertions(+), 10 deletions(-) create mode 100644 .RFC-0000-template.md.swp diff --git a/.RFC-0000-template.md.swp b/.RFC-0000-template.md.swp new file mode 100644 index 0000000000000000000000000000000000000000..266d93873b0ba2d8bdfd3e822433894bf40e30eb GIT binary patch literal 16384 zcmeI3O^h5z6~{Xc333SnA-F)efCQ090ynruNF0$+6mm@Xy{hW*u49r&Tq4vf z{cZ0|b=75fcVvy=M%1EX4mB9WMwj|%PJi$SA*28Y@R;3d>cQ@wHdQ+ZF=ME?Pu*qbPRM1 zyfX%FNbb1(=xs7~;p4Zcn?G>jojul#>KN!4=osi2=osi2=osi2=osi2_`hJFo*YWP zf+mNeHfQ4BS6uS<8}aAX`1#pOzK_3le{~FW40H^140H^140H^140H^140H^140H^1 z47>{&(A=;E_2pCqq?SHbVW@4z>~jo@!LCdqR*B*`t{uIrQJukTBe zr@%V620VFPlJvmOuT7HU;F)WZY&f?g4j$yTF}b2^$S_~(0)D@Hntw1h&A<;J5H5+&$4YOy1VUOKNMRYn2=4%EGBs z7piagRab?{)SxQIs;oz*QkIt>%-9sQuC0tL2dW-frv^sXO=V6bOX^f<+<_03*|V-z zwup1&wqNPW8h0YBYW60EUyIbJk(Nh|tyEf$$4z1D8J}e*l2g{DjdN;J*|Ji)s?}m2 ziPW$xGo_16^>ubwmZ%IbX&{=a%OJ#r_drUg^VMWwK`kw9_zBjkdT9w(yw*Xp%s4wl zqmkYB+3dW3VVfj!ZsLHqRT!GjPZvL}G^iz2szc($(a=#9=hW z&`7-TsGFG74umoBsk9`XtK}5qBCdQAbh)U6a5yTbp41rA;wFtVS$IIc#A&986BLtB z*yqFXA=guDI7(rd3dgiXnU}*^D;~KeGXpE`Ek4x?vd8f1H#vSGdBrH+zUry__J{@X za6Sl8%F4xz z4D@};P9~T~U@!XChR-hOL1^t8>s_stn}>uXq5HCanmhof(Qpu51zW|p(d1V$Y(dcU z%chop&$I0jZ>p&eK5P=#5y_`|HrG3v??gy;_&loSMp~F+%EUCO!|R zSs7#hTr(c)Y8J!vvjtBV#GxYS=D#FMOKVNd;%-&OEvQG&+d4Oog&pn~@9>E`xZ_E( ztR9jbMdPP!_^ORfxFtNtrW%SvM;H71SHDN(u%$Il<+^Y3b_2vkd48svNd{$^NV}cp zCBd-WOUMLjS^21wQ1%(cI!7Lgwv-re{dVp+VG}6KUd@VTLY$lt<1(|i>?JeFJ{H12 zEyf7gG_K4Wzr`I_56V2}WFF;6W7y{~il65Z?O9Tb7)CO->_C-a#f*sYuQ{6Pa4V>n zrkt?lY4v6S?k9+vzT^%jn3RRjd|uuVpDc(!z@AUQ+d56ngmtzhL9?HFR}dwbfMjj- z*zfZAjX@ZcVoNVlOY{U1+T^M8n4xfNCp(S=HrgCyk~l0xMIRgUue@imB-tXZY@c=7 zT-Fpd*+Mdfsx5y+QU{HkURs-OJ2LLBs6a+XbmSJVDcCO9lzmDbmz)yx2&qDn@d`FD zRMe8RyhpXM)l*v%k$h7axa1TStUZtExH+~`x6efn+s-OIs9RZD!xpgTC8^A`tZ<;< zCI=R7C%9-5^wF@AS|>*&Ic@oq&RDaLwf=SbWE(p&XBG}#?Qmym!cEerlj9PzNs13a z(`Q@BRCHN3OyY<%?wZMjT(KI3S)V6%pK3D9?g?ImU9zOzl_Q_n`F2hZX2%_#FD251 zg|_iFat-#0Zn&}8L*~ylmQ#qF+DzdHWtqiql`Wy{(D5K3ydNoTaVnYOlEmGTP@~x z$Ef?ua>#V2%G4cZbAb&B1!2Q){2C)qmCx5KnIl2T@rkYDGh%(t3}hhjAX!1SCOo#1 z2(NLRv2ev+wqtr~os*XA0zN@sWRK0W>E|}(8@$K%k;I><*h9$ZL;_R$%1M+A9(u0n zkHyJ&(r-&oG^At_k%ZJ`t*}jgTUIBgZiUU_G| zHY$2*gL6eH8E)!iou0r?!L#7IU9|0c**MZlN>)+s2 z@G^J_ya-+Z&jFFU0^Lu?K*vDGK*vDGK*vDGK*zwlmVpQ91UH3Rqfiurs4F7-`f`tCZ*XeCIa z=0NL<-!elhA)}s0xY&MgP1epKmm+bYc6zz-^YYOPuH${6P~y8oQAmnCxnqGJ-^IW# z^{DdQdkLet%U}L}tZ&eUo@%Z8H1(hKgZ}qLg&6pkehV!yzk=6@ixU^y!?E>!p@>3f zG_)z#C9GqMdSBn(KioDMp}E#vPI#5)mwN3zgm3YC4ZKZ+g3zFoSeSbQ!ue}U-$jKl zS_ttTqwPQhy2e}z$z5$_mdk3W3wl+!6dh5yh2-Y4HIH=C!^Oh>`xxIk>`N0OxQxO3 zaVbe(VK7$$CzR-kgL)n1+KKgeVgfICi>%>qR%H|yet5ZDws1`mS=x{;2_02B4w{Qz Wf5}Gkm#Y92WEt Date: Wed, 13 Jul 2022 21:30:29 -0500 Subject: [PATCH 3/6] Add more details and add description of existing messaging APIs --- .RFC-0000-template.md.swp | Bin 16384 -> 0 bytes RFC-0026-logging-system.md | 459 +++++++++++++++++++++++++++++++++---- 2 files changed, 410 insertions(+), 49 deletions(-) delete mode 100644 .RFC-0000-template.md.swp diff --git a/.RFC-0000-template.md.swp b/.RFC-0000-template.md.swp deleted file mode 100644 index 266d93873b0ba2d8bdfd3e822433894bf40e30eb..0000000000000000000000000000000000000000 GIT binary patch literal 0 HcmV?d00001 literal 16384 zcmeI3O^h5z6~{Xc333SnA-F)efCQ090ynruNF0$+6mm@Xy{hW*u49r&Tq4vf z{cZ0|b=75fcVvy=M%1EX4mB9WMwj|%PJi$SA*28Y@R;3d>cQ@wHdQ+ZF=ME?Pu*qbPRM1 zyfX%FNbb1(=xs7~;p4Zcn?G>jojul#>KN!4=osi2=osi2=osi2=osi2_`hJFo*YWP zf+mNeHfQ4BS6uS<8}aAX`1#pOzK_3le{~FW40H^140H^140H^140H^140H^140H^1 z47>{&(A=;E_2pCqq?SHbVW@4z>~jo@!LCdqR*B*`t{uIrQJukTBe zr@%V620VFPlJvmOuT7HU;F)WZY&f?g4j$yTF}b2^$S_~(0)D@Hntw1h&A<;J5H5+&$4YOy1VUOKNMRYn2=4%EGBs z7piagRab?{)SxQIs;oz*QkIt>%-9sQuC0tL2dW-frv^sXO=V6bOX^f<+<_03*|V-z zwup1&wqNPW8h0YBYW60EUyIbJk(Nh|tyEf$$4z1D8J}e*l2g{DjdN;J*|Ji)s?}m2 ziPW$xGo_16^>ubwmZ%IbX&{=a%OJ#r_drUg^VMWwK`kw9_zBjkdT9w(yw*Xp%s4wl zqmkYB+3dW3VVfj!ZsLHqRT!GjPZvL}G^iz2szc($(a=#9=hW z&`7-TsGFG74umoBsk9`XtK}5qBCdQAbh)U6a5yTbp41rA;wFtVS$IIc#A&986BLtB z*yqFXA=guDI7(rd3dgiXnU}*^D;~KeGXpE`Ek4x?vd8f1H#vSGdBrH+zUry__J{@X za6Sl8%F4xz z4D@};P9~T~U@!XChR-hOL1^t8>s_stn}>uXq5HCanmhof(Qpu51zW|p(d1V$Y(dcU z%chop&$I0jZ>p&eK5P=#5y_`|HrG3v??gy;_&loSMp~F+%EUCO!|R zSs7#hTr(c)Y8J!vvjtBV#GxYS=D#FMOKVNd;%-&OEvQG&+d4Oog&pn~@9>E`xZ_E( ztR9jbMdPP!_^ORfxFtNtrW%SvM;H71SHDN(u%$Il<+^Y3b_2vkd48svNd{$^NV}cp zCBd-WOUMLjS^21wQ1%(cI!7Lgwv-re{dVp+VG}6KUd@VTLY$lt<1(|i>?JeFJ{H12 zEyf7gG_K4Wzr`I_56V2}WFF;6W7y{~il65Z?O9Tb7)CO->_C-a#f*sYuQ{6Pa4V>n zrkt?lY4v6S?k9+vzT^%jn3RRjd|uuVpDc(!z@AUQ+d56ngmtzhL9?HFR}dwbfMjj- z*zfZAjX@ZcVoNVlOY{U1+T^M8n4xfNCp(S=HrgCyk~l0xMIRgUue@imB-tXZY@c=7 zT-Fpd*+Mdfsx5y+QU{HkURs-OJ2LLBs6a+XbmSJVDcCO9lzmDbmz)yx2&qDn@d`FD zRMe8RyhpXM)l*v%k$h7axa1TStUZtExH+~`x6efn+s-OIs9RZD!xpgTC8^A`tZ<;< zCI=R7C%9-5^wF@AS|>*&Ic@oq&RDaLwf=SbWE(p&XBG}#?Qmym!cEerlj9PzNs13a z(`Q@BRCHN3OyY<%?wZMjT(KI3S)V6%pK3D9?g?ImU9zOzl_Q_n`F2hZX2%_#FD251 zg|_iFat-#0Zn&}8L*~ylmQ#qF+DzdHWtqiql`Wy{(D5K3ydNoTaVnYOlEmGTP@~x z$Ef?ua>#V2%G4cZbAb&B1!2Q){2C)qmCx5KnIl2T@rkYDGh%(t3}hhjAX!1SCOo#1 z2(NLRv2ev+wqtr~os*XA0zN@sWRK0W>E|}(8@$K%k;I><*h9$ZL;_R$%1M+A9(u0n zkHyJ&(r-&oG^At_k%ZJ`t*}jgTUIBgZiUU_G| zHY$2*gL6eH8E)!iou0r?!L#7IU9|0c**MZlN>)+s2 z@G^J_ya-+Z&jFFU0^Lu?K*vDGK*vDGK*vDGK*zwlmVpQ91UH3Rqfiurs4F7-`f`tCZ*XeCIa z=0NL<-!elhA)}s0xY&MgP1epKmm+bYc6zz-^YYOPuH${6P~y8oQAmnCxnqGJ-^IW# z^{DdQdkLet%U}L}tZ&eUo@%Z8H1(hKgZ}qLg&6pkehV!yzk=6@ixU^y!?E>!p@>3f zG_)z#C9GqMdSBn(KioDMp}E#vPI#5)mwN3zgm3YC4ZKZ+g3zFoSeSbQ!ue}U-$jKl zS_ttTqwPQhy2e}z$z5$_mdk3W3wl+!6dh5yh2-Y4HIH=C!^Oh>`xxIk>`N0OxQxO3 zaVbe(VK7$$CzR-kgL)n1+KKgeVgfICi>%>qR%H|yet5ZDws1`mS=x{;2_02B4w{Qz Wf5}Gkm#Y92WEt Date: Tue, 2 Aug 2022 19:07:49 -0500 Subject: [PATCH 4/6] More updates from discussion --- RFC-0026-logging-system.md | 59 +++++++++++++++++++++----------------- 1 file changed, 32 insertions(+), 27 deletions(-) diff --git a/RFC-0026-logging-system.md b/RFC-0026-logging-system.md index cfd8cd6f..482cca11 100644 --- a/RFC-0026-logging-system.md +++ b/RFC-0026-logging-system.md @@ -24,9 +24,6 @@ Create a message logging system for PyTorch with the following requirements: - **Error**: Emits a message as an error. If an error is never caught, the application will print the error to stderr and quit. - - TODO: Do we also need a **Fatal** severity for integration with Meta's - internal logging system (glog)? A fatal message terminates the program - * Offer different message classes under each severity level. - Every message is emitted as an instance of a message class. @@ -48,35 +45,34 @@ Create a message logging system for PyTorch with the following requirements: we should probably have unit tests for it. See [documentation](https://docs.python.org/3/library/warnings.html#the-warnings-filter) -* Settings to disable specific message classes and severity levels - - - TODO: Error classes should never be disableable, right? +* Settings to disable specific **Warning** or **Info** classes - Disabling warnings in Python is already possible with the `warnings` module filter. See [documentation](https://docs.python.org/3/library/warnings.html#the-warnings-filter). There is no similar system in C++ at the moment, and building one is probably low priority. - - Filtering out **Info** severity messages would also be nice to have, since - excessive printouts can degrade the user experience. Related to - issue [#68768](https://github.com/pytorch/pytorch/issues/68768) + - Filtering out **Info** messages would be nice to have because excessive + printouts can degrade the user experience. Related to issue + [#68768](https://github.com/pytorch/pytorch/issues/68768) * Settings to enable/disable emitting duplicate messages generated by multiple `torch.distributed` ranks. Related to issue [#68768](https://github.com/pytorch/pytorch/issues/68768) -* Ability to make a particular warning only warn once. Warn-once should be the - default for most warnings. +* Ability to make a particular **Warning** or **Info** message only emit once. + Warn-once should be the default for most warnings. - Currently `TORCH_WARN_ONCE` does this in C++, but there is no Python equivalent + - Offer a filter to override warn- and log-once, so that they always emit. + The filter could work similarly to the Python `warnings` filter. This is + a low priority feature. + - TODO: `torch.set_warn_always()` currently controls some warnings (maybe only the ones from C++? I need to find out for sure.) - - TODO: Should there be a setting to turn a warn-once into a warn-always and - vice versa for an entire message class? - * Settings can be changed from Python, C++, or environment variables - Filtering warnings with Python command line arguments should @@ -170,10 +166,14 @@ exists in C++, and it is not implemented as a C++ class that can be inherited * **`c10::BetaWarning`** - Python `torch.BetaWarning` - Emitted when a beta feature is called. See [PyTorch feature classifications](https://pytorch.org/blog/pytorch-feature-classification-changes/). + - TODO: This warning type might not be very useful--find out if we really + want this * **`c10::PrototypeWarning`** - Python `torch.PrototypeWarning` - Emitted when a prototype feature is called. See [PyTorch feature classifications](https://pytorch.org/blog/pytorch-feature-classification-changes/). + - TODO: This warning type might not be very useful--find out if we really + want this * **`c10::NondeterministicWarning`** - Python `torch.NondeterministicWarning` - Emitted when `torch.use_deterministic_algorithms(True)` and @@ -198,9 +198,18 @@ In order to emit messages, developers can use the APIs defined in this section. These APIs all have a variable length argument list, `...` in C++ and `*args` in Python. When a message is emitted, these arguments are concatenated into -a string, and the string becomes the body of the message. In C++, the arguments -must all have the `std::ostream& operator<<` function defined so that they can -be concatenated, and in Python, they must all have a `__str__` function. +a string, and the string becomes the body of the message. + +In C++, the arguments in `...` must all have the `std::ostream& operator<<` +function defined so that they can be concatenated. + +In Python, each element in `*args` must either have a `__str__` function or it +must be a callable that, when called, produces another object that has +a `__str__` fuction. Providing the body of a message as a callable can provide +better performance in cases where the message would not be emitted, as in +`torch.check(True, lambda: expensive_function())` if `cond == True`, since the +`expensive_function()` would not be called in that case. + #### Error APIs @@ -335,13 +344,6 @@ TODO: Should we have a `TOCH_WARN_RANK` (and others) in C++ as well? Is there an existing use case for it? -### Other details - -At the moment in PyTorch, the Python `warnings` module is being publicly -included in `torch` as `torch.warnings`. This should probably be removed or -renamed to `_warnings` to avoid confusion. - - # PyTorch's current messaging API The rest of this document contains details about the current messaging API in @@ -414,6 +416,7 @@ Python error class: | C++ error class | Python error class | | ------------------------------- | -------------------------- | +| `std::exception` | `RuntimeError` | | `c10::Error` | `RuntimeError` | | `c10::IndexError` | `IndexError` | | `c10::ValueError` | `ValueError` | @@ -446,13 +449,15 @@ message using `operator<<`. `c10::Error` and its subclasses are translated into their corresponding Python errors [in `CATCH_CORE_ERRORS`](https://github.com/pytorch/pytorch/blob/72e4aab74b927c1ba5c3963cb17b4c0dce6e56bf/torch/csrc/Exceptions.h#L54-L100). -However, not all of the `c10::Error` subclasses in the table above appear here. -I'm not sure yet what's up with that. +However, not all of the `c10::Error` subclasses in the table above appear here, +which could just be an oversight. `CATCH_CORE_ERRORS` is included within the `END_HANDLE_TH_ERRORS` macro that -every Python-bound C++ function uses for handling errors. For instance, +most Python-bound C++ functions use for handling errors. For instance, `THPVariable__is_view` uses the error handling macro [here](https://github.com/pytorch/pytorch/blob/72e4aab74b927c1ba5c3963cb17b4c0dce6e56bf/tools/autograd/templates/python_variable_methods.cpp#L76). +There is also a similar `END_HANDLE_TH_ERRORS_PYBIND` macro that is used for +pybind-based bindings. #### `torch::PyTorchError` From 4b75803bf90c16b0120787fa0557bfe79ace1ef3 Mon Sep 17 00:00:00 2001 From: Kurt Mohler Date: Thu, 13 Oct 2022 15:12:56 -0500 Subject: [PATCH 5/6] Add c10::DeprecationWarning and fix default warning type to c10::UserWarning --- RFC-0026-logging-system.md | 14 ++++++++++++-- 1 file changed, 12 insertions(+), 2 deletions(-) diff --git a/RFC-0026-logging-system.md b/RFC-0026-logging-system.md index 482cca11..015425e6 100644 --- a/RFC-0026-logging-system.md +++ b/RFC-0026-logging-system.md @@ -247,15 +247,25 @@ corresponding Python signature on the right. #### Warning APIs **`TORCH_WARN(...)`** - `torch.warn(*args)` - - C++ warning: `c10::Warning` + - C++ warning: `c10::UserWarning` - Python warning: `UserWarning` **`TORCH_WARN_ONCE(...)`** - `torch.warn_once(*args)` - - C++ warning: `c10::Warning` + - C++ warning: `c10::UserWarning` - Python warning: `UserWarning` - For a given callsite, the warning is emitted only upon the first time it is called. +**`TORCH_WARN_DEPRECATION(...)`** - `torch.warn_deprecation(*args)` + - C++ warning: `c10::DeprecationWarning` + - Python warning: `UserWarning` + +**`TORCH_WARN_DEPRECATION_ONCE(...)`** - `torch.warn_deprecation_once(*args)` + - C++ warning: `c10::DeprecationWarning` + - Python warning: `DeprecationWarning` + - For a given callsite, the warning is emitted only upon the first time it is + called. + **`TORCH_WARN_WITH(warning_t, ...)`** - `torch.warn_with(warning_type, ...)` - C++ warning: Specified by `warning_t` argument - Python warning: Specified by `warning_type` argument From 1a859d8c13257b192e105c696a2cee8728caab79 Mon Sep 17 00:00:00 2001 From: Kurt Mohler Date: Tue, 15 Nov 2022 18:37:24 -0600 Subject: [PATCH 6/6] Make Python functions private --- RFC-0026-logging-system.md | 41 ++++++++++++++++++++------------------ 1 file changed, 22 insertions(+), 19 deletions(-) diff --git a/RFC-0026-logging-system.md b/RFC-0026-logging-system.md index 015425e6..17c84cb9 100644 --- a/RFC-0026-logging-system.md +++ b/RFC-0026-logging-system.md @@ -207,7 +207,7 @@ In Python, each element in `*args` must either have a `__str__` function or it must be a callable that, when called, produces another object that has a `__str__` fuction. Providing the body of a message as a callable can provide better performance in cases where the message would not be emitted, as in -`torch.check(True, lambda: expensive_function())` if `cond == True`, since the +`torch._check(True, lambda: expensive_function())` if `cond == True`, since the `expensive_function()` would not be called in that case. @@ -219,58 +219,61 @@ in the following signatures, and throw an error if that condition is false. The error APIs are listed below, with the C++ signature on the left and the corresponding Python signature on the right. -**`TORCH_CHECK(cond, ...)`** - `torch.check(cond, *args)` +**`TORCH_CHECK(cond, ...)`** - `torch._check(cond, *args)` - C++ error: `c10::Error` - Python error: `RuntimeError` -**`TORCH_CHECK_INDEX(cond, ...)`** - `torch.check_index(cond, *args)` +**`TORCH_CHECK_INDEX(cond, ...)`** - `torch._check_index(cond, *args)` - C++ error: `c10::IndexError` - Python error: `IndexError` -**`TORCH_CHECK_VALUE(cond, ...)`** - `torch.check_value(cond, *args)` +**`TORCH_CHECK_VALUE(cond, ...)`** - `torch._check_value(cond, *args)` - C++ error: `c10::ValueError` - Python error: `IndexError` -**`TORCH_CHECK_TYPE(cond, ...)`** - `torch.check_type(cond, *args)` +**`TORCH_CHECK_TYPE(cond, ...)`** - `torch._check_type(cond, *args)` - C++ error: `c10::TypeError` - Python error: `TypeError` -**`TORCH_CHECK_NOT_IMPLEMENTED(cond, ...)`** - `torch.check_not_implemented(cond, *args)` +**`TORCH_CHECK_NOT_IMPLEMENTED(cond, ...)`** - `torch._check_not_implemented(cond, *args)` - C++ error: `c10::NotImplementedError` - Python error: `NotImplementedError` -**`TORCH_CHECK_WITH(error_t, cond, ...)`** - `torch.check_with(error_type, cond, *args)` +**`TORCH_CHECK_WITH(error_t, cond, ...)`** - `torch._check_with(error_type, cond, *args)` - C++ error: Specified by `error_t` argument - Python error: Specified by `error_type` argument +Additionally, `cond` for the Python overloads is allowed to be a boolean tensor +with one element. + #### Warning APIs -**`TORCH_WARN(...)`** - `torch.warn(*args)` +**`TORCH_WARN(...)`** - `torch._warn(*args)` - C++ warning: `c10::UserWarning` - Python warning: `UserWarning` -**`TORCH_WARN_ONCE(...)`** - `torch.warn_once(*args)` +**`TORCH_WARN_ONCE(...)`** - `torch._warn_once(*args)` - C++ warning: `c10::UserWarning` - Python warning: `UserWarning` - For a given callsite, the warning is emitted only upon the first time it is called. -**`TORCH_WARN_DEPRECATION(...)`** - `torch.warn_deprecation(*args)` +**`TORCH_WARN_DEPRECATION(...)`** - `torch._warn_deprecation(*args)` - C++ warning: `c10::DeprecationWarning` - Python warning: `UserWarning` -**`TORCH_WARN_DEPRECATION_ONCE(...)`** - `torch.warn_deprecation_once(*args)` +**`TORCH_WARN_DEPRECATION_ONCE(...)`** - `torch._warn_deprecation_once(*args)` - C++ warning: `c10::DeprecationWarning` - Python warning: `DeprecationWarning` - For a given callsite, the warning is emitted only upon the first time it is called. -**`TORCH_WARN_WITH(warning_t, ...)`** - `torch.warn_with(warning_type, ...)` +**`TORCH_WARN_WITH(warning_t, ...)`** - `torch._warn_with(warning_type, ...)` - C++ warning: Specified by `warning_t` argument - Python warning: Specified by `warning_type` argument -**`TORCH_WARN_ONCE_WITH(warning_t, ...)`** - `torch.warn_with(warning_type, ...)` +**`TORCH_WARN_ONCE_WITH(warning_t, ...)`** - `torch._warn_with(warning_type, ...)` - C++ warning: Specified by `warning_t` argument - Python warning: Specified by `warning_type` argument - For a given callsite, the warning is emitted only upon the first time it is @@ -286,7 +289,7 @@ differently--if two warning messages emitted from the same location differ even slightly (for instance, if the value of some variable is included in the message and that value differs between two different `warnings.warn` calls), then both warnings are emitted. `TORCH_WARN_ONCE` does not check whether -messages differ. But we could probably implement `torch.warn_once` in a similar +messages differ. But we could probably implement `torch._warn_once` in a similar way to how the `warnings` module filter is implemented. @@ -296,7 +299,7 @@ Just like the error and warning APIs, the info APIs each have a variable length argument list, `...` in C++ and `*args` in Python. These arguments are concatenated into the info message. -**`TORCH_LOG_INFO(...)`** - `torch.log_info(*args)` +**`TORCH_LOG_INFO(...)`** - `torch._log_info(*args)` - C++ info class: `c10::Info` - Python warning: `torch.Info` - TODO: Is there a better name than `log_info`? I didn't want to call it @@ -306,7 +309,7 @@ concatenated into the info message. [`torch.log`](https://pytorch.org/docs/stable/generated/torch.log.html?highlight=torch%20log#torch.log) is already taken. -**`TORCH_LOG_INFO_WITH(info_t, ...)`** - `torch.log_info_with(info_type, *args)` +**`TORCH_LOG_INFO_WITH(info_t, ...)`** - `torch._log_info_with(info_type, *args)` - C++ info class: Specified by `info_t` argument - Python info class: Specified by `info_type` argument @@ -336,7 +339,7 @@ However, implementing the duplicate filter like this is not ideal. It would be better to have dedicated message system API calls for this. In the case of warnings, the following signature could be used: -**`torch.warn_rank(my_rank, *args, warn_rank=0)`** +**`torch._warn_rank(my_rank, *args, warn_rank=0)`** * Args: - `my_rank` - Rank of the subprocess calling this function - `args` - Warning message @@ -344,10 +347,10 @@ warnings, the following signature could be used: * The warning is only emitted if `my_rank == warn_rank` TODO: Add APIs for the rest of the message classes, like -`torch.log_info_rank()`, etc. +`torch._log_info_rank()`, etc. TODO: There should also be a global setting to enable emitting the duplicates. -`torch.warn_rank` could check the setting, and if it's turned on, then it would +`torch._warn_rank` could check the setting, and if it's turned on, then it would emit the warning for all ranks. TODO: Should we have a `TOCH_WARN_RANK` (and others) in C++ as well? Is there