Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

🚚 port ma.arg{min,max} and MaskedArray.arg{min,max} #468

Open
wants to merge 2 commits into
base: main
Choose a base branch
from

Conversation

@guan404ming guan404ming changed the title 🚚 port ma.arg{min,max} and MaskedArray.arg{min,max} 🚚 port ma.arg{min,max} and MaskedArray.arg{min,max} Apr 4, 2025
@guan404ming guan404ming marked this pull request as draft April 4, 2025 17:16
@guan404ming guan404ming changed the title 🚚 port ma.arg{min,max} and MaskedArray.arg{min,max} [WIP] 🚚 port ma.arg{min,max} and MaskedArray.arg{min,max} Apr 4, 2025
@guan404ming guan404ming force-pushed the port-ma-argmax-argmin branch 6 times, most recently from 98a5905 to 54319e0 Compare April 7, 2025 04:28
@guan404ming guan404ming changed the title [WIP] 🚚 port ma.arg{min,max} and MaskedArray.arg{min,max} 🚚 port ma.arg{min,max} and MaskedArray.arg{min,max} Apr 7, 2025
@guan404ming guan404ming added numpy.ma.* port: from numpy NumPy PR's that should be ported to NumType labels Apr 7, 2025
@guan404ming
Copy link
Member Author

guan404ming commented Apr 7, 2025

Hi @jorenham I've completed the migration, but there are still issues with the CI that I can't resolve. I was hoping you could help. It looks like a runtime type error. Thanks!

Stub: in file src/numpy-stubs/ma/__init__.pyi:1176
Overload(def [_ArrayT <: numpy.ndarray[Any, Any]] (a: _ArrayT`-1, axis: None =, fill_value: Union[Union[complex, bytes, str, numpy.generic[Any]], None] =, out: None =, *, keepdims: Union[Literal[False], numpy._globals._NoValueType] =) -> numpy.int64, def [_ArrayT <: numpy.ndarray[Any, Any]] (a: _ArrayT`-1, axis: Union[typing.SupportsIndex, None] =, fill_value: Union[Union[complex, bytes, str, numpy.generic[Any]], None] =, out: None =, *, keepdims: Union[bool, numpy._globals._NoValueType] =) -> Any, def [_ArrayT <: numpy.ndarray[Any, Any]] (a: _ArrayT`-1, axis: Union[typing.SupportsIndex, None] =, fill_value: Union[Union[complex, bytes, str, numpy.generic[Any]], None] =, *, out: _ArrayT`-1, keepdims: Union[bool, numpy._globals._NoValueType] =) -> _ArrayT`-1, def [_ArrayT <: numpy.ndarray[Any, Any]] (a: _ArrayT`-1, axis: Union[typing.SupportsIndex, None], fill_value: Union[Union[complex, bytes, str, numpy.generic[Any]], None], out: _ArrayT`-1, *, keepdims: Union[bool, numpy._globals._NoValueType] =) -> _ArrayT`-1)
Runtime:
def (a, *args, **params)

error: numpy.ma.argmin is not a function
Stub: in file src/numpy-stubs/ma/__init__.pyi:1138
Overload(def [_ArrayT <: numpy.ndarray[Any, Any]] (a: _ArrayT`-1, axis: None =, fill_value: Union[Union[complex, bytes, str, numpy.generic[Any]], None] =, out: None =, *, keepdims: Union[Literal[False], numpy._globals._NoValueType] =) -> numpy.int64, def [_ArrayT <: numpy.ndarray[Any, Any]] (a: _ArrayT`-1, axis: Union[typing.SupportsIndex, None] =, fill_value: Union[Union[complex, bytes, str, numpy.generic[Any]], None] =, out: None =, *, keepdims: Union[bool, numpy._globals._NoValueType] =) -> Any, def [_ArrayT <: numpy.ndarray[Any, Any]] (a: _ArrayT`-1, axis: Union[typing.SupportsIndex, None] =, fill_value: Union[Union[complex, bytes, str, numpy.generic[Any]], None] =, *, out: _ArrayT`-1, keepdims: Union[bool, numpy._globals._NoValueType] =) -> _ArrayT`-1, def [_ArrayT <: numpy.ndarray[Any, Any]] (a: _ArrayT`-1, axis: Union[typing.SupportsIndex, None], fill_value: Union[Union[complex, bytes, str, numpy.generic[Any]], None], out: _ArrayT`-1, *, keepdims: Union[bool, numpy._globals._NoValueType] =) -> _ArrayT`-1)
Runtime:
def (a, *args, **params)

error: numpy.ma.core.argmax is not a function
Stub: in file src/numpy-stubs/ma/core.pyi:1176
Overload(def [_ArrayT <: numpy.ndarray[Any, Any]] (a: _ArrayT`-1, axis: None =, fill_value: Union[Union[complex, bytes, str, numpy.generic[Any]], None] =, out: None =, *, keepdims: Union[Literal[False], numpy._globals._NoValueType] =) -> numpy.int64, def [_ArrayT <: numpy.ndarray[Any, Any]] (a: _ArrayT`-1, axis: Union[typing.SupportsIndex, None] =, fill_value: Union[Union[complex, bytes, str, numpy.generic[Any]], None] =, out: None =, *, keepdims: Union[bool, numpy._globals._NoValueType] =) -> Any, def [_ArrayT <: numpy.ndarray[Any, Any]] (a: _ArrayT`-1, axis: Union[typing.SupportsIndex, None] =, fill_value: Union[Union[complex, bytes, str, numpy.generic[Any]], None] =, *, out: _ArrayT`-1, keepdims: Union[bool, numpy._globals._NoValueType] =) -> _ArrayT`-1, def [_ArrayT <: numpy.ndarray[Any, Any]] (a: _ArrayT`-1, axis: Union[typing.SupportsIndex, None], fill_value: Union[Union[complex, bytes, str, numpy.generic[Any]], None], out: _ArrayT`-1, *, keepdims: Union[bool, numpy._globals._NoValueType] =) -> _ArrayT`-1)
Runtime:
def (a, *args, **params)

error: numpy.ma.core.argmin is not a function
Stub: in file src/numpy-stubs/ma/core.pyi:1138
Overload(def [_ArrayT <: numpy.ndarray[Any, Any]] (a: _ArrayT`-1, axis: None =, fill_value: Union[Union[complex, bytes, str, numpy.generic[Any]], None] =, out: None =, *, keepdims: Union[Literal[False], numpy._globals._NoValueType] =) -> numpy.int64, def [_ArrayT <: numpy.ndarray[Any, Any]] (a: _ArrayT`-1, axis: Union[typing.SupportsIndex, None] =, fill_value: Union[Union[complex, bytes, str, numpy.generic[Any]], None] =, out: None =, *, keepdims: Union[bool, numpy._globals._NoValueType] =) -> Any, def [_ArrayT <: numpy.ndarray[Any, Any]] (a: _ArrayT`-1, axis: Union[typing.SupportsIndex, None] =, fill_value: Union[Union[complex, bytes, str, numpy.generic[Any]], None] =, *, out: _ArrayT`-1, keepdims: Union[bool, numpy._globals._NoValueType] =) -> _ArrayT`-1, def [_ArrayT <: numpy.ndarray[Any, Any]] (a: _ArrayT`-1, axis: Union[typing.SupportsIndex, None], fill_value: Union[Union[complex, bytes, str, numpy.generic[Any]], None], out: _ArrayT`-1, *, keepdims: Union[bool, numpy._globals._NoValueType] =) -> _ArrayT`-1)
Runtime:
def (a, *args, **params)

Found 4 errors (checked 325 modules)
deleted /home/runner/work/numtype/numtype/.venv/lib/python3.13/site-packages/numpy/py.typed (1)
deleted /home/runner/work/numtype/numtype/.venv/lib/python3.13/site-packages/numpy/**/*.pyi ([23](https://github.com/numpy/numtype/actions/runs/14300819613/job/40074866251?pr=468#step:4:24)7)

@guan404ming guan404ming marked this pull request as ready for review April 7, 2025 04:39
@guan404ming guan404ming self-assigned this Apr 7, 2025
@jorenham
Copy link
Member

jorenham commented Apr 7, 2025

Hi @jorenham I've completed the migration, but there are still issues with the CI that I can't resolve. I was hoping you could help. It looks like a runtime type error. Thanks!

Stub: in file src/numpy-stubs/ma/__init__.pyi:1176
Overload(def [_ArrayT <: numpy.ndarray[Any, Any]] (a: _ArrayT`-1, axis: None =, fill_value: Union[Union[complex, bytes, str, numpy.generic[Any]], None] =, out: None =, *, keepdims: Union[Literal[False], numpy._globals._NoValueType] =) -> numpy.int64, def [_ArrayT <: numpy.ndarray[Any, Any]] (a: _ArrayT`-1, axis: Union[typing.SupportsIndex, None] =, fill_value: Union[Union[complex, bytes, str, numpy.generic[Any]], None] =, out: None =, *, keepdims: Union[bool, numpy._globals._NoValueType] =) -> Any, def [_ArrayT <: numpy.ndarray[Any, Any]] (a: _ArrayT`-1, axis: Union[typing.SupportsIndex, None] =, fill_value: Union[Union[complex, bytes, str, numpy.generic[Any]], None] =, *, out: _ArrayT`-1, keepdims: Union[bool, numpy._globals._NoValueType] =) -> _ArrayT`-1, def [_ArrayT <: numpy.ndarray[Any, Any]] (a: _ArrayT`-1, axis: Union[typing.SupportsIndex, None], fill_value: Union[Union[complex, bytes, str, numpy.generic[Any]], None], out: _ArrayT`-1, *, keepdims: Union[bool, numpy._globals._NoValueType] =) -> _ArrayT`-1)
Runtime:
def (a, *args, **params)

error: numpy.ma.argmin is not a function
Stub: in file src/numpy-stubs/ma/__init__.pyi:1138
Overload(def [_ArrayT <: numpy.ndarray[Any, Any]] (a: _ArrayT`-1, axis: None =, fill_value: Union[Union[complex, bytes, str, numpy.generic[Any]], None] =, out: None =, *, keepdims: Union[Literal[False], numpy._globals._NoValueType] =) -> numpy.int64, def [_ArrayT <: numpy.ndarray[Any, Any]] (a: _ArrayT`-1, axis: Union[typing.SupportsIndex, None] =, fill_value: Union[Union[complex, bytes, str, numpy.generic[Any]], None] =, out: None =, *, keepdims: Union[bool, numpy._globals._NoValueType] =) -> Any, def [_ArrayT <: numpy.ndarray[Any, Any]] (a: _ArrayT`-1, axis: Union[typing.SupportsIndex, None] =, fill_value: Union[Union[complex, bytes, str, numpy.generic[Any]], None] =, *, out: _ArrayT`-1, keepdims: Union[bool, numpy._globals._NoValueType] =) -> _ArrayT`-1, def [_ArrayT <: numpy.ndarray[Any, Any]] (a: _ArrayT`-1, axis: Union[typing.SupportsIndex, None], fill_value: Union[Union[complex, bytes, str, numpy.generic[Any]], None], out: _ArrayT`-1, *, keepdims: Union[bool, numpy._globals._NoValueType] =) -> _ArrayT`-1)
Runtime:
def (a, *args, **params)

error: numpy.ma.core.argmax is not a function
Stub: in file src/numpy-stubs/ma/core.pyi:1176
Overload(def [_ArrayT <: numpy.ndarray[Any, Any]] (a: _ArrayT`-1, axis: None =, fill_value: Union[Union[complex, bytes, str, numpy.generic[Any]], None] =, out: None =, *, keepdims: Union[Literal[False], numpy._globals._NoValueType] =) -> numpy.int64, def [_ArrayT <: numpy.ndarray[Any, Any]] (a: _ArrayT`-1, axis: Union[typing.SupportsIndex, None] =, fill_value: Union[Union[complex, bytes, str, numpy.generic[Any]], None] =, out: None =, *, keepdims: Union[bool, numpy._globals._NoValueType] =) -> Any, def [_ArrayT <: numpy.ndarray[Any, Any]] (a: _ArrayT`-1, axis: Union[typing.SupportsIndex, None] =, fill_value: Union[Union[complex, bytes, str, numpy.generic[Any]], None] =, *, out: _ArrayT`-1, keepdims: Union[bool, numpy._globals._NoValueType] =) -> _ArrayT`-1, def [_ArrayT <: numpy.ndarray[Any, Any]] (a: _ArrayT`-1, axis: Union[typing.SupportsIndex, None], fill_value: Union[Union[complex, bytes, str, numpy.generic[Any]], None], out: _ArrayT`-1, *, keepdims: Union[bool, numpy._globals._NoValueType] =) -> _ArrayT`-1)
Runtime:
def (a, *args, **params)

error: numpy.ma.core.argmin is not a function
Stub: in file src/numpy-stubs/ma/core.pyi:1138
Overload(def [_ArrayT <: numpy.ndarray[Any, Any]] (a: _ArrayT`-1, axis: None =, fill_value: Union[Union[complex, bytes, str, numpy.generic[Any]], None] =, out: None =, *, keepdims: Union[Literal[False], numpy._globals._NoValueType] =) -> numpy.int64, def [_ArrayT <: numpy.ndarray[Any, Any]] (a: _ArrayT`-1, axis: Union[typing.SupportsIndex, None] =, fill_value: Union[Union[complex, bytes, str, numpy.generic[Any]], None] =, out: None =, *, keepdims: Union[bool, numpy._globals._NoValueType] =) -> Any, def [_ArrayT <: numpy.ndarray[Any, Any]] (a: _ArrayT`-1, axis: Union[typing.SupportsIndex, None] =, fill_value: Union[Union[complex, bytes, str, numpy.generic[Any]], None] =, *, out: _ArrayT`-1, keepdims: Union[bool, numpy._globals._NoValueType] =) -> _ArrayT`-1, def [_ArrayT <: numpy.ndarray[Any, Any]] (a: _ArrayT`-1, axis: Union[typing.SupportsIndex, None], fill_value: Union[Union[complex, bytes, str, numpy.generic[Any]], None], out: _ArrayT`-1, *, keepdims: Union[bool, numpy._globals._NoValueType] =) -> _ArrayT`-1)
Runtime:
def (a, *args, **params)

Found 4 errors (checked 325 modules)
deleted /home/runner/work/numtype/numtype/.venv/lib/python3.13/site-packages/numpy/py.typed (1)
deleted /home/runner/work/numtype/numtype/.venv/lib/python3.13/site-packages/numpy/**/*.pyi ([23](https://github.com/numpy/numtype/actions/runs/14300819613/job/40074866251?pr=468#step:4:24)7)

Ah yes I was afraid that stubtest wouldn't be able to understand these. and I unfortunately was right about that...

If you look at the numpy source, you'll see the problem (
https://github.com/numpy/numpy/blob/69f4df3b67002f537d816468d8b5c29f06d5f8e3/numpy/ma/core.py#L7236-L7237):

argmin = _frommethod('argmin')
argmax = _frommethod('argmax')

Confusingly, _frommethod is a type (https://github.com/numpy/numpy/blob/69f4df3b67002f537d816468d8b5c29f06d5f8e3/numpy/ma/core.py#L7078-L7117):

class _frommethod:
    """
    Define functions from existing MaskedArray methods.

    Parameters
    ----------
    methodname : str
        Name of the method to transform.

    """

    def __init__(self, methodname, reversed=False):
        self.__name__ = methodname
        self.__qualname__ = methodname
        self.__doc__ = self.getdoc()
        self.reversed = reversed

    def getdoc(self):
        "Return the doc of the function (from the doc of the method)."
        meth = getattr(MaskedArray, self.__name__, None) or\
            getattr(np, self.__name__, None)
        signature = self.__name__ + get_object_signature(meth)
        if meth is not None:
            doc = """    %s\n%s""" % (
                signature, getattr(meth, '__doc__', None))
            return doc

    def __call__(self, a, *args, **params):
        if self.reversed:
            args = list(args)
            a, args[0] = args[0], a

        marr = asanyarray(a)
        method_name = self.__name__
        method = getattr(type(marr), method_name, None)
        if method is None:
            # use the corresponding np function
            method = getattr(np, method_name)

        return method(marr, *args, **params)

It is a similar situation as with ufunc, but in this case we only have __call__ to deal with. Given your experience with ufuncs, you're probably able to figure out the solution now. If not, then don't hesitate to ask for another hint.


You could also consider splitting this PR, if you feel that that would help.

@jorenham
Copy link
Member

jorenham commented Apr 7, 2025

Could you also reference the original numpy PR's here, and co-author @MarcoGorelli in 54319e0 (unless he objects)?

@guan404ming
Copy link
Member Author

guan404ming commented Apr 7, 2025

Thanks for hint, it currently works well now!

@guan404ming guan404ming force-pushed the port-ma-argmax-argmin branch from c3b35df to ce5de3b Compare April 7, 2025 15:51
@guan404ming guan404ming requested a review from jorenham April 7, 2025 15:59
Copy link
Member

@jorenham jorenham left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The _frommethod type is not only used for argmin and argmax, but also for count that have a different callable signature. So I was thinking of a more generic approach where by default, the callable signature has a broad "one-size-fits-all" signature, but that can be customized with a more specific callable signature, e.g. with Callable or a callable Protocol.

@jorenham

This comment was marked as resolved.

@guan404ming guan404ming force-pushed the port-ma-argmax-argmin branch from ce5de3b to db60ac4 Compare April 8, 2025 04:14
@guan404ming guan404ming force-pushed the port-ma-argmax-argmin branch 3 times, most recently from 86554bb to 68ae1b9 Compare April 8, 2025 04:59
@guan404ming guan404ming force-pushed the port-ma-argmax-argmin branch from 68ae1b9 to 4909fe6 Compare April 8, 2025 05:19
@guan404ming
Copy link
Member Author

This week is my midterm week haha, I would try fix this at this weekend~

@jorenham
Copy link
Member

jorenham commented Apr 8, 2025

This week is my midterm week haha, I would try fix this at this weekend~

There's no rush :) Good luck with your midterm!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
numpy.ma.* port: from numpy NumPy PR's that should be ported to NumType
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants