Replacing vectorize brakes code

[bencebecsy]

I used uniform from a previous release, and it worked fine. Since the latest update that replaced vectorize with jit my code stopped working. Here is a minimal working example:

import numpy as np
import numba as nb
from numba import njit

from numba_stats import uniform as uniform_numba

@njit()
def get_lnprior_helper():
    """jittable helper for calculating the log prior"""
    prior = 1.0

    #loop through uniform parameters
    for i in range(3):
       prior *= uniform_numba.pdf(0.5, 0.0, 1.0)

    return prior

print(get_lnprior_helper())

This now gives the following error:

No implementation of function Function(<function pdf at 0x114f8bdc0>) found for signature:
 
 >>> pdf(float64, float64, float64)

This seems to only happen if it's within a jitted function. If I remove @njit() from above, it works fine. I'm at numba 0.53.0.

Any idea why this might be happening?

[HDembinski]

Hi @bencebecsy, this looks like a bug, I am investigating.

[HDembinski]

I will try to restore that functionality, but using numba_stats in this way is not going to give you good performance. numba_stats is optimised for vectorised operation on arrays. Internally, it is now using arrays everywhere. So if you feed it a single number, it is converted into an array and then the implementation operates on that array. It is possible that the numba optimizer recognizes this special case and reduces the code again to a calculation on numbers, but I am not sure.

In any case, to reap the performance benefits of numba_stats, you should process chunks of number at once and write your code in this way.

[bencebecsy]

Hi @HDembinski, thanks for looking into this. Thanks for the info on the best way to use numba_stats. To be honest, I got decent performance this way, and the main reason I used uniform_numba.pdf instead of uniform.pdf from scipy.stats is not the performance but the fact that I couldn't get that working in a jitted function. If I do from scipy.stats import uniform in the above example, and replace uniform_numba.pdf(0.5, 0.0, 1.0) with uniform.pdf(0.5, 0.0, 1.0), I get the following error:

Untyped global name 'uniform': Cannot determine Numba type of <class 'scipy.stats._continuous_distns.uniform_gen'>

Do you know if there's any way around this other than using numba_stats?

[HDembinski]

No there is no other way, that's the raison d'etre for numba_stats. I will try to restore this functionality, but if you care about performance, you should organise your code to use arrays of values as the smallest unit instead of individual values. Otherwise you cannot use the powerful SIMD instructions of modern CPUs.