[BUG]: Freezing/Getting Stuck on Python 3.7/PySR 0.18.3

[amirhszd]

What happened?

Hi there, I have not been able to find a similar issue to this. It is probably something silly. But I am running on Python 3.7, on larger number of iterations and bigger datasets (2000 samples with 3 features), the progress print gets stuck at "compiling Julia backend" or in the middle of the optimization. I am using the very basic functionality code in the readme file, so nothing crazy. Any ideas?

Thanks,
Amir

Version

0.18.3

Operating System

macOS

Package Manager

Conda

Interface

Script (i.e., python my_script.py)

Relevant log output

Compiling Julia backend...
[ Info: Started!
2.4%┣██                                                                                  ┫ 71/3.0k [00:02<01:07, 44it/s]Expressions evaluated per second: 7.85e+04. Head worker occupation: 22.4%                                               Press 'q' and then <enter> to stop execution early.                                                                     Hall of Fame:                                                                                                           ----------------------------------------------------------------------------------------------------------------------- Complexity  Loss       Score     Equation                                                                               1           6.077e-01  1.594e+01  y = x2                                                                                2           6.057e-01  3.288e-03  y = cube(-0.33556)                                                                    3           5.736e-01  5.450e-02  y = -0.18654 * x1                                                                     4           5.716e-01  3.527e-03  y = cube(-0.21912 * x1)                                                               5           5.017e-01  1.303e-01  y = 0.24248 * (x0 - x1)                                                               6           3.090e-01  4.849e-01  y = cube(0.33309 * (x0 - x1))                                                         8           2.241e-01  1.606e-01  y = 0.24248 * (square(x0 - x1) - x0)                                                  12          2.219e-01  2.427e-03  y = ((square(x0 - x1) + (x2 - x1)) * 0.21079) + -0.24809                              14          2.136e-01  1.898e-02  y = 0.24248 * (((square((x0 - x1) - -0.17614) - x0) - 1.3057) - -0.22216)             16          2.097e-01  9.353e-03  y = (x2 + ((0.1745 * 1.3059) * (square(x1 - x0) - (0.54558 * x1)))) + -0.34857        17          2.013e-01  4.082e-02  y = (((square(x0 - x1) + x2) * 0.21079) - square((0.096751 * -2.0963) * x1)) + -0....                                   24809                                                                                 19          1.975e-01  9.550e-03  y = (((square(x0 - x1) + x2) * 0.21079) - square(((0.096751 * x0) * x1) * 0.57378)...                                   ) + -0.24809                                                                          20          1.954e-01  1.073e-02  y = (((square(inv(-0.7479)) * square((x0 + -0.34297) - x1)) - x1) * (-0.083091 * i...                                   nv(cube(-0.82306)))) - 0.2591

Extra Info

No response

[MilesCranmer]

Hi @amirhszd,
Are you able to use a newer Python? Python 3.7 stopped receiving security updates 1.5 years ago, so is no longer recommended to use in any context:

The most recent PySR version is 1.4.0. This version is only supported on Python 3.10 and above, and will not be available on older Python versions. Are you able to update Python?

Cheers,
Miles

[amirhszd]

Thank you very much for this!

[amirhszd]

Unfortunately, I am still seeing the same behavior with python 3.11. I am able to run it with 500 iterations and sampling 1000 from my dataset, but anything above that freezes.

[MilesCranmer]

What version of PySR? Can you update to 1.4?

[MilesCranmer]

Also please share your entire script if possible

[amirhszd]

I am using PySR 1.4.

below is my code:

    df = pd.read_csv("data.csv")

    x = np.vstack([df["w"].to_numpy(),
                   df["T1"].to_numpy() - df["T2"].to_numpy(),
                   (df["e1"].to_numpy() + df["e2"].to_numpy())/2, 
                   df["e1"].to_numpy() - df["e2"].to_numpy()]
                  ).T.astype(np.float32) 
    y = df['error'].to_numpy()

    from pysr import PySRRegressor
    model = PySRRegressor(
        maxsize=20,
        niterations=1000,  # < Increase me for better results
        binary_operators=["*", "+", "-", "/"],
        unary_operators=["square", "cube", "inv(x) = 1/x"],
        extra_sympy_mappings={"inv": lambda x: 1 / x},
    )

    indices = np.random.choice(np.arange(len(x)), 1000)
    model.fit(x[indices], y[indices])

[MilesCranmer]

Can you share data.csv too? I really am not sure what could cause freezing though. Can you share the entire output from start to finish, even stderr if it is available? And if you monitor the process in the "Activity Monitor", is it still running?

[JinningLianggithub]

Hi Miles,

I encountered similar problem. I am using a Python version of 3.10.12, about 5000 data point, 10 features. The parameter is below. When it stuck, I use htop and cannot see it's running. But at beginning I can see it in htop. Do you have idea how to fix this? Thank you in advance.

Cheers
Jinning

#log
model = PySRRegressor(
populations=150,
# ^ Assuming we have 4 cores, this means 2 populations per core, so one is always running. Better set a 3proc
#population_size=50,
ncycles_per_iteration=500,
# ^ Generations between migrations.
niterations=300, # Run forever
early_stop_condition=(
"stop_if(loss, complexity) = loss < 1e-4 && complexity < 50"
# Stop early if we find a good and simple equation
),
timeout_in_seconds=60 * 60 * 8,
# ^ Alternatively, stop after 1 hours have passed.
maxsize=100,
# ^ Allow greater complexity.
#maxdepth=10,
# ^ But, avoid deep nesting.
binary_operators=["","^","+"],
unary_operators=["exp","log10"],
constraints={
"+": (-1, -1),
#"-": (8, 8),
"*": (10, 10),
#"/": (3, 3),
"^": (10, 2),
"exp": 10,
"log10": 10,
#"log10": 3,
},
complexity_of_variables=3,
# ^ Limit the complexity within each argument.
# "inv": (-1, 9) states that the numerator has no constraint,
# but the denominator has a max complexity of 9.
# "exp": 9 simply states that exp can only have
# an expression of complexity 9 as input.
nested_constraints={
#"^" : { "exp": 0, "log10": 0, "^":0},
"exp": { "exp": 0, "log10": 0, "^":0},
"log10": { "exp": 0, "log10": 0, "^":0},},
#"log10": { "exp": 0, "log": 0, "log10": 0, "^":0},
#},
# ^ Nesting constraints on operators. For example,
# "square(exp(x))" is not allowed, since "square": {"exp": 0}.
#complexity_of_operators={"/": 2, "exp": 3},
# ^ Custom complexity of particular operators.
complexity_of_constants=1,
# ^ Punish constants more than variables
#select_k_features=4,
# ^ Train on only the 4 most important features
progress=True,
# ^ Can set to false if printing to a file.
#weight_randomize=0.1,
# ^ Randomize the tree much more frequently
#precision=32,
# ^ Higher precision calculations.
#turbo=True,
# ^ Faster evaluation (experimental),
#batching=True,
#batch_size=2000,
#parallelism="multiprocessing",
procs=50,
turbo=True,
weight_optimize=0.001,
elementwise_loss=elementwise_loss,
#loss_function=loss_function,
#warm_start=True
#parsimony=1e-5/5,
#weight_optimize=0.001,
#bumper=True
verbosity=1,
)

[MilesCranmer]

Could you edit your comment to use the triple backticks? Then it will display as code and be easier to read. Thanks!

[JinningLianggithub]

Thanks for reply! It is attached below. I deleted commented lines

model = PySRRegressor(
populations=150,
ncycles_per_iteration=500,
niterations=300, # Run forever
early_stop_condition=(
"stop_if(loss, complexity) = loss < 1e-4 && complexity < 50"
),
timeout_in_seconds=60 * 60 * 8,
maxsize=100,
binary_operators=["","^","+"],
unary_operators=["exp","log10"],
constraints={
"+": (-1, -1),
"*": (10, 10),
"^": (10, 2),
"exp": 10,
"log10": 10,
},
complexity_of_variables=3,
nested_constraints={
"exp": { "exp": 0, "log10": 0, "^":0},
"log10": { "exp": 0, "log10": 0, "^":0},
}
complexity_of_constants=1,
progress=True,
procs=50,
turbo=True,
weight_optimize=0.001,
elementwise_loss=elementwise_loss,
verbosity=1,
)

[MilesCranmer]

Hm not sure. But what is binary_operators=["","^","+"]? Is the operator not shown?

[JinningLianggithub]

It's ["*","^","+"], not sure why it didn't show. Would it be possible that the data is too large? Or the tree is too deep?