Predator Prey Lotka-Volterra Model

Looking at Predator Prey Model and applying Neural ODE, UDE and PINNs to fit the model The Lotka-Volterra model, also known as the predator-prey model, describes the dynamics of biological systems in which two species interact, one as a predator and the other as prey. The model is represented by a pair of first-order, nonlinear differential equations:

Summary

The Lotka-Volterra model captures the cyclical nature of predator-prey interactions:

$$ \begin{cases} \frac{dx}{dt} = \alpha x - \beta xy \\ \frac{dy}{dt} = \delta xy - \gamma y \end{cases} $$

where:

• $x$ represents the number of prey (e.g., rabbits).
• $y$ represents the number of predators (e.g., foxes).
• $\alpha$ is the natural growth rate of prey in the absence of predators.
• $\beta$ is the rate at which predators destroy prey.
• $\gamma$ is the natural death rate of predators in the absence of prey.
• $\delta$ is the rate at which predators increase by consuming prey.

Plot:

Dynamics:

1. When prey is abundant, predators have plenty to eat, leading to an increase in the predator population.
2. As the predator population grows, it consumes more prey, leading to a decrease in the prey population.
3. With fewer prey available, the predator population starts to decline due to starvation.
4. As the predator population decreases, the prey population begins to recover, starting the cycle anew.

Code Files:

1. The Lotka_Volterra_Model.jl files contains synthetic data(noiseless) using Lotka-Volterra model, application of Neural ODE and forecasting using
   • relu activation function
   • 3 layers (including input and output layer)
   • Adam Optimizer with maximum iterations of 1000 and learning rate of 0.001
2. The Lotka_Volterra_Model_rbf_BFGS.jl file contains same data and follows same methodology except change in following ones:
   • rbf activation function
   • 4 layers (including input and output layer)
   • Adam Optimizer with maximum iterations of 500 and learning rate of 0.001
   • BFGS Optimizer with maximum iterations of 100 and learning rate of 0.01
3. The Lotka_Volterra_Model_forecast.jl file contains the forecasting breakdown case scenarios:
   • 90% training data
   • 70% training data
   • 50% training data
   • 30% training data
   • 10% training data
4. The UDE/Lotka_Volterra_UDE_shallow.jl file contains optimized model fit code for UDE system

Neural ODE

Using true data from the above and then fitting Neural ODE to fit the data Using Adam Optimizer to optimize:

After using RBF Activation Function and both ADAM and BFGS Optimizer, Final Output:

Hyperparameter Tuning Plots

Logarithmic Loss Variation with Different Activation Functions

Loss Variation with Different Number of Hidden Units

Logarithmic Loss Variation with Different Adam Step Sizes

Forecast:

Purpose: The forecasting part is intended to use the trained neural ODE model to predict future states of the system beyond the time range it was originally trained on. Steps:

• Defined an extended time span for prediction.
• Specified the time points for which predictions are needed.
• Created a neural ODE object configured for forecasting.
• Ran the neural ODE to generate forecasts using the optimized model parameters.
• Plotted both the training and forecasted data to compare model predictions against actual data and observe expected future behavior.

After Further Optimization with BFGS and rbf activation function :

Forecast Break Down Point:

Case 1: Training Neural ODE with 90% data and forecasting on remaining 10%

Case 2: Training Neural ODE with 70% data and forecasting on remaining 30%

Case 3: Training Neural ODE with 50% data and forecasting on remaining 50%

Case 4: Training Neural ODE with 40% data and forecasting on remaining 60%

Case 5: Revised Plot: Training Neural ODE with 35% data and forecasting on remaining 65%

Case 6: Training Neural ODE with 30% data and forecasting on remaining 70%

Case 7: Training Neural ODE with 10% data and forecasting on remaining 90%

UDE

The generated data for the prey and predator populations over a time span of 10 units is then used to train a UDE model. The UDE incorporates neural networks to learn the unknown interaction terms in the Lotka-Volterra equations. Specifically, we treat the interaction terms $\beta xy$ and $\delta xy$ as functions that can be learned from data using neural networks. UDE Construction In this implementation:

The learned system using neural networks is described as:

$$ \begin{cases} \frac{dx}{dt} = \alpha x - \text{NN}_1(x, y) \\ \frac{dy}{dt} = -\gamma y + \text{NN}_2(x, y) \end{cases} $$

Where:

• $\text{NN}_1(x, y)$ is a neural network that approximates the interaction term $\beta xy$.
• $\text{NN}_2(x, y)$ is a neural network that approximates the interaction term $\delta xy$.

Two neural networks (NN1 and NN2) model the interaction terms $\beta xy$ and $\delta xy$ respectively. These networks are trained to minimize the loss between the generated data and the predicted populations.

Optimization Strategy

To train the UDE, the following optimization process is used:

Adam Optimizer: The UDE is first trained using the Adam optimizer with a learning rate of 0.001 over 20,000 iterations. RMSProp Optimizer: The model is then fine-tuned using RMSProp with a learning rate of 0.001 and momentum ρ = 0.9 for 5,000 iterations.

Optimized Fitted Data:

Neural Network Architecture

Most importantly in UDE it works perfectly with shallow networks

The neural networks $\text{NN}_1$ and $\text{NN}_2$ are constructed using the Lux library in the following architecture:

NN1 = Lux.Chain(
    Lux.Dense(2, 10, relu),
    Lux.Dense(10, 10, relu),
    Lux.Dense(10, 10, relu),
    Lux.Dense(10, 1)
)

NN2 = Lux.Chain(
    Lux.Dense(2, 10, relu),
    Lux.Dense(10, 10, relu),
    Lux.Dense(10, 10, relu),
    Lux.Dense(10, 1)
)

Results

The final trained UDE model is used for forecasting the population dynamics over a longer time span of 20 units. The results show that the neural network accurately captures the cyclical behavior of the predator-prey dynamics. The predictions are visualized alongside the original data for comparison.

Visualization

The following plot shows the learned dynamics:

The solid lines represent the generated data for the prey and predator populations. The dashed lines show the UDE model's predictions for both populations over the forecasting period. The model accurately predicts the oscillations in prey and predator populations, indicating the successful learning of the interaction dynamics.

Forecasting

The UDE model is trained on lesser data (% of original time plan) and forecasted on remaining part to find the forecasting breakdown point

From the above Neural ODE model we found that the forecast breakdown point was 0.35 % of training data and the forecast was completely wrong

Whereas in case of UDE for the same 0.35% training data, the model is able to capture the underlying data very well

Case: Training Neural ODE with 35% data and forecasting on remaining 65%

In addition to it, we observed the UDE trains well with even shallower neural network of 5 neurons of 3 layers whereas for Neural ODE for the same data point we used 100 neurons and 5 layers

Forecasting on further timestamps till 30 timesteps with training on just 3.5 timesteps

We went ahead with 31% training data and found although it was able to capture the underlying data it started deviating

Breakdown Edge Case 1 (Partial Breakdown): Training Neural ODE with 31% data and forecasting on remaining 69%

Forecasting on further timestamps till 30 timesteps with training on just 3.1 timesteps

Breakdown Edge Case 2 (Complete Breakdown): Training Neural ODE with 30% data and forecasting on remaining 70%

Optimizers Plot

How Loss differs with Optimizers along with iterations

Adding Gaussian Noise and Forecasting

With 0.005 sd Noise

Data:

Forecast:

With 0.01 sd Noise

Data:

Forecast:

With 0.3 sd Noise

Data:

Forecast: