Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
148 commits
Select commit Hold shift + click to select a range
43e94db
Symbolic Outputs branch - new parameter dict & symbolic tensor
ushham Jul 6, 2023
2e8b461
Working symbolic parameters
ushham Jul 7, 2023
e1d6d5d
Update to ip - working for symbolic, broken the numeric
ushham Jul 7, 2023
eec1dd8
Ocean symbolic ip updated
ushham Jul 10, 2023
0e8b1c7
Ground symbolic ip updated
ushham Jul 10, 2023
7b01564
working symbolic ip - not tested
ushham Jul 10, 2023
d554839
Symbolic tensor DE started - not working
ushham Jul 10, 2023
07518fa
Update to parameter locations
ushham Jul 11, 2023
e02720f
edit to subs in IP
ushham Jul 11, 2023
0573894
Bug fixing the symbolic tensor
ushham Jul 11, 2023
d85c1da
working out a way to take tensor products
ushham Jul 14, 2023
2963976
Addition of tensordot and calculating inverse matricies properly
ushham Jul 18, 2023
411cb28
Update of addition to dict
ushham Jul 18, 2023
7f3728e
using tensorproduct in all calculations
ushham Jul 19, 2023
7d583ee
Rough draft of dynT
ushham Jul 19, 2023
f64422b
Addition of useful functions
ushham Jul 20, 2023
b600ad5
Changing the way subs work in IP
ushham Jul 20, 2023
cb29fd4
Edits to the way printing of the tensor works
ushham Jul 20, 2023
9b618f9
change to definition of n in basis
ushham Jul 21, 2023
7e658f6
First working symbolic tensor - outputs incorrect4
ushham Jul 24, 2023
a8903d0
Fix of sign errors - Stored Linear Tensor checked for atm-ocn
ushham Jul 25, 2023
046cadc
Edit to functions to allow them to work with tensors of any rank
ushham Jul 25, 2023
5b5a39f
Bugs in DE tensor calc fixes
ushham Jul 25, 2023
20da196
Update to dynamic_T loop for speed
ushham Jul 25, 2023
7005af0
cleaning up the file
ushham Jul 25, 2023
222fc0a
notebooks used to test tensor
ushham Jul 25, 2023
5012c93
Update to do all calculations using dictionaries not symbolic tensors
ushham Jul 26, 2023
ce956d0
issue with DE qg tensor - not fixed
ushham Jul 26, 2023
f509edf
Edit to structure of additional functions
ushham Jul 27, 2023
3d094dc
Dyn T tensor working - output of results draft
ushham Aug 3, 2023
746126a
First working equation outputs
ushham Aug 4, 2023
fed0c47
addition of T4 tensor calc
ushham Aug 5, 2023
45289e7
Fixed non-stored versions
ushham Aug 7, 2023
dbf03ba
Bugs fixed in numerically testing
ushham Aug 8, 2023
7364af2
Finding error in DE version
ushham Aug 8, 2023
e1a4ad5
Working version of variable substitution
ushham Aug 9, 2023
c4133a6
Lambified function output working for multiple variables
ushham Aug 9, 2023
b69e5d5
Trying to fix O IP
ushham Aug 10, 2023
547cbed
Cleaning up Linear notebook
ushham Aug 10, 2023
014402c
Bug fixes for ground version
ushham Aug 16, 2023
e7f03cd
Addition of all variables
ushham Aug 17, 2023
613f60b
Inner products can be calculated numerically and output symbolically
ushham Aug 18, 2023
bc49dc5
Tests added for symbolic version
ushham Aug 18, 2023
7317705
Minor edits to structure of outputs
ushham Aug 18, 2023
f4c09c8
Draft auto output
ushham Aug 19, 2023
8e77fae
Addition of some documentation
ushham Aug 19, 2023
6ff0df5
Bug fixes and addition of some documentation
ushham Aug 20, 2023
c8257f1
Notebooks tested
ushham Aug 20, 2023
f98664e
Minor bug fixes - tests now passing
ushham Aug 20, 2023
f48ff3e
Change to subs to work with Sympy==1.12
ushham Aug 24, 2023
9a5e5c4
Bugs in substitutions fixed
ushham Aug 24, 2023
af0b5a5
Added a property symbol for the parameter class
jodemaey Aug 30, 2023
41f066a
Reverting some of the previous changes
jodemaey Aug 30, 2023
fa50cab
Created a ScalingParameter to address a longstanding problem
jodemaey Aug 31, 2023
4d1d2e2
Implemented an array parameters class
jodemaey Aug 31, 2023
8387996
Added some slight modifications to the previous commit
jodemaey Aug 31, 2023
e304a47
Implemented symbols in Parameter and ArrayParameter
jodemaey Sep 1, 2023
60bf7d3
Changed the name of parameters array
jodemaey Sep 13, 2023
64836d7
Modifying pipeline to work with new parameters functionality - Not wo…
ushham Sep 14, 2023
ae084ac
New symbolic pipeline - Linear version working
ushham Sep 15, 2023
935c412
Symbolic pipeline - dyn_T working - all tests passing
ushham Sep 15, 2023
33c32cf
Update to auto outputs
ushham Sep 15, 2023
752e985
Cleaning library imports
ushham Sep 15, 2023
526e28b
Edit to julia output to make it work in DifferentialEquations library
ushham Sep 29, 2023
596c2ba
Tidy up of julia output
ushham Sep 29, 2023
182722c
Notebook cleaning
ushham Sep 29, 2023
9d559d2
bug and catches fixed in non-stored version
ushham Sep 30, 2023
8a25fae
Reverting change to make the params setter unified for ground and ocean
jodemaey Oct 6, 2023
5f62fcf
Merge branch 'feature/symbolic_outputs' of gitlab-me.oma.be:ClimDyn/q…
jodemaey Oct 6, 2023
a24017a
Removing sparsesymbols property
jodemaey Oct 6, 2023
47dcb98
Cosmetics
jodemaey Oct 6, 2023
45ea945
Cosmetics
jodemaey Oct 6, 2023
e253114
Cosmetics + Comments
jodemaey Oct 6, 2023
2ada3c5
Continued cosmetics + comments
jodemaey Oct 9, 2023
600772e
Added a symbolic expression property for the parameters
jodemaey Oct 9, 2023
4936a41
Implemented basic operations for parameters
jodemaey Oct 10, 2023
11a6f93
Finished operations definition for the Parameter class
jodemaey Oct 10, 2023
4b33c2b
Cont'd
jodemaey Oct 10, 2023
2dd9859
Finished operations definition for the all classes
jodemaey Oct 10, 2023
f5cbf37
Solved some bugs
jodemaey Oct 10, 2023
fd16c09
Implemented sqrt for parameters arithmetic
jodemaey Oct 10, 2023
685365d
Comments
jodemaey Oct 10, 2023
3c08362
Updated description strings
jodemaey Oct 11, 2023
e990360
ParametersArray operations are now units and symbolic aware
jodemaey Oct 11, 2023
fd1dc26
Corrected the way dimensionality is managed in arithmetic operations
jodemaey Oct 11, 2023
dff6919
Fall back to default arithmetic operations if things go wrong
jodemaey Oct 11, 2023
1eb29f3
Adapted the symbolic QgTensor to the new paramters symbolics
jodemaey Oct 11, 2023
f7a9e43
Moved the default AUTO templates inside the symbolic output module
jodemaey Oct 18, 2023
08e03c0
Deleting the hidden template files.
jodemaey Oct 18, 2023
31ca66c
Continued comments
jodemaey Oct 18, 2023
1871e4d
Documentation added and fix of auto tab spacing
ushham Oct 25, 2023
da18fa8
Bug fix for auto output
ushham Oct 25, 2023
1d7348a
Error fixed in funciton print out
ushham Oct 25, 2023
5d7c888
Merge including AUTO edits
ushham Oct 26, 2023
7274f8e
Auto mistakes fixed
ushham Oct 26, 2023
a652e28
Solving a bug in the AUTO config file
jodemaey Nov 19, 2023
aba7750
Added some AUTO standard parameters name
jodemaey Nov 19, 2023
d03bd81
Adding a first notebook to show AUTO computation
jodemaey Nov 19, 2023
6304452
Letting the user to choose if params and solutions are initialized in
jodemaey Nov 20, 2023
5204f73
Merge branch 'feature/symbolic_outputs' of gitlab-me.oma.be:ClimDyn/q…
jodemaey Nov 20, 2023
709c0d7
Update of the AUTO notebook
jodemaey Nov 20, 2023
325c5b3
Corrected a bug in the base class for tests of symbolic outputs
jodemaey Nov 20, 2023
e624198
Corrected a bug in the symbol of the gamma_a parameter
jodemaey Nov 20, 2023
e4edaef
Added a test for the symbolic output of the land-atm model
jodemaey Nov 20, 2023
4c5755a
Merge branch 'feature/symbolic_outputs' of gitlab-me.oma.be:ClimDyn/q…
jodemaey Nov 20, 2023
0b71693
Cont'd
jodemaey Nov 20, 2023
f064cca
symbolic dict can be provided not in equation order
ushham Nov 27, 2023
f4d6509
Update of the Notebook
jodemaey Dec 1, 2023
b64e574
Corrected a bug in commit https://gitlab-me.oma.be/ClimDyn/qgs/-/comm…
jodemaey Dec 1, 2023
dd68ebb
Changed the name of the AUTO variables (using var_string)
jodemaey Dec 1, 2023
3277e90
Update of the creation mechanisms for Params Array
jodemaey Jan 7, 2024
db2a5ce
Trying to symplify the parameters substitutions
jodemaey Jan 12, 2024
d7d5532
Obvious bug corrected
jodemaey Jan 12, 2024
6b19b0f
Another bug
jodemaey Jan 12, 2024
2e9f816
Trying again
jodemaey Jan 12, 2024
d4b6c7c
Abandonning the idea of passing a list of symbols as continuation
jodemaey Jan 12, 2024
e3a56ab
Commenting
jodemaey Jan 12, 2024
8e8fa60
Orography spectral coefficients now have symbols
jodemaey Jan 13, 2024
42e6da1
Cleaning and commenting
jodemaey Jan 13, 2024
b57d446
Renamed symbolic_output to symbolic_tendencies
jodemaey Jan 13, 2024
ff9cdb0
Cleaning
jodemaey Jan 14, 2024
0435b24
Cont'd
jodemaey Jan 14, 2024
e364c40
Trying to accelerate a bit the symbolic equations
jodemaey Jan 15, 2024
b31a353
Fixed a bug in the plot functions of AUTO notebook
jodemaey Jan 26, 2024
ab4fd63
Output of jacobian as function - ERROR IN JACOBIAN
ushham Feb 15, 2024
36bd031
Merge remote-tracking branch 'refs/remotes/origin/feature/symbolic_ou…
ushham Feb 15, 2024
2445742
Symbolic tensor contraction bug fix
ushham Feb 16, 2024
27763d3
Correction of bugs in symbolic dynT - Tests not passing yet
ushham Feb 22, 2024
e614e6d
Fix of missue of 'connect_to_ocean' in test
ushham Feb 23, 2024
55f0882
Clean up of function outputs - Tests passing
ushham Feb 23, 2024
02d4bf6
Cleanup of notebook
ushham Feb 28, 2024
2ce9cd0
Symbolic outputs documentation
ushham Jun 4, 2024
17c8033
intro notebook added
ushham Oct 7, 2024
af6c51d
Bug fix - ground modes
ushham Oct 7, 2024
75d9a23
Inclusion of translating final outputs - bug found where EPSILON was …
ushham Oct 7, 2024
f093252
removal of rogue print statement
ushham Oct 7, 2024
f420fe7
Working function translation - fortran EPSILON bug fix
ushham Oct 8, 2024
ffbbb89
Bug fix - replacing of strings before eq is split across rows - fortr…
ushham Oct 9, 2024
157905c
Replacing lambda string in python symbolic outputs
ushham Dec 21, 2024
3da2ff5
Bug fix in non-stored symbolic tensor
ushham Jan 10, 2025
e905c63
Cleaning notebooks
jodemaey Mar 11, 2025
66c9b58
Added a new notebook to show how to use the symbolic tendencies
jodemaey Mar 11, 2025
9ff415f
Cleaning again
jodemaey Mar 11, 2025
b7a395b
Added a notebook for the AUTO symbolic tendencies
jodemaey Mar 11, 2025
b0bb3f3
Moving the symbolic outputs notebook to a given folder
jodemaey Mar 11, 2025
55b84ec
Merge branch 'develop' into feature/symbolic_outputs
jodemaey Mar 11, 2025
af2aac5
Cleaning again
jodemaey Mar 11, 2025
61eaa03
Solved a bug that prevented reuse of the test classes
jodemaey Mar 12, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 16 additions & 0 deletions documentation/source/files/technical_description.rst
Original file line number Diff line number Diff line change
Expand Up @@ -94,6 +94,20 @@ The computational flow is as follows:

Sketch of the computational flow.


Generating Symbolic Equations
-----------------------------
Using `Sympy`_ qgs offers the functionality to return the ordinary differential equations of the projected model as a string, with any parameters the user chooses to be returned as varibales in the equations. In addition, the resulting equations can be returned already formatted in the programming language of the users' choice. This allows the qgs framework to feed directly into pipelines in other programming languages. Currently the framework can return the model equations in the following languages:

* Python
* Julia
* Fortran-90
* Mathematica
* AUTO-p07 continuation software

This also allows the user to specify their own integration method for solving the model equations in python.


Additional technical information
--------------------------------

Expand All @@ -103,6 +117,8 @@ Additional technical information
* qgs has a `tangent linear model`_ optimized to run ensembles of initial conditions as well, with a broadcast
integration of the tangent model thanks to `Numpy`_.

* The symbolic output functionality of the qgs model relies on `Sympy`_ to perform the tensor calculations. This library is significantly slower than the numerical equivilent and as a result it is currently only feasible to generate the symbolic model equations for model resolutions up to :math:`4x4`.

References
----------

Expand Down
38 changes: 32 additions & 6 deletions documentation/source/files/user_guide.rst
Original file line number Diff line number Diff line change
Expand Up @@ -462,7 +462,33 @@ This concludes the initialization of qgs, the function :meth:`f` hence produced
An example of the construction exposed here, along with plots of the trajectories generated, can be found in the section :ref:`files/examples/manual:Manual setting of the basis`.
See also the following section for the possible usages.

4. Using qgs (once configured)

4. Symbolic outputs
-------------------

If the user wants to generate the model tendencies with non-fixed parameters, use the tendencies in another programming language, or use their own integrator in python, the qgs framework can use `Sympy`_ to proform the above calculations symbolically rather than numerically.

To return the tendencies including specified parameter values, or to format the tendencies in another programming language, the qgs model is configured as shown in section :ref:`files/user_guide:Configuration of qgs`, however the **Symbolic** inner product is always used in this pipeline. To create the symbolic tendencies, the instance of :class:`.QgParams` is passed to the function :func:`.create_symbolic_tendencies`:

.. code:: ipython3

from qgs.functions.symbolic_tendencies import create_symbolic_equations

parameters = [model_parameters.par1, model_parameters.par2, model_parameters.par3]

f = create_symbolic_equations(model_parameters, continuation_variables=parameters, language='python')

The varibale :meth:`f` is a string of the model tendencies, formatted in the programming language specified by the keyword :meth:`language`. The model tendencies will contain the specified :meth:`continuation_variables` as free parameters.

Currently the framework can format the equations in the following programming languages:
* :meth:`python`
* :meth:`julia`
* :meth:`fortran`
* :meth:`mathematica`
* :meth:`auto`


1. Using qgs (once configured)
---------------------------------

Once the function :math:`\boldsymbol{f}` giving the model's tendencies has been obtained, it is possible to use it with
Expand All @@ -482,7 +508,7 @@ the qgs built-in integrator to obtain the model's trajectories:

Note that it is also possible to use other ordinary differential equations integrators available on the market, see for instance the :ref:`files/examples/diffeq_example:Example of DiffEqPy usage`.

4.1 Analyzing the model's output with diagnostics
5.1 Analyzing the model's output with diagnostics
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

.. include:: diagnostic.rst
Expand Down Expand Up @@ -524,7 +550,7 @@ Note that it is also possible to use other ordinary differential equations integ

More diagnostics will be implemented soon.

4.2 Toolbox
5.2 Toolbox
^^^^^^^^^^^

The toolbox regroups submodules to make specific analysis with the model and are available in the :mod:`qgs.toolbox` module.
Expand All @@ -536,7 +562,7 @@ Presently, the list of submodules available is the following:

More submodules will be implemented soon.

4.2.1 Lyapunov toolbox
5.2.1 Lyapunov toolbox
""""""""""""""""""""""

This module allows you to integrate the model and simultaneously obtain the *local* `Lyapunov vectors`_ and `exponents`_.
Expand All @@ -555,10 +581,10 @@ See also :cite:`user-KP2012` for a description of these methods.

Some example notebooks on how to use this module are available in the `notebooks/lyapunov <../../../../notebooks/lyapunov>`_ folder.

5. Developers information
6. Developers information
-------------------------

5.1 Running the test
6.1 Running the test
^^^^^^^^^^^^^^^^^^^^^

The model core tensors can be tested by running `pytest`_ in the main folder: ::
Expand Down
103 changes: 103 additions & 0 deletions model_test/test_aotensor_sym.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,103 @@

import sys
import os

path = os.path.abspath('./')
base = os.path.basename(path)
if base == 'model_test':
sys.path.extend([os.path.abspath('../')])
else:
sys.path.extend([path])


import unittest
import numpy as np

from qgs.params.params import QgParams
from qgs.inner_products import symbolic
from qgs.tensors.qgtensor import QgsTensor
from qgs.tensors.symbolic_qgtensor import SymbolicQgsTensor

from model_test.test_base_symbolic import TestBaseSymbolic

real_eps = np.finfo(np.float64).eps


class TestSymbolicAOTensor(TestBaseSymbolic):
'''
Test class for the Linear Symbolic Tensor
The tensor is tested against the reference file, and then the numerical tensor calculated in qgs.
'''

filename = 'test_aotensor.ref'

def test_sym_against_ref(self):
self.check_lists_flt()

def test_sym_against_num(self):
self.check_numerical_lists_flt()

def symbolic_outputs(self, output_func=None):

if output_func is None:
self.symbolic_values.clear()
tfunc = self.save_ip_symbolic
else:
tfunc = output_func

params = QgParams({'rr': 287.e0, 'sb': 5.6e-8})
params.set_atmospheric_channel_fourier_modes(2, 2, mode="symbolic")
params.set_oceanic_basin_fourier_modes(2, 4, mode="symbolic")

# Setting MAOOAM default parameters
params.set_params({'kd': 0.04, 'kdp': 0.04, 'n': 1.5})

aip = symbolic.AtmosphericSymbolicInnerProducts(params, return_symbolic=True, make_substitution=True)
oip = symbolic.OceanicSymbolicInnerProducts(params, return_symbolic=True, make_substitution=True)

aip.connect_to_ocean(oip)

sym_aotensor = SymbolicQgsTensor(params=params, atmospheric_inner_products=aip, oceanic_inner_products=oip)

subbed_tensor = sym_aotensor.sub_tensor()

for coo, val in zip(subbed_tensor.keys(), subbed_tensor.values()):
_ip_string_format(tfunc, 'sym_aotensor', coo, val)

def numerical_outputs(self, output_func=None):

if output_func is None:
self.numerical_values.clear()
tfunc = self.save_ip_numeric
else:
tfunc = output_func

params = QgParams({'rr': 287.e0, 'sb': 5.6e-8})
params.set_atmospheric_channel_fourier_modes(2, 2, mode="symbolic")
params.set_oceanic_basin_fourier_modes(2, 4, mode="symbolic")

# Setting MAOOAM default parameters
params.set_params({'kd': 0.04, 'kdp': 0.04, 'n': 1.5})

aip = symbolic.AtmosphericSymbolicInnerProducts(params, return_symbolic=False)
oip = symbolic.OceanicSymbolicInnerProducts(params, return_symbolic=False)

aip.connect_to_ocean(oip)

num_aotensor = QgsTensor(params=params, atmospheric_inner_products=aip, oceanic_inner_products=oip)

for coo, val in zip(num_aotensor.tensor.coords.T, num_aotensor.tensor.data):
_ip_string_format(tfunc, 'num_aotensor', coo, val)


def _ip_string_format(func, symbol, indices, value):
if abs(value) >= real_eps:
s = symbol
for i in indices:
s += "["+str(i)+"]"
s += " = % .5E" % value
func(s)


if __name__ == "__main__":
unittest.main()
97 changes: 97 additions & 0 deletions model_test/test_aotensor_sym_dynT.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,97 @@

import sys
import os

path = os.path.abspath('./')
base = os.path.basename(path)
if base == 'model_test':
sys.path.extend([os.path.abspath('../')])
else:
sys.path.extend([path])


import unittest
import numpy as np

from qgs.params.params import QgParams
from qgs.inner_products import symbolic
from qgs.tensors.qgtensor import QgsTensorDynamicT
from qgs.tensors.symbolic_qgtensor import SymbolicQgsTensorDynamicT

from model_test.test_base_symbolic import TestBaseSymbolic

real_eps = np.finfo(np.float64).eps

class TestSymbolicAOTensorDynT(TestBaseSymbolic):
'''
Test class for the Dynamic T Symbolic Tensor
The tensor is tested against the reference file, and then the numerical tensor calculated in qgs.
'''

def test_sym_against_num(self):
self.check_numerical_lists_flt()

def symbolic_outputs(self, output_func=None):

if output_func is None:
self.symbolic_values.clear()
tfunc = self.save_ip_symbolic
else:
tfunc = output_func

params = QgParams({'rr': 287.e0, 'sb': 5.6e-8}, dynamic_T=True)
params.set_atmospheric_channel_fourier_modes(2, 2, mode="symbolic")
params.set_oceanic_basin_fourier_modes(2, 4, mode="symbolic")

# Setting MAOOAM default parameters
params.set_params({'kd': 0.04, 'kdp': 0.04, 'n': 1.5})

aip = symbolic.AtmosphericSymbolicInnerProducts(params, return_symbolic=True, make_substitution=True)
oip = symbolic.OceanicSymbolicInnerProducts(params, return_symbolic=True, make_substitution=True)

if not aip.connected_to_ocean:
aip.connect_to_ocean(oip)

sym_aotensor = SymbolicQgsTensorDynamicT(params=params, atmospheric_inner_products=aip, oceanic_inner_products=oip)

subbed_tensor = sym_aotensor.sub_tensor()

for coo, val in zip(subbed_tensor.keys(), subbed_tensor.values()):
_ip_string_format(tfunc, 'sym_aotensor', coo, val)

def numerical_outputs(self, output_func=None):

if output_func is None:
self.numerical_values.clear()
tfunc = self.save_ip_numeric
else:
tfunc = output_func

params = QgParams({'rr': 287.e0, 'sb': 5.6e-8}, dynamic_T=True)
params.set_atmospheric_channel_fourier_modes(2, 2, mode="symbolic")
params.set_oceanic_basin_fourier_modes(2, 4, mode="symbolic")

# Setting MAOOAM default parameters
params.set_params({'kd': 0.04, 'kdp': 0.04, 'n': 1.5})

aip = symbolic.AtmosphericSymbolicInnerProducts(params, return_symbolic=False)
oip = symbolic.OceanicSymbolicInnerProducts(params, return_symbolic=False)

if not aip.connected_to_ocean:
aip.connect_to_ocean(oip)

num_aotensor = QgsTensorDynamicT(params=params, atmospheric_inner_products=aip, oceanic_inner_products=oip)

for coo, val in zip(num_aotensor.tensor.coords.T, num_aotensor.tensor.data):
_ip_string_format(tfunc, 'num_aotensor', coo, val)

def _ip_string_format(func, symbol, indices, value):
if abs(value) >= real_eps:
s = symbol
for i in indices:
s += "["+str(i)+"]"
s += " = % .5E" % value
func(s)

if __name__ == "__main__":
unittest.main()
Loading