Merge pull request #30 from elmbeech/master

pcdl release v4.0.1

Author: elmbeech

.github/workflows/apple.yml

@@ -7,18 +7,16 @@ run-name: ${{ github.actor }}::pytest pcdl library on mac os x; the latest pytho
 
 on:
   push:
-    branches: ["master", "v3", "v4"]
-  #pull_request:
-  #  branches: ["master", "v3", "v4"]
+    branches: ["utest"]  # ["master", "v3", "v4"]
 
 jobs:
   build-macosx:
     runs-on: macos-latest
     strategy:
       fail-fast: false
       matrix:
-        python-version: ["3.13"]
-        #python-version: ["3.9", "3.10", "3.11", "3.12", "3.13"]
+        #python-version: ["3.14"]
+        python-version: ["3.10", "3.11", "3.12", "3.13"]
 
     env:
       PYTHONPATH: /Users/runner/work/physicelldataloader/physicelldataloader

.github/workflows/linux.yml

@@ -7,18 +7,16 @@ run-name: ${{ github.actor }}::pytest pcdl library on linux os; all python3 vers
 
 on:
   push:
-    branches: ["master", "v3", "v4"]
-  #pull_request:
-  #  branches: ["master", "v3", "v4"]
+    branches: ["utest"]  # ["master", "v3", "v4"]
 
 jobs:
   build-linux:
     runs-on: ubuntu-latest
     strategy:
       fail-fast: false
       matrix:
-        python-version: ["3.13"]
-        #python-version: ["3.9", "3.10", "3.11", "3.12", "3.13"]
+        #python-version: ["3.14"]
+        python-version: ["3.10", "3.11", "3.12", "3.13"]
 
     env:
       PYTHONPATH: /home/runner/work/physicelldataloader/physicelldataloader

.github/workflows/windows.yml

@@ -7,18 +7,16 @@ run-name: ${{ github.actor }}::pytest pcdl library on windows os; the latest pyt
 
 on:
   push:
-    branches: ["master", "v3", "v4"]
-  #pull_request:
-  #  branches: ["master", "v3", "v4"]
+    branches: ["utest"]  # ["master", "v3", "v4"]
 
 jobs:
   build-windows:
     runs-on: windows-latest
     strategy:
       fail-fast: false
       matrix:
-        python-version: ["3.13"]
-        #python-version: ["3.9", "3.10", "3.11", "3.12", "3.13"]
+        #python-version: ["3.14"]
+        python-version: ["3.10", "3.11", "3.12", "3.13"]
 
     env:
       PYTHONPATH: D:\a\physicelldataloader\physicelldataloader

README.md

@@ -8,17 +8,18 @@ into [python3](https://en.wikipedia.org/wiki/Python_(programming_language)).
 
 pcdl was forked from the original [PhysiCell-Tools](https://github.com/PhysiCell-Tools) [python-loader](https://github.com/PhysiCell-Tools/python-loader) implementation.
 
-The pcdl python3 library maintains three branches:
+The pcdl python3 library maintains four branches:
 
 + **Branch version 1** is the original PhysiCell-Tools/python-loader code.
 + **Branch version 2** will be strictly compatible with the original PhysiCell-Tools/python-loader code, although pip installable.
 + **Branch version 3** might break with old habits, although tries to be as much downward compatible as possible.
   The aim of the v3 branch is to get a very lean and agile python3 physicell output interface for the ones coming from the python3 world.
++ Finally, **Branch version 4** reimplemented the backend in a more python3, less C++ like manner.
 
 
 ## Header:
 
-+ Language: python [>= 3.9](https://devguide.python.org/versions/)
++ Language: python [>= 3.10](https://devguide.python.org/versions/)
 + Library dependencies: anndata, bioio, matplotlib, numpy, pandas, (requests), scipy, vtk
 + Date of origin original PhysiCell-Tools python-loader: 2019-09-02
 + Date of origin pcdl fork: 2022-08-30
@@ -28,12 +29,12 @@ The pcdl python3 library maintains three branches:
 + Source code: [https://github.com/elmbeech/physicelldataloader](https://github.com/elmbeech/physicelldataloader)
 
 
-## HowTo Guide:
+## ✨ HowTo Guide:
 
 + [installation and troubleshooting](https://github.com/elmbeech/physicelldataloader/tree/master/man/HOWTO.md)
 
 
-## Tutorial:
+## ✨ Tutorial:
 
 Basics Tutorials:
 
@@ -63,17 +64,15 @@ Extras tutorials for GUI software:
 + [pcdl and paraview](https://github.com/elmbeech/physicelldataloader/tree/master/man/TUTORIAL_paraview.md)
 + [pcdl and blender](https://github.com/elmbeech/physicelldataloader/tree/master/man/TUTORIAL_blender.md)
 + [pcdl and napari](https://github.com/elmbeech/physicelldataloader/tree/master/man/TUTORIAL_python3_napari.md)
-+ [pcdl and fiji imagej](https://github.com/elmbeech/physicelldataloader/tree/master/man/TUTORIAL_fijiimagej.md)
-<!--
-+ [pcdl and neuroglancer](https://github.com/elmbeech/physicelldataloader/tree/master/man/TUTORIAL_(python3)_neuroglancer.md)
--->
++ [pcdl and fiji imagej, icy, qupath](https://github.com/elmbeech/physicelldataloader/tree/master/man/TUTORIAL_fijiimagej.md)
++ [pcdl and neuroglancer](https://github.com/elmbeech/physicelldataloader/tree/master/man/TUTORIAL_neuroglancer.md)
 
 Slides:
 
 + [presentations given](https://github.com/elmbeech/physicelldataloader/tree/master/man/lecture)
 
 
-## Reference Manual:
+## &#x2728; Reference Manual:
 
 + [API application interface](https://github.com/elmbeech/physicelldataloader/tree/master/man/REFERENCE.md)
 
@@ -95,11 +94,14 @@ Within the pcdl library, we tried to stick to the documentation policy laid out
 + fork pcdl co-programmer: Furkan Kurtoglu, Jennifer Eng, Heber Rocha
 + fork pcdl continuous testing and feedbacks: Aneequa Sundus, John Metzcar
 + student prj on pcdl:
-  Benjamin Jacobs (make\_gml),
+  Benjamin Jacobs (make\_graph\_gml),
+  Jason Lu (render\_neuroglancer),
   Katie Pletz (beta testing),
+  Leena Sohail (beta testing),
   Marshal Gress (plot\_scatter),
   Nick Oldfather (unit test model),
-  Thierry-Pascal Fleurant (plot\_timeseries)
+  Thierry-Pascal Fleurant (plot\_timeseries),
+  Viviana Kwong (render\_neuroglancer)
 
 Developers, please make pull requests to the https://github.com/elmbeech/physicelldataloader/tree/development branch. Thanks!
 
@@ -121,13 +123,44 @@ Developers, please make pull requests to the https://github.com/elmbeech/physice
 ## Road Map:
 
 + evt generate lineage tree graph output files.
-+ evt add neuroglancer ome.tiff support.
-+ evt add DataDiVR support.
 
 ## Release Notes:
++ version 4.0.1 (2025-06-24): elmbeech/physicelldataloader
+    + man updated.
+    + minor bugfixes.
+
++ version 4.0.0 (2025-05-13): elmbeech/physicelldataloader
+    + v4 was forked from v3.3.4!
+    + **mcds.data** struct was rewritten in more python less c++ way.
+    + pyMCDS.py and part of pyAnnData.py was fused to **timestep.py**.
+    + pyMCDSts.py and part of pyAnnData.py was fused to **timeseries.py**.
+    + pyCLI.py was renames to **commandline.py**.
+    + data\_timeseries.py was renamed to **output_data.py**.
+    + TimeStep function **get_concentration** was deprecated because pandas already has this functionlity.
+    + TimeStep function **get_concentration_at** was deprecated because pandas already has this functionlity.
+    + TimeStep function **get_cell_df_at** was deprecated because pandas already has this functionlity.
+    + **make_conc_vtk** and **make_cell_vtk** on the fly visualization was removed because paraview is good enough.
+    + new TimeStep **get_cell_attribute_list** function, to retrieve a list of all tracked cell attribute labels.
+    + new **pcdl_get_cell_attribute_list** function, to retrieve a list of all tracked cell attribute labels.
+    + new **render_neuroglancer** function, to render ome tiff image into neuroglancer.
+    + new **pcdl_render_neuroglancer** function, to render ome tiff images into neuroglancer.
+
++ version 3.3.7 (2025-06-01): elmbeech/physicelldataloader
+    + compatible with current (non end-of-life cycle) python versions.
+    + minor bugfixes.
+
++ version 3.3.6 (2025-05-13): elmbeech/physicelldataloader
+    + compatible with numpy >= 2.0.0 and current (non end-of-life cycle) python versions.
+
++ version 3.3.5 (2025-05-13): elmbeech/physicelldataloader
+    + compatible with numpy < 2.0.0 and current (non end-of-life cycle) python versions.
+    + remove pyMCDS and pyMCDSts **make_ome_tiff** and pyCLI **pcdl_make_ome_tiff** to make pyMCS.py stand alone again.
+    + new TimeStep **get_cell_attribute_list** function, to retrieve a list of all tracked cell attribute labels.
+    + new **pcdl_get_cell_attribute_list** function, to retrieve a list of all tracked cell attribute labels.
+
 + version 3.3.4 (2025-03-07): elmbeech/physicelldataloader
     + replace the **aicsimageio** library dependency with its successor **bioio**. special thanks to Joel Eliason!
-    + **make_ome_tiff** can handle automatically generated file names with > 255 characters. special thank to Genevieve Stein-O'Brien and DanielBergman!
+    + **make_ome_tiff** can now handel generated file names with > 255 characters. special thank to Genevieve Stein-O'Brien and DanielBergman!
     + **get_mesh_spacing** handels now an edge case correctly that would have resulted in a division by zero. special thanks to Randy Heiland!
 
 + version 3.3.3 (2025-01-10): elmbeech/physicelldataloader

man/HOWTO.md

@@ -146,5 +146,6 @@ pyMCDS.py and the pyMCDS class is very lightweight.
 Besides the python3 core library, this code has only matplotlib, numpy, pandas, scipy, and vtk library dependencies.\
 The pyMCDS class evolved into the pcdl.TimeStep class, which has additionally anndata dependency, which makes the library slightly heavier but much more powerful for downstream data analysis.
 Apart from that, pcdl offers the pcdl.TimeSeries class to handle the mcds snapshots from an entire PhysiCell run, and a set of functions that can be run straight from the command line, without even having to fire up a python3 shell.
-Future branch version 4 will abandon this ancient library structure to become more concise.
-Don't fear. pyMCDS.py is there to last. We will keep on maintaining pyMCDS.py from branch version 3.
+Finally, branch version 4 broke with this ancient library structure because it is just out of time to run the code like this.\
+Stay assured, if you like pyMCDS.py, it is there to last.
+We will keep on maintaining pyMCDS.py from branch version 3.

man/REFERENCE.md

@@ -80,8 +80,6 @@ Basically, there are four types of functions:
 + [help(mcds.get_substrate_list)](https://github.com/elmbeech/physicelldataloader/tree/master/man/docstring/mcds.get_substrate_list.md)  #! workhorse function
 + [help(mcds.get_substrate_dict)](https://github.com/elmbeech/physicelldataloader/tree/master/man/docstring/mcds.get_substrate_dict.md)
 + [help(mcds.get_substrate_df)](https://github.com/elmbeech/physicelldataloader/tree/master/man/docstring/mcds.get_substrate_df.md)
-+ [help(mcds.get_concentration)](https://github.com/elmbeech/physicelldataloader/tree/master/man/docstring/mcds.get_concentration.md)
-+ [help(mcds.get_concentration_at)](https://github.com/elmbeech/physicelldataloader/tree/master/man/docstring/mcds.get_concentration_at.md)
 + [help(mcds.get_conc_df)](https://github.com/elmbeech/physicelldataloader/tree/master/man/docstring/mcds.get_conc_df.md)  #! workhorse function
 + [help(mcds.plot_contour)](https://github.com/elmbeech/physicelldataloader/tree/master/man/docstring/mcds.plot_contour.md)  #! workhorse function
 + [help(mcds.make_conc_vtk)](https://github.com/elmbeech/physicelldataloader/tree/master/man/docstring/mcds.make_conc_vtk.md)  #! workhorse function
@@ -90,7 +88,6 @@ Basically, there are four types of functions:
 + [help(mcds.get_celltype_list)](https://github.com/elmbeech/physicelldataloader/tree/master/man/docstring/mcds.get_celltype_list.md)  #! workhorse function
 + [help(mcds.get_celltype_dict)](https://github.com/elmbeech/physicelldataloader/tree/master/man/docstring/mcds.get_celltype_dict.md)
 + [help(mcds.get_cell_df)](https://github.com/elmbeech/physicelldataloader/tree/master/man/docstring/mcds.get_cell_df.md)  #! workhorse function
-+ [help(mcds.get_cell_df_at)](https://github.com/elmbeech/physicelldataloader/tree/master/man/docstring/mcds.get_cell_df_at.md)
 + [help(mcds.plot_scatter)](https://github.com/elmbeech/physicelldataloader/tree/master/man/docstring/mcds.plot_scatter.md)  #! workhorse function
 + [help(mcds.make_cell_vtk)](https://github.com/elmbeech/physicelldataloader/tree/master/man/docstring/mcds.make_cell_vtk.md)  #! workhorse function
 + [help(mcds.get_anndata)](https://github.com/elmbeech/physicelldataloader/tree/master/man/docstring/mcds.get_anndata.md)  #! workhorse function
@@ -102,13 +99,14 @@ Basically, there are four types of functions:
 
 ### TimeStep microenvironment and cells
 + [help(mcds.make_ome_tiff)](https://github.com/elmbeech/physicelldataloader/tree/master/man/docstring/mcds.make_ome_tiff.md)  #! workhose function
++ [help(mcds.make_neuroglancer)](https://github.com/elmbeech/physicelldataloader/tree/master/man/docstring/mcds.make_neuroglancer.md)  #! workhose function
 
 ### TimeStep internal functions
 + [help(pcdl.scaler)](https://github.com/elmbeech/physicelldataloader/tree/master/man/docstring/pcdl.scaler.md)  # anndata
 + [help(pcdl.graphfile_parser)](https://github.com/elmbeech/physicelldataloader/tree/master/man/docstring/pcdl.graphfile_parser.md)  # mcds
 ```python3
-help(pcdl.pyMCDS._read_xml)
-help(pcdl.pyAnnData._anndextract)
+help(pcdl.TimeStep._read_xml)
+help(pcdl.TimeStep._anndextract)
 ```
 
 
@@ -147,6 +145,7 @@ Basically, there are four types of functions:
 
 ### TimeSteries microenvironment and cells
 + [help(mcdsts.make_ome_tiff)](https://github.com/elmbeech/physicelldataloader/tree/master/man/docstring/mcdsts.make_ome_tiff.md)  #! workhose function
++ [help(mcdsts.make_neuroglancer)](https://github.com/elmbeech/physicelldataloader/tree/master/man/docstring/mcdsts.make_neuroglancer.md)  #! workhose function
 
 ### Timeseries timeseries
 + [help(mcdsts.plot_timeseries)](https://github.com/elmbeech/physicelldataloader/tree/master/man/docstring/mcdsts.plot_timeseries.md)  #! workhorse function
@@ -160,8 +159,8 @@ Basically, there are four types of functions:
 ### TimeSeries internal functions
 + [help(pcdl.scaler)](https://github.com/elmbeech/physicelldataloader/tree/master/man/docstring/pcdl.scaler.md)  # anndata
 ```python3
-help(pcdl.pyMCDSts._handle_magick)
-help(pcdl.pyAnnData._anndextract)
+help(pcdl.TimeSeries._handle_magick)
+help(pcdl.TimeSeries._anndextract)
 ```
 
 
@@ -196,6 +195,7 @@ The command line interface functions mimic the name and parameter arguments as c
 
 ### Command line cells and microenvironment
 + [pcdl_make_ome_tiff --help](https://github.com/elmbeech/physicelldataloader/tree/master/man/docstring/pcdl_make_ome_tiff.md)  #! workhorse function
++ [pcdl_make_neuroglancer --help](https://github.com/elmbeech/physicelldataloader/tree/master/man/docstring/pcdl_make_neuroglancer.md)  #! workhorse function
 
 ### Command line timeseries
 + [pcdl_plot_timeseries --help](https://github.com/elmbeech/physicelldataloader/tree/master/man/docstring/pcdl_plot_timeseries.md)  #! workhorse function

man/TUTORIAL_commandline.md

@@ -1,7 +1,5 @@
 # PhysiCell Data Loader Tutorial: pcdl from the Commandline
 
-<!-- bue 2024-08-22: have to check if this works from dos and power shell. it will work somehow, because i can run the cli unit tests -->
-
 The most important commands for down stream data analysis,
 available in the pcdl TimeStep and TimeSeries class,
 can be run straight from a command line shell, like [bash](https://en.wikipedia.org/wiki/Bash_(Unix_shell)), [csh](https://en.wikipedia.org/wiki/C_shell), [dos](https://en.wikipedia.org/wiki/DOS), [fish](https://en.wikipedia.org/wiki/Fish_(Unix_shell)), [ksh](https://en.wikipedia.org/wiki/KornShell), [powershell](https://en.wikipedia.org/wiki/PowerShell), [sh](https://en.wikipedia.org/wiki/Bourne_shell), [tsh](https://en.wikipedia.org/wiki/Tcsh), or [zsh](https://en.wikipedia.org/wiki/Z_shell), to name a view.
@@ -196,6 +194,19 @@ pcdl_get_celltype_list output/output00000000.xml
 pcdl_get_celltype_list -h
 ```
 
+### &#x2728; pcdl\_get\_cell\_attrribute\_list
+
+Output all recorded cell attributes.
+
+```bash
+pcdl_get_cell_attribute_list output
+```
+```bash
+pcdl_get_cell_attribute_list output/output00000000.xml
+```
+```bash
+pcdl_get_cell_attribute_list -h
+```
 
 ### &#x2728; pcdl\_get\_cell\_attribute
 
@@ -398,7 +409,7 @@ However, any numerical (bool, int, float) cell\_attribute can be outputted.
 For example: dead, cells\_per\_voxel, or pressure.
 
 These ome.tiff files can be further analyzed,
-for example with the [napari](https://napari.org/stable/) or [fiji imagej](https://fiji.sc/) or [blender](https://www.blender.org/) software,
+for example with the [Napari](https://napari.org/stable/) or [Fiji Imagej](https://fiji.sc/) or [Neuroglancer](https://research.google/blog/an-interactive-automated-3d-reconstruction-of-a-fly-brain/) or [Blender](https://www.blender.org/) or similar software,
 as described in the extra tutorials.
 
 ```bash
@@ -414,25 +425,33 @@ pcdl_make_ome_tiff -h
 Further readings:
 + [TUTORIAL_python3_napari.md](https://github.com/elmbeech/physicelldataloader/blob/master/man/TUTORIAL_python3_napari.md)
 + [TUTORIAL_fiji_imagej.md](https://github.com/elmbeech/physicelldataloader/blob/master/man/TUTORIAL_fijiimagej.md)
++ [TUTORIAL_neuroglancer.md](https://github.com/elmbeech/physicelldataloader/blob/master/man/TUTORIAL_neuroglancer.md)
 + [TUTORIAL_blender.md](https://github.com/elmbeech/physicelldataloader/blob/master/man/TUTORIAL_blender.md)
 
 
+### &#x2728; pcdl\_render\_neuroglancer
 
-## [Making movies](https://en.wikipedia.org/wiki/Making_Movies)
-
-
-### &#x2728; pcdl\_make\_movie
-
-Make a [mp4](https://en.wikipedia.org/wiki/MP4_file_format) movie from the jpeg plots from a time series.
+With this command, you can render a time step ome.tiff file or a time step from a whole time series ome.tiff file straight into [Neuroglancer](https://research.google/blog/an-interactive-automated-3d-reconstruction-of-a-fly-brain/), which is a [WebGL](https://en.wikipedia.org/wiki/WebGL)-based viewer that will render the ome.tiff straight in your browser.
 
+Below, we render a time step into Neuroglancer, first utilizing the time step ome.tiff, then using the whole time series ome.tiff.
+You can only render one time step at a time and not a entire time series, like, for example, in napari.
 ```bash
-pcdl_plot_scatter output
-pcdl_make_movie output/cell_cell_type_z0.0/
+pcdl_make_ome_tiff output/output00000000.xml
+pcdl_render_neuroglancer output_2d/output00000000_oxygen_water_default_blood_cells_ID.ome.tiff
 ```
 ```bash
-pcdl_make_movie -h
+pcdl_make_ome_tiff output
+pcdl_render_neuroglancer output_2d/timeseries_oxygen_water_default_blood_cells_ID.ome.tiff 3  # render time step 3 from the time series
+```
+```bash
+pcdl_make_ome_tiff -h
 ```
 
+Further readings:
++ [TUTORIAL_neuroglancer.md](https://github.com/elmbeech/physicelldataloader/blob/master/man/TUTORIAL_neuroglancer.md)
+
+
+## [Making movies](https://en.wikipedia.org/wiki/Making_Movies)
 
 ### &#x2728; pcdl\_make\_gif
 
@@ -446,7 +465,17 @@ pcdl_make_gif output/cell_cell_type_z0.0/
 pcdl_make_gif -h
 ```
 
+### &#x2728; pcdl\_make\_movie
+
+Make a [mp4](https://en.wikipedia.org/wiki/MP4_file_format) movie from the jpeg plots from a time series.
 
+```bash
+pcdl_plot_scatter output
+pcdl_make_movie output/cell_cell_type_z0.0/
+```
+```bash
+pcdl_make_movie -h
+```
 
 ## Data Clean Up