Enhance Plotting and Display Features, Improve Calibration Handling and Consistency (#59)

* Adds d-spacing parameter to project.py

* Adds d parameter to plotting.py

* Enhances notebook conversion with Jupytext

* Adds 2025 DMSC workshop tutorial to documentation

* Adds quick and dirty d-spacing plotting functionality

* Adds 1st draft of the dmsc summer school 2025 tutorial

* Updates Summer School tutorial

* Includes d-spacing branch in documentation build

* Relocates easydiffraction installation cell

* Refines powder diffraction tutorial

* Enhances Plotly figure display in Jupyter

Improves the display of Plotly figures in Jupyter notebooks by checking for IPython availability. Converts figures to HTML for better compatibility, avoiding warnings related to unknown MIME types.

* Suppresses Cryspy warnings by redirecting stderr

Wraps Cryspy calculation in stderr redirection to prevent
warning messages from cluttering the output. This is a
temporary measure until Cryspy handles warnings more
efficiently.

* Adjusts default quadratic calibration parameter

* Adds function to extract value from XYE file header

* Updates the summer school tutorial

* Configures Plotly to hide certain mode bar buttons

* Refactors CIF display functionality

* Enhances progress tracking with notebook support

* Improves phase handling in experiments

* Removes CIF display call

* Converts error logging to printed messages

* Adds d-spacing conversion for constant wavelength

* Adjusts default plotting engine based on environment

* Simplifies LBCO quick tutorial

* Standardizes variable naming for 2-theta calculations

* Add units to b_iso parameter in AtomSite class

* Formats fitter method arguments for readability

* Enhances fitting progress tracker with updated display

* Refactors header defaults for table rendering

* Update category key naming for consistency

* Corrects category key in atom site test

* Updates tracker start message in the unit test

* Expands parameter table with more details following discussion #24

* Switches data download to 'd-spacing' branch in tests

* Enhances Descriptor and Parameter string representation

* Improves user notifications for calculator engine imports

* Refactors calculator methods for improved readability

* Refactors report generation to improve table rendering

* Enhances table margins for improved layout

* Enhances TOF to d-spacing conversion

* Use env variable for default branch data download

Author: AndrewSazonov

.github/workflows/building-deploying-docs.yml

@@ -4,7 +4,7 @@ on:
   # Trigger the workflow on push
   push:
     # To the develop and master branches
-    branches: [develop, master, docs]
+    branches: [develop, master, docs, d-spacing]
 
   # Allows you to run this workflow manually from the Actions tab
   workflow_dispatch:
@@ -101,7 +101,7 @@ jobs:
       - name: Convert ${{ env.NOTEBOOKS_DIR }}/*.py to docs/${{env.NOTEBOOKS_DIR }}/*.ipynb
         run: |
           cp -R ${{ env.NOTEBOOKS_DIR }}/data docs/${{ env.NOTEBOOKS_DIR }}/
-          jupytext ${{ env.NOTEBOOKS_DIR }}/*.py --to ipynb
+          jupytext ${{ env.NOTEBOOKS_DIR }}/*.py --from py:percent --to ipynb
           mv ${{ env.NOTEBOOKS_DIR }}/*.ipynb docs/${{ env.NOTEBOOKS_DIR }}/
 
       # The following step is needed to avoid the following message during the build:

.github/workflows/testing-code.yaml

@@ -23,6 +23,11 @@ concurrency:
     ${{ github.workflow }}-${{ github.event.pull_request.number || github.ref }}
   cancel-in-progress: true
 
+env:
+  # Set the environment variables to be used in all jobs defined in this workflow
+  # Set the CI_BRANCH environment variable to be the branch name
+  CI_BRANCH: ${{ github.head_ref || github.ref_name }}
+  
 jobs:
   testing-code:
 

.github/workflows/testing-tutorials.yaml

@@ -14,6 +14,11 @@ concurrency:
     ${{ github.workflow }}-${{ github.event.pull_request.number || github.ref }}
   cancel-in-progress: true
 
+env:
+  # Set the environment variables to be used in all jobs defined in this workflow
+  # Set the CI_BRANCH environment variable to be the branch name
+  CI_BRANCH: ${{ github.head_ref || github.ref_name }}
+
 jobs:
   testing-tutorials:
 

docs/mkdocs.yml

@@ -77,6 +77,8 @@ nav:
           - Ni pd-neut-cwl: tutorials/pdf_pd-neut-cwl_Ni.ipynb
           - Si pd-neut-tof: tutorials/pdf_pd-neut-tof_Si-NOMAD.ipynb
           - NaCl pd-xray: tutorials/pdf_pd-xray_NaCl.ipynb
+      - Workshops & Schools:
+          - 2025 DMSC: tutorials/dmsc-summer-school-2025_analysis-powder-diffraction.ipynb
   - API Reference:
       - API Reference: api-reference/index.md
       - core: api-reference/core.md

docs/tutorials/index.md

@@ -66,3 +66,11 @@ The tutorials are organized into the following categories.
 - [NaCl `pd-xray`](pdf_pd-xray_NaCl.ipynb) –
   Demonstrates a PDF analysis of NaCl using data collected from an X-ray
   powder diffraction experiment.
+
+## Workshops & Schools
+
+- [2025 DMSC](dmsc-summer-school-2025_analysis-powder-diffraction.ipynb) –
+  A workshop tutorial that demonstrates a Rietveld refinement of the 
+  La0.5Ba0.5CoO3 crystal structure using time-of-flight neutron powder 
+  diffraction data simulated with McStas. This tutorial is designed for 
+  the ESS DMSC Summer School 2025.

src/easydiffraction/__init__.py

@@ -19,7 +19,10 @@
 from easydiffraction.summary import Summary
 
 # Utils
-from easydiffraction.utils.utils import download_from_repository
+from easydiffraction.utils.utils import (
+    download_from_repository,
+    get_value_from_xye_header
+)
 from easydiffraction.utils.formatting import (
     chapter,
     section,
@@ -39,5 +42,6 @@
     "chapter",
     "section",
     "paragraph",
-    'download_from_repository'
+    "download_from_repository",
+    "get_value_from_xye_header"
 ]

src/easydiffraction/analysis/analysis.py

@@ -2,7 +2,10 @@
 import numpy as np
 from typing import List, Optional, Union
 
-from easydiffraction.utils.utils import render_table
+from easydiffraction.utils.utils import (
+    render_cif,
+    render_table
+)
 from easydiffraction.utils.formatting import (
     paragraph,
     warning
@@ -204,8 +207,15 @@ def how_to_access_parameters(self) -> None:
             print(warning(f"No parameters found."))
             return
 
-        columns_headers = ['Code variable', 'Unique ID for CIF']
-        columns_alignment = ["left", "left"]
+        columns_headers = ['datablock',
+                           'category',
+                           'entry',
+                           'parameter',
+                           'How to Access in Python Code',
+                           'Unique Identifier for CIF Constraints']
+
+        columns_alignment = ['left', 'left', 'left', 'left', 'left', 'left']
+
         columns_data = []
         project_varname = self.project._varname
         for datablock_type, params in params.items():
@@ -215,12 +225,17 @@ def how_to_access_parameters(self) -> None:
                     category_key = param.category_key
                     entry_id = param.collection_entry_id
                     param_key = param.name
-                    variable = f"{project_varname}.{datablock_type}['{datablock_id}'].{category_key}"
+                    code_variable = f"{project_varname}.{datablock_type}['{datablock_id}'].{category_key}"
                     if entry_id:
-                        variable += f"['{entry_id}']"
-                    variable += f".{param_key}"
-                    uid = param._generate_human_readable_unique_id()
-                    columns_data.append([variable, uid])
+                        code_variable += f"['{entry_id}']"
+                    code_variable += f".{param_key}"
+                    cif_uid = param._generate_human_readable_unique_id()
+                    columns_data.append([datablock_id,
+                                         category_key,
+                                         entry_id,
+                                         param_key,
+                                         code_variable,
+                                         cif_uid])
 
         print(paragraph("How to access parameters"))
         render_table(columns_headers=columns_headers,
@@ -384,14 +399,19 @@ def fit(self):
 
         if self.fit_mode == 'joint':
             print(paragraph(f"Using all experiments 🔬 {experiment_ids} for '{self.fit_mode}' fitting"))
-            self.fitter.fit(sample_models, experiments, calculator, weights=self.joint_fit_experiments)
+            self.fitter.fit(sample_models,
+                            experiments,
+                            calculator,
+                            weights=self.joint_fit_experiments)
         elif self.fit_mode == 'single':
             for expt_name in experiments.ids:
                 print(paragraph(f"Using experiment 🔬 '{expt_name}' for '{self.fit_mode}' fitting"))
                 experiment = experiments[expt_name]
                 dummy_experiments = Experiments()  # TODO: Find a better name
                 dummy_experiments.add(experiment)
-                self.fitter.fit(sample_models, dummy_experiments, calculator)
+                self.fitter.fit(sample_models,
+                                dummy_experiments,
+                                calculator)
         else:
             raise NotImplementedError(f"Fit mode {self.fit_mode} not implemented yet.")
 
@@ -417,14 +437,6 @@ def as_cif(self):
         return "\n".join(lines)
 
     def show_as_cif(self) -> None:
-        cif_text = self.as_cif()
-        lines = cif_text.splitlines()
-        max_width = max(len(line) for line in lines)
-        padded_lines = [f"│ {line.ljust(max_width)} │" for line in lines]
-        top = f"╒{'═' * (max_width + 2)}╕"
-        bottom = f"╘{'═' * (max_width + 2)}╛"
-
-        print(paragraph(f"Analysis 🧮 info as cif"))
-        print(top)
-        print("\n".join(padded_lines))
-        print(bottom)
+        cif_text: str = self.as_cif()
+        paragraph_title: str = paragraph(f"Analysis 🧮 info as cif")
+        render_cif(cif_text, paragraph_title)

src/easydiffraction/analysis/calculators/calculator_base.py

@@ -3,6 +3,7 @@
 from typing import List, Any
 
 from easydiffraction.core.singletons import ConstraintsHandler
+from easydiffraction.sample_models.sample_model import SampleModel
 from easydiffraction.sample_models.sample_models import SampleModels
 from easydiffraction.experiments.experiment import Experiment
 
@@ -22,16 +23,22 @@ def engine_imported(self) -> bool:
         pass
 
     @abstractmethod
-    def calculate_structure_factors(self, sample_model: SampleModels, experiment: Experiment) -> None:
+    def calculate_structure_factors(
+            self,
+            sample_model: SampleModel,
+            experiment: Experiment
+    ) -> None:
         """
         Calculate structure factors for a single sample model and experiment.
         """
         pass
 
-    def calculate_pattern(self,
-                          sample_models: SampleModels,
-                          experiment: Experiment,
-                          called_by_minimizer: bool = False) -> np.ndarray:
+    def calculate_pattern(
+            self,
+            sample_models: SampleModels,
+            experiment: Experiment,
+            called_by_minimizer: bool = False
+    ) -> np.ndarray:
         """
         Calculate the diffraction pattern for multiple sample models and a single experiment.
 
@@ -84,10 +91,12 @@ def calculate_pattern(self,
         return y_calc_total
 
     @abstractmethod
-    def _calculate_single_model_pattern(self,
-                                        sample_model: SampleModels,
-                                        experiment: Experiment,
-                                        called_by_minimizer: bool) -> np.ndarray:
+    def _calculate_single_model_pattern(
+            self,
+            sample_model: SampleModels,
+            experiment: Experiment,
+            called_by_minimizer: bool
+    ) -> np.ndarray:
         """
         Calculate the diffraction pattern for a single sample model and experiment.
 
@@ -101,7 +110,11 @@ def _calculate_single_model_pattern(self,
         """
         pass
 
-    def _get_valid_linked_phases(self, sample_models: SampleModels, experiment: Experiment) -> List[Any]:
+    def _get_valid_linked_phases(
+            self,
+            sample_models: SampleModels,
+            experiment: Experiment
+    ) -> List[Any]:
         """
         Get valid linked phases from the experiment.
 
@@ -126,4 +139,4 @@ def _get_valid_linked_phases(self, sample_models: SampleModels, experiment: Expe
         if not valid_linked_phases:
             print('Warning: None of the linked phases found in Sample Models. Returning empty pattern.')
 
-        return valid_linked_phases
+        return valid_linked_phases

src/easydiffraction/analysis/calculators/calculator_crysfml.py

@@ -1,16 +1,18 @@
 import numpy as np
 from typing import Any, Dict, List, Union
-from .calculator_base import CalculatorBase
-from easydiffraction.utils.formatting import warning
+
 from easydiffraction.sample_models.sample_models import SampleModels
 from easydiffraction.sample_models.sample_models import SampleModel
 from easydiffraction.experiments.experiment import Experiment
 from easydiffraction.experiments.experiments import Experiments
 
+from .calculator_base import CalculatorBase
+
 try:
     from pycrysfml import cfml_py_utilities
+    print("✅ 'pycrysfml' calculation engine is successfully imported.")
 except ImportError:
-    print(warning('"pycrysfml" module not found. This calculator will not work.'))
+    print("⚠️ 'pycrysfml' module not found. This calculation engine will not be available.")
     cfml_py_utilities = None
 
 
@@ -25,7 +27,11 @@ class CrysfmlCalculator(CalculatorBase):
     def name(self) -> str:
         return "crysfml"
 
-    def calculate_structure_factors(self, sample_models: SampleModels, experiments: Experiments) -> None:
+    def calculate_structure_factors(
+            self,
+            sample_models: SampleModels,
+            experiments: Experiments
+    ) -> None:
         """
         Call Crysfml to calculate structure factors.
 
@@ -61,7 +67,11 @@ def _calculate_single_model_pattern(
             y = []
         return y
 
-    def _adjust_pattern_length(self, pattern: List[float], target_length: int) -> List[float]:
+    def _adjust_pattern_length(
+            self,
+            pattern: List[float],
+            target_length: int
+    ) -> List[float]:
         """
         Adjusts the length of the pattern to match the target length.
 
@@ -77,7 +87,11 @@ def _adjust_pattern_length(self, pattern: List[float], target_length: int) -> Li
             return pattern[:target_length]
         return pattern
 
-    def _crysfml_dict(self, sample_model: SampleModels, experiment: Experiment) -> Dict[str, Union[Experiment, SampleModel]]:
+    def _crysfml_dict(
+            self,
+            sample_model: SampleModels,
+            experiment: Experiment
+    ) -> Dict[str, Union[Experiment, SampleModel]]:
         """
         Converts the sample model and experiment into a dictionary format for Crysfml.
 
@@ -95,7 +109,10 @@ def _crysfml_dict(self, sample_model: SampleModels, experiment: Experiment) -> D
             "experiments": [experiment_dict]
         }
 
-    def _convert_sample_model_to_dict(self, sample_model: SampleModels) -> Dict[str, SampleModel]:
+    def _convert_sample_model_to_dict(
+            self,
+            sample_model: SampleModel
+    ) -> Dict[str, Any]:
         """
         Converts a sample model into a dictionary format.
 
@@ -133,7 +150,10 @@ def _convert_sample_model_to_dict(self, sample_model: SampleModels) -> Dict[str,
 
         return sample_model_dict
 
-    def _convert_experiment_to_dict(self, experiment: Experiment) -> Dict[str, Any]:
+    def _convert_experiment_to_dict(
+            self,
+            experiment: Experiment
+    ) -> Dict[str, Any]:
         """
         Converts an experiment into a dictionary format.
 
@@ -148,8 +168,8 @@ def _convert_experiment_to_dict(self, experiment: Experiment) -> Dict[str, Any]:
         peak = getattr(experiment, "peak", None)
 
         x_data = experiment.datastore.pattern.x
-        two_theta_min = float(x_data.min())
-        two_theta_max = float(x_data.max())
+        twotheta_min = float(x_data.min())
+        twotheta_max = float(x_data.max())
 
         exp_dict = {
             "NPD": {
@@ -163,9 +183,9 @@ def _convert_experiment_to_dict(self, experiment: Experiment) -> Dict[str, Any]:
                 #"_pd_instr_reflex_s_l": peak_asymm.s_l.value if peak_asymm else 0.0,
                 #"_pd_instr_reflex_d_l": peak_asymm.d_l.value if peak_asymm else 0.0,
                 "_pd_meas_2theta_offset": instrument.calib_twotheta_offset.value if instrument else 0.0,
-                "_pd_meas_2theta_range_min": two_theta_min,
-                "_pd_meas_2theta_range_max": two_theta_max,
-                "_pd_meas_2theta_range_inc": (two_theta_max - two_theta_min) / len(x_data)
+                "_pd_meas_2theta_range_min": twotheta_min,
+                "_pd_meas_2theta_range_max": twotheta_max,
+                "_pd_meas_2theta_range_inc": (twotheta_max - twotheta_min) / len(x_data)
             }
         }