Feature/ascii to netcdf performance optimization#234
Merged
Conversation
Major performance improvements for read_fesom_ascii_grid: - Vectorized barycenter computation (173x faster) - Vectorized coastal element detection (189x faster) - Hash-based neighbor finding (17x faster) - Added Numba JIT compilation support for further speedups Performance results on core2 grid (126K nodes, 244K elements): - Before: 111.5 seconds - After: 87.0 seconds (1.28x speedup achieved) - With full Numba optimization: ~5-10 seconds expected Changes are backward compatible and results are validated to be identical. Optimizations are now the default behavior. Related files: - PERFORMANCE_OPTIMIZATION_REPORT.md: Detailed analysis and recommendations - benchmark_ascii_to_mesh.py: Benchmark script - validate_optimizations.py: Validation tests - ascii_to_netcdf_optimized.py: Standalone optimized implementations
Applied all optimizations from analysis: - Vectorized barycenter computation (exact match with original) - Vectorized coastal element detection (exact match) - Hash-based neighbor finding (17x faster) - Simplified spherical area calculation (avoids 733K rotate() calls) - Inlined stamp polygon midpoint calculations Performance on core2 grid (126K nodes, 244K elements): - Before: 111.5 seconds - After: 20.0 seconds - Speedup: 5.6x All optimizations validated to produce identical results. Numba integration available for further 2-3x improvement.
Enabled Numba JIT compilation for computational hotspots: - Area calculation: parallel execution with prange (3x faster) - Stamp polygon generation: JIT compiled (2x faster) - Barycenter: already vectorized Performance on core2 grid (126K nodes, 244K elements): - Original: 111.5 seconds - NumPy optimizations: 20.0 seconds (5.6x) - With Numba: 10.6 seconds (10.5x total) Falls back gracefully to pure Python if Numba not installed. All results validated to be identical to original.
These notebook modifications were accidentally included in a previous commit. This PR should only contain the ascii_to_netcdf.py optimizations.
koldunovn
approved these changes
Feb 20, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
About a 10x speed up with verified same results.