Skip to content

Commit b215f1e

Browse files
jonhealy1jonhealy1
authored
Create shared sfeos helpers module (#376)
**Related Issue(s):** - #369 **Description:** - This pr moves shared elasticsearch/ opensearch logic out of `core` and into the `sfeos_helpers` module. - Other code duplicated in es/ os `database_logic` files has been moved into the `sfeos_helpers` module as well. - The readme has been updated/ reorganized and a table of contents has been added. **PR Checklist:** - [x] Code is formatted and linted (run `pre-commit run --all-files`) - [x] Tests pass (run `make test`) - [x] Documentation has been updated to reflect changes, if applicable - [x] Changes are added to the changelog
1 parent f2d4807 commit b215f1e

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

58 files changed

+1982
-1428
lines changed

‎.github/workflows/cicd.yml

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -96,6 +96,10 @@ jobs:
9696
run: |
9797
pip install ./stac_fastapi/core
9898
99+
- name: Install helpers library stac-fastapi
100+
run: |
101+
pip install ./stac_fastapi/sfeos_helpers
102+
99103
- name: Install elasticsearch stac-fastapi
100104
run: |
101105
pip install ./stac_fastapi/elasticsearch[dev,server]

‎.github/workflows/publish.yml

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -35,6 +35,18 @@ jobs:
3535
# Publish to PyPI
3636
twine upload dist/*
3737
38+
- name: Build and publish sfeos_helpers
39+
working-directory: stac_fastapi/sfeos_helpers
40+
env:
41+
TWINE_USERNAME: "__token__"
42+
TWINE_PASSWORD: ${{ secrets.PYPI_API_TOKEN }}
43+
run: |
44+
# Build package
45+
python setup.py sdist bdist_wheel
46+
47+
# Publish to PyPI
48+
twine upload dist/*
49+
3850
- name: Build and publish stac-fastapi-elasticsearch
3951
working-directory: stac_fastapi/elasticsearch
4052
env:

‎CHANGELOG.md

Lines changed: 13 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -5,24 +5,33 @@ All notable changes to this project will be documented in this file.
55
The format is based on [Keep a Changelog](http://keepachangelog.com/en/1.0.0/)
66
and this project adheres to [Semantic Versioning](http://semver.org/spec/v2.0.0.html).
77

8+
89
## [Unreleased]
910

11+
## [v5.0.0a0] - 2025-05-29
12+
1013
### Added
1114

15+
- Created new `sfeos_helpers` package to improve code organization and maintainability [#376](https://github.com/stac-utils/stac-fastapi-elasticsearch-opensearch/pull/376)
1216
- Added introduction section - What is stac-fastapi-elasticsearch-opensearch? - to README [#384](https://github.com/stac-utils/stac-fastapi-elasticsearch-opensearch/pull/384)
1317

1418
### Changed
1519

20+
- Refactored utility functions into dedicated modules within `sfeos_helpers` [#376](https://github.com/stac-utils/stac-fastapi-elasticsearch-opensearch/pull/376):
21+
- Created `database` package with specialized modules for index, document, and utility operations
22+
- Created `aggregation` package for Elasticsearch/OpenSearch-specific aggregation functionality
23+
- Moved shared logic from core module to helper functions for better code reuse
24+
- Separated utility functions from constant mappings for clearer code organization
25+
- Updated documentation to reflect recent code refactoring [#376](https://github.com/stac-utils/stac-fastapi-elasticsearch-opensearch/pull/376)
1626
- Improved README documentation with consistent formatting and enhanced sections [#381](https://github.com/stac-utils/stac-fastapi-elasticsearch-opensearch/pull/381):
1727
- Added sfeos logo and banner
1828
- Added a comprehensive Quick Start guide
1929
- Reorganized sections for better navigation
2030
- Reformatted content with bullet points for improved readability
2131
- Added more detailed examples for API interaction
22-
32+
2333
### Fixed
2434

25-
2635
## [v4.2.0] - 2025-05-15
2736

2837
### Added
@@ -397,7 +406,8 @@ and this project adheres to [Semantic Versioning](http://semver.org/spec/v2.0.0.
397406
- Use genexp in execute_search and get_all_collections to return results.
398407
- Added db_to_stac serializer to item_collection method in core.py.
399408

400-
[Unreleased]: https://github.com/stac-utils/stac-fastapi-elasticsearch-opensearch/compare/v4.2.0...main
409+
[Unreleased]: https://github.com/stac-utils/stac-fastapi-elasticsearch-opensearch/compare/v5.0.0a0...main
410+
[v5.0.0a0]: https://github.com/stac-utils/stac-fastapi-elasticsearch-opensearch/compare/v4.2.0...v5.0.0a0
401411
[v4.2.0]: https://github.com/stac-utils/stac-fastapi-elasticsearch-opensearch/compare/v4.1.0...v4.2.0
402412
[v4.1.0]: https://github.com/stac-utils/stac-fastapi-elasticsearch-opensearch/compare/v4.0.0...v4.1.0
403413
[v4.0.0]: https://github.com/stac-utils/stac-fastapi-elasticsearch-opensearch/compare/v3.2.5...v4.0.0

‎Makefile

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -95,7 +95,8 @@ pybase-install:
9595
pip install -e ./stac_fastapi/api[dev] && \
9696
pip install -e ./stac_fastapi/types[dev] && \
9797
pip install -e ./stac_fastapi/extensions[dev] && \
98-
pip install -e ./stac_fastapi/core
98+
pip install -e ./stac_fastapi/core && \
99+
pip install -e ./stac_fastapi/sfeos_helpers
99100

100101
.PHONY: install-es
101102
install-es: pybase-install

‎README.md

Lines changed: 19 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -3,7 +3,6 @@
33
<!-- markdownlint-disable MD033 MD041 -->
44

55

6-
76
<p align="left">
87
<img src="https://raw.githubusercontent.com/stac-utils/stac-fastapi-elasticsearch-opensearch/refs/heads/main/assets/sfeos.png" width=1000>
98
</p>
@@ -68,6 +67,7 @@ This project is built on the following technologies: STAC, stac-fastapi, FastAPI
6867
## Table of Contents
6968

7069
- [Documentation & Resources](#documentation--resources)
70+
- [Package Structure](#package-structure)
7171
- [Examples](#examples)
7272
- [Performance](#performance)
7373
- [Quick Start](#quick-start)
@@ -95,6 +95,21 @@ This project is built on the following technologies: STAC, stac-fastapi, FastAPI
9595
- [Gitter Chat](https://app.gitter.im/#/room/#stac-fastapi-elasticsearch_community:gitter.im) - For real-time discussions
9696
- [GitHub Discussions](https://github.com/stac-utils/stac-fastapi-elasticsearch-opensearch/discussions) - For longer-form questions and answers
9797

98+
## Package Structure
99+
100+
This project is organized into several packages, each with a specific purpose:
101+
102+
- **stac_fastapi_core**: Core functionality that's database-agnostic, including API models, extensions, and shared utilities. This package provides the foundation for building STAC API implementations with any database backend. See [stac-fastapi-mongo](https://github.com/Healy-Hyperspatial/stac-fastapi-mongo) for a working example.
103+
104+
- **sfeos_helpers**: Shared helper functions and utilities used by both the Elasticsearch and OpenSearch backends. This package includes:
105+
- `database`: Specialized modules for index, document, and database utility operations
106+
- `aggregation`: Elasticsearch/OpenSearch-specific aggregation functionality
107+
- Shared logic and utilities that improve code reuse between backends
108+
109+
- **stac_fastapi_elasticsearch**: Complete implementation of the STAC API using Elasticsearch as the backend database. This package depends on both `stac_fastapi_core` and `sfeos_helpers`.
110+
-
111+
- **stac_fastapi_opensearch**: Complete implementation of the STAC API using OpenSearch as the backend database. This package depends on both `stac_fastapi_core` and `sfeos_helpers`.
112+
98113
## Examples
99114

100115
The `/examples` directory contains several useful examples and reference implementations:
@@ -181,6 +196,7 @@ There are two main ways to run the API locally:
181196
- **Compatibility**: The most recent Elasticsearch 7.x versions should also work. See the [opensearch-py docs](https://github.com/opensearch-project/opensearch-py/blob/main/COMPATIBILITY.md) for compatibility information.
182197

183198

199+
184200
## Configuration Reference
185201

186202
You can customize additional settings in your `.env` file:
@@ -518,5 +534,6 @@ You can customize additional settings in your `.env` file:
518534
- Limits each client to a specified number of requests per time period (e.g., 500 requests per minute)
519535
- Helps prevent API abuse and maintains system stability
520536
- Ensures fair resource allocation among all clients
521-
537+
522538
- **Examples**: Implementation examples are available in the [examples/rate_limit](examples/rate_limit) directory.
539+

‎compose.yml

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -9,7 +9,7 @@ services:
99
environment:
1010
- STAC_FASTAPI_TITLE=stac-fastapi-elasticsearch
1111
- STAC_FASTAPI_DESCRIPTION=A STAC FastAPI with an Elasticsearch backend
12-
- STAC_FASTAPI_VERSION=4.2.0
12+
- STAC_FASTAPI_VERSION=5.0.0a0
1313
- STAC_FASTAPI_LANDING_PAGE_ID=stac-fastapi-elasticsearch
1414
- APP_HOST=0.0.0.0
1515
- APP_PORT=8080
@@ -42,7 +42,7 @@ services:
4242
environment:
4343
- STAC_FASTAPI_TITLE=stac-fastapi-opensearch
4444
- STAC_FASTAPI_DESCRIPTION=A STAC FastAPI with an Opensearch backend
45-
- STAC_FASTAPI_VERSION=4.2.0
45+
- STAC_FASTAPI_VERSION=5.0.0a0
4646
- STAC_FASTAPI_LANDING_PAGE_ID=stac-fastapi-opensearch
4747
- APP_HOST=0.0.0.0
4848
- APP_PORT=8082

‎dockerfiles/Dockerfile.ci.es

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -12,6 +12,7 @@ RUN apt-get update && \
1212
COPY . /app/
1313

1414
RUN pip3 install --no-cache-dir -e ./stac_fastapi/core && \
15+
pip3 install --no-cache-dir -e ./stac_fastapi/sfeos_helpers && \
1516
pip3 install --no-cache-dir ./stac_fastapi/elasticsearch[server]
1617

1718
USER root

‎dockerfiles/Dockerfile.ci.os

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -12,6 +12,7 @@ RUN apt-get update && \
1212
COPY . /app/
1313

1414
RUN pip3 install --no-cache-dir -e ./stac_fastapi/core && \
15+
pip3 install --no-cache-dir -e ./stac_fastapi/sfeos_helpers && \
1516
pip3 install --no-cache-dir ./stac_fastapi/opensearch[server]
1617

1718
USER root

‎dockerfiles/Dockerfile.deploy.es

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -13,6 +13,7 @@ WORKDIR /app
1313
COPY . /app
1414

1515
RUN pip install --no-cache-dir -e ./stac_fastapi/core
16+
RUN pip install --no-cache-dir -e ./stac_fastapi/sfeos_helpers
1617
RUN pip install --no-cache-dir ./stac_fastapi/elasticsearch[server]
1718

1819
EXPOSE 8080

‎dockerfiles/Dockerfile.deploy.os

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -13,6 +13,7 @@ WORKDIR /app
1313
COPY . /app
1414

1515
RUN pip install --no-cache-dir -e ./stac_fastapi/core
16+
RUN pip install --no-cache-dir -e ./stac_fastapi/sfeos_helpers
1617
RUN pip install --no-cache-dir ./stac_fastapi/opensearch[server]
1718

1819
EXPOSE 8080

‎dockerfiles/Dockerfile.dev.es

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -16,4 +16,5 @@ WORKDIR /app
1616
COPY . /app
1717

1818
RUN pip install --no-cache-dir -e ./stac_fastapi/core
19+
RUN pip install --no-cache-dir -e ./stac_fastapi/sfeos_helpers
1920
RUN pip install --no-cache-dir -e ./stac_fastapi/elasticsearch[dev,server]

‎dockerfiles/Dockerfile.dev.os

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -16,4 +16,5 @@ WORKDIR /app
1616
COPY . /app
1717

1818
RUN pip install --no-cache-dir -e ./stac_fastapi/core
19+
RUN pip install --no-cache-dir -e ./stac_fastapi/sfeos_helpers
1920
RUN pip install --no-cache-dir -e ./stac_fastapi/opensearch[dev,server]

‎dockerfiles/Dockerfile.docs

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -12,6 +12,7 @@ WORKDIR /opt/src
1212

1313
RUN python -m pip install \
1414
stac_fastapi/core \
15+
stac_fastapi/sfeos_helpers \
1516
stac_fastapi/elasticsearch \
1617
stac_fastapi/opensearch
1718

‎examples/auth/compose.basic_auth.yml

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -9,7 +9,7 @@ services:
99
environment:
1010
- STAC_FASTAPI_TITLE=stac-fastapi-elasticsearch
1111
- STAC_FASTAPI_DESCRIPTION=A STAC FastAPI with an Elasticsearch backend
12-
- STAC_FASTAPI_VERSION=4.2.0
12+
- STAC_FASTAPI_VERSION=5.0.0a0
1313
- STAC_FASTAPI_LANDING_PAGE_ID=stac-fastapi-elasticsearch
1414
- APP_HOST=0.0.0.0
1515
- APP_PORT=8080
@@ -43,7 +43,7 @@ services:
4343
environment:
4444
- STAC_FASTAPI_TITLE=stac-fastapi-opensearch
4545
- STAC_FASTAPI_DESCRIPTION=A STAC FastAPI with an Opensearch backend
46-
- STAC_FASTAPI_VERSION=4.2.0
46+
- STAC_FASTAPI_VERSION=5.0.0a0
4747
- STAC_FASTAPI_LANDING_PAGE_ID=stac-fastapi-opensearch
4848
- APP_HOST=0.0.0.0
4949
- APP_PORT=8082

‎examples/auth/compose.oauth2.yml

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -9,7 +9,7 @@ services:
99
environment:
1010
- STAC_FASTAPI_TITLE=stac-fastapi-elasticsearch
1111
- STAC_FASTAPI_DESCRIPTION=A STAC FastAPI with an Elasticsearch backend
12-
- STAC_FASTAPI_VERSION=4.2.0
12+
- STAC_FASTAPI_VERSION=5.0.0a0
1313
- STAC_FASTAPI_LANDING_PAGE_ID=stac-fastapi-elasticsearch
1414
- APP_HOST=0.0.0.0
1515
- APP_PORT=8080
@@ -44,7 +44,7 @@ services:
4444
environment:
4545
- STAC_FASTAPI_TITLE=stac-fastapi-opensearch
4646
- STAC_FASTAPI_DESCRIPTION=A STAC FastAPI with an Opensearch backend
47-
- STAC_FASTAPI_VERSION=4.2.0
47+
- STAC_FASTAPI_VERSION=5.0.0a0
4848
- STAC_FASTAPI_LANDING_PAGE_ID=stac-fastapi-opensearch
4949
- APP_HOST=0.0.0.0
5050
- APP_PORT=8082

‎examples/auth/compose.route_dependencies.yml

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -9,7 +9,7 @@ services:
99
environment:
1010
- STAC_FASTAPI_TITLE=stac-fastapi-elasticsearch
1111
- STAC_FASTAPI_DESCRIPTION=A STAC FastAPI with an Elasticsearch backend
12-
- STAC_FASTAPI_VERSION=4.2.0
12+
- STAC_FASTAPI_VERSION=5.0.0a0
1313
- STAC_FASTAPI_LANDING_PAGE_ID=stac-fastapi-elasticsearch
1414
- APP_HOST=0.0.0.0
1515
- APP_PORT=8080
@@ -43,7 +43,7 @@ services:
4343
environment:
4444
- STAC_FASTAPI_TITLE=stac-fastapi-opensearch
4545
- STAC_FASTAPI_DESCRIPTION=A STAC FastAPI with an Opensearch backend
46-
- STAC_FASTAPI_VERSION=4.2.0
46+
- STAC_FASTAPI_VERSION=5.0.0a0
4747
- STAC_FASTAPI_LANDING_PAGE_ID=stac-fastapi-opensearch
4848
- APP_HOST=0.0.0.0
4949
- APP_PORT=8082

‎examples/rate_limit/compose.rate_limit.yml

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -9,7 +9,7 @@ services:
99
environment:
1010
- STAC_FASTAPI_TITLE=stac-fastapi-elasticsearch
1111
- STAC_FASTAPI_DESCRIPTION=A STAC FastAPI with an Elasticsearch backend
12-
- STAC_FASTAPI_VERSION=4.2.0
12+
- STAC_FASTAPI_VERSION=5.0.0a0
1313
- STAC_FASTAPI_LANDING_PAGE_ID=stac-fastapi-elasticsearch
1414
- APP_HOST=0.0.0.0
1515
- APP_PORT=8080
@@ -43,7 +43,7 @@ services:
4343
environment:
4444
- STAC_FASTAPI_TITLE=stac-fastapi-opensearch
4545
- STAC_FASTAPI_DESCRIPTION=A STAC FastAPI with an Opensearch backend
46-
- STAC_FASTAPI_VERSION=4.2.0
46+
- STAC_FASTAPI_VERSION=5.0.0a0
4747
- APP_HOST=0.0.0.0
4848
- APP_PORT=8082
4949
- RELOAD=true

0 commit comments

Comments
 (0)