Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refactoring: towards Fink v4 #948

Merged
merged 38 commits into from
Feb 4, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
38 commits
Select commit Hold shift + click to select a range
e0d99ea
Split ztf and rubin scripts
JulienPeloton Jan 28, 2025
b6e39c5
Rename imports
JulienPeloton Jan 28, 2025
b0aea38
Modify fink launcher
JulienPeloton Jan 28, 2025
5466319
Simplify fink
JulienPeloton Jan 28, 2025
3dfced3
Update test script
JulienPeloton Jan 29, 2025
56538f9
Factorize args into configuration file, and remove unused arguments
JulienPeloton Jan 29, 2025
db51f93
Update fink launcher, and replace producer by survey
JulienPeloton Jan 29, 2025
c563e8b
Update distribution conf by removing unused variable
JulienPeloton Jan 29, 2025
719e577
Use another criterion to switch to simulation mode in stream2raw
JulienPeloton Jan 29, 2025
22866bb
Remove unused bin
JulienPeloton Jan 29, 2025
34ba422
Unique default conf files for ZTF
JulienPeloton Jan 29, 2025
03f9443
Update conf files
JulienPeloton Jan 29, 2025
cdaf422
remove unused file
JulienPeloton Jan 29, 2025
c60729d
Save...
JulienPeloton Jan 29, 2025
22060af
... and remove
JulienPeloton Jan 29, 2025
0f39b02
Update conf file path
JulienPeloton Jan 29, 2025
ecc3913
Update conf
JulienPeloton Jan 29, 2025
5d7a256
Update streaming scheduler
JulienPeloton Jan 29, 2025
a5f6275
Parametrize db operations
JulienPeloton Jan 29, 2025
3d916ac
Rename DB launcher
JulienPeloton Jan 29, 2025
38962a9
Remove unused db script
JulienPeloton Jan 29, 2025
b45df8c
Change path
JulienPeloton Jan 29, 2025
4aea38f
Update README
JulienPeloton Jan 29, 2025
3f4e0cb
Sentinel should work
JulienPeloton Jan 29, 2025
70ae4e9
Ruff
JulienPeloton Jan 29, 2025
2c62e30
Add new survey argument
JulienPeloton Jan 29, 2025
3d36637
Increase resources
JulienPeloton Jan 29, 2025
33e9597
Update path for main scripts in k8s
JulienPeloton Jan 31, 2025
7a11986
Replace the producer name
JulienPeloton Jan 31, 2025
447f5ba
debug
JulienPeloton Jan 31, 2025
c0af67e
k8s workflow uses its own kafka server name
JulienPeloton Jan 31, 2025
3fdc7cb
Debug
JulienPeloton Jan 31, 2025
1003928
debug: kafka sasl
JulienPeloton Jan 31, 2025
a47657a
Add new parameter to control Kaffka producer security protocol
JulienPeloton Feb 3, 2025
e77df74
Add new arg to binary
JulienPeloton Feb 3, 2025
b8e0f6b
kafka: Fix typo in the key name
JulienPeloton Feb 3, 2025
c6dfe55
Disable tmate
JulienPeloton Feb 3, 2025
6253698
Bump to 4.0
JulienPeloton Feb 4, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 4 additions & 4 deletions .github/workflows/linter.yml
Original file line number Diff line number Diff line change
Expand Up @@ -26,9 +26,9 @@ jobs:
pip install ruff
- name: fink-broker
run: |
ruff check --statistics --preview fink_broker/*.py
ruff format --check --preview fink_broker/*.py
ruff check --statistics --preview fink_broker/*/*.py
ruff format --check --preview fink_broker/*/*.py
- name: bin
run: |
ruff check --preview --statistics bin/*.py
ruff format --check --preview bin/*.py
ruff check --preview --statistics bin/*/*.py
ruff format --check --preview bin/*/*.py
4 changes: 2 additions & 2 deletions .github/workflows/test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -59,14 +59,14 @@ jobs:
echo "PYTHONPATH: $PYTHONPATH"
echo "JAVA_HOME: $JAVA_HOME"
echo `python -V`
- name: Run test suite
- name: Run ZTF test suite
run: |
# to remove
export KAFKA_OPTS="-Djava.net.preferIPv4Stack=True"
cd $USRLIBS
source scripts/start_services.sh --kafka-version ${KAFKA_VERSION} --hbase-version ${HBASE_VERSION}
cd $FINK_HOME
fink_test -c conf/fink.conf.dev --stream-integration --db-integration --mm-offline --unit-tests
fink_test -s ztf -c conf/fink.conf.dev --stream-integration --db-integration --mm-offline --unit-tests
curl -s https://codecov.io/bash | bash
- uses: act10ns/slack@v1
with:
Expand Down
64 changes: 0 additions & 64 deletions bin/access_science_db.py

This file was deleted.

70 changes: 0 additions & 70 deletions bin/checkstream.py

This file was deleted.

File renamed without changes.
Loading
Loading