v3.2.0-rc0

What's Changed

• Switch to self-hosted-v2 runner by @fjammes in #745
• Update processing for anomaly detection by @JulienPeloton in #743
• Debug self hosted runner by @fjammes in #749
• Active learning loop by @JulienPeloton in #747
• Add features to HBase tables using full schema (master, anomaly) by @JulienPeloton in #751
• [CI] split Sentinel tests into units, integration (DB), integration (stream) by @JulienPeloton in #755
• Update to Spark 3.4.1 by @JulienPeloton in #759
• Update code dependencies by @JulienPeloton in #764
• Deploy blazar filter by @JulienPeloton in #768
• Run ciux-based CI by @fjammes in #761
• SSO index table by @JulienPeloton in #774
• [HBase] update rowkeys for resolvers by @JulienPeloton in #777
• Upgrade pip before installation of packages by @JulienPeloton in #775
• Add redshift to the TNS table by @JulienPeloton in #780
• 748 add support for argocd by @fjammes in #781
• Replace local crossmatch for VSX with a call to Vizier using the CDS xmatch service by @JulienPeloton in #790
• Improve e2e science, self-hosted test by @fjammes in #796
• Cache only required fields when computing statistics by @JulienPeloton in #792
• Add to HBase the fields generated by the mag_rate science module by @JulienPeloton in #791
• Update Slack channel names by @JulienPeloton in #789
• Add more columns for uppervalid alerts by @JulienPeloton in #785
• Add crossmatch with the SPICY catalog by @JulienPeloton in #803
• Bump fink-science to 5.7.1, with the memory leak fixed by @JulienPeloton in #814
• Push all data at once in science_archival by @JulienPeloton in #815
• 799 avoid image build when not needed by @fjammes in #801
• Rename spicy_name into spicy_class by @JulienPeloton in #813
• Restore per-file push for science archival by @JulienPeloton in #817
• Add new filter for YSO by @JulienPeloton in #822
• Add a new option to control the number of offsets per trigger by @JulienPeloton in #820
• 816 deploy fink broker on fink integration platform by @fjammes in #824
• Use gitlab private registry in CI by @fjammes in #826
• Update integration test documentation by @fjammes in #827
• New filter for dwarf AGN by @JulienPeloton in #833
• add filter criteria by @anaismoller in #831
• Update debug_self_hosted.md by @fjammes in #836
• Active learning loop: sent high probability candidates by @anaismoller in #835
• Improve logging of error when importing science modules by @JulienPeloton in #844
• Update requirements by @JulienPeloton in #834
• Check & format using Ruff by @JulienPeloton in #847
• Discard alerts with i band measurements by @JulienPeloton in #839
• Improve CD process by @fjammes in #861
• 865 incompatible pyarrow version causing issues with parquet file reading in fink science library by @fjammes in #866
• Change partitioning strategy for online processing by @JulienPeloton in #793
• Known TDE by @JulienPeloton in #873
• Add missing argument by @JulienPeloton in #884
• Improve ci/cd design by @fjammes in #876
• Bump actions/download-artifact from 3 to 4.1.7 in /.github/workflows by @dependabot in #882
• Bump requirement versions by @JulienPeloton in #893
• Update SSOFT generation by @JulienPeloton in #880
• hostless: check if the candidate has been reported by @JulienPeloton in #896
• Update thresholds for hostless detection by @JulienPeloton in #895
• Update SSO index table (resolver) by @JulienPeloton in #898
• 899 update integration tests by @fjammes in #901
• fink-mm: various updates by @JulienPeloton in #906
• Include lower bounds when filtering for hostless by @JulienPeloton in #905
• SSHG1G2 by @JulienPeloton in #908
• Add option to install prometheus stack by @fjammes in #911
• Add TNS column by @JulienPeloton in #910
• Set the topic number to 8 for science in the CI by @JulienPeloton in #918
• adding last limit per filter by @anaismoller in #916
• Update model name to activate upper limits by @JulienPeloton in #920
• Stop pushing cutout data to HBase main table by @JulienPeloton in #922
• Add image database service by @JulienPeloton in #926
• Calculating anomaly_score and sending notifications for user models by @Knispel2 in #929
• Various cleaning operations by @JulienPeloton in #928
• Bump requirements for fink-science and fink-utils by @JulienPeloton in #933
• New filter to count incoming alerts by @JulienPeloton in #935
• Update Python dependencies by @JulienPeloton in #941
• Remove the ztf.jd table from HBase by @JulienPeloton in #939
• Catch json decoding issues by @JulienPeloton in #938
• Drop images for the anomaly table by @JulienPeloton in #943
• 913 replace minio s3 with hdfs by @fjammes in #930

New Contributors

• @Knispel2 made their first contribution in #929

Full Changelog: v3.1...v3.2.0-rc0

v3.1

What's Changed

• k8s pipeline by @fjammes in #697
• Update pixel table by @JulienPeloton in #717
• [Hbase] Push data incrementally in science_archival by @JulienPeloton in #721
• 698 k8s create a noscience pipeline by @fjammes in #718
• Bump dependencies by @JulienPeloton in #725
• Use schema converter from fink_utils by @JulienPeloton in #726
• HBase table for TNS object by @JulienPeloton in #728
• Update arguments for rfscore_sigmoid_elasticc by @JulienPeloton in #732
• Add local alerts by @JulienPeloton in #735
• Shift integration tests in k8s by @JulienPeloton in #737
• Add filter for area-restricted anomalies by @JulienPeloton in #740
• Add support for self-hosted runner in CI by @fjammes in #738

Full Changelog: 3.0...v3.1

3.0

What's Changed

• Externalise schemas for the CI by @JulienPeloton in #677
• Add a new function to select only relevant columns when pushing to HBase by @JulienPeloton in #678
• Clean repo by @JulienPeloton in #680
• Build fink-broker image from fink-broker repository by @fjammes in #675
• New filter targeting the Gravitation category in SIMBAD by @JulienPeloton in #687
• Update README by @JulienPeloton in #690
• [WIP] Resolve Kafka failure by @JulienPeloton in #693
• Anomaly detection filter by @JulienPeloton in #696
• Install fink-science from URL by @JulienPeloton in #702
• New arg for the feature extraction module by @JulienPeloton in #706
• Revert feature extraction by @JulienPeloton in #709
• Push anomaly candidates to HBase by @JulienPeloton in #707
• Put fink-FAT operations at the end by @JulienPeloton in #711
• Update the constructor for feature extraction by @JulienPeloton in #714
• Elasticc round two! by @JulienPeloton in #705

New Contributors

• @fjammes made their first contribution in #675

Full Changelog: 2.9...3.0

2.9

What's Changed

• [Kafka] Attach the schema to the payload, in the key by @JulienPeloton in #670
• Update columns to push to HBase tables by @JulienPeloton in #673

Full Changelog: 2.8...2.9

2.8

What's Changed

• Update the path to the simulator by @JulienPeloton in #663
• SNAD processor and anomaly detection by @VoidDruid in #554
• Add crossmatch with 4LAC and 3HSP by @JulienPeloton in #666
• Add T2 layer by @JulienPeloton in #667
• Add mangrove xmatch (and fix t2 syntax) by @JulienPeloton in #668

New Contributors

• @VoidDruid made their first contribution in #554

Full Changelog: 2.7.1...2.8

2.7.1

What's Changed

• online & offline GRB module to join streams by @JulienPeloton in #656
• New connection to ZTF by @JulienPeloton in #658
• Update AGN path by @JulienPeloton in #660

Full Changelog: 2.7...2.7.1

2.7

What's Changed

• Rename fink-fat index column by @JulienPeloton in #648
• [Elasticc] various updates by @JulienPeloton in #651
• [stream2raw] Parametrise the producer by @JulienPeloton in #654

Full Changelog: 2.4...2.7

2.5

What's Changed

• Add new fields when ingesting to hBase by @JulienPeloton in #631
• [Elasticc] Fix the column name by @JulienPeloton in #633
• Bump pyarrow from 0.14.1 to 0.15.1 by @dependabot in #636
• Fix timezone, and date format for elasticc by @JulienPeloton in #637
• Add modules for elastic (fink-science 3.0.0) by @JulienPeloton in #640
• Deploy EarlySN and T2 for elasticc (dummy) by @JulienPeloton in #643
• [Elasticc] various updates by @JulienPeloton in #645
• Change the way binary classifiers are decoded in elasticc by @JulienPeloton in #646

New Contributors

• @dependabot made their first contribution in #636

Full Changelog: 2.2...2.5

2.2

What's Changed

• Bump to 2.1 by @JulienPeloton in #589
• Remove temp files at the end of the database operations by @JulienPeloton in #591
• Add the new filter on YSO by @JulienPeloton in #595
• Stop the broker at 5pm Paris Time (4pm UTC) by @JulienPeloton in #598
• Fix the index for the TNS table to avoid overwriting cells by @JulienPeloton in #599
• Process streams from ELAsTICC by @JulienPeloton in #597
• [Bug] Fix wrong slicing when correlating with TNS catalog by @JulienPeloton in #601
• [Bug] Fix the healpix tables generation by adding a reference to the objectId in the index by @JulienPeloton in #602
• Fink-FAT: scheduler extension by @FusRoman in #605
• Update the conf file for fink-fat by @JulienPeloton in #609
• [HBase] new index tables for SSO candidates by @JulienPeloton in #611
• Fix reader syntax for recent versions of fastavro by @JulienPeloton in #617
• [HBase] refactoring by @JulienPeloton in #614
• [Elasticc] smoother processing by @JulienPeloton in #628

New Contributors

• @FusRoman made their first contribution in #605

Full Changelog: 2.1...2.2

2.1

What's Changed

• Schedule the CI every day-of-week at 5 UTC by @JulienPeloton in #584
• Remove old reference to fink_filters and use fink_utils by @JulienPeloton in #586
• [Prod] Spark 2 to Spark 3 by @JulienPeloton in #588

Full Changelog: 2.0...2.1