Refactor mbon stats

[ocefpaf]

Start addressing #79

[ocefpaf]

@MathewBiddle we need to merge #80 first and I'd like to ask you a few clarifications before moving forward with this one. So far all I did was to reduce the amount for for-loops but the data changed in subtle ways that I'm not sure they are related to the code change. I believe it may due too the time of the query.

[MathewBiddle]

Where does this stand? Is there more work to be done before a review?

[ocefpaf]

Let's close and reopen to toggle the CIs. If it still pass, I'd like to check if the changed data is expected, due to a later run, or if my changes are responsible for it. Hard for me to say b/c I'm not familiar with MBON.

[MathewBiddle]

closed and reopened to trigger checks

[MathewBiddle]

@ocefpaf this now passes all checks. Are we good to merge?

[ocefpaf]

@ocefpaf this now passes all checks. Are we good to merge?

IMO, yes. It would be nice if you, or someone more familiar with mbon, could run it and comment if the results make sense. At the time I recall we had almost 100% comparison, with the exception of https://github.com/ioos/ioos_metrics/pull/81/files#diff-c1e3298f59885519b1b10f023299eae49fedc0616aaf818f2155ed5a11a5371bR607

[MathewBiddle]

Something isn't right with the dates, the data only goes back to 2023-04.

I'm testing using the code in https://github.com/ioos/ioos_metrics/blob/main/notebooks/mbon_citation_visualizations.ipynb

[MathewBiddle]

The yearly totals plots match:

But, the total citations via GBIF don't match:

df = mbon_stats()
print(f"{len(df.drop_duplicates(subset='doi',keep='first')[['doi']])} dataset DOI's at GBIF")
print(f"{len(df)} total citations via GBIF")

41 dataset DOI's at GBIF
41 total citations via GBIF

[ocefpaf]

Thanks! Now I know were to start looking for code differences and try to fix this.

[MathewBiddle]

Here's what the numbers should be:

41 dataset DOI's at GBIF
2920 total citations via GBIF

OBIS downloads:

      downloads
year           
2018         27
2019       7312
2020      87692
2021     457248
2022     213969
2023     221396
2024     430666
2025      16494

OBIS: 1434804 downloads

GBIF Downloads:

      number_downloads
year                  
2016              3369
2017             30080
2018             48361
2019             29966
2020             41590
2021             51336
2022             76654
2023             97296
2024            110224

GBIF: 488876 downloads