Potencial bug in CGMS2DayByDay implementation

[staskh]

While developing IGLU-PYTHON translation, I discovered a few discrepancies in CGMS2DayByDay. Unfortunately, my knowledge of R is quite limited, so I beg your pardon for using Pythonic test cases to illustrate my finding:

Shifting one sample to the left

data = pd.DataFrame({
    'id': ['subject1', 'subject1', 'subject1', 'subject1'],
    'time': pd.to_datetime([
        '2020-01-01 00:00:00',  # 0 min
        '2020-01-01 00:05:00',  # 5 min
        '2020-01-01 00:10:00',  # 10 min
        '2020-01-01 00:15:00',  # 15 min
    ]),
    'gl': [150, 155, 160, 165]
})

r_result = my_CGMS2DayByDay(data)         # calls R imlementation with iglu_py.bridge

gd2d = r_result['gd2d']
actual_dates = r_result['actual_dates']

instead of expected [[150. 155. 160. 165. nan,.....]], gd2d returns [[155. 160. 165. nan nan]... - loosing sample associated with timestamp '00:00:00', while shifting all samples left on time line

IMHO, the problem is line 209 where daily time clock starts at 5min (or dt0 value) instead of proper 0.
IMHO, the correct code has to be

  dti = rep(dt0, (ndays * 24 * 60 /dt0)-1)
  dti_cum = c(0,cumsum(dti))

Actual Dates sometimes have one day more than required

In the sample above, it is expected that actual_dates returns a single date of '2020-01-01',
However, actual_dates returns 2 dates : ['2020-01-01',''2020-01-02']. The similar issue is propagated into gd2d shape: instead of expected (1,288), it is returned as (2,288)
Imho, the problem is in Line 208, which has to be something like:

  ndays = difftime(as.Date(max(tr)), as.Date(min(tr)), units = "days") + 1

use of tz=UTC shift timeline to one day

With example above, my_CGMS2DayByDay(data,tz="UTC") will return active_dates as ['2019-12-31', '2020-01-01'] and associate data samples with early hours of '2019-12-31' , instead of late evening ( I'm running this code in UTC+3). I didn't make an analysis of root cause, but IMHO this issue can be critical for day/night analysis of glucose trends.

Please see example at https://github.com/staskh/iglu_python/blob/main/iglu_r_discrepancies.ipynb

Regards
Stas

[irinagain]

Thank you for bringing this to our attention! I have other deadlines in the queue to get this looked at right away, but it was on our list to rewamp the processing function anyway (to make it more consistent across metrics), so we would check these things among the list and get back, I anticipate in July