Surveying the incoming waveform data from sites

[del42]

Let sites give as much information about their incoming waveform:

• Format
• Channels
• Sampling rate
• Number of leads(if known)
• De-identified or expect de-identification tool from CHoRUS
• Representation of sample and its units
• How gap is handled
• Compression used
• Type of converter used

[del42]

EMORY

Hu Lab

• Format: We have Bedmaster GE system. Original format is STP, converted format is XML then ADIBIN, HDF5
• Channels: --
• Sampling rate: 240hz across all channels
• Number of leads: mostly 12 leads
• De-identified: Yes, de-identified with WFtool
• Representation of sample and its units: Float, (ECG) is millivolts (mV), for others need to check.
• How gap is handled: Gap handled with filler number for continuity
• Compression used: No, due to random access capability for streaming
• Type of converter used:
  https://pypi.org/project/binfilepy/
  https://github.com/hulab-ucsf/wftools
  https://github.com/Ostrich-Emulators/PreVent#formatconverter

Gari Lab

• Format: Bedmaster STP data to XML then WFDB pipeline
• Channels: ECG, PPG, Resp, ABP, and others
• Sampling rate: 240hz across all channels
• Number of leads: 1 to 3 ECG leads (but there are other possible channels of data ... max=32 channels)
• De-identified: De-identified by BMI pipeline
• Representation of sample and its units: 16-bit integer
• How gap is handled: ---- Note: When broken into variable-length segments, the gap was marked only at the top level .hea file (without real data being written to the data file) by a line starting with ~, followed by a number showing how many samples of data were skipped (gap), e.g. ~ 185280, which means 185280 samples were skipped.
• Compression: None
• Type of converter used: --

[bemoody]

MIT

• Format: WFDB
• Channels: ECG, PPG, Resp, ABP, and others
• Sampling rate:
  • 249.88 Hz (ECG)
  • 124.94 Hz (PPG, ABP)
  • 62.47 Hz (Resp)
• Number of leads: 1 to 3 ECG leads
• De-identified: De-identified by the MIMIC team
• Representation of sample and its units: 16-bit integer
• How gap is handled: Record is broken into variable-length segments; some segments can be empty
• Compression: FLAC
• Type of converter used: https://github.com/MIT-LCP/downcast

[wa6gz]

UVA

• Format: Bedmaster STP (GE Monitors) to UVA-format HDF5
• Channels:
  • Waveform: ECG and PPG (typical), multiple pressures (ART, FEM, CVP, ICP, UAC)
  • Numerics: HR and SPO2 (typical), low-resolution pressures, CO2, extracted waveform features (e.g. ST length, PVC)
• Sampling rate:
  • 240 Hz ECG
  • 120 Hz ABP/CVP/ICP
  • 60 Hz PPG
  • 0.5 Hz numerics
• Number of leads(if known): 1 to 3 ECG typically, up to 6
• De-identified or expect de-identification tool from CHoRUS: deidentified by converter below
• Representation of sample and its units: 16-bit integer
• How gap is handled: Sample interval is 2s, signal padded with NAN value to full sample interval, per-channel gaps denoted by consecutive timestamps with greater than 2s difference
• Compression used: gzip
• Type of converter used:
  • Bedmaster STPToolkit
  • https://github.com/Ostrich-Emulators/PreVent

[edilbertoamorim]

• UCSF Format: Philips => WFDB => HDF5 if we can get the converter from UVa to work
• Channels:
  • Waveform: ECG and PPG (typical), multiple pressures (ART, CVP, ICP)
  • Numerics: HR and SPO2 (typical), low-resolution pressures, CO2 (might have more)
• Sampling rate: 62.5, 125, 250, 500 Hz - varies by channel
• Number of leads(if known): 1 to 3 ECG but may have more
• De-identified or expect de-identification tool from CHoRUS: deidentified by UVa converter
• Representation of sample and its units: 16-bit integer
• How gap is handled: not sure in DWC conversion with Philips
• Compression used: none for wfdb gzip for HDF5
• Type of converter used: Philips DWC to WFDB tool; UVa converter

[Chesterguan]

UF:

• Format: original format is CRF, I can convert it to CSV and tested UVA tool to convert it to HDF5
• Channels:
  • Waveform: ECG , ART, FEM, CVP, ICP, UAC
  • Vitals: HR,SPO2, CO2, ST Segment, PVC
• Sampling rate:
  • 0.5 Hz Vitals
  • Various for waveforms, I have full list of each signal
• Number of leads(if known): unknown
• De-identified or expect de-identification tool from CHoRUS: De-identified already
• Representation of sample and its units : 5 digits integer
• How gap is handled: filled with missing value indicator
• Compression used: N/A
• Type of converter used:
  • UVA converter

[RGanta]

-- NCH

• Format: parquet (Philips)
• Channels:
  • Numerics: HR, SPo2, Pulse, Perf, NBP, RR, Co2
  • Waveform: ECG, RESP, ABP/ART, ICP/CVP, Pleth
• Sampling rate:
  • 250 Hz/500 Hz ECG
  • 125 Hz ABP/Pleth
  • 62.5 Hz RESP/Co2
  • 1024 ms sample interval numerics
• Number of leads(if known): up to 6 (ECG)
• De-identified or expect de-identification tool from CHoRUS: deidentified by custom script
• Representation of sample and its units: 16-bit integer (waveforms), float (numerics)
• How gap is handled: waveforms are packaged in 5.12s-long wavesamples packets, with a timestamp for each packet. Gaps are encoded by non-consecutive timestamps
• Compression used: gzip
• Type of converter used:

[dcn2123]

-- Columbia

• Format: Columbia HDF5
• Channels:
o Waveform: ECG, Pleth, ART/ABP, ICP,CVP,Resp
o Numeric: currently in CSV not HDF5 but planning to convert soon: HR, RR, rso2, pbto2, Perf, Spo2, Temp, btbHR, CO2, NBP, ST, Pulse
• Sampling Rate:
o 250 Hz (ECG)
o 125 Hz (ABP/ART, Pleth, ICP)
o 62.5 (RESP)
o 1 Hz (numerics)
• Number of leads: Up to 5 leads
• De-identified or expect de-identification tool from CHoRUS: deidentified by custom script
• Representation of sample and its units: Float, millivolts (mV)
• How gap is handled: We stored raw waveforms along with time, so in the h5 there was no gap handling as such
• Compression used: N/A
• Type of converter used: no conversion, we are just decoding the encrypted waveform and compressing into HDF5

[vijaym01]

Seattle Childrens:

Format: parquet (Philips). The data is stored in database tables so we have the option to export in other formats such as CSV.
Channels:
Numerics: HR, SPo2, Pulse, Perf, NBP, RR, Co2
Waveform: ECG, RESP, ART, CVP, Pleth
Sampling rate:
250 Hz/500 Hz ECG
125 Hz ABP/Pleth
62.5 Hz RESP/Co2
1024 ms sample interval numerics
Number of leads(if known): up to 6 (ECG)
De-identified or expect de-identification tool from CHoRUS: deidentified by custom script
Representation of sample and its units: 16-bit integer (waveforms), float (numerics)
How gap is handled: waveforms are packaged in 5.12s-long wavesamples packets, with a timestamp for each packet. Gaps are encoded by non-consecutive timestamps
Compression used: N/A
Type of converter used:

[amg369]

MGH

Format: Bedmaster HDF5
Channels: ECG, PPG, Resp, ABP, ICP, and others
Sampling rate: Varies between 240 Hz, 120 Hz, and there may be others.
Number of leads(if known): Up to 6 generally
De-identified or expect de-identification tool from CHoRUS: As far as I understand, the main source of identification for our waveforms is the filename, which is the PatientEncounterID. I'm unaware of any other source of PHI.
Representation of sample and its units: Float, and ECG is mV, ICP is mmHG. Others may differ.
How gap is handled: HDF5 files will be raw, so gap is not handled.
Compression used: None
Type of converter used: None

[clermontg]

Pitt:

Our data is h5 and csv's.

We collect the following:
A. For numerics
Label|Total_hours|Mean_hours|NumFINs
ABPd|54251.643|96.1908564|564
ABPm|57052.621|97.3594215|586
ABPs|54251.643|96.1908564|564
ARTd|192240.782|95.5471083|2012
ARTm|199926.181|97.1458605|2058
ARTs|192311.765|95.5823882|2012
awRR|6553.478|84.0189487|78
BIS|172.075|24.5821429|7
btbHR|710358.392|116.4712891|6099
C.I.|1852.627|57.8945938|32
C.O.|1731.043|55.8400968|31
CPP|1725.572|95.8651111|18
CVPm|10010.788|57.8658266|173
eeFlow|0.313|0.3130000|1
EMG|172.075|24.5821429|7
etCOâ‚‚|7382.675|87.8889881|84
HR|801318.723|106.7713155|7505
ICPm|14125.858|135.8255577|104
InsTi|0.313|0.3130000|1
IPI|6.872|6.8720000|1
ΔQTc|24466.165|86.4528799|283
LAPm|261.753|87.2510000|3
NBPd|676896.589|116.8070041|5795
NBPm|678305.436|117.0501184|5795
NBPs|676885.232|116.8050443|5795
PAPd|2883.205|54.4000943|53
PAPm|3212.213|56.3546140|57
PAPs|2883.205|54.4000943|53
PAVsup|0.313|0.3130000|1
PAWP|927.891|115.9863750|8
Pd|2.382|1.1910000|2
PEEP|0.313|0.3130000|1
Perf|708528.000|117.3448162|6038
Perf l|5245.622|74.9374571|70
PIP|0.313|0.3130000|1
Pm|2.443|1.2215000|2
Ps|2.382|1.1910000|2
Pulse (ABP)|56480.174|97.2120034|581
Pulse (ART)|196992.020|96.3286161|2045
Pulse (NBP)|668328.553|115.4679601|5788
Pulse (P)|2.442|1.2210000|2
Pulse (PAP)|3204.051|57.2151964|56
Pulse (SpOâ‚‚)|702799.363|117.1918231|5997
Pulse (UAP)|19128.523|273.2646143|70
Pulse(SpOâ‚‚l)|5211.073|75.5227971|69
Pulse(SpOâ‚‚T)|3686.186|43.3668941|85
PVC|799835.073|106.6162454|7502
QT|24466.565|86.4542933|283
QT-HR|24466.564|86.4542898|283
QTc|24466.565|86.4542933|283
RR|790928.972|106.5511211|7423
SEF|90.319|90.3190000|1
SpOâ‚‚|708530.590|117.3452451|6038
SpOâ‚‚l|5245.622|74.9374571|70
SpOâ‚‚T|97808.310|58.6029419|1669
SQI (BIS)|172.075|24.5821429|7
SR|172.075|24.5821429|7
ST-aVF|740336.420|103.4422831|7157
ST-aVL|740336.488|103.4422926|7157
ST-aVR|740336.421|103.4422832|7157
ST-I|740732.878|103.4832185|7158
ST-II|742680.889|103.4662704|7178
ST-III|740708.098|103.4942152|7157
ST-MCL|85010.400|53.7700190|1581
ST-V|733602.240|103.8067412|7067
ST-V1|5620.663|38.4976918|146
ST-V2|5620.663|38.4976918|146
ST-V3|5620.663|38.4976918|146
ST-V4|5620.663|38.4976918|146
ST-V5|5620.663|38.4976918|146
ST-V6|5620.663|38.4976918|146
STindx|5709.969|38.8433265|147
Tblood|1898.467|59.3270938|32
Tcore|338.266|37.5851111|9
Temp|79787.939|96.7126533|825
Tesoph|516.264|51.6264000|10
Tin/Tt|0.313|0.3130000|1
TP|90.319|90.3190000|1
Trect|599.754|54.5230909|11
Tskin|181.312|60.4373333|3
Tven|170.252|170.2520000|1
TVexp|0.313|0.3130000|1
TVin|0.313|0.3130000|1
UAPm|19985.862|285.5123143|70
Unknwn|8.361|1.1944286|7
UVPm|636.142|63.6142000|10
VC|0.313|0.3130000|1

The last column is the number of encounters (out of 8400) with data in the labels

B. For waveforms
Label|Total_hours|Mean_hours|NumFINs
ABP|59345.851|95.1055304|624
AI|7117.385|41.8669706|170
ART|217709.320|97.5400179|2232
AS|7048.107|42.2042335|167
aVF|12495.240|87.3793007|143
aVL|12363.723|98.1247857|126
aVR|767920.056|114.7348059|6693
COâ‚‚|7566.265|87.9798256|86
CVP|10554.078|56.1387128|188
EEG|189.610|27.0871429|7
ES|7132.333|42.2031538|169
I|319665.682|132.3667420|2415
ICP|14768.655|134.2605000|110
II|888392.638|107.4235354|8270
III|420367.373|95.9742861|4380
LAP|261.627|87.2090000|3
MCL|108170.926|58.1251617|1861
nolabel|46735.321|147.4300347|317
P|2.441|1.2205000|2
PAP|3360.806|55.0951803|61
Pleth|785273.528|117.7674757|6668
PLETHl|6213.322|82.8442933|75
PlethT|50238.787|63.5130051|791
RAP|73.213|73.2130000|1
Resp|882419.069|107.7303222|8191
UAP|20146.977|272.2564459|74
UVP|636.007|63.6007000|10
V|878471.348|107.7086008|8156
V1|7124.601|39.3624365|181
V2|10.904|5.4520000|2
V3|16.262|16.2620000|1
V5|10.423|10.4230000|1

Only lead II is 512Hz.
Other leads are 256Hz
Pleth and resp are 128Hz (I need to doublecheck that)
Nothing is upsampled/downsampled

C. We also collect enumerations and alarms - no defined sampling rate

A single file per patient
Signals are appended to each other, so gaps are not handled other than carrying their specific collection time.
We have a "csv2h5" converter which also scales each signal to actual units (when applicable)
Nothing