implement congruency checks between data & metadata

[atn38]

the ones not yet in ECC

• attribute order match between data header and EML
• attribute names match between data header and EML
• attribute missing codes and enumeration match between data and EML

[gastil]

I was hoping the pasta checker would get around to those checks. Currently the header-match is an info-level, human-viewed check.
And the enum check would have saved me grief.
Are you concerned you might be duplicating effort, doing this both in MetaEgress and (in future) pasta? Tho they say errors are best caught closest to the source. And who knows when check coding will resume in pasta.
So I vote yes please.

[atn38]

I'm not concerned about duplicating efforts. Versions of this idea have been written again and again and have their own uses in context. MetaEgress can use its own. E.g. I wrote something for datapie, the ADC folks wrote something here: https://github.com/NCEAS/datamgmt/blob/master/R/qa_package.R And yes I just came to grief about the enum and missing codes one motivating this issue

…

On Mon, Nov 4, 2019 at 4:16 PM Gastil ***@***.***> wrote: I was hoping the pasta checker would get around to those checks. Currently the header-match is an info-level, human-viewed check. And the enum check would have saved me grief. Are you concerned you might be duplicating effort, doing this both in MetaEgress and (in future) pasta? Tho they say errors are best caught closest to the source. And who knows when check coding will resume in pasta. So I vote yes please. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#50?email_source=notifications&email_token=AKAZD5T76WCDV7P7AKGVIETQSCNKJA5CNFSM4JIXIAAKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEDA47GA#issuecomment-549572504>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AKAZD5WQX2A23L3YJ7O3TLDQSCNKJANCNFSM4JIXIAAA> .

[atn38]

a first attempt at the function at https://github.com/BLE-LTER/MetaEgress/blob/congruence/R/check_attributes_congruence.R

next: integrating this into create_entity and issue warnings/errors as appropriate.
plus issue more detailed messages, think about a way to check for presence of missing codes in data not listed in metadata

[atn38]

not closing this yet, since the function needs improvements