Skip to content

JSON diff has unreasonable threshold for big diff #70

@jmarrec

Description

@jmarrec

As stated on NatLabRockies/EnergyPlus#8833 (comment)

The json big diffs: I think the epregression script is unreasonable: https://github.com/NREL/EnergyPlusRegressionTool/blob/237b8a38c280fd0ce6563b6849ae51376e326d66/epregressions/runtests.py#L630-L643

It assumes an absolute difference > 0.0001 is a big diff. This for eg:

image

It's categorized as a big diff, but in fact it's a 1e-12 difference in %...

In [1]: f_1 = 30310597.103115804

In [2]: f_2 = 30310597.102913737

In [3]: abs(f_1 - f_2) > 0.00001
Out[6]: True

In [7]: (f_1 - f_2) / f_1
Out[7]: 6.6665528125937005e-12

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions