fix merge.py

[MatthewSteen]

@gtfierro I got this error when running merge.py. This was after going through the HIL merging process.

Traceback (most recent call last):
  File "merge.py", line 19, in <module>
    G = merge_type_cluster(g1, g2, BLDG, similarity_threshold=0.1)
  File "/home/ubuntu20/.local/lib/python3.8/site-packages/brickschema/merge.py", line 237, in merge_type_cluster
    linked_records, leftover_g1, leftover_g2 = _merge_features(
  File "/home/ubuntu20/.local/lib/python3.8/site-packages/brickschema/merge.py", line 186, in _merge_features
    linked_entities = _unpack_linked_records(linked_records)
  File "/home/ubuntu20/.local/lib/python3.8/site-packages/brickschema/merge.py", line 35, in _unpack_linked_records
    for (e1, e2), _ in linked_records:
ValueError: too many values to unpack (expected 2)

[gtfierro]

@MatthewSteen I rewrote the matching algorithm this morning; it's definitely slower (if you don't provide much initial mapping), but seems to work more reliably.

You can investigate in this branch: https://github.com/NREL/tasty/tree/fixing-up-merge/examples/nrel_pnnl/merge

It will let you give initial seedings to the merge function:

--  ------------------------------------------------------  ----------------------------------------------  --                                                                                                    
 0  Entity[Core_bottom-ZN, "Core_bottom-ZN"]                Entity[HVACZone/1036676, "CORE_BOTTOM"]          0                                                                                                    
 1  Entity[Core_mid-ZN, "Core_mid-ZN"]                      Entity[HVACZone/1036677, "PERIMETER_BOT_ZN_1"]   1                                                                                                    
 2  Entity[Core_top-ZN, "Core_top-ZN"]                      Entity[HVACZone/1036678, "PERIMETER_BOT_ZN_2"]   2                                                                                                    
 3  Entity[Perimeter_bot_ZN_1-ZN, "Perimeter_bot_ZN_1-ZN"]  Entity[HVACZone/1036679, "PERIMETER_BOT_ZN_3"]   3                                                                                                    
 4  Entity[Perimeter_bot_ZN_2-ZN, "Perimeter_bot_ZN_2-ZN"]  Entity[HVACZone/1036680, "PERIMETER_BOT_ZN_4"]   4                                                                                                    
 5  Entity[Perimeter_bot_ZN_3-ZN, "Perimeter_bot_ZN_3-ZN"]  Entity[HVACZone/1036711, "PERIMETER_MID_ZN_1"]   5                                                                                                    
 6  Entity[Perimeter_bot_ZN_4-ZN, "Perimeter_bot_ZN_4-ZN"]  Entity[HVACZone/1036712, "PERIMETER_MID_ZN_4"]   6                                                                                                    
 7  Entity[Perimeter_mid_ZN_1-ZN, "Perimeter_mid_ZN_1-ZN"]  Entity[HVACZone/1036713, "PERIMETER_MID_ZN_3"]   7                                                                                                    
 8  Entity[Perimeter_mid_ZN_2-ZN, "Perimeter_mid_ZN_2-ZN"]  Entity[HVACZone/1036714, "PERIMETER_MID_ZN_2"]   8                                                                                                    
 9  Entity[Perimeter_mid_ZN_3-ZN, "Perimeter_mid_ZN_3-ZN"]  Entity[HVACZone/1036716, "PERIMETER_TOP_ZN_1"]   9                                                                                                    
10  Entity[Perimeter_mid_ZN_4-ZN, "Perimeter_mid_ZN_4-ZN"]  Entity[HVACZone/1036717, "PERIMETER_TOP_ZN_2"]  10                                                                                                    
11  Entity[Perimeter_top_ZN_1-ZN, "Perimeter_top_ZN_1-ZN"]  Entity[HVACZone/1036718, "PERIMETER_TOP_ZN_3"]  11                                                                                                    
12  Entity[Perimeter_top_ZN_2-ZN, "Perimeter_top_ZN_2-ZN"]  Entity[HVACZone/1036719, "PERIMETER_TOP_ZN_4"]  12                                                                                                    
13  Entity[Perimeter_top_ZN_3-ZN, "Perimeter_top_ZN_3-ZN"]  Entity[HVACZone/1049702, "CORE_MID"]            13                                                                                                    
14  Entity[Perimeter_top_ZN_4-ZN, "Perimeter_top_ZN_4-ZN"]  Entity[HVACZone/1049703, "CORE_TOP"]            14                                                                                                    
--  ------------------------------------------------------  ----------------------------------------------  --                                                                                                    
Matching pair? type '3 4' to match row 3 in col1 with row 4 in col2                                                                                                                                               
Type 'done' to finish                                                                                                                                                                                             
> 0 0                                                                                                                                                                                                             
--  --------

Once you provide those, then it will attempt to match the rest up on the similarity of their URI and rdfs:label

[MatthewSteen]

Hey Gabe,

I'm trying to merge PNNL's latest GraphDB model (20230406_statements.ttl.txt), which ran overnight without finishing. Could you take a look?

[gtfierro]

I pushed a small change which seems to fix it? I pushed the new merged.ttl file to the nrel-pnnl branch

[MatthewSteen]

Way faster. Probably took less time than my input. I don't see as much output in the terminal compared to before.

[]
0
------------------------------
Unifying http://example.org/building/Core_bottom-ZN and http://example.org/building/HVACZone/1036676
Unifying http://example.org/building/Core_mid-ZN and http://example.org/building/HVACZone/1049702
Unifying http://example.org/building/Core_top-ZN and http://example.org/building/HVACZone/1049703
Unifying http://example.org/building/Perimeter_bot_ZN_1-ZN and http://example.org/building/HVACZone/1036677
Unifying http://example.org/building/Perimeter_bot_ZN_2-ZN and http://example.org/building/HVACZone/1036678
Unifying http://example.org/building/Perimeter_bot_ZN_3-ZN and http://example.org/building/HVACZone/1036679
Unifying http://example.org/building/Perimeter_bot_ZN_4-ZN and http://example.org/building/HVACZone/1036680
Unifying http://example.org/building/Perimeter_mid_ZN_1-ZN and http://example.org/building/HVACZone/1036711
Unifying http://example.org/building/Perimeter_mid_ZN_2-ZN and http://example.org/building/HVACZone/1036714
Unifying http://example.org/building/Perimeter_mid_ZN_3-ZN and http://example.org/building/HVACZone/1036713
Unifying http://example.org/building/Perimeter_mid_ZN_4-ZN and http://example.org/building/HVACZone/1036712
Unifying http://example.org/building/Perimeter_top_ZN_1-ZN and http://example.org/building/HVACZone/1036716
Unifying http://example.org/building/Perimeter_top_ZN_2-ZN and http://example.org/building/HVACZone/1036717
Unifying http://example.org/building/Perimeter_top_ZN_3-ZN and http://example.org/building/HVACZone/1036718
Unifying http://example.org/building/Perimeter_top_ZN_4-ZN and http://example.org/building/HVACZone/1036719

real    1m17.816s
user    0m11.829s
sys     0m0.132s