Performance issue when using sr_get_data()

[DDavisZhou]

Hi, Team

I am experiencing a performance issue when using sr_get_data() to retrieve a tree whose root nodes match a provided XPath.
Normally, this operation takes under 0.01 seconds. However, when I use an XPath defined by _3gpp-common-subnetwork.yang, it takes about 0.3 seconds.
Example xpth:
/_3gpp-common-subnetwork:SubNetwork[id='1']/_3gpp-nr-nrm-externalgnbcucpfunction:ExternalGNBCUCPFunction[id='2']/attributes/gNBId

The _3gpp-common-subnetwork.yang file uses yangmnt:mount-point as follows:

yangmnt:mount-point children-of-SubNetwork {
  description "Mountpoint for ManagedElement";
  reference "RFC8528 YANG Schema Mount";
}

I suspect that the use of yangmnt:mount-point might be related to the increased retrieval time.

Could you provide any suggestions or insights into why this might be happening? Are there known performance issues with yangmnt:mount-point, or are there any optimizations I can apply to improve the retrieval time?

Thank you for your assistance.

The related version I used as the follow:

libyang:     3.4.2						
libnetconf2: 3.5.1		
Sysrepo:     2.11.7
netopeer2:   2.2.31	

[michalvasko]

Are there known performance issues with yangmnt:mount-point, or are there any optimizations I can apply to improve the retrieval time?

Yes, the main performance git is when using an inline schema, which needs to create a new libyang context every time it is parsing data. The optimization is to use shared schema, which creates the context only once and then reuses it.

[DDavisZhou]

Yes, the main performance git is when using an inline schema, which needs to create a new libyang context every time it is parsing data. The optimization is to use shared schema, which creates the context only once and then reuses it.

Thank you very much for your response. I have adjusted my database. Since the defined yangmnt:mount-point children-of-SubNetwork is not actually used, mount data as shown below:

root@localhost:~# sysrepocfg -X -x /ietf-yang-schema-mount:schema-mounts/mount-point -d operational
<schema-mounts xmlns="urn:ietf:params:xml:ns:yang:ietf-yang-schema-mount">
  <mount-point>
    <module>_3gpp-common-subnetwork</module>
    <label>children-of-SubNetwork</label>
    <config>true</config>
    <shared-schema>
      <parent-reference>NULL</parent-reference>
      <parent-reference>'parent:/subnet3gpp:SubNetwork'</parent-reference>
    </shared-schema>
  </mount-point>
</schema-mounts>

Unfortunately, the performance issues still persist. Could you please provide some additional suggestions?

[michalvasko]

How exactly are you testing it? When a process parses the data the first time, the context will need to be created whether you use inline or shared schema, the difference should be seen on repeated data parsing.

[DDavisZhou]

How exactly are you testing it?

The data of ietf-yang-schema-mount in the operational database:

<schema-mounts xmlns="urn:ietf:params:xml:ns:yang:ietf-yang-schema-mount">
  <mount-point>
    <module>_3gpp-common-subnetwork</module>
    <label>children-of-SubNetwork</label>
    <config>true</config>
    <shared-schema>
      <parent-reference>NULL</parent-reference>
    </shared-schema>
  </mount-point>
</schema-mounts>

The data of _3gpp-common-subnetwork in the running database:

<SubNetwork xmlns="urn:3gpp:sa5:_3gpp-common-subnetwork">
  <id>1</id>
  <attributes>
    <dnPrefix>C=TW,ST=Taipei,L=Taipei,O=Sercomm,OU=PSIRT,CN=NIRD0011160</dnPrefix>
    <userLabel/>
    <userDefinedNetworkType/>
    <setOfMcc>001</setOfMcc>
    <priorityLabel>0</priorityLabel>
    <parents>1</parents>
    <containedChildren>1</containedChildren>
  </attributes>
  <ExternalENBFunction xmlns="urn:3gpp:sa5:_3gpp-nr-nrm-externalenbfunction">
    <id>1</id>
    <attributes>
      <priorityLabel>0</priorityLabel>
      <eNBId>1</eNBId>
    </attributes>
  </ExternalENBFunction>
  <ExternalGNBCUCPFunction xmlns="urn:3gpp:sa5:_3gpp-nr-nrm-externalgnbcucpfunction">
    <id>1</id>
    <attributes>
      <priorityLabel>0</priorityLabel>
      <gNBId>142</gNBId>
      <gNBIdLength>22</gNBIdLength>
      <pLMNId>
        <mcc>001</mcc>
        <mnc>01</mnc>
      </pLMNId>
    </attributes>
    <ExternalNRCellCU xmlns="urn:3gpp:sa5:_3gpp-nr-nrm-externalnrcellcu">
      <id>1</id>
      <attributes>
        <priorityLabel>0</priorityLabel>
        <cellLocalId>1</cellLocalId>
        <nRPCI>122</nRPCI>
        <pLMNIdList>
          <mcc>001</mcc>
          <mnc>01</mnc>
        </pLMNIdList>
        <pLMNIdList>
          <mcc>311</mcc>
          <mnc>48</mnc>
        </pLMNIdList>
        <nRFrequencyRef>[email protected],DC=example,DC=com CN=John Smith,OU=Sales,O=ACME Limited,L=Moab,ST=Utah,C=US</nRFrequencyRef>
      </attributes>
    </ExternalNRCellCU>
  </ExternalGNBCUCPFunction>
</SubNetwork>

I use the sr_get_data() to retrieve data belonging to the _3gpp-common-subnetwork module, which operation takes about 0.3 seconds.

Test case example：

/_3gpp-common-subnetwork:SubNetwork[id='1']/_3gpp-nr-nrm-externalgnbcucpfunction:ExternalGNBCUCPFunction[id='1']/attributes/gNBId
/_3gpp-common-subnetwork:SubNetwork[id='1']/_3gpp-nr-nrm-externalgnbcucpfunction:ExternalGNBCUCPFunction[id='1']/attributes/gNBIdLength
/_3gpp-common-subnetwork:SubNetwork[id='1']/_3gpp-nr-nrm-externalgnbcucpfunction:ExternalGNBCUCPFunction[id='1']/_3gpp-nr-nrm-externalnrcellcu:ExternalNRCellCU[id='1']/attributes/cellLocalId
/_3gpp-common-subnetwork:SubNetwork[id='1']/_3gpp-nr-nrm-externalgnbcucpfunction:ExternalGNBCUCPFunction[id='1']/_3gpp-nr-nrm-externalnrcellcu:ExternalNRCellCU[id='1']/attributes/nRPCI

I obtained the values of the above four xpaths in sequence through sr_get_data(), and each xpath would take about 0.3 seconds?

I would like to ask
(1) Is my use of mount data correct ？
(2) are there any optimizations I can apply to improve the retrieval time?

Thanks.

[michalvasko]

Firstly, you should remove the line <parent-reference>NULL</parent-reference> from your operational data. This is not C but YANG and NULL is not a valid value for the parent-reference node. But it probably caused no issues. So, if you want me to help you, you will need to provide all the YANG modules with imports so that I can reproduce the test case and analyze where the time is spent. But note that schema-mount was definitely not designed for efficiency so this may be the best we can do.

[DDavisZhou]

Firstly, you should remove the line <parent-reference>NULL</parent-reference> from your operational data. This is not C but YANG and NULL is not a valid value for the parent-reference node.

Thank you very much for your correction

So, if you want me to help you, you will need to provide all the YANG modules with imports so that I can reproduce the test case and analyze where the time is spent.

I have prepare YANG modules with imports, you can get them via https://github.com/DDavisZhou/_3gpp-common-subnetwork-files

But note that schema-mount was definitely not designed for efficiency so this may be the best we can do.

But when I want to get some subnetwork values using XPath, it takes me too much time.

[michalvasko]

It seems you have not included the mounted modules, please do. Have you installed them into sysrepo? Otherwise you would have to provide them in ietf-yang-library operational data together with the schema-mounts data.

[DDavisZhou]

It seems you have not included the mounted modules, please do.

The _3gpp-common-subnetwork.yang file uses yangmnt:mount-point as follows:

yangmnt:mount-point children-of-SubNetwork {
  description "Mountpoint for ManagedElement";
  reference "RFC8528 YANG Schema Mount";
}

But actually , the node of "children-of-SubNetwork" is not in use in my database.
Therefore, I previously set <parent-reference>NULL</parent-reference> as NULL in operational database of ietf-yang-schema-mount which is used to avoid errors caused by importing XML configuration without mount data.
The error caused by importing XML configuration without mount data was mention in CESNET/libyang#2316

Will this affect performance？

[michalvasko]

In your running data you posted before there are data of (likely) modules _3gpp-nr-nrm-externalenbfunction, _3gpp-nr-nrm-externalgnbcucpfunction, and _3gpp-nr-nrm-externalnrcellcu. They are not defined in the YANG schema so they must be mounted, the children-of-SubNetwork is not an actual YANG data node, it is just the mount point (explained in the schema-mount RFC). So yes, in the running data you posted you are parsing mounted data.