vdk-test-utils: Measure payload size with len, not getsizeof (#3157)

Previously, the size of payloads was measured using sys.getsizeof when
testing. This is suboptimal, since different versions of Python might
give different sizes (and in particular, when using Python3.12, the size
of payloads is 8 bytes less than in previous versions). To address this,
we now use len to measure the payload size, which should be consistent
across all Python versions.

The relevant tests in vdk-core have been adjusted to address this
change.

Testing done: cicd

Signed-off-by: Gabriel Georgiev <[email protected]>

projects/vdk-core/tests/functional/ingestion/test_ingest_chain.py

@@ -29,9 +29,9 @@ def get_shared_result_data():
         bool_key=str(True),
         float_key=str(1.23),
         nested=str(dict(key="value")),
-        payload_size=163,
+        payload_size=114,
     )
-    expected_rows_object = {"first": "two", "payload_size": 80, "second": "2"}
+    expected_rows_object = {"first": "two", "payload_size": 31, "second": "2"}
 
     return expected_object, expected_rows_object
 

projects/vdk-plugins/vdk-test-utils/src/vdk/plugin/test_utils/ingest_util_plugins.py

@@ -2,7 +2,6 @@
 # SPDX-License-Identifier: Apache-2.0
 import datetime
 import logging
-import sys
 from dataclasses import dataclass
 from typing import Dict
 from typing import List
@@ -63,7 +62,7 @@ def pre_ingest_process(
         start_time = datetime.datetime(year=2022, month=1, day=27, hour=16)
         metadata = IngestionMetadata({"ingestion_submission_start_time": start_time})
         for i in payload:
-            payload_size = sys.getsizeof(str(i))
+            payload_size = len(str(i))
             i["payload_size"] = payload_size
             processed_payloads.append(i)