Mismatch of data in Elastic and DB on deletes

[pinakipb2]

Describe the bug
When a field in an aspect is updated to empty, DB gets updated, but ES keeps the previous value.
For e.g. there is a field called description in an aspect for some entity. And currently it contains some value, which is present in DB as well as ES. When the description is updated to empty string. DB is updated to empty string and UI shows accordingly, but ES still holds the previous description value. Although the hasDescription field in ES gets updated to false.

To Reproduce
Steps to reproduce the behavior:

1. Create any entity which has a valid description.
2. Update / Ingest the description to be empty string.
3. We can check the value in DB being empty string.
4. But the value of description in ES is still the previous description.

Expected behavior
Value of the description should be empty as well, matching what is in DB.

Desktop (please complete the following information):

• OS: MacOS 14.6.1
• Browser Chrome
• Datahub-Version - 0.13.3

Additional context
As per initial analysis, its found that the document is upserted. So, the previous and current document are merged, hence the description from the previous document gets retained in the process.

[jjoyce0510]

@david-leifker Do you mind taking a look?

[david-leifker]

This will treat empty strings as null for ES indexing -> #11514

[pinakipb2]

@david-leifker - Thanks !