Improve enrichment logic to handle type and datatype mismatches

[srikanthccv]

Currently, our enrichment logic has some limitations that can lead to poor user experience and silent failures. This issue aims to address these concerns and improve the robustness of our enrichment process.

Problem:

1. When a user requests an attribute with a specific type and datatype that doesn't match what's in the database, the current logic continues to search for the exact match, potentially leading to empty results.
2. This behavior can be especially problematic for saved views where metadata might change over time due to third-party instrumentation changes or colleague modifications.
3. The current approach of letting queries fail silently can be frustrating and difficult for users to troubleshoot.

Current behavior:

• If both field.Type and field.DataType are specified and don't match what's in the DB, the query continues with user-provided values, potentially resulting in no data.
• The fields stored in the database are permanent and not bound to time, which can lead to outdated metadata persisting.

Proposed improvements:

1. Implement a fallback mechanism when the requested type and datatype don't exist for an attribute.
2. Provide clear feedback to users when there's a mismatch between requested and available attribute metadata.
3. Consider how to handle cases where attribute metadata changes over time, especially for saved assets such as panels, views etc...
4. Explore options for periodically updating or validating stored attribute metadata.

Questions to consider:

• Should we reject requests with mismatched type/datatype, or attempt to use available metadata?
• How can we improve error messages and suggestions to guide users towards correct attribute usage?
• Should we implement a mechanism to track attribute metadata changes over time?

Next steps:

1. Discuss and agree on the best approach to handle type and datatype mismatches.
2. Design a solution that balances user expectations with data integrity.
3. Implement the chosen solution and update relevant documentation.
4. Add tests to cover various scenarios of type and datatype mismatches.

This improvement will enhance the reliability of our enrichment process and provide a better user experience when dealing with changing or mismatched attribute metadata.