Handle OpenAI/Azure content filter failures gracefully in chart history processing task

[raptium]

Problem

Some small fraction of our chart history data triggers OpenAI content filtering (exact fraction unknown, or I can pre-process and remove problematic data beforehand). When this happens, the whole task fails with an error code and does not complete.

Example error:

Error code: 400 - {'error': {'message': "The response was filtered due to the prompt triggering Azure OpenAI's content management policy. Please modify your prompt and retry. To learn more about our content filtering policies please read our documentation: https://go.microsoft.com/fwlink/?linkid=2198766", 'type': None, 'param': 'prompt', 'code': 'content_filter', 'status': 400, 'innererror': {'code': 'ResponsibleAIPolicyViolation', 'content_filter_result': {'hate': {'filtered': False, 'severity': 'safe'}, 'jailbreak': {'filtered': True, 'detected': True}, 'self_harm': {'filtered': False, 'severity': 'safe'}, 'sexual': {'filtered': False, 'severity': 'safe'}, 'violence': {'filtered': False, 'severity': 'safe'}}}}}

Expected behavior

• The task should still finish even if some conversations are invalid or filtered.
• It is acceptable to skip invalid conversations instead of failing the entire task.

Proposed solution

• If a content filter is triggered for a conversation, catch the error and skip that particular conversation.
• Ensure the remaining valid data processes and completes the overall task successfully.
• Log filtered/skipped items for further review.

Additional context

• Current behavior causes complete task failure for a rare fraction of data.
• Content filtering is required by Azure OpenAI, but failure handling should be improved.