Random missing DAG from DagBag

[Harshil-Jani]

Apache Airflow version

Other Airflow 2 version (please specify below)

If "Other Airflow 2 version" selected, which one?

2.10.0

What happened?

There is a DAG which we wrote. It randomly pops up in UI sometimes and goes missing. When we hit that URL it shows DAG missing from DAGBAG. and even if nobody changes anything, It just pops up from somewhere. On logs we have checked that even when the DAG is missing from DagBag it still runs and does it's tasks (we verified this via db and pod). Can we please get some light on why this randomness ?

What you think should happen instead?

Dag should either be visible all the time or stay missing all the time.

How to reproduce

No idea

Operating System

Linux

Versions of Apache Airflow Providers

No response

Deployment

Docker-Compose

Deployment details

No response

Anything else?

No response

Are you willing to submit PR?

• Yes I am willing to submit a PR!

Code of Conduct

• I agree to follow this project's Code of Conduct

[boring-cyborg[bot]]

Thanks for opening your first issue here! Be sure to follow the issue template! If you are willing to raise PR to address this issue please do so, no need to wait for approval.

[potiuk]

Look ad dag parsing log. Very likely it takes a very long time to parse your dag file or it fails to parse it randomly. Airflow continuously re-parses the DAGs and when it cannot parse the DAG for some time - timeouts or similar.

You can see this discussion #44495 (comment) for more detailed explanation of what could go wrong and how to approach it.