Asset backfill fails to check for complete status of runs without output

[Zan-L]

What's the issue?

The current logic for determining whether a backfill has completed is flawed for assets with optional output, because only materialized and failed runs are considered complete:

dagster/python_modules/dagster/dagster/_core/execution/asset_backfill.py

Lines 177 to 182 in 701562d

	return (
	(
	self.materialized_subset | self.failed_and_downstream_subset
	).num_partitions_and_non_partitioned_assets
	== self.target_subset.num_partitions_and_non_partitioned_assets
	)

This issue has two consequences:

1. Backfills are stuck in in progress from the UI
2. Automation Condition backfill_in_progress() evaluates to true and blocks downstream automation

What did you expect to happen?

Change the logic from getting all that is materialized to all that has RUN_SUCCESS events for this function:

dagster/python_modules/dagster/dagster/_core/execution/asset_backfill.py

Line 1301 in 701562d

def get_asset_backfill_iteration_materialized_partitions(

How to reproduce?

No response

Dagster version

1.9.6

Deployment type

None

Deployment details

No response

Additional information

No response

Message from the maintainers

Impacted by this issue? Give it a 👍! We factor engagement into prioritization.

[stigmarl]

I've been seeing something similar for a observable source using that is configured with an automation condition and a MultiPartitionKey where the output is of type DataVersionsByPartition. Each observation creates a backfill which never completes, even though the underlying runs have finished. This causes the Runs page to be filled with these backfills, since we want to observe every hour. Could this be related somehow? We're on Dagster 1.9.6, and didn't experience this issue before upgrading.

[Zan-L]

@stigmarl Yes, because observable source assets don't return output, either.