Force async processing for large data sources on restricted domains

[mkangia]

Product Description

When saving a large data source (current default threshold is 1M), users will see the following error message to enforce asynchronous processing

Additionally, Minor improvement to show an error message on top if save fails for a data source configuration

Technical Summary

https://dimagi.atlassian.net/browse/SC-4001

Reused the existing feature flag that limits data source rebuilds to enforce asynchronous processing & slightly modifying the existing restriction by that feature flag to apply only for synchronous data sources.

Feature Flag

RESTRICT_DATA_SOURCE_REBUILD

Safety Assurance

Safety story

Tested locally.

Automated test coverage

QA Plan

https://dimagi.atlassian.net/browse/QA-7317

Rollback instructions

• This PR can be reverted after deploy with no further considerations

Labels & Review

• Risk label is set correctly
• The set of people pinged as reviewers is appropriate for the level of risk of the change

[Charl1996]

@mkangia

Why don't we mark it automatically for the user if know the data source covers a lot of data? It doesn't seem like a good UX when we only tell a user that they should have done something after they've tried to do it, especially if we can know that beforehand.

Any thoughts?

[mkangia]

Hey @Charl1996

Great points.

Couple of things that I considered:

1. I was hesitant to process the data source asynchronously if it was not marked to do so
2. I was hesitant to set something on the data source, which was created by the user & that too without the user knowing about it. We could else then mark it as async but also notify user we did so, along with explanation, which seemed to get too stretched for the implementation. So, I chose to rather tell the user to do so, which has an additional benefit of user getting aware of this feature and using it more actively in future.

Happy to reconsider if you think we are adding unnecessary step for the user. I was just thinking of being more explicit.

[Charl1996]

Happy to reconsider if you think we are adding unnecessary step for the user

I don't have too strong feelings either way, so fine to go with the way you have it. It does have the benefit of being more explicit and making the user aware of the setting.

[zandre-eng]

Looks good. Just some minor nits.

[zandre-eng]

This isn't part of your changes so feel free to ignore, but it seems we are returning an exception message back to the user here. Might be good to confirm that this is simply returning a general message and not any sort of stack trace.