[docs] Attention backends

[stevhliu]

Docs for enabling different attention backends.

[HuggingFaceDocBuilderDev]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

[sayakpaul]

Thanks for starting this! Looking good. Before talking about specific attention backends, we could educate the users about the forms in which they can use them.

1. Through setting attention backend name model. set_attention_backend("<backend_name>").
2. Through the context manager.

Then we could maintain a table of attention backend names, their GitHub/official pages, and notes.

We could then take one example for a backend and make it complete.

This way, I think it will be leaner and easier to follow. WDYT?

[sayakpaul]

1. Flex uses a different path in torch:
   

   diffusers/src/diffusers/models/attention_dispatch.py

   Line 847 in 5e181ed

   out = flex_attention.flex_attention(

2. The backends that leverage nn.functional.scaled_dot_product_attention() can be found in https://github.com/huggingface/diffusers/blob/5e181eddfe7e44c1444a2511b0d8e21d177850a0/src/diffusers/models/attention_dispatch.py (search with scaled_dot_product_attention().

[stevhliu]

Sounds good, feel free to contribute to the table with any notes you may have!

I skipped making an additional example at the end to avoid being repetitive since I've already included an example when introducing set_attention_backend and with attention_backend. I think once users learn that, its easy for them to just plug in the attention backend names.

[sayakpaul]

Superb stuff!

I would also make it clear that attention backends are an experimental feature.