Flash attention?

[jkosinski]

Consider asking for this pull request?
google-deepmind/alphafold#931 (comment)
Apparently it gives significant speed up!

[DimaMolod]

Supported in the latest version of alphafold2

[DimaMolod]

...but not merged yet :-)
google-deepmind/alphafold#931

[DimaMolod]

AlphaPulldown already exposes flash-attention controls for AF3 at run_structure_prediction.py and run_multimer_jobs.py. But for official DeepMind AF2, the flash-attention PR is still open and unmerged, so implementing this here means maintaining an AlphaFold2 fork/patch with numerical and portability risk.

[jkosinski]

yeah, i think can be closed now.