Attention Flow

Abnar & Zuidema have gathered more than 250 citation in 2 years for their paper Quantifying Attention Flow in Transformers (ACL’20), on interpreting attention patterns in Transformers.

Here’s a nice video explainer of the paper.

Reference

Samira Abnar and Willem Zuidema. 2020. Quantifying Attention Flow in Transformers. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 4190–4197, Online. Association for Computational Linguistics.