[N] DeepMind, Microsoft, Allen AI & UW Researchers Convert Pretrained Transformers into RNNs, Lowering Memory Cost While Retaining High Accuracy

  • by

A research team from University of Washington, Microsoft, DeepMind and Allen Institute for AI develop a method to convert pretrained transformers into efficient RNNs. The Transformer-to-RNN (T2R) approach speeds up generation and reduces memory cost.

Here is a quick read: DeepMind, Microsoft, Allen AI & UW Researchers Convert Pretrained Transformers into RNNs, Lowering Memory Cost While Retaining High Accuracy

The paper Finetuning Pretrained Transformers into RNNs is on arXiv.

submitted by /u/Yuqing7
[link] [comments]

Leave a Reply

Your email address will not be published. Required fields are marked *