Google Translate is turning 20! 🎉. There are 20 fun facts and tips in the thread below.

Translate is one of my favorite Google products because it brings us all closer together!

I've been involved with a couple of things over the years. The first was our deployment of the initial system in 2006, which provided a huge leap forward in quality because it used a much larger 5-gram language model trained on trillions of words of text (indeed, probably the first trillion token language model training in the world: paper has some nice heads showing scaling-law-like quality improvement from scaling to more data/compute).

See "Large Language Models in Machine Translation", Thorsten Brants, Ashok C. Popat, Peng Xu, Franz J. Och and Jeffrey Dean, https://t.co/QnK7lllpoj

The second major collaboration was in 2016 when we moved Translate over from a statistical machine translation approach to using deep neural networks.  This approach relied on two key innovations.  The first was Google's work on Sequence-to-Sequence models (https://t.co/W9c0a0PXoV).  The second was our development of TPUs, custom cups that improved the performance of inference for deep neural networks by 30-80X over existing CPUs and GPUs of the day (and reduced latency by 15-30X).  This made launching compute-intensive language model services like Translate feasible for hundreds of millions of users. See "In-Datacenter Performance Analysis of a Tensor Processing Unit",  Norman P. Jouppi et al.  https://t.co/qpJl7FM6EO

GNMT paper:
"Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation",  Yonghui Wu, Mike Schuster, Zhifeng Chen, Quoc V. Le, Mohammad Norouzi, Wolfgang Macherey, Maxim Krikun, Yuan Cao, Qin Gao, Klaus Macherey, Jeff Klingner, Apurva Shah, Melvin Johnson, Xiaobing Liu, Łukasz Kaiser, Stephan Gouws, Yoshikiyo Kato, Taku Kudo, Hideto Kazawa, Keith Stevens, George Kurian, Nishant Patil, Wei Wang, Cliff Young, Jason Smith, Jason Riesa, Alex Rudnick, Oriol Vinyals, Greg Corrado, Macduff Hughes, and Jeffrey Dean, https://t.co/YasV0MEpxM

Most recently, we have advanced Translate further using Gemini models.

Each of these advances relied on research that have major quality leaps over the existing status quo translation approaches, bringing better quality and connectedness to all of our Translate users! 🎉

likes: 455 | retweets: 65 | replies: 31 | views: 178815