BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding

Pre-trains a Transformer encoder on unlabeled text by hiding random words and recovering them from context on both sides. One pre-trained model can then be adapted to many tasks with a small output layer and brief fine-tuning.

Why it matters

It improved many benchmarks at once and made pre-train-then-fine-tune the standard recipe in NLP.

Read the paper: arXiv:1810.04805

Plain-language summary written by SolxLab. Read the full paper at the official link; all rights remain with the authors.