멘토님과 Q&A

2021-09-24 피어세션에서 나온 질문들

• Q1. Electra에서 pretrain을 할 때, Generator도 train시키는 건가 아니면 discriminator만 train시키는 건가요?
• Q2. KoNLPy에서 mecab이 일본어 tokenizing 기반으로 만들어진 걸로 아는데, 이게 제일 빠르고 성능이 좋은 이유는 뭘까?

NLP Papers listing

Follows Chronological Order

• Attention is All you Need | 2017
• GPT | 2018
• BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding | 2019
• GPT 2: Language Models are Unsupervised Multitask Learners | 2019
• XLNet: Generalized Autoregressive Pretraining for Language Understanding | 2019
• RoBERTa | 2019
• ERNIE: Enhanced Representation through Knowledge Integration | 2019
• ALBERT: A Lite BERT for Self-supervised Learning of Language Representations | 2020
• ELECTRA: Pre-training Text Encoders as Discriminators Rather Than Generators | 2020
• BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension | 2020