NaverAIBoostCamp_7기_paper

🔥25/02/21자로 모든 paper를 추가하였습니다!🔥

🚀 NaverAIBoostCamp 논문 및 further reading모음 📖

Collaborator

GitHub

Commit convention rule : 날짜-[주제]-내용-상태

ex) 2025-01-04 [CV Recent Trends] All/Only_Paper/Only_Further Add/Update/Delete

잘못된 내용은 이슈와 PR로 알려주세요 💡

🙏도움을 주신 분들🙏

더 좋은 컨텐츠를 제공받길 원한다면 💖후원하기!!💝

1. 📌Introduction

NaverAIboostCamp에서 소개한 논문들을 주제별로 정리한 폴더입니다. 현재는 CV(Computer Vision) 트랙의 논문을 중심으로 정리하고 있으며, 추후 모든 트랙으로 확장할 예정입니다.

2에서 5까지는 논문만 있습니다.

• 6부터 논문외의 사이트들을 정리하였습니다.

2. 📌CV 트랙 정리(Only Paper)

2.1 CV 이론

• VGGNet
• ResNet
• ViT
• Grad-CAM
• mixup
• CutMix
• Fully Convolutional Networks for Semantic Segmentation
• SAM
• DETR
• Real-World Single Image Super-Resolution: A New Benchmark and A New Model
• Real-World Blur Dataset for Learning and Benchmarking Deblurring Algorithms
• Blind Super-Resolution Kernel Estimation using an Internal-GAN
• SpatialTracker: Tracking Any 2D Pixels in 3D Space
• CLIP huggingface implementation
• ImageBIND official implementation
• LanguageBIND
• Flamingo pytorch implementation
• LLaVA
• BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generation
• BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models
• DDPM
• LDM (Stable Diffusion)
• DDIM
• 3D MACHINE LEARNING
• Mesh R-CNN
• NeRF
• 3DGS
• DreamFusion
• Loper et al., SMPL: A Skinned Multi-Person Linear Model: SIGGRAPH 2015.
• Bogo et al., Keep it SMPL: Automatic Estimation of 3D Human Pose and Shape from a Single Image: ECCV 2016.
• Anguelov et al., SCAPE: Shape Completion and Animation of People: SIGGRAPH 2005.

2.2 CV 기초 프로젝트

• A survey on Image Data Augmentation for Deep Learning
• AutoAugment: Learning Augmentation Strategies from Data
• RandAugment: Practical automated data augmentation with a reduced search space
• Fine-Grained Image Analysis with Deep Learning: A Survey
• A ConvNet for the 2020s
• An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
• CoAtNet: Marrying Convolution and Attention for All Data Sizes
• ConViT: Improving Vision Transformers with Soft Convolutional Inductive Biases
• Multimodal Learning with Transformers: A Survey
• Self-supervised Learning: Generative or Contrastive
• Ensemble deep learning: A review

2.3 Object Detection

• R-CNN
• Fast R-CNN
• Faster R-CNN
• SPPNet
• FPN
• PAFPN
• DetectoRS
• EfficientDet (BiFPN)
• NasFPN
• AugFPN
• YOLO survey
• Retinanet (focal loss)
• SSD
• EfficientNet
• EfficientDet
• DCN
• DETR
• Swin
• YOLO v4
• M2Det
• CornerNet

2.4 Data-Centric AI

• DMLR
• Convolutional Character Networks
• EAST
• Data and its (dis)contents: A survey of dataset development and use in machine learning research
• Human-In-The-Loop에 대한 survey 논문
• 다양한 task에 적용 가능한 IAA에 관한 논문
• LLM을 활용한 data annotation에 관한 survey 논문
• A survey on Image Data Augmentation for Deep Learning
• A survey of synthetic data augmentation methods in computer vision

2.5 Semantic Segmentation

• FCN
• DeconvNet
• SegNet
• FCDenseNet
• Unet
• DeepLabv1
• DilatedNet
• DeepLabv2
• PSPNet
• DeepLabv3
• DeepLabv3+
• Unet
• Unet++
• Unet3+
• EfficientUnet
• DenseUnet
• ResidualUnet
• SWA
• HRNet
• SegFormer
• ViT
• Weakly Supervised Object Localization and Detection: A Survey

2.6 CV Recent Trends

• NIPS 2016 Tutorial: Generative Adversarial Networks
• Variational Diffusion Models
• NVAE: A Deep Hierarchical Variational Autoencoder
• Denoising Diffusion Probabilistic Model
• Improved Denoising Diffusion Probabilistic Model
• High-Resolution Image Synthesis with Latent Diffusion Models
• Denoising Diffusion Implicit Models
• Progressive Distillation for Fast Sampling of Diffusion Models
• Consistency Models
• Diffusion Models Beat GANs on Image Synthesis
• Classifier-Free Diffusion Guidance
• Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding
• Hierarchical Text-Conditional Image Generation with CLIP latents
• SDEdit: Guided Image Synthesis and Editing with Stochastic Differential Equations
• Dual Diffusion Implicit Bridges for Image-to-Image Translation
• Adding Conditional Control to Text-to-Image Diffusion Models
• An Image is Worth One Word: Personalizing Text-to-Image Generation using Textual Inversion
• Dreambooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generation
• Erasing Concepts from Diffusion Models
• Unified Concept Editing in Diffusion Models
• Video Diffusion Models
• Video Probabilistic Diffusion Models in Projected Latent Space
• Align your Latents: High-Resolution Video Synthesis with Latent Diffusion Models
• DreamFusion: Text-to-3D using 2D Diffusion
• zero-1-to-3: zero-shot one image to 3d object
• Consistent-1-to-3: Consistent Image to 3D View Synthesis via Geometry-aware Diffusion Models
• Cascaded Diffusion Models for High Fidelity Image Generation
• DiffBIR: Towards Blind Image Restoration with Generative Diffusion Prior
• Solving Inverse Problem in Medical Imaging with Score-Based Generative Models
• Label-Efficient Semantic Segmentation with Diffusion Models
• ODISE: Open-Vocabulary Panoptic Segmentation with Text-to-Image Diffusion Models
• Your Diffusion Model is secretly a zero-shot classifier
• Emergent Correspondence from Image Diffusion
• AnoDDPM: Anomaly Detection with Denoising Diffusion Probabilistic Models using Simplex Noise

3. 📌NLP 트랙 정리(Only Paper)

3.1 NLP 이론

• Distributed Representations of Words and Phrases and their Compositionality
• GloVe: Global Vectors for Word Representation
• LSTM
• Highway Networks
• Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation
• Sequence to Sequence Learning with Neural Networks
• Neural Machine Translation by Jointly Learning to Align and Translate
• Effective Approaches to Attention-based Neural Machine Translation
• Sparse is Enough in Scaling Transformers
• Attention Is All You Need
• Layer Normalization
• Group Normalization
• Attention is not Explanation
• Attention is not not Explanation
• BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
• XLNet: Generalized Autoregressive Pretraining for Language Understanding
• Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation
• Neural Network Acceptability Judgments
• A Broad-Coverage Challenge Corpus for Sentence Understanding through Inference
• SQuAD: 100,000+ Questions for Machine Comprehension of Text
• Hierarchical Neural Story Generation
• The Curious Case of Neural Text Degeneration

3.2 NLP 기초 프로젝트

• Attention Is All You Need
• Quantifying Attention Flow in Transformers
• LoRA
• Non-Autoregressive & Autoregressive

3.3 MRC(Machine Reading Comprehension)

• BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
• BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension
• Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer (T5)
• Dense Passage Retrieval for Open-Domain Question Answering
• Reading Wikipedia to Answer Open-Domain Questions
• A Survey on Machine Reading Comprehension: Tasks, Evaluation Metrics and Benchmark Datasets
• Latent Retrieval for Weakly Supervised Open Domain Question Answering
• Dense Passage Retrieval for Open-Domain Question Answering
• Exploring the limits of transfer learning with a unified text-to-text transformer(T5)
• How much knowledge can you pack into the parameters of language model?
• UNIFIEDQA: Crossing Format Boundaries with a Single QA System
• LLaMA: Open and Efficient Foundation Language Models
• Real-Time Open-Domain Question Answering with Dense-Sparse Phrase Index
• Contextualized Sparse Representations for Real-Time Open-Domain Question Answering

3.4 Data-Centric NLP

• DMOps: Data Management Operation and Recipes
• A Survey on Awesome Korean NLP Datasets
• ProsocialDialog: A Prosocial Backbone for Conversational Agents
• Natural Language Processing: State of The Art, Current Trends and Challenges
• Understanding Back-Translation at Scale
• EDA: Easy Data Augmentation Techniques for Boosting Performance on Text Classification Tasks
• Data Augmentation using Pre-trained Transformer Models
• AugGPT: Leveraging ChatGPT for Text Data Augmentation
• Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processing
• Everyone's Voice Matters: Quantifying Annotation Disagreement Using Demographic Information
• GPT-4 Technical Report
• LLaMA: Open and Efficient Foundation Language Models
• Llama 2: Open Foundation and Fine-Tuned Chat Models

3.5 Generative for NLP

• Emergent Abilities
• Few-shot Learning (GPT-3)
• Chain of Thought
• Flash Attention 2
• RoPE
• A sensitivity analysis of (and practitioners' guide to) convolutional neural networks for sentence classification
• Scaling laws for neural language models
• Training compute-optimal large language models
• Llama: Open and efficient foundation language models
• SuperNLI: Super-naturalinstructions: Generalization via declarative instructions on 1600+ nlp tasks
• Bert: Pre-training of deep bidirectional transformers for language understanding
• GPT-3: Language models are few-shot learners.
• Few-shot fine-tuning vs. in-context learning: A fair comparison and evaluation
• Training language models to follow instructions with human feedback
• Learning to summarize with human feedback
• PPO: Proximal policy optimization algorithms
• DPO: Direct preference optimization: Your language model is secretly a reward model
• ORPO: Reference-free monolithic preference optimization with odds ratio
• The RefinedWeb dataset for Falcon LLM: outperforming curated corpora with web data, and web data only.
• The curious case of neural text degeneration
• The llama 3 herd of models
• Self-instruct: Aligning language models with self-generated instructions
• A Survey on Evaluation of Large Language Models
• LLM의 위치 편향
• WildBench 논문
• Lost in the Middle: How Language Models Use Long Contexts
• Extending Context Window of Large Language Models via Positional Interpolation
• Train Short, Test Long: Attention with Linear Biases Enables Input Length Extrapolation
• Ring Attention with Blockwise Transformers for Near-Infinite Context
• RAG의 시작
• Contrastive Learning에 대하여
• LLM 기반 임베딩 모델
• GraphRAG
• Reflexion
• ReAct
• Flashattention: Fast and memory-efficient exact attention with io-awareness
• Flashattention-2: Faster attention with better parallelism and work partitioning
• Fast inference from transformers via speculative decoding
• Efficient memory management for large language model serving with pagedattention

3.6 NLP Recent Trends

• A Survey of Large Language Models
• Tree-of-Thoughts
• ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIs (Qin et al., ICLR 2024)
• Describe, Explain, Plan, and Select (Wang et al., NeurIPS 2023)
• Generative Agents: Interactive Simulacra of Human Behavior (Park et al., UIST 2023)
• The Rise and Potential of Large Language Model Based Agents: A Survey (Xi et al., arXiv 2023)
• Survey of Hallucination in Natural Language Generation (Ji et al., arXiv 2022)
• Siren's Song in the AI Ocean: A Survey on Hallucination in Large Language Models (Zhang et al., arXiv 2023)
• Bias and Fairness in Large Language Models: A Survey (Gallegos, arXiv 2023)
• Toxicity in ChatGPT: Analyzing Persona-assigned Language Models (Deshpande et al., EMNLP 2023)
• ACL 2023 Tutorial: Retrieval-based Language Models and Applications
• Prompting GPT-3 To Be Reliable (Si et al., ICLR 2023)
• TemporalWiki (Jang et al., EMNLP 2022)
• Retrieval-Augmented Black-Box Language Models (Shi et al., arXiv 2023)
• Rethinking with Retrieval: Faithful Large Language Model Inference (He et al., arXiv 2022)

4. 📌Recsys 트랙 정리(Only Paper)

4.1 Recsys 이론

• Variational Autoencoders for Collaborative Filtering
• Diffusion Recommendation Model
• Bayesian Probabilistic Matrix Factorization using Markov Chain Monte Carlo
• Probabilistic Matrix Factorization
• Understanding Black-box Predictions via Influence Functions, ICML 2017
• Data Shapley: Equitable Valuation of Data for Machine Learning,ICML 2019
• Data Valuation using Reinforcement Learning, ICML 2020
• Data-OOB: Out-of-bag Estimate as a Simple and Efficient Data Value, ICML 2023
• Efficient Neural Causal Discovery without Acyclicity Constraints
• A Meta-Transfer Objective for Learning to Disentangle Causal Mechanisms
• Causal Reasoning and Large Language Models: Opening a New Frontier for Causality

4.2 ML 기초 프로젝트

• A Brief Introduction to Machine Learning for Engineers
• Zheng, Alice, and Amanda Casari. Feature engineering for machine learning: principles and techniques for data scientists. " O'Reilly Media, Inc.", 2018.
• Hastie, Trevor, et al. The elements of statistical learning: data mining, inference, and prediction. Vol. 2. New York: springer, 2009.
• SMOTE: synthetic minority over-sampling technique. Journal of artificial intelligence research, 16, 321-357.
• Gradient Boosting Paper
• XGBoost Paper
• LightGBM Paper
• Lasso Regression Paper
• Ridge Regression Paper
• Ensemble deep learning: A review
• Random Search for Hyper-Parameter Optimization
• Practical Bayesian Optimization of Machine Learning Algorithms
• Hyperparameters and Tuning Strategies for Random Forest
• Management of Machine Learning Lifecycle Artifacts: A Survey

4.3 Competitive DS

• ArcFace: Additive Angular Margin Loss for Deep Face Recognition
• Attention Is All You Need
• LONG SHORT-TERM MEMORY
• SAINT Model Paper
• Deep Knowledge Tracing

4.4 RecSys 기초프로젝트

• Improved Apriori Algorithm for Mining Association Rules
• Matrix Factorization Techniques for Recommender Systems
• BPR: Bayesian personalized ranking from implicit feedback
• Neural Collaborative Filtering
• Deep Neural Networks for YouTube Recommendations
• AutoRec: Autoencoders Meet Collaborative Filtering
• Collaborative Denoising Auto-Encoders for Top-N Recommender Systems
• Neural Graph Collaborative Filtering
• LightGCN: Simplifying and Powering Graph Convolution Network for Recommendation
• When Recurrent Neural Networks meet the Neighborhood for Session-Based Recommendation
• Factorization Machines
• Field-aware Factorization Machines for CTR Prediction
• Greedy Function Approximation: A Gradient Boosting Machine
• XGBoost: A Scalable Tree Boosting System
• LightGBM: A Highly Efficient Gradient Boosting Decision Tree
• CatBoost: unbiased boosting with categorical features
• Wide & Deep Learning for Recommender Systems
• DeepFM: A Factorization-Machine based Neural Network for CTR Prediction
• Deep Interest Network for Click-Through Rate Prediction
• Behavior Sequence Transformer for E-commerce Recommendation in Alibaba
• An Empirical Evaluation of Thompson Sampling
• A Contextual-Bandit Approach to Personalized News Article Recommendation
• RecSys Challenge 2023: Deep Funnel Optimization with a Focus on User Privacy

4.5 Movie Rec

• A survey of collaborative filtering techniques
• Restricted Boltzmann Machines for Collaborative Filtering
• AutoRec: Autoencoders Meet Collaborative Filtering
• Neural Collaborative Filtering
• Variational Autoencoders for Collaborative Filtering
• Wide & Deep Model
• DeepFM
• Latent Cross
• Temporal-Contextual Recommendation in Real-Time
• SASRec
• BERT4Rec
• Sequential Recommender Systems: Challenges, Progress and Prospects
• A Case Study on Sampling Strategies for Evaluating Neural Sequential Item Recommendation Models

4.6 Recsys Recent Trends

• Neural Graph Collaborative Filtering
• Simplifying and Powering Graph Convolution Network for Recommendation
• Disentangled Graph Collaborative Filtering
• Self-supervised Graph Learning for Recommendation
• Hypergraph Contrastive Collaborative Filtering
• Semi-supervised classification with graph convolutional networks
• Simplifying Graph Convolutional Networks
• Hypergraph Neural Networks
• A Survey of Graph Neural Networks for Recommender Systems: Challenges, Methods, and Directions
• Self-Attentive Sequential Recommendation
• Sequential Recommendation with Bidirectional Encoder Representations from Transformer
• Lighter and Better: Low-Rank Decomposed Self-Attention Networks for Next-Item Recommendation
• Feature-level Deeper Self-Attention Network for Sequential Recommendation
• Noninvasive Self-attention for Side Information Fusion in Sequential Recommendation
• Decoupled Side Information Fusion for Sequential Recommendation
• Sequential Modeling with Multiple Attributes for Watchlist Recommendation in E-Commerce
• DropoutNet: Addressing Cold Start in Recommender Systems
• Recommendation for New Users and New Items via Randomized Training and Mixture-of-Experts Transformation
• Variational Autoencoders for Collaborative Filtering (Liang et al., WWW 2018)
• Local Latent Space Models for Top-N Recommendation (Christakopoulou et al., KDD 2018)
• TALLRec: An Effective and Efficient Tuning Framework to Align Large Language Model with Recommendation

5. 📌공통 강의에서 소개한 논문 정리

5.1 Generative AI

• LLM Survey 논문 (2023)
• GAN Survey 논문 (2020)
• Diffusion Models Survey 논문 (2024)
• RLHF 제안 논문
• Large Language Model 서베이 논문
• GPT-3 논문
• A Survey of Large Language Models
• Self-Instruct 논문
• Self-Rewarding 논문
• GAN 논문
• cGAN 논문
• Pix2Pix 논문
• CycleGAN 논문
• StarGAN 논문
• ProgressiveGAN 논문
• StyleGAN 논문
• VAE 논문
• VQ-VAE 논문
• DDPM 논문
• DDIM 논문
• Classifier Guidance 논문
• Classifier-free Guidance 논문
• LDM 논문
• Latent Diffusion Model
• Stable Diffusion XL
• SDXL Turbo (Adversarial Diffusion Distillation)

5.2 Product Serving

• paper 없음

5.3 최적화/경량화

• Rethinking the Value of Network Pruning
• Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inference
• The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networks:
• Pruning Convolutional Neural Networks for Resource Efficient Inference
• Compressing BERT: Studying the Effects of Weight Pruning on Transfer Learning
• Distilling the Knowledge in a Neural Network
• Parameter-Efficient Transfer Learning for NLP
• Prompt tuning
• Prefix tuning
• AdapterFusion: Non-Destructive Task Composition for Transfer Learning
• QLoRA: Efficient Finetuning of Quantized LLMs

---

여기서부터는 논문외의 읽을거리들로 있었던 것들을 정리하였습니다.

6. 📌Further Reading에 있었던 것들.

6.1 공통코스

6.1.1 Pytorch

• Introduction to PyTorch — PyTorch Tutorials documentation
• 텐서(Tensor) — 파이토치 한국어 튜토리얼 (PyTorch tutorials in Korean)
• torch.Tensor — PyTorch documentation
• 부동소수점 - 백과사전
• torch.randn — PyTorch documentation
• torch.Tensor — PyTorch documentation
• GPU와 AI - 네이버 지식백과
• torch.Tensor.view — PyTorch main documentation
• torch.reshape — PyTorch main documentation
• Difference between view, reshape, transpose and permute in PyTorch
• torch.squeeze — PyTorch main documentation
• Tensor 모양 설명
• $L_p$ norm
• 선형회귀
• 경사하강법 학습방법
• Preprocessing data
• PyTorch DataLoader — PyTorch main documentation
• BCELoss — PyTorch main documentation
• BCEWithLogitsLoss — PyTorch main documentation
• CrossEntropyLoss — PyTorch main documentation

6.1.2 ML LifeCycle

• A survey on Image Data Augmentation for Deep Learning
• Clipping
• LSTM — PyTorch main documentation
• Attention Is All You Need
• BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
• An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale

6.1.3 EDA & DataViz

• [Harvard Business Review] Boost Your Team's Data Literacy
• [Sequoia Capital] Why Data Science Matters
• [Sequoia Capital] Role of Data Scientist
• 데이터 시각화 교과서 (클라우스 윌케 저)
• Data Viz Project
• Misleading Data Visualization
• [The Economists] Mistakes, we've drawn a few
• [Google ML Course] Types of Bias
• [Github] Awesome Feature Engineering
• [서울대 AI 연구원] 다차원 데이터 시각화와 AI (컴퓨터공학부 서진욱 교수)
• [Distill] How to Use t-SNE Effectively
• Forecasting: Principles & Practice
• Hugging Face - NLP Course
• Text Visualization Browser
• [Github] eugeneyan/applied-ml
• [Material Design] Data Visualization
• 잘못 사용된 시각화 모음 WTF.viz
• Startup Metrics for Pirates: AARRR! - Dave McClure
• 데이터 시각화, 인지과학을 만나다
• 도널드 노만의 UX 디자인 특강
• UX/UI의 10가지 심리학 법칙

6.1.4 AI 개발 기초

• "Clean Code: A Handbook of Agile Software Craftsmanship" by Robert C. Martin:
• "Design Patterns: Elements of Reusable Object-Oriented Software" by Erich Gamma, Richard Helm, Ralph Johnson, and John Vlissidesn
• "모던 리눅스 교과서" by 마이클 하우센블라스
• "The Linux Command Line" by William Shotts:
• Streamlit 공식 문서
• Python 3.x Docs: Virtual Environments and Packages

6.2 CV

6.2.1 CV 이론

• 한겨례 뉴스 "빌게이츠도 감탄한 최예진 교수, 생성형 AI 학습 데이터 공개해야"

6.2.2 CV 기초 프로젝트

• Kaggle Competition
• Image file format
• Multilabel Image Classification Using Deep Learning
• Training with Pytorch
• Pytorch: Automatic Mixed Precision
• Nvidia: Mixed-Precision-Training
• Tensorboard
• WandB

6.2.3 Object Detection

• mmdetection github
• detectron2 github
• Paperswithcode Object Detection
• Kaggle

6.2.4 Data-Centric AI

• AI 최신 활용 사례
• 현실 세계에서의 데이터중심 AI
• SynthText in the Wild Dataset
• Tesseract (Off-the-shelf OCR open source)
• Data annotation에 대한 OpenCV의 blogpost

6.2.5 Semantic Segmentation

• Segmentation models library

6.2.6 CV Recent Trends

• Generative Modeling by Estimating Gradients of the Data Distribution

6.3 NLP

6.3.1 NLP 이론

• Byte-Pair Encoding tokenization
• The Illustrated Word2vec
• The Unreasonable Effectiveness of Recurrent Neural Networks
• cs231n Lecture 10: Recurrent Neural Networks
• The Exploding and Vanishing Gradients Problem in Time Series
• Understanding LSTM Networks
• The Illustrated Transformer
• The Annotated Transformer
• BertViz: Visualize Attention in NLP Models
• Foundations of NLP Explained Visually: Beam Search, How It Works
• How to generate text: using different decoding methods for language generation with Transformers

6.3.2 NLP 기초 프로젝트

• Improving Language Understanding by Generative Pre-Training
• Tokenizer 에 대해
• Hugging Face Tokenizer 활용하기
• Summary of the tokenizers
• Hugging Face Tutorial
• Hugging Face 문서
• T-value 와 P-value
• BertViz
• Tune Hyperparameters with Sweeps (wandb.ai)
• PEFT (Hugging Face)
• Parameter와 Hyperparameter의 차이
• Text classification (Hugging Face)
• Image Embedding과 Triplet Loss 간단 설명 (tistory.com)
• k-최근접 이웃 알고리즘 - 위키백과, 우리 모두의 백과사전 (wikipedia.org)
• Embedding Distance | 🦜️🔗 LangChain
• BertForSequenceClassification
• transformers/src/transformers/models/bert/modeling_bert.py at v4.34.1 · huggingface/transformers (github.com)
• What is Summarization? - Hugging Face
• BART
• OpenAI GPT2
• Trainer(Hugging Face)
• WandB
• Model Hub(Hugging Face)
• Model Card (Hugging Face)
• Streamlit tutorial
• Annotated Model Card Template (Hugging Face)

6.3.3 MRC(Machine Reading Comprehension)

• 문자열 type에 관련된 정리글
• KorQuAD 데이터 소개 슬라이드
• Naver Engineering: KorQuAD 소개 및 MRC 연구 사례 영상
• SQuAD 데이터셋 둘러보기
• The Illustrated BERT, ELMo, and co. (How NLP Cracked Transfer Learning)
• Huggingface datasets
• Introducing BART
• Pyserini BM25 MSmarco documnet retrieval 코드
• Sklearn feature extractor
• Open domain QA tutorial: Dense retrieval
• FAISS blog
• FAISS github
• FAISS tutorial
• Getting started with Faiss
• ACL 2020 ODQA tutorial
• Phrase Retrieval and Beyond

6.3.4 Data-Centric NLP

• Week 35 - 모델 중심에서 데이터 중심의 AI 개발로
• Crowdsourced Data Collection Benefits & Best Practices ['25]
• KLUE benchmark
• EIGHTH CONFERENCE ON MACHINE TRANSLATION (WMT23)
• Tokenizers_huggingface
• Summary of the tokenizers
• 2023년 1분기 인공지능(AI) 및 자연어처리(NLP) 주요 뉴스
• LLM에 환각이 발생하는 원인과 해결 방안

6.3.5 Generative for NLP

• LLaMA 3.2
• LLaMA 구조 설명
• LLaMA Github 모델링 코드
• Stanford Alpaca
• What is reinforcement learning from human feedback
• Needle In A Haystack
• Jamba 1.5 Open Model Family: The Most Powerful and Efficient Long Context Models

6.3.6 NLP Recent Trends

• Prompt-engineering
• Introduction to LLM Agents
• LLM Powered Autonomous Agents
• AutoGPT
• BingAI
• Biases in Large Language Models: Origins, Inventory, and Discussion (Navigli et al., Journal of Data and Information Quality 2023)
• WhatsApp’s AI shows gun-wielding children when prompted with ‘Palestine’
• 주제별로 알아보는 continual learning
• SearchGPT 기능
• Reducing Toxicity in Language Models
• Red Teaming Language Models with Language Models (Perez et al., EMNLP 2022)
• LangChain
• LangChain Explained in 13 Minutes

6.4 Recsys

6.4.1 Recsys 이론

• 중심극한정리, Central Limit Theorem
• 최대우도법(MLE)
• 가우시안 혼합 모델(Gaussian Mixture Model)
• KL divergence
• VI(Variational Inference)
• Markov Chain Monte Carlo - 공돌이의 수학정리 노트
• Influence Function에 대한 이해
• Trustworthy Machine Learning (Chapter 3 p.116-p.219)
• Python으로 하는 인과추론 : 개념부터 실습까지

6.4.2 ML 기초 프로젝트

• Is there a rule-of-thumb for how to divide a dataset into training and validation sets? - stackoverflow
• Data Leakage in The ICML 2013 Whale Challenge
• regression-metrics-for-mashine-learning
• Cross-Validation-Techniques
• Weights & Biases 공식 튜토리얼 문서
• Weights & Biases 공식 예제 깃허브 저장소
• The Kaggle Book: Data analysis and machine learning for competitive data science. Packt Publishing Ltd, 2022.

6.4.3 Competitive DS

• An Introduction to Statistical Learning
• Pattern Recognition and Machine Learning
• Time Series Analysis: Forecasting and Control
• The Elements of Statistical Learning
• Python Data Science Handbook
• Scikit-learn Documentation
• Deep Learning book
• PyTorch Documentation
• Introduction to Data Mining
• XGBoost Documentation
• LightGBM Documentation
• CatBoost Documentation
• Deep Learning for Time Series Forecasting
• Feature Engineering and Selection: A Practical Approach for Predictive Models
• Data Science for Business
• Hands-On Machine Learning with Scikit-Learn, Keras, and TensorFlow
• Ensemble Methods: Foundations and Algorithms
• Automated Machine Learning: Methods, Systems, Challenges
• Hyperparameter Optimization: A Practical Guide
• Optuna Documentation

6.4.4 RecSys 기초프로젝트

• [추천 시스템] 성능 평가 방법 - Precision, Recall, NDCG, Hit Rate, MAE, RMSE
• 한국어와 NLTK, Gensim의 만남 @ PyCon Korea 2015
• 17 types of similarity and dissimilarity measures used in data science.
• Word2Vec 그리고 추천 시스템의 Item2Vec
• TOROS N2 - lightweight approximate Nearest Neighbor library
• ANN-Benchmarks
• (영상) XGBoost - StatQuest
• (영상) 04-8: Ensemble Learning - LightGBM (앙상블 기법 - LightGBM)
• Catboost 주요 개념과 특징 이해하기
• (영상) 뭐볼까? : 네이버 AiRS 인공지능 콘텐츠 추천의 진화
• 논문 리뷰 A Contextual-Bandit Approach to Personalized News Article Recommendation
• RecSys Challenge 2023 Homepage

6.4.5 Movie Rec

• Paperswithcode(MovieLens)
• 01. 추천시스템 이해
• awesome-RecSys
• Collaborative Denoising Auto-Encoders for Top-N Recommender Systems
• Artwork Personalization at Netflix
• Learning a Personalized Homepage

6.4.6 Recsys Recent Trends

• (블로그) Enhancing Recommendation Systems with Large Language Models

6.5 Generative AI

• LLM 모델 튜닝, 하나의 GPU로 가능할까? Parameter Efficient Fine-Tuning(PEFT)을 소개합니다!
• Alpaca 홈페이지

6.6 Product Serving

• Uber의 Michelangelo 플랫폼에 관한 글로, 대규모 머신러닝 모델 서빙 방법에 대한 통찰을 살펴볼 수 있습니다.
• Machine learning is going real-time
• 쏘카의 Airflow 구축기
• 쏘카의 Airflow 구축기2
• 버킷플레이스 Airflow 도입기
• 라인 엔지니어링 Airflow on Kubernetes
• Day1, 2-2. 그런 REST API로 괜찮은가
• HTTP response status codes
• FastAPI Documentation
• Advanced User Guide for FastAPI
• Awesome-fastapi
• 도커(Docker) 입문편: 컨테이너 기초부터 서버 배포까지
• GCP 공식 문서
• AWS를 이용한 MLOps 구축 사례 살펴보기
• MLOps Principles - Automation
• Experiments Tracking
• Reproducibility
• Data Engineer Jobs

6.7 최적화/경량화

• Softmax Temperature
• Lecture 10 - Knowledge Distillation | MIT 6.S965
• A brief overview of Imitation Learning
• Dynamic Quantization
• 딥러닝의 Quantization (양자화)와 Quantization Aware Training:
• Transfer Learning for Computer Vision Tutorial
• Brief Overview of Parallelism Strategies
• Data Parallelism on CNN:
• Pipeline Parallelism Algorithm
• 단일 머신을 사용한 모델 병렬화 모범 사례

7. 📌Contribution 하는 방법

7.1 Fork를 한다.

• 원본 저장소를 내 계정으로 복사(Fork)

7.2 PR를 보낸다.

• Fork한 저장소에서 변경 내용을 작업한 뒤 Pull Request(Pull Request)를 생성

7.3 Approve를 받으면 Merge!

• 리뷰 후 승인을 받으면 원본 저장소에 변경사항이 병합(Merge)

이 문서는 계속 업데이트 예정입니다. 궁금한 사항이나 제안이 있다면 Issue 혹은 Pull Request로 알려주세요! 감사합니다.

8. 📌Updates (Changelog) - finished at 25-02-21🔥

날짜	변경 사항
2025-01-01	🎨 CV 업데이트 - CV 섹션 논문 목록을 추가하고 깔끔하게 정렬했습니다.
2025-01-02	✨ NLP 이론 섹션 업데이트 - NLP 이론 파트에 주요 논문 및 레퍼런스를 추가했습니다.
2025-01-04	🎨 CV Recent Trends 섹션 업데이트 - CV recent trends 섹션에 최근 논문 목록을 추가했습니다.
2025-01-13	✨ NLP 기초 프로젝트 섹션 업데이트 - NLP 기초 프로젝트 파트에 주요 논문 및 레퍼런스를 추가했습니다.
2025-01-13	🌱 Recsys 이론 및 ML 기초 프로젝트 섹션 업데이트 - Recsys 기초 프로젝트 파트에 주요 논문 및 레퍼런스를 추가했습니다.
2025-01-14	✨ NLP MRC & Data-Centric 섹션 업데이트 - NLP 기초 프로젝트 파트에 주요 논문 및 레퍼런스를 추가했습니다.
2025-01-14	🌱 Recsys Competitive DS & Recsys 기초프로젝트 섹션 업데이트 - Recsys 기초 프로젝트 파트에 주요 논문 및 레퍼런스를 추가했습니다.
2025-01-21	✨ NLP Generation & Recent 섹션 업데이트 - NLP 파트에 주요 논문 및 레퍼런스를 추가했습니다.
2025-02-21	🔥 모든 paper를 추가하였습니다

9. 📌Star History

Star History