Lecture 12: Evaluation Metrics
About
📝 100 AI Papers with Code
About this series
Transformer
Vision Transformer
🎓 Stanford CS336: LLM from Scratch
About this course
Lecture 01: Introduction & BPE
Lecture 02: PyTorch Basics & Resource Accounts
Lecture 03: Transformer LM Architecture
Lecture 04: MoE Architecture
Lecture 05&06: GPU Optimization, Triton & FlashAttention
Lecture 07&08: Parallelism
Lecture 09&11: Scaling Laws
Lecture 10: Inference & Deployment
Lecture 12: Evaluation
Lecture 13&14: Data Collection & Processing
Lecture 15: LLM Alignment SFT & RLHF(PPO, DPO)
Lecture 16 & 17: LLM Alignment SFT & RLVR(GRPO)
Assignment 01: BPE Tokenizer & Transformer LM
Assignment 02: Flash Attention & Parallelism
Assignment 05: SFT & GRPO
📖 Deep Learning Foundation & Concepts
About this book
Lecture 12: Evaluation Metrics
Lecture 12介绍了评估语言模型性能的各种指标和方法。内容涵盖了传统的评估指标,如困惑度(Perplexity)、准确率(Accuracy)等,以及更适用于生成任务的指标,如BLEU、ROUGE和METEOR等。此外,还讨论了人类评估的重要性及其在模型评估中的作用。通过这些评估方法,可以更全面地了解语言模型的表现和改进方向。
Back to top
Lecture 10: Inference & Deployment
Lecture 13&14: Data Collection & Processing