👋🏻Welcome to Yuyang’s Blog

A Roadmap of Large Language Models (LLMs)

在这篇文章中，我们将探讨大语言模型的发展史，以及如何训练出一个强大的LLM。我们从数据的收集和预处理开始，逐步介绍模型架构，训练方法，后训练，评估，部署，以及应用。之后我们将探讨LLM的应用，包括LLM-Based Agents，等等。最后我们将展望LLM的未来发展趋势。这篇文章不是一个详尽的技术指南，而是一个概览，旨在帮助读者理解LLM的基本概念和发展历程。

2026-04-17

Yuyang Zhang

4450 words

17 mins

LLM Series Part2: Speed up & Scale up LLM

在这篇文章中，我们将深入探讨LLM的高性能推理（Inference）技术。随着LLM的规模不断扩大，并且LLM相关的应用推理的计算资源需求也在不断增加，因此如何提高LLM的推理效率成为了一个重要的问题。在这篇文章中，我们将介绍一些常用的高性能推理技术，比如量化（Quantization），剪枝（Pruning），知识蒸馏（Knowledge Distillation）等，这些技术可以有效地减少模型的计算复杂度，提高模型的推理效率。另外，我们还会介绍一些针对LLM推理优化的硬件加速器，比如GPU，TPU等，以及一些针对LLM推理优化的软件库，比如ONNX…

2026-04-08

Yuyang Zhang

What is Context Engineering & Harness Engineering?

随着OpenClaw在2026年初的爆火，Context Engineering和Harness Engineering的概念被提出并且重新被人关注。Context Engineering主要是指在使用LLM的过程中，如何有效地管理和利用上下文信息，以提高模型的性能和效率。而Harness Engineering则是指在使用LLM的过程中，如何设计和构建一个高效的系统架构，以支持模型的训练和部署。在这篇文章中，我们将深入探讨Context Engineering和Harness Engineering的概念，以及它们在LLM应用中的重要性和实践方法。

2026-03-21

Yuyang Zhang

LLM Series Part1: Architecture and Training Stages

在这篇文章中，我们将深入探讨不同的LLM的模型架构。随着LLM的不断发展，模型的架构也在不断的演变，我们将从最初的Transformer架构展开，逐步的介绍不同Component的演变和改进。在最后，我们还会介绍除了Transformer架构之外的其他架构，比如Mamba, RetNet等。最后我们还会介绍不同的Optimizer，Loss Functions: Cross-Entropy Loss，Diffusion Loss等，以及Regularization techniques在LLM训练中的应用。

Reinforcement Learning in LLM

Transformer

Large Language Model

Reinforcement Learning

In this blog, we explore how reinforcement learning (RL) is used to post-train large language models (LLMs) to improve performance, robustness, and alignment. We begin by formulating LLM generation as an RL problem (states, actions, policies, rewards), then introduce widely used RL-from-preference…

2026-01-25

Yuyang Zhang

From Entropy to KL Divergence: A Comprehensive Guide

Mathematics

KL Divergence, also known as Kullback-Leibler Divergence, is a fundamental concept in information theory and statistics. In this blog post, we will first explore the concept of entropy, which is the foundation of KL Divergence. Then, we will delve into the concept of KL Divergence, its…

2025-12-02

Yuyang Zhang

All kinds of “Learnings” in Deep Learning

Deep Learning

Summary

In the blog, I will cover all kind of learnings in Deep Learning, including supervised learning, unsupervised learning, self-supervised learning, reinforcement learning, and more. By the end, you will have a solid understanding of different learning paradigms in Deep Learning.

2025-12-02

Yuyang Zhang

The Evolution of Position Encoding in the Transformer

Transformer

Large Language Model

In the blog, I will go through different position encoding machnism used in the Transformer-based neural network architecture. We will explore why the position encoding is important and different types of position encoding. In the end, we will extend the position encoding to visual field, such as…

2025-11-25

Yuyang Zhang

All About Diffusion & Flow Models

Generative-Model

Diffusion-Model

This article offers a comprehensive overview of diffusion models from multiple perspectives. We begin with the foundations—DDPM, DDIM, and Score Matching—and explore their relationships. From there, we introduce the ODE/SDE framework, showing how DDPM can be derived from stochastic…

2025-10-15

Yuyang Zhang

Speed Up Training for Neural Networks

Training Tricks

Training large neural networks can be time-consuming and resource-intensive. This blog post explores various techniques to speed up the training process, including optimizing calculating in the single GPUs, and parallelizing the training across multiple GPUs.

2025-09-10

Yuyang Zhang

Categories