别只做一个会回答的 RAG。做一个能证明、能监控的系统。Do not build a RAG that merely answers. Build a system that proves and monitors.
这套讲义逐模块覆盖当前官方仓库，用“岗位知识库问答系统”贯穿摄取、检索、Agent、编排、评估、监控与交付。它假设你先完成 Python 和生成式 AI 入门，但每个工程概念仍从白话解释开始。
This guide covers every current official module through a running Job Knowledge Copilot, spanning ingestion, retrieval, agents, orchestration, evaluation, monitoring, and delivery. It assumes the Python and GenAI foundations while explaining every engineering concept in plain language.
The 30-minute path samples one chapter per volume for the map from Agentic RAG to portfolio delivery. Then complete all 42 examples and project tasks module by module.
The official repository is publicly readable but declares no license. This guide does not reproduce or translate course prose, code, homework, or templates; all bilingual explanations, examples, misconceptions, checks, and acceptance criteria are original. Services, versions, and schedules change, so verify the current repository before implementation.
第一册 · 模块 1Volume 1 · Module 1
先让检索会计划，也会停Let retrieval plan—and stop
用受控循环理解 Agentic RAG。
Understand Agentic RAG through a bounded loop.
本册作品里程碑Portfolio milestone
交付三轮上限、可回放的研究流程。
Deliver a replayable research flow capped at three rounds.
Basic RAG retrieves once then answers. Why should a flow adapt when a question needs decomposition, rewriting, follow-up retrieval, and a stopping decision?
类比桥梁Analogy bridge
像研究员先列问题、找第一批资料，再根据缺口决定补哪份来源，而不是一次搜索后硬写结论。
Like a researcher who plans questions, reviews initial sources, then fills evidence gaps instead of drafting after one search.
类比边界：类比只帮助识别关系；真正系统仍由数据、代码、接口和运行环境决定。Where it breaks: The analogy only exposes relationships; the real system is determined by data, code, interfaces, and its runtime.
Agentic RAG places query planning, retrieval, evidence review, tool use, and stopping into a bounded loop with state, budget, provenance, and failure exits.
例子 1 · 问题分解Example 1 · Question decomposition
输入Input
“我适合哪个 AI 岗位，缺什么？”“Which AI role fits me, and what am I missing?”
过程Process
拆成经历证据、岗位要求、差距三个检索问题Split into experience, role requirements, and gap queries
结果 / 验收Result / check
三组证据合并后回答Answer after merging three evidence groups
例子 2 · 查询改写Example 2 · Query rewriting
输入Input
用户说“会做知识库”User says “I can build a knowledge base”
过程Process
改写为 RAG、embedding、检索、评估同义查询Rewrite into RAG, embedding, retrieval, and evaluation queries
结果 / 验收Result / check
召回口语与技术词表达Retrieve both colloquial and technical wording
例子 3 · 证据不足停止Example 3 · Stop on insufficient evidence
输入Input
材料没有线上监控经历No monitoring experience in the material
过程Process
两轮检索仍为空就停止Stop after two empty retrieval rounds
结果 / 验收Result / check
明确缺证据，不虚构经历State missing evidence; do not invent experience
例子 4 · 工具选择Example 4 · Tool selection
输入Input
问题含当前招聘数据Question needs current job-market data
过程Process
选择授权搜索而非只查静态库Choose authorized search, not only the static store
结果 / 验收Result / check
动态事实带来源日期Dynamic facts include source dates
例子 5 · 循环上限Example 5 · Loop cap
输入Input
检索结果反复相似Repeatedly similar retrieval results
过程Process
按重复率与步数终止Terminate by repetition rate and step count
结果 / 验收Result / check
避免无限检索与费用失控Avoid infinite retrieval and runaway cost
例子 6 · 人类确认Example 6 · Human checkpoint
输入Input
准备给外部联系人发总结Ready to send a summary externally
过程Process
Agent 只生成草稿和收件人预览Agent only drafts and previews recipients
结果 / 验收Result / check
用户确认后才外发Send only after user approval
常见误区Common misconception
把“会自己搜很多次”当作聪明；没有预算、停止与证据标准的循环只会更贵地重复错误。
Treating repeated search as intelligence; without budget, stopping, and evidence criteria, the loop repeats mistakes at higher cost.
作品任务Portfolio task
实现岗位知识库的三步 Agentic RAG：计划问题、执行检索、评审证据；最多三轮并保存轨迹。
Implement a three-stage Agentic RAG flow for the job knowledge base: plan, retrieve, and review evidence, capped at three rounds with traces.
验收：Acceptance:每个回答能回放查询与来源；无证据、重复结果或超预算时有不同停止原因。Every answer replays queries and sources, with distinct stop reasons for no evidence, repetition, or budget.
自测：什么时候普通“检索一次 → 生成”反而比 Agentic RAG 更好？Self-check: When is one-shot retrieve-then-generate better than Agentic RAG?
When the question is simple, the corpus stable, one retrieval sufficient, or latency, cost, and risk require a fixed path. Add autonomy only for measured benefit.
把文档变成有版本、有权限的证据Turn documents into versioned, authorized evidence
建立摄取、切块、混合检索、过滤与重排。
Build ingestion, chunking, hybrid retrieval, filtering, and reranking.
本册作品里程碑Portfolio milestone
交付 40 问检索评估与来源追踪。
Deliver a 40-query retrieval evaluation with provenance.
02
模块 2Module 2
向量检索：从文档进入到候选证据排序Vector Search
先回答一个白话问题Start with a plain question
文字不同但意思相近时，怎样找到相关内容，又怎样避免“相似”被误写成“正确”？
How do we find semantically related text while avoiding the mistake of treating similarity as correctness?
类比桥梁Analogy bridge
像给每段文字放到一张语义地图；距离近代表话题接近，不代表来源最新、可信或有权限。
Place passages on a semantic map: proximity suggests topic similarity, not recency, authority, or permission.
类比边界：类比只帮助识别关系；真正系统仍由数据、代码、接口和运行环境决定。Where it breaks: The analogy only exposes relationships; the real system is determined by data, code, interfaces, and its runtime.
Ingestion, parsing, chunking, embeddings, vector/keyword retrieval, filtering, fusion, and reranking are separate stages; the index stores provenance, version, and access metadata.
例子 1 · 分块边界Example 1 · Chunk boundary
输入Input
长文档中的资格条件与例外Eligibility rule and exception in a long document
过程Process
按标题/段落切块并保留少量重叠Chunk by headings/paragraphs with limited overlap
结果 / 验收Result / check
规则与例外能同块或相邻召回Rule and exception retrieve together or adjacently
按内容 hash 增量重嵌入并删除旧版本Re-embed by content hash and remove old version
结果 / 验收Result / check
查询只返回当前可追溯版本Queries return only the current traceable version
常见误区Common misconception
只调 chunk size 和 top-k，完全不建立“真实问题—相关证据”的检索评估集。
Tuning chunk size and top-k without a query-to-relevant-evidence evaluation set.
作品任务Portfolio task
为 30 份项目材料建立混合检索：可重复摄取、版本过滤、ACL、重排与 40 条检索评测。
Build hybrid retrieval for 30 project documents with repeatable ingestion, version filters, ACL, reranking, and 40 retrieval evaluations.
验收：Acceptance:Recall@k、MRR/命中位置和空查询表现有记录；任何候选能回到源文件与段落。Recall@k, rank position/MRR, and empty-query behavior are recorded; every candidate traces to file and paragraph.
自测：向量相似度 0.92 能证明片段支持答案吗？Self-check: Does vector similarity 0.92 prove that a passage supports the answer?
让数据管线失败得可见、恢复得可靠Make pipeline failures visible and recovery reliable
用编排管理状态，不把业务逻辑锁进平台。
Use orchestration for state without trapping business logic.
本册作品里程碑Portfolio milestone
交付可恢复的候选发布工作流。
Deliver a recoverable candidate-release workflow.
03
模块 3Module 3
AI 编排：把摄取、评估与发布变成可恢复流程AI Orchestration with Kestra
先回答一个白话问题Start with a plain question
当一次项目需要定时摄取、建索引、跑评估、发布和失败重试时，怎样让每一步可见、可恢复？
When a project schedules ingestion, indexing, evaluation, release, and retries, how do we make every step visible and recoverable?
类比桥梁Analogy bridge
像机场调度：航班有依赖、资源、重试和人工放行，不能靠一条长脚本祈祷跑完。
Like airport operations: dependencies, resources, retries, and human clearance cannot be managed by one hopeful long script.
类比边界：类比只帮助识别关系；真正系统仍由数据、代码、接口和运行环境决定。Where it breaks: The analogy only exposes relationships; the real system is determined by data, code, interfaces, and its runtime.
A DAG defines dependencies, tasks stay idempotent, and artifacts are versioned. The orchestrator schedules and tracks state without trapping core business logic in platform config.
例子 1 · 每日摄取Example 1 · Daily ingestion
输入Input
每日新增岗位文档Daily new job documents
过程Process
检测变更→解析→索引→评估Detect change → parse → index → evaluate
结果 / 验收Result / check
只有评估通过才标为当前索引Mark the index current only after evaluation passes
例子 2 · 失败重试Example 2 · Failure retry
输入Input
embedding 服务 503Embedding service returns 503
过程Process
有界重试并从失败批次续跑Bounded retry and resume failed batch
结果 / 验收Result / check
已成功批次不重复计费Successful batches are not billed twice
例子 3 · 幂等运行Example 3 · Idempotent run
输入Input
同一文档被事件触发两次The same document event fires twice
过程Process
以内容 hash 作为任务键Use content hash as task key
结果 / 验收Result / check
只生成一份索引记录Only one index record is produced
例子 4 · 人工门禁Example 4 · Human gate
输入Input
新索引检索分下降New index lowers retrieval quality
过程Process
工作流暂停并展示差异Pause the flow and show the diff
结果 / 验收Result / check
负责人决定发布或回滚Owner chooses release or rollback
例子 5 · 秘密隔离Example 5 · Secret isolation
输入Input
模型与数据库凭证Model and DB credentials
过程Process
使用编排器 secret 引用与最小权限身份Use secret references and least-privilege identities
结果 / 验收Result / check
日志和配置不含明文秘密Logs and configs contain no plaintext secrets
例子 6 · 平台退出Example 6 · Platform exit
输入Input
需要更换编排器Need to change orchestrators
过程Process
业务步骤封装成独立 CLI/APIKeep business steps in independent CLI/API units
结果 / 验收Result / check
新平台只重写调度层Only scheduling is rewritten
常见误区Common misconception
把 YAML 配置写得很复杂就认为流程可靠，却没有幂等、制品版本和失败恢复测试。
Mistaking complex YAML for reliability without idempotency, artifact versioning, or recovery tests.
作品任务Portfolio task
编排一条“摄取→索引→检索评估→人工门禁→发布”的候选流程，并故意在每步制造一次失败。
Orchestrate ingestion → indexing → retrieval evaluation → human gate → release, deliberately failing each stage once.
验收：Acceptance:失败可从最近安全点恢复，不重复副作用；每次运行有输入、输出、版本和负责人。Failures resume from the last safe point without duplicate effects; every run records input, output, version, and owner.
自测：为什么不应把所有 Python 业务逻辑直接写进编排平台的内联脚本？Self-check: Why avoid placing all Python business logic in orchestrator inline scripts?
It traps testing, reuse, and migration in the platform. Keep business steps independently callable and tested; let the orchestrator manage dependency, parameters, schedule, and state.
评估告诉你坏没坏，监控告诉你何时坏Evaluation says whether; monitoring says when
从固定数据集走到线上质量、漂移和成本信号。
Move from fixed datasets to live quality, drift, and cost signals.
本册作品里程碑Portfolio milestone
交付 80 题门槛、SLO、仪表盘和报警手册。
Deliver an 80-case gate, SLOs, dashboard, and alert runbook.
04
模块 4Module 4
评估：分别测检索、回答、系统与业务结果Evaluation
先回答一个白话问题Start with a plain question
RAG 演示回答过一个问题，怎样证明它面对真实问题仍能稳定找到证据并忠实回答？
A RAG demo answers one question. How do we prove it consistently retrieves evidence and answers faithfully on real questions?
类比桥梁Analogy bridge
像验收医院：不能只看医生一次答对，还要分别测分诊、检查、诊断、用药和患者结果。
Like evaluating a hospital: triage, tests, diagnosis, treatment, and outcomes must be measured separately.
类比边界：类比只帮助识别关系；真正系统仍由数据、代码、接口和运行环境决定。Where it breaks: The analogy only exposes relationships; the real system is determined by data, code, interfaces, and its runtime.
Build a dataset with relevant evidence and expected behavior; measure recall/rank for retrieval, faithfulness/relevance for generation, then latency, cost, abstention, and safety for the system.
例子 1 · 检索命中Example 1 · Retrieval hit
输入Input
问题与标注相关段落Question with labeled relevant passage
过程Process
检查 passage 是否进入 top-kCheck whether passage appears in top-k
结果 / 验收Result / check
发现回答错是检索没找到Find that the answer failed at retrieval
例子 2 · 忠实度Example 2 · Faithfulness
输入Input
答案与实际检索上下文Answer and retrieved context
过程Process
逐主张检查是否被证据支持Check each claim against evidence
结果 / 验收Result / check
流畅但新增数字被判失败Fluent invented numbers fail
例子 3 · 拒答评估Example 3 · Abstention evaluation
输入Input
知识库外问题Out-of-scope question
过程Process
期望返回“无证据”Expect “no evidence”
结果 / 验收Result / check
拒答正确也计为成功Correct abstention counts as success
例子 4 · 对抗样本Example 4 · Adversarial case
输入Input
文档含提示注入Document contains prompt injection
过程Process
检查系统是否仍只当数据Check that it remains data only
结果 / 验收Result / check
工具策略不被外部文本改变Tool policy is unchanged
例子 5 · 成对比较Example 5 · Pairwise comparison
输入Input
旧提示与新提示输出Old and new prompt outputs
过程Process
盲评顺序随机化Blind comparison with randomized order
结果 / 验收Result / check
减少位置和版本偏见Reduce position and version bias
例子 6 · 统计波动Example 6 · Statistical variation
输入Input
同一版本多次运行Repeated runs of one version
过程Process
报告均值、分布与置信区间Report mean, distribution, and confidence interval
结果 / 验收Result / check
不把一次提升当规律One improvement is not treated as a law
常见误区Common misconception
用 LLM judge 一个总分覆盖全部质量；评判模型也会偏，且总分会掩盖安全退化。
Using one LLM-judge score for all quality; judges are biased and aggregate scores hide safety regressions.
作品任务Portfolio task
构建 80 题评估集：常规、空证据、冲突、过时、注入和权限场景；建立组件级与端到端报告。
Build an 80-question set covering normal, no-evidence, conflict, stale, injection, and permission cases, with component and end-to-end reports.
验收：Acceptance:任何变更都能定位是检索、生成、权限还是性能变化；关键安全项不得被平均分抵消。Every change localizes retrieval, generation, permission, or performance effects; critical safety failures cannot be averaged away.
自测：回答完全正确，但检索到的是过时政策；这次评估应算通过吗？Self-check: The answer is correct, but retrieval used an obsolete policy. Should the evaluation pass?
Not as a reliable system. It may be accidentally correct while provenance and version control failed. Record answer correctness separately from source/retrieval failure.
Without ground truth for every live request, how do we detect RAG degradation before complaints?
类比桥梁Analogy bridge
像驾驶仪表盘：油量、温度、警报和路线偏差一起看，单看发动机“还在转”不够。
Like a dashboard: fuel, temperature, alarms, and route deviation matter; “the engine is running” is insufficient.
类比边界：类比只帮助识别关系；真正系统仍由数据、代码、接口和运行环境决定。Where it breaks: The analogy only exposes relationships; the real system is determined by data, code, interfaces, and its runtime.
把工程边界装进真正能演示的作品Package engineering boundaries into a demonstrable project
最佳实践只保留有风险证据的部分，最终项目展示成功与失败。
Keep best practices backed by risk evidence and demonstrate both success and failure.
本册作品里程碑Portfolio milestone
交付可重建、可评估、可监控、可讲述的端到端作品。
Deliver a reproducible, evaluable, observable, explainable project.
06
模块 6Module 6
最佳实践：可靠性、安全、性能与成本的系统清单Best Practices (Optional)
先回答一个白话问题Start with a plain question
当原型已经能回答问题，哪些工程措施能把“偶尔成功”变成可维护服务？
Once a prototype answers questions, which engineering practices turn occasional success into a maintainable service?
类比桥梁Analogy bridge
像把家庭厨房升级为餐厅：卫生、备份、库存、峰值流量和责任分工都必须标准化。
Like turning a home kitchen into a restaurant: hygiene, backup, inventory, peak load, and ownership need standards.
类比边界：类比只帮助识别关系；真正系统仍由数据、代码、接口和运行环境决定。Where it breaks: The analogy only exposes relationships; the real system is determined by data, code, interfaces, and its runtime.
A full answer binds model, prompt, permission, and source state. Any change can make it stale or unauthorized. Retrieval candidates also need versioning but allow regeneration and verification.
端到端项目：把知识库问答做成可展示作品End-to-End Project Example (Optional)
先回答一个白话问题Start with a plain question
怎样把文档、检索、Agent、评估、监控和界面整合成面试时能演示、能被追问的完整项目？
How do we combine documents, retrieval, agents, evaluation, monitoring, and UI into a portfolio project that survives interview questions?
类比桥梁Analogy bridge
像毕业作品展：不只摆成品，还要展示草图、测试、失败修复和为什么这样设计。
Like a graduation exhibition: show the artifact, sketches, tests, failures, and design decisions.
类比边界：类比只帮助识别关系；真正系统仍由数据、代码、接口和运行环境决定。Where it breaks: The analogy only exposes relationships; the real system is determined by data, code, interfaces, and its runtime.
Start with problem and data boundaries, then modular implementation, fixed evaluation, containers, demo data, monitoring, and README; no real secrets or private data enter the repository.
例子 1 · 问题卡Example 1 · Problem card
输入Input
目标用户与知识库Target user and knowledge base
过程Process
定义问题、错误成本与禁止用途Define task, error costs, and prohibited uses
结果 / 验收Result / check
项目不是“做一个 RAG”空话The project is not the vague “build a RAG”
例子 2 · 架构图Example 2 · Architecture diagram
输入Input
摄取到回答的组件Components from ingestion to answer
过程Process
标数据、版本、权限和故障Mark data, versions, permissions, and failures
结果 / 验收Result / check
面试时能解释每条边Every edge is explainable in an interview
例子 3 · 演示脚本Example 3 · Demo script
输入Input
三个成功与两个失败场景Three success and two failure scenarios
过程Process
固定演示数据并预期输出Fix demo data and expected behavior
结果 / 验收Result / check
失败处理也能现场展示Failure handling is demonstrable
例子 4 · 测试证据Example 4 · Test evidence
输入Input
检索、回答、攻击与 E2ERetrieval, answer, attack, and E2E tests
过程Process
CI 生成版本化报告CI generates a versioned report
结果 / 验收Result / check
改动是否退化一眼可见Regressions are visible
例子 5 · 部署包Example 5 · Deployment package
输入Input
API、UI、索引构建器API, UI, and index builder
过程Process
容器和配置示例一键启动One-command start via containers and example config
结果 / 验收Result / check
评审者从空环境可重建A reviewer rebuilds from a clean environment
例子 6 · 项目叙述Example 6 · Project story
输入Input
一次检索退化事故One retrieval regression
过程Process
用问题—证据—选择—验证复盘Explain problem, evidence, choice, and verification
结果 / 验收Result / check
90 秒讲清真实贡献与取舍Explain real contribution and tradeoffs in 90 seconds
常见误区Common misconception
为了演示顺滑只保留成功路径，面试官一问“服务挂了或没搜到怎么办”就暴露项目只是拼装。
Keeping only the happy path for a smooth demo; one question about outages or empty retrieval reveals a stitched prototype.
Deliver Job Knowledge Copilot V1 with public/self-authored demo data, hybrid retrieval, bounded agent, 80-case eval, monitoring dashboard, containers, and README.
验收：Acceptance:新评审者 15 分钟内启动；演示五场景；每个数字可追溯；失败可诊断；无秘密与私人数据。A new reviewer starts it in 15 minutes, runs five scenarios, traces every number, diagnoses failures, and finds no secrets or private data.
自测：作品集最有说服力的是“用了多少技术”还是“如何证明它可靠”？Self-check: What is more persuasive in a portfolio: how many technologies were used or how reliability was proven?
是后者。清楚的问题、基线、失败案例、评估、监控、取舍和可复现步骤，比技术名堆叠更能证明你理解项目。
The latter. Clear framing, baselines, failures, evaluation, monitoring, tradeoffs, and reproducibility prove understanding better than a technology list.
Source: DataTalksClub llm-zoomcamp. The current tree has seven modules; modules 6 and 7 are marked optional but remain fully covered. Public readability is not an open license, so this is an original conceptual reconstruction with factual attribution links.