| Item | Details |
|---|
| Course Title | AI Systems (Machine Learning Systems: Agentic Workflows and Harness Engineering) |
| Course Code | AI4001 |
| Credits | 3 credits (2 lecture + 1 lab) |
| Target Students | AI Department, 4th year |
| Contact Hours | 3 hours/week × 16 weeks |
| Instructor | Young Joon Lee (yj.lee@chu.ac.kr) |
| Classroom | AI Lab (DGX H100 Cluster) |
| Course Site | https://ai-systems-2026.halla.ai/ |
The AI industry of 2025–2026 has decisively shifted from conversational models to autonomous agentic systems. This course is organized around two central themes: harness engineering — the discipline of controlling non-deterministic AI agents with deterministic systems — and the Human-on-the-Loop (HOTL) paradigm, in which humans take on strategic oversight rather than step-by-step approval.
Students will directly operate the NVIDIA DGX H100 server in the Cheju Halla University AI Lab (partitioned via MIG technology), gaining hands-on experience deploying and running open-source models such as DeepSeek-Coder-V2 without relying on commercial APIs. The ultimate goal is to design an autonomous software development pipeline (multi-agent SDLC) and implement it in the capstone project Ralphthon.
Upon completing this course, students will be able to:
- Explain the differences among HITL, HOTL, and HIC governance architectures and implement them as Governance-as-Code
- Design and implement harness engineering systems using the Ralph Loop methodology
- Optimize long-running agent loops through context window management and instruction tuning
- Build a multi-agent SDLC by connecting planner, coder, and QA agents via MCP
- Deploy open-source LLMs in a DGX H100 + MIG + vLLM environment and measure their performance
- Automatically evaluate the quality of agent systems using LLM-as-Judge and telemetry
| Week | Theory | Lab |
|---|
| Week 1 | Course orientation, AI system paradigm shift, HITL vs HOTL | Dev environment setup, AI coding CLI installation (Lab 01) |
| Week 2 | HOTL governance in depth, EU AI Act compliance, Governance-as-Code | Implementing the first agentic loop (Lab 02) |
| Week 3 | MCP architecture deep dive, TBAC/governance gateway, MIG compute isolation | MCP server implementation and security verification (Lab 03) |
| Week | Theory | Lab |
|---|
| Week 4 | Loop paradigms — Test-time Compute Scaling, Ralph Loop/RLM/autoresearch | Ralph Loop implementation and cumulative learning (Lab 04) |
| Week 5 | Context window management, preventing Context Rot, state tracking file design | Implementing a context management system (Lab 05) |
| Week 6 | Instruction tuning, the “Sign” metaphor, designing permanent context payloads | PROMPT.md tuning practice (Lab 06) |
| Week | Theory | Lab |
|---|
| Week 7 | Agent role division, traditional SDLC vs agentic SDLC | Multi-agent pipeline design (Lab 07) |
| Week 8 | Project proposal presentation (midterm replacement) | Individual capstone proposal presentation — no exam |
| Week 9 | Planner agent (2-phase separation), QA independence, automated test pipeline | Planner + QA agent implementation (Lab 09) |
| Week | Theory | Lab |
|---|
| Week 10 | DeepSeek-Coder-V2 architecture, open-source vs commercial API, tool ecosystem | vLLM deployment practice (Lab 10) |
| Week 11 | vLLM high-throughput inference, CUDA optimization, MIG slice utilization | Building a high-throughput inference server |
| Week 12 | Telemetry design, LLM-as-Judge evaluation framework, cost optimization | Telemetry & LLM-as-Judge (Labs 11 & 12) |
| Week | Content |
|---|
| Week 13 | Individual project kickoff, architecture design document (expanding the Week 8 proposal) |
| Week 14 | Ralphthon execution — harness implementation, agent integration, iterative improvement |
| Week 15 | System integration, automated testing, presentation preparation |
| Week 16 | Final presentation and demo, peer evaluation, course wrap-up |
| Item | Weight | Description |
|---|
| Lab Assignments (Lab 01–12) | 40% | Individual submission via GitHub PR |
| Midterm Project | 20% | Individual project proposal presentation (Week 8, midterm replacement) |
| Capstone Ralphthon | 30% | Individual project with final presentation |
| Contribution & Participation | 10% | GitHub PR contributions, class discussion |
- Huntley, G. (2025). The Ralph Loop: Deterministic Agentic Engineering
- Anthropic. (2026). Claude Code Documentation
- NVIDIA. (2025). DGX H100 MIG Configuration Guide
- Karpathy, A. (2025). autoresearch: Automated ML Research
- Zhang, A. et al. (2025). Recursive LM: Language Models that Call Themselves
- Snell, C. et al. (2024). Scaling LLM Test-Time Compute Optimally
- Hong, S. et al. (2024). MetaGPT: Meta Programming for Multi-Agent Collaborative Framework (ICLR 2024)
- Qian, C. et al. (2024). ChatDev: Communicative Agents for Software Development (ACL 2024)
- Damani, S. et al. (2025). Towards a Science of Scaling Agent Systems (DeepMind + MIT)
- Python 3.10+ programming proficiency
- Basic Git/GitHub usage
- Linux command-line fundamentals
- Machine learning basics (prerequisite courses: AI Fundamentals, Deep Learning)
For course-related inquiries, please reach out via GitHub Issue or email (yj.lee@chu.ac.kr).
If you find errors in course materials, please report them via GitHub Issue.
