https://shengxu.pages.dev/en/tags/aiops/Cloud architecture & DevOps notes by Shengxu: Kubernetes, Cilium, observability, LLM infra, AI agents.Hugo 0.153.2 & FixIt v0.4.0-alpha.3-20251225101113-8ffb9a95enMon, 19 Jan 2026 15:00:00 +0800https://shengxu.pages.dev/en/posts/llm-observability-guide-2026/Mon, 19 Jan 2026 15:00:00 +0800https://shengxu.pages.dev/en/posts/llm-observability-guide-2026/AIObservability<p>As large language models (LLMs) evolve from &ldquo;novelty toys&rdquo; into the &ldquo;productivity backbone&rdquo; of enterprises, a question that every technical leader keeps coming back to has surfaced: <strong>When API calls become a black box, how do we manage these massive, expensive AI models with the same rigor we apply to databases or microservices?</strong></p> <p>If 2024 was the year everyone was busy &ldquo;getting demos to work,&rdquo; then 2026 marks the dawn of &ldquo;fine-grained governance.&rdquo; The simple &ldquo;call succeeded/failed&rdquo; logs of the past can no longer answer today&rsquo;s complex operational questions: <em>&ldquo;Why was this agent so smart yesterday, but today it&rsquo;s spouting nonsense?&rdquo;</em>, <em>&ldquo;Why did our token costs suddenly double last month?&rdquo;</em>, <em>&ldquo;Is someone trying to attack our customer service bot with a prompt injection?&rdquo;</em></p>