https://blog.lameiro0x.com/tags/langkit/Recent content in Langkit on Miguel Lameiro | Cybersecurity Blog & Security WriteupsHugo -- 0.161.1en-usThu, 07 May 2026 00:00:00 +0000https://blog.lameiro0x.com/notes/ai-security/measuring-quality-and-safety-in-llm-applications/Thu, 07 May 2026 00:00:00 +0000https://blog.lameiro0x.com/notes/ai-security/measuring-quality-and-safety-in-llm-applications/<h2 id="introduction">Introduction</h2> <p>Before adding controls to an LLM application, you need to know what is happening. Quality and safety measurement gives you that visibility.</p> <p>This course focuses on metrics and monitoring rather than runtime blocking. It uses chat datasets, WhyLogs, LangKit, custom UDFs, model-based scoring, and active monitoring patterns to inspect hallucinations, data leakage, toxicity, refusals, and prompt injection.</p> <p>The useful lesson is practical: do not treat &ldquo;the model seems fine&rdquo; as evidence. Log the interactions, compute signals, inspect critical examples, and evaluate filtered subsets.</p>