Playbooks
Opinionated, copy‑and‑adapt guides for common MLOps scenarios. Use these as starting points, not as rigid standards.
CI/CDMonitoringLLM & MCP
CI/CD for an ML service
PlaybookTurn a single ML model service into a pipeline with linting, tests, and safe deploys.
Checklist
- Define one Git repo as the source of truth for the service.
- Add lint, typecheck and test commands (`npm run lint`, `npm run test`).
- Create a minimal GitHub Actions workflow that runs on PR and `main`.
- Fail the pipeline on any lint/test/typecheck error.
- Build a container image or artifact only on `main` or tagged releases.
Where to start
Use the Continuous Integration page for a more detailed breakdown and copy‑pasteable workflow example.
Monitoring & drift signals
PlaybookDefine what you monitor for a model in production before you deploy it.
Signals to define
- Request latency: P50 / P95 / P99.
- Error rate: HTTP and application‑level errors.
- Business metric: e.g. approval rate, fraud catch rate.
- Data quality: missing values, ranges, category cardinality.
- Drift: feature distributions vs. training data.
How to apply
Start by writing these down for one model using the Production environment mindset, then wire them into your existing monitoring stack (Prometheus, cloud metrics, etc.).
MCP & tools integration
PlaybookExpose your existing tools and data sources to LLMs using Model Context Protocol.
High‑level steps
- Pick 1–2 high‑value tools (e.g. metrics query, model registry lookup).
- Implement a small MCP server that wraps those tools.
- Expose safe parameters only (no raw SQL from the model).
- Log all MCP calls for later analysis and debugging.
Where to learn more
The MCP page under the Development environment walks through core concepts, MLOps benefits, and example server/client code.