Production Monitoring

Curriculum75 min

• Production Monitoring Foundations
• Concept | Process governance for MLOps
• Concept | Model comparisons
• Concept | Model evaluation stores
• Concept | Monitoring model performance and drift in production
• Model Monitoring with a Model Evaluation Store
• Get started
• Create two model monitoring pipelines
• Compare and contrast model monitoring pipelines
• Run more model evaluations
• Conduct drift analysis
• Automate model monitoring
• Create additional model monitoring assets
• Model Monitoring Feedback Loops
• Concept | Monitoring and feedback in the AI project lifecycle
• Tutorial | API endpoint monitoring
• Optional: Production Deployment Automation
• Get started
• Start with a retrain model scenario
• Add a Create bundle or API service version step
• Add an Update project or API deployment step
• Run the scenario & observe the outcome
• Plan for a more robust setup
• Optional: Model Monitoring in Different Contexts
• Get started
• A batch workflow within Dataiku
• An API endpoint on a Dataiku API node
• An exported Python model scored externally
• An exported Java model scored externally
• What's next?
• Wrap Up
• Course checkpoint | Production Monitoring
• Tell us what you think | Production Monitoring
• Course complete | Production Monitoring