Course URL: https://www.coursera.org/learn/introduction-to-machine-learning-in-production/
Co instructors from the Tensorflow team:
- Robert Crowe
- Laurence Moroney (AI Advocacy; “AI and machine learning for coders”)
- Data drift - distribution of data has changed.
- overview of entire lifecycle of production ml project - scoping, development, deployment
- evolution of data pipelines. TFX. data provanance
- ml models in production; minimize costs, analytics, explainability
- deployment : serve users request. deployment pipelines. diff infras. best practices.. continuously operating pipelines.
- concept drift or data drift
- holdout test set
- proof of concept
- jupyter notebook
- POC to production gap; sheer amount of work. the doughut around the hole.
- D Sculley et. al NIPS 2015: Hidden technical debt in machine learning systems.
- “Other pieces of software” for production Ml systems.
- Framework - systematically planout the lifecycle.
STEPS OF FULL LIFECYCLE ML PROJECTS
- scoping – define project
- data – define and establish baseline, label and organize data
- modeling – select and train model, platform error analysis. (iterative process)
- Deployment – deploy in prod, monitor and maintain system.
(diagram) Learning AI’s Steven Layett and Daniel Pipryata.
LandingLens - MLOps -
- Key challenges in building production ready MODELS
- model centric ai dev vs data centric ai dev
- AI system = code + data
- model development = model + hyperparameters + data
- LandingLens - Error Analysis
Define data and establish baseline. eg: iguana labeling, phone scratch/pitmark.
Label Ambiguity - combinatorially different ways to transcribe.