Building a machine learning model is hard. Building and serving individual machine learning models for thousands of customers every single day is harder. Model training time must be kept to a minimum, memory constraints are exacerbated and the love and insight that one would usually apply to refining a single model must be automated and scaled. At Zendesk we successfully use pandas, scikit-learn, flask and hadoop to run a machine learning ecosystem that builds, tunes and serves account specific models via an API for many of our customers. This talk is about what we learnt building it and how Python's data stack let us get it to production really really fast.