15 - 16 June 2022

MLOps Summit MLOps Summit schedule

MLOps Summit San Francisco



Download PDF
  • 08:00

    Coffee & Registration

  • 09:00

    MLOps Stage: Chair Welcome

  • 09:15
    Adam Kraft

    Where Are We With AutoML?

    Adam Kraft - Machine Learning Engineer - Google Brain

    Down arrow blue

    Where are we With AutoML?

    AutoML aims to help everyone achieve state-of-the-art AI for their specific problems. Techniques such as Neural Architecture Search (NAS) push the boundary of finding efficient and high quality models. How is AutoML being used today and where is the field headed in the future? This talk gives an overview of AutoML techniques, exploring how they work and the challenges of applying them across different AI tasks and settings.

    Adam Kraft is a machine learning engineer on the Google Brain Team, working on AutoML for a wide variety of AI tasks. Before Google, Adam spent eight years in computer vision and machine learning, working with satellite imagery at Orbital Insight and helping customers shop with their camera phones at Amazon.

    Twitter Linkedin
  • 09:50

    MLOps: Summit Presentation with Sama

  • 10:15
    Emily Curtin

    Making Code and Humans GPU-Capable at Mailchimp

    Emily Curtin - Senior Machine Learning Engineer - Mailchimp

    Down arrow blue

    Making Code and Humans GPU-Capable at Mailchimp

    What happens when you have a bunch of data scientists, a bunch of new and old projects, a big grab-bag of runtime environments, and you need to get all those humans and all that code access to GPUs? Come see how the ML Eng team at Mailchimp wrestled first with connecting abstract containerized processes to very-not-abstract hardware, then scaled that process across tons of humans and projects. We’ll talk through the technical how-to with Docker, Nvidia, and Kubernetes, but all good ML Engineers know that wrangling the tech is only half the battle and the human factors can be the trickiest part.

    3 Key Takeaways • An overview of the call stack from container, orchestration framework, OS, and all the way down to real GPU hardware • How ML Eng at Mailchimp provides GPU-compatible dev environments for many different projects and data scientists • An experienced take on how to balance data scientist’s human needs against heavy system optimization (spoiler alert: favor the humans)

    Emily May Curtin is a Senior Machine Learning Platform Engineer at Mailchimp, which is definitely what she thought she’d be doing back when she went to film school. She combines her wealth of experience in DevOps, data engineering, distributed systems, and “cloud stuff” to enable Data Scientists at Mailchimp to do their best work. Truthfully, she’d rather be at her easel painting hurricanes and UFOs. Emily lives (and paints) in her hometown of Atlanta, GA, the best city in the world, with her husband Ryan who’s a pretty cool guy.

    Twitter Linkedin
  • 10:45

    Morning Break

  • 11:00

    Models as Business Assets

  • Mac Macoy

    SPEAKER

    Mac Macoy - Senior Software Engineer - MLOps - Chick-fil-A

    Down arrow blue

    Models as Business Assets

    Linkedin
  • Jimmy Simmons

    SPEAKER

    Jimmy Simmons - Principal Team Leader, Machine Learning Operations and Big Data - Chick-fil-A

    Linkedin
  • 11:35

    Summit Presentation - Best Practices for High Quality Data

  • 12:00

    Panel Discussion: Rolling Out the Practice of MLOps Organizationally

  • Korri Jones

    PANELIST

    Korri Jones - Senior Lead Machine Learning Engineer - Chick-fil-A

    Down arrow blue

    Korri Jones is a Sr Lead Machine Learning Engineer and Innovation Coach at Chick-fil-A, Inc. in Atlanta, Georgia where he is focused on MLOps. Prior to his work at Chick-fil-A, he worked as a Business Analyst and Product Trainer for NavMD, Inc., was an adjunct professor at Roane State Community College, and instructor for the Project GRAD summer program at Pellissippi State Community College and the University of Tennessee, Knoxville. His accolades are just as diverse, and he was in the inaugural 40 under 40 for the University of Tennessee in 2021, Volunteer of the year with the Urban League of Greater Atlanta with over 1000 hours in a single calendar year and has received the “Looking to the Future” award within his department at Chick-fil-A among many others, including best speaker awards in business case competitions.

    Twitter Linkedin
  • Supreet Kaur

    PANELIST

    Supreet Kaur - Assistant Vice President - Morgan Stanley

    Down arrow blue

    MLOps/Trusted AI Summit: Closing General Session: Complexity vs Simplicity in ML and AI Projects

    Women in AI Reception: Pivoting into AI

    Supreet is an AVP at Morgan Stanley. Prior to Morgan Stanley, she was a management consultant at ZS Associates where she automated different workflows and built data driven solutions for fortune 500 clients. She is extremely passionate about technology and AI and hence started her own community called DataBuzz where she engages the audience by sharing the latest AI and Tech trends and also mentors people who want to pivot in this field.

  • Lakshmi Ravi

    PANELIST

    Lakshmi Ravi - Applied Scientist - Amazon

    Down arrow blue

    Selecting ML Algorithms and Validating

    ML Practitioners often have a dilemma in identifying the right ML Model for the problem space. In this talk, I will be going over the common questions that will help in narrowing down the right next step. Developed model will have to meet certain validation metrics. The next common question is how the validation metrics proposed by scientists will have to be explained to business leaders and help them decide if the model is eligible to deployed. The next step is to find mechanisms to develop and study the online validation metrics. Often online metrics of an ML model launched will require studying the results in Treatment-Control fashion. In this talk, I will describe common development practices that helps in A/B testing of experiments.

    Lakshmi is an Applied Scientist with Amazon.She has been working with Amazon Machine Learning teams for the last 4.5 years. She had the chance to be part of Alexa's NLP team, Behavior Analytics (a causal Inference division in Amazon) and Amazon Music teams (improving the voice experience in Alexa).

    Linkedin
  • 12:45

    Lunch

  • 13:45

    A Model Development Journey

  • 14:20

    Round Table Discussions

  • Shilpi Agarwal

    Round Table Topic Leader: Data Ethics in Business - The Cornerstone of Customer Trust

    Shilpi Agarwal - Founder & Chief Data Ethics Officer - DataEthics4All

    Down arrow blue

    Shilpi Agarwal is a Data Philanthropist, Adjunct Faculty at Stanford and MIT $100K Launch Mentor.

    Armed with the technical skills from her Bachelor of Engineering in Computer Science, design thinking skills from her Masters in Design, combined with 20+ years of Business and Marketing know-how by working as a Marketing Consultant for some really big and some small brands, Shilpi started DataEthics4All, troubled with the unethical use of data around her on social media, in business and in political campaigns.

    DataEthics4All is a Community bringing the STEAM in AIᵀᴹ Movement for Youth and celebrating Ethics 1stᵀᴹ Champions of today and tomorrow pledging to help 5 Million economically disadvantaged students in the next 5 years by breaking barriers of entry in tech and creating awareness on the ethical use of data in data science and artificial intelligence in enterprise, working towards a better Data and AI World.

    Twitter Linkedin
  • Subramanian "Subbu" Iyer

    Round Table Topic Leader: AI vs Simpler Solutions

    Subramanian "Subbu" Iyer - Sr. Director of AI - Target

    Linkedin
  • Ban Kawas

    Round Table Topic Leader: Explainable AI (XAI) and Its Role in Building Trusted AI

    Ban Kawas - Senior Research Scientist - Reinforcement Learning - Meta

    Down arrow blue

    Ban is a Senior AI Research Scientist at Meta. She is working on democratizing Reinforcement Learning and enabling its use in the real world, spanning several application areas from compiler optimization to embodied AI. Ban and her team are developing ReAgent; an end-to-end platform for applied RL, checkout open source version at https://reagent.ai/

    Linkedin
  • Naman Kohli

    Round Table Topic Leader: Causal Analysis

    Naman Kohli - Applied Scientist - Amazon

    Linkedin
  • Lakshmi Ravi

    Round Table Topic Leader: Causal Analysis

    Lakshmi Ravi - Applied Scientist - Amazon

    Down arrow blue

    Selecting ML Algorithms and Validating

    ML Practitioners often have a dilemma in identifying the right ML Model for the problem space. In this talk, I will be going over the common questions that will help in narrowing down the right next step. Developed model will have to meet certain validation metrics. The next common question is how the validation metrics proposed by scientists will have to be explained to business leaders and help them decide if the model is eligible to deployed. The next step is to find mechanisms to develop and study the online validation metrics. Often online metrics of an ML model launched will require studying the results in Treatment-Control fashion. In this talk, I will describe common development practices that helps in A/B testing of experiments.

    Lakshmi is an Applied Scientist with Amazon.She has been working with Amazon Machine Learning teams for the last 4.5 years. She had the chance to be part of Alexa's NLP team, Behavior Analytics (a causal Inference division in Amazon) and Amazon Music teams (improving the voice experience in Alexa).

    Linkedin
  • 15:15

    Afternoon Networking Break

  • 15:45
    Jerry Xu

    ML at Scale

    Jerry Xu - Architect for Business Integrity's Machine Learning Infrastructure - Facebook

    Down arrow blue

    Jerry Xu is the Architect for Business Integrity's Machine Learning Infrastructure at Facebook. Jerry has over 20 years of experience building high-performance and large-scale systems with multiple engineering and leadership positions at Lyft, Box, Twitter, Zynga, and Microsoft. Before joining Facebook, he was the co-founder and CEO at Datatron Technologies, a pioneer in MLOps automation. Jerry holds several patterns on machine learning and data storage.

    Linkedin
  • 16:20

    Summit Presentation - Model Management, Deployment, Lineage & Monitoring

  • 16:45
    David Liu

    Taming Signals, Features, and Training Datasets: How Data Management Shaped Pinterest ML

    David Liu - Head of ML Platform - Pinterest

    Down arrow blue

    Taming Signals, Features, and Training Datasets: How data management shaped Pinterest ML

    This talk examines how Pinterest evolved its handling of three types of data -- raw signals, ML features and ML training datasets -- and the effects on ML practitioners at Pinterest. Data management is the core complexity of production ML engineering, especially in Web-scale applications with billions of entities and training examples. Pinterest “signals,” or raw data about Pins, boards, and other entities, started as monolithic datasets that grew unmaintainable. We split them into individually owned datasets on a standardized “Signal Platform,” improving governance around lineage, ownership, and monitoring. We standardized ML features from highly custom formats to a flat “Unified Feature Representation,” enabling a shared feature store and model inference. Finally, we are transitioning ML training datasets from ad-hoc row-oriented datasets to standardized columnar table groups, enabling improved storage efficiency and shared training pipelines in the future.

    David is the Head of ML Platform at Pinterest, which comprises ML Data, ML Training, and ML Serving teams. These teams provide infrastructure for 200 engineers and data scientists for applications spanning ads, recommendations, search, and trust/safety, handling billions of events per day. Previously at Pinterest, David also started the Related Pins recommendations and visual search teams and built one of the first ML-based recommender systems at Pinterest. He holds a bachelor's and master's degree in computer science from Stanford.

    Linkedin
  • 17:15

    Networking Reception

  • 18:15

    End of Day One

  • THIS EVENT STARTS AT 8:45

  • 08:45

    Coffee & Registration

  • 09:45

    MLOps Stage: Chair Welcome

  • 10:00
    Zachary Hanif

    Beyond MLOps: A Closer Look at Operational Machine Learning & Why it Matters

    Zachary Hanif - Vice President of Machine Learning - Capital One

    Down arrow blue

    Beyond MLOps: A Closer Look at Operational Machine Learning & Why it Matters

    While critical aspects of MLOps like infrastructure, CI/CD pipelines, logging and monitoring are much discussed in the machine learning world today, there has been less of a focus on the actual operations of machine learning. This talk will explore key topics and questions related to the operational considerations of developing and deploying machine learning models in real-world environments, including: how to respond when something goes awry; data-centric approaches to model development and how to course-correct for suboptimal training data; and what happens when novel external environments (e.g. a global pandemic) create outputs that run counter to developers’ expectations.

    Zachary Hanif is Capital One’s Vice President of Machine Learning, currently working to democratize and de-risk access to modeling capabilities across a data-driven enterprise. He has a career background focused on applications of machine learning and advanced modeling innovation, research, and product delivery across diverse problem domains in startups and large enterprises. Zachary is passionate about identifying novel uses of machine learning and large-scale distributed systems technologies that generate measurable value, pragmatic innovation, and ensuring that technical solutions, particularly those leveraging AI/ML, are responsible, ethically sound, and robust to challenge.

    Twitter Linkedin
  • 10:35

    Summit Presentation - MLOps in the Real World

  • 11:00

    Morning Break

  • 11:15
    Lakshmi Ravi

    Selecting ML Algorithms and Validating

    Lakshmi Ravi - Applied Scientist - Amazon

    Down arrow blue

    Selecting ML Algorithms and Validating

    ML Practitioners often have a dilemma in identifying the right ML Model for the problem space. In this talk, I will be going over the common questions that will help in narrowing down the right next step. Developed model will have to meet certain validation metrics. The next common question is how the validation metrics proposed by scientists will have to be explained to business leaders and help them decide if the model is eligible to deployed. The next step is to find mechanisms to develop and study the online validation metrics. Often online metrics of an ML model launched will require studying the results in Treatment-Control fashion. In this talk, I will describe common development practices that helps in A/B testing of experiments.

    Lakshmi is an Applied Scientist with Amazon.She has been working with Amazon Machine Learning teams for the last 4.5 years. She had the chance to be part of Alexa's NLP team, Behavior Analytics (a causal Inference division in Amazon) and Amazon Music teams (improving the voice experience in Alexa).

    Linkedin
  • 11:50

    Summit Presentation - Peak Performance Tips for ML Models

  • 12:15

    Lunch

  • 13:15
    Hiranmayi Ranganathan

    Data-Driven Modeling Approaches in Computational Drug Discovery

    Hiranmayi Ranganathan - Machine Learning Specialist - Accelerating Therapeutics for Opportunities in Medicine (ATOM)

    Down arrow blue

    Data-Driven Modeling Approaches in Computational Drug Discovery

    This session will introduce ATOM ( Accelerated Therapeutics for Opportunities in Medicine) and the work going on in the consortium. We will then talk about the issues around building predictive models for small molecule drug discovery, focusing on both target specific drug discovery approaches and structure based multi-target modeling.

    Some topics covered include: - Drug Discovery Cheminformatics Models - Introduction to Quantitative Structure Activity Relationships (QSAR) - ATOM Modeling Pipeline - Structural models

    Linkedin
  • 13:50
    Diego Klabjan

    MLOps for Deep Learning

    Diego Klabjan - Professor - Northwestern University

    Down arrow blue

    MLOps for Deep Learning

    In model serving, two important decisions are when to retrain the model and how to efficiently retrain it. Having one fixed model during the entire often life-long inference process is usually detrimental to model performance, as data distribution evolves over time, resulting in a lack of reliability of the model trained on historical data. It is important to detect drift and retrain the model in time. We present an ensemble drift detection technique utilizing three different signals to capture data and concept drifts. In a practical scenario, ground truth labels of samples are received after a lag in time, which we consider appropriate. Our framework automatically decides what data to use to retrain based on the signals. It also triggers a warning indicating a likelihood of drift.

    Model training in serving is not a one-time task but an incremental learning process. We address two challenges of life-long retraining: catastrophic forgetting and efficient retraining. To solve these two issues, we design a retraining model that can select important samples and important weights utilizing multi-armed bandits. To further address forgetting, we propose a new regularization term focusing on synapse and neuron importance.

    Only a significant minority of companies unlock the true potential of AI as trained models accumulate dust due to challenges in MLOps. Serving reliable AI predictions to customers involves cost, effort, and planning to set up a continuous deployment pipeline. MLOps for Deep Learning demands a carefully crafted deployment pipeline. We discuss our open-source project which is a robust continuous deployment pipeline by integrating our unique drift detection and model retrain algorithms for serving DL models. We show how to efficiently deploy, monitor, and maintain DL models in production using our solution which is a Kubernetes native POC solution.

    Diego Klabjan is a professor at Northwestern University, Department of Industrial Engineering and Management Sciences. He is also Founding Director, Master of Science in Analytics, and the Deep Learning Lab. His expertise is focused on data science and deep learning with a concentration in finance, insurance, and healthcare. Professor Klabjan has led projects with large companies such as The Chicago Mercantile Exchange Group, Intel, General Motors and many other, and he is also assisting numerous start-ups with their analytics needs. He is also a founder of Opex Analytics.

    Twitter Linkedin
  • 14:25
    Supreet Kaur

    Closing General Session: Complexity vs Simplicity in ML and AI Projects

    Supreet Kaur - Assistant Vice President - Morgan Stanley

    Down arrow blue

    MLOps/Trusted AI Summit: Closing General Session: Complexity vs Simplicity in ML and AI Projects

    Women in AI Reception: Pivoting into AI

    Supreet is an AVP at Morgan Stanley. Prior to Morgan Stanley, she was a management consultant at ZS Associates where she automated different workflows and built data driven solutions for fortune 500 clients. She is extremely passionate about technology and AI and hence started her own community called DataBuzz where she engages the audience by sharing the latest AI and Tech trends and also mentors people who want to pivot in this field.

  • 14:45

    End of Summit

MLOps Summit San Francisco

MLOps Summit San Francisco

15 - 16 June 2022

Get your ticket
This website uses cookies to ensure you get the best experience. Learn more