Low(er) Code ML Pipelines with Conduit
11-03, 14:15–14:55 (America/New_York), Music Box (Room 5411)

Conduit is a Python package and terminal user interface (TUI) designed to streamline the process of building, deploying, and managing machine learning (ML) pipelines on GCP’s Vertex AI platform. Its primary goal is to improve efficiency for data science teams by reducing the time it takes to deploy ML models while observing MLOps best practices.


In the absence of Conduit, users would need to work directly with KubeFlow Pipelines (KFP) or TensorFlow Extended (TFX) APIs/DSLs to create machine learning pipelines in Vertex AI. Both of these processes are time-consuming and have steep learning curves. In addition, these methods make it difficult to reuse custom pipeline components across teams or products. The need for efficient pipeline creation also stems from challenges that data scientists commonly face, such as time-consuming setup, manual file creation, and error-prone configurations.

Conduit was developed to not only address these key efficiency issues, but to promote a healthier user focus on what really matters: exploring data and building great models. With Conduit, users can create fully functional Vertex AI pipelines that enforce MLOps best practices more simply by answering a straightforward series of questions through a Terminal User Interface (TUI) developed with the Textual framework, without having to leave their development environment. The result is rapidly produced pipelines composed of reusable and standardized components that can be run or scheduled on Vertex AI. 

The expected user base for Conduit includes data scientists and/or engineers who are deploying models/pipelines built with Python ML frameworks using Vertex AI in GCP. We're excited about the productivity and efficacy benefits of Conduit and eager to share our perspective on why TUIs are a great way to optimize workflow when locality and low(er) code solutions are of interest.


Prior Knowledge Expected

No previous knowledge expected

Piero Ferrante is an AVP & Data Science Fellow at CVS Health, a Fortune 6 health solutions company, where he and his team are focused on building scalable machine learning systems and developing tools to enhance the productivity and efficacy of hundreds of fellow data scientists and engineers.

Piero has 15+ years of applied AI/ML experience in healthcare, telecom, insurance, mobile advertising, and fintech at companies ranging in size from unicorn startups to Fortune 500s. He holds an M.S. in Predictive Analytics from Northwestern University, a B.S. in Finance and Management Information Systems from the University of Delaware, and has served as an adjunct at New York University, the University of Kansas, and Rockhurst University. Piero also advises Play-it Heath, a digital health startup, on algorithms and data strategy.

Viren Bajaj is a Senior Machine Learning Engineer at CVS Health.

He works on products to improve the productivity and efficacy of data scientists and engineers.

Currently, he is working on Conduit - a python package that streamlines the process of deploying and managing kubeflow pipelines on Vertex AI, Google's AI Platform.

Previously, he's worked at NASA Langley Research Center, Logical Systems Lab at Carnegie Mellon University, and the Nuclear & Particle Physics Department at Carnegie Mellon University.