Once the custom container images are built, and storage is configured, our Kubeflow environment will look like the below screenshot: This gives the teams the flexibility they need to run their respective tasks. The following image depicts how the Notebook Servers leverage the storage engine.Įach Jupyter Notebook Server uses its own container image with the appropriate tools and frameworks. The trained model is stored in another shared volume used by DevOps engineers to package and deploy the model for inference. The data preparation and inference environments will target the CPU, while the Jupyter Notebook used for training will run on a GPU host.ĭata scientists will perform the data processing task and save the final dataset to a shared volume used by machine learning engineers training the model. Each step is associated with a dedicated Jupyter Notebook Server environment. There are three independent steps involved in this exercise: data preparation, model training, and inference. In the first part of this series, we will build custom container images for the Kubeflow Notebook Server that we will use in the remainder of this tutorial.
For model serving, we will leverage KFServing, one of the core building blocks of Kubeflow.
This scenario will be further extended to run MLOps based on Kubeflow Pipelines.
#DOCKER HUB INTEL PYTHON JUPYTER NOTEBOOK HOW TO#
The objective is not to train the most sophisticated model but explore how to build ML pipelines with Kubeflow Notebook Server. We will train a Convolutional Neural Network (CNN) to classify the images of dogs and cats. To follow this guide, you need to have Kubeflow installed in your environment with a storage engine like Portworx supporting shared volumes for creating PVCs with RWX support.
We will delve deeper into these tasks over the next few installments. In this installment, we will start exploring building an end-to-end machine learning pipeline for data preparation, training, and inference.
0 kommentar(er)
