Kubeflow + k3s: an Enterprise MLOps Platform in Your Home Lab

Introduction

MLOps is to Machine Learning what DevOps was to software development: the discipline that turns ML experiments into reproducible, automated and maintainable production systems. Without MLOps, your model is a Jupyter notebook that works on your laptop. With MLOps, it is a pipeline that trains, validates, deploys and monitors models continuously.

Kubeflow is the reference open-source MLOps platform. It was born at Google in 2017, graduated as a CNCF project and today it is used by organizations such as Bloomberg, Spotify, Cisco, the US Department of Defense and hundreds of Fortune 500 companies. Its purpose: to make ML deployments on Kubernetes simple, portable and scalable.

The problem: Kubeflow is designed for production Kubernetes clusters. Running it locally seems unfeasible.

The solution: k3s — a certified Kubernetes distribution that consumes 512MB of RAM and installs in 30 seconds. Same API, same manifests, same experience. What you deploy here works the same on EKS, GKE or AKS.

In this lab we are going to set up a complete enterprise-grade MLOps platform on your local machine:

Kubeflow Pipelines — Orchestration of ML workflows as DAGs
SeaweedFS — S3-compatible artifact storage
ML Metadata (MLMD) — Experiment tracking and lineage
KServe — Production model serving
A real pipeline — Train, evaluate and serve a working model

All with commands you can copy and paste directly.

Lab architecture

CODE

┌─────────────────────────────────────────────────────────────┐
│                        Your Machine (k3s)                   │
│                                                             │
│  ┌──────────────┐    ┌──────────────────┐                  │
│  │   Kubeflow   │    │  Kubeflow        │                  │
│  │   Dashboard  │◄───│  Pipelines API   │                  │
│  │   (UI)       │    │  (Orchestrator)  │                  │
│  └──────────────┘    └────────┬─────────┘                  │
│                               │                             │
│         ┌─────────────────────┼─────────────────┐          │
│         │                     │                 │          │
│         ▼                     ▼                 ▼          │
│  ┌─────────────┐    ┌──────────────┐   ┌─────────────┐    │
│  │  SeaweedFS   │    │     ML       │   │   KServe    │    │
│  │  (Artifacts) │    │   Metadata   │   │  (Serving)  │    │
│  │  S3-compat  │    │   (MySQL)    │   │             │    │
│  └─────────────┘    └──────────────┘   └─────────────┘    │
│                                                             │
│  ┌──────────────────────────────────────────────────────┐  │
│  │               k3s (lightweight Kubernetes)            │  │
│  │          Same API as EKS / GKE / AKS                 │  │
│  └──────────────────────────────────────────────────────┘  │
└─────────────────────────────────────────────────────────────┘

Component	Function	Enterprise equivalent
k3s	Lightweight Kubernetes	EKS, GKE, AKS
Kubeflow Pipelines	ML orchestration	Vertex AI Pipelines, SageMaker Pipelines
SeaweedFS	Artifact storage (S3-compatible)	AWS S3, Azure Blob, GCS
ML Metadata	Lineage and tracking	MLflow Tracking, Vertex Experiments
KServe	Model serving	SageMaker Endpoints, Vertex Prediction

Prerequisites

Resource	Minimum	Recommended
RAM	8 GB	16 GB
CPU	4 cores	8 cores
Disk	30 GB free	50 GB free
OS	Linux (any distro)	Ubuntu 22.04+ / Debian 12+

Important: Disk space is critical. If your partition has less than 30 GB free, Kubernetes will trigger the disk-pressure taint and no pod will be able to start. Check with df -h / before you begin.

Required software:

BASH

# Check that you have these tools
docker --version    # Docker 20.10+
curl --version      # For downloads
kubectl version --client  # If you don't have it, k3s includes it

If you don't have kubectl installed, don't worry — k3s includes its own binary and we'll configure it later.

Step 1: Install k3s

k3s is certified Kubernetes in a single 60MB binary. One command and you have a working cluster:

BASH

curl -sfL https://get.k3s.io | sh -s - --write-kubeconfig-mode 644

This installs k3s as a systemd service. Let's verify the cluster is operational:

BASH

# Check nodes
sudo k3s kubectl get nodes

# You should see something like:
# NAME         STATUS   ROLES                  AGE   VERSION
# your-machine Ready    control-plane,master   30s   v1.36.x+k3s1

Configure kubectl to use it without sudo:

BASH

# Copy the kubeconfig for your user
mkdir -p ~/.kube
sudo cp /etc/rancher/k3s/k3s.yaml ~/.kube/config
sudo chown $(id -u):$(id -g) ~/.kube/config
export KUBECONFIG=~/.kube/config

# Add it to .bashrc so it persists
echo 'export KUBECONFIG=~/.kube/config' >> ~/.bashrc

Verification:

BASH

kubectl cluster-info
# Kubernetes control plane is running at https://127.0.0.1:6443

What we just did: Installed a complete Kubernetes cluster with API server, scheduler, controller-manager, etcd (SQLite in k3s) and kubelet. All in a single process that consumes ~500MB of RAM.

Step 2: Deploy Kubeflow Pipelines

Kubeflow Pipelines (KFP) is the central component of the platform. It manages ML workflows as directed acyclic graphs (DAGs) where each node is an independent container.

2.1 Deploy Kubeflow Pipelines with official manifests

BASH

# Define the version (use the latest stable)
export PIPELINE_VERSION="2.16.1"

# Apply the Kubeflow Pipelines manifests (standalone)
kubectl apply -k "github.com/kubeflow/pipelines/manifests/kustomize/cluster-scoped-resources?ref=$PIPELINE_VERSION"

# Wait for the CRDs to register
kubectl wait --for condition=established --timeout=60s crd/applications.app.k8s.io

# Deploy the components into the kubeflow namespace
kubectl apply -k "github.com/kubeflow/pipelines/manifests/kustomize/env/platform-agnostic?ref=$PIPELINE_VERSION"

2.2 Verify the deployment

The first time takes 8-10 minutes because it needs to download the container images. Be patient:

BASH

# Wait for all pods to be Running
kubectl -n kubeflow wait --for=condition=Ready pods --all --timeout=600s

# Check the pod status
kubectl -n kubeflow get pods

You should see something similar to:

CODE

NAME                                               READY   STATUS    RESTARTS   AGE
cache-deployer-deployment-xxxxx                    1/1     Running   0          5m
cache-server-xxxxx                                 1/1     Running   0          5m
metadata-envoy-deployment-xxxxx                    1/1     Running   0          5m
metadata-grpc-deployment-xxxxx                     1/1     Running   0          5m
metadata-writer-xxxxx                              1/1     Running   0          5m
ml-pipeline-xxxxx                                  1/1     Running   0          5m
ml-pipeline-persistenceagent-xxxxx                 1/1     Running   0          5m
ml-pipeline-scheduledworkflow-xxxxx                1/1     Running   0          5m
ml-pipeline-ui-xxxxx                               1/1     Running   0          5m
ml-pipeline-viewer-crd-xxxxx                       1/1     Running   0          5m
ml-pipeline-visualizationserver-xxxxx              1/1     Running   0          5m
mysql-xxxxx                                        1/1     Running   0          5m
seaweedfs-xxxxx                                    1/1     Running   0          5m
workflow-controller-xxxxx                          1/1     Running   0          5m

Note: It is normal for metadata-grpc to have a few initial restarts while it waits for MySQL to start. It stabilizes on its own.

2.3 Access the Dashboard

BASH

# Port-forward the Kubeflow Pipelines UI
kubectl -n kubeflow port-forward svc/ml-pipeline-ui 8080:80 &

# Open in the browser
echo "Dashboard available at: http://localhost:8080"

Open http://localhost:8080 in your browser. You'll see the Kubeflow Pipelines dashboard with sections for Pipelines, Experiments, Runs and Artifacts.

What we have now: An enterprise ML orchestrator with a web UI, S3-compatible artifact storage (SeaweedFS), a metadata database (MySQL) and a pipeline execution engine (Argo Workflows). All running on your machine.

Step 3: Install the Kubeflow Pipelines SDK

The Python SDK lets us define pipelines as code and upload them to the cluster:

BASH

# Create a virtual environment (mandatory on modern distros with PEP 668)
python3 -m venv ~/mlops-venv
source ~/mlops-venv/bin/activate

# Install the SDK (same version as the server)
pip install kfp==2.16.1

# Verify the installation
python3 -c "import kfp; print(f'KFP SDK version: {kfp.__version__}')"

Important: Always activate the venv before working with the SDK: source ~/mlops-venv/bin/activate

Step 4: Create your first ML Pipeline

We are going to create a real pipeline that:

Downloads data — A classification dataset
Preprocesses — Cleaning and a train/test split
Trains — A classification model with scikit-learn
Evaluates — Accuracy, precision and recall metrics
Exports — Saves the serialized model to SeaweedFS

4.1 Define the components

Create a file mlops_pipeline.py:

PYTHON

"""
MLOps Pipeline: Wine classification with Kubeflow Pipelines
Each function decorated with @component runs as an independent container
"""
from kfp import dsl
from kfp.dsl import Input, Output, Dataset, Model, Metrics

# ─────────────────────────────────────────────
# Component 1: Data download
# ─────────────────────────────────────────────
@dsl.component(
    base_image="python:3.11-slim",
    packages_to_install=["scikit-learn", "pandas"]
)
def download_data(dataset_out: Output[Dataset]):
    """Download the sklearn Wine dataset and save it as CSV."""
    from sklearn.datasets import load_wine
    import pandas as pd

    wine = load_wine(as_frame=True)
    df = wine.frame
    df.to_csv(dataset_out.path, index=False)
    print(f"Dataset downloaded: {df.shape[0]} samples, {df.shape[1]} columns")


# ─────────────────────────────────────────────
# Component 2: Preprocessing
# ─────────────────────────────────────────────
@dsl.component(
    base_image="python:3.11-slim",
    packages_to_install=["scikit-learn", "pandas"]
)
def preprocess_data(
    dataset_in: Input[Dataset],
    train_data: Output[Dataset],
    test_data: Output[Dataset],
    test_size: float = 0.2
):
    """Split the data into train/test with stratification."""
    import pandas as pd
    from sklearn.model_selection import train_test_split

    df = pd.read_csv(dataset_in.path)

    # Separate features and target
    X = df.drop("target", axis=1)
    y = df["target"]

    # Stratified split
    X_train, X_test, y_train, y_test = train_test_split(
        X, y, test_size=test_size, random_state=42, stratify=y
    )

    # Save the splits
    train = pd.concat([X_train, y_train], axis=1)
    test = pd.concat([X_test, y_test], axis=1)

    train.to_csv(train_data.path, index=False)
    test.to_csv(test_data.path, index=False)

    print(f"Train: {train.shape[0]} samples | Test: {test.shape[0]} samples")


# ─────────────────────────────────────────────
# Component 3: Training
# ─────────────────────────────────────────────
@dsl.component(
    base_image="python:3.11-slim",
    packages_to_install=["scikit-learn", "pandas", "joblib"]
)
def train_model(
    train_data: Input[Dataset],
    model_out: Output[Model],
    n_estimators: int = 100,
    max_depth: int = 10
):
    """Train a RandomForest with the given hyperparameters."""
    import pandas as pd
    from sklearn.ensemble import RandomForestClassifier
    import joblib

    # Load data
    df = pd.read_csv(train_data.path)
    X = df.drop("target", axis=1)
    y = df["target"]

    # Train the model
    model = RandomForestClassifier(
        n_estimators=n_estimators,
        max_depth=max_depth,
        random_state=42,
        n_jobs=-1
    )
    model.fit(X, y)

    # Save the serialized model
    joblib.dump(model, model_out.path)

    print(f"Model trained: {n_estimators} trees, depth {max_depth}")
    print(f"Train accuracy: {model.score(X, y):.4f}")


# ─────────────────────────────────────────────
# Component 4: Evaluation
# ─────────────────────────────────────────────
@dsl.component(
    base_image="python:3.11-slim",
    packages_to_install=["scikit-learn", "pandas", "joblib"]
)
def evaluate_model(
    model_in: Input[Model],
    test_data: Input[Dataset],
    metrics_out: Output[Metrics]
):
    """Evaluate the model with classification metrics."""
    import pandas as pd
    from sklearn.metrics import accuracy_score, precision_score, recall_score, f1_score
    import joblib

    # Load the model and the test data
    model = joblib.load(model_in.path)
    df = pd.read_csv(test_data.path)
    X = df.drop("target", axis=1)
    y = df["target"]

    # Predictions
    y_pred = model.predict(X)

    # Compute metrics
    accuracy = accuracy_score(y, y_pred)
    precision = precision_score(y, y_pred, average="weighted")
    recall = recall_score(y, y_pred, average="weighted")
    f1 = f1_score(y, y_pred, average="weighted")

    # Log metrics in Kubeflow
    metrics_out.log_metric("accuracy", accuracy)
    metrics_out.log_metric("precision", precision)
    metrics_out.log_metric("recall", recall)
    metrics_out.log_metric("f1_score", f1)

    print(f"--- Evaluation Results ---")
    print(f"Accuracy:  {accuracy:.4f}")
    print(f"Precision: {precision:.4f}")
    print(f"Recall:    {recall:.4f}")
    print(f"F1-Score:  {f1:.4f}")


# ─────────────────────────────────────────────
# Pipeline: Full orchestration
# ─────────────────────────────────────────────
@dsl.pipeline(
    name="wine-classification-pipeline",
    description="MLOps pipeline: download, preprocess, train and evaluate a wine classification model"
)
def ml_pipeline(
    test_size: float = 0.2,
    n_estimators: int = 100,
    max_depth: int = 10
):
    """Parameterizable pipeline — the same parameters you would tune in production."""

    # Step 1: Download data
    download_task = download_data()

    # Step 2: Preprocess
    preprocess_task = preprocess_data(
        dataset_in=download_task.outputs["dataset_out"],
        test_size=test_size
    )

    # Step 3: Train the model
    train_task = train_model(
        train_data=preprocess_task.outputs["train_data"],
        n_estimators=n_estimators,
        max_depth=max_depth
    )

    # Step 4: Evaluate
    evaluate_model(
        model_in=train_task.outputs["model_out"],
        test_data=preprocess_task.outputs["test_data"]
    )

4.2 Compile and upload the Pipeline

BASH

# Compile the pipeline to YAML format (portable artifact)
python3 -c "
from kfp import compiler
from mlops_pipeline import ml_pipeline

compiler.Compiler().compile(
    pipeline_func=ml_pipeline,
    package_path='wine_pipeline.yaml'
)
print('Pipeline compiled: wine_pipeline.yaml')
"

The file wine_pipeline.yaml is a portable artifact — you can upload it to any Kubeflow installation (local, cloud, enterprise) and it will run identically.

4.3 Run the Pipeline

BASH

# Run via the SDK
python3 -c "
import kfp

# Connect to the local cluster
client = kfp.Client(host='http://localhost:8080')

# Create an experiment (groups related runs)
experiment = client.create_experiment(name='wine-classification')

# Launch the pipeline with parameters
run = client.create_run_from_pipeline_package(
    pipeline_file='wine_pipeline.yaml',
    experiment_name='wine-classification',
    run_name='wine-run-v1',
    arguments={
        'test_size': 0.2,
        'n_estimators': 150,
        'max_depth': 12
    }
)

print(f'Run launched: {run.run_id}')
print(f'Dashboard: http://localhost:8080/#/runs/details/{run.run_id}')
"

Open the dashboard link and you'll see the pipeline running in real time. Each component appears as a node in the graph, with individual logs, input/output artifacts and metrics.

Note: The first run takes ~7-8 minutes because each component pulls its base image (python:3.11-slim) and installs dependencies. Subsequent runs use the cache and take ~2 minutes.

Step 5: Artifact storage with SeaweedFS

Kubeflow Pipelines deploys SeaweedFS as artifact storage. SeaweedFS exposes an API that is 100% compatible with S3 — any tool or SDK that works with AWS S3 works without changes.

BASH

# Port-forward the SeaweedFS S3 endpoint
kubectl -n kubeflow port-forward svc/seaweedfs 9000:9000 &

# Default credentials
echo "S3 Endpoint: http://localhost:9000"
echo "Access Key: minio"
echo "Secret Key: minio123"

Verify artifacts

BASH

# Install the MinIO client (compatible with any S3)
curl -sL https://dl.min.io/client/mc/release/linux-amd64/mc -o /usr/local/bin/mc
chmod +x /usr/local/bin/mc

# Configure the alias
mc alias set local http://localhost:9000 minio minio123

# List buckets
mc ls local/

# View the pipeline artifacts
mc ls local/mlpipeline/ --recursive

You'll see the artifacts organized by pipeline/run/component:

CODE

v2/artifacts/wine-classification-pipeline/<run-id>/download-data/.../dataset_out
v2/artifacts/wine-classification-pipeline/<run-id>/preprocess-data/.../train_data
v2/artifacts/wine-classification-pipeline/<run-id>/preprocess-data/.../test_data
v2/artifacts/wine-classification-pipeline/<run-id>/train-model/.../model_out

In production, you simply point to S3/GCS/Azure Blob and your pipeline code doesn't change.

Step 6: Serve models with KServe

KServe is the standard for model serving on Kubernetes. It provides:

Multi-framework (sklearn, TensorFlow, PyTorch, XGBoost, ONNX)
Canary deployments (gradual traffic to new versions)
Request batching and GPU scheduling
RawDeployment mode — works without Knative or Istio (ideal for k3s)

6.1 Install KServe

BASH

# Install Cert-Manager (KServe prerequisite)
kubectl apply -f https://github.com/cert-manager/cert-manager/releases/download/v1.16.0/cert-manager.yaml

# Wait for cert-manager to be ready
kubectl -n cert-manager wait --for=condition=Ready pods --all --timeout=180s

# Install KServe (server-side apply needed due to CRD size)
kubectl apply --server-side --force-conflicts \
  -f https://github.com/kserve/kserve/releases/download/v0.14.0/kserve.yaml

# Install the default runtimes (sklearn, tensorflow, etc.)
kubectl apply --server-side --force-conflicts \
  -f https://github.com/kserve/kserve/releases/download/v0.14.0/kserve-cluster-resources.yaml

# Wait for KServe to be operational
kubectl -n kserve wait --for=condition=Ready pods --all --timeout=180s

6.2 Configure KServe for k3s (RawDeployment)

On k3s we don't have Knative, so we configure KServe in RawDeployment mode (it uses standard Kubernetes Deployments and Services):

BASH

# Change the deployment mode to RawDeployment
kubectl -n kserve patch configmap inferenceservice-config \
  --type merge \
  -p '{"data":{"deploy":"{\"defaultDeploymentMode\": \"RawDeployment\"}"}}'

6.3 Allow cross-namespace access to the storage

KServe needs to access SeaweedFS (in the kubeflow namespace) from the default namespace. We remove the restrictive NetworkPolicy:

BASH

# Remove the NetworkPolicy that blocks cross-namespace access
kubectl -n kubeflow delete networkpolicy seaweedfs

6.4 Create S3 credentials for KServe

BASH

cat <<'EOF' | kubectl apply -f -
apiVersion: v1
kind: Secret
metadata:
  name: seaweedfs-secret
  namespace: default
  annotations:
    serving.kserve.io/s3-endpoint: seaweedfs.kubeflow.svc.cluster.local:9000
    serving.kserve.io/s3-usehttps: "0"
    serving.kserve.io/s3-region: us-east-1
type: Opaque
stringData:
  AWS_ACCESS_KEY_ID: minio
  AWS_SECRET_ACCESS_KEY: minio123
---
apiVersion: v1
kind: ServiceAccount
metadata:
  name: kserve-sa
  namespace: default
secrets:
  - name: seaweedfs-secret
EOF

6.5 Prepare and deploy the model

We copy the trained model to a clean path in the storage:

BASH

# Copy the pipeline model to a KServe-compatible path
# (KServe sklearn expects a model.joblib file in the directory)
mc cp local/mlpipeline/v2/artifacts/wine-classification-pipeline/<RUN_ID>/train-model/<ARTIFACT_ID>/model_out /tmp/model.joblib
mc mb local/mlpipeline/models/wine-classifier/
mc cp /tmp/model.joblib local/mlpipeline/models/wine-classifier/model.joblib

Tip: Replace and with the real values. You can see them with mc ls local/mlpipeline/v2/artifacts/ --recursive | grep model_out

Now we deploy the InferenceService:

BASH

cat <<'EOF' | kubectl apply -f -
apiVersion: serving.kserve.io/v1beta1
kind: InferenceService
metadata:
  name: wine-classifier
  namespace: default
spec:
  predictor:
    serviceAccountName: kserve-sa
    model:
      modelFormat:
        name: sklearn
      storageUri: "s3://mlpipeline/models/wine-classifier"
      resources:
        requests:
          cpu: "100m"
          memory: "256Mi"
        limits:
          cpu: "500m"
          memory: "512Mi"
EOF

We wait for it to be Ready:

BASH

# Check that the model is serving
kubectl get inferenceservice wine-classifier

# Wait for it to be Ready (may take 1-2 min due to image pull)
kubectl wait --for=condition=Ready inferenceservice/wine-classifier --timeout=180s

6.6 Make predictions against the model

BASH

# Port-forward to the model service
kubectl port-forward svc/wine-classifier-predictor 8081:80 &

# Make a prediction (one wine sample)
curl -s -X POST http://localhost:8081/v1/models/wine-classifier:predict \
  -H "Content-Type: application/json" \
  -d '{
    "instances": [
      [14.23, 1.71, 2.43, 15.6, 127.0, 2.80, 3.06, 0.28, 2.29, 5.64, 1.04, 3.92, 1065.0]
    ]
  }' | python3 -m json.tool

Expected response:

JSON

{
    "predictions": [0]
}

Try it with multiple samples to check the 3 wine classes:

BASH

curl -s -X POST http://localhost:8081/v1/models/wine-classifier:predict \
  -H "Content-Type: application/json" \
  -d '{
    "instances": [
      [14.23, 1.71, 2.43, 15.6, 127.0, 2.80, 3.06, 0.28, 2.29, 5.64, 1.04, 3.92, 1065.0],
      [12.37, 0.94, 1.36, 10.6, 88.0, 1.98, 0.57, 0.28, 0.42, 1.95, 1.05, 1.82, 520.0],
      [13.73, 1.50, 2.70, 22.5, 101.0, 3.0, 2.60, 0.26, 1.86, 5.10, 1.04, 3.57, 1190.0]
    ]
  }' | python3 -m json.tool

The model is serving predictions in real time with sub-second latency.

Step 7: Monitor the platform

7.1 Metrics with Prometheus + Grafana

k3s doesn't include monitoring by default, but we can add it with the kube-prometheus stack:

BASH

# Add the Helm repository (you need Helm installed)
helm repo add prometheus-community https://prometheus-community.github.io/helm-charts
helm repo update

# Install kube-prometheus-stack (Prometheus + Grafana + alerts)
helm install monitoring prometheus-community/kube-prometheus-stack \
  --namespace monitoring \
  --create-namespace \
  --set grafana.adminPassword=admin \
  --set prometheus.prometheusSpec.serviceMonitorSelectorNilUsesHelmValues=false

BASH

# Access Grafana
kubectl -n monitoring port-forward svc/monitoring-grafana 3000:80 &

echo "Grafana: http://localhost:3000"
echo "User: admin | Password: admin"

7.2 Recommended dashboard

In Grafana, import dashboard 13332 (Kubernetes cluster monitoring) to view cluster resources. You'll be able to monitor the CPU/RAM consumption of each MLOps platform component and detect bottlenecks during training.

Step 8: Automate with CI/CD (enterprise bonus)

In an enterprise environment, ML pipelines are triggered automatically. Here is a GitHub Actions example that ties everything together:

YAML

# .github/workflows/ml-pipeline.yml
name: ML Pipeline - Train & Deploy

on:
  push:
    paths:
      - 'ml/**'  # Only when ML code changes

jobs:
  run-pipeline:
    runs-on: self-hosted  # Runner with cluster access
    steps:
      - uses: actions/checkout@v4

      - name: Install KFP SDK
        run: pip install kfp==2.16.1

      - name: Compile pipeline
        run: python3 ml/compile_pipeline.py

      - name: Submit pipeline run
        env:
          KFP_HOST: ${{ secrets.KUBEFLOW_HOST }}
        run: |
          python3 -c "
          import kfp
          client = kfp.Client(host='${KFP_HOST}')
          client.create_run_from_pipeline_package(
              pipeline_file='wine_pipeline.yaml',
              experiment_name='production',
              run_name='auto-$(date +%Y%m%d-%H%M%S)'
          )
          "

      - name: Wait for completion
        run: |
          # Wait for the run to finish and validate metrics
          python3 ml/wait_and_validate.py

Comparison: your lab vs. enterprise Cloud

Aspect	Your Lab (k3s)	Production (EKS/GKE)
Kubernetes	k3s (single-node)	EKS/GKE (multi-node, HA)
Storage	SeaweedFS (local, S3-compat)	S3/GCS/Azure Blob
Pipeline engine	Same Kubeflow Pipelines	Same Kubeflow Pipelines
Model serving	KServe RawDeployment	KServe + Knative + GPU nodes
Auth	None (lab)	Dex + OIDC + RBAC
Monitoring	Prometheus + Grafana	Datadog/CloudWatch + PagerDuty
Networking	localhost	Istio service mesh + mTLS
Scaling	Manual	HPA + Cluster Autoscaler

The key difference: your pipeline code doesn't change. The components, the DAG, the logic — everything is identical. Only the underlying infrastructure changes.

Cleanup

If you want to remove the whole lab:

BASH

# Delete the served model
kubectl delete inferenceservice wine-classifier

# Delete KServe
kubectl delete namespace kserve

# Delete Kubeflow Pipelines
kubectl delete namespace kubeflow

# Delete monitoring
helm uninstall monitoring -n monitoring
kubectl delete namespace monitoring

# Delete cert-manager
kubectl delete namespace cert-manager

# Uninstall k3s completely
/usr/local/bin/k3s-uninstall.sh

Next steps

Once you master this lab, the next steps toward enterprise production are:

Multi-tenancy — Add Dex + OIDC for authentication and per-team namespaces
GPU scheduling — Add the NVIDIA device plugin for accelerated training
Feature Store — Integrate Feast to manage features shared across teams
A/B testing — Configure canary deployments in KServe for gradual rollouts
Data versioning — Add DVC to version datasets alongside the code
Model registry — Implement approval workflows before promoting models to production

Conclusion

You have deployed the same MLOps platform that Fortune 500 organizations use in production — on your own machine, with 100% open-source tools. It is not a mock-up: the manifests you applied, the SDK you used and the concepts of pipelines as DAGs are exactly the same ones you would find in a 200-node cluster on GKE.

The difference between your lab and an enterprise environment is not the technology — it is the scale, the authentication and the redundancy. But the knowledge you have gained is directly transferable.

Kubeflow Pipelines gives you something no Jupyter notebook can offer: reproducibility, auditability and automation. Every run is recorded, every artifact versioned, every metric stored. That is what separates an experiment from a production system.

Introduction

Lab architecture

Prerequisites

Step 1: Install k3s

Step 2: Deploy Kubeflow Pipelines

2.1 Deploy Kubeflow Pipelines with official manifests

2.2 Verify the deployment

2.3 Access the Dashboard

Step 3: Install the Kubeflow Pipelines SDK

Step 4: Create your first ML Pipeline

4.1 Define the components

4.2 Compile and upload the Pipeline

4.3 Run the Pipeline

Step 5: Artifact storage with SeaweedFS

Verify artifacts

Step 6: Serve models with KServe

6.1 Install KServe

6.2 Configure KServe for k3s (RawDeployment)

6.3 Allow cross-namespace access to the storage

6.4 Create S3 credentials for KServe

6.5 Prepare and deploy the model

6.6 Make predictions against the model

Step 7: Monitor the platform

7.1 Metrics with Prometheus + Grafana

7.2 Recommended dashboard

Step 8: Automate with CI/CD (enterprise bonus)

Comparison: your lab vs. enterprise Cloud

Cleanup

Next steps

Conclusion

Comments