没有 Google Cloud Storage 的 Kubeflow

问题描述

是否可以使用替代的本地解决方案替换 Google Cloud Storage 存储桶的使用，以便可以运行例如Kubeflow Pipelines 完全独立于 Google Cloud Platform?

Is it possible to replace the usage of Google Cloud Storage buckets with an alternative on-premises solution so that it is possible to run e.g. Kubeflow Pipelines completely independent from the Google Cloud Platform?

推荐答案

是的，这是可能的.您可以使用 minio，它类似于 s3/gs，但它在本地存储的持久卷上运行.

Yes it is possible. You can use minio, it's like s3/gs but it runs on a persistent volume of your on-premises storage.

以下是有关如何将其用作 kfserving 推理存储的说明:

Here are the instructions on how to use it as a kfserving inference storage:

验证 minio 是否在您的 kubeflow 安装中运行:

Validate that minio is running in your kubeflow installation:

$ kubectl get svc -n kubeflow |grep minio
minio-service                                  ClusterIP   10.101.143.255   <none>        9000/TCP            81d

为您的 minio 启用隧道:

Enable a tunnel for your minio:

$ kubectl port-forward svc/minio-service -n kubeflow 9000:9000
Forwarding from 127.0.0.1:9000 -> 9000
Forwarding from [::1]:9000 -> 9000

浏览 http://localhost:9000 以进入 minio UI 并创建存储桶/上传您的模型.凭证 minio/minio123.或者，您可以使用 mc 命令从终端执行此操作:

Browse http://localhost:9000 to get to the minio UI and create a bucket/upload your model. Credentials minio/minio123. Alternatively you can use the mc command to do it from your terminal:

$ mc ls minio/models/flowers/0001/
[2020-03-26 13:16:57 CET]  1.7MiB saved_model.pb
[2020-04-25 13:37:09 CEST]      0B variables/

为minio访问创建secret&ser​​viceaccount，注意s3-endpoint定义了minio的路径，keyid&acceskey是base64编码的凭证:

Create a secret&serviceaccount for the minio access, note that the s3-endpoint defines the path to the minio, keyid&acceskey are the credentials encoded in base64:

$ kubectl get secret mysecret -n homelab -o yaml
apiVersion: v1
data:
  awsAccessKeyID: bWluaW8=
  awsSecretAccessKey: bWluaW8xMjM=
kind: Secret
metadata:
  annotations:
    serving.kubeflow.org/s3-endpoint: minio-service.kubeflow:9000
    serving.kubeflow.org/s3-usehttps: "0"
  name: mysecret
  namespace: homelab

$ kubectl get serviceAccount -n homelab sa -o yaml
apiVersion: v1
kind: ServiceAccount
metadata:
  name: sa
  namespace: homelab
secrets:
- name: mysecret

最后，创建您的 inferenceservice 如下:

Finally, create your inferenceservice as follows:

$ kubectl get inferenceservice tensorflow-flowers -n homelab -o yaml
apiVersion: serving.kubeflow.org/v1alpha2
kind: InferenceService
metadata:
  name: tensorflow-flowers
  namespace: homelab
spec:
  default:
    predictor:
      serviceAccountName: sa
      tensorflow:
        storageUri: s3://models/flowers

这篇关于没有 Google Cloud Storage 的 Kubeflow的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持！