机器学习平台整理

开源系列

cube开源一站式云原生机器学习平台:https://blog.csdn.net/luanpeng825485697/article/details/123619334

github:https://github.com/tencentmusic/cube-studio

kubeflow参考

官网:https://www.kubeflow.org/docs/started/

参考:https://www.jianshu.com/p/192f22a0b857

AirFlow/NiFi/MLFlow/KubeFlow进展:https://blog.csdn.net/chenhuipin1173/article/details/100913909

最好的任务编排工具:Airflow vs Luigi vs Argo vs MLFlow

总结

一句话总结就是:kubeflow是一个为 Kubernetes 构建的可组合,便携式,可扩展的机器学习技术栈。

支持的训练架构-https://www.kubeflow.org/docs/components/training/

英文对比:

https://aicurious.io/posts/airflow-mlflow-or-kubeflow-for-mlops/

https://devsamurai.vn/blog/ml-platform-kuberflow-mlflow-argo-airflow/

通用型选airflow

机器学习偏向大规模选kubeflow

机器学习偏向小规模选mlflow

bash 复制代码
5. How to choose between Airflow+Mlflow, and Kubeflow?

To sum up, I have some recommendations from my personal perspective:

    If your system needs to deal with multiple types of workflow, not just machine learning, Airflow may support you better. It is a mature workflow orchestration frameworks with support for a lot of operators besides machine learning.
    If you want a system with predesigned patterns for machine learning, and run at large scale on Kubenetes clusters, you may want to consider Kubeflow. Many ML specific components in Kubeflow can save your time implementing from scratch in Airflow.
    If you want to deploy MLOps in a small scale system (for example, a workstation, or a laptop), picking Airflow+MLflow stack can eliminate the need of setting up and running a Kubenetes system, and save more resources for the main tasks.

This blog post has briefly shown the differences between three popular MLOps frameworks (Airflow, MLflow and Kubeflow). Hope that it helps you in making decision between 2 stacks (Airflow + MLflow and Kubeflow). If you want to talk more about these frameworks or recommend others, please comment beflow. Thank you very much!
相关推荐
三月七(爱看动漫的程序员)1 分钟前
LM2 : A Simple Society of Language Models Solves Complex Reasoning
人工智能·gpt·语言模型·自然语言处理·chatgpt·langchain·1024程序员节
学不会lostfound28 分钟前
三、计算机视觉_01图像的基本操作
人工智能·opencv·计算机视觉·pil·卷积池化
小黄人软件1 小时前
【AI协作】让所有用电脑的场景都能在ChatGPT里完成。Canvas :新一代可视化交互,让AI易用易得
人工智能·chatgpt·canvas
知来者逆1 小时前
基于集成Whisper 与 Pepper-GPT改进人机交互体验并实现顺畅通信
人工智能·gpt·语言模型·自然语言处理·whisper·人机交互
摆烂仙君1 小时前
《Probing the 3D Awareness of Visual Foundation Models》论文解析——单图像表面重建
人工智能·深度学习·计算机视觉
摆烂仙君1 小时前
《Probing the 3D Awareness of Visual Foundation Models》论文解析——多视图一致性
人工智能·深度学习
Elastic 中国社区官方博客2 小时前
Elasticsearch retrievers 通常与 Elasticsearch 8.16.0 一起正式发布!
大数据·数据库·人工智能·elasticsearch·搜索引擎·ai·全文检索
LKID体2 小时前
ChatGPT提问prompt范例模板
人工智能·chatgpt·prompt
MinIO官方账号2 小时前
使用 Prompt API 与您的对象聊天
人工智能
爱喝矿泉水的猛男3 小时前
Prompt设计技巧和高级PE
java·人工智能·prompt·cot·tot·pe·ape