机器学习平台整理

开源系列

cube开源一站式云原生机器学习平台:https://blog.csdn.net/luanpeng825485697/article/details/123619334

github:https://github.com/tencentmusic/cube-studio

kubeflow参考

官网:https://www.kubeflow.org/docs/started/

参考:https://www.jianshu.com/p/192f22a0b857

AirFlow/NiFi/MLFlow/KubeFlow进展:https://blog.csdn.net/chenhuipin1173/article/details/100913909

最好的任务编排工具:Airflow vs Luigi vs Argo vs MLFlow

总结

一句话总结就是:kubeflow是一个为 Kubernetes 构建的可组合,便携式,可扩展的机器学习技术栈。

支持的训练架构-https://www.kubeflow.org/docs/components/training/

英文对比:

https://aicurious.io/posts/airflow-mlflow-or-kubeflow-for-mlops/

https://devsamurai.vn/blog/ml-platform-kuberflow-mlflow-argo-airflow/

通用型选airflow

机器学习偏向大规模选kubeflow

机器学习偏向小规模选mlflow

bash 复制代码
5. How to choose between Airflow+Mlflow, and Kubeflow?

To sum up, I have some recommendations from my personal perspective:

    If your system needs to deal with multiple types of workflow, not just machine learning, Airflow may support you better. It is a mature workflow orchestration frameworks with support for a lot of operators besides machine learning.
    If you want a system with predesigned patterns for machine learning, and run at large scale on Kubenetes clusters, you may want to consider Kubeflow. Many ML specific components in Kubeflow can save your time implementing from scratch in Airflow.
    If you want to deploy MLOps in a small scale system (for example, a workstation, or a laptop), picking Airflow+MLflow stack can eliminate the need of setting up and running a Kubenetes system, and save more resources for the main tasks.

This blog post has briefly shown the differences between three popular MLOps frameworks (Airflow, MLflow and Kubeflow). Hope that it helps you in making decision between 2 stacks (Airflow + MLflow and Kubeflow). If you want to talk more about these frameworks or recommend others, please comment beflow. Thank you very much!
相关推荐
卡梅德生物科技小能手4 分钟前
卡美德生物科普:LOXL2(赖氨酰氧化酶样蛋白2)
人工智能·经验分享·生活
Dragon Wu6 分钟前
AI视频创作笔记(五)短剧制作流程
人工智能·ai
一切皆是因缘际会8 分钟前
AI产业的深度变革与未来思辨
人工智能·ai·架构
不爱土豆唯爱马铃薯9 分钟前
MONKEYCODE 教程系列MC-025 | 实战AI客服机器人
人工智能·数据挖掘
keykey6.9 分钟前
用 PyTorch 训练图像分类器:完整实战
开发语言·人工智能·深度学习·机器学习
刘婉晴11 分钟前
【火山「AI安全攻防」】恶意Skill检测引擎设计思路分享
人工智能·安全
小王毕业啦21 分钟前
2009-2024年 各国清廉指数CPI(xlsx)
大数据·人工智能·数据挖掘·数据分析·社科数据·实证分析·经管数据
syounger31 分钟前
从遗留系统到AI运营:富士通转型折射日本企业的数字化再考
人工智能
DogDaoDao43 分钟前
【GitHub】CodeGraph 深度解析:为 AI 编程代理构建预索引代码知识图谱
人工智能·程序员·github·知识图谱·ai编程·ai agent·codegraph
小O的算法实验室43 分钟前
2025年IEEE TASE,基于双层耦合平均场博弈的大规模智能体集成任务分配与轨迹规划
人工智能·算法·机器学习