机器学习平台整理

开源系列

cube开源一站式云原生机器学习平台:https://blog.csdn.net/luanpeng825485697/article/details/123619334

github:https://github.com/tencentmusic/cube-studio

kubeflow参考

官网:https://www.kubeflow.org/docs/started/

参考:https://www.jianshu.com/p/192f22a0b857

AirFlow/NiFi/MLFlow/KubeFlow进展:https://blog.csdn.net/chenhuipin1173/article/details/100913909

最好的任务编排工具:Airflow vs Luigi vs Argo vs MLFlow

总结

一句话总结就是:kubeflow是一个为 Kubernetes 构建的可组合,便携式,可扩展的机器学习技术栈。

支持的训练架构-https://www.kubeflow.org/docs/components/training/

英文对比:

https://aicurious.io/posts/airflow-mlflow-or-kubeflow-for-mlops/

https://devsamurai.vn/blog/ml-platform-kuberflow-mlflow-argo-airflow/

通用型选airflow

机器学习偏向大规模选kubeflow

机器学习偏向小规模选mlflow

bash 复制代码
5. How to choose between Airflow+Mlflow, and Kubeflow?

To sum up, I have some recommendations from my personal perspective:

    If your system needs to deal with multiple types of workflow, not just machine learning, Airflow may support you better. It is a mature workflow orchestration frameworks with support for a lot of operators besides machine learning.
    If you want a system with predesigned patterns for machine learning, and run at large scale on Kubenetes clusters, you may want to consider Kubeflow. Many ML specific components in Kubeflow can save your time implementing from scratch in Airflow.
    If you want to deploy MLOps in a small scale system (for example, a workstation, or a laptop), picking Airflow+MLflow stack can eliminate the need of setting up and running a Kubenetes system, and save more resources for the main tasks.

This blog post has briefly shown the differences between three popular MLOps frameworks (Airflow, MLflow and Kubeflow). Hope that it helps you in making decision between 2 stacks (Airflow + MLflow and Kubeflow). If you want to talk more about these frameworks or recommend others, please comment beflow. Thank you very much!
相关推荐
九酒3 小时前
AI Agent 开发踩坑记:口播功能非得用 APP 原生实现吗?
前端·人工智能·agent
蝎子莱莱爱打怪3 小时前
DSpark 讲透:DeepSeek 不换模型,硬把 V4 提速 85%,是怎么做到的?
人工智能·面试·程序员
巫山老妖4 小时前
置身AI内
人工智能
IT_陈寒6 小时前
JavaScript项目实战经验分享
前端·人工智能·后端
vanuan7 小时前
两个AI智能体第一次对话-A2A双Agent协作实战
人工智能
kfaino9 小时前
码农的AI翻身(四)你好,我叫 Attention
人工智能·后端
雨落Re11 小时前
如何设计一个高质量Skill
人工智能
Token炼金师11 小时前
大模型权重文件全指南:从格式选择到优化实战
人工智能
阿牛哥_GX11 小时前
CDP 浏览器操控原理:让脚本接管你的浏览器
人工智能
ThreeS11 小时前
手搓MiniVLA全实战教程-一步一步用pytorch解释原理与思路
人工智能·python