机器学习平台整理

开源系列

cube开源一站式云原生机器学习平台:https://blog.csdn.net/luanpeng825485697/article/details/123619334

github:https://github.com/tencentmusic/cube-studio

kubeflow参考

官网:https://www.kubeflow.org/docs/started/

参考:https://www.jianshu.com/p/192f22a0b857

AirFlow/NiFi/MLFlow/KubeFlow进展:https://blog.csdn.net/chenhuipin1173/article/details/100913909

最好的任务编排工具:Airflow vs Luigi vs Argo vs MLFlow

总结

一句话总结就是:kubeflow是一个为 Kubernetes 构建的可组合,便携式,可扩展的机器学习技术栈。

支持的训练架构-https://www.kubeflow.org/docs/components/training/

英文对比:

https://aicurious.io/posts/airflow-mlflow-or-kubeflow-for-mlops/

https://devsamurai.vn/blog/ml-platform-kuberflow-mlflow-argo-airflow/

通用型选airflow

机器学习偏向大规模选kubeflow

机器学习偏向小规模选mlflow

bash 复制代码
5. How to choose between Airflow+Mlflow, and Kubeflow?

To sum up, I have some recommendations from my personal perspective:

    If your system needs to deal with multiple types of workflow, not just machine learning, Airflow may support you better. It is a mature workflow orchestration frameworks with support for a lot of operators besides machine learning.
    If you want a system with predesigned patterns for machine learning, and run at large scale on Kubenetes clusters, you may want to consider Kubeflow. Many ML specific components in Kubeflow can save your time implementing from scratch in Airflow.
    If you want to deploy MLOps in a small scale system (for example, a workstation, or a laptop), picking Airflow+MLflow stack can eliminate the need of setting up and running a Kubenetes system, and save more resources for the main tasks.

This blog post has briefly shown the differences between three popular MLOps frameworks (Airflow, MLflow and Kubeflow). Hope that it helps you in making decision between 2 stacks (Airflow + MLflow and Kubeflow). If you want to talk more about these frameworks or recommend others, please comment beflow. Thank you very much!
相关推荐
虾球xz11 分钟前
游戏引擎学习第281天:在房间之间为摄像机添加动画效果
c++·人工智能·学习·游戏引擎
冷yan~17 分钟前
GitHub文档加载器设计与实现
java·人工智能·spring·ai·github·ai编程
willhu200822 分钟前
Tensorflow2保存和加载模型
深度学习·机器学习·tensorflow
Humbunklung1 小时前
从数据层面减少过拟合现象
机器学习
AI大模型系统化学习1 小时前
Excel MCP: 自动读取、提炼、分析Excel数据并生成可视化图表和分析报告
人工智能·ai·大模型·ai大模型·大模型学习·大模型入门·mcp
lboyj1 小时前
填孔即可靠:猎板PCB如何用树脂塞孔重构高速电路设计规则
人工智能·重构
Blossom.1182 小时前
从虚拟现实到混合现实:沉浸式体验的未来之路
人工智能·目标检测·机器学习·计算机视觉·语音识别·vr·mr
赵青临的辉2 小时前
简单神经网络(ANN)实现:从零开始构建第一个模型
人工智能·深度学习·神经网络
KALC2 小时前
告别“知识孤岛”:RAG赋能网络安全运营
人工智能·网络安全
2303_Alpha2 小时前
深度学习入门:深度学习(完结)
人工智能·笔记·python·深度学习·神经网络·机器学习