机器学习平台整理

开源系列

cube开源一站式云原生机器学习平台:https://blog.csdn.net/luanpeng825485697/article/details/123619334

github:https://github.com/tencentmusic/cube-studio

kubeflow参考

官网:https://www.kubeflow.org/docs/started/

参考:https://www.jianshu.com/p/192f22a0b857

AirFlow/NiFi/MLFlow/KubeFlow进展:https://blog.csdn.net/chenhuipin1173/article/details/100913909

最好的任务编排工具:Airflow vs Luigi vs Argo vs MLFlow

总结

一句话总结就是:kubeflow是一个为 Kubernetes 构建的可组合,便携式,可扩展的机器学习技术栈。

支持的训练架构-https://www.kubeflow.org/docs/components/training/

英文对比:

https://aicurious.io/posts/airflow-mlflow-or-kubeflow-for-mlops/

https://devsamurai.vn/blog/ml-platform-kuberflow-mlflow-argo-airflow/

通用型选airflow

机器学习偏向大规模选kubeflow

机器学习偏向小规模选mlflow

bash 复制代码
5. How to choose between Airflow+Mlflow, and Kubeflow?

To sum up, I have some recommendations from my personal perspective:

    If your system needs to deal with multiple types of workflow, not just machine learning, Airflow may support you better. It is a mature workflow orchestration frameworks with support for a lot of operators besides machine learning.
    If you want a system with predesigned patterns for machine learning, and run at large scale on Kubenetes clusters, you may want to consider Kubeflow. Many ML specific components in Kubeflow can save your time implementing from scratch in Airflow.
    If you want to deploy MLOps in a small scale system (for example, a workstation, or a laptop), picking Airflow+MLflow stack can eliminate the need of setting up and running a Kubenetes system, and save more resources for the main tasks.

This blog post has briefly shown the differences between three popular MLOps frameworks (Airflow, MLflow and Kubeflow). Hope that it helps you in making decision between 2 stacks (Airflow + MLflow and Kubeflow). If you want to talk more about these frameworks or recommend others, please comment beflow. Thank you very much!
相关推荐
HelloRevit20 分钟前
机器学习、深度学习、大模型 是什么关系?
人工智能·深度学习·机器学习
共享笔记24 分钟前
Adobe Photoshop Elements 2026 正式发布:AI 引擎让修图更简单!
人工智能·adobe·photoshop
芝士AI吃鱼27 分钟前
我为什么做了 Cogniflow?一个开发者关于“信息流”的思考与实践
人工智能·后端·aigc
Juchecar1 小时前
文字与电的相似性:中间载体
人工智能
kyle-fang1 小时前
pytorch-张量
人工智能·pytorch·python
算家计算1 小时前
告别繁琐文档处理!PaddleOCR-VL-vLLM-OpenAI-API本地部署教程:精准解析文本/表格/公式
人工智能·开源
woshihonghonga1 小时前
Dropout提升模型泛化能力【动手学深度学习:PyTorch版 4.6 暂退法】
人工智能·pytorch·python·深度学习·机器学习
该用户已不存在1 小时前
AI编程工具大盘点,哪个最适合你
前端·人工智能·后端
机器学习ing.1 小时前
Vision Transformer(ViT)保姆级教程:从原理到CIFAR-10实战(PyTorch)!
人工智能·深度学习·机器学习
算家计算1 小时前
国产模型新王登基!刚刚,Kimi K2 Thinking发布,多项能力超越GPT-5
人工智能·开源·资讯