机器学习平台整理

开源系列

cube开源一站式云原生机器学习平台:https://blog.csdn.net/luanpeng825485697/article/details/123619334

github:https://github.com/tencentmusic/cube-studio

kubeflow参考

官网:https://www.kubeflow.org/docs/started/

参考:https://www.jianshu.com/p/192f22a0b857

AirFlow/NiFi/MLFlow/KubeFlow进展:https://blog.csdn.net/chenhuipin1173/article/details/100913909

最好的任务编排工具:Airflow vs Luigi vs Argo vs MLFlow

总结

一句话总结就是:kubeflow是一个为 Kubernetes 构建的可组合,便携式,可扩展的机器学习技术栈。

支持的训练架构-https://www.kubeflow.org/docs/components/training/

英文对比:

https://aicurious.io/posts/airflow-mlflow-or-kubeflow-for-mlops/

https://devsamurai.vn/blog/ml-platform-kuberflow-mlflow-argo-airflow/

通用型选airflow

机器学习偏向大规模选kubeflow

机器学习偏向小规模选mlflow

bash 复制代码
5. How to choose between Airflow+Mlflow, and Kubeflow?

To sum up, I have some recommendations from my personal perspective:

    If your system needs to deal with multiple types of workflow, not just machine learning, Airflow may support you better. It is a mature workflow orchestration frameworks with support for a lot of operators besides machine learning.
    If you want a system with predesigned patterns for machine learning, and run at large scale on Kubenetes clusters, you may want to consider Kubeflow. Many ML specific components in Kubeflow can save your time implementing from scratch in Airflow.
    If you want to deploy MLOps in a small scale system (for example, a workstation, or a laptop), picking Airflow+MLflow stack can eliminate the need of setting up and running a Kubenetes system, and save more resources for the main tasks.

This blog post has briefly shown the differences between three popular MLOps frameworks (Airflow, MLflow and Kubeflow). Hope that it helps you in making decision between 2 stacks (Airflow + MLflow and Kubeflow). If you want to talk more about these frameworks or recommend others, please comment beflow. Thank you very much!
相关推荐
peterfei5 分钟前
若爱 IfAI v0.4.2 发布:技能市场上线,重新定义 AI 编辑器的可扩展性
人工智能·开源
阿杰学AI11 分钟前
AI核心知识129—大语言模型之 向量数据库(简洁且通俗易懂版)
数据库·人工智能·ai·语言模型·自然语言处理·向量数据库·vector database
PILIPALAPENG17 分钟前
第3周 Day 2:Function Calling —— 让 Agent 听懂人话,自己干活
前端·人工智能·python
阿里云大数据AI技术29 分钟前
PAI Physical AI Notebook详解8:Isaac Lab Arena 全身机器人机动+操控工作流
人工智能
高木木的博客43 分钟前
数字架构智能化测试平台(1)--总纲
人工智能·python·nginx·架构
wanghowie44 分钟前
11. AI 客服系统架构设计:不是调 API,而是系统工程
人工智能·系统架构
袋鼠云数栈UED团队1 小时前
基于 OpenSpec 实现规范驱动开发
前端·人工智能
Raink老师1 小时前
【AI面试临阵磨枪】什么是 Tokenization?子词分词(Subword)的优缺点?
人工智能·ai 面试
迷你可可小生1 小时前
面经(三)
人工智能·rnn·lstm
云烟成雨TD1 小时前
Spring AI Alibaba 1.x 系列【28】Nacos Skill 管理中心功能说明
java·人工智能·spring