Apache DolphinScheduler 3.2.1 版本发布:增强功能与安全性的全面升级

近期,Apache DolphinScheduler 社区激动地宣布 3.2.1 版本的发布。此次更新不仅着力解决了前一版本(3.2.0)中遗留的问题,而且引入了一系列的功能增强和优化措施。

原先的问题主要源于部分重要代码在发布过程中未能成功合并(cherry-pick),加之这部分代码的合并过程较为复杂,因此,3.2.1 版本基于 2024年2月的 dev 分支代码,剔除了一些不兼容的特性后发布。

全部 Changeloghttps://github.com/apache/dolphinscheduler/releases/tag/3.2.1

下载地址https://dolphinscheduler.apache.org/zh-cn/download/3.2.1

主要修复和功能增强

新特性和优化

  • SQL 任务现支持使用 druid 进行 SQL 分割,支持设置 maxRows
  • 支持自定义 HTTP body 渲染。
  • Kubernetes (k8s) 现支持自定义标签 (label)。
  • 新增支持阿里云语音告警源。
  • Helm chart 现支持 JDBC 注册中心;支持任务类型过滤。

关键问题修复

  • 修复从 3.1.x 升级到 3.2.x 的失败问题。
  • 解决任务组件在使用资源中心时只能使用绝对全路径的限制。
  • 修复启动参数优先级设置错误。
  • 解决数据质量任务无法执行的问题。
  • 修复任务组队列失效问题。
  • 解决任务定义列表修改时任务消失的问题。
  • 修复特殊情况下删除工作流实例导致的空指针异常(NPE)。
  • 解决 Master 和 Worker 之间的通信问题。
  • 修复 Kyuubi 数据源在 UI 中不显示的问题。

安全性改进

此版本也对几个关键的 CVE 问题进行了修复,包括:

  • CVE-2023-49250
  • CVE-2023-51770
  • CVE-2023-50270
  • CVE-2023-49068
  • CVE-2023-49109

BugFix

  • fix: Resource relate path invalid when tenant change (#15581)

  • fix\]\[worker\]:Fix the issue of missing disk usage reporting information in the worker (#15565)

  • Fix create parent directory will cause FileAlreadyExistsException (#15576)

  • Fix Recover WorkflowInstance will casue workflow Instance state is success but task insatnce is killed/paused (#15574)

  • fix: data quality may fail in docker mode (#15563)

  • fix: start param for wf not work (#15544)

  • fix: ddl without drop exists (#14128)

  • fix switch js (#15487)

  • fix: data quality can not use (#15551)

  • Fix createFile with permission will not work (#15556)

  • Bug\]\[force-success\] force success add end time (#15144)

  • Fix TaskGroupQueue will never be wakeup due to wakeup failed at one time (#15528) Exit JVM when OOM (#15538)

  • Fix exception occur in RpcServer side, it will not be sent to RpcClient (#15536)

  • front: When you edit a task in the task definition list, the front task list is displayed (#12819)

  • Fix\] \[Bug\] Change default version of Workflow/TaskDefinition to `1` (#15498)

  • fix workflow will have same updatetime when import (#14810)

  • BUG\] #15013 Fix retryInterval in RetryPolicy will never be used in RetryUtils (#15014)

  • Fix PostgresqlDatabaseContainerProvider get Image is incorrect (#15434)

  • Bug\]\[Api\] Fix NPE when deleting a workflow instance (#15485)

  • Fix notify failover WorkflowInstance will cause NPE (#15499)

  • HotFix\] Fix createTaskInstanceWorkingDirectory failed if the old path exist (#15377)

  • Bug\]\[Master\]serial_wait strategy workflow unable to wake up (#15270)

  • Bug\] \[Audit log\] Fix Audit log UI query error (#15427)

  • Set TaskGroupQueue updateTime when force start (#15510)

  • TaskGroupPriority only compare When TaskGroup is same (#15486)

  • Remove taskQueue and looper in worker (#15292)

  • Display the resource file doesn't exist message in task create page (#15350)

  • Recreate new TaskInstance Working Directory when exist in worker (#15358)

  • Bug\] Close SSH session after remote shell finish (#15348)

  • Fix resource file usage(Delete Resource/ResourceUser which is deprecated)

  • Bug\]\[Master\] send ACK event timeout (#15346)

  • Fix #15129\] \[Dependent\] The date rules of the dependent node are ambiguous. (#15289)

  • HotFix\] Fix TaskOutputParameterParser might OOM if meed a bad output param expression (#15264)

  • Set the tenant as the owner in final stage (#15256)

  • Use chown to set the file owner (#15240)

  • Fix\] Change HTTP plugin timeout param to number type (#15234)

  • Fix docs style is incorrect by CI pass (#15167)

  • Expire session when update user password (#15219)

  • Fix home page workflow instance miss status (#15193)

  • fix security issue (#15192)

  • fix can't stop bug (#15191)

  • Remove API Result in Service (#15181)

  • Exclude DataSourceAutoConfiguration in worker server (#15169)

  • Bug\] Fix TriggerRelationMapper cannot work due to miss DatabaseIdProvider (#15153)

  • Fix incorrect button display text (#15160)

  • Fix\]\[Data Quality\] Change t_ds_dq_rule_input_entry field name fix PostgreSQL not support value issue (#14992)

  • Bug\] Fix endless loop (#15092)

  • Fix confusing constant string for unit convertor (#15126)

  • fix-#11726\] fix error when set connection proerty both in the URL and an argument (#15093)

  • E2E\]\[Bug\] Fix k8s-e2e (#15098)

  • Fix-15036\] \[API\] Fix task definition edit doesn't work (#14801)

  • Fix missing Kyuubi type in UI (#15051)

  • Fix-14885\]\[pom\] fix spotless format file path (#14889)

  • Revert "[Bug] [Resource] fix resource delete bug (#15003)

  • Bug\] \[Resource\] fix resource delete bug (#15003) Delete File generated by UT (#15022)

  • Improvement\]\[UT\] Improve Worker registry coverage (#15380)

  • Improvement\]\[HTTP\] support custom rendering of http body (#15531)

  • Feature-15475\]\[DinkyTask\] DinkyTask supports Dinky-1.0.0 and common sql (#15479)

  • Enable set ServerLoadProtection fot Master/Worker (#15439)

  • Feature\] timed scheduler Improvement (#15449)

  • Enhancement\]\[API\]Enhance mysql connection properties (#15433)

  • Add config for defaultTenantEnabled (#15391)

  • Use DefaultUncaughtExceptionHandler to log the uncached exception (#15496)

  • adjust the sequence of alarm group and add validate (#15382)

  • Use Druid to split sql (#15367)

  • optimize add select filter (#15378)

  • Improvement\]\[Helm\] using helm-docs to generate docs automatically (#15299)

  • Optimize server startup log (#15362)

  • Improvement\]\[E2E\] support e2e compose v2 fix code style (#15325)

  • delete debugger (#15316)

  • Set maxRows in SqlTask (#15342)

  • Feature-15146\]\[dolphinscheduler-task-sqoop\] add sqoop source/target type (#15146)

  • Improvement-15260\]\[dolphinscheduler-datasource-hana\] add hana related dependencies (#15260)

  • Improvement\] Move delay calculation to Master (#15278)

  • Support parse task output params under multiple log (#15244)

  • Improvement-15009\]\[Parameter\] Change project parameter value to text (#15010)

  • Improvement\] Clean up Scheduler logic (#15198)

  • Improvement\]\[Helm\] support task type filter (#15179)

  • Improvement\]\[Alert\] Add timeout params for HTTP plugin (#15174)

  • Improvement\]\[K8S\] Remove ResourceQuota (#14991)

  • DSIP-19\] Support sagemaker connections in the connection center, as well as external connections to the connection center in sagemaker tasks (#14976)

  • Feature-14832\]\[Listener\]Implementation of Listener Mechanism (#14981)

  • Feature-14678\]\[Master\]\[UI\]Dependent task parameter passing (#14702)

  • Add IT for dolphinscheduler-tools module (#15043)

  • Set kubectl version to v1.28.3 (#15053)

  • Add dolphinscheduler-dao-plugin module (#15019)

  • improvement\]\[Resources\] Improve details page return to the previous list page (#14951)

  • Improvement\]\[Registry\]\[Jdbc\] Add jdbc registry config in helm charts (#14431)

Document

  • Doc\]\[Docker\] fix typo on start with docker (#15534)

  • Change download url in backend.yml (#15526)

  • Doc\]\[K8S\] Add DS K8S Operator into k8s deployment character (#15516)

  • Remove unused cache-evict.png (#15220)

  • Doc-15500\]\[Task\] Update cli opts of spark and flink (#15501)

  • Doc\]remove temporary markdown comments (#15385)

  • Add deploy on Terraform on README (#15189)

  • Modify the documentation that python task will not work properly when '\n' indicates the presence of a variable and needs to use 'repr(value)' (#15145)

  • Docs\] fix typo (#15032)

  • merge schema 330 into 321 and change docs (#15582)

  • Set the workflow instance ready state to running in failover (#15572)

  • cp: Reduce the size of tarball to continue ASF release (#15004)

  • chore: Docs change for 3.2.1 release (#15539)

  • DS-15489\]\[style\]rename the vo object suffix (#15504)

感谢所有贡献者的辛勤付出,特别是以下成员(排名不分先后): | 致谢名单 | | | |--------------|--------------|----------------| | AliceXiaoLu | Gallardot | clovelll | | BaiJv | John Huang | davidzollo | | CommandCV | JohnZp | dbac | | Dyqer | Orange-Summer | destroydestiny | | EricGao888 | Radeity | devosend | | SbloodyS | SusurHe | fuchanghai | | aiwenmo | arlendp | imizao | | c3Vu | caishunfeng | imp2002 | | izualzhy | kevinkelin | lenboo | | lgcareer | liunaijie | liyou | | lizhenglei | ly109974 | pegasas | | qingwli | reele | rickchengx | | ruanwenjun | sdhzwc | sleo | | tangjiaolong | xdu-chenrj | xinxingi | | xjlgod | xujiaqiang | zhanqian | | zhaohehuhu | zhihuasu | zhongjiajie | | zhuangchong | zhutong6688 | |

本文由 白鲸开源科技 提供发布支持!

相关推荐
你觉得20511 小时前
哈尔滨工业大学DeepSeek公开课:探索大模型原理、技术与应用从GPT到DeepSeek|附视频与讲义下载方法
大数据·人工智能·python·gpt·学习·机器学习·aigc
啊喜拔牙11 小时前
1. hadoop 集群的常用命令
java·大数据·开发语言·python·scala
别惊鹊12 小时前
MapReduce工作原理
大数据·mapreduce
8K超高清12 小时前
中国8K摄像机:科技赋能文化传承新图景
大数据·人工智能·科技·物联网·智能硬件
2401_8712905813 小时前
MapReduce 的工作原理
大数据·mapreduce
SelectDB技术团队14 小时前
Apache Doris 2025 Roadmap:构建 GenAI 时代实时高效统一的数据底座
大数据·数据库·数据仓库·人工智能·ai·数据分析·湖仓一体
你觉得20514 小时前
浙江大学朱霖潮研究员:《人工智能重塑科学与工程研究》以蛋白质结构预测为例|附PPT下载方法
大数据·人工智能·机器学习·ai·云计算·aigc·powerpoint
益莱储中国15 小时前
世界通信大会、嵌入式展及慕尼黑上海光博会亮点回顾
大数据
Loving_enjoy15 小时前
基于Hadoop的明星社交媒体影响力数据挖掘平台:设计与实现
大数据·hadoop·数据挖掘
浮尘笔记15 小时前
go-zero使用elasticsearch踩坑记:时间存储和展示问题
大数据·elasticsearch·golang·go