apache seatunnel 2.3.12 Changelog

Docs

$Doc$ Added deepwiki to README.md #9354 (#9356)
$Doc$ Fix link error and DynamoDB parameter error (#9447)
$Doc$ $Improve$ update projection supported in Paimon (#9425)
$Doc$ $Zeta$ Add zeta tuning guide (#9539)
$doc$ The link in the document is 404 (#9561)
$Docs$ update jdbc related document (#9679)
$Docs$ $Connector-Iceberg$ Update Iceberg doc with support S3 Tables rest catalogs (#9686)
$Fix$ $Doc$ Fix rest api finished-jobs doc miss SAVEPOINT_DONE (#9676)
$Fix$ $Doc$ Fix transform-v2 TableFilter title (#9528)
$Fix$ $Doc$ Sink's plugin_input is incorrect configurations (#9492)
$Fix$ $Doc$ Update StarRocks doc change schema necessity to true (#9656)
$Improve$ $Doc$ Add GenericDialect support in jdbc doc (#9763)
$Improve$ $Doc$ Add multimodal support description to docs (#9652)
$Improve$ $Doc$ Add SeaTunnel tools into Readme (#9707)
$Improve$ $Doc$ Fix formatting of parameter types (#9753)
$Improve$ $Doc$ Update sql function result type (#9703) (#9711)

Bug Fix

$Bug$ $Connector-V2$ Fix the issue of writing the ORC format Iceberg report "Illegal provider-class name" (#6754) (#9588)
$Bug$ $Connector-V2$ Updates Iceberg version to 1.6.1 (#9387) (#9451)
$Bug$ $seatunnel-connector-http-base$ An NPE (NullPointerException) will occur when the pageField is null (#9498)
$Bug$ $connector-http$ Fix paging request running infinitely (#9504)
$Bug$ $Connector-V2$ fix NPE when decimal type precision is incompatible for Paimon (#9452)
$Bug$ $Connector-V2$ NoSuchMethodError caused by Netty version conflict on Spark 3.3.0 (#9632)
$bug$ $dist$ fix the release packaging exception (#9436)
$bugfix$ $Connector-V2$ Fixed the load driver inaccurate situation (#9468)
$Fix$ $connector-jdbc$ prevent precision loss in Float to BigDecimal conversion (#9670)
$Fix$ $Transform-V2$ Fix embedding output columns vector dimension (#9646)
$Fix$ $CI$ Update iris docker version to fix ci (#9360)
$Fix$ $Connecotr-kafka$ Fix kafka IllegalArgumentException when offset is -1 (#9376)
$Fix$ $Connector-clickhouse$ Fix SeaTunnelRow tableId set error (#9585)
$Fix$ $Connector-File$ Fix parquet support user config schema (#9596)
$Fix$ $Connector-HTTP$ Add default content-type when user not set (#9497)
$Fix$ $Connector-Http$ fix Invalid mime type (#9363)
$Fix$ $connector-http$ fix parsing httpjson, the number of two fields is inconsistent with the import failure (#9103)
$Fix$ $Connector-Iceberg$ Fix Time Zone Issue for Iceberg Timestamp Type (#9460)
$Fix$ $Connector-Jdbc$ Supports reading and writing Postgres network dress types (#9618)
$Fix$ $Connector-Jdbc$ Fixed Vertica data source cannot upsert data. (#9607)
$Fix$ $connector-rabbitmq$ Set default value for durable, exclusive and auto-delete (#9631)
$Fix$ $Connector-V2$ Add Filter for Partitions to Prevent Blocking in KafkaConsumer StreamMode (#9598)
$Fix$ $Connector-V2$ ArrowToSeatunnelRowReader convertSeatunnelRowValue add handle Second TIMESTAMP type (#9393)
$Fix$ $Connector-V2$ Correct typo in batch-size-per-scan option key (#9434)
$Fix$ $Connector-V2$ Fix misleading parameter name in DorisStreamLoad (#9685)
$Fix$ $Connector-V2$ Fix OceanBase Oracle create unsupported data type (#9383)
$Fix$ $Connector-V2$ Fix prometheus check time can not parse double value (#9311)
$Fix$ $Connector-V2$ Fix starrocks decimal column definition generation(#9470) (#9471)
$Fix$ $Connector-V2$ Optimize Paimon DECIMAL type check to prevent precision loss (#9480)
$Fix$ $Connector-V2$ Optimize start mode of kafka recovery job (#9736)
$Fix$ $Connector-V2$ Oracle cdc not update transaction commit when LOB enabled (#9412)
$Fix$ $Connector-V2$ set default tableid in SeaTunnelRow for Debezium JSON deserialization (#9431)
$Fix$ $Connector-V2$ Update catalog table schema of debezium json (#9525)
$Fix$ $Connector-V2$ Update file filter pattern compilation to remove unnecessary quoting (#9658)
$Fix$ $Connector-V2$ Update waitCompaction value for batch mode and writeonly (#9479)
$Fix$ $Connectors-Jdbc$ Postgres supports streaming and batch reading and writing of the interval data type (#9590)
$Fix$ $Connectors-v2$ fix dynamic bucket for paimon sink (#9595)
$fix$ $connectors-v2$ repeated commit cause task exceptions (#9665)
$Fix$ $Core$ fix kotlin jar conflict (#9683)
$Fix$ $E2E$ Fixed e2e test cases of Kafka did not work as expected (#9359)
$Fix$ $engine-server$ Fix the issue of garbled characters in log output (#9594)
$Fix$ $Mongo-CDC$ Fix the issue where mongo isExactlyOnce defaults to true, causing room to malfunction (#9454)
$Fix$ $paimon-e2e$ e2e test error (#9467)
$Fix$ $paimon-e2e$ Optimize Paimon E2E Cases (#9612)
$Fix$ $seatunnel-dist$ fix duplicate dependency (#9458)
$Fix$ $Spark$ Fix source parallelism not working with Spark engine (#9319)
$Fix$ $Transform-V2$ correct handling of 'T' in date format strings (#9406)
$Fix$ $Transform-v2$ Fix custom udf throw exception (#9195)
$Fix$ $Transform-V2$ Reduce embedding precision from double to float (#9635)
$Fix$ $Transform$ Support integer input for from_unixtime function (#9738)
$Fix$ $Zeta$ Avoid redundant checkpoint reads when disabled checkpoint (#9552)
$Fix$ $Zeta$ change parallelStream to stream in SeaTunnelTask to avoid classloader mismatch (#9426)
$Fix$ $Zeta$ Enhance job state display by including pending jobs in status retrieval (#9489)
$Fix$ $Zeta$ Fix Imap resource leak (#9696)
$Fix$ $Zeta$ Fix local mode can not finish (#9549)
$Fix$ $Zeta$ Fix operation thread leak when master switch (#9464)
$Fix$ $Zeta$ Use dynamic port allocation for HTTP in RestApiHttpsForTruststoreTest and update Prometheus image (#9792)
$Fix$ $POM$ Fix transform udf dependency version (#9803)
$Fix$ $Core$ Fix the placeholders cannot pass parameters when using complex config (#9800)
$Future$ $Connector-V2$ Support the automatic creation of non-primary key table (#9219)
$HotFix$ $CI$ The Some CI subtasks are not executed (#9684)
$Hotfix$ $Connector-Jdbc$ Write MySQL to support set collection data type (#9553)
$Hotfix$ $Transform-V2$ Fix some model return number of dimensions (#9644)
$Hotfix$ $Zeta$ Fix custom https port configuration not working (#9705)

Feature

$Chore$ fix typos filed -> field (#9757)
$Chore$ $Core$ Remove useless parameter in ConfigBuilder (#9494)
$Feature$ $connector-file$ Add configurable sheet_max_rows support for Excel sink connector (#9668)
$Feature$ Version from 2.3.11 to 2.3.12 (#9366)
$Feature$ $Connector-Clickhouse$ Support Clickhouse multi table source read (#9704)
$Feature$ $Connector-File-Hadoop$ Support multi table sink feature for HdfsFile (#9651)
$Feature$ $connector-hive$ hive sink connector support overwrite mode #7843 (#7891)
$Feature$ $connector-kudu$ implement the filter (#9405)
$Feature$ $connector-paimon$ Paimon connector supports paimon privilege (#9722)
$Feature$ $connector-tdengine$ Support subtable and fieldNames in tdengine source (#9593)
$Feature$ $connector-v2$ Add Sensorsdata Connector Support #9323 (#9432)
$Feature$ $Connector-V2$ Jdbc mysql support read tinyint(1) to byte(tinyint) (#9373)
$Feature$ $Connector-V2$ Support databend source/sink connector (#9331)
$Feature$ $Connector-V2$ Support like predicate pushdown in paimon (#9484)
$Feature$ $Connector-v2$ Support multi paimon source (#9759)
$Feature$ $Connector-V2$ $JDBC$ Add presto/trino dialect (#9388)
$Feature$ $Connectors-v2$ Optimize the size of CDC JAR Files (#9546)
$Feature$ $Connectors-v2$ Support auto-increment id for FakeSource (#9505)
$feature$ $connectors-v2$ Support in predicate pushdown in paimon (#9379)
$Feature$ $Connectors-v2$ Support Mysql8.4+ for mysql-cdc (#9720)
$Feature$ $Connectors-V2$ $Elasticsearch$ Support vector transformation sink (#9330)
$Feature$ $Connectors-v2$ Paimon version upgrade to 1.1.1 (#8074)
$Feature$ $Core$ Add plugin directory support for each connector (#9650)
$Feature$ $Core$ Support show cluster members information in seatunnel-cluster scripts (#9502)
$Feature$ $elasticsearch-connector$ Add API key authentication support (#9610)
$Feature$ $Format$ Improve maxwell_json,canal_json,debezium_json format add ts_ms and table (#9701)
$Feature$ $Jdbc$ Support read multiple tables by regular expressions (#9380)
$Feature$ $MySQL CDC$ MySQL cdc support start by time (#9735)
$Feature$ $Redis$ Add redis key into the result record (#9574)
$Feature$ $Sink$ File support new format: maxwell_json,canal_json,debezium_json (#9278) (#9336)
$Feature$ $SQL function$ Enhance SQL Transform COALESCE functions to support type cast (#9299)
$Feature$ $SQL Transform$ Check SQL Cast will fail or not during the SQL parsing stage (#9600)
$Feature$ $Transform-V2$ Add TRIM_SCALE function for sql transform (#9700)
$Feature$ $transform-v2$ Data Validator Transform support (#9445)
$Feature$ $transform-v2$ sql transform support multi_if function (#9154)
$Feature$ $Transform-V2$ Support jsonpath extracting multiple fields in a configuration block (#9712)
$Feature$ $Transform-V2$ Support vector series sql function (#9765)
$Feature$ $Transform$ Add Murmur64 function to obtain the lower 64 bit (#9748)
$Feature$ $Transform$ Support single/batch mode vectorization using Amazon Titan & cohere embedding model (#9120)
$Feature$ $TransForms-V2$ Support scala dynamic compile (#9625)
$Feature$ $Transform V2$ Add vector dimension reduction transform (#9783)
$Feature$ $Zeta$ Add metrics for task intermediate queue size (#9550)
$Feature$ $Zeta$ Add startTime to the job info (#9400)
$Feature$ $Zeta$ extended jobStateEvent event listening for zeta (#9689)
$Feature$ $Zeta$ Support SQL format in REST API (#9802)
$improve$ jdbc options (#9541)
$Improve$ JdbcInputFormat nextRecord Exception throw TableId (#9374)
$improve$ pulsar options (#9180)
$improve$ rocketmq options (#9251)
$improve$ selectdb options (#9252)
$improve$ socket options (#9517)
$Improve$ table_store options (#9515)
$improve$ tdengine options (#9399)
$improve$ typesense options (#9398)
$Improve$ $API$ Add metadata schema into catalog table (#9586)
$Improve$ $API$ Optimize the enumerator API semantics and reduce lock calls at the connector level (#9671)
$Improve$ $CI$ Disable close stable issue and JdbcVerticaIT (#9742)
$Improve$ $CI$ Update state action fix not all pr/issue be checked (#9557)
$Improve$ $connector-clickhouse$ Clickhouse support parallelism reading schema (#9446)
$Improve$ $Connector-file$ Add configurable binary chunk size support to BinaryReadStrategy (#9391)
$Improve$ $Connector-Hudi$ Add pre-combine field option for hudi sink (#9496)
$improve$ $Connector-jdbc$ add comments when schema not include all columns (#9559)
$Improve$ $Connector-milvus$ update milvus-sdk-java to 2.5.11 (#9710)
$Improve$ $Connector-V2$ Optimize Milvus doc and e2e test case (#9766)
$Improve$ $Connector-V2$ Add customizable row delimiter support for text file processing (#9608)
$Improve$ $Connector-V2$ Add tunnel_endpoint option to MaxCompute source for emulator test (#9548)
$improve$ $Connector-V2$ delete jdbc param support_upsert_by_query_primary_key_exist (#9408)
$Improve$ $Connector-V2$ File Source Support filtering files by last modified time. (#9526)
$Improve$ $Connector-V2$ Replace deprecated createDownloadSession by buildDownloadSession (#9555)
$Improve$ $Connector-V2$ Support like predicate pushdown in paimon (#9653)
$Improve$ $Connector-V2$ Support maxcompute sink writer upsert/delete action with upsert session mode (#9462)
$Improve$ $Connector-V2$ Support maxcompute sink writer with timestamp field type (#9234)
$Improve$ $Connector-V2$ Use key_field_name option when reading Redis hash data (#9642)
$Improve$ $Core$ Configure spotless to auto-replace imports with shaded packages. (#9655)
$Improve$ $Core$ Unify the aws-sdk-v2 version to 2.31.30 (#9698)
$Improve$ $Core$ Update apache common to apache common lang3 (#9694)
$Improve$ $Csv$ support configurable CSV delimiter in file connector (#9660)
$Improve$ $E2E$ Optimize Hudi E2E Cases (#9752)
$Improve$ $E2E$ Optimize iceberg E2E Cases (#9769)
$Improve$ $Oracle-CDC$ Remove duplicate load table names (#9357)
$Improve$ $Transform-V2$ Improve sql transform exception to locate error expression (#9227)
$Improve$ $Transform-V2$ Remove unnecessary unquoting method and streamline date parsing (#9478)
$Improve$ $Transform-V2$ SQL transform EXTRACT function support more fields (#9342)
$Improve$ $Transform$ $SQL$ Support cast to TINYINT/SMALLINT (#9347)
$Improve$ $Zeta$ Add verification for Env options (#9569)
$Improve$ $Zeta$ Improve local mode doc and default config (#9770)
$Improve$ $Zeta$ Make sure the pending job schedule always works (#9532)
$Improve$ $Zeta$ Optimize unstable CheckpointErrorRestoreEndTest cases (#9619)
$Improve$ $Zeta$ Remove distributed lock when store metrics imap (#9776)
$Improve$ $Zeta$ Set the default slot-num value to twice the number of CPU cores (#9601)
$improve$ improve FE node failover logging for better observability (#9657)