IDEA编写flinkSQL(快速体验版本,--无需配置环境)

相关资料

文档内容 链接地址
datagen生成器 https://nightlies.apache.org/flink/flink-docs-release-1.16/docs/connectors/table/datagen/
print 生成器 https://nightlies.apache.org/flink/flink-docs-release-1.16/docs/connectors/table/print/

准备工作

优点就是下载个idea就能体验,无需配置的环境(如 数据源等)

1、idea 开发工具

2、创建 maven 项目 -- archetype 选择quickstart 表示java开发

java代码

代码逻辑

1、采用datagen 生成器,作为数据 source

2、采用print 作为打印器,作为sink 直接输出

java 复制代码
package org.example;

import org.apache.flink.streaming.api.environment.StreamExecutionEnvironment;
import org.apache.flink.table.api.Table;
import org.apache.flink.table.api.bridge.java.StreamTableEnvironment;

public class FlinkSqlDemo {
    public static void main(String[] args) throws Exception {
        // 创建执行环境
        StreamExecutionEnvironment env = StreamExecutionEnvironment.getExecutionEnvironment();
        StreamTableEnvironment tableEnv = StreamTableEnvironment.create(env);

        // 创建输入表(使用DataGen生成测试数据)
        String sourceDDL = "CREATE TABLE user_behavior (\n" +
                "    user_id BIGINT,\n" +
                "    behavior STRING,\n" +
                "    ts TIMESTAMP(3)\n" +
                ") WITH (\n" +
                "    'connector' = 'datagen',\n" +
                "    'rows-per-second' = '1',\n" +
                "    'fields.user_id.kind' = 'random',\n" +
                "    'fields.user_id.min' = '1',\n" +
                "    'fields.user_id.max' = '2',\n" +
                "    'fields.behavior.length' = '2'\n" +
                ")";

        // 创建输出表(打印结果)
        String sinkDDL = "CREATE TABLE print_table (\n" +
                "    behavior STRING,\n" +
                "    cnt BIGINT\n" +
                ") WITH (\n" +
                "    'connector' = 'print'\n" +
                ")";

        // 执行DDL
        tableEnv.executeSql(sourceDDL);
        tableEnv.executeSql(sinkDDL);

        // 执行查询并插入结果
        Table resultTable = tableEnv.sqlQuery(
                "SELECT behavior, COUNT(*) AS cnt " +
                        "FROM user_behavior " +
                        "GROUP BY behavior"
        );

        // 插入到输出表
        resultTable.executeInsert("print_table").await();

        // 执行任务(流式任务需要保持运行)
        env.execute("Flink SQL Demo");
    }
}

pom依赖配置

`

4.0.0

复制代码
<groupId>org.example</groupId>
<artifactId>flinklearn</artifactId>
<version>1.0-SNAPSHOT</version>
<packaging>jar</packaging>


<properties>
    <maven.compiler.source>8</maven.compiler.source>
    <maven.compiler.target>8</maven.compiler.target>
    <flink.version>1.17.1</flink.version>
</properties>

<dependencies>
    <!-- Flink Core -->
    <dependency>
        <groupId>org.apache.flink</groupId>
        <artifactId>flink-java</artifactId>
        <version>${flink.version}</version>
    </dependency>
        <!-- Flink实时流-->
    <dependency>
        <groupId>org.apache.flink</groupId>
        <artifactId>flink-streaming-java</artifactId>
        <version>${flink.version}</version>
    </dependency>
    <dependency>
        <groupId>org.apache.flink</groupId>
        <artifactId>flink-clients</artifactId>
        <version>${flink.version}</version>
    </dependency>

    <!-- Flink Table -->
    <dependency>
        <groupId>org.apache.flink</groupId>
        <artifactId>flink-table-planner_2.12</artifactId>
        <version>${flink.version}</version>
    </dependency>
    <dependency>
        <groupId>com.ververica</groupId>
        <artifactId>flink-sql-connector-mysql-cdc</artifactId>
        <version>2.0.1</version>
    </dependency>
</dependencies>

<build>
    <plugins>
        <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-shade-plugin</artifactId>
            <version>3.2.4</version>
            <executions>
                <execution>
                    <phase>package</phase>
                    <goals>
                        <goal>shade</goal>
                    </goals>
                    <configuration>
                        <transformers>
                            <transformer implementation="org.apache.maven.plugins.shade.resource.ManifestResourceTransformer">
                                <mainClass>com.example.FlinkSqlDemo</mainClass>
                            </transformer>
                        </transformers>
                    </configuration>
                </execution>
            </executions>
        </plugin>
    </plugins>
</build>
相关推荐
❀͜͡傀儡师1 天前
docker 部署Flink和传统部署
docker·容器·flink
舒一笑1 天前
信息的建筑学:MyBatis Log Panda 如何重构开发者的认知地图
后端·sql·intellij idea
core5121 天前
不借助框架实现Text2SQL
sql·mysql·ai·大模型·qwen·text2sql
张人玉1 天前
SQLite 快速入门 Cheat Sheet
数据库·sql·sqlite
DarkAthena1 天前
【DuckDB】活用marco以兼容GaussDB的SQL执行
数据库·sql·duckdb
x***01061 天前
SQL 注入漏洞原理以及修复方法
网络·数据库·sql
q***31891 天前
mysql 迁移达梦数据库出现的 sql 语法问题 以及迁移方案
数据库·sql·mysql
Gauss松鼠会1 天前
【openGauss】OPENGAUSS/POSTGRESQL 中float类型到int类型的隐式转换
数据库·sql·database·opengauss
Linux Huang1 天前
【Dinky】IDEA运行出现HistoryServer异常
java·hadoop·flink·intellij idea
渣渣盟1 天前
Flink分布式文件Sink实战解析
分布式·flink·scala·1024程序员节