提交入口
org.apache.flink.table.api.internal.TableEnvironmentImpl#executeSql
@Override
public TableResult executeSql(String statement) {
List<Operation> operations = getParser().parse(statement);
if (operations.size() != 1) {
throw new TableException(UNSUPPORTED_QUERY_IN_EXECUTE_SQL_MSG);
}
Operation operation = operations.get(0);
return executeInternal(operation);
}
提交过程概述
sql--》operation--》Transformation--》pipeline(StreamGraph)--》JobGraph--》deployJobCluster
具体过程
1、查询转成operation
sql-->SqlNodeList-->flinkPlanner.validate(sqlNode)-->Optional<Operation> operation = SqlNodeConverters.convertSqlNode(validated, context);
2、operation转成pipeline(StreamGraph)
org.apache.flink.table.api.internal.TableEnvironmentImpl#executeQueryOperation
executeQueryOperation--》
2.1、operation转成Transformation
List<Transformation<?>> transformations = translate(Collections.singletonList(sinkOperation));--》
org.apache.flink.table.planner.delegation.PlannerBase#translate
2.2、Transformation转成pipeline
Pipeline pipeline =
execEnv.createPipeline(
transformations, tableConfig.getConfiguration(), defaultJobName);
2.3、pipeline转成streamGraph
final StreamGraph streamGraph = executionEnvironment.generateStreamGraph(transformations); // StreamGraph是Pipeline接口的实现类
3、将StreamGraph转成JobGraph
org.apache.flink.streaming.api.environment.StreamExecutionEnvironment#executeAsync(org.apache.flink.streaming.api.graph.StreamGraph)
JobClient jobClient = execEnv.executeAsync(pipeline);
3.1、final PipelineExecutor executor = getPipelineExecutor();
CompletableFuture<JobClient> jobClientFuture =
executor.execute(streamGraph, configuration, userClassloader);
3.2、createJobGraph#new StreamingJobGraphGenerator(
userClassLoader, streamGraph, jobID, serializationExecutor)
.createJobGraph();
4、将jobGraph提交到集群
org.apache.flink.client.deployment.executors.AbstractJobClusterExecutor#execute
4.1、获取集群信息
final ClusterClientProvider<ClusterID> clusterClientProvider =
clusterDescriptor.deployJobCluster(
clusterSpecification, jobGraph, configAccessor.getDetachedMode
4.2、提交任务到集群
return CompletableFuture.completedFuture(
new ClusterClientJobClientAdapter<>(
clusterClientProvider, jobGraph.getJobID(), userCodeClassloader));