Hive 源码 - 技术栈

hive 编译

issue

Failed to execute goal com.github.os72:protoc-jar-maven-plugin:3.5.1.1:run (default) on project hive-standalone-metastore: Error resolving artifact: com.google.protobuf:protoc:2.5.0: The following artifacts could not be resolved: com.google.protobuf:protoc:exe:osx-aarch_64:2.5.0 (absent): Could not transfer artifact com.google.protobuf:protoc:exe:osx-aarch_64:2.5.0 from/to maven-default-http-blocker (http://0.0.0.0/)

可以增加编译选项

复制代码

 -Dos.arch=x86_64

可参见

https://qileq.com/article/202202230001/

issue

ERROR\] Failed to execute goal on project hive-service: Could not resolve dependencies for project org.apache.hive:hive-service:jar:3.1.3: Failed to collect dependencies at org.apache.directory.server:apacheds-server-integ:jar:1.5.6 -\> org.apache.directory.client.ldap:ldap-client-api:jar:0.1-SNAPSHOT: Failed to read artifact descriptor for org.apache.directory.client.ldap:ldap-client-api:jar:0.1-SNAPSHOT: The following artifacts could not be resolved: org.apache.directory.client.ldap:ldap-client-api:pom:0.1-SNAPSHOT (absent): Could not transfer artifact org.apache.directory.client.ldap:ldap-client-api:pom:0.1-SNAPSHOT from/to maven-default-http-blocker (http://0.0.0.0/): Blocked mirror for repositories: \[tbds (http://tbdsrepo.oa.com/repository/tbds/, default, disabled), 添加 repo Codehaus repository codehaus-mule-repo https://repository-master.mulesoft.org/nexus/content/groups/public/ default 该问题，可以参见 这个 hive issue ： https://issues.apache.org/jira/browse/HIVE-21777 * issue [ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:3.6.1:compile (default-compile) on project hive-webhcat: Compilation failure [ERROR] hive/hcatalog/webhcat/svr/src/main/java/org/apache/hive/hcatalog/templeton/Main.java:[258,31] 对于FilterHolder(java.lang.Class), 找不到合适的构造器 [ERROR] 构造器 org.eclipse.jetty.servlet.FilterHolder.FilterHolder(org.eclipse.jetty.servlet.Source)不适用 [ERROR] (参数不匹配; java.lang.Class无法转换为org.eclipse.jetty.servlet.Source) [ERROR] 构造器 org.eclipse.jetty.servlet.FilterHolder.FilterHolder(java.lang.Class)不适用 [ERROR] (参数不匹配; java.lang.Class无法转换为java.lang.Class) [ERROR] 构造器 org.eclipse.jetty.servlet.FilterHolder.FilterHolder(javax.servlet.Filter)不适用 [ERROR] (参数不匹配; java.lang.Class无法转换为javax.servlet.Filter) [ERROR] 将 maven 版本 从 3.9.4 降低到 3.2.5 后该问题解决 ##### Hive 写入 即使 设置了 set hive.exec.mode.local.auto=true; 也会回滚，当 input 文件较多时。 Query ID = hadoop_20240109112435_62531b28-9651-4d82-968d-4ff26142326d Total jobs = 3 Launching Job 1 out of 3 Number of reduce tasks not specified. Estimated from input data size: 1 In order to change the average load for a reducer (in bytes): set hive.exec.reducers.bytes.per.reducer= In order to limit the maximum number of reducers: set hive.exec.reducers.max= In order to set a constant number of reducers: set mapreduce.job.reduces= Cannot run job locally: Number of Input Files (= 7) is larger than hive.exec.mode.local.auto.input.files.max(= 4)