提交spark任务运次报错
java
06/03 18:27:50 INFO Client: Setting up container launch context for our AM
25/06/03 18:27:50 INFO Client: Setting up the launch environment for our AM container
25/06/03 18:27:50 INFO Client: Preparing resources for our AM container
25/06/03 18:27:50 WARN Client: Neither spark.yarn.jars nor spark.yarn.archive is set, falling back to uploading libraries under SPARK_HOME.
25/06/03 18:27:59 INFO Client: Uploading resource file:/tmp/spark-27957b63-3767-457f-a931-cc7f84917c3e/__spark_libs__545909112269949051.zip -> hdfs://everdc/user/hdfs/.sparkStaging/application_1748248020756_0040/__spark_libs__545909112269949051.zip
25/06/03 18:28:02 INFO Client: Uploading resource file:/opt/apps/everdc/spark_jars/everdc-stand-model-engine-spark-4.9.0.jar -> hdfs://everdc/user/hdfs/.sparkStaging/application_1748248020756_0040/everdc-stand-model-engine-spark-4.9.0.jar
25/06/03 18:28:02 INFO Client: Uploading resource file:/app/logs/model_run_2.json -> hdfs://everdc/user/hdfs/.sparkStaging/application_1748248020756_0040/model_run_2.json
25/06/03 18:28:02 INFO Client: Uploading resource file:/tmp/spark-27957b63-3767-457f-a931-cc7f84917c3e/__spark_conf__4210376678626642445.zip -> hdfs://everdc/user/hdfs/.sparkStaging/application_1748248020756_0040/__spark_conf__.zip
25/06/03 18:28:03 INFO SecurityManager: Changing view acls to: root,hdfs
25/06/03 18:28:03 INFO SecurityManager: Changing modify acls to: root,hdfs
25/06/03 18:28:03 INFO SecurityManager: Changing view acls groups to:
25/06/03 18:28:03 INFO SecurityManager: Changing modify acls groups to:
25/06/03 18:28:03 INFO SecurityManager: SecurityManager: authentication disabled; ui acls disabled; users with view permissions: Set(root, hdfs); groups with view permissions: Set(); users with modify permissions: Set(root, hdfs); groups with modify permissions: Set()
25/06/03 18:28:03 INFO Client: Deleted staging directory hdfs://everdc/user/hdfs/.sparkStaging/application_1748248020756_0040
Exception in thread "main" java.lang.NoSuchMethodError: org.apache.hadoop.conf.Configuration.getPassword(Ljava/lang/String;)[C
at org.apache.spark.SSLOptions$.$anonfun$parse$8(SSLOptions.scala:194)
at scala.Option.orElse(Option.scala:447)
at org.apache.spark.SSLOptions$.parse(SSLOptions.scala:194)
at org.apache.spark.SecurityManager.<init>(SecurityManager.scala:98)
at org.apache.spark.deploy.yarn.Client.createContainerLaunchContext(Client.scala:1137)
at org.apache.spark.deploy.yarn.Client.submitApplication(Client.scala:220)
at org.apache.spark.deploy.yarn.Client.run(Client.scala:1327)
at org.apache.spark.deploy.yarn.YarnClusterApplication.start(Client.scala:1764)报错中org.apache.hadoop.conf.Configuration.getPassword，这个方法找不到，hadoop在2.7以上版本中才有这个方法，查看运行hadoop环境hadoop是2.4.5的版本的，升级到hadoop2.7后正常