提示：文章写完后，目录可以自动生成，如何生成可参考右边的帮助文档
[Sqoop看这篇文章就够了_must contain 'conditions' in where clause._SoWhat1412的博客-CSDN博客](https://blog.csdn.net/qq_31821675/article/details/106682454 "Sqoop看这篇文章就够了_must contain 'conditions' in where clause._SoWhat1412的博客-CSDN博客")
大数据环境
C:\Windows\System32\drivers\etc
修改ip和hostname的对应关系
127.0.0.1 view-localhost # view localhost server
192.168.52.150 hadoop01 hadoop01.itcast.cn
192.168.52.151 hadoop02 hadoop02.itcast.cn
检查环境
ping 通环境测试
检查hadoop的版本
root@hadoop01 \~# hadoop version
Hadoop 3.0.0-cdh6.2.1
Source code repository http://github.com/cloudera/hadoop -r 7ce7dcdb76fd7f700cd7749bbc6390f76ff2abae
Compiled by jenkins on 2019-09-11T08:04Z
Compiled with protoc 2.5.0
From source with checksum b6cc25ff15ff575f159489abbee34c5
This command was run using /opt/cloudera/parcels/CDH-6.2.1-1.cdh6.2.1.p0.1425774/jars/hadoop-common-3.0.0-cdh6.2.1.jar
检查hive的版本
root@hadoop01 \~# hive --version
WARNING: Use "yarn jar" to launch YARN applications.
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in jar:file:/opt/cloudera/parcels/CDH-6.2.1-1.cdh6.2.1.p0.1425774/jars/log4j-slf4j-impl-2.8.2.jar!/org/slf4j/impl/StaticLoggerBinder.class
SLF4J: Found binding in jar:file:/opt/cloudera/parcels/CDH-6.2.1-1.cdh6.2.1.p0.1425774/jars/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type org.apache.logging.slf4j.Log4jLoggerFactory
Hive 2.1.1-cdh6.2.1
Subversion file:///container.redhat7/build/cdh/hive/2.1.1-cdh6.2.1/rpm/BUILD/hive-2.1.1-cdh6.2.1 -r fc0582dc4d2e2342d1eef8cb702bad68e1e2cdd2
Compiled by jenkins on Wed Sep 11 01:31:00 PDT 2019
From source with checksum 6bf22b68335469e1b010bcfecc8ea570
检查 hive服务是否启动了
root@hadoop01 \~# hive
WARNING: Use "yarn jar" to launch YARN applications.
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in jar:file:/opt/cloudera/parcels/CDH-6.2.1-1.cdh6.2.1.p0.1425774/jars/log4j-slf4j-impl-2.8.2.jar!/org/slf4j/impl/StaticLoggerBinder.class
SLF4J: Found binding in jar:file:/opt/cloudera/parcels/CDH-6.2.1-1.cdh6.2.1.p0.1425774/jars/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type org.apache.logging.slf4j.Log4jLoggerFactory
Logging initialized using configuration in jar:file:/opt/cloudera/parcels/CDH-6.2.1-1.cdh6.2.1.p0.1425774/jars/hive-common-2.1.1-cdh6.2.1.jar!/hive-log4j2.properties Async: false
WARNING: Hive CLI is deprecated and migration to Beeline is recommended.
hive>
WARNING: Hive CLI is deprecated and migration to Beeline is recommended.
hive>
>
>
> show databases;
OK
default
test
Time taken: 4.039 seconds, Fetched: 2 row(s)
hive>
>
检查 sqoop是否安装
root@hadoop01 \~# sqoop help
Warning: /opt/cloudera/parcels/CDH-6.2.1-1.cdh6.2.1.p0.1425774/bin/../lib/sqoop/../accumulo does not exist! Accumulo imports will fail.
Please set $ACCUMULO_HOME to the root of your Accumulo installation.
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in jar:file:/opt/cloudera/parcels/CDH-6.2.1-1.cdh6.2.1.p0.1425774/jars/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class
SLF4J: Found binding in jar:file:/opt/cloudera/parcels/CDH-6.2.1-1.cdh6.2.1.p0.1425774/jars/log4j-slf4j-impl-2.8.2.jar!/org/slf4j/impl/StaticLoggerBinder.class
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type org.slf4j.impl.Log4jLoggerFactory
23/09/20 13:48:09 INFO sqoop.Sqoop: Running Sqoop version: 1.4.7-cdh6.2.1
usage: sqoop COMMAND ARGS
Available commands:
codegen Generate code to interact with database records
create-hive-table Import a table definition into Hive
eval Evaluate a SQL statement and display the results
export Export an HDFS directory to a database table
help List available commands
import Import a table from a database to HDFS
import-all-tables Import tables from a database to HDFS
import-mainframe Import datasets from a mainframe server to HDFS
job Work with saved jobs
list-databases List available databases on a server
list-tables List available tables in a database
merge Merge results of incremental imports
metastore Run a standalone Sqoop metastore
version Display version information
See 'sqoop help COMMAND' for information on a specific command.
检查mysql数据库
root@hadoop01 \~# mysql -uroot -p123456
mysql: Warning Using a password on the command line interface can be insecure.
Welcome to the MySQL monitor. Commands end with ; or \g.
Your MySQL connection id is 70
Server version: 5.7.26 MySQL Community Server (GPL)
Copyright (c) 2000, 2019, Oracle and/or its affiliates. All rights reserved.
Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.
Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
mysql>
mysql> show databases;
+--------------------+
| Database |
+--------------------+
| information_schema |
| hive |
| hue |
| mysql |
| nev |
| oozie |
| performance_schema |
| scm |
| sys |
| test |
+--------------------+
10 rows in set (0.00 sec)
sqoop list-databases --connect jdbc:mysql://192.168.52.150:3306/ --username root --password 123456
测试sqoop数据库的连接
root@hadoop01 \~# sqoop list-databases --connect jdbc:mysql://192.168.52.150:3306/ --username root --password 123456
Warning: /opt/cloudera/parcels/CDH-6.2.1-1.cdh6.2.1.p0.1425774/bin/../lib/sqoop/../accumulo does not exist! Accumulo imports will fail.
Please set $ACCUMULO_HOME to the root of your Accumulo installation.
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in jar:file:/opt/cloudera/parcels/CDH-6.2.1-1.cdh6.2.1.p0.1425774/jars/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class
SLF4J: Found binding in jar:file:/opt/cloudera/parcels/CDH-6.2.1-1.cdh6.2.1.p0.1425774/jars/log4j-slf4j-impl-2.8.2.jar!/org/slf4j/impl/StaticLoggerBinder.class
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type org.slf4j.impl.Log4jLoggerFactory
23/09/20 13:54:17 INFO sqoop.Sqoop: Running Sqoop version: 1.4.7-cdh6.2.1
23/09/20 13:54:17 WARN tool.BaseSqoopTool: Setting your password on the command-line is insecure. Consider using -P instead.
23/09/20 13:54:17 INFO manager.MySQLManager: Preparing to use a MySQL streaming resultset.
Wed Sep 20 13:54:18 CST 2023 WARN: Establishing SSL connection without server's identity verification is not recommended. According to MySQL 5.5.45+, 5.6.26+ and 5.7.6+ requirements SSL connection must be established by default if explicit option isn't set. For compliance with existing applications not using SSL the verifyServerCertificate property is set to 'false'. You need either to explicitly disable SSL by setting useSSL=false, or set useSSL=true and provide truststore for server certificate verification.
information_schema
hive
hue
mysql
nev
oozie
performance_schema
scm
sys
test
root@hadoop01 \~#
练习环境部分信息
页面部分的补充
admin/admin
hue/hue
