八、hdfs文件系统副本块数量的配置

1、配置方式

2、实际操作演示

(1)在Hadoop用户的根目录下创建text.txt文件

(2)上传文件

bash 复制代码
hadoop@node1:~$ hdfs dfs -ls hdfs://node1:8020/
Found 4 items
drwxr-xr-x   - hadoop supergroup          0 2023-11-21 23:06 hdfs://node1:8020/home
-rw-r--r--   2 hadoop supergroup         35 2023-11-26 22:54 hdfs://node1:8020/text.txt
-rw-r--r--   3 hadoop supergroup         27 2023-11-26 22:57 hdfs://node1:8020/three_text.txt
drwx------   - hadoop supergroup          0 2023-11-18 12:21 hdfs://node1:8020/user

(3)修改副本数

bash 复制代码
hadoop@node1:~$ hdfs dfs -setrep 1 hdfs://node1:8020/three_text.txt
Replication 1 set: hdfs://node1:8020/three_text.txt
hadoop@node1:~$ hdfs dfs -ls hdfs://node1:8020/
Found 4 items
drwxr-xr-x   - hadoop supergroup          0 2023-11-21 23:06 hdfs://node1:8020/home
-rw-r--r--   2 hadoop supergroup         35 2023-11-26 22:54 hdfs://node1:8020/text.txt
-rw-r--r--   1 hadoop supergroup         27 2023-11-26 22:57 hdfs://node1:8020/three_text.txt
drwx------   - hadoop supergroup          0 2023-11-18 12:21 hdfs://node1:8020/user

3、fsck命令检查文件的副本数

(1)结果演示:

bash 复制代码
hadoop@node1:~$ hdfs fsck hdfs://node1:8020/text.txt -files -blocks -locations
Connecting to namenode via http://node1:9870/fsck?ugi=hadoop&files=1&blocks=1&locations=1&path=%2Ftext.txt
FSCK started by hadoop (auth:SIMPLE) from /192.168.88.101 for path /text.txt at Sun Nov 26 23:07:44 CST 2023

/text.txt 35 bytes, replicated: replication=2, 1 block(s):  OK
0. BP-1383297113-192.168.88.101-1699274849934:blk_1073741835_1012 len=35 Live_repl=2  [DatanodeInfoWithStorage[192.168.88.103:9866,DS-89bf9985-fca9-4717-bdd1-1476f0b98a0f,DISK], DatanodeInfoWithStorage[192.168.88.101:9866,DS-bc41ebe7-7da0-4097-813b-7a025fba1b99,DISK]]


Status: HEALTHY
 Number of data-nodes:  3
 Number of racks:               1
 Total dirs:                    0
 Total symlinks:                0

Replicated Blocks:
 Total size:    35 B
 Total files:   1
 Total blocks (validated):      1 (avg. block size 35 B)
 Minimally replicated blocks:   1 (100.0 %)
 Over-replicated blocks:        0 (0.0 %)
 Under-replicated blocks:       0 (0.0 %)
 Mis-replicated blocks:         0 (0.0 %)
 Default replication factor:    3
 Average block replication:     2.0
 Missing blocks:                0
 Corrupt blocks:                0
 Missing replicas:              0 (0.0 %)
 Blocks queued for replication: 0

Erasure Coded Block Groups:
 Total size:    0 B
 Total files:   0
 Total block groups (validated):        0
 Minimally erasure-coded block groups:  0
 Over-erasure-coded block groups:       0
 Under-erasure-coded block groups:      0
 Unsatisfactory placement block groups: 0
 Average block group size:      0.0
 Missing block groups:          0
 Corrupt block groups:          0
 Missing internal blocks:       0
 Blocks queued for replication: 0
FSCK ended at Sun Nov 26 23:07:44 CST 2023 in 1 milliseconds


The filesystem under path '/text.txt' is HEALTHY

(2)分析

相关推荐
大数据追光猿5 小时前
Python应用算法之贪心算法理解和实践
大数据·开发语言·人工智能·python·深度学习·算法·贪心算法
人类群星闪耀时6 小时前
物联网与大数据:揭秘万物互联的新纪元
大数据·物联网·struts
桃林春风一杯酒12 小时前
HADOOP_HOME and hadoop.home.dir are unset.
大数据·hadoop·分布式
桃木山人13 小时前
BigData File Viewer报错
大数据·java-ee·github·bigdata
B站计算机毕业设计超人13 小时前
计算机毕业设计Python+DeepSeek-R1高考推荐系统 高考分数线预测 大数据毕设(源码+LW文档+PPT+讲解)
大数据·python·机器学习·网络爬虫·课程设计·数据可视化·推荐算法
数造科技13 小时前
紧随“可信数据空间”政策风潮,数造科技正式加入开放数据空间联盟
大数据·人工智能·科技·安全·敏捷开发
逸Y 仙X16 小时前
Git常见命令--助力开发
java·大数据·git·java-ee·github·idea
caihuayuan417 小时前
PHP建立MySQL持久化连接(长连接)及mysql与mysqli扩展的区别
java·大数据·sql·spring
B站计算机毕业设计超人17 小时前
计算机毕业设计Hadoop+Spark+DeepSeek-R1大模型民宿推荐系统 hive民宿可视化 民宿爬虫 大数据毕业设计(源码+LW文档+PPT+讲解)
大数据·hadoop·爬虫·机器学习·课程设计·数据可视化·推荐算法
(; ̄ェ ̄)。18 小时前
在nodejs中使用ElasticSearch(二)核心概念,应用
大数据·elasticsearch·搜索引擎