Apache Knox 2.0.0使用

向他一样rap2024-05-12 19:37

目录

介绍

使用

gateway-site.xml

users.ldif

my_hdfs.xml

my_yarn.xml

其它

介绍

The Apache Knox Gateway is a system that provides a single point of authentication and access for Apache Hadoop services in a cluster. The goal is to simplify Hadoop security for both users (i.e. who access the cluster data and execute jobs) and operators (i.e. who control access and manage the cluster). The gateway runs as a server (or cluster of servers) that provide centralized access to one or more Hadoop clusters. In general the goals of the gateway are as follows:

  • Provide perimeter security for Hadoop REST APIs to make Hadoop security easier to setup and use
    • Provide authentication and token verification at the perimeter
    • Enable authentication integration with enterprise and cloud identity management systems
    • Provide service level authorization at the perimeter
  • Expose a single URL hierarchy that aggregates REST APIs of a Hadoop cluster
    • Limit the network endpoints (and therefore firewall holes) required to access a Hadoop cluster
    • Hide the internal Hadoop cluster topology from potential attackers

使用

解压后，目录如下：

进入conf目录:

文件说明：

gateway-site.xml

网关文件，修改如下:

XML 复制代码 
    <property>
        <name>gateway.port</name>
        <value>18483</value>
        <description>The HTTP port for the Gateway.</description>
    </property>
    <property>
        <name>gateway.path</name>
        <value>my/mimi</value>
        <description>The default context path for the gateway.</description>
    </property>
    <property>
        <name>gateway.dispatch.whitelist</name>
        <value>.*$</value>
    </property>

users.ldif

用户文件，全部如下

XML 复制代码 
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements.  See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership.  The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License.  You may obtain a copy of the License at
#
#     http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

version: 1

# Please replace with site specific values
dn: dc=hadoop,dc=apache,dc=org
objectclass: organization
objectclass: dcObject
o: Hadoop
dc: hadoop

# Entry for a sample people container
# Please replace with site specific values
dn: ou=people,dc=hadoop,dc=apache,dc=org
objectclass:top
objectclass:organizationalUnit
ou: people

dn: ou=myhadoop,dc=hadoop,dc=apache,dc=org
objectclass:top
objectclass:organizationalUnit
ou: myhadoop

# Entry for a sample end user
# Please replace with site specific values
dn: uid=guest,ou=people,dc=hadoop,dc=apache,dc=org
objectclass:top
objectclass:person
objectclass:organizationalPerson
objectclass:inetOrgPerson
cn: Guest
sn: User
uid: guest
userPassword:123456

dn: uid=myclient,ou=people,dc=hadoop,dc=apache,dc=org
objectclass:top
objectclass:person
objectclass:organizationalPerson
objectclass:inetOrgPerson
cn: Myclient
sn: Client
uid: myclient
userPassword:qwe

dn: uid=myhdfs,ou=myhadoop,dc=hadoop,dc=apache,dc=org
objectclass:top
objectclass:person
objectclass:organizationalPerson
objectclass:inetOrgPerson
cn: myhdfs
sn: myhdfs
uid: myhdfs
userPassword:123456

dn: uid=myyarn,ou=myhadoop,dc=hadoop,dc=apache,dc=org
objectclass:top
objectclass:person
objectclass:organizationalPerson
objectclass:inetOrgPerson
cn: myyarn
sn: myyarn
uid: myyarn
userPassword:123


# entry for sample user admin
dn: uid=admin,ou=people,dc=hadoop,dc=apache,dc=org
objectclass:top
objectclass:person
objectclass:organizationalPerson
objectclass:inetOrgPerson
cn: Admin
sn: Admin
uid: admin
userPassword:123456

dn: uid=root,ou=myhadoop,dc=hadoop,dc=apache,dc=org
objectclass:top
objectclass:person
objectclass:organizationalPerson
objectclass:inetOrgPerson
cn: Admin
sn: Admin
uid: root
userPassword:123456

# entry for sample user sam
dn: uid=sam,ou=people,dc=hadoop,dc=apache,dc=org
objectclass:top
objectclass:person
objectclass:organizationalPerson
objectclass:inetOrgPerson
cn: sam
sn: sam
uid: sam
userPassword:sam-password

# entry for sample user tom
dn: uid=tom,ou=people,dc=hadoop,dc=apache,dc=org
objectclass:top
objectclass:person
objectclass:organizationalPerson
objectclass:inetOrgPerson
cn: tom
sn: tom
uid: tom
userPassword:tom-password

# create FIRST Level groups branch
dn: ou=groups,dc=hadoop,dc=apache,dc=org
objectclass:top
objectclass:organizationalUnit
ou: groups
description: generic groups branch

# create the analyst group under groups
dn: cn=analyst,ou=groups,dc=hadoop,dc=apache,dc=org
objectclass:top
objectclass: groupofnames
cn: analyst
description:analyst  group
member: uid=sam,ou=people,dc=hadoop,dc=apache,dc=org
member: uid=tom,ou=people,dc=hadoop,dc=apache,dc=org
member: uid=myhdfs,ou=myhadoop,dc=hadoop,dc=apache,dc=org
member: uid=myyarn,ou=myhadoop,dc=hadoop,dc=apache,dc=org

# create the scientist group under groups
dn: cn=scientist,ou=groups,dc=hadoop,dc=apache,dc=org
objectclass:top
objectclass: groupofnames
cn: scientist
description: scientist group
member: uid=sam,ou=people,dc=hadoop,dc=apache,dc=org

# create the admin group under groups
dn: cn=admin,ou=groups,dc=hadoop,dc=apache,dc=org
objectclass:top
objectclass: groupofnames
cn: admin
description: admin group
member: uid=admin,ou=people,dc=hadoop,dc=apache,dc=org
member: uid=root,ou=myhadoop,dc=hadoop,dc=apache,dc=org

注意，我在这里面添加了两个用户 myhdfs，myyarn

进入knox-2.0.0/conf/topologies 文件，会发现一个sandbox.xml文件

它是一个模板文件，使用时改一个名字，我的如下

my_hdfs.xml

XML 复制代码 
<?xml version="1.0" encoding="UTF-8"?>
<topology>
   <name>my_hdfs</name>
   <gateway>
      <provider>
         <role>authentication</role>
         <name>ShiroProvider</name>
         <enabled>true</enabled>
         <param>
            <name>sessionTimeout</name>
            <value>30</value>
         </param>
         <param>
            <name>main.ldapRealm</name>
            <value>org.apache.knox.gateway.shirorealm.KnoxLdapRealm</value>
         </param>
         <param>
            <name>main.ldapContextFactory</name>
            <value>org.apache.knox.gateway.shirorealm.KnoxLdapContextFactory</value>
         </param>
         <param>
            <name>main.ldapRealm.contextFactory</name>
            <value>$ldapContextFactory</value>
         </param>


		<param>
			<name>main.ldapRealm.contextFactory.systemUsername</name>
			<value>uid=myclient,ou=people,dc=hadoop,dc=apache,dc=org</value>
		</param>
		<param>
			<name>main.ldapRealm.contextFactory.systemPassword</name>
			<value>${ALIAS=ldcSystemPassword}</value>
		</param>
		<param>
			<name>main.ldapRealm.userSearchBase</name>
			<value>ou=myhadoop,dc=hadoop,dc=apache,dc=org</value>
		</param>
		<param>
			<name>main.ldapRealm.userSearchAttributeName</name>
			<value>uid</value>
		</param>
		<param>
			<name>main.ldapRealm.userSearchFilter</name>
			<value>(&amp;(objectclass=person)(uid={0})(uid=myhdfs))</value>
		</param>

         <param>
            <name>main.ldapRealm.contextFactory.url</name>
            <value>ldap://localhost:33389</value>
         </param>
         <param>
            <name>main.ldapRealm.contextFactory.authenticationMechanism</name>
            <value>simple</value>
         </param>
         <param>
            <name>urls./**</name>
            <value>authcBasic</value>
         </param>
      </provider>
      <provider>
         <role>identity-assertion</role>
         <name>Default</name>
         <enabled>true</enabled>
      </provider>
      <provider>
         <role>hostmap</role>
         <name>static</name>
         <enabled>true</enabled>
         <param>
            <name>localhost</name>
            <value>my_hdfs</value>
         </param>
      </provider>
   </gateway>
<!--
   <service>
      <role>NAMENODE</role>
      <url>hdfs://hadoop02:9000</url>
   </service>
   <service>
      <role>WEBHDFS</role>
      <url>http://hadoop02:50070/webhdfs</url>
   </service>
-->   
   <service>
      <role>HDFSUI</role>
      <url>http://hadoop02:50070</url>
      <version>2.7.0</version>
   </service>

</topology>

my_yarn.xml

XML 复制代码 
<?xml version="1.0" encoding="UTF-8"?>
<topology>
   <name>my_yarn</name>
   <gateway>
      <provider>
         <role>authentication</role>
         <name>ShiroProvider</name>
         <enabled>true</enabled>
         <param>
            <name>sessionTimeout</name>
            <value>30</value>
         </param>
         <param>
            <name>main.ldapRealm</name>
            <value>org.apache.knox.gateway.shirorealm.KnoxLdapRealm</value>
         </param>
         <param>
            <name>main.ldapContextFactory</name>
            <value>org.apache.knox.gateway.shirorealm.KnoxLdapContextFactory</value>
         </param>
         <param>
            <name>main.ldapRealm.contextFactory</name>
            <value>$ldapContextFactory</value>
         </param>
         
		 <!-- 登录测试 -->
        <param>
			<name>main.ldapRealm.contextFactory.systemUsername</name>
			<value>uid=myclient,ou=people,dc=hadoop,dc=apache,dc=org</value>
		</param>
		<param>
			<name>main.ldapRealm.contextFactory.systemPassword</name>
			<value>${ALIAS=ldcSystemPassword}</value>
		</param>
		<param>
			<name>main.ldapRealm.userSearchBase</name>
			<value>ou=myhadoop,dc=hadoop,dc=apache,dc=org</value>
		</param>
		<param>
			<name>main.ldapRealm.userSearchAttributeName</name>
			<value>uid</value>
		</param>
		<param>
			<name>main.ldapRealm.userSearchFilter</name>
			<value>(&amp;(objectclass=person)(uid={0})(uid=myyarn))</value>
		</param>
		 <!-- 安全测试-->
		<param><name>csrf.enabled</name><value>true</value></param>
		<param><name>csrf.customHeader</name><value>X-XSRF-Header</value></param>
		<param><name>csrf.methodsToIgnore</name><value>GET,OPTIONS,HEAD</value></param>
		<param><name>cors.enabled</name><value>false</value></param>
		<param><name>xframe.options.enabled</name><value>true</value></param>
		<param><name>xss.protection.enabled</name><value>true</value></param>
		<param><name>strict.transport.enabled</name><value>true</value></param>
		<param><name>rate.limiting.enabled</name><value>true</value></param>

         <param>
            <name>main.ldapRealm.contextFactory.url</name>
            <value>ldap://localhost:33389</value>
         </param>
         <param>
            <name>main.ldapRealm.contextFactory.authenticationMechanism</name>
            <value>simple</value>
         </param>
         <param>
            <name>urls./**</name>
            <value>authcBasic</value>
         </param>
      </provider>
      <provider>
         <role>identity-assertion</role>
         <name>Default</name>
         <enabled>true</enabled>
      </provider>
      <provider>
         <role>hostmap</role>
         <name>static</name>
         <enabled>true</enabled>
         <param>
            <name>localhost</name>
            <value>my_yarn</value>
         </param>
      </provider>
   </gateway>
   <service>
      <role>YARNUI</role>
      <url>http://hadoop03:8088</url>
   </service>
   <service>
    <role>JOBHISTORYUI</role>
    <url>http://hadoop03:19888</url>
  </service>

</topology>

说明，配置中的${ALIAS=ldcSystemPassword}是生成的密文

参考：

如果测试不成功改名实际密码测试

之后进入knox-2.0.0/bin

执行(注意，不能使用root用户)

XML 复制代码 
./ldap.sh start
./knoxcli.sh create-master
./gateway.sh start

网站访问：

https://192.168.200.11:18483/my/mimi/my_yarn/yarn

账号：myyarn 密码 123

https://192.168.200.11:18483/my/mimi/my_hdfs/hdfs

账号：myhdfs 密码 123456

其它

通过我的多次测试，成功实现了，不同用户访问不同页面。下面是几个问题

1 必须有多个单独的xml才能实现不同用户访问不同页面

2 我测试成功的只有hdfs、yarn、hbase。之后准备测试hue，发现死活不成功

注意：文中xml单用户访问页面可以直接复制，然后修改。官网描述很多测试失败。

相关推荐
微学AI5 小时前
2026年时序数据库选型指南：Apache IoTDB从大数据与物联网视角深度剖析
apache·时序数据库·iotdb
路baby1 天前
Pikachu安装过程中常见问题(apache和MySQL无法正常启动）
计算机网络·mysql·网络安全·adb·靶场·apache·pikachu
努力打怪升级2 天前
Apache HTTP Server 2.4 Windows 版完整配置与运维手册
运维·http·apache
航Hang*2 天前
第2章：进阶Linux系统——第10节：Linux 系统编程与 Shell 脚本全解笔记（GCC+Make+Vim+Shell Script）
linux·运维·服务器·学习·vim·apache·vmware
倔强的石头1063 天前
时序数据库选型指南：可视化与分析协同怎么选？——以 Apache IoTDB + Grafana + 大数据引擎为例
apache·时序数据库·iotdb
zz0723203 天前
Apache Kafka 开源的分布式事件流平台
分布式·kafka·apache
A-刘晨阳3 天前
流批一体架构下的时序数据库选型：Apache IoTDB实时计算能力深度解析与国际化对比
架构·apache·时序数据库
wei_shuo3 天前
工业物联网数据基础设施：Apache IoTDB 与 TimechoDB 的云原生与 AI 进化之路
物联网·apache·iotdb
橘子编程5 天前
Apache Hadoop知识全解析
大数据·hive·hadoop·apache
zzj_2626106 天前
实验三 循环结构程序设计（Python）
服务器·python·apache
热门推荐
01GitHub 镜像站点02一周AI热点速览（2026.03.31-04.06）：GPT-6曝光、谷歌开源Gemma 4、资本狂飙与模型军备竞赛03OpenClaw 请求超时 llm request timed out 怎么解决？3 种方案实测，附完整排查流程04AI 编程效率翻倍：Superpowers Skills 上手清单 + 完整指南05Oh My Codex 快速使用指南06CodeBuddy与WorkBuddy深度对比：腾讯两款AI工具差异及实操指南07VMware Workstation Pro 17 虚拟机完整安装教程（2026最新）08Qwen3.5-Omni与Qwen3.6模型全面解析（含测评/案例/使用教程）09UV安装并设置国内源10MySQL表约束详解：8大核心约束实战指南