Apache Knox 2.0.0使用

向他一样rap2024-05-12 19:37

目录

介绍

使用

gateway-site.xml

users.ldif

my_hdfs.xml

my_yarn.xml

其它

介绍

The Apache Knox Gateway is a system that provides a single point of authentication and access for Apache Hadoop services in a cluster. The goal is to simplify Hadoop security for both users (i.e. who access the cluster data and execute jobs) and operators (i.e. who control access and manage the cluster). The gateway runs as a server (or cluster of servers) that provide centralized access to one or more Hadoop clusters. In general the goals of the gateway are as follows:

  • Provide perimeter security for Hadoop REST APIs to make Hadoop security easier to setup and use
    • Provide authentication and token verification at the perimeter
    • Enable authentication integration with enterprise and cloud identity management systems
    • Provide service level authorization at the perimeter
  • Expose a single URL hierarchy that aggregates REST APIs of a Hadoop cluster
    • Limit the network endpoints (and therefore firewall holes) required to access a Hadoop cluster
    • Hide the internal Hadoop cluster topology from potential attackers

使用

解压后，目录如下：

进入conf目录:

文件说明：

gateway-site.xml

网关文件，修改如下:

XML 复制代码 
    <property>
        <name>gateway.port</name>
        <value>18483</value>
        <description>The HTTP port for the Gateway.</description>
    </property>
    <property>
        <name>gateway.path</name>
        <value>my/mimi</value>
        <description>The default context path for the gateway.</description>
    </property>
    <property>
        <name>gateway.dispatch.whitelist</name>
        <value>.*$</value>
    </property>

users.ldif

用户文件，全部如下

XML 复制代码 
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements.  See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership.  The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License.  You may obtain a copy of the License at
#
#     http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

version: 1

# Please replace with site specific values
dn: dc=hadoop,dc=apache,dc=org
objectclass: organization
objectclass: dcObject
o: Hadoop
dc: hadoop

# Entry for a sample people container
# Please replace with site specific values
dn: ou=people,dc=hadoop,dc=apache,dc=org
objectclass:top
objectclass:organizationalUnit
ou: people

dn: ou=myhadoop,dc=hadoop,dc=apache,dc=org
objectclass:top
objectclass:organizationalUnit
ou: myhadoop

# Entry for a sample end user
# Please replace with site specific values
dn: uid=guest,ou=people,dc=hadoop,dc=apache,dc=org
objectclass:top
objectclass:person
objectclass:organizationalPerson
objectclass:inetOrgPerson
cn: Guest
sn: User
uid: guest
userPassword:123456

dn: uid=myclient,ou=people,dc=hadoop,dc=apache,dc=org
objectclass:top
objectclass:person
objectclass:organizationalPerson
objectclass:inetOrgPerson
cn: Myclient
sn: Client
uid: myclient
userPassword:qwe

dn: uid=myhdfs,ou=myhadoop,dc=hadoop,dc=apache,dc=org
objectclass:top
objectclass:person
objectclass:organizationalPerson
objectclass:inetOrgPerson
cn: myhdfs
sn: myhdfs
uid: myhdfs
userPassword:123456

dn: uid=myyarn,ou=myhadoop,dc=hadoop,dc=apache,dc=org
objectclass:top
objectclass:person
objectclass:organizationalPerson
objectclass:inetOrgPerson
cn: myyarn
sn: myyarn
uid: myyarn
userPassword:123


# entry for sample user admin
dn: uid=admin,ou=people,dc=hadoop,dc=apache,dc=org
objectclass:top
objectclass:person
objectclass:organizationalPerson
objectclass:inetOrgPerson
cn: Admin
sn: Admin
uid: admin
userPassword:123456

dn: uid=root,ou=myhadoop,dc=hadoop,dc=apache,dc=org
objectclass:top
objectclass:person
objectclass:organizationalPerson
objectclass:inetOrgPerson
cn: Admin
sn: Admin
uid: root
userPassword:123456

# entry for sample user sam
dn: uid=sam,ou=people,dc=hadoop,dc=apache,dc=org
objectclass:top
objectclass:person
objectclass:organizationalPerson
objectclass:inetOrgPerson
cn: sam
sn: sam
uid: sam
userPassword:sam-password

# entry for sample user tom
dn: uid=tom,ou=people,dc=hadoop,dc=apache,dc=org
objectclass:top
objectclass:person
objectclass:organizationalPerson
objectclass:inetOrgPerson
cn: tom
sn: tom
uid: tom
userPassword:tom-password

# create FIRST Level groups branch
dn: ou=groups,dc=hadoop,dc=apache,dc=org
objectclass:top
objectclass:organizationalUnit
ou: groups
description: generic groups branch

# create the analyst group under groups
dn: cn=analyst,ou=groups,dc=hadoop,dc=apache,dc=org
objectclass:top
objectclass: groupofnames
cn: analyst
description:analyst  group
member: uid=sam,ou=people,dc=hadoop,dc=apache,dc=org
member: uid=tom,ou=people,dc=hadoop,dc=apache,dc=org
member: uid=myhdfs,ou=myhadoop,dc=hadoop,dc=apache,dc=org
member: uid=myyarn,ou=myhadoop,dc=hadoop,dc=apache,dc=org

# create the scientist group under groups
dn: cn=scientist,ou=groups,dc=hadoop,dc=apache,dc=org
objectclass:top
objectclass: groupofnames
cn: scientist
description: scientist group
member: uid=sam,ou=people,dc=hadoop,dc=apache,dc=org

# create the admin group under groups
dn: cn=admin,ou=groups,dc=hadoop,dc=apache,dc=org
objectclass:top
objectclass: groupofnames
cn: admin
description: admin group
member: uid=admin,ou=people,dc=hadoop,dc=apache,dc=org
member: uid=root,ou=myhadoop,dc=hadoop,dc=apache,dc=org

注意，我在这里面添加了两个用户 myhdfs，myyarn

进入knox-2.0.0/conf/topologies 文件，会发现一个sandbox.xml文件

它是一个模板文件，使用时改一个名字，我的如下

my_hdfs.xml

XML 复制代码 
<?xml version="1.0" encoding="UTF-8"?>
<topology>
   <name>my_hdfs</name>
   <gateway>
      <provider>
         <role>authentication</role>
         <name>ShiroProvider</name>
         <enabled>true</enabled>
         <param>
            <name>sessionTimeout</name>
            <value>30</value>
         </param>
         <param>
            <name>main.ldapRealm</name>
            <value>org.apache.knox.gateway.shirorealm.KnoxLdapRealm</value>
         </param>
         <param>
            <name>main.ldapContextFactory</name>
            <value>org.apache.knox.gateway.shirorealm.KnoxLdapContextFactory</value>
         </param>
         <param>
            <name>main.ldapRealm.contextFactory</name>
            <value>$ldapContextFactory</value>
         </param>


		<param>
			<name>main.ldapRealm.contextFactory.systemUsername</name>
			<value>uid=myclient,ou=people,dc=hadoop,dc=apache,dc=org</value>
		</param>
		<param>
			<name>main.ldapRealm.contextFactory.systemPassword</name>
			<value>${ALIAS=ldcSystemPassword}</value>
		</param>
		<param>
			<name>main.ldapRealm.userSearchBase</name>
			<value>ou=myhadoop,dc=hadoop,dc=apache,dc=org</value>
		</param>
		<param>
			<name>main.ldapRealm.userSearchAttributeName</name>
			<value>uid</value>
		</param>
		<param>
			<name>main.ldapRealm.userSearchFilter</name>
			<value>(&amp;(objectclass=person)(uid={0})(uid=myhdfs))</value>
		</param>

         <param>
            <name>main.ldapRealm.contextFactory.url</name>
            <value>ldap://localhost:33389</value>
         </param>
         <param>
            <name>main.ldapRealm.contextFactory.authenticationMechanism</name>
            <value>simple</value>
         </param>
         <param>
            <name>urls./**</name>
            <value>authcBasic</value>
         </param>
      </provider>
      <provider>
         <role>identity-assertion</role>
         <name>Default</name>
         <enabled>true</enabled>
      </provider>
      <provider>
         <role>hostmap</role>
         <name>static</name>
         <enabled>true</enabled>
         <param>
            <name>localhost</name>
            <value>my_hdfs</value>
         </param>
      </provider>
   </gateway>
<!--
   <service>
      <role>NAMENODE</role>
      <url>hdfs://hadoop02:9000</url>
   </service>
   <service>
      <role>WEBHDFS</role>
      <url>http://hadoop02:50070/webhdfs</url>
   </service>
-->   
   <service>
      <role>HDFSUI</role>
      <url>http://hadoop02:50070</url>
      <version>2.7.0</version>
   </service>

</topology>

my_yarn.xml

XML 复制代码 
<?xml version="1.0" encoding="UTF-8"?>
<topology>
   <name>my_yarn</name>
   <gateway>
      <provider>
         <role>authentication</role>
         <name>ShiroProvider</name>
         <enabled>true</enabled>
         <param>
            <name>sessionTimeout</name>
            <value>30</value>
         </param>
         <param>
            <name>main.ldapRealm</name>
            <value>org.apache.knox.gateway.shirorealm.KnoxLdapRealm</value>
         </param>
         <param>
            <name>main.ldapContextFactory</name>
            <value>org.apache.knox.gateway.shirorealm.KnoxLdapContextFactory</value>
         </param>
         <param>
            <name>main.ldapRealm.contextFactory</name>
            <value>$ldapContextFactory</value>
         </param>
         
		 <!-- 登录测试 -->
        <param>
			<name>main.ldapRealm.contextFactory.systemUsername</name>
			<value>uid=myclient,ou=people,dc=hadoop,dc=apache,dc=org</value>
		</param>
		<param>
			<name>main.ldapRealm.contextFactory.systemPassword</name>
			<value>${ALIAS=ldcSystemPassword}</value>
		</param>
		<param>
			<name>main.ldapRealm.userSearchBase</name>
			<value>ou=myhadoop,dc=hadoop,dc=apache,dc=org</value>
		</param>
		<param>
			<name>main.ldapRealm.userSearchAttributeName</name>
			<value>uid</value>
		</param>
		<param>
			<name>main.ldapRealm.userSearchFilter</name>
			<value>(&amp;(objectclass=person)(uid={0})(uid=myyarn))</value>
		</param>
		 <!-- 安全测试-->
		<param><name>csrf.enabled</name><value>true</value></param>
		<param><name>csrf.customHeader</name><value>X-XSRF-Header</value></param>
		<param><name>csrf.methodsToIgnore</name><value>GET,OPTIONS,HEAD</value></param>
		<param><name>cors.enabled</name><value>false</value></param>
		<param><name>xframe.options.enabled</name><value>true</value></param>
		<param><name>xss.protection.enabled</name><value>true</value></param>
		<param><name>strict.transport.enabled</name><value>true</value></param>
		<param><name>rate.limiting.enabled</name><value>true</value></param>

         <param>
            <name>main.ldapRealm.contextFactory.url</name>
            <value>ldap://localhost:33389</value>
         </param>
         <param>
            <name>main.ldapRealm.contextFactory.authenticationMechanism</name>
            <value>simple</value>
         </param>
         <param>
            <name>urls./**</name>
            <value>authcBasic</value>
         </param>
      </provider>
      <provider>
         <role>identity-assertion</role>
         <name>Default</name>
         <enabled>true</enabled>
      </provider>
      <provider>
         <role>hostmap</role>
         <name>static</name>
         <enabled>true</enabled>
         <param>
            <name>localhost</name>
            <value>my_yarn</value>
         </param>
      </provider>
   </gateway>
   <service>
      <role>YARNUI</role>
      <url>http://hadoop03:8088</url>
   </service>
   <service>
    <role>JOBHISTORYUI</role>
    <url>http://hadoop03:19888</url>
  </service>

</topology>

说明，配置中的${ALIAS=ldcSystemPassword}是生成的密文

参考：

如果测试不成功改名实际密码测试

之后进入knox-2.0.0/bin

执行(注意，不能使用root用户)

XML 复制代码 
./ldap.sh start
./knoxcli.sh create-master
./gateway.sh start

网站访问：

https://192.168.200.11:18483/my/mimi/my_yarn/yarn

账号：myyarn 密码 123

https://192.168.200.11:18483/my/mimi/my_hdfs/hdfs

账号：myhdfs 密码 123456

其它

通过我的多次测试，成功实现了，不同用户访问不同页面。下面是几个问题

1 必须有多个单独的xml才能实现不同用户访问不同页面

2 我测试成功的只有hdfs、yarn、hbase。之后准备测试hue，发现死活不成功

注意：文中xml单用户访问页面可以直接复制，然后修改。官网描述很多测试失败。

相关推荐
zhojiew10 小时前
基于Pyspark对Apache Iceberg核心功能的使用实践
apache
SeaTunnel1 天前
AI 让 SeaTunnel 读源码和调试过时了吗？
大数据·数据库·人工智能·apache·seatunnel·数据同步
http阿拉丁神猫2 天前
ansible脚本解读
服务器·ansible·apache
小羊Yveesss2 天前
商家小程序外卖订单打印方案：云打印机对接、分单逻辑与模板配置实战
小程序·apache
@insist1232 天前
信息安全工程师-Apache/IIS安全增强与OWASP漏洞防护
安全·apache·软考·信息安全工程师·软件水平考试
Geek_Vison2 天前
如何借助小程序容器技术实现跨端APP的敏捷开发
小程序·apache·敏捷流程
Apache RocketMQ2 天前
Apache RocketMQ 5.0 架构解析：如何基于云原生架构支撑多元化场景
云原生·架构·apache·rocketmq·java-rocketmq
RingWu3 天前
微服务架构-全链路追踪：Apache SkyWalking
微服务·架构·apache
Donk_674 天前
高可用-Keepalived 解析
运维·服务器·apache
小羊Yveesss6 天前
门店小程序外卖配送搭建实战：配送对接、运费策略与多门店调度方案
小程序·apache
热门推荐
01GitHub 镜像站点02Codex 接入 DeepSeek API 完整配置文档03CC-Switch & Claude 基于 Linux 服务器安装使用指南04【踩坑记录 | 第一篇】微软商店无法使用时，如何手动安装 OpenAI Codex？附`.msix`文件系统错误解决方法05几个好用的ip纯净度检测网站06裂开！ChatGPT 居然开始要手机号验证，附详细解决方法07装上就回不去了：CodeGraph 让 AI 编程效率飙升 92%，它到底做了什么？08用了半年 OpenRouter，我换到了 Ofox.ai — 两个 AI API 聚合平台的真实对比09【AI】2026 年具身智能模型和世界模型总结10codex app每次打开重连5次Reconnecting问题解决