Apache Knox 2.0.0使用

向他一样rap2024-05-12 19:37

目录

介绍

使用

gateway-site.xml

users.ldif

my_hdfs.xml

my_yarn.xml

其它

介绍

The Apache Knox Gateway is a system that provides a single point of authentication and access for Apache Hadoop services in a cluster. The goal is to simplify Hadoop security for both users (i.e. who access the cluster data and execute jobs) and operators (i.e. who control access and manage the cluster). The gateway runs as a server (or cluster of servers) that provide centralized access to one or more Hadoop clusters. In general the goals of the gateway are as follows:

  • Provide perimeter security for Hadoop REST APIs to make Hadoop security easier to setup and use
    • Provide authentication and token verification at the perimeter
    • Enable authentication integration with enterprise and cloud identity management systems
    • Provide service level authorization at the perimeter
  • Expose a single URL hierarchy that aggregates REST APIs of a Hadoop cluster
    • Limit the network endpoints (and therefore firewall holes) required to access a Hadoop cluster
    • Hide the internal Hadoop cluster topology from potential attackers

使用

解压后，目录如下：

进入conf目录:

文件说明：

gateway-site.xml

网关文件，修改如下:

XML 复制代码 
    <property>
        <name>gateway.port</name>
        <value>18483</value>
        <description>The HTTP port for the Gateway.</description>
    </property>
    <property>
        <name>gateway.path</name>
        <value>my/mimi</value>
        <description>The default context path for the gateway.</description>
    </property>
    <property>
        <name>gateway.dispatch.whitelist</name>
        <value>.*$</value>
    </property>

users.ldif

用户文件，全部如下

XML 复制代码 
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements.  See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership.  The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License.  You may obtain a copy of the License at
#
#     http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

version: 1

# Please replace with site specific values
dn: dc=hadoop,dc=apache,dc=org
objectclass: organization
objectclass: dcObject
o: Hadoop
dc: hadoop

# Entry for a sample people container
# Please replace with site specific values
dn: ou=people,dc=hadoop,dc=apache,dc=org
objectclass:top
objectclass:organizationalUnit
ou: people

dn: ou=myhadoop,dc=hadoop,dc=apache,dc=org
objectclass:top
objectclass:organizationalUnit
ou: myhadoop

# Entry for a sample end user
# Please replace with site specific values
dn: uid=guest,ou=people,dc=hadoop,dc=apache,dc=org
objectclass:top
objectclass:person
objectclass:organizationalPerson
objectclass:inetOrgPerson
cn: Guest
sn: User
uid: guest
userPassword:123456

dn: uid=myclient,ou=people,dc=hadoop,dc=apache,dc=org
objectclass:top
objectclass:person
objectclass:organizationalPerson
objectclass:inetOrgPerson
cn: Myclient
sn: Client
uid: myclient
userPassword:qwe

dn: uid=myhdfs,ou=myhadoop,dc=hadoop,dc=apache,dc=org
objectclass:top
objectclass:person
objectclass:organizationalPerson
objectclass:inetOrgPerson
cn: myhdfs
sn: myhdfs
uid: myhdfs
userPassword:123456

dn: uid=myyarn,ou=myhadoop,dc=hadoop,dc=apache,dc=org
objectclass:top
objectclass:person
objectclass:organizationalPerson
objectclass:inetOrgPerson
cn: myyarn
sn: myyarn
uid: myyarn
userPassword:123


# entry for sample user admin
dn: uid=admin,ou=people,dc=hadoop,dc=apache,dc=org
objectclass:top
objectclass:person
objectclass:organizationalPerson
objectclass:inetOrgPerson
cn: Admin
sn: Admin
uid: admin
userPassword:123456

dn: uid=root,ou=myhadoop,dc=hadoop,dc=apache,dc=org
objectclass:top
objectclass:person
objectclass:organizationalPerson
objectclass:inetOrgPerson
cn: Admin
sn: Admin
uid: root
userPassword:123456

# entry for sample user sam
dn: uid=sam,ou=people,dc=hadoop,dc=apache,dc=org
objectclass:top
objectclass:person
objectclass:organizationalPerson
objectclass:inetOrgPerson
cn: sam
sn: sam
uid: sam
userPassword:sam-password

# entry for sample user tom
dn: uid=tom,ou=people,dc=hadoop,dc=apache,dc=org
objectclass:top
objectclass:person
objectclass:organizationalPerson
objectclass:inetOrgPerson
cn: tom
sn: tom
uid: tom
userPassword:tom-password

# create FIRST Level groups branch
dn: ou=groups,dc=hadoop,dc=apache,dc=org
objectclass:top
objectclass:organizationalUnit
ou: groups
description: generic groups branch

# create the analyst group under groups
dn: cn=analyst,ou=groups,dc=hadoop,dc=apache,dc=org
objectclass:top
objectclass: groupofnames
cn: analyst
description:analyst  group
member: uid=sam,ou=people,dc=hadoop,dc=apache,dc=org
member: uid=tom,ou=people,dc=hadoop,dc=apache,dc=org
member: uid=myhdfs,ou=myhadoop,dc=hadoop,dc=apache,dc=org
member: uid=myyarn,ou=myhadoop,dc=hadoop,dc=apache,dc=org

# create the scientist group under groups
dn: cn=scientist,ou=groups,dc=hadoop,dc=apache,dc=org
objectclass:top
objectclass: groupofnames
cn: scientist
description: scientist group
member: uid=sam,ou=people,dc=hadoop,dc=apache,dc=org

# create the admin group under groups
dn: cn=admin,ou=groups,dc=hadoop,dc=apache,dc=org
objectclass:top
objectclass: groupofnames
cn: admin
description: admin group
member: uid=admin,ou=people,dc=hadoop,dc=apache,dc=org
member: uid=root,ou=myhadoop,dc=hadoop,dc=apache,dc=org

注意，我在这里面添加了两个用户 myhdfs，myyarn

进入knox-2.0.0/conf/topologies 文件，会发现一个sandbox.xml文件

它是一个模板文件，使用时改一个名字，我的如下

my_hdfs.xml

XML 复制代码 
<?xml version="1.0" encoding="UTF-8"?>
<topology>
   <name>my_hdfs</name>
   <gateway>
      <provider>
         <role>authentication</role>
         <name>ShiroProvider</name>
         <enabled>true</enabled>
         <param>
            <name>sessionTimeout</name>
            <value>30</value>
         </param>
         <param>
            <name>main.ldapRealm</name>
            <value>org.apache.knox.gateway.shirorealm.KnoxLdapRealm</value>
         </param>
         <param>
            <name>main.ldapContextFactory</name>
            <value>org.apache.knox.gateway.shirorealm.KnoxLdapContextFactory</value>
         </param>
         <param>
            <name>main.ldapRealm.contextFactory</name>
            <value>$ldapContextFactory</value>
         </param>


		<param>
			<name>main.ldapRealm.contextFactory.systemUsername</name>
			<value>uid=myclient,ou=people,dc=hadoop,dc=apache,dc=org</value>
		</param>
		<param>
			<name>main.ldapRealm.contextFactory.systemPassword</name>
			<value>${ALIAS=ldcSystemPassword}</value>
		</param>
		<param>
			<name>main.ldapRealm.userSearchBase</name>
			<value>ou=myhadoop,dc=hadoop,dc=apache,dc=org</value>
		</param>
		<param>
			<name>main.ldapRealm.userSearchAttributeName</name>
			<value>uid</value>
		</param>
		<param>
			<name>main.ldapRealm.userSearchFilter</name>
			<value>(&amp;(objectclass=person)(uid={0})(uid=myhdfs))</value>
		</param>

         <param>
            <name>main.ldapRealm.contextFactory.url</name>
            <value>ldap://localhost:33389</value>
         </param>
         <param>
            <name>main.ldapRealm.contextFactory.authenticationMechanism</name>
            <value>simple</value>
         </param>
         <param>
            <name>urls./**</name>
            <value>authcBasic</value>
         </param>
      </provider>
      <provider>
         <role>identity-assertion</role>
         <name>Default</name>
         <enabled>true</enabled>
      </provider>
      <provider>
         <role>hostmap</role>
         <name>static</name>
         <enabled>true</enabled>
         <param>
            <name>localhost</name>
            <value>my_hdfs</value>
         </param>
      </provider>
   </gateway>
<!--
   <service>
      <role>NAMENODE</role>
      <url>hdfs://hadoop02:9000</url>
   </service>
   <service>
      <role>WEBHDFS</role>
      <url>http://hadoop02:50070/webhdfs</url>
   </service>
-->   
   <service>
      <role>HDFSUI</role>
      <url>http://hadoop02:50070</url>
      <version>2.7.0</version>
   </service>

</topology>

my_yarn.xml

XML 复制代码 
<?xml version="1.0" encoding="UTF-8"?>
<topology>
   <name>my_yarn</name>
   <gateway>
      <provider>
         <role>authentication</role>
         <name>ShiroProvider</name>
         <enabled>true</enabled>
         <param>
            <name>sessionTimeout</name>
            <value>30</value>
         </param>
         <param>
            <name>main.ldapRealm</name>
            <value>org.apache.knox.gateway.shirorealm.KnoxLdapRealm</value>
         </param>
         <param>
            <name>main.ldapContextFactory</name>
            <value>org.apache.knox.gateway.shirorealm.KnoxLdapContextFactory</value>
         </param>
         <param>
            <name>main.ldapRealm.contextFactory</name>
            <value>$ldapContextFactory</value>
         </param>
         
		 <!-- 登录测试 -->
        <param>
			<name>main.ldapRealm.contextFactory.systemUsername</name>
			<value>uid=myclient,ou=people,dc=hadoop,dc=apache,dc=org</value>
		</param>
		<param>
			<name>main.ldapRealm.contextFactory.systemPassword</name>
			<value>${ALIAS=ldcSystemPassword}</value>
		</param>
		<param>
			<name>main.ldapRealm.userSearchBase</name>
			<value>ou=myhadoop,dc=hadoop,dc=apache,dc=org</value>
		</param>
		<param>
			<name>main.ldapRealm.userSearchAttributeName</name>
			<value>uid</value>
		</param>
		<param>
			<name>main.ldapRealm.userSearchFilter</name>
			<value>(&amp;(objectclass=person)(uid={0})(uid=myyarn))</value>
		</param>
		 <!-- 安全测试-->
		<param><name>csrf.enabled</name><value>true</value></param>
		<param><name>csrf.customHeader</name><value>X-XSRF-Header</value></param>
		<param><name>csrf.methodsToIgnore</name><value>GET,OPTIONS,HEAD</value></param>
		<param><name>cors.enabled</name><value>false</value></param>
		<param><name>xframe.options.enabled</name><value>true</value></param>
		<param><name>xss.protection.enabled</name><value>true</value></param>
		<param><name>strict.transport.enabled</name><value>true</value></param>
		<param><name>rate.limiting.enabled</name><value>true</value></param>

         <param>
            <name>main.ldapRealm.contextFactory.url</name>
            <value>ldap://localhost:33389</value>
         </param>
         <param>
            <name>main.ldapRealm.contextFactory.authenticationMechanism</name>
            <value>simple</value>
         </param>
         <param>
            <name>urls./**</name>
            <value>authcBasic</value>
         </param>
      </provider>
      <provider>
         <role>identity-assertion</role>
         <name>Default</name>
         <enabled>true</enabled>
      </provider>
      <provider>
         <role>hostmap</role>
         <name>static</name>
         <enabled>true</enabled>
         <param>
            <name>localhost</name>
            <value>my_yarn</value>
         </param>
      </provider>
   </gateway>
   <service>
      <role>YARNUI</role>
      <url>http://hadoop03:8088</url>
   </service>
   <service>
    <role>JOBHISTORYUI</role>
    <url>http://hadoop03:19888</url>
  </service>

</topology>

说明，配置中的${ALIAS=ldcSystemPassword}是生成的密文

参考：

如果测试不成功改名实际密码测试

之后进入knox-2.0.0/bin

执行(注意，不能使用root用户)

XML 复制代码 
./ldap.sh start
./knoxcli.sh create-master
./gateway.sh start

网站访问：

https://192.168.200.11:18483/my/mimi/my_yarn/yarn

账号：myyarn 密码 123

https://192.168.200.11:18483/my/mimi/my_hdfs/hdfs

账号：myhdfs 密码 123456

其它

通过我的多次测试，成功实现了，不同用户访问不同页面。下面是几个问题

1 必须有多个单独的xml才能实现不同用户访问不同页面

2 我测试成功的只有hdfs、yarn、hbase。之后准备测试hue，发现死活不成功

注意：文中xml单用户访问页面可以直接复制，然后修改。官网描述很多测试失败。

相关推荐
闲云野鹤在人间2 天前
项目实战：LNMP-电商平台-ECshop
linux·运维·nginx·apache·lvs
隔窗听雨眠2 天前
湖仓一体架构深度解读：Apache Doris如何打破数据边界实现查询提速30倍
架构·apache
南城以南溫暖如初1472 天前
社区健身小程序开发流程详解：技术选型与实施经验分享
java·经验分享·spring boot·mysql·vue·apache
北极糊的狐2 天前
钉钉小程序接入若依（只做增删改查，先不管工作台免登）
apache
SelectDB技术团队3 天前
阶跃星辰 Agent 可观测：Apache Doris / SelectDB 的技术能力与实践
django·apache·知识图谱
万岳科技系统开发3 天前
医疗诊所小程序如何实现患者管理和会员运营？
java·小程序·apache
行业研究员8 天前
腾讯位置服务核心功能与应用场景解析
apache·lbs·腾讯位置服务
SeaTunnel9 天前
Apache SeaTunnel 提交一个任务都经过了什么？
java·大数据·服务器·apache·etl·seatunnel
SelectDB12 天前
基于 Apache Doris 搭建 RAG 系统：从基础 RAG 到知识图谱增强的完整实现与选型指南
apache
SelectDB12 天前
SelectDB Enterprise 4.0.5：企业级实时分析与 AI 数据底座怎么选？安全合规配置指南
apache
热门推荐
01GitHub 镜像站点02国内可直接用、免费额度/永久免费的大模型API清单（含 SiliconFlow、火山、阿里、智谱、百度、Kimi、DeepSeek、DMXAPI 等）03微信历史版本含下载地址（ Windows PC | 安卓 | MAC ）及设置微信不更新042026 国产 AI 大模型横评：DeepSeek、通义千问、Kimi、文心一言、星火、豆包谁更能打？05Agnes AI 免费 API 接入指南：文本、生图、生视频，一套接口全免费06VSCode+CodeX扩展，无法启动07历年考研数学一、数学二、数学三真题试卷及答案PDF08AI 编程 IDE 全景解析 2026：Agent 全面接管开发链路09如何新建文件夹？ 电脑新建文件夹的4种方法102026年7月AI圈大地震：GPT-5.6被政府限制、Claude入驻Slack、Anthropic自研芯片