19.Hive外部表权限用Sentry管理

19.1 环境介绍

操作系统版本：RedHat6.5
CM版本：CM 5.11.1
集群已启用Kerberos和Sentry
采用具有sudo权限的ec2-user用户进行操作

19.2 实验准备

19.2.1外部表数据父目录创建

使用hive用户登录Kerberos

[root@ip-172-31-8-141 1874-hive-HIVESERVER2]# kinit -kt hive.keytab hive/ip-172-31-8-141.ap-southeast-1.compute.internal@CLOUDERA.COM
[root@ip-172-31-8-141 1874-hive-HIVESERVER2]# klist
Ticket cache: FILE:/tmp/krb5cc_0
Default principal: hive/ip-172-31-8-141.ap-southeast-1.compute.internal@CLOUDERA.COM

Valid starting     Expires            Service principal
09/01/17 11:10:54  09/02/17 11:10:54  krbtgt/CLOUDERA.COM@CLOUDERA.COM
        renew until 09/06/17 11:10:54
[root@ip-172-31-8-141 1874-hive-HIVESERVER2]#

创建HDFS目录
- 使用如下命令在HDFS的根目录下创建Hive外部表的数据目录/extwarehouse

[root@ip-172-31-8-141 ec2-user]# hadoop fs -mkdir /extwarehouse
[root@ip-172-31-8-141 ec2-user]# hadoop fs -ls /
drwxr-xr-x   - hive   supergroup          0 2017-09-01 11:27 /extwarehouse
drwxrwxrwx   - user_r supergroup          0 2017-08-23 03:23 /fayson
drwx------   - hbase  hbase               0 2017-09-01 02:59 /hbase
drwxrwxrwt   - hdfs   supergroup          0 2017-08-31 06:18 /tmp
drwxrwxrwx   - hdfs   supergroup          0 2017-08-30 03:48 /user
[root@ip-172-31-8-141 ec2-user]# hadoop fs -chown hive:hive /extwarehouse
[root@ip-172-31-8-141 ec2-user]# hadoop fs -chmod 771 /extwarehouse
[root@ip-172-31-8-141 ec2-user]# hadoop fs -ls /
drwxrwx--x   - hive   hive                0 2017-09-01 11:27 /extwarehouse
drwxrwxrwx   - user_r supergroup          0 2017-08-23 03:23 /fayson
drwx------   - hbase  hbase               0 2017-09-01 02:59 /hbase
drwxrwxrwt   - hdfs   supergroup          0 2017-08-31 06:18 /tmp
drwxrwxrwx   - hdfs   supergroup          0 2017-08-30 03:48 /user
[root@ip-172-31-8-141 ec2-user]#

配置外部表数据父目录的ACL同步
- 确保HDFS已开启sentry并启用ACL同步
- 配置sentry同步路径，19.2.1创建的Hive外部表数据目录
- 配置完成，重启服务。

19.3 Hive外部表创建

使用beeline命令行连接hive，创建Hive外部表
- 建表语句：

create external table if not exists student(
        name string,
        age int,
        addr string
)
ROW FORMAT DELIMITED FIELDS TERMINATED BY ','
LOCATION '/extwarehouse/student';

终端操作：

[root@ip-172-31-8-141 1874-hive-HIVESERVER2]# beeline 
Beeline version 1.1.0-cdh5.11.1 by Apache Hive
beeline> !connect jdbc:hive2://localhost:10000/;principal=hive/ip-172-31-8-141.ap-southeast-1.compute.internal@CLOUDERA.COM
...
0: jdbc:hive2://localhost:10000/> create external table if not exists student(
. . . . . . . . . . . . . . . . >         name string,
. . . . . . . . . . . . . . . . >         age int,
. . . . . . . . . . . . . . . . >         addr string
. . . . . . . . . . . . . . . . > )
. . . . . . . . . . . . . . . . > ROW FORMAT DELIMITED FIELDS TERMINATED BY ','
. . . . . . . . . . . . . . . . > LOCATION '/extwarehouse/student';
...
INFO  : OK
No rows affected (0.236 seconds)
0: jdbc:hive2://localhost:10000/>

向student表中load数据
- 准备测试数据

[root@ip-172-31-8-141 student]# pwd
/home/ec2-user/student
[root@ip-172-31-8-141 student]# ll
total 4
-rw-r--r-- 1 root root 39 Sep  1 11:37 student.txt
[root@ip-172-31-8-141 student]# cat student.txt 
zhangsan,18,guangzhou
lisi,20,shenzhen
[root@ip-172-31-8-141 student]#

将student.txt文件put到hdfs的/tmp/student目录

[root@ip-172-31-8-141 student]# hadoop fs -mkdir /tmp/student
[root@ip-172-31-8-141 student]# ll
total 4
-rw-r--r-- 1 hive hive 39 Sep  1 11:37 student.txt
[root@ip-172-31-8-141 student]# hadoop fs -put student.txt /tmp/student
[root@ip-172-31-8-141 student]# hadoop fs -ls /tmp/student
Found 1 items
-rw-r--r--   3 hive supergroup         39 2017-09-01 11:57 /tmp/stu
dent/student.txt
[root@ip-172-31-8-141 student]#

在beeline命令行下，将数据load到student表

0: jdbc:hive2://localhost:10000/> load data inpath '/tmp/student' into table student;
...
INFO  : Table default.student stats: [numFiles=1, totalSize=39]
INFO  : Completed executing command(queryId=hive_20170901115858_5a76aa76-1b24-40ce-8254-42991856c05b); Time taken: 0.263 seconds
INFO  : OK
No rows affected (0.41 seconds)
0: jdbc:hive2://localhost:10000/>

执行完load命令后，查看表数据

0: jdbc:hive2://localhost:10000/> select * from student;
...
INFO  : OK
+---------------+--------------+---------------+--+
| student.name  | student.age  | student.addr  |
+---------------+--------------+---------------+--+
| zhangsan      | 18           | guangzhou     |
| lisi          | 20           | shenzhen      |
+---------------+--------------+---------------+--+
2 rows selected (0.288 seconds)
0: jdbc:hive2://localhost:10000/>

19.4 给fayson用户student表读权限

使用fayson用户的principal初始化Kerberors的票据

[ec2-user@ip-172-31-8-141 cdh-shell-master]$ kinit fayson
Password for fayson@CLOUDERA.COM: 
[ec2-user@ip-172-31-8-141 cdh-shell-master]$ klist
Ticket cache: FILE:/tmp/krb5cc_500
Default principal: fayson@CLOUDERA.COM

Valid starting     Expires            Service principal
09/01/17 12:27:39  09/02/17 12:27:39  krbtgt/CLOUDERA.COM@CLOUDERA.COM
        renew until 09/08/17 12:27:39
[ec2-user@ip-172-31-8-141 cdh-shell-master]$

[ec2-user@ip-172-31-8-141 ~]$ hadoop fs -ls /extwarehouse/student
ls: Permission denied: user=fayson, access=READ_EXECUTE, inode="/extwarehouse/student":hive:hive:drwxrwx--x
[ec2-user@ip-172-31-8-141 ~]$

beeline命令行查看

[ec2-user@ip-172-31-8-141 ~]$ beeline 
Beeline version 1.1.0-cdh5.11.1 by Apache Hive
beeline> !connect jdbc:hive2://localhost:10000/;principal=hive/ip-172-31-8-141.ap-southeast-1.compute.internal@CLOUDERA.COM
...
INFO  : OK
+-----------+--+
| tab_name  |
+-----------+--+
+-----------+--+
No rows selected (0.295 seconds)
0: jdbc:hive2://localhost:10000/> select * from student;
Error: Error while compiling statement: FAILED: SemanticException No valid privileges
 User fayson does not have privileges for QUERY
 The required privileges: Server=server1->Db=default->Table=student->Column=addr->action=select; (state=42000,code=40000)
0: jdbc:hive2://localhost:10000/>

impala-shell命令行查看

[ec2-user@ip-172-31-8-141 cdh-shell-master]$ impala-shell 
...
[Not connected] > connect ip-172-31-10-156.ap-southeast-1.compute.internal:21000;
Connected to ip-172-31-10-156.ap-southeast-1.compute.internal:21000
Server version: impalad version 2.8.0-cdh5.11.1 RELEASE (build 3382c1c488dff12d5ca8d049d2b59babee605b4e)
[ip-172-31-10-156.ap-southeast-1.compute.internal:21000] > show tables;
Query: show tables
ERROR: AuthorizationException: User 'fayson@CLOUDERA.COM' does not have privileges to access: default.*

[ip-172-31-10-156.ap-southeast-1.compute.internal:21000] > select * from student;
Query: select * from student
Query submitted at: 2017-09-01 12:33:06 (Coordinator: http://ip-172-31-10-156.ap-southeast-1.compute.internal:25000)
ERROR: AuthorizationException: User 'fayson@CLOUDERA.COM' does not have privileges to execute 'SELECT' on: default.student

[ip-172-31-10-156.ap-southeast-1.compute.internal:21000] >

通过hive用户创建的外部表，未给fayson用户赋予student表读权限情况下，无权限访问hdfs的(/extwarehouse/student)数据目录，在beeline和impala-shell命令行下，fayson用户均无权限查询student表数据。
为fayson用户赋予student表读权限
- 注：以下操作均在hive管理员用户下操作
创建student_read角色

0: jdbc:hive2://localhost:10000/> create role student_read;
...
INFO  : Executing command(queryId=hive_20170901124848_927878ba-0217-4a32-a508-bf29fed67be8): create role student_read
...
INFO  : OK
No rows affected (0.104 seconds)
0: jdbc:hive2://localhost:10000/>

将student表的查询权限授权给student_read角色

0: jdbc:hive2://localhost:10000/> grant select on table student to role student_read;
...
INFO  : Executing command(queryId=hive_20170901125252_8702d99d-d8eb-424e-929d-5df352828e2c): grant select on table student to role student_read
...
INFO  : OK
No rows affected (0.111 seconds)
0: jdbc:hive2://localhost:10000/>

将student_read角色授权给fayson用户组

0: jdbc:hive2://localhost:10000/> grant role student_read to group fayson;
...
INFO  : Executing command(queryId=hive_20170901125454_5f27a87e-2f63-46d9-9cce-6f346a0c415c): grant role student_read to group fayson
...
INFO  : OK
No rows affected (0.122 seconds)
0: jdbc:hive2://localhost:10000/>

再次测试，使用fayson用户登录Kerberos，访问HDFS目录
- 访问student数据所在hdfs目录/extwarehouse/student

[ec2-user@ip-172-31-8-141 ~]$ hadoop fs -ls /extwarehouse/student
Found 1 items
-rwxrwx--x+  3 hive hive         39 2017-09-01 14:42 /extwarehouse/student/student.txt
[ec2-user@ip-172-31-8-141 ~]$

beeline查询student表

[ec2-user@ip-172-31-8-141 ~]$ klist
Ticket cache: FILE:/tmp/krb5cc_500
Default principal: fayson@CLOUDERA.COM

Valid starting     Expires            Service principal
09/01/17 12:58:59  09/02/17 12:58:59  krbtgt/CLOUDERA.COM@CLOUDERA.COM
        renew until 09/08/17 12:58:59
[ec2-user@ip-172-31-8-141 ~]$ 
[ec2-user@ip-172-31-8-141 ~]$ beeline 
Beeline version 1.1.0-cdh5.11.1 by Apache Hive
beeline> !connect jdbc:hive2://localhost:10000/;principal=hive/ip-172-31-8-141.ap-southeast-1.compute.internal@CLOUDERA.COM
...
INFO  : OK
+-----------+--+
| tab_name  |
+-----------+--+
| student   |
+-----------+--+
1 row selected (0.294 seconds)
0: jdbc:hive2://localhost:10000/> select * from student;
...
INFO  : OK
+---------------+--------------+---------------+--+
| student.name  | student.age  | student.addr  |
+---------------+--------------+---------------+--+
| zhangsan      | 18           | guangzhou     |
| lisi          | 20           | shenzhen      |
+---------------+--------------+---------------+--+
2 rows selected (0.241 seconds)
0: jdbc:hive2://localhost:10000/>

impala-shell查询student表

[ec2-user@ip-172-31-8-141 cdh-shell-master]$ klist
Ticket cache: FILE:/tmp/krb5cc_500
Default principal: fayson@CLOUDERA.COM

Valid starting     Expires            Service principal
09/01/17 12:58:59  09/02/17 12:58:59  krbtgt/CLOUDERA.COM@CLOUDERA.COM
        renew until 09/08/17 12:58:59
[ec2-user@ip-172-31-8-141 cdh-shell-master]$ impala-shell 
...
[Not connected] > connect ip-172-31-10-156.ap-southeast-1.compute.internal:21000;
Connected to ip-172-31-10-156.ap-southeast-1.compute.internal:21000
Server version: impalad version 2.8.0-cdh5.11.1 RELEASE (build 3382c1c488dff12d5ca8d049d2b59babee605b4e)
[ip-172-31-10-156.ap-southeast-1.compute.internal:21000] > show tables;
Query: show tables
+---------+
| name    |
+---------+
| student |
+---------+
Fetched 1 row(s) in 0.02s
[ip-172-31-10-156.ap-southeast-1.compute.internal:21000] > select * from student;
...
+----------+-----+-----------+
| name     | age | addr      |
+----------+-----+-----------+
| zhangsan | 18  | guangzhou |
| lisi     | 20  | shenzhen  |
+----------+-----+-----------+
Fetched 2 row(s) in 0.13s
[ip-172-31-10-156.ap-southeast-1.compute.internal:21000] >

通过hive用户创建的外部表，给fayson用户赋予student表读权限后，可正常访问hdfs的(/extwarehouse/student)数据目录，在beeline和impala-shell命令行下，fayson用户均可查询student表数据。
- 开启外部表的数据父目录ACL同步后，不需要单独的维护外部表数据目录权限。

人面猴
序言：七十年代末，一起剥皮案震惊了整个滨河市，随后出现的几起案子，更是在滨河造成了极大的恐慌，老刑警刘岩，带你破解...
沈念sama阅读 204,053评论 6赞 478
死咒
序言：滨河连续发生了三起死亡事件，死亡现场离奇诡异，居然都是意外死亡，警方通过查阅死者的电脑和手机，发现死者居然都...
沈念sama阅读 85,527评论 2赞 381
救了他两次的神仙让他今天三更去死
文/潘晓璐我一进店门，熙熙楼的掌柜王于贵愁眉苦脸地迎上来，“玉大人，你说我怎么就摊上这事。” “怎么了？”我有些...
开封第一讲书人阅读 150,779评论 0赞 337
道士缉凶录：失踪的卖姜人
文/不坏的土叔我叫张陵，是天一观的道长。经常有香客问我，道长，这世上最难降的妖魔是什么？我笑而不...
开封第一讲书人阅读 54,685评论 1赞 276
港岛之恋（遗憾婚礼）
正文为了忘掉前任，我火速办了婚礼，结果婚礼上，老公的妹妹穿的比我还像新娘。我一直安慰自己，他们只是感情好，可当我...
茶点故事阅读 63,699评论 5赞 366
恶毒庶女顶嫁案：这布局不是一般人想出来的
文/花漫我一把揭开白布。她就那样静静地躺着，像睡着了一般。火红的嫁衣衬着肌肤如雪。梳的纹丝不乱的头发上，一...
开封第一讲书人阅读 48,609评论 1赞 281
城市分裂传说
那天，我揣着相机与录音，去河边找鬼。笑死，一个胖子当着我的面吹牛，可吹牛的内容都是我干的。我是一名探鬼主播，决...
沈念sama阅读 37,989评论 3赞 396
双鸳鸯连环套：你想象不到人心有多黑
文/苍兰香墨我猛地睁开眼，长吁一口气：“原来是场噩梦啊……” “哼！你这毒妇竟也来了？” 一声冷哼从身侧响起，我...
开封第一讲书人阅读 36,654评论 0赞 258
万荣杀人案实录
序言：老挝万荣一对情侣失踪，失踪者是张志新（化名）和其女友刘颖，没想到半个月后，有当地人在树林里发现了一具尸体，经...
沈念sama阅读 40,890评论 1赞 298
护林员之死
正文独居荒郊野岭守林人离奇死亡，尸身上长有42处带血的脓包…… 初始之章·张勋以下内容为张勋视角年9月15日...
茶点故事阅读 35,634评论 2赞 321
白月光启示录
正文我和宋清朗相恋三年，在试婚纱的时候发现自己被绿了。大学时的朋友给我发了我未婚夫和他白月光在一起吃饭的照片。...
茶点故事阅读 37,716评论 1赞 330
活死人
序言：一个原本活蹦乱跳的男人离奇死亡，死状恐怖，灵堂内的尸体忽然破棺而出，到底是诈尸还是另有隐情，我是刑警宁泽，带...
沈念sama阅读 33,394评论 4赞 319
日本核电站爆炸内幕
正文年R本政府宣布，位于F岛的核电站，受9级特大地震影响，放射性物质发生泄漏。R本人自食恶果不足惜，却给世界环境...
茶点故事阅读 38,976评论 3赞 307
男人毒药：我在死后第九天来索命
文/蒙蒙一、第九天我趴在偏房一处隐蔽的房顶上张望。院中可真热闹，春花似锦、人声如沸。这庄子的主人今日做“春日...
开封第一讲书人阅读 29,950评论 0赞 19
一桩弑父案，背后竟有这般阴谋
文/苍兰香墨我抬头看了看天上的太阳。三九已至，却和暖如春，着一层夹袄步出监牢的瞬间，已是汗流浃背。一阵脚步声响...
开封第一讲书人阅读 31,191评论 1赞 260
情欲美人皮
我被黑心中介骗来泰国打工，没想到刚下飞机就差点儿被人妖公主榨干…… 1. 我叫王不留，地道东北人。一个月前我还...
沈念sama阅读 44,849评论 2赞 349
代替公主和亲
正文我出身青楼，却偏偏与公主长得像，于是被迫代替她去往敌国和亲。传闻我的和亲对象是个残疾皇子，可洞房花烛夜当晚...
茶点故事阅读 42,458评论 2赞 342

19.Hive外部表权限用Sentry管理

19.1 环境介绍

19.2 实验准备

19.2.1外部表数据父目录创建

19.3 Hive外部表创建

19.4 给fayson用户student表读权限

推荐阅读更多精彩内容