code1
feature1 = "id, application_id, user_profile_id, amount"
sql1 = """SELECT %s FROM tb_source_data.loan_applications LIMIT %d"""%(feature1, 3)
hiveContext.sql(sql1).show(1000, truncate=False)
等价于
hiveContext.sql("""SELECT %s FROM tb_source_data.loan_applications LIMIT %d"""%("id, application_id, user_profile_id, amount", 3)).show(1000, truncate=False)
输出结果
+-----+----------------------------+---------------+--------+
|id |application_id |user_profile_id|amount |
+-----+----------------------------+---------------+--------+
|18132|AAAA17071813423573529711111 |17322 |0.0 |
|18133|BBBBB17071813472976219211111|17323 |100000.0|
|18134|CCCC17071813490193476111111 |17324 |0.0 |
+-----+----------------------------+---------------+--------+
code2
查看一列的数据
for i in hiveContext.sql("DESC tb_source_data.loan_applications").collect():
print i[0]+","
[output]:
id,
application_id,
user_profile_id,
amount,
tenor,
