mongodb 数据库导出csv文件
- 找到mongoexport所在文件夹，选中文件夹，右键shift，选在此处打命令窗口
- 进入cmd窗口，输入以下命令
- mongoexport -d douban -c top250 --csv -f id,place,level,time,quote,name -o douban.csv
- d douban为数据库名 c top250为新建表名 f为要导出的字段 o为导出表的名称
- 导出的文件存放在bin文件夹中，会发现乱码
- 用记事本打开选择UTF-8保存再用excel打开即不会乱码
沈梦辰微博评论爬取
hello world
import requests
import json
import time
import pymongo
client = pymongo.MongoClient('localhost', 27017)
weibo = client['weibo']
comment_shengmengc = weibo['comment_shengmengc']
headers = {
"Cookies":'xxxxxxxxxxx',
"User-Agent":'Mozilla/5.0 (iPhone; CPU iPhone OS 9_1 like Mac OS X) AppleWebKit/601.1.46 (KHTML, like Gecko) Version/9.0 Mobile/13B143 Safari/601.1'
}
url_comment = ['http://m.weibo.cn/api/comments/show?id=4060977869675098&page={}'.format(str(i)) for i in range(0,1000)]
def get_comment(url):
wb_data = requests.get(url,headers=headers).text
data_comment = json.loads(wb_data)
try:
datas = data_comment['data']
for data in datas:
comment = {"comment":data.get("text")}
comment_shengmengc.insert_one(comment)
except KeyError:
pass
for url in url_comment:
get_comment(url)
time.sleep(2)
