接一位微信好友所托叫爬取腾讯视频的原地址,以此可以去掉烦人的广告。虽有一些插件可以支持,但代码撸出来带劲点吧。。。
步骤不完整,就简单说些注意点和发现的东西。
抓包首先从h5,app端抓起,web端花招会多点。。。
直接看几个接口吧。
1.https://bkvv.video.qq.com/getinfo?_qv_rmt={$u1}&_qv_rmt2={$u2}&defn=auto&platform={$plt}&otype=json&sdtfrom={$std}&_rnd={$ts}&appVer=0.0.1&dtype=3&vid={$vid}&newnettype=1
2.https://h5vv.video.qq.com/getinfo
3.https://h5vv.video.qq.com/getkey
4.https://vv.video.qq.com/getinfo
5.https://vv.video.qq.com/getkey
接口的不同和利用接口里面参数的不同可以获得一个视频的各片段各集的url,或者视频的M3u8文件,我是利用2,3接口直接获取一个完整视频的url。
2接口中getinfo的参数如下
params = {
'charge': 0,
'vid': vid, *url或html获取
'defaultfmt': 'auto',
'otype': 'json',
'guid': '8fffd19befa1413953bb108f58e49b3b', *发觉有问题用不了就要换,抓包看
'platform': plt,
'defnpayver': 1,
'appVer': '3.0.83',
'sdtfrom':std,
'host':'v.qq.com',
'ehost':'https%3A%2F%2Fv.qq.com%2Fx%2Fcover%2Fnuijxf6k13t6z9b%2Fl0023olk3g4.html',
'defn':'mp4',
'fhdswitch': 0,
'show1080p':1,
'isHLS':0,
'newplatform':'v1010',
'defsrc':1,
'_0': 'undefined',
'_1': 'undefined',
'_2': 'undefined',
'_': int(round(time.time() * 1000)),
'callback':jsonpCallback, *返回json的前缀
}
r = requests.get('https://h5vv.video.qq.com/getinfo', params=params).content
上面的参数基本固定也最好不要落下,vid要从自己获取。这个接口获取到的信息是为了获得vkey而作准备。
从2中返回的json获得
- 视频url前缀
url_prefix = data['vl']['vi'][0]['ul']['ui'][0]['url'] - MP4文件名字,q0200qbrzbk.mp4这种全集的,q0200qbrzbk.p201.1.mp4这种分段的
fn_pre = data['vl']['vi'][0]['lnk']
filename = fn_pre + '.mp4'
接着请求3接口
参数:
params = {
'charge': 0,
'vid': vid, *视频vid
'format':2,
'otype': 'json',
'guid': '8fffd19befa1413953bb108f58e49b3b',
'platform': 10901,
'defnpayver': 0,
'appVer': '3.0.83',
'vt':0,
'sdtfrom':'v1010',
'_rnd':rmt['t'], *时间戳重要,没有直接20k速度
'_qv_rmt': rmt['u1'], *限速算法,重要,没有直接20k速度
'_qv_rmt2': rmt['u2'], *同上
'ui_host': 2,
'filename':filename,
'callback':jsonpCallback,
'_':int(round(time.time() * 1000)), *13位时间戳,我测没有会卡顿
}
r = requests.get('https://h5vv.video.qq.com/getkey', params=params).content
核心来了,限速折腾了一天,直到爬各种数据拿到js的算法整合而成。qvrmt这两个经过算法而生成的参数腾讯出不久,所以觉得这种爬取视频方法短时间不会失效。
ok,从以下算法弄出三个参数扔到上面的接口上去请求。
- qvrmt 请求方法 rmt = getQv(plt, vid, std, str(1)),分别为platform,vid,sdtfrom
# coding: utf-8
import time
import hashlib
Seed = "#$#@#*ad"
urlStr = "ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789+/="
def hexToString(h):
r = ''
index = 2 if h[0:2] == '0x' else 0
indes = []
indes.append(index)
while(index <len(h)-2):
index+=2
indes.append(index)
for i in indes:
b = int(h[i:i+2],16)
r+=chr(b)
return r
def getQv(plt, vid, std, sts, ts=int(time.time())):
global Seed
ts = str(ts)
p = {
"plt":plt,
"vid":vid,
"std":std,
"sts":sts,
"ts":ts,
}
md = hashlib.md5()
md.update((str(plt) + vid + ts + Seed + sts + std).encode('utf-8'))
result = hexToString(md.hexdigest())
u = urlenc(tempcalc(result, Seed),sts[0],ts)
c = urlenc(tempcalc(result, '86FG@hdf'), sts[0],ts)
u1 = U1(u, 0)
u2 = U1(u, 1)
data = {
'p':p,
'u':u,
'c':c,
'u1':u1,
'u2':u2,
't':ts
}
return data
def urlenc(input,sts,ts):
global urlStr, output
chr1 = chr2 = chr3 = enc1 = enc2 = enc3 = enc4 = ''
chr5 = ''
chr6 = ''
output = ''
i = 0
while(i < len(input)):
chr1 = ord(input[i])
i += 1
m1 = i
i += 1
if(len(input)>m1):
chr2 = ord(input[m1])
else:
chr5='NaN'
m = i
i += 1
if(m>len(input) or m==len(input)):
chr6='NaN'
else:
chr3=ord(input[m])
if(i==15):
output = output+'A'
output = output+sts
output = output+ts
enc1 = chr1 >> 2
enc2 = ((chr1 & 3) << 4) | (chr2 >> 4)
enc3 = ((chr2 & 15) << 2) | (chr3 >> 6)
enc4 = chr3 & 63
if (chr5 == 'NaN'):
enc3 = enc4 = 64
elif(chr6=='NaN'):
enc4 = 64
output = output+urlStr[enc1]+urlStr[enc2]+urlStr[enc3]+urlStr[enc4]
return output
def tempcalc(a,b):
r = ''
for i in range(len(a)):
chr1 = (ord(a[i])^ord(b[i%4]))
r = r+chr(chr1)
return r
def U1(a,b):
r = ''
index = b
indes = []
indes.append(index)
while (index < len(a) - 2):
index += 2
indes.append(index)
for i in indes:
r+= a[i]
i+=2
return r
最后从返回json中获得重要的vkey拼接视频地址
url = '{}{}?sdtfrom=v1010&guid=8fffd19befa1413953bb108f58e49b3b&vkey={}'.format(url_prefix,filename,data['key'])
如
[url]= http://222.73.132.155/om.tc.qq.com/ACQ0qWH8FNLvmirYrJFrmcK_-3iFtXdnx2wbFY1zuvv8/p0556o7jft0.p712.1.mp4?sdtfrom=v1010&guid=ffc2f4cc36a272e31b71035fcda35910&vkey=072F6FDB970690CADB8E3EA44A8839FDD911D535BB7D17B3C7AB942B26D59AF7F92F1DE22B1288C9052FDD9D2594195D257F707E7B4063FE62E47C56C90B1C87DB44D16E3502D9F87333807E7E3DD2C62519608B6646CFE2A07519D28FAD923BC2934A7DCDFB51D8D7400910FCBE5FF73A4F6A3056F96209
获得m3u8文件貌似不会限速,大家研究吧。
后续转源码上来,flask部署爬虫放云上爽歪歪,说得不细,重点是说明限速问题处理,