GCloud在ubuntu 初始化的问题解决
sudo chown shootime -R /home/shootime/.config/gcloud
1. Install the latest Cloud Tools version (154.0.1)
https://cloud.google.com/sdk/docs/#deb
2. 下载task_events部分数据[http://www.cnblogs.com/instant7/p/4102818.html]
importurllib2
url='https://commondatastorage.googleapis.com/clusterdata-2011-1/'f= open('C:\\SHA256SUM')
l=f.readlines()
f.close()foriinl:ifi.count('task_events')>0:
fileAddr= i.split()[1][1:]
fileName= fileAddr.split('/')[1]print'downloading', fileName
data= urllib2.urlopen(url+fileAddr).read()print'saving', fileName
fileDown= open('C:\\task_events\\'+fileName,'wb')
fileDown.write(data)
fileDown.close()
3. 云服务器ssh远程下载到本地文件夹
在本地终端打开下列命令
sudo scp cluster@13.94.41.xxx:~/data/clusterdata-2011-2/task_usage/part-00000-of-00500.csv.gz ~/Downloads/
sudo scp -r cluster@13.94.41.xxx:~/data/clusterdata-2011-2/task_usage /media/shootime/TFV/cluster
4. 下载数据集
gsutil cp -R gs://clusterdata-2011-2/task_usage ~/data/
