对于上篇中,预处理后的结构化数据,本篇希望对其进行数据的格式化输出。
主要目标是:根据地点id,起止时间段的设定,在条件筛选下得到对应的数据,并格式化输出到本地文件中。
上篇中最后得到的结构化数据:
2017-08-28 23:37:00,14,84742aa602e4
2017-08-28 23:38:00,14,84742aa602e4
2017-08-28 23:39:00,14,84742aa602e4
2017-08-28 23:40:00,14,84742aa602e4
2017-08-28 23:41:00,14,84742aa602e4
2017-08-28 23:42:00,14,84742aa602e4
2017-08-28 23:43:00,14,84742aa602e4
2017-08-28 23:44:00,14,84742aa602e4
2017-08-29 05:16:00,13,84742ab11871
2017-08-29 05:17:00,13,84742ab11871
2017-08-29 05:18:00,13,b0e235c0398a
2017-08-29 05:18:00,13,84742ab11871
2017-08-29 05:19:00,13,b0e235c0398a
2017-08-29 05:19:00,13,84742ab11871
2017-08-29 05:20:00,13,b0e235c0398a
2017-08-29 05:20:00,13,84742ab11871
2017-08-29 05:21:00,13,b0e235c0398a
2017-08-29 05:21:00,13,84742ab11871
2017-08-29 05:22:00,13,b0e235c0398a
2017-08-29 05:22:00,13,84742ab11871
2017-08-29 05:23:00,13,b0e235c0398a
2017-08-29 05:23:00,13,84742ab11871
2017-08-29 05:24:00,13,b0e235c0398a
2017-08-29 05:24:00,13,84742ab11871
2017-08-29 05:25:00,13,b0e235c0398a
2017-08-29 05:25:00,13,84742ab11871
2017-08-29 05:26:00,13,b0e235c0398a
2017-08-29 05:27:00,13,b0e235c0398a
2017-08-29 06:53:00,18,84742aac34d7
2017-08-29 06:54:00,18,84742aac34d7
2017-08-29 06:55:00,18,84742aac34d7
2017-08-29 06:56:00,18,84742aac34d7
2017-08-29 06:57:00,18,84742aac34d7
2017-08-29 06:58:00,13,5cf7c3756af6
2017-08-29 06:58:00,18,84742aac34d7
2017-08-29 06:59:00,13,5cf7c3756af6
2017-08-29 06:59:00,18,84742aac34d7
2017-08-29 07:00:00,13,5cf7c3756af6
2017-08-29 07:00:00,18,84742aac34d7
2017-08-29 07:01:00,13,5cf7c3756af6
2017-08-29 07:01:00,18,84742aac34d7
2017-08-29 07:02:00,13,5cf7c3756af6
2017-08-29 07:02:00,18,84742aac34d7
2017-08-29 07:03:00,13,5cf7c3756af6
2017-08-29 07:04:00,13,5cf7c3756af6
2017-08-29 07:05:00,13,5cf7c3756af6
2017-08-29 07:06:00,13,5cf7c3756af6
2017-08-29 07:07:00,13,5cf7c3756af6
2017-08-29 07:08:00,12,742344af2efd
2017-08-29 07:09:00,12,742344af2efd
2017-08-29 07:09:00,13,84742ab11871
2017-08-29 07:10:00,12,742344af2efd
2017-08-29 07:10:00,13,84742ab11871
2017-08-29 07:11:00,12,742344af2efd
2017-08-29 07:11:00,13,84742ab11871
2017-08-29 07:12:00,12,742344af2efd
2017-08-29 07:12:00,13,84742ab11871
2017-08-29 07:12:00,18,84742aac34f3
2017-08-29 07:13:00,12,742344af2efd
2017-08-29 07:13:00,12,acc1eef136c1
2017-08-29 07:13:00,13,84742ab11871
2017-08-29 07:13:00,18,84742aac34f3
2017-08-29 07:14:00,12,a4717460508e
2017-08-29 07:14:00,12,742344af2efd
2017-08-29 07:14:00,12,acc1eef136c1
2017-08-29 07:14:00,13,84742ab11871
2017-08-29 07:14:00,18,84742aac34f3
2017-08-29 07:15:00,12,a4717460508e
2017-08-29 07:15:00,12,742344af2efd
2017-08-29 07:15:00,12,a8c83abd75f6
2017-08-29 07:15:00,12,acc1eef136c1
2017-08-29 07:15:00,13,84742ab11871
2017-08-29 07:15:00,18,84742aac34f3
2017-08-29 07:16:00,12,a4717460508e
2017-08-29 07:16:00,12,742344af2efd
2017-08-29 07:16:00,12,a8c83abd75f6
2017-08-29 07:16:00,12,acc1eef136c1
2017-08-29 07:16:00,13,84742ab11871
2017-08-29 07:16:00,18,84742aac34f3
2017-08-29 07:17:00,12,a4717460508e
2017-08-29 07:17:00,12,742344af2efd
2017-08-29 07:17:00,12,a8c83abd75f6
2017-08-29 07:17:00,12,acc1eef136c1
2017-08-29 07:17:00,13,84742ab11871
2017-08-29 07:17:00,18,84742aac34f3
2017-08-29 07:18:00,12,a4717460508e
2017-08-29 07:18:00,12,acc1eef136c1
2017-08-29 07:18:00,12,a8c83abd75f6
2017-08-29 07:18:00,13,84742ab11871
2017-08-29 07:18:00,18,84742aac34f3
2017-08-29 07:19:00,12,a4717460508e
2017-08-29 07:19:00,12,acc1eef136c1
2017-08-29 07:19:00,12,a8c83abd75f6
2017-08-29 07:19:00,18,84742aac34f3
2017-08-29 07:20:00,12,a4717460508e
2017-08-29 07:20:00,12,acc1eef136c1
2017-08-29 07:20:00,12,742344af2efd
2017-08-29 07:20:00,12,a8c83abd75f6
2017-08-29 07:20:00,18,84742aac34f3
2017-08-29 07:21:00,12,acc1eef136c1
2017-08-29 07:21:00,12,742344af2efd
2017-08-29 07:21:00,12,a8c83abd75f6
2017-08-29 07:21:00,12,a4717460508e
2017-08-29 07:21:00,13,dc742ab11aa2
2017-08-29 07:21:00,13,58404eb6d01d
2017-08-29 07:21:00,18,84742aac34f3
2017-08-29 07:22:00,12,a4717460508e
2017-08-29 07:22:00,12,acc1eef136c1
2017-08-29 07:22:00,12,742344af2efd
2017-08-29 07:22:00,12,a8c83abd75f6
2017-08-29 07:22:00,13,58404eb6d01d
2017-08-29 07:22:00,13,dc742ab11aa2
2017-08-29 07:22:00,18,205d47dfd85b
2017-08-29 07:22:00,18,84742aac33ed
2017-08-29 07:22:00,18,84742aac34f3
2017-08-29 07:23:00,12,742344af2efd
2017-08-29 07:23:00,12,a8c83abd75f6
2017-08-29 07:23:00,12,acc1eef136c1
2017-08-29 07:23:00,12,a4717460508e
2017-08-29 07:23:00,13,58404eb6d01d
2017-08-29 07:23:00,13,dc742ab11aa2
2017-08-29 07:23:00,18,205d47dfd85b
2017-08-29 07:23:00,18,fc1a11f6c610
2017-08-29 07:23:00,18,84742aac33ed
2017-08-29 07:23:00,18,84742aac34f3
2017-08-29 07:23:00,18,84742aac34d7
2017-08-29 07:23:00,18,84742aac34e1
......
比如我想得到id为12的place,其时间区间为 2017-08-28 00:00:00 到 2017-08-29 12:00:00
优化后的python程序如下:
# -*- coding: UTF-8 -*-
#import numpy as np
import pandas as pd
from pandas import Series, DataFrame
#from dateutil.parser import parse
records = pd.read_csv('./macdata/normal_origin_info_test.txt',names=['time','idplace','mac'])
#读取文件,records的数据结构为dataframe,设定列名为[time,idplace,mac。
pid = 12
#设定place id 为12
start_time = '2017-08-28 00:00:00'
#设定起始时间
end_time = '2017-08-29 12:00:00'
#设定截止时间
def getInfoByIdpidtime(idplace,stime,etime):
#定义查询函数
s = stime[5:7]+stime[8:10]
#格式化起始时间
e = etime[5:7]+etime[8:10]
#格式化截止时间
idplaceinfo_test = records[ (records['idplace'] == idplace) & (records['time'] > stime) & (records['time'] < etime) ]
#条件筛选 依照dataframe语法规范
outputpath = './macdata/selectinfo/' + 'place_' + str(idplace) + '_' + s + '-' + e + '.csv'
#设定输出路径和输出文件名
idplaceinfo_test.to_csv(outputpath, header=False, index=False)
getInfoByIdpidtime(pid,start_time,end_time)
#执行查询
最终得到的数据如下:
2017-08-29 07:08:00,12,742344af2efd
2017-08-29 07:09:00,12,742344af2efd
2017-08-29 07:10:00,12,742344af2efd
2017-08-29 07:11:00,12,742344af2efd
2017-08-29 07:12:00,12,742344af2efd
2017-08-29 07:13:00,12,742344af2efd
2017-08-29 07:13:00,12,acc1eef136c1
2017-08-29 07:14:00,12,a4717460508e
2017-08-29 07:14:00,12,742344af2efd
2017-08-29 07:14:00,12,acc1eef136c1
2017-08-29 07:15:00,12,a4717460508e
2017-08-29 07:15:00,12,742344af2efd
2017-08-29 07:15:00,12,a8c83abd75f6
2017-08-29 07:15:00,12,acc1eef136c1
2017-08-29 07:16:00,12,a4717460508e
2017-08-29 07:16:00,12,742344af2efd
2017-08-29 07:16:00,12,a8c83abd75f6
2017-08-29 07:16:00,12,acc1eef136c1
2017-08-29 07:17:00,12,a4717460508e
2017-08-29 07:17:00,12,742344af2efd
2017-08-29 07:17:00,12,a8c83abd75f6
2017-08-29 07:17:00,12,acc1eef136c1
2017-08-29 07:18:00,12,a4717460508e
2017-08-29 07:18:00,12,acc1eef136c1
2017-08-29 07:18:00,12,a8c83abd75f6
2017-08-29 07:19:00,12,a4717460508e
2017-08-29 07:19:00,12,acc1eef136c1
2017-08-29 07:19:00,12,a8c83abd75f6
2017-08-29 07:20:00,12,a4717460508e
2017-08-29 07:20:00,12,acc1eef136c1
2017-08-29 07:20:00,12,742344af2efd
2017-08-29 07:20:00,12,a8c83abd75f6
2017-08-29 07:21:00,12,acc1eef136c1
2017-08-29 07:21:00,12,742344af2efd
2017-08-29 07:21:00,12,a8c83abd75f6
2017-08-29 07:21:00,12,a4717460508e
2017-08-29 07:22:00,12,a4717460508e
2017-08-29 07:22:00,12,acc1eef136c1
2017-08-29 07:22:00,12,742344af2efd
2017-08-29 07:22:00,12,a8c83abd75f6
2017-08-29 07:23:00,12,742344af2efd
2017-08-29 07:23:00,12,a8c83abd75f6
2017-08-29 07:23:00,12,acc1eef136c1
2017-08-29 07:23:00,12,a4717460508e
又是毫无建树的一天啊,时间过得飞快
(:з」∠)