问题描述
我需要处理大量CSV文件,其中时间戳始终是一个字符串,代表以毫秒为单位的unix时间戳.我找不到有效修改这些列的方法.
I need to process a huge amount of CSV files where the time stamp is always a string representing the unix timestamp in milliseconds. I could not find a method yet to modify these columns efficiently.
这是我想出的,但是,这当然只重复了该列,而我不得不以某种方式将其放回原始数据集中.我确定创建DataFrame
This is what I came up with, however this of course duplicates only the column and I have to somehow put it back to the original dataset. I'm sure it can be done when creating the DataFrame
?
import sys
if sys.version_info[0] < 3:
from StringIO import StringIO
else:
from io import StringIO
import pandas as pd
data = 'RUN,UNIXTIME,VALUE\n1,1447160702320,10\n2,1447160702364,20\n3,1447160722364,42'
df = pd.read_csv(StringIO(data))
convert = lambda x: datetime.datetime.fromtimestamp(x / 1e3)
converted_df = df['UNIXTIME'].apply(convert)
这将选择"UNIXTIME"列并将其更改为
This will pick the column 'UNIXTIME' and change it from
0 1447160702320
1 1447160702364
2 1447160722364
Name: UNIXTIME, dtype: int64
进入此
0 2015-11-10 14:05:02.320
1 2015-11-10 14:05:02.364
2 2015-11-10 14:05:22.364
Name: UNIXTIME, dtype: datetime64[ns]
但是,我想使用pd.apply()
之类的方法来获取转换后的列返回的整个数据集,或者像我已经写的那样,仅在从CSV生成DataFrame时创建日期时间.
However, I would like to use something like pd.apply()
to get the whole dataset returned with the converted column or as I already wrote, simply create datetimes when generating the DataFrame from CSV.
推荐答案
您可以使用并传递arg unit='ms'
:
In [5]:
df['UNIXTIME'] = pd.to_datetime(df['UNIXTIME'], unit='ms')
df
Out[5]:
RUN UNIXTIME VALUE
0 1 2015-11-10 13:05:02.320 10
1 2 2015-11-10 13:05:02.364 20
2 3 2015-11-10 13:05:22.364 42
这篇关于 pandas 将具有unix时间戳(以毫秒为单位)的行转换为日期时间的文章就介绍到这了,希望我们推荐的答案对大家有所帮助,也希望大家多多支持!