包含python3中JSON数据的不同压缩方法

问题描述

因此，我想使用其他压缩器压缩JSON数据.我用它来压缩JSON.

So, I want to compress my JSON data using different compressor. I used this to compress the JSON.

import gzip
import JSON

with gzip.GzipFile('2.json', 'r') as isfile:
    for line in isfile:
        obj = json.loads(line)

这会引起错误.

raise OSError('Not a gzipped file (%r)' % magic)

OSError: Not a gzipped file (b'[\n')

我也尝试使用直接压缩.

I also tried direct compressing using.

zlib_data= zlib.compress(data)

这会引发错误.

return lz4.block.compress(*args, **kwargs)

TypeError: a bytes-like object is required, not 'list'

因此，基本上，我想使用所有方法来压缩JSON，并计算使用不同方法进行压缩所花费的时间.

So, Basically i want to compress a JSON using all the methods and to compute the time taken for the compression in different methods.

推荐答案

在python2.7上

这似乎是您的数据类型的问题

On python2.7

it seems to be a problem of the type of your data

要压缩的数据应为"str"类型

import gzip
import json
import lz4
import time

with gzip.GzipFile('data.gz','w') as fid_gz:
    with open('data.json','r') as fid_json:
        # get json as type dict
        json_dict = json.load(fid_json)
        # convert dict to str
        json_str = str(json_dict)
    # write string
    fid_gz.write(json_str)

# check well maded
with gzip.GzipFile('data.gz','r') as fid_gz :
    print(fid_gz.read())

即使gzip压缩

gzip.zlib.compress(json_str,9)

即使lz4压缩

lz4.block.compress(json_str)

时间检查将是

# set start time
st = time.time()
# calculate elasped time
print(time.time() - st)

---

在python3.5上

python2.7和python 3之间的区别在于您要压缩的数据类型

---

On python3.5

the difference between python2.7 and python 3 is the type of your data to compress

要压缩的数据应通过bytes()为字节"类型

制作.gz文件时

with gzip.GzipFile('data.gz','w') as fid_gz:
    with open('data.json','r') as fid_json:
        json_dict = json.load(fid_json)
        json_str = str(json_dict)
        # bytes(string, encoding)
        json_bytes = bytes(json_str,'utf8')
    fid_gz.write(json_bytes)

或仅使用gzip.compress(data，compresslevel = 9)进行压缩

or just compress with gzip.compress(data, compresslevel=9)

# 'data' takes bytes
gzip.compress(json_bytes)

或仅使用zlib.compress(bytes，level = -1，/)进行压缩

or just compress with zlib.compress(bytes, level=-1, /)

gzip.zlib.compress(json_bytes,9)

或仅使用lz4.bloc.compress(source，compression = 0)进行压缩

or just compress with lz4.bloc.compress(source, compression=0)

# 'source' takes both 'str' and 'byte'
lz4.block.compress(json_str)
lz4.block.compress(json_bytes)

测量时间取决于您的意图.

the measuring time is on your intention.

欢呼

这篇关于包含python3中JSON数据的不同压缩方法的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持！