问题描述
尽管我可能正在诊断事件的根本原因,确定事件影响了多少用户,或者为了评估最近代码更改对性能和吞吐量的影响而提取了时序日志,但我的工具却保持不变:grep
,awk
,sed
,tr
,uniq
,sort
,zcat
,tail
,head
,join
和split
.为了将它们粘合在一起,Unix提供了管道,对于更高级的过滤,我们提供了xargs
.如果这些令我失望,总会出现perl -e
.
Though I may be diagnosing the root cause of an event, determining how many users it affected, or distilling timing logs in order to assess the performance and throughput impact of a recent code change, my tools stay the same: grep
, awk
, sed
, tr
, uniq
, sort
, zcat
, tail
, head
, join
, and split
. To glue them all together, Unix gives us pipes, and for fancier filtering we have xargs
. If these fail me, there's always perl -e
.
这些工具非常适合处理CSV文件,制表符分隔的文件,可预测的行格式的日志文件或具有逗号分隔的键/值对的文件.换句话说,每行几乎没有上下文的文件.
These tools are perfect for processing CSV files, tab-delimited files, log files with a predictable line format, or files with comma-separated key-value pairs. In other words, files where each line has next to no context.
我最近需要遍历千兆字节的XML,以建立用户使用情况的直方图.使用我拥有的工具,这已经足够容易了,但是对于更复杂的查询,常规方法会失效.假设我的文件中包含以下内容:
I recently needed to trawl through Gigabytes of XML to build a histogram of usage by user. This was easy enough with the tools I had, but for more complicated queries the normal approaches break down. Say I have files with items like this:
<foo user="me">
<baz key="zoidberg" value="squid" />
<baz key="leela" value="cyclops" />
<baz key="fry" value="rube" />
</foo>
假设我要生成一个从用户到每个<foo>
平均<baz>
个数的映射.不再逐行处理:我需要知道我当前正在检查的用户<foo>
,所以我知道要更新的平均值.能够完成此任务的任何类型的Unix衬垫都可能是难以理解的.
And let's say I want to produce a mapping from user to average number of <baz>
s per <foo>
. Processing line-by-line is no longer an option: I need to know which user's <foo>
I'm currently inspecting so I know whose average to update. Any sort of Unix one liner that accomplishes this task is likely to be inscrutable.
幸运的是,在XML领域,我们拥有诸如XPath,XQuery和XSLT之类的出色技术来帮助我们.
Fortunately in XML-land, we have wonderful technologies like XPath, XQuery, and XSLT to help us.
以前,我已经习惯于使用精美的XML::XPath
Perl模块来完成上述查询,但是找到了可以在当前窗口上运行XPath表达式的TextMate插件,我停止编写一次性的Perl脚本来查询XML.我刚刚发现有关 XMLStarlet 的内容,该内容在我输入时会安装,并且我希望将来会使用
Previously, I had gotten accustomed to using the wonderful XML::XPath
Perl module to accomplish queries like the one above, but after finding a TextMate Plugin that could run an XPath expression against my current window, I stopped writing one-off Perl scripts to query XML. And I just found out about XMLStarlet which is installing as I type this and which I look forward to using in the future.
所以这引出了我的问题:是否有类似JSON的工具?某些调查任务要求我对JSON文件执行类似的查询只是时间问题,并且如果没有XPath和XSLT之类的工具,那么这样的任务将变得更加困难.如果我有一堆看起来像这样的JSON:
So this leads me to my question: are there any tools like this for JSON? It's only a matter of time before some investigation task requires me to do similar queries on JSON files, and without tools like XPath and XSLT, such a task will be a lot harder. If I had a bunch of JSON that looked like this:
{
"firstName": "Bender",
"lastName": "Robot",
"age": 200,
"address": {
"streetAddress": "123",
"city": "New York",
"state": "NY",
"postalCode": "1729"
},
"phoneNumber": [
{ "type": "home", "number": "666 555-1234" },
{ "type": "fax", "number": "666 555-4567" }
]
}
并且想要找到每个人的平均电话号码,我可以使用XPath做这样的事情:
And wanted to find the average number of phone numbers each person had, I could do something like this with XPath:
fn:avg(/fn:count(phoneNumber))
问题
- 是否有任何命令行工具可以在此查询" JSON文件方式吗?
- 如果您必须处理大量Unix命令行上的JSON文件,您使用什么工具?
- 哎呀,还有工作要做吗制作这样的查询语言JSON吗?
- 如果您确实在你的日常工作,你是什么喜欢/不喜欢他们?在那里有陷阱吗?
- Are there any command-line toolsthat can "query" JSON files in thisway?
- If you have to process a bunch ofJSON files on a Unix command line,what tools do you use?
- Heck, is there even work being doneto make a query language like thisfor JSON?
- If you do use tools like this inyour day-to-day work, what do youlike/dislike about them? Are thereany gotchas?
我注意到越来越多的数据使用JSON进行序列化,因此在将来分析大型数据转储时,像这样的处理工具将至关重要. JSON的语言库非常强大,编写脚本来进行这种处理非常容易,但实际上需要人们玩弄数据外壳工具.
I'm noticing more and more data serialization is being done using JSON, so processing tools like this will be crucial when analyzing large data dumps in the future. Language libraries for JSON are very strong and it's easy enough to write scripts to do this sort of processing, but to really let people play around with the data shell tools are needed.
- Grep and Sed Equivalent for XML Command Line Processing
- Is there a query language for JSON?
- JSONPath or other XPath like utility for JSON/Javascript; or Jquery JSON
推荐答案
我刚刚发现了这一点:
http://stedolan.github.com/jq/
"jq是轻量级且灵活的命令行JSON处理器."
"jq is a lightweight and flexible command-line JSON processor."
2014更新:
@ user456584提及:
@user456584 mentioned:
在 json
自述文件中,位于 http://github.com/trentm/json 有很多类似的东西
这篇关于什么是JSON的良好CLI工具?的文章就介绍到这了,希望我们推荐的答案对大家有所帮助,也希望大家多多支持!