使用jq将CSV转换为JSON

问题描述

如果您有这样的csv数据集:

If you have a csv dataset like this:

name, age, gender
john, 20, male
jane, 30, female
bob, 25, male

你能做到这一点吗?

[ {"name": "john", "age": 20, "gender: "male"},
  {"name": "jane", "age": 30, "gender: "female"},
  {"name": "bob", "age": 25, "gender: "male"} ]

仅使用jq?

我发现了这篇文章，该文章展示了我正在尝试执行此操作，但它使用标头字段到值的手动"映射.我不需要/不想重命名标题字段，并且有很多.我也不想每次布局更改时都必须更改脚本/命令.

I found this article which shows what I'm trying to do, but it uses a 'manual' mapping of the header fields to the values. I don't need/want to rename the header fields and have quite a few of them. I would also not want to have to change a script/command every time the layout changes.

是否可以动态提取标头，然后使用jq单线将它们与值组合?

Is it possible to dynamically extract the headers and then combine them with the values with a jq one-liner?

推荐答案

简而言之-是的，除了单线位.

In short - yes, except maybe for the one-liner bit.

jq通常非常适合文本拼写，对于具有正则表达式支持的版本尤其如此.例如，有了正则表达式支持，给定问题语句所需的修整就变得微不足道了.

jq is often well-suited to text wrangling, and this is especially true of versions with regex support. With regex support, for example, the trimming required by the given problem statement is trivial.

由于jq 1.5rc1包含正则表达式支持并且自2015年1月1日开始可用，因此以下程序假定jq 1.5为版本；如果您希望使其与jq 1.4一起使用，请参阅两个对于jq 1.4"注释.

Since jq 1.5rc1 includes regex support and has been available since Jan 1, 2015, the following program assumes a version of jq 1.5; if you wish to make it work with jq 1.4, then see the two "For jq 1.4" comments.

也请注意，该程序不能处理CSV的所有一般性和复杂性. (有关更通用地处理CSV的类似方法，请参见 https://github.com/stedolan/jq/wiki/Cookbook#convert-a-csv-file-with-headers-to-json )

Please also note that this program does not handle CSV in all its generality and complexity. (For a similar approach that does handle CSV more generally, see https://github.com/stedolan/jq/wiki/Cookbook#convert-a-csv-file-with-headers-to-json)

# objectify/1 takes an array of string values as inputs, converts
# numeric values to numbers, and packages the results into an object
# with keys specified by the "headers" array
def objectify(headers):
  # For jq 1.4, replace the following line by: def tonumberq: .;
  def tonumberq: tonumber? // .;
  . as $in
  | reduce range(0; headers|length) as $i ({}; .[headers[$i]] = ($in[$i] | tonumberq) );

def csv2table:
  # For jq 1.4, replace the following line by:  def trim: .;
  def trim: sub("^ +";"") |  sub(" +$";"");
  split("\n") | map( split(",") | map(trim) );

def csv2json:
  csv2table
  | .[0] as $headers
  | reduce (.[1:][] | select(length > 0) ) as $row
      ( []; . + [ $row|objectify($headers) ]);

csv2json

示例(假设csv.csv是给定的CSV文本文件):

Example (assuming csv.csv is the given CSV text file):

$ jq -R -s -f csv2json.jq csv.csv
[
  {
    "name": "john",
    "age": 20,
    "gender": "male"
  },
  {
    "name": "jane",
    "age": 30,
    "gender": "female"
  },
  {
    "name": "bob",
    "age": 25,
    "gender": "male"
  }
]

这篇关于使用jq将CSV转换为JSON的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持！