本文介绍了通过bash脚本获取域头的处理方法,对大家解决问题具有一定的参考价值,需要的朋友们下面随着小编来一起学习吧!
问题描述
我在test.csv中编写了10个域,并试图获取这10个域的标头.但不会一一读取内容.
I write 10 domains in test.csv and trying to get the header of these 10 domains. but it won't read contents 1 by 1.
并运行该脚本
for j in test.csv
do
awk -F',' '{ print "$1" }' $j | curl -Is | cat >> b.txt
done
我大约有1000万个域,并尝试通过脚本获取标头.有可能吗?
I have about 10 million domains, and trying to get header by script. Any way possible ?
推荐答案
curl
不会告诉您来自stdin的URL,除非您告知这样做.而且您不需要为此进行循环(假设您有多个文件).您正在寻找的是:
curl
won't read URLs from stdin unless you tell it to do so. And you don't need a loop for this (assuming you have multiple files). What you're looking for is:
awk -F',' '{ print "url=" $1 }' file1 file2 file3 ... | curl -s -I -K- > out
注意-K-
,它告诉curl
从stdin读取命令行参数.
Notice the -K-
, it tells curl
to read command line arguments from stdin.
这篇关于通过bash脚本获取域头的文章就介绍到这了,希望我们推荐的答案对大家有所帮助,也希望大家多多支持!