4.shell编程-文本处理三剑客之sed

4.1.sed的选项

 sed，流编辑器。对标准输出或文件进行逐行处理。

语法格式

• 第一种：stdout | sed [option] "pattern command"
• 第二种：sed [option] "pattern command" file

 选项

• -n    只打印模式匹配行
• -e    直接在命令行进行sed编辑，默认选项
• -f    编辑动作保存在文件中，指定文件执行
• -r    支持扩展正则表达式
• -i    直接修改文件内容

实例

[root@VM_0_9_centos shell_learn]# cat test.txt 
I love python
I love PYTHON
Hadoop is bigdata frame
[root@VM_0_9_centos shell_learn]# sed -n 'p' test.txt 
I love python
I love PYTHON
Hadoop is bigdata frame
[root@VM_0_9_centos shell_learn]# sed -n '/python/p' test.txt 
I love python
[root@VM_0_9_centos shell_learn]# sed -n -e '/python/p' -e '/PYTHON/p' test.txt 
I love python
I love PYTHON
[root@VM_0_9_centos shell_learn]#

-f 选项，把编辑动作放到文本中

[root@VM_0_9_centos shell_learn]# cat edit.txt 
/python/p
[root@VM_0_9_centos shell_learn]# sed -n -f edit.txt test.txt 
I love python
[root@VM_0_9_centos shell_learn]#

 -i   修改

sed -i 's/love/like/g' test.txt

4.2.sed中的pattern详解

pattern用发表

（1）直接指定行号

[root@VM_0_9_centos shell_learn]# sed -n '17p' /etc/passwd
dbus:x:81:81:System message bus:/:/sbin/nologin

（2）指定起始行号和结束行号

[root@VM_0_9_centos shell_learn]# sed -n '10,13p' /etc/passwd
operator:x:11:0:operator:/root:/sbin/nologin
games:x:12:100:games:/usr/games:/sbin/nologin
ftp:x:14:50:FTP User:/var/ftp:/sbin/nologin
nobody:x:99:99:Nobody:/:/sbin/nologin
[root@VM_0_9_centos shell_learn]# 

（3）指定起始行号，然后后面N行

[root@VM_0_9_centos shell_learn]# sed -n '10,+5p' /etc/passwd
operator:x:11:0:operator:/root:/sbin/nologin
games:x:12:100:games:/usr/games:/sbin/nologin
ftp:x:14:50:FTP User:/var/ftp:/sbin/nologin
nobody:x:99:99:Nobody:/:/sbin/nologin
avahi-autoipd:x:170:170:Avahi IPv4LL Stack:/var/lib/avahi-autoipd:/sbin/nologin
systemd-bus-proxy:x:999:997:systemd Bus Proxy:/:/sbin/nologin
[root@VM_0_9_centos shell_learn]# 

（4）/pattern1/    正则表达式匹配的行

[root@VM_0_9_centos shell_learn]# sed -n '/derek/p' /etc/passwd
derektest:x:1001:1001::/home/derektest:/bin/bash
derek:x:1002:1002::/home/derek:/bin/bash
[root@VM_0_9_centos shell_learn]# 

（5）/pattern1/,/pattern2/

[root@VM_0_9_centos shell_learn]# sed -n '/nginx/,/derek/p' /etc/passwd
nginx:x:993:991:Nginx web server:/var/lib/nginx:/sbin/nologin
memcached:x:992:990:Memcached daemon:/run/memcached:/sbin/nologin
redis:x:991:989:Redis Database Server:/var/lib/redis:/sbin/nologin
derektest:x:1001:1001::/home/derektest:/bin/bash
[root@VM_0_9_centos shell_learn]# 

（6）linenumber,/pattern1/

从第30行开始匹配，直到匹配到derek行结尾

[root@VM_0_9_centos shell_learn]# sed -n '30,/derek/p' /etc/passwd
mysql:x:27:27:MySQL Server:/var/lib/mysql:/bin/bash
nginx:x:993:991:Nginx web server:/var/lib/nginx:/sbin/nologin
memcached:x:992:990:Memcached daemon:/run/memcached:/sbin/nologin
redis:x:991:989:Redis Database Server:/var/lib/redis:/sbin/nologin
derektest:x:1001:1001::/home/derektest:/bin/bash
[root@VM_0_9_centos shell_learn]#

（7）/pattern1/,linenumber

从nginx行开始到35行结束

[root@VM_0_9_centos shell_learn]# sed -n '/nginx/,35p' /etc/passwd
nginx:x:993:991:Nginx web server:/var/lib/nginx:/sbin/nologin
memcached:x:992:990:Memcached daemon:/run/memcached:/sbin/nologin
redis:x:991:989:Redis Database Server:/var/lib/redis:/sbin/nologin
derektest:x:1001:1001::/home/derektest:/bin/bash
derek:x:1002:1002::/home/derek:/bin/bash
[root@VM_0_9_centos shell_learn]# 

4.3.sed中的删除

（1）p

查询

sed -n '1p' test.txt

（2）d  删除

删除1~3行

sed -i '1,3d' test.txt

删除以 ‘’Beau‘’开头，到以“Simp”开头，中间所有的行

test.txt

[root@VM_0_9_centos shell_learn]# cat test.txt
Beautiful is better than ugly.
Explicit is better than implicit.
Simple is better than complex.
Complex is better than complicated.
Flat is better than nested.
Sparse is better than dense.
Readability counts.
[root@VM_0_9_centos shell_learn]#

sed -i '/^Beau/,/^Simp/d' test.txt

[root@VM_0_9_centos shell_learn]# cat test.txt 
Complex is better than complicated.
Flat is better than nested.
Sparse is better than dense.
Readability counts

4.4.sed中的增加

（3）a

匹配到的行后追加内容

[root@VM_0_9_centos shell_learn]# cat test.txt 
Complex is better than complicated.
Flat is better than nested.
Sparse is better than dense.
Readability counts.
[root@VM_0_9_centos shell_learn]# sed -i '/Flat/a zhang-derek' test.txt 
[root@VM_0_9_centos shell_learn]#
[root@VM_0_9_centos shell_learn]# cat test.txt 
Complex is better than complicated.
Flat is better than nested.
zhang-derek
Sparse is better than dense.
Readability counts.
[root@VM_0_9_centos shell_learn]# 

（4）i

匹配到的行前追加内容

[root@VM_0_9_centos shell_learn]# cat test.txt 
Complex is better than complicated.
Flat is better than nested.
Sparse is better than dense.
Readability counts.
[root@VM_0_9_centos shell_learn]# sed -i '/Flat/i zhang-derek' test.txt 
[root@VM_0_9_centos shell_learn]#
[root@VM_0_9_centos shell_learn]# cat test.txt 
Complex is better than complicated.
zhang-derek
Flat is better than nested.
Sparse is better than dense.
Readability counts.
[root@VM_0_9_centos shell_learn]# 

（5）r

将后面指定文件的内容追加到爬匹配的行后面

[root@VM_0_9_centos shell_learn]# cat list.txt 
xxxxxxxxxxx
yyyyyyyyyyy
[root@VM_0_9_centos shell_learn]# cat test.txt 
Complex is better than complicated.
Flat is better than nested.
Sparse is better than dense.
Readability counts.
[root@VM_0_9_centos shell_learn]# sed -i '/Flat/r list.txt' test.txt 
[root@VM_0_9_centos shell_learn]#
[root@VM_0_9_centos shell_learn]# cat test.txt 
Complex is better than complicated.
Flat is better than nested.
xxxxxxxxxxx
yyyyyyyyyyy
Sparse is better than dense.
Readability counts.
[root@VM_0_9_centos shell_learn]#

（6）w

将匹配到的行内容另存到其它文件

[root@VM_0_9_centos shell_learn]# cat test.txt 
Complex is better than complicated.
Flat is better than nested.
xxxxxxxxxxx
yyyyyyyyyyy
Sparse is better than dense.
Readability counts.
[root@VM_0_9_centos shell_learn]# touch 1.txt
[root@VM_0_9_centos shell_learn]# sed -i '/^xxx/,/^yyy/w 1.txt' test.txt 
[root@VM_0_9_centos shell_learn]#
[root@VM_0_9_centos shell_learn]# cat 1.txt 
xxxxxxxxxxx
yyyyyyyyyyy
[root@VM_0_9_centos shell_learn]# 

4.5.sed中的修改

• s/pattern/string         只替换一行中的第一个
• s/pattern/string/g      全部行内全部替换
• s/pattern/string/ig      全部替换，并且不区分大小写 

 实例

[root@VM_0_9_centos shell_learn]# cat 2.txt 
i like python
i like english
I like django
I like flask,flask,flask
[root@VM_0_9_centos shell_learn]# sed -i 's/flask/Flask/g' 2.txt 
[root@VM_0_9_centos shell_learn]#
[root@VM_0_9_centos shell_learn]# cat 2.txt 
i like python
i like english
I like django
I like Flask,Flask,Flask

4.6.反向引用

实例一

[root@VM_0_9_centos shell_learn]# cat 3.txt 
hadAAp
hadBBp
hadCCp
hadDDp
[root@VM_0_9_centos shell_learn]# sed -i 's/had..p/&ss/g' 3.txt 
[root@VM_0_9_centos shell_learn]#
[root@VM_0_9_centos shell_learn]# cat 3.txt 
hadAApss
hadBBpss
hadCCpss
hadDDpss
[root@VM_0_9_centos shell_learn]#

说明：“&”表示前面匹配到的内容，结果就是在匹配到的所有内容后面加上“ss”

实例二

[root@VM_0_9_centos shell_learn]# cat 3.txt 
hadAApss
hadBBpss
hadCCpss
hadDDpss
[root@VM_0_9_centos shell_learn]# sed -i 's/\(had\)...../\1derek/g' 3.txt 
[root@VM_0_9_centos shell_learn]#
[root@VM_0_9_centos shell_learn]# cat 3.txt 
hadderek
hadderek
hadderek
hadderek
[root@VM_0_9_centos shell_learn]#

说明：“\1”和“&”的区别是“\1”可以反向引用匹配到的内容的一部分，然后对其修改，“&”只能对匹配的内容整体修改，不能拆分