How to remove empty/blank lines from a file in Unix (including spaces)?

Question

How do I remove empty/blank (including spaces only) lines in a file in Unix/Linux using the command line?

contents of file.txt

Line:Text
1:<blank>
2:AAA
3:<blank>
4:BBB
5:<blank>
6:<space><space><space>CCC
7:<space><space>
8:DDD

output desired

1:AAA
2:BBB
3:<space><space><space>CCC
4:DDD

For awk, see: [Remove blank lines in awk](http://unix.stackexchange.com/q/128328/21471), or [using grep](http://stackoverflow.com/q/1611809/55075), in general, see: [How to remove blank lines from a file in shell?](http://unix.stackexchange.com/q/101440/21471) — kenorb, May 05 '15 at 16:11
[same question and same answer given over here but with much more votes](https://stackoverflow.com/a/16414489/52074) — Trevor Boyd Smith, Feb 21 '18 at 19:18
This is an epic answer that pretty much gives every possible solution using grep, sed, awk: https://stackoverflow.com/questions/16414410/delete-empty-lines-using-sed/39139322#39139322 — wisbucky, Apr 25 '19 at 23:03

score 166 · Answer 1 · edited Mar 30 '19 at 23:36

166

This sed line should do the trick:

sed -i '/^$/d' file.txt

The -i means it will edit the file in-place.

edited Mar 30 '19 at 23:36

Eddie C.

487
1
3
12

answered Mar 28 '11 at 22:19

Martijn Heemels

7,438
6
39
62

24

It actually needs to be "/^ *$/d" to remove lines that only contain spaces. – Sean Reifschneider Mar 28 '11 at 23:36
3

@SeanReifschneider That requirement was not in the question when this answer was written? – kasperd Apr 18 '16 at 10:02
7

@SeanReifschneider Would "/^\s*$/d" not be better as it would include tabs? Whilst it's not mentioned in the original post, it seems a stronger option to me. – mrswadge Sep 20 '17 at 12:57
2

[adding the `\s*` is the same solution given by stackoverflow question](https://stackoverflow.com/a/16414489/52074) – Trevor Boyd Smith Feb 21 '18 at 19:17
I get `bad flag in substitute command: 'e'` – ishandutta2007 Apr 13 '19 at 02:12
This command is not compatible with macOS/BSD, secondly it does not produce the same output as OP requested (it produces 5 lines, not 4). – kenorb Jul 22 '19 at 09:30

kenorb · Answer 2 · 2019-07-22T09:54:43.540

82

`grep`

Simple solution is by using grep (GNU or BSD) command as below.

Remove blank lines (not including lines with spaces).
```
grep . file.txt
```
Remove completely blank lines (including lines with spaces).
```
grep "\S" file.txt
```

Note: If you get unwanted colors, that means your grep is aliases to grep --color=auto (check by type grep). In that case, you can add --color=none parameter, or just run the command as \grep (which ignores the alias).

`ripgrep`

Similar with ripgrep (suitable for much larger files).

Remove blank lines not including lines with spaces:

rg -N . file.txt

or including lines with spaces:

rg -N "\S" file.txt

See also:

How to remove blank lines from a file (including tab and spaces)?
With sed: Delete empty lines using sed
With awk: Remove blank lines using awk

edited Jul 22 '19 at 09:54

answered May 05 '15 at 16:12

kenorb

5,943
1
44
53

6

`grep .` seems to be the simplest solution. – Leo Mar 21 '18 at 21:36
The downside of `grep .` compared to the other solutions is that it will highlight all the text in red. The other solutions can preserve the original colors. Compare `unbuffer apt search foo | grep .` to `unbuffer apt search foo | grep -v ^$` – wisbucky Apr 25 '19 at 23:09
3

@wisbucky You see colors, because `grep` is aliased to `grep --color=auto` on your system (check by: `type grep`). You can run it as `\grep` or use `--color=none` parameter. – kenorb Apr 25 '19 at 23:26
@kenorb If you use `grep --color=none .`, you will get all white text, which overrides the color formatting of the original command (example: `apt search foo`) – wisbucky Apr 26 '19 at 02:01
`grep .` will match lines containing only spaces, which the OP says is not desired. – Jim L. Jul 19 '19 at 21:53
@JimL. A dot (`.`) in `grep` matches any character, so it'll print all non-empty lines. Even the lines with a single empty space are going to be printed. – kenorb Jul 20 '19 at 14:50
Yes, and the OP wishes lines containing only spaces to be removed. – Jim L. Jul 20 '19 at 20:05
@JimL. I've improved answer with the solution removing lines with spaces. – kenorb Jul 22 '19 at 09:08
Actually, I think none of the other answers providing what OP requested (exempt [Steven one](https://serverfault.com/a/915814/130437)). – kenorb Jul 22 '19 at 09:33

score 33 · Answer 3 · answered Mar 28 '11 at 22:18

33

sed '/^$/d' file.txt

d is the sed command to delete a line. ^$ is a regular expression matching only a blank line, a line start followed by a line end.

answered Mar 28 '11 at 22:18

Kamil Kisiel

11,946
7
46
68

This command does not produce the same output as OP requested (it produces 5 lines, not 4). – kenorb Jul 22 '19 at 09:31

score 22 · Answer 4 · answered Mar 28 '11 at 22:20

22

You can use the -v option with grep to remove the matching empty lines.

Like this

grep -Ev "^$" file.txt

answered Mar 28 '11 at 22:20

jdabney

321
1
3

4

I don't believe you need the `-E`, at least not with GNU grep, but apart from that I'm so pleased to see this done with grep! It's what I reach for in preference to sed, every time; in-line filters seem to me to be better than in-line editors. – MadHatter Mar 28 '11 at 22:45
If you want to skip the commented and blank lines, especially while dealing with conf files use `grep -Ev '^#|^$' file.txt` – Govind Kailas Mar 07 '19 at 04:11
`grep (GNU grep) 3.4` requires -E if you are using @GovindKailas' command – Yet Another User Aug 15 '20 at 04:07

score 16 · Answer 5 · edited Jul 22 '19 at 09:34

16

Here is an awk solution:

awk NF file.txt

With Awk, NF only set on non-blank lines. When this condition match, Awk default action is to print the whole line.

edited Jul 22 '19 at 09:34

kenorb

5,943
1
44
53

answered Jun 08 '18 at 16:48

Zombo

1
1
16
18

score 9 · Answer 6 · answered Jul 26 '17 at 10:35

9

To remove empty lines, you could squeeze new line repeats with tr:

cat file.txt | tr -s '\n' '\n'

answered Jul 26 '17 at 10:35

siddhadev

191
1
1

This produces 6 lines, not 4 as OP requested. – kenorb Jul 22 '19 at 09:34

score 1 · Answer 7 · answered Apr 12 '18 at 21:24

Ex/Vim

Here is the method using ex editor (part of Vim):

ex -s +'v/\S/d' -cwq test.txt

For multiple files (edit in-place):

ex -s +'bufdo!v/\S/d' -cxa *.txt

^{Note: The :bufdo command is not POSIX.}

Without modifying the file (just print on the standard output):

cat test.txt | ex -s +'v/\S/d' +%p +q! /dev/stdin

score 1 · Answer 8 · answered Apr 26 '19 at 23:51

1

xargs if you dont mind stripping leading whitespace

$ docker run -it --rm alpine sh
/ # cat <<eof > /tmp/file
> one
>
>   two
> three
>
>
>   four
> eof
/ # cat /tmp/file
one

  two
three


  four
/ # cat /tmp/file | xargs -n1
one
two
three
four

answered Apr 26 '19 at 23:51

christianlc

121
3

score 0 · Answer 9 · answered Apr 13 '19 at 02:15

0

For me @martigin-heemels command was throwing error this fixed it (ie a dummy param to i),

sed -i '' '/^$/d' file.txt

answered Apr 13 '19 at 02:15

ishandutta2007

153
1
6

How to remove empty/blank lines from a file in Unix (including spaces)?

9 Answers9

grep

ripgrep

Ex/Vim

`grep`

`ripgrep`