Joining lines of text on Linux

There are number of ways to join multiple lines of text and change delimiters if needed. This article shows two of the easier ways to do this and explains the commands.

Using the tr command

The tr command is quite versatile. It’s used to make many types of changes to text files, but it can also flatten multiple lines into one by replacing newline characters with blanks. It does, however, remove the final newline as well. Note the $ prompt at the end of the second line. That’s a clue!

$ tr 'n' ' '  newfile

To fix this problem, you can add a newline to the end of the file with an echo command like this:

$ echo "" >> newfile
$ od -bc newfile
0000000 124 150 151 163 040 151 163 040 141 040 146 151 154 145 040 164
          T   h   i   s       i   s       a       f   i   l   e       t
0000020 150 141 164 040 111 040 143 141 156 040 165 163 145 040 146 157
          h   a   t       I       c   a   n       u   s   e       f   o
0000040 162 040 164 145 163 164 151 156 147 056 040 012
          r       t   e   s   t   i   n   g   .      n	 

Using the paste command

The paste command was developed solely for the purpose of merging the lines in a file into a single line. So this command is a great choice for flattening multi-line text files. If we start with a file like the one below, the result will look like what is shown at the bottom.

$ cat testfile
This is
a file that
I can
use
for testing.

$ paste -sd ' ' testfile
This is a file that I can use for testing.

Redirect the output to a second file to save the flattened text. It will include a newline at the end, so nothing more will need to be done.

$ paste -sd ' ' testfile > testfile2

The -s (use a single file) and -d (add a delimiter between each of the processed lines) in the command shown above will ensure that the joined lines are all separated by blanks. You can, however, use whatever delimiter is right for the task.

If you add multiple file names with the paste command, each file will be turned into a single line in the output.

$ paste -sd ' ' testfile testfile2
This is a file that I can use for testing.
This is a second file.

To save the result, redirect the output to a third file.

$ paste -sd ' ' testfile testfile2 > newfile
$ cat newfile
This is a file that I can use for testing.
This is a second file.

To turn only the first three lines of a file into a single line, add a head command like this.

$ head -3 testfile3 | paste -sd ' '
This is a file that I can

If you want to turn every consecutive group of five lines in a file into a separate line, you have to work a little harder. This script joins five lines at a time into one by putting those lines to a separate file and then flattening it.

#!/bin/bash

bottom=5

echo -n "file> "
read file
len=`wc -l $file | awk '{print $1}'`

while [ $bottom -le $len ]; do
    head -$bottom $file | tail -5 > temp
    paste -sd ' ' temp
    bottom=`expr $bottom + 5`
done

# join any remaining lines
remaining=`expr $len - $bottom + 5`
if [ "$remaining" -gt 0 ]; then
    tail -$remaining $file | paste -sd ' '
fi

Here’s an example of running it:

$ cat biggerfile
This
is
a
bigger
file.
Let's
turn it
into
fewer
lines.

$ joinlines
file> biggerfile
This is a bigger file.
Let's turn it into fewer lines.

You could easily adjust the number of lines that are sequentially joined together by modifying the script or make it even more versatile by prompting the user to enter the number of lines to join together. Here’s a script with those changes:

#!/bin/bash

echo -n "file> "
read file
len=`wc -l $file | awk '{print $1}'`

echo -n "Number of lines to join> "
read lines
bottom=$lines

while [ $bottom -le $len ]; do
    head -$bottom $file | tail -$lines > temp
    paste -sd ' ' temp
    bottom=`expr $bottom + $lines`
done

# join any remaining lines
remaining=`expr $len - $bottom + $lines`
if [ "$remaining" -gt 0 ]; then
    tail -$remaining $file | paste -sd ' '
fi

Here’s an example of running the script:

$ joinlines2
file> biggerfile
Number of lines to join> 3
This is a
bigger file. Let's
turn it into fewer
lines.

Wrap-up

Linux provides quite a few commands that can combine multiple lines of text into one. The paste command makes the job surprisingly easy. In fact, if you expect to use the paste -sd ‘ ‘ command often, you could make the task even easier by turning it into an alias like this:

$ alias paste="paste -sd ' '"
$ paste testfile
This is a file that I can use for testing.