Unix tip: Arranging data and running commands with xargs

The xargs command is one that many Unix users take advantage of in only some familiar context, such as using it along with a find command to do something with the particular files once you find them. But it can used for doing a lot of other things on the command line that might surprise you.

Xargs is generally used to take input generated by some other commands and then using it to create and run other commands. This can be a big time saver. When used with the find command, for example, xargs allows you to create commands to manipulate the found files in some way. You might want to find all files in the certain folder that end with .pl or .sh and make them executable. Or you might want to remove all files that end in .old. These tasks are very easy to do with an xargs command appended to your find command. Here are some simple examples:

$ find . -name "*.pl" | xargs chmod 750
$ find . -name "*.old" | xargs rm

In the command below, we’re looking for all C source code files that refer to the standard IO library, stdio.h

$ find . -name '*.c' | xargs grep 'stdio.h'
./libcaca-0.99.beta19/src/aafire.c:#   include 
./libcaca-0.99.beta19/src/aafire.c:#include 
./libcaca-0.99.beta19/src/cacaclock.c:#   include 
./libcaca-0.99.beta19/src/cacaserver.c:#include 
./libcaca-0.99.beta19/src/cacademo.c:#   include 
./libcaca-0.99.beta19/src/cacadraw.c:#   include 

These are all fairly common use of xargs. At the same time, xargs has many other uses that make it considerably more versatile. Let’s take a closer look at how it works and what it can do.

First, the xarg command’s basic function is pretty simple. It echoes what you toss at it. In a sense, it’s not very different from echo, except that content needs to be redirected to it, not presented as an argument, or it can be provided after you type xargs and before you press control-d. First, the most simple form.

$ echo a b c
a b c
$ echo a b c | xargs
a b c

Obviously not much is gained in the example above. The pipe and the use of xargs are just making the command more complex and a bit more time-consuming. But the command illustrates how xargs just takes each string that is passed to it and repeats it.

In the example below, we just type xargs and then hit enter. Then we type some text. I typed control-d after the text and pressing the enter key. You won’t see the control-d (^d), of course, but I included it below to show where it was entered. The xargs command displays the text again. You could enter as many lines of text as you cared to and all would be repeated after the control-d.

$ xargs
Hello, world
Goodbye then!
^dHello, world Goodbye then!

In a similar fashion, if you need to have the contents of a simple file show up on a single line, basically flattening the file, xargs will comply with your wishes.

First, the file:

$ cat data
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15

Then the flattening of its contents with xargs:

$ cat data | xargs
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15

You can also view your data by squeezing a given number of lines together. In this example, we’re displaying every 5 lines from the file as a single line.

$ cat data | xargs -l5
1 2 3 4 5
6 7 8 9 10
11 12 13 14 15

And this can work with numbers generated by brace expansion as well. In this example, we’re using the echo command and the range of values expressed in the braces to feed the numbers between 1 and 100 to xargs and then grouping them five at a time.

$ echo {1..100} | xargs -n 5
1 2 3 4 5
6 7 8 9 10
11 12 13 14 15
16 17 18 19 20
21 22 23 24 25
26 27 28 29 30
31 32 33 34 35
36 37 38 39 40
41 42 43 44 45
46 47 48 49 50
51 52 53 54 55
56 57 58 59 60
61 62 63 64 65
66 67 68 69 70
71 72 73 74 75
76 77 78 79 80
81 82 83 84 85
86 87 88 89 90
91 92 93 94 95
96 97 98 99 100

And in this last command, we’re adding each set of three lines together from the data file together and then displaying the sums.

$ cat data|tr ' ' 'n'|xargs -l3|tr ' ' '+'|bc
6
15
24
33
42

The xargs command can be handy for a number of data manipulation tasks. While its primarily used to construct new commands so that you don’t have to do that by hand, it also allows you to do some unusual rearrangements of data that might just come in handy.