Miskatonic University Press

Adventures in /usr/bin and the likes, a blog post by Alexander Blagoev, is full of all sorts of very useful short examples of Unix/Linux command line tools. It mentions cut, which I use a lot, and paste, which I don’t but will remember now, Building on the Unix philosophy, here’s a way to put them together with bc to sum a column of numbers in a CSV file, which is something I often want to do.

Let’s say we have expenses.csv, a list of things a librarian has been spending money on:

date,category,amount
2017-02-24,books,12.99
2017-02-25,cardigans,50.00
2017-02-27,books,18.50
2017-02-27,catfood,24.50

We want to add up what’s the amount column. And we don’t want to use awk or anything more complex, because I never remember how AWK works. (If I’m going to use a scripting language I’d use R or Ruby here, but this is about keeping it to basic shell commands.)

It’s easy to use cut to pick out the amount column:

$ cut -d, -f3 expenses.csv
amount
12.99
50.00
18.50
24.50

We don’t want the column header there, so we need to grep it out.

$ cut -d, -f3 expenses.csv | grep -v amount
12.99
50.00
18.50
24.50

Now we can use paste to turn that column into one line, with a delimiter (this is taken from a Stack Overflow answer, as so many things are):

$ cut -d, -f3 expenses.csv | grep -v amount | paste -s -d+
12.99+50.00+18.50+24.50

That looks like arithmetic, and we can pipe it into bc, a calculator that does a lot more than just adding up numbers, but that’s all we need. The -- is the common way to tell it to read its input from STDIN:

$ cut -d, -f3 expenses.csv | grep -v amount | paste -s -d+ | bc --
105.99