May 292013
 

There are times when debugging system behaviour when it can be very useful to see what the inputs and/or outputs of a process are: to read its STDIN and STDOUT filehandles. It can be handy to know what’s being printed to an unseen screen, particularly if there is no log file, or it may be handy to snoop on what a user is typing. This is a yet another way of using the Swiss Army knife that is strace.

In its most basic form, with few options, strace can print out every system call that a process makes, and if the “-f” or “-ff” switches are used, also print out all the system calls from the child processes that the process forks. Like this:

  # strace -f -p <PID>

This provides far too much information though, so it’s often desirable to run strace with filter expressions to narrow the search down, specified with the “-e” switch. The man page provides an exhaustive list of these filters.

To specify a filter for reading STDIN, STDOUT and STDERR, the following expression will do the trick:

  # strace -ff -e trace=write -e write=1,2 -p <PID>

Make use of this how you will.


Matt Parsons is a freelance Linux specialist who has designed, built and supported Unix and Linux systems in the finance, telecommunications and media industries.

He lives and works in London.

May 272013
 

Bash is the de facto shell these days, and for good reason. Most of us who peck away at our shells all day become pretty good with Bash’s keyboard shortcuts, but there’s one handy and little known shortcut that’s worth mentioning – how to recall the first word from the previous line only.

You’re probably used to pulling up the previous Bash line with the “Up” arrow. You probably also frequently substitute the last word from the previous line with “Ctrl-Underscore”. But next time you need to execute the previous command, but with a different argument, rather than hitting “Up”, deleting the last argument and retyping the new argument, try using !:- in place of the first word (that is, the command).

So if the previous command was:

   # /opt/splunk/bin/splunk start

And you next wanted to execute it again, but with a “stop” argument, do this:

   # !:- stop

The Bash shell will substitute !:- with the text from the previous command. No grabbing for the mouse and copying and pasting. Remember, according to Larry Wall, creator of Perl, Laziness is the second virtue of the great programmer.


Matt Parsons is a freelance Linux specialist who has designed, built and supported Unix and Linux systems in the finance, telecommunications and media industries.

He lives and works in London.

May 222013
 

I needed to maintain two separate SVN repositories that should have ostensibly been the same, but which had a few minor differences. Most of these were differences trivial, but all needed to be checked individually and resolved on a case by case basis. This could have been a complicated proposition, but fortunately, the “diff” command is recursive, and I came up with a little hack of a script to further simplify the process.

The “diff” command is a handy utility that reports the differences between files, on a line by line basis. It has a lot of switches which control the amount of output, and the man page is a fine resource for this. Two useful ones are “-u” which simplifies or “unifies” the output (try it with and without to see the difference), and as mentioned above, “-r” which makes the command recursive, comparing files within subsequent subdirectories as well.

The problem is, doing a recursive diff on two directories can produce massive amounts of output. If the aim of the exercise is to resolve the differences completely, you’d just do a straight copy, but more often than not, it’s to root out a few differences or make some slight modifications. The trick is to weed out the signal from the noise on subsequent refinements.

One way of doing this (rather than just a “grep -v” on the output which would become unnecessarily unwieldy) is to use the “-X” switch to supply the name of a file which contains filenames (or patterns of filenames) to exclude from the check. Better yet, write a script to execute the diff command, submitting the script itself as the argument to the “-X” switch. In the script, list the filename patterns after the “diff” invocation, like this:

Example script:
recursive-diff.sh

diff -X $0 -ur dropbear/puppet/ bunyip/puppet/
exit

.svn
amanda.conf
icinga
backup.sh
ldap
fileserver.conf
site.pp

The “$0” argument means that the executing filename will be substituted. And the “exit” command means that nothing following it in the script will be executed.

To use this script, start with nothing after “exit”, then as you explore and resolve inconsistencies between files, exclude any false positives by listing the filename at the end of the script. Re-run the script, and repeat until the differences have been resolved or ignored.

This is one of those long, drawn-out processes, but there are some few things that just can’t be automated.


Matt Parsons is a freelance Linux specialist who has designed, built and supported Unix and Linux systems in the finance, telecommunications and media industries.

He lives and works in London.

May 092013
 

It’s a common task when writing shell scripts to need to source a set of environment variables from an external configuration file. The advantage of this is that the dynamic variables can be abstracted from the script itself, which can then be more securely version controlled. In shell scripts, it’s a simple matter of assigning values to variables in a literal shell fashion, and then sourcing the config file from within the executable script with the dot “.” operator. Perl scripts will not accept this syntax, but it can often be useful to re-use shell config files. Here’s a simple and intuitive method to do so.

Short Answer

Iterate over each line in the shell config file and parse it like this:

      $$1 = $2 if /^([^=]+)=(.+)$/;

Long Example

In this example there is a file called “auth.cfg” which contains a username and password, assigned to the variables USER and PASSWORD respectively:

auth.cfg:
USER="backup"
PASSWD="password123"

The code displayed in the “short answer” above can be used an readably expanded to this:

   #!/usr/bin/perl

   my $cfgFile = "auth.cfg";
   open(CFG,"<$cfgFile") || die "Cannot open config file $cfgFile\n";
   while() {
      if (/^([^=]+)=(.+)$/) {
         $$1 = $2;     # Take the variable value as the literal name of a variable
      }
   }
   print "USER = $USER\n";
   print "PASSWD = $PASSWD\n";

The only catch is that the config file must accurately specify the name of the variable that the script expects (in “$$1”), otherwise the variable will be unassigned and the script’s behaviour could be unpredictable. Some form of error checking is therefore advised, but this will be left as an exercise to the reader.


Matt Parsons is a freelance Linux specialist who has designed, built and supported Unix and Linux systems in the finance, telecommunications and media industries.

He lives and works in London.