Mar 212014
 

I’m a big fan of using MultiSSH for performing quick hacks on multiple hosts at once. ClusterSSH is nice too. I hate myself for using it, because frankly any form of parallel execution should really use a formal tool, something like Capistrano, but there are times when quick and dirty is what’s required.

The problem is, if the parallel commands you’re executing are all hitting the same network resource at exactly the same time, you can get timeouts, failures, or possibly even look up the daemon. I’m looking at you, Puppet (sometimes “puppetd -t –splay” just doesn’t cut it).

What I like to do to prevent everything being too simultaneous is to precede the command with a random sleep pause, like this:

sleep $(( $RANDOM*8/1000 )); puppetd -t --noop

This will randomly sleep for any time between 1 and 8 seconds before continuing with the Puppet agent update, saving the Puppetmaster and its precious thread count.

This could similiarly be used during YUM or APT updates, but really, if you have to, you should probably be automating these kinds of solutions.


Matt Parsons is a freelance Linux specialist who has designed, built and supported Unix and Linux systems in the finance, telecommunications and media industries.

He lives and works in London.

Sep 182013
 

Indenting lines with leading tabs is an important technique for writing clear and readable code, and are essential in Python where they have syntactic meaning. Many IDEs insert tabs automatically, but if you’re using vi it can be tedious to insert tabs individually. Fortunately, there is an easy shortcut, and it’s a very handy command to remember.

In the vi session, enter command mode by pressing ESC. Then, set the size of the tabs with the following command:

  :set shiftwidth=2

To make this the default, put “set shiftwidth=2” in your ~/.exrc file.

Then, to tab shift the next 13 lines, you’d type a command like this:

13>>


Matt Parsons is a freelance Linux specialist who has designed, built and supported Unix and Linux systems in the finance, telecommunications and media industries.

He lives and works in London.

Jul 302013
 

The IP address that a host presents to the Internet is probably not the same one on its network interface, thanks to the magic of NAT. The difficulty of obtaining this address comes from the fact that one must effectively “ask” an outside source for the answer, since the host and internal network is not in charge of allocating this address, and therefore doesn’t know it. Fortunately, this can be accomplished easily with a simple command line.

One can use wget or curl for this (this example uses wget) and the external site ipecho.net, which offers an unformatted page displaying the IP address.

   wget -q -O - http://ipecho.net/plain && echo

This expression could be executed inside a subshell and the result written to a variable. From here, this information could be useful for checking that a VPN client is connected.


Matt Parsons is a freelance Linux specialist who has designed, built and supported Unix and Linux systems in the finance, telecommunications and media industries.

He lives and works in London.

Jun 242013
 

As a freelance tech contractor, I get a lot of emails every day from recruiters about prospective jobs. Many of these are unsolicited, but this is a good thing, as over time, quite a large list of contacts can be built up. I don’t have the time to reply to each one – particularly those that don’t suit my skillset, or if I’m happily indentured – but I do file these emails into a separate Gmail Mailbox which I’ve labelled “Employment”.

I’m not going to add each recruiter to my Contacts as it arrives – that would be time consuming to start and difficult to maintain. But when it comes time to make a great big recruiter contact list, I’ve written the Python script below to scrap the entire mailbox and output each unique email address.

#!/usr/bin/python

import imaplib
import sys
import email
import re

#FOLDER=sys.argv[1]
FOLDER='Employment'
LOGIN='example.address@gmail.com'
PASSWORD='xxxxxxxxxxxx'
IMAP_HOST = 'imap.gmail.com'  # Change this according to your provider

email_list = []
email_unique = []

mail = imaplib.IMAP4_SSL(IMAP_HOST)
mail.login(LOGIN, PASSWORD)
mail.select(FOLDER) 

result, data = mail.search(None, 'ALL')
ids = data[0]
id_list = ids.split()
for i in id_list:
	typ, data = mail.fetch(i,'(RFC822)')
	for response_part in data:
		if isinstance(response_part, tuple):
			msg = email.message_from_string(response_part[1])
			sender = msg['from'].split()[-1]
			address = re.sub(r'[<>]','',sender)
# Ignore any occurences of own email address and add to list
	if not re.search(r'' + re.escape(LOGIN),address) and not address in email_list:
		email_list.append(address)
		print address

I’ve hard-coded my email login, password and the mailbox name, although it’s easy enough to modify the script to enter them as argmuents (I’ve commented out a line demonstrating this).

In a later post, I’m going to discuss how I use this script for job-seeking.


Matt Parsons is a freelance Linux specialist who has designed, built and supported Unix and Linux systems in the finance, telecommunications and media industries.

He lives and works in London.