John Hawthorn

Work, work. Zug zug.

2011-07-16T07:00:00Z

I wrote a little script to block websites which would destract me from work. Inspired by get-shit-done and the Orc voices from Warcraft 2. The script can be called as either workwork or readytowork to enable the block, and workcomplete to unblock the websites.

Warning: Contains a list of websites which will waste your time.

#!/usr/bin/env ruby

SiteList = %w{
  reddit.com
  news.ycombinator.com
  twitter.com
  plus.google.com
  slashdot.org
  cracked.com
  vimeo.com
  youtube.com
  digg.com
}

StartToken = '## workwork'
EndToken = "## zugzug\n"

HostsFile = '/etc/hosts'
ConfigFile = '~/.workwork'

def disabled_sites
  if File.exist?(ConfigFile)
    open(ConfigFile).read.split("\n")
  else
    SiteList
  end
end

def workwork
  lines = []
  lines << StartToken
  disabled_sites.each do |site|
    lines << "127.0.0.1\t#{site}"
    lines << "127.0.0.1\twww.#{site}"
  end
  lines << EndToken

  open(HostsFile, 'a+') do |file|
    file.write lines.join("\n")
  end
end

def workcomplete
  open(HostsFile, 'r+') do |file|
    string = file.read
    string.gsub!(/#{StartToken}.*#{EndToken}/m, '')
    file.seek(0)
    file.truncate(0)
    file.write(string)
  end
end

# switch to root since we need to change /etc/hosts
if ENV["USER"] != "root"
  exec("sudo #{ENV['_']} #{ARGV.join(' ')}")
end

case File.basename($0)
when 'workwork' then
when 'readytowork' then
  workwork
when 'workcomplete' then
  workcomplete
else
  puts "Unknown command"
end

Now I just need a script to remove executable permissions from minecraft.

Dealing with the MySQL slow log on Amazon RDS

2011-07-13T07:00:00Z

Amazon RDS is a web service offered by amazon which provides users with MySQL databases in the cloud. It's an attractive service due to its convenient backups and easy replication.

Amazon doesn't provide shell access to RDS instances. For this reason, the slow query log, which is an invaluable tool for anyone with a somewhat large or complex data set, is not available in the traditional way. Amazon, however, does provide a method for enabling and retrieving the log.

Setup

To enable the slow query log on RDS, you should create or modify a DB parameter group which can be done from the AWS management console (online interface) or through various other tools. There are two attributes that should be changed, slow_query_log, which should be set to 1 to enable, and long_query_time, which should be set to the shortest length to be considered a slow query.

The slow query log is stored in the table mysql.slow_log. Rotating the slow query log is done by calling the stored procedure CALL rds_rotate_slow_log.

Accessing

Update: this script is now available as a gem: rds_slow_log

I've written a little ruby script to print out the slow log in the usual format. From this it can be read, or analyzed with tools like request-log-analyzer.

require 'rubygems'
require 'active_record'

class SlowLog < ActiveRecord::Base
  set_table_name 'slow_log'

  def query_time
    time_parse query_time_before_type_cast
  end
  def lock_time
    time_parse lock_time_before_type_cast
  end
  def to_s
      <<EOS
# Time: #{start_time.strftime('%y%m%d %H:%M:%S')}
# User@Host: #{user_host}
# Query_time: #{query_time}  Lock_time: #{lock_time}  Rows_sent: #{rows_sent}  Rows_examined: #{rows_examined}
#{sql_text};
EOS
  end

  def self.print_all
    find(:all, :order => :start_time).each do |query|
      puts query
    end
  end

  private
  def time_parse string
    hours, minutes, seconds = string.to_s.split(/:/)
    hours.to_i * 60 * 60 + minutes.to_i * 60 + seconds.to_i
  end
end

ActiveRecord::Base.establish_connection :adapter  => 'mysql',
                                        :database => 'mysql',
                                        :username => 'username',
                                        :password => 'password'
SlowLog.print_all

This was written to be compatible with both activerecord 2.3 and 3.0. If compatability with 2.3 were to be dropped, the time_parse hacks could be removed.

Introducing the dkim gem

2011-07-04T07:00:00Z

DKIM (defined in RFC 4871) is a way for email senders to claim responsibility for an email message. It works by adding an extra header to every email sent which cryptographically signs the message's headers and body. By verifying the signature, the receiver of DKIM-signed email can be certain the emails content has been unaltered and comes from who is claimed. By taking responsibility for these messages, the sender's reputation can allow the message to be less scrutinized by spam/phishing filters on the receivers end. For this reason, DKIM is desirable because is should increase deliverability.

I had the need to send a large amount of email through Amazon SES from a rails application. This was made very easy by the terrific fog library for accessing cloud services (I would recommend version 0.8.2 or above, which includes a bug fix I submitted). However I found no good way to DKIM sign email from ruby. There was a rubydkim gem, but it didn't work correctly in my testing, and required an external C library.

This prompted me to create the dkim gem: a simple ruby library for DKIM signing email messages. dkim depends only on ruby built with openssl support (or jruby-openssl if you're using jruby).

Most basic usage (which should be all that is necessary for most) is to configure the gem globally

1
2
3

Dkim::domain      = 'example.com'
Dkim::selector    = 'mail'
Dkim::private_key = open('private.pem').read

and then to sign any messages as needed

mail = <<eos
To: someone@example.com
From: john@example.com
Subject: hi

Howdy
eos

Dkim.sign(mail)
# =>
# To: someone@example.com
# From: john@example.com
# Subject: hi
# DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=example.com; q=dns/txt; s=mail; t=1305917829;
#   bh=qZxwTnSM1ywsrq0Ag9UhQSOtVIG+sW5zDkB+hPbuX08=; h=from:subject:to;
#   b=0mKnNOkxFGiww63Zu4t46J7eZc3Uak3I9km3IH2Le3XcnSNtWJgxiwBX26IZ5yzcT
#   VwJzcCnPKCScIJMQ7yfbfXmNsKVIOV6eSUqu1YvJ1fgzlSAXuDEMNFTjoto5rrdA+
#   BgX849hEY/bWHDl1JJgNpiwtpl4t0Q7M4BVJUd7Lo=
# 
# Howdy

A more detailed explanation can be found in the project's README on github.

I hope the library proves useful to others, and would love to hear from anyone using it.

Minecraft to IRC bridge

2011-06-10T07:00:00Z

Like almost 2.5 million others, my recent productivity has taken a hit due to the game Minecraft. I have a home server set up to play with roommates and friends. Funnily enough, Minecraft turned into a frequent medium of communication for us. To take part in this when not logged into minecraft, and to get notifications to when friends were playing, I set out to make a bridge between minecraft and IRC.

The result is mcirc, a simple minecraft to irc gateway. It tails the minecraft log looking for player joins quits and messages, and relays them to the IRC server. This script requires that there is a FIFO for input to the server.

It is worth noting that this probably not the best minecraft <-> irc bridge which exists, but perhaps the simplest, and runs on a vanilla minecraft server. There are several which function as bukkit plugins: 1 2 3 .

Usage is

1	ruby irc.rb -s irc.freenode.org -c balmoralmc -f minecraft/console.pipe -l minecraft/server.log

The script itself is very small and largely command line parsing

#!/usr/bin/env ruby

require 'socket'
require 'uri'

class MinecraftIrcBot
  def initialize(options)
    uri = URI.parse("irc://#{options[:server]}")
    @channel = options[:channel]
    @socket = TCPSocket.open(uri.host, uri.port || 6667)
    @mclog = IO.popen("tail -f -n0 '#{options[:log]}'", "r")
    @name = options[:name]
    @pipe = options[:pipe]
    say "NICK #{@name}"
    say "USER #{@name} 0 * #{@name}"
    say "JOIN ##{@channel}"
  end

  def say(msg)
    puts msg
    @socket.puts msg
  end

  def say_to_chan(msg)
    say "PRIVMSG ##{@channel} :#{msg}"
  end

  def say_to_minecraft(msg)
    msg = "say #{msg}"
    puts msg

    File.open(@pipe, "w") do |console|
      console.puts msg
    end
  end

  def run
    loop do
      read, write, error = IO.select([@socket, @mclog])
      if read.include? @socket
        msg = @socket.gets

        # connection lost
        return unless msg

        case msg.strip
        when /^PING :(.+)$/i
          say "PONG #{$1}"
        when /^:(.+?)!.+?@.+?\sPRIVMSG\s.+?\s:(.+)$/i
          say_to_minecraft("<#{$1}> #{$2}")
        end
      end

      if read.include? @mclog
        msg = @mclog.gets
        msg.gsub!(/^\d{4}-\d{2}-\d{2} \d{2}:\d{2}:\d{2} /, '')
        case msg.strip
        when /^\[INFO\] ([a-z0-9]*) lost connection/i
          say_to_chan("#{$1} has left")
        when /^\[INFO\] ([a-z0-9]*) \[[^\]]*\] logged in/i
          say_to_chan("#{$1} has joined")
        when /^\[INFO\] <([a-z0-9]*)> (.*)$/i
          say_to_chan("<#{$1}> #{$2}")
        end
      end
    end
  end

  def quit
    say "PART ##{@channel} :bye bye"
    say 'QUIT'
  end
end


def parse
  require 'optparse'
  options = {}
  optparse = OptionParser.new do |opts|
    opts.banner = "Usage: #{$0} options"

    opts.on("-s", "--server SERVER", "IRC server") do |v|
      options[:server] = v
    end
    opts.on("-c", "--channel CHAN", "IRC channel") do |v|
      options[:channel] = v
    end
    opts.on("-f", "--fifo FIFO", "named pipe into minecraft server's console") do |v|
      options[:pipe] = v
    end
    opts.on("-l", "--log LOG", "minecraft servers's log file for reading") do |v|
      options[:log] = v
    end

    options[:name] = "mcirc"
    opts.on("-n", "--name NAME", "name of the irc user") do |v|
      options[:name] = v
    end
  end
  begin
    optparse.parse!

    required = [:server, :channel, :pipe, :log]
    required.each do |arg|
      raise OptionParser::MissingArgument, arg if options[arg].nil?
    end
  rescue OptionParser::InvalidOption, OptionParser::MissingArgument
    puts $!.to_s
    puts optparse
    exit 1
  end
  options
end

def run
  options = parse
  bot = MinecraftIrcBot.new(options)

  trap("INT"){ bot.quit }

  bot.run
end

run

Switching from Jekyll to nanoc

2011-06-10T07:00:00Z

Apologies to anyone who reads my atom feed in their reader of choice. I switched this blog from Jekyll to nanoc over the weekend which caused my old posts to show up as new.

When I started this blog in 2009 I decided on Jekyll, a static site generator. Jekyll is perhaps best known for powering Github Pages, a simple way to get a site at yourname.github.com. Due to a couple small differences, I've grown to prefer nanoc, and decided to switch this blog.

I'm certainly not the first to convert their site from Jekyll to nanoc. It was pretty straightforward, as both programs are fairly similar. However, some rules I used made nanoc more closely emulate Jekyll.

nanoc has a blogging helper, which provides similar functionality to Jekyll's post handling. Unlike Jekyll, which keeps posts in /_posts, blogging helper considers any item with a :kind of article to be a post. Jekyll also determines an articles' creation time from its title. I created the following preprocess rule to make nanoc act like Jekyll in that regard.

require 'time'
preprocess do
  items.each do |item|
    if item.identifier =~ %r{^/articles/(.*)$}
      item[:created_at] = Time.parse($1)
      item[:kind] = 'article'
    end
  end
end

The other rule was for the routes of articles, for which I used this regex to make articles go in the same location as Jekyll posts. There's probably a cleaner solution, and the regex might need adjustment for others' use of Jekyll, but this worked quite well for me.

1
2
3

item.identifier.gsub(%r{^/articles/(\d+)-(\d+)-\d+_\d+-\d+-(.*)$}) do |match|
  "/#{$1}/#{$2}/#{$3}/index.html"
end

With these changes I was able to use all of my Jekyll posts in nanoc without modification.