I today completed a twelve-day stint working with Westfield London in collaboration with Delete London, working on Ruby on Rails development.

The very lovely digital marketing team had some ideas for quick wins to improve the user experience on the London Westfield website, including adding affiliate product data to the retailer pages, and creating an events booking system. I was brought in to help realise these ideas, and turned around two rapid solutions in the allotted time.
Affiliates
The challenge was to find a way to manage the variety of retailer and affiliate partners that Westfield deal with, in an efficient way. I developed an XML parser using REXML’s stream listener to read XML information from an arbitrary affiliate partner, and create product objects from the live stream. These objects then interface with the existing CMS system and generate XML documents ready to be picked-up in the view. Not only does this vastly improve the look of the retailer pages and raise the profile of their clients, but their click-throughs will earn affiliate fees. Win!
Events
Responding to a very tight deadline, I developed an events booking system, designed for the digital marketing team to administer events with minimal effort. The system, implemented in two days, has an administrative back-end that allows the creation of events, with multiple dates attached, composed in an AJAX-form. I used unobtrusive_date_picker to render a user-friendly date picker, as well as restful_authentication to handle the login process. In order to launch within the timeframe, we decided to build the system as a stand-alone webapp, which was supposed to be hosted on a third-party, but after an eleventh-hour issue, I ended up offering my own server as a host, and therefore have my first corporate client! However, the rendered form sits within an IFRAME to hide the URL from the customer.

Meanwhile, hundreds are signing up to Westfield’s events, and the marketing team can download their required CSV data on demand, and the form will deactivate once individual dates become full.

If you’re into fashion, you should sign-up! http://uk.westfield.com/london/offers-events/events/william-baker-booking

I like to think that I stay ahead of the times. In 2000, I used to think that my Creative Nomad jukebox with 6GB of hard-disk space was the most incredible thing ever, and I took mine to University, proud of its ability to reduce a shelf-load of CDs into an object no larger than one or two discs. It was cool how you could take a bunch of music, let them all get strung together into a continuous stream, and have them played back in a random order, like your own personal radio station without the ‘personalities’. Back then, Creative had to market them as jukeboxes- they were still too bulky to be used like iPod shuffles are now. But that was nearly ten years ago, and times must move on.

I have found that the biggest problem after moving to mp3s is keeping up with my collection. I stopped collecting after around 20GB and began to just have smaller collections here and there: some music on my PC laptop, some on my Mac, some on my PC, some on a mp3 player. Overall, it became difficult to keep it all in one place, and so whenever I’d buy a CD, download an album or mix one of my own tracks my collection would become increasingly fragmented. Its got to the stage now where I haven’t actually listened to parts of my collection for many months, so I decided to do something about it.

I wanted to setup my own jukebox, streaming music over the web, so that I could access it wherever I was, from any platform, at any time. That way, all my music is in one place and I save heaps of disc space on my local computers.

So I decided to setup Icecast, an open-source streaming server. It works by receiving an audio stream from somewhere, and converting it into itty-bitty-bytes that are downloaded to clients over the Internet, such as iTunes. The audio stream can be a live feed from a soundcard, or taken from a playlist running off your hard drive. I used the latter approach, because I have a large directory of mp3 files that I want to listen to. This involves using another piece of software called Ices, which is responsible for building the audio stream sent to Icecast. Its a handy piece of software; working from a playlist, it can randomly choose a track, re-sample it to a lower bitrate (to save your bandwidth costs and keep the music playing smoothly) and even cross-fade between tracks.

While I was busy rsyncing my music collection to my server, I downloaded and installed icecast 2.3.2 and ices 0.4 from source. I chose the older version of ices, because it deals with mp3s, whereas the newer version can only read ogg Vorbis files, and my collection consists mainly of mp3 files. I needed to install a couple of dependencies thrown up by configure, specifically libxslt, which I did using apt-get. So far so good.

The first configuration step was configuring Icecast. By default, icecast creates a configuration file in /usr/local/etc/ called icecast.xml which you can edit. The file is commented with examples, and can be tweaked as needed. Don’t forget as this is an XML file that it should be well-formed with matching opening and closing tags, as well as opening and closing comments (and no nesting comments!). I reduced the number of clients (it is a private stream for me!), set the hostname/port, log directory and the source, relay and admin passwords (they’re ‘hackme’ by default!). Following that, I was able to test the server was working by running icecast and checking the error.log file:
icecast -c /usr/local/etc/icecast.xml
tail /var/log/icecast/error.log
I could also visit the admin pages at the hostname and port I had configured, which show a basic admin interface.

The next step was to provide audio for Icecast by configuring Ices. Similarly, the default install created an example configuration file in /usr/local/etc/ called ices.conf.dist. I moved this to ices.conf and began to edit. I changed:

• <File> – I set this to be the playlist file I wanted for the stream. The playlist file is simply a text file with a path to an mp3 file on each line, so I generated it automatically from the music files I kept in /music.
  find /music -name *.mp3 > /usr/local/etc/playlist.txt
• <Randomize> – I set this to 1 so that Ices jumped around the list, creating a radio station style mix, rather than playing through sequentially.
• <Crossfade> – I set this to 5 so that each song fades-in/fades-out over the course over five seconds, resulting in nice, continuous play.
• <BaseDirectory> – I set this to be the same as the log directory of Icecast so that the logs were nearby each other.
• <Hostname> – I set this to match the hostname I configured in Icecast.
• <Port> – Ditto…
• <Password> – Same…
• <Name> – I set this to “Dan’s Jukebox”, which is what the stream is listed under in iTunes.
• <Genre> – I set this to “Anything”.
• <Description> – This also appears on the web interface and iTunes so I wrote something in there too.
• <URL> – This URL is a page used to describe the stream, and appears on rotation in the iTunes interface so I used my website URL.
• <Public> – I set this to 0 so that the stream isn’t listed publicly and I get subpoenaed by the RIAA…
• <Bitrate> – Very important! This is the setting that Ices uses to determine what quality to send the audio in, and has implications on the amount of bandwidth you use; which in turn determines the financial cost and smoothness of your stream. If your stream is available to multiple users, you have to consider that this figure would be multiplied by the number of listeners, so be careful not to set it too high. 64kbps is about the quality you get listening to MySpace and is low-quality; but you won’t really notice if listening on a laptop speaker anyway. Bearing in mind that the mp3s were mostly encoded at 128kps originally, I decided to keep my bandwidth use down and started out with 64kbs. It sounds alright to me, maybe I’ll ramp it up to about 80-something later if I feel like it.

Once configured, I tested Ices by running it without any options:
ices
By default, Icecast and Ices run in the foreground, which means that once you start running them they don’t return the console back to you and you have to open another to keep using Linux. I wanted them to run in the background as daemons so that when my server boots up / restarts, or I log-out, the music keeps going. I so put together two init.d scripts, starting from /etc/skeleton, to start/stop Icecast and Ices. Below is the icecast init.d script I’m using, and Ices is similar.
#! /bin/sh
# Author: Dan Garland dan@NOSPAMdangarland.co.uk>

# Do NOT “set -e”

# PATH should only include /usr/* if it runs after the mountnfs.sh script
PATH=/sbin:/usr/sbin:/bin:/usr/bin
DESC=”Icecast Server”
NAME=icecast
DAEMON=/usr/local/bin/$NAME
DAEMON_ARGS=”-b -c /usr/local/etc/icecast.xml”
ICES=/usr/local/bin/ices
USER=$NAME
GROUP=$NAME
PIDFILE=/var/run/$NAME.pid
ICES_PIDFILE=/var/run/ices.pid
SCRIPTNAME=/etc/init.d/$NAME

# Exit if the package is not installed
|| exit 0

# Read configuration variable file if it is present
&& . /etc/default/$NAME

# Load the VERBOSE setting and other rcS variables
. /lib/init/vars.sh

# Define LSB log_* functions.
# Depend on lsb-base (>= 3.0-6) to ensure that this file is present.
. /lib/lsb/init-functions

#
# Function that starts the daemon/service
#
do_start()
{
# Return
# 0 if daemon has been started
# 1 if daemon was already running
# 2 if daemon could not be started

start-stop-daemon –start –quiet –chuid $USER:$GROUP –pidfile $PIDFILE –exec $DAEMON –test > /dev/null \
|| return 1

start-stop-daemon –start –quiet –chuid $USER:$GROUP –pidfile $PIDFILE –exec $DAEMON — \
$DAEMON_ARGS \
|| return 2
# Add code here, if necessary, that waits for the process to be ready
# to handle requests from services started subsequently which depend
# on this one. As a last resort, sleep for some time.
}

#
# Function that stops the daemon/service
#
do_stop()
{
# Return
# 0 if daemon has been stopped
# 1 if daemon was already stopped
# 2 if daemon could not be stopped
# other if a failure occurred
start-stop-daemon –stop –quiet –retry=TERM/30/KILL/5 –pidfile $PIDFILE –name $NAME
RETVAL=”$?”
&& return 2
# Wait for children to finish too if this is a daemon that forks
# and if the daemon is only ever run from this initscript.
# If the above conditions are not satisfied then add some other code
# that waits for the process to drop all resources that could be
# needed by services started subsequently. A last resort is to
# sleep for some time.
start-stop-daemon –stop –quiet –oknodo –retry=0/30/KILL/5 –exec $DAEMON
&& return 2
# Many daemons don’t delete their pidfiles when they exit.
rm -f $PIDFILE
return “$RETVAL”
}

#
# Function that sends a SIGHUP to the daemon/service
#
do_reload() {
#
# If the daemon can reload its configuration without
# restarting (for example, when it is sent a SIGHUP),
# then implement that here.
#
start-stop-daemon –stop –signal 1 –quiet –pidfile $PIDFILE –name $NAME
return 0
}

case “$1″ in
start)
&& log_daemon_msg “Starting $DESC” “$NAME”
do_start
case “$?” in
0|1) && log_end_msg 0 ;;
2) && log_end_msg 1 ;;
esac
;;
stop)
&& log_daemon_msg “Stopping $DESC” “$NAME”
do_stop
case “$?” in
0|1) && log_end_msg 0 ;;
2) && log_end_msg 1 ;;
esac
;;
#reload|force-reload)
#
# If do_reload() is not implemented then leave this commented out
# and leave ‘force-reload’ as an alias for ‘restart’.
#
#log_daemon_msg “Reloading $DESC” “$NAME”
#do_reload
#log_end_msg $?
#;;
restart|force-reload)
#
# If the “reload” option is implemented then remove the
# ‘force-reload’ alias
#
log_daemon_msg “Restarting $DESC” “$NAME”
do_stop
case “$?” in
0|1)
do_start
case “$?” in
0) log_end_msg 0 ;;
1) log_end_msg 1 ;; # Old process is still running
*) log_end_msg 1 ;; # Failed to start
esac
;;
*)
# Failed to stop
log_end_msg 1
;;
esac
;;
*)
#echo “Usage: $SCRIPTNAME {start|stop|restart|reload|force-reload}” >&2
echo “Usage: $SCRIPTNAME {start|stop|restart|force-reload}” >&2
exit 3
;;
esac

:
Note the -b option to run in the background and the –chuid option to run the program as the icecast user and group (Icecast doesn’t run as root). If you don’t have a icecast user or group, create them as root with the adduser and addgroup commands:
adduser icecast
addgroup icecast
I could then tell Debian to run these scripts at start-up using update-rc.d:
update-rc.d icecast defaults
update-rc.d ices defaults
Once running, I could point iTunes to the stream by choosing Advanced -> Open Audio Stream, and typing in the full url to the mointpoint. Now I’m listening to some track I swear I haven’t heard since the mid 90′s, and I want to tell the world.

This post is about running the XPay client provided by SecureTrading for on-line e-commerce payment processing, but it might also be useful for anyone trying to run a java application on a Debian server, particularly if they require it to be 32-bit compatible, such as running a JRE 1.4.2, or to start-up on boot.

SecureTrading’s payment processing solution provides a Java application responsible for all the encryption and secure communication with their payment gateways, leaving the application developer only responsible for forming an XML document and transmitting it on an unsecured, local socket. I have recently completed an integration using Ruby on Rails, and things were going too smoothly; something had to be problematic!

And it was this: I was running Debian Etch on a 64-bit server, and the XPay client requires Java runtime 1.4.2, which is no longer supported by Sun and only has a 32-bit linux version (unless you’re running Intel). I thought I was stuck: SecureTrading even offered to refund me at one point. I was certain there had to be solution other than getting hold of another 32-bit box, and it was, thanks to the excellent Debian o/s.

Some boffin has packaged the libraries necessary to run 32-bit apps in the ia32-libs package, so that older 32-bit applications can run alongside up-to-date 64-bit ones. It is installed in one command:
apt-get install ia32-libs
I could then download and run the binary installer for the Java runtime environment. I could then follow the instructions to setup the client certificate in the keystore and run xpay.
/usr/local/java/j2re1.4.2_19/bin/keytool -import -alias xpay -file /var/whereever/xpay/securetradingxpay.cer \
-keystore /usr/local/java/j2re1.4.2_19/lib/security/cacerts
This leaves the task of having XPay run at startup, as an init.d script. Typically, Java doesn’t play nice with init.d scripts, but I managed to get it running with Debian’s start-stop-daemon script. Starting from the /etc/init.d/skeleton file, I trimmed most of it away to end up with:
#! /bin/sh
# Author: Dan Garland <dan@dangarland.co.uk>
# Do NOT “set -e”
# PATH should only include /usr/* if it runs after the mountnfs.sh script
JAVA_HOME=/usr/local/java/j2re1.4.2_19
CLASSPATH=/var/whereever/xpay/XPay.jar
PATH=$JAVA_HOME/bin:/sbin:/usr/sbin:/bin:/usr/bin
DESC=”Xpay Client”
NAME=XPay
DAEMON=$JAVA_HOME/bin/java
DAEMON_ARGS=”-classpath $CLASSPATH -Djava.security.manager -Djava.security.policy=/var/whereever/xpay/xpaypolicy XPay ”
PIDFILE=/var/run/$NAME.pid
SCRIPTNAME=/etc/init.d/$NAME

# Read configuration variable file if it is present
&& . /etc/default/$NAME

# Load the VERBOSE setting and other rcS variables
. /lib/init/vars.sh

# Define LSB log_* functions.
# Depend on lsb-base (>= 3.0-6) to ensure that this file is present.
. /lib/lsb/init-functions

#
# Function that starts the daemon/service
#
do_start()
{
  start-stop-daemon –start –background -p $PIDFILE -m –exec $DAEMON — $DAEMON_ARGS
  echo “$NAME started…”
}

#
# Function that stops the daemon/service
#
do_stop()
{
  start-stop-daemon –stop -p $PIDFILE
  echo “$NAME stopped…”
}

#
# Function that sends a SIGHUP to the daemon/service
#
do_reload() {
        #
        # If the daemon can reload its configuration without
        # restarting (for example, when it is sent a SIGHUP),
        # then implement that here.
        #
        start-stop-daemon –stop –signal 1 –quiet –pidfile $PIDFILE –name $NAME
        return 0
}

case “$1″ in
  start)
        && log_daemon_msg “Starting $DESC” “$NAME”
        do_start
        case “$?” in
                0|1) && log_end_msg 0 ;;
                2) && log_end_msg 1 ;;
        esac
        ;;
  stop)
        && log_daemon_msg “Stopping $DESC” “$NAME”
        do_stop
        case “$?” in
                0|1) && log_end_msg 0 ;;
                2) && log_end_msg 1 ;;
        esac
        ;;
  #reload|force-reload)
        #
        # If do_reload() is not implemented then leave this commented out
        # and leave ‘force-reload’ as an alias for ‘restart’.
        #
        #log_daemon_msg “Reloading $DESC” “$NAME”
        #do_reload
        #log_end_msg $?
        #;;
  restart|force-reload)
        #
        # If the “reload” option is implemented then remove the
        # ‘force-reload’ alias
        #
        log_daemon_msg “Restarting $DESC” “$NAME”
        do_stop
        case “$?” in
          0|1)
                do_start
                case “$?” in
                        0) log_end_msg 0 ;;
                        1) log_end_msg 1 ;; # Old process is still running
                        *) log_end_msg 1 ;; # Failed to start
                esac
                ;;
          *)
                # Failed to stop
                log_end_msg 1
                ;;
        esac
        ;;
  *)
        #echo “Usage: $SCRIPTNAME {start|stop|restart|reload|force-reload}” >&2
        echo “Usage: $SCRIPTNAME {start|stop|restart|force-reload}” >&2
        exit 3
        ;;
esac

:
This script then runs XPay on boot. Make sure that the paths to the jar and the policy files are correct, and that the policy file has the correct path to the Java 1.4.2 installation directory.

Today is a milestone in my ambitions as a freelance web developer. I have installed and deployed my own server in a datacenter, running Debian linux. This development now allows me to offer full-cycle web development, from concept, through design, implementation and testing, and I can even host your website when its done.

Contact me for a quote for your freelance development or hosting needs.

I have invested some time and money into acquiring my own web platform which I will be using to host my upcoming websites for myself and for my clients. I have called my new system Titan, after the people who bought it, and because a Titan is a powerful deity; Titan is both powerful and omnipresent (provided you have a web connection).
I’ve spent the day assembling my components and installing Debian Etch 4.0. The system is a dual-core, dual-CPU AMD 64-bit Tyan Transport GT24, with 16GB RAM and 1TB RAID. So far, I’m happy with how the system is holding out, although I had a bit of bother deciding on how to handle the RAID functionality. It turns out that the RAID capacity of the controller is what linux geeks refer to as ‘Fake RAID’, because it is not a true hardware RAID controller that presents the seperate disks to the OS as one array; so I was confused when Debain was showing all four disks. In the end, I used a software RAID 5 setup, which is working rather well.
Ultimately, Titan will be handling all of the traffic for my band’s upcoming music website…. more on that soon. However now that I have the capacity to host some extra sites I will be able to take on some hosting arrangements, and I plan to host the websites I will be developing in future.