I reached the second milestone of my project to create a virtual reality model of the Conic Bellophone, an instrument comprised of 144 cone-shaped bells being developed for a PhD project. Moving on from the early prototype, which has six bells, I have managed to complete a version that has 144 bells, creating a ghostly harmonic when played.

The first challenge was dealing with the size and complexity of the model. I was supplied VRML97 exported from CATIA, describing a much more complicated and detailed model. In this model, the author had gone to the detail of individual screws, brackets, stands and bells, and rather than defining the geometry as primitive shapes, the exporter program naively creates large numbers of IndexedFaceSets; defining each point on a plane individually. Consequently, the supplied VRML97 was about 15MB in size, some 40,000 lines of code. This instantly presented problems with the tools I was using; Netbeans would regularly crash and the load-up time was taking about 4-5 minutes, even on my MacBook. At any rate, to deliver such a file on the web would be useless. So my first task, before I could really begin, was to reduce the filesize of the model. To accomplish this, I had to first depart from the arcane VRML97 format and into X3D, an XML version of VMRL that I could work with.

I noticed from inspecting the file that the exporter had introduced a large number of seemingly useless group nodes, such as:
<Group>
<Group>
<Group>
<Group USE=”somegroup”>
…
</Group>
</Group>
</Group>
</Group>
I decided to remove them with a simple parser written in Java. I created an application that read in the XML file, performed a recursive operation to remove the nodes and write out the result.
private void deleteUseless(Node n)
{
     Node parent = n.getParentNode();

     NodeList children = n.getChildNodes();
     for(int p = 0; p < children.getLength(); p++)
     {
       Node child = children.item(p);

       if(!child.getNodeName().equals(“#text”))
       {
         deleteUseless(child);
       }
     }

     if(n.getNodeName().equals(“Group”)
             && n.getAttributes().getNamedItem(“USE”) == null
             && n.getAttributes().getNamedItem(“DEF”) == null)
     {
       for(int p = 0; p < children.getLength(); p++)
       {
         Node child = children.item(p);
         if(!child.getNodeName().equals(“#text”))
         {
             parent.insertBefore(child, parent.getFirstChild());
         }
       }
       parent.removeChild(n);
       replacecount++;
     }
}
This method removed 2,500 unnecessary nodes. This still wasn’t enough to make the scene workable, so I began to simplify the scene by replacing IndexedFaceSets with primitive geometry.

The most important element in the scene were the bells themselves. I created a new node to represent a bell, that would be scaled to the desired size and re-used multiple times. In X3D jargon, this new node is called a prototype. A prototype defines a new node, and has its own scope for ROUTEs and DEFs. A prototype can be parametrised by defining a number of fields, which permit re-use, towards a component-based method of building complex scenes.

At this stage each bell had three requirements, other than rendering the bell in 3D. Firstly, the bell had to sound when clicked, as if the bell were stuck in real-life. Secondly, the bell had to change colour when clicked, to visualise what parts of the instrument were playing. Thirdly, the bell had to be able to play when the mouse was hovering over the bell, so that a run of bells could be played in a glissando effect. The behaviour between a single-bell sound and a glissando would be toggled by pressing the ‘G’ key on the keyboard.
Single sound of AudioClip
The first requirement is the most straightforward to implement. It is a text-book use of the TouchSensor node, along with a Sound and AudioClip node. Keeping all the nodes grouped together, it is important that the TouchSensor is a sibling of the grouping node of the geometry, for the TouchSensor acts upon the child nodes of its parent. Thus:
<Group>
<TouchSensor />
<Sound>
<AudioClip />
</Sound>

<!– Geometry –>
<Shape>
…
</Shape>
</Group>
The AudioClip defines the sound to play with its url parameter. X3D allows multiple URLs to be specified, to allow for local and remote access. ROUTEs are used to implement the sound behaviour:
<ROUTE fromNode=”singleTouch” fromField=”touchTime” toNode=”chime” toField=”stopTime”/>
<ROUTE fromNode=”singleTouch” fromField=”touchTime” toNode=”chime” toField=”startTime”/>
N.B. I include both stopTime and startTime as I had desired the bell-sound to restart when clicked again, much like a bell would if hit successive times. According to the X3D specification, passing the same time event for the stop and the start should result in the audio clip restarting. However I found mixed results with this, and mostly I see start/stop behaviour instead.
Colour Highlight with ColorInterpolator
Like all complicated concepts, using interpolators in X3D is straightforward once you understand it. I wanted to choose a colour for the bell’s material, which would change like a roll-over in CSS. In X3D, its easier to consider the problem as charting the node’s material to pass through a series of colour values, which is precisely what ColorInterpolator does. In this sense its more akin to a robot walking between waypoints than a roll-over paradigm.

An interpolator (or the similar sequencer nodes) have two main variables. The key, which defines the points in the series, and the keyValue, which define the corresponding values at those points. The key is always between 0 and 1, representing the start and end of the cycle, with any number of fractions in between, seperated by whitespace. For my roll-over I defined 0, 0.5 and 1, so that I can pass through three colours, finally returning to the original colour at the end of the cycle. For the corresponding values, in this case a colour, I would need three-pairs of RGB values to correspond to the colours at 0, 0.5 and 1.

In order to cause succession between the points, in X3D a TimeSensor is used to generate time events, so that the browser moves through the series, like a sequencer. When used alongside the touchsensor, these nodes can now be routed to achieve the roll-over effect.
<TimeSensor DEF=”colourTimer” />
<ColorInterpolator DEF=”flash” key=”0 0.5 1″ keyValue=”0 1 0 1 1 1 0.8 0.8 0.8″/>
<TouchSensor DEF=”glissTouch” enabled=”false”>

<ROUTE fromField=’touchTime’ fromNode=’singleTouch’ toField=’set_startTime’ toNode=’colourTimer’/>
<ROUTE fromField=’fraction_changed’ fromNode=’colourTimer’ toField=’set_fraction’ toNode=’flash’/>
<ROUTE fromField=’value_changed’ fromNode=’flash’ toField=’set_diffuseColor’ toNode=’material’/>
Glissando effect with AudioClip
The best result of the model was the glissando, simultaneously playing several samples of micro-tonal bells, creating a haunting and harmonic sound when the mouse cursor passes over a bell. This effect was a slight change from the single-touch, using a different field to drive the behaviour. touchTime would be too late; for this is the time value when the touch sensor’s geometry was clicked. Instead, TouchSensor provides an isOver event, which is set to true when the mouse cursor intersects the geometry. As I required a time value rather than a boolean value, I needed another node to convert the values, called TimeTrigger. TimeTrigger is an invisible node that acts like a black-box; accepting a boolean input and returning a timestamp output, triggerTime. I used this to route between the TouchSensor and the AudioClip to obtain the glissando effect, and could even use this to drive the colour flash.
<ROUTE fromNode=”glissTouch” fromField=”isOver” toNode=”gliss” toField=”set_boolean” />
<ROUTE fromNode=”gliss” fromField=”triggerTime” toNode=”chime” toField=”set_startTime” />
<ROUTE fromField=’triggerTime’ fromNode=’gliss’ toField=’set_startTime’ toNode=’colourTimer’/>
Toggle play mode with keyboard button press
Finally, I needed to switch between glissando and the single-click modes by pressing a keyboard key. To keep the behaviours seperate, I defined two seperate TouchSensors, each routed differently to achieve the different behaviours. I defined the TouchSensor associated with the glissando effect to be disabled, with the enabled=”false” attribute on the touch sensor. The process of toggling thus became an alternation of the enabled attribute of the TouchSensor. The current state of the toggle is stored using a BooleanToggle node.

The browser picks up keyboard events when you define a KeySensor within your scene. The KeySensor can then be routed to a script node, for processing. Below I wrote a short script to detect whether the letter ‘G’ had been pressed, and routed the result to the touch sensor.
<KeySensor DEF=”glissMode” />
<BooleanToggle DEF=”playMode” />

<Script DEF=”logic” directOutput=”true”>
<field name=’keyInput’ type=’SFString’ accessType=’inputOnly’/>
<field name=’toggleGliss’ type=’SFBool’ accessType=’outputOnly’/>
<![CDATA[ecmascript:
 function keyInput (inputValue)
{
if(inputValue == 'g' || inputValue == 'G')
{
toggleGliss = true;
}
}
]]>
</Script>

<ROUTE fromNode=”logic” fromField=”toggleGliss” toNode=”playMode” toField=”set_boolean” />
<ROUTE fromNode=’glissMode’ fromField=’keyPress’ toNode=’logic’ toField=’keyInput’/>
<ROUTE fromNode=”playMode” fromField=”toggle_changed” toNode=”glissTouchSensor” toField=”enabled” />
Putting it all together
Now I could define a prototype and re-use the bell logic and geometry. I would embed each prototype inside a transform element, so that I could scale and translate each individually. The only alteration I needed to make was with the button toggle; for the state of the toggle had to be held within the main scene, not within each prototype, because each prototype has its own scope. I modified the prototype to accept the boolean value of the BooleanToggle as a field, as well as the URL for the sound to play.
<ProtoDeclare name=’Bell’>
<ProtoInterface>
  <field accessType=’initializeOnly’ name=’soundURL’ type=’MFString’ />
    <field accessType=’inputOnly’ name=’playMode’ type=’SFBool’/>
  </ProtoInterface>
  <ProtoBody>
  <Group DEF=”BellMain”>
      <TimeSensor DEF=”colourTimer” />
      <ColorInterpolator DEF=”flash” key=”0 0.5 1″ keyValue=”0 1 0 1 1 1 0.8 0.8 0.8″/>
      <TouchSensor DEF=”glissTouch” enabled=”false”>
      <IS>
          <connect nodeField=’enabled’ protoField=’playMode’/>
        </IS>
      </TouchSensor>
      <TouchSensor DEF=”singleTouch” />
      <TimeTrigger DEF=”gliss” />
      <Sound maxFront=”100000″ maxBack=”100000″>
        <AudioClip DEF=”chime”>
       <IS>
            <connect nodeField=’url’ protoField=’soundURL’/>
          </IS>
        </AudioClip>
      </Sound>

      <!– Bell Geometry –>

<BooleanToggle DEF=”playMode”>
<IS>
<connect nodeField=’set_boolean’ protoField=’playMode’/>
</IS>
</BooleanToggle>

<!– Single-touch sound –>
<ROUTE fromNode=”singleTouch” fromField=”touchTime” toNode=”chime” toField=”stopTime”/>
<ROUTE fromNode=”singleTouch” fromField=”touchTime” toNode=”chime” toField=”startTime”/>

<!– Glissando –>
<ROUTE fromNode=”glissTouch” fromField=”isOver” toNode=”gliss” toField=”set_boolean” />
<ROUTE fromNode=”gliss” fromField=”triggerTime” toNode=”chime” toField=”set_startTime” />
<ROUTE fromField=’triggerTime’ fromNode=’gliss’ toField=’set_startTime’ toNode=’colourTimer’/>

<!– Colour flash –>
<ROUTE fromField=’touchTime’ fromNode=’singleTouch’ toField=’set_startTime’ toNode=’colourTimer’/>
<ROUTE fromField=’fraction_changed’ fromNode=’colourTimer’ toField=’set_fraction’ toNode=’flash’/>
<ROUTE fromField=’value_changed’ fromNode=’flash’ toField=’set_diffuseColor’ toNode=’material1′/>
</ProtoBody>
</ProtoDeclare>
Rather than rigging up 144 route statements to pass each prototype the playmode value, I picked up this neat trick from the X3D mailing list to dynamically add routes via the SAI. In order for the script to work, all of the prototypes were grouped together in a single group, so that they could be passed to the script.
<Script DEF=”logic” directOutput=”true”>
<field name=”touchs” type=”SFNode” accessType=”initializeOnly”>
<Group USE=”Touchable”/>
</field>
<field name=”playMode” type=”SFNode” accessType=”initializeOnly”>
<BooleanToggle USE=”playMode” />
</field>

<![CDATA[ecmascript:
function initialize()
{
scene = Browser.currentScene;

for (i = 0; i < 144; i++)
{
bell = touchs.children;
scene.addRoute(playMode, 'toggle_changed', bell, 'playMode');
}
}
]]>
</Script>
Next steps
I was left only to rig-up the geometry for the stands and the bases. Overall, the model is working well. After the optimisations, the file was reduced to 1,500 lines of code, and renders at an acceptable framerate. The next challenge will be to reduce the file-size of the audio-clips: currently 10 second .wav files, before it can be delivered onto the web proper.

The remaining area of functionality for this project is to work with MIDI to read-in compositions and play them using the new instrument, in effect, writing a basic MIDI sequencer in X3D. I’m currently working on this now, so watch this space! You can listen to a sample of the sounds being played on my music blog.

Thanks to all those in the X3D Mailing List whose comments have helped me progress with this project.

One of the projects I’m working on right now is an ambitious attempt to realise a 3D-model of the Conic Bellophone, an experimental musical instrument under developed as part of a PhD project. The requirements were to develop a 3D model that can be viewed from any angle, and when a bell is ‘struck’ to sound a sample captured from the real thing.

I hoped to dust-off my Xj3D knowledge from hacking around at University during my thesis, and despite some wrestling I managed to get as far as a 3D model with sounding bells, and a glissando effect when you run the mouse cusor over the bells (press the red box).

Still to do: I have to figure out how to persuade the time/event model to accept restarting sound. Currently, the spec says that passing the current time to both the startTime and stopTime of an AudioNode should restart the node, but I am finding this not to be the case, and I’m trying workarounds with script nodes. After that’s solved, I will be working with MIDI to allow the instrument to play arrangements made in MIDI; all in 3D.

The instrument uses the applet-launcher Java web start program, to wrap an applet which renders the Xj3D browser and the scene. I’ve found this has issues when used in windows, so there is still some work to do.

But take a sneak preview: http://www.dangarland.co.uk/virtualinstrument . Don’t forget to press ‘trust’ on the security prompts that appear.

When delivering on-line content, it is always desirable to ensure that the content is accessible to the widest possible audience. This objective is well-documented in most areas of web development, but when it comes to dealing with 3D scenes and worlds on-line, things are far from straightforward. A variety of formats, proprietary plug-ins and target platforms make delivering 3D content on the web a challenge.

The simplest method of publishing 3D is probably to link straight to a scene file, and rely on a browser plug-in capable of reading it. This approach is limited in that it depends on end-users having such a plug-in installed; and these plug-ins are normally platform-dependant. They also pose problems when trying to fit the object within the webpage; all too often frames are used to compensate, with ugly results.

The best fit to this solution takes the form of the X3D format. X3D specifies a clean XML syntax for 3D worlds, compatible with existing languages and tools, and comes with its own Java implementation, called Xj3D. This open-source codebase contains the code for the scenegraph nodes and also for a browser, allowing 3D objects to be embedded in webpages using Java applets; hence delivering a platform-independent solution as well using a plug-in that is widely supported. I wanted to embed a 3D scene into a webpage for a client, so I found out how to employ Xj3D as an applet.

The Xj3D browser uses OpenGL as the default renderer, which introduces platform-dependent requirements on end-users. The Jogl and Joal libraries depend on native extensions that wouldn’t be known until runtime. Fortunately, there is a mechanism using Java Web Start to embed an applet within Web Start, which determines the user’s platform and downloads the relevant libraries, called applet launcher. Using this mechanism, it is possible to run native code, while coding the applet in a platform-independent way.

First of all, some straightforward Java is used to define the applet which we intend to embed, which has the task of setting up the Xj3D browser. This code was based on the example from xj3d.org.
package com.yourserver.applet;

import java.applet.Applet;
import java.awt.*;
import java.util.HashMap;

import org.web3d.x3d.sai.*;
import org.xj3d.sai.*;
import org.web3d.vrml.scripting.external.sai.*;

public class XJ3DApplet extends Applet {

ExternalBrowser browser;

public XJ3DApplet() {
}

public void init() {
setLayout(new BorderLayout());
browser = getBrowser();
loadScene();
}

// Generate the browser
private ExternalBrowser getBrowser() {
HashMap requestedParameters = new HashMap();
requestedParameters.put(“Xj3D_ConsoleShown”, Boolean.TRUE);
requestedParameters.put(“Xj3D_LocationShown”, Boolean.FALSE);

X3DComponent comp = BrowserFactory.createX3DComponent(requestedParameters);
Xj3DBrowser browser = (Xj3DBrowser) comp.getBrowser();

setBackground(Color.blue);
add((Component) comp, BorderLayout.CENTER);
setVisible(true);

return browser;
}

// Create the scene implementation and add it to the world
private void loadScene() {
X3DScene mainScene = browser.createX3DFromURL(new String[] {getParameter(“modelURL”)});
browser.replaceWorld(mainScene);
}
}
This class can then be packaged up into a jar, ready to be deployed. This class would also be the place to hook into the browser for any SAI-scripting, or to pass other parameters to the browser, such changing cursors or the taskbar look + feel.

The next stage was to obtain the Xj3D libraries to deploy alongside the applet. These can be found on xj3d.org. Xj3D provides a jar file to contain the runtime used with applets, packaging a jar for each X3D profile seperately. You will need to choose which jar file is appropriate for your scene; I was using an immersive scene with touch sensors, so I needed xj3d-immersive-applet-av3d_2.0.0.jar.

Before any of the jars could be used however, it is necessary to sign them using the jarsigner command. If you haven’t already got a keystore, then in order to make jarsigner work you must create a key using the keytool command:
keytool -genkey
Which will prompt you for information necessary to create a self-signed certificate and a key, which defaults to ‘mykey’. I used the domain of the server I would be running off as the common name (it prompts you as ‘Your Name’). Once I had a key, I could sign my applet’s jar and each of the Xj3D jars so that I could host them from my server.
jarsigner -storepass changeit soandso.jar mykey
Once I had signed copies of the latest jar files from xj3d.org, I was ready to define the applet tag. The remaining code resides within inside the applet tag within the HTML itself:
<applet code=”org.jdesktop.applet.util.JNLPAppletLauncher”
width=600
height=400
archive=”http://download.java.net/media/applet-launcher/applet-launcher.jar,

http://www.yourserver.com/jogl.jar,

http://www.yourserver.com/gluegen-rt.jar,

http://www.yourserver.com/joal.jar,

http://www.yourserver.com/uri.jar,

http://www.yourserver.com/vecmath.jar,

http://www.yourserver.com/j3d-org-all_0.9.0.jar,

http://www.yourserver.com/xj3d-immersive-applet-av3d_2.0.0.jar,

http://www.yourserver.com/js.jar,

http://www.yourserver.com/xj3d-sai_2.0.0.jar,

http://www.yourserver.com/xj3d-external-sai_2.0.0.jar,

http://www.yourserver.com/httpclient.jar,

http://www.yourserver.com/aviatrix3d-all_2.0.0.jar,

http://www.yourserver.com/xj3d-script-base_2.0.0.jar,

http://www.yourserver.com/yourbrowserapplet.jar”>

<param name=”codebase_lookup” value=”false”/>
<param name=”subapplet.classname” value=”com.yourserver.applet.XJ3DApplet”/>
<param name=”subapplet.displayname” value=”Your Applet”/>
<param name=”subapplet.image” value=”http://www.yourserver.com/splash.jpg”/>
<param name=”noddraw.check” value=”true”/>
<param name=”progressbar” value=”true”/>

<param name=”jnlpNumExtensions” value=”2″/>
<param name=”jnlpExtension1″
value=”http://download.java.net/media/jogl/builds/archive/jsr-231-webstart-current/jogl.jnlp”/>
<param name=”jnlpExtension2″
value=”http://download.java.net/media/joal/webstart/joal.jnlp”/>

<param name=”modelURL” value=”http://www.yourserver.com/bellsnew.x3d”/>
</applet>
Notice that you can host all the jars yourself. I found that I ran into trouble when I was hosting the applet-launcher jar file; this is related to the necessity of having the native extensions signed by the same source as the applet launcher. I’m sure this is fixable but I didn’t discover how.

One difficulty that I found when using this approach was that the link between Web Start and the applet runtime suppressed errors stemming from the Xj3D browser, making debugging very difficult. I found that it was far more efficient to get my scene working perfectly in the stand-alone browser, which reported errors immediately, before trying to get the applet running. Whenever I had trouble getting the scene to load, it was usually because of a jar file that was missing or not signed correctly.

Now when users visit the HTML page, a Java applet appears that downloads all the necessary software, and runs the Xj3D browser, displaying the 3D content in the most accessible way.

A long-standing interest of mine in the field of computer graphics is the progress of the X3D standard for virtual reality, established as an ISO standard in 2004. I was first taken in as a student, working on a project to develop my own network protocol for X3D environments, impressed by the intuitive XML-syntax and standards-driven approach. An open-source Java-implementation exists, although sadly their documentation / user-guide page hasn’t moved on at all since I was at University. Ultimately, I found working with a draft specification, no documentation and a severe lack of tools too much for X3D to remain compelling, although when I was recently approached to develop an on-line model of a musical instrument that has recently been invented, my first inclination was to catch up with X3D and see if it had become any more mature or easier to work with.

My first stop was checking out what tools were available. The Web3D consoritum keep a list of players and editors that is currently more useful than a Google search, mainly because so many tools are propreitary and only run on Windows. I was running on Mac OSX, and it took a while to find any editors that were helpful at all. But before I nearly gave in to running parallels and WinXP, I found that X3D-Edit has become much more useful and featured. A Java-based Netbeans mod, it comprises a built-in Xj3D browser for easy preview (just hit right-mouse and its the first option) as well as tight integration with any other players; I was checking my scene in the excellent FreeWRL. I found that the navigation tree and code-snapping features took a great deal of complexity out of the complex model I was supplied with, and the import from VRML 2.0 worked like a charm. Despite the more powerful features, I found that the packaging of X3DEdit is rather awkward and un-mac like; the executable is a shell command to kick-start the Java VM, and the memory-hungry app forced me to quit and restart the editor many, many times.

However, to suddenly have found a tool that I could put up with for X3D development renewed those questions buzzing around in my head since discovering virtual reality: could we all easily develop 3D-environments, as easy as we’re writing blogs today? Will we all be wondering around with HUDs and building social networks in 3D-psychedelic tripped-out universes?

And then I get amongst the code, and I see that X3D is still very much back in the 90s. While 3D has come from Wolfenstein3D to Ice Age in the last ten or so years, its hard to see how I could sit in front of this computer for more than that long and achieve nothing more than a cone with X3D; and even a cone would be a bit 99 (pardon the pun). Thankfully, for this job I was supplied with a model; but even the user-interface of the browsers with their unusual viewing paradigms seem designed for little more than tracking a spherical object in a scene. My little brother uses Wii-motes to make music in a little 3D world; and yet even with a degree in computer science I don’t feel confident that I could take this technology anywhere near that, as cool as would be.

Nonetheless the ideals of the standard and the potential applications are too vital to ignore. Perhaps hope will come in the form of more open-source technology. I haven’t had the chance to check out blender, but it seems to be a fully-fledged graphics package like Maya, without the several thousand pound price tag. The flux player has found an application and become Vivaty, which looks promising as a standards-based second-life styled network. So for now, I’ll continue the trial-and-error with X3D.