The greatest influence on web applications today is desktop application software. In the October issue of Treehosue, John Zeratsky, wrote about how the original interface developers for the Mac and Windows took their inspiration from the real world. Real world interfaces that are controlled by touch or physical manipulation are called “haptic interfaces.” Your mouse and keyboard are examples of haptic interfaces as are TIVO remotes, the ipod and a Playstation controller.
I’ve found that one of the best places to study a wide range of haptic interfaces are at concerts. Musical instruments are remarkable devices with learning curves that vary as much as the software interfaces we find in the digital world. Pianos, like to-do list makers and instant messaging software, are amazingly simple to understand and interact with even though under the hood sits a complex matrix of wires and hammers. A violin, on the other hand, is a bit more like Photoshop or Shake, requiring a bit more training and effort to coax something pleasant in the beginning. While some of the most compelling software gets people “kicking ass” the fastest, often mastery is a product of discipline (or coercion) and repetitive usage.
Because it’s so easy to get in the mindset that the best inspiration comes from our operating systems, I’ve highlighted some music-makings applications below to show there’s plenty of room for innovation when it comes to creating unique interfaces.
- Audiopad : Audiopad is a composition and performance instrument for electronic music that tracks the positions of objects on a tabletop surface and converts their motion into music. Developed by the brains over at MIT, you have to check out the videos to see the elegance of this interface. With some additional features and development, this could become every DJ’s dream instrument. There’s also no reason why this interface couldn’t be adapted for video remixing or even a touch screen interface.
- Max/MSP : Max/MSP is a graphical environment for music, audio, and multimedia. This software has been used worldwide for over fifteen years by performers, composers, artists, teachers, and students to build interactive instruments. As a general rule of thumb, programming is not a pleasant experience for me. I’ll do it, but I try to pass off as much to Ryan as possible. In college, I was introduced to Max/MSP and NATO to help me build virtual video remixing applications and I have to say it is one of the most exciting environments to program I have ever worked in. Like Flash, there’s an immediacy to the manipulations that makes hacking, tweaking and modding MAX patches easy and accessible. Definitely check out the trial demos. Comes in both Mac and PC flavors.
- Reason 3.0 : Propellerhead Software’s virtual studio rack literally translates real world synthesizers into a convenient software environment. A single keypress will turn Reason’s rack around, to reveal virtual patch cords that you can repatch by dragging the plug to the desired connector. Most audio connections are made automatically and when a new device is created, it appears immediately below the currently selected device and patched into the system in the most logical way. Reason is well-designed and ridiculously customizable.
I think the reason these interfaces are so appealing is that they successfully show the relationships of the choices we make. In Audiopad, distance from one puck to another translates to volume level. In MAX/MSP and Reason, we can easily hear and see the consequences of our logic thanks to their visualization environments and patch cords. Successful interfaces create successful relationships. I think that’s one of the reasons why “dampening” in Ajax applications seems to be so important right now. If people don’t witness the changes they’re making, they won’t feel empowered. And if they don’t feel like they’re in control, they’ll probably won’t be coming back.
The original GUI interfaces made a distinction. The real-world metaphor only applied to a very limited set of objects (nouns), whereas many Reason-like applications take that metaphor to extremes. From a pure interface design standpoint, they are horrible, and yet, because musicians are used to their real-world counterparts, the transition from “real” amp to “virtual” amp isn’t so bad. It would be rather easy to create an interface that was more efficient, however, that interface would violate the cardinal law of user experience and design: “Give the user what they expect”
Musicians use Reason. Sally in accounting, doesn’t. Propellerhead and others like them have come up with a specialised interface that allows musicians to work in an environment in which they feel comfortable. They “Give the user what they expect”.
Oh, and I wouldn’t necessarily consider a piano “easier” to learn than a violin. I play cello, and I took to it much faster than a piano. You really don’t want to hear me play piano.
I’d have to agree with Mr. Harrington. I play the saxophone, and previous to that the clarinet. I have little to no clue on how to plunk on the piano, though my oldest son has taken lessons for about a year and is doing quite well. Reading more than a single line of music frightens me. So the example of a piano being easier… I think that’s a personal example that doesn’t exactly pertain to all your readers.
As in web design, give the readers/users what they expect and they’ll use it quickly, no matter how easy or hard the GUI actually is to understand/use for a novice.
I have been discussing this for a while with an Interface Designer I work with. I showed him a few pictures of Traktor DJ Studio 3, a DJ software that uses a lot of knobs. That co-worker says that knobs on a software interface are “stupid”, because a knob is a real object that you can twist around with two finders, and you can’t do that with a mouse.
A agree with him, they are not used in similar ways, and I pretty much find it boring to keep dragging the mouse around on the knobs till they start to turn, but I don’t see how vertical sliders or any other control could solve this issue.
Now, about Reason, which I use too, the cables ideas is excellent specially because in the real world you can’t right-click a cable and have it show a list of ports it can connect to. A nice “reality extension” I might say.
Wonderful article, by the way!
Audiopad is very interesting since it’s really innovative. There is another article on it by IBM [1].
[1] http://www-306.ibm.com/ibm/easy/eou_ext.nsf/publish/4876
I disagree with CM & G R. :) I play viola, and while the piano may be a difficult instrument (I know I would have a lot of difficulty playing so many notes at once), it is SO much easier to produce a good tone (i.e. one that doesn’t make other people try to get as far away from you as possible). Anybody can sit down at the piano and make pleasing noises. Not so with a violin!
By the way, I think I used Max/MSP (or something very like it) for a class in the MIT Media Lab once. It was kind of fun to play with, though the Mac kept crashing (didn’t help that Macs were completely foreign to me). It was hooked up to a keyboard and synthesizer, and I “wrote” a little program that would “listen” to a short phrase played on the keyboard, and play back a similar, but different response. So basically it would let you play an improvised duet with the computer.
I have played the violin (a few years in junior high), the piano (since age 9), and have integrated Reason into my digital studio in the last couple of years (I’m now 41).
Reason’s most recent upgrade (3.0) significantly extended the ability to integrate the software with hardware devices called control surfaces. This seems to be the direction that software based music will continue in for the next few years.
A control surface might have a piano style keyboard or it may have drum pads, but typically either will have a quantity of rotary knobs and sliders that can be mapped (very easily) to the virtual synth’s knobs and sliders. Thus, you control the software with the hardware, but this time, instead of the 104-keyboard and mouse setup, you have something with which you can easily set variable values.
M-Audio makes control surface, as does Korg, Roland, Yamaha, and many others.
Here’s an example of the M-Audio control surfaces, just to give you an idea.
M-Audio control surfaces