Institution: Synthesis / David Reisner Consulting
Principal Investigator: David Reisner
E-mail: dar at synthesis.com
General Type of Work: Human-Oriented User Interface Design
Projects and Future Work:
We are interested in the creation of user interfaces that are
truly human- (as opposed to computer-) oriented. We are
especially interested in interfaces which make much better
use of human world-models and the perceptual system,
including "peripheral" perception - information acquired
about environment outside of the area of primary attention or
focus thru "background" or "ambient" sound, peripheral
vision, and the "subtle" action of other senses. We are
attempting to design user interfaces which use these
processes to provide information about a computer or
In a recent experiment, we added a layer on top of the Unix
c-shell command interface running under X windows. Upon
completion, each user command made a unique sound - a "chord"
based on the literal text of the command. Informal
observation suggested that even this simplistic mechanism
allowed users to monitor the status of several simultaneous
commands with reduced attention.
We are presently considering systems that use frequent or
continuous audio cues. Using audio loosely modeled on natural
sounds, we hope to have system activity sound like a
"cocktail party" - a general but informative babble, with the
ability to focus on any of a large number of pieces of
information from moment to moment. We hope to use "timbral",
temporal/rhythmic, and spatial cues, coupled to display
systems with several large screens and an analogous visual
For sonification of complex data, in addition to using
characteristics of sound to represent additional degrees of
freedom, we are interested in representing "derived"
information like rates of change or "smoothness" of a
function, differences between multiple data sets, or user
defined conditions/constraints (e.g. values within ranges).
We are also interested in the use of audio in multi- and
interactive- media and in understanding the evolution of
We have found that mu-law audio encoding does not work well
for some classes of sound processing. The presentation of
sound with complex timbres or many distinct parts requires
reasonable resolution and bandwidth audio hardware (e.g. 16
bit, 44.1KHz) not available on many installed workstations.
For improved interfaces we would also like to see much
larger, high-resolution display systems and more autonomous
physical interface devices.
Even at low bandwidths significant sound processing is
expensive. We believe that the presentation of complex sound
(and visuals) representing dynamic systems or on-going
computations will greatly benefit from multi-processor
systems, with a full special or general purpose processor
dedicated to the audio (visual) interface.
Synthesis / David Reisner Consulting
provides research, design, and
implementation in diverse areas including: digital audio,
user interface, Unix systems, and the entertainment industry.
Projects range from killer whale training using sound, to an
early "clipboard" computer (`80), digital recording systems,
biological fingerprinting, robot control, and one of the
first retargetable source-level debuggers. Clients include
Sun Microsystems Laboratories, Time-Warner, and Sea World.