Sound in Product Design
Thesis Final Draft – 12/1/08
Advisor Dave Kaufer
As designers, we are fundamentally concerned with solving problems through
the medium of products. Solutions (and by extension products) can be good, bad,
or somewhere in between. If we hope to be good designers, we strive to create
good products. But what makes a product good? This question has been widely
debated and has a multitude of equally viable answers. Personally, I am fond of the
“useful-usable-desirable” triumvirate, which states that products should serve a
needed function in an efficient and painless manner while providing an experience
that people want.
I did not set out, however, to write a paper on what makes a good product; I
1set out to write a paper on non-speech sound in products. I began by thinking of
the different ways in which I could talk about sound. I could argue, for instance,
that sound is worthwhile in-and-of-itself and we simply don’t use it enough. I could
also argue that many products do not convey information efficiently enough – that
the proper use of sound in products could greatly enhance products’ ease and
efficiency of use. Finally, I might argue that digital products often lack a certain
richness incumbent in many of their analog predecessors because they do not
properly engage our sensory modalities. I realized, however, that what I really
wanted was to discuss good products and why sound is an element of them.
Fortunately, I had already found my argument. Sound makes products useful –it is
a powerful tool for making products that serve a function people want to have.
Sound makes products usable - through the careful inclusion of sound in products
we can enable unrealized efficiencies. Sound makes products desirable – sound is
an integral part of any truly rich experience, and it is rich experiences that people
desire. This paper will focus on how sound can be used to create useful, usable, and
1 In the interest of brevity, I will hereafter refer to non-speech sound simply as “sound.”
The first question to examine, however, is why sound is even worth talking about. Clearly, designers should use every tool available to them (when appropriate)
to create good products. Sound is one of these tools, and is a particularly
interesting one at our current stage of technological development. For the first time,
digital technology is at a point where high quality sound can be cheaply included in
essentially any digital device.
On the other hand, sound has rarely received much attention by product designers and the design community at large. This is not surprising, given that
Interaction Design as a discipline arose largely from the fields of human-computer
interaction (itself based in computer science), and communication design (which
has traditionally been concerned with issues of visual display and aesthetics).
Neither of these realms of study have a history of working with sensory modalities
other than vision. The result is that sound is rarely used in modern products and
often implemented badly when it is present.
When design professionals have examined sound, the focus has been function. In the 1970’s, Deatherage (a human factors researcher) published an
early framework for considering sounds in human-operated equipment. His
guidelines, which remain enlightening today, focus strictly on the use of sound for
functional purposes and rest largely on the notion that sound is most appropriate as
a means of transmitting information while relieving the eyes (1972).
In the 1980’s sound began to receive attention again, beginning with the work of Sarah Bly. In 1982, Bly demonstrated the functional use of data sonification
techniques. Bly experimentally determined that sonic representation could improve
people’s ability to find patterns in certain data.
In 1989 a special issue of the journal Human-Computer Interaction was
devoted to non-speech sound at the interface. The issue contained an introduction
by Bill Buxton and three important articles by Bill Gaver, Meera Blattner et. al, and
Alistair Edwards. Gavers article detailed the SonicFinder, an auditory overlay for the
Apple Macintosh operating system that revolved around the use of everyday
listening skills (dubbed “auditory icons”) to transmit information at the interface.
Blattner et al introduced the concept of “earcons” - single pitches or rhythmic
sequences of pitches used to represent information. Finally, Edwards detailed the
design and testing of an auditory word processing program for blind users. All of
these designs focused on methods for transmitting discrete information with sound
2 rather than talking about sound as means for creating better products.
In the 1990’s research into sound focused primarily on computer interfaces and remained an inquiry into its functional potential. Beginning in 1993 the
International Community for Auditory Display held annual meetings, publishing the
proceedings of their first conference in a volume edited by Gregory Kramer. This
publication, which remains a key resource today, contained articles related to
psychoacoustics, data sonification, auditory icons, earcons, and sound synthesis
methods (1994). All of the articles, however, were concerned with how to use
sound, rather than developing a framework for considering sound as an element of
Other research has also provided interesting frameworks for discussing sound. Ferrington has proposed a set of five categories of information that can be
presented with sound, including physical activities, invisible structures (thing that
cannot be seen), dynamic changes, abnormal structures (invisible structures that
are not functioning as expected), and events in the world (spatial information)
(1994). Macauley et al. revised these categories to include visible entities and event,
hidden entities/event, imagined entities (considering what a sound might mean),
patterns of events, the passing of time, emotion, and position in space (1998). Both
of these frameworks center on the transmittal of information.
The issue I take with the majority of research done to date is an over-great emphasis on the functional use of sound, and too little focus on how sound can
positively impact the gestalt of a user’s experience with a product. Researchers
have provided many examples of how sound can be used functionally and written at
length of audio and audition’s unique properties, but few have given much formal
thought to the role sound plays as an element of user experience (outside of strict
functionalism). If we consider, instead, the design community’s treatment of visual
2 This is not to say that no mind was paid to why sound should be used. Gaver, notably,
began his paper with a section titled “Why Use Sound?” The focus of his paper, however,
was on how sound could be used to provide information, rather than how it improved the
interfaces, we find a focus on the way people interact with products, rather than
the information that can be provided. Though much work focuses on our perceptual
abilities as they relate to visual information and the types of information that can
be so provided, great emphasis is also given to using visual interfaces as a tool for
creating engaging interactions. By framing the issue of sound in products through
the useful-usable-desirable lens, we can begin to better see how people interact
with sound at present, and how this contributes to the creation of good products.
The useful-usable-desirable placements are an effective tool for considering
the gestalt of product experience because each overlaps, contributes to, and
influences the others but can also be examined discretely. If a product is not useful,
it cannot be easy to use (usable), or particularly desirable. On the other hand, if a
product serves a truly needed function that cannot be satisfied in other ways, it will
likely be desirable even if it is not very usable. Similarly, a product that is not
usable will not generally be desirable unless no substitute exists and may be
useless if its usability problems are extreme. Desirableness can influence usability
and usefulness as well – if a user finds a product desirable enough, this will influence their perception of the product’s usefulness and efficiency as well. By
considering the three dimensions together and looking at how they overlap in
regards to sound, we can paint a clearer picture of audition’s role in products.
It is important to recognize that “useful-usable-desirable” is a set of
perspectives for considering all aspects of any product while designing. The terms
do not represent categories, but rather is a means for designers to place their
thinking about a design. An Eames Lounge chair can be considered useful insofar as
it affords sitting, usable because it is comfortable and small enough to fit through
the average home doorway, and desirable because it is beautiful. None of these
stands alone, however. As Don Norman has argued, for example, beauty actually
improves users’ perception of usability (2004). Designers Jonathan Cagan and Craig
Vogel (who popularized the useful-usable-desirable framework), have argued that a
product is only truly valuable when it excels in all three areas (2002).
This paper will make the case for sound by examining the many ways in
which sound can be used to create or promote usefulness, usability, and desirability
in products, with a focus on how the interplay between the three creates engaging
interactions. I cite research where it exists and give examples (both good and bad)
as often as possible. It is my hope that after reading this paper the reader will
3 agree that sound is an important consideration for anyone designing a product.
Sound and Usefulness
Merriam-Webster defines useful as “capable of being put to use; especially:
serviceable for an end or purpose.” Alternatively, Cagan and Vogel have defined a
useful product as “one that satisfies a human need, is capable of being produced at
reasonable cost, and has a clear market” (2002, p. 57). In design, useful might be
considered the basest aspect of products because it is entirely relative to context. A
scrap of cloth may be a useful filter if one is lost in the desert and comes across a
dirty puddle, but would never be considered so in a normal situation. Furthermore,
usefulness is a vague concept because one can reasonably take a broad definition
of the term. I own, for instance, a decorative treacle can. I purchased the can some
years ago in London and I have never put it to any particular use (I poured the
treacle down the drain), but it may be considered useful because I like it. It
performs the function of sitting on a shelf collecting dust and stirring certain
emotions and memories in me when I gaze upon it.
On the other hand, usefulness is potentially the most important aspect of any product. If a product truly serves no purpose, then it cannot, by definition, be
usable and will be desirable to very few. I define usefulness in regards to products
as “serving a particular needed function.” That is, for a product to be useful it must
serve as a tool for a specific and required purpose.
Sound is an important tool for creating useful products. Sound can be a tool by itself and can also augment other modes of communication to allow functionality
that could not otherwise occur. In order to best understand how sound can
contribute to usefulness, it is helpful to first examine some unique properties of
sound and audition.
Stephen Brewster, a prolific researcher of sound in digital interfaces, has proposed a number of “advantages offered by sound,” that elucidate sound’s
3 I have an inclusive definition of product, using it to refer to anything human-made and
monetized, whether physical artifact, service, or system.
functional capacity. One of the most well-known aspects of sound is that it is attention grabbing and does not require focus: people can choose not to look at something, but it is much more difficult to avoid hearing it (2007). Our ears are always on, even (to some extent) while sleeping. This aspect of audition no doubt helped us survive as a species in earlier stages of our development, and continues to offer important opportunities for product designers today.
In particular, the “always on” nature of sound means that it serves as a
highly effective medium for alarms and alerts. From the toaster in the kitchen, to fire alarms and phone calls, designers have taken advantage of sound as a way to inform users that some event has occurred that may require immediate attention. Generally speaking, this use of sound has fallen into two categories: alerts signaling completion of an operation, and alarms drawing attention to an immediate concern. The first category includes products such as microwaves, 1960’s Teletype machines (which rang a bell after a transmission was completed), and teakettles. The second category includes fire alarms, ambulance sirens, and alarm clocks. Sonic alarms work well because sound does not require focus or line-of-sight. Without sound, we would not see that an alarm had gone off until we actually looked at it.
Perhaps because alarms and alerts are one of the most common uses of
sound, they are also the most widely abused. Anyone who has used a PC has undoubtedly suffered the jarring noise of a Windows alert. More often than not, these alert sounds accompany a modal dialog box that pops up in front of whatever program is currently active. In such cases, the visual interface suffices to alert the user, making the sonic alert redundant. Redundancy in alerts is only appropriate when there is substantial reason to believe that the user may not otherwise notice the alert message, which is clearly not an issue when a modal dialog box pops up over a piece of software that a user is currently engaged in using.
The Windows example also illustrates the interdependent nature of
usefulness, usability, and desirableness. Cooper and Reimann have pointed out that error alert noises act as “a public announcement of a user’s failure” because they are “negative audible feedback” (2003, p. 453). Many alert sounds, which serve a specific function, therefore reduce the desirability of a software product (because
no one wants their failures publicly announced), and may even harm usability if
users go out of their way to avoid them.
A much better implementation of sound as alert can be heard in Google’s Gmail. Gmail, a web-based email application that allows a user’s contacts to initiate
a chat session any time one is logged on, makes a sound (redundant to a
simultaneous visual alert) when a chat message arrives. The sonic alert is
appropriate in this case because people often keep their email open while they work
in other applications (and thus are unlikely to see the visual alert). Furthermore,
the sound represents a positive rather than negative alert. In this example, sound
is used to serve the function of alerting a user of an immediate concern, and in so
doing also increases the usability of the application (by making it easier to perceive
The “always on” nature of our ears also means that humans are very good at ignoring sounds that they do not want to pay attention to (what Brewster has called
habituation) (2007). John Cage is said to have once sat in an anechoic chamber for
some time. When he exited he remarked to the engineer on duty that he heard two
distinct sounds – one high and one low. The engineer told him that the high-pitched
sound was his nervous system and the low one his circulatory system. Most of us,
however, rarely (if ever) perceive these sounds because we have learned from birth
to ignore them and are usually surrounded by other louder sounds regardless. The
implication of our propensity for ignoring sounds is that sound can be a useful
medium for providing ambient information because a well-designed continuous
4 sound is often not noticeable until it changes or stops.
Automobile engines, for example, produce a wide range of sounds that we rarely pay attention to. Sometimes, however, they produce unusual sounds,
signaling that a trip to the repair shop is needed. Another example is the clicking of
4 The greatest danger in any use of ambient sound is, of course, annoyance. People often
complain that they find product sounds annoying or distracting. The problem can most often
be circumvented, however, by paying careful attention to the design of the sound itself and
the sonic ecology in which it will exist. Cooper has argued that the reason users complain of
annoyance is that most interface sounds are used almost exclusively for negative feedback
(2003). Alternatively, Brewster has argued that loudness of interface sounds is primarily to
a computer’s hard drive as the disk is accessed. We rarely notice this sound, but can listen for it at will to know whether their computer is reading or writing to disk. The use of ambient sound to gain information is actually a basic fact of human psychology. Our ears constantly scan our physical environment, ignoring sounds that provide little information and alerting us to ones that might signal something more interesting.
In a functional manner, ambient sound is most commonly found in
mechanical products. Sound, after all, is only vibrations in the air, and so anything with moving parts will produce some sound. Ambient sound is often how we
discover and diagnose mechanical problems, be it in a moped or a laser printer. Skilled mechanics, for example, can often properly diagnose an engine problem
from the sound alone. In this way, sound serves the purpose of providing users with information that cannot be easily obtained by other means. Again, this use of sound may also be considered an aspect of usability because it makes solving these mechanical problems easier, or even desirability because consumers are likely to prefer a product that can be fixed easily over one that cannot.
Ambient sound is also commonly used by retail stores, which play a
particular kind of music to help flavor their brand. Starbucks has taken this to an extreme, publishing albums of the music played in their stores, perhaps as a tool for securing brand loyalty. Similarly, nightclubs often base much of their brand in the music the play, using the sounds of their establishment as a primary lure to customers. Sound is thus used as a tool with the purpose of enhancing brand.
Alternatively, this use of sound makes products more desirable by helping to create an appealing brand image in customers’ minds.
It should be mentioned that the ambient use of sound for functional purposes
can have a formidable negative impact on a product’s desirability as well. If ambient sounds are not providing desired information, then they quickly become noise. For this reason designers must be very careful when designing ambient
sound. Such sounds must be acoustically designed to minimize the risk of becoming noise and should generally turn themselves off if the information being provided is unlikely to remain continually helpful.
Another aspect of sound and audition that proves useful is its “superior
temporal resolution” (Kramer 1994, p. 9). That is, people are capable of perceiving
shorter duration sound events than visual. Animations generally run at thirty frames per second because humans cannot readily perceive visual stimuli that are shorter. The human ear, however, easily perceives sonic events of less than 10 milliseconds in length. Perhaps because of this, humans can sometimes react faster to auditory stimuli that to visual. Our auditory system is thus well suited to real-time monitoring of information for which reaction time is a factor.
Sound is extremely effective at imbuing products with emotion. This aspect of sound is closely related to desirability, but can be considered a function as well. Video games have used the emotionality of sound to great advantage. The simple music of Space Invaders sped up as the player progressed through the game, creating a purposefully heightened state of anxiety in the player and adding intensity to the experience of play. More modern 3D and virtual reality games use recognizable sound effects to inform players of events occurring outside of their direct field of virtual vision and also make use of music to enhance the emotion of
5 Sound in these examples, serves the users’ need for intense in-game situations.
engagement with a game.
Thus we see that sound is fundamentally useful because it enables products
to serve purposes that are otherwise difficult or impossible to facilitate. Sound can grab attention in ways that are impractical via other sensory modalities, serving effectively as an alert or alarm, can be used to display real-time events too short to be perceived by the eye, is well suited to serving up information ambiently, can enhance brand, engender specific emotional response with surprising accuracy, and provide information about events that are impractical to display visually.
5 Many modern role-playing games, for example, have cut scenes in which animation, dialog, and music are used much as they are in film.
Sound and Usability
According to Cagan and Vogel “a usable product is one that is easy to
operate, easy to learn how to operate, and reliable” (2003, p.57). The Usability
Professionals’ Association defines it as “the degree to which something - software, hardware or anything else - is easy to use and a good fit for the people who use it”
(2008). There are many definitions of usability, but almost all center around ease of
use and, when relevant, efficiency. Generally speaking, usability is measured by
asking users to relate their subjective satisfaction with a product, and by recording
a user’s success and error rate in performing tasks with a product and measuring
6 the amount of time a task requires for completion.
Sound provides great potential for making products easier and more efficient
to use. In particular, sound is a wonderful mechanism for giving feedback and
providing predictive information, compliments graphic user interfaces well, and can
make products more accessible to the visually impaired.
Humans have used hearing to obtain feedback as long as we have been a
species. Auditory feedback is present in any physical product with which we interact:
we know when we have finished tearing a paper towel off of a roll from the sound
as much as the sight and touch; we know that a bottle of Windex is working from
the squirting sound that it makes when we pull its lever; we know that we are
7indeed scrolling on our ipod from the clicking of the wheel.
Jonathan Grudin, a well-known HCI researcher, relates the following story
regarding sound and feedback in digital products in the early 1980’s:
“Computer keyboards no longer clicked the way mechanical and electric
typewriter keyboards did, but people expected the click, so we programmed
the computer to make a clicking sound. The question that arose was, when
someone tried to type text into a read-only field, should one click or not? By
not clicking you could signal that the key wasn't accepted, but you would
violate the mechanical key metaphor. Which would people prefer? … it was
decided to issue the click (preserving the illusion it was a mechanical sound)
6 Well known usability expert Jakob Nielsen has written at length on usability metrics, some
of which can be found at http://www.useit.com/alertbox/20010121.html.
7 Interestingly, the scroll wheel on many iPods is designed to produce both a physical
clicking sound and an electronically generated one, thus providing feedback to the user
whether or not they are wearing headphones.