Toward More Sensitive Mobile Phones
Ken Hinckley and Eric Horvitz
Microsoft Research, One Microsoft Way, Redmond, WA 98052
Although cell phones are extremely useful, they can be ringing in response to simulated phone calls. CHOOSING A NOTIFICATION MODALITY annoying and distracting to owners and others nearby. We The first set of interaction techniques addresses the issue of describe sensing techniques intended to help make mobile choosing an appropriate notification modality for an phones more polite and less distracting. For example, our incoming call. Cell phones typically provide some phone’s ringing quiets as soon as the user responds to an combination of audio alerting (ringing, playing music), incoming call, and the ring mutes if the user glances at the vibrotactile feedback (vibrating battery), and a visual caller ID and decides not to answer. We also eliminate the indication on the display that there is an incoming call, need to press a Toften with caller ID information to help decide whether or ALK button to answer an incoming call by not to take the call. When the phone knows if it is being recognizing if the user picks up the phone and listens to it. held, and if it is likely that the user may actually be looking Keywords at the display, one can make a better choice of the Input, sensors, context-aware, mobile devices, cell phones notification modality (possibly including silence).
INTRODUCTION Quiet Ringing. When the phone starts ringing, if the user is
not already holding the phone, then simply touching the Mobile phones have become ubiquitous communication
phone automatically lowers the volume of the ring. Hence, devices, but they often employ naïve alerting policies that
once the phone attracts your attention, and realizes that it can transform them into nuisances. In particular, cell
has done so, it is no longer necessary to ―shout‖ and the phones typically are unaware of how they are being used.
interaction transitions to a more private, one-on-one setting Thus, they may act in a way that is disruptive to others or
by softening the volume. We also explored muting the ring, embarrassing to the owner. Many cell phones support
but this can lead to an ambiguous situation where the user profiles that allow the user to manually set an appropriate
cannot distinguish muting from a caller being disconnected. response for different contexts. However, the user must
remember to turn on the correct profile, and the user must Acknowledging and Ignoring Calls. Once the user has again remember to turn off the profile when it is no longer grabbed the phone, the next step is often to look at the relevant or appropriate. For example, a device erroneously display and see who the caller is. This is a naturally left in a meeting profile may lead to missed calls in a occurring ―gesture‖ that can be recognized by our sensors. subsequent outdoor context. Automatic sensing techniques As soon as (1) the user is holding the device and (2) the may help eliminate these kinds of problems . device is moved so that it is tilted towards the user, it is
very likely that the user has just looked at the display. At Furthermore, many interactions with cell phones can be
this point, the software flags that the user has demanding of cognitive and visual attention. Machinery
acknowledged the call—it is aware that the user has and perceptual apparatus endowing cell phones to
received the notification. If the user is already holding the recognize explicit, but natural and minimally demanding
phone at the time the call arrives, this acknowledgment gestures of use, offers a set of strategies that promises to
gesture triggers the quiet ringing behavior as above (if the limit the attentional demands of commonly used features.
user is already holding the phone, the fact that he is holding We have prototyped several interaction techniques by it does not indicate that he has noticed the incoming call). augmenting a Cassiopeia E105 PocketPC with three extra
The user can then accept the call using the call answering sensors (as detailed in ): a two-axis linear accelerometer
gesture described below. However, if the user chooses not (tilt sensor), a capacitive touch sensor that detects when the
to take the call, the natural response is to put the phone user is holding the device, and an infrared proximity sensor
back in one’s pocket, or back on a nearby table, for that detects range to nearby objects. Our device has no
example. In all of these cases after an acknowledgment, the cellular connectivity, but we prototype the interaction by
tilt angles change, or the user is no longer holding the
device, allowing the software to infer that the user is no
longer looking at the display. Once the software senses that
the user is ignoring the call, it mutes the ring, but leaves a
visual indication that there is an incoming call. This way,
To appear in ACM UIST 2001 Symposium on User Interface Software &
the user can still change his or her mind and decide to take
the call before the caller hangs up or drops into voice mail.
Together, these techniques use the naturally occurring
gestures implicit in using a cell phone to help ensure that
the phone quiets, and then mutes, as soon as feasible, thus
minimizing potential disruption to others while also being Backlight Activation. Picking up and looking at the phone certain that notifications get through if possible. turns on the display backlight for 15 seconds, allowing the
viewer to see the display without any extra action such as Target Device for Notifications. We use physical contact hitting a button. This gesture also turns on the phone if it is with the device (touch sensing) and any recent activity currently powered off . Furthermore, if the user is still registered by the tilt sensor to decide if a device is being looking at the display when the backlight goes off, a slight used currently, or has been used recently. The most recently RELATED WORK readjustment (quick change in tilt angles while holding the used device typically will be available for subsequent We recently reported related techniques for hand-held phone) turns the backlight back on. If the phone is held in notifications, in the absence of other evidence to the computers . Other than location awareness, we are aware an orientation where it is unlikely the user is looking at it, contrary. We have only constructed one prototype of our of little published work on sensing techniques specifically the backlight is turned off to save power. device, so we have not yet been able to fully explore this tailored to cell phones. Schmidt et. al.  describe a cell technique when multiple devices are present. phone that switches profiles by sensing when it is sitting on Vibration Notification. If the user is holding the phone, then a table, being held by the user, or being used outdoors, for the phone knows that the user can feel a vibrotactile example. One commercial product, the Ericsson R520 cell response and hence it is not necessary to distract others by phone , features a speakerphone mode with a proximity ringing as well. However, if the user is not touching the switch that automatically reduces the volume if you hold phone, the user still may be able to feel it vibrate– the the phone to your ear, but reverts to speakerphone when phone may be in a pocket, for example, where it is not you put it down. We have also implemented this technique being touched but its vibration can still be felt. Hence it on our prototype device; it seems to work well. should both vibrate and ring in that situation (as opposed to There are other approaches which could be combined with a similar technique proposed by Schmidt ). Our our work to address the problem of notification in context prototype device does not have a vibrating battery, so at as a whole. Sawhney & Schmandt  propose several present we visually indicate the ―vibration‖ notification. related techniques for dynamically adapting notification
modality and calculating a usage level, as well as audio
processing techniques which could be used to augment the
techniques reported here. Horvitz et al.  describe a REDUCING ATTENTIONAL DEMANDS notification architecture that uses probabilistic techniques
to prioritize notifications, allowing the system to either Call Answering Gesture. Current cell phones require a
suppress them or deliver them at an appropriate time, to an specific action in order to answer the call. Examples
appropriate device, using evidence such as the user’s include pressing a TALK button, which requires looking at calendar, or sensed events (keyboard, mouse, microphone). the button or searching for it by feel, or opening a flip cover,
which can be awkward to achieve with one hand, especially
when the user is already engaged in another real-world task.
When there is an incoming call, we use lifting the phone CONCLUSION and listening to it as an implicit gesture to answer the call. We have proposed techniques to help make cell phones less The gesture, which is the same as that used in our previous intrusive to others, and less demanding of the owner’s work to trigger voice recording on a PDA , is a attention. Such techniques provide examples of how the combination of holding the device, tilting it in a pose coming age of ubiquitous sensors may provide a richer, typical of talking into a cell phone, and detecting the head more sensitive user experience for mobile phones. in close proximity. We currently have no corresponding
gesture to hang up the phone, as putting down the phone
REFERENCES activates a speakerphone mode (, see below), and thus
1. Ericsson, R520 Mobile Phone, www.ericsson.com/r520 does not necessarily indicate that the user wants to hang up.
2. Hinckley, K., Pierce, J., Sinclair, M., Horvitz, E., Sensing Voice Recognition Context. At other times, when no Techniques for Mobile Interaction, ACM UIST 2000, 91-100. incoming call is pending, the ―answering‖ gesture can be
used to activate the built-in microphone for a voice 3. Horvitz, E., Jacobs, A., Hovel, D., Attention-Sensitive recognition context. Cell phones with voice recognition Alerting, Proceedings of UAI '99, Conference on Uncertainty capabilities are available now on the market: for example, and Artificial Intelligence, 1999, 305-313. the user can speak the name of a contact, such as ―Call Jeff 4. Sawhney, N., Schmandt, C., Nomadic Radio: Scaleable and at home.‖ However, the user must speak a magic word or Contextual Notification for Wearable Audio Messaging, press the TCHI'99, 96-103.
ALK button to trigger the functionality. Our 5. Schmidt, A., Aidoo, K., Takaluoma, A., Tuomela, U., Van approach eliminates the need for these extra steps. We have Laerhove, K., Van de Velde, W., Advanced Interaction in not implemented voice recognition on our device; we Context, Handheld and Ubiquitous Computing '99, 89-101. currently only record the utterance for testing purposes.