Matt Gemmell

Touch Notation

8 min read

Within the last twelve to eighteen months, I’ve crossed a threshold whereby the vast majority of my work is now aimed at touch-screen devices. I often have to sketch out feature specs, interaction designs and so forth, and I enjoy working on paper whenever I can. I quickly encountered a problem: touch-screen gestures are difficult to describe concisely. To solve this problem, I created a means of talking about such gestures symbolically; I call it Touch Notation.

There are three primary problems I face when talking about multi-touch gestures in documents:

  1. Elaborateness. To visually describe a gesture, it’s fairly common to see actual life-like drawings of fingers and hands; that’s far too much trouble to sketch, and takes up too much room when reproduced. It also doesn’t scale to small sizes.
  2. Verbosity. To instead textually describe a gesture, disproportionately many words are required: “double-tap-and-hold the object then drag to the right”. That’s tiring, and again takes up far too much room.
  3. Ambiguity. There’s no absolutely standard mapping of terminology to gestures, so we’re faced with deciding whether we really want “swipe”, “flick” or something else, and whether the term will be understood later.

Such situations are usually helped by creating a formal terminology or notation of some kind, and indeed we all find ourselves doing so automatically and organically in our own notes. My own notation has developed over the course of a few months, and has stabilised sufficiently to be worth sharing; I find it very useful. Touch Notation’s purpose is to help me talk about and design features and interactions for touch-screen (and usually multi-touch) software.

That’s not all it’s useful for, though – I also use it to display gestural input in a concise, reference-suitable way. This usage is analogous to lists of keyboard shortcuts on the desktop, which make use of standard modifier-key symbols like Shift, Alt, Ctrl etc.

Touch Notation lets me make gestural-input “cheat sheets” for touch-screen apps just as easily as I can make lists of keyboard shortcuts for desktop apps. The goal is to provide one possible way of talking precisely and accurately (I hesitate to quite say “scientifically”) about touch-screen interaction. I thought it might be of interest to you too, so I’ve decided to share it.

Design Priorities

When developing the notation, my priorities were:

  • It should be (relatively) easy to sketch quickly, and definitely easier than textual descriptions or finger-drawings.
  • It should be scaleable, so it can be reproduced at any size (particularly at small sizes).
  • Each symbol should be unambiguous within the set.
  • We should prefer using the simplest, strongest forms for the most common gestures.
  • All symbols should fit within constant vertical space, and have similar vertical visual weight.

These are simple, utilitarian and common-sense constraints, with an emphasis on making something that’s usable rather than conceptually perfect (which is almost always the right approach for anything).

Fingers vs Gestures

Before I list the symbols, I want to briefly mention the number of fingers issue. I’ve seen plenty of sketches and icons where, for example, a two-finger swipe-right is represented as two right-pointing arrows, one above the other. That’s fine in itself, but it completely breaks scaleability at small sizes – particularly if you’re requiring a four-finger swipe for some reason.

I wanted to avoid that problem with Touch Notation, so my approach was to separate the number of fingers from the gesture. After all, a tap is conceptually the same “gesture” whether you’re tapping with one finger or with two. I specify the number of fingers first (which is optional for the most common case of only one finger), then the actual gesture (with a gap between the two sections).

There are a few alternate notations I’ve experimented with for showing the number of fingers:

  • Number Only. This is the one I actually use on a day-to-day basis, because it’s as simple as writing/typing a number.
3 fingers; number only
  • Number and Symbol. As with Number Only, but followed by a “Finger” symbol (shown below in the full list of symbols). Since the Finger symbol is a sort of vertical lozenge shape, you’d likely want to distinguish this from (say) the numeral 1 by using colour or some such thing.
3 fingers; number and symbol
  • Multiple Symbols. No number, and instead repeating the Finger symbol the appropriate number of times.
3 fingers; symbols only

It really depends what you find most useful or pleasing. There’s actually less visual ambiguity from the Finger symbol than you might think, since (for example) confusing it for the numeral 1 doesn’t change its meaning in isolation, and in Number and Symbol style you’re unlikely to think it’s an additional numeral 1 after the actual number because it’s very uncommon indeed to require a double-digits number of fingers to perform a gesture!

Personally, I find the Number Only style to be the easiest to write, and least visually ambiguous at smaller sizes. I’ll be using it in all the examples below.

Touch Notation Symbols

Here’s the full set of symbols, and their meanings.

Tap

Tap symbol; a filled circle

A standard tap (and release).

Tap and Hold

Tap and Hold symbol; a hollow circle

A tap and hold for a moment (without releasing). This gesture is sometimes also called a long tap.

I deliberated for a while about whether this symbol should actually be Tap, and the solid circle be Tap and Hold (the hollow area representing releasing the tap, in that case), but I decided against it since Tap is by far the most common gesture and I wanted it to have as strong and simple a form as possible.

Swipe

Swipe symbol; a filled arrow

A swipe or flick across the screen, usually with no particular origin. The arrow is rotated to indicate the actual direction of the swipe.

Rotate

Rotate symbol; two small, filled circles with curved arrows pointing clockwise

A rotation gesture, involving at least two fingers.

Pinch Out

Pinch Out symbol; two diagonally-opposed filled arrows

An expansion/pinch-out gesture, involving at least two (and usually exactly two) fingers.

It’s worth noting that the Pinch Out/In symbols are horizontal reflections of the fairly-standard “Enter/Exit Full Screen” icons used on Mac OS X and elsewhere. The context of use will remove any ambiguity, and the symbols are too meaningful to pass up on that basis alone.

Pinch In

Pinch In symbol; two diagonally-facing filled arrows

A pinch (inwards) gesture, involving at least two (and usually exactly two) fingers.

Tap and Drag

Tap and Drag symbol; a filled arrow with a filled circle at its tail

A tap and then drag, without releasing and without any explicit pause between tapping and dragging. This is distinct from a Swipe in that a Tap and Drag usually has a specific origin point: the object you’re dragging.

Tap, Hold and Drag

Tap, Hold and Drag symbol; a filled arrow with a hollow circle at its tail

A tap, hold for a moment, and then drag (commonly involving a transition in object-state after the hold period has elapsed).

Finger

Finger symbol; a vertical, filled rounded-rectangle or lozenge

The symbol for a finger, which can optionally be used when specifying the number of fingers required for a gesture (as described above).

Examples

Here are a few examples of using Touch Notation to encode various gestures. When I’m creating formal documents using Touch Notation, I tend to use black for the number of fingers and blue for the entire gestural part, just to give a subtle sense of energy and motion to the gesture. You’ll find that the gesture instructions in tutorials in the iPad versions of the iWork apps do the same thing.

  • Tap (with 1 finger)
Tap with one finger
  • Double-tap with 3 fingers
Double tap with three fingers
  • Swipe right with 2 fingers
Swipe right with two fingers

I find that the best way to read these gestures is to first read the black number, prepare that number of fingers, then follow along with the blue symbols of the gesture: “with this number of fingers, do that”.

Quantity Modifiers

I sometimes want to specify that, say, the user will tap and hold for two seconds, or will rotate for 45 degrees. I use leading or trailing numbers to indicate these situations, with the position of the number depending on the actual affinity of the value to the symbol. There are a couple of examples below.

  • Tap and Hold for 2 seconds, then drag
Tap and Hold for two seconds, then drag
  • Tap and Hold (for some default period), then drag for 2 seconds
Tap and Hold, then drag for two seconds

Sequential and Split Input

By default, since we’re dealing with multi-touch gestures, two consecutive tap symbols (for example) indicate performing two taps quickly, i.e. a double-tap. Sometimes, however, I want to express that the user should tap, wait a moment, then tap again – as two sequential gestures. I use the comma for this, since it has a fairly standard meaning of indicating a brief pause.

  • Double Tap
Double Tap
  • Tap, then Tap again
Tap then Tap again

In touch-screen gestures, there’s also the (not so well-known) concept of a split tap; this is where you’re performing one action with one hand, and then (simultaneously) perform an action with the other hand, as a single gesture.

This is commonly used in VoiceOver, where for example the user can roll a finger over the keyboard until they hear that they’re arrived at the correct key, then tap a finger of their other hand anywhere on screen to enter that key. Split taps in VoiceOver allow for much easier text-entry where precision is only important for one hand.

I think the general concept of split input is interesting and useful, so I included it in the notation: I use a forward-slash to indicate the split. I don’t consider this to create ambiguity, since I only ever encode one gesture in a notation sequence, so I wouldn’t be using a forward-slash to mean “or”, as it can in English.

  • Tap and Hold, then Drag, and then Tap elsewhere with another finger
Tap and Hold, then Drag, and then Tap elsewhere with another finger

Download your own set of Touch Notation symbols

I find this notation useful, and you’re welcome to use it too. If you use it anywhere other than in-house, I’d appreciate attribution. I’m offering a Photoshop (.psd) file containing each symbol as a Shape Layer (with all symbols at their original relative sizes), a Photoshop Custom Shapes (.csh) file so you can use Touch Notation in any Photoshop document, and also the original Illustrator (.ai) file I created the symbols in.

You can download Touch Notation here.

License info is included with the download; it’s a Creative Commons Attribution-ShareAlike license.

Oliver Toscan also made an OmniGraffle Stencil of the Touch Notation symbols; you can download it here.

Final Thoughts

I’ve found Touch Notation to be really useful in my own work; I just keep a quick symbol-reference around (though I rarely need to refer to it now) for when I’m sketching, and I’ll occasionally make formal lists of gestures for features electronically using the templates. I find that it has just the right balance of simplicity and expressiveness that I actually do use it.

I hope you find a use for it in your workflow, or at the very least get some value from comparing it to your own notation style; I always welcome any feedback you may have. If you’re feeling grateful/generous, feel free to take a look at my Amazon Wishlist, or the Paypal Donate link on my Cocoa Source Code page.

Remember that you can always keep up to date with what I’m doing by following me (mattgemmell) on Twitter. Now, back to work.