Touching Technology
taking the physical world seriously in digital design

Notes of talk given, in slightly different variants at:

Winter School for "Interactive Technologies for the End-User", Bangalore, 2nd–3rd February 2009,

Greek SIGCHI Workshop “Human-Computer Interaction: Theory and Practice of design of usable and accessible technologies”, Athens, 5th March 2009,

Seminar at Department of Computer Science and Technology, University of Peloponnese, Tripolis, 9th March 2009,

Also in longer form at:

Swiss Universities’ Winter School on "Usability Engineering", Anzere, 26th_30th January 2009,

Abstract

Although we live in an increasingly digital world, our bodies and minds are designed to interact with the physical. When designing purely physical artefacts we do not need to understand how their physicality makes them work - they simply have it. However, as we design hybrid physical/digital products, we must now understand what we lose or confuse by the added digitality. With two and half millennia of philosophical ponderings since Plato and Aristotle, several hundred years of modern science, and perhaps one hundred and fifty years of near modern engineering - surely we know sufficient about the physical for ordinary product design? While this may be true of the physical properties themselves, it is not the fact for the way people interact with and rely on those properties. It is only when the nature of physicality is perturbed by the unusual and, in particular the digital, that it becomes clear what is and is not central to our understanding of the world. This talk discusses some of the obvious and not so obvious properties that make physical objects different from digital ones. We see how we can model the physical aspects of devices and how these interact with digital functionality.

References

1. [[AE09]] AHRC/EPSRC. Designing for the 21st Century. AHRC/EPSRC research initiative, (accessed march 2009). http://www.design21.dundee.ac.uk/
2. [BS03]] Benford, S., H. Schnadelbach, B. Koleva, B., Gaver, A. Schmidt, A., Boucher, A., Steed, R. Anastasi, C. Greenhalgh, T. Rodden and H. Gellersen (2003). Sensible, Sensable and Desirable: A Framework for Designing Physical Interfaces, Technical Report Equator-03-003, Equator, 2003.
3. [[BS05]] Benford, S., H. Schnadelbach, B. Koleva, B., Gaver, A. Schmidt, A., Boucher, A., Steed, R. Anastasi, C. Greenhalgh, T. Rodden and H. Gellersen (2005), “Expected, sensed, and desired: A framework for designing sensing-based interaction”,ACM Transactions on Computer-Human Interaction (TOCHI) journal, Volume 12, Issue 1, March 2005, ACM Press, pp. 3-30.
4. [[BC07]] Bertini, E., T. Catarci, A. Dix, S. Gabrielli, S. Kimani, G. Santucci (2007). Appropriating Heuristic Evaluation Methods for Mobile Computing. Chapter XLVI in Handbook of Research on User Interface Design and Evaluation for Mobile Technology, J. Lumsden (ed.) Information Science Reference also published in the International Journal of Mobile Human Computer Interaction, Vol. 1, Issue 1, pp. 20-41
5. [[CF03]] Cheverst, K., D. Fitton and A. Dix. “Exploring the Evolution of Office Door Displays”, in: Public and Situated Displays: Social and Interactional aspects of shared display technologies. K. O’Hara, M. Perry, et al (Eds). Chapter 6, pp. 141-169, Kluwer. ISBN 1-4020- 1677-8. 2003.
6. [[Cl98]] Clark, A.: Being There: Putting Brain, Body and the World Together Again. MIT Press, Cambridge, MA (1998)
7. [[Co00]] Cotterill, R. Enchanted looms: conscious networks in brains and computers. Cambridge University Press, 2000, ISBN 0521794625, 9780521794626
8. [[De97]] Dehaene, S. (1997). The Number Sense. Penguin, London.
9. [[DE09]] DEPtH, “Designing for Physicality", AHRC/EPSRC Project AH/E507646, http://www.physicality.org/DEPtH/
10. [[Dx91]] Dix, A.: Status and events: static and dynamic properties of interactive systems, Proc. of the Eurographics Seminar: Formal Methods in Computer Graphics, 1991. http://www.hcibook.com/alan/papers/euro91/
11. [[Dx94]] Dix, A. (1994). Que sera sera - The problem of the future perfect in open and cooperative systems. Proceedings of HCI'94: People and Computers IX, Eds. G. Cockton, S. W. Draper and G. R. S. Weir. Glasgow, Cambridge University Press. 397-408. http://www.hcibook.com/alan/papers/loop-HCI94/
12. [[DA96]] Dix, A. and Abowd, G.: Modelling Status and Event Behaviour of Interactive Systems, Software Engineering Journal, 11:6, 334–346 (1996). http://www.hcibook.com/alan/papers/SEJ96-s+e/
13. [[Dx00]] Dix, A. (2000). Welsh Mathematician walks in Cyberspace (the cartography of cyberspace). (keynote) Proceedings of the Third International Conference on Collaborative Virtual Environments - CVE2000. ACM Press. pp. 3-7. http://www.hcibook.com/alan/papers/CVE2000/
14. [[DR00]] Dix, A., T. Rodden, N. Davies, J. Trevor, A. Friday, K. Palfreyman (2000). Exploiting space and location as a design framework for interactive mobile systems. ACM Transactions on Computer-Human Interaction (TOCHI), 7(3), pp. 285-321, September 2000. http://doi.acm.org/10.1145/355324.355325
15. [[DC04]] Dix, A., K. Cheverst, D. Fitton and A. Friday (2004). The auditability of public space - approaching security through social visibility. 2nd UK-UbiNet Workshop Security, trust, privacy and theory for ubiquitous computing. 5-7th May 2004, University of Cambridge, UK
16. [[Dx05]] Dix, A. (2005). the brain and the web - a quick backup in case of accidents. Interfaces, 65, pp. 6-7. Winter 2005. http://www.hcibook.com/alan/papers/brain-and-web-2005/
17. [[DL07]] Dix, A., Leite, J. and Friday, A.: XSED – XML-based Description of Status–Event Components and Systems, Proc. of Engineering Interactive Systems 2007 (EIS 2007), LNCS 4940, Springer, 2008. http://www.hcibook.com/alan/papers/EIS-DSVIS-XSED-2007/
18. [[Dx09]] Dix, A (2009). Paths and Patches: Patterns of Geonosy and Gnosis. Chapter 1 in Exploration of Space, Technology, and Spatiality: Interdisciplinary Perspectives, P. Turner, S. Turner, and E. Davenport (eds), Information Science Reference, ISBN: 978-1-60566-020-2, pp. 1–16
19. [[DG09]] Dix, A, M. Ghazali, S. Gill, J. Hare and D. Ramduny-Ellis (2009). Physigrams: Modelling Devices for Natural Interaction. Formal Aspects of Computing , Springer (in press), doi:10.1007/s00165-008-0099-y http://www.hcibook.com/alan/papers/FAC-physical-2009/
20. [[DG10]] Dix, A, S. Gill, J. Hare and D. Ramduny-Ellis (2009). TouchIT: Designing for Physicality, The Book (to come) http://www.physicality.org/TouchIT
21. [[En95]]Endsley, M. (1995). Toward a theory of situation awareness in dynamic systems. Human Factors, 37(1), 32-64.
22. [[EK95]] Ericsson, K., and Kintsch, W. 1995, Long-term working memory. Psychological Review, 102, 211-245.
23. [[Gi79]] Gibson, James J. (1979): The Ecological Approach to Visual Perception. New Jersey, USA, Lawrence Erlbaum Associates
24. [[GD03]] Ghazali, M. and A. Dix (2003). Aladdin’s Lamp: Understanding New from Old, Proc. of 1st UK-UbiNet Workshop, Imperial College London, 2003. http://www.hcibook.com/alan/papers/ubinet-2003/
25. [[GD05]] Ghazali, M. and A. Dix (2005). Visceral Interaction, Proc. of the 10th British HCI Conf., Vol 2, pp. 68–72, 2005. http://www.hcibook.com/alan/papers/visceral-2005/
26. [[GD06]] Ghazali, M. and A. Dix (2006). Natural Inverse: Physicality, Interaction & Meaning, Let’s Get Physical: Tangible Interaction and Rapid Prototyping in, for, and about Design Workshop at 2nd International Conf. on Design Computing & Cognition 2006, 2006.
27. [[GR06]] Ghazali, M., D. Ramduny-Ellis, E. Hornecker and A. Dix (editors) (2006). Physicality 2006: First International Workshop on Physicality, (Proceedings of Worskshop, 6-7 Feb. 2006), Lancaster University. ISBN 1862201781. http://www.physicality.org/
28. [[GL08]] S. Gill, G. Loudon, A. Woolley, J. Hare, D. Walker, A. Dix and D. Ramduny-Ellis (2008) Rapid Development of Tangible Interactive Appliances: Achieving the Fidelity/Time Balance. In Special Issue on Tangible and Embedded Interaction, International Journal of Arts and Technology, Vol. 1, No.3/4, 2008, pp. 309 - 331, DOI: 10.1504/IJART.2008.022365
29. [[GD05]] Gilleade K., A. Dix and J. Allanson J(2005). Affective Videogames and Modes of Affective Gaming: Assist Me, Challenge Me, Emote Me. Online Proceedings of DIGRA 2005, June 2005. http://www.digra.org/dl/db/06278.55257.pdf
30. [[GG23]] Grimm,, Jacob W. and Wilhelm K. Grimm. The Frog Prince. Grimm's Fairy Tales, Edgar Taylor and Marian Edwardes (Translators), 1823. Current editions: Norilana Books, 2008, Penguin Books 2007. Online text at Project Guttenberg: http://www.gutenberg.org/files/2591/2591-h/2591-h.htm#2H_4_0013
31. [[HG09]] Hare, J., S. Gill, G. Loudon, D. Ramduny-Ellis and A. Dix. Physical Fidelity: Exploring the Importance of Physicality on Physical-Digital Conceptual Prototyping, Proceedings of Interact 2009 (to appear)
32. [[HH00]] Hollan, J., E. Hutchins and D. Kirsh (2000) ‘Distributed cognition: toward a new foundation for human-computer interaction research’. ACM transactions on computer-human interaction, 7(2), 174-196.
33. [[Hu83]] Hutchins E. Understanding Micronesian navigation. In D. Gentner & A. Stevens (Eds.), Mental models. Hillsdale, NJ: Lawrence Erlbaum, pp 191-225, 1983.
34. [[Hu95]] Hutchins E. Cognition in the Wild. MIT Press, 1995.
35. [[HY09]] HYSCOM , IEEE Control System Society (CSS) has a Technical Committee on Hybrid Systems, accessed March 2009. http://www.dii.unisi.it/hybrid/ieee/
36. [[IU97]] Ishii, H. and B. Ullmer (1997). Tangible bits: towards seamless interfaces between people, bits and atoms. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (Atlanta, Georgia, United States, March 22 - 27, 1997). S. Pemberton, Ed. CHI '97. ACM, New York, NY, 234-241. DOI= http://doi.acm.org/10.1145/258549.258715
37. [[KV09]] Katifori, A., C. Vassilakis and A. Dix (2009). Ontologies and the Brain: Using Spreading Activation through Ontologies to Support Personal Interaction. Cognitive Systems Research (in press). http://www.hcibook.com/alan/papers/Ontologies-and-the-Brain-2009/
38. [[KB03]] Koleva, B, S. Benford, Kher Hui Ng, and T. Rodden (2003). “A Framework for Tangible User Interfaces”, Physical Interaction (PI03) - Workshop on Real World User Interfaces”, a workshop at the Mobile HCI Conference 2003 in Udine (Italy), September 8, 2003.
39. [[Mc09]] McGrath, R. (2009). "Species-Appropriate Computer Mediated Interaction (to be presented)", alt.chi, ACM CHI 2009, April 8, 2009. http://www.chi2009.org/altchisystem/login.php?action=showsubmission&id=199
40. [[Mi56]] Miller, G. The Magical Number Seven, Plus or Minus Two: Some Limits on Our Capacity for Processing Information. The Psychological Review, 1956, vol. 63, pp. 81-97. http://www.musanim.com/miller1956/
41. [[No88]] Norman, D. The Psychology of Everyday Things. Basic Books, New York, 1988.
42. [[No90]] Norman, D. The Design of Everyday Things. Doubleday, New York, 1990
43. [[No08]] Nova, N., LIFT-Labs. tangible@home. MobiKUI2008, First International Workshop on Mobile and Kinetic User Interfaces, Fribourg, Switzerland, 13-14 Oct. 2008.
44. [[PC08]] PalCom: making computing palpable, Dated 21 April 2008, accessed March 2009. http://www.ist-palcom.org/
45. [[PF08]] Physical Fidelity in Design: A Shared Exploration. 3-4 July 2008 The National Centre for Product Design and Development Research (PDR), University of Wales Institute Cardiff (UWIC), UK
46. [[PP04]] Prinz, W., U. Pankoke-Babatz, W. Graethe, T. Gross, S. Kolvenbach and L. Schäfer (2004). Presenting activity information in an inhabited information space. In D. Snowdon (Ed), Inhabited information spaces. Living with your data. Berlin: Springer. pp.181-208.
47. [[RD07]] Ramduny-Ellis, D., A. Dix, J. Hare and S. Gill (eds.) (2007). Physicality 2007, Proceedings of the Second International Workshop on Physicality (September 2007, Lancaster University), UWIC Press, ISBN 978-1-905617-60-9. http://www.physicality.org/
48. [[RD09a]] Ramduny-Ellis, D., A. Dix, S. Gill and J. Hare. Physicality and Interaction (Editorial). Interacting with Computers. Volume 21, Issues 1-2, January 2009, Pages 64–65. doi:10.1016/j.intcom.2008.10.003
49. [[RD09b]] Ramduny-Ellis, D., A. Dix, S. Gill and J. Hare (editors). Special Issue on Physicality and Interaction. Interacting with Computers. Volume 21, Issues 1-2, January 2009, Pages 64–124.
50. [[RH08]] Ramduny-Ellis, D., J. Hare, A. Dix and S. Gill (2008) Exploring Physicality in the Design Process. Proceedings of the 2008 Design Research Society International Conference, (16-19 Jul. 2008), Sheffield, UK.
51. [[ST09]] STELARC (official website), accessed March 2009. http://www.stelarc.va.com.au/
52. [[Ta09]] TARDIS, Wikipedia page. http://en.wikipedia.org/wiki/TARDIS
53. [[TL06]] Teh, K., S. Lee and A. Cheok, Poultry.Internet and Internet Pajama: Novel Systems for Remote Haptic Interaction. Technologies for E-Learning and Digital Entertainment, LNCS 3942, Springer, 2006, pp. 1288-1291. DOI: 10.1007/11736639_161.
54. [[Th07]] Thimbleby, H.: "Press On: Principles of Interaction Programming", MIT Press (2007).
55. [[Wa03]] Warwick , K. "A study in Cyborgs", Ingenia, Journal of the Royal Academy of Engineering, Vol.16, pp.15-22 (2003)
56. [[We91]] Weiser, M. (1991). The Computer for the Twenty-First Century, Scientific American, September 1991.
57. [[UI01]] Ullmer, B., and H. Ishii (2001). Emerging Frameworks for Tangible User Interfaces. In HCI in the New Millenium, John M. Carroll, Ed. 579–601.
58. [[UI05]] Ullmer, B., H. Ishii and R. Jacob (2005). Token+constraint systems for tangible interaction with digital information. ACM Trans. Comput.-Hum. Interact. 12, 1 (Mar. 2005), 81-118. DOI=http://doi.acm.org/10.1145/1057237.1057242
59. [[VG08]] Voida, A. and S. Greenberg (2008) Wii All Play: The Console Game as a Computational Meeting Place. Research report 2008-912-25, Department of Computer Science, University of Calgary, Calgary, Alberta, Canada, September.
60. [[WD04]] Wensveen, S., J. Djajadiningrat and C. Overbeeke (2004). Interaction Frogger: A Design Framework to Couple Action and Function, Proc. of the DIS’04, ACM, pp. 177–184, 2004.