Category Archives: HCI and usability

why software need never hang

Posted on by

Over 20 years ago I wrote “The Myth of the Infinitely Fast Machine“, about the way software developers effectively assume that everything on the machine side of human interaction happens instantly. Often interaction is programmed in a turn-taking style:

  1. wait for user action
  2. process the event
  3. display changes
  4. back to step 1

This assumption of instant (or at least infinitely fast) response at step 2 often ignores network delays, disk IO or heavy computation. This tends to work fine on a high-spec development or test machine, with a fast network and clean install of all system software … but when the software hits a real machine, a few years old, untidy system, slow network … things fall to pieces.

So 20 years later (as I described in my post last week) I am sitting watching the spinning rainbow ball as Word struggles to save a document (over an hour now, I think I will need to kill it). To be fair I think the root ’cause’ of the problem … or at least one problem … may be the printer as the Cannon printer driver has never worked properly on an Intel Mac (maybe new driver when I upgrade to Leopard?) and perhaps some change in the rest of the system (maybe the Office install) has tipped it over into not working at all.

As far as I can tell Word then decides to ask the printer things in order to set the margins properly when saving the document, and then gets stuck. I found a post on a Microsoft forum about a different print related problem and the ‘helpful’ tech support from MS simply said “not our fault, re-install everything”.

So to recap:

  • user asks Word to save – probably the most critical operation in the system, or the system auto-saves, again to ensure safety against crashes, so really critical
  • Word decides it needs information from the printer (although it has been displaying the page to the users using some existing information on page properties).
  • Word asks for info from the printer driver of the currently selected printer
  • if the printer doesn’t respond Word hangs and blocks all user interaction

However, the printer driver may be third party, may be connecting to a shared printer hanging off a different network, or in the case of a laptop on a network currently disconnected from the computer … and any resulting delay is not the fault of the developers of Word??!

The annoying thing is that such ‘hanging’ delays need never happen.

Basically there are four main causes for delays:

  1. ordinary computation takes a long time due to it being too complex for the available hardware
  2. unbounded internal computation -for example iterative algorithms
  3. waiting for external resources (disk, network, etc.)
  4. bugs that lead to the system going crazy (effectively case 2 by accident!)

Type 1 will surface during testing and may require re-design of the interaction, but is simply ‘slow’ rather than ‘hanging’. Typically it leads to things gradually getting slower as the document or data gets larger or more complicated. This requires standard profiling and optimisation.

Type 4 is hard to deal with – bugs do happen. However, the majority of the problems I’m experiencing in Word at the moment are not a failure of this kind as Word does, most of the time, eventually complete without crashing.

Types 2 and 3, especially the latter, should be detected and then dealt with in the design of the user interface.

Some real-time programming languages have ways of automatically working out how long code will take to run in order to be able to assert “this will respond within a 10 ms interrupt cycle”. However, this is hard, even for relatively simple embedded systems; so not practical for complex operating systems or user interfaces.

However, a simpler version of the above is possible. Certain system functions invoke external resources such as the disk, or the network. If any function or method in your own application invokes one of these system functions, then it could potentially hang – and should be documented to say so or return some sort of ‘promise’: “I’ve started to do X, please check back later to see if it is ready”. Of course the methods that call these themselves need to be documented as potentially hanging … and so forth.

If the response to any form of user interaction ends up calling a potentially hanging function, then it is in danger of having a delay of type 3 above. However, so long as this is known, it can be dealt with at the user interface level by spawning a thread to do the work so that some form of progress indicator or at least “Cancel” button can be active – it should never ‘hang’.

This marking of functions as potentially ‘hanging’ could be done by programmers themselves, but equally can be automated as a form of static analysis, simply starting with a known set of hanging system functions and recursively ‘colouring’ functions that call them. This kind of automated checking should be standard practice in any large software project.

The type 2 hanging is a little more complicated. The ADA programming language has a ‘safe’ subset that only allows loops where the bounds are fixed at compile time. This is probably too restrictive for complex software, but certainly any loop with unknown limits could be flagged. If as part of a code walk through or similar practice it is decided that the loop is ‘safe’ it can be annotated as such, otherwise, just like the case of system calls, the system can propagate the fact that certain functions may have unbounded computation and then the UI adjusted accordingly.

For small bespoke software development I can be forgiving, but for large vendors like Microsoft, Apple or Adobe, there is no excuse for this form of culpable failure.

… but I have a bad feeling that in 20 years time I may be writing again …

[[ News flash – 1.5 hours later Word has finished saving the document! … 14 pages obviously hard work. … but then it has hung again 🙁 ]]

pain, tears and office 2008

Posted on by

Some weeks ago I upgraded Microsoft Office to Office 2008 (yes it does still have menus on the Mac!), and life since has been constant trouble.

OK first there are ‘minor’ niggles like it eating 1/2 my screen space in huge tool bars replicated at the top of every window, or eveytime I read in an Excel spreadsheet it telling me that old macros no longer work … actually I don’t use Excel macros, but f you do and have lots of spreadsheets that use them what then? … and don’t get me started in the fact that I can no longer cut and paste directly between Word and Dreamweaver.

… and then, just over 2 weeks ago, I was at the AVI conference and, as one does, writing the slides for the presentation the day before. I had produced all the diagrams for the presentation in Powerpoint and then copied them into Word, so thought it would be easy – start with the Powerpoint file with all the diagrams in it and add a few words around them – after all pictures always best. However, this was reckoning without Office 2008. The figures had been produced in PPT 2004, and when I opened them in Office 2008 half the images just disappeared. I tried opening in the old version of office, but it simply crashed every time I tried to update a file, I assume the Office 2008 install broke the old Office 2004 install in some way. In desperation I tried cutting and pasting the slides between PPT 2004 and PPT 2008, but that failed (I guess because Powerpoint thought it was pasting back into itself!). Eventually I managed to get the crucial images by cutting and pasting via a third program.

But the reason I am blogging now, rather than doing the pile of work that I need to do, is that Word has decided that about every 10 minutes it needs a 15 minute break and disappears into a little spinning rainbow – it does eventually come back, but only after several cups of tea.

To be fair most of the problems seem to be with compatibility mode … but surely backward compatibility is not so difficult … after all we have a lot of old files out here .. or if they can’t code it properly simply produce one-off converters rather than pretending to work when they don’t!

But the spinning disk has at last stopped … so back to another 10 minutes work before it halts again.

local URIs … mashing up the desktop

Posted on by

I’ve worried for a while about desktop URLs.

Within the web it is easy to link things together. If I want to refer to my home page I just add a link like this. However, on the desktop things are not so simple and I end up copying chunks of mail messages into the notes field in iCal rather than simply being able to link to the mail message where I arranged the meeting.

Links from the desktop to the web are easy … just use the URL … many desktop applications including mail clients and word processors will allow you to embed clickable links. Indeed it is often easier to link to a web page than to another object on the desktop! However, things get more difficult if you want to link the other way round, from a web page to a local file or resource. In my browser’s favourites I have several links to local files, but you cannot easily do the same if your bookmarks are in a web service like del.icio.us or even my own Snip!t. It is hard to seamlessly weave your desktop into the global web.

A couple of events brought this issue to a head for me.

First at the CHI workshop on PIM entitled the Disappearing Desktop, I asked if anyone knew of work in the area and I heard from Leo Sauermann that they had made some progress on this as part of the Gnowsis project. Their proposal for a Desktop URI Scheme (edited by Leo) is targeted principally at the first of the scenarios above, being able to link between things within the desktop.

The second event was at the AVI workshop on designing multi-touch interaction techniques for coupled public and private displays. During discussions abut touch-based interactions such as the Microsoft Surface or Apple iPhone, we considered scenarios where peole got together for a meeting (as we were) in a hotel bar (where we split for small group discussion) and had screens on table tops and walls, laptops, tablets, phones … and wanted to seamlessly move material between devices. Clearly an essential requirement for which is some way to identify resources across ad hoc collections of devices.

Finally I was in Athens working with George Lepouras, Akrivi Katifori and others. George had developed a Thunderbird extension to allow Snip!t to snip from mail messages … but while we could snip the text there was no way for the Snip!t page to link back to the mail message. We need full round trip URIs that link desktop and web with no distinction – URIs that can be embedded in a web page and (assuming you have the right permissions and are in an appropriate place) can be clicked and the appropriate mail message, calendar entry or whatever is opened.

Based on this and discussions we had, I drafted a discussion document on globally accessible local URIs. Any feedback very welcome.

Over the summer we hope to put together a demonstrator / reference implementation – if anyone is interested let me know.

Tags and Tagging: from semiology to scatology

Posted on by

I’ve just been at a two-day workshop on “Tags and Tagging” organised by the “Branded Meeting Places” project.

Tags are of course becoming ubiquitous in the digital world: Flickr photos, del.icio.us bookmarks; at the digital/physical boundary: RFID and barcodes; and in the physical world: supermarket price stickers, luggage labels and images of Paddington Bear or wartime evacuees each with a brown paper label round their necks. Indeed we started off the day being given just such brown paper tags to design labels for ourselves.

Alan's tag

As well as being labels so we know each other, they were also used as digital identifiers using a mobile-phone-based image-recognition system, which has been used in a number of projects by the project team at Edinburgh (see some student projects here). We could photograph each others tags with our own phones, MMS the picture to a special phone number, then a few moments later an SMS message would arrive with the other person’s profile.

Being focused on a single topic and even single word ‘tag’ soon everything begins to be seen through the lens of “tagging”, so that when we left the building and saw a traffic warden at work outside the building, instantly the thought came “tagging the car”!

Vocal Thumbs logoThe workshop covered loads of ground and included the design and then construction of a real application – part of the project’s methodology of research through design. However, two things that I want to write about. The first is the way the workshop made me think about the ontology or maybe semiology of tags and tagging, and the second is a particular tag (or maybe label, notice?) … on a toilet door … yes the good old British scatological obsession.

Continue reading →

mobile design workshop

Posted on by

A couple of weeks ago I attended a mobile design workshop at microsoft labs in Cambridge. Great 2 days … people from academia and industry (and no not just MS, also Google, Yahoo, Sony, Nokia, …!)
Alan in Helmet

Most of the time was spent splitting into small working groups then coming back together for plenaries. I took part in groups discussing:

(1) tools to make it easier for those in developing countries to design mobile phone applications that suit their needs [session notes], rather than simply passing on applications and designs fitted for very different needs and infrastructure. During the discussion various applications of phone technology were cited that were completely different form those we would expect in the UK, US or Europe, but fitted the situations of people. These included using the address book as a ‘who owes what’ list for a trader … the ‘telephone numbers’ were in act amounts of money! This use of ‘ancillary’ parts of the phone rather than simply being a glorified communication device. Although he context of this was Africa, it also echoes studies of domestic phone use by Malay women in the UK by Fariza (who has just had her PhD viva :-)). She found alarm, calculator and things like that, at least as important as phone & text for the people she studied.

(2) ‘mindfulness’ and mobile phones … and of course the fact that normally they do the opposite interupting etc. … but just to not make us all agree too much, I said that mindfulness sounded like we should all become like rabbits; it is the looking forward and back, with all its stress, that is one of the things that make us human.

(3) task/data oriented interaction … escaping from the ‘application’. This was particularly relevant to me given onCue at aQtive was in this space as are Snip!t and work on TIM project with colleagues at Rome, Athens and recently new collaborators Madrid … with whom I had a short but lovely visit after CHI.

Task session

the electronic village shop – update and kit

Posted on by

I forgot last week when I wrote my post “the electronic village shop – enhancing local community through global network” that Fiona had mentioned a blog, Silversprite, from the Outer Hebrides that mentioned the flip side of this; in a post “Tesco comes to the Outer Hebrides“, he mentions the potential conflicts between Tesco online and village shops. It is interesting that Tesco does not itself deliver direct, but you can give a local delivery firm as your drop off point and they do the last leg of the delivery … of course this delivery point could be the local shop!

I also remembered after that some years ago (2002) I had an email enquiry from someone who had found a reference to the electronic village shop in one of my talks. He was wondering how to revitalise his own village shop which was about to close and asked if I had any advice. The following is edited version of my reply where I pondered on what was possible as a single shop without buy-in from one of the big franchise chains:

Continue reading →

tales from/for Berlin – appropriation, adoption and physicality

Posted on by

A few weeks ago I had a short visit to Berlin as a guest of Prometei, a PhD training program at the University of Technology of Berlin focused on “prospective engineering of human-technology-interaction”. While there I gave an evening talk on “Designing for adoption and designing for appropriation” and spent a very pleasant afternoon seminar with the students on “Physicality and Interaction”.

I said I would send some links, so this is both a short report on the visit and also a few links to appropriation and adoption and a big long list of links to physicality!

Continue reading →

the electronic village shop – enhancing local community through global network

Posted on by

The internet seems to be about remote connections, international communities and globalisation. We may surf the web, scan blogs or talk to friends across the globe, but may not know the person next door. Indeed in Channel 4’s recent documentary “My Street“, Sue Bourne, who produces TV documentaries seen by millions, gets to meet her neighbours for the first time.

But there is another side, where global networks could help local communities to grow and reconnect with one another. Some things are happening grass roots up, some need changes in public policy or intervention, and some may never happen.

The electronic village shop is a dream I’ve had for now well over 15 years, and may happen, or may not … and maybe definitely will not if I don’t do something myself! However, there are other signs of local connections growing.

I have talked about these issues and the electronic village shop at different times over the years1, but have never previouslly written about them, so eventually …
Continue reading →

  1. Talks mentioning the electronic village shop: Understanding the e-Market and Designing Products to Fit, London, Jan. 2000, Cyber-economies and the Real World, Pretoria, Sept. 2001. and Toys for the Boys or Jobs for the Girls, Cheltenham, Nov. 2001[back]

mirrors and estrangement

Posted on by

One of my PhD students, Fariza, has been extensively studying a single person, a women not so disimilar from herself. In doing so they have become friends and much of Fariza’s thesis (soon to be submitted) concerns the methodological issues surrounding this.

pianoOne issue we have discussed at length is the importance of estrangement, that distancing oneself from the commonplace to make the taken for granted become apparent. We do not see the things closest to us: the dirty toenail, the crumpled sheet, the asymmetric fall of the piano music stand, the things lost because they are in the open.

Artists and comedians often open our eyes to the unseen-because-unnoticed aspects of life, such as Emin’s own crumpled sheets or the poignant woman on the platform in “The Fall and Rise of Reggie Perrin”1. Garfinkel’s breaching experiments attempt to bring this incisive comic eye to social science. Of novelists Gabriel Garcí­a Márquez is the master of this; through his true to life yet surreal accounts, with sometimes tenderness and sometimes almost cruel dispassion, he describes in intricate detail the intimate yet insignificant.

The Last Battle, C.S. LewisBerger talks about the way artists look at their work in a mirror to see it afresh and he himself sees the sunshine of lilacs in the mirror, lilacs that show him only their shadows2. I was reminded of one of C.S. Lewis’ Narnia books where the supra-reality of the mirror image is more crystal sharp, more ‘real’ than the ordinary world3.

Mirrors bring to mind reflective practice, just as Fariza has to write herself into her own accounts, the mirror often shows not only the unseen side of an object but also oneself and oneself in relation to the object. The seer and seen are themselves seen and, like Berger’s lilacs, the partiality of one’s seeing becomes more obvious. The mirror is not so much important because it shows the hidden sun-glowed lilacs, but because it says that the shadowed petals are not the whole flower.

Arnolfini Mariage: mirror closeupI am reminded of that Dutch painting by van Eyck where the apparently pregnant woman and her husband stand in their home with their lap dog between them and discarded outdoor shoes untidily dropped4. On the wall behind the couple is a convex mirror capturing the whole scene from behind, and in the reflected doorway where you stand is the tiny image of van Eyck and another. Unlike supposedly ‘good’ scientific writing in incomprehensible passive speech, the artist has not painted himself out of his picture.

Arnolfini Mariage: dog closeupBerger also talks about paintings being painted for the moment of seeing, yet so many portraits do not look towards you, or even the artist, but through you, beyond you, to an unseen landscape. This may be because it is hard to paint eyes, or because it is hard to stare into the eyes of one’s painter. In life it is only with the deepest lovers or friends that we dare to share like this, hence perhaps the growing friendship that Fariza experienced when gazing deeply together into her subject’s life. Also, perhaps why the most intimate paintings have often been of the painter’s wives or lovers. Unusually in van Eyck’s painting, clearly of close friends, the couple do not gaze either aloof or uncaring past the viewer, but instead gaze within the painting demurely to one another … it is only the small dog that stares back.

Mona LisaThis may be part of the Mona Lisa’s allure. I have never seen it ‘in the flesh’, but everyone talks about the eyes that they follow you. However, look again, it is not so much that they follow you, but, unusual amongst paintings, they actually look at you and I suppose at Leonardo himself.

I know Fariza has found this, and I am sure it is universal, that if we look closely and honestly at those around us, we begin more clearly to see ourselves .

  1. This was a popular BBC TV series, but it is the book which I remember as most moving, I laughed out loud and shed tears equally.[back]
  2. John Berger, “and our faces, my heart, brief as photos“, Bloomsbury, 2005[back]
  3. end of Chapter 15 in “The Last Battle“[back]
  4. The page about the Arnolfini Portrait at the National Gallery site allows you to zoom into the image. It also explains that the woman is not actually pregnant, but simply in the full skirt style of the day.[back]

What is Computing? The centrality of systemics

Posted on by

Recently I was in a meeting where the issue of ‘core’ computer science came up. One person listed a few areas, but then this was challenged by another member of the group who said (to be fair, partly in jest), that core computer science should certainly include computer architecture, but not the ‘human stuff’.

I felt a little like a teenager complete with T-shirt and iPod dropped into Jurassic Park arguments that I thought had been put to bed in the 1980s suddenly resurfacing – how do you explain this white thing that makes sounds from its earphones to a caveman wearing skins?

However, I also felt a certain sympathy as I often wonder about computer science as a whole; indeed it has its own arguments in the 1960s and ’70s as to whether it was a ‘discipline’ as opposed to just an application domain for maths or electronics, or just a tool for business. Maybe one of the clinchers was the theoretical foundations of computing in the work of Church and Turing … but strangely enough at Lancaster the closest to this, the course on algorithmic complexity, is taught by a HCI person!

One of my worries in computing is that these theoretical foundations are still weak, there is black hole in the theoretical centre of computer science1. However, these theoretical issues were certainly not what was bothering my colleague. To answer his challenge and my own worries about the discipline we really need to know – what is computing?

Continue reading →

  1. This demands a discussion of its own, but the basic problem is that while Church and Turing gave us understanding of disembodied computation, we still do not have clear understanding of generic computation when embodied in devices in general only particular architectures. [back]