Firefox 3 seems to have fixed memory problems

I had been reluctantly considering giving up using Firefox as it crawled to a halt so often on so many sites. To be fair I think it is because I keep lots of tabs open and Firefox did not seem to deal will with pages with many refreshing elements … many air and train ticketing sites were particular problems. However, Firefox 3 has been running continuously for some time now and looking at ‘top’ in terminal window has about 1/3 the real memory footprint compared with Firefox 2 … now it is comparable with Word, Dreamweaver, etc. … I had been sticking with Firefox largely because the Firefox Snip!t bookmarklet works better than the Safari one, so now I can continue to do so without the machine crawling to a halt – well done team Mozilla 🙂

eprints: relaxed and scalable interfaces

A story, a bit of a moan … and then I hope some constructive ideas .

It is time for the University annual report, which includes a list of all publications across the University. In previous years this was an easy job. I keep an up-to-date web page with all my publications for each year, so I simply gave our secretaries a link to the web publication list, they cut and paste it into Word, tidied the format a little … and job done. However, this year things are different … a short while ago the department installed an EPrints server. This year the department is making its submission to the University by downloading from the EPrints server, which means we have to upload to it :-/

The citation adding page runs to several screen fulls including breaking author names down into surname forename … the thought of that was somewhat daunting.

Fortunately you can import into EPrints from BibTeX and EndNote bibliographies … unfortunately mine is in plain HTML 🙁

Now the 10 million AKT project that Southampton was a lead partner in developed a free text bibliography server … but, unfortunately, not included in EPrints 🙁

So a few regular expression substitutions and a lot of hand edits later and I convert my 2007 pub list into BibTeX (actually couple of hours in total including ‘bug fixing’ syntax errors in the BibTeX).

Then upload the clean .bib file … beautiful – I get a list of all the uploaded items … but they are my ‘user workspace’ and not properly deposited. This I have to do one-by-one and not allowed to do so until I have filled in various additional fields, scattered liberally over several forms including one form for adding subjects that requires several clicks to open up a lovely tree browser that in the end has only 2 leaves.

Now after grouching the lessons.

There seems to be a few key problems:

(1) First the standard usability issues: the inclusion pages are oriented around the data in the system not the user, there are no shortcuts for previously entered authors, etc.

(2) The system will not allow data to be entered if it is not complete. Of course the institution wants full data (e.g. whether it is refereed, etc.), but making it difficult to enter data makes it likely that user will not bother. That is the alternative to perfect data may be no data!

(3) The interface to enter and edit is fine for a small number of entries, but becomes a pain when processing a complete publication list. Contrarily, the page for setting the subject categories is designed for handling large trees of categories but does not gracefully handle a small number.

Both (2) and (3) are also common problems, but not so well considred in usability iterature.

A useful inofmration systems heuristic that I often advocate is

“don’t enforce consistency, but highlight inconsistency”

In this case why not allow me to deposit incomplete records and then leave me a ‘to do list’ page … yes and maybe even badger me periodically with automatic emails to check it.

Anther maxim that applies to (2) is:

“Make it easy for the user to do what you want”

If you want people to upload references make it as easy as possible to do so. Now I’m sure the designers intend this to be the case, but it is easy sometimes to focus on usability of individual screens and interactions rather than the wider context.

In fact, this was the second time that I was faced with problem (3) today. Fiona had accidentally double clicked a large number of archived files when she was trying to drag them to Trash. She had to kill the application as it blindly started to open dozens of files (why not ask?). However, it was clearly coded resiliently and kept backup copies of the files it had started to open, so, when she tried to re-open it, InDesign started to ask her whether she wanted to recover the files … but did so one-by-one and wouldn’t let her do anything else until she had laboriously answered every dialogue box.

In this case the solution is fairly obvious, if there are many (or even ore than one) files to be recovered why not list them and aks about them all, perhaps with check boxes so you can recover some but not others. In general tabular or list-style views tend to work better with large numbers of items, allowing you to perform edits to many items in a single transaction.

Similarly in EPrints, after the import there were just a few fields required for each entry, some form of tabular view would have allowed me to scan down the link and select ‘refereed/not refereed’ for each entry.

With the subject categories, it was in a sense the opposite problem, but a symptom of the way we, as designers, often have some idea in out heads about how large a particular set is likely to be and then design around that idea. However, if you can notice this tendency one can often produce variant interaction styles depending on the size of the set. For example, in web-based systems to browse hierarchies I have often (but not always!) added code that effectively says, “if the number of entries at this level is not to great, then show this level as headings with the next level as well.”


fully expanded EPrints subjects menu

The EPrints server clearly expects that the subject tree will be far bigger, as it would be on a University-wide installation. Although even if the list is very large the number of items used by an individual would be small.

So as general design advice, if there is some form of collection:

  • are there any absolute lower or upper bounds on the size?
  • check, within these absolute bounds, what the interface would be like with 1, 3, 10, 100, 1000 in the collection
  • if the potential collection is large, is the likely size needed for a particular usre, situation, smaller?

To be fair I am an unusual user with my pretty complete HTML publication lists, if I had no systematic way of keeping my own publications then I would appreciate EPrints more. However, there will be many with word processor lists, so maybe I’m not so unusual. I assume other people just knuckle down and get on with it. So the real problem is that I am impatient user!

Which brings us to the last and most valuable piece of advice. When it comes to ussr testing cussed users are worth their weight in gold. Users that are too nice are useless,; they cope, they manage and would hate to hurt your feelings by telling you your system is not perfect. So find the nasty users, the impatient users, the ones who complain at the slightest things … they are true treasure.

escape from distraction

Last week I was away in Cornwall and lost (but later found) my phone, so was both without a phone and with no internet connection … and it was amazingly liberating. My life is driven by the never ending stream of incoming mails and while in principle I could ignore them, in fact I find myself constantly breaking off what I do and seeing what has come in.

This reminded me of a Times article Haliyana pointed out to be a couple of weeks ago “Stoooopid …. why the Google generation isn’t as smart as it thinks“. We make a virtue of the never ending stream of interruptions that assail us; “multi-tasking” we call it, but in fact they not only mean we are less focused, but are possibly loosing the ability to concentrate at all.

While reading the article itself I found myself fighting not to want to follow the numerous links to other stories that littered the Times online page … and I would like to tell you more about it, but I never managed to read to the end before succumbing to the next interruption.

Basic Numeracy

When the delayed SATS results eventually arrive, I’m sure there will be the regular navel gazing at the state of basic numeracy and literacy in UK schools. But what about those who were in primary schools 30 years ago?

This morning on BBC News Channel an interviewer was talking to an economist from the City. They were discussing the reduction in bank lending (a fall of 3% during June, with 32% year-on-year drop ) and its implications for the housing market and the economy in general. The interviewer asked if it was accelerating and the economist agreed, mentioning how the year-on-year drop had gone from 10% in one quarter to 20% in the next and now over 30%.

Of course these figures are all based on a year-on-year average that includes the period before the credit crunch began last autumn and in fact are consistent with a steady linear fall of around 3% per month for the 9 months since the Northern Rock collapse. That is an alarming rate of fall, but not evidence of an accelerating fall.

This apparent lack of basic numeracy reminds me of a discussion some years ago with senior financial executives who dismissed any attempt to quantify projected company income as ‘just numbers’. Having lost money in the Northern Rock collapse I wonder whether the executives in Northern Rock and other banks had a similar attitude!

I know it is easy for me as a trained mathematician to hold up my hands in horror, but still these are people who are playing not only with their own livelihoods, but also the lives of their investors, ordinary people and even the state of the entire economy.

We do have a peculiar attitude in the UK where it is acceptable for highly educated people (including many computer scientists) to just ‘not do math’, and furthermore say so with a level of pride, whereas to say the same about reading would be unconscionable. Other European countries seem far more numerate, so this seems to be a cultural phenomena not an intellectual problem.

I have heard that one of the best predictors of educational success is if a child is willing to put off a treat for another day. Mathematics does require doing work at one stage to see benefit maybe many years later, but this to some extent runs counter to the increasingly common expectation of students to want to know fully and completely how something is useful to them now.

Maybe the answer is for schools to have lessons in leaving sweeties until tomorrow … and perhaps remedial lessons for City economists who matured during the Thatcher years.

Thusly, he wrote

Was with Kiel, one of my PhD students the other day, reading a draft chapter of his PhD thesis. He used the word ‘thusly’ and at first I thought this was a Kiel-ism (sorry Kiel, you do have a few). However, he assured me it was common usage … and not just for him. I suspected it was a regional idiom and indeed a bit of web searching finds many archaic uses of thusly, but several more recent Lancashire uses.

I suspect the word arose because ‘thus’ often precedes a verb and so has acquired an adverb ‘-ly’ ending … a form of lexical over-generalisation. … and if you have not already stopped reading because you can’t understand why anyone would be even interested in this level of language … as I did the web search found a lovely discussion site called INTERACTION (and no, this one is not the synonymous BCS HCI Group) where one person mildly mocked another for using ‘thusly’ in the 21st Century … obviously not a Lancashire lad.

This has also made me reflect on my own frequent use of ‘thus’ to mean ‘therefore’ as opposed to ‘in this way’ … argh am I really a grammar nerd :-/

why software need never hang

Over 20 years ago I wrote “The Myth of the Infinitely Fast Machine“, about the way software developers effectively assume that everything on the machine side of human interaction happens instantly. Often interaction is programmed in a turn-taking style:

  1. wait for user action
  2. process the event
  3. display changes
  4. back to step 1

This assumption of instant (or at least infinitely fast) response at step 2 often ignores network delays, disk IO or heavy computation. This tends to work fine on a high-spec development or test machine, with a fast network and clean install of all system software … but when the software hits a real machine, a few years old, untidy system, slow network … things fall to pieces.

So 20 years later (as I described in my post last week) I am sitting watching the spinning rainbow ball as Word struggles to save a document (over an hour now, I think I will need to kill it). To be fair I think the root ’cause’ of the problem … or at least one problem … may be the printer as the Cannon printer driver has never worked properly on an Intel Mac (maybe new driver when I upgrade to Leopard?) and perhaps some change in the rest of the system (maybe the Office install) has tipped it over into not working at all.

As far as I can tell Word then decides to ask the printer things in order to set the margins properly when saving the document, and then gets stuck. I found a post on a Microsoft forum about a different print related problem and the ‘helpful’ tech support from MS simply said “not our fault, re-install everything”.

So to recap:

  • user asks Word to save – probably the most critical operation in the system, or the system auto-saves, again to ensure safety against crashes, so really critical
  • Word decides it needs information from the printer (although it has been displaying the page to the users using some existing information on page properties).
  • Word asks for info from the printer driver of the currently selected printer
  • if the printer doesn’t respond Word hangs and blocks all user interaction

However, the printer driver may be third party, may be connecting to a shared printer hanging off a different network, or in the case of a laptop on a network currently disconnected from the computer … and any resulting delay is not the fault of the developers of Word??!

The annoying thing is that such ‘hanging’ delays need never happen.

Basically there are four main causes for delays:

  1. ordinary computation takes a long time due to it being too complex for the available hardware
  2. unbounded internal computation -for example iterative algorithms
  3. waiting for external resources (disk, network, etc.)
  4. bugs that lead to the system going crazy (effectively case 2 by accident!)

Type 1 will surface during testing and may require re-design of the interaction, but is simply ‘slow’ rather than ‘hanging’. Typically it leads to things gradually getting slower as the document or data gets larger or more complicated. This requires standard profiling and optimisation.

Type 4 is hard to deal with – bugs do happen. However, the majority of the problems I’m experiencing in Word at the moment are not a failure of this kind as Word does, most of the time, eventually complete without crashing.

Types 2 and 3, especially the latter, should be detected and then dealt with in the design of the user interface.

Some real-time programming languages have ways of automatically working out how long code will take to run in order to be able to assert “this will respond within a 10 ms interrupt cycle”. However, this is hard, even for relatively simple embedded systems; so not practical for complex operating systems or user interfaces.

However, a simpler version of the above is possible. Certain system functions invoke external resources such as the disk, or the network. If any function or method in your own application invokes one of these system functions, then it could potentially hang – and should be documented to say so or return some sort of ‘promise’: “I’ve started to do X, please check back later to see if it is ready”. Of course the methods that call these themselves need to be documented as potentially hanging … and so forth.

If the response to any form of user interaction ends up calling a potentially hanging function, then it is in danger of having a delay of type 3 above. However, so long as this is known, it can be dealt with at the user interface level by spawning a thread to do the work so that some form of progress indicator or at least “Cancel” button can be active – it should never ‘hang’.

This marking of functions as potentially ‘hanging’ could be done by programmers themselves, but equally can be automated as a form of static analysis, simply starting with a known set of hanging system functions and recursively ‘colouring’ functions that call them. This kind of automated checking should be standard practice in any large software project.

The type 2 hanging is a little more complicated. The ADA programming language has a ‘safe’ subset that only allows loops where the bounds are fixed at compile time. This is probably too restrictive for complex software, but certainly any loop with unknown limits could be flagged. If as part of a code walk through or similar practice it is decided that the loop is ‘safe’ it can be annotated as such, otherwise, just like the case of system calls, the system can propagate the fact that certain functions may have unbounded computation and then the UI adjusted accordingly.

For small bespoke software development I can be forgiving, but for large vendors like Microsoft, Apple or Adobe, there is no excuse for this form of culpable failure.

… but I have a bad feeling that in 20 years time I may be writing again …

[[ News flash – 1.5 hours later Word has finished saving the document! … 14 pages obviously hard work. … but then it has hung again 🙁 ]]

pain, tears and office 2008

Some weeks ago I upgraded Microsoft Office to Office 2008 (yes it does still have menus on the Mac!), and life since has been constant trouble.

OK first there are ‘minor’ niggles like it eating 1/2 my screen space in huge tool bars replicated at the top of every window, or eveytime I read in an Excel spreadsheet it telling me that old macros no longer work … actually I don’t use Excel macros, but f you do and have lots of spreadsheets that use them what then? … and don’t get me started in the fact that I can no longer cut and paste directly between Word and Dreamweaver.

… and then, just over 2 weeks ago, I was at the AVI conference and, as one does, writing the slides for the presentation the day before. I had produced all the diagrams for the presentation in Powerpoint and then copied them into Word, so thought it would be easy – start with the Powerpoint file with all the diagrams in it and add a few words around them – after all pictures always best. However, this was reckoning without Office 2008. The figures had been produced in PPT 2004, and when I opened them in Office 2008 half the images just disappeared. I tried opening in the old version of office, but it simply crashed every time I tried to update a file, I assume the Office 2008 install broke the old Office 2004 install in some way. In desperation I tried cutting and pasting the slides between PPT 2004 and PPT 2008, but that failed (I guess because Powerpoint thought it was pasting back into itself!). Eventually I managed to get the crucial images by cutting and pasting via a third program.

But the reason I am blogging now, rather than doing the pile of work that I need to do, is that Word has decided that about every 10 minutes it needs a 15 minute break and disappears into a little spinning rainbow – it does eventually come back, but only after several cups of tea.

To be fair most of the problems seem to be with compatibility mode … but surely backward compatibility is not so difficult … after all we have a lot of old files out here .. or if they can’t code it properly simply produce one-off converters rather than pretending to work when they don’t!

But the spinning disk has at last stopped … so back to another 10 minutes work before it halts again.

local URIs … mashing up the desktop

I’ve worried for a while about desktop URLs.

Within the web it is easy to link things together. If I want to refer to my home page I just add a link like this. However, on the desktop things are not so simple and I end up copying chunks of mail messages into the notes field in iCal rather than simply being able to link to the mail message where I arranged the meeting.

Links from the desktop to the web are easy … just use the URL … many desktop applications including mail clients and word processors will allow you to embed clickable links. Indeed it is often easier to link to a web page than to another object on the desktop! However, things get more difficult if you want to link the other way round, from a web page to a local file or resource. In my browser’s favourites I have several links to local files, but you cannot easily do the same if your bookmarks are in a web service like del.icio.us or even my own Snip!t. It is hard to seamlessly weave your desktop into the global web.

A couple of events brought this issue to a head for me.

First at the CHI workshop on PIM entitled the Disappearing Desktop, I asked if anyone knew of work in the area and I heard from Leo Sauermann that they had made some progress on this as part of the Gnowsis project. Their proposal for a Desktop URI Scheme (edited by Leo) is targeted principally at the first of the scenarios above, being able to link between things within the desktop.

The second event was at the AVI workshop on designing multi-touch interaction techniques for coupled public and private displays. During discussions abut touch-based interactions such as the Microsoft Surface or Apple iPhone, we considered scenarios where peole got together for a meeting (as we were) in a hotel bar (where we split for small group discussion) and had screens on table tops and walls, laptops, tablets, phones … and wanted to seamlessly move material between devices. Clearly an essential requirement for which is some way to identify resources across ad hoc collections of devices.

Finally I was in Athens working with George Lepouras, Akrivi Katifori and others. George had developed a Thunderbird extension to allow Snip!t to snip from mail messages … but while we could snip the text there was no way for the Snip!t page to link back to the mail message. We need full round trip URIs that link desktop and web with no distinction – URIs that can be embedded in a web page and (assuming you have the right permissions and are in an appropriate place) can be clicked and the appropriate mail message, calendar entry or whatever is opened.

Based on this and discussions we had, I drafted a discussion document on globally accessible local URIs. Any feedback very welcome.

Over the summer we hope to put together a demonstrator / reference implementation – if anyone is interested let me know.