Interfaces and Infrastructure for Mobile Multimedia Applications
EPSRC GR/L64140 & GR/L64157

about the project
research areas
  user interface
ongoing work



Alan Dix, 21/2/2002

User Interface Work

Work on Separable Notification for Feedthrough and Pace Impedance

A major strand of the UI work has been work on separable notification services to provide feedthrough of one user's actions to other users. This is critical in collaborative advanced mobile applications both for task-oriented update of shared information and also for peripheral awareness that has been widely recognised as essential for collaborative activity.

The basic problem is quite simple. In a collaborative setting one user A makes a modification to some shared data, or performs some event (this may be as simple as moving a cursor, or as major as replacing a whole file). We call this user's computer/device the active client (AC). In the system as a whole we want some other user P to be aware (either peripherally or more explicitly) of the event or change. We call this user's computer/device the passive client (PC). How do we propagate the event information from the active client to the passive client in a manner that is timely and effective for the users and efficient in terms of network and computational resources? Note that there are typically several passive clients and that the roles of active and passive client alternate during interaction. Furthermore, the phrase "timely and effective for the users" is itself a dynamic issue depending not just on the current application, but also on the current context including device characteristics, the user's current task, focus and location.

Figure 1 : Client-data interaction with notification server

It is common for data to be stored centrally in a database or other form of resource server. A notification server is a similar centralised component whose job is to manage this transmission of events. At a broader level a notification service (c.f. server) is the general system of components that enable appropriate notification and may or may not involve a notification server. Notification services will use event or messaging services, and may sometimes be bundled with them, but satisfy a different requirement. Within the context of the project we have made several significant theoretical advances within this area.

  • The exhaustive analysis of the different potential event interactions between the active client, data server, notification server and passive client. This lead to the identification of the importance of a separable notification service, i.e. one which is not intimately connected to the data repository. The advantages of this include the ability to locate notification and data services in different physical locations and the ability to create collaboration and data aware user interfaces on top of legacy data sources such as traditional databases, web data etc. Our experimental notification server, Getting-to-Know (GtK), was built deliberately to be able to operate in this separable mode (and also more tightly data-coupled modes). This strand was principally reported in [2].
  • A more formal view of this strand focused on the issues of initiative within chains of event notification. For example, during polling the event notification moves in the 'opposite direction' to the initiative, whereas in more explicit notification both operate in the same direction. To see this, imagine you are waiting for your holiday tickets to arrive at your travel agent. Every day you might ring your travel agent to ask whether they have arrived (polling). Alternatively, you may ask your travel agent to ring you (explicit notification). In both cases the knowledge that the tickets have arrived lies with the travel agent, and the notification direction is always from agent to you, but in the first case (polling) initiative lies with you, and in the latter with the travel agent. Within a user interface there are typically multiple levels of notification and initiative both within the infrastructure (e.g. notification service) and at the interface (active beep to tell user, or passive change to status). This work was reported in [3]. The insights from this were used in refinements of our GtK notification server and were also incorporated into a commercial product (see dissemination).
  • The pace at which events are generated, whether through user actions, environmental monitoring, or media streaming, may not match that required by the users who wish to view or be aware of the events. For example, in a shared drawing application, one may not want to be aware of pixel-by-pixel drawing of an object. In some applications the requirements are similar for all recipients, in which case the production of events can simply be throttled. In others this differs on a user-to-user basis. For the latter the notification server is the ideal place to filter events and pass them on at an appropriate rate. This has been called 'pace impedance matching'. The concept was introduced at workshops (and mentioned briefly in [3]), and then implemented within GtK. The results of this have been reported in [4] and in Ramduny-Ellis' PhD thesis [5]).

A journal paper is in final preparation bringing together the work on separability and pace impedance and experience of their use within Getting-to-Know. Furthermore, a web version of GtK is due to be launched shortly, focused on lower pace interactions and http-based transactions including client polling of the server (the current version of GtK is a Java based server intended for largely synchronous interactions with permanent client-server connections).

Point-of-presence as a Location for Components in Shared Collaborative Applications

In mobile applications shared data and notification will typically need to be mediated by central servers and databases. This means that all feedthrough to users encounters the delays of potentially global network transmission and consumes correspondingly high network resources. For remote interactions between users this is acceptable as they will typically require a lower pace of feedthrough (potentially managed through pace impedance matching). However, as users come together, for example, at a face-to-face meeting, they perceive their mobile devices as 'close' in physical space, but they may still be interacting via very remote servers. A proposed solution is to migrate aspects of interface/data/notification services closer to collaborating mobile users. The analysis of the options and issues arising was reported in [6] and [7], but has not been incorporated into current versions of GtK as it requires extensive support within the network infrastructure. While this is not currently practical, emerging all-IP based solutions for wireless networks (as typified by the Cellular-IP proposals) may offer such support and we expect our results in this area to be applicable in such cases.

Issues of Location and Context Awareness and the Nature of Cyberspace

One of the important aspects of mobile applications is that they operate at changing physical locations and in changing physical contexts. Awareness of these contexts is growing due to technologies such as GPS, phone-cell identification, bluetooth and device specific sensors. An early analysis of these issues was reported in [8] and a more detailed analysis of the nature of context aware applications and specifically location awareness was reported in [1]. This includes various levels of analysis: an informal taxonomic classification of context awareness and location aware characteristics; a formal model of space for location services; and a computational model implemented on top of Limbo, a Linda-based shared tuple space [9]. Developments of this strand were also presented at various keynotes [10,11]. This work was highly praised by referees because the majority of work within ubiquitous computing and context-aware interfaces is very closely tied to specific systems. In contrast, our work takes a more analytic view, building a framework to understand these many disparate systems.

Other UI Issues of Mobile and Related Technologies

As a side-effect of the focused streams within the project, a number of more disparate UI aspects of mobile and related interfaces were studied leading to various keynotes [12,13], journal and other publications [14, 15]. In particular, the work with Liu and Sun (Staffordshire University) and Narasipuram (Hong Kong City), on Norm-based analysis of network agents formed a bridge between the more technical issues and the social and economic impact of a network-connected society. These issues were at the periphery of this project, but were the subject of various invited talks, keynotes and articles [11,16,17,18].