REF Redux 2 – world ranking of UK computing

Posted on August 11, 2015 by alan

This is the second of my posts on the citation-based analysis of REF, the UK research assessment process in computer science. The first post set the scene and explained why citations are a valid means for validating (as opposed generating) research assessment scores.

Spoiler: for outputs of similar international standing it is ten times harder to get 4* in applied areas than more theoretical areas

As explained in the previous post amongst the public domain data available is the complete list of all outputs (except a very small number of confidential reports), this does NOT include the actual REF 4*/3*/2*/1* score, but does include Scopus citation data from late 2013 and Google scholar citation data from late 2014.

From this seven variations of citation metrics were used in my comparative analysis, but essentially all give the same results.

For this post I will focus on one of them, which is perhaps the clearest, effectively turning citation data into world ranking data.

As part of the pre-submission materials, the REF team distributed a spreadsheet, prepared by Scopus, which lists for different subject areas the number of citations for the best 1%, 5%, 10% and 25% of papers in each area. These vary between areas, in particular more theoretical areas tend to have more Scopus counted citations than more applied areas. The spreadsheet allows one to normalise the citation data and for each output see whether it is in the top 1%, 5%, 10% or 25% of papers within its own area.

The overall figure across REF outputs in computing is as follows:

Top 1%      16.9%
Top 1-5%:   27.9%
Top 6-10%:  18.0%
Top 11-25%: 23.8%
Lower 75%:  13.4%

The first thing to note is that about 1 in 6 of the submitted outputs are in the top 1% worldwide and not far short of a half (45%) in the top 5%. Of course this is the top publications, so one would expect the REF submissions to score well, but still this feels like a strong indication of the quality of UK research in computer science and informatics.

According to the REF2014 Assessment criteria and level definitions, the definition of 4* is “quality that is world-leading in terms of originality, significance and rigour“, and so these world citation rankings correspond very closely to “world leading”. In computing we allocated 22% of papers as 4*, that is, roughly, if a paper is in the top 1.5% of papers world wide in its area it is ‘world leading’, which sounds reasonable.

The next level 3* “internationally excellent” covers a further 47% of outputs, so approximately top 11% of papers world wide, which again sounds a reasonable definition of “internationally excellent”. Validating the overall quality criteria of the panel.

As the outputs include a sub-area tag, we can create similar world ‘league tables’ for each sub-area of computing, that is ranking the REF submitted outputs in each area amongst their own area worldwide:

As is evident there is a lot of variation, with some top areas (applications in life sciences and computer vision) with nearly a third of outputs in the top 1% worldwide, whilst other areas trail (mathematics of computing and logic), with only around 1 in 20 papers in top 1%.

Human computer interaction (my area) is split between two headings “human-centered computing” and “collaborative and social computing” between them just above mid point; AI also in the middle and Web in top half of the table.

Just as with the REF profile data, this table should be read with circumspection – it is about the health of the sub-area overall in the UK, not about a particular individual or group which may be at the stronger or weaker end.

The long-tail argument (that weaker researchers and those in less research intensive institutions are more likely to choose applied and human-centric areas) of course does not apply to logic, mathematics and formal methods at the bottom of the table. However, these areas may be affected by a dilution effect as more discursive areas are perhaps less likely to be adopted by non-first-language English academics.

This said, the definition of 4* is “Quality that is world-leading in terms of originality, significance and rigour“, and so these world rankings seem as close as possible to an objective assessment of this.

It would therefore be reasonable to assume that this table would correlate closely to the actual REF outputs, but in fact this is far from the case.

Compare this to the REF sub-area profiles in the previous post:

Some areas lie at similar points in both tables; for example, computer vision is near the top of both tables (ranks 2 and 4) and AI a bit above the middle in both (ranks 13 and 11). However, some areas that are near the middle in terms of world rankings (e.g. human-centred computing (rank 14) and even some near the top (e.g. network protocols at rank 3) come out very poorly in REF (ranks 26 and 24 respectively). On the other hand, some areas that rank very low in the world league table come very high in REF (e.g. logic rank 28 in ‘league table’ compared to rank 3 in REF).

On the whole, areas that are more applied or human focused tend to do a lot worse under REF than they appear to be when looked in terms of their world rankings, whereas more theoretical areas seem to have inflated REF rankings. Those that are traditional algorithmic computer science’ (e.g. vision, AI) are ranked similarly in REF and in the world rankings.

We will see other ways of looking at these differences in the next post, but one way to get a measure of the apparent bias is by looking at how high an output needs to be in world rankings to get a 4* depending on what area you are in.

We saw that on average, over all of computing, outputs that rank in the top 1.5% world-wide were getting 4* (world leading quality).

For some areas, for example, AI, this is precisely what we see, but for others the picture is very different.

In applied areas (e.g. web, HCI), an output needs to be in approximately the top 0.5% of papers worldwide to get a 4*, whereas in more theoretical areas (e.g. logic, formal, mathematics), a paper needs to only be in the top 5%.

That is looking at outputs equivalent in ‘world leading’-ness (which REF is trying to measure), it is 10 times easier to get a 4* in theoretical areas than applied ones.

REF Redux 1 – UK research assessment for computing; what it means and is it right?

Posted on August 3, 2015 by alan

REF is the 5 yearly exercise to assess the quality of UK university research, the results of which are crucial for both funding and prestige. In 2014, I served on the sub-panel that assessed computing submissions. Since, the publication of the results I have been using public domain data from the REF process in order to validate the results using citation data.

The results have been alarming suggesting that, despite the panel’s best efforts to be fair, in fact there was significant bias both in terms of areas of computer science and types of universities. Furthermore the first of these is also likely to have led to unintentional emergent gender bias.

I’ve presented results of this at a bibliometrics workshop at WebSci 2015 and at a panel at the British HCI conference a couple of weeks ago. However, I am aware that the full data and spreadsheets can be hard to read, so in a couple of posts I’ll try to bring out the main issues. A report and mini-site describes the methods used in detail, so in these posts I will concentrate on the results, and implications, starting in this post by setting the scene seeing how REF ranked sub-areas of computing and the use of citations for validation of the process. The next post will look at how UK computing sits amongst world research, and whether this agrees with the REF assessment.

Few in UK computing departments will have not seen the ranking list produced as part of the final report of the computing REF panel.

Here topic areas are ranked by the percentage of 4* outputs (the highest rank). Top of the list is Cryptography, with over 45% of outputs ranked 4*. The top of the list is dominated by theoretical computing areas, with 30-40% 4*, whilst the more applied and human areas are at the lower end with less than 20% 4*. Human-centred computing and collaborative computing, the areas where most HCI papers would be placed, are pretty much at the bottom of the list, with 10% and 8.8% of 4* papers respectively.

Even before this list was formally published I had a phone call from someone in an institution where the knowledge of it had obviously leaked. Their department was interviewing for a lectureship and the question being asked was whether they should be recruiting candidates from HCI as this will clearly not be good looking towards REF 2020.

Since then I have heard of numerous institutions who are questioning the value of supporting these more applied areas, due to their apparent poor showing under REF.

In fact, even taken at face value, the data says nothing at all about the value in particular departments., and the sub-panel report includes the warning “These data should be treated with circumspection“.

There are three possible reasons any, or all of which would give rise to the data:

the best applied work is weak — including HCI :-/
long tail — weak researchers choose applied areas
latent bias — despite panel’s efforts to be fair

I realised that citation data could help disentangle these.

There has been understandable resistance against using metrics as part of research assessment. However, that is about their use to assess individuals or small groups. There is general agreement that citation-based metrics are a good measure of research quality en masse; indeed I believe HEFCE are using citations to verify between-panel differences in 4* allocations, and in Morris Sloman’s post REF analysis slides (where the table above first appeared), he also uses the overall correlation between citations and REF scores as a positive validation of the process.

The public domain REF data does not include the actual scores given to each output, but does include citations data provided by Scopus in 2013. In addition, for Morris’ analysis in late 2014, Richard Mortier (then at Nottingham, now at Cambridge) collected Google Scholar citations for all REF outputs.

Together, these allow detailed citation-based analysis to verify (or otherwise) the validity of the REF outputs for computer science.

I’ll go into details in following posts, but suffice to say the results were alarming and show that, whatever other effects may have played a part, and despite the very best efforts of all involved, very large latent bias clearly emerged during the progress.

WebSci 2015 – WebSci and IoT panel

Posted on June 29, 2015 by alan

Sunshine on Keble quad, brings back memories of undergraduate days at Trinity, looking out toward the Wren Library.

Yesterday was first day of WebSci 2015. I’m here largely as I’m giving my work on comparing REF outcomes with citation measures, “Citations and Sub-Area Bias in the UK Research Assessment Process”, at the workshop on “Quantifying and Analysing Scholarly Communication on the Web” on Tuesday.

However, yesterday I was also on a panel on “Web Science & the Internet of Things”.

These are some of the points I made in my initial positioning remarks. I talked partly about a few things sorting round the edge of Internet of Things (IoT) and then some concerts examples of IoT related rings I;ve been involved with personally and use these to mention few themes that emerge.

Not quite IoT

Talis

Many at WebSci will remember Talis from its SemWeb work. The SemWeb side of the business has now closed, but the education side, particularly Reading List software with relationships between who read what and how they are related definitely still clear WebSci. However, the URIs (still RDF) of reading items are often books, items in libraries each marked with bar codes.

Years ago I wrote about barcodes as one of the earliest and most pervasive CSCW technologies (“CSCW — a framework“), the same could be said for IoT. It is interesting to look at the continuities and discontinuities between current IoT and these older computer-connected things.

The Walk

In 2013 I walked all around Wales, over 1000 miles. I would *love* to talk about the IoT aspects of this, especially as I was wired up with biosensors the whole way. I would love to do this, but can’t , because the idea of the Internet in West Wales and many rural areas is a bad joke. I could not even Tweet. When we talk about the IoT currently, and indeed anything with ‘Web’ or ‘Internet’ in its name, we have just excluded a substantial part of the UK population, let alone the world.

REF

Last year I was on the UK REF Computer Science and Informatics Sub-Panel. This is part of the UK process for assessing university research. According to the results it appears that web research in the UK is pretty poor. In the case of the computing sub-panel, the final result was the outcome of a mixed human and automated process, certainly interesting HCI case study of socio-technical systems and not far from WeSci concerns.

This has very real effects on departmental funding and on hiring and investment decisions within universities. From the first printed cheque, computer systems have affected the real world, while there are differences in granularity and scale, some aspects of IoT are not new.

Later in the conference I will talk about citation-based analysis of the results, so you can see if web science really is weak science 😉

Clearly IoT

Three concrete IoT things I’ve been involved with:

Firefly

While at Lancaster Jo Finney and I developed tiny intelligent lights. After more than ten years these are coming into commercial production.

Imagine a Christmas tree, and put a computer behind each and every light – that is Firefly. Each light becomes a single-pixel network computer, which seems like technological overkill, but because the digital technology is commoditised, suddenly the physical structures of wires and switches is simplified – saving money and time and allowing flexible and integrated lighting.

Even early prototypes had thousands of computers in a few square metres. Crucially too the higher level networking is all IP. This is solid IoT territory. However, like a lot of smart-dust, and sensing technology based around homogeneous devices and still, despite computational autonomy, largely centrally controlled.

While it may be another 10 years before it makes the transition from large-scale display lighting to domestic scale; we always imagined domestic scenarios. Picture the road, each house with a Christmas tree in its window, all Firefly and all connected to the internet, light patterns more form house to hose in waves, coordinate twinkling from window to window glistening in the snow. Even in tis technology issues of social interaction and trust begin to emerge.

FitBit

My wife has a FitBit. Clearly both and IoT technology and WebSci phenomena with millions of people connecting their devices into FitBit’s data sharing and social connection platform.

The week before WebSci we were on holiday, and we were struggling to get her iPad’s mobile data working. The Vodafone website is designed around phones, and still (how many iPads!) misses crucial information essential for data-only devices.

The FitBit’s alarm had been set for an early hour to wake us ready to catch the ferry. However, while the FitBit app on the iPad and the FitBit talk to one another via Bluetooth, the app will not control the alarm unless it is Internet connected. For the first few mornings of our holiday at 6am each morning …

Like my experience on the Wales walk the software assumes constant access to the web and fails when this is not present.

Tiree Tech Wave

I run a twice a year making, talking and thinking event, Tiree Tech Wave, on the Isle of Tiree. A wide range of things happen, but some are connected with the island itself and a number of island/rural based projects have emerged.

One of these projects, OnSupply looked at awareness of renewable power as the island has a community wind turbine, Tilly, and the emergence of SmartGrid technology. A large proportion of the houses on the island are not on modern SmartGrid technology, but do have storage heating controlled remotely, for power demand balancing. However, this is controlled using radio signals, and switched as large areas. So at 4am each morning all the storage heating goes on and there is a peak. When, as happens occasionally, there are problems with the cable between the island and the mainland, the Island’s backup generator has to deal with this surge, it cannot be controlled locally. Again issuss of connectivity deeply embedded in the system design.

We also have a small but growing infrastructure of displays and sensing.

We have, I believe, the worlds first internet-enabled shop open sign. When the café is open, the sign is on, this is broadcast to a web service, which can then be displayed in various ways. It is very important in a rural area to know what is open, as you might have to drive many miles to get to a café or shop.

We also use various data feeds from the ferry company, weather station, etc., to feed into public and web displays (e.g. TireeDashboard). That is we have heterogeneous networks of devices and displays communicating through web apis and services – good Iot and WebSCi!

This is part of a broader vision of Open Data Islands and Communities, exploring how open data can be of value to small communities. On their own open environments tend to be most easily used by the knowledgeable, wealthy and powerful, reinforcing rather than challenging existing power structures. We have to work explicitly to create structures and methods that make both IoT and the potential of the web truly of benefit to all.

If the light is on, they can hear (and now see) you

Posted on April 1, 2015 by alan

Following Samsung’s warning that its television sets can listen into your conversations¹, and Barbie’s, even more scary, doll that listens to children in their homes and broadcasts this to the internet², the latest ‘advances’ make it possible to be seen even when the curtains are closed and you thought you were private.

For many years it has been possible for security services, or for that matter sophisticated industrial espionage, to pick up sounds based on incandescent light bulbs.

The technology itself is not that complicated, vibrations in the room are transmitted to the filament, which minutely changes its electrical characteristics. The only complication is extracting the high-frequency signal from the power line.

However, this is a fairly normal challenge for high-end listening devices. Years ago when I was working with submarine designers at Slingsby, we were using the magnetic signature of power running through undersea cables to detect where they were for repair. The magnetic signatures were up to 10,000 times weaker than the ‘noise’ from the Earth’s own magnetic field, but we were able to detect the cables with pin-point accuracy³. Military technology for this is far more advanced.

The main problem is the raw computational power needed to process the mass of data coming from even a single lightbulb, but that has never been a barrier for GCHQ or the NSA, and indeed, with cheap RaspberryPi-based super-computers, now not far from the hobbyist’s budget⁴.

Using the fact that each lightbulb reacts slightly differently to sound, means that it is, in principle, possible to not only listen into conversations, but work out which house and room they come from by simply adding listening equipment at a neighbourhood sub-station.

The benefits of this to security services are obvious. Whereas planting bugs involves access to a building, and all other techniques involve at least some level of targeting, lightbulb-based monitoring could simply be installed, for example, in a neighbourhood known for extremist views and programmed to listen for key words such as ‘explosive’.

For a while, it seemed that the increasing popularity of LED lightbulbs might end this. This is not because LEDs do not have an electrical response to vibrations, but because of the 12V step down transformers between the light and the mains.

Of course, there are plenty of other ways to listen into someone in their home, from obvious bugs to laser-beams bounced of glass (you can even get plans to build one of your own at Instructables), or even, as MIT researchers recently demonstrated at SIGGRAPH, picking up the images of vibrations on video of a glass of water, a crisp packet, and even the leaves of a potted plant⁵. However, these are all much more active and involve having an explicit suspect.

Similarly blanket internet and telephone monitoring have applications, as was used for a period to track Osama bin Laden’s movements⁶, but net-savvy terrorists and criminals are able to use encryption or bypass the net entirely by exchanging USB sticks.

However, while the transformer attenuates the acoustic back-signal from LEDs, this only takes more sensitive listening equipment and more computation, a lot easier than a vibrating pot-plant on video!

So you might just think to turn up the radio, or talk in a whisper. Of course, as you’ve guessed by now, and, as with all these surveillance techniques, simply yet more computation.

Once the barriers of LEDs are overcome, they hold another surprise. Every LED not only emits light, but acts as a tiny, albeit inefficient, light detector (there’s even an Arduino project to use this principle). The output of this is a small change in DC current, which is hard to localise, but ambient sound vibrations act as a modulator, allowing, again in principle, both remote detection and localisation of light.

If you have several LEDs, they can be used to make a rudimentary camera⁷. Each LED lightbulb uses a small array of LEDs to create a bright enough light. So, this effectively becomes a very-low-resolution video camera, a bit like a fly’s compound eye.

While each image is of very low quality, any movement, either of the light itself (hanging pendant lights are especially good), or of objects in the room, can improve the image. This is rather like the principle we used in FireFly display⁸, where text mapped onto a very low-resolution LED pixel display is unreadable when stationary, but absolutely clear when moving.

pix-11 pix-21
pix-12 pix-22
LEDs produce multiple very-low-resolution image views due to small vibrations and movement⁹.

Sufficient images and processing can recover an image.

So far MI5 has not commented on whether it uses, or plans to use this technology itself, nor whether it has benefited from information gathered using it by other agencies. Of course its usual response is to ‘neither confirm nor deny’ such things, so without another Edward Snowden, we will probably never know.

So, next time you sit with a coffee in your living room, be careful what you do, the light is watching you.

Not in front of the telly: Warning over ‘listening’ TV. BBC News, 9 Feb 2015. http://www.bbc.co.uk/news/technology-31296188[back]
Privacy fears over ‘smart’ Barbie that can listen to your kids. Samuel Gibbs, The Guardian, 13 March 2015. http://www.theguardian.com/technology/2015/mar/13/smart-barbie-that-can-listen-to-your-kids-privacy-fears-mattel[back]
“Three DSP tricks”, Alan Dix, 1998. https://alandix.com/academic/papers/DSP99/DSP99-full.html[back]
“Raspberry Pi at Southampton: Steps to make a Raspberry Pi Supercomputer”, http://www.southampton.ac.uk/~sjc/raspberrypi/[back]
A. Davis, M. Rubinstein, N. Wadhwa, G. Mysore, F. Durand and W. Freeman (2014). The Visual Microphone: Passive Recovery of Sound from Video. ACM Transactions on Graphics (Proc. SIGGRAPH), 33(4):79:1–79:10 http://people.csail.mit.edu/mrub/VisualMic/[back]
Tracking Use of Bin Laden’s Satellite Phone, all Street Journal, Evan Perez, Wall Street Journal, 28^th May, 2008. http://blogs.wsj.com/washwire/2008/05/28/tracking-use-of-bin-ladens-satellite-phone/[back]
Blinkenlight, LED Camera. http://blog.blinkenlight.net/experiments/measurements/led-camera/[back]
Angie Chandler, Joe Finney, Carl Lewis, and Alan Dix. 2009. Toward emergent technology for blended public displays. In Proceedings of the 11th international conference on Ubiquitous computing (UbiComp ’09). ACM, New York, NY, USA, 101-104. DOI=10.1145/1620545.1620562[back]
Note using simulated images; getting some real ones may be my next Tiree Tech Wave project.[back]

Statistics and individuals

Posted on March 23, 2015 by alan

Ramesh Ramloll recently posted on Facebook about two apparently contradictory news reports on vitamin D, one entitled “Recommendation for vitamin D intake was miscalculated, is far too low, experts say” and the other “High levels of vitamin D is suspected of increasing mortality rates“.

While specifically about diet and vitamin D intake, there seems to be a number of lessons from this: about communication of science (Ramesh’s original reason for posting this), widespread statistical ignorance amongst scientists (amongst others), and the fact that individuals are not averages.

Ramesh remarked:

Science reporting is broken, or science itself is broken … the masses are like deer in headlights when contradictory recommendations through titles like these appear in the mass media, one week or so apart.

I know that rickets is currently on the increase in the UK, due partly to poverty and poor diets leading to low dietary vitamin D intake, and due partly to fear of harmful UV and skin cancer leading to under-exposure of the skin to sunlight, our natural means of vitamin D production. So these issues are very important, and as Ramesh points out, clarity in reporting is crucial.

Looking at the two articles, the ‘too low’ article came from North America, the ‘too much’ article, although reported in AAAS ‘EurekaAlert!’ news, originated in University of Copenhagen, so I thought that maybe the difference is that health conscious Danes are simply overdosing.

However, even as a scientist, making sense of the reports is complicated by the fact that they talk in different units. The ‘too low’ one is about dietary intake of vitamin D measured in ‘IU/day’, and the Danish ‘too much’ report discusses blood levels in ‘nanomol per litre’. Wow that makes things easy!

Furthermore the Danish study (based on 247,574 Danes, real public health ‘big data’) showed the difference between ‘too much’ and ‘too little’, was a factor of two, 50 vs 100 nanomol/litre. It suggests, Goldilocks fashion, that 70 nanomol/liter is ‘just right’. Note however, the ‘EurekaAlert!’ news article does NOT quantify the relative risks of over and under dosing, which does make a big difference to the way they should be read as practical advice, and does not give a link to the source article to find out (this is the AAAS!).

Digging a little deeper into the “too low” news report, it is based on an academic article in the journal ‘Nutrients’, “A Statistical Error in the Estimation of the Recommended Dietary Allowance for Vitamin D“, which is re-assessing the amount of dietary vitamin D to achieve the same 50 nanomol/litre level used as the ‘low’ level by the Danish researchers. The Nutrients article is based not on a new study, but a re-examination of the original meta-study that gave rise to the (US and Canadian) Institute of Medicines current recommendations. The new article points out that the original analysis confused study averages and individual levels, a pretty basic statistical mistake.

Graphs from “A Statistical Error in the Estimation of the Recommended Dietary Allowance for Vitamin D“. LHS is study averages, RHS taking not account variation within studies.

A few things I took from this:

1) The level of statistical ignorance amongst those making major decisions (in this case at the Institute of Medicine) is frightening. This is part of a wider issue of innumeracy, which I’ve seen in business/economic news reporting on the BBC, reporting of opinion polls in the Times, academic publishing and reviewing in HCI, and the list goes on. This is an issue that has worried me for some time (see “Cult of Ignorance“, “Basic Numeracy“).

2) Just how spread the data is for the studies. I guess this is because individual differences and random environmental factors are so great. This really brings home the importance of replication, which is so hard to get funded or published in many areas of academia, not least in HCI where individual differences and variations within studies are also very high. But it also emphasises the importance of making sure data is published in such a way that meta-analysis to compare and combine individual studies is possible.

3) Individual difference are large. Based on the revised suggested limits for dietary vitamin D, designed to bring at least 39/40 people over the recommended blood lower limit of 50 nanomol/litre, half of people would end up with blood levels higher than four or five times that lower limit, that is more than twice as high as the level the other study says leads to deleterious over-consumption levels. This really brings home that diet and metabolism vary such a lot between people and we need to start to understand individual variations for health advice, not simply averages. This is difficult, as illustrated by the spread of studies in the ‘too low’ article, but may become possible as more mass data, as used by the Danish study, becomes available.

In short:

individuals matter in statistics

and

statistics matter for individuals

It started with a run … from a conversation at Tiree Tech Wave to an award-winning project

Posted on March 16, 2015 by alan

Spring has definitely come to Tiree and in the sunshine I took my second run of the year. On Soroby beach I met someone else out running and we chatted as we ran. It reminded me of another run two years ago …

It was spring of 2013 and a busy Tiree Tech Wave with the launch of Frasan on the Saturday evening. A group had come from the Catalyst project in Lancaster, including Maria Ferrario and she had mentioned running when she arrived, so I said I’d do a run with her. Only later did I discover that her level of running was somewhat daunting, competing in marathons with times that made me wonder if I’d survive the outing.

Happily, Maria modified her pace to reflect my abilities, and we took a short run from the Rural Centre to Chocolates and Charms (good to have a destination), indirectly via Soroby Beach, where I ran today.

Running across the sand we talked about smart grids, and the need to synchronise energy use with renewable supply, and from the conversation the seeds of an idea grew.

I started my walk round Wales almost immediately after (with the small matter of my daughter’s wedding in between), but Maria went back to Lancaster and talked to Adrian Friday, who put together a project proposal (with the occasional, very slow email interchange when I could get Internet connections). Towards the end of the summer we heard we had been short-listed and I joined Adrian via Skype for an interview in July.

… and we were successful 🙂

The OnSupply project was born.

OnSupply was a sub-project of the Lancaster Catalyst project. The wider Catalyst project’s aims were to understand better the processes by which advanced technology could be used by communities. OnSupply was the main activity for nine-months of the last year of Catalyst.

OnSupply itself was focused on how people can better understand the availability of renewable energy. Our current model of energy production assumes electricity is always available ‘on demand’ and the power generation companies’ job is to provide it when wanted. However, renewable energy does not come when we want it, but when the wind blows, the tides run and the sun shines. That is in the future we need to shift to a model where energy is used when it is available, ‘on supply’ rather than ‘on demand’.

The Lancaster team, led by Adrian consisted of four full time researchers, Will, Steve, Peter, and of course, Maria, and the other project partners were Tiree Tech Wave, the Tiree Development Trust, Goldsmiths University, and Rory Gianni, an independent developer based in Scotland specialising in environmental issues.

The choice of Tiree was of course partly because of Tiree Tech Wave and my presence here, but also because of Tilly, the Tiree community wind turbine, and the slightly parlous state of the electricity cable between Tiree and the mainland. In many ways the island is just like being on the mainland, you flick the switch and electricity is there. While Tilly can provide nearly a megawatt at full capacity, this simply feeds into the grid, just like the wind farms you see over many hillsides.

However, there is also an extent to which we, as an island population, are more sensitised to issues of electricity and renewable energy.

First is the presence of Tilly, which can be seen from much of the island; while the power goes into the grid, when she turns this generates income, which funds various island projects and groups.

But, the same wind that drives Tilly (incidentally the most productive land-based turbine in the UK), shakes power lines, and at its wildest causes shorts and breakages. The fragile power reduces the lifetime of the sophisticated wireless routers, which provide broadband to half the island, and damages fridge compressors.

Furthermore, the aging sea-cable (now happily replaced) frequently broke so that island power was provided for months at a time from backup diesel generator. As well as filling the ferry with oil tankers, the generator cannot cope with the fluctuating power from Tilly, and so for months she is braked, meaning no electricity and so no money.

So, in some ways, a community perfect for investigating issues of awareness of energy production, sensitised enough that it will be easier to see impact, but similar enough to those on the mainland that lessons learnt can be transferred.

The project itself proceeded through a number of workshops and iterative stages, with prototypes designed to provoke discussions and engagement. My favourites were machines that delivered brightly coloured ping-pong balls as part of a game to explore energy uses, and wonderful self-assembly kits for the children, incorporating a wind and solar energy gauge.

The project culminated in a display at the Tiree Agricultural Show.

While OnSupply finished last summer, the reporting continues and a few weeks ago a paper about the project, to be presented at the CHI’2015 conference in South Korea in April, was given a best paper award at the CHI’2015 conference.

… and all this from a run on the beach.

toys for Tech Wave – MicroView

Posted on February 14, 2015 by alan

I’m always on the lookout for interesting things to add to the Tiree Tech Wave boxes to join Arduinos, Pis, conductive fabric, Lilypad, Lego Technic, etc., and I had chance to play with a new bit of kit at Christmas ready for the next TTW in March.

Last year I saw a Kickstarter campaign for MicroView by GeekAmmo, tiny ‘chip-sized’ Arduinos with a built in OLED display. So I ordered a ‘Learning Kit’ for Tiree Tech Wave, which includes two MicroViews and various components for starter projects.

Initially, the MicroView was ahead of schedule and I hoped they would arrive in time for TTW 8 last October, but they hit a snag in the summer. The MicroViews are manufactured by Sparkfun who are very experienced in the maker space, but the production volume was larger than they were previously used to and a fault (missing boot loader) was missed by the test regime, leading to several thousands of faulty units being delivered.

Things go wrong and it was impressive to see the way both GeekAmmo and Sparkfun responded to the fault, analysed their quality processes and, particularly important, keeping everyone informed.

So, no MicroViews for TTW8, but they arrived before Christmas, and so one afternoon over Christmas I had a play 🙂

When you power up the MicroView (I used a USB from the computer as power source, but it can be battery powered also) the OLED screen first of all shows a welcome and then takes you through a mini tutorial, connecting up jumpers on the breadboard, and culminating with a flashing LED. It is amazing that you can do a full tutorial, even a starter one, on a 64×48 OLED!

Although it is possible to program the MicroView from a download IDE, the online tutorials suggest using codebender.cc, which allows you to program the Micriview ‘from the cloud’ and share code (sketches).

The results of my first effort are on the left above 🙂

Can you think of any projects for two tiny Ardunos? Come to Tiree Tech Wave in March and have a go!

the year that was 2014

Posted on December 20, 2014 by alan

While 2013 was full of momentous events (Miriam getting married, online HCI course and walking 1000 years around Wales), 2014 seems to have relatively little to report.

A major reason for that is the REF panel and the time taken, inter alia, to read and assess 1000 papers. I am not at all convinced by the entire research assessment process – however, if it is to happen it is needs to be done as well as possible, hence while still reeling from the walk (indeed asked whilst on the walk) I agreed to be on the panel at a relatively late stage late in 2013.

At the end of the year with the results out I guess the other members of the REF panels and I are either loved hated deepening on how different institutions fared … maybe it is good that I live on an island so far from anyone :-/

I guess I am no more convinced at the end of the process than I was at the beginning. It was good to read so much over such a wide range of topics, I feel I have an overview of UK computing that I have never had before. This was often depressing (so many niche areas that clearly will never affect anything else in computer science, let alone the world), but also lifted by the occasional piece of work that was theoretically deep, well reported and practically useful.

Beyond the many many hours of reading for REF, the world has moved on:

Fiona has begun to sell more textile art online and at events, including a stall at Fasanta where the Tiree Tapestry was also exhibited.
Miriam passed her driving test and has a car.
Esther has had a number of performances including a short film (although sadly I’ve not managed to attend any this year :-()

Personally and work-wise (the boundary is always hard to draw):

I eventually managed to fill in the remaining day blogs for Alan Walks Wales in time for 1st anniversary!
I’m gradually managing to spread the word about the unique data I collected at various talks including at events in Bangelore and Athens.
The OnSupply project about awareness of renewable energy production was wonderfully successful with several workshops in Tiree, a best paper nomination at ICT4S and an accepted CHI paper.
Work with Rachel on Musicology data, which has been slowly ticking along informally, has now been funded as the InConcert project and we ran an exciting symposium on concert-related data in November.
At Talis I am looking at the benefits of learning analytics and published my first journal paper in the area, as well as using it practically in teaching.
Tiree Tech Wave has gone from strength to strength with capacity attendance and digital fabrication workshops for the Tiree community in the autumn.
… and not least competed in the 35 mile round Tiree Ultra-marathon in September 🙂

… and in 2015

who knows, but I’ve already entered for next year’s ultra – why not join me 🙂

Tech Wave is coming

Posted on October 8, 2014 by alan

The eighth Tiree Tech Wave is just over two weeks away. We have some participants coming from GRAND NCE Canada’s Digital Media Research Network as well as those closer to home including the Code for Europe Fellows working in Nesta’s Open Data Scotland project.

There will be the normal open agenda, and also a few special activities. Jacqui Bennet has a little friendly competition planned and Steve Foreshaw from Lancaster will run a workshop on using low-cost 3D scanners, which we hope to then use to scan some of the lug boats around the island in collaboration with the Tiree Maritime Trust.

FabLab Cardiff are bringing a sort of mini-FabLab-in-a-van. During the Tech Wave they will be making things themselves, including re-installing the Tiree touchable in a glorious new enclosure. They will also run some short tutorial/workshops on using some of the equipment for TTW attendees and Tiree locals.

Although time is getting tight, I am hoping we might also have a couple of MicroViews, a miniature Arduino with built in OLED display. I ordered a Learning Kit through their Kickstarter campaign with two MicroViews (Blinking Eyes), so looking forward to some winking teddy bears 🙂 After being ahead of schedule, they had a slight production problem with their second batch, and TTW is in the third batch, so keeping fingers crossed, but, if not this time, certainly at the spring 2015 TTW.

running and talking

Posted on October 3, 2014 by alan

September saw two events on Tiree; both exciting but each very different: at the beginning of the month the first Tiree Ultramarathon organised by Will Wright our very own Tiree superhero and towards the end ‘Re-Thinking Architecturally‘, a workshop of the European Network of Excellence on Internet Science organised by Clare Hooper from Southampton University. I was lucky enough to take part in both.

In addition, in a few weeks time (23-27 Oct) there will be the eighth Tiree Tech Wave, which will include participants from Canada, Scotland, Wales and England (but no-one from Ireland yet). As well as the usual unstructured serendipity, we will have some tutorial workshops of 3D scanning, 3D printing and laser cutting … but more about that in another post.

The Tiree Ultra

photo Rhoda Meek

I should first emphasise that I did not run all of the 35 mile ultra-marathon course around the coast of Tiree, but did about 50:50, walk/run. I’d never done anything remotely like this before. The longest I’d run was the Tiree half marathon back in May and the longest I’d walked during the Wales walk was a 29 mile day (although I did that over more than 12 hours). I had as a schoolboy once done a 34 mile day hike up into the coal valleys above Cardiff, and one of the spurs to sign up for the Ultra was to beat my 17 year old self!

With my usual level of preparation it came to the beginning of August and I realised I’d not run at all, nor even taken a walk longer than to the beach and back, since the half-marathon in May. Just before the event I found a couple of sites with training schedules for marathons, all of them were several months long and the ‘beginner’ level was “runs 15-25 miles a week regularly” … what about zero miles a week? Anyway my lengthy one month training schedule consisted mainly of short (2.5 mile) beach runs with the occasional 7.5 mile run to Hynish and towards the end a run of 8.5 miles along part of the route of the Ultra round the base of Ben Hynish. I was aware I’d not done any really substantial runs and so, rather foolishly, on the Wednesday 4 days before the event I ran (and walked!) 21 miles around part of the route on the east end of the island and down to Hynish and back. A good last run before the event, but one I should have done a week earlier.

Somehow or other, despite my foolish training schedule, I managed to get round without any serious injury. My main problem was eating, or rather failing to eat. I found I could only mange food during walking stages, and then just a small amount of Kendal Mint Cake and few bananas, I guess overall I managed to burn around 4000 calories, but ate less than 1000, which really made the legs start to tire as I got to the latter parts of the course. I’ve already entered for the 2015 Tiree Ultra next September (half the places are already gone and entries have only been open a week), but I will have to work out how to eat better before then. Crucially I got round the course … and not even last. The serious contenders managed times not much more than 4 1/2 hours whilst I got round in just over 8:20, but I was simply happy to get to the end.

Although it was further and faster than any day walking last year, it was in many ways a lot easier than the Wales walk. About 2/3 of the way round the Tiree course my right ankle and calf started to stiffen, which was where I’d had Achilles ankle problems a couple of years ago. Although I did try to ease a little I was not terribly worried; the worst would be that I’d be hobbling for a month or so after the run. In contrast, last year I knew that I would be walking again the next day, and the next,and the one after that; with each ache I worried whether it would be the injury that did not get better, and stopped the walk. And, despite the worry of this, I had also learnt quite how resilient the body is, that most pains and strains did get better, albeit slowly, despite unrelenting exercise.

I was also reminded very much of the walk when I finished. I took off my running shoes and socks, the latter sodden from beach and bog, and filled with fine layer of sand. The soles of my feet, which had been subjected to 35 miles of damp sandpaper, felt like I was walking on coals and I hobbled about between van and Ceadhar where they laid on a post-walk pizza and party. It was just like that so many days last year, I would get to where I was staying, ease off my boots, put on sandals, and then hobble out in search of food, wondering if I would get as far as the closest pub or cafe, let alone walk again the next day.

Re-thinking Architecturally

The Internet Science workshop was equally enjoyable, but a little less physically challenging! It brought together economists, architects, lawyers, policy advisors, and some with more technical background from as far afield as Umea in northern Sweden. It was lovely being able to attend a technology workshop on Tiree that I hadn’t organised 🙂

Clare had been to one of the Tiree Tech Waves, and then, when she was organising a workshop for the European Network of Excellence on Internet Science, she thought of Tiree. The logistics were not without problems, but after the event I’ve been getting together with the Tiree Trust to make an information pack for future organisers to make the process easier. So if if you would like to organise an event on Tiree, get in touch!

The participants all seemed utterly taken with the venue, several brave souls even swimming in the mornings off the Hynish pier, in the words of one of the participants on the way back after the event “I’d rather be in Tiree!”. Another said:

“one great consequence of the week in Tiree was a kind of intellectual regeneration that let me set aside the stresses of the coming academic year and…think openly a bit.”
Alison Powell (London School of Economics)

In fact this is precisely the feedback I get from many who have been to Tiree Tech Wave. It is hard to capture in words the way the open horizon and being at the wild-edge opens up the mind, especially when meeting with others equally committed to exploring new ideas freely and openly.

One of the memorable moments from the week was a debate of internet freedom and regulation. We started off half on one side half the other and then part way through we all had to swop sides. I can’t believe how passionate I got about both sides of the argument … and I managed to wheel in my school English teacher as authority on benevolent dictatorship¹.

photo Parag Deshpande

photo Rory at Balevullin

People

While two very different events, a common story was the wonderful welcome of the people of Tiree. During the Ulta-marathon, at every way-station there was a glorious array of tray bakes, chocolates, pre-sliced fruit, and above all smiling faces. This was great for me seeing faces I knew, but clearly very special for the participants who came from afar being greeted as if they too were friends. For the Re-thinking Architecturally workshop many people made it a success: the wonderful team at the Hynish Centre, especially Lesley who kept on smiling despite a seven hour wait for participants whose travel was disrupted, everyone at Ceadhar, Ring n Ride, and at the airport re-arranging travel across Europe when the Thursday plane was cancelled.

I wrote what I thought to be a masterful mock O’level essay that asserted that Macbeth was actually a good king taking a ‘he made the trains run on time’ argument — my English teacher, Miss Griffiths, was not impressed.[back]