book: Nightingale, Peter Dorward

Posted on by

Peter Dorward’s Nightingale is a truly beautiful tale, both in language and story.  Not beautiful in a pink ribbons sense, but with a harsh, sometimes almost brutal directness.  Dorward is a Scot, so perhaps the image of whisky is pertinent.  Certainly not a liquor like limoncello, strong beneath but covered over with sweetness, like aspects of the Italy Dorward portrays, but like a South-East Islay Malt, a smoke-tar flavour that almost makes you gag and yet all the richer for its lack of compromise.

Nightingale, takes us into Italy’s “Years of Lead” (Anni di piombo), the period of political terrorism from left and right that left  thousands dead, and in particular the 1980 railway station bombing in Bologna, which killed eighty five one hot holiday morning.  This is hardly an easy topic to deal with.  The jacket describes the novel as a ‘literary thriller’, but it is at heart about people: the almost comic, but bloody, naivete of political extremism, and the tenuous glory of love.

Although, the central character in the novel is Scottish, and the protagonists include a German Baader-Meinhoff acolyte and an Egyptian bartender, Italians and Italy form not just the backdrop, but permeate the pages of Nightingale. Dorward describes Italy with sensitivity and straightforwardness, and I think loves the country and the people in the same way I have come to; yet aware of the dark undercurrents that often underlie the Formica-tabled pizzeria and high fashion boutiques.

I recall a  few years ago seeing flowers around a plaque on the wall, just opposite the entrance to the University of Rome “La Sapienza” in via Salaria.  I had been visiting occasionally for several years, but not noticed the plaque before.  I was told it was to commemorate a Professor of the University, Massimo D’Antona, who had been assassinated some years earlier (1999) for serving on a government committee looking into the reform of labour law.  In the UK it sometimes seems we have lost our passion, that politics and life end up in a lassitude and compromise, that we need some of the passion of the south.  And yet, this passion comes at a cost.

I came to Nightingale through reading Andrew Greig’s At the Loch of the Green Corrie.  The central part of Greig’s semi-biographical, semi-autobiographical book is his journey to fish at the loch of the title, accompanied with two close friends, brothers, one of which was Peter. One evening, camping beside another loch, in conversation oiled with whisky drunk from camping mugs, Peter shares his early ideas for a story.  He is a GP in London at the time, and dabbling in writing, but yet to write a full novel.

I was captivated by this real story, of the man and his desires, and instantly reached for the internet to find him.  It was with so much joy that I saw he had written the novel, and was now an award-winning author (and still a doctor, but now in Scotland). Greig’s account had opened up such an intimacy with these brothers, so wonderful to see those nascent ideas, on that midge-plagued, peat-mattressed shoreline, bear fruit.

using the Public Suffix list

Posted on by

On a number of occasions I have wanted to decompose domain names, for example in the URL recogniser in Snip!t.  However, one problem has always been the bit at the end.  It is clear that ‘com’ and ‘ac.uk’ are the principle suffixes of ‘www.alandix.com’ and ‘www.cs.bham.ac.uk’ respectively.  However, while I know that for UK domains it is the last two components that are important (second level domains), I never knew how to work this out in general for other countries.  Happily, Mozilla and other browser vendors have an initiative called the Public Suffix List , which provides a list of just these important critical second level (and deeper level) suffixes.

I recently found I needed this again as part of my Talis research.  There is a Ruby library and a Java sourceforge project for reading the Public Suffix list, and an implementation by the DKIM Reputation project, that transforms the list into generated tables for C, PHP and Perl.  However, nothing for easily and automatically maintaining access to the list.  So I have written a small PHP class to parse, store and access the Public Suffix list. There is an example in the public suffix section of the ‘code’ pages in this blog, and it also has its own microsite including more examples, documentation and a live demo to try.

spice up boring lists of web links – add favicons using jQuery

Posted on by

Earlier today I was laying out lists of links to web resources, initially as simple links:

However, this looked a little boring and so thought it would be good to add each site’s favicon (the little icon it shows to the left on a web browser), and have a list like this:

  jQuery home page

  Wikipedia page on favicons

  my academic home page

The pages with the lists were being generated, and the icons could have been inserted using a server-side script, but to simplify the server-side code (for speed and maintainability) I put the fetching of favicons into a small JavaScript function using jQuery.  The page is initially written (or generated) with default images, and the script simply fills in the favicons when the page is loaded.

The list above is made by hand, but look at this example page to see the script in action.

You can use this in your own web pages and applications by simply including a few JavaScript files and adding classes to certain HTML elements.

See the favicon code page for a more detailed explanation of how it works and how to use it in your own pages.

Gordon’s example to us all

Posted on by

Last night I read a BBC article on Gordon Brown’s earnings since he stopped being Prime Minister a few years ago.  I felt a lump coming to my throat as I read the story.  Ex-PMs typically have lucrative post-government careers with lecture tours and the like.  Gordon Brown has similarly earned 1.4 million pounds in lecture fees and book royalties, but then given it all away.

In the run up to the General Election in 2010 I wrote how I gradually warmed to Gordon Brown during the campaign as it became increasingly clear that he was a man of true integrity.  This is another indication of that integrity, and utterly amazing to see in the modern world.

Of course he was not pretty like David Cameron or Nick Clegg, nor could he control his irritation when faced with objectionable, if popular, views.  In short, not a showman, nor a celebrity, not slick, not ‘political – just a genuinely good man.

It is sad that that is not sufficient to impress the 21st-century voter.

Dinner or tea, lunch or dinner – signs of class or the times

Posted on by

I was pondering the words of the old advertising jingle1:

I like a nice cup of tea in the morning,
Just to start the day you see;
And at half past eleven,
Well my idea of heaven,
Is a nice cup of tea.

I like a nice cup off tea with my dinner,
And a nice cup of tea with my tea,
And about this time of night,
What goes down a treat, you’re right,
It’s a nice cup of tea.

As well as the deep truth underlying the words, I suddenly became aware of the beginning of the second stanza: “a nice cup of tea with my dinner, and a nice cup of tea with my tea“.

I’d guess the last part of this may be confusing to a non-UK audience, or it may conjure up images of period-drama afternoon tea with cucumber sandwiches and parasols over a game of croquet.

Now the meaning of ‘dinner’ has been a matter of discussion in my household for years.

When I was a child ‘dinner’ was the light meal in the middle of the day, whereas ‘tea’ was the main meal at around 6 o’clock.

In contrast, Fiona takes a more pragmatic approach: ‘dinner’ is the main meal whether taken midday or in the evening.

My impression is that, when I was a child, this was part of a general class distinction. Posh (middle class) people ate lunch at midday, dinner in the evening, watched BBC and drank coffee. The working class ate dinner at midday, ate tea in the evening, watched ITV (the channel with adverts), and drank tea.

Weirdly in school one had ‘school dinners’ or ‘free dinners’ if on benefits, but had ‘packed lunches’.

We have sometimes discussed whether the tea/dinner distinction was more a Welsh-ism. But the advertising jingle clearly shows it was widespread2.

Now-a-days I tend to use the words rather interchangeably, and certainly happy to use ‘lunch’. Is this because I have become part of the professional classes or a general shift of language?

What do you call meals? Is it the same as when you were little? Is it still a class distinction?

  1. According to responses in AnswerBank, this was from an original 1937 song for Brook Bond ‘D’ brand … and in fact the word ‘tea’ was replaced by ‘D’ … but I obviously missed this and remember it as ‘tea’!  The original lyrics have slightly different final lines, “And when it’s time for bed, There’s a lot to be said, For a nice cup of tea“, or maybe I simply misremembered the advert.[back]
  2. even in 1937[back]

Lies vs. facts: the 26k benefits ceiling

Posted on by

In the UK the government is proposing a ceiling on benefits of £26,000. This sounds a large figure, indeed it is the median income, so seems reasonable that someone out of work should not receive more than the average working person. The press is, of course, polarised on the issue, as is the Church of England.

I was particularly interested in the coverage in last Wednesday’s Daily Mail, partly as this was where the former Archbishop of Canterbury chose to issue a statement about the issue, and partly because I was on a BA flight and it is one of the free newspapers! This issue of the Mail contained an article, “The hard workers who are proud not to claim”1, detailing the circumstances of three different working and tax-paying households living below or close to the proposed £26,000 limit, who can’t understand why they are working and paying taxes to support others to live on more than them.

I wondered about the truth behind these stories.  As you might imagine, the Mail’s stories were, to be generous, disingenuous, and most probably misleading, both to their readers and those they interviewed. When you work out the actual figures and facts behind the stories, things turn out rather differently then they were projected.

The issue of the proposed £26,000 benefits ceiling was particularly hot in the news after the House of Lords made radical amendments to the bill. The opposition in the Lords to proposed benefits reforms comes not just from the Labour benches, but includes some LibDems and Conservatives, and, vocally, several Church of England bishops2.

Lord Carey, the former Archbishop of Canterbury, weighed into this debate chastising his fellow bishops in the Lords, on the grounds that the weight that the national debt lays on our children is a major moral issue and the runaway benefits bill is a crucial part of controlling this.

There are of course differing views on how fast and how radically we should be attempting to cut national debt and how this should be accomplished. What is notable is that Carey chose to make this statement in the Daily Mail. My guess is he chose the Mail, rather than, say, the Times or the Telegraph (let alone the Independent or Guardian, who might have published it alongside contrary views), is that the Mail is much more a paper for ordinary Middle England folk, the ‘squeezed middle’, who feel they are paying the bulk of the taxes that fund the burgeoning benefits budget.

Whilst the ‘quality’ newspapers push their own particular viewpoint, they do follow a certain journalistic ethic, and normally within their articles you find the full facts, as they know them. Now, this is sometimes very deeply buried, to the point of disinformation, but is at least present; the careful reader can see the counter arguments through the opinion.

The Mail has no such scruples; it is unashamedly a newspaper of persuasion not information.

Given this, however much the Mail is targeting a particular demographic, Carey’s choice seemed misguided or naive.

In particular, in the same copy as Carey’s statement, there was the article describing the three households, all in tight economic circumstances, but who are working, paying tax to fund benefits, but not on benefits themselves. This is, in fact, excellent journalism, cold figures are hard to comprehend, real examples can convey the truth better than abstractions.

One household was a single woman, Rachel, living on her own; the second, Lauren and David, an engaged couple with a baby living with one of their parents; and the third, Emma and Darren, a married couple with two small twins, living in a rented house. They all had net incomes below or close to the proposed £26,000 benefits cap, and in each case the description ends with a personal statement, which expresses their frustration that, while they manage to cope on their income, why should people need £26,000 when not in work:

“I don’t understand why people would need to claim more than £26,000 in benefits if I can live comfortably on this“, Rachel

“It’s crazy that people say they can’t live of £26,000. People need to make sacrifices like the rest of us have.“, Lauren

“It makes us very angry that my husband works so hard and pays tax on his income, which goes to pay the benefits bills of all those people who don’t work and who receive more money than us.“, Emma

What the Mail reporters clearly failed to tell any of these families is what they would be receiving on benefits if they were suddenly made redundant and out of work.

Just to see I put each of these people’s circumstances into the government benefits calculator and a housing benefit calculator3.

Rachel, lives alone with £16,000 gross income and £13,000 net income. She describes rent (£500) and bills taking up most of her income, but leaving her with £250 a month for “recreational and leisure activities“, allowing her to “live comfortably“. If she lost her job her benefits including housing benefit to contribute to rent would total £9,774 per annum (£53.45 job seekers allowance, £19.38 council tax rebate4, £115.30 housing benefit). That is just what she describes as her basic bills with none of her recreation or leisure. I’m sure if asked whether she would be happy to live on this, her answer would be different.

Lauren and David fare worst; they have a gross salary of £33,000, with a net income of £27,560 (including child benefit and child tax credits). If they were both to lose their job, they would take home a total of £200.61 a week, around £10,500 per annum5. It was Lauren who said, “People need to make sacrifices like the rest of us have“. If the Mail reporter had explained to her that she would have to cope on 2/5 of their current take-home money would she feel the same?

It is the last family however, that does appear to highlight anomalies in the benefits system. Darren works in public transport and has a gross pay of precisely £26,000, leaving Emma and Darren with a take home pay of £21,608 (including child benefit). If Darren lost his job (or found himself unable to work as he has a medical condition) and both of them registered as job seekers (although Emma is currently looking after the children at home) then they would receive a total of £24,295 a year (just over £15,000 of this is basic benefit, the rest council tax relief6 and housing benefit), more than their current take home pay.

The reason for this disparity is that Emma and Darren do not attempt to claim benefits: “We are proud that we’re not on benefits, although sometimes it can be really hard“. In fact they would be eligible for substantial housing benefits7, which would presumably make all the difference for them and their children.

The shame of being on welfare runs deep, and, assuming Emma and Darren are Mail readers, no doubt fanned by the constant stories of welfare scroungers and the ‘feckless’. They quite rightly want to instil an ethic of hard work into their children, but do not feel able to claim benefits, which they will have contributed to through tax and national insurance throughout their previous working lives, in order to help as they bring up those same children now.

Interestingly, they are happy to accept child benefit (and I assume child tax credit, although not explicitly mentioned), and when the children are of school age will not send them to a fee-paying school, but happy to send them to a state school, effectively an educational ‘benefit’ of around £16,000 a year, let alone insist on paying for hospital and doctors fees for delivery of the twins and subsequent medical care.

The difference is that these benefits, allowances, and services are universal, and so seen as ‘rights’ as a taxpayer, even if, as in the case of this family, you are a net beneficiary.

This very much strengthens the case for maintaining child benefit as a non-means tested benefit. In general, many benefits are not claimed, whether through pride, principles or ignorance. The one exception is child benefit, which is both universally accepted and well targeted8.

Maybe if appraised of the full facts each of the people interviewed by the Mail might still feel the same, particularly Emma and Darren. Maybe too Mail readers would feel the same if presented with the truth. But clearly the Mail does not trust its readers to make up their own minds if given the full facts, and sadly Lord Carey has leant his weight behind this deliberate disinformation; unintentionally, but very persuasively helping to mislead the public.

  1. “The hard workers who are proud not to claim”, Daily Mail, Wednesday, January 25, 2012, p. 7.[back]
  2. Whether they should be in the second house in the first place is another issue![back]
  3. I used the Tonbridge & Malling Bourough Council’s web site as this has an online housing benefit calculator.  While currently housing benefit is similar across the country, this may change in the future with government plans for ‘localising support‘, the potential impact of which has been under-reported.[back]
  4. For Rachel on a one bedroom flat I estimated a council tax bill of £1000.[back]
  5. This figure is particularly low as  they live with parents.  While the government makes strong statements about family values, there are equally strong disincentives to support close family.  If Lauren and David were out of work, but with friends rather than parents, they would be able to pay rent to contribute to household costs, which they could then claim against housing benefit.  Furthermore, if a grown-up child receives cash support from parents, it is regarded as income for the calculation of benefits.[back]
  6. For Emma and Darren I estimated an annual council tax bill of £1500.[back]
  7. Housing benefit is perhaps the greatest cause of anomalies in the systems. Even Boris Johnson was against a cap in housing benefit, as the proposed, albeit apparently high, limit would still make large areas of London (not just the fancy bits!) no go areas for anyone on an average wage including nurses, transport workers, etc.. The situation gets even more complicated with those with a mortgage, as mortgage interest is deemed a cost for benefits calculation when you are out of work, but not when you have a job.[back]
  8. More broadly there is a minority suggestion (I believe only the Green Party in the UK support this) to replace all tax allowances and basic benefits, with a universal wage or ‘basic income‘, effectively an amount for every adult and child, deemed high enough for basic survival (probably close to current basic benefit levels). Indeed the amount you gain through the personal tax allowance, the amount you can earn without paying tax, is very close to a single person’s job seekers allowance, so this is very nearly a ‘zero sum’ for tax payers without children.[back]

If Kodak had been more like Apple

Posted on by

Finally Kodak has crumbled; technology and the market changed, but Kodak could not keep up. Lots of memories of those bright yellow and black film spools, and memories in photographs piled in boxes beneath the bed.

But just imagine if Kodak had been more like Apple.

I’m wondering about the fallout from the Kodak collapse. I’m not an investor, nor an employee, or even a supplier, but I have used Kodak products since childhood and I do have 40 years of memories in Kodak’s digital photo cloud. There are talks of Fuji buying up the remains of the photo cloud service, so it maybe that they will re-emerge, but for the time being I can no longer stream my photos to friend’s kTV enabled TV sets when I visit, nor view them online.

Happily, my Kodak kReader has a cache of most of my photos. But, how many I’m not sure, when did I last look at the photos of those childhood holidays or my wedding, will they be in my reader, I’ll check my kPhone as well. I’d hate to think I’d lost the snaps of the seaside holiday when my hat blew into the water; I only half remember it, but every time I look at it I remember being told and re-told the story by my dad.

The kReader is only a few months old. I usually try to put off getting a new one as they are so expensive, but even after a couple of years the software updates put a strain on the old machines.  I had to give up when my three year old model seemed to take about a minute to show each photo. It was annoying as this wasn’t just the new photos, but ones I recall viewing instantly on my first photo-reader more than 30 years ago (I can still remember the excitement as I unwrapped it one Christmas, I was 14 at the time, but now children seem to get their first readers when they are 4). The last straw was when the software updates would no longer work on the old processor and all my newer photos were appearing in strange colours.

Some years ago, I’d tried using a Fuji-viewer, which was much cheaper than the Kodak one. In principle you could download your photo cloud collection in an industry standard format and then import them into the Fuji cloud. However, this lost all the notes and dates on the photos and kept timing out unless I downloaded them in small batches, then I lost track of where I was. Even my brother-in-law, who is usually good at this sort of thing, couldn’t help.

But now I’m glad I’ve got the newest model of kReader as it had 8 times the memory of the old one, so hopefully all of my old photos in its cache. But oh no, just thought, has it only cached the things I’ve looked at since I’ve got it?  If so I’ll have hardly anything. Please, please let the kReader have downloaded all it could.

Suddenly, I remember the days when I laughed a little when my mum was still using her reels of old Apple film and the glossy prints that would need scanning to share on the net (not that she did use the net, she’d pop them in the post!). “I know it is the future”, she used to say, “but I never really trust things I can’t hold”. Now I just wish I’d listened to her.

Wikipedia blackout and why SOPA winging gets up my nose

Posted on by

Nobody on the web can be unaware of the Wikipedia blackout, and if they haven’t heard of SOPA or PIPA before will have now.  Few who understand the issues would deny that SOPA and PIPA are misguided and ill-informed, even Apple and other software giants abandoned it, and Obama’s recent statement has effectively scuppered SOPA in its current form.  However, at the risk of apparently annoying everyone, am I the only person who finds some of the anti-SOPA rhetoric at best naive and at times simply arrogant?

Wikipedia Blackout screenshot

The ignorance behind SOPA and a raft of similar legislation and court cases across the world is deeply worrying.  Only recently I posted about the recent NLA case in the UK, that creates potential copyright issues when linking on the web reminiscent of the Shetland Times case nearly 15 years ago.

However, that is no excuse for blinkered views on the other side.

I got particularly fed up a few days ago reading an article “Lockdown: The coming war on general-purpose computing”1  by copyright ativist Cory Doctorow based on a keynote he gave at the Chaos Computer Congress.  The argument was that attempts to limit the internet destroyed the very essence of  the computer as a general purpose device and were therefore fundamentally wrong.  I know that Sweden has just recognised Kopimism as a religion, but still an argument that relies on the inviolate nature of computation leaves one wondering.

The article also argued that elected members of Parliament and Congress are by their nature layfolk, and so quite reasonably not expert in every area:

“And yet those people who are experts in policy and politics, not technical disciplines, still manage to pass good rules that make sense.“

Doctorow has trust in the nature of elected democracy for every area from biochemistry to urban planning, but not information technology, which, he asserts, is in some sense special.

Now even as a computer person I find this hard to swallow, but what would a geneticist, physicist, or even a financier using the Black-Scholes model make of this?

Furthermore, Congress is chastised for finding unemployment more important than copyright, and the UN for giving first regard to health and economics — of course, any reasonable person is expected to understand this is utter foolishness.  From what parallel universe does this kind of thinking emerge?

Of course, Doctorow takes an extreme position, but the Electronic Freedom Foundation’s position statement, which Wikipedia points to, offers no alternative proposals and employs scaremongering arguments more reminiscent of the tabloid press, in particular the claim that:

“venture capitalists have said en masse they won’t invest in online startups if PIPA and SOPA pass“

This turns out to be a Google sponsored report2 and refers to “digital content intermediaries (DCIs)“, those “search, hosting, and distribution services for digital content“, not startups in general.

When this is the quality of argument being mustered against SOPA and PIPA is there any wonder that Congress is influenced more by the barons of the entertainment industry?

Obviously some, such as Doctorow and more fundamental anti-copyright activists, would wish to see a completely unregulated net.  Indeed, this is starting to be the case de facto in some areas, where covers are distributed pretty freely on YouTube without apparently leading to a collapse in the music industry, and offering new bands much easier ways to make an initial name for themselves.  Maybe in 20 years time Hollywood will have withered and we will live off a diet of YouTube videos :-/

I suspect most of those opposing SOPA and PIPA do not share this vision, indeed Google has been paying 1/2 million per patent in recent acquisitions!

I guess the idealist position sees a world of individual freedom, but it is not clear that is where things are heading.  In many areas online distribution has already resulted in a shift of power from the traditional producers, the different record companies and book publishers (often relatively large companies themselves), to often one mega-corporation in each sector: Amazon, Apple iTunes. For the latter this was in no small part driven by the need for the music industry to react to widespread filesharing.  To be honest, however bad the legislation, I would rather trust myself to elected representatives, than unaccountable multinational corporations3.

If we do not wish to see poor legislation passed we need to offer better alternatives, both in terms of the law of the net and how we reward and fund the creative industries.  Maybe the BBC model is best, high quality entertainment funded by the public purse and then distributed freely.  However, I don’t see the US Congress nationalising Hollywood in the near future.

Of course copyright and IP is only part of a bigger picture where the net is challenging traditional notions of national borders and sovereignty.  In the UK we have seen recent cases where Twitter was used to undermine court injunctions.  The injunctions were in place to protect a few celebrities, so were ‘fair game’ anyway, and so elicited little public sympathy.  However, the Leveson Inquiry has heard evidence from the editor of the Express defending his paper’s suggestion that the McCann’s may have killed their own daughter; we expect and enforce (the Expresss paid £500,000 after a libel case) standards in the print media, would we expect less if the Express hosted a parallel new website in the Cayman Islands?

Whether it is privacy, malware or child pornography, we do expect and need to think of ways to limit the excess of the web whilst preserving its strengths.  Maybe the solution is more international agreements, hopefull not yet more extra-terratorial laws from the US4.

Could this day without Wikipedia be not just a call to protest, but also an opportunity to envision what a better future might be.

  1. blanked out today, see Google cache[back]
  2. By Booz&Co, which I thought at first was a wind-up, but appears to be a real company![back]
  3. As I write this, I am reminded of the  corporation-controlled world of Rollerball and other dystopian SciFi.[back]
  4. How come there is more protest over plans to shut out overseas web sites than there is over unmanned drones performing extra-judicial executions each week.[back]

tread lightly — controlling user experience pollution

Posted on by

When thinking about usability or user experience, it is easy to focus on the application in front of us, but the way it impacts its environment may sometimes be far more critical. However, designing applications that are friendly to their environment (digital and physical) may require deep changes to the low-level operating systems.

I’m writing this post effectively ‘offline’ into a word processor for later upload. I sometimes do this as I find it easier to write without the distractions of editing within a web browser, or because I am physically disconnected from the Internet. However, now I am connected, and indeed I can see I am connected as a FTP file upload is progressing, it is just that anything else network-related is stalled.

The reason that the FTP upload is ‘hogging’ the network is, I believe, due to a quirk in the UNIX scheduling system, which was, paradoxically, originally intended to improve interactivity.

UNIX, which sits underneath Mac OS, is a multiprocessing operating system running many programs at once. Each process has a priority, called its ‘niceness‘, which can be set explicitly, but is also tweaked from moment to moment by the operating system. One of the rules for ‘tweaking’ it is that if a process is IO-bound, that is if it is constantly waiting for input or output, then its niceness is decreased, meaning that it is given higher priority.

The reason for this rule is partly to enhance interactive performance in the old days of command line interfaces; an interactive program would spend lots of time waiting for the user to enter something, and so its priority would increase meaning it would respond quickly as soon as the user entered anything. The other reason is that CPU time was seen as the scarce resource, so that processes that were IO bound were effectively being ‘nicer’ to other processes as they let them get a share of the precious CPU.

The FTP program is simply sitting there shunting out data to the network, so is almost permanently blocked waiting for the network as it can read from the disk faster than the network can transmit data. This means UNIX regards it as ‘nice’ and ups its priority. As soon as the network clears sufficiently, the FTP program is rescheduled and it puts more into the network queue, reads the next chunk from disk until the network is again full to capacity. Nothing else gets a chance, no web, no email, not even a network trace utility.

I’ve seen the same before with a database server on one of Fiona’s machines — all my fault. In the MySQL manual it suggested that you disable indices before large bulk updates (e.g. ingesting a file of data) and then re-enable them once the update is finished as indexing is more efficient on lots of data than one at a time. I duly did this and forgot about it until Fiona noticed something was wrong on the server and web traffic had ground to a near halt. When she opened a console on the server, she found that it seemed quiet, very little CPU load at all, and was puzzled until I realised it was my indexing. Indexing requires a lot of reading and writing data to and from disk, so MySQL became IO-bound, was given higher priority, as soon as the disk was free it was rescheduled, hit the disk once more … just as FTP is now hogging the network, MySQL hogged the disk and nothing else could read or write. Of course MySQL’s own performance was fine as it internally interleaved queries with indexing, it is just everything else on the system that failed.

These are hard scenarios to design for. I have written before (“why software need never hang“) about the way application designers do not think sufficiently about potential delays due to slow networks, or broken connections. However, that was about the applications that are suffering. Here the issue is not that the FTP program is badly designed for its delays, it is still responding very happily, just that it has had a knock on effect on the rest of the system. It is like cleaning your sink with industrial bleach — you have a clean house within, but pollute the watercourse without.

These kind of issues are not related solely to network and disk, any kind of resource is limited and profligacy causes damage in the digital world as much as in the physical environment.

Some years ago I had a Symbian smartphone, but it proved unusable as its battery life rarely exceeded 40 minutes from full charge. I thought I had a duff battery, but later realised it was because I was leaving applications on the phone ‘open’. For me I went to the address book, looked up a number, and that was that, I then maybe turned the phone off or switched  to something else without ‘exiting’ the address book. I was treating the phone like every previous phone I had used, but this one was different, it had a ‘real’ operating system, opening the address book launched the address book application, which then kept on running — and using power — until it was explicitly closed, a model that is maybe fine for permanently plugged in computers, but disastrous for a moble phone.

When early iPhones came out iOS was criticised for being single threaded, that is not having lots of things running in the ‘background’. However, this undoubtedly helped its battery life. Now, with newer versions of iOS, it has changed and there are lots of apps running at once, and I have noticed the battery life reducing, is that simply the battery wearing out with age or the effect of all those apps running?

Power is of course not just a problem for smartphones, but for any laptop. I try to closedown applications on my Mac when I am working without power as I know some programs just eat CPU when they are apparently idle (yes, Firefox, it’s you I’m talking about). And from an environmental point of view, lower power consumption when connected would also be good. My hope was that Apple would take the lessons learnt in the early iOS to change the nature of their mainstream OS, but sadly they succumbed to the pressure to make iOS a ‘proper’ OS!

Of course the FTP program could try to be friendly, perhaps when it is not the selected window deliberately throttle its network activity. But then the 4 hour upload would take 8 hours, instead of 20 minutes left at this point, I’d be looking forward to another 4 hours and 20 minutes, and I’d be complaining about that.

The trouble is that there needs to be better communication, more knowledge shared, between application and operating system. I would like FTP to use all the network capacity that it can, except when I am interacting with some other program. Either FTP needs to say to the OS “hey here’s a packet, send it when there’s a gap”1, or the OS needs some way for applications to determine current network state and make decisions based on that. Sometimes this sort of information is easily available, more often it is either very hard to get at or not available at all.

I recall years ago when internet was still mainly through pay-per-minute dial-up connections. You could set your PC to automatically dial when the internet was needed. However, some programs, such as chat, would periodically check with a central server to see if there was activity, this would cause the PC to dial-up the ISP. If you were lucky the PC also had an auto-disconnect after a period of inactivity, if you were not lucky the PC would connect at 2am and by the morning you’d find yourself with a phone bill more than your weeks’ wages.

When we were designing onCue at aQtive, we wanted to be able to connect to the Internet when it was available, but avoid bankrupting our users. Clearly somewhere in the TCP/IP stack, the layers of code over the network, at some level deep down it knew whether we were connected. I recall we found a very helpful function in the Windows API called something like “isConnected”2. Unfortunately, it worked by attempting to send a network packet and returning true if it succeeded and false if it failed. Of course sending the test packet caused the PC to auto-dial …

And now there is just 1 minute and 53 seconds left on the upload, so time to finish this post before I get on to garbage collection.

  1. This form of “send when you can” would also be useful in cellular networks, for example when syncing photos.[back]
  2. I had a quick peek, and fund that Windows CE has a function called InternetGetConnectedState.  I don’t know if this works better now.[back]

New Year and New Job

Posted on by

It is a New Year and I am late with my Christmas crackers again!

If you are expecting the annual virtual cracker from me it is coming … but maybe not before Twelfth Night :-/

The New Year is bringing changes, not least, as many already know, I am moving my academic role and taking up a part-time post as professor down in Birmingham University.

At Birmingham I will be joining an established and vibrant HCI centre, including long-term colleague and friend Russell Beale.  The group has recently had substantial  investment from the University leading to several new appointments including Andrew Howes (who coincidentally also has past Lancaster connections).

The reasons for the move are partly to join this exciting group and partly to simplify life as Talis is based in Birmingham, so just one place to travel to regularly, and one of my daughters also there.

Of course this also means I will be leaving many dear colleagues and friends at Lancaster, but I do expect to continue to work with many and am likely to retain a formal or informal role there for some time.

As well as moving institutions I am also further reducing my percentage of academic time — typically I’ll be just one day a week academic.  So, apologies in advance if my email responses becomes even more sporadic and I turn down (or fail to answer :-() requests for reviews, PhD exams, etc.

Although moving institutions, I will, of course, continue to live up in Tiree (wild and windy, but, at the moment, so is everywhere!), so will still be travelling up and down the country; I’ll wave as I pass!

… and there will be another Tiree Tech Wave in March 🙂