Query-by-Browsing has user explanations

Query-by-Browsing now has ‘user explanations’, ways for users to tell the machine learning component which features are significant in the user provided examples.  As promised in my blog about local AI explanations in QbB a few weeks ago, this version of QbB is released to coincide with our paper “Talking Back: human input and explanations to interactive AI systems” that Tommaso Turchi is presenting at the Workshop on Adaptive eXplainable AI (AXAI) at IUI 2025 in Cagliari, Italy,

As part of the EU Horizon Tango project, on hybrid human–AI decision making, we have been thinking about what it would mean for users to provide the AI with explanations of their human reasoning in order to guide machine learning and improve the AI’s explanations of its outputs.

As an exemplar of this I have modified QbB to include forms of user explanation.  These are of two kinds, global user explanations to guide the overall machine learning and local user explanations focused on individual examples.

Play with this version of QbB or see the QbB documentation in Alan Labs.

Basic operation

Initially you use QbB as normal: you select examples of records you do and don’t want included and the system infers a query using a variant of ID3 that can be presented as a decision tree or an SQL query.

Global user guidance

At any point you can click row headers to toggle between important (red border), ignore (grey) or standard.  The query refreshes taking into account these preferences .  Columns marked ‘ignore’ are not used at all by the machine learning, whereas those marked as ‘important’ are given preference when it creates the query.

In the screenshot below the Wage column is marked as important.  Compare this to the previous image where the name ‘Tom’ was used in the query.

 

Local user explanations

In addition you can click data cells in individual rows to toggle between important (red border), not important (grey) or standard.  This means that for this particular example the relevant field is more or less important.  Note that is a local explanation, just because a field is important for this record selection, it does not mean it is important for them all.

See below the same example with the column headers all equally important, but the cell with contents ‘Tom’ annotated as unimportant (grey).  The generated query does not use this value.  However, note that while the algorithm does its best to follow the preferences, it may not always be able to do so.

 

Under the hood

Query-by-Browsing uses a modified version of Quinlan’s ID3 decision tree induction algorithm, which has been one of the early and enduring examples of practical machine learning.  The variant used in previous versoins of QbB includes cross-column comparisons (such as ‘outgoings>income‘), but otherwise use the same information-entropy-based procedure to build the decision tree top down.

The modified version to take into account global user guidances and local user explanations still follows the top-down approach.

For the global column-based selections, the ‘ignore’ columns are not included at all and the entropy-score of the ‘important’ columns are multiplied by a weighting to make the algorithm more likely to select decisions based on these columns.  Currently this is a fixed scaling factor, but could be made variable to allow levels of importance to be added to columns.

For the local user explanations, a similar process is used except: (a) the columns for unimportant cells are scaled-down to make them less likely to be chosen rather than forbidden entirely; (b) the scaling up/down for the columns of important/unimportant cells depends on the proportion of labelled cells below the current node.  This means that the local explanation makes little difference in the higher-level nodes, where an individual cell is one amongst many unless several have similar cell-level labels, however, as one comes closer to the nodes that drive the decision for a particular annotated record its cell labelings become more significant.

Note that this is a relatively simple modification of the current algorithm.  One of the things we point out in the ‘talking back‘ paper is that user explanations open up a wide range of challenges in both user interfaces and fundamental algorithms.

 

 

 

 

 

fresh version of calQ available – bluring the boundary between calculator and spreadsheet

Fresh version of calQ available, an experimental calculator that gradually blurs the boundary between calculator, spreadsheet and coding.

At first use it as a simple online 4-function calculator, but with a ’till roll’ showing you your calculation history – yes just like the old mechanical desktop ones!

If you want, when you want and when you feel comfortable, click past values and to use them in your current calculation, or to reuse constants such as tax rates. Update past values and seen later ones change, just like a mini-spreadsheet. Name lines of calculations to them to help you remember what you’ve done.

If you find yourself doing the same thing repeatedly, copy an old till roll and edit it, export the till roll into a spreadsheet or use it to make a custom function.

But if you want just add things up 🙂

Politics of Water

Water has been at the heart of Welsh politics for many years as highlighted by an article in the BBC News today1.  However, the impacts of climate change means this is a growing issue across the world.

Man Turns on Water tap

BBC News: Wales ‘missing out on fortune’ over water powers – An ex-minister says he is “aghast” the Welsh government still hasn’t taken control of water policy.

I recall hearing on the radio about the Free Wales Army attacking water pipelines in the 1960s. I was a small child at the time and thought it all sounded very exciting, but I had no idea of the politics behind this.

It was brought home to me when I first paid water rates myself.

As a child, after my dad died, my mum was incredibly good at managing the finances and saving for bills.  Each year the household rate bill would arrive (the UK housing tax for local services).  It would be huge, but as she was on widow’s benefit there was a rebate of 90% of the bill, so the remainder was manageable.  However, this did not include water and sewage.  Once a year the water rates bill would arrive.  It was as big as the standard ‘rates’ bill, but this time there was no rebate, and at that time there was no monthly payment option.  As I said, mum was good at saving towards these big bills, but it was so big, we always knew when it hit the carpet!

Roll on the years and I am paying my own water rates bill for the first time in the early 1980s.  We lived in Bedfordshire, a county of England not known for high rainfall.  My water rates bill was £60 (about £300 today’s prices), but when I talked to my mum, in Cardiff, at the base of the Brecon Beacons with multiple reservoirs, her water bill was £300 (~£1500 today), five times higher.

The reason for this was that the water companies were semi-autonomous.  Wales is full of mountains and consequently expensive to pipe water around the country, hence the high bills.  However, Wales also has lots of water, but as this was piped across the border, there was no commensurate flow of cash back.

Happily this disparity no longer seems to be the case, I assume due to different subsidies to the water companies, but certainly highlights why the control of water is a political issue.

Separating the Waters

In the early months of 2020 the news in the UK was dominated by flooding; Covid-19 was still a distant and uncertain problem compared to the images of homes, shops, and whole communities inundated, often with filthy water contaminated by effluent forced out of drains and sewers. In insurance terms, flood is one of the natural disasters commonly referred to as “Acts of God”. Water seems to be the ultimate blessing or curse of God: the “rain falls on the just and the unjust” (Matt. 5:45) a liberal outpouring for all.

At the same time in the east of Australia bushfire raged, following an unprecedented heat wave. A little further west, in the Murray Darling Basin, a report highlighted that gigalitres of water were being wasted as large volumes of water were directed through the constricted river to almond groves near the sea, by-passing farms ravaged by drought on the way. Farmers looked on helpless as water flowed past by their parched land2.

Almond fields near Mildura. There are fears the Murray-Darling water management regime may not be able to handle the boom in the water-intensive crop. Photograph: Mike Bowers/The Guardian

Guardian 25 May 2019. Tough nut to crack: the almond boom and its drain on the Murray-Darling – Demand for the thirsty crop has created a gold rush but irrigators and growers fear there might not be enough water

As the coronavirus lockdown restricted movement across the world, it highlighted the plight of 15 million US citizens who each year have their water supply cut off for non-payment of water bills. In the UK water companies can sue and eventually bailiffs seize goods, but they cannot, by law, turn off the water supply. Water is deemed essential for human life and dignity. Not so in the US.

In normal times this is bad enough, but at least family members can use toilets and wash in their workplace or public buildings. However during lock down, confined in one’s home, there was no such recourse; bottled water could be brought in, but without a water supply faeces had to be collected and thrown out with the rubbish. One BBC report told the harrowing story of a woman with a family of eleven, who refused to let helpers drop off water at her home for shame of the smell.

How does water, the universal gift of God, become a commodity and privilege?

Water is not mentioned explicitly in the Universal Declaration of Human Rights (UDHR), but  Article 25 guarantees the right to:

“a standard of living adequate for the health and well-being of himself and of his family, including food, clothing, housing and medical care”

In addition, Article 22’s right to “social security” and the “economic, social and cultural rights indispensable for his dignity and the free development of his personality” seems highly pertinent.

Although the order of the 30 articles in the UDHR is not a priority list, it is perhaps telling that Article 25 comes well behind Article 17 which guarantees the right to private property.

Water and War

For many years there have been warnings of the impact of climate change on water supplies and the potential for conflict.  Sometimes this is largely within states as in the case of Australia and the Colorado River in the US (although the latter also impacts North Mexico).  However others cross national borders, such as the long-running disputes on the Nile, including between Egypt and Ethiopia about the construction of the Grand Ethiopian Renaissance Dam.  There have always been water wars, but the likelihood and severity are expected to rise.

It was significant that one of the first actions of the Russian invasion into Ukraine was to reopen the North Crimean Canal, which had been dammed by the Ukrainian government in 2014 cutting off the majority of the fresh water supply to the two and half million people of the Crimean peninsula.  The crisis was largely silent for many years, perhaps in part because Russia does not like to admit weakness, but back in 2020 there were warnings from open source news sites of the ever growing human, ecological and geopolitical crisis. and by 2021 this was picked by Bloomberg and the FT, the latter describing it as a ‘water war‘.  Although there are many interlinked reasons for the conflict in Ukraine, it may be we are already seeing the first major modern water war.

North Crimean Canal. Connects the Denpr at the Kakhovka reservoir with the east of Crimea.

Wikipedia: North Crimean Canal (image: Berihert, CC BY-SA 3.0)

 

  1. Thanks to Alan Sandry for pointing out the BBC article.[back]
  2. See 9News “Struggling Aussie farmers enraged by incredible water wastage” and full Australia Institute report “Southern discomfort: water losses in the southern Murray Darling Basin“.[back]

The seeing coat – y gôt gwylio

(N.B. strictly the ‘watching coat’)

Dw i’n dysgu Cymraeg …
I am learning Welsh …
felly, dw i’n gwneud llawer o camsyniad.
so I make lots of mistakes.

Neithiwr, ro’n i eisiau dweud “côt gwisgo” …
Last night I wanted to say “dressing gown” …
ond yn wir,  dwedais i “côt gwylio”.
but really, I said “watching coat”.

Yn fy nghwrs creadigrwydd, dw I’n dweud …
In my creativity course I say …
“camsyniad yw cyfle”
“a mistake is an opportunity”

Felly, y bore ‘ma ro’n i’n meddwl ..
So, this morning I thought …
‘Sai côt gwylio gyda fi …
If I had a seeing coat …

Taswn i’n edrych ar y coed,
If I were to look at the trees,
Baswn i’n gweld pob un goeden, pob boncyff, cangen, brigyn, a dailen,
I would see each tree, each trunk, branch, twig and leaf.

Taswn i’n edrych ar blodyn,
If I were to look at a flower,
Baswn i’n gweld petlau ac llwch pail, un gwenyn yn yfed.
I would see petals and pollen dust, a single bee drinking.

Taswn i’n edrych ar y pobl yn y stryd,
If I were to look at the people in the street,
Baswn i’n gweld tad, merch, ffrind a diethryn, yn hapus a thrist, yn gyffrous, yn hiraethu
I would see father, daughter, friend and stranger, happy and sad, excited, longing
Dim pobl, ond pob un ohonynt yn berson unigryw.
Not people, but each one of them a person, unique.

Taswn i’n edrych i’r awyr,
If I were to look to the sky,
Baswn i’n gweld pob un gwmwl ac aderyn, ac y tu hwnt i’r awyr
I would see each cloud and bird, and beyond the sky
Yr wybren wedi’i phaentio i mewn mil o liwiau o glas.
The firmament painted in a thousand colours of blue.

Ond yn wir, nid oes eisiau “cot gwylio” arnaf i,
But really, I do not need a “seeing coat”
y cyfan sydd ei angen yw edrych.
all that is needed is to look.

Meta-tags made easy

I wanted to add OpenGraph/Twitter Card meta-tags to a few static web pages.  It would have easily taken 5-10 minutes each, which seemed tedious, so I took … well let’s say a little longer (!) … and made a general purpose online tool for creating the markup.

If you want your web pages to look good when shared on Twitter and Facebook, it is important to have the right meta tags in the <head> section of the page.  However, as these are invisible it is easy to forget.

It is not that complicated to do, copying your page title and description into a few different meta tags.  The bit I find hardest is remembering (well looking up, no way I’d remember), which of the meta tags are name–content ones and which are property–content ones … I’m sure there is a logical explanation …

The tool allows you to enter the details of your page in a web form and then dynamically creates a preview of the Twitter Card and, most important, generates the meta tags to copy into your web page <head> section.

I’d come across JsViews/JsRender recently, which is used in several Microsoft products.  It is similar to Angular (from Google stable) in that it allows you to create two-way templates, dynamically binding data to HTML content.  My colleagues when i was at Talis used Angular and I’ve used it myself for one project, but I found it hard to adapt when what I wanted didn’t quite fit its model … although, to be fair, I do tend to use everything to breaking point!

JsViews is closely aligned to jQuery, which I use pretty much everywhere, so it seemed sensible to have a go at using JsViews for this micro-project as this was essentially a single page web page with no backend.  So far so good with JsViews; although, again to be fair, this was a lot easier than what I’d been doing with Angular.

I provided a few click-to-see examples using my own pages and of course an obvious example was the page of the online tool itself … only no image. I did emphasise in the tool how useful it is to have a good image for social media sharing, and the Twitter Card looked a bit boring.

So the fastest logo ever (thanks to Wikimedia and OpenGraph for the building blocks!

Scopus vs Google Scholar in Computer Science

In response to a Facebook thread about my recent LSE Impact Blog, “Evaluating research assessment: Metrics-based analysis exposes implicit bias in REF2014 results“, Joe Marshall commented,

“Citation databases are a pain, because you can’t standardise across fields. For computer science, Google scholar is the most comprehensive, although you could argue that it overestimates because it uses theses etc as sources. Scopus, web of knowledge etc. all miss out some key publications which is annoying”

 

My answer was getting a little too complicated for a Facebook reply; hence a short blog post.

While for any individual paper, you get a lot of variation between Scopus and Google Scholar, from my experience with the data, I would say they are not badly correlated if you look at big enough units.  There are a few exceptions, notably bio-tech papers which tend to get more highly placed under Scopus than GS.

Crucial for REF is how this works at the level of whole institution data.  I took a quick peek at the REF institution data, comparing top quartile counts for Scopus and Google Scholar. That is, the proportion of papers submitted from each institution that were in top 25% of papers when ranked by citation counts.  Top quartile is chosen as it should be a reasonably predictor of 4* (about 22% of papers).

The first of these graphs shows Scopus (x-axis) vs Google Scolar (y-axis) for whole institutions.  The red line is at 45 degree, representing an exact match.  Note that, many institutions are relatively small, so we would expect a level of spread.

inst-scopus-vs-google-top-quartile-with-line

While far from perfect, there is clustering around the line and crucially for all types of institution.  The major outlier (green triangle to the right) is Plymouth which does have a large number of biomed papers. In short, while one citation metric might be better than the other, they do give roughly similar outcomes.

This is very different from what happens in you compare either with actual REF 4* results:

inst-scopus-top-quartile-vs-REF-4star-with-line   inst-google-top-quartile-vs-REF-4star-with-line

In both cases not only is there far less agreement, but also there are systematic effects.  In particular, the post-1992 institutions largely sit below the red line; that is they are scored far less highly by REF panel than by either Scopus or Google Scholar.  This is a slightly different metric, but precisely the result I previously found looking at institutional bias in REF.

Note that all of these graphs look far tighter if you measure GPA rather than 4* results, but of course it is 4* that is largely what is funded.

hope and despair

I have spent a good part of the day drafting my personal response to Lord Stern’s review of the Research Excellence Framework; trying to add some positive suggestions to an otherwise gloomy view of the REF process.

My LSE impact blog “Evaluating research assessment: Metrics-based analysis exposes implicit bias in REF2014 results” also came out today, good to see and important to get the message out, but hardly positive; my final words were:

“despite the best efforts of all involved, the REF output assessment process is not fit for purpose”,

and this on a process that consumed a good part of a year of my life … depressing.

However, then on Facebook I saw the announcement:

Professor Tom Rodden announced as EPSRC's Deputy CEO

Yay, a sensible voice near the heart of UK research … a glimmer of light flicker’s on the horizon.

 

 

Big themes

We were talking about the big themes, and what bigger theme than Christmas.  John talks of the Word, that pre-exists all, the Logos, the ruliness we seek in random events, the laws of the universe examined in CERN and comet-hugging satellite, the conversation between God and creation, the Word that was singly, irrevocably and powerfully spoken, the Word that says all and is all, the Word that is “Love”.

It was a Word declared through 10 billion years of the dark star-thread universe, a Word sung by incomprehensible angels, a Word of cosmic significance; but it was no abstract Word, no Word of plain intellectual study, but a Word made Flesh.

Wriggling, scrawny, damp-wrinkled flesh, still flecked with the drying blood of Mary torn in childbirth, prefiguring another bloody day, like and unlike every other baby, letting out one unignorable, earth-shaking cry.

why Ukraine upsets me – the death of democracy

As is probably evident from occasional Tweets or Facebook comments, I can get hot under the collar about the events in Ukraine.  This is for several reasons, but for now the first and most important reason.

It should shock us all.

For the first time since the Second World War, we have seen the violent overthrow of a democratically elected government in Europe.

Let that sink in.

It does not matter whether you are pro-European or pro-Russian, left wing or right wing. In February we saw the overthrow of democracy in Europe.

That is shocking.

We have seen conflict in Europe before.  The breakup of the Iron Curtain was largely peacefully, with just the the odd tank fire on the Russian Parliament, but that can be passed by as the final effects of totalitarian government.  In Yugoslavia we saw a democratic state implode after the secession of part of its people, the violent reaction of central government and eventual ethnic cleansing.

We have seen democratic governments overthrown elsewhere: in Chile in the 1970s, in Egypt more recently.  However, the first was during the Cold War days, when anything was acceptable to defend against communism (the CIA even made contingency plans to  overthrow the Harold Wilson Labour government!), and in the latter case the Muslim Brotherhood government were arguably working to remove aspects of democracy.

But Ukraine was different.  It is clearly a complex situation, and it is easy to be critical from a distance.  The politics there is factional, rather like Northern Ireland, and clearly driven very strongly by the super-rich, not so unlike the US.  So, whether the pre-February Government in Ukraine was a good or bad one is a matter of debate, but its election was not.  The 2010 poll was operated by the previous pro-European administration, so there was no element of Gerrymandering, and, as far as any poll in the area could be, it was regarded as fair.

That is, there was no challenge to the fundamental democratic legitimacy of the pre-Feb 2014 government.

Yet, we watched and encouraged its violent downfall.

It is almost hard to recall now.  Certainly the BBC tends to remind us of the events in Crimea and the rise of pro-Russin separatists as the start of the conflict, but of course these were the response not the start.

The start was the Maidan protests; the daily images of police lines with fire bombs raining down, the arming of the protestors, the eventual bloody clashes with large numbers of protesters and also many police left dead on the streets, and the ensuing decision of the president to give up power rather than bring the army onto the streets of Kiev.

I imagine this was Britain.  Thatcher in 1990 and the Poll Tax riots; Blair in 2003 and the anti-Iraq War protests; Cameron in 2012 and the ‘Occupy’ movement.  In all cases, my sympathy was with the demonstrators, in all cases I wished the government had listened more to them, just as I’m fairly sure most of the Ukrainians I know would have supported the original causes of Maidan.  However, neither I nor anyone in the UK, whether they sided with the Poll Tax, Anti-War or Occupy protesters or with the government, would have wished for this to end with the overthrow of the government and the remaining parliament making decisions with the masked ex-protestors standing with automatic weapons at the doors of Westminster.

This is what happened in Ukraine.

And we in the West supported it, indeed encourage it.  US senator John McCain visited the demonstrators early, while they were still a peaceful ‘occupy’-like movement.  However,  EU representatives were there after the far-right elements had armed.

The press dressed this as people power against autocratic government.

It never was, simply a democratic government that made a decision that a large minority of its people (often violently) disagreed with, and most fundamentally was not one that we in the west agreed with.

And for the majority of people in Ukraine, for those who voted a government they trusted, we taught them that democracy does not pay, that democracy is a sham, that democracy is only good if the government you elect does the ‘right’ things as judged by the western media.

We have witnessed the death of democracy.

And applauded it.

 

just running (and the odd walk)

I now feel  little more prepared for Sunday’s 35 mile Tiree ultramarathon, which is following the coats of the island as closely as possible.  Sort of a bit like my walk around Wales, but on a smaller scale!


Tiree Ultamarathon Route

I’m planning to do a mix of run and walk.  The target is to get round in 10 hours.  This would be a doodle if it were all road and beaches as I can average 4 miles an hour fast walk, but the rough ground sections will be slow, so I need to run when I can to make up.

Today I ran and walked the section of the route that goes round the east end of the island, 11.5 miles on Sunday’s route, then 2 miles back across the island to home, and then, to cap off threw in a little 7.5 mile run down to Hynish and back.  In all 21 miles in five and a quarter hours, so the running and slow walking averaging out at 4 miles an hour, on track. I’m bound to slow a little as the day wears on, and the weather is set to be less good on Sunday, but it’s good to know I’m in the right ballpark.

My only problem is eating enough while moving.  I did manage to eat a Mars bar, but find it hard to eat when I’ve just been running, so had to wait for the long walk sections — I guess why the professional runners all use those gel packs.  I think I’ll get some more Kendal Mint Cake as that is far easier to simply suck/crunch and swallow, and no load on the stomach — straight sugar!

Of course, now it is evening I have to try and catchup with all the work I should have been doing in those five and quarter hours :-/

I’m not really doing it ‘for’ anything , but if you feel inspired the JustGiving pages on AlanWalksWales are still open for donations.