Croydon (Parliament Politics Magazine) – Croydon Council begins testing an artificial intelligence system with only 80% accuracy, raising concerns over reliability and implications for decision-making.
A flawed method for handling Council Tax inquiries has been released to the public by the borough’s administrators without any prior notice.
Croydon Council has begun to be overrun by the robots. The council acknowledges that they are only operating effectively 80% of the time at this point.
Naturally, the local government that forbids its employees from learning about events occurring in their own borough is Croydon Council.
Additionally, it appears that the council does not want the people of Croydon to know that they are being used in an experiment for the new “customer-facing” artificial intelligence that the borough’s bureaucrats are developing.
Yesterday, the council discreetly posted an announcement on a far-flung section of the authority’s official, little used website.
Whether they like it or not, Croydon residents will be the test subjects for the council’s cost-cutting experiment. The council even claims in internal documents obtained by Inside Croydon that its faulty AI Assistant system cannot send gullible members of the public to a staff worker for assistance with their questions.
The council acknowledges on its internal computer system today that their contact center lacks the resources necessary to handle the one million inquiries they receive annually. On its intranet, the council informed employees that the volume of calls is “putting immense pressure on service teams.”
Even the “digital first” council doesn’t seem to be quite sure that the data on its website is correct or current.
“The project group has been working closely with our Council Tax team to make sure our website content is up-to-date, accurate and as user-friendly as possible,”
staff were told.
In the message to staff, the council admits that what they have unleashed on the public is not entirely reliable:
“It currently works by searching the council website for the information it needs. This is why we have been working closely with teams to ensure our web content is as clear and accurate as possible.
In its current format it can’t search for information beyond the council website and it can’t transfer residents to a contact centre representative if needed, although it can give them the contact details for onward support.
As is the nature of this tech, with each iteration it’ll keep getting better and evolving as we go.”
So that’s alright then.
Croydon’s AI Assistant is based on Microsoft Copilot. On its public-facing website, the council says that Croydon’s AI Assistant pilot aims to help residents find answers to their questions about Council Tax more quickly and easily, using the information on the website.
The council receives more than 1 million contacts per year, and the aim is to speed up response times at a time when lots of residents want to quickly find information about their Council Tax.
The problem is that the council acknowledges that its AI robots have only answered 80% of the questions correctly in tests.
When managing residents’ Council Tax bills, there appears to be a huge chance of making expensive mistakes.
Staff have been told:
“Where the assistant gives the wrong answer or comes back with nothing, it has mainly been due to our website not having the correct information or being unclear.”
Hardly reassuring.
Introducing a system that is only 80% reliable to the general public may appear strange, even a bit premature.
However, this is a financially stressed Croydon that has been criticized by government commissioners on numerous occasions for failing to make operating savings in a timely manner.
The council’s £204,000 chief executive, Katherine Kerswell, oversaw the £6.4 million expenditure on outside consultants last year in order to get their opinion on how to make additional cutbacks.
One of the easier answers to that exercise was the use of AI robots.
The council’s AI project may also raise significant data protection concerns, particularly if the council doesn’t adequately and beforehand notify the public about how it plans to use their data.
On the council website, they say: “This is a pilot, so the council will be gathering lots of feedback and working with residents to see how the assistant works for them.” Hmmm.
Introducing a system that is only 80% reliable to the general public may appear strange, even a bit premature.
However, this is a financially stressed Croydon that has been criticized by government commissioners on numerous occasions for failing to make operating savings in a timely manner.
The council’s £204,000 chief executive, Katherine Kerswell, oversaw the £6.4 million expenditure on outside consultants last year in order to get their opinion on how to make additional cutbacks.
One of the easier answers to that exercise was the use of AI robots.
The council’s AI project may also raise significant data protection concerns, particularly if the council doesn’t adequately and beforehand notify the public about how it plans to use their data.
How is the Croydon council planning to address the 2200% failure rate of the AI system?
Regularly updating and retraining the AI system with fresh, high-quality data is one of the best strategies to lower AI failure rates. Since the AI assistant depends on this information to provide proper answers to questions, it is imperative that the content on the council’s website is current and accurate.
Improving data management is a crucial first step because AI failures are frequently caused by outdated or poor quality data.
Before implementing the system more widely, the council can find flaws and improve it by using a phased approach, beginning with trial projects, getting input, and making small adjustments. As certain failure sites appear, this iterative method aids in addressing them.
The council can assess accuracy by clearly defining the AI system’s goals and key performance indicators (KPIs).