AI hallucinations: A risk or a threat in the government sector?

The integration of artificial intelligence (AI) technologies in government funded agencies has garnered considerable attention, with a growing body of work dedicated to safely realising the benefits AI offers. Early initiatives such as Australia’s voluntary AI ethics principles, draw upon the development of Australia’s AI ethics framework consisting of eight principles that organisations can use to:

• achieve superior outcomes with AI
• minimse the risks of adverse impacts
• uphold the highest standards of ethical business and governance.

This effort will be further supported and expanded through the Australian Government’s recently announced AI expert group, which will provide guidance on testing, transparency, and accountability measures for AI in legitimate yet high risk contexts. The group comprises expertise in Indigenous Cultural and Intellectual Property, Law, technology and ethics.

The Australian Government is behind a push to reach 1.2 million tech related jobs by 2030 and is offering a new free course in AI 101 to small and medium business owners. The program covers topics including challenges and risks, common misconceptions, real world applications, and advice from industry experts to start your career in AI.

Relevance of AI in the Public Sector

The 2023 Report from the Australian Government Department of Prime Minister and Cabinet, titled "How might artificial intelligence affect the trustworthiness of public service delivery?" offers valuable insights. Citizens increasingly demand higher standards of care, personalised services, and greater efficiency when interacting with government services. AI holds the potential to revolutionise public service delivery, offering enhanced experiences and outcomes for the community. Current AI applications in the public sector include chatbots, virtual assistants, document and image recognition for border control, fraud detection and data mapping.

Relevance of AI in the Local Government Sector

AI has the potential to transform how local councils deliver services, enhancing efficiency in areas such as planning applications, rate collection, and aiding in data analysis and cost-preventative maintenance. Data-driven decision-making can assist local councils in making well-informed choices regarding resource distribution, financial planning, and policy development. Advanced sensors and AI-driven algorithms can anticipate maintenance or repair needs for infrastructure components such as roads, bridges, and utility systems.

Challenges and Risks Associated with AI

Emerging evidence suggests that unregulated AI can exacerbate societal disparities. Studies, like a recent examination of AI bias in America, reveal prejudices against marginalised groups, potentially influencing practices and perpetuating endemic biases in employment, education, insurance, and housing sectors. Furthermore, using AI for data collection and analysis raises privacy and security concerns that must be effectively managed. Without proper controls, AI systems may rely on flawed algorithms, making it difficult to track or explain decisions, akin to the issues seen with Robodebt.

Framework for Trustworthy Stewardship

In response to these challenges, public agencies must adopt a framework that ensures trustworthy stewardship of AI systems by:

• Establishing ethical guidelines aligned with community values, human rights principles, and legal obligations.
• Ensuring transparency in data collection processes, decision-making algorithms, and outcomes, while addressing potential biases.
• Implementing accountability mechanisms to hold individuals and teams responsible for AI-related decisions, supported by formal governance, oversight, and escalation procedures.
• Strengthening data governance policies to ensure data quality, integrity, and privacy, clarifying ownership, access controls, sharing agreements, and compliance with regulations.
• Conducting comprehensive risk management exercises to identify and mitigate biases, discrimination, security vulnerabilities, and unintended consequences.
• Building AI systems on the principles of fairness and equity, and which include strategies for monitoring discriminatory impacts.
• Maintaining human oversight to ensure accountability, transparency, and intervention capabilities in critical decisions.
• Continuously monitoring and improving AI systems over time.

As AI capabilities advance rapidly, public organisations in Australia must position themselves to maximise opportunities for improving government service provision. By adopting a framework for trustworthy stewardship, agencies can mitigate risks and harness the full potential of AI in serving the community.

About Us

Centium is an independent assurance and audit firm that focuses on helping clients manage their risks. We partner with Australian state and local government, not-for-profit organisations and private sector clients to provide a complete solution to managing organisational risk, enhancing governance and improving operational performance. We believe that the principles of effective governance and risk management apply to all aspects of managing an organisation, from procurement, to probity to technology and business transformation.

Useful References

ACS urges action as AI disruption looms | Information Age

Artificial Intelligence Ethics Policy | Digital.NSW and

Mandatory Ethical Principles for the use of AI - Digital.NSW