The Canada Revenue Agency considers its multimillion-dollar artificial intelligence chatbot a success even though internal records show it provides inaccurate answers to taxpayers 10% of the time.Blacklock's Reporter says Access to Information documents reveal the CRA set a benchmark requiring its chatbot, known as Charlie, to maintain a 90% accuracy rate when answering questions from Canadians. The federal tax agency did not explain why a system delivering incorrect information one out of every 10 times was deemed acceptable.“No generative artificial intelligence system can achieve 100% accuracy,” managers wrote in an internal memo outlining the project's success criteria.The chatbot was launched in 2025 as part of the agency's effort to modernize taxpayer services and provide quicker access to information. According to figures tabled in Parliament, the project cost taxpayers $18.06 million, including $3.2 million paid to outside consultants.The spending details were disclosed in response to a parliamentary inquiry from Conservative MP Eric Melillo, who requested a breakdown of development, operational and maintenance costs associated with the Charlie chatbot.“The agency has embarked on a journey to explore the transformative potential of artificial intelligence,” said internal management speaking notes. “Our beta chatbot deployment launched earlier is already showcasing the immense opportunity to enhance how we serve Canadians.”Officials described the chatbot as a key part of a broader effort to modernize government services.“Our chatbot service is not just a technological upgrade,” the notes stated. “It is a transformative opportunity to modernize how we serve Canadians.”Despite that optimism, internal reports indicate taxpayers were largely unimpressed with the service.An Oct. 29 report found only 36.4% of users who provided feedback expressed satisfaction, while 63.6% reported a negative experience. The report noted frequent requests from users seeking to speak with a live CRA agent after interacting with the chatbot.“Challenges remain, including issues with contextual accuracy and user feedback functionality such as incorrect information,” one memo stated. “It falls short of meeting users’ expectations.”.Officials acknowledged many users expected the chatbot to resolve problems rather than simply point them toward information.Records show Charlie struggled to answer a variety of common taxpayer questions, including whether tax returns had been received, how to establish HST instalments, how to update phone numbers for multi-factor authentication and how to recover account access codes.In one instance, a taxpayer asked what an “OCCR underpayment for April 2025” meant. The chatbot replied that the question might be outside its expertise.The system also directed users to obsolete tax forms and incorrectly advised that direct deposit information could still be changed over the phone.The findings come after previous audits revealed human CRA call-centre agents also provided inaccurate information at troubling rates, with errors occurring as much as 46% of the time.Revenue Commissioner Bob Hamilton has maintained the agency is not liable when taxpayers suffer financial consequences after acting on incorrect information provided by CRA staff.During testimony before the House of Commons public accounts committee last October, Conservative MP Gérard Deltell challenged Hamilton on whether Canadians should bear the cost of government mistakes.“Who is going to pay for the mistake you made, the honest citizen or the Canada Revenue Agency?” Deltell asked.“It is a problem,” Hamilton replied.Deltell then questioned whether the CRA had calculated how much money Canadians had lost as a result of inaccurate information.“I very much appreciate the frustration,” Hamilton responded.
