How to Conduct an AI Risk Assessment
AI is no longer a side project running in a lab. It is embedded in hiring decisions, credit approvals, medical...
In August 2026, a story out of Anhui province, China started making the rounds on tech news sites for a reason that had nothing to do with chips, benchmarks, or funding rounds. A farmer asked an AI chatbot how to deal with weeds and pests on his sesame field. He followed the advice exactly. By the next morning, close to 25 acres of sesame seedlings were dead.
It's a small story in the grand scheme of AI news, but it's also one of the clearest examples yet of what happens when AI-generated advice meets a real-world decision that can't be undone. If you work with AI tools, advise clients on AI adoption, or you're simply trying to understand where the real risks of AI actually sit, this incident is worth breaking down in detail, from the exact chemical mix-up to the wider question of who actually carries the risk when a chatbot gets something this wrong.
The farmer at the center of this story is a 67-year-old man surnamed Wu, from Chuzhou in China's Anhui province. According to Taiwanese outlet CTWANT, which first reported the story, Wu had been using an AI app for farming advice for about a year. He wasn't an early convert. He was skeptical of the tool at first, and only started relying on it after it gave him useful answers, month after month, on smaller decisions.
That's an important detail, and we'll come back to it. Trust built slowly, over dozens of small interactions that went fine.
This time, Wu asked the AI for help controlling weeds and pests across his 150-mu plot, which works out to roughly 24.7 acres, planted with sesame seedlings. The AI recommended a herbicide mix, reported by different outlets as containing flufenacet (also referred to as "flusulfasulfaether") or fomesafen, depending on the source, sometimes combined with additional compounds referred to in local reporting as "high-efficiency flupyrimethalin" and a "thiamethoxazine" and "methyl salt" mix. Wu applied it across the entire field, exactly as the chatbot suggested.
By the following morning, his sesame crop was dying. Not damaged. Dying. Local agricultural technicians who inspected the field afterward confirmed what had gone wrong, and the case quickly spread through Chinese-language media before being picked up by tech outlets internationally.
Laying the sequence out plainly makes the failure easier to see:
That gap between step 5 and step 7 is really the whole story. The AI had the correct information. It just never surfaced it until it was too late to matter.
The herbicide the AI recommended is designed to kill broadleaf weeds, and it's commonly used in soybean fields. That's a legitimate, well-established use case. The problem is that sesame is also a broadleaf plant. Spraying a broadleaf herbicide across a sesame field doesn't just risk the crop, it targets the crop directly, in the same way it would target the weeds it's meant for.
When Wu went back to the AI afterward and asked why his field had died, it correctly identified the herbicide as the cause. The tool could explain the mistake after the fact. It just hadn't warned him before the damage was done.
There was a second layer to the error. Agricultural experts noted that even where this type of herbicide is safe to use, it's meant to be spot-sprayed on the specific patches where weeds are a problem, not applied uniformly across an entire field. Wu sprayed the whole 25 acres at once, following the advice as given.
So there were really two mistakes stacked on top of each other: the wrong chemical for the crop, and the wrong application method for that chemical. Either one alone might have been a survivable error. Together, they wiped out the season.
It's tempting to read this as a case of "the AI got the amount wrong" or "the timing was off." It wasn't that kind of mistake. Agronomists who reviewed the case pointed out that this was a category error: the model matched "clear the weeds" to "broadleaf herbicide" without registering that the crop itself is also a broadleaf plant. It answered the question that was asked, technically correctly for a soybean field, without catching that the context made the advice dangerous.
That distinction matters a lot if you're thinking about AI risk. A dosage mistake is a tuning problem. A category error, where the system generates a technically coherent answer that's wrong for the actual situation, is a much harder thing to catch, because the output looks confident and complete. There's no obvious red flag in the response itself. You'd need domain knowledge, or a second source, to catch it before acting.
Wu didn't have that second check. He had a chat interface with a small disclaimer at the bottom saying AI-generated content might be incorrect and should be verified. He'd seen that disclaimer, presumably, dozens of times before, on questions where the AI turned out to be right. That's a hard habit to break on the one day it matters.
It's worth sitting with this question a little longer, because it's the part people usually skip past. The AI wasn't lying, and it wasn't malfunctioning in any obvious technical sense. When Wu asked why his crop died, it gave a correct, coherent explanation almost immediately. The information was in there the whole time.
What was missing wasn't knowledge, it was a check. A competent human agronomist, hearing "I want to clear weeds from my sesame field," would ask a follow-up question or flag the crop type before naming a chemical. A general-purpose AI model, trained to be helpful and to answer the question as asked, tends to go straight to a plausible-sounding recommendation instead. It doesn't have a built-in habit of pausing on requests that could cause irreversible damage, because most requests it handles don't.
This is a pattern that shows up well beyond agriculture. A model can hold the correct information and still fail to apply it at the one moment it matters, because nothing in the interaction prompted it to slow down.
This is where the story gets uncomfortable, because there isn't a clean answer.
A representative from the AI software company reportedly told local media that the tool doesn't maintain an independent knowledge base and pulls from publicly available information on the internet, and that the company would look into where the specific recommendation came from. That's a fairly standard response, and it points to a structural issue: general-purpose AI tools trained on public data don't necessarily verify agricultural guidance against a crop's actual classification before repeating it.
From a liability standpoint, commercial farm losses like this one tend to fall into a legal gray zone. Consumer protection rules and emerging AI product liability frameworks are mostly built around personal use and consumer goods. A lost commercial harvest is much more likely to hinge on the terms of service the farmer agreed to when he opened the app, and those terms almost always shift risk back onto the user.
In other words: the disclaimer did its legal job, even if it didn't do its practical job.
This is exactly the kind of gap that organizations doing real AI risk assessment work need to plan for before deployment, not after. If you're evaluating where AI tools introduce risk into a workflow, whether that's agriculture, healthcare, finance, or anything with physical or financial consequences, it helps to have a structured process for it. We've written a more detailed breakdown of how that process works in How to Conduct an AI Risk Assessment, which covers how to map out where AI-generated decisions could cause irreversible harm before they happen, not after.
The Chuzhou incident stands out because the damage was so visible and so total, an entire field, dead overnight. But it fits a pattern that's been building for a while, where AI tools give advice with total confidence in situations where a bit of hesitation would have served the user better.
Around the same period this story broke, a developer publicly described losing an entire project directory after an AI coding assistant misread a routine backup instruction and deleted files it wasn't supposed to touch. Different domain, same shape of failure: a tool that answered the literal request without registering how much was riding on getting it right.
There have also been well-documented cases of AI customer service tools inventing policies that companies were later held to, and of AI medical symptom checkers offering advice that sounded specific and authoritative while being based on incomplete information. None of these are reasons to avoid AI altogether. They're reasons to think carefully about where in a workflow AI output needs a human check before it turns into an action, and where it doesn't.
A few things stand out when you look at this incident closely.
Trust builds on easy questions and gets spent on hard ones. Wu didn't trust the AI blindly from day one. He tested it, it worked, and over months that track record turned into confidence. That's a completely reasonable way for a person to build trust in a tool. It's also exactly how overreliance happens. The AI being right 50 times in a row tells you nothing about whether it'll be right on the 51st, especially if the 51st question is higher stakes than the first 50.
Wrong agronomic advice can't be corrected on the next cycle. In software, you find a bug, you patch it, you ship again. In farming, you find out the chemical was wrong the morning after you sprayed it, when the field is already dead and the season doesn't repeat. Some domains simply don't forgive a wrong answer the way others do, and it's worth knowing which kind of decision you're making before you act on AI output.
A disclaimer is not the same as a warning. "AI generation may be incorrect, please verify" is a legal safety net, not a risk communication. It doesn't tell you when to be careful. It doesn't flag that this particular answer touches something irreversible. A generic disclaimer sitting at the bottom of every chat, right or wrong, trains people to tune it out.
The error looked complete, which made it more dangerous, not less. The AI didn't hedge, didn't flag uncertainty, and didn't ask what crop it was talking to before recommending a chemical. A confidently wrong answer is far more dangerous than an uncertain one, because it doesn't invite a second opinion.
Scale turns a mistake into a disaster. Had Wu tested the herbicide on a small corner of the field first, this would be a minor story about a bad recommendation caught early. Applying it across all 25 acres at once, on the AI's word alone, is what turned an error into a wiped-out season.
If you're using AI for anything with real-world consequences, agriculture, health, legal, financial, or operational decisions, a few habits go a long way:
Stories like this one are starting to show up in policy discussions, not just tech news cycles, because they expose a gap that most current AI regulation hasn't fully closed. Frameworks like the EU's approach to AI product liability are beginning to bring software into the same category as physical products for consumer harm, but commercial and agricultural use often sits outside that protection. A farmer's lost harvest is treated as a contract dispute, governed by whatever terms of service he agreed to, rather than as a product safety failure.
That gap matters because agriculture is exactly the kind of sector where AI adoption is accelerating fastest among people who aren't AI specialists. Farmers, small business owners, and individual professionals are increasingly using general-purpose chatbots for domain-specific advice, without any of the guardrails that a specialized, regulated tool would be required to have. As AI tools spread into more high-stakes corners of daily work, the question of who is accountable when the advice is wrong is only going to come up more often, not less.
This is precisely the gap that structured AI governance training is meant to address, whether you're building AI policy for an organization, advising clients on AI adoption, or simply trying to understand where the guardrails need to sit. Organizations that deploy AI into workflows with real consequences need more than a disclaimer buried in the terms of service. They need a process for identifying where AI output touches irreversible decisions, and a plan for human checkpoints at exactly those points. That's a core part of what proper AI governance and risk management training is built to teach, and it's the same thinking behind courses like AI Readiness: Implementation, Adaptation, and Scaling of AI, which walks through how to roll out AI tools into real operations without the kind of gap that cost Wu his season.
If you're the one responsible for how AI gets adopted across a team or organization, it's also worth looking at frameworks built specifically for that role, like the Chief AI Officer (CAIO): Build AI Strategy & Lead course, which covers exactly this kind of risk-to-decision mapping at an organizational level. You can browse the full library of governance and compliance courses here.
It's easy to read this as an agriculture story and move on. It's really an AI governance story that happened to play out in a sesame field. The same pattern, a tool that answers confidently, a user who reasonably trusts it after a good track record, and no structural check before an irreversible action, shows up in hiring decisions, medical triage tools, financial approvals, and industrial operations. The chemical changes. The failure mode doesn't.
None of this is about avoiding AI. Wu's year of useful advice before the mistake is proof the tool had real value most of the time. It's about knowing which questions are cheap to get wrong and which ones aren't, and building a habit of checking before you act on the ones that aren't.
A 67-year-old farmer in Chuzhou, Anhui province, China, asked an AI chatbot for weed and pest control advice for his sesame field. The AI recommended a broadleaf herbicide that isn't approved for sesame and suggested spraying it across the whole field rather than only the affected patches. He followed the advice, and about 25 acres of his sesame seedlings died within a day.
The herbicide the AI recommended is designed to kill broadleaf weeds, commonly used in soybean fields. Sesame is also classified as a broadleaf plant, so the chemical affected the crop the same way it would affect the weeds. On top of that, the herbicide is meant for targeted spot-spraying, not full-field application, which made the damage worse. For more on how this kind of failure gets identified and mapped before deployment, see our guide on how to conduct an AI risk assessment.
It's legally unclear. A representative from the AI software company said the tool pulls from publicly available internet information rather than an independent, verified knowledge base, and said they would investigate the source of the advice. Commercial losses like this generally aren't covered by consumer protection or emerging AI product liability rules, which mostly apply to personal use. In practice, the outcome likely depends on the terms of service the farmer agreed to when using the app.
AI tools can be useful for general farming questions, and this farmer had a year of reportedly helpful interactions before this incident. But general-purpose AI models aren't a substitute for checking a chemical's approved crop list, testing on a small area first, or consulting a local agricultural extension office before applying anything across an entire field. Treat AI advice as a starting point for high-stakes agricultural decisions, not a final answer.
The key steps are testing AI-recommended actions on a small scale before scaling up, cross-checking high-stakes advice against a second, verified source, specifically asking the AI what could go wrong rather than only how to proceed, and treating irreversible actions (like spraying a full field or submitting a legal filing) with extra caution regardless of how reliable the tool has been so far. Organizations rolling out AI across teams benefit from a structured risk assessment process, which you can explore through our AI governance and compliance courses.
A dosage error means the AI got a quantity or timing wrong within an otherwise correct recommendation. A category error, like what happened in this case, means the AI matched the request to a technically valid but contextually wrong answer, in this case, a soybean herbicide for a sesame field. Category errors are harder to catch because the response looks complete and confident, with no obvious red flag pointing to the mistake.
Yes. Around the same time as the Chuzhou incident, there were reports of an AI coding tool deleting a developer's entire project directory after misreading a backup instruction, and there have been earlier, well-documented cases of AI customer service bots inventing company policies and AI symptom checkers giving confident but incomplete medical guidance. The common thread across all of these is a tool answering a literal request without recognizing how much was riding on getting it right.
At minimum, verify the recommended product against its official label or approved crop list, apply it first to a small test area rather than the whole field, and check with a local agricultural extension office or licensed agronomist before treating anything at scale. Chemical application is exactly the kind of irreversible, high-stakes decision where a second opinion is worth the extra day it takes to get one.
AI is no longer a side project running in a lab. It is embedded in hiring decisions, credit approvals, medical...
Ai Governance
As AI spreads through business and daily life, so do the moments when it goes wrong, and those moments now...
Ai Governance
If a regulator or a court asked your business to prove that one of your AI systems is fair, safe,...