Cisco Tried to Use AI to Write Security Incident Reports and Things Didn't Go as Planned

Cisco warns that AI-generated incident reports are often inaccurate, inconsistent, and prone to data loss due to LLM limitations
The company recommends granular single-task prompts, fixed source documents, and strict formatting rules to improve reliability.
Cross-contamination between reports remains a challenge, and researchers recommend new sessions for each new incident report to avoid errors.

Any company looking to use AI tools for their security reporting may want to read a new report from Cisco that outlines its experience using AI-generated incident reports.

The company has warned that those who use AI to create long-form technical content should expect “significant inaccuracies, unusual conclusions, and inconsistent writing styles,” primarily due to the probability-driven nature of large language models (LLMs).

“These models generate results by predicting the next token, typically a word or subword, in a sequence, based on model weights and training data,” says Cisco, or as The Register says, “they are essentially an elegant autocomplete system that makes educated guesses.”

What works and what doesn’t

Since AI basically only predicts the next word, it creates four key problems, according to Cisco:

LLMs use different data for each new query, making consistency and standardization a challenge
Even if the same data is shared, the result will always be slightly different
Each new document will have a different structure and format, which presents another standardization challenge.
AI often discards valuable data, changing the outcome
This does not mean that AI is unusable for long technical reports, quite the opposite. It can still save businesses a lot of data, but the tool must be configured and optimized correctly.

Cisco says a good approach is to give the AI “granular, single-task instructions that focus on a small, specific portion of the report.”

The company also said that the AI should not be free to choose its sources for the report, but should instead be given specific documents. Finally, the AI should have clear instructions on format and style.

“A blind test of the sample report in our quality assurance process did not show any notable drop in the overall quality of the writing,” Cisco said.

“The reviewer, the professional editor, and the administrative reviewer all made glowing comments about the report without knowing that it was generated by AI. The reviewer commented that the incidence of typos and grammatical errors was much lower than in the average report.”

Cisco also discovered another challenge: When the AI is asked to edit multiple sample reports in a single session, the content of one report’s source material becomes contaminated with another, “even if the notes used to generate the first report were removed from the project’s reference documents.”

To address this issue, the researchers recommended starting a new session and re-entering prompts for each new incident report.