How to improve technical documentation with generative AI

Devops teams have a love-hate relationship with writing and consuming technical documentation. Developers loathe reading and maintaining undocumented code. Architecture diagrams can tell a great story, but much of it is fiction compared to the implemented architecture. Even IT service management (ITSM) process flows for incident, request, and change management are rarely followed as specified in the documentation.

CIOs, CTOs, and other digital trailblazers insist on documentation. However, project budgets rarely include technical writers, and agile teams rarely have time to do more than code-level documentation, README files, and other basics.

While product owners capture requirements in agile user stories, the documentation guiding an application’s business rules, journey maps, architecture, APIs, and standard operating procedures is rarely complete and up-to-date.

I’ve previously written about using generative AI to write requirements and agile user stories. Now, I am asking the follow-up question: How can developers, engineers, and architects use genAI tools to write and maintain accurate documentation?

Using genAI for devops and ITSM documentation

Several years ago, I worked with a development team that said documentation was worthless. They believed good code that followed naming conventions and had robust unit testing, with high code coverage and strong error handling, was all the documentation they needed. Even if they dedicated more time to documenting existing features (at the expense of developing new ones), the materials would become obsolete with the next deployment’s changes, they said.

But proponents say using generative AI tools can help devops teams maintain documentation at the pace of code changes and deployments. “Generative AI is shifting the role of software documentation from static reference material to a dynamic layer within the product experience,” says Erik Troan, CTO of Pendo. By capturing user flows and generating contextual guidance automatically, we’re seeing how documentation can now evolve in real time alongside the software itself to reduce friction and improve user efficiency.”

Dominick Profico, CTO at Bridgenext, believes AI-generated knowledge will eventually replace documentation altogether. “GenAI will allow us to reach the point where documentation that leaders have been craving and developers have been avoiding for decades will become obsolete. LLMs will reach a level where documentation is truly dynamic, generated on the fly in response to a question, a chat, or a prompt, and extracted from the model’s knowledge of the codebase, industry standards, documentation, and even support tickets and system logs.”

Regardless of what the future brings, genAI is already making it possible for devops teams to address documentation requirements more efficiently. Additionally, there are new reasons to invest in documentation, as devops teams utilize AI agents and other genAI development tools.

Define the target audience for your documentation

Before investing in documentation, it’s a best practice to define the audience and how they will use the materials. Let this be the baseline standard for defining “good enough” and “up-to-date” documentation. Here are the major audiences and needs devops teams should consider:

Newly onboarded developers seek documentation that covers the architecture, devops non-negotiable requirements, the software development process, and high-level code structure to become productive quickly and produce acceptable solutions according to standards.

External development teams want to review an API’s documentation, README files in Git repositories, data definitions in data catalogs, and guides on log files and other observability artifacts.

Architects, security specialists, and site reliability engineers (SREs) require documentation when recommending app modernizations and programs to address technical debt. They also need documentation to aid in incident response and perform root-cause analysis.

Data scientists, data governance specialists, and engineers working on data pipelines are often consuming data created by APIs and applications to be used in reports, data visualizations, analytics, and AI models. They’ll want to see the updated data catalog and to understand data lineage, which they’ll use for data-driven decision making.

Product managers, product owners, change leaders, and other subject matter experts (SMEs) want to know “how the system works.” While they want to avoid diving into code, they need more detail than what’s typically provided in release notes.

Auditors for ISO 27001, ISO 9001, SSDF, CMMI, SOC 2, and other compliance standards will want to review the required documentation.

GenAI coding assistants and AI agents will consume documentation to improve their relevance and accuracy.

How to use genAI tools for technical documentation

We know different audiences have different documentation needs. How can we use generative AI tools to meet these needs in targeted ways?

Document how features work

“The docs for Google Cloud’s API’s are written in code, and they proved that the only way to keep literally tens of thousands of API docs up to date was to reduce that effort to automation,” says Miles Ward, CTO at SADA. “We dumped our technical documentation into NotebookLM, now I can get a podcast explaining the nuance of feature interactions to me in plain English. The state of the art is changing rapidly, and new tools like Gemini, NotebookLM, and Mariner can help customers get their documentation to be an asset, rather than a chore.”

To document how features work, consider writing and maintaining the following:

A feature specification documenting requirements, including end-user documentation.

A short technical design that includes architecture, dependencies, testing, security, configuration, and deployment sections.

References, including links to agile user stories and IT service management tickets.

Some tools for functional-level documentation include Microsoft Teams, Atlassian Confluence, Google Workspace, Notion, and MediaWiki.

Document APIs, data dictionaries, and data pipelines

“One of the most exciting shifts we’re seeing with genAI in the CTO office is how it transforms documentation from an afterthought into a natural byproduct of the development process itself,” says Armando Franco, director of technology modernizations at TEKsystems. “For example, as teams build microservices, genAI can automatically produce and maintain OpenAPI specifications that accurately reflect endpoints, payloads, and authentication methods. For data teams, AI can generate lineage diagrams and data catalogs directly from SQL code and ETL pipelines, ensuring consistency across environments.”

Devops teams should remember that they are not the target audience of technical documentation. Developers who join the program or take over where the original development team leaves off are the primary audience, along with any external developers who utilize APIs or other externalized capabilities.

Different kinds of technical documentation leverage different tools:

The best place to document data dictionaries is in data catalogs such as Alation, Atlan, Ataccama, AWS Glue Data Catalog, Azure Data Catalog, Collibra, Data.world, Erwin Data Catalog, Google Dataplex Universal Catalog, Informatica Enterprise Data Catalog, and Secoda.

Dataops teams using data pipelines, data integration platforms, or other integration platforms can use visual design tools to provide data flow and lineage diagrams.

Tools for documenting APIs include Postman, Redocly, Swagger, and Stoplight.

Document the runtime and standard operating procedures

“Traditional documentation practices haven’t kept pace with the dynamic, real-time nature of today’s AI-driven cloud systems,” says Kevin Cochrane, CMO at Vultr. “CTOs are now using genAI tools to turn logs, configs, and runtime data into living documentation that evolves with the system, helping teams reduce friction and accelerate development. This approach turns documentation into a continuity tool: preserving shared context, reducing single points of failure, and preventing execution breakdowns across the stack.”

Devops best practices focus on workflow, tools, and configuration, leaving it to teams to decide how to document handoffs from development to operational functions. The following types of tools can help address the gaps:

Tools for creating operational knowledge bases and standard operating procedures: Atlassian Jira Service Manager, Freshservice Knowledge Base, ServiceNow Knowledge Management, and Zendesk Guide.

Log-file analysis tools with AI: Datadog, Dynatrace, LogicMonitor, Logz.io, New Relic, Splunk, and Sumo Logic.

Tools for visualizing public cloud infrastructure: Cloudcraft, Hava, and Lucidscale.

Tools to diagram the architecture, sequences, and other flows: Draw.io, Figma, Eraser, Lucidchart, Miro, and Visio.

Provide AI agents with documentation they can use

While many code-generating AI agents analyze the codebase, a growing number of them can also analyze software documentation for added context.

“When every code change is documented, AI agents can understand not just what the code does, but why it was written that way, and this historical context transforms AI from a coding assistant to a knowledgeable team member,” says Andrew Filev, CEO and founder of Zencoder. “This institutional knowledge, previously locked in developers’ heads or scattered across Slack threads, becomes searchable, actionable intelligence that improves every subsequent AI interaction.”

Devops teams should consider feeding AI code-generators documentation on APIs, user stories with acceptance criteria, coding standards, architecture principles, README files, secure coding guidelines, data privacy rules, and compliance references.

Filev adds, “LLMs work three times better with detailed documentation because they can understand context, constraints, and intentions. Teams using this approach report that after six months, their AI agents become dramatically more effective at understanding their specific codebase patterns and conventions.”

Document legacy applications

One additional use case is to address undocumented applications, especially when the original developers are no longer with the organization. Sanjay Gidwani, COO of Copado, shared three key AI capabilities that make documenting existing systems easier:

GenAI is great at summarizing vast amounts of material, so it can easily read existing source code and summarize the intent.

Many business application systems rely on configuration metadata, and AI with metadata awareness can read configurations and document them.

AI can analyze your data to determine the actual processes used, complete with the length of time in various stages and the identity of the participants.

While undocumented systems are problematic and may be a compliance issue, creating overly verbose documentation is also challenging. Long-form documentation is hard for humans to consume and expensive to maintain, even when assisted by generative AI. The best approach is to keep your audience in mind and maintain just enough documentation. All documentation should be targeted for the people who will review it and the LLMs that will use it to answer their questions.