A year of documentation-driven development

For many software teams, documentation is written after features are built and design decisions have already been made. When that happens, questions about how a feature is understood or used often don’t surface until much later. 

A little over one year ago, our team began to recognize this pattern in our own work. Features generally functioned as intended but were difficult to use or explain. Documentation lagged behind releases. Important design decisions lived mostly in conversation or code, rather than in shared artifacts that could be reviewed, challenged, or built upon by others.

Documentation-driven development is the framework that grew out of these observations. In this article, I’ll describe what led us to adopt it, how it changed our day-to-day engineering work, and what we observed after a year of using it in practice.

The reflections in this article are drawn from my experience working on Landscape, a systems administration product with a long history at Canonical. Our team includes backend engineers, web engineers, documentation, product management, and UX design. Like many teams responsible for a mature product, we balance the ongoing maintenance and introduction of new features, often while collaborating with other engineering teams.

What documentation-driven development means in practice

Documentation-driven development is an approach to software engineering that treats documentation as an integral part of the development process itself, rather than something that happens afterward. 

Documentation here doesn’t just mean user documentation. It includes engineering design documentation, user-facing documentation, and any internal documentation used to collaborate within the team or with other teams. In practice, this process gave our feature work a clearer sequence to follow.

This is how documentation-driven development looks on our team:

New features begin with a design document, which clearly and concisely describes the proposed design of the feature, including the existing state of the system, the problem being addressed, why solving it matters, and the expected design to be implemented. Writing that design documentation forced us to make our decisions explicit and early on, and opened the door to more collaborative discussion around the feature design.

Once the design is agreed upon, drafting the user documentation begins before any code is written. This stage forces engineers to closely engage with the user experience. If a feature is difficult to explain cleanly, that reflects something unresolved in the design itself.

As development progresses, code and documentation continue to evolve together. The user documentation should reflect the current state of the feature, and the act of writing the documentation continues to inform the implementation decisions. 

By the time the feature is complete, the documentation is complete as well, not because it was rushed at the end, but because it helped shape the work along the way.

Documentation-driven development isn’t an effort to improve user documentation in isolation. It’s a way of viewing the user experience as a core concern of the product and engineering practice.

Before we adopted it: What wasn’t working

Before we adopted documentation-driven development, feature development followed a general pattern that may feel familiar to many engineers: a need would surface, some members of the team would talk through an approach, informal notes or partially-completed design documentation might get recorded, and implementation would begin.

It’s not that feature design work didn’t happen, but much of it was worked out in conversation or reflected in the code. User documentation, when it existed, was written near the end of development and often wasn’t ready by the time the feature was complete.

There wasn’t a consistent point where we tried to articulate the user experience as a whole. Early discussions centered around the software behavior and feasibility; often, these decisions came as we were problem solving, rather than defining beforehand how we would solve those problems. Questions about how a feature would be discovered, understood, or used by our end-users often surfaced quite late, when revisiting the feature design was difficult.

Over time the negative effects of the previous system became clear: missing documentation, confusing edge cases, and feedback or ideas that arrived after the feature had already assumed a fixed shape.

The major shift: Documentation as part of engineering

One of the first effects I observed after our team adopted documentation-driven development was that it changed how engineers interacted with their own assumptions about new features. Writing documentation earlier in the process made gaps in design and implementation more visible. While it’s possible to implement a feature without fully resolving edge cases or ambiguities, it’s much harder to explain a feature without noticing the areas that are unclear.

This shift was especially apparent during the feature design stage. Design documents became stable points of reference that informed every stage of development. Formalizing the role of design documentation transformed our approach to development and the character of our discussions. Conversations were less likely to only revolve around the internal structure of the feature, and they were more likely to include questions about how the feature would be experienced by our users. Because design documentation made assumptions and tradeoffs explicit, it also provided a stronger foundation for building user documentation later.

Example: Avoiding a costly mistake

At one point during the year, we were designing a feature that was closely related to the work done by another team and to the future integration with their system. Our original approach involved Landscape taking on more ownership over managing user identities, which is an area that’s relatively new for us as a product.

We wrote a detailed design document and created the accompanying UX mockups, carefully considering the implementation details, user experience, and longer-term plans for Landscape. Since this work intersected closely with the work done by an internal identity management team, we shared this document with them for review. 

The identity management team’s input had two important effects. First, they brought domain knowledge about identity management. This helped us see that the approach we were proposing would be complicated to maintain over time, and would be a burden on our team’s capacity. Second, they pointed out that our design could complicate future integration with their system because it was misaligned with their product’s plans. For example, our proposed database schema wasn’t compatible with their data model, and making them compatible would be too large of an effort at this stage.

As a result of this feedback, we ended up reevaluating the feature itself, and later chose not to implement that design as proposed. We substantially reduced the scope of the feature, and avoided spending time on unnecessary work that was complex and would’ve needed rework in the future. 

Without documentation-driven development, we likely wouldn’t have explored the design in this level of depth or produced a document substantial enough to support this kind of review. The feature we chose not to build was a direct result of treating documentation as a fundamental part of our engineering process.

What changed over time

As documentation-driven development became part of our normal workflow, several changes followed.

User documentation as part of the development process

User documentation became an integral part of feature development, rather than something produced at the end. Because it was written and revised alongside the code, engineers were more engaged with the documentation and more invested in how features would be understood by our users. Over time, this engagement led to a noticeable improvement in quality. The documentation became clearer, more complete, and better aligned with how the product actually behaved.

That engagement also had some secondary effects. Engineers also became more familiar with our documentation as a whole, not because they were instructed to be, but because they contributed to it and understood it. They were more likely to notice gaps, question existing documentation, and volunteer to make improvements.

Clearer artifacts and better collaboration

Collaboration across different roles on our team improved as well, largely because we were producing higher-quality documentation earlier in the process. Having clearer, more complete designs and user experiences written down made it easier to share work for review without requiring deep prior context about Landscape or its codebase. Discussions between backend engineering, web engineering, documentation, product management, and UX design were grounded in a shared understanding of the feature, rather than summaries or recollection. It became easier to align on what a feature was meant to do, why it was meant to do that, and what the user experience would be.

A shift in team culture

And one of the most significant changes I observed was in our team culture. Documentation, design, and the user experience became easier to talk about because they became part of our everyday engineering work rather than peripheral concerns. The perspective across our team changed: improving the user-facing components of our product became a shared practice, not just a shared value.

Limitations

Although we experienced many positive outcomes from adopting documentation-driven development, it didn’t address every problem. The process is designed for new feature development, so it didn’t directly resolve existing issues in our documentation or its ongoing maintenance challenges.

First of all, adoption was gradual and uneven. Our team was quick to adopt the design documentation stage, but drafting user-facing documentation early in the process required a larger shift in how we were used to working. It took time for that part of the practice to feel natural and useful, and it didn’t take hold at the same time for each member of the team.

We also learned that the process itself needed some flexibility. Not every feature warranted the same depth of documentation at each stage: some features benefitted more from deeper design work, while others were more complex in the user documentation and implementation stages. We iterated on our process as we went, adjusting how we applied it to different kinds of challenges rather than following it rigidly.

But along the way, something quieter happened. Adapting the new process to fit our needs as we went gave the team shared experience in tackling unfamiliar problems and learning to work through uncertainty together.

Looking ahead

Documentation-driven development wasn’t the solution to all our problems, but it did give us a foundation to build from. By making our user experience and feature design part of our everyday engineering work, it created conditions for other practices to grow, and as a result, the quality of our work is now much higher. Designs are more deliberate and better thought-out before implementation begins, and new user documentation is more clearly written for its audience, more closely aligned with the behavior of the product, and ready when new features are complete. We’ve also been able to build on this foundation by introducing practices like user-story mapping in our engineering work, drawing on the same attention to processes and user journeys that documentation-driven development helped establish.

One year in, documentation-driven development has helped us build a shared process and language for thinking about our product, code, documentation, and the user experience together. Our team practices and product continue to evolve, but now with more clarity, a stronger connection with how users engage with what we build, and fewer assumptions left unexamined.