Managing Complexity in an Agile Project

Most of you have probably been through some form of Agile training, and probably have a basic understanding of the Scrum framework. But when you returned to your project team, perhaps you discovered that some of the ideas and techniques you just learned in class are not so easily adopted on your team.  The reality of our project is so much more complex than the theory we learn in the classroom.  So what do you do?

Well, remember that Agile promotes an iterative and incremental approach to software development; that is, taking a complex problem and tackling it one chunk at a time, rather than spending lots of time developing master plans and designs upfront that are out of date as soon as we finish them.

But how do you chunk out a complex project?

Well maybe it would be helpful to understand the different aspects of complexity.  We can view the complexity challenges we face in terms of:

1.       Process Complexity
2.       Product Complexity
3.      Organizational complexity
4.       Team Complexity
5.       Architectural and design complexity

Each type of complexity is difficult to address, but the Architectural and design complexity is probably the most challenging.  So let’s focus on this one during for now.

You can talk to just about any developer and they will tell you that the architecture and design of software and systems are increasingly complicated. Every year we get new requirements to provide more even functionality to already complex systems. And add to this the need to interconnect software and systems to other systems, and the design complexity increases exponentially.

So how do you address architectural and design complexity in an Agile work environment?

If we were working on a waterfall project, the approach would be to spend weeks or months putting together a detailed architectural model and detailed design.  But the problem with this approach is it locks down a design at the beginning of a project and limits the ability of the developers and testers to make improvements to the design as they build out the product.

Now in an Agile project, an architectural model would still be valuable, but it doesn’t need to be initially at the same level of detail as would be expected on a waterfall project.  Instead the model should be an Agile model, that is, one that is continually enhanced over a series of iterations, as new information is learned from the development team and as the business partners gain a clearer understanding of what they really want.

On an Agile project, models should be viewed more as useful tools of discovery.  In fact, the process of creating a model is often more useful than the model itself.  Don’t try to get everything right the first time.  As one person put it, Agile models are more like paintings, than photographs.

Frankly, it may be unrealistic to expect an architect or lead designer to create a perfect and detailed design up-front for systems as complex and interconnected as those we see today.  But by using an iterative and incremental approach and creating an initial architectural roadmap, a better design, and therefore a better product will evolve throughout the project.

So on your next Agile project that involves complex systems and integration, consider building an Agile model and Agile design that evolves over time, that takes in consideration what is learned by the development team who is actually building the product.  Let the design evolve as the product evolves through a series of iterations.