Based on my 13+ years of experience as a professional software developer; I would like to discuss about “Agile”; its virtues and its dangers, its myths and misconceptions, its pros and cons; and also some of the common pitfalls I have encounter during the years.
Agile, not Agile? What the Heck are we Talking About?
For the last 15 years we have heard software development companies, software development teams, IT departments, developers and the Industry in general talk about the agile way to software development, its benefits, virtues and how several businesses have had a productivity boost in their software development projects by doing “Agile”, and as a result, we have seen tons of businesses (not necessarily related to software development) jump into the agile way.
However not everything is milk and honey, recently we have heard many developers, managers, clients, and businesses complain about “Agile” (or some of its implementations as SCRUM) and how it have failed to deliver what it had promised.
But WHY?
Well, I believe this effect has to do with a lack of understanding in what “Agile” is, what it’s for and what it isn’t for… as with any tool, you should know when and how to use it.
So, based on my 13+ years of experience as a professional software developer; I would like to discuss about “Agile”; its virtues and its dangers, its myths and misconceptions, its pros and cons; and also some of the common pitfalls I have encounter during the years; also I will mention some of the methodologies, tools and techniques commonly used in the “Agile” world.
But first, we need to agree on what “Agile” is.
What is Agile? What does Agile mean?
In software development, it’s a term used to describe a set of principles for people; involved on software development projects; to follow.
The goal of “Agile” is to build better software by promoting collaboration and self organization in cross functional teams while embracing change, not fighting against it.
“Agile” was described on the “Manifesto for Agile Software Development” or “Agile Manifesto”created 15 years ago (February 2001) by 17 industry experts and it reads as follows:
“We are uncovering better ways of developing
software by doing it and helping others do it.
Through this work we have come to value:
Individuals and interactions over processes and tools
Working software over comprehensive documentation
Customer collaboration over contract negotiation
Responding to change over following a plan
That is, while there is value in the items on
the right, we value the items on the left more.”
There are 12 principles behind this manifesto, you can read them here (http://www.agilemanifesto.org/principles.html)
These principles are not a recipe and not a process; they will not tell you how to satisfy the customer, or how to communicate with your team, or how to manage your project, neither will they tell you how to embrace the change or how to do your work. They will not tell you to use X or Y technology, they are technology agnostic and in a higher level of abstraction (less specific).
“Agile” won’t impose you the HOWs, WHYs and WHENs; “Agile” will only give you the “WHATs”; and by only stating the what, “Agile” gains a valuable and important set of characteristics like: the ability to be loose and flexible, to be reusable, adaptable and extendable.
However It also makes it vague and incomplete, we cannot simply use it as if it was a methodology (it is not; a common and dangerous mistake), we must first tailor it to fit our needs, culture and our software development project context; And we must prepare ourselves and the team; we must pick our processes techniques and tools correctly if we are to succeed.
In a nutshell, “Agile” is not a methodology but a framework, and as such It shares the goods and evils common to them.
We have a Tight Deadline, We Need to Work Faster, Let’s Use “Agile”…
How many of you have heard the speed justification as a reason to use “Agile” in a software development project? My guess would be: Many
Is it wrong? Well, let’s say the most common misconception is that “Agile” means working fast, but If you carefully read the manifesto and its principles, you will notice the authors have never used words such as: fast, speed or velocity; clearly on projects with such justifications people have missed the real meaning.
I have heard a lot of devs complain about how working in “sprints” is not sustainable; and based on the thought that a “sprint” in sports (and in particular in running) is the act of going at top speed for a short distance; then no; it’s not sustainable but that is actually a “SCRUM” (the most popular “Agile” implementation) misconception, and probably more related to poor choosing of words than its actual meaning, but I will go to that in detail later on when discussing scrum.
I do agree, however, that increasing the pace, rhythm and tempo of the development process for a long period of time isn’t sustainable; as a matter of fact, “Agile” is against the going fast idea, and you can read it in the 7th principle:
”Agile processes promote sustainable development. The sponsors, developers, and users should be able to maintain a constant pace indefinitely.”
So there you have it, “Agile” does not promote working fast or working at an increased pace. “Agile” promotes that each team find their own pace, their own regular rhythm to work by without ever getting into a “sprint” mode that will burn out the team and prove unsustainable.
If we are to compare “Agile” with running sports then “Agile” is more like guidelines to a marathon than guidelines to speed racing. “Agile” principles do not promote working fast; “Agile” principles promote working steady.
But the Client’s to Blame, he Continuously Changes What he Wants at the Last Minute…
Ok, let’s read the 2nd principle:
“Welcome changing requirements, even late in development. Agile processes harness change for the customer’s competitive advantage.”
“Agile” welcomes change, because being able to adapt to changes is a crucial asset in any business (Just ask Kodak) so by going “Agile” you must focus on this, rather than getting more done in less time (in some cases that could be a side effect of doing agile, but that is not an inherent principle of it).
It should be clear now; “Agile” does not mean working fast; it means you should be able to adapt… and for that to happen, your team, your development processes and the software being developed should be adaptable.
We Need to do some Refactor of the Code to Reduce our Technical Debt as Part of our Next Delivery… No, that Does Not add Value to our Client!
Another misconception (and one that is particularly harmful) is that “Agile” strips the engineers, devs and the technical people on the team from its powers and handles the wheel to business people; meaning that technical requirements are not as important as business requirements.
Again, this is not the case; in fact the 9th principle reads:
“Continuous attention to technical excellence and good design enhances agility.”
This principle accepts the fact that to be better at being “Agile”; in order to be ADAPTABLE, the technical needs must be on the spotlight too, must be attended and resolved; and not just the business needs.
Senior developers and software architects know that in order for software to be adaptable, easy to change and easy to extend and maintain; such software needs to be designed that way, its implementation needs to be carefully thought, it will require a good separation of concerns, it will need loosed coupling between the different pieces that conform it, or you will end with a rigid software which is hard to change, extend and maintain (not necessarily bad software just bad at being adaptable).
And if the team is doing “Agile”; the software it produces should be easy to update, extend and maintain, it should be ready for changes, it should be adaptable…it should be “Agile” too.
And as we already said, engineers, devs and architects should design it and develop it that way, in order to do so, the technical needs and requirements have to be addressed too. They can’t be tossed away or ranked low in the list of priorities just because someone claims “it does not add any visible value to the client”; of course it’s adding value, it is adding the value of letting you do “Agile” development. This is a crucial mistake some organizations make and the main reason “Agile” won’t work for them.
I want to make clear that, business and technical requirements are equally important, and both need the same level of attention, both need to be addressed; It is up to the team to decide what is to be addressed on each delivery; they are the experts, they know what problem they are trying to solve or the goal they want to achieve.
There are other common misconceptions people has about “Agile”, however the ones I have mentioned are those I believe are the most common and harmful of all.
Then You are Traditional, not Agile…What is “Not Agile”?
Well, to answer this question I would like to show the differences of the 2 approaches, by bringing in the ideas that Martin Fowler (one of those 17 experts that wrote the manifesto back in 2001) exposed on many of his papers and publications; in fact there is a video I highly recommend you to watch where he and Neil Ford explained Agile (https://www.youtube.com/watch?v=GE6lbPLEAzc).
In this video Fowler explained the difference between “Agile”and not agile or the traditional approach in a really clear and simple way,I liked it a lot:
He said that for the traditional approach first the plan is made, then the planned work gets done, as a common management saying states:
“Plan your work, then work your plan”
Thus, becoming “a plan-driven approach”.
Fowler also refers to this approach as a predictive approach, as the team predicts first (by doing the plan) the outcome of their software development project, then the team works according such plan while monitoring and controlling the work so that it meets the plan at all moments, when a deviation happens (unexpected stuff comes) there are some mechanisms in place that will keep the plan and the project in sync. Keeping the plan in sync can be very time consuming and depending on the size or complexity of the project it can become a full time job, this is why some organizations use a dedicated role for this job (Project Manager) that will also be responsible for most administrative related task within the project like resource management, risk management, change management, etc.
In this approach a good plan is crucial, as you might have imagined. But, what is needed to create a good plan then? Well, there are many factors and variables at play for that and I’m not going to dig into them here; however there is one that I do want to mention: in order to have a good plan you need to have an initial set of STABLE requirements.
Note that being stable does not mean unchangeable; stable means that the change won’t come often; so stable requirements are those that are not set in stone, but rather, are requirements that have been developed enough that they will rarely change during the development of the software.
To do a plan-driven approach it is imperative that you have a stable set of requirements, here is where the “Agile” approach not only is much more tolerant to changes in requirements, it goes further and sees the adaptability as an asset, as an advantage and to do it properly all the underlying disciplines, the team, and the software produced must be able to adapt.
To be able to adapt, “Agile” adopts some tools and techniques like: adaptive planning and evolutionary software design, as a result the software produced is designed for change; but that is not enough. If the software being developed is changing often, the integration of the software will need to be done more oftenly so that the changes are deployed constantly as part of the regular releases; and the software would need to be tested as often as well (at least the regression testing of all the impacted pieces). Continuous Integration and Automated Testing are useful techniques to achieve this.
Finally, another key difference that Fowler clearly points out is how in the plan-driven approach the process comes first, then the people.
Traditional software development organizations first choose their software development processes (and roles), then they’ll find the people that will fit in each role to assemble the software development team.
In contrast “Agile” inverts this idea by choosing the right people first (assembling a team that is actually functional as a team, a multidisciplinary team able to work and collaborate together) then the team is in charge of choosing the development process they’ll use; aiming at choosing the best process for them.
All this will produce a shift in the entire organization and in its way of doing software; so to use “Agile” you will require a commitment and willingness from the people and organization to change their processes, tools and techniques in favor for more adaptable ones.
Those are the key differences between “Agile” and “Plan-Driven” a.k.a. “Traditional way”
In Summary…
So, for the remainder of this blog series we can agree that “Agile” is a framework for software development that rely on people the development of better software (working software) by means of their expertise (multidisciplinary) and teamwork (good collaboration and communication), through iterative development with a focus on adaptability.
In general you should understand that for software development organizations or teams being “Agile” or “Not Agile” is not a black and white, on or off characteristic; it is a scale where you can be more or less agile than others:
How much you lean to one or the other side of this scale is dependant on how good you and your organization and teams are at following the agile manifesto and its principles, and what tools and techniques are adopted.
On the next part I will describe common “Agile” implementations (extensions of the agile framework), techniques and tools suited for the “Agile” approach.
