Agile Development - Lean. Eliminate Waste

I am back from a nice vacation in Brazil and ready to write a little more about Agile development. Last article I gave a 30,000 feet view about Lean, today I am going to get one of the principles (Eliminate Waste) and get a little deeper.

Waste

What is waste in software development? I described that on my last post as "everything not adding value to the customer is considered waste. In other words: In software development, waste is anything that does not improve the quality of code, reduces the amount of time and effort it takes to produce code, or does not deliver business value to the customer. That is too generic.

On Poppendiek's book she describes complexity as a great candidate on promoting waste. "Complexity calcifies our code and causes it to turn brittle and break. The prescription for complexity in software development is simple: Write Less Code !!

Complexity

Just about any company that starts with a single technology is agile by definition. It can deliver and respond to the customer with easy. As the company grows and achieve success it starts to become slow, unresponsive. They have developed a complex code base, array of products and technologies. Unless they get complexity under control they will strangle what made them successful. The responsiveness, the agility.

The problem with complexity is that is does not grow linearly it does in a exponential fashion. Soon it starts to dominate all other costs in the software development process. Wise software development organizations place top priority on keeping the software clean, small and simple.

So, how to deal with complexity? Here are some food for thought

Justify Every Feature

The very first step on controlling complexity is limiting the features and functions that make to the software base. It is like a diet... you will be able to control your weight if you control what goes in your body. So, every feature that gets developed should pass to a rigorous evaluation and prove that it will create more economic value than the cost throughout the lyfecycle. Loading a software with a laundry list of feature is the lazy way to do marketing. Worse than that it is a recipe for disaster since the complexity will increase with useless features (not so valuable).

It takes guts to limit feature set but it mostly pays off. A company that delivers a product with just the right amount of features shows that it understands its customer.

Minimum Useful Feature Set

On agile the ideal approach do deliver software is dividing the product in small feature sets that immediately deliver value to the customer; and release these feature sets in descending order or importance. That way the customer can quickly start using the software. So, a minimum feature set is the one that helps the customer do a portion of its job faster.

From the sustainability point of view software developed on a incremental way is easier to maintain because incremental development can be sustained for the life time of the software. Of course other disciplines surrounds the incremental process and promotes the easy to change characteristics of the software. Refactoring supported by unit testing is one of them.

Don't Automate Complexity

Sometimes software companies solve complex problems by wrapping the existing process in a complex web of software. Never automate complexity. Understand the problem and simplify the process then, if necessary automate. In other words you domain the technology, you customer has the domain knowledge of the business. It is your job to simplify and automate (solve) that problem the simplest way possible.

What is Waste in Software development then?

Poppendieck define 7 wastes

Waste	Description
Partially Done Work	The inventory of software is partially done work. The objective is to move from the start of work on a system to integrated, tested, documented, deployable code in a single, rapid flow. The only way to accomplish this is to divide work in small batches, or interactions. Example of partial work is: uncoded documentation, unsynchronized code, untested code, undocumented code, undeployed code.
Extra Features	Waste in software is adding features that are not needed to get customer's current job done. If there isn't a clear and present economic need for the feature, it should not be developed.
Relearning	Rediscovering something we once knew and have forgotten is perhaps the best definition of "rework" in development. Knowledge needs to be captured and needs to be wide available to the development team. Another way to waste knowledge is to ignore the knowledge people bring to the workplace by failing to engage them in the development process. This is even more serious than loosing track of the knowledge generated. It is critical to leverage the knowledge of all workers by drawing on the experience that they have built up over time.
Handoffs	Passing knowledge in a strange science. You can document your architecture and give them to an engineer and most likely it will not be of much help. On the other hand if you sit with this person and be with him on the transfer, give him some pointers on some aspects of the software, testing, the history behind what was done, most likely, sooner your engineer will be able to ride by himself. This kind of knowledge is called tacit knowledge and it is very difficult to hand off using documentation. When a work is handed off a vast amount of tacit knowledge is left behind in the mind of the originator. If each handoff leaves 50% of the knowledge behind, 3 % of the whole knowledge is left after 5 handoffs. So, to minimize lost: Reduce the number of handoffs, Use design build teams (complete, cross-functional teams) so that people can teach each other how to ride the code. Use high bandwidth communication Release partial or preliminary work for consideration and feedback
Task Switching	Concentrate on one tasks at a time and finish it before switching to another task. Switching to a different task is not only distracting , it takes time and often detracts from the result of both tasks. Task switching time is waste.
Delays	Waiting for someone in order to have your work done is a big waste. That is why a process like scrum puts so much effort on maintaining the engineering unblocked. Again, communication plays a very important role on minimizing delays. Have your team collocated or collaboration tools that promotes open communication. Have your daily meeting and try to identify and correct any roadblock.
Defects	Every code base should include a set of mistake-proofing tests that do not let defects into the code, both at the unit testing level and acceptance level. Somehow software still finds devious ways to fail, so testing experts should start testing early and often to find as many of these unexpected errors as possible. Whenever a defect is found, a test should be created so that it can never happen again. A good agile team has an extremely low defect rate, because the primary focus is on mistake-proofing the code and making defects unusual. The secondary focus is on finding defects as early as possible and looking for ways to keep that kind of defect from recurring.

Conclusion

We touched several subjects related to waste in software development and the text is self explanatory. I personally put a lot of importance of the defect aspect of building software. If we could "easily" convince engineer and QA that testing first enhances development speed, mistake-proof the code and helps on refactoring our lives maintaining and enhancing the software would be much easier.

In addition applying acceptance and unit testing has a deeper meaning than just mistaking-proof the code. Acceptance tests are best when they constitute the design of the product and match the that design to the structure of the domain. Unit tests are best considered the design of the code; writing unit tests before writing code leads to simpler, more understandable and more testable code. These tests tell us exactly and in detail how we expect the code and the ultimate product to work. As such, they also constitute the best documentation of the system, documentation that is always current because the tests must always pass.

Reference

Implementing Lean Software Development - From concept to cash. Mary and Tom Poppendieck

Agile Development - Lean

Last article I started mumbling about Agile and why companies were adopting Agile methods. The focus was on why projects were failing using traditional software development methods.

Until recently I was mostly focused on Scrum and extreme programming but I always heard good things about the Lean approach. So I went to the net and got a book that I recommend Lean software development by Mary and Tom Poppendieck(http://www.amazon.com/Implementing-Lean-Software-Development-Addison-Wesley/dp/0321437381/ref=sr_1_2?ie=UTF8&s=books&qid=1245442337&sr=8-2). This article is an attempt to summarize the ideas as I understood and give you guys started on the subject.

Lean Software Development

Let’s start with a more formal definition. Lean software development is a translation of lean manufacturing principles and practices to the software development domain. Adapted from the Toyota Production System, a pro-lean subculture is emerging from within the Agile community.

Lean Software Development is not a management or development methodology per se, but it offers principles that are applicable in any environment to improve software development.

So in that way the tenants or principles of this methods can be mixed with the process defined by Scrum for example. More and more I feel that there is not a right method but smart leaders or managers should be able to use the best of each approach adapting to the reality of his/her group. Here are the Lean principles:

1. Eliminate Waste

Basically everything not adding value to the customer is considered waste. In other words: “In software development, waste is anything that does not improve the quality of code, reduces the amount of time and effort it takes to produce code, or does not deliver business value to the customer.

2. Amplify Learning

Constant learning will minimize the numbers of problems/bugs. The best approach for improving a software development environment is to amplify learning. The creation of bugs should be prevented by running tests as soon as the code is written (TDD). Instead of adding more documentation or detailed planning, different ideas could be tried by writing code (Spikes).

For programmers to develop a system that delivers business value, they will have to learn about many things. Some are technical some are requirements but overall the leaning will benefit both.

3. Decide as late as possible

The development process is always associated with some uncertainty. By delaying decisions to the “last responsible moment” you are toying with the possibility of avoid change. In a nutshell the principal states that you should delay decisions as much as possible until they can be made based on facts and not on uncertain assumptions and predictions.

4. Deliver as fast as possible

Deliver as fast as possible has a lot of relationship with the iterative SCRUM model. We are not talking about the whole product as fast as possible but verticals that deliver value to the customer.

If we can deliver quickly we can receive feedback quicker and therefore fine tune our direction with mode certainty. Constant cross correction is important on a agile method.

5. Empower the team

The quality of a software team (the people factor) is the most important element in successfully delivering software. In order to get people to take responsibility, get motivated, and gel as a team, they need to be responsible for the outcome and authorized to make it happen.

The pyramid leadership model is long gone. You can not do everything by yourself. You have to empower your team; give them responsibilities and of course make them accountable.

6. Build integrity in

The ever demanding Customer needs to have an overall experience of the System – this is the so called perceived integrity: how it is being advertised, delivered, deployed, accessed, how intuitive its use is, price and how well it solves problems.

Conceptual integrity means that the system’s separate components work well together as a whole with balance between flexibility, maintainability, efficiency, and responsiveness. This could be achieved by understanding the problem domain and solving it at the same time, not sequentially.

7. See the whole

Software systems nowadays are not simply the sum of their parts, but also the product of their interactions. Defects in software tend to accumulate during the development process – by decomposing the big tasks into smaller tasks, and by standardizing different stages of development, the root causes of defects should be found and eliminated.

Conclusion

I will continue talking about Agile on the next article (probably some basics in scrum or tools used in Lean) but the lesson I take from this is to understand the principles and slowly bring to the methodology being used in your team.

If you can’t convince your management that Agile is the way to go at least you can use some of the concepts (TDD, pair programming, sprints, etc.). I guarantee that picking at least one will bring benefits to the overall success you the team. Next time try Test Driven Development for a while and you will see the code quality improve.

Enough for today… until next time

Reference

Lean Software Development Overview

Lean software development (Wikipedia)