This is a post put together from not my successes, though I've had some. This is rather born out of my failures. The times where I've been through product life cycles and not achieved the quality standards that I set for myself and my team.
TDD is great. I've seen it done successfully. Heck, I've even done it successfully. The think I like about it most is that when I am done, I feel good about my code and myself. I'm coming off a stint where my team ditched TDD and other quality practices in favor of initial delivery speed. Before anyone starts... I know. The overall speed of the project was slowed by fixation on hitting internal delivery dates. The internal dates were met, but the quality of the delivery caused a test-fix tail that we are still processing. Its a march I've treaded before, and its what led me to this post...
Over the years I've noticed a few cultural aspects of the companies I've worked for that puts real barriers in from of the adoption of quality practices, especially TDD. It seems counter intuitive to the end goal that we all have of delivering quality software, but it still seems to happen.
Fixed Date and Feature Set
Its very tempting to get something big to market quickly. To overcommit on something in order to impress. The pressures can come from different sources. Below are just a few.
- An exec trying to make a name for himself or the group he is responsible for
- Project leadership in fear of losing out to other companies, or groups within the same company for the work.
- A technical lead trying to prove his worth
- A business-side person or a salesman fearing they will lose clients unless something gets delivered to match or exceeds the competition
Let me preface that I think that none of these people are inherently evil. These people face very real pressures, and are also judged by the success of the software you put out. I've seen these pressures time and again have an impact on how a project is developed.
I frequently get "We need to act like a startup and get this out to market quickly!". Well, you can guess what happens to quality practices. Let me be clear, I am 100 percent in favor of releasing production code quickly, but there is a way to do this, and it is not without tradeoffs.
There are 3 aspects to software development that can be tuned: Function, Date, and Quality. The general belief is that you can tune 2 out of the three while the third would be a function of the other 2. So when a product owner says they want this specific feature set on this specific date, that delivery is going to have quality that is a function of those 2. When those 2 are very aggressive, you know that quality will be low. Shortcuts have to happen to get there. Unfortunately, one of the first things that gets jettisoned are upfront quality practices.
Fixation on Initial Delivery Speed
So when you have an aggressive schedule, and when you have separate groups that do test, performance, internationalization testing, etc. The brunt of the pressure comes upon the initial development of the code to meet aggressive internal dates so the downstream groups can be successful. If the initial delivery is sub-standard in quality, then there are multiple test/fix cycles that that need to happen in order to prepare the project for delivery.
In reality, this becomes a disastrous form of thinking. The initial build of the project is the most important part to ensuring long term product quality and maintainability. Just ask any developer that has to work on a legacy codebase!
When a team rushes the initial development of software, the following side affects happen:
- The code is not well thought through and not cleanly crafted, making future changes to it much more time consuming and risky.
- Issues in the code are introduced and not found until downstream test cycles, extending the "tail" of the release (i.e. longer test cycles)
- Because of the longer test cycles, the overall development time of the project extends, because of the added burden on the test teams, or worst the project goes out the door with sub-standard quality.
Misconception that Developers Need to be "Pushed"
I've run into this several times, so I feel like this must be common. Managers and leaders of projects feel that the development team will not work at full capacity unless they are "pushed" with over-aggressive schedules.
I really don't know where this comes from. I think people take a bit too much stock in Parkinson's Law, where if someone is giving more time than needed then they won't finish early, they will take up the time allotted. The abuse of this adage is the the thought that teams must overcommit to avoid this. Most of the software developers I've worked with are extremely professional, and take pride in their work, and are most of the time dare I say, over-aggressive. After 15 years of software development, I've never seen a gross underestimation of a total project by the programmers involved. I've seen good, solid estimates, and I've seen misses, but not much "hey we've done it all early". I've seen it at a small scale but never a large.
I know I don't speak on behalf of all developers here, but what I've noticed is that most developers, including myself, want to please, and we want to be heroes. We have confidence in our abilities and think we can overcome challenging projects with aggressive schedules. We are easily persuaded into committing to these marches.
What do we do?
I'll go out on a limb and say that multitudes of developers find themselves in this very situation. What to do? In the cases where I have overcome these pressures, here were some things that I thought were key. They are things that I carry with me from here forward...
Own the Estimates, Own the Code
Don't let anyone fix date and feature set on you in a way that compromises the quality of your work. In the end the code we develop is ours. We need to be able to stand behind its quality. Not everyone develops mission critical software that people's lives depend on, but we all do develop software with the goal of making people's lives better. Pressure to over-commit is gonna come from multiple directions, but the buck has to stop with us.
Don't be afraid to deliver bad news
As I mentioned before, various players are going to have real reasons for pressuring the development team to over-commit. Its our responsibility as developers of the project to be honest about what we can and cannot do and clearly communicate that back. This can be done in a respectful way, but it has to be done. Often you'll find that there is room to negotiate functionality where it didn't seem there was before, once the stakeholders understand what we are truly capable of. This can't happen until they know that what they are proposing is not possible as-is.
Really be Agile
In the Agile I've learned, the one based on the actual manifesto, we don't make big, over-aggressive commitments.
- We develop valuable, small, and incremental working software with superior quality.
- We put extra focus and attention into how the code is developed early and up-front, because we know the cost of skimping on quality early elongates the quality of the end result.
- We set high standards for our deliveries. At the end of an iteration the code should be a production quality. Period.
This might be quite different than the way you are currently dealing with your stakeholders. You might even be calling yourselves "agile" and work in 2 week sprints, but if you are not in control of the work you do, and at the end of your iterations you are not producing valuable software with superior quality, then something has to change!
I believe this is a fight worth fighting. You may take some lumps early on for your stand, but the cases in my career where I have stood strong in the face of pressure have always benefited my teams and I in the long run. At the end of the day I think we all want to be proud of what we do and the work we put out. Don't let anyone take that from you.