Probably a really good title for a horror or a rock climbing movie, but sadly its neither, its just my thoughts on the software development process and how to dice a project into manageable chunks, to get maximum feedback.
When developers start to break a project up, they very naturally slice the project using the tiers of the application as the boundaries, then they work on each tier producing a completed product.
Over the years there have been many arguments over where best to start the development process, either with the UI (top down development), the database (bottom up development) or from the business services (middle out development). What all of these slicing and dicing methods have in common is that there is no program until right at the very end of the development cycle within the final rush to production.
This means that the user (the person who wants the application) doesn’t see anything until right before release. While the project is in the “black hole” of development the user is left with nothing to see, and invariably thinks up new and cunning things that the “application” could do and problems it could solve, which has become know as “feature creep”, and is the bane of every software manager!
In 1970 Dr Winston Royce published a paper entitled “Managing the Development of Large Software Systems”, the paper contained many illustrations of how software development could be managed, “completely designed before coded and completely coded before tested”, this image of software development became known as waterfall. Sadly, simply looking at the pictures and ignoring the text, has lead to the dominance of the waterfall technique. The text of the paper actually describes an iterative design process where each iteration has design, coding and testing elements within it. Only now some 40 years later is this style of development becoming popular.
The trouble is waterfall is very attractive to developers, simply look at the first paragraph, we tend to split projects in a way that dictates waterfall, there may not be the huge planning up front, but development still takes place in a black hole, where nothing escapes until the final rush of release Whereupon the bug, debug cycle starts as the user tries to mould the application into the thing they now realise they wanted.
Instead of a black hole, we need a revolving door, where software flows out, is viewed by the user, and flows back to the developers with notes and changes, a continuous stream of questions “is this what you want?” being asked by the developers in the code and partially completed application, and answers returning from the user “yes but like this” as change requests, and new features. Eventually a stage is reached where the user is happy and the software is released, this does not have to stop the flow of the conversation, changes can still be requested, and the application evolved, it is mealy that the application has enough functionality to be useful.
Change is no longer seen as the green eyed demon, that must be chained, and controlled, change is now the assistant is should be, guiding the application through to completion. The user moves from being the annoying person over the wall causing problems, to an integral part of the development process, a touchstone, for the application.
The iterative methodologies of SCRUM and LEAN extol the practice of quick turnarounds and getting feedback, and have iterative development at their core. But each can be subverted, if the product is not seen by the user before it is completed then is it still iterative? Where is the feedback loop?
To avoid subverting the point of the agile methodologies, the project should be sliced in a way that promotes feedback, and dissuades the practice of waterfall.
One way this can be achieved is to slice the project virtually down through all the application layers from UI to database, providing one clean slice of the application. The user can view a slice, one at a time as they are completed, each new slice being added to the previous, eventually enough slices are provided that the application is fit for release. Because each slice is so small it can be easily changed during the development process, these changes help to evolve the application, and because each slice is seen by the user, there is no change, debug cycle at the conclusion of the project.
Deciding what and where to slice is a problem that can be solved during the design process, and requirements gathering stage, having well defined features that slice the application naturally for the developer will promote iterative development and provide a vehicle for user feedback.
The makeup of a feature now becomes important, too big, and feedback won’t be received quick enough to gain the advantages of iterative development, too small and the user wont have enough to give valid feedback on, delaying valuable information getting to the developer.
To be well defined a feature should in my view.
Define a single action that the user takes and the result of that action
The defined action should result in data passing through each tier of the application, or the generation of a new view within the application.
The defined feature should be able to completed within a short period of time.
Because each feature is distinct and contained, development can be started and stopped on a product very easily, at the end of each feature, the next feature can be picked up, or a different product feature selected, in essence each feature becomes a bit like a bug fix, a single piece of functionality is added to the application, and then passed for checking to the user.
This is a radically different way of thinking about software, and requires a lot of practice to get the best from the approach, but the rewards are that the user of the software controls the release cycle, because they know where the project is at each point, and what features provide the minimum requirement for release, for the developer release is simply another iteration out the door