Agile vs Waterfall

There are several types of agile methodologies, however the Scrum Methodology is the most popular. In fact, the vast majority of software organizations use Scrum. The method’s primary tenet is executing shorter development increments to be more responsive to customer needs. Note the word “needs” rather than “requests”. Although the title of this page is “Agile vs Waterfall”, it’s really a comparison of the Scrum Methodology to the Waterfall Methodology, given that Scrum is the most commonly used agile methodology and best illustrates the contrast of agile to waterfall.

As Henry Ford said (or did not say, depending upon who you ask, but it still makes for a great analogy):

The point is that customers often don’t know what they want. They know the problem they’re trying to solve, but they often are unable to translate that into the best solution. It’s the job of product managers and UX/I Designers to interpret what customers say they want and define what they need. But we (and, yes, the author of this content is a veteran Product Manager) don’t always get it “right”. This results in the need to iterate software. The Scrum methodology is purpose-designed to achieve this.

The Scrum method centers on short development increments in which developers build a set of features, test them, and (potentially) release them to customers. Each increment, termed “sprints”, should result in a “potentially shippable product”. The word “potentially” is added to recognize the reality that in many cases features built in a single sprint cannot be released because features to be built in subsequent sprints are needed to complete a product release.

Consider building a v1.0 product (a “minimum viable product” or MVP). In the first sprint, the team completes features for creating user accounts, resetting a user’s password and logging in. But upon login, no functionality is available because no one has developed it yet. Thus, the team has built a “potentially” ship-able product in that the features they built are customer-ready (they are “done” based on the acceptance criteria defined for them). But in reality, the product is not yet ready to ship.

The Scrum manifesto states the following values:

• Individuals and their interactions over processes and tools.
• Working software over comprehensive documentation.
• Customer collaboration over contract negotiation.
• Responding to change over following a plan.

In short, Scrum believes that the objective of a software development team (and a team includes ALL job functions from planning to maintenance and everyone in between) is to as quickly as possible build software that meets the needs of customers. Documentation, process and tools only exist to make sure the team is aligned. A plan is only as good as its outcome, which means if the plan isn’t working, change it.

Waterfall is a traditional development methodology that breaks the key stages of building software into sequential, linear phases. Five phases were common:

The definition of market and product requirements typically culminating with the publication of Market and Product Requirements Documents (MRD / PRD). These documents contained all information about target customers and users, their needs, and the features that would satisfy those needs. Each product release was represented in a single document that might contain dozens, hundreds or even thousands of individual functional and non-functional requirements.

Based on the requirements, the architecture and user experience/interface (UX/I) design teams would go to work. The architecture would include the technology stack and individual system components or services and their method of interaction. The UX/I designs will illustrate the screens though which end-users would interact with the software.

In this phase, developers are coding the features of the product. Only coding. No testing intertwined other than perhaps unit testing done by the developers, which is tantamount to smoke testing (light testing of primary use cases of a given feature). Typically, the quality assurance team is defining their test plan, including the environment (what devices, screen sizes, etc. do they need to test on) and the test cases (the individual testing tasks that determine if each feature meets specific acceptance criteria or not. This would also include plans for load/performance testing as well as regression/system testing.

The quality assurance team executes their test plans including functional testing, regression/system testing, and load/performance testing. If those terms are unclear, functional testing looks at each feature independently to determine if it meets acceptance criteria or not. Regression/system testing determines if all features together result in defects that individual features did not produce. Load/performance testing determines if the software is able to handle the actual burden that real customers will put on the software.

The final phase is ongoing support and maintenance of the software.

Organizations commonly used Waterfall when shipping packaged software or enterprise premise-hosted applications. The emergence and increasing popularity of the Scrum methodology has generally aligned with the growth of hosted software, initially under the heading of “Application Software Providers (ASP)” and more recently Software-as-a-Service (SaaS).

The Waterfall methodology has numerous drawbacks for today’s modern software development organizations:

• Release Cycle Length: Waterfall releases too months and in some cases a year or more to complete. Each phase could be weeks or months long. Some organizations strived for quarterly releases, but few succeeded in optimizing Waterfall under this release cadence. Most took at least six months per release.
• Detailed Planning: The method was reliant upon upfront, accurate planning of requirements. In other words, you had to know while planning a given release all requirements for that release because once you complete planning, and the design phase begins, you can’t make any changes.
• Documentation Fatigue: Each phase resulted in the production of mountains of documentation starting with the beloved MRDs and PRDs. The documentation produced typically required approval of each department prior to phase in which the document was produced being completed to the point that the documentation became almost as important as the software it was meant to define.
• No Integration of Coding and Testing: Rather than partnering developers with testers to ensure that each feature is tested functionally when coding is complete, Waterfall stipulated that all coding be complete, followed by a “feature freeze”, which was then followed by testing of all features. This often resulted in systemic defects – those so deep in the code that finding and fixing it took far longer than if they tested in tandem with coding.
• The Blame Game: The transitions from each phase of Waterfall to the next were “gates” that required approvals of stakeholders. That approval process often led to a CYA behavior. For some, it became less important that the software built was actually successful than it was to make sure if/when the software failed that you or your team were not to blame.

The short version of the comparison is that the iteration length under the Waterfall method is too long to gather customer feedback and puts more importance upon adherence to the process than upon the success of the software in satisfying the needs of customers.

Agile methodologies, and Scrum in particular, exist to correct the shortcomings of Waterfall and similar methods. It’s tenets are focusing myopically on rapidly iterating and delivering working software that satisfies customer needs.