Agile Requirements Analysis and Modeling

Agile software development is a software development approach that aims to solve the imperfections of the traditional waterfall approach. In the waterfall approach, the main software development activities such as analysis, design, development, and tests are strictly separated. So before the implementation starts, the future solution is analyzed and described in a comprehensive document. The specification is subsequently passed to the development team that implements the solution accordingly. Once the solution is ready, testers take it and start testing it as a whole.

Over time, it turned out that this approach is not the best one for most software development projects. The main reason is that in the software industry, it is usually not possible to fix the scope of the solution. Software is complex, and there is a big amount of inputs influencing the final solution. But what is worst, the inputs continuously change. As a result, software projects used to be multiple times more expensive than estimated and delivered later than expected due to the changing requirements. The reaction to this situation is the agile software development process. It advocates an iterative, incremental, and more lightweight approach based on collaboration and face to face communication to handle the changing requirements better (for more details see Agile Manifesto). In practice, the software is developed in small parts (increments), which are analyzed, developed, tested, and deployed (iterative process). So after a couple of weeks, the team always has a small chunk of working software that has been fully verified and could be shown to the customer (for more detailed explanation check out this website, for example). This approach helps sooner discover the inconsistencies between what the development team is building and what stakeholders really need. Agile teams collaborate more, communicate more with the stakeholders, use more lightweight tools, and regularly present working parts of the final solution. Besides, they consider changes as an inevitable part of the process, so they do not fight it, they welcome it.

The agile approach is applicable not only to the software development process as a whole but could as well be used solely during the requirements analysis. Sure, developing software the agile way requires analysts to analyze requirements the agile way too, but it is not necessarily needed the other way around. The benefits of the agile requirements analysis and modeling could be used even on the waterfall projects. What is more, they should be.

In this chapter, we are going to introduce the main principles and practices which support the agility of the analysis activities.

Requirements Analysis On Agile Projects

At this point, it is worth mentioning that there is nothing like "agile requirements". Developing software using agile does not mean opting out of detailed requirements analysis. Each software product must be analyzed before it is built, the only difference is when the analysis is done. The agile analysis does not prescribe how much information about the solution is needed, it just says when it is needed. While in the waterfall, all information must be ready before the actual development starts, in agile, it is analyzed right before it is needed. On an agile project, the requirements analysis is not a single activity that is done at the beginning of the project. It is carried out throughout the whole software development lifecycle. Instead of describing all requirements upfront, only the skeleton of the solution is defined at the beginning, which is detailed enough just to enable making the main architecture decisions and planning the resources.

Because the software is developed in small increments (a fully tested and working piece of software), it is enough to analyze just the small part which is going to be developed next. This means that there is no comprehensive documentation at the beginning of the project, and only the necessary documentation for the upcoming iteration is created. This makes agile analysis incremental. It is also highly iterative, which means that there is no sequential transition from analysis to design, but both rely on each other. The high-level requirement is transformed into a more detailed requirement, but at the same time, a newly discovered low-level requirement might influence back some of the high-level requirements.

The downside of the agile approach is it requires not just the development team to be agile, but also the stakeholders. Since there is no single timeframe when the requirements are being elicited, stakeholders need to be actively involved in the course of the whole project and be flexible in providing information and making decisions anytime it is needed. They must be prepared to be much more supportive of the development team than it is in the traditional approaches. The key stakeholders literally become part of the development team, which is time-consuming and may put pressure on them.

Requirements Management On Agile Projects

Agile development is driven by a prioritized queue of requirements called a product backlog. This queue contains requirements which are usually described in the detail that enables creating rough estimates and assigning a priority. Each of them is typically just a paragraph of text. The requirements are then taken from the backlog according to their priority and elaborated in detail just before it is going to be developed so that there is only a minimal time between the analysis and development, which helps minimize the risk of change and rework.

While the product backlog may contain practically any requirement, in today's agile methodologies such as SCRUM, it mostly consists of user stories.

User Stories

A user story is not a full specification of a system feature. It is a "promise to have a future conversation". It articulates what the stakeholders want or need from the system and why so that it could be prioritized and registered in the product backlog.

USER STORY

In software development and product management, a user story is an informal, natural language description of one or more features of a software system.
(source: Wikipedia)

User Stories are great as an activity in collecting and prioritizing the high-level features. It is a short description of something that the stakeholders need to do with the system, focused on the value or result they get from doing it. They are written from the stakeholder's point of view, in the language that the stakeholders would use. They may have various formats, below is the list of the most common:

1. As a <role> I can <capability>, so that <receive benefit>
2. In order to <receive benefit> as a <role>, I can <goal/desire>
3. As <who> <when> <where>, I <want> because <why>

Example: As an e-shop administrator, I want the shop to include a contact form so that it is easier for the customers to reach the customer support and the technical problems are announced as soon as possible.

As you can see, the user story does not state which input fields the contact form must include, what data validations are going to be present or where the data are going to be sent to. The goal is not to fully describe the feature, but just to indicate who wants what and why. The details will be analyzed just in time it is needed, specifically before the user story is about to be picked up from the product backlog.

Benefits of User Stories

User stories are not a requirement analysis technique, so in no way could they help analysis with eliciting and modeling requirements. On the other hand, correctly applied user stories come with many benefits. As they are written from the user's point of view, they promote focusing on the user's needs and abstracting from the technical solution. The "receive benefit/why" part also naturally captures the rationale. And finally, telling a story is more vivid, therefore more effective, than just listing dozens of requirements in the table.

Mistakes to Avoid

1. User stories should be used just for capturing user goals. Sometimes we see teams write every single requirement as a user story, even the technical ones, which leads to funny statements.
2. Another mistake is blindly following one prescribed user story format. The added value of the story does not lie in the format, but in the information it holds.
3. Use cases are always related to a single system. User stories, on the other hand, are written from the user's perspective, not explicitly stating which system is going to implement the story: "As an officer, I need a report of today's orders so that I can check whether everything is in order". Such user story represents what needs to be implemented within the project, not within any particular system. Analysts need to distinguish whether the requirement may impact multiple systems or if it is a use case of a particular product.

IMPORTANT

For most agile approaches, the basic entity which is estimated and prioritized is the user story. But the true agility means using what is best at the given situation. If the basic artifact of the project is a use case, the backlog will include use cases. If the project is process-oriented, the backlog will consist of process activities.

Agile Requirements Principles

The following list presents principles that increase the agility of the requirements analysis and modeling. Although they are most beneficial in combination with the agile software development process, they could be successfully adopted to some extent, even on traditional projects. You really do not need to work in a fully agile environment to take advantage of the agile way of analyzing and modeling systems.

1. Iterative Approach to Requirements Elicitation and Analysis

As described at the beginning of this chapter, analyzing all requirements upfront is ineffective on most projects. Instead, requirements should be specified in iterations, which enables the team to develop a set of the most critical parts of the system while the analysis of the rest can be postponed. This gives them the chance to discuss the basic features with the stakeholders very early and get instant feedback, which might positively impact the rest of the analysis.

2. Changing Requirements Are Normal

The main problem of the waterfall, and the core motivation for more and more teams switching to agile, was the inability to react to changing requirements. New core requirements discovered during the acceptance tests caused many project delays or even failures. As a result, many teams were trying hard to prevent requirements from changing. They were building comprehensive specifications trying to tune up every detail, they made stakeholders sign off requirements specifications before the actual development started to make stakeholders think twice before changing something. But this all has shown counter-productive, as it just made stakeholders put everything they could think off to the specifications. This just led to inflating projects with features that were rarely or never used in production and which only made projects more expensive. What is more, this all ruined the relationships between the delivery team and the stakeholders.

The truth is, no matter how hard we try, it is impossible to prevent requirements from changing. In fact, if the requirements do not change, likely, the stakeholders are not interested in the solution we are building. Creating comprehensive requirements specifications before the development starts is good for managers and sales representatives, but neither for the development team nor for the stakeholders. Stakeholders are in stress that they cannot change their minds regarding their requirements. Analysts are in stress that they need to specify everything in detail upfront. And developers are angry that they very often need to throw away features on which they have spent a lot of time and which are newly considered out of scope.

The better approach is to first map all requirements from the high level to have an idea about the scope. This enables the team to create a high-level overview, basic solution architecture, and analysis plan. Then the goal is to identify the most important, risky, or complex requirements, which will become the first candidates for a deeper analysis. The analysis is driven by priorities, and the most important features are analyzed and developed first. Features are not analyzed upfront, but only when its priority allows for the development. Such an approach makes stakeholders more relaxed as they are not forced to think through everything in one go as the process is open to requirements changes.

3. Simple Requirements

Agile analysts keep the requirements formality and the amount of detail on the level, which enables them to deliver the solution successfully, yet at the same time, which requires analysts to spend as little time analyzing requirements as possible. A good practice is to always start specifying requirements in the form which is least detailed and formal and which enables fast progress. Adding formality and details then takes place only if there is justification for it (stakeholders or developers do not understand the requirements, for example). This applies to all forms of requirements, be it textual, visual, or models.

1. The Best Tool For the Job: Agile analysts do not have a fixed set of techniques to capture requirements. They are aware that each project is different, consisting of different types of requirements, so they always use the right tool for the job. This usually means combining multiple techniques, so it is quite normal to describe some requirements using use cases, some using diagrams and the rest is described in a table.
2. Lightweight Techniques And Models: Because agile analysts know that requirements are going to change, they do not waste time on creating pixel-perfect screen designs, formal over-detailed models, and heavy-weight specifications. They create only the minimal artifacts which are needed to successfully communicate the requirements with stakeholders and which enable smooth implementation.

4. Communication And Active Participation

Agile analysts prefer communication and collaboration when eliciting requirements. Unlike traditional analysts, they try to avoid discussing requirements using documents. They rather discuss requirements with stakeholders personally on workshops or live product demonstrations to verify they have understood everything. In general, agile analysts use more visual techniques such as sketches or prototypes. Also, to be effective, as many requirements as possible should be in an executable form. This means that agilists try not to waste time describing every field on the screen, and instead, they create a development version of the application and give it to the users to obtain rapid feedback.

In general, agile analysis involves more communication. Textual specifications are ambiguous, requirements are changing, and the only way to verify that analysts and stakeholders understand each other is frequent and effective communication. Analysts should always try to discuss as many requirements as possible personally to avoid possible misunderstandings stemming from the written communication. They should also prefer presenting working solutions instead of creating specifications.

5. Agile Requirements Documentation

Agile analysts spend definitely less time documenting requirements than their traditional counterparts. If the specification is not the primary tool for communicating requirements, and the requirements are not analyzed upfront, then the analysis outputs could be briefer and more lightweight. It is a dangerous antipattern to think, there is no place for documentation on agile projects. Each project requires a particular amount of requirements documentation, to capture information which should not or cannot be stored just in the team members' heads. Not having any documentation is frustrating, no matter if you do agile or not. The difference is why the documentation is created, what amount of it is created, in what form, and when.

Why

Requirements should not be documented just because somebody said so. It must have a purpose as there is an effort associated with creating it, and we definitely do not want to create something which is not going to be used, right? Also, documentation should not be created as a communication tool unless there is no other, more effective way. If you could sit down with stakeholders and show them a demo, wireframes, or a process flow, prefer it rather than sending them a document. Documentation (be it a document, model, or any other form) should serve as a shared team memory for information that has a justification for being documented. For example, the information is complex, it took a lot of time to find it, and it would be a big loss to lose it.

What

Team members should understand that creating documentation not only takes time but it also requires effort to maintain it. There must be a balance between the costs of creating the document and the value it is expected to provide. It is not a good approach to spend half of each working day creating documents and models as your primary goal is to create software. On the other hand, it is frustrating to have nothing to show to the new team member, which would provide him with the context of the business, product, and architecture. We focus on documentation more in this chapter, here we just outline the basic points:

• Document overviews not details: Overviews are more valuable than detailed specifications as they provide a context and enables the reader to learn where to get more detailed information. Of course, it is frustrating to have requirements documentation presenting just general things and nobody knows the details, but it is not feasible to document everything. Since requirements could change a lot in agile, it must be chosen wisely, which details to document. Overviews are general enough to be resistant from frequent changing.
• Document stable things: Since agile analysts do not try to prevent requirements from changing, they naturally must document less as reworking requirements documentation each time something changes would be frustrating. Documenting just general things in overviews is the way, but there are cases when the details are worth capturing too. Agile analysts do not defy documenting details if it has a justification. What they do instead is they postpone the documentation activities to as late as possible to increase the chances the requirements are stable and the risk of rework is lower.
• Do not create perfect documentation: Requirements should be documented in the form and quality, which is good enough for everybody to understand them, implement them, and test them. They do not have to be perfect. Perfection is time-consuming and mostly not necessary. What is good enough depends on the project, team, and other aspects that need to be assessed case by case. Do not insist on formal models if a quick sketch does the job, for example. Take the lightweight approach, use whiteboard, pen and paper, and other lightweight techniques that have the best value to effort ratio.

When

The lack of documentation is frustrating, but at the same time, we advise not to document much. In fact, the truth is somewhere in between. Documentation is great when it is created with a purpose, it is really needed, and when the benefit of having it is greater than the cost of creating and maintaining it. Also, documentation is not a source of frustration if it is up to date and contains mostly overviews and stable things as described in the previous point.