Introduction

The vast majority of the current organizations are absolutely dependent on their information systems and software applications. Almost all company information and processes are now managed by these systems, which implies two important facts. First, the IT ecosystems powering today's large organizations are so complex that there is not a single person who could describe how it all works end to end. Second, organizations ended up in a situation when their capabilities are more or less dependent on the capabilities of their IT systems. In other words, changing an organization always involves changing its IT systems, be it introducing a new system or removing or modifying a legacy one. The combination of the two aspects mentioned above puts enormous pressure on people responsible for discovering how the planned business changes will impact the IT systems. With the increasing amount of information managed by the systems, the impacts get bigger and bigger, and finding the right solution to the business problems becomes harder. This requires people searching for solutions to work more effectively with such a big amount of information. To achieve this, knowledge of the organization must be captured in a structured and consistent way, which enables getting the right information at the right time.

Although there are many parties in the organization whose work is highly dependent on the quality information, the one for whom it is especially important is the analyst. He or she usually drives the change and is mostly the only person to see the big picture who also uses it to help business people to make the right decisions. This is why the analyst very quickly becomes the most important person behind the change, and the quality information is crucial for this role. The need is even more amplified in the case of big organizations supporting dozens of processes and operating dozens of IT systems. In such environments, it is inevitable to set at least basic rules on how to structure the information. Whether the analyst describes what the company has (its structure or IT architecture), what it is capable of (products or processes), or what is required to be changed, it must be documented in an organized way. What kind of information to capture, when, and how to structure it is a vital part of the Effective Analysis and Documentation. But it looks at the analysis from a much broader view. It starts with introducing what analysis is, elaborates on different types of analysis, explains the role of the analyst, and what qualities the analyst should have. Then it describes the most frequent mistakes which cause ineffectiveness that block analysts from working faster and making fewer mistakes together with the advice on how to get rid of them. After that, it elaborates on what information analysts usually need to make decisions, what information they produce, and how they structure it. The last part is focused on how the described practices and techniques could be implemented using real tools.

It should be pointed out that there is no best approach to analysis and documentation. On the other hand, we want to avoid presenting just high-level abstract concepts. There are many quality books focused on how to analyze, model, and describe software. However, they all tend to be very high-level just because they want to cover all possible situations, projects, organizations, and teams. Analysis is more of an art than a science, and the rule "one size does not fit all" applies to analysis more than to any other discipline.
Despite this, we have decided to create a handbook, which is general enough to cover most of the situations analyst might encounter, yet to provide concrete guidelines on how to proceed with the particular tasks on various projects in daily life. Comparing analysis to software development, which has its own best practices, methodologies, and automation tools, analysis is still in its infancy. Most teams do not have any rules or guidelines and their approach to analysis is "do it the best way you can". Our motivation is to take analysis closer to the state of software development by providing concepts, good practices, and tools for automating it. We also aim to inspire analysts by demonstrating the techniques on concrete, real-life examples instead of just presenting abstract concepts. The goal is to give them the confidence that the particular approach is ready to be used in the production environment.

We believe that there should exist a set of basic rules and guidelines which may be effective and, at the same time, could work in a wide range of environments. So the aim is to present a unified guide on how to analyze and document organizational components (processes, IT systems, organizational structure, etc.) that is usable in most environments. Plus, we help automate it!

Real-life Motivation

A project manager (PM) approaches an analyst (AN) on a big project and asks questions:

PM: "We had a status meeting with our key stakeholders, and I couldn't answer their question of how we calculate the fee when the client is due with the payment for the service. The project gets bigger and bigger, and I can't remember such details. Can you?"

AN: "Mmm, this was analyzed by Mike, who has already left the company. I heard that the requirements are of poor quality. Let me check..."

AN: "Got it. The requirement FR.039 states that 'Penalty fee for the late payment is calculated automatically. The algorithm is included in the overall calculation spreadsheet on disk O:\'"

PM: "Good job. Can you find the algorithm in the spreadsheet, please?"

AN: "The rule in the spreadsheet states that the fee is calculated as 0.5% from the price for the service and is calculated every day. The bad news is, the spreadsheet was used only during our requirements workshops and has not been updated since."

PM: "How is that possible?"

AN: "The requirements had been written by the business analyst. Then they were transformed to use cases by another analyst. After an hour of searching, I found the right use case. It was named 'Calc penalty' instead of 'fee', so it wasn't easy to identify it. It says the fee is 0.6 percent."

PM: "Ok, thanks. Besides, they asked me when we calculate the fee. I guess it is some nightly batch job, right?"

AN (after another hour): "The use case is apparently run every night. But it processes only low-value clients."

PM: "Who are low-value clients and how we charge the fee for the other clients? I can remember we had a Sharepoint page containing all the business terms, is a low-value client there?"

AN: "I have already checked it, and it contains only the term Client. Give me a minute."

AN (after another hour): "Ok, I checked with Steve, the team leader of the Client care department. He had a meeting, so it took me an hour to get the information. According to him, the low-value client is determined by the account balance. So I searched through the analysis and found the definition in business rules, which were in another spreadsheet. Low-value is under $250,000."

PM: "Alright. So the last question is how we charge the fee for the high-value clients."

AN: "I tried to find this too. But apart from the business rules, I didn't find anything regarding the high-value clients. I need to go to the 3rd floor and ask the people responsible for the fees directly."

PM: "Ok, but please hurry up, the project is delayed, and this issue takes our valuable time. Thank you."

AN (after another 40 minutes): "Heureka! They told me that high-value clients are contacted directly by the bank employees through the phone. There is a report delivered to the manager's email address containing the list of the high-value clients who should be contacted. I found that report in our analysis, it's REP-028, and the reason why I couldn't find it sooner is, it talks just about clients with the account balance > $250,000. No mention about high-value clients."

PM: "Great job, I appreciate it. I will sum it up and send it to the stakeholders."

A cruel story, isn't it? Unfortunately, this scenario is quite common. Most of us working on big IT projects with a lot of stakeholders have gotten into this situation We know how frustrating it is trying to modify some part of the system, but it is first needed to find out how it actually works at the moment. Another example is joining an already running project and trying to learn its background, goals, and requirements. It is often not possible without talking to dozens of people, taking their valuable time. People on projects come and go, even stakeholders may leave the company during the project. Sometimes the processes and business logic are so complex that even stakeholders get lost, which could cause a funny paradox: business stakeholders who have defined the requirements eventually ask the project team to remind them how the hack they have defined the requirements. Sounds silly that the people responsible for telling you what to build asks you what they have told you to build? It is surprisingly more common than one would think. Replacing legacy system or radically modifying it is usually being performed during the standard operation of the company and defining the requirements, attending workshops, providing consultations or carrying out user testing are tasks which are done alongside stakeholders' daily agenda, which in the long term might cause exhaustion of the business team, apathy, and lack of drive. As a result, the project team very often becomes also a "support team," which needs to have the right information, needs to be able to connect the dots, and use it to help stakeholders to define the solution clearly. This is especially true of analysts who, unlike the stakeholders, should see the big picture which spans across the borders of the individual business units. Consequently, analysts very often become the most important people on the project, as they have the helicopter view, and at the same time, they are actually subject-matter experts in the individual business domains which they have learned during the project. It is then very common that in the late phases of the project, they got asked by business people: "How have we eventually designed the algorithm?" or "Do you remember if we send an email or not in this case?".

It is not an easy task for analysts to work with such a big amount of various information. They need to know where to find the information, which information to record, when and how to record it, and also, how to structure it in the way it remains manageable. Effective Analysis and Documentation focuses on these aspects and elaborates them in the context of various project types and different individual project phases. It discusses problematic analysis and documentation approaches that are frequently being adopted on projects and proposes ways to improve them. It also presents guidelines that will help analysts to produce more structured and consistent outputs that bring more value. These guidelines aim mainly to streamline the analysis effort and reduce the frustration of everybody who will try to learn about the project or understand how existing enterprise components work. It also comes with a ready-to-use framework, which may be used to document projects or parts of an organization. On top of that, it provides a reference implementation that demonstrates how the analysis and documentation could be improved using the right tools and automation.

It is also fair to say that many of the presented principles and ideas are not entirely new, and a big part of it is just an aggregation of the existing resources. The goal was not to reinvent already proven principles, but rather to combine the best approaches together to provide a comprehensive framework and to complement them with real-life examples to demonstrate how particular problems could actually be solved.

Who Should Read This Textbook

The intended audience is mainly people who aspire to become analysts on IT projects or who have already been working as analysts but want to improve the way they carry out their tasks. There is no prerequisite for reading since it starts with the basics: introduces what it is analysis, what are the responsibilities, tasks and skills of an analyst, along with the basic artifacts and models analysts typically produce. The main part focuses on the main causes of the frustration associated with the analysis and documentation, which brings ineffectiveness and delays to analysis work. It shows approaches to reduce the frustration and to make analysts more effective. This part is valuable for all analysts regardless of seniority. The third part introduces a unique way to create both development and post-release documentation. Since analysts are not the only ones who document their outputs, reading this section will be beneficial for anybody who produces documentation or suffers from other's poor-quality documentation.

Here are some examples of who might be interested in reading Effective Analysis and Documentation:

• You are new to analysis, you are now responsible for collecting requirements, preparing inputs for the development team and you are curious how to effectively proceed in this new role
• You have already seen many different approaches to documenting the analysis outputs or the current state of the enterprise, realized that it is not an easy task and now you are curious if it could be improved to make you more effective
• You are an experienced analyst, who has already read a lot of about UML, business analysis and system analysis, has got a decent knowledge of what should be done during the analysis and when it should be done but you still miss the "how." You miss the connection between the sources you know, and you ask how to connect the information together.
• You might be a developer who needs to create a specification of a low-level algorithm or to show how the technical components of the system are integrated
• You are a project manager who communicates with stakeholders often, and you always find it hard to get the answers from them
• You feel that outputs from the analysts have nothing in common, are of poor quality and you believe there should be a better approach
• You feel that discovering impacts of the business changes on the organization is more expensive than it should be

What Is Not Covered

Just like an experienced analyst should know that a vital part of the scope definition is also an explicit statement of what is not in scope to save stakeholders from disappointment, the scope of the Effective Analysis and Documentation is also limited.

Following topics are not covered in detail:

• How to choose the right analysis approach (agile, waterfall, etc.)
  • Although we are in favor of agile approaches and many herein presented principles are inspired by agile, we think that the concrete approach depends on many factors, so we leave the selection up to you
• How to elicit, develop and manage requirements, how to deal with change requests
  • A significant part of EA is dedicated to requirements, but it doesn't state concrete approaches as it often depends on the overall analysis approach
• How to facilitate requirements workshops, how to communicate during analysis or how to lead a team of analysts
  • Even though EA mentions the required quality the good analyst should have, it doesn't directly cover these topics

Real-life Story - Conclusion

You thought that our story from the beginning ended, right? The answers were found, stakeholders are happy, so all sorted. Well, kind of. Until the end of the month:

PM: "Hey Steve, I see 4 hours in your timesheet on Thursday the 2^nd spent on some "Administrative task." What was that?"

AN: "It's the time spent on searching how we calculate the fee for the delayed payments, you remember?"

PM: "Oh, yes. But I can't charge the customer for these 4 hours, it was our fault that we were not able to get the answer faster. Our company needs to pay for this. We definitely need to improve our documentation..."