The Purpose of Modeling

My client wants a complete domain model, including every field and relationship...

I just finished a laborious, and somewhat exhaustive review of a relatively elaborate and extensive domain model with a set of business stakeholders last week. I know that a lot of upfront documentation is never considered a good thing on an agile project, but sometimes, when the scale of a project is incredibly large (think $20 million budget, and we inherited several thousand pages of requirements documentation before I came on board) it is necessary to capture business concepts in a form capable of providing effective guidance for future development.

The primary goal of this domain model was to take the vast ream of requirements given to us, and present them in a form that would be digestible by both business and developer/techie resources. In terms of total volume, the model consisted of approximately 16 diagrams, each describing a different domain concept, with approximately a page of text going along each diagram. In my opinion, this is not an awful lot of documentation compared to the source requirements we inherited which were thousands and thousands of pages long. When creating the domain model, we relied on many of the domain driven design approaches that can be found at http://www.domaindrivendesign.org/.


What exactly is the purpose of modeling?
While going over our models with business stakeholders, the biggest problem that I kept running into was that the client wished to see every single field and every single relationship on every class. They were under the impression that if the model wasn't perfect/complete it wasn't going to be useful. I did my best to explain that models did not need to be 100% perfect/complete in order to be useful, quite the contrary in fact. I'm not sure I did a perfect job in doing this, so I thought I would write up a quick blurb on what I believe are the primary purposes for modeling in the hopes that this will help me to defend my position better on in the future.

I should also point out that Scott ambler is an excellent job of describing modeling principles and best practices on his agile modeling method site

1. Modeling enables Understanding
   
   IMHO this is the most important part of modeling. Modeling enables people to understand what has gone on before their project, and also allows them to get a better understanding of what needs to happen next.
   
   As an example of the "gone on before", take the all to typical situation where a developer team inherits requirements; sometimes of good quality, sometimes of not so good quality. Developers are asked to design or develop based on these requirements, and frequently don't even know where to start. Modeling and diagramming is an excellent way to comb through these artifacts and apply some critical thinking around them so as to make some initial directional setting decisions.
   
   As an example of the "what needs to happen next", the same example also applies. The modeling done by developer type resources can also be used to apply critical thinking around what the data structures will look like, what the interface screens will look like, what services will be required, et cetera.
   
   This modeling exercise is also the best way for these developer resources to actually understand any legacy documentation. Simply asking someone to read a pound of documentation will rarely result in comprehension of that documentation. Giving someone some documentation and asking them to perform a discrete activity will result in exponentially more understanding. Examples include reading requirements to do initial analysis/domain modeling, initial vendor analysis, usability/interface modeling, actual development, et cetera.
   


2. Modeling Enables Communication
   
   Simple models that abstract and structure a large amount of requirements (or any other system information for that matter) are worth their weight in gold. The maxim "a picture is worth a thousand words" is no truer than in the system development space. Being able to illustrate a vast and incredibly complex system with 10 to 20 pictures (or 1 or 2 depending on the level of abstraction) goes a long way toward ensuring that a lot of distributed teams have the same understanding of what the system needs to do.
   

Complete models, be careful what you wish for...

So what does this have to do with my original clients desire to "have every field and every relationship described". You'll note that nowhere in these above two points did I mention "modeling to specify". Creating a specification means detailing every field, detailing every constraint, and every relationship, regardless of whether its fundamental to the point you are trying to make on a specific diagram or model. Creating a "complete" model actually makes it much more difficult to achieve the above two objectives. And here IMHO are the reasons why...

1. Important concepts are obscured: if every relationship, every field, and every business constraint is modeled and presented how can we tell the important ones from the unimportant ones? A model that is too detailed has the unintended consequence of intermingling important concepts with unimportant concepts. The art of actually hiding unimportant information is crucial to creating good models, it's probably harder to create models that only display relevant information than to create models that show everything. But the result is certainly a better model, one that enables understanding of the system across a variety of stakeholders.


2. The model is impossible to maintain: having been on several projects with a model oriented bent, I've had plenty of hands-on experience on the overhead required to maintaining readable and accurate models. In my experience, it is simply impossible to create maintainable models that are 100% precise down to the field level. At best, models are created that are "true when created" and get more and more irrelevant as time goes by. There are various experts out there who claim that modern tools make it easy to keep detailed diagrams and technical code in sync with each other. These experts need more actual hands-on practice with some of these tools, because this notion is still the stuff of ivory tower dreams, do not underestimate the effort involved in maintaining these types of models, it will equal, and often surpass your actual coding effort.


3. Complete models are really just "code-umation”: code-umation” is documentation that approximate the detail level of actual code itself. This type of documentation is no easier to navigate than code, it is certainly harder to change than code, and is not executable or testable. Run, run, run as far away from this approach as you can. It doesn't matter if you're in sourcing, outsourcing, working on a large or small projects, this type of detailed documentation does nothing to help developers do their work, try picking up the phone, having a face-to-face conversation, or any other method to communicate detail of this nature. Better yet, follow some of the excellent implementation patterns found in Kent Beck's excellent text here to ensure that your code is readable and self-documenting.


4. Drudgery versus innovation: creating detailed, field level, specification oriented models is mindnumbing, monotonous work. Good models are the result of critical thinking, innovative solutioning, and thoughtful analysis. These two styles of work are the opposite of each other. The more you focus on low-level details the less you'll focus on figuring out what the right things to do are, if you had the choice of receiving a high quality modeling backed by creative intelligent thinking or a complete model that was a result of monotonous repetitive effort what would you rather have. Don't ask for both, because it's simply not realistic.
   
   So in short, when modeling, focus on what's important, focus on what you need to communicate, and focus modeling on helping you understand what you need to do next. In most cases the simple advice will go a long way in streamlining your modeling efforts in ensuring that your modeling and models stay relevant.

Can Use Cases Be Used in an Agile Way?

Can one use use cases on a agile project? do use cases provide any value for agile teams?
I' m going to put out my opinion unequivocally Yes, Yes and Yes.

Most agile purists would disagree with me on all the three above points. Typically agile development approaches use user stories, backlog items, or other simpler mechanisms to describe requirements. They often have the format of "as a student i can purchase books". These requirements are typically put on an index card, and then not elaborated until it's time to code, and usually only by conversation, sometimes with a couple lines of text.

First of all let me state that I find user stories very useful for estimating , planning and tracking. I don't however find them very useful for providing context, structure, or abstraction. My experience has been that using user stories alone does not scale very well when working on large projects where multiple views of requirements are needed. For instance, a high-level view for project directors and other executives or a more detailed view for developers.

I can appreciate the bias that most agile practitioners have against use cases, because frankly, most use cases that I have come across in my experience are very poorly done. They are hard to read, they have way too much structure, rely way too much on UML syntax (extensions and usage relationships) and can often digress into what I call "functional decomposition". In order for use cases to provide value to any Project, agile or not, they need to be written properly. I specifically recommend two excellent books, Patterns for Effective Use Cases and Writing Effective Use Cases both of which Alastair Cockburn either authored or co-authored.

What I have euphemistically coined the "Cockburn Use Case Approach" describes how to write use cases with just enough formality and structure to allow proper abstracting, good context, multiple views, etc.; while still being readable by end-users, and not requiring a massive amount of effort to build and maintain. Most importantly, this technique focuses on writing scenarios using natural text, not on creating uber complex UML diagrams.

Alistair Cockburn has posted a pretty good piece justifying his use of use cases which I recommend taking a look at.

Some other comments on the topic can also be found here

A Brief Case Study

I have decided to share a brief case study regarding the own experience with user stories and use cases in general.

About five years ago I was asked to take a fairly large public sector program that was being previously conducted in a waterfall fashion and focus on getting some very quick functionality out the door. The client was previously frustrated by the millions of dollars spent with only paper documentation (and not very good documentation at that) to show for it. I lead a small task force of around five developers for about six months, our goal was to try to win back client confidence by delivering "something" quickly, and iteratively. We followed the XP process as it existed at the time, using user stories, velocity, etc.

While overall it was very successful, we quickly ran into many of the problems stated above. We had a lot of problems organizing our stories into a higher context, and we had no executive view which would allow client leadership to let us know if we were generally in the right direction. As a result, we wasted a lot of time developing functionality that was never actually used. (Often users on the ground working with agile teams do not have proper executive context, just a fact of life at least in all of my experiences)

As the program started to grow in size with multiple teams, I started looking into use cases, going through a variety of use case books until I came upon Writing Effective Use Cases and Patterns for Effective Use Cases. We started writing use cases according to the advice in these books, and saw great value immediately.

In all other respects, we still followed an iterative process, we developed use case briefs for a large portion of the use cases upfront. These then acted as our user stories in terms of planning, and estimating. The first thing we then did at the beginning of each iteration was to develop scenarios and extensions for use cases assigned to the iteration. Still agile, still iterative, but we used use cases.

Why Aren't These Cases Accepted within the Agile Community?

Throughout the last five years I have subsequently been a lead architect and process lead on no less than five separate projects (some very large and some very small) where I have used these techniques for use cases. I combined these use cases with other agile approaches (CRC, story points, test driven development, etc.) and have found them to be fantastically useful. This whole agile bias against use cases stems from the fact that use cases are probably one of the most abused SDLC related artifact. There are many different ways to do use cases, and from what I've read, most of them are just "wrong". But when done according to the advice found in these two books, they can really support an agile approach. I'm also confused by the agile maxim that you should "do what works", as long as you don't use use cases, never do any upfront modeling, don't use any case tools, etc., etc.

IMHO this sometimes religious bias against other tools can sometimes limit the appeal of agile by nonbelievers. I'm probably going little off-topic and will stop there.

Maybe the whole agile bias against use cases is a reaction against previous BUFD. But I still agree a little bit of upfront thinking is always going to benefit any development effort, as long as the emphasis is not on creating a requirements document, but doing the necessary thinking required to ensure that the right product is developed.

I think use cases can easily mapped to user stories, and can fit into almost any agile process
summary use cases > epics, themes
system use case > regular user stories
use case briefs >user story descriptions
use case details >part of iteration planning, determining the iteration objectives.

I will continue to use use cases on future projects, but combining them with more traditional agile techniques.

Why domain driven design is important

For several years I've been convinced that one of the core concepts of a system that should be modeled is the business domain. Being able to tell a common story using language, diagrams, and actual code that is focused on the business problem being addressed is one of the incredibly powerful capabilities of object-oriented technology.
This approach is being increasingly known as "Domain Driven Design". The term comes from an incredibly good book by Eric Evans. While browsing the Internet I came across a pretty good video that describes the benefits.

Interview with Rebecca

CRC cards-incredibly successful so far

During the last week of December I submitted a post discussing my intention to use CRC modeling techniques on what I would call a "massive" project. I needed a way to collaboratively build an overall domain model for an incredibly complex solution that was currently expressed in a thousand plus pages of linear, non-scenario-based business requirements. I said that I would report on my progress in a later post, and I'm happy to state that CRC modeling has been a fantastic success. While the overall constraints and structure of this project cannot in any way be called "Agile", applying this agile modeling approach across our internal team is providing a fantastic amount of value.

As I stated in my previous post, I initially followed a more traditional modeling approach, which involved a technical team:
1) Reviewing the requirements and creating models
2) Placing white board models into an official UML tool (Rational Software Architect)
3) Holding various workshops with business experts on our team to make sure that the models properly reflected the requirements

Again, as stated in my previous post, this approach had very limited success. The main problems being
1) Business resources were simply not a core part of creating the model. Reviews were simply not that successful/productive

2) UML repositories are really hard to share and maintain on a large team. Especially when rapid change is required.

Note: A lot of agile signatories (Scott Ambler especially) have strong cautions concerning using complex UML case tools; I never understood this before I tried using one with a team of 7 plus resources working on a shared model. It is difficult to overstate the aggravation involved in keeping this model in sync. An excellent resource on case modeling tools can be found on Scott Ambler's site here.

3) The case tool itself was causing the team (myself especially) to model with much more detail than required for this stage of the process. There's a lot of good advice out there on modeling that states that modeling should be at a high level until they can be validated through real implementation. UML diagrams by their very nature can sometimes seem too detailed for this initial analysis.

Within week one of the project restarting after Christmas break, (sometime around the second week of January) I had organized three parallel teams of two technical/modeler types and one business/functional resource. I myself played a floater role of mentor/facilitator/advisor across the three teams. Of course, I only had a week more experience than any of the other teams, but I'm used to doing what I call "just-in-time training". (meaning I train myself just in time to train somebody else)

The technical/modeling resources all had a pretty good understanding of OO and UML, while the business/functional resources had spent the last year and a half collecting 7000 plus business requirements, and thus had a pretty good understanding of what the solution needed to do in order to be successful.

We set up a process that involved spending three hours in the morning conducting modeling sessions using the separate teams mentioned above, then the modelers excluding the functional resources would spend the next half day consolidating the models, removing duplication, creating common structures and making sure that the separate work was combined into a cohesive whole.

Personally, I was personally concerned that CRC cards would not effectively scale, that it would be too hard to organize the cards//we organize them as necessary, that the model would be hard to maintain, and that we would be unable to get the right amount of detail. I was also concerned that the business/functional resources (who have no object oriented training whatsoever) would simply not be able to properly participate.

I have to say that my concerns had been completely unwarranted, every team appears to have had very good success in restating requirements into a CRC format that is now digestible in a matter of minutes, as opposed to hours. Even more important, functional/business resources have all played a vital role in deciphering/explaining business requirements and contributing to the quality/accuracy of our CRC models. I was personally satisfied we are on the right track when I saw specific CRC cards being built by these functional resources. Even more important, the proposed system appears to be a lot simpler than perceived by simply looking at the original requirements. Certainly there still is a lot of complexity, and this will by no means be an easy system to build, but it has become apparent that there is a lot of shared pieces that apply to 70 plus percent of the requirements.
Our client has also been very firm in wanting a Rational Software Architect version of the model. What that means is that we've spent the last week or so entering our CRC cards into this tool. The work involved in ensuring the integrity of the model within the tool, and the level of effort required to make simple changes more than validates the incredible flexibility provided by using simple index cards and particleboards. Changing the contents of a CRC-based model is incredibly easy, incredibly quick, and painless. I'll probably never attempt any serious modeling exercise without CRC cards again. And actually probably little embarrassed that I've never used a CRC before. So two thumbs up IMHO for CRC cards.