The Purpose of Modeling

My client wants a complete domain model, including every field and relationship...

I just finished a laborious, and somewhat exhaustive review of a relatively elaborate and extensive domain model with a set of business stakeholders last week. I know that a lot of upfront documentation is never considered a good thing on an agile project, but sometimes, when the scale of a project is incredibly large (think $20 million budget, and we inherited several thousand pages of requirements documentation before I came on board) it is necessary to capture business concepts in a form capable of providing effective guidance for future development.

The primary goal of this domain model was to take the vast ream of requirements given to us, and present them in a form that would be digestible by both business and developer/techie resources. In terms of total volume, the model consisted of approximately 16 diagrams, each describing a different domain concept, with approximately a page of text going along each diagram. In my opinion, this is not an awful lot of documentation compared to the source requirements we inherited which were thousands and thousands of pages long. When creating the domain model, we relied on many of the domain driven design approaches that can be found at http://www.domaindrivendesign.org/.


What exactly is the purpose of modeling?
While going over our models with business stakeholders, the biggest problem that I kept running into was that the client wished to see every single field and every single relationship on every class. They were under the impression that if the model wasn't perfect/complete it wasn't going to be useful. I did my best to explain that models did not need to be 100% perfect/complete in order to be useful, quite the contrary in fact. I'm not sure I did a perfect job in doing this, so I thought I would write up a quick blurb on what I believe are the primary purposes for modeling in the hopes that this will help me to defend my position better on in the future.

I should also point out that Scott ambler is an excellent job of describing modeling principles and best practices on his agile modeling method site

1. Modeling enables Understanding
   
   IMHO this is the most important part of modeling. Modeling enables people to understand what has gone on before their project, and also allows them to get a better understanding of what needs to happen next.
   
   As an example of the "gone on before", take the all to typical situation where a developer team inherits requirements; sometimes of good quality, sometimes of not so good quality. Developers are asked to design or develop based on these requirements, and frequently don't even know where to start. Modeling and diagramming is an excellent way to comb through these artifacts and apply some critical thinking around them so as to make some initial directional setting decisions.
   
   As an example of the "what needs to happen next", the same example also applies. The modeling done by developer type resources can also be used to apply critical thinking around what the data structures will look like, what the interface screens will look like, what services will be required, et cetera.
   
   This modeling exercise is also the best way for these developer resources to actually understand any legacy documentation. Simply asking someone to read a pound of documentation will rarely result in comprehension of that documentation. Giving someone some documentation and asking them to perform a discrete activity will result in exponentially more understanding. Examples include reading requirements to do initial analysis/domain modeling, initial vendor analysis, usability/interface modeling, actual development, et cetera.
   


2. Modeling Enables Communication
   
   Simple models that abstract and structure a large amount of requirements (or any other system information for that matter) are worth their weight in gold. The maxim "a picture is worth a thousand words" is no truer than in the system development space. Being able to illustrate a vast and incredibly complex system with 10 to 20 pictures (or 1 or 2 depending on the level of abstraction) goes a long way toward ensuring that a lot of distributed teams have the same understanding of what the system needs to do.
   

Complete models, be careful what you wish for...

So what does this have to do with my original clients desire to "have every field and every relationship described". You'll note that nowhere in these above two points did I mention "modeling to specify". Creating a specification means detailing every field, detailing every constraint, and every relationship, regardless of whether its fundamental to the point you are trying to make on a specific diagram or model. Creating a "complete" model actually makes it much more difficult to achieve the above two objectives. And here IMHO are the reasons why...

1. Important concepts are obscured: if every relationship, every field, and every business constraint is modeled and presented how can we tell the important ones from the unimportant ones? A model that is too detailed has the unintended consequence of intermingling important concepts with unimportant concepts. The art of actually hiding unimportant information is crucial to creating good models, it's probably harder to create models that only display relevant information than to create models that show everything. But the result is certainly a better model, one that enables understanding of the system across a variety of stakeholders.


2. The model is impossible to maintain: having been on several projects with a model oriented bent, I've had plenty of hands-on experience on the overhead required to maintaining readable and accurate models. In my experience, it is simply impossible to create maintainable models that are 100% precise down to the field level. At best, models are created that are "true when created" and get more and more irrelevant as time goes by. There are various experts out there who claim that modern tools make it easy to keep detailed diagrams and technical code in sync with each other. These experts need more actual hands-on practice with some of these tools, because this notion is still the stuff of ivory tower dreams, do not underestimate the effort involved in maintaining these types of models, it will equal, and often surpass your actual coding effort.


3. Complete models are really just "code-umation”: code-umation” is documentation that approximate the detail level of actual code itself. This type of documentation is no easier to navigate than code, it is certainly harder to change than code, and is not executable or testable. Run, run, run as far away from this approach as you can. It doesn't matter if you're in sourcing, outsourcing, working on a large or small projects, this type of detailed documentation does nothing to help developers do their work, try picking up the phone, having a face-to-face conversation, or any other method to communicate detail of this nature. Better yet, follow some of the excellent implementation patterns found in Kent Beck's excellent text here to ensure that your code is readable and self-documenting.


4. Drudgery versus innovation: creating detailed, field level, specification oriented models is mindnumbing, monotonous work. Good models are the result of critical thinking, innovative solutioning, and thoughtful analysis. These two styles of work are the opposite of each other. The more you focus on low-level details the less you'll focus on figuring out what the right things to do are, if you had the choice of receiving a high quality modeling backed by creative intelligent thinking or a complete model that was a result of monotonous repetitive effort what would you rather have. Don't ask for both, because it's simply not realistic.
   
   So in short, when modeling, focus on what's important, focus on what you need to communicate, and focus modeling on helping you understand what you need to do next. In most cases the simple advice will go a long way in streamlining your modeling efforts in ensuring that your modeling and models stay relevant.

CRC cards-incredibly successful so far

During the last week of December I submitted a post discussing my intention to use CRC modeling techniques on what I would call a "massive" project. I needed a way to collaboratively build an overall domain model for an incredibly complex solution that was currently expressed in a thousand plus pages of linear, non-scenario-based business requirements. I said that I would report on my progress in a later post, and I'm happy to state that CRC modeling has been a fantastic success. While the overall constraints and structure of this project cannot in any way be called "Agile", applying this agile modeling approach across our internal team is providing a fantastic amount of value.

As I stated in my previous post, I initially followed a more traditional modeling approach, which involved a technical team:
1) Reviewing the requirements and creating models
2) Placing white board models into an official UML tool (Rational Software Architect)
3) Holding various workshops with business experts on our team to make sure that the models properly reflected the requirements

Again, as stated in my previous post, this approach had very limited success. The main problems being
1) Business resources were simply not a core part of creating the model. Reviews were simply not that successful/productive

2) UML repositories are really hard to share and maintain on a large team. Especially when rapid change is required.

Note: A lot of agile signatories (Scott Ambler especially) have strong cautions concerning using complex UML case tools; I never understood this before I tried using one with a team of 7 plus resources working on a shared model. It is difficult to overstate the aggravation involved in keeping this model in sync. An excellent resource on case modeling tools can be found on Scott Ambler's site here.

3) The case tool itself was causing the team (myself especially) to model with much more detail than required for this stage of the process. There's a lot of good advice out there on modeling that states that modeling should be at a high level until they can be validated through real implementation. UML diagrams by their very nature can sometimes seem too detailed for this initial analysis.

Within week one of the project restarting after Christmas break, (sometime around the second week of January) I had organized three parallel teams of two technical/modeler types and one business/functional resource. I myself played a floater role of mentor/facilitator/advisor across the three teams. Of course, I only had a week more experience than any of the other teams, but I'm used to doing what I call "just-in-time training". (meaning I train myself just in time to train somebody else)

The technical/modeling resources all had a pretty good understanding of OO and UML, while the business/functional resources had spent the last year and a half collecting 7000 plus business requirements, and thus had a pretty good understanding of what the solution needed to do in order to be successful.

We set up a process that involved spending three hours in the morning conducting modeling sessions using the separate teams mentioned above, then the modelers excluding the functional resources would spend the next half day consolidating the models, removing duplication, creating common structures and making sure that the separate work was combined into a cohesive whole.

Personally, I was personally concerned that CRC cards would not effectively scale, that it would be too hard to organize the cards//we organize them as necessary, that the model would be hard to maintain, and that we would be unable to get the right amount of detail. I was also concerned that the business/functional resources (who have no object oriented training whatsoever) would simply not be able to properly participate.

I have to say that my concerns had been completely unwarranted, every team appears to have had very good success in restating requirements into a CRC format that is now digestible in a matter of minutes, as opposed to hours. Even more important, functional/business resources have all played a vital role in deciphering/explaining business requirements and contributing to the quality/accuracy of our CRC models. I was personally satisfied we are on the right track when I saw specific CRC cards being built by these functional resources. Even more important, the proposed system appears to be a lot simpler than perceived by simply looking at the original requirements. Certainly there still is a lot of complexity, and this will by no means be an easy system to build, but it has become apparent that there is a lot of shared pieces that apply to 70 plus percent of the requirements.
Our client has also been very firm in wanting a Rational Software Architect version of the model. What that means is that we've spent the last week or so entering our CRC cards into this tool. The work involved in ensuring the integrity of the model within the tool, and the level of effort required to make simple changes more than validates the incredible flexibility provided by using simple index cards and particleboards. Changing the contents of a CRC-based model is incredibly easy, incredibly quick, and painless. I'll probably never attempt any serious modeling exercise without CRC cards again. And actually probably little embarrassed that I've never used a CRC before. So two thumbs up IMHO for CRC cards.

Collaboration Responsibility Cards-domain modeling made easy

I recently just started work on what I consider to be a fairly large project with a large number of complex requirements. A massive amount of requirements had already been collected, (over a thousand pages worth). Unfortunately, a large majority of the requirements were not scenario driven, but of the "the system should make sure that..." variety. Extremely hard to follow especially giving this scale.
I was tasked with applying some structure and some modeling driven techniques to help frame the requirements and set the stage for downstream delivery. Needless to say, the environment for this project could be definitely considered less than ideal from an agile point of view. Project management did not want any development to start until requirements were better structured, and we had a better understanding of how different requirements mapped to different technology components.
One of the recommendations I quickly made was to develop a business domain model using package, class, and sequence diagrams. I'd read an excellent book by Eric Evans called Domain Driven Design, which nicely combined business centric modeling, UML, and a pragmatic, agile approach. I was hoping to use the functional resources that were responsible for putting together the business requirements in a business domain expert/reviewer capacity, as well as use a team of more intermediate resources to do the bulk of the modeling effort, with myself providing direction and guidance . We were constrained with using Rational Architect as our modeling tool, as that is what the client owned .
After about a month of modeling efforts, I had to admit I was not entirely pleased with our progress. Given a couple of days to reflect, I believe that the source of much of our inertia was due to the following
1) white board-based models are too hard to change once significant amount of effort has been put into creating the model. Whiteboards, while inclusive, need to be erased eventually, and require way too much redrawing.
2) most UML modeling tools might be a little bit easier to change, but are not collaborative in nature. There is way too much overhead in either printing off the models so that a large number of resources can collaboratively review and modify, or setting up a projector. Any changes require further work with the tool, and the turnaround time between making changes and putting them in the tool take way too long.
3) if business domain experts are not involved in directly creating the model, they do not focus enough on the model to see if they are correct. Our functional resources had several chances to take a look at many of our models before showing them to the client, but it was only until we did a live session with the client where the functional resources were actually asked to help the modeling team defend some of the modeling decisions did it become apparent how wrong our models were.
4) UML and the associated tools, just offer way too much detail. This detail actually can get in the way of initial modeling efforts. Despite my best intentions, I frequently found myself concerned with issues that were just not relevant at this point in the process. I can personally reflect on many situations where I spent too much time concerned with directionality of relationships, the use of inheritance over other more flexible typing mechanisms, and other issues.
CRC cards to the rescue
Now I admit to having heard of CRC cards before, but I've never paid much attention. I've always been happy to use TogetherJ or some other modeling tool for the majority of my diagrams, reviewing my work with other resources, or handing them off as necessary. Whoever, I believe that the the incredible business complexity of this project, the relatively large number of resources on the team, and scope of the proposed application dictates a different approach. I was doing some Internet research on various modeling tools and techniques and came upon CRC cards by accident. It didn't take very much reading for me to realize this approach had fascinating potential. I'm not going to go into too much detail around what CRC modeling is, however in brief CRC modeling involves placing responsibilities (methods and properties for classes) onto index cards (one index card for each class) as well as placing the collaborations necessary (with other classes) to fill these responsibilities. Index cards that represent classes with the lot of collaboration are placed close to each other, index cards that represent classes that implement or extend other classes/interfaces can be placed on top of each other. Potential models can be rapidly re-factored by rearranging these index cards. One can then trace through a set of classes and their responsibilities to see if a model will support specific requirements. One excellent resource concerning CRC modeling can be found here.
I quickly played with some CRC cards for about half a day, and was amazed at the speed I was able to model a large portion of requirements. I believe CRC cards will enable me to be effective on this project for the following reasons.
1) east of use-unlike UML, as well as most UML modeling tools, CRC cards are incredibly easy to learn, and do not require a lot of technical skill or object oriented knowledge to understand.
2) collaborative-CRC cards are incredibly collaborative, many resources (both technical and business oriented) can sit around a large table, and create CRC cards together. The result is a model that will have a very high alignment with true business concept situations and processes.
3) low fidelity-CRC cards natively support a lower precision fidelity then UML modeling notation. This lack of detail is actually a benefit when trying to focus on the nature of responsibility and collaboration, parameter ordering, datatypes, and other details found in UML just get in the way of this kind of modeling.
4) extremely easy to change-models made from CRC cards are very easy to reorganize, modify, and extend. Cards can be very easily ripped up, replaced, and changed.
Once I get back to work from the Christmas holidays I plan to introduce this technique to the modeling team, I would like to use CRC cards until the majority of the model has been completed, then translate the results to a more official modeling tool. I will certainly post the results here.