« Impressions from JAOO 2008 (2) | Main | Wednesday’s comments winners »
Impressions from JAOO 2008 (2 ½)
By Jan Reher | October 2, 2008
By Jan Reher, Senior Developer, Systematic Software Engineering A/S. jan.reher@systematic.com.
Tuesday, September 30, 2008 (continued)
Peter Zimmerer: Software Architecture and Testing
The talk concerned the interface between the test field of knowledge and the architecture field of knowledge. Or: What every architect should know about test, and how architects’ involvement is essential for successful testing.
Historically, testing has moved from focusing on detection of failures, over prevention of failures, to today’s more comprehensive focus on providing information about the quality of the product. Detection and prevention of faults are still very important activities; only they are not sufficient.
If you choose—conciously or not—to perform little or no testing, you save a lot of money initially. However, experience has shown again and again that as the release date approaches, the cost of testing rises dramatically. And, worse, you are shipping a faulty product, and those faults will show up in production, driving your costs even higher.
So you need to invest in testing. The return will not show up until your project is well under way but don’t be tempted to skimp on testing. Produce a test strategy and test plan based on your business goals and priorities. Focus on risks, especially those related to architecture. And this is where the architect first enters the game, because—by definition—he knows the strengths and weaknesses of the architecture better than the test manager.
Testing the product in itself is one thing but how do you test an architecture? Given the requirements, the architect should think about how to test them before he thinks about how to realise them. This rule is usually applied at the unit and unit testing level and is part of the test-driven development viewpoint. It fosters good design if not applied indiscriminately. But it also applies at the architecture level, and especially for non-functional requirements, which are generally difficult to realise, and also difficult to test.
Architecture testing happens mostly at the integration level. Note that while stub components are a handy tool for the architect who has to construct the complete system incrementally, they make integration tests difficult and, at worst, meaningless. So the integration sequence must be driven not only by architectural and construction needs but also by test needs.
When it comes to regression testing, the architect must again help the test manager. The test manager knows which test cases must be executed. The architect must supplement this with his knowledge about the product’s weak spots.
In my experience, many other software team roles that architects could do with a lot more insight into what test is for and how it is done. Peter did not try to tell everyone what they should know about test but only what the architect should know. I like that focus.
Eoin Woods: The Top Ten Architecture Mistakes
Yet another experience report from a person who is actually out there solving real world problems. JAOO has a healthy percentage of this kind of presentations. This is good.
Eoin claimed to have many of these mistakes not just once but several times. His talk was lively, engaging and fun. And very serious, too.
Often the solution to a mistake is simple and obvious. Trouble is, the implementation is not. This reminds me of tightrope walking: The mistake is to fall off the rope. The solution is to keep your balance. Simple, really.
I don’t think we were told whether this was a prioritised list, so I will assume it wasn’t, and that the mistakes are a bit like the seven deadly sins—i.e. deadly all of them.
Scoping woes: Functional scope creep is difficult to avoid, non-functional more so. The customer says he wants 24/7 but does he really need it? And is he really willing to pay for that last extra nine? As the architect, you need to focus ruthlessly on the problem which the system is supposed to solve. Keep asking questions like “how will this increase your productivity”
Not casting your net widely: Who are the stakeholders? They are not just the users. The stakeholders are all those people whose cooperation you are going to need. Compliance people can be thought of as negative stakeholders, service and maintenance people too: Basically, they would like your system to be as small as possible. So make a list of stakeholders, and talk to them.
Focusing on function: Users want function, and talk about function. They tend to take non-functional qualities for granted. Do not forget the non-functional stuff. Functionality is “easy”, -abilities are not. So work your way through the -ability list. You cannot cover them all in depth, and you cannot achieve them all, because they are in fundamental conflict with each other. This is about making trade-offs.
Boxes and lines descriptions: Different people (stakeholders) care about different things, and one big diagram won’t serve all. So use a well-define notation and present several views. Use several pictures to present each view. Use UML, but keep it simple and tidy, and add conventions and stereotypes. Be accurate, also at high levels of abstraction. Abstract does not mean imprecise.
Forgetting it needs to be built: Beware of choosing new and achingly trendy technologies for your system. Every choice entails risks, but new technologies come with their special kind of risks. Make sure the service people will be able to run it. Can you justify choosing a new technology?
Lack of platform precision: There are stacks of platform dependencies out there. Versioning nightmares lurk in every corner. You need to specify exactly what the platforms are. It is frequently better to comply with what is already in place in the execution environment than inventing new needs.
Performance assumptions: Everyone cares about performance. Testing and predicting performance is difficult. Frequently, we only manage to test small and limited scenarios. Simply multiplying these up realistic scales won’t work. You must assume nothing. Be paranoid. Test it. There is no easy way out. But ignoring the problem is worse.
Do-it-yourself security: Security is difficult. Do not assume it is not. Reuse existing infrastructure but remember that this has its costs, too.
Lack of disaster recovery: This is expensive, and complicated. Do not fall into the trap of thinking “This is no time to worry about things that might never happen”. But if you start thinking about this problem early on, it is not that expensive. You need to practice, and then practice some more.
No backup plan: You will need this when you deploy the next new version of the system. Your upgrade procedure must have a reverse gear.
Awareness helps. Here’s some.
October 2nd, 2008 at 11:38 pm
[...] lorrie . Excerpt: The talk concerned the interface between the test field of knowledge and the architecture field of knowledge. Or: What every architect should know about test, and how architects’ involvement is essential for successful testing. … [...]