In his recent blog post Scott Bellware expresses skepticism about using Domain Specific Language tools like Fitness for application that are at the very beginning of developing their model and domain specific language. He argues that for applications with unstable domain model the maintenance overhead is too big.
http://codebetter.com/blogs/scott.bellware/archive/2007/10/28/170346.aspx
The more I deal with severely distributed teams (long way from Vancouver to Siberia!) the more I have come to view Test First and Test Driven Design/Development as a must for success. It is an uncontested fact that the lack of co-location is the largest downside to a distributed agile team, you can view the communication link within an agile team as a water system.
For a co-located team, the communication is a vast pool of knowledge that everyone can dip into whenever they please. As soon as you divide the team up, even if it is from one room to another, you need to put a pipe in place between these pools. The further apart you place the teams, the narrower the pipe becomes.
Now one needs to ask, how best can we fill this pipe? Are there any ways we can shore up the pipe with other means of communication. This is where I see Test First, TDD, executable requirements, Fit, whatever you want to call it but activities that help the team communicate the requirements and validate that the design and development underway meets those requirements without the constant need to fill up the pipe with clarifying questions.
Now pardon my belaboured use of the plumbing analogy (can't help it, I am an engineer!) but after an initial flood of communication clarifying tests and requirements at the beginning of the sprint/iteration/whatever (we need to start to have a unified language for these things ... a blog post for another time), these agreed upon, well understood, and continuously executed tests leave the pipe free to be used to communicate about creative problem solving and more truly profound questions about requirements, not questions like "should the discount be applied AT $50 dollars or AFTER $50".
In our development process, we attempt to create Fit tests for new user stories up front in collaboration with business owners (let’s call this “Fit-driven development” - because we love the “X-driven” paradigms). This has several advantages for developers. First, we learn the language and rules of the business. This is crucial to any attempt at domain-driven design (see?), in which we carry the language and concepts of the business domain deep into our applications. Second, it gets us collaborating with the business owners on a day to day basis beyond the bounds of the SCRUM meeting. This builds knowledge of each other’s worlds and, more importantly, empathy, which is key to a building a positive working relationship. Third, we add to the test coverage in our system – tests that are usually at a level higher than unit tests, like integration or acceptance tests.
Now, to be honest, we don’t always develop Fit-first but when we don’t I often find myself wishing that we had – for example, the time when I discovered that the language I used in my code was woefully inconsistent with the language the business uses, or the time when we implemented a core piece of logic and had to rewrite it when we learned that we had misinterpreted the partially-erased scribble on the whiteboard - shock! As a developer who must admit to rewriting code too frequently, tools, like Fit, that help to get it right the first time are whole-heartedly welcomed.
Fit is a very flexible tool for testing due in large part to the pre-defined fixture types (ActionFixture, ColumnFixture, RowFixture, fitlibrary’s DoFixture, etc.) This flexibility means it’s usually possible to write test tables in a way that is intuitive and easy to read. However, we’ve come across a situation where the test tables we want to write aren’t executable by any of Fit’s built-in fixture types.
The feature we’re working with is a Fit HTML table builder. Our test provides a fixture classname as input and expects a Fit HTML table as output. Here’s an example of what we wanted to test:
Unfortunately, when we ran this test using a ColumnFixture, Fit interpreted the nested table as more parsable cells rather than expected HTML output. And, while there are other ways to structure the test, such as using raw HTML, none of them seem very easy to work with.
What we really need is a new type of fixture, which interprets the table data in the last column as HTML content to be compared with the result of the fixture’s verification method (in the case above, table()).
public class HtmlTableFixture extends ColumnFixture {
@Override
public void check(Parse cell, TypeAdapter adapter) {
String expected = cell.body;
try {
String result = (String) adapter.get();
if (expected.equals(result)) {
right(cell);
} else {
wrong(cell, result);
}
} catch (Exception e) {
exception(cell, e);
}
}
@Override
protected Binding createBinding(int column, Parse heads)
throws Throwable {
Binding binding = super.createBinding(column, heads);
if (column == columnBindings.length - 1) {
binding = new HtmlTableBinding(binding);
}
return binding;
}
private static final class HtmlTableBinding
extends Binding {
private final Binding internalBinding;
private HtmlTableBinding(Binding binding) {
this.internalBinding = binding;
}
@Override
public void doCell(Fixture fixture, Parse cell)
throws Throwable {
Unparse unparse = new Unparse(cell.parts);
String html = unparse.text;
cell.body = html;
internalBinding.doCell(fixture, cell);
}
}
}
public class Unparse {
public String text;
public Unparse(Parse parse) {
text = unparse(parse);
}
private String unparse(Parse parse) {
StringBuffer sb = new StringBuffer();
sb.append(parse.tag);
if (parse.body != null) {
sb.append(parse.body);
}
if (parse.parts != null) {
sb.append(unparse(parse.parts));
}
sb.append(parse.end);
if (parse.more != null) {
sb.append(unparse(parse.more));
}
return sb.toString();
}
}
One thing I’ve learned in software: business people and developers speak a different language. My Stanford friends like to lump people under fuzzies or techies (fuzzies being someone who doesn't know a lot about computers) and this generalization has some merit – at times it can feel like you’re speaking to another species (and buzzword spouting executives are from another planet!)
But I’ve been using a solution called Fit, or the Framework for Integrated Testing, that bridges the gap between fuzzies and techies. In Fit, product managers supplement written word requirements with HTML or Excel Fit tables that express concrete examples. These Fit tables can easily be written by fuzzies in a HTML WYSWIG editor or Microsoft Word or Excel. There is something almost mathematical about writing out examples in this way, forming what domain-driven design philosophers might dub a Ubiquitous Language. These examples are both requirements and tests, saving you a step (and an opportunity for business needs to be lost in translation).
How easy is it? Well let’s go through an example (from the Fit bible, “Fit for Developing Software” by Fit pioneers Rick Mugridge and Ward Cunningham.)
Business rule: A 5% discount is provided whenever the total purchase is greater than $1,000.
Fit test table:
Kinda making sense?
Well, row one names the fixture, or the code that hooks the test into the system under test.
Row two has the headers, which label the two columns. The amount column acts as a given column, which specifies the test inputs (amount = purchase price in the business rule). The discount column acts as a calculated value column, which specifies the expected output of the calculation. The calculated value columns are designated by the () brackets after the name. The rest of the rows are effectively our test cases, trying a variety of inputs into the system.
Once a developer has written a fixture to hook this test into the system, the Fit test can be run, producing a report (see my report below). The calculated value column is colour-coded in the report: green=test success, red=test failure. Failed tests list the expected value and actual value calculated by the system, so you can investigate the discrepancy. In this case, the developers seem to have misinterpreted the requirement to provide the discount when the purchase is greater than $1000.
The value of Fit is not only in more precise communication of requirements, but as an automated acceptance testing framework. With Fit, your requirements can be tested regularly to indicate to developers if new development efforts meet expectations, and if refactoring or new development has broken existing functionality.
Well, that’s all for now. To learn more about Fit, check out http://fit.c2.com/
Why using FIT?
Test Driven development is an Integral part of Agile processes. As part of Test Driven Development FIT is a good choice to be used as an automated test framework for acceptance testing.
Why using CruiseControl?
Usually agile processes suggest that continuous build and testing must be part of the development process. CruiseControl,in a very flexible and configurable manner,accommodates automating and scheduling the build and testing of an under development code.
As the above suggests, CC and FIT are very likely to be used in an agile project and I thought sharing some information on how CC can run FIT tests and display FIT reports can be beneficial. If you know of any better solution please let me know.
Please Note that CC provides a plug-in for displaying FITNESS test results and not FIT test results. If you are working with FITNESS and need to configure CC to show the test results for Fitness tests then see Fitness Reporting posting on gmane.comp.java.cruise-control.user newsgroup for an example.
I assume the reader of this posting already knows how to configure a project on Cruise Control. In case you do not know how to do it you can find more info on the following sites:
-Getting Started with CC document: CruiseControl Getting Started
-11 easy steps to get CC going: CruiseControl - Getting Started in 11 easy steps
Let’s assume the target project is already using FIT. Below I show you how to create an ant target for running your fit tests and also configure CC to use it:
1. Create two properties that are available to your build.xml for FIT input and output folders:
For example:
fit.storytests.dir=src/test/com/sentinel/test/fit/storytests
fit.results.dir=${build.dir}/fitReports
2. Create an ant target in your build.xml for running FIT Tests in your project:
* Add fitlibrary classpath:
<path id="fitlibrary.classpath">
<fileset dir="${libs}" includes="fitlibraryRunner.jar"/>
</path>
* Add fit test classpath:
<path id="fit.test.classpath">
<pathelement location="${build.test.classes.dir}"/>
…
<pathelement location="${build.classes.dir}"/>
<path refid="fitlibrary.classpath"/>
</path>
* Add build targets:
<target name="fit.test.prepare" depends="compile.test,fit.test.clean"
description="Prepare to do fit test"/>
<target name="fit.test.clean">
<delete dir="${fit.results.dir}"/>
</target>
<target name="test.fit" depends="fit.test.prepare" description="runs fit tests">
<echo>Please see the Fit Test Report page for more details on following
results:</echo>
<java classname="fitlibrary.runner.FolderRunner" fork="yes" failonerror="yes">
<classpath refid="fit.test.classpath"/>
<arg value="${fit.storytests.dir}"/>
<arg value="${fit.results.dir}"/>
</java>
</target>
3. To test you changes and run the fit tests run: ant test.fit
4. Create an snapshot of your project under CC and create a Project element in CC config.xml.
5. Create a file called fitLink.jsp and place it under [Cruise Control Home]\webapps\cruisecontrol directory. fileLinks.jsp provides a link to your FitReport folder under artifacts directory. Leave the ouput folder point to fitReports folder under build directory inorder to have the FIT test reorts published under CC artifacts. This is the content of fileLink.jsp:
6. Modify the buildResults.jsp file in the above directory to include fileLink.jsp. To do so include the line below at the bottom of buildresults.jsp:
<jsp:include page="./fitLink.jsp"/>
7. Inlcude the test.fit target in the build.xml file used by this project on CC.
<project name="myProject" default="build" basedir=".">
<target name="build">
<!-- Call the target that does everything -->
<ant antfile="build.xml" target="clean"/>
<ant antfile="build.xml" target="deploy-all"/>
<ant antfile="build.xml" target="test"/>
<ant antfile="build.xml" target="test.fit"/>
</target>
</project>
8. Restart your CC and open CC home page in your browser.
9. Press the build button for your new poject.
10. After the build is completed you should see a summary report of FIT test results on your project build result page in Errors\Warning section. At the bottom of the page you should also see a FIT Test section that provides a link to FIT Reports page.
