Acceptance Test Patterns

Recently I have spent some time writing many acceptance tests (AT) in a row for a project while working at Quinary. Usually I spend not too much time with the customer, helping her writing a few AT for each iteration. It can be 30′ or 1 hour a week on average, rarely more than that (and only at the beginning of the project or a new release). Working that way you end up writing AT in little bursts.

This time it was different. We were using Fitnesse like a tool for writing executable specifications (more on this in further posts). It is not interesting here to discuss why we were doing that, but in the end it meant writing tens of AT without even a single line of code. The project itself was just a brainstorming about a possible product, therefore there was no intention (yet) to really code anything. But writing executable specs helped understanding the different areas and functionalities that we eventually need to address.

Whatever the reason, I now have many AT, some of which are really similar to each other. Even more: some of those tests are slightly equal, they look to be different instances of the same pattern. For example: whenever we have forms used to enter new entities (ala DDD), there is always a test that tries to insert duplicated data, check that it fails and check the actual error.

This is just a simple example, but it already contains all the elements that are typical of a pattern language, expressed in the vocabulary of functional elements.

While I’m pretty sure that this concept isn’t really new, it seems that it is different from what specified for example with the BuildOperateCheck pattern: this pattern works at the Fit level, and it is so general to comprise almost any Fit test that doesn’t follow the other well-known pattern represented by flow-style actions in FitLibrary.

To put it simpler, what I’m trying to say is that there seems to be emerging patterns at the functional level, working at a lower level than general-purpose Fit/Fitnesse patterns. It may also be that specific domains will have their own (even more) concrete patterns.

This opens interesting possibilities.

5 Responses to “Acceptance Test Patterns”

  1. Marco Says:

    If we consider UAT as a way to communicate instead of a way to test (like TDD is a way to design instead of a way to test) then IMHO the emergence of a domain language is unavoidable. On the other hand I think it would be hard to re-use that domain language with a different customer (even if the domain is the same) because the dictionary in use comes from the people involved in the project.

    Maybe a high-level example of what you have in mind speaking of patterns at the functional level could help me focus on topic.

  2. Matteo Says:

    Examples, please!

    Could you show a simplified example of the kind of tests you are talking about?

  3. theboxx Says:

    Fair point, Marco. But let me explain, hopefully a little bit better.

    Suppose that your application has some user interface through which the user is allowed to enter new entities into the system. Suppose also that those entities are uniquely identified by a few key attributes among all that can be specified. This is pretty common, especially in systems where users may register themselves or enter new data that play a significant role in business rules.

    For example, the customer may easily depict a scenario where a new user may register herself, choose her own login name, but that login name got to be unique. Or the tuple composed by first name, last name, birth date, birth place has to be unique (bad choice anyway, usually true but not necessarily always).

    Whatever the criteria, the customer may want to check that the system is compliant with this little business rule. Typically she will write an AT like the following.

    !|UserRegistration|

    |”insert user”|vam|”first name”|Max|”second name”|Van|”email”|maxvan@test.org|”city”|Milan|”country”|Italy|
    |check|”users”|1|
    |”insert user”|vam|”first name”|Pasquale|”second name”|Gargiulo|”email”|pcg@test.org|”city”|Nice|”country”|France|
    |check|”users”|1|
    |check|”message”|Sorry, user “vam” is already assigned, please choose another login name. The following are still free: vam1, gap, pgargiulo|
    |”insert user”|gap|”first name”|Pasquale|”second name”|Gargiulo|”email”|pcg@test.org|”city”|Nice|”country”|France|
    |check|”users”|2|

    |”user list”|
    |”user”|”first name”|”second name”|”email”|”city”|”country”|
    |vam|Max|Van|maxvan@test.org|Milan|Italy|
    |gap|Pasquale|Gargiulo|pcg@test.org|Nice|France|

    Disclaimer: I’m not talking here about testing through the UI ==> these are business rules that need to be verified apart from UI logic.

    Doesn’t make more sense?

    I tend to think that reusing the language among customers and/or projects is actually harder if the AT is more concrete, that is to say if it is expressed more strictly in domain terms. But I have no proof yet.

  4. Olorin Says:

    [quote]
    I tend to think that reusing the language among customers and/or projects is actually harder if the AT is more concrete, that is to say if it is expressed more strictly in domain terms. But I have no proof yet.
    [/quote]

    So… hmm… is this a case of generality vs specificity ?

    The proof might simply be that, when the tests are expressed in concrete terms (specific to a given project), the chance of re-using them as they are goes down because there are less chance of another product using the same terms.

    Say that, instead of writing the tests with ‘User Name’ and ‘Password’ we wrote them in terms of ‘Entity Name’ and ‘Entity Attribute Name’ and ‘Entity Attribute Value’. The ‘Entity’ language is more likely to be used in other pdocuts, hence more re-usable.

    However, you will find that people prefer something more concrete (and -alas!- specific).
    There might.. just might.. be some correlation between the partition ‘people that prefer generality vs specificity’ and ‘business people vs software engineers’, but I don’t have the time to try and prove that one🙂

    A similar example, I think, is the type of fixture people prefer to see.

    I *suspect* software engineers tend to quickly get to action fixtures (because they can be reused🙂, but business people like column fixtures better.

    For instance, your Userregistration fixture above is an action fixture (or, at least, it looks like one). But the business people I work with would make me change it into a column fixture:

    !|User Registration|
    |User Id|First Name|Last Name|Email|City|Country|
    |vam|Max|Van|someone@somewhere.org|Milan|Italy|

    Has anyone else encountered this phenomenon ?

    F.O.R.

  5. theboxx Says:

    Olorin,

    I tend to agree with you, I think it could be the case you describe. I’m quite convinced that the more concrete the language, the less reusable the fixtures, especially out of their context.

    But I also think that there is more. And I would never write a fixture in terms of “Entity name”, etc.

    Regarding fixture choice I like the point made by Bill Wake regarding Procedural vs Declarative Tests ( http://xp123.com/xplor/xp0503/index.shtml ) ==> from my point of view they are equivalent and they don’t influence the pattern iself.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s


%d bloggers like this: