Posts Tagged ‘java’

Running JRuby with Terracotta

Tuesday, October 27, 2009

Lately I have been doing some interesting work integrating Terracotta in our application. We use Terracotta in a very limited scenario: as a tool to implement asynchronous DB replication on PostgreSQL (unfortunately the existing tools out there are not suitable for the kind of usage that we do of PostgreSQL – anyways, I’ll talk more in details about this in further posts).
At a certain point in time we decided to drive some java code with JRuby, in order to process information like log files more efficiently. The java code needed to integrate with Terracotta, so we have had to modify jruby scripts in the following way (eg.: jruby_tc.bat on Windows):

@echo off
rem ---------------------------------------------------------------------------
rem jruby.bat - Start Script for the JRuby Interpreter
rem for info on environment variables, see internal batch script _jrubyvars.bat


rem Terracotta
set TC_INSTALL_DIR="C:\Program Files\terracotta\terracotta-3.0.1"
set TC_CONFIG_PATH="C:\whatever\tc-config.xml"
call %TC_INSTALL_DIR%\bin\dso-env.bat -q

rem Sometimes, when jruby.bat is being invoked from another BAT file,
rem %~dp0 is incorrect and points to the current dir, not to JRuby's bin dir,
rem so we look on the PATH in such cases.
IF EXIST "%~dp0_jrubyvars.bat" (set FULL_PATH=%~dp0) ELSE (set FULL_PATH=%~dp$PATH:0)

call "%FULL_PATH%_jrubyvars.bat" %*

if %JRUBY_BAT_ERROR%==0 "%_STARTJAVA%" %_VM_OPTS% -Xbootclasspath/a:"%JRUBY_CP%" -classpath ^
   "%CP%;%CLASSPATH%" -Djruby.home="%JRUBY_HOME%" -Djruby.lib="%JRUBY_HOME%\lib" ^"cmd.exe" -Djruby.script=jruby.bat org.jruby.Main %JRUBY_OPTS% %_RUBY_OPTS%

call "%FULL_PATH%_jrubycleanup"

rem 1. exit must be on the same line in order to see local %E% variable!
rem 2. we must use cmd /c in order for the exit code properly returned!
rem    See JRUBY-2094 for more details.
endlocal & cmd /d /c exit /b %E%

The relevant differences are in bold in the batch file above. I've called the batch jruby_tc.bat in order to differentiate it from usual jruby.bat. As you can see these changes are for Terracotta 3.0.1 but they should work just the same for Terracotta 3.1

If it’s for test then it has to be good

Wednesday, March 25, 2009

You are dealing with some old code: it’s not too bad, you have definitely seen much worse. There’s a class within a home-grown framework that doesn’t allow you to test some application code. You spend some time understanding how to change things to make it more testable. After a while you come up with the idea of adding a method to that class solely for testing purposes – it’s not a fantastic thing, but the alternatives are frightening and this will make a lot of code much more testable. Because you are a good person you want to make it very clear that that code is intended to be used for testing only, not in real life:

public void setMickeyMouseForTest(Class cheeseClass,
                                  String cheeseName, Mouse mouse) {
    cheeseClass = resolveAssociatedClass(cheeseClass);

    if (getLocalFactory().getCheeseName(cheeseClass) != null)
        throw new RuntimeException("Use standard setMickeyMouse instead");

    mouses.put(cheeseName, mouse);

This method is similar to the real thing:

public void setMickeyMouse(Class cheeseClass, Mouse mouse);

Method setMickeyMouse determines cheeseName by calling getLocalFactory().getCheeseName(cheeseClass), while method setMickeyMouseForTest assumes that cheeseName is not set (even stronger: if it is not null it throws an exception) => instead, it uses the parameter provided in the call.
In the end, the difference between the two lies in the way a certain condition is handled in the code: the method written for testing purposes is capable of dealing with an exceptional condition differently. You’ve created that method exactly to simplify the amount of setup required for tests, so that it doesn’t matter if getLocalFactory().getCheeseName(cheeseClass) is not capable of providing a cheeseName.

So, the code is committed, lots of tests are now possible, the team celebrates your genius.

Some weeks later you’re pair programming with a colleague. You’re still dealing with the same framework, but this time it’s a different corner-case. After lots of mumbling, back and forth, debugging sessions, you superficially look at the signature of that method, obviously without remembering what you have done a month before. You just vaguely remember that it was capable of dealing in a better way with a corner-case. And the method is called “…ForTest” so it has to be good, in the end we are testing indeed.

It doesn’t work. An exception is thrown. The exception hints to “Use standard setMickeyMouse instead…“. Oh gosh, let’s take a look: ops, I know what it is, we’ve forgotten to set cheeseName for cheeseClass in the local factory. Let’s do that.

<type> <type> <type> <type> <type> <type> <type> <type> <type> <type> <type> <type> <type> <type> <type> <type>

Damn, it’s still not working. What is it happening?!?

<debug> <debug> <debug> <debug> <debug> <debug> <debug> <debug> <debug> <debug> <debug> <debug> <debug> <debug> <debug> <debug> <debug> <debug> <debug> <debug> <debug> <debug> <debug> <debug> <debug> <debug> <debug> <debug> <debug> <debug> <debug> <debug> <debug> <debug> <debug> <debug> <debug> <debug> <debug> <debug> <debug> <debug>

It’s very strange.

getLocalFactory().getCheeseName(cheeseClass) is not null, so how can it possibly say that it is null instead?!?

<more debug> <more debug> <more debug> <more debug> <more debug> <more debug> <more debug> <more debug> <more debug> <more debug> <more debug> <more debug> <more debug> <more debug> <more debug> <more debug> <more debug> <more debug> <more debug> <more debug> <more debug> <more debug> <more debug> <more debug> <more debug> <more debug> <more debug> <more debug> <more debug> <more debug> <more debug> <more debug> <more debug> <more debug> <more debug> <more debug> <more debug> <more debug> <more debug> <more debug> <more debug> <more debug> <more debug> <more debug> <more debug> <more debug> <more debug> <more debug> <more debug> <more debug> <more debug> <more debug> <more debug> <more debug> <more debug> <more debug> <more debug> <more debug> <more debug> <more debug> <more debug> <more debug> <more debug> <more debug> <more debug> <more debug>

Oh my gosh! The exception is actually thrown WHEN getLocalFactory().getCheeseName(cheeseClass) is not null, not viceversa!

What happened is that you and your colleague just wasted 20′ because you misread “!= null” (reading “== null” instead). What does that tell you?

It told me a few things:

  • Managing null values is always evil and error prone. At the very least, checking for a null disrupts the business logic and it makes the code less readable, ultimately more likely to break.
  • It’s too easy to misread a check based on != vs ==. Not only that: the fact that an exception is being thrown if the expression evaluates to true hints that the check is to verify if the value is null, because that is what you would typically do when you are adding a guard to your code. Which brings me to the next point.
  • Moreover, the intent of the code is not clear. It’s not enough to call a method “somethingMeaningfulForTest“, that is the suffix “ForTest” is misleading if the method is not truly applicable for every test. This is even reinforced by the fact that the exception message is not enough communicative.
  • As usual, test code has to follow the same standard of quality as application code, which means readability, expressiveness, simplicity, lack of duplication. This is particularly important for accessory code that is written to support tests.

So, in the end, how should that method be called? What about:

public void setMickeyMouseForTestWhenThereIsNoAvailableCheeseName(
    Class cheeseClass, String cheeseName, Mouse mouse);

The underlying precondition is that method getLocalFactory().getCheeseName(cheeseClass) returns null for this code to execute properly. This should be made clearer with a checked exception being thrown by the method, or, even better, by a custom annotation that marks the method with the defined precondition. In Java land this doesn’t go very much beyond documentation purposes for a variety of reasons:

  • First of all there are limitations on the attributes that an Annotation can exhibit. Specifically, only String, Class and primitive types are allowed, not even object version of scalar type like Integer can be used.
  • Second and foremost, annotations can be queried to provide metadata about code, but they don’t really affect code execution. An annotation is useful only to the degree it is recognized by the runner executing some code. There is in the end no native and effective support for Design By Contract in Java and there will probably never be. A useful result can be achieved by combining together AOP with Annotations, but I have mixed feelings about it => in the end I always have the impression that the mumbo jumbo typical of the supporting infrastructure it’s there just to circumvent a limitation that should be inherently addressed by the language itself.

Or shouldn’t it?

Using functors to handle test connections

Thursday, October 9, 2008

A few days ago I was working on some functional tests that have the following structure:

      public void testSomething() throws Exception {
        // 1. istantiate the system
        // 2. interact with it through the UI
        // 3. assert the state of the system by examining the DB

        // 4. interact a bit more through the UI
        // 5. assert again the state of the system by examining the DB

        // etc.

Some notes about them:

  • Tests are implemented with JUnit 3.8.2, but this is irrilevant to what will follow.
  • The application is a webapp implemented with a proprietary web framework.
  • Those tests are implemented running the servlet engine in process through ServletUnit.
  • The DB is interfaced through a proprietary framework and it is accessed behind a mock JDBC driver that works translating SQL statements in memory.
  • The state of the system is examined by fetching either an Entity or EntityList, abstract classes that represent respectively a single record or a collection of records in a table.
  • Methods connectDB() and relinquishDB() are used to manage the test connection, which needs to be different than the one used by the application for structural reasons.

All the tests that follow the above structure inherit from an abstract TestCase which implements its own setUp and tearDown, treated like template methods whose behaviour can be refined further (for example, choosing to install a different mock DB). connectDB() and relinquishDB() use connection parameters that are potentially established differently for every test, but in reality they are seamlessly copied over from test to test.

While this structure makes sense, I thought that repeating connectDB() and relinquishDB() several times in the test is extremely annoying. It would be nice if we can wrap some logic into a block that takes care of the repetition for us: calling connectDB() first, executing our code and then calling relinquishDB() in the end. If I was programming in ruby, that would have been extremely easy to implement with proper blocks, wouldn’t it be? Well, in java is not that difficult either, if you tolerate the syntactic noise of course. Let’s see how.

To start, we can define a Functor interface that can be used to implement a block without parameters:

public interface Functor<T> {
    T exec();

Now we need to wrap a bit of logic around it that takes care of connecting the DB and relinquishing it afterwards:

public class Fetcher {
    public static <T> T queryStateFor(Functor<T> f) {
        T result = f.exec();
        return result;
    private static void connectDB() {
        // ...
    private static void relinquishDB() {
        // ...

The actual implementations of Fetcher.connectDB() and Fetcher.relinquishDB() are fairly trivial and the methods are in reality inlined: I’ve written them in a Compose Method style just to make them easier to understand and to trace back Fetcher structure to the test structure above.

The Fetcher can be used in the following way (assuming that it is imported statically in the test case):

    public void testSomething() throws Exception {
        // 1. istantiate the system
        // 2. interact with it through the UI
        final PurchaseOrder po = //...
        PurchaseOrderStatus status = queryStateFor(new Functor<PurchaseOrderStatus>() {
            public PurchaseOrderStatus exec() {
                return po.getLastStateChangeStatus();

        // etc.

As you can see, this refactoring is not about saving keystrokes.

A developer was asking me which kind of advantage this style achieves over the other. In general, I think that functors are a powerful tool, often underestimated in statically typed languages like java. But these are the reasons why I think they are particularly relevant in the context of queries and tests:

  1. Avoid repeating connectDB(); … relinquishDB(); without the introduction of methods that would have to be rewritten for every test.
  2. Ensure that queries for checking purposes correctly connect to the proper DB and relinquish it in the end.
  3. Make the test more readable, without hiding relevant details (connecting and relinquishing the DB are irrelevant details to the test, only the query matters in the eyes of the reader / maintainer).
  4. In general, make the test less dependant upon lower level details whose relevance is marginal.
  5. Using a functor the query can be executed as a generified method, hence it definitely scores better in terms of type safety.

I’ve also thought about any disadvantage:

  1. The syntax looks a bit odd for the java world, but it’s not entirely a stranger.
  2. Current functor implementation can handle only one result per call, but it can certainly be extended and in general it doesn’t hurt significantly calling several functors in a row.

And you? What do you think about this way of handling a test connection to the DB?

PS: While I think that this style provides a better alternative to repeat calls to connectDB() and relinquishDB() all over the code, there’s actually a simpler way to solve the problem ==> the tests can be provided with their own connection that has nothing to deal with the connection handled by application code running in the servlet engine (credits to Mark for recognizing it). Nevertheless, functors are a powerful tool typically neglected in the java world and they definitely deserve more attention in situations like this one.


Get every new post delivered to your Inbox.

Join 258 other followers