Archive for October, 2009

replacer.rb and deleter.rb

Friday, October 30, 2009

From time to time I find myself needing to replace certain strings globally (eg: in a directory tree), or deleting lines based on a match. I find sed extremely annoying and I don’t manage to get around its syntax. So ruby comes in handy!

replacer.rb

exp_search = '<%=launchBean\.getCustomVersion\(\)%>'
exp_replace = '${xsession.info.launchBean.customVersion}'
directory = '**/'
file_mask = '*.jsp'

Dir.glob(directory+file_mask).each do |file|
  File.open(file, 'r+') do |f|
    content = f.read
    if /#{exp_search}/ =~ content
      s = content.gsub(/#{exp_search}/, exp_replace)
      f.rewind
      f.write s
      puts "#{file}: #{content.split(/#{exp_search}/).size-1} matches"
    end
  end
end

deleter.rb

exp_search = 'useBean.*launchBean'
directory = '**/'
file_mask = '*.jsp'

Dir.glob(directory+file_mask).each do |file|
  count = 0
  content = ""
  File.open(file, 'r+') do |f|
    while line = f.gets
      if /#{exp_search}/ =~ line
        count += 1
      else
        content = content << line
      end
    end
  end
  if count > 0
    File.open(file, 'w') do |f|
      f.write content
      puts "#{file}: #{count} lines deleted"
    end
  end
end

These commands are especially useful if you have to refactor a codebase that doesn’t provide direct support with refactoring tools. In this case, I was refactoring a few hundred JSPs of a legacy system.

Credits to RSF & Ruby One-Liners for most of the code & ideas. rcscript seems to be a bit too much for me at this time, I’m perfectly happy modifying replacer.rb and deleter.rb when I need it.

Advertisements

Our opposable thumb makes us all professional mechanics, doesn’t it?!?

Thursday, October 29, 2009
Usage of duct tape

Duct tape on Apollo 17, courtesy of Nasa

After watching the video of Uncle Bob’s Clean Code presentation at NDC 2009 last Friday, I’ve thought a bit about the paradox in which our profession lives, especially in light of the recent debate around testing ignited collaterally once again by Joel Spolsky’s blog post. This  paradox is interesting, because it is multifaceted.

Facet 1: most of the companies prefer to deal with many cheap, sloppy programmers instead of investing in a few very good programmers.

Facet 2: despite all the knowledge about software development is easily available a) on the internet b) in multiple ways and c) at a very convenient price (if not free), most of the developers don’t learn much after the university and don’t take any pride in exercising their profession.

Facet 1 may be a side effect of the whole “IT Doesn’t Matter” phenomenon: if investing in IT doesn’t give you anymore a competitive advantage, then maybe it’s not worth investing in the best developers you can find, but just in “developers”. While I generally agree with Carr, especially with regards to “simple” tasks like Office Automation or building corporate websites, there are still huge areas of inefficiency where IT can play a significant role. Maybe it will not give you a significant advantage over your competitors, but it can still save you a lot of money. I personally know a few examples in the logistics domain, but unfortunately I cannot disclose any details because of existing NDAs. Let’s just say that we are talking about many millions of dollars of savings because of problems that in theory have been solved already at the beginning of the nineties. IT wouldn’t matter, if only it was applied and used consistently!

I have also experienced personally the 10x productivity factor: I have seen sloppy programmers being 10x slower than good programmers (not exceptional programmers, just good ones), whose salary wasn’t even nearly 10x bigger than the former ones (let’s be optimistic, let’s talk about 30-40% more). So, it still seems kind of dumb to resolve to hire mediocre people, especially when you are a medium-large company: you certainly have bigger capacity to absorb inefficiencies, but those numbers are starting to be so big, that unbelievable amounts of money go down the drain just because of this. Having said that, there are not nearly enough good programmers in the world and it seems that HR and head hunters have just given up. Sidenote: I happen to know many good programmers and all of them are having a successful career, so it seems that there is always a need for good programmers, at least in this continent. Good programmers, people who care, please don’t give up!

Is there a cause-effect relationship between 1 and 2? I don’t think so. It seems simply that there is a bit of pressure to level software development discipline. I’m sure that this is not the intent of Joel Spolsky, quite the contrary, but singing an ode to the duct tape programmer, even when the programmer is a good one, doesn’t seem the smartest idea to me. Put duct tape in the hands of an exceptional programmer and he’ll do great things. Put duct tape in the hands of a sloppy programmer and he’ll do an awful job. Have you ever seen a participant to Canada’s Worst Handyman assembling a shelf?

For some reasons, everybody thinks he/she can program, even without the appropriate knowledge or background. After some practice in Excel, everybody can certainly write VBA macros (and do a terrible job anyway), but it’s like saying that because everybody can use a wrench then we are all professional mechanics. I guess it’s all about perception, so a lot of people oversimplify the complexity behind implementing and maintaining a code base for any significantly complicated business problem. I’m not saying that every software programmer needs to be as good as Ward Cunningham, but in my mind there is a clear trend towards embracing more and more people in the industry with less and less capable minds. Even worse than that: a lot of programmers seem simply to not care at all.

What is wrong about letting more people get into software development? Generally speaking, nothing – assuming that the complexity at hand has been reduced over the years, thanks to astonishing advancements in tools and technologies. To some degrees this is true: the things that we can do today would have been virtually impossible twenty years ago, but during the same period of time the industry has just raised the bar, tackling more and more complicated problems. The revolution that started with personal computing isn’t just done yet.

Hence, we need smart people, willing to learn new things every day, constantly looking for ways to improve their work. It’s like having your legs solidly planted in today’s problems, but your eyes are always looking at tomorrow.

Duct tape is an exceptional tool when you need to fix something quickly and for whatever reason you don’t have access to the appropriate tool, knowledge, etc. Maybe sometimes you leave duct tape in place for years (hell, my plastic pipe that conduct warm and humid air from the dryer outside of my house sticks into its place thanks to duct tape), but this is generally not a good solution, capable of lasting for a long time. Sooner or later somebody will have to come and clean the mess that the duct tape expert has left. Two, three times a year I have to replace the duct tape because it simply wears out over time: it’s annoying and I would have fixed it already if I wasn’t renting my house.

There may be specific situations where this is tolerable, but not in much broader terms. And yet, there seems to be a tendency to underestimate the effects over time of such short-sighted decisions in our profession. Is it a tolerance to mediocrity? Are we all getting dumber?

Think about it this way: would you accept to have a sloppy pediatrician taking care of your child’s health? Would you go into the surgery room with a mediocre surgeon? Would you put your life on the line in a judicial trial with an incompetent lawyer? Would you trust an ignorant plumber to do the plumbing of your house? Etc., the examples are endless. We are not talking about NASA-like situations – these are more or less very common problems.

The point is that you would NOT. And yet, many managers and entrepreneurs, even when they are technical, do with software development what they would not do in their personal lives. Is it because it’s not their personal wealth or beloved ones at stake?

I’ve always found that the best managers and entrepreneurs are the ones which know the difference between cost and investment. There is no return from a cost, but there better be from investments. If you are on the upper side of the food chain, which return do you expect from the developers working for you? How can you maximize it? But if you are on the lower side: how can you provide value to your company? How can you be always on top of your profession?

Running JRuby with Terracotta

Tuesday, October 27, 2009

Lately I have been doing some interesting work integrating Terracotta in our application. We use Terracotta in a very limited scenario: as a tool to implement asynchronous DB replication on PostgreSQL (unfortunately the existing tools out there are not suitable for the kind of usage that we do of PostgreSQL – anyways, I’ll talk more in details about this in further posts).
At a certain point in time we decided to drive some java code with JRuby, in order to process information like log files more efficiently. The java code needed to integrate with Terracotta, so we have had to modify jruby scripts in the following way (eg.: jruby_tc.bat on Windows):

@echo off
rem ---------------------------------------------------------------------------
rem jruby.bat - Start Script for the JRuby Interpreter
rem
rem for info on environment variables, see internal batch script _jrubyvars.bat

setlocal

rem Terracotta
set TC_INSTALL_DIR="C:\Program Files\terracotta\terracotta-3.0.1"
set TC_CONFIG_PATH="C:\whatever\tc-config.xml"
call %TC_INSTALL_DIR%\bin\dso-env.bat -q
set JAVA_OPTS=%JAVA_OPTS% %TC_JAVA_OPTS%

rem Sometimes, when jruby.bat is being invoked from another BAT file,
rem %~dp0 is incorrect and points to the current dir, not to JRuby's bin dir,
rem so we look on the PATH in such cases.
IF EXIST "%~dp0_jrubyvars.bat" (set FULL_PATH=%~dp0) ELSE (set FULL_PATH=%~dp$PATH:0)

call "%FULL_PATH%_jrubyvars.bat" %*

if %JRUBY_BAT_ERROR%==0 "%_STARTJAVA%" %_VM_OPTS% -Xbootclasspath/a:"%JRUBY_CP%" -classpath ^
   "%CP%;%CLASSPATH%" -Djruby.home="%JRUBY_HOME%" -Djruby.lib="%JRUBY_HOME%\lib" ^
   -Djruby.shell="cmd.exe" -Djruby.script=jruby.bat org.jruby.Main %JRUBY_OPTS% %_RUBY_OPTS%
set E=%ERRORLEVEL%

call "%FULL_PATH%_jrubycleanup"

rem 1. exit must be on the same line in order to see local %E% variable!
rem 2. we must use cmd /c in order for the exit code properly returned!
rem    See JRUBY-2094 for more details.
endlocal & cmd /d /c exit /b %E%

The relevant differences are in bold in the batch file above. I’ve called the batch jruby_tc.bat in order to differentiate it from usual jruby.bat. As you can see these changes are for Terracotta 3.0.1 but they should work just the same for Terracotta 3.1