Strangeloop 2012

Interesting two days so far. My main track of attendance has been the Data sessions - and a varied menu it has been. Firstly, let me say the Opening conference speaker, Stonebraker, may be right about a few things but he misses some big points.

Seeing as he is from VoltDB, he has to pay hommage to their architecture and implementation but I don't think one should do it to the point of damaging one's credibility. To be certain, his critique of NoSQL being non-performant for high volume transactions is technically correct, but high volume transactions are not the reason you implement NoSQL.

I am basically the same age as Stonebraker and his recollection of history is as equally curious as his understanding of JDBC drivers and their speed, but that's another story.

During the 1970's the data storage war was waged between two or three vendors, but mostly everyone competed against IBM's ISAM and then they're VSAM data storage or you used IBM's IMS DB/DC framework...unless you used IDMS, which is still in use to day as far as I know. Oracle was still fermenting in one of the Beltway bandits in government work.

More to come...

And in other news...

I was asked a question about classpath and it wasn't in my short term memory...so I had to go searching again...turns out this blog has some interesting information.

Hadoop

So I am using Hadoop Pro to review Hadoop and I can't say I recommend the book. It's not that it's bad, but I can't see where it really helps me get a Hadoop cluster up and running locally.

In fairness, I've been installing this on Mac OS X Lion and Hadoop is really twitchy when it comes to JAVA_HOME. Java is optional with Lion and when it is installed, the normal/Library/Java/Home doesn't work any more. I had to go digging around and put the following into my .bash_profile:

export JAVA_EXEC_HOME="/System/Library/Frameworks/JavaVM.framework/Versions/Current/Commands"

$JAVA_EXEC_HOME/java_home > .java_home.file

export JAVA_HOME="$(less .java_home.file)"

That solved the JAVA_HOME issue. I went to do the maven build of the 0.23 release but alas, the tests failed. Violating my own rule, I included -DskipTests to get a jar file without passing the tests, but still no joy. Sigh.

So...I downloaded the "stable" version - 0.20.203 and of course, this one was built with ANT. So after grepping the targets I ran the jar target and naturally, they used Ivy for the dependencies...so after downloading the Interwebs (sic) again...it finally built.

I ran some of the examples and so far, so good. I am going to move onto building a Hadoop cluster, but seriously, I think I'm going to go build it on my Ubuntu box. Lion is still giving me fits...

Hmm. I didn't realize I was two whole releases behind on my Ubuntu box. I thought I had already done 10.x. So...I started the upgrade and behold, I was getting an error that indicated that 11.04 was already installed...blah, blah, blah. Changed sources.list to comment out 11.04 and that went well...installed 11.10 and KABOOM! It's asking for my user login and password and it's bombing out.

I guess I'll have to make a flash bootable Ubuntu 11.10 drive and totally reinstall from there...I may never get Hadoop installed on this box...

Spring Roo - What I've seen so far

I refactored the RabbitMQ tutorial project to use Gradle, Core Spring and NoXML - meaning I used annotations instead of the ubiquitous XML ApplicationContext file and it worked just fine. I now have a nice base of code to work from should I need it.

I decided to continue my Spring Roo excursion by coding the bitboard section of a chess project I'm working on for the Tulsa Java Users Group.

Some observations: I don't seem to be able to add ArrayLists easily using the Roo command line. So far I have created the fields using the Set and then refactoring the code inside of the SpringSource Tool Suite. I was concerned about the .aj files automagically created by Roo, but after making the change I opened them up and there seemed to be no issue.

Also, it seems that while Roo does do quite a bit of scaffolding for you, there is quite a bit of work you must do on your own - don't get me wrong - I appreciate not having to write all the AOP code, but still I feel something empty about the tool. For instance, I would like to use annotations on the Components, etc., but I don't see any support for that yet. It may be there, I just haven't found it.

Writing new methods is definitely something you must do in the IDE, but no big deal.

Maven is the default build tool and I think there is Gradle support in the future releases, but for now I suggest visiting this page for ideas and tools for converting your project to Gradle.

I haven't found the commands to undo a command just executed from the Roo shell - I'll dig through the doc for an answer but for now I just change it in STS if I make a mistake.

So far, Roo is fun and easy to use. I'm looking forward to really exercising it on the rest of the project.

P. S. Now this is interesting. I had an error in SpringSource Tool Suite (which is essentially Eclipse with the Spring IDE), I corrected it and the annoying little red x was still showing up. F5- Refresh. Still there. Close. Open. Still there.

So I went to the Error Console, clicked on the error, thinking it would take me to the line of code in error but instead took me to a perfectly fine line of code.

Frustrated and disgusted, I deleted the error message out of the Error Console and Voila! No more annoying little x. Time to file a bug report...

Modding DdlUtils to automagically configure SQLFire instances

After digging through the source code for DdlUtils, I was able to figure out how to mod the code to add the SQLFire parameters to the DDL that creates the SQLFire database and tables.

By intercepting the statements in the SQLBuilder.java class, I was able to write a routine that looked for an sqlfire.parameters file and add any SQLFire configuration parameters to the DDL. I have tested the write through parameter (AsyncEventListener) by running the ant task to create the db and then immediately dumped the DDL to an SQL file to examine the structure. Qu'pla!

Next I'll run the modded code with a whole host of parameters for a non trivial database with a number of tables. I've even modded the code to change only a single table if I want by identifying the table in question. Otherwise, the parameters will apply to all the tables.

This is sweet. Now I can convert databases quickly by creating a simple parameter file. No more hacking SQL to get it done, which is not a problem if you're only dealing with a handful of tables, but if you have hundreds of tables, then this is a real timesaver. I attached the code below in case you are interested.

L8r.

Code. Disclaimer: I do not warrant, guarantee or in any way suggest this code is covered by any other agreement other than it does what I want it to do. Use at your own risk.

Homebrew and RabbitMQ - Mac OS X install solved...mostly

Scott, a coworker at Springsource, read about my troubles with Mac OS X and RabbitMQ, so he recommended I try Homebrew, the MacPorts alternative.

It turned out quite well for my Snow Leopard machine and everything works great, but the Lion install is...well, complicated.

It seems there are some issues with libcurl so now I have to try and figure that one out.

Right now I'm putting the finishing touches to DdlUtils so that I can have it automagically append the SQLFire parameters and then test it.

I'm thinking I want to convert the RabbitMQ tutorial to use core Spring and then after that I wanted to finish up my Spring Roo project soooooo...if it sounds like I'm not real interested in solving the Lion problem...

On the other hand, using Homebrew was a big success for the Snow Leopard machine. Go to the install page for the Homebrew install instructions, and then install Erlang and RabbitMQ:

$ brew install erlang

$ brew install rabbitmq

Or just follow the instructions on this page.

L8r.

Setting up a RabbitMQ project using Gradle

Journal Notes:

1. Download Gradle and install. I have the STS plugin but I want to know how to do this without all the magic underneath.
2. Set up a Java project. Right now I include the rabbitmq-client.jar when setting up the project. Later we'll change this to use gradle dependency.
3. Right click the project name, select Configure>Convert to Gradle project.
4. Go into the top level of the project directory and issue line command $ touch gradle.build.
5. Open up the gradle.build file and add the follow code:

apply plugin: 'java'

task "create-dirs" << {

sourceSets.all*.java.srcDirs*.each { it.mkdirs() }

sourceSets.all*.resources.srcDirs*.each { it.mkdirs() }

}

From here I ran $ gradle create-dirs. A lot of useful information can be found at the Gradle Cookbook site so if you run into issues this page helps. After running this line command, I refresh the project view in STS and the project structure is all set up. In fact, it is exactly the same as a Maven project structure, but I guess most folks know that already.

Underneath src/main/java I add the package I want - in this case com.springsource.rabbitmq.

Since the Gradle java plugin already has a number of built in tasks, I think I'm pretty much done with this file for now. Onto the code. I'm going to use the tutorial code from the RabbitMQ site, so I go to the command line and download the source code using the instructions on this page.

Since the rabbitmq-client.jar file is in the Referenced Libraries, all my code resolves the imports for the rabbitmq-client code, but I want to use Gradle (well, really Gradle uses Ivy) to resolve my dependencies.

The Gradle Dependency Management documentation page is pretty helpful. It looks like I can install the rabbitmq-client.jar file in my local Maven repository.

$ mvn install:install-file -Dfile=/home/jim/Documents/rabbitmq-java-client-bin-2.5.1/rabbitmq-client.jar -DgroupId=com -DartifactId=rabbitmq -Dversion=2.5.1 -Dpackaging=jar -DlocalRepositoryPath=/home/jim/Documents/.m2/repository

Now I try to run the build just to see if it works, but it doesn't, as I expected.

$ gradle build

OK I get the compile errors I expected. Now to setup the dependencies in gradle.build. First , I remove the rabbitmq-client.jar file from the Referenced Libraries folder in the Project Explorer. Then I add the following code to build.gradle:

repositories {

mavenLocal()

}

dependencies {

compile group: 'com', name: 'rabbitmq', version: '2.5.1'

testCompile group: 'junit', name: 'junit', version: '4.+'

}

I save the build.gradle file, and rerun the build from the command line.

:compileJava

Download file:/home/jim/.m2/repository/com/rabbitmq/2.5.1/rabbitmq-2.5.1.pom

Download file:/home/jim/.m2/repository/com/rabbitmq/2.5.1/rabbitmq-2.5.1.jar

:processResources UP-TO-DATE

:classes

:jar

:assemble

:compileTestJava UP-TO-DATE

:processTestResources UP-TO-DATE

:testClasses UP-TO-DATE

:test

Download file:/home/jim/.m2/repository/junit/junit/4.8.1/junit-4.8.1.pom

Download file:/home/jim/.m2/repository/junit/junit/4.8.1/junit-4.8.1.jar

:check

:build

BUILD SUCCESSFUL

Sweet. Now I can start running the client and play around with the code.

That's it. I took quite a bit of fumbling around, several restarts of projects and rereading the doc, but once you get the hang of it, Gradle works really well.

L8r.

P.S. Later I went back into STS and discovered all kinds of errors in the classes. Just click on the project, Gradle > Refresh Dependencies and it's all fixed.