Was good to share use cases and experiences in person with people who I had just spoken with in forums.
I spend the first two days in the hackathon doing some research and test of different ASF projects. Put special interest in Pig.

There were really interesting chats. I found specially great Mahout project. I had discovered it in ApacheCon 2008 in New Orleans, I almost just heard about it in there but paid more atention this time and looks full of possibilities. It is used for machine learning and runs under Hadoop.
Was also good to get some info about Servlet 3.0 and learn about servlets doFilter function and some other stuff.
HBase is another project I was interested in. Looks good to be used as a “data warehouse” but seems really difficult (at least at first impression) to deal with the stored data.

Meetups were so good too. There was a presentation about the new Lucene contrib TrieRangeQuery. It is still not available in the official release but you can use it graving a nightly build. In the next few days I will try to write with more detail about this and other presented projects.

After checking the example with detail, what comes now to my mind is if it would be possible to create a Lucene index using Pig and MapReduce jobs retrieving data from a distributed HBase data store system… I am wandering if there would be Lucene analyzers problems (or any other), for example.

I have read that Pig is not specially fast accessing to data. However, in indexing cases, probably this would be more than compensated with the MapReduce jobs.

How fast would it be? I still have lots of research and tests to do…