grep for content at the end of a file

By rainkinz on Mar 10, 2014, comments 40
find . -type f -name "*.log" -exec /bin/bash -c '(tail -n 500 "$1" | grep -q "not in gzip format") && echo "$1"' _ {} \;

Install ruby via rbenv with shared libraries

By rainkinz on Sep 10, 2013, comments 44

Sometimes we need ruby installed with shared libraries. It can be done easily as per:

CONFIGURE_OPTS=--enable-shared  rbenv install 1.9.3-p194

Mapping a key to run the currently visible ruby file in VIM

By rainkinz on Sep 09, 2013, comments 65

Gary Bernhardt uses this in his fantastic Destroy All Software series.

:map ,t :w\|:!ruby %<cr>  

Create UTF-8 mysql database

By rainkinz on Apr 02, 2013, comments 47
CREATE DATABASE dbname CHARACTER SET utf8 COLLATE utf8_general_ci;

Lucene Segment Merges

By rainkinz on Apr 02, 2013, comments 204

Great article on what goes on here:

Deleting a remote git tag

By rainkinz on Feb 18, 2013, comments 41

I can't claim credit for this, but we recently removed a bunch of 'dev tags'. The git command to do so is:

git tag -d tag_name
git push origin :refs/tags/tag_name

Using Viscosity to route through private networks

By rainkinz on Feb 17, 2013, comments 301

Viscosity is my OpenVPN client of choice. Recently I wanted to be able to route through my office to connect to some clients that have ip rules limiting the source of the connection. It's easily done:

  1. Go to the Viscosity menu and select Preferences.
  2. Select the VPN connection you want to route through.
  3. Select Networking.

For each IP you wish to route through the VPN:
1. In the Routing section click the + button to add a route. An Add Route dialog will pop up.
2. Add the IP address of the host you're trying to connect to in Destination.
3. Add the bitmask
4. For the Gateway, select VPN Gateway.
5. Click Add.

Once you're all done, click Save.

Next time you connect traffic to the specified host should be routed through your VPN connection.

Check your work using:

 netstat -nr


Issues with installing mysql gem after Mountain Lion upgrade

By rainkinz on Jan 24, 2013, comments 48

I was having a problem with ruby. After upgrading I reinstalled bundler and bundle installed only to find the mysql gem was failing to install. Weird, it installed fine way back when. This is the error:

$ gem install mysql
Building native extensions.  This could take a while...
ERROR:  Error installing mysql:
    ERROR: Failed to build gem native extension.

        /Users/brendan/.rbenv/versions/1.9.3-p374/bin/ruby extconf.rb
checking for mysql_ssl_set()... *** extconf.rb failed ***
Could not create Makefile due to some reason, probably lack of
necessary libraries and/or headers.  Check the mkmf.log file for more
details.  You may need configuration options.

Provided configuration options:
/Users/brendan/.rbenv/versions/1.9.3-p374/lib/ruby/1.9.1/mkmf.rb:381:in `try_do': The compiler failed to generate an executable file. (RuntimeError)
You have to install development tools first.
    from /Users/brendan/.rbenv/versions/1.9.3-p374/lib/ruby/1.9.1/mkmf.rb:461:in `try_link0'
    from /Users/brendan/.rbenv/versions/1.9.3-p374/lib/ruby/1.9.1/mkmf.rb:476:in `try_link'
    from /Users/brendan/.rbenv/versions/1.9.3-p374/lib/ruby/1.9.1/mkmf.rb:619:in `try_func'
    from /Users/brendan/.rbenv/versions/1.9.3-p374/lib/ruby/1.9.1/mkmf.rb:894:in `block in have_func'
    from /Users/brendan/.rbenv/versions/1.9.3-p374/lib/ruby/1.9.1/mkmf.rb:790:in `block in checking_for'
    from /Users/brendan/.rbenv/versions/1.9.3-p374/lib/ruby/1.9.1/mkmf.rb:284:in `block (2 levels) in postpone'
    from /Users/brendan/.rbenv/versions/1.9.3-p374/lib/ruby/1.9.1/mkmf.rb:254:in `open'
    from /Users/brendan/.rbenv/versions/1.9.3-p374/lib/ruby/1.9.1/mkmf.rb:284:in `block in postpone'
    from /Users/brendan/.rbenv/versions/1.9.3-p374/lib/ruby/1.9.1/mkmf.rb:254:in `open'
    from /Users/brendan/.rbenv/versions/1.9.3-p374/lib/ruby/1.9.1/mkmf.rb:280:in `postpone'
    from /Users/brendan/.rbenv/versions/1.9.3-p374/lib/ruby/1.9.1/mkmf.rb:789:in `checking_for'
    from /Users/brendan/.rbenv/versions/1.9.3-p374/lib/ruby/1.9.1/mkmf.rb:893:in `have_func'
    from extconf.rb:45:in `<main>'

Gem files will remain installed in /Users/brendan/.rbenv/versions/1.9.3-p374/lib/ruby/gems/1.9.1/gems/mysql-2.9.0 for inspection.
Results logged to /Users/brendan/.rbenv/versions/1.9.3-p374/lib/ruby/gems/1.9.1/gems/mysql-2.9.0/ext/mysql_api/gem_make.out

I thought the error about developer tools was referring to a lack of a proper compiler, but I have Xcode and Command Line Tools installed.

Anyway, long story short. The 'trick' was to uninstall mysql and re-install it. My guess there was a link error, though I'm sorry to say I didn't dig into it too much. I'm using homebrew and I installed mysql before upgrading to Mountain Lion, so:

brew uninstall mysql
brew cleanup # I also did a cleanup but not sure if that is necessary
brew install mysql
gem install mysql

My Git Workflow

By rainkinz on Jan 08, 2013, comments 31

I found it took me a little time to find a workflow I like with git. This is what I've settled on for now:

  • Branch
git checkout -b some_topic
  • Make some changes to some_topic branch.
  • Notice that master has been updated.
  • Make sure my changes have been committed in some_topic branch.
  • Checkout master and pull the latest changes:
git checkout master
git pull
  • Rebase those changes into the some_topic branch:
git checkout some_topic
git rebase master
  • Optionally make more changes on the some_topic branch:
  • Merge some_topic into master:
git checkout master
git pull
git merge some_topic

Check this out too:

As an alternative you could do this:

git fetch
git rebase origin/master

Displaying Unicode Characters in Ruby

By rainkinz on Mar 15, 2012, comments 73

If you know the Unicode character code and want to decode it you can do something like this:

  • Ruby 1.8
puts ["2665".hex].pack("U")
=> nil
  • Ruby 1.9 In Ruby 1.9 the \u{} escape sequence can be used to decode unicode codepoints:
puts "\u{2665}"
=> nil

Splitting a file into many pieces

By rainkinz on Nov 11, 2011, comments 28

The other day I was trying to copy a large gzip file onto a USB drive and it kept erroring out when it hit the 4Gb mark. I realized (face palm) that the drive is formatted as a vfat filesystem. Obviously vfat is limited to max a max file size of 4Gb. So simply split the file into 4000MB chunks with:

split --line-bytes=4000m foo.tar.gz foo_ &

Copy the pieces onto the USB drive. Take the USB drive to the other copmputer. Copy the files off the drive and rejoin the file with:

cat foo_a* > foo.tar.gz &

Downgrading Rubygems

By rainkinz on Jul 11, 2011, comments 29

Sadly I need to work with a rails 2.3.3 and 1.2.3 app now and then and the latest rubygems has issues with these old versions of rails. For example:

rake my:task:name
rake/rdoctask is deprecated.  Use rdoc/task instead (in RDoc 2.4.2+)
rake aborted!
uninitialized constant ActiveSupport::Dependencies::Mutex

So I have to downgrade. Not a huge deal. I'm now using rbenv

gem update --system 1.4.2

DWR, Spring and Capture (Using Spring and DWR 2.x Filters)

By rainkinz on Jan 27, 2007, comments 360

Recently a bot figured out how to submit my comment form. It was pretty funny I had more than 700 comments telling me how interesting my post on how to install a jar in maven was. I decided to add captcha support and while I was at it, make the comment form use DWR for submission so that the page didn't refresh etc.

I have created a little example application demonstrating how I integrated JCaptcha with DWR. The example is also (hopefully) serves as an example of how to use the new Spring Namespace support that has been added to DWR. I think if you're using DWR with Spring 2 and you want to keep all your configuration within Spring, you should use the Namespace support for configuring DWR. It's just easier. Please note this example only runs in a Java 5 environment and that if you're using DWR 2.0.rc2 there is a small issue with using Spring 2.0.1 or better so I've had to use Spring 2.0. This issue is fixed in CVS HEAD so if you want to build DWR yourself from source you should have no problems with Spring 2.0.1 or 2.0.2.

DWR Configuration in Spring

To keep things simple, in this example all the DWR configuration is done in the spring-catpcha-controllers.xml spring configuration. First we register the DwrController using:

<dwr:controller id="dwrController" debug="false" >

Now we need to tell DWR what domain objects we'd like to expose using javascript. DWR does this using Converters. In this case I'm really only exposing the Comment domain object:

<dwr:convert type="bean" class="com.kuripai.example.domain.Comment">
  <dwr:include method="id" />
  <dwr:include method="title" />
  <dwr:include method="postedBy" />
  <dwr:include method="website" />
  <dwr:include method="email" />
            <dwr:include method="body" />
            <dwr:include method="createdAt" />
            <dwr:include method="humanResponse" />

The next step is to configure and expose the service that saves comments. I've created a very simple WeblogService for this purpose which just saves the comments into a java.util.List. Obviously a production version would save the comments in a database or similar. Here is the configuration:

 <bean id="weblogService" class="com.kuripai.example.service.WeblogService" >
        <dwr:remote javascript="weblogService">

            <!--  Methods that are allowed to be exposed via javascript -->
            <dwr:include method="getComments" />
            <dwr:include method="saveComment" />

            <!-- Filter to handle Captcha -->
            <dwr:filter class="com.kuripai.example.dwr.captcha.CaptchaAjaxFilter" />

        <property name="validators">
                <ref local="captchaValidator" />
                <bean class="com.kuripai.example.domain.CommentValidator" />

As you can see you configure your service as normal with the additional element telling DWR what the name of the service should be in Javascript and the elements telling DWR what methods are allowed to be exposed. Also note that I've exposed the service directly via DWR. Personally I don't normally do this. Instead I write a very simple proxy around the service and expose that instead. One reason I like to do this is to invoke regular Spring Validators and return a regular org.springframework.validation.Errors object to DWR. I'm not really sure if this is such a great idea. Just something I'm playing with at the moment.

    public Errors saveComment(Comment comment) {
        Errors errors = new BindException(comment, "comment");
        for (int i = 0; i < validators.length; i++) {
            ValidationUtils.invokeValidator(validators[i], comment, errors);

        if (!errors.hasErrors()) {
            comment.setId(Long.valueOf((long) comments.size()));
        return errors;

The org.directwebremoting.AjaxFilter

You might have notices the element in the previous configuration code snippet. DWR 2 introduced the org.directwebremoting.AjaxFilter that can be configured to be invoked per request for a method exposed by DWR. I use a filter to here to get the current Captcha ID in Session and put it in a threadlocal based holder object for use in the Captcha validator.

    public Object doFilter(Object object, Method method, Object[] params, AjaxFilterChain chain) throws Exception {
        if (logger.isDebugEnabled()) {
  "Processing method '" + method.getName() + "' on service '" +
                        object + "'");

        HttpSession session = WebContextFactory.get().getSession();

        Object reply = chain.doFilter(object, method, params);

        return reply;

The CaptchaValidator then is a regular org.springframework.validation.Validator:

    public void validate(Object target, Errors errors) {
        ValidationUtils.rejectIfEmpty(errors, CAPTCHA_FIELD_NAME,
                new Object[] { new DefaultMessageSourceResolvable(
                        "form.captcha") }, "Please enter the validation word");

        if (errors.getFieldError(CAPTCHA_FIELD_NAME) != null) {

        CaptchaAware captchaAware = (CaptchaAware) target;
        if (!isCaptchaValid(captchaAware.getHumanResponse())) {
            errors.rejectValue(CAPTCHA_FIELD_NAME, "form.error.captchaInvalid",
                    "Please enter the security word again");

    private boolean isCaptchaValid(Object response) {
        String captchaId = CaptchaResponseHolder.getCaptchaId();
        if (logger.isDebugEnabled()) {
            logger.debug("Validating captcha response '" + response
                    + "' for captchaId '" + captchaId + "'");
        Boolean isValid = captchaService.validateResponseForID(captchaId,
        return isValid.booleanValue();

Running the Example

That's pretty much it. I've zipped up the source code for the example here -> (I will put this up in the next few days). Unzip this into a directory of your choice and type mvn tomcat:run after completing the steps below providing of course you have maven 2 installed. You should also be able to import the project into eclipse to take a look at it.

Installing JCaptcha and Other Required Libraries in your Maven Repository

Unfortunately there seems to be a problem with the version of JCaptcha in on ibiblio. So you will need download and install the jar in you maven repository yourself following these steps (assuming you have Maven 2 installed):

  1. Download jcaptcha-1.0-RC3 binary from the projects download page.

  2. Unzip the downloaded jcaptcha binary and cd into the directory where you unzipped it.

  3. Install the jar using this command:

  mvn install:install-file -DgroupId=jcaptcha -DartifactId=jcaptcha-all \
  -Dversion=1.0-RC3 -Dpackaging=jar -Dfile=jcaptcha-all-1.0-RC3.jar

h2. Running the Example

You should now be ready to run the example. From the command line, cd into the directory where you downloaded the example. Type:

mvn clean tomcat:run 

Maven should download a whole bunch of jar files including a enough to run an embedded version of Tomcat. Which will start up and run. Navigate to http://localhost:8080/dwr-captcha/example/comments.html and you should see a regular looking comment form with captcha that never refreshes.