Using Riak 2.x on Mac OS X with 7DBin7Weeks

I've been trying to work through Seven Databases in Seven Weeks, and have run into problems a couple of times installing Riak and given up. I finally got through it. One issue is that the book uses Riak 1.x, whereas Riak 2.x is the current version. Also, the default ports and commands have changed a bit, so the commands in the book don't work.

The key part to getting Riak running is to install the correct version of both Erlang and Riak, both from source.

Installing Riak requires the custom Erlang from Basho. Don't install Erlang using OS X's brew (you can only get version 17) or from the standard Erlang bistro (you get R16A). You must install R16B02 from Basho per the instructions at here. I added '–prefix=/usr/local/otp_R16B02' to the 'configure' command, and then after installing added it to my path with 'export PATH=$PATH:/usr/local/otp_R16B02/bin' in .bash_profile.

If you try to use Erlang R17 installed by brew, you get a version mismatch error when trying to build Riak (17 instead of R16B). Manually changing this will not work either, because Riak actually uses some custom flags not available in the standard Erlang tooling.

If you try to use the standard Erlang R16A package, you will get an error on the console like:

xxx:riak-2.0.4 phil.varner$ dev/dev1/bin/riak start
!!!!
!!!! WARNING: ulimit -n is 32768; 65536 is the recommended minimum.
!!!!
riak failed to start within 15 seconds,
see the output of 'riak console' for more information.
If you want to wait longer, set the environment variable
WAIT_FOR_ERLANG to the number of seconds to wait.

and then from riak console

LT-A10-122189:riak-2.0.4 phil.varner$ dev/dev1/bin/riak console
config is OK
-config /Users/phil.varner/Downloads/riak-2.0.4/dev/dev1/data/generated.configs/app.2015.02.20.23.56.47.config -args_file /Users/phil.varner/Downloads/riak-2.0.4/dev/dev1/data/generated.configs/vm.2015.02.20.23.56.47.args -vm_args /Users/phil.varner/Downloads/riak-2.0.4/dev/dev1/data/generated.configs/vm.2015.02.20.23.56.47.args
!!!!
!!!! WARNING: ulimit -n is 32768; 65536 is the recommended minimum.
!!!!
Exec:  /Users/phil.varner/Downloads/riak-2.0.4/dev/dev1/bin/../erts-5.10/bin/erlexec -boot /Users/phil.varner/Downloads/riak-2.0.4/dev/dev1/bin/../releases/2.0.4/riak               -config /Users/phil.varner/Downloads/riak-2.0.4/dev/dev1/data/generated.configs/app.2015.02.20.23.56.47.config -args_file /Users/phil.varner/Downloads/riak-2.0.4/dev/dev1/data/generated.configs/vm.2015.02.20.23.56.47.args -vm_args /Users/phil.varner/Downloads/riak-2.0.4/dev/dev1/data/generated.configs/vm.2015.02.20.23.56.47.args              -pa /Users/phil.varner/Downloads/riak-2.0.4/dev/dev1/bin/../lib/basho-patches -- console
Root: /Users/phil.varner/Downloads/riak-2.0.4/dev/dev1/bin/..
bad scheduling option -sfwi
Usage: beam.smp [flags]...

The error "bad scheduling option -sfwi" is because it's trying to use the custom flags in Basho's Erlang, which obviously don't exist in the standard Erlang tools.

After installing the Basho Erlang, install Riak from here and follow the instructions

You'll probably need to change your ulimit settings also, for which there are excellent instructions here

There are quite a few changes in Riak 2.x from Riak 1.x used in the book. The Basho Riak Five-Minute Install has good examples of the new syntax.

The riak-admin command has changed since 7DBin7Weeks p2.0 was published, so the 'join' command is now like 'dev/dev2/bin/riak-admin cluster join dev1@127.0.0.1', and it only stages joins, without actually affecting them. You can see the staged changes with 'dev/dev1/bin/riak-admin cluster plan' and affect them with 'dev/dev2/bin/riak-admin cluster commit'.

The default port for the HTTP interface is now 10018 instead of 8091, so all of the URLs are now like 'http://localhost:10018/stats'.

I also ran into an issue where curl couldn't connect to the Riak port 10018 using "localhost", but directly as 127.0.0.1 worked fine.

Using Apache CXF 2.7, Struts2 2.3, and ASM 5 with Maven

In an application I work on, we recently upgraded Struts2 to 2.3.20. We then started getting exceptions at runtime with the cause of:

Caused by: java.lang.VerifyError: class
net.sf.cglib.core.DebuggingClassWriter overrides final method
visit.(IILjava/lang/String;Ljava/lang/String;Ljava/lang/String;[Ljava/lang/String;)V

The problem here is that in cglib 2.2.2, net.sf.cglib.core.DebuggingClassWriter extends the ASM class org.objectweb.asm.ClassVisitor and overrides the visit method. The only works in ASM 3, since in ASM 4 and 5 there's a different pattern of extension to follow, and the visit method is final.

So the dependencies are

  • CXF 2.7.14 depends on cglib 2.2 and ASM 3.3.1
  • Struts2 2.3.20 depends
    on cglib-nodeps 2.1 and ASM 5.0.2
  • Our webapp has a dep on cglib-nodeps 2.2.2

I first tried downgrading everything, but ran into some issues with Java 8 bytecode with ASM 3 and 4, so we had to upgrade to ASM 5. The only use of ASM is in xwork-core by com.opensymphony.xwork2.util.finder.ClassFinder, which is only used by
PackageBasedActionConfigBuilder in the "convention" plugin, which my application doesn't use, so theoretically if we were compiling to target 1.6 or 1.7 we could downgrade that to ASM 3 or 4. When struts2 2.3.22 comes out, it looks like the resolution of this issue CXF broken with upgrade of ASM 5 since Struts 2.3.20 will allow that.

Using CXF and our app with cglib 3.1 and ASM 5 seems to be working fine, at least so far. All of the CXF tests (apart from a few of the CORBA ones) passed with ASM 5.

Updated the parent pom's properties and dependencyManagement dependencies to use the right versions and exclude the wrong ones. One change was that we previously used the cglib-nodeps artifact that shadow jars the ASM classes into net.sf.cglib.asm instead of having the real ASM artifact as a transitive dependency.

Add properties for the versions, since there are a few dependencies for each one:

<properties>
    <cxf.version>2.7.14</cxf.version>
    <struts2-version>2.3.20</struts2-version>
    <asm.version>5.0.2</asm.version>
    <cglib.version>3.1</cglib.version>
</properties>

In the parent pom, set the versions to use and the transitive exclusions for the wrong ones:

<dependencyManagement>
...
        <dependency>
            <groupId>org.apache.cxf</groupId>
            <artifactId>cxf-bundle</artifactId>
            <version>${cxf.version}</version>
            <exclusions>
                <exclusion> <!-- exclude version 3 of asm -->
                    <groupId>asm</groupId>
                    <artifactId>asm</artifactId>
                </exclusion>
           ...
      </dependency>

        <dependency> 
            <groupId>org.apache.struts</groupId>
            <artifactId>struts2-core</artifactId>
            <version>${struts2-version}</version>
            <exclusions>
                <exclusion>
                    <groupId>javassist</groupId>
                    <artifactId>javassist</artifactId>
                </exclusion>
                <exclusion> <!-- exclude version 2.1-->
                    <groupId>cglib</groupId>
                    <artifactId>cglib-nodep</artifactId>
                </exclusion>
                <exclusion> <!-- we prefer our explicit version, though it should be the same -->
                    <groupId>org.ow2.asm</groupId>
                    <artifactId>asm</artifactId>
                </exclusion>
                <exclusion> <!-- we prefer our explicit version, though it should be the same -->
                    <groupId>org.ow2.asm</groupId>
                    <artifactId>asm-commons</artifactId>
                </exclusion>
            </exclusions>
        </dependency>
...
        <dependency>
            <groupId>cglib</groupId>
            <artifactId>cglib</artifactId>
            <version>${cglib.version}</version>
            <exclusions>
                <exclusion> <!-- exclude v. 4.2-->
                    <groupId>org.ow2.asm</groupId>
                    <artifactId>asm</artifactId>
                </exclusion>
                <exclusion> <!-- exclude v. 4.2 -->
                    <groupId>org.ow2.asm</groupId>
                    <artifactId>asm-util</artifactId>
                </exclusion>
                <exclusion> <!-- no ant! -->
                    <groupId>ant</groupId>
                    <artifactId>ant</artifactId>
                </exclusion>
            </exclusions>
        </dependency>

explicitly add the asm version we want:

<dependency>
            <groupId>org.ow2.asm</groupId>
            <artifactId>asm</artifactId>
            <version>${asm.version}</version>
        </dependency>
        <dependency>
            <groupId>org.ow2.asm</groupId>
            <artifactId>asm-util</artifactId>
            <version>${asm.version}</version>
        </dependency>
        <dependency>
            <groupId>org.ow2.asm</groupId>
            <artifactId>asm-commons</artifactId>
            <version>${asm.version}</version>
        </dependency>

and then in the dependencies section of the build pom, explicitly add cglib and asm:

<dependencies>
...
    <dependency>
        <groupId>cglib</groupId>
        <artifactId>cglib</artifactId>
    </dependency>
    <dependency>
        <groupId>org.ow2.asm</groupId>
        <artifactId>asm</artifactId>
    </dependency>
    <dependency>
        <groupId>org.ow2.asm</groupId>
        <artifactId>asm-util</artifactId>
    </dependency>
    <dependency>
        <groupId>org.ow2.asm</groupId>
        <artifactId>asm-commons</artifactId>
    </dependency>
...
</dependencies>

That seems to be working, but please comment if you have any additions to this!