Compiling hdfs-fuse bundled with Hadoop

Question

I am trying to compile the hdfs-fuse extension from Hadoop 0.20.2 on a machine running Fedora 14. Below are the packages I have installed:

fuse-2.8.5-2.fc14.x86_64
fuse-libs-2.8.5-2.fc14.x86_64
fuse-devel-2.8.5-2.fc14.x86_64

Then, I have followed the tutorial available on the Hadoop wiki:
http://wiki.apache.org/hadoop/MountableHDFS

To summarize, here, are the steps I am using:

1. In go into $HADOOP_HOME and I launch the compilation of the libhdfs by using:

   ant compile-c++-libhdfs -Dislibhdfs=1

2. In a second step I copy the libhdfs library to the libhdfs folder:

   mkdir build/libhdfs
   cp build/c++/Linux-amd64-64/lib/* build/libhdfs

3. I compile the fuse_hdfs executable :

   ant compile-contrib -Dislibhdfs=1 -Dfusedfs=1 -Dlibhdfs-fuse=1

My problem is that in the last stage I get the following error:

     [exec] Making all in .
     [exec] make[1]: Entering directory `/local/opt/hadoop-0.20.2/src/contrib/fuse-dfs'
     [exec] make[1]: Nothing to be done for `all-am'.
     [exec] make[1]: Leaving directory `/local/opt/hadoop-0.20.2/src/contrib/fuse-dfs'
     [exec] Making all in src
     [exec] make[1]: Entering directory `/local/opt/hadoop-0.20.2/src/contrib/fuse-dfs/src'
     [exec] gcc  -Wall -O3 -L/local/opt/hadoop-0.20.2/build/libhdfs -lhdfs -L/lib -lfuse -L/user/lpellegr/home/local/opt/jdk//jre/lib/amd64/server -ljvm  -o fuse_dfs fuse_dfs.o fuse_options.o fuse_trash.o fuse_stat_struct.o fuse_users.o fuse_init.o fuse_connect.o fuse_impls_access.o fuse_impls_chmod.o fuse_impls_chown.o fuse_impls_create.o fuse_impls_flush.o fuse_impls_getattr.o fuse_impls_mkdir.o fuse_impls_mknod.o fuse_impls_open.o fuse_impls_read.o fuse_impls_release.o fuse_impls_readdir.o fuse_impls_rename.o fuse_impls_rmdir.o fuse_impls_statfs.o fuse_impls_symlink.o fuse_impls_truncate.o fuse_impls_utimens.o fuse_impls_unlink.o fuse_impls_write.o  
     [exec] /usr/bin/ld: fuse_stat_struct.o: undefined reference to symbol 'ceil@@GLIBC_2.2.5'
     [exec] /usr/bin/ld: note: 'ceil@@GLIBC_2.2.5' is defined in DSO /lib64/libm.so.6 so try adding it to the linker command line
     [exec] /lib64/libm.so.6: could not read symbols: Invalid operation
     [exec] collect2: ld returned 1 exit status
     [exec] make[1]: *** [fuse_dfs] Error 1
     [exec] make[1]: Leaving directory `/local/opt/hadoop-0.20.2/src/contrib/fuse-dfs/src'
     [exec] make: *** [all-recursive] Error 1

BUILD FAILED
/local/opt/hadoop-0.20.2/build.xml:497: The following error occurred while executing this line:
/local/opt/hadoop-0.20.2/src/contrib/build.xml:30: The following error occurred while executing this line:
/local/opt/hadoop-0.20.2/src/contrib/fuse-dfs/build.xml:57: exec returned: 2

What is important to notice here is that I haven't define FUSE_HOME. Do you think that defining FUSE_HOME can help ? or do you have an idea about the problem ?

Answer 1

I don't think defining FUSE_HOME would help. The important lines are

 [exec] /usr/bin/ld: fuse_stat_struct.o: undefined reference to symbol 'ceil@@GLIBC_2.2.5'
 [exec] /usr/bin/ld: note: 'ceil@@GLIBC_2.2.5' is defined in DSO /lib64/libm.so.6 so try adding it to the linker command line
 [exec] /lib64/libm.so.6: could not read symbols: Invalid operation
 [exec] collect2: ld returned 1 exit status

The linker is telling you that the symbol 'ceil' in fuse_stat_struct.c can't be found.. but it knows where it is. It's in libm.so -- probably provided by glibc-devel. Maybe you need a 32-bit version of libm.so. I haven't seen this before.

A bit of searching yields this: http://lists.fedoraproject.org/pipermail/devel/2010-March/133601.html

As a stab at it, try setting your LDFLAGS environment variable and recompile.

 ant compile-contrib -Dislibhdfs=1 -Dfusedfs=1 -Dlibhdfs-fuse=1 LDFLAGS=-lm