Hadoop

編譯與 Hadoop 捆綁的 hdfs-fuse

  • February 3, 2011

我正在嘗試在執行 Fedora 14 的機器上從 Hadoop 0.20.2 編譯 hdfs-fuse 擴展。以下是我安裝的軟體包:

fuse-2.8.5-2.fc14.x86_64

fuse-libs-2.8.5-2.fc14.x86_64

fuse-devel-2.8.5-2.fc14.x86_64

然後,我按照 Hadoop wiki 上的教程進行操作:

http ://wiki.apache.org/hadoop/MountableHDFS

總結一下,這裡是我正在使用的步驟:

  1. 進入 $HADOOP_HOME 並使用以下命令啟動 libhdfs 的編譯:

ant compile-c++-libhdfs -Dislibhdfs=1

  1. 在第二步中,我將 libhdfs 庫複製到 libhdfs 文件夾:

mkdir build/libhdfs

cp build/c++/Linux-amd64-64/lib/* build/libhdfs

  1. 我編譯 fuse_hdfs 執行檔:

ant compile-contrib -Dislibhdfs=1 -Dfusedfs=1 -Dlibhdfs-fuse=1

我的問題是在最後階段我收到以下錯誤:

    [exec] Making all in .
    [exec] make[1]: Entering directory `/local/opt/hadoop-0.20.2/src/contrib/fuse-dfs'
    [exec] make[1]: Nothing to be done for `all-am'.
    [exec] make[1]: Leaving directory `/local/opt/hadoop-0.20.2/src/contrib/fuse-dfs'
    [exec] Making all in src
    [exec] make[1]: Entering directory `/local/opt/hadoop-0.20.2/src/contrib/fuse-dfs/src'
    [exec] gcc  -Wall -O3 -L/local/opt/hadoop-0.20.2/build/libhdfs -lhdfs -L/lib -lfuse -L/user/lpellegr/home/local/opt/jdk//jre/lib/amd64/server -ljvm  -o fuse_dfs fuse_dfs.o fuse_options.o fuse_trash.o fuse_stat_struct.o fuse_users.o fuse_init.o fuse_connect.o fuse_impls_access.o fuse_impls_chmod.o fuse_impls_chown.o fuse_impls_create.o fuse_impls_flush.o fuse_impls_getattr.o fuse_impls_mkdir.o fuse_impls_mknod.o fuse_impls_open.o fuse_impls_read.o fuse_impls_release.o fuse_impls_readdir.o fuse_impls_rename.o fuse_impls_rmdir.o fuse_impls_statfs.o fuse_impls_symlink.o fuse_impls_truncate.o fuse_impls_utimens.o fuse_impls_unlink.o fuse_impls_write.o  
    [exec] /usr/bin/ld: fuse_stat_struct.o: undefined reference to symbol 'ceil@@GLIBC_2.2.5'
    [exec] /usr/bin/ld: note: 'ceil@@GLIBC_2.2.5' is defined in DSO /lib64/libm.so.6 so try adding it to the linker command line
    [exec] /lib64/libm.so.6: could not read symbols: Invalid operation
    [exec] collect2: ld returned 1 exit status
    [exec] make[1]: *** [fuse_dfs] Error 1
    [exec] make[1]: Leaving directory `/local/opt/hadoop-0.20.2/src/contrib/fuse-dfs/src'
    [exec] make: *** [all-recursive] Error 1

BUILD FAILED
/local/opt/hadoop-0.20.2/build.xml:497: The following error occurred while executing this line:
/local/opt/hadoop-0.20.2/src/contrib/build.xml:30: The following error occurred while executing this line:
/local/opt/hadoop-0.20.2/src/contrib/fuse-dfs/build.xml:57: exec returned: 2

這裡需要注意的重要一點是我沒有定義 FUSE_HOME。你認為定義 FUSE_HOME 有幫助嗎?或者你對這個問題有什麼想法?

我不認為定義 FUSE_HOME 會有所幫助。重要的線路是

[exec] /usr/bin/ld: fuse_stat_struct.o: undefined reference to symbol 'ceil@@GLIBC_2.2.5'
[exec] /usr/bin/ld: note: 'ceil@@GLIBC_2.2.5' is defined in DSO /lib64/libm.so.6 so try adding it to the linker command line
[exec] /lib64/libm.so.6: could not read symbols: Invalid operation
[exec] collect2: ld returned 1 exit status

連結器告訴您找不到 fuse_stat_struct.c 中的符號“ceil”。但它知道它在哪裡。它在 libm.so 中——可能由 glibc-devel 提供。也許您需要 32 位版本的 libm.so。我以前沒見過這個。

稍作搜尋即可得出: http: //lists.fedoraproject.org/pipermail/devel/2010-March/133601.html

作為一個嘗試,嘗試設置您的 LDFLAGS 環境變數並重新編譯。

ant compile-contrib -Dislibhdfs=1 -Dfusedfs=1 -Dlibhdfs-fuse=1 LDFLAGS=-lm

引用自:https://serverfault.com/questions/230318