Hadoop
編譯與 Hadoop 捆綁的 hdfs-fuse
我正在嘗試在執行 Fedora 14 的機器上從 Hadoop 0.20.2 編譯 hdfs-fuse 擴展。以下是我安裝的軟體包:
fuse-2.8.5-2.fc14.x86_64
fuse-libs-2.8.5-2.fc14.x86_64
fuse-devel-2.8.5-2.fc14.x86_64
然後,我按照 Hadoop wiki 上的教程進行操作:
http ://wiki.apache.org/hadoop/MountableHDFS
總結一下,這裡是我正在使用的步驟:
- 進入 $HADOOP_HOME 並使用以下命令啟動 libhdfs 的編譯:
ant compile-c++-libhdfs -Dislibhdfs=1
- 在第二步中,我將 libhdfs 庫複製到 libhdfs 文件夾:
mkdir build/libhdfs
cp build/c++/Linux-amd64-64/lib/* build/libhdfs
- 我編譯 fuse_hdfs 執行檔:
ant compile-contrib -Dislibhdfs=1 -Dfusedfs=1 -Dlibhdfs-fuse=1
我的問題是在最後階段我收到以下錯誤:
[exec] Making all in . [exec] make[1]: Entering directory `/local/opt/hadoop-0.20.2/src/contrib/fuse-dfs' [exec] make[1]: Nothing to be done for `all-am'. [exec] make[1]: Leaving directory `/local/opt/hadoop-0.20.2/src/contrib/fuse-dfs' [exec] Making all in src [exec] make[1]: Entering directory `/local/opt/hadoop-0.20.2/src/contrib/fuse-dfs/src' [exec] gcc -Wall -O3 -L/local/opt/hadoop-0.20.2/build/libhdfs -lhdfs -L/lib -lfuse -L/user/lpellegr/home/local/opt/jdk//jre/lib/amd64/server -ljvm -o fuse_dfs fuse_dfs.o fuse_options.o fuse_trash.o fuse_stat_struct.o fuse_users.o fuse_init.o fuse_connect.o fuse_impls_access.o fuse_impls_chmod.o fuse_impls_chown.o fuse_impls_create.o fuse_impls_flush.o fuse_impls_getattr.o fuse_impls_mkdir.o fuse_impls_mknod.o fuse_impls_open.o fuse_impls_read.o fuse_impls_release.o fuse_impls_readdir.o fuse_impls_rename.o fuse_impls_rmdir.o fuse_impls_statfs.o fuse_impls_symlink.o fuse_impls_truncate.o fuse_impls_utimens.o fuse_impls_unlink.o fuse_impls_write.o [exec] /usr/bin/ld: fuse_stat_struct.o: undefined reference to symbol 'ceil@@GLIBC_2.2.5' [exec] /usr/bin/ld: note: 'ceil@@GLIBC_2.2.5' is defined in DSO /lib64/libm.so.6 so try adding it to the linker command line [exec] /lib64/libm.so.6: could not read symbols: Invalid operation [exec] collect2: ld returned 1 exit status [exec] make[1]: *** [fuse_dfs] Error 1 [exec] make[1]: Leaving directory `/local/opt/hadoop-0.20.2/src/contrib/fuse-dfs/src' [exec] make: *** [all-recursive] Error 1 BUILD FAILED /local/opt/hadoop-0.20.2/build.xml:497: The following error occurred while executing this line: /local/opt/hadoop-0.20.2/src/contrib/build.xml:30: The following error occurred while executing this line: /local/opt/hadoop-0.20.2/src/contrib/fuse-dfs/build.xml:57: exec returned: 2
這裡需要注意的重要一點是我沒有定義 FUSE_HOME。你認為定義 FUSE_HOME 有幫助嗎?或者你對這個問題有什麼想法?
我不認為定義 FUSE_HOME 會有所幫助。重要的線路是
[exec] /usr/bin/ld: fuse_stat_struct.o: undefined reference to symbol 'ceil@@GLIBC_2.2.5' [exec] /usr/bin/ld: note: 'ceil@@GLIBC_2.2.5' is defined in DSO /lib64/libm.so.6 so try adding it to the linker command line [exec] /lib64/libm.so.6: could not read symbols: Invalid operation [exec] collect2: ld returned 1 exit status
連結器告訴您找不到 fuse_stat_struct.c 中的符號“ceil”。但它知道它在哪裡。它在 libm.so 中——可能由 glibc-devel 提供。也許您需要 32 位版本的 libm.so。我以前沒見過這個。
稍作搜尋即可得出: http: //lists.fedoraproject.org/pipermail/devel/2010-March/133601.html
作為一個嘗試,嘗試設置您的 LDFLAGS 環境變數並重新編譯。
ant compile-contrib -Dislibhdfs=1 -Dfusedfs=1 -Dlibhdfs-fuse=1 LDFLAGS=-lm