WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform – 开源中国社区

然而完整的运行返回数据为:

MacdeMacBook-Pro-2:2.2.0 mac$ sbin/start-all.sh

This script is Deprecated. Instead use start-dfs.sh and start-yarn.sh

14/03/19 22:34:33 DEBUG lib.MutableMetricsFactory: field org.apache.hadoop.metrics2.lib.MutableRate org.apache.hadoop.security.UserGroupInformation$UgiMetrics.loginSuccess with annotation @org.apache.hadoop.metrics2.annotation.Metric(valueName=Time, about=, value=[Rate of successful kerberos logins and latency (milliseconds)], always=false, type=DEFAULT, sampleName=Ops)

14/03/19 22:34:33 DEBUG lib.MutableMetricsFactory: field org.apache.hadoop.metrics2.lib.MutableRate org.apache.hadoop.security.UserGroupInformation$UgiMetrics.loginFailure with annotation @org.apache.hadoop.metrics2.annotation.Metric(valueName=Time, about=, value=[Rate of failed kerberos logins and latency (milliseconds)], always=false, type=DEFAULT, sampleName=Ops)

14/03/19 22:34:33 DEBUG impl.MetricsSystemImpl: UgiMetrics, User and group related metrics

14/03/19 22:34:33 DEBUG security.Groups:  Creating new Groups object

14/03/19 22:34:33 DEBUG util.NativeCodeLoader: Trying to load the custom-built native-hadoop library…

14/03/19 22:34:33 DEBUG util.NativeCodeLoader: Failed to load native-hadoop with error: java.lang.UnsatisfiedLinkError: no hadoop in java.library.path

14/03/19 22:34:33 DEBUG util.NativeCodeLoader: java.library.path=.:/Library/Java/Extensions:/System/Library/Java/Extensions:/usr/lib/java

14/03/19 22:34:33 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform… using builtin-java classes where applicable

14/03/19 22:34:33 DEBUG security.JniBasedUnixGroupsMappingWithFallback: Falling back to shell based

14/03/19 22:34:33 DEBUG security.JniBasedUnixGroupsMappingWithFallback: Group mapping impl=org.apache.hadoop.security.ShellBasedUnixGroupsMapping

14/03/19 22:34:33 DEBUG security.Groups: Group mapping impl=org.apache.hadoop.security.JniBasedUnixGroupsMappingWithFallback; cacheTimeout=300000

14/03/19 22:34:33 DEBUG security.UserGroupInformation: hadoop login

14/03/19 22:34:33 DEBUG security.UserGroupInformation: hadoop login commit

14/03/19 22:34:33 DEBUG security.UserGroupInformation: using local user:UnixPrincipal?? mac

14/03/19 22:34:33 DEBUG security.UserGroupInformation: UGI loginUser:mac (auth:SIMPLE)

14/03/19 22:34:33 DEBUG security.UserGroupInformation: PrivilegedAction as:mac (auth:SIMPLE) from:org.apache.hadoop.hdfs.tools.GetConf.run(GetConf.java:314)

14/03/19 22:34:33 DEBUG impl.MetricsSystemImpl: StartupProgress, NameNode startup progress

Incorrect configuration: namenode address dfs.namenode.servicerpc-address or dfs.namenode.rpc-address is not configured.

Starting namenodes on []

localhost: ssh: connect to host localhost port 22: Connection refused

localhost: ssh: connect to host localhost port 22: Connection refused

Starting secondary namenodes [0.0.0.0]

0.0.0.0: ssh: connect to host 0.0.0.0 port 22: Connection refused

14/03/19 22:34:37 DEBUG lib.MutableMetricsFactory: field org.apache.hadoop.metrics2.lib.MutableRate org.apache.hadoop.security.UserGroupInformation$UgiMetrics.loginSuccess with annotation @org.apache.hadoop.metrics2.annotation.Metric(valueName=Time, about=, value=[Rate of successful kerberos logins and latency (milliseconds)], always=false, type=DEFAULT, sampleName=Ops)

14/03/19 22:34:37 DEBUG lib.MutableMetricsFactory: field org.apache.hadoop.metrics2.lib.MutableRate org.apache.hadoop.security.UserGroupInformation$UgiMetrics.loginFailure with annotation @org.apache.hadoop.metrics2.annotation.Metric(valueName=Time, about=, value=[Rate of failed kerberos logins and latency (milliseconds)], always=false, type=DEFAULT, sampleName=Ops)

14/03/19 22:34:37 DEBUG impl.MetricsSystemImpl: UgiMetrics, User and group related metrics

14/03/19 22:34:37 DEBUG security.Groups:  Creating new Groups object

14/03/19 22:34:37 DEBUG util.NativeCodeLoader: Trying to load the custom-built native-hadoop library…

14/03/19 22:34:37 DEBUG util.NativeCodeLoader: Failed to load native-hadoop with error: java.lang.UnsatisfiedLinkError: no hadoop in java.library.path

14/03/19 22:34:37 DEBUG util.NativeCodeLoader: java.library.path=.:/Library/Java/Extensions:/System/Library/Java/Extensions:/usr/lib/java

14/03/19 22:34:37 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform… using builtin-java classes where applicable

14/03/19 22:34:37 DEBUG security.JniBasedUnixGroupsMappingWithFallback: Falling back to shell based

14/03/19 22:34:37 DEBUG security.JniBasedUnixGroupsMappingWithFallback: Group mapping impl=org.apache.hadoop.security.ShellBasedUnixGroupsMapping

14/03/19 22:34:37 DEBUG security.Groups: Group mapping impl=org.apache.hadoop.security.JniBasedUnixGroupsMappingWithFallback; cacheTimeout=300000

14/03/19 22:34:37 DEBUG security.UserGroupInformation: hadoop login

14/03/19 22:34:37 DEBUG security.UserGroupInformation: hadoop login commit

14/03/19 22:34:37 DEBUG security.UserGroupInformation: using local user:UnixPrincipal?? mac

14/03/19 22:34:37 DEBUG security.UserGroupInformation: UGI loginUser:mac (auth:SIMPLE)

14/03/19 22:34:37 DEBUG security.UserGroupInformation: PrivilegedAction as:mac (auth:SIMPLE) from:org.apache.hadoop.hdfs.tools.GetConf.run(GetConf.java:314)

starting yarn daemons

starting resourcemanager, logging to /usr/local/Cellar/hadoop/2.2.0/libexec/logs/yarn-mac-resourcemanager-MacdeMacBook-Pro-2.local.out

localhost: ssh: connect to host localhost port 22: Connection refused

这是在网上找到的最贴切的解决方法
Run the example with an argument like this -Djava.library.path={YOUR_HADOOP_HOME}/lib/native/Linux-amd64-64/ will probably fix the problem on the condition that Mac has something native under native directory
大概的意思和解决思路我已经差不多了,因为我是用Brew安装的Hadoop,可能存在库不同的情况,但是是可以解决的嘛。

问题在于我hadoop下没有lib目录,更没有native这样的目录,里面的文件是从哪来的呢?没有可以借鉴的地方呀,哪位高手知道呢?我本人是MAC,我好去download…

这个问题困扰我一晚了,呵呵。

来源URL:http://www.oschina.net/question/154627_147919