mahout安装配置 – 爱嘉牛LA – 博客园

1.下载mahout

   下载地址:http://mahout.apache.org

   我下载的最新版

:mahout-distribution-0.9

2.把mahout解压到你想存放的文档,我是放在/Users/jia/Documents/hadoop-0.20.2,即hadoop的安装目录上。

3.为mahout配置环境

   打开终端,打开profile文件所在的目录

JIAS-MacBook-Pro:~ jia$ open /etc

   把profile文件复制到桌面,然后编辑,在它后面加入环境变量

复制代码

export JAVA_HOME=/Library/Java/JavaVirtualMachines/jdk1.8.0.jdk/Contents/<span style="color: #000000;">Home</p><p>export HADOOP_HOME</span>=Documents/hadoop-0.20.2<span style="color: #000000;"></p><p>export MAHOUT_HOME</span>=Documents/hadoop-0.20.2/mahout-distribution-0.9<span style="color: #000000;"></p><p>export MAVEN_HOME</span>=Documents/apache-maven-3.2.2<span style="color: #000000;"></p><p>export PATH</span>=$PATH:$JAVA_HOME/bin:$HADOOP_HOME/bin:$MAVEN_HOME/bin:$MAHOUT_HOME/<span style="color: #000000;">bin</p><p>export HADOOP_CONF_DIR</span>=Documents/hadoop-0.20.2/<span style="color: #000000;">conf</p><p>export MAHOUT_CONF_DIR</span>=Documents/hadoop-0.20.2/mahout-distribution-0.9/<span style="color: #000000;">conf</p><p>export classpath</span>=$classpath:$JAVA_HOME/lib:$MAHOUT_HOME/lib:$HADOOP_CONF_DIR:$MAHOUT_CONF_DIR

复制代码

    然后把桌面上的profile文件覆盖/etc上的profile,期间要输入管理员密码

    注意:

1.如果在ubuntu下安装的是hadoop2.6进行配置的话,路径为:

export JAVA_HOME=/usr/lib/jvm/java-<span style="color: #800080;">1.7</span>.<span style="color: #800080;">0</span>-openjdk-<span style="color: #000000;">amd64</p><p>export HADOOP_HOME</span>=/home/sendi/hadoop-<span style="color: #800080;">2.6</span>.<span style="color: #800080;">0</span><span style="color: #000000;"></p><p>export MAHOUT_HOME</span>=/home/sendi/mahout-distribution-<span style="color: #800080;">0.9</span><span style="color: #000000;"></p><p>export MAVEN_HOME</span>=/home/sendi/apache-maven-<span style="color: #800080;">3.3</span>.<span style="color: #800080;">3</span><span style="color: #000000;"></p><p>export PATH</span>=$PATH:$JAVA_HOME/bin:$HADOOP_HOME/bin:$MAVEN_HOME/bin:$MAHOUT_HOME/<span style="color: #000000;">bin</p><p>export HADOOP_CONF_DIR</span>=/home/sendi/hadoop-<span style="color: #800080;">2.6</span>.<span style="color: #800080;">0</span>/etc/<span style="color: #000000;">hadoop</p><p>export MAHOUT_CONF_DIR</span>=/home/sendi/mahout-distribution-<span style="color: #800080;">0.9</span>/<span style="color: #000000;">conf</p><p>export classpath</span>=$classpath:$JAVA_HOME/lib:$MAHOUT_HOME/lib:$HADOOP_CONF_DIR:$MAHOUT_CONF_DIR

View Code

    2.配置MAHOU_CONF_DIR时有些网站说时export MAHOUT_CONF_DIR=Documents/hadoop-0.20.2/mahout-distribution-0.9/src/conf

 0.9版本的正确配置是:export MAHOUT_CONF_DIR=Documents/hadoop-0.20.2/mahout-distribution-0.9/conf ,因为当你打开mahout文件夹时,发现没有src这个目录<br> mahout官网上0.9版本有几个压缩文件,我自己试过,前面两个小压缩文件不行。<br> 

    

    这里我选择的是第5个78M的。


4.检验mahout是否配置成功

   4.1启动hadoop

JIAS-MacBook-Pro:hadoop-0.20.2 jia$ bin/start-all.sh 

   4.2查看mahout

复制代码

JIAS-MacBook-Pro:mahout-distribution-0.9 jia$ bin/<span style="color: #000000;">mahout </p><p>MAHOUT_LOCAL is not set; adding HADOOP_CONF_DIR to classpath.</p><p>hadoop binary is not in PATH,HADOOP_HOME</span>/bin,HADOOP_PREFIX/<span style="color: #000000;">bin, running locally</p><p>An example program must be given as the first argument.</p><p>Valid program names are:</p><p>  arff.vector: : Generate Vectors from an ARFF file or directory</p><p>  baumwelch: : Baum</span>-Welch algorithm <span style="color: #0000ff;">for</span><span style="color: #000000;"> unsupervised HMM training</p><p>  canopy: : Canopy clustering</p><p>  cat: : Print a file or resource as the logistic regression models would see it</p><p>  cleansvd: : Cleanup and verification of SVD output</p><p>  clusterdump: : Dump cluster output to text</p><p>  clusterpp: : Groups Clustering Output In Clusters</p><p>  cmdump: : Dump confusion matrix in HTML or text formats</p><p>  concatmatrices: : Concatenates </span>2<span style="color: #000000;"> matrices of same cardinality into a single matrix</p><p>  cvb: : LDA via Collapsed Variation Bayes (0th deriv. approx)</p><p>  cvb0_local: : LDA via Collapsed Variation Bayes, in memory locally.</p><p>  evaluateFactorization: : compute RMSE and MAE of a rating matrix factorization against probes</p><p>  fkmeans: : Fuzzy K</span>-<span style="color: #000000;">means clustering</p><p>  hmmpredict: : Generate random sequence of observations by given HMM</p><p>  itemsimilarity: : Compute the item</span>-item-similarities <span style="color: #0000ff;">for</span> item-<span style="color: #000000;">based collaborative filtering</p><p>  kmeans: : K</span>-<span style="color: #000000;">means clustering</p><p>  lucene.vector: : Generate Vectors from a Lucene index</p><p>  lucene2seq: : Generate Text SequenceFiles from a Lucene index</p><p>  matrixdump: : Dump matrix in CSV format</p><p>  matrixmult: : Take the product of two matrices</p><p>  parallelALS: : ALS</span>-<span style="color: #000000;">WR factorization of a rating matrix</p><p>  qualcluster: : Runs clustering experiments and summarizes results in a CSV</p><p>  recommendfactorized: : Compute recommendations using the factorization of a rating matrix</p><p>  recommenditembased: : Compute recommendations using item</span>-<span style="color: #000000;">based collaborative filtering</p><p>  regexconverter: : Convert text files on a per line basis based on regular expressions</p><p>  resplit: : Splits a set of SequenceFiles into a number of equal splits</p><p>  rowid: : Map SequenceFile</span>&lt;Text,VectorWritable&gt; to {SequenceFile&lt;IntWritable,VectorWritable&gt;, SequenceFile&lt;IntWritable,Text&gt;<span style="color: #000000;">}</p><p>  rowsimilarity: : Compute the pairwise similarities of the rows of a matrix</p><p>  runAdaptiveLogistic: : Score </span><span style="color: #0000ff;">new</span><span style="color: #000000;"> production data using a probably trained and validated AdaptivelogisticRegression model</p><p>  runlogistic: : Run a logistic regression model against CSV data</p><p>  seq2encoded: : Encoded Sparse Vector generation from Text sequence files</p><p>  seq2sparse: : Sparse Vector generation from Text sequence files</p><p>  seqdirectory: : Generate sequence files (of Text) from a directory</p><p>  seqdumper: : Generic Sequence File dumper</p><p>  seqmailarchives: : Creates SequenceFile from a directory containing gzipped mail archives</p><p>  seqwiki: : Wikipedia xml dump to sequence file</p><p>  spectralkmeans: : Spectral k</span>-<span style="color: #000000;">means clustering</p><p>  split: : Split Input data into test and train sets</p><p>  splitDataset: : split a rating dataset into training and probe parts</p><p>  ssvd: : Stochastic SVD</p><p>  streamingkmeans: : Streaming k</span>-<span style="color: #000000;">means clustering</p><p>  svd: : Lanczos Singular Value Decomposition</p><p>  testnb: : Test the Vector</span>-<span style="color: #000000;">based Bayes classifier</p><p>  trainAdaptiveLogistic: : Train an AdaptivelogisticRegression model</p><p>  trainlogistic: : Train a logistic regression using stochastic gradient descent</p><p>  trainnb: : Train the Vector</span>-<span style="color: #000000;">based Bayes classifier</p><p>  transpose: : Take the transpose of a matrix</p><p>  validateAdaptiveLogistic: : Validate an AdaptivelogisticRegression model against hold</span>-<span style="color: #000000;">out data set</p><p>  vecdist: : Compute the distances between a set of Vectors (or Cluster or Canopy, they must fit in memory) and a list of Vectors</p><p>  vectordump: : Dump vectors from a sequence file to text</p><p>  viterbi: : Viterbi decoding of hidden states from given output states sequence</span>

复制代码

      这里需要说明下,当你看到下面的代码时,以为是错的,其实不是,原因:

      MAHOUT_LOCAL:设置是否本地运行,如果设置这个参数就不会运行hadoop了,一旦设置这个参数,那HADOOP_CONF_DIR 和HADOOP_HOME 这两个参数的

      设置就自动失效了。

      当初我就在这个问题上纠结了很久。 

<span style="color: #000000;">MAHOUT_LOCAL is not set; adding HADOOP_CONF_DIR to classpath.</p><p>hadoop binary is not in PATH,HADOOP_HOME</span>/bin,HADOOP_PREFIX/bin, running locally

5.运行mahout的算法

   5.1到下面的地址去下载测试数据

        http://archive.ics.uci.edu/ml/databases/synthetic_control/synthetic_control.data

   5.2创建测试目录testdata,并把数据导入到这个testdata目录中

JIAS-MacBook-Pro:hadoop-0.20.2 jia$ bin/hadoop fs -mkdir testdata

   5.3把测试数据上传到hdfs上,不能把测试数据存在mac上用pages建立的文档上,而是建立一个新的文件命令:touch data 

JIAS-MacBook-Pro:hadoop-0.20.2 jia$ bin/hadoop fs -put workspace/data testdata/

  5.4运行mahout上的kmeans算法

JIAS-MacBook-Pro:hadoop-0.20.2 jia$ bin/hadoop jar mahout-distribution-0.9/mahout-examples-0.9-job.jar org.apache.mahout.clustering.syntheticcontrol.kmeans.Job

 5.5查看结果

复制代码

JIAS-MacBook-Pro:~ jia$ cd Documents/hadoop-0.20.2/<span style="color: #000000;"></p><p>JIAS</span>-MacBook-Pro:hadoop-0.20.2 jia$ bin/hadoop fs -ls output/<span style="color: #000000;"></p><p>Found </span>15<span style="color: #000000;"> items</p><p></span>-rwxrwxrwx   1 jia staff        194 2014-08-03 14:42 /Users/jia/Documents/hadoop-0.20.2/output/<span style="color: #000000;">_policy</p><p>drwxr</span>-xr-x   - jia staff        136 2014-08-03 14:42 /Users/jia/Documents/hadoop-0.20.2/output/<span style="color: #000000;">clusteredPoints</p><p>drwxr</span>-xr-x   - jia staff        544 2014-08-03 14:41 /Users/jia/Documents/hadoop-0.20.2/output/clusters-0<span style="color: #000000;"></p><p>drwxr</span>-xr-x   - jia staff        204 2014-08-03 14:41 /Users/jia/Documents/hadoop-0.20.2/output/clusters-1<span style="color: #000000;"></p><p>drwxr</span>-xr-x   - jia staff        204 2014-08-03 14:42 /Users/jia/Documents/hadoop-0.20.2/output/clusters-10-<span style="color: #0000ff;">final</span><span style="color: #000000;"></p><p>drwxr</span>-xr-x   - jia staff        204 2014-08-03 14:41 /Users/jia/Documents/hadoop-0.20.2/output/clusters-2<span style="color: #000000;"></p><p>drwxr</span>-xr-x   - jia staff        204 2014-08-03 14:41 /Users/jia/Documents/hadoop-0.20.2/output/clusters-3<span style="color: #000000;"></p><p>drwxr</span>-xr-x   - jia staff        204 2014-08-03 14:41 /Users/jia/Documents/hadoop-0.20.2/output/clusters-4<span style="color: #000000;"></p><p>drwxr</span>-xr-x   - jia staff        204 2014-08-03 14:41 /Users/jia/Documents/hadoop-0.20.2/output/clusters-5<span style="color: #000000;"></p><p>drwxr</span>-xr-x   - jia staff        204 2014-08-03 14:41 /Users/jia/Documents/hadoop-0.20.2/output/clusters-6<span style="color: #000000;"></p><p>drwxr</span>-xr-x   - jia staff        204 2014-08-03 14:41 /Users/jia/Documents/hadoop-0.20.2/output/clusters-7<span style="color: #000000;"></p><p>drwxr</span>-xr-x   - jia staff        204 2014-08-03 14:42 /Users/jia/Documents/hadoop-0.20.2/output/clusters-8<span style="color: #000000;"></p><p>drwxr</span>-xr-x   - jia staff        204 2014-08-03 14:42 /Users/jia/Documents/hadoop-0.20.2/output/clusters-9<span style="color: #000000;"></p><p>drwxr</span>-xr-x   - jia staff        136 2014-08-03 14:41 /Users/jia/Documents/hadoop-0.20.2/output/<span style="color: #000000;">data</p><p>drwxr</span>-xr-x   - jia staff        136 2014-08-03 14:41 /Users/jia/Documents/hadoop-0.20.2/output/random-seeds

复制代码

 

来源URL:http://www.cnblogs.com/aijianiula/p/3888386.html