Item-Based Recommendations with Hadoop

Mahout在MapReduce上实现了Item-Based Collaborative Filtering,这里我尝试运行一下。

  1. 安装Hadoop

  2. 从下载Mahout并解压

  3. 准备数据

    下载1 Million MovieLens Dataset,解压得到ratings.dat,用

    sed 's/:😦[0-9]{1,}):😦[0-9]{1})::[0-9]{1,}$/,\1,\2/' ratings.dat

    处理成需要的格式。

  4. 运行

    mahout recommenditembased -s SIMILARITY_LOGLIKELIHOOD -i /path/to/input/file -o /path/to/desired/output -n 25

    参数:

    MAHOUT-JOB: /home/laxe/apple/mahout/mahout-examples-0.11.0-job.jar
    Job-Specific Options:
    --input (-i) input Path to job input directory.
    --output (-o) output The directory pathname for output.
    --numRecommendations (-n) numRecommendations Number of recommendations per user.
    --usersFile usersFile File of users to recommend for.
    --itemsFile itemsFile File of items to recommend for.
    --filterFile (-f) filterFile File containing comma-separated userID,itemID pairs. Used to exclude the item from the recommendations for that user(optional).
    --userItemFile (-uif) userItemFile File containing comma-separated userID,itemID pairs(optional). Used to include only these items into recommendations. Cannot be used together with usersFile or itemsFile.
    --booleanData (-b) booleanData Treat input as without prefvalues.
    --maxPrefsPerUser (-mxp) maxPrefsPerUser Maximum number of preferences considered per user in final recommendation phase.
    --minPrefsPerUser (-mp) minPrefsPerUser Ignore users with less preferences than this in the similarity computation (default: 1).
    --maxSimilaritiesPerItem (-m) maxSimilaritiesPerItem Maximum number of similarities considered per item.
    --maxPrefsInItemSimilarity (-mpiis) maxPrefsInItemSimilarity Max number of preferences to consider per user or item in the item similarity computation phase, users or items with more preferences will be sampled down(default: 500).
    --similarityClassname (-s) similarityClassname Name of distributed similarity measures class to instantiate,
    alternatively use one of the predefined similarities([SIMILARITY_COOCCURRENCE, SIMILARITY_LOGLIKELIHOOD, SIMILARITY_TANIMOTO_COEFFICIENT, SIMILARITY_CITY_BLOCK, SIMILARITY_COSINE, SIMILARITY_PEARSON_CORRELATION, SIMILARITY_EUCLIDEAN_DISTANCE])
    --threshold (-tr) threshold Discard item pairs with a similarity value below this.
    --outputPathForSimilarityMatrix (-opfsm) outputPathForSimilarityMatrix Write the items imilarity matrix to this path(optional).
    --randomSeed randomSeed Use this seed for sampling.
    --sequencefileOutput Write the output into a Sequence File instead of a text file.
    --help (-h) Print out help.
    --tempDir tempDir Intermediate output directory.
    --startPhase startPhase First phase to run.
    --endPhase endPhase Last phase to run specify HDFS directories while running on hadoop; else specify local file system directories.

参考
Introduction to Item-Based Recommendations with Hadoop
mahout分布式:Item-based推荐

相关推荐
理智的煎蛋28 分钟前
CentOS/Ubuntu安装显卡驱动与GPU压力测试
大数据·人工智能·ubuntu·centos·gpu算力
赵孝正1 小时前
GitLab 分支管理与 Push 问题全解析
大数据·elasticsearch·gitlab
JAVA学习通1 小时前
【RabbitMQ】---RabbitMQ 工作流程和 web 界面介绍
分布式·rabbitmq
嘉禾望岗5031 小时前
Yarn介绍与HA搭建
大数据·hadoop·yarn
小麦矩阵系统永久免费1 小时前
小麦矩阵系统:让短视频分发实现抖音快手小红书全覆盖
大数据·人工智能·矩阵
IT研究室2 小时前
大数据毕业设计选题推荐-基于大数据的国家药品采集药品数据可视化分析系统-Spark-Hadoop-Bigdata
大数据·hadoop·信息可视化·spark·毕业设计·数据可视化·bigdata
安卓开发者2 小时前
鸿蒙NEXT应用数据持久化全面解析:从用户首选项到分布式数据库
数据库·分布式·harmonyos
Lx3523 小时前
Hadoop性能瓶颈分析:从JVM到磁盘IO的全链路优化
大数据·hadoop
大数据点灯人3 小时前
【Flink】Flink Runtime 开发指南
大数据·flink
一个java开发3 小时前
distributed.client.Client 用户可调用函数分析
大数据·python