Item-Based Recommendations with Hadoop

Mahout在MapReduce上实现了Item-Based Collaborative Filtering,这里我尝试运行一下。

  1. 安装Hadoop

  2. 从下载Mahout并解压

  3. 准备数据

    下载1 Million MovieLens Dataset,解压得到ratings.dat,用

    sed 's/:😦[0-9]{1,}):😦[0-9]{1})::[0-9]{1,}$/,\1,\2/' ratings.dat

    处理成需要的格式。

  4. 运行

    mahout recommenditembased -s SIMILARITY_LOGLIKELIHOOD -i /path/to/input/file -o /path/to/desired/output -n 25

    参数:

    MAHOUT-JOB: /home/laxe/apple/mahout/mahout-examples-0.11.0-job.jar
    Job-Specific Options:
    --input (-i) input Path to job input directory.
    --output (-o) output The directory pathname for output.
    --numRecommendations (-n) numRecommendations Number of recommendations per user.
    --usersFile usersFile File of users to recommend for.
    --itemsFile itemsFile File of items to recommend for.
    --filterFile (-f) filterFile File containing comma-separated userID,itemID pairs. Used to exclude the item from the recommendations for that user(optional).
    --userItemFile (-uif) userItemFile File containing comma-separated userID,itemID pairs(optional). Used to include only these items into recommendations. Cannot be used together with usersFile or itemsFile.
    --booleanData (-b) booleanData Treat input as without prefvalues.
    --maxPrefsPerUser (-mxp) maxPrefsPerUser Maximum number of preferences considered per user in final recommendation phase.
    --minPrefsPerUser (-mp) minPrefsPerUser Ignore users with less preferences than this in the similarity computation (default: 1).
    --maxSimilaritiesPerItem (-m) maxSimilaritiesPerItem Maximum number of similarities considered per item.
    --maxPrefsInItemSimilarity (-mpiis) maxPrefsInItemSimilarity Max number of preferences to consider per user or item in the item similarity computation phase, users or items with more preferences will be sampled down(default: 500).
    --similarityClassname (-s) similarityClassname Name of distributed similarity measures class to instantiate,
    alternatively use one of the predefined similarities([SIMILARITY_COOCCURRENCE, SIMILARITY_LOGLIKELIHOOD, SIMILARITY_TANIMOTO_COEFFICIENT, SIMILARITY_CITY_BLOCK, SIMILARITY_COSINE, SIMILARITY_PEARSON_CORRELATION, SIMILARITY_EUCLIDEAN_DISTANCE])
    --threshold (-tr) threshold Discard item pairs with a similarity value below this.
    --outputPathForSimilarityMatrix (-opfsm) outputPathForSimilarityMatrix Write the items imilarity matrix to this path(optional).
    --randomSeed randomSeed Use this seed for sampling.
    --sequencefileOutput Write the output into a Sequence File instead of a text file.
    --help (-h) Print out help.
    --tempDir tempDir Intermediate output directory.
    --startPhase startPhase First phase to run.
    --endPhase endPhase Last phase to run specify HDFS directories while running on hadoop; else specify local file system directories.

参考
Introduction to Item-Based Recommendations with Hadoop
mahout分布式:Item-based推荐

相关推荐
Yvonne9782 小时前
创建三个节点
java·大数据
bug404_4 小时前
分布式大语言模型服务引擎vLLM论文解读
人工智能·分布式·语言模型
OJAC近屿智能5 小时前
苹果新品今日发布,AI手机市场竞争加剧,近屿智能专注AI人才培养
大数据·人工智能·ai·智能手机·aigc·近屿智能
lucky_syq5 小时前
Spark算子:大数据处理的魔法棒
大数据·分布式·spark
昔我往昔7 小时前
项目中分库分表的分布式ID如何生成
分布式
m0_748233648 小时前
【分布式】Hadoop完全分布式的搭建(零基础)
大数据·hadoop·分布式
圣享科技SMARTLIC8 小时前
企业软件合规性管理:构建高效、安全的软件资产生态
大数据·安全·浮动许可证监控·许可证管理·浮动许可证优化·软件资产管理·浮动许可证管理
roman_日积跬步-终至千里8 小时前
【分布式理论14】分布式数据库存储:分表分库、主从复制与数据扩容策略
数据库·分布式
京东零售技术8 小时前
京东广告基于 Apache Doris 的冷热数据分层实践
大数据
D愿你归来仍是少年9 小时前
解决Python升级导致PySpark任务异常方案
大数据·开发语言·python·spark