文章目录
-
-
- [1.anaconda install](#1.anaconda install)
- 2.model
-
1.anaconda install
https://anaconda.com/download/success
https://www.anaconda.com/docs/getting-started/anaconda/install#macos-linux-installation
cmd
# 打开终端，进入下载目录（假设安装包下载到 Downloads 目录）
cd ~/Downloads
# 给安装脚本添加执行权限
chmod +x Anaconda3-xxxx.xx-MacOSX-x86_64.sh # 请将 Anaconda3-xxxx.xx-MacOSX-x86_64.sh 替换为实际下载的文件名
# 执行安装脚本
./Anaconda3-xxxx.xx-MacOSX-x86_64.sh # 请将 Anaconda3-xxxx.xx-MacOSX-x86_64.sh 替换为实际下载的文件名
# 安装过程中会提示你阅读许可协议，按空格键翻页，阅读完后输入 yes 同意协议
# 之后会提示你选择安装路径，默认即可，按回车键确认
# 最后会询问是否初始化 Anaconda，输入 yes 完成初始化
conda --version2.model
https://www.modelscope.cn/home
https://github.com/modelscope/modelscope/blob/master/README_zh.md
cmd
#!/usr/bin/python
# -*- coding: UTF-8 -*-
conda create -n modelscope python=3.8
conda activate modelscope
(modelscope) ➜ modelscope python modelscope_demo.py
2025-07-18 17:16:49,118 - modelscope - WARNING - Model revision not specified, use revision: v1.0.0
Downloading Model from https://www.modelscope.cn to directory: /Users/wgg/.cache/modelscope/hub/models/damo/nlp_raner_named-entity-recognition_chinese-base-ecom
2025-07-18 17:16:50,390 - modelscope - WARNING - Model revision not specified, use revision: v1.0.0
2025-07-18 17:16:50,590 - modelscope - INFO - initiate model from /Users/wgg/.cache/modelscope/hub/models/damo/nlp_raner_named-entity-recognition_chinese-base-ecom
2025-07-18 17:16:50,590 - modelscope - INFO - initiate model from location /Users/wgg/.cache/modelscope/hub/models/damo/nlp_raner_named-entity-recognition_chinese-base-ecom.
2025-07-18 17:16:50,592 - modelscope - INFO - initialize model from /Users/wgg/.cache/modelscope/hub/models/damo/nlp_raner_named-entity-recognition_chinese-base-ecom
2025-07-18 17:16:52,224 - modelscope - INFO - head has no _keys_to_ignore_on_load_missing
2025-07-18 17:16:52,632 - modelscope - INFO - All model checkpoint weights were used when initializing ModelForTokenClassificationWithCRF.
2025-07-18 17:16:52,632 - modelscope - INFO - All the weights of ModelForTokenClassificationWithCRF were initialized from the model checkpoint If your task is similar to the task the model of the checkpoint was trained on, you can already use ModelForTokenClassificationWithCRF for predictions without further training.
2025-07-18 17:16:52,635 - modelscope - WARNING - No preprocessor field found in cfg.
2025-07-18 17:16:52,635 - modelscope - WARNING - No val key and type key found in preprocessor domain of configuration.json file.
2025-07-18 17:16:52,635 - modelscope - WARNING - Cannot find available config to build preprocessor at mode inference, current config: {'model_dir': '/Users/wgg/.cache/modelscope/hub/models/damo/nlp_raner_named-entity-recognition_chinese-base-ecom'}. trying to build by task and model information.
2025-07-18 17:16:52,641 - modelscope - INFO - cuda is not available, using cpu instead.
2025-07-18 17:16:52,642 - modelscope - WARNING - No preprocessor field found in cfg.
2025-07-18 17:16:52,642 - modelscope - WARNING - No val key and type key found in preprocessor domain of configuration.json file.
2025-07-18 17:16:52,642 - modelscope - WARNING - Cannot find available config to build preprocessor at mode inference, current config: {'model_dir': '/Users/wgg/.cache/modelscope/hub/models/damo/nlp_raner_named-entity-recognition_chinese-base-ecom', 'sequence_length': 512}. trying to build by task and model information.
2025-07-18 17:16:52,650 - modelscope - WARNING - No preprocessor field found in cfg.
2025-07-18 17:16:52,650 - modelscope - WARNING - No val key and type key found in preprocessor domain of configuration.json file.
2025-07-18 17:16:52,650 - modelscope - WARNING - Cannot find available config to build preprocessor at mode inference, current config: {'model_dir': '/Users/wgg/.cache/modelscope/hub/models/damo/nlp_raner_named-entity-recognition_chinese-base-ecom', 'sequence_length': 512}. trying to build by task and model information.
* Serving Flask app 'modelscope_demo'
* Debug mode: off
WARNING: This is a development server. Do not use it in a production deployment. Use a production WSGI server instead.
* Running on all addresses (0.0.0.0)
* Running on http://127.0.0.1:9080
* Running on http://10.168.88.14:9080
Press CTRL+C to quit
/Users/wgg/anaconda3/envs/modelscope/lib/python3.8/site-packages/transformers/modeling_utils.py:1161: FutureWarning: The `device` argument is deprecated and will be removed in v5 of Transformers.
warnings.warn(
--------ner-------
{'output': [{'type': 'HP', 'start': 0, 'end': 2, 'prob': 0.6007019, 'span': '苹果'}, {'type': 'HC', 'start': 10, 'end': 16, 'prob': 0.16767623, 'span': 'iPhone'}, {'type': 'XH', 'start': 17, 'end': 19, 'prob': 0.6483761, 'span': '15'}]}
--------ner-------
curl -X POST http://0.0.0.0:9080/ner \
-H "Content-Type: application/json" \
py
#!/usr/bin/env python
import warnings
warnings.filterwarnings("ignore", category=UserWarning, module="modelscope.utils.plugins", message="pkg_resources is deprecated as an API")
# pip install modelscope flask numpy pandas datasets pyarrow addict -i https://pypi.tuna.tsinghua.edu.cn/simple
from flask import Flask, request, jsonify
from modelscope.pipelines import pipeline
from modelscope.utils.constant import Tasks
from modelscope.models import Model
from modelscope.preprocessors import TextRankingTransformersPreprocessor
from modelscope.preprocessors import TokenClassificationTransformersPreprocessor
import json
import numpy as np
import pandas as pd
# 1.命名实体识别任务
def simple() -> None:
# 创建命名实体识别任务的管道
ner_pipeline = pipeline(Tasks.named_entity_recognition, model='damo/nlp_raner_named-entity-recognition_chinese-base-ecom')
# 输入文本
text = "苹果公司发布了新款 iPhone 15。"
# 执行命名实体识别
result = ner_pipeline(text)
print("--------simple-------")
print(result)
# 2. 文本分词任务
def tokenize() -> None:
# 加载预训练模型
model_id = 'damo/nlp_structbert_word-segmentation_chinese-base-ecommerce'
model = Model.from_pretrained(model_id)
# 创建预处理器
preprocessor = TokenClassificationTransformersPreprocessor(model.model_dir)
# 创建分词任务的管道
seg_pipeline = pipeline(task=Tasks.word_segmentation, model=model, preprocessor=preprocessor)
# 输入文本
text = "苹果公司发布了新款 iPhone 15。"
# 执行分词任务
result = seg_pipeline(text)
print("--------tokenize-------")
print(result)
app = Flask(__name__)
# 创建命名实体识别任务的管道
ner_pipeline_http = pipeline(Tasks.named_entity_recognition, model='damo/nlp_raner_named-entity-recognition_chinese-base-ecom')
@app.route('/ner', methods=['POST'])
def ner():
data = request.get_json()
text = data.get('text', '')
result = ner_pipeline_http(text)
print(result)
return jsonify(result)
if __name__ == "__main__":
#simple()
#tokenize()
#app.run(debug=True)
app.run(host='0.0.0.0', port=9080)