MindIE 基于昇腾910B2 aarch64环境profile

MindIE镜像获取

https://www.hiascend.com/developer/ascendhub/detail/af85b724a7e5469ebd7ea13c3439d48f

MindIE使用说明

https://www.hiascend.com/document/detail/zh/mindie/10RC2/mindiellm/llmdev/mindie_llm0004.html

docker生成和启动

编写 docker 启动脚本 start-docker.sh

IMAGES_ID=$1
NAME=$2
if [ $# -ne 2 ]; then
    echo "error: need one argument describing your container name."
    exit 1
fi
docker run --name ${NAME} -it -d --net=host --shm-size=500g \
    --privileged=true \
    -w /home \
    --device=/dev/davinci_manager \
    --device=/dev/hisi_hdc \
    --device=/dev/devmm_svm \
    --entrypoint=bash \
    -v /usr/local/Ascend/driver:/usr/local/Ascend/driver \
    -v /usr/local/dcmi:/usr/local/dcmi \
    -v /usr/local/bin/npu-smi:/usr/local/bin/npu-smi \
    -v /usr/local/sbin:/usr/local/sbin \
    -v /home:/home \
    -v /tmp:/tmp \
    -v /usr/share/zoneinfo/Asia/Shanghai:/etc/localtime \
    -e http_proxy=$http_proxy \
    -e https_proxy=$https_proxy \
    ${IMAGES_ID}

参数说明：

• IMAGES_ID 为镜像版本号。（docker images 命令回显中的 IMAGES ID）
• NAME 为启动容器名，可自定义设置。

启动容器(确保启动容器前，本机可访问外网)

bash start-docker.sh ${image_id} ${custom_image_name}# 进入容器
docker exec -itu root ${custom_image_name} bash

环境变量配置

cd /opt/package

安装CANN包

source install_and_enable_cann.sh

若退出后重新进入容器，则需要重新加载 CANN 环境变量，执行以下四行命令

source /usr/local/Ascend/ascend-toolkit/set_env.sh
source /usr/local/Ascend/nnal/atb/set_env.sh
source /usr/local/Ascend/mindie/set_env.sh
source /usr/local/Ascend/llm_model/set_env.sh

CHECKPOINT=/home/zhaojiacheng/workspace/weight/Qwen2-7B-bak

权重文件转换（可选）

当前ATB Models 推理仅支持加载safetensor格式的权重文件，若下载的权重文件中已有safetensor格式的文件，则无需进行权重转换，若环境中仅有bin格式的权重文件，请按照如下方式进行转换。

cd /usr/local/Ascend/llm_model

权重转 safetensor

python examples/convert/convert_weights.py --model_path ${CHECKPOINT}

执行对话推理脚本

执行推理脚本

python examples/run_pa.py --model_path ${CHECKPOINT}

默认推理内容为"What's deep learning?"，batch size为1，output_len为20，修改配置在run_pa.py文件中

Profiling 长度1024tokens的问题，因为是超长输入text，msprof会报错，因此需要将texts文字写入到examples/run_pa.py下面，

In the tapestry of human history, threads of invention, exploration, and cultural fusion intertwine to form a narrative rich in complexity and diversity. From the ancient civilizations that laid the foundations of modern society such as the Egyptians, who mastered the art of irrigation and monumental architecture, to the Greeks, whose philosophical inquiries and democratic ideals still resonate today the march of progress has been relentless and multifaceted. The Roman Empire, with its unparalleled network of roads, legal systems, and military strategies, exemplified the heights of administrative and organizational prowess, setting the stage for the medieval period. This era, often misconstrued as dark and stagnant, was, in reality, a crucible of innovation and cross-cultural exchange, as seen in the Silk Road's bustling trade and the scholarly pursuits within the Islamic Golden Age. The Renaissance, a rebirth of art and knowledge, emerged from the depths of the Middle Ages, casting a spotlight on humanism and reviving interest in classical antiquity. This period saw luminaries like Leonardo da Vinci and Michelangelo break new ground in art and science, embodying the spirit of inquiry and the pursuit of excellence. The Industrial Revolution marked another pivotal turn, transforming societies with the power of steam, mechanization, and later, electricity. The rapid pace of change brought about by inventors and entrepreneurs like James Watt and Thomas Edison reshaped daily life, economies, and the environment, ushering in the modern era. The 20th century, with its world wars, technological advancements, and social movements, highlighted humanity's capacity for both destruction and profound creativity. The advent of the digital age, led by visionaries such as Steve Jobs and Tim Berners-Lee, has connected the world in ways previously unimaginable, making information and communication virtually instantaneous. As we gaze into the future, the challenges of climate change, resource scarcity, and technological ethics loom large. Yet, the story of human progress is one of resilience and innovation. The quest for sustainable development, the exploration of space, and the advancements in artificial intelligence and biotechnology present new frontiers for humanity to explore. This narrative, weaving through millennia, showcases the indomitable spirit of human curiosity and the relentless pursuit of knowledge. As we stand on the shoulders of giants, we look forward to writing the next chapters of this ongoing saga, hopeful that the future will reflect the best of our collective endeavors. In this continuing saga, the emphasis shifts towards a more interconnected and technologically advanced world. The 21st century has ushered in an era where the digital realm and artificial intelligence not only augment human capabilities but also challenge our ethical frameworks and societal norms. Innovations in machine learning and AI have the potential to revolutionize industries, from healthcare, where personalized medicine and robotic surgery become the norm, to environmental science, with predictive models and solutions for combating climate change. However, these advancements are not without their quandaries. The digital divide and issues of privacy, surveillance, and data security raise critical questions about the kind of society we wish to cultivate. The ethical use of technology, equitable access to its benefits, and the safeguarding of individual rights become paramount in ensuring that progress does not come at the cost of human dignity or freedom. Simultaneously, the specter of climate change and environmental degradation demands a reevaluation of our relationship with the natural world. The pursuit of sustainable living, renewable energy sources, and green technologies is more than an environmental imperative it's a moral one. As we harness the power of wind, solar, and bioengineering, we embark on a path that seeks harmony with the planet, aspiring to leave a livable earth for future generations. Moreover, the exploration of space represents the boundless curiosity of the human spirit. The successful landing of rovers on Mars, the search for extraterrestrial life, and plans for human colonization of other planets speak to our unyielding desire to understand our place in the universe. These endeavors not only push the boundaries of science and technology but also ignite imaginations, inspiring a sense of wonder and possibility. Yet, amidst these strides in science and technology, the cultural and social fabric of humanity continues to evolve. The global exchange of ideas, facilitated by the internet, has led to a cultural renaissance where art, music, and literature transcend geographical boundaries, fostering a global community that celebrates diversity yet acknowledges the challenges of integration and identity. As we navigate the complexities of the 21st century, the story of humanity is one of adaptation and resilience. The challenges we face, from ethical dilemmas posed by technology to the existential threat of climate change, are daunting. However, history has shown that in moments of crisis, there lies opportunity for growth and transformation. The future is a canvas yet to be painted, and it is up to us, collectively, to decide the colors and shapes of the world to come. With wisdom gleaned from the past and an eye towards the future, we step forward, ready to contribute our verses to the enduring epic of human civilization. As the narrative of humanity unfurls into the latter ha

然后 执行如下命令

 msprof --application="python examples/run_pa.py --model_path ${CHECKPOINT} " --output=/home/workspace/mindie/qwen2_7b_1024input_1output_no_warmup_batch8/qwen2_7b_1024input_128output_no_warmup_batch8_test1

数据分析

1024input 128output nowarmup

测试1时长(ms)

测试2时长

测试3时长

平均时长

1024input 128output nowarmup

=3309-2915

=3481-3087

=4691-4288

test1

Start time

2024-10-18 16:26:52,915 INFO pid: 148067 logging.py-53: ---------------begin inference---------------

Endtime

2024-10-18 16:26:53,309 INFO pid: 148067 logging.py-53: ---------------end inference---------------

test2

Start time

2024-10-18 16:27:33,087 INFO pid: 149071 logging.py-53: ---------------begin inference---------------

Endtime

2024-10-18 16:27:33,481 INFO pid: 149071 logging.py-53: ---------------end inference---------------

test3

Start time

2024-10-18 16:28:04,288 INFO pid: 150075 logging.py-53: ---------------begin inference---------------

Endtime

2024-10-18 16:28:04,691 INFO pid: 150075 logging.py-53: ---------------end inference---------------

QA

1. 默认状态会报python依赖的错误

pip install transformers -U

2. childFailedError

pip3 install pytz
pip3 install datautils