pi0.5机器人VLA大模型昇腾使用指南【免费下载链接】cann-recipes-embodied-intelligence本项目针对具身智能业务中的典型模型、加速算法提供基于CANN平台的优化样例项目地址: https://gitcode.com/cann/cann-recipes-embodied-intelligencepi0.5整体介绍论文题目π0.5: a Vision-Language-Action Model with Open-World Generalization中文译文π0.5: 一种具备开放世界泛化能力的视觉–语言–动作模型功能介绍pi0.5一种基于pi0的新模型它通过在异构任务上进行协同训练co-training实现更广泛的泛化能力。pi0.5利用来自多种机器人平台的数据、高层语义预测、网络数据以及其他来源使其能够在真实世界的机器人操作任务中实现更强的通用性。它结合了协同训练与混合多模态样例这些样例将图像观测、语言指令、目标检测、语义子任务预测以及底层动作整合在一起通过知识迁移实现有效泛化。pi0.5首次展示一个端到端、由学习驱动的机器人系统能够在全新的家庭环境中执行长时序且高灵巧度的操作技能例如在完全陌生的住宅里完成厨房或卧室清洁等任务。pi0.5的相关代码仓拉取、数据集和模型下载# 进入需要放置代码仓的本地xxx目录下 cd xxx git clone https://gitcode.com/cann/cann-recipes-embodied-intelligence.git chmod x cann-recipes-embodied-intelligence/manipulation/pi05/infer_with_torch/download_code_and_data.sh ./cann-recipes-embodied-intelligence/manipulation/pi05/infer_with_torch/download_code_and_data.sh完成上述操作之后最终lerobot根目录中相关代码目录树详见附录lerobot根目录相关代码目录树。pi0.5在昇腾310P上的运行环境配置与昇腾服务器无关的环境配置# 创建运行环境 conda create -y -n lerobot python3.10 conda activate lerobot # 回到lerobot根目录安装lerobot。 cd lerobot pip install torch2.5.1 torchvision0.20.1 --index-url https://download.pytorch.org/whl/cpu pip install -e . pip install transformers githttps://github.com/huggingface/transformers.gitfix/lerobot_openpi与昇腾平台相关的环境配置安装CANN软件包。本样例的编译执行依赖CANN开发套件包cann-toolkit与CANN二进制算子包cann-kernels支持的CANN软件版本为CANN 8.2.RC1。 请从软件包下载地址下载Ascend-cann-toolkit_8.2.RC1_linux-x86_64.run与Ascend-cann-kernels-310p_8.2.RC1_linux-x86_64.run软件包并参考CANN安装文档依次进行安装。# ${cann_install_path}为CANN包的实际安装目录注意每次新建终端时首先source一下set_env.sh。 # 方式1默认路径安装以root用户为例 source /usr/local/Ascend/ascend-toolkit/set_env.sh # 方式2指定路径进行安装 source ${cann_install_path}/ascend-toolkit/set_env.sh # 在上述运行环境中继续安装对应版本torch-npu pip install numpy1.26.4 pip install torch_npu-2.5.1.post1 wget https://gitcode.com/Ascend/pytorch/releases/download/v7.1.0-pytorch2.5.1/torch_npu-2.5.1.post1-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl pip install torch_npu-2.5.1.post1-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whlpi0.5在昇腾上的推理步骤运行下面的代码即可自动构造mock输入进行pi05模型推理打印推理性能及机器人动作。# 进入lerobot代码仓根目录 cd lerobot conda activate lerobot chmod x run_pi05_inference.sh ./run_pi05_inference.sh pi05_model float16 1 3 npu基于上述运行过程得到pi05的单次推理时间及结果如下所示推理性能单次推理时间约860ms示例输出如下INFO - Starting inference timing (3 iterations)... INFO - ---------------------------------------- INFO - Inference Results for pi05_model INFO - Device: npu:0, Dtype: torch.float16 INFO - Action shape: torch.Size([1, 32]) INFO - Total time for 3 runs: 2.5864 s INFO - Average latency: 862.1430 ms INFO - Throughput: 1.16 FPS INFO - ----------------------------------------推理结果单次推理结果为50组机械臂关节角度序列shape为[50,32]每次推理后保存在queue中action输出一组。pi05在昇腾上的精度验证步骤基于mock的数据输入NPU与原始CPU/GPU Pytorch输出相似度对比构造固定输入如全0图像 固定指令 token测试 PyTorch CPU/GPU 和 310P NPU 的输出精度对比python verify_pi05_accuracy_ascend.py \ --pretrained_model_name_or_path pi05_model \ --device npu:0示例输出如下Global Cosine Similarity: 1.000000 Per-timestep Cosine Similarity: Step 0: 1.000000 ... Step 49: 0.999999 Minimum Per-step Similarity: 0.999999 Average Per-step Similarity: 0.999999 MSE Loss: 0.000000 Verification SUCCESS: All similarities 0.99可能遇到的问题运行推理时若使用网络环境下载google/paligemma-3b-pt-224模型需提前取模型对应的 huggingface 页面请求访问 Access参考详见https://huggingface.co/docs/huggingface_hub/main/cn/quick-start和https://huggingface.co/docs/hub/models-gated若网络环境下载huggingface模型较慢遇到下载google/paligemma-3b-pt-224卡顿可手动下载模型到本地路径再修改lerobot/src/lerobot/policies/pi05/processor_pi05.py中对应145行处google/paligemma-3b-pt-224为本地路径。Citationmisc{intelligence2025pi05visionlanguageactionmodelopenworld, title{$\pi_{0.5}$: a Vision-Language-Action Model with Open-World Generalization}, author{Physical Intelligence and Kevin Black and Noah Brown and James Darpinian and Karan Dhabalia and Danny Driess and Adnan Esmail and Michael Equi and Chelsea Finn and Niccolo Fusai and Manuel Y. Galliker and Dibya Ghosh and Lachy Groom and Karol Hausman and Brian Ichter and Szymon Jakubczak and Tim Jones and Liyiming Ke and Devin LeBlanc and Sergey Levine and Adrian Li-Bell and Mohith Mothukuri and Suraj Nair and Karl Pertsch and Allen Z. Ren and Lucy Xiaoyang Shi and Laura Smith and Jost Tobias Springenberg and Kyle Stachowicz and James Tanner and Quan Vuong and Homer Walke and Anna Walling and Haohuan Wang and Lili Yu and Ury Zhilinsky}, year{2025}, eprint{2504.16054}, archivePrefix{arXiv}, primaryClass{cs.LG}, url{https://arxiv.org/abs/2504.16054}, }附录lerobot根目录相关代码目录树检查整体代码目录树经过上述的复制及替换操作pi05适配昇腾的lerobot根目录中的最终相关代码目录树如下所示├── src # pi05模型训练及推理框架 | ├── lerobot | | ├── policies | | | ├── pi05 | | | | ├── modeling_pi05.py # pi05的模型训练及推理代码 ├── pi05_model # pi05 base模型 └── pyproject.toml # 运行环境第三方包的安装版本 └── README.md # 昇腾上运行pi05推理的环境配置及操作指导 └── run_pi05_inference.sh # 昇腾上运行pi05推理过程一键启动脚本 └── run_pi05_example.py # 昇腾上运行pi05推理示例代码 └── verify_pi05_accuracy_ascend.py # 昇腾上运行pi05推理结果精度验证代码 └── infer_utils.py # 推理与验证脚本共用工具函数【免费下载链接】cann-recipes-embodied-intelligence本项目针对具身智能业务中的典型模型、加速算法提供基于CANN平台的优化样例项目地址: https://gitcode.com/cann/cann-recipes-embodied-intelligence创作声明：本文部分内容由AI辅助生成（AIGC），仅供参考