昇腾环境300v pro 搭建qwen3 vl

news2026/5/24 8:54:53

1.启动dockerdocker run -itd \--name qwen-vl-serve \--nethost \--device/dev/davinci0 \--device/dev/davinci_manager \--device/dev/devmm_svm \--device/dev/hisi_hdc \-v /home/zhouty/Qwen3-VL-8B-Instruct:/workspace/models \-v /usr/local/Ascend/driver:/usr/local/Ascend/driver \quay.io/ascend/vllm-ascend:v0.18.0-310p-openeuler \/bin/bash2.启动服务export TORCH_COMPILE_DISABLE1export VLLM_USE_V10export VLLM_ASCEND_DISABLE_DYNAMIC_QUANT1vllm serve /workspace/models \--dtype float16 \--host 0.0.0.0 \--port 8000 \--tensor-parallel-size 1 \--trust-remote-code \--max-model-len 81923.双卡的启动docker run -itd \--name qwen-vl-serve \--nethost \--device/dev/davinci0 \--device/dev/davinci2 \--device/dev/davinci_manager \--device/dev/devmm_svm \--device/dev/hisi_hdc \-e ASCEND_RT_VISIBLE_DEVICES0,1 \-v /home/zhouty/Qwen3-VL-8B-Instruct:/workspace/models \-v /usr/local/Ascend/driver:/usr/local/Ascend/driver \quay.io/ascend/vllm-ascend:v0.18.0-310p-openeuler \/bin/bashexport TORCH_COMPILE_DISABLE1export VLLM_USE_V10export VLLM_ASCEND_DISABLE_DYNAMIC_QUANT1vllm serve /workspace/models \--dtype float16 \--host 0.0.0.0 \--port 8000 \--tensor-parallel-size 1 \--trust-remote-code \--max-model-len 8192

本文来自互联网用户投稿，该文观点仅代表作者本人，不代表本站立场。本站仅提供信息存储空间服务，不拥有所有权，不承担相关法律责任。如若转载，请注明出处：http://www.coloradmin.cn/o/2637126.html

如若内容造成侵权/违法违规/事实不符，请联系多彩编程网进行投诉反馈，一经查实，立即删除！