跳至正文
OpenAgentSkill
浏览已开放 · 目录记录日期 2026-11-21 · 自动更新和账户服务暂时关闭。
AI 与知识库

torch-xpu-run

Run an arbitrary Hugging Face safetensors model on an Intel GPU using **upstream PyTorch** (>= 2.8) with the built-in `torch.xpu` device. Covers loading from the Hub, picking the right dtype, autocast, multi-GPU with accelerate's `device_map`, and the CUDA -> XPU code translation a user has to do once. Use for the Transformers / Accelerate / Diffusers path. Not for OpenAI-compatible serving (use vllm-xpu-run); explicitly not via intel-extension-for-pytorch (ipex) or ipex-llm — those paths are end-of-life and upstream PyTorch supersedes them.

intel · 已有静态检查记录

安装与使用

运行命令前先阅读来源与依赖说明。复制命令不会自动安装或执行。

npx skills add intel/gpu-ai-skills --skill torch-xpu-run

技能说明


name: torch-xpu-run description: Run an arbitrary Hugging Face safetensors model on an Intel GPU using upstream PyTorch (>= 2.8) with the built-in torch.xpu device. Covers loading from the Hub, picking the right dtype, autocast, multi-GPU with accelerate's device_map, and the CUDA -> XPU code translation a user has to do once. Use for the Transformers / Accelerate / Diffusers path. Not for OpenAI-compatible serving (use vllm-xpu-run); explicitly not via intel-extension-for-pytorch (ipex) or ipex-llm — those paths are end-of-life and upstream PyTorch supersedes them.

torch-xpu-run

Upstream PyTorch has a torch.xpu namespace mirroring torch.cuda (prototype since 2.5; this skill assumes >= 2.8, which is where the native xccl collective backend and the coverage below are dependable). Don't use intel-extension-for-pytorch (ipex) or ipex-llm — both are end-of-life (March 2026), upstream PyTorch supersedes them.

CUDA -> XPU code translation

CUDAXPU
torch.cuda.is_available()torch.xpu.is_available()
torch.cuda.device_count()torch.xpu.device_count()
torch.cuda.empty_cache()torch.xpu.empty_cache()
torch.cuda.synchronize()torch.xpu.synchronize()
torch.cuda.memory_allocated(0)torch.xpu.memory_allocated(0)
model.to("cuda")model.to("xpu")
tensor.to("cuda:1")tensor.to("xpu:1")
with torch.autocast("cuda", torch.bfloat16)with torch.autocast("xpu", torch.bfloat16)
torch.cuda.amp.GradScaler()torch.amp.GradScaler("xpu") — needs FP64 support, so disable it (enabled=False) on Arc A-Series, which lacks native FP64
device_map="auto" (Accelerate)same; Accelerate detects XPU directly
dist.init_process_group(backend="nccl")dist.init_process_group(backend="xccl") <- only non-mechanical change

Where to get PyTorch with XPU

XPU wheels are not on the default PyPI index. A plain pip install torch gets the CUDA/CPU build, where torch.xpu exists as a namespace but reports no devices. Install from the XPU index:

# stable
pip3 install torch torchvision torchaudio \
    --index-url https://download.pytorch.org/whl/xpu

# nightly — only when you need an unreleased fix
pip3 install --pre torch torchvision torchaudio \
    --index-url https://download.pytorch.org/whl/nightly/xpu

Pinning works the same way, e.g. pip install torch==2.11.0 torchvision==0.26.0 torchaudio==2.11.0 --index-url https://download.pytorch.org/whl/xpu. Take the three versions from one release row — mixing rows breaks the ABI.

The Intel GPU driver must already be installed on the host (xpu-discover / xpu-runtime-preflight verify this). Binary wheels do not need Intel Deep Learning Essentials; only source builds do.

Verify:

python3 -c "import torch; print(torch.__version__, torch.xpu.is_available(), torch.xpu.device_count())"

torch.xpu.is_available() is True is the authoritative signal. Wheels from the XPU index also carry a +xpu local version suffix (e.g. 2.13.0+xpu), as do the vendor serving images, but a PyTorch built from source does not unless the build sets it — so treat a missing suffix as a prompt to check is_available(), not as proof XPU is absent. is_available() returning False on XPU hardware almost always means a missing host driver or a container that can't see /dev/dri (that case also logs XPU device count is zero!).

Three options:

  1. Local venv — same install command, no container.
  2. Build a thin Dockerfile on ubuntu:24.04 or python:3.12-slim and run the XPU-index install above. Canonical docs are the PyTorch XPU notes: https://docs.pytorch.org/docs/stable/notes/get_start_xpu.html (the https://pytorch.org/get-started/locally/ selector emits the same command once you pick Linux + Pip + Python + Intel GPU, but it is JS-rendered and shows nothing XPU-related when fetched as text).
  3. Reuse a serving image — vllm/vllm-openai-xpu:latest ships a working torch-xpu inside; start with --entrypoint /bin/bash.

Launch with the GPU visible (see xpu-container-run for full flags):

docker run --rm -it \
    --device /dev/dri \
    --ipc=host \
    -e ZE_AFFINITY_MASK=0 \
    -e HF_TOKEN="$HF_TOKEN" \
    -v "$HOME/.cache/huggingface:/root/.cache/huggingface" \
    --entrypoint /bin/bash \
    <torch-xpu-image>

Quickstart

STOP — confirm before proceeding. Before installing packages or downloading weights, ask the user to confirm:

  1. The model ID (and dtype if not bf16)
  2. That installing packages (torch from the XPU index plus transformers / accelerate) and downloading multi-GB weights is acceptable

Do not run pip install, uv pip install, or model download commands until the user explicitly confirms. This is a hard requirement.

Check before installing. Always verify packages are already present before running pip install:

python3 -c "import torch; print(torch.__version__, torch.xpu.is_available())" 2>&1 && \
python3 -c "import transformers; print(transformers.__version__)" 2>&1 && \
python3 -c "import accelerate; print(accelerate.__version__)" 2>&1

Only install what the import check reports missing:

pip install --quiet --break-system-packages 'transformers>=4.56' accelerate

Never add torch to that line — it must come from the XPU index (see Where to get PyTorch with XPU), and an unpinned pip install alongside other packages can silently replace a working +xpu build with the PyPI CUDA/CPU wheel.

(--break-system-packages is needed under PEP 668 in Ubuntu 24.04+; omit in older images, or use a venv.)

from transformers import AutoConfig, AutoModelForCausalLM, AutoTokenizer
import torch

model_id = "Qwen/Qwen2.5-1.5B-Instruct"

# Fetch config first to apply pre-load patches.
cfg = AutoConfig.from_pretrained(model_id)

# Rope-scaling: older models omit the 'type' field required by
# Transforme
阅读完整来源 ↗