torch-xpu-run
Run an arbitrary Hugging Face safetensors model on an Intel GPU using **upstream PyTorch** (>= 2.8) with the built-in `torch.xpu` device. Covers loading from the Hub, picking the right dtype, autocast, multi-GPU with accelerate's `device_map`, and the CUDA -> XPU code translation a user has to do once. Use for the Transformers / Accelerate / Diffusers path. Not for OpenAI-compatible serving (use vllm-xpu-run); explicitly not via intel-extension-for-pytorch (ipex) or ipex-llm — those paths are end-of-life and upstream PyTorch supersedes them.
intel · 已有静态检查记录
安装与使用
运行命令前先阅读来源与依赖说明。复制命令不会自动安装或执行。
npx skills add intel/gpu-ai-skills --skill torch-xpu-run技能说明
name: torch-xpu-run
description: Run an arbitrary Hugging Face safetensors model on an Intel GPU using upstream PyTorch (>= 2.8) with the built-in torch.xpu device. Covers loading from the Hub, picking the right dtype, autocast, multi-GPU with accelerate's device_map, and the CUDA -> XPU code translation a user has to do once. Use for the Transformers / Accelerate / Diffusers path. Not for OpenAI-compatible serving (use vllm-xpu-run); explicitly not via intel-extension-for-pytorch (ipex) or ipex-llm — those paths are end-of-life and upstream PyTorch supersedes them.
torch-xpu-run
Upstream PyTorch has a torch.xpu namespace mirroring torch.cuda
(prototype since 2.5; this skill assumes >= 2.8, which is where the
native xccl collective backend and the coverage below are dependable).
Don't use intel-extension-for-pytorch
(ipex) or ipex-llm — both are end-of-life (March 2026), upstream
PyTorch supersedes them.
CUDA -> XPU code translation
| CUDA | XPU |
|---|---|
torch.cuda.is_available() | torch.xpu.is_available() |
torch.cuda.device_count() | torch.xpu.device_count() |
torch.cuda.empty_cache() | torch.xpu.empty_cache() |
torch.cuda.synchronize() | torch.xpu.synchronize() |
torch.cuda.memory_allocated(0) | torch.xpu.memory_allocated(0) |
model.to("cuda") | model.to("xpu") |
tensor.to("cuda:1") | tensor.to("xpu:1") |
with torch.autocast("cuda", torch.bfloat16) | with torch.autocast("xpu", torch.bfloat16) |
torch.cuda.amp.GradScaler() | torch.amp.GradScaler("xpu") — needs FP64 support, so disable it (enabled=False) on Arc A-Series, which lacks native FP64 |
device_map="auto" (Accelerate) | same; Accelerate detects XPU directly |
dist.init_process_group(backend="nccl") | dist.init_process_group(backend="xccl") <- only non-mechanical change |
Where to get PyTorch with XPU
XPU wheels are not on the default PyPI index. A plain
pip install torch gets the CUDA/CPU build, where torch.xpu exists
as a namespace but reports no devices. Install from the XPU index:
# stable
pip3 install torch torchvision torchaudio \
--index-url https://download.pytorch.org/whl/xpu
# nightly — only when you need an unreleased fix
pip3 install --pre torch torchvision torchaudio \
--index-url https://download.pytorch.org/whl/nightly/xpu
Pinning works the same way, e.g. pip install torch==2.11.0 torchvision==0.26.0 torchaudio==2.11.0 --index-url https://download.pytorch.org/whl/xpu. Take the three versions from one
release row — mixing rows breaks the ABI.
The Intel GPU driver must already be installed on the host
(xpu-discover / xpu-runtime-preflight verify this). Binary wheels do
not need Intel Deep Learning Essentials; only source builds do.
Verify:
python3 -c "import torch; print(torch.__version__, torch.xpu.is_available(), torch.xpu.device_count())"
torch.xpu.is_available() is True is the authoritative signal. Wheels
from the XPU index also carry a +xpu local version suffix (e.g.
2.13.0+xpu), as do the vendor serving images, but a PyTorch built from
source does not unless the build sets it — so treat a missing suffix as
a prompt to check is_available(), not as proof XPU is absent.
is_available() returning False on XPU hardware almost always means a
missing host driver or a container that can't see /dev/dri (that case
also logs XPU device count is zero!).
Three options:
- Local venv — same install command, no container.
- Build a thin Dockerfile on
ubuntu:24.04orpython:3.12-slimand run the XPU-index install above. Canonical docs are the PyTorch XPU notes: https://docs.pytorch.org/docs/stable/notes/get_start_xpu.html (the https://pytorch.org/get-started/locally/ selector emits the same command once you pick Linux + Pip + Python + Intel GPU, but it is JS-rendered and shows nothing XPU-related when fetched as text). - Reuse a serving image —
vllm/vllm-openai-xpu:latestships a working torch-xpu inside; start with--entrypoint /bin/bash.
Launch with the GPU visible (see xpu-container-run for full flags):
docker run --rm -it \
--device /dev/dri \
--ipc=host \
-e ZE_AFFINITY_MASK=0 \
-e HF_TOKEN="$HF_TOKEN" \
-v "$HOME/.cache/huggingface:/root/.cache/huggingface" \
--entrypoint /bin/bash \
<torch-xpu-image>
Quickstart
STOP — confirm before proceeding. Before installing packages or downloading weights, ask the user to confirm:
- The model ID (and dtype if not bf16)
- That installing packages (torch from the XPU index plus
transformers/accelerate) and downloading multi-GB weights is acceptable
Do not run pip install, uv pip install, or model download commands
until the user explicitly confirms. This is a hard requirement.
Check before installing. Always verify packages are already present
before running pip install:
python3 -c "import torch; print(torch.__version__, torch.xpu.is_available())" 2>&1 && \
python3 -c "import transformers; print(transformers.__version__)" 2>&1 && \
python3 -c "import accelerate; print(accelerate.__version__)" 2>&1
Only install what the import check reports missing:
pip install --quiet --break-system-packages 'transformers>=4.56' accelerate
Never add torch to that line — it must come from the XPU index (see
Where to get PyTorch with XPU), and an unpinned pip install
alongside other packages can silently replace a working +xpu build
with the PyPI CUDA/CPU wheel.
(--break-system-packages is needed under PEP 668 in Ubuntu
24.04+; omit in older images, or use a venv.)
from transformers import AutoConfig, AutoModelForCausalLM, AutoTokenizer
import torch
model_id = "Qwen/Qwen2.5-1.5B-Instruct"
# Fetch config first to apply pre-load patches.
cfg = AutoConfig.from_pretrained(model_id)
# Rope-scaling: older models omit the 'type' field required by
# Transforme
阅读完整来源 ↗