跳至正文
OpenAgentSkill
浏览已开放 · 目录记录日期 2026-11-21 · 自动更新和账户服务暂时关闭。
浏览器与自动化

sglang-xpu-bench

Benchmark a **running SGLang-XPU server** on an Intel GPU using `sglang.bench_serving`. Measures TTFT, TPOT, ITL, end-to-end latency, and throughput against the OpenAI-compatible endpoint. Use after sglang-xpu-run. Not for vLLM servers (use vllm-xpu-bench) or no-server PyTorch (use torch-xpu-bench).

intel · 已有 AI 审核记录

安装与使用

运行命令前先阅读来源与依赖说明。复制命令不会自动安装或执行。

npx skills add intel/gpu-ai-skills --skill sglang-xpu-bench

技能说明


name: sglang-xpu-bench description: Benchmark a running SGLang-XPU server on an Intel GPU using sglang.bench_serving. Measures TTFT, TPOT, ITL, end-to-end latency, and throughput against the OpenAI-compatible endpoint. Use after sglang-xpu-run. Not for vLLM servers (use vllm-xpu-bench) or no-server PyTorch (use torch-xpu-bench).

sglang-xpu-bench

sglang.bench_serving is SGLang's online benchmark client (the counterpart to vllm bench serve). Speaks the OpenAI-compatible API exposed by sglang-xpu-run.

Step 0 — verify SGLang server is running on Intel XPU

REQUIRED: Before benchmarking, you must confirm a SGLang server is running, identify its port, and verify it's using Intel XPU (not CPU fallback, not NVIDIA). This prevents benchmarking a server that silently fell back to CPU.

Run the checks below in a single shell session (later blocks reuse $SGLANG_PID, $SGLANG_PORT, $MODEL from earlier ones).

Find the server process and its container:

# 1. Check if SGLang is running and find its container
SGLANG_PID=$(ps aux | grep -iE 'sglang|launch_server' | grep -v grep | awk 'NR==1 {print $2}')
if [ -z "$SGLANG_PID" ]; then
    echo "❌ No SGLang server found running. Start one with sglang-xpu-run."
    exit 1
fi
echo "✓ SGLang server found (PID: $SGLANG_PID)"

CONTAINER_NAME=$(docker ps --format '{{.Names}}' 2>/dev/null | while read name; do
    if docker top "$name" -o pid 2>/dev/null | awk 'NR>1' | grep -qxF "$SGLANG_PID"; then echo "$name"; break; fi
done)
if [ -z "$CONTAINER_NAME" ]; then
    echo "❌ No container matched PID $SGLANG_PID. This skill runs the bench client"
    echo "   via 'docker exec' because the sglang package is only installed inside"
    echo "   the server container. A host-only SGLang install is not supported here."
    exit 1
fi
echo "✓ SGLang container: $CONTAINER_NAME"

Read the launch args once — they give you both the container-internal port and the --device flag. The bench client runs via docker exec inside the container, so the port must be the one SGLang binds inside the container (host ss can't see the container-namespaced socket). The --device xpu check is REQUIRED — it catches a server that silently fell back to CPU:

# 2. Read port + device from the launch args in one pass (REQUIRED)
ARGS=$(ps -p "$SGLANG_PID" -o args=)
SGLANG_PORT=$(echo "$ARGS" | awk '{for(i=1;i<=NF;i++) if($i=="--port" && i<NF) print $(i+1)}' | head -1)
SGLANG_PORT=${SGLANG_PORT:-30000}   # sglang default
DEVICE_ARG=$(echo "$ARGS" | awk '{for(i=1;i<=NF;i++) if($i=="--device" && i<NF) print $(i+1)}' | head -1)
echo "DEVICE_ARG=${DEVICE_ARG:-none}"
if [ "$DEVICE_ARG" = "cuda" ]; then
    echo "❌ Server is on NVIDIA CUDA, not Intel XPU."; exit 1
elif [ "$DEVICE_ARG" = "xpu" ]; then
    echo "✓ SGLang server has --device xpu"
else
    echo "⚠️  Could not confirm --device xpu (got: '${DEVICE_ARG:-none}'); relying on XPU memory check."
fi

# 3. Verify reachable from inside the container and read model name
if ! docker exec "$CONTAINER_NAME" curl -s http://127.0.0.1:$SGLANG_PORT/v1/models >/dev/null 2>&1; then
    echo "❌ SGLang server not responding on container port $SGLANG_PORT."
    exit 1
fi
MODEL=$(docker exec "$CONTAINER_NAME" curl -s http://127.0.0.1:$SGLANG_PORT/v1/models | python3 -c "import sys,json; print(json.load(sys.stdin)['data'][0]['id'])")
echo "✓ Responding on container port $SGLANG_PORT — model: $MODEL"

Reject NVIDIA GPU usage, then confirm XPU memory is in use (rules out CPU fallback):

# 4a. Reject NVIDIA GPU usage — scan only the compute-app PID list and match
# the whole line, so a short PID can't collide with memory/temp/other numbers
# elsewhere in nvidia-smi's output.
if command -v nvidia-smi >/dev/null 2>&1 && \
   nvidia-smi --query-compute-apps=pid --format=csv,noheader 2>/dev/null \
     | tr -d ' ' | grep -qxF "$SGLANG_PID"; then
    echo "❌ Server is running on NVIDIA GPU, not Intel XPU."; exit 1
fi

# 4b. Verify Intel XPU memory usage — -m 18 = "GPU Memory Used (MiB)", CSV output
GPU_COUNT=$(xpu-smi discovery 2>/dev/null | grep -cE "^\| +[0-9]|Device [0-9]+:")
[ "${GPU_COUNT:-0}" -gt 0 ] || GPU_COUNT=1
XPU_IN_USE=0
for id in $(seq 0 $((GPU_COUNT - 1))); do
    MEM_USED=$(xpu-smi dump -d "$id" -m 18 -i 1 -n 1 2>/dev/null | awk -F',' 'NR==2 {gsub(/ /,"",$NF); print int($NF)}')
    echo "  Device $id: ${MEM_USED:-0} MiB"
    [ "${MEM_USED:-0}" -gt 500 ] && XPU_IN_USE=1
done
if [ "$XPU_IN_USE" -eq 0 ]; then
    echo "❌ XPU memory near zero — server may have fallen back to CPU. Restart with --device xpu."
    exit 1
fi
echo "✓ SGLang is confirmed running on Intel XPU"

Everything needed for the benchmark is now confirmed:

# 5. Summary — CONTAINER_NAME, SGLANG_PORT, MODEL were set above
echo "=== Ready to benchmark === Port: $SGLANG_PORT  Model: $MODEL  Container: $CONTAINER_NAME"

If no SGLang server is found, if it's running on NVIDIA instead of Intel XPU, or if it's fallen back to CPU, the checks exit with guidance.

Metrics

  • TTFT — wall time to first generated token.
  • TPOT — mean per-token time after the first.
  • ITL — per-token inter-arrival; percentiles meaningful for latency SLAs.
  • E2EL — wall time of one request.
  • Throughput — output_tokens / wall_seconds.

Prerequisites

A running SGLang-XPU server (per sglang-xpu-run). Step 0 above will verify the server is running, identify its port, and confirm it's using Intel XPU.

Note on ALL_PROXY: if the host has ALL_PROXY=socks://... set, the bench client's HTTP requests may be routed through the SOCKS proxy. Unset it before benching a local server:

unset ALL_PROXY all_proxy

Online bench

Important: The benchmark client must run inside the same container where the SGLang server is running. The sglang package is only installed in the container environment, not on the host.

Always use docker exec (not nsenter) to run benchmarks. The nsenter appro

阅读完整来源 ↗