dpnp-memory
Device memory management for dpnp arrays on Intel CPUs and GPUs. Use when a dpnp script grows in memory until it fails, when a dataset does not fit in device memory, when an array turns out to be on a different device than expected, or when a loop allocates a new array on every iteration. Covers USM allocation, inspecting placement and queues with dpctl, reusing an output buffer, chunking a workload larger than the device, and the tools that report device memory use.
intel · Static checks on file
Installation
Read the source and dependencies before running a command. Copying does not install or execute anything.
npx skills add intel/skills --skill dpnp-memoryAbout this skill
name: dpnp-memory description: >- Device memory management for dpnp arrays on Intel CPUs and GPUs. Use when a dpnp script grows in memory until it fails, when a dataset does not fit in device memory, when an array turns out to be on a different device than expected, or when a loop allocates a new array on every iteration. Covers USM allocation, inspecting placement and queues with dpctl, reusing an output buffer, chunking a workload larger than the device, and the tools that report device memory use. license: Apache-2.0 compatibility: "Requires dpnp and dpctl. Device memory reporting needs xpu-smi (data center GPUs) or intel_gpu_top (client GPUs)." metadata: intel-skill-type: "tool-skill" version: "1.0"
dpnp memory and device placement
Purpose
Answers where a dpnp array lives, how much room the device has, and how to keep
a long-running script from filling it. dpnp arrays are allocated in SYCL unified
shared memory on a device, not on the CPU heap the way NumPy arrays are, so the
questions that matter are different: which device, whose queue, and when the
allocation is released.
Prefer this skill over guessing from symptoms — a script that slows down over hours, an out-of-memory error, or a GPU that turns out to have been a CPU all along are all answered by reading state the runtime already exposes.
When to Use This Skill
Use this skill when:
- Memory use climbs over the life of a script or a notebook session.
- A dataset is larger than the device and has to be processed in pieces.
- The user needs to confirm which device or queue an array is on.
- A tight loop allocates a new array per iteration.
- The user asks how to see device memory use from outside Python.
Do not use this skill for host-side NumPy memory questions, for file I/O
(that is dpnp-io), or to decide whether dpnp is worth using at all.
Quick Start
import dpnp
arr = dpnp.arange(1000)
print(arr.sycl_device) # e.g. level_zero:gpu:0
print(arr.sycl_device.name) # human-readable name
print(arr.sycl_device.global_mem_size / 1e9, "GB") # capacity, not free space
global_mem_size is the total the device reports. There is no dpnp API for
free memory — that comes from the tools in Gotchas.
Implementation Guide
-
Read the placement before changing anything. Every array carries
sycl_deviceandsycl_queue;dpctl.get_devices()lists what is visible. Filter strings arebackend:device_type:index, solevel_zero:gpu:0andopencl:cpu:0name specific devices. Level Zero is the lower-overhead backend for Intel GPUs.import dpctl for device in dpctl.get_devices(): print(device.filter_string, device.name) -
Target a device explicitly when the default is wrong.
dpnppicks a default device at import time using the SYCL default selector, which scores the visible devices — it is not "the first GPU". Passdevice=orsycl_queue=rather than relying on it:gpu = dpctl.SyclDevice("level_zero:gpu:0") arr = dpnp.arange(1000, device=gpu) queue = dpctl.SyclQueue(gpu) shared = dpnp.arange(1000, sycl_queue=queue) -
Reuse the output buffer in loops. Universal functions take
out=, which writes into an existing allocation instead of making one:a = dpnp.arange(10000, dtype=dpnp.float64) b = dpnp.arange(10000, dtype=dpnp.float64) result = dpnp.empty(10000, dtype=dpnp.float64) # allocate once for _ in range(1000): dpnp.add(a, b, out=result) # no new allocationUse
dpnp.empty()rather thandpnp.zeros()when the initial values are overwritten anyway, and pre-allocate the output ofdpnp.matmul(A, B, out=C)the same way. -
Keep conversions out of the loop body.
dpnp.asnumpy()copies device to host anddpnp.array()copies host to device. Calling a NumPy function on adpnparray, or mixing the two in one expression, does the same thing implicitly. Hoist the conversion above the loop. -
Chunk a workload that does not fit. Size each chunk so the input and the intermediates together stay under the device capacity — roughly half to two thirds of it is a workable starting point — then release the arrays before the next iteration:
import gc import numpy import dpnp chunk = 10_000_000 for start in range(0, 100_000_000, chunk): host = numpy.load(f"data_chunk_{start}.npy") device_array = dpnp.array(host) total = dpnp.sum(device_array ** 2) numpy.save(f"result_{start}.npy", dpnp.asnumpy(total)) del device_array, total, host gc.collect() -
Watch the device while it runs rather than reasoning about it afterwards:
xpu-smi dump -m 1on data center GPUs,intel_gpu_topon client GPUs,clinfofor OpenCL limits,ze_infofor Level Zero. Steadily climbing memory is the signature of a leak.
Performance
No measured numbers ship with this skill. Whether pre-allocation or chunking is worth it depends on array size, device, and driver, so measure the specific case:
- Pre-allocation matters most for small arrays in loops with many iterations, where allocation is a large share of the work. For large arrays the allocation cost is amortized over the compute.
- Chunking trades memory for repeated allocation and transfer. If disk I/O dominates, that trade is invisible; if compute dominates, it is not.
- Warm up before timing anything: the first call on a new shape includes compilation.
Gotchas & Limitations
deldoes not free device memory immediately. It drops a reference. The allocation goes back when the object is collected, and in a notebook an output cell can hold the last reference.gc.collect()encourages collection; it does not guarantee the allocator returns the memory at th