Move `usm_ndarray` into `dpctl_ext.tensor` by vlad-perevezentsev · Pull Request #2807 · IntelPython/dpnp

vlad-perevezentsev · 2026-03-06T01:19:00Z

This PR proposes to migrate the tensor interface (usm_ndarray, dlpack, flags) into dpctl_ext/tensor making dpnp independent of dpctl's tensor module.

Updates:

Introduce dpctl_ext_capi.h

Implement a clean CMake interface library DpctlExtCAPI to properly propagate generated headers to consumers

Update remaining imports from dpctl.tensor to dpctl_ext.tensor

Link all backend extensions against DpctlExtCAPI to ensure consistent access to the C-API

Have you provided a meaningful PR description?
Have you added a test, reproducer or referred to an issue with a reproducer?
Have you tested your changes locally for CPU and GPU devices?
Have you made sure that new changes do not introduce compiler warnings?
Have you checked performance impact of proposed changes?
Have you added documentation for your changes, if necessary?
Have you added your changes to the changelog?

…ency

…y_impl_part_3

…gration

dpnp/__init__.py

ndgrigorian · 2026-03-19T05:26:13Z

@antonwolfy @vlad-perevezentsev
A thought but should we consider flattening the directory structure? Since currently it's
dpnp -> dpctl_ext -> tensor (actual files...)

Maybe could we flatten to something like dpnp -> tensor (or dpctl_ext)

antonwolfy · 2026-03-19T16:21:40Z

@antonwolfy @vlad-perevezentsev A thought but should we consider flattening the directory structure? Since currently it's dpnp -> dpctl_ext -> tensor (actual files...)

Maybe could we flatten to something like dpnp -> tensor (or dpctl_ext)

dpnp.tensor sounds reasonable as for me.

antonwolfy · 2026-03-19T16:24:30Z

Based on warning report seems we still using import dpnp.tensor somewhere in code or tests.
Is it intentional?

dpctl_ext/tensor/_scalar_utils.py

dpctl_ext/tensor/_set_functions.py

dpctl_ext/tensor/_ctors.py

antonwolfy · 2026-03-19T16:59:02Z

dpnp/backend/extensions/blas/CMakeLists.txt

 pybind11_add_module(${python_module_name} MODULE ${_module_src})
 add_sycl_to_target(TARGET ${python_module_name} SOURCES ${_module_src})

+target_link_libraries(${python_module_name} PRIVATE DpctlExtCAPI)


Why do we need that and how it worked previously without linking with dpctl lib?

Previously dpnp4pybind11.hpp included dpctl_capi.h from the external dpctl package which was found via ${Dpctl_INCLUDE_DIRS} set by find_package(Dpctl).

Now dpnp4pybind11.hpp includes dpctl_ext_capi.h which is an internal header,and without linking to DpctlExtCAPI I got an error

dpnp4pybind11.hpp:41:10: fatal error: 'dpctl_ext_capi.h' file not found 41 | #include "dpctl_ext_capi.h"

But the error is about include, not about linkage.

This logic has been removed in #2830 so I do not see the point in fixing it here

In any case DpctlExtCAPI is an interface library that collects include paths and build dependencies , no actual linking takes place, just only handles build order and header locations

The comment was about target_link_libraries usage, see docs:

Specify libraries or flags to use when linking a given target and/or its dependents.

Btw, I'm fine to resolve that in the follow-up PR.

antonwolfy · 2026-03-19T17:03:08Z

CMakeLists.txt

 endif()

-add_subdirectory(dpnp)
+# DpctlExtCAPI: Interface library for dpctl_ext C-API


Is it a temporary approach, or do we need to define interface lib even when it will be exposed by dpctl when the migration is completed?

It's a design decision either way: dpctl exposed PyUSMArrayObject for numba-dpex's sake. So this way, usm_ndarray would be exposed to C-API and could be interacted with from C code conveniently.

It could be removed, in which case, the api methods like i.e., cdef api char* UsmNDArray_GetData could also be removed (barring causing any problems in the dpctl_ext).

I don't think we need to migrate the code, which will not be used by someone, I'd propose to drop that.
In any case we will able to port that in the future if someone will be interesting in.

The PR removing С-API logic has been pushed in #2830
Please check if I have implemented your idea correctly

antonwolfy · 2026-03-30T11:38:41Z

dpctl_ext/tensor/include/dlpack/README.md

@@ -0,0 +1,7 @@
+# DLPack header
+
+The header `dlpack.h` downloaded from `https://github.com/dmlc/dlpack.git` remote at tag v1.0rc commit [`62100c1`](https://github.com/dmlc/dlpack/commit/62100c123144ae7a80061f4220be2dbd3cbaefc7).


I believe we are using the newer tag than v1.0rc

I have updated README.md and dlapck.h to v1.3
thank you!

are you sure we support 1.3 version of DLPack?

dpctl_ext/tensor/_dlpack.pyx

dpctl_ext/tensor/_usmarray.pyx

antonwolfy · 2026-03-30T11:43:41Z

dpnp/dpnp_algo/dpnp_arraycreation.py

 import math
 import operator

-import dpctl.tensor as dpt


We still have from dpctl.tensor interface in some dpnp comments

All TODO comments have been removed in #2829
The remaining comments and docstrings containing dpctl.tensor will be updated in the following PR

antonwolfy · 2026-03-30T11:52:25Z

dpnp/backend/extensions/blas/CMakeLists.txt

 pybind11_add_module(${python_module_name} MODULE ${_module_src})
 add_sycl_to_target(TARGET ${python_module_name} SOURCES ${_module_src})

+target_link_libraries(${python_module_name} PRIVATE DpctlExtCAPI)


The comment was about target_link_libraries usage, see docs:

Specify libraries or flags to use when linking a given target and/or its dependents.

Btw, I'm fine to resolve that in the follow-up PR.

antonwolfy · 2026-03-30T11:57:47Z

dpnp/dpnp_iface_statistics.py

-# pylint: disable=no-name-in-module
 import dpnp.backend.extensions.statistics._statistics_impl as statistics_ext
 from dpctl_ext.tensor._numpy_helper import normalize_axis_index
 from dpnp.dpnp_utils.dpnp_utils_common import (


Suggested change

from dpnp.dpnp_utils.dpnp_utils_common import (

from .dpnp_utils.dpnp_utils_common import (

Updated
Thank you

antonwolfy · 2026-03-30T12:07:38Z

dpnp/__init__.py

-with warnings.catch_warnings():
-    warnings.simplefilter("ignore", DeprecationWarning)
-    from dpctl.tensor import __array_api_version__, DLDeviceType
+from dpctl_ext.tensor import __array_api_version__, DLDeviceType


TODO comment is missing

I have added it
Thank you !

vlad-perevezentsev added 30 commits February 6, 2026 03:05

Use tril/triu/_tril from dpctl_ext.tensor in dpnp

f1d6e56

Disable pylint no-name-in-module for dpctl_ext

6680790

Add TODO comments

263b717

Use default_device_complex_type from dpctl_ext on test_array_api_info.py

4130c1b

Remove unused build_dpctl_ext function

17ca9ab

Apply remarks for CMake files

79cb2a4

Apply remarks for c++ files

4bf080e

Remove linear-sequence implementations

cfa6cd6

Merge move_tensor_impl_ext into move_tensor_impl_ext_part_2

e0e50ac

Use _tensor_impl from dpctl_ext in dpnp

087a2ec

Add missing include

f4492fb

Use nested namespace syntax

b367c9f

Add missing include complex

3113716

Add missing memory and queue checks

978afee

Move ti._copy_numpy_ndarray_into_usm_ndarray()

fec84ec

Move asnumpy(),from_numpy(), to_numpy() to dpctl_ext/tensor

497e810

Update dpnp.asnumpy to use dpctl_ext functions

3be4e14

Move copy(), astype() to dpctl_ext/tensor

1d88365

reuse astype(), copy() from dpctl_ext

fd18db0

Move _copy_usm_ndarray_for_reshape

0976171

Move reshape() to dpctl_ext/tensor

318692e

Reuse reshape from dpctl_ext in dpnp

3c0c113

Move _copy_usm_ndarray_for_roll

30f2c53

Move roll() to dpctl_ext/tensor

85c29da

Update dpnp.roll to use dpctl_ext

6e8d857

Update .gitignore to ignore .so files in dpctl_ext

19e93b9

Remove unused includes in tensor_ctors.cpp

b111e49

Use Python::Module for dpctl_ext static lib to avoid libpython depend…

c082224

…ency

Merge move_tensor_impl_ext into move_tensor_impl_ext_part_2

9e7deb3

Merge include-dpctl-tensor into move_tensor_impl_ext_part_2

1a736f7

vlad-perevezentsev added 8 commits March 18, 2026 10:14

Apply remarks

b0a4a6e

Merge move_elementwise_binary_impl_part_2 into move_elementwise_binar…

21f1aec

…y_impl_part_3

Merge include-dpctl-tensor into move_elementwise_binary_impl_part_3

255a20b

Clean up includes

d00ebe4

Merge move_elementwise_binary_impl_part_3 into finalize_functional_mi…

1a4f450

…gration

Merge finalize_functional_migration into move_usm_ndarray

ed51252

Increase timeout_minutes for build dpnp with coverage step

b4395ea

Increase timeout-minutes for Build and Deploy Docs

7327502

antonwolfy reviewed Mar 18, 2026

View reviewed changes

dpnp/__init__.py Outdated Show resolved Hide resolved

vlad-perevezentsev added 5 commits March 19, 2026 04:25

Merge include-dpctl-tensor into finalize_functional_migration

42a6ada

Update test_from_dlpack_with_dpt to use empty from dpctl_ext

bc52234

Fully switch to using dpctl_ext functions for dpnp_elementwise_common.py

8adaa68

Update _testing.py in dpctl_ext/tensor

0277672

Merge finalize_functional_migration into move_usm_ndarray

11562a2

Base automatically changed from finalize_functional_migration to include-dpctl-tensor March 19, 2026 14:53

Merge include-dpctl-tensor into move_usm_ndarray

fd763f3

antonwolfy reviewed Mar 19, 2026

View reviewed changes

vlad-perevezentsev added 4 commits March 19, 2026 12:38

Apply remarks

b30e6cc

Remove the warning suppress

a9aadd9

Increase timeout-minutes for generate-coverage job

6ea695a

Undo accidentally removed import DLDeviceType

76b1a0f

antonwolfy reviewed Mar 30, 2026

View reviewed changes

vlad-perevezentsev added 4 commits March 30, 2026 06:22

Update dlpack.h to v1.3 and README.md

fe3b2db

Update examples in dpctl_ext.tensor cython files

9087bbf

Add TODO comment in __init__.py

aa934f7

Apply remark

33fc8fc

		@@ -0,0 +1,7 @@
		# DLPack header

		The header `dlpack.h` downloaded from `https://github.com/dmlc/dlpack.git` remote at tag v1.0rc commit [`62100c1`](https://github.com/dmlc/dlpack/commit/62100c123144ae7a80061f4220be2dbd3cbaefc7).

	from dpnp.dpnp_utils.dpnp_utils_common import (
	from .dpnp_utils.dpnp_utils_common import (

Conversation

vlad-perevezentsev commented Mar 6, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

ndgrigorian commented Mar 19, 2026

Uh oh!

antonwolfy commented Mar 19, 2026

Uh oh!

antonwolfy commented Mar 19, 2026

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

vlad-perevezentsev commented Mar 6, 2026 •

edited

Loading