# Failure on USM shared malloc with large sizes

**URL:** <https://community.khronos.org/t/failure-on-usm-shared-malloc-with-large-sizes/107449>\
**Category:** SYCL\
**Created:** [June 21, 2021, 12:02pm UTC](https://community.khronos.org/t/failure-on-usm-shared-malloc-with-large-sizes/107449 "2021-06-21T12:02:23Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![sajjadrsm](https://community.khronos.org/letter_avatar_proxy/v4/letter/s/7993a0/32.png) [@sajjadrsm](https://community.khronos.org/u/sajjadrsm)\
**Post date:** [June 21, 2021, 12:02pm UTC](https://community.khronos.org/t/failure-on-usm-shared-malloc-with-large-sizes/107449/1 "2021-06-21T12:02:23Z")

</div>

Hi everyone,

I have a question about USM shared malloc arguments.

In my program, I use a USM shared pointer with size `2*N*N`:

```auto
std::vector<float> A(2 * N * N);
auto A_acc = (Array<float, 2, N, N> *)malloc_shared( sizeof(Array<float, 2, N, N>), 
                         deviceQueue.get_device(), deviceQueue.get_context());
new (A_acc) Array<float, 2, N, N>(A.data());

```

Inside the kernel I reference the shared memory via `A_acc[0][0][i][j]` and `A_acc[0][1][i][j]`.

My program works when N is set to values smaller than 24 (like 8, 16 or 24), but it fails at size 32.

May I ask if there is any limitations on using USM pointers inside kernels?

Thanks & Regards  
Sajjad

---

<div class="post-metadata">

**Author:** ![keryell1](https://community.khronos.org/letter_avatar_proxy/v4/letter/k/48db29/32.png) [@keryell1](https://community.khronos.org/u/keryell1)\
**Post date:** [June 21, 2021, 3:58pm UTC](https://community.khronos.org/t/failure-on-usm-shared-malloc-with-large-sizes/107449/2 "2021-06-21T15:58:12Z")

</div>

Can you post the full example or at least a minimum test case showing the problem, which compiler & system you are using, which accelerator are you targeting? Otherwise it is difficult to help. For example I have no idea about what is “Array” and so on.

---

<div class="post-metadata">

**Author:** ![sajjadrsm](https://community.khronos.org/letter_avatar_proxy/v4/letter/s/7993a0/32.png) [@sajjadrsm](https://community.khronos.org/u/sajjadrsm)\
**Post date:** [June 21, 2021, 9:07pm UTC](https://community.khronos.org/t/failure-on-usm-shared-malloc-with-large-sizes/107449/3 "2021-06-21T21:07:30Z")

</div>

Thanks for your response. This is how I am defining Array:

```auto
template<typename T, int N, int... Rest>
struct Array : std::array<Array<T, Rest...> , N>{
  using std::array<Array<T, Rest...> , N>::operator[];
};

template<typename T, int N>
struct Array<T, N> : std::array<T, N>{
  using std::array<T, N>::operator[];
};

```

I am using the CUDA backend on DPC++ where the `malloc_shared` calls cuda\_piextUSMSharedAlloc in the Cuda plugin API and then it makes a call to `cuMemAllocManaged`.

---

<div class="post-metadata">

**Author:** ![keryell1](https://community.khronos.org/letter_avatar_proxy/v4/letter/k/48db29/32.png) [@keryell1](https://community.khronos.org/u/keryell1)\
**Post date:** [June 22, 2021, 1:44am UTC](https://community.khronos.org/t/failure-on-usm-shared-malloc-with-large-sizes/107449/4 "2021-06-22T01:44:07Z")

</div>

I see, your `Array` is multidimensional array defined by using recursively a `std::array`.  
I cannot see any obvious reason for the failure.  
Perhaps some alignment constraints?  
For N = 32, this requires 8192 bytes, which might hit a bug when using more than one 4K page?  
Anyway, it looks related to a specific implementation with a specific back-end. so I suggest you open an issue on [GitHub - intel/llvm: Intel staging area for llvm.org contribution. Home for Intel LLVM-based projects.](https://github.com/intel/llvm) with a complete example which can compile and exhibits the bug at run time so they can directly try the code.

---

<div class="post-metadata">

**Author:** ![sajjadrsm](https://community.khronos.org/letter_avatar_proxy/v4/letter/s/7993a0/32.png) [@sajjadrsm](https://community.khronos.org/u/sajjadrsm)\
**Post date:** [June 22, 2021, 8:57pm UTC](https://community.khronos.org/t/failure-on-usm-shared-malloc-with-large-sizes/107449/5 "2021-06-22T20:57:46Z")

</div>

Thank you very much for the pointers.

---

<div class="post-metadata">

**Author:** ![system](https://community.khronos.org/uploads/default/original/2X/b/ba89cb49313a0e869198c8be354afde8bcae027e.png) [@system](https://community.khronos.org/u/system)\
**Post date:** [December 19, 2021, 8:58pm UTC](https://community.khronos.org/t/failure-on-usm-shared-malloc-with-large-sizes/107449/6 "2021-12-19T20:58:39Z")

</div>

This topic was automatically closed 180 days after the last reply. New replies are no longer allowed.