Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ENH: workunit typing and type sizing feedback #168

Open
tylerjereddy opened this issue Feb 2, 2023 · 1 comment
Open

ENH: workunit typing and type sizing feedback #168

tylerjereddy opened this issue Feb 2, 2023 · 1 comment
Labels

Comments

@tylerjereddy
Copy link
Contributor

While working on the implementation of pk.arange() the following scenario caused me a substantial headache. Example workunit:

@pk.workunit
def arange_impl_1d_uint32(tid: int,
                         start: int,
                         stop: int,
                         step: int,
                         out: pk.View1D[pk.uint32]):
    counter: int = 0
    for i in range(start, stop, step):
        if tid == counter:
            out[tid] = i
            break
        counter += 1

Example of the NumPy calculation exceeding the largest allowed signed 32-bit integer:

# this works just fine
np.arange(0, 2147483648, 100000, dtype=np.uint32)

In PyKokkos not only do we not get a helpful error message (below), but I don't even know how to fix this:

Traceback (most recent call last):
  File "/home/tyler/github_projects/pykokkos/test.py", line 10, in <module>
    main()
  File "/home/tyler/github_projects/pykokkos/test.py", line 6, in main
    pk.arange(0, 2147483648, 100000, dtype=pk.uint32) # error
  File "/home/tyler/github_projects/pykokkos/pykokkos/lib/create.py", line 62, in arange
    _ufunc_kernel_dispatcher(tid=size,
  File "/home/tyler/github_projects/pykokkos/pykokkos/lib/ufuncs.py", line 50, in _ufunc_kernel_dispatcher
    ret = sub_dispatcher(tid, desired_workunit, **kwargs)
  File "/home/tyler/github_projects/pykokkos/pykokkos/interface/parallel_dispatch.py", line 175, in parallel_for
    func(**args)
RuntimeError: Unable to cast Python instance of type <class 'int'> to C++ type 'int'

I believe the issue boils down to range() in the workunit and how that gets translated over to C++ -- it obviously can't be the same as Python range() since Python can use arbitrary precision integers. Nonetheless, it would be helpful if:

  1. the actual violating line in the workunit kernel were in the traceback
  2. we had clear documentation of how to properly handle these scenarios, where the range includes input outside the normal positive (or negative) limit of C++ signed int32. Maybe we could provide a custom typing for i. It is really hard for me to judge here because it isn't Python or C++ anymore at this point, so I'm not even sure what syntax to use.

@JBludau may have something in the works to help out a bit here perhaps

For now, I'll just push my WIP on feature branch treddy_arange until I'm unstuck here.

@tylerjereddy
Copy link
Contributor Author

I should also mention--it isn't necessarily even clear to me that we'd want to make this multi-threaded/parallel -- nonetheless, the correct way forward would still be instructive for the purposes of crafting more sophisticated workunits.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

1 participant