Here's the example given from a kubectl plugin, kubectl-view-allocations:
> kubectl-view-allocations -r gpu
Resource Requested Limit Allocatable Free
nvidia.com/gpu (71%) 10.0 (71%) 10.0 14.0 4.0
├─ node-gpu1 (0%) __ (0%) __ 2.0 2.0
├─ node-gpu2 (0%) __ (0%) __ 2.0 2.0
├─ node-gpu3 (100%) 2.0 (100%) 2.0 2.0 __
│ └─ fah-gpu-cpu-d29sc 2.0 2.0 __ __
├─ node-gpu4 (100%) 2.0 (100%) 2.0 2.0 __
│ └─ fah-gpu-cpu-hkg59 2.0 2.0 __ __
├─ node-gpu5 (100%) 2.0 (100%) 2.0 2.0 __
│ └─ fah-gpu-cpu-nw9fc 2.0 2.0 __ __
├─ node-gpu6 (100%) 2.0 (100%) 2.0 2.0 __
│ └─ fah-gpu-cpu-gtwsf 2.0 2.0 __ __
└─ node-gpu7 (100%) 2.0 (100%) 2.0 2.0 __
└─ fah-gpu-cpu-x7zfb 2.0 2.0 __ __
Seems to offer what you are looking for.
Furthermore apparently, sharing GPUs is possible (with Nvidia-provided software) - might be useful to you or whoever finds this in the future: