That was kind of a nuisance but I got it done. I settled on using a push descriptor for the cubemap texture but I'll have to extend my bindless texture system sooner than later.