Got the golden dice. Closest fight was probably the shadow-cursed shambling mound in act 2 because I messed up the preparation. Second was phase spider queen for the same reason, ended up fighting it with two awake sets of hatchlings to start with.