hate how diffusion models really really really want to average out the brightness of the image

i put "completely dark, night time photo, barely visible, pure blackness" in positives and "white, bright, sunlight, studio light, brightness, light" in the negatives, and chroma still fills half of my images with bright white lighting 90% of the time

started feeding it a completely black latent with a bit of noise but that still isn't enough 90% of the time