I generated a controlnet + region mapped image. The problem is that if the strength and end_percent of the "Apply ControlNet" node are above 0.2, then one of my character's hair gets based on the reference image which is wrong.

Meanwhile if I keep the strength and end_percent at 0.2, while the result is fine, it deviates a bit from the reference pose, which kind of defeats the purpose of masking a specific image.

In the rentry guide I see this line:
>We won't use inpaint ControlNets and dedicated inpainting models in this guide, but it's good to know they exist.

Is this the kind of thing I need to look into? Otherwise, what is the best way to replace a character's hairstyle with their original hairstyle in a multi-character controlnet image?