Imagine if Axeman pulls a fast one and it turns out that Loid isn't in his room because he stepped out for some emergency WISE call or something.

Yor probably wouldn't get angry at him for leaving Anya alone considering she walks home from school all the time, but it could go there since she's not in the right mindset right now.