In retrospect it does make sense for Denji to not immediately assume that "Fami" was also behind Nayuta's murder. Denji did hear Barem calling Asa dumb and threatening to kill her, so he knew she was being used, which is why he didn't blame her for anything.
Maybe he assumed this "Fami" was the same? he knew Barem was a piece of shit via his association with Makima + his direct actions, but Denji didn't know anything about "Fami" beyond her name.
Not remembering Miri is just a gag though.