Ralsei's not a headband or any other basic object.
In chapter 1 Ralsei says that he has been waiting his whole life completely alone for Kris and Susie to arrive. In chapter 2 it is established that what might be a relatively short amount of time for Kris and Susie is a very long time for Ralsei, because Ralsei has no frame of reference for what is a short or long amount of time. From this I deduce that Ralsei has not existed for a very long time. Darkners retain memories of events they witnessed as objects, and it wouldn't make sense for Ralsei to say he's been alone his whole life if he was a normal object that existed within the storage closet. If that were the case, surely he would've seen other Lightners such as Alphys or Toriel.
I theorize that
A) Whatever object Ralsei is based on was created not very long ago, possibly during the creation of the Dark Kingdom's fountain or shortly thereafter.
OR
B) Ralsei was conceived from pure darkness, not tied to any object, allowing him to travel between Dark Worlds freely.
Ralsei does seem to imply that he is *something* in the Light World in his explanation for the Dark Worlds in chapter 3, so I'm more inclined towards option A.