I had this exact same question but one day I had a conversation that blew my mind. I sat down on a bench outside of a Target waiting for my Lyft driver, next to me was an old woman eating a sandwich. Not sure how or why we began to talk but we had a pretty lively Convo about Legos. She talked about how much she loved collecting them and sharing her hobbies with her grandkids.
She may not have realized it, but she gave me a whole new perspective in life. It IS possible to be a responsible adult AND still be able to enjoy what it is typically seen as "for kids". The trick I think is to come up with a healthy balance between the "kid stuff" and your responsibilities.