How do I handle this?

I'm 44 and four years ago I met a man who is 54. He has never been married and always lived at home with his mom. She is in good health, so it's not an issue of taking care of her. In the very beginning, I really thought this would be a "normal" relationship, I figured he was just living with her to help her out, but it became evident to me rather quickly that he has no intention of ever moving out or having a life of his own, he is perfectly content with this arrangement. He treats me very well. We spend a lot of time together and I like him very much. I know this relationship has no future other than staying exactly the way it is right now. Recently, I started talking to an old friend of mine that I first met 20 some years ago. He went through a divorce a few years back, and I would like to spend some time with him and to explore where this may or may not go. Should I end whatever it is I have with the first guy before starting to spend more time with my friend? We have no commitment, but are in the habit of spending time together. I don't want to hurt his feelings but for some reason, I feel obligated to him.