You do not owe this person a single word of explanation. Your reasons for not hiring him when he, evidently, needs money, are absolutely none of his business. Just tell him firmly that when you wish to hire him again he will be the first to know, that you have his contact, and you have don’t need reminders.
I would be concerned about this relationship. There may be unfortunate reasons he seems to need money. Desperate people can do bad things. It may be best to pause and not engage in his life for some time.
This is one reason why I rarely do appointments at my home. I do not want to bring people I do not know well that close to my life.