Is there a problem in taking a loan/borrowing money from my 20 year old special needs son?

There is no problem taking a loan or even using the money of this special needs son by his parents because his status is as a son who is maintained by his father that all his money belongs to his father. The exception to the rule is in a case where the money was given to the son on condition that it is for his use only and his parents do not have permission to use it for their personal needs. (See Shulchan Aruch Choshen Mishpat 202, 2. and Sema 8).