If someone looks good in a piece of clothing, it is fair to assume they look less good with it on.
“Really, dude?! You’re gonna pick on this stupid shit?”, you ask.
You bet your sweet ass!
The compliment goes like this…
“That looks good on you!”
This implies that the person has qualities that improve the cloth on their body vs. the article of clothing has a quality that improves how the person looks.
It is the difference in “That makes you better.” and “You make that better.”.