Who cares? not me, I never watch that crap! Reality/ contest tv, performing, art is not a contest, for tv ratings.
BUT
With regard to dressing for the part, maybe you are not the demographic the producers are looking to attract and hold. And maybe the shirtless guy tests well with THAT demographic.
Just like when my parents ridiculed the Beatles long hair, and the first time performers came on without ties and suits. If it's about the music, then listen to the music and forget the clothes.
Personally I play best in shorts, t-shirt and bare feet. But I put up with wearing more at some gigs. I have quit bands and given up gigs because they insist I wear a "costume" and I want to enjoy myself and play my best not impress with my wardrobe.