I get the feeling that's because gluten-free food is - for some people - medically necessary, while people with, say, allergies to some animal product e.g. eggs will usually just buy normal food and be careful not to buy anything containing egg. As a result, "vegan" food has a negative perception that "gluten free" escapes.
Gluten free does not escape a negative reputation. There are many people who follow a gluten free diet because it's somewhat of a fad, and there are many others who, as a result, don't take it seriously.
As someone who is gluten free because I can't tolerate gluten, I would add that, for those who don't have that issue, gluten free food is not always the healthiest choice. In particular, gluten free baking is tricky, and gluten free baked goods are generally much higher in sugar.
