I think you are missing the point. Nazi Germany was by no means a vegetarian country. I think a better question to ask would be - could the Holocaust have happened in the mostly-vegetarian India? Millions of people were murdered under the pretense that these people are pigs or rats. But what if your culture (e.g. Hinduism or whatever) also condemns the killing of pigs and rats? The entire argument that justifies murdering these people breaks down.

India long had a caste system. Do you argue that the long-standing violence and discrimination against "untouchables" could not have led to a genocide the way long-standing anti-Semitism in Europe did?

First, India isn't mostly vegetarian (about a 1/3 of the population is vegetarian).

Second, India has had and continues to have horrible acts of violence committed against various social and religious segments of the population. See: Partition of India, Ayodhya, Caste System violence.

Third, I haven't seen any study, nor any evidence that suggests that violence is unique to non-vegetarians.