chalav yisrael again

Question

Discussing the issue of chalav israel with someone, and he accepts the principle that assuming it's just a way of assuring we are getting only cow's milk, there's no reason to drink only milk from Jewish cows at this point. But he questions that perhaps at this point it's become a minhag among certain segments of our population to assure themselves of the fact that it's cow's milk by drinking only from Jewish cows, and that would obligate them to stick with it. Thoughts? Thanks!