Shatnez

Question

You answered earlier that in Yoreh Deah #300:4 it states that you are not to wear a wool sweater over a linen shirt. However, it also says in the Rema (and noseh kailim) that this is only true when you cannot take off the bottom garment without taking off the top garment. Especially how the Taz describes the undressing process, why can we not say that all undergarments can be removed without taking off the garment on top? Perhaps a very tight sweater I will agree, but in most cases why should it be true? (Sorry for the long essay!)