If you attack nations who surrendered from your alliance while the war is on-going, you wind up draining resources away from fighting your actual enemy. I suppose you could justify it as an example to keep people from deserting, but if the only way to keep the members of your alliance from bailing during war time is to threaten them, your alliance is already a corpse.
I understand people getting upset by what feels like abandonment when you need people, but, honestly, if someone doesn't want to fight for the alliance, then I'd rather not have them there during a war. It's not like they took anything from me by being in my alliance during peace time. If they engaged in economic transactions with the alliance, it's pretty likely that we benefitted from that either from money going to newbs or tech going to our bigger nations. If they just sat there, then I'm not sure why I sure care one way or another whether they were ever on the AA or continue to remain there. If they contributed to the alliance, then we benefited from their stay. If they made trouble during peace time, then they were probably coasting towards getting booted anyway.
I really don't understand why people bother. I have much more important scores to settle than with random people I probably don't know the names of anyway. I have no interest in setting up a deterrence for people who want to jump ship because, uh, I don't want people who aren't willing to fight getting in the way when I'm trying to fight a war. I'm also not all that afraid of having my members suddenly jump ship, so setting up a precedent to deter people from doing so isn't very high on my priority list.
Frankly, unless you like being vindictive or are afraid of having your own members bail on you, I don't actually see what reason anyone has to give enough of a damn to do anything other than wave to people's backs as the run from war.