How do you know when it's really time to leave?

I am realizing my spouse is emotionally abusive towards me. He is not that way with our children. I am about at my wits end. We are in counseling, but I can't say I feel like it's doing any good. I think about leaving, but when it comes to actually truly planning it out and turning my kids' life upside down, it's so terrifying. I don't want to destroy my kids for my mistakes. If you have been in a similar situation, how did you know it had crossed the line and it was time to move on - no question?