My (mid 20s F) husband (late 20s M) suddenly said he wants us to see a marriage counselor. I'm hurt.

First post, kind of rambling because I've never shared our issues with anyone. Not incredibly specific in case he finds this.
Some background: my husband and I have been together almost 4 years. We've never been the best communicators. He had a rough childhood and any conflict makes him just shut down and let me "win," which I can't stand because it feels like we never come to a true resolution on disagreements. My issue in conflict is that I become incredibly defensive and my feelings are easily hurt. Also important to note is that for entire relationship until my recent difficult pregnancy, I handled pretty much all domestic duties like cooking and cleaning while working at least one full time job, and sometimes a part time job as well, while he's only ever worked one full time job.
So we recently welcomed our first child into the world. I've been staying home with the baby and trying my best to keep up with domestic duties, but it's really hard for me to do everything with this newborn still needing so much attention and care. The baby comes first, the vacuuming can wait. After our entire relationship of me begging him for help with cleaning and giving up asking after awhile, just cleaning up after him, he has decided there is nothing more important in the world than having a clean house and he has been lecturing and nitpicking every day. This is really difficult for me to handle after so many years of not getting any help from him at all. I'm also just tired from caring for the baby. I try to do what I can and whenever I get something done, I feel really great about it until he points out something else I've messed up/haven't done. He keeps saying he's not pointing things out as a personal attack on me, and that "we" need to work on things, but every time he does point something out, it's something I've personally left. Meanwhile, he still leaves messes/doesn't complete his very few agreed upon chores and I don't say anything about it because I don't want to upset him. I've tried to tell him that I'm happy to do some cleaning if he could just watch the baby for awhile when he gets home from work, but he says he's too tired. Some days he doesn't do anything involving our baby, not even holding the baby, and not only is it frustrating because I struggle to get things done, but it's absolutely heartbreaking to see him not want anything to do with the baby.
So the other day we got into an argument where, again, I was feeling good and happy for getting some things done until he pointed out something small that I didn't get to. I really just completely broke down, telling him that it's really hard caring for a newborn and doing everything else. So then he shut down, said he wants to see a marriage counselor, and we've barely talked since then. Counseling has never been brought up between us, ever. Since we're not talking and I'm with the baby 24/7, that also means he hasn't done anything at all with the baby, which hurts me the most. I usually would try to talk to him in the past, but I don't want to make things worse.
For me, with my parents' failed relationship, marriage counseling means all else has failed, you're about to give up, and divorce is coming. It scares me so much that he feels our marriage is heading in that direction. I'm worried we're going to try counseling and nothing will change and that will be the end of it.
Anyone in a similar situation that has gone to counseling? What was your experience? I'm so afraid we're going to spend time and money on something that won't change anything.
TLDR; my husband and I have a history of poor communication and he suddenly said he wants to see a marriage counselor. I'm afraid that means it's the beginning of the end.
EDIT: Wow, this really blew up a lot more than I expected! I really appreciate the encouragement and insight from all of the responses. I'm seeing that counseling could be really good for our communication and for a neutral third party to talk about realistic expectations for both of us. I started looking into local counselors and I will talk to my husband about choosing one so he doesn't feel like I'm picking one to side with me.


Source.