My husband made a stupid comment in a bad situation and my Brother in law can't/won't forgive him

So this is very very long and there is alot af context, but I really Hope someone will read this and advise me. For added context, check my post history and read about the last time my Bil(34) hated me/us.
In early september my whole family packed up and went on vacation to spain. We are a small family, we were 14 people. Me (f34), my husband (36) our two boys, my mom and dad, aunt and uncle, grandmother, brother and SIL and my sister(f27) her husband (BIL) and their daugter (~5 years old). Dont worry you dont have to keep track of all these people!
The vacation went smoothly from sunday to thursday evening, where my husband made the stupid comment that my BIL wont forgive.
So we made the idiotic decision to go to a mall in the evening after a full day of activities. We agree on a time and a place to meet, to get dinner. Everybody scatters off in different directions. Me and my family walks around with my brother and SIL, but eventually we get separated from them too. Its caos. My son loses his backpack and so him and I wander around the mall to find it, wich we ended up doing. We are tired, sweaty and hungry.
We end up all gathering in front of three restaurants, but nobody can agree on wich one to eat at. My children are hungry and restless, so I end up just walking in to the nearest restaurant with them and husband, and the rest of the family follows.
We sit down to eat, but I get the vibe that not everyone are happy about my choice. We eat, time to pay, my mother freaks out because she is trying to find out how much everyone needs to pay. My uncle fixes this by paying for us all and we all just pay him.
We get out of the restaurant and I thought that we were on our way back to the hotel, so I, husband and boys make our way down the escalator and my BIL is right behind me. As soon as we get of the escalator I turn around to look and see if everybody is behind us. They are not. I see my sister at the top, making some movements with her arms and BIL immidiatly gets back on the escalator and goes back up. I follow, but when I get to the top, all I se is my BIL dissapering in to the crowd. Everybody is gone.
So I call some of them, but no one answers. I eventually get ahold of my mom. Turns out that her and my grandmother went to the bathroom and also got left behind. We decide to exit the mall, as my aunt sent a message to the family groupchat saying that she was outside.
So we end up outside the mall, me, husband, two boys, mom, grandmom, aunt and uncle. So again chaos: are we staying? Are we going home? Where are x,y,z? Should I go get us some cabs?
At one point my brother and sister in law shows up. And a few minuts later, my niece is here. (My sister and BIL were not there) But then five minuts later, my niece is nowhere to be seen. We freak out! On top of all the other stress, my niece is now missing. This is where my memory gets kind foggy. We are running around looking, but my husband is staying put with our boys. Mom is irate. At some point in all this, my husband says this: "Not my child, not my problem". I did not hear this, it must have been said about the time I ran to check if she had wandered in to the parking garage or maby it just didnt register through all the panic.
Anyway... My mom ends up sprinting through the mall, finding niece safe and sound with my sister and BIL, aparently they took her from outside, without making sure that we knew or saw them take her. (This is about the same time that it hit me, that I should call my sister and see if they had her ?) Apparently, the things my mom thought to tell them, was: 1. omg we thought she was gone, why didnt you tell us that you took her! 2. (Husbandsname) said not my child not my problem.
Now, we can all agree that it was a ****ty thing to say. When I found out that he said that I was furious with him. But I know my husband, and he did not mean it. He is the kindest man I know. Kinder than all of the poeple in my family combined. He is not formally diagnosed with adhd, but he is on a waitinglist to get to talk to someone to get the diagnosis and help. No excuses, at all, it was the worst possible thing to say in that situation. Maby some of you recognizes the "says something before thinking it through" or no filter thing.
So husband and I decide to abandon ship. Niece is found. The kids are tired, we are tired. I hail a cab and want to go home, but wait! Half the people still doesnt want to go home. They tell us to take grandmom, but we cant find a cab with enough room for all of us, that want to go home. So aunt and uncle decides to walk, because they dont want to wait any longer. People run off again and we end up having to leave my dad and oldest son outside the mall to get grandmom home to the hotel.
The next days are terrible. BIL is clealy angry and avoiding us. At that point I still didnt know why. We decide to do things without the rest of the family and have a great time just us four. At the airport back home, BIL rushes off and the vacation ended on a bit of a sour note.
So fast forward to last monday. We havent seen BIL at two different family gatherings since then, so I message him. I dont know how to translate the danish word, but its something to the effect of are you mad at me?. He answers that he is not mad, but more dissapointed. And that is when I find out what my husband said. I was on my BILs side and wanted my husband to apologize, until my BIL wrote this to me.
"In the future I wont let you have our children over (sister is pregnant) because im scared that if you arent there husband will just let them run in to the street and get run over".
record scratch
What the hell kind of reasoning is that? They have known eachother for nearly a decade. Six words said in a high stress situation, just erased all that he knows about my husband. He is convinced that husband is negligant and malicious. I hit the brake and told him that he was out of line. He told me that that is what he has decided and that he wont change his mind. That my husband isnt welcome at their home anymore.
Nieces birthday is comming up and my sister just invited me and the boys, but not my husband.
Am I an idiot for not wanting to go?
I want to be the bigger person because, after all, it was my husband that said that idiotic thing that started it all. I understand. But on the other hand, I cant really look my BIL in the eyes after what he said about my husband.
Help?
Edit: for clarification, my sister and BIL dropped my niece of with us, then went somewhere
Then they came by and collected her, WHITHOUT making sure that we saw them take her. We were all trying to figure out what the hell our plan were. So at no point were my niece actually missing. She was at no point I danger or alone. This whole situation lasted maby five minutes.
Tl:dr On a familyvacation in spain, we got in a very stressfull situation. My husband made a stupid comment in regards to my niece, that my brother in law can't and won't forgive. He has decided that my husband, that he has known for nearly a decade, is dangerous for his daughter to be around. My nieces birthday is comming up, and they invited me and our son, but not my husband. Now I dont know what to do.


Source.