My boyfriend [21M] doesn’t understand why I [19F] didn’t stay at his house for his birthday after he raised his hand at me. Should I feel guilty?

So last month I visited my boyfriend’s house, a four hour train journey from my house. It was the first time I’d been there. I was supposed to be there for five days, including his birthday, the Thursday.
On Tuesday night we had a huge argument about how uncomfortable I felt in the house. He kept leaving me alone to go and talk to his siblings, shouted at me about shutting the door too hard, and shouted about me leaving milk in the bowl after I had cereal.
After the initial row was over he sat for an hour calling me names and calling me ungrateful. I was crying and he suggested I stop before his brother or sister heard. He then got angry again and raised his hand at me. I flinched and was genuinely scared of him in that moment. He’s always been angry but I saw another side of him when I wasn’t in the comfort of my own hometown or home.
I left on the Wednesday, before his birthday. I just didn’t want to be around him. And whilst we did make up and have been doing okay, last night we had another blow up row where he blamed me leaving on our relationship going to ****. I don’t know whether I should feel guilty about it or whether I did the right thing.
TL/DR: boyfriend raised hand at me and is making me feel guilty for not staying at his house after.


Source.