Boyfriend's (29m) parents (60s) let themselves into our house

I moved into my boyfriends house about 6 months ago and have been struggling to feel like it is "home". One thing contributing to this is that his parents who live down the street will come around about once a week while we are not there. Usually to do something useful like mow the lawn, take out the trash, feed the cat. One time the let themselves in while I was home because they thought I was out (my car wasn't in its usual spot) and I had just gotten out of the shower.
My boyfriend doesn't mind this at all.... This has been the status quo since he bought the house 7 years ago. They are very close and after all, they are just being helpful!
It bothers me because I would absolutely not accept this dynamic with my own parents. I feel judged if the house is a mess (his mom has commented before about the mess) and just on edge. I don't feel like I have control over my environment and I'd honestly rather mow the lawn myself if it meant reclaiming that zone of privacy. It's hard to articulate to him why it upsets me so much and I don't even know if I'm being unreasonable. I also see the other side.... And I don't want to cause tension between my boyfriend and his family.
We have spoken about it and came to a compromise that they would not go upstairs and that bf would try to let me know when they were likely to come round so I could tidy up a bit. However, bf never spoke to them (he said they wouldn't go upstairs anyway so there was no need) and this morning his mom came over while we were out and he didn't warn me. He said that he doesn't always know if/when they will come. So I've basically been told just to live with it.
Should I just get over it? Is it normal for families to act like this? How do I approach this with bf? How do I draw some boundaries? What boundaries are appropriate?
*TL;DR - boyfriends parents come into our home, is this weird? How to I assert some boundaries? *
Edit: Thanks for all of your advice! Really feel supported and validated for feeling the way I do. So I decided to bite the bullet and sit down with my boyfriend while things were fresh in my mind. I explained how this was stopping it feeling like my home and that he needed to enforce some boundaries with his parents. I said that I need them to text me to check before they come over regardless of whether I am here or not. I also told him that he needs to shut down comments his mom makes about cleanliness as I find it very rude. He agreed in principle and told me that he would talk to them. He has gone over to theirs now to help his mom with some tech problem so I'll see if he actually follows through. I am prepared to keep asserting this boundary as long as it takes for them to respect it. It's definitely just a first step...but a positive one. Thanks everyone!
Edit 2: He did it! Apparently he used the example of what if I'm walking around in my underwear and they understood. Let's see if they actually ask in reality and what happens if I say no! Baby steps!


Source.