Workplace Like a Family?

[Nikia]

1. If you don't enjoy this culture, I would look for something else

while you keep your current job.
With my past experience, I personally would never work for a small company again. Whatever the culture and circumstances are, they are very much dictated by the owner or owning family. While some owners are much more reasonable than others, the business is usually run by irrational preferences of them and everything is much more personal. While some employees may be treated very good by such employers, everything seems based on how much they personally like you.
In your situation, they like employees who go to all their events and dislike employees who don't. If this is how they feel, you probably can't change their mind.