There is something that I have been learning and concluding as I get older and get the feeling that this is not some sort of unique epiphany. When you are younger and starting out, you will go to the end of the earth to make customers happy which does pay off. Nights, weekends, give up vacations, answer the phones at all hours of the night, take your brow beatings like a man, pretty much whatever it takes. In your 20s-30s you really have a lot of drive but eventually you hit a comfort level and that drive diminishes, it turns more into maintenance. Maintain your current income level, keep your bills paid, plan your later in life exit from the rat race and most importantly, maintain your sanity. I can only assume that as time goes on, you begin to have less tolerance for employees, customers and vendors. Not saying to turn into a pig headed my way or the highway guy but your life experiences combined with the fact that your hierarchy of needs are met (food/shelter, companionship, self esteem) you have one part left over and that is yourself. So while listening is very important, sometimes it becomes a battle between that and your sanity and your sanity wins out because of what I said earlier about your needs being met.
When I work with older business owners, many of them have slimmed down and choose to only do what they want to do and deal with who they want to deal with. Maybe 1-2 employees and many times just some part time helper. Earlier in their careers, many of them had some pretty big operations but it sucked the life out of them. They hit that wall and get done with listening to the noise from every direction and every single one of them will say that they don't have any regrets whatsoever with where they are in life. Most are way more content. With that said, you take it and listen for awhile but eventually you have to put yourself at the top. Have you ever seen the Seinfeld soup nazi episode? That is reality that they simply made into a funny bit.