Relationships and Coping Community

So, I fell in love with an amazing man. Intelligent, educated, business savvy, mature,handsome and fun. He was the bank president and I a manger in a sister bank. We had an instant attraction for each other, it was weird, both of us were outgoing n spoke with every except each other. I didn't know much about him and didn't dare show an interest in him to anyone. I seen him once a week in passing at office meetings. I was farely knew to the company when we had a xmas party we both had a couple of drinks and started a conversation and were mostly speaking with our eyes and soon left for a after party where we eventual kissed. After that night we didnt speak for 6 months. He was on my mind all day everyday and I on his. I volunteered to get a message to him for a co-worker and then he asked me if I felt anything for him I said yes and that started our relationship. We both fell hard for each other. We were both in our mid 30's had children, i divorced he, was not with his baby momma. I had never dated a man who gave me everything I was looking for in a relationship. According to him I fulfilled every need he had ever wished for and more. We were perfect for each other.
We spent alot of time together, weekdays n weekends. My schedule with him was always day to day. I had 2 young girls that I didnt want to confuse with this relationship. So for the first almost full year I hid him from my girls. Because he had a young son and was going through a custody battle I was very understanding with his schedule. After almost a year I introduced the love of my life to my family. They fell hard for him too. He was/is great and everyone loves him. We quickly bought a house and moved in. things were great. About 2 years in he started to spend more and more time away from home bacuase his ex was uncomfortable with their son coming over,. She didn't allow it. He was only to see him at her house, his moms or sisters house. He is a great dad but its now been 7 years we are together and his son is his everything. We sold our house 3 years ago and his mother who was also his sons babysitter had a heart attack and so my bf moved back home with his mom to care for her and his son . He and his ex haven't resolved their custody issues and she still doesn't allow their son to see me. His mom is not fond of me because his ex convinced her that I was just after his money and that I don't like their son. That however is not true, i love my bf's son and in fact am very sad i am not able to be more a part of his life, He is the son of the love of my life, i Iove that little boy even though I hardly ever get to see him. I always have. As for his mother, she was fed horrible things about me and my bf never tried to defend me, i don't blame her. So, Ive given my bf 7 years, I still love this man and cant imagine my life without him, but i only see him Saturday nights and sunday evenings to Monday when i'm lucky. He is very involved with his son and their sports activities. I'm not allowed at those events. Last couple of years he has taken my dedication and faithfullness for granted, he takes his son to sporting events without me to respect his ex's wishes and when she goes out with her friends on the weekends he stays with his son at her house we do spend a lot of the time on the phone and his son knows its me, but according to my bf that's ok. We have had extensive talks over the years about this and he says he will never leave me, he cant love anyone the way he loves me. I told him 2 years ago if this didn't change I would have to end this because I am hurting. He has 2 lives the one with me and the other. I have been in and out of depression, and my kids miss the happy mom they had. My bf acknowledges that things have to change but he has to settle arrangements with his ex first. At this pace nothing will change til his son us 30. I'm so confused, I don't want to be without him but i cant keep this up much longer.......Help please