When kids are young, like yours, it is easy to over schedule them, but as time goes on and school becomes much more demanding you will really have to scale back their activities. We make school #1 with homework being the first line of business after school. When my kids were really small I didn't have that problem as we didn't have the money for the activities and my son was to sick to join anything.
As much as I love sports, I am a bigger fan of free play for kids, we have our whole adult lives to follow schedules and demanding routines. Childhood is the only time you can just hang out if you want to or do spur of the moment things. I am an avid reader and trips to the library each week are a must.
If you have a pool, how about giving up swim lessons during the warm months when you can use your own pool? We joined the YMCA to use their indoor pool during the winter months. My kids no longer take swim lessons, they swim very well and are not interested in joining a swim team, so we let the lessons go. Using the pool during the summer creates lots of free play time with or without friends.
How about karate once a week, is there really a need for twice a week? Are they going to go into competition? Soccer and Basketball are popular with my kids and they have opposite seasons, so there is no doubling up on sports. My son who is 7 now plays baseball in spring and early summer but he doesn't like soccer. We have no scheduled sports or activities over the summer, lots of free time for family and friends.
We also leveled out an area in the backyard, used railroad ties as a frame, line with plastic, then flood it in December for outdoor ice skating. My kids took lessons for a short time just to learn how to skate, now we have pick up hockey games in the backyard with neighborhood kids which is great as my son loves hockey but due to health issues he can not play on a team.
With this winter being so snowy (we are just north of Boston MA) my kids have discovered the joys of sledding on the bigger hills. We bring hot chocolate and snacks and have our own tailgate party while sledding. I had forgotten how much I liked sledding, I have a few spectacular bruises but many many more laughs. We do wear helmuts for sledding, it is dangerous, that is why I never took my kids when they were younger. My kid are 5, 7, 9 and 12 which makes sledding eaiser as they all can bring their own sleds back up the hill.
Good luck, you will figure out what is best for your kids as time goes on.