Why cant I refinance my second condo into a 30 year fixed?

I am the owner of 2 unit low rise condo in Virginia Beach Va. We live in the top condo unit and rent out the lower condo unit.  I had no problem getting a fixed mortgage for my top live-in unit. I have a ARM for the rental unit. Over the years I have tried to refi the lower rental unit into a 15 or 30 year fixed mortgage. I am always told I can not do that because I own both units. Any ideas on how I can solve this problem?