That's because when the sun and rain combine to make a rainbow, they really make a full-circle rainbow. We usually don't see the full circle because the horizon of the Earth is in the way.