When you are out of body you don't leave your body behind, you are firmly tethered to your body, which is firmly tethered to the physical plane. When you go out into the astral plane you are no longer in the physical earth (well, your etheric body is, but your astral body isn't, since it's not physical at all), so whatever you see isn't the physical earth, but what the collective consciousness tells you about your surroundings. It's a place made of mind, not the physical earth. That's why there's all kinds of distortions that don't always correspond to the physical earth.