the odds are not stacked against them, plants (and other living creatures) do their best to colonize new places. That's why trees have soooo many seeds. Reproduction is the goal, not longevity.
Living in an urban forest, ripping up wild young trees is part of normal yard maintenance. I probably get at least a dozen a year. Two of my full-sized trees in my yard were wild seedlings that I allowed to grow because I like those particular kinds of trees (a catalpa and a sumac), and they were in good locations.
This also includes invasive tree species like buckthorn, which is a villain in our area. Buckthorns pop up constantly.
I guess one could view urban tree dynamics as extreme selection for certain traits. They have to be pretty, and in a location that won't inconvenience humans, and they can't do anything to inconvenience humans, like cause allergies, and they have to be extremely resilient.
