I can agree with the "Angels or Apes" notion. Think of how long humans have had any civilization to speak of above small independent villages--ten thousand years or so? Now compare that to the age of the whole galaxy--ten BILLION years or so. Life on Earth has existed for a thousand times longer than anything human has, and a hundred thousand times longer than anything that we would call civilization. That is why I expect that we will find dozens of pre-civilized yet intelligent species for every civilized one that we find (i.e. hunter-gatherers).
One the other end of things is the Fermi Paradox. Even if faster-than-light travel is impossible, and we assume that a colony world is settled for one thousand years before it sends out its own colony missions (compare to 300 years between first settlement of North America and first USA attempts to colonize distant places such as the Philippines), it would still take maybe ten million years to colonize the entire galaxy--only a thousandth of the galaxy's current age. Why, then, has no galactic empire/federation already colonized the whole galaxy? You would have to assume that NOBODY in the whole galaxy is willing to break the Prime Directive and settle Earth even for the billions of years before there were any people around here--and it's pretty certain that there has been no industrial civilization on Earth prior to our own since before the dinosaurs became extinct, or else they would have already mined all those surface veins of minerals that early humans found.
Thus, either there is NO galaxy-spanning civilization that likes to colonize Earth-like planets that have biospheres but no people, OR they have been able to resist the temptation to occupy this nice uninhabited planet for millions or billions of years--or else there is something about the conditions or biology on Earth that makes it undesirable for whatever races have been in control of this part of the galaxy for all this time. The mere notion that aliens could reach Earth but didn't want to exploit it before humans even arose implies that they are either so different from our biology that our biosphere is uninhabitable to them, or else that they have some psychological aversion to settling already-established biospheres that is stronger than their desire to have an additional world to own for even the greediest among them over thousands of generations.