Galaxies like the Milky Way indeed could not have formed as a unit at such an early era after the Big Bang. The first galactic objects were smaller and denser than present-day galaxies, and thus could collapse and pull in gas on timescales of tens, rather than hundreds, of millions of years. Once gas compressed inside such a protogalaxy, stars could begin to form т?? an especially tricky step that is not fully understood т?? and then galactic evolution could proceed quickly.
Astronomy magazine subscribers can read the full answer for free. Just make sure you're registered with the website.