Wow! There's nuthin' 'round the ol' joint.
Shown above, via a tweet by Old Baseball Photos, is Ebbets Field just prior to its opening in 1913. You can see the grandstands and the backside of the outfield walls, but nothing else (except a few buildings here and there). I always assumed Brooklyn was built out the way it is today, but clearly that is wrong. The immediate area was as barren as a bald head.
A little bit of research tells me that Pigtown, the neighborhood this area was called due to the existence of several large pig farms, was economically depressed.