Draft "haters" will often point to the fact that sure-fire hall of famer Tom Brady was selected in the 6th round. So was Antonio Brown. The Packers own David Bakhtiari has claim to being the best left tackle in football. He was a fourth round pick by Ted Thompson. A list of the outliers could certainly go on and on.
The draft does matter. People will point to Brady and Brown and even some undrafted guys, but by and large players selected eariler are better. Brown's a bit of a unicorn and there are a lot of quarterbacks that would've succeded with the Belichek / Kraft infrastructure.