Kenny Lofton was a Hall-of-Fame caliber baseball player. He fell off of the ballot in the first year.
Jim Edmonds was a Hall-of-Fame caliber baseball player. He fell off of the ballot in the first year.
Andruw Jones was a Hall-of-Fame caliber baseball player. He drew only 7.3% of the vote his first year on the ballot, and will likely fall off next year as Mariano Rivera, Roy Halladay, Todd Helton, and Andy Pettitte join an already stacked list of candidates.
You can argue that none of these players belong in the Hall.