Kalvin Phillips says that "narratives" which followed former manager Pep Guardiola's comments about his weight left him feeling "frustrated".
Manchester City boss Guardiola has since apologised for saying in December 2022 that Phillips had come back from the Qatar World Cup "overweight".
Midfielder Phillips, who is on a season-long loan at Ipswich, told the "My Mate’s a Footballer" podcast that speculation on social media, "just kind of grew and grew," following those comments.
Speaking as a guest alongside his former Leeds team-mate Patrick Bamford and comedian Joe Wilkinson in the latest episode of the BBC Sounds series, the 28-year-old said: "Every club that I'd go to, where I spoke to the manager and the nutritionist and stuff like that, they'd always speak about weight before they'd say anything else".