Clifford departure comes after three seasons coaching the Magic, who made the playoffs in his first two years but stumbled to a record of 21-51 this past season.
Clifford, a former assistant for the Magic under Stan Van Gundy came to the Magic in 2018 after 5 seasons coaching the Charlotte Hornets.
“We would like to thank Steve for his contributions to the Orlando Magic,” said President of Basketball Operations Jeff Weltman. “We appreciate the many sacrifices he has made as our head coach and understand the timeline of our new path does not align with his goals as a head coach in our league.