The Denmark striker only joined Forest September but struggled to make an impact for the Championship club.
He made 15 league appearances - seven of which were starts - and netted just twice.
Bendtner, who had not featured for Forest since the end of January, will now look to kickstart his career in Norway.
The former Arsenal man told Rosenborg's official website: "I am not yet expert in Norwegian football, but I know that Rosenborg is the epitome of a stable club that gets the best out of their players.
"That kind of stability I have missed since my time at Arsenal, and now we'll see if I can experience it again.