Terzic was interim manager of Dortmund from December 2020 to May 2021 and guided the team to German Cup success.
The 39-year-old was assistant to Lucien Favre in 2018 and, following the Frenchman's sacking, took over on a temporary basis until the end of the 2020-21 season before becoming the club's technical director.
"By now, people should know how important BVB is in my life," Terzic said. "We will give everything every day for the success of the club."
Rose, who joined Dortmund from Borussia Moenchengladbach in 2021, left by mutual consent last week after they ended the Bundesliga campaign eight points behind champions Bayern Munich in second place.
Dortmund dropped down to the Europa League after finishing third in their Champions League group before being eliminated by Rangers, and they suffered a shock German cup exit at the hands of second division St Pauli in January.
"We are convinced that this personnel decision is the right one for Dortmund," sporting director Sebastian Kehl said.
"Edin knows our club, the environment, a large part of the team and the adjustments we want to make in order to be able to offer our fans successful football."