Jokanovic was installed as the permanent replacement for Chris Wilder, who had left in March with the club bottom of the Premier League.
The 53-year-old, who had previously guided Watford and Fulham to promotion to the Premier League, was hoping to take the relegated club back into the English top-flight at the first attempt.
But Jokanovic won just six of his 19 league games in charge to leave United languishing in 16th spot in the table, eight points off the playoff places, despite Tuesday's 1-0 victory at Reading.
That has prompted the club to turn back to former Leeds boss Heckingbottom, who served as caretaker manager at Bramall Lane after Wilder left the club last season.
"Following several months of strategic planning and a lacklustre start to the season, the board decided that a change was in the best long-term interests of the club," said a statement on Sheffield United's website.
Chairman Yusuf Giansiracusa said: "We look forward to embarking on a new era in Blades football with Paul at the helm. Paul possesses the personal leadership qualities and technical football skills the board feels are essential to successfully implement the board's strategic directives."