Interim President, Michel Kafando was freed after the African Union suspended Burkina Faso’s membership and threatened to impose sanctions on coup leaders, if they did not reinstate the transitional government and release its leaders, the new junta leaders said.
However, Prime Minister Isaac Zida, who was also detained when the elite presidential guard (RSP) stormed into a cabinet meeting on Wednesday and abducted President, Michel Kafando along with the Prime Minister, remains under house arrest.
Prime Minister, Isaac Zida was the army officer who took control following the ousting of former President Blaise Compaore, the country had been seen as a role model by pro-democracy campaigners across sub-Saharan Africa.
The AU statement came after a meeting of the Peace and Security Council in Addis Ababa, in which it gave coup leaders 96 hours, or until September 22, to restore the transitional government or face travel bans and asset freezes.
"Council decides to suspend, with immediate effect, the participation of Burkina Faso in all AU activities," it said, pointing out that members of the RSP, who are linked to the kidnappings, would be held legally accountable for their actions.
Coup leader, General Gilbert Diendere who was the former chief-of-staff of President Compaore, before the citizens of Burkina Faso toppled the president last year as he sought to extend his 27-year rule, told reporters that Kafando has now moved back to his official residence.
It was reported that the interim president was freed on Thursday, but he has not yet made a public appearance.
Two other ministers have also been released, the coup leaders added.
The decision to release the leaders of the transitional government was made "as a sign of easing tensions and in the general interest", a statement read on national television said.
The RSP have been trained mainly by the US and it is considered to be the most powerful armed faction in Burkina Faso.
The armed group is known for its constant disruptions of the activities of the transitional government, as it tries to cling to the privileges it enjoyed under Compaore's rule.
Compaore is currently living in exile and has been accused of committing list of abuses, including his attempt to try and change the constitution to extend his term in office.
General Diendere has stated that he has not contacted Compaore and is willing do everything to "avoid violence that could plunge the country into chaos."