The announcement follows a deal signed between the government and FARC, the largest guerrilla movement in the country. The decades-long conflict between rebel factions and Colombia has resulted in the deaths of over 250,000 people.
Colombian President Juan Manuel Santos and the country's second-largest guerrilla group, the National Liberation Army (ELN), will start formal peace talks on October 27. The negotiations with the 2,000-strong ELN will begin in Ecuador, according to an agreement signed in Caracas by both sides under the auspices of Venezuela's government.
Santos negotiated an agreement with the FARC, the left-leaning rebel group in a bid to end over 50 years of conflict. He won a Nobel Peace Prize for his efforts even though Colombian voters rejected the deal in a plebiscite held on October 2. Talks with the ELN are seen as a boost for Santos whose popularity has taken a beating.
"We are a special nation that grows despite adversities," Santos said in a speech after the announcement on Monday.
"Peace won't slip through our fingers. On the contrary, it will be stronger, and now that we will advance with the ELN, it will be complete."
Head negotiators from the Colombian government and the left-wing ELN said in March that they were ready to launch peace talks. However, Santos did not agree to sit at the table until the group had freed all the hostages.
Over the last week, ELN freed three hostages and Santos said in a televised speech that the group would not take hold more people captive.
The two leftist guerrilla groups FARC and ELN have been fighting with the Colombian state since 1964. The ELN has attacked large landholders and multinational companies. The group has been accused of bombing oil pipelines and other installations linked to Colombia's oil industry.
More than 260,000 people have died in the Colombian conflict, in which leftist rebel groups and drug gangs have played an important role.
On Sunday the Colombian president announced that he would donate the Nobel prize money to victims of the conflict.