Around 120 migrants were aboard the inflatable craft when it ran into trouble, according to survivors. Only 16 people were rescued, while the missing included two babies and three children under the age of 12.
Three babies died off the coast of Libya on Friday while 100 people remained missing in the latest Mediterranean migrant shipwreck as European leaders reached a hard-fought deal to tackle the issue.
Around 120 migrants were aboard the inflatable craft when it ran into trouble, according to survivors taken to Al Hmidiya, east of the capital Tripoli.
Just 16 were rescued, while the missing included two babies and three children under the age of 12.
Survivors who made it back to Libyan soil recounted their ordeal following their pre-dawn departure from Garaboulli, east of Tripoli.
During the journey there was an explosion on board, after which the motor caught fire and the vessel began taking on water, they told AFP.
The passengers included several Moroccan families, none of whom survived, as well as Syrians and Sudanese, they said.
"When I saw the number of people on the boat, I refused to get on board, because we had been told that we would be 20 (passengers)," said survivor Amri Swileh, from Yemen.
Showing his bruised arms, the 26-year-old said smugglers threatened to shoot him and forced him onto the boat which was just eight metres long.
"I saw people burn," he said.
"I lost all of my Yemeni friends who were with me. All five of them are missing."
While there were up to 15 women on board, the 16 rescued passengers were all young men.
Another survivor, Bakari Badi, said early in the journey a man fell into the water and had to climb back onto the overcrowded craft.
"People were saying to the captain to go back to Libya but by then our engine exploded. So many people were wounded," the 32-year-old Gambian said.
'I saw people die'
The shipwreck came after European Union leaders reached an agreement aimed at sharing the responsibility for hosting migrants more fairly across the bloc, although strong differences between countries remain.
EU President Donald Tusk warned there was no guarantee it would work, while France and Austria ruled themselves out of hosting migrant centres.
The poorly-equipped Libyan coastguard were forced to abandon most of the bodies of migrants who died in Friday's shipwreck "for lack of resources", a coastguard captain said.
Salem al Qadhi said he was shocked when his vessel reached the site.
"It was horrible to see," he said.
Fishermen had spotted the vessel in difficulty and contacted the Libyan coastguard.
Kobrem, a 17-year-old survivor, said he clung onto a rope attached to the wrecked boat for two hours until rescuers arrived.
"Each of the other survivors held onto something, mostly empty containers," added the teenager, who said he paid smugglers around $400 (340 euros) for the crossing.
"I saw people die: babies, children, women, men," he said.
There were further rescues off the coast of Garaboulli on Friday, with 345 migrants saved from three boats, navy spokesman General Ayoub Kacem said.
Libyan authorities have been involved in the rescue of hundreds of migrants in recent days.
On June 18, five bodies were recovered and more than 100 people were saved after being shipwrecked off the Libyan coast.
Nearly 1,000 migrants were rescued on June 24 from multiple inflatable boats which had run into trouble during perilous bids to cross the Mediterranean to Europe.
The agreement reached by EU leaders overnight includes plans to set up secure centres in Europe to process arrivals' arrival claims, but cracks are already starting to emerge, with France and Austria ruling themselves out as hosts.
The Brussels pact also proposes setting up "disembarkation platforms" outside the EU as a way to reduce the number of people taking the perilous sea journey.
But the forces of Libyan strongman Khalifa Haftar on Friday rejected any foreign military presence in the south of the country to stem migration.
Haftar heads the self-styled Libyan National Army, which controls much of eastern Libya and supports a parallel government that challenges the authority of the country's UN-backed unity government in Tripoli.