Swedish Minister for Home Affairs Anders Ygeman said that Sweden is planning to expel up to 80,000 refugees who did not qualify for asylum.
Refugees would be deported by charter aircraft over several years, said Ygeman.
"We are talking about 60,000 people but the number could climb to 80,000," he said.
Around 163,000 refugees applied for asylum in Sweden in 2015 which is the highest per capita number among European countries.
58,800 cases were processed last year, of which 55 percent were approved.
Swedish government has carried out temporary border checks aiming to control the refugee influx into the country.
Scandinavian countries, Germany and Austria are the main countries that refugees are choosing to go to and seek asylum.
Tensions are running high in Sweden after a 15-year-old refugee was arrested in Molndal, near Gothenburg, after stabbing a 22-year-old asylum centre employee to death.
Officials say at least 35,000 unaccompanied minors sought asylum in Sweden in 2015. The number is five times more than the numbers in 2014.
Swedens neighbour Denmark passed measures on Tuesday aimed at limiting the rights of refugees, some laws include the confiscation of valuables and a three year of family reunification delay.
Greece to blame?
The announcement came shortly after European Commission circulated a draft that accused Greek government of not fulfilling its duty and seriously neglecting its obligations to control the external frontier of Europes Schengen area.
"The draft report concludes that Greece seriously neglected its obligations and that there are serious deficiencies in the carrying out of external border controls that must be overcome and dealt with by Greek authorities," European Commission Vice President Valdis Dombrovskis said in a press conference.
Schengen area entitles every EU citizen to travel, work and live in any EU country without special formalities.
On the other hand Greek government spokeswoman Olga Gerovasili accused the European Commission of playing "blame games" and said it failed to keep its promise about a programme that was agreed upon last year to relocate tens of thousands of refugees stranded in Greece.
Europe is struggling to cope with the refugee crisis as tens of thousands refugees arrive on Greek beaches while many refugees die during the dangerous journey across the Mediterranean Sea.
Most of refugees are arriving by boats and are facing cold wintry conditions.
The United Nations said at east 46,000 people have arrived in Greece in 2016 and at least 170 people were killed during the crossing.