The incident came as Belarus, which has said it wants to defuse the border crisis, prepared a first repatriation flight for refugees to Iraq.
The Polish security forces have detained a group of about 100 refugees who crossed the Belarus border during the night, accusing Belarusian forces of leading the operation.
The Polish defence ministry said on Thursday that Belarusian forces had first carried out reconnaissance and "most likely" damaged the barbed wire fence along the border.
"Then the Belarusians forced the refugees to throw stones at Polish soldiers to distract them. The attempt to cross the border took place several hundred metres away," it said.
"A group of about 100 refugees was detained," it said, adding that the incident happened near the village of Dubicze Cerkiewne.
"Belarusian special forces led yesterday's attack," the ministry said.
Video footage released by the defence ministry appeared to show Polish soldiers surrounding a large group of refugees crouched down in a wooded area at night next to some barbed wire.
Thursday's incident came as Belarus, which has said it wants to defuse the crisis, prepared a first repatriation flight for refugees to Iraq that will have between 200 and 300 people on board.
On Wednesday, Belarus President Alexander Lukashenko and German Chancellor Angela Merkel agreed in a telephone call that Minsk and Brussels should discuss the crisis.
Thousands of refugees, mainly from the Middle East, are camped out or staying close to the Poland-Belarus border in dire conditions aiming to cross into the European Union, in a crisis that began over the summer.
The EU says Belarus engineered the crisis in retaliation for sanctions on the ex-Soviet country.
Minsk and its main ally Russia have rejected the charges and have criticised the EU for not taking in the refugees seeking to cross over.
Aid groups say at least 11 migrants have died since the crisis began in the summer.
They have called for a de-escalation and a humanitarian response to the problem.