If Sinn Fein had not "corrected" its position on Gerry Adams' arrest they should have been put out of Northern Ireland's power-sharing government, the First Minister said.
Peter Robinson said his deputy Martin McGuinness needed to clarify his views after he alleged a rump of officers left over from before policing reforms was responsible for the republican party leader's arrest.
He said his partner at Stormont needed to make clear whether he had breached his pledge of office which requires support for the police.
Mr Adams, 65, was released after four days questioning by police investigating the 1972 murder of Jean McConville.
Mr Robinson said: "Why would we leave the (ministerial) executive because Sinn Fein breaches the pledge of office?
"If Sinn Fein breaches the pledge of office they should be put out of Government."
The Democratic Unionist leader added: "We would not be slow in bringing forward a motion for their exclusion. Indeed, if Sinn Fein had not corrected their position the motion would have gone down."
Wednesday's arrest triggered a bitter political row at Stormont, with Sinn Fein accusing an "anti-peace process rump" within the PSNI of orchestrating the detention with the aim of damaging the party ahead of the European and local government elections.
This was angrily rejected by political rivals, whose fury intensified when senior Sinn Fein figures indicated that their support for the police - a critical plank in the peace process - would be "reviewed" if Mr Adams was charged.
Mr Robinson denounced those remarks as "bully-boy" tactics.
The rapturous welcome the Sinn Fein president received in a west Belfast hotel on his first public appearance after his release on Sunday was in marked contrast to the angry scenes outside the police station as loyalists protested at the decision to free him.
There was disorder in the loyalist Sandy Row area of Belfast, with petrol bombs and stones thrown, though no-one was injured.
Mr Robinson leads an often-fractious coalition of Northern Ireland's five main parties, of which the DUP and Sinn Fein are the largest.
While the East Belfast assembly member and Mr McGuinness have united to condemn the actions of dissident republicans opposed to the peace process who have killed police officers, soldiers and a prison warder, there have been disagreements between the parties over policing and justice before.
Differences surround whether whether police should continue to make arrests for a whole range of conflict killings before the Good Friday Agreement of 1998, including by the IRA.
Mr Robinson has urged that everybody should be equally subject to the rule of law, while Sinn Fein favours a South African-style international truth and reconciliation commission.
Paramilitary prisoners were released shortly after the Agreement but that did not include those yet to be convicted of troubles crimes, who could serve two years in prison as part of special provisions.
The actions of senior Sinn Fein member Gerry Kelly have also been scrutinised by unionists angered after he wrote a book about his 1983 escape from the main paramilitary prison, the Maze.
His comments at a republican commemoration of two IRA men killed by their own bomb in Co Tyrone also prompted a DUP complaint, ultimately dismissed by a standards watchdog.
Sinn Fein has urged unionists to do more to influence loyalists during the riots which often accompany some flashpoint parades and Mr McGuinness has blamed the loyalist paramilitary Ulster Volunteer Force, who Northern Ireland's Police Federation believes are no longer on ceasefire, for recent crime in Belfast.
The five parties held months of talks around Christmas about dealing with the toxic legacy of the conflicts, contentious parades by loyalists and republicans and the flying of the Union flag from public buildings.
Those negotiations broke up without agreement.