Former Guantanamo Bay detainee Moazzam Begg has been arrested on suspicion of Syria-related terrorism offences.
Mr Begg, 45, from Hall Green, in Birmingham, who was held in the US-run military prison in Cuba for nearly three years, is suspected of attending a terrorist training camp and facilitating terrorism overseas, West Midlands Police said.
A 36-year-old man from Shirley, in Solihull, and a 44-year-old woman and her 20-year-old son, both from Sparkhill, in Birmingham, have also been arrested on suspicion of facilitating terrorism overseas.
All four are being held at a police station in the West Midlands, police said, while their three home addresses are being searched by officers from the West Midlands Counter Terrorism Unit. Vehicles and electronic equipment are being removed for forensic analysis.
Non-governmental organisation Cage, which campaigns for the rights of people detained during counter-terrorism operations and for whom Mr Begg is an outreach director, said it was "outraged" by his arrest.
Asim Qureshi, research director of Cage, said: " Cage calls on all defenders of civil liberties and the rule of law to stand up and protest against the serious curtailment of yet another victim.
"The message may be unpalatable to those who wish to shroud their abuse in secrecy but that can never justify an attack on the messengers.
"We are disgusted that Moazzam Begg is being retraumatised with the same guilt by association accusations that resulted in his unlawful incarceration in Guantanamo Bay.
"We fully support our colleague and see his arrest as politically motivated and as part of a campaign to criminalise legitimate activism."
Around a dozen plain-clothed police officers visited Mr Begg's home in Boden Road at different times throughout the day following the low-key operation to arrest him.
Neighbours claimed an officer had been seen taking photographs of a green people carrier parked near the property before it and another vehicle were removed on a low-loader.
Peter Miles said: "By about eight o'clock this morning there were several vehicles over there and people around the front door with blue gloves on, carrying bags. Then a low-loader arrived and took two cars away, and after that a police van arrived.
"I have never seen the man and I had no idea who lived there until the media told me."
Other residents said they were fully aware of the campaign to secure Mr Begg's release and his subsequent high-profile, but did not know he was living in their street.
One man, who declined to be named, said: "I looked out of the window and saw a couple of guys going in and out of the house but they weren't in uniform.
"I know of Moazzam Begg but I didn't know he lived there even though I have spoken to him a few times."
Another resident, who also asked not to be identified by the media, described Mr Begg as a model neighbour who was always very quiet and well-mannered.
"My husband went out to work at about 7.30am and then phoned me and said it looked like there had been a raid going on over the road," she said.
"There were four unmarked vehicles outside but it was utterly silent, there were no lights and no sirens."
West Midlands Police said it had decided to name Mr Begg as a result of an "anticipated high public interest" but added his identification does "not imply any guilt".
Head of investigations for West Midlands Counter Terrorism Unit, Detective Superintendent Shaun Edwards, said: "All four arrests are connected. They were pre-planned and intelligence led. There was no immediate risk to public safety.
"We continue to urge anyone planning to travel to Syria to read the advice issued by the Foreign Office."
Originally from Birmingham, Mr Begg moved to Afghanistan with his family in 2001 before taking them to Pakistan in 2002 when the war began.
He was detained in Islamabad, Pakistan, as an " enemy combatant" in January 2002 and was taken to the Bagram internment centre for about a year before being transferred to Guantanamo.
The British citizen was released along with three others in January 2005 and was allowed to return to the UK where he was arrested by the police before being released without charge.
Mr Begg has always maintained that he was only involved in charity business and that he has never been involved in any kind of terrorist activity.
He wrote about his travels to Syria in a publicly-available blog entry dated December 24 2013 on the Cage website, in which he also described having his passport confiscated to prevent further travel.
In July 2012, Mr Begg wrote, he visited Syria and met former prisoners who had been held by the Assad regime.
In a second visit made in December 2012, he met current and former prisoners and also visited refugee camps.
As well as witnessing British aid being brought in, he claimed he also met British fighters.
Around 250 British-based extremists who went to train and fight in Syria have returned to the UK, it has emerged.
Ministers have been told that over the past two years more than 400 Britons have gone to Syria and it is now thought just over half have returned.
In January alone, 16 people were arrested on suspicion of terrorism offences related to Syria compared with 24 arrests in the whole of last year.