Police and security officials said they believe one of those arrested is the Karachi head of the militant network's new South Asian wing.
Al-Qaeda chief Ayman al-Zawahiri announced the creation of the new branch in September to "wage jihad" in Myanmar, Bangladesh and India, which has a large but traditionally moderate Muslim population.
"Five terrorists including the Karachi chief of Al-Qaeda have been arrested from LakriGali, Old Haji Camp neighbourhood of Karachi," a police statement said.
Local security sources confirmed the arrests and an intelligence official told AFP the men had sworn allegiance to the Al-Qaeda wing's leader, Pakistani militant Asim Umar.
"Those arrested include ShahidUsma, chief of the Karachi chapter of Al-Qaeda's South Asia chapter," another security official told AFP.
Karachi, a teeming metropolis of around 18 million people, is rife with political, ethnic, sectarian and criminal violence.
Police said the five arrested men "operate Al-Qaeda's network in the city" and were behind several "acts of terrorism".
They gave no details of these acts, but the Al-Qaeda South Asia branch claimed responsibility for a daring raid on a naval dockyard in Karachi in September.
At least three Navy officers have been arrested over the attack, raising questions about Al-Qaeda's infiltration in Pakistan's military ranks.
© 1994-2014 Agence France-Presse