HSBC in New Malden put up four flags during the Olympics to represent each nationality of staff members who work at the branch.
But Tamils were enraged at the gesture, which they said brought back memories of torture, rape and massacre from the country many of them have fled and consider a genocidal state.
K Ilankovan, of the Institute of Tamil Culture, said if the flag was part of a bigger display it may not have caused offence, but he said people felt it targeted the community directly.
He said: “It is insensitive. They should be more aware. If you are in the Olympics mood put all of the flags up, not just a few.
“Personally I am behind the British flag, that’s who we need to support.”
The manager of the bank, in New Malden High Street, removed the flags after residents complained and a spokesman said they have no other cases of anything similar.
Councillor Yogan Yoganathan said he has been a customer of the bank, which brands itself as the world’s local bank, for 44 years.
He said: “Obviously some of the people are going through pain still. I can understand their frustration. “We need people to work together and respect each other.
“HSBC has to be careful. They are serving the community and have to make sure they look after the community as a whole.”
HSBC was the first foreign bank to initiate a branch in Jaffna, Sri Lanka, and now has 16 branches on the island.
An HSBC spokesman said: “The flags were meant with the very best intentions and the bank unreservedly apologises to any members of the Tamil community who were offended.”
The people of Sri Lanka endured more than 25 years of violence and conflict during the civil war, which arose from tensions between the majority Sinhalese and the Tamil minority.
A UN report published in 2011 said both sides in the conflict committed war crimes against civilians, but the Sri Lankan government rejected the report, describing it as biased. (YLG)