Some fans are using the “political atmosphere as a cover for their own racism and prejudice”, says anti-discrimination group Fare.
It comes after Chelsea condemned a vocal minority of their fans for anti-Semitic chants during Thursday’s 2-2 draw with MOL Vidi in Hungary.
Earlier this week, four Blues fans were suspended following the alleged racial abuse of Raheem Sterling.
Fare executive director Piara Powar says such attitudes must be defeated.
“The sad fact is that in recent years Chelsea have done an incredible amount of work to tackle anti-Semitism, much of it highly innovative and impactful,” said Powar.
“But there remains throughout football a rump of people who in 2019 will see the political atmosphere as a cover for their own racism and prejudice.”
A Chelsea spokesman said the offensive songs about Tottenham fans have “shamed the club”.
Ben Holman, from educational anti-racism charity Show Racism the Red Card, says anti-Semitic abuse must be “treated seriously in mainstream society” for it to be tackled accordingly in football.
“In some incidences the message has got to the fans it’s not acceptable,” Holman told BBC Sport. “The problem is some of the chants are more historic and in that way fans don’t realise the problem with it.
“Until it’s treated seriously in mainstream society as racism you will always see it shunted off in football as not so serious.
“Racism isn’t a problem intrinsic to football. These fans are at a football match for two hours a week, but for the other 166 are members of society, taking the bus, going to work.”
According to incidents recorded by charity the Community Security Trust, anti-Semitism has been on the rise in the UK.
Meanwhile, a report by watchdog Her Majesty’s Inspectorate of Constabulary earlier this year said hate crime rose after the 2016 Brexit referendum and the same could happen when the UK leaves the European Union in 2019.
“Our organisation has always believed sport and football are a microcosm of society,” added Holman.
“We think if society is racist, football is part of society and will always have racism, so we try to educate young people in society and hope football will follow.”
Powar described the episode as a “sad indictment” of “where some people are in their understanding of racism and the impact it can have”.
“They stare history in the face and think they are somehow exempt from the judgements it will make on their actions,” Powar added.
“We should give a lot of credit to those Chelsea fans who highlighted what was going on at the match on social media or directly to the authorities.”
What have Chelsea done about it?
Last year, Chelsea condemned an anti-Semitic chant by their fans during a win at Leicester, with Blues supporters using a song about their striker Alvaro Morata to abuse London rivals Tottenham, who have a large Jewish fanbase.
Spain international Morata also posted on social media asking fans to “respect everyone”.
Holman says the Stamford Bridge club have been “progressive and forward thinking” in their efforts to eradicate anti-Semitisim from their fanbase.
In October, chairman Bruce Buck told the Sun newspaper the club may require fans found guilty of anti-Semitic abuse to visit the site of Nazi concentration camp Auschwitz rather than banning them.
The Premier League club’s Say No to Anti-Semitism scheme, which began in January, also provides one-to-one education courses.
A group of 150 people, consisting of club staff, stewards and supporters, visited Auschwitz in June to learn about the deaths of more than a million people killed there between 1940 and 1945.
“Chelsea have really stepped up their efforts to eradicate anti-Semitism from Stamford Bridge, and among their fans,” Holman told BBC Sport.
“[Chelsea owner] Roman Abramovich has ploughed a lot of his personal money into tackling anti-Semitism.
“The club realise they could ban a fan and wash their hands with them, but that person will still hold those anti-Semitic views. If they can educate them then that person may benefit society.”
Powar added: “I have no doubt that in the end these types of attitudes will be defeated.
“That necessity for cultural change applies across the football industry, not just the terraces, from the governing institutions, to clubs and the media.
“We should look at what’s been happening in the last two weeks to get more creative and bring about that culture change more urgently.”
Brighton boss Chris Hughton, one of two black Premier League managers, said clubs are on top of what they see, but eradicating anti-Semitic and racial abuse from football or society is “about a culture and making sure people are respectful of all colours and creeds”.
“Racial events in our game, which we are trying as hard as we can to eradicate, are always going to happen,” he added.
“You hope it’s something that doesn’t escalate. Sometimes when times are harder they become more relevant – but racism holds no place in our game.
“It holds no place in society but unfortunately there are always going to be incidences.”
‘Thoroughly depressing’ – Jewish community reacts
Uefa, European football’s governing body, said it will await the referee’s report of Thursday’s Europa League match before deciding on whether any action will be taken.
Incidents of anti-Semitic and racial abuse are a criminal offence.
For those that take place in the English game, governing body the Football Association works with clubs and the police to identify individuals and make sure they face appropriate action through the courts, which can impose banning orders.
Chelsea fan and writer Ivor Baddiel told the Victoria Derbyshire programme that some fans think they are “just being anti-Tottenham”.
He added: “They aren’t, they are being hugely and horrendously anti-Semitic.
“When you sing ‘Spurs are on their way to Auschwitz’, that is what you are really chanting about. Clearly there are people who think it’s OK and maybe they don’t understand why Jewish people are so offended by it.
“You would think that all but the most hardened fascists would think that was wrong.”
Simon Johnson, chief executive of the Jewish Leadership Council, said the incident was “thoroughly depressing, especially in light of the dedicated work Chelsea have done to address the problem”.
He added: “We completely endorse the club’s strong statement and would support them in any robust action which they now take against the perpetrators.”
Board of Deputies vice-president Amanda Bowman said the “disgraceful behaviour must be challenged and the perpetrators identified and punished”.
She said the organisation is “fully behind Chelsea’s ‘Say No to Anti-Semitism’ campaign” but added that “football still has much work to do before racism on the terraces is eradicated”.