Tom Cleverley grabbed a dramatic late winner to consign Arsenal to yet another dispiriting defeat in the Premier League.
Per Mertesacker had given Arsene Wenger's side a first-half lead when he met Granit Xhaka's corner to score his first league goal since December 2013.
However, substitute Mesut Ozil missed a glaring chance to double his side's lead and they were made to pay, as Richarlison was awarded a penalty after he was felled by Hector Bellerin.
Troy Deeney, who came on as a second-half substitute, took the spot kick and scored, but the drama wasn't over.