The Hammers raced into a three-goal lead at London Stadium through Jesse Lingard, Jarrod Bowen and Tomas Soucek, who touched in a Michail Antonio header.
But forward Alexandre Lacazette headed a late equaliser after own goals from Soucek and Craig Dawson set up an unlikely comeback.
This game is going to give me a few nightmares because it is really difficult to stop some of the things we have been doing to hurt ourselves. Arteta told BBC Sport.
The result leaves the Hammers fifth, two points behind fourth-placed Chelsea, while Arsenal move above Aston Villa into ninth.
Arteta was full of praise for his side’s second-half performance, declaring it the best [Arsenal have] played all season.
At 3-0 down, Arsenal looked almost certain to lose further ground in the race for European places.
Lacazette gave the Gunners hope when found the net with a volley which took a deflection off Soucek for Arsenal’s opener just before the break.
And Dawson turned in Calum Chambers’ cross in the 61st minute to further cut the deficit, before the France striker rose highest to level the game late on after good work from Nicolas Pepe.
That is the level we can show, Arteta added. We could have scored six or seven goals and won the game, but we cannot have two faces.
We stuck to the plan we had because we were getting further away from it. We had the belief we could do something.
A defeat would have opened up a 10-point gap between the two sides and arguably ended Arsenal’s hopes of qualifying for a European competition through a top-six finish.
And while questions will be asked about their poor first-half performance, Arteta will take heart from his side’s superb reaction against a disciplined West Ham.