Mohamed Salah added a second, but then Arsenal hit back with three goals in five chaotic minutes.
First Alexis Sanchez pounced on defensive hesitation, then Liverpool goalkeeper Simon Mignolet blundered to allow a leveller from Granit Xhaka, and finally Mesut Ozil put the Gunners ahead.
But incredibly then, an error from Arsenal goalkeeper Petr Cech allowed Roberto Firmino to haul Liverpool level late in the game.
A frustrated Klopp, who exchanged angry words with nearby fans as he threw down a water bottle after Firmino’s goal and appeared to have water thrown at him, said: “Of course we should have won but we had five minutes where we had obvious problems.
“It was not team defending. The first goal and the second were misjudgements. We opened the door for them in these five minutes. I’m disappointed.
“I was aggressively happy at our equaliser, I have made my apologies, but if the poeple were to apoligize to me it would take a bit longer! I did nothing, I threw a bottle onto the ground, I was not happy at that moment.
“It was not about controlling the game. In 99.9% of these situations Mignolet saves these balls - but not tonight. We should be three, four even five-nil up before we even defend a cross.”
Arsenal manager Arsene Wenger said: “I’m frustrated we did not win - but to be honest it could have been over at half time. Give credit to the players because even at 2-0 we did not show weakness and came back.
“I will keep private what we said at half time - but the positive thing was that we were only 1-0 down.
“In the first half we were paralysed, frozen, second best everywhere. I said to the players the good thing is we’re only 1-0 down. What we did was not good enough. We had nothing to lose.
“But at 3-2 we were naive defensively. If I tried to be fair overall maybe it’s a fair result.”
Jack Wilshere said: “We played like we were scared in the first half.” ends PLACE YOUR TEXT ADVERT BELOW:>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> How I Made Me =N=2.5 MILLION In 7 Days from 2 Common Product I Imported and how you can do the same. Click HERE FOR FREE Details right now Warning To Men: This 3 Foods Is Killing Your Erection. Click Here To Know Them REVEALED!! Reduce Your Blood Sugar Level And Reverse Diabetics Within A Short Period Of Time. Click Here To See How Warning: Got Arthritis? 3 Simple Ways To Get Cured And Be Permanently Free From Joint Pains. Click Here To See Them