A Maxence Lacroix own goal gave Arsenal the lead in the 80th minute before Palace's Marc Guehi levelled in stoppage time to take the tie to penalties.
Arrizabalaga then made it an even more miserable night for Lacroix by saving the Frenchman's effort as Arsenal won the shootout 8-7.
“Emotionally, you have to be focused, be strong," Arrizabalaga told Arsenal's official website.
"Because obviously when you concede in the last minute and then you have to go to penalties, you need to 100% focused on your penalty takers you need to save.
"So it was a change of mindset, and it worked. And then credit to the guys, they took amazing penalties.
"They keep us in the game, and they give me an opportunity to make a save."
Premier League leaders Arsenal next host Brighton & Hove Albion on Saturday.