Ok, I've seen a lot of people talking about why Aziraphale decides to go with heaven, but I haven't seen anyone talking about how, in Aziraphale's eyes, he has just seen proof that heaven can be fixed.
Gabriel was the embodiment of everything wrong with heaven. He didn't care about the earth or the humans on it. He claimed to be the good guy, but he never actually cared about being good. He was cruel to so many people, but especially Aziraphale.
And yet that he found love, and because of that love, he began to care about the world. He became kind. He became caring. He became good.
When Gabriel shows up on Aziraphale's doorstep with no memories, he gives him a second chance because he's so sure that even someone like Gabriel can be saved. And he's right.
Aziraphale wants so badly to believe that heaven is fundamentally good, just deeply flawed and misguided. After all, if the Supreme Archangel Gabriel can change, can learn to be better, then surely the system itself can too.
What Aziraphale doesn't realize is that it was only by going outside of the system that Gabriel found love and happiness, and that once he had, there was no longer a place for him in it. Heaven is broken, and Aziraphale won't be able to fix it from within. But he loves God and her creations so much that he has to try.