Poor baby. Definitely keep the vet appointment. It sounds like a bad infection to me. He'll likely need some antibiotics and possibly surgery on his lower jaw, depending on how bad it is. I'm glad he's still eating though.
Let us know what the vet thinks! Fingers crossed he just needs antibiotics!