I'm firmly in the "Caleb is damned" camp. Caleb is possessed in that scene and the sexual overtones are totally there. He fakes out the family by finishing the prayer with them, which makes them rejoice, then he goes into that strange homoerotic rant that stuns the family into silence, then he arches his back, lets out a contented, practically post-orgasmic sigh, then dies.
It's a total mockery of prayer. Katherine is later seen crying out that Caleb is damned, and William's best response is basically a hesitant "we don't know that for sure." Caleb's regurgitation of an entire apple, which do not exist anywhere near the farm and which reflects Caleb's earlier lie, proves to the family that he is a victim of witchcraft. Eggers also specifically points out in the commentary that the scene was difficult in part because they couldn't explain to Harvey Scrimshaw its exact significance, because I guess he was too young to understand just how fucked up it was.
Besides, Caleb is arguably the most rational and loyal member of the family, so seeing him go in such a way is extra horrifying.
If you look, there's actually a stain in the crotch area at that point but I have no idea if it was deliberate or not...just something I've noticed since I've seen the film about four times, several times after seeing discussions about this scene. Don't look at me like that.
Besides, Caleb is arguably the most rational and loyal member of the family, so seeing him go in such a way is extra horrifying.