Use spear to get him down to 1 shield left, then use abide, then break him, and on the next turn boost up the spear special ability to max. Repeat that a few times and you'll win in no time.
Thanks for the tips! The two extra guys weak to swords does throw a wrench into things, is it better to just keep them alive and wail on the boss or kill them ASAP before focusing on the boss?