This is just not a good excuse.
It covers somethings, but not others. First time he talks about his Dad's friends, and there's Jimmy the Hat, they ask if he always wore a hat, and Karl says he didn't see him wear one, but he didn't see him that much, and they joke that maybe it's ironic. So when the story reappears, that's actually why he had the nickname. They tried a piece of material, they riffed on it, and the modified version was repeated.
I'll give you an even more blatant one. Anecdotes in the first person are more effective, a common theme in comedy writing is always tell story style jokes in the first person. So when Karl says his dad met a man that looks like Ken Dodd, and said 'nice to meet you Ken', it's not nearly as effective as when Karl later tells the story and makes himself the person who met the guy.
Memory isn't accurate, but it won't put you in a place your dad was. Maybe it happened to his dad, maybe it never happened, I have no idea, but it's blatantly purely comedy when it's repeated from the first person.
Other clear indications it's bullshit is characters who are attributed to things. Like when he first talks about getting a mattress from the back of a van, it's one of his dad's friends. Then when they've established his 'uncle' that has two TVs and sleeps in a boat, it becomes his old mattress, because it's a comical addition to the sleeping in a boat thing.
He says things which blatantly contradict other things. He said his dad cut a mattress, which fucked it up, so he had to replace it and the catalog company ripped him off. Then magically he suddenly has it again years later.
We'll never really know, but massive amounts of the show are completely made up. His brother never stole a tank, his dad didn't crash a train, and his friend doesn't live in a hole in the dirt.