He is the only one who doesn't like Kel. This is mostly due to the fact that Kel often makes a sarcastic remark to Roger's baldness. Kel tends to injure him and damage his house during his visits but he usually tolerates Kel's presence, but Roger gets mad at Kel when he goes too far of his stupid actions whenever they get him hurt and injured. Unlike Sheryl, Roger is quick to punish Kenan which explains why Kenan is more close to Sheryl. In the pilot episode Roger even punished Kel even though he's not his son which Kel points out.
In the episode "Get the Kel Outta Here" Roger bans Kel from his house for good, after Kel almost killed him and refuses to forgive him. At the end of the episode Roger forgives Kel after suffering from amnesia as a result of a tree accident.