It's easy to chill, when your team is winning. Just let Rob & Kim start struggling again, and you'll see them (Rob) fighting.
I personally don't think Peter is being that bad. I do think he is pushing Sarah a little bit hard, especailly with her having trouble with the hydrolics in her artificial leg. But I heard him say last episode something about them just needing to have fun. So I don't think he is overly concerned if they lost. Sarah on the other hand, is the one shown wanting to give up right in the middle of tasks. Maybe it is the stuff we are not being shown, that she is mad about.