I think special consideration should be given to 'keep on racing' legs that involve a lot more travel. Examples that come to mind are season 10 from Madagascar - Finland - Ukraine. All 1 leg! Or season 22 from Bora Bora to New Zealand to Bali. Sure you may be able to catch a few z's on the flights to places, but it's still a very brutal schedule and sleep deprivation has to play a big factor that we don't always account for.
I think some of the underrated hardest legs are the ones with excessive amounts of driving as well. Season 2, leg 11 comes to mind, where they drove the entire length of New Zealand. Or Season 4, leg 4, going all throughout France, with every route marker somewhere else in the country. Some tasks might be tough, but long self-driving sections I imagine would be toughest.