Hilariously, all of the non-story-related NPCs in the second game were voiced by the same person.
Nightmare Fuel: Don't eat those mushrooms. Everyone will laugh. Or cry.
The Ghost Pig in the sequel is quite creepy compared to the others who are at least goofy.
The Water Temple after time has frozen. Not only is the whole world in black and white now, everything frozen in place, even the water makes it creepy. To make it worse, it's followed with this music!
That One Boss: The Red Pig is considered the most difficult.
The Orange Pig in the jungle is also pretty bad due the fight being in a tall and vertical room and the bag was rotating at the top of a series of tiny platforms.
The evil Earth Pig in the second game may also count.
That One Sidequest: The Race mini-game in the first game, and the Trolley one in the second.
The race in the first game was actually pretty simple when you learned the trick, avoid jumping as much as possible unless there's a direct obstacle, and stay on the ground as long as you can so you can stay at top speed which would allow you to finish the race.
From the second game, ladies and gentlemen, let me introduce you to the Secret Towers. Each of which requires a pair of Guide Dang It sidequests to even get the key to open. Then you have to find the door to each (also a Guide Dang It, as the game gives no hint as to where the doors are, and they're invisible). Oh, and there are three of them. Most players give up long before discovering the Platform Hell within.