While I find Supernatural creepy and horrific a lot of the time, I don't find it particularly scary. I can watch it alone in the dark (my preferred way of watching, actually) and it doesn't give me nightmares after.
That said, an episode that always makes me jump out of my skin during 1 scene in particular regardless of how many times I watch it (and it being my all-time favourite, I've watched it too many times to remember) is Mystery Spot. The scene that gets me every single time in Dean getting hit by the car. I know it's coming and always try to prepare myself for it, but I always seem to gasp in shock when it happens.
The other time I've gotten a shiver down my back (the closest I can think of to being truly scared) is in Season 6's You Can't Handle The Truth when Soulless!Sam is interrogating the victim's sister. It's the combination of looks he gives the sister and Dean that, to me, is truly terrifying. He goes after the sister without the least bit of empathy we are so used to seeing Sam have with witnesses,
basically tells Dean to f*ck off without saying a word
and the look of triumph when she finally breaks is about the creepiest look I've ever seen on Jared's face.
It's right up there with the infamous smirk from Live Free & Twihard. OMG I love Soulless!Sam so friggin' much!!!