Onward into Madness

 Part of the huge appeal of horror is the descent into madness. Its something we can all relate to. Who hasn't stayed up late nights carving the inestimable names of nether gods into their flesh? No? That's OK. I guess we'll just have to fake it.
The World of Darkness has an excellently mod capable trait called Morality. Love it or hate it, its there to add to the game. It goes all the way back to Vampire the Masquerade and of course the fishmalk. They've already shown time and again how to tweak, adjust, and fine tune it to suit the sort of descent into crazy that you're into. Or to reflect the damning consequences of some preventable actions, such as Hubris in Mage the Awakening or Harmony in Werewolf the Forsaken. Its also spelled out how to do this in World of Darkness Mirrors, if you have it read it. If you don't get it. Its just that useful and its become a constant companion of mine.
The default system (for mortals anyway) is Poe's, "the Tell-Tale Heart" insanity crops up from the internalization of guilt or the consequence of cognitive dissonance brought about by some traumatic experience. Poe's narrator kills an old man, he finds, like it or not he's violated some social norm. He's transgressed and he knows it, this is the Degeneration roll in action, in this case our hero's failed the first roll. Throughout the tale he runs his mouth quiet a bit, mostly trying to justify what he just did. A Morality roll is called for just for this reason the character's actions aren't just a threat to their system of ethics, its also a threat to how they see themselves. Fail the roll and your character can't come to terms and just like Poe's narrator, insanity follows.
All in all, well done White Wolf. Call of Cthulhu, and its variations (there are many, Cthulhutech anyone?) on the other hand defines insanity as a static, largely irrecoverable value. Sanity as a zero sum game. Like poor Danforth once you've known enough of the Mythos (see: cthulhu) the your fragile world view comes crumbling down. The greater the horror the greater the cost. Insanity is broken down between temporary an immediate reaction, long temporary lasting a number of hours, and indefinite which is severe and persistent.
I've tried to create a hybrid of the two system, something which fits the World of Darkness, but has that same certain cost Call of Cthulhu evokes.  If you're interested in using this system in the old world of darkness, I would turn you toward the Hunter the Reckoning Player's Guide which contains rules for essentially mortal player characters called Bystanders and systems for how they keep or lose their minds in the face of the supernatural.
For another mad variant, World of Darkness Mirrors offers its own traits, Forbidden Lore and Reason, since I put together my game before the book was released I've developed my own traits and system.

Sanity 

The measure of character's mind in this game is represented by Sanity.  Unlike standard morality or willpower Sanity is the measure of your character's long term psychological wellness.  As the horrors of the Mythos slowly seep into every aspect of the character's life the psychological structure by which the person sustains himself collapses. Sanity begins at 10 for all characters.  Each dot of Mythos reduces maximum Sanity by 1, meaning Sanity may never rise higher than 10-Mythos rating.

Sanity Loss 

Whenever the character is exposed to horrors equal to or less than his current Sanity rating roll a number of dice associated with the horror experienced. If the roll succeeds the character manages to avoid any long term harm but must make a Sanity roll to determine if the character becomes Unhinged. On a failure the character may develop a derangement. The character's Sanity rating drops, he has become inured to the lesser horrors of the Mythos. Regardless of success or failure on the degeneration roll a Sanity roll is made, if the rating fell the new rating is rolled and a failure here means the character gains a minor or severe derangement. In addition if a character fails to achieve a number of successes more than the lowest of Resolve or Composure - 5, they become Unhinged.

At Sanity of 10-9 a character is immune to developing a derangement.

Sanity may only be regained when appropriate at the Storyteller's discretion, suggestions for such an improvement include: spending 10 points of experience, at the end of a story, upon reaching 5 in a single ability, defeated or destroying the Mythos, psychotherapy. Drugs may treat the systems by providing temporary dice bonuses when resisting derangements, but provide no permanent solution.

Unhinged 

A character who is Unhinged is no longer in control of their immediate actions, the sheer terror they have faced has temporarily stricken them mad. Whenever this occurs the storyteller rolls a single dice to randomly determine how the character is affected. Storytellers may simply assign an effect if they choose, players and Storytellers alike are encouraged to add to the list of responses.
For example: On a shadowy country road psychologist Sylvia Benson's car strikes something in the road. Getting out of her car reveals a man in desperate straights, he's clutching a book tightly to his chest, but his lower half is mangled beyond recognition. She's hit someone! The storyteller rules this horror is more than she can tolerate (her Sanity is 10, the Storyteller decides this is an 8 rated event). She rolls no successes on the degeneration roll, her Sanity drops as Sylvia realizes she's accidentally killed someone. She rolls her new Sanity rating of 9, if her Sanity was lower she would garnish a derangement, instead she rolls one die a 7 and flees into the woods.

  Sanity

Sanity	Horror	Dice
10	Looking something disconcerting or  particularly confusing.(surprised to see a human corpse, skim a minor mythos tome)	5
9	Encountering something frightening, disgusting, or extremely confusing.(running across a small stream of flowing blood)	5
8	Dealing with something causing severe panic, disorientation, or loathing.(reading a minor Mythos tome, seeing an ethereal or fleeting entity)	4
7	Seeing something that causes nausea or stupefaction.(witnessing a friend's violent death, awaking trapped in a coffin, skimming a major Mythos tome)	4
6	Find something otherworldly or shocking.(meeting someone you know to be dead, witnessing a minor or vaguely humanoid entity, utilizing a minor Mythos Rite)	3
5	Harming Intentionally('necessary' torture) or Major Property Damage (arson).  Observing something utterly terrifying. (be subject to a terrible Mythos Rite, fighting with a particularly dangerous entity, reading a major Mythos tome, utilizing a moderate Mythos Rite)	3
4	Viewing something that is extremely shocking.(being subject to a major Mythos Rite, utilizing a powerful Mythos Rite, undergo severe torture, seeing a corpse rise from its grave, witness an utterly alien entity)	3
3	Run afoul of mind-damaging horror. (be forced to engage in cannibalism, summon an Old One, read the dread Necronomicon, give worship to one of the Mythos entities)	2
2	Happening upon extreme horror. (being resurrected after death, in the presence of a Great Old One, witnessing widespread catastrophic destruction, become a cult founder)	2
1	Incomprehensible cosmic evil. (brought before the seat of an Elder God, raising dread Cthulhu, committing mass murder )	2

Becoming Unhinged

Random Results	Response	Effect
1	Faint	Proximity to the horror causes the character to pass out, they remain Incapacitated for 5 minutes. If someone attempts to revive the character, the Incapacitated character may make a Stamina+Resolve at -3 to see if they come to each minute.
2	Flight	Terrified the character is forced to flee regardless of the situation, the character runs   away from the source of his fear for 5 -Composure (minimum 1) minutes without any regard for anyone's (including his own) personal safety. Any action not related to escape suffers a -5 penalty, furthermore particularly injured or out of shape character's may need to roll to avoid passing out from the   strain.
3	Nightmares	The sight is ingrained in the psyche of the character, they do not regain their next point of Willpower from sleep, further they are treated as though they did not sleep at all for the purpose of fatigue.
4	Self Destruction	Unable to manage the horror before the character tries to cope through self injury. This can manifest as cutting, punching or hitting, burning, picking at wounds or sutures, self-biting, poking or stabbing, or ingesting of harmful items or substances, this is not suicidal in nature. The character must cause at least 3 Bashing or 1 Lethal point of damage to themselves.
5	Crippled Virtue	The character temporarily finds herself unable to restrain her worst indulgences as her sense of right and wrong is assaulted. The character must indulge in her Vice whenever able for 24 hours and does not receive  willpower for doing so.
6	Cower	Fear overwhelms the character driving them into a fetal state. Suffer a -2 to all actions for the remainder of the scene and must make a Composure+Resolve to take any actions against the object of the character's fear.
7	Stunned	Shocked beyond reason the character temporarily retreats inside themselves. Twitching and drooling the character halves his Speed and suffers a -3 penalty to Mental Skills for the scene.
8	Scream	Faced with something the character's mind cannot come to terms with the only response is to cry out. They cannot speak until their next available action (or roughly one minute). The character suffers a one time -5 penalty on their next action as well as making themselves very obvious targets to any hostiles.
9	Lose Bodily Control	Raw power of the fear manifests in a physical way. Temporarily lose control of bowel function and suffer a -1 to actions for the remainder of the scene as well the subsequent embarrassment. Alternatively the character retches and vomits profusely suffering 1 Bashing but avoiding the smell.
10	Shaken	Experience has left a temporary blight on the character's sense of self-worth losing a point of willpower immediately. For the remainder of the scene no actions can benefit from willpower or the 10 again rule.