Ramzi, I just started running an Esteren campaign a few weeks ago, so I think I'm qualified to give some advice on this very topic, being a new GL myself.
Do you have Book 0? If so, I think the scenarios in there provide a good measure for how often to ask for mental resistance checks.
Rolling a few mental resistance checks during a session might make it seem like your players' characters are going to go crazy really quickly. However, failed mental resistance rolls only give temporary trauma points, which go away if the characters are able to rest for a week in a safe place. So the most important thing is
pacing, which I think is an important skill to pick up on as a GM in general. So if you run your PCs through the ringer in an extended, horrifying situation where they have to make a bunch of mental resistance rolls in a short period of time, they may very well go temporarily mad. However, you can decide to have a "break" in the campaign a short time later where the PCs have a chance to rest, so everything is more or less back to normal.
However, it's important to remember that the more mental resistance checks you have your players make, the more likely their PCs are to get permanent trauma points by rolling a 1 on the check. So it's a careful balancing act.
Here are a few examples from our campaign that I think are illustrative of some lessons I've learned:
1) You can use mental resistance checks to enforce a certain atmosphere when a player isn't giving a convincing performance. One of the PCs had an illness that drove him temporarily mad, experiencing severe delusions and whatnot. To add some flavor during the session, when the player said that his PC was going to fill his water canteen from a nearby mountain spring, I said that his PC saw in the water the faces of his dead family staring up at him. The player said, "... oookay, I'm not doing that, then." I thought this was a very unconvincing response, so I had him roll a mental resistance check. My thinking was that the result of the roll -- whether a success, a failure, or scarring -- would create a more interesting narrative event than what had just happened. He succeeded on the roll, so I narrated his PC shutting his eyes and, upon opening them, the faces were gone.
2) A mental resistance check can help spur a player's imagination. One of the PCs, a priest of the Temple, tried using the Healing miracle -- which he had never been able to use before -- on another PC, who was a stubborn, fanatical support of Magience. As the priest touched the other PCs forehead, I narrated his sudden awakening to the Healing miracle and the miraculous effects that it had on the Magientist's wounds. I asked the player of the Magientist what her PC was thinking at the moment, and she thought for a moment and said, "... I have no idea how to roleplay this." So I asked her to make a mental resistance check as this inexplicable phenomenon shook the foundation of how she believed the world worked. She not only failed, but the trauma points took her up to the syndrome stage of Mysticism. So I told the player that while her PC was shaken by this inexplicable event, she suddenly felt more resolved than ever to prove to the priest that his system of beliefs is wrong and that she knows the truth. Inspired by this, the player RPed her PC shoving the priest into the snow and screaming at him, since one way that people have of dealing with trauma is "anger."
3) Sometimes it's better to not roll. Running a published scenario, there was one scene that called for a mental resistance roll as the PCs looked at a man's face horribly disfigured by some illness. However, I forgot to call for the check at the right moment, and with the way the scene was going, the roll and its results just interfered with the pacing of the scene.
Do you have any specific concerns about the sanity system? One other thing to consider is that it requires a lot of bookkeeping if you keep track of it all yourself in secret.