This one probably isn't going to make much sense unless you know a little bit about the story of Job, which attempts to reconcile the existence of suffering or evil with the existence of God. Job was blessed with wealth and family, so basically, God made a bet with Satan that allowed Satan to remove those things, and still, Job would continue to praise God. Once his possessions and children are taken away, Job's wife suggests that Job "curse God and die." Despite his difficult circumstances, he does not curse God, but rather curses the day of his birth.
Knowing this, we can find the links and similarities between Job and Larry Gopnik. Larry never curses God; this is proven in the scene at the pool with his brother, where his brother blames Hashem for all his problems, but Larry does not agree. When Larry goes to the senior Rabbi, Rabbi Marshak, or the one who is closest to God, he is too busy to see him. This symbolizes the ridiculousness of spirituality being able to answer life's questions. And of course, the tornado at the end alludes to God speaking to Job from the whirlwind, saying that He will not explain why these bad things are happening to him. "Accept the mystery," Clive's father tells Larry, when confronted about the envelope of money left on his desk. Larry also says in his lecture: "The Uncertainty Principle. It proves we can't ever really know...what's going on. So it shouldn't bother you. Not being able to figure anything out. Although you will be responsible for it on the mid-term."
There are other religious allusions as well: Larry looking from his roof to see Mrs. Samsky sunbathing nude in the backyard is similar to King David seeing Bathsheba. My own interpretation is that Mrs. Samsky is a pleasing form of the devil, trying to tempt Larry to commit adultery.
What really ties the film together nicely is the allusion to Schrodinger's cat. Very simply stated, Schrodinger proposed an experiment where a cat in a closed box was either alive or dead depending on the state of a subatomic particle. The thought experiment serves to illustrate the bizarre nature of quantum mechanics. We see this at the end, when Larry is agonizing over whether or not to accept Clive's bribe. Almost as soon as he does, he receives a phone call from his doctor indicating bad medical news.
When all else fails, know that the first five minutes (the Eastern Europe scene) explains the rest of the movie: "Receive with simplicity everything that happens to you." The alternative, according to the Coens, is to curse God and die.
This can be a frustrating picture to get through at first, especially if you don't get some of the references, since the story doesn't seem to go anywhere and nothing seems to happen. But a little research goes a long way to really appreciating this one.