I'm writing this here, so I can link to it thereafter. Many of our users have noticed confusing errors where LaTeX code does not behave as expected. If you are a one-time user, you probably just want to leave the buggy code in place and figure that a moderator will clean it up. But, if you come here frequently, you might want to understand what is going on. This is my attempt at an explanation a bit more detailed than the one in the FAQ. I should note that the real expert here is Anton, who has put in a great deal of effort getting things to work as well as they do; I'm just a bystander.
When mathoverflow displays your question (or answer), there are two javascript programs which parse your text. The first is wmd. This inserts links and images, makes words bold and italic, and strips out any HTML that might be inserted by malicious users. It probably also does some other things that I am not thinking of right now. The second is jsmath. This creates equations. The important thing to understand is that jsmath is operating on the output of wmd, not on your raw input.
This can cause problems when wmd modifies your input in unexpected ways. The main workaround is to enclose your equations in back-single-quotes. (This character `.) This tells wmd to leave them alone. You can also try to make wmd produce output which will be valid LaTeX. I do this sometimes, but it is probably not good practice because wmd may change and then your solution will likely break. (Paging the experts: are there any known cases where the back-single-quote solution does not work as expected?)
Here are the main known issues:
* and _ are special characters for wmd, involved in the encoding of bold and italics. Fortunately, the contexts in which wmd modifies these characters are largely, but not completely, disjoint from those in these characters appear in equations. If you have used one of these characters, and something is going horribly wrong, you should definitely try back-single-quoting the equation that contains it.
\{ , \} and \\ Both jsmath and wmd use backslash for special purposes. When wmd sees \* , for example, it replaces it by * . When wmd sees \{ , \} or \\ , it replaces them by { , } and \ respectively. Then jsmath sees these characters and responds accordingly; which is to say, it usually does nothing which is visible to the user. Users tend to find this experience frustrating. :)
PREVIEW VERSUS DISPLAY
There is a further issue. Remember how I said that there are two javascript programs? There are actually three. For reasons known only to Fog Creek (who wrote StackExchange, the underlying software of this website), there are two different versions of wmd. One of these is used to produce the preview that you see while writing a question/answer and the other one of which is used to produce the final displayed version. They are very subtly different, but the differences tend to be exactly in the areas which effect the interaction with jsmath. To make things worse, we don't have access to the code for the wmd which does the final display, and we do not have the ability to modify it
Anton has responded by trying to write a clone of the final display wmd and using it in place of the standard preview wmd that came with the website. At this point, his clone is very good, so you probably won't see any differences between the previewed version and the final one. However, if you do spot such a difference, please mention it at http://tea.mathoverflow.net/discussion/30/2/report-bugs-with-latex-jsmath-rendering-here/#Item_34