Anton now says that backticks are bad practice since the behaviour will not be stable under changes to the software. The fact that Markdown doesn't look inside block-level tags is part of the specification of Markdown so anything that pretends to be Markdown has to accept that. But Markdown does look inside backticks and what it does there can depend on the configuration.
]]>\{ to { (and \\ to a single backslash, and similar for other things). Then MathJaX only sees { which it interprets as a TeX group and so doesn't display the brace. One solution is to use \lbrace and \rbrace as voloch says. Another is to put the mathematics inside an HTML tag so that Markdown doesn't look at it. If the paragraph doesn't contain anything else that Markdown should look at, you can just put <p> ... </p> around it.
]]>