Vanilla 1.1.9 is a product of Lussumo. More Information: Documentation, Community Support.
Someone who knows better should confirm this, but I think that questions with an answer that has been voted for don't get this bumping treatment (the two on the question in question - I love writing that! - both have no votes) so a fairly innocuous solution is simply to vote for one of the answers.
If none of the answers deserves a vote, or if there are no answers, then you could provide one. In particular, if it has been answered in the comments then that can be transferred to an answer. Custom dictates that such an answer should be community wiki. You would then have to hope that someone else voted for the answer (you could even write that in the answer: "I'm writing this answer to stop it getting bumped to the front page, I need someone to vote for it for my nefarious plot to succeed"). This is what you say in your last paragraph with the addition of the "community wiki" feature.
Of course, this presupposes that the question shouldn't have been closed for a genuine reason (wrong level for the site or some such) in which case vote to close (and/or bring it to the attention of the +3000 users by commenting on it here on meta).
Indeed, some people only answer in comments (something that I consider a bit unhelpful to the community) and I see no problem with copying the answer into a (suitably attributed and community wiki) answer for the questioner to accept.
Hmm, according to Anton's answer on meta.stackexchange, that shouldn't have happened as David Roberts' answer got voted.
At this point, I'd wait for Anton or Scott to wake up to give you the definitive answer on how the community user is meant to behave and whether or not there's something else going on behind the scenes that us mere mortals can't see.
The community user's actions aren't logged and are pretty hard to track. As far as I know, the community user "pokes" questions that haven't had any activity in a while and are "unanswered". The SE lead developer confirmed what "unanswered" means. See also this meta.SE post. I'm not sure what happened here.
Joseph, it seems possible to me that someone voted the answer up between the time the community user bumped the question and the time you looked at it. Are you sure the answer had been voted up before the question was bumped?
Scott and Joseph: I came to this thread to share my wonderful solution to the mystery, but I see that I wasn't the first :( In fact, I think this happens rather often. I, too, noticed more than a few questions that have been raked up by "Mathoverflow" and had positive votes when I looked at them. However, they are usually in the middle of the stack, so presumably, they have been sitting on the front page for several hours by then.
It is important to remember that the periodic stirring that the MathOverflow User does is intended as a good thing: to bring back to our attention questions that, as far as it can tell, haven't been resolved. As with any automatic process, it sometimes gets it wrong, so now we know how to help it: ensuring that there is an answer with a positive vote count. If there is no suitable answer, but somehow the question has been resolved (maybe in the comments), then a community wiki answer explaining the situation is more than likely to fix it.
1 to 13 of 13