Not signed in (Sign In)

Vanilla 1.1.9 is a product of Lussumo. More Information: Documentation, Community Support.

  1.  

    Pre-empetive disclaimer: Writing this up feels incredibly egotistical, even for me. I'm not sure anyone will care. But I am going to forge ahead and ignore that.

    As proposed in this thread (near the top of the second page), I have been conducting a randomized experiment. The last 20 times that I have answered a question on MO, I wrote out my answer, then flipped a coin and either submitted the answer as myself or as anonymous. I made no edits after the coin flip, so that my writing could not be influenced by the flip. I had pre-decided to stop the experiment with 20 answers, regardless of the results.

    The questions that I answered under my own name are the 9 answers from "Algebraic cycles of dimension 2 on the square of a generic abelian surface" to "Non-diagonalizable complex symmetric matrix", with the exception of "Choice vs. countable choice". Bring up my user page and sort the answers by "newest" to see them all. I kept "Choice vs. Counteable Choice" outside the experiment, because I felt that a vague answer like this would only be of interest if signed by a known mathematician; maybe I wimped out here.

    The Gravatar algorithm always assigned me the same image as anonymous, so my anonymous posts are also easy to pick out. You can see the icon in question on my answer to "What is torsion in differential geometry intuitively?" If you want to find them all, go to the user page, search for "Anonymous" and look for the 11 anonymous users with that Gravatar.

    Without further ado, the results:

    As myself: Vote totals: 8, 2, 5, 12, 1, 5, 5, 7, 3 Mean: 5.3 Std. Dev: 3.4

    As anonymous: Vote totals: 9, 3, 5, 1, 4, 8, 6, 0, 5, 2, 4 Mean: 4.3 Std. Dev: 2.8

  2.  

    I'm not feeling awake enough to do a decent analysis. Clearly, any effect based on signing my name is much smaller than the variation within each sample. I strongly suspect that this data is not enough to reject the null hypothesis (that my name has no effect). Note that simply dropping my stupid answer here removes almost half the difference.

    It would be worth determining, based on this trial, how long an experiment we would have to run to get some truly meaningful data.

    A few other notes: I tried not to respond to comments on my answers as myself, as I could not do so as anonymous. (Although I see that I forgot and did so on occasion.) I made sure not to use too many links. On the other hand, I made no attempt to disguise my familiarity with the idiosyncracies of jsMath, which probably meant that I did better than a true new-comer.

    I answered a number of questions I ordinarily wouldn't have bothered with, because I wanted to gather the data quickly. But, because of my protocol, these questions were randomly distributed between myself and anonymous.

    The first few times that I answered anonymously, I didn't fully understand how MO cookies worked, and left some odd deleted answers that are visible to 10K+ users. Ben contacted me and I explained what was going on; I don't know if anyone else figured it out.

  3.  
    +1 David!
  4.  
    (I remember Anton mentioned to me he'd noticed you doing this, I don't remember how he knew.)
    • CommentAuthorBen Webster
    • CommentTimeMay 5th 2010 edited
     

    Because David's (recognizable) email address was visible to moderators. I thought he was having login troubles and offered to merge the accounts.

  5.  

    I'm happy you carried this out David. Thanks for sharing the results with us.

    Partly inspired by David's original comment and partly as a joke, I recently posted an answer under a pseudonym. (I think Anton noticed, but he probably figured out the joke and didn't say anything.) I wonder if anyone else has done this?

  6.  

    +1 David! I had noticed that David was doing this (I think I was looking for highly duplicated email addresses or something), but I was pretty sure he was just running this experiment, so I didn't email him about it. @fgdorais: I don't remember noticing anything like that. What was the post?

  7.  

    In case anybody is interested, here's a scatterplot of log(current reputation of post owner) against score(post):
    scatterplot

    Perhaps a more readable graph would be log(current rep) against average score of a post. I don't want to make that graph right now, but someone else is welcome to.

  8.  

    @Anton: Thanks for the graph! (The answer was this one. The joke works better if you're a Kubrick fan.)

  9.  

    @fgdorais: I did notice that was a clever username, but I didn't realize it was you.

  10.  

    Ok, here's my second attempt at getting some useful information on this issue out of the database dump.

    For readability, I've rounded everybody's reputation to the (logarithmically) nearest power of ∜2. Among all users with reputation close to 2^{n/4}, I've plotted the average score of a post, with error bars given by one standard deviation:
    graph

    Note that the numbers are especially weird for users with less than 10 reputation because it's hard to get less than 10 reputation unless you do something funny.

    This was also my attempt to learn how to tell python to make graphics. I can't say it was a great success. If you want to make one of these graphs yourself, you're welcome to use my terrible code:

    from scipy import stats
    from numpy import mean, std, log
    from lxml import etree
    from pyx import *
    
    fileobj = open("posts.xml","r")
    posts   = etree.parse(fileobj).getroot()
    fileobj.close()
    fileobj = open("users.xml","r")
    users   = etree.parse(fileobj).getroot()
    fileobj.close()
    
    userScores = {}
    for post in posts.findall("row"):
        owner = post.get("OwnerUserId")
        if owner==None:
            continue # ignore posts where the owner has been deleted
        if owner in userScores.keys():
            userScores[owner]+=[int(post.get("Score"))]
        else:
            userScores[owner]=[int(post.get("Score"))]
    
    repScores = {}
    base=2.0**0.25
    for uid in userScores.keys():
        user = users.xpath('//row[@Id="'+uid+'"]')[0]
        rep = int(user.get("Reputation"))
        effectiveRep = base**round(log(rep)/log(base))
        if effectiveRep in repScores.keys():
            repScores[effectiveRep]+=userScores[uid]
        else:
            repScores[effectiveRep]=userScores[uid]
    
    repScoreStddev=[]
    for rep, scores in repScores.items():
        repScoreStddev+=[(rep,mean(scores),std(scores))]
    
    g = graph.graphxy(width=22,x=graph.axis.logarithmic(min=1,max=22000),y=graph.axis.linear(min=-10,max=15))
    g.plot(graph.data.list(repScoreStddev, x=1, y=2, dy=3), [graph.style.symbol(graph.style.symbol.square), graph.style.errorbar()])
    g.writePDFfile("repVSscores")
    
  11.  

    I don't know how to read that graph, so if somebody could do a quick analysis, I'd appreciate it!

  12.  

    There's is a positive correlation between the votes an answer receives and the reputation of the user giving it. There are large standard deviations, but I would have got away with claiming I was seeing something real in my undergraduate physics courses.

    Given that we'd expect that high reputation users tend to give good answers (or would we? are they just incessant? :-), it seems hard to even guess much about causation in the other direction.

    • CommentAuthorRegenbogen
    • CommentTimeMay 11th 2010
     

    Which reminds me:

    For the one who has will be given more, and he will have more than enough. But the one who does not have, even what he has will be taken from him.

  13.  

    The large standard deviations are probably unavoidable. High reputation users are likely to give high-yield answers, but also very technical low-yield answers. Anyone have thoughts on how to account for this and other types of skewing effects? Maybe giving a bonus for accepted answers?