tea.mathoverflow.net - Discussion Feed (Experimental Results) Sun, 04 Nov 2018 13:54:00 -0800 http://mathoverflow.tqft.net/ Lussumo Vanilla 1.1.9 & Feed Publisher François G. Dorais comments on "Experimental Results" (5606) http://mathoverflow.tqft.net/discussion/385/experimental-results/?Focus=5606#Comment_5606 http://mathoverflow.tqft.net/discussion/385/experimental-results/?Focus=5606#Comment_5606 Tue, 18 May 2010 11:03:06 -0700 François G. Dorais The large standard deviations are probably unavoidable. High reputation users are likely to give high-yield answers, but also very technical low-yield answers. Anyone have thoughts on how to account for this and other types of skewing effects? Maybe giving a bonus for accepted answers?

]]>
Regenbogen comments on "Experimental Results" (5522) http://mathoverflow.tqft.net/discussion/385/experimental-results/?Focus=5522#Comment_5522 http://mathoverflow.tqft.net/discussion/385/experimental-results/?Focus=5522#Comment_5522 Tue, 11 May 2010 12:07:54 -0700 Regenbogen Which reminds me:

For the one who has will be given more, and he will have more than enough. But the one who does not have, even what he has will be taken from him.

]]>
Scott Morrison comments on "Experimental Results" (5521) http://mathoverflow.tqft.net/discussion/385/experimental-results/?Focus=5521#Comment_5521 http://mathoverflow.tqft.net/discussion/385/experimental-results/?Focus=5521#Comment_5521 Tue, 11 May 2010 12:02:59 -0700 Scott Morrison There's is a positive correlation between the votes an answer receives and the reputation of the user giving it. There are large standard deviations, but I would have got away with claiming I was seeing something real in my undergraduate physics courses.

Given that we'd expect that high reputation users tend to give good answers (or would we? are they just incessant? :-), it seems hard to even guess much about causation in the other direction.

]]>
Harry Gindi comments on "Experimental Results" (5489) http://mathoverflow.tqft.net/discussion/385/experimental-results/?Focus=5489#Comment_5489 http://mathoverflow.tqft.net/discussion/385/experimental-results/?Focus=5489#Comment_5489 Sun, 09 May 2010 23:13:18 -0700 Harry Gindi I don't know how to read that graph, so if somebody could do a quick analysis, I'd appreciate it!

]]>
Anton Geraschenko comments on "Experimental Results" (5488) http://mathoverflow.tqft.net/discussion/385/experimental-results/?Focus=5488#Comment_5488 http://mathoverflow.tqft.net/discussion/385/experimental-results/?Focus=5488#Comment_5488 Sun, 09 May 2010 22:58:33 -0700 Anton Geraschenko Ok, here's my second attempt at getting some useful information on this issue out of the database dump.

For readability, I've rounded everybody's reputation to the (logarithmically) nearest power of ∜2. Among all users with reputation close to 2^{n/4}, I've plotted the average score of a post, with error bars given by one standard deviation:
graph

Note that the numbers are especially weird for users with less than 10 reputation because it's hard to get less than 10 reputation unless you do something funny.

This was also my attempt to learn how to tell python to make graphics. I can't say it was a great success. If you want to make one of these graphs yourself, you're welcome to use my terrible code:

from scipy import stats
from numpy import mean, std, log
from lxml import etree
from pyx import *

fileobj = open("posts.xml","r")
posts   = etree.parse(fileobj).getroot()
fileobj.close()
fileobj = open("users.xml","r")
users   = etree.parse(fileobj).getroot()
fileobj.close()

userScores = {}
for post in posts.findall("row"):
    owner = post.get("OwnerUserId")
    if owner==None:
        continue # ignore posts where the owner has been deleted
    if owner in userScores.keys():
        userScores[owner]+=[int(post.get("Score"))]
    else:
        userScores[owner]=[int(post.get("Score"))]

repScores = {}
base=2.0**0.25
for uid in userScores.keys():
    user = users.xpath('//row[@Id="'+uid+'"]')[0]
    rep = int(user.get("Reputation"))
    effectiveRep = base**round(log(rep)/log(base))
    if effectiveRep in repScores.keys():
        repScores[effectiveRep]+=userScores[uid]
    else:
        repScores[effectiveRep]=userScores[uid]

repScoreStddev=[]
for rep, scores in repScores.items():
    repScoreStddev+=[(rep,mean(scores),std(scores))]

g = graph.graphxy(width=22,x=graph.axis.logarithmic(min=1,max=22000),y=graph.axis.linear(min=-10,max=15))
g.plot(graph.data.list(repScoreStddev, x=1, y=2, dy=3), [graph.style.symbol(graph.style.symbol.square), graph.style.errorbar()])
g.writePDFfile("repVSscores")
]]>
Anton Geraschenko comments on "Experimental Results" (5416) http://mathoverflow.tqft.net/discussion/385/experimental-results/?Focus=5416#Comment_5416 http://mathoverflow.tqft.net/discussion/385/experimental-results/?Focus=5416#Comment_5416 Thu, 06 May 2010 16:18:08 -0700 Anton Geraschenko @fgdorais: I did notice that was a clever username, but I didn't realize it was you.

]]>
François G. Dorais comments on "Experimental Results" (5412) http://mathoverflow.tqft.net/discussion/385/experimental-results/?Focus=5412#Comment_5412 http://mathoverflow.tqft.net/discussion/385/experimental-results/?Focus=5412#Comment_5412 Thu, 06 May 2010 15:58:08 -0700 François G. Dorais @Anton: Thanks for the graph! (The answer was this one. The joke works better if you're a Kubrick fan.)

]]>
Anton Geraschenko comments on "Experimental Results" (5410) http://mathoverflow.tqft.net/discussion/385/experimental-results/?Focus=5410#Comment_5410 http://mathoverflow.tqft.net/discussion/385/experimental-results/?Focus=5410#Comment_5410 Thu, 06 May 2010 15:45:08 -0700 Anton Geraschenko In case anybody is interested, here's a scatterplot of log(current reputation of post owner) against score(post):
scatterplot

Perhaps a more readable graph would be log(current rep) against average score of a post. I don't want to make that graph right now, but someone else is welcome to.

]]>
Anton Geraschenko comments on "Experimental Results" (5401) http://mathoverflow.tqft.net/discussion/385/experimental-results/?Focus=5401#Comment_5401 http://mathoverflow.tqft.net/discussion/385/experimental-results/?Focus=5401#Comment_5401 Thu, 06 May 2010 14:48:18 -0700 Anton Geraschenko +1 David! I had noticed that David was doing this (I think I was looking for highly duplicated email addresses or something), but I was pretty sure he was just running this experiment, so I didn't email him about it. @fgdorais: I don't remember noticing anything like that. What was the post?

]]>
François G. Dorais comments on "Experimental Results" (5381) http://mathoverflow.tqft.net/discussion/385/experimental-results/?Focus=5381#Comment_5381 http://mathoverflow.tqft.net/discussion/385/experimental-results/?Focus=5381#Comment_5381 Wed, 05 May 2010 21:18:19 -0700 François G. Dorais I'm happy you carried this out David. Thanks for sharing the results with us.

Partly inspired by David's original comment and partly as a joke, I recently posted an answer under a pseudonym. (I think Anton noticed, but he probably figured out the joke and didn't say anything.) I wonder if anyone else has done this?

]]>
Ben Webster comments on "Experimental Results" (5380) http://mathoverflow.tqft.net/discussion/385/experimental-results/?Focus=5380#Comment_5380 http://mathoverflow.tqft.net/discussion/385/experimental-results/?Focus=5380#Comment_5380 Wed, 05 May 2010 21:03:16 -0700 Ben Webster Because David's (recognizable) email address was visible to moderators. I thought he was having login troubles and offered to merge the accounts.

]]>
Scott Morrison comments on "Experimental Results" (5379) http://mathoverflow.tqft.net/discussion/385/experimental-results/?Focus=5379#Comment_5379 http://mathoverflow.tqft.net/discussion/385/experimental-results/?Focus=5379#Comment_5379 Wed, 05 May 2010 20:56:10 -0700 Scott Morrison Scott Morrison comments on "Experimental Results" (5378) http://mathoverflow.tqft.net/discussion/385/experimental-results/?Focus=5378#Comment_5378 http://mathoverflow.tqft.net/discussion/385/experimental-results/?Focus=5378#Comment_5378 Wed, 05 May 2010 20:54:06 -0700 Scott Morrison David Speyer comments on "Experimental Results" (5377) http://mathoverflow.tqft.net/discussion/385/experimental-results/?Focus=5377#Comment_5377 http://mathoverflow.tqft.net/discussion/385/experimental-results/?Focus=5377#Comment_5377 Wed, 05 May 2010 20:53:15 -0700 David Speyer I'm not feeling awake enough to do a decent analysis. Clearly, any effect based on signing my name is much smaller than the variation within each sample. I strongly suspect that this data is not enough to reject the null hypothesis (that my name has no effect). Note that simply dropping my stupid answer here removes almost half the difference.

It would be worth determining, based on this trial, how long an experiment we would have to run to get some truly meaningful data.

A few other notes: I tried not to respond to comments on my answers as myself, as I could not do so as anonymous. (Although I see that I forgot and did so on occasion.) I made sure not to use too many links. On the other hand, I made no attempt to disguise my familiarity with the idiosyncracies of jsMath, which probably meant that I did better than a true new-comer.

I answered a number of questions I ordinarily wouldn't have bothered with, because I wanted to gather the data quickly. But, because of my protocol, these questions were randomly distributed between myself and anonymous.

The first few times that I answered anonymously, I didn't fully understand how MO cookies worked, and left some odd deleted answers that are visible to 10K+ users. Ben contacted me and I explained what was going on; I don't know if anyone else figured it out.

]]>
David Speyer comments on "Experimental Results" (5376) http://mathoverflow.tqft.net/discussion/385/experimental-results/?Focus=5376#Comment_5376 http://mathoverflow.tqft.net/discussion/385/experimental-results/?Focus=5376#Comment_5376 Wed, 05 May 2010 20:40:04 -0700 David Speyer Pre-empetive disclaimer: Writing this up feels incredibly egotistical, even for me. I'm not sure anyone will care. But I am going to forge ahead and ignore that.

As proposed in this thread (near the top of the second page), I have been conducting a randomized experiment. The last 20 times that I have answered a question on MO, I wrote out my answer, then flipped a coin and either submitted the answer as myself or as anonymous. I made no edits after the coin flip, so that my writing could not be influenced by the flip. I had pre-decided to stop the experiment with 20 answers, regardless of the results.

The questions that I answered under my own name are the 9 answers from "Algebraic cycles of dimension 2 on the square of a generic abelian surface" to "Non-diagonalizable complex symmetric matrix", with the exception of "Choice vs. countable choice". Bring up my user page and sort the answers by "newest" to see them all. I kept "Choice vs. Counteable Choice" outside the experiment, because I felt that a vague answer like this would only be of interest if signed by a known mathematician; maybe I wimped out here.

The Gravatar algorithm always assigned me the same image as anonymous, so my anonymous posts are also easy to pick out. You can see the icon in question on my answer to "What is torsion in differential geometry intuitively?" If you want to find them all, go to the user page, search for "Anonymous" and look for the 11 anonymous users with that Gravatar.

Without further ado, the results:

As myself: Vote totals: 8, 2, 5, 12, 1, 5, 5, 7, 3 Mean: 5.3 Std. Dev: 3.4

As anonymous: Vote totals: 9, 3, 5, 1, 4, 8, 6, 0, 5, 2, 4 Mean: 4.3 Std. Dev: 2.8

]]>