tea.mathoverflow.net - Discussion Feed (For your statistical amusement: questions/comments/votes per day.)

Ben Webster comments on "For your statistical amusement: questions/comments/votes per day." (4022)

2010-03-23T14:53:32-07:00

Incidentally, how many of us would have believed before this project that it would average as low as 2 closed questions a day? Not I.

Mark Meckes comments on "For your statistical amusement: questions/comments/votes per day." (3981)

2010-03-22T10:56:26-07:00

More seriously, thanks for the help with the grep command, which will save me time if I think of some other graph I want to see.

Steve Huntsman comments on "For your statistical amusement: questions/comments/votes per day." (3978)

2010-03-22T09:33:47-07:00

http://mathoverflow.net/questions/11084/what-programming-languages-do-mathematicians-use/11135

Harry Gindi comments on "For your statistical amusement: questions/comments/votes per day." (3977)

2010-03-22T09:33:22-07:00

+1 for bible knowledge. I never read the bible much, but I think what you wrote there is correct.

;)

Anton Geraschenko comments on "For your statistical amusement: questions/comments/votes per day." (3974)

2010-03-22T09:05:46-07:00

... try to teach a man to fish, and he'll say, "Haha, you're a fishing rod!"

Mark Meckes comments on "For your statistical amusement: questions/comments/votes per day." (3968)

2010-03-22T04:40:50-07:00

I've apparently decided that the easiest way to see the graph I'm interested in is to write a program in the language MO Moderator (and I seem to be right).

Anton Geraschenko comments on "For your statistical amusement: questions/comments/votes per day." (3962)

2010-03-21T22:36:09-07:00

Yup, the bodies of posts are stored in attributes. That's not how it is in the actual dump of the database, but that's the way the SO public dump is done, and I thought it was worthwhile to copy their format pretty exactly since I assumed they'd thought it out carefully.

Mariano comments on "For your statistical amusement: questions/comments/votes per day." (3961)

2010-03-21T22:10:51-07:00

Wow. The bodies of posts are stored in attributes! Wow.

Anton Geraschenko comments on "For your statistical amusement: questions/comments/votes per day." (3960)

2010-03-21T21:35:23-07:00

By the way, Scott is getting these stats out of the public dump, and you can easily harvest this (and other) data yourself. For example, to get the number of questions closed on each day, you could simply run the command

grep -o 'ClosedDate="[^"]*"' posts.xml | sed -e 's/ClosedDate="//' -e 's/T.*$//' | sort | uniq -c

This finds questions which have a "ClosedDate" property, extracts the date, and then sorts and counts the number of times each date appears. You can then paste the result into a spreadsheet and have fun. You have to do slightly more to get the days where no questions were closed. Here's a graph of number of questions closed each day, along with a running average:

(This only counts questions that were closed but not deleted. We don't include deleted posts in the public dump.)

Mark Meckes comments on "For your statistical amusement: questions/comments/votes per day." (3959)

2010-03-21T20:04:18-07:00

It would also be easier to judge the answer to fpqc's question (did closures spike during finals) looking at the actual per-day graph. But it looks like probably not.

Ben Webster comments on "For your statistical amusement: questions/comments/votes per day." (3958)

2010-03-21T19:03:00-07:00

So wait, you deliberately confused us? Is the closed per day graph too bumpy? a running average would be nice to see, since while I can tell that things deflect downward, it's hard to say by how much. Maybe you could put up the data on many eyes, or as a google doc?

Harry Gindi comments on "For your statistical amusement: questions/comments/votes per day." (3954)

2010-03-21T12:00:25-07:00

@Scott: I saw the shape of that graph and didn't even bother to look at the axes.

Scott Morrison comments on "For your statistical amusement: questions/comments/votes per day." (3952)

2010-03-21T10:21:32-07:00

I presumed everyone would correctly infer that although I'd extracted "questions closed per day" I'd graphed the partial sums, per Anton's suggestion...

Harry Gindi comments on "For your statistical amusement: questions/comments/votes per day." (3942)

2010-03-20T21:14:54-07:00

@Steve: That's per day according to Scott.

Edit: Ah, I see, he was mistaken. That's most certainly not per day.

Steve Huntsman comments on "For your statistical amusement: questions/comments/votes per day." (3941)

2010-03-20T20:59:22-07:00

Fewer bad posts being made (not caught). The slope is decreasing.

Harry Gindi comments on "For your statistical amusement: questions/comments/votes per day." (3940)

2010-03-20T20:32:33-07:00

More bad posts being made, or more bad posts being caught do you think?

Scott Morrison comments on "For your statistical amusement: questions/comments/votes per day." (3934)

2010-03-20T17:35:26-07:00

Here's the data for questions closed per day

Anton Geraschenko comments on "For your statistical amusement: questions/comments/votes per day." (3933)

2010-03-20T17:34:20-07:00

I think it's easier to see what's happening if you plot the cumulative values (i.e. "total number of X up to a given day").

@fgdorais: comments don't show the strong initial peak because you need 50 reputation to comment on other peoples' posts. Votes also has a subdued peak because you need to get a small amount of reputation (15) before you can vote.

Tom Leinster comments on "For your statistical amusement: questions/comments/votes per day." (3930)

2010-03-20T16:22:23-07:00

Thanks, Scott! Interesting.

Harry Gindi comments on "For your statistical amusement: questions/comments/votes per day." (3926)

2010-03-20T14:39:51-07:00

Oh, could you add a graph for questions closed per day? I want to see if it really spiked during finals.

Mariano comments on "For your statistical amusement: questions/comments/votes per day." (3925)

2010-03-20T14:22:07-07:00

Could you adorn the graphs with trend lines? /me is lazy...

François G. Dorais comments on "For your statistical amusement: questions/comments/votes per day." (3924)

2010-03-20T14:14:32-07:00

Neat! Thanks Scott!

Any technical reason why comments don't show as strong initial peak in October as the other graphs do?

Scott Morrison comments on "For your statistical amusement: questions/comments/votes per day." (3923)

2010-03-20T13:16:02-07:00

Questions per day

All posts, per day

Votes per day

Comments per day

New users per day