tea.mathoverflow.net - Discussion Feed (For your statistical amusement: questions/comments/votes per day.)

Ben Webster comments on "For your statistical amusement: questions/comments/votes per day." (4022)

Ben Webster — Tue, 23 Mar 2010 14:53:32 -0700

Incidentally, how many of us would have believed before this project that it would average as low as 2 closed questions a day? Not I.

Mark Meckes comments on "For your statistical amusement: questions/comments/votes per day." (3981)

Mark Meckes — Mon, 22 Mar 2010 10:56:26 -0700

More seriously, thanks for the help with the grep command, which will save me time if I think of some other graph I want to see.

Steve Huntsman comments on "For your statistical amusement: questions/comments/votes per day." (3978)

Steve Huntsman — Mon, 22 Mar 2010 09:33:47 -0700

http://mathoverflow.net/questions/11084/what-programming-languages-do-mathematicians-use/11135

Harry Gindi comments on "For your statistical amusement: questions/comments/votes per day." (3977)

Harry Gindi — Mon, 22 Mar 2010 09:33:22 -0700

+1 for bible knowledge. I never read the bible much, but I think what you wrote there is correct.

;)

Anton Geraschenko comments on "For your statistical amusement: questions/comments/votes per day." (3974)

Anton Geraschenko — Mon, 22 Mar 2010 09:05:46 -0700

... try to teach a man to fish, and he'll say, "Haha, you're a fishing rod!"

Mark Meckes comments on "For your statistical amusement: questions/comments/votes per day." (3968)

Mark Meckes — Mon, 22 Mar 2010 04:40:50 -0700

I've apparently decided that the easiest way to see the graph I'm interested in is to write a program in the language MO Moderator (and I seem to be right).

Anton Geraschenko comments on "For your statistical amusement: questions/comments/votes per day." (3962)

Anton Geraschenko — Sun, 21 Mar 2010 22:36:09 -0700

Yup, the bodies of posts are stored in attributes. That's not how it is in the actual dump of the database, but that's the way the SO public dump is done, and I thought it was worthwhile to copy their format pretty exactly since I assumed they'd thought it out carefully.

Mariano comments on "For your statistical amusement: questions/comments/votes per day." (3961)

Mariano — Sun, 21 Mar 2010 22:10:51 -0700

Wow. The bodies of posts are stored in attributes! Wow.

Anton Geraschenko comments on "For your statistical amusement: questions/comments/votes per day." (3960)

Anton Geraschenko — Sun, 21 Mar 2010 21:35:23 -0700

By the way, Scott is getting these stats out of the public dump, and you can easily harvest this (and other) data yourself. For example, to get the number of questions closed on each day, you could simply run the command

grep -o 'ClosedDate="[^"]*"' posts.xml | sed -e 's/ClosedDate="//' -e 's/T.*$//' | sort | uniq -c

This finds questions which have a "ClosedDate" property, extracts the date, and then sorts and counts the number of times each date appears. You can then paste the result into a spreadsheet and have fun. You have to do slightly more to get the days where no questions were closed. Here's a graph of number of questions closed each day, along with a running average:

(This only counts questions that were closed but not deleted. We don't include deleted posts in the public dump.)

Mark Meckes comments on "For your statistical amusement: questions/comments/votes per day." (3959)

Mark Meckes — Sun, 21 Mar 2010 20:04:18 -0700

It would also be easier to judge the answer to fpqc's question (did closures spike during finals) looking at the actual per-day graph. But it looks like probably not.

Ben Webster comments on "For your statistical amusement: questions/comments/votes per day." (3958)

Ben Webster — Sun, 21 Mar 2010 19:03:00 -0700

So wait, you deliberately confused us? Is the closed per day graph too bumpy? a running average would be nice to see, since while I can tell that things deflect downward, it's hard to say by how much. Maybe you could put up the data on many eyes, or as a google doc?

Harry Gindi comments on "For your statistical amusement: questions/comments/votes per day." (3954)

Harry Gindi — Sun, 21 Mar 2010 12:00:25 -0700

@Scott: I saw the shape of that graph and didn't even bother to look at the axes.

Scott Morrison comments on "For your statistical amusement: questions/comments/votes per day." (3952)

Scott Morrison — Sun, 21 Mar 2010 10:21:32 -0700

I presumed everyone would correctly infer that although I'd extracted "questions closed per day" I'd graphed the partial sums, per Anton's suggestion...

Harry Gindi comments on "For your statistical amusement: questions/comments/votes per day." (3942)

Harry Gindi — Sat, 20 Mar 2010 21:14:54 -0700

@Steve: That's per day according to Scott.

Edit: Ah, I see, he was mistaken. That's most certainly not per day.

Steve Huntsman comments on "For your statistical amusement: questions/comments/votes per day." (3941)

Steve Huntsman — Sat, 20 Mar 2010 20:59:22 -0700

Fewer bad posts being made (not caught). The slope is decreasing.

Harry Gindi comments on "For your statistical amusement: questions/comments/votes per day." (3940)

Harry Gindi — Sat, 20 Mar 2010 20:32:33 -0700

More bad posts being made, or more bad posts being caught do you think?

Scott Morrison comments on "For your statistical amusement: questions/comments/votes per day." (3934)

Scott Morrison — Sat, 20 Mar 2010 17:35:26 -0700

Here's the data for questions closed per day

Anton Geraschenko comments on "For your statistical amusement: questions/comments/votes per day." (3933)

Anton Geraschenko — Sat, 20 Mar 2010 17:34:20 -0700

I think it's easier to see what's happening if you plot the cumulative values (i.e. "total number of X up to a given day").

@fgdorais: comments don't show the strong initial peak because you need 50 reputation to comment on other peoples' posts. Votes also has a subdued peak because you need to get a small amount of reputation (15) before you can vote.

Tom Leinster comments on "For your statistical amusement: questions/comments/votes per day." (3930)

Tom Leinster — Sat, 20 Mar 2010 16:22:23 -0700

Thanks, Scott! Interesting.

Harry Gindi comments on "For your statistical amusement: questions/comments/votes per day." (3926)

Harry Gindi — Sat, 20 Mar 2010 14:39:51 -0700

Oh, could you add a graph for questions closed per day? I want to see if it really spiked during finals.

Mariano comments on "For your statistical amusement: questions/comments/votes per day." (3925)

Mariano — Sat, 20 Mar 2010 14:22:07 -0700

Could you adorn the graphs with trend lines? /me is lazy...

François G. Dorais comments on "For your statistical amusement: questions/comments/votes per day." (3924)

François G. Dorais — Sat, 20 Mar 2010 14:14:32 -0700

Neat! Thanks Scott!

Any technical reason why comments don't show as strong initial peak in October as the other graphs do?

Scott Morrison comments on "For your statistical amusement: questions/comments/votes per day." (3923)

Scott Morrison — Sat, 20 Mar 2010 13:16:02 -0700

Questions per day

All posts, per day

Votes per day

Comments per day

New users per day