tea.mathoverflow.net - Discussion Feed (Help ensure the public database dump is sane)

Andrew Stacey comments on "Help ensure the public database dump is sane" (3556)

Andrew Stacey — Mon, 01 Mar 2010 05:15:35 -0800

If these database dumps are not sanitised, then I suspect that there would be serious privacy issues about releasing even encrypted versions, since what is nice and secure today may not be quite so secure after watching tomorrow's episode of Spooks (nice to see that they use Macs in MI5).

But I don't see why, if Anton and Scott and the others do go completely doolally, an unsanitised database dump should be needed. The quickest way to rescue MO in the event of such a crash would be to start a wholly new site and then turn the sanitised database dumps into a read-only archive. If an official new site isn't up and running within 24hrs of Scott and Anton's disappearance, it's almost certain that someone will set up an unofficial scion of MO.

I can also understand the need to have database dumps in case Fog Creek goes doolally, but there even more so it's pointless to have anything above the sanitised data since a new system would probably have stored the data in a completely different manner.

The only situation in which I can envisage needing full database dumps is if Fog Creek gets hit by an EMP temporarily wiping their systems and needing some restore from backup. But unless the EMP also took out Scott and Anton (after all, we've no proof that they aren't robots) they could keep copies of the unsanitised data privately.

Harald Hanche-Olsen comments on "Help ensure the public database dump is sane" (3553)

Harald Hanche-Olsen — Mon, 01 Mar 2010 03:55:42 -0800

@Sonia: That is not a bad idea, really. The database dumps could be encrypted and the encrypted copies could even be left out in the open, and the encryption key shared the way you suggest. (Google tells me there is free secret sharing software out there, but I don't know how usable it is.)

Sonia Balagopalan comments on "Help ensure the public database dump is sane" (3550)

Sonia Balagopalan — Mon, 01 Mar 2010 00:47:45 -0800

You could use a secret sharing scheme.

Anton Geraschenko comments on "Help ensure the public database dump is sane" (3548)

Anton Geraschenko — Sun, 28 Feb 2010 23:24:19 -0800

But what if that person goes crazy?

Scott Morrison comments on "Help ensure the public database dump is sane" (3545)

Scott Morrison — Sun, 28 Feb 2010 17:43:54 -0800

Perhaps we should consider finding someone responsible and respectable who would agree to keeping a copy of every unsanitized database dump we produce, as insurance against the moderators going crazy. :-) Of course, it should be clear that that person should be embarrassed if they released those dumps for any reason other than us losing our marbles and being unwilling to continue maintaining the site.

Anton Geraschenko comments on "Help ensure the public database dump is sane" (3532)

Anton Geraschenko — Sun, 28 Feb 2010 01:30:17 -0800

I've just finished putting together a sanitized dump of the MO database. I'm looking for a handful of people to have a look at it before I release it to make sure I haven't accidentally included anything that I shouldn't have. If you or someone you know is interested, please send me an email. Since I'm pretty sure the SO public dumps are well thought-out, I've modeled the MO dump after them very closely, so it helps if you've looked at those. See this meta.SO post for a description of what data is included.

The dump should include any information that you could in principle get by scraping the site, but it doesn't. For example, it currently doesn't include revision histories. If you have suggestions for things the MO dump should contain that aren't in the SO dump, please post them here.