tea.mathoverflow.net - Discussion Feed (Trackbacks to arXiv?) Sun, 04 Nov 2018 13:47:14 -0800 http://mathoverflow.tqft.net/ Lussumo Vanilla 1.1.9 & Feed Publisher Scott Morrison comments on "Trackbacks to arXiv?" (5200) http://mathoverflow.tqft.net/discussion/278/trackbacks-to-arxiv/?Focus=5200#Comment_5200 http://mathoverflow.tqft.net/discussion/278/trackbacks-to-arxiv/?Focus=5200#Comment_5200 Wed, 28 Apr 2010 21:05:11 -0700 Scott Morrison +1 all round.

]]>
Anton Geraschenko comments on "Trackbacks to arXiv?" (5176) http://mathoverflow.tqft.net/discussion/278/trackbacks-to-arxiv/?Focus=5176#Comment_5176 http://mathoverflow.tqft.net/discussion/278/trackbacks-to-arxiv/?Focus=5176#Comment_5176 Wed, 28 Apr 2010 11:35:00 -0700 Anton Geraschenko
Edit: I've emailed the new trackbacks to the arXiv admin. Hopefully they'll be up soon.
More Edit: They're up. That was quick. ]]>
jc comments on "Trackbacks to arXiv?" (5171) http://mathoverflow.tqft.net/discussion/278/trackbacks-to-arxiv/?Focus=5171#Comment_5171 http://mathoverflow.tqft.net/discussion/278/trackbacks-to-arxiv/?Focus=5171#Comment_5171 Wed, 28 Apr 2010 10:24:07 -0700 jc I posted arxiv links in comments a few times and they don't seem to have been picked up by the scraper so far:

http://mathoverflow.net/questions/21014/isotropic-deformation-retract-of-weinstein-manifolds http://mathoverflow.net/questions/21236/weight-filtration-for-smooth-analytic-manifolds

]]>
Anton Geraschenko comments on "Trackbacks to arXiv?" (3946) http://mathoverflow.tqft.net/discussion/278/trackbacks-to-arxiv/?Focus=3946#Comment_3946 http://mathoverflow.tqft.net/discussion/278/trackbacks-to-arxiv/?Focus=3946#Comment_3946 Sun, 21 Mar 2010 00:19:39 -0700 Anton Geraschenko "I count 2 mathematics trackbacks out of the last 100 that aren't from MO." That's a little unfair since we submitted over 1000 trackbacks all at once from scraping the database. Assuming people keep posting arXiv links at about the same rate they've been doing, we should see between 5 and 10 new trackbacks per day.

]]>
Anton Geraschenko comments on "Trackbacks to arXiv?" (3945) http://mathoverflow.tqft.net/discussion/278/trackbacks-to-arxiv/?Focus=3945#Comment_3945 http://mathoverflow.tqft.net/discussion/278/trackbacks-to-arxiv/?Focus=3945#Comment_3945 Sun, 21 Mar 2010 00:10:59 -0700 Anton Geraschenko @Andrew: Actually, nothing in a post ever starts a new line or ends a line since posted content is always within <p> tags. The reason for including the end-of-line character at the end is really just for debugging purposes.

]]>
Ben Webster comments on "Trackbacks to arXiv?" (3943) http://mathoverflow.tqft.net/discussion/278/trackbacks-to-arxiv/?Focus=3943#Comment_3943 http://mathoverflow.tqft.net/discussion/278/trackbacks-to-arxiv/?Focus=3943#Comment_3943 Sat, 20 Mar 2010 23:35:40 -0700 Ben Webster @figueroa- 28 out of 40 actually understates their impact. I count 2 mathematics trackbacks out of the last 100 that aren't from MO (one from Terry Tao, one from NCC); all the other ones are for physics. I think we're going to dominate math trackbacks for a very long time.

]]>
Scott Morrison comments on "Trackbacks to arXiv?" (3936) http://mathoverflow.tqft.net/discussion/278/trackbacks-to-arxiv/?Focus=3936#Comment_3936 http://mathoverflow.tqft.net/discussion/278/trackbacks-to-arxiv/?Focus=3936#Comment_3936 Sat, 20 Mar 2010 17:40:28 -0700 Scott Morrison @Andrew -- we also catch old-style identifiers via different regexes. Anton was just showing how we catch new ones. You're right about starting lines, and our testing wouldn't have picked up those. Note that we handle "fake new-style identifiers", introduced on the Front, by querying the Front for the correct old-style identifier.

]]>
Andrew Stacey comments on "Trackbacks to arXiv?" (3935) http://mathoverflow.tqft.net/discussion/278/trackbacks-to-arxiv/?Focus=3935#Comment_3935 http://mathoverflow.tqft.net/discussion/278/trackbacks-to-arxiv/?Focus=3935#Comment_3935 Sat, 20 Mar 2010 17:36:35 -0700 Andrew Stacey You miss out on old-style identifiers, and you miss out on references that start a line.

]]>
Anton Geraschenko comments on "Trackbacks to arXiv?" (3932) http://mathoverflow.tqft.net/discussion/278/trackbacks-to-arxiv/?Focus=3932#Comment_3932 http://mathoverflow.tqft.net/discussion/278/trackbacks-to-arxiv/?Focus=3932#Comment_3932 Sat, 20 Mar 2010 17:24:21 -0700 Anton Geraschenko

I have one question: will something like '1001.1234' be picked up without the 'arXiv:' prefix?

Yes. Anything matching the following regex will be picked up: [^\w.]\d\d[01]\d.\d{4}(?:[v\W]|$) . Based on five months of data, this doesn't seem to have picked up any false positives. If we do get false positives, we'll refine our search. Feel free to use this thread to imagine situations where somebody might reasonably match this regex without meaning to refer to an arXiv article.

]]>
Scott Morrison comments on "Trackbacks to arXiv?" (3931) http://mathoverflow.tqft.net/discussion/278/trackbacks-to-arxiv/?Focus=3931#Comment_3931 http://mathoverflow.tqft.net/discussion/278/trackbacks-to-arxiv/?Focus=3931#Comment_3931 Sat, 20 Mar 2010 17:22:11 -0700 Scott Morrison @figueroa, yes, 1001.1234 will be picked up without the arXiv: prefix, but subject to some other complicated constraints on the adjacent characters (e.g., there has to something other than an alphanumeric character other than v immediately afterwards...).

And yes, at least for users with javascript enabled (almost everyone), we'll see in Anton's google analytics statistics how many people arrive via arxiv trackbacks. Remind us again in a month if we haven't reported on this --- I'm really interested to find out how effective this will be.

]]>
José Figueroa comments on "Trackbacks to arXiv?" (3929) http://mathoverflow.tqft.net/discussion/278/trackbacks-to-arxiv/?Focus=3929#Comment_3929 http://mathoverflow.tqft.net/discussion/278/trackbacks-to-arxiv/?Focus=3929#Comment_3929 Sat, 20 Mar 2010 14:59:09 -0700 José Figueroa
I agree. 28 of the 40 most recent trackbacks point to MO. This ought to increase MOs exposure and potentially increase participation. Is there a way to measure this? i.e., a way to see whether someone reached MO via a link from the arXiv?

I have one question: will something like '1001.1234' be picked up without the 'arXiv:' prefix? ]]>
Harry Gindi comments on "Trackbacks to arXiv?" (3928) http://mathoverflow.tqft.net/discussion/278/trackbacks-to-arxiv/?Focus=3928#Comment_3928 http://mathoverflow.tqft.net/discussion/278/trackbacks-to-arxiv/?Focus=3928#Comment_3928 Sat, 20 Mar 2010 14:57:31 -0700 Harry Gindi Sure, I was responding to Scott's call for more links of the form [link text](link)

]]>
Anton Geraschenko comments on "Trackbacks to arXiv?" (3927) http://mathoverflow.tqft.net/discussion/278/trackbacks-to-arxiv/?Focus=3927#Comment_3927 http://mathoverflow.tqft.net/discussion/278/trackbacks-to-arxiv/?Focus=3927#Comment_3927 Sat, 20 Mar 2010 14:53:28 -0700 Anton Geraschenko arXiv links in comments are picked up just fine. Even if you refer to an article but don't link to the arXiv, the reference is picked up. You can write stuff like "arxiv:math/0504123" or "arxiv: 1001.1234v3". The main advantage of including the actual URL in a comment (or a link in a post) is that it's easier for people to get to the article ... they don't have to type "arxiv.org/abs/XYZW".

]]>
Harry Gindi comments on "Trackbacks to arXiv?" (3922) http://mathoverflow.tqft.net/discussion/278/trackbacks-to-arxiv/?Focus=3922#Comment_3922 http://mathoverflow.tqft.net/discussion/278/trackbacks-to-arxiv/?Focus=3922#Comment_3922 Sat, 20 Mar 2010 13:09:57 -0700 Harry Gindi This is a great reason to make markdown-style links work in comments...

]]>
Scott Morrison comments on "Trackbacks to arXiv?" (3921) http://mathoverflow.tqft.net/discussion/278/trackbacks-to-arxiv/?Focus=3921#Comment_3921 http://mathoverflow.tqft.net/discussion/278/trackbacks-to-arxiv/?Focus=3921#Comment_3921 Sat, 20 Mar 2010 12:31:54 -0700 Scott Morrison After a few delays, and some back and forth with the arxiv folks to resolve some issues, we now have trackbacks working!

We've submitted all the "historical" trackbacks from earlier posts, and these have been incorporated into the appropriate pages. There were ~1000 of these. We have a cron job running on the same server that hosts meta, which periodically looks for new arxiv links in recently edited questions, and submits those. (That is, we're doing this entirely without StackExchange support.)

We locate references to arxiv papers very aggressively -- it's not even necessary to actually have a link. We've implemented a bunch of tricks to locate arxiv identifiers, and checked the output carefully for the historical links, but won't be reviewing things routinely in future. If you happen to see an incorrect trackback, please let us know, as we'd like to play nice for the arxiv folks (who've been extremely helpful and accommodating!). If you suspect that something should have generated a trackback, make sure it wasn't the result of a recent edit (there's at most a 24 hour delay before you'll see things on the arxiv, it seems) and then let us know here or elsewhere.

Even though we try to find other arxiv references, I'd strongly encourage everyone to use proper links to the arxiv, with [link text](link url) style links. This makes it easier for readers than just bare arxiv identifiers, and means that the links will be detected by the less error prone parts of our program!

Go out there and discuss some arxiv papers! You can also use http://arxiv.org/tb/recent to see recent trackbacks on the arxiv -- I expect that mathoverflow will dominate this listing for now.

]]>
José Figueroa comments on "Trackbacks to arXiv?" (3853) http://mathoverflow.tqft.net/discussion/278/trackbacks-to-arxiv/?Focus=3853#Comment_3853 http://mathoverflow.tqft.net/discussion/278/trackbacks-to-arxiv/?Focus=3853#Comment_3853 Sun, 14 Mar 2010 21:10:02 -0700 José Figueroa Scott Morrison comments on "Trackbacks to arXiv?" (3817) http://mathoverflow.tqft.net/discussion/278/trackbacks-to-arxiv/?Focus=3817#Comment_3817 http://mathoverflow.tqft.net/discussion/278/trackbacks-to-arxiv/?Focus=3817#Comment_3817 Fri, 12 Mar 2010 18:24:50 -0800 Scott Morrison Anton and I made a lot of behind the scenes progress today. We've talked to the arxiv administrators, and you can already see a sample trackback at http://arxiv.org/tb/math/0307263. Quite likely, we'll submit all the historical trackbacks for existing questions soon, and then process new trackbacks as the links are made.

]]>
Ryan Budney comments on "Trackbacks to arXiv?" (3812) http://mathoverflow.tqft.net/discussion/278/trackbacks-to-arxiv/?Focus=3812#Comment_3812 http://mathoverflow.tqft.net/discussion/278/trackbacks-to-arxiv/?Focus=3812#Comment_3812 Fri, 12 Mar 2010 10:27:27 -0800 Ryan Budney Scott Morrison comments on "Trackbacks to arXiv?" (3806) http://mathoverflow.tqft.net/discussion/278/trackbacks-to-arxiv/?Focus=3806#Comment_3806 http://mathoverflow.tqft.net/discussion/278/trackbacks-to-arxiv/?Focus=3806#Comment_3806 Thu, 11 Mar 2010 12:53:41 -0800 Scott Morrison I have a little program (written in mathematica, because my perl/bash/python-fu is weak) that can either take an RSS feed from MathOverflow, or the database dump, and generate trackbacks. I only ran the final step of pinging the arxiv on one page, and got back a status 200 OK, but didn't see a link appear. Let's wait until we see if the arxiv is agreeable, and then someone can write a proper script.

(Interestingly, in the database dump I already find ~650 links to the arxiv.)

]]>
Scott Morrison comments on "Trackbacks to arXiv?" (3800) http://mathoverflow.tqft.net/discussion/278/trackbacks-to-arxiv/?Focus=3800#Comment_3800 http://mathoverflow.tqft.net/discussion/278/trackbacks-to-arxiv/?Focus=3800#Comment_3800 Thu, 11 Mar 2010 10:32:26 -0800 Scott Morrison This looks like it might be really easy. Sending a trackback (i.e. notifying the arxiv of a link) is just a matter of making a http POST request to a certain URL, filling out certain fields. See the spec. The arxiv has a semi-automated trackback spam filter in place, but I'll go now and ask what we'd need to do to make sure we can get through this.

I think the technically easiest solution is to run a cron job that watches for links to the arxiv from mathoverflow, then constructs the appropriate request to the arxiv. If someone wants to get thinking about a little command-line program that finds recent links from mathoverflow to the arxiv, please go ahead and report back here. For now, just something that outputs pairs (mathoverflow URL, arxiv URL) would be great.

]]>
Scott Morrison comments on "Trackbacks to arXiv?" (3799) http://mathoverflow.tqft.net/discussion/278/trackbacks-to-arxiv/?Focus=3799#Comment_3799 http://mathoverflow.tqft.net/discussion/278/trackbacks-to-arxiv/?Focus=3799#Comment_3799 Thu, 11 Mar 2010 10:25:44 -0800 Scott Morrison Thanks figueroa! I hope we get somewhere on this. At some point I understood the trackback system, but not anymore. I think we should seriously think about the possibility of implementing this ourselves (even if it's a script run externally that provides the glue), as this could be a really great feature. I think having trackbacks would make questions about points raised in specific papers even more useful.

]]>
José Figueroa comments on "Trackbacks to arXiv?" (3777) http://mathoverflow.tqft.net/discussion/278/trackbacks-to-arxiv/?Focus=3777#Comment_3777 http://mathoverflow.tqft.net/discussion/278/trackbacks-to-arxiv/?Focus=3777#Comment_3777 Thu, 11 Mar 2010 04:18:42 -0800 José Figueroa Tom Leinster comments on "Trackbacks to arXiv?" (3775) http://mathoverflow.tqft.net/discussion/278/trackbacks-to-arxiv/?Focus=3775#Comment_3775 http://mathoverflow.tqft.net/discussion/278/trackbacks-to-arxiv/?Focus=3775#Comment_3775 Thu, 11 Mar 2010 03:47:26 -0800 Tom Leinster On Scott's second point, it might be worth mentioning it to Greg Kuperberg - he'd surely know who to talk to at the arXiv.

]]>
Scott Morrison comments on "Trackbacks to arXiv?" (3770) http://mathoverflow.tqft.net/discussion/278/trackbacks-to-arxiv/?Focus=3770#Comment_3770 http://mathoverflow.tqft.net/discussion/278/trackbacks-to-arxiv/?Focus=3770#Comment_3770 Wed, 10 Mar 2010 21:02:35 -0800 Scott Morrison This is an excellent idea, but it would require cooperation from multiple people. First, we'd need StackExchange to use the trackback protocol, and this would require a feature request at meta.stackexchange.com (please link here if you make it!). Second, the trackbacks on the arxiv end are run on "whitelist", not "blacklist" basis, and we would have to talk to the appropriate people there. I think I know who this is, and I think it is likely they would be agreeable.

]]>
José Figueroa comments on "Trackbacks to arXiv?" (3765) http://mathoverflow.tqft.net/discussion/278/trackbacks-to-arxiv/?Focus=3765#Comment_3765 http://mathoverflow.tqft.net/discussion/278/trackbacks-to-arxiv/?Focus=3765#Comment_3765 Wed, 10 Mar 2010 18:58:37 -0800 José Figueroa