This regards question 82952, a beginner question. An answer appeared, which had a brief sentence with mathematical content, followed by this:

But this is hardly a research level question, so to avoid annoyed remarks and rapid closure of your question please read the FAQ and probably prefer http://math.stackexchange.com for asking such questions.

In a comment under the answer, I wrote:

Marc, if you agree that this question is not appropriate for MO, then neither is giving an answer. The point is that asking such questions on MO should not be encouraged; please confine remarks like the second sentence to a comment instead. Thanks. -- Todd Trimble

The answerer wrote:

@Todd: I don't agree with the fact that answering a question is encouraging similar questions. The question was by a newcomer, who has gained no reputation by doing so and probably will now go elsewhere; I see no harm in the fact that he should do so without feeling the door was slammed on him. Also I think using comments as happens often (and here) for a many-party discussion is rather confusing, and not well supported by the software; I did not feel inclined to add to this confusion. -- Marc van Leeuwen

I thought surely I was espousing a widely held view on MO etiquette, but instead of embarking on a discussion with Marc in comment boxes, I thought any discussion should go here. I feel sure this type of thing has come up before and has been discussed on meta, but I find it hard to search for relevant entries. If anyone can point to such discussions, that would be great. Otherwise, please feel free to weigh in on this subject.

I agree that off-topic answers should not be answered (in general). However, to me it makes almost no differences whether answering happens via a comment or an answer. [To avoid a misunderstanding, I understand that Todd's 'second sentence' is the one he quoted so this is not a contradiction to what he said.]

Although to some extent though not fully I share Marc van Leeuwen's opinion that the 'encouraging'-aspect is not that important. Indeed, I think the direct encouraging effect is very small in particular after the existence of math.SE; I see no reason why somebody would continue to ask things on MO that would be well-received on math.SE after being made aware of it. [In the precise form the question also does not seem like a good math.SE question, but with some minor editing I think it would have been alright.] There is however, or at least so I believe, an indirect encouraging effect like so: the material is on MO, internet searches for related keywords out of related needs bring this up, if it then seems like the question is 'alright' (after all it had an answer, and perhaps it is not obvious what this closing is about) it might encourage similar questions by somebody else.

However, if now something like this gets answer, I think it should be properly answered; not some half-way, cryptic, possibly misleading answer in a comment. (This is also not about comment vs. answer, a proper answer might also fit in a comment box).

In fact in this case this happened. Somebody gave a misleading 'answer' in a comment. (It seems this was done in good faith, although initially due to the combination with the second IMO somewhat dissmisive comment, I had a different impression. But this got clarified and is tangential to the present discussion.) While this got also somehow clarified by a comment I still think in these special circumstances it was justified to give an answer to fully clarify the situation (and actually personally was glad when I noticed it).

In brief, my general opinion (I know I do not always follow this myself, but abstractly I think this is what should happen):

1. No answering at all of off-topic questions. Neither by comments nor answers. Also no hints.

2. If somebody for some reasons still answers, it needs to be done well. The worst thing of all are low-quality answers (half-way, cryptic, not quite correct as written,...) to low-quality questions.

In brief, my opinion for the specific question:

I think it could have been handled better in general (including me). But, well, things do not always go perfectly. Marc van Leeuwen tried to save a situation that got slightly tricky or could have gotten so, and I think suceeded.

My comment to Marc was more of the sort, "it's the principle of the thing". I can sort of see a pragmatic ground for acting as Marc did, and there's no reason to believe this had anything to do with "point-grubbing" (which is nevertheless a real phenomenon at MO, I believe); however, I'd like to put some general principles up for discussion.

(1) I do agree with quid that actual Answers on MO (items in answer boxes) ought to be answers: as definitive as one can make them. I think that for the case we're talking about -- the first sentence anyway -- it wasn't really an answer; it's more like the kind of thing one says in office hours to coax the student in a right direction. In a related vein, answers that amount to coy hints (I can think of at least one specific case) are similarly not the kind of Answers we should be giving.

(2) If one feels reluctant to give an answer in full because the question seems to the would-be answerer like homework, i.e., if a definitive answer would be out of place, that means the question is out of place (or "off-topic"). It should be closed, gently perhaps but firmly. Half-answers in the answer box, however well-meaning, ought to be avoided.

I confess to giving the occasional hint in a comment, even as I cast a vote for closure. I think I do that almost exclusively for things that look like homework, although I don't 100% guarantee that.

While an isolated instance of an answer (or half-answer) given to a beginner question might not by itself encourage hordes of other such questions being asked (either by OP or others), repeated instances of this type of thing can lead in this direction. Slippery slope, and all that. I think we can agree that MO gets quite a lot of beginner questions as it is, and we need to be careful in this regard.