How can we encourage/enforce the backing up of answers?

Question

There's already been a discussion about back-it-up rules and relevancy of answers a few times on this site.

• What should we do with answers that are not backed-up?
• Related Answers: Why your Pakistani answer won't always work for India
• Do we want references in our answers?
• Should we encourage writing from experience?

There may have been even more, that I missed, but these are the most obvious. Now, all of these questions and their answers pretty much seem to agree on one thing: having backed up answers is good. It makes the site a better place.

Yet, although we've been asking users to explain how or why their answer works, there still is very little written that comes with sources or describes the actual experiences had so that they can be compared to the situation in the question. There is often no disclosure of the culture these experiences are taking place in, even though we are encouraging the use of location tags.

Like said here:

The thing that I worry though, is that people may be answering questions or upvoting answers that demonstrate a lack of experience.

That worry was made in the very beginning of the site, and I am seeing it come true lately. I spent half an hour yesterday evening trying to convince one of the site's more active members to include some back up for the claims their answer was making, and in the end, the only thing added was a line like 'Speaking from experience here'. That doesn't give any information on how the experience related to the situation the asker was experiencing, and the cultural background behind that experience. I'm seeing it in other answers too, they're basically coming across as brainstorming on possible solutions, sometimes explained in detail as to how or why they should work, but those claims aren't backed up with anything, there's no proof they have worked in the past.

From here, it is suggested that:

If you do not know your solution works because you've used it yourself or you've seen it recommended by a reputable* source, do not supply the answer.

Now, that quote comes from an answer where it is argued that answers must back themselves up with either sources or experience, and it seems to suggest that enforcing such a thing is possible. All other answers to that question seem to be in agreement, although they are shorter versions.Same goes for the question here, it suggests that enforcing a back it up rule works and is possible. I personally would love to see this happening then, since it would mean easier vetting of answers that actually fit the question and a higher answer quality overall.

Now, looking at the first posts queue or searching this site for the newest answers (is:a, sort by newest), it becomes clear there's a whole lot of common sense answers and brainstorming going on, and that to the outside world it looks like this is okay here (hence the first posts almost never being backed up).

Now, I've been wondering how we could achieve having backed up answers, or discouraging the ones that aren't. And I could use some help with it, so:

How can we encourage/enforce the backing up of answers?

Answer 1

I'm just going to throw this out there but I don't really think there's much we should do to enforce such a rule. I think judging the quality of someones reasoning becomes too opinion based of a practice, and is going to lead to a lot of arguments over answers that should/should not have been deleted. For example, what if they provide a reason and it's "not good enough"? Some would vote to delete, others might think it's sufficient and want to leave open.

Encouragement/discouragement of this behavior would have to be reinforced with voting/commenting practices. If a short answer has a lot of upvotes, it's probably because the people upvoting it think it's good advice, which is a different kind of "back-it-up", but still a back-it-up nonetheless. Inversely, if someone is giving advice that no one else supports and fails to list good reasons for why they are suggesting such a thing, they're naturally going to be downvoted and learn they need to do work on the quality of their answers.

Answer 2

How can we encourage/enforce the backing up of answers?

It should not be enforced! Encourage by explaining what you are missing.

Quality of the answers should not be a concern for moderation, because the StackExchange tries to solve exactly this problem by democratic vote mechanism. While it is nice to have backups of claims, especially in interpersonal matters it is impossible to know your solution works! That´s why it is really valuable to have a great diversity of approaches and also be able to put them into perspective by popular vote and comments provided.

If the back-it up rule is strictly enforced, we will loose a lot of answers. A lot of the users here are not scientist working in this field and having research an citations ready. Also a lot of question are not specifically seeking scientific information but more of an outsiders view or even wisdom from others.

What you where are asking for, here, is a mandate to to moderate more aggressively to subjective opinion, instead of relying on the idea this whole community is build upon, just because you may not agree in some cases with that popular vote. This would render the unique mechanism of StackExchange Q/A moot.

I suggest to try an interpersonal approach if you think an answer can be improved. Instead of waving with site-rules, ask the author for clarification or context. Most will happily improve their posts when made aware of what exactly is unclear.

Answer 3

One possibility might be to edit in sources for some of the better, but un-cited, answers. This would take a lot of extra work, and some very diplomatic commenting, but if done well it could simultaneously create some good examples of "backed up" answers and reduce the number of highly-voted, unsupported answers, all without dinging folks for not having done this already.

Direct editing would work best with answers that advocate techniques that can be found described in scholarship or at least some reasonable online source. Obviously this would work much less well for personal experience, since we can't attribute our experiences to the original answerer. One approach for this kind of answer/evidence might be to include the relevant experience in a comment, with a note inviting the answerer to incorporate it into the answer if found useful.

I've taken both approaches on English.SE, where citation requirements are more regularly enforced, especially with new users. For example, if a new user gives an answer to a "single-word-request" that sounds reasonable but without any definition to back it up, I might find a relevant definition and edit it into the answer. I'd typically then leave a comment saying something like

Hi, NewUser, welcome to [this SE]! I think your answer is a good idea. I've added a dictionary definition with a link, as answers with some support are generally better received here. Please feel free to edit further (or substitute your own definition if that's not what you were thinking of). You can have a look at our help center for further tips about writing answers--I look forward to seeing more contributions from you!

Answers that are just a definition sometimes get a comment about adding a sentence or two explaining how this fits the request.

On other occasions, I've been researching an answer when someone posts essentially my response, but without some of the citations I've found (especially etymology questions). In that case I might only add a comment, rather than directly editing:

+1 I've also found the term used in several books from that decade, including [quotes and links]. Please feel free to incorporate any of this information into your answer if you find it helpful.

Obviously in the realm of IPS the types of support that could be added would be rather different and less straightforward, but I think for some answers it would be doable. Some answers advocate using a specific IPS strategy or technique that has Wikipedia entries or Psychology Today articles or even scholarly articles that could be linked (setting boundaries, active listening, negotiation, etc.), while in others someone other than the OP might be able to offer an example of having used the recommended strategy effectively.

Answer 4

First: I completely agree with Jess K.'s answer.

But if you, for some reason, decide that backing things up with anecdotes is a must for this site, then 1) it must be immediately clear to casual users what is backup and how much is enough/not enough, and 2) it must be consistently enforced.

Ad 1): I tried finding the "how much is enough backup" rules in the help section - and it didn't hit me in the face within the first minute of searching for it. From my experience as a software dev: Linking to lengthy meta discussions / 500 page manuals / referring to email communication / etc will not get users to use those resources, extract the rules "correctly", and apply them consistently. You need clear, practical examples of what backup looks like. E.g. please give a concrete example how you back up general "rules" of etiquette in Western culture - if it's not enough that 37 people upvoted it.

Ad 2): In the last days, I've repeatedly stumbled upon questions where an upvoted/accepted answer was "decorated" by mods with "Can you back this up?" while other (and, in my opinion: less relevant and less well-worded answers) were not. To an outsider, this is not distinguishable from mods trolling some users and leaving other users alone. My recommendation to make things look "fair" would be: if you feel you need to leave a back-it-up-comment on one answer, check the other answers on the same question if they need the same comment. Particularly, if you threaten to delete the answer or parts of it. Otherwise, you're sending a signal that the other answers are OK! It doesn't matter if you don't intend to send that signal, you're sending it.

I find it much more easy to learn the back it up rules from seeing one question where it has been consistently applied to all answers than from 10 questions where it has been haphazardly applied to some answers but not all. At least I more frequently compare answers on the same question with each other than answers on different questions.

You may say: "But it's a lot of work to look at all answers on a question!" Yeah, it is - that's why I find the voting system of SE so efficient. It doesn't rely mainly on moderators deciding what is good advice and what isn't, it relies on the Wisdom of the Crowd - so back to Jess K.'s answer.