There's already been a discussion about back-it-up rules and relevancy of answers a few times on this site.

There may have been even more, that I missed, but these are the most obvious. Now, all of these questions and their answers pretty much seem to agree on one thing: having backed up answers is good. It makes the site a better place.

Yet, although we've been asking users to explain how or why their answer works, there still is very little written that comes with sources or describes the actual experiences had so that they can be compared to the situation in the question. There is often no disclosure of the culture these experiences are taking place in, even though we are encouraging the use of location tags.

Like said here:

The thing that I worry though, is that people may be answering questions or upvoting answers that demonstrate a lack of experience.

That worry was made in the very beginning of the site, and I am seeing it come true lately. I spent half an hour yesterday evening trying to convince one of the site's more active members to include some back up for the claims their answer was making, and in the end, the only thing added was a line like 'Speaking from experience here'. That doesn't give any information on how the experience related to the situation the asker was experiencing, and the cultural background behind that experience. I'm seeing it in other answers too, they're basically coming across as brainstorming on possible solutions, sometimes explained in detail as to how or why they should work, but those claims aren't backed up with anything, there's no proof they have worked in the past.

From here, it is suggested that:

If you do not know your solution works because you've used it yourself or you've seen it recommended by a reputable* source, do not supply the answer.

Now, that quote comes from an answer where it is argued that answers must back themselves up with either sources or experience, and it seems to suggest that enforcing such a thing is possible. All other answers to that question seem to be in agreement, although they are shorter versions.Same goes for the question here, it suggests that enforcing a back it up rule works and is possible. I personally would love to see this happening then, since it would mean easier vetting of answers that actually fit the question and a higher answer quality overall.

Now, looking at the first posts queue or searching this site for the newest answers (is:a, sort by newest), it becomes clear there's a whole lot of common sense answers and brainstorming going on, and that to the outside world it looks like this is okay here (hence the first posts almost never being backed up).

Now, I've been wondering how we could achieve having backed up answers, or discouraging the ones that aren't. And I could use some help with it, so:

How can we encourage/enforce the backing up of answers?

  • My meta question along the same lines as yours got such a weak/ lukewarm response (2 downvotes, 1 answer scoring -4 and just 3 comments) in August 2017: May I remind members that we had agreed to include references/citations in our answers whenever possible? – English Student Mar 21 '18 at 20:50
  • 2
    What qualifies as backing up a response when sharing based on personal experience? Do we expect users to share their own experience as a large part of an answer? When can this rule be ignored? because "common sense"? – spiral succulent Mar 22 '18 at 0:18
  • 3
    @spiralsucculent here is already a pretty good meta on how to write experience based answers interpersonal.meta.stackexchange.com/a/183/1599. Basically, it needs enough info so their experience can be compared to that in the question, if there were differences pointing those out as well is good. I'd love to see the rule not be ignored, because like I said underneath Jesse's answer, experiences or references are much easier compared than reasoning, so if we do this we can eradicate a huge part of opinion based answers that we can't validate if they've ever worked- see the quote in the q – Tinkeringbell Mar 22 '18 at 6:11

I'm just going to throw this out there but I don't really think there's much we should do to enforce such a rule. I think judging the quality of someones reasoning becomes too opinion based of a practice, and is going to lead to a lot of arguments over answers that should/should not have been deleted. For example, what if they provide a reason and it's "not good enough"? Some would vote to delete, others might think it's sufficient and want to leave open.

Encouragement/discouragement of this behavior would have to be reinforced with voting/commenting practices. If a short answer has a lot of upvotes, it's probably because the people upvoting it think it's good advice, which is a different kind of "back-it-up", but still a back-it-up nonetheless. Inversely, if someone is giving advice that no one else supports and fails to list good reasons for why they are suggesting such a thing, they're naturally going to be downvoted and learn they need to do work on the quality of their answers.

  • 1
    I was afraid this might happen :P Which is why I linked the quote about 'people may be answering questions or upvoting answers that demonstrate a lack of experience.' I think I've seen that happening quite a few times now, how would letting go of the back it up rule (as it was originally, so references or experiences, not votes) prevent this? If you think votes can be used to encourage/discourage certain behavior, that might be a good way to go, but could you then elaborate on how to encourage a 'good' voting culture? – Tinkeringbell Mar 21 '18 at 20:59
  • 2
    Also, judging the quality of someones reasoning is exactly what a back it up rule would take away, so it will remove a large opinion based aspect of this site, instead of add one. Experience or references are much easier compared than reasoning/opinions.. – Tinkeringbell Mar 21 '18 at 21:04
  • 9
    @Tinkeringbell judging the quality of someones reasoning is exactly what a back it up rule would take away I don't think that is quite right. A lot of reasoning is subjective in IPS. What references are good enough? How much experience must they demonstrate? Do they have to have nearly identical life experiences or can they merely have vaguely/questionably applicable experience to the OP? The answer is somewhere in the middle, which is why enforcing a rule to display experience or reasoning would require quite a bit of subjective arbitration. – BlackThorn Mar 22 '18 at 21:29
  • @BlackThorn that's a discussion we can have after we start encouraging backed up answers, fact remains that backed up remains way better than having to judge common sense. Because what's common sense for me isn't automatically applicable to someone in a culture as different from mine as Pakistan for example. – Tinkeringbell Mar 23 '18 at 6:26
  • So, if I'm going to write an answer, I'd better damn well include that I'm from the Netherlands and explain why I think my experience is applicable in their situation as well, exactly as described here interpersonal.meta.stackexchange.com/questions/83/… – Tinkeringbell Mar 23 '18 at 6:26
  • Now, you can tell me that that will be worse than having to judge the quality of someone's reasoning if there's no back up, but it will make it easier for people to see that my answer comes from completely different cultural experience and it will make it a lot easier to judge than if I'd just included the common sense.... – Tinkeringbell Mar 23 '18 at 6:29
  • 4
    @Tinkeringbell But at the same time that's just part of what makes your answer quality better, but it still doesn't seem like something we can effectively regulate. I once had someone vote to close this question interpersonal.stackexchange.com/questions/7673/… of mine because it didn't have a location tag... There's no need for one. I've already stated exactly how every involved party feels about the given situation. So why should it need a location tag... It literally adds nothing to the post (cont.) – Jess K. Mar 23 '18 at 13:46
  • 5
    @Tinkeringbell and it's those kind of judgement calls (where not everyones opinion on what is good vs bad) that make implementing a rule such as "back it up" too much of a judging game. We start voting to delete answers based on how good we think the reasoning is and it'll just come off as bullying, and discourage anyone from improving. Unless you can think of concrete, measurable qualifications for "backing up your answers" and how we can regulate it, I don't think this will be as helpful... I think we're better off using voting system and comments to encourage users to improve their answer – Jess K. Mar 23 '18 at 13:48
  • Well, a location or culture description isn't so much about how both parties feel, but also about how a certain kind of behavior might be received by the other party. I think site culture has improved now, and we're asking for cultural background more than location (at least I try, because I understand that just a location tag doesn't give enough information most of the time). But, I don't hesitate to cast a close-vote if I can't see if what is described is what can be expected from a certain culture, or e.g. if there is a cultural restriction on the solutions that can be tried. – Tinkeringbell Mar 23 '18 at 14:11
  • @JessK. I have, I believe, nowhere in the question mentioned that we should delete stuff. I asked how we can encourage/enforce a back it up rule and better answer quality (although my original title left something to be desired). So, if you want to use the voting system and comments to do so, maybe you can explain in your answer how we can use that. When do we vote, when not? When do we leave comments, is a stock comment linking to meta okay? – Tinkeringbell Mar 23 '18 at 14:11
  • 1
    @JessK "it'll just come off as bullying" exactly, and on a website where some mods routinely resort to ad-hominem and encourage downvoting + flagging to attack other users, it doesn't come as a surprise. – Andrea Lazzarotto Jun 5 '18 at 22:47
  • 1
    @AndreaLazzarotto just for completeness sake, Shog isn't a mod here. He's a StackExchange Community Manager (he's employed by SE). If you have a problem with him or any of the mods here (or any SE site), you should use the "contact" link at the bottom of every page. – scohe001 Jun 5 '18 at 22:53
  • @scohe001, yeah even worst. BTW of course I've already used the contact form more than once, they do not particularly care. I had luck on Ask Ubuntu though, several of his offensive comments were pruned by mods. – Andrea Lazzarotto Jun 5 '18 at 23:15

How can we encourage/enforce the backing up of answers?

It should not be enforced! Encourage by explaining what you are missing.

Quality of the answers should not be a concern for moderation, because the StackExchange tries to solve exactly this problem by democratic vote mechanism. While it is nice to have backups of claims, especially in interpersonal matters it is impossible to know your solution works! That´s why it is really valuable to have a great diversity of approaches and also be able to put them into perspective by popular vote and comments provided.

If the back-it up rule is strictly enforced, we will loose a lot of answers. A lot of the users here are not scientist working in this field and having research an citations ready. Also a lot of question are not specifically seeking scientific information but more of an outsiders view or even wisdom from others.

What you where are asking for, here, is a mandate to to moderate more aggressively to subjective opinion, instead of relying on the idea this whole community is build upon, just because you may not agree in some cases with that popular vote. This would render the unique mechanism of StackExchange Q/A moot.

I suggest to try an interpersonal approach if you think an answer can be improved. Instead of waving with site-rules, ask the author for clarification or context. Most will happily improve their posts when made aware of what exactly is unclear.

  • Hey Daniel! Seems we've just crossed each other: I have since edited the title to ask about how to encourage backed up answers, since other meta's are already discussing the enforcing of the back it up rule... – Tinkeringbell Mar 22 '18 at 11:22
  • 4
    The "Back it up" policy doesn't require scientific citations or anything; it just means talking in detail about the experience that led you to a recommendation. – HDE 226868 Mar 22 '18 at 13:04
  • 6
    @HDE226868: A lot of interpersonal "wisdom" does not originate from one pivotal experience but more or less 1000´s of interactions throughout ones life. So you could just tack a disclaimer onto each and every post personal opinion upon accumulated experiences over <age> years Don´t know if that improves anything. – user6109 Mar 22 '18 at 13:28
  • 4
    @Daniel, if that's the case, and there's years of experience with a single skill, then why not pick one that was similar to the one someone is asking about as a case-study to back up an answer? Explain the similarities, preferably also the differences, then work to how you think the answer might work or what is uncertain because you've never been in the same situation.. – Tinkeringbell Mar 22 '18 at 16:25
  • 5
    @Tinkeringbell: Nobody is ever in the same situation as this always depends on who you are talking to, and we do not all share the same one problematic friend ;). Answers here never come with a success guarantee. If there just is not any single memorable event but just a feeling what would be proper ... do we want to discard such answers then? Even if any rational human thinks this is still good advice? – user6109 Mar 22 '18 at 16:33
  • @Daniel, if I am going someone to suggest not getting into an argument between their divorced parents, but to stay out of it, I explain how it worked for me with two arguing colleagues that tried to put me in the middle... I try and consider if there are power balances that might make this situation different, or in the case of parents they're of course closer than colleagues, so you'd want to be extra careful not to hurt any feelings... If there's just a feeling, I'd like to refer you to the quote in my question saying 'don't write the answer'. – Tinkeringbell Mar 22 '18 at 16:52
  • I gave you a link to the good subjective, bad subjective blog post in chat already, it states something similar: answers should include back up. A feeling is no more than an opinion, and opinions should be backed up with references (sources or experience). I'm looking for a way to make the distinction between a good and bad answer (purely based on being backed up or not) clearer – Tinkeringbell Mar 22 '18 at 16:54

One possibility might be to edit in sources for some of the better, but un-cited, answers. This would take a lot of extra work, and some very diplomatic commenting, but if done well it could simultaneously create some good examples of "backed up" answers and reduce the number of highly-voted, unsupported answers, all without dinging folks for not having done this already.

Direct editing would work best with answers that advocate techniques that can be found described in scholarship or at least some reasonable online source. Obviously this would work much less well for personal experience, since we can't attribute our experiences to the original answerer. One approach for this kind of answer/evidence might be to include the relevant experience in a comment, with a note inviting the answerer to incorporate it into the answer if found useful.

I've taken both approaches on English.SE, where citation requirements are more regularly enforced, especially with new users. For example, if a new user gives an answer to a "single-word-request" that sounds reasonable but without any definition to back it up, I might find a relevant definition and edit it into the answer. I'd typically then leave a comment saying something like

Hi, NewUser, welcome to [this SE]! I think your answer is a good idea. I've added a dictionary definition with a link, as answers with some support are generally better received here. Please feel free to edit further (or substitute your own definition if that's not what you were thinking of). You can have a look at our help center for further tips about writing answers--I look forward to seeing more contributions from you!

Answers that are just a definition sometimes get a comment about adding a sentence or two explaining how this fits the request.

On other occasions, I've been researching an answer when someone posts essentially my response, but without some of the citations I've found (especially etymology questions). In that case I might only add a comment, rather than directly editing:

+1 I've also found the term used in several books from that decade, including [quotes and links]. Please feel free to incorporate any of this information into your answer if you find it helpful.

Obviously in the realm of IPS the types of support that could be added would be rather different and less straightforward, but I think for some answers it would be doable. Some answers advocate using a specific IPS strategy or technique that has Wikipedia entries or Psychology Today articles or even scholarly articles that could be linked (setting boundaries, active listening, negotiation, etc.), while in others someone other than the OP might be able to offer an example of having used the recommended strategy effectively.

  • No, only the OP should be adding substantial information to their answer. (And please stop adding research to posts at ELU!) You are always free to write your own answer with your own sources though. – curiousdannii Mar 23 '18 at 8:18
  • 3
    @curiousdannii interpersonal.meta.stackexchange.com/a/2344/1599 seems to suggest that editing in links is encouraged... Especially since this site already gets lot of answers that are similar.. – Tinkeringbell Mar 23 '18 at 9:01

First: I completely agree with Jess K.'s answer.

But if you, for some reason, decide that backing things up with anecdotes is a must for this site, then 1) it must be immediately clear to casual users what is backup and how much is enough/not enough, and 2) it must be consistently enforced.

Ad 1): I tried finding the "how much is enough backup" rules in the help section - and it didn't hit me in the face within the first minute of searching for it. From my experience as a software dev: Linking to lengthy meta discussions / 500 page manuals / referring to email communication / etc will not get users to use those resources, extract the rules "correctly", and apply them consistently. You need clear, practical examples of what backup looks like. E.g. please give a concrete example how you back up general "rules" of etiquette in Western culture - if it's not enough that 37 people upvoted it.

Ad 2): In the last days, I've repeatedly stumbled upon questions where an upvoted/accepted answer was "decorated" by mods with "Can you back this up?" while other (and, in my opinion: less relevant and less well-worded answers) were not. To an outsider, this is not distinguishable from mods trolling some users and leaving other users alone. My recommendation to make things look "fair" would be: if you feel you need to leave a back-it-up-comment on one answer, check the other answers on the same question if they need the same comment. Particularly, if you threaten to delete the answer or parts of it. Otherwise, you're sending a signal that the other answers are OK! It doesn't matter if you don't intend to send that signal, you're sending it.

I find it much more easy to learn the back it up rules from seeing one question where it has been consistently applied to all answers than from 10 questions where it has been haphazardly applied to some answers but not all. At least I more frequently compare answers on the same question with each other than answers on different questions.

You may say: "But it's a lot of work to look at all answers on a question!" Yeah, it is - that's why I find the voting system of SE so efficient. It doesn't rely mainly on moderators deciding what is good advice and what isn't, it relies on the Wisdom of the Crowd - so back to Jess K.'s answer.

  • 3
    Not sure if you meant to, but I really enjoy how this answer is "backed up" with your dev background :) that being said, I'd imagine mods deal with answers here like a hospital--the patient who needs the most attention now is going to get it now. I think it's more important to get comments on up-and-coming answers before they get high voted and deleting them becomes controversial. If you think some answers are missing comments for back it up, then add them! Enforcing this policy is a community effort :) – scohe001 Jun 7 '18 at 17:20
  • @scohe001 I don't like the way the enforcement of this policy happens so I should be doing it myself? Sorry, but: no. In writing this answer I took 1/2h out of my day to describe the POV of a casual user - and the only reaction is 4 downvotes (probably from the mods who don't want to hear criticism of their new idea) and a comment on how I should be doing the work they want done. I'm not willing to invest any more in a stack that goes the way this one is going. – AllTheKingsHorses Jun 8 '18 at 7:52
  • 2
    All StackExchange sites are moderated to some degree by the community. The idea of "back it up" isn't just some idea the mods came up with on their own. Of course nobody but mods are required to help moderate, but I don't think it's really fair to complain about other users not doing things that you aren't willing to do either. – Em C Jun 8 '18 at 12:53
  • @EmC I've got no problem with the back it up policy in general. In the case of this stack it's taken to a level of arbitrariness that I don't comprehend. So I wouldn't even know when to help moderate and when to keep quiet. My request for more clarity and explanation just got a lot of downvotes as reaction. And: I'm not complaining because the mods aren't posting back-it-up-comments. I'm put off because I can't see a fair pattern on when they do and when they don't. I'm put off to the point where I don't want to invest a lot of time... So I'll just keep quiet and see how things develop. – AllTheKingsHorses Jun 10 '18 at 12:26

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .