Do we have a problem with people answering questions that they don't know the answer to?

Question

It seems like we have users who tend to answer out of what they believe to be "common sense" and in some of our more general, everyday sort of questions some of these even sound like reasonable answers. The common sense applied seems to fit, the OP is content with the answer, future readers seem to find the answer helpful and things work out fine.

On the other hand...

Sometimes we have users who tend to answer out of what they believe to be "common sense" on some of our more niche questions. Basically they're applying what they believe to be common sense, to a question about an uncommon or less common situation.

It seems like this may be causing some problems.

• The OP may be getting advice from someone who's effectively guessing at what the solution ought to be.

• These answers often aren't exactly well supported. The reasoning may be explained, but there's often holes in the reasoning that the answerer doesn't see.

• These answers can often be somewhat tone deaf, or offensive to people who have had experience with the situation in the question.

• There's often no indication in the answer that it isn't based on experience, or research.

• And perhaps worst of all... These answers being written by people who don't know that they don't know, are often voted to the top by other people who don't know that they don't know. This sometimes leaves other answers written by users who have direct knowledge of the subject unread by future readers, as they're sometimes 3 or more answers down the list.

This seems like it may be a problem that's somewhat unique to more subjective Stacks. To use a Stack Overflow comparison, if someone who'd never written a line of C# wrote an answer to a question specifically about C# it would probably end up voted into a hole, or deleted entirely. That doesn't happen here, because most of our answers aren't so easily tested, and many of our questions and answers are largely voted upon by people who may never be in the exact situation the question is about.

It seems that our "back it up" rule helps with this situation, but from what I'm seeing, I'm not sure that it helps enough.

Is this a problem? And if it is, what can we do about it?

Answer 1

Yes, this is a problem. It may be difficult to decide in exactly which cases it's "bad", but there are definitely answers without the necessary experience or knowledge to support them. (I don't think it's helpful to hunt for examples here; it feels a bit too close to name-calling.)

This is completely unsurprising. Sure, it's rare for people to knowingly post bad answers, but we often don't know what we don't know. As long as people are allowed to post confidently clueless answers, they will do so. And as long as those answers are present, the sufficiently convincing ones will gather upvotes - there will be other people who have similar gaps in experience or knowledge.

Realistically, I think the only solution is to actually enforce the back it up policy, e.g. with deletion of answers. There is ample precedent across the network for going beyond "not an answer" deletion. From the network-wide FAQ on deletion (in the "What are the criteria for deletion?" section, after covering "not an answer" and such):

These are general guidelines; some communities in the network may uphold more specific reasons to delete posts or not. For example, on Puzzling.SE, answers to a puzzle without explanation are subject to deletion, and some technical sites will delete answers which are not only wrong but also harmful when tried.

This is, of course, not an easy thing to do. A lot of the judgments to be made are difficult. And we can always offer guidance before enforcing via deletion. However, I see no way around it, so I think it's a tough path we are forced to follow if we wish the site to flourish.

---

So how would this look?

For process: what Monica describes from Workplace seems good. Essentially, suggest improvements to add support/explanation, if they aren't made, flag to get a post notice added, if that doesn't get anything done, flag again to delete.

For what to address, I've tried to think of some deliberately vague examples. As with everything in life, things may vary depending on details.

• Probably good: an answer based on direct personal experience, with an explanation that makes it clear this is the case. ("I've dealt with A before, and I've found that doing X helps achieve your goal.")
• Probably good: an answer based on indirect personal experience, which is clearly applicable/extensible to the case at hand. ("I haven't seen precisely A, but I have seen B, and I've found that doing X helps achieve your goal.")
• Probably bad: an answer that provides arguments but not experience, especially if it reaches a conclusion that differs strongly from those based on experience. ("Obviously if someone wants A, they also want B, and so you should do X.")
• Probably bad: an answer based on opinion/belief. ("It's just common sense that you should do X.")

We should also keep in mind the context and potential for harm when flagging. If people's feelings are getting hurt, that's a sign we should be increasingly cautious about unsupported answers. On the other hand, if it's a low-stakes situation that no one has direct experience with and people are tossing out potentially useful ideas, we can probably be a bit more flexible.

And this is about mitigating an issue, so getting even catching even 50% of the bigger problems is already a good start, and could even be effective about shifting the general site attitude and expectations about answers. We don't have to try to hunt down every last instance (especially on older questions), and we don't have to try to apply this strictly in cases that are extremely unclear.

---

To take a step back, I think that it's really valuable for us to provide guidance and support for the moderators here. They're the ones who are ultimately going to have to handle our flags, and in some cases delete upvoted answers. In order to do that, they need to have meta posts/policies to stand on, and they need to be able to look forward to support when they're inevitably challenged.

If we agree about this path, it may be that we eventually need to discuss some specific examples to help get this started, and we will likely need to discuss some after the fact when challenged. If and when we do so, let's approach it carefully. We do not want to put people on trial for their answers.

Answer 2

We sometimes have this problem over on The Workplace, where we also expect people to back up their answers but accept experience. (And, realistically, a lot of answers aren't explicit about the backup and they stand, like you're seeing here.)

One factor that comes up for us -- I'm not sure how common it is on IPS -- is questions (not) stating things that ought to restrict answers, or answerers missing or ignoring such things if they are present. For example, somebody answers a question about notice periods saying "2 weeks is the norm; just go" and either the OP didn't say or the answerer missed that this is in India, where notice periods are longer and you need a relieving letter from your previous employer (so walking out on them is counter-indicated). Or somebody answers a question about escalating a problem to management and misses that the OP is in Japan (where deferential phrasing might be expected) or Germany (where straightforwardness might be expected).

The informal practice that has evolved for us is approximately the following:

• If an answer makes unsupported claims and anybody objects, comments ensue. Well-written comments requesting clarification are a good thing here. All those other comments get deleted or moved to chat.

• If those unsupported claims are irritating enough, somebody usually flags to request (or a moderator notices and adds) the "citation needed" post notice. If the post remains unchanged for long enough, somebody will flag again and the moderators will probably delete it.

• If the unsupported claims don't bother anybody, nothing special happens.

• If the question hits HNQ, things can get messier.

Many of our users read questions (and answers) carefully and ask for missing information. For us it's often location or type of company or roles of the people involved; for IPS it's cultural context or location or other things. You don't need to be knowledgable about a particular context to ask for it if it's missing. I leave comments all the time asking for clarifications where those clarifications won't help me answer the question (I don't know enough about, say, writing an Arbeitszeugnis to help) but I know something important wasn't specified. I leave those comments to (a) help other people be able to answer and (b) alert the community that there is missing information. Similarly, if our users see that an answer doesn't fit with some details in the question (he said he's in the Ukraine, so your advice to involve OSHA won't help him), they take action.

Figuring out what counts as backing up an answer when personal experience is relevant can be tricky. Before trying to formalize rules, it's worth seeing if we can all be attentive to missing or missed details and be diligent in writing good constructive requests for improvement. Or, if necessary, flagging for those post notices -- but I've found it's usually better to first try to get the author to fix the problem without that badge of shame hanging on the answer.

Answer 3

The correct way is to write a better answer.

The SE model is quite easy,you are responsible that you reflect your experience or research to a degree that people can understand your answer, are convinced that you are right and upvote the answer.

So if you think that an answer is wrong, explain in detail what is wrong about it and how your experience and research invalidates it. Especially interesting is the part how do you know that an answerer does not know and that people do not know. Yes, you are responsible to validate your claims. Yes, it is your task to demonstrate that an answer is tone-deaf and offensive apart from your own opinion.

So the described problem is inevitably linked to the quality of the people providing answers. If they cannot convince people, it is not a "problem" which can be solved. If the community provides good convincing answers, the "problem" disappears. There is no independent judgement possible, who apart from the community which includes the answerers could judge?

Answer 4

Essentially just a mirror of Cascabel's answer, with a few alterations in terms of how to go about solving it:

Yes it is a problem. Yes, scenario's like this are one of they key reasons we encourage users to back up their answer. However, I do not think we should change our rules and start forcing users to back up their answers.

All of the reasons here (bar one I will talk about later) are exactly in line with why we encourage users to back up their answers. Why doesn't this site have a back it up rule clearly outlines essentially the same problem here and similarly, I think the top voted answer clearly outlines how we should go about handling this problem:

TL;DR

Yes, we should have a "back it up rule". It should be added to our help center.

However, we need to be very careful about how we "enforce" this rule, because good advice on interpersonal skills can be based upon an amalgam of personal experiences built up over time, and doesn't require specific parallels to what the OP is experiencing in order to be valid, high quality advice.

This rule should be used as guidance to help improve low-quality answers, and not a strict yardstick that should be applied universally.

There is only one single exception to this, and that is in answers that are inherently offensive. (I am talking about the one problem I did not address before where you said: "These answers can often be somewhat tone deaf, or offensive to people who have had experience with the situation in the question.") This is far trickier to deal with.

In these cases we can have an answer that is both not backing itself up and offending the OP. When the answer includes things that are obviously, and inherently offensive then we all know to delete it. The problem is that most of the time, the offence is made through ignorance and there are no actual rude words used and it is only offensive from the OP's subjective viewpoint. These cases are almost impossible to police as often those moderating the question may even disagree whether or not the answer was offensive or not and really, we should only be deleting answers like this if we can agree that it violates the be nice policy. So, answers that are not backed up, nicely worded but promoting what I view as an ignorant viewpoint I think down-voting is the way to go.

In the end, I see this as just yet another example of why we should be actively encouraging and up-voting answers that have backed their reasons up, even if we disagree because these answers are not leaving it ambiguous and are far more useful for our site.

Answer 5

Backing it up is a great concept but how will we enforce it ?

Saying "I have experience in this subject" is easy to say and hard to disprove online. Asking for experience is so easy to fake that people are probably already lying about it.

Linking to studies can be cool and give an option for people to go more in depth with the source material, but it also has a couple of problems. You don't necessarily know where to find studies, you may have a read a study but have trouble finding it again, you may also link to a fake study.

A "back it up" policy is hard to enforce. Requiring experience or studies from people who just want to voice their opinion or use "common sense" will just make you see how easily they will be able to justify themselves by "experience" or other "studies".

So at this point we have few options :

• We start to doubt every answer, asking for sources or experience.
• We trust answers backed up by source and experience to be naturally better and trust people to recognize their value, hence upvoting them.
• We get rid of the policy
• We trust the OP to be able to pick the best answer, based on it's efficiency rather than by how it looks on paper (won't always work as OP can't just test all the answers and do his own little study.)
• Something else

We have to decide who or what we trust. Studies are nice, but we can't do much else than to trust people with their advice or not.

Answer 6

Use a soft approach to improve the Answer. If that fails, deal with it!

• Comment, best by asking the author for help to understand his answer. A subjectively bad advice with a highly up voted comment balancing it may show controversy on a topic better to future readers than any number of unrelated answers can.
• Vote.
• Post your own answer.
• Edit, if you have information to improve the answer.
• (Maybe:) Feature to mark answers as option/experience-based as a flag for readers?

Never delete those answers. (Popular answers, that don´t go against the be nice policy or are objectively wrong)

Stack overflow has a specific model that tries to reach maximum helpfulness by self-moderation. Picking and deleting specific answers because they seem subjectively low quality undermines this model. It´s basically saying the stack... platform is not working for IPS, we need a different format.

This is like democracy - sometimes you get things like Brexit or Trump which you may personally dislike, but overruling those decisions would invalidate the whole system.

Answer 7

It's a pity there is not a tool that when applied, keeps an answer slightly lower in the list, despite having a higher score, and adds a note at the bottom to explain that's the case.

Explanation

The issue seems to be that once an answer is at the top of the list, it's also that bit more likely to get more attention and more upvotes. Usually that works fine, and it's what is wanted, but in a few pathological cases, an answer that is ignorant, guesswork, or harmful, gets established and at that point feeds off its own success, in a feedback loop, whether right or wrong.

Most times and in most stacks that self corrects - but not all, and this Q is asking about how to break that outcome for the few answers where there is real doubt they should be at the top, without harming the general flow of things or imposing some version of "mod decision" as to validity.

My thought is to break the linkage between votes, and position in answer list, for a few days.

Suggestion

Suppose a flag existed, call it the dubious flag. A question with the dubious flag will at most be allowed to rank as the 3rd highest answer on the question, and will display a message at its foot, something like "This question is being held back for a few days because of doubts whether it is valid or appropriate. You can still upvote and downvote it."

This will encourage users to consider its validity without forcing or overriding them. Users above a certain level might be able to flag that this flag is correct or incorrect in their view, as feedback on the flag itself.

The outcome would be that for a while, the dubious answer doesn't have pole position and is marked as one where a doubt may exist, so with luck over the next couple of days we will see specific efforts to addressing that, and more attention to answers not flagged as dubious, at which point the flag is removed as the harm has been countered.

In the case that this flag is applied where it shouldn't be, the response would (hopefully) be that users will still upvote it and mods will see support for the answer (and lack of agreement that the answer is dubious) by those users having an appropriate level, which will quickly suggest the flag should be removed. At which point the question is ranked according to votes as usual and no longer held back.