4

We've had the "back it up" policy for almost a year now. It was one of the earlier things we seemed to think important for the quality of answers on the site.

We've had more than a few discussions about supporting answers with experience or references, as well as a few discussion about the problems that unsupported answers seem to be causing:

What should we do with answers that are not backed-up?

Related Answers: Why your Pakistani answer won't always work for India

Do we want references in our answers?

Should we encourage writing from experience?

How can we encourage/enforce the backing up of answers?

Do we have a problem with people answering questions that they don't know the answer to?

(I'm sure I've missed some, feel free to add to the list)

Admittedly, I once thought that votes would handle this issue. I'm generally not a big fan of hard rules and I perhaps had a little too much faith in people's common sense (pun intended) when it came to voting. Unfortunately it looks like we've had a lot of cases where votes have made the problem worse...

It seems like the majority of users seem to agree that unsupported answers are a problem, and most seem to agree that we can, and probably should, handle it the way that other, more subjective, SE sites have handled it.

https://interpersonal.meta.stackexchange.com/a/2985/59

https://interpersonal.meta.stackexchange.com/a/2988/59

https://interpersonal.meta.stackexchange.com/a/1265/59

(Again I'm sure there are others, feel free to add them to the list)

To paraphrase the above...

User side:

  1. Someone posts an unsupported answer.
  2. Add a nicely worded comment asking for the answer to be supported.
  3. Flag the answer, using a custom mod flag, to let them know that the answer isn't supported.

Moderator side:

  1. A moderator responds to the flag by commenting and/or adding a post notice.
  2. If the answer isn't edited to add support, or the answer is significantly problematic, the moderator may then delete the answer.

Please realize that that's a paraphrased tl;dr of the three answers above. Please read the above answers for the fine details.

The question is... Are we ready to bite the bullet and start enforcing our back it up policy?

I would like to think that we are.

(Note that this is tagged feature-request, we've discussed this plenty, but this isn't a duplicate of the previous discussions because I'm asking that we finally take action.)

36
  • 2
    Notably, the voting on the last (most recent) question you linked is strongly in favor of essentially this kind of enforcement, so in the spirit of mods enforcing policy created by the community, it seems like the answer has to be yes, unless the mods realize something I've missed (very possible!).
    – Cascabel
    May 22, 2018 at 21:25
  • 1
    Helpful query: interpersonal.stackexchange.com/…
    – Shog9
    May 22, 2018 at 21:32
  • 7
    Custom mod flags are not the first step. This site needs the users to actually do some of this work. Moderator deletions can not be undone by users, which means that if we delete stuff, we make more work for ourselves in needing to watch them and undelete them if they're fixed.
    – Catija StaffMod
    May 22, 2018 at 21:35
  • 2
    @Shog9 point taken ...
    – Tinkeringbell Mod
    May 22, 2018 at 21:36
  • 2
    @AndreaLazzarotto Not really, that question is about identifying a problem and possible solutions, this one is requesting the implementation of a solution offered there.
    – Tinkeringbell Mod
    May 22, 2018 at 21:37
  • 5
    NAA flags are better. By the standards of this site, unsupported answers are not answers.
    – Catija StaffMod
    May 22, 2018 at 21:39
  • 2
    @Catija That's a pretty easy tweak to make to the question, as long as y'all are sure that it's fine. We don't want to NAA flag if it's not going to be clear enough; that's a good way to get mistakenly declined flags.
    – Cascabel
    May 22, 2018 at 21:41
  • 1
    Well, they'd be "disputed", technically... not "declined"... assuming you mean users disagreeing with the flags, @Cascabel ? I'd really love to have the NAA explanation changed but I don't really think we can do that here.
    – Catija StaffMod
    May 22, 2018 at 21:47
  • 1
    @Catija But aren't we technically allowed to do something if people are 'abusing' the review queue? Like educate them that answers that lack back up should be considered NAA? If we can agree here that we're going to enforce such a policy, that's another thing on the list of enforcing that stuff then: Watch the queues for that.
    – Tinkeringbell Mod
    May 22, 2018 at 21:50
  • 1
    @Catija I am mainly concerned about the ease of looking at a generic flag like NAA, on an answer that perhaps provides some stuff that looks on the surface like backing it up but in fact isn't really solid, and saying "nah, that's an answer." That is, I think that generic flags could conceivably make this more difficult for y'all to handle, by forcing you to be very careful about all flags you handle.
    – Cascabel
    May 22, 2018 at 21:52
  • 1
    @Cascabel I'm not saying to never use custom mod flags... but going straight to a mod flag completely excises the entire site from helping to deal with these posts. We're exception handlers, not judge, jury and (post) executioner. Many of these do get deleted by regular users. The mods should not be the first line of defense.
    – Catija StaffMod
    May 22, 2018 at 21:54
  • 2
    I got downvoted for not backing up answers to low-quality questions that (for me) just lack some "general understanding", some life experience or only need a hint to think on. Should we write "source of this tip: life" after every answer? Should we keep answering even without knowing a recent situation that was exactly the same and we helped out by doing x and y? What to do with questions with no clear "do this" answer but a vague hint could help more than leaving it blank? Please avoid rules that nobody can follow. Too many downvotes push people off. Better forbid downvoting without comment.
    – puck
    Jun 3, 2018 at 19:12
  • 3
    @puck that looks like it could be a new question.
    – apaul
    Jun 3, 2018 at 19:39
  • 3
    SSight3 said it right, nobody knows if the "personal experience" is correct. Even if, people and situations are different so who knows the same solution will work the same in a different situation. Another thougt: An answer should not provide false safety for the asker. Instead they should think about it and adapt it to their problem, situation and other people involved. Explaining with "this could work because [feasible reason]" may be better than "it worked for me so will it work for you".
    – puck
    Jun 5, 2018 at 4:15
  • 3
    @puck Again, if you'd like to challenge the policy or the way it's enforced, write up your thoughts in an answer or a new question. Comments don't change policy.
    – apaul
    Jun 5, 2018 at 4:37

2 Answers 2

Reset to default
8

Finally!

Here's what I'd propose as a strategy for moderators:

  1. Moderators see an unsubstantiated answer, they either delete it outright with a comment, or slap a post notice on it and revisit in a few days and delete if not substantiated.

...that's it. Pretty simple. But... We can't exactly expect moderators to review every answer posted here. Nor can we expect moderators to recognize every unsubstantiated answer that they come across. This system doesn't work that way - many hands make light work, and so if we're gonna do this we need many hands...

So here's what I'd propose as a strategy for everyone using the site:

  1. See an answer that's unsubstantiated

  2. Post a comment to the effect of... No, y'know what, I had a generic example here but that was a bad idea. Post a comment specific to the answer:

    • point out any claims that aren't supported by evidence of some sort - something the author has experienced, something they've read, anything tangible.
    • explain why you're having trouble accepting these claims without backup
    • politely ask the author to provide supporting evidence for their claims.

    This step is important. I cannot emphasize that enough. This site isn't an informal support forum, but sometimes it sure looks like one - and so you can forgive authors for writing in the style they would on such a forum, and gently guide them towards something a bit more rigorous. If you get into a disagreement, stay polite and ask them to raise the matter for discussion here on meta.

  3. Downvote the answer

  4. Flag the answer as "Not an answer"
  5. Go to review and review posts flagged by other members of the site, voting or recommending deletion in cases where they too fail to provide backup for their assertions.

how this can work

If the post collects a significant number of delete / "recommend deletion" reviews, and scores <= 0, it'll be deleted - but in such a way that the author can undelete it if they wish to.

If the deleted answer is undeleted or scores more than 0, a moderator flag will automatically be raised.

Moderators, now informed of the need, can then follow the proposed "strategy for moderators" outlined at the top of this post.

24
  • 8
    Keep in mind that the first time you delete an answer for being "unsubstantiated", you are sending the message that ALL other non-deleted answers are deemed "substantiated". Hope the mods have a lot of time on their hands dedicated to overriding the community. Sounds like a full time job.
    – Clay07g
    May 23, 2018 at 15:38
  • 7
    @Clay07g That... really isn't the case. That's like saying that all comments are allowed because we haven't deleted them all. If they aren't brought to the attention of the users or moderators of the site, they're simply not handled. That doesn't mean that they're acceptable.
    – Catija StaffMod
    May 23, 2018 at 15:41
  • 6
    Yes, there's an element of subjectivity to this, @Cashbee - that's par for the course for "not answers", which is why we have a review process. But honestly... Y'all are probably over-thinking this. This doesn't have to be as strict as, say, Skeptics - you don't have to start by compiling criteria for what sources can be trusted, or debate the validity of original research... We can get quite a ways by just asking, "why?" When someone asserts that something is true, that a solution will work, that people will react a certain way, does the answer also explain why the author thinks this?
    – Shog9
    May 23, 2018 at 15:46
  • 5
    @Catija Not acceptable, but accepted nonetheless. Betting on comments staying up in IPS could be a fun gambling game with the way they are handled. Perhaps acceptable, given the problem that unregulated comments cause. Do "unsubstantiated" answers really cause so much trouble that it's worth it to implement another inconsistent policy? At what point am I going to ask "Are answers deleted because they are unsubstantiated or are they deleted because my interpretation of substantiation is different that a single moderator?"
    – Clay07g
    May 23, 2018 at 15:48
  • 4
    @Shog9 Can the difference between a substantiated and unsubstantiated answer come down to including "In my experience, [claim]"?
    – Clay07g
    May 23, 2018 at 15:51
  • 10
    There are sites where over half the answers posted during the first year ended up deleted, @Clay07g. Do we need to be that strict here? Probably not... Do we need to be more strict than we're currently being? That's the purpose of this and numerous past discussions (see linked sidebar). You're essentially arguing that because we can't be perfect we shouldn't even try to be better than we are now - that's kind of a defeatist attitude, but if you want to make that argument then please post your own answer.
    – Shog9
    May 23, 2018 at 15:55
  • 7
    Believe it or not, that's already happening @Beofett - for the past 6 months, a majority of answers here have been deleted by vote, not by their authors and not by moderators. Similarly, low-quality review handles over 70% of VLQ and NAA flags. This wouldn't have had a chance of working back when we first discussed it; now it actually might: the greater community of IPS has the ability to do this sort of moderation, they just need to decide that they want to.
    – Shog9
    May 24, 2018 at 16:56
  • 5
    @Shog9 except now you have the opposite problem, good answers being deleted for not being PC/groupthink. Vote to delete simply has become double-plus-downvote. Which is turning IPX into an echo chamber, you must write to an editorial stance no one can even describe.
    – Harper - Reinstate Monica
    Jun 3, 2018 at 13:49
  • 6
    If you're sore about the deletion of your speculative post that made no effort to answer the question, post a question about it here, @harper. I see no evidence that it was removed for not backing up its solution; for that to have happened it would have had to have contained a solution.
    – Shog9
    Jun 3, 2018 at 14:31
  • 4
    @shog9 first, I am sick of being accused of being "sore" about something. I expect beyter behavior from a mod on an interpersonal skills forum... since you've taken it ad-hominem. I did not attack you and don't know to which post you refer. This all is a distraction; it does nothing to address my point, which threatens the forum itself.
    – Harper - Reinstate Monica
    Jun 3, 2018 at 14:56
  • 7
    First, I told you to ask a question here, @harper. So go do that if you wanna talk. You came in lobbing unsubstantiated claims on a Sunday morning, so I went and did the research that you didn't and posted my conclusion. If you don't LIKE my conclusion, then go start a discussion where you can include actual evidence to support your assertions.
    – Shog9
    Jun 3, 2018 at 15:11
  • 4
    I have no doubt you did research; I just have no idea why you wiuld, or what you found. You do seem to be laboring to infer a specific concern from my general one -- what you may be overlooking is I now have sufficient rep to see everyone's deleted answers. With me, don't look for passive aggressive inferences. I can, do and did say exactly what I mean.
    – Harper - Reinstate Monica
    Jun 3, 2018 at 15:29
  • 6
    I have no idea, @harper. Either you started this conversation because your answer got deleted and you want to talk about why, or you raised it for some other reason and want to talk about that. But so far you've wasted a lot of comments not talking about anything specific, so maybe with the generous 30,000 character length limit of a question you could eventually get to the point.
    – Shog9
    Jun 3, 2018 at 15:48
  • 6
    Well, noted then. I'll file your complaint with the rest of em.
    – Shog9
    Jun 3, 2018 at 16:20
  • 7
    @Harper If you have an example of a good answer being deleted that shouldn't have been, post a question about it. I doubt that people are deleting things for reasons of political correctness, but if you have a doubt the best place to discuss would be in a dedicated question.
    – apaul
    Jun 3, 2018 at 20:44
3

I agree that we should be "backing answer up", but I'm quite disappointed that everyone seems takes that to mean "present some similar experience you've personally gone through" (and literally nothing else).

Justification, explanation or elaboration is the way we should be backing answers up.

What we should be looking for in answers is:

  • Reasoning about the situation (e.g. "I'm recommending A because people like B and don't like C, and this can be seen by the fact that they did D")
  • Addressing possible criticism (e.g. "This may cause A, because B. If you want an approach that is less likely to do so, you can try C instead of D")
  • Explaining how to handle possible responses (e.g. "They are likely to say A because B, to which you can respond with C, which will do D")

Because we want others to:

  • Understand why you think the advice you're giving would work (for reasons other than "well, it worked before")
  • Judge the advice by the quality of the advice (instead of by the user posting it or by just taking their word that it works)
  • Generalise the advice to other situations and handle unexpected ways the situation can play out

What we should not be looking for is: (for reasons explained below)

  • I'm right because I went through something similar
  • I'm right because I'm also X
  • I'm right because I'm a psychologist

Experience (by itself) is not a good way to back an answer up.

Adding some experience to an answer might have some uses, but overall I'm not convinced about the benefit of an anecdote (or otherwise justifying experience).

  • People aren't all the same. Something working once, or in a specific environment, doesn't mean it will generally work or it's good advice. Providing an anecdote of how one person responded may, in fact, mislead the reader into believing this is how something is likely to be received.

  • Some of these scenarios are extremely specific, meaning there may not be anyone able to speak (much less give the best advice) from experience.

    Others may be able to speak from experience, but have privacy concerns about sharing said experience (since plenty of the questions here are about what can be considered very sensitive and private matters).

    I'll bet we'll start haemorrhaging our best answerers, have our unanswered rate skyrocket and eventually kill the site if we start strictly trying to enforce this, because there simply aren't enough questions to limit the number of questions any given person can answer by such a large amount, while still keeping them interested in staying here, and there probably simply aren't enough answerers to prevent such a large portion of people from answering (at least at this stage - it might work if we get to Stack Overflow numbers).

  • Interpersonal skill is about being able to respond well to a new situation or respond well to many situations - this means you can give good advice without having lived through a similar experience.

  • All the experience in the world doesn't mean you actually managed to figure out how to deal well with this situation. Would you assume someone who's gone on 10000 first dates is a good person to ask for first date advice? I wouldn't.

  • Every interaction you've ever had with another person is experience. One can learn things about people based on a set of related or unrelated experiences and translate that to new experience (that's learning in a nutshell).

    Some of these experiences may be too mundane to remember or be noteworthy, other times you may be basing advice on a collection of experiences, which you can't really briefly, or at all, summarise.

  • This disregards a large part of what voting is for. Other experts vote to show how useful an answer is, and may be voting based on their experience. If someone posts an answer, it should be voted for based on how useful the voter believes the advice is, not because of an anecdote that shows that it might work.

    How well does voting actually succeed in this goal? That's a good question, but, even if it fails miserably, that still wouldn't really make this a good alternative.

  • Lying is a thing. If we're forced to back up answers with experiences, what's to stop people from just making up an anecdote so their answer is acceptable (at least in as far as possible)?

    Why would they do this? Well, because they want to contribute here, but, to answer any given question, there's this arbitrary hurdle of needing to have lived through that specific experience or something close enough to it to get over.

    What's to stop lying on other circumstances? Motivation, firstly - there's much less reason to lie in other scenarios (at least in as far as answering on this site goes). Also, other experts - no-one can really say "there's absolutely no way that anecdote is true" (because unbelievably improbable things do occasionally happen). If, on the other hand, you're presenting flawed reasoning, a fabricated reference or things that are simply objectively not true, other experts would easily (or at least reasonably) be able poke holes in that.

  • This is not the way things work on other sites.

    On technical sites, i.e. Stack Overflow, you can be asked for proof (which an anecdote is not) or you can be asked for an explanation, but no-one will ask you to provide an anecdote or convince them why you're qualified to give this answer (or at least such requests are extremely rare... and usually not constructive).

    It also doesn't work this way on Workplace, for that matter (which has been mentioned at some point in this discussion) - no-one expects answers to be backed up with anecdotes there (but explanations - sure).

  • Anecdotes are bad at teaching. If you say "I'm recommending A", adding "because I once tried A and it worked" will not teach the reader anything. That will only allow the reader to handle that specific scenario, not generalise it to other similar scenarios, and possibly totally mess up because they are unable to modify the approach in case something unexpected happens. One needs to understand the underlying reasoning for interpersonal advice to be useful.

16
  • 4
    If people put in enough effort to make up believable stories to support their answer, great? "Anecdotes are bad at teaching", really? What about myths, fables, public speaking 101? If the person with 1000 first dates has bad advice, downvote - but at least you know where they're coming from, unlike the guy writing based on the "logic" of how women (tm) work who neglects to mention that he's never actually been on a date.
    – Em C
    Jun 6, 2018 at 19:14
  • 4
    @EmC The point of an anecdote would be to "prove" that something works (or am I missing the point of all of this?). Making up a story, however believable, does not do that, full stop. The first dates example might've been an intentional example of "more is worse" (as in someone who's been on that many first dates is probably doing something seriously and horribly wrong), but honestly I'd rather take dating advice from someone who's never been on a date but can tell me why the advice would work as opposed to from someone who just says "trust me, I've been on many dates" or "it worked once".
    – NotThatGuy
    Jun 6, 2018 at 19:46
  • 3
    Asking for anecdotes is not intended to get people to say "Trust me", it's to encourage "Here is an example of how this advice can be applied and a potential outcome of that". Yeah we can't tell if people are lying: trolls are always going to be an issue on the internet. We can at least require effort to pass. That'd be an improvement over now, where people throw in their two cents on any old question, even when they don't actually have any experience to know if it'll work in reality. IPS issues have so many moving parts and relationships are not logic puzzles.
    – Em C
    Jun 6, 2018 at 20:01
  • 6
    @EmC Lying is only part of the problem - for our primary method of backing answers up, we're completely unable to tell what's the truth and what's a lie, what's exaggerated, that there weren't any times where it went very differently for the answerer and what's simply an exception to the rule. So yes, we just need to trust that it will work. Everyone on here seems to be pretending that all of psychology is basically just grasping at straws and that we have absolutely no idea how people think, so the best we can do is these very flawed anecdotes, and I honestly just don't understand it.
    – NotThatGuy
    Jun 6, 2018 at 20:48
  • 2
    That sounds like an argument for requiring actual sources, not an argument for allowing answers like "You should do X, because even though I haven't experienced this, it makes sense in my head". I mean I definitely agree with you that anecdotes aren't perfect, but it's at least a slightly higher bar than that.
    – Em C
    Jun 6, 2018 at 20:53
  • 6
    @EmC Sure, scientific sources would probably be best, but actually being able to explain what you're advising seems much closer to that than saying "I have no idea why this works, but it'll probably work again because it worked this one time". Now maybe that would make a good argument for requiring many anecdotes, because then at least it's statistically significant, but that still doesn't address all the problems I mentioned, and we'd be cutting away yet another massive section of possible useful answers, and questions, not to mention that answers would be a mess and enforcement a nightmare
    – NotThatGuy
    Jun 6, 2018 at 21:08
  • 3
    Yes I am hoping for people to actually explain why their experience is valid and lessons learned. And we can always ask for clarification ("how does this relate to OP's situation", "how did you handle xyz risk"). I think Shog put it pretty well - I've seen a lot of "reasoning-based" answers that basically say "People are X and therefore my answer", or worse "it's just common sense".
    – Em C
    Jun 6, 2018 at 21:21
  • 1
    re your edit: being able to extrapolate to new situations is indeed good IPS. We are here to help people develop IPS. So if an answerer can't figure out a way to extrapolate their experience to the OP's situation, then maybe they shouldn't be answering. I really don't see that as a bad thing..
    – Em C
    Jun 6, 2018 at 21:24
  • 2
    @EmC "people are X" and "common sense" are clearly not explanations. We can still require effort to pass and we can still make sure readers are able to understand "where they are coming from". None of this is exclusive to personal experience/references.
    – Jesse
    Jun 6, 2018 at 23:08
  • 1
    @Jesse I agree with you on those not being good explanations, but I have seen multiple users on the main site use that as their justification as to why their answer should remain.
    – Em C
    Jun 6, 2018 at 23:28
  • 1
    @NotThatGuy Anecdotes don't "prove" that something works, but they provide context in which I can judge your advice and relate it to myself. Due to the nature of IPS, almost no two situations are alike and I need to know the context of yours to apply your advice to mine.
    – Magisch
    Jun 7, 2018 at 8:25
  • @Magisch You make a good argument for just killing the site and getting it over with, because what you're describing would work on a discussion forum, not a Q&A where both questions and answers are expected to apply for anyone having a similar question and others vote to indicate how useful an answer is based on their own expertise.
    – NotThatGuy
    Jun 7, 2018 at 11:53
  • 1
    @NotThatGuy With that I mean that in order to serve the function of a stack exchange Q/A, answers need to be somewhat broadly applicable. One way to make that work for incredibly specific situations (like interpersonal relations) is to give a lot of context so readers themselves can judge which parts of an answer can be useful to them and which ones may be colored by bias.
    – Magisch
    Jun 7, 2018 at 11:56
  • 5
    Yup! Backing up can be faked and one experience won't justify the other. So this idea was planned to help but easily could turn to the opposite. I sometimes feel like answering because I claim I could see it either from the OP's view or from the others' view and I could give a hint why "they" might react like this or what I'd like to hear in a situation. But I tend to ignore this site because the number of "I don't know better but I can press the downvote button" people seems to increase and new motivations to vote will make it increase even more. Don't make rules for the sake of making rules.
    – puck
    Jun 8, 2018 at 12:33
  • 3
    ... and exactly this is why I didn't write an answer to this topic. Downvote yourself - not me ;-) (sorry but this is so true)
    – puck
    Jun 8, 2018 at 12:34

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .