Why doesn't this site have a back it up rule?

Question

I'm looking at the answers on this site. I've found that very few of them are supported by any sort of evidence. For example, in the question How to discourage missionaries?, the top voted answer doesn't back any of its claims up:

Have a no cold caller sign. While yes it might irritate them, as long as you're not there for them to express their irritation on, you should be fine.

If you know it's them you can simply ignore the door, and not open it, you can burn (or recycle) anything they put through your door.

Finally, if they still don't get the message, write a note on the door, or perhaps even leave a polite letter for them, informing them that you do not appreciate what they do and have no interest in following their religion, however should you have a change of heart you know where to go.

If they end up being a large nuisance, simply call the police to remove them from your property for trespassing.

How should I know that any of these suggestions will work? Some of these suggestions seem like jokes: for example, why on earth would it be a good idea for me to "burn... anything they put through your door"?

As it stands right now, that question is a polling question. Various solutions are proposed in answers, but with no explanation as to why any solution is a good one. The community upvotes the suggestions that appeal to them personally, and downvotes the ones that they don't like. But fortunately or unfortunately, the Stack Exchange platform is not designed for polling questions.

I would like to suggest that this site establish a back-it-up rule, as described on the blog post Good Subjective, Bad Subjective:

The folks at Moms4mom owned up to the subjective issue and came up with a set of principles to create useful subjective discussions on parenting: the Back It Up! Principle. Back It Up! means that your answers must be based on either:

• Something that happened to you personally
• Something you can back up with a reference

If you're interested in seeing how this would work in practice, I highly recommend taking a look at the RPG Stack Exchange, which is a site that frequently answers questions about interpersonal issues related to role playing games.

Answer 1

TL;DR

Yes, we should have a "back it up rule". It should be added to our help center.

However, we need to be very careful about how we "enforce" this rule, because good advice on interpersonal skills can be based upon an amalgam of personal experiences built up over time, and doesn't require specific parallels to what the OP is experiencing in order to be valid, high quality advice.

This rule should be used as guidance to help improve low-quality answers, and not a strict yardstick that should be applied universally.

Long Version

On the Parenting.se, there was a similar "back it up" rule, based on the blog you referenced. The rule is part of the custom content for the site's help center:

Please note that opinions shared here should be backed up either with a reference, or experiences that happened to you personally. Also, posts that primarily exist to push a specific agenda (propaganda), and soap-boxing, are not welcome.

If we want a rule like this, I suggest placing it on our help center, which is currently nearly empty.

Having moderated for a while on the Parenting site, I can speak to how that policy worked for us.

In practice, that policy was primarily there as an "official" policy that we could point to in order to help provide guidance for low-quality answers. It was not something that was expected for 100% of our answers.

It is unrealistic to expect 100% of answers to explicitly spell out what personal experiences the author is drawing upon to provide their advice. Interpersonal skill, like parenting, is built upon years and years of personal experiences, and it is impossible to distinguish good advice based upon this personal body of experience and "common sense".

If my child never threw a tantrum on the floor of a restaurant, am I qualified to give advice to someone whose child has? I've never experienced that, but I would argue that I have experience in avoiding this situation.

If the only people who can provide advice to an answer are people who have personally experienced a very similar situation, this site will not succeed. Worse, that's not a particularly enforceable position to take. Do moderators spend all of their time deleting answers (which may be highly upvoted by people who aren't familiar with our policy) that don't spell out the author's personal experience? Do we diligently copy and paste a standard comment onto every answer that doesn't spell out how their personal experience applies (which, as I mentioned above, is not always a realistic request to begin with)?

Attempting to do these things will accomplish little beyond discouraging new members of our community.

If an answer calls out directly relevant personal experience, or reference materials, those are signs that the answer likely deserves an upvote. If the answer instead provides what seems like sensible advice, backed with an explanation of why that advice seems applicable to the situation, fully answers the question, but doesn't explicitly state what parallel experiences happened to the author that they are basing their advice on, that's fine, too.

If instead, an answer is brief, low quality, and doesn't seem to stand on its own, it is appropriate to downvote and add a comment saying "answers on this site should be backed up with either a reference, or experiences that happened to you personally", with a link to (our hopefully updated) help center.

Answer 2

To a greater extent than most SE sites, this is an "action" (what to do) site, not a "knowledge site. Therefore. references to academic knowledge or links to wikipedia are less helpful here than on other sites.

There should be a "back it up" rule, but in most cases, this "back up" is reasonably based on personal experience. For instance, "I was bullied as a child, and have found that one way to deal with bullying is..." is a more helpful answer on this site than on most others. On this site, "My mother/father/older sibling told me..." is more helpful than on most others. And, of course, many other users have had similar experiences, and are in a better position to judge the merits of an answer based on experience or common sense.

If a person has no experience in an area, then the more "academic" type of back up is necessary.

Answer 3

Tl/Dr: A back-up rule for the site would be detrimental. It's negative reinforcement, rather than positive reinforcement, and other stack exchanges have show alternatives.

We dealt with the subjectivity problem during the beta for WorldBuilding.SE. What makes the subjectivity problem tricky is that everyone has a different opinion about how subjective a question or how justified an answer should be. Some people want fully backed up and justified answers with links. Others just want common sense or logic. Others simply want the answers. For an example of how extreme these differences are, look at Mathematics.SE vs MathOverflow. A good question on Mathematics.SE could get brutally downvoted and closed on MathOverflow because their target demographics are very different, leading them come to a different line between what is a good question/answer and what is not.

On WorldBuilding, this was a rather unique problem because every question is asking for a part of an answer. You can't ask WB to write your novel for you, so you always have to ask for pieces, and then incorporate those pieces into your novel. I think IPS operates the same way. Everyone is writing questions containing a piece of their life, and must re-integrate those answers into their life before determining if they truly have merit or not.

Because these questions are asking for pieces, there really is no right or wrong answer. This isn't Physics.SE, where you can cite wikipedia's answer, or do some math. The answers aren't complete until they are integrated into the story. IPS is the same way. An answer isn't an answer until you've integrated it into your life and found that it fits correctly.

This downplays the value of backing up the answers. In many cases, there is no official answer to these questions, so all we're doing is providing subjective evidence that the answer is good.

What constitutes backing up information anyways? When it comes to Physics.SE, it's easy because there is a well understood curriculum. For IPS, the schoolroom is life. We're learning every day, and we're learning in different ways. If I had to justify many of my answers, they would be using the language of Rational Actor Theory and Chinese internal martial arts, because those are the most prominent ways I view interpersonal relationships. My justifications may have very little value to you, unless also think in these terms (which I assume you do not). Whatever justification scheme you use, I guarantee you the OP uses a different one! Indeed, those who want a "suggestions" site have a justification scheme of their own -- may the most upvoted question win! These justifications clearly have some value, but its hard to come to a consensus on how valuable they are compared to the answers themselves.

On WorldBuilding, we found that people don't all agree on how much backing up is "good." Sometimes, people are just looking for quick answers which they can then integrate. Sometimes, people are looking for detailed proofs that the answer is good. I think IPS is similar. Some people are looking to use this site more as a "suggestions site" some are looking to use it as a "expert system" which links up expert information.

The solution on WorldBuilding was to rely on tags. An OP could tag a question to indicate what level of backing up of the answer they wanted to see. Tags which have had an effect on this are:

• Science-based - Everything should be backed up to science, but it can be a loose connection
• Hard-science - An acceptable answer must reference their sources, and typically rely on mathematical calculations.

Then we let the community go wild with them, and collected statistics. What we found is that the community really liked the rules of Science-based questions. It's the most popular tag on the site. The community of people who ask and answer science-based questions have evolved towards a balance of their own. If you try to be too loose with a science-based answer, you get called on it. Meanwhile, those who want hard-science can get it.

As a general rule in interpersonal skills, it's more helpful in the long run to use positive reinforcement rather than negative reinforcement.

"Our research found that negative reinforcement is actually far more effective for sparking initial habit change.... But here’s where things get interesting: If you were continually penalized ..., the negative reinforcement would eventually stop working. You’d become resentful of the constant punishment." - Pavlok

Rules are universally backed up with negative reinforcement (I can't think of a counter example. Can anyone think of one?). What we want is to raise the health of IPS in the long run, and that's best done with positive reinforcement. Tagging the level of backup expected gives us handholds with which to provide that positive reinforcement while simultaneously listening to the community and their desires.

Answer 4

1. A back it up rule is only useful if it is enforced by the community.

If the community is going to continue to answer with opinions, thoughts, and suggestions, and continue to reward these answers with votes, then the policy is irrelevant.

2. Questions on how to make people stop doing something that bothers you are never going to get good answers.

You can not make someone else change their behavior with a simple answer. Changing other peoples behaviors takes time, conditioning, and a desire of the other person to change. You can not explain how to do that in an answer. Throw in the ubiquitous "Politely" and it's even more difficult.

3. The problem here is that the scope for allowed questions is FAR TOO BROAD!

The scope of "Interacting with another person" includes practically 100% of all SE's that are not pure technology based. There is a reason these sites have defined scopes, because if the scope is too wide you end up with a mess of questions; no or few real experts to help answer the good questions; and to many garbage questions to sift through looking to find a good one.

People ask questions here, then get 10 mediocre answers that get highly voted because they have suggestions that the community likes, but the mass volume makes it really difficult for the asker to find the really good answers. Combined with the fact that at least one of them probably hit on their bias so it gets selected as the "Correct" answer even if the help was not actually helpful.

Until the question scope is reduced so that the signal to noise ratio is much better, you are going to continue to see garbage answers. Garbage in => Garbage out!

Answer 5

The problem that this site currently has is that the answers are mostly suggestions. What this means is that answers will simply suggest suggestions to the problem, but there is no way for anyone reading the answer to know if the suggestion will work unless they actually try it. This leads to problems; if someone uses a suggestion found on this site and the suggestion didn't work, well, they're just one more person who's been tricked by the internet.

The mission of Stack Exchange is to make the internet a better place. (Or rather, this should be the mission of Stack Exchange; whether that actually happens is a different question.) If the only thing that answers on this site do is spam suggestions that may or may not be right, well, then the Interpersonal Skills Stack Exchange will be making the internet a worse place, not a better one.

So how can this problem be solved. How can this community teach its members to write answers that aren't suggestions but rather solutions? How can this community write answers that are helpful and that people can trust?

In most cases, answers on this site will be written from experience. This makes sense; there aren't academic sources for basic social skills. But it's not easy to write a good experience-based answers. Many people believe that an experience-based answer simply means changing "do x" to "From my experience, doing x is best". But if you think about it, there's no difference between the two quotes. Both are ultimately suggestions; the difference is the second quote attempts to gain the reader's trust by claiming an identity. Since anyone can lie on the internet without repercussion, it's a poor attempt indeed.

After some thought, some frustration, and a lot of conversation on chat, I've come up with a way of thinking about the issue that is helpful and easy to understand.

answer = information + argument

A good answer has two components: information and an argument. Both components are crucial to ensuring that an answer is more than a list of suggestions. To explain why, let me go through each component one by one.

What information should answers include?

When someone receives advice, the first thing they need to know is whether the advice is applicable to them. "Divorce your spouse" might be good advice if you are in an abusive relationship, and it might be bad advice if you're having an argument about what tv show to watch. As has been discussed elsewhere on this site, good advice for one culture might be bad advice for another culture. When someone reads an answer, they need to be able to know that the answer applies to their situation.

To fix this, provide information about the specific context your experience comes from. For example, in a question about retail etiquette, including a statement such as "as someone who has both worked in retail and enjoys shopping, I can understand both sides" contains enough information that a reader will be able to tell whether the subsequent advice will apply to them.

It's important to understand that since this is the internet, it's impossible to verify whether the information is true. So the only critique one can realistically make of information is that an answer doesn't have enough of it. But trust is not the function of the information portion of an answer. Trust is where the second portion of the answer, the argument, comes into play.

Arguments transform information into solutions people can trust.

So once a reader knows that an answer is applicable, how does a reader know that the ultimate conclusion of the answer (e.g. "do [x]") is a good one? The answer: it's the job of the answer to convince the reader that the answer's advice is good. How do you convince someone? You give an argument.

Here's what a good argument might look like: "I was in an abusive relationship. Nothing I did improved things. I later found out from a psychologist that this is normal. Meeting with the psychologist helped give me courage to walk away. The very act of talking about the issue with someone face to face helped my organize my thoughts and feelings. I suggest that you meet with a psychologist, or someone who is a professional who can offer advice." Of course, this argument is missing the first part of the equation: information.

Frequently Asked Questions

Do we need a consensus to enact this as a policy?

You don't need a consensus to downvote and/or leave constructive comments. Don't underestimate the power of downvotes and comments to change people's behavior! At the same time, downvoting and leaving constructive comments are perhaps the only way to build grassroots support for establishing a consensus about requirements for answers. So the answer to this question is: downvote and leave constructive comments!

Here are some examples of constructive comments:

What country is your answer applicable to? I ask because different countries have very different social conventions.

Answers shouldn't just list suggestions, but should explain why each suggestion would be beneficial to the reader. If I use your suggestions, how can I be sure that they will make my situation better?

There's no need to worry about answer quality; the votes take care of that on their own

I'm going to redirect you to this answer by Shog, which does an excellent job of breaking down some of the philosophy of Stack Exchange with regard to voting, answer quality, and closing questions.

Where can I read more about this?

• The blog post Good Subjective, Bad Subjective

• This excellent answer on the RPG meta about how to write an experience based answer.

• This excellent answer about why a back it up policy is essential.