We've spent the last few weeks talking about what it takes to make an answer that meets the site's guidelines for quality and I think that these discussions are valuable and will help us decide where to go when we're ready to make the next step forward in setting expectations for answer quality... but I'm not sure that the wider community - the users not active on IPS Meta but active in the review queues - share the opinions of the users who are... and it seems that even the users who are active here on Meta aren't necessarily of the same opinions.
So... here's what I'm thinking... this comment says:
Believe it or not, that's already happening @Beofett - for the past 6 months, a majority of answers here have been deleted by vote, not by their authors and not by moderators. Similarly, low-quality review handles over 70% of VLQ and NAA flags. This wouldn't have had a chance of working back when we first discussed it; now it actually might: the greater community of IPS has the ability to do this sort of moderation, they just need to decide that they want to. Shog9♦
Because of this comment and some other discussion, it looked like there would be general support for creating a post notice that moderators could use to indicate when an answer failed to meet our back it up guidelines, so I requested it and now, a couple of weeks later it's been mentioned that we're not actually ready for this since the site isn't enforcing the policy.
Yeah, there's a reason I haven't commented on that, @Catija; a post notice should follow from what's working on the site already, in spirit if not in word. And we're a ways away from that. Shog9♦
I admit that I pretty much agree with this and that's without actually looking at any data. We have a queue that sees many NAA flags that are disputed by the community and later flagged again. Since NAA and VLQ flags only go through the LQP queue once (if that - accepted answers never go through it at all), it falls on the moderators to determine whether an answer is "sufficient" or not.
And many of these flags sort of sit around for a long time because, as Shog says, "we're a ways away" from knowing what the community wants us to delete as insufficiently supported.
What numbers can we get that will compare the quantity of answers that are flagged as VLQ or NAA that are:
- Deleted by non-moderators
- Deleted by moderators
- Disputed (and never reflagged)
- Disputed and reflagged and deleted
- Disputed and reflagged but declined
There may be other things that are relevant that I'm not thinking of - deleted by users/moderators but never flagged, perhaps? Score of the deleted/disputed/declined posts?
I'm also not quite sure how to use the data after we have it - what does it tell us? How are we trending? Is there a way we can sort through the data to determine why an answer was deleted and whether it failed to meet our guidelines or something else.
This all sounds really complex but maybe I'm making it more complicated than it needs to be... so, maybe we should start simple and then look deeper after a bit.