3.9 out of 5 stars

By Steve Outing

That’s how my latest Editor & Publisher Online column is rated by 11 reviewers (as I write this) on Newstrust. The column is about the larger issue of the proliferation of alternative news sources online (blogs, grassroots media, et al), and the lack of any competent guidance in telling consumers whether those sources are credible or not — and it discusses the NewsTrust non-profit initiative as one specific approach to alleviating the problem.

I don’t feel like it was a brilliant column by any means, so 3.9 ain’t such a bad score. Editor & Publisher itself is rated as an overall source as 3.7 currently on NewsTrust, so I guess I’m a bit ahead of the game. Still, the score feels a little like getting a “B” on a writing assignment in college. :)

You can read the column yourself if you want to learn more about the online source-credibility issue and NewsTrust in particular.

But seeing NewsTrust in action and my own work reviewed, a few other thoughts are worth dissecting here (and weren’t covered in my column):

1. Journalists will need to get used to this public grading. Sure, we all get reader feedback: letters to the editor, private e-mail praise and rants, bloggers opining on the quality and/or content of our reporting, etc. But this statistical public assessment of what we write is another thing altogether. I do think that it will allow journalists willing to take in criticism to learn how to do better. It’s a good thing, but it could make some journalists uncomfortable.

2. I suspect that NewsTrust will turn into a useful hiring tool. The reporter candidate who talks a good game but whose NewsTrust-ranked stories consistently get ranked in the 2 out of 5 range may have trouble getting hired. Handy tool for hiring editors. Maybe not so great for average reporters who want to find a new job.

3. If NewsTrust does have an effect on individual careers, I can envision some journalists trying to game the system.

Author: Steve Outing Steve Outing is a Boulder, Colorado-based media futurist, digital-news innovator, consultant, journalist, and educator. ... Need assistance with media-company future strategy? Get in touch with Steve!

3 Responses to "3.9 out of 5 stars"

  1. Fabrice Florin
    Fabrice Florin 9 years ago .Reply

    Steve, 3.9 is a really good rating for NewsTrust, so pat yourself on the back for a job well done!

    I wish there was a way to filter the news without having to rate it for quality, but it’s the most reliable method we’ve found so far – computers need numeric data to sort with, and the popularity measurements used by Google and Digg are simply not effective indicators of quality in their current form.

    I understand how awkward it must feel to have your work evaluated in this way, but note that we try really hard to get our reviewers to rate the actual work product, not the individual. And journalists are welcome to rate our reviews as well, which can dramatically affect our own member levels. You can even rate your own work, as long as you disclose in your comments that you are the author.

    The intent is to keep everyone honest by making everything transparent – including editorial filtering processes that used to take place behind closed doors. So far, most journalists we’ve talked to about this project are supportive of our overall mission, but some get a bit nervous when their work gets evaluated by the public. I hope that over time, this will become less of an issue, as long as we’re careful in designing systems that reward fairness and respect from our reviewers.

    Thanks again for your outstanding coverage of our initiative — and for participating in our experiment.

    All the best,

    Fabrice Florin
    Executive Director, NewsTrust

  2. Fabrice Florin
    Fabrice Florin 9 years ago .Reply

    Steve, 3.9 is a really good rating for NewsTrust, so pat yourself on the back for a job well done! I wish there was a way to filter the news without having to rate it for quality, but it's the most reliable method we've found so far – computers need numeric data to sort with, and the popularity measurements used by Google and Digg are simply not effective indicators of quality in their current form. I understand how awkward it must feel to have your work evaluated in this way, but note that we try really hard to get our reviewers to rate the actual work product, not the individual. And journalists are welcome to rate our reviews as well, which can dramatically affect our own member levels. You can even rate your own work, as long as you disclose in your comments that you are the author. The intent is to keep everyone honest by making everything transparent – including editorial filtering processes that used to take place behind closed doors. So far, most journalists we've talked to about this project are supportive of our overall mission, but some get a bit nervous when their work gets evaluated by the public. I hope that over time, this will become less of an issue, as long as we're careful in designing systems that reward fairness and respect from our reviewers. Thanks again for your outstanding coverage of our initiative — and for participating in our experiment. All the best, Fabrice Florin Executive Director, NewsTrust

  3. […] Steve Outing » 3.9 out of 5 stars Newstrust: “1. Journalists will need to get used to this public grading. 2. I suspect that NewsTrust will turn into a useful hiring tool. 3. If NewsTrust does affect individual careers, I can envision some journalists trying to game the system.” (tags: services reviews statistics credibility ethics journalism career news+biz tidbits+fodder) […]

Leave your comment