Friday, January 25, 2019

The DFRLab and Reddit

In November, the Digital Forensic Research Lab published a report titled "RT and R/The_Donald."  It's an examination of how stories from perform in the subreddit /r/The_Donald.  As I read the report, some of the author's statements struck me as amiss, and I want to take a few minutes to articulate what bothered me. 

Nitpick #1:

Early in the article, the authors says:
Over the past year, more than 500 articles from USA Really and the were submitted on Reddit. Only two, however, were upvoted more than 1,000 times, indicating a low uptake of the content shared from the two domains.
That's wrong.  I went to and found seven articles from that received more than 1,000 upvotes:  One, two, three, four, five, six, seven.  This is an example of how data can be different depending on where you look.

Nitpick #2:

Later in the report, the author embeds an image from BuzzSumo, showing the six most-popular articles from the past six months, ranked in terms of total social media engagement.  The gist is that none of the top RT articles had any engagement on Reddit:

What I've been trying to figure out is why the same article is listed five times in a row?  It's the one titled: "Going to the gym is a sign of being gay, says Malaysian newspaper."

For what it's worth, that RT story was posted in multiple subreddits, and it spurred user engagements in /r/Malaysia (40 upvotes. 57 comments), /r/Forsen (27 upvotes, 4 comments), and /r/LGBT (25 upvotes, 1 comment).  I'm assuming the author didn't count those engagements because they occurred back in February, 2017, and she was only searching for results from the past six months.....but that still leaves a puzzling question:  Why does the BuzzSumo image indicate that all of the RT articles about the Malaysian newspaper were printed in October, when a simple Google search shows the article was printed in February?

I created an account on BuzzSumo and tried to replicate the DFRLab's results, to no avail.  I searched for the most-popular English-language articles for the six-month period up to November 6, 2018 (the publication date of the DFRLab's report).  My results showed that five of the top six stories had some engagement on Reddit.  The Malaysian newspaper story didn't appear in my results:

Nitpick #3:

Near the end of the report, the author explains how "if it was not for The_Donald, RT’s articles would fall on deaf ears on Reddit."

This statement ignores the fact that RT articles have been highly upvoted on /r/WorldNews, which is an active subreddit with 20 million subscribers.  If you look at the most upvoted RT articles on Reddit from the past year, the top four submissions were all in /r/WorldNews.  In their report, the DFRLab says the top 20 RT links in The_Donald received a combined 132,000 upvotes.  By comparison, the top 20 RT links on /r/WorldNews from the past year received over 143,000 upvotes.

Like I said earlier, this type of data depends on where you look.  (And I know my time frame isn't the same as the DFRLab's time frame.)  My point is:  It's wrong to suggest that /r/The_Donald is the only place on Reddit where RT articles gain a foothold.

Nitpick #4:

At the end of the report, the author says, "by upvoting RT’s articles thousands of times, The_Donald Redditors made it possible for RT’s content to reach Reddit’s home page, which is the first page users see when they enter the site."

I don't think the DFRLab understands how the "front page" of Reddit works.  For years, the front page consisted of 25 default subreddits.  Later, this was expanded to 50 default subreddits, and /r/The_Donald was never part of the default group.  In February, 2017, the Reddit admins redesigned the front page to display links from a large swathe of "popular" subreddits, and /r/The_Donald was not included as part of the original list, either.  It's possible that /r/The_Donald was included on the "popular" list at some point, but my impression is it never happened.  Thus, I doubt /r/The_Donald has ever pushed an RT article to the front page of Reddit.

Incidentally, I did find an example of RT making the front page on Reddit in 2018, courtesy of /r/WorldNews:


I do agree with the DFRLab's overall thesis: There are lots of RT links being submitted to the /r/The_Donald, and this is concerning in light of the user base's potential susceptibility to Russian propaganda.  That said, I felt the DFRLab's efforts to contextualize their data mostly fell flat.  The report's conclusion mentions how RT is "seeding" it's content in /r/The_Donald, and I would've preferred if that part received more of an explanation.

No comments: