Towards Understanding and Defending Against Algorithmically Curated Misinformation

Abstract

Search engines and online social media platforms have become important sources of information for users worldwide. Despite their popularity and ubiquitousness, online platforms are not always trustworthy sources of information. The platforms are driven by black box algorithms that optimize for engagement over the credibility of information. There are increasing concerns that online platforms amplify inaccurate information, making it easily accessible via search results and recommendations. In this thesis, I explore the role of online algorithms in promoting misinformation and design defenses against online misinformation by incorporating human-centered insights from stakeholders such as fact-checking organizations and news agencies. My research recognizes the multifaceted nature of online misinformation and explores the algorithmic, policy, fact-checking, and design aspects of the problem through three distinct research threads. In the first thread of my research, I investigate and audit online platforms such as YouTube and Amazon to understand the role of algorithms driving these platforms in surfacing and amplifying misinformative content to users. Through the audits, I found that performing certain real-world actions on misinformative content (e.g. watching a conspiratorial video on YouTube, or adding a misinformative book to the cart on Amazon) could lead users into problematic echo chambers of misinformation. Additionally, I identified vulnerable user populations who could be targets for specific misinformative topics on online platforms. In the second research thread, I explore ways to support the fact-checking process to combat online misinformation. For this work, I interviewed 14 fact-checking organizations and news agencies across four continents to understand their current fact-checking processes, challenges, and needs. This research establishes fact-checking process as a socio-technical phenomenon, revealing the collaborative efforts of various stakeholder groups and technological infrastructure in facilitating effective fact-checking endeavors. It also highlights the technical, policy, and informational barriers to fact-checking and emphasizes the need for systematic changes in civic, informational, and technological contexts to improve the overall quality of fact-checking. In the final thread of my dissertation research, I collaborated with Pesacheck, Africa's largest indigenous fact-checking organization, to design and develop YouCred---a fact-checking system that enables monitoring of algorithmically driven online platforms for misinformation. To create YouCred, I incorporated insights from previous research threads as well as the expertise and feedback of Pesacheck's fact-checkers throughout the development and design stages. YouCred specifically facilitates misinformation discovery and credibility assessments on the YouTube platform. It automatically generates search queries related to important events and topics of interest to fact-checkers and also offers an intuitive interface for annotating videos for misinformation. Through a nine-month evaluation period at Pesacheck, YouCred demonstrates its practical value and usefulness for fact-checkers, underscoring the importance of ongoing collaboration between fact-checking organizations and technology developers in combating online misinformation.