Collection, Management, and Analysis of Twitter Data: Using the Twitter API for Academic Research and BERT

📍 virtual MZES, Mannheim

📆 May 04, 2022

As a highly relevant platform for political as well as social online interactions, social scientists increasingly analyze Twitter data. As of 01/2021, Twitter renewed its API, which now includes access to the full history of tweets for academic usage. In this talk, I will first present a detailed walkthrough of the data collection process, from applying to access to storing the data. Following a brief discussion of data processing routines, I then introduce an application from my own methodological research that uses textual contents of tweets from German members of parliament. It combines the state-of-the-art NLP method BERT with hierarchical shrinkage estimators to obtain legislator-level salience and position metrics for specific policy domains and sub-domains.

📝 Slides

👤 Andreas Küpfer is a graduate of the Mannheim Master in Data Science and an incoming doctoral researcher at the Technical University of Darmstadt. His interdisciplinary research interests include text as data, applying machine learning technologies, and substantial inference in the fields of political communication and political competition.