Abstract

This paper contributes to a critical methodological discussion that has direct ramifications for policy studies: how computational methods can be concretely incorporated into existing processes of textual analysis and interpretation without compromising scientific integrity. We focus on the computational method of topic modeling and investigate how it interacts with two larger families of qualitative methods: content and classification methods characterized by interest in words as communication units and discourse and representation methods characterized by interest in the meaning of communicative acts. Based on analysis of recent academic publications that have used topic modeling for textual analysis, our findings show that different mixed‐method research designs are appropriate when combining topic modeling with the two groups of methods. Our main concluding argument is that topic modeling enables scholars to apply policy theories and concepts to much larger sets of data. That said, the use of computational methods requires genuine understanding of these techniques to obtain substantially meaningful results. We encourage policy scholars to reflect carefully on methodological issues, and offer a simple heuristic to help identify and address critical points when designing a study using topic modeling.
Original languageEnglish
JournalPolicy Studies Journal
ISSN0190-292X
DOIs
Publication statusE-pub ahead of print - 19 Jun 2019
MoE publication typeA1 Journal article-refereed

Fields of Science

  • 113 Computer and information sciences
  • 517 Political science

Cite this

@article{51e0007a31654c779b04ca374a377c11,
title = "Topic Modeling and Text Analysis for Qualitative Policy Research",
abstract = "This paper contributes to a critical methodological discussion that has direct ramifications for policy studies: how computational methods can be concretely incorporated into existing processes of textual analysis and interpretation without compromising scientific integrity. We focus on the computational method of topic modeling and investigate how it interacts with two larger families of qualitative methods: content and classification methods characterized by interest in words as communication units and discourse and representation methods characterized by interest in the meaning of communicative acts. Based on analysis of recent academic publications that have used topic modeling for textual analysis, our findings show that different mixed‐method research designs are appropriate when combining topic modeling with the two groups of methods. Our main concluding argument is that topic modeling enables scholars to apply policy theories and concepts to much larger sets of data. That said, the use of computational methods requires genuine understanding of these techniques to obtain substantially meaningful results. We encourage policy scholars to reflect carefully on methodological issues, and offer a simple heuristic to help identify and address critical points when designing a study using topic modeling.",
keywords = "113 Computer and information sciences, 517 Political science",
author = "Karoliina Isoaho and Daria Gritsenko and Eetu M{\"a}kel{\"a}",
year = "2019",
month = "6",
day = "19",
doi = "10.1111/psj.12343",
language = "English",
journal = "Policy Studies Journal",
issn = "0190-292X",
publisher = "Wiley Blackwell",

}

Topic Modeling and Text Analysis for Qualitative Policy Research. / Isoaho, Karoliina; Gritsenko, Daria; Mäkelä, Eetu.

In: Policy Studies Journal, 19.06.2019.

Research output: Contribution to journalArticleScientificpeer-review

TY - JOUR

T1 - Topic Modeling and Text Analysis for Qualitative Policy Research

AU - Isoaho, Karoliina

AU - Gritsenko, Daria

AU - Mäkelä, Eetu

PY - 2019/6/19

Y1 - 2019/6/19

N2 - This paper contributes to a critical methodological discussion that has direct ramifications for policy studies: how computational methods can be concretely incorporated into existing processes of textual analysis and interpretation without compromising scientific integrity. We focus on the computational method of topic modeling and investigate how it interacts with two larger families of qualitative methods: content and classification methods characterized by interest in words as communication units and discourse and representation methods characterized by interest in the meaning of communicative acts. Based on analysis of recent academic publications that have used topic modeling for textual analysis, our findings show that different mixed‐method research designs are appropriate when combining topic modeling with the two groups of methods. Our main concluding argument is that topic modeling enables scholars to apply policy theories and concepts to much larger sets of data. That said, the use of computational methods requires genuine understanding of these techniques to obtain substantially meaningful results. We encourage policy scholars to reflect carefully on methodological issues, and offer a simple heuristic to help identify and address critical points when designing a study using topic modeling.

AB - This paper contributes to a critical methodological discussion that has direct ramifications for policy studies: how computational methods can be concretely incorporated into existing processes of textual analysis and interpretation without compromising scientific integrity. We focus on the computational method of topic modeling and investigate how it interacts with two larger families of qualitative methods: content and classification methods characterized by interest in words as communication units and discourse and representation methods characterized by interest in the meaning of communicative acts. Based on analysis of recent academic publications that have used topic modeling for textual analysis, our findings show that different mixed‐method research designs are appropriate when combining topic modeling with the two groups of methods. Our main concluding argument is that topic modeling enables scholars to apply policy theories and concepts to much larger sets of data. That said, the use of computational methods requires genuine understanding of these techniques to obtain substantially meaningful results. We encourage policy scholars to reflect carefully on methodological issues, and offer a simple heuristic to help identify and address critical points when designing a study using topic modeling.

KW - 113 Computer and information sciences

KW - 517 Political science

U2 - 10.1111/psj.12343

DO - 10.1111/psj.12343

M3 - Article

JO - Policy Studies Journal

JF - Policy Studies Journal

SN - 0190-292X

ER -