Overview of Methods for Computational Text Analysis to Support the Evaluation of Contributions in Public Participation

In this publication in Digital Government: Research and Practice Julia Romberg and Tobias Escher offer a review of the computational techniques that have been used in order to support the evaluation of contributions in public participation processes. Based on a systematic literature review, they assess their performance and offer future research directions.

Abstract

Public sector institutions that consult citizens to inform decision-making face the challenge of evaluating the contributions made by citizens. This evaluation has important democratic implications but at the same time, consumes substantial human resources. However, until now the use of artificial intelligence such as computer-supported text analysis has remained an under-studied solution to this problem. We identify three generic tasks in the evaluation process that could benefit from natural language processing (NLP). Based on a systematic literature search in two databases on computational linguistics and digital government, we provide a detailed review of existing methods and their performance. While some promising approaches exist, for instance to group data thematically and to detect arguments and opinions, we show that there remain important challenges before these could offer any reliable support in practice. These include the quality of results, the applicability to non-English language corpora and making algorithmic models available to practitioners through software. We discuss a number of avenues that future research should pursue that can ultimately lead to solutions for practice. The most promising of these bring in the expertise of human evaluators, for example through active learning approaches or interactive topic modelling.

Key findings

  • There are a number of tasks in the evaluation processes that could be supported through Natural Language Processing (NLP). Broadly speaking, these are i) detecting (near) duplicates, ii) grouping of contributions by topic and iii) analyzing the individual contributions in depth. Most of the literature in this review focused on the automated recognition and analysis of arguments, one particular aspect of the task of in-depth analysis of contribution.
  • We provide a comprehensive overview of the datasets used as well as the algorithms employed and aim to assess their performance. Generally, despite promising results so far the significant advances of NLP techniques in recent years have barely been exploited in this domain.
  • A particular gap is that few applications exist that would enable practitioners to easily apply NLP to their data and reap the benefits of these methods.
  • The manual labelling efforts required for training machine learning models risk any efficiency gains from automation.
  • We suggest a number of fruitful future research avenues, many of which draw upon the expertise of humans, for example through active learning or interactive topic modelling.

Publication

Romberg, Julia; Escher, Tobias (2023): Making Sense of Citizens’ Input through Artificial Intelligence. In: Digital Government: Research and Practice, Artikel 3603254. DOI: 10.1145/3603254.

2nd workshop for practitioners on automated text analysis for citizen contributions

Part of the efforts of the research group is to develop tools that support the evaluation of citizen contributions from participation processes. On 10 December 2021 the research group hosted a workshop with practitioners (including local planning officials, participation officers and planning experts) to discuss our recent developments, part of which have been published in the Proceedings of the 8th Workshop on Argument Mining.

More information on the insights from the workshop is available in German.

Kick-Off Conference of Junior Research Groups in Bonn

On 9 & 10 March 2020 the kick-off conference of all junior research groups took place that are funded by the BMBF programme on Research for Sustainable Development (FONA). At the moment there are about 20 junior research group that receive funding and that reported during the two-day conference.

As part of the latest funding round of 2019 we presented our group in the form of a presentation and a poster.

For an overview of all junior research group see the BMBF website.

Conference Future City 2019

On December 2nd and 3rd, 2019, the Conference Future City 2019 of the Federal Ministry of Education and Research took place in Münster. The CIMT group – represented by Laura Mark and Katharina Huseljić – presented itself there with a poster.

The main goal of our group was to network with science and practice in order to facilitate future cooperation in the evaluation of participation procedures. In addition, there was a strong exchange with former and currently funded projects from social-ecological research. Among other things, the focus was on inter- and transdisciplinary work in junior research groups.

Additionally, there was room for disciplinary interaction and networking and workshops deepening knowledge in the differing research attempts. There was the opportunity to attend various workshops with and without disciplinary focuses. The members of the junior research group attended events on working with data at local municipality level and on the possibility of bottom up climate protection.