Supporting the Manual Evaluation Process of Citizen’s Contributions Through Natural Language Processing

Doctoral thesis of Julia Romberg

Engaging citizens in decision-making processes is a widely implemented instrument in democracies. On the one hand, such public participation processes serve the goal of achieving a more informed process and thus to potentially improve the process outcome, i.e. resulting policies, through the ideas and suggestions of the citizens. On the other hand, involving the citizenry is an attempt to increase the public acceptance of decisions made.

As public officials try to evaluate the often large quantities of citizen input, they regularly face challenges due to restricted resources (e.g. lack of personnel,time limitations). When it comes to textual contributions, natural language processing (NLP) offers the opportunity to provide automated support for the evaluation, which to date is still carried out mainly manually. Although some research has already been conducted in this area, important questions have so far been insufficiently addressed or have remained completely unanswered. In my thesis, I have focuses on the sub-tasks of thematic structuring and argument analysis of public participation data.

For the thematic structuring of the contributions, I have chosen a supervised learning approach based on classification algorithms and active learning. On the one hand, I have investigated how much manual effort can be reduced by such strategies, using three case studies from German municipalities as examples (for more insights, please refer to this article). On the other hand, I have developed evaluation metrics that reflect the public analysts’ needs when designing topic classification methods with active learning.

In argument mining, on the one hand, I looked at how robustly argument identification and classification methods perform across different participation processes (for more insights, please refer to this article). On the other hand, I focused on the concreteness of arguments. In addition to predicting a three-level concreteness label, I developed a methodology to take into account the subjectivity of concreteness ratings and their impact on the prediction result (for more insights, please refer to this article).