Tag: publication

The political shaping of the sustainability transformation: participatory and ecological?

In this article in regierungsforschung.de, the academic online magazine of the NRW School of Governance, Tobias Escher explores the question of whether the participation formats required for a participatory development of the transformation to a more sustainable society can also be implemented in an ecologically sustainable manner. In other words, the question is how the principle of (ecologically) sustainable governance can be upheld in the democratically important area of political participation.

Abstract

The transformation to a sustainable society can only succeed through the involvement of all stakeholders. At the same time, sustainable governance requires that these participation processes also fulfil the requirements of sustainability. Based on a systematic discussion of the greenhouse gas emissions generated by various political participation formats, this article argues that in view of the functional and normative significance of political participation, its environmental footprint does usually not contradict the demand for sustainable governance.

Key findings

  • The implementation of participation formats generates greenhouse gas emissions due to the energy required, the mobility of the participants and, where applicable, catering and accommodation. Crucially, the level of emissions depends on whether the formats are held in person or digitally (see table below).
  • For local face-to-face formats, the emissions from the event venue and mobility of participants are comparatively low and only amount to around 2kg of CO2-equivalents per person in the scenario analysed. The emissions increase considerably as soon as participants have to travel further (using fossil-fuelled transport) – in the underlying scenario by a factor of twelve to 24kg. This is roughly equivalent to the average emissions from a 100km car journey.
  • In contrast, online participation formats only generate very low emissions (around 0.3kg per person in the scenario). Nevertheless, the choice of participation formats cannot be based solely on their short-term ecological costs, but must also take into account normative requirements for inclusion and popular control. Digital participation formats in particular represent significant barriers to participation for already disadvantaged population groups (see also our DigiBeSt-report).
  • Overall, participation has an ecological footprint, but this is generally justified by functional and normative considerations because after all, governance should not only be sustainable, but above all democratic!
face-to-face participationdigital participation
local(inter)-regionalasynchronsynchron
energyheating & climatisation of event venueoperation of end user devices & data centresmore data intensive communication
mobilitytraveltravelnot applicable
catering & accommodationcateringaccommodationnot applicable
Direct environmental impact of different participation formats. The darker the colour, the greater the respective emissions from this source. The emissions generally increase with the number of participants and the duration of the participation format.

Publication

Escher, Tobias (2024): Die politische Gestaltung der Nachhaltigkeitstransformation: partizipativ und ökologisch? Essay. In: Regierungsforschung.de (15. Jahrgang). https://regierungsforschung.de/die-politische-gestaltung-der-nachhaltigkeitstransformation-partizipativ-und-oekologisch/

Is that fair? An evaluation tool for local mobility measures

In this article in the journal Internationales Verkehrswesen, Laura Mark, Annika Busch-Geertsema, Jessica LeBris, Gesa Matthes and Kerstin Stark present a practical approach to assess the justice of transport measures in various dimensions. The approach and the article were developed in the context of the working group “Mobilität, Erreichbarkeit und soziale Teilhabe” of the Academy for Spatial Development in the Leibniz Association (ARL).

Summary

This paper presents a practical tool for taking a systematic “second look” at mobility transition measures through the lens of justice. It can be used for conceptual support during planning and implementation, for reflection during or after the process as well as ongoing monitoring. Three dimensions of justice can be used to analyse which population-groups benefit from these measures. The dimensions used are distributive justice, recognition of different realities of life and procedural justice, which have been further differentiated and combined in an easy-to-use matrix.

To differentiate the recognition of different realities of life, the Persona approach is employed. In a Persona, specific characteristics are combined that can influence and restrict mobility options, decisions, and activity chains. The article proposes a system for developing custom Personas, but the Personas already developed by the authors can also be used for the application.

The article provides a detailed presentation of the dimensions of justice and the Persona approach. It can be downloaded here:

IV Tool for justiceHerunterladen

Furthermore, the evaluation tool, including a user manual and suggestions for Personas, can also be downloaded in Excel format from the ARL Mobility Working Group website (scroll down). The authors welcome feedback and encourage free use and further development.

Annotation and Provision of Datasets

As part of our project, we worked on the manual annotation of a large number of datasets with the aim of supporting the development of AI methods for evaluating public participation contributions.

Supervised machine learning requires training datasets in order to learn patterns related to the respective codings. In the area of citizen participation, there is a lack of comprehensively coded German-language datasets. In order to meet this need, we have therefore worked on annotating German-language participation processes from the field of mobility according to four dimensions:

  • Firstly, we have thematically classified contributions according to modes of transportation, other requirements for public space, and defects that need to be fixed immediately.
  • Second, we coded processes by argumentative sentences and divided them into premises and conclusions.
  • Thirdly, we have assigned argumentative units of meaning to how concrete they are.
  • Fourthly, we have coded textual location information.

A more detailed description of the datasets – as of June 2022 – can be found in our publication: Romberg, Julia; Mark, Laura; Escher, Tobias (2022, June). A Corpus of German Citizen Contributions in Mobility Planning: Supporting Evaluation Through Multidimensional Classification. Since then, we have continued to work on the thematic coding of the datasets and revised our scheme of modes of transport.

The following table shows the current status of annotation and is updated on an ongoing basis (in German): Google-Sheet

In accordance with our open source policy, the annotated datasets are made available to the public under Creative Commons CC BY-SA License when possible.

A number of publications have been produced based on these data sets. These can be found at https://www.cimt-hhu.de/gruppe/romberg/romberg-veroeffentlichungen/.

Effects of online citizen participation on legitimacy beliefs

In this article in the journal Policy & Internet, Tobias Escher and Bastian Rottinghaus explore the question of how participation in local consultation processes (on planning of cycling infrastructure) affects attitudes towards local politics. To this end, in 2018 they examined a total of three participation procedures in which the cities of Bonn, Cologne (district Ehrenfeld) and Moers consulted their citizens on local cycling infrastructure. In each case, for five weeks citizens were able to submit, comment on and evaluate proposals through an online platform. In total, more than 3,000 proposals were collected which were to be incorporated into the subsequent cycling planning (see further information on the Cycling Dialogues project).

Abstract

In order to generate legitimacy for policies and political institutions, governments regularly involve citizens in the decision-making process, increasingly so via the Internet. This research investigates if online participation does indeed impact positively on legitimacy beliefs of those citizens engaging with the process, and which particular aspects of the participation process, the individual participants and the local context contribute to these changes. Our surveys of participants in almost identical online consultations in three German municipalities show that the participation process and its expected results have a sizeable effect on satisfaction with local political authorities and local regime performance. While most participants report at least slightly more positive perceptions that are mainly output-oriented, for some engagement with the process leads not to more, but in fact to less legitimacy. We find this to be the case both for those participants who remain silent and for those who participate intensively. Our results also confirm the important role of existing individual resources and context-related attitudes such as trust in and satisfaction with local (not national) politics. Finally, our analysis shows that online participation is able to enable constructive discussion, deliver useful results and attract people who would not have participated offline to engage.

Key findings

  • The participation processes we studied and to which citizens were invited by their respective councils do indeed have an influence on the attitudes of those who participate in such consultations.
  • For many of the participants, the positive effect that was hoped for does indeed occur: they are more positive about the local institutions (mayor, administration) and local politics as a whole. The decisive factor for the assessment is whether one expects local politics to take the citizens’ proposals seriously and act upon them. In other words, the result of the process is more important for attitudes than the process itself.
  • It is noteworthy that this holds true also for those who have rather negative views of local politics to begin with. However, previous experience with local politics also plays a role: those who already have a higher level of satisfaction and trust in the municipality are becoming more positive by participation.
  • At the same time, participation can also lead to less satisfaction. We were able to show this, on the one hand, for those who were intensively involved in the participation process and made a lot of proposals. On average, this group was less satisfied in the end, probably because their expectations of the impact of their efforts were disappointed. Those who did not actively participate but only visited the online procedure without making suggestions themselves were also more dissatisfied. These people were apparently mainly concerned about the fact that the process took place exclusively online.
  • Overall, however, our results show that such online participation processes not only enable constructive participation, but that they also reach additional groups: Almost half of the respondents would not have participated if the process had only been conducted with on-site formats requiring physical presence.

Publication

Escher, Tobias; Rottinghaus, Bastian (2023): Effects of online citizen participation on legitimacy beliefs in local government. Evidence from a comparative study of online participation platforms in three German municipalities. In: Policy & Internet, Artikel poi3.371. DOI: 10.1002/poi3.371.

Consultation of citizens in mobility projects: The participation landscape at municipal level in Germany

In this article in the journal Raumforschung und Raumordnung, Laura Mark, Katharina Holec and Tobias Escher present the results of a survey on the scope and organisation of consultation in mobility-related municipal planning. From these results, conclusions can be drawn about the participation landscape in Germany.

The results were presented in an earlier version at the 18th annual conference of the Mobility and Transport Working Group (AK MoVe) in June 2023.

Abstract

Municipalities as key actors in the transport transition are increasingly using consultative public participation in planning. So far, however, it is unclear to what extent they use participatory processes in mobility-related planning and how these are designed. Given the challenges associated with the transition to a climate-neutral transport system, taking stock of existing efforts is highly relevant in order to assess the practical significance of participation processes and to better investigate the role of different types of procedures and contexts.

This study fills this gap based on an analysis of the consultative, discursive participation processes for mobility-related planning in German cities since 2015. The study examined ‘participation-oriented’ cities with guidelines for citizen participation, which were compared to a random selection of ‘typical’ municipalities in North Rhine-Westphalia, Baden-Württemberg and Saxony as well as the three German city states.

Based on these approximately 180 cities and 350 procedures, it becomes clear that discursive consultations are carried out regularly, in particular in municipalities with guidelines and larger cities. Worth criticizing is that the formats used can usually reach only certain groups of the population and that for a significant proportion of the processes examined no information on the results of participation can be found. This means that the potentials of discursive citizen participation in addressing the municipal transport transition have not yet been sufficiently utilised.

Key Findings

  • Participation in municipal planning procedures related to mobility is no longer an exception, but not yet the rule either. Based on the data of our sample, it can be presumed that in most municipalities in Germany there was no possibility to participate in such procedures in the period under consideration.
  • In general, cities with guidelines involved their citizens more frequently, more often and with more diverse topics and formats. Medium-sized and large cities consulted their citizens significantly more often than small towns.
  • Weaknesses are evident in the participation formats used: The majority of municipalities relied on self-selected selection processes. First attempts with target group-specific formats or random selection can be found mainly in the municipalities with guidelines and in the city states. A large proportion of the procedures were also carried out purely online.
  • For 5 to 10% of the procedures, no current status could be found, and for a larger proportion it was unclear what happened after the consultation. This is true for all municipalities, although less so for those with guidelines, and can be regarded as a lack of transparency and impact of participation.

Publication

Mark, Laura; Holec, Katharina; Escher, Tobias (2024): Die Beteiligung von Bürgerinnen und Bürgern bei kommunalen Mobilitätsprojekten: Eine quantitative Erhebung konsultativer Beteiligungsverfahren in Deutschland. In: RuR (Spatial Research and Planning). DOI: 10.14512/rur.2239

Database

The database, i.e. the systematically collected compilation of the participation processes and their coding according to specific aspects, can be researched and downloaded here.

Measures for a more user-centered evaluation of classification quality

This paper, presented at the 14th International Conference on Recent Advances in Natural Language Processing, introduces measures to evaluate practical requirements for the usability of AI-based tools.

Summary

A solution to limited annotation budgets is active learning (AL), a collaborative process between humans and machines for the strategic selection of a small but informative set of examples. While current approaches optimize AL from a machine learning perspective, we argue that successful real-world deployment requires additional criteria that target the second pillar of AL: the human annotators and their needs. For example, the usefulness of AL methods in text classification is typically assessed using common performance measures such as accuracy or F1. However, such measures fall short when applied to real-world datasets, which often contain a higher number of imbalanced classes. In these scenarios, additional criteria—such as quickly identifying all classes (e.g., topics) or detecting rare cases—become important. We therefore introduce four measures that reflect the class-specific requirements users have for data collection and content analysis.

In a comprehensive comparison of uncertainty-, diversity-, and hybrid-based data selection strategies across six different datasets, we find, for example, that strong F1 performance does not necessarily correspond to complete class coverage (i.e., not all topics are identified), and that different data selection strategies exhibit varying strengths and weaknesses with respect to class-specific requirements. Our empirical findings highlight that a holistic perspective is essential when evaluating AL approaches to ensure their practical usefulness. To this end, standard measures for evaluating machine-based text classification must be complemented by those that better reflect user needs.

Selected results

  • This publication proposes four new class-specific evaluation metrics for AL approaches that take into account how well and how quickly rare or all classes are detected. These criteria are not captured in detail by standard metrics such as F1.
  • The new measures enable practice-relevant insights into performance, particularly on datasets with varying frequently occurring classes and a wide range of different classes – characteristics that are common in real-world applications (e.g., topic detection in public participation processes).
  • It becomes clear that the choice of an appropriate AL strategy should not be based solely on standard performance measures. For example, the top-performing approaches according to the F1 score may not ensure that all classes are identified, despite this being a crucial requirement in the automated analysis of participation contributions: no topic should be overlooked, no voice should go unheard. The measures we developed can provide additional guidance in strategy selection and thus support the choice of practice-oriented solutions.

Publication

Romberg, J. (2023). Mind the User! Measures to More Accurately Evaluate the Practical Value of Active Learning Strategies. Proceedings of the 14th International Conference on Recent Advances in Natural Language Processing, 996–1006. https://aclanthology.org/2023.ranlp-1.107/

MA-thesis on participation of pupils during the Corona pandemic

In her thesis for the MA Social Sciences: Social Structures and Democratic Governance at Heinrich-Heine-University Düsseldorf, Maria Antonia Dausner has investigated the possibilities of pupil participation during the Covid-19-related school closures, focusing on an analysis of selected elementary schools in North Rhine-Westphalia.

More information is available in German.

Overview of Methods for Computational Text Analysis to Support the Evaluation of Contributions in Public Participation

In this publication in Digital Government: Research and Practice Julia Romberg and Tobias Escher offer a review of the computational techniques that have been used in order to support the evaluation of contributions in public participation processes. Based on a systematic literature review, they assess their performance and offer future research directions.

Abstract

Public sector institutions that consult citizens to inform decision-making face the challenge of evaluating the contributions made by citizens. This evaluation has important democratic implications but at the same time, consumes substantial human resources. However, until now the use of artificial intelligence such as computer-supported text analysis has remained an under-studied solution to this problem. We identify three generic tasks in the evaluation process that could benefit from natural language processing (NLP). Based on a systematic literature search in two databases on computational linguistics and digital government, we provide a detailed review of existing methods and their performance. While some promising approaches exist, for instance to group data thematically and to detect arguments and opinions, we show that there remain important challenges before these could offer any reliable support in practice. These include the quality of results, the applicability to non-English language corpora and making algorithmic models available to practitioners through software. We discuss a number of avenues that future research should pursue that can ultimately lead to solutions for practice. The most promising of these bring in the expertise of human evaluators, for example through active learning approaches or interactive topic modelling.

Key findings

  • There are a number of tasks in the evaluation processes that could be supported through Natural Language Processing (NLP). Broadly speaking, these are i) detecting (near) duplicates, ii) grouping of contributions by topic and iii) analyzing the individual contributions in depth. Most of the literature in this review focused on the automated recognition and analysis of arguments, one particular aspect of the task of in-depth analysis of contribution.
  • We provide a comprehensive overview of the datasets used as well as the algorithms employed and aim to assess their performance. Generally, despite promising results so far the significant advances of NLP techniques in recent years have barely been exploited in this domain.
  • A particular gap is that few applications exist that would enable practitioners to easily apply NLP to their data and reap the benefits of these methods.
  • The manual labelling efforts required for training machine learning models risk any efficiency gains from automation.
  • We suggest a number of fruitful future research avenues, many of which draw upon the expertise of humans, for example through active learning or interactive topic modelling.

Publication

Romberg, Julia; Escher, Tobias (2023): Making Sense of Citizens’ Input through Artificial Intelligence. In: Digital Government: Research and Practice, Artikel 3603254. DOI: 10.1145/3603254.

Expert evidence: State of research on opportunities, challenges and limitations of digital participation

As set out in the German Site Selection Act (StandAG), the Federal Office for the Safety of Nuclear Waste Management (BASE) is charged with the comprehensive information and participation of the public in regards procedure for the search and selection of a repository site for the final disposal of high-level radioactive waste. In this context, in February 2022 BASE commissioned an expert report on the “Possibilities and limits of digital participation tools for public participation in the repository site selection procedure (DigiBeSt)” from the Düsseldorf Institute for Internet and Democracy (DIID) at Heinrich Heine University Düsseldorf in cooperation with the nexus Institute Berlin. For this purpose, lead by Tobias Escher a review of the state of research and current developments (work package 2) was prepared has been summarised in a detailed report (in German).

Selected findings from the report are:

  • Social inequalities in digital participation are mainly based on the second-level digital divide, i.e. differences in the media- and content-related skills required for independent and constructive use of the internet for political participation.
  • Knowledge about the effectiveness of activation factors is still often incomplete and anecdotal, making it difficult for initiators to estimate the costs and benefits of individual measures.
  • Personal invitations have been proven to be suitable for (target group-specific) mobilisation, but the established mass media also continue to play an important role.
  • Broad and inclusive participation requires a combination of different digital and analogue participation formats.
  • Participation formats at the national level face particular challenges due to the complexity of the issues at stake and the size of the target group. Therefore, these require the implementation of cascaded procedures (interlocking formats of participation at different political levels) as well as the creation of new institutions.

Publication

Lütters, Stefanie; Escher, Tobias; Soßdorf, Anna; Gerl, Katharina; Haas, Claudia; Bosch, Claudia (2024): Möglichkeiten und Grenzen digitaler Beteiligungsinstrumente für die Beteiligung der Öffentlichkeit im Standortauswahlverfahren (DigiBeSt). Hg. v. Düsseldorfer Institut für Internet und Demokratie und nexus Institut. Bundesamt für die Sicherheit der nuklearen Entsorgung (BASE). Berlin (BASE-RESFOR 026/24). Available online https://www.base.bund.de/DE/themen/fa/sozio/projekte-ende/projekte-ende.html .

Enriching Machine Prediction with Subjectivity Using the Example of Argument Concreteness in Public Participation

In this publication in the Workshop on Argument Mining, Julia Romberg develops a method to incorporate human perspectivism in machine prediction. The method is tested on the task of argument concreteness in public participation contributions.

Abstract

Although argumentation can be highly subjective, the common practice with supervised machine learning is to construct and learn from an aggregated ground truth formed from individual judgments by majority voting, averaging, or adjudication. This approach leads to a neglect of individual, but potentially important perspectives and in many cases cannot do justice to the subjective character of the tasks. One solution to this shortcoming are multi-perspective approaches, which have received very little attention in the field of argument mining so far.

In this work we present PerspectifyMe, a method to incorporate perspectivism by enriching a task with subjectivity information from the data annotation process. We exemplify our approach with the use case of classifying argument concreteness, and provide first promising results for the recently published CIMT PartEval Argument Concreteness Corpus.

Key findings

  • Machine learning often assumes a single ground truth to learn from, but this does not hold for subjective tasks.
  • PerspectifyMe is a simple method to incorporate perspectivism in existing machine learning workflows by complementing an aggregated label with a subjectivity score.
  • An example of a subjective task is the classification of the concreteness of an argument (low, medium, high), a task whose solution can also benefit the machine-assisted evaluation of public participation processes.
  • First approaches to classifying the concreteness of arguments (aggregated label) show an accuracy of 0.80 and an F1 value of 0.67.
  • The subjectivity of concreteness perception (objective vs. subjective) can be predicted with an accuracy of 0.72 resp. an F1 value of 0.74.

Publication

Romberg, Julia (2022, October). Is Your Perspective Also My Perspective? Enriching Prediction with Subjectivity. In Proceedings of the 9th Workshop on Argument Mining (pp.115-125), Gyeongju, Republic of Korea. Association for Computational Linguistics. https://aclanthology.org/2022.argmining-1.11