AI for the evaluation of participation? The potential of language models to recognise modes of transport in participation contributions

In this article in the journal Internationales Verkehrswesen, Laura Mark, Julia Romberg and Tobias Escher present a language model that can be used to reliably recognise modes of transport in participation contributions. They show that supervised machine learning can usefully support the evaluation of participation contributions in mobility-related online participation processes.

Summary

Consultations are an important part of transport planning and can help to integrate knowledge from the public into the planning process. However, online formats in particular often result in large volumes of contributions, the thorough evaluation of which is resource-intensive. It is hoped that the use of AI will support this.

The language model presented in this article is based on the concept of supervised machine learning for text classification. Pre-trained models are re-fined using smaller data sets. In this way, a model can be adapted to a specific area of application, such as mobility-related consultation processes.

A pre-trained German-language version of the high-performance RoBERTa language model was used as a starting point. Using a categorisation scheme that mainly distinguishes between the modes of transport mentioned, 1,700 contributions from seven transport planning consultation processes were manually coded. The resulting data was used partly as training data for fine-tuning the language model and partly for evaluation.

Results

  • Overall, it was shown that language models already available today are suitable for supporting the evaluation of consultation processes in practice. The language model developed here for recognising the modes of transport can serve as the basis for a specific application in municipal planning practice.
  • The post-trained RoBERTa language model is very effective at assigning the appropriate modes of transport. The model presented by us can always reliably assign well over 90% of the entries correctly to the modes of transport they contain.
  • For the processes on whose contributions the model had been trained, an average of 97% of the categories could be correctly assigned (on a separate test set). For contributions from other transport-related participation procedures, the appropriate modes of transport could still be assigned very reliably with an accuracy of 91 to 94%.
  • The performance of the model therefore hardly deteriorates when it is applied to previously unknown data from mobility-related participation procedures. This means that manual coding in advance can be omitted, at least for similarly structured participation procedures, which significantly reduces the effort involved.

Publication

Mark, Laura; Romberg, Julia; Escher, Tobias (2024): KI zur Auswertung von Beteiligung. In: Internationales Verkehrswesen 76 (1), S. 12–16.

The political shaping of the sustainability transformation: participatory and ecological?

In this article in regierungsforschung.de, the academic online magazine of the NRW School of Governance, Tobias Escher explores the question of whether the participation formats required for a participatory development of the transformation to a more sustainable society can also be implemented in an ecologically sustainable manner. In other words, the question is how the principle of (ecologically) sustainable governance can be upheld in the democratically important area of political participation.

Abstract

The transformation to a sustainable society can only succeed through the involvement of all stakeholders. At the same time, sustainable governance requires that these participation processes also fulfil the requirements of sustainability. Based on a systematic discussion of the greenhouse gas emissions generated by various political participation formats, this article argues that in view of the functional and normative significance of political participation, its environmental footprint does usually not contradict the demand for sustainable governance.

Key findings

  • The implementation of participation formats generates greenhouse gas emissions due to the energy required, the mobility of the participants and, where applicable, catering and accommodation. Crucially, the level of emissions depends on whether the formats are held in person or digitally (see table below).
  • For local face-to-face formats, the emissions from the event venue and mobility of participants are comparatively low and only amount to around 2kg of CO2-equivalents per person in the scenario analysed. The emissions increase considerably as soon as participants have to travel further (using fossil-fuelled transport) – in the underlying scenario by a factor of twelve to 24kg. This is roughly equivalent to the average emissions from a 100km car journey.
  • In contrast, online participation formats only generate very low emissions (around 0.3kg per person in the scenario). Nevertheless, the choice of participation formats cannot be based solely on their short-term ecological costs, but must also take into account normative requirements for inclusion and popular control. Digital participation formats in particular represent significant barriers to participation for already disadvantaged population groups (see also our DigiBeSt-report).
  • Overall, participation has an ecological footprint, but this is generally justified by functional and normative considerations because after all, governance should not only be sustainable, but above all democratic!
face-to-face participationdigital participation
local(inter)-regionalasynchronsynchron
energyheating & climatisation of event venueoperation of end user devices & data centresmore data intensive communication
mobilitytraveltravelnot applicable
catering & accommodationcateringaccommodationnot applicable
Direct environmental impact of different participation formats. The darker the colour, the greater the respective emissions from this source. The emissions generally increase with the number of participants and the duration of the participation format.

Publication

Escher, Tobias (2024): Die politische Gestaltung der Nachhaltigkeitstransformation: partizipativ und ökologisch? Essay. In: Regierungsforschung.de (15. Jahrgang). https://regierungsforschung.de/die-politische-gestaltung-der-nachhaltigkeitstransformation-partizipativ-und-oekologisch/

Pushback for the municipal mobility transition? Joint closing event of SÖF Junior Research Groups CIMT and MoveMe

The two junior research groups in Social-Ecological Research CIMT and MoveMe are holding a joint final event showcasing some results of their research into the transition to sustainable mobility. The event is scheduled to take place online on 26. April 2024. More information is available in German.

Is that fair? An evaluation tool for local mobility measures

In this article in the journal Internationales Verkehrswesen, Laura Mark, Annika Busch-Geertsema, Jessica LeBris, Gesa Matthes and Kerstin Stark present a practical approach to assess the justice of transport measures in various dimensions. The approach and the article were developed in the context of the working group “Mobilität, Erreichbarkeit und soziale Teilhabe” of the Academy for Spatial Development in the Leibniz Association (ARL).

Summary

This paper presents a practical tool for taking a systematic “second look” at mobility transition measures through the lens of justice. It can be used for conceptual support during planning and implementation, for reflection during or after the process as well as ongoing monitoring. Three dimensions of justice can be used to analyse which population-groups benefit from these measures. The dimensions used are distributive justice, recognition of different realities of life and procedural justice, which have been further differentiated and combined in an easy-to-use matrix.

To differentiate the recognition of different realities of life, the Persona approach is employed. In a Persona, specific characteristics are combined that can influence and restrict mobility options, decisions, and activity chains. The article proposes a system for developing custom Personas, but the Personas already developed by the authors can also be used for the application.

The article provides a detailed presentation of the dimensions of justice and the Persona approach. It can be downloaded here:

Furthermore, the evaluation tool, including a user manual and suggestions for Personas, can also be downloaded in Excel format from the ARL Mobility Working Group website (scroll down). The authors welcome feedback and encourage free use and further development.

Expert evidence: State of research on opportunities, challenges and limitations of digital participation

As set out in the German Site Selection Act (StandAG), the Federal Office for the Safety of Nuclear Waste Management (BASE) is charged with the comprehensive information and participation of the public in regards procedure for the search and selection of a repository site for the final disposal of high-level radioactive waste. In this context, in February 2022 BASE commissioned an expert report on the “Possibilities and limits of digital participation tools for public participation in the repository site selection procedure (DigiBeSt)” from the Düsseldorf Institute for Internet and Democracy (DIID) at Heinrich Heine University Düsseldorf in cooperation with the nexus Institute Berlin. For this purpose, lead by Tobias Escher a review of the state of research and current developments (work package 2) was prepared has been summarised in a detailed report (in German).

Selected findings from the report are:

  • Social inequalities in digital participation are mainly based on the second-level digital divide, i.e. differences in the media- and content-related skills required for independent and constructive use of the internet for political participation.
  • Knowledge about the effectiveness of activation factors is still often incomplete and anecdotal, making it difficult for initiators to estimate the costs and benefits of individual measures.
  • Personal invitations have been proven to be suitable for (target group-specific) mobilisation, but the established mass media also continue to play an important role.
  • Broad and inclusive participation requires a combination of different digital and analogue participation formats.
  • Participation formats at the national level face particular challenges due to the complexity of the issues at stake and the size of the target group. Therefore, these require the implementation of cascaded procedures (interlocking formats of participation at different political levels) as well as the creation of new institutions.

Publication

Lütters, Stefanie; Escher, Tobias; Soßdorf, Anna; Gerl, Katharina; Haas, Claudia; Bosch, Claudia (2024): Möglichkeiten und Grenzen digitaler Beteiligungsinstrumente für die Beteiligung der Öffentlichkeit im Standortauswahlverfahren (DigiBeSt). Hg. v. Düsseldorfer Institut für Internet und Demokratie und nexus Institut. Bundesamt für die Sicherheit der nuklearen Entsorgung (BASE). Berlin (BASE-RESFOR 026/24). Available online https://www.base.bund.de/DE/themen/fa/sozio/projekte-ende/projekte-ende.html .

3rd workshop for practitioners on first results from surveys in case study municipalities

On 30 November we invited representatives of the municipalities with whom we cooperate in order to discuss the first results of the extensive surveys conducted by our research group. The focus was on the question of how the respective participation procedures are assessed by those participating and which aspects motivate or discourage such participation.

Despite the diversity of the five projects we examined (and the still small number of participants), the assessments of the people participating in such processes show a relatively high degree of agreement. Overall, the evaluations of the participation processes are rather positive with regard to the course of discussion and transparency. At the same time, however, there are also comparable challenges in all processes. For example, the representation of one’s own interests is rated as relatively good, but gaps in the representation of other opinions are perceived. Also, a balance of interests is not always achieved. Furthermore, the participants are rather sceptical about the actual impact of the participation results on the political process, even though they still deem such an impact possible.

There is more information available in German.

2nd workshop for practitioners on automated text analysis for citizen contributions

Part of the efforts of the research group is to develop tools that support the evaluation of citizen contributions from participation processes. On 10 December 2021 the research group hosted a workshop with practitioners (including local planning officials, participation officers and planning experts) to discuss our recent developments, part of which have been published in the Proceedings of the 8th Workshop on Argument Mining.

More information on the insights from the workshop is available in German.

New working group on mobility, accessibility and social inclusion at the ARL – Academy for Territorial Development in the Leibniz Association

We are pleased that Laura Mark is part of the aforementioned working group and can discuss our research with colleagues. Practitioners and researchers meet regularly in the working group to discuss various topics related to mobility and social inclusion. The working group started in the middle of 2021 and the content-related work is now taking more and more shape: Areas of interface with our research include the question of procedural justice in planning processes for the mobility transition – who participates and whose voices are heard? How should planning and participation processes for a sustainable mobility transition be designed in the future in order to include everyone? Here we will report on the further work and publications and events that develop within the context of this working group!

New working group on mobility, accessibility and social participation at the ARL – Academy for Spatial Development in the Leibniz Association

We are pleased to have Laura Mark participating in the mentioned working group to contribute to discussions with colleagues about our research. Practitioners and academics meet regularly in the working group to work on various topics related to mobility and social participation. The working group was started in mid-2021 and the substantive work is gradually taking shape: Intersections with our research include the question of procedural justice in planning procedures for the mobility transition – who participates and whose voices are heard? How should planning and participation processes be designed in the future for a sustainable mobility transition that includes everyone? We will report on the ongoing work, publications and events that emerge within the framework of this working group!

Results of the first practical workshop of the junior research group CIMT

Our first practical workshop in summer 2020 focused on the question of how the evaluation of citizen contributions can be technically supported and what requirements practitioners have for a software solution designed to (partially) automate the evaluation.

More information can be found in the working paper (German version only!):