Scientific quality assurance

The initial SQA team was convened for the development and selection of the study design, and subsequently, while the study is being awarded, it will add more members to cover specific topics and methods. The SQA is tasked with scrutinising the methods, their applicability in the studies, and their suitability.

In the Forum Flughafen und Region (FFR, Airport and Region Forum) it is common practice to have external scientific quality assurance. This ensures that the studies conducted on behalf of the FFR satisfy the highest scientific standards and reflect the status quo in the research.

The SQA was inaugurated on 25 August 2021, after which it established its rules of procedure. Although the SQA is appointed by the Coordination Council and works closely with the consortium and the FFR’s bodies, it is completely independent in all other respects.

Selection of SQA members and current composition

During selection, special attention was paid to ensuring that the SQA team has a multidisciplinary composition with at least five experts bringing expertise in measurement and modelling, the physics and chemistry of constituents of air, knowledge of UFP sources and data management.

The FFR developed its own list of criteria for the search for and selection of quality assurance experts:

• Technical expertise in one of the above-mentioned disciplines, evidence of participation in relevant publications or ongoing research projects
• Professional experience in working with and managing multidisciplinary teams
• Experience as a reviewer or appraiser in (national / international) evaluation procedures
• Experience in research management and communication, and publications in the public eye
• (Good) knowledge of German – at least sufficient to read scientific studies in German.

Alongside these criteria, there was another important restriction: experts who could be considered for participation in the study, or who actually took part in the invitations to tender, were of course not considered for membership of the SQA, as they might find themselves in a conflict of interests. The SQA also applied the Guidelines for Safeguarding Good Research Practice issued by the German Research Foundation (DFG) when identifying potential conflicts of interests.

Based both on the criteria listed above and on assessments, the experts were then placed in order to be contacted. The SQA currently has the following members:

• Emanuel Fleuti, Zurich Airport (immissions)
• Prof. David Groneberg, Goethe University Frankfurt (impact research)
• Dr. Martin Gysel-Beer, Paul Scherer Institute (immissions)
• Prof. Christian Hasse, TU Darmstadt (emissions)
• Prof. Andreas Held, TU Berlin (immissions)
• Dr Bryan Hellack, German Environment Agency (immissions)
• Prof. Wolfgang Hoffmann, University of Greifswald (impact research)
• Dr Stephan Nordmann, German Environment Agency (modelling)
• Theo Rindlisbacher, Swiss Federal Office for Civil Aviation (emissions)
• Prof. Erich Wichmann, Institute of Epidemiology at Helmholtz Munich (impact research).

How is quality assurance carried out by the SQA?

The SQA has the following tasks:

• Checking whether the methodology of the procedure selected by the contractors for realising the chosen study design meets the applicable scientific quality standards
• Checking the extent to which the proposed investigational design is suitable for answering the research questions
• Checking whether the evaluation, representation and interpretation of data are consistent, are based on the agreed methods, and satisfy scientific quality standards
• Checking whether research results of third parties have been sufficiently identified and acknowledged in the generation and interpretation of the study’s own results
• Formulating recommendations for the contractors
• Preparing opinions on draft reports and on the final reports of the consortium. The final reports will be published at the end along with the study. The draft reports are intended solely to aid internal communication between the participating scientists (including scientific support), and will be passed to the FFR exclusively for internal discussion.

The process:

The contractors must report to the SQA. This involves submitting draft reports, and any other documents to be prepared, to the SQA for discussion and a formal opinion. The SQA meets at least twice every year, with part of at least one meeting taking place in camera, followed by a joint meeting with the contractors. The consortium is obliged to take the SQA’s recommendations regarding scientific findings into consideration and, if they are not followed, to present a comprehensive rationale explaining why this was the case (e.g. scientifically untenable, not covered in the specifications, etc.). Similarly to the contractors, the SQA will also report regularly to the UFP Working Group on the status of its work, ideally at the same time as the contractors’ reports or at times coordinated with these reports. Prior to taking decisions, the Coordination Council receives both from the contractors and from the SQA all the relevant documents for comment and for drawing up recommendations. This comprises all the draft reports, including the SQA’s formal opinions (if the SQA has already commented on them and the consortium has had an opportunity to examine and integrate the remarks), studies, interim reports and results, etc.