Benchmarking of European Medicines Agencies (BEMA)

HMA (Joint Human and Veterinary) has established a benchmarking programme among the human and veterinary medicines agencies known as the Benchmarking of European Medicines Agencies (BEMA).  The programme has the following broad aim:

• To contribute to the development of a world-class medicines regulatory system based on a network of agencies operating to best practice standards.

BEMA is based on assessment of the systems and processes in individual agencies against a set of indicators which have been agreed in the following areas:

• Management systems;
• Assessment of marketing authorisation applications;
• Pharmacovigilance (drug safety) activities; and
• Inspection services.

The assessment is an opportunity to identify strengths and best practices in agencies and any opportunities for improvement; it is not an audit designed to identify non-compliance.  The exercise is not designed for direct comparison or ranking of agencies.  Agencies are encouraged to install suitable best practices in order to enable an improved operation of the network of agencies.

The BEMA is composed of representatives of AT, DE, FI, FR, GR, HR, NL, PT, SK and EMA.

Contact  Points:

Rui Santos Ivo (PT, co-chair)
E-Mail: rui.ivonoSpam@noSpaminfarmednoSpam.pt

Siniša Tomić (HR, co-chair)
E-Mail: sinisa.tomicnoSpam@noSpamhalmednoSpam.hr

BEMA Secretariat (HR)
E-Mail: Bema.SecretariatnoSpam@noSpamhalmednoSpam.hr

A Steering Group has been established to develop the programme and oversee its roll-out.  The group has drafted the questionnaire used in the assessment, devised the methodology, and it is also responsible for training the assessors.

It can be contacted at Bema.SecretariatnoSpam@noSpamhalmednoSpam.hr

Logistical support for the scheduling of visits is provided by the HMA Permanent Secretariat and support for the training workshops and assessor seminars is provided by the European Medicines Agency (EMA).

The BEMA methodology includes both self-assessment and peer review assessment, and is broadly based on ISO 9004 guidelines.

Self-assessment enables agencies to assess their own strengths and weaknesses in relation to the areas targeted by the performance indicators.

The peer review assessment is important as it allows each agency to be visited by teams of specially trained assessors (drawn from a pool of volunteers from other agencies) for an independent assessment against the performance indicators.

The outcome of each visit is an anonymised report produced by the assessment team and agreed with the visited agency.  All results are stored on a central database and a report is generated for the agencies to use as a tool for identification of best practices and for improvement.

BEMA I

The BEMA I programme began in January 2004 with the establishment of the Steering Group and the development of the KPIs, questionnaire and methodology.  Visits to agencies took place between June 2005 and May 2006 and successfully established the methodology as a tool for benchmarking and improvement of the network of medicines agencies in the EU/EEA.  The Steering Group published an overview document in May 2006, with a summary and a full report on the outcome of the first cycle published in December 2006.

BEMA II

Development of the second cycle of benchmarking was based on formal feedback gathered from agencies and assessors at the end of the first cycle, and the experiences of the Steering Group and logistics team, in order to make amendments to the programme, including the KPIs for the revised questionnaire.  Visits took place between June 2008 and July 2011.  Because the time period for visits to agencies was longer than in BEMA I, an interim summary report on the first half of the visits has been published.  The final report was published in September 2012.

BEMA III

For the third cycle of benchmarking, the HMA Strategy for 2011-2015 was used to direct an extensive review of the KPIs resulting in a more streamlined and focused questionnaire. Formal feedback gathered from agencies and assessors at the end of the second cycle, and the experiences of the Steering Group and logistics team, were used to make amendments to the programme methodology and supporting documentation. Visits in the BEMA III cycle began in September 2012 with a total of 47 agencies to be visited by October 2014.

The final report was adopted by the HMA in June 2016.

BEMA IV

The BEMA IV cycle was established following a positive outcome of the previous BEMA cycles, allowing its development into a tool for the improvement of regulatory quality management system and consistency of best practices across the EU network of medicines agencies.

The Steering Group and the Secretariat began with the cycle preparation in the spring of 2015. The programme methodology and supporting documentation were amended and adopted based on the formal feedback, gathered from agencies and assessors at the end of third cycle. Revision of the KPIs and updates in line with the EU Medicines Agencies Network Strategy to 2020 resulted in development of comprehensive, yet focused questionnaire.

BEMA IV assessments began in June 2016, with a total of 44 agencies assessed by the end of 2018.

The final report was adopted by the HMA in September 2019.

BEMA V

The development of BEMA V cycle was established in 2019 as a continuous activity for detecting best practices and improvement of the European Medicines Regulatory Network (EMRN). The programme methodology, BEMA Questionnaire and supporting documents were amended and adopted based on the formal feedback from HMA, BEMA assessors and NCAs. As a new addition to the Questionnaire, certain KPIs have introduced flexible assessment criteria, in line with the 2021-2025 EMRN strategy.

The initial start of BEMA assessments was proposed to begin in second half of 2020, yet postponed due to restrictions caused by the COVID-19 pandemic outbreak. BEMA V assessments began in December 2022 with estimated finalisation in 2025.

The interim report on BEMA V cycle was adopted by the HMA in September 2024.