Guidance on clinical evidence requirements for medical devices: Clinical data and evaluation

Clinical data are the safety or effectiveness information that is generated from the clinical use of a medical device. Sources of such data include device-specific clinical investigations, published clinical literature and post-market surveillance data.

Factors that may influence the need for clinical data include:

• post-market performance of the comparator device
• the seriousness of the condition being treated
• devices assessed to have a high risk of potential safety concerns
• higher levels of uncertainties associated with identifying the risks and benefits of the technology
• devices with inconclusive evidence to support safety and effectiveness or with controversial safety or effectiveness information in the literature
• introduction of a novel device technology
• devices that have unique or novel manufacturing processes, raw material sources or specifications
• duration of use
• anticipated variation in patient response to treatment, including differential performance due to sex, gender or other characteristics (such as age or race and ethnicity)
• devices with limited benefit to the intended patient population
• devices that may cause rare adverse events

Note: Rare adverse events could include, for example, unintended events, disease or injury, or clinical signs that occur so infrequently that they cannot be evaluated in a pre-market study. The World Health Organization uses the term "rare" when describing an adverse event that occurs between 1 in 1,000 people and 1 in 10,000 people.

For example, an application may be subject to an increased need for device-specific clinical data (such as undergoing sufficient statistical analysis to justify study endpoints and sample size, demographics) if there is a lack of safety and effectiveness evidence for:

• an emerging device technology or
• a complex device designed to be implanted for a long period of time

Where safety and effectiveness have been adequately demonstrated but clinical data collection is ongoing, results from ongoing tests may need to be submitted following licensing. This requirement is set out under terms and conditions at the time of licensing. Clinical data collection may be ongoing due to the emerging nature of the technology (for example, to gather information on long-term use or on use in rare patient populations in a real-world setting or to study a specific population). The results from this ongoing clinical data collection must demonstrate that the device's continued safety and effectiveness are maintained post-market.

When analyzing clinical data, it's important to consider the quality of the data in terms of sources of uncertainty, missing data or the degree of disaggregated population data. Manufacturers should provide estimates on the amount of under-reporting, for example:

• as may be seen from differences in reported rates in randomized, controlled clinical trials versus market history
• the number of problems, complaints or incidents that have been reported to the manufacturer that were not required to be reported to Health Canada or
• the number of patients who have been lost to follow-up.

The value of the clinical data is diminished and may result in a need for higher-quality device-specific clinical data if:

• the number of patients who were lost to follow-up is greater than 10% or
• under-reporting of adverse events is high

It's also important that any reports or collection of data represent robust evidence. There should be sufficient information to make an objective assessment of the device's safety and effectiveness. For example, randomized, controlled clinical trials provide a higher quality of evidence. By contrast, reports of clinical experience, such as anecdotal reports, individual case reports or expert clinical opinion, are of lower quality. This type of information is generally limited to cases where it's not feasible or practical to collect high-quality clinical data (for example, in rare disease states, under-represented patient populations and emergent interventions).

Health Canada expects that clinical data referred by manufacturers adequately represent the Canadian population and clinical practice. Any clinical data used by the manufacturer to demonstrate a device's safety and effectiveness should reflect the population for whom the device is intended.

Clinical data that is representative of the population

Sex- and Gender-based analysis plus (SGBA Plus) is an analytical process used to assess how diverse groups of people may be impacted by product or federal initiatives. Diverse groups of people include women, men, girls, boys, gender-diverse people, racial and ethnic minorities, persons with disabilities, and First Nations, Inuit and Métis people. Consideration is given to biological factors related to sex, race and ethnicity, socio-cultural factors related to gender and other identity factors.

The "Plus" recognizes that people have multiple identity factors that intersect and accumulate, that privilege or disempower, impacting their lived experiences and health. In other words, SGBA Plus considers many critical identity factors (for example, race and ethnicity, religion, age, mental and physical disability, geography, income, education). The ways they intersect inherently govern their social, economic and health outcomes.

Evidence demonstrates that biological, economic and social differences between diverse groups of women and men contribute to differences in health risks, health services use, health system interaction and health outcomes. The integration of SGBA Plus throughout the life cycle of a medical device will lead to sound science that addresses the different needs of people effectively.

Given the potential for different impacts of medical devices for diverse subpopulations, clinical studies should include adequate representation in a disaggregated manner:

• Sex refers to a person's biological and physiological characteristics. A person's sex is most often designated by a medical assessment at the moment of birth. This is also referred to as birth-assigned sex.
• Gender refers to the roles, behaviours, activities and attributes that a given society may construct or consider appropriate for the categories of "men" and "women". It can result in stereotyping and limited expectations about what people can and cannot do.
• Gender-diverse people refers to people who may or may not have a gender that is congruent with their birth-assigned sex or do not identify with either of the genders ("man/boy" or "woman/girl"). Instead, they identify with a gender that is either between or outside of the traditional binary gender roles.
• Race refers a social construct. It's not grounded in biology but can influence how people access programs and services. Therefore, the impacts of racialization need to be measured and assessed along with other identity factors as a determinant of health.
• Ethnicity refers to a broader term than race that's used to categorize groups of people according to their cultural expression and identification. Commonalities such as racial, national, tribal, religious, linguistic or cultural origin may be used to describe someone's ethnicity.

As a guideline, manufacturers should consider the following key questions when integrating SGBA Plus in the design of clinical trials:

• Is the device sex-specific (female or male biology has specific impact on the device's use and effectiveness)? If yes, was the device tested specifically for that biologic sex?
• Might the product behave differently according to sex, race and ethnicity, age or gender identity?
• Have safety or adverse events differences been noted according to subgroups (for example, sex, gender, race and ethnicity)?
• During the clinical trials, what representation of different sex and gender identities or various age, race and ethnic subpopulation groups will be included in testing?
• To what extent will the results of the clinical trials submitted as part of the licence application be disaggregated by sex, gender, age, pregnancy or breastfeeding status, or race and ethnicity?

Device design should take into account the unique anatomical and physiological characteristics of all sexes and genders to the extent that this is practicable. When clinical studies are not sufficiently sized or powered to draw conclusions from these subgroups, manufacturers are encouraged to consider how certain types of other clinical data (in particular real world evidence (RWE) and post-market clinical experience) can be used to demonstrate the differential impacts of a device on different sexes and genders.

Further, where feasible, the differential impacts of a device on under-represented populations, including racial and ethnic groups, should be considered in clinical trials or investigations. This might be done through clinical trial design approaches such as stratification or by requiring a minimum number of patients from these subgroups within a larger clinical trial.

For more information and guidance, please consult the following items:

• Guidance document: Considerations for inclusion of women in clinical trials and analysis of sex differences (Health Canada)
• Integrating sex and gender into research (Canadian Institutes of Health Research)
• Gender-based analysis plus research guide (Women and Gender Equality Canada)

Clinical data for under-represented populations

When considering clinical data for under-represented populations, including children or pregnant individuals, manufacturers may have limited clinical data. Sometimes, this data gap can be mitigated by conducting small, well-designed clinical trials or by including a group of patients from the under-represented population within a larger clinical trial. Alternately, a device might be granted a licence with terms and conditions that stipulate more follow-up to demonstrate ongoing safety and effectiveness. Labelling limitations may also be required where there is not sufficient clinical data.

Manufacturers may also consider other ways to extract the needed clinical data from existing data. This can include leveraging RWE that may reflect the device's use in sub-population groups who don't usually participate in clinical investigations. Manufacturers should also consult existing institutional research practices or guidelines on including under-represented populations in clinical trials or investigations when it's safe to do so.

Pediatric devices (neonates, infants, children and adolescents)

Biologically, children (especially 6 years and younger) and adults are different. As infants and children have smaller organs, medical devices designed for adults may not be appropriate.

Pediatric populations are under-represented in clinical trials and investigations. This is an area of concern. As such, care must be taken to evaluate the limited datasets in the pre-market evaluation stage and in the assessment of post-market signals.

Manufacturers need to consider the following factors, including:

• age (as an indirect indicator of patient maturity and physiological development)
• anatomical size and weight
• future biologic growth

These pediatric-related factors should be considered when designing the product, as well as during the assessment, interpretation of data, indications for use, labelling and risk mitigation phases. Intended patient age and/or patient weight should be stated in the intended use if it will affect the device's performance or safety. Consideration should also be given to different sub-groups of the pediatric population (for example, neonates, adolescents), which have different needs and risk profiles.

Pre-market applications for a product used by the pediatric population should show how the design considers this group. For example, if a device will be used by adolescents down to neonates, then the pre-market application should show how these targeted populations were considered in the design. Product testing or theoretical justification, whichever is appropriate, may be used to demonstrate this. For example:

• devices that physically interact with the patient's anatomy (for example, blood pressure cuff)
  • the size of the device should align with the expected size of the patient populations
  • implantable medical devices should allow for or accommodate growth of the pediatric patient after implantation
  • the algorithms should be trained and validated in the appropriate patient populations (for example, arrhythmias are different in certain pediatric populations due to higher baseline heart rates)
  • precision and accuracy may be impacted by differences in concentration, thickness or level of interaction of the device as a function of patient age and other characteristics (for example, sex, race and ethnicity, and anatomic size)
  • may also impact the hazard posed to the patient (for example, sensitivity to radiation exposure during radiography procedures)
  • may be impacted by variables related to age as well other characteristics such as sex, race and ethnicity, and anatomic size (for example, hemodialysis device, infusion pumps)

  Device usability is another important aspect to consider in all populations, including in pediatric populations, especially if the device includes a user interface and relies on patient inputs, oversight or options. The pre-market application should make it clear whether the use of the device requires adult supervision or can be operated by the pediatric patient.

  Human factors and usability studies should demonstrate that the person who is to operate the device (parent/caregiver or child of the intended age) can follow the prescribed labelling or training. Intended operators should be able to operate the device without making errors that could be hazardous or undermine the device's effectiveness. Manufacturers need to analyze these studies to demonstrate that possible hazards have been identified and successfully mitigated. Note: The use of labelling to mitigate such a risk may be ineffective in pediatric populations, especially for younger patients.

  Generating clinical data from clinical investigations

  A clinical investigation may be used to collect information about the safety and effectiveness of a medical device. Manufacturers that conduct a properly designed device-specific clinical investigation may be highly confident in their ability to evaluate the device's safety and effectiveness.

  Clinical investigations may be necessary to support a pre-market licence application, especially in devices with limited characterization of performance within a clinical setting.

  In Canada, no person shall conduct a clinical investigation in respect of a medical device unless the person holds an authorization issued under Part 3 of the Regulations. In addition, clinical investigations carried out by or on behalf of the manufacturer should follow the ethical principles of the Declaration of Helsinki and the Tri-Council policy statement (2nd edition): Ethical conduct for research involving humans. They should also conform to good clinical practices outlined in ISO 14155 - Clinical investigation of medical devices for human subjects.

  Health Canada does accept clinical data from other jurisdictions. However, countries that have different standard clinical practices, different patient demographics or different standards than Canada may influence how we interpret the data.

  For more information on the sale and import of a medical device for investigational testing involving human subjects, please consult the Applications for medical device investigational testing authorizations guidance document.

  Generating clinical data from post-market clinical experience

  Once a medical device is on the market, more safety and effectiveness data can be obtained as information about the device continues to accumulate from real-world use. This clinical data, known as real-world data (RWD), can provide supporting information on the ongoing safety and effectiveness of a device. For example, the device could be used for longer durations or in patient populations that are broader compared to those studied in a pre-market clinical investigation. Devices may also be marketed for different uses in different countries.

  Various types of post-market data may be included to support the product's safety and effectiveness in a new licence application or in an application for an amendment to a licence. The data may be based on the subject device or a comparable device.

  Marketing history information in Canada and other jurisdictions

  This includes information on the marketing history in Canada and other jurisdictions, including for:

  • devices accessed through the Special Access Program (SAP)
  • devices with an investigational testing authorization (ITA) application
  • modified devices for which the previous version of the device is licensed in Canada
  • similar devices on which the current subject device is based and which are not licensed in Canada but have been in clinical use in other regulatory jurisdictions

  Marketing history should include a summary of the total number of sales per year of each model of the subject device, broken down by country or regulatory jurisdiction.

  Manufacturers should note that marketing history is generally associated with significant under-reporting of adverse events. Therefore, while marketing history can be helpful in identifying possible risks associated with the device, it's limited in its ability to demonstrate safety or effectiveness.

  Incidents and corrective actions

  A device that is the subject of a licence application in Canada may have been sold for use previously, in Canada or internationally. This could be through earlier market authorization in a foreign regulatory jurisdiction or an authorization in Canada through SAP or ITA.

  As part of the post-market monitoring of the subject device, a manufacturer may have:

  • received complaints of incidents or reported adverse events associated with its use and
  • implemented corrective actions to address an identified problem

  Information gathered by the manufacturer who handles a complaint or adverse event or recalls the device may provide valuable context as part of a licence application. However, incident reporting by consumers has limitations due to under-reporting and incomplete information. Manufacturers are also encouraged to inform consumers about incident reporting.

  To submit device incident data to Health Canada, manufacturers should:

  • separate incidents into relevant categories
  • present both the number of incidents as well as the number of products used or sold to provide context about the rate of occurrence of these events
  • include the dates for which the data are applicable
  • include marketing history such as incidents and corrective actions if using clinical evidence from a comparator device

  If a recall has occurred or other corrective action has been taken to address an identified problem with a device, detailed information should be submitted as part of the licence application. For more information and guidance, please consult the Guide to recall of medical devices (GUI-0054).

  Real-world data (RWD) and real-world evidence (RWE)

  Examples of RWD include:

  • data from electronic health records (EHRs)
  • claims and billing data
  • data from product and disease registries
  • patient-generated data (including in home use settings)
  • data gathered from other sources that can inform on health status, such as mobile devices

  RWD sources can be used as data collection and analysis infrastructure to support many types of trial designs. These include randomized trials, such as large simple trials, pragmatic clinical trials and observational studies (prospective and/or retrospective).

  To inform market authorization decisions, well-designed and -planned clinical investigations are the most robust tool for providing evidence of device safety and effectiveness. Following specific patient populations in a highly controlled environment facilitates the collection of high-quality data. However, it can limit the generalizability of the device's use in real-world settings. Moreover, conducting clinical investigations is not always feasible and may not be considered ethical for certain diseases/disorders (such as rare diseases) or certain patient populations. In other situations, excessive investigation costs or small available patient populations may introduce constraints.

  Expanding evidence sources to include RWD and RWE may address some of these concerns and offer opportunities to:

  • gain insight on public health
  • identify late occurring or rare adverse reactions
  • identify additional sex, gender, race and ethnicity and other identity characteristics or factors affecting usage
  • increase both the extent and rate of health product access for patient populations
  • obtain data on safety and effectiveness in subgroups that were not well represented in clinical studies such as racial and ethnic minorities, and different age groups;
  • clearly elucidate real-world performance of the product under real-world use

  With the proper analysis, RWD of sufficient quality, quantity and relevance may be used to generate useful RWE. Robust RWD and RWE may be used to support Health Canada's regulatory decision-making before and after a device is available in the Canadian marketplace (pre- and post-market).

  It's not the intent of this guidance document to elaborate on the methodological approaches that can be used to generate or analyze RWD. Manufacturers filing a medical device licence application or amendment application with RWD/RWE to Health Canada should:

  • identify the test device/model/version that is the subject of the RWD and any differences between the comparator and subject device (the impact of these differences in interpretation of the RWE should be discussed in the application)
  • include a rationale to explain why RWE is appropriate to provide evidence of safety and/or effectiveness (for example, rare population, ethical considerations)
  • include a detailed explanation of how the RWD was gathered and analyzed to support the quality and relevance of the evidence as it pertains to the application
  • identify any limitations of the data or of the evidence generated through analysis of the data and identify how these were mitigated to the extent possible (for example, identified or potential biases, known confounders and how they were addressed in the analyses)
  • verify relevant use of the device in the health care system

  For more information about the use of RWD and RWE, please consult Elements of real world data/evidence quality throughout the prescription drug product life cycle. Although the document focuses on pharmaceuticals, some information is also relevant to the elements of protocol development and data quality with respect to generating RWD and RWE for medical devices.

  Literature reviews

  Clinical data can also be obtained from sources that are already publicly available. This includes data on a subject device that has been published in the scientific literature or published clinical data from a comparator device with similar technological characteristics.

  A literature review is considered the systematic identification, synthesis and analysis of the body of available scientific literature on the subject device when used for its intended purpose(s). When providing a literature review as evidence of safety or effectiveness, manufacturers should ensure that it does not rely on devices that are not comparable. Ideally, the literature review is for the subject device.

  If the clinical history of a comparator device has been well established, a manufacturer may rely solely on a literature review to establish safety and effectiveness for the subject device. If a manufacturer cannot demonstrate that the comparator device is suitable, then device-specific clinical data may be required.

  A literature review should include a critical analysis of all available evidence, including both favourable and unfavourable reports, as well as the validity or limitations of the study designs. Manufacturers are expected to provide a summary of each publication and an analysis of the data contained in the publication. All publications that are cited should be included in the application.

  Generating clinical data from usability studies and simulations

  Usability studies and simulation data sets are usually performed outside of the scope of investigational testing authorizations in Canada. However, they can also provide valuable information about the safety and effectiveness of certain medical devices.

  Usability or human factors studies often involve typical users of medical devices who test the medical device in a simulated clinical environment. Outcomes from these studies often focus on indicators such as user errors and adequacy of following instructions.

  The IEC 62366-1 international standard on medical devices - Part 1: Application of usability engineering to medical devices defines a usability test as a method for exploring or evaluating a user interface with intended users within a specified intended use environment. For this guidance, usability or human factors studies should include an appropriate user group and be conducted in a representative clinical setting, where the device is used in a manner consistent with the provided labelling.

  Simulation studies generally refer to numerical simulations of real-world use of a medical device. For the purpose of this guidance, Health Canada defines simulation as:

  • the imitation of the characteristics of a system, entity, phenomena or process using a computational model or
  • a specific "run" of the model with 1 set of parameters that results in the quantity of interest or multiple quantities of interest

  Simulations can also provide useful information on safety and effectiveness, especially when based on validated clinical databases. Information on the databases, numerical simulation software or other tools used should be clearly described. Validation of the tools and data used should be clearly demonstrated so that the results can be applied to the actual clinical use of the proposed device.

  In some cases, human factors evidence may be deemed necessary to demonstrate that all known risks have been reduced to the extent possible.

  Data considerations for devices with cosmetic indications for use

  Some devices may have several therapeutic applications that may either:

  • diagnose, treat, mitigate or prevent a disease, disorder or abnormal physical state or
  • restore, modify or correct the body structure

  This includes devices with cosmetic applications or cosmetic indications for use. Examples include lasers for hair removal, lasers for corrective refractive eye surgery, wrinkle reduction devices and devices for liposuction or circumference reduction.

  Benefit-risk considerations

  All medical devices are reviewed to confirm that the benefit of the device outweighs the risks. In the case of devices with cosmetic indications for use, Health Canada will review the clinical evidence submitted by the manufacturer to confirm:

  • patients are not exposed to unnecessary risks and
  • residual risks are mitigated to the extent possible, taking into consideration the clinical benefit

  When applicable, information on how a device with cosmetic indications for use could affect diverse patient subpopulations may be considered in assessing the benefits or risks related to the device. Diverse patient subpopulations include various groups of females, males, women, men and gender-diverse people, racial and ethnic minorities, pediatric and elderly patients, and patients with special needs.

  Providing objective clinical information

  In some cases, it may be difficult to quantify the benefit of the cosmetic indication for use. Make sure that any data are gathered as objectively as possible. Manufacturers can reduce or eliminate potential bias by:

  • comparing a baseline measurement to a measure taken after treatment
  • blinding the patient, and if possible the assessors, to whether a patient received treatment or was in a control group

  If the device is intended to improve the appearance of the patient, image comparisons from before and after treatment are often used. Data collected in the form of pictures should be taken in a controlled environment, using the same conditions before and after treatment, including location, angle, lighting, resolution and camera type/model. Data collected should also be disaggregated by sex and gender across age and racialized groups as appropriate. Note these considerations in the application.

  Literature provided as evidence should include enough details to allow Health Canada to confirm that the conclusions are not biased or confounded by poor controls during the assessment. Include key journal articles in the application. Provide at least one objective measure of effectiveness (for example, count and length of wrinkles before and after, weight before and after procedure).

  Supporting the effectiveness of the device

  Manufacturers must provide evidence to support all parts of the indications for use of their medical device. For example, if the product is indicated for use to achieve permanent results, then the manufacturer needs to provide data from a clinical investigation that follows patients over an extended period of time. The length of follow-up should be based on scientifically grounded evidence that the results are indeed "permanent" as opposed to "long-term" or "short-term."

  When the product is indicated for temporary results, the data must clarify how long the results will last. If specific areas of the body are targeted for treatment, then the clinical data provided in the application must reflect that specific target area. Manufacturers should justify that the clinical data they used was gathered in a large enough number of healthy patients to prove that the results are statistically significant. The manufacturer should use scientifically supported and validated assessment tools and methods to collect measures of patient impact, quality of life or satisfaction.

  Supporting the safety of the device

  Health Canada will be critical in its review of the safety evidence provided. Manufacturers must take measures to ensure that the design of the device eliminates, or reduces as much as possible, any dangers that can arise from its use, when used in accordance with the instructions for use.

  For requirements to mitigate potential thermal damage of energy-based medical devices, please refer to the Risk of thermal harm from therapeutic and cosmetic energy-based medical devices: Notice to industry.

  If the device is to be used by the public or non-health care professionals who have had limited training, manufacturers must demonstrate that the design of the device is safe for all users. Manufacturers are encouraged to consider usability or human factors testing to confirm that interactions between the device and its users occur as intended and are safe.

  If a certain frequency of treatment is required to maintain the effectiveness of treatment, then clinical data supporting the safety of that frequency of treatment is required. As well, patient characteristics that could affect safety, such as dermatological conditions or skin pigmentation, must be clearly considered. Clinical data must also be provided to support the safety in those populations of patients.

  Medical devices with cosmetic indications for use are subject to the same stringent Health Canada review as other devices to ensure the benefits outweigh the risks. Manufacturers should consider whether the clinical evidence provided to support these applications is objective and addresses all aspects of the indications for use.

  Clinical evaluation

  A clinical evaluation should consider all relevant information from all sources of clinical data.

  Through proper clinical evaluation, it's expected that the manufacturer will have demonstrated the following:

  • the device is effective during normal and expected use conditions
  • the known and foreseeable risks, and any adverse events, are minimized and acceptable when weighed against the benefits of the intended effectiveness
  • any claims made about the device's safety and effectiveness (for example, product labelling, marketing material (including website) and indications for use) are supported by suitable clinical evidence

  If a comparison is being made to a licensed or marketed device, manufacturers may also provide objective evidence that the outcomes observed when using the proposed device are not inferior to the existing standard of care approaches. This comparison should include an equivalent patient population (where applicable) and contemporary outcomes from an existing equivalent intervention or for an accepted standard of care treatment. Confidence levels and uncertainties in the datasets should also be discussed.

  The evaluation should also address any clinical claims made about the device, the adequacy of device labelling and product information (particularly contraindications, precautions and warnings) and the suitability of instructions for use.

  When deciding upon the scope of a clinical evaluation, a manufacturer should consider:

  • any design features of the device or aspects of the target treatment populations that require specific attention
  • evidence gathered through device-specific clinical investigations
  • evidence from comparator devices that may be used to support the safety and/or effectiveness of the subject device
  • possible differences based on sex, race and ethnicity, and gender-related characteristics in safety and effectiveness

  Manufacturers may provide a Clinical Evaluation Report (CER) with their Class III or IV applications. A CER provides the conclusions of the clinical evaluation of the subject device. The CER should include all analyzed clinical data and demonstrate that the subject device achieves its intended purpose while maintaining a favourable benefit-risk profile. The CER is not mandatory in Canada.

  When providing a CER with a medical device licence application, additional supporting evidence may be required to ensure the comparator device and the subject device are discussed given their Canadian regulatory status. Once provided, the CER becomes subject to the Public Release of Clinical Information.