| Clinical trial sponsors can collect patient reported | | | | data quality controls, rapid data access, and so on. |
| outcomes electronically in multinational clinical trials. | | | | To deliver consistently reliable systems, the |
| Advantages include significant improvements in | | | | Electronic Patient Reported Outcomes provider |
| clinic and subject compliance, data quality, data | | | | must also have the skills to develop and execute |
| access and monitoring, as well as reduced staff | | | | high quality system test plans. Test plans must |
| and subject burden. Yet, ePRO clinical solutions can | | | | include thoughtful test cases that address every |
| also present challenges. By understanding some | | | | requirement in the systems requirements |
| important ePRO best practices, however, | | | | document. |
| sponsors can overcome major challenges.ePRO | | | | To ensure every systems requirement is |
| CLINICAL SOLUTIONS: BRIEF SUMMARY OF THE | | | | covered, there must be a traceability matrix |
| CHALLENGES | | | | which matches each requirement in the systems |
| One of the major challenges faced when using | | | | requirements document to the relevant test |
| Electronic Patient Reported Outcomes (ePRO) in | | | | cases in the test plan. This may seem obvious, |
| multinational trials is ensuring the quality and | | | | but it is often not done. |
| reliability of the ePRO systems and data files. | | | | For each systems requirement there must also |
| After all, even small problems with the systems | | | | be detailed test cases and scenarios for both valid |
| or data can sometimes damage credibility, which | | | | and invalid data, as well as standard and |
| can lead to decreased system usage, increased | | | | non-standard user actions. The expected results |
| enrollment, queries, and other trial costs, and at | | | | for each test must also be documented in |
| worst, the discrediting of all the data. | | | | advance of the test cases to ensure the integrity |
| Another challenge is the very large number of | | | | of the test. In addition, the test plan needs to |
| changing variables that need to be managed. | | | | include large data input sets that represent a wide |
| These variables include: | | | | variety of diverse scenarios. |
| - Complex protocols and lengthy questionnaires | | | | Furthermore, it is vital that the test plan include |
| - Long lists of countries, languages and translations | | | | significant load and stress testing to ensure that |
| - Complex system requirements | | | | the systems will function properly in real world |
| - Interdependent schedules | | | | conditions, when an exceptionally large set of data |
| - Long growing lists of issues | | | | has been collected or when stressful situations are |
| - A multitude of first patient dates | | | | encountered. |
| - International user support across different time | | | | When executing the test plan, the testing |
| zones in many languages. | | | | specialists must document whether the tests pass |
| Validity, or measurement equivalence with | | | | satisfactorily and ensure that every data file is |
| validated paper questionnaires, can also be a | | | | correct, without exception. When evaluating ePRO |
| significant challenge. Due to the nature of the | | | | providers, the sponsor should also review test |
| hardware and software, some systems cannot | | | | plans and traceability matrices from previous |
| fully support existing validated paper | | | | projects to ensure that the provider has the skills |
| questionnaires. | | | | to develop and execute test plans that meet the |
| For instance, small hand-held devices generally | | | | standards described above. |
| have very restricted screen real estate. IVR | | | | In addition to the formal testing performed by the |
| systems are limited because subjects can only | | | | provider, the sponsor must also carefully review |
| remember short questions and answers. In these | | | | the software by performing a user acceptance |
| and other cases, the questions and answers from | | | | test. The purpose of this test is to ensure that |
| a validated paper questionnaire often have to be | | | | both the ePRO team and the sponsor team |
| modified in order to accommodate the | | | | agree that all systems requirements have been |
| technology. | | | | implemented as described in thesystems |
| In order to then ensure that the data from the | | | | requirements document. |
| electronic systems are equivalent to the | | | | If the provider has executed a quality test plan, |
| paper-based questionnaire, it is often necessary to | | | | this sponsor process should be quick and easy, |
| conduct a validation study comparing the data | | | | and should provide the sponsor team final control |
| collected on the new electronic tool with the data | | | | and peace of mind before delivering systems for |
| collected on the validated paper questionnaire. | | | | the first patient date. However, if the provider |
| These validation studies can add significantly to | | | | has not performed adequate testing, quality issues |
| both costs and project timelines. | | | | may be revealed and the sponsor then has the |
| Finally, a challenge faced by many when using | | | | opportunity to delay shipment and require |
| Electronic Patient Reported Outcomes in | | | | corrective action before the systems are |
| multinational trials is quality assurance and auditing. | | | | distributed to sites and become a burden in the |
| Some auditors have limited experience or | | | | field.ePRO CLINICAL SOLUTIONS: COMPETENT |
| knowledge of ePRO systems, leading to varied | | | | PROJECT MANAGEMENT |
| interpretations of industry regulations. These | | | | Because there are so many moving parts in |
| diverse regulatory interpretations can cause | | | | multinational trials, it is critical that the provider can |
| confusion, and also have considerable cost and | | | | develop and maintain complicated project plans. |
| timeline implications.ePRO CLINICAL SOLUTIONS: | | | | Project plans must include hundreds, sometimes |
| BEST PRACTICES | | | | thousands, of detailed tasks and expected |
| The challenges described above sometimes | | | | durations, dependencies between them, and |
| appear daunting to a study team, even though | | | | contingencies for unexpected events. |
| they appreciate the potential. However, there are | | | | Because the ePRO project schedule typically |
| solutions. The best practices described below | | | | involves hundreds of tasks, the provider must |
| offer a method to manage the challenges.ePRO | | | | demonstrate competencies with an electronic |
| CLINICAL SOLUTIONS: SYSTEM QUALITY AND | | | | project management tool so that a changing |
| RELIABILITY | | | | environment can be accommodated quickly. |
| All successful systems start with an unambiguous | | | | During the provider evaluation process, sponsors |
| systems requirements document. The purpose of | | | | should also look at project schedules and project |
| this document is to describe the system(s) to be | | | | status reports from previous studies to help |
| delivered in detail, so that all team members have | | | | determine the providers' project management |
| exactly the same expectations of what the | | | | skills. |
| system will do and what data the system will | | | | In addition, good communication among the |
| provide. | | | | project team members needs to be the rule and |
| The process of writing and reviewing a detailed | | | | not the exception. Regularly scheduled project |
| planning document, whether it be systems | | | | management team meetings with key staff from |
| requirements, a clinical trial protocol, or otherwise, | | | | both the sponsor and the provider are critical to |
| forces the author and the reviewers to think | | | | review the status of project tasks, determine if |
| through options for processes and to discuss and | | | | changes are needed, and to resolve any issues |
| tweak them until there is confidence among all | | | | that arise. These project status meetings are |
| participants that the best solution has been | | | | crucial in order to ensure that sponsor and ePRO |
| designed. | | | | team member efforts are complementary and |
| So, the systems requirements definition process, | | | | effective.ePRO CLINICAL SOLUTIONS: CHOOSE |
| properly executed, produces quality thinking on | | | | SYSTEMS THAT SIGNIFICANTLY REDUCE |
| system design and is the first step towards | | | | VALIDATION ISSUES |
| producing quality systems. In addition, because the | | | | To resolve the challenge of measurement |
| deliverable is a written detailed document | | | | equivalence between a validated paper |
| describing how the system will work, it prevents | | | | questionnaire and an electronic device, sponsors |
| misunderstandings, confusion and frustration at a | | | | can choose a technology where the validated |
| later stage when the systems are implemented. | | | | paper questionnaire text does not need to be |
| A systems requirements document must show all | | | | changed. Touchscreen tablet or internet-based |
| screen shots exactly as they will look in the | | | | systems typically offer this kind of flexibility. |
| completed system. Every edit for every data | | | | Although it is beyond the scope of this article, the |
| entry field, all screen and system validation | | | | validation requirements for these technologies are |
| checks, and all system logic (skips, conditions, | | | | very minimal.ePRO CLINICAL SOLUTIONS: |
| calculations, error conditions, messages, and so | | | | MANAGE QUALITY ASSURANCE AUDITORS |
| on) must be described clearly and in detail. | | | | Given that many auditors have little experience |
| This documented presentation of the system | | | | with Electronic Patient Reported Outcomes, |
| both 'on the surface' and 'beneath the surface' | | | | sometimes their statements can be confusing, |
| (logic checks, and so on) enables the sponsor | | | | and even frightening. |
| team and the ePRO provider to think about the | | | | If an auditor imposes requirements that seem |
| issues carefully before programming starts. It will | | | | unreasonable, it is important to review the |
| also make certain that the system design will | | | | regulation(s) with the auditor and find more |
| ensure compliance with protocol, and that the | | | | practical solutions. In most cases, there are |
| resulting data files will meet the needs of the | | | | various avenues to ensure regulatory compliance, |
| study. | | | | and many of them are simple and |
| The final systems requirements document must | | | | cost-effective.ePRO CLINICAL SOLUTIONS: |
| be signed by the key sponsors and ePRO | | | | CONCLUSION |
| provider team members, to show agreement and | | | | The best practices discussed above can provide |
| commitment from both parties on the details of | | | | real tools for addressing the challenges that can |
| all systems functions and operations. When | | | | arise in the implementation of Electronic Patient |
| evaluating providers, the sponsor should ask to | | | | Reported |
| see systems requirements documents from | | | | Outcomes in multinational clinical trials. While these |
| previous projects. | | | | best practices do not cover every possible |
| The sponsor should ensure that the providers | | | | situation that can occur, they will provide |
| have the skills to deliver a thoughtfully prepared | | | | significant assistance in addressing key challenges, |
| systems requirements document which describes | | | | and put their clinical trials on a solid path to |
| a high quality system with the attributes the | | | | success. |
| sponsor desires: ease of use, protocol compliance, | | | | |