Too many clinical trials fail – where are the errors and how to remove them?
When looking at the overall process of a clinical trial, we can separate the sources of errors into the following classes:
- hypothesis: poor quality of the underlying hypothesis
- knowledge about the patient
- inherent statistical errors
- data errors during a trial
Looking at those types of trials where a paper-version of a CRF (case report form) is used, then one obvious source of error lies in the transcription phase: while the Form Filler at the study site manually enters data (i.e. answers to the predefined questions) into the CRF, the Data Enterer transcripts these data into a validated database. Since this manual step is erroneous, using an electronic CRF (eCRF) with direct or indirect link to the database can definitely improve data consistency.
An interesting source of error lies in the knowledge about the patient. Do we really know that much about the person and its physiology and its health status so that a correct diagnosis can be made? Is abdominal pain a side effect of the drug under test or does the patient have IBS (irritable bowel syndrome)? While not knowing everything about the patient (making a gene test with every patient before the study is often not doable), the diagnosis may be wrong. The more we know about the patient, i.e. the more data we have, the better a diagnosis can be. Studies then may be converted from a hypothesis based one to a data-driven one.
This Article demonstrates the usage of a new toolbox, the Apple ResearchKit, as a eCRF and how more and more reliable data from the patients can be collected during a clinical trial.
Overview of the ResearchKit
ResearchKit was announced in March 2015 and published to the community as open source software one month later, with magnificent feedback from the development community. It was developed by Apple with the help of the leading medical institutions and foundations around the world.
Such is the popularity of the iPhone, ResearchKit was an immediate success. Just days after being announced, the system helped Stanford University gather a year’s worth of applicants (11,000) for a cardiovascular study in just 24 hours. Praising „the power of the phone,“ Alan Yeung, medical director of Stanford Cardiovascular Health, said to get over 10,000 applicants enrolled on a study without ResearchKit „would take a year and 50 medical centers around the country.“ ibtimes.co.uk
Apple published ResearchKit including a short video giving a rough overview. Now let us dive deeper into the insights: In general ResearchKit supports the creation of native iOS applications for research studies while focusing on three major aspects:
- Inform about the study and obtain patients consent
- Create surveys
- Create active tasks and collect sensor data
Strictly spoken, ResearchKit is a set of templates for fast creation of eCRFs with the additional aspect of having the patient filling in the data without the necessity of going to the study center.
Having an eye on data security, it is important to inform the patients in research studies about which sensitive information will be collected and shared as part of their enrollment and involvement. Appropriate customizable templates can be used to inform the patient about the details of the study as well obtaining his consent. Part of this is a dedicated template that asks the user to provide a signature directly on their device.
Example screenshots of the “Consent” module of the ResearchKit
The central “survey” module of the framework contains templates for the creation of dynamic surveys. This means that the authors of a study can select from a choice of currently more than 10 templates that best fit to the types of questions defined in the trial plan. Creating a question form is quite straightforward: just provide the text for the question, the type of answer expected and, if appropriate (e.g. multiple choice questions), the possible answers the patient can choose of.
Example screenshots of the “survey” module of the ResearchKit
All data entered (i.e. answers from multiple choices or free text) is being stored in a defined format and needs to be transferred to the backend of the research study. At this point it needs to be said that Apple or the contributors of ResearchKit do not have access to the patient’s data at any time. It is upon the choice of the developer to implement a secure data transmission to some database.
So far these templates cover the possibilities of an eCRF.
Now, the most interesting part of the framework is the ability of collecting sensor data from the iOS device while asking the patient to perform certain tasks. Hereby, a variety of templates as part of the “active tasks” module help gathering data for different categories like fitness, voice, gait and balance, spatial memory and tapping speed.
Typically, an active task is structured into several steps:
- Provide information about the task itself and what the patient needs to do
- Counter indicating when the task starts. Now the patient can prepare to perform the required task, e.g. put the iOS device onto a table
- Record data during the duration of the task
- Inform about the completion of the task
A tasks ends with a confirmation screen that informs about the successful completion and about the next steps in the research study. The recorded data are stored as raw data and will not be interpreted by ResearchKit. This remains at the owner of the study.
The following active tasks are currently supported:
|Motor activity||Gait and Balance||AccelerometerGyroscope||Device motionPedometer|
|Tapping speed||Multi-Touch displayAccelerometer (optional)||Touch activity|
|Fitness||Fitness||Accelerometer||Device motion PedometerLocationHeart rate|
|Cognition||Spatial memory||Multi-Touch display||Touch activityCorrect and actual sequence|
|Voice||Sustained phonation||Microphone||Uncompressed audio|
Example screenshots of active tasks currently integrated in the Research Kit
With the help of ResearchKit the creation speed of the eCRFs is dramatically increased. Instead of wasting time and money on the creation of forms, the developer can now concentrate on the data only.
With the decision to publish ResearchKit as Open Source on githup, the community of iOS developers are able to push the framework forwards as they have already done with more than 600 updates to this day since the first release in April. By this approach, one can be sure that only the owner of the study sees the data.
The restriction to the Apple ecosystem ensures that the gathered data is not influenced by different hardware suppliers. The minimum hardware requirement is an iPod touch that can be handed over to a closed user group if they do not own an iOS device. ResearchKit supports external hardware like the Apple Watch to measure data like the heart rate and it can be adopted to support other external devices that collect medical data.
Currently not supported by ResearchKit is the collection of data in the background as it will be done within Apple’s health app, but as the framework and the health app is under control of Apple this can only be a matter of time. Another point that is yet not supported by ResearchKit and has to be realized by the developer is the ability to schedule surveys and active tasks. For example if the patient should perform an active task every morning, the developer has to setup a notification to remind the patient.
It will be exciting to see how ResearchKit develops in the next month and according to the news the iPhone could become a new tool in genetic studies.
Active Task: Example Case
To give you an impression how an active task can look like we would like to give an example for voice recording. The goal of the task is to investigate the ability of sustained phonation that uses the microphone of the device and creates uncompressed audio. The described example demonstrates the usage of the various templates as well as how the forms can be filled with content. All the artwork is provided by ResearchKit and can easily be customized or replaced by other individual images or animations.
||Step 1: Inform about the task.Typically the patient will be informed about the task in general on a first screen.In this case the patient will be informed that the voice will be evaluated, using the microphone of the device.|
||Step 2: Inform about the details.This screen informs about the concrete steps the patient will have to complete in the following.Here the patient will be informed that he needs to take a deep breath and say “Aaaaah” as long as possible. He will receive visual feedback in form of a power spectrum to involve his vocal volume.|
||Step 3: Prepare for the activity.It is possible to display a countdown timer so that the patient can prepare for the evaluation.|
||Step 4: Guide and give feedback.Within this step the patient will be guided to say “Aaaaah” and receives visual feedback from the microphone that helps him to keep his vocal volume constantly. No data analysis is done by ResearchKit; this can be defined according to the requirements.This step ends automatically as soon as the patient has stopped his voice.|
||Step 5: Inform about the task result.This steps displays that the evaluation is successfully completed and what will happen next with the results.|
Summary and Outlook
The ResearchKit provides templates for rapid development of iOS based eCRFs covering not only basic question types but also active tasks by utilizing available sensor data. Data can be entered everywhere and the patient is not required to go to the study center. This makes it very comfortable and in the end comes with less costs for a study.
Notice that the questionnaire is just part of a clinical trial; the clinical diagnostics has still be done (e.g. once in a month). Although this is the more expensive portion of a study, having the ability of collecting data on a higher frequency base (e.g. every day or on demand) helps the researchers finding out more about the person/drug under test and as such improve the quality of diagnosis.
The integration of sensor data opens the range of applicable trial types to areas, where sensor data was not available at all or where the study center required some additional measurement tools. Hereby, one can think of the measurement of the Range-of-Motion after surgical intervention, e.g. on the shoulder. Another idea would be the observation and tracking of activity and movement of elderly people.
From the trial plan to the integrated solution using the Apple Research Kit
Clinical Trials and Zühlke?
When moving from the ResearchKit to an integrated solution for clinical trials, the following tasks need to be done:
- create the customized App based on your trial plan and questionnaire and bring it onto your iOS devices (or put it in the AppStore)
- ensure a secure data transmission to a database and implement the whole data backend
- incorporate special tasks not integrated in the ResearchKit right now (e.g. Range-of-Motion tasks, optional)
- perform Data Analytics tasks, such as providing some sort of first analysis helping the researchers interpreting the raw sensor data (optional)
- develop and integrate dedicated sensor hardware/software, e.g. vitality sensors for iOS devices (optional)
Zühlke is supporting you at all these steps.
What is your experience with the ResearchKit? We are looking forward to your feedback to this blog.