|
|
||||||||
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
METHODS |
From Daiichi Sankyo Pharma Development, Edison, New Jersey (Dr Sarapa); Mortara Instrument, Milwaukee, Wisconsin (Dr Mortara, Mr Brown); and AMPS-LLC, New York City, New York (Mr Isola, Dr Badilini).
Address for correspondence: Nenad Sarapa, MD, Johnson & Johnson PRD, Welsh and McKean Roads, Spring House, PA 19477; e-mail: nsarapa{at}prdus.jnj.com.
| ABSTRACT |
|---|
|
|
|---|
Key Words: FDA QTc warehouse digital annotated ECG quality
The digital environment offers ease of acquisition, processing, analysis, storage, and regulatory submission of ECGs from clinical trials.4 The US Food and Drug Administration (FDA) recommends that ECG data from TQTS in the new drug application (NDA) are submitted in digital form with annotations detailing the exact onset and offset points of ECG intervals.5 The FDA has occasionally requested digital submission of ECGs from other studies in the NDA.6 The Health Level 7 (HL7) Version 3.0, a standard format for digital annotated ECG files based on the Extended Markup Language (XML), incorporates the elements necessary to describe the amplitude resolution and sampling rate of annotated ECG waveforms.7 A digital ECG warehouse was developed under the Cooperative Research and Development Agreement between the FDA and Mortara Instrument (Milwaukee, Wisconsin)8 to effectively store and analyze large numbers of digital annotated ECG waveforms after they are uploaded online. The warehouse consists of the section exclusive to the FDA reviewers and a "private" section (E-Scribe Warehouse)9 for the fee-for-service perusal by sponsors before the NDA submissions. The functionality of the FDA digital ECG warehouse has been reviewed recently.10 The warehouse assigns each ECG a Global Unique ID (GUID) number, which links the annotated waveform to other SAS files within the NDA dossier (eg, demography, QT/QTc measurements on that ECG, drug concentrations, and adverse events). FDA reviewers view annotated ECG waveforms online to assess the quality of ECG data during the NDA review using a standardized approach.6 They initially randomly spot-check individual ECGs from all parts of the database for problems with the ECG signal quality and the placement of ECG interval annotations. Thereafter, they scrutinize specific ECG waveforms identified as outliers in the statistical results or due to intriguing waveform descriptions or other relevant criteria. In addition, the FDA reviewers may use the digital warehouse algorithms that produce quantitative scores for ECG quality metrics and sort individual ECGs by the score value in tables and histogram distribution plots.10
This study sought to test the ability of the E-Scribe Warehouse to detect certain quality issues with digital ECGs obtained from robust QTc assessment in a clinical study in healthy subjects.
| SUBJECTS AND METHODS |
|---|
|
|
|---|
ECG Acquisition and Analysis
Digital 12-lead ECGs were obtained by an ELI 200 electrocardiograph (Mortara Instrument; 500 samples/second). Four single resting supine ECGs were obtained in each subject at baseline (30 minutes predose) and at 2, 3, and 4 hours after dosing. Subjects rested quietly in fully supine position for 5 minutes before each ECG and remained semirecumbent in bed during 4 hours postdose. The ECGs consisted of 10 seconds of raw data along with the set of representative beats computed by the ELI 200 algorithm. The ECGs were processed by a digital caliper software system (CalECG2 Version 1.0, AMPSLLC, New York) on a high-resolution computer screen. The filter settings of the electrocardiograph (0.05-Hz low-frequency cutoff and 150-Hz high-frequency cutoff) were maintained during processing with no additional filtering applied during storage and processing of ECG waveforms.
A semiautomated QT interval measurement method was applied as follows: the CalECG algorithm prepositioned the Q-wave onset and T-wave offset annotations, a trained reader visually inspected the placement of QT interval annotations by CalECG, and, wherever necessary, the reader manually adjusted the position of calipers to achieve the optimal placement of QT annotations. Within 1 day, ECGs were analyzed twice by the same reader in the randomized sequence using the standard-of-care single-lead annotation protocol that makes QT and RR measurements on 3 consecutive sinus rhythm beats on the raw ECG signal from limb lead II.12,13 The reader was blinded to treatment period (baseline vs postdose) and postdose time. The first analysis was done in a precise manner as described above ("reference" ECG set). The second analysis was done in a purposefully erroneous manner ("test" ECG set) whereby the reader had intentionally made a predefined number of errors expected to occur during ECG processing and analysis by the central laboratory. The following interventions were made in the "test" set:
The "reference" and "test" ECG sets were uploaded in the E-Scribe Warehouse (ESW, Mortara Instrument).9
ESW Metrics
For the overall ECG data in the test versus reference set, ESW produced summary reports for the integrity of study data, the ECG acquisition quality, and the adherence to the QT annotation protocol by the reader.
The VERITAS digital algorithm (Mortara Instrument) within ESW produced a quantitative score for the T-wave signal strength on the T-waves in the annotated beats of individual ECGs in both data sets. The T-wave signal strength is a measure of the T-wave power, and as such, it is a squared amplitude measurement that is positive definite. Higher power T-waves are characterized by larger amplitude as well as faster rise and descent times. The median, mean, and minimum-maximum range of score values were computed for the T-wave signal strength in the test versus reference data set. ESW ranked individual ECGs in each set in order of increasing T-wave signal strength score and displayed them on histograms. In addition, the number and proportion of ECGs was computed with the T-wave signal strength score, with the threshold indicative of the questionable ECG quality regarding that metric (
20 000).
The VERITAS algorithm within ESW also produced the T-offset Bias metric on the 10 beats where the T-offset annotation has been intentionally moved by the reader to create the test set. VERITAS generates a "global" QT interval from the analysis of a single cardiac cycle from ECG rhythm data and determines the QT interval duration using the same cardiac cycle from all 12-leads. The T-offset bias constitutes the temporal difference (ms) between the fiducial point of the T-offset annotated by the precise versus imprecise semiautomated method (reference and test set, respectively) and by the VERITAS algorithm.
| RESULTS |
|---|
|
|
|---|
|
|
ESW scores for the T-wave signal strength in the reference and test sets are summarized in Table II, and the histogram distributions of ECGs for this quality metric for the reference versus test set are presented in Figures 2A and 2B. The ESW score for the T-wave amplitude on the same ECG ranked lower in the test set (after intentional misannotation) than in the reference set. For example (Figure 3), the ECG from the test set misannotated in the lead V1 with low T-wave amplitude was assigned a T-wave signal strength of 5480 by the ESW and ranked as 103rd among 104 ECGs. The same ECG in the reference set, having been correctly annotated in limb lead II with normal T-wave amplitude, had a T-wave signal strength score of 63 948 and the 19th ranking. Individual ECGs of questionable quality (identified by the T-wave signal score below the ESW threshold) were evident on the histograms, and the respective waveforms were viewable by double-clicking on the ECG record in the rank-order tables or on the bars in histograms.
|
|
|
Figure 4 shows the QT interval duration by the 3 methods (the fully automated VERITAS algorithm and the semiautomated method executed precisely [reference] and intentionally imprecisely [test]) on the 9 beats where the T-offset annotation errors were intentionally introduced in the test set. Table III presents the listing and the descriptive statistics for the ESW metric of T-offset bias for the same 9 beats.
|
|
| DISCUSSION |
|---|
|
|
|---|
The adherence to the QT annotation protocol by the readers is recognized as an important aspect of the central laboratory performance in clinical trials.6,16 If the readers deviate from a standard QT annotation protocol due to low ECG quality in a given lead, then this deviation may be a relevant quality metric. The ESW summary report in our study revealed that the algorithms correctly detected no ECGs with less than 3 expected QT annotations in the reference and test sets, which corresponds to no errors committed or intentionally introduced by the reader. The ESW correctly detected full compliance with the lead choice protocol in the reference set. On the other hand, for the subset of 20 ECGs from 5 subjects in the test set where the reader has intentionally violated the protocol for lead selection during QT annotation, ESW reported 19 and 21 ECGs with QT annotations in a lead different from the primary choice and in multiple leads, respectively, which translates to failing to detect 1 of 20 ECGs with this type of protocol incompliance. Good QT interval measurement practice would require that when encountering unacceptable noise or other quality issue midway through the analysis of serial ECGs from 1 subject, all ECGs from that subject should be resubmitted to the blinded reader at the central laboratory for remeasurement in the backup lead. The ability to assess QT annotations from 1 time point to the next in serial ECGs from 1 subject would improve the functionality of the ESW in assessing the adherence to the QT annotation protocol by the central laboratory and should be incorporated in future releases.
ECG signal quality influences the reliability of QT interval measurement by every QT interval measurement method,6,15,21,22 and issues in this regard are not uncommon in ECGs from robust QTc studies. Hnatkova et al23 found that 0.38% of 15 194 and 2.31% of 29 806 ECGs from 2 TQTS and that 13.18% of 4090 ECGs from another drug study in healthy subjects24 were immeasurable due to unacceptable noise content. The global low-frequency (LF) and high-frequency (HF) noise metrics in the ESW indicate the overall quality of the ECG signal and are associated purely with ECG acquisition at the study site. ESW has detected no content of LF and HF noise in the test and reference sets, which corresponds to the careful attention to the quality of ECG acquisition by the clinical site staff in our study. Identical reports for the global LF and HF content reflect are in keeping with identical electrical ECG signals in both sets; however, due to the small number of ECGs in our data sets, we could not conclusively assess the performance of the ESW in this functional area.
The T-wave signal strength is the ESW metric that reflects the ECG waveform feature that critically influences the precision and reproducibility of QT measurement by fully manual, semiautomated (manually adjusted), and fully automated QT interval measurement methods alike.10,21,22 Inspection of this particular ECG property by both the clinical site staff and the readers at the central laboratory is advisable in every robust QTc study,25 and it has been recommended that sponsors request of central laboratories the study-specific reports on the exact number of ECGs incompatible with reliable QT measurement.10 The quantitative scores by the relevant ESW algorithm may add value in this context. The low score and/or low ranking for the T-wave signal strength would point the FDA reviewers toward the ECGs that, according to the good QT interval measurement practice, should have been considered for exclusion from the analysis by the central laboratory. The statistic for the ECGs with the T-wave signal strength score below the threshold in our study (Table III) has approximately reflected the proportion of ECGs in the test set where the T-offset was intentionally annotated in the setting of the low T-wave amplitude. With endorsement of the FDA, Mortara Instrument initially set an arbitrary threshold of 20 000 for the T-wave signal strength on the basis of qualified manual review of several thousand digital ECGs from clinical trials used in the development of ESW.10 It is meant primarily to allow the FDA reviewers to quickly detect ECGs of questionable quality within 1 study or to compare different studies based on the proportion of ECGs outside of the range of acceptable quality. The FDA did not disclose the role of the mean and median values for the T-wave signal strength score during the NDA review. In fact, it is uncertain if the FDA reviewers currently use these summary parameters at all because they are not instantly available to download from the warehouse (instead, the FDA reviewers have to rely on Mortara support staff to provide the individual scores in SAS or similar format and compute the summary statistics). With increasing numbers of ECGs in the FDA warehouse, it is expected that the threshold for T-wave signal strength will be adjusted. Ideally, the means/medians for the T-wave signal strength and other relevant metrics could be used for comparisons between studies within 1 NDA or across the FDA ECG warehouse. Further studies are needed to assess if the warehouse scores for a particular quality metric will vary according to different clinical trial designs or the pharmacological classes of drugs. Given consistent performance of the warehouse in similar settings, the quality of ECG data from TQTS of the same design and/or TQTS with the same class of compounds could be compared by the summary scores for the warehouse metrics. When larger numbers of digital ECGs are submitted to the FDA, the mean/median values for the whole warehouse, or for the particular TQTS known to the FDA reviewers, may replace the thresholds as the "signpost" for the ECG data that warrant closer scrutiny during the NDA review.10 The same logic applies to the sponsor's use of ESW before or outside of the NDA submissions, whereby the ESW scores for a given study would be compared with the historic data.
Limitations of the Study
This was an observational study not intended to test a hypothesis, but the performance of the ESW was challenged according to a predefined study protocol with quantitative endpoints. The data sets have included a much smaller number of ECGs than the one typically obtained in the TQTS or the phase I study; however, the FDA reviewers currently spot-check a small number of ECGs in the warehouse and consider them to be informative of the quality of ECGs in the whole NDA under review.6 We have acquired single ECGs at each time point instead of triplicates, the current standard for robust QTc assessment in clinical trials. On the other hand, the ESW and the FDA warehouse consider each ECG from a triplicate as a single observation,10 so our results from 2 sets of single ECGs are representative of the functionality of both parts of the digital ECG warehouse. Although it is likely that the FDA reviewers routinely use the algorithms in the warehouse to identify which ECGs deserve closer scrutiny during the NDA review,10 the specific digital tools in the FDA ECG warehouse, applicable metrics, and minimal acceptable standard for regulatory approval have not been disclosed. Instead of the FDA section of the ECG warehouse, we have performed the analysis within the commercially available "private" section (the E-Scribe Warehouse). It should have very similar performance characteristics to the FDA ECG warehouse because both use the same technology provided by Mortara Instrument. Thus, our results would be relevant for the performance of the FDA ECG warehouse in the regulatory review of ECG data quality included in the NDA.
| CONCLUSIONS |
|---|
|
|
|---|
Validation of the utility of the ECG warehouse will require that the results of functional assessment in larger sets of digital annotated ECGs be put in the context of the statistical and clinical outcomes from large cardiac safety studies. Even today, though, the ECG warehouse facilitates the implementation of the FDA requirement for submission of digital ECG data within the NDA5 and clearly offers potential to improve the quality of proarrhythmic risk assessment during clinical development and regulatory review of novel drug candidates.
| ACKNOWLEDGEMENTS |
|---|
|
|
|---|
Dr Sarapa's current affiliation is Johnson & Johnson PRD, Spring House, Pennsylvania.
| REFERENCES |
|---|
|
|
|---|
1. International Conference on Harmonization of Technical Requirements for Registration of Pharmaceuticals for Human Use (ICH). E14 Guidance on clinical evaluation of QT/QTc interval prolongation and proarrhythic potential for non-antiarrythmic drugs. 2005. Available at: http://www.fda.gov/cder/guidance/6922fnl.pdf.
2. Stockbridge N, Throckmorton DC. Regulatory advice on evaluation of the proarrhythmic potential of drugs. J Electrocardiol. 2004;37(suppl): 40-42.[CrossRef][Web of Science][Medline] [Order article via Infotrieve]
3. Strnadova C. The assessment of QT/QTc interval prolongation in clinical trials: a regulatory perspective. Drug Inform J. 2005;39: 407-434.[Web of Science]
4. Cabell CH, Noto TC, Krucoff MW. Clinical utility of the Food and Drug Administration Electrocardiogram Warehouse: a paradigm for the critical pathway initiative. J Electrocardiol. 2005;38: 175-179.[CrossRef][Web of Science][Medline] [Order article via Infotrieve]
5. Food and Drug Administration. CDER. Providing digital electrocardiogram (ECG) data. 2006. Available at: http://www.fda.gov/cder/regulatory/ersr/default.htm#ECG.
6. Stockbridge N, Brown BD. Annotated ECG waveform data at FDA. J Electrocardiol. 2004;37(suppl): 63-64.[CrossRef][Web of Science][Medline] [Order article via Infotrieve]
7. HL7 Version 3 Standard. Ann Arbor, MI: Health Level Seven, Inc; 2003. Available at: http://www.hl7.org/V3AnnECG/foundationdocuments/welcome/index.htm.
8. Mortara Instrument press release: Mortara Instrument announces collaboration with US Food and Drug Administration (FDA) Milwaukee, WI, 8 June 2004. Available at: http://www.mortara.com/assets/client_files/File/news_CRADA.pdf.
9. E-Scribe Digital Annotated ECG Warehouse. Milwaukee, WI: Mortara Instrument; 2006. Available at: https://www.ecgWarehouse.com/index.php.
10. Sarapa N. Quality assessment of digital annotated ECG data from clinical trials by the FDA Warehouse. Exp Opin Drug Saf. 2007;6: 1-13.[CrossRef]
11. Sarapa N, Morganroth J, Couderc JP, et al. Electrocardiographic identification of drug-induced QT prolongation: assessment by different recording and measurement methods. Ann Noninvasive Electrocardiol. 2004;9: 48-57.[CrossRef][Web of Science][Medline] [Order article via Infotrieve]
12. Committee for Proprietary Medicinal Products. Points to consider: the assessment of the potential for QT interval prolongation by non-cardiovascular medicinal products. 1997. Available at: http://www.coresearch.biz/regulations/cpmp.pdf.
13. Morganroth J, Silber SS. How to obtain and analyze electrocardiograms in clinical trials: focus on issues in measuring and interpreting changes in the QTc interval duration. Ann Noninvasive Electrocardiol. 1999;4: 425-433.[CrossRef]
14. Morganroth J. Focus on issues in measuring and interpreting changes in the QTc interval duration. Eur Heart J Suppl. 2001; 3(suppl K): K105-K111.[Abstract]
15. Batchvarov V, Hnatkova K, Malik M. Assessment of noise in digital electrocardiograms. Pacing Clin Electrophysiol. 2002;25: 499-503.[CrossRef][Medline] [Order article via Infotrieve]
16. Malik M. Errors and misconceptions in ECG measurement used for the detection of drug induced QT interval prolongation. J Electrocardiol. 2004;37(suppl): 25-33.[Web of Science][Medline] [Order article via Infotrieve]
17. Fenichel RR, Malik M, Antzelevitch C, et al. Drug-induced torsades de pointes and implications for drug development. J Cardiovasc Electrophysiol. 2004;15: 475-495.[CrossRef][Web of Science][Medline] [Order article via Infotrieve]
18. Le Coz F, Funck-Brentano C, Poirier JM, Kibleur Y, Mazoit FX, Jaillon P. Prediction of sotalol-induced maximum steady-state QTc prolongation from single-dose administration in healthy volunteers. Clin Pharmacol Ther. 1992;52: 417-426.[Web of Science][Medline] [Order article via Infotrieve]
19. Barbey JT, Sale ME, Woosley RL, Shi J, Melikian AP, Hinderling PH. Pharmacokinetic, pharmacodynamic, and safety evaluation of an accelerated dose titration regimen of sotalol in healthy middle-aged subjects. Clin Pharmacol Ther. 1999;66: 91-99.[CrossRef][Web of Science][Medline] [Order article via Infotrieve]
20. Murray A, McLaughlin NB, Bourke JP, Doig JC, Furniss SS, Campbell RW. Errors in manual measurement of QT intervals. Br Heart J. 1994;71: 386-390.
21. Badilini F, Sarapa N. Implications of methodological differences in digital electrocardiogram interval measurement. J Electrocardiol. 2006;39: S152-S156.[CrossRef][Web of Science][Medline] [Order article via Infotrieve]
22. Kligfield P, Gettes LS, Bailey JJ, et al. Recommendations for the standardization and interpretation of the electrocardiogram: Part I. The electrocardiogram and its technology: a scientific statement from the American Heart Association Electrocardiography and Arrhythmias Committee, Council on Clinical Cardiology; the American College of Cardiology Foundation; and the Heart Rhythm Society. Circulation. 2007;115: 1306-1324.
23. Hnatkova K, Gang Y, Batchvarov VN, Malik M. Precision of QT interval measurement by advanced electrocardiographic equipment. PACE. 2006;29: 1277-1284.[Medline] [Order article via Infotrieve]
24. Hnatkova K, Malik M. Automatic adjustment of manually measured QT intervals in digital electrocardiograms improves precision of electrocardiographic drug studies. Comp Cardiol. 2002;20: 697-700.
25. Darpo B, Agin M, Kazierad DJ, et al. Man versus machine: is there an optimal method for QT measurements in thorough QT studies? J Clin Pharmacol. 2006;46: 598-612.![]()
CiteULike
Connotea
Del.icio.us
Digg
Reddit
Technorati What's this?
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH | TABLE OF CONTENTS |