Validation of Automated Coronary Autoplaque Quantification

Context

Designed and implemented the APQ algorithm for scan-specific attenuation thresholding, coordinated multi-center validation studies comparing automated results against manual expert tracing and Intravascular Ultrasound (IVUS) and performed statistical analysis (Bland-Altman, Pearson correlation) to determine algorithm accuracy and bias.

Primary Risks Identified

• Intersite Variability: Differences in CT scanner hardware and reconstruction kernels can significantly affect Hounsfield Unit (HU) values for plaque, making fixed thresholds unreliable.
• Manual Subjectivity: Expert readers often disagree on manual contour placement, leading to inconsistent "gold standards" for validation.
• Image Quality: Calcified segments can cause "blooming" artifacts, potentially overestimating calcified plaque volume.
• Validation Complexity: Comparing 3D CT data with 2D IVUS cross-sections requires precise spatial registration to ensure the same plaque segment is analyzed.

Equation Explanations

The APQ algorithm and validation studies utilize specific quantitative metrics to characterize plaque morphology and assess the clinical severity of coronary artery disease.

• Plaque Burden (%): (Total Plaque Volume / Vessel Wall Volume) × 100. This metric represents the percentage of the total vessel space occupied by plaque, where higher percentages (e.g., >38%) indicate significant atherosclerotic progression.
• Remodeling Index (RI): Vessel Area at the Lesion / Vessel Area at a Reference Point. An RI greater than 1.1 signifies "positive remodeling," which is often a marker for high-risk, vulnerable plaques.
• Area Stenosis (%): (1 - Minimal Luminal Area / Reference Luminal Area) × 100. This determines the functional severity of a blockage by quantifying how much of the original vessel lumen has been restricted.
• Noncalcified Plaque (NCP) Volume: Calculated by integrating voxels within the 3D vessel wall that fall between -30 and 300 HU. This identifies lipid-rich and fibrous "soft" plaques.
• Calcified Plaque (CP) Volume: Calculated using high-attenuation thresholds (typically >300 HU or >550 HU). This measures hardened calcium deposits that contribute to the total plaque volume.

Testing Approach

Testing focused on environment-driven failure analysis rather than pure functional coverage. I combined exploratory testing with log-driven diagnostics to isolate licensing and rendering failures. CI results were reviewed alongside local reproduction to distinguish configuration issues from product defects. Risk-based prioritization was used to validate startup stability and core workflows before secondary features.

• Phase 1 (Manual Comparison): APQ was tested on 29 focal plaques from 24 patients, comparing results to manual slice-by-slice quantification by two independent expert readers.
• Phase 2 (IVUS Comparison): To establish true accuracy, APQ was compared against Intravascular Ultrasound (IVUS) in 20 patients (22 plaques) who underwent both CCTA and IVUS within a 48-hour window.
• Statistical Metrics: Used correlation coefficients ($r$), $p$-values, and bias analysis to quantify performance.

Outcome

Over three release cycles, production incidents dropped noticeably and release discussions shifted from opinions to evidence.

• Rapid Processing: Reduced analysis time from several minutes (manual) to under 2 seconds per lesion.
• High Accuracy: Demonstrated an excellent correlation with IVUS ($r = 0.94$, $p < 0.001$) for noncalcified plaque volume.
• Improved Consistency: Showed significantly lower bias ($32.6\text{ mm}^3$ vs. $64.4\text{ mm}^3$) compared to interactive threshold adjustment (ITA).
• Reliability: Established strong correlation with expert readers ($R = 0.90$) for overall plaque composition.

Key Lessons

Quality maturity doesn't come from tools - it comes from aligning testing effort with real risk and making results visible.

• Dynamic Thresholding is Essential: Patient-specific and scan-specific thresholds for HU (Hounsfield Units) are more accurate than fixed values due to variations in luminal contrast.
• 3D vs. 2D: Volumetric (3D) analysis captures the true extent of disease more effectively than traditional 2D area-based stenosis measurements.
• Automation as a Gatekeeper: Reliable automated tools can reduce unnecessary invasive catheterization by providing a faster, more accurate risk assessment.