Automated all-in-one device for handling Magnetic Bead-Based Samples in immunoassays

Overall design architecture of MECHA

Figure 1a shows a standard workflow of ECL assay. The process begins by mixing a sample with target-specific magnetic beads (MBs) to capture the analytes of interest. The resulting MB-target complexes are then labeled with probe antibodies conjugated to the ECL luminophore [Ru(bpy)₃]²⁺. These labeled MBs are deposited onto an electrode together with the coreactant tripropylamine (TPrA) for signal generation (Figure. 1b). Upon applying a potential between the working and reference electrodes, both [Ru(bpy)₃]²⁺ and TPrA are oxidized, generating the highly oxidizing species [Ru(bpy)₃]³⁺ and the strongly reducing radical TPrA*. Their subsequent redox reaction forms the excited-state [Ru(bpy)₃]²⁺*, which emits light^10,18,19. This MB-based assay can be directly applied to native samples such as plasma or urine without purification, achieving a rapid turnaround time of ~1 hour. Furthermore, MBs can be magnetically concentrated on the electrode surface to enhance signal intensity^20,21.

We designed the MECHA system to automate sample preparation, including target capture, labeling, mixing, and washing. The system design mimicked the manual workflow by adopting a linear-well format (Figure. 1c): necessary reagents (e.g., MBs, antibodies) and wash buffers were preloaded in separate wells, and the MBs were sequentially transferred between these wells. The bead transfer was executed using a magnet rod encased in a removable plastic sheath. This rod captured MBs in one well and transported them to the next. MBs were released by retracting the magnet rod from the sheath, assisted by sonic agitation. Sample mixing within the wells was achieved by vortexing the entire well structure.

We constructed a prototype (Figure. 1d) compatible with a conventional 96-well plate (12 columns × 8 rows). Each row on the plate constituted the linear well path for a separate sample, enabling the system to run eight concurrent assays. To support this operation, an array of eight magnetic rods was installed on a mechanical actuator, and the plate holder was designed to perform an orbital rotation for sample mixing. The system featured a small form factor (w x d x h (in cm) = 33 × 44 × 26 ] as a benchtop machine. It included a touchscreen user interface and could run user-defined command scripts for customized workflows.

Matching the MECHA's throughput, we built an ECL setup for eight parallel measurements. An ECL chip was fabricated, comprising eight working electrodes, a shared counter electrode, and a shared reference electrode (Figure. 1e). This chip was mounted onto a custom socket (Figure. 1f) that incorporated pogo pins for electrical contacts and an embedded magnet array to concentrate MBs onto the working electrodes. During the ECL reaction, the chip was imaged using an electron-multiplying charge-coupled device (EMCCD) camera, and the luminescence intensity from each working electrode was extracted for quantitative analysis.

Design of the magnetic manipulators

Four types of MB manipulations are needed for automated sample preparation, i.e. bead capture, transport, release, and dispersion. To execute these functions, we designed and integrated (Figure. 2a), a magnetic manipulator, a plate translator, and an orbital shaker into MECHA (see Figure. S1 and Data S1-S4 for details).

The magnetic manipulator consisted of eight magnet assemblies, each comprising a vertically aligned pair of magnets. The upper magnet was rod-shaped, encased in a detachable plastic sheath (Figure. S2 and Plastic sheath.STL for plastic sheath); this assembly could be immersed in a sample to collect MBs. The lower magnet was positioned beneath each well to attract MBs downward (Bottom magnet holder.STL). Both magnets and the sheath were mounted on separate motorized stages, which independently operated along the z-axis (as defined in Figure. 2a, and PS holder part1&2.STL, Top magnet holder.STL for assembly of top magnet control module). The sheath stage also included a vibration motor that generated oscillatory motion (Figure. 2b). For lateral movement, the 96-well plate was mounted on another motorized stage that moved along the x-axis; this stage also served as an orbital shaker (Plate bed.STL, Shaking plate.STL, Cam 1 & 2.STL). All parts were assembled onto the MECHA base (Base plate.STL and Main bracket.STL) and then covered in the cases. (Base case.STL, Back case.STL, and Front case.STL).

Figure 2c illustrates the MB transfer sequence. (i) The initial MECH actuation involves stage movements to enclose the upper magnets with the sheaths (Video 1). (ii) The upper magnet assembly is immersed in the sample wells to collect MBs (Video 2). The lower magnet is located away from the well bottom. (iii) The upper magnet assembly is raised, and the plate is translated in the x direction, positioning MBs above the next well. (iv) The upper assembly descends into the well. The sheath is detached (Video 3), and the rod magnet is withdrawn. (v) The vibration module is activated to release MBs into the solution (Video 4). Concurrently, the lower magnet is raised to the bottom of the plate to accelerate MB recovery. (vi) The plastic sheath is raised to cover the upper magnet, and the lower magnet is retracted. The entire plate then undergoes orbital rotation to disperse the MBs.

The system control was implemented using a microcontroller integrated with position sensors and stepper motors. An embedded software, developed in Python, coordinated all actuator movements and timing sequences. Key operational parameters, including manipulator movements, assay duration, vibration period, vortex speed, and mixing time, were programmable through a touchscreen interface (Figure. S3) or by uploading command scripts.

Design of the orbital shaker

We chose an orbital shaker to generate uniform flow profiles across all 96 wells. The shaking module consisted of four eccentric cams, with a main cam (orbiting radius, 1.5 mm) driven by a direct-current (DC) motor (Figure. 2d). To determine the optimal operating conditions, we performed three-dimensional transient computational fluid dynamics (CFD) simulations. We used the actual well geometry (height: 10.67 mm; bottom diameter: 6.35 mm; top diameter: 6.86 mm) and the input sample volume (100 μL). The simulation model incorporated inertial forcing, gravity, surface tension, and wetting boundary conditions (see Methods) to evaluate free-surface deformation and internal flow structures over a range of rotational speeds. The design objective was to maximize MB dispersion while preventing liquid spillovers.

The simulation identified three hydrodynamic regimes (Figure. 2e and Figure. S4). At low rotational speeds (~600 rpm), circulation was weak, and the fluid was largely confined near the bottom of the well (Regime I; Video 5). At intermediate speeds (~600–1000 rpm), the flow displayed a toroidal vortex structure spanning most of the liquid volume (Regime II; Video 6): the closed circulation loop indicated continuous exchange of fluid between the well bottom and the free surface, thereby enhancing convective transport and bead dispersion. At higher speeds (≳1000 rpm), pronounced free-surface deformation increased the risk of spillovers (Regime III; Video 7).

Experimental observations were consistent with these predictions (Figure. 2f). At ≤400 rpm, MBs remained aggregated after 1 min of shaking. At 600 rpm, beads were mobilized but remained concentrated near the well center, indicating incomplete volumetric circulation. At 800 rpm, aggregated beads were uniformly dispersed throughout the well volume. Based on the combined CFD and experimental results, 800 rpm was selected as the operating condition for rapid MB dispersion.

Performance evaluation of MECHA in sample preparation

We next evaluated MECHA’s performance in automated sample preparation. A key metric was MB transport efficiency, defined as the ratio of transferred beads to input beads. This ratio was estimated by counting residual beads remaining in a well after the transfer step (Figure. S5). To reduce non-specific bead binding, we treated both the well plates and plastic sheath with 1% BSA (see Methods).

We tested magnetic beads of two sizes (2.8 μm and 1 μm) to evaluate the effect of bead size on transport efficiency. MECHA demonstrated strong performance with 2.8-μm magnetic beads, whereas manipulating 1-μm beads was suboptimal (Figure. S6). Magnetic capture force scales with particle volume, while viscous drag scales linearly with particle radius. Therefore, smaller beads experience reduced capture efficiency and slower transport velocities under identical field conditions. This limitation can be addressed by using a larger magnet, which will increase both the magnetic force and the bead-capture area.

In the current design, we opted to use 2.8 μm beads. The average bead transport efficiency between adjacent wells reached approximately 91% (Figure. 3a). After nine sequential transfers across the well array, the final bead retention was 47% (Figure. 3b), consistent with the inter-well transport efficiency. The retention values were highly reproducible, with a coefficient of variation (CV) below 5%.

We assessed the uniformity of ECL signals by processing multiple samples in parallel. A parent EV sample was aliquoted into eight input wells, and MECHA prepared eight ECL-ready MB complexes by completing automated steps: EV capture on MBs, washing, labeling with ECL probes, and final buffer rinses. ECL measurements exhibited high signal consistency (Figure. 3c). The CV values were <10% across varying input EV concentrations (Figure. 3d), highlighting MECHA’s capability for reproducible sample handling.

We compared ECL signal intensities between MECHA-prepared and manually processed samples. Two types of analytical targets, EVs (Figure. 3e) and soluble protein (IL-6; Figure. S7), were tested at varying concentrations. The immunoassay workflow involved five MB-transfer steps, corresponding to an expected final bead recovery of ~68%. Manual sample processing yielded a comparable recovery of ~75% (Figure. S8). Consistent with these results, ECL signals obtained from MECHA-processed samples showed strong agreement with those from manual preparation (Pearson’s r = 0.90), supporting the analytical equivalence of the automated workflow to conventional manual methods. Bland-Altman analysis supported this agreement, showing a mean bias of 0.025 ± 0.11 (95% limits of agreement: −0.19 to 0.24), indicating negligible systematic deviation between the two preparation methods (Figure. S9).

EV assays using MECHA-prepared samples

We optimized the MECHA protocol for EV protein detection through ECL assays (see Table S1 for operation parameters). The sample wells were preloaded with reagents for sequential automated processing (Figure. 4a). For EV capture, MBs (diameter, 2.8 μm) were functionalized with antibodies against CD63, a tetraspanin expressed on EV surfaces. To generate probe antibodies conjugated to ECL reporters, we employed a click-chemistry approach using half-antibodies (Figure. 4a, bottom). Full antibodies were first reduced to cleave interchain disulfide bonds, exposing free sulfhydryl groups on the heavy chains. These thiols were then reacted with a maleimide-functionalized [Ru(bpy)₃]²⁺ complex. This conjugation approach consistently produced a labeling ratio of 0.6 [Ru(bpy)₃]²⁺ per half antibody (Figure. S8).

After MECHA processing, labeled MBs were mixed along with the coreactant TPrA and deposited onto an electrode for ECL measurements. An electrical potential of 1.2 V was applied between the working and counter electrodes to trigger light emission. ECL microscopy (see Methods) confirmed successful sample preparation, revealing individual MBs with strong luminescence signals (Figure. 4b). We subsequently used MECHA to process samples with varying EV concentrations. ECL measurements were performed on the custom-designed multiplex chip. The signal level was dose-dependent (Figure. 4c), indicating that quantitative EV detection was achieved using MECHA-prepared samples.

Using a CD63-specific ECL probe, we constructed an EV titration curve based on the mean ECL intensity per electrode (Figure. 4d). The assay achieved a limit of detection (LOD) of 4 × 10⁴ EVs/mL (technical replicates, n = 3) and a dynamic range spanning four orders of magnitude. The ECL assay outperformed conventional ELISA in both sensitivity and dynamic range, with an LOD approximately 1,800 times lower and a dynamic range more than 100 times wider.

Application demonstration using EV proteins in clinical samples

As a pilot utility study, we applied the MECHA-ECL pipeline to profile EV proteins in clinical plasma samples, with the demographic information presented in Table S2. Figure 5a shows the study design. Plasma was collected from 37 patients with advanced colorectal cancer (CRC) undergoing preoperative chemotherapy with the FOLFOX regimen (folinic acid, fluorouracil, and oxaliplatin). After completing a 2-month treatment course, all patients underwent curative surgery. Surgical tissue specimens were evaluated by histopathology to assess therapeutic response to chemotherapy, classifying 20 patients as good responders and 17 as poor responders. Longitudinal plasma samples were obtained at three defined time points: before chemotherapy, after chemotherapy, and after surgery. MECHA-ECL analysis was performed to quantify treatment-associated changes in EV marker expression.

For EV capture, we prepared MBs functionalized with antibodies against CD63 or CD9. A mixture of CD63- and CD9-conjugated MBs was used to capture EVs directly from plasma. The captured EVs were profiled for six CRC-associated surface markers: epithelial cell adhesion molecule (EpCAM), epidermal growth factor receptor (EGFR), glycoprotein 33 (GPA33), multidrug resistance protein (MDR), carcinoembryonic antigen (CEA), and prominin-1 (CD133). These markers are previously validated as EV biomarkers for CRC diagnostics¹⁴. Two additional targets were included: CD63 to estimate EV loading and control IgG to account for nonspecific background signals. CD63 was included to estimate total EV loading, and IgG served as a nonspecific control. For each marker (M), the net signal was defined as ΔI_M = I_M − I_IgG. Marker expression was normalized to EV abundance as ΔI_M/ΔI_CD63, with CD63 expression serving as an estimate of EV amount¹⁴.

Figure 5b shows the normalized expression levels of EV markers before and after chemotherapy, stratified by treatment response determined from histopathology. Although marker expression was generally elevated in poor responders relative to good responders, no single marker provided sufficient discriminative power. To identify an EV signature prognostic of therapeutic response, we evaluated all candidate marker combinations using logistic regression applied to pretreatment samples. We then selected the optimal model that simultaneously minimized the Akaike information criterion (AIC) and maximized the area under the receiver operating characteristic (ROC) curve. This combined criterion identified the EGFR-CD133 pair as the most informative and parsimonious marker combination (Figure. S9). The resulting model yielded an AUC of 0.74 (Figure. 5c), indicating fair predictive performance.

We applied the same regression model (EGFR + CD133) to calculate risk scores for pre- and post-surgery plasma samples (Figure. 5d). All CRC patients exhibited decreased risk scores after surgery, likely reflecting a reduction in tumor burden. This observation was consistent with previous studies that reported the decline of CRC EV markers following tumor resection^14,22.

MATERIAL

4-(dimethylamino)pyridine (4-DMAP, 98%), Amicon centrifugal filter (3 kDa, 0.5 mL), Tripropylamine (TPrA), tris(2-carboxyethyl)phosphine hydrochloride (TECP), Aminomaleimide, Dimethyl sulfoxide (DMSO), and Ru(bpy)₂(mcbpy-O-Su-ester)(PF₆)₂ were purchased from Milipore Sigma. Bovine serum albumin, magnetic beads (1μm and 2.8 μm), 1xphosphate buffer saline (PBS), and 96-well plate were purchased from Thermo Fisher Scientific. Size-exclusion chromatography (SEC) column was purchased from iZon. ECL chip was printed from UXN. The submitted drawing was manufactured through the following machining: 3D Printing parts (ABS): Back case.STL, Base case.STL, bottom magnet holder.STL, Cam 1.STL, and Cam2.STL (Data S1); CNC Machining parts (Aluminum): Base plate.STL, Main bracket.STL, plate bed.STL, PS holder part1.STL, PS holder part2.STL, shaking plate.STL, and Top magnet holder.STL (Data S2); Injection molding (polystyrene): Plastic sheath (PS).STL (Data S3); Laser cutting and acrylic bonding (acrylic plate): Front case.STL (Data S4).

MECHA design and fabrication

The system was designed to process eight samples simultaneously in a 96-well plate format (see Figure. S1 for details). The assembly included one motorized x-axis stage for the sample plate and three independently driven z-axis stages supporting two magnet modules and removable plastic sheaths. Two vibration motors were mounted on the sheath stage, and an additional motor was attached to the plate stage to generate orbital vortex motion. Metal components were fabricated via computer numerical control (CNC) machining. The device’s plastic parts were produced by 3D printing or laser cutting. The plastic sheath was fabricated by injection molding. Embedded control software was developed in Arduino C++ and deployed on a Teensy 4.0 microcontroller. A touchscreen interface (Nextion HMI LCD touchscreen) was integrated to provide real-time user interaction and system monitoring. Through this interface, users can adjust key operational parameters, including assay time, vibration duration, vortex speed (revolutions per minute, RPM), and vortex time. Each assay sequence was executed along the column direction of the 96-well plate.

ECL chip fabrication and detector setup.

The electrode pattern for the ECL chip was designed using EAGLE software (Autodesk) and patterned onto a gold film by UXN Research Center (UXN, South Korea). To facilitate handling, the thin gold film was supported by a 1 mm-thick acrylic plate, which was laser-cut to the desired size and bonded to the back of the gold film using double-sided adhesive (468MP, 3M). A second, donut-shaped acrylic plate (3–4 mm thick) was also laser-cut and attached to the front side of the gold film, precisely aligned with the reaction electrode area, to form a reservoir for the ECL reagent. The ECL connector was fabricated using a 3D printer and assembled with pogo pins to interface with the potentiostat (Digi-Ivy Inc, DY2216). The detection module consisted of an EM-CCD camera (Falcon) equipped with a 2–6× zoom lens to capture the ECL signal emitted from the electrode surface.

CFD modeling of orbital shaking

Three-dimensional transient CFD simulations were conducted to optimize orbital shaking conditions in a 96-well plate. The modeled geometry replicated a standard well (height 10.67 mm; bottom diameter 6.35 mm; top diameter 6.86 mm) with a working liquid volume of 100 μL, corresponding to an initial liquid height of approximately 3.09 mm. The incompressible, unsteady Navier–Stokes equations were solved using a transient finite-element framework implemented in COMSOL Multiphysics (COMSOL Inc., USA). Orbital motion was introduced as a time-dependent body acceleration representing circular translation of the plate^30,31. The liquid-air interface was captured using a phase-field method based on the Cahn–Hilliard formulation, with surface tension incorporated through a free-energy functional^32,33. Wetting at the polystyrene wall was represented by a prescribed static contact angle of 87.4°. No-slip boundary conditions were applied to solid surfaces, and the top boundary of the air domain was set to atmospheric pressure. Simulations were advanced until a periodic steady state was reached, after which velocity fields and free-surface morphologies were extracted for analysis.

Magnetic bead preparation

Two milligrams of streptavidin-coated magnetic beads (Dynabeads M-280 Streptavidin, Invitrogen) were suspended in 1× phosphate-buffered saline (PBS) solution (500 μL) at 20 °C and washed three times (1× PBS). The beads were then resuspended in 200 μL of 1× PBS. This suspension was divided into two 100 μL aliquots. To one aliquot, 20 μg of biotinylated anti-CD63 antibodies (Ancell) was added, and to the other, 20 μg of biotinylated anti-CD9 antibodies (Abcam) was added. The resulting mixtures were incubated for 2 hours (20 °C) with continuous rotation. After incubation, the beads were washed three times with 1× PBS, and each antibody-conjugated bead solution was resuspended in 50 μL SuperBlock (Thermo Fisher Scientific) in PBS.

Antibody cleavage and preparation of the ruthenium complex

Antibody disulfide bonds were reduced to generate free thiol groups. Briefly, 10 μL of antibody solution (1 mg/mL) was diluted with 180 μL PBS (pH 7.4), followed by the addition of 10 μL of tris(2-carboxyethyl)phosphine hydrochloride (TCEP; 100 mM in deionized water, MilliporeSigma). The mixture was incubated at 25 °C for 30 min with gentle mixing using a HulaMixer (Thermo Fisher Scientific). For preparation of the maleimide-functionalized ruthenium complex, 0.5 μL of amino-maleimide (50 mM in dimethyl sulfoxide, DMSO) was mixed with 5 μL of Ru(bpy)₂(mcbpy-O-Su-ester)(PF₆)₂ (4.927 mM in deionized water), followed by dilution with 194.5 μL of 1× PBS to yield a maleimide-activated [Ru(bpy)₃]²⁺ solution.

Ruthenium conjugation with cleaved antibody

The reduced antibody was mixed with the maleimide-functionalized [Ru(bpy)₃]²⁺ and incubated at 25 °C for 1 h in the dark with gentle mixing using a HulaMixer (Thermo Fisher Scientific) to allow thiol–maleimide coupling. Following the reaction, the mixture was purified twice using a 3 kDa molecular weight cutoff centrifugal filter (Amicon Ultra, MilliporeSigma) to remove unreacted ruthenium complex. The purified antibody–[Ru(bpy)₃]²⁺ conjugate was diluted to 10 μg/mL in 0.1% bovine serum albumin (BSA) and stored at −20 °C until use. Antibodies used in this study are listed in Table S4.

Sample preparation sequence

The automated sample processing was performed using the MECHA system (see Table S1 for the operation settings). To reduce non-specific bead binding, we treated both the well plates and plastic sheath with 1% BSA for 30 min at 20-24 °C before use. A 96-well plate was preloaded with reagents according to the programmed sequence: mixed MB solution (2 μL), anti-CD63 and anti-CD9 antibodies (1 μL each), wash buffers, and labeling reagents. EV capture. EV samples (100 μL of EV-spiked PBS or 100 μL of plasma) were loaded into the MB well, and the mixture was incubated for 30 min with vortexing (200 rpm). Washing. MBs were magnetically collected and transferred through a well containing 0.05% PBS-Tween 20 (PBS-T). EV labeling. MBs were transferred to a well containing 100 μL of [Ru(bpy)₃]²⁺ labeled antibodies (2 μg/mL in 0.1% BSA) and incubated for 15 min. Final wash and collection. Labeled MBs were washed three times with 1× PBS and resuspended in 30 μL of PBS for ECL measurement.

ECL measurement

For ECL measurements, 7 μL of each prepared MB suspension was deposited onto the working electrode (8 electrodes in total), followed by the addition of 1 mL of 180 mM tripropylamine (TPrA) in PBS (pH 7.4). ECL signals were recorded using the custom-built detection system described above. Chronoamperometric measurements were initiated, during which light emission from the electrode surface was acquired and quantified as the ECL intensity.

Cell culture and EV isolation

SW480 and SW620 colorectal cancer cell lines were obtained from the American Type Culture Collection (ATCC) and maintained in Dulbecco’s Modified Eagle Medium (DMEM; Cellgro) supplemented with 10% fetal bovine serum (FBS) and 1% penicillin–streptomycin (Cellgro). Cells were routinely tested and confirmed to be free of mycoplasma contamination using a MycoAlert Mycoplasma Detection Kit (Lonza, LT07-418). Cells at passages 1–20 were cultured in complete medium until they reached ~80% confluency, then incubated for 48 hours in medium containing 5% EV-depleted FBS. Conditioned medium was collected, sequentially filtered through a 0.2 μm membrane (Millipore), and centrifuged at 300 × g for 10 min to remove cell debris. The supernatant was concentrated to 0.5 mL using a Centricon Plus-70 centrifugal filter unit (10 kDa cutoff; Millipore) and loaded onto a qEV 70 nm size-exclusion chromatography (SEC) column (iZon). During elution, 4 mL of 1× PBS was added in increments of 100–200 μL. The first 3 mL of eluate was discarded, and the subsequent 1.5 mL fraction, containing EVs, was collected. The collected EVs were further concentrated using an Amicon Ultra-2 centrifugal filter unit (Millipore) and stored at -80 °C until use.

Clinical samples

Clinical plasma samples were collected under protocols approved by the Institutional Review Board (IRB) of Massachusetts General Hospital (2019P003441 and 2019P003472; principal investigator, H.L.) and Chonnam National University Hwasun Hospital (CNUHH 2019-228; principal investigator, C.H.K.). All procedures were conducted in accordance with institutional guidelines. Written informed consent was obtained from all participants. Peripheral blood samples (~10 mL) were collected from patients with advanced colorectal cancer at three defined time points: before chemotherapy, after chemotherapy, and within 5 days of surgery. Samples were centrifuged at 2,000 × g for 3 min to separate plasma from the red blood cell and buffy coat layers. The resulting plasma fractions were aliquoted and stored at -80 °C until use.

Tumor tissue analysis

Tumor regression was evaluated on surgical specimens obtained after preoperative chemotherapy. Histopathological analysis was performed at Chonnam National University Hwasun Hospital. Tissue sections were stained with hematoxylin and eosin (H&E) and graded according to the American Joint Committee on Cancer / College of American Pathologists (AJCC/CAP) tumor regression grading (TRG) system. Tumor response was classified as follows: TRG 1, near complete response (<10% residual tumor); TRG 2, moderate response (10–50%); TRG 3, minimal response (>50%); and TRG 4, poor or no response (extensive residual tumor without regression). For analysis, TRG 1–2 were grouped as good responders and TRG 3 and 4 as poor responders.

Data processing and statistical analysis

ECL intensity values were obtained for each target marker (I_M) and the IgG control (I_IgG). The net signal for each marker was calculated as ΔI_M = I_M − I_IgG. Marker expression levels were normalized to CD63 as ΔI_M/IΔ_CD63. For statistical evaluation, data were analyzed using GraphPad Prism (version 10.4) and R (version 4.2.2). Comparisons between groups were performed using unpaired two-tailed t-tests unless otherwise specified. To identify predictive markers of chemotherapy response, logistic regression was performed using marker data collected prior to chemotherapy. Risk scores were computed from regression coefficients, and receiver operating characteristic (ROC) curves were generated to evaluate model performance. The area under the ROC curve (AUC) was used as a measure of predictive accuracy.