A feedback-driven brain organoid platform enables automated maintenance and high-resolution neural activity monitoring

2.1. An integrated microfluidic, electrophysiology, and imaging organoid research platform

We have developed an integrated platform (Fig. 1) that automates organoid culture and data collection in individual microenvironments. While microfluidics (Fig. 1a) controls the media environment, digital microscopy captures the morphogenic features. The neural activity is recorded by local field potential measurements using complementary-metal-oxide semiconductor (CMOS) high-density microelectrode arrays (HD-MEA) [62] (Fig. 1b). The Internet of Things (IoT) cloud network brokers the communication between all devices and facilitates data storage, processing, and presentation services including an interactive webpage (Fig. 1d). Through touch-free automation, samples remain undisrupted in the incubator, increasing the consistency of images and allowing for higher frequencies of feeding and recording.

At user-defined intervals, conditioned media is aspirated by a syringe pump through a system of distribution valves (Fig. 1a), stored in a collection reservoir (without passing through the syringe pump vial) (Fig. 1c), and replaced by an equivalent volume of fresh media. Both types of media are perfused through flexible fluorinated ethylene propylene (FEP) tubing at 110 mm/s, which leads to low shear forces [63] (see Materials and methods, Microfluidic cell culture). This equates to a flow rate of 44.1 μL/s.

The digital microscope (Fig. 1e) is attached using 3D-printed parts on aluminum posts. The 3D printed culture chambers integrate the microfluidics and HD-MEAs. A liquid-impermeable O-ring gasket ensures media retention inside the chamber. The well lid includes a polished glass rod submerged in the media, improving image quality and removing the effects of condensation. Alignment grooves in the glass rod lid prevent rotation and incorrect fitting. The lid exchanges gas with the incubator conditions through ventilating air ducts (Fig. 1g), similar to a cell culture well plate. The removable and re-attachable lid reduces manufacturing complexity and enables future use of other lids with applications beyond imaging.

Fig. 1c shows the cross-section of the culture chamber attached to the HD-MEA. The media flows in (red) and out (blue). The sinuous media path and well geometry ensure minimum disturbance to the biological sample [63]. Fresh media is delivered on top of the volume present in the chamber, similar to partial media changes found in manual feeding protocols [64,65]. The ideal operating range is between 350 to 700 μL (see Supplementary Materials and Methods, Figure 1 and Table 1 for numerical volume limits). In the case of over-aspiration, media drops to a minimum of 170 μL before aspirating air from the chamber’s headspace. The 3D-printed catch tray guards against overflow, collecting up to 1.5 ml (200% of the chamber’s capacity) to protect the recording equipment from liquid damage.

2.2. Computer vision for microfluidic flow feedback

We developed a computer vision volume estimation system to monitor the accumulation of aspirated media and identify anomalies during culture feeding events. A critical technical challenge arose from media crystallization and debris accumulation, which could obstruct microfluidic lines during aspiration. We resolved this by implementing a computer vision-based feedback loop that flagged incomplete aspiration cycles, triggering corrective actions until the target volume was achieved or user assistance was requested. These safeguards enabled autonomous error recovery and prevented feed cycle failure throughout the study. Fig. 2a–b provides a detailed view of the setup inside a refrigerator, which includes three main components: a collection reservoir support system, an LED panel, and a camera module (see Materials and methods, 3D-printed components). The camera module remains on standby for image capture requests made by other IoT devices or users. Upon request, computer vision techniques are employed to estimate the media volume within the reservoirs accurately. This circuit converts camera-captured images into quantitative volume measurements in real time, enabling closed-loop feedback to the pump during culture maintenance.

Fig. 2c shows the computer vision process (see Materials and methods, Computer vision for fluid volume estimation) for segmenting area related to the media in the reservoir. A calibration was required to establish the relationship between the segmented area in pixels and volume in milliliters. We captured 184 images of the collection reservoirs containing volumes of media ranging from 0 to 12 mL (several pictures for each volume), with each volume confirmed by a scale, accurate to 1 μL. This calibration process yielded a robust polynomial mapping between segmented pixel area and true volume. We trained two regression models to account for the conical and cylindrical regions of the reservoir, achieving high fidelity across the full 0–12 mL range. For each specific volume in Fig. 2d, multiple points overlap and are all accounted for to calculate the polynomial regression lines. To accommodate the reservoir’s conical section (volumes <1.5 mL) and cylindrical section (volumes >1.5 mL), two distinct regressions were applied, ensuring a high degree of precision for each geometrical shape.

A Leave-One-Out cross-validation (LOO) [66] approach was employed to quantify the model’s error. This method tests the model’s accuracy and generalizability in an unbiased manner, ensuring that the calibration results in a model that performs reliably across different samples. The effectiveness of the model is assessed quantitatively with the following metrics: an average Mean Absolute Error (MAE) of 0.56% (equivalent to 27 μL), an average standard deviation of errors at 0.53% (22 μL), and an average Root Mean Square Error (RMSE) of 0.77% (35 μL). The polynomial models exhibit R-squared values of >0.99, denoting an optimal fit of pixel area to liquid volume. Fig. 2e shows the average absolute error percentage at a specific volume, with the bar indicating the error range from minimum to maximum.

2.3. IoT infrastructure creates an ecosystem of devices and cloud-based services

We built a cloud-based IoT ecosystem that enables communication between users, devices, and services to implement actions, record data, and streamline upload, storage, and analysis. All devices (here: pumps, microscopes, and microelectrode arrays) run software using the device-class Python framework (Fig. 3a and Supplementary Materials and Methods). Devices operate collectively with shared core software and complementary behaviors: they can request jobs from each other, yield during sensitive operations, and ensure collaborative functions and smooth operation (Fig. 3d). One challenge during multi-device coordination was managing asynchronous job execution across devices (e.g., electrophysiology, fluidics, imaging). We addressed this by implementing a device-level state queue that logs jobs and defers them opportunistically, ensuring time-sensitive tasks (e.g., recordings) were uninterrupted while preserving non-conflicting tasks (e.g., uploading). This prevented job loss and allowed error-tolerant, coordinated automation across modalities. Devices update their shadow in the database whenever their state information changes (i.e, assigned experiment, schedule, current job and estimated completion time, and other dynamic variables) to eliminate the need for device polling. Messages (i.e., job requests) between devices and services are sent through a centralized MQTT broker via the publish/subscribe protocol. This decoupled architecture allows for independent and extensible deployment of components. Data generated by devices is immediately uploaded to an S3 object storage in a predefined structure using an experiment Universally Unique IDentifier (UUID) as the top-level key. Uploading large electrophysiology datasets presented bandwidth-related challenges due to inconsistent internet connectivity. We developed a file queue system with automatic retry and checksum validation to ensure upload fidelity and prevent data corruption or loss during transfer. A ‘metadata.json’ file stores experiment details, sample information, notes, and an index of the produced data. Raw data is stored separately from analyzed data under different sub-keys. Cloud jobs, which operate as shared services, process raw data from S3 and write results back to S3, reporting status via MQTT messages. To utilize the IoT ecosystem, users initiate experiments, control devices, and visualize data through a website (see Materials and methods, Website and screenshots in Supplemental Figure 2), with the typical user workflow in Fig. 3c.

2.4. Automated study of cerebral cortex organoids

The integrated research platform was used to study the effects of automation on the neuronal activity of pluripotent stem-cell-derived mouse cerebral cortex organoids. Embryonic stem cells were aggregated, patterned, and expanded to generate organoids using a previously defined differentiation protocol [57,67]. Day 32 post-aggregation, organoids were plated two-per-chip directly onto HD-MEAs. Two were plated to maximize use of the HD-MEA surface (3.85 × 2.10 mm²). For the 7-day study, 8 chips across two batches were split into groups that were fed and recorded with standard manual procedures (Controls, N = 4), manual feeding and automated recording (AR, N = 1), automatic feeding and manual recording (AF, N = 1), or automatic feeding and automatic recording (AFAR, N = 2). Manual feeding for the Control and AR groups involved aliquoting and pre-warming fresh media within a biosafety cabinet, transferring the culture from the incubator to the cabinet, aspirating conditioned media by pipette, replenishing with fresh media, and returning the sample to the incubator. Automated feeding for the AF and AFAR groups involved microfluidic aspirating of conditioned media and microfluidic dispensing of fresh media while the sample remained undisturbed in the incubator. All chips were imaged in the incubator every hour, each using a dedicated upright digital microscope (DinoLite).

Automated microfluidic feeds were used to increase the consistency and frequency of cell culture media replacement. We removed conditioned cell supernatant from the well and dispensed the equivalent volume of fresh media for each feed cycle. The controls had 1.0 mL media replacement every 48 h, consistent with standard protocols. AF and AFAR were placed on a protocol in which 143 μL media were replaced every 6 h, matching the total media volume turnover across groups for the 7-day study. The schedule of automated media feeds was defined at the experiment’s launch and initiated by a timed feeding job command sent to the microfluidic pump. The fidelity of feeding was controlled through a computer vision volumetric feedback loop on the aspirated conditioned media (Figs. 2, 4a). To assess operational efficiency, we quantified feedback performance across microfluidic feeds. The system was designed to maintain a strict schedule of four feeds per day (28 total replenishment cycles over the 7-day study). For AFAR 1, 61% of the scheduled replenishment cycles (17 of 28) triggered feedback corrections, comprising 81 aspiration, 11 dispense, and 9 pull actions. Feedback events per trigger ranged from 1 to 10 with a mean of 6. For AF, 54% of replenishment cycles (15 of 28) required feedback, with 69 aspirations, 31 dispenses, and 22 pulls, and a slightly higher mean of 8 feedback actions per trigger (range 2–11).

Conditioned media has a high protein content, contains cellular debris, and is susceptible to forming salt crystals [68,69]. In microfluidic systems, this leads to clogs, error accumulation, and failure modes [70]. To overcome this, a volume estimation feedback loop was initiated each time the pump performed a job. At the time the medium was perfused to/from a specific well, the pump sent a job request to the camera module responsible for imaging the well’s collection reservoir. The image was captured, uploaded to the cloud, its volume estimated by the computer vision Estimator, and returned to the pump for feedback interpretation. The pump compared this measured volume with its expected target and responded accordingly: initiating further aspiration or dispensing jobs if a discrepancy exceeded the tolerance range, or logging success if the measurement matched. This closed-loop interaction occurred autonomously without user intervention, with exception handling only triggered if repeated corrections failed. Within tolerance, the action was declared a success (marked as a green check mark in Fig. 4a), and no further action was taken. Outside of tolerance, the pump scheduled itself a new job proportional to the volume discrepancy and in relation to the number of previous feedback attempts (see Materials and methods, Feedback interpreter).

The system was designed to resolve errors in media exchange using feedback. If feedback actions exceed set thresholds, which were perfusion of >2 mL or >20 retry attempts, the system sends an alert via Slack and halts operations. This was designed to prevent overflow or underflow events and ensure media volumes remain within tolerance. In both 7-day experiments, no interventions were required.

The automated feeding and feedback results for AF and AFAR 1 are visually represented in Fig. 4b–d. Fig. 4c shows the traces of expected volume and computer vision estimated volume for AFAR 1 (left) and AF (right) for the 7-day study (Days 5 to 12 post-plating). There was a collection reservoir change on Day 8 in which the 15 mL conical was replaced with a fresh tube. In both samples, the drop in estimated and expectation reflects the collection reservoir exchange. For AFAR 1 (Fig. 4b, left), a zoomed-in view of the feedback loop following the scheduled feeding cycle at 7:12 on Day 9 highlights feedback actions taken to remedy a volumetric discrepancy. In this instance, the volume estimation was less than expected after the feed cycle. Five consecutive aspiration jobs were carried out, and the estimated volume still remained under expectation. At the 6th iteration of feedback, a pull job was sent to the pumps, which raised the collection volume above the expected volume. In the 7th and 8th iterations of feedback, two dispense jobs were engaged to supplement the well for the over-aspiration. In a similar case, for AF (Fig. 4b, right), a total of 6 iterations of feedback were engaged to bring the estimated volume into tolerance with the expected volume; however, in this example, no dispense jobs were required. Fig. 4d shows histograms of the sum of pump events per day by subcategory. Each feeding cycle (four per day) was scheduled, and all other events occurred through feedback.

2.5. High-frequency HD-MEA recordings and automated feeding do not disrupt neuronal activity

To evaluate organoid neuronal activity, extracellular field potentials were measured using 26,400 electrode HD-MEAs, which can record up to 1020 electrodes simultaneously. We conducted daily activity scans to monitor neural activity. Activity heat maps derived from the first and final activity scans for each sample are presented in Fig. 5b, with an outline of the organoid edge based on alignment of corresponding microscopy images. To optimize electrode coverage, we generated specific configuration files for electrode selection based on the regions with the highest activity, which remained constant for seven of the eight chips. In one case (AF, Day 32+6), we updated the configuration due to the emergence of a new high-signal area on the second day of recording. Stable configuration maps allowed for automated electrode recordings over days, optimizing long-term analysis.

Manual recordings involved an experimenter placing each HD-MEA on the recording unit and initiating 10-min recordings via software. In contrast, the hourly recordings (AFAR 1-2 and AR) featured the HD-MEA remaining on the recording unit while automated software handled the entire process, from power management to data uploading. AFAR 1-2 and AR each amassed over 158 recordings, totaling over 26.3 h (525 GB) of electrophysiology data per sample. Conversely, all manually recorded samples (Controls 1-4 and AF) accumulated 7 recordings each, amounting to 1.2 h (23 GB) of electrophysiology data. The studies generated over 2 TB of data from over 100 h of mouse organoid recordings.

From these data, we analyzed the effects of our automated microfluidic, imaging, and recording system on the neuronal activity of the brain organoids housed therein. Imaging of the chips from above (Fig. 5a) allowed us to align the body of the organoid with neural activity (Fig. 5b). In some instances, such as in Control 1, neurite outgrowths were evident in the images and activity scans.

Initial activity scans were used to distribute samples into experimental and control conditions. We sought an even distribution with respect to baseline activity to reduce biases from initial conditions. Groups were assigned based on median unit counts and neural activity. Both Controls and Automated samples included organoids with fewer than 100 units and organoids with greater than 100 units. Samples with fewer than 25 units were excluded to ensure reliable electrophysiology analysis. Neural units identified from initial recordings by spike sorting with Kilosort2 [71] were: Control 1: 87 units, Control 2: 292 units, Control 3: 144 units, Control 4: 80 units, AR: 173 units, AFAR 1: 43, AFAR 2: 250, AF: 29. Three chips were omitted from the study for having a unit count less than 25. Throughout the 7-day study, all samples with unit counts 29 to 80 (Control 4, AFAR 1, and AF) increased their detected unit counts during the 7-day study, while all samples with unit counts 87 and above (Control 1-3, AR, and AFAR 2) decreased their detected unit counts (Fig. 5c).

The average firing rate per neural unit increased for every sample, irrespective of feeding or recording schedules (Fig. 5c). All samples started with an average firing rate of 1.95 Hz ($\sigma =0.48$), increased by 0.24 Hz per day ($\sigma =0.09$), and concluded with an average of 3.98 Hz ($\sigma =1.22$). The automated conditions (AR, AFAR, and AF) presented no divergence from the controls on neural unit count, firing rate, or morphology resulting from increased frequency of feeding and/or increased frequency of recording.

2.6. High-frequency HD-MEA recordings reveal dynamic neuronal activity states in organoids

The hourly recorded conditions (AFAR 1-2 and AR) revealed transient states, not apparent with single daily recordings (Fig. 5c–e). Linear regression trendlines comparing the hourly and daily recordings for a single sample are congruent, however daily recordings do not capture the prominent oscillatory dynamics of neuronal unit count and median firing rate captured by the hourly recordings. Median firing rates were observed to fluctuate as much as 3-fold over the course of a day and are not well-characterized by linear regression fit (AFAR 1 R² = 0.31, AFAR 2 R² = 0.42, AR R² = 0.69).

We inspected the effect of feeding on these dynamics. The AFAR samples had a six-hour automation cycle (Fig. 5f) that included one 143 μL feed and six 10-min recordings. We examined effect by aligning recordings to a six-hour ‘time since feed’ cycle. Fig. 5g and h present the composite graphs of aggregated neuronal firing rates of AFAR 1 and AFAR 2, comprising 26 feeding cycles with all recordings binned with respect to their time since feeding. The superimposed feeding cycles did not show a trend in units (not shown) nor firing rate in relation to feeding cycles. The variance presented in Fig. 5d and e do not align with the six-hour feeding cycle.

2.7. Effects of feeding during recording

We further investigated the effect of feeding with a follow-up experiment that included microfluidic feeds during electrophysiology recordings to capture the immediate response on neuronal activity to a media injection. The AR sample from the 7-day study (Fig. 5) was equipped with a microfluidic culture chamber and set on a new feeding and recording schedule. Automated 15-min recordings were performed every hour for 36 h. Feedings occurred every third hour that began at minute 5 of the ongoing recording (Fig. 6a). Each feeding cycle was defined as an aspiration and dispense of 150 μL. Automated feeds increased in their number of cycles each third hour for four experimental conditions (1 cycle = 150 μL, 2 cycles = 300 μL, 4 cycles = 600 μL, and 6 cycles = 900 μL). Each of these conditions were performed three times (Fig. 6b).

Fig. 6c presents neural unit raster plots and the average neuron firing rate for representative recordings of each condition. Raster plots of neural unit firing over time show no change in unit activity during microfluidic manipulation (light purple overlay). The average firing rate did not show unusual variability during or after the feeding window and did not correlate to pump actions (Dispense, Aspirate, or Cycle). To further inspect this computationally, we performed normalization and z-score analysis (detailed description in Materials and methods, Normalization and z-score calculation for effect of feeding during recording). Within each recording, a 90-s sliding window was applied to the spike raster with 1-s steps. The firing rate of a single unit at any particular window was normalized to the same unit’s average firing rate across all windows within a recording. This scales changes in activity for any neural unit to be comparable irrespective of average firing rate. For all non-feeding recordings (N = 28), we calculated the firing rate mean and standard deviation (STD) of all normalized units in each window. The results of the experimental conditions (Fig. 6d) (N = 3 per condition) were calculated with z-scores generated for each neural unit in each window to relate how much firing rate changed with respect to baseline variability. The STD of ±1 in relation to the non-feed activity are marked (dashed blue line). In the 10 min following the onset of microfluidic feeding, the largest increase in firing rate was +0.7 STD above the mean at 3.9-min (150 μL condition), largest decrease was 0.6 STD below the mean at 0.8-min (300 μL condition). Z-scores for all conditions (150 μL, 300 μL, 600 μL, and 900 μL) remained within ±0.7 STD showing no significant change in activity during microfluidic manipulations.

Feed cycles of different total volume and time length did not elicit immediate changes in firing rates during or minutes after feeding. Combined with our findings in Fig. 5 showing no changes in firing rate distributions for six hours following feeding cycles, these results suggest that neural activity remains stable despite media exchange and the associated fluid movement.

4.1. Assembled devices and custom 3D-printed components

The Bill of Materials listing components and costs are provided in Supplementary Materials. STL files for 3D printing are provided in PrintedAccessories.zip. All parts were printed using biocompatible resins on a Formlabs Form 3B+ printer with 100 μm layer height, post-processed for 15-min with isopropyl alcohol in the Formlabs Form Wash, and UV cured for 15-min at 60 °C in the Form Cure. The sterilization of the 3D printed parts was completed by a 30 min dry autoclave cycle at 121 °C. See Supplementary Materials section 3D-printed components for more details.

4.2. Microfluidic cell culture

The automated microfluidic pump system builds on previous work [63]. The microfluidic system was configured for this study to support two chips (AF and AFAR) and their respective collection reservoirs (right and left) were imaged by the camera setup. Replicates of the conditions were achieved by repeating the experiment on the following batch of organoids.

Fresh cell culture media is kept at 4 °C refrigeration and accessed by the pump through flexible FEP tubing routed into a benchtop refrigerator and to a media bottled with a reagent delivery cap (Cole-Parmer VapLock). Fresh media is kept refrigerated to increase longevity and may be replaced during experimentation. To dispense, the syringe pump and distribution valves draw fresh media into the syringe vial and distribute the programmed volume into flexible FEP tubing routed through an access port in the incubator. Here, the media is heated in incubator conditions prior to being delivered to the organoid inside the culture chamber. To keep media dispenses available on demand, a preheated 450 μL reserve (59% of the chamber’s volumetric capacity) of fresh media remains idle in the FEP tubing so that upon dispensing, 37 °C media is delivered to the well in less than 10 s. The FEP tubing is interfaced with the fluidic module with threaded ferrule lock and nut fittings (Cole-Parmer VapLock). Outflow from the fluidic module is drawn away with FEP tubing routed out of the incubator and into a refrigerator containing the collection reservoirs and computer vision camera setup.

For the collection reservoirs, we selected 15 mL Polyethylene Terephthalate (PET) conical tubes (430 055, Corning) for high optical clarity, ease of replacement, and durability in downstream analysis and cold storage. To enhance visibility for computer vision imaging, we removed the factory-printed writing area on the conical PET tubes using generic, multipurpose tape. Flexible FEP tubing was interfaced with the PET tubes using a rubber cork plug (#6448K95, McMaster-Carr). The cork was pierced with 8-gauge steel needles that served as supportive conduits for the tubing. The tubing was secured inside the needle with glue (Loctite 4011) to create a hermetic seal at the point of interface. The steel encasing of the needles ensures a smooth, unobstructed flow within the flexible FEP tubes. Each collection reservoir had two flexible FEP tubes: one for media coming from the fluidic module and one for pressurized operation connected to the syringe pump. This ensured that spent media never entered the syringe (only air). The air is expelled into a filtered (Millipore AA 0.22 μm syringe filter) safety container (not shown in Fig. 1).

For the 7-day studies described here, we designed for equivalent media exchange across conditions. The Controls were fed 4 times at 1 mL per feed, totaling 4 mL of replacement media. AF and AFAR were fed 28 times at 143 μL per feed, totaling 4 mL of replacement media over the week. Summing the scheduled feeds and feedback adjustments, a single collection reservoir could store conditioned media for 2–3 weeks.

4.3. Computer vision for fluid volume estimation

The Estimator converts captured images into volume estimates using HSV (Hue, Saturation, and Value) segmentation and a calibrated polynomial regression. It distinguishes conical vs. cylindrical regions of the collection tube for accuracy. The result is passed to the pump’s feedback interpreter, which determines corrective actions. The computer vision setup, located inside a 4 °C refrigerator, included a support for the collection reservoir, a camera module, and an LED panel positioned behind the conical tubes. The LED panel served as backlighting to enhance the clarity and contrast of the images. The reservoir support was a two-plex 3D-printed system capable of multiplexity to tailor alternate experiments (see 3D-printed components). The camera and LED panel were both controlled by a Raspberry Pi.

To generate the calibration dataset, the camera module captured images of media in the collection reservoirs at select volumes over the entire range of the tube (0–12 mL), totaling 184 images. The volumes associated with each image were measured using a high-precision scale (30 029 077, Mettler Toledo). This approach enabled a correlation between the visual representation of media in the images and its actual volume (see Results).

To ensure image quality, our study introduced two checks to validate the integrity of the captured images: Lighting and blurriness. A region of interest (ROI) was designated within the panel’s area to verify the lighting conditions by checking that the average RGB color values each exceeded a minimum threshold of 20 out of 255. Blurriness was assessed by computing the variance of the Laplacian for the image, with a necessary threshold of 50 to pass. The thresholds were empirically determined using the calibration dataset.

Fig. 2c illustrates the methodology applied to fluid segmentation, outlined in the Results section. The process begins with capturing an RGB image of the collection reservoirs that are fixed in place by the setup. To facilitate better segmentation and feature extraction, the RGB image is transformed into the HSV (Hue, Saturation, and Value) color space. A summation of the HSV values row-wise from the bottom to the top of the collection reservoir results in three distinctive profiles that allow differentiation between the liquid and background. Each profile, as illustrated in Fig. 2c, presents a vertex at the boundary. A row value was established by averaging three rows identified in each HSV channel: an abrupt rise in the curve for the Hue channel, the absolute maximum for the Saturation channel, and the absolute minimum for the Value channel. From the average row value, the first segmentation was created. Everything below this row was set as white pixels, and everything above it was set as black pixels. A local evaluation around the average row was made to incorporate the meniscus in this segmentation. Utilizing HSV thresholds, the meniscus was accurately characterized and incorporated into the initial segmentation, culminating in the final image segmentation, in which white pixels represented the liquid portion.

The estimated volume was given by Eq. (1), where x represents the segmented area in pixels, and the resultant volume is in microliters. Two different curves are used to account for the conical section for volumes under 1.5 mL (and pixel area less than 4446) and the cylindrical section for larger volumes.

The image segmentation and estimation based on the mathematical model (Eq. (1)) is carried out by a software program named the “Estimator”. The process initiates with a feeding cycle, which triggers a picture request. Upon receiving the image of the collection reservoir, the “Estimator” analyzes the image and returns the estimated value of the fluid volume. The volume is relayed to the next module for feedback interpretation within the pump system (see Feedback interpreter).

4.4. Feedback interpreter

Computer vision volume estimations were compared to expectation values based on the sum total of pump action jobs. The feedback interpreter classified estimations into four categories: within tolerance, out-of-tolerance, anomaly, and tube change. Tolerance was a static volume selected at the start of the experiment. For the results shown here, the tolerance was 143 μL. If the volume estimation received was within the expectation value ± the tolerance, the pump action was determined a success, and feedback ceased. If the volume estimation received was beyond the expectation value ± the tolerance and also less than ±2000 μL, another cycle of feedback was engaged. When the volume was less than expected, for the first 5 iterations of feedback, aspiration jobs were sent to the pump with the difference of expectation and estimation. For iterations 6 to 19, pull jobs were sent to the pump, increasing by one for each subsequent interaction. A “pull” is a 1000 μL aspiration at 10× the standard syringe speed (applying a 1.1 × 103 mm/s flow rate), shown to generate the force required to break through variably high resistance in the conditioned media. At 20 iterations, the feedback interpreter requests manual intervention via the messaging application, and all further pump actions are suspended until the issue is resolved. When the volume was more than expected, dispense jobs were sent to the pump with the difference of expectation and estimation. Dispense actions were limited to 200 μL per action and 2 iterations of feedback in total to prevent overflow. A volume estimation that was 2000 μL or more above the expectation value was determined as an anomaly and requested manual intervention via the messaging application, and all further pump actions were suspended until the issue was resolved. The feedback interpreter automatically detected collection reservoir tube changes when the volume estimation dropped by 2000 μL or more compared to the previous estimation and the total volume present was estimated as less than 2000 μL.

4.5. Computer vision for in-incubator organoid culture imaging

4.6. Measuring neural activity

Extracellular field potential recordings were performed using CMOS-based high-density microelectrode arrays (HD-MEAs) (MaxOne, Maxwell Biosystems). Each HD-MEA contains 26,400 recording electrodes within a sensing area of 3.85 mm × 2.1 mm (each electrode has a diameter of 7.5 μm, spaced 17.5 μm apart center-to-center). A subset of up to 1020 electrodes (defined spatially by a configuration) can be selected for simultaneous recording [82]. Across one configuration, neuronal activity in microvolts was sampled over time at 20 kHz and stored in HDF5 file format.

The experiment involved each chip’s daily activity scans and recordings (described below). Each chip underwent an activity scan and subsequent recording every day, consistently conducted within the same one-hour time window. All chips shared the same recording unit and were recorded one at a time. For the AFAR condition, beyond the daily recordings and activity scans, the chip remained on the HD-MEA for automated hourly recordings. To ensure reliability of spike sorting and neural signal analysis, we applied quality control metrics commonly used in extracellular recordings [83]. The gain was set to 1024× with a 1 Hz high pass filter for both activity scans and recordings. The recording was set up to save 5 RMS thresholded spike times as well as all raw voltage data for downstream analysis and plotting.

All neural activity measurements were performed inside the incubator at 36.5 °C, 5% CO2.

4.7. Normalization and z-score calculation for effect of feeding during recording

In section Effects of feeding during recording, Fig. 6d, the firing rate variability to feeding was derived as follows:

For every recording, a 90-s sliding window was used to further analyze the sorted and curated spike raster with steps of 1-s. For every window, the average firing rate per unit was computed. To account for intrinsic variability in firing rate between units, the average firing rate per window was normalized for each unit as $\Vert x\Vert =(a-m)/(a+m)$, where $\Vert x\Vert$ is the norm, a is the window rate and m is the mean value across all window rates in the recording. This normalization yields a values between −1 and 1, reflecting the deviation from the average firing rate of each unit. A value of 0 means that the average firing rate for a unit in a given window is the same as the average firing rate for that unit. A value of ±1/3 means that the firing rate for a unit in a given window is approximately twice (for +1/3) or half (for −1/3) the mean firing rate for that unit.

Subsequently, the normalized firing rates per unit for all no-feed recordings were taken and for each window relative to the start of the recording, the mean and standard deviation were computed over all units in all no-feed recordings. These values were then used to z-score normalize the normalized firing rates of every unit in every window throughout all feed recordings as $z=(\Vert x_{feed}\Vert -{\mu }_{nofeed})/{\sigma }_{nofeed}$, where z is the z-score, $\Vert x_{feed}\Vert$ is the norm in the feeding condition, ${\mu }_{nofeed}$ is the mean of the norm in the no feeding condition, and ${\sigma }_{nofeed}$ is the standard deviation of the norm in the no feeding condition. This normalization yields a value for each unit per fed recording at every window that reflects the variability from the average firing rate of that unit. This is expressed as the number of standard deviations that any particular value is removed from the average variability from the mean firing rate of all units over all no-feed recordings at that same window in the recording. If the firing rate of the unit during any point in the feed recording substantially increases or decreases relative to the average firing rate over the whole recording and if this is not the case for units in the no-feed recording, this will be reflected as a positive or negative z-score normalized value. In addition, the z-scores were averaged over all units in all feed recordings with the same feed volume to see the effects for each individual feed volume.

4.8. Internet of Things (IoT)