A primary insight driving our research is that a sensing technology has the ability to preserve specific structure in data to substantially enhance generalization of learning, and thereby advance cognitive capabilities. Following this insight, we pursue systems based on physically-integrated (PI) sensing. In contrast to remote sensing (e.g., vision) where the embedded signals coupled to each sensor (e.g., imager pixel) change in a dynamic environment with whatever is in the field of view, PI sensing directly associates sensors with physical objects, and, in doing so, enforces invariant semantic structure. For instance, in the area of human-interactive spaces, shown in the figure, semantic structure arises because the ways humans naturally interact with objects in their environments says something about the activities they are engaged in and the intentions they have. Today’s deep-learning models (e.g., convolutional neural networks) have achieved great success applied to remote sensing (e.g., vision), by first detecting what embedded signals are being sensed (i.e., by striding correlation filters within a layer) and then forming these into semantic features (i.e., by composition through layers). But, such ground-up learning contends with a large space of possibilities, requiring proportionately complex models, which are difficult to train and limit interpretability. Of course, we still make extensive use of remote sensing and deep learning. But we think about when and how to combine such approaches with PI sensing, to enable structure in sensing data towards regimes of faster learning, greater robustness, and increased adaptability/transferability.


So, PI sensing sounds good, but it raises two key technology challenges, which are big focus areas of our systems research:

  1. We must now deploy sensors, potentially on a very large scale, to capture the embedded signals arising from the many natural interactions between physical objects; but, such deployment must not be disruptive to those interactions and the information signals they generate. To address this, we research systems based on Large-Area Electronics (LAE), a technology capable of creating diverse, expansive, and form-fitting arrays of transducers.
  2. We must now perform continuous computations for learning, state estimation, and action planning from all of the sensor data; but, the embedded and distributed nature of that data raises critical resource-constraint challenges (energy, bandwidth, etc.). To address this, we research machine-learning algorithms together with new circuits and architectures, to exploit the statistical nature of both algorithms, for learning and inference, as well as energy-aggressive architectures, based on emerging devices and compute models.

Below are some examples from our experimental research in these areas; many of these examples combine results from our theoretical and algorithmic research.


heterogeneous microprocessorProgrammable Approximate Acceleration for Sensor-Inference: This work presents a heterogeneous microprocessor for low-energy sensor-inference applications. Hardware acceleration has shown to enable substantial energy-efficiency and throughput gains, but raises significant challenges where programmable computations are required, as in the case of feature extraction. To overcome this, a programmable feature- extraction accelerator (FEA) is presented that exploits genetic programming for automatic program synthesis. This leads to approximate, but highly structured, computations, enabling: 1) a high degree of specialization; 2) systematic mapping of programs to the accelerator; and 3) energy scalability via user-controllable approximation knobs. A microprocessor integrating a CPU with feature-extraction and classification accelerators is prototyped in 130-nm CMOS. [paper][paper]


""Frequency-Hopping DCO Architecture for Large-Area Pressure Sensing: Hybrid systems combine large-area electronics (LAE) with silicon-CMOS ICs for sensing and computation, respectively. In such systems, interfacing a large number of distributed LAE sensors with the CMOS domain poses a key limitation. This work presents an architecture that aims to greatly reduce both the number of physical connections and the time for accessing all of the sensors. Each sensor modulates the amplitude of a thin-film transistor (TFT) digitally controlled oscillator (DCO). All DCO outputs are combined, but each follows a unique frequency-hopping pattern (controlled by a code from CMOS), allowing recovery of the individual sensors. The architecture enables much greater scalability of sensors for a given number of connections than active-matrix and binary- addressing schemes. [paper][paper]


""Direct Classification of Analog-sensor Data via Clocked Comparators:This work presents a system, where clocked comparators directly derive classification decisions from analog sensor signals, thereby replacing instrumentation amplifiers, ADCs, and digital MACs, as typically required. A machine-learning algorithm for training the classifier is presented, which enables circuit non-idealities as well as severe energy/area scaling in analog circuits to be overcome. Furthermore, a noise model of the system is presented and experimentally verified, providing a means to predict and optimize classification error probability in a given application. The noise model shows that superior noise efficiency is achieved by the comparator-based system compared with a system based on linear low-noise amplifiers. [paper]


""In-memory Computing Architecture for a Machine-learning Classifier:This work presents a machine-learning classifier, where computations are performed in a standard 6T SRAM array, which stores the machine-learning model. This eliminates the need for explicit memory accesses, instead performing accessing of a computational result over a large amount of data stored in the array, thus amortizing accessing energy/delay. Peripheral circuits implement mixed-signal weak classifiers via columns of the SRAM, and a training algorithm enables a strong classifier through boosting and also overcomes circuit non-idealities, by combining multiple columns. [paper][paper]


Chopper-stabilized LNAFlexible-electronics system for EEG acquisition and Processing: This work presents an electroencephalogram (EEG) acquisition and biomarker-extraction system based on flexible, thin-film electronics. There exist commercial, single-use, flexible, pre-gelled electrode arrays; however, these are fully passive, requiring cabling to transfer sensitive, low-amplitude signals to external electronics for readout and processing. This work presents an active EEG acquisition system on flex, based on amorphous silicon (a-Si) thin-film transistors (TFTs). The system incorporates embedded chopper- stabilized a-Si TFT low-noise amplifiers, to enhance signal integrity, and a-Si TFT compressive-sensing scanning circuits, to enable reduction of EEG data from many channels onto a single interface, for subsequent processing by a CMOS IC. [paper][paper]



Large-area Image-sensing and Feature-extraction System: This work presents a sensing and compression system for image detection, based on large-area electronics (LAE). LAE allows us to create expansive, yet highly-dense arrays of sensors, enabling integration of millions of pixels. However, the thin-film transistors (TFTs) available in LAE have low performance and high variability, requiring the sensor data to be fed to CMOS ICs for processing. This results in a large number of interconnections, which raises system cost, and limits system scalability and robustness. To overcome this, the presented system employs random projection, a method from statistical signal processing, to compress the pixel data from a large array of image sensors in the LAE domain using TFTs. [paper]


""Large-area Microphone Phased Array: This work presents a system for reconstructing independent voice commands from two simultaneous speakers, based on an array of spatially distributed microphones. It adopts a hybrid architecture, combining large-area electronics (LAE), which enables a physically expansive array (>1 m width), and a CMOS IC, which provides superior transistors for readout and signal processing. The array enables us to: 1) select microphones closest to the speakers to receive the highest SNR signal; 2) use multiple spatially diverse microphones to enhance robustness to variations due to microphones and sound propagation in a practical room. Each channel consists of a thin-film transducer formed from polyvinylidene fluoride (PVDF), a piezopolymer, and a localized amplifier composed of amorphous silicon (a-Si) thin-film transistors (TFTs). Each channel is sequentially sampled by a TFT scanning circuit, to reduce the number of interfaces between the large-area electronics (LAE) and CMOS IC. [paper][paper][paper]


""Large-area Image-sensing and Classification System: This work presents a large-area image-sensing and detection system that integrates, on glass, sensors and thin-film transistor (TFT) circuits for classifying images from sensor data. Large-area electronics (LAE) enables the formation of millions of sensors spanning physically large areas; however, to perform processing functions, thousands of sensor signals must be interfaced to CMOS ICs, posing a critical limitation to system scalability. This work presents an approach whereby image detection of shapes is performed using simple circuits in the LAE domain based on amorphous silicon (a-Si) TFTs. This reduces the interfaces to the CMOS domain. The limited computational capability of TFT circuits as well as high variability and high density of process defects affecting TFTs and sensors is overcome using a machine-learning algorithm known as error-adaptive classifier boosting (EACB) to form embedded weak classifiers. [paper][paper]


""Matrix-multiplying ADC for Linear Feature Extraction and Classification in the A-D Process: In wearable and implantable medical-sensor applications, low-energy classification systems are of importance for deriving high-quality inferences locally within the device. Given that sensor instrumentation is typically followed by A-D conversion, this paper presents a system implementation wherein the majority of the computations required for classification are implemented within the ADC. To achieve this, first an algorithmic formulation is presented that combines linear feature extraction and classification into a single matrix transformation. Second, a matrix-multiplying ADC (MMADC) is presented that enables multiplication between an analog input sample and a digital multiplier, with negligible additional energy beyond that required for A-D conversion. [paper][paper][paper]


""Frequency Readout of Post-processed Thin-Film MEMS Resonators:Thin-film MEMS resonators fabricated at low temperatures can be processed on CMOS ICs, forming high-sensitivity transducers within complete sensing systems. A key focus for the MEMS devices is increasing the resonant frequency, enabling, among other benefits, operation at atmospheric pressure. How- ever, at increased frequencies, parasitics associated with both the MEMS-CMOS interfaces and the MEMS device itself can severely degrade the detectability of the resonant peak. This work attempts to overcome these parasitics while providing isolation of the CMOS IC from potentially damaging sensing environments. To achieve this, an interfacing approach is proposed based on capacitive coupling across the CMOS IC passivation, and a detection approach is proposed based on synchronous readout. [paper][paper]


""Large-Area 3D Multi-gesture Sensing System: This work presents a flexible 40×40cm2 gesture- sensing sheet for large-area interactive spaces. The system achieves out-of-plane sensing to 16cm. Self-capacitance readout of individual electrode pixels in a 4×4 array enables multiple gestures to be sensed simultaneously without ghost effects. For high-sensitivity readout, pixel self-capacitance is converted to frequency via high-Q LC oscillators formed from amorphous-silicon (a-Si) thin-film transistors (TFTs) and planar inductors patterned directly on flex. Frequency readout is then performed by a CMOS IC. Scalability in the number and scan rate of pixels is achieved by (1) inductively coupling all oscillators to the CMOS IC through a single interface, and (2) reading out all pixels in a row simultaneously in separated frequency channels. [paper]


""Large-Area 3D Gesture-sensing System: In this work, we present a 3D sensing system with 40×40cm2 area and sensing distance to 30cm. This distance is achieved via two approaches. First, capacitance sensing is performed via frequency modulation, and the sensitivity of frequency readout is enhanced by high-Q oscillators capable of filtering noise sources in the readout system as well as stray noise sources from display coupling. Second, the capacitance signal is enhanced by eliminating electrostatic coupling between the sensing electrodes and surrounding ground planes. [paper]


""Self-powered Large-area Strain-Sensing Sheet for Structural Monitoring: This work presents a 2nd-generation system for high-resolution structural-health monitoring of bridges and buildings. The system combines large-area electronics (LAE) and CMOS ICs via scalable interfaces based on inductive and capacitive coupling. This enables architectures where the functional strengths of both technologies can be leveraged to enable large-scale strain sensing scalable to cm resolution yet over large-area sheets. The system consists of three subsystems: (1) a power-management subsystem, where LAE is leveraged for solar-power harvesting, and CMOS is leveraged for power conversion and regulation; (2) a sensing subsystem, where LAE is leveraged for dense strain sensing, and CMOS is leveraged for multi-sensor acquisition; and (3) a communication subsystem, where LAE is leveraged for long-range interconnects, and CMOS is leveraged for low-power transceivers. [paper][paper]


Large-area Strain-Sensing Sheet for Structural Monitoring: Early-stage damage detection for bridges requires continuously sensing strain over large portions of the structure, yet with centimeter-scale resolution. To achieve sensing on such a scale, this work presents a sensing sheet that combines CMOS ICs, for sensor control and readout, with large-area electronics (LAE), for many-channel distributed sensing and data aggregation. Bonded to a structure, the sheet thus enables strain sensing scalable to high spatial resolutions. In order to combine the two technologies in a correspondingly scalable manner, non-contact interfaces are used. Inductive and capacitive antennas are patterned on the LAE sheet and on the IC packages, so that system assembly is achieved via low-cost sheet lamination without metallurgical bonds. The LAE sheet integrates thin-film strain gauges, thin-film transistors, and long interconnects on a 50-μm-thick polyimide sheet, and the CMOS ICs integrate subsystems for sensor readout, control, and communication over the distributed sheet in a 130 nm process. [paper][paper]


""Configurable Low-power Microprocessor for Medical-sensor Inference: Data-driven methods based on machine learning enable powerful frameworks for analyzing complex physiological signals in medical-sensor applications; however, these methods are not well supported by traditional DSPs. A general-purpose microprocessor is presented in 130nm CMOS that integrates configurable accelerators, enabling low-energy hardware to support the broadest range of machine-learning frameworks reported to date. In addition to computational energy, memory limitations due to the high-order data-driven models are overcome by an embedded compression/decompression accelerator, which reduces the memory footprint by 4× with overhead <8%. [paper]


""Low-power Microprocessor for Medical-sensor Inference: Low-power sensing technologies have emerged for acquiring physiologically indicative patient signals. However, to enable devices with high clinical value, a critical requirement is the ability to analyze the signals to extract specific medical information. Yet given the complexities of the underlying processes, signal analysis poses numerous challenges. Data-driven methods based on machine learning offer distinct solutions, but unfortunately the computations are not well supported by traditional DSP. This work presents a custom processor that integrates a CPU with configurable accelerators for discriminative machine-learning functions. A support-vector-machine accelerator realizes various classification algorithms as well as various kernel functions and kernel formulations, enabling range of points within an accuracy-versus-energy and -memory trade space. An accelerator for embedded active learning enables prospective adaptation of the signal models by utilizing sensed data for patient-specific customization, while minimizing the effort from human experts. [paper][paper]


""Large-area Super-regenerative Radios on Wallpaper:  This work presents a super-regenerative (SR) transceiver with integrated antenna on plastic that leverages the attribute of large area to create high- quality passives; this enables resonant TFT circuits at high frequencies (near ƒt) and allows for large antennas, maximizing the communication distance. The resulting carrier frequency is 900kHz, and the range is over 12m (at 2kb/s). This will enable sheets with integrated arrays of radio frontends for distributing a large number of communication links over large areas. [paper]


""Large-area Wireless Charging System based on TFT LC Oscillators:This work presents an energy-harvesting system consisting of amorphous-silicon (a-Si) solar cells and thin-film-transistor (TFT) power circuits on plastic. Along with patterned planar inductors, the TFTs realize an LC- oscillator that provides power inversion of the DC solar- module output, enabling a low-cost sheet for inductively- coupled wireless charging of devices. Despite the low performance of the TFTs, the oscillator can operate above 2MHz by incorporating the device parasitics into the resonant tank. This enables increased quality factor for the planar inductors, improving the power-transfer efficiency and the power delivered. [paper]


""Large-Area Wireless Charging System based on TFT Switching Power Inverters: With the explosion in the number of battery-powered portable devices, ubiquitous powering stations that exploit energy harvesting can provide an extremely compelling means of charging. This work presents a system on a flexible sheet that, for the first time, integrates the power electronics using the same thin-film amorphous-silicon (a-Si) technology as that used for established flexible photovoltaics. This demonstrates a key step towards future large-area flexible sheets which could cover everyday objects, to convert them into wireless charging stations. In this work, we combine the thin-film circuits with flexible solar cells to provide embedded power inversion, harvester control, and power amplification. This converts DC outputs from the solar modules to AC power for wireless device charging through patterned capacitive antennas. [paper]