The ISTC-EC will explore new algorithms that can advance the state of embedded processors as well as innovative systems architectures and processor capabilities to support novel applications in the retail, automotive and home environments of the future. The overarching goals of the center’s research will be to enable real-time personalization of applications; spontaneous interaction with the environment (the automatic delivery of “right here, right now, right for me” services); and crowdsourcing (sharing, interpreting, verifying and acting upon synthesized information from diverse, distributed inputs.) With these goals in mind, the ISTC-EC will organize its research agenda around four key technology themes, outlined below.
Collaborative Perception. Embedded systems interact with the physical environment and with humans through multi-modal sensing (e.g., video and audio, accelerometers, temperature sensors, and new sensors such as 3D video and modulated light). This research theme will focus on the use of computer vision to sense and understand the environment as well as human behavior and intent, to enable natural interaction. Perception in embedded-computing applications involves unique challenges, as it must be performed online and in real time in the face of resource constraints. The ISTC-EC will investigate new ways to do this robustly, by adopting advanced sensors, synthesizing multi-modal sensor data, leveraging prior learning, incorporating contextual information, and collaborating with humans in the machine-perception process (e.g., having humans provide additional input to help embedded systems make sense of visual data). Topics to be explored might include face recognition; analysis of attention, intent and emotion; behavior modeling; and interactions between humans and the environment.
Real-time Knowledge Discovery. The ISTC-EC will explore data mining and machine learning in real time to support future retail, automotive and home applications that require immediate responses. Specific topics to be explored might include real-time machine learning on heterogeneous (or customized) cores, discovering patterns in event streams (e.g., data generated in the home by diverse sources, from video recordings to location sensors and online calendars); anomaly mining; online learning; and learning from heterogeneous, dynamic and/or high-dimensional data. Machine learning in embedded-computing applications presents a number of challenges, such as operating effectively in resource-constrained environments, dealing with multiple specialized sensing modalities, and managing the complex tradeoffs involved in determining when to offload computation to the cloud and when to process data locally.
Robotics. Embedded systems, particularly in the robotics space, involve technologies in unique, often autonomously mobile, multi-sensory exploration. This research theme focuses on technologies that contribute to unique new automation capabilities in the store, car and home of the future. To date, robots have primarily been successful at manipulation in simulation and tightly controlled environments such as factory automation. The ISTC-EC will explore technologies that can contribute to new automated assistive capabilities in the store, car and home of the future. Specific topics to be investigated might include multi-modal sensing hardware; real-time interaction between robots and humans via haptic (tactile, or touch-sensitive) interfaces; fine-grained tracking of people and objects; indoor and urban navigation in a dynamic environment; motion sensing and planning; and detection and avoidance of obstacles.
Embedded Systems Architecture. Embedded systems pose challenges of resource constraints, such as low power, low bandwidth, limited memory, limited storage and limited computational capability. This research theme focuses on hardware and software innovations that can enable large-scale, real- time data-processing algorithms (collaborative perception, knowledge discovery), tailored to these unique resource constraints, the specific context (physical location, proximity, usage models) in which they are used, as well as domain-specific requirements posed by the store, car and home of the future. The ISTC-EC will explore technologies to incorporate large-scale algorithms in hardware, when to offload computations to the cloud vs. performing them in silicon, and to address the architectural challenges of porting digital profiles across heterogeneous platforms in real- time (e.g., to seamlessly migrate entertainment content from the home to a car, so that a child can resume playing a game in the car while her parents run errands). Specific topics to be investigated might include power management, as energy consumption is a major issue when dealing with embedded systems and the intensive computation that will be required to support the store, car and home of the future. Other topics for potential investigation include large-scale parallel algorithms, implementing machine learning in silicon, novel platforms for crowdsourced computation and communication, and location/proximity-aware scheduling.
