The quiet revolution of AI on the edge
AI is running on mobile phones, sensors, and home appliances thanks to the engineering innovations that make the most of the limited memory, reduced computational power, and little energy available in these devices.
The recent popularity of large generative models such as GPT immediately makes us think of AI running in large computing farms that consume significant electricity and take a long time to execute. A parallel revolution, though perhaps more quiet, is happening in bringing AI capabilities to all kinds of small devices (eg mobile phones, environmental sensors, home appliances) that have to run AI with limited memory, reduced computational power, and little energy. This has become known as AI on edge computing.
A recent USC AI Futures Symposium highlighted USC research in AI on the edge, a multidisciplinary arena that spans engineering research in computing architectures, microelectronics, new materials, and neuromorphic computing as well as computer science research in AI algorithms and models.
The sessions featured presentations from twenty USC faculty and researchers in the departments of Electrical and Computer Engineering, Computer Science, Biomedical Engineering, and Chemical Engineering and Materials Science, as well as researchers from the Information Sciences Institute (ISI). The symposium had more than 200 registered participants. In addition to USC faculty and students, there were many attendees from industry, government, and other universities.
USC Viterbi School of Engineering Dean Yannis C. Yortsos gave the opening remarks for the symposium. He remarked on the school’s strengths in both AI and edge computing. “In the AI front, Viterbi has been pioneering AI research in large language models, machine translation, multi-agent systems, knowledge graphs, and social robotics. We pioneered the use of neural networks for large language models, by showing that neural transducers could generate better machine translations. In the hardware and architectures front, Viterbi has demonstrated leadership in microelectronics, advanced materials and technology for computing, biologically-inspired architectures and devices, and quantum computing. In 1981, the USC Information Sciences Institute pioneered multi-project wafer fabrication services with MOSIS, which has since provided researchers and students unique opportunities to create and learn. Viterbi hosted the first quantum computer in academia in 2011. And in 2021, the school opened the O’Brien Nanofabrication Laboratory to translate novel materials into transformational devices that will move us beyond conventional silicon processing. Viterbi continues to innovate in both AI and edge computing.”
The symposium had four sessions of technical presentations. The first session, devoted to TinyML, showed that navigation can be managed in ultra-resource constrained devices, that deep learning models can be run in the edge to support devices in the internet of things (IoT) that continuously collect data, and that privacy is improved if sensitive data is processed upon collection on the edge rather than moved to the cloud for processing.
The second session covered neuromorphic computing, and included presentations about brain machine interfaces for stimulation of memory formation, sensing through electronic skin, disentangling brain functions to improve diagnosis and treatment, and biologically inspired retinas for improving vision.
The third session on hardware-software co-design introduced architectures and algorithms especially designed to process graphs, images and videos, and architectures to create multiple independent models in distributed locations that can be combined into a better performing model following a federated approach.
The fourth and final session was devoted to post-CMOS hardware and device materials, such as phase-change materials, single-crystal indium phosphide (InP), and memristors.
Two featured speakers reported on large multi-site multi-disciplinary efforts in this area. An invited presentation from Yiran Chen of Duke University discussed a variety of approaches to AI on the edge for improving network communications as part of an NSF-funded AI Institute. Steve Crago of the USC Information Sciences Institute discussed the CHIPS and Science Act recently passed by the US Congress, and the opportunities under the DoD Microelectronics Commons. He highlighted USC’s leadership in forming a consortium of academic, government, and industry partners in the Southern California region.
A panel of experts discussed new directions for research in AI on the edge. An exciting area is in biologically inspired devices, particularly in the area of sensing, perception, and memory. Another interesting direction for future research is fine-tuning edge devices, where both the device and the software can evolve over time to adapt to a changing environment or to incorporate new technologies. The panel highlighted that open source has accelerated research in this area, and that research would benefit from agreements on metrics to measure efficiency, adaptability, and other desirable properties of AI on the edge. The panel pointed out the importance of close collaborations with industry, as new products are designed to offer customers new capabilities and technologies and those can be a very effective driver for research.
USC Senior Vice-President for Research and Innovation Ishwar Puri gave the closing remarks. He remarked that AI is a fractal community and it is crucial to build bridges across disciplines. Dr Puri is also Professor of Aerospace and Mechanical Engineering, and highlighted that more AI research is needed to better understand the natural world, to predict unique but plausible events, and to characterize unanticipated consequences.
The symposium was co-chaired by Joshua Yang, professor of Electrical and Computer Engineering, and Yolanda Gil, Director of New Initiatives in AI and Data Science in Viterbi and at ISI, and Research Professor in Computer Science. Yang noted: “We have an impressive concentration of researchers working in this area in different departments at Viterbi, it is very exciting to see us coming together and envision the future of AI on the edge”.
The event was part of the USC AI Futures Symposium series that has featured USC research under the themes “AI with Common Sense,” “AI and Data Science,” and “Will AIs Ever Be One of Us?” Recordings of the prior symposia as well as the recent “AI on the Edge” symposium are available from tinyurl.com/ai-edge.
