As the third and final day of VentureBeat’s Transform 2020 digital conference kicked off, Nvidia’s vice president and general manager of embedded and edge computing, Deepu Talla, gave a fire-related talk about the increasingly important role of AI in edge in business computing, a topic that has been widely discussed in the past year but has remained somewhat amorphous. Talla presented a clear thesis: Edge AI exists to solve specific business problems that demand a combination of internal computing, high speed and low latency that cloud AI cannot offer.
As of today, most next-generation AIs run in the cloud, or at least generate AI-powered responses in the cloud, based on spatially and temporally aggregated data from devices with some edge processing capabilities. But as explained by Maribel López, founder of Talla and López Research, some AI response processes are already moving to the limit, in part because the sensors are now generating an increasing volume of data that cannot be sent to the cloud for processing.
It’s not just about managing all that data, Talla explained; Edge AI located within or near the data collection point may, in some cases, be a more practical or socially beneficial approach. For a hospital, which can use sensors to monitor patients and gather requests for medications or assistance, edge processing means keeping medical data private at home rather than sending it to cloud servers. Similarly, a retail store could use numerous cameras for self-management and inventory management and to monitor pedestrian traffic. Such granular details could slow down a network, but can be replaced by an on-site edge server with lower latency and lower total cost.
Talla said that last year AI has benefited from the availability of excellent hardware and architectures, including GPUs with tensor cores for dedicated AI processing, plus high-performance and secure network equipment. Unlike smartphones, which are replaced every 2-3 years, Edge Servers will remain in the field for 5, 10, or more years, making software-focused updates critical. To that end, Nvidia’s EGX Edge Computing software brings traditional cloud capabilities to Edge Servers and will be updated to improve over time. The company has also released industry-specific edge frameworks such as Metropolis (smart cities), Clara (healthcare), Jarvis (conversational AI), Isaac (robotics), and Aerial (5G), each of which supports forms of AI on Nvidia GPU.
Talla explained that it is possible to combine features of multiple frameworks, such as using Clara Guardian to help hospitals stop playing, with Jarvis monitoring cameras in patient rooms and then automatically handling spoken requests like “I want water.” Using Metropolis smart city tools, the same system could handle AI processing for the entire hospital camera fleet, dynamically counting the number of people in the building or in the rooms. Some of these tasks can happen today with AI in the cloud, but moving most or all of it to the limit for faster responsiveness makes sense, for certain companies.
However, Talla did not suggest that cloud AI is coming out or outdated. In fact, she noted that the responses generated by artificial intelligence in the cloud are currently fantastic, and said that the appeal of artificial intelligence will depend on its ability to solve a specific business problem better than a cloud alternative. It remains to be seen whether an in-house AI system will have an equal, lower, or higher total cost of ownership for businesses compared to cloud platforms, as well as which approach ultimately delivers the best overall experience for the business and its customers. .
Still, Talla said during a question-and-answer session that a significant amount of processing will move from cloud to edge over the next five years, although an AI-generated response may also be just one component of a more AI system. Large that merges edge and AI processing in the cloud. In addition, she noted that Edge Servers will become increasingly useful for multiple functions simultaneously, so that a single Edge Computer can handle 5G communications, video analytics, and conversational AI for a business, rather than a single purpose.
