The need for real-time insights into video streams is driving the use of artificial intelligence (AI) techniques such as deep neural networks for classification, object detection and extraction, and anomaly detection. Srikumar Venugopal, research scientist and a member of the High-Performance Systems group in IBM Research, Ireland, and his team performed an experimental evaluation of object classification and achieved mixed results.
The IBM team used commercially available cloud-hosted services to conduct the evaluation. The best result they could secure was a classification output, which was far below the standard video production rate of 24 frames per second. Therefore, they conducted a similar experiment on a representative edge device (NVIDIA Jetson TK1). Though this helped them achieve the latency requirements, the process used up most of the resources available on the device.
To break this duality, the IBM team proposes a semantic cache approach. They provide the details in their HotEdge 2018 conference paper "Shadow Puppets: Cloud-level Accurate AI Inference at the Speed and Economy of Edge.”
The semantic cache approach combines the low latency of edge deployments with the near-infinite resources available in the cloud. The team leveraged the well-known technique of caching to mask latency by executing AI inference for a particular input (e.g. video frame) in the cloud and storing the results on the edge against a "fingerprint", or a hash code, based on features extracted from the input.
Click here to read how the semantic cache approach works.
Please give your feedback on this article or share a similar story for publishing by clicking here.
