CS7387 Independent Research Presentation
Title: Deep Learning-based Object detection on Heterogeneous Processors with TensorRT
Presenter: Yuxiao Zhou
Advisor: Dr. Kecheng Yang
Date/time: May 1st, 2023 @4:00 p.m.
Location: Zoom Meetingļ¼https://txstate.zoom.us/my/yangk
Abstract:
Deep neural networks have shown remarkable capabilities in computer vision applications. However, their complex architectures can pose challenges for efficient real-time deployment, as they require significant computational resources and energy costs. To overcome these challenges, TensorRT has been developed to optimize neural network models trained on major frameworks to speed up inference and minimize latency. It enables inference optimization using techniques such as model quantization which reduces computations by lowering the precision of the datatype. The focus of our paper is to evaluate the effectiveness of TensorRT for model quantization. We conduct a comprehensive assessment of the accuracy, inference time, and throughput of a TensorRT quantized model on an edge device. Our findings indicate that the quantization in TensorRT significantly enhances the efficiency of inference metrics while maintaining a high level of inference accuracy. Additionally, we explore various workflows for implementing quantization using TensorRT and discuss their advantages and disadvantages. Based on our analysis of these workflows, we provide recommendations for selecting an appropriate workflow for different application scenarios.
Deadline: May 26, 2023, midnight