NUS Enterprise

Video Visual Relation Detection

Technology #2017-117

Questions about this technology? Ask a Technology Manager

Download Printable PDF

Image Gallery
High Res
Tat Seng CHUA
Managed By
Ms Asha Srinivasan (
Associate Director (65)65161671
Patent Protection

US Patent Pending
Files and Attachments
Technology Offer [PDF]
Video Visual Relation Detection Image-High Res [JPG]

As compared to still images, videos provide a more natural set of features for detecting visual relations, in which features like the dynamic interactions between objects enable visual relations like "A-follow-B" and "A towards-B" to be detected in videos. However, detecting visual relations in videos is more technically challenging than in images due to the difficulties in accurate object tracking and diverse relation appearances in the video domain. 

VidVRD is a novel method that helps overcome the technical challenges in detecting visual relations in videos by using object tracklet proposal, short-term relation prediction and greedy relational association. Moreover, we contribute the first dataset for VidVRD evaluation, which contains 1,000 videos with manually labeled visual relations, to validate our proposed method. On this dataset, our method achieves the best performance in comparison with the state-of-the-art baselines. 

Video analysis/Deep Learning / Artificial Intelligence 

Analytical and laboratory studies to validate analytical predictions 

This technique may effectively underpin numerous visual-language tasks, such as captioning, visual search, and visual question-answering. 

1. It can detect the visual relations between object pairs within the given videos. 
2. It can describe different types of visual relations between a pair of objects. 
3. It can predict all the visual relations between object pairs within a short-term video clip. 
4. It can associate short-term relations into video visual relations. 

Patent pending. Available for licensing.