Video Visual Relation DetectionTechnology #2017-117
Questions about this technology? Ask a Technology Manager
- Image Gallery
- Tat Seng CHUA
- Managed By
- Ms Asha Srinivasan Associate Director (65)65161671
- Patent Protection
- US Patent Pending
As compared to still images, videos provide a more natural set of features for detecting visual relations, in which features like the dynamic interactions between objects enable visual relations like "A-follow-B" and "A towards-B" to be detected in videos. However, detecting visual relations in videos is more technically challenging than in images due to the difficulties in accurate object tracking and diverse relation appearances in the video domain.
VidVRD is a novel method that helps overcome the technical challenges in detecting visual relations in videos by using object tracklet proposal, short-term relation prediction and greedy relational association. Moreover, we contribute the first dataset for VidVRD evaluation, which contains 1,000 videos with manually labeled visual relations, to validate our proposed method. On this dataset, our method achieves the best performance in comparison with the state-of-the-art baselines.
Video analysis/Deep Learning / Artificial Intelligence
STAGE OF DEVELOPMENT
Analytical and laboratory studies to validate analytical predictions
This technique may effectively underpin numerous visual-language tasks, such as captioning, visual search, and visual question-answering.
1. It can detect the visual relations between object pairs within the given videos.
2. It can describe different types of visual relations between a pair of objects.
3. It can predict all the visual relations between object pairs within a short-term video clip.
4. It can associate short-term relations into video visual relations.
Patent pending. Available for licensing.