Open Journal Systems

Efficient Human-Robot Interaction using Deep Learning with Mask R-CNN: Detection, Recognition, Tracking and Segmentation

Than Le, Dang Huynh




Human robot interaction; deep neural net- work; tracking robotics; detection; mini-parallel kinematic.

Full Text:



Rekha N, M.Z.Kurian, "Face Detection in Real Time Based on HOG", International Journal of Advanced Research in Computer Engineering & Technology (IJARCET), 2014.

L. R. Cerna, G. Cámara-Chávez, D. Menotti, "Face Detection: Histogram of Oriented Gradients and Bag of Feature Method", Proceed-ings of the International Conference on Image Processing, Computer Vision, and Pattern Recognition (IPCV), 2013.

J. Barreto, P. Menezes, J. Dias, "Human-robot interaction based on haar-like features and eigenfaces." International Conference on Robotics and Automation (ICRA), 2004.

P. Viola, M. Jones, "Rapid object detection using a boosted cascade of simple features." Computer Vision and Pattern Recognition (CVPR), 2001.

R. Ranjan, V.M. Patel, R. Chellappa, "HyperFace, A Deep Multitask Learning Framework for Face Detection, Landmark Localization, Pose Estimation, and Gender Recognition.", The IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 2016.

K. Zhang, Z. Zhang, Z. Li, Y. Qiao, "Joint Face Detection and Alignment using Multi-task Cascaded Convolutional Networks", IEEE Signal Processing Letters (SPL) 2016.

S, Yang, P. Luo, C. Loy, X. Tang, "From Facial Parts Responses to Face Detection: A Deep Learning Approach", The IEEE International Conference on Computer Vision (ICCV), 2015.

F. Schroff, D. Kalenichenko, J. Philbin, "FaceNet: A Unified Em-bedding for Face Recognition and Clustering", Computer Vision and Pattern Recognition (CVPR), 2015.

R. Girshick, "Fast R-CNN", The IEEE International Conference on Computer Vision (ICCV), 2015.

S. Ren, K. He, R. Girshick, J. Sun, "Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks", Conference on Neural Information Processing Systems (NIPS), 2015

K. He, G. Gkioxari, P. Dollár, R. Girshick, "Mask R-CNN", The IEEE International Conference on Computer Vision (ICCV), 2017.

Quan H. Nguyen ; Trinh N. P. Tran ; Dung D. Huynh ; An T. Le ; Than D. Le, "Real-Time Localization and Tracking System with Multiple-Angle Views for Human Robot Interaction", The IEEE International Conference on Robotic Computing (IRC), 2017.

"INRIA Person",, INRIA.

"40 Actions",, Stan-ford.

"VGG Image Annotator", vgg/, Oxford.

Luis Perez, Jason Wang, "The Effectiveness of Data Augmentation in Image Classification using Deep Learning", arXiv, 2017.

Jason Yosinski, Jeff Clune, Yoshua Bengio, Hod Lipson, "How transferable are features in deep neural networks?" Advances in Neural Information Processing Systems, 2014.

CNN Features off-the-shelf: an Astounding Baseline for Recognition Ali S. Razavian, Hossein Azizpour, Josephine Sullivan Stefan Carlsson, "CNN Features off-the-shelf: an Astounding Baseline for Recognition", arXiv, 2017.

Jeff Donahue, Yangqing Jia, Oriol Vinyals, Judy Hoffman, Ning Zhang, Eric Tzeng, Trevor Darrell, "DeCAF: A Deep Convolutional Activation Feature for Generic Visual Recognition", The International Conference on International Conference on Machine Learning, 2017.

Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, Li Fei-Fei, "ImageNet: A large-scale hierarchical image database", IEEE Conference on Computer Vision and Pattern Recognition, 2009.

Alessandro De Luca, Alin A. SchafferSami HaddadinGerd Hirzinger, "Collision Detection and Safe Reaction with the DLR-III Lightweight Manipulator Arm", IEEE/RSJ International Conference on Conference: Intelligent Robots and Systems, 2006.

Sebastian Thrun, Wolfram Burgard, Dieter Fox, "Probabilistic Robotics"

Hoi V. Nguyen, Than D. Le, Dung D. Huynh, Peter Nauth, "Forward kinematics of a human-arm system and inverse kinematics using vector calculus", International Conference on Control, Automation, Robotics and Vision (ICARCV), 2016.

Miao Li, Hang Yin, Kenji Tahara, Aude Billard, "Learning Object-level Impedance Control for Robust Grasping and Dexterous Manipulation," IEEE International Conference on Robotics and Automation (ICRA), 2014.

Joseph Redmon, Ali Farhadi, "YOLO9000: Better, Faster, Stronger",IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2017.

Emanuele Magrini, Fabrizio Flacco, Alessandro De Luca, "Control of generalized contact motion and force in physical human-robot interaction", IEEE International Conference on Robotics and Automation (ICRA), 2015

An T. Le, Than D. Le, "Search-based Planning and Replanning on Robotics and Autonomous Systems", Advanced Path Planning for Mobile Entities, IntechOpen, 2018.

John J. Craig. Introduction to Robotics: Mechanics and Control. PEARSON, 2009.

Lung-Wen Tsai. Robot analysis: The mechanics of serial and parallel manipulators 1st edition. pages 118129. John Wiley and Sons, Inc, 1999.

Ian Goodfellow and Yoshua Bengio and Aaron CourvilleMask R-CNN implementation, RCNN

Ian Goodfellow and Yoshua Bengio and Aaron Courville, "Deep Learning (Adaptive Computation and Machine Learning)", MIT Press, 2016.

Ross Girshick, Jeff Donahue, Trevor Darrell, Jitendra Malik, " Rich feature hierarchies for accurate object detection and semantic segmentation", Techical report, 2014.

(62 Abstract Views, 79 PDF Downloads)


  • There are currently no refbacks.

Copyright (c) 2018 Than Le, Dang Huynh

Creative Commons License
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.