ETRI: Visual Sensing for Natural Human-Robot Interaction
This project aims to create and implement Stevi, a high-level vision subsystem for use with personal service robots. In order to properly function, service robots will require the ability to detect the location and intentions of potential users. To meet these requirements, this system will be able to perform localization, detection and tracking of people, as well gesture recognition in real time using stereo and omnidirectional video.
While computer vision has made significant progress over the years in all of these areas, there is still a significant divide between the abilities of even the most advanced algorithms and the human visual system. The reduced computational power and dynamic nature of most robotic platforms further intensify the need for better vision systems. To address these issues, we make use of the Software Architecture for Immersipresence (SAI) framework to accomplish a fusion of preexisting efficient algorithms at the symbolic level, resulting in a highly robust system.
To test this approach we make use of the WEVER-N, a prototype personal service robot platform developed by ETRI in Korea. This provides us with a working platform for research, including propulsion, servo controlled stereo color cameras, and speech synthesis. Future research will augment this system with static cameras placed strategically throughout the environment to increase Stevi's situational awareness.
Department of Justice: Design, implement, and evaluate the operating performance of a complete prototype system to locate, track and identify people.
We propose to design, implement, and evaluate the operating performance of a complete prototype system to locate, track and identify people going through a predefined zone, indoors or outdoors. This is accomplished by processing images taken from a video camera, inferring the location of the subjects’ head, using this information to capture high resolution face image sequences from a second, dedicated zoom camera mounted on a pan-tilt head, building a 3D face model from this face image sequence, and using this 3D model to perform biometric identification. We show that this approach is applicable with subjects approximately 25 feet away, indoors or outdoors, using off-the-shelf components, and can be extended to 50ft.
We choose faces as our biometric basis, because most of the distinctive and permanent biometric features (such as fingerprints, hand shape, iris or retinal scans) require cooperative subjects in close proximity to the biometric system. Unfortunately, even top 2D face recognition systems today are neither reliable nor accurate enough. We propose instead to perform face recognition in 3D. This allows the use of true shape invariants for recognition, and circumvents difficulties associated with pose and lighting. Our results validate our claim of significant improvement over 2D systems.
To generate these 3D descriptions, we use an image sequence, as natural head and body motion provides enough viewpoint variation to perform stereo-motion for 3D face reconstruction. We have validated this approach on real data.
For recognition, we will use the commercial Geometrix 3D face recognition engine, accessed through their SDK. This is the top-performing package in recent tests conducted by NIST in the Face Recognition Grand Challenge (FRGC) experiment. We will further enhance recognition performance through the use of fused texture information, and face symmetry.
We will first enroll a number (between 10 and 50) of subjects under cooperative conditions. These subjects will be taken from the USC laboratories roster of staff, faculty and students. They will be asked to sign a consent form (appendix E), in accordance to human subjects protocol defined by University regulations. The goal is to then track and recognize these individuals as they walk through the zone of interest. Subjects not enrolled are tracked, but not recognized. We will produce quarterly reports describing our experiments, and quantitatively describe our progress. A final report will relate the recognition rates in terms of image quality, distance, environmental conditions, number of enrollees, and other appropriate parameters.
CIA Post Doctoral Fellowship: 3D Modeling of Faces from Video
3D face models are important for a wide array of applications including surveillance, gaming, military simulation, virtual teleconferencing/chat, and surgical simulation. Of particular interest to the intelligence community is the application to face recognition. Face recognition methods based on 3D face models have been shown to significantly outperform methods based on 2D images alone. The models are insensitive to lighting variations and changes in facial characteristics over time. While the surface of the face may change cosmetically, the underlying 3D geometry remains constant. Acquiring face data from cameras promises unobtrusive recognition, potentially from large distances
The goal of this project is to reconstruct 3D faces from video, leveraging ubiquitous commercial and consumer imaging devices ranging from cell phones to surveillance cameras. Our method relies solely on observed data and does not require the use of strong prior models to constrain the reconstruction, as this often results in models with a strong bias towards the model prior.
NSF Funded Project: Integrated Media Systems Center
This project is part of the Sensory Interfaces (SI) at IMSC. The goals of this project are to bridge gaps between Computer Vision and Computer Graphics.
- Ongoing Projects:
- Completed Projects:
NSF Funded Project: Tensor Voting
This project addresses the central issue of Perceptual Grouping in Computer Vision. Over the past few years, we have developed a unified Tensor Based framework to formalize this problem. The method is not iterative, uses no hard thresholds, and there is a single free parameter.
Ongoing research:
NSF Funded Project: Difficult Problems in Stereo and Motion Analysis
This project addresses two difficult problems in Stereo and Motion analysis. These two areas of computer vision have been explored with numerous approaches and very good results have been achieved, although for limited types of scenes, such as a single, smooth and textured surface. Occlusion is a cause of many remaining difficulties, since most algorithms cannot operate on areas visible in one image only. Furthermore, even though the human visual system has no difficulty estimating the depth or velocity of occluded objects, computer vision systems are incapable of that. They either produce arbitrary results for the occluded regions, or, at best, mark them as occluded. The second issue we need to address is the need for processing at multiple scales. Many real scenes contain objects that are perceived at different scales, both due to variations in size and level of detail and due to variations in texture density. A multiple scale processing scheme is necessary for processing such scenes. It must be capable of detecting and preserving fine details, not allowing larger or richer in texture regions to dominate smaller or less textures ones, achieving good continuation of structures, and filling in missing data.
Ongoing research:
ARDA Funded project: Video Analysis and Content Extraction
The goal of this project is to develop tools for automated analysis of video sequences. This will include methods for detection and tracking of moving objects resulting in a structured description of the video. This representation will then be used for extracting contents that will allow understanding of the events occurring in the scene. Such automated tools will be essential for analysts in the intelligence community to cope with and make effective use of the vast quantities of video data that are becoming increasingly available.
- Research topics:
- Data and Formats:
|
