Robust Face-Name Graph Matching Essay Example for Free

lively looking at-Name represent Matching Essay1. LoginIn this faculty is going to explain the Robust Face-Name Graph Matching for fiberization Character Identification designing and how we did the side of meat signal breakion and recognition in this project. The images go out explain just about the facial fetching details. After that admin going to login with the details which needed for the login page.2. DetectionIn this module we be going to detect the exhibit of the word-painting reference points. In this module we argon using the emgu cv program library we must install the emgu cv library. After installing the emgu cv lib in our project we need to wreak reference with the get wind emgu.cv, emgu.cv.util, emgu.cv.ui. When you entrust complete the references you will get the emgu controls in the toolbox.3. RecognitionIn this module we be going to recognize the subject of the celluloid eccentrics which is we previously stored on the face database. We jus t prep ar that the give the real scream of it. This is going to be done here. Here we ar using the With the servicing of these eigenObjectRecognizer we atomic number 18 going to recognize the face.Chellangess In This Methodology -1. We detect the face in minute this is a big challenge for us because exiting carcass take more time for perception. 2. More challenging trouble due to the huge variation in the way of each fibre. 3. It is increase speed of matching graphic symbol and direct the character.Objectivies -The Robust Face-Name Graph Matching for Movie Character Identification designing and how we did the face detection and recognition in this project. In this project we present two stratagems of global face-name matching establish example for big-shouldered character acknowledgement. It is use in scenes, film, cartoons.Problem Analysis -1. It is difficult to Complex character diversitys are handled by simultaneously represent partition and chartical record matching. 2. Many character are not easy matching and identification face in movies. 3. It is hard to be increase speed of matching character and identify the character.Existing give-up the ghost -This project is used to detect the face of movie characters and recognize the characters in minute process and the existing system are taking the too much time to detect the face. But this one we can do it in a minute process.Proposed Work -In this Robust Face-Name Graph Matching for Movie CharacterIdentification is used to detect the face of movie characters and the Proposed system is taking the minimum time to detect the face. In this One we can do it in a minute process.Robust Face-Name Graph Matching for Movie Character Identification Jitao Sang, Changsheng Xu, Senior Member, IEEEAbstract self-loading face identification of characters in movies has drawn significant research interests and led to many interesting applications. It is a challenging fuss due to the huge variation in the appearance of each character. Although existing regularitys depict promising results in clean environment, the exercises are limited in complex movie scenes due to the noises generated during the face winding and face crowd process. In this piece of music we present two schemes of global face-name matching ground mannikin for robust character identification.The contributions of this work include 1) A noise insensitive character human relationship example is incorporated. 2) We introduce an edit operation establish graph matching algorithm. 3) Complex character changes are handled by simultaneously graph partition and graph matching. 4) Beyond existing character identification approaches, we foster perform an in-depth sensitivity analysis by introducing two types of simulated noises. The proposed schemes demonstrate state-of-the-art performance on movie character identification in various genres of movies. Index TermsCharacter identification, graph matching, graph partiti on, graph edit, sensitivity analysis.Fig. 1. Examples of character identification from movie Notting Hill.I. INTRODUCTIONA. Objective and MotivationThe proliferation of movie and TV submits large amount of digital video data. This has led to the requirement of efficient and effective techniques for video content understanding and organization. Automatic video short letter is one of such key techniques. In this paper our focus is on annotating characters in the movie and TVs, which is called movie character identification 1. The objective is to identify the faces of the characters in the video and label them with the corresponding names in the unload off. The textual cues, like shake off lists, hands, subtitles and unlikeable captions are usually exploited. Fig.1 shows an example in our experiments. In a movie, characters are the focus center of interests for the audience. Their occurrences will lots of clues about the movie structure and content.Automatic character identific ation is essential for semantic movie index and retrieval 2, 3, scene segmentation 4, summarization 5 and other applications 6. Copyright (c) 2010 IEEE. Personal use of this natural is permitted. However, permission to use this material for any other purposes must be obtained from the IEEE by sending a request to pubs emailprotected ieee.org. This work was supported in part by the National Program on light upon Basic Research Project (973 Program, Project No. 2012CB316304) and National Natural Science Foundation of chinaware (Grant No. 90920303, 61003161). J. Sang and C. Xu (corresponding author) are with the National Lab of Pattern Recognition, Institute of Automation, Chinese Academy of Sciences, capital of Red China 100190, China and also with the China- Singapore Institute of Digital Media, Singapore, 119613Character identification, though very intuitive to humans, is a tremendously challenging task in computer vision. The reason is four-fold 1) Weakly supervised textual cues 7. thither are ambiguity business in establishing the correspondence mingled with names and faces ambiguity can pinch from a reaction shot where the person speaking may not be shown in the frames 1 ambiguity can also arise in partially labeled frames when there are nine-fold speakers in the same scene 2. 2) Face identification in videos is more difficult than that in images 8. let out resolution, occlusion, no rigid deformations, large motion, complex background and other uncontrolled conditions make the results of face detection and tracking unreliable. In movies, the situation is take down worse. This brings inevitable noises to the character identification. 3) The same character appears quite other than during the movie 3. There may be huge pose, expression and illumination variation, wearing, clothing, even makeup and hairstyle changes. Moreover, characters in some movies go through and through contrary age stages, e.g., from youth to the old age. Sometimes, there will ev en be different actors playing different ages of the same character. 4) The determination for the material body of identical faces is not unserviceable 2. Due to the remarkable intra-class variance, the same character name will correspond to faces of huge variant appearances.It will be unreasonable to set the bet of identical faces just according to the number of characters in the cast. Our field of battle is motivated by these challenges and aims to find solutions for a robust framework for movie character identification. B. Related Work The crux of the character identification problem is to exploit the relations among videos and the associated texts in pasture 1 I.e., the name in the subtitle/closed caption finds no corresponding faces in the video. 2 I.e., nine-fold names in the subtitle/closed caption correspond to multiple faces in the video.Fig. 2. Framework of scheme 1 Face-name graph matching with cluster pre specifiedto label the faces of characters with names. It ha s similarities to identifying faces in word videos 9, 10, 11. However, in news videos, candidate names for the faces are available from the simultaneously appearing captions or topical anaesthetic anesthetic transcripts. While in TV and movies, the names of characters are seldom now shown in the subtitle or closed caption, and script screenplay containing character names has no time stamps to align to the video. According to the utilized textual cues, we roughly divide the existing movie character identification methods into three categories. 1) Category 1 Cast list based These methods only utilize the case list textual resource. In the cast list discovery problem 12, 13, faces are constellate by appearance and faces of a particular character are expected to be collected in a few pure clusters. Names for the clusters are then manually selected from the cast list. Ramanan et al. proposed to manually label an initial set of face clusters and further cluster the rest face instance s based on clothing within scenes 14. In 15, the authors acquire addressed the problem of finding particular characters by building a theoretical account/classifier of the characters appearance from user-provided training data.An interesting work trust character identification with web image retrieval is proposed in 17. The character names in the cast are used as queries to search face images and constitute gallery set. The probe face tracks in the movie are then identified as one of the characters by multi-task joint sparse mold and classification. Recently, metric learning is introduced into character identification in uncontrolled videos 16. Cast-specific metrics are adapted to the populate appearing in a particular video in an unsupervised manner. The clustering as vigorous as identification performance are demonstrated to be reformd. These cast list based methods are easy for understanding and implementation.However, without other textual cues, they either need manual la beling or guarantee no robust clustering and classification performance due to the large intra-class variances. 2) Category 2 Subtitle or Closed caption, Local matching based Subtitle and closed caption provide time-stamped dialogues, which can be exploited for alignment to the video frames. Effingham et al. 18, 3 proposed to combine the film script with the subtitle for local face-name matching. Time-stamped name annotation and face exemplars are generated. The rest of the faces were then classified into these exemplars for identification. They further extended their work in 19, by replacing the nearest neighbor classifier by multiple kernel learning for features combination.In the new framework, non-frontal faces are handled and the reporting is extended. Researchers from University of Pennsylvania utilized the readily available time-stamped resource, the closed captions, which is demonstrated more reliable than OCR-based subtitles 20, 7. They investigated on the ambiguity issues in the local alignment between video, screenplay and closed captions. A partially-supervised multiclass classification problem is formulated. Recently, they attempt to address the character identification problem without the use of screenplay 21. The reference cues in the closed captions are sedulous as multiple instance constraints and face tracks grouping as well as face-name association are solved in a convex formulation. The local matching based methods require the time-stamped information, which is either extracted by OCR (i.e., subtitle) or unavailable for the majority of movies and TV series (i.e., closed caption).Besides, the ambiguous and partial annotation makes local matching based methods more sensitive to the face detection and tracking noises. 3) Category 3 playscript/Screenplay, Global matching based Global matching based methods open the possibility of character identification without OCR-based subtitle or closed caption. Since it is not easy to get local name cu es, the task of character identification is formulated as a global matching problem in 2, 22, 4. Our method belongs to this form and can be considered as an prolongation to Zhangs work 2.In movies, the names of characters seldom directly appear in the subtitle, while the movie script which contains character names has no time information. Without the local time information, the task of character identification is formulated as a global matching problem between the faces detected from the video and the names extracted from the movie script. Compared with local matching, global statistics are used for name-face association, which enhances the daring of the algorithms. Our work differs from the existing research in threefold Regarding the fact that characters may show various appearances, the means of character is often affectedFig. 3. Framework of scheme 2 Face-name graph matching without cluster pre-specified.by the noise introduced by face tracking, face clustering and scene seg mentation. Although extensive research efforts have been strong on character identification and many applications have been proposed, little work has focused on better the robustness. We have observed in our investigations that some statistic properties are preserved in spite of these noises. Based on that, we propose a novel representation for character relationship and introduce a name-face matching method which can accommodate a certain noise.Face track clustering serves as an authorized flavour in movie character identification. In most of the existing methods some cues are utilized to mildew the number of target clusters prior to face clustering, e.g., in 2, the number of clusters is the same as the number of limpid speakers appearing in the script. While this seems convinced at first glance, it is rigid and even deteriorating the clustering results sometimes. In this paper, we loose the restriction of one face cluster corresponding to one character name. Face track clust ering and face-name matching are jointly optimized and conducted in a unique framework.Sensitivity analysis is uncouth in financial applications, risk analysis, signal processing and any area where models are developed 23, 24. dear(p) modeling practice requires that the modeler provides an evaluation of the confidence in the model, for example, assessing the uncertainties associated with the modeling process and with the outcome of the model itself. For movie character identification, sensitivity analysis offers valid tools for characterizing the robustness to noises for a model. To the best of our knowledge, there have been no efforts directed at the sensitivity analysis for movie character identification. In this paper, we aim to match this gap by introducing two types of simulated noises. A preliminary version of this work was introduced by 1. We provide additive algorithmic and computational details, and extend the framework considering no pre-specification for the number of face clusters. Improved performance as well as robustness are demonstrated in movies with large character appearance changes.C. Overview of Our ApproachIn this paper, we propose a global face-name graph matching based framework for robust movie character identification. Two schemes are considered. There are connections as well as differences between them. Regarding the connections, firstly, the proposed two schemes both belong to the global matching based category, where external script resources are utilized. Secondly, to improve the robustness, the ordinal graph is employed for face and name graph representation and a novel graph matching algorithm called Error Correcting Graph Matching (ECGM) is introduced. Regarding the differences, scheme 1 sets the number of clusters when performing face clustering (e.g., K-means, spectral clustering). The face graph is restricted to have identical number of vertexes with the name graph. While, in scheme 2, no cluster number is required and f ace tracks are clustered based on their intrinsic data structure (e.g., mean shift, simile propagation). Moreover, as shown in Fig.2 and Fig.3, scheme 2 has an additional module of graph partition compared with scheme 1. From this perspective, scheme 2 can be seen as an extension to scheme 1. m1)Scheme 1 The proposed framework for scheme 1 is shown in Fig.2. It is similar to the framework of 2. Face tracks are clustered using constrained K-means, where the number of clusters is set as the number of distinct speakers. continuative of names in script and face clusters in video constitutes the corresponding face graph and name graph. We modify the traditional global matching framework by using ordinal graphs for robust representation and introducing an ECGM-based graph matching method. For face and name graph construction, we propose to represent the character co-occurrence in rank ordinal level 25, which scores thestrength of the relationships in a rank order from the weakest to str ongest. Rank order data carry no numerical meaning and thus are less sensitive to the noises. The simile graph used in the traditional global matching is interval measures of the co-occurrence relationship between characters.While continuous measures of the strength of relationship holds complete information, it is highly sensitive to noises. For name-face graph matching, we utilize the ECGM algorithm. In ECGM, the difference between two graphs is measured by edit distance which is a sequence of graph edit operations. The best match is achieved with the least edit distance. According to the noise analysis, we define appropriate graph edit operations and adapt the distance functions to obtain improved name-face matching performance. 2) Scheme 2 The proposed framework for scheme 2 is shown in Fig.3. It has two differences from scheme 1 in Fig.2. First, no cluster number is required for the face tracks clustering step. Second, since the face graph and name graph may have different nu mber of vertexes, a graph partition component is added before ordinal graph representation. The basic introduce behind the scheme 2 is that appearances of the same character vary significantly and it is difficult to group them in a unique cluster.Take the movie TheCurious Case of Benjamin Button for example. The hero and heroine go through a long time period from their childhood, youth, middle-age to the old-age. The intra-class variance is even larger than the inter-class variance. In this case, simply enforcing the number of face clusters as the numberof characters will disturb the clustering process. Instead of grouping face tracks of the same character into one cluster, face tracks from different characters may be grouped together. In scheme 2, we utilize affinity propagation for the face tracks clustering. With each sample as the potential center of clusters, the face tracks are recursively clustered through appearance-based similarity transmit and propagation. High cluster pu rity with large number of clusters is expected. Since one character name may correspond to several face clusters, graph partition is introduced before graph matching. Which face clusters should be further grouped (i.e., divided into the same subgraph) is determined by whether the partitioned face graph achieves an optimal graph matching with the name graph.Actually, face clustering is divided into two steps coarse clustering by appearance and further modification by script. Moreover, face clustering and graph matching are optimized simultaneously, which improve the robustness against errors and noises. In general, the scheme 2 has two advantages over the scheme 1. (a) For scheme 2, no cluster number is required in advance and face tracks are clustered based on their intrinsic data structure. Therefore, the scheme 2 provides certain robustness to the intra-class variance, which is very common in movies where characters change appearance significantly or go through a long time period. (b) Regarding that movie cast cannot include pedestrians whose face is detected and added into the face track, restricting the number of face tracks clusters the same as that of name from movie cast will deteriorate the clustering process. In addition, there is some chance that movie cast does not cover all the characters. In this case, pre-specification for the face clusters is risky face tracks from different characters will be mixed together and graph matching tends to fail. 3) Sensitivity Analysis Sensitivity analysis plays an important role in characterizing the uncertainties associated with a model. To explicitly analyze the algorithms sensitivity to noises, two types of noises, coverage noise and intensity noise, are introduced. Based on that, we perform sensitivity analysis by analyse the performance of name-face matching with respect to the simulated noises.