Computer face recognition and identification is an important problem with numerous applications in intelligent user interfaces, security, and accessibility. For example, identification of faces can be used for access to restricted areas, authentication without keyboards, and personalized interfaces. Traditional algorithms that solve this problem identify faces individually or use co-occurrence information for a given set of people (For example, person X is often photographed with person Y). However, such systems have many errors, and cannot generalize well to people the system has not seen together previously. In contrast, humans are much better at this because we have "common sense." We can look at a photograph and recognize that people tend to photograph with family, friends, and co-workers. We can further guess why the photograph (birthday, conference) was taken, and even where (person's house, a certain park). This abstract knowledge is very difficult for computer algorithms to learn because there are not sets of these "rules" available explicitly. In our work, we tackle this problem using structured relational graphs like Freebase, a large online graph of billions of facts about the world. Given many specific examples of people who are in photographs together and data about those photographs, we can see what connects these people on graphs like Freebase, and generalize to the "common sense" rules that are notoriously difficult to learn. Then, given a new image and possible candidates for identification of people in it, we can compute the most likely set of correct identifications. We can also hypothesize unknown faces to our existing faces. This allows us to learn, on-the-fly, a new recognizer for that specific person. Finally, by knowing relationships behind photograph subjects, we can discover and generate useful contextual information about the photograph, such as location and possible reason for the photograph. Broadly, this means that computer systems can more accurately identify meaning behind photographs; a problem that has been historically hard for computers to accomplish.