RNNs Implicitly Implement Tensor Product Representations

Recurrent neural networks (RNNs) can learn continuous vector representations of symbolic structures such as sequences and sentences; these representations often exhibit linear regularities (analogies). Such regularities motivate our hypothesis that RNNs that show such regularities implicitly compile symbolic structures into tensor product representations (TPRs; Smolensky, 1990), which additively combine tensor products of vectors representing roles (e.g., sequence positions) and vectors representing fillers (e.g., particular words). To test this hypothesis, we introduce Tensor Product Decomposition Networks (TPDNs), which use TPRs to approximate existing vector representations. We demonstrate using synthetic data that TPDNs can successfully approximate linear and tree-based RNN autoencoder representations, suggesting that these representations exhibit interpretable compositional structure; we explore the settings that lead RNNs to induce such structure-sensitive representations. By contrast, further TPDN experiments show that the representations of four models trained to encode naturally-occurring sentences can be largely approximated with a bag of words, with only marginal improvements from more sophisticated structures. We conclude that TPDNs provide a powerful method for interpreting vector representations, and that standard RNNs can induce compositional sequence representations that are remarkably well approximated by TPRs; at the same time, existing training tasks for sentence representation learning may not be sufficient for inducing robust structural representations.Comment: Accepted to ICLR 201

Lyrics of Love and Laughter

Lyrics of Love and Laughter was crafted following another severe round of pneumonia. With this work, however, Dunbar secured his reputation as the premier African-American poet in the United States at the time. Like earlier volumes, this book contained poems in both dialect and standard English, reflecting its mix of both sentimental and realistic depictions of African-American life. The book included eleven poems not previously collected within a book.https://corescholar.libraries.wright.edu/dunbar/1008/thumbnail.jp

The Strength of Gideon

Dunbar\u27s second collection of short stories echoed his works of poetry in that it was a combination of works full of sentimental humor and works that told of the harsher sides of the African-American experience, both before and after emancipation. Critics tended to promote the sentimental stories at the expense of his harsher accounts of that experience. Later critics, however, suggest these stories demonstrate Dunbar\u27s criticism of racial inequality in American society.https://corescholar.libraries.wright.edu/dunbar/1015/thumbnail.jp

The Love of Landry

Recovering from a strong bout with pneumonia, Dunbar spent time in Colorado attempting to recoup his health. One result of his time there was his second novel, The Love of Landry. The heroine, much like Dunbar, is in Colorado to recuperate and falls in love with a cowboy. Like his previous novel, his second attempt did not find favor with critics, and many focused on what they considered a weak presentation of characters.https://corescholar.libraries.wright.edu/dunbar/1014/thumbnail.jp

Joggin\u27 Erlong

Joggin’ Erlong was the final collection of Dunbar’s poetry published during his lifetime. His health continued to decline after his diagnosis with tuberculosis. Separated from his wife in 1902, Dunbar eventually returned home to Dayton to live with his mother, Matilda. Much of his later work between 1902-1906 reflects his increasingly ill health.https://corescholar.libraries.wright.edu/dunbar/1005/thumbnail.jp

The Heart of Happy Hollow

In his final collection of short stories, Dunbar’s focus remained on life in America for African-Americans, but rather than a singular focus on plantation society, this set of stories presented a variety of perspectives. One story, which grew in favor during the 20th century, focused on the immorality, or what he called the folly, of lynching.https://corescholar.libraries.wright.edu/dunbar/1018/thumbnail.jp

When Malindy Sings

When Malindy Sings is illustrated with photographs taken by the Hampton Institute Camera Club, and also features decorations created by Margaret Armstrong. The title for this book comes from one of Dunbar\u27s most well-known dialect poems, one which emphasizes the more soulful aspects and the heartfelt, impromptu music of many of his poems. The poem was inspired by his mother Matilda\u27s habit of singing in the kitchen and was first published in 1896.https://corescholar.libraries.wright.edu/dunbar/1006/thumbnail.jp

Howdy Honey Howdy

Illustated with photographs by Leigh Richmond Miner, Howdy Honey Howdy contains many previously published poems by Dunbar. One difference from previous covers is the photograph of a woman in a doorway, hand extended, as if welcoming the reader and inviting one to enter.https://corescholar.libraries.wright.edu/dunbar/1004/thumbnail.jp