7,788 research outputs found

    Backpropagation Beyond the Gradient

    Get PDF
    Automatic differentiation is a key enabler of deep learning: previously, practitioners were limited to models for which they could manually compute derivatives. Now, they can create sophisticated models with almost no restrictions and train them using first-order, i. e. gradient, information. Popular libraries like PyTorch and TensorFlow compute this gradient efficiently, automatically, and conveniently with a single line of code. Under the hood, reverse-mode automatic differentiation, or gradient backpropagation, powers the gradient computation in these libraries. Their entire design centers around gradient backpropagation. These frameworks are specialized around one specific task—computing the average gradient in a mini-batch. This specialization often complicates the extraction of other information like higher-order statistical moments of the gradient, or higher-order derivatives like the Hessian. It limits practitioners and researchers to methods that rely on the gradient. Arguably, this hampers the field from exploring the potential of higher-order information and there is evidence that focusing solely on the gradient has not lead to significant recent advances in deep learning optimization. To advance algorithmic research and inspire novel ideas, information beyond the batch-averaged gradient must be made available at the same level of computational efficiency, automation, and convenience. This thesis presents approaches to simplify experimentation with rich information beyond the gradient by making it more readily accessible. We present an implementation of these ideas as an extension to the backpropagation procedure in PyTorch. Using this newly accessible information, we demonstrate possible use cases by (i) showing how it can inform our understanding of neural network training by building a diagnostic tool, and (ii) enabling novel methods to efficiently compute and approximate curvature information. First, we extend gradient backpropagation for sequential feedforward models to Hessian backpropagation which enables computing approximate per-layer curvature. This perspective unifies recently proposed block- diagonal curvature approximations. Like gradient backpropagation, the computation of these second-order derivatives is modular, and therefore simple to automate and extend to new operations. Based on the insight that rich information beyond the gradient can be computed efficiently and at the same time, we extend the backpropagation in PyTorch with the BackPACK library. It provides efficient and convenient access to statistical moments of the gradient and approximate curvature information, often at a small overhead compared to computing just the gradient. Next, we showcase the utility of such information to better understand neural network training. We build the Cockpit library that visualizes what is happening inside the model during training through various instruments that rely on BackPACK’s statistics. We show how Cockpit provides a meaningful statistical summary report to the deep learning engineer to identify bugs in their machine learning pipeline, guide hyperparameter tuning, and study deep learning phenomena. Finally, we use BackPACK’s extended automatic differentiation functionality to develop ViViT, an approach to efficiently compute curvature information, in particular curvature noise. It uses the low-rank structure of the generalized Gauss-Newton approximation to the Hessian and addresses shortcomings in existing curvature approximations. Through monitoring curvature noise, we demonstrate how ViViT’s information helps in understanding challenges to make second-order optimization methods work in practice. This work develops new tools to experiment more easily with higher-order information in complex deep learning models. These tools have impacted works on Bayesian applications with Laplace approximations, out-of-distribution generalization, differential privacy, and the design of automatic differentia- tion systems. They constitute one important step towards developing and establishing more efficient deep learning algorithms

    LIPIcs, Volume 251, ITCS 2023, Complete Volume

    Get PDF
    LIPIcs, Volume 251, ITCS 2023, Complete Volum

    Spectrum auctions: designing markets to benefit the public, industry and the economy

    Get PDF
    Access to the radio spectrum is vital for modern digital communication. It is an essential component for smartphone capabilities, the Cloud, the Internet of Things, autonomous vehicles, and multiple other new technologies. Governments use spectrum auctions to decide which companies should use what parts of the radio spectrum. Successful auctions can fuel rapid innovation in products and services, unlock substantial economic benefits, build comparative advantage across all regions, and create billions of dollars of government revenues. Poor auction strategies can leave bandwidth unsold and delay innovation, sell national assets to firms too cheaply, or create uncompetitive markets with high mobile prices and patchy coverage that stifles economic growth. Corporate bidders regularly complain that auctions raise their costs, while government critics argue that insufficient revenues are raised. The cross-national record shows many examples of both highly successful auctions and miserable failures. Drawing on experience from the UK and other countries, senior regulator Geoffrey Myers explains how to optimise the regulatory design of auctions, from initial planning to final implementation. Spectrum Auctions offers unrivalled expertise for regulators and economists engaged in practical auction design or company executives planning bidding strategies. For applied economists, teachers, and advanced students this book provides unrivalled insights in market design and public management. Providing clear analytical frameworks, case studies of auctions, and stage-by-stage advice, it is essential reading for anyone interested in designing public-interested and successful spectrum auctions

    Art and design learning journey: interactions between learners and materials

    Get PDF
    This thesis is an empirical explorative and new materialist qualitative research journey representing a secondary school art and design teacher’s awakening to the importance and vitality of art education to young learners with regards to their own intrinsic learning journey and their subsequent wider outlook on life. Secondary education and specifically art education is vulnerable and prone to political whims, lack of interest and shifts of policy since 1768 and the founding of the Royal Academy. The historical and political lineage of art and design education is outlined along with the lasting impact of language used within more recent statutory documentation. Little research currently exists that specifically looks at what is generated within the processes of making and doing that are intrinsic to creative activity and are lived out in every art and design classroom environment. Within this thesis I will explore the rich potential for haptic and tacit knowledge to be generated within the creative process, driven by heuristic experiences. I will also highlight the generation of powerful emotional relationships generated between human and non-human actants which occur as students engage with making and doing within the art classroom. Through working directly with different creative processes and materials, including research, poetry, design, and ceramics, two classes of year 9 students explored both collaboratively and individually the value of making and responding to both their own learning experience and that of working with others. The physicist and academic Karen Barad offers a novel platform of diffractive analysis with which to interpret the research project data in order to challenge the accepted positionality of merely working through a creative process in a procedural way. Diffractive analysis is also central in the analysis of the intra-actions between human and nonhuman actants opening up further discussions challenging established hierarchy and status quo presently found in secondary education. The genesis of the creative process is explored through the material discursive phenomena created through the intra-actions between human and nonhuman matter

    Making Connections: A Handbook for Effective Formal Mentoring Programs in Academia

    Get PDF
    This book, Making Connections: A Handbook for Effective Formal Mentoring Programs in Academia, makes a unique and needed contribution to the mentoring field as it focuses solely on mentoring in academia. This handbook is a collaborative institutional effort between Utah State University’s (USU) Empowering Teaching Open Access Book Series and the Mentoring Institute at the University of New Mexico (UNM). This book is available through (a) an e-book through Pressbooks, (b) a downloadable PDF version on USU’s Open Access Book Series website), and (c) a print version available for purchase on the USU Empower Teaching Open Access page, and on Amazon

    Amazonian Vision: Representations of Women Artists in Victorian Fiction

    Get PDF
    Title from PDF of title page, viewed June 14, 2023Dissertation advisors: Jennifer Phegley and Linda MitchellVitaIncludes bibliographical references (pages 320-337)Dissertation (Ph.D.)--Department of English Language and Literature. Department of History. University of Missouri--Kansas City, 2023This dissertation examines representations of women artists—writers, musicians, painters, and photographers—in nineteenth-century British novels and poetry written by Charlotte Brontë, Elizabeth Barrett Browning, George Eliot, Anne Brontë, Dinah Craik, Charlotte Yonge, and Amy Levy. It analyzes how their heroines wield literal and metaphorical vision to navigate the male gaze and male surveillance of the Victorian art world. These authors utilize the symbiotic relationship between vision and art to contest binary societal definitions that insisted men were creative and women imitative. This study is arranged by forms of vision adopted by the characters addressed in each chapter. Chapter one examines how the heroines of Charlotte Brontë’s Jane Eyre and Elizabeth Barrett Browning’s Aurora Leigh exercise “spiritual vision,” which facilitates Miltonic artistic agency as they author autobiographies following the blinding of their (male) romantic counterparts. Chapter two examines George Eliot’s use of contrasting characters in Middlemarch and Daniel Deronda to show how Eliot’s women must step outside of the frame as art objects and wield “moral vision” to realize her vision of the artist as an instrument of human sympathy. Chapter three examines the “Amazonian vision” adopted by women painters in Anne Brontë’s The Tenant of Wildfell Hall, Dinah Craik’s Olive, and Charlotte Yonge’s The Pillars of the House; they forge entry into the historically male-dominated visual art world and achieve financial self-sufficiency by selling their work. Finally, chapter four examines how adopting “metropolitan vision” empowers the speaker of Amy Levy’s “A London Plane-Tree” poems and the Lorimer sisters in her novel The Romance of a Shop, respectively, as a poet and as professional photographers. This work utilizes an interdisciplinary approach to synthesize discussion of the novels with historical sources—primarily art histories, biographies, the authors’ diaries and letters, and nineteenth-century periodical press articles. It finds that, in consideration of historical circumstances, the women authors under discussion exercised progressive vision of their own. This vision was surprisingly radical in its early manifestations but often reliant on spiritualization and abstraction; over time, in fiction as in history, women artists’ presence in the art world gained immediacy and strength.Spiritual vision: the miltonic artist in Jane Eyre and Aurora Leigh -- Moral vision: sympathy, vanity, and art in Middlemarch and Daniel Deronda -- Amazonian vision: painterly success in The tenant of Wildfell Hall, Olive, and The pillars of the house -- Metropolitan vision: London-inspired art in Amy Levy's "A London plane-tree" poems and The romance of a sho

    Constitutions of Value

    Get PDF
    Gathering an interdisciplinary range of cutting-edge scholars, this book addresses legal constitutions of value. Global value production and transnational value practices that rely on exploitation and extraction have left us with toxic commons and a damaged planet. Against this situation, the book examines law’s fundamental role in institutions of value production and valuation. Utilising pathbreaking theoretical approaches, it problematizes mainstream efforts to redeem institutions of value production by recoupling them with progressive values. Aiming beyond radical critique, the book opens up the possibility of imagining and enacting new and different value practices. This wide-ranging and accessible book will appeal to international lawyers, socio-legal scholars, those working at the intersections of law and economy and others, in politics, economics, environmental studies and elsewhere, who are concerned with rethinking our current ideas of what has value, what does not, and whether and how value may be revalued

    Performing Both Sides of the Glass: Videogame Affordances and Live Streaming on Twitch

    Get PDF
    This thesis examines the performative dimensions videogame affordances assume within online, live streaming environments. This approach considers how streamers configure their videogame play in terms of a potential audience, drawing on five semi-structured, in-depth interviews with Australian-based Twitch streamers to analyse how streamers leverage videogame affordances to produce “meaningful moments”. Guiding this thesis is the question of how the player-videogame relationship is maintained, fractured or altered within live-streaming environments such as Twitch

    Moving away from the Monoplot: conventional narrative structure in the screenplay and the unconventional alternatives

    Get PDF
    Through a combination of creative and critical practice, this project seeks to explore the conventionalisation effect that popular screenwriting handbooks have had on screenwriting practice, the unconventional structural models that are available to the screenwriter once they move away from the conventional model, and the meanings which conventional and unconventional structural approaches create. By examining the most influential screenwriting handbooks, and tracing the historical development of screenwriting conventions, a model for quantifying and understanding conventional narrative structure will be proposed: Conventional Monoplot. An exploration of the cinematic canon, box office statistics and Academy Awards success will attempt to show that the conventionalisation of structural practice within the screenplay has led to an increased homogeneity of form and meaning in mainstream cinema and a concurrent reduction in narrative sophistication and critical esteem. By applying the Conventional Monoplot model in their practice, this project will argue that the screenwriter can quantify and understand divergence from conventional structural practice by negative correlation to the model, and through such practice the homogeneity of film form might be challenged. Through an examination of a wide body of film texts a taxonomy of alternative structural models will be proposed, and the meanings which these models create will be explored. The creative element, a feature screenplay, will demonstrate practical application of one of these models, and a critical reflection will explore the meanings created by use of this unconventional structural model, locating a methodology for unconventional practice in the screenplay. The project will propose that the influence of screenwriting handbooks has led to homogeneity and conventionalisation in the culture of the screenplay, and that by consciously focusing on unconventional structural practices the screenwriter can access a greater diversity of meaning at the structural level

    Esports events: classification and impact of business model of video games on size

    Get PDF
    Esports events are not commonly researched in academic literature. This research aims to provide a higher degree of understanding around esports, esports events and their size, and to develop a framework for future research. Video game business models are considered, as their link with esports is not often examined. Overwatch is also investigated as a case study of a single video game and its associated esport. The methodology employed is based on mixed methods. A pragmatic approach is utilised, adjusting the research philosophy based on the most suitable approach for each part of the study. The research design evolves based on the findings and the methods used in the previous chapters. Chapter 4 utilises a mixed methods approach, chapters 5 and 6 both use a quantitative method, before chapter 7 uses a qualitative, case study like technique. Chapter 4 explores the determination of a framework to measure size of events and such framework is created, with 1 event being classed as ''giga'', 16 as ''mega'', 15 as ''major'', and 11 as ''minor''. Chapter 5 undertakes a similar pursuit but utilising an index to rank sizes. There are no large differences in score or in class, and there is a high degree of correlation between the index and the classification from the previous chapter. Chapter 6 explores event size vs. video game business model, finding that events associated to buy-to-play and free-to-play games have a larger size than events associated to pay-to-pay games. Chapter 7 analyses Overwatch and concludes that a switch to a free-to-play model would be beneficial for Overwatch, and for its associated esport Overwatch League. A number of recommendations are made as a result of the research undertaken. Better collection and organisation of data on esports would be beneficial for future research. A centralised governing body would help with a number of aspects in esports. More research could be undertaken into business models, into the implication of choosing one over another and switching between them. A research centre at the European level would also be beneficial, as would the growth of formal structures around esports and esports research
    • …
    corecore