Education at the interface of cultural practice and cultural technology

Der Beitrag entwickelt einen (medien-)pädagogischen Orientierungsrahmen, der nicht von digitalen Technologien als Produkt, sondern von der Technikgenese als Prozess ausgeht. Der Prozess der Digitalisierung wird als performativer Vorgang der diskretisierenden, abstrahierenden und formalisierenden Beschreibung praktischer Vollzüge gefasst. Dies erlaubt, den genuinen Einfluss digitaler Technologien und ihres praktischen Gebrauchs auf Bildungsprozesse zu untersuchen und kritisch zu reflektieren. Der Rahmen offenbart das grundlegende Spannungsverhältnis zwischen der unterstellten Annahme einer prinzipiellen Geregeltheit praktischer Vollzüge und der unhintergehbaren Komplexität und Kontingenz kultureller Praxis. Die Perspektive erweitert damit den Blick über die Betrachtung digitaler Technologien hinaus hin zu der Frage nach den kulturellen Praktiken und den in ihnen tradierten operationalen Formen.This article develops a (media)educational framework that is based on the genesis of technology as a process rather than on digital technologies as a product. The process of digitalization is understood as a performative process of discretized, abstracted, and formalized description of cultural practice. This allows to identify and reflect the genuine impact of digital technologies on educational processes. The framework reveals the fundamental tension between the underlying assumption of practice as describable and the inevitable complexity and contingency of cultural practice. The perspective thus goes beyond the analysis of digital technologies as it reflects the cultural practices and its operational forms

Representations and representation learning for image aesthetics prediction and image enhancement

With the continual improvement in cell phone cameras and improvements in the connectivity of mobile devices, we have seen an exponential increase in the images that are captured, stored and shared on social media. For example, as of July 1st 2017 Instagram had over 715 million registered users which had posted just shy of 35 billion images. This represented approximately seven and nine-fold increase in the number of users and photos present on Instagram since 2012. Whether the images are stored on personal computers or reside on social networks (e.g. Instagram, Flickr), the sheer number of images calls for methods to determine various image properties, such as object presence or appeal, for the purpose of automatic image management and curation. One of the central problems in consumer photography centers around determining the aesthetic appeal of an image and motivates us to explore questions related to understanding aesthetic preferences, image enhancement and the possibility of using such models on devices with constrained resources. In this dissertation, we present our work on exploring representations and representation learning approaches for aesthetic inference, composition ranking and its application to image enhancement. Firstly, we discuss early representations that mainly consisted of expert features, and their possibility to enhance Convolutional Neural Networks (CNN). Secondly, we discuss the ability of resource-constrained CNNs, and the different architecture choices (inputs size and layer depth) in solving various aesthetic inference tasks: binary classification, regression, and image cropping. We show that if trained for solving fine-grained aesthetics inference, such models can rival the cropping performance of other aesthetics-based croppers, however they fall short in comparison to models trained for composition ranking. Lastly, we discuss our work on exploring and identifying the design choices in training composition ranking functions, with the goal of using them for image composition enhancement

COMPUTATIONAL MODELLING OF HUMAN AESTHETIC PREFERENCES IN THE VISUAL DOMAIN: A BRAIN-INSPIRED APPROACH

Following the rise of neuroaesthetics as a research domain, computational aesthetics has also known a regain in popularity over the past decade with many works using novel computer vision and machine learning techniques to evaluate the aesthetic value of visual information. This thesis presents a new approach where low-level features inspired from the human visual system are extracted from images to train a machine learning-based system to classify visual information depending on its aesthetics, regardless of the type of visual media. Extensive tests are developed to highlight strengths and weaknesses of such low-level features while establishing good practices in the domain of study of computational aesthetics. The aesthetic classification system is not only tested on the most widely used dataset of photographs, called AVA, on which it is trained initially, but also on other photographic datasets to evaluate the robustness of the learnt aesthetic preferences over other rating communities. The system is then assessed in terms of aesthetic classification on other types of visual media to investigate whether the learnt aesthetic preferences represent photography rules or more general aesthetic rules. The skill transfer from aesthetic classification of photos to videos demonstrates a satisfying correct classification rate of videos without any prior training on the test set created by Tzelepis et al. Moreover, the initial photograph classifier can also be used on feature films to investigate the classifier’s learnt visual preferences, due to films providing a large number of frames easily labellable. The study on aesthetic classification of videos concludes with a case study on the work by an online content creator. The classifier recognised a significantly greater percentage of aesthetically high frames in videos filmed in studios than on-the-go. The results obtained across datasets containing videos of diverse natures manifest the extent of the system’s aesthetic knowledge. To conclude, the evolution of low-level visual features is studied in popular culture such as in paintings and brand logos. The work attempts to link aesthetic preferences during contemplation tasks such as aesthetic rating of photographs with preferred low-level visual features in art creation. It questions whether favoured visual features usage varies over the life of a painter, implicitly showing a relationship with artistic expertise. Findings display significant changes in use of universally preferred features over influential vi abstract painters’ careers such an increase in cardinal lines and the colour blue; changes that were not observed in landscape painters. Regarding brand logos, only a few features evolved in a significant manner, most of them being colour-related features. Despite the incredible amount of data available online, phenomena developing over an entire life are still complicated to study. These computational experiments show that simple approaches focusing on the fundamentals instead of high-level measures allow to analyse artists’ visual preferences, as well as extract a community’s visual preferences from photos or videos while limiting impact from cultural and personal experiences

Choosing an aesthetically optimal photographic image from a sequence of similar or nearly identical images

Digital Photography has caused an exponential increase of captured images. The process of reviewing and selecting the most beautiful images, in regard to some well-known aesthetic criteria, is a time consuming task. Excellent and repeatable face and object-recognition and image classification tasks of various kinds of images (portrait, landscape, macro, sports) are the result of applied machine learning algorithms. Because of the increase in hardware computing-power deep neural networks are becoming more and more popular. This master thesis analyses solutions and tools which can (semi)automatically find the most aesthetically pleasing images from a sequence of images. In this thesis the methods and tools of classical machine learning and those of deep convolutional neural networks are compared. On the basis of my professional photographic experience and images in various fields of photography (documentary, wedding, sport) a comercial prototype application is tested and some solutions to the problem are suggested

複雑さに関連した特徴を用いた主観的印象予測

学位の種別: 課程博士審査委員会委員 : （主査）東京大学准教授 山﨑 俊彦, 東京大学教授 相澤 清晴, 国立情報学研究所教授 佐藤 真一, 東京大学教授 佐藤 洋一, 東京大学教授 苗村 健University of Tokyo(東京大学

Sparse Gradient Optimization and its Applications in Image Processing

Millions of digital images are captured by imaging devices on a daily basis. The way imaging devices operate follows an integral process from which the information of the original scene needs to be estimated. The estimation is done by inverting the integral process of the imaging device with the use of optimization techniques. This linear inverse problem, the inversion of the integral acquisition process, is at the heart of several image processing applications such as denoising, deblurring, inpainting, and super-resolution. We describe in detail the use of linear inverse problems in these applications. We review and compare several state-of-the-art optimization algorithms that invert this integral process. Linear inverse problems are usually very difficult to solve. Therefore, additional prior assumptions need to be introduced to successfully estimate the output signal. Several priors have been suggested in the research literature, with the Total Variation (TV) being one of the most prominent. In this thesis, we review another prior, the l0 pseudo-norm over the gradient domain. This prior allows full control over how many non-zero gradients are retained to approximate prominent structures of the image. We show the superiority of the l0 gradient prior over the TV prior in recovering genuinely piece-wise constant signals. The l0 gradient prior has shown to produce state-of-the-art results in edge-preserving image smoothing. Moreover, this general prior can be applied to several other applications, such as edge extraction, clip-art JPEG artifact removal, non-photorealistic image rendering, detail magnification, and tone mapping. We review and evaluate several state-of-the-art algorithms that solve the optimization problem based on the l0 gradient prior. Subsequently we apply the l0 gradient prior to two applications where we show superior results as compared to the current state-of-the-art. The first application is that of single-image reflection removal. Existing solutions to this problem have shown limited success because of the highly ill-posed nature of the problem. We show that the standard l0 gradient prior with a modified data-fidelity term based on the Laplacian operator is able to sufficiently remove unwanted reflections from images in many realistic scenarios. We conduct extensive experiments and show that our method outperforms the state-of-the-art. In the second application of haze removal from visible-NIR image pairs we propose a novel optimization framework, where the prior term penalizes the number of non-zero gradients of the difference between the output and the NIR image. Due to the longer wavelengths of NIR, an image taken in the NIR spectrum suffers significantly less from haze artifacts. Using this prior term, we are able to transfer details from the haze-free NIR image to the final result. We show that our formulation provides state-of-the-art results compared to haze removal methods that use a single image and also to those that are based on visible-NIR image pairs

Spuren digitaler Artikulationen

Scheinbar unablässig wird in den Sozialen Medien gepostet und geliked. Soziale Medien fungieren als Orte informeller und direkter Kommunikation sowie Interaktion, an denen neue kulturelle Ausdrucksformen verhandelt und weitergetragen werden. Hier zeigt sich eine derart komplexe Gemengelage unterschiedlichster und zugleich aufeinander bezogener digitaler Mediennutzungspraktiken, dass die theoretische Annäherung an die Überschüssigkeit sozialer Praktiken in Sozialen Medien nur im Denken eines Entzugs stattfinden kann. Ausgehend von der Interdependenz von Aisthesis, Artikulation und Technik unternehmen die Beiträger*innen des Bandes Versuche, die Spuren kultureller Bildungs- und Transformationsprozesse in Sozialen Medien aus unterschiedlichen disziplinären Perspektiven (auf-)zulesen

Spuren digitaler Artikulationen: Interdisziplinäre Annäherungen an Soziale Medien als kultureller Bildungsraum

Scheinbar unablässig wird in den Sozialen Medien gepostet und geliked. Soziale Medien fungieren als Orte informeller und direkter Kommunikation sowie Interaktion, an denen neue kulturelle Ausdrucksformen verhandelt und weitergetragen werden. Hier zeigt sich eine derart komplexe Gemengelage unterschiedlichster und zugleich aufeinander bezogener digitaler Mediennutzungspraktiken, dass die theoretische Annäherung an die Überschüssigkeit sozialer Praktiken in Sozialen Medien nur im Denken eines Entzugs stattfinden kann. Ausgehend von der Interdependenz von Aisthesis, Artikulation und Technik unternehmen die Beiträger*innen des Bandes Versuche, die Spuren kultureller Bildungs- und Transformationsprozesse in Sozialen Medien aus unterschiedlichen disziplinären Perspektiven (auf-)zulesen