Enhancing graphics quality and optimizing power consumption considering the human visual system in mobile devices

학위논문 (박사)-- 서울대학교 대학원 : 전기·컴퓨터공학부, 2016. 2. 신현식.최근까지 GPU의 하드웨어가 눈에 띄게 발전하고 있지만, 아직도 60fps를 만족하면서 높은 품질의 그래픽 요구사항을 만족하기 어렵다. 또한 최근 높은 해상도의 요구사항은 전력 소모와 온도 문제 관점에서도 매우 어려운 문제이다. GPU의 전력 소모는 GPU의 연산량과 정비례하기 때문에, 사람의 인지 능력 관점에서 이득이 없음에도 불구하고, 고정된 높은 해상도와 높은 프레임 속도로 인한 GPU 높은 연산량은 의미가 없다. 본 논문에서는 사람의 인지 능력을 고려한 GPU 연산량을 줄이는 새로운 방법들을 제안한다. 사람의 인지 능력을 고려한 GPU 연산량을 줄이는 시작 단계로, 전력 소모의 주요 요인들을 상용화된 LG G3 모바일 기기로 분석한다. 이 과정을 통해 모바일 GPU의 전력 소모의 3 가지 주요 요인인 해상도, 프레임 속도 그리고 데이터 중복성에 대해 분석한다. 이러한 주요 요인들을 기반으로 사람의 인지 능력을 고려한 새로운 렌더링 기법들을 통해 연산량을 효과적으로 절감하는 기법들을 제안한다. 첫째로 해상도 관점에서 GPU에서의 해상도 변경 기반 연산량 감소 기법들에 대해 제안한다. 최근의 연구들은 사람의 인지능력과 콘텐츠의 특성을 반영하지 못하여, 그래픽 결점이 지속적으로 관찰된다. 기존 연구들과는 다르게, 제안하는 동적 렌더링 화질 개선 스케일링 (Dynamic Rendering Quality Scaling: DRQS)은 최소한의 추가비용으로 변환 행렬을 활용한 프레임 간 변화량을 이용하여 해상도 조절 및 품질 개선 스케일링을 통해 성능을 최대 38%까지 개선한다. 또한 저 사양 그래픽스 응용프로그램의 경우에서는 사람의 인지 능력관점에서 그래픽 품질의 감소 없이 GPU의 연산량을 24%까지 줄인다. 둘째로 프레임 보간 기법을 활용한 그래픽 품질 향상 기법들에 대해서 제안한다. 최근의 프레임 보간 방식은 모션 보상 기반의 알고리즘 기반으로 중간프레임을 생성하기 때문에 요구되는 높은 비용은 모바일에서 적용할 수 없다. 이러한 문제를 개선하기 위해, 새로운 접근 방식인 GPU의 타일 렌더링을 이용한 중간 프레임 전달 방식의 프레임 보간 기법은 지연과 추가적인 높은 비용 없이 중간 프레임을 생성한다. 제안하는 기법을 통해 기존의 연구들 대비 시스템 관점에서 절반의 연산 비용으로 사람의 인지 능력 관점에서 동등한 그래픽 품질을 얻을 수 있다. 마지막으로 가장 최근에 발표된 OpenGL ES 3.0에서 제안된 기술인 Multi render target(MRT) 기술을 재사용 관점에서 최적화하기 위한 방법을 제안한다. MRT 는 지연 쉐이딩을 통한 복잡한 라이팅 연산을 효율적으로 처리하기 위해 많이 사용되는 기술이다. 하지만, 한꺼번에 렌더 타깃에 렌더링을 해야 하기 때문에 큰 메모리 대역폭을 요구한다, 이러한 문제는 제한된 모바일 환경에서는 큰 장애이다. 이 문제를 개선하기 위해 시간적 중복성을 이용한 데이터 재사용을 통해 이미 쓰인 렌더 타깃의 데이터를 선택적으로 재사용하여 GPU의 연산량 및 메모리 사용을 감소시킨다. 실험을 통해 사람의 인지 능력 과점에서 그래픽 품질을 유지하면서 18%의 시스템 레벨의 전력 소모 감소를 얻을 수 있다.제 1 장 서 론 1 1.1 연구 목적 1 1.2 연구 공헌 3 1.3 논문 구성 7 제 2 장 연구 배경 10 2.1 모바일 그래픽스의 발전 10 2.1.1 모바일 그래픽스 하드웨어의 진화 10 2.1.2 모바일 그래픽스 소프트웨어 진화 14 2.2 모바일 환경의 소모 전력 분석 19 2.3 해상도 23 2.4 프레임 속도 25 2.5 데이터 중복 26 제 3 장 가변 해상도 기반 최적화 29 3.1 거리 기반 가변 해상도 변환 기법 29 3.2 응용 프로그램 특성 기반 해상도 변환 기법 32 3.3 동적 렌더링 기반 전력 소모 최적화 및 품질 개선 33 3.3.1 인간 시각 시스템 기반 동적 렌더링 35 3.3.2 변환 행렬을 통한 변화량 계산 38 3.3.3 그래픽 품질 개선 스케일링 44 제 4 장 프레임 속도 기반 최적화 47 4.1 프레임 보간 47 4.2 정방향 재 투영 기법 49 4.3 역방향 재 투영 기법 52 4.4 폐색 영역 처리 및 한계 54 4.5 인간 시각 시스템 기반 홀드-타입 뭉개짐 55 4.6 타일 기반 GPU의 전력 소모 최적화 및 품질 개선 58 4.6.1 타일 기반 렌더링 60 4.6.2 중간 프레임 전달 기법 기반 프레임 속도 증가 63 4.6.3 인간 시각 시스템 기반 프레임 분석 69 4.6.4 렌더링 우선순위 계산 및 합성 72 제 5 장 데이터 재사용을 통한 최적화 77 5.1 데이터 재사용 77 5.2 멀티 렌더 타깃의 데이터 재사용을 통한 최적화 78 5.2.1 멀티 렌더 타깃 80 5.2.2 시간적 일관성 기반 데이터 재사용 83 5.2.3 렌더 타깃 저장 87 5.2.4 인간 시각 시스템 기반 렌터 타깃 재사용 88 제 6 장 성능 분석 92 6.1 실험 환경 93 6.1.1 구현 및 환경 93 6.1.2 실험 벡터 95 6.1.3 시각 시스템 기반 화질 평가 기준 96 6.2 성능 및 소모 전력 평가 99 6.2.1 프레임 간 변화량을 이용한 동적 렌더링 기법 99 6.2.2 타일 기반 GPU 를 위한 프레임 속도 증가 기법 104 6.2.3 멀티 렌더 타깃을 위한 데이터 재사용 기법 108 제 7 장 결론 115 참고 문헌 118 Abstract 126Docto

Exploring mesh shaders

Dissertação de mestrado integrado em Engenharia InformáticaEvery artist is somewhat limited by the mean by which they expose their art. This is also true for the field of Computer Graphics, where there are many limiting factors that developers must go out of their way to avoid. The most limiting of these factors is the computing performance, which directly limits the complexity of what an artist can fabricate in a piece of hardware. As such, Computer Graphics’ investigators keep an eye out for the improvements made in the hardware department that enables them to introduce more complexity to the scenes they create on their computers. Three years ago, a novel approach to compute the geometric complexity of three-dimensional (3D) scenes was introduced: Mesh shaders. Mesh shaders pose as an alternative to the traditional geometric processing method and can be a more performant approach to handle specific geometric workloads. Notwithstanding, little attention has been given to these shaders. Thus, this thesis presents an investigative effort to evaluate the value proposition of these shaders across different scenarios. To do so, this thesis puts Mesh shaders against traditional implementations and measures their differences both in method and performance. By the end of this thesis, the reader should have a concise understanding of Mesh shaders, but not a clear cut answer regarding their use. These shaders can provide performance benefits in specific scenarios over the traditional approach, but not without considerable care by the developer. In fact, the flexibility provided by the Mesh shaders’ approach gives the developer a significant responsibility regarding their final performance. When incorrectly set up, these shaders can result in mediocre performances compared to those of the traditional pipeline. Ultimately, these shaders should be used by experienced users intending to avoid specific bottlenecks of the traditional approach. For others, the traditional pipeline offers a more streamlined approach, thoroughly optimised by default.Todos os artistas são de alguma forma limitados pelo meio de exposição da sua arte. Isto não deixa de ser verdade com Computação Gráfica, onde existem vários fatores limitadores que os programadores têm de con tornar. Entre estes, o mais impeditivo é a velocidade de computação, que limita diretamente a complexidade da arte que pode ser produzida por uma peça de hardware. Deste modo, os investigadores da área de Computação Gráfica mantêm-se atentos às inovações que ocorrem no campo do hardware e lhes permitem introduzir mais complexidade nos cenários que criam. Há três anos, um método inédito para tratar a complexidade geométrica de cenas tridimensionais foi intro duzido: Mesh shaders. Os Mesh shaders apresentam-se como uma alternativa ao método tradicional de pro cessamento de geometria, que pode obter melhor desempenho em certos cenários geométricos. No entanto, não tem sido dada muita atenção a esta alternativa. Assim, esta tese apresenta uma investigação destes shaders com o intuito de avaliar a sua proposta de valor em diferentes situações. Para o fazer, esta tese irá colocar estes shaders frente a frente com os shaders tradicionais e medirá as diferenças entre ambos, tanto em desempenho como em método. No final, o leitor deverá possuir uma ideia coesa sobre os Mesh shaders, mas não terá uma perceção binária quanto ao uso dos mesmos. Isto porque estes shaders podem oferecer um benefício em termos de desempenho em certas situações, mas requerem cuidados adicionais por parte do programador. Da flexibilidade oferecida pelos Mesh shaders advém uma responsabilidade significativa para o programador no que toca ao desempenho final dos mesmos. Quando programados incorretamente, estes shaders resultarão num desempenho medíocre comparado ao desempenho oferecido pelo método tradicional. Fundamentalmente, estes shaders deverão ser utilizados por utilizadores mais experientes que pretendem evitar bottlenecks específicos do método tradicional. Para todos os outros, o pipeline tradicional oferece um método mais simples que possui por predefinição otimizações acentuadas

Interactive web-based visualization

The visualization of large amounts of data, which cannot be easily copied for processing on a user’s local machine, is not yet a fully solved problem. Remote visualization represents one possible solution approach to the problem, and has long been an important research topic. Depending on the device used, modern hardware, such as high-performance GPUs, is sometimes not available. This is another reason for the use of remote visualization. Additionally, due to the growing global networking and collaboration among research groups, collaborative remote visualization solutions are becoming more important. The additional use of collaborative visualization solutions is eventually due to the growing global networking and collaboration among research groups. The attractiveness of web-based remote visualization is greatly increased by the wide availability of web browsers on almost all devices; these are available today on all systems - from desktop computers to smartphones. In order to ensure interactivity, network bandwidth and latency are the biggest challenges that web-based visualization algorithms have to solve. Despite the steady improvements in available bandwidth, these improvements are still significantly slower than, for example, processor performance, resulting in increasing the impact of this bottleneck. For example, visualization of large dynamic data in low-bandwidth environments can be challenging because it requires continuous data transfer. However, bandwidth improvement alone cannot improve the latency because it is also affected by factors such as the distance between server and client and network utilization. To overcome these challenges, a combination of techniques is needed to customize the individual processing steps of the visualization pipeline, from efficient data representation to hardware-accelerated rendering on the client side. This thesis first deals with related work in the field of remote visualization with a particular focus on interactive web-based visualization and then presents techniques for interactive visualization in the browser using modern web standards such as WebGL and HTML5. These techniques enable the visualization of dynamic molecular data sets with more than one million atoms at interactive frame rates using GPU-based ray casting. Due to the limitations which exist in a browser-based environment, the concrete implementation of the GPU-based ray casting had to be customized. Evaluation of the resulting performance shows that GPU-based techniques enable the interactive rendering of large data sets and achieve higher image quality compared to polygon-based techniques. In order to reduce data transfer times and network latency, and improve rendering speed, efficient approaches for data representation and transmission are used. Furthermore, this thesis introduces a GPU-based volume-ray marching technique based on WebGL 2.0, which uses progressive brick-wise data transfer, as well as multiple levels of detail in order to achieve interactive volume rendering of datasets stored on a server. The concepts and results presented in this thesis contribute to the further spread of interactive web-based visualization. The algorithmic and technological advances that have been achieved form a basis for further development of interactive browser-based visualization applications. At the same time, this approach has the potential for enabling future collaborative visualization in the cloud.Die Visualisierung großer Datenmengen, welche nicht ohne Weiteres zur Verarbeitung auf den lokalen Rechner des Anwenders kopiert werden können, ist ein bisher nicht zufriedenstellend gelöstes Problem. Remote-Visualisierung stellt einen möglichen Lösungsansatz dar und ist deshalb seit langem ein relevantes Forschungsthema. Abhängig vom verwendeten Endgerät ist moderne Hardware, wie etwa performante GPUs, teilweise nicht verfügbar. Dies ist ein weiterer Grund für den Einsatz von Remote-Visualisierung. Durch die zunehmende globale Vernetzung und Kollaboration von Forschungsgruppen gewinnt kollaborative Remote-Visualisierung zusätzlich an Bedeutung. Die Attraktivität web-basierter Remote-Visualisierung wird durch die weitreichende Verfügbarkeit von Web-Browsern auf nahezu allen Endgeräten enorm gesteigert; diese sind heutzutage auf allen Systemen - vom Desktop-Computer bis zum Smartphone - vorhanden. Bei der Gewährleistung der Interaktivität sind Bandbreite und Latenz der Netzwerkverbindung die größten Herausforderungen, welche von web-basierten Visualisierungs-Algorithmen gelöst werden müssen. Trotz der stetigen Verbesserungen hinsichtlich der verfügbaren Bandbreite steigt diese signifikant langsamer als beispielsweise die Prozessorleistung, wodurch sich die Auswirkung dieses Flaschenhalses immer weiter verstärkt. So kann beispielsweise die Visualisierung großer dynamischer Daten in Umgebungen mit geringer Bandbreite eine Herausforderung darstellen, da kontinuierlicher Datentransfer benötigt wird. Dennoch kann die alleinige Verbesserung der Bandbreite keine entsprechende Verbesserung der Latenz bewirken, da diese zudem von Faktoren wie der Distanz zwischen Server und Client sowie der Netzwerkauslastung beeinflusst wird. Um diese Herausforderungen zu bewältigen, wird eine Kombination verschiedener Techniken für die Anpassung der einzelnen Verarbeitungsschritte der Visualisierungspipeline benötigt, angefangen bei effizienter Datenrepräsentation bis hin zu hardware-beschleunigtem Rendering auf der Client-Seite. Diese Doktorarbeit befasst sich zunächst mit verwandten Arbeiten auf dem Gebiet der Remote-Visualisierung mit besonderem Fokus auf interaktiver web-basierter Visualisierung und präsentiert danach Techniken für die interaktive Visualisierung im Browser mit Hilfe moderner Web-Standards wie WebGL und HTML5. Diese Techniken ermöglichen die Visualisierung dynamischer molekularer Datensätze mit mehr als einer Million Atomen bei interaktiven Frameraten durch die Verwendung GPU-basierten Raycastings. Aufgrund der Einschränkungen, welche in einer Browser-basierten Umgebung vorliegen, musste die konkrete Implementierung des GPU-basierten Raycastings angepasst werden. Die Evaluation der daraus resultierenden Performanz zeigt, dass GPU-basierte Techniken das interaktive Rendering von großen Datensätzen ermöglichen und eine im Vergleich zu Polygon-basierten Techniken höhere Bildqualität erreichen. Zur Verringerung der Übertragungszeiten, Reduktion der Latenz und Verbesserung der Darstellungsgeschwindigkeit werden effiziente Ansätze zur Datenrepräsentation und übertragung verwendet. Des Weiteren wird in dieser Doktorarbeit eine GPU-basierte Volumen-Ray-Marching-Technik auf Basis von WebGL 2.0 eingeführt, welche progressive blockweise Datenübertragung verwendet, sowie verschiedene Detailgrade, um ein interaktives Volumenrendering von auf dem Server gespeicherten Datensätzen zu erreichen. Die in dieser Doktorarbeit präsentierten Konzepte und Resultate tragen zur weiteren Verbreitung von interaktiver web-basierter Visualisierung bei. Die erzielten algorithmischen und technologischen Fortschritte bilden eine Grundlage für weiterführende Entwicklungen von interaktiven Visualisierungsanwendungen auf Browser-Basis. Gleichzeitig hat dieser Ansatz das Potential, zukünftig kollaborative Visualisierung in der Cloud zu ermöglichen

Towards Computational Efficiency of Next Generation Multimedia Systems

To address throughput demands of complex applications (like Multimedia), a next-generation system designer needs to co-design and co-optimize the hardware and software layers. Hardware/software knobs must be tuned in synergy to increase the throughput efficiency. This thesis provides such algorithmic and architectural solutions, while considering the new technology challenges (power-cap and memory aging). The goal is to maximize the throughput efficiency, under timing- and hardware-constraints

Kommunikation und Bildverarbeitung in der Automation

In diesem Open-Access-Tagungsband sind die besten Beiträge des 9. Jahreskolloquiums "Kommunikation in der Automation" (KommA 2018) und des 6. Jahreskolloquiums "Bildverarbeitung in der Automation" (BVAu 2018) enthalten. Die Kolloquien fanden am 20. und 21. November 2018 in der SmartFactoryOWL, einer gemeinsamen Einrichtung des Fraunhofer IOSB-INA und der Technischen Hochschule Ostwestfalen-Lippe statt. Die vorgestellten neuesten Forschungsergebnisse auf den Gebieten der industriellen Kommunikationstechnik und Bildverarbeitung erweitern den aktuellen Stand der Forschung und Technik. Die in den Beiträgen enthaltenen anschaulichen Beispiele aus dem Bereich der Automation setzen die Ergebnisse in den direkten Anwendungsbezug

Optimization techniques for computationally expensive rendering algorithms

Realistic rendering in computer graphics simulates the interactions of light and surfaces. While many accurate models for surface reflection and lighting, including solid surfaces and participating media have been described; most of them rely on intensive computation. Common practices such as adding constraints and assumptions can increase performance. However, they may compromise the quality of the resulting images or the variety of phenomena that can be accurately represented. In this thesis, we will focus on rendering methods that require high amounts of computational resources. Our intention is to consider several conceptually different approaches capable of reducing these requirements with only limited implications in the quality of the results. The first part of this work will study rendering of time-­¿varying participating media. Examples of this type of matter are smoke, optically thick gases and any material that, unlike the vacuum, scatters and absorbs the light that travels through it. We will focus on a subset of algorithms that approximate realistic illumination using images of real world scenes. Starting from the traditional ray marching algorithm, we will suggest and implement different optimizations that will allow performing the computation at interactive frame rates. This thesis will also analyze two different aspects of the generation of anti-­¿aliased images. One targeted to the rendering of screen-­¿space anti-­¿aliased images and the reduction of the artifacts generated in rasterized lines and edges. We expect to describe an implementation that, working as a post process, it is efficient enough to be added to existing rendering pipelines with reduced performance impact. A third method will take advantage of the limitations of the human visual system (HVS) to reduce the resources required to render temporally antialiased images. While film and digital cameras naturally produce motion blur, rendering pipelines need to explicitly simulate it. This process is known to be one of the most important burdens for every rendering pipeline. Motivated by this, we plan to run a series of psychophysical experiments targeted at identifying groups of motion-­¿blurred images that are perceptually equivalent. A possible outcome is the proposal of criteria that may lead to reductions of the rendering budgets

Dynamic data structures and saliency-influenced rendering

With increasing heterogeneity of modern hardware, different requirements for 3d applications arise. Despite the fact that real-time rendering of photo-realistic images is possible using today’s graphics cards, still large computational effort is required. Furthermore, smart-phones or computers with older, less powerful graphics cards may not be able to reproduce these results. To retain interactive rendering, usually the detail of a scene is reduced, and so less data needs to be processed. This removal of data, however, may introduce errors, so called artifacts. These artifacts may be distracting for a human spectator when gazing at the display. Thus, the visual quality of the presented scene is reduced. This is counteracted by identifying features of an object that can be removed without introducing artifacts. Most methods utilize geometrical properties, such as distance or shape, to rate the quality of the performed reduction. This information used to generate so called Levels Of Detail (LODs), which are made available to the rendering system. This reduces the detail of an object using the precalculated LODs, e.g. when it is moved into the back of the scene. The appropriate LOD is selected using a metric, and it is replaced with the current displayed version. This exchange must be made smoothly, requiring both LOD-versions to be drawn simultaneously during a transition. Otherwise, this exchange will introduce discontinuities, which are easily discovered by a human spectator. After completion of the transition, only the newly introduced LOD-version is drawn and the previous overhead removed. These LOD-methods usually operate with discrete levels and exploit limitations of both the display and the spectator: the human. Humans are limited in their vision. This ranges from being unable to distinct colors at varying illumination scenarios to the limitation to focus only at one location at a time. Researchers have developed many applications to exploit these limitations to increase the quality of an applied compression. Some popular methods of vision-based compression are MPEG or JPEG. For example, a JPEG compression exploits the reduced sensitivity of humans regarding color and so encodes colors with a lower resolution. Also, other fields, such as auditive perception, allow the exploitation of human limitations. The MP3 compression, for example, reduces the quality of stored frequencies if other frequencies are masking it. For representation of perception various computer models exist. In our rendering scenario, a model is advantageous that cannot be influenced by a human spectator, such as the visual salience or saliency. Saliency is a notion from psycho-physics that determines how an object “pops out” of its surrounding. These outstanding objects (or features) are important for the human vision and are directly evaluated by our Human Visual System (HVS). Saliency combines multiple parts of the HVS and allows an identification of regions where humans are likely to look at. In applications, saliency-based methods have been used to control recursive or progressive rendering methods. Especially expensive display methods, such as pathtracing or global illumination calculations, benefit from a perceptual representation as recursions or calculations can be aborted if only small or unperceivable errors are expected to occur. Yet, saliency is commonly applied to 2d images, and an extension towards 3d objects has only partially been presented. Some issues need to be addressed to accomplish a complete transfer. In this work, we present a smart rendering system that not only utilizes a 3d visual salience model but also applies the reduction in detail directly during rendering. As opposed to normal LOD-methods, this detail reduction is not limited to a predefined set of levels, but rather a dynamic and continuous LOD is created. Furthermore, to apply this reduction in a human-oriented way, a universal function to compute saliency of a 3d object is presented. The definition of this function allows to precalculate and store object-related visual salience information. This stored data is then applicable in any illumination scenario and allows to identify regions of interest on the surface of a 3d object. Unlike preprocessed methods, which generate a view-independent LOD, this identification includes information of the scene as well. Thus, we are able to define a perception-based, view-specific LOD. Performance measures of a prototypical implementation on computers with modern graphic cards achieved interactive frame rates, and several tests have proven the validity of the reduction. The adaptation of an object is performed with a dynamic data structure, the TreeCut. It is designed to operate on hierarchical representations, which define a multi-resolution object. In such a hierarchy, the leaf nodes contain the highest detail while inner nodes are approximations of their respective subtree. As opposed to classical hierarchical rendering methods, a cut is stored and re-traversal of a tree during rendering is avoided. Due to the explicit cut representation, the TreeCut can be altered using only two core operations: refine and coarse. The refine-operation increases detail by replacing a node of the tree with its children while the coarse-operation removes the node along with its siblings and replaces them with their parent node. These operations do not rely on external information and can be performed in a local manner. These only require direct successor or predecessor information. Different strategies to evolve the TreeCut are presented, which adapt the representation using only information given by the current cut. These evaluate the cut by assigning either a priority or a target-level (or bucket) to each cut-node. The former is modelled as an optimization problem that increases the average priority of a cut while being restricted in some way, e.g. in size. The latter evolves the cut to match a certain distribution. This is applied in cases where a prioritization of nodes is not applicable. Both evaluation strategies operate with linear time complexity with respect to the size of the current TreeCut. The data layout is chosen to separate rendering data and hierarchy to enable multi-threaded evaluation and display. The object is adapted over multiple frames while the rendering is not interrupted by the used evaluation strategy. Therefore, we separate the representation of the hierarchy from the rendering data. Due to its design, this overhead imposed to the TreeCut data structure does not influence rendering performance, and a linear time complexity for rendering is retained. The TreeCut is not only limited to alter geometrical detail of an object. The TreeCut has successfully been applied to create a non-photo-realistic stippling display, which draws the object with equal sized points in varying density. In this case the bucket-based evaluation strategy is utilized, which determines the distribution of the cut based on local illumination information. As an alternative, an attention drawing mechanism is proposed, which applies the TreeCut evaluation strategies to define the display style of a notification icon. A combination of external priorities is used to derive the appropriate icon version. An application for this mechanism is a messaging system that accounts for the current user situation. When optimizing an object or scene, perceptual methods allow to account for or exploit human limitations. Therefore, visual salience approaches derive a saliency map, which encodes regions of interest in a 2d map. Rendering algorithms extract importance from such a map and adapt the rendering accordingly, e.g. abort a recursion when the current location is unsalient. The visual salience depends on multiple factors including the view and the illumination of the scene. We extend the existing definition of the 2d saliency and propose a universal function for 3d visual salience: the Bidirectional Saliency Weight Distribution Function (BSWDF). Instead of extracting the saliency from 2d image and approximate 3d information, we directly compute this information using the 3d data. We derive a list of equivalent features for the 3d scenario and add them to the BSWDF. As the BSWDF is universal, also 2d images are covered with the BSWDF, and the calculation of the important regions within images is possible. To extract the individual features that contribute to visual salience, capabilities of modern graphics card in combination with an accumulation method for rendering is utilized. Inspired from point-based rendering methods local features are summed up in a single surface element (surfel) and are compared with their surround to determine whether they “pop out”. These operations are performed with a shader-program that is executed on the Graphics Processing Unit (GPU) and has direct access to the 3d data. This increases processing speed because no transfer of the data is required. After computation, each of these object-specific features can be combined to derive a saliency map for this object. Surface specific information, e.g. color or curvature, can be preprocessed and stored onto disk. We define a sampling scheme to determine the views that need to be evaluated for each object. With these schemes, the features can be interpolated for any view that occurs during rendering, and the according surface data is reconstructed. These sampling schemes compose a set of images in form of a lookup table. This is similar to existing rendering techniques, which extract illumination information from a lookup. The size of the lookup table increases only with the number of samples or the image size used for creation as the images are of equal size. Thus, the quality of the saliency data is independent of the object’s geometrical complexity. The computation of a BSWDF can be performed either on a Central Processing Unit (CPU) or a GPU, and an implementation requires only a few instructions when using a shader program. If the surface features have been stored during a preprocess, a reprojection of the data is performed and combined with the current information of the object. Once the data is available, the computation of the saliency values is done using a specialized illumination model, and a priority for each primitive is extracted. If the GPU is used, the calculated data has to be transferred from the graphics card. We therefore use the “transform feedback” capabilities, which allow high transfer rates and preserve the order of processed primitives. So, an identification of regions of interest based on the currently used primitives is achieved. The TreeCut evaluation strategies are then able to optimize the representation in an perception-based manner. As the adaptation utilizes information of the current scene, each change to an object can result in new visual salience information. So, a self-optimizing system is defined: the Feedback System. The output generated by this system converges towards a perception-optimized solution. To proof the saliency information to be useful, user tests have been performed with the results generated by the proposed Feedback System. We compared a saliency-enhanced object compression to a pure geometrical approach, common for LOD-generation. One result of the tests is that saliency information allows to increase compression even further as possible with the pure geometrical methods. The participants were not able to distinguish between objects even if the saliency-based compression had only 60% of the size of the geometrical reduced object. If the size ratio is greater, saliency-based compression is rated, on average, with higher score and these results have a high significance using statistical tests. The Feedback System extends an 3d object with the capability of self-optimization. Not only geometrical detail but also other properties can be limited and optimized using the TreeCut in combination with a BSWDF. We present a dynamic animation, which utilizes a Software Development Kit (SDK) for physical simulations. This was chosen, on the one hand, to show the universal applicability of the proposed system, and on the other hand, to focus on the connection between the TreeCut and the SDK. We adapt the existing framework, and include the SDK within our design. In this case, the TreeCut-operations not only alter geometrical but also simulation detail. This increases calculation performance because both the rendering and the SDK operate on less data after the reduction has been completed. The selected simulation type is a soft-body simulation. Soft-bodies are deformable in a certain degree but retain their internal connection. An example is a piece of cloth that smoothly fits the underlying surface without tearing apart. Other types are rigid bodies, i.e. idealistic objects that cannot be deformed, and fluids or gaseous materials, which are well suited for point-based simulations. Any of these simulations scales with the number of simulation nodes used, and a reduction of detail increases performance significantly. We define a specialized BSWDF to evaluate simulation specific features, such as motion. The Feedback System then increases detail in highly salient regions, e.g. those with large motion, and saves computation time by reducing detail in static parts of the simulation. So, detail of the simulation is preserved while less nodes are simulated. The incorporation of perception in real-time rendering is an important part of recent research. Today, the HVS is well understood, and valid computer models have been derived. These models are frequently used in commercial and free software, e.g. JPEG compression. Within this thesis, the Tree-Cut is presented to change the LOD of an object in a dynamic and continuous manner. No definition of the individual levels in advance is required, and the transitions are performed locally. Furthermore, in combination with an identification of important regions by the BSWDF, a perceptual evaluation of a 3d object is achieved. As opposed to existing methods, which approximate data from 2d images, the perceptual information is directly acquired from 3d data. Some of this data can be preprocessed if necessary, to defer additional computations during rendering. The Feedback System, created by the TreeCut and the BSWDF, optimizes the representation and is not limited to visual data alone. We have shown with our prototype that interactive frame rates can be achieved with modern hardware, and we have proven the validity of the reductions by performing several user tests. However, the presented system only focuses on specific aspects, and more research is required to capture even more capabilities that a perception-based rendering system can provide