A Robust Quantile Huber Loss With Interpretable Parameter Adjustment In Distributional Reinforcement Learning

Distributional Reinforcement Learning (RL) estimates return distribution mainly by learning quantile values via minimizing the quantile Huber loss function, entailing a threshold parameter often selected heuristically or via hyperparameter search, which may not generalize well and can be suboptimal. This paper introduces a generalized quantile Huber loss function derived from Wasserstein distance (WD) calculation between Gaussian distributions, capturing noise in predicted (current) and target (Bellman-updated) quantile values. Compared to the classical quantile Huber loss, this innovative loss function enhances robustness against outliers. Notably, the classical Huber loss function can be seen as an approximation of our proposed loss, enabling parameter adjustment by approximating the amount of noise in the data during the learning process. Empirical tests on Atari games, a common application in distributional RL, and a recent hedging strategy using distributional RL, validate the effectiveness of our proposed loss function and its potential for parameter adjustments in distributional RL. The implementation of the proposed loss function is available here.Comment: 6 pages, 1 figure, to be published in ICASSP 202

Scalable Inference in Hardware Verification and Social Graph Reasoning

With the growing abundance of raw data and the increasing processing power of parallel computing systems, various industries nowadays embrace data mining and are rapidly replacing traditional methods with machine learning solutions. One area that is currently undergoing such transition is the semiconductor industry. In this domain several problems call for data-driven solutions, with failure triage being one of the most prominent. Failure triage is an emerging regression verification problem where the goal is to sort through thousands of hardware failures in order to characterize and prioritize them before they are analyzed in detail and fixed. This preprocessing step is critical in deciding how resources (i.e., engineering effort) will be allocated in fixing these failure. Today, industries assign these decisions to expert engineers. Such manual approaches are unsustainable long-term with design complexity on the rise and verification data becoming more and more sensitive to human interpretation. In the first part of this thesis we show that existing inference methods can be employed towards automating failure triage. Experiments demonstrate that the proposed inference methods can make correct decisions at various triage stages with at least 80% average accuracy, and only a 6.5% runtime overhead when used in modern debugging flows. In contrast to the semiconductor industry, a long thread of work based on data mining has already been established in the social media analytics sector, with applications ranging from sentiment analysis to community detection and viral marketing. In the second part, we study one of the dominant problems in computational social science, known as influence maximization. The problem relates to identifying a subset of a population that can spread information over a social network in a maximal way. We address influence maximization in the continuous-time domain, where the propagation of influence between members of the population obeys stochastic transmission times. Maximizing influence in the continuous-time domain is generally intractable and, therefore, the process calls for approximations. This dissertation proposes novel approximation algorithms that are specifically tailored for parallel computing, and that achieve between 5x-85x performance improvements over the state-of-the-art.Ph.D

High Level Debugging Techniques for Modern Verification Flows

Early closure to functional correctness of the final chip has become a crucial success factor in the semiconductor industry. In this context, the tedious task of functional debugging poses a significant bottleneck in modern electronic design processes, where new problems related to debugging are constantly introduced and predominantly performed manually. This dissertation proposes methodologies that address two emerging debugging problems in modern design flows. First, it proposes a novel and automated triage framework for Register-Transfer-Level (RTL) debugging. The proposed framework employs clustering techniques to automate the grouping of a plethora of failures that occur during regression verification. Experiments demonstrate accuracy improvements of up to 40% compared to existing triage methodologies. Next, it introduces new techniques for Field Programmable Gate Array (FPGA) debugging that leverage reconfigurability to allow debugging to operate without iterative executions of computationally-intensive design re-synthesis tools. Experiments demonstrate productivity improvements of up to 30 x vs. conventional approaches.MAS

Accelerating Post Silicon Debug of Deep Electrical Faults

Abstract—With the growing complexity of current designs and shrinking time-to-market, traditional ATPG methods fail to detect all electrical faults in the design. Debug teams have to spend considerable amount of time and effort to identify these faults during post silicon debug. This work proposes off-chip analysis to speed-up the effort of identifying hard-to-find electrical faults that are not detected using conventional test methods, but cause the chip to crash during functional testing or silicon-bring-up. With the goal of reducing the search space for reconstructing the failure trace path formal methodology is used to analyze the reachable states along the path. Isolating the root cause of failure is also accelerated. Moreover, we propose a forward traversal technique on a selected few possible faults to generate a complete failure trace starting from the initial state to the crash state. Experimental results show that the proposed approach can significantly reduce the actual silicon run thereby reducing the overall debug time. I

Leveraging Reconfigurability to Raise Productivity in FPGA Functional Debug

Abstract—We propose new hardware and software techniques for FPGA functional debug that leverage the inherent reconfigurability of the FPGA fabric to reduce functional debugging time. The functionality of an FPGA circuit is represented by a programming bitstream that specifies the configuration of the FPGA’s internal logic and routing. The proposed methodology allows different sets of design internal signals to be traced solely by changes to the programming bitstream followed by device reconfiguration and hardware execution. Evidently, the advantage of this new methodology vs. existing debug techniques is that it operates without the need of iterative executions of the computationally-intensive design re-synthesis, placement and routing tools. In essence, with a single execution of the synthesis flow, the new approach permits a large number of internal signals to be traced for an arbitrary number of clock cycles using a limited number of external pins. Experimental results using commercial FPGA vendor tools demonstrate productivity (i.e. run-time) improvements of up to 30 × vs. a conventional approach to FPGA functional debugging. These results demonstrate the practicality and effectiveness of the proposed approach. I