From Optimization to Control: Quasi Policy Iteration

Recent control algorithms for Markov decision processes (MDPs) have been designed using an implicit analogy with well-established optimization algorithms. In this paper, we make this analogy explicit across four problem classes with a unified solution characterization. This novel framework, in turn, allows for a systematic transformation of algorithms from one domain to the other. In particular, we identify equivalent optimization and control algorithms that have already been pointed out in the existing literature, but mostly in a scattered way. With this unifying framework in mind, we then exploit two linear structural constraints specific to MDPs for approximating the Hessian in a second-order-type algorithm from optimization, namely, Anderson mixing. This leads to a novel first-order control algorithm that modifies the standard value iteration (VI) algorithm by incorporating two new directions and adaptive step sizes. While the proposed algorithm, coined as quasi-policy iteration, has the same computational complexity as VI, it interestingly exhibits an empirical convergence behavior similar to policy iteration with a very low sensitivity to the discount factor

Optimizing Hardware Simulation and Realization of Discrete Cosine Transform Using VHDL Hardware Description Language

Discrete cosine transform (DCT) is the fundamental part of JPEG compressor and is one of the most widely used conversion technique in digital signal processing (DSP) and image compression. Due to importance of the discrete cosine transform in JPEG standard, an algorithm is proposed that is in parallel structure thus intensify hardware implementation speed of discrete cosine transform and JPEG compression procedure. The proposed method is implemented by utilizing VHDL hardware description language in structural format and follows optimal programming tips by which, low hardware resource utilization, low latency, high throughput and high clock rate are achieved. Inputs are 8-bit long, 4 separate units are considered and CSA and CLA adders are used to realize discrete cosine transform. Working frequency for this implementation is 100 MHz and each stage delay is 10ns which is optimum in comparison with other methods. This proposed method can be easily utilized in any hardware applications such as JPEG compressor, image/signal processing and etc. by minimum change in design parameters. Also, it can be used as a hard-core in embedded systems, system on chips (SOC), system on programmable chips (SOPC) and network on chips (NOC)

Condicionamento osmótico das sementes e disponibilidade de enxofre na estabilidade da membrana celular e produtividade de soja em solo salino

The objective of this work was to determine the effects of seed priming and sulfur application on cell membrane characteristics, seedling emergence, chlorophyll content and grain yield of soybean (Glycine max) in saline soil. A complete-block design in 4x3 factorial arrangement with three replicates was used to test four types of seed priming (water, auxin, gibberellin and non-priming) and three levels of sulfate availability (0, 70 and 140 kg ha-1 K2SO4). The soil had a silty loam texture with an electrical conductivity of 3.61 ds m-1, a pH of 8.2 and a saturation percentage of about 46%. Seed priming had significant effects on mean emergence rate (MER), emergence percentage, relative water content (RWC) of leaves, relative chlorophyll content, time of maturity, shoot length and grain yield. The highest values for these variables were observed in the priming treatments, except for the time of maturity. Sulfur application had significant effects on MER, shoot length, RWC, membrane injury index and grain yield. Priming treatments provide greater emergence rates and grain yields and interact sinergicaly with sulfur rates.O objetivo deste trabalho foi determinar os efeitos do condicionamento osmótico das sementes e da disponibilidade de enxofre sobre características da membrana celular, emergência de plântulas, conteúdo relativo de clorofila e produtividade de soja (Glycine max) em solo salino. O experimento foi conduzido em delineamento de blocos ao acaso, em arranjo fatorial 4x3, com três repetições, para avaliar quatro tipos de condicionamento osmótico das sementes (água, auxina, giberelina e ausência de condicionamento) e três níveis de disponibilidade de enxofre para as plantas (aplicação ao solo de 0, 70 e 140 kg ha-1 de K2SO4). O solo utilizado apresentava textura média, com condutividade elétrica de aproximadamente 3,61 ds m-1, pH de 8,2 e percentagem de saturação em torno de 46%. O condicionamento osmótico das sementes teve efeito significativo sobre a taxa média de emergência (MER), percentagem de emergência, conteúdo relativo de água das folhas (RWC), conteúdo relativo de clorofila, época de maturação, comprimento da parte aérea e produtividade de grãos. Os maiores valores para essas variáveis, exceto época de maturação, foram observados nos tratamentos com condicionamento osmótico. A aplicação de enxofre teve efeito significativo sobre MER, comprimento da parte aérea, RWC, índice de dano à membrana celular, e produtividade de grãos. O condicionamento osmótico proporciona maiores taxas de emergência e produtividade de grãos e interage significativamente com a aplicação de enxofre

Modulation of Drug Craving in Crystalline-Heroin Users by Transcranial Direct Current Stimulation of Dorsolateral Prefrontal Cortex

Background: Drug craving, the main cause of relapse and a major motivator for drug use, is a challenging obstacle in substance use treatment. Transcranial direct current stimulation (tDCS), a non-invasive neuromodulatory technique, has shown promising outcomes in treating different neuropsychiatric disorders such as drug addiction, more specifically on drug craving. The aim in the current study was to examine the effects of applying tDCS on dorsolateral prefrontal cortex (DLPFC) in reducing drug cravings in former crystalline-heroin users enrolled in methadone maintenance (MMT) programs. Methods: The present study was a semi-experimental, crossover study with pre/post-test, and a control group. 40 right-handed men were selected from former crystalline-heroin users enrolled in MMT programs in Tehran, Iran. They were then divided into two matched groups based on age, education, and age of onset crystalline-heroin abuse. Desire for Drug Questionnaire (DDQ) was administered two times to all of the subjects, before first brain stimulation, and at the end of the last session. Experimental group received TDCS on DLPFC, and sham stimulation was applied on control subjects. The data were analyzed by analysis of covariance (ANCOVA) method using SPSS software. Findings: The study results indicated anodal tDCS over right and cathodal TDCS over left DLPFC, and in parallel with sham, significantly decreased drug cravings among former crystalline-heroin users (P < 0.050). Conclusion: This study showed that applying TDCS on DLPFC of former crystalline-heroin users reduces drug craving. The findings of this study expanded the results of previous studies on effects of this neuromodulatory technique for drug craving reduction in other drug type settings

Correlation and path coefficient analysis of yield and yield component in some of broad bean (Vicia faba l.) genotypes

Faba bean is a grain legume and grown for its high protein content in the seed. It is also serves as a rotational crop which play great role in controlling disease epidemics in areas were cereal mono-cropping is abundant. Yield in faba bean, similar to the other crops, is a complex trait and constitute by many of morphological and physiological traits. This study was carried out during 2011-12 and 2012-13 in two region of Iran including Guilan and Lorestan provinces. Field experiments were conducted in a randomized complete block design with three replications and ten genotypes. The results of combined analysis of variance indicated that the studied genotypes differed significantly for all of the studied traits. The results indicated also environment effect and environment × genotype interaction effects were significant or highly significant for all of the traits. The highest seed yield were determined for genotype 1 (3159.9 and 4016.9 kg ha-1 at 2012 and 2013, respectively) in Guilan and genotype 5 (495.44 kg ha-1) in Lorestan. The results of correlation analysis indicated that there were positive significant correlation coefficients between seed yield and seed length (LS), seed width (WS), pod length (PL) and hundred seed weight (HSW) in Guilan province at two cropping season. Path coefficient analysis indicated that traits containing number of pod per plant, number of steam per plant, pod length, seed length/width ratio and hundred seed weight had the highest positive direct effects on dry seed yield in studied faba bean genotypes. Attention should be paid to some of characters such as pod length, hundred seed weight, number of pods per plant and number of stems per plant for augmentation of seed yield and these traits could be used as selection criteria in faba bean breeding programs. These findings indicate that selection for each or full of the above traits would be accompanied by high yielding ability under such conditions. It could be concluded that the high yielding genotypes, such as 1 and 2 could be used to improve faba beans and making possibilities of extending production of this legume crop