Leveraging Factored Action Spaces for Off-Policy Evaluation

Off-policy evaluation (OPE) aims to estimate the benefit of following a counterfactual sequence of actions, given data collected from executed sequences. However, existing OPE estimators often exhibit high bias and high variance in problems involving large, combinatorial action spaces. We investigate how to mitigate this issue using factored action spaces i.e. expressing each action as a combination of independent sub-actions from smaller action spaces. This approach facilitates a finer-grained analysis of how actions differ in their effects. In this work, we propose a new family of "decomposed" importance sampling (IS) estimators based on factored action spaces. Given certain assumptions on the underlying problem structure, we prove that the decomposed IS estimators have less variance than their original non-decomposed versions, while preserving the property of zero bias. Through simulations, we empirically verify our theoretical results, probing the validity of various assumptions. Provided with a technique that can derive the action space factorisation for a given problem, our work shows that OPE can be improved "for free" by utilising this inherent problem structure.Comment: Main paper: 8 pages, 7 figures. Appendix: 30 pages, 17 figures. Accepted at ICML 2023 Workshop on Counterfactuals in Minds and Machines, Honolulu, Hawaii, USA. Camera ready versio

Baggrund for rekonstruktion af ovnene i 1987: Konstruktion, teknik og materiale

Background for the reconstruction of the kilns in 1987. Construction, technology and materials By Birgit Als Hansen and Morten Aaman Sørensen In the space of a few years, three medieval pottery kilns were discovered and excavated in Jutland. The kilns appear to be of varying construction, representing three types not previously found in this country. This inspired the excavators and a small group of kiln enthusiasts to try a reconstruction and subsequent firing of all three kilns. The reconstruction of the kiln from Hellum afforded few problems. The preserved substructure indicated a simple round, updraft kiln, where the floor was supported by a central pillar with radiating clay bars. A type well known all over Europe from ancient times to our day, and in our case very similar to a kiln from Torksey in Lincolnshire (fig. 1). The kiln from Barmer was constructed differently. What was left under ground level looked like a hip bath, with stoke hole and fire chamber in the lower part, and a slightly sloping floor for the pots in the upper part, where an opening at the rear was presumably used for stacking the pottery, and during the firing as a vent for the exhaust. The two chambers were separated by a fire screen in the shape of three solid pillars – very much like kilns found in France and Germany (fig. 2). No traces of the upper parts of the kiln remained in Barmer, but a row of small holes with charred wood in the sidewalls indicated a barrow vault moulded on a wooden structure. In Kragelund, only the front part of the kiln was preserved. The stoke hole, with a flue on either side of a raised platform, indicated a type of kiln found in the Netherlands, France and Germany. None of these kilns offered much information about the structure above ground level, except possibly one of a series of kilns from Brunssum (fig. 3), where the foremost part of a domed roof seemed to have rested on the front part of the platform. A reconstruction along these lines seemed a permissible experiment

Ovnen fra Kragelund: Keramik, rekonstruktion og brænding

The Kragelund kiln. Reconstruction of a kiln and firing of the pottery By Birgit Als Hansen, Morten Aaman Sørensen and Jens Christian Ørting. The kiln from Kragelund was the largest and most complicated of the three kilns in the experiment. The lower part, with the stoke hole, flues, and the front part of an elevated platform that could safely be copied from the plan of the excavation; whereas the domed roof, its shape and the number of openings and vents partly had to be decided on the basis of experience with previous experiments in kiln building. There were no traces of a wooden structure to support the vault during construction and the dome had to be hand-moulded from several layers of clay. A regular hemispherical form was attempted, with moderate success. A small hole was left open in the top of the dome to let steam out in the early phases of the firing. An opening was made to the rear for stacking the pottery, and later used in a diminished form as a vent to let out exhaust. Another vent in the front of the dome was to allow for regulating the draught during firing. After a couple of firings to dry out the kiln, copies of the original pottery, consisting of glazed jars and pots, were stacked on the platform, and the firing could begin. The next 28 hours gave every opportunity to experiment with adjusting the draught and changing the methods of firing. For a 20-hour period, it seemed impossible to force the temperature up to the desired 900 degrees Celsius. Ultimately, it was decided to try an extensive firing, regardless of the amount of wood consumed. Within a few hours the temperature reached 900-1000 degrees, and the firing was stopped. The stokehole and other openings were blocked up, and to intensify a reduced atmosphere in the kiln, a small mountain of previously removed charcoal was put back into the firing chamber. The experiment could hardly be called a success. The kiln was difficult to manage and consumed far too much wood, and although the pottery was well fired, the glaze was grey and lustreless, with small particles of metallic lead. The sudden rise in temperature in the final phase indicated that intensifying the firing at an early stage could save time and wood, and the lustreless glaze might be caused by the large amount of charcoal, producing an atmosphere excessively deficient in oxygen

The intronic BRCA1 c.5407-25T>A variant causing partly skipping of exon 23—a likely pathogenic variant with reduced penetrance?

Rare sequence variants in the non-coding part of the BRCA genes are often reported as variants of uncertain significance (VUS), which leave patients and doctors in a challenging position. The aim of this study was to determine the pathogenicity of the BRCA1 c.5407-25T>A variant found in 20 families from Norway, France and United States with suspected hereditary breast and ovarian cancer. This was done by combining clinical and family information with allele frequency data, and assessment of the variant’s effect on mRNA splicing. Mean age at breast (n = 12) and ovarian (n = 11) cancer diagnosis in female carriers was 49.9 and 60.4 years, respectively. The mean Manchester score in the 20 families was 16.4. The allele frequency of BRCA1 c.5407-25T>A was 1/64,566 in non-Finnish Europeans (gnomAD database v2.1.1). We found the variant in 1/400 anonymous Norwegian blood donors and 0/784 in-house exomes. Sequencing of patient-derived cDNA from blood, normal breast and ovarian tissue showed that BRCA1 c.5407-25T>A leads to skipping of exon 23, resulting in frameshift and protein truncation: p.(Gly1803GlnfsTer11). Western blot analysis of transiently expressed BRCA1 proteins in HeLa cells showed a reduced amount of the truncated protein compared with wild type. Noteworthily, we found that a small amount of full-length transcript was also generated from the c.5407-25T>A allele, potentially explaining the intermediate cancer burden in families carrying this variant. In summary, our results show that BRCA1 c.5407-25T>A leads to partial skipping of exon 23, and could represent a likely pathogenic variant with reduced penetrance.publishedVersio

Plasma equol concentration is not associated with breast cancer and fibrocystic breast conditions among women in Shanghai, China

Equol (a bacterial metabolite of the soy isoflavone daidzein) is produced by 30% to 50% of humans and may be associated with health outcomes. We hypothesized that plasma equol would be inversely associated with risks of fibrocystic breast conditions (FBC) and breast cancer (BC). Plasma from women in a breast self-examination trial in Shanghai with BC (n = 269) or FBC (n = 443), and age-matched controls (n = 1027) was analyzed for isoflavones. Equol was grouped into categories (= 45 nmol/L) and, among women with daidzein >= 20 nmol/L, the log(10) equol:daidzein ratio was grouped into tertiles. Where available, non-cancerous tissue (NCT) adjacent to the carcinomas from women with BC were classified as non-proliferative or proliferative (n = 130 and 172, respectively). The lesions from women with FBC were similarly classified (n = 99 and 92, respectively). Odds ratios (OR) and 95% confidence intervals (CI) were calculated across equol categories and tertiles of log(10) equol:daidzein ratio. Equol categories were not associated with FBC or BC >.05). For log(10) equol:daidzein, compared to controls there were positive associations in the mid tertile for proliferative FBC (OR 2.06, 95% CI 1.08-3.93), BC with proliferative NCT (OR 2.95, 95% CI 1.37-6.35), and all BC regardless of histology (OR 2.37, 95% CI 1.43-3.95). However, trends in ORs with increasing plasma equol values or equol:daidzein ratios were not observed (P >.05). The results of this study do not provide evidence that equol plays a role in the etiology of these breast conditions. However, further work is needed to confirm or refute this conclusion. (C) 2016 Elsevier Inc. All rights reserved.Peer reviewe

Pre-Clinical Evaluation of a Replication-Competent Recombinant Adenovirus Serotype 4 Vaccine Expressing Influenza H5 Hemagglutinin

Influenza virus remains a significant health and social concern in part because of newly emerging strains, such as avian H5N1 virus. We have developed a prototype H5N1 vaccine using a recombinant, replication-competent Adenovirus serotype 4 (Ad4) vector, derived from the U.S. military Ad4 vaccine strain, to express the hemagglutinin (HA) gene from A/Vietnam/1194/2004 influenza virus (Ad4-H5-Vtn). Our hypothesis is that a mucosally-delivered replicating Ad4-H5-Vtn recombinant vector will be safe and induce protective immunity against H5N1 influenza virus infection and disease pathogenesis.The Ad4-H5-Vtn vaccine was designed with a partial deletion of the E3 region of Ad4 to accommodate the influenza HA gene. Replication and growth kinetics of the vaccine virus in multiple human cell lines indicated that the vaccine virus is attenuated relative to the wild type virus. Expression of the HA transgene in infected cells was documented by flow cytometry, western blot analysis and induction of HA-specific antibody and cellular immune responses in mice. Of particular note, mice immunized intranasally with the Ad4-H5-Vtn vaccine were protected against lethal H5N1 reassortant viral challenge even in the presence of pre-existing immunity to the Ad4 wild type virus.Several non-clinical attributes of this vaccine including safety, induction of HA-specific humoral and cellular immunity, and efficacy were demonstrated using an animal model to support Phase 1 clinical trial evaluation of this new vaccine

Setting the seeds for a green growth -A Study of biofuel development in Indonesia’s transport sector

Climate change poses as one of the major environmental issues on a global scale today, with one of the largest contributor to the climate change being the greenhouse gas (GHG) emissions. GHG emissions in turn is known to a large extent originate in anthropogenic energy use, especially fossil fuel usage. This leads to almost a quarter of the global emissions today being emitted from the transport sector as the sector is primary fueled by fossil fuels. Biofuels have been promoted as a strong candidate for fossil fuel substitution as it has similar properties while being renewable. However, even as biofuels have been increasing annually since 2008 globally, there are still concerns associated with the usages that have hindered its replacement of fossil fuels. Indonesia, one of the most populated countries in Southeast Asia, is projected to be one of the world’s leading economies in 2050. In 2030, Indonesia in projected to have doubled its energy consumption since 2014, indicating that the decisions and actions taken today in Indonesia will have a significant impact on the future fuel consumption. This, in combination with being the largest producer of palm oil in the world, gives it a great potential to be in a leading position in the future production of biofuels, especially biodiesel from palm oil. Additionally, Indonesia has set mandatory targets for biofuel share in the transport sector which indicates that the country also seeks to promote the usages of biofuels. This thesis intends to investigate Indonesia’s potential for biofuel development in the transport sector and which factors that could hinder it. More specifically, this study answers the two questions: (i) Which factors are hampering the development of biofuels in the Indonesian transportation sector from a multi-level perspective? and (ii) What is the projection of the Indonesian transport sector by 2030 in terms of fuel consumption and global warming potential (GWP) and what role could biofuels play in reducing those? The data used were collected by interviews with stakeholders in Indonesia and a literature study, which afterwards was processed with the tools Multi-Level Perspective (MLP) and Long-range Energy Alternatives Planning (LEAP). MLP is a concept aiming to describe how system transitions happens with the help of an examination of the linkages in between technology and society, whereas LEAP is an energy modeling tool used in order to create energy projections in different policy scenario options. In this research, MLP were used to capture and analyze the factors that influence the future implementation of biofuels in Indonesia from a socio-technical perspective, putting forward key barriers for biofuel implementation in Indonesia. LEAP was used to construct a model of the Indonesian transport sector to project the future fuel consumption and GWP emissions. This was used to examine biofuels importance as a fuel through scenarios where different Indonesian policy actions were appraised. Together, these two methods will provide an enlightening and concluding remark on the future possibilities for biofuels in Indonesia's transport sector in this thesis. The major finding of the first research question were that biofuels in Indonesia were being introduced in Indonesia as the government saw economic benefits and a solution to the increased energy demand in Indonesia and an increased energy security in going towards a domestic produced energy source. The need for biofuels were also increased due to an increased global sustainability awareness, which also reached Indonesia. Today, biofuels in Indonesia is in a socio-technical transition pathway to go from a niche innovation to a technology in the regime level, but in order for a breakthrough, a number of barriers needs to be addressed. The most mentioned barriers were the institutional and regulatory barriers, which mainly lays in a lack of cooperation amongst the regulatory institutions and a low rate of success of biofuel laws and regulations. Other barriers were the market barriers, closely connected in a subsidization of fossil fuels in Indonesia and a need for further subsidization of biofuels for the market for biofuels to exist in Indonesia. Furthermore, a technical barrier with the vehicle engine were seen as the engine needs improvements in order for a higher blending of biofuels in the fuel. There was also a concern of the perceived sustainability of biofuels in general (e.g. water scarcity and pollution) which was identified as a hindrance. It was also clear a social change is needed in order to push the biofuel breakthrough and enable it to reach its full potential. Today, the interviewees mainly see a development for the biofuel biodiesel and not for the biofuel bioethanol in Indonesia, which they mainly concluded as there is currently none or very little production and demand for bioethanol in Indonesia, and as well a lack of governmental support for bioethanol development. For the second research question four scenarios were used; Business as Usual (a continuation of current trends), Improved Standards (an investigation of higher emissions standards and an increasing fuel efficiency), Biofuel Mandate (a mandatory biofuel share in fuels) and the Low Carbon scenario (a combination of the previous two as well as introduction of electric vehicles, changed car preferences and higher biofuel blending targets). Cars represents a tenth of the vehicle fleet and the share of diesel car amongst the cars are 5% while gasoline stands for 95%, the rest of the vehicles are gasoline driven motorcycles. The main findings were: the total vehicle fleet will have doubled by 2030. This rapid increase could cause stress on the domestic fuel supply, as the yearly fuel consumption is expected to grow from 770 million GJ in 2014 to 1850 million GJ in 2030, an increase by 140%. In the projection the fossil fuels are blended with biofuels, diesel is mixed with biodiesel while gasoline is blended with bioethanol. The annual diesel fuel consumption is projected to increase from 350 million liters to 1100 million in the Business as Usual scenario where the current trend was expected to continue. The implementation of biodiesel and bioethanol in the Biofuel Mandate scenario. Additionally, the Biofuel Mandate scenario resulted in a 12.6% reduction of GHG emissions during the projected period. The cumulative GHG emissions in the projection is estimated to be 1630 million tonnes of carbon dioxide equivalents in 2030 for the Business as Usual scenario. In the transport sector, gasoline is projected to increase from 23 billion liters to 54 billion liters over the period. In the Improved Standards scenario, where the emission standards are increased from Euro2 to Euro6 in 2030 for cars and Euro4 for motorcycles in 2025 as well as an annual fuel economy improvement of 2%, the total fuel consumption is reduced with 20% and GWP by 18%. It was found that the annual GWP could be reduced by almost 31% below the 2014 level, at 47 million metric tonnes of CO2Eq, if the actions of the Low Carbon scenario were achieved. The barriers associated with the Improved Standards scenario were technical, institutional and regulatory while the Biofuel Mandate scenario also had financial, market and distribution barriers. The Low Carbon scenario had barriers from all of the identified barrier categories. The institutional and regulatory barrier was the most prevailing barrier for all of the scenarios. The recommended actions based on the content of this thesis is to firstly promote collaboration between governmental institutes, stakeholders and authorities and include all stakeholders in the decision progress, this way, frameworks and regulations will have a chance to improve and increase the knowledge about biofuels in all levels. It is also important to implement a stronger biofuel R&D culture, promote a more sustainable biofuel development and increase the public awareness of biofuels. The implementation of biofuels will have a reducing effect on the total GWP and fossil fuel consumption. Gasoline is projected to remain the predominant fuel in the transport sector. Therefore will actions targeting the reduction or substitution of gasoline be more impactful than those towards diesel. However due to the availability and current production capacity of biodiesel there is still potential for higher share of biodiesel in fuels