Training of Reinforcement Learning Agents for Autonomous Driving in Simulated Environments

Et system som er i stand til autonom kjøring, må ha flere funksjoner. For det første må kjøretøyet kunne sanse omgivelsene sine, og for det andre må det bruke informasjonen om omgivelsene for å manøvrere til ønsket destinasjon. Denne oppgaven er fokusert på bruken av Reinforcement Learning (RL) for å manøvrere kjøretøyet i dets miljø - den autonome kjøresimulatoren med åpen kildekode CARLA. Tidligere RL-systemer i CARLA-simulatoren bruker enkle syns-enkodere for å registrere omgivelsene, noe som muligens begrenser ytelsen. Denne oppgaven undersøker bruken av mer komplekse forhåndstrente syns-enkodere i sammenheng med Reinforcement Learning (RL). Flere agenter ble opplært i miljøer i CARLA-simulatoren, med varierende kompleksitet og utfordringer. En ren RL agent ble trent for å gi ett sammenligninigsgrunnlag, ved bruk av Proximal Policy Optimization (PPO)-algoritmen med Transfuser enkoderen og ga suboptimale resultater, ettersom kjøretøyet konsekvent svingte av veien og kolliderte. Inntroduksjon av ekspertdemonstrasjoner gjennom General Reinforced Imitation for Autonomous Driving (GRIAD)-tilnærmingen forbedret ikke ytelsen, og resulterte i at kjøretøyet sto stille på veien. For å overvinne disse begrensningene ble det laget et forenklet miljø med redusert trafikkkompleksitet, noe som resulterte i store fremskritt i påfølgende treninger. I det forenklede miljøet ble både TransFuser-enkoderen og en enkel Convolutional Neural Network (CNN) brukt. CNN-enkoderen demonstrerte bedre ytelse sammenlignet med TransFuser. Likevel møtte begge tilnærmingene utfordringer med å unngå kollisjoner med andre kjøretøy. Det er viktig å merke seg at opplæringsvarigheten bare var en million steg, noe som nødvendiggjorde ytterligere undersøkelser for å trekke definitive konklusjoner. Fremtidig forskning bør fokusere på å vurdere virkningen av mer komplekse synskodere på opplæringen av RL agenter. Å forlenge treningstiden kan føre til en mer grundig forståelse av de potensielle fordelene og ulempene forbundet med sofistikerte syns-enkodere i RL-scenarier. Ved å adressere disse områdene kan det gjøres fremskritt i utviklingen av effektive opplæringsmetoder for RL-agenter som opererer i komplekse miljøer i den virkelige verden.A system capable of autonomous driving needs to have several capabilities. Firstly, the vehicle needs to be able to sense its environment, and secondly, it needs to use the information about its surroundings to maneuver to its desired destination. This thesis is focused on the use of Reinforcement Learning (RL) to maneuver the vehicle in its environment - the open-source autonomous driving simulator CARLA. Previous RL systems in the CARLA simulator use simple vision encoders to sense their surroundings, possibly limiting their performance. This thesis investigates the utilization of more complex pre-trained vision encoders in the context of Reinforcement Learning (RL) for autonomous driving. Multiple agents were trained in environments within the CARLA simulator, varying in complexity. The baseline training using the Proximal Policy Optimization (PPO) algorithm with a complex transformer-based vision encoder produced suboptimal results, as the vehicle consistently veered off the road and encountered crashes. Incorporating expert demonstrations through the General Reinforced Imitation for Autonomous Driving (GRIAD) approach did not enhance performance, leaving the vehicle stationary on the road. To overcome these limitations, a simplified environment with reduced traffic complexity was created, resulting in notable advancements in subsequent training runs. In the simplified environment, both the complex encoder and a simple Convolutional Neural Network (CNN) was employed. The CNN encoder demonstrated superior performance compared to the TransFuser encoder. Nevertheless, both approaches encountered challenges in avoiding collisions with other vehicles. It is important to note that the training duration only was 1 million steps, necessitating further investigation to draw definitive conclusions. Future research should focus on assessing the impact of more complex vision encoders on the training of RL agents. Extending the training time and gradually increasing traffic complexity can lead to a more thorough understanding of the potential benefits and drawbacks associated with sophisticated vision encoders in RL scenarios. By addressing these areas, advancements can be made in the development of effective training methodologies for RL agents operating in complex real-world environments

Интеллект-карта как средство оценивания качества знаний обучающихся: возможности и ограничения структурно-информационного подхода

Обсуждаются возможности и ограничения использования интеллект-карт как средства оценивания качества знаний обучающихся в рамках структурно-информационного подхода. Интеллект-карта рассматривается, с одной стороны, как логико-смысловая вербально-образная модель учебного материала, отображающая его содержание и структуру, с другой стороны – как модель индивидуальных знаний обучающихся. Для расчета структурных и информационных характеристик интеллект-карты предлагается использовать структурные формулы древовидных графов. Приведен алгоритм расчета структурных и информационных характеристик эталонной интеллект-карты и индивидуальных интеллект-карт, относительного показателя упорядоченности знаний обучающегося

Crowdfunding - From an Investor Perspective

Det nye markedet for crowdfunding, som oppstod fra FinTech industrien, har revolusjonert tradisjonell bankvirksomhet når det gjelder finansiering til oppstartsselskaper og små- og mellomstore bedrifter. Til forskjell fra tradisjonell ekstern finansiering, involverer ikke crowdfunding tredjeparter slik som banker, men i stedet blir privatpersoner eller selskaper som trenger finansiering direkte koblet til investorer som er villige til å investere i dem. Til tross for markedets enorme vekst i løpet av det siste tiåret, er forskning på investorer i dette markedet sjelden. Formålet med denne masteroppgaven er å skape en forståelse for hvem investorene er og hvordan de ser på crowdfunding som en investeringsmulighet. På bakgrunn av dette formålet, har vi utviklet følgende forskningsspørsmål: «Hvordan kan vi kategorisere investorene i det norske crowdfunding markedet i dag, og hvordan ser de på crowdfunding som en investeringsmulighet?» For å besvare problemstillingen har vi benyttet oss av et forskningsdesign som består av en triangulering av både kvalitativ og kvantitativ metode. Metoden innebærer innhenting av data gjennom tre intervjuer med ulike plattformer, fem intervju med investorer og en spørreundersøkelse besvart av 25 investorer. Spørreundersøkelsen hadde som mål å enten komplementere, divergere eller konvergere våre empiriske funn fra intervjuer med investorer. Resultatene viser at investorer i det norske crowdfunding markedet kan deles inn i fire ulike kategorier; «Lotto Investors», «Traders», «Business Angels» og «Analytical Investors». Disse kategoriene ble etablert på bakgrunn av investorenes aktivitetsnivå, kompetanse, preferanse for prosjekter og hva deres investeringsbeslutninger var basert på. Vi gir også innsikt i hvordan investorene oppfatter det norske crowdfunding markedet. I denne sammenheng, utviklet vi en ‘oppfatningsmodell’ bestående av ulike elementer som påvirker investorenes oppfatning av crowdfunding som en investeringsmulighet. Disse elementene var; ulike egenskaper ved investorene, kvalitet ved plattformene og markedet/samfunnet. I tillegg ble formålet med investorenes investering, deres oppfatning av risiko og deres motivasjon drøftet i analysen. Dermed har vi dekket flere hull i litteraturen, både med tanke på investorenes motivasjon og deres oppfatninger. Samlet sett bidrar vår masteroppgave til verdifull kunnskap om investorene i crowdfunding som kan være gunstig for både ledere av crowdfundingselskaper og andre investorer.The new market of crowdfunding, which have emerged from the FinTech industry, has revolutionized traditional banking when it comes to funding for start-up companies and small and medium-sized enterprises (SMEs). Unlike traditional financing, crowdfunding does not involve third parties, such as banks, but instead directly links private persons or firms who needs funding with investors willing to fund. Despite the market’s enormous growth over the last decade, research on the investors in this market is rare. The purpose of this thesis is to create an understanding of the investors and how they perceive crowdfunding as an investment opportunity. For this purpose, we have developed the following research question; “How can we categorize the investors that participate in the Norwegian crowdfunding market today, and how do they perceive crowdfunding as an investment opportunity?” To answer this research question, we used a triangulation of a mixed method research design, with both a qualitative and quantitative approach. The method involves three platform interviews, five investor interviews and a survey answered by 25 investors. The survey aimed to either complement, diverge or converge the empirical findings from the interviews. The results of this thesis show that it was possible to categorize the investors from the Norwegian crowdfunding market into four different groups; “Lotto Investors”, “Traders”, “Business Angels” and “Analytic Investors”. These categories were made on the basis of the investors’ investment activity level, competence, preference for projects and what their investment decisions were based on. We also provide insights about how the investors perceive the Norwegian crowdfunding market. In this context, we developed a perception model consisting of various elements that affects the investors’ perception of crowdfunding as an investment opportunity. These elements are different investor characteristics, platform quality and the market/community. In addition, the purpose of the investors’ investment, their perception of risk and their motivation was added to the analysis. Thus, we have addressed several gaps in the literature, both concerning the investors’ motivation and their perception. Overall, this thesis contributes with valuable knowledge about the investors that is beneficial both to managers of crowdfunding platforms and to investors of crowdfunding

Assessment of the quality and content of website health information about herbal remedies for menopausal symptoms

Objective: To assess quality, readability and coverage of website information about herbal remedies for menopausal symptoms. Study design: A purposive sample of commercial and non-commercial websites was assessed for quality (DISCERN), readability (SMOG) and information coverage. Main outcome measures: Non-parametric and parametric tests were used to explain variability of these factors across types of websites and to assess associations between website quality and information coverage. Results: 39 sites were assessed. Median quality and information coverage scores were 44/80 and 11/30 respectively. The median readability score was 18.7, similar to UK broadsheets. Commercial websites scored significantly lower on quality (p=0.014), but there were no statistical differences for information coverage or readability. There was a significant positive correlation between information quality and coverage scores irrespective of website provider (r=0.69, p<0.001, n=39). Conclusion: Overall website quality and information coverage is poor and the required reading level high

Patients' online access to their electronic health records and linked online services: a systematic review in primary care

Background Online access to medical records by patients can potentially enhance provision of patient-centred care and improve satisfaction. However, online access and services may also prove to be an additional burden for the healthcare provider. Aim To assess the impact of providing patients with access to their general practice electronic health records (EHR) and other EHR-linked online services on the provision, quality, and safety of health care. Design and setting A systematic review was conducted that focused on all studies about online record access and transactional services in primary care. Method Data sources included MEDLINE, Embase, CINAHL, Cochrane Library, EPOC, DARE, King’s Fund, Nuffield Health, PsycINFO, OpenGrey (1999–2012). The literature was independently screened against detailed inclusion and exclusion criteria; independent dual data extraction was conducted, the risk of bias (RoB) assessed, and a narrative synthesis of the evidence conducted. Results A total of 176 studies were identified, 17 of which were randomised controlled trials, cohort, or cluster studies. Patients reported improved satisfaction with online access and services compared with standard provision, improved self-care, and better communication and engagement with clinicians. Safety improvements were patient-led through identifying medication errors and facilitating more use of preventive services. Provision of online record access and services resulted in a moderate increase of e-mail, no change on telephone contact, but there were variable effects on face-to-face contact. However, other tasks were necessary to sustain these services, which impacted on clinician time. There were no reports of harm or breaches in privacy. Conclusion While the RoB scores suggest many of the studies were of low quality, patients using online services reported increased convenience and satisfaction. These services positively impacted on patient safety, although there were variations of record access and use by specific ethnic and socioeconomic groups. Professional concerns about privacy were unrealised and those about workload were only partly so

Institusjonelt entreprenørskap og regional bærekraftig næringsutvikling : EIMUR i Nordøst-Island

Island er et land som ønsker å være i førersetet når det kommer til ”grønn vekst”. Over hele landet finnes det ambisiøse aktører som ser etter ulike muligheter å gjøre lokalsamfunnene de lever i mer bærekraftig. Målet med denne masteroppgaven er å forstå hvordan slike aktører blir motivert, målrettet og klar for å promotere regional bærekraftig næringsutvikling og kartlegge hvilke nye spilleregler/institusjoner de innfører for å nå sine mål. Hovedproblemstillingen i denne masteroppgaven er: ”Hvordan kan institusjonelt entreprenørskap bidra til mer innovasjon og økt bærekraftighet i Nordøst-Island?”. Videre spør jeg 1) hvilke kjennetegn på institusjonelt entreprenørskap ser jeg hos informantene i studien? 2) hvordan påvirker erfaringer, forventninger og visjoner aktørenes handlinger og strategier i organisasjonen og hvordan bruker de makt og innflytelse for å påvirke politikken i organisasjonen i tidlig fase? 3) hvordan ser aktørene på betydningen av uformelle institusjoner i utviklingen av et regional innovasjonssystem i embryonisk fase? 4) hvordan vil aktørene fasilitere nærhet og kunnskap? og 5) hvilke utviklingsbaner og stier mener aktørene er viktig i framtiden for at EIMUR skal nå sine mål, og hvordan ønsker aktørene å påvirke valget av disse. Gjennom dybdeintervju og ulike dokumenter har jeg samlet inn empiri som kan si noe om aktørenes subjektive meninger og regionens objektive kvaliteter. Denne studien viser at informantene langt på vei kan kalles institusjonelle entreprenører med lederkapasitet. Disse aktørene har vært sterkt delaktig i opprettelsen av et innovativt partnerskap med klare mål om å gjøre Nordøst-Island til en mer bærekraftig region. Erfaringer, forventninger, visjoner og maktposisjon har påvirket aktørenes evne til å innføre nye spilleregler/institusjoner for regional bærekraftig næringsutvikling. EIMUR ønsker å utvikle et regionalt innovasjonssystem rundt bærekraftighet i regionen og aktørene i studien ønsker å promotere framtidige utviklingsbaner i form av stietablering og stifornyelse som innebærer ny næringsvirksomhet og nye tjenester/produkter. Denne studien setter fokus på sammenhengen mellom institusjonelt entreprenørskap og regional bærekraftig næringsutvikling, og jeg mener slik forskning kan være med å avdekke noen av kreftene som ligger bak “grønn vekst”

Teknologi til fallforbygging – prosjektrapport Falltek.

Fallforebygging er høyt prioritert i kommunene og Falltek (www.falltek.no) hadde som mål å utvikle og bruke motiverende teknologi til fallforebyggende trening. Gjennom å kombinere følgeforskning og utvikling av teknologi kan vi • Bedre fange opp brukerbehov • Støtte innovative prosesser i kommunene og • Tilpasse teknologiutviklingen til brukerbehov og kommunens organiserin

Categorizing Vaccine Confidence With a Transformer-Based Machine Learning Model: Analysis of Nuances of Vaccine Sentiment in Twitter Discourse.

BACKGROUND: Social media has become an established platform for individuals to discuss and debate various subjects, including vaccination. With growing conversations on the web and less than desired maternal vaccination uptake rates, these conversations could provide useful insights to inform future interventions. However, owing to the volume of web-based posts, manual annotation and analysis are difficult and time consuming. Automated processes for this type of analysis, such as natural language processing, have faced challenges in extracting complex stances such as attitudes toward vaccination from large amounts of text. OBJECTIVE: The aim of this study is to build upon recent advances in transposer-based machine learning methods and test whether transformer-based machine learning could be used as a tool to assess the stance expressed in social media posts toward vaccination during pregnancy. METHODS: A total of 16,604 tweets posted between November 1, 2018, and April 30, 2019, were selected using keyword searches related to maternal vaccination. After excluding irrelevant tweets, the remaining tweets were coded by 3 individual researchers into the categories Promotional, Discouraging, Ambiguous, and Neutral or No Stance. After creating a final data set of 2722 unique tweets, multiple machine learning techniques were trained on a part of this data set and then tested and compared with the human annotators. RESULTS: We found the accuracy of the machine learning techniques to be 81.8% (F score=0.78) compared with the agreed score among the 3 annotators. For comparison, the accuracies of the individual annotators compared with the final score were 83.3%, 77.9%, and 77.5%. CONCLUSIONS: This study demonstrates that we are able to achieve close to the same accuracy in categorizing tweets using our machine learning models as could be expected from a single human coder. The potential to use this automated process, which is reliable and accurate, could free valuable time and resources for conducting this analysis, in addition to informing potentially effective and necessary interventions

Mental Disorder Recovery Correlated with Centralities and Interactions on an Online Social Network

Recent research has established both a theoretical basis and strong empirical evidence that effective social behavior plays a beneficial role in the maintenance of physical and psychological well-being of people. To test whether social behavior and well-being are also associated in online communities, we studied the correlations between the recovery of patients with mental disorders and their behaviors in online social media. As the source of the data related to the social behavior and progress of mental recovery, we used PatientsLikeMe (PLM), the world's first open-participation research platform for the development of patient-centered health outcome measures. We first constructed an online social network structure based on patient-to-patient ties among 200 patients obtained from PLM. We then characterized patients' online social activities by measuring the numbers of "posts and views" and "helpful marks" each patient obtained. The patients' recovery data were obtained from their self-reported status information that was also available on PLM. We found that some node properties (in-degree, eigenvector centrality and PageRank) and the two online social activity measures were significantly correlated with patients' recovery. Furthermore, we re-collected the patients' recovery data two months after the first data collection. We found significant correlations between the patients' social behaviors and the second recovery data, which were collected two months apart. Our results indicated that social interactions in online communities such as PLM were significantly associated with the current and future recoveries of patients with mental disorders.Comment: 20 pages, 5 figures, 5 tables; accepted for publication in Peer