Finite Horizon Q-learning: Stability, Convergence, Simulations and an application on Smart Grids

Q-learning is a popular reinforcement learning algorithm. This algorithm has however been studied and analysed mainly in the infinite horizon setting. There are several important applications which can be modeled in the framework of finite horizon Markov decision processes. We develop a version of Q-learning algorithm for finite horizon Markov decision processes (MDP) and provide a full proof of its stability and convergence. Our analysis of stability and convergence of finite horizon Q-learning is based entirely on the ordinary differential equations (O.D.E) method. We also demonstrate the performance of our algorithm on a setting of random MDP as well as on an application on smart grids

Tooth-Support over Dentures: An approach to Preventive Prosthodontics

Over denture is a favoured treatment modality for elderly patients with few remaining teeth. Roots maintained under the denture base preserve the alveolar ridge, provide sensory feedback and improve the stability of the dentures. The concept of conventional tooth-retained over dentures is a simple and cost effective treatment than the implant over dentures. When few firm teeth are present in an otherwise compromised dentition, they can be retained and used as abutments for over denture fabrication. The concept of over dentures may not be the elixir, but it is a positive means for delaying the process of complete endentulism and helps in the preservation of bone. To top it all, it gives the patient the satisfaction of having prosthesis with his natural teeth still present. The present case reports discuss the rehabilitation of an edentulous patient with a tooth supported over denture with metal copings

Actor-Critic or Critic-Actor? A Tale of Two Time Scales

We revisit the standard formulation of tabular actor-critic algorithm as a two time-scale stochastic approximation with value function computed on a faster time-scale and policy computed on a slower time-scale. This emulates policy iteration. We begin by observing that reversal of the time scales will in fact emulate value iteration and is a legitimate algorithm. We compare the two empirically with and without function approximation (with both linear and nonlinear function approximators) and observe that our proposed critic-actor algorithm performs better empirically though with a marginal increase in the computational cost

EXPLORATION OF E-LEARNING VS TRADITIONAL LEARNING IN INDIA

Purpose of Study: The purpose of the study is to carry out the comparison of traditional learning and e-learning with reference to university education. Methodology: A comparative analysis method that has been selected to fulfill the purpose of the study. The research data has been collected by various sources internet, previous research studies published in the Journal, universities which offer different technical and nontechnical programs .and further analyzed by in-depth understanding approach. Result: The outcome of this study will show the effect of e-learning in the current era. As we have compared e-learning with traditional learning and the result shows us that e-learning has filled the gap between the universities and the real-life industries' demands. Main finding: The result of this study showed that e-learning is the need of the present era in order to fulfill the gap between the universities and the real-life industries' demands. Additionally, e-learning based courses will have more impact and provide more skill and exposure to students as compared to traditional learning mode. Implications/Applications: This article can help the different universities and students to introduce more e-learning courses in their curriculum and they can fill the real-time industries' demands. Novelty/originality of the study: Our research can ensure that e-learning has a huge impact on our students and can help to increase the skills and exposure of them according to the current demands

INTERNET OF THINGS IN SMART AGRICULTURE: APPLICATIONS AND OPEN CHALLENGES

Purpose of Study: The IoT is an emerging field nowadays and that can be used anywhere in automation, agriculture, controlling as well as monitoring of any object, which exists in the real world. We have to make use of IoT in Agriculture to increase productivity. Agro-industry processes could be more efficient by using IoT. It gives automation to agro-industry by reducing human intervention. In the current scenario, the sometime farmer doesn’t know the current status of the soil moisture and other things related to their land and don’t produce productive results towards crops. The purpose of this research study is to explore the usage of IoT devices and application areas that are being used in agriculture.  Methodology: The methodology behind this study is to identify trends and review the open challenges, application areas and architectures for IoT in agro-industry. This survey is based on a systematic literature review where related research is grouped into four domains such as monitoring, control, prediction, and logistics.  Main Findings: This research study presents a detailed work of the eminent researchers and designs of computer architecture that can be applied in agriculture for smart farming. This research study also highlights various unfolded challenges of IoT in agriculture. Implications: This study can be beneficial for farmers, researchers, and professionals working in agricultural institutions for smart farming. Novelty/Originality of the study: Various eminent researchers have been making efforts for smart farming by using IoT concepts in agriculture. But, a bouquet of unfolded challenges is still in a queue for their effective solution. This study makes some efforts to discuss past research and open challenges in IoT based agriculture

POLLUTION MANAGEMENT THROUGH INTERNET OF THINGS: A SUBSTANTIAL SOLUTION FOR SOCIETY

Purpose of Study: In this age where population and industrialization have extended exponentially, contamination is a noteworthy emergency looked by humanity and essential measures ought to be taken to control it quickly. High levels of pollutants in the atmosphere, which are responsible for causing pollution that causes human heart problems, breathing and other all health problems to our society. This paper focused to provide a substantial solution to pollution in our society as a result of which major social, personal health and environmental pollution problems can be monitored and by taking effective steps they can be reduced. Methodology: The proposed framework is an IOT unit for handling these issues, including powerful use of the different sensor, to capture the estimations of various sorts of toxins in the earth and show insights on a site which can be gotten to by all clients for nothing. The research data is collected from the different sensors installed at different locations in our surroundings. After collecting data from sensors, a comparative study of data is carried out with the standards provided by the Central Pollution Control Board (CPCB). Results: The proposed system helps to reduce the pollution in our environment and as a result of which pollution can be reduced by taking major steps towards it. Application/Implications: The framework we are proposing is a propelled answer for observing the ecological contamination at a specific spot and make data available for each person. Novelty/originality of the study: Our proposed system ensures that pollution which ultimately causes health, environmental and other social problems can be tracked from time to time and each and every person in the society can track and check pollution level in their surroundings. &nbsp

Process optimization for recycling of bread waste into bioethanol and biomethane: a circular economy approach

Bread is the second most wasted food in the UK with annual wastage of 292,000 tons. In the present work, bread waste (BW) was utilized for fermentative production of ethanol by Saccharomyces cerevisiae KL17. Acidic and enzymatic saccharification of BW was carried out resulting in the highest glucose release of 75 and 97.9 g/L which is 73.5 and 95.9% of theoretical yield, respectively. The obtained sugars were fermented into ethanol initially in shake flask followed by scale up in bioreactor in batch and fed-batch mode. In the fed-batch mode of cultivation, the maximum ethanol titers of 111.3, 106.9, and 114.9 g/L with conversion yield and productivity of 0.48, 0.47, and 0.49 g/g, and 3.1, 3.0, and 3.2 g/L.h was achieved from pure glucose, glucose-rich acidic and enzymatic hydrolysates, respectively. Further to improve the process economics, the solid residues after acidic (ABW) and enzymatic (EBW) hydrolysis of BW along with respective fermentation residues (FR) obtained after the ethanol production were pooled and subjected to anaerobic digestion. The solid residue from ABW + FR, and EBW + FR yielded a biochemical methanation potential (BMP) of 345 and 379 mL CH4/g VS, respectively. Life cycle assessment of the process showed that the total emissions for ethanol production from BW were comparable to the emissions from more established feedstocks such as sugarcane and maize grain and much lower when compared to wheat and sweet potato. The current work demonstrates BW as promising feedstock for sustainable biofuel production with the aid of circular biorefining strategy. To the authors knowledge, this is the first time, such a sequential system has been investigated with BW for ethanol and biomethane production. Further work will be aimed at ethanol production at pilot scale and BMP will be accessed in a commercial anaerobic digester

Bread waste – a potential feedstock for sustainable circular biorefineries

The management of staggering volume of food waste generated (∼1.3 billion tons) is a serious challenge. The readily available untapped food waste can be promising feedstock for setting up biorefineries and one good example is bread waste (BW). The current review emphasis on capability of BW as feedstock for sustainable production of platform and commercially important chemicals. It describes the availability of BW (>100 million tons) to serve as a feedstock for sustainable biorefineries followed by examples of platform chemicals which have been produced using BW including ethanol, lactic acid, succinic acid and 2,3-butanediol through biological route. The BW-based production of these metabolites is compared against 1G and 2G (lignocellulosic biomass) feedstocks. The review also discusses logistic and supply chain challenges associated with use of BW as feedstock. Towards the end, it is concluded with a discussion on life cycle analysis of BW-based production and comparison with other feedstocks

Physics Potential of the ICAL detector at the India-based Neutrino Observatory (INO)

The upcoming 50 kt magnetized iron calorimeter (ICAL) detector at the India-based Neutrino Observatory (INO) is designed to study the atmospheric neutrinos and antineutrinos separately over a wide range of energies and path lengths. The primary focus of this experiment is to explore the Earth matter effects by observing the energy and zenith angle dependence of the atmospheric neutrinos in the multi-GeV range. This study will be crucial to address some of the outstanding issues in neutrino oscillation physics, including the fundamental issue of neutrino mass hierarchy. In this document, we present the physics potential of the detector as obtained from realistic detector simulations. We describe the simulation framework, the neutrino interactions in the detector, and the expected response of the detector to particles traversing it. The ICAL detector can determine the energy and direction of the muons to a high precision, and in addition, its sensitivity to multi-GeV hadrons increases its physics reach substantially. Its charge identification capability, and hence its ability to distinguish neutrinos from antineutrinos, makes it an efficient detector for determining the neutrino mass hierarchy. In this report, we outline the analyses carried out for the determination of neutrino mass hierarchy and precision measurements of atmospheric neutrino mixing parameters at ICAL, and give the expected physics reach of the detector with 10 years of runtime. We also explore the potential of ICAL for probing new physics scenarios like CPT violation and the presence of magnetic monopoles.Comment: 139 pages, Physics White Paper of the ICAL (INO) Collaboration, Contents identical with the version published in Pramana - J. Physic

Assessing the feasibility of integrating ecosystem-based with engineered water resource governance and management for water security in semi-arid landscapes: A case study in the Banas catchment, Rajasthan, India

Much of the developing world and areas of the developed world suffer water vulnerability. Engineering solutions enable technically efficient extraction and diversion of water towards areas of demand but, without rebalancing resource regeneration, can generate multiple adverse ecological and human consequences. The Banas River, Rajasthan (India), has been extensively developed for water diversion, particularly from the Bisalpur Dam from which water is appropriated by powerful urban constituencies dispossessing local people. Coincidentally, abandonment of traditional management, including groundwater recharge practices, is leading to increasingly receding and contaminated groundwater. This creates linked vulnerabilities for rural communities, irrigation schemes, urban users, dependent ecosystems and the multiple ecosystem services that they provide, compounded by climate change and population growth. This paper addresses vulnerabilities created by fragmented policy measures between rural development, urban and irrigation water supply and downstream consequences for people and wildlife. Perpetuating narrowly technocentric approaches to resource exploitation is likely only to compound emerging problems. Alternatively, restoration or innovation of groundwater recharge practices, particularly in the upper catchment, can represent a proven, ecosystem-based approach to resource regeneration with linked beneficial socio-ecological benefits. Hybridising an ecosystem-based approach with engineered methods can simultaneously increase the security of rural livelihoods, piped urban and irrigation supplies, and the vitality of river ecosystems and their services to beneficiaries. A renewed policy focus on local-scale water recharge practices balancing water extraction technologies is consistent with emerging Rajasthani policies, particularly Jal Swavlamban Abhiyan (‘water self-reliance mission’). Policy reform emphasising recharge can contribute to water security and yield socio-economic outcomes through a systemic understanding of how the water system functions, and by connecting goals and budgets across multiple, currently fragmented policy areas. The underpinning principles of this necessary paradigm shift are proven and have wider geographic relevance, though context-specific research is required to underpin robust policy and practical implementation