Combining predictions from linear models when training and test inputs differ

Methods for combining predictions from different models in a supervised learning setting must somehow estimate/predict the quality of a model's predictions at unknown future inputs. Many of these methods (often implicitly) make the assumption that the test inputs are identical to the training inputs, which is seldom reasonable. By failing to take into account that prediction will generally be harder for test inputs that did not occur in the training set, this leads to the selection of too complex models. Based on a novel, unbiased expression for KL divergence, we propose XAIC and its special case FAIC as versions of AIC intended for prediction that use different degrees of knowledge of the test inputs. Both methods substantially differ from and may outperform all the known versions of AIC even when the training and test inputs are iid, and are especially useful for deterministic inputs and under covariate shift. Our experiments on linear models suggest that if the test and training inputs differ substantially, then XAIC and FAIC predictively outperform AIC, BIC and several other methods including Bayesian model averaging.Comment: 12 pages, 2 figures. To appear in Proceedings of the 30th Conference on Uncertainty in Artificial Intelligence (UAI2014). This version includes the supplementary material (regularity assumptions, proofs

Ritual repetition : Creating safe havens for sufferers or boring experiences?

Peer reviewedPublisher PD

Inconsistency of Bayesian Inference for Misspecified Linear Models, and a Proposal for Repairing It

We empirically show that Bayesian inference can be inconsistent under misspecification in simple linear regression problems, both in a model averaging/selection and in a Bayesian ridge regression setting. We use the standard linear model, which assumes homoskedasticity, whereas the data are heteroskedastic, and observe that the posterior puts its mass on ever more high-dimensional models as the sample size increases. To remedy the problem, we equip the likelihood in Bayes' theorem with an exponent called the learning rate, and we propose the Safe Bayesian method to learn the learning rate from the data. SafeBayes tends to select small learning rates as soon the standard posterior is not `cumulatively concentrated', and its results on our data are quite encouraging.Comment: 70 pages, 20 figure

The oxidative dehydrogenation of methanol to formaldehyde over silver catalysts in relation to the oxygen-silver interaction

The properties of silver in the oxidative dehydrogenation of methanol were studied in a flow reactor under near industrial conditions. The influences of temperature, concentration of both reactants, gas velocity, space velocity, the form of the silver catalyst and surface composition of the catalyst were studied. A model for the reaction is proposed which is based on the experimental observations and on the nature of the interaction of silver with oxygen. It issuggested that different oxygen species on the silver surface play different roles in the reactions to CO, CO2 and H2CO. Gas phase reactions only contribute to the conversion to CO

The silver-oxygen interaction in relation to oxidative dehydrogenation of methanol

The interaction of unsupported silver with oxygen at atmospheric pressure and at temperatures between 100 and 600°C has been studied using temperature programmed reduction and desorption experiments with temperatures ranging up to 900°C. In addition, the interaction of an oxygen-loaded silver surface with methanol has been studied using both these techniques and temperature programmed reaction. It appears that the silver-oxygen chemistry is influenced strongly by hydrogen dissolved in the silver during the pretreatment of the catalyst, the hydrogen giving rise to a new type of sub-surface species, possibly sub-surface OH groups, and also to an increase of the amount of sub-surface oxygen formed. Sub-surface oxygen can be converted into a strongly bound species that is not present to a measurable extent after normal oxidation. Defects, partly generated as a consequence of the interaction between oxygen and hydrogen in the sub-surface region of the silver, probably generate this strongly bound oxygen species. The presence of the sub-surface oxygen species appears to activate the silver for methanol dehydrogenation

The influence of hydrogen treatment and catalyst morphology on the interaction of oxygen with a silver catalyst

The interaction of an unsupported silver catalyst which had been pretreated by hydrogen at various temperatures with oxygen at 210°C has been studied using Temperature Programmed Reduction (TPR) over a temperature range up to 900°C. Hydrogen treatment at 500°C or above before the oxidation step causes the formation of extra species, thought to be OH groups in the sub-surface of the sample. A peak in the spectra attributable to oxygen strongly bound in the vicinity of surface defects is found to be dependent on the surface roughness and grain size of the silver sample used; hydrogen pretreatment causes the strongly bound oxygen in the vicinity of surface defects to be converted to sub-surface OH. It is also shown that the TPR measurements themselves influence the morphology of the sample and that these changes are comparable with structural changes which occur during the use of the catalysts for oxidative dehydrogenation of methanol. It is suggested that these structural changes are caused by the interaction of the sub-surface of the silver with both oxygen and hydrogen

The interaction between silver and N2O in relation to the oxidative dehydrogenation of methanol

The interaction of N2O with pure silver at temperatures up to 900 °C has been studied using temperature-programmed reduction and desorption; the interaction is compared with that of oxygen with silver. The effect of addition of N2O, as well as of the complete replacement of oxygen by N2O, on the oxidative dehydrogenation of methanol on a silver catalyst has also been studied. It was found that the interaction of silver with N2O was much slower than that of O2; no atomic surface oxygen species were observed, probably because the formation of subsurface species was not complete; selective adsorption appears to take place on the surface defects and grain boundaries involved in the formation of the subsurface species. Addition of small amounts of N2O to the reaction mixture (CH3OH + O2) for the oxidative dehydrogenation of methanol had almost no influence on the conversion or on the product distribution measured. However, the conversions were considerably lower when oxygen was totally replaced by N2O; only above 600 °C was the N2O exhausted. At the same level of conversion of the methanol, the amount of CO2 produced was lowered compared to the case of O2. This is in agreement with the suggestion that CO2 is formed via weakly bound surface oxygen