Weight Averaging Improves Knowledge Distillation under Domain Shift

Knowledge distillation (KD) is a powerful model compression technique broadly used in practical deep learning applications. It is focused on training a small student network to mimic a larger teacher network. While it is widely known that KD can offer an improvement to student generalization in i.i.d setting, its performance under domain shift, i.e. the performance of student networks on data from domains unseen during training, has received little attention in the literature. In this paper we make a step towards bridging the research fields of knowledge distillation and domain generalization. We show that weight averaging techniques proposed in domain generalization literature, such as SWAD and SMA, also improve the performance of knowledge distillation under domain shift. In addition, we propose a simplistic weight averaging strategy that does not require evaluation on validation data during training and show that it performs on par with SWAD and SMA when applied to KD. We name our final distillation approach Weight-Averaged Knowledge Distillation (WAKD).Comment: ICCV 2023 Workshop on Out-of-Distribution Generalization in Computer Vision (OOD-CV

Developing Collective Teaching Computer Software for the Course “Decision Theory”

Improvement of training students using modern information technologies, like collective developing teaching computer software, is discussed. Organizational, technical, technological advices are given. Experience of using information technologies in educational course “Decision Theory” is described

Surface ZnSe:Ca layers with hole conductivity

The authors investigate the effect of treating n-ZnSe substrates with boiling aqueous Ca(NO₃)₂ suspension on their electrical and luminescent properties. Base substrates were cut from bulk pure zinc selenide crystals grown from a stoichiometric melt by the Bridgman method. It was found that the Ca-doping of the substrates causes an almost complete “quenching” of the low-energy orange emission band with a maximum near hωmax ≈ 1,95 eV and a significant increase in the efficiency of the edge blue luminescence band.Исследовано влияние обработки подложек n-ZnSe в кипящей водной суспензии соли Ca(NO₃)₂ на их элек-трические и люминесцентные свойства.Досліджено вплив обробки підкладинок n-ZnSe в киплячій водній суспензії солі Ca(NO₃)₂ на їхні електричні та люмінесцентні властивості

CALCULATION OF HEAT CAPACITY IN MEAT DURING ITS FREEZING CONSIDERING PHASE CHANGE

As a consequence of insufficient study of water phase change in meat accompanied by water crystallization, its modeling is currently based on the empirical dependence of the frozen water portion on temperature. Such model does not allow answering a number of questions such as of metrological order, and also of physicochemical interpretation of processes occurring in meat during water crystallization. In this paper, we propose an approach to modeling the phase change process of meat during its freezing on the basis of the phonon theory of Debye crystallization, which allows to obtain physically justified dependences of heat capacity on temperature in the phase change region. The obtained dependences may serve as a simple method for calculating the heat capacity of meat in the temperature range of 113 K to the cryoscopic temperature of the given meat type, or as a basis for the analysis and correction of factors affecting the meat freezing in the temperature range of the phase change

Three Problems of the Theory of Choice on Random Sets

This paper discusses three problems which are united not only by the common topic of research stated in the title, but also by a somewhat surprising interlacing of the methods and techniques used. In the first problem, an attempt is made to resolve a very unpleasant metaproblem arising in general choice theory: why the conditions of rationality are not really necessary or, in other words, why in every-day life we are quite satisfied with choice methods which are far from being ideal. The answer, substantiated by a number of results, is as follows: situations in which the choice function "misbehaves" are very seldom met in large presentations. In the second problem, an overview of our studies is given on the problem of statistical properties of choice. One of the most astonishing phenomenon found when we deviate from scalar-extremal choice functions is in stable multiplicity of choice. If our presentation is random, then a random number of alternatives is chosen in it. But how many? The answer isn't trivial, and may be sought in many different directions. As we shall see below, usually a bottleneck case was considered in seeking the answer. It is interesting to note that statistical information effects the properties of the problem very much. The third problem is devoted to a model of a real life choice process. This process is typically spread in time, and we gradually (up to the time of making a final decision) accumulate experience, but once a decision is made we are not free to reject it. In the classical statement (i.e. when "optimality" is measured by some number) this model is referred to as a "secretary problem", and a great deal of literature is devoted to its study. We consider the case when the notions of optimality are most general. As will be seen below, the best strategy is practically determined by only the statistical properties of the corresponding choice function rather than its specific form

ПОРІВНЯЛЬНА ХАРАКТЕРИСТИКА РЕЗУЛЬТАТІВ ТЕСТУ НА ФІЗИЧНУ ПІДГОТОВЛЕНІСТЬ НАСЕЛЕННЯ УКРАЇНИ З БІГУ НА 100 МЕТРІВ НА ПОЧАТКУ І В КІНЦІ НАВЧАЛЬНОГО РОКУ СТУДЕНТІВ І КУРСУ КЕІКНЕУ У 1999/2000 ТА 2002/2003 НАВЧАЛЬНИХ РОКАХ

Attentive vision at the results of the state test on speed

A calculation model for the heat capacity of beef with different moisture during freezing taking into account free water crystallization

The paper proposes a model for the process of free moisture crystallization in beef within the framework of the Debye concept with establishment of dependencies of model parameters on the initial moisture content. Model adequacy was validated by comparison of the calculation results with the results of the experiments on determination of values of heat capacity and phase transition enthalpy in beef with different initial moisture obtained by the differential scanning calorimetry method. It is shown that the end of free water phase transition in beef with initial moisture in a range of 37% to 80% occurs at a temperature of 243 К. Calculation dependencies of parameters of the model used for calculation of beef heat capacity are presented