Learning Sparse & Ternary Neural Networks with Entropy-Constrained Trained Ternarization (EC2T)

Deep neural networks (DNN) have shown remarkable success in a variety of machine learning applications. The capacity of these models (i.e., number of parameters), endows them with expressive power and allows them to reach the desired performance. In recent years, there is an increasing interest in deploying DNNs to resource-constrained devices (i.e., mobile devices) with limited energy, memory, and computational budget. To address this problem, we propose Entropy-Constrained Trained Ternarization (EC2T), a general framework to create sparse and ternary neural networks which are efficient in terms of storage (e.g., at most two binary-masks and two full-precision values are required to save a weight matrix) and computation (e.g., MAC operations are reduced to a few accumulations plus two multiplications). This approach consists of two steps. First, a super-network is created by scaling the dimensions of a pre-trained model (i.e., its width and depth). Subsequently, this super-network is simultaneously pruned (using an entropy constraint) and quantized (that is, ternary values are assigned layer-wise) in a training process, resulting in a sparse and ternary network representation. We validate the proposed approach in CIFAR-10, CIFAR-100, and ImageNet datasets, showing its effectiveness in image classification tasks.Comment: Proceedings of the CVPR'20 Joint Workshop on Efficient Deep Learning in Computer Vision. Code is available at https://github.com/d-becking/efficientCNN

Модели общественных явлений и сценарные подходы в принятии решений

Предложен новый класс моделей с ассоциативной памятью для изучения явлений в больших социальных системах. Модели имеют структуру, сходную со структурой нейросетевых моделей хопфилдовского класса. Учет в предложенной концепции интеллектуальных свойств субъектов общественных процессов позволил значительно расширить круг явлений, моделирование которых становится возможным. В частности, учет способности субъектов строить прогнозы ситуаций и принимать решения на базе этих прогнозов приводит к совершенно новым свойствам решений, главное из которых — возникновение многозначных решений, что на уровне большой общественной системы приводит к появлению множества способов поведения такой системы, т.е. сценариев развития событий. Обсуждаются также некоторые аналогии с поведением квантово-механических систем.Запропоновано новий клас моделей з властивостями асоціативної пам’яті для вивчення явищ у великих соціальних системах. Моделі мають структуру, подібну до структури нейронних мереж хопфілдівського класу. Врахування в запропонованій концепції інтелектуальних властивостей, притаманних суб’єктам суспільного процесу, дозволило значно розширити коло явищ, моделювання яких стає можливим. Зокрема, спроможність суб’єктів будувати прогнози ситуацій та приймати рішення, засновані на цих прогнозах, призводять до цілком нових властивостей розв’язків, основним з яких є можливість появи багатозначних розв’язків. На рівні великої суспільної системи це призводить до появи великої кількості способів поведінки такої системи, тобто сценаріїв розвитку подій. Обговорюються також деякі аналогії з поведінкою квантово- механічних систем.A new class of models is proposed with the properties of associative memory to be used in investigation of large social systems. The models have the structure similar to neuronets models of the Hopfield type. Accounting the intellectual properties allowed extending the scope of modelable phenomena. In particular, the individual ability for forecasting and decision-making results in new properties of solutions. The main new property is possible appearance of multivalued solutions. For large social systems, this leads to the existence of many trajectories of system, that is, many scenarios of behavior. Some analogies with quantum-mechanical phenomena are discussed

Change Agents in Global Initiatives

In December 2012, the President of Fort Hays State University attended a meeting with faculty and staff from the Department of Advanced Education Programs. During the meeting, he asked an important question,” What are you doing for Kansans?” The question became a challenge for a few faculty members that helped guide the direction of an initiative to provide professional development support for teachers on technology integration in the 21st Century classroom. The President’s question provided the impetus to keep going and to develop a more succinct plan to provide support for schools; especially as schools developed Google Domains and began to use the free applications offered by Google and others. As a result, 19 faculty and staff worked together to become Google Certified Educators. The process, itself, helped form a team of six individuals who went on to complete Google for Education Trainer certification. The gTrainers quickly developed a vision for what could be accomplished based on the belief that technology integration is more than an “add-on.” It is a mindset where the tools become the means for creating challenging, engaging projects and an environment for student learning

Implications of the Digital Divide for Technology Integration in Schools: A White Paper

The implications of the Digital Divide in 2019 vary slightly from the onset of Internet use. Max Weber’s (1922, 1978) theory of stratification provides a foundation for understanding why the growth is slow. Defining the Digital Divide is difficult. In reality, it has moved from a situation of access to acquiring sufficient skills to use it effectively. We examines the issue from the perspective of the need for technology literate leaders in schools. Technology literate leaders not only embrace technology: they model it. The leaders provide teachers with access and training to implement the best pedagogically sound teaching strategies so that students have optimum opportunities to learn with technology

Cultural Capital and the Digital Divide: A Literature Review

The internet holds promise of rapid diffusion of information to the global community and the potential to change our way of life. Those without access or with limited access to the internet are at a disadvantage. This paper examined the implications and issues surrounding the Digital Divide as it relates to Bourdieu\u27s theory of cultural capital. It is an examination of the literature on the Digital Divide in education