Parallel growing and training of neural networks using output parallelism

In order to find an appropriate architecture for a large-scale real-world application automatically and efficiently, a natural method is to divide the original problem into a set of sub-problems. In this paper, we propose a simple neural network task decomposition method based on output parallelism. By using this method, a problem can be divided flexibly into several sub-problems as chosen, each of which is composed of the whole input vector and a fraction of the output vector. Each module (for one sub-problem) is responsible for producing a fraction of the output vector of the original problem. The hidden structure for the original problem’s output units are decoupled. These modules can be grown and trained in parallel on parallel processing elements. Incorporated with a constructive learning algorithm, our method does not require excessive computation and any prior knowledge concerning decomposition. The feasibility of output parallelism is analyzed and proved. Some benchmarks are implemented to test the validity of this method. Their results show that this method can reduce computational time, increase learning speed and improve generalization accuracy for both classification and regression problems

An Analysis of Differences in Approaches to Systems of Linear Equations Problems Given Multiple Choice Answers

This descriptive study focuses on the approaches college students (ages 20 -24) use when solving systems of linear equations problems that have multiple choice answers. Participants were from a midsize public university in the northeast. Four approaches were considered – three forwards approaches: 1) substitution, 2) elimination, and 3) graphing, and one backwards approach: plugging in the x and y values from each multiple choice option. Participants solved systems of linear equations problems and answered questions based on their methods in a structured clinical interview. Each participant also filled out a questionnaire. It was shown from the results of this study that the major of a student does not change the approach used to solve a problem by very much. Most students in the study chose to use substitution to solve the problems, usually because this was the method students remembered most and was deemed the “easiest” method by the students

Taking Turing by Surprise? Designing Digital Computers for morally-loaded contexts

There is much to learn from what Turing hastily dismissed as Lady Lovelace s objection. Digital computers can indeed surprise us. Just like a piece of art, algorithms can be designed in such a way as to lead us to question our understanding of the world, or our place within it. Some humans do lose the capacity to be surprised in that way. It might be fear, or it might be the comfort of ideological certainties. As lazy normative animals, we do need to be able to rely on authorities to simplify our reasoning: that is ok. Yet the growing sophistication of systems designed to free us from the constraints of normative engagement may take us past a point of no-return. What if, through lack of normative exercise, our moral muscles became so atrophied as to leave us unable to question our social practices? This paper makes two distinct normative claims: 1. Decision-support systems should be designed with a view to regularly jolting us out of our moral torpor. 2. Without the depth of habit to somatically anchor model certainty, a computer s experience of something new is very different from that which in humans gives rise to non-trivial surprises. This asymmetry has key repercussions when it comes to the shape of ethical agency in artificial moral agents. The worry is not just that they would be likely to leap morally ahead of us, unencumbered by habits. The main reason to doubt that the moral trajectories of humans v. autonomous systems might remain compatible stems from the asymmetry in the mechanisms underlying moral change. Whereas in humans surprises will continue to play an important role in waking us to the need for moral change, cognitive processes will rule when it comes to machines. This asymmetry will translate into increasingly different moral outlooks, to the point of likely unintelligibility. The latter prospect is enough to doubt the desirability of autonomous moral agents

Twenty Years of Edtech

An opinion often cited among educational technology (edtech) professionals is that theirs is a fast-changing field. This statement is sometimes used as a motivation (or veiled threat) to senior managers to embrace edtech because if they miss out now, it’ll be too late to catch up. However, amid this breathless attempt to keep abreast of new developments, the edtech field is remarkably poor at recording its own history or reflecting critically on its development. When Audrey Watters recently put out a request for recommended books on the history of educational technology, I couldn’t come up with any beyond the handful she already had listed. There are edtech books that often start with a historical chapter to set the current work in context, and there are edtech books that are now part of history, but there are very few edtech books dealing specifically with the field’s history. Maybe this reflects a lack of interest, as there has always been something of a year-zero mentality in the field. Edtech is also an area to which people come from other disciplines, so there is no shared set of concepts or history. This can be liberating but also infuriating. I’m sure I was not alone in emitting the occasional sigh when during the MOOC rush of 2012, so many “new” discoveries about online learning were reported—discoveries that were already tired concepts in the edtech field

Optimal rates and adaptation in the single-index model using aggregation

We want to recover the regression function in the single-index model. Using an aggregation algorithm with local polynomial estimators, we answer in particular to the second part of Question~2 from Stone (1982) on the optimal convergence rate. The procedure constructed here has strong adaptation properties: it adapts both to the smoothness of the link function and to the unknown index. Moreover, the procedure locally adapts to the distribution of the design. We propose new upper bounds for the local polynomial estimator (which are results of independent interest) that allows a fairly general design. The behavior of this algorithm is studied through numerical simulations. In particular, we show empirically that it improves strongly over empirical risk minimization.Comment: 36 page