Going Big: A Large-Scale Study on What Big Data Developers Ask

Software developers are increasingly required to write big data code. However, they find big data software development challenging. To help these developers it is necessary to understand big data topics that they are interested in and the difficulty of finding answers for questions in these topics. In this work, we conduct a large-scale study on Stackoverflow to understand the interest and difficulties of big data developers. To conduct the study, we develop a set of big data tags to extract big data posts from Stackoverflow; use topic modeling to group these posts into big data topics; group similar topics into categories to construct a topic hierarchy; analyze popularity and difficulty of topics and their correlations; and discuss implications of our findings for practice, research and education of big data software development and investigate their coincidence with the findings of previous work

Learning Visual Routines with Reinforcement Learning

Reinforcement learning is an ideal framework to learn visual routines since the routines are made up of sequences of actions. However, such algorithms must be able to handle the hidden state (perceptual aliasing) that results from visual routine's purposefully narrowed attention. The U-Tree algorithm successfully learns visual routines for a complex driving task in which the agent makes eye movements and executes deictic actions in order to weave in and out of traffic on a four-laned highway. The task involves hidden state, time pressure, stochasticity, a large world state space, and a large perceptual state space. U-Tree uses a tree-structured representation, and is related to work on Prediction Suffix Trees (Ron, Singer, & Tishby 1994), Parti-game (Moore 1993), Galgorithm (Chapman & Kaelbling 1991), and Variable Resolution Dynamic Programming (Moore 1991). UTree is a direct descendant of Utile Suffix Memory (McCallum 1995c), which used short-term memory, but not selective perception..

Learning to Use Selective Attention and Short-Term Memory in Sequential Tasks

This paper presents U-Tree, a reinforcement learning algorithm that uses selective attention and shortterm memory to simultaneously address the intertwined problems of large perceptual state spaces and hidden state. By combining the advantages of work in instance-based (or "memory-based") learning and work with robust statistical tests for separating noise from task structure, the method learns quickly, creates only task-relevant state distinctions, and handles noise well. U-Tree uses a tree-structured representation, and is related to work on Prediction Suffix Trees [Ron et al., 1994] , Parti-game [Moore, 1993] , G-algorithm [Chapman and Kaelbling, 1991] , and Variable Resolution Dynamic Programming [Moore, 1991] . It builds on Utile Suffix Memory [McCallum, 1995c] , which only used short-term memory, not selective perception. The algorithm is demonstrated solving a highway driving task in which the agent weaves around slower and faster traffic. The agent uses active perception with ..

Employing EM in pool-based active learning for text classification

This paper shows how a text classifier’s need for labeled training documents can be reduced by taking advantage of a large pool of unlabeled documents. We modify the Query-by-Committee (QBC) method of active learning to use the unlabeled pool for explicitly estimating document density when selecting examples for labeling. Then active learning is combined with Expectation-Maximization in order to “fill in ” the class labels of those documents that remain unlabeled. Experimental results show that the improvements to active learning require less than two-thirds as many labeled training examples as previous QBC approaches, and that the combination of EM and active learning requires only slightly more than half as many labeled training examples to achieve the same accuracy as either the improved active learning or EM alone.

Learning Task-Relevant State Spaces with a Utile Distinction Test

This paper presents a reinforcement learning algorithm that learns an agent-internal state space on-line, in response to the demands of the task---thus avoiding the need for the agent designer to delicately engineer the agent's internal state space. The algorithm scales well with (1) large perceptual state spaces by pruning away unnecessary features, and (2) "overly small" perceptual state spaces (i.e. hidden state) by augmenting the provided features with short-term memory of past features. The algorithm, called U-Tree, uses a tree-structured representation, and is related to work on Prediction Suffix Trees (Ron, Singer, & Tishby 1994), Parti-game (Moore 1993), Galgorithm (Chapman & Kaelbling 1991), and Variable Resolution Dynamic Programming (Moore 1991). U-Tree is a direct descendant of Utile Suffix Memory (McCallum 1995c), which used short-term memory, but not selective perception. U-Tree is demonstrated solving a highway driving task in which the agent weaves around slower and fas..

Using reinforcement learning to spider the Web efficiently

Consider the task of exploring the Web in order to find pages of a particular kind or on a particular topic. This task arises in the construction of search engines and Web knowledge bases. This paper argues that the creation of efficient web spiders is best framed and solved by reinforcement learning, a branch of machine learning that concerns itself with optimal sequential decision making. One strength of reinforcement learning is that it provides a formalism for measuring the utility of actions that give benefit only in the future. We present an algorithm for learning a value function that maps hyperlinks to future discounted reward using a naive Bayes text classifier. Experiments on two real-world spidering tasks show a threefold improvement in spidering efficiency over traditional breadth-first search, and up to a two-fold improvement over reinforcement learning with immediate reward only