Finding Minimal Cost Herbrand Models with Branch-Cut-and-Price

Given (1) a set of clauses $T$ in some first-order language $\cal L$ and (2) a cost function $c : B_{{\cal L}} \rightarrow \mathbb{R}_{+}$, mapping each ground atom in the Herbrand base $B_{{\cal L}}$ to a non-negative real, then the problem of finding a minimal cost Herbrand model is to either find a Herbrand model $\cal I$ of $T$ which is guaranteed to minimise the sum of the costs of true ground atoms, or establish that there is no Herbrand model for $T$. A branch-cut-and-price integer programming (IP) approach to solving this problem is presented. Since the number of ground instantiations of clauses and the size of the Herbrand base are both infinite in general, we add the corresponding IP constraints and IP variables `on the fly' via `cutting' and `pricing' respectively. In the special case of a finite Herbrand base we show that adding all IP variables and constraints from the outset can be advantageous, showing that a challenging Markov logic network MAP problem can be solved in this way if encoded appropriately

Accelerating Reinforcement Learning by Composing Solutions of Automatically Identified Subtasks

This paper discusses a system that accelerates reinforcement learning by using transfer from related tasks. Without such transfer, even if two tasks are very similar at some abstract level, an extensive re-learning effort is required. The system achieves much of its power by transferring parts of previously learned solutions rather than a single complete solution. The system exploits strong features in the multi-dimensional function produced by reinforcement learning in solving a particular task. These features are stable and easy to recognize early in the learning process. They generate a partitioning of the state space and thus the function. The partition is represented as a graph. This is used to index and compose functions stored in a case base to form a close approximation to the solution of the new task. Experiments demonstrate that function composition often produces more than an order of magnitude increase in learning rate compared to a basic reinforcement learning algorithm

Power Allocation and Time-Domain Artificial Noise Design for Wiretap OFDM with Discrete Inputs

Optimal power allocation for orthogonal frequency division multiplexing (OFDM) wiretap channels with Gaussian channel inputs has already been studied in some previous works from an information theoretical viewpoint. However, these results are not sufficient for practical system design. One reason is that discrete channel inputs, such as quadrature amplitude modulation (QAM) signals, instead of Gaussian channel inputs, are deployed in current practical wireless systems to maintain moderate peak transmission power and receiver complexity. In this paper, we investigate the power allocation and artificial noise design for OFDM wiretap channels with discrete channel inputs. We first prove that the secrecy rate function for discrete channel inputs is nonconcave with respect to the transmission power. To resolve the corresponding nonconvex secrecy rate maximization problem, we develop a low-complexity power allocation algorithm, which yields a duality gap diminishing in the order of O(1/\sqrt{N}), where N is the number of subcarriers of OFDM. We then show that independent frequency-domain artificial noise cannot improve the secrecy rate of single-antenna wiretap channels. Towards this end, we propose a novel time-domain artificial noise design which exploits temporal degrees of freedom provided by the cyclic prefix of OFDM systems {to jam the eavesdropper and boost the secrecy rate even with a single antenna at the transmitter}. Numerical results are provided to illustrate the performance of the proposed design schemes.Comment: 12 pages, 7 figures, accepted by IEEE Transactions on Wireless Communications, Jan. 201