Motivated by a wide-spread use of convex optimization techniques, convexity
properties of bit error rate of the maximum likelihood detector operating in
the AWGN channel are studied for arbitrary constellations and bit mappings,
which may also include coding under maximum-likelihood decoding. Under this
generic setting, the pairwise probability of error and bit error rate are shown
to be convex functions of the SNR in the high SNR regime with
explicitly-determined boundary. The bit error rate is also shown to be a convex
function of the noise power in the low noise/high SNR regime.Comment: to be presented at IZS-10, March 201

Gagnon, Francois

Kostina, Victoria

Loyka, Sergey

English

arXiv

Gagno, Francois

Repository for Publications and Research Data

Bit error rate is convex at high SNR

Motivated by a wide-spread use of convex optimization techniques, convexity properties of bit error rate of the maximum likelihood detector operating in the AWGN channel are studied for arbitrary constellations and bit mappings, which may also include coding under maximum-likelihood decoding. Under this generic setting, the pairwise probability of error and bit error rate are shown to be convex functions of the SNR in the high SNR regime with explicitly-determined boundary. The bit error rate is also shown to be a convex function of the noise power in the low noise/high SNR regime

Gagnon, François

Caltech Authors - Main

   
Bit Error Rate is Convex at High SNR 
 
Sergey Loyka 
SITE 
University of Ottawa 
Ottawa, K1N 6N5, Canada 
e-mail: sergey.loyka@ieee.org 
Victoria Kostina 
Department of Electrical Engineering 
Princeton University 
Princeton, NJ, 08544, USA 
e-mail: vkostina@princeton.edu. 
Francois Gagnon 
Department of Electrical Engineering 
Ecole de Technologie Superieure 
Montreal, H3C 1K3, Canada 
e-mail: francois.gagnon@etsmtl.ca 
 
Abstract— Motivated by a wide-spread use of convex 
optimization techniques, convexity properties of bit error rate of 
the maximum likelihood detector operating in the AWGN 
channel are studied for arbitrary constellations and bit mappings, 
which may also include coding under maximum-likelihood 
decoding. Under this generic setting, the pairwise probability of 
error and bit error rate are shown to be convex functions of the 
SNR in the high SNR regime with explicitly-determined 
boundary. The bit error rate is also shown to be a convex function 
of the noise power in the low noise/high SNR regime. 
I. INTRODUCTION 
Optimization problems of various kinds simplify 
significantly if the goal and constraint functions involved are 
convex. Indeed, a convex optimization problem has a unique 
global solution, which can be found either analytically or, with 
a reasonable effort, by several efficient numerical methods; its 
numerical complexity grows only moderately with theproblem 
dimensionality and required accuracy; convergence rat s and 
required step size can be estimated in advance; there are 
powerful analytical tools that can be used to attack  problem 
and that provide insights into such problems even if solutions, 
either analytical or numerical, are not found yet [1][2]. 
Contrary to this, not only generic nonlinear optimizat on 
problems do not possess these features, they are not solvable 
numerically, i.e. their complexity grows prohibitively fast with 
problem dimensionality and required accuracy [2]. Thus, there 
is a great advantage in formulating or at least in approximating 
an optimization problem as a convex one. 
In the world of digital communications, one of the major 
performance measures is either symbol or bit error rate (SER 
or BER). Consequently, when an optimization of a 
communication system is performed, either SER or BER often 
appears as goal or constraint functions. Examples include 
optimum power/rate allocation in spatial multiplexing systems 
(BLAST) [3], optimum power/time sharing for a transmitter 
and a jammer [4], rate allocation or precoding in multicarrier 
(OFDM) systems [5], optimum equalization [6], optimu  
multiuser detection [7], and joint Tx-Rx beamforming 
(precoding-decoding) in MIMO systems [8]. Symbol and bit 
error rates of the maximum likelihood (ML) detector have 
been extensively studied and a large number of exact or 
approximate analytical results are available for various 
modulation formats, for both non-fading and fading AWGN 
channels [9][10]. On the other hand, convexity properties of 
error rates are not understood so well, especially for 
constellations of complicated geometry, large dimensio ality 
or when coding is used. Results in this area are scarce. Many 
known closed-form error rate expressions can be verified by 
differentiation to be convex, but this approach does not 
provide any generic conclusions. Convexity properties for 
binary modulations have been studied in-depth in [4], 
including applications to transmitter and jammer 
optimizations, and were later extended to arbitrary 
multidimensional constellations in [11][12] in terms of the 
SER under maximum-likelihood detection. A log-concavity 
property of the SER as a function of the SNR [dB] for the 
uniform square-grid constellations has been establihed by 
Conti et al [13]. 
Unfortunately, convexity of SER does not say anything in 
general about convexity of the BER, since the latter depends 
on pairwise probabilities of error (PEP) and not on the SER 
[14]. Since the BER is an important performance indicator and 
thus appears as an objective in many optimization problems, 
we study its convexity in the present paper using a eneric 
geometrical framework developed in [11][12]. Our setting is 
generic enough so that the results apply to constellations of 
arbitrary order, shape and dimensionality, which may also 
include coding 
First, we establish convexity properties of the PEP as a 
function of SNR: it is convex at high SNR regime, con ave at 
the low one, and has an odd number of inflection points in-
between. Based on this, convexity of the BER at high SNR is 
established for arbitrary constellation and coding. Thus, this 
property is a consequence of Gaussian noise density and 
maximum likelihood detection rather than particular 
constellation or coding technique. We also show that t e BER 
is a convex function of the noise power in the small noise/high 
SNR mode. 
II. SYSTEM MODEL 
The standard baseband discrete-time system model with an 
AWGN channel, which includes matched filtering and 
sampling, is 
 = +r s ξ  (1) 
where s  and r  are n-dimensional vectors representing the Tx 
and Rx symbols respectively, { }1 2, ,..., M∈s s s s , a set of M 
constellation points, ξ  is the additive white Gaussian noise 
(AWGN), 20~ ( , )σξ 0 IN , whose probability density function 
(PDF) is 
 ( ) 2 20/ 2 220( ) 2 np e− − σξ = πσ xx  (2) 
where 20σ  is the noise variance per dimension, and n is the 
constellation dimensionality; lower case bold letters denote 
vectors, bold capitals denote matrices, ix  denotes i-th 
   
component of x , x  denotes L2 norm of x , T=x x x , where 
the superscript T denotes transpose, ix  denotes i-th vector. 
The average (over the constellation points) SNR is defined as 
2
01/γ = σ , which implies the appropriate normalization, 21
1
1
M
iM i= =∑ s . 
Consider the maximum likelihood detector, which is 
equivalent to the minimum distance one in the AWGN 
channel, ̂ arg min i i= −ss r s . The probability of symbol error 
eiP  given that i=s s  was transmitted is 
[ ]ˆPr 1ei i i ciP P= ≠ = = −s s s s , where ciP  is the probability of 
correct decision. The SER averaged over all constellation 
points is [ ]
1
Pr 1
M
e ei i ci
P P P== = = −∑ s s . eiP  can be expressed 
as 
 1 ( )
i
eiP p dξΩ
= − ∫ x x  (3) 
where iΩ  is the decision region (Voronoi region), and is
corresponds to 0=x , i.e. the origin is shifted for convenience 
to the constellation point is . iΩ  can be expressed as a convex 
polyhedron [1],  
 { } ( ) 1,   ,   
2
j iT
i j ij j
j i
b
−
Ω = ≤ = = −
−
s s
x Ax b a s s
s s
 (4) 
where Tja  denotes j-th row of A , and the inequality in (4) is 
applied component-wise. Clearly, eiP  and ciP  posses the 
opposite convexity properties. 
Another important performance indicator is the pairwise 
probability of error (PEP) i.e. a probability 
{ } ˆPr Pri j j i→ = = =  s s s s s s  to decide in favor of js  given 
that is , i j≠ , was transmitted, which can be expressed as 
 { }Pr ( )
j
i j p dξΩ
→ = ∫s s x x (5) 
where jΩ  is the decision region for js  when the reference 
frame is centered at is . The SER can now be expressed as 
 { }Prei i jj iP ≠= →∑ s s  (6) 
and the BER can be expressed as a positive linear combination 
of PEPs [14] 
 { } { }
21
BER Pr Pr
log
M
ij
i i j
i j i
h
M= ≠
= = →∑∑ s s s s  (7) 
where ijh  is the Hamming distance between binary sequences 
representing is  and js . 
Note that the model and error rate expressions we are using 
are generic enough to apply to arbitrary constellations, which 
may also include coding under maximum-likelihood decoding 
(codewords are considered as points of an extended 
constellation). We now proceed to establish convexity 
properties of error rates in this generic setting. 
 
III.  CONVEXITY OF SYMBOL ERROR RATES 
Convexity properties of symbol error rates for arbitrary 
constellations in the SNR and noise power have been 
established in [11][12] and are summarized below for
completeness and comparison purpose. 
Theorem 1 (Theorem 1 and 2 in [11]): The SER eP  is a 
convex function of the SNR γ  for any constellation (which 
may also include coding) if 2n ≤ , 
 2 2 0e ed P d P γ′′γ = >  (8) 
For 2n > , the following convexity properties hold: 
• eiP  is convex in the large SNR mode, 
 ( ) 2min,2 in n dγ ≥ +  (9) 
where min,id  is the minimum distance from is  to its 
decision region boundary, 
• eiP  is concave in the small SNR mode, 
 ( ) 2max,2 in n dγ ≤ −  (10) 
where max,id  is the maximum distance from is  to its 
decision region boundary, 
• there are an odd number of inflection points, 
0ci eiP Pγ γ= =′′ ′′ , in the intermediate SNR mode, 
 ( ) ( )2 2max, min,2 2i in n d n n d− ≤ γ ≤ +  (11) 
• the SER eP  is convex at high SNR, 
 ( ) 2min2n n dγ ≥ +  (12) 
where { }min min,mini id d=  is the minimum distance to 
decision region boundary in the constellation. 
 
Theorem 2 (Theorem 4 in [11]): Symbol error rates have 
the following convexity properties in the noise power 
2
0NP = σ , for any n and constellation geometry, 
• eiP  is concave in the large noise mode, 
 ( ) 12max, 2 2( 2)N iP d n n −≥ + − +  (13) 
• eiP  is convex in the small noise mode, 
 ( ) 12min, 2 2( 2)N iP d n n −≤ + + +  (14) 
• there are an odd number of inflection points for 
intermediate noise power, 
( ) ( )1 12 2max,min, 2 2( 2) 2 2( 2)N iid n n P d n n− −+ + + ≤ ≤ + − +  (15) 
• the SER eP  is convex in the small noise/high SNR mode, 
 ( ) 12min 2 2( 2)NP d n n −≤ + + +  (16) 
While the convexity properties above are important for many 
optimization problems, they do not lend any conclusion  about 
convexity of the BER, since the latter is not directly related to 
eP  or eiP  in general. While, in some cases, the BER can be 
expressed as linear combination of eiP , there are positive and 
negative terms so that no conclusion about convexity can be 
made in this case either. On the other hand, the BER can be 
expressed as a positive linear combination of pairwise 
probabilities of error so that the convexity of the latter implies 
the convexity of the former. Thus, we study below the 
   
convexity property of the PEP, from which the convexity 
property of the BER will follow. 
 
IV.  CONVEXITY OF PAIRWISE PROBABILITY OF ERROR 
In many cases, it is a pairwise error probability that is a key 
point in the analysis (e.g. for constructing a union bound and 
other performance metrics). Furthermore, it is also a basic 
building block for the BER in (7), so that we establish its 
convexity property first. 
 
Theorem 3:  
a) The pairwise error probability { }Pr i j→s s  is a convex 
function of the SNR at the high SNR region, 
2
min,( 2 ) / in n dγ ≥ + , for any n; 
b) for 1, 2n = , it is concave at the low SNR region, 
2
max,( 2 ) / ( )ij jn n d dγ ≤ + + , where ij i jd = −s s  is the 
distance between is  and js , and there is an odd number of 
inflection points, { }Pr 0i j ′′→ =s s , in the intermediate SNR 
mode, 
 2 2max, min,( 2 ) / ( ) ( 2 ) /ij j in n d d n n d+ + ≤ γ ≤ +  (17) 
c) for 2n > , the PEP is convex at the low SNR region, 
2
max,( 2 ) / ( )ij jn n d dγ ≤ − + , and there is an even number of 
inflection points in-between, 
2 2
max, min,( 2 ) / ( ) ( 2 ) /ij j in n d d n n d− + ≤ γ ≤ +  
Proof: See Appendix. 
 
We note that Theorem 3(a) is stronger than Theorem 1 at the 
high SNR region since the latter follows from the former but 
the opposite is not always true (as the other SNR ranges in 
Theorem 3 above indicate). Unlike the SER, the pairwise error 
probability can be concave at low SNR even for 1, 2n = . 
Since Theorem 3 holds for any constellation and bit 
mapping, it follows that the convexity property of the PEP at 
high SNR is a consequence of Gaussian noise density rather 
than particular modulation/coding used, where the latter 
determines only the SNR threshold. 
V. CONVEXITY OF THE BER AT HIGH SNR 
We are now in a position to establish the main result of this 
paper. 
Theorem 4: The BER is a convex function of the SNR, for 
any constellation and bit mapping, which may also include 
coding under maximum-likelihood decoding, at the high SNR 
regime, 
 2min( 2 ) /n n dγ ≥ + , (18) 
where { }min min,mini id d=  is the minimum distance to the 
boundary in the constellation. 
Proof: Using the relationship between the BER and the 
pairwise error probabilities in (7) and observing that a positive 
linear combination of convex functions is convex. Q.E.D. 
We remark that the condition in (18) guarantees the
convexity of all PEP, BER and SER. In some cases (Gray 
encoding and when nearest neighbor errors dominate), th  
BER is approximated as 2SER/ log M , so that it inherits the 
same convexity properties as in Theorems 1 and 2 above. 
VI.  CONVEXITY OF THE PEP AND BER IN NOISE POWER 
In a jammer optimization problem, it is convexity pro erties in 
noise power that are important [4]. Motivated by this fact, we 
study below convexity of the PEP and BER in the noise power. 
Theorem 5: The PEP { }Pr i j→s s  is a convex function of 
the noise power 20NP = σ , for any n, in the small noise/high 
SNR mode, 
 ( ) 12min, 2 2( 2)N iP d n n −≤ + + +  (19) 
and in the large noise/low SNR mode, 
 ( ) 12max,( ) 2 2( 2)N ij jP d d n n −≥ + + − +  (20) 
Proof: See Appendix. 
Based on this Theorem, the following convexity property of 
the BER is established. 
Corollary 5.1: For any constellation geometry and 
dimensionality, which may also include coding under ML 
decoding, the BER is a convex function of the noise power in 
the small noise/high SNR mode: 
 ( ) 12min 2 2( 2)NP d n n −≤ + + +  (21) 
where specifics of the constellation/code determine o ly the 
upper bound in (21). 
VII.  REFERENCES 
[1] S. Boyd, L. Vandenberghe, Convex Optimization, Cambridge 
University Press, 2004. 
[2] A. Ben-Tal, A. Nemirovski, Lectrures on Modern Convex 
Optimization, MPS-SIAM Series on Optimization, Philadelphia, 
2001. 
[3] V. Kostina, S. Loyka, On Optimum Power Allocation for the 
V-BLAST, IEEE Transactions on Communications, v. 56, N. 6, 
pp. 999-1012, June 2008. 
[4] M. Azizoglu, Convexity Properties in Binary Detection 
Problems, IEEE Trans. Inform. Theory, v. 42, N. 4, pp. 1316-
1321, July 1996. 
[5] Y.-P. Lin, S.-M. Phoong, BER Minimized OFDM Systems 
With Channel Independent Precoders, IEEE Trans. Signal 
Processing, v.51, N.9, pp. 2369-2380, Sep. 2003. 
[6] C.C. Yeh, J.R. Barry, Adaptive Minimum Bit-Error Rate 
Equalization for Binary Signaling, IEEE Trans. 
Communications, v.48, N.7, pp. 1226-1235, Jul. 2000. 
[7] X. Wang, W.S. Lu, A. Antoniou, Constrained Minimum-BER 
Multiuser Detection, IEEE Trans. Signal Processing, v.48, N.10, 
pp. 2903-2909, Oct. 2000. 
[8] D.P. Palomar, J.M. Cioffi, M.A. Lagunas, Joint Tx-R 
Beamforming Design for Multicarrier MIMO Channels: A
Unified Framework for Convex Optimization, IEEE Trans. 
Signal Processing, v.51, N.9, pp. 2381-2401, Sep. 2003. 
[9] J.M. Wozencraft, I.M. Jacobs, Principles of Communication 
Engineering, Wiley, 1965. 
   
[10] J.R. Barry, E.A. Lee, D.G. Messerschmitt, Digital 
Coomunications (3rd Ed.), Kluwer, Boston, 2004. 
[11] S. Loyka. V. Kostina, F. Gagnon, Symbol Error Rates of 
Maximum-Likelihood Detector: Convex/Concave Behavior and 
Applications, IEEE International Symposium on Information 
Theory (ISIT’07), June 2007, Nice, France. 
[12] S. Loyka, V. Kostina, F. Gagnon, Error Rates of the 
Maximum-Likelihood Detector for Arbitrary Constellations: 
Convex/Concave Behavior and Applications, IEEE Transactions 
on Information Theory, accepted, 2009. 
[13] A. Conti et al., Log-Concavity Property of the Error 
Probability with Application to Local Bounds for Wireless 
Communications, IEEE Trans. Information Theory, June 2009. 
[14] J. Lassing et al, Computation of the Exact Bit-Error Rate of 
Coherent M-ary PSK with gray Code Bit Mapping, IEEE Trans. 
Communications, v. 51, N. 11, pp. 1758-1760, Nov. 2003. 
VIII.  APPENDIX 
Proof of Theorem 3: The pairwise probability of error 
{ }Prij i jP = →s s  can be presented as 
 ( )
j
ijP p dξΩ
= ∫ x x  (22) 
where jΩ  is the decision region for js  when the reference 
frame is centered at is . Its second derivative in the SNR is 
 
2
2
( )
j
ij
d p
P d
d
ξ
Ω
′′ =
γ∫
x
x  (23) 
where the derivative is 
 ( )2
/ 22
2/2
2
( ) 1
e
4 2
nd p
f
d
ξ −γγ =  γ π 
xx x  (24) 
and ( ) ( )1 2( ) / /f t t t= − α γ − α γ , 1 2 0n nα = + > , 
2 12n nα = − < α . Consider three different cases. 
(i) If 2 1min, /id ≥ α γ , where min, min ( )j jid b=  is the 
minimum distance from the origin to the boundary of iΩ , then 
2( ) 0f ≥x j∀ ∈ Ωx  so that the integral in (23) is clearly 
positive since the integrand is non-negative everywhere in the 
integration region and positive in some parts of it. Fig. 1 
illustrates this case. This is a high SNR mode since 
2
1 min,/ idγ ≥ α . 
(ii) If 2 1max,( ) /ij jd d+ ≤ α γ  and 1, 2n = , where max,jd  is 
the maximum distance from the center of jΩ  to its boundary, 
then 2( ) 0f ≤x  j∀ ∈ Ωx  so that the integral in (23) is clearly 
negative and the result follows. Fig. 2 illustrates thi  case. This 
is a low-SNR mode since 21 max,/ ( )ij jd dγ ≤ α + . An odd 
number of inflection points in Theorem 3(b) follows from the 
continuity argument (ijP′′  is a continuous function of the SNR). 
(iii) Part (c) follows from the same argument as in ( i). 
Q.E.D. 
 
Proof of Theorem 5: follows the same geometric technique 
as for Theorem 3. 2nd derivative of the PEP in the noise power 
can be expressed as 
 
2 2
2 2
( )
j
ij
N N
d P d p
d
dP P
ξ
Ω
= ∫
x
x  (25) 
where 
 
( )
( ) ( ) ( )
2
2 2 22 *
2 4
*
1 2
1 2
( ) 1 1
e
4 2
,
2 2( 2),   2 2( 2)
N
n
P
N N N
N N
d p
f
dP P P
f t t P t P
n n n n
−
ξ  =  π 
= − β − β
β = + + + β = + − +
x
x
x
 (26) 
 
and 1 2 0β > β > . Since ( )*f t  has the same structure as ( )f t  
in (24), the proof follows the same steps. In particular, if 
2
1min, Nid P≥ β , then 2 2/ 0 jNd p dPξ > ∀ ∈ Ωx  so that the 
integral in (25) is clearly positive. The other case i  proved in a 
similar way. Q.E.D. 
 
 
min,id
1α
γ
iΩ
1x
2x
+ +
+
+
2
( ) 0f >x
 
Fig. 1. Two-dimensional illustration of the problem geometry for 
Case 1. The decision region iΩ  is shaded. 
2( )f x  has a sign as 
indicated by “+” and “-“. 
 
ijd
1α
γ
iΩ
1x
2x
−
−
−
−
+
+
+
+
2
( ) 0f <x2( ) 0f >x
jΩ +
max,jd
 
Fig. 2. Two-dimentional illustration of the problem geometry for 
Case 2.  


Bit Error Rate is Convex at High SNR

  
 
Bit Error Rate is Convex at High SNR 
 
Sergey Loyka 
SITE 
University of Ottawa 
Ottawa, K1N 6N5, Canada 
e-mail: sergey.loyka@ieee.org 
Victoria Kostina 
Department of Electrical Engineering 
Princeton University 
Princeton, NJ, 08544, USA 
e-mail: vkostina@princeton.edu. 
Francois Gagnon 
Department of Electrical Engineering 
Ecole de Technologie Superieure 
Montreal, H3C 1K3, Canada 
e-mail: francois.gagnon@etsmtl.ca 
 
Abstract— Motivated by a wide-spread use of convex 
optimization techniques, convexity properties of bit error rate of 
the maximum likelihood detector operating in the AWGN 
channel are studied for arbitrary constellations and bit mappings, 
which may also include coding under maximum-likelihood 
decoding. Under this generic setting, the pairwise probability of 
error and bit error rate are shown to be convex functions of the 
SNR in the high SNR regime with explicitly-determined 
boundary. The bit error rate is also shown to be a convex function 
of the noise power in the low noise/high SNR regime. 
I. INTRODUCTION 
Optimization problems of various kinds simplify 
significantly if the goal and constraint functions involved are 
convex. Indeed, a convex optimization problem has a unique 
global solution, which can be found either analytically or, with 
a reasonable effort, by several efficient numerical methods; its 
numerical complexity grows only moderately with the problem 
dimensionality and required accuracy; convergence rates and 
required step size can be estimated in advance; there are 
powerful analytical tools that can be used to attack a problem 
and that provide insights into such problems even if solutions, 
either analytical or numerical, are not found yet [1][2]. 
Contrary to this, not only generic nonlinear optimization 
problems do not possess these features, they are not solvable 
numerically, i.e. their complexity grows prohibitively fast with 
problem dimensionality and required accuracy [2]. Thus, there 
is a great advantage in formulating or at least in approximating 
an optimization problem as a convex one. 
In the world of digital communications, one of the major 
performance measures is either symbol or bit error rate (SER 
or BER). Consequently, when an optimization of a 
communication system is performed, either SER or BER often 
appears as goal or constraint functions. Examples include 
optimum power/rate allocation in spatial multiplexing systems 
(BLAST) [3], optimum power/time sharing for a transmitter 
and a jammer [4], rate allocation or precoding in multicarrier 
(OFDM) systems [5], optimum equalization [6], optimum 
multiuser detection [7], and joint Tx-Rx beamforming 
(precoding-decoding) in MIMO systems [8]. Symbol and bit 
error rates of the maximum likelihood (ML) detector have 
been extensively studied and a large number of exact or 
approximate analytical results are available for various 
modulation formats, for both non-fading and fading AWGN 
channels [9][10]. On the other hand, convexity properties of 
error rates are not understood so well, especially for 
constellations of complicated geometry, large dimensionality 
or when coding is used. Results in this area are scarce. Many 
known closed-form error rate expressions can be verified by 
differentiation to be convex, but this approach does not 
provide any generic conclusions. Convexity properties for 
binary modulations have been studied in-depth in [4], 
including applications to transmitter and jammer 
optimizations, and were later extended to arbitrary 
multidimensional constellations in [11][12] in terms of the 
SER under maximum-likelihood detection. A log-concavity 
property of the SER as a function of the SNR [dB] for the 
uniform square-grid constellations has been established by 
Conti et al [13]. 
Unfortunately, convexity of SER does not say anything in 
general about convexity of the BER, since the latter depends 
on pairwise probabilities of error (PEP) and not on the SER 
[14]. Since the BER is an important performance indicator and 
thus appears as an objective in many optimization problems, 
we study its convexity in the present paper using a generic 
geometrical framework developed in [11][12]. Our setting is 
generic enough so that the results apply to constellations of 
arbitrary order, shape and dimensionality, which may also 
include coding 
First, we establish convexity properties of the PEP as a 
function of SNR: it is convex at high SNR regime, concave at 
the low one, and has an odd number of inflection points in-
between. Based on this, convexity of the BER at high SNR is 
established for arbitrary constellation and coding. Thus, this 
property is a consequence of Gaussian noise density and 
maximum likelihood detection rather than particular 
constellation or coding technique. We also show that the BER 
is a convex function of the noise power in the small noise/high 
SNR mode. 
II. SYSTEM MODEL 
The standard baseband discrete-time system model with an 
AWGN channel, which includes matched filtering and 
sampling, is 
 = +r s ξ  (1) 
where s  and r  are n-dimensional vectors representing the Tx 
and Rx symbols respectively, { }1 2, ,..., M∈s s s s , a set of M 
constellation points, ξ  is the additive white Gaussian noise 
(AWGN), 20~ ( , )σξ 0 IN , whose probability density function 
(PDF) is 
 ( ) 2 20/ 2 220( ) 2 np e− − σξ = piσ xx  (2) 
where 20σ  is the noise variance per dimension, and n is the 
constellation dimensionality; lower case bold letters denote 
vectors, bold capitals denote matrices, ix  denotes i-th 
  
 
component of x , x  denotes L2 norm of x , T=x x x , where 
the superscript T denotes transpose, ix  denotes i-th vector. 
The average (over the constellation points) SNR is defined as 
2
01 /γ = σ , which implies the appropriate normalization, 21
1 1
M
iM i= =∑ s . 
Consider the maximum likelihood detector, which is 
equivalent to the minimum distance one in the AWGN 
channel, ˆ arg min i i= −ss r s . The probability of symbol error 
eiP  given that i=s s  was transmitted is 
[ ]ˆPr 1ei i i ciP P= ≠ = = −s s s s , where ciP  is the probability of 
correct decision. The SER averaged over all constellation 
points is [ ]1 Pr 1
M
e ei i ciP P P== = = −∑ s s . eiP  can be expressed 
as 
 1 ( )
i
eiP p dξΩ= − ∫ x x  (3) 
where iΩ  is the decision region (Voronoi region), and is  
corresponds to 0=x , i.e. the origin is shifted for convenience 
to the constellation point is . iΩ  can be expressed as a convex 
polyhedron [1],  
 { } ( ) 1,   ,   
2
j iT
i j ij j
j i
b
−
Ω = ≤ = = −
−
s s
x Ax b a s s
s s
 (4) 
where Tja  denotes j-th row of A , and the inequality in (4) is 
applied component-wise. Clearly, eiP  and ciP  posses the 
opposite convexity properties. 
Another important performance indicator is the pairwise 
probability of error (PEP) i.e. a probability 
{ } ˆPr Pri j j i→ = = =  s s s s s s  to decide in favor of js  given 
that is , i j≠ , was transmitted, which can be expressed as 
 { }Pr ( )
j
i j p dξΩ→ = ∫s s x x  (5) 
where jΩ  is the decision region for js  when the reference 
frame is centered at is . The SER can now be expressed as 
 { }Prei i jj iP ≠= →∑ s s  (6) 
and the BER can be expressed as a positive linear combination 
of PEPs [14] 
 { } { }
21
BER Pr Pr
log
M
ij
i i j
i j i
h
M
= ≠
= = →∑∑ s s s s  (7) 
where ijh  is the Hamming distance between binary sequences 
representing is  and js . 
Note that the model and error rate expressions we are using 
are generic enough to apply to arbitrary constellations, which 
may also include coding under maximum-likelihood decoding 
(codewords are considered as points of an extended 
constellation). We now proceed to establish convexity 
properties of error rates in this generic setting. 
 
III. CONVEXITY OF SYMBOL ERROR RATES 
Convexity properties of symbol error rates for arbitrary 
constellations in the SNR and noise power have been 
established in [11][12] and are summarized below for 
completeness and comparison purpose. 
Theorem 1 (Theorem 1 and 2 in [11]): The SER eP  is a 
convex function of the SNR γ  for any constellation (which 
may also include coding) if 2n ≤ , 
 
2 2 0e ed P d P γ′′γ = >  (8) 
For 2n > , the following convexity properties hold: 
• eiP  is convex in the large SNR mode, 
 ( ) 2min,2 in n dγ ≥ +  (9) 
where min,id  is the minimum distance from is  to its 
decision region boundary, 
• eiP  is concave in the small SNR mode, 
 ( ) 2max,2 in n dγ ≤ −  (10) 
where max,id  is the maximum distance from is  to its 
decision region boundary, 
• there are an odd number of inflection points, 
0ci eiP Pγ γ= =′′ ′′ , in the intermediate SNR mode, 
 ( ) ( )2 2max, min,2 2i in n d n n d− ≤ γ ≤ +  (11) 
• the SER eP  is convex at high SNR, 
 ( ) 2min2n n dγ ≥ +  (12) 
where { }min min,min i id d=  is the minimum distance to 
decision region boundary in the constellation. 
 
Theorem 2 (Theorem 4 in [11]): Symbol error rates have 
the following convexity properties in the noise power 
2
0NP = σ , for any n and constellation geometry, 
• eiP  is concave in the large noise mode, 
 ( ) 12max, 2 2( 2)N iP d n n −≥ + − +  (13) 
• eiP  is convex in the small noise mode, 
 ( ) 12min, 2 2( 2)N iP d n n −≤ + + +  (14) 
• there are an odd number of inflection points for 
intermediate noise power, 
( ) ( )1 12 2max,min, 2 2( 2) 2 2( 2)N iid n n P d n n− −+ + + ≤ ≤ + − +  (15) 
• the SER eP  is convex in the small noise/high SNR mode, 
 ( ) 12min 2 2( 2)NP d n n −≤ + + +  (16) 
While the convexity properties above are important for many 
optimization problems, they do not lend any conclusions about 
convexity of the BER, since the latter is not directly related to 
eP  or eiP  in general. While, in some cases, the BER can be 
expressed as linear combination of eiP , there are positive and 
negative terms so that no conclusion about convexity can be 
made in this case either. On the other hand, the BER can be 
expressed as a positive linear combination of pairwise 
probabilities of error so that the convexity of the latter implies 
the convexity of the former. Thus, we study below the 
  
 
convexity property of the PEP, from which the convexity 
property of the BER will follow. 
 
IV. CONVEXITY OF PAIRWISE PROBABILITY OF ERROR 
In many cases, it is a pairwise error probability that is a key 
point in the analysis (e.g. for constructing a union bound and 
other performance metrics). Furthermore, it is also a basic 
building block for the BER in (7), so that we establish its 
convexity property first. 
 
Theorem 3:  
a) The pairwise error probability { }Pr i j→s s  is a convex 
function of the SNR at the high SNR region, 
2
min,( 2 ) / in n dγ ≥ + , for any n; 
b) for 1, 2n = , it is concave at the low SNR region, 
2
max,( 2 ) / ( )ij jn n d dγ ≤ + + , where ij i jd = −s s  is the 
distance between is  and js , and there is an odd number of 
inflection points, { }Pr 0i j ′′→ =s s , in the intermediate SNR 
mode, 
 
2 2
max, min,( 2 ) / ( ) ( 2 ) /ij j in n d d n n d+ + ≤ γ ≤ +  (17) 
c) for 2n > , the PEP is convex at the low SNR region, 
2
max,( 2 ) / ( )ij jn n d dγ ≤ − + , and there is an even number of 
inflection points in-between, 
2 2
max, min,( 2 ) / ( ) ( 2 ) /ij j in n d d n n d− + ≤ γ ≤ +  
Proof: See Appendix. 
 
We note that Theorem 3(a) is stronger than Theorem 1 at the 
high SNR region since the latter follows from the former but 
the opposite is not always true (as the other SNR ranges in 
Theorem 3 above indicate). Unlike the SER, the pairwise error 
probability can be concave at low SNR even for 1, 2n = . 
Since Theorem 3 holds for any constellation and bit 
mapping, it follows that the convexity property of the PEP at 
high SNR is a consequence of Gaussian noise density rather 
than particular modulation/coding used, where the latter 
determines only the SNR threshold. 
V. CONVEXITY OF THE BER AT HIGH SNR 
We are now in a position to establish the main result of this 
paper. 
Theorem 4: The BER is a convex function of the SNR, for 
any constellation and bit mapping, which may also include 
coding under maximum-likelihood decoding, at the high SNR 
regime, 
 
2
min( 2 ) /n n dγ ≥ + , (18) 
where { }min min,min i id d=  is the minimum distance to the 
boundary in the constellation. 
Proof: Using the relationship between the BER and the 
pairwise error probabilities in (7) and observing that a positive 
linear combination of convex functions is convex. Q.E.D. 
We remark that the condition in (18) guarantees the 
convexity of all PEP, BER and SER. In some cases (Gray 
encoding and when nearest neighbor errors dominate), the 
BER is approximated as 2SER/ log M , so that it inherits the 
same convexity properties as in Theorems 1 and 2 above. 
VI. CONVEXITY OF THE PEP AND BER IN NOISE POWER 
In a jammer optimization problem, it is convexity properties in 
noise power that are important [4]. Motivated by this fact, we 
study below convexity of the PEP and BER in the noise power. 
Theorem 5: The PEP { }Pr i j→s s  is a convex function of 
the noise power 20NP = σ , for any n, in the small noise/high 
SNR mode, 
 ( ) 12min, 2 2( 2)N iP d n n −≤ + + +  (19) 
and in the large noise/low SNR mode, 
 ( ) 12max,( ) 2 2( 2)N ij jP d d n n −≥ + + − +  (20) 
Proof: See Appendix. 
Based on this Theorem, the following convexity property of 
the BER is established. 
Corollary 5.1: For any constellation geometry and 
dimensionality, which may also include coding under ML 
decoding, the BER is a convex function of the noise power in 
the small noise/high SNR mode: 
 ( ) 12min 2 2( 2)NP d n n −≤ + + +  (21) 
where specifics of the constellation/code determine only the 
upper bound in (21). 
VII. REFERENCES 
[1] S. Boyd, L. Vandenberghe, Convex Optimization, Cambridge 
University Press, 2004. 
[2] A. Ben-Tal, A. Nemirovski, Lectrures on Modern Convex 
Optimization, MPS-SIAM Series on Optimization, Philadelphia, 
2001. 
[3] V. Kostina, S. Loyka, On Optimum Power Allocation for the 
V-BLAST, IEEE Transactions on Communications, v. 56, N. 6, 
pp. 999-1012, June 2008. 
[4] M. Azizoglu, Convexity Properties in Binary Detection 
Problems, IEEE Trans. Inform. Theory, v. 42, N. 4, pp. 1316-
1321, July 1996. 
[5] Y.-P. Lin, S.-M. Phoong, BER Minimized OFDM Systems 
With Channel Independent Precoders, IEEE Trans. Signal 
Processing, v.51, N.9, pp. 2369-2380, Sep. 2003. 
[6] C.C. Yeh, J.R. Barry, Adaptive Minimum Bit-Error Rate 
Equalization for Binary Signaling, IEEE Trans. 
Communications, v.48, N.7, pp. 1226-1235, Jul. 2000. 
[7] X. Wang, W.S. Lu, A. Antoniou, Constrained Minimum-BER 
Multiuser Detection, IEEE Trans. Signal Processing, v.48, N.10, 
pp. 2903-2909, Oct. 2000. 
[8] D.P. Palomar, J.M. Cioffi, M.A. Lagunas, Joint Tx-Rx 
Beamforming Design for Multicarrier MIMO Channels: A 
Unified Framework for Convex Optimization, IEEE Trans. 
Signal Processing, v.51, N.9, pp. 2381-2401, Sep. 2003. 
[9] J.M. Wozencraft, I.M. Jacobs, Principles of Communication 
Engineering, Wiley, 1965. 
  
 
[10] J.R. Barry, E.A. Lee, D.G. Messerschmitt, Digital 
Coomunications (3rd Ed.), Kluwer, Boston, 2004. 
[11] S. Loyka. V. Kostina, F. Gagnon, Symbol Error Rates of 
Maximum-Likelihood Detector: Convex/Concave Behavior and 
Applications, IEEE International Symposium on Information 
Theory (ISIT’07), June 2007, Nice, France. 
[12] S. Loyka, V. Kostina, F. Gagnon, Error Rates of the 
Maximum-Likelihood Detector for Arbitrary Constellations: 
Convex/Concave Behavior and Applications, IEEE Transactions 
on Information Theory, accepted, 2009. 
[13] A. Conti et al., Log-Concavity Property of the Error 
Probability with Application to Local Bounds for Wireless 
Communications, IEEE Trans. Information Theory, June 2009. 
[14] J. Lassing et al, Computation of the Exact Bit-Error Rate of 
Coherent M-ary PSK with gray Code Bit Mapping, IEEE Trans. 
Communications, v. 51, N. 11, pp. 1758-1760, Nov. 2003. 
VIII. APPENDIX 
Proof of Theorem 3: The pairwise probability of error 
{ }Prij i jP = →s s  can be presented as 
 ( )
j
ijP p dξΩ= ∫ x x  (22) 
where jΩ  is the decision region for js  when the reference 
frame is centered at is . Its second derivative in the SNR is 
 
2
2
( )
j
ij
d p
P d
d
ξ
Ω
′′ =
γ∫
x
x  (23) 
where the derivative is 
 ( )2/ 22 2/ 22( ) 1 e4 2
nd p f
d
ξ
−γγ 
=  γ pi 
xx x  (24) 
and ( ) ( )1 2( ) / /f t t t= − α γ − α γ , 1 2 0n nα = + > , 
2 12n nα = − < α . Consider three different cases. 
(i) If 2 1min, /id ≥ α γ , where min, min ( )j jid b=  is the 
minimum distance from the origin to the boundary of iΩ , then 
2( ) 0f ≥x j∀ ∈ Ωx  so that the integral in (23) is clearly 
positive since the integrand is non-negative everywhere in the 
integration region and positive in some parts of it. Fig. 1 
illustrates this case. This is a high SNR mode since 
2
1 min,/ idγ ≥ α . 
(ii) If 2 1max,( ) /ij jd d+ ≤ α γ  and 1, 2n = , where max, jd  is 
the maximum distance from the center of jΩ  to its boundary, 
then 2( ) 0f ≤x  j∀ ∈ Ωx  so that the integral in (23) is clearly 
negative and the result follows. Fig. 2 illustrates this case. This 
is a low-SNR mode since 21 max,/ ( )ij jd dγ ≤ α + . An odd 
number of inflection points in Theorem 3(b) follows from the 
continuity argument ( ijP′′  is a continuous function of the SNR). 
(iii) Part (c) follows from the same argument as in (ii). 
Q.E.D. 
 
Proof of Theorem 5: follows the same geometric technique 
as for Theorem 3. 2nd derivative of the PEP in the noise power 
can be expressed as 
 
2 2
2 2
( )
j
ij
N N
d P d p d
dP P
ξ
Ω
= ∫
x
x  (25) 
where 
 
( )
( ) ( ) ( )
2
2 2 22 *
2 4
*
1 2
1 2
( ) 1 1
e
4 2
,
2 2( 2),   2 2( 2)
N
n
P
N N N
N N
d p f
dP P P
f t t P t P
n n n n
−ξ  
=  
pi 
= − β − β
β = + + + β = + − +
x
x
x
 (26) 
 
and 1 2 0β > β > . Since ( )*f t  has the same structure as ( )f t  
in (24), the proof follows the same steps. In particular, if 
2
1min, Nid P≥ β , then 2 2/ 0 jNd p dPξ > ∀ ∈ Ωx  so that the 
integral in (25) is clearly positive. The other case is proved in a 
similar way. Q.E.D. 
 
 
min,id
1α
γ
iΩ
1x
2x
+ +
+
+
2( ) 0f >x
 
Fig. 1. Two-dimensional illustration of the problem geometry for 
Case 1. The decision region iΩ  is shaded. 2( )f x  has a sign as 
indicated by “+” and “-“. 
 
ijd
1α
γ
iΩ
1x
2x
−
−
−
−
+
+
+
+
2( ) 0f <x2( ) 0f >x jΩ +
max, jd
 
Fig. 2. Two-dimentional illustration of the problem geometry for 
Case 2.  


Bit Error Rate is Convex at High SNR

Abstract

Similar works

Full text

Available Versions

Repository for Publications and Research Data

Caltech Authors - Main

Caltech Authors - Main