In this paper we prove that there exists a smooth classical solution to the
HJB equation for a large class of constrained problems with utility functions
that are not necessarily differentiable or strictly concave. The value function
is smooth if admissible controls satisfy an integrability condition or if it is
continuous on the closure of its domain. The key idea is to work on the dual
control problem and the dual HJB equation. We construct a smooth, strictly
convex solution to the dual HJB equation and show that its conjugate function
is a smooth, strictly concave solution to the primal HJB equation satisfying
the terminal and boundary conditions.Comment: 18 page