Deciding Isomorphy using Dehn fillings, the splitting case

We solve Dehn's isomorphism problem for virtually torsion-free relatively hyperbolic groups with nilpotent parabolic subgroups. We do so by reducing the isomorphism problem to three algorithmic problems in the parabolic subgroups, namely the isomorphism problem, separation of torsion (in their outer automorphism groups) by congruences, and the mixed Whitehead problem, an automorphism group orbit problem. The first step of the reduction is to compute canonical JSJ decompositions. Dehn fillings and the given solutions of the algorithmic problems in the parabolic groups are then used to decide if the graphs of groups have isomorphic vertex groups and, if so, whether a global isomorphism can be assembled. For the class of finitely generated nilpotent groups, we give solutions to these algorithmic problems by using the arithmetic nature of these groups and of their automorphism groups.Comment: 76 pages. This version incorporates referee comments and corrections. The main changes to the previous version are a better treatment of the algorithmic recognition and presentation of virtually cyclic subgroups and a new proof of a rigidity criterion obtained by passing to a torsion-free finite index subgroup. The previous proof relied on an incorrect result. To appear in Inventiones Mathematica

Time complexity of the conjugacy problem in relatively hyperbolic groups

If u and v are two conjugate elements of a hyperbolic group then the length of a shortest conjugating element for u and v can be bounded by a linear function of the sum of their lengths, as was proved by Lysenok in [Some algorithmic properties of hyperbolic groups, Izv. Akad. Nauk SSSR Ser. Mat.53(4) (1989) 814-832, 912]. Bridson and Haefliger showed in [Metrics Spaces of Non-Positive Curvature (Springer-Verlag, Berlin, 1999)] that in a hyperbolic group the conjugacy problem can be solved in polynomial time. We extend these results to relatively hyperbolic groups. In particular, we show that both the conjugacy problem and the conjugacy search problem can be solved in polynomial time in a relatively hyperbolic group, whenever the corresponding problem can be solved in polynomial time in each parabolic subgroup. We also prove that if u and v are two conjugate hyperbolic elements of a relatively hyperbolic group then the length of a shortest conjugating element for u and v is linear in terms of their lengths