How to do an evaluation: pitfalls and traps

The recent literature is replete with papers evaluating computational tools (often those operating on 3D structures) for their performance in a certain set of tasks. Most commonly these papers compare a number of docking tools for their performance in cognate re-docking (pose prediction) and/or virtual screening. Related papers have been published on ligand-based tools: pose prediction by conformer generators and virtual screening using a variety of ligand-based approaches. The reliability of these comparisons is critically affected by a number of factors usually ignored by the authors, including bias in the datasets used in virtual screening, the metrics used to assess performance in virtual screening and pose prediction and errors in crystal structures used

How to do an evaluation: pitfalls and traps

The recent literature is replete with papers evaluating computational tools (often those operating on 3D structures) for their performance in a certain set of tasks. Most commonly these papers compare a number of docking tools for their performance in cognate re-docking (pose prediction) and/or virtual screening. Related papers have been published on ligand-based tools: pose prediction by conformer generators and virtual screening using a variety of ligand-based approaches. The reliability of these comparisons is critically affected by a number of factors usually ignored by the authors, including bias in the datasets used in virtual screening, the metrics used to assess performance in virtual screening and pose prediction and errors in crystal structures used

In vitro and in vivo activity of novel small-molecule inhibitors targeting the pleckstrin homology domain of protein kinase B/AKT

The phosphatidylinositol 3-kinase/AKT signaling pathway plays a critical role in activating survival and antiapoptotic pathways within cancer cells. Several studies have shown that this pathway is constitutively activated in many different cancer types. The goal of this study was to discover novel compounds that bind to the pleckstrin homology (PH) domain of AKT, thereby inhibiting AKT activation. Using proprietary docking software, 22 potential PH domain inhibitors were identified. Surface plasmon resonance spectroscopy was used to measure the binding of the compounds to the expressed PH domain of AKT followed by an in vitro activity screen in Panc-1 and MiaPaCa-2 pancreatic cancer cell lines. We identified a novel chemical scaffold in several of the compounds that binds selectively to the PH domain of AKT, inducing a decrease in AKT activation and causing apoptosis at low micromolar concentrations. Structural modifications of the scaffold led to compounds with enhanced inhibitory activity in cells. One compound, 4-dodecyl-N-(1,3,4-thiadiazol-2-yl) benzenesulfonamide, inhibited AKT and its downstream targets in cells as well as in pancreatic cancer cell xenografts in immunocompromised mice; it also exhibited good antitumor activity. In summary, a pharmacophore for PH domain inhibitors targeting AKT function was developed. Computer-aided modeling, synthesis, and testing produced novel AKT PH domain inhibitors that exhibit promising preclinical properties. ©2009 American Association for Cancer Research

Conformer Generation with OMEGA: Algorithm and Validation Using High Quality Structures from the Protein Databank and Cambridge Structural Database

Here, we present the algorithm and validation for OMEGA, a systematic, knowledge-based conformer generator. The algorithm consists of three phases: assembly of an initial 3D structure from a library of fragments; exhaustive enumeration of all rotatable torsions using values drawn from a knowledge-based list of angles, thereby generating a large set of conformations; and sampling of this set by geometric and energy criteria. Validation of conformer generators like OMEGA has often been undertaken by comparing computed conformer sets to experimental molecular conformations from crystallography, usually from the Protein Databank (PDB). Such an approach is fraught with difficulty due to the systematic problems with small molecule structures in the PDB. Methods are presented to identify a diverse set of small molecule structures from cocomplexes in the PDB that has maximal reliability. A challenging set of 197 high quality, carefully selected ligand structures from well-solved models was obtained using these methods. This set will provide a sound basis for comparison and validation of conformer generators in the future. Validation results from this set are compared to the results using structures of a set of druglike molecules extracted from the Cambridge Structural Database (CSD). OMEGA is found to perform very well in reproducing the crystallographic conformations from both these data sets using two complementary metrics of success