3 research outputs found
CSAR 2014: A Benchmark Exercise Using Unpublished Data from Pharma
The 2014 CSAR Benchmark
Exercise was the last community-wide exercise
that was conducted by the group at the University of Michigan, Ann
Arbor. For this event, GlaxoSmithKline (GSK) donated unpublished crystal
structures and affinity data from in-house projects. Three targets
were used: tRNA (m1G37) methyltransferase (TrmD), Spleen Tyrosine
Kinase (SYK), and Factor Xa (FXa). A particularly strong feature of
the GSK data is its large size, which lends greater statistical significance
to comparisons between different methods. In Phase 1 of the CSAR 2014
Exercise, participants were given several protein–ligand complexes
and asked to identify the one near-native pose from among 200 decoys
provided by CSAR. Though decoys were requested by the community, we
found that they complicated our analysis. We could not discern whether
poor predictions were failures of the chosen method or an incompatibility
between the participant’s method and the setup protocol we
used. This problem is inherent to decoys, and we strongly advise against
their use. In Phase 2, participants had to dock and rank/score a set
of small molecules given only the SMILES strings of the ligands and
a protein structure with a different ligand bound. Overall, docking
was a success for most participants, much better in Phase 2 than in
Phase 1. However, scoring was a greater challenge. No particular approach
to docking and scoring had an edge, and successful methods included
empirical, knowledge-based, machine-learning, shape-fitting, and even
those with solvation and entropy terms. Several groups were successful
in ranking TrmD and/or SYK, but ranking FXa ligands was intractable
for all participants. Methods that were able to dock well across all
submitted systems include MDock, Glide-XP, PLANTS, Wilma, Gold, SMINA, Glide-XP/PELE, FlexX, and MedusaDock. In fact, the submission based on Glide-XP/PELE cross-docked
all ligands to many crystal structures, and it was particularly impressive
to see success across an ensemble of protein structures for multiple
targets. For scoring/ranking, submissions that showed statistically
significant achievement include MDock using
ITScore, with a flexible-ligand term, SMINA using Autodock-Vina,, FlexX using HYDE, and Glide-XP using XP DockScore with and without ROCS shape similarity. Of course, these
results are for only three protein targets, and many more systems
need to be investigated to truly identify which approaches are more
successful than others. Furthermore, our exercise is not a competition
CSAR 2014: A Benchmark Exercise Using Unpublished Data from Pharma
The 2014 CSAR Benchmark
Exercise was the last community-wide exercise
that was conducted by the group at the University of Michigan, Ann
Arbor. For this event, GlaxoSmithKline (GSK) donated unpublished crystal
structures and affinity data from in-house projects. Three targets
were used: tRNA (m1G37) methyltransferase (TrmD), Spleen Tyrosine
Kinase (SYK), and Factor Xa (FXa). A particularly strong feature of
the GSK data is its large size, which lends greater statistical significance
to comparisons between different methods. In Phase 1 of the CSAR 2014
Exercise, participants were given several protein–ligand complexes
and asked to identify the one near-native pose from among 200 decoys
provided by CSAR. Though decoys were requested by the community, we
found that they complicated our analysis. We could not discern whether
poor predictions were failures of the chosen method or an incompatibility
between the participant’s method and the setup protocol we
used. This problem is inherent to decoys, and we strongly advise against
their use. In Phase 2, participants had to dock and rank/score a set
of small molecules given only the SMILES strings of the ligands and
a protein structure with a different ligand bound. Overall, docking
was a success for most participants, much better in Phase 2 than in
Phase 1. However, scoring was a greater challenge. No particular approach
to docking and scoring had an edge, and successful methods included
empirical, knowledge-based, machine-learning, shape-fitting, and even
those with solvation and entropy terms. Several groups were successful
in ranking TrmD and/or SYK, but ranking FXa ligands was intractable
for all participants. Methods that were able to dock well across all
submitted systems include MDock, Glide-XP, PLANTS, Wilma, Gold, SMINA, Glide-XP/PELE, FlexX, and MedusaDock. In fact, the submission based on Glide-XP/PELE cross-docked
all ligands to many crystal structures, and it was particularly impressive
to see success across an ensemble of protein structures for multiple
targets. For scoring/ranking, submissions that showed statistically
significant achievement include MDock using
ITScore, with a flexible-ligand term, SMINA using Autodock-Vina,, FlexX using HYDE, and Glide-XP using XP DockScore with and without ROCS shape similarity. Of course, these
results are for only three protein targets, and many more systems
need to be investigated to truly identify which approaches are more
successful than others. Furthermore, our exercise is not a competition
CSAR 2014: A Benchmark Exercise Using Unpublished Data from Pharma
The 2014 CSAR Benchmark
Exercise was the last community-wide exercise
that was conducted by the group at the University of Michigan, Ann
Arbor. For this event, GlaxoSmithKline (GSK) donated unpublished crystal
structures and affinity data from in-house projects. Three targets
were used: tRNA (m1G37) methyltransferase (TrmD), Spleen Tyrosine
Kinase (SYK), and Factor Xa (FXa). A particularly strong feature of
the GSK data is its large size, which lends greater statistical significance
to comparisons between different methods. In Phase 1 of the CSAR 2014
Exercise, participants were given several protein–ligand complexes
and asked to identify the one near-native pose from among 200 decoys
provided by CSAR. Though decoys were requested by the community, we
found that they complicated our analysis. We could not discern whether
poor predictions were failures of the chosen method or an incompatibility
between the participant’s method and the setup protocol we
used. This problem is inherent to decoys, and we strongly advise against
their use. In Phase 2, participants had to dock and rank/score a set
of small molecules given only the SMILES strings of the ligands and
a protein structure with a different ligand bound. Overall, docking
was a success for most participants, much better in Phase 2 than in
Phase 1. However, scoring was a greater challenge. No particular approach
to docking and scoring had an edge, and successful methods included
empirical, knowledge-based, machine-learning, shape-fitting, and even
those with solvation and entropy terms. Several groups were successful
in ranking TrmD and/or SYK, but ranking FXa ligands was intractable
for all participants. Methods that were able to dock well across all
submitted systems include MDock, Glide-XP, PLANTS, Wilma, Gold, SMINA, Glide-XP/PELE, FlexX, and MedusaDock. In fact, the submission based on Glide-XP/PELE cross-docked
all ligands to many crystal structures, and it was particularly impressive
to see success across an ensemble of protein structures for multiple
targets. For scoring/ranking, submissions that showed statistically
significant achievement include MDock using
ITScore, with a flexible-ligand term, SMINA using Autodock-Vina,, FlexX using HYDE, and Glide-XP using XP DockScore with and without ROCS shape similarity. Of course, these
results are for only three protein targets, and many more systems
need to be investigated to truly identify which approaches are more
successful than others. Furthermore, our exercise is not a competition