You are here

2015/2016 Model Challenge

Goals

  • Establish a benchmark set of 3DEM maps in the 3.0-4.5 Å resolution range, where significant growth in the number of maps is anticipated over the next few years and where a number of technical challenges exist to map interpretation and fitting

  • Encourage developers of modelling software packages and biological end users to analyze these maps and present modelling results with the best practice

  • Evolve criteria for evaluation and validation of 3DEM map-derived models

  • Compare and contrast the various modelling and analysis approaches in a positive spirit

Model Challenge Registrants: 39
Model Challenge Submissions: 106

Model Committee

Paul Adams (Chair), Axel Brunger, Randy Read, Torsten Schwede, Maya Topf, Gerard Kleywegt, Cathy Lawson, Wah Chiu

Model Challenge How to Participate

All members of the Scientific Community--at all levels of experience--are invited to participate as Challengers, and/or as Assessors.

Challengers will create and submit their own atomic coordinate models of one or more challenge targets using the supplied target maps.

There are four modelling categories:

  1. Optimize the current cryoEM model for the target, improving fit to the cryo-EM map density and/or model stereochemistry
  2. Fit known related cryoEM, crystallographic, or comparative models to the cryo-EM map, optionally followed by model optimization
  3. Ab initio model building, without reference to any existing cryo-EM or crystallographic models related to the cryo-EM map
  4. Any other method that seeks to interpret, or create a model based on, the cryo-EM map

For each submission, challengers will provide their final model, target map (if modified from original), and basic information about their modelling process. Researchers are expected to disclose all of the prior information used in their method. For full challenge rules and additional information see the model challenge guide.

Assessors will contribute to the challenge assessment phase.  Following a short initial review period by the model committee, challenge data and files will be made publicly available (entry authorship suppressed) for anyone to assess. The intention is to enable comparisons of the various modelling methodologies available and their options in a positive spirit.  Assessors will be able to share their results via a planned workshop (~Fall 2016) as well as manuscript submissions to a Journal special issue.

****All participants--challengers and assessors--are required to register****

Model Challenge Timeline

2015
DEVELOPMENT PHASE
February-June Model Committee meets monthly to discuss possible scope of challenge
June 20/21 Face-to-face meeting to define challenge targets, goals, and parameters
July-September Finalization of challenge formulation. Requests to map contributors to provide any missing data (e.g., half-maps, unfiltered, unsharpened, unmasked maps)
2015/2016
CHALLENGE PHASE
October 14 Pre-Challenge Announcement, Challenger and Assessor Registration Opens
January-June 17 Challenge Site Open for Model Entry Submissions
June 17 21:00 UTC
Model entry submission window closes
2016/2017 ASSESSMENT PHASE

June-November

Challenge Data review by the Model Committee
December-May
Challenge Data review opened to volunteer assessors (blind analysis--entry authorship suppressed)
May 29 Face-to-Face Meeting to discuss preliminary outcomes of assessments
mid July Assessors provide written reports on their results
mid July Reports on Models sent to Challengers; Full workflow info provided to all challengers and assessors
Aug 15 Feedback from Challengers on their reports
Sept 1 Deadline for Assessors to update their reports
Oct 5-8 Challenges Meeting (Maps and Models)
Post-workshop Challenge Writeups (multiple articles) for a Journal Special Issue

Model Challenge Targets

The eight targets chosen by the model challenge committee are shown below.  All are based on recently reported 3DEM structures.  The target order is according to unique molecular weight, from 19 kDa (TMV) at left to 2500 kDa (ribosome) at right. For challenge rules and additional information about each target see the model challenge guide.  

April 28: Corrected symmetry center info and BIOMT matrix files for the proteasome and beta-galactosidase targets, see the model challenge guide.

May 3: Corrected BIOMT matrix file containing Icosahedral transformations for Brome Mosaic Virus, see the model challenge guide.

Target Map Download: You can use this rsync script to download target maps from one of three wwPDB ftp sites. Alternately, you can download individual maps from EMDB atlas pages (click on EMDB entry link in the table below, select "download" tab).

  1. Tobacco Mosaic Virus 2. T20S Proteasome 3. GroEL 4. TRPV1 Channel 5. Brome Mosaic Virus 6. β-Galactosidase 7. γ-Secretase 8. 70S Ribosome

target

Map "A" EMDB entry
Primary Citation
Reported Resolution (Å)
EMD-2842
Fromm et al
3.3
EMD-5623
Li et al
3.3
EMD-6422
--
4.1
EMD-5778
Liao et al
3.3
EMD-6000
Wang et al
3.8
EMD-5995
Bartesaghi et al
3.2
EMD-2677
Lu et al
4.5
EMD-2847
Fischer et al
2.9
Map "B" EMDB entry
Primary Citation
Reported Resolution (Å)
  EMD-6287
Campbell et al
2.8
      EMD-2984
Bartesaghi et al
2.2

EMD-3061
Bai et al
3.4

EMD-6316
Li et al
3.6

Imposed Map Symmetry

Helical

Dihedral (D7)

Dihedral (D7)

 Cyclic (C4) Icosahedral (I)  Dihedral (D2) None (C1) None (C1)
 Sample MW (MDa) --  0.7 0.8  0.3  4.6  0.47  0.17 2.5
 Unique MW (kDa) 19  50 56  80 80  120  170 2500
Map Contributors (Thank You!)

Simon Fromm, Carsten Sachse

Jean-Paul Armache, Yifan Cheng

Melody Campbell, Bridget Carragher

Soung-Hun Roh, Corey Hryc, Wah Chiu Jean-Paul Armache, Maofu Liao, Yifan Cheng Zhao Wang, Wah Chiu Alberto Bartesaghi, Sriram Subramaniam Xiaochen Bai, Sjors Scheres

Niels Fischer, Holger Stark

Wen Li, Zheng Liu, Joachim Frank

EMDataBank Validation Challenges are supported by NIH National Institute of General Medical Sciences

Please send your challenge questions, comments and feedback to challenges@emdatabank.org

Theme by Danetsoft and Danang Probo Sayekti inspired by Maksimer