CASP 15

We expect to run the experiment on the usual CASP timetable: targets will be available from approximately the beginning of May 2022 through August, and the CASP conference will be in December.

The core of CASP remains the same: Double-blind testing with independent assessment against experiment to establish and advance the state of art in modeling proteins and protein-complexes.

CASP categories will change: Existing categories will be modified; new ones added (tentatively RNA structure, protein-organic ligand complexes, and conformational variability); and it is proposed that some of old ones (refinement, single protein related accuracy estimates, and contacts/distances) will be dropped. Each of these changes is outlined below. CASP has very limited resources to make these changes, and where appropriate we will explore collaborations.

Single proteins and domains: This category has been the core of CASP and will continue. The AlphaFold2 and RoseTTAFold methods set a very high bar, but it is important that there is further objective testing and analysis. We expect there will be a number of new competitive methods. Assessment is likely to focus more on the conditions under which poorer structures or parts of structures are obtained and whether any modeling methods produce more accurate models in those circumstances.

Assessment will place more emphasis on side chain accuracy. Related to that, as outlined further below, one proposed change is that participant provided accuracy estimates will be required to use lDDT units, not Angstroms as previously, and should be provided for all atoms, not just per residue.

---

Some news:

CASP15’s success depends on generosity of the experimental community in providing targets as ground truth against which to assess the computation methods. Over the years more than 150 structure determination groups have provided over 1100 targets for CASP challenges. For CASP15, we are requesting submission of all types of experimental structures determined by X-ray crystallography, cryo-electron microscopy and NMR as potential targets, but are particularly interested in the following:

1. High resolution structures of single proteins. Because of the high accuracy of the new computational methods, it is becoming difficult to distinguish experimental error from computational error in low resolution structures.

2. Structures with few or no known sequence relatives. Consistently accurate computed structures for this class of target requires methods that do not depend on evolutionary relationships.

3. Protein complexes. Deep learning methods already show increased performance in this area, and a range of complexes is needed to establish exactly how powerful these are. Assessment will be in partnership with CAPRI, as in other recent CASPs.

4. RNA structures, RNA complexes, and protein RNA complexes. Many more RNA structures are now being determined experimentally, opening this area for more extensive rigorous assessment.

5. Proteins with clearly determined alternative conformations. An obvious extension beyond single protein structures is the calculation of ensembles of conformations. The new computational methods are already being applied to this problem, but there is a paucity of definitive experimental data to assess these against, which may limit this category.

6. Protein-organic ligand complexes. Deep learning methods are also being applied to these structures. We are exploring including this category in CASP15. A major challenge is obtaining suitable targets.

We also plan to include modeling assisted by sparse experimental data, in collaboration with experimental groups in NMR, SAXS, and crosslinking mass spectrometry. For that, protein material is needed (this is not expected for most targets, but if available, it would be much appreciated!).

So, if you have suitable targets in any if these areas, we would very much appreciate you getting in touch by replying to this email or writing to [email protected] or suggesting your target directly through the CASP15 target entry page: http://www.predictioncenter.org/casp15/targets_submission.cgi.

---

CASP15 registration will open on April 4, server testing on April 18, and the first target will be released on May 2, 2022. The experiment will run May-August and culminate with a conference in Europe in December. Please check the CASP15 web page (https://predictioncenter.org/casp15/) for more details on the upcoming experiment.

Given the tremendous progress of the last two experiments and the clear potential for further progress, especially in the assembly category, we look forward to another exciting experiment. Success will primarily depend on two things: your participation and the identification of targets, particularly for complexes, RNA , protein-ligand, and conformational ensemble prediction. Please help us find suitable targets - talk to your colleagues now.

CASP organizers wish you a successful prediction season!

---

How is CASP distributed?
Didn't this project used to grab some and run it here?
Do you make an account at CASP and work directly from their server?

How is CASP distributed?
Didn't this project used to grab some and run it here?
Do you make an account at CASP and work directly from their server?

No, you can't run CASP work directly. You have to be a research group.
In the past editions of CASP, Rosetta@home released wus clearly related to the competition with, for example, "_casp_" label in the name
I don't know if in this edition it will be.
I hope.

---

Dear CASPers,

CASP15 registration will open on April 4, server testing on April 18, and the first target will be released on May 2, 2022. The experiment will run May-August and culminate with a conference in Europe in December. Please check the CASP15 web page (https://predictioncenter.org/casp15/) for more details on the upcoming experiment.

Given the tremendous progress of the last two experiments and the clear potential for further progress, especially in the assembly category, we look forward to another exciting experiment. Success will primarily depend on two things: your participation and the identification of targets, particularly for complexes, RNA , protein-ligand, and conformational ensemble prediction. Please help us find suitable targets - talk to your colleagues now.

CASP organizers wish you a successful prediction season!

---

Dear method developers,

We slightly modified format of CASP15 TS prediction by requesting atom-level confidence scores in the percentage form (0.00-100.00) instead of originally suggested fraction form (0.00-1.00). This way you will be able to provide increased precision estimates within the PDB format allowing two decimal places in the B-factor field.

CASP organizers

---

Dear CASP15 server predictors,

We have started the first round of checking connectivity and correctness of prediction format for servers registered for tertiary structure prediction. We sent a test target T1031 to all registered servers. If you had registered a TS server, but have not received at least one request from us please check the Server Queries link from the main CASP15 web page to see the status of the queries to your server. In case you notice any problems, please check your registration settings and contact us if in doubt. Also, if you do not see your prediction(s) as accepted in our server status page (link Server Prediction Status) or through the Model Viewer /My CASP15 Profile pages, then either your prediction did not reach us or was rejected due to unregistered with CASP submission email or format issues.

A request to EMAIL-server curators. Please, make sure that your servers reply to our distribution server [email protected] immediately after you have received a query. Please put the following text into the Subject of the email: "T1031 - query received by MY_SERVER". Failing to do this may affect the timely addressing of unexpected connectivity failures.

We will continue testing TS servers for two more days.

---

Today we sent a request to the registered RNA servers for modeling a test target R1099.

Please make sure your server sees it and generates prediction(s) as expected. In case of any issues - get in touch with us at [email protected].

---

We want to inform you that AlphaFold2 and RosettaFOLD models will be available to all human predictors in CASP15 immediately after closing server prediction (typically three days after the target release). The models will be generated by three participating CASP15 servers (af2-standard, af2-multimer and Baker-server) and posted on the Prediction Center website at https://predictioncenter.org/download_area/CASP15/server_predictions. The RosettaFOLD models will be provided directly by David Baker's group, while AlphaFold2 models by Arne Elofsson's group, who will run standard monomeric and multimeric AFv2.2 protocols suggested by Deepmind.

Additionally, the Elofsson group will post a larger set of alphafold-standard and alphafold-multimer models (25 normally) and the corresponding multiple-sequence alignments (MSAs) for all CASP15 targets at http://duffman.it.liu.se/casp15/.

---

We have cleaned our databases and the file system after the server test runs, and are ready now to roll CASP15 modeling experiment. The prediction season starts Monday, May 2 at 9 am PDT.

Eight targets are planned for the first week:
Monday: 1 TS target
Tuesday: 1 TS target
Wednesday: 1 TS target and 2 RNA targets
Thursday: 2 oligomeric targets (both selected for CAPRI)
Friday: 1 TS target with ligand

Plans for the second week will be announced next Saturday.

---

Does anyone know if we'll be running any of these tasks?

The first ones are out today it seems:
https://predictioncenter.org/news.cgi#421

---

Does anyone know if we'll be running any of these tasks?

The first ones are out today it seems:
https://predictioncenter.org/news.cgi#421

Nothing on the Robetta server so far.
https://robetta.bakerlab.org/queue.php?id=&username=casp&target=&seq=

---

CASP15: week 2 (May 9-15) (2022-5-7)
1. In week 1, we released 11 targets in different categories (including two subunits of hetero-complex H1106) and collected over 1,300 models.

2. Some clarifications about the targets released last Friday.

Target H1111 is a target for assembly prediction only. We did not release its individual subunits for prediction as two of them, YscX and YscY, had been released as targets T1106s1 and T1106s2 forming H1106, and the third, YscV, is a domain with known structure (PDB ID 7alw). The challenge here is to model the 9:9:9 complex of YscX:YscY:YscV. Please stick to the target template posted at https://predictioncenter.org/casp15/target.cgi?id=39&view=all.

Target T1105v1 is a target for server/ligand prediction only. As such, we are expecting protein receptor predictions from servers only (T1105v1 sequence is almost identical to that of T1105, with one-point mutation). The challenge for human-expert groups is to predict this protein mutant in complex with the queuosine ligand.

Targets R1107 and R1108 are targets for RNA prediction. These targets have been also released separately as RNA-puzzles PZ35 and PZ36. Note that future CASP RNA targets will be released through the CASP website only, but will be fully shared with the RNA-puzzles organization. The RNA-puzzles organizers posted the following information about the targets: == The structures have been crystallized and solved using the U1A co-crystallization technique (a loop contains the U1A recognition sequence). In these puzzles, the RNAs form a dimer structure. Please predict each of these two puzzles as dimers. ==

3. Reminding that we are releasing AlphaFold2 and RosettaFOLD models through https://predictioncenter.org/download_area/CASP15/server_predictions. The models are posted at this address two hours after closing server prediction on the target (i.e. at 2pm PDT).

4. Plans for week 2 (subject to change)
Monday: 1 target (TS)
Tuesday: 1 target (oligo/CAPRI)
Wednesday: 1 target (heteromer/no_CAPRI, subunits, ligands)
Thursday: 1 target (oligo/CAPRI)
Friday: 3 targets including 2 RNA (one with ligand), and 1 TS (with ligand)

---

CASP15: week 3 (May 16-22)
Next week we plan to release ten targets, including 4 or 5 multimeric (also CAPRI), 2 with ligands, and 2 RNA (on Friday).

---

CASP15: week 4 (May 23-29)
1. RNA update.
At the request of predictors, we started posting server models submitted on RNA targets at predictioncenter.org/download_area/CASP15/server_predictions.
Models for R1107, R1108, R1116 and R1117 were posted yesterday (May 20); models for currently active server targets (R1126 and R1128) and all future RNA targets will be posted at 2pm PDT on the server expiration date. Currently, there are 9 server groups submitting RNA predictions.

2. Next week we will see first targets expiring for human-expert prediction, including a few multimeric targets. This means the time for QA prediction is coming. The QA prediction window for upcoming targets is reported through the Target List page.

3. Plans for week 4 (subject to change).
Monday: 1 target - H1129 (heterodimer/CAPRI, one subunit for TS)
Tuesday: 2 targets - T1130 (TS), T1131 (TS)
Wednesday: 2 targets - T1132 (homomer/CAPRI); , T1133 (TS)
Thursday: 1 target - H1134 (heterodimer/CAPRI, two subunits for TS)
Friday: 3 targets - H1135 (heteromultimer/CAPRI with ligands); R1136v1, R1136v2 (RNA, apo and holo conformations)

---

CASP15: week 5 (May 30 - June 5)
1. This week we will have 9 TS targets spread Mon-Thu, one very large multimeric target (CAPRI) on Friday, and two variants of the same RNA (young and mature) on Friday.

2. Attention QA predictors: today we erroneously sent a duplicated request for estimating accuracy of H1111 models (first at 9:35 and then at 10 am). This is a big target - so please make sure your server does not do the same work twice.

---

CASP15: week 6 (June 6-12) (2022-6-5)
1. This week Monday through Friday, we will have one target per day for assembly prediction. All these targets will also be part of CAPRI. The targets feature an CNPase (structure known) and five different nanobodies. The challenge is predicting the binding site and the complex conformation. These will be assembly-only targets, i.e. no TS prediction for subcomplexes will be requested.

2. Additionally we will have five TS-designated targets spread over the week.

3. As usually, we will also have an RNA target on Friday.

---

1. With six weeks of prediction behind us, we have passed the CASP15 equator.

After the outstanding performance of AphaFold2 on single-domain proteins in CASP14, we expected a shift of interest in the prediction community from single proteins to complexes. And that indeed happened. In CASP15/CAPRI, we have 81 groups participating in the assembly category compared to 49 in CASP14. That's a 65% increase!

The participation in the single-protein prediction continues to be very strong with 127 groups sending models. This represents only 15% drop from the previous CASP (149 groups in CASP14). At the same time, we are happy to welcome new groups coming to the field, mainly from the AI community. A teaser: first evaluation results show that some single-protein targets still constitute significant predictive challenge and that the accuracy of AlphaFold2 models can be surpassed. Thus, the single-protein prediction category is very much alive and still worth investing your time.

The accuracy assessment category took a big dip in participation with the switch of the prediction scope from monomeric to multimeric targets. The refurbished category drew attention of 26 groups specializing in assessing accuracy of interfaces and overall multimeric fold compared to 73 groups specializing in accuracy of monomers in CASP14.

Introduction of two new categories, RNA prediction and Ligand prediction, got a great response from the community with 35 and 31 groups participating, correspondingly. At this time, we already secured more than 10 targets in each of these categories allowing us to carry out a meaningful evaluation of the results. We are very eager to determine the state of the art in these prediction areas.

2. Next week plans.

Assembly targets (also CAPRI) will be released on Monday, Tuesday and Friday.

We will have four single-protein targets Mon-Thu, additionally to subunits of the assembly targets. The Wednesday target is on a larger side with ~1400 res. The Tuesday target is a monomer, but we will formally release it as a dimer to allow proper placement of the ligand for those participating in this prediction category.

An RNA target will be released not on Friday as it became usual in this CASP, but on Thursday. Please note that the target has some heterogeneity and you might consider submitting different models (out of 5 permitted) representing alternative conformations.

---

CASP15: week 8 (June 20-26) (2022-6-18)
Next week we will release a series of 5 targets for ligand prediction (apo and one-point mutant holo with 4 different ligands). Even though the Monday target in this series is an apo structure, we encourage dedicated ligand prediction groups to model it as well to be able later to compare unbound and bound-state models.

On Monday, Wednesday and Thursday will have one additional monomeric target for prediction.

On Tuesday, we will release two small multimeric targets with similar sequences but different conformations due to the crystallization conditions (also selected for CAPRI).

On Friday, we will have a huge assembly target (also CAPRI). The target is a 6-unit complex of around 7000 residues total. Five out of six units have 100% hits to the PDB, and the sixth unit has high sequence identity. However the authors warned us that some subunits of the published structure from another group have serious modeling errors and that their own structure cover more residues, essentially solving three new complex components (so - go beyond what you will see in the published structure). We will post links to the related publication and cryo-EM map in the next-week communique.

---