The long-anticipated Judging Systems Trials, a brain child of the FEI Dressage Task Force, took place in the Deutsche Bank Dressage Stadium in Aachen, Germany, on September 7-9, 2009.
This well-organized event, which was open to an audience and press, was a first concrete step forwards in the modernization of the dressage judging system.
A large number of especially Belgian and Dutch riders volunteered to compete their experienced and upcoming horses in this event. The trial was not a competition and the riders will not receive their score sheets. However, for them it was an exclusive once-in-a-lifetime opportunity to show in the famous Aachen ring. Unfortunately only one Olympic rider turned up, Imke Schellekens-Bartels.
The actual testing of the different systems happened on the first two days of the three-day event and it was separated into five trials. The entire trials were discussed on the third day amongst judges and Task Force Members. The actual results of the trials are not directly known as they are sent to Switzerland, where maths-wizard David Stickland will calculate the results of each trial and estimate the level of improvement of each system in comparison to the old format.
Half Marks and Bonus-Malus System To Improve the Sport
On Monday the first two trials tested various methods of judging and scoring dressage. Trial number one was a Grand Prix level test with the judges at different positions: five in the standard position (E, H, C, M, B), whose marks were used as reference scores, and two on the short side by A. They tested the effect of increasing the judges' panel from five to seven judges and investigated the effect of dropping the high and low score. Afterwards, four judges were asked to score each two components of the training scale: impulsion and rhythm, contact and submissiveness, collection and suppleness, and precision and straightness.
German judge Peter Holler commented that the component judging "very much corresponds with what I would do normally," but it still was "a different way of judging in the brain." Component judging posed the problem that some parts can not be separated from another or that "sometimes a horse can lose the rhythm but still have good impulsion." The advantage of this style of judging was that "you don't have to look so much at the mistakes. It is more like judging young horses," Holler confirmed. An issue the judges also faced was the intertwining of the various parts. What is submission? Is it simply throughness, or also impulsion and contact?
On Monday afternoon ten judges were seated round the show ring. Five of them gave the conventional marks as reference and five were allowed to use half marks. The idea of half mark judging was initially discussed by Wayne Channon on Eurodressage.com and accepted by the FEI Dressage Task Force to be tested at this trial following a request from the riders. Very often judges tend towards a "small" seven or a "big" six but in the end this problem could cause a 10% point difference in a test, because the panel is restricted to using full marks. The half-mark system would encourage judges to raise scores.
Task Force member Robert Dover stated that "it gives us a chance to get closer and closer to scores that are commensurate with the words of the judges. It will also be easier for the spectators." As the sport gets more challenging and top riders are scoring much closer together by riding "on the edge," the decimal system is probably the future. "One hundred of a second or of a point can make the difference," Dover added. Though it took some time to get adapted to the half mark system, the judges were very enthusiastic about it and considered it easy to apply to the current system at all levels.
On Monday evening, a closed session took place in which O-judge Katrina Wüst gave a very detailed presentation on how to quantify the degree of difficulty in the freestyle to music based on a newly invented "bonus-malus" system. On an artistic level, the choreography and music can be quantified by establishing a catalogue of movements which can earn bonus points depending on their difficulty. A rider would have to indicate the degree of difficulty of his freestyle beforehand and the judges will score this using the catalogue. A 6 would be the neutral score if a rider has ridden all the requirements once. "He can then build up the degree of difficulty on the condition that it is well executed. He then gets a score between 6 and 10," Wüst explained. "If it is badly executed, he gets malus points. Riders should not over-ask their horses because they will get malus points for incorrect execution, or it is unbalanced." The new freestyle scoring system is an attempt to reduce partially the subjectivity of the judging. "Riders will know what they can do to get good points," Wüst added. "It is no longer a finger in the wind," Davison quipped.
The Dressage Task Force also expressed its interest in splitting up the tasks between judges: some scoring the technical part of the test, others the artistic part. Wüst showed a video of the 2007 European Championships in Turin to demonstrate how judges could divide tasks by using newly designed test sheets. "The most positive thing about this is that those judges doing the artistic judging could, for the very first time, sit back and have the luxury of simply watching the entire test before giving marks for that part of the performance," Dover stated.
New Angles Explored
Tuesday morning started off with a trial to split up the technical and collective marks. A panel of ten judges divided tasks. Five gave marks for the movements, five for the collectives using a newly designed test sheet. This trial challenged the variation between the average marks for movements and for collectives, which were separated in five categories: walk trot canter, submission, execution of the movements, rider's seat and overall performance. Would it be higher or lower?
Wüst's freestyle ideas were put to practice in the fourth trial in which four judges at H, C, M and B marked the technical part using half marks and three judges at B, between C and H and between C and M scored the artistic part with half points.
In the afternoon the final fifth trial examined the effect of putting the judges closer together. Five judges on the short side and five on the long side all had to mark an Intermediaire I test. Stickland unofficially confirmed that the scores were much closer together with the judges all having the same view. The variation of angles was also assessed by repositioning two judges' boxes at M and H on the long side instead of the traditional short side corner spots. FEI Dressage Director Trond Asmyr stipulated that "the public looks at a test from all angles, so why not judge from the same angle as the public."
Judges Supervisory Panel
In the background of the trials in the main ring a side project was being tested which delighted many officials attending: the Judges Supervisory Panel (JSP). A panel of three (retired) judges would follow on a tv-screen the running scores of their officiating colleagues and flag on screen any major discrepancies between the scores for a movement. They can decide for themselves whether the difference was acceptable. The idea is that these judges could overrule a mark and correct it so major mistakes would no longer happen. How often do we see one judge scoring a movement a 4 while the others give it a seven, or the other way round, when one judges missed a mistake and gave an 8 while his colleagues marked it straight 4s.
O judges Uwe Mechlem and Eric Lette and trainer David Hunt carried out this peer review on both days. "Any error could be redressed and it will only make the sport more fair," Richard Davison summarized. Elisabeth Max-Theurer confirmed the importance of the JSP: "We need to get the JSP to avoid nationalistic judging. It is a way for fairness in the future."
A few issues need to be taken into consideration. It will be necessary that the marks flashing on the tv-screen will be anonymous so that JSP panelist will not feel pressure to overrule a mark. The educational value of this system is immense as it can help in the instruction of the judges. Furthermore, judges who messed up too much could run the possibility of getting demoted. At the moment the technology is not up-to-date yet to implement it in the current system, but this is definitely something which will be worked on. Robert Dover suggested that the JSP should not only include judges, but could also have riders or trainers in it.
Positive and Appreciative About the Judging System Trials
The Tuesday Trials came to an end with a press conference in the main building on the Aachen show grounds. The entire FEI Dressage Task Force, including David Holmes (FEI Executive Sports Director) and Trond Asmyr (FEI Dressage Director) briefed the press, consisting of a mere five journalists and two photographers that were on assignment to report on this historic event.
DTF chair Frank Kemperman informed that changing the judging system is only one of the many tasks the DTF has taken at hand for the betterment of the sport. The results and analyses of the scores will be gathered in one big report which will be first presented to the FEI at the General Assembly in Copenhagen, Denmark, November 15-20, 2009, before it is released to the world.
Wüst stated that "our aim is to stop the criticism on the judging. We don't think the system is bad but we try to be pro-active and modernize our system."
David Holmes explained that prior to the trials in Aachen, the FEI thoroughly researched judging systems of different sports and had met with representatives of ice-skating and gymnastics. "The key thing that came out of both sports is their evaluation panel," Holmes confirmed, which validates the installment of the Judges Supervisory Panel.
Before the trials there was a subdermal fear that the judges would not comply to a new system. "We were not sure how they would take to it and adapt to some very radical ideas. Before they were iffy, but afterwards they got more open-minded and were able to pick out the positive sides," Davison admitted. "They judges became very open and were ready to embrace improvements."
In conclusion Davison specified that "the system needs to get better, not the judges."
Related Links
Visitors Welcome at Dressage Task Force Judging Trials in Aachen
Dressage Task Force July Conference Call Summary
FEI Dressage Task Force Letter To Trainers And Riders
Dressage Task Force Announces Judging System Trial at Aachen
Trond Asmyr, New FEI Director of Dressage