An anonymous evaluator: The three key skills of an evaluator

An evaluator’s expertise, the time they spend reviewing an application, and their experience of encountering another human being in the process are central to the assessment work. Not only do these speak of the evaluator’s skills, but they also raise discrepancies that are equally relevant to the success of the evaluation.

In my understanding, there are three key factors that particularly affect the assessment.

First, the evaluator’s expertise in the field which has led them to be assigned the evaluation task. Secondly, the limited time they have for each application. And third, their ability to both get excited about and be indifferent to everything they experience in the process. In other words, being a unique person.

This means that an evaluator has at least three discrepancies to resolve.

At the margins of expertise

When structuring a large number of applications, I often look at similarities between themes and subjects, between different forms and sub-forms of art, and at other interests that arise intuitively and are shared by different applications. This may happen quite loosely, at an early stage and largely based on the title of the application. These quickly provide an overview of the field within which the selection will take place. If any application is clearly outside my expertise, this also becomes evident at this point.

If we consider the evaluator’s expertise in their field to be the right hand, the left hand is the field that an application speaks about and that requires a specific type of expertise. The evaluator tries to align this left hand as well as possible with the right hand, without ever quite succeeding. Experimental art operates at the margins of a well-known field of art. The work taking place between different art forms may turn its back on all the known frames of reference from which it has sprung. I admit that some of the applications I’m tasked with reviewing sometimes require expertise that I as an evaluator don’t have direct access to. In cases where this is blatantly true, the application will be transferred to another evaluator. In more subtle cases, the applicant’s work may fall within the margins of a context I’m familiar with, and my enthusiasm for the project and trust in the applicant takes precedence over my expertise, as does the time I spend with the application.

If only there was more time

At the start of the evaluation process, the number of applications I’m assigned sets the pace. It determines how much time on average I have to review each application. However, this too creates a conflict. I am expected to give a fair evaluation of the applications. For example, this could mean that I review each application for approximately the same amount of time. Yet I find myself spending a different amount of time on each application, depending on how much time it requires, even if it differs from other applications. This is also a natural result of the reading experience itself: after all, the writing styles of the applications sometimes vary greatly. Some include a wide range of references that I find myself exploring. Others are concise and quick to read. Some provide extensive portfolios, others shorter ones.

The applicant’s and the evaluator’s ways of thinking meet in a surreal space, because the time spent writing the application is completely different to the time the evaluator spends reading it. In this regard, the applicant and evaluator do not meet at all. Yet, the applications convey the other person’s way of thinking very strongly.

An experience of meeting another person

The time I spend reading the applications also communicates what I am excited about and what doesn’t excite me. I spend considerably more time than average reading some applications. For one reason or another, I get immersed in them. Using search engines, I look for more information about the applicant and the topics they raise. I may come back to the application to review it in different contexts.

Other applications I browse through. They don’t seem to evoke anything in me, nothing resonates. When this happens, I mark the application as unread, inevitably returning to it again after some time. If the application then feels remotely familiar, but I still don’t want to spend any more time with it, I will not give it any more attention.

However, some applications stay at the back of my mind even though I’ve decided not to give them more time. They may niggle at me subconsciously, and as I continue reading the next applications, an idea they highlight may bring the earlier application to my mind. Applications help each other by highlighting and strengthening a common interesting phenomenon or theme of our time.

A particular work plan may inspire me because it clearly conveys what the applicant intends to do with the grant. Another one may intrigue me because I, as an evaluator, can’t fully grasp what the applicant is saying: despite their explanation, I can’t imagine what the end result will be like – instead, I find myself relying on trust. If I feel that this has created a kind of precarious trust relationship between the applicant and myself (we don’t know each other, there is only an application, one of many, between us), there is a strong chance that I will find the application and the applicant interesting.

As an evaluator, I constantly come face to face with my capriciousness as a human who has the ability to get interested in things and a strong need to encounter others, while also having a limited capacity for both stimuli and sociability. Spreading the review process over several weeks and months is essential for balancing this capriciousness.

When assessing, often the same few applications, for weeks on end, an evaluator may get closer to the applicant, and to the time the applicant has spent writing their application.