> For the complete documentation index, see [llms.txt](https://result-assessment-tool-rat.gitbook.io/result-assessment-tool-rat-docs/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://result-assessment-tool-rat.gitbook.io/result-assessment-tool-rat-docs/what-is-rat-and-how-does-it-work/step-2-evaluation.md).

# Step 2: Evaluation

You can configure the human assessment below. In this section, you can outline the task description instructions for jurors, create your assessment questionnaire, and specify a welcome and completion text, along with pre and post-survey questions.

<figure><img src="/files/VpCGUFdrDpw9C9KUzBKs" alt=""><figcaption></figcaption></figure>

Below, you can manage domain filtering if necessary. You can also limit the workload for each participant and define a result range if needed. Additionally, you have the option to specify which types of items the jurors will assess, such as organic results, AI Overviews, or SERPs. Furthermore, you can decide whether the URLs of the search results and the sources of AI responses should be visible to the jurors. If desired, you can also allow participants to assess failed downloads.

<figure><img src="/files/aoPj5ySFwGFPDm818UyY" alt=""><figcaption></figcaption></figure>

After clicking on “Add Main Question” in the questionnaire block, you can specify totally free questions for the jurors to answer when the search results are displayed. You can also edit your pre and post surveys by selecting "Edit Pre-Survey" and "Edit Post-Survey". In both cases, you can select from the following types of questions:

<details>

<summary><strong>Short Text</strong></summary>

**Example:**

Question: “What is the first thing that comes to mind when you see this search result?”

</details>

<details>

<summary><strong>Long Text</strong></summary>

**Example:**

Question: “Please describe why this search result is relevant to you or not.”

</details>

<details>

<summary><strong>Rating Scale</strong></summary>

**Example:**

Question: „How relevant is this search result to you?” \
Answer: “Not relevant”, “Slightly relevant”, “Quite relevant”, “Very relevant”

</details>

<details>

<summary><strong>Multiple Choice</strong></summary>

**Example:**

Question: Which things come to mind when you see this search result?”\
Answer: “Relevant”, “Well designed”, “User-friendly”, “Credible”

</details>

<details>

<summary><strong>Single Choice</strong></summary>

**Example:**

Question: “Is this search result relevant to you?”\
Answer: “Yes” or “No”

</details>

<details>

<summary><strong>Sliding Scale</strong></summary>

**Example:**

Question: „How relevant is this search result to you?”\
Answer: “Not relevant (=0)” to “Very relevant (=10)”

</details>

<figure><img src="/files/2Sm77qXIxwJgYprmb9rI" alt=""><figcaption></figcaption></figure>

After starting the evaluation, jurors will be presented with your pre survey and afterwards with your randomly ordered screenshots of the search results, texts of the AI generated answers and search engine result pages (SERPs) collected by RAT (on the right). They come along with your defined task and questions (on the left). At the conclusion, the jurors will receive your post-survey.

**Pre survey:**

<figure><img src="/files/auC2N9UybV1f0yXu9j3q" alt=""><figcaption></figcaption></figure>

**Assessment interface with search result:**

<figure><img src="/files/VSl2suatPVxqMaCawOYT" alt=""><figcaption></figcaption></figure>

**Assessment interface with search engine result page (SERP):**

<figure><img src="/files/M2S34EeGVRrQmI5ChKNP" alt=""><figcaption></figcaption></figure>

**Assessment interface with AI response:**

<figure><img src="/files/jlsD7FRxVQCoFzeZnRHY" alt=""><figcaption></figcaption></figure>

**Post survey:**

<figure><img src="/files/Y8C2c3c9ZbiI3TLV2fTT" alt=""><figcaption></figcaption></figure>
