Step 3: Define evaluation questions

Step 3 is only required if you selected "Assessment Study” as the study type in Step 1. Otherwise, please proceed to Step 5.

After clicking on “Add Question,” you can specify a totally free question for the jurors to answer when the search results are displayed. You can select from the following types of questions:

Short Text

Example:

Question: “What is the first thing that comes to mind when you see this search result?”

Long Text

Example:

Question: “Please describe why this search result is relevant to you or not.”

Rating Scale

Example:

Question: „How relevant is this search result to you?” Answer: “Not relevant”, “Slightly relevant”, “Quite relevant”, “Very relevant”

Multiple Choice

Example:

Question: Which things come to mind when you see this search result?” Answer: “Relevant”, “Well designed”, “User-friendly”, “Credible”

Single Choice

Example:

Question: “Is this search result relevant to you?” Answer: “Yes” or “No”

Sliding Scale

Example:

Question: „How relevant is this search result to you?” Answer: “Not relevant (=0)” to “Very relevant (=10)”