@pushkin-templates/exp-grammaticality-judgment

The grammaticality-judgment template includes an experiment in which participants rate the acceptability of target sentences.

Installing the exp-grammaticality-judgment template

In your Pushkin site directory, run:

pushkin i exp

Name the experiment and select the main Pushkin distribution. Select @pushkin-templates/exp-grammaticality-judgment from the list of available templates and choose which version you want (the latest is typically recommended).

Config options

In the experiment's /web page/src direrectory, you'll find the config file config.js (note that this is different from the config.yaml file in the root of the experiment). Here you'll find some options which change the behavior of the experiment:

• fontColor: the color for experiment text
• fontSize: the size for experiment text
• fontFamily: the font for experiment text (note that multiple backups are specified, in case specific fonts are not available for particular participants)
• correctiveFeedback: when set to true, two-alternative forced choice questions will indicate if the participant's response was correct (in green font) or not (in red font). For the likert scale or slider, text indicating if the sentence was grammatical or ungrammatical will show. For all response types, when this is set to false, a fixation cross appears instead of corrective feedback
• responseType: controls whether the response type is two-alternative forced choice ("2afc"), five-point likert scale ("likert"), or a slider from 0-100 ("slider")

Stimuli

sentence_grammatical	sentence_ungrammatical
This is a sentence.	This a sentence.
He went to the park last night.	He went the park last night.
The cow is jumping.	The cow are jumping.
Which friend did she meet with?	Which friend she meet with did?

The stimuli for this experiment have two parameters:

• sentence_grammatical: the grammatically correct version of the sentence
• sentence_ungrammatical: the grammatically incorrect version of the sentence

Customizing a grammaticality-judgment experiment

Finding experiment files to modify

If you have installed an experiment using the grammaticality judgment experiment template and called it gram, you should have a directory called gram in your experiments folder. This directory should be structured like this:

gram
├── api controllers
├── config.yaml
├── LICENSE
├── migrations
├── README.md
├── web page
└── worker

In order to customize your simple grammaticality judgment experiment, you will need to access two files, config.js and stim.js. These files can be found in web page/src/, a directory that looks like this:

src
├── assets
├── config.js
├── consent.js
├── debrief.js
├── experiment.js
├── index.js
└── stim.js

Modifying config.js

This file controls the aesthetics of your experiment, including font color, font size, and font family. If you wanted to set the font color to green, set the font size to 24px, and set the font family to a monospace font such as Courier New, you would modify config.js as follows:

// Custom stylin'

var experimentConfig = {
    fontColor: "green",
    fontSize: "24px",
    fontFamily: "'Courier New', Courier, monospace",
    correctiveFeedback: true
    responseType: "2afc"
}

export default experimentConfig;

You'll notice that 'Courier New' is not fontFamily's only specification. This is because it's important to list backup fonts in case your preferred font can't be loaded. You can read more about this practice here and see other CSS font combination ideas here.

You'll also notice that correctiveFeedback is set to true. You can change this to false so that participants don't receive any feedback. Once you make this change, your config.js should look like this:

// Custom stylin'

var experimentConfig = {
    fontColor: "green",
    fontSize: "24px",
    fontFamily: "'Courier New', Courier, monospace",
    correctiveFeedback: false
    responseType: "2afc"
}

export default experimentConfig;

By default, responseType is set to 2afc. You can change this to likert or slider to change the response type to a 5-item likert scale or a slider from 0-100. If you wanted to use a likert scale, your final config.js should look like this:

// Custom stylin'

var experimentConfig = {
    fontColor: "green",
    fontSize: "24px",
    fontFamily: "'Courier New', Courier, monospace",
    correctiveFeedback: false
    responseType: "likert"
}

export default experimentConfig;

You can run pushkin prep and pushkin start to see your changes.

Modifying stim.js

This file controls the stimuli presented to participants. It specifies the sentences for each trial and denotes which is grammatical (sentence_grammatical) and which is not (sentence_ungrammatical).

Say you have created the following table of stimuli for your experiment:

sentence_grammatical	sentence_ungrammatical
This is an example.	This an example.
He went for a walk.	He went a walk.
The frogs are jumping.	The frogs is jumping.
Where did she go?	Where she did go?

In order to be able to use these stimuli in the grammaticality judgment experiment, you must use a table-to-JSON converter such as this one to format it correctly for jsPsych. Once it has been converted, paste the JSON into the stim.js file so that the new object of stimuli is assigned to the stimArray variable.

Run pushkin prep and pushkin start again, and your experiment should be ready to go!