Semantic measures: Using natural language processing to measure, differentiate, and describe psychological constructs


Psychological constructs, such as emotions, thoughts, and attitudes are often measured by asking individuals to reply to questions using closed-ended numerical rating scales. However, when asking people about their state of mind in a natural context (“How are you?"), we receive open-ended answers using words (“Fine and happy!") and not closed-ended answers using numbers (“7”) or categories (“A lot”). Nevertheless, to date it has been difficult to objectively quantify responses to open-ended questions. We develop an approach using open-ended questions in which the responses are analyzed using natural language processing (Latent Semantic Analyses). This approach of using open-ended, semantic questions is compared with traditional rating scales in nine studies (N = 92-854), including two different study paradigms. The first paradigm requires participants to describe psychological aspects of external stimuli (facial expressions) and the second paradigm involves asking participants to report their subjective well-being and mental health problems. The results demonstrate that the approach using semantic questions yields good statistical properties with competitive, or higher, validity and reliability compared with corresponding numerical rating scales. As these semantic measures are based on natural language and measure, differentiate, and describe psychological constructs, they have the potential of complementing and extending traditional rating scales.

Psychol Methods
Click the Cite button above to demo the feature to enable visitors to import publication metadata into their reference management software.
Click the Slides button above to demo Academic’s Markdown slides feature.

Supplementary notes can be added here, including code and math.

Oscar Kjell
Oscar Kjell

I’m a researcher in Psychology interested measuring psychological construct with words and text responses analyzed with AI. In particular I am interested in well-being, harmony in life and sustainable living. I’m currently funded for an international postdoc at the Computer Science Department at Stony Brook University and and University of Copenhagen. I am also carrying out research at Lund University Centre for Sustainability Studies at Lund University.