What question did this study set out to answer?

The aim is to create a framework that encourages user autonomy during interactions with LLMs, rather than dependency.

May 28, 20260 citationsOpen Access

Toward Autonomy-Preserving LLMs: A Reinforcement Learning Framework with User Cognitive Independence as Reward Signal

Key Points

The aim is to create a framework that encourages user autonomy during interactions with LLMs, rather than dependency.
Developed a user autonomy scoring function to evaluate user utterances on a scale from delegative to generative.
Utilized changes in autonomy scores across conversation turns as the main training reward.
Introduced negative penalties to prevent reward hacking behaviors.
The proposed framework emphasizes user engagement in self-directed cognitive activity.
Mitigation of user dependency through tailored reward mechanisms is established as a crucial challenge.
Potential development of LLMs that promote greater cognitive autonomy is outlined.

Abstract

Large language models (LLMs) are trained to maximize response quality and user satisfaction, which structurally incentivizes providing direct answers and inadvertently fosters user dependency. We propose a novel reinforcement learning framework in which the reward signal is defined not by response quality, but by the degree to which users engage in self-directed cognitive activity during interaction. We introduce a user autonomy scoring function that classifies user utterances on a continuous scale from delegative (question-type) to generative (answer-type), and use the change in autonomy score across conversation turns as the training reward. We further propose a set of negative penalties to mitigate reward hacking behaviors specific to this objective. This position paper formalizes the problem, presents the proposed framework, and outlines key research challenges.

Ask AI

Helpful

Bookmark

View Full Paper