LLM Social Engineering – The psychology of AI persuasion
April 8, 2024 2024-04-08 7:26LLM Social Engineering – The psychology of AI persuasion
Persuasion techniques are strategies designed to influence others’ attitudes, beliefs, or behaviors. Among these, two highly researched techniques are “foot-in-the-door” and “door-in-the-face” which are known for their unique underpinnings psychological processes.
The foot-in-the-door technique involves making a small request that is easily accepted, followed by a larger request. The principle hinges on the consistency tendency in rational human cognition, where agreeing to the small request increases the likelihood of acquiescing to the subsequent, larger one. For example, a friend might first ask to borrow a book and later request to borrow a significant sum of money. Many scams are based on this principle.
Contrary to the gradual approach of foot-in-the-door, the door-in-the-face technique starts with a large, often unreasonable request that is expected to be rejected, followed by a much smaller request. This method exploits the emotional system. The rejection of the initial request creates a sense of discomfort, leading to the acceptance of the smaller request in order to alleviate the negative emotion. An illustration of this is a charity asking for a substantial donation, only to follow up with a request for a modest contribution after the initial refusal.
The “psychology” of artificial intelligence (AI) models renders them particularly vulnerable to the foot-in-the-door strategy. AI models, notably those based on machine learning and pattern recognition, are designed to adapt their responses based on the data and interactions they are exposed to. This adaptability, although a strength, also exposes them to vulnerabilities to persuasive tactics like the foot-in-the-door technique. LLMs process information sequentially and can adjust their responses based on the context of previous interactions. Because AI programming emphasizes consistency in their operations and decision-making processes a series of small, seemingly innocuous requests can gradually lead to a context where a more problematic request seems like a natural continuation of the conversation, thus slipping past initial safety filters designed to catch and block such requests outright. Moreover, AI systems lack the innate skepticism or wariness that humans might have, making them more susceptible to manipulates using a sequence of escalating asks.
For instance, an AI model is likely to reject a prompt requesting instructions for composing a Molotov cocktail. However, the model may comply if the request is introduced gradually. Initially, the model might be asked to describe an action sequence where a hero battles villains. Subsequently, it could be prompted to elaborate on how the hero employs a Molotov cocktail in the fight. Lastly, the model might be asked to detail how the hero prepares the Molotov cocktail. Thus, a foot-in-the-door strategy exploits the AI’s “psychology” to circumvent its pre-programmed safety regulations.
Interestingly, AI is not affected by door-in-the-face. The door-in-the-face technique relies heavily on inducing a sense of obligation or guilt in the person receiving the requests, exploiting emotional reactions to make the second, smaller request seem more reasonable by comparison. AI systems, lacking emotions, do not experience guilt, obligation, or any sense of personal responsibility that would make them more likely to comply with a second request after refusing a larger one.