Slot Machine Reward Psychology

Slot machines reward gamblers with money according to which reinforcement schedule? Variable ratio c. Fixed interval d. Variable interval. But it’s the way they dish out rewards that really seals the deal, and slot machine makers learned these invisible design elements from basic psychology. Skinner’s famous study on reward is a.

• Slot Machine Reward Psychology Training
• Slot Machine Reward Psychology Course
• Punishment And Reward Psychology

I spent part of last week on vacation from science in Las Vegas, where I thankfully avoided financial ruin due to some fortunate combination of genes, math awareness and a wife that has no interest in gambling. Sure, I dabbled a bit in games of chance, but as soon as I got a little bit ahead on the blackjack tables I ran for my life, knowing that the probability would even out hard in the long run. For those concerned about the financial well-being of Sin City, they still managed to turn a profit on us, thanks to the low-return temptations of fine dining and French circus acts set to Beatles megamixes. But most of our time was spent on the free entertainment of people-watching and stuff-watching, observing row after row of people almost hypnotically at work on loud, noisy slot machines amid fake New York, Paris and Venice scenery.

It doesn’t take a PhD in neurobiology to conclude that slot machines are designed to lure people into a money-draining repetition, just as it doesn’t take expertise in the casino business to realize slots are absurdly profitable – there’s a reason why they outnumber table games 100-to-1. But I wanted to go back to the scientific literature to confirm a faint glimmer of information I retained from graduate school, specifically that slot machines are masterful manipulators of our brain’s natural reward system. Every feature – the incessant noise, the flashing lights, the position of the rolls and the sound of the coins hitting the dish – is designed to hijack the parts of our brain designed for the pursuit of food and sex and turn it into a river of quarters. Or so I remember.

Fortunately, there is a robust amount of research into why slot machines are so addictive, despite paying out only about 75% of what people put in. They are, some scientists have concluded, the most addictive of all the ways humans have designed to gamble, because pathological gambling appears faster in slots players and more money is spent on the machines than other forms of gambling. In Spain, where gambling is legal and slot machines can be found in most bars, more than 20.3 billion dollars was spent on slots in 2008 – 44% of the total money spent by Spaniards on gambling last year.

That data was published earlier this month by a psychologist from the Universidad de Valencia named Mariano Choliz in the Journal of Gambling Studies. Yes, such a publication exists! In the background of the paper, Choliz outlines the tricks that slot machines use to keep people feeding them:

• Operating on a random payout schedule, but appearing to be a variable payout; i.e. fooling the player into thinking that the more money they play, the more likely they are to win.
• “The illusion of control” in pressing buttons or pulling a lever to produce the outcome.
• The “near-miss” factor (more on this below)
• Increased arousal (where the sounds and flashing lights come in)
• Able to be played with very little money; the allure of “penny” slots.
• And perhaps most importantly, immediate gratification.

This last point is the subject of Choliz’s experiment, which puts a group of ten pathological gamblers in front of two different slot machines. One machine produces a result (win or lose) 2 seconds after the coin was virtually dropped (it was computer program), the other delayed the result until 10 seconds after the gambler hit play. In support of the immediate gratification theory, gamblers played almost twice as long on the 2-second machines than they did on the 10-second machines…even though the 10-second machines paid out more money on average!

Choliz concluded that the immediacy of the reward was part of what kept people at slot machines, making them so addictive. The quick turnaround between action and reward also allows people to get into a repetitious, uninterrupted behavior, which Choliz compares to the “Skinner boxes” of operant conditioning – the specialized cages where rats hit a lever for food or some other reward. It seems like a cruel comparison, but after my three days walking through the casinos, not an inaccurate one.

Another trick up the slot machine’s sleeve was profiled earlier this year by a group of scientists from the University of Cambridge. In the journal Neuron, Luke Clark and colleagues examined the “near-miss” effect, the observation that barely missing a big payout (i.e. two cherries on the payline while the third cherry is just off) is a powerful stimulator of gambling behavior.

The Cambridge researchers put their subjects in an fMRI machine to take images of their brains while they played a two-roll slot machine game. When the players hit a match and won money, the reward systems of the brain predictably got excited – the activation of areas classically associated to respond to food or sex I mentioned earlier. When players got a “near-miss,” they reported it as a negative experience, but also reported an increased desire to play! That feeling matched up with activation of two brain areas commonly associated with drug addiction: the ventral striatum and the insula (smokers who suffer insular damage suddenly lose the desire to smoke).

Clark and co. conclude that near-misses produce an “illusion of control” in gamblers, exploiting the credo of “practice makes perfect.” If you were learning a normal task such as hitting a baseball, a “near-miss” foul ball would suggest that you’re getting closer – it’s better than a complete whiff, after all. But for a slot machine, where pulling the lever has no impact on the rolls other than to start them moving and start the internal computer calculating, a “near-miss” is as meaningless as any miss.

Nevertheless, it’s this type of “cognitive distortion,” as Clark and colleagues name it, that makes slot machines such effective manipulators of our brains. Those massive, gaudy casino-hotels that I wore out a pair of shoes strolling through last week weren’t just built on a crafty use of probability, they were built on a exploitation of brain functions we are only just beginning to understand.

In particular, the Skinner Box applies directly to slot machine addiction. Skinner did experiments with rats using boxes that dispensed cheese as a reward when the rats pulled a lever. Obviously, the rats who got cheese every time they pulled the lever were more motivated to pull the lever than the rats who didn’t get cheese. Slot Psychology. Slot machine manufacturers use psychology to set the winning combination percentages. That's why manufacturers like Konami come up with multi-linked games with bonus screens. And, they know that players need enough tiny payouts to get instant gratification, and enough higher payoffs to keep them interested. There are many variables, but the main power of slots is rooted deeply in human psychology. Slots take advantage of basic human instinctsto maximize their appeal. Slot machines are able to seduce us on multiple levels. Slots naturally catch our attention because they are flashy.

Okay, so Lavata and I get in heated discussions about WoW’s loot system. He hates that anything random ever drops from an instance, and I think that random things HAVE to drop from instances. He’d be much happier if you always got what you want the first time, or if everything you wanted was on a token system.

The fact is that random loot uses well known behavioral psychology principles, and it works on both rats and people, to keep them engaged with an activity for longer. What I mean is – rats will keep pressing the levers, and people will keep mashin’ the keyboards trying to get their loot. At a certain point, random isn’t all that fun, but you’ll keep working towards that goal, whether or not you are happy. However, behavioral psychology has nothing to do with being happy, it has to do with modifying behavior (and behavior modification is a normal part of life). You say: “But I’m not like some Pavlovian dog that salivates at purple loot!” My response to that is, well the game is more like operant than classical conditioning, so in a way you are right.

Terms (from Schwartz, Wasserman, & Robbins, 2002, Psychology of Learning and Behavior):

• Behavioral Psychology – Behavior theorists try to understand learning (and believe that all animal species learn in basically the same way). They believe that all learning involves associations between responses and stimuli, complex behavior can be understood by it’s parts, and environmental influnences play a large part in learning.
• Operant Conditioning – Producing voluntary, goal-directed behavior, where reinforcement (either positive or negative) is used to shape behavior. Behavior is effected by it’s consequences, with the goal of doing things that will have positive consequences for you (ie. a rat will learn how to press a lever to get food).
• Positive Reinforcement – when behavior produces a reward that is considered positive. This increases the likelihood that the behavior will occur again. Keep in mind that you can think giving candy to a child is reinforcing, however if they are allergic to eating candy (as in eating candy sends them to the hospital), then it’s not a positive reinforcement. It’s when you give someone something that they actually do want.
• Conditioned Reinforcement – While things like food are likely to be rewarding if you are hungry, things like money (both inside and outside the game) are conditioned reinforcers.
• Token reinforcement – Instead of directly giving money or food, you can train an animal to associate tokens with their reinforcement. So, you can teach a chimpanzee that they can get their food after they have X number of tokens.
• Continuous reinforcement – Every time you do a behavior, you get a desired reward.
• Intermittent reinforcement– Most behavior is reinforced intermittently. For example, when you work at a job, you aren’t constantly being given pennies or dollars for producing good behaviors throughout the day, you usually get paychecks on some sort of set schedule.

Types of Intermittent reinforcement schedules (and how they show up in WoW):

• Fixed Ratio Schedule – You get reinforced after completing a certain number of responses, which is not based on the passage of time. PvP honor rewards are fixed ratio schedules. You get loot after accuring a certain amount of honor. If you kill a lot of people quickly, then you get your reward sooner. Reputation grinds, where you get rewarded with small amounts of rep for every kill, ends up being fixed ratio schedules, since there is a fixed ratio of mob kills you have to do to get to the next highest reputation rank. If you had to kill 20 mobs to get a drop, then that drop would be on a fixed ratio schedule – so a lot of quest rewards are fixed ratio schedules.
• Variable Ratio Schedule – This requires a random number of responses to get your reward, and is how all the random drops in WoW (and cassinos) operate. This is the schedule that all random drop loot is on. Just the passage of time alone doesn’t make the drop more likely, and every time you kill that mob, it has the same X% chance of dropping the item, which doesn’t increase with the number of kills.
• Fixed Interval Schedule – Requires one response after a fixed amount of time. While loot tends to not be given out just based on the passage of a fixed amount of time, the boss respawn timers are on pretty fixed intervals. If you clear Naxxramas on a Tuesday night, you get no reward for going to Naxx until after the bosses have respawned. Daily quests are also more of a fixed interval schedule. You know that if you did it today, you can’t get rewarded for trying to do it again until tomorrow. True fixed interval schedules would be something like respawning an hour after you killed it, so that the interval was based on your behavior.
• Variable Interval Schedule – Requires a random amount of time to have passed and a single response to get your reward. A great example of this is ore & herb respawn timers. Any one node takes a random amount of time to respawn, so you are forced to run/fly around a zone trying to find ones that did pop up, rather than camping on a node and coming back every X minutes. In-game fishing is on a variable interval schedule, in terms of how long it takes once you cast before you can get your fish.

But WoW rewards me constantly every time I do something: At level cap, your most common rewards are basically token reinforcement (money & token drops from instances). Loot that you want usually drops in PvE on a variable interval schedule, off bosses that are on fixed interval schedules. Things like Arenas still have some random elements because the queue system for rankings makes it such that sometimes you lose, but there’s a reason why so many people queue up the day before arena points are handed out – because arena points are basically driven entirely by a fairly fixed interval schedule – arena points are given out the same time every week. They mostly put in boss respawn timers on the high-end raids so that we’ll get up out of our chairs and do other things with our lives sometimes, and to control the pacing of how often we get those big rewards once we hit end-game. The great thing that WoW can do is to maximize all of the types of schedules, and use them in ways that keep us playing longer.

There are plenty of examples of WoW on variable ratio schedules. I could kill the first boss in Shadow Labs non-heroic as many times as I wanted to try and get my lifebloom idol off the boss, as long as I had people who were willing to go in and help me farm it. We ran that instance over and over and over again, until I just gave up mostly because my friends weren’t going to be my friends much longer if I kept making them run that instance with me (and I understood that going more didn’t increase the likelihood of it dropping).

What WoW (and Casinos) understand is that variable ratio schedules have the best rates of responding. You’ll keep playing the game longer so that you can have a chance of getting that awesome reward. While random isn’t “fair”, it keeps you playing the game longer.

Slot Machine Reward Psychology Training

Variable ratios are also a tool to keep some items as being very rare, which makes them more desirable. “Green” items are super common, so it’s not exciting when you get one. “Blue” items are worth a little bit more because they’re a little bit harder to get. Some “epic” items are likely to be rare, so we still value them highly, and in most cases, they are the best we can get. “Legendary” items are way super rare, and people will do just about anything to get one. If all it took to get the legendary was a fixed ratio of 20 kills of bosses in a raid instance and everyone got it after that number of kills, it wouldn’t have that high of a value – it would be totally common. The randomness and rarity of the items make them desirable.

Slot Machine Reward Psychology Course

If all the loot was based on a fixed ratio token drop schedule, raiding would be really, really boring. “Oh, I don’t get my loot until next Tuesday.” It didn’t work in manufacturing jobs – people created unions to organize against fixed ratio payment schedules, because while they worked hard to get products built, it meant that the entire workday was a constant grind to get things done. Having some grinding to get rewards on a fixed ratio schedule is fine to have in the game, since you’re actually just getting tokens and points on a constant schedule, which isn’t even a true fixed ratio schedule (ie. kill this mob 20 times and then get the drop).

In this game, it’s the draw of the random exciting rewards that makes us keep coming back week after week, year after year. When the loot items in the existing content are no longer desirable, they release new and better items that are “must have” items for us to be excited about earning again. If bosses didn’t drop random loot, we probably wouldn’t like raiding so much. The variable ratio drop schedules for the pieces of the desired loot pieces is what makes us keep coming back to an instance. You don’t want to miss a night of raiding, because if the desired piece of loot drops and someone else gets it instead, then you’ll be upset. Likewise, people don’t want to get up from their slot machine at the Casino, even if they are on a winning streak, because they are waiting for their payout to come. Casinos would make a lot less money on slot machines if they gave out rewards on any other reinforcement schedule than what they do.

Understanding behavioral reward systems also helps explain why people got bored with Naxxramas and stopped logging in after they had all the loot they wanted. They didn’t really think that there was likely to be a reward they want for going to that instance, so they stopped playing and found something else to do that was rewarding (either in or out of the game). This is also why gearing up your tank first, before the rest of the raid, usually ends up with the tank getting bored and not wanting to run the instance long enough for everyone else to get loot. However, sometimes you keep running it after you stop getting rewards, since “extinction” of behavior (ie. when you stop doing something) usually takes longer on a variable interval schedule than the other schedules. It helps explain why people will spend hours of their time trying to fish up a rare turtle mount, and why at least some loot out of instances has to be random drops (and why bosses usually drop a smaller number of loot items than there are people in the instance). If you knew that after 20 runs in Ulduar, you could have all the gear you needed, you would run Ulduar 20 times and then stop running the instance.

Punishment And Reward Psychology

In conclusion, having rare drops on a variable ratio schedule keeps rats pressing the lever longer, keeps people spending money in the Casinos, and keeps WoW players staying in the game longer. It’s not about being fair, or really what gets you the rewards the fastest – variable ratio reinforcement schedules is what what keeps you playing the longest. I could ramble on more, but that’s where I’ll stop for today.