This is about challenge, opposition, danger, and fear. Suppose I say to you, hey let’s play this game. You say 1, I say 2, you say 3, and so on, and when we get to 10, then it’s over! Sounds kind of boring right? There’s nothing for you to put your mind to, no challenge to overcome, because you’re pretty good at counting to 10 already. Once we add an element of difficulty, such as a taboo on multiples of 6 (or whatever), so that instead of saying those numbers you have to clap your hands (or something) whenever one of them comes up, then we see how high we can count before someone makes a mistake and it becomes a classic ice-breaker game. The key element is the possibility, however small, of failure, danger, and fear. In a game, we hunger for the unknown, we thirst for victory against the enemy. And that enemy is ultimately ourselves.
Of course games like football, chess, and rock-paper-scissors have made use of opposition to provide this excitement and interest for countless years: It’s me and my team against you and yours — which of us will win? The interplay of strategies, skills, and intuition makes such games fascinating to play over and over again, or even to watch without playing.
Roleplaying games are a very special case. The gamemaster (or “GM”) is often the individual tasked with creating a series of monsters, dangers, or other problems that the rest of the group (usually called “the players”) must overcome. The inherent imbalance in the situation — one person versus all the rest — means that the GM needs a lot of power. Most roleplaying games tend to give the GM virtual omnipotence within the game and story environment, but they protect the players from this unlimited power by changing the GM’s role from “opponent” to “opponent and ally.”
The GM has to create just enough opposition that the players will feel properly opposed and challenged, but not so much that they feel overwhelmed and discouraged. The goal here is to actually help the players engage with the story and the game by being their worthy opponent, the darkness that they must dispel. If the players successfully overcome the challenges you set out for them, then you rightly feel happy about their accomplishment, and if they fail, you can of course give them some sort of second chance or way out, albeit with consequences attached. The right balance of success and failure leads to story and character development that feels very real, because even though the story will eventually reach some kind of conclusion, along the way we see what the characters had to sacrifice to get there. This is the real win condition for the group as a whole, players and gamemaster together, and the apparent competition between them is just a tool to help them get there.