Minigame Point System Improvements

No TL;DR for this since I highly recommend reading everything to gain an understanding of the purpose of the post and how the suggestions work exactly. If you really must, then read the formulas posted below, that's the only change I'm suggesting.

Spoiler: TL;DR

Suggesting this new formula, modelled directly off the Elo system:

1) Change in rating ΔR = K*(S - E)

2) ΔX = RandInt(ΔR*0.8, ΔR*1.2)

3) New Rating R' = R + ΔX

K is a constant to be determined, recommended value of 40.

S is the score of the match = 1 or 0.5 or 0

E = 1/(1 + 10^[(R* - R)/400]), which is the expected score of the player.

R is the player's current ratng.

R* is the opponent's current rating.

Introduction

The points system in GMS is a replica of rating systems widely used in games like chess (with an added random range for points gain/loss at the end). However, the in-game system is somewhat out of proportion right now, so I'm suggesting this improvement (in particular, the Elo system) to make the numbers more sensible.

The general idea of a rating system is to measure the skill level of a player using a number. Quoting from Wikipedia on the Elo rating system: "A player's Elo rating is represented by a number which increases or decreases based upon the outcome of games between rated players. After every game, the winning player takes points from the losing one. The difference between the ratings of the winner and loser determines the total number of points gained or lost after a game. In a series of games between a high-rated player and a low-rated player, the high-rated player is expected to score more wins. If the high-rated player wins, then only a few rating points will be taken from the low-rated player. However, if the lower rated player scores an upset win, many rating points will be transferred. The lower rated player will also gain a few points from the higher rated player in the event of a draw. This means that this rating system is self-correcting. A player whose rating is too low should, in the long run, do better than the rating system predicts, and thus gain rating points until the rating reflects their true playing strength."


To summarise:

1. Your rating is a representation of your skill and performance at the game.
The number per se is meaningless unless there is a comparison to be made. For example, let's say a chess player has a rating of 2400. That means nothing unless I tell you (or you know) that the average/starting chess player rating is at 1500, and the highest rated player is ~2800. This gives you an understanding that this player is pretty good at chess.

2. When compared with another rating, it reflects the expectation of the outcome of matches between the two players.
A person with a higher rating is expected to win more games against a lower-rated person. If the two players then proceed to play a series of matches and the outcome is very close to the expectation, then their ratings shouldn't change much overall since it is still accurate. However, if the lower-rated player does better than expected, his rating will increase and the higher-rated player will lose points since the system believes that their rating are inaccurate (given the evidence that is the result of the matches), and will adjust it to reflect the skill levels better (lower rated player goes up since he is probably better than what his ratings currently suggest).

3. The amount of points exchanged depends on the difference between the two ratings.
If you beat a player of equal rating, you would gain a certain amount of points. If you played someone with a much lower rating, the points you gain should be less than what you got from the equally rated player. That is because you are expected to win the lower rated player in the first place. Similarly, if you beat someone with a much higher rating (or if you lose to someone of much lower rating), you would gain (lose) a much greater amount of points because you did something that was unexpected.



Why is there so much emphasis on expectation?

It gives mathematical meaning to the numbers when compared with one another.

Before I go on, you need to know that in rating systems a win is worth 1 "point", a draw is half a point to both players, and a loss is worth no points. That means for each game that is played, there is 1 point up for grabs. If you draw the game, it is split evenly (ie you achieve 50% result since the match was even). If you won the game, you get a "point" (ie you achieve a result of 100%, the best outcome of the game and the outcome that everyone aim for). Similarly, if you lose you get no points (it is the worst result of the game, hence you scored 0%). From here on I will use the word "points" to refer to the result of the match, and "rating" to refer to the number that represents your skill level.

In the standard Elo rating system, players of equal rating are expected to be of equal skill, and hence both of them should achieve 50% result across all matches (0.5 points per match on average).

A player who is rated 200 higher than another has an expected result of about 75% (0.75 points per match on average). This translates to roughly 3 wins + 1 loss every 4 matches, or 2 wins + 2 draws (both cases you get 3/4 points).

A player who is rated 400 higher than another has an expected result of about 90% (0.9 points per match on average).

To summarise, say we have 4 players:
A rated 2000.
B rated 2200.
C rated 2400.
D rated 2600.

Going down the list, each player has an expected result of 0.25 against the player below him. A has an expected result of 0.1 against C, same as B against D.



This is the main motivation for my writing of this post - to assign this meaning to the numbers so that it makes more sense and is of more use the win/loss ratio (which is also already recorded in-game).

Think about the accuracy of win/loss ratios. If I kept playing with a weaker friend, I would get a 100/0/0 record. But you cannot say that I am better than someone who players against strong players and achieve a 70/30/0 record. The rating system is supposed to do that job, but the current system in-game does it poorly, and I dare say it is better to use win/loss ratio as an estimate of skill levels rather than the current rating system.



GMS system versus current system

No one knows the exact formula that GMS uses, but by observation it is clear that they do model a standard rating system, meaning they achieved all the points that I wrote above. The only difference is that when points are supposed to be exchanged at the end of the match, they fluctuate it a little on purpose to allow for variety and a bit of "luck". But because it is random, in the long run it should average out and still work like a proper rating system.

The Legends system we are using now does not emulate a rating system properly. It does roughly tell you the skill level of the player (and even this may be inaccurate due to the formulas used), but the proportions are not in order. Unlike in the case of the Elo system, say if you are rated at 2400, you do not know exactly how much better a player rated 2600 is compared to yourself. He might be much stronger than you think, being able to beat you 19 games out of 20, or he may be very close to your skill level. The difference of 200 points here does not tell you much because of the way the system is designed. There is a lack of consistency throughout the rating scale and in point exchanges. By consistency I do not mean not using random factors (which GMS does), instead I refer to a formula that generates a meaningful result each time.

An example of Legends' system:

Players with about 2000 rating usually gains/loses 2 points when playing with each other.
Compare this to a match between 4000 and 2000 player. The 4000 player gains/loses about 400 points each match.

Points to note:

1. 2 points per match is really low for evenly skilled opponents.

2. 400 point gain is too much even if the 2000 player beats the 4000 player. 1 win alone does not justify the huge transfer.

3. The most important point: The 4000 player also gains 400 points from beating the 2000 player. This does not make sense seeing a fellow 2000 player only gains 2 points from the same 2000 player. Why would someone who is far better than their opponent gain so much more points from winning compared to two people who are evenly skilled?


This is the reason why the numbers are meaningless. A 4000 player can easily hit 6000 by beating 2000 rated players a few times, whereas the 2000 only reaches about 2050 after the same number of matches. In this case, the 2000 player is of similar skill level to the 2050 player, and the 4000 player is of similar skill level to the 6000 player, but the difference in numbers are not consistent in reflecting that. In short, the number mean different things at different rating tiers, making it extremely difficult compare and understand the numbers.



I'm guessing that the amount of points exchanged is proportional to the aggregate rating of both players involved. That explains why a match between 4000 and 2000 player yields such a high point exchange compared to a match between two 2000s. I am not 100% sure that this is the case, but the fact remains that the rating system isn't effective.

Hence, I would like to suggest a switch to the Elo model as described below.



The Elo Rating System (tweaked)


Change in rating ΔR = K*(S - E)


-- S is the result of the match. It doesn't have to be one match, but since in Maple the ratings are updated after each match, we only need to use the score of one match. In this case, a win is 1, a draw is 0.5, a loss is 0.

-- E is the expected result of the player, calculated by:

E = 1/(1 + 10^[(R* - R)/400])

where R* is the opponent's rating.

-- K is a rating change constant decided upon by the user of the system. I recommend a value of 40 for this (I estimate this is the rough value that GMS uses). This means that the amount of points exchanged between 2 equally rated players is 20 (half of K, you can check this using the formula). This also caps the rating change at 40 per match for extremely unexpected results. The more extreme, the closer it gets to the limit of 40.



The points exchanged this way would be fixed, but GMS made this a little random, so we shall do the same via:

ΔX = RandInt(ΔR*0.8, ΔR*1.2)

Allow a 20% discrepancy and choose a random integer from the range. Then the new rating for the player would be:

New Rating R' = R + ΔX

where R is the current rating of the player.

Note: GMS randoms the change for each player individually, hence the points exchange may not be a zero sum equation, ie a player may gain 10 points and the other loses only 8 because of the random factor adjusting the change for each player separately. So this process has to be done twice, one for each player.

Note 2: A few of these calculations should probably be floored, but that might be done naturally with the code. It doesn't matter much though.

Examples of the system in action:

I will write some examples without using the randomiser I added above, just to show you the average effect of the system.

A (2000) beats B (2000)

New rating of A: 2020 (+20)
New rating of B: 1980 (-20)

A (3000) beats B (2000)

New rating of A: 3000 (the change is 0.1, which will probably be floored.)
New rating of B: 2000

A (2000) beats B (3000)

New rating of A: 2039 (+39)
New rating of B: 2961 (-39)

A (2000) draws with B (2400)

New rating of A: 2016 (+16)
New rating of B: 2384 (-16)

Additional Comments (/FAQ)

Why percentage based randomisation?
If a fixed +/- value is used for range, a very strong player who beats a very weak player would get much more points than he should. On average, he only gains a couple of points (check with GMS videos), but with fixed +/- value of say 3~5 points, that would allow him to gain 4~6 points per match, up from 2. Also, percentage randomisation is better to keep in line with the meaning of numbers in the long run.

Final Notes

In my opinion, Elo is not the best rating system. It is good no doubt, but there are other systems that have made improvements to make it more accurate and effective. However, those system are really really complicated, whereas the Elo system is very simple to calculate and code (relatively speaking), so it is the ideal system to use for a game within a game.

This may be a trivial improvement to some, but since mini-games are working, I would like to see them functioning properly in all aspects at least.

If you have questions of clarification or anything, feel free to post them

Links

http://en.wikipedia.org/wiki/Elo_rating_system

or just Google "Elo Rating System" and see what you find.

 

Wasn't planning on making a TL;DR, but fine I shall OnionDown

 

Like 2 metres or so.

 

Omok is a lot more complicated than Tic Tac Toe. It is similar to other piece-placing games like Go and Reversi, they all involve a lot of thinking and strategy and there are high level competitive play out there just like other games such as Chess (as well as rating sytems to rank professional players for it). Tic Tac Toe is an extremely simplified version of Omok, so simple that if you study it a little you should not lose any games at all (I would say it is a "child's game", and Omok is the proper "adult" version of it). So while it is silly to use Elo for Tic Tac Toe, Omok is definitely complex enough for it.

Also, in GMS the points system was made to act like a rating system. No one knows exactly what their formulas are, so I simply picked Elo to be used since it is one of the simplest systems out there.