Forum topic models: suggestion thread breakdowns

Hi all, geospiza here again with another forum statistic thread (see forum summarization). With a large scrape of the forums on my computer, I wanted to take a look at some of the suggestions for broad themes.

The breakdown of the feedback/suggestion subforum follows:

Code:

+-----------------------+-----+
|subforum               |count|
+-----------------------+-----+
|feedback               |1322 |
|rejected.86            |340  |
|accepted.85            |320  |
|not-possible-to-add.126|80   |
+-----------------------+-----+

There are currently quite a few open suggestions, with a relatively smaller amount of closed threads (either accepted or rejected). Over time:



There's a pretty obvious spike in the amount of feedback in 2020 (which corresponds with the big population spike).



The cumulative number of threads put the an inflection point around 2017, where the number of new suggestions outpaces the rate of being able to address feedback. Feedback is addressed at a fairly constant rate though.

I decided to try clustering the threads using a topic model (latent dirichlet allocation). It clusters documents into a fixed number of topics by learning a set of key words that identify a topic. The goodness of fit is evaluated using a measure of information called perplexity. Lower is better. I made a plot using different number of topics to decide which one I would use:



I tried out both 5 and 10 (one using the elbow method, the other the minimum perplexity). The following topics are defined by the top 7 words ordered by their "importance" to the topic.

Code:

The lower bound on the log likelihood of the entire corpus: -6795805.6480941605
The upper bound on perplexity: 6.9517259084958125

topic 0: map go player kill exp change say
topic 1: weapon luk sword attack dex nt level
topic 2: item drop scroll nx price player rate
topic 3: hp player level wash server quest event
topic 4: skill damage buff class attack change hit

Code:

The lower bound on the log likelihood of the entire corpus: -6767257.28175798
The upper bound on perplexity: 6.922522539803226

topic 0: moderator server gm guild player bos thing
topic 1: bubble song select random loop combat foresee
topic 2: scroll drop item price player mesos rate
topic 3: hp wash player game level server quest
topic 4: item name id nx cash donor pet
topic 5: map level exp pq player go leech
topic 6: event islander item maple island new play
topic 7: music track listen song battle youtube favorite
topic 8: charge change stance ban ht energy abuse
topic 9: skill damage buff class attack change job

The decided to go with the first set of topics, since a smaller number of them was much easier to interpret and analyze. The final model took on the following topics with their top 10 important words.

Code:

topic 0: skill damage buff attack class change hit job boss need
topic 1: hp player scroll wash game item server level play event
topic 2: quest drop level pq ring exp rate reward player party
topic 3: map player server go change say way game thing take
topic 4: item id name nx cash pet hair donor look shop

After taking a look the top words and the top threads for each of the topics, I decided on the following topic names:

• 0: Balance
• 1: Voting and Economy
• 2: Questing
• 3: General Server
  
• 4: Cash Shop

These names are not hard and fast; I took a look at the top names and the top resulting names to find something that fits.
Each thread in the dataset is given a score between 0-1 for each of these topics, where the sum of the scores adds up to 1. The scores are summed across all the threads to get a sense of which topics are the most prevalent:



Here's the plot of the weights for each topic over time:



It seems like there's a recent trend for more balance. Here are the top posts sorted by how likely they are to be part of topic. I used these to better name the topics.

Top 20 posts for general_server

1. (0.999861): map-owner-abuse.18724
2. (0.999824): mapowner-timeouts-and-abuse-of-the-system.32870
3. (0.999799): prototype-client-results.11116
4. (0.999786): npc-to-guild-hall.3058
5. (0.999694): ability-to-mute-some-all-smegas.31788
6. (0.999666): make-channel-exclusive-gm-buffs-and-announce-when-theyre-being-given-on-each-individual-channel.11910
7. (0.999657): remove-mapowner-automatically-if-the-person-who-has-the-map-is-dead.39649
8. (0.999631): skele-maps.6703
9. (0.999617): gm-suggestion.33078
10. (0.999616): and-while-im-at-it.8802
11. (0.999585): the-challenge-command-and-other-mapowner-feedback.7138
12. (0.999559): removal-of-1st-job-congratulations.11311
13. (0.999545): new-host.9284
14. (0.999513): pin-changes.2391
15. (0.999434): intentionally-spacing-fm-stores.38674
16. (0.999268): map-owner.3264
17. (0.999246): suggestion-site-light-spot.2925
18. (0.999238): fix-for-mapowner.3468
19. (0.999157): minor-community-poll-henesys.10136
20. (0.999146): better-navigation-in-ranking-page.24628

Spoiler: the rest

Top 20 posts for vote_econ

1. (0.999937): some-feedback-in-regards-to-the-gacha-revamp.6238
2. (0.999923): lets-solve-hp-washing.21020
3. (0.999905): make-reverse-weapons-tradable-even-after-equipping.36375
4. (0.999831): fixing-the-economy.15612
5. (0.999733): improve-gachapon.12470
6. (0.999719): chaos-white-scrolls.10620
7. (0.999635): suggestion-nx-rewards-for-enormous-effort.3021
8. (0.999527): suggestion-ap-reset.35814
9. (0.999470): alternative-possbility-to-messing-with-ap-reset.13495
10. (0.999252): repealing-the-trade-tax.14191
11. (0.999218): i-think-nx-cash-may-raise-for-one-vote.2921
12. (0.999183): change-ap-resets-to-1k-nx.36965
13. (0.998917): gachapon-problem.3858
14. (0.998838): add-maple-weapons-to-shiny-pearl-exchange-list.13547
15. (0.998830): remove-cash-gach-entirely.18284
16. (0.998790): scroll-for-hp-gain-increase.11671
17. (0.998625): gachapon-on-maple-island.36160
18. (0.998544): add-facestompers-and-stormcaster.24514
19. (0.998207): ap-resets.3044
20. (0.997339): random-lottery-nx.7336

Top 20 posts for questing

1. (0.999530): quest-level-restriction.10182
2. (0.998875): increase-apq-key-droprate-lower-apple-droprate.21823
3. (0.998631): slime-shoes.2364
4. (0.998497): buff-the-boss-card-drop-rates-for-mpq-opq.30549
5. (0.998014): orbis-party-quest-rewards-and-boss-card.21902
6. (0.997949): questss-exp.3347
7. (0.997905): enable-johns-quests-repeat.36971
8. (0.997562): adjust-mr-shims-request.29825
9. (0.997560): flashback-events.39957
10. (0.997216): raising-kpq-maximum-level-to-35.3265
11. (0.996941): pq-dummies.16893
12. (0.996838): balrog-pq.3553
13. (0.996307): suggestio-increase-the-drop-rate-of-dull-crystal.12029
14. (0.995847): make-monster-book-sellable-at-the-npc.36418
15. (0.995256): make-bundles-of-goby-the-etc-exchange-item-for-frankenroid.23600
16. (0.994674): card-drop-rate-on-anti-bot-mobs.38798
17. (0.994377): quests-in-progress.18131
18. (0.994122): needing-a-piece-of-crack-just-to-enter-pap.5118
19. (0.993948): lower-the-drop-rate-for-ores.2748
20. (0.992846): remove-the-boat-tickets-grendel-gives-in-the-messenger-of-the-fairy-forest.11265

Top 20 posts for balance

1. (0.999908): imminent-buccaneer-changes.31224
2. (0.999882): my-next-skill-changes-wish-list.29224
3. (0.999847): buff-non-fire-charges-maybe.28274
4. (0.999839): petition-for-buccaneer-super-transformation-to-have-90-stance.28896
5. (0.999754): making-parts-of-horntail-elementally-weak.38239
6. (0.999623): little-change-for-paladins-blast-skill-and-bandishs-hero-skill.24679
7. (0.999606): skills-balance-increase-shadow-stars-time.35273
8. (0.999527): mage-element-amplification.6312
9. (0.999443): yet-another-bucc-buff-post-perhaps-for-halloween-o-o.34051
10. (0.999371): fixing-elemental-composition-for-both-mages.7560
11. (0.999363): alternative-marksman-buff.30945
12. (0.999356): making-paladins-better-at-element-neutral-targets.13013
13. (0.999314): a-suggestion-im-not-sure-can-be-added-but-i-wanted-to-bring-up-sniper.38779
14. (0.999183): prevent-speed-infusion-from-being-overwritten-by-booster.32708
15. (0.999080): fix-enrage-buff-early-levels.24869
16. (0.999028): buff-level-1-blast.33586
17. (0.999015): explosion-casting-speed-dilemma.9353
18. (0.999000): strafe-the-good-and-the-bad.30705
19. (0.998930): feedback-regarding-pink-bean-balance-changes-and-some-suggestions.36323
20. (0.998835): a-reasonable-hex-of-the-beholder-buff.23359

Top 20 posts for cash_shop

1. (0.995659): list-of-nx-clothes.3584
2. (0.991836): my-buddy-whale.4028
3. (0.989661): white-seal-cushion.9959
4. (0.989106): a-request-for-a-face-and-a-hair.22210
5. (0.987857): more-donor-items-please.39205
6. (0.987799): suggestion-this-visor.3046
7. (0.986641): cash-item-equips-moved-to-cash-slot-in-item-inventory.34356
8. (0.982172): suggestion-cosmetic-items-and-donor-nx.9026
9. (0.981400): some-nx-weapons-not-rares.7700
10. (0.980549): cs-item-for-transparent-face-eyes-glasses.4394
11. (0.980355): request-pets-here.3542
12. (0.979532): wanted-some-new-hairstyles-for-a-cosplay.23383
13. (0.977321): cash-items.36324
14. (0.977292): bladed-falcons-katana-cash-item.24269
15. (0.974662): aran-skin-color.6578
16. (0.972966): remove-mont-blanc-hair-shadow.32132
17. (0.972958): nx-item-addition.4101
18. (0.970935): picture-thread-suggestion.2923
19. (0.968995): transparent-glasses.4432
20. (0.968987): unisex-hair-and-face.33425

Overall, the topics (except for the general server suggestions) seem cohesive at a glance.

I was curious what the posts that were assigned to multiple topics looked like. I took the topic vector (probabilities that the thread is part of a topic) and sorted them from distances to the unit vector.

Code:

-RECORD 0------------------------------------------------------------------------------
 thread_id | 52-use-slots-just-isnt-enough.4257                                   
 distance  | 0.18276617                                                           
 balance   | 0.25                                                                 
 cash_shop | 0.25                                                                 
 map_owner | 0.26                                                                 
 questing  | 0.05                                                                 
 vote_econ | 0.19                                                                 
-RECORD 1------------------------------------------------------------------------------
 thread_id | bunch-of-good-changes.35101                                           
 distance  | 0.18904829                                                           
 balance   | 0.15                                                                 
 cash_shop | 0.07                                                                 
 map_owner | 0.25                                                                 
 questing  | 0.22                                                                 
 vote_econ | 0.32                                                                 
-RECORD 2------------------------------------------------------------------------------
 thread_id | make-alt-lvl-100-weapons-usable-for-crafting-materials.34804         
 distance  | 0.19307376                                                           
 balance   | 0.22                                                                 
 cash_shop | 0.14                                                                 
 map_owner | 0.16                                                                 
 questing  | 0.13                                                                 
 vote_econ | 0.36                                                                 
-RECORD 3------------------------------------------------------------------------------
 thread_id | official-feedback-inventory-slot-upgrades.2814                       
 distance  | 0.20978586                                                           
 balance   | 0.02                                                                 
 cash_shop | 0.23                                                                 
 map_owner | 0.22                                                                 
 questing  | 0.25                                                                 
 vote_econ | 0.28                                                                 
-RECORD 4------------------------------------------------------------------------------
 thread_id | remove-ellin-pq-transformations.32745                                 
 distance  | 0.23225385                                                           
 balance   | 0.26                                                                 
 cash_shop | 0.2                                                                   
 map_owner | 0.24                                                                 
 questing  | 0.3                                                                   
 vote_econ | 0.0                                                                   
-RECORD 5------------------------------------------------------------------------------
 thread_id | whats-coming-up-next.9379                                             
 distance  | 0.23338255                                                           
 balance   | 0.0                                                                   
 cash_shop | 0.22                                                                 
 map_owner | 0.23                                                                 
 questing  | 0.3                                                                   
 vote_econ | 0.25                                                                 
-RECORD 6------------------------------------------------------------------------------
 thread_id | make-zakum-diamonds-stackable.4390                                   
 distance  | 0.23668121                                                           
 balance   | 0.3                                                                   
 cash_shop | 0.21                                                                 
 map_owner | 0.22                                                                 
 questing  | 0.27                                                                 
 vote_econ | 0.0                                                                   
-RECORD 7------------------------------------------------------------------------------
 thread_id | back-to-school-event.16132                                           
 distance  | 0.23884948                                                           
 balance   | 0.0                                                                   
 cash_shop | 0.24                                                                 
 map_owner | 0.31                                                                 
 questing  | 0.19                                                                 
 vote_econ | 0.25                                                                 
-RECORD 8------------------------------------------------------------------------------
 thread_id | community-poll-drop-filter-made-official.23286                       
 distance  | 0.23966599                                                           
 balance   | 0.0                                                                   
 cash_shop | 0.2                                                                   
 map_owner | 0.22                                                                 
 questing  | 0.3                                                                   
 vote_econ | 0.28                                                                 
-RECORD 9------------------------------------------------------------------------------
 thread_id | suggestion-add-level-50-mounts.5340                                   
 distance  | 0.24361697                                                           
 balance   | 0.0                                                                   
 cash_shop | 0.19                                                                 
 map_owner | 0.32                                                                 
 questing  | 0.27                                                                 
 vote_econ | 0.21                                                                 
-RECORD 10-----------------------------------------------------------------------------
 thread_id | petition-to-lower-level-cap-for-pumpkin-slices-halloween-event-2020.35149
 distance  | 0.24780038                                                           
 balance   | 0.2                                                                   
 cash_shop | 0.0                                                                   
 map_owner | 0.26                                                                 
 questing  | 0.2                                                                   
 vote_econ | 0.33                                                                 
-RECORD 11-----------------------------------------------------------------------------
 thread_id | compensation-for-the-120-bishops.6358                                 
 distance  | 0.24819347                                                           
 balance   | 0.31                                                                 
 cash_shop | 0.0                                                                   
 map_owner | 0.2                                                                   
 questing  | 0.19                                                                 
 vote_econ | 0.3                                                                   
-RECORD 12-----------------------------------------------------------------------------
 thread_id | allow-valentine-etc-to-be-stackable.28235                             
 distance  | 0.25478587                                                           
 balance   | 0.12                                                                 
 cash_shop | 0.09                                                                 
 map_owner | 0.28                                                                 
 questing  | 0.38                                                                 
 vote_econ | 0.12                                                                 
-RECORD 13-----------------------------------------------------------------------------
 thread_id | some-ideas.33217                                                     
 distance  | 0.25825295                                                           
 balance   | 0.14                                                                 
 cash_shop | 0.27                                                                 
 map_owner | 0.3                                                                   
 questing  | 0.29                                                                 
 vote_econ | 0.0                                                                   
-RECORD 14-----------------------------------------------------------------------------
 thread_id | stop-leveling-command.31847                                           
 distance  | 0.26122317                                                           
 balance   | 0.17                                                                 
 cash_shop | 0.0                                                                   
 map_owner | 0.36                                                                 
 questing  | 0.23                                                                 
 vote_econ | 0.24                                                                 
-RECORD 15-----------------------------------------------------------------------------
 thread_id | dks-endgame-spear-pa.3554                                             
 distance  | 0.26224607                                                           
 balance   | 0.18                                                                 
 cash_shop | 0.0                                                                   
 map_owner | 0.35                                                                 
 questing  | 0.18                                                                 
 vote_econ | 0.29                                                                 
-RECORD 16-----------------------------------------------------------------------------
 thread_id | november-events.6466                                                 
 distance  | 0.2654662                                                             
 balance   | 0.02                                                                 
 cash_shop | 0.21                                                                 
 map_owner | 0.12                                                                 
 questing  | 0.31                                                                 
 vote_econ | 0.34                                                                 
-RECORD 17-----------------------------------------------------------------------------
 thread_id | guy-fawkes-day-event.12798                                           
 distance  | 0.26782817                                                           
 balance   | 0.0                                                                   
 cash_shop | 0.37                                                                 
 map_owner | 0.18                                                                 
 questing  | 0.26                                                                 
 vote_econ | 0.19                                                                 
-RECORD 18-----------------------------------------------------------------------------
 thread_id | name-change-quest-reward-lower-price-nickname-in-expedition-feature.35545
 distance  | 0.26870826                                                           
 balance   | 0.0                                                                   
 cash_shop | 0.2                                                                   
 map_owner | 0.38                                                                 
 questing  | 0.2                                                                   
 vote_econ | 0.23                                                                 
-RECORD 19-----------------------------------------------------------------------------
 thread_id | unwelcome-visitor-event-pq.22637                                     
 distance  | 0.27275556                                                           
 balance   | 0.08                                                                 
 cash_shop | 0.11                                                                 
 map_owner | 0.12                                                                 
 questing  | 0.31                                                                 
 vote_econ | 0.38                                                                 
only showing top 20 rows

Spoiler: links

1. 52-use-slots-just-isnt-enough.4257
2. bunch-of-good-changes.35101
3. make-alt-lvl-100-weapons-usable-for-crafting-materials.34804
4. official-feedback-inventory-slot-upgrades.2814
5. remove-ellin-pq-transformations.32745
6. whats-coming-up-next.9379
7. make-zakum-diamonds-stackable.4390
8. back-to-school-event.16132
9. community-poll-drop-filter-made-official.23286
10. suggestion-add-level-50-mounts.5340
11. petition-to-lower-level-cap-for-pumpkin-slices-halloween-event-2020.35149
12. compensation-for-the-120-bishops.6358
13. allow-valentine-etc-to-be-stackable.28235
14. some-ideas.33217
15. stop-leveling-command.31847
16. dks-endgame-spear-pa.3554
17. november-events.6466
18. guy-fawkes-day-event.12798
19. name-change-quest-reward-lower-price-nickname-in-expedition-feature.35545
20. unwelcome-visitor-event-pq.22637

Not entirely sure what to make of these; the model seems to have a difficult time too.

Finally, I decided to take the output of the topic model and run this through a classifier. I ignore the open feedback section and focus on accepted and rejected (with not-possible going into rejected). The balance of labels looks like this:



I plot the principal components (PCA n=2) for the topics and get a nice plot of the model's geometry. This gives an idea of the space that the classifier will have to look at:



You can imagine a hyper-prism with 5 dimensions getting squashed into 2 dimensions. Topics are nicely distributed across each the corners (the paper describes this as a topic simplex).

Sticking it into a logistic regression model with default parameters via sklearn, I get an accuracy of 0.6 and an AUC of 0.63. Not great, but I'm sure there are other features that could help improve the model performance. We can evaluate this by taking a look at the ranking of the results. The number next to the thread is the likelihood of rejecting the thread.

First we look at the false positives, or the threads that were actually rejected but predicted to be accepted:

Spoiler

Rejected but predicted accepted

1. 0.387016 nx-items-suggestion.3904
2. 0.388747 invisible-nx-for-all-weapons.2032
3. 0.390621 removing-fat-lips-from-cash-shop.27308
4. 0.392867 donor-nx.11406
5. 0.395522 transparent-wand.5857
6. 0.402494 v-86-paladin-changes.4174
7. 0.404261 buff-axe-and-bw.25448
8. 0.409212 suggestion-for-new-cash-shop-throwing-stars.5594
9. 0.413070 nerf-the-new-capes.24199
10. 0.432610 gold-and-purple-lvl-15-and-95-surfboard.8994
11. 0.441399 taking-the-big-out-of-big-bang.22566
12. 0.446905 bigfoots-hitbox.27681
13. 0.448806 add-trihard-as-a-discord-emote.13026
14. 0.450065 ufo-event.4219
15. 0.451920 make-ciders-stackable.16014
16. 0.457394 offensive-cashshop-content-rant.15655
17. 0.458367 assassin-attacks.23365
18. 0.459343 monster-level.3353
19. 0.464498 donating-idea.2655
20. 0.471050 hair-style-npc.12481

Many of these are cash shop or balance related. Next are the false negatives, or the things that were accepted but predicted rejected:

Spoiler

Accepted but predicted rejection

1. 0.677543 some-feedback-in-regards-to-the-gacha-revamp.6238
2. 0.677388 remove-cash-gach-entirely.18284
3. 0.677144 increasing-vote-streak-nx-bonus.24902
4. 0.677011 the-removal-of-unwanted-untradable-potions-from-gachapon.11913
5. 0.676657 constant-votes.22928
6. 0.673812 gachapon.8748
7. 0.669772 a-discussion-on-the-economic-systemic-influences-of-gachapon.3270
8. 0.669462 maple-items.2160
9. 0.668836 suggestion-allow-gifts-on-cs.2948
10. 0.666954 feedback-gtop100-changes.23684
11. 0.664727 gachapon-pots-pies-chance-reduced.4143
12. 0.662895 suggestion-how-to-reduce-hp-washing.20965
13. 0.662237 slightly-buff-legendary-items-from-gachapon.20243
14. 0.660754 another-hp-equip-horntail-req.22169
15. 0.660299 reduce-summer-raffle-cost.26142
16. 0.660239 s-ap-reset-in-a-bundle.24835
17. 0.659651 omok-points-rating-fix.16693
18. 0.657913 a-better-server.5238
19. 0.652005 transparency-on-event-exchange-removal.34797
20. 0.648531 change-voting-to-once-per-day-rather-than-once-every-24h.14805

Many of these have to do with voting and raffles/gacha.

Finally, let's look at the threads that might be accepted based on this model:

Most likely to be accepted

1. 0.384090 white-seal-cushion.9959
2. 0.384197 a-request-for-a-face-and-a-hair.22210
3. 0.384436 more-donor-items-please.39205
4. 0.385066 request-pets-here.3542
5. 0.385527 suggestion-cosmetic-items-and-donor-nx.9026
6. 0.385675 some-nx-weapons-not-rares.7700
7. 0.386035 wanted-some-new-hairstyles-for-a-cosplay.23383
8. 0.386460 cash-items.36324
9. 0.386467 bladed-falcons-katana-cash-item.24269
10. 0.387296 remove-mont-blanc-hair-shadow.32132
11. 0.387304 susanoo-donor-effect.32048
12. 0.388061 unisex-hair-and-face.33425
13. 0.389145 guild-leadership-transfer-with-no-cost.30458
14. 0.389464 suggestion-change-elf-ear-item-to-earring-or-eye-slot.36102
15. 0.389792 stirgeman-items-not-working.19058
16. 0.389850 can-we-label-speargrave-flairgraves-by-weap-type-in-library.27137
17. 0.390619 7631
18. 0.391644 tricolor-corgi-pup.24358
19. 0.392729 instead-of-crossbows.7356
20. 0.392893 bloody-jeanne-unisex.26454

These are almost all cash-shop requests. It kind of makes sense, since cash shop items are relatively simple compared to things like massive balance overhauls to the game. These are somewhat tedious to do from my understanding of WZ editing though.

Here are the threads that are most likely to be rejected according to the model.

Most likely to be rejected:

1. 0.677541 lets-solve-hp-washing.21020
2. 0.677539 make-reverse-weapons-tradable-even-after-equipping.36375
3. 0.677528 fixing-the-economy.15612
4. 0.677514 improve-gachapon.12470
5. 0.677512 chaos-white-scrolls.10620
6. 0.677486 suggestion-ap-reset.35814
7. 0.677477 alternative-possbility-to-messing-with-ap-reset.13495
8. 0.677447 repealing-the-trade-tax.14191
9. 0.677437 change-ap-resets-to-1k-nx.36965
10. 0.677382 scroll-for-hp-gain-increase.11671
11. 0.677359 gachapon-on-maple-island.36160
12. 0.677348 add-facestompers-and-stormcaster.24514
13. 0.677178 random-lottery-nx.7336
14. 0.677057 islander-announcements.6328
15. 0.676817 extend-expiration-date-for-halloween-rings.40276
16. 0.676751 blackfist-cloak.34739
17. 0.676497 make-the-event-ap-resets-tradeable.30519
18. 0.676381 exclude-scrolled-gear-from-being-used-as-crafting-mats.31747
19. 0.676091 extend-the-summer-event-due-to-ddos-attacks.13618
20. 0.675904 npc-that-sells-ws-cs.40260

Many of these suggestions are controversial, and involve changes to the economy or voting.

I wouldn't read deeply into the logistic regression model, since the accuracy on the training set was low. It's possible that categorizing some of the open threads into accepted or rejected would help improve performance though. The results for the top 20 and bottom 20 seem to align with my [limited] understanding of suggestions.

I hope this has been interesting; I enjoyed seeing the suggestions laid out this like by topics instead of by the default forum view. If I had access to my desktop (with my GPU), one improvement I would make would be to include summaries of each of threads. I'll probably follow up with some other visualizations as I think of them, but I think this is a solid foundation for other explorations. I'll also upload my notebooks somewhere that's visible (since the repo is private).

Let me know if you have ideas or want me to dig deeper into a point of interest.

 

There are plenty of other features that I suspect aren't linearly related: the counts of different reactions, the number of replies, who has replied (staff or not), whether the author has a long post history, etc.. Using the topic features as part of a regression model seems like one way of making it generally less abstract

Bigrams are a good idea; I'll probably try this as my next follow up. The trickiest part of incorporating this are the useless pairs of adjacent words, but a good excuse to try out more of the NLTK features. It would be great to catch things like "hp washing" in the topics though.

 

This is ambiguous, since it could be read as no responses from staff as opposed to no responses in general. I've included a list of both using accept/reject predictions to break ties (topic features using 5 topics, fed into a logistic regression model). The total number of likes is the sum from all of the posts in the thread.

Top reacted suggestions

1. [p-reject 0.71 | like x 1155 | reply x 280] making-hp-washing-truly-optional-a-solution.39773
2. [p-reject 0.68 | like x 1066 | reply x 240] storage-on-maple-island.36458
3. [p-reject 0.71 | like x 986 | reply x 160] manifesto-on-the-3-main-problems-plaguing-the-server.37538
4. [p-reject 0.69 | like x 888 | reply x 300] formal-proposal-questers-ring.29962
5. [p-reject 0.52 | like x 741 | reply x 500] ask-for-new-cash-items-here.7418
6. [p-reject 0.60 | like x 390 | reply x 80] making-towns-henesys-more-alive.39232
7. [p-reject 0.57 | like x 377 | reply x 100] class-balance-should-warriors-be-viable.36968
8. [p-reject 0.67 | like x 347 | reply x 180] potential-wipe-bishop-changes-discussion-thread.8986
9. [p-reject 0.65 | like x 293 | reply x 60] make-horntail-1-run-per-day.38022
10. [p-reject 0.58 | like x 288 | reply x 100] change-meso-up-from-being-a-party-buff-to-a-self-buff.39574
11. [p-reject 0.59 | like x 279 | reply x 60] make-bossing-great-again.38244
12. [p-reject 0.64 | like x 262 | reply x 60] the-current-state-of-affairs.8554
13. [p-reject 0.67 | like x 254 | reply x 20] now-that-pink-bean-is-released-there-needs-to-be-more-sources-of-hp-available.35255
14. [p-reject 0.59 | like x 249 | reply x 100] maplelegends-future-economy-solution.18348
15. [p-reject 0.70 | like x 248 | reply x 80] the-hp-buff-item-in-depth.36925
16. [p-reject 0.63 | like x 237 | reply x 120] agent-earrings.19913
17. [p-reject 0.69 | like x 235 | reply x 60] bring-back-summoning-sacks.15270
18. [p-reject 0.56 | like x 227 | reply x 100] warrior-balanced-change-thread-all.36721
19. [p-reject 0.69 | like x 213 | reply x 100] your-feedback-and-experience-of-maplelegends.35702
20. [p-reject 0.59 | like x 208 | reply x 80] actually-balance-the-game.28136

Spoiler

1. [p-reject 0.71 | like x 1155 | reply x 280] making-hp-washing-truly-optional-a-solution.39773
2. [p-reject 0.68 | like x 1066 | reply x 240] storage-on-maple-island.36458
3. [p-reject 0.71 | like x 986 | reply x 160] manifesto-on-the-3-main-problems-plaguing-the-server.37538
4. [p-reject 0.69 | like x 888 | reply x 300] formal-proposal-questers-ring.29962
5. [p-reject 0.52 | like x 741 | reply x 500] ask-for-new-cash-items-here.7418
6. [p-reject 0.60 | like x 390 | reply x 80] making-towns-henesys-more-alive.39232
7. [p-reject 0.57 | like x 377 | reply x 100] class-balance-should-warriors-be-viable.36968
8. [p-reject 0.67 | like x 347 | reply x 180] potential-wipe-bishop-changes-discussion-thread.8986
9. [p-reject 0.65 | like x 293 | reply x 60] make-horntail-1-run-per-day.38022
10. [p-reject 0.58 | like x 288 | reply x 100] change-meso-up-from-being-a-party-buff-to-a-self-buff.39574
11. [p-reject 0.59 | like x 279 | reply x 60] make-bossing-great-again.38244
12. [p-reject 0.64 | like x 262 | reply x 60] the-current-state-of-affairs.8554
13. [p-reject 0.67 | like x 254 | reply x 20] now-that-pink-bean-is-released-there-needs-to-be-more-sources-of-hp-available.35255
14. [p-reject 0.59 | like x 249 | reply x 100] maplelegends-future-economy-solution.18348
15. [p-reject 0.70 | like x 248 | reply x 80] the-hp-buff-item-in-depth.36925
16. [p-reject 0.63 | like x 237 | reply x 120] agent-earrings.19913
17. [p-reject 0.69 | like x 235 | reply x 60] bring-back-summoning-sacks.15270
18. [p-reject 0.56 | like x 227 | reply x 100] warrior-balanced-change-thread-all.36721
19. [p-reject 0.69 | like x 213 | reply x 100] your-feedback-and-experience-of-maplelegends.35702
20. [p-reject 0.59 | like x 208 | reply x 80] actually-balance-the-game.28136
21. [p-reject 0.57 | like x 199 | reply x 40] convert-scar-targa-helms-to-minimum-1-drop.34971
22. [p-reject 0.70 | like x 193 | reply x 120] make-it-fair-for-those-unable-to-multi-account-vote-12-2-2019.27075
23. [p-reject 0.71 | like x 167 | reply x 140] quest-ring-0.24384
24. [p-reject 0.69 | like x 165 | reply x 80] we-need-a-brainstorm-thread-to-make-the-island-more-active.7330
25. [p-reject 0.71 | like x 163 | reply x 80] make-hp-washing-optional.17813
26. [p-reject 0.56 | like x 149 | reply x 20] dunas.35894
27. [p-reject 0.68 | like x 148 | reply x 40] selling-map.37718
28. [p-reject 0.58 | like x 143 | reply x 40] revamping-horntail-drop-table.39667
29. [p-reject 0.70 | like x 139 | reply x 18] a-second-look-storage-on-maple-island.38350
30. [p-reject 0.58 | like x 138 | reply x 80] neo-tokyo-bosses-droprates.26458
31. [p-reject 0.50 | like x 131 | reply x 40] ability-to-mute-some-all-smegas.31788
32. [p-reject 0.67 | like x 131 | reply x 40] ensuring-new-players-know-tos-maple-island-quizzes.36715
33. [p-reject 0.39 | like x 130 | reply x 60] petition-for-buccaneer-super-transformation-to-have-90-stance.28896
34. [p-reject 0.67 | like x 130 | reply x 60] modify-neo-tokyo-staves-and-wands.32433
35. [p-reject 0.68 | like x 129 | reply x 60] suggestion-make-halloween-monster-sacks-useable-in-towns-and-non-mob-maps-only.16850
36. [p-reject 0.50 | like x 128 | reply x 80] dark-knight-bitching-thread.27014
37. [p-reject 0.58 | like x 127 | reply x 80] lets-talk-about-heroes.15957
38. [p-reject 0.69 | like x 127 | reply x 40] area-bosses-limited-entry-mini-dungeons-anego-samurai-bf.38290
39. [p-reject 0.60 | like x 125 | reply x 100] poll-2007-ms-nx-vs-private-server-nx.6315
40. [p-reject 0.65 | like x 125 | reply x 40] shaolin-5-6f-eph-data-and-suggestions-for-new-changes.36073
41. [p-reject 0.45 | like x 122 | reply x 40] making-parts-of-horntail-elementally-weak.38239
42. [p-reject 0.68 | like x 121 | reply x 100] suggestion-lower-the-price-of-ap-resets-updated.11327
43. [p-reject 0.70 | like x 117 | reply x 40] request-to-re-open-summoning-sack-mayhem-contest.16917
44. [p-reject 0.70 | like x 117 | reply x 40] mapownership.37621
45. [p-reject 0.71 | like x 117 | reply x 120] self-bishop-leech.10685
46. [p-reject 0.58 | like x 115 | reply x 20] make-reverse-weapons-tradable-even-after-equipping.36375
47. [p-reject 0.64 | like x 111 | reply x 80] golden-gach-tickets-a-new-way-to-obtain-ap-resets.13491
48. [p-reject 0.52 | like x 104 | reply x 40] make-axe-viable-again.27213
49. [p-reject 0.65 | like x 104 | reply x 60] luk-mage-avoidability-buff.30091
50. [p-reject 0.60 | like x 102 | reply x 20] pink-bean-and-its-current-state.36123
51. [p-reject 0.54 | like x 101 | reply x 40] marksmen-should-be-buffed-further.30704
52. [p-reject 0.60 | like x 97 | reply x 20] removing-the-whiting-system.35836
53. [p-reject 0.59 | like x 95 | reply x 40] adding-facestompers-and-making-red-socks-permanent.13477
54. [p-reject 0.39 | like x 88 | reply x 160] skill-balances-for-all-jobs.24624
55. [p-reject 0.57 | like x 87 | reply x 20] opq-buffs-to-compete-with-gs.35363
56. [p-reject 0.64 | like x 86 | reply x 17] islander-weddings.14424
57. [p-reject 0.69 | like x 85 | reply x 40] deep-ludibrium-teleport-portals.35807
58. [p-reject 0.71 | like x 85 | reply x 40] lets-solve-hp-washing.21020
59. [p-reject 0.58 | like x 83 | reply x 20] ws-drops-from-end-game-bosses.38464
60. [p-reject 0.70 | like x 83 | reply x 6] ring-for-hp-scroll.39945
61. [p-reject 0.59 | like x 82 | reply x 40] a-few-suggestions-to-revive-the-drks-class.24113
62. [p-reject 0.65 | like x 82 | reply x 20] christmas-2020-feedback.36784
63. [p-reject 0.60 | like x 81 | reply x 17] nerf-veetron-drop-rate.39468
64. [p-reject 0.52 | like x 79 | reply x 60] make-the-quest-bartols-request-repeatable.3400
65. [p-reject 0.62 | like x 79 | reply x 20] make-mw20-drop-from-zak-mw30-drop-from-ht.13679
66. [p-reject 0.71 | like x 79 | reply x 60] thoughts-and-suggestions-on-washing-mechanic.36535
67. [p-reject 0.69 | like x 78 | reply x 20] more-channels.32992
68. [p-reject 0.71 | like x 78 | reply x 80] hp-washing-solution.24832
69. [p-reject 0.68 | like x 77 | reply x 40] re-balance-cds-or-buff-other-relevant-content.33606
70. [p-reject 0.69 | like x 77 | reply x 20] a-solution-for-mages.39589
71. [p-reject 0.69 | like x 75 | reply x 40] poll-islander.7359
72. [p-reject 0.58 | like x 74 | reply x 40] timeless-reverse-weapons-balancing-issue.35084
73. [p-reject 0.70 | like x 74 | reply x 20] ban-multi-client-at-pianus.36880
74. [p-reject 0.71 | like x 74 | reply x 12] delete-hp-mp-washing-system-and-forbid-multiclient.38227
75. [p-reject 0.38 | like x 71 | reply x 60] my-next-skill-changes-wish-list.29224
76. [p-reject 0.57 | like x 71 | reply x 15] repopulating-party-quests.38540
77. [p-reject 0.60 | like x 71 | reply x 15] a-solution-for-bad-mage-nerf-threads.39592
78. [p-reject 0.59 | like x 70 | reply x 20] a-better-mark-of-the-squad.30503
79. [p-reject 0.59 | like x 70 | reply x 20] how-to-include-monster-carnival-in-maplelegends.27789
80. [p-reject 0.66 | like x 69 | reply x 20] remove-middle-shelves-at-shaolin-5-6.31066
81. [p-reject 0.69 | like x 69 | reply x 20] questing-ring.22415
82. [p-reject 0.51 | like x 68 | reply x 20] buff-marksmen.27868
83. [p-reject 0.56 | like x 67 | reply x 20] reduce-cwk-signs-to-one-per-job.39450
84. [p-reject 0.58 | like x 66 | reply x 40] paladin-changes.4133
85. [p-reject 0.67 | like x 66 | reply x 20] tos-suggestion-account-sharing-and-aggro-abuse-should-not-be-the-same-ban-sequence.36009
86. [p-reject 0.70 | like x 66 | reply x 20] mapowner.30557
87. [p-reject 0.59 | like x 65 | reply x 18] npc-to-convert-standard-equips-into-cosmetics.40023
88. [p-reject 0.55 | like x 64 | reply x 13] scrolling-ability-should-not-autotick-ws-use.38490
89. [p-reject 0.70 | like x 64 | reply x 20] remove-mapowner-automatically-if-the-person-who-has-the-map-is-dead.39649
90. [p-reject 0.52 | like x 63 | reply x 60] transparent-knuckles-new-faces-rare-nx-for-donors.7353
91. [p-reject 0.56 | like x 63 | reply x 40] dragon-blood-improvement.25709
92. [p-reject 0.66 | like x 62 | reply x 40] nx-clothes-cancel-gender-dichotomy.7212
93. [p-reject 0.69 | like x 62 | reply x 11] this-hurts-to-see.13583
94. [p-reject 0.61 | like x 61 | reply x 20] shaolin-spawn-rates.37156
95. [p-reject 0.70 | like x 60 | reply x 20] mapowner-timeouts-and-abuse-of-the-system.32870
96. [p-reject 0.71 | like x 60 | reply x 20] blood-wash-should-not-be-penalized-with-max-hp-loss.40148
97. [p-reject 0.43 | like x 59 | reply x 20] skills-balance-increase-shadow-stars-time.35273
98. [p-reject 0.45 | like x 59 | reply x 7] making-sense-of-mages-visual-clutter-summons-and-big-bang.39475
99. [p-reject 0.56 | like x 59 | reply x 20] the-real-warrior-cleave-balance-we-need.37020
100. [p-reject 0.64 | like x 58 | reply x 14] castellan-toad-x2.33355

Top reacted suggestions with less than 3 replies

1. [p-reject 0.71 | like x 21 | reply x 2] wedding-ring-as-nx-item.31995
2. [p-reject 0.56 | like x 20 | reply x 3] add-spinel-to-el-nath-aka-spinel-nath.35909
3. [p-reject 0.58 | like x 18 | reply x 3] neo-tokyo-exchange-quest.36649
4. [p-reject 0.61 | like x 18 | reply x 3] make-alt-lvl-100-weapons-usable-for-crafting-materials.34804
5. [p-reject 0.55 | like x 17 | reply x 3] make-ntpq-easier.26308
6. [p-reject 0.54 | like x 16 | reply x 3] can-we-label-speargrave-flairgraves-by-weap-type-in-library.27137
7. [p-reject 0.65 | like x 15 | reply x 3] improve-aggro-system-at-map-of-zipangu-2100-odaiba.38327
8. [p-reject 0.71 | like x 15 | reply x 1] extend-the-summer-event-due-to-ddos-attacks.13618
9. [p-reject 0.47 | like x 14 | reply x 2] unpopular-opinion.37755
10. [p-reject 0.66 | like x 14 | reply x 2] change-pink-bean-log-out-log-in-mechanics-to-match-horntail.36504
11. [p-reject 0.48 | like x 13 | reply x 3] party-quest-npc-tweak.25236
12. [p-reject 0.49 | like x 13 | reply x 1] implement-a-rankings-board-for-jump-quests-in-game-and-boss-expeditions-party-quests-possibly.31799
13. [p-reject 0.42 | like x 12 | reply x 1] add-mana-reflection-damage-to-dpm-output.38013
14. [p-reject 0.50 | like x 12 | reply x 3] suggestion-blind-works-as-hamstring.25873
15. [p-reject 0.50 | like x 12 | reply x 3] put-monsterbook-cards-in-the-item-ignore-pendant-for-pets.32269
16. [p-reject 0.53 | like x 12 | reply x 3] shop-improvement-to-prevent-mistyping.34973
17. [p-reject 0.55 | like x 12 | reply x 3] neo-tokyo-dialoge-nerf.34903
18. [p-reject 0.63 | like x 12 | reply x 1] make-leaving-party-in-cwk-after-passing-leader-safe.37085
19. [p-reject 0.39 | like x 11 | reply x 3] aesthetic-and-quality-of-life-changes-for-dk.22120
20. [p-reject 0.49 | like x 11 | reply x 1] use-a-date-format-for-in-game-screenshots-that-sorts-in-chronological-order.32843

Spoiler

1. [p-reject 0.71 | like x 21 | reply x 2] wedding-ring-as-nx-item.31995
2. [p-reject 0.56 | like x 20 | reply x 3] add-spinel-to-el-nath-aka-spinel-nath.35909
3. [p-reject 0.58 | like x 18 | reply x 3] neo-tokyo-exchange-quest.36649
4. [p-reject 0.61 | like x 18 | reply x 3] make-alt-lvl-100-weapons-usable-for-crafting-materials.34804
5. [p-reject 0.55 | like x 17 | reply x 3] make-ntpq-easier.26308
6. [p-reject 0.54 | like x 16 | reply x 3] can-we-label-speargrave-flairgraves-by-weap-type-in-library.27137
7. [p-reject 0.65 | like x 15 | reply x 3] improve-aggro-system-at-map-of-zipangu-2100-odaiba.38327
8. [p-reject 0.71 | like x 15 | reply x 1] extend-the-summer-event-due-to-ddos-attacks.13618
9. [p-reject 0.47 | like x 14 | reply x 2] unpopular-opinion.37755
10. [p-reject 0.66 | like x 14 | reply x 2] change-pink-bean-log-out-log-in-mechanics-to-match-horntail.36504
11. [p-reject 0.48 | like x 13 | reply x 3] party-quest-npc-tweak.25236
12. [p-reject 0.49 | like x 13 | reply x 1] implement-a-rankings-board-for-jump-quests-in-game-and-boss-expeditions-party-quests-possibly.31799
13. [p-reject 0.42 | like x 12 | reply x 1] add-mana-reflection-damage-to-dpm-output.38013
14. [p-reject 0.50 | like x 12 | reply x 3] suggestion-blind-works-as-hamstring.25873
15. [p-reject 0.50 | like x 12 | reply x 3] put-monsterbook-cards-in-the-item-ignore-pendant-for-pets.32269
16. [p-reject 0.53 | like x 12 | reply x 3] shop-improvement-to-prevent-mistyping.34973
17. [p-reject 0.55 | like x 12 | reply x 3] neo-tokyo-dialoge-nerf.34903
18. [p-reject 0.63 | like x 12 | reply x 1] make-leaving-party-in-cwk-after-passing-leader-safe.37085
19. [p-reject 0.39 | like x 11 | reply x 3] aesthetic-and-quality-of-life-changes-for-dk.22120
20. [p-reject 0.49 | like x 11 | reply x 1] use-a-date-format-for-in-game-screenshots-that-sorts-in-chronological-order.32843
21. [p-reject 0.50 | like x 11 | reply x 1] qol-automatically-stack-items-in-storage.40200
22. [p-reject 0.61 | like x 11 | reply x 2] request-to-have-a-built-in-function-for-mesos-per-minute-hour.36606
23. [p-reject 0.30 | like x 10 | reply x 1] paladins-elemental-charge-animation.38070
24. [p-reject 0.48 | like x 10 | reply x 3] pickpoket.35785
25. [p-reject 0.51 | like x 10 | reply x 3] reverting-changes.30709
26. [p-reject 0.53 | like x 10 | reply x 1] remove-pickpocket-cap-of-200.23928
27. [p-reject 0.54 | like x 10 | reply x 3] guild-chair-as-reward-for-guild-point-ranking.24516
28. [p-reject 0.60 | like x 10 | reply x 2] alternative-sources-of-raw-mesos-entering-the-ml-economy-bossing.32902
29. [p-reject 0.63 | like x 10 | reply x 1] what-to-do-with-hunting-gach-tickets.8567
30. [p-reject 0.70 | like x 10 | reply x 1] allow-unlimited-people-to-enter-manons-forest.30264
31. [p-reject 0.48 | like x 9 | reply x 2] suggestion-cosmetic-items-and-donor-nx.9026
32. [p-reject 0.51 | like x 9 | reply x 3] discord-donor-rank.18975
33. [p-reject 0.57 | like x 9 | reply x 3] gender-restrictions.25432
34. [p-reject 0.58 | like x 9 | reply x 2] change-the-timeless-weapon-crafting-recipe-to-consume-reverse-weapon-instead-of-nt-weapons.36497
35. [p-reject 0.59 | like x 9 | reply x 3] cpq-where-you-at-fam.28313
36. [p-reject 0.60 | like x 9 | reply x 1] a-simple-qol-change-drops-in-boss.39296
37. [p-reject 0.62 | like x 9 | reply x 1] make-full-monster-cards-unpickable.22211
38. [p-reject 0.67 | like x 9 | reply x 3] add-nx-consumables-to-gift-point-shop.36291
39. [p-reject 0.69 | like x 9 | reply x 3] defense-scroll-buff.24822
40. [p-reject 0.71 | like x 9 | reply x 3] easter-event.39810
41. [p-reject 0.37 | like x 8 | reply x 3] can-we-buff-dunas-exp-again.28874
42. [p-reject 0.51 | like x 8 | reply x 1] regarding-perma-beginners-buffs.33918
43. [p-reject 0.52 | like x 8 | reply x 1] turning-pets-permanent-with-donor-cash.26084
44. [p-reject 0.52 | like x 8 | reply x 1] add-command-to-turn-off-voting-reminders.23085
45. [p-reject 0.53 | like x 8 | reply x 2] some-balancing-suggestions.28868
46. [p-reject 0.54 | like x 8 | reply x 3] add-craftable-by-tab-for-craftable-weapons-items-in-the-library.31236
47. [p-reject 0.56 | like x 8 | reply x 2] bunch-of-good-changes.35101
48. [p-reject 0.63 | like x 8 | reply x 1] revisiting-the-nullexp-command.34585
49. [p-reject 0.66 | like x 8 | reply x 3] scisors-of-karma-like-item.29389
50. [p-reject 0.67 | like x 8 | reply x 3] to-magnify-sense-of-accomplishment.31716
51. [p-reject 0.44 | like x 7 | reply x 1] pop-out-smegas.30322
52. [p-reject 0.48 | like x 7 | reply x 3] make-shop-slots-able-to-sell-more-than-200-items-per-stack.35154
53. [p-reject 0.49 | like x 7 | reply x 2] colored-item-list.35661
54. [p-reject 0.50 | like x 7 | reply x 3] using-donor-pts-to-make-owned-pets-permanent.37728
55. [p-reject 0.52 | like x 7 | reply x 2] name-change-quest-reward-lower-price-nickname-in-expedition-feature.35545
56. [p-reject 0.54 | like x 7 | reply x 3] old-fm-shops.6434
57. [p-reject 0.55 | like x 7 | reply x 3] use-other-identifier-instead-of-loginid-for-voting-website.38507
58. [p-reject 0.62 | like x 7 | reply x 2] pet-equips.26939
59. [p-reject 0.36 | like x 6 | reply x 3] make-a-skill-usable-by-double-clicking-it-on-the-quick-slot-window.26709
60. [p-reject 0.42 | like x 6 | reply x 1] decrease-xbow-arrow-costs-to-match-bow-arrow-costs-please.36763
61. [p-reject 0.48 | like x 6 | reply x 2] searching-guild-members-in-website-rankings.12086
62. [p-reject 0.48 | like x 6 | reply x 3] cash-items.36324
63. [p-reject 0.56 | like x 6 | reply x 3] more-donor-items-please.39205
64. [p-reject 0.57 | like x 6 | reply x 1] allowing-all-event-chairs-to-be-tradable.31607
65. [p-reject 0.57 | like x 6 | reply x 1] nx-item-suggestions.6948
66. [p-reject 0.58 | like x 6 | reply x 1] make-amorian-baskets-stack.24746
67. [p-reject 0.58 | like x 6 | reply x 3] glittering-altaire-earrings.26767
68. [p-reject 0.59 | like x 6 | reply x 1] add-30-more-seconds-for-cwkpq-bonus-stage.33888
69. [p-reject 0.60 | like x 6 | reply x 1] increase-level-cap-for-magatia-pq.24439
70. [p-reject 0.61 | like x 6 | reply x 2] summer-events-beach-slippers.26145
71. [p-reject 0.64 | like x 6 | reply x 1] fun-surprise-gm-mob-boss-spawn-event-like-neckson-did.30921
72. [p-reject 0.65 | like x 6 | reply x 2] oven-mitten-request.10772
73. [p-reject 0.70 | like x 6 | reply x 2] map-rebalance.21164
74. [p-reject 0.33 | like x 5 | reply x 3] bucc-make-snatch-usable-while-jumping.27745
75. [p-reject 0.37 | like x 5 | reply x 2] database-oblivion-mob-of-temple-of-time-weakness.38980
76. [p-reject 0.39 | like x 5 | reply x 1] mounts-in-pink-bean.36575
77. [p-reject 0.48 | like x 5 | reply x 3] client-mod-soft-hard-keyboard-pin-keybind.30689
78. [p-reject 0.48 | like x 5 | reply x 2] make-npc-chat-better.36496
79. [p-reject 0.48 | like x 5 | reply x 1] change-the-colour-of-alliance-members-on-the-minimap-to-green.30617
80. [p-reject 0.49 | like x 5 | reply x 3] suggestion.8485
81. [p-reject 0.49 | like x 5 | reply x 2] changing-item-sorting-in-inventory.24293
82. [p-reject 0.51 | like x 5 | reply x 3] suggestion-change-elf-ear-item-to-earring-or-eye-slot.36102
83. [p-reject 0.52 | like x 5 | reply x 3] buyback-feature.36734
84. [p-reject 0.54 | like x 5 | reply x 2] buff-flairgrave-dagger.35128
85. [p-reject 0.54 | like x 5 | reply x 2] gpq-bonus-stg-remove-kick-status.28219
86. [p-reject 0.55 | like x 5 | reply x 2] giving-the-option-to-craft-blue-dragon-shield-timeless-kite-shield.39527
87. [p-reject 0.56 | like x 5 | reply x 3] add-stirgeman-utility-pants-to-exchange-rewards.31233
88. [p-reject 0.57 | like x 5 | reply x 3] request-to-have-ellin-pq-reward-exp-in-a-graded-fashion.36596
89. [p-reject 0.57 | like x 5 | reply x 3] request-for-owl-of-minerva-to-report-summary-statistics-particularly-measures-of-centre-and-spread.36619
90. [p-reject 0.59 | like x 5 | reply x 3] the-super-chair-something-for-the-campers.33674
91. [p-reject 0.60 | like x 5 | reply x 2] couple-rings-purchasable-with-donor-nx.20650
92. [p-reject 0.61 | like x 5 | reply x 1] add-spectator-mode-to-mini-games.36908
93. [p-reject 0.62 | like x 5 | reply x 2] allow-beginner-in-pink-bean-event-pq.31033
94. [p-reject 0.63 | like x 5 | reply x 3] add-topwear-bottomwear-hp-scrolls-to-gachapon.36159
95. [p-reject 0.71 | like x 5 | reply x 3] buffffffff-olaf-and-instructorrrrrrrrr.20580
96. [p-reject 0.34 | like x 4 | reply x 1] duey-prime.35858
97. [p-reject 0.34 | like x 4 | reply x 2] increase-the-range-of-buccs-snatch.36190
98. [p-reject 0.43 | like x 4 | reply x 3] guardian-skill-change.24741
99. [p-reject 0.44 | like x 4 | reply x 2] put-a-cd-on-herons-skill.36556
100. [p-reject 0.47 | like x 4 | reply x 3] qol-in-game-window-locations-saving-resetting.28128

 

I played with this a bit more. Instead of feeding the threads into a topic model and then into a classification model, I threw the word count features directly a naive bayes (0.65 accuracy) and a factorization machine (0.79 accuracy) classifier. The first considers the effects of each word independently on whether it's accepted or rejected; the latter uses interactions between all pairs to make the decision. I was surprised by how effective the latter was on the train/test set since it improved by 0.2.

I took a look at the results of the factorization machine:

Code:

+----------+-----+
|prediction|count|
+----------+-----+
|       0.0| 1939|
|       1.0|  123|
+----------+-----+

Only 123 results predicted to be rejected? I must be doing something wrong...

Spoiler: results

Most likely to be accepted

1. 0.000000 skill-balances-for-all-jobs.24624
2. 0.000000 class-balance-should-warriors-be-viable.36968
3. 0.000000 warrior-balanced-change-thread-all.36721
4. 0.000001 change-meso-up-from-being-a-party-buff-to-a-self-buff.39574
5. 0.000003 neo-tokyo-bosses-droprates.26458
6. 0.000005 ask-for-new-cash-items-here.7418
7. 0.000009 thoughts-and-suggestions-on-washing-mechanic.36535
8. 0.000015 yeah-mpq-it%E2%80%99s-pretty-obscure-you%E2%80%99ve-probably-never-heard-of-it%E2%80%A6.34976
9. 0.000018 actually-balance-the-game.28136
10. 0.000026 marksmen-should-be-buffed-further.30704
11. 0.000031 dark-knight-bitching-thread.27014
12. 0.000050 manifesto-on-the-3-main-problems-plaguing-the-server.37538
13. 0.000054 the-hp-buff-item-in-depth.36925
14. 0.000057 shaolin-5-6f-eph-data-and-suggestions-for-new-changes.36073
15. 0.000060 imminent-buccaneer-changes.31224
16. 0.000095 make-bossing-great-again.38244
17. 0.000099 luk-mage-avoidability-buff.30091
18. 0.000121 christmas-2020-feedback.36784
19. 0.000131 on-the-recent-shad-buffs.26904
20. 0.000167 modify-neo-tokyo-staves-and-wands.32433
21. 0.000195 lets-talk-about-heroes.15957
22. 0.000225 timeless-reverse-weapons-balancing-issue.35084
23. 0.000250 re-balance-cds-or-buff-other-relevant-content.33606
24. 0.000277 my-next-skill-changes-wish-list.29224
25. 0.000277 paladin-changes.4133
26. 0.000329 make-reverse-weapons-tradable-even-after-equipping.36375
27. 0.000449 dragon-blood-improvement.25709
28. 0.000534 formal-proposal-questers-ring.29962
29. 0.000661 a-few-suggestions-to-revive-the-drks-class.24113
30. 0.000677 a-thread-about-crafting-or-trying-to-craft-nrcs.6361
31. 0.000770 reasonable-changes-to-buccaneers.28347
32. 0.000910 make-horntail-1-run-per-day.38022
33. 0.001224 buff-mage-cast-speed-on-specific-non-ultimate-skills.34366
34. 0.001306 petition-for-buccaneer-super-transformation-to-have-90-stance.28896
35. 0.001465 returning-scammed-items.6659
36. 0.001521 make-enrage-to-be-worth-skill.35910
37. 0.001532 making-hp-washing-truly-optional-a-solution.39773
38. 0.002035 the-current-state-of-affairs.8554
39. 0.002203 golden-gach-tickets-a-new-way-to-obtain-ap-resets.13491
40. 0.002226 making-towns-henesys-more-alive.39232
41. 0.002366 lets-solve-hp-washing.21020
42. 0.002394 hp-washing-solution.24832
43. 0.002703 bowmaster-vs-marksman-under-the-new-changes.30717
44. 0.002738 make-hp-washing-optional.17813
45. 0.002929 nerf-veetron-drop-rate.39468
46. 0.003055 deep-ludibrium-teleport-portals.35807
47. 0.003539 buff-marksmen.27868
48. 0.003960 make-axe-viable-again.27213
49. 0.004280 revamping-horntail-drop-table.39667
50. 0.004327 your-feedback-and-experience-of-maplelegends.35702

Most likely to be rejected

1. 0.478435 cash-shop-clown-set.6588
2. 0.476573 storage-for-untradeable-items.29669
3. 0.475797 new-cash-item-suggestion.33809
4. 0.475207 duey-prime.35858
5. 0.474403 bloody-jeanne-unisex.26454
6. 0.473965 20-pqs-for-glasses.30873
7. 0.472585 add-skill-packet-skill-reset.24098
8. 0.472091 colored-item-list.35661
9. 0.471412 tricolor-corgi-pup.24358
10. 0.471024 add-cassandra-and-gaga-and-future-event-npcs-to-temple-of-time.30881
11. 0.469901 find-partner-command.23868
12. 0.469375 party-chat-guild-chat.29815
13. 0.469278 make-the-diamond-arrows-a-full-stack-instead-of-2k.36293
14. 0.469000 stackable-gachapon-tickets.30911
15. 0.468927 notification-if-someone-remove-you-in-buddy.29859
16. 0.468579 guild-leadership-transfer-with-no-cost.30458
17. 0.468055 have-a-server-on-west-coast-for-better-ping.28471
18. 0.467805 energy-charge.24952
19. 0.467738 make-gpq-earrings-an-nx-equip.36514
20. 0.467081 can-you-make-items-auto-stackable-in-storage.26824
21. 0.466841 a-mute-player-function.30961
22. 0.466781 thank-you.7092
23. 0.466204 quest-helper-fix.31012
24. 0.466133 can-we-get-stirgeman-gear.17817
25. 0.465736 spring-festival-coin.30770
26. 0.465721 new-key-to-teleport-pet-to-your-location.35264
27. 0.465710 suggestion.8485
28. 0.465629 qol-character-item-list.27423
29. 0.465222 mpq-quality-of-life-improvements.35013
30. 0.465142 opq-goddess-wristband.26415
31. 0.464784 stirgeman-items-not-working.19058
32. 0.464659 can-we-buff-dunas-exp-again.28874
33. 0.464545 marriage.15318
34. 0.464365 bladed-falcons-katana-cash-item.24269
35. 0.463633 rooneys-location.19196
36. 0.463284 quest-shammos-awakening.10018
37. 0.463223 add-those-nostalgic-quests-from-the-past.36916
38. 0.463176 make-scarlion-and-zakum-require-the-same-amount-of-people-to-enter-the-boss.26382
39. 0.462933 stop-booster-from-overwriting-si.26547
40. 0.462922 7590
41. 0.462815 light-of-circle-like-common-gacha.30854
42. 0.462631 ems-pirate-ring.22204
43. 0.462554 craftable-stormcaster-gloves-facestompers-cilbis-etc.10994
44. 0.462489 allow-beginner-in-pink-bean-event-pq.31033
45. 0.462455 islander-monster-card-ring.34787
46. 0.462043 request-for-more-alternative-links-to-download-game-patchers.30794
47. 0.461727 opq-ppq.12401
48. 0.461597 assassin-critical-hit-is-useless.31734
49. 0.461260 glittering-altaire-earrings.26767
50. 0.460966 7631

In this model, it looks like the suggestions that are likely to be rejected are the ones with the least amount text, while the ones that are most likely to be accepted are longer. To validate this claim, I made a plot of the number of words vs the probability of rejection.



Not quite what I expected. It looks like the count of words gives the model more confidence on whether it should be predicted or rejected, but the decision likely depends on the words themselves.