1. Hello!

    First of all, welcome to MapleLegends! You are currently viewing the forums as a guest, so you can only view the first post of every topic. We highly recommend registering so you can be part of our community.

    By registering to our forums you can introduce yourself and make your first friends, talk in the shoutbox, contribute, and much more!

    This process only takes a few minutes and you can always decide to lurk even after!

    - MapleLegends Administration-
  2. Experiencing crashes with 'Unknown Error'? Make sure you are on the latest MapleLegends version. The current latest version is found by clicking here.
    Dismiss Notice

Monster Book Transcription

Discussion in 'Programming' started by geospiza, Sep 7, 2020.

  1. geospiza
    Offline

    geospiza Web Developer Staff Member Web Developer

    188
    395
    196
    Apr 16, 2020
    9:21 AM
    geospiza
    Dark Knight
    146
    Funk
    Ever feel frustrated figuring out what cards to hunt next because the poor Monster Book layout? The Monster Book Transcription app will machine-read screenshots of your monster book so you can sort it against the information provided by the Monster Book Efficient Farming Guide by Precel and Bambo.

    I really appreciate the level of detail that was put into the card farming guide, thank you.

    Warning that this is a work in progress (WIP). I currently don't have my desktop on me, so this will only work if for screenshots taken on macOS with default client settings and a retina display (cmd-shift-4 + space + click resulting in a 1824x1468 image). I'd like to eventually support all resolutions in the Windows client.

    After capturing your Monster Book in screenshots, you can transcribe them in the app. Image processing is done locally in your browser. Afterwards, you can check the grid view to see if the results look okay:

    upload_2020-9-7_12-22-48.png
    By default, all of the sets that have been completed are hidden from view. If for some reason you'd like to view them, there are some options to play with.

    The table view is more informative. You can sort by the order in the book, by card route, or by the number of cards collected.

    By book order:
    upload_2020-9-7_12-24-50.png
    This is the default view, but it's only useful for checking that the transcription is done correctly.

    By card route (tour) order:
    upload_2020-9-7_12-25-54.png
    This is the order that's suggested by the efficient farming guide. This effectively orders monsters by their location in the world map. This is the view that I wanted for my own personal card hunting.

    By card collected order:
    upload_2020-9-7_12-26-35.png
    This order sorts by the number of cards collected, then by the tour order. If you only need a few more sets to get to the next tier, this will probably be useful.

    The source code can be found on Github at geospiza-fortis/monsterbook. I've attached a single image (Screen Shot 2020-09-05 at 5.51.46 PM.png) that can be transcribed for testing. You can also download a zip file of my monster book for a larger test.
     

    Attached Files:

    • Great Work Great Work x 11
    • Like Like x 1
  2. Tate
    Offline

    Tate Capt. Latanica

    309
    207
    273
    Apr 16, 2020
    New Zealand
    6:21 AM
    Potayto, Protecc, Attacc, Smegaphone, VoteReminder
    Shadower
    170
    Beaters
    will definitely try this out when im home later.
     
  3. OP
    OP
    geospiza
    Offline

    geospiza Web Developer Staff Member Web Developer

    188
    395
    196
    Apr 16, 2020
    9:21 AM
    geospiza
    Dark Knight
    146
    Funk
    I've collected 94 cards since I first wrote this, so it's been a useful to me. If you have a resolution that's not supported and want to help out, you can post or PM me a single screenshot with the monster book open.
     
  4. OP
    OP
    geospiza
    Offline

    geospiza Web Developer Staff Member Web Developer

    188
    395
    196
    Apr 16, 2020
    9:21 AM
    geospiza
    Dark Knight
    146
    Funk
    After over a year, I've made a few updates to the program with the goal of making this completely functional. I found out that the monster book position in the client is saved, which makes the fixed offset approach completely moot. I rewrote most of the image processing stuff in Rust and built a small application to stitch the monster book together.

    [​IMG]

    Download it here on Github: https://github.com/geospiza-fortis/monsterbook/releases/tag/v1.0.0

    To use it:
    1. Take consecutive screenshots of your monsterbook in order, using the screenshots functionality in the client.
    2. Copy the images from the screenshots folder into a separate folder
    3. Run monsterbook-gui.exe and select the screenshots folder that you created
    I'm planning on compiling this program into wasm so it can be run directly in a web-browser. In addition, there are quite a few things missing from this, like being able to sort your missing cards to figure out what to hunt next. The stitched images are pretty cool though.

    I had a lot of fun implementing the algorithm for trying to find the offset into the screenshot where all of the cards were. I initially used this library called imageproc, which includes a template_matching algorithm that performs poorly. I implemented the phase correlation algorithm, which basically finds the position in the frequency domain where the signal of the two images are most closely aligned. It's simple to implement if you have access to the 2d-fft and it's inverse:

    Code:
    psudocode for phase correlation that returns offset of template to screenshot
    
    template is a cropped image with the 5x5 grid of cards
    screenshot is the image
    
    let padded be the template padded to the size of the screenshot
    let X = fft2d(screenshot)*conjugate(fft2d(padded))
    let inverted = ifft(X/abs(X))
    let offset be the argmax of inverted (aka x, y position of the max value)
    return offset
    
    It works surprisingly well, although I ran into bugs when I was implementing because I had to reimplement the 2d fft using a 1d fft. This is done by taking the fft of all the rows, then all the columns. The issue with the library that I am using (rustfft) is that the data structure I was using (ndarray) only returns references to the underlying buffer if they are contiguous in memory. There was a struggle to figure out how to transpose the matrix, or at least generate a new buffer that could be used to transform the column.
     
    • Great Work Great Work x 8
    • Friendly Friendly x 1
  5. UnknownCode
    Offline

    UnknownCode Selkie Jr.

    204
    54
    205
    Jun 6, 2021
    Female
    9:21 AM
    Bowmaster, Thief
    PerionSpirit
    Hey, Geospiza will macOS have its own Monster Book Stitcher app in the future?
     
  6. OP
    OP
    geospiza
    Offline

    geospiza Web Developer Staff Member Web Developer

    188
    395
    196
    Apr 16, 2020
    9:21 AM
    geospiza
    Dark Knight
    146
    Funk
    It's probably doable since I plan on making the web app the primary method of use. That being said, my abundance of free time is less so nowadays and I don't have a timeline on when I'll finish this project.
     
    • Friendly Friendly x 1

Share This Page