OwlRepo - a repository of transcribed owl searches

The one possibility that I'm aware of is using HDClient=1 on macOS. I actually didn't realize this was possible until a few weeks ago and haven't touched the site since. The other possibility is that you're taking a screenshot using Window's snipping feature. If this is the case, you should make sure that the screenshot is coming from the screenshot folder in the MapleLegends install directory.

If you go into the developer console (ctrl-shift-j on Chrome, ctrl-shift-k on Firefox) and then upload a few screenshots, it should give you more hints into what the processing error is. If you let me know what your system/resolution and the error is, I can make a few updates if need be.

 

I've put some time into site while the server was down today to push out small things I've been working on. First is a toy idea to show the store names of recent uploads.



Here are the most popular channel/fm using the same data across all time. I still get confused between fm-channel or channel-fm, so take it with a grain of salt.



I'm also looking to build a clean dataset to improve the quality of transcribing screenshots (https://owlrepo.com/curate). I put together a small tool for editing and submitting changes. Try it if you want to help out, but be mindful that I'm going to have to review each one manually.

Spoiler: a preview of editing/curation ui

Every task you contribute will get recorded locally and in a database: https://owlrepo.com/personal

 

Thanks for trying it out and for the great ideas! The goal of curating the images is to build a ground truth data set that can be used to fine-tune a pre-trained OCR model. Without a ground truth, it's hard to quantify confidence on the quality of the results that are coming out. I spent a lot of time playing with knobs and parameters to get the half-decent results on the site now, but there's room for improvement.

That being said, I *think* it's important that all of the text is correct when it's fed back into the model. Feeding in half-baked data might lead to wonkier results. I have a pretty simple heuristic for gathering up candidate images, but it might be too lax if the store names are the only things that have typos.

Spoiler: curation/candidate/view.sql

Code:

WITH
  bad_rows AS (
  SELECT
    *
  FROM
    owlrepo.upload_flat
  WHERE
    price = 0
    OR quantity = 0
    OR bundle = 0
    -- sometimes a 1 is mistaken for a 4
    OR bundle = 4 )
SELECT
  MIN(task_id) AS task_id,
  screenshot_sha1,
  screenshot_name
FROM
  bad_rows
GROUP BY
  screenshot_sha1,
  screenshot_name
ORDER BY
  screenshot_sha1

I haven't considered adding in confidence to the entries. It looks like tesseract exposes internal statistics about box boundaries and confidence, but I'd estimate a few weekends of dedicated work to get to something like "table entry confidence" in the owlrepo api. It's appealing, but it's a bit too much to implement for this UI atm. Having it show up on the site would require reprocessing all of the images, which is something that I'm saving for a v2 (with the fine-tuned model).

 

I added a checkbox you can mark if a row is missing. It's not too uncommon (e.g % uploaded is 0.98), but they haven't been explicitly added to candidate tasks. The SQL is a bit trickier, but the pages with missing rows are probably important.

 

I've added a new page at owlrepo.com/merchants that lists the items that have been seen across uploaded owl searches by a seller. Not very useful information, but it's neat to be able to look yourself up.

 

I've updated the original post since some of the screenshots are wildly out of date. The new interface looks like this:



I am now cleaning the data before creating this summary table (which means anything outside of 3 MAD are thrown out). The box plot contains all of the important information including the min, p25, p50, p75, max, average, and standard deviation. I also now include the sum of bundle sizes to get a total number. The min and bundle sizes are the most unreliable out of all of these measures because there are cases when the price text overflows into the bundle column (look for this when people price things over 10B). If things look wonky, it's probably because they are.

The scrolls on the front page now rotate through the 6 scrolls that I've been uploading every 4-5 days instead of being stuck with just Claw for ATT 30. Thanks again Nise for the scroll suggestions.

I also swapped over to Plot.ly for all charts. If you are looking at any of the time series plots, you can click on the legend at the bottom to remove lines from the plot. For example, here is the median price for CSS 20%.



Finally I've added some interesting cryptography to prove that an upload was made by you. I'm using JSON Web Keys (JWK) and JSON Web Tokens (JWT) to help secure the upload page. In the personal page, there's a new section:



This is unique for every device and is automatically generated for you. The thumbprint is included on every upload, alongside a local timestamp so the owl screenshots align to server time instead of local time. Treat this like you would a bitcoin wallet (with about the same level of anonymity) and don't lose your keys. I am planning on adding a way to claim a nickname, as well as an option to create new keys. I'll put together a small thank you/ranking page now that there's an account-like system in place.

Usage has been steadily growing which I finding exciting. The last day of November (2020-11-30) had 346 unique visitors to the site which is something like 5% of the daily active player base. I am looking for feedback on the front page because I know that it is incredibly information dense and probably intimidating to folks. Is it too much? Is it confusing? Are there things you want to know more about? Would you rather see prices in a different way?

Let me know your thoughts -- feel free to reach out to me here or on discord.

 

owlrepo.com/guide is my attempt at a noob-friendly cheatsheet with aesthetic, inspired by Mii's scroll guide and Fruit's scroll guide.

 

The OwlRepo dataset is public with the upload history and SQL views that the website uses. You can query the data for free with a BigQuery sandbox account. You can run SQL queries like this:

Code:

SELECT * from owlrepo.owlrepo.uploads_v1

Here's a starter notebook. If you're familiar with the tools, it only takes a few lines to make this:

 

Yeah, I can add some stats to that page for progress. It's an ever growing list though

 

I made a few minor updates, and realized that I haven't updated this thread in some time. First off, I fixed some recent usability issues:

• The search was not returning correct results until you typed something and cleared it. Search is tolerable again (please let me know if it still sucks)
• 100% scrolls were not showing up in the scroll guide. I was wondering why there were two listings for earring int 10%. It turns out that "10" is also a pattern in "100"...

Anyhow, since the time of last writing (2021-01-14), I've made some more significant changes to the site. A notable one is that the site has some new styling that's easier on the eyes during the hours I look at it. I expanded the scroll guide to include mastery books and some random etc. (let me know if you have suggestions). Finally, I refactored the site to use sapper for routing, which lays the groundwork for optimizing page loads.

It's been great seeing people other than me uploading to the site. To date, there have been ~48 unique contributors.



Thanks all for helping out

 

I can't reproduce this in Firefox 87, Chrome 88, and whatever version of Safari comes built into iOS 13. I get helmet dex 60, followed by helmet int 60. What browser are you using? Do you know if this is an issue on OwlRepo 2.2.1 (aa4cd7b0) specifically (bottom of the page)?

 

Yet another small update: the guide will let you set a lower bound on the scroll prices. The prices might be more out of date the less valuable the scroll is. I use the recommendation page to remind me what to upload, and it prioritizes more expensive scrolls over cheaper scrolls. If something is out-of-date and you happen to owl it, consider uploading and updating the guide.