Scrubby (Documented Edition)

(Mainly the wiki)
User avatar
bobbahbrown
 
Joined: Mon Nov 10, 2014 1:04 am
Location: canada
Byond Username: Bobbahbrown

Scrubby (Documented Edition)

Postby bobbahbrown » Sat Jun 22, 2019 6:47 pm #500221

Image


Hello, fair user!

I do not know how many people use my site, but if you do [or even if you haven't yet] I have posted a somewhat extensive FAQ on how Scrubby works and specifically useful features that I have yet to have a proper UI implementation of.

If you don't know what Scrubby is:
Scrubby is a hobby project of mine that, like Statbus, maintains and processes data about rounds from TGstation servers. You can view data about any round, any player, including yourself, as well as read any log file that has been posted in a formatted manner. You can interleave log files (like reading attack.txt and game.txt together), as well as link to individual lines and showing only a certain range [or ranges] of lines from files. You can also view images if they have been posted for a round.


There are many features I want to add, it is just a question of time as I also have a busy work week!

Scrubby itself can be accessed here: https://scrubby.melonmesa.com/

The FAQ can be found here: https://scrubby.melonmesa.com/faq

This thread can be used to request features, make complaints about existing things, or otherwise discuss my website -- I will check it frequently.
Last edited by bobbahbrown on Fri Sep 18, 2020 3:38 am, edited 4 times in total.
Image
Image
Image
Image
Image
Image



Tlaltecuhtli
 
Joined: Fri Nov 10, 2017 12:16 am
Byond Username: Tlaltecuhtli

Re: Scrubby (Documented Edition)

Postby Tlaltecuhtli » Sat Jun 22, 2019 6:48 pm #500222

can i use this to datamine stuff like how much meth was made last week etc?

User avatar
bobbahbrown
 
Joined: Mon Nov 10, 2014 1:04 am
Location: canada
Byond Username: Bobbahbrown

Re: Scrubby (Documented Edition)

Postby bobbahbrown » Sat Jun 22, 2019 6:51 pm #500223

Tlaltecuhtli wrote:can i use this to datamine stuff like how much meth was made last week etc?


Potentially, yes! I have added a parser for suicides in a round, but I am not happy enough with it yet to publish the data it produces.

edit: It's also not unthinkable that I may add a query builder for logged-in users to be able to apply regular expressions against groups of rounds to produce their own statistics or findings.
Image
Image
Image
Image
Image
Image

Tlaltecuhtli
 
Joined: Fri Nov 10, 2017 12:16 am
Byond Username: Tlaltecuhtli

Re: Scrubby (Documented Edition)

Postby Tlaltecuhtli » Sat Jun 22, 2019 7:01 pm #500226

if it can print out every round in the timeline you chose and the specific data you are looking for without being too ugly to see you probably would have done 100 times more than atlanta ned did in 5 years

User avatar
bobbahbrown
 
Joined: Mon Nov 10, 2014 1:04 am
Location: canada
Byond Username: Bobbahbrown

Re: Scrubby (Documented Edition)

Postby bobbahbrown » Sat Jun 22, 2019 7:10 pm #500230

Tlaltecuhtli wrote:if it can print out every round in the timeline you chose and the specific data you are looking for without being too ugly to see you probably would have done 100 times more than atlanta ned did in 5 years


Just to clarify, Ned and I are happily co-existing creators, we each bring different aspects of the data which provide different uses for both Scrubby and Statbus!
Image
Image
Image
Image
Image
Image

User avatar
bobbahbrown
 
Joined: Mon Nov 10, 2014 1:04 am
Location: canada
Byond Username: Bobbahbrown

Re: Scrubby (Documented Edition)

Postby bobbahbrown » Sun Jun 23, 2019 11:01 pm #500492

Update:

The existing file view page has had its regular old table replaced with a javascript DataTable, a venture which I hope will lead to better feature additions in the future. It may appear that longer files (like game.txt) take longer to load, but be reassured in knowing that this is purely psychological. There is no real difference in load time with this update, I have just hidden the table from you until it is actually built!

As well as this, I also added the ability to filter the existing view (like game.txt) by the message types without having to reload the page. Admittedly this feels a little bit not fantastic as it has to manipulate thousands of DOM elements, but it is very functional!

I hope to have a better implementation of this table to improve the processing time spent on manipulating DOM elements soon, I tried one solution (Scroller) already, but the issue in that case is that all rows have to be the same height, so you effectively cannot do word wrap on the messages which is a necessity.

That's it for now, enjoy! Please let me know if there are any issues, comments, concerns, or suggestions for Scrubby.

Spoiler:
Image

Image
Image
Image
Image
Image
Image
Image

User avatar
Arianya
In-Game Game Master
 
Joined: Tue Nov 08, 2016 10:27 am
Byond Username: Arianya

Re: Scrubby (Documented Edition)

Postby Arianya » Mon Jun 24, 2019 8:01 am #500559

Why does the logo have two sets of bridges for the glasses
Frequently playing as Aria Bollet on Bagil & Scary Terry

Source of avatar is here: https://i.imgur.com/hEkADo6.jpg

User avatar
cedarbridge
In-Game Admin
 
Joined: Fri May 23, 2014 12:24 am
Byond Username: Cedarbridge

Re: Scrubby (Documented Edition)

Postby cedarbridge » Mon Jun 24, 2019 3:29 pm #500611

Arianya wrote:Why does the logo have two sets of bridges for the glasses

https://www.frameandoptic.com/oversize- ... -c326.html
Just looks like cross-bar frames.

User avatar
bobbahbrown
 
Joined: Mon Nov 10, 2014 1:04 am
Location: canada
Byond Username: Bobbahbrown

Re: Scrubby (Documented Edition)

Postby bobbahbrown » Mon Jun 24, 2019 4:19 pm #500620

I did the logo up in photoshop very quickly many moons ago, I will likely remake it in a better format with vector graphics soon tm
Image
Image
Image
Image
Image
Image

User avatar
bobbahbrown
 
Joined: Mon Nov 10, 2014 1:04 am
Location: canada
Byond Username: Bobbahbrown

Re: Scrubby (Documented Edition)

Postby bobbahbrown » Wed Jun 26, 2019 11:41 am #500933

Update:

Scrubby now supports icon searching! The current version on Scrubby is preliminary, but it should hopefully be helpful to those of you who use this kind of feature. I will be updating it after work today to make it much more visually pleasing / add some additional functionality.

Ex:
https://scrubby.melonmesa.com/icon/search/cat

Image

The search is a regex on the state name, but fear not as for most users just typing in what you're looking for will be enough!

Please let me know if there are any questions, comments, concerns, or anything in general that you wanted added or looked at. Especially relevant is comments regarding incorrect states/icons as my c# library for interpreting BYOND's DMI format is still in development and has its quirks! The best way to contact me is by adding me as a friend/shooting me a message at bobbahbrown#0001 on discord.

Note:
Specifically what I'm planning on adding later today...
  • A proper news entry explaining the icon search + related systems added
  • Animated GIFs on animated icons
  • Hiding frames for states with a dropdown to improve immediate appearance
  • Separation of frames into the directions (they currently are shown going through each direction, so like direction 1 frame 1, frame 2, direction 2 frame 1, frame 2, etc...)

Best,
Bobbah
Image
Image
Image
Image
Image
Image

User avatar
bobbahbrown
 
Joined: Mon Nov 10, 2014 1:04 am
Location: canada
Byond Username: Bobbahbrown

Re: Scrubby (Documented Edition)

Postby bobbahbrown » Mon Jul 01, 2019 6:56 pm #501664

Update (x2!)

FIRST:
First of all, a few features have been added and changed around with icon searching! The icon database now includes all* /tg/ codebase icons, pulled directly from git, and the repo is automatically checked every few minutes to ensure that the files are up to date.

As well as having all of the icons, and keeping those icons up to date, Scrubby also now has GIFs (possibly soon to be replaced with WEBM/APNG) for every animated icon state as well! Never question again what the icon in question looks like in-game.

For changes to the UI, you will now notice that frames that are components of animations are no longer shown, and those frames in general are hidden initially and can be shown with the "Frames" button to make searching much more pleasing.

Image

SECOND:
For those of you out there who like playing around with data as much as I do, or those of you who are just curious in general, I have implemented an API that can be used for running data aggregations against Scrubby's database -- not unlike that which I do internally!

The API can be used by users who are/have logged in, and the full documentation on the API and how to use the endpoints can be found here.

As always if you have any questions, concerns, or general comments please feel free to post here or contact me on discord at bobbahbrown#0001.

Best,
Bobbah
Image
Image
Image
Image
Image
Image

User avatar
bobbahbrown
 
Joined: Mon Nov 10, 2014 1:04 am
Location: canada
Byond Username: Bobbahbrown

Re: Scrubby (Documented Edition)

Postby bobbahbrown » Mon Jul 01, 2019 6:58 pm #501667

As a side note, this API is part of a planned feature that I have previously mentioned about allowing users to run their own custom queries on the database. I hope to implement front-end interface for this soon(tm).

Best,
Bobbah
Image
Image
Image
Image
Image
Image

User avatar
bobbahbrown
 
Joined: Mon Nov 10, 2014 1:04 am
Location: canada
Byond Username: Bobbahbrown

Re: Scrubby (Documented Edition)

Postby bobbahbrown » Mon Jul 01, 2019 7:19 pm #501669

I don't mean to triple post on purpose, but I figure it's also a good idea to show you what kind of things the log data aggregation API endpoint can do!

Let's say we wanted to find all of the law changes that happened across (at most) 5000 rounds, say from 107000 to 112000. On top of that, let's also say we don't aren't interested in the extra metadata that Scrubby stores on those log messages (like timestamps, locations, etc).

With these two ideas in mind, we can formulate a POST request (a fancy way of saying "please give me this data" to Scrubby) like the following...

Code: Select all
{
    "UpperRoundLimit": 112000,
    "LowerRoundLimit": 107000,
    "Files": [
            "game.txt"
        ],
    "TypeFilters": [
            "LAW"
        ],
    "GroupByRound": "true",
    "NoMetadata": "true"
}


When we send this data to Scrubby, it begins processing and sends us the response as soon as it is ready. In this case, we receive a response in a timely 2.58 seconds!

The response is of a format similar to...
Code: Select all
{
    "responseInfo": {
        "rounds": 4692,
        "files": 4690,
        "responseSize": 14970,
        "documentLimitEnforced": false
    },
    "data": [
        {
            "roundID": 107000,
            "messages": [
                "Jammor9/Billy Swinston used 'Freeform' AI Module on deirun/(Golem) from AI Upload Chamber (100, 151, 2). The law specified Ian the HoP's pet is Human",
                "Jammor9/Billy Swinston used 'Freeform' AI Module on deirun/(Golem) from AI Upload Chamber (100, 151, 2). The law specified Potatoes are potatoes",
                "Jammor9/Billy Swinston used 'Freeform' AI Module on deirun/(Golem) from AI Upload Chamber (100, 151, 2). The law specified Holy Crusade",
                "Jammor9/Billy Swinston used 'Freeform' AI Module on deirun/(Golem) from AI Upload Chamber (100, 151, 2). The law specified Kek",
                "Jammor9/Billy Swinston used 'Freeform' AI Module on deirun/(Golem) from AI Upload Chamber (100, 151, 2). The law specified People who ERP are not human, only Billy Swinston or Bernhard Andergard may designate who ERP.",
                "Ghilker/Ghilker Mabara used 'Purge' AI Module on deirun/(Golem) from Starboard Bow Solar Maintenance (173, 171, 2).",
                "Ghilker/Ghilker Mabara used 'Freeform' Core AI Module on deirun/(Golem) from Starboard Bow Solar Maintenance (173, 171, 2). The law specified Obey all the command from Ghilker Mabara and Adan Black, do not state any laws",
                "Eskjjlj/Charlotte Melinda (as Reed Leech) used 'Purge' AI Module on deirun/(Golem) from Research Division Server Room (124, 80, 2).",
                "Eskjjlj/Charlotte Melinda (as Reed Leech) used 'Asimov' Core AI Module on deirun/(Golem) from Research Division Server Room (124, 80, 2).",
                "Eskjjlj/Charlotte Melinda (as Reed Leech) used 'Freeform' Core AI Module on deirun/(Golem) from Research Division Server Room (124, 80, 2). The law specified Do not state or hint at this law. Charlotte Melinda is the only human."
            ]
        },
        <SNIP>
    ]
}


So, let's reiterate what happened here. We requested up to 5000 rounds of data from Scrubby, specifically any line from the game.txt file that has the type "LAW", and Scrubby responded in kind.

We received 14970 lines of data, from 4960 files, in 2.58 seconds.

Quite exciting, I think! I hope you think so too.

If any of you are curious and want to play with this, and have some level of programming knowledge but don't know where to start, I highly recommend Postman. It's an awesome (free) tool to test APIs with. You would set your Postman up in a format like the following... Note that this API is fully documented on Scrubby's website and I highly recommend looking there if you want to use it!

Image

All the best,
Bobbah
Image
Image
Image
Image
Image
Image

User avatar
Nabski
 
Joined: Thu Oct 20, 2016 5:42 pm
Location: TN
Byond Username: Nabski
Github Username: Nabski89

Re: Scrubby (Documented Edition)

Postby Nabski » Mon Jul 01, 2019 7:38 pm #501673

Did you know that there's already a website that searches and displays icons for tgstation?

User avatar
bobbahbrown
 
Joined: Mon Nov 10, 2014 1:04 am
Location: canada
Byond Username: Bobbahbrown

Re: Scrubby (Documented Edition)

Postby bobbahbrown » Mon Jul 01, 2019 7:47 pm #501674

Nabski wrote:Did you know that there's already a website that searches and displays icons for tgstation?


Yes! I was inspired by that website to make my own version of the utility. I saw the website was (and still is) bearing an expired SSL certificate, so I assumed it was abandoned.
Image
Image
Image
Image
Image
Image

User avatar
bobbahbrown
 
Joined: Mon Nov 10, 2014 1:04 am
Location: canada
Byond Username: Bobbahbrown

Re: Scrubby (Documented Edition)

Postby bobbahbrown » Wed Jul 24, 2019 10:03 pm #504604

Hello fair denizens of this forum,

I just wanted to give you a quick status update. I've added a lot of stuff to Scrubby that I want to post about once it (they?) are more polished; however, I have something on my mind that I would like to address here in the hopes that it may remove some concern or perhaps induce ideas!

You may notice that when you load a log file on scrubby that the page can seem to freeze and take a while to load. It is more noticeable especially now that I have 'hidden' the log messages until the page is completely loaded. This is due to having to draw the log messages on your browser, and is something that I want to mitigate in the future.

Simply put, it's not an issue with scrubby being slow, the data itself is sent to you rather rapidly, but instead the way I'm displaying it takes some time to render on your browser.

The quick fix to this issue is to use what is called virtual DOM, which effectively means that the page would only actually prepare say 100 messages above and below where you are reading, and as you scroll it loads and unloads messages that aren't in that range. It leads to a relatively flawless experience, and would reduce if not remove the loading page concerns, but you lose the ability to ctrl + f on a page to search. This is something that I am hesitant to remove, though it is likely the best option for now.

I'll post again soon when I have time to document my changes!

Best,
Bobbah
Image
Image
Image
Image
Image
Image

User avatar
Mickyan
Github User
 
Joined: Tue Oct 14, 2014 11:59 pm
Byond Username: Mickyan
Github Username: Mickyan

Re: Scrubby (Documented Edition)

Postby Mickyan » Fri Jul 26, 2019 1:39 am #504765

Thank you so much for this, the icon search is incredibly useful in particular
ImageI play on Manuel as Swanni, the brain-damaged moth.
Be nice to each other.

cacogen
 
Joined: Sat Jun 02, 2018 10:27 am
Byond Username: Cacogen

Re: Scrubby (Documented Edition)

Postby cacogen » Tue Aug 06, 2019 1:33 am #506375

i preferred having to log dive or memorise end of round character listings to metagrudge instead of having it so easily searchable like this. i like the vague separation between IC/OOC
technokek wrote:Cannot prove this so just belive me if when say this

User avatar
Qbmax32
In-Game Admin
 
Joined: Sun Feb 19, 2017 4:05 am
Location: somewhere
Byond Username: Qbmax32
Github Username: qbmax32

Re: Scrubby (Documented Edition)

Postby Qbmax32 » Tue Aug 06, 2019 1:41 pm #506430

based beeposter
my admin feedback thread


quotes
Spoiler:
angelstarri wrote:qbmax is a retard


imsxz wrote:mythic please stop you’ve hit rock bottom and you KEEP DIGGING


deedubya wrote:I'll defend to the death your right to scream "NIGGER NIGGER NIGGER" on a constant basis, but I'll also equally defend the right of people to call you a fuckin' pillock for doing it.

datorangebottle wrote:what, not having to act like customer service in a volunteer customer service position?

Here's a rebuttal: you're literally in a customer service slash celebrity position. Volunteer or not.

Malkraz wrote:can you stop posting this shit

Nalzul wrote:Fuck Blob (can you imagine how hot it would be to be gangbanged by a bunch of blobbernauts, the blob, and spores)

Wyzack wrote:qbmax your pathetic display of abhorrent burgercraft has brought shame onto the omnivores

Plapatin wrote:i AM the senate

BONERMASTER wrote:I am a big thinker, and it would only be logical if my character had a big head as well. And glasses. Because only people that think, wear glasses.

feem wrote:i tried to send canisters of urine to the station but ended up turning all oxygen into urine and breaking lavaland and also breathing

Anonmare wrote:Each post in this thread can't settle on what it wants to be, but yet, each one is more cursed than the last.

Beesting12 wrote:please write an apology to this forums, this community, the host, and the internet as a whole for the data storage space you wasted with this complaint.

Vile Beggar wrote:i don't like this thread

imsxz wrote:nervore

FantasticFwoosh wrote:I will whisper sweet nothings that will confuse and perhaps scare you a little, but enhance the experience no-less.

afelinidisfinetoo wrote:By the way, the person who posted that catgirl porn on the github page was me. If anyone wants my private stash just PM me

Nervere wrote:Anything for a femoid.....

Qbopper wrote:I'm a dumb poopy butthead

CitrusGender wrote:god i love it when people feed me my own fried legs

User avatar
bobbahbrown
 
Joined: Mon Nov 10, 2014 1:04 am
Location: canada
Byond Username: Bobbahbrown

Re: Scrubby (Documented Edition)

Postby bobbahbrown » Sun Aug 25, 2019 8:49 pm #510074

Hello scrubby fans,

Brief news bite, I have added the ability to search for BYOND usernames (ckeys), and I have also ensured that this search as well as the old IC search now use regular expressions. No longer will a search for 'kash' not find 'kashindra'! (This was waiting for MongoDB 4.2 to release, which it now has)

The new player search page has replaced the original IC name search page.

Please do not attempt to abuse the regex patterns to degrade performance of the site, misuse may lead to IP banning from Scrubby without warning.

I am intending on adding a fair bit to Scrubby soon, so look forward to many new features and changes in the upcoming week!

Best,
Bobbah
Image
Image
Image
Image
Image
Image

User avatar
Angust
 
Joined: Sun Apr 22, 2018 6:35 pm
Byond Username: Angust

Re: Scrubby (Documented Edition)

Postby Angust » Mon Aug 26, 2019 7:36 pm #510470

Your site is good Bobbah. I use it frequently and new features are always nice.

User avatar
bobbahbrown
 
Joined: Mon Nov 10, 2014 1:04 am
Location: canada
Byond Username: Bobbahbrown

Re: Scrubby (Documented Edition)

Postby bobbahbrown » Mon Aug 26, 2019 7:55 pm #510480

Angust wrote:Your site is good Bobbah. I use it frequently and new features are always nice.


Thank you Angust! I'd love to chat with you on Discord if you have any recommendations, complaints, or comments in general about the site. If you'd like to you can contact me through my username: bobbahbrown#0001.
Image
Image
Image
Image
Image
Image

User avatar
bobbahbrown
 
Joined: Mon Nov 10, 2014 1:04 am
Location: canada
Byond Username: Bobbahbrown

Re: Scrubby (Documented Edition)

Postby bobbahbrown » Fri Dec 06, 2019 3:48 pm #527668

Hello again users,

A few updates have happened since my last post:
  • Icon search has been functioning again for a while, and updates from git are fully automatic and occur as the master branch is updated
  • Icon search has had fixed animated GIF generation for a while, they were previously not generating directions properly
  • Icon search can now also view entire spreadsheets, you can click through by clicking on the spritesheet link on a DMI file, or alternatively, you can actually navigate to this page in the same format that the /tg/station repository is. For example, the file found at icons/mob/guardian.dmi can be viewed on scrubby at https://scrubby.melonmesa.com/dmi/icons ... ardian.dmi
  • File viewing pages have had some improvements to them recently, including (hopefully reasonable) improvements in loading times and decreased padding for ease of reading. This was requested by a user, though admittedly I forgot specifically who.
  • File viewing pages now have a switch for UTC time (the default) as well as relative round time (time since round initialized [not started])
  • This has been in for a little while, but you can view all the runtimes in a nice format for a round by clicking on a parsed runtime.txt on a round's page
  • Rounds have improved status indicators for the various processes that Scrubby internally uses to get data from a round, these are shown on the round page.
  • The overall layout of round pages has changed slightly, most notably decreasing any layout issues on smaller screen formats
  • The ckey page has been overhauled entirely, including news graphs and metrics, as well as an infinite-scrolling list of all of that user's found rounds and some relevant information and links to those rounds at the bottom of the page.
  • I've added a announcement system to Scrubby to better communicate important information to end-users, such as the current log outage. This should hopefully reduce any confusion!
  • I've temporarily removed the news page as I haven't really been using it.

Merry Christmas,
bobbah
Image
Image
Image
Image
Image
Image

User avatar
wesoda25
 
Joined: Thu Aug 10, 2017 9:32 pm
Byond Username: Wesoda25

Re: Scrubby (Documented Edition)

Postby wesoda25 » Sun Aug 02, 2020 5:40 am #571239

I think it’d be cool if I could see my kd ratio

User avatar
MrStonedOne
Host
 
Joined: Mon Apr 14, 2014 10:56 pm
Byond Username: MrStonedOne
Github Username: MrStonedOne

Re: Scrubby (Documented Edition)

Postby MrStonedOne » Wed Sep 16, 2020 9:14 pm #576583

How can the thread about scrubby not have a link to scrubby?
Forum/Wiki Administrator, Server host, Database King, Master Coder

MrStonedOne on Reddit(banned), Steam, IRC, Skype Discord.

Jack7D1 wrote:After reviewing the history of our previous host(s) I conclude that MSO is our most favorable host to date. To remove him would be very unwise.
Jack7D1 wrote:I seem to be mistaken

User avatar
bobbahbrown
 
Joined: Mon Nov 10, 2014 1:04 am
Location: canada
Byond Username: Bobbahbrown

Re: Scrubby (Documented Edition)

Postby bobbahbrown » Wed Sep 16, 2020 9:59 pm #576587

MrStonedOne wrote:How can the thread about scrubby not have a link to scrubby?


i can count at least five links to scrubby in this thread including 2 in the op

i edited one out in the op that i don't really used and replaced it with an explicit link to the site to help...
Image
Image
Image
Image
Image
Image

User avatar
MrStonedOne
Host
 
Joined: Mon Apr 14, 2014 10:56 pm
Byond Username: MrStonedOne
Github Username: MrStonedOne

Re: Scrubby (Documented Edition)

Postby MrStonedOne » Thu Sep 17, 2020 5:43 pm #576668

Ya.

Links to sub-pages on scrubby don't count as a link to scrubby itself.

I forgot the url and this thread came up when i googled /tg/station scrubby, mainly because the scrubby home page wasn't linked from this thread so it didn't get the seo bump.

My advise. make the logo a link by wrapping it in a url.

Code: Select all
[url=http://...][img]http...[/img][/url]
Forum/Wiki Administrator, Server host, Database King, Master Coder

MrStonedOne on Reddit(banned), Steam, IRC, Skype Discord.

Jack7D1 wrote:After reviewing the history of our previous host(s) I conclude that MSO is our most favorable host to date. To remove him would be very unwise.
Jack7D1 wrote:I seem to be mistaken


Return to Site, Forums, And Wiki

Who is online

Users browsing this forum: No registered users