Posts

• Apr 15, 2022

  The Dawn of Do What I Mean

  Boy, last week was busy for deep learning. Let’s start with the paper I worked on.

  SayCan is a robot learning system that we’ve been developing for about the past year. The paper is here, and it builds on a lot of past work we’ve done in conditional imitation learning and reinforcement learning.

  Suppose you have a robot that can do some small tasks we tell it to do with natural language, like “pick up the apple” or “go to the trash can”. If you have these low-level tasks, you can chain them together into more complex tasks. If I want the robot to throw away the apple, I could say, “pick up the apple, then go to the trash can, then place it in the trash can”, and assuming those three low-level tasks are learned well, the robot will complete the full task.

  Now, you wouldn’t want to actually say “pick up the apple, then go to the trash can, then place it in the trash can”. That’s a lengthy command to give. Instead, we’d like to just say “throw away the apple” and have the rest be done automatically. Well, in the past few years, large language models (LLMs) have shown they can do well at many problems, as long as you can describe the input and output with just language. And this problem of mapping “throw away the apple” to “pick / go to trash / place” fits that description exactly! With the right prompt, the language model can generate the sequence of low-level tasks to perform.

  

  This, by itself, is not enough. Since the LLM is not aware of the robot’s surroundings or capabilities, using it naively may generate sentences the robot isn’t capable of performing. This is handled with a two-pronged approach.

  1. The language generation is constrained to the skills the robot can (currently) perform.
  2. Each generated instruction is scored based on a learned value function, which maps the image + language to the estimated probability the robot can complete the task.

  You can view this as the LLM estimating the best-case probability an instruction helps the high-level goal, and the value function acting as a correction to that probability. They combine to pick a low-level task the robot can do that’s useful towards the high-level goal. We then repeat the process unless the task is solved.

  This glosses over a lot of work on how to learn the value function, how to learn the policy for the primitive tasks, prompt engineering for the large language model, and more. If you want more details, feel free to read the paper! My main takeaway is that LLMs are pretty good. The language generation is the easy part, while the value function + policy are the hard parts. Even assuming that LLMs don’t get better, there is a lot of slack left for robot capabilities to get better and move towards robots that do what you mean.

  * * *

  LLMs are not the bottleneck in SayCan, but they’re still improving (which should be a surprise to no one). As explained in the GPT-3 paper, scaling trendlines showed room for at least 1 order of magnitude, and recent work suggests there may be more.

  DeepMind put out a paper for their Chinchilla model. Through more careful investigation, they found that training corpus size had not increased in size relative to parameter count as much as it could have. By using about 4x more training data (300 billion tokens → 1.4 trillion tokens), they reduced model size by 4x (280B parameters → 70B parameters) while achieving better performance.

  

  Estimated compute-optimal scaling, using larger datasets and fewer parameters than previous scaling laws predicted.

  Meanwhile, Google Brain announced their PaLM language model, trained with 540B parameters on 780 billion tokens. That paper shows something similar to the GPT-2 → GPT-3 shift. Performance increases on many tasks that were already handled well, but on some tasks, there are discontinuous improvements, where the increase in scale leads to a larger increase in performance than predicted from small scale experiments.

  

  Above is Figure 5 of the PaLM paper. Each plot shows model performance on a set of tasks where PaLM’s performance vs model size is log-linear (left), “discontinuous” (middle), or relatively flat (right). I’m not even sure the flat examples are even that flat, they look slightly under log-linear at worst. Again, we can say that loss will go down as model size goes up, but the way that loss manifests in downstream tasks doesn’t necessarily follow the same relationship.

  The emoji to movie and joke explanation results are especially interesting to me. They feel qualitatively better in a way that’s hard to describe, combining concepts with a higher level of complexity than I expect.

  

  Neither of these works have taken the full 1 order of magnitude suggested by prior work, and neither indicates we’ve hit a ceiling on model scaling. As far as I know, no one is willing to predict whether or what qualitatively new capabilities we’ll see from the next large language model. This is worth emphasizing - people genuinely don’t know. Before seeing the results of the PaLM paper, I think you could argue that language models would have more trouble learning math-based tasks, and the results corroborate this (both navigate and mathematical_induction from the figure above are math-based). You could also have predicted that at least one benchmark would get qualitatively better. I don’t see how you could have predicted that english_proverbs and logical_sequence in particular would improve faster than their power low curve.

  The blog post for the Chinchilla model notes that given the PaLM compute budget, they expect you could match it with 140B params if you used a dataset of 3 trillion tokens of language. In other words, there’s room for improvement without changing the model architecture, as long as you crawl more training data. I don’t know how hard that is, but it has far less research uncertainty than anything from the ML side.

  Let’s just say it’s not a good look for anyone claiming deep learning models are plateauing.

  * * *

  That takes us to DALL·E 2.

  

  On one hand, image generation is something that naturally captures the imagination. You don’t have to explain why it’s cool, it’s just obviously cool. Similar to language generation, progress here might overstate the state of the field, because it’s improving things we naturally find interesting. And yet, I find it hard to say this doesn’t portend something.

  From a purely research standpoint, I was a bit out of the loop on what was state-of-the-art in image generation, and I didn’t realize diffusion based image synthesis was outperforming autoregressive image synthesis. Very crudely, the difference between the two is that diffusion gradually updates the entire image towards a desired target, while autoregressive generation draws each image patch in sequence. Empirically, diffusion has been working better, and some colleagues told me that it’s because diffusion better handles the high-dimensional space of image generation. That seems reasonable to me, but, look, we’re in the land of deep learning. Everything is high-dimensional. Are we going to claim that language is not a high-D problem? If diffusion models are inherently better in that regime, then diffusion models should be taking over more of the research landscape.

  Well, maybe they are. I’ve been messing around with Codex a bit, and would describe it as “occasionally amazing, often frustrating”. It’s great when it’s correct and annoying when it’s not. Almost-correct language is amusing. Almost-correct code is just wrong, and I found it annoying to continually delete bad completions when trying to coax the model to generate better ones. There was a recent announcement of improving Codex to edit and insert text, instead of just completing it. It’s better UX for sure, and in hindsight, it’s likely using the same core technology DALL-E uses for image editing.

  

  We’re taking an image and dropping a sofa in it, or we’re taking some text and changing the sentence structure. It’s the same high level problem, and maybe it’s doing diffusion-based generation under the hood.

  * * *

  Where does this leave us?

  In general, there is a lot of hype and excitement about models with a natural language API. There is a building consensus that text is a rich enough input space to describe our intentions towards ML models. It may not be the only input space, but it’s hard to see anything ignoring it. If you believe the thesis that language unlocked humanity’s ability to share complex ideas in short amounts of time, then computers learning what to do based on language should be viewed as a similar sea change in how we interact with ML models.

  It feels like we are heading for a future where more computer systems are “do what I mean”, where we hand more agency to models that we believe have earned the right to that agency. And we’ll do so as long as we can convince ourselves that we understand how these systems work.

  I don’t think anyone actually understands how these systems work. All the model disclosure analysis I’ve read feels like it’s poking the outside of the model and cataloging how the black box responds, without any generalizable lesson aside from “consider things carefully”. Sure, that’s fine for now, but that approach gets harder when your model is capable of more things. I hope people are paying attention.

  Comments
• Jan 22, 2022

  MIT Mystery Hunt 2022

  This has spoilers for MIT Mystery Hunt 2022. Spoilers are interspersed and will not be labeled ahead of time.

  So, teammate won Mystery Hunt.

  Since the winner of MIT Mystery Hunt has to write the next year’s hunt, things will be…interesting for the next year. I still need to decide what level of involvement I’ll have in that, but to be safe this post isn’t going to do any speculation about the future of Hunt.

  This year was the first year where teammate didn’t do open signups. In previous years, teammate was around 60-80 people, and based on a team survey after Mystery Hunt 2021, people generally felt the team was growing too big. So for 2022, we did a closed roster of dedicated team members, which was defined as people who helped write Teammate Hunt, or had hunted with teammate in 2021 and at least one previous year. I think this is always a tricky thing to navigate, since the natural state of hunt teams is to grow larger over time, and people will always end up close to whatever line you draw.

  We had 53 people this year, which puts us around the size of Left Out. And this year was a really close race. Left Out was basically tied with us all through Saturday, and Death & Mayhem was ahead of us until they slowed down around Sunday 4 AM. (We…kept going.)

  I don’t think we made any big changes from last year with regards to Sheets norms or remote solving tools. Based on what Wei-Hwa said in the afterparty Discord, I think the reason teammate edged out Left Out was that we were more gung ho about going for the win, whereas Left Out was more uncertain and didn’t fully let go of the brakes until the end. More specifically, Left Out saved all 3 free answers until they were sure about things, hammering them all on Sci-Ficisco. We had used one free answer in New You City, and two free answers in Howtoona, based on our observation that it would be easy to backsolve after we finished the meta. That likely sped up our meta solves and let us make up for our slightly slower puzzle solving average (relative to Left Out and Death & Mayhem).

  I’m not sure how this compares to other teams, but teammate has always had a decent number of people who really like the meta-level strategizing of how to get from the start of Hunt to the end as effectively as possible, and team policy has always been that it’s okay to hunt this way. This means that, for example, you never need to ask permission to backsolve, and every year we’ve had a puzzle that got sniped from someone who was about to finish their forward solve. Depending on viewpoint, this is either fine or horrifying. But, well, it does work, and it can be fun to improvise a good speedrun route.

  Thoughts on Hunt

  Good hunt.

  Fun theming, many puns, solid puzzles, good art, and cool that there was an intermediate tier of accomplishment between the first round and Pen Station. I would have preferred the team interactions to not be recorded ahead of time, since I felt that made it easier to skip the story, but I realize that it’s logistically a lot harder to make that happen while also running New You City.

  So still. Good hunt.

  Pre-Hunt

  Our pre-hunt socials were mostly different groups of 2-4 people solving Star Rats, and I think literally every group backsolved The Rescuers. This probably says something about what puzzles we tend to start working on first. We also played some escape room games and other board games.

  Like last year, we had a #conspiracies Discord channel, where I claimed that it would be important that MLK day was a full moon. Nope, that didn’t matter, but we did find a Scriptnotes episode that seemed too good to be true. It mentioned puzzles, palindromes, and “anagrams of MATE”, and a tweet made on January 17, which was MLK day this year. Based on that and the mention of crossword alignment charts, it really seemed like it was seeding puzzle content. It wasn’t, and in retrospect, the date of the podcast (2 weeks after winning Hunt) should have been a sign that it probably wasn’t important.

  For this hunt, we did three optional in-person hubs, with one in Bay Area, one in Seattle, and one in Boston. This was a bit risky, but I went to the Bay Area one since I felt it was within my risk tolerance. No one was flying in, everyone attending had gotten a booster, masks required, it was in South Bay where people generally take COVID pretty seriously, and you had to show a negative rapid test day of. I’m not too sure how much the in-person hub helped solving, because we still needed to join voice chat to talk to people outside the Bay Area, but it definitely helped on the physical puzzles.

  I had been playing through the Ace Attorney series prior to Hunt. Right now, I’m on the 2nd case of Apollo Justice, which features a noodle shop called Eldoon’s Noodles. The characters mention it’s an almost-palindrome in game, and even talk about “Team Meat” when you inspect it more closely, which added up to an eerie pre-hunt coincidence. And then the first round was called The Investigation! Please, try to convince me this wasn’t a work of the divine.

  

  The Investigation

  The Missing Piece - First puzzle I worked on in Hunt. I recognized the Palindrome name badge theming right away. I did find the 2009 Star Rats badge during hunt, which was a bit of a trip, but I figured there was no way Palindrome would make a puzzle in the first round depend on previous real-world badges. We never figured out what the years meant - after we counted the lanyard overlaps, we were pretty sure that was correct and figured we had just skipped a step somewhere.

  Where the Wild Things Are - This was the first physical puzzle, and mostly got claimed by the Boston hub, but the Bay Area hub got the jigsaw puzzle, which ended up being one of the longer ones. With no patterned edges, pairing pieces up was pretty tough. We figured out all of the mechanics, but our data was incomplete and the meta got solved before we could patch it up.

  The Ministry

  Harold and the Purple Crayon - fun dataset, although I came in after most of the IDing was done and mostly threw around extraction ideas with other people. I don’t love that you’re actually supposed to throw out the ones that don’t fit, but it is neat to see the final step.

  Oxford Children’s Dictionary - We pretty quickly decided that “one side will be regular definitions, and one side will be jank definitions”, but it took us a long time to determine exactly what form of jank it would be. Once we got a few examples, it was pretty fun, although we ended up skipping about half of the letters out of difficulty, solving from “??? budweiser? + ??? seashore? ???”.

  The Talking Tree - Pretty cute puzzle, and somehow these diagrams were a lot easier to fill out by intuition than the ones in the similar puzzle from Silph Puzzle Hunt. I did think it was funny that one person solving had written a phonetics puzzle for Teammate Hunt. Also I just realized how relevant the title is, it’s literally talking syntax trees.

  Dinotopia - I view puzzles as creating order out of chaos. To borrow an analogy from Alex Rosenthal’s talk, it is a coincidence that the number of piano keys matches the number of constellations, but once you know this coincidence exists, you kind of have to make a puzzle that pretends this coincidence is vitally important. Sometimes you have to do a bunch of work to make the contrived coincidence work, but sometimes the pieces are already there and it feels like you’re just discovering it.

  In this sense, it’s really cool that the writing system referenced in this puzzle is inherently ambiguous if you arrange the symbols in the right way, and that parsing out readable text is a satisfying challenge. I mostly figured out a-has of what we were supposed to do and how to extract, then got lunch while watching other people do all the work, which is really a 10/10 solve experience, would recommend.

  The Ministry - When we unlocked The Ministry, we had 25/25 feeders thanks to backsolves. We noticed “bit = binary” right away, and I mentioned that COLORFUL HEAD could be a predicate for “starts with ROYGBIV”, at which point we quickly inferred all 5 mechanics. Since this was the only puzzle we had left, I got to witness the terrifying sight of Sheets locusts descend on the 5 x 25 bit matrix. I think we filled out all the bits in 30 seconds. Very scary.

  Fruit Around - We didn’t pre-solve the “hungry caterpillar” connection before The Ministry, but we had a few minutes between solving The Ministry and doing the interaction with Palindrome, and we guessed “bookwyrm = very hungry caterpillar” in that gap, so Fruit Around was pretty straightforward for us.

  In general, the construction of The Ministry is pretty impressive. I didn’t even realize that the meta answers were also semantic descriptions of the bookwyrm until after we finished Hunt. There are a lot of layers of constraints going on and it’s nice that the circle closes and it all comes together.

  Noirleans

  I mostly missed this round, aside from…

  Curious and Determined - We ended up having to backsolve this puzzle, since we didn’t see the way to map letters to each clue. However, the realization that “wait it’s these colors” was fantastic, and it was funny we said “The Shawshank Redemption isn’t really any color aside from blue and orange, in the same way that every movie poster is blue and orange. I guess the main character’s named Red…wait a second.”

  Lake Eerie

  Large-scale Anthropomorphism - after IDing a few of the animals, we dumped all of the animals into Google at once, and got search results for taikyoku shogi, which translates to “ultimate chess”, a terrifyingly complicated game played on a 36x36 board. We IDed the appropriate adjectives, guessed the Chinese (well, Japanese) numerals were cluing rows, and figured it was a taikyoku shogi chess problem from hell. That seemed like exactly the level of ridiculousness to expect from Mystery Hunt.

  

  Now, if you solved the puzzle yourself, you might have noticed one issue - the game that’s used is actually dai shogi, which is played on a 15x15 board. Most piece movement is the same, but notably, the king in taikyoku shogi is allowed to move up to 2 spaces, which made it significantly harder to decide on the best reply. When we got the king out of checkmate according to taikyoku rules, we figured we had made an error somewhere.

  Interestingly, whether you interpret it as taikyoku shogi or dai shogi, you extract FIC first, and we said “it’s probably something like FICTION or FICTIONAL or FICKLE” many times. We tried the first two, came back after someone told us we were using the wrong game, then got stuck with weird letters from a non-optimal mate where the falcon didn’t capture anything. After a closer reading of the rules, we figured out the igui rule to take a piece without moving out of the pin, extracted a K, and realized it was FICKLE an hour after we talked about guessing it. On one hand, we could have saved a lot of grief, but we also found the most unique part of the shogi logic, so I’m not too upset in the end.

  The Graveyard - If I remember correctly, this was a meta that was only unlocked partway through the round. I took a look when it opened, stared a bit, tried to think of appropriately eerie connections, then said “hang on, aren’t these the Pacman ghost colors?” I confirmed the year lined up, someone else found the Japanese ghost names, and then we went on a backsolve spree. It was a little fuzzy, I believe we ended up solving from ROFA, TIONS, and a penciled in IRAPP on the group we had 3/4 on. I do think it’s a little odd that the Ghostbusters group was ordered by credits, when the names appear after 1/2/3/4 letters of the string.

  The Quest Coast

  A Number of Games - When this unlocked, I figured I would have to work on it because I’d done some combinatorial game theory before. But then it turned out teammate is just a bunch of nerds, because lots of us had done combinatorial game theory before. I drifted off of this puzzle in favor for…

  Something Command - I’m heavily biased, but this is my favorite puzzle from Hunt. We did the math on the Eldrazi one first, in case the indexing was based on how much over lethal you could get. After getting lethal exactly with 2 different lines (attacking with Nettle Drone or not), we believed the extraction would only be based on the missing card name, solving at 4/7 correct cards. I think the puzzle is doable if you don’t know MTG, but knowing MTG definitely made it faster, and it was cool that you could intuit your way to figuring out the missing card even if you didn’t exactly work out all the math.

  Sorcery for Dummies - Cool puzzle - I mostly came in after all the individual letters were IDed, to try to figure out paths for each monster. This ended up overlapping with technical difficulties that took down interactive puzzles though, and during the downtime I started on another puzzle. Once the backend went back-up, I decided I’d rather finish what I started and this puzzle was completed by the time I looked at it again.

  Once Upon a Time in the Quest - When I went to bed around 3 AM Saturday morning, we hadn’t made much progress on this meta. I woke up at 7 AM, earlier than I planned to. I was going to go back to sleep, but I saw a message asking about Dinotopia’s mechanics. The overnight crew had broken into the Quest meta. I got on voice chat, described Dinotopia, then went back to sleep for an hour. Waking up to more forward solves in Quest, I read over the work, saw that THE DARK hadn’t been done yet, considered clue phrases that needed to start “THE DARK” instead of just “DARK”, proposed “The Dark Knight”, and was pleasantly surprised when that was correct. That got us to 6/8 of the Step 3s, which was enough to wheel of fortune the answer. I still think it’s a bit weird that there’s an extra IT’S at the start, but maybe it makes it easier to find good words or get to a nice round length.

  New You City

  Does Any Kid Still Do This Anymore? - I did this as my last puzzle before sleeping for Saturday, so it was mostly about grinding trigrams in a daze. I got Antioch in the first one from Nutrimatic, and decided that was fake until we did a few more examples. I remember saying, “I can confirm that kids still take the BART.” That’s technically true, but the meaning of the title became clearer after we got the main a-ha.

  Bad Beginnings - We didn’t get this until after we won Hunt, but I’m mentioning it here anyways because the dataset used is great and it’s worth revisiting.

  Proof by Induction - I came in at the end, after all the ideas were figured out, to help ID some of the missing lines. I hadn’t heard of this language before and it’s a pretty good one.

  Recipeoria

  Sunday Dinner - Okay, I didn’t actually work on this puzzle, but I clutched the finale. We have a channel where people can crowdsource help on anything, and the call for help was “we have a cluephrase EIGHT PAST, about a NYT Sunday crossword with a food theming”. I tossed some terms into Google and said “SPAGHETTI”? It was right. I didn’t understand the clue at all. (They later explained it was an anagram and wordplays.com must have indexed a cryptic that used it before.)

  Heartford

  Somehow I saw nothing in this round, not even the meta.

  Whoston

  Rotten Little Scamps - I pitched in a bit to some of this puzzle, which sparked an ask for “does anyone have an Icelandic Nutrimatic”, which might be the best request I heard all Hunt. We did not find an Icelandic Nutrimatic.

  Reference Point

  You Took the Fifth - I got tagged to work on this puzzle the moment it opened. More of a word puzzle than an Ace Attorney puzzle, but still good, and it was cool once we figured out the reason it was presented as an objection.lol. It did take us quite a while to decide on the right interpretation of each line, but I believe that was part of the intended difficulty.

  On Second Thought - Now that I think about it, I haven’t seen the phonetic step in very many puzzles before, but it didn’t feel too bad when we were working on this. The puzzle is kinda ISISy, but not in a bad way.

  Diced Turkey Hash - Something Command was my favorite puzzle of the Hunt, but Diced Turkey Hash was by far the most memorable. This was the second physical puzzle, and I do think it’s a shame it unlocked so late. We figured out the binary grid pretty quickly, as well as the Mayan numerals and Tarot symbols. A bit more work got us the Dzonghka numbers as well. From there, we figured that “face-to-face” was a clue indicating we should take a walk between adjacent faces of the d20s. Based on the given text, we started identifying mappings between pairs of dice, noticing more relevant text every time we reread the given text. It’s impressively dense, and the “director’s chair” realization was great.

  We struggled with it a lot, but given it was one of the least solved puzzles in Hunt, it wasn’t just us. Our difficulties came from two rabbit holes. First, the flavor about going on a journey really felt like we needed to trace a path along both pairs of dice, but the final extraction only relies on the mapping between the two dice, which could be determined without looking at the face topology. The given text about the journey definitely helped us confirm we were doing the right thing, but we also managed to translate it into a reasonably unique path that visited each face once. The most extreme argument was when we did the Tarot dice. We aimed for a path through all prime numbers in descending order, taking the shortest path when possible. At one point, the two primes were on completely opposite faces, and there was no unique shortest path, but I argued that “be a walker, not a rider - but do not wait” could be interpreted as “take the shortest path that does not go through The Chariot”, which left only one path on the d20. This was not the intention of that text, but it did lead us to writing down the correct pairing (via a circuitous route).

  The other rabbit hole is something Palindrome definitely did not predict. We had rolled the dice, and didn’t spot anything weird in what face it landed on, but then we decided to try rolling them into water, which would magnify any weight difference. To our surprise, there was a consistent face that would point up. Given this didn’t show up at all when rolling onto a table, I was not a believer, until I tried for myself. I’ve recreated a video below.

  I was still not convinced, and argued that other d20s might show the same behavior, even if they were fair. One person drove home and back to bring control group d20s, while we recruited help from remote solvers.

  

  All the control group d20s landed on different faces between trials. And thus our solve group was split in twain - we could not deny the dice were weighted, but did it matter to the puzzle? Or was it just a manufacturing defect?

  Arguments for it mattering:

  • This could be a reason this was a physical puzzle.
  • The dice feel high quality, which is evidence against manufacturing defects.
  • Previous Mystery Hunt puzzles have used gimmicked dice before.
  • The given text mentioned “an ocean of blue” - perhaps this was a hint to use water to figure out the dice were weighted.

  Arguments against:

  • The distribution of faces when rolled onto a table is not clearly weighted (tested by doing the eye test of rolling the dice many many times). It would be very easy to miss the weighting unless you thought of the test we did.
  • Nothing about the puzzle presentation suggested the die would be weighted.
  • The gimmicked dice from the previous Mystery Hunt puzzle were more clearly gimmicked than our dice.

  No one was making much progress convincing anybody else, so we decided to settle it by seeing whether the virtual version of Diced Turkey Hash left any way for a virtual solver to learn the dice were weighted. Only problem was that it wouldn’t unlock for another hour. So, we spun wheels a bit, until the virtual version unlocked and we learned that no, the virtual version only showed the faces of the each dice, and the weighting did not matter in the slightest.

  It wasn’t real, it certainly wasn’t intentional, but it was a very entertaining argument, so thanks Palindrome, and thanks to the sponsor HRT for manufacturing slightly unfair dice.

  Reference Desk - We random anagrammed the answer, and just could not figure out the ordering step when trying to backsolve. Without the ordering idea, backsolving was pretty impossible, so we just left the round incomplete. This likely contributed to our lower total solve count in the end.

  Howtoona

  How to Install a Handle - The moment this unlocked, I scrolled down the page, and said “YOOOOO IT’S MATHDANIEL SQUIRREL”. I had actually emailed the relevant dataset to Ryan North a week before Hunt thanks to this Dinosaur Comics, so it was fresh in my mind. I can confirm that he didn’t know about the bracket before, but does now.

  How to Make the Right Move - The pair of us that first looked at this puzzle immediately figured out the puzzle mashup, then tagged other people to work on it. I credit working on the Sleeping Beauty meta from Inception Hunt. I also believe we sent an errata request about board 9, claiming it was impossible. It wasn’t, that board was just too bigbrain. We got it eventually.

  How to Require Some More Assembly (Picture Puzzle 3): I got to come in to save extraction, which is always fun. Saw a completed grid and a highlighted entry, interpreted it slightly differently than the group that filled out the grid, and then we solved. Although it took us a while because we read “invert Y” as “flip across Y-axis”, which made the final image look more like a vampire bat than the actual answer.

  How to Do Quality Reviews - Helped with initial data entry, but then it hit the part of the puzzle that was not very parallelizable and I drifted back to…

  How to Find a Component - I opened the meta at the start of the round, didn’t see how to do anything, and left. A while later, I opened our meta sheet again, and saw someone had written “these numbers are the alphabetical permutation of the given words”. Seeing that all our answers were 9 letters long with unique letters, I wrote up my theories for how I wanted the meta to work, then tried starting from 3/9. We didn’t have enough pieces to place anything, but I suspected the mechanic was very constraining on the answers. I was right. After applying the constraints that the answer was 9 letters, all letters were unique, and numbers in each column should be unique, my Scrabble dictionary was reduced to just 226 words.

  A teammate and I spent the next 30 minutes looking for thematic backsolves on abandoned puzzles, and failed to backsolve anything. In retrospect, most of the puzzle answers were not very thematic, and I suspect this was intentional. The constraints are really strong if you squeeze them for everything you can.

  Around this time, we were down to 1 manuscrip left, and win-comm (the group watching overall strategy) proposed using the free answer in Howtoona. We had gone up to 5/9 Howtoona answers, and our estimate was that we’d need 1 more answer to crack the meta open. However, multiple puzzles in Howtoona were moving forward without getting stuck, and if one of those got finished, we could instead have an extra feeder in Sci-Fi. We knew that teammate was in contention to win (since we had gotten a phone call saying so), but we also knew we were not in the lead on unlock progress (because one puzzle had an errata issued 30 minutes before we unlocked it). After some discussion, we said to give us 15-20 minutes to solve the meta, then ask again. About 20 minutes later, we were still at 5/9, increasingly confident we’d need a 6th answer, and unwilling to backsolve for it because our candidate word list was too big. Given that, we pulled the trigger and redeemed our manuscrip on a Howtoona puzzle no one was looking at…right as the 6th feeder was forward solved. At 7/9 it was pretty trivial to backsolve the remaining answers and we finished the meta within 5 minutes.

  I think the win-Hunt play would have been to use the manuscrip 20 minutes before we did, and the max-fun play would have been to use it on a Sci-Fi puzzle and let people finish their Howtoona puzzles. Instead we did something in-between that was somewhat unsatisfying on both ends. It wasn’t a perfect call, but it was a tricky decision given that teammate agreed we were both going to have fun and go for the win. I think the one we came to was acceptable.

  In case you were wondering, that previous section is what I mean by “meta-level strategizing”. A conversation like that tends to happens every year.

  The Plot Device

  (This is wildly non-chronological, because I worked on this throughout the Hunt, but this felt like the approximate time where I did the most work on it.)

  Narnia Beeswax - We got our first Plot Device solve in the middle of this puzzle. My understanding is that the group working on Order of Apparitions was getting really fed up with the “cluephrase” they had, and tried guessing the whole thing in frustration. In Narnia, we confirmed that we could submit single answers, and good thing too - it took us so many tries to solve EQUALLY SNUG. In general, the answer guess timeout was more annoying in this round, and we often spent 5 minutes locked out from guesses.

  A Crying Shamus - I was not good enough to help on any of the cryptics, but I was good enough at searching all the words with “mystery” and “detective” to figure out the answer ordering. I was also good enough at reading last letters to finish the puzzle. For whatever reason the last letters popped out for me more than the first ones.

  Synonym Toast - Just a neat word puzzle in general, although we ran into an errata on enumeration. I think we explained the issue to Palindrome quite poorly, because it took a few iterations for us to explain what we thought the error was. Sorry!

  Step by Step Ladder - We had tried to presolve the Plot Device meta by building a word ladder, so when we unlocked the puzzle that was actually a word ladder, we sped through it pretty quickly (10 minutes based on our solve info).

  Sci-Ficisco

  I missed everything from this round aside from the meta, but I’ve heard good things about both Replicator Droid and Lists of Large Integers

  Communicating With the Aliens - When we got to this puzzle, it was about 3:30 AM on the West Coast, and I was listening to ideas about lines of symmetry without really having much input. I felt I needed some sleep but also knew we were really close to finishing, so I decided to take a 30 minute nap and hope we’d have 1 or 2 more feeders by the time I woke up. I set one alarm (and 5 backup alarms), woke up at 4 AM, physically got out of bed at 4:20 AM (nice), and tried again. It took us an embarrassingly long time to reorder our puzzles in the given round order, and after doing so it also took a while to get just 1 of our answers in the grid. We had a lot of competing theories about what the symbols meant, none of which let us place the DES MOINES answer. Eventually, we got it in, and once we noticed how the three-dot symbols lined up, the rest was pretty fast. I decided to start the Nutrimatic query while most people were placing words, and mentioned a lot of the regexes were matching phrases starting with FIRST, which oriented us to the right pun idea.

  Endgame

  We entered endgame knowing that the coin hadn’t been found yet, but we also knew we were probably very close with other teams, given we were behind on puzzle unlocks at one point. Our best hope was to assume we had leapfrogged teams on the metas and could finish endgame before they caught up.

  Battery Pack - Unlike other teams, we did not pre-solve the Plot Device meta, but it was pretty clear what to do once we saw the shape on the meta page. We complicated it for ourselves a bit by assuming it would mix-and-match across all answers, rather than coming pre-grouped.

  The Tollbooth - It took us about 1.5 hours to finish this puzzle, and I’m really curious what the solve time was for other finishing teams. We split into three groups: one to find books, one to solve book titles, and one to solve the printer’s devilry clues. We got most of the data, and assumed the extraction would be based on the pairing the puzzles described with the printer’s devilry puzzle. When we failed to notice anything special from that pairing, we tried increasingly conspiratorial ideas, including “take random letters out of each word to spell a punny phrase”, which got surprisingly far on building a reasonable yet totally incorrect answer.

  With the entire Hunt as a potential data source, we tried a bunch of things, like the number of lines on each PDF, or the map on the Pen Station round page. After what felt like an eternity, someone who decided “borders” was important noticed the leaves on the PDF. We did the indexing, and won Hunt.

  Personally, it didn’t really feel real. For me, it felt like, “oh, we won, wooooo, I’m going to sleep.” Even now, Mystery Hunt 2023 feels like this thing that doesn’t exist yet, and it’s hard for me to have an opinion on it before it becomes more solid.

  Maybe I’m a little jaded, but winning Mystery Hunt didn’t light any big fires of motivation for me. I’ve already exorcised all my puzzle demons and written the puzzles I had to write. The ones left in my puzzle ideas file are generally uninspiring and don’t feel good enough for Mystery Hunt. After writing puzzles fairly continuously for 3 years (MLP: Puzzles are Magic into Teammate Hunt 2020 into Teammate Hunt 2021), I have a better sense of how easy it is for me to let puzzles consume all my free time and destroy my ability to make small talk or start any other projects that need concentrated effort. Sure, making puzzles is rewarding, but lots of things are rewarding, and I feel I need to set stricter boundaries on the time I allocate to this way of life - boundaries that are likely to get pushed the hardest by working on Mystery Hunt of all things.

  The opportunity to write for Mystery Hunt doesn’t come around very often, which makes me think I should go for it, but I’m not expecting to write anything super crazy. Hunt is Hunt, and I am cautiously optimistic that I have enough experience with the weight of expectations to get through the writing process okay.

  Comments
• Dec 31, 2021

  "My Soul is Pony-Scarred for Life Because of You"

  (Content warning: more swearing than usual, lots of ponies.)

  I’ve really put off writing this post.

  I’ve been watching My Little Pony for 10 years. 10 years. Good lord, that’s a long time. I thought I would leave the fandom in 2019, after the Friendship is Magic series finale. Then I thought I would stop watching in 2020, after finishing my fan projects. Then I thought I would stop in 2021…and here I am.

  I’ve known for a long time that I wanted to write something about what My Little Pony and its fandom mean to me, but I’ve put it off for a long time because it brings up a complicated group of emotions and ideas that are hard to explain. This post is going to be long, so let’s get started.

  How Did I Get Into My Little Pony?

  I first got into My Little Pony via the show. Then I got really into it because of the fandom.

  This post is not trying to get anybody else into the fandom. I decided long ago that it wasn’t worth doing so. But, I can’t explain what the Friendship is Magic fandom means to me if I don’t explain why the show appealed to so many people, and how that laid the foundation for everything that came after.

  I started watching in 2011, during the hiatus between Season 1 and 2. I was a high school student attending Canada/USA Mathcamp that summer. Mathcamp is a summer math program that introduces talented high school students to interesting university level math, like topology or complexity theory. One thing special about Mathcamp is that it encourages student-organized events. As long as you get enough staff to agree to supervise, students can organize anything they want. The brony phenomenon was in full swing, a few students were fans, some staff were fans, and they organized a watch party. I went, watched a few episodes, saw the appeal, and resolved to watch the rest of the show later.

  Not everyone who went got it. I distinctly remember one staff member who wanted to give the show a chance, but was completely baffled why MLP was popular after watching a few episodes. We had just finished “Call of the Cutie”, so we all called him a blank flank, which just sparked more confusion. Good times.

  Season 2 started out really strong, and I stuck with the show all the way until its final season in 2019. Friendship is Magic has its ups and downs, but at its core it’s a wholesome, well-animated cartoon that shifts between light low continuity slice of life episodes, and high fantasy adventures where the fate of the world is at stake. The characters have fun personalities, they play off each other well, and then occasionally it turns into a magical girl anime with big rainbow laser beams. What’s not to like?

  Of course, people got the wrong idea. No, it’s not about the 1980s Generation 1 cartoon, it’s almost exclusively about the Friendship is Magic cartoon from 2010. No, it’s not some fetish thing, most of the fandom just likes the show. (There is some porn, because people are always horny, but it’s not a focal point.) No, it wasn’t some elaborate gaslighting ironic joke. Maybe for some it was, but, again, most just genuinely liked the show.

  There was other criticism too. “I can’t believe you’re watching a kid’s show.” When people said this, I don’t think they actually meant “shows made for kids can’t be good.” All of Pixar and Disney’s best work is a clear counterexample. I think they really meant that it’s easier to get away with a bad cartoon that’s all flash and no substance if you market it to kids. A lot of people mentally bucketed My Little Pony as one of those shows. That may have been true before (Generation 3.5 is legendarily bad), but every reboot is another chance to make things right.

  In short, yeah, My Little Pony: Friendship is Magic is an advertising vehicle where the toys come first, but it’s a fun advertising vehicle. Sometimes making something good is the right way to make money. I wouldn’t say the writers succeeded all the time, some episodes are really bad, but the good ones are great. In context, the story told in “The Perfect Pear” is exceptionally cute and emotional.

  

  A good show explains why a fandom exists, but it doesn’t explain why the fandom grew so explosively during 2010-2013. I’ve heard a brony describe it as a lightning in a bottle, and I think that’s true. It was dumb luck, amplified through the Internet.

  In 2006, the sociology researchers Salganik, Dodds, and Watts had a question: given that every publisher wants to identify the next hit song, why is predicting song popularity so hard? They came up with the Music Lab, an artificial music platform. Participants were asked to listen and rank new songs. Optionally, some groups were allowed to see rankings from previous participants.

  The paper is here, and the short version is that song popularity was just genuinely unpredictable. A popular song in one group would be near the bottom with the next. Interestingly, the more people observed other people’s rankings, the more volatile the popularity would be. That suggests some level of herd mentality - initial popularity is very random, but once you get popular, it is easier to retain popularity because people defer to the crowd and attention is zero-sum. I assume these dynamics are even stronger now compared to 2006, given a bigger Internet and stronger recommendation systems.

  In most realities, My Little Pony: Friendship is Magic is a quietly successful cartoon that sells a bunch of toys, and otherwise leaves no mark on popular culture, outside a bit of nostalgia many years later. In our reality, it won the cosmic lottery, and after it did, the premise and size made it too novel to ignore. Not the premise of the show: the premise of the fandom. “Several 13 to 30 year old men swear up and down that the new My Little Pony show is worth watching” was a bizarre enough meme to spread far and overpower many existing aversions. The only reason I gave MLP a chance was because I was into Homestuck at the time, and MLP profile pics and references were appearing on the MSPA forums.

  The show staff certainly didn’t expect it. In a panel with showrunner Lauren Faust and writer M. A. Larson, they talked about what it was like to work on the show during Season 1, and in particular, the day they watched a bunch of Russian teenagers singing along to Winter Wrap-Up. In that moment, the brony fandom they never predicted clarified into something real. The only thing left was deciding what to do about it.

  “Fans are Great. Fandom is Weird”

  There were haters, like there always are. Every culture creates its counterculture, and bronies are no different. Even so, bronies got it especially bad. It was astoundingly difficult for fans to explain the appeal to outsiders, and even if they did, it was incredibly easy to write off the whole thing as a bunch of weirdos.

  This may be an unpopular opinion, but the MLP fandom totally deserved it. Kids didn’t deserve to be bullied for liking My Little Pony, but the fandom as a whole definitely deserved some gentle teasing. You know the old saying:

  “Fans are great. Fandom is weird.”

  (Terry Pratchett, supposedly, but I haven’t found a primary source.)

  I’ve yet to see any fandom that wasn’t weird in one way or another. I mean this in the most endearing way possible, but I’ve been in Touhou fandom. I’ve been in Homestuck fandom. Let’s say I have a pretty calibrated sense of how weird fandoms can get. MLP fandom was and is weird. That’s why I love it.

  We’ll see what I mean later, but for example, it was a meme to say “welcome to the herd” when someone declared they were a new member of the fandom. During the animating process, ponies were copy-pasted in the backgrounds of different episodes to save on animation costs. The fandom made up entire names and backstories for those characters, before any of them had gotten a speaking line. The character designs for Derpy, Whooves, Lyra, Bon-Bon, Octavia, Vinyl, and others were set instantly, but all the canon stories afterwards were heavily influenced by what fans wanted to see. Then there was the endless fan music, which ranged from remixing musical numbers in the show, to original music that sampled pony vocal chops, to instrumental music that was just inspired by the show.

  Elements of this appear in many fandoms, but brony fandom was unusually prolific. The only good explanation for this level of content is passion, but that’s not a very satisfying answer. It just punts the question to why people were so passionate about a show that was good, but certainly not the Second Coming.

  For their part, the show staff supported the fandom, and Hasbro said they didn’t plan to shut the fandom down, given it was free publicity. They made a poster for San Diego Comic Con 2011, and starting with Season 2, they made slight nods to the adult fandom here and there.

  

  I don’t think Hasbro could have shut down the fandom even if they wanted to. It was bigger than them now.

  I think the explosion of MLP fan content is most easily explained by the MLP universe’s untapped potential. If someone does good worldbuilding, you bet people are going to do more things in that world. In this respect, Friendship is Magic was almost hyperoptimized for fandom. The ensemble cast made it easy for fans to find at least one character they found compelling. The simple pony templates made it easy for people to make original characters. The show makes frequent allusions to world mythology, the history of Equestria, and other kingdoms ruled by different creatures, but rarely dwells on them for too long because they don’t serve the plot. This is the right move from a storytelling point of view, but unanswered questions beg fan answers.

  There are also out-of-universe factors. When Hasbro wants a new toy line, the writers have to write it into the show, whether it makes sense or not. Sometimes, network executive intervention makes the show worse. According to rumor, the Season 3 finale felt so rushed because the writers begged (and failed) to get approval for a 2-part episode. Other times, it isn’t even the executives’ faults. The writers just drop the ball and put out a stinker. And, since Friendship is Magic has to stay family-friendly, there’s inherent constraints on what the show staff is allowed to depict.

  My fave note on #MLPSeason4Finale “We cannot show Twilight punch Tirek in the face”

  — Meghan McCarthy (@MMeghanMcCarthy) May 10, 2014

  

  (Original storyboard of Twilight v Tirek fight.)

  You couldn’t ask for a better sandbox to play in. And fans did play in it. They did all kinds of crazy shit. Sturgeon’s Law still applies, but the MLP fandom was prolific enough to produce lots of content in the 10% that wasn’t crap.

  Want a voice-acted 9 hour Ace Attorney x MLP crossover video series? Sure, why not. Turnabout Storm is slow at times, but it does a great job at capturing the tone of Ace Attorney, has lots of shout-outs to both IPs, and the case’s resolution is excellent. Similarly, when I reread Fallout Equestria, I felt it still held up as a case study in how to write a protagonist who uses their abilities to their full potential, and villains whose actions are consistent with their end goals.

  Those are both stupidly long, so if you want something shorter, how about Ponycraft 2 or Shingeki no Pony? Both are playing in the what-if space of “what if My Little Pony was a darker, more violent show”, crossing over with Starcraft and Attack on Titan respectively, and there’s some inherent humor in the juxtaposition between the two.

  

  Mad Pony: Friendship Road, by Yulyeen

  Want something without a crossover? Well, it’s not as popular, but I’ll always talk up the pony music video The Stars Will Aid in Her Escape, set to “Cosmic Love” by Florence and the Machine. It’s a cool discovered association, with a lot of nice visual editing. The pairing between “The stars, the moon, they have all been blown out, You left me in the dark” from Cosmic Love and the story of Luna’s turn to Nightmare Moon is uncanny, as well as the observation that both twilight the time and Twilight the character act as the bridge between Sun and Moon.

  In some ways, fan content can be better than original content. Fanwork is like painting a tapestry, but instead of the standard colors like red and blue, you get more exotic colors like Reimu or Papyrus. This is inherently more limiting, and narrows the target audience to niche audiences, but in exchange, that fan work can convey very specific feelings or ideas to fans that have the required background. Those references can then further build on each other, creating very dense networks where every fan creation has ties to several other ones, producing endless variations on a common theme. Original work can’t do this, or at least not to the same degree. The reference pool is just too wide for this level of interaction.

  The fan work may not objectively be higher quality, but it can subjectively be better, and when it comes to entertainment, you like what you like.

  Falling Down the Rabbit Hole

  Perhaps it’s silly that bronies ascribed so much meaning to an IP that has minimal real world value, and could carry on without them. But, again, they like what they like. We like what we like. The real world is inherently a pretty brutal place, and the universe really doesn’t care about anything. When faced with an unstoppable meaning-eating monster, anything that fights against that is good.

  I remember during the aughts, when I was first trying to work my way into sports media, the popular line among the cool kids was that things like sports are a distraction that monopolizes peoples’ attention and energy that otherwise would go into enacting real political change. But things like sports are the fucking point! MMA, or learning how to play the lap steel, or thrift fashion, or Counter-Strike, or Scrubs fan fiction, or whatever in the world it is for you. That’s what you’re fighting for, if you’re fighting. Every hour you get to spend in that world is your victory against all this.

  (Fighting in the Age of Loneliness: A Conversation with Felix and Jon)

  I’ve come to realize that I put a lot of respect on people who take a nonsense idea, with minimal usefulness to the real world, and just run with it to completion because they want to. The lack of usefulness is almost the entire point. This doesn’t make me real money. It doesn’t make me smarter. It doesn’t help me on dates. It’s just a thing I do, in the time I have. Nonsense only exists because we’re serious about making it. There’s too many things that get dropped at the first sign of resistance, and I’d rather live in a world where people follow through on what they’re passionate about.

  When it comes to fandom in particular, it’s interesting because a critical mass of passion leads to content that brings in more fans, leading to more content, and more fans, and so on. You get these engines of creation that twirl further and further away from show canon, and it’s fun to trace the web of inspiration. Fallout Equestria was an influential fanfic, which then spawned its own subgenre, a few game making projects, and not one but two different fanfiction print projects. The former focuses on just the Fallout Equestria universe, while the latter takes its name from FoE and prints a wider selection of MLP fanfic, if you can stomach the shipping cost from Russia.

  Fighting is Magic was a 2D fighting game made by a group of fans with solid fighting game design skills. They made enough progress to show it off at side events for Evo, one of the biggest fighting game tournaments in the world. The high profile drew a cease and desist letter from Hasbro, and they complied, shutting down all development. In response, Lauren Faust offered to do some character design work, and Skullgirls licensed their engine to the team for free thanks to meeting a crowdfunding goal for Skullgirls DLC. Given a lifeline conditioned on redoing all art assets and implementation in the new engine, they renamed to Them’s Fightin’ Herds, ran a crowdfunding campaign, and released in early access on Steam in 2018, about 6 years after their first build. The game still receives updates today, and although story mode (still) isn’t done, its existence at all is a bit of a miracle.

  I dunno how familiar people are with fan projects, but there are probably hundreds of ideas for fan projects starting every day. I would say 99% of them never go beyond the idea phase. It’s exceedingly unlikely that any given fan project will go anywhere at all, let alone stick together for as long as we have. Of the ones that do get past the idea phase, most of them are filled with passion, but are lacking skills. […] The tiny team we started with, from a random collection of people who happened to see a single post on a single forum at a certain time happened to get along and have the drive and skills that complemented each other and most of the gaps were filled right off the bat.

  (Interview with TFH dev team, circa 2015)

  The game has been divorced from its My Little Pony origins for years, but it’s thanks to the show that it exists at all, and I’m happy it’s moving towards its own identity. They released their first new character this year, and she’s a pirate goat that can cling to walls how is this not the greatest?

  Sometimes, you don’t have a team. You have one crazy person. Consider Lullaby for a Princess, an animation by WarpOut. It’s 7 minutes long, and you could make a good argument that it’s the best thing the fandom’s ever made.

  The backing song “Lullaby for a Princess” is by ponyphonic, a duo who imagined a song Celestia would sing to Luna after banishing her to the moon. WarpOut was inspired by the song, and decided to animate it with an exceptionally high quality bar. Although the music and backgrounds were done by other artists, the meat and potatoes of animating was done by one person, frame by frame, averaging 0.5 seconds of footage per day.

  Footage from behind the scenes video

  Much of my interaction with the fandom now is just through the pony music scene (which, yes, is a thing). You’ve got Ponies at Dawn as the “big” label, but then there are smaller labels like Equinity and new labels like VibePoniez curating more lofi music. Which all feeds into sites like the Equestrian Trot 100 (no longer maintained), or Horse Music Herald, as well as pony radio stations like PonyvilleFM.

  

  (Source)

  These songs get a few thousand views. The launch parties get around 100 to 200 listeners. Nothing crazy, but good enough for people to keep tuning in. Sometimes, you have to write a song. If My Little Pony is the inspiration, so be it. Maybe somewhat fittingly, my favorite song from the pony music community is itself a remix of another fan song - The Wasteland Wailers’s acoustic cover of On Hold by YourEnigma.

  When making this post, as an exercise I tried drawing out the web of MLP fan content I could recall off the top of my head. I didn’t get very far before realizing this was insane and I needed to stop.

  

  (Click for full size)

  There was a certain energy in the fandom, where people felt empowered to make whatever they wanted, along with an assuredness worn by fans who knew the community would keep growing and last forever. And like every group in history, they were wrong.

  Time Marches On

  Compared to what it was in the early years, the brony fandom’s a shell of what it was. The golden age is over and what’s left is much less definable. People in the fandom will get upset if you claim it’s dead, but it feels unlikely it’ll ever recover in size to what it once was.

  You could try to blame it on Season 3. Controversial at the time, and still near the bottom of “best season” polls. You could blame it on the shifts in the show’s tone. As the show evolved, it “grew up”, making more nods to the adult audience, which not everybody liked. Some felt this cheapened the innocence of the show, or that giving fans what they wanted made the show worse on an objective level. Or, maybe it’s just seasonal rot. Slice of life shows tend to suffer after they exhaust the obvious storylines. I think the most likely explanation is that it’s the same thing that happened to Pokemon Go. For a short time, the world collectively had a fever dream and got in on the hype. Then the hype faded and the world moved on.

  I’ve long hesitated to call myself a brony, even though I’ve watched the entire series, read about 2 million words of fanfic, created a custom MLP-themed Dominion expansion that I tried to get into Equestria Daily, and most recently led writing for My Little Pony: Puzzles are Magic, an MLP-themed puzzlehunt.

  I will let the defunct ethnography blog Research is Magic explain why.

  In my dissertation research, I study people who self-identify as Vietnamese and how they form connections with one another in local communities, on national and continental levels, and even transnationally. I’ve purposefully avoided categories like “Vietnamese Americans” or “the Vietnamese diaspora” because they tend to presuppose certain sorts of relationships that people may or may not have with each other […]. Therefore, I have often used terminology like ‘self-identify as Vietnamese” because the identification of oneself as Vietnamese means that a person is self-consciously engaged in identity work of a sort that produces potential links to other people who also self-identify as Vietnamese.

  […]

  Kurt and I say often, “I’m not sure I’m a brony,” and we have both wondered about what keeps either of us from taking the plunge, so to speak. I would venture a guess that our anxiety stems from an incongruity between the moment we started to construct for ourselves an idea of a brony community and its relationship to our other affiliations. For me at least, I had been a casual viewer of the show for a long time. Even once I heard that bronies were a thing, I did not feel a need to reevaluate my watching of the show in those terms. I was part of the public/counterpublic in the sense we began with—of a kind of rapt attention to a cultural object—but since there was no utility for me in identifying with others, I felt no need to do the associated identity work.

  Once Kurt and I decided we were going to do research on bronies, then the community became a “thing” for us…but it also meant that we at that same moment already had identities vis-a-vis that thing: brony researchers. Having produced the relationship with the fandom as one of a scholarly gaze, it’s been difficult to take on the brony label, even though we’re often doing exactly the same thing bronies are. The way the imagined community is imagined, the moment at which it is imagined, and a person’s relationship to that imagined thing all become important considerations.

  Research is Magic: “What Sort of Group are Bronies?”

  I’ve always preferred calling myself an “MLP fan” or “member of the fandom”, and I know I’m not alone. When I went through the feedback for Puzzles are Magic, I saw someone say they’ve seen every episode, but don’t consider themselves a brony. Or a couple said they and their partner called each other Flutter Butter and Dashie, but they didn’t consider themselves bronies. And then meanwhile, I go to a pony convention, where you would assume everyone self-identifies as a brony, and an attendee says they’re 2 seasons behind the show, and a panel gives a warning that their presentation includes spoilers for the currently running season, when you’d think catching up on the show would be a prerequisite of showing up.

  The people who call themselves bronies are the ones who get use out of calling themselves bronies. They get a sense of fellowship in adopting the label. I never found that fellowship. It was fun to watch the show, and watch fan content, but I never felt the need to stand up for a generic brony, nor did I ever take the jump to actively participate in fandom discussion, or work on fan projects with people I hadn’t met before IRL.

  It’s for these reasons that I found myself having less fun at brony conventions. The first one was great, but afterwards, not so much. Fandoms all have a pyramid of intensity. Of all the viewers, some become fans. Of those fans, some produce content or go to fan sites. Of those people, only some go to conventions. So at conventions, you don’t meet the average fan. You meet the super hardcore fan. Which, to me, is split between very cool people, and people I never want to interact with in my life. Dealing with the people I don’t want to interact with takes up all the energy I have to meet everyone else. (I feel I have a similar reaction to Bay Area rationalists.)

  The fans I get along with realize the absurdity of the brony fandom, and just go along with it. They can seamlessly jump between hating on your favorite pony, and asking if you want to join them for lunch. Maybe they acknowledge that they’re sort of there for the ponies, but really there for the people. Or, they come up with genuinely impressive surrealist humor. I have yet to hear a more brain-stopping question than “If you could ship Twilight Sparkle with any pre-2000 US President, which would you pick?”

  Maybe it’s just because of the demographics, but man is there a lot of shitposting. Like donating an empty box of Froot Loops for the charity auction. It is at least signed by the Toucan Sam voice actor, but it’s also a cereal box.

  

  Or, well, this nonsense from 2018.

  

  (A dabbing Ugandan Knuckles, spinning fidget spinners while eating a Tide pod, attached to a container of Tide pods not to be opened until after Friendship is Magic ends.)

  You know if this kind of humor works for you. For me, it does! I appreciated getting to watch someone do a live rendition of “We Are Number One” in full Robbie Rotten cosplay. But, memes often don’t have any deeper meaning, and trying to live off just memes is like trying to live on only french fries. Talking to someone who only lives off memes is even worse.

  The more time I spent in these convention spaces, the more I started to see the dreary plain white conference room walls instead of the magic. The decline in popularity was becoming starker. Going to a panel hosted by someone trying their best to work a room and failing would be fine if the seats were packed, but it was 1/3rd full at best and that just made it depressing.

  There were some good conversations, but there were also content creators whose work I respected acting like assholes. Or panels that felt like excuses to hoard the meager clout still achievable in horse fame, rather than put on a show for fans. There were too many people who were there just to feel something. To get excited about being part of something greater, and having no avenue for that besides ponies. A sort of repressed nerddom exploding out at once, with no sense of perspective or self-awareness on what it meant in the grand scheme of things - absolutely nothing. Ponies are cool, but I don’t want them to be the only thing in my life. I’m perfectly okay with spending an irresponsible amount of time on MLP, as long as I know it’s irresponsible.

  Unfortunately, there are people who don’t know it’s irresponsible, lack perspective on its irresponsibility, and can’t leave the MLP community because all their friendships and connections are based on MLP. They are also the people who always go to conventions. Horse conventions are their churches. The places where, for one brief weekend, the fandom is the real world, and as all-encompassing as they want it to be.

  This is a subtle point, and I want to be very clear about it: I don’t have a problem with people like this existing. I think every hardcore fandom has some people who truly had no friends, community, or stable family situation before discovering the fandom. There are tons of stories of people who went through dark times, who likely would have taken their own life if they hadn’t found something to live for, in something as wonderfully goofy as My Little Pony.

  It is clearly good that they found that something.

  “People have come together and formed relationships and got married and had kids during the course of the show, because of the show … and we were wondering, how many kids owe their existence to this show?”

  (Jayson Thiessen, chief director of My Little Pony for multiple seasons, when asked about the show’s impact)

  I wish them all the best. I just don’t want to talk to them. Maybe that makes me stupid, or selfish, or hypocritical to be so judgmental in a fandom whose motto is “love and tolerate”. I don’t know. That’s always been one of the struggles in MLP fandom. The show drew together people from around the world, including opposite sides of the political spectrum, and when the highly online far left and far right learn about each other, the peace can only stay for so long.

  When your entire world is a single fandom, small conflicts become life-threatening attacks. Bad episodes become existential threats. Every time drama happens in the brony fandom (and there is always drama), I wish I could force everyone involved to watch and understand “This is Phil Fish”, a discussion on Internet fame (or more specifically, what it’s like to be a media object defined by people hating you).

  Imagine you’re at some Manhattan cocktail party, and in conversation with a stranger you’ve only just met, you say, “Can you believe this bullshit Phil Fish said on Twitter?” Do you think the more plausible response is

  A: God I know, what an asshole, or
  B: I don’t really care.

  The correct answer is C: Who the hell is Phil Fish?

  Phil is not famous the way we are used to thinking about celebrities. Despite being in a movie (a documentary about video games), the average random passerby has no idea who he is. […] The world at large does not know or care who makes video games. FEZ has shipped a million units, so in a random sampling of 7000 strangers, it would have been played by 1 of them. Phil is subculturally important, not culturally important.

  He’s only famous to us.

  In 2019, I had given myself a resolution: after I finished Puzzles are Magic, I’d move on from My Little Pony. It was getting smaller, I’d seen everything conventions had to offer, and I had some merch to remember it by. What else was there to do?

  One Last Ride

  BronyCon was the largest My Little Pony convention. Starting from small meetups in the New York City area, it later moved to New Jersey, then Baltimore, reaching 10,000 attendees by 2015.

  I say “was”, because BronyCon ended in 2019. There were a few factors going into it. Attendance was declining about 20% year-over-year from its peak in 2015. Friendship is Magic’s final season was airing that October, just over 9 years after its first episode. Being the biggest con was part of BronyCon’s identity, and organizing staff decided they would rather end BronyCon with one big finale coinciding with the last season, rather than carrying on and watching it grow smaller each year. Given there were other MLP conventions with no plans of stopping, I thought it was a good call.

  (To this day, my favorite video about conventions, from any fandom)

  I waffled back and forth over whether to go, and decided that if I didn’t go, I would regret it for the rest of my life. So I went.

  Evidently, others had the same idea. In 2018, they got 5,465 attendees. Knowing it was the last BronyCon, organizers expected a bump to around 7,000 or 8,000. They got 10,000. All sorts of people turned out. Lauren Faust, who had moved on from the show after Season 2. Old fandom members, who got super into it, then left the fandom in 2014, coming back to catch up with old friends for one weekend because it felt right. New fans who started just 1 or 2 years ago, well after the hype had faded. My Little Pony is now at the stage where kids in the target demographic of 6-7 years old at the start are almost adults now, and a bunch grew up with the show in the same way my generation grew up with Harry Potter. Every year, BronyCon got international attendees, but this year was the last one. If there was a time to splurge on international flights, now was it.

  BronyCon. Was. Nuts.

  

  I felt the fandom had told itself a collective story: the show’s ending. People’s lives are moving on. There will always be a core group of fans keeping the fire lit, like the Trekkies between the Original Series and Next Generation, but that’ll be it. The fandom believed it was not capable of pulling off big conventions, and the entire weekend was spent proving that wrong. People just needed a reason to come out of the woodwork.

  We all knew this wouldn’t last. BronyCon was only so big because of the FOMO from existing fans, not new ones. It wasn’t going to kickstart a new wave of growth. Still, that didn’t make it any less entertaining. I saw a fan show off his Rainbow Dash decaled Ford Mustang. The Chipotle across the street seemed to have at least one person in a pony T-shirt at all hours of the day.

  At the vendor hall, I talked to one of the few Asians I saw, buying the literal last Kirin shirt they had available in black. It was a little short, and I heavily suspected it’d be too small for me after going through the wash, but there wasn’t any choice. They had run out of merch on the first day.

  

  They told me to come to their panel about the South East Asia brony scene, which sounded interesting, but I completely forgot to. Sorry!

  I did talk to another Asian brony in the games room, while watching a Them’s Fightin’ Herds tournament match. While people in the background were yelling about a game they obviously didn’t understand, he talked to me about the Chinese brony scene. A lot of the Chinese fanbase is younger (like 10 years old), but there are still adult fans and they pulled in around 1200 attendees at their most recent convention. He was from Shenzhen, and was part of a group of 30 that made it past the visa issues to the US. He was spending his time buying merch to bring back for everyone who couldn’t come. I suspected he was from a rich family, since he was only 18, but he lived in Shenzhen, had gotten a sponsor tier badge ($250+, guarantees shorter lines and other perks), was carrying around a really high quality Starlight plushie (which he said cost $530 to commission), and he offered me $50 if I’d stand in line to get Lauren Faust’s autograph. Given that said line was 4 hours long because everyone wanted to tell Lauren Faust their life story, I declined.

  The convention center was getting used by other events too. That year, it was co-located with the Rubik’s Cube National Championships. I saw some cubers walking through the same halls, and can only imagine what they were thinking when passing by the occasional fursuit.

  

  I had privately made the Elements of Harmony ↔ Infinity Stones connection before BronyCon, and was gratified to see someone else had done the same.

  In the evening, I went to one of the nighttime 18+ panels about the history of pony conventions, got a pony-themed cocktail, and listened to a bunch of people talk about how they got here, and all the horror stories of past cons. The one with the fire, the one where the organizers ran out of money and were completely gone while shit was hitting the fan, and so on. You know what they say, shared trauma brings everyone together. The cocktail was vodka, apple juice, triple sec, cranberry syrup, and small pieces of strawberry, and it was way too sweet. Shouldn’t have been surprised.

  A number of brony conventions run concerts hosting fan musicians, who play their music to a crowd. Sometimes they play remixes of songs from the show, but more frequently they play original music that wouldn’t be out of place at an indie EDM concert. Because pony is what brought people together, but it’s not the reason people stay.

  Usually, these events suck. Hard. Not by any fault of the music acts, it’s just that they’re filled with a bunch of awkward people, in a room that’s far too big, where no one’s brave enough to dance. The one at the final BronyCon? The Last Ride? It was lit. Okay, well, not by the standard of a real nightclub or music concert, but it was packed at 2 AM. Someone was waving an enormous Equestria flag. A person starts an impromptu conga line, yelling “JOIN US! IT’S OKAY! We’re all retarded.” Someone else signals everyone to clear a circle, and when we do, he starts showing off his sick breakdance moves. Between the insane circumstances, and the casual self-directed use of “retard” in the year 2019, I’m not sure I could give a better summary of the state of the fandom.

  Despite everything, the rise and the fall in popularity, the content creators who moved on and see their pony work as an old shame, the outing of community figureheads as racists or sexual abusers, I could see the magic that originally drew me in. Enough that although I wouldn’t call myself a brony, if someone did, I wouldn’t feel the need to correct them. It’d be too hard to explain.

  “Sometimes We’re Self-Conscious About Our Dumb Sport”

  I don’t have any plans to go to another con. None of them are going to top BronyCon anyways.

  I probably won’t watch Generation 5. I’m not sure how much longer I’ll stay in the fandom. We called it a ride because the MLP fandom loves horse puns, but it’s a fitting name. The point of rides is to stop. It’s okay for them to stop. Or, at least it’s okay for me to get off it if I want to. I think that’ll happen one day. For now, I still get something out of listening to new songs from the pony music community, and the few fan artists I follow.

  I have spent an irrational amount of time and money on My Little Pony paraphernalia, and there is only one thing I regret. Once, I was wandering vendor tables, and saw someone was selling Arduineighs - a pony-themed Arduino board. They were themed after Twilight Sparkle’s Secret Shipfic Folder, a PG rated board game making fun of tropes in poorly written romantic fanfiction.

  

  I bought one for $5, and regretted it almost immediately. I’ve never soldered in my life, what am I going to do with an Arduino board you have to assemble yourself? The coolness of the plain metallic look wore off pretty quickly.

  Everything else? No regrets. I don’t regret the plushies, the prints, the Daybreaker shadow box, the badge of Flurry Heart labeled “DEMON BABY” that I bought from a teenager who offered 10% off if I subscribed to Pewdiepie on YouTube. (I didn’t.) I don’t regret the time spent carefully trawling the MLP wiki for the perfect Lyra thumbnail. I certainly don’t regret spending my senior year writing terrible Java applets to playtest MLP Dominion cards - eventually, I parlayed that into a SWE internship interview.

  I come back to this question of what MLP all meant to me, and I’ve decided that’s just the wrong question to ask. Pony fandom was a sizable part of my life for 10 years. It wasn’t separate from reality, it was an exaggeration of reality, at its worst and at its best. The drama, the weirdos, the cool content creators, the jerk content creators, the toxic arguments, the jaw-dropping fan work, all of it. My thoughts on it don’t fit in a nice tidy box, because life doesn’t fit in a nice tidy box.

  In the late 2000s, there was a Youtuber named ChaosAngel. Every few days, he uploaded a metal remix of a Touhou song, usually from the most recent Comiket. The songs ranged from prog rock to black metal, and his work was pretty influential on my music tastes. His channel was hit with copyright strikes from various doujin circles, and it was eventually taken down without appeal.

  In response, he created a new channel, reuploading every song except the ones with copyright strikes. To announce the channel, he uploaded a “song 0” - a remix of Mima’s theme Reincarnation.

  What were the reasons of the evil spirit Mima for her wish to destroy humankind, and why she goes from light to darkness was never known. Mima always denied her death, and she claims to be just a wandering spirit instead of a ghost, while she found the way to revive herself.

  So let us continue what she couldn’t accomplish, by reviving ourselves.

  Thanks for watching.

  “No one shall be able to drive us from the wonderland that ZUN created for us.”

  Sure, Hasbro’s the ones who created Wonderland, but it’s the fans that make it a place worth visiting. Together we’ve created bizarre cathedrals of horses and magic, stylized with cringeworthy stained glass windows, and it needs someone to bear witness. As long as people maintain it, I’ll check in from time to time, even if it’s only out of nostalgia. Even if it’s all fake compared to the real world. The cathedrals are real to us.

  Maybe you have a brony fandom of your own. Something you’re a fan of, that you can’t explain. I think it’s more common than people think. I got this feeling when I watched a documentary about MMA.

  There’s no magic out there when it comes to the contest over resources that governs our lives and the lives of our children. […] The magic that we wish we saw everywhere else was in the cage, because it was conjured by people who were just too fucked up to make it in the world outside. In a world where everything seemed to get slightly better then slightly worse at a more constant rate, at least there was one place where unthinkable things actually happened. At least if you put two weird people with incredible abilities in front of each other, their combined experiences and opposing martial abilities would create a beautiful, maddening story.

  “Fighting in the Age of Loneliness”, 1:24:20

  Dana White is an MMA fan, and there’s a very real insecurity in MMA fans. Whether you’re watching a shitty UK stream on your friend’s off-brand tablet, or you’re the president of the world’s biggest promotion, sometimes you may feel a little self-conscious about our dumb sport.

  Everyone who loves this sport has had a moment where they watch an event with a friend, a family member, or a romantic partner. It was probably during the pay-per-view golden age of the late 2000s, where the person you hoped would become just as obsessed as you [in MMA] saw men in awful tattoos wearing shorts that said Condom Depot in huge letters on them, [pushing] one another against a fence for 15 minutes. […] Your target audience found it too weird, too boring, too terrifying, or all three. And the cultural image of MMA fighters is a bunch of enormous, angry men, with weird tattoos, who always seem to be yelling. And the image of fans is a sea of guys with big guts and chiseled arms, wearing Affliction shirts and getting wasted before insulting random passersby. But does it matter? We love this sport! […] We love it, and that’s all that should matter. And who gives a shit if we don’t have hundreds of millions of people watching with us every time. And why do we care if people think we’re weird or fucked up for watching it? We know what our sport is, and we know who we are! From the most stereotypical ones, to the grandmas and grad students who get just as excited as the Affliction shirt guys.

  “Fighting in the Age of Loneliness”, 1:27:30

  Maybe you’re trying to justify why it’s worth cheering for the Seattle Mariners, possibly the most tragic team in baseball.

  The Mariners aren’t special on account of their lack of success, it’s just that success is entirely irrelevant. We’ve entered another realm here, one that’s far larger and doesn’t operate on the dead currency of winning and losing. Unless you let those limits go, you’re an astronaut who brought your wallet. The Seattle Mariners are not competitors. They’re protagonists. […]

  With the benefit of hindsight, we know that on-field contention wasn’t in the cards for the Seattle Mariners, and has never been to this day. That was never happening. The only fight left was for happiness.

  “The History of the Seattle Mariners”, 2:26:50 and 2:44:50

  I’ve long given up on getting anyone else into this fandom, this thing where observing it is hard and pretending it doesn’t exist is easy. That’s fine. People don’t have to see it or understand it. They just have to accept it. If they can’t, they can go to hell.

  Over the years, I’ve come around to the “relentless march of the normal” theory. Any culture that gets big - any culture at all - cannot grow and stay big without losing its unique charm, because that unique charm can’t survive in a culture that tries to appeal to everyone. So that culture has a choice: accept it’ll change, or find a way to grow slowly enough to protect itself. We all wish we could get both the charm and the popularity. I’ve yet to see anything succeed. All those early brony thinkpieces examining why people don’t watch the show were a waste of time. If they had gotten what they wanted, we wouldn’t have recognized what came out. Sometimes it’s worth fighting for legitimacy, but bronies lost that fight ages ago. Trying to argue “we’re better than the furries” is not going to save you from the wider world. Make fun of yourself by saying you’re trash, and move on.

  The kids, when they like something on the Internet, they call themselves the trash of the thing.

  - Lin Manuel Miranda

  BronyCon was lucky that their final run was the year before COVID-19 swept through and cancelled all pony conventions for 2020. It was really unclear how the brony ecosystem would fare. Con treasuries took a big hit, and artists who rely on con sales as supplementary income were in trouble. In response, some organizers set up PonyFest, a free online brony convention with remote panels and a vendor hall hosted in Discord. Cool idea, a lot of work required. I bought some cute MLP-themed boba stickers in support.

  Once again, they held a music concert, but this one was virtual in Pony Town, a social MMORPG where people can make pony OCs to chat and hang out. Looking at that concert of fan music, streamed live into an actively developed fan game that hit 15k concurrent users this year, I realized this fandom actually might never die. Where are these people even coming from?

  

  {Click for full size)

  Sure, Friendship is Magic is over, but the fans left have transcended the show. They’re doing what makes them happy. The recursive engine of fan art of fan art is going to keep spinning. The Generation 5 movie kick-started things a little bit, and maybe the TV series will be good. Maybe it won’t. It doesn’t matter either way. Friendship will always be magic. Once you’re in, a part of you never leaves.

  Letting go is harder than it seems
  I still wonder if you remember me

  It’s been so long, you’ve gone so far
  Through space and time between us in my heart
  I’ve lost my way, now I can’t see
  I still wonder if you remember me

  Thanks to Alex Mun and Diyang Tang for reviewing earlier drafts.

  Comments