gammagurke

Karma: 45

Simply reverse engineering gpt2-small (Layer 0, Part 1: Attention)

gammagurke22 Jul 2025 14:59 UTC

21 points

0 comments27 min readLW link

gammagurke 14 Aug 2022 22:21 UTC
3 points
0
on: Dwarves & D.Sci: Data Fortress
My selection:
4 miners, 2 smiths, 1 woodcutter, 1 farmer, 1 brewer, 2 warriors, 2 crafters
Considerations:
Miners:
Miners are clearly the best at producing value, but, as others have pointed out, some forts seem to inexplicably die if they have a lot of miners. I’m too lazy to do some deeper analysis of whether some combinations of metals make this more or less likely or something, so I’m just gonna cap our miners at 4.
Food:
2 farmers + 2 warriors or 1 farmer + 1 brewer + 2 warriors seems to be (exactly) enough for such a small expedition in all cases, and a single brewer seems to be good for value
Value:
As many miners as possible, at least one crafter and smith.
Fuel:
We should be good with one woodcutter, in light forest at scarce coal.
for the last 2 dwarves, I’m going with one smith and one crafter. I’m not sure the 3rd smith does anything (one smith can make bronze gear, the other iron gear, while the silver goes to the crafters who maybe also get stuff from the woodcutter).

gammagurke 18 Jul 2022 1:14 UTC
3 points
0
on: Ars D&D.sci: Mysteries of Mana
I’m recommending Fireball+Levee+Vambrace.
It seems like it’s a turn-based format where wizards don’t have control over which of 2 spells of a type(offensive or defensive) they’re using, if they have more than 1 prepared. So with knowledge of what our opponent has prepared, it would be best if we could only prepare the best 2 spells (1 of each type) to counter his, and not have to randomly switch to the 2nd best some of the time. With that, we need to find out what is the least bad 2nd-best option for either type.

In defensive spells, there is nothing that counters both our opponent’s offensive spells. Since it’s presumably random which one they’re using, we can’t really pick anything that gets countered by either of the opponent’s offensive spells, leaving two options. Levee, which can counter some of the water damage on the missiles, and vambrace, which can use the high water mana itself. Not sure which one to prefer here.

In offensive spells, there are the spells that counter our opponent’s solar shield, one of them (Fireball) with a reasonably higher mana total than the other, and spells that use the high water mana, one of them matching the shield on light, the other having higher mana with earth. Out of these, I am reasonably confident that Fireball gives the best dpr (or whatever the relevant metric turns out to be).
Taking all that into account, it seems like we get the smallest loss in picking the 2 defensive spells. If we can recommend our master to only prepare 2 spells or have a 100%-0% split instead of a 50%-50% between 2 spells of a type, I’d recommend to cut the Levee and only rely on the Vambrace for defense.

gammagurke 13 Jun 2022 0:53 UTC
3 points
0
on: D&D.Sci June 2022: A Goddess Tried To Reincarnate Me Into A Fantasy World, But I Insisted On Using Data Science To Select An Optimal Combination Of Cheat Skills!
Notes:

-Traits:
Most nerds are otakus and vice versa. Nerds are great at time travel, otakus are great at everything except having a dark side. Postulating some underlying stats (and tentatively drawing the causal arrow stats → traits, because of the uneven and correlated distribution of traits), it would seem that anyone with enough points in different stats, probably including something we can call intelligence, has a high chance of being an otaku, while only intelligence contributes to the chance of being a nerd. A dark side just switches your underlying stats to something much-better-distributed than normal, which is why it’s often a good thing for non-otakus to take and bad for otakus, who already have good stats and have better uses for the skill slot.
Most office workers aren’t Hikkikomoris and vice versa. It would make sense for this to be linked to some sort of sociability stat, though curiously sociopaths aren’t part of this interaction. Office worker special power: The Grind (reliably gain power through XP gain), Hikkikomori special power: Death from isolation (high chance to get trapped in one’s own mind palace and instantly die (or something))
-Skills (most of this agrees with aphyer’s analysis):
If you want to stay alive, you’ll want to specialise in some way to defend. There are 3 ways of doing this: Shielding yourself from damage (works best at medium hp and tankiness), Lots of regeneration (works best at high hp and tankiness), or just Dodging everything that comes at you (works best at low hp and tankiness).
You’ll also want some way of gaining power and doing damage, otherwise you’ll never work your way up to defeating the Demon King. If your stats are bad, the best thing you can do is just swap them out for a powerful and mysterious dark side. Enough Intelligence and the power of time travel might work for you. With hard work and diligence, you can try to amass enough XP to get there all by yourself. If you don’t mind the opinions of others or if you’re into that kind of thing, you can turn yourself into a big and tanky dinosaur (which then means regeneration is the best defense to specialise in). It’s possible you can scrape by just getting more lucky than others would, but usually there are better options. Similarly, a Mind Palace can be sufficient for some people, but usually there are better options, especially for Hikkikomoris.
Anyone good enough or righteous enough or something to feel compelled to pick some combination of Enlightenment, Splendour or Luck should follow that impulse, everyone else should stay clear.
My choice:
Temporal Distortion, Anomalous Agility
Unfortunately, we lack any information about ourselves other than the two traits. We don’t know whether enlightenment/splendour is a newcomb-like or smoker’s lesion-like decision or whether we’ve lost our chance at goodly and righteously declaring that of course we’ll be taking enlightenment/splendour by just taking the time to look at the data.
As for the decision between Agility and Barriers, and possibly even between Temporal and XP or Mind palace, it seems to be down to what our underlying stats are, which we know nothing about, so we just have to go by the average from the data.

gammagurke 16 May 2022 2:05 UTC
5 points
0
on: D&D.Sci Divination: Nine Black Doves
My budget:
30000 on grain shipments. Red moons, backwards geese and new constellations all aren’t good signs for food production and with tons of doves and this being the highest-ev thing to do in a vacuum anyway it seems good enough, even though Italy’s just had a famine.
10000 on the protection of Asclepius for Grecia. Stopping the spread >> preventing the outbreak
20000 on fire protection in Grecia and Hispania. Italia’s new constellations should help with fires over there.
Defenses against pillaging are nice, but 50% doesn’t seem enough to get the ev there higher than any of the others unless there are some major indications.

gammagurke 5 Apr 2022 5:04 UTC
6 points
0
on: Duels & D.Sci March 2022: Evaluation and Ruleset
This was the most fun I’ve had analysing data and writing code probably ever. Unfortunately I missed the previous editions, but I’m looking forward to the next one. If I had played the previous ones, I might have steered further away from trying to explain effects by complex interactions that are common in this sort of card game in favour of simpler interactions that are more likely to be put into this sort of ruleset (for example things having one combat-relevant stat instead of two).
Because this was such a blast to play around with, I don’t really have any specific things I would change.
The way the decks in the dataset were generated, along with the datasets size, made it easy to check how some specific decktype did in general, but hard to check how it did against some other specific decktype, which seems like the perfect middleground.

I really appreciate the work that was put into the theme, letting me roleplay as Kaiba playing children’s card games and trying to win using the power of arcane computer analysis tools.

gammagurke 2 Apr 2022 13:31 UTC
3 points
0
in reply to: gammagurke’s comment on: Duels & D.Sci March 2022: It’s time for D-d-d-d-d-d-d-d-d-d-d-d-d-d-data!
My PvE list:
5 Lotus
3 Angel
2 Emperor
2 Pirate

gammagurke 31 Mar 2022 14:09 UTC
10 points
0
on: Duels & D.Sci March 2022: It’s time for D-d-d-d-d-d-d-d-d-d-d-d-d-d-data!
A couple of people have already pointed out similarities to mtg/hearthstone so I thought it might be a good way to think explicitly about the rules/specific cards (some reasoning behind it, but these aren’t predictions, just a way to solidify my intuitions):
- Hearthstone-like mana system since no lands (Lotus works like innervate?)
- Creatures regenerate like in mtg, otherwise bigger creatures could just be traded off with 2 or 3 smaller ones and it wouldn’t be so big a deal to lotus one of them out
- Starting life total is balanced in a way that good aggro decks and good control (that is lotus ramp) decks have approximately even odds against each other
- Archetypes:
- Good guy tribal: Battalion is some sort of lord for guards , knights, vigilantes and copies of itself (making it one of the inexpensive cards that doesn’t do too badly with lotuses, because it makes sense to ramp out things that lord each other)
- (intuition stats: guard: 1 mana ¹⁄₄, knight: 3 mana ³⁄₃, vigilante: 3 mana ⁴⁄₂, battalion: 2 mana ²⁄₂ and buff all others by +1/+1. something like this would explain why battalion/guard > battalion/knight > battalion/vigilante)
- Lotus ramp: Lotus out a big thing, big thing clears the board and wins the game. Dragons seem worse at this than Angels, emperors are slower than both
- (intuition stats: Angel: 5 mana ⁴⁄₆, Dragon: 6 mana ⁵⁄₆, Emperor: 7 mana ⁷⁄₇, Lotus: gain 2-3 mana this turn only)
- Evil equip: Sword of shadows is presumably some kind of equipment you put on Pirates, Hooligans or Minotaurs to give them stats and it can be reused when the creature has died (or maybe it’s an anthem sort of thing, but that would go against its name). Pirates have a high winrate in general, and work well with both Swords and Angels, so I would expect Pirate to be overstatted for its cost and slightly defensive-leaning. Minotaurs don’t have a great winrate and are not great with Swords, but are the other “low-cost” thing that works reasonably well with lotuses, and game nr. 185896 gives a hint of what minotaur does (mainly, having a minotaur unopposed gives you a short clock, 2-3 turns maybe?).
- (intuition stats: Pirate: 1 mana ²⁄₃, Hooligan: 2 mana ³⁄₂, Minotaur: 4 mana ⁷⁄₁, Sword: 0 or 1 mana +2/+1 equip for 0)
- presumably, Swords technically also work with Dragons and Emperors, but those are big enough that it doesn’t make much of a difference, same with Angels and battalions.
Strategy:
- diverse decks do well because it’s important to curve out. Yugi’s gonna be hitting the perfect curve, which means we have to defend early and outvalue late. With my completely made-up stats Yugi’s curve would look something like: Pirate/Sword into Hooligan into Lotus Angel into Minotaur into Knight/Battalion into Dragon into Emperor.
- one way to do the “outvalue late” part of the plan is going with angels and at least 1 emperor. Which means we’ll need lotuses. For the “defend early” part, this strongly depends on how exactly lotuses work. It might be that if we have, say, 5 Lotuses and 5 Angels, we can double Lotus into Angel turn 1 which already stops the aggro cold. But if you can only play one Lotus a turn or something, we’re gonna need some smaller creatures.
- especially for smaller creatures, exact stats and things like whether double-blocking is a thing are hugely important. The first idea would be to just take a bunch of pirates, but then a pirate with a sword probably just eats those and it’s not clear whether the saved life total makes up for the lost value.
- another way is stacking swords on things and hoping that that suffices to at least trade with Yugi’s big creatures in the later turns or, depending on the cost of swords, to out-aggro him (seems unlikely against a perfect curve)
- starting Hand size is really hard to speculate on, but we can try anyway: Main observation is that lotuses don’t work with aggro cards. If the starting hand size was something like 7, there would be no reason for that. It seems like, unless they have a sword/battalion or some good high-cost cards, aggro decks quickly run out of steam. Aggro can stay aggressive for about as many turns as it can double-spell to keep board control and it would seem that that limit is reached pretty early on, if a lotus decreasing that number by 1 is more relevant than a big boost in turn 1-2 board control. So my guess would be something like 2-3 cards in opening hand, with one drawn every turn. Yugi loves a small hand size, so this is something we’ll need to keep in mind.

gammagurke

Sim­ply re­verse en­g­ineer­ing gpt2-small (Layer 0, Part 1: At­ten­tion)

Simply reverse engineering gpt2-small (Layer 0, Part 1: Attention)