Minding Our Way

Nate Soares

I faked confidence, and it soon became real. I found that my social limitations had been largely psychological, and that the majority of my classmates were more than willing to be friends.

Yet even these simple ideas were absent in the actual system. Corruption and inefficiency ran rampant. Worse, my peers didn't seem particularly perturbed: they took the system as a given, and merely memorized the machinery for long enough to pass a test. Even the grownups were apathetic: they dickered over who should have power within the system, never suggesting we should alter the system itself.

My childhood illusions fell to pieces. I realized that nothing was meticulously managed, that the smartest people weren't in control, making sure that everything was optimal. All the world problems, the sicknesses and the injustices and the death: these weren't necessary evils, they were a product of neglect. The most important system of all was poorly coordinated, bloated, and outdated — and nobody seemed to care.

When grownups say you can't do something, what they mean is that they can't do it.

"So you want to be an idealist?", the book asked. "Very well, but what is your ideal?"

Nobody found flaws in my logic. Nobody challenged my bold claims. Instead, they simply failed to understand. They got stuck three or four points before the interesting points, and could go no further. I learned that most people don't understand basic economics or game theory. Many others were entrenched in bluegreensmanship and reflexively treated my suggestions as attacks. Aspiring politicians balked at the claim that Democracy, while perhaps an important step in our cultural evolution, can't possibly be the end of the line. Still others insisted that it's useless to discuss ideals, because they can never be achieved.

I learned a long time ago that most people are content to accept the way things are. Everyone wants the world to change, but most are cowed by the fact that they can't change it themselves.

if the chance that one person can save the world is one in a million, then there had better be a million people trying.

When "Make Good Music" is an instrumental goal, she schedules practice time on a sitar and grinds out the hours. But she doesn't really like it, so she cuts corners whenever akrasia comes knocking. She lacks inspiration and spends her spare hours dreaming of stardom. Her songs are shallow and trite.

Ideally, we would be just as motivated to carry out instrumental goals as we are to carry out terminal goals. In reality, this is not the case. As a human, your motivation system does discriminate between the goals that you feel obligated to achieve and the goals that you pursue as ends unto themselves. As such, it is sometimes in your best interest to modify your terminal goals.

Goal-hacking in the name of consistency isn't really a Dark Side power. This power is only Dark when you use it like the musician in our example, when you adopt terminal goals for instrumental reasons. This form of goal hacking is less common, but can be very effective.

You don't get good at programming by sitting down and forcing yourself to practice for three hours a day. I mean, I suppose you could get good at programming that way. But it's much easier to get good at programming by loving programming, by being the type of person who spends every spare hour tinkering on a project. Because then it doesn't feel like practice, it feels like fun. This is the power that you can harness, if you're willing to tamper with your terminal goals for instrumental reasons.

My Willpower Does Not Deplete

Ego depletion is a funny thing. If you don't believe in ego depletion, you suffer less ego depletion. This does not eliminate ego depletion. Knowing this, I have a compartment in which My Willpower Does Not Deplete. I go there often, when I'm studying. It's easy, I think, for one to begin to feel tired, and say "oh, this must be ego depletion, I can't work anymore." Whenever my brain tries to go there, I wheel this bad boy out of his cage. "Nope", I respond, "My Willpower Does Not Deplete".

I have to actually be tired out, in a way that doesn't trigger the My Willpower Does Not Deplete safeguards. This doesn't let me keep going forever, but it prevents a lot of false alarms.

Knowing this, you can easily activate the placebo effect manually. Feeling sick? Here's a freebie: drink more water. It will make you feel better. No? It's just a placebo, you say? Doesn't matter. Tell yourself that water makes it better. Put that in a nice little compartment, save it for later. It doesn't matter that you know what you're doing: your brain is easily fooled.

Want to be more productive, be healthier, and exercise more effectively? Try using Anything is a Placebo! Pick something trivial and non-harmful and tell yourself that it helps you perform better. Put the belief in a compartment in which you act as if you believe the thing. Cognitive dissonance doesn't matter! Your brain is great at ignoring cognitive dissonance. You can "know" you're wrong in the global case, while "believing" you're right locally.

I use Intentional Compartmentalization to "locally believe" things that I don't "globally believe", in cases where the local belief makes me more productive. In this case, the beliefs in the compartments are things that I tell myself. They're like mantras that I repeat in my head, at the System 2 level. System 1 is fragmented and compartmentalized, and happily obliges.

I have invoked Willful Inconsistency on only two occasions, and they were similar in nature. Only one instance of Willful Inconsistency is currently active, and it works like this: I have completely and totally convinced my intuitions that unfriendly AI is a problem. A big problem. System 1 operates under the assumption that UFAI will come to pass in the next twenty years with very high probability. You can imagine how this is somewhat motivating.

Explicitly, I believe UFAI is one possibility among many and that the timeframe should be measured in decades rather than years. I've concluded that it is my most pressing concern, but I don't actually believe we have a hard 15 year countdown. That said, it's hard to understate how useful it is to have a gut-level feeling that there's a short, hard timeline. This "knowledge" pushes the monkey brain to go all out, no holds barred. In other words, this is the method by which I convince myself to actually specialize. This is how I convince myself to deploy every available resource, to attack the problem as if the stakes were incredibly high. Because the stakes are incredibly high, and I do need to deploy every available resource, even if we don't have a hard 15 year timer.

Your brain was optimized with no concern for truth, and optimal performance may require self deception.

I remind the uncomfortable that instrumental rationality is not about being the most consistent or the most correct, it's about winning.

From another perspective, you could say that I deregulated a black market on distractions: By lifting the mental ban on entertainment, I was able to price it accurately and weigh the tradeoffs. If there is a new book I want to read, the answer is not an outright and unenforcible "No". Rather, it's "can we afford to be underproductive for the next few days?". And when the answer is negative, it's significantly easier for me to postpone gratification than to resist the temptation entirely. The end result is that I have much more control over when I indulge in escapism.

A number of my teachers took it upon themselves to press upon me just how much I could do if I actually applied myself. I didn't bother correcting them. If they weren't going to invent a grade higher than 'A', why should I waste my efforts in the classroom? I had better things to do.

This experience in school had two important repercussions. First, it taught me to seek out the gap between the intended rules and the actual rules. I developed a knack for it, and this has served me well in many walks of life. Noticing the space between what you meant and what you said is a fundamental skill for programmers. Math is a tool designed to narrow such gaps. Logical incompleteness theorems are statements about the gap between what logic can say and what mathematicians want to say.

school helped me make explicit the virtue of putting in the minimum possible effort.

This leads us to my second trick for avoiding akrasia: I am not Trying Really Hard. People who are Trying Really Hard give themselves rewards for progress or punishments for failure. They incentivize the behavior that they want to have. They keep on deciding to continue doing what they're doing, and they engage in valiant battle against akrasia. I don't do any of that. Instead, I simply Move Towards the Goal.

I don't will myself to study. It is not a chore, it is not something I force myself to do. That's not to say I enjoy studying, per se: it's hard work, and the reward structure is pathetic compared to programming.

I don't Try Really Hard. I simply Move Towards the Goal.

This is, internally, an immutable fact, made so both by habit and by crude Pavlovian training. None of this is explicit, mind you, it's just the nature of goals. I can change the goal and I can drop the goal, but I can't hold the goal and not pursue it.

I never decided to study really hard. You can "decide" not to watch the next episode of that TV show only to sternly berate yourself three episodes later. My decision to study hard was made on a lower level, it's been internalized.

Acting on goals is the thing that System 1 does regardless of what System 2 "decides".

System 2 controls things by picking the goals. It was a long and arduous process to internalize my most recent set of goals, the ones that have driven me to study hard and become a research associate and so on. It took a few months and a bit of mindhacking, and that's a story for another day. But once the goal was chosen, marching towards it was out of my hands.

System 2 isn't in control of whether I move towards the goal. Instead, it spends its time doing something it's very good at: finding the most efficient path. Minimizing effort.

When I'm Moving Towards the Goal, I don't worry about whether things will be done. I've outsourced that concern to habit. Instead, mental effort is spent looking for the shortest path, the easiest route. Difficult paths do not require additional willpower, because the internal narrative is not one of expending effort. If anything, a difficult path is worth extra points, because it means I'm pursuing admirable goals. Internally, I'm not Struggling Against Akrasia. I'm Finding an Efficient Route.

studying math at high speed for five months was hard. However, I have built myself a headspace where hardness is not an obstacle to overcome but a feature of the terrain.

System 2 doesn't have to spend effort convincing System 1 to move forward, because System 1 is going to move forward come hell or high water. Thus, System 2 spends its time making sure that the march is as easy as possible.

It seemed clear that if the goal had been 60k, many of the same people would have eeked out a victory with similar margins and the same narrative of butting against their limits. The natural conclusion was that I can't trust myself to feel out my own limits.

These days, I occasionally throw wrenches into my study plans when I think I'm growing complacent.

Often, this fails spectacularly. Sometimes, I am at or near my limits, and skipping an intro logic textbook to dive straight into Model Theory is a really bad idea. Other times, I find out that I actually was just hovering around an anchor point, seduced by a narrative of linear improvement.

There are no limits. There are plateaus, but you must not stay there, you must go beyond them. If it kills you, it kills you. A man must constantly exceed his level. - Bruce Lee

I'm not wondering whether I will be able to convince myself to study each day. Instead, I'm gauging whether I'm reading the most effective material. I'm noticing that it won't be enough for me to just learn the material, I also have to signal that I've learned the material (and that I should start doing book reviews). I'm monitoring to see when I've grown complacent and looking for ways to keep me on my toes. This is process is doubly useful: It helps me sidestep akrasia and it also helps me become more effective.

Productivity is a habit of mine. As I mentioned in the previous post, I've been following a similar schedule for years: two days doing social things, five days doing something constructive. Before I turned my efforts towards FAI research, this mainly consistent of programming, writing, and self-education.

I deplore fun. Ok, not really. However, I do have a strong aversion to activities that I find unproductive.

"KILLING TIME!" roared the dog—so furiously that his alarm went off. "It's bad enough wasting time without killing it." And he shuddered at the thought.

Before long, I began to view escapism as a guilty pleasure: fun and addictive, but unsatisfying. Things like hiking and going to parties became almost a chore: I superficially enjoyed them, sure, but I yearned to be elsewhere, doing something permanent. Even reading fiction took on a pang of guilt. I valued things that moved me forward, that honed my skills or moved me closer to my terminal goals. I wanted to be building things, improving things.

These days, I aim to spend about two evenings a week (one on weekdays, one on weekends) doing something that's traditionally fun. I spend the rest of my time doing things that sate my neverending desire to march towards my goals. It's interesting

These days, I aim to spend about two evenings a week (one on weekdays, one on weekends) doing something that's traditionally fun. I spend the rest of my time doing things that sate my neverending desire to march towards my goals.

When I was quite young, one of the guests at our house refused to eat processed food. I remember that I offered her some fritos and she refused. I was fairly astonished, and young enough to be socially inept. I asked, incredulous, how someone could not like fritos. To my surprise, she didn't brush me off or feed me banal lines about how different people have different tastes. She gave me the answer of someone who had recently stopped liking fritos through an act of will. Her answer went something like this: "Just start noticing how greasy they are, and how the grease gets all over your fingers and coats the inside of the bag. Notice that you don't want to eat things soaked in that much grease. Become repulsed by it, and then you won't like them either."

This woman's technique stuck with me. She picked out a very specific property of a thing she wanted to stop enjoying and convinced herself that it repulsed her.

If I were trying to start hating fun (and I remind you that I'm not trying, because I already do, and that you shouldn't try, because it's no fun) then this is the route I would recommend: Recognize those little discomforts that underlie your escapism, latch on to them, and blow them completely out of proportion.

When skiing, partying, or generally having a good time, try remembering that this is exactly the type of thing people should have an opportunity to do after we stop everyone from dying.

When doing something transient like watching TV or playing video games, reflect upon how it's not building any skills that are going to make the world a better place, nor really having a lasting impact on the world.

Notice that if the world is to be saved then it really does need to be you who saves it, because everybody else is busy skiing, partying, reading fantasy, or dying in third world countries.

If you can structure your life such that productive things are the things that you do by default, the things that you do in your free time when you have nothing else on your plate, then you will be in good shape. When "do something that forwards your goals" is the fallback plan then it becomes much easier to scale your efforts up.

I was careful not to let schooling get in the way of my education.

I'm writing this because I was able to learn a lot very quickly. In the space of eighteen weeks I went from being a professional programmer to helping Benja discover Fallenstein's Monster, a result concerning tiling agents (in the field of mathematical logic).

I estimate my total study time to be slightly less than 500 hours. I achieved high retention and validated my understanding against other participants of the December workshop. I did this without seriously impacting my job or my social life.

the normal weekday I studied for an hour and a half in the morning, a half hour during lunch, and three to four hours in the evening. On the average weekend day I studied 8 to 12 hours on and off throughout the day.

While my studying did not affect my schedule much, it definitely affected my pacing. Don't get me wrong; this sprint was not easy. I suspended many other projects and drastically increased my intensity and my pace. I spent roughly the same amount of time per day studying as I used to spend on side projects, but there is a vast difference between spending three hours casually tinkering on open source code and spending three hours learning logic as fast as possible.

had a number of techniques for handling difficult exercises. First, I'd put them aside and come back to them later. If that failed, I'd restate the problem (and all relevant material) in my own words. If this didn't work, it at least helped me identify the point of confusion, which set me up for a question math.stackexchange.com.

I had a number of techniques for handling difficult exercises. First, I'd put them aside and come back to them later. If that failed, I'd restate the problem (and all relevant material) in my own words. If this didn't work, it at least helped me identify the point of confusion, which set me up for a question math.stackexchange.com.

Upon finishing a book, I would immediately start the next one. Concurrently, I would start writing a review of the book I'd finished. I generally wrote the first draft of my book reviews on the Sunday after completing the book, alternating between studying the new and summarizing the old. On subsequent weekdays I'd edit in the morning and study in the evening until I was ready to post my review. It's worth noting that summarizing content, especially the research papers, went a long way towards solidifying my knowledge and ensuring that I wasn't glossing over anything.

I live with two close friends. This meant that social contact was never out of reach. Even when spending an entire day sequestered in my room pouring over a textbook I was able to maintain a small amount of social interaction. If ever I had a spare hour and a thirst for company, I found it readily available. My primary partner was, up until early 2014, going to school full time while holding down a full time job. Thus, her schedule was more restrictive than my own and we had been working around it for some time. Our relationship was not further constrained by my efforts. My core friend groups knew and respected what I was doing. I was more tense and exhausted than usual, but I had warned my friends to expect this and no friendships suffered as a result.

Most important of all, I had friends I could call on when I needed a mental health day. I could rely on them to find time where we could just sit around, play with LEGO bricks, and shoot the breeze. This went a long way towards keeping me sane.

I tentatively believe that someone could sustain my pace for significantly longer than I did, so long as they were willing to live with the strain. I don't plan to test this myself: I'll be slowing down both to improve performance at work and to reduce my general stress levels. Five months of fervent studying is no walk in the park.

There is no magic to it. If you study the right material, do the exercises, and write what you've learned in your own words, then you can indeed learn MIRI-relevant math in a reasonable amount of time.

Learning fast does not need to dominate your life. There can be time for social activities and even significant side projects. You will have to work really hard, but that work does not have to consume your life.

If you're going to do something like this, let people know what you're doing. This is much easier if you have people you can turn to for support who don't mind you being extra snappy, people who can drag you away for a day every week or two. Also, stating your goals publicly helps to stop you from giving up.

The thing I miss most about college is tight feedback loops while learning. When autodidacting, the feedback loop can be long.

There are a number of little misconceptions you have when you're familiar with something but you've never applied it, and I found myself having to clean those out just to understand what Model Theory was trying to say to me.

The third pass was necessary to understand the greater theory. I've never been particularly good at memorizing things, and it's not sufficient for me to believe and memorize a theorem. If it's going to stick, I have to understand why it's important. I have to understand why this theorem in particular is being stated, rather than another. I have to understand the problem that's being solved. A third pass was necessary to figure out the context in which the text made sense.

After a third pass of any given chapter, the next chapter didn't seem quite so random. When the upcoming content started feeling like a natural progression instead of a random walk, I knew I was making progress.

but I did not anticipate requiring three passes. Mostly, I didn't anticipate gaining as much as I did from a re-read; I would have guessed that something opaque on the first pass would remain opaque on a second pass.

In the vein of cognitive exhaustion, there were a few times while reading Model Theory where I seem to have become cognitively exhausted before becoming physically exhausted. This was a first for me. I'm not referring to those times when you've done a lot of mental work and you shy away from doing anything difficult, that's happened to me plenty. Rather, in this case, I felt fully awake and ready to keep reading. And I did keep reading. It just… didn't work. I'd have trouble following simple proofs. I'd fail at parsing sentences that were quite clear after resting.

I found that this went a long way towards helping me track down places where I'd thought I learned something, but actually hadn't. If you're having trouble, go explain the concept to somebody (or to a text file). This can bridge the gap between "I read it" and "I can do the problems" quite well. For me, this technique often took problems from "unapproachable" to "easy" in one fell swoop.

I try to move the needle on at least two projects a day (more on weekends).

This tendency to see patterns in randomness is occasionally harmful. In tests where a green light is lit up 80% of the time (according to some random process) and a red light is lit up the other 20% of the time, where the subject must guess which light will be lit up, rats and pigeons always pick green while humans try to predict the pattern. In these scenarios, the rats (who always guess green) are right 80% of the time, while the humans (who tend to guess green 80% of the time but not the right 80% of the time) are correct only about 68% of the time.

Whether you use my tools or some other method, it might well be worth a few hours of your life to retrain your intuition for randomness. It may help you avoid premature judgements.

In part, this gave me a greater appreciation of small large numbers like a hundred. But mostly, it lowered my bar for what counts as a "large number". Which worked, in a way — my subconscious now has a much deeper respect for large numbers in that it finds anything greater than a few dozen to be tediously huge. When I encounter big numbers (such as, say, fifty three), it sort of throws up its hands and says "this is like that thing with the dice — way too big for it to be worth my time to comprehend."

This makes it much easier for me to disregard my (broken) intuitions about large numbers, and rely more on doing the multiplication.

counting wildlife threatened by other oil spills. And if he

if he cares that much about de-oiling birds, then how much does he actually care about factory farming, nevermind hunger, or poverty, or sickness? How much does he actually care about wars that ravage nations? About neglected, deprived children? About the future of humanity? He actually cares about these things to the tune of much more money than he has, and much more time than he has. For the first time, Daniel sees a glimpse of of how much he actually cares, and how poor a state the world is in. This has the strange effect that Daniel's reasoning goes full-circle, and he realizes that he actually can't care about oiled birds to the tune of 3 minutes or $3: not because the birds aren't worth the time and money (and, in fact, he thinks that the economy produces things priced at $3 which are worth less than the bird's survival), but because he can't spend his time or money on saving the birds. The opportunity cost suddenly seems far too high: there is too much else to do! People are sick and starving and dying! The very future of our civilization is at stake!

In the original mindstate, the reason he didn't drop everything to work on ALS was because it just didn't seem… pressing enough. Or tractable enough. Or important enough. Kind of. These are sort of the reason, but the real reason is more that the concept of "dropping everything to address ALS" never even crossed his mind as a real possibility. The idea was too much of a break from the standard narrative. It wasn't his problem. In the new mindstate, everything is his problem. The only reason he's not dropping everything to work on ALS is because there are far too many things to do first.

You don't get to feel the appropriate amount of "care", in your body. Sorry — the world's problems are just too large, and your body is not built to respond appropriately to problems of this magnitude. But if you choose to do so, you can still act like the world's problems are as big as they are. You can stop trusting the internal feelings to guide your actions and switch over to manual control.

I think that at least part of it comes from a certain sort of desperate perspective. It's not enough to think you should change the world — you also need the sort of desperation that comes from realizing that you would dedicate your entire life to solving the world's 100th biggest problem if you could, but you can't, because there are 99 bigger problems you have to address first.

becoming a philanthropist is really really hard. (If you're already a philanthropist, then you have my acclaim and my affection.) First it requires you to have money, which is uncommon, and then it requires you to throw that money at distant invisible problems, which is not an easy sell to a human brain.

Courage isn't about being fearless, it's about being able to do the right thing even if you're afraid. And similarly, addressing the major problems of our time isn't about feeling a strong compulsion to do so. It's about doing it anyway, even when internal compulsion utterly fails to capture the scope of the problems we face.

When you do the multiplication, you realize that addressing global poverty and building a brighter future deserve more resources than currently exist. There is not enough money, time, or effort in the world to do what we need to do.