How I'm thinking about giving now vs. later for AI safety
For the last eight years, I’ve been trying to come up with a good model of the value of giving now vs. later for reducing existential risk. I have no fewer than four half-finished drafts on the subject (realistically, more like 10%-finished). None of those attempts got very far.
Earlier this year, I gave up on those attempts and made a super simple model instead. But that model was kind of confusing and hard to use if you’re not fluent with probability distributions. I recently realized I had already written a script to do the same thing six months earlier, and my earlier script was better and easier to understand, but then I forgot it existed. Today I’m going to talk about the earlier script because it’s the one I’m actually using.
Cross-posted to the EA Forum.
The basic concept behind my recent attempts:
After the singularity, my money doesn’t matter anymore. I want to donate an equal amount of money every year from now until the singularity. But I don’t know when the singularity will happen. How much should I donate each year (in terms of % of starting wealth)?
The reason for donating an equal amount each year is that there’s a tradeoff between early and late donations:
- Early donations can have compounding effects.
- Late donations happen when you have better information.
If you’re unsure about which side of the tradeoff matters more, then it’s reasonable to distribute donations over time.
If I had perfect knowledge of when the singularity will happen, I’d donate an equal amount each year. But I’m uncertain about the timeline. Instead, I can calculate how much to donate given a distribution over possible timelines.
I wrote two scripts to calculate my optimal spending rate given the above model, plus some model assumptions. As I mentioned, I didn’t set out to write two scripts—I wrote one a year ago, forgot it existed, and then wrote another slightly different one six months ago. But the two scripts ended up serving as a robustness check, because they use two different methods to represent my beliefs:
- The first script takes a discrete list of
<years until point-of-no-return>and<probability for that year>with a selection of representative points (2 years, 3 years, 5 years, 10 years, etc.). Then it draws a curve showing how much I should donate each year. - The second script uses two continuous distributions—a log-normal and a power law distribution—and asks me for median and variance parameters. Then it draws three curves showing how much I should donate: one for each of the two distributions, and one for a combination of the two.
My previous post talked about the second script. The downside of the second approach is that it requires choosing distribution parameters, which can be unintuitive. (I set α = 1.5 on the Pareto distribution. What the heck is an “α = 1.5”?)
The first script takes a list of years along with probabilities. The probabilities represent how likely it is that it will be that many years before we reach the point of no return—the point at which AI is inevitably going to control the future, and therefore our dollars can’t influence anything.
Here’s the table I used (these probabilities are weakly held):
| Years until point of no return | Probability |
|---|---|
| 1 | 0.1 |
| 2 | 0.1 |
| 3 | 0.1 |
| 5 | 0.2 |
| 10 | 0.25 |
| 15 | 0.05 |
| 20 | 0.05 |
| 50 | 0.15 |
The script then fit this table to a distribution and produced a donation schedule for me:
Here is the donation schedule in table form:
| original | smooth | |
|---|---|---|
| 2026 | 25.7% | 25.4% |
| 2027 | 15.7% | 15.9% |
| 2028 | 10.7% | 10.6% |
| 2029 | 7.4% | 7.6% |
| 2030 | 7.4% | 5.7% |
| 2031 | 3.4% | 4.4% |
| 2032 | 3.4% | 3.6% |
| 2033 | 3.4% | 2.9% |
| 2034 | 3.4% | 2.4% |
| 2035 | 3.4% | 2.1% |
My thinking on why I chose these specific probabilities:
- Assuming AI development doesn’t hit a wall and we don’t deliberately slow down, we probably hit the point of no return within the next five years, and there’s a good chance that it happens on the early end of that range (e.g., as portrayed in AI 2027).
- AI timelines on the order of 10 years would also not be surprising.
- If AI development goes slower than expected, then it might end up going a lot slower, so there’s a long tail of probability going out to 50 years.
My first draft of this post had two years as the minimum time until the point of no return, but I don’t think I was being sufficiently doomy. In fact, there’s a meaningful chance that we have already passed the point of no return, and donations don’t matter anymore. But there’s no point in including that in the model, since it doesn’t affect donation decisions.
I’m tempted to put even more probability mass on the earlier years, but I spread out my distribution a bit on the basis of epistemic humility plus not wanting to front-load my donations too much.
My plan
For lack of any better strategy, I plan to donate roughly according to the schedule shown in the last section. This year I plan to donate more like 20% rather than 25%, purely based on vibes—donating a quarter of my DAF feels like too much.
In 2024, I donated basically a made-up amount that felt right on a gut level. In 2025, I thought about following a strategy like this, but donating 20% felt scary—I don’t want donate most of my money only to realize later on that I actually should’ve donated it somewhere else—so I only donated 10% or so. It still feels scary this year, but I’m going to follow the plan anyway (or close to it, at least).
I expect things will change next year: my distribution over AI timelines will either get narrower or wider. When that happens, I will re-evaluate the plan.
Source code is available on GitHub.