Making Inferences about the Effect of the Stimulus

Did the stimulus help the US economy? In the wake of the debt ceiling debacle, Obama cannot expect to get additional stimulus passed, because of Republican opposition to increasing spending when the debt has become as large as it is.  I want to stop and think about how we'd know if it had worked. One approach is what's called an interrupted time-series research design: the economy was bad before the stimulus, then there was the stimulus, and the economy is not now "good." So the stimulus had no effect. The trouble with this is that it's a pretty crummy research design. The post-stimulus world is a different world than the pre-stimulus one, and it's pretty hard to compare the two. We can really only tell what effect the stimulus had if we could compare a pair of USAs. They are exactly identical, except that one got stimulus and the other didn't. Unfortunately, we don't see the no-stimulus world; we need to be able to see that alternate universe in order to make a causal claim. This is what's known as the Fundamental Problem of Causal Inference (FPCI). If you want to know how a cause (or "treatment") affects an outcome, you need to compare the real world that received the treatment with an identical world in which the only different thing is whether or not a treatment was received.

We don't know how well the stimulus worked because the possible states of the world include the following:

1. The economic stimulus had no effect; the economy has not changed as a result..
2. The economic stimulus had a large effect; the economy would have been much worse without it.
3. The economic stimulus had a large effect, the economy would have been much better without it.

Not to mention the many states of the world between those three. We could be in state 2 or 3 and we wouldn't be able to differentiate between them. State 2 includes the possible worlds in which economic and job growth both grow largely as well as the world where economic indicators grow more than they might have otherwise, but they still aren't positive.

Just to be clear: this means that we cannot tell if the stimulus was effective by comparing the state of the economy before the stimulus with the state of the economy afterward. We could compare the behavior of firms that received stimulus funds with those that received no funds. It's not quite the stimulus vs. no-stimulus world, but it's closer, especially if we think that direct and indirect effects across firms were minimal (this is a big problem that I do not address here). A recent paper by Jones and Rothschild (h/t to Tyler Cowen for covering the paper) attempts part of this comparison. The study claims that the stimulus should have been larger to be effective, because only 42 percent of the workers hired with stimulus funds were unemployed. Jones and Rothschild, the authors of the study survey firms that received stimulus funds to find out how they behaved given that they used those funds.

What's wrong with this picture? It's the FPCI: By studying firms that received stimulus funds, we don't know how firms that did not receive funds reacted. Not only can we not assess how an outcome varied by using a variable that doesn't change--in this case, a binary indicator of whether a firm received funds or not, but we cannot assess how an outcome varied when we don't observe the counterfactual state. That is, how would these funds behave if they had not received stimulus. The authors attempt to mitigate this by trying to get their respondents to think couterfactually, but this fails to solve the problem. We can't expect the individuals involved in decision-making to know how they would have behaved in different circumstances. We only know how they did behave. More specifically, in the case of this paper, we don't know whether non-stimulus firms were shedding workers at drastic rates (so that 42% looks really, really good in comparison) or were hiring from the ranks of the unemployed at much higher rates (so that 42% looks pretty poor).

There are at least two ways to solve this problem. One may already be lurking in their data. The other requires significant additional work. One way to (attempt) to solve the FIPC is to make relevant comparisons. We could compare the US to another, similar country that didn't attempt stimulus, or that attempted less stimulus. But more specifically, we could focus on the implicit comparison the authors make: We can compare firms that received lots of stimulus to those that received less stimulus. Or we can compare firms that received stimulus to those that received no stimulus. As I mentioned, the first might already be in their data--it makes sense that they would have asked how much stimulus funds the firm received, then they could calculate a per-employee or per-output measure of stimulus received to make the numbers more comparable. Or they can conduct another survey and attempt to match firms that didn't receive stimulus with those that did.

Both of these approaches may still be problematic in terms of FIPC. No matter how many factors these firms share, unless stimulus was awarded via a lottery, it will be difficult to know for certain that the difference made by the stimulus is not, in fact, a result of some other factor.

Obama Doesn't Deserve Any Credit for Egypt

With Mubarak gone, we (rightly) want to assess Obama's handling of an entirely unexpected crisis. Niall Ferguson criticizes Obama's team for not appropriately gaming such a difficult to image scenario. Sure, this would have been advisable, but at some point your foreign policy team has to decide which in conceivable threats to deal with, and which are, well, simply inconceivable (This makes some sense, whether or not revolutions are inherently unpredictable (as Scott Wolford argues) or if a group of Egypt experts had, in fact, predicted Mubarak’s weakness (as Dan Drezner claims)). Slate’s John Dickerson weighs in in Obama’s favor, arguing that when events move quickly, doing little isn’t such a bad option. Foreign Policy's Blake Hounshell argues similarly that Obama, while spending too much time trying to "thread the needle," came down in favor of the protests, but in a way that avoided making the protesters look like foreign stooges.

All of these arguments miss an important point about what Obama did. Under pressure from Middle Eastern allies, the Obama administration responded carefully, finally deciding to "support" the protestors in their calls for reform, but by maintaining calls for stability. Mubarak and his government could stay in power, handle the transition, and work toward September elections. Some commenters (like Stephen Walt and Blake Hounshell) claim that this represents Obama’s ultimate support for the protesters.

I don't see it that way. Unless Obama and his team are much more naive than I give them credit for, they, like any other politician, know the value of political promises. Without strict incentives to follow up, political promises aren't credible. And any promise by Mubarak to work towards "transition" wasn't credible. The Egyptian protesters knew this: that's why they stayed on the streets. If they had gone home, Mubarak could simply have loaded up Facebook and started hunting down the organizers, one by one. By September, any new protests that could have been generated by the new elections could have been shut down by arresting the organizers and putting security forces out in force to quell new protests. End result: Mubarak stays in power, with Obama's support for reform giving him the out he needed. 

Good thing the Egyptians didn't fall for it.

Here's the real rub, though. The US can pretty much afford to piss off Arab dictators--Mubarak and the rest, since, as far as foreign policy goes, their interests are our interests. Saudis don't want high oil prices, but not because they're worried about the US. Mubarak's Egypt will always play along on combating terrorism and maintaining the treaty with Israel. They can't afford not to. Both because of the outrageous levels of development and military aid we give Egypt, but also because it's in their interests to do so.

If Mubarak hadn't fallen, angering him wouldn't have mattered so much, because our interests are aligned. If he did, then it's critical that we played the right game during the protests. And that game was to unambiguously side with the protesters. We weren't, and I'm certain that if the next government is a non-military one, they'll remember who got them there and who didn't.