The Effect of Financial Education on the Quality of Decision Making [pdf] by Sandro Ambuehl, B. Douglas Bernheim, and Annamaria Lusardi.
We introduce a method for measuring the quality of financial decision making built around a notion of financial competence, which gauges the alignment between individuals’ choices and those they would make if they properly understood their opportunities. We use it to document the potential pitfalls of the types of brief rhetoric-laden interventions commonly used for adult financial education. Motivational rhetoric can render the effects of such interventions indiscriminate even when people appear to understand and internalize the targeted concepts. Conventional methods of evaluation involving financial literacy, self-reported decision strategies, and directional effects on choices do not reliably detect these deficiencies.
Ambuehl et al introduce a new method for measuring the quality of financial decision making that assesses a consumer’s willingness to pay (WTP) for pairs of equivalent consumption opportunities, where one member of each pair is a simplified version of the other.
They used this method and another more conventional method to look at the effects of an online video on compound interest, and more specifically the rule-of-thumb (rule of 72) “that the percentage interest rate on an investment multiplied by the number of periods required for it to double equals 72 (approximately)“. So with a yearly interest of 8%, your $100 will double in 9 years (72/8 = 9).”Our experiment involves a web-based financial education intervention narrowly focused on the concept of compound interest.”
Study was done with N=504 subjects via Amazon Mechanical Turk. Average length of a session was 62 minutes, which I think is quite long. This study did pay at least double the hourly rate MTurkers make on average, which is $5/hour.
How does the new method work?
Respondents got 2 (simple or complex) x 5 =10 choices where in each case they had to state their willingness to pay. Rationally, WTP between simple and complex shouldn’t differ, because the pay-out is the same (spoiler alert: it does differ for real people, average difference is -13.3 percentage points)
If the difference [WTPcomplex] – [WTPsimple] < 0, then you underestimate the power of compound interest. You want to pay less to have the complex option with the (compound) interest, although the total value at pay-out is identical to the simple version.
How the method is explained in the paper:
Subjects performed 10 paired valuation tasks. Each task elicited an equivalent current dollar value for a reward r to be received in either 36 or 72 days. With simple framing, the reward was described as follows: “We will pay you $r in t days.” With complex framing, the same reward was described in terms of a return on an initial investment, as follows: “We will invest $a at an interest rate of R% per day. Interest is compounded daily. We will pay you the proceeds in t days.” Subjects made two sets of choices pertaining to each future reward, one with simple framing, the other with complex framing
So using the second row of the table below, an example of the simple framing would be: “We will pay you $18 in 72 days” and the complex framing: “We will invest $4,50 at an interest rate of 2% per day. Interest is compounded daily. We will pay you the proceeds in 72 days”.
The complex framing also amounts to $18 in 72 days, because $4.50 doubles in 72/2 = 36 days to $9, which doubles to $18 in another 36 days. Hence the number of Doublings =2 in the last column.
Not surprisingly, the valuation tasks (what is willingness to pay/WTP?) with complex framing take subjects nearly three times as long to complete than those with simple framing (59 seconds versus 22 seconds, p < 0.001).
Effects of online video
“according to standard measures, the interventions that include substantive material [i.e. explanation of Rule of 72] are highly effective at promoting financial literacy“.
Financial literacy was measured with the five questions below, and subjects in Substance-treatment (Rule of 72 explanation) scored +1.3 to +1.4 questions more correct than controls that on average had 2 correct answers.
[correct answers I think are: 7.2 years; 18%; about 20 years; $250; 36%]
Incidentally, 12,8% of the sample that did not see the video on the Rule of 72 (not in a Substance-treatment), later stated when explicitly asked that they had used that heuristic.
With the new measurement method (financial competence via WTP for equivalent products) Ambuehl, Bernheim and Lusardi conclude:
Thus, the provision of substantive information appears to induce greater effort and deliberation, but the addition of simplistic rhetorical assertions concerning the power of compound interest seem to negate that effect, perhaps because they point to a less cognitively demanding heuristic. (…)
Unfortunately, as we have seen, behavior may respond primarily to motivational rhetoric even when people appear to understand and internalize the substantive elements of instruction.
Earlier in their introduction, they also stated this objection to other methods:
The most common alternative is to evaluate the prevalence of dominated choices (…) In effect, each decision boils down to solving a math problem that has one and only one correct answer. Consequently, the approach amounts to administering an incentivized test of financial literacy. (…) In contrast, the vast majority of real-world financial decisions are not simply math problems.
Implications for public policy
- Devise educational methods that more effectively lead people to put pertinent knowledge into practice when making decisions, and to do so correctly. (…) Rhetorical prods may be useful for the purpose of marketing educational programs and boosting participation, but counterproductive when incorporated into pedagogy.
- Deploy educational programs targeted at populations known to manifest particular biases in order to create countervailing biases (challenging because highly context-specific).
- simplify the framing of naturally occurring decision problems, either by developing and deploying better tools for visualizing opportunities and consequences, or by requiring suppliers of financial products to characterize them in simple terms.
To sum up
Although the paper is clearly written, I found the content sometimes hard to follow. In my summary I have skipped many complications and robustness-test. Results were hard to interpret (I had to study the tables) because the metrics are not intuitive (subjects’ normalized valuations). It’s complicated and if you make the effort, I think it is worthwhile.