Why have happiness researchers been so slow to recognize the problems in using surveys to measure progress?

In my last post I pointed out that it is not possible to measure perceptions of progress accurately by using surveys to measure average life satisfaction at different times and then observe to what extent it has risen or fallen. As a result of changing reference norms, people who value an expansion of economic opportunities cannot necessarily be expected to show rising satisfaction with their lives in successive happiness surveys.

I have just discovered that a similar point was made by Francis Heylighten and Jan Bernheim over a decade ago, in an article that seems to have attracted little attention. The authors made the point as follows:

‘Progress could in principle be measured through the change over time of average scores of subjective well-being. However, the existing longitudinal data show little improvement. These survey results are intrinsically insensitive to developments over time, because SWB is typically evaluated relative to proximate, and therefore salient, reference points, such as peers or expectations based on recent experience’. See: Heylighen F. & Bernheim J.(2001): "Measuring Global Progress  Through Subjective Well-Being", in: Proceedings of the III Conference of the ISQOLS.

One of the suggestions that Heylighten and Bernheim made to correct this distortion was to develop a progress indicator from variables that explain a high proportion of cross-country differences in life satisfaction.

If that approach was followed to develop an indicator to measure perceptions of  progress, recent research by John Helliwell and Christopher Barrington-Leigh suggests that the relevant variables to include might be: the log of household income; whether the respondents had relatives or friends to count on if needed; whether the respondents were satisfied with their freedom to choose what to do with their lives; whether corruption was widespread in business and government; and whether they had donated money to a charity in the past month. Their analysis suggests that people in both high-income and low-income countries place about the same value on log income (use of logs allows for declining marginal utility of income) but people in high-income countries place more value on variables other than income. See: ‘Measuring and Understanding Subjective Well-Being’ Canadian Journal of Economics, 43 (3), 2010.

However, I’m not sure that the suggested approach would entirely solve the problem. It seems likely that perceptions that people in low-income countries have of the best possible life would involve a less opulent life-style than the perceptions of people in high-income countries i.e. perceptions of the best possible life rise with increasing wealth (and the marginal utility of income may not decline as rapidly as cross-country regressions seem to imply). In my view, that means it would be preferable to measure perceptions of progress directly using the method suggested in my last post, i.e. by comparing the answers that survey respondents provide when asked to rate their past lives at the same time as their current lives. An even better approach to measurement of progress, as suggested in the book I am writing, would be to identify the characteristics of good societies and measure to what extent societies were adopting those characteristics.

There may be a case to be made that the well-being of people in high-income countries would be higher if the move toward post-materialistic societies was more rapid. But the people who want to make that case should argue it openly, rather than pretending that responses to happiness surveys indicate that most people do not place much value on material progress.

Can happiness surveys help us to measure progress?

I have written about similar questions here before, but I’m not sure that I managed to get the message across to many people. The issues are not all that complex. I probably just need more practice in trying to explain them in simple terms.

The most obvious way to use happiness surveys to measure progress would be to use such surveys to measure average life satisfaction at different times and then observe to what extent it has risen or fallen.

Where is the problem in that? The main problem is that as a result of changing reference norms people who value an expansion of economic opportunities cannot necessarily be expected to show rising satisfaction with their lives in successive surveys.

What do I mean by changing reference norms? When we are asked to rate our satisfaction with life we do so relative to reference norms, such as by comparing our standard of living with that of people we know. Some surveys ask people to rate their lives relative to ‘the best possible life’, but our perceptions of ‘the best possible life’ may also change. For example, education may cause people to expand their horizons so they become less satisfied with a modest standard of living. The same kind of thing can happen when people move from rural to urban areas or obtain access to TV and the internet.  

So, if education tends to make people less satisfied with a modest standard of living, does that mean that they do not value the opportunities that education provides? It obviously doesn’t. Some people make large sacrifices to obtain educational opportunities, so it would be difficult to argue that they don’t value them.

The same reasoning applies to the benefits of technological progress. No-one could expect that people living in 1950 could have felt unhappy or dissatisfied - or sad, or angry even - because they did own personal computers or any of the numerous other amenities of modern life that had not then been invented.

The fact that we do not feel dissatisfied that we do not yet possess the products of future technological progress does not mean that such products will not enhance our future wellbeing and that of our descendants. It just means that we are fortunate to have emotional systems that enable us to give a high rating to our current lives if we can attain a standard of living that is somewhere near the upper bound of what it is currently possible for humans to attain.

Changing reference norms help our emotional system to adapt to changes in external circumstances, but that doesn’t mean that we should allow them to bias our judgements about changes in the quality of our lives.

Derek Bok, former president of Harvard, unwittingly provided a good example of the distorted perception that can arise when we ignore changing reference norms when he wrote:

‘As Americans adapt and yesterday’s luxuries turn into today’s necessities, people are naturally unwilling to give them up, but that does not mean that they are any happier than they were before the process began. Neither does it suggest that the products they yearn for in future will bring them any greater pleasure. What then is the justification for future economic growth?’ See: ‘The Politics of Happiness’, 2010, p 67.

The fallacy in that argument becomes obvious if it is applied to advances in medical science. Does the fact that people in high-income countries have adapted to advances such as the development of antibiotics, and now tend to view them as a normal part of life, mean that such advances have no value? In deciding whether or not we would be happier without advances in medical science, or any other product of technological change, the pertinent question to ask is whether we are obtaining a net benefit from it now. Adaptation may cause us to take for granted the benefits of technological progress, but it is our judgement of where our interests lie that makes us unwilling to give up the those benefits.

One way to eliminate the possible impact of changing reference norms is to ask survey participants to rate their lives at some point in the past (for example, five years ago) at the same time as they are asked to rate their current lives. Responses to such questions enable levels of individual flourishing to be gauged against historical benchmarks to show to what extent people feel that their lives have improved over time. My analysis of such data collected by the World Gallup Poll suggests that people tend to perceive the greatest improvement in their lives over the previous five years in countries where a high percentage of people consider that the national economy is ‘getting better’ and where rates of economic growth have been relatively high.*

Happiness surveys can help us to measure progress if they are used in the right way.

 ----------------------------------------------------------------------------------------------

*The estimated regression equation is as follows:

LIFETODAY = -0.330 + 1.003*PASTLIFE + 0.015*ECONOMY + 0.037*GROWTH + 0.299*IMPGOV

                       0.290)  (0.044)                     (0.003)                      (0.017)                    (0.243)

Adjusted R² = 0.84. The figures in brackets are standard errors of the estimated coefficients.

102 countries were included in the analysis.

LIFETODAY is the average rating ‘life today’ from the Gallup World Poll (around 2008) which asks respondents to rate their current lives on a ladder scale with the ‘best possible life’ as the top rung.

PASTLIFE is the average rating of ‘life five years ago’ from the Gallup World Poll.

ECONOMY is percentage of participants in the Gallup World Poll who perceive that economic conditions in their country are getting better.

GROWTH is the estimated rate of growth in per capita GDP (rgdpl) from Penn World Tables over the preceding five years (2002-07).

IMPGOV is the improvement over the period 2002-07 in the average of the six World Bank Governance Indicators.

How should we describe the current imbalances within western democracies?

Nicholas Eberstadt’s answer to this question is fairly clear from the title of his recently published book, ‘A Nation of Takers, America’s Entitlement Epidemic’. Eberstadt describes the growth of welfare payments in the US, the decline of stigma against accepting help from the government and the growth of dependence on entitlements. He establishes that about half of the US population now live in households receiving some government benefits and more than 30 percent now receive means-tested benefits. He suggests that, with the growing numbers living on disability benefits, ‘gaming and defrauding of the entitlement system have emerged as a mass phenomenon in modern America’. He also suggests that the ‘taker mentality’ has gravitated toward ‘taking from a pool of citizens who can offer no resistance to such schemes: the unborn descendants of today’s entitlement-seeking population’.

The book also presents two dissenting views. William Galston argues that although many people have come to depend on entitlements to fund their living expenses, they have not become ‘dependent’ in the way that children are dependent on their parents. He suggests that much of the growth of welfare entitlements rests on ‘temporally extended interdependence’. One generation consents to helping to fund the retirement of their parents, with the expectation that the next generation will do the same for them. He acknowledges, however, that ‘something has gone awry’ when the current generation discharges its obligations by imposing heavier sacrifices on the next generation. He suggests that the moral issue is ‘generational selfishness’ rather than dependence. He agrees with Eberstadt that disability benefits are subject to serious abuse, but suggests that the willingness of people to take advantage of the system is not necessarily evidence of deep cultural change.

The main point made by Yuval Levin is that differences in vision about the relationship between government and the citizen – collectivism versus radical individualism – overlook the importance of the ‘space between the individual and the state’, which is occupied by the family, civil society and the private economy. He argues that the state gravely threatens the space for private life. He suggests that rather than dependence, the problem is more ‘a draining away’ of ‘civic energies by the effort required to sustain the liberal welfare state. The country ‘is increasingly exhausting itself’ not just because of the size of the entitlement and benefit regime but also because of its ‘immense inefficiency’. Levin suggests that rather than a nation of takers, America is ‘a nation at risk of becoming incapable of rising to the challenge of self-government’.

The different viewpoints presented in this book are highly relevant to some issues discussed in the book I am writing. One of the points I am making is that when governments relieve us of the need to exercise our power of self-direction, then our skills in running our lives will not develop properly and we are likely to remain dependent on government throughout our lives. That means I am in sympathy with the points that Nicholas Eberstadt is making. At the same time, the US does not seem to me to be a particularly promising place to look for evidence of dependence on welfare having a widespread adverse impact on the social fabric.

I also suggest in the book I am writing that there is a growing gap in many wealthy countries between the responsibilities that many people expect democratic governments to discharge and what governments are actually capable of delivering. Perhaps it could be described as a problem of dependence, in the sense of governments becoming addicted to ever more spending (despite rising debt levels or increased reliance on unstable revenue sources).

At times, I have described the problem as an expectations gap, implying that it has arisen because of inflated public expectations of what governments can do. But it isn’t particularly helpful to blame ‘the public’. The underlying problem is that political leaders who seek to place responsibilities on government that are beyond its capability do not suffer appropriate political consequences. So, we should be thinking about how political leaders could be persuaded to moderate their promises and focus more effort on delivering efficient government.

The diagram presented below seems to me to be a useful way to think of the issues involved.

It is interesting to consider where particular countries should be located on the diagram. The countries of southern Europe should obviously be placed near the bottom right hand side and the Scandinavian countries would be at the top right. Hong Kong might be toward the left at the top. But where should we place the US, or Australia?

Is there more economic freedom in Australia than in 'the land of the free'?

It is difficult to believe that there could be less economic freedom in the United States than in Australia, but that is what economic freedom indexes seem to show.

The Heritage Foundation’s index currently has Australia in 3^rd place, behind Singapore and Hong Kong, and the US in 10^th place. The Fraser Institute’s index currently has Australia in 5^th place and the US in 18^th place.

Both the Heritage Foundation and the Fraser Institute have economic freedom in the US declining below that of Australia around 2008.  See Figures 1 and 2 below.

The timing of the decline in US economic freedom as indicated by the Heritage Foundation’s index suggests that it may be largely associated with the aftermath of the Global Financial Crisis, but the Fraser Institute’s index has the decline beginning around the turn of the century. According to the Heritage index the main decline in recent years has been with respect to financial and investment regulation, but the Fraser index also shows a decline in other areas, including freedom to trade internationally.

If the decline in US economic freedom was related solely to re-regulation of financial markets, it might be tempting to dismiss it as some kind of necessary evil. Well, it isn’t, but that will not stop me from suggesting that financial re-regulation has been far from benign in any case. Rather than side-track myself on that issue, however, I will just link to a highly relevant recent article by Ken Rogoff.

There are several reasons why Australians should not take comfort from indexes suggesting that we now have greater economic freedom that the US. The most obvious is that it has occurred as a result of a decline in economic freedom in the US, rather than any recent reform efforts in Australia.

The second reason is because the measurement of economic freedom is difficult. I think the Fraser Institute and Heritage Foundation should be applauded for their efforts to define economic freedom precisely enough to enable it to be measured, but I don’t think their indicators adequately capture all relevant aspects of the regulatory environment. For example, in 2008, just before he left Australia, Phil Burgess made some important points about the regulatory environment in this country that would be difficult to capture adequately in economic freedom indexes.

Before I proceed further, I should remind you that Phil Burgess was one of the ‘three amigos’ who came here to help Telstra through a difficult period. If that doesn’t ring any bells, you might remember the very wise - and very public – investment advice he gave to his mother about not buying Telstra shares because of what the government was then doing to squeeze profit out of that company, not long after selling it to gullible investors.

  

The comment that Burgess made concerns the degree to which the ‘public order’ tends to dominate the ‘civic order’ in Australia. What he was talking about was that, compared with other liberal democracies, government leaders in Australia have a very high capacity to frame and control the public dialogue by virtue of agenda setting, money and expectations management. Counterbalancing voices of legitimacy and authority in the civic order – including business - are, by contrast, often muted and ineffectual. He suggests that is associated with a greater tendency for the civic leadership groups to play an insider’s game and focus on influencing politicians.

I think Phil Burgess is probably correct in his judgement, but I don’t have sufficient knowledge of the US to be sure. If he is correct, it would be fair to say that economic freedom indexes tend to over-state the extent of economic freedom in Australia relative to the US.

My third reason for not taking comfort from indexes showing economic freedom higher in Australia than in the US is that public attitudes are still more supportive of economic freedom in the US than in Australia. Data from the World Values Survey suggests that Americans are more strongly in favour of the existence of large income disparities as incentives (i.e. less in favour of redistribution to make incomes more equal), less in favour of public ownership of business, somewhat more inclined to say that competition is good and less cynical on the question of whether success comes from luck and connections rather than hard work.

   

All that suggests to me that it would not make sense to bet my life savings that the economic freedom will be greater in Australia than in the US over the next few decades. The only problem is that my modest life savings are actually almost exclusively allocated towards investment in Australia. It might be time for a re-think!