Vatsal Khandelwal on Pessimistic Belief Correction and its Impact on Well-Being

SHRUTI RAJAGOPALAN: Welcome to Ideas of India, a podcast where we examine academic ideas that can propel India forward. My name is Shruti Rajagopalan, and this is the 2023 job market series, where I speak with young scholars entering the academic job market about their latest research on India. 

I spoke with Vatsal Khandelwal, who is a junior research fellow at Merton College and an associate member of the Department of Economics at University of Oxford. His main research interest are development and behavioral economics, with a focus on social and economic networks.

We discussed his job market paper titled “Silent Networks: The Role of Inaccurate Beliefs in Reducing Useful Social Interactions,” (coauthored with Ronak Jain). We spoke about pessimistic beliefs and their impact on individual’s well-being, the methods to correct incorrect beliefs about social norms, the importance of social networks, support groups, students’ expectations’ impact on performance more.

For a full transcript of this conversation, including helpful links of all the references mentioned, click the link in the show notes or visit mercatus.org/podcasts.

Hi, Vatsal. Welcome to the show. It’s a pleasure to have you here.

VATSAL KHANDELWAL: Hi, Shruti. It’s wonderful to be here. Thank you for the invite.

RAJAGOPALAN: What you’re really looking at in this particular paper, is the effect of incorrect or pessimistic beliefs that people have—in this case, relatively low-income workers—the belief that they have about the social norms within the community which impacts how they interact with them. The very specific thing we’re talking about is the willingness to share details about financial stress, other kinds of social and economic anxiety, any kind of mental health issues, and so on. There seems to be some kind of general stigmatization or shame associated with these kinds of questions, so, it’s important to have a good gauge or the right beliefs about the immediate social cohort that one operates in.

If it’s wrong, it has bad consequences, which is that people don’t rely on their social networks as much because there’s no dialogue, and it has other bad consequences. What you do is quite interesting, which is you run this randomized controlled trial [RCT], and you provide the treatment group, relative to the control group, you provide the treatment group the correct information about other people’s (in the group) a willingness to engage in dialogue about financial health, mental health, those kinds of questions.

And you find that actually this kind of intervention—to correct a previously misguided or incorrect belief—significantly increases that individual’s engagement with their immediate community. Then, of course, you find some very, very specific results, like, these participants are 15 percentage points more likely to sign up for savings group with the community members. They’re more likely to become volunteers in mental health and social anxiety discussions, and so on. And the effects are quite big.

Can you talk to me about what this intervention does exactly? What’s the mechanism by which you’re getting all these results that you’re seeing in what seems like a fairly complex social experiment?

KHANDELWAL: Right. Thank you so much for the introduction to the paper, and thanks for the question. I think our motivation while designing the intervention was that, as you mentioned, individuals rely on their social ties to provide and seek support. We know vast literature that shows that social networks are used to smooth consumption, spread information, seek advice, but the ability of the network to perform these functions can be severely constrained if individuals don’t demand it in the first place.

This lack of demand, as you mentioned, can arise due to concerns around violating social norms. For example, if I believe that the majority of my community is not going to view me positively or is going to disapprove if I reach out to them for financial advice, that’s going to prevent me from doing so. Now, these norms are deeply ingrained and they’re hard to change, but misperceptions about the norms can be easier to work with. What we do is we study how inaccurate beliefs about your peers or your community’s willingness to engage can be a first-order concern that generates what we call silent networks: networks where there’s a high potential benefit of interacting but limited useful interactions.

We, as you mentioned, combined evidence from RCT survey experiments and structural estimation, to answer three questions. The first is, do inaccurate beliefs exist in the first place? Second, how can belief-correction interventions reduce these inaccuracies, improve network interaction, strengthen network ties? And third, I think there’s a more structural question, which is, okay, we had this one approach, this one mechanism towards solving this issue: There could be so many other policy instruments, and we want to be able to make a comment on that by leveraging the random variation that we induce in the RCT.

In terms of what we do, we work with a sample of low-income workers in Delhi, and we note in our baseline that there’s low levels of dialogue. They are not interacting with each other on mental health and financial health-related concerns, especially— this is surprising given that they lack access to formal assistance of any kind. They also have very few links and advice networks to do with mental health and financial health. At the same time, majority of them underestimate the proportion of individuals in their community who are willing to engage with them on these topics.

When we provide them with an information shock—so, this is like a first step. The RCT is like a proof of concept where we shock your beliefs and want to study the impact of that on outcomes. When we do that, we find, as you mentioned, [an] increase in willingness to participate in sittings groups and listening services, but more than that, we also find that individuals are willing to finance these, to some extent where they are willing to pay for them. I think that to us, was a very surprising result, that network ties are inefficient to the extent, and inaccurate beliefs are inaccurate to the extent, that once you correct them, people are willing to pay to set up these channels to interact.

In terms of mechanisms, we run additional experiments and use survey evidence to tease them apart. We find that the primary channel through which this is happening is a reduction in the cost of violating the norm. People are concerned about reputational factors or interaction-related shame. It is precisely the pessimists, the ones who are our target group for the intervention, we find in a later experiment that it is exactly those people who are more worried about these concerns as well. We try to causally tease apart different costs of violating the norm and find that that might be the thing that’s happening.

RAJAGOPALAN: One of the things this is in particular on the question of financial health and financial advice, not just mental health—what I was thinking was, in addition to just shame, it seems to me almost like the pessimism is not just associated with violating the social norm. There might even be a thought process that if I share this information about my lack of information or my lack of access, then I’m even less likely after that to get any kinds of financial access or get credit from networks and so on. Is that what’s partially going on in the financial health story, which is a little bit different from the mental health story?

KHANDELWAL: That’s a great point. This is definitely something we were thinking about. Like I mentioned, cost of violating the norm, what exactly do I mean by that? These costs I mentioned could be reputational, it could be interaction-based, but as you mentioned, it could also be signaling. If I signal that I’m a certain type, you might not reach out to me to offer me jobs later on. These are individuals who want to get these jobs. 70% of them, for example, say that they use the network ties for hearing about jobs.

That might be a concern, and specifically so for financial health. In a later experiment, which we do, where we ask individuals to predict whether a link would exist between any two randomly chosen people in the community, and we vary the characteristics of the adviser, we say, "Okay, the adviser has contacts in private jobs or not, the adviser is network central." It’s very important in the networks to primarily to get a gossip, so maybe they are network central or not, and they’ve attended a sensitivity training by the NGO or not.

We vary these, and we find that signaling is not something that has a causal impact on people’s predictions of whether these links would exist or not. It’s precisely interaction and reputation that’s affecting their perception. Of course, this is a hypothetical sort of game that we play. More than that, it’s not differential for mental health and financial health-related advice taking. That’s something that we also noticed.

RAJAGOPALAN: Yes. That tells us something about the social stigma and social shame that’s going on, especially amongst the low-income groups, right?

KHANDELWAL: Exactly.

RAJAGOPALAN: One question I had was, how much of what’s going on in the experiment is driven by the pandemic-related stress? Because part of this is playing out during the pandemic, and we know that low-income workers, especially in Delhi, had a lot of pandemic-related stress, in terms of both the pandemic and the lockdown and the effects after that. What’s going on with that story?

KHANDELWAL: I think two points on that. One is, whether the pandemic affected the levels of stress, which it certainly did; however, the question is whether these are specific to the pandemic. To that effect, we find, for example, in 2023, when we went back and did in-person surveys, we found that a huge majority of them, around 75%, report facing consumption crises events in the last six months even now. I think stress for them is, unfortunately, not something that is unique to the pandemic.

But, I think the second thing, in addition to the point about stress, is some of the patterns that we observe in terms of inaccuracies, or in terms of low dialogue, or in terms of the gaps that we observed between overall networks and advice networks: Are these patterns specific to the pandemic? Do I just have inaccurate beliefs, or am I just not interacting because we are in a period of just low introductions in general? We find that not to be the case. Still, there’s a significant proportion who rarely or never talk about these issues with their peers in 2023. More than that, there is still a huge proportion that underestimates the willingness of others. These patterns carry forward to a time outside of the pandemic.

I think the last thing I would mention here, which is one of the things we don’t do in the paper, which is something I would definitely want to think about in future work as well, is the production function of these beliefs. What we do find here is that majority of people have been living in these centers and in these localities for a long time and it’s not correlated with having more accurate beliefs.

It’s not like, “if I’ve been living here and I’ve been interacting in close proximity with others, I have more accurate beliefs.” That doesn’t happen to be the case. Again, that reinforces that it’s not like a pandemic-related low interaction, a feature of the pandemic that is causing inaccurate beliefs.

RAJAGOPALAN: You know how much of this is also a function of the nature of the economy, which is like whether they’re part of the informal economy or whether they are more integrated into formal associations and institutions? I’m asking for multiple reasons. Now, we know that the informal part of the economy may also overlap with lower levels of income or wealth, may also increase vulnerability and financial stress because it’s informal and you don’t have stable contracts.

That’s just one part of it. The other part of it is, if people have more stable jobs—for instance, they belong to the same firm, right?—or members of a particular industrial area all have access to the same gym or to the same entertainment area and so on, you start forming associations with people, either formally through the firm or informally outside the firm. But you know that all of you belong to a similar space or a similar network, which is almost always the social grease that keeps these wheels running.

How much of what you are observing is because these people are trapped in an informal economy where you don’t have better, more stable arrangements to network through the employment situation or other situations?

KHANDELWAL: I’m going to address this in, I think, two points. One is, of course, the informal economy. There’s a lack of avenues to interact in, as you pointed out, and you’re going to talk about that. Even before that, I think the fact that this is an informal setup makes network interactions absolutely critical, which may not be true in formal settings as much. I think the treatment effects thereby also what we observe—might very much—the magnitude is very dependent on the fact that this is a formal setup.

I think regarding the point about avenues to interact, I would maybe not distinguish between the informal or formal because I can imagine, for example, a variety of settings in firms or schools or those kinds of things where individuals still have avenues to interact, but advice networks are not forming.

People, for example, students don’t want to ask questions in classrooms because they feel they’re signaling something, or employees don’t want to seek advice about certain things that can improve their productivity because they are worried about signaling. I think despite those avenues, advice networks can still be useful and constrained in terms of how many links are forming on those.

I think the distinction I would draw is precisely that what you alluded to, which is whether or not there is an institutional platform or an institutional premise on which people can act. You can think of informal rural settings, maybe caste and the in-links that people have within their caste provide that platform that you’re not worried about violating norms when you’re seeking financial advice. We have so many papers on risk-sharing networks in rural India where this is not the dominant concern [for] people.

There are other concerns like trust and enforcement and those kinds of things, but norm violation is not a concern. Maybe because castes provides that apparatus on the basis on which people can interact. I think I would distinguish between environments that have that institutional apparatus and don’t have it. Even formal environments may not have it, and informal environments might have it.

RAJAGOPALAN: This is where I think one part of the story may also be a question of migration, right? Again in Delhi, specifically the areas that you’re looking at, it’s a migrant-driven informal economy, if I can call it that, right? They’re not long-term native members. They’ve come here very specifically looking for jobs. Their housing situation or segregation is very much determined by the fact that they are migrants.

How much of what you are observing is a function of the fact that they have all these social networks back home? Here, it’s a new norm because it’s a new place, it’s a new city, it’s a new housing slum, or new construction situation where they’re suddenly employed—they don’t know these people. It’s quite reasonable that they don’t want to interact with them, and they are either feeling shy and awkward, or they’re worried about reputation because they’re basically job-seeking migrants who do need to make a good impression and find the next gig and so on.

Is there a substitution effect where there are networks back home that are advising people on mental health questions, on financial health questions in a way that they are less likely to tap into—or more worried about some sticky social norms or social shame—in the migrant, migrated-to, area like New Delhi?

KHANDELWAL: No, that’s a great point and that’s definitely something that we also had in mind, that what if this is not the relevant network. In our sample, what we find is that there are 30% of them who are migrants and the remaining are non-migrants. Out of those 30% as well, majority report not being in contact with their home communities for these issues.

I think that makes it [being a migrant] a setting where we think this might not be as much of a reason for inaccurate beliefs as other features that we highlight in the paper. I think the other thing is on average, in our sample, the average years that people have stayed in these localities is 20. That’s quite high. It’s even more surprising that on average, people who have stayed for 20 years are having inaccurate beliefs about these kinds of things. I think if the sample did have more migrants, the inaccuracies would be as you mentioned, even worse.

RAJAGOPALAN: Therefore, the treatment effect would be even bigger in some sense.

KHANDELWAL: I think the substitution that we do observe, which was very interesting, is between informal and formal support systems. One of the results that we do find—apart from an increase in take up for, an increase in demand for network engagement and paying to do so—we do find some evidence of substituting away from formal assistance.

We, for example, give our participants a chance to listen to helpline numbers, and treated participants are significantly less likely to do that. Or we give them a chance to participate in a depression scoring where they learn about their own health by something that is implemented by the enumerator, and treated participants are less likely to do that.

That suggests a tension between the experimenter or a helpline coming to help you versus you now knowing that a network is going to provide support and feeling more comfortable using that support. I think that tension was very interesting. In a different world, you could imagine the two being compliments, but here, we don’t find that.

RAJAGOPALAN: Now that I’ve understood your setup so much better, I want to come to the interventions, and what that really means. It’s quite clear that the treatment effects that you see, they’re quite big. They’re definitely significant, worth thinking about whether they can be translated into some meaningful intervention.

Before we get into that, how much of this is about specific beliefs in a very targeted situation? You’re talking about very specific questions about employment, or financial health, or mental health, or in some cases it could be pandemic-related health, or… how do we solve for a situation where there is an infectious disease out there and so on.

One is the question of, is this generalizable to other questions that also require social interaction, that also require social dialogue, that also require this network to come in to support? The second is, can any of these interventions really be long-term, or will you just see a short-term effect? Then the expectation is that people will adjust to that as some new normal, and then these effects start disappearing. What is a good way to think about what are we really doing with these interventions?

KHANDELWAL: Yes. On your first point, I think we are interested in gauging inaccuracies to do with a network that might influence other misperceptions as well. This is not inaccuracy about a particular, let’s say, opinion or a particular topic, but this is inaccuracies about interaction in a network.

If you think most people are not going to engage, you will not interact, thereby the learning that takes place in the network presumably is going to be affected. Thereby you will develop misperceptions on a variety of different other things.

RAJAGOPALAN: Can I ask a follow-up there? For instance, I can quite easily imagine a situation where people like to discuss politics, or how they should think about voting in the next election or something like that. That’s another area which is rife with misinformation and incorrect beliefs and incorrect facts.

There might be people who enjoy discussing this and actually the network informing them or advising them on specific political questions or political facts may actually improve both their access to voting, their ability to vote for the better candidate suited to their situation, and so on. But it’s not clear to me that I would see the same social awkwardness or shame in discussing political information that I would see in discussing financial information.

So, to me, it seems like it’s not just about engaging with a network that can potentially offer support and advice and suggest better outcomes. There seems something more specific about that. Maybe I’m not explaining myself very well.

KHANDELWAL: No, no, no. That’s a great point. I think there are certainly various other functions that the network could perform such as be a platform to discuss politics that would potentially be unrelated to inaccuracies around financial or mental health–related advice taking. Now, this is outside the scope of the paper, but I can think of multi-layered networks where there are complementarities and links across. The links are stronger if you are also someone I take advice from, if you’re also someone who might interact with to borrow or lend. Thereby the strength of a link on the intensive margin could still be affected in a network of political interactions, for example.

RAJAGOPALAN: Fair enough.

KHANDELWAL: That’s one. The second is I think we are deliberately vague while talking about engagement with financial health and mental health. We are specific enough so that participants are all on the same platform and they think about these issues, but vague enough to be able to then say that this is about financial wellbeing rather than contacting someone, let’s say, about a very specific job. I think then that kind of network would also have effects on, “should I reach out to someone telling them I’m in financial distress and I need a job, or should I reach out to someone telling them I’m in financial distress and I need money?”

I think that those kinds of different ambit of interactions would still fall within the umbrella of our definition. Definitely, I would not want to claim that this covers all kinds of interactions. It definitely does not.

RAJAGOPALAN: On the question of short- versus long-run effects of these interventions, what is a good way to think about that?

KHANDELWAL: This is precisely where the model and structural estimation comes in. That was the key reason to have that, to be able to predict the long-run impacts of the RCT. We find that our intervention is not going to have long-run impacts.

RAJAGOPALAN: I have a follow-up on that. When you say your intervention doesn’t have a long-term impact, your intervention is basically a belief correction about the group—

KHANDELWAL: Exactly.

RAJAGOPALAN: —in question. What do you mean when you say it doesn’t have a long-run impact? Is it that they go reverse back to the original, or is it a new misperception? What’s going on there?

KHANDELWAL: I think like us to imagine setting where my beliefs, let’s say, affect my choices. In the next period, people observe choices, each other’s choices, update their beliefs, and they process this immediately between beliefs and choices that continues over time. When I shock your beliefs, you either come back to that original equilibrium where you were, where you started with. Or you can think of a threshold, and you cross that threshold, and you go to this new, beautiful equilibrium where everybody’s talking about these issues.

What we find is, as you mentioned, it’s precisely coming back to that old equilibrium. Within the confines of the model and the estimation and the assumptions that they have, we can make that prediction that individuals are going to go back to that equilibrium.

Of course, how short is the short run is a question to ponder about. Two interesting things that we can say is, [one,] how costly alternative interventions would be, interventions that could translate the short-run effect of the RCT into something long-run. The estimation allows us to do that. The NGO in the setting, we’ve been talking about a story to do with beliefs. It could be so many different things. We could make people interact in savings groups, allow them to update the benefits of interacting. In the most literal sense, the benefits of interacting could be increased by a cash transfer—of course, infeasible and not sensible at all, but that is something that you could think of it in those lines. Now, if you did do that, what we find is the mean of the benefit distribution has to be shifted by half of its current value, which is a really strong effect, to generate effects that are at least as large as the short-term effect of the RCT long-run effects as large as the short-term effects.

Now, not only that, given the empirical evidence that individuals are willing to pay to set up informal avenues, you can now see how belief-correction interventions, even if they don’t have long-run impacts, can be used to finance—to allow—communities to self-finance big push interventions.

I might not be able to set up a savings group as an NGO, and there’s a huge policy gap here because I don’t have enough funds, but we can generate the funds from within these communities by shifting their beliefs, which we do in this setting and use those funds to finance something big push.

RAJAGOPALAN: This is where I start thinking about how the RCT is a very interesting tool to figure out what’s going on, but it also exposes the difficulties or limitations in fixing it as easily as it exposes what’s going on. Is that also the sense that you get when you’re at the end of this long journey having done the RCT, the surveys, and so on?

KHANDELWAL: Our prior was that the belief-correction intervention obviously is not going to have long-run effects. What we thought it would be useful for is to gauge, firstly, the demand for network interactions, and how that shifts with shifting your perceptions about the norm. And more importantly, what we then did is, as I mentioned, use the estimation to say, “Okay, what if we changed this thing or this other thing or this third thing, and what if we changed these things? How would that compare to the RCT effect?” I think using the RCT to, of course, do that proof-of-concept exercise to increase this willingness to generate those funds is important, and it’s something that we definitely did want to do; but also using it [RCT] to be able to make bigger comments using the model is something that we were very interested in.

RAJAGOPALAN: You just spoke about one of the priors that you and your co-author had was that this is not going to have a long-term effect. You had that sense going in. I want to talk about a different prior. Why is it the prior that support groups and networks can actually form easily or automatically or without frictions? Why is that a prior in the first place? Because I am not a low-income worker, I am not a migrant in the informal economy of New Delhi. I am relatively privileged. I live in the greater Washington, DC area. I have formal employment. I have health insurance, everything. It’s not clear to me that I would do that much better [laughter] in overcoming some of these questions when it comes to financial health or mental health. My assumption would also be some form of an incorrect belief except the weight pans out normally—at least in the world I observe, which is a very first-world elite situation—that this gets solved through trusted arbiters.

If there’s a demand for support group—I’ll give you the extreme example, which is Alcoholics Anonymous or something like that—there’s clearly a problem. This is an issue that has a social shame and stigma, but there are also massive benefits in forming a support group that can advise you on how you cope with a substance-abuse problem and so on. The way that the market or society has corrected for it is through a trusted arbiter. In this case, Alcoholics Anonymous is that brand or that service. You can imagine village situations where there’s a trusted arbiter who’s a local leader or the village elder, or someone who went away to study and then came back, and now is the educated person in that village, and offers advice of all sorts to his neighbors.

My question is why would we assume a different prior in the first place? Now, I’m not trying to negate the reason for doing the RCT, but just as a curiosity. Why is the prior different from the obvious prior, which is, of course, people have incorrect beliefs about social interactions and networks.

KHANDELWAL: I think that’s a very good question, and this relates back to our conversation briefly about whether there’s an institution provision to interact, which is close to what you described as an arbiter. We were earlier talking about rural networks and castes where the caste acts as an institutional arbiter, and people interact on the basis of that. I think for us, the prior was really updated. We updated our prior very strongly when we realized the huge inaccuracy here. Of course, nominal inaccuracies would exist regardless in various contexts, but the fact that 70% of them are underestimating it and that it leads to that big an increase, that challenged our priors quite a bit. We definitely weren’t thinking that it’s going to be anything but organic. We were gradualist in the way we were thinking about this as well. The model that I described to you also thinks of it in a sense where beliefs affect outcomes, outcomes affect beliefs, and the process then leads up to this good equilibrium or a bad equilibrium. We were thinking along those lines, but the fact that these really pronounced inaccuracies would exist, I think that was surprising to us, especially in a setting where social safety nets are key.

That we don’t interact with, let’s say other individuals when we meet them in the gym and don’t take advice on certain things, that doesn’t have a huge cost to us. In a setting where there is absolutely limited formal assistance, this magnitude of inaccurate beliefs was something that challenged our priors quite a lot. I think one of the things that we’ve learned by doing this also is that we cannot and should not take social networks as granted in terms of their ability to function as social safety nets. In various settings, they might not function at all.

RAJAGOPALAN: One thing I got out of this is that social networks are extremely important, but they’re also extremely hard to engineer and intervene. Is that your sense at the end of this experiment?

Given that the effects are short-term, given that people go back to their original priors, after the intervention. Given that to have a long-term effect, the intervention has to be so large and so expensive. Those are the things that lead me to believe that this is a bigger question than just a simple policy intervention. That bigger question of social networks is something that is so important for economic outcomes but equally hard to artificially engineer.

KHANDELWAL: No, I entirely agree. I think one of the findings here that the model points us towards is that individuals are very much stuck in this “low interactions poverty trap” if I can call it. In that, the low interactions actually worsen outcomes. Now, whether it’s easier to maneuver, it is not easy at all. We have a result which shows that these short-term outcomes are really huge. Whatever our definition of long run will be, that those outcomes do not persist.

What I do find very reassuring, though, from a policy perspective, at least in the world where we know that there is no policy assistance, what I find reassuring is that you can finance your way out of it. You need a costly intervention. You don’t have a government that is actively treating you as number one priority. You can then self-finance your way out of it by forming these interactions using costly approaches, but you yourself are paying for it [intervention]. I think the fact that belief-correction interventions can generate that fund, and we’ve collected that fund, and it’s been given to the—

RAJAGOPALAN: It’s a big effect. You said they’re more likely to make a 29% higher contribution towards setting up the service. That’s a large contribution.

KHANDELWAL: I think they’re still very much gradualist about it in the sense that we would not expect belief correction interventions to have this one-off impact and then think we’ve changed. In fact, the paper shows that you will not change it to an equilibrium. That’s good knowledge to have so that we don’t assume otherwise. It’s also a positive thing to know that you can gradually build up the funds to finance something stronger.

RAJAGOPALAN: Can I bother you on a different paper that you’ve been working on and ask you a couple of questions? This is your work with your co-author Minahil Asim and Ronak Jain. This is an experiment you conduct in Pakistani schools. What you find is that teachers communicating their expectations to students impacts learning outcomes. This is not just the regular stuff that the teachers are teaching students. This is quite specific. This is teachers telling them what they think of their current performance and what they expect from them in the future in a very targeted way, not at the classroom level, but at the individual student level.

You find that this has quite a big impact on learning outcomes. Specifically, when teachers expect something more, that is, they have a higher expectation of the student than what they are than their current performance, they tend to do particularly well.

What is the underlying mechanism that’s driving this result? Is it something as simple as it’s a signal that the teachers give a shit, and that’s prompting the students to do better? Or is it something like they’re getting highly individualized feedback which is specific to them, which is prompting them to do better because that’s tailored to their situation? What is going on here because the effects are large?

KHANDELWAL: Firstly, thank you so much for asking a question about this paper. It’s not a bother at all. This was the first paper I started working on during my PhD. I’m very excited to be talking about this. I think you are exactly right. The fact that we observed such large effects on academic outcomes was also very surprising to us. Again, relating it back to our previous conversation, this also happens to be a setting of limited communication between teachers and students about expectations. It’s not every day that teachers deliver expectations to students.

Now, what we wanted to do is precisely to get at the mechanism, like, does the magnitude of the expectation matter? Maybe I give an expectation that is too high compared to your performance, and then you get demotivated and frustrated and perform even worse. We randomly varied whether students get a framing as the teacher expects you to achieve at most this much or at least this much. We, of course, elicited this student-by-student, tailored from the teachers to make sure that we don’t give them insensible numbers. What we find is that even when they get this upper value, they perform even better.

I think the mechanism that we think is operating here is you were exactly spot on. I think it’s something to do with the message being exactly very tailored to the student with a follow-up survey when they were asked to interpret, "Okay, you got this infographic with this particular score and target. What did you think? How did you interpret it?" Majority of students thought that that was motivating, because they interpreted it as a goal.

I think some key aspects of the design were also baked in to get at some of the mechanisms. For example, is it just about being contacted by the teacher and getting information, something you learned about yourself that you were probably not paying attention to, so we had an information arm that just gives information about current performance? Or we want to see how classroom norms get into this, or classroom support gets into this whole story. We randomly match you with a peer who also gets an expectation then you both motivate each other. We want to see how that affects outcomes.

I’m just going to briefly allude to those results here, which is very counterintuitive. We find that the information actually has an effect. It’s just information about the past performance that has an effect. When you add on the peer angle to the expectations, the classroom networks angle, we find the effect goes away. That’s mainly because there are specific types of pairs that are benefiting and specific that are losing out, and the average effect goes away. The paper should hopefully be out by the time the episode is released. I’m very curious to hear more thoughts and feedback on it.

RAJAGOPALAN: We have something called Emergent Ventures, which does moonshot grants. One of Tyler’s big themes is that he thinks a mentor can bring in an enormous amount of value by increasing the aspiration level of the mentee. That’s one of the biggest value-adds that a mentor can provide if you can raise the aspiration value. I was wondering if something like that is going on here. If you say you can do so much better than your current performance, which is also leading to the largest effect in your paper, I think, it seems like there’s something going on about that, which is that you are subliminally or explicitly switching the aspiration levels and what impact that has on learning.

KHANDELWAL: That’s definitely something that could be at work. One of the things we do find about the mentors and aspirational mentors whom you can view as role models you can think of among your classroom peers. Someone who’s doing better than you when you’re matched with them maybe you perform better. That doesn’t happen to be the case here. There’s other research that shows that in some cases it does work. Here we find that that actually is the negative impact, and the positive impact comes when you’re paired with someone who’s not as good as you.

The point that you made about mentors also reminds me of my work with Juni Singh, who’s also on the market this year. She’s a postdoc at Caltech and our work in rural Nepal, where we are studying where the network ties can be used to improve entrepreneurship among women in rural areas. Interestingly, there we find that—and these are tentative results, but we find that—when the person is matched with someone who’s central in the network, more central than them and happens to be a friend, that’s where the positive impacts are concentrated.

That’s where we think some of this motivational role that you matched with someone who’s central in the network, who’s very important in the network, and you matched with them and you attend the training with them, and then you have better outcomes, that could be something to do with aspirations and to do with mentorship. Take that to a classroom setting, and things are completely different. I think just how heterogeneous all of these things are. It’s super interesting.

RAJAGOPALAN: I would actually be even more specific and say take that to this classroom setting and it’s different, because these things are highly specific to where you’re running the experiment. This line of research is fascinating. Thank you so much for sharing this with me. This was a lot of fun.

KHANDELWAL: Thank you. This was a lot of fun for me too. Thank you so much.