Knowing It All
The credibility dilemma: When acknowledging a (perceived) lack of credibility can make a boast more believable
Kristina Wald, Shereen Chaudhry & Jane Risen
Organizational Behavior and Human Decision Processes, July 2024
Abstract:
People who are judged negatively by others (e.g., as low in competence) often face a dilemma: They may want to self-promote (to improve others’ impressions of them), but worry their claims may not seem believable. We term this type of situation the “credibility dilemma,” and investigate how people can self-promote most effectively in such cases. In particular, we examine the impact of explicitly acknowledging one’s perceived lack of credibility while self-promoting (e.g., “I’m not that smart, but…” or “I know this may seem hard to believe, but…”). Across ten studies, we find that credibility disclaimers improve perceptions of the self-promoter (compared to self-promoting without them) by increasing perceptions of the speaker’s self-awareness and sincerity. In contrast, credibility disclaimers are ineffective (and sometimes backfire) when the speaker is already perceived as credible. Our findings suggest that common advice to avoid drawing attention to one’s flaws may sometimes be unwarranted.
Whose Pants Are on Fire? Journalists Correcting False Claims are Distrusted More Than Journalists Confirming Claims
Randy Stein & Caroline Meyersohn
Communication Research, forthcoming
Abstract:
Do people trust journalists who provide fact-checks? Building upon research on negativity bias, two studies support the hypothesis that people generally trust journalists when they confirm claims as true, but are relatively distrusting of journalists when they correct false claims. In Study 1, participants read a real fact-check that corrected or confirmed a claim about politics or economics. In Study 2, participants read a real report that corrected or confirmed a marketing claim for one of several products. Participants in both studies had higher levels of distrust for journalists providing corrections, perceiving them as more likely to be lying and possessing ulterior motives. This effect held even among corrections consistent with respondents’ prior beliefs (i.e., for claims that participants thought might be false). The results represent a novel reason why people distrust journalists and resist belief correction. We discuss implications for transparency in journalism, and for how journalists frame fact-checks.
The AI Scientist: Towards Fully Automated Open-Ended Scientific Discovery
Chris Lu et al.
University of Oxford Working Paper, August 2024
Abstract:
One of the grand challenges of artificial general intelligence is developing agents capable of conducting scientific research and discovering new knowledge. While frontier models have already been used as aides to human scientists, e.g. for brainstorming ideas, writing code, or prediction tasks, they still conduct only a small part of the scientific process. This paper presents the first comprehensive framework for fully automatic scientific discovery, enabling frontier large language models to perform research independently and communicate their findings. We introduce The AI Scientist, which generates novel research ideas, writes code, executes experiments, visualizes results, describes its findings by writing a full scientific paper, and then runs a simulated review process for evaluation. In principle, this process can be repeated to iteratively develop ideas in an open-ended fashion, acting like the human scientific community. We demonstrate its versatility by applying it to three distinct subfields of machine learning: diffusion modeling, transformer-based language modeling, and learning dynamics. Each idea is implemented and developed into a full paper at a cost of less than $15 per paper. To evaluate the generated papers, we design and validate an automated reviewer, which we show achieves near-human performance in evaluating paper scores. The AI Scientist can produce papers that exceed the acceptance threshold at a top machine learning conference as judged by our automated reviewer. This approach signifies the beginning of a new era in scientific discovery in machine learning: bringing the transformative benefits of AI agents to the entire research process of AI itself, and taking us closer to a world where endless affordable creativity and innovation can be unleashed on the world's most challenging problems.
End-To-End Causal Effect Estimation from Unstructured Natural Language Data
Nikita Dhawan et al.
University of Toronto Working Paper, July 2024
Abstract:
Knowing the effect of an intervention is critical for human decision-making, but current approaches for causal effect estimation rely on manual data collection and structuring, regardless of the causal assumptions. This increases both the cost and time-to-completion for studies. We show how large, diverse observational text data can be mined with large language models (LLMs) to produce inexpensive causal effect estimates under appropriate causal assumptions. We introduce NATURAL, a novel family of causal effect estimators built with LLMs that operate over datasets of unstructured text. Our estimators use LLM conditional distributions (over variables of interest, given the text data) to assist in the computation of classical estimators of causal effect. We overcome a number of technical challenges to realize this idea, such as automating data curation and using LLMs to impute missing information. We prepare six (two synthetic and four real) observational datasets, paired with corresponding ground truth in the form of randomized trials, which we used to systematically evaluate each step of our pipeline. NATURAL estimators demonstrate remarkable performance, yielding causal effect estimates that fall within 3 percentage points of their ground truth counterparts, including on real-world Phase 3/4 clinical trials. Our results suggest that unstructured text data is a rich source of causal effect information, and NATURAL is a first step towards an automated pipeline to tap this resource.
Intelligence Disclosure and Cooperation in Repeated Interactions
Marco Lambrecht et al.
American Economic Journal: Microeconomics, August 2024, Pages 199–231
Abstract:
How does the information on players’ intelligence affect strategic behavior? Game theory, based on the assumption of common knowledge of rationality, does not provide useful predictions. We experimentally show that in the Prisoners’ Dilemma disclosure hampers cooperation; higher intelligence players trust their partners less when playing against someone of lower ability. Similarly, in the Battle of Sexes with low payoff inequality, disclosure disrupts coordination, as higher intelligence players try to force their most preferred outcome. Instead, with higher payoff inequality, behavior changes and higher intelligence players concede. We analyze the reasons for these patterns of behavior.
Fast & slow decisions under risk: Intuition rather than deliberation drives advantageous choices
Aikaterini Voudouri, Michał Białek & Wim De Neys
Cognition, September 2024
Abstract:
Would you take a gamble with a 10% chance to gain $100 and a 90% chance to lose $10? Even though this gamble has a positive expected value, most people would avoid taking it given the high chance of losing money. Popular “fast-and-slow” dual process theories of risky decision making assume that to take expected value into account and avoid a loss aversion bias, people need to deliberate. In this paper we directly test whether reasoners can also consider expected value benefit intuitively, in the absence of deliberation. To do so, we presented participants with bets and lotteries in which they could choose between a risky expected-value-based choice and a safe loss averse option. We used a two-response paradigm where participants made two choices in every trial: an initial intuitive choice under time-pressure and cognitive load and a final choice without constraints where they could freely deliberate. Results showed that in most trials participants were loss averse, both in the intuitive and deliberate stages. However, when people opted for the expected-value-based choice after deliberating, they had predominantly already arrived at this choice intuitively. Additionally, loss averse participants often showed an intuitive sensitivity to expected value (as reflected in decreased confidence). Overall, these results suggest that deliberation is not the primary route for expected-value-based responding in risky decision making. Risky decisions may be better conceptualized as an interplay between different types of “fast” intuitions rather than between two different types of “fast” and “slow” thinking per se.
Recurring suboptimal choices result in superior decision making
Supratik Mondal, Dominik Lenda & Jakub Traczyk
Decision, forthcoming
Abstract:
A vast body of research has indicated that intensified deliberation on choice problems often improves decision accuracy, as evidenced by choices that maximize expected value (EV). However, such extensive deliberation is not always feasible due to cognitive and environmental constraints. In one simulation study and three well-powered fully incentivized empirical studies, using the decision-from-experience task, we found that individuals who maximized EV without time constraints accumulated higher total gain. The trend reversed in the following two studies. Under time constraints, participants who made more suboptimal (or random in terms of EV maximization) decisions earned more money than those who spent more time maximizing EV. By comparing sampling and decision strategies among people with higher and lower statistical numeracy, we found that more numerate individuals made quicker suboptimal choices, resulting in better overall earnings than less numerate individuals. Detailed analysis indicated that skilled decision makers sampled information more rapidly and dynamically. They adaptively relied on varying search strategies, initially focusing on reducing uncertainty and later discovering unobserved outcomes. Finally, adaptive exploration was accompanied by the development of a metacognitive understanding of the task structure and choice environment. Participants who recognized the effectiveness of the random selection strategy earned more rewards. Taken together, these findings suggest that people (especially those with higher numeracy) in time-constrained environment adaptively changed their decision-making strategies and developed a metacognitive understanding of the task structure and decision environment. This resulted in making recurring suboptimal choices that led to superior long-term performance in the decision task.
How Numerical Cognition Explains Ambiguity Aversion
Marina Lenkovskaya & Steven Sweldens
Journal of Consumer Research, forthcoming
Abstract:
Consumers generally prefer precise probabilities or outcomes over imprecise ranges with the same expected value, a bias known as ‘ambiguity aversion.’ We argue that two elementary principles of numerical cognition explain great heterogeneity in this bias, affecting consumer choices in many domains where options are characterized by varying levels of uncertainty (e.g., lotteries, discounts, investment products, vaccines, etc). The first principle, the ‘compression effect,’ stipulates that consumers’ mental number lines are increasingly compressed at greater number magnitudes. This alone suffices to predict ambiguity aversion as it causes a midpoint (e.g., $40) to be perceived as closer to the upper bound of a range (e.g., $60) compared to its lower bound (e.g., $20). Furthermore, as the compression effect distorts the mental number line especially at lower numbers, it follows that ambiguity aversion should decrease around greater numbers. The second principle, the ‘left-digit effect’ causes a range’s relative attractiveness to decrease (increase) disproportionately with every left-digit transition in its lower (upper) bound, thus increasing (decreasing) ambiguity aversion. Due to the overall compression effect, the impact of the left-digit effect increases at greater numbers. We present 34 experiments (N = 10634), to support the theory’s predictions and wide applicability.
When Half Is at Least 50%: Effect of “Framing” and Probability Level on Frequency Estimates
David Mandel & Megan Kelly
Journal of Behavioral Decision Making, July 2024
Abstract:
Expert judgment often involves estimating magnitudes, such as the frequency of deaths due to a pandemic. Three experiments (Ns = 902, 431, and 755, respectively) were conducted to examine the effect of outcome framing (e.g., half of a threatened group expected to survive vs. die), probability level (low vs. high), and probability format (verbal, numeric, or combined) on the estimated frequency of survivals/deaths. Each experiment found an interactive effect of frame and probability level, which supported the hypothesis that forecasted outcomes received by participants were implicitly quantified as lower bounds (i.e., “at least half”). Responding in a manner consistent with a lower-bound “at least” interpretation was unrelated to incoherence (Experiments 1 and 2) and positively related to numeracy (Experiments 1 and 3), verbal reasoning (Experiment 3), and actively open-minded thinking (Experiments 2 and 3). The correlational results indicate that implicit lower bounding is an aspect of linguistic inference and not a cognitive error. Implications for research on framing effects are discussed.
Dynamic inconsistency in great apes
Laura Salas-Morellón, Ignacio Palacios-Huerta & Josep Call
Scientific Reports, August 2024
Abstract:
When presented with the option of either an immediate benefit or a larger, later reward, we may behave impatiently by choosing instant gratification. Nonetheless, when we can make the same decision ahead of time and plan for the future, we tend to make more patient choices. Here, we explored whether great apes share this core feature of human decision-making, often referred to as dynamic inconsistency. We found that orangutans, bonobos, and gorillas tended to act impatiently and with considerable variability between individuals when choosing between an immediate reward and a larger-later reward, which is a commonly employed testing method in the field. However, with the inclusion of a front-end delay for both alternatives, their decisions became more patient and homogeneous. These results show that great apes are dynamically inconsistent. They also suggest that, when choosing between future outcomes, they are more patient than previously reported. We advocate for the inclusion of diverse time ranges in comparative research, especially considering the intertwinement of intertemporal choices and future-oriented behavior.