We make only one point in this article. Every quantitative study must be able to answer thequestion: what is your estimand? The estimand is the target quantity—the purpose of thestatistical analysis. Much attention is already placed on how to do … Continued
The rise of many-analysts studies has highlighted substantial variability in research outcomes when different teams independently analyze the same datasets and hypotheses. This paper makes two key contributions to this emerging field. First, we demonstrate how the many-analysts framework can … Continued
The potential of using Large Language Models (LLMs) themselves to evaluate LLM outputs offers a promising method for assessing model performance across various contexts. Previous research indicates that LLM-as-a-judge exhibits a strong correlation with human judges in the context of … Continued
AI-based interactive assistants are advancing human-augmenting technology, yet their effects on users’ mental and physiological states remain under-explored. We address this gap by analyzing how Copilot for Microsoft Word, a LLM-based assistant, impacts users. Using tasks ranging from objective (SAT … Continued
Recent gains in popularity of AI conversational agents have led to their increased use for improving productivity and supporting well-being. While previous research has aimed to understand the risks associated with interactions with AI conversational agents, these studies often fall … Continued
Agents built on large language models (LLMs) have excelled in turn-by-turn human-AI collaboration but struggle with simultaneous tasks requiring real-time interaction. Latency issues and the challenge of inferring variable human strategies hinder their ability to make autonomous decisions without explicit … Continued
Artificial Intelligence (AI) systems have historically been used as tools that execute narrowly defined tasks. Yet recent advances in AI have unlocked possibilities for a new class of models that genuinely collaborate with humans in complex reasoning, from conceptualizing problems … Continued
Using AI to create autonomous researchers has the potential to accelerate scientific discovery. A prerequisite for this vision is understanding how well an AI model can identify the underlying structure of a black-box system from its behavior. In this paper, … Continued
There is significant discussion about the opportunities and issues associated with the use of artificial intelligence in higher education. However, issues, such as academic integrity and privacy, continue to dominate the conversation. This has limited how well instructors and higher … Continued
AI writing assistants powered by Large Language Models (LLMs) are increasingly used to make autocomplete suggestions to people as they write text. Can these AI writing assistants impact people’s attitudes in this process? In two large-scale preregistered experiments (N=2,582), we … Continued