Publications
Publications
Show Results For
- All HBS Web
(3,769)
- Faculty Publications (497)
- 2024
- Conference Paper
Quantifying Uncertainty in Natural Language Explanations of Large Language Models
By: Himabindu Lakkaraju, Sree Harsha Tanneru and Chirag Agarwal
Large Language Models (LLMs) are increasingly used as powerful tools for several
high-stakes natural language processing (NLP) applications. Recent prompting
works claim to elicit intermediate reasoning steps and key tokens that serve as
proxy explanations for LLM... View Details
- 2023
- Article
Benchmarking Large Language Models on CMExam—A Comprehensive Chinese Medical Exam Dataset
By: Junling Liu, Peilin Zhou, Yining Hua, Dading Chong, Zhongyu Tian, Andrew Liu, Helin Wang, Chenyu You, Zhenhua Guo, Lei Zhu and Michael Lingzhi Li
Recent advancements in large language models (LLMs) have transformed the field of question answering (QA). However, evaluating LLMs in the medical field is challenging due to the lack of standardized and comprehensive datasets. To address this gap, we introduce CMExam,... View Details
- 2023
- Article
Post Hoc Explanations of Language Models Can Improve Language Models
By: Satyapriya Krishna, Jiaqi Ma, Dylan Slack, Asma Ghandeharioun, Sameer Singh and Himabindu Lakkaraju
Large Language Models (LLMs) have demonstrated remarkable capabilities in performing complex tasks. Moreover, recent research has shown that incorporating human-annotated rationales (e.g., Chain-of-Thought prompting) during in-context learning can significantly enhance... View Details
- 2023
- Other Article
The Harvard USPTO Patent Dataset: A Large-Scale, Well-Structured, and Multi-Purpose Corpus of Patent Applications
By: Mirac Suzgun, Luke Melas-Kyriazi, Suproteem K. Sarkar, Scott Duke Kominers and Stuart Shieber
Innovation is a major driver of economic and social development, and information about many kinds of innovation is embedded in semi-structured data from patents and patent applications. Though the impact and novelty of innovations expressed in patent data are difficult... View Details
- December 2023
- Article
When Should the Off-Grid Sun Shine at Night? Optimum Renewable Generation and Energy Storage Investments
By: Christian Kaps, Simone Marinesi and Serguei Netessine
Globally, 1.5 billion people live off the grid, their only access to electricity often limited to operationally-expensive fossil fuel generators. Solar power has risen as a sustainable and less costly option, but its generation is variable during the day and... View Details
- 2023
- Article
Which Models Have Perceptually-Aligned Gradients? An Explanation via Off-Manifold Robustness
By: Suraj Srinivas, Sebastian Bordt and Himabindu Lakkaraju
One of the remarkable properties of robust computer vision models is that their input-gradients are often aligned with human perception, referred to in the literature as perceptually-aligned gradients (PAGs). Despite only being trained for classification, PAGs cause... View Details
- November 2023
- Case
Open Source Machine Learning at Google
Set in early 2023, the case exposes students to the challenges of managing open source software at Google. The case focuses on the challenges for Alex Spinelli, Vice President of Product Management for Core Machine Learning. He must set priorities for Google’s efforts... View Details
- November 2023 (Revised March 2024)
- Case
Infarm: Betting the (Indoor) Farm on Food Security
By: Elie Ofek
In the summer of 2023, the co-founders of Infarm, a controlled environment agriculture (CEA) company, were contemplating a major pivot going forward. While Infarm had successfully shown it could grow over 75 products—mainly herbs, leafy greens and mushrooms—in modular... View Details
- November 2023 (Revised April 2024)
- Case
Khanmigo: Revolutionizing Learning with GenAI
By: William A. Sahlman, Allison M. Ciechanover and Emily Grandjean
Already a leader in the edtech space since its 2008 launch, Khan Academy was now one of the first edtech organizations to embrace generative artificial intelligence ("genAI"). In March 2023, Khan Academy began beta testing Khanmigo, a genAI “guide” and tutor built with... View Details
- November–December 2023
- Article
Keep Your AI Projects on Track
By: Iavor Bojinov
AI—and especially its newest star, generative AI—is today a central theme in corporate boardrooms, leadership discussions, and casual exchanges among employees eager to supercharge their productivity. Sadly, beneath the aspirational headlines and tantalizing potential... View Details
- November 2023
- Article
Open Source Software and Global Entrepreneurship
By: Nataliya Langburd Wright, Frank Nagle and Shane Greenstein
This is the first study to consider the relationship between open source software (OSS) and
entrepreneurship around the globe. This study measures whether country-level participation on
the GitHub OSS platform affects the founding of innovative ventures, and where it... View Details
- October 2023 (Revised February 2024)
- Case
Loris
By: Shunyuan Zhang, Das Narayandas, Stacy Straaberg and David Lane
In December 2022, Loris’s executive team considered their go-to-market strategy. Loris was an artificial intelligence (AI) software startup for the customer service industry with two products on the market: 1) Agent Assist which provided customer service agents (CSAs)... View Details
- October 2023 (Revised November 2023)
- Case
Recycle & Re-Match: The Future of Soccer Turfs
By: George Serafeim, Lena Duchene and Carlota Moniz
By August 2023, Re-Match, an artificial turf waste-to-value company, had operations in Denmark and the Netherlands and had recycled over 160,000 tons of waste and plastic fiber. With recent capital injection from the VC firm Verdane and a dual revenue business model,... View Details
Keywords: Carbon Emissions; Carbon Abatement; Sustainability; Recycling; Waste Management; Technology; Entrepreneurial Management; Business Growth and Maturation; Business Model; Decisions; Energy Conservation; Investment Return; Profit; Technological Innovation; Patents; Growth and Development Strategy; Market Entry and Exit; Digital Platforms; Wastes and Waste Processing; Business Strategy; Competition; Expansion; Technology Adoption; Sports; Environmental Sustainability; Entrepreneurship; Green Technology Industry; Service Industry; Manufacturing Industry; Rubber Industry; Sports Industry; Denmark; Netherlands; France; United States; Pennsylvania; Europe
- 2023
- Working Paper
Black-box Training Data Identification in GANs via Detector Networks
By: Lukman Olagoke, Salil Vadhan and Seth Neel
Since their inception Generative Adversarial Networks (GANs) have been popular generative models across images, audio, video, and tabular data. In this paper we study whether given access to a trained GAN, as well as fresh samples from the underlying distribution, if... View Details
- 2023
- Working Paper
Causal Interpretation of Structural IV Estimands
By: Isaiah Andrews, Nano Barahona, Matthew Gentzkow, Ashesh Rambachan and Jesse M. Shapiro
We study the causal interpretation of instrumental variables (IV) estimands of nonlinear, multivariate structural models with respect to rich forms of model misspecification. We focus on guaranteeing that the researcher's estimator is sharp zero consistent, meaning... View Details
Keywords: Mathematical Methods
- October 2023
- Article
Improving Regulatory Effectiveness Through Better Targeting: Evidence from OSHA
By: Matthew S. Johnson, David I. Levine and Michael W. Toffel
We study how a regulator can best target inspections. Our case study is a U.S. Occupational Safety and Health Administration (OSHA) program that randomly allocated some inspections. On average, each inspection averted 2.4 serious injuries (9%) over the next five years.... View Details
- 2023
- Working Paper
In-Context Unlearning: Language Models as Few Shot Unlearners
By: Martin Pawelczyk, Seth Neel and Himabindu Lakkaraju
Machine unlearning, the study of efficiently removing the impact of specific training points on the
trained model, has garnered increased attention of late, driven by the need to comply with privacy
regulations like the Right to be Forgotten. Although unlearning is... View Details
- September 2023 (Revised September 2024)
- Case
IBJ, Inc. (A): Seeking Matrimony in Japan
By: Ramon Casadesus-Masanell and Akiko Saito
In March 2020, Shigeru Ishizaka, founder and CEO of IBJ, Inc., Japan's largest marriage matching service provider, faced a critical decision regarding the company’s planned ¥3.5 billion (US$32.8 million) acquisition of competitor ZWEI Co., Ltd. IBJ, founded in 2006,... View Details
- September 2023 (Revised January 2024)
- Case
AI21 Labs in 2023: Strategy for Generative AI
By: David Yoffie, Orna Dan and Elena Corsi
Israeli generative artificial intelligence company AI21 Labs was founded in 2017 to realize the vision of true machine intelligence. It sought to reinvent writing and reading and in 2020 it launched Wordtune, an app using GenAI software to offer alternate text... View Details
- September 2023 (Revised March 2024)
- Case
ReMo Energy: Sizing Up Investors
By: Jeffrey J. Bussgang and Tom Quinn
In 2023, executives with ReMo Energy (founded 2020) were deciding which size ammonia plant to build as their first project. Their innovative model produced ammonia—useful for making fertilizer and for energy storage—from renewable energy, and they had received funding... View Details
Keywords: Factories, Labs, and Plants; Business Startups; Cost vs Benefits; Design; Energy Conservation; Energy Generation; Renewable Energy; Venture Capital; Investment Return; Goods and Commodities; Size; Infrastructure; Risk and Uncertainty; Science-Based Business; Commercialization; Technological Innovation; Chemical Industry; Energy Industry; Green Technology Industry; United States; Boston