Airbnb NYC Data Analysis

Creating an enhanced AI-assisted workflow to conduct data analysis on Airbnbs in NYC.

View Project File

Context

Product teams often use insights from product analytics to inform product strategy. This project explores how AI can be applied to improve the quality of insights generated from standard data analysis workflows. The goal is to evaluate how different AI-assisted approaches improve the data analysis performed on an existing dataset. We aim to identify how AI can uncover overlooked patterns, deepen analytical rigor, and produce more actionable insights with minimal human context.

Improving The Data Analysis Insights

We have analyzed a dataset using Python and extracted certain insights. We want to use AI to enhance the data analysis, so we can uncover useful new information. There are three ways in which we can apply AI to accomplish this:

1. Enhanced Insights: We instruct the AI model to review the existing data analysis and draw new conclusions based on the results of this analysis. The goal is to reveal interesting patterns, observations, or details that were initially overlooked.

2. Expanded Analysis: We instruct the AI model to review the existing data analysis and dataset, and generate an updated analysis, using the initial analysis as a reference. The goal is to improve the quality of insights obtained by having the model conduct the analysis using new techniques to expand upon the insights extracted from the existing analysis.

3. Independent Analysis: We instruct the AI model to review the existing dataset and generate a new analysis from scratch. The goal is to improve the quality of insights obtained by giving the model complete freedom to conduct the analysis, without being influenced by the existing analysis.

Note: In all three cases, the AI-generated output will be manually refined as well.

Input Files

We can provide the following files as context for the AI:

• Dataset: The raw dataset.

• Data Analysis: A Jupyter Notebook containing the data analysis, along with details about the dataset, the analysis methodology, data analysis results, visualizations, and insights.

• Overview: A markdown file containing the summary of the data analysis.

Objective

The overall objective is to identify a new reproducible data analysis workflow that can be used by data analysts (or interested stakeholders) to extract insights from similar datasets (via an AI tool or an AI agent). This is a low-context workflow that is meant to explore the insights generated when the AI is given limited context on the data analysis goals (why the analysis is being done) and the analysis methodology (what to analyze and how).

Workflow

Tools: Cursor, ChatGPT

Step 1: Generate Enhanced Insights

Step 1A: Craft a data analysis prompt

We construct an initial prompt with basic requirements. We optimize this prompt using meta prompting with GPT-5. We review and refine the generated prompt as needed. Refer to my meta prompting resource for more details on how to do this.

Step 1B: Conduct data analysis

We run the optimized prompt using GPT-5 to review the existing data analysis and generate new insights. We review and refine the AI-generated output as needed.

Step 1C: Update the data analysis overview

We combine the new AI-generated insights with the original insights, generating a new comprehensive data analysis overview.

Output

• Enhanced Insights Data Analysis

• Enhanced Insights Data Analysis Overview

Note: The initial and optimized analysis prompts, along with the standardization prompt, can be found here.

Step 2: Generate Expanded Analysis

Step 2A: Craft a data analysis prompt

We construct an initial prompt with basic requirements. We optimize this prompt using meta prompting with GPT-5. We review and refine the generated prompt as needed. Refer to my meta prompting resource for more details on how to do this.

Step 2B: Conduct data analysis

We run the optimized prompt using GPT-5 to review the existing data analysis and dataset and generate new insights. We review and refine the AI-generated output as needed.

Step 2C: Update the data analysis overview

We combine the new AI-generated insights with the original insights, generating a new comprehensive data analysis overview.

Step 2D: Standardize data analysis overview

Each approach will generate insights in a certain format. We want to compare the outputs from all three approaches. However, this can be hard to do when the format varies between each approach. Therefore, we use the overview generated in approach 1 (Step 1C) as a baseline and use AI to restructure the output from this approach to match this format, essentially standardizing it.

Alternatively, we could explicitly state the format in our initial prompt. However, this might bias or limit the analysis to the methods used in the existing data analysis. Therefore, we use a prompt to standardize the output, only after the analysis has been completed.

Output

• Expanded Analysis Data Analysis

• Expanded Analysis Data Analysis Overview

Note: The initial and optimized analysis prompts, along with the standardization prompt, can be found here.

Step 3: Generate Independent Analysis

Step 3A: Craft a data analysis prompt

We construct an initial prompt with basic requirements. We optimize this prompt using meta-prompting with GPT-5. We review and refine the generated prompt as needed. Refer to my meta prompting resource for more details on how to do this.

Step 3B: Conduct data analysis

We run the optimized prompt using GPT-5 to review the dataset and generate new insights. We review and refine the AI-generated output as needed.

Note: GPT-5 hallucinated features in the dataset when it had no frame of reference, so these sections had to be removed from the analysis.

Step 3C: Standardize data analysis overview

Each approach will generate insights in a certain format. We want to compare the outputs from all three approaches. However, this can be hard to do when the format varies between each approach. Therefore, we use the overview generated in approach 1 (Step 1C) as a baseline and use AI to restructure the output from this approach to match this format, essentially standardizing it.

Alternatively, we could explicitly state the format in our initial prompt. However, this might bias or limit the analysis to the methods used in the existing data analysis. Therefore, we use a prompt to standardize the output, only after the analysis has been completed.

Output

• Independent Analysis Data Analysis

• Independent Analysis Data Analysis Overview

Note: The initial and optimized analysis prompts, along with the standardization prompt, can be found here.

Step 4: Compare Analysis Insights From All Approaches

We now have AI-generated insights from the dataset using three different approaches. We can now compare the results to answer the following questions:

• How did the quality of insights in each AI approach compare to the original analysis?

• Which approach (1, 2, or 3 yielded the greatest insight into the dataset?

• How did the insights generated vary between the three approaches?

Step 4A: Craft a data analysis comparison prompt

We construct an initial prompt with basic requirements. We optimize this prompt using meta-prompting with GPT-5. We review and refine the generated prompt as needed. Refer to my meta prompting resource for more details on how to do this.

Step 4B: Conduct data analysis comparison

We run the optimized prompt using GPT-5 to review the insights generated from each approach. We review and refine the AI-generated output as needed.

Output

• Data Analysis Comparison

Note: The initial and optimized analysis prompts can be found here.

Conclusion

We explored how AI-assisted approaches can enhance standard data analysis insights under low-context conditions—that is, when the AI receives minimal guidance about analytical goals or methods.

• The Manual Analysis serves as a strong descriptive baseline, capturing clear trends in pricing, availability, and neighborhood differences.

• The Enhanced Insights approach advances this by reframing insights through a product lens, translating findings into concrete actions such as dynamic pricing adjustments and review-based incentives.

• The Expanded Analysis approach further strengthens the analytical rigor by integrating regression and clustering methods, linking quantitative evidence directly to product strategy. It provides the clearest balance between statistical robustness and business relevance.

• The Independent Analysis approach, operating independently of prior analyses, delivers the most original discoveries—especially around host concentration, supply diversity, and professional host behavior—but would benefit from more explicit analytical directives and measurable goals.

Strategic Takeaway

These approaches are complementary when integrated into a unified analytics workflow:

• Use the Enhanced Insights approach for exploratory discovery and identifying new strategic levers.

• Use the Expanded Analysis approach for statistical validation, segmentation, and experiment design.

• Use the Independent Analysis approach for translating validated insights into actionable, stakeholder-facing product initiatives.

This tiered model ensures a balanced framework that combines innovation, analytical rigor, and operational applicability—supporting Airbnb’s ongoing goal to optimize supply quality, pricing strategy, and market performance.