Amplitude, a leader in product analytics, has unveiled its proactive agent, Wave, significantly accelerating the product improvement process. By integrating with OpenAI’s advanced models, Wave can transform analytical data and user feedback into actionable solutions ready for implementation in just 1-2 days, a process that previously took up to a month.
Wave leverages OpenAI’s models, including GPT‑6.1 Sol, GPT‑6 Luna, and GPT‑Image‑2.5 Flare, for 80% of its operations. The agent meticulously analyzes product usage data, user session recordings, customer feedback, and source code to identify and prioritize areas for enhancement.
“AI can now find useful work that we never thought to delegate to it. OpenAI’s models can explore a problem, write and execute code, and then use the results to make decisions about next steps,” stated Spencer Skates, CEO of Amplitude.
From Product Data to Deployed Enhancements
Wave consolidates Amplitude’s analytical data, session recordings, customer feedback, and code to pinpoint and prioritize improvement opportunities. It then develops solutions, coordinates their implementation with development teams and coding agents, and monitors outcomes to inform further refinements.
Amplitude strategically employs different OpenAI models for specific tasks. GPT‑6.1 Sol is used for deep code and data exploration and supports the opportunity manager. GPT‑6 Luna handles frequent, less resource-intensive tasks like code scanning and minor checks. GPT‑Image‑2.5 Flare is utilized for generating product prototypes, using screenshots from session recordings as visual guides.
Once a development proposal is ready, teams can send it directly from Wave to coding agents like Codex, providing a technical blueprint and acceptance criteria to guide the implementation process.
How Wave Drove a 160% Increase in Engagement
“The idea behind Wave is to have a unified platform with agents that review massive amounts of session recordings, analytics, and code to find opportunities for product improvement,” explained Jacob Newman, Principal AI Product Manager at Amplitude.
Amplitude tested Wave on its own homepage, where standard suggestion cards were designed to prompt users to interact with an AI agent. Wave observed that users often overlooked these prompts. In a sample of 100 session recordings, 80 users typed their own questions, while only 12 selected a suggestion card.
Wave proposed personalizing these cards, drawing context from recently viewed charts, dashboards, or the page the user originated from. It generated mockups and a technical plan, ensuring the option for free-text input remained and that generic suggestions were displayed when personalization context was absent. After a brief review, the team handed off the work to a coding agent and conducted an A/B test of the changes.
Experiment Results
According to Amplitude’s reported results, the number of messages sent via suggestion cards on the homepage increased by 160% compared to a control group within just one day. “It’s a great example of how not-so-revolutionary, ignored things can surprisingly boost feature usage,” commented Eric Carlson, Lead AI Engineer at Amplitude.
Selecting the Right OpenAI Model for Every Task
Wave’s diverse tasks impose varying demands on its underlying models. Deep investigation requires strong reasoning and evidence-based conclusions. Small, repetitive checks, performed hundreds of times, must be fast and cost-effective. Amplitude evaluates quality, latency, and cost for each task before assigning the appropriate models.
“OpenAI has provided us with both powerful models for reasoning and smaller, faster models, allowing us to balance quality, latency, and inference cost across the entire product,” said Spencer Skates, CEO of Amplitude.
For Wave’s “Opportunity Investigator,” Amplitude assessed how well models validated findings, avoided false positives, incorporated reviewer feedback, recovered from tool failures, and maintained faithfulness to evidence. GPT‑6.1 Sol achieved an overall score of 4.06 out of 5, outperforming three alternative models which scored 3.62, 3.52, and 2.94. It excelled in all five investigation criteria and demonstrated the fastest median investigation time at 54 minutes.
In a separate evaluation of the Data Science Agent, GPT‑6.1 Sol scored 67.5 out of 100, compared to 69.3 for another leading model, at an average cost of $1.28 per send versus $2.63. GPT‑6.1 Sol also scored higher in adherence to tool contracts, evaluation and uncertainty, and justification of evidence.
Data Science Agent GPT‑6.1 Sol Evaluation
For high-volume tasks like code scanning and minor checks, Amplitude utilizes GPT‑6 Luna to reduce latency and costs. Distributing work among specialized agents allows the team to reserve more expensive computations for tasks that truly require them.
For generating frameworks and prototypes, OpenAI’s image generation model achieved a score of 80.3 out of 100 for solution prototypes, surpassing another image generation model’s score of 63.9. Amplitude’s evaluation considered communication quality, product fit, instruction adherence, text and UI readability, and visual consistency across 15 opportunities.
Maintaining Team Control Over Releases
Clients across various industries are already using Wave in its beta phase to identify and implement product improvements. Wave analyzes session and analytics data, then generates recommendations complete with a proposed fix, a technical plan, and acceptance criteria.
From there, teams can control the level of automation. Most teams send bug fixes directly to coding agents, while new features and larger projects undergo human review.
For changes that reach production, the path from discovery to deployment can be remarkably short: among experiments, features, and guides launched by clients, one in four were deployed the same day Wave identified the opportunity, half were live in less than four days, and two-thirds were implemented within a week.
What’s Next
“Our ambition with OpenAI is to make self-improving products the standard for software development,” said Skates. “As OpenAI’s models get better at reasoning and coding, teams will be able to delegate more exploration and implementation tasks to agents, ultimately leading to faster releases of better products. Amplitude is already seeing this as agents find and help implement improvements in its own product while the team works on other priorities. I think teams will be surprised by how much they can achieve when their software works alongside them to improve the product.”
