Love it or hate it, AI integration has officially become table stakes for retail brands who want to remain competitive. Over the past year, Walmart, Target, Instacart, Ralph Lauren, and other big names have all debuted Gen AI-powered chatbots to assist (and perhaps gently nudge) consumers on their purchase journeys. While the bots are designed in part to personalize the shopping experience and help customers make more informed decisions, these retailers are also betting that the new rollouts will lead to increased conversions. But is that true?

The age of agentic shopping is still in its infancy, and brands will no doubt continue to optimize and refine their AI features as more data becomes available. But as a first stab at gauging just how effective these tools are, we analyzed mobile engagement and conversion data from Amazon’s Rufus during the weeks leading up to Black Friday.

To assess the efficacy of these tools, there are two initial questions that must be answered.

One: are people actually *using *it?

And two: are the people using it *more likely to convert? *Let’s dive in.

Gaining Steam: Rufus Usage Throughout November

At the start of the month, customers using Rufus made up 30% of Amazon app sessions. But as we get closer to Black Friday (which fell on 11/28 this year), we can see the gap gradually shrinking and coming closer to parity. The portion of users employing Rufus peaked on Black Friday itself at 40% — a 33% increase from the 11/1 share. While these figures are not massive, a trend is revealing itself: as consumers get closer to key shopping (and sales) dates, they utilize AI more heavily to sway their purchasing decisions.

It’s possible that increased urgency causes users to outsource some of their decision-making, or that shoppers browsing closer to major sales dates are more likely to be cost-sensitive, thus utilizing chatbots to determine the best deals. Whatever the catalyst, the share of AI visits starts to drop immediately after Black Friday — finishing off the month at 38%.

**Deals, Deals, Deals: Purchases Made With (and Without) Rufus **

People are using Rufus — but is the chatbot actually contributing to real-life conversions? Take a look at this graph that compares the volume of purchases made by mobile users who employed Rufus, to the volume of purchases made by those who didn’t. For analytical purposes we normalized the data, setting November 1st as the baseline for both groups. Rather than looking at absolute numbers, the chart shows relative changes in conversion levels throughout the month. While in actuality the net number of non-AI sessions consistently remains higher, the trend lines displayed here show us the relative rates of growth between the two factions, and the relationship between their up- and downturns.

The subset of customers making purchases through Rufus remains relatively stable through the first half of the month, with some minor fluctuations. Interestingly, the first three local maximas — smaller spikes in activity before the global peak on 11/28 — all fell on Sundays (11/2, 11/9, and 11/16). On these dates, conversions attached to AI visits grew at a faster rate than non-AI visits. This suggests that the boosted sales are not merely a result of generally heightened e-commerce activity on Sundays, but rather something to do specifically with Rufus’s involvement.

We see another peak in AI conversions on 11/20, the first day of Amazon’s Black Friday Week. This is followed by a final mini-spike on 11/23 (another Sunday), before finishing the month off strong with a dramatic 91-point day-over-day increase on Black Friday. Again — not only did AI conversions rise during the aforementioned windows, but they also increased disproportionately relative to non-AI conversions. While the data is preliminary, this initial snapshot shows conversion activity linked to AI clustering around major sales events, as well as weekend days, when most people have more downtime to shop intentionally.

As retailers continue to make their foray into Generative AI, it will be a fierce race to determine the most impactful use cases for the tech. When do people rely on AI to make decisions? *Who *relies on AI to make decisions? And, crucially, are tools like Rufus actually driving sales — or are those most likely to engage with the chatbot already more predisposed to converting (i.e., higher income, or further down the funnel)?

In other words, the jury is still out on how much of Rufus’ contribution can be written off as a spurious correlation, versus how much is truly determinative. Sensor Tower will be keeping a close eye on fast-moving developments in the space. And as ChatGPT’s new shopping research tool picks up steam, we’ll be there with the insights you need to understand its impact.

Until then,** sign up for our monthly email newsletter **to keep up with all the most salient trends affecting the digital economy.