What happens when our experts go head-to-head with the bots they created?
Margin Geek’s bots never get tired and they are constantly learning with new data. We knew we were better than the code-based scanner tools out there, but we started asking ourselves, are we even better than a human scan?
Introducing the contestants...
Bots aren't perfect, but they do have a few secret weapons.
Margin Geek's Bots:
- Adaptive AI search with 4+ search methods
- Fine-tuned large language model, continuously learning
- Redundant methods to make sure no products are missed
- A little help from the QA team
The bot's true secret weapon is allowing the AI to take a risk on matches it is unsure about, and then getting a QA team to efficiently validate those guesses. That and the built-in redundancy is what drives our high performance.
In-House Experts (Human):
- Experts at Amazon scanning
- Allotted 2 minutes / SKU (standard scanning pace)
Yes, we gave our bot a QA team because that is our offering. We also compare to a human in a real-life circumstance of scanning efficiently. 2 minutes per SKU for a typical 1000 SKUs source list is almost a full work week of scanning (33 hours). Few sellers spend that much time on 1000 rows, and if they do it's given to an offshore resource that is even more rushed, with misaligned incentives.
Three face-offs
We increased the stakes by pitting the AI not just against our own team, but our clients as well.
- Face-off 1: AI vs. our experts
- Face-off 2: AI vs. our clients
- Face-off 3: A simulation of various scenarios
Results:
Across the first two face-off methods, the profit identified increased 25%. We also drove an additional 3% in margin, and if we were to run that trial across 6000 SKUs, we would have saved 100+ hours, but we have better things to do.
In the third face-off method, things get a bit more varied. Many things can happen to profits identified by either contestant (human or AI). A competitor can come into the picture, best seller rank can vary, a price war could get triggered. More subtly, the AI will always lose if the dataset is incomplete or messy. To factor in the unknown is difficult, but our simulation points out that there can be variance around these results.
AI vs. Client Face-Off
One of our clients was sourcing from a direct relationship with a Proctor and Gamble division. This distributor client does >$100M / year in revenue across all channels, and they hired Margin Geek and our business partners as e-commerce experts to list their products on wholesale listings.
Before they had access to Margin Geek’s Product Scan, they manually searched their product list for matches, finding up to 2 ASINs per SKU. With a margin cutoff of 5%, their average margin was 12%. After our bots scanned the same list, we provided 18% margin with over 2x the options - that means, if they had the investment firepower, they could have made 2x more profit.
We cross-checked these profit forecasts with profit guru, just to assure you that our opportunities were stronger, regardless of the tool you use.
AI vs our own searches
Compared to our own in-house experts who were limited to 2 minutes per SKU, we found a similar outcome to our face-off against clients. The AI pair found more than 2-3x more opportunities with the AI, regardless of the margin threshold. Margin was also boosted 4% or more, regardless of how much cash was available to deploy.
Additionally, because there were so many more high margin opportunities, there was also so much more investment potential at the same margin threshold, meaning you could better diversify your SKUs and avoid over-purchasing inventory.
Running simulations tells us the value of more opportunities
Our simulation assumed that a normal source list has product opportunities of different profit margins and volumes. We then compared the margin when one randomly finds 50% of opportunities versus 75% of opportunities. That was repeated for 10 simulation runs.
We found profit increased 25% with 1.5x opportunities, minus any differences in analysis cost.
This was less than in our AI v. client and AI vs. our own searches, and that is probably because our simulation doesn’t factor in the fact that Margin Geek tends to find the opportunities that others can’t find. For example, we can find text matches that other tools can’t. We also see other tools missing on opportunities that don’t have BSRs, yet we have 3 volume methods (e.g., using review count for backup). When other tools don't identify an opportunity, there are less sellers competing, and higher margins as a result.
What should you expect?
Data quality is the biggest barrier for an AI driven scan. If we don’t get good data from you, you can’t expect the AI to be able to deliver great results.
However, if the data is clean, then for the subset of products you send to us, you will get a margin bump if you fall into the following buckets
- You manually review your source lists, and you do so efficiently (i.e., 2 min per SKU)
- You scan SKUs via traditional scanning tools (e.g., UPC scanners).
If you don’t see a margin improvement at all, we’ll happily refund your money if you show us that your manual scan found more matches than we did. After all, we are constantly improving an want to make sure you are satisfied.
