Which? Tested Boots’ AI Skin Scanner: 37% Got Results That Didn’t Match Their Skin

You know the drill. Open the app, take a selfie under the bathroom light, wait three seconds and receive a verdict on your pores from a robot. AI skin scanners are now everywhere, from the Boots website to the beauty hall, and they are very good at sounding confident. The question is whether they are right.

Which? decided to find out. The consumer champion asked 30 people to test the Boots Skincare Diagnostic Tool, running more than 120 scans and tracking hundreds of data points in early September. The results, published on 15 October, were not flattering. And shortly after Which? shared its findings with Boots, the tool vanished from the retailer’s website.

What Which? Actually Did

Which? started by counting. It found 17 online AI skin-scanning tools across retailers like Boots and Superdrug and brand sites including La Roche-Posay, L’Oréal and Neutrogena. Crucially, at least seven of them, Boots included, run on the same white-label engine, built by Finnish beauty-tech company Revieve, which launched the Boots tool in October 2022 with a promise of analysing “around 20 main metrics” and more than 200 sub-metrics. Some big brands, such as L’Oréal with its ModiFace framework, use their own.

So Which? focused on the Boots version, which it described as “the most mainstream option, run by the most common AI skin analysis platform”. Each tester did two back-to-back scans in good lighting, a third where they deliberately gave wrong answers on the pre-scan questionnaire, and a final scan in dim or patchy light.

The Headline Number: 37 Per Cent

The good news first. In good lighting, the tool is consistent with itself. Scores barely moved between the first and second scans, “by less than a single point on average”, and only four of the 30 testers saw a swing of five points or more.

The bad news is that consistency is not the same as accuracy. “All in all, about 63% of testers received scores that matched what they saw in the mirror, while 37% got results that completely mismatched their actual skin concerns,” Which? reported.

The misses were specific. Of 14 people with visible acne and breakouts who listed them as their main concern, the scanner “failed to spot this or directly contradicted them 10 times”. Nine testers had dark spots; the app rated their pigmentation as perfectly healthy six times. And of seven testers with redness, “not a single one had redness flagged as a concern, and three were even awarded top health scores for clear skin.”

The tool also appeared to perform worse on darker skin. Which? found it “struggled to detect hyperpigmentation and dark spots on participants with darker skin tones (Fitzpatrick Types IV and V), giving them higher overall health scores on average than those with lighter skin types.”

Lighting Changed Everything

If you have ever wondered why your scan results differ from one bathroom to the next, here is your answer. For about half the group, “simply shifting light sources caused overall skin scores to swing wildly, jumping up to +39 points or dropping by as much as -25 points.” Even moving the light from the right to the left of the face noticeably changed the result. Lighter skin tones saw the biggest swings.

The Questionnaire Problem

Then there is the bit most of us skim. Which? found the pre-scan questions “don’t change your overall score much, but they massively skew category-specific ratings” and, more to the point, the shopping list. Twenty-five out of 30 testers “got completely different morning product lists just by tweaking their pre-scan answers.”

In one case, a tester with clear skin, correctly rated as having little to no acne, simply ticked spots as a concern on their third scan. The tool then gave them a low acne score, as if the breakouts had materialised on request. In another, a tester with rosacea was recommended products containing retinol and salicylic acid, which Which? noted “could irritate their skin”.

That tester put it bluntly: “I’m amazed. I have rosacea and am recovering from a bit of an outbreak, so I have some spots. I said spots and blemishes were my main concern and that I had sensitive, dry skin. Yet I have scored highly for spots and been recommended products that contain things like salicylic acid and retinol, which I think would make my skin so much more irritated. I guess this tool is not for people with skin conditions, as I have obviously visible rosacea!”

Which? also clocked a pattern in the recommendations themselves. L’Oréal and Cosrx products “were suggested far more frequently than other brands – 214 and 154 times respectively – while other popular brands like Olay and La Roche-Posay barely showed up.” When it asked Boots how products are chosen, Which? says the retailer “didn’t answer our specific questions but did inform us that the Skin Diagnostic Tool was no longer available online.”

What The Dermatologist Says

Consultant dermatologist Dr Emma Wedgeworth gave Which? a measured take. “There is currently an overwhelming choice of skincare, so tools like this can be helpful to tailor your skincare to suit your own skin,” she said. “However, it is important not to confuse these sorts of diagnostic tools with medical diagnosis. Parameters like radiance or dullness and even skin typing – normal, combination and oily – are not recognised in clinical dermatology. There is no doubt that AI will continue to evolve, but for now, if you are suffering from a skin condition, it’s important to seek expert, professional help.”

So, Should You Bother?

Which?’s verdict is cautious: The tools “can be fun to try out and help narrow down your choices”, it says, but “it might not be clear why you are being offered specific products, and when it comes to actual skin health concerns, it’s no replacement for seeing a dermatologist or medical professional.”

If you are going to scan, the advice is simple. Fill in the questionnaire properly, shoot in good, even lighting, and treat the product list as a starting point rather than a prescription. If something is being recommended for a condition you actually have, check it against a human before you add to basket.

Worth remembering, too, that this is one tool and 30 testers. Which? did not test the other 16 scanners, and the white-label engines behind them will keep improving. But with “scan your face” screens multiplying across beauty halls and brand websites, it is a useful reminder that a confident-looking score is not the same thing as a diagnosis.