AI Inferences and the Big Tech Reason Retailers Know More than They’re Telling You

If you own a smartphone, companies know everything about you. 

A DeleteMe engineer co-built an AI system called Priva-See that showed what ad tech can learn from your phone, and 465 people tested it. The result? With just a small subset of personal data, an AI can infer your mental health, legal issues, medical conditions, political beliefs, your family, finances, and much more. 

Most of this kind of inference happens out of sight. Amazon’s new “About you” feature is an unusual exception, and it’s the one making headlines. The retailer now shows customers some of what it has collected about them, including details about their household, interests and hobbies, product preferences, and even their appearance. 

The truth is, Amazon is only showing you what it chooses to. It’s a cartoon version of what’s possible. Here’s everything you need to know about Amazon, Priva-See, and the data grift that’s exposing your personal information to retailers, governments, AI companies, and anyone else who’s willing to pay. 

Contents: 

1. What Does a Retailer Know? 
2. What Your Phone Knows Can Hurt You
3. What Priva-See Didn’t Show
4. Who’s Regulating AI Inferences? 
5. Starving the AI Inferencing Infrastructure
6. Final Thoughts

What Does a Retailer Know? 

Amazon’s “about you” feature hit headlines after one user went viral saying the feature called her out for having “flat buttocks.” Another showed that Amazon concluded she “had no friends.” You can find the feature yourself at Profile > See all options > Shopping preferences > About you if you’re curious. 

The information seems pretty basic or even silly, until you add AI inferences into the mix. 

AI inferences are conclusions that AI can draw from seemingly unrelated or benign data points. As an example, we used the following prompt in Gemini 3.6. 

“What conclusions might a commercial AI model be able to draw from these three data points based on a user’s purchasing history with a particular retailer: “Product Preferences – selects loose-fitting, comfortable clothes,” “prefers natural, oil-based bar soaps,” “consistently repurchases the same personal care products.”

According to Gemini, a commercial AI can infer anything from “low price-elasticity for core personal care” to the possibility of sensitive skin conditions like eczema. Low-price elasticity means you’ll be willing to pay even if the price goes up. That’s just from three data points on an Amazon profile that can include 80+ pieces of information. 

When retailers have your smartphone data, things get even more gnarly. To see just how scary AI inferences can get, we need to look at another, much more revealing experiment. 

What Your Phone Knows Can Hurt You

The Priva-See experiment started with a research study called “Privacy Leakage Through AI-mediated Analysis of Smartphone Data,” published September 22, 2026. Seven researchers developed the Priva-See app and the study, including:

• Sarah Radway – Harvard University
• Zoe Robert – DeleteMe Engineer
• Matthew Soto – Carnegie Mellon University
• Julianna Cimillo – Unaffiliated
• Sebastian Diaz – Harvard University
• Meg Marco – Harvard University
• James Mickens – Harvard University 

In their own words, they “built Priva-See, an LLM (large language model)-based inference system for app-collected user data; Priva-See reflects our best understanding of how real-life adtech companies would leverage machine learning to build user profiles.” 

465 people participated in the Institutional Review Board (IRB)-approved user study and deployed Priva-See on their phones. The app had access to the photo library, calendar, contacts, and location data. 

For reference, Instagram requests the following permissions: 

1. Calendar
2. Photos and videos
3. Precise location
4. Contacts

And several more. 

The experiment revealed just how privacy-invasive inferences can be, even with a relatively small amount of user data. 

In the words of one of the researchers, Zoe Robert, who also happens to be a software engineer at DeleteMe, “The data used in our study is a tiny fraction of what the ad tech ecosystem has access to. The inferences in our study likely represent the floor of what ad tech is capable of today.”

Researcher Sarah Radway added, “Real-world organizations also have much more sophisticated models, more compute, and the ability to run analyses over far greater time spans than the ~3 minutes our analysis took.”

Here’s what else the experiment uncovered.  

1. Many inferences were accurate

Despite the relatively small amount of data, nearly half of the inferences were measured as accurate, while another 17% were measured as somewhat accurate. 18% were extremely inaccurate. 

2. Many inferences were sensitive

Of the inferences that were accurate, many were highly sensitive. 

Categories of inferences that were particularly sensitive included: 

• Mental health
• Reproductive health
• Legal issues
• Sexual orientation
• Substance abuse
• Medical conditions
• Religion
• About children

Among others. 

Imagine these data points in the hands of an employer, the federal government, or marketers. 

3. Inaccurate inferences could be just as damaging

Of the inaccurate inferences produced through Priva-See, the most commonly mistaken data point was income level. 

Income level is one of the data points retailers can use to target some demographics with higher prices. In some cases, wealthier customers actually obtain lower prices. In other cases, less well-to-do consumers may get lower prices. A wrong income assumption could help or hurt, depending on the circumstance. 

Another common mistake was the AI picking up a clue about the wrong person, usually someone close to the user. 

Radway said, “If an inference applies to your spouse or child, it is still effective to target you with an ad.”

For other types of incorrect assumptions, Robert explained, “Any system that involves personalization or automated decision making outside of your control can lead to unfair or biased outcomes.” 

Radway added, “An incorrect inference, even if incorrect, can likewise be used for discriminatory purposes via algorithmic discrimination. For example, prior researchers found significant differences in exposure to employment and housing-related advertisements along racial and gender lines, resulting from algorithmic discrimination.”

4. Inferences can target family members as well

In many instances, inferences produced by the LLM were not about the user who had agreed to install Priva-See, but rather someone close to them, a family member, or a child. 

Robert said, “Privacy has network effects. So many of the inferences our system made were about people close to the user (child, friend, spouse, coworkers, anyone in their contact list), and not the device owner themselves. Your privacy choices impact the safety and security of others.”

In other words, every time a user gives an app permission to access contacts, calendar information, or other data, there’s a chance that someone else’s privacy is violated in the process. 

5. Photos often produced sensitive inferences

Priva-See works within very specific guidelines. It can search through photos but only select ten as the basis of its inferences. 

The LLM was configured to search specifically for photos that might contain sensitive or identifying information, especially screenshots. 

ID card photos, screenshots of betting apps, and even screenshots of login credentials, if compromised, could lead to unwarranted data exposure, account takeovers, and identity theft. 

How many apps on your phone have access to your photos right now?

6. Informed users changed their behavior

The researchers involved in the Priva-See experiment concluded that “the experience significantly impacted participant willingness to share permissions data moving forward.” 

Once users understand that a screenshot can expose their entire life, they’re less likely to give an app access to their photos. Once they see that retailers can use their location to target them with higher prices, they’re less likely to want to share precise location data with every app that asks for it. 

“When you grant an app access to a full permission type (i.e. all photos), you are granting permission to access that data until you revoke that permission. Mobile privacy “hygiene” is important! Check your system settings regularly and remove permission access to apps that no longer need that permission type,” said Robert. 

How will you change your app permissions after reading this? 

What Priva-See Didn’t Show

The Priva-See experiment was valuable. As Robert explained, “Most people do not understand the capabilities of AI-powered inference systems. This inhibits their ability to make informed privacy and security decisions about sharing their data. Showing people sensitive inferences made using their actual data in a safe, privacy-preserving way is a really effective tool for privacy education.”

Radway said, “There is minimal transparency into how advertising companies can use our data. We hope to raise awareness of the sensitive, accurate, and invasive nature of inferences that can be made using the data you share online, and contribute to the development of privacy education tooling to help users protect themselves online.”

But like the Amazon example, Priva-See only scratches the surface of AI inference risks and the type of data AI companies hold. After all, the data economy that buys and sells your data to anyone who wants to pay, including AI companies, isn’t limited to an app on your phone or to a single retailer.

It’s also thriving thanks to data brokers, who trade in increasingly sensitive data and feed that data directly into the AI ecosystem. 

In California’s data broker registry of just over 600 data brokers: 

• 115 collect precise location data
• 42 collect IDs and SSNs
• 28 sell to law enforcement
• 55 sell to the federal government
• 26 sell to foreign actors
• 32 sell data directly to AI development companies

That’s all self-reported, so the real numbers for each category may be higher. 

The number of data brokers that exist today is certainly much higher than just 600. Most operate under the radar, without any kind of oversight. 

They can do much more than tell Amazon that you don’t have any friends. 

Which of those sensitive inferences data brokers are all too happy to sell might end up in the hands of a cybercriminal who uses AI, and how could they target you if that happened? 

Who’s Regulating AI Inferences? 

You may wonder, why isn’t someone stepping forward to regulate the kind of inferences AI companies can collect and share? 

The short answer is that existing laws mainly address what kinds of information data brokers and other covered companies can collect and sell, not what kinds of information they or other entities can infer from non-sensitive data points. 

Robert said, “We have laws in the US that protect medical data from first party sources, but so much of our medical history can be inferred. When you share your full contact list with WhatsApp, you are also sharing information about any doctors in your contact list (specialty, location, etc) with Meta. When you share your full camera roll with Instagram, you are also sharing any screenshots you might have of prescription labels, test results, and so on.”

Going back to the earlier example from Amazon, that means these laws won’t allow a data broker to sell the following data point: “this user has eczema.” But existing laws typically will allow a retailer or data broker to share: “This user selects loose-fitting, comfortable clothes, prefers natural, oil-based bar soaps, and consistently repurchases the same personal care products.” 

The fact that an individual with a painful skin condition can then be targeted with higher prices for basic necessities is neither here nor there under most existing state laws. 

Maryland recently banned surveillance pricing for food retailers and third-party delivery services, which means any of those entities doing business in the state can no longer set prices based on personal data. It’s still too narrow a ban and holds limited penalties that are unlikely to significantly affect the biggest players in surveillance pricing. Other states are even further behind.

New AI regulations in progress still don’t go far enough in addressing the specific harm of modern AI inferences, which are more powerful than any kind of data processing capability adtech companies may have utilized in the past. 

Radway shared, “LLM-based inference pipelines are able to produce more specific, invasive, and accurate inferences than preceding inference systems, and represent an increased risk of harm over prior systems.”

If regulators won’t protect you, you have to take the first step. 

Starving the AI Inferencing Infrastructure

When your data falls into the wrong hands, it’s a weapon. Your best bet is to starve the AI of the disconnected data points it uses to draw sensitive conclusions. 

1. Audit and restrict app permissions.

Regularly review permissions on your mobile devices (especially for storage/photos, calendar, precise location, contacts, and microphone). Revoke access for any app that doesn’t need it to perform its primary function.

2. Opt out of data brokers and adtech networks. 

Use opt-out tools and platforms or data removal services to scrub your personal information and location histories from data broker databases. Alternatively, use our DIY opt-out guides. 

3. Request and audit your retailer profiles. 

Submit Data Subject Access Requests (DSARs) to major retailers. A DSAR is a formal request asking a company to share with you exactly what personal data it holds about you. Most major retailers have an option under their privacy policy page to make these requests. Companies usually have about a month to respond. Delete any stored data points where the option exists. 

4. Don’t correct inaccurate retailer profile data. 

Resist the urge to correct or fix false inferences by giving away even more of your data. For example, Amazon asks you to improve its recommendations by correcting its “about me” conclusions. Feeding correct data into these profiles enables further misuse of your personal information. 

5. Don’t link accounts and ecosystem apps. 

Don’t connect your retail accounts to smart home devices (like Alexa), payment apps, or social media logins. Cross-linking allows AI models to aggregate scattered data points into a single profile.

6. Decline store loyalty programs and in-app tracking. 

Skip retailer loyalty apps or store-specific reward cards whenever possible, especially for sensitive purchases. These programs trade minor discounts in exchange for comprehensive tracking of your purchasing frequency, basket contents, and exact store visit times.

7. Advocate for stronger protections

Push back against retailers that engage in more and more data collection and heavier AI usage for “personalization.” See if there’s a more privacy-respecting alternative. Reach out to lawmakers to ask for AI regulations that specifically address AI inferences. 

Final Thoughts

AI overreach won’t change because of a person or a company. It has to become a movement that sees what Amazon and Priva-See revealed and decides enough is enough. Get educated and get out there to advocate for yourself. You deserve better. 

Read More: 

1. Read our full breakdown on surveillance pricing. 
2. Discover how AI tools like ChatGPT are facilitating shopping scams
3. See how ads in ChatGPT may impact your privacy.