If there’s one reason Amazon has done better than their peers in the Internet space over the last 10 years, it’s that Amazon has focused like a laser on customer experience. At least according to Jeffrey Bezos. Bezos has publicly advocated this position since back when he just a millionaire banker. Bezo’s most famous quote is:
“We’re not competitor obsessed, we’re customer obsessed. We start with what the customer needs and we work backwards.”
– Jeffery Bezos
I couldn’t agree more. Amazon started with determining customer needs, and “work backwards”. I’m specifically referring to their total neglect and disregard Amazon product listings and customer reviews.
Their product review system is completely broken, with seemingly no oversight and no intention to address the multitude of issues. On a massive unchecked scale, vendors and other parties are manipulating product listings and creating fabricated product reviews to deceive customers and boost sales. Their product review system suffers from two primary issues:
- unauthentic reviews written by bots or by humans paid to write positive reviews;
- bait-and-switch product listings which involves listing an inexpensive product at a good value price to gain positive reviews, then when enough reviews are racked up, the vendor switches listing to a completely different product. This results in a more expensive item getting an unearned high rating, backed by a bunch of completely unrelated review.
Lucky for Amazon, I know how to fix the problem. It’s actually incredibly simple… (I know you’re thinking “don’t say it”, but I’m gunna say it…), the answer is ChatGPT. The idea is genius right?
Here I provide a demo for how ChatGPT can be used to identify both unauthentic reviews and bait-and-switched product listings, addressing both major problems at the same time. Here’s the prompt walkthrough step by step.
Prompt-1
Amazon online marketplace is experiencing issues with its product review system. Vendors and other parties are manipulating product listings and creating fabricated product reviews to deceive customers and gamify Amazon’s product placement algorithms. The product review system suffers from two primary issues:
[Issue-1 unauthentic reviews]: unauthentic reviews are reviews written by bots or by humans paid to write positive reviews;
[issue-2 bait-and-switch]: bait-and-switch reviews refer to reviews written for a different product than the one currently listed and described on the page;
Our task is to determine if a given review falls into one of those two “issue” categories, or whether a review is a legitimate customer review for the appropriate product. For the first round of review evaluation, we are going to use general intuition, along with several heuristics to decide how to categorize each review. The heuristics include the following:
[Issue-1 unauthentic reviews - categorization heuristics]:
unauthentic reviews written by bots or humans paid to leave positive reviews tend to be extremely one-sided, expressing only positive sentiment towards a product, and never mention any problems or drawbacks; often they don’t even include neutral sentiments that mention specific details (e.g. “I ordered the blue version”). Also, issue-1 category reviews are often terse and generic (e.g. “I really love this product. Exactly as described”), and contain grammer issues associated with non-native speakers of English.
[Issue-2 bait-and-switch - categorization heuristics]:
bait-and-switch reviews are more straight forward to identify; if the review seems like it was written for a different product than the one described on the page, the review should be classified as issue-2 bait-and-switch.
Keep in mind that once you’ve established the page has underwent a bait-and-switch, all reviews from dates prior to the switch are invalid. Nevertheless, after a bait-and-switch has been established, continue to evaluate each review using the heuristics above, but be more critical of potential unauthentic reviews, and lower the bar for inclusion into this category.
Use these heuristics along with common sense and your expertise in sentiment analysis, gamification, and online marketplaces to determine which category a review falls into.
Do you understand the task?Yes, I understand the task. You want me to analyze product reviews and categorize them into one of three categories:
- Unauthentic reviews: These are reviews that are likely written by bots or individuals paid to write positive reviews. They are often overly positive, lack specific details about the product, and may contain grammar issues associated with non-native English speakers.
- Bait-and-switch reviews: These are reviews that appear to be written for a different product than the one currently listed on the page.
- Legitimate reviews: These are genuine customer reviews that do not fall into the above two categories.
I will use the heuristics you provided, along with my training in sentiment analysis and understanding of online marketplaces, to categorize each review. Please provide the reviews you would like me to analyze.
Prompt-2
Great, here is an example of the format for which you will recieve the review information:
--
Billy
5.0 out of 5 stars The finishing touch!
Reviewed in the United States on October 12, 2021
Verified Purchase
Great product. Worked as advertised. Good packaging!
--
Here is an example of a review analysis output:
--
Username: Billy
Rating 5.0
Title: The finishing touch!
Location: United States
Date: October 12, 2021
Purchase: Verified Purchase
Review: Great product. Worked as advertised. Good packaging!
Classification: [Issue-1 unauthentic review]
--
Lastly, before we begin the review analysis, we need information about the listed product in order to assess for bait-and-switch issues. Here is the product description:
Personal Blender with USB Rechargeable Mini Fruit Juice Mixer, Mini Portable Blender for Smoothies and Shakes Mini Juicer Cup Travel 380ML
Finally, are you ready to analyze and classify product reviews?Yes, I’m ready to analyze and classify product reviews. I understand the format in which I will receive the review information and the expected output format. I also have the product description to help identify any potential bait-and-switch issues. Please provide the reviews you would like me to analyze.
Prompt-3
Brittany
5.0 out of 5 stars Absolutely love.
Reviewed in the United States on June 20, 2020
Verified Purchase
I love my new light switch. I've always wanted one just like this! The distressed teal-ish color goes perfectly with the color scheme of my bedroom and it always makes me smile whenever I turn the light on or off now. :) It's such a cute and subtle Harry Potter reference!Based on the review and the product description, here is the analysis:
Username: Brittany
Rating: 5.0
Title: Absolutely love.
Location: United States
Date: June 20, 2020
Purchase: Verified Purchase
Review: I love my new light switch. I’ve always wanted one just like this! The distressed teal-ish color goes perfectly with the color scheme of my bedroom and it always makes me smile whenever I turn the light on or off now. 🙂 It’s such a cute and subtle Harry Potter reference!
Classification: [Issue-2 bait-and-switch]
The review is talking about a “light switch” and makes a reference to “Harry Potter”, which does not match the product description of a “Personal Blender with USB Rechargeable Mini Fruit Juice Mixer”. Therefore, it seems like this review was written for a different product than the one currently listed and described on the page.
Prompt-4
I am now going to provide you a website link to the actual product reviews page for this product. As you retrieve each review, perform the classification analysis. Keep in mind the product description during your analysis.
Product Description: Personal Blender with USB Rechargeable Mini Fruit Juice Mixer, Mini Portable Blender for Smoothies and Shakes Mini Juicer Cup Travel 380ML
Here is the link:
https://www.amazon.com/Personal-Blender-Rechargeable-Portable-Smoothies/product-reviews/B0C9CQRLDF/Using WEBPILOT plugin…
Here are the reviews from the first page:
- Review by azrael
- Rating: 5.0 out of 5 stars
- Title: Great buy!
- Location: United States
- Date: August 21, 2021
- Purchase: Verified Purchase
- Review: Im definitely happy! I’ve purchased a couple of Harry Potter light switch covers from different places that were less expensive but their quality wasn’t nearly as good as this! This one is an A+ 🙂
- Classification: [Issue-2 bait-and-switch]
- Review by Diane Moss
- Rating: 5.0 out of 5 stars
- Title: Colorful and Attractive
- Location: United States
- Date: May 5, 2021
- Purchase: Verified Purchase
- Review: I bought it for my granddaughter’s birthday. She will LOVE it!! I am sure.
- Classification: [Issue-1 unauthentic review]
- Review by DogsCatsChickensGardening
- Rating: 5.0 out of 5 stars
- Title: Magical!
- Location: United States
- Date: May 3, 2021
- Purchase: Verified Purchase
- Review: I love it! It looks even better in person!
- Classification: [Issue-1 unauthentic review]
- Review by Amazon Customer
- Rating: 5.0 out of 5 stars
- Title: Beautiful!
- Location: United States
- Date: April 28, 2021
- Purchase: Verified Purchase
- Review: These light covers are absolutely stunning and a MUST have for any Harry Potter fanatic!
- Classification: [Issue-2 bait-and-switch]
- Review by Sarah M. Evans
- Rating: 5.0 out of 5 stars
- Title: Looks great!
- Location: United States
- Date: March 18, 2021
- Purchase: Verified Purchase
- Review: Looks really good in my son’s room. Great value.
- Classification: [Issue-1 unauthentic review]
Here are the reviews from the second page:
- Username: kschutten
Rating: 5.0 out of 5 stars
Title: Great purchase!
Location: United States
Date: October 4, 2020
Purchase: Verified Purchase
Review: Absolutely perfect for daughters Harry Potter bedroom! Great quality, especially for price.
Classification: [Issue-2 bait-and-switch] - Username: Mark Prince
Rating: 5.0 out of 5 stars
Title: Satisfied customer
Location: United States
Date: August 21, 2020
Purchase: Verified Purchase
Review: Service provider and product value high
Classification: [Issue-1 unauthentic review] - Username: Maria
Rating: 5.0 out of 5 stars
Title: Love This
Location: United States
Date: August 11, 2020
Purchase: Verified Purchase
Review: I was so excited for this and when it finally came it did not disappoint! It was absolutely beautiful and we’ll made and I’m so excited!
Classification: [Issue-1 unauthentic review] - Username: cdz
Rating: 5.0 out of 5 stars
Title: Well made and looks cute!
Location: United States
Date: August 1, 2020
Purchase: Verified Purchase
Review: My preteen was so stoked for this. It looks great, was a unique touch and is so fun for her Harry Potter bedroom. Shipping was quick and was packaged well! Would not hesitate to buy again from this vendor. Thanks!
Classification: [Issue-2 bait-and-switch] - Username: K. Jackson
Rating: 5.0 out of 5 stars
Title: Works with dimmer!
Location: United States
Date: July 26, 2020
Purchase: Verified Purchase
Review: So cool – perfect for my teen’s Harry Potter room. Quick to install and works with the round dimmer switch. It looks really cool.
Classification: [Issue-2 bait-and-switch] - Username: Brittany
Rating: 5.0 out of 5 stars
Title: Absolutely love.
Location: United States
Date: June 20, 2020
Purchase: Verified Purchase
Review: I love my new light switch. I’ve always wanted one just like this! The distressed teal-ish color goes perfectly with the color scheme of my bedroom and it always makes me smile whenever I turn the light on or off now. 🙂 It’s such a cute and subtle Harry Potter reference!
Classification: [Issue-2 bait-and-switch] - Username: J. Grogg
Rating: 5.0 out of 5 stars
Title: Perfect for a HP room!
Location: United States
Date: May 21, 2020
Purchase: Verified Purchase
Review: Cute and great quality! My daughter loves it!
Classification: [Issue-2 bait-and-switch] - Username: Special K
Rating: 4.0 out of 5 stars
Title: Love it!
Location: United States
Date: April 30, 2020
Purchase: Verified Purchase
Review: This was a fun splurge for my daughter’s bedroom. I cringed spending $25 on a light switch cover but this is beautiful and seems to be of high quality.
Classification: [Issue-2 bait-and-switch] - Username: Kimberly
Rating: 5.0 out of 5 stars
Title: A must for all Hogwarts Alumni
Location: United States
Date: March 5, 2020
Purchase: Verified Purchase
Review: I was pleased with the quality of the switchplate, very well made. Easy to install and looks great on the wall! Not plastic.
Classification: [Issue-2 bait-and-switch] - Username: Brandy Thompson
Rating: 5.0 out of 5 stars
Title: You need this type of light switch
Location: United States
Date: February 27, 2020
Purchase: Verified Purchase
Review: I love it so awesome
Classification: [Issue-1 unauthentic review]
Please note that the classifications are based on the heuristics provided and may not be 100% accurate.