Nowadays nearly every online shop utilizes some sort of product recommendation engine, which is no wonder, as these systems, if set up and configured properly can significantly boost revenues, CTRs, conversions, and other important metrics.
Moreover, they can have considerable positive effects on the user experience as well, which translates into metrics that are harder to measure, but are nonetheless of paramount importance to online businesses, such as customer satisfaction and retention.
What is a recommendation engine, exactly?
Recommendation engines are, at their core, information filtering tools that utilize algorithms and data to recommend the most relevant items to a particular user in a given context.
Here, an item can mean a piece of content, a product, or even a person (in the case of dating sites, for instance). Recommendations can be powered by aggregate data, which determines the relevance of a certain item in relation to a given context, or user specific data, that fuels personalized recommendations.
Naturally, the types of products a certain eCommerce site sells and the kind of audience it caters to have an enormous impact on how recommendations should be used and presented in the store, and also on the kind of logics that work best for them.
Logic is kind of a niche word, although I will use it often in this article. For clarity, by logic, I mean the set of predefined rules and algorithms based on which recommendations are provided (i.e. “Popular products”, “Customers who bought this also bought…”, etc.)
How Recommendation Engines Benefit Sites
One doesn’t need to carry out thorough market research to understand that a customer will be more willing to make a purchase at a shop where they feel they’re getting maximum help in finding what they’re looking for as fast and easy as possible.
They’re also much more likely to return to such a shop in the future. While there’s a lot more to creating this kind of experience than recommendations, implementing a sophisticated solution is without a doubt a cornerstone of a relevant and personalized customer journey.
To get an idea about the business value of recommender systems: A few months ago, Netflix estimated, that its recommendation engine is worth a yearly $1billion for the company.
Another telling example, is when back in 2011 eBay acquired a 2-year-old recommendation engine startup, Hunch for $80 million.
Amazon founder Jeff Bezos understood the value of recommender systems as early as 1998:
In the online world, businesses have the opportunity to develop very deep relationships with customers, both through accepting preferences of customers and then observing their purchase behavior over time, so that you can get that individualized knowledge of the customer and use that individualized knowledge of the customer to accelerate their discovery process.
If we can do that, then the customers are going to feel a deep loyalty to us, because we know them so well.
Now, jump to 2015: according to a study published by VentureBeat last year, 77% of “digital natives” (roughly, generation Y and a huge part of X) expect their online experience to be personalized. This means that personalization and related technologies left the “nice to have” category for good.
Nowadays, employing such techniques is downright a prerequisite of doing business online successfully.
The most famous example is Amazon’s much-cited statistic, according to which 35% of their total revenues are coming from purchases of products that customers found through recommendations.
This is a very important metric when evaluating a recommender system. In our experience, businesses don’t always get it at first, what metrics to consider in relation to the performance of such as system.
Here are a few metrics that can describe recommender performance:
Conversion Rate from Recommendations – Conversion rate for people who clicked on recommendations.
GMV/1000 Recommendations – This is often the hardest to understand at first, but it generally means, the average revenue / 1000 recommendations, that comes from people purchasing products that they’ve found in recommender boxes.
CTRs – While obvious, it is a very important metric to consider. One thing to look out for, though: widgets in unfavorable locations (footer, sidebar) can skew these numbers (and, in fact, all others as well), this should be kept in mind during evaluation.
% of Revenue Through Recommendations – It is one of the most often used metrics (also highlighted in the Amazon example above). This simply means revenues through recommendations / total revenues.
Number of Products Viewed – The number of products viewed by people, who are actively using recommendations during their sessions. While intuitively, more browsing can also mean that users are having a hard time finding what they’re looking for, a study by Wolfgang Digital (cited in more detail later) concluded, that the time spent on the site by a user and the number of pages they open actually correlate with conversions positively.
These are the metrics based on which recommender systems are usually evaluated during POCs and tenders (as an example, see this press release about eBay Turkey’s recommender system tender, in which our company participated in).
It is important to note, that alongside the actual recommender performance, lots of external factors play into these metrics as well, such as widget location, general page layout, seasonal/campaign-related factors, etc.
The Difference Between Recommendations and Personalization
While in the media, the words recommendations and personalization are pretty much used interchangeably, they’re not at all the same thing.
Recommendations can be personalized, but this is by no means the best and only option in each situation.
Also, there’s a lot more to recommendations than personalization, and vice-versa. Personalization cannot be achieved only through recommendation engines (nevertheless, real personalization, at scale, is always powered by a recommendation engine).
The underlying technologies of recommender systems are becoming more and more polished, both technically and in terms of possible business applications.
With the proliferation of SaaS solutions in the field, more and more businesses can leverage the power of sophisticated recommendation systems and stay competitive in the evolving eCommerce space.
Where Can You Use Recommendations?
There are plenty of places where recommendations can be displayed on an eCommerce or content site. To narrow the topic down, in this article, I’ll discuss the basic kinds of logics and recommendation techniques that work best in widgets on the main, product, cart, category, and 404 pages of eCommerce stores.
Main Page Recommendations
The main page is the first thing than users coming from direct traffic see when they visit a site. As these visitors don’t necessarily come looking for anything in specific, main page recommendations serve the purpose of informing customers about your latest deals and discounts, and showcasing your product portfolio.
1. Popular Products
This is a very basic, yet powerful recommendation logic that works splendidly in nearly all eCommerce stores. The popularity of a product is determined most easily by the number of times it has been purchased (weighted with how long it’s been available).
However, more sophisticated recommender systems incorporate other event data into their logics, in order to serve even more accurate recommendations (clicks, views, add-to-cart events, etc.)
Getting this right is extremely important, given that according to the Pareto’s rule in marketing, 80% of product sales come from 20% of products.
In the case of content sites (news sites, video portals) other factors, such as time spent on page or percentage scrolled, or seconds watched (of a video) can also be important popularity factors.
You should always go with the metrics that best define what item in your stock or piece of content on your site is the most popular with your visitors.
2. Rating-Based Recommendations
Another indicator of popularity can be high user ratings.
A bit of a side note here: recommender system jargon calls user ratings, comments, etc., in other words, preference signals users give consciously and willingly, “explicit feedback”.
On the other hand, clicks, purchases, and browsing habits are termed “implicit feedback”, as they represent data points that are generated when a user naturally interacts with the platform or website.
Recommender systems deployed in the real world (not in academia) utilize almost solely implicit feedback, due to the abundance, richness, and structure of such data, as opposed to a review, for instance, which is highly unstructured information, or ratings, which are extremely scarce data (only a small percentage of users write reviews, or even give ratings).
Nevertheless, while machines might not have a lot to benefit from ratings, users do. In fact, According to a survey by BrightLocal, 88% of users trust online reviews as much as personal recommendations.
If you think that due to the habits and preferences of your user base reviews can be a prominent factor in influencing purchase decisions, you should probably consider featuring a “Top rated” box on your main page, or at least highlight the ratings of products in widgets served by different logics throughout your site.
3. Personalized Recommendations
Personalized recommender widgets display different products to each user based on their past purchase and browsing history. Recommendation algorithms used for this task can differ greatly in terms of how they’re implemented exactly and the factors they consider (and often times, “the devil is in the details”, as they say).
Nevertheless, as popular product recommendations are good for catering for the mainstream, one of the main benefits of personalization in product recommendations, is that they can increase the sales of “long tail” items.
However, personalized recommendations require considerable amounts of behavioral data on users, which a system does not have in the case of new visitors, for example (this is called the cold start problem for recommender systems).
In order to provide a personalized user experience for returning customers, but also cater to first-time visitors, the industry best practice is to define a “fallback scenario”, or a sequence of such scenarios, in other words, an automated plan B, for personalized recommendations.
In practice, this means that the system detects whether it has enough data about a certain customer to serve him personalized item suggestions.
If it does not, the recommendation system will “fall back” to a more general (like popular), or category filtered logic, for which it utilizes aggregate or meta-data rather than customer-specific information.
The fallback scenario should be planned with great care, and should utilize the data you have as effectively as possible, in order to provide reasonable accurate recommendations for new visitors as well.
Recommendations on the Product Page
The product page or product information page is where visitors find detailed descriptions of a product and its features and can choose to add it to their cart or order it right away.
The main aim of recommendations on these pages is to display the most relevant items and therefore provide a “next step” in their search to your visitor and keep them browsing your site.
In most cases, the more time they spend in your shop, the higher the chances they’ll actually make a purchase. In fact, in a 2014 study, Wolfgang Digital found, that both time spent on site, and pages viewed have good, positive correlations with conversions.
4. Similar Products
Similar product boxes can be based on very different logics. The least complex one is simple category-based filtering, which can be implemented even without a recommendation engine (needless to say, it lags behind in performance as well).
If you combine this simple filtering method with meta-data (descriptions, product titles, tags, prices, etc.) based similarity, you can greatly enhance the performance (such as, recommending items of the same brand or same color, from the current category), but for this, you will need to have advanced recommender functionality available on your site.
One of the best-performing similarity based logics is a method called “item-to-item collaborative filtering” (a method that was pioneered by Amazon). I’ll elaborate on this below.
5. “Customer Who Bought/Viewed This…” Collaborative Filtering
Collaborative filtering in eCommerce product recommendations was first implemented at scale by Amazon (they filed their initial patent for item-to-item collaborative filtering as early as 1999). I believe, their results speak for themselves.
At its core, collaborative filtering works by collecting preferences or taste information from many users (collaborating). Collaborative filtering can fuel item-to-item (product similarity based) and personalized recommendations as well.
Item-to-item collaborative filtering recommendations: This type of recommendation logic basically determines the similarity of two products by looking at how often they’re present together in the browsing or purchase histories of users.
In practice, widgets using this kind of logics are named, for example “Customers who viewed this, also viewed…” which pretty much explains the basic idea.
The preference models these algorithms build, are very authentic and specific to your website and your users, as they quantify the real interactions between your visitors and platform.
Moreover, with ample amount of data, even problems, such as automated accessory recommendations can be answered with a reasonable accuracy (more on this below, at the cart page accessory recommendation section).
6. Personalized Recommendations
Taking the context into consideration (the product being looked at), alongside the history of the current user, personalized product page recommendations can be fine-tuned even further. Personalized collaborative filtering algorithms can be applied for this scenario.
To get a general idea about how these work, look at the simplified example below:
Personalized collaborative filtering recommendations: The most common way of personalized recommendations is focusing on the average similarity of products, to the last x number of products a user has viewed.
The system uses the same item-to-item collaborative similarity for this task, only it does not compare one item to another, but to all items in the user’s history. Weighing the output based on the recency of interactions usually improves accuracy further.
Cart Recommendations
Recommending products similar, or related to the content of the visitor’s cart on cart pages can be a very effective way of boosting average order quantities and values.
Cart page recommendations reach the customer in a very favorable psychological state, where he most likely already decided to make a purchase, therefore he’ll be more willing to say yes to further offers. There are a couple of recommendation logics that are a very good fit for this task.
In our experience, cart page recommendations tend to provide well above average GMV/1000 recommendations numbers.
Moreover, according to a 2015 study carried out on 300 ecommerce stores by Braillance and Marketingsherpa, widgets that recommend other items to items in the user’s cart were among the 10 best performing recommendation types, based on the revenue from recommendations metric.
If we consider, how much less traffic cart pages receive than a product page, for instance, this is not to be sneezed at.
7. Recommending Accessories
Recommending accessories for products can significantly increase average order size and value on your site.
Moreover, implementing such widgets can be quite easy, technically speaking. However, depending on the size of your catalog and your category structure, it can be relatively administration-heavy, as it is very hard to automate the process of recommending compatible accessories for each item.
Therefore, such logics are usually fueled by manually assigned item relations.
Optimally, you can recommend categories to categories directly (i.e. iPhone -> iPhone Accessories). This is a lot easier to automate and scale, but requires extremely conscious category structure planning.
Of course, with enough data and a bit of creativity, one can abstract rules on how to analyze behavioral information, so that the output of the system will be actual, compatible accessories.
One of such easily codifiable rules that can define an accessory in relation to a product is items that are often bought together with it, but cost significantly (2x, 3x or more) less. This is a method we utilize at some of our enterprise clients.
This is where volume comes into play, as with a few purchases, results generated by this algorithm would be unpredictable, but reaching a critical amount of data, all statistics tend to correlate with reality more and more (given that the premises are correct).
8. Frequently Bought Together
Displaying frequently bought together products on cart pages can be very effective.
However, in order for virtually any cart page recommendation to provide results, the checkout process must lead customers through the cart page, where the actual recommendations are featured, and it is also a relatively data-heavy recommendation technique.
The layout of this page is also an important factor. If you have the time and resources, A/B testing different layouts and designs can yield excellent results and insights (needless to say, this holds true globally, by the way, not just for recommendations).
Netflix, for instance, uses A/B tests extensively for home screen layouts on different platforms, featured shows, and even movie artworks. There’s certainly a lot to learn form their approach.
Category Page Recommendations
When a customer is looking at a category page, he already gave you valuable information on what he’s looking for. Your aim here should be to provide assistance, so he won’t have to browse through the whole category, before arriving at his desired product or piece of content.
While the best option in this scenario is advanced filtering and faceted search, recommendations can provide value in these cases as well.
9. Popular Products
One of the most obvious ways to help users on category pages find what they’re looking for, is to show them what most people are looking for.
A little trick, is to use not singular categories for category page recommendations, but teach the system your category structure as well. That way, child categories can be recommended for parent categories, which provides more diversity in recommendations.
Another, and perhaps even better option is to reorder items in the category based on popularity, by default (just remember the 80:20 rule for popular products).
10. Zero Result and 404 Pages
Getting to a “zero search result” or 404 page often means the end of a customer’s session on a site, and, therefore, a potential lost conversion.
These pages are often the ones with the highest exit rates, and also one of the few, where exit rate is actually a very important metric.
I like the premise of this article on exit pages, which states, that the exit rate numbers are rather meaningless, except for in relation to pages connected to the checkout process, some “functional” pages (zero results can fit into this category), and 404 error pages.
There are several tactics, to provide recommendations that can get customers to continue browsing in these situations.
One of the basic methods, is that if your site couldn’t display results for the query only due to filters the customer had set up before, the system can perform an additional search on the complete product catalog, and display them the matches (if there are any).
Similarly, a search box on 404 pages can sometimes make up for the UX glitch.
Displaying personalized product recommendations on these pages can also help customer to resume the discovery thread that the error or no result query interrupted.
Conclusion
Naturally, the ideas, tactics, and techniques featured in this article are not all there is to say about product recommendations in eCommerce.
It’s merely meant to provide some food for thought to store owners or managers who are planning on implementing a recommender system or taking their current practices to the next level.
With the new breed of SaaS recommender systems available on the market nowadays, one can get familiar with product recommendations and start testing out their ideas right away, and experience the results first hand.