- RETAIL SEGMENT: B2B Maintenance, Repair, and Operations (MRO)
- PRODUCT: Recommend™
- CHALLENGE: Test and implement an extensible solution that can personalize every aspect of the shopping journey, and provide a robust platform for future development.
- 2% lift in RPS
- 1% lift in AOV
- 1% lift in CVR
MonotaRO Co., Ltd. is a leading B2B ecommerce company in the maintenance, repair and operations (MRO) space in Japan. MonotaRO.com offers customers approximately nine million items from a multitude of categories—including construction tools, industrial clothing and tools, office supplies/equipment, machinery parts, and automotive parts. Their customer base of 1.5 million users continues to grow by an average of 30,000 new customers per month. The company went public on the Tokyo Stock Exchange December 2006, and has been listed on the First Section of the TSE (for large companies) since December 2009. As of June 2015, MonotaRO’s trailing 12-month online revenue is between $400–450 million and growing.
Since inception, MonotaRO has been a data-driven company, which has facilitated its shift from a catalog-based business to a predominantly ecommerce company. The company developed advanced internal software development skills that it applied to its ecommerce and B2B supply chain, and was an early adopter of online optimization techniques/technologies, analytics, data mining and predictive modeling.
In spite of the availability of numerous recommendation software tools and services in the Japanese market, MonotaRO developed their own internal recommendations system, which produced very good results, relative to available alternatives. They continued their investment in the development of customer behavioral analytics methodologies and technologies.
In the summer of 2014, RichRelevance (headquarted in San Francisco, CA) was still assessing the market opportunity for its cloud-based personalization solutions in Japan and lacked a strong presence in the country. Meanwhile, MonotaRO had developed and was using its own personalization solutions, but recognized that they could best leverage their technology, know-how and expertise by partnering with and taking advantage of RichRelevance’s advanced personalization technology and platform, and engaged in a.study to examine the suitability of RichRelevance for its business.
MonotaRO was aware of RichRelevance’s market leadership and credibility in the USA and Europe, and its renowned King-of-the-Hill™ (“KotH”) approach, which forces a real-time competition amongst over 100 different personalization algorithms, in order to select the most relevant recommendations for each individual shopper at that precise moment in time.
Of great interest to MonotaRO was RichRelevance’s ability to utilize consider contextual data—not only page type, device type, and vertical/category, but also a wealth of behavioral data from online shoppers, both known and anonymous. This data is brought together inside a vast, real-time, Hadoop cloud infrastructure, enabling not only personalization services, but also open, easy access to data from other relevant and future applications MonotaRO has, or plans to deploy
A/B Testing Using RichRelevance Personalization Technologies
Though MonotaRO had already deployed their home-grown recommendation software on their website and had invested heavily in building a tool combining product and customer attributes specific to their customer base, they chose to conduct an A/B test pitting their technology against RichRelevance’s personalization capabilities . Whereas RichRelevance offers a number of services for personalization through its Relevance Cloud™ (category page and search result sorting with Discover™, content and campaign optimization with Engage™, and product recommendations with Recommend™), the A/B test was limited to recommendations. Lift in various KPIs were measured over the course of the A/B test.
The RichRelevance personalization engine is unique for its ensemble learning personalization engine, which learns over time and continuously improves various KPIs to deliver competitive advantage.
During the “listening mode” phase of implementation (when the engine gathers online behavioral information for analysis, without displaying recommendations), MonotaRO could track how RichRelevance was learning and optimizing relative to the product catalog structure (categories, sub categories, etc.) and associated attributes. It could also see how the collected behavioral data gradually improved the quality and confidence of recommendations, and how the various predictive models were continuously optimizing performance as time went on. This learning and improvement continues to this date, even after going live.
“What was immediately evident after starting A/B testing with RichRelevance Recommend was the rapid and dramatic reduction in the bounce rate, which was continuously reduced by over 1% on the RichRelevance side throughout testing,” says Masaya Suzuki, CEO and President of MonotaRO Co. Ltd.
“We invest heavily in Internet advertising and the purchase of Ad Words at our company, so a lower bounce rate directly translates to higher ROI in our advertising budget. This alone persuaded us that RichRelevance was very valuable to MonotaRO.”
Suzuki adds, “At the immediate start of A/B testing, we honestly believed that our own recommendations would prove to be pretty good compared to those of RichRelevance, as we had a big advantage from the cumulative optimization of our own algorithms over several years’ worth of purchase data. However, after two to three weeks, the RichRelevance engine started showing evident progress in its automated learning and fine-tuning, and our KPIs continuously improved.” He continues, “Very soon after the A/B test results started to stabilize, we realized that we should consider moving beyond testing as soon as possible in order to avoid losing a real revenue opportunity! By the first calendar quarter of 2015, we decided to contract with RichRelevance for a long-term subscription to their Recommend product and other services.”
Masato Kubo, MonotaRO’s Data Analytics Group Director, was responsible for overseeing the day-to-day validation of RichRelevance’s A/B test results. He comments that: “As the personalization engine continuously learned and optimized, we definitely saw a lift in revenue. The improvement was clear— around 2% in revenue per session, 1% in average order value, and about 1% lift in conversion rate.”
“Leveraging optimal algorithms and customer-specific segment information, in combination with fine-tuning and support from RichRelevance, proved instrumental in driving concrete and positive results,” adds Kubo. “We originally thought that our deep understanding of our B2B business, which was embedded in our own in-house recommendations, would result in comparable results to those of RichRelevance. But the A/B test results clearly showed otherwise.”
An Extensible Personalization Platform
Another factor that supported MonotaRO’s decision to implement RichRelevance was the open, service-oriented architecture (SOA) of its Build™ platform, since MonotaRO plans to utilize their data sources across a number of personalization innovation initiatives.
CEO Suzuki comments, “Before RichRelevance, we were far along in building our own internal Hadoop infrastructure to collect, store and analyze big data from customer behavioral information; this was required for us to remain a data-driven innovative company. We then found out that RichRelevance offered access to their high-performance and proven platform. After much detailed research, we determined that their system was exactly what we wanted, and that we could use it to our advantage.” He continues, “We would have needed an enormous level of internal investment to build something similar to what RichRelevance offers, let alone the years of planning and development.”
Suzuki further adds, “One of the great things about RichRelevance is that it offers a wide range of deep personalization applications with depth in capabilities, in addition to offering a large number of personalization algorithms to customize every shopper experience. Pure-play recommendation systems tend to overemphasize data analytical aspects, but RichRelevance take this one step further by offering personalization features for marketers and merchandisers, as well as providing tools to improve the overall user experience.
“The next step for MonotaRO is the implementation of Discover™, to provide personalized sorting on category pages based on an individual shopper’s behavior. We hope this will dramatically improve the ease and comfort with which our customers find the products they need and want as soon as possible.”
Suzuki concludes, “RichRelevance initially described their product as a merchandising solution, not an IT tool. After implementation, we have now come to understand the true meaning of this.”
Kosuke Furuhata, leader of MonotaRO’s Content Development Group, was intimately involved in the RichRelevance implementation. He notes that, “The implementation of RichRelevance required almost no modifications at all to our ecommerce system and infrastructure. I was amazed by the number of personalization solutions they offered; had we tried to build this internally, it would have required resources and time greatly in excess of what we could afford.”
Email and Beyond to Mobile
At MonotaRO, efforts are already underway to deploy sort personalization on category pages using Discover, as well as to include personalized product recommendations in email using Recommend. Further, MonotaRO plans to expand the use of RichRelevance personalization to its mobile site and applications, while also ingesting segmentation scoring data into the Build platform—after which it will experiment with building its own personalization strategies to exploit segmentation data.
“Our traditional category pages did not include any personalization in the product display order, so we are naturally excited to have Discover implemented in hopes of overall customer user experience improvements,” says Analytics Group Director Kubo. “Our email recommendations in the past have been very rudimentary in logic. However, with RichRelavance’s ability to optimize the latest recommendations at the time the email is opened, we are confident that we will be able to provide a better experience to our customers.”
“In keeping with market trends, mobile devices are increasingly more important for our customers at MonotaRO. We are expecting good results from personalization of the mobile experience as well. And it doesn’t end there—there is still much more to be done, such as customizing algorithms that take into account purchase records and product attributes, as well as advanced personalization through customer segmentation. Our goal is to expand the horizon and grow our maturity level, in order to reap maximum benefit from the system.”DOWNLOAD