An introduction by way of some marketing prose.
Cutting edge intelligent systems. Big data at scale. World class ad tech to drive phenomenal growth in a multi-billion dollar business. Real-time tracking and personalization to engage millions of shoppers daily: Ad Tech & Customer Intelligence Engineering @ Wayfair.
Meet the two halves of an amazing engineering team.
Ad Tech: Responsible for the development of all Wayfair proprietary Ad Tech, continuously delivering solutions which outperform all available commercial alternatives. Keyword search optimization for hundreds of millions of dollars of annual ad spend and a product catalog with millions of distinct products. Display retargeting to outperform specialized industry leaders, fueled by real-time product recommendations and user-level bidding. Intelligent campaign automation that exceeds the configurability, intuitions and power of commercial alternatives, enabling Wayfair to get the perfect content in front of each user with the best timing and optimal channel, for hundreds of concurrent campaigns and tens of millions of customers.
Customer Intelligence: Provides the underpinnings for our award winning ad tech as well as customer analytics and personalization for the entire customer experience. Responsible for the development of an industry-best tracking event bus and curation of a holistic customer file, making hundreds of data points available for real-time consumption. Delivering systems which allow for real-time product and content recommendations at massive scale, enabled by the best combination of configurability and machine learning.
I pulled the above from one of our job descriptions. Somewhat sensationalized prose, but you get the idea: We build cool stuff.
My name is Franklyn. I build ad tech.
Five and a half years ago, I joined Wayfair as product manager responsible for the evolution of our ad tech. This after 10 years in Japan, where I was first a software engineer developing video games and later at the intersection of product management and business development creating a display ad network out of virtual ‘in game’ billboards, and then a brief stint getting my MBA.
Five years ago, I moved into Wayfair Engineering. I was an engineer before shifting to product. Wayfair was still a relatively young brand; we were just getting our feet wet with initiatives to modernize our paid search and make display advertising effective; we were dabbling in email personalization and starting to think about technology decisions to power our advertising at scale. We needed strong engineering focus on the development of our marketing tech. The move made sense.
That was the rebirth of Wayfair’s ad tech platform. 10 engineers at the start, and some -huge- goals: Build world class ad tech; empower an ambitious marketing department; drive massive growth.
The team’s evolution
Wayfair Chief Architect, Ben Clark, calls us secretive. He jokes about how little attention we call to ourselves, suggesting that we’re a covert organization bent on world domination. He even referred to this inaugural blog post as “the de-cloaking.”
I prefer to think of us as NASA: (i) We’re constantly innovating and (ii) much of what we build starts off powering our ads but ends up enabling all kinds of new features elsewhere in the customer experience.
We’re also an especially nimble team in a huge tech organization. As a result, we’ve piloted a number of technologies over the years, driving their addition to Wayfair’s Tech Radar for general use. For example, in 2018 we paved the way for company-wide adoption of Aerospike following a bake off with Cassandra and HBase. Looking for something with minimal read latency that would support our customer data graph, we originally thought Cassandra; we netted out adopting Aerospike when we took into account a number of features (reduced maintenance overhead, substantially lower hardware costs, and performance comparable to a pure in-memory cache) that made it virtually a drop in upgrade for persistent Redis company-wide.
Now 95 people strong, Ad Tech & Customer Intelligence Engineering has delivered some major wins for the company. And I plan on borrowing the podium here from time to time to introduce you to our team, our platforms, as well as our development philosophy and learnings.
What about our work positions us as a Wayfair innovation hub? A preview of posts to come, courtesy of Wayfair Ad Tech & Customer Intelligence Engineering.
Project “Scribe” – At the absolute core of our customer intelligence engine lies “Scribe,” our real-time tracking pipeline. “Real-time” is key here; the platform was developed in a Lambda architecture to function both as the source of truth for analytics -and- the event-bus that would eventually power our online customer data repository, real-time product recommendations, and streaming notifications.
Project “Waystack” – “Waystack” is the codename for our first-party ad server, enabling advanced tracking, segmentation, and delivery of personalized content for every customer impression. But the story of “Waystack,” developed as our in-house display retargeting solution, is actually the story of a few -other- projects: “Libra,” “Customer Information” and “Recommendations.” With the daunting task of beating specialized industry leaders at the game of display advertising, we had to get ‘creative’ (pun intended) by introducing:
Project “Libra” – An entirely stateless, platform agnostic, data-center agnostic A/B testing engine to learn what would and wouldn’t work.
Projects “Customer Information” and “Recommendations” – Two services, one which provides the relevant customer browse context, reflecting up-to-the-second customer activity, and a second which joins that to product and sale event data in real-time, allowing us to quickly configure and deliver new product, sale event, content and messaging recommendations, immediately, for any customer, with minimal pre-compute.
Project “Archimedes” – Ad Tech & Customer Intelligence Engineering’s answer to machine learning, Project “Archimedes” is actually the blanket code-name for all of our customer intelligence efforts, and thereby inclusive of much of the above. It also encompasses customer data aggregation (otherwise known as our ‘facts to features’ engine), customer segmentation, and prediction for many of our machine learning-based models. Little known fact: Archimedes is also the name of my dog.
Project “GrayMatter” – The ‘brains’ of our notifications platform, “GrayMatter” empowers the business to define timed series of notification actions, interwoven with testing and conditional logic, with customizable content and templating strategies. Designed with separation of concerns in mind, “GrayMatter” focuses on configurability in the logic layer, leveraging other external services to check A/B tests, gather recommendations, compile and send email messages and push notifications, etc. “GrayMatter” is quickly evolving into the decision engine backing all customer touch-points.
Project “Smart Routing” – In collaboration with teams in Operational Engineering, Project “Smart Routing” represents an interesting use of both our technology (“Scribe” and “Customer Information”) and design patterns (“Libra”) to route customers to the most appropriate customer service agent when they call in. This has gone through multiple iterations since its initial 2016 deployment, and now incorporates predictive modeling.
Projects “Athena” and “Rio” – With a product catalog containing more than 14 million distinct purchasable items, generating keywords for SEM is no small task, nor is bidding for them. Add to that the complexity of keeping price and inventory information in sync with platform partners like Google and Facebook. “Athena” and “Rio” address each of these concerns in turn, making hundreds of millions of dollars of annual ad spend highly efficient.
Just the beginning
We come from humble beginnings: Ten engineers doing their best to get out ahead of an ambitious marketing department with sizeable goals. And having continuously delivered solutions like the above, solutions that outperform available commercial alternatives, we have evolved.
We’re a Wayfair success story: An internal ad tech company providing best-in-class technology solutions in our own right, and a key driver of Wayfair’s overall success. We have a lot more to share; we’re sure this inaugural article has piqued your interest. Watch this space for deep dives on the above projects and more.