In April, I attended the Data Science Research Symposium at University of Massachusetts Amherst and gave a short talk highlighting Wayfair’s marketing data science projects. The full video and slides are below, but here’s an overview.

Goals of marketing data science at Wayfair

As the largest online retailer of furnishings and home decor, each year Wayfair invests hundreds of millions of dollars in different types of advertising (which we call “marketing channels”, e.g. search engines, TV, email, direct mail, social media) to bring in more customers and therefore drive revenue. As marketing data scientists, our mission is to provide data-driven solutions to optimize the marketing budget allocation and to maximize the ROI of each marketing channel.

An overview of Wayfair’s marketing data science projects

At Wayfair, data scientists are active players in the research, development, and production cycle of our products. We leverage our interdisciplinary knowledge and experience, which typically includes 1) business and operational research; 2) machine learning and statistical modeling; and 3) engineering. Our projects are always motivated by the needs of our stakeholders – in this case, Wayfair’s marketing teams. Usually, our work starts with defining the problem and setting quantifiable goals . Then we develop solutions to achieve those goals by building machine learning models. In this step, prediction and optimization are the two major themes for our marketing data science products. To verify that our predictive and/or optimization strategies indeed work, our gold standard is to perform an A/B test to show that the new method outperforms any existing method as measured by marketing key performance indicators (KPI). We productionize our model only after its success is validated by an A/B test. During productionization, we scale up our models, generalize our strategies and set up for automation.

Athena: an automated bidding platform

Search engine marketing is currently Wayfair’s biggest marketing channel. Every day, we bid on millions of keywords so that the corresponding Wayfair products will show up in search results pages. At Wayfair, we developed an in-house platform (named after Athena, the Goddess of wisdom) to automate the bidding process. Our mission is to bring in as much revenue and/or visits as possible without overspending. To achieve this goal, we first define it as an optimization problem: that is, to maximize revenue as long as the efficiency constraint is met such that the net revenue is greater than or equal to ad cost. To solve this optimization problem, next we build up predictive models to characterize the relationship of bid value change and its corresponding effects (net revenue and ad cost changes). Specifically, we ensemble multiple machine learning algorithms (linear models, random forest, and XGBoost) to more accurately describe causal relationships. Then we search for optimal bid values for each keyword using the ensembled model to either maximize revenue if the efficiency constraint suffices, or to maximize bidding efficiency if the efficiency constraint does not suffice. To demonstrate that this strategy indeed works, we ran an A/B test to show that when compared to the existing bidding algorithm, this new approach brings in more visits and revenue without changing the bidding efficiency. We are currently working on automating this new strategy to scale up the bidding capacity to millions of keywords and Wayfair products.

I hope this post has provided some insights about the goals and products of Wayfair’s marketing data science team. In my next blog post, I’ll introduce our in-house automated A/B test platform itself (named after Gemini), which allows us to simultaneously test multiple strategies and methods in an objective and efficient way.