Customer Journey Funnel - A Framework for Answering Product Analytics Interview Questions
In Data Science interviews in the Product space, it is common to get questions like -
Visits/Revenue/Deliveries/Metric A dropped by 20% for an ecommerce website/Product, how will you identify the issue?
How will you improve a particular product? How will you prioritize which feature to pick up?
One good way to answer the above questions is through the lens of a customer funnel.
What is a customer funnel?
It’s a framework which can explain the customer journey from the start till end state of a product. For example (see the figure below) - For a simple ecommerce website, it can start from Site Visitors and end in a product purchase. There are also several intermediary steps that are important in this journey. It’ll be worthwhile looking at each step of the funnel below and practise making these for some of your favourite products — could be Netflix Streaming or sign-up process for Instagram.
Laying out your understanding of the funnel is an important step in assuring your interviewer that you think through problems systematically with clearly defined customer flow in mind.
What are the inputs and outputs at each step of the funnel?
The inputs are all possible levers that can impact the output of a particular step. For example - Step 1 here talks about Total Site Visitors. This is directly related to how Marketing, SEO are going. Seasonality is another factor that strongly affects this.
Step 2 is around how many users actually viewed a detail page. This is directly impacted by price, aesthetics and relevance. One gets the idea that if you influence these levers, you can also increase the proportion of users who view Item Detail Page.
Next up, “Clicking on Add to Cart” button is directly influenced by product quality, description, # of images and Product Reviews. Subsequently, “Go to Check out” is a result of Calls to Actions, Promotions and Discounts.
Finally, “Purchases” are a product of how seamless the check out process is.
Another thing to note here is what are the inputs that can be controlled vs that can’t be controlled? For example - Marketing & SEO can be controlled while Seasonality can not. Grasping Seasonality (for example - around holiday season for an ecommerce website) is something that’s really important and most interviews would like you to touch upon that.
Broadly, interviewer is looking for how quickly you’re able to come up with inputs after laying out the funnel. One way to practice would be to look at services you use like Facebook Events or Google Calendar - can you draw a funnel for that — and get the inputs and outputs? The quicker you do that, the better your interview will go because you have laid out at the framework at the onset.
Another Important to caveat here is to understand the ethics of what’s possible and what’s not. Manipulating Product Reviews is not a ethical way to get increase outcomes!
As a data scientist, you will often work with PM team to possibly design features that will impact these inputs so that drop-off after each step is reduced. Some of these inputs are difficult to change (Price, Relevancy (because of model limitations)) but some aren’t and have more leeway. What features you choose to work on has to be aligned with the rest of the website and the strategy of the company.
Another set of questions which can follow the above is identification of different features (usually provided by interviewer) that can be implemented at different steps of the funnel journey.
Which feature should you choose to prioritize to implement?
Important thing to note here is that you need to tie in with the top-line goal here. How much will this feature impact the metric that actually matters! For an early stage company, growth is most important. For a mature company, that may be profitability. Identify metrics most important with the strategy and which features will hit that metric the most. For competing features 1 & 2 above operating at different steps of the funnel, it’s important to understand which feature will impact your top-line goal. In this case, we have Revenue as the top-line goal — and we go ahead with Feature 2.
This customer funnel framework will also help you with troubleshooting. Usually interview will ask you something that’s really down the funnel. “Revenue is down 20%. Can you diagnose that?” To reiterate, you can identify all the steps leading up-to that metric. You can then backtrack and see where’s the dip. Once you’ve identified the right step in the funnel, it becomes easier. You can also now divide it by other segments like platforms, marketplaces, even screen sizes etc.
References
How to Think Product Analytics in PM Interviews by Amazon Sr PM, Vivek Pandey (Highly recommended. This post was heavily inspired by this)
To practice these kind of case study questions and more with actual data science interviewers, sign up to be mock interviewed at https://insider-training.co
