AWS Made Easy

AWS Insiders Podcast: Episode 1 – Transforming Customer Experience with Amazon Personalize

With Ankur Mehrotra, Director and General Manager for AWS AI

On this episode, Ankur Mehrotra dives deep into Amazon Personalize, a fully managed machine learning service that goes beyond static, rule-based recommendation systems and trains, tunes, and deploys custom ML models.

Listen now on

Speakers

Portrait of Rahul Subramaniam

Rahul Subramaniam

AWS superfan and CTO of ESW Capital

Rahul is currently the CEO of CloudFix and DevGraph and serves as the Head of Innovation at ESW Capital. Over the course of his career, Rahul has acquired and transformed 140+ software products in the last 13 years.
Portrait of Ankur Mehrotra

Ankur Mehrotra

Director and General Manager for AWS AI

General management responsibilities for various AWS AI products and services, including personalization, forecasting, anomaly detection, healthcare & life sciences, ML at the edge.

Transcript

  1. Speaker

    Hello and welcome to AWS insiders. On this podcast, we will uncover how today’s tech leaders can stay ahead of
    the constantly evolving pace of innovation at AWS. You’ll hear the secrets and strategies of Amazon’s top
    product managers on how to reduce costs, and improve performance.

  2. Speaker

    Today’s episode features an interview with Ankur Mehrotra, director and general manager for AWS AI. On this
    episode, Ankur dives deep into Amazon Personalize, a fully managed machine learning service that goes beyond
    static, rule-based recommendation systems, and trains, tunes, and deploys custom ML models. It allows for
    delivery of highly customized recommendations to customers across industries. But before we get into it, here’s
    a brief word from our sponsor.

  3. Speaker

    This podcast is brought to you by CloudFix. Ready to save money on your AWS bill? CloudFix finds and implements
    100% safe, AWS recommended account fixes that can save you 10 to 20% on your AWS bill. Visit cloudfix.com for a
    free savings assessment.

  4. Speaker

    And now here’s your host, AWS super fan and CTO of ESW capital, Rahul Subramaniam.

  5. Rahul Subramaniam

    So Ankur, welcome to the show, AWS Insiders. Really excited to talk today about AWS personalized. We’d like to
    understand a little bit about what you guys are doing with your customers with AWS Personalized. What os the
    problem that you’re really trying to solve with it?

  6. Ankur Mehrotra

    Thanks, Rahul. Thanks for having me. Yeah. Over the last few years, you can see that businesses are engaging
    with the users more and more through digital channels and that has only accelerated during the pandemic.

  7. Ankur Mehrotra

    So every digital touch point provides businesses an opportunity to create an experience for the users that’s
    more tailored to their needs. And that’s where Amazon Personalized comes in. It’s an AWS service that helps
    businesses create more personalized experiences for the users by creating recommendations that are powered by
    machine learning. They’ll be seeing a lot of customer interest in adoption, and a lot of AWS customers are able
    to build a high performing personalization system using Amazon Personalized and they’re seeing great results.

  8. Rahul Subramaniam

    Great. And I completely understand the need for providing this personalized content. These days, we get
    inundated by choices, especially around all the digital channels and getting the right kind of recommendations
    for what customers want to consume is incredibly valuable. So what are the changes since the beginning of the
    pandemic that you’re seeing in the kinds of use cases that customers are applying AWS Personalized?

  9. Ankur Mehrotra

    As consumer habits are changing and the way they’re engaging with different businesses are changing. We are
    seeing the need for personalization or the need for creating better user experiences in digital channels
    increase, and we’re seeing that happen across different [inaudible 00:03:20 ]. So obviously as retail has been a
    great example, and also where Amazon really pioneered personalization in the late nineties. But now we are
    seeing the need in all different kinds of workloads.

  10. Ankur Mehrotra

    For instance, in banking we are seeing customers or businesses trying to create more personalized experiences
    for the users. And we see even in other cases, for example, a very unique case that I will point out to you is,
    there’s an app called Calm. It’s an app for meditation and mindfulness. So they use Amazon Personalized to
    create recommendations for the users for mindfulness exercises. And they’ve told us how much impact that’s
    created during the pandemic. These kind of stories really make our efforts work.

  11. Rahul Subramaniam

    That sounds pretty interesting. If somebody were to start with Personalize, I think the thing that most
    businesses want to know is, how is going to impact my business and how do I measure impact of starting to use a
    service like Personalized? So how do you see customers measuring those metrics? What kind of metrics are they
    really looking at to measure the success?

  12. Ankur Mehrotra

    So I think this really depends on the kind of business. For instance, in an online retail scenario, businesses
    are looking to improve conversion rates, improve revenue for order. So this is where if you can add the right
    kind of personalization at the right customer touch points… For example, at the right point in the customer
    journey on the retail website, if you can add recommendations that drive more up sales, then you can change, you
    can increase your average revenue per order. It just changes from business to business.

  13. Ankur Mehrotra

    For example, in the media and entertainment space, if you have a, let’s say video or on-demand application,
    that’s where you may want to optimize for the video engagement and video watch time. And the great thing is that
    Amazon Personalize gives you an ability to optimize, to specify a metric, that you want to optimize your
    recommendations from.

  14. Rahul Subramaniam

    So, as I understand it, there are basically three constructs or fundamental constructs of how you end up using
    AWS Personalized. There’s the concept of the user, there are the items list that you have, and then a whole lot
    of events that you feed it. The events basically are interactions or impressions that the user has with those
    particular items.

  15. Ankur Mehrotra

    That’s right. So the way you can get started with Amazon Personalized is you have your interaction events, which
    is interactions between your users and items and these items can be products in your, if you are an online
    retail business, these can be products in your catalog. Or if you are, let’s say in media and entertainment,
    this can be videos or titles in a catalog. So in the interaction dataset, which sometimes also sends a spark of
    click stream dataset, that the most important dataset that you need to start using Amazon Personalize to build a
    recommendation system.

  16. Ankur Mehrotra

    Then if you have other metadata about your items or users. So for instance, if you have metadata such as product
    category or brand about the items in your catalog, then that is optional for you to use, but that can really
    improve the quality of your recommendations because Amazon Personalize is able to automatically understand
    trends between products that have similar metadata, for instance.

  17. Ankur Mehrotra

    Then if you have any information metadata about your users, such as their location, or their preferences, age
    group, et cetera, that also helps. So when you start using Amazon Personalize, you take these three types of
    datasets, the interactions one being most important, and you can add it to Amazon Personalize, either through a
    set of APIs, by just making API calls, or by moving this data through an Amazon S3 bucket. And then once you’ve
    done that, you can use a set of APIs to then select your use case or the type of recommendations that you’re
    trying to generate. At Reinvent recently we launched more use case specific recommenders for retail on retail
    and media entertainment. So if your use case falls within those two buckets, then you can really pick the domain
    specific use case within the service through the API.

  18. Ankur Mehrotra

    So in retail, for example, you have frequently bought together. So you can directly pick the recommenders that
    map to your use case. And if your use case does not fall in the retail or M&E bucket, then you can pick the
    recommendation types that most closely match a use case. So for in instance, we have recommendation types such
    as, user level recommendations. If you’re trying to generate a set of item recommendations for a particular user
    or a personalized ranking, if you have a set of items in mind, but you want to rank them differently for each
    user, you can select that what we call as recipe. If you want to create similar items product widget on a
    product detail page, then that recipe can be really useful.

  19. Ankur Mehrotra

    Then recently we also launched a new type of recommendations called intelligent user segmentation, where we flip
    the problem. We say, instead of generating recommendations for a given user you can specify an item or let’s say
    an item category or a genre, and generate a segment of users who may be interested in that item or genre. It can
    be used with different marketing tools, for email campaigns, ads, and even for other market [inaudible 00:09:17]
    that are upper in funnel or in the consideration.

  20. Ankur Mehrotra

    So these are the different kind of recommendations that you can choose within Amazon Personalized and it can be
    done through a simple API call. And once you do that, then in the background, Amazon Personalized uses machine
    learning, trains a bunch of different machine learning models to create recommendations that optimize for these
    use cases.

  21. Ankur Mehrotra

    Then you can also specify… You have all these knobs and dials that you can use to optimize or augment these
    recommendations. We see that our customers don’t want to directly display recommendations that are coming out of
    the machine learning model, they want to specify specific rules and constraints on their recommendations.

  22. Ankur Mehrotra

    For example, you may want to filter recommendations for a given product category or metrics. So you can do that
    easily through a filtering and business rules capability. It’s super easy to retrieve the recommendations
    through a low latency API and integrate that into your application or website or whatever experience you’re
    trying to create for your users. So that’s basically the end to end flow. You start with this data, you can add
    it to the service through APIs. Then you can pick the type of recommendations you want to create.

  23. Rahul Subramaniam

    Most folks who haven’t been in the AWS ecosystem feel like if they were to get onto using any of the AWS
    services, they literally have to create these plans to migrate to the cloud and have these enormously long
    exercises of moving to the cloud. All you really need is to take your data, move it into an S3 bucket, and then
    basically pass that on to the service that can create these models and then just access the outcomes of the
    recommendations directly via an API. And all of that comes prebuilt for them. They miss out on these amazing
    higher order services that you guys are creating. That can literally be leveraged by anybody without having to
    migrate their entire applications or data centers over to the cloud.

  24. Ankur Mehrotra

    Yeah. That’s absolutely correct, Rahul. I think if you were to implement such a recommendation system on frame,
    you would have to think about servers and instances and how you scale. When you’re using Amazon Personalized you
    don’t have to have any of your existing infrastructure data on AWS. This is a completely managed service. Which
    means all you have to do is haul a set of APIs and you’ve got highly relevant recommendations ready for you to
    integrate into your application.

  25. Rahul Subramaniam

    Right. So I think given that almost every application has tons of data or items of some sort that they’re
    presenting, whether it be documents, whether it be whatever comes up basically on your screen, can be treated
    pretty much like an item. And you have to optimize your UX to make sure that you’re only recommending stuff that
    is most relevant to the customer. So this becomes even more relevant, and given that there’s such low friction
    to get started with something like this, if your product doesn’t today have a recommendation capability using
    AWS, Personalize seems like the easiest way to get started. Take your data, put it in S3 and start getting
    recommendations right out of the box.

  26. Ankur Mehrotra

    Exactly. Given how personalization can directly impact the business output, all businesses should try and use
    the service to see how this drives overall improvement in the business metrics.

  27. Rahul Subramaniam

    Are there any constraints or limits on the number of items? I think sometime ago we were running into some
    limits of how many documents or items you can actually have in the system. Are there limits or constraints
    around that, that have changed?

  28. Ankur Mehrotra

    There is a limit where for one type of recommendation, which we also refer to as a recommended or a campaign,
    you can have to, to 750,000 items in the catalog. Now that usually is sufficient. You can split it into multiple
    recommendation solutions, but then if you need higher limits, then we are happy to work with you to look at your
    specific use case and see how we can help you.

  29. Rahul Subramaniam

    So what does the cycle time really look like with Personalized? And how are folks really measuring relevance of
    their recommendations that they’re getting?

  30. Ankur Mehrotra

    That’s a great question. We always tell our customers that when you are starting out with Amazon Personalize or
    building a personalization system, always think about how you’re going to measure success. In success for you
    would mean, an uptake in [consumer 00:14:00] rates or an uptake in conversion rates. When you’re using Amazon
    Personalize as a member of service, underneath is training machine learning models, it does give you offline
    metrics in terms of how well the system thinks that the machine learning model form. But those are offline
    metrics and understand the efficacy of those recommendations to a certain extent through those offline metrics.

  31. Ankur Mehrotra

    We always recommend our customers do experiments with real customer traffic to measure the performance against
    the business metrics that they’re trying to optimize. So we see the classic way of doing this is, run an AB test
    compared to where you have coming from Amazon Personalize, running in an experiment against an experience that
    has no recommendations, that’s not personalized, or any baseline.

  32. Rahul Subramaniam

    Great. Coming to the AB test set up. What are some easy recommendations for getting started with AB testing?
    Because it seems like a really hairy problem for someone just getting started with a simple AWS service like
    Personalize.

  33. Ankur Mehrotra

    Yeah. You’re right, Rahul. We’re working on making this much more easy for Amazon Personalize customers. We do
    have some resources, including documentation blocks that walk our customers through some of the best practices
    they should use while setting up AB tests. But one recommendation I would make is, don’t start from scratch.
    Don’t start implementing your own AB testing framework just to do this. There are a lot of other… Lot of a AB
    testing frameworks out there that are widely used and leverage them. And some of them, we also do have reference
    architectures where they can seamlessly integrate with Amazon Personalize and the sample code out there that you
    can use. So that’s that’s one advice I would give. If you don’t have an AB testing framework already in place,
    try and look around and see what exists out there and what can be used.

  34. Rahul Subramaniam

    Great. I think that’s great advice. I think folks who are usually coming in from the on-prem data center world
    are in the habit of trying to create everything from scratch, because they don’t have that support system of all
    these other services and recipes that other people have created that can literally be deployed with a click,
    either by the marketplace or via other tools and services that are available online.

  35. Rahul Subramaniam

    So let me use this to segue into your three pieces of advice that you would give customers as they’re getting
    started.

  36. Ankur Mehrotra

    Sure. When we see customers starting out with a personalization or a recommendation system project, I think it’s
    starting out small by building out a proof of concept is the right thing to do. However, sometimes what we see
    is that when customers are trying to start small, they take a very small amount of data and try to build a
    recommendation system.

  37. Ankur Mehrotra

    Now, the challenge is that if you don’t have sufficient data that presents your overall data set well, then the
    quality of recommendations that you may get will not be as good as what you may get if you’re using a larger set
    of your data with Amazon Personalize. So that is one thing to think about as you’re doing a POC or you’re trying
    to build a smaller version of a recommendation system before you scale.

  38. Ankur Mehrotra

    So having sufficient data in the right, paying attention to the quality of data is really important as you start
    out. That’s the first thing I would say.

  39. Rahul Subramaniam

    So is there a thumb rule for the amount of data that you need to have? I assume most of it is specifically
    around the events’ data. Is there a thumb rule that you’d recommend to folks about what they should think about
    in terms of quantum of data that they should at least have?

  40. Ankur Mehrotra

    There is no hard and fast rule as such, but the high level guidance I would say is, make sure… So if let’s say
    you are in an eCommerce business, make sure that the data that you’re using is representative of your entire
    catalog, or at least all product categories, rather than just picking the data from just one or two categories
    and then later you may end up wondering why recommendations for other categories aren’t good enough. And the
    same goes for users as well. If let’s say you have users of, if you have different features on a website and
    different kinds of users, you want to make sure that’s also representative of your larger user base. It differs
    from use case, to use case, but that’s the high level of guidance I would provide.

  41. Ankur Mehrotra

    And this does require some experimentation to get right. Yeah. So there are other things that I would say is the
    ability to provide click stream events in real time. Consumer habits can change quite rapidly. So when you’re
    browsing something, you may be looking for something right now that you were not looking for, let’s say three
    days ago.

  42. Ankur Mehrotra

    So if your recommendation system is not aware of your recent browsing behavior, then sometimes the
    recommendations can end up being not relevant, aligned with what you’re looking for right now. We always advise
    our customers to try and use the legal client events capability and stream those events to Amazon Personalize,
    so you can do it quite easily, either using an API that the service provides or using the [edifice amplify
    00:19:39] as the creative part integrations with Amazon Personalize. That really improves the quality of
    recommendations you may receive in to the service. And it can also really make a big difference in the overall
    success.

  43. Rahul Subramaniam

    So I have a very specific question around this specific scenario because we struggled with it for a while as we
    were using Personalize. So I thought this would be a good learning experience for everyone else as well.
    Especially around the holiday seasons, the recommendations might vary quite a bit from what you typically end up
    recommending, let’s say in a usual, or through the rest of the year. So in such scenarios where there are these
    unique events that you can foresee when they’re going to come up, but your recommendation style needs to change,
    how do as one end up using Personalize to drive the new mechanism of recommendations for such scenarios? What
    are the customers doing in those scenarios?

  44. Ankur Mehrotra

    Good question. There are two ways to address this. The first one is, which I just mentioned that, if you have
    real time events flowing into the service then the ML models will latch onto the latest, the most recent
    browsing behavior of your users and change recommendations in real time. So to solve this so that as your
    browsing trends and shopping trends start changing due to seasonality, the ML models will quickly adapt to it,
    changing its recommendations.

  45. Ankur Mehrotra

    The second way to do this, to feed into the service. And when I say data, the historical data. The historically
    interactions data that you use to give to the service, and the service uses that are train machinery models. So
    in that historical data, if you have sufficient coverage where you have data from previous, let’s say holidays
    or seasons built in, and if you have metadata that indicates that yes, this day was a holiday. Let’s say your
    trends on Black Friday or Cyber Monday may be different. So if you can annotate your data set by saying, and let
    the service know that while this day was actually a holiday, then that can be really used for the service to
    learn how consumer behavior and patterns and shopping patterns change during a particular [inaudible 00:22:06].

  46. Rahul Subramaniam

    That’s actually really useful. My last question is going to be around your recommendation towards avoiding any
    costs overruns, or losing control over costs. Now at the face of it, AWS Personalize seems like a ridiculously
    cheap service for the value that it provides. If I’m not mistaken, you guys price it in terms of tens of
    thousands of users and it’s a fraction of a dollar for providing recommendations up to 4,000 recommendations,
    it’s practically free.

  47. Rahul Subramaniam

    So what are the mistakes that a customer could make, where they suddenly find that the way they’re using the
    service, or leveraging the service, they end up losing control. And this is common with a lot of these services
    where you find that customers make one mistake. They realize only later because of the way consumption based
    billing works, you realize that only when the bill arrives that you’ve made a big mistake. So what would be the
    one or two recommendations that you would have for someone using Personalize to avoid crazy cost overruns?

  48. Ankur Mehrotra

    Yeah. So first of all, you’re right, we get this from so many of our customers. The amount of positive impact or
    the uptake in the business output that they saw using Amazon Personalize is far, far greater than what they’re
    paying for the service. And we’re happy to see that customers are deriving so much value from the service.

  49. Ankur Mehrotra

    There aren’t too many ways where you could end up cost over runs with Amazon Personalize, but there are a couple
    things I will say. So Amazon Personalize can scale automatically as your traffic increases. Let’s say, if you
    have more traffic during the daytime and less traffic during the nighttime, then the service can scale up and
    down automatically to provision the right amount of resources to serve your recommendations.

  50. Ankur Mehrotra

    However, we also do allow the ability for you to reserves a certain recommendations per second. And that can be
    really useful when you’re expecting the traffic to suddenly spike up. So for those cases, the service does
    provide you a way to reserve higher, let’s say TPS. So one advice I would give is, think about when you really
    need to reserve that provisioning because the increased reserved provisioning you’re also paying more. So use
    that feature where you need it. And when it’s business as usual, just rely on the auto scaling features.

  51. Rahul Subramaniam

    Would you recommend that folks, especially in auto scaling, set up certain kinds of alarms and alerts so that
    when it exceeds beyond what you anticipate, in terms of traffic or flow of events and so on, you’re at least
    aware of it in real time, rather than realizing way later that you had some kind of a service event within your
    application that just suddenly caused a gazillion API calls to be made to Personalize, and it just kept auto
    scaling up without you really expecting or anticipating that event?

  52. Ankur Mehrotra

    Yeah, I think that’s always a good idea to keep some alarms to monitor how the overall integration with
    Personalize and how the number of feedback calls being made are trending and alerted. If there’s something
    that’s out of the ordinary.

  53. Rahul Subramaniam

    Great. I’d love for you to share a few success stories and non-retail would actually be really helpful, because
    I think the number of people who are looking for recommendations as a service that actually really works well
    for their non-retail scenario. So are there examples that you could share with us in those spaces or in those
    domains?

  54. Ankur Mehrotra

    Sure. Yeah. Yeah. So especially during the pandemic, the consumption of media and entertainment services has
    certainly spiked. So we see personalization becoming more and more relevant. One of our AWS customers discovery
    a few months ago, they launched their first director consumer streaming service called Discovery Plus. They had
    a very tight timeline to build a service end, to end and launch it.

  55. Ankur Mehrotra

    They chose Amazon Personalize to bar their recommendations and they were able to do that from absolutely nothing
    to this running and production at scale, ready for prime time in a matter of few months, just going from POC to
    just integrating it all the way into their system. So that’s been a great story and love working with the team
    at Discovery. They’ve been great. They recommended videos and titles and similar videos and other
    recommendations that you see in the Discovery Plus app are programs on Personalize.

  56. Ankur Mehrotra

    The other one that comes to my mind is Intuit is a data list customer. They’ve bought an app called Mint, which
    helps users track their budget and their spending, et cetera. Intuit used Amazon Personalize to create
    recommendations within the Mint app for financial offers and recommendations for financial products. It is a
    great use case where you’re able to apply the service in all kinds of different use cases. And they’re seeing
    great value from it. That’s another one that I really, really like.

  57. Ankur Mehrotra

    So we are seeing use cases across different industries and pleasant piece of price to see how well the models
    can adapt to different use cases. We’re excited and we’re working on many new features and launches, and it’s
    going to be a great year for the service.

  58. Rahul Subramaniam

    Thanks so much, Ankur for sharing all of this knowledge with us. As always, we remain huge fans of all of the
    higher order services at AWS, including Forecast Personalize and the likes. And I look forward to working with
    you and your team through the year with all the new stuff that you guys are coming up with.

  59. Rahul Subramaniam

    For the audience I would really urge you to go try it Personalize. It’s really simple to get started with. And
    for any application on prem or otherwise, personalization is now becoming pretty much a standard function that
    you have to offer. AWS Personalize is probably one of the easiest ways to get that into your app without doing a
    lot of heavy lifting. So go check it out and thank you once again for coming here and talking to us.

  60. Ankur Mehrotra

    Thanks, Rahul. It’s always great talking to you.

  61. Speaker

    We hope you enjoyed this episode of AWS Insiders. If so, please take a moment to rate and review the show. For
    more information on how to implement 100% safe AWS-recommended account fixes that can save you an average of 25%
    on every AWS bill, visit cloudfix.com. Join us again next time for more secrets and strategies from Amazon’s top
    product managers. Thank you for listening

Leave a Reply

Your email address will not be published.