Ask Us Anything: Episode 8

Rahul

Stephen, you might have to unmute. Good morning, everyone.

Rahul

The weekend is awesome. We made a trip to keep this appointment and to be honest, not disappointing at all. It was fabulous. Though it is a little wet and cold, but we had family, friends, it was quite an amazing weekend camping out there. And this week looks like really awesome weather here in Seattle. So yeah, it’s been bright and sunny for the last two or three days. Finally, just as we were wrapping up our camping, we packed all of our stuff into the cars and we were just waiting to kinda head out, that’s when the sun came out. For the entire two days of the weekend, it was dark and gloomy. But you know, we had a lovely campsite. We had you know glamping of sorts, we had a yurt. And had a nice fire going. And we had really nice neighbors in the camping area who let us use their canopy, so we could all stay dry. So it was pretty awesome.

Rahul

How about you?

Rahul

How was your weekend?

Rahul

All right. Cool.

Rahul

It looks like AWS is having some sort of festivities themselves. As you and I were discussing on Friday, things were going berserk. We had the AWS announcements. I think we’ve had a record number of announcements this week across all the different services. And my phone was just going crazy with all the alerts that are coming in. So, this is gonna be an interesting week to discuss all the different announcement that I got. I’m not sure we’re gonna have time to do all of them. But I [crosstalk 00:13:18].

Rahul

Yeah, I think that would be a fun thing. Or to make it even more entertaining, you’re gonna have these electric buzzers that you have kind of where you put your app monitor it buzzes every time you can overshoot. It’s really hard to kind of not overshoot, right?

Rahul

So I think the first time we saw something like this was with Aurora, the database service. And the tech that, know you, AWS built to separate out the computer and the storage is a pattern that now we see them applying across various different services. So, when Aurora first, you know, was designed, they had this multi-easy replication that they created that allowed, you know, you to A, have all these new instances in the cluster kind of failover very quickly. Replication you know followed asset properties and you can manage all of that stuff within this replication.

A lot of the constraints that were created by the Gap Theorem around the trade-outs that you need to make, started kind of blurring a little bit because of this tricky separation of compute and storage. They took it to another level with Aurora where they started creating a better mechanism of replicating across regions. Now, of course, there are higher latencies when you’re replicating across regions, but given that they have the kind of backbone that they have, they managed to create some optimizations where it isn’t too bad, it is possible to now, you know, replicate data across the regions and failover over there or load balance over there if necessary.

Now, they seem to be bringing that same tech over to a bunch of other storage technologies. Like, I foresee this you know, showing up in, I mean, of course, we are seeing it now in OpenSearch, but the same thing happening in Neptune, the same thing happening in, you know, a lot of other services like the Time Series Database, Timestream. I see this being a common pattern that you’ll see… You’ll also see that what was built for Aurora Serverless is probably something that’s going to happen with a whole lot of other databases like the same pattern of having your compute units being able to scale up and down within 0.5 or 1 times the compute units. That whole mechanism and pattern that they’re able to stamp out across this service is pretty remarkable.

So, you know, I would anchor on the Aurora kind of pattern because right now what I see is AWS Aurora is kind of like the leading service that is trying out, testing out a bunch of these new patterns of replication patterns of separating out compute and storage, maintaining certain kinds of consistencies and guarantees, and SLAs. I would keep my eye out and you know look at all the other data storage services and I expect to see all of those patterns being replicated in the near future.

Rahul

Correct. So just for context, you know, the big breakthrough that Aurora had was the ability to take the data and use the S3 backbone under the hood. And because they had built this amazing Multi-AZ replication into S3, they were able to leverage it pretty much as is to build the storage layer for Aurora. So the minute you write something, it immediately gets replicated across three different AZs and that’s available to you right out of the box as a service. So now that you could look at storage as, you know, through one single lens and all the other replication and all the other needs that you have around storage are taken care of for you, you could then add the compute layer on top of it independent of the storage. So you could scale up compute, you could scale up storage separately, you could replicate stuff, you could failover a lot easier. And that’s been the big breakthrough for Aurora. And now, you’ll get that with OpenSearch. I foresee this happening with Neptune, I foresee this happening with a bunch of other services, yeah.

Rahul

Yeah, at the end of the day, OpenSearch is basically up fork off [SP] Elasticsearch. So, maintaining compatibility given that they haven’t really divulged that much yet has been, I think, fairly straightforward from an AWS standpoint. So, yeah, and when they did Elasticsearch or their older version of Elasticsearch, the managed version, right from then, they had kind of started the process of separating out the compute and storage.

Rahul

Yeah, let’s go ahead.

Rahul

While they OpenSearched large chunks of Elasticsearch which AWS basically took and turned into their, you know, Elasticsearch or OpenSearch service, Elasticsearch as a company still has the entire ELK stack that includes Kibana and Logstash. And ELK as a whole is basically what a lot of customers want to leverage. Then Kibana visualization beats anything that AWS has, and customers want it. So, it makes sense for Elasticsearch to leverage the marketplace, to sell the whole stack as a whole with their terms and their pricing even though it’s on the AWS marketplace and have it be available to AWS customers. So rather than fight AWS on just Elasticsearch part, they are now dealing with the entire stack and are offering it in the marketplace. And that makes life a lot easier for both parties. They you know, fight a lot less and I think customers end up benefiting the most out of this.

Rahul

Yeah, I think it’s a fairly symbiotic relationship at this point for a lot of these providers. I mean, you see a similar story with Snowflake. Snowflake’s entire success is because of Amazon or with AWS, and you know, vice versa. Like, AWS is basically getting a huge amount of workloads moved to their setup because Snowflake, you know, brings a whole lot of those customers for those kind of use cases under their platform. So, I think it is a very, very mutually beneficial relationship. I think you just have to make sure that each party is not stepping on the other one’s toes too much. There is going to be some overlap for sure, but as long as they can see the mutual benefit of that relationship and each one knows how to ride the other’s success, I think customers will stand to benefit the most out of it.

Rahul

Yeah.

Rahul

So, the problem with a lot of your metrics and your data is that there’s just, again, it’s like a fire hose. And to be able to sift through that is incredibly hard. And as more and more of the applications move to this microservices architecture and there’s more Lambdas, there are more subsystems that are producing tons and tons of logs, sifting through that is incredibly hard. And that’s at the lower level. At a higher level, you know, your application could be monitoring customer journeys, it could be monitoring calls coming in, traffic. It could be measuring pretty much anything. And to be able to stay on top of it, to figure out what you should pay attention to at any point of time from an operations standpoint, becomes incredibly hard.

If you have too many metrics, you know, it’s like when something goes wrong in an aircraft and if you had 25 bulbs bleeping and alarms and sirens and all going off, you wouldn’t know where to get started. In code, you know, quality, and this one kind of rings home very close to me, which is a lot of us have started to always use tools like Sonar, SonarQube, or Sonar for our code quality measures. And the one thing that I find happening pretty much all the time is within about a few weeks of setting up a tool like SonarQube or CAST or whatever tool you pick, you start seeing developers shutting off those rules. Because there’s just so much noise that keeps coming in. You don’t know how to filter out the pure signal out of it and then just start, more often than not, just ignoring the rule or the signal or whatever you thought was important to you.

So, while you have all of these metrics, the metrics are only useful when you have a mechanism of A, identifying the outlier very quickly and easily, and B, even amongst those outliers, being able to hone in and filter into only what you really care about actioning on. And not everything is actionable. Sometimes you just want to know about anomalies but they help you do a post-mortem at a later stage. But you need to focus on only what is actionable. You know, what are your decision-making alerts going to be, filter them down to just those, and focus on those. Otherwise your ops team is going to be all over the place. They’re going to be running like chickens with their heads on fire. And that’s not an uncommon scene in a lot of organizations.

Rahul

Correct. And it’s also important to I think, more often than not, it is really hard to create a tool-based anomaly detection. Like you can say, hey, if my threshold crosses this, you know, raise an alert. Because it’s very, in an elastic environment like AWS where your loads might weary your… You know, all the metrics might be all over the place within a week. It’s really hard to set rules because those rules are going to get violated in a very short order. You need a machine-learning-like approach which starts to understand patterns a lot better. And then you’ll have to detect anomalies based on that.

And that’s why something like Lookout for Metrics is really valuable. You’re not just adding simple rules which, again, will overwhelm you with outcomes or completely hide anomalies. Like, you have to be flexible. You have to follow the pattern of the data stream that is coming in and find anomalies within that rather than set very gut-based rules that determine what you think might be an anomaly might not actually be an anomaly in the data pattern. So, I think Lookout for Metrics plays a very, very important role in that first part, which is making sure that you understand and know the right kinds of anomalies that are being raised or that you see in your data. And second is the filtering and simplifying the way you consume all of those insights or those anomalies is what is being made easy over here. So, incredibly valuable.

We actually have a very interesting use case that the team is working on right now which is… We have a product in which we have tons and tons of customer journeys. Like, we have tons of website data that comes in for how…it’s a customer experience product. There’s tons of data that’s coming in about what a customer’s journey is and for some very, very large organizations, a journey could take you to a thousand different bots. I think the largest one we have can have anywhere from 5,000 to 10,000 different bots that a customer could take through a simple journey. And in all of those cases, it is manually impossible to create the idea of an anomaly. Like, I want you to know where to start with creating rules.

So, you need to let the ML kind of decide what is normal. Right? It’s hard to objectively define that in rules as to what normal is, but once you feed the data to Lookout for Metrics, you let it learn what normal is, and then you detect anomalies in that. Suddenly, you know your service goes down and you have, you know, the volume on one particular path dropping off dramatically and moving to another path because some service was down or your login didn’t work or the lead didn’t get generated and therefore didn’t give access to the customer, the potential customer or whatever. You might have a number of scenarios, but to be able to detect that anomaly, create an alert and have someone look at it and fix it right away, is incredibly valuable and it’s not just cloud watch metrics that you can use this for. You can use this for a whole lot of use cases where you have time series data and you want to go figure out anomalies.

Rahul

Correct. And that is not something that you could just determine based on rules, or you couldn’t set threshold based on very simple rules, you would need an ML kind of approach. Or if you have daily anomaly, like you know, for certain services you find that between 9:00 and 10:30 a.m. during weekdays, you see these huge spike of consumption within this microservice and then you find that the rest of the day, everything is pretty much flat or low. But then how would you model that as a rule? Like, you want to make sure that…you’re not going to change your rules every day, you’re not going to change rules by the hour, and you don’t know… Okay, you know, what if that 10:30 a.m. spills over to 11:00 a.m.? Like, would you treat that as an anomaly? It’s just really hard to articulate those kind of rules. It’s easier to let a machine learning algorithm kind of learn what the pattern is and find anomalies within them.

Rahul

Yeah. And if you have a bunch of automation, by the way, you could have this going to an event bridge set up where you might have different routers or rules that then take this event and process it in different ways. I mean, the number of services that you could then you know plumb kind of one after the other in succession is enormous. So, it all depends on what you want to do. In fact, we’re going to talk about another service with Connect where they have incident management or cases that are now available in preview. You could probably take an event of an anomaly and then pass it on to AWS Connect cases. And you don’t need to set up a separate instance or a separate setup for that at all.

Rahul

Correct. And you know, this feels so close to home because like the expression on their faces, I have seen dev teams have the same expression when they get these alerts from their CICD system or things failing, and then the manager’s under pressure to kind of deliver, you know, an outcome and you have a deadline to kind of get your service published and get the software out. And this is exactly the expression that developers have on their faces. They all look at each other and like cross their fingers. All of that, ah, we could probably get through this alert. Like, it shouldn’t matter that much and then they let it go, and then next thing you know is it’s a disaster at the customer deployment but…

Rahul

Yes. So in general, if you look at any of the AWS services or a lot of cloud services in general, the fact that they are elastic in nature, you can scale up or down almost infinitely, one of the DynamoDB’s big claims is that they can scale up infinitely. Right? One of the problems with that is that you can very quickly end up in a scenario where some wrong setup within your application or maybe it’s an external attack of some kind can just suddenly cause a massive spike in utilization of a service like DynamoDB and can give you a massive bill to go deal with. And you wouldn’t be able to stop it because DynamoDB will scale up to deal with the load that’s kind of thrown at it. And that’s a risk.

Now, the proven way to kind of go about this is to say I want to know when my system exceeds a certain threshold of utilization and when it does, that’s when I want to make important decisions. I want to decide whether this is near a real load, where hey, it’s great, I’d had a marketing campaign and suddenly I have tons of customers trying out my product. That’s awesome. So I want to let it go. Or this just feels like an attack. This has come out of nowhere. I didn’t expect it. I don’t know why suddenly there’s this massive load. It’s not going to impact businesses or going to give me a crazy high bill at the end of the month. So I want to make a decision on whether to let this continue or not.

So, Service Quotas are actually an amazing way of managing up a bounce off your elasticity where it puts you back in control. And with that control, you are then allowing yourself to make decisions that can ensure that you are not causing waste, you’re not spending money unnecessarily, and those services don’t proliferate unbridled or unchecked.

Rahul

Yes.

Rahul

Correct. Like, I mean, there are scenarios where you do expect that elastic nature to kind of kick in and move. So for example when Amazon themselves, one of the interesting documents that they published after every prime day is the services and how much they were put under stress and what they could handle. So yeah, when you have something like a prime day come up, you know DynamoDB definitely kind of shoots up in utilization and you suddenly start seeing many millions of transactions per second executing well through it. But on a typical day, it’s not something that you would expect. I would think that you would create some sort of an upper bound on the different API requests and stuff like that, and if something exceeds over there, you raise a flag or raise an alarm to say something is not right.

You know, I think this also goes back to what we talked about in terms of Lookout for Metrics. Imagine you could actually look at your Lookout for Metrics. You can get the statistical averages like figure out what your one sigma or one standard deviation looks like for your particular metric and you could choose that to set up your service quota for that service. So, your system knows and learns what one standard deviation implies and you use that as a mechanism of managing your service quota that way. Even if you have an anomaly, okay, you basically you know, manage it to an upper bound and you can take a call on what you want to do next. So that’s another way of using Lookout for Metrics as well.

Rahul

Yes.

Rahul

Yeah. I have known of a number of AWS customers who have had the scenario where there are these crypto miners lurking around. And the minute they can get access to an API which allows you to launch an instance, they will use it. They will launched the P instance types, which are basically your GPU-centric ones. They are expensive to launch. And they will launch as many instances as they possibly can. And then they will run all of their mining stuff, you know, on it. Now, AWS has done a phenomenal job of creating multiple checks and stuff like that, but one of the big instruments that you have in your armory against that kind of attack is your service quotas because if you limit tech as a policy within our org, we set at literally at the org level down to every account.

Our service policies ensure that you cannot launch a P instance type. If you want to launch a P instance type for machine learning kind of use cases, then specific exceptions are created. And also, it’s just not possible to launch those instances in 99% of our accounts just because that’s control mechanism that ensures that not just external attacks but even internally. That you are not wasting money on launching this instance because unless you have a justifiable case for launching those kind of GPU-based instances, you’re just wasting money. So it’s an incredibly valuable tool in your arsenal to fight waste.

Rahul

Correct.

Rahul

Yeah, let’s talk about Lex.

Rahul

Yeah. So I’m gonna start with a personal confession. Lex is one of my favorite services of just… It’s enabled me to do a lot of things that I would not have been able to do otherwise.

So, the way I end up using Lex, and this is just for some background about why this chatbot tool is so valuable. I have a terrible sense UInUX. And every time, you know, like a big part of my job is to work on product. And I think our strengths lie more in the backend engines that drive all these amazing products. But when it comes to you UInUX, I find myself fairly handicapped. And thankfully we have a good team to kind of overcome those limitations. But I found when Lex first came out, again, one of those light bulb moments for me, where I found that my team and I were really, really good at writing up API to do some very specific operations of things. And it was amazing that I could take a Lex intent and map it directly to an API. And the fact that it was available in the chatbots, whatever output I wanted out of an API, I did not have to build out a full-fledged UI with a bunch of dropdowns and figure out how to get to the right data to pass on to that API call.

I could literally define an intent with slots, the slots being the API parameters, and the output would come right to me in the chat client. Right? So, I could directly interact whatever question I wanted answered. I’d just write an API and that API would be the intent and the parameters that are needed were the slots. And then I could have a conversational mechanism of asking that question. And sometimes I would forget what the parameters are, so I’ll just say, “Hey, I want data on X.” And it would say, “Hey, great. What date range do you want it for?” And then, you know what particular dimension you wanted it on and it would ask me all these questions back to help me fill out these parameters. And it would get me the answer. And I wouldn’t have to build out a full-fledged UI with dropdowns and selectors, and date selectors, and figure out how to construct this amazing UX that wouldn’t feel painful.

Rahul

Correct.

Rahul

Yeah. In fact, like, I think one of the simplest… Here’s a use case that I use extensively. We use a lot of QuickSight dashboards that have tons of data. And I personally feel very handicapped kind of navigating through those QuickSight dashboards because there are so many dials, and unless you really knew the underlying data and how to slice and dice it, it just becomes really, really complicated. So, every time I manage to work with somebody to create a dashboard that I like, to create a view that I like of the world, I immediately take that and write up a very simple API endpoint that says, here’s what I want and here are the four parameters that go into determining the right data.

And once I’ve got that little piece implemented, it’s usually about 10 to 15 lines of code that go into a Lambda, I literally can ask my Alexa device or I could just type into my chatbot saying, “Hey, can you pull up this particular data?” And then it says, “Hey, give me the date range, give me parameter B, give me parameter C.” I’m like, “Okay, what are choice over parameter A, B, or C?” It can give me those answers I fill it up right away. And it gives me the report right there in my chatbot. So it gives me a QuickSight URL, I click on it, and it takes me to the dashboard with all the parameters filled in. And I don’t have to fiddle with the UI to figure out how to get there. It’s one link, it’s got everything built-in, I click on it, it takes me there.

And that I find incredibly liberating because, you know, now, I’m not bound by all the UInUX and I’m not in your UIHell, which is what I kind of dig in for myself. But back to this particular announcement, I think why this is so incredibly valuable is that back when I started, I’m talking about five, six years ago when Lex first came out, I had to write Lambdas for every little chat handle. You know, unless you knew how to write Lambdas, deploy those functions, figure out how you can scale them. If you had 10 parts of the conversation, then you’d have 10 different Lambdas dealing with all this complexity. Now, if you go to the chatbot designer, you literally don’t have to do any of this stuff. You don’t even have to deal with Lambda. You can literally just write up a bunch of your intent, define your slots, give it just a few phrases words so there are phrases that you can understand how to parse those intents, you can set context right there in the UI, and you can try it out right in the UI.

And if it works, great. If it doesn’t, you want to tweak some of those phrases because there’s a different way that you’re adding it, teach it. You can give it a few more, three or four more, and then you’re all set. It works. So that’s the, I think, I would expect to see folks really leverage Lex to create a bunch of automations. The chatbot interface is so underutilized today. If you feel very, very comfortable with APIs, chatbots are probably the best way for you to start automating a whole bunch of stuff and getting value right away without needing to build out complicated UIs.

Rahul

Yeah, and in fact, it’s just the API call. I mean, if you know how to design an API endpoint, the Lex interface feels so intuitive, literally map the intent to the API function name, that’s your intent, and each parameter in your function call is a slot. There couldn’t be a better match for a UX to the backend system. I find it incredibly awesome. It’s very, very liberating.

Rahul

I’ve loved Connect, and I think we’ve had an earlier episode where I talked about how many amazing things they’re adding to this particular service. In most organizations, the human resources that is involved in talking to customers is incredibly valuable and important on so many dimensions. Number one, they are your one touch point with your customer more than any other, you know… Like, your sales folks, your finance folks, your product folks don’t talk to the customer or interact with the customer quite as much as your support folks do. So making sure that you have consistency in the way these people or this organization interacts with the customer is incredibly important to your brand, to your reputation, and you know all that stuff. The more you can automate, the more checks you can put in place, the more consistently you can deliver that interaction or the customer experience, the better your brand gets in the long term.

If you have, you know, two agents, one being really, really customer-centric and focused and the other one not so much, you’re gonna have a very inconsistent experience from a customer standpoint. So, tools like this where you can focus your agents to only handle a narrow set of things, which are really important and valuable, where machines cannot possibly kind of play that particular role, I think adds to the overall brand. A, you’re using people to only focus on the really, really hard stuff that requires judgment, that requires a certain skill and you’re not having them do a very wide range of things in which case you will end up with fluctuations in customer experience. So, the more tools that come out of the Amazon Connect space, I think they are creating a pattern that could potentially allow you to build one of the best customer service organizations on the planet. And I love every one of these new capabilities that they’re coming up with.

I mean, can you imagine reaching out to your customers in near real-time when let’s say a service is down or let’s say you are about to get on the flight and the flight is delayed. Or let’s say there’s an emergency of some kind and you need to get to the customers right away, you need reach millions of customers, tell them exactly what they should do. There is no way to handle this effectively when you’re doing this with people. Humans don’t scale quite the same way. So, the more tools like this which meet all the compliances, meet all the laws, keeping your brand identity, keeping your brand reputation intact. I mean, tools like this are invaluable.

Rahul

Yeah. And to give you a simple example. I mean, I’ll just bring up one real-world use case of this. Every time you receive an Amazon delivery, within 24 hours you’ll actually receive an email that says, “Did you like the product? Did you receive it? Was it in good shape? Did the delivery happen correctly?” And very recently, I had ordered something where I’d ordered two items. One of them came in, the second one didn’t. And I was dreading going into the Amazon site and dealing with support. But because I got the…I received this email, it literally said, “Hey, just tell me whether you received it or not.” And there were four options in that email. I was able to provide feedback without having to log into the Amazon portal or going to support. I mean, Amazon hides their… Getting to support is not the easiest thing on the Amazon website. So yeah, I was able to literally give feedback. They got back within 24 hours. And they had another set of the product shipped back to me, whatever was missing. And that customer experience is night and day versus me having to actually find a person, talk to that person, give them the feedback which is again a very inconsistent interaction that I would have in that scenario.

Rahul

Exactly. Absolutely. I have Amazon Prime accounts in three different countries depending on where I am. And yeah, I do that because of the consistency of the experience.

Rahul

Exactly.

Rahul

Correct. And I mean today the reality is that the case management has become fairly commodity or relative commodity where you use ServiceNow or Zendesk or at licensed products to basically decide. That becomes your case management system and then the pain really is to have all these systems integrate. You have to go through that process, make sure the data is flowing through correctly. More often than not, the data actually resides in multiple different data sources where you might have some data in Salesforce, some data in Zendesk, and you need data from both of them to match up. Disambiguation becomes a problem so Connect kind of helps solve a lot of that problem where you have matches capability within Connect where even if you’re pulling data from different sources, the matches will disambiguate it.

So, I think it’s already there and is certainly going to get better as Connect becomes the center of your entire CRM, customer support operations. The center of gravity is moving into the Connect product or Connect service itself where that’s where your single pane of glass, your single source of truth is going to reside. All the other peripheral systems that you end up using are gonna have some slice of the data, but if you wanted that holistic view, Connect is probably the place to go look at to find out where the real truth lies. And it’s also then the right place to have your agents look into as they’re interacting with your customer because that’s where they have the 360-degree view of what is important to the customer, what do they care about, what should I not say, what resonates with them in terms of a tone. All of that as it resides in one place in Connect. I mean, makes it much easier for agents instead of having to go to four or five or six different systems and try to pull the data out.

Rahul

Completely agree.

Rahul

Absolutely.

Rahul

Thanks, everyone.