AWS Made Easy
Search
Close this search box.

Ask Us Anything: Episode 17

Latest podcast & videos

Summary

In this episode, Rahul and Stephen begin part 1 of a 3-part series in showing #AWS-powered automation, developed with DevSpaces and DevFlows, to show how they have streamlined the livestream process.

You can read more about their process in this blog post.

AWS Made Easy

Transcript

  1. [music]

  2. Man

    Alexa, coffee on.

  3. Alexa

    Okay.

  4. [music]

  5. Man

    Alexa, coffee off.

  6. Alexa

    Okay.

  7. [music]

  8. Stephen

    All right, hello and welcome to the “AWS Made Easy Ask Us Anything” live stream, episode number 17, and this is our “Behind the Scenes” episode. We’re really excited to share this with you. How are you doing Rahul?

  9. Rahul

    Doing very well and really, really excited about the episode today. I think this has been a long time, the planning. We wanted to get it just right to present it to all of you, so couldn’t be more excited. How was your weekend, Stephen?

  10. Stephen

    Oh, really, really good. Just a really relaxing weekend. The kids were getting over a little bit of a cold, so they’re back in business now. How about yours?

  11. Rahul

    Oh, we had a really awesome weekend. We went to this national reserve called Nagarahole National Reserve, which is in the state of Karnataka. Beautiful, beautiful place. Weather was just absolutely awesome. And we spent the weekend, did a safari. Yeah, was overall fantastic. The unfortunate thing is I’ve kind of contracted an extreme case of laryngitis. So, I’m glad I’m not the one speaking for the most part today. But would love to quiz you on all the stuff that’s happening. I think this is gonna be a really, really fun topic today.

  12. Stephen

    Well, that sounds great. Well, let’s get into it. And then let’s see. For some reason, my connection is a bit fuzzy, but is it coming through okay?

  13. Rahul

    Yeah, it seems to be coming through just fine.

  14. Stephen

    Okay, great. Well, on that case, I’m gonna cue up the first segment, and then we’ll get right into it. So, let’s see. Here we go. And then what I’ll do after this, I’m gonna share my screen, and so this will be totally transparent process. Okay, here goes.

    Okay, so you notice it said a “What’s New Review.” Now that’s intentional. This isn’t a “What’s New Review” segment, this is a “Behind the Scenes” segment. But I wanted to show you as part of how the segments get generated for that part. So, I wanted to make sure that we use that very process. Okay, let’s start with…

  15. Rahul

    Yeah, why don’t we just start by telling us a little bit about all the different phases that go into producing this live stream?

  16. Stephen

    Sure. Well, I guess I’ll go through the different ideas, and I’ll share my screen. Actually, let me do a real quick behind-the-scenes here. So, what we do in the live stream, I’ve got the streaming on StreamYard. And then off to the… So, I’ve got my main monitor here, which is you, and then off to the side, I’ve got some notes that I want to go through. And so I’ll just bring those notes onto the screen so we can kind of do this in real time.

    But one of the streams…there’s different phases that a stream goes through. It’s basically… Here, let me share this window. Share, and here we go. Bring this onto the screen, move that StreamYard out of the way so we don’t get that recursive effect. So, stream goes from Planned, to Scheduled, to Ready, Recorded, Post-Processed, Highlighted, Finalized, and Completed. And it seems a bit excessive, but I got to the point where I was sort of tracking all these different pieces of the stream. And then I wrote down, okay, the full flowchart of how it works, and it ended up being quite involved. So, I thought, “Okay, I have to manage this.”

    Let me see if I can share the flowchart of how the stream process ended up looking like because when we first started this, I was doing everything manually. We do the live stream, all the preparation manually. I thought, “I need to do a brain dump of exactly what’s going on.” And I ended up with this flowchart here. I don’t know how visible this is.

  17. Rahul

    It’s pretty tiny. Can we zoom in a bit?

  18. Stephen

    Here we go. Here we go. So, is that okay?

  19. Rahul

    And by the way, it looks super tiny.

  20. Stephen

    That’s exactly the point is I got to the point where, okay, we have this stream, and it’s going through certain stages, it starts with Planning. And then when we go from Planned to scheduled, it means, okay, we’ve got a date locked in, we’ve possibly got a guest locked in, we know what episode number is going to be. And then when we go from Scheduled to Ready, it means that, okay, we’re ready to shoot the live stream. We’ve got the event made. And so that’s what these little tick marks are saying is all the different bits of data that I need to gather up along the way. I’ll zoom in here.

    And then when we do the stream and it’s recorded, that means we have a URL of our live stream. That’s our Episode 17.mp4, which we’ll have at the end of this session. And then we want to post-process it, which means we chop it up into different highlights and segments that we share out onto social media. And so I’ll show you what that looks like. And this is a shout-out to our own AWS Made Easy channel. But if we go down to AWS Made Easy. So, we see that throughout the week, we distribute these highlights. This is our…you and I were in Anaheim.

  21. Rahul

    Yeah.

  22. Stephen

    Talking about the config. We went over this article that’s adding 20 new resources. This is the Rekognition article that we went through. And so we bring out this highlights throughout the week. And then finally, at the very end, everything is completed, which means we’re no longer tracking it. And so I kind of wrote all this stuff down, and I was managing this all in my head. And I thought, Okay, well, we need to automate it plus, you know, we’re programmers, right? So if it’s anything we have to do, it’s more than once, we have to automate it.

    In general, I’m using this tool called ClickUp and we can get into that a little bit later. But you can see my ClickUp We can think of ClickUp as like a programmable Kanban board, and it moves a stream to these different phases. And then basically what the way I think about this stream is that for every phase that we talked about, we say, “Okay, do we have data to move into the next phase?” If not, then it’ll kind of kick me back.

  23. Rahul

    So, it’s basically a simple workflow.

  24. Stephen

    Exactly. It’s just a workflow tracker. But the cool thing is I can stick a webhook during any point in the process, and that webhook can then call out to an AWS function, a Lambda function, it can call out to a DevFlow, which we’ll talk about in a little bit. It’s really flexible. And so that way we can enforce doing processing certain data with at each segment, at each transition.

  25. Rahul

    Got it. That makes a lot of sense. In fact, I think automation in general, just talking about automation. I feel like it’s a really cool way to document complex processes and complex systems. Just because English as a language is probably the worst way to articulate these systems. Just, you know, every time I’ve looked at a wiki, to look at how to do something, I have never managed to get it to run, I invariably need to call up a developer or, you know, have them come online, so that we, you know, do a joint session, and they walked me through exactly what that text meant I was supposed to do.

    So automation couldn’t be a better way to document, you know, all the complexities around these workflows, processes, even if it is just to be done, you know, occasionally, it’s still worth documenting it in that form.

  26. Stephen

    Well, I agree, especially when it’s done occasionally…

  27. Rahul

    Yeah, exactly.

  28. Stephen

    …then I’ll talk about it later but I don’t know how many times I’ve re-taught myself AWS Sam, because I don’t use it often enough to have it burned into my L2 cache of my brain or even the ram of my brain it’s in my, you know, the glacier part of my brain. And so you know that retaught, restarting that process every single time, it’s just often enough where I think I know what I’m doing. But it’s not often enough where I’m sure I know what I’m doing. And so I’ve got a little bit of automation around that. So that way, I don’t have to think about it every single time. And I could just know that it works.

  29. Rahul

    Couldn’t agree more.

  30. Stephen

    Oh, well. So, I think that kind of makes the case for the automation in terms of live streams. Do you have any other questions about what we’re doing? Or should we go on for the next one?

  31. Rahul

    No, why don’t we go on to the next one?

  32. Stephen

    All right, so then I’m going to trigger transition to. Okay. Oh, I see. One thing ClickUp has done is it’s reshuffled the order of my transitions. So, I have to deal with that in a moment. That’s really strange. Okay. So, in terms of tools, what we use, so I use ClickUp, as we said for…this is the Kanban Board, this is the issue tracker, this is what’s tracking my stream.

  33. Rahul

    Can you zoom in a little bit on it?

  34. Stephen

    Sure.

  35. Rahul

    It looks super tiny.

  36. Stephen

    I’ve got the… Okay, so we’ll go over it as we get to a new stream. But the idea is, I track… I’ve got a list called Live Streams. And in that list, I’ve got two, what we call sub-lists. One is called Streams, and one are called LS Tasks or Live Stream Tasks. And those are associated tasks with the stream. And so if I wanted to make a new live stream… Here, I’ll show you what it looks like for Episode 16 that we’ve done recently. Okay, Episode 16 is in the Post-Process state. It’s got a few segments that I can review.

    Let’s look at Episode 15. Okay, so Episode 15, we’ve made a few segments, we’ve made a few highlights from those segments. We’ve made social media posts from those highlights, or there’s one here where I’m still fleshing out the design of the thumbnail. So, we’ll make sure that that gets posted, and then that will get promoted through the process. And they all tie back to this stream process. And so the stream process, it says, okay, this is a task of type Stream. It’s Episode 15, here’s the title. Here’s the LinkedIn event that it said, here’s our promotional image for that episode. Here’s the recording of it in an S3 bucket, here’s the show notes that came out of that. That popped up on my different window, so I’ll move that here. Here’s the show notes where we talked about each thing that we did. So, this is kind of a one-stop place to manage the stream. And all these fields get populated as the automation progresses through. And then…

  37. Rahul

    This is a pretty neat way to document all of this workflow.

  38. Stephen

    And so the part that I really like, and we’ll get through… So, if I go over to Streams, and I can go to Automations. And this is saying, okay, if I change the status of…well, let’s look at this automation. So, this automation is saying when the status goes from Planned, and then it advances out of the Planned state…Oh, this is saying, “If I don’t have this episode number, then kick it back.”

  39. Rahul

    Yeah.

  40. Stephen

    But then I’ll have another automation that says…Here we go. This is where it gets interesting. When the status changes from Planned to Scheduled, then it’s going to call a DevFlow webhook. And we’ll look into this DevFlow webhook in the next one. And so it’s saying, “Well, now that I’ve actually nailed down a date, well, let’s automatically send out a Google Calendar invite.”

    So, the tools we use to manage this, we use ClickUp for the management, we use DevSpaces as our IDE to code all these. And then DevFlows is our automation tool. And we can talk about that. That’s a tool for wiring up AWS services together. We’re gonna use AWS Rekognition. We use StreamYard for the streaming, and then ClickUp for task management.

    Now, one second here, again, this is behind the scenes. In our StreamYard, I’ve got these transition tasks. And I noticed that somehow transitions three, four, and five vanished. So, I’m gonna these back in. So, we can actually queue them up. There we go. Transitions three, four, and five are processing. Okay. So, should we go on to the next bit of the conversation here?

  41. Rahul

    Yeah, absolutely. Go ahead.

  42. Stephen

    Let’s do it.

    Okay, cool. So, we saw that transition three ran, and that’s what we’re saying. For segment number three, we wanna talk about scheduling the live stream in Google Calendar using DevFlows. So, I’ll open up this DevFlow here. Now, I really like this tool, DevFlows. I think what you said earlier, it’s about documenting the automation. And in this case, it’s saying when something happen… The best way for me to document things is a flowchart, right? If you have some really big, technically correct, philosophically complete document that painfully texturally enumerates every edge case, it’s painful. Whereas, this is a pretty simple flowchart of exactly what’s happening.

    So, just to zoom in, it’s saying…so here’s the other end of this transaction. Here’s a webhook that comes in, and I’ll go to the Edit View here. So, this webhook is gonna come in, and then some of the data that’s going to be there… I’ve used this Reshape tool. Let’s look at the configuration here. I have an event coming in, and the event has a body. And in this body is going to be guest email, appointment start date, appointment end date, appointment summary. Those are what’s going to come out of that event.

  43. Rahul

    Yeah.

  44. Stephen

    And then I’m sending this to Google Calendar. And so in this Google Calendar, what I’m saying is I want… I’m setting it to… Okay, I’m experimenting with the times here. I’m giving you the appointment start date and end date. And this is conference calls to plan the meeting for the live stream. And then we can populate a Zoom one. There’s some personal information in the production one, so I’m showing a demo on here. But yeah, the important parts here are the start and end dates, and that gets translated from the incoming webhook into something that Google Calendar can then use to put something on a schedule.

    So, if we try that, let’s show how that works. So, let’s come up. Let’s go back to this view here. We’ll filter. And I’ve got a demo episode here in our live stream view, episode number 9995. Okay, so right now it’s in the Planned state. And if we locked down a scheduled date, let’s schedule this hypothetical episode for tomorrow at 7:45 in the morning. Okay. And then if we advance this from Planned to Scheduled, then this webhook is going to get called, and it’s going to say, “Okay, take this.” The webhook will get called, it will get a date, and then it will translate that date into one for the live stream and then the other one for the live stream minus 24 hours.

    I’ve got a JSON audit in there. I just heard my Google Chat ping. And then so I should look on my Calendar now and I will get… This is a behind-the-scenes. This will pop up in a moment. I’ll get a… whoops, let me just turn off that Calendar. This will pop up in a second with the appointment. Let’s refresh that.

  45. Rahul

    So, while that’s coming up, I do wanna, you know, talk a little bit about this new paradigm…

  46. Stephen

    Yeah, please.

  47. Rahul

    …which is what DevFlows is kind of pushing. So, this is what has been known as the flow programming paradigm. One of the big changes that has happened out there is that as the cloud providers have been providing this amazing set of services, the new skill that developers need is not so much about writing code, but more about stitching together these services to achieve the outcomes that you want.

    And in those kind of scenarios, I think the most readable, the most well-structured or well-architected solutions are the ones that are best represented by a flow because invariably, there’s data that’s flowing through and transforming through all of these different services. And what you need is the right way to visualize the way the data is actually flowing through all of these different, you know, endpoints. So, it’s not so much about writing code, but about how you’re stitching these together. And that’s where I think flow programming and DevFlows as the platform, you know, comes in really handy. It’s a pretty neat platform. We’d love to create a bigger, deep dive at one point of time on DevFlows itself.

  48. Stephen

    Yeah, I really enjoy. And just to show the kind of the power of it, you know, here’s an incoming webhook. You can say, “Okay, well, what do I want to do after that webhook?” You can do a bit of math, you can post it to Google Chat, you can manipulate that and have it feed S3. There’s lots of different options.

    All right. Well, let’s go into the next one. I really like DevFlows. Like you said, we’ll do a deep dive on it. But this is just showing one little piece of that puzzle of, okay, we have a live stream set up, we wanna make a planning appointment, and we wanna make sure we pencil in the live stream appointment, and then we can inject the StreamYard URL, which you’re gonna need into that Google Calendar. All right. Well, let’s go on to the next one, which is then making the segments. All right. Okay.

  49. Rahul

    You got a bunch of voodoo going on with this one.

  50. Stephen

    This one’s cool. I’m really, really happy with how this one works out. Okay, so let’s look at our test episode number 9995. And then again, since this is a behind-the-scenes, usually I have my notes off to the side, but I’m just going to do it right here. So, what is a segment? This is cool. Now a segment is a pre-planned thing that we’re going to talk about. So, as we’ve been talking throughout this live stream, a segment is, okay, here’s a discrete piece of the conversation, we know ahead in advance what that’s going to be.

    Now we generate this little video transition. And what we’re generating that video transition for is for Rekognition to later chop up our video into different segments so that we don’t have to do that manually. So, that’s what I was doing for the first 12 or 13 episodes was watching the episode in advance, taking a note of the time in milliseconds that it started and ended, and then making the highlights from there. But that was a really painstaking process.

    And then plus Rekognition has… Well, we’ll talk about what Rekognition is later, but the idea is we basically want to make our show so that it’s really, really easy for Rekognition to discuss when we’re talking about different segments. So, I think what I can do is just show you what it does, and then we’ll go through how it works.

    Okay, so here’s our live stream view. We’ve got this demo live stream number 9995. And then, in that demo live stream, we want to…let’s say that we wanted to talk about some segments that we wanted to talk about. I think I wrote this down here. Let me do a different filter so I can show you this one. Greater than 99, 100. Okay. Here’s another test episode that we’ve made. And we have these type of segments. So, let’s make another segment that’s basically the same. I’ll show you what the test episode looks like.

    So, I’ve got a test episode that starts out with our standard introduction. And then I pull up a segment and then we talk about something on the browser window, and then another transition plays. And so I want to generate these videos for these transitions. Okay, so now that I think I’ve set up the problem, actually let’s go ahead and do it. So, let’s make a new task, and let’s call this Segment, and we’ll call this the Intro Segment. And this, we’re gonna call it a Task of Type Segment. It’s gonna be episode number 9995. And this relates to our streaming segment. Sorry, stream, excuse me. This is our test stream up there. Okay.

    Okay, so here’s our test stream with new transitions that we’re making. All right. Here we go, Demo Stream. Sorry. Let me filter this back to equals 9995. Okay. Let’s make our segment. So, we’re going to make a segment in LS Tasks. Segment: Introduction YouTube DL has a JavaScript Interpreter. Okay, so this is of type Segment. It’s episode number 9995. It is associated with Stream Task of Demo Stream. And then we’re gonna call this segment title, “YouTube DL has a JavaScript Interpreter.” Okay, let’s create that task. So, that’s gonna show up here.

    Now if I make this view a little bit wider, right now, this is an open task. We don’t have anything. But then what I can do is if I advanced this to In Progress, it’s going to then call a webhook, which is going to give us one of these transition videos. So, if we refresh this. Refresh the ClickUp UI. And you should get it back fairly quickly. It might take another second or two.

  51. Rahul

    And usually, Stephen, how many episodes or how many segments do we create in our one-hour video?

  52. Stephen

    So, it’s usually about six. Okay. All right, so now it came back, and here’s what I’ve got.

  53. Rahul

    That looks pretty neat.

  54. Stephen

    Yeah, and so what it did is, again, within the LS Tasks automation, I’ve got an automation which says, “When this happens, okay, generate transition video when segment goes from Open to In Progress.” And this is calling this…which I have to change now. It’s calling this Lambda function URL. And so what all that does…well, it’s actually an interesting pattern that evolves. Let me go back to this view. So, I found that when we have this automation that uses this webhooks, we actually have to do this two-step pattern.

    Now, usually, automations, they want to return instantly. But it takes a few seconds to generate this video, and so what we end up doing is the first Lambda will push a task to SQS, and then the second Lambda function will… I’m allowing that to take the 10, 15, 20 seconds that it needs in order to generate that video. So, that’s what we’re doing here. We generate this segment, and later, we wanna say, “Okay, this is the segment order. This is going to be the first segment in our live stream.”

    We can say, “Okay, is there an article that we’re discussing here?” Let’s see if there’s an article that we’re discussing. I had this in my notes here. Yeah. We’re gonna discuss this article here. Article URL there, and this is… And in the segments also, once we’ve discussed the segments, sometimes we rate them as either Simplicity or Complexity. And so this is where I’m tracking that. After the fact, I’ll say, “Okay, this one is Simplicity, and then we’ll give it our number of clouds that are five. And so I’ll put this here. Okay, so that’s our segment.

    And so what we do is once this video gets generated, we then go into StreamYard and we have to populate. So, just sharing this here. These are the different segments that we populate so that while we’re talking, we can then queue those segments. So, showing you the source of that code again, and that’s a bit… Here we go. We open this up here. I have another window here that has Lambda function. Give me one second. Okay. My green window is the one that has… I thought I had refreshed this session. It had expired.

  55. Rahul

    And once you get that working Stephen, can we zoom in a little bit into that?

  56. Stephen

    Absolutely. I’m gonna need to use my phone because I’m going to need to use my MFA. So, give me one second.

  57. Rahul

    Always the best practice, use two-factor authentication.

  58. Stephen

    All right. And my phone is also my camera, so excuse me for one second. Authenticator 242609. Okay. There we go.

  59. Rahul

    And we lost you, Stephen. Lost your video.

  60. Stephen

    There we go. So, this will be a separate behind-the-scenes, but what the camera does is it runs this thing called Camo because my iPhone camera is so much better than a webcam, and so I just use that as a camera. Okay, so here’s my Lambda function that’s really doing all this stuff. It’s calling this API called Shotstack. And so Shortstack basically takes a JSON payload that represents the video that we wanna make. So, what it starts with is it starts with this blank transition template. It starts with this blank transition template. And then there’s this big spot right here. And then what the Lambda function does is it gets the right data out of the task. It says, “Okay, here we go.” It gets the right data out of the task and then injects the title into that video.

    And then this Lambda function has a longer timeout. And then here is where we update the ClickUp custom fields to have the URL of the video. And so then once that populates, I can then download them and populate StreamYard, I don’t forget to do that because I’ve done that several times when, oh, gosh, I forgot to generate the transition videos. It’s going live in five minutes. So, having the automation done is really, really helpful.

  61. Rahul

    Yeah, by the way, even though this seems like a live stream, and, you know, might seem like everything is happening, you know, as it happens, that does happen as well. But there is a ton of planning that goes into the structure of each show. We do plan for a lot more than we actually get through in every episode, usually. So, that planning is incredibly important if you wanna do a show and, you know, keep it interesting.

  62. Stephen

    Oh, I really underestimated it. But now that we’re in that streaming world, I’ve seen some ratio of minutes of live contents to minutes of planning and it’s, like, 1 to 20.

  63. Rahul

    Yeah.

  64. Stephen

    Or 1 to 50. It’s a lot. Now this one, we’re showing all the live stream, all the automation that went into it. But anyway, I think what we end up with when we do all these transitions is we’ll have this live video that StreamYard will give us a recording at the very end of it. And because these transitions have these black frames here, this is what will then give us our highlight detection, which we’ll demo coming up. All right, well, anything, other questions for the segment generator?

  65. Rahul

    No, this seems really cool. The fact that you can actually take a base template and then just insert all the specific headers and it generates all the standard segment, you know, markers for you, that’s pretty cool.

  66. Stephen

    And just as a little shout-out, the API is called Shotstack. And they have the cleverest advertising that I’ve seen because what they actually do is they publish a ton of how-tos and how to use FFmpeg. And then what you realize when you’re reading this is I’m never ever, ever, ever gonna be able to get this right scripting this together. FFmpeg is so fragile. And at the very end, they’re just like, “Oh, you can just pay us a couple of bucks and we’ll do it all for you. Here’s a JSON payload that you can do.” And yeah, scripting FFmpeg is a… I’m glad that they’ve done that, and I’m happy to pay them. And it goes with our stitching things together, right? Don’t invent it yourself.

  67. Rahul

    Exactly.

  68. Stephen

    Stitch together an existing service. This is theirs. I don’t wanna know how many special edge cases they’ve dealt with, but I’m happy to share that cost with all the other Shotstack clients.

  69. Rahul

    Completely agree.

  70. Stephen

    All right. Well, let’s do a quick break, and then when we come back, we’re gonna show DevSpaces to deploy Lambdas with function URLs because that’s been a really handy bit of this, especially with these automations when you want a quick webhook. Lambdas with function URLs are the way to go. So, let me trigger this break.

  71. Female

    Public cloud costs going up and your AWS bill growing without the right cost controls? CloudFix saves you 10% to 20% on your AWS bill by focusing on AWS-recommended fixes that are 100% safe with zero downtime and zero degradation in performance. The best part, with your approval, CloudFix finds and implements AWS fixes to help you run more efficiently. Visit cloudfix.com for a free savings assessment.

  72. Stephen

    Okay, cool. So, in this segment, I’m gonna be a little bit more generic. But when we’re talking about automation, a lot of times, like I said in the previous moment, you want a quick function to exist in a way that’s callable in public. You have a little bit of Python code, it’s really just applying a little bit of logic and updating something somewhere. And that’s a pretty common pattern, and so you want a function to exist somewhere that’s publicly callable.

    Now, Lambda has been around for a long time, but then making it public involved API gateway and dealing with cores, and it was a bit more involved.

  73. Rahul

    Don’t forget IAM.

  74. Stephen

    [crosstalk 00:41:35.200] IAM. Luckily, there’s now a cool thing called Lambda function URLs. And the idea is that you can deploy a Lambda function and then right away you can… You deploy Lambda function and then it just has a URL and you can easily select your authentication mechanism. And I usually just have it open and the fact that the web… I don’t publish this URL as kind of the security I need for these webhooks. So, what I did, I made a template for how to very, very quickly deploy a function to Lambda. So, I wanna show how that works, and this is using DevSpaces.

    So, the way that this works, again, as I mentioned earlier, I always find myself re-teaching myself SAM, re-teaching myself some of these things, you know, because I don’t do it all the time. So, what I wanted to do is kind of automate this process. So, what’s the easiest way to get some code public? So, we’ve got this repository, and I’ll put this in the chat, but it’s called SAM Lambda pi function. And the idea is I wanted to make it as easy as possible to have a function made public.

    So, let’s start this with eyk.devspaces.com. Now what this is doing, it’s cloning that repository. And then I’ve got my DevSpaces set up. So, I have my AWS environment. My credentials that work with SAM are already built in.

  75. Rahul

    And the good thing is, you’re not setting this up on your bespoke machine.

  76. Stephen

    No.

  77. Rahul

    It’s available and identical to literally anyone out there who, you know, launches this DevSpace.

  78. Stephen

    Yeah, yeah, that’s exactly right. Anyone can use this. They can just clone this and say, okay, you have to make sure… And it’s documented right here. You wanna make sure that you have your AWS access key ID, AWS secret access key, a default region set, and then you’re gonna wanna set this environment variable, which is the name of your stack. And I’ll show you how this works. It’s pretty cool.

    So, if you look at the DevSpaces, and I’ll be very careful with what I show here, but within my variables, I’ve got access keys that are context-sensitive for different projects. And what I’ve set it up is that for all my projects that start with SAM, I’ve got a certain set of access keys that are pre-programmed in there. So, when I fork this repository, I can just call it SAM something else. And then I’ve already got the right access keys built in. Now I’ve got one setup step here that takes a little bit longer. Here we go.

    I’m waiting for this one step to finish. I just have a… I should have moved that into the base image of that. But this has been a busy process. So, let me pull up CloudFormation while we’re waiting for this to finish.

  79. Rahul

    So, the nice thing about DevSpace is that on startup, it’ll pretty much install and deploy and do whatever else, the Grunt work with compiling and deploying your applications is, and it’ll do that for you without even, you know, setting up a terminal and then getting it all to work. So, it makes it really simple and standard to get started. Looks like it completed.

  80. Stephen

    Yeah, it’s all set. So, say I wanted to deploy some function, I’ll make it really easy here. Instead of Hello World, I’ll say Hello DevSpaces and it’s automatically going to save. And then I’ll go back to the documentation because I always forget what I’m doing here, and then say, “Okay, those three variables get defined through my DevSpaces config.” So, all I really want to do is I can do here. I’ll call this the hellodevspacesstack./ build.sh. There we go. And now it’s kicked off a… what’s going on here? User. DevSpaces EYK.One second here.

  81. Rahul

    You don’t have a right user here.

  82. Stephen

    It should have been, let me check my environment really quick, though I have to hide the screen for a second.

  83. Rahul

    No problem. And by the way, for the audience, while Stephen is trying to get this to work, check out DevSpaces at devspaces.com or devspaces.com. It’s a pretty neat tool where you don’t need to set up your local environments. You can take any application, get started with coding. It’s probably the only environment out there that supports ARM-based instances for you to deploy all of your development stuff. And it is all on AWS.

    So, if your resources are running on AWS, there couldn’t be a better mechanism of, you know, deploying your development environment, coding over there, maybe testing it, and then literally doing a pull request right there and setting it off for deployment or production. So, it can be beautifully integrated with whatever CI/CD mechanism you have. It all operates via your GitHub or GitLab, or whatever, you know, Git-based solution you have for your source control. It’s really that simple. So, give it a shot, give it a try, or reach out to us. We’d love to get you set up.

  84. Stephen

    And we’re back. I guess I hadn’t rotated those keys through. But yeah, so there we go. Right away it’s doing this change set, and it’s creating this whole SAM template, and you can see it in CloudFormation. I’ll refresh over here. And there we go. This is creating in progress, the Hello DevSpaces stack. Again, all we did, the only change we made is we modified this app.pi, writes that little bit of business logic that you wanted to throw in there. There it is. And then as soon as this finishes creating, we’re gonna get an output and the output is gonna… There we go. I can refresh that.

  85. Rahul

    Almost done. Yeah. Now it should.

  86. Stephen

    Let me check out the resources. This is the last thing, of course. Yeah, the last one.

  87. Rahul

    There.

  88. Stephen

    There it is. So, now I’ve got this endpoint. And not the most sophisticated bit of code, but look, this took just a few seconds and we’ve got a publicly running bit of Python code that’s right there on AWS and you’ve got your whole config saved. You can commit this back and it’s just done. So, really happy to be using DevSpaces for a lot of these bits of automation behind the scenes.

  89. Rahul

    Yeah. And I have to admit AWS Lambda function URLs are currently my favorite service because it makes stitching together multiple services together. They’re tied with some Lambda to code so easy that you can just literally take any functionality deployed as a function URL. And that’s now available to you to stitch together as part of a larger chain. So, yeah, couldn’t be more delighted about function URLs.

  90. Stephen

    And what I’ll do, I’ll modify this this template here to be a proper GitHub template, or you can just…what do you call it? Fork this and populate your own environment variables within DevSpaces. Off to update mine. Yeah, but all you have to do is fork this, set your AWS keys correctly, and there you go. You don’t have to go through that whole process. You don’t have to install SAM CLI locally. You don’t have to make sure that you’re running the right version of Python locally. No local setup at all. So, I really like that.

    All right, well, let’s go to the next segment. And I can see already that we’re gonna have to do a Part 2 and a maybe Part 3 of this behind-the-scenes series, but let’s keep going here and see where we get. Okay, here it goes.

    All right, now I’m going to turn off the camera on ourselves because we’re about to get this recursive effect. That’s not too bad. Okay, so here is StreamYard. Now, StreamYard is what we use to stream with. And we’d love to get someone from StreamYard on the show eventually, just because it’s such a really useful tool. So, you can see right now we’re streaming to LinkedIn, to YouTube, to Twitch, to Twitter, all simultaneously. All those transitions that we’ve made, we’ve got them up on the side. We can inject other things into the stream. So, when Rahul and I talk about an article, we can give it four and a half clouds.

    Oh, and I forgot about my other banner that I made for the previous segment. So, you can really have a lot of fun in StreamYard. It’s made live streaming really easy. The first live stream I did that was for a different event, I tried to do all this custom work in OBS, and we came to the Q&A session and everything was in the wrong place. It was really hard to manage. And so, StreamYard is a lot less flexible than OBS, but it’s one of those…like, it just works. Once we started using it, we haven’t had to think about it. It’s been really nice. Yeah. It would be really nice that they had an API so I can stack this deck automatically and put it into my transitions, but they do not have an API at present. So…

  91. Rahul

    I think we could do, you know, something like a Selenium script or something like that to kind of do this in a headless manner?

  92. Stephen

    That will be cool.

  93. Rahul

    Yeah, I think that’s something to try out. But just recollecting, didn’t we do our first episode with OBS?

  94. Stephen

    Yes, exactly.

  95. Rahul

    And really struggled with.

  96. Stephen

    Yeah, so OBS, it works really well until we got the live Q&A at the very end. And what we were trying to do is we were trying to use Zoom and then stream capture parts of Zoom and push it into OBS. So, like with StreamYard…and again, I’ll show you. I’ve got my Stream Deck over here. And if you look over here, I’ve got buttons programmed in that represent different StreamYard views. And so if I want to switch… Let’s see if I can do this here. If I want to get rid of the split screen and just do group, I can hit this and we’re there.

    And then I’ve got…so this is really this. There we go. Got a cinema mode, and we talk with the article. And so I’ve got this kind of by my right hand while we’re doing the live streams. So, that’s a Stream Deck, and it pairs with StreamYard. And I’ve got some macros programmed in that will switch the different views. So, I’ve got a little bit of control over. I always think back to the future when Emmett Brown is looking at the Sony handycams. This is a portable television studio. I think this is like, I don’t know, a TV station as a service. So, I’m very happy to be using this.

  97. Rahul

    If I’m not mistaken for the first episode, Stephen, I think you had most of your family being part of the coordination, you know, and production team.

  98. Stephen

    Oh, gosh, and my brother was moderating the live stream. And yeah, StreamYard gives you everything all in one place. It’s really, really neat.

  99. Rahul

    I think we had, like, more people behind the scenes who were literally just making sure that all the other pieces were being scheduled and, you know, showing up at the right time. It was quite a pain, and I think we’ve come a really long way since then.

  100. Stephen

    It’s been really fun. And I’m excited to do a behind-the-scenes episode, you know, in six months from now and seeing where it’s going to get to. All right, well, let’s do… I think we have time for the using Rekognition. I wanna give the remaining 10 minutes for that. And I think we have to do the other things another day. But let me queue up the Rekognition transition. Here it goes.

    Okay, so clearing up my screen here. So, let’s go back to our view here. I’ll make this a bit bigger. So, we’ve got a stream, and this stream is referencing this little mini-episode that I made where we’re going over, you know, just a news article on Hacker News. And we’ve got all those things there. So, let’s go over how we would kind of post-process this. So, we’ve got one segment there. We need to make another segment. And let’s call this segment Java Commentary.

    And this is forcing me to make this of type Segment, 9995. Adding the Stream Task. This will populate well Demo Stream. And then we’ll call this Java Commentary. Okay. Save that. And I will make one more. Let me clean that up a bit. Okay. Segment. And now this will make sense in a second. Wrap up. Okay. We’re gonna make this a segment. So, the segments are things that we’ve planned in advance, and they have a title. We’ll call this Java Commentary. This one is going to be… We’ll save that. Link it to the parent. We need to call this a Segment. We need to link it to the parent task. Let’s see, Demo Stream, there we go. That should be what we need. Oh, I forgot the… Let’s see. Where is it?

    One second. I must have forgot a field here because I have this view. I think what’s happened… Let me disable this filter quickly. I wanna make sure that this works exactly how I want it to. Okay. So, we’re in the segment view. Oh, yeah, that’s what it is. I forgot the… Now I can turn my filter back on. So, we’ll filter by the episode number as 9995. Okay, sorry for the little bit of setup.

    Yeah. Now it’s here. We’ll call this Wrap-Up. These are all in progress. So, our previous automation should have… And so let’s say we’ve gone through the show, this one’s Complexity. We’re gonna give this a one. This one is Simplicity. We’re gonna give this a five. Let’s do a refresh. Our other automation will have run and we should have these transitions. Oh, and this is important, the order that we’re gonna go through them is two, and three. And let’s also give them URLs so I have something to reference. And let’s have that one be that URL.

    Okay, that sounds good. Okay, so say we’ve done this live stream, and it’s all set. Then what I would do, and this part is still manual because StreamYard doesn’t have an API is that I would set this recorded S3 URI field. Now I’m gonna set this to this. Here we go. Set that here, code it S3 URI. Okay, that’s happy with that now. So, now when I move this from, so say it was in the recorded state because…actually, I think some of my automation, I wanna make sure, okay, it was ready and then we’ve recorded it.

    And so now that it’s in the recorded state, the automation is gonna fire, which is going to launch a rekognition job. And what rekognition is going to do is it’s going to look for all the segments, and it’s going to figure out how many segments were in the show. And then if the amount of discrete clips it found matches the segments, so, in this case, it will find three like this. It’ll look for this pattern. It’ll look for a black frame, and then that transition video, and then some amount of content, a black frame, that transition video, and then some amount of content. If it finds the same amount that we’ve already pre-planned, it’s going to make a highlight that matches to that.

    So, let’s refresh. And if we go to an overall view, we should see that start to work. There they are. So, cool. Okay, so what ended up happening…and it seems like we got it just on the cusp of an update there. Okay, so what it did, and let’s go to the highlight view, and this will make a lot more sense. So, this is 9995. Okay. So, what we’re seeing here is we’ve got… This is the cool thing. Rekognition populated these fields. I’ll zoom in a little bit more. Rekognition populated the fact that this happened here at 00:00:35:10 and it ended at 00:03:11:08. And so now if we go over to our video and we go to a certain timestamp, and let’s see, what was it? We’ll go to 00:35.

  101. Rahul

    And there we go.

  102. Stephen

    Yeah, that’s it. And now if we go to the next one. So, the next one, so see our transition segment is about 10 seconds. So, this one ended at 11 and this next one starts at 28. So, the way that works in the background, and I’ll show you the script here. We’re getting close to our time. We have this huge Rekognition. Now Rekognition, it’s kind of fascinating. The key here is that we’re looking for this, what’s called a technical queue. And the technical queue that we’re looking for is that there’s a black frame here. And what this coverage percentage is saying is if this black frame is covering more than 60% of the screen.

    Now if you dive into the documentation, one thing that’s interesting is a black frame technical queue only works if there’s also no audio. And StreamYard does a thing where when you trigger a transition, it tapers off the audio over about half a second. So, that’s why that black frame has to be a second long. If you had full control over the video, you could just do it for, I don’t know, a couple of frames, but it’s a bit extra long at the moment because I’m compensating for that audio taper off to make sure we get a little piece of it where there’s no audio.

  103. Rahul

    Stephen, if we can take just one minute to tell the audience a little bit more about Rekognition because I’m not sure we’ve actually covered that in depth. [crosstalk 01:03:53.686] earlier.

  104. Stephen

    Oh, sure sorry. I got so excited I kind of jumped ahead, sorry.

  105. Rahul

    Just very quickly, Rekognition is actually a suite of API that deals specifically with both image and video-related AI services. That could be anything from detecting people, detecting faces, detecting, you know, sentiment, detecting, you know, not safe for work kind of content. So, there are a lot of very interesting things that you can do with Rekognition. They have a very vast set of API that are there to be leveraged. So, go check it out.

  106. Stephen

    So, this one, in particular, is called the Rekognition Segment API. And what it’s doing is it’s detecting… Sorry, I got so excited and I was also watching the clock, and I got this and it worked.

  107. Rahul

    It always happens to us.

  108. Stephen

    So, yeah, what you do is you give it a file in a bucket and you tell it what to look for. And in this case, we’re using a technical queue. I have typed “black frame,” and then once this job runs, it will broadcast out to an SNS topic when it’s finished. And then so I have… this is actually using that two-function paradigm where one of them starts it and then another one is waiting for the result to then update it.

  109. Rahul

    Yeah.

  110. Stephen

    But the neat thing is…yeah. And that’s one thing I’ve seen through this whole automation process when you’re making these webhooks is you really want to have that pairing. You don’t want webhooks to be synchronous, and most of your webhook providers don’t want that either. And ClickUp says, you know, if you make a webhook that takes a minute, they’re gonna disable that automation.

  111. Rahul

    Yeah. Completely understand.

  112. Stephen

    Yeah, so this is Rekognition. I’m working on a blog post that’ll kind of show all this stuff and have a repository that you can play with in DevSpaces and deploy the automation stack in SAM. You just have to fill out your AWS variables, and then you can basically have a very similar stack. The idea is…and we’ll have to go over this next time, but now that we’ve got these highlights, I’ll just give a quick preview.

    So, when we have these highlights, this is saying here’s the full length of the segment. Now on LinkedIn, you can do a 10-minute video as a highlight, but on Twitter, you can only do 60 seconds. So, what we do is we have a preview highlight. So if we did say…and again I’m doing this blind, but say if we start at 35, but then we wanna end it at 1 minute and 15, that’ll be okay. So, we’ve got that. And then let’s jump into this one. And we’ll add a thumbnail because we need a thumbnail.

    Let’s see. Thumbnail image, upload the file. This will be Episode 17. There we go. This will be the thumbnail for our video is that. And then if I move this over to In Progress, then this is going to then take these parameters and it’s going to render them in Shortstack into these videos that we can then share out on YouTube, on Twitter, or LinkedIn. And it’ll make the full version for LinkedIn and it’ll make the shortened version for Twitter. Let’s see how that’s going. That will run. We’ll have to do that next time because now it’s 10:00. We’re already two minutes past.

    But yeah, that’s how we use Rekognition to do this segment analysis. And this has been a real time saver because the first 13, 14 episodes, it was really just moving…it was in VLC, with my notepad writing down these timestamps and getting that to work. It was time-consuming. But if you can control your transition videos and have that black frame, then you can basically make Rekognition do all that work for you. The Rekognition Segment Analysis API. All right. Well, I think [crosstalk 01:08:33.459]

  113. Rahul

    Thanks so much, Stephen, for walking us through that. And I know that there’s tons more to go through, but look forward to the blog posts as well.

  114. Stephen

    Yeah, I’ll get that finished up and post it along with some code. And we have to do a Part 2 and maybe a Part 3 of this. And I guess that’s where… I realized there’s so much there. And there’s a lot of little tasks, like getting the thumbnail lined up, making sure the thumbnail is ready when I wanna make the highlights. And that was a problem I ran into earlier is I’d be all ready to go and I say, “Oh, I forgot to ask the designer to make the thumbnails,” and then so that would be a 24-hour turnaround on there.

  115. Rahul

    Yeah.

  116. Stephen

    And so being able to have all this flow controlled and all these reminders in place to add this structure, it’s been really fun, and using DevSpaces and DevFlows because it makes it really easy to just have these quick functions that…yeah. I couldn’t think of a faster way to get a Lambda function just sitting out there on the public internet than using a DevSpaces template.

  117. Rahul

    Couldn’t agree more. Anyway, thank you so much for the wonderful episode. And the audience, please feel free to subscribe. Leave us a comment. Let us know what else you would like to know. We will see you soon. Next week.

  118. Stephen

    Yeah, let’s see you next week. Maybe we’ll just do the… There’s Part 2 next week.

  119. Rahul

    Absolutely.

  120. Stephen

    All right. All right. Well, thanks again. And I’ll cue the intro.

  121. Rahul

    Bye.

  122. Stephen

    Bye, bye. Here we go. I gotta cue the credits, and there we go.

  123. Female

    Is your AWS public cloud bill growing? Well, most solutions focus on visibility. CloudFix saves you 10% to 20% on your AWS bill by finding and implementing AWS recommended fixes that are 100% safe. There’s zero downtime and zero degradation in performance. We’ve helped organizations save millions of dollars across tens of thousands of AWS instances. Interested in seeing how much money you can save? Visit CloudFix…