Ask Us Anything: Episode 17

Stephen

All right, hello and welcome to the “AWS Made Easy Ask Us Anything” live stream, episode number 17, and this is our “Behind the Scenes” episode. We’re really excited to share this with you. How are you doing Rahul?

Stephen

Oh, really, really good. Just a really relaxing weekend. The kids were getting over a little bit of a cold, so they’re back in business now. How about yours?

Stephen

Well, that sounds great. Well, let’s get into it. And then let’s see. For some reason, my connection is a bit fuzzy, but is it coming through okay?

Stephen

Okay, great. Well, on that case, I’m gonna cue up the first segment, and then we’ll get right into it. So, let’s see. Here we go. And then what I’ll do after this, I’m gonna share my screen, and so this will be totally transparent process. Okay, here goes.

Okay, so you notice it said a “What’s New Review.” Now that’s intentional. This isn’t a “What’s New Review” segment, this is a “Behind the Scenes” segment. But I wanted to show you as part of how the segments get generated for that part. So, I wanted to make sure that we use that very process. Okay, let’s start with…

Stephen

Sure. Well, I guess I’ll go through the different ideas, and I’ll share my screen. Actually, let me do a real quick behind-the-scenes here. So, what we do in the live stream, I’ve got the streaming on StreamYard. And then off to the… So, I’ve got my main monitor here, which is you, and then off to the side, I’ve got some notes that I want to go through. And so I’ll just bring those notes onto the screen so we can kind of do this in real time.

But one of the streams…there’s different phases that a stream goes through. It’s basically… Here, let me share this window. Share, and here we go. Bring this onto the screen, move that StreamYard out of the way so we don’t get that recursive effect. So, stream goes from Planned, to Scheduled, to Ready, Recorded, Post-Processed, Highlighted, Finalized, and Completed. And it seems a bit excessive, but I got to the point where I was sort of tracking all these different pieces of the stream. And then I wrote down, okay, the full flowchart of how it works, and it ended up being quite involved. So, I thought, “Okay, I have to manage this.”

Let me see if I can share the flowchart of how the stream process ended up looking like because when we first started this, I was doing everything manually. We do the live stream, all the preparation manually. I thought, “I need to do a brain dump of exactly what’s going on.” And I ended up with this flowchart here. I don’t know how visible this is.

Stephen

Here we go. Here we go. So, is that okay?

Stephen

That’s exactly the point is I got to the point where, okay, we have this stream, and it’s going through certain stages, it starts with Planning. And then when we go from Planned to scheduled, it means, okay, we’ve got a date locked in, we’ve possibly got a guest locked in, we know what episode number is going to be. And then when we go from Scheduled to Ready, it means that, okay, we’re ready to shoot the live stream. We’ve got the event made. And so that’s what these little tick marks are saying is all the different bits of data that I need to gather up along the way. I’ll zoom in here.

And then when we do the stream and it’s recorded, that means we have a URL of our live stream. That’s our Episode 17.mp4, which we’ll have at the end of this session. And then we want to post-process it, which means we chop it up into different highlights and segments that we share out onto social media. And so I’ll show you what that looks like. And this is a shout-out to our own AWS Made Easy channel. But if we go down to AWS Made Easy. So, we see that throughout the week, we distribute these highlights. This is our…you and I were in Anaheim.

Stephen

Talking about the config. We went over this article that’s adding 20 new resources. This is the Rekognition article that we went through. And so we bring out this highlights throughout the week. And then finally, at the very end, everything is completed, which means we’re no longer tracking it. And so I kind of wrote all this stuff down, and I was managing this all in my head. And I thought, Okay, well, we need to automate it plus, you know, we’re programmers, right? So if it’s anything we have to do, it’s more than once, we have to automate it.

In general, I’m using this tool called ClickUp and we can get into that a little bit later. But you can see my ClickUp We can think of ClickUp as like a programmable Kanban board, and it moves a stream to these different phases. And then basically what the way I think about this stream is that for every phase that we talked about, we say, “Okay, do we have data to move into the next phase?” If not, then it’ll kind of kick me back.

Stephen

Exactly. It’s just a workflow tracker. But the cool thing is I can stick a webhook during any point in the process, and that webhook can then call out to an AWS function, a Lambda function, it can call out to a DevFlow, which we’ll talk about in a little bit. It’s really flexible. And so that way we can enforce doing processing certain data with at each segment, at each transition.

Stephen

Well, I agree, especially when it’s done occasionally…

Stephen

…then I’ll talk about it later but I don’t know how many times I’ve re-taught myself AWS Sam, because I don’t use it often enough to have it burned into my L2 cache of my brain or even the ram of my brain it’s in my, you know, the glacier part of my brain. And so you know that retaught, restarting that process every single time, it’s just often enough where I think I know what I’m doing. But it’s not often enough where I’m sure I know what I’m doing. And so I’ve got a little bit of automation around that. So that way, I don’t have to think about it every single time. And I could just know that it works.

Stephen

Oh, well. So, I think that kind of makes the case for the automation in terms of live streams. Do you have any other questions about what we’re doing? Or should we go on for the next one?

Stephen

All right, so then I’m going to trigger transition to. Okay. Oh, I see. One thing ClickUp has done is it’s reshuffled the order of my transitions. So, I have to deal with that in a moment. That’s really strange. Okay. So, in terms of tools, what we use, so I use ClickUp, as we said for…this is the Kanban Board, this is the issue tracker, this is what’s tracking my stream.

Stephen

Sure.

Stephen

I’ve got the… Okay, so we’ll go over it as we get to a new stream. But the idea is, I track… I’ve got a list called Live Streams. And in that list, I’ve got two, what we call sub-lists. One is called Streams, and one are called LS Tasks or Live Stream Tasks. And those are associated tasks with the stream. And so if I wanted to make a new live stream… Here, I’ll show you what it looks like for Episode 16 that we’ve done recently. Okay, Episode 16 is in the Post-Process state. It’s got a few segments that I can review.

Let’s look at Episode 15. Okay, so Episode 15, we’ve made a few segments, we’ve made a few highlights from those segments. We’ve made social media posts from those highlights, or there’s one here where I’m still fleshing out the design of the thumbnail. So, we’ll make sure that that gets posted, and then that will get promoted through the process. And they all tie back to this stream process. And so the stream process, it says, okay, this is a task of type Stream. It’s Episode 15, here’s the title. Here’s the LinkedIn event that it said, here’s our promotional image for that episode. Here’s the recording of it in an S3 bucket, here’s the show notes that came out of that. That popped up on my different window, so I’ll move that here. Here’s the show notes where we talked about each thing that we did. So, this is kind of a one-stop place to manage the stream. And all these fields get populated as the automation progresses through. And then…

Stephen

And so the part that I really like, and we’ll get through… So, if I go over to Streams, and I can go to Automations. And this is saying, okay, if I change the status of…well, let’s look at this automation. So, this automation is saying when the status goes from Planned, and then it advances out of the Planned state…Oh, this is saying, “If I don’t have this episode number, then kick it back.”

Stephen

But then I’ll have another automation that says…Here we go. This is where it gets interesting. When the status changes from Planned to Scheduled, then it’s going to call a DevFlow webhook. And we’ll look into this DevFlow webhook in the next one. And so it’s saying, “Well, now that I’ve actually nailed down a date, well, let’s automatically send out a Google Calendar invite.”

So, the tools we use to manage this, we use ClickUp for the management, we use DevSpaces as our IDE to code all these. And then DevFlows is our automation tool. And we can talk about that. That’s a tool for wiring up AWS services together. We’re gonna use AWS Rekognition. We use StreamYard for the streaming, and then ClickUp for task management.

Now, one second here, again, this is behind the scenes. In our StreamYard, I’ve got these transition tasks. And I noticed that somehow transitions three, four, and five vanished. So, I’m gonna these back in. So, we can actually queue them up. There we go. Transitions three, four, and five are processing. Okay. So, should we go on to the next bit of the conversation here?

Stephen

Let’s do it.

Okay, cool. So, we saw that transition three ran, and that’s what we’re saying. For segment number three, we wanna talk about scheduling the live stream in Google Calendar using DevFlows. So, I’ll open up this DevFlow here. Now, I really like this tool, DevFlows. I think what you said earlier, it’s about documenting the automation. And in this case, it’s saying when something happen… The best way for me to document things is a flowchart, right? If you have some really big, technically correct, philosophically complete document that painfully texturally enumerates every edge case, it’s painful. Whereas, this is a pretty simple flowchart of exactly what’s happening.

So, just to zoom in, it’s saying…so here’s the other end of this transaction. Here’s a webhook that comes in, and I’ll go to the Edit View here. So, this webhook is gonna come in, and then some of the data that’s going to be there… I’ve used this Reshape tool. Let’s look at the configuration here. I have an event coming in, and the event has a body. And in this body is going to be guest email, appointment start date, appointment end date, appointment summary. Those are what’s going to come out of that event.

Stephen

And then I’m sending this to Google Calendar. And so in this Google Calendar, what I’m saying is I want… I’m setting it to… Okay, I’m experimenting with the times here. I’m giving you the appointment start date and end date. And this is conference calls to plan the meeting for the live stream. And then we can populate a Zoom one. There’s some personal information in the production one, so I’m showing a demo on here. But yeah, the important parts here are the start and end dates, and that gets translated from the incoming webhook into something that Google Calendar can then use to put something on a schedule.

So, if we try that, let’s show how that works. So, let’s come up. Let’s go back to this view here. We’ll filter. And I’ve got a demo episode here in our live stream view, episode number 9995. Okay, so right now it’s in the Planned state. And if we locked down a scheduled date, let’s schedule this hypothetical episode for tomorrow at 7:45 in the morning. Okay. And then if we advance this from Planned to Scheduled, then this webhook is going to get called, and it’s going to say, “Okay, take this.” The webhook will get called, it will get a date, and then it will translate that date into one for the live stream and then the other one for the live stream minus 24 hours.

I’ve got a JSON audit in there. I just heard my Google Chat ping. And then so I should look on my Calendar now and I will get… This is a behind-the-scenes. This will pop up in a moment. I’ll get a… whoops, let me just turn off that Calendar. This will pop up in a second with the appointment. Let’s refresh that.

Stephen

Yeah, please.

Stephen

Yeah, I really enjoy. And just to show the kind of the power of it, you know, here’s an incoming webhook. You can say, “Okay, well, what do I want to do after that webhook?” You can do a bit of math, you can post it to Google Chat, you can manipulate that and have it feed S3. There’s lots of different options.

All right. Well, let’s go into the next one. I really like DevFlows. Like you said, we’ll do a deep dive on it. But this is just showing one little piece of that puzzle of, okay, we have a live stream set up, we wanna make a planning appointment, and we wanna make sure we pencil in the live stream appointment, and then we can inject the StreamYard URL, which you’re gonna need into that Google Calendar. All right. Well, let’s go on to the next one, which is then making the segments. All right. Okay.

Stephen

This one’s cool. I’m really, really happy with how this one works out. Okay, so let’s look at our test episode number 9995. And then again, since this is a behind-the-scenes, usually I have my notes off to the side, but I’m just going to do it right here. So, what is a segment? This is cool. Now a segment is a pre-planned thing that we’re going to talk about. So, as we’ve been talking throughout this live stream, a segment is, okay, here’s a discrete piece of the conversation, we know ahead in advance what that’s going to be.

Now we generate this little video transition. And what we’re generating that video transition for is for Rekognition to later chop up our video into different segments so that we don’t have to do that manually. So, that’s what I was doing for the first 12 or 13 episodes was watching the episode in advance, taking a note of the time in milliseconds that it started and ended, and then making the highlights from there. But that was a really painstaking process.

And then plus Rekognition has… Well, we’ll talk about what Rekognition is later, but the idea is we basically want to make our show so that it’s really, really easy for Rekognition to discuss when we’re talking about different segments. So, I think what I can do is just show you what it does, and then we’ll go through how it works.

Okay, so here’s our live stream view. We’ve got this demo live stream number 9995. And then, in that demo live stream, we want to…let’s say that we wanted to talk about some segments that we wanted to talk about. I think I wrote this down here. Let me do a different filter so I can show you this one. Greater than 99, 100. Okay. Here’s another test episode that we’ve made. And we have these type of segments. So, let’s make another segment that’s basically the same. I’ll show you what the test episode looks like.

So, I’ve got a test episode that starts out with our standard introduction. And then I pull up a segment and then we talk about something on the browser window, and then another transition plays. And so I want to generate these videos for these transitions. Okay, so now that I think I’ve set up the problem, actually let’s go ahead and do it. So, let’s make a new task, and let’s call this Segment, and we’ll call this the Intro Segment. And this, we’re gonna call it a Task of Type Segment. It’s gonna be episode number 9995. And this relates to our streaming segment. Sorry, stream, excuse me. This is our test stream up there. Okay.

Okay, so here’s our test stream with new transitions that we’re making. All right. Here we go, Demo Stream. Sorry. Let me filter this back to equals 9995. Okay. Let’s make our segment. So, we’re going to make a segment in LS Tasks. Segment: Introduction YouTube DL has a JavaScript Interpreter. Okay, so this is of type Segment. It’s episode number 9995. It is associated with Stream Task of Demo Stream. And then we’re gonna call this segment title, “YouTube DL has a JavaScript Interpreter.” Okay, let’s create that task. So, that’s gonna show up here.

Now if I make this view a little bit wider, right now, this is an open task. We don’t have anything. But then what I can do is if I advanced this to In Progress, it’s going to then call a webhook, which is going to give us one of these transition videos. So, if we refresh this. Refresh the ClickUp UI. And you should get it back fairly quickly. It might take another second or two.

Stephen

So, it’s usually about six. Okay. All right, so now it came back, and here’s what I’ve got.

Stephen

Yeah, and so what it did is, again, within the LS Tasks automation, I’ve got an automation which says, “When this happens, okay, generate transition video when segment goes from Open to In Progress.” And this is calling this…which I have to change now. It’s calling this Lambda function URL. And so what all that does…well, it’s actually an interesting pattern that evolves. Let me go back to this view. So, I found that when we have this automation that uses this webhooks, we actually have to do this two-step pattern.

Now, usually, automations, they want to return instantly. But it takes a few seconds to generate this video, and so what we end up doing is the first Lambda will push a task to SQS, and then the second Lambda function will… I’m allowing that to take the 10, 15, 20 seconds that it needs in order to generate that video. So, that’s what we’re doing here. We generate this segment, and later, we wanna say, “Okay, this is the segment order. This is going to be the first segment in our live stream.”

We can say, “Okay, is there an article that we’re discussing here?” Let’s see if there’s an article that we’re discussing. I had this in my notes here. Yeah. We’re gonna discuss this article here. Article URL there, and this is… And in the segments also, once we’ve discussed the segments, sometimes we rate them as either Simplicity or Complexity. And so this is where I’m tracking that. After the fact, I’ll say, “Okay, this one is Simplicity, and then we’ll give it our number of clouds that are five. And so I’ll put this here. Okay, so that’s our segment.

And so what we do is once this video gets generated, we then go into StreamYard and we have to populate. So, just sharing this here. These are the different segments that we populate so that while we’re talking, we can then queue those segments. So, showing you the source of that code again, and that’s a bit… Here we go. We open this up here. I have another window here that has Lambda function. Give me one second. Okay. My green window is the one that has… I thought I had refreshed this session. It had expired.

Stephen

Absolutely. I’m gonna need to use my phone because I’m going to need to use my MFA. So, give me one second.

Stephen

All right. And my phone is also my camera, so excuse me for one second. Authenticator 242609. Okay. There we go.

Stephen

There we go. So, this will be a separate behind-the-scenes, but what the camera does is it runs this thing called Camo because my iPhone camera is so much better than a webcam, and so I just use that as a camera. Okay, so here’s my Lambda function that’s really doing all this stuff. It’s calling this API called Shotstack. And so Shortstack basically takes a JSON payload that represents the video that we wanna make. So, what it starts with is it starts with this blank transition template. It starts with this blank transition template. And then there’s this big spot right here. And then what the Lambda function does is it gets the right data out of the task. It says, “Okay, here we go.” It gets the right data out of the task and then injects the title into that video.

And then this Lambda function has a longer timeout. And then here is where we update the ClickUp custom fields to have the URL of the video. And so then once that populates, I can then download them and populate StreamYard, I don’t forget to do that because I’ve done that several times when, oh, gosh, I forgot to generate the transition videos. It’s going live in five minutes. So, having the automation done is really, really helpful.

Stephen

Oh, I really underestimated it. But now that we’re in that streaming world, I’ve seen some ratio of minutes of live contents to minutes of planning and it’s, like, 1 to 20.

Stephen

Or 1 to 50. It’s a lot. Now this one, we’re showing all the live stream, all the automation that went into it. But anyway, I think what we end up with when we do all these transitions is we’ll have this live video that StreamYard will give us a recording at the very end of it. And because these transitions have these black frames here, this is what will then give us our highlight detection, which we’ll demo coming up. All right, well, anything, other questions for the segment generator?

Stephen

And just as a little shout-out, the API is called Shotstack. And they have the cleverest advertising that I’ve seen because what they actually do is they publish a ton of how-tos and how to use FFmpeg. And then what you realize when you’re reading this is I’m never ever, ever, ever gonna be able to get this right scripting this together. FFmpeg is so fragile. And at the very end, they’re just like, “Oh, you can just pay us a couple of bucks and we’ll do it all for you. Here’s a JSON payload that you can do.” And yeah, scripting FFmpeg is a… I’m glad that they’ve done that, and I’m happy to pay them. And it goes with our stitching things together, right? Don’t invent it yourself.

Stephen

Stitch together an existing service. This is theirs. I don’t wanna know how many special edge cases they’ve dealt with, but I’m happy to share that cost with all the other Shotstack clients.

Stephen

All right. Well, let’s do a quick break, and then when we come back, we’re gonna show DevSpaces to deploy Lambdas with function URLs because that’s been a really handy bit of this, especially with these automations when you want a quick webhook. Lambdas with function URLs are the way to go. So, let me trigger this break.

Stephen

Okay, cool. So, in this segment, I’m gonna be a little bit more generic. But when we’re talking about automation, a lot of times, like I said in the previous moment, you want a quick function to exist in a way that’s callable in public. You have a little bit of Python code, it’s really just applying a little bit of logic and updating something somewhere. And that’s a pretty common pattern, and so you want a function to exist somewhere that’s publicly callable.

Now, Lambda has been around for a long time, but then making it public involved API gateway and dealing with cores, and it was a bit more involved.

Stephen

[crosstalk 00:41:35.200] IAM. Luckily, there’s now a cool thing called Lambda function URLs. And the idea is that you can deploy a Lambda function and then right away you can… You deploy Lambda function and then it just has a URL and you can easily select your authentication mechanism. And I usually just have it open and the fact that the web… I don’t publish this URL as kind of the security I need for these webhooks. So, what I did, I made a template for how to very, very quickly deploy a function to Lambda. So, I wanna show how that works, and this is using DevSpaces.

So, the way that this works, again, as I mentioned earlier, I always find myself re-teaching myself SAM, re-teaching myself some of these things, you know, because I don’t do it all the time. So, what I wanted to do is kind of automate this process. So, what’s the easiest way to get some code public? So, we’ve got this repository, and I’ll put this in the chat, but it’s called SAM Lambda pi function. And the idea is I wanted to make it as easy as possible to have a function made public.

So, let’s start this with eyk.devspaces.com. Now what this is doing, it’s cloning that repository. And then I’ve got my DevSpaces set up. So, I have my AWS environment. My credentials that work with SAM are already built in.

Stephen

No.

Stephen

Yeah, yeah, that’s exactly right. Anyone can use this. They can just clone this and say, okay, you have to make sure… And it’s documented right here. You wanna make sure that you have your AWS access key ID, AWS secret access key, a default region set, and then you’re gonna wanna set this environment variable, which is the name of your stack. And I’ll show you how this works. It’s pretty cool.

So, if you look at the DevSpaces, and I’ll be very careful with what I show here, but within my variables, I’ve got access keys that are context-sensitive for different projects. And what I’ve set it up is that for all my projects that start with SAM, I’ve got a certain set of access keys that are pre-programmed in there. So, when I fork this repository, I can just call it SAM something else. And then I’ve already got the right access keys built in. Now I’ve got one setup step here that takes a little bit longer. Here we go.

I’m waiting for this one step to finish. I just have a… I should have moved that into the base image of that. But this has been a busy process. So, let me pull up CloudFormation while we’re waiting for this to finish.

Stephen

Yeah, it’s all set. So, say I wanted to deploy some function, I’ll make it really easy here. Instead of Hello World, I’ll say Hello DevSpaces and it’s automatically going to save. And then I’ll go back to the documentation because I always forget what I’m doing here, and then say, “Okay, those three variables get defined through my DevSpaces config.” So, all I really want to do is I can do here. I’ll call this the hellodevspacesstack./ build.sh. There we go. And now it’s kicked off a… what’s going on here? User. DevSpaces EYK.One second here.

Stephen

It should have been, let me check my environment really quick, though I have to hide the screen for a second.

Stephen

And we’re back. I guess I hadn’t rotated those keys through. But yeah, so there we go. Right away it’s doing this change set, and it’s creating this whole SAM template, and you can see it in CloudFormation. I’ll refresh over here. And there we go. This is creating in progress, the Hello DevSpaces stack. Again, all we did, the only change we made is we modified this app.pi, writes that little bit of business logic that you wanted to throw in there. There it is. And then as soon as this finishes creating, we’re gonna get an output and the output is gonna… There we go. I can refresh that.

Stephen

Let me check out the resources. This is the last thing, of course. Yeah, the last one.

Stephen

There it is. So, now I’ve got this endpoint. And not the most sophisticated bit of code, but look, this took just a few seconds and we’ve got a publicly running bit of Python code that’s right there on AWS and you’ve got your whole config saved. You can commit this back and it’s just done. So, really happy to be using DevSpaces for a lot of these bits of automation behind the scenes.

Stephen

And what I’ll do, I’ll modify this this template here to be a proper GitHub template, or you can just…what do you call it? Fork this and populate your own environment variables within DevSpaces. Off to update mine. Yeah, but all you have to do is fork this, set your AWS keys correctly, and there you go. You don’t have to go through that whole process. You don’t have to install SAM CLI locally. You don’t have to make sure that you’re running the right version of Python locally. No local setup at all. So, I really like that.

All right, well, let’s go to the next segment. And I can see already that we’re gonna have to do a Part 2 and a maybe Part 3 of this behind-the-scenes series, but let’s keep going here and see where we get. Okay, here it goes.

All right, now I’m going to turn off the camera on ourselves because we’re about to get this recursive effect. That’s not too bad. Okay, so here is StreamYard. Now, StreamYard is what we use to stream with. And we’d love to get someone from StreamYard on the show eventually, just because it’s such a really useful tool. So, you can see right now we’re streaming to LinkedIn, to YouTube, to Twitch, to Twitter, all simultaneously. All those transitions that we’ve made, we’ve got them up on the side. We can inject other things into the stream. So, when Rahul and I talk about an article, we can give it four and a half clouds.

Oh, and I forgot about my other banner that I made for the previous segment. So, you can really have a lot of fun in StreamYard. It’s made live streaming really easy. The first live stream I did that was for a different event, I tried to do all this custom work in OBS, and we came to the Q&A session and everything was in the wrong place. It was really hard to manage. And so, StreamYard is a lot less flexible than OBS, but it’s one of those…like, it just works. Once we started using it, we haven’t had to think about it. It’s been really nice. Yeah. It would be really nice that they had an API so I can stack this deck automatically and put it into my transitions, but they do not have an API at present. So…

Stephen

That will be cool.

Stephen

Yes, exactly.

Stephen

Yeah, so OBS, it works really well until we got the live Q&A at the very end. And what we were trying to do is we were trying to use Zoom and then stream capture parts of Zoom and push it into OBS. So, like with StreamYard…and again, I’ll show you. I’ve got my Stream Deck over here. And if you look over here, I’ve got buttons programmed in that represent different StreamYard views. And so if I want to switch… Let’s see if I can do this here. If I want to get rid of the split screen and just do group, I can hit this and we’re there.

And then I’ve got…so this is really this. There we go. Got a cinema mode, and we talk with the article. And so I’ve got this kind of by my right hand while we’re doing the live streams. So, that’s a Stream Deck, and it pairs with StreamYard. And I’ve got some macros programmed in that will switch the different views. So, I’ve got a little bit of control over. I always think back to the future when Emmett Brown is looking at the Sony handycams. This is a portable television studio. I think this is like, I don’t know, a TV station as a service. So, I’m very happy to be using this.

Stephen

Oh, gosh, and my brother was moderating the live stream. And yeah, StreamYard gives you everything all in one place. It’s really, really neat.

Stephen

It’s been really fun. And I’m excited to do a behind-the-scenes episode, you know, in six months from now and seeing where it’s going to get to. All right, well, let’s do… I think we have time for the using Rekognition. I wanna give the remaining 10 minutes for that. And I think we have to do the other things another day. But let me queue up the Rekognition transition. Here it goes.

Okay, so clearing up my screen here. So, let’s go back to our view here. I’ll make this a bit bigger. So, we’ve got a stream, and this stream is referencing this little mini-episode that I made where we’re going over, you know, just a news article on Hacker News. And we’ve got all those things there. So, let’s go over how we would kind of post-process this. So, we’ve got one segment there. We need to make another segment. And let’s call this segment Java Commentary.

And this is forcing me to make this of type Segment, 9995. Adding the Stream Task. This will populate well Demo Stream. And then we’ll call this Java Commentary. Okay. Save that. And I will make one more. Let me clean that up a bit. Okay. Segment. And now this will make sense in a second. Wrap up. Okay. We’re gonna make this a segment. So, the segments are things that we’ve planned in advance, and they have a title. We’ll call this Java Commentary. This one is going to be… We’ll save that. Link it to the parent. We need to call this a Segment. We need to link it to the parent task. Let’s see, Demo Stream, there we go. That should be what we need. Oh, I forgot the… Let’s see. Where is it?

One second. I must have forgot a field here because I have this view. I think what’s happened… Let me disable this filter quickly. I wanna make sure that this works exactly how I want it to. Okay. So, we’re in the segment view. Oh, yeah, that’s what it is. I forgot the… Now I can turn my filter back on. So, we’ll filter by the episode number as 9995. Okay, sorry for the little bit of setup.

Yeah. Now it’s here. We’ll call this Wrap-Up. These are all in progress. So, our previous automation should have… And so let’s say we’ve gone through the show, this one’s Complexity. We’re gonna give this a one. This one is Simplicity. We’re gonna give this a five. Let’s do a refresh. Our other automation will have run and we should have these transitions. Oh, and this is important, the order that we’re gonna go through them is two, and three. And let’s also give them URLs so I have something to reference. And let’s have that one be that URL.

Okay, that sounds good. Okay, so say we’ve done this live stream, and it’s all set. Then what I would do, and this part is still manual because StreamYard doesn’t have an API is that I would set this recorded S3 URI field. Now I’m gonna set this to this. Here we go. Set that here, code it S3 URI. Okay, that’s happy with that now. So, now when I move this from, so say it was in the recorded state because…actually, I think some of my automation, I wanna make sure, okay, it was ready and then we’ve recorded it.

And so now that it’s in the recorded state, the automation is gonna fire, which is going to launch a rekognition job. And what rekognition is going to do is it’s going to look for all the segments, and it’s going to figure out how many segments were in the show. And then if the amount of discrete clips it found matches the segments, so, in this case, it will find three like this. It’ll look for this pattern. It’ll look for a black frame, and then that transition video, and then some amount of content, a black frame, that transition video, and then some amount of content. If it finds the same amount that we’ve already pre-planned, it’s going to make a highlight that matches to that.

So, let’s refresh. And if we go to an overall view, we should see that start to work. There they are. So, cool. Okay, so what ended up happening…and it seems like we got it just on the cusp of an update there. Okay, so what it did, and let’s go to the highlight view, and this will make a lot more sense. So, this is 9995. Okay. So, what we’re seeing here is we’ve got… This is the cool thing. Rekognition populated these fields. I’ll zoom in a little bit more. Rekognition populated the fact that this happened here at 00:00:35:10 and it ended at 00:03:11:08. And so now if we go over to our video and we go to a certain timestamp, and let’s see, what was it? We’ll go to 00:35.

Stephen

Yeah, that’s it. And now if we go to the next one. So, the next one, so see our transition segment is about 10 seconds. So, this one ended at 11 and this next one starts at 28. So, the way that works in the background, and I’ll show you the script here. We’re getting close to our time. We have this huge Rekognition. Now Rekognition, it’s kind of fascinating. The key here is that we’re looking for this, what’s called a technical queue. And the technical queue that we’re looking for is that there’s a black frame here. And what this coverage percentage is saying is if this black frame is covering more than 60% of the screen.

Now if you dive into the documentation, one thing that’s interesting is a black frame technical queue only works if there’s also no audio. And StreamYard does a thing where when you trigger a transition, it tapers off the audio over about half a second. So, that’s why that black frame has to be a second long. If you had full control over the video, you could just do it for, I don’t know, a couple of frames, but it’s a bit extra long at the moment because I’m compensating for that audio taper off to make sure we get a little piece of it where there’s no audio.

Stephen

Oh, sure sorry. I got so excited I kind of jumped ahead, sorry.

Stephen

So, this one, in particular, is called the Rekognition Segment API. And what it’s doing is it’s detecting… Sorry, I got so excited and I was also watching the clock, and I got this and it worked.

Stephen

So, yeah, what you do is you give it a file in a bucket and you tell it what to look for. And in this case, we’re using a technical queue. I have typed “black frame,” and then once this job runs, it will broadcast out to an SNS topic when it’s finished. And then so I have… this is actually using that two-function paradigm where one of them starts it and then another one is waiting for the result to then update it.

Stephen

But the neat thing is…yeah. And that’s one thing I’ve seen through this whole automation process when you’re making these webhooks is you really want to have that pairing. You don’t want webhooks to be synchronous, and most of your webhook providers don’t want that either. And ClickUp says, you know, if you make a webhook that takes a minute, they’re gonna disable that automation.

Stephen

Yeah, so this is Rekognition. I’m working on a blog post that’ll kind of show all this stuff and have a repository that you can play with in DevSpaces and deploy the automation stack in SAM. You just have to fill out your AWS variables, and then you can basically have a very similar stack. The idea is…and we’ll have to go over this next time, but now that we’ve got these highlights, I’ll just give a quick preview.

So, when we have these highlights, this is saying here’s the full length of the segment. Now on LinkedIn, you can do a 10-minute video as a highlight, but on Twitter, you can only do 60 seconds. So, what we do is we have a preview highlight. So if we did say…and again I’m doing this blind, but say if we start at 35, but then we wanna end it at 1 minute and 15, that’ll be okay. So, we’ve got that. And then let’s jump into this one. And we’ll add a thumbnail because we need a thumbnail.

Let’s see. Thumbnail image, upload the file. This will be Episode 17. There we go. This will be the thumbnail for our video is that. And then if I move this over to In Progress, then this is going to then take these parameters and it’s going to render them in Shortstack into these videos that we can then share out on YouTube, on Twitter, or LinkedIn. And it’ll make the full version for LinkedIn and it’ll make the shortened version for Twitter. Let’s see how that’s going. That will run. We’ll have to do that next time because now it’s 10:00. We’re already two minutes past.

But yeah, that’s how we use Rekognition to do this segment analysis. And this has been a real time saver because the first 13, 14 episodes, it was really just moving…it was in VLC, with my notepad writing down these timestamps and getting that to work. It was time-consuming. But if you can control your transition videos and have that black frame, then you can basically make Rekognition do all that work for you. The Rekognition Segment Analysis API. All right. Well, I think [crosstalk 01:08:33.459]

Stephen

Yeah, I’ll get that finished up and post it along with some code. And we have to do a Part 2 and maybe a Part 3 of this. And I guess that’s where… I realized there’s so much there. And there’s a lot of little tasks, like getting the thumbnail lined up, making sure the thumbnail is ready when I wanna make the highlights. And that was a problem I ran into earlier is I’d be all ready to go and I say, “Oh, I forgot to ask the designer to make the thumbnails,” and then so that would be a 24-hour turnaround on there.

Stephen

And so being able to have all this flow controlled and all these reminders in place to add this structure, it’s been really fun, and using DevSpaces and DevFlows because it makes it really easy to just have these quick functions that…yeah. I couldn’t think of a faster way to get a Lambda function just sitting out there on the public internet than using a DevSpaces template.

Stephen

Yeah, let’s see you next week. Maybe we’ll just do the… There’s Part 2 next week.

Stephen

All right. All right. Well, thanks again. And I’ll cue the intro.

Stephen

Bye, bye. Here we go. I gotta cue the credits, and there we go.