davethemaker

Tag: AWS

  • Amazon Data Firehose

    This is a service that moves data from a source location to a target location. Producers can be apps, clients, SDK, Kinesis Agent, or even Kinesis Data Streams (we’ll be exploring this in a minute), AWS IoT, CloudWatch. Records can be up to 1MB, and can be transformed potentially by a lambda function. Potential destinations…

  • Kinesis Data Stream

    Kinesis is a service offered by AWS for streaming data. This could come from IoT devices or applications, but essentially we are looking at a large number of producers and with a use case of using the data in real time. For example, this might be useful for r/t analytics, such as with a stock…

  • SNS

    Simple Notification Service (SNS) is helpful, like SQS, to decouple events with messages. Let’s walk through an example. Amazon SNS > Topics > Create Topic. Type is Standard (not FIFO), name is FirstTopic. Next, we create a subscription: I select Email and enter my email address. So the subscription is created. Right now we have…

  • SQS: Dead Letter Queue

    If a message is repeatedly sent back from the consumer back to the queue for some reason – perhaps it’s a malformed message – then we need a mechanism for flagging that message, perhaps for analysis and debugging. That’s where the Dead Letter Queue comes into play. We can configure a message being sent to…

  • Publishing S3 Event Notifications to SQS Queue

    In this exercise, we’re going to practice setting permissions and configuration so once an event occurs within an s3 bucket, that event notification will be sent to the SQS Queue. This is handy because with this framework, we could take that message held in the queue, save it to a database, or send a SNS…

  • SQS: Producing,Receiving

    SQS is a queing service that decouples applications. This is really interesting as it points to creative building of data pipelines, ingesting data from events and then doing all sorts of cool stuff with it! Let’s get some hands on: Amazon SQS > Queue > Create queue. I’m going for tthe Standard There’s the configuration.…

  • Instance Meta-data

    We can access the meta data of an instance using IMDS v1, which doesn’t require a token, or IMDS v2, which is much more secure, and which also requires more work including generating a token. I created a Linux EC2 instance and connected via shell. Here’s all I had to run to get the instance…

  • Mapping DNS Alias to EC2 via ALB

    With Route 53, you have different records: A, AAAA, CNAME, and more. These can point to AWS resources- for example, CNAME and Alias can point to resources in different ways. I want to explore creating several EC2 instances, and Application Load Balancer (ALB), and then pointing that ALB to a domain name. First I create…

  • Network Hardening Using Amazon Inspector and AWS Systems Manager

    During my graduate studies in Information Systems, I was introduced to cybersecurity and became really interested in it. Since then, I’ve been focusing on learning cloud services and IaC, but I’m interested in pulling the security aspect in as well. To that effect, I’m going to be using Amazon Inspector and AWS Systems Manager to…

  • From local to EC2: Docker Apache Web App

    I’ve been following a tutorial which describes how to use containers within the AWS environment- it’s well done (Cantrill) but it does require the user (me) to simply download a pre-packaged container-ready file as part of a Cloud Formation deployment. That works for the intention of the author – provide a good, birds-eye view of…