davethemaker

Category: Uncategorized

  • Amazon Data Firehose

    This is a service that moves data from a source location to a target location. Producers can be apps, clients, SDK, Kinesis Agent, or even Kinesis Data Streams (we’ll be exploring this in a minute), AWS IoT, CloudWatch. Records can be up to 1MB, and can be transformed potentially by a lambda function. Potential destinations…

  • Kinesis Data Stream

    Kinesis is a service offered by AWS for streaming data. This could come from IoT devices or applications, but essentially we are looking at a large number of producers and with a use case of using the data in real time. For example, this might be useful for r/t analytics, such as with a stock…

  • SNS

    Simple Notification Service (SNS) is helpful, like SQS, to decouple events with messages. Let’s walk through an example. Amazon SNS > Topics > Create Topic. Type is Standard (not FIFO), name is FirstTopic. Next, we create a subscription: I select Email and enter my email address. So the subscription is created. Right now we have…

  • SQS: Dead Letter Queue

    If a message is repeatedly sent back from the consumer back to the queue for some reason – perhaps it’s a malformed message – then we need a mechanism for flagging that message, perhaps for analysis and debugging. That’s where the Dead Letter Queue comes into play. We can configure a message being sent to…

  • Publishing S3 Event Notifications to SQS Queue

    In this exercise, we’re going to practice setting permissions and configuration so once an event occurs within an s3 bucket, that event notification will be sent to the SQS Queue. This is handy because with this framework, we could take that message held in the queue, save it to a database, or send a SNS…

  • SQS: Producing,Receiving

    SQS is a queing service that decouples applications. This is really interesting as it points to creative building of data pipelines, ingesting data from events and then doing all sorts of cool stuff with it! Let’s get some hands on: Amazon SQS > Queue > Create queue. I’m going for tthe Standard There’s the configuration.…

  • Docker and Containers

    Containers are lightweight computer services and I’m going to work on exploring this technology. First, to create a cluster, which groups tasks, services together. First, go to ECS and create a cluster. For this example, I’m going to select both Fargate, which is serverless provisioning, and EC2 Instances, which requires manual configuration. Since I selected…

  • Instance Meta-data

    We can access the meta data of an instance using IMDS v1, which doesn’t require a token, or IMDS v2, which is much more secure, and which also requires more work including generating a token. I created a Linux EC2 instance and connected via shell. Here’s all I had to run to get the instance…

  • Mapping DNS Alias to EC2 via ALB

    With Route 53, you have different records: A, AAAA, CNAME, and more. These can point to AWS resources- for example, CNAME and Alias can point to resources in different ways. I want to explore creating several EC2 instances, and Application Load Balancer (ALB), and then pointing that ALB to a domain name. First I create…

  • Backend for AK – explore RDS

    Previously, I uploaded an html file to the relevant s3 bucket, setting the permissions for the public to access the contents, setting the CORS policy, and then deleting the cloudfront setup which I had created previously but wasn’t working correctly. I’ll add that in later but the focus is on building the functionality so that’s…