Monday 1/17/22 Cloud Studies Update: AWS Lambda parts 1 & 2

Adrian Cantrill’s SAA-C02 study course, 60 minutes: Serverless and Application services section: AWS Lambda Parts 1 & 2

[ASSOCIATESHARED] AWS Lambda – Part 1

This three-part lesson focused on Lambda. The first thing we learned was that Lambda is a function as a service (FAAS) product. You provide specialized short running and focused code to Lamda, and it takes care of running it and only billing you for what you consume. A Lambda function is a piece of code, which Lambda runs and every Lambda function is using a supported runtime. According to the lesson, Python 3.8 is an example of a supported runtime.

Moving on, it was further stated that when creating a Lambda function, the end user defines which runtime that piece of code uses. When the code is provided to Lambda, it is loaded into and exectued within a runtime environment, and this runtime environment is specifically created to run code using a certain runtime, a certain language. So, when a Lambda function is created that uses the Python 3.8 runtime, then the runtime environment that’s created is itself specifically designed to run Python 3.8 code.

When a Lambda function is created the amount of resource the runtime environment needs is also defined, by allocating a certain amount of memory, which indirectly allocates a certain amount of virtual CPU; you don’t choose the amount of CPU.

The key takeaway from all of this is that Lambda as a service is designed for short running and focused functions, is that you’re only billed for the duration that a function runs. Based on the amount of resource allocated to an environment and based on the duration that functions run for per invocation, all this determines how much you’re billed for the Lambda product; you’re billed for the duration of function executions

Lambda is a key part of serverless architectures running within AWS. To use Lambda, you define a Lambda function, which is akin to a unit of configuration. A Lambda function can be thought of as the code itself plus all the associated wrappings and configuration. At its most basic, the Lambda function is a deployment package, which Lambda executes. This requires defining the language which the function is written in, provide a deployment package and set some resources. At the time of invocation, the deployment package is downloaded and executed within the runtime environment.

Lambda supports many different runtimes, including common version of Pythong, Ruby, Java, Go, C# and various versions of Node JS. Lambda layers allow for the creation of custom runtimes, and many Lambda layers are created by the community using the Lambda Layers functionality, including Rust.

It was emphasized at this point if the Docker is involved, to consider this as not Lambda; Docker is anti-pattern for Lambda. Generally, Docker is not used with Lambda.

Selecting the runtime to use when creating the function determines the components which are available inside the runtime environment. Every time a Lambda function is invoked, a new runtime environment is created with all needed components. Lambda functions are stateless which means that no data is left over from a previous invocation. This isn’t always the case, but from an architectural standpoint it is best to assume this to be the case. Basically, the code running within Lambda needs to be able to work 100% of the time if the environment is new. Lambda runtime environments have no state.

Along with this, the resources needed for the Lambda functions are also defined, which determines resource allocation for the runtime environment. Memory is directly defined, between 128 mb and 3 GB. The amount of virtual CPU is not directly controlled; this scales linearly with memory. There is also disk space allocation; 512 mb is mounted as /tmp within the runtime environment. This should only be viewed as a temporary space, as it is blank for every separate invocation.

Lambda functions can run for up to 900 seconds (15 minutes), and this is known as the function timeout. For anything beyond that duration Lambda cannot be used directly. Other things like step functions can be used to create longer running workflows.

The security for a Lambda function is controlled using execution roles, IAM roles assumed by the Lambda function with permissions to interact with all of the AWS products and services. Any permission are delivered by creating an execution role and attaching that to a specific Lambda function.

Lambda forms a core part of the delivery of serverless applications within AWS and uses products such as S3, API gateway, and Lambda, and these threse products are often used to deliver serverless applications.

Lambda can be used for file processing using S3, S3 events and Lambda, for things like watermarking images, database triggers using DynamoDB, DynamoDB streams and Lambda. Lambda can also be used to implement a form of serverless Cron, using Eventbridge or CloudWatchEvents to invoke Lambda functions at specific times.

Lambda can also be used to perform real time stream data processing, which involves configuring Lambda to invoke whenever data is added to a Kinesis stream, which is important because Lambda is scalable and so it can scale with the amount of data being streamed into a Kinesis stream.

[ASSOCIATESHARED] AWS Lambda Part 2

Picking up from lesson one, this second part of Lambda focused on Lambda networking, Lambda permissions, and Lambda monitoring. It was explained that Lambda has two networking modes that one needs to be aware of. First is public, which is the default, and second is VPC networking, and the architecture of both needs to be understood.

Public networking starts with an AWS environment and inside it is a single Lambda function, which is part of a wider application. Lambda running by default using public networking means that it has network connectivity to public space AWS services on the public internet. It can connect to both of these from a networking perspective and as long as it has the required methods of authentication and authorization, then it can access all of those services.

Public networking offers the best performance for Lambda because no customer specific networking is required. Lambda functions can run on shared hardware and networking with nothing specific to one particular customer. However, any Lambda functions running with this default have no access to services running within a VPC unless those services are configured with public addressing as well as security rules to allow external access.

It was explained that in most cases, Lambda is used with its public networking model, but there are situations where this isn’t enough. For these situations, Lambda can be configured to run inside a VPC.

For this situation, the important thing about Lambdas running inside a VPC is that they obey all of the same rules as anything else running in a VPC because they’re running within that VPC.

Lambda functions running inside a VPC can freely access other VPC based resources, assuming any network ACL’s and security groups allow that access. The limitation is that they can’t access anything outside of the VPC unless networking configuration exists within the VPC to allow this external access.

The upshot is that if you need to design any solutions which involve Lambda functions running within a VPC, treat them like anything else running in the VPC. You could use a VPC endpoint, a gateway endpoint, or if the Lambda function needed access to AWS Public Services or the internet, you could deploy another gateway and a public subnet and then attach an internet gateway to the VPC.

After this we discussed that because Lambda may need to create network interfaces within the VPC, it requires those permissions. There used to be disadvantages to running Lambda in a VPC. The reason was that the networking architecture Lambda used. VPC-based Lambda functions don’t actually run within the VPC; the way they work is similar to Fargate.

Historically, when Lambda functions were invoked they would create an elastic network interface within the customer VPC and traffic would flow between this service VPC and the customer VPC. Configuring these elastic interfaces on a per function, per invocation basis would take time and add delay to the execution of the Lambda function code. Also, this architecture doesn’t scale well, because parallel function executions or concurrency required additional elastic network interfaces, and the more popular a system, the worse the problems. With larger systems there were more performance issues and more and more issues with keeping VPC capacity available for larger and larger numbers of ENIs.

The new way is that instead of requiring an elastic network interface per function execution, AWS analyzes all of the functions running in a region and an account and builds a unique combination of security groups and SOAP NAT’s.

If all your functions used a collection of subnets but the same security groups, then one network interface will be required per subnet. If they used the same subnet and all used the same security group, then all the Lambda functions could use the single elastic network interface, so a single connection between the Lambda service VPC and your VPC would be required for every unique combination of security groups and subnets used by your Lambda functions. The network interface is created when you’re using this architecture when you configure the Lambda function, which takes about 90 seconds, but this is done once. This means that now you can use private networking at scale without increasing the number of elastic network interfaces required. It was reiterated that we need to treat Lambda functions running within a VPC like any other VPC based resource.

Moving on, we looked at security of Lambda functions. With Lambda permissions there are actually two key parts of the permissions model that need to be understood. For Lambda functions to work, they need to be granted an execution role, which is a role assumed by Lambda, and the role grants permissions based on the roles permission policy. A role is created which has a trust policy which trusts Lambda, and the permissions policy that role has is used to generate the temporary credentials that the Lambda function uses to interact with other resources. In many ways this is the same as an EC2 instance role.

The other part is Lambda resource policies, which in many ways is like a bucket policy in S3, it controls who can interact with a specific Lambda function. It’s the resource policy which can be used to allow external accounts to invoke a Lambda function or certain services to use a Lambda function such as SNS or S3. The resources policy is something changed when you integrate all the services with Lambda and you can manually change it via the CLI or the API, but not the UI.

Before ending part two, we looked at logging. It was explained that Lambda uses CloudWatch, CloudWatch logs, and X-Ray for various aspects of it’s logging and monitoring. Any logging information generated from Lambda executions goes into CloudWatch logs. Any outputs from Lambda, messages output to the log, any errors, details on the duration of the execution is all stored into CloudWatch logs. Any metrics, are stored in CloudWatch. CloudWatch is the thing that stores metrics. Logging is stored in CloudWatch logs. Any details on the number of invocations, successes or failures go into CloudWatch.

This can be integrated into X-Ray in order to add distribured tracing capability, to enable something like tracing the path of a user or the path of a session through a serverless application which uses Lambda. X-Ray is associated with distributed tracing.

It was emphasized that for the exam for Lambda to be be able to log into CloudWatch logs to generate the output of any of the executions, Lambda needs permissions granted via the execution role, and there’s actually a prebuilt policy and role within AWS sepcifically designed to give Lambda functions the basic permissions that they require to log information into CloudWatch logs.  

Published by pauldparadis

Working towards cloud networking security as a profession.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: