Implementing Magic Links with Amazon Cognito: A Step-by-Step Guide

The user initiates the sign-in process by entering their email in your application.
They receive an email with a time-limited URL.
The user clicks on the URL and is authenticated into the application.

Again, this is not something that Cognito supports out of the box. But we can implement this custom authentication flow using its Lambda hooks [2].

How it works

Cognito has three Lambda hooks for implementing custom authentication flows:

However, we can’t rely on these alone to implement magic links.

Let me explain.

The problems with Session

The official documentation page [3] explains the roles of these hooks and how they fit together. However, it doesn’t tell you that a Session string issued by the user pool needs to be passed back and forth between the frontend client and the user pool.

You can see this in the response of the InitiateAuth [4] and the request of the RespondToAuthChallenge [5] APIs. This Session value is how we are able to pass data from the CreateAuthChallenge function to the VerifyAuthChallengeResponse function using privateChallengeParameters.

This is a problem when implementing magic links.

When the user clicks on the magic link, it would most likely open a new browser window and the previous session data is lost on the client. Of course, there are ways to work around this using various forms of local storage. But what if the magic link is opened on a different browser? The user could have also closed the previous browser tab as well.

What if we include the Session value in the magic link? Wouldn’t that allow the client to continue the authentication flow in the new browser window?

Sadly, that won’t work either.

The initial Session is generated AFTER the CreateAuthChallenge function was called. This is because the session contains information that was returned by the CreateAuthChallenge function, such as the privateChallengeParameters and challengeMetadata values above.

Instead, what we can do is introduce a custom API endpoint to kick off the authentication process and send the magic link to the user. Only when the user opens the application via the magic link do we use Cognito’s InitiateAuth API to kick off a custom Cognito authentication flow.

The Solution

Here’s how the solution works:

1. The user enters their email to start the authentication flow. The front end calls a POST /login endpoint on our API Gateway REST API. This triggers a Lambda function to:

Generate a secret token. The front end would later need to present this token when it responds to the custom auth challenge.
Generate the magic URL including the secret token as a query string parameter.
Send an email to the user including the magic URL. Here, I will use the Amazon Simple Email Service (SES) to send emails. If you wish to try it out yourself, you would need to create and verify a domain identity in SES. Please refer to the official documentation page [6] for more details on how to do that.
To enforce a time limit, we can use a JSON string that contains an expiration datetime and encrypt it using the Key Management Service (KMS). This encrypted string would be our secret token.
To make sure the magic links are invalidated when the user generates a new one, we need a way to track the CURRENT token for a user. We can do this by setting it as a custom attribute on the Cognito user. This is preferred to using a DynamoDB table because the Cognito user pool would pass it along to the CreateAuthChallenge function. Which saves us from having to read it out of the database ourselves.

Sidenote: as Otto Kruse pointed out on Twitter, the decision to use Cognito’s user attributes has an impact on the number of users that can sign in at the same time. Because Cognito has a hard limit of 25 reqs/sec on AdminSetUserAttribute. If you’re likely to experience thundering herd problems then you should consider using DynamoDB to record the secret token instead.

2. The user clicks on the magic link in the email and is directed back to the application. The link contains an email and a token query string parameter. The frontend client uses the email to initiate the Cognito authentication flow.

3. The user pool calls the DefineAuthChallenge Lambda function to decide what it should do. The function indicates that it should present a custom auth challenge to the user.

4. To create the custom challenge, the user pool calls the CreateAuthChallenge Lambda function. Because we had saved the secret token as a custom attribute on the Cognito user (step 1.), the function can find the token in its invocation event.

At this point, we can “lock in” the secret token we should use for this authentication flow. So even if there’s a delay in the front end responding to this challenge and the user starts another authentication flow elsewhere, it wouldn’t affect the current flow.

We can do this by saving the secret token in the response.privateChallengeParameters and make sure the VerifyAuthChallengeResponse function uses it.

5. Immediately after receiving the custom auth challenge, the front end responds with the token it received through the query string parameters. There’s no need for user input.

6. The user pool calls the VerifyAuthChallengeResponse function to validate the token:

Make sure it matches what the CreateAuthChallenge function had saved in the privateChallengeParameters object.
Make sure the token hasn’t expired.
Make sure the user the token was issued for matches the user who’s trying to sign in.

If all the checks pass then the challenge is answered correctly.

7. The user pool calls the DefineAuthChallenge function again to decide what happens next. If the challenge was answered correctly then JWT tokens are issued. Otherwise, we fail the authentication attempt.

There’s no need for retries. If the token is invalid for any reason, no amount of retries can change that.

Okay, so that’s how this solution works at a conceptual level.

Let’s see how we can implement it.

How to implement it

1. Set up a Cognito User Pool

First, we need to set up a Cognito User Pool.

PasswordlessMagicLinksUserPool:
  Type: AWS::Cognito::UserPool
  Properties:
    UsernameConfiguration:
      CaseSensitive: false
    UsernameAttributes:
      - email
    Policies:
      # this is only to satisfy Cognito requirements
      # we won't be using passwords, but we also don't
      # want weak passwords in the system ;-)
      PasswordPolicy:
        MinimumLength: 16
        RequireLowercase: true
        RequireNumbers: true
        RequireUppercase: true
        RequireSymbols: true
    Schema:
      - AttributeDataType: String
        Mutable: false
        Required: true
        Name: email
        StringAttributeConstraints: 
          MinLength: '8'
      - AttributeDataType: String
        Mutable: true
        Required: false
        Name: authChallenge
        StringAttributeConstraints: 
          MinLength: '8'
    LambdaConfig:
      PreSignUp: !GetAtt PreSignUpLambdaFunction.Arn
      DefineAuthChallenge: !GetAtt DefineAuthChallengeLambdaFunction.Arn
      CreateAuthChallenge: !GetAtt CreateAuthChallengeLambdaFunction.Arn
      VerifyAuthChallengeResponse: !GetAtt VerifyAuthChallengeResponseLambdaFunction.Arn

Notice that we configured a custom attribute called authChallenge. When the user initiates an authentication flow, our API function would save the secret token in this attribute. The value of this attribute would be passed along to the DefineAuthChallenge, CreateAuthChallenge and VerifyAuthChallengeResponse functions.

- AttributeDataType: String
  Mutable: true
  Required: false
  Name: authChallenge
  StringAttributeConstraints: 
    MinLength: '8'

Also, it’s important to note that passwords are still required even if you don’t intend to use them. I have set a fair strong password requirement here, but the passwords would be generated by the front end and they are never exposed to the user.

Our user pool is not going to verify the user’s email when they sign up. Because every time the user tries to sign in, we would send them an email with a magic link. Which would verify their ownership of the email address at that point.

Sidenote: I made this decision for the demo to make the demo really easy to use. In practice, you should still enable auto-verification for your application. It adds minor friction to the sign-up process but ensures the user accounts in your production system are all valid emails. Additionally, you should clean up unconfirmed Cognito users after X number of days.

2. Set up the User Pool Client for the frontend

The frontend application needs a client ID to talk to the user pool. Because we don’t want the users to log in with passwords, we will only support the custom authentication flow with ALLOW_CUSTOM_AUTH.

WebUserPoolClient:
  Type: AWS::Cognito::UserPoolClient
  Properties:
    ClientName: web
    UserPoolId: !Ref PasswordlessMagicLinksUserPool
    ExplicitAuthFlows:
      - ALLOW_CUSTOM_AUTH
      - ALLOW_REFRESH_TOKEN_AUTH
    PreventUserExistenceErrors: ENABLED

3. A KMS customer-managed key (CMK)

We will generate the secret token by encrypting a JSON payload like this:

{
  "email": "me@example.com",
  "expiration": "2023-03-19T02:27:45.768Z"
}

To do that, we will need a KMS key.

EncryptionKey:
  Type: AWS::KMS::Key
  Properties: 
    Enabled: true
    EnableKeyRotation: true
    KeyPolicy:
      Version: '2012-10-17'
      Statement:
        - Sid: Enable IAM User Permissions
          Effect: Allow
          Principal: 
            AWS: !Sub arn:aws:iam::${AWS::AccountId}:root
          Action: kms:*
          Resource: '*'
        - Sid: Allow access for Key Administrators
          Effect: Allow
          Principal: 
            AWS: !Sub arn:aws:iam::${AWS::AccountId}:role/Administrator
          Action:
            - kms:Create*
            - kms:Describe*
            - kms:Enable*
            - kms:List*
            - kms:Put*
            - kms:Update*
            - kms:Revoke*
            - kms:Disable*
            - kms:Get*
            - kms:Delete*
            - kms:TagResource
            - kms:UntagResource
            - kms:ScheduleKeyDeletion
            - kms:CancelKeyDeletion
          Resource: '*'
    MultiRegion: false
    PendingWindowInDays: 7

4. The Lambda function behind POST /login

As mentioned above, we will add a custom API endpoint to initiate the authenticate flow. In the Serverless framework, I can do this by declaring a logIn function like this:

logIn:
  handler: functions/log-in.handler
  events:
    - http:
        path: login
        method: post
        cors: true
  environment:
    SES_FROM_ADDRESS: noreply@${self:custom.domain}
    KMS_KEY_ID: !Ref EncryptionKey
    BASE_URL: passwordless-cognito.theburningmonk.com
    USER_POOL_ID: !Ref PasswordlessMagicLinksUserPool
  iamRoleStatements:
    - Effect: Allow
      Action: ses:SendEmail
      Resource: 
        - !Sub arn:aws:ses:${AWS::Region}:${AWS::AccountId}:identity/${self:custom.domain}
        - !Sub arn:aws:ses:${AWS::Region}:${AWS::AccountId}:configuration-set/*
    - Effect: Allow
      Action: kms:Encrypt
      Resource: !GetAtt EncryptionKey.Arn
    - Effect: Allow
      Action: cognito-idp:AdminUpdateUserAttributes
      Resource: !GetAtt PasswordlessMagicLinksUserPool.Arn

The function needs a number of environment variables, including:

The ID of the KMS key.
The base URL of the magic link.
The ID of the Cognito User Pool. Because we will add the secret token as a custom attribute on the Cognito user.
The email sender’s address. This sender must be from a verified domain in SES or it must be a verified email address in SES.

So let’s have a look at the code for this function.

const Cognito = require('aws-sdk/clients/cognitoidentityserviceprovider')
const cognito = new Cognito()
const SES = require('aws-sdk/clients/sesv2')
const ses = new SES()
const { TIMEOUT_MINS } = require('../lib/constants')
const { encrypt } = require('../lib/encryption')
const qs = require('querystring')
const middy = require('@middy/core')
const httpErrorHandler = require('@middy/http-error-handler')
const cors = require('@middy/http-cors')

const { SES_FROM_ADDRESS, USER_POOL_ID, BASE_URL } = process.env
const ONE_MIN = 60 * 1000

module.exports.handler = middy(async (event) => {
  const { email } = JSON.parse(event.body)
  if (!email) {
    return {
      statusCode: 400,
      body: JSON.stringify({
        message: 'You must provide a valid email.'
      })
    }
  }

  // only send the magic link on the first attempt
  const now = new Date()
  const expiration = new Date(now.getTime() + ONE_MIN * TIMEOUT_MINS)
  const payload = {
    email,
    expiration: expiration.toJSON()
  }
  const tokenRaw = await encrypt(JSON.stringify(payload))
  const tokenB64 = Buffer.from(tokenRaw).toString('base64')
  const token = qs.escape(tokenB64)
  const magicLink = `https://${BASE_URL}/magic-link?email=${email}&token=${token}`  

  try {
    await cognito.adminUpdateUserAttributes({
      UserPoolId: USER_POOL_ID,
      Username: email,
      UserAttributes: [{
        Name: 'custom:authChallenge',
        Value: tokenB64
      }]
    }).promise()
  } catch (error) {
    return {
      statusCode: 404,
      body: JSON.stringify({
        message: 'User not found'
      })
    }
  }
  
  await sendEmail(email, magicLink)
  return {
    statusCode: 202
  }
})
.use(httpErrorHandler())
.use(cors())

I want to point out a few things in this function:

The sendEmail function has been omitted here for brevity’s sake. It does what you’d expect and sends the magic link to the user by email.
The request validation can be delegated to API Gateway instead. API Gateway supports request validation. In fact, it would be my preferred way to validate POST bodies because invalid requests would be rejected by API Gateway without hitting my function. So I don’t have to write custom code to validate them, and importantly, API Gateway does not charge for these invalid requests.
If the user’s email is not found in Cognito, it will return an HTTP 404. This is fine in most cases, but it allows malicious actors to find out if an account exists in your application. If you had enabled the PreventUserExistenceErrors setting on the user pool client, then you should ensure this function follows the same behaviour and to not return a 404.
This function uses the Middy [7] middleware engine to handle unhandled errors and add CORS headers in the response.
The encrypted token is saved in the user’s authChallenge attribute described earlier. However, when using custom attributes, the attribute names have to be prefixed with custom:. Hence it appears as custom:authChallenge here.
The encrypted token (which results in a Buffer object) is converted to a base64 string and then URI encoded. But it’s the base64 version that is saved in the Cognito user’s attributes. When we send this token back during sign-in, we have to first URI decode it. This happens in the front end.
The encrypt function exists in another module (see below), which uses the aforementioned KMS key.

const KMS = require('aws-sdk/clients/kms')
const KmsClient = new KMS()

const { KMS_KEY_ID } = process.env

const encrypt = async (input) => {
  const resp = await KmsClient.encrypt({
    KeyId: KMS_KEY_ID,
    Plaintext: input
  }).promise()

  return resp.CiphertextBlob
}

const decrypt = async (ciphertext) => {
  const resp = await KmsClient.decrypt({
    CiphertextBlob: Buffer.from(ciphertext, 'base64')
  }).promise()

  return resp.Plaintext
}

module.exports = {
  encrypt,
  decrypt
}

5. (Front end) Initiate the authentication flow

Once registered, a user can initiate the authentication flow by just entering their email. In the front end, we will make an HTTP request to the POST /login endpoint above.

async function sendMagicLink() {
  const response = await fetch('https://xxx.execute-api.eu-west-1.amazonaws.com/dev/login', {
    method: 'POST',
    headers: {
      'Content-Type': 'application/json'
    },
    body: JSON.stringify({
      email: email.value
    })
  }).catch(err => {
    alert(`Failed to send magic link: ${err.message}`)
  })

  if (response.status !== 202) {
    const responseBody = await response.json()
    alert(`Failed to send magic link: ${responseBody.message}`)
  } else {
    signInStep.value = 'SENT_MAGIC_LINK'
  }
}

The user would then receive an email like this:

6. (Front end) Sign in with Cognito

Upon clicking the magic link, the user would be taken back to the website. Instead of showing the user the original UI, we need to extract the email and token values from the query string and sign the user in (assuming the token is still valid).

In Vue, we can use the onMounted lifecycle hook to execute code when the component has been mounted to the DOM. At this point, we can check if the email and token query string parameters are present. If so, initiate Cognito’s sign-in flow and respond to the custom challenge.

import { onMounted } from 'vue'
import { Amplify, Auth } from 'aws-amplify'

onMounted(async () => {
  // the search string looks like "?email=xxx&token=yyy"
  if (window.location.search) {
    const qs = window.location.search.substring(1)
    const qsParams = qs.split(['&'])
    const qsEmail = qsParams.find(x => x.startsWith('email='))
    const qsToken = qsParams.find(x => x.startsWith('token='))
    if (qsToken) {
      const email = decodeURIComponent(qsEmail.substring(6))
      const cognitoUser = await Auth.signIn(email)
      
      const token = decodeURIComponent(qsToken.substring(6))
      try {
        const challengeResult = await Auth.sendCustomChallengeAnswer(cognitoUser, token)
      } catch (err) {
        console.log(err)
        alert('The token is invalid.')
      }
    }
  }  
})

Please note that, if we submitted a valid token then the response from Auth.sendCustomChallengeAnswer would contain the JWT tokens for the user. If the token is invalid or has expired, then the call will err because we only allow one attempt in the DefineAuthChallenge function.

7. The DefineAuthChallenge function

Cognito’s custom authentication flow behaves like a state machine. The DefineAuthChallenge function is the decision maker and instructs the user pool on what to do next every time something important happens.

As you can see from the overview of the solution, this function is engaged multiple times during an authentication session:

when the user initiates authentication, and
every time the user responds to an auth challenge.

With magic links, there is no point in giving users multiple attempts at responding to the auth challenge. If the link is invalid or expired, it will never become valid again.

So this is the state machine we want to implement:

And here’s what my DefineAuthChallenge function looks like.

const _ = require('lodash')

module.exports.handler = async (event) => {
  if (event.request.userNotFound) {
    event.response.issueTokens = false
    event.response.failAuthentication = true
    return event
  }

  if (_.isEmpty(event.request.session)) {
    // Issue new challenge
    event.response.issueTokens = false
    event.response.failAuthentication = false
    event.response.challengeName = 'CUSTOM_CHALLENGE'
  } else {
    const lastAttempt = _.last(event.request.session)
    if (lastAttempt.challengeResult === true) {
      // User gave right answer
      event.response.issueTokens = true
      event.response.failAuthentication = false
    } else {
      // User gave wrong answer
      event.response.issueTokens = false
      event.response.failAuthentication = true
    }
  }

  return event
}

When the client initiates the Cognito authentication flow, the user pool calls the DefineAuthChallenge function. At this point in the flow, the request.session array would be empty. So we set response.challengeName to CUSTOM_CHALLENGE and this instructs the user pool to invoke the CreateAuthChallenge function next.

When the client responds to the auth challenge, the user pool would call the VerifyAuthChallengeResponse function to check the user’s answer. The user pool then calls the DefineAuthChallenge function again to decide what to do next.

As mentioned before, we don’t need to give the user multiple attempts to enter the right code (as we did in the OTP example [1]). So we can make an on-the-spot decision to issue tokens or fail the authentication attempt.

8. The CreateAuthChallenge function

The CreateAuthChallenge function only needs to do one thing – to pass the encrypted token to the VerifyAuthChallengeResponse function.

module.exports.handler = async (event) => {
  event.response.publicChallengeParameters = {
    email: event.request.userAttributes.email
  }

  // the verify step would decrypt this and check the user's answer
  event.response.privateChallengeParameters = {
    challenge: event.request.userAttributes['custom:authChallenge']
  }

  return event
}

Sidenote: you might be wondering why this step is necessary since the user attributes are also available to the VerifyAuthChallengeResponse function. This is a design choice I made to “lock in” the secret token an ongoing authentication flow would use. It shouldn’t matter in this case because the front-end calls Auth.signIn and then Auth.sendCustomChallengeAnswer immediately after. There’s a very small time window where the user can initiate another authentication flow (in another window perhaps) between these two calls.

9. The VerifyAuthChallengeResponse function

We need to do several things when verifying a user’s answer:

The most obvious is, does the secret token they submitted match what we had locked in at the CreateAuthChallenge step?
Decrypt the token with our KMS key and extract the email and expiration datetime from the decrypted JSON string.
Make sure the token hasn’t expired.
Make sure the current user’s email matches what’s in the token.

If all these checks pass, then we can say the auth challenge has been answered correctly.

So, here’s my VerifyAuthChallengeResponse function.

const { decrypt } = require('../lib/encryption')

module.exports.handler = async (event) => {
  const email = event.request.userAttributes.email

  const expected = event.request.privateChallengeParameters.challenge
  if (event.request.challengeAnswer !== expected) {
    console.log("answer doesn't match current challenge token")
    event.response.answerCorrect = false
    return event
  }

  const json = await decrypt(event.request.challengeAnswer)
  const payload = JSON.parse(json)
  console.log(payload)
  
  const isExpired = new Date().toJSON() > payload.expiration
  console.log('isExpired:', isExpired)

  if (payload.email === email && !isExpired) {    
    event.response.answerCorrect = true
  } else {
    console.log("email doesn't match or token is expired")
    event.response.answerCorrect = false
  }
  
  return event
}

And that’s it.

These are the ingredients you need to implement passwordless authentication with magic links with Cognito.

Trying it out for yourself

To get a sense of how this passwordless authentication mechanism works, please feel free to try out the demo application here.

And you can find the source code for this demo on GitHub:

Wrap up

I hope you have found this article useful and helps you get more out of Cognito, a somewhat underloved service.

If you want to learn more about building serverless architecture, then check out my upcoming workshop [10] where I will be covering topics such as testing, security, observability and much more.

Hope to see you there.