技術（tech）

AWS Lambda Python: How to Fix ‘Unable to import module ‘lambda_function’: No module named ‘regex._regex” Error

 

While writing Python code for Lambda, I needed to use external modules, so I tried to load them using Lambda layers.

During this process, I got the error "AWS Lambda python: Unable to import module ‘lambda_function’: No module named ‘regex._regex’" and spent 2-3 hours figuring out how to solve it.

Now that I’ve found the solution, I’m writing this article to share it.

I hope this helps anyone facing the same issue.

Reference: layer

Contents
  1. Environment
  2. Issue
  3. Cause
  4. Solution

Environment

 

  • macOS Big Sur 11.7
  • Processor: Intel
  • Python 3.9

 

Issue

 

# First, downloaded external modules locally
python3.9 -m pip install python-binance -t python/

# Then compressed them into a zip file
zip -r layer.zip python

# Registered it as a layer in the AWS Lambda console, and

# Tried to run a Lambda function like this
import os
from binance import Client, ThreadedWebsocketManager, ThreadedDepthCacheManager

def lambda_handler(event, context):
  print('--- Start...')

 

Then…

AWS Lambda python: Unable to import module 'lambda_function': No module named 'regex._regex'

 

I got the error mentioned in the title.

I was confused because I had properly compressed the modules and loaded them, so why wasn’t it working?

 

Cause

 

The cause was the environment difference between where I ran pip install and compressed the zip file (macOS) and the Lambda execution environment. (I didn’t trace the root cause of what specifically changes due to the environment difference)

It seems that if you install on a Linux OS, register it as a layer, and load it, it works properly.

This article was the key to solving the problem:
https://stackoverflow.com/questions/64498145/aws-lambda-python-unable-to-import-module-lambda-function-no-module-named-r

The official documentation was also somewhat helpful:
https://aws.amazon.com/jp/premiumsupport/knowledge-center/lambda-import-module-error-python/

 

Solution

 

As a simple method, you can use a Python 3.9 Docker container to pip install and compress the modules, then register them as a layer.

# Execute on the host (macOS)
docker pull python:3.9
docker run -itd --name python python:3.9
docker exec -it python /bin/sh 

# Execute in the container
cd var/tmp
mkdir python                                                                                                  
python3.9 -m pip install python-binance -t python/

## Install the zip command since it's not available
apt update && apt install -y zip
## Compress to zip
zip -r layer.zip python

# Execute on the host (macOS)
## Copy the zip file created in the container
docker cp python:/var/tmp/layer.zip ./

 

Then, register the zip file created here as a layer.

When you run the Lambda function again… the error disappears and the module is properly loaded.

 

I hope this helps someone.

aws lambda
RELATED POST