Serverless, real-time, ClamAV+Yara scanning for your S3 Buckets
Join the Slack Channel
What is clara
Clara is Python based project used to scan your S3 bucket files with ClamAV and Yara signatures. It is built on top of Upside Travel's bucket-antivirus-function project and Airbnb's (BinaryAlert)[https://github.com/airbnb/binaryalert] project. Clara combines the two functionalities into a single project with some additional improvements:
- Yara and ClamAV Scanning feature with signature updates.
- Slack/Email SNS alerts.
- DynamoDB storage support.
- Python 3.7 support.
Step 1: Building the Lambda Zip file from source
change current directory to clara and run
This will create a Build directory with the zip file containing the lambda fuction code and requirements.
Step 2: Creating the S3 buckets.
We will need three different S3 buckets for this project. One for storing the files to be scanned, second for storing clamAV definitions and third for storing your Yara rules.
- Create an S3 bucket with named
file-scanning-upload. This bucket will hold the files to be scanned by clara.
Create a second bucket to store the clamAV definitions. Name the bucket as
For this same bucket, navigate to Permissions Tab and select Bucket Policy. Copy and paste the json policy from
clamav-definition-updates-policy.jsonto this field and save. This policy gives public access to the bucket allowing it to download definitions from the internet.
Now, create the third and final bucket and name it
You can now upload your yara signature sets to this bucket. Once done, again navigate to Permissions tab and select Bucket Policy. Copy and paste the json policy defined in
Now we are all set with creating our S3 buckets and assigning the required access policies.
- Creating the rules and definition update Lambda function
Proceed to your AWS Lambda dashboard UI and create a new lambda fuction. Select "From scratch" to define our own parameters.
Name the function as
bucket-defs-updateand select the runtime as Python 3.7.
In the Designer tab, click on the left side "Add trigger" option. Select
CloudWatch Eventsand then add
rate(3 hours)for Scheduled expression. Make sure that enable trigger option is selected. Save it.
In the Function code section, select "upload from zip file" and select the Build.zip file that we created in step 1. Again, select the runtime as Python 3.7. In the Handler field add
In the environment variables field, we will have to provide the bucket names where our ClamAV and Yara rules recide. Define the following key and value in this field: Key Value AV_DEFINITION_S3_BUCKET clamav-definition-updates YARA_RULES_S3_BUCKET yara-rule-updates
For the Execution role section, we will have to create a new role to be attached to our lambda function. Head to your IAM dashboard, Click on "Roles" and "create new role". i. Name your role as
bucket-defs-update. ii. Click on "Attach policies" and then create a new policy. iii. Copy-paste the json policy from
bucket-defs-update-policy.jsonand save it. Once the policy is attached to the role, go back to your Lambda creation page and scroll to the "Execution Role" section.
In the "Execution role" section, select "Use an existing role" and select the newly created role. You might want to click on the Refresh button for your role to reflect.
In the "Basic Settings", select 512MB memory and Timout as 3 minutes.
(Optional) Choose the VPC where you want to add your lambda function.
- Creating the scanner lambda function
Create a new lambda function and name it as
bucket-scanner-function. Select the runtime environment as Python 3.7.
In the Designer tab, add a new trigger and select S3 Event.
Select your bucket name where the scanning files will be uploaded. In this example we have named it as
file-scanning-upload. In the Event Type select All object create event. Make sure that enable trigger is configured. Then click on Add.
In the Function code section slect the zip file created in Step 1. Set Python3.7 as the runtime environment. Set the value of Handler as
Define the below two environment variable key values: AV_DEFINITION_S3_BUCKET clamav-definition-updates YARA_RULES_S3_BUCKET yara-rule-updates
For the Execution Role, create a new role and name it as
bucket-scanner-function. Select Attach Policy and create a new policy. You can again name the policy as above. Copy-paste the
bucket-scanner-function-policy.jsoninto the JSON editor of the policy and save it.
Go back to the lambda function creation dashboard. In the Execution role, select use an existing role and choose the newly created role.
In the Basic settings, set the memory to 2048MB and timeout as 3 minutes. Save the lambda function.
Testing the Lambda functions
Click on Test functions and let it run with the default values.
Come back to your lambda function page. You should now have two functions created. Click on the
bucket-defs-updatefunction to launch its UI. Select the monitoring tab and you should see the graph with execution of test event details.
You can click on View Logs in CloudWatch to see the log lines created by the lambda function. You should see print lines about definitions and rules getting updated.
Go to your S3 bucket
file-scanning-uploadand add a new file there.
Now, open the monitoring page of the
bucket-scanner-function. Select View logs in CloudWatch. You should see the scanning getting initiated. If there is a detection, it should print in the log line.