Django: UnicodeDecodeError while trying to read template 500.html

I am trying to deploy my django application to a production environment with AWS Elastic Beanstalk. In my staging environment, where I have DEBUG=True, everything is fine, but when DEBUG=False I am getting the error UnicodeDecodeError while trying to read template /home/docker/code/django-app/templates/500.html

Here is my 500.html template:

{% extends "base.html" %} 
{% load i18n %}

{% block title_html %}{% trans 'Server error (500)' %}{% endblock %}

{% block content %} 
    <h1>{% trans 'Server Error <em>(500)</em>' %}</h1> 
    <p>
        {% trans "There has been an error. It's been reported to the site administrators and should be fixed shortly. Thank you for your patience." %}
    </p>

{% endblock %}