Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

converts bad photo file extensions (supercedes 404/412) #691

Closed
wants to merge 1 commit into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion OpenOversight/app/config.py
Original file line number Diff line number Diff line change
Expand Up @@ -37,7 +37,8 @@ class BaseConfig(object):

# Upload Settings
MAX_CONTENT_LENGTH = 50 * 1024 * 1024
ALLOWED_EXTENSIONS = set(['png', 'jpg', 'jpeg', 'gif'])
ALLOWED_EXTENSIONS = set(['png', 'jpg', 'jpeg', 'gif', 'psd', 'tiff'])
PHOTO_REQUIRES_CONV = set(['tiff', 'psd'])

# User settings
APPROVE_REGISTRATIONS = os.environ.get('APPROVE_REGISTRATIONS', False)
Expand Down
5 changes: 4 additions & 1 deletion OpenOversight/app/main/views.py
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,8 @@
ac_can_edit_officer, add_department_query, add_unit_query,
create_incident, get_or_create, replace_list,
set_dynamic_default, create_note, get_uploaded_cropped_image,
create_description, filter_by_form, dept_choices)
create_description, filter_by_form, dept_choices,
requires_conversion, convert_image)

from .forms import (FindOfficerForm, FindOfficerIDForm, AddUnitForm,
FaceTag, AssignmentForm, DepartmentForm, AddOfficerForm,
Expand Down Expand Up @@ -848,6 +849,8 @@ def upload(department_id):
file_to_upload = request.files['file']
if not allowed_file(file_to_upload.filename):
return jsonify(error="File type not allowed!"), 415
if requires_conversion(file_to_upload.filename):
convert_image(file_to_upload.filename)
original_filename = secure_filename(file_to_upload.filename)
image_data = file_to_upload.read()

Expand Down
15 changes: 15 additions & 0 deletions OpenOversight/app/utils.py
Original file line number Diff line number Diff line change
Expand Up @@ -189,6 +189,21 @@ def allowed_file(filename):
filename.rsplit('.', 1)[1].lower() in current_app.config['ALLOWED_EXTENSIONS']


def requires_conversion(filename):
return '.' in filename and \
filename.split('.', 1)[1].lower() in current_app.config['PHOTO_REQUIRES_CONV']


def convert_image(filename):
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

does this convert e.g. tiff and psd files to jpegs or just rename them? we can use imagemagick for the conversion, i think it should work for both tiff and psds (though we should test that, each are good candidates for unit tests)

Copy link
Member Author

@ssempervirens ssempervirens Oct 15, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Right, so I didn't realize imagemagick had a good python binding? Should I just use PIL?

(So there is no way to avoid the os.open/decompression bomb issue?)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

oh right right, Pillow (Python 3 version of PIL) should do the trick! there's also a decent imagemagick API called wand if you find that Pillow is lacking support for a filetype that you want to support.

But yeah I'm not seeing a way to convert these images without using something like Pillow (short of using a third-party service).

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

According to the Pillow docs for this method (https://pillow.readthedocs.io/en/stable/reference/Image.html#PIL.Image.Image.save),

"Saves this image under the given filename. If no format is specified, the format to use is determined from the filename extension, if possible."

So it would appear this is actually converting the files to the new format, not just changing the name.

However, maybe we should have a test for this if we don't already?

try:
img_to_change = Pimage.open(filename)
except Pimage.DecompressionBombWarning:
return None
basename = os.path.splitext(filename)[0].split('.')
fixed_img = img_to_change.save(basename + 'jpeg')
return fixed_img


def get_random_image(image_query):
if image_query.count() > 0:
rand = random.randrange(0, image_query.count())
Expand Down
2 changes: 1 addition & 1 deletion OpenOversight/tests/test_utils.py
Original file line number Diff line number Diff line change
Expand Up @@ -151,7 +151,7 @@ def test_s3_upload_jpeg(mockdata):


def test_user_can_submit_allowed_file(mockdata):
for file_to_submit in ['valid_photo.png', 'valid_photo.jpg', 'valid.photo.jpg', 'valid_photo.PNG', 'valid_photo.JPG']:
for file_to_submit in ['valid_photo.png', 'valid_photo.jpg', 'valid.photo.jpg', 'valid_photo.PNG', 'valid_photo.JPG', 'valid_photo.TIFF', 'valid_photo.tiff', 'valid_photo.psd', 'valid_photo.PSD']:
assert OpenOversight.app.utils.allowed_file(file_to_submit) is True


Expand Down