diff --git a/.github/workflows/main.yml b/.github/workflows/main.yml
new file mode 100644
index 000000000..a1505f01e
--- /dev/null
+++ b/.github/workflows/main.yml
@@ -0,0 +1,48 @@
+name: CI
+
+# Controls when the action will run.
+on:
+ # Triggers the workflow on push or pull request events but only for the develop branch
+ push:
+ branches:
+ - develop
+ - main
+ pull_request:
+ branches:
+ - develop
+ - main
+
+ # Allows you to run this workflow manually from the Actions tab
+ workflow_dispatch:
+
+# A workflow run is made up of one or more jobs that can run sequentially or in parallel
+jobs:
+ # This workflow contains a single job called "build"
+ build:
+ # The type of runner that the job will run on
+ runs-on: ubuntu-latest
+ env:
+ FLASK_APP: OpenOversight.app
+ strategy:
+ matrix:
+ python-version: [3.5, 3.6, 3.7, 3.8]
+
+ # Steps represent a sequence of tasks that will be executed as part of the job
+ steps:
+ # Checks-out your repository under $GITHUB_WORKSPACE, so your job can access it
+ - uses: actions/checkout@v2
+ - name: Set up Python ${{ matrix.python-version }}
+ uses: actions/setup-python@v2
+ with:
+ python-version: ${{ matrix.python-version }}
+ - name: Install dependencies
+ run: |
+ python -m pip install --upgrade pip
+ pip install -r requirements.txt -r dev-requirements.txt
+
+ # Run flake8
+ - name: Run flake8
+ run: flake8 --ignore=E501,E722,W504,W503
+ # Runs tests
+ - name: Run tests
+ run: FLASK_ENV=testing pytest --doctest-modules -n 4 --dist=loadfile -v OpenOversight/tests/ OpenOversight/app;
diff --git a/CONTRIB.md b/CONTRIB.md
index d7ea1f28c..ccec299f9 100644
--- a/CONTRIB.md
+++ b/CONTRIB.md
@@ -2,12 +2,24 @@
First, thanks for being interested in helping us out! If you find an issue you're interested in, feel free to make a comment about how you're thinking of approaching implementing it in the issue and we can give you feedback. Please also read our [code of conduct](/CODE_OF_CONDUCT.md) before getting started.
-## Submitting a PR
+## Submitting a Pull Request (PR)
-When you come to implement your new feature, you should branch off `develop` and add commits to implement your feature. If your git history is not so clean, please do rewrite before you submit your PR - if you're not sure if you need to do this, go ahead and submit and we can let you know when you submit.
+When you come to implement your new feature, clone the repository and then create a branch off `develop` locally and add commits to implement your feature.
+
+If your git history is not so clean, please do rewrite before you submit your PR - if you're not sure if you need to do this, go ahead and submit and we can let you know when you submit.
+
+To submit your changes for review you have to fork the repository, push your new branch there and then create a Pull Request with `OpenOversight:develop` as the target.
Use [PULL_REQUEST_TEMPLATE.md](/PULL_REQUEST_TEMPLATE.md) to create the description for your PR! (The template should populate automatically when you go to open the pull request.)
+### Recommended privacy settings
+Whenever you make a commit with `git` the name and email saved locally is stored with that commit and will become part of the public history of the project. This can be an unwanted, for example when using a work computer. We recommond changing the email-settings in the github account at https://github.com/settings/emails and selecting "Keep my email addresses private" as well as "Block command line pushes that expose my email". Also find your github-email address of the form `+@users.noreply.github.com` in that section. Then you can change the email and username stored with your commits by running the following commands
+```
+git config user.email ""
+git config user.name ""
+```
+This will make sure that all commits you make locally are associated with your github account and do not contain any additional identifying information. More detailed information on this topic can be found [here](https://docs.github.com/en/free-pro-team@latest/github/setting-up-and-managing-your-github-user-account/setting-your-commit-email-address).
+
### Linting / Style Checks
`flake8` is a tool for automated linting and style checks. Be sure to run `flake8` and fix any errors before submitting a PR.
@@ -207,7 +219,19 @@ Next, in your terminal run `docker ps` to find the container id of the `openover
## Debugging OpenOversight - Use pdb with a test
If you want to run an individual test in debug mode, use the below command.
-```yml
-`docker-compose run --rm web pytest --pdb -v tests/ -k `
+```bash
+docker-compose run --rm web pytest --pdb -v tests/ -k
```
+
+where `` is the name of a single test function, such as `test_ac_cannot_add_new_officer_not_in_their_dept`
+
+Similarly, you can run all the tests in a file by specifying the file path:
+
+```bash
+docker-compose run --rm web pytest --pdb -v path/to/test/file
+```
+
+where `path/to/test/file` is the relative file path, minus the initial `OpenOversight`, such as
+`tests/routes/test_officer_and_department.py`.
+
Again, add `import pdb` to the file you want to debug, then write `pdb.set_trace()` wherever you want to drop a breakpoint. Once the test is up and running in your terminal, you can debug it using pdb prompts.
\ No newline at end of file
diff --git a/DEPLOY.md b/DEPLOY.md
index 41d5cbfe2..86759b676 100644
--- a/DEPLOY.md
+++ b/DEPLOY.md
@@ -38,14 +38,20 @@ You'll need to create an AWS account, if you don't already have one. Then, you'l
For the officer identification UI to work, you'll need to create a CORS policy for the S3 bucket used with OpenOversight. In the AWS UI, this is done by navigating to the listing of buckets, clicking on the name of your bucket, and choosing the Permissions tab, and then "CORS configuration". Since we're not doing anything fancier than making a web browser GET it, we can just use the default policy:
```
-
-
- *
- GET
- 3000
- Authorization
-
-
+[
+ {
+ "AllowedOrigins": [
+ "*"
+ ],
+ "AllowedMethods": [
+ "GET"
+ ],
+ "MaxAgeSeconds": 3000,
+ "AllowedHeaders": [
+ "Authorizations"
+ ]
+ }
+]
```
If you don't click "Save" on that policy, however, the policy will not actually be applied.
diff --git a/OpenOversight/app/__init__.py b/OpenOversight/app/__init__.py
index 2ae49edb0..b609d5cb3 100644
--- a/OpenOversight/app/__init__.py
+++ b/OpenOversight/app/__init__.py
@@ -11,6 +11,7 @@
from flask_mail import Mail
from flask_migrate import Migrate
from flask_sitemap import Sitemap
+from flask_wtf.csrf import CSRFProtect
from .config import config
@@ -26,6 +27,7 @@
default_limits=["100 per minute", "5 per second"])
sitemap = Sitemap()
+csrf = CSRFProtect()
def create_app(config_name='default'):
@@ -40,6 +42,7 @@ def create_app(config_name='default'):
login_manager.init_app(app)
limiter.init_app(app)
sitemap.init_app(app)
+ csrf.init_app(app)
from .main import main as main_blueprint # noqa
app.register_blueprint(main_blueprint)
diff --git a/OpenOversight/app/auth/forms.py b/OpenOversight/app/auth/forms.py
index 5a5b2f071..df006c5b3 100644
--- a/OpenOversight/app/auth/forms.py
+++ b/OpenOversight/app/auth/forms.py
@@ -88,7 +88,11 @@ class EditUserForm(Form):
ac_department = QuerySelectField('Department', validators=[Optional()],
query_factory=dept_choices, get_label='name', allow_blank=True)
is_administrator = BooleanField('Is administrator?', false_values={'False', 'false', ''})
- submit = SubmitField(label='Update')
+ is_disabled = BooleanField('Disabled?', false_values={'False', 'false', ''})
+ approved = BooleanField('Approved?', false_values={'False', 'false', ''})
+ submit = SubmitField(label='Update', false_values={'False', 'false', ''})
+ resend = SubmitField(label='Resend', false_values={'False', 'false', ''})
+ delete = SubmitField(label='Delete', false_values={'False', 'false', ''})
def validate(self):
success = super(EditUserForm, self).validate()
diff --git a/OpenOversight/app/auth/views.py b/OpenOversight/app/auth/views.py
index feb9393d1..d2102c15e 100644
--- a/OpenOversight/app/auth/views.py
+++ b/OpenOversight/app/auth/views.py
@@ -1,7 +1,4 @@
-from future.utils import iteritems
-
from flask import render_template, redirect, request, url_for, flash, current_app
-from flask.views import MethodView
from flask_login import login_user, logout_user, login_required, \
current_user
from . import auth
@@ -101,7 +98,7 @@ def register():
return render_template('auth/register.html', form=form, jsloads=jsloads)
-@auth.route('/confirm/')
+@auth.route('/confirm/', methods=['GET'])
@login_required
def confirm(token):
if current_user.confirmed:
@@ -237,144 +234,76 @@ def change_dept():
return render_template('auth/change_dept_pref.html', form=form)
-class UserAPI(MethodView):
- decorators = [admin_required]
-
- def get(self, user_id):
- # isolate the last part of the url
- end_of_url = request.url.split('/')[-1].split('?')[0]
+@auth.route('/users/', methods=['GET'])
+@admin_required
+def get_users():
+ if request.args.get('page'):
+ page = int(request.args.get('page'))
+ else:
+ page = 1
+ USERS_PER_PAGE = int(current_app.config['USERS_PER_PAGE'])
+ users = User.query.order_by(User.username) \
+ .paginate(page, USERS_PER_PAGE, False)
- if user_id is None:
- if request.args.get('page'):
- page = int(request.args.get('page'))
- else:
- page = 1
- USERS_PER_PAGE = int(current_app.config['USERS_PER_PAGE'])
- users = User.query.order_by(User.email) \
- .paginate(page, USERS_PER_PAGE, False)
+ return render_template('auth/users.html', objects=users)
- return render_template('auth/users.html', objects=users)
- else:
- user = User.query.get(user_id)
- if not user:
- return render_template('403.html'), 403
-
- actions = ['delete', 'enable', 'disable', 'resend', 'approve']
- if end_of_url in actions:
- action = getattr(self, end_of_url, None)
- return action(user)
- else:
- form = EditUserForm(
- email=user.email,
- is_area_coordinator=user.is_area_coordinator,
- ac_department=user.ac_department,
- is_administrator=user.is_administrator)
- return render_template('auth/user.html', user=user, form=form)
- def post(self, user_id):
- # isolate the last part of the url
- end_of_url = request.url.split('/')[-1].split('?')[0]
+@auth.route('/users/', methods=['GET', 'POST'])
+@admin_required
+def edit_user(user_id):
+ user = User.query.get(user_id)
+ if not user:
+ return render_template('404.html'), 404
- user = User.query.get(user_id)
+ if request.method == 'GET':
+ form = EditUserForm(obj=user)
+ return render_template('auth/user.html', user=user, form=form)
+ elif request.method == 'POST':
form = EditUserForm()
+ if form.delete.data:
+ # forward to confirm delete
+ return redirect(url_for('auth.delete_user', user_id=user.id))
+ elif form.resend.data:
+ return admin_resend_confirmation(user)
+ elif form.submit.data:
+ if form.validate_on_submit():
+ # prevent user from removing own admin rights (or disabling account)
+ if user.id == current_user.id:
+ flash('You cannot edit your own account!')
+ form = EditUserForm(obj=user)
+ return render_template('auth/user.html', user=user, form=form)
+ form.populate_obj(user)
+ db.session.add(user)
+ db.session.commit()
+ flash('{} has been updated!'.format(user.username))
+ return redirect(url_for('auth.edit_user', user_id=user.id))
+ else:
+ flash('Invalid entry')
+ return render_template('auth/user.html', user=user, form=form)
- if user and end_of_url and end_of_url == 'delete':
- return self.delete(user)
- elif user and form.validate_on_submit():
- for field, data in iteritems(form.data):
- setattr(user, field, data)
-
- db.session.add(user)
- db.session.commit()
- flash('{} has been updated!'.format(user.username))
- return redirect(url_for('auth.user_api'))
- elif not form.validate_on_submit():
- flash('Invalid entry')
- return render_template('auth/user.html', user=user, form=form)
- else:
- return render_template('403.html'), 403
-
- def delete(self, user):
- if request.method == 'POST':
- username = user.username
- db.session.delete(user)
- db.session.commit()
- flash('User {} has been deleted!'.format(username))
- return redirect(url_for('auth.user_api'))
-
- return render_template('auth/user_delete.html', user=user)
- def enable(self, user):
- if not user.is_disabled:
- flash('User {} is already enabled.'.format(user.username))
- else:
- user.is_disabled = False
- db.session.add(user)
- db.session.commit()
- flash('User {} has been enabled!'.format(user.username))
- return redirect(url_for('auth.user_api'))
+@auth.route('/users//delete', methods=['GET', 'POST'])
+@admin_required
+def delete_user(user_id):
+ user = User.query.get(user_id)
+ if not user or user.is_administrator:
+ return render_template('403.html'), 403
+ if request.method == 'POST':
+ username = user.username
+ db.session.delete(user)
+ db.session.commit()
+ flash('User {} has been deleted!'.format(username))
+ return redirect(url_for('auth.get_users'))
- def disable(self, user):
- if user.is_disabled:
- flash('User {} is already disabled.'.format(user.username))
- else:
- user.is_disabled = True
- db.session.add(user)
- db.session.commit()
- flash('User {} has been disabled!'.format(user.username))
- return redirect(url_for('auth.user_api'))
+ return render_template('auth/user_delete.html', user=user)
- def resend(self, user):
- if user.confirmed:
- flash('User {} is already confirmed.'.format(user.username))
- else:
- token = user.generate_confirmation_token()
- send_email(user.email, 'Confirm Your Account',
- 'auth/email/confirm', user=user, token=token)
- flash('A new confirmation email has been sent to {}.'.format(user.email))
- return redirect(url_for('auth.user_api'))
- def approve(self, user):
- if user.approved:
- flash('User {} is already approved.'.format(user.username))
- else:
- user.approved = True
- db.session.add(user)
- db.session.commit()
- token = user.generate_confirmation_token()
- send_email(user.email, 'Confirm Your Account',
- 'auth/email/confirm', user=user, token=token)
- flash('User {} has been approved!'.format(user.username))
- return redirect(url_for('auth.user_api'))
-
-
-user_view = UserAPI.as_view('user_api')
-auth.add_url_rule(
- '/users/',
- defaults={'user_id': None},
- view_func=user_view,
- methods=['GET'])
-auth.add_url_rule(
- '/users/',
- view_func=user_view,
- methods=['GET', 'POST'])
-auth.add_url_rule(
- '/users//delete',
- view_func=user_view,
- methods=['GET', 'POST'])
-auth.add_url_rule(
- '/users//enable',
- view_func=user_view,
- methods=['GET'])
-auth.add_url_rule(
- '/users//disable',
- view_func=user_view,
- methods=['GET'])
-auth.add_url_rule(
- '/users//resend',
- view_func=user_view,
- methods=['GET'])
-auth.add_url_rule(
- '/users//approve',
- view_func=user_view,
- methods=['GET'])
+def admin_resend_confirmation(user):
+ if user.confirmed:
+ flash('User {} is already confirmed.'.format(user.username))
+ else:
+ token = user.generate_confirmation_token()
+ send_email(user.email, 'Confirm Your Account',
+ 'auth/email/confirm', user=user, token=token)
+ flash('A new confirmation email has been sent to {}.'.format(user.email))
+ return redirect(url_for('auth.get_users'))
diff --git a/OpenOversight/app/commands.py b/OpenOversight/app/commands.py
index 2ca6b6989..0983ff113 100644
--- a/OpenOversight/app/commands.py
+++ b/OpenOversight/app/commands.py
@@ -3,7 +3,8 @@
import csv
import sys
from builtins import input
-from datetime import datetime
+from datetime import datetime, date
+from dateutil.parser import parse
from getpass import getpass
from typing import Dict, List
@@ -11,9 +12,10 @@
from flask import current_app
from flask.cli import with_appcontext
+from .models import db, Assignment, Department, Officer, User, Salary, Job
+from .utils import get_officer, str_is_true, normalize_gender, prompt_yes_no
+
from .csv_imports import import_csv_files
-from .models import Assignment, Department, Job, Officer, Salary, User, db
-from .utils import get_officer, prompt_yes_no, str_is_true
@click.command()
@@ -155,6 +157,8 @@ def set_field_from_row(row, obj, attribute, allow_blank=True, fieldname=None):
val = datetime.strptime(row[fieldname], '%Y-%m-%d').date()
except ValueError:
val = row[fieldname]
+ if attribute == 'gender':
+ val = normalize_gender(val)
setattr(obj, attribute, val)
@@ -162,9 +166,12 @@ def update_officer_from_row(row, officer, update_static_fields=False):
def update_officer_field(fieldname):
if fieldname not in row:
return
- if row[fieldname] == '':
- row[fieldname] = None
+
+ if fieldname == 'gender':
+ row[fieldname] = normalize_gender(row[fieldname])
+
if row[fieldname] and getattr(officer, fieldname) != row[fieldname]:
+
ImportLog.log_change(
officer,
'Updated {}: {} --> {}'.format(
@@ -175,6 +182,7 @@ def update_officer_field(fieldname):
update_officer_field('last_name')
update_officer_field('first_name')
update_officer_field('middle_initial')
+
update_officer_field('suffix')
update_officer_field('gender')
@@ -190,7 +198,24 @@ def update_officer_field(fieldname):
if row[fieldname] == '':
row[fieldname] = None
old_value = getattr(officer, fieldname)
- new_value = row[fieldname]
+ # If we're expecting a date type, attempt to parse row[fieldname] as a datetime
+ # This also normalizes all date formats, ensuring the following comparison works properly
+ if isinstance(old_value, (date, datetime)):
+ try:
+ new_value = parse(row[fieldname])
+ if isinstance(old_value, date):
+ new_value = new_value.date()
+ except Exception as e:
+ msg = 'Field {} is a date-type, but "{}" was specified for Officer {} {} and cannot be parsed as a date-type.\nError message from dateutil: {}'.format(
+ fieldname,
+ row[fieldname],
+ officer.first_name,
+ officer.last_name,
+ e
+ )
+ raise Exception(msg)
+ else:
+ new_value = row[fieldname]
if old_value is None:
update_officer_field(fieldname)
elif str(old_value) != str(new_value):
@@ -264,8 +289,9 @@ def try_else_false(comparable):
def process_assignment(row, officer, compare=False):
assignment_fields = {
- 'required': ['job_title'],
+ 'required': [],
'optional': [
+ 'job_title',
'star_no',
'unit_id',
'star_date',
@@ -292,13 +318,13 @@ def process_assignment(row, officer, compare=False):
i += 1
if i == len(assignment_fieldnames):
job_title = job.job_title
- if (job_title and 'job_title' in row and row['job_title'] == job_title) or \
+ if (job_title and row.get('job_title', 'Not Sure') == job_title) or \
(not job_title and ('job_title' not in row or not row['job_title'])):
# Found match, so don't add new assignment
add_assignment = False
if add_assignment:
job = Job.query\
- .filter_by(job_title=row['job_title'],
+ .filter_by(job_title=row.get('job_title', 'Not Sure'),
department_id=officer.department_id)\
.one_or_none()
if not job:
@@ -384,7 +410,15 @@ def process_salary(row, officer, compare=False):
@with_appcontext
def bulk_add_officers(filename, no_create, update_by_name, update_static_fields):
"""Add or update officers from a CSV file."""
+
+ encoding = 'utf-8'
+
+ # handles unicode errors that can occur when the file was made in Excel
with open(filename, 'r') as f:
+ if u'\ufeff' in f.readline():
+ encoding = 'utf-8-sig'
+
+ with open(filename, 'r', encoding=encoding) as f:
ImportLog.clear_logs()
csvfile = csv.DictReader(f)
departments = {}
diff --git a/OpenOversight/app/config.py b/OpenOversight/app/config.py
index 92e3e0cdd..597c8d6db 100644
--- a/OpenOversight/app/config.py
+++ b/OpenOversight/app/config.py
@@ -50,6 +50,7 @@ def init_app(app):
class DevelopmentConfig(BaseConfig):
DEBUG = True
+ SQLALCHEMY_ECHO = True
SQLALCHEMY_DATABASE_URI = os.environ.get('SQLALCHEMY_DATABASE_URI')
NUM_OFFICERS = 15000
SITEMAP_URL_SCHEME = 'http'
@@ -79,5 +80,5 @@ def init_app(cls, app): # pragma: no cover
'development': DevelopmentConfig,
'testing': TestingConfig,
'production': ProductionConfig,
- 'default': DevelopmentConfig
}
+config['default'] = config.get(os.environ.get('FLASK_ENV', ""), DevelopmentConfig)
diff --git a/OpenOversight/app/main/choices.py b/OpenOversight/app/main/choices.py
index 2d3cbd7a3..2f159eb00 100644
--- a/OpenOversight/app/main/choices.py
+++ b/OpenOversight/app/main/choices.py
@@ -9,8 +9,7 @@
('PACIFIC ISLANDER', 'Pacific Islander'),
('Other', 'Other'), ('Not Sure', 'Not Sure')]
-GENDER_CHOICES = [('M', 'Male'), ('F', 'Female'), ('Other', 'Other'),
- ('Not Sure', 'Not Sure')]
+GENDER_CHOICES = [('Not Sure', 'Not Sure'), ('M', 'Male'), ('F', 'Female'), ('Other', 'Other')]
STATE_CHOICES = [('', '')] + [(state.abbr, state.name) for state in states.STATES]
LINK_CHOICES = [('', ''), ('link', 'Link'), ('video', 'YouTube Video'), ('other_video', 'Other Video')]
diff --git a/OpenOversight/app/main/forms.py b/OpenOversight/app/main/forms.py
index fc40c130f..1e739237c 100644
--- a/OpenOversight/app/main/forms.py
+++ b/OpenOversight/app/main/forms.py
@@ -17,6 +17,14 @@
import datetime
import re
+# Normalizes the "not sure" option to what it needs to be when writing to the database.
+# Note this should only be used for forms which save a record to the DB--not those that
+# are used to look up existing records.
+db_genders = list(GENDER_CHOICES)
+for index, choice in enumerate(db_genders):
+ if choice == ('Not Sure', 'Not Sure'):
+ db_genders[index] = (None, 'Not Sure') # type: ignore
+
def allowed_values(choices, empty_allowed=True):
return [x[0] for x in choices if empty_allowed or x[0]]
@@ -205,8 +213,12 @@ class AddOfficerForm(Form):
validators=[AnyOf(allowed_values(SUFFIX_CHOICES))])
race = SelectField('Race', default='WHITE', choices=RACE_CHOICES,
validators=[AnyOf(allowed_values(RACE_CHOICES))])
- gender = SelectField('Gender', default='M', choices=GENDER_CHOICES,
- validators=[AnyOf(allowed_values(GENDER_CHOICES))])
+ gender = SelectField(
+ 'Gender',
+ choices=GENDER_CHOICES,
+ coerce=lambda x: None if x == 'Not Sure' else x,
+ validators=[AnyOf(allowed_values(db_genders))]
+ )
star_no = StringField('Badge Number', default='', validators=[
Regexp(r'\w*'), Length(max=50)])
unique_internal_identifier = StringField('Unique Internal Identifier', default='', validators=[Regexp(r'\w*'), Length(max=50)])
@@ -258,8 +270,12 @@ class EditOfficerForm(Form):
validators=[AnyOf(allowed_values(SUFFIX_CHOICES))])
race = SelectField('Race', choices=RACE_CHOICES, coerce=lambda x: x or None,
validators=[AnyOf(allowed_values(RACE_CHOICES))])
- gender = SelectField('Gender', choices=GENDER_CHOICES, coerce=lambda x: x or None,
- validators=[AnyOf(allowed_values(GENDER_CHOICES))])
+ gender = SelectField(
+ 'Gender',
+ choices=GENDER_CHOICES,
+ coerce=lambda x: None if x == 'Not Sure' else x,
+ validators=[AnyOf(allowed_values(db_genders))]
+ )
employment_date = DateField('Employment Date', validators=[Optional()])
birth_year = IntegerField('Birth Year', validators=[Optional()])
unique_internal_identifier = StringField('Unique Internal Identifier',
diff --git a/OpenOversight/app/main/views.py b/OpenOversight/app/main/views.py
index 43fdf0e05..65dcfed0e 100644
--- a/OpenOversight/app/main/views.py
+++ b/OpenOversight/app/main/views.py
@@ -5,11 +5,12 @@
import re
from sqlalchemy.exc import IntegrityError
from sqlalchemy.orm.exc import NoResultFound
+from sqlalchemy.orm import selectinload
import sys
from traceback import format_exc
from flask import (abort, render_template, request, redirect, url_for,
- flash, current_app, jsonify, Response, Markup)
+ flash, current_app, jsonify, Response)
from flask_login import current_user, login_required, login_user
from . import main
@@ -330,23 +331,6 @@ def edit_salary(officer_id, salary_id):
return render_template('add_edit_salary.html', form=form, update=True)
-@main.route('/user/toggle/', methods=['POST'])
-@login_required
-@admin_required
-def toggle_user(uid):
- try:
- user = User.query.filter_by(id=uid).one()
- if user.is_disabled:
- user.is_disabled = False
- elif not user.is_disabled:
- user.is_disabled = True
- db.session.commit()
- flash('Updated user status')
- except NoResultFound:
- flash('Unknown error occurred')
- return redirect(url_for('main.profile', username=user.username))
-
-
@main.route('/image/')
@login_required
def display_submission(image_id):
@@ -375,6 +359,9 @@ def display_tag(tag_id):
def classify_submission(image_id, contains_cops):
try:
image = Image.query.filter_by(id=image_id).one()
+ if image.contains_cops is not None and not current_user.is_administrator:
+ flash('Only administrator can re-classify image')
+ return redirect(redirect_url())
image.user_id = current_user.get_id()
if contains_cops == 1:
image.contains_cops = True
@@ -472,9 +459,7 @@ def edit_department(department_id):
if Assignment.query.filter(Assignment.job_id.in_([rank.id])).count() != 0:
failed_deletions.append(rank)
for rank in failed_deletions:
- formatted_rank = rank.job_title.replace(" ", "+")
- link = '/department/{}?name=&badge=&unique_internal_identifier=&rank={}&min_age=16&max_age=100&submit=Submit'.format(department_id, formatted_rank)
- flash(Markup('You attempted to delete a rank, {}, that is in use by the linked officers.'.format(rank, link)))
+ flash('You attempted to delete a rank, {}, that is still in use'.format(rank))
return redirect(url_for('main.edit_department', department_id=department_id))
for (new_rank, order) in new_ranks:
@@ -540,15 +525,23 @@ def list_officer(department_id, page=1, race=[], gender=[], rank=[], min_age='16
form_data['gender'] = request.args.getlist('gender')
unit_choices = [(unit.id, unit.descrip) for unit in Unit.query.filter_by(department_id=department_id).order_by(Unit.descrip.asc()).all()]
- rank_choices = [jc[0] for jc in db.session.query(Job.job_title, Job.order).filter_by(department_id=department_id, is_sworn_officer=True).order_by(Job.order).all()]
+ rank_choices = [jc[0] for jc in db.session.query(Job.job_title, Job.order).filter_by(department_id=department_id).order_by(Job.order).all()]
if request.args.get('rank') and all(rank in rank_choices for rank in request.args.getlist('rank')):
form_data['rank'] = request.args.getlist('rank')
- officers = filter_by_form(form_data, Officer.query, department_id).filter(Officer.department_id == department_id).order_by(Officer.last_name, Officer.first_name, Officer.id).paginate(page, OFFICERS_PER_PAGE, False)
+ officers = filter_by_form(
+ form_data, Officer.query, department_id
+ ).filter(Officer.department_id == department_id)
+ officers = officers.options(selectinload(Officer.face))
+ officers = officers.order_by(Officer.last_name, Officer.first_name, Officer.id)
+ officers = officers.paginate(page, OFFICERS_PER_PAGE, False)
for officer in officers.items:
- officer_face = officer.face.order_by(Face.featured.desc()).first()
- if officer_face:
- officer.image = officer_face.image.filepath
+ officer_face = sorted(officer.face, key=lambda x: x.featured, reverse=True)
+
+ # could do some extra work to not lazy load images but load them all together
+ # but we would want to ensure to only load the first picture of each officer
+ if officer_face and officer_face[0].image:
+ officer.image = officer_face[0].image.filepath
choices = {
'race': RACE_CHOICES,
@@ -773,6 +766,9 @@ def label_data(department_id=None, image_id=None):
image = get_random_image(image_query)
if image:
+ if image.is_tagged and not current_user.is_administrator:
+ flash('This image cannot be tagged anymore')
+ return redirect(url_for('main.label_data'))
proper_path = serve_image(image.filepath)
else:
proper_path = None
diff --git a/OpenOversight/app/models.py b/OpenOversight/app/models.py
index f03605988..3b1731530 100644
--- a/OpenOversight/app/models.py
+++ b/OpenOversight/app/models.py
@@ -1,9 +1,10 @@
import re
+from datetime import date
from flask_sqlalchemy import SQLAlchemy
from flask_sqlalchemy.model import DefaultMeta
from sqlalchemy.orm import validates
-from sqlalchemy import UniqueConstraint
+from sqlalchemy import UniqueConstraint, CheckConstraint
from werkzeug.security import generate_password_hash, check_password_hash
from itsdangerous import TimedJSONWebSignatureSerializer as Serializer
from itsdangerous import BadSignature, BadData
@@ -98,25 +99,28 @@ class Officer(BaseModel):
middle_initial = db.Column(db.String(120), unique=False, nullable=True)
suffix = db.Column(db.String(120), index=True, unique=False)
race = db.Column(db.String(120), index=True, unique=False)
- gender = db.Column(db.String(120), index=True, unique=False)
+ gender = db.Column(db.String(5), index=True, unique=False, nullable=True)
employment_date = db.Column(db.Date, index=True, unique=False, nullable=True)
birth_year = db.Column(db.Integer, index=True, unique=False, nullable=True)
assignments = db.relationship('Assignment', backref='officer', lazy='dynamic')
assignments_lazy = db.relationship('Assignment')
- face = db.relationship('Face', backref='officer', lazy='dynamic')
+ face = db.relationship('Face', backref='officer')
department_id = db.Column(db.Integer, db.ForeignKey('departments.id'))
department = db.relationship('Department', backref='officers')
unique_internal_identifier = db.Column(db.String(50), index=True, unique=True, nullable=True)
- # we don't expect to pull up officers via link often so we make it lazy.
+
links = db.relationship(
'Link',
secondary=officer_links,
- lazy='subquery',
backref=db.backref('officers', lazy=True))
notes = db.relationship('Note', back_populates='officer', order_by='Note.date_created')
descriptions = db.relationship('Description', back_populates='officer', order_by='Description.date_created')
salaries = db.relationship('Salary', back_populates='officer', order_by='Salary.year.desc()')
+ __table_args__ = (
+ CheckConstraint("gender in ('M', 'F', 'Other')", name='gender_options'),
+ )
+
def full_name(self):
if self.middle_initial:
middle_initial = self.middle_initial + '.' if len(self.middle_initial) == 1 else self.middle_initial
@@ -135,24 +139,20 @@ def race_label(self):
return label
def gender_label(self):
+ if self.gender is None:
+ return 'Data Missing'
from .main.choices import GENDER_CHOICES
for gender, label in GENDER_CHOICES:
if self.gender == gender:
return label
def job_title(self):
- if self.assignments.all():
- return self.assignments\
- .order_by(self.assignments[0].__table__.c.star_date.desc())\
- .first()\
- .job.job_title
+ if self.assignments_lazy:
+ return max(self.assignments_lazy, key=lambda x: x.star_date or date.min).job.job_title
def badge_number(self):
- if self.assignments.all():
- return self.assignments\
- .order_by(self.assignments[0].__table__.c.star_date.desc())\
- .first()\
- .star_no
+ if self.assignments_lazy:
+ return max(self.assignments_lazy, key=lambda x: x.star_date or date.min).star_no
def __repr__(self):
if self.unique_internal_identifier:
diff --git a/OpenOversight/app/templates/auth/email/new_confirmation.html b/OpenOversight/app/templates/auth/email/new_confirmation.html
index cdf4fd245..3397904fa 100644
--- a/OpenOversight/app/templates/auth/email/new_confirmation.html
+++ b/OpenOversight/app/templates/auth/email/new_confirmation.html
@@ -4,9 +4,9 @@
Username: {{ user.username }}
Email: {{ user.email }}
-To view or delete this user, please click here.
+To view or delete this user, please click here.
Alternatively, you can paste the following link in your browser's address bar:
-{{ url_for('auth.user_api', _external=True) }}
+{{ url_for('auth.edit_user', user_id=user.id, _external=True) }}
Sincerely,
The OpenOversight Team
Note: replies to this email address are not monitored.
diff --git a/OpenOversight/app/templates/auth/email/new_confirmation.txt b/OpenOversight/app/templates/auth/email/new_confirmation.txt
index c81db9693..a6328ac0e 100644
--- a/OpenOversight/app/templates/auth/email/new_confirmation.txt
+++ b/OpenOversight/app/templates/auth/email/new_confirmation.txt
@@ -7,7 +7,7 @@ Email: {{ user.email }}
To view or delete this user, click on the following link:
-{{ url_for('auth.user_api', _external=True) }}
+{{ url_for('auth.edit_user', user_id=user.id, _external=True) }}
Sincerely,
diff --git a/OpenOversight/app/templates/auth/email/new_registration.html b/OpenOversight/app/templates/auth/email/new_registration.html
index c86e42dc7..120ad3309 100644
--- a/OpenOversight/app/templates/auth/email/new_registration.html
+++ b/OpenOversight/app/templates/auth/email/new_registration.html
@@ -4,9 +4,9 @@
Username: {{ user.username }}
Email: {{ user.email }}
-To approve or delete this user, please click here.
+To approve or delete this user, please click here.
Alternatively, you can paste the following link in your browser's address bar:
-{{ url_for('auth.user_api', _external=True) }}
+{{ url_for('auth.edit_user', user_id=user.id, _external=True) }}
Sincerely,
The OpenOversight Team
Note: replies to this email address are not monitored.
diff --git a/OpenOversight/app/templates/auth/email/new_registration.txt b/OpenOversight/app/templates/auth/email/new_registration.txt
index 20f8abf1f..5840085a9 100644
--- a/OpenOversight/app/templates/auth/email/new_registration.txt
+++ b/OpenOversight/app/templates/auth/email/new_registration.txt
@@ -7,7 +7,7 @@ Email: {{ user.email }}
To approve or delete this user, click on the following link:
-{{ url_for('auth.user_api', _external=True) }}
+{{ url_for('auth.edit_user', user_id=user.id, _external=True) }}
Sincerely,
diff --git a/OpenOversight/app/templates/auth/user.html b/OpenOversight/app/templates/auth/user.html
index 2099964c2..0d5764ab7 100644
--- a/OpenOversight/app/templates/auth/user.html
+++ b/OpenOversight/app/templates/auth/user.html
@@ -13,7 +13,7 @@
diff --git a/OpenOversight/app/templates/auth/user_delete.html b/OpenOversight/app/templates/auth/user_delete.html
index afa8db1ed..9e8756a09 100644
--- a/OpenOversight/app/templates/auth/user_delete.html
+++ b/OpenOversight/app/templates/auth/user_delete.html
@@ -11,7 +11,8 @@
Are you sure you want to delete this user?
This cannot be undone.
-
diff --git a/OpenOversight/app/templates/auth/users.html b/OpenOversight/app/templates/auth/users.html
index 8ca8aca26..e32752b9a 100644
--- a/OpenOversight/app/templates/auth/users.html
+++ b/OpenOversight/app/templates/auth/users.html
@@ -10,8 +10,8 @@
{% with paginate=objects,
- next_url=url_for('auth.user_api', page=objects.next_num),
- prev_url=url_for('auth.user_api', page=objects.prev_num),
+ next_url=url_for('auth.get_users', page=objects.next_num),
+ prev_url=url_for('auth.get_users', page=objects.prev_num),
location='top' %}
{% include "partials/paginate_nav.html" %}
{% endwith %}
@@ -29,20 +29,11 @@
{% for user in objects.items %}
- {{ user.username }}
+ {{ user.username }}
|
{{ user.email }} |
@@ -80,8 +71,8 @@
{% with paginate=objects,
- next_url=url_for('auth.user_api', page=objects.next_num),
- prev_url=url_for('auth.user_api', page=objects.prev_num),
+ next_url=url_for('auth.get_users', page=objects.next_num),
+ prev_url=url_for('auth.get_users', page=objects.prev_num),
location='bottom' %}
{% include "partials/paginate_nav.html" %}
{% endwith %}
diff --git a/OpenOversight/app/templates/description_delete.html b/OpenOversight/app/templates/description_delete.html
index 1a33ff981..1003a109a 100644
--- a/OpenOversight/app/templates/description_delete.html
+++ b/OpenOversight/app/templates/description_delete.html
@@ -15,6 +15,7 @@
Are you sure you want to delete this description?
This cannot be undone.
diff --git a/OpenOversight/app/templates/image.html b/OpenOversight/app/templates/image.html
index 9e8d54ea1..64f777f2a 100644
--- a/OpenOversight/app/templates/image.html
+++ b/OpenOversight/app/templates/image.html
@@ -73,6 +73,7 @@ Classification
Classify Admin only
diff --git a/OpenOversight/app/templates/link_delete.html b/OpenOversight/app/templates/link_delete.html
index 4330ae4dd..c516d7fb7 100644
--- a/OpenOversight/app/templates/link_delete.html
+++ b/OpenOversight/app/templates/link_delete.html
@@ -25,6 +25,7 @@
Are you sure you want to delete this link?
This cannot be undone.
diff --git a/OpenOversight/app/templates/list_officer.html b/OpenOversight/app/templates/list_officer.html
index f362e62aa..5d11e1a28 100644
--- a/OpenOversight/app/templates/list_officer.html
+++ b/OpenOversight/app/templates/list_officer.html
@@ -67,10 +67,10 @@ Gender
diff --git a/OpenOversight/app/templates/note_delete.html b/OpenOversight/app/templates/note_delete.html
index 15bc4d8ba..a4d0d85b3 100644
--- a/OpenOversight/app/templates/note_delete.html
+++ b/OpenOversight/app/templates/note_delete.html
@@ -15,6 +15,7 @@
Are you sure you want to delete this note?
This cannot be undone.
diff --git a/OpenOversight/app/templates/profile.html b/OpenOversight/app/templates/profile.html
index 1235f9f7f..f85427f4d 100644
--- a/OpenOversight/app/templates/profile.html
+++ b/OpenOversight/app/templates/profile.html
@@ -37,14 +37,10 @@
Account Status
{% endif %}
{% if current_user.is_administrator and user.is_administrator == False %}
-
Toggle (Disable/Enable) User Admin only
-
-
-
+
+
{% endif %}
{% if current_user.is_administrator %}
diff --git a/OpenOversight/app/templates/sort.html b/OpenOversight/app/templates/sort.html
index 63b2fcccb..df43826a5 100644
--- a/OpenOversight/app/templates/sort.html
+++ b/OpenOversight/app/templates/sort.html
@@ -38,6 +38,7 @@
Do you see uniformed law enforcement officers in the photo?