Merge branch 'develop' into 171/s3-local

.github/workflows/main.yml

@@ -0,0 +1,48 @@
+name: CI
+
+# Controls when the action will run. 
+on:
+  # Triggers the workflow on push or pull request events but only for the develop branch
+  push:
+    branches: 
+    - develop
+    - main
+  pull_request:
+    branches:
+    - develop
+    - main
+
+  # Allows you to run this workflow manually from the Actions tab
+  workflow_dispatch:
+
+# A workflow run is made up of one or more jobs that can run sequentially or in parallel
+jobs:
+  # This workflow contains a single job called "build"
+  build:
+    # The type of runner that the job will run on
+    runs-on: ubuntu-latest
+    env:
+      FLASK_APP: OpenOversight.app
+    strategy:
+      matrix:
+        python-version: [3.5, 3.6, 3.7, 3.8]
+
+    # Steps represent a sequence of tasks that will be executed as part of the job
+    steps:
+      # Checks-out your repository under $GITHUB_WORKSPACE, so your job can access it
+      - uses: actions/checkout@v2
+      - name: Set up Python ${{ matrix.python-version }}
+        uses: actions/setup-python@v2
+        with:
+          python-version: ${{ matrix.python-version }}
+      - name: Install dependencies
+        run: |
+          python -m pip install --upgrade pip
+          pip install -r requirements.txt -r dev-requirements.txt
+
+      # Run flake8
+      - name: Run flake8
+        run: flake8 --ignore=E501,E722,W504,W503
+      # Runs tests
+      - name: Run tests
+        run: FLASK_ENV=testing pytest --doctest-modules -n 4 --dist=loadfile -v OpenOversight/tests/ OpenOversight/app;

CONTRIB.md

@@ -2,12 +2,24 @@
 
 First, thanks for being interested in helping us out! If you find an issue you're interested in, feel free to make a comment about how you're thinking of approaching implementing it in the issue and we can give you feedback.  Please also read our [code of conduct](/CODE_OF_CONDUCT.md) before getting started.
 
-## Submitting a PR
+## Submitting a Pull Request (PR)
 
-When you come to implement your new feature, you should branch off `develop` and add commits to implement your feature. If your git history is not so clean, please do rewrite before you submit your PR - if you're not sure if you need to do this, go ahead and submit and we can let you know when you submit.
+When you come to implement your new feature, clone the repository and then create a branch off `develop` locally and add commits to implement your feature.
+
+If your git history is not so clean, please do rewrite before you submit your PR - if you're not sure if you need to do this, go ahead and submit and we can let you know when you submit.
+
+To submit your changes for review you have to fork the repository, push your new branch there and then create a Pull Request with `OpenOversight:develop` as the target.
 
 Use [PULL_REQUEST_TEMPLATE.md](/PULL_REQUEST_TEMPLATE.md) to create the description for your PR! (The template should populate automatically when you go to open the pull request.)
 
+### Recommended privacy settings
+Whenever you make a commit with `git` the name and email saved locally is stored with that commit and will become part of the public history of the project. This can be an unwanted, for example when using a work computer. We recommond changing the email-settings in the github account at https://github.com/settings/emails and selecting "Keep my email addresses private" as well as "Block command line pushes that expose my email". Also find your github-email address of the form `<id>+<username>@users.noreply.github.com` in that section. Then you can change the email and username stored with your commits by running the following commands
+```
+git config user.email "<your-github-email>"
+git config user.name "<your-github-username>"
+```
+This will make sure that all commits you make locally are associated with your github account and do not contain any additional identifying information. More detailed information on this topic can be found [here](https://docs.github.com/en/free-pro-team@latest/github/setting-up-and-managing-your-github-user-account/setting-your-commit-email-address).
+
 ### Linting / Style Checks
 
  `flake8` is a tool for automated linting and style checks. Be sure to run `flake8` and fix any errors before submitting a PR.
@@ -207,7 +219,19 @@ Next, in your terminal run `docker ps` to find the container id of the `openover
 
 ## Debugging OpenOversight - Use pdb with a test
 If you want to run an individual test in debug mode, use the below command.
-```yml
-`docker-compose run --rm web pytest --pdb -v tests/ -k <test_name_here>`
+```bash
+docker-compose run --rm web pytest --pdb -v tests/ -k <test_name_here>
 ```
+
+where `<test_name_here>` is the name of a single test function, such as `test_ac_cannot_add_new_officer_not_in_their_dept`
+
+Similarly, you can run all the tests in a file by specifying the file path:
+
+```bash
+docker-compose run --rm web pytest --pdb -v path/to/test/file 
+```
+
+where `path/to/test/file` is the relative file path, minus the initial `OpenOversight`, such as 
+`tests/routes/test_officer_and_department.py`.
+
 Again, add `import pdb` to the file you want to debug, then write `pdb.set_trace()` wherever you want to drop a breakpoint.  Once the test is up and running in your terminal, you can debug it using pdb prompts.

DEPLOY.md

@@ -38,14 +38,20 @@ You'll need to create an AWS account, if you don't already have one. Then, you'l
 For the officer identification UI to work, you'll need to create a CORS policy for the S3 bucket used with OpenOversight. In the AWS UI, this is done by navigating to the listing of buckets, clicking on the name of your bucket, and choosing the Permissions tab, and then "CORS configuration". Since we're not doing anything fancier than making a web browser GET it, we can just use the default policy:
 
 ```
-<CORSConfiguration>
-	<CORSRule>
-		<AllowedOrigin>*</AllowedOrigin>
-		<AllowedMethod>GET</AllowedMethod>
-		<MaxAgeSeconds>3000</MaxAgeSeconds>
-		<AllowedHeader>Authorization</AllowedHeader>
-	</CORSRule>
-</CORSConfiguration>
+[
+    {
+        "AllowedOrigins": [
+            "*"
+        ],
+        "AllowedMethods": [
+            "GET"
+        ],
+        "MaxAgeSeconds": 3000,
+        "AllowedHeaders": [
+            "Authorizations"
+        ]
+    }
+]
 ```
 
 If you don't click "Save" on that policy, however, the policy will not actually be applied.

OpenOversight/app/__init__.py

@@ -11,6 +11,7 @@
 from flask_mail import Mail
 from flask_migrate import Migrate
 from flask_sitemap import Sitemap
+from flask_wtf.csrf import CSRFProtect
 
 from .config import config
 
@@ -26,6 +27,7 @@
                   default_limits=["100 per minute", "5 per second"])
 
 sitemap = Sitemap()
+csrf = CSRFProtect()
 
 
 def create_app(config_name='default'):
@@ -40,6 +42,7 @@ def create_app(config_name='default'):
     login_manager.init_app(app)
     limiter.init_app(app)
     sitemap.init_app(app)
+    csrf.init_app(app)
 
     from .main import main as main_blueprint  # noqa
     app.register_blueprint(main_blueprint)

OpenOversight/app/auth/forms.py

@@ -88,7 +88,11 @@ class EditUserForm(Form):
     ac_department = QuerySelectField('Department', validators=[Optional()],
                                      query_factory=dept_choices, get_label='name', allow_blank=True)
     is_administrator = BooleanField('Is administrator?', false_values={'False', 'false', ''})
-    submit = SubmitField(label='Update')
+    is_disabled = BooleanField('Disabled?', false_values={'False', 'false', ''})
+    approved = BooleanField('Approved?', false_values={'False', 'false', ''})
+    submit = SubmitField(label='Update', false_values={'False', 'false', ''})
+    resend = SubmitField(label='Resend', false_values={'False', 'false', ''})
+    delete = SubmitField(label='Delete', false_values={'False', 'false', ''})
 
     def validate(self):
         success = super(EditUserForm, self).validate()

OpenOversight/app/auth/views.py

@@ -1,7 +1,4 @@
-from future.utils import iteritems
-
 from flask import render_template, redirect, request, url_for, flash, current_app
-from flask.views import MethodView
 from flask_login import login_user, logout_user, login_required, \
     current_user
 from . import auth
@@ -101,7 +98,7 @@ def register():
     return render_template('auth/register.html', form=form, jsloads=jsloads)
 
 
-@auth.route('/confirm/<token>')
+@auth.route('/confirm/<token>', methods=['GET'])
 @login_required
 def confirm(token):
     if current_user.confirmed:
@@ -237,144 +234,76 @@ def change_dept():
     return render_template('auth/change_dept_pref.html', form=form)
 
 
-class UserAPI(MethodView):
-    decorators = [admin_required]
-
-    def get(self, user_id):
-        # isolate the last part of the url
-        end_of_url = request.url.split('/')[-1].split('?')[0]
+@auth.route('/users/', methods=['GET'])
+@admin_required
+def get_users():
+    if request.args.get('page'):
+        page = int(request.args.get('page'))
+    else:
+        page = 1
+    USERS_PER_PAGE = int(current_app.config['USERS_PER_PAGE'])
+    users = User.query.order_by(User.username) \
+        .paginate(page, USERS_PER_PAGE, False)
 
-        if user_id is None:
-            if request.args.get('page'):
-                page = int(request.args.get('page'))
-            else:
-                page = 1
-            USERS_PER_PAGE = int(current_app.config['USERS_PER_PAGE'])
-            users = User.query.order_by(User.email) \
-                .paginate(page, USERS_PER_PAGE, False)
+    return render_template('auth/users.html', objects=users)
 
-            return render_template('auth/users.html', objects=users)
-        else:
-            user = User.query.get(user_id)
-            if not user:
-                return render_template('403.html'), 403
-
-            actions = ['delete', 'enable', 'disable', 'resend', 'approve']
-            if end_of_url in actions:
-                action = getattr(self, end_of_url, None)
-                return action(user)
-            else:
-                form = EditUserForm(
-                    email=user.email,
-                    is_area_coordinator=user.is_area_coordinator,
-                    ac_department=user.ac_department,
-                    is_administrator=user.is_administrator)
-                return render_template('auth/user.html', user=user, form=form)
 
-    def post(self, user_id):
-        # isolate the last part of the url
-        end_of_url = request.url.split('/')[-1].split('?')[0]
+@auth.route('/users/<int:user_id>', methods=['GET', 'POST'])
+@admin_required
+def edit_user(user_id):
+    user = User.query.get(user_id)
+    if not user:
+        return render_template('404.html'), 404
 
-        user = User.query.get(user_id)
+    if request.method == 'GET':
+        form = EditUserForm(obj=user)
+        return render_template('auth/user.html', user=user, form=form)
+    elif request.method == 'POST':
         form = EditUserForm()
+        if form.delete.data:
+            # forward to confirm delete
+            return redirect(url_for('auth.delete_user', user_id=user.id))
+        elif form.resend.data:
+            return admin_resend_confirmation(user)
+        elif form.submit.data:
+            if form.validate_on_submit():
+                # prevent user from removing own admin rights (or disabling account)
+                if user.id == current_user.id:
+                    flash('You cannot edit your own account!')
+                    form = EditUserForm(obj=user)
+                    return render_template('auth/user.html', user=user, form=form)
+                form.populate_obj(user)
+                db.session.add(user)
+                db.session.commit()
+                flash('{} has been updated!'.format(user.username))
+                return redirect(url_for('auth.edit_user', user_id=user.id))
+            else:
+                flash('Invalid entry')
+                return render_template('auth/user.html', user=user, form=form)
 
-        if user and end_of_url and end_of_url == 'delete':
-            return self.delete(user)
-        elif user and form.validate_on_submit():
-            for field, data in iteritems(form.data):
-                setattr(user, field, data)
-
-            db.session.add(user)
-            db.session.commit()
-            flash('{} has been updated!'.format(user.username))
-            return redirect(url_for('auth.user_api'))
-        elif not form.validate_on_submit():
-            flash('Invalid entry')
-            return render_template('auth/user.html', user=user, form=form)
-        else:
-            return render_template('403.html'), 403
-
-    def delete(self, user):
-        if request.method == 'POST':
-            username = user.username
-            db.session.delete(user)
-            db.session.commit()
-            flash('User {} has been deleted!'.format(username))
-            return redirect(url_for('auth.user_api'))
-
-        return render_template('auth/user_delete.html', user=user)
 
-    def enable(self, user):
-        if not user.is_disabled:
-            flash('User {} is already enabled.'.format(user.username))
-        else:
-            user.is_disabled = False
-            db.session.add(user)
-            db.session.commit()
-            flash('User {} has been enabled!'.format(user.username))
-        return redirect(url_for('auth.user_api'))
+@auth.route('/users/<int:user_id>/delete', methods=['GET', 'POST'])
+@admin_required
+def delete_user(user_id):
+    user = User.query.get(user_id)
+    if not user or user.is_administrator:
+        return render_template('403.html'), 403
+    if request.method == 'POST':
+        username = user.username
+        db.session.delete(user)
+        db.session.commit()
+        flash('User {} has been deleted!'.format(username))
+        return redirect(url_for('auth.get_users'))
 
-    def disable(self, user):
-        if user.is_disabled:
-            flash('User {} is already disabled.'.format(user.username))
-        else:
-            user.is_disabled = True
-            db.session.add(user)
-            db.session.commit()
-            flash('User {} has been disabled!'.format(user.username))
-        return redirect(url_for('auth.user_api'))
+    return render_template('auth/user_delete.html', user=user)
 
-    def resend(self, user):
-        if user.confirmed:
-            flash('User {} is already confirmed.'.format(user.username))
-        else:
-            token = user.generate_confirmation_token()
-            send_email(user.email, 'Confirm Your Account',
-                       'auth/email/confirm', user=user, token=token)
-            flash('A new confirmation email has been sent to {}.'.format(user.email))
-        return redirect(url_for('auth.user_api'))
 
-    def approve(self, user):
-        if user.approved:
-            flash('User {} is already approved.'.format(user.username))
-        else:
-            user.approved = True
-            db.session.add(user)
-            db.session.commit()
-            token = user.generate_confirmation_token()
-            send_email(user.email, 'Confirm Your Account',
-                       'auth/email/confirm', user=user, token=token)
-            flash('User {} has been approved!'.format(user.username))
-        return redirect(url_for('auth.user_api'))
-
-
-user_view = UserAPI.as_view('user_api')
-auth.add_url_rule(
-    '/users/',
-    defaults={'user_id': None},
-    view_func=user_view,
-    methods=['GET'])
-auth.add_url_rule(
-    '/users/<int:user_id>',
-    view_func=user_view,
-    methods=['GET', 'POST'])
-auth.add_url_rule(
-    '/users/<int:user_id>/delete',
-    view_func=user_view,
-    methods=['GET', 'POST'])
-auth.add_url_rule(
-    '/users/<int:user_id>/enable',
-    view_func=user_view,
-    methods=['GET'])
-auth.add_url_rule(
-    '/users/<int:user_id>/disable',
-    view_func=user_view,
-    methods=['GET'])
-auth.add_url_rule(
-    '/users/<int:user_id>/resend',
-    view_func=user_view,
-    methods=['GET'])
-auth.add_url_rule(
-    '/users/<int:user_id>/approve',
-    view_func=user_view,
-    methods=['GET'])
+def admin_resend_confirmation(user):
+    if user.confirmed:
+        flash('User {} is already confirmed.'.format(user.username))
+    else:
+        token = user.generate_confirmation_token()
+        send_email(user.email, 'Confirm Your Account',
+                   'auth/email/confirm', user=user, token=token)
+        flash('A new confirmation email has been sent to {}.'.format(user.email))
+    return redirect(url_for('auth.get_users'))