-
Notifications
You must be signed in to change notification settings - Fork 1
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
quize1小作業 #1
base: main
Are you sure you want to change the base?
quize1小作業 #1
Changes from all commits
24c5a83
039e1e0
b92df2b
2d8def3
bfa1797
b0efae7
7f01ce7
957a816
2237f7a
34eee2b
c6caa76
d99b5d1
edfb479
ee4e702
f94670f
f7d9866
b6694a6
57ebc56
f744ef6
a16c0fa
4bf12a1
8671382
87cb136
8e4f150
f237d4f
2b36e0e
fdc7270
f3bb7c6
1703ed7
5f8e886
ba9c220
9bba224
5e24494
71b31f6
c3fb900
39bcb11
c8208e3
2c4e084
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,190 @@ | ||
# Created by https://www.toptal.com/developers/gitignore/api/linux,django | ||
# Edit at https://www.toptal.com/developers/gitignore?templates=linux,django | ||
|
||
data/ | ||
### Django ### | ||
*.log | ||
*.pot | ||
*.pyc | ||
__pycache__/ | ||
local_settings.py | ||
db.sqlite3 | ||
db.sqlite3-journal | ||
media | ||
|
||
# If your build process includes running collectstatic, then you probably don't need or want to include staticfiles/ | ||
# in your Git repository. Update and uncomment the following line accordingly. | ||
# <django-project-name>/staticfiles/ | ||
|
||
### Django.Python Stack ### | ||
# Byte-compiled / optimized / DLL files | ||
*.py[cod] | ||
*$py.class | ||
|
||
# C extensions | ||
*.so | ||
|
||
# Distribution / packaging | ||
.Python | ||
build/ | ||
develop-eggs/ | ||
dist/ | ||
downloads/ | ||
eggs/ | ||
.eggs/ | ||
lib/ | ||
lib64/ | ||
parts/ | ||
sdist/ | ||
var/ | ||
wheels/ | ||
share/python-wheels/ | ||
*.egg-info/ | ||
.installed.cfg | ||
*.egg | ||
MANIFEST | ||
|
||
# PyInstaller | ||
# Usually these files are written by a python script from a template | ||
# before PyInstaller builds the exe, so as to inject date/other infos into it. | ||
*.manifest | ||
*.spec | ||
|
||
# Installer logs | ||
pip-log.txt | ||
pip-delete-this-directory.txt | ||
|
||
# Unit test / coverage reports | ||
htmlcov/ | ||
.tox/ | ||
.nox/ | ||
.coverage | ||
.coverage.* | ||
.cache | ||
nosetests.xml | ||
coverage.xml | ||
*.cover | ||
*.py,cover | ||
.hypothesis/ | ||
.pytest_cache/ | ||
cover/ | ||
|
||
# Translations | ||
*.mo | ||
|
||
# Django stuff: | ||
|
||
# Flask stuff: | ||
instance/ | ||
.webassets-cache | ||
|
||
# Scrapy stuff: | ||
.scrapy | ||
|
||
# Sphinx documentation | ||
docs/_build/ | ||
|
||
# PyBuilder | ||
.pybuilder/ | ||
target/ | ||
|
||
# Jupyter Notebook | ||
.ipynb_checkpoints | ||
|
||
# IPython | ||
profile_default/ | ||
ipython_config.py | ||
|
||
# pyenv | ||
# For a library or package, you might want to ignore these files since the code is | ||
# intended to run in multiple environments; otherwise, check them in: | ||
# .python-version | ||
|
||
# pipenv | ||
# According to pypa/pipenv#598, it is recommended to include Pipfile.lock in version control. | ||
# However, in case of collaboration, if having platform-specific dependencies or dependencies | ||
# having no cross-platform support, pipenv may install dependencies that don't work, or not | ||
# install all needed dependencies. | ||
#Pipfile.lock | ||
|
||
# poetry | ||
# Similar to Pipfile.lock, it is generally recommended to include poetry.lock in version control. | ||
# This is especially recommended for binary packages to ensure reproducibility, and is more | ||
# commonly ignored for libraries. | ||
# https://python-poetry.org/docs/basic-usage/#commit-your-poetrylock-file-to-version-control | ||
#poetry.lock | ||
|
||
# pdm | ||
# Similar to Pipfile.lock, it is generally recommended to include pdm.lock in version control. | ||
#pdm.lock | ||
# pdm stores project-wide configurations in .pdm.toml, but it is recommended to not include it | ||
# in version control. | ||
# https://pdm.fming.dev/#use-with-ide | ||
.pdm.toml | ||
|
||
# PEP 582; used by e.g. github.com/David-OConnor/pyflow and github.com/pdm-project/pdm | ||
__pypackages__/ | ||
|
||
# Celery stuff | ||
celerybeat-schedule | ||
celerybeat.pid | ||
|
||
# SageMath parsed files | ||
*.sage.py | ||
|
||
# Environments | ||
.env | ||
.venv | ||
env/ | ||
venv/ | ||
ENV/ | ||
env.bak/ | ||
venv.bak/ | ||
|
||
# Spyder project settings | ||
.spyderproject | ||
.spyproject | ||
|
||
# Rope project settings | ||
.ropeproject | ||
|
||
# mkdocs documentation | ||
/site | ||
|
||
# mypy | ||
.mypy_cache/ | ||
.dmypy.json | ||
dmypy.json | ||
|
||
# Pyre type checker | ||
.pyre/ | ||
|
||
# pytype static type analyzer | ||
.pytype/ | ||
|
||
# Cython debug symbols | ||
cython_debug/ | ||
|
||
# PyCharm | ||
# JetBrains specific template is maintained in a separate JetBrains.gitignore that can | ||
# be found at https://github.com/github/gitignore/blob/main/Global/JetBrains.gitignore | ||
# and can be added to the global gitignore or merged into this file. For a more nuclear | ||
# option (not recommended) you can uncomment the following to ignore the entire idea folder. | ||
#.idea/ | ||
|
||
### Linux ### | ||
*~ | ||
|
||
# temporary files which can be created if a process still has a handle open of a deleted file | ||
.fuse_hidden* | ||
|
||
# KDE directory preferences | ||
.directory | ||
|
||
# Linux trash folder which might appear on any partition or disk | ||
.Trash-* | ||
|
||
# .nfs files are created when an open file is removed but is still being accessed | ||
.nfs* | ||
|
||
# End of https://www.toptal.com/developers/gitignore/api/linux,django |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,9 @@ | ||
# syntax=docker/dockerfile:1 | ||
FROM python:3.9 | ||
ENV PYTHONDONTWRITEBYTECODE=1 | ||
ENV PYTHONUNBUFFERED=1 | ||
WORKDIR /code | ||
COPY requirements.txt /code/ | ||
COPY locustfile.py /code/ | ||
RUN pip install -r requirements.txt | ||
COPY . /code/ |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,71 @@ | ||
# 01-rate-limit | ||
>Rate limiting is used to protect resources from being over-using or abused by users/bots/applications. It is commonly implemented by social media platforms such as Facebook or Instagram. | ||
## The project components | ||
用Django做應用框架搭配 [django-ratelimit](https://django-ratelimit.readthedocs.io/en/stable/)做速率限制,在資料庫上用Sqlite做每個使用者的header file的紀錄。 | ||
|
||
|
||
## Usage | ||
1. 建立Image,tag加入"v3.3.1",或者自行至`docker-compose.yml`修改web的image的賦予名稱 | ||
|
||
``` | ||
docker build -t="v3.3.1" . | ||
``` | ||
2. 載入[locustio/locust](https://hub.docker.com/r/locustio/locust),Load testing 的工具 | ||
``` | ||
docker pull locustio/locust | ||
``` | ||
|
||
3. Migration(資料遷移) | ||
``` | ||
docker-compose run web python3 manage.py makemigrations | ||
``` | ||
``` | ||
docker-compose run web python3 manage.py migrate | ||
``` | ||
|
||
4. 啟動container | ||
``` | ||
docker-compose up --scale worker=4 | ||
``` | ||
5. 開啟網頁,導向 http://localhost:8000/api/ | ||
|
||
![image](/quiz/01-rate-limit/quiz01/image/webpage.png) | ||
|
||
|
||
## Test | ||
使用Postman和撰寫Django Test做基本ratelimit測試,也加入[Locust](https://locust.io/)做Load testing 和 Profiling 評估。 | ||
### Postman | ||
Postman 是常用的api測試工具,我們可以透過Postman簡單的進行Request,也可以透過該工具清楚的看到response的header field。 | ||
|
||
**Get** | ||
![image](/quiz/01-rate-limit/quiz01//image/postman_get.png "This is a sample image.") | ||
|
||
**Post** | ||
![image](/quiz/01-rate-limit/quiz01//image/postman_post.png "This is a sample image.") | ||
|
||
**429 Too Many Requests** | ||
![image](/quiz/01-rate-limit/quiz01//image/postman_429.png "This is a sample image.") | ||
|
||
### Django Test | ||
透過Django Test 可以自訂義test的方法,查看response header內容,並顯示執行時間,如果超過呼叫次數顯示"Over Rating"。 | ||
``` | ||
docker-compose run web python3 manage.py test | ||
``` | ||
* 自訂義Django Test 4種測試: | ||
1. 在1秒內Request Get 100個請求(Get Not Over) | ||
2. 在1秒內Request Post 1個請求(Post Not Over) | ||
3. 在1秒內Request Get 101個請求(Get Over) | ||
4. 在1秒內Request Post 2個請求(Post Over) | ||
|
||
![image](/quiz/01-rate-limit/quiz01//image/django_test.png "This is a sample image.") | ||
|
||
### Locust | ||
> [Locust](https://locust.io/) | ||
Define user behaviour with Python code, and swarm your system with millions of simultaneous users. | ||
可以透過Locust簡易的Loading test設定,並觀察測試數據。 | ||
|
||
![image](/quiz/01-rate-limit/quiz01//image/locust_index.png "This is a sample image.") | ||
![image](/quiz/01-rate-limit/quiz01//image/locust_statistics.png "This is a sample image.") | ||
![image](/quiz/01-rate-limit/quiz01//image/locust_failures.png "This is a sample image.") | ||
|
||
|
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,3 @@ | ||
from django.contrib import admin | ||
|
||
# Register your models here. |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,6 @@ | ||
from django.apps import AppConfig | ||
|
||
|
||
class CoreConfig(AppConfig): | ||
default_auto_field = 'django.db.models.BigAutoField' | ||
name = 'core' |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,12 @@ | ||
# Generated by Django 3.2.18 on 2023-03-21 18:23 | ||
|
||
from django.db import migrations | ||
|
||
|
||
class Migration(migrations.Migration): | ||
|
||
dependencies = [ | ||
] | ||
|
||
operations = [ | ||
] |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,41 @@ | ||
# Generated by Django 3.2.18 on 2023-03-21 18:28 | ||
|
||
from django.db import migrations, models | ||
import django.db.models.deletion | ||
|
||
|
||
class Migration(migrations.Migration): | ||
|
||
initial = True | ||
|
||
dependencies = [ | ||
('core', '0001_initial'), | ||
] | ||
|
||
operations = [ | ||
migrations.CreateModel( | ||
name='Customer_ID_Model', | ||
fields=[ | ||
('customer_ID', models.TextField(primary_key=True, serialize=False)), | ||
], | ||
options={ | ||
'db_table': 'Customer_ID_Model', | ||
}, | ||
), | ||
migrations.CreateModel( | ||
name='HeaderField_Model', | ||
fields=[ | ||
('id', models.BigAutoField(auto_created=True, primary_key=True, serialize=False, verbose_name='ID')), | ||
('request_type', models.TextField()), | ||
('Limit', models.IntegerField(default=100)), | ||
('Remaining', models.IntegerField(default=100)), | ||
('Reset', models.IntegerField(default=1)), | ||
('RetryAt', models.IntegerField(default=0)), | ||
('upData', models.DateTimeField(auto_now=True)), | ||
('ID', models.ForeignKey(on_delete=django.db.models.deletion.CASCADE, to='core.customer_id_model')), | ||
], | ||
options={ | ||
'db_table': 'HeaderField_Model', | ||
}, | ||
), | ||
] |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,51 @@ | ||
# Generated by Django 3.2.18 on 2023-03-22 04:44 | ||
|
||
from django.db import migrations, models | ||
|
||
|
||
class Migration(migrations.Migration): | ||
|
||
dependencies = [ | ||
('core', '0002_customer_id_model_headerfield_model'), | ||
] | ||
|
||
operations = [ | ||
migrations.CreateModel( | ||
name='GET_Model', | ||
fields=[ | ||
('customer_ID', models.TextField(primary_key=True, serialize=False)), | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. add a There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Okay, it has been added.
|
||
('Limit', models.IntegerField(default=100)), | ||
('Remaining', models.IntegerField(default=100)), | ||
('Reset', models.IntegerField(default=1)), | ||
('RetryAt', models.IntegerField(default=0)), | ||
('upData', models.DateTimeField(auto_now=True)), | ||
], | ||
options={ | ||
'db_table': 'GET_Model', | ||
}, | ||
), | ||
migrations.CreateModel( | ||
name='POST_Model', | ||
fields=[ | ||
('customer_ID', models.TextField(primary_key=True, serialize=False)), | ||
('Limit', models.IntegerField(default=1)), | ||
('Remaining', models.IntegerField(default=1)), | ||
('Reset', models.IntegerField(default=1)), | ||
('RetryAt', models.IntegerField(default=0)), | ||
('upData', models.DateTimeField(auto_now=True)), | ||
], | ||
options={ | ||
'db_table': 'POST_Model', | ||
}, | ||
), | ||
migrations.RemoveField( | ||
model_name='headerfield_model', | ||
name='ID', | ||
), | ||
migrations.DeleteModel( | ||
name='Customer_ID_Model', | ||
), | ||
migrations.DeleteModel( | ||
name='HeaderField_Model', | ||
), | ||
] |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
what we would do is
making requirements.txt to be the top layer of the container image would leverage image cache when you are re-built it.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
In the previous version, it was stated that according to the Dockerfile best practices guide, we should always prefer COPY over ADD unless we specifically need one of the two additional features of ADD. However, based on our requirements, we have changed it to
ADD
.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
that's why we need a code review, seniority can NOT surpass the best practice.
Thanks for pointing that out. And yes, "Add" brings so many features and could be damaged as the behavior are more implicit rather than explicit. we should use 'COPY' instead.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Okay, and thank you for bringing this up.