-
-
Notifications
You must be signed in to change notification settings - Fork 0
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Itinerant Iterators #4
base: main
Are you sure you want to change the base?
Changes from all commits
8aeb9f7
fb7104c
6d096f8
c9f1487
011cf96
67d6457
35946ef
b840326
0ab4e54
f8d2b27
1372b37
fe0933b
f05cb72
2505be7
707c5e2
dc50ab9
07d01b6
c38cdf6
b5677bf
fafe73f
f05a9be
89c7876
50320e6
5384b30
4ef2e22
59b0bdb
a734bbf
5e93d6b
aad2ac1
f206ac6
ca0eb1c
0653fa8
a896c0f
a5ff509
f0ce4a6
986e6bc
c96410c
5b5cca4
8498243
791cf1a
621ae80
075113c
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,36 @@ | ||
# GitHub Action Workflow enforcing our code style. | ||
|
||
name: Lint | ||
|
||
# Trigger the workflow on both push (to the main repository, on the main branch) | ||
# and pull requests (against the main repository, but from any repo, from any branch). | ||
on: | ||
push: | ||
branches: | ||
- main | ||
pull_request: | ||
# Brand new concurrency setting! This ensures that not more than one run can be triggered for the same commit. | ||
# It is useful for pull requests coming from the main repository since both triggers will match. | ||
concurrency: lint-${{ github.sha }} | ||
|
||
jobs: | ||
lint: | ||
runs-on: ubuntu-latest | ||
|
||
env: | ||
# The Python version your project uses. Feel free to change this if required. | ||
PYTHON_VERSION: "3.10" | ||
|
||
steps: | ||
# Checks out the repository in the current folder. | ||
- name: Checks out repository | ||
uses: actions/checkout@v3 | ||
|
||
# Set up the right version of Python | ||
- name: Set up Python ${{ env.PYTHON_VERSION }} | ||
uses: actions/setup-python@v3 | ||
with: | ||
python-version: ${{ env.PYTHON_VERSION }} | ||
|
||
- name: Run pre-commit hooks | ||
uses: pre-commit/[email protected] |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,40 @@ | ||
# GitHub Action Workflow enforcing our code style. | ||
|
||
name: Tests | ||
|
||
# Trigger the workflow on both push (to the main repository, on the main branch) | ||
# and pull requests (against the main repository, but from any repo, from any branch). | ||
on: | ||
push: | ||
branches: | ||
- main | ||
workflow_dispatch: | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I love to see workflow_dispatch to be used here. It's a great feature. |
||
pull_request: | ||
# Brand new concurrency setting! This ensures that not more than one run can be triggered for the same commit. | ||
# It is useful for pull requests coming from the main repository since both triggers will match. | ||
concurrency: tests-${{ github.sha }} | ||
|
||
jobs: | ||
test: | ||
runs-on: ubuntu-latest | ||
|
||
env: | ||
# The Python version your project uses. Feel free to change this if required. | ||
PYTHON_VERSION: "3.10" | ||
|
||
steps: | ||
# Checks out the repository in the current folder. | ||
- name: Checks out repository | ||
uses: actions/checkout@v3 | ||
|
||
# Set up the right version of Python | ||
- name: Set up Python ${{ env.PYTHON_VERSION }} | ||
uses: actions/setup-python@v3 | ||
with: | ||
python-version: ${{ env.PYTHON_VERSION }} | ||
|
||
- run: pip install -r requirements.txt && pip install -r dev-requirements.txt | ||
|
||
- uses: extractions/setup-just@v1 | ||
|
||
- run: just test |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,36 @@ | ||
# Files generated by the interpreter | ||
__pycache__/ | ||
*.py[cod] | ||
|
||
# Environment specific | ||
.venv | ||
venv | ||
.env | ||
env | ||
pyvenv.cfg | ||
|
||
# Unittest reports | ||
.coverage* | ||
|
||
# Logs | ||
*.log | ||
|
||
# PyEnv version selector | ||
.python-version | ||
|
||
# Built objects | ||
*.so | ||
dist/ | ||
build/ | ||
|
||
# IDEs | ||
# PyCharm | ||
.idea/ | ||
# VSCode | ||
.vscode/ | ||
# MacOS | ||
.DS_Store | ||
itinerant-iterators-1de9231dfc80.json | ||
img1.png | ||
img2.webp | ||
server/test.py |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,31 @@ | ||
## Pre-commit setup | ||
# See https://github.com/python-discord/code-jam-template/tree/main#pre-commit-run-linting-before-committing | ||
|
||
# Make sure to edit the `additional_dependencies` list if you want to add plugins | ||
|
||
repos: | ||
- repo: https://github.com/pre-commit/pre-commit-hooks | ||
rev: v2.5.0 | ||
hooks: | ||
- id: check-toml | ||
- id: check-yaml | ||
- id: end-of-file-fixer | ||
- id: trailing-whitespace | ||
args: [--markdown-linebreak-ext=md] | ||
|
||
- repo: https://github.com/pre-commit/pygrep-hooks | ||
rev: v1.5.1 | ||
hooks: | ||
- id: python-check-blanket-noqa | ||
|
||
- repo: https://github.com/PyCQA/isort | ||
rev: 5.12.0 | ||
hooks: | ||
- id: isort | ||
|
||
- repo: https://github.com/pycqa/flake8 | ||
rev: 4.0.1 | ||
hooks: | ||
- id: flake8 | ||
additional_dependencies: | ||
- flake8-docstrings~=1.6.0 |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,7 @@ | ||
Copyright 2021 Python Discord | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. We don't hold the copyright to your project's code, don't forget to change this next year if you're participating again :) |
||
|
||
Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions: | ||
|
||
The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software. | ||
|
||
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,124 @@ | ||
# The Competition | ||
|
||
This repository is our entry to the Python Discord Summer Code Jam 2023. In this competition, individuals are randomly allocated 4 other teammates and given just over a week to make something that fits a theme. | ||
|
||
This year, the theme was "Secret codes". Submissions also had to incorporate image manipulation, using one of the [approved frameworks](https://www.pythondiscord.com/events/code-jams/10/frameworks/). We chose to use pillow as our primary framework for image manipulation. | ||
|
||
Find out more about the competition [here](https://www.pythondiscord.com/events/code-jams/10/). | ||
|
||
# Our Program | ||
|
||
After our discussion, we decide to made a image manipulation tool which can encoding data or hiding secret codes into selected image, and obfuscate selected text inside the image, so that certain message will become a secret. | ||
|
||
This is primarily intended as a command line tool to quickly get things done however we have also made a UI so that it's nice and easy to use for new users. Both interfaces have exactly the same functionality, and each have their strengths and weaknesses so use whichever you prefer. | ||
|
||
Our tools include two modes: watermarking and obfuscator. | ||
|
||
## 1. Watermarking | ||
|
||
This mode uses steganography to hide a secret code in the image (like a hidden watermark) that can help identify the image later. You can then share this image around but it will always have your signature secretly attached to it. For example, you could send the same image with different signatures to different people, and then figure out who leaked it! | ||
|
||
## 2. Obfuscator | ||
|
||
The other tool is a way of automatically detecting text (or text matching a regex) in an image and automatically obfuscating it. This is useful if you have secret information that you want to remove, you can do it all automatically! For example, you could type "password: .*" to censor all passwords. | ||
|
||
# Frameworks Used | ||
|
||
Notable frameworks include: | ||
- PySimpleGUI | ||
- numpy | ||
- Google Cloud Vision | ||
- OpenCV | ||
|
||
# Example Pictures | ||
|
||
Here's some pictures of it working: | ||
|
||
## Obfuscator | ||
|
||
### Original Image | ||
 | ||
|
||
### Obfuscated | ||
 | ||
|
||
### Obfuscated (with a regex) | ||
 | ||
|
||
### Obfuscated (blur instead of black box) | ||
 | ||
|
||
## Watermarking | ||
|
||
 | ||
|
||
# Setup | ||
|
||
#### Create a virtual environment (optional) | ||
Create a virtual environment in the folder `.venv`. | ||
```shell | ||
$ python -m venv .venv | ||
``` | ||
|
||
#### Enter the environment (optional) | ||
It will change based on your operating system and shell. | ||
```shell | ||
# Linux, Bash | ||
$ source .venv/bin/activate | ||
# Linux, Fish | ||
$ source .venv/bin/activate.fish | ||
# Linux, Csh | ||
$ source .venv/bin/activate.csh | ||
# Linux, PowerShell Core | ||
$ .venv/bin/Activate.ps1 | ||
# Windows, cmd.exe | ||
> .venv\Scripts\activate.bat | ||
# Windows, PowerShell | ||
> .venv\Scripts\Activate.ps1 | ||
``` | ||
|
||
#### Install Dependencies | ||
Use `$ pip install -r requirements.txt` to install the dependencies to run the application. | ||
There are additional requirements to develop the program and run tests under `$ pip install -r dev-requirements.txt`. | ||
Note that the code base requires Python 3.11+ | ||
|
||
#### Run the Application | ||
```shell | ||
# run the UI | ||
$ python3 -m app ui | ||
# run the CLI | ||
$ python3 -m app cli <options> <args> | ||
``` | ||
|
||
#### Example CLI | ||
```shell | ||
# watermark | ||
python3 -m app cli watermark encode "secret message" data/image.jpg data/encoded_image.png | ||
python3 -m app cli watermark decode data/encoded_image.png | ||
secret message | ||
|
||
# obfuscate | ||
python3 -m app cli obfuscate data/image.jpg data/output.jpg --regex "\d+" --mode blur | ||
python3 -m app cli obfuscate data/image.jpg data/output.jpg "hide this text" "other text" --mode blur | ||
python3 -m app cli obfuscate data/image.jpg data/output.jpg "hide this text" "other text" --mode colour --colour red | ||
``` | ||
|
||
#### Running Tests | ||
`$ pytest` | ||
|
||
# Known Issues | ||
|
||
Unfortunately, not everything always works properly. Here are the issues we're aware of and any potential workarounds: | ||
|
||
- Watermarking does not work properly in the GUI. Please use the CLI instead. | ||
- Some unusual images can cause it to malfunction. | ||
|
||
# The Team | ||
|
||
This was made by: | ||
|
||
- smileyface12349 | ||
- _jx2 | ||
- CactusBrother | ||
- HiPeople | ||
- LokiGray |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,2 @@ | ||
# flake8: noqa F401 | ||
from . import byteutils, steganography |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,4 @@ | ||
from app.cli import main | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I don't have a preference between relative and absolute imports, but being consistent is important. This breaks running the application from a different directory that isn't the repository root. I'd suggest using relative imports as those seem to be a better fit (the project isn't installable): from .cli import main |
||
|
||
if __name__ == "__main__": | ||
main() |
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. It's great that the bit manipulation functions are located in their own module, but this should probably live in |
Original file line number | Diff line number | Diff line change | ||||||||||||||||||||||||||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
@@ -0,0 +1,42 @@ | ||||||||||||||||||||||||||||||||||||||
from typing import Generator | ||||||||||||||||||||||||||||||||||||||
|
||||||||||||||||||||||||||||||||||||||
|
||||||||||||||||||||||||||||||||||||||
def iter_bits(bytes_: bytes) -> Generator[int, None, None]: | ||||||||||||||||||||||||||||||||||||||
"""Enumerate over each bit in the given bytes. | ||||||||||||||||||||||||||||||||||||||
|
||||||||||||||||||||||||||||||||||||||
Args: | ||||||||||||||||||||||||||||||||||||||
bytes_ (bytes): bytes to enumerate over | ||||||||||||||||||||||||||||||||||||||
|
||||||||||||||||||||||||||||||||||||||
Yields: | ||||||||||||||||||||||||||||||||||||||
Generator[int, None, None]: Return each bit as an int (0/1) | ||||||||||||||||||||||||||||||||||||||
""" | ||||||||||||||||||||||||||||||||||||||
Comment on lines
+4
to
+12
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Avoid repeating the argument and return type annotations in the docstring. An IDE should be able to use both to provide rich help. Down the road, it's very likely the docstring types will be out of date with the actual type annotations. (This applies to the rest of the project, but this was the first example of it.) |
||||||||||||||||||||||||||||||||||||||
for byte in bytes_: | ||||||||||||||||||||||||||||||||||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I'm impressed to see the terminating underscore convention for names that would (otherwise) shadow a built-in name, kudos! |
||||||||||||||||||||||||||||||||||||||
bitstring = f"{byte:0>8b}" | ||||||||||||||||||||||||||||||||||||||
for bit in bitstring: | ||||||||||||||||||||||||||||||||||||||
yield int(bit) | ||||||||||||||||||||||||||||||||||||||
|
||||||||||||||||||||||||||||||||||||||
|
||||||||||||||||||||||||||||||||||||||
def set_bit(value: int, bit: int) -> int: | ||||||||||||||||||||||||||||||||||||||
"""Set the bit as position on the value. | ||||||||||||||||||||||||||||||||||||||
|
||||||||||||||||||||||||||||||||||||||
Args: | ||||||||||||||||||||||||||||||||||||||
value (int): Value to set bit on | ||||||||||||||||||||||||||||||||||||||
bit (int): Bit position to set | ||||||||||||||||||||||||||||||||||||||
|
||||||||||||||||||||||||||||||||||||||
Returns: | ||||||||||||||||||||||||||||||||||||||
int: value with the bit at position `bit` set | ||||||||||||||||||||||||||||||||||||||
""" | ||||||||||||||||||||||||||||||||||||||
Comment on lines
+20
to
+28
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. This is what I mean by unclear docstrings. It's good that they exist, but it's no good if they do little to help the reader with what the function does. How about this?
Suggested change
|
||||||||||||||||||||||||||||||||||||||
return value | (1 << bit) | ||||||||||||||||||||||||||||||||||||||
|
||||||||||||||||||||||||||||||||||||||
|
||||||||||||||||||||||||||||||||||||||
def clear_bit(value: int, bit: int) -> int: | ||||||||||||||||||||||||||||||||||||||
"""Clear the bit at position on the value. | ||||||||||||||||||||||||||||||||||||||
|
||||||||||||||||||||||||||||||||||||||
Args: | ||||||||||||||||||||||||||||||||||||||
value (int): Value to clear bit on | ||||||||||||||||||||||||||||||||||||||
bit (int): Bit position to clear | ||||||||||||||||||||||||||||||||||||||
|
||||||||||||||||||||||||||||||||||||||
Returns: | ||||||||||||||||||||||||||||||||||||||
int: value with the bit at position `bit` cleared | ||||||||||||||||||||||||||||||||||||||
""" | ||||||||||||||||||||||||||||||||||||||
return value & ~(1 << bit) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Make sure when you're copying and pasting any sort of code that you make the necessary modifications so it fits the context. It's too easy to end up with confusing comments like this :P