Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Itinerant Iterators #4

Open
wants to merge 42 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
42 commits
Select commit Hold shift + click to select a range
8aeb9f7
Initial commit
smileyface12349 Aug 28, 2023
fb7104c
setup project structure
jj-style Sep 2, 2023
6d096f8
fix justfile
jj-style Sep 2, 2023
c9f1487
implement steganography encoding with LSB
jj-style Sep 2, 2023
011cf96
implement LSB encode/decode. Add Steg interface to allow more impleme…
jj-style Sep 2, 2023
67d6457
add tests and fix imports
jj-style Sep 3, 2023
35946ef
Replace opencv with PIL for image steganography.
jj-style Sep 4, 2023
b840326
Merge feature/11-unit-tests-cli
jj-style Sep 4, 2023
0ab4e54
Implement colour box obfuscation and unimplemented blur box.
jj-style Sep 5, 2023
f8d2b27
Update base GUI
cloki0610 Sep 7, 2023
1372b37
Fix lint failed error
cloki0610 Sep 7, 2023
fe0933b
Fix lint failed error
cloki0610 Sep 7, 2023
f05cb72
Fix Image size bug, Save btn error, rm grey rect
cloki0610 Sep 7, 2023
2505be7
Facilitated generator to choose pixels for LSB encoding
Sep 9, 2023
707c5e2
Fixed lint
Sep 9, 2023
dc50ab9
fix steganography encoding and fmt and lint
jj-style Sep 10, 2023
07d01b6
implement blur obfuscating
jj-style Sep 10, 2023
c38cdf6
Functions retrieves bounding boxes of an image based on text
HiPeople21 Sep 10, 2023
b5677bf
Fixing linting issues
HiPeople21 Sep 10, 2023
fafe73f
hook up UI to steganoraphy
jj-style Sep 10, 2023
f05a9be
Merge branch 'main' of github.com:smileyface12349/itinerant-iterators
jj-style Sep 10, 2023
89c7876
Works with regex now
HiPeople21 Sep 10, 2023
50320e6
update README
jj-style Sep 10, 2023
5384b30
Merge branch 'main' of github.com:smileyface12349/itinerant-iterators
jj-style Sep 10, 2023
4ef2e22
Implement watermark on cli
jj-style Sep 10, 2023
59b0bdb
Implement CLI commanf for obfuscation.
jj-style Sep 10, 2023
a734bbf
Set up server to handle requests to api
HiPeople21 Sep 10, 2023
5e93d6b
Fixed conflicts
HiPeople21 Sep 10, 2023
aad2ac1
Added quick note about python 3.11 required
Sep 10, 2023
f206ac6
UI fix
HiPeople21 Sep 10, 2023
ca0eb1c
Fix obfuscator
HiPeople21 Sep 10, 2023
0653fa8
UI accepts Regex now
HiPeople21 Sep 10, 2023
a896c0f
Force conversion to RGB mode
Sep 10, 2023
a5ff509
Lint, this is not the time...
Sep 10, 2023
f0ce4a6
Merge branch 'main' of https://github.com/smileyface12349/itinerant-i…
HiPeople21 Sep 10, 2023
986e6bc
Update README.md
smileyface12349 Sep 11, 2023
c96410c
Update README.md
cloki0610 Sep 13, 2023
5b5cca4
Update README.md
smileyface12349 Sep 14, 2023
8498243
Edited images on readme
smileyface12349 Sep 14, 2023
791cf1a
Update README.md
smileyface12349 Sep 14, 2023
621ae80
Censored sensitive info
smileyface12349 Sep 14, 2023
075113c
Add 'itinerant-iterators/' from commit '621ae80d7a6d6eb468dded5c00958…
janine9vn Sep 28, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
36 changes: 36 additions & 0 deletions itinerant-iterators/.github/workflows/lint.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,36 @@
# GitHub Action Workflow enforcing our code style.

name: Lint

# Trigger the workflow on both push (to the main repository, on the main branch)
# and pull requests (against the main repository, but from any repo, from any branch).
on:
push:
branches:
- main
pull_request:
# Brand new concurrency setting! This ensures that not more than one run can be triggered for the same commit.
# It is useful for pull requests coming from the main repository since both triggers will match.
concurrency: lint-${{ github.sha }}

jobs:
lint:
runs-on: ubuntu-latest

env:
# The Python version your project uses. Feel free to change this if required.
PYTHON_VERSION: "3.10"

steps:
# Checks out the repository in the current folder.
- name: Checks out repository
uses: actions/checkout@v3

# Set up the right version of Python
- name: Set up Python ${{ env.PYTHON_VERSION }}
uses: actions/setup-python@v3
with:
python-version: ${{ env.PYTHON_VERSION }}

- name: Run pre-commit hooks
uses: pre-commit/[email protected]
40 changes: 40 additions & 0 deletions itinerant-iterators/.github/workflows/tests.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,40 @@
# GitHub Action Workflow enforcing our code style.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Make sure when you're copying and pasting any sort of code that you make the necessary modifications so it fits the context. It's too easy to end up with confusing comments like this :P


name: Tests

# Trigger the workflow on both push (to the main repository, on the main branch)
# and pull requests (against the main repository, but from any repo, from any branch).
on:
push:
branches:
- main
workflow_dispatch:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I love to see workflow_dispatch to be used here. It's a great feature.

pull_request:
# Brand new concurrency setting! This ensures that not more than one run can be triggered for the same commit.
# It is useful for pull requests coming from the main repository since both triggers will match.
concurrency: tests-${{ github.sha }}

jobs:
test:
runs-on: ubuntu-latest

env:
# The Python version your project uses. Feel free to change this if required.
PYTHON_VERSION: "3.10"

steps:
# Checks out the repository in the current folder.
- name: Checks out repository
uses: actions/checkout@v3

# Set up the right version of Python
- name: Set up Python ${{ env.PYTHON_VERSION }}
uses: actions/setup-python@v3
with:
python-version: ${{ env.PYTHON_VERSION }}

- run: pip install -r requirements.txt && pip install -r dev-requirements.txt

- uses: extractions/setup-just@v1

- run: just test
36 changes: 36 additions & 0 deletions itinerant-iterators/.gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,36 @@
# Files generated by the interpreter
__pycache__/
*.py[cod]

# Environment specific
.venv
venv
.env
env
pyvenv.cfg

# Unittest reports
.coverage*

# Logs
*.log

# PyEnv version selector
.python-version

# Built objects
*.so
dist/
build/

# IDEs
# PyCharm
.idea/
# VSCode
.vscode/
# MacOS
.DS_Store
itinerant-iterators-1de9231dfc80.json
img1.png
img2.webp
server/test.py
31 changes: 31 additions & 0 deletions itinerant-iterators/.pre-commit-config.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
## Pre-commit setup
# See https://github.com/python-discord/code-jam-template/tree/main#pre-commit-run-linting-before-committing

# Make sure to edit the `additional_dependencies` list if you want to add plugins

repos:
- repo: https://github.com/pre-commit/pre-commit-hooks
rev: v2.5.0
hooks:
- id: check-toml
- id: check-yaml
- id: end-of-file-fixer
- id: trailing-whitespace
args: [--markdown-linebreak-ext=md]

- repo: https://github.com/pre-commit/pygrep-hooks
rev: v1.5.1
hooks:
- id: python-check-blanket-noqa

- repo: https://github.com/PyCQA/isort
rev: 5.12.0
hooks:
- id: isort

- repo: https://github.com/pycqa/flake8
rev: 4.0.1
hooks:
- id: flake8
additional_dependencies:
- flake8-docstrings~=1.6.0
7 changes: 7 additions & 0 deletions itinerant-iterators/LICENSE
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
Copyright 2021 Python Discord
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We don't hold the copyright to your project's code, don't forget to change this next year if you're participating again :)


Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
124 changes: 124 additions & 0 deletions itinerant-iterators/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,124 @@
# The Competition

This repository is our entry to the Python Discord Summer Code Jam 2023. In this competition, individuals are randomly allocated 4 other teammates and given just over a week to make something that fits a theme.

This year, the theme was "Secret codes". Submissions also had to incorporate image manipulation, using one of the [approved frameworks](https://www.pythondiscord.com/events/code-jams/10/frameworks/). We chose to use pillow as our primary framework for image manipulation.

Find out more about the competition [here](https://www.pythondiscord.com/events/code-jams/10/).

# Our Program

After our discussion, we decide to made a image manipulation tool which can encoding data or hiding secret codes into selected image, and obfuscate selected text inside the image, so that certain message will become a secret.

This is primarily intended as a command line tool to quickly get things done however we have also made a UI so that it's nice and easy to use for new users. Both interfaces have exactly the same functionality, and each have their strengths and weaknesses so use whichever you prefer.

Our tools include two modes: watermarking and obfuscator.

## 1. Watermarking

This mode uses steganography to hide a secret code in the image (like a hidden watermark) that can help identify the image later. You can then share this image around but it will always have your signature secretly attached to it. For example, you could send the same image with different signatures to different people, and then figure out who leaked it!

## 2. Obfuscator

The other tool is a way of automatically detecting text (or text matching a regex) in an image and automatically obfuscating it. This is useful if you have secret information that you want to remove, you can do it all automatically! For example, you could type "password: .*" to censor all passwords.

# Frameworks Used

Notable frameworks include:
- PySimpleGUI
- numpy
- Google Cloud Vision
- OpenCV

# Example Pictures

Here's some pictures of it working:

## Obfuscator

### Original Image
![Original image](https://cdn.discordapp.com/attachments/1145778261549404243/1151922040547319868/res.jpg)

### Obfuscated
![Obfuscated](https://cdn.discordapp.com/attachments/1145778261549404243/1151922113100398592/image.png)

### Obfuscated (with a regex)
![Obfuscated Regex](https://cdn.discordapp.com/attachments/1145778261549404243/1151922321834115202/image.png)

### Obfuscated (blur instead of black box)
![Obfuscated Blur](https://cdn.discordapp.com/attachments/1145778261549404243/1151922555549138974/image.png)

## Watermarking

![Watermarking](https://cdn.discordapp.com/attachments/1145778261549404243/1151925841807286272/Screenshot_2023-09-14_175629.png)

# Setup

#### Create a virtual environment (optional)
Create a virtual environment in the folder `.venv`.
```shell
$ python -m venv .venv
```

#### Enter the environment (optional)
It will change based on your operating system and shell.
```shell
# Linux, Bash
$ source .venv/bin/activate
# Linux, Fish
$ source .venv/bin/activate.fish
# Linux, Csh
$ source .venv/bin/activate.csh
# Linux, PowerShell Core
$ .venv/bin/Activate.ps1
# Windows, cmd.exe
> .venv\Scripts\activate.bat
# Windows, PowerShell
> .venv\Scripts\Activate.ps1
```

#### Install Dependencies
Use `$ pip install -r requirements.txt` to install the dependencies to run the application.
There are additional requirements to develop the program and run tests under `$ pip install -r dev-requirements.txt`.
Note that the code base requires Python 3.11+

#### Run the Application
```shell
# run the UI
$ python3 -m app ui
# run the CLI
$ python3 -m app cli <options> <args>
```

#### Example CLI
```shell
# watermark
python3 -m app cli watermark encode "secret message" data/image.jpg data/encoded_image.png
python3 -m app cli watermark decode data/encoded_image.png
secret message

# obfuscate
python3 -m app cli obfuscate data/image.jpg data/output.jpg --regex "\d+" --mode blur
python3 -m app cli obfuscate data/image.jpg data/output.jpg "hide this text" "other text" --mode blur
python3 -m app cli obfuscate data/image.jpg data/output.jpg "hide this text" "other text" --mode colour --colour red
```

#### Running Tests
`$ pytest`

# Known Issues

Unfortunately, not everything always works properly. Here are the issues we're aware of and any potential workarounds:

- Watermarking does not work properly in the GUI. Please use the CLI instead.
- Some unusual images can cause it to malfunction.

# The Team

This was made by:

- smileyface12349
- _jx2
- CactusBrother
- HiPeople
- LokiGray
2 changes: 2 additions & 0 deletions itinerant-iterators/app/__init__.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
# flake8: noqa F401
from . import byteutils, steganography
4 changes: 4 additions & 0 deletions itinerant-iterators/app/__main__.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
from app.cli import main
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't have a preference between relative and absolute imports, but being consistent is important. This breaks running the application from a different directory that isn't the repository root. I'd suggest using relative imports as those seem to be a better fit (the project isn't installable):

from .cli import main


if __name__ == "__main__":
main()
42 changes: 42 additions & 0 deletions itinerant-iterators/app/byteutils.py
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's great that the bit manipulation functions are located in their own module, but this should probably live in app.steganography since the obfuscation code makes no use of this code.

Original file line number Diff line number Diff line change
@@ -0,0 +1,42 @@
from typing import Generator


def iter_bits(bytes_: bytes) -> Generator[int, None, None]:
"""Enumerate over each bit in the given bytes.

Args:
bytes_ (bytes): bytes to enumerate over

Yields:
Generator[int, None, None]: Return each bit as an int (0/1)
"""
Comment on lines +4 to +12
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Avoid repeating the argument and return type annotations in the docstring. An IDE should be able to use both to provide rich help. Down the road, it's very likely the docstring types will be out of date with the actual type annotations.

(This applies to the rest of the project, but this was the first example of it.)

for byte in bytes_:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm impressed to see the terminating underscore convention for names that would (otherwise) shadow a built-in name, kudos!

bitstring = f"{byte:0>8b}"
for bit in bitstring:
yield int(bit)


def set_bit(value: int, bit: int) -> int:
"""Set the bit as position on the value.

Args:
value (int): Value to set bit on
bit (int): Bit position to set

Returns:
int: value with the bit at position `bit` set
"""
Comment on lines +20 to +28
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is what I mean by unclear docstrings. It's good that they exist, but it's no good if they do little to help the reader with what the function does. How about this?

Suggested change
"""Set the bit as position on the value.
Args:
value (int): Value to set bit on
bit (int): Bit position to set
Returns:
int: value with the bit at position `bit` set
"""
"""Set a bit in a byte to one.
Args:
value: Byte to set bit on
bit: Bit index to set
Returns:
int: Value with the bit at :bit: location set.
"""

return value | (1 << bit)


def clear_bit(value: int, bit: int) -> int:
"""Clear the bit at position on the value.

Args:
value (int): Value to clear bit on
bit (int): Bit position to clear

Returns:
int: value with the bit at position `bit` cleared
"""
return value & ~(1 << bit)
Loading