Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Slow Uploads using Google Drive API with Python #1852

Open
nammonjv opened this issue Nov 7, 2024 · 0 comments
Open

Slow Uploads using Google Drive API with Python #1852

nammonjv opened this issue Nov 7, 2024 · 0 comments
Assignees

Comments

@nammonjv
Copy link

nammonjv commented Nov 7, 2024

Expected Behavior

Automatically upload large files daily to Google Drive using Python, with upload speeds matching my internet connection.

Actual Behavior

The upload speed is stuck at around 20 Mbps, even when I use a faster network or adjust the chunk size in the script. I initially thought the issue was a limit within the Python code, but after testing the network speed in the same script, it showed speeds over 100 Mbps.

Here is my script:

file_name = os.path.basename(file_path)
file_size = os.path.getsize(file_path) / (1024 * 1024)

media = MediaFileUpload(file_path, mimetype='application/octet-stream', chunksize=256*1024 * 1024, resumable=True) 

file_metadata = {
    'name': file_name,
    'parents': [folder_id] if folder_id else []
}

try:
    request = service.files().create(media_body=media, body=file_metadata)
    start_time = time.time()
    response = request.execute()
    print(f"Uploaded '{file_name}' successfully at {datetime.now().strftime('%Y-%m-%d %H:%M:%S')}\n")
    elapsed_time = time.time() - start_time
    upload_speed = file_size / elapsed_time
    print(f"Time used: {elapsed_time}\nUpload speed: {upload_speed:.2f} MB/s or {upload_speed*8:.2f} Mbps")
    return True
except HttpError as error:
    print(f"Failed to upload '{file_name}': {error}")
    return False

I also tried changing the response part to:

response = None
while response is None:
    status, response = request.next_chunk()
    if status:
        print("Uploaded %d%%." % int(status.progress() * 100))

but it showed almost the same results.

Specifications

  • Python version: 3.12.7
  • OS: Windows
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants