Manual test fails but runtests.py passes

[jcorbettfrank]

Hi, first I wanted to say thanks for making this library. It will be very helpful to me.

I've been getting an error when trying to work with nested repeated Protobuf messages. Whenever I migrate I get "django.db.utils.OperationalError: duplicate column name: <repeated_field_name>_index". In my attempts to debug this, I worked with the test code provided in django-pb-model/pb_model/tests. When I run django-pb-mode/runtests.py everything passes. However, if I create a separate Django project/app and copy over the models.py, tests.py, and models_pb2.py files from django-pb-model/pb_model/tests I get the same error "django.db.utils.OperationalError: duplicate column name: map_string_to_message_field_index".

Below are the steps I followed to reliably produce this error with the supplied test code.

Steps to Reproduce (Ubuntu 18.04):

1. Install Django
   $ pip install Django
2. Install django-pb-model
   $ pip install django-pb-model
3. Make an overall test directory
   $ mkdir test
4. Clone django-pb-model
5. In your test directory, create a Django test project and app
   $ cd test
   $ django-admin startproject pb_test
   $ cd pb_test
   $ python manage.py startapp pb_app_test
6. Delete the automatically created models.py and tests.py in the pb_app_test directory
   $ rm pb_app_test/models.py pb_app_test/tests.py
7. Copy the test models.py, tests.py, and models_pb2.py from the cloned django-pb-model/pb_model/tests directory into your test app
   $ cd test/django-pb-model/pb_model/tests
   $ cp tests.py models.py models_pb2.py test/pb_test/pb_app_test/
8. Edit test/pb_test/pb_test/settings.py to include pb_model and pb_app_test apps
   INSTALLED_APPS = [
   'pb_model',
   'pb_app_test.apps.PbAppTestConfig',
   ...,
   ]
9. Make migrations in your test app (from within test/pb_test)
   $ python manage.py makemigrations
   • Create model Embedded
   • Create model ListWrapper
   • Create model M2MRelation
   • etc.
10. Now I can get the reproducible error from either running
    $ python manage.py migrate
    or
    $ python manage.py test

I've put the stack trace at the bottom. So essentially what I did was manually create a fully fledged Django project/app with the test models from django-pb-models/pb_model/tests and get an error that I do not get when I just run runtests.py. This is the same error that I've been seeing when I initially tried to use django-pb-model with my custom protobuf messages.

Creating test database for alias 'default'... Traceback (most recent call last): File "/home/jo28861/miniconda3/lib/python3.7/site-packages/django/db/backends/utils.py", line 84, in _execute
return self.cursor.execute(sql)
File "/home/jo28861/miniconda3/lib/python3.7/site-packages/django/db/backends/sqlite3/base.py", line 394, in execute
return Database.Cursor.execute(self, query)
sqlite3.OperationalError: duplicate column name: map_string_to_message_field_index

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "manage.py", line 22, in
execute_from_command_line(sys.argv)
File "/home/jo28861/miniconda3/lib/python3.7/site-packages/django/core/management/init.py", line 401, in execute_from_command_line
utility.execute()
File "/home/jo28861/miniconda3/lib/python3.7/site-packages/django/core/management/init.py", line 395, in execute
self.fetch_command(subcommand).run_from_argv(self.argv)
File "/home/jo28861/miniconda3/lib/python3.7/site-packages/django/core/management/commands/test.py", line 23, in run_from_argv
super().run_from_argv(argv)
File "/home/jo28861/miniconda3/lib/python3.7/site-packages/django/core/management/base.py", line 328, in run_from_argv
self.execute(*args, **cmd_options)
File "/home/jo28861/miniconda3/lib/python3.7/site-packages/django/core/management/base.py", line 369, in execute
output = self.handle(*args, **options)
File "/home/jo28861/miniconda3/lib/python3.7/site-packages/django/core/management/commands/test.py", line 53, in handle
failures = test_runner.run_tests(test_labels)
File "/home/jo28861/miniconda3/lib/python3.7/site-packages/django/test/runner.py", line 684, in run_tests
old_config = self.setup_databases(aliases=databases)
File "/home/jo28861/miniconda3/lib/python3.7/site-packages/django/test/runner.py", line 606, in setup_databases
self.parallel, **kwargs
File "/home/jo28861/miniconda3/lib/python3.7/site-packages/django/test/utils.py", line 173, in setup_databases
serialize=connection.settings_dict.get('TEST', {}).get('SERIALIZE', True),
File "/home/jo28861/miniconda3/lib/python3.7/site-packages/django/db/backends/base/creation.py", line 72, in create_test_db
run_syncdb=True,
File "/home/jo28861/miniconda3/lib/python3.7/site-packages/django/core/management/init.py", line 168, in call_command
return command.execute(*args, **defaults)
File "/home/jo28861/miniconda3/lib/python3.7/site-packages/django/core/management/base.py", line 369, in execute
output = self.handle(*args, **options)
File "/home/jo28861/miniconda3/lib/python3.7/site-packages/django/core/management/base.py", line 83, in wrapped
res = handle_func(*args, **kwargs)
File "/home/jo28861/miniconda3/lib/python3.7/site-packages/django/core/management/commands/migrate.py", line 233, in handle
fake_initial=fake_initial,
File "/home/jo28861/miniconda3/lib/python3.7/site-packages/django/db/migrations/executor.py", line 117, in migrate
state = self._migrate_all_forwards(state, plan, full_plan, fake=fake, fake_initial=fake_initial)
File "/home/jo28861/miniconda3/lib/python3.7/site-packages/django/db/migrations/executor.py", line 147, in _migrate_all_forwards
state = self.apply_migration(state, migration, fake=fake, fake_initial=fake_initial)
File "/home/jo28861/miniconda3/lib/python3.7/site-packages/django/db/migrations/executor.py", line 245, in apply_migration
state = migration.apply(state, schema_editor)
File "/home/jo28861/miniconda3/lib/python3.7/site-packages/django/db/migrations/migration.py", line 124, in apply
operation.database_forwards(self.app_label, schema_editor, old_state, project_state)
File "/home/jo28861/miniconda3/lib/python3.7/site-packages/django/db/migrations/operations/models.py", line 92, in database_forwards
schema_editor.create_model(model)
File "/home/jo28861/miniconda3/lib/python3.7/site-packages/django/db/backends/base/schema.py", line 324, in create_model
self.execute(sql, params or None)
File "/home/jo28861/miniconda3/lib/python3.7/site-packages/django/db/backends/base/schema.py", line 142, in execute
cursor.execute(sql, params)
File "/home/jo28861/miniconda3/lib/python3.7/site-packages/django/db/backends/utils.py", line 68, in execute
return self._execute_with_wrappers(sql, params, many=False, executor=self._execute)
File "/home/jo28861/miniconda3/lib/python3.7/site-packages/django/db/backends/utils.py", line 77, in _execute_with_wrappers
return executor(sql, params, many, context)
File "/home/jo28861/miniconda3/lib/python3.7/site-packages/django/db/backends/utils.py", line 86, in _execute
return self.cursor.execute(sql, params)
File "/home/jo28861/miniconda3/lib/python3.7/site-packages/django/db/utils.py", line 90, in exit
raise dj_exc_value.with_traceback(traceback) from exc_value
File "/home/jo28861/miniconda3/lib/python3.7/site-packages/django/db/backends/utils.py", line 84, in _execute
return self.cursor.execute(sql)
File "/home/jo28861/miniconda3/lib/python3.7/site-packages/django/db/backends/sqlite3/base.py", line 394, in execute
return Database.Cursor.execute(self, query)
django.db.utils.OperationalError: duplicate column name: map_string_to_message_field_index