Skip to content

table.tometadata() fails on index that was created on interim version of table #4279

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
sqlalchemy-bot opened this issue Jun 15, 2018 · 10 comments
Labels
bug Something isn't working schema things related to the DDL related objects like Table, Column, CreateIndex, etc.
Milestone

Comments

@sqlalchemy-bot
Copy link
Collaborator

Migrated issue, originally created by Michael Bayer (@zzzeek)

This seems to be a regression in 1.2 due to #4147

from sqlalchemy import *

m1 = MetaData()

t1 = Table(
    'test',
    m1,
    Column('id', Integer, primary_key=True),
    Column('eventtime', DateTime),
)

Index("ix_test", t1.c.eventtime)

t1 = Table(
    'test',
    m1,
    Column('id', Integer, primary_key=True),
    Column('eventtime', DateTime),
    extend_existing=True
)


m2 = MetaData()

t2 = t1.tometadata(m2)

output:

#!

Traceback (most recent call last):
  File "test.py", line 25, in <module>
    t2 = t1.tometadata(m2)
  File "/home/classic/dev/sqlalchemy/lib/sqlalchemy/sql/schema.py", line 911, in tometadata
    **index.kwargs)
  File "/home/classic/dev/sqlalchemy/lib/sqlalchemy/sql/schema.py", line 3447, in __init__
    self._set_parent(table)
  File "/home/classic/dev/sqlalchemy/lib/sqlalchemy/sql/schema.py", line 3458, in _set_parent
    table.description
sqlalchemy.exc.ArgumentError: Index 'ix_test' is against table 'test', and cannot be associated with table 'test'.

@sqlalchemy-bot
Copy link
Collaborator Author

Changes by Michael Bayer (@zzzeek):

  • edited description

@sqlalchemy-bot
Copy link
Collaborator Author

Michael Bayer (@zzzeek) wrote:

ok that is up at https://gerrit.sqlalchemy.org/#/q/I521aa2c9f3baa0e84598bbdd6ffe4bf07b6e3ba8. the gerrit queue is enormous right now, will take a few days for this to get in.

@sqlalchemy-bot
Copy link
Collaborator Author

colladoman wrote:

Thank you!

For me it would a be perfect workaround (and a simple and great new feature) doing something like that in tometadata method:

    def tometadata(self, metadata, schema=RETAIN_SCHEMA,
                   referred_schema_fn=None, name=None, copy_indexes=True):
...
...
...
    if copy_indexes:
        for index in self.indexes:
            ...
            ...
            ...

@sqlalchemy-bot
Copy link
Collaborator Author

Michael Bayer (@zzzeek) wrote:

fine grained control over tometadata() would be nice but if you make it "COPY_X=True" kind of thing, what about: unique constraints, primary key constraints, check constraints, Posgresql-specific EXCLUDES contraints, etc, then you have an API like:

 def tometadata(self, metadata, schema=RETAIN_SCHEMA,
               referred_schema_fn=None, name=None, copy_indexes=True, copy_pk=True, copy_check=True, copy_unqiue=True, postgresql_copy_exclude=True, ...):

dialects like Postgresql need to be able to inject new "mydialect_copy_myddl=True" options into it, so that would be **kw as well.

I usually do exclusion APIs more like this:

 def tometadata(self, metadata, schema=RETAIN_SCHEMA,
               referred_schema_fn=None, name=None, include_object=None):

include_object is a function that receives each object, it can then return True/False to indicate if that object should be included. metadata.reflect() has this as does the autogenerate system in Alembic.

@sqlalchemy-bot
Copy link
Collaborator Author

colladoman wrote:

Yours is really a much better aproach, but i can't fully understand how it should be used. I would appreciate an example when it is implemented.
Thank you again!

@sqlalchemy-bot
Copy link
Collaborator Author

Michael Bayer (@zzzeek) wrote:

the bug is fixed, you use tometadata() like you were doing.

@sqlalchemy-bot
Copy link
Collaborator Author

Michael Bayer (@zzzeek) wrote:

Lookup index columns in parent table by key for copy

Fixed regression in 1.2 due to 🎫4147 where a :class:.Table that
has had some of its indexed columns redefined with new ones, as would occur
when overriding columns during reflection or when using
:paramref:.Table.extend_existing, such that the :meth:.Table.tometadata
method would fail when attempting to copy those indexes as they still
referred to the replaced column. The copy logic now accommodates for this
condition.

Change-Id: I521aa2c9f3baa0e84598bbdd6ffe4bf07b6e3ba8
Fixes: #4279

8f7766c

@sqlalchemy-bot
Copy link
Collaborator Author

Changes by Michael Bayer (@zzzeek):

  • changed status to closed

@sqlalchemy-bot
Copy link
Collaborator Author

Michael Bayer (@zzzeek) wrote:

Lookup index columns in parent table by key for copy

Fixed regression in 1.2 due to 🎫4147 where a :class:.Table that
has had some of its indexed columns redefined with new ones, as would occur
when overriding columns during reflection or when using
:paramref:.Table.extend_existing, such that the :meth:.Table.tometadata
method would fail when attempting to copy those indexes as they still
referred to the replaced column. The copy logic now accommodates for this
condition.

Change-Id: I521aa2c9f3baa0e84598bbdd6ffe4bf07b6e3ba8
Fixes: #4279
(cherry picked from commit 8f7766c)

b6f479b

@sqlalchemy-bot
Copy link
Collaborator Author

colladoman wrote:

Awesome! Thanks!

@sqlalchemy-bot sqlalchemy-bot added schema things related to the DDL related objects like Table, Column, CreateIndex, etc. bug Something isn't working labels Nov 27, 2018
@sqlalchemy-bot sqlalchemy-bot added this to the 1.2.x milestone Nov 27, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working schema things related to the DDL related objects like Table, Column, CreateIndex, etc.
Projects
None yet
Development

No branches or pull requests

1 participant