Skip to content

eager_defaults does not work with single table inheritance #3908

New issue
New issue
Closed
Closed
eager_defaults does not work with single table inheritance#3908
Labels
bugSomething isn't workingorm
Milestone
 
1.1.xx
@sqlalchemy-bot

Description

@sqlalchemy-bot
sqlalchemy-bot
opened on Feb 9, 2017
Collaborator

Migrated issue, originally created by Jack Zhou (@univerio)

When using single table inheritance, eager_defaults seems to be affected by the columns that the subclasses add to the base table:

class Foo(Base):
    __tablename__ = "foo"
    id = Column(Integer, primary_key=True)
    type = Column(String)
    created_at = Column(DateTime(), server_default=func.now())

    __mapper_args__ = {
        "polymorphic_on": type,
        "polymorphic_identity": "foo",
        "eager_defaults": True,
    }


class Bar(Foo):
    bar = Column(String)
    __mapper_args__ = {
        "polymorphic_identity": "bar",
    }

foo = Foo()
session.add(foo)
session.flush()

This results in this exception

Traceback (most recent call last):
  File "sqla.py", line 59, in <module>
    main()
  File "sqla.py", line 51, in main
    session.flush()
  File "/home/jack/dev/test/.venv/lib/python3.5/site-packages/sqlalchemy/orm/session.py", line 2139, in flush
    self._flush(objects)
  File "/home/jack/dev/test/.venv/lib/python3.5/site-packages/sqlalchemy/orm/session.py", line 2259, in _flush
    transaction.rollback(_capture_exception=True)
  File "/home/jack/dev/test/.venv/lib/python3.5/site-packages/sqlalchemy/util/langhelpers.py", line 60, in __exit__
    compat.reraise(exc_type, exc_value, exc_tb)
  File "/home/jack/dev/test/.venv/lib/python3.5/site-packages/sqlalchemy/util/compat.py", line 187, in reraise
    raise value
  File "/home/jack/dev/test/.venv/lib/python3.5/site-packages/sqlalchemy/orm/session.py", line 2223, in _flush
    flush_context.execute()
  File "/home/jack/dev/test/.venv/lib/python3.5/site-packages/sqlalchemy/orm/unitofwork.py", line 389, in execute
    rec.execute(self)
  File "/home/jack/dev/test/.venv/lib/python3.5/site-packages/sqlalchemy/orm/unitofwork.py", line 548, in execute
    uow
  File "/home/jack/dev/test/.venv/lib/python3.5/site-packages/sqlalchemy/orm/persistence.py", line 181, in save_obj
    mapper, table, insert)
  File "/home/jack/dev/test/.venv/lib/python3.5/site-packages/sqlalchemy/orm/persistence.py", line 856, in _emit_insert_statements
    value_params)
  File "/home/jack/dev/test/.venv/lib/python3.5/site-packages/sqlalchemy/orm/persistence.py", line 1043, in _postfetch
    dict_[mapper._columntoproperty[col].key] = row[col]
  File "/home/jack/dev/test/.venv/lib/python3.5/site-packages/sqlalchemy/orm/mapper.py", line 2991, in __missing__
    (column, self.mapper))
sqlalchemy.orm.exc.UnmappedColumnError: No column foo.bar is configured on mapper Mapper|Foo|foo...

When attempting to add a Bar instead,

bar = Bar()
session.add(bar)
session.flush()

A different exception is raised:

Traceback (most recent call last):
  File "sqla.py", line 62, in <module>
    main()
  File "sqla.py", line 54, in main
    session.commit()
  File "/home/jack/dev/test/.venv/lib/python3.5/site-packages/sqlalchemy/orm/session.py", line 874, in commit
    self.transaction.commit()
  File "/home/jack/dev/test/.venv/lib/python3.5/site-packages/sqlalchemy/orm/session.py", line 461, in commit
    self._prepare_impl()
  File "/home/jack/dev/test/.venv/lib/python3.5/site-packages/sqlalchemy/orm/session.py", line 441, in _prepare_impl
    self.session.flush()
  File "/home/jack/dev/test/.venv/lib/python3.5/site-packages/sqlalchemy/orm/session.py", line 2139, in flush
    self._flush(objects)
  File "/home/jack/dev/test/.venv/lib/python3.5/site-packages/sqlalchemy/orm/session.py", line 2259, in _flush
    transaction.rollback(_capture_exception=True)
  File "/home/jack/dev/test/.venv/lib/python3.5/site-packages/sqlalchemy/util/langhelpers.py", line 60, in __exit__
    compat.reraise(exc_type, exc_value, exc_tb)
  File "/home/jack/dev/test/.venv/lib/python3.5/site-packages/sqlalchemy/util/compat.py", line 187, in reraise
    raise value
  File "/home/jack/dev/test/.venv/lib/python3.5/site-packages/sqlalchemy/orm/session.py", line 2223, in _flush
    flush_context.execute()
  File "/home/jack/dev/test/.venv/lib/python3.5/site-packages/sqlalchemy/orm/unitofwork.py", line 389, in execute
    rec.execute(self)
  File "/home/jack/dev/test/.venv/lib/python3.5/site-packages/sqlalchemy/orm/unitofwork.py", line 548, in execute
    uow
  File "/home/jack/dev/test/.venv/lib/python3.5/site-packages/sqlalchemy/orm/persistence.py", line 193, in save_obj
    update_version_id in states_to_update
  File "/home/jack/dev/test/.venv/lib/python3.5/site-packages/sqlalchemy/orm/persistence.py", line 1008, in _finalize_insert_update_commands
    only_load_props=toload_now)
  File "/home/jack/dev/test/.venv/lib/python3.5/site-packages/sqlalchemy/orm/loading.py", line 223, in load_on_ident
    return q.one()
  File "/home/jack/dev/test/.venv/lib/python3.5/site-packages/sqlalchemy/orm/query.py", line 2749, in one
    ret = self.one_or_none()
  File "/home/jack/dev/test/.venv/lib/python3.5/site-packages/sqlalchemy/orm/query.py", line 2719, in one_or_none
    ret = list(self)
  File "/home/jack/dev/test/.venv/lib/python3.5/site-packages/sqlalchemy/orm/query.py", line 2786, in __iter__
    context = self._compile_context()
  File "/home/jack/dev/test/.venv/lib/python3.5/site-packages/sqlalchemy/orm/query.py", line 3320, in _compile_context
    "No column-based properties specified for "
sqlalchemy.exc.InvalidRequestError: No column-based properties specified for refresh operation. Use session.expire() to reload collections and related items.

When I specify "with_polymorphic": "*" on Foo,

class Foo(Base):
    __tablename__ = "foo"
    id = Column(Integer, primary_key=True)
    type = Column(String)
    created_at = Column(DateTime(), server_default=func.now())

    __mapper_args__ = {
        "polymorphic_on": type,
        "polymorphic_identity": "foo",
        "eager_defaults": True,
        "with_polymorphic": "*",
    }

bar = Bar()
session.add(bar)
session.flush()

Yet another exception is raised:

Traceback (most recent call last):
  File "sqla.py", line 63, in <module>
    main()
  File "sqla.py", line 55, in main
    session.commit()
  File "/home/jack/dev/test/.venv/lib/python3.5/site-packages/sqlalchemy/orm/session.py", line 874, in commit
    self.transaction.commit()
  File "/home/jack/dev/test/.venv/lib/python3.5/site-packages/sqlalchemy/orm/session.py", line 461, in commit
    self._prepare_impl()
  File "/home/jack/dev/test/.venv/lib/python3.5/site-packages/sqlalchemy/orm/session.py", line 441, in _prepare_impl
    self.session.flush()
  File "/home/jack/dev/test/.venv/lib/python3.5/site-packages/sqlalchemy/orm/session.py", line 2139, in flush
    self._flush(objects)
  File "/home/jack/dev/test/.venv/lib/python3.5/site-packages/sqlalchemy/orm/session.py", line 2259, in _flush
    transaction.rollback(_capture_exception=True)
  File "/home/jack/dev/test/.venv/lib/python3.5/site-packages/sqlalchemy/util/langhelpers.py", line 60, in __exit__
    compat.reraise(exc_type, exc_value, exc_tb)
  File "/home/jack/dev/test/.venv/lib/python3.5/site-packages/sqlalchemy/util/compat.py", line 187, in reraise
    raise value
  File "/home/jack/dev/test/.venv/lib/python3.5/site-packages/sqlalchemy/orm/session.py", line 2223, in _flush
    flush_context.execute()
  File "/home/jack/dev/test/.venv/lib/python3.5/site-packages/sqlalchemy/orm/unitofwork.py", line 389, in execute
    rec.execute(self)
  File "/home/jack/dev/test/.venv/lib/python3.5/site-packages/sqlalchemy/orm/unitofwork.py", line 548, in execute
    uow
  File "/home/jack/dev/test/.venv/lib/python3.5/site-packages/sqlalchemy/orm/persistence.py", line 193, in save_obj
    update_version_id in states_to_update
  File "/home/jack/dev/test/.venv/lib/python3.5/site-packages/sqlalchemy/orm/persistence.py", line 1008, in _finalize_insert_update_commands
    only_load_props=toload_now)
  File "/home/jack/dev/test/.venv/lib/python3.5/site-packages/sqlalchemy/orm/loading.py", line 223, in load_on_ident
    return q.one()
  File "/home/jack/dev/test/.venv/lib/python3.5/site-packages/sqlalchemy/orm/query.py", line 2749, in one
    ret = self.one_or_none()
  File "/home/jack/dev/test/.venv/lib/python3.5/site-packages/sqlalchemy/orm/query.py", line 2719, in one_or_none
    ret = list(self)
  File "/home/jack/dev/test/.venv/lib/python3.5/site-packages/sqlalchemy/orm/loading.py", line 90, in instances
    util.raise_from_cause(err)
  File "/home/jack/dev/test/.venv/lib/python3.5/site-packages/sqlalchemy/util/compat.py", line 203, in raise_from_cause
    reraise(type(exception), exception, tb=exc_tb, cause=cause)
  File "/home/jack/dev/test/.venv/lib/python3.5/site-packages/sqlalchemy/util/compat.py", line 187, in reraise
    raise value
  File "/home/jack/dev/test/.venv/lib/python3.5/site-packages/sqlalchemy/orm/loading.py", line 57, in instances
    for query_entity in query._entities
  File "/home/jack/dev/test/.venv/lib/python3.5/site-packages/sqlalchemy/orm/loading.py", line 57, in <listcomp>
    for query_entity in query._entities
  File "/home/jack/dev/test/.venv/lib/python3.5/site-packages/sqlalchemy/orm/query.py", line 3636, in row_processor
    polymorphic_discriminator=self._polymorphic_discriminator
  File "/home/jack/dev/test/.venv/lib/python3.5/site-packages/sqlalchemy/orm/loading.py", line 299, in _instance_processor
    mapper._props[k] for k in only_load_props)
  File "/home/jack/dev/test/.venv/lib/python3.5/site-packages/sqlalchemy/orm/loading.py", line 299, in <genexpr>
    mapper._props[k] for k in only_load_props)
KeyError: 'bar'

Activity

sqlalchemy-bot

sqlalchemy-bot commented on Feb 9, 2017

@sqlalchemy-bot
sqlalchemy-bot
on Feb 9, 2017
CollaboratorAuthor

Michael Bayer (@zzzeek) wrote:

hint: you're using postgresql, bugs are related to returning

sqlalchemy-bot

sqlalchemy-bot commented on Feb 9, 2017

@sqlalchemy-bot
sqlalchemy-bot
on Feb 9, 2017
CollaboratorAuthor

Jack Zhou (@univerio) wrote:

This happens on SQLite as well:

class Foo(Base):
    __tablename__ = "foo"
    id = Column(Integer, primary_key=True)
    type = Column(String)
    created_at = Column(DateTime(), server_default=func.now())

    __mapper_args__ = {
        "polymorphic_on": type,
        "polymorphic_identity": "foo",
        "eager_defaults": True,
    }


class Bar(Foo):
    bar = Column(String)
    __mapper_args__ = {
        "polymorphic_identity": "bar",
    }

bar = Bar()
session.add(bar)
session.commit()

Traceback:

Traceback (most recent call last):
  File "sqlite.py", line 55, in <module>
    main()
  File "sqlite.py", line 51, in main
    session.commit()
  File "/home/jack/dev/test/.venv/lib/python3.5/site-packages/sqlalchemy/orm/session.py", line 874, in commit
    self.transaction.commit()
  File "/home/jack/dev/test/.venv/lib/python3.5/site-packages/sqlalchemy/orm/session.py", line 461, in commit
    self._prepare_impl()
  File "/home/jack/dev/test/.venv/lib/python3.5/site-packages/sqlalchemy/orm/session.py", line 441, in _prepare_impl
    self.session.flush()
  File "/home/jack/dev/test/.venv/lib/python3.5/site-packages/sqlalchemy/orm/session.py", line 2139, in flush
    self._flush(objects)
  File "/home/jack/dev/test/.venv/lib/python3.5/site-packages/sqlalchemy/orm/session.py", line 2259, in _flush
    transaction.rollback(_capture_exception=True)
  File "/home/jack/dev/test/.venv/lib/python3.5/site-packages/sqlalchemy/util/langhelpers.py", line 60, in __exit__
    compat.reraise(exc_type, exc_value, exc_tb)
  File "/home/jack/dev/test/.venv/lib/python3.5/site-packages/sqlalchemy/util/compat.py", line 187, in reraise
    raise value
  File "/home/jack/dev/test/.venv/lib/python3.5/site-packages/sqlalchemy/orm/session.py", line 2223, in _flush
    flush_context.execute()
  File "/home/jack/dev/test/.venv/lib/python3.5/site-packages/sqlalchemy/orm/unitofwork.py", line 389, in execute
    rec.execute(self)
  File "/home/jack/dev/test/.venv/lib/python3.5/site-packages/sqlalchemy/orm/unitofwork.py", line 548, in execute
    uow
  File "/home/jack/dev/test/.venv/lib/python3.5/site-packages/sqlalchemy/orm/persistence.py", line 193, in save_obj
    update_version_id in states_to_update
  File "/home/jack/dev/test/.venv/lib/python3.5/site-packages/sqlalchemy/orm/persistence.py", line 1008, in _finalize_insert_update_commands
    only_load_props=toload_now)
  File "/home/jack/dev/test/.venv/lib/python3.5/site-packages/sqlalchemy/orm/loading.py", line 223, in load_on_ident
    return q.one()
  File "/home/jack/dev/test/.venv/lib/python3.5/site-packages/sqlalchemy/orm/query.py", line 2749, in one
    ret = self.one_or_none()
  File "/home/jack/dev/test/.venv/lib/python3.5/site-packages/sqlalchemy/orm/query.py", line 2719, in one_or_none
    ret = list(self)
  File "/home/jack/dev/test/.venv/lib/python3.5/site-packages/sqlalchemy/orm/loading.py", line 90, in instances
    util.raise_from_cause(err)
  File "/home/jack/dev/test/.venv/lib/python3.5/site-packages/sqlalchemy/util/compat.py", line 203, in raise_from_cause
    reraise(type(exception), exception, tb=exc_tb, cause=cause)
  File "/home/jack/dev/test/.venv/lib/python3.5/site-packages/sqlalchemy/util/compat.py", line 187, in reraise
    raise value
  File "/home/jack/dev/test/.venv/lib/python3.5/site-packages/sqlalchemy/orm/loading.py", line 57, in instances
    for query_entity in query._entities
  File "/home/jack/dev/test/.venv/lib/python3.5/site-packages/sqlalchemy/orm/loading.py", line 57, in <listcomp>
    for query_entity in query._entities
  File "/home/jack/dev/test/.venv/lib/python3.5/site-packages/sqlalchemy/orm/query.py", line 3636, in row_processor
    polymorphic_discriminator=self._polymorphic_discriminator
  File "/home/jack/dev/test/.venv/lib/python3.5/site-packages/sqlalchemy/orm/loading.py", line 299, in _instance_processor
    mapper._props[k] for k in only_load_props)
  File "/home/jack/dev/test/.venv/lib/python3.5/site-packages/sqlalchemy/orm/loading.py", line 299, in <genexpr>
    mapper._props[k] for k in only_load_props)
KeyError: 'bar'
sqlalchemy-bot

sqlalchemy-bot commented on Feb 9, 2017

@sqlalchemy-bot
sqlalchemy-bot
on Feb 9, 2017
CollaboratorAuthor

Michael Bayer (@zzzeek) wrote:

note the eager_defaults is also doing a superfluous SELECT here and I've added #3909 for that.

sqlalchemy-bot

sqlalchemy-bot commented on Feb 9, 2017

@sqlalchemy-bot
sqlalchemy-bot
on Feb 9, 2017
CollaboratorAuthor

Changes by Michael Bayer (@zzzeek):

  • removed labels: low priority
sqlalchemy-bot

sqlalchemy-bot commented on Feb 9, 2017

@sqlalchemy-bot
sqlalchemy-bot
on Feb 9, 2017
CollaboratorAuthor

Changes by Michael Bayer (@zzzeek):

  • added labels: orm
sqlalchemy-bot

sqlalchemy-bot commented on Feb 9, 2017

@sqlalchemy-bot
sqlalchemy-bot
on Feb 9, 2017
CollaboratorAuthor

Changes by Michael Bayer (@zzzeek):

  • set milestone to "1.1.x"
sqlalchemy-bot

sqlalchemy-bot commented on Feb 9, 2017

@sqlalchemy-bot
sqlalchemy-bot
on Feb 9, 2017
CollaboratorAuthor

Michael Bayer (@zzzeek) wrote:

yes SQLIte the "foo" cases work though

sqlalchemy-bot

sqlalchemy-bot commented on Feb 9, 2017

@sqlalchemy-bot
sqlalchemy-bot
on Feb 9, 2017
CollaboratorAuthor

Michael Bayer (@zzzeek) wrote:

Check for columns not part of mapping, correct mapping for eager_defaults

Fixed two closely related bugs involving the mapper eager_defaults
flag in conjunction with single-table inheritance; one where the
eager defaults logic would inadvertently try to access a column
that's part of the mapper's "exclude_properties" list (used by
Declarative with single table inheritance) during the eager defaults
fetch, and the other where the full load of the row in order to
fetch the defaults would fail to use the correct inheriting mapper.

Fixes: #3908
Change-Id: Ie745174c917d512e2c46d9e3cc14512cde53cc9a

→ 540bcff

sqlalchemy-bot

sqlalchemy-bot commented on Feb 9, 2017

@sqlalchemy-bot
sqlalchemy-bot
on Feb 9, 2017
CollaboratorAuthor

Changes by Michael Bayer (@zzzeek):

  • changed status to closed
sqlalchemy-bot
closed this as completedon Feb 9, 2017
sqlalchemy-bot
added
bugSomething isn't working
orm
on Nov 27, 2018
sqlalchemy-bot
added this to the 1.1.x milestone on Nov 27, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't workingorm

    Type

    No type

    Projects

    No projects

    Milestone

    Relationships

    None yet

      Development

      No branches or pull requests

        Participants

        @sqlalchemy-bot

        Issue actions
          eager_defaults does not work with single table inheritance · Issue #3908 · sqlalchemy/sqlalchemy