Skip to content

Support for Inspecting/Reflecting PostgreSQL column defaults using GENERATED #5324

Closed
@sigmavirus24

Description

@sigmavirus24

Is your feature request related to a problem? Please describe.

Today, I use SQLAlchemy to reflect and inspect Postgres tables. People are starting to move away from SERIAL types to using GENERATED BY DEFAULT AS IDENTITY and GENERATED ALWAYS AS IDENTITY.

For example if I:

CREATE TABLE testidentity (id BIGINT PRIMARY KEY GENERATED BY DEFAULT AS IDENTITY, name VARCHAR NOT NULL, optional VARCHAR);

When I use sqlalchemy.inspect to eventually get_columns I'm given:

[{'name': 'id',
  'type': BIGINT(),
  'nullable': False,
  'default': None,
  'autoincrement': False,
  'comment': None},
 {'name': 'name',
  'type': VARCHAR(),
  'nullable': False,
  'default': None,
  'autoincrement': False,
  'comment': None},
 {'name': 'optional',
  'type': VARCHAR(),
  'nullable': True,
  'default': None,
  'autoincrement': False,
  'comment': None}]

But there is a default and it does autoincrement

Describe the solution you'd like

Support for describing these generated defaults as having a default and as autoincrementing (although the latter is less important)

Describe alternatives you've considered

Trying to inspect the metadata via some other library directly and working around SQLAlchemy

Additional context

N/A

Have a nice day!

Activity

sigmavirus24

sigmavirus24 commented on May 12, 2020

@sigmavirus24
Author

I'm also willing to assist with implementing this but I have no clue as to where to start as I've never contributed to this project before.

zzzeek

zzzeek commented on May 12, 2020

@zzzeek
Member

this is possibly not trivial, not on the reflection side, but on the mechanics of INSERT. all of the PG internals assume SERIAL for autoincrement and getting this format to work would be the hard part.

added
reflectionreflection of tables, columns, constraints, defaults, sequences, views, everything else
schemathings related to the DDL related objects like Table, Column, CreateIndex, etc.
use casenot really a feature or a bug; can be support for new DB features or user use cases not anticipated
on May 12, 2020
added this to the 1.4.x milestone on May 12, 2020
jvanasco

jvanasco commented on May 12, 2020

@jvanasco
Member

Further Reading:

zzzeek

zzzeek commented on May 12, 2020

@zzzeek
Member

we do support computed columns in general however i think PG's "identity" is a separate syntax.

jvanasco

jvanasco commented on May 12, 2020

@jvanasco
Member

Aside from reflection:

@zzzeek check out this link that goes into some of what is done regarding the auto-creation of serials. https://www.depesz.com/2017/04/10/waiting-for-postgresql-10-identity-columns/

CaselIT

CaselIT commented on May 12, 2020

@CaselIT
Member

We should check if the generated always as identity is already piked up by the reflection logic in 1.4, since the syntax is the same as a normal compute column.

zzzeek

zzzeek commented on May 12, 2020

@zzzeek
Member

We should check if the generated always as identity is already piked up by the reflection logic in 1.4, since the syntax is the same as a normal compute column.

the syntax is not what we generate, I tried this:


class A(Base):
    __tablename__ = 'a'

    id = Column(Integer, Computed("AS IDENTITY"), primary_key=True)
    data = Column(String)
e = create_engine("postgresql://scott:tiger@localhost/test", echo='debug')

Base.metadata.drop_all(e)
Base.metadata.create_all(e)


and got

sqlalchemy.exc.ProgrammingError: (psycopg2.errors.SyntaxError) syntax error at or near "("
LINE 3:  id INTEGER GENERATED ALWAYS AS (AS IDENTITY) STORED NOT NUL...
                                        ^

[SQL: 
CREATE TABLE a (
	id INTEGER GENERATED ALWAYS AS (AS IDENTITY) STORED NOT NULL, 
	data VARCHAR, 
	PRIMARY KEY (id)
)

I think this looks like a totally different (but very similar) syntax. One proposal is to add this as a column option:


class A(Base):
    __tablename__ = 'a'

    id = Column(Integer, postgresql_identity="BY DEFAULT", primary_key=True)
    data = Column(String)


class A(Base):
    __tablename__ = 'a'

    id = Column(Integer, postgresql_identity="ALWAYS", primary_key=True)
    data = Column(String)

49 remaining items

Loading
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    postgresqlreflectionreflection of tables, columns, constraints, defaults, sequences, views, everything elseschemathings related to the DDL related objects like Table, Column, CreateIndex, etc.sqluse casenot really a feature or a bug; can be support for new DB features or user use cases not anticipated

    Type

    No type

    Projects

    No projects

    Relationships

    None yet

      Development

      No branches or pull requests

        Participants

        @zzzeek@jvanasco@sigmavirus24@sqla-tester@CaselIT

        Issue actions

          Support for Inspecting/Reflecting PostgreSQL column defaults using GENERATED · Issue #5324 · sqlalchemy/sqlalchemy