Skip to content

postgresql+psycopg2: scalar_subquery with bindparam in insert returns only one value when inserting multiple rows with SQLAlchemy 2.0.0b1 #8639

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
samahav opened this issue Oct 15, 2022 · 3 comments
Labels
bug Something isn't working great mcve An issue with a great mcve insertmanyvalues insertmanyvalues feature near-term release addition to the milestone which indicates this should be in a near-term release regression something worked and was broken by a change sql
Milestone

Comments

@samahav
Copy link

samahav commented Oct 15, 2022

Describe the bug

I am very excited for SQL Alchemy 2.0 so I ran some projects with the first beta release. With PostgreSQL (postgresql+psycopg2 dialect) a scalar subquery with bindparam in insert returns only one value per batch instead of one value per row when inserting multiple rows.

The code example works great against MySQL database (mysql+pymysql dialect).

To Reproduce

from sqlalchemy import (Column, ForeignKey, Integer, String, bindparam,
                        create_engine, insert, select)
from sqlalchemy.engine.url import URL
from sqlalchemy.orm import declarative_base, relationship

Base = declarative_base()


class ProductClass(Base):
    __tablename__ = "product_class"
    id = Column(Integer, primary_key=True)
    product_class_name = Column(String(length=255), nullable=False)


class Product(Base):
    __tablename__ = "product"
    id = Column(Integer, primary_key=True)
    product_name = Column(String(length=255), nullable=False)
    product_class_id = Column(Integer, ForeignKey("product_class.id"), nullable=False)
    product_class = relationship("ProductClass")


engine = create_engine(
    URL.create(
        drivername="postgresql+psycopg2",
        username="postgres",
        password="postgres",
        host="localhost",
        port=5432,
        database="test_db",
    ),
    future=True,
    echo=True,
)

Base.metadata.drop_all(bind=engine)
Base.metadata.create_all(bind=engine)


with engine.begin() as conn:
    conn.execute(
        insert(ProductClass),
        [
            {"id": 1, "product_class_name": "soda"},
            {"id": 2, "product_class_name": "candy"},
        ],
    )

# we're supposed to get product_class_id
#     1 for "coca-cola"
#     2 for "chocolate"
insert_stmnt = insert(Product).values(
    product_class_id=select(ProductClass.id)
    .where(ProductClass.product_class_name == bindparam("product_class_name"))
    .scalar_subquery()
)
insert_rows = [
    {"product_class_name": "soda", "product_name": "coca-cola"},
    {"product_class_name": "candy", "product_name": "chocolate"},
]
with engine.begin() as conn:
    conn.execute(insert_stmnt, insert_rows)

# let's check the results
select_stmnt = select(Product.product_name, ProductClass.product_class_name).join(
    ProductClass
)
with engine.begin() as conn:
    result = conn.execute(select_stmnt)
    result_rows = [
        {"product_class_name": row.product_class_name, "product_name": row.product_name}
        for row in result
    ]

if insert_rows == result_rows:
    print("Everything is fine :)")
else:
    print("Something is seriously wrong!")
    print("Rows from database:")
    print(result_rows)

Error

There is no error, the code runs fine, but the inserted data is not fine. We're supposed to have different product_class_id for coca-cola and soda. Now we have product_class_id = 1 for both of them!

Versions

  • OS: Ubuntu 20.04
  • Python: 3.10.7
  • SQLAlchemy: 2.0.0b1 (there is no problem with sqlalchemy==1.4.41)
  • Database: PostgreSQL 15.0, had the same issue with 13.4
  • DBAPI: postgresql+psycopg2 (psycopg2-binary==2.9.4)

Additional context

No response

@samahav samahav added the requires triage New issue that requires categorization label Oct 15, 2022
@zzzeek zzzeek added bug Something isn't working sql regression something worked and was broken by a change near-term release addition to the milestone which indicates this should be in a near-term release insertmanyvalues insertmanyvalues feature and removed requires triage New issue that requires categorization labels Oct 15, 2022
@zzzeek zzzeek modified the milestones: 2.0 final, 2.0beta2 Oct 15, 2022
@zzzeek
Copy link
Member

zzzeek commented Oct 15, 2022

thanks! our first regression

psycopg2's fast execution helper feature used in 1.4 can successfully render this, so we can keep using insertmanyvalues if we can fix the rendering for this:

 INSERT INTO product (product_name, product_class_id) VALUES ('coca-cola', (SELECT product_class.id 
	FROM product_class 
	WHERE product_class.product_class_name = 'soda')),('chocolate', (SELECT product_class.id 
	FROM product_class 
	WHERE product_class.product_class_name = 'candy'))

@zzzeek zzzeek added the great mcve An issue with a great mcve label Oct 15, 2022
@sqla-tester
Copy link
Collaborator

Mike Bayer has proposed a fix for this issue in the main branch:

accommodate arbitrary embedded params in insertmanyvalues https://gerrit.sqlalchemy.org/c/sqlalchemy/sqlalchemy/+/4130

@zzzeek
Copy link
Member

zzzeek commented Oct 15, 2022

OK that was hard and this patch might need some more work

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working great mcve An issue with a great mcve insertmanyvalues insertmanyvalues feature near-term release addition to the milestone which indicates this should be in a near-term release regression something worked and was broken by a change sql
Projects
None yet
Development

No branches or pull requests

3 participants