Skip to content

Allow usage of Values as a column element in postgres #6289

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
CaselIT opened this issue Apr 15, 2021 · 11 comments
Closed

Allow usage of Values as a column element in postgres #6289

CaselIT opened this issue Apr 15, 2021 · 11 comments
Assignees
Labels
postgresql public api change that require a stub package update sql use case not really a feature or a bug; can be support for new DB features or user use cases not anticipated
Milestone

Comments

@CaselIT
Copy link
Member

CaselIT commented Apr 15, 2021

Is your feature request related to a problem? Please describe.
Postgresql supports using VALUES as a column element, in query of the type

select bar from foo where bar in (values (1),(2),(42));
select bar from foo where bar = any (values (1),(2),(42));

Describe the solution you'd like
it would be nice if these two queries could be generated using

sa.select(t).where(t.c.bar.in_(sa.values(t.c.bar).data([[1],[2],[42]])))
sa.select(t).where(t.c.bar == sa.any_(sa.values(t.c.bar).data([[1],[2],[42]])))

Describe alternatives you've considered
the first one produces

SELECT bar
FROM t, (VALUES (:param_1), (:param_2), (:param_3))
WHERE bar IN (SELECT bar
FROM (VALUES (:param_4), (:param_5), (:param_6)))

that does not work.
While the second raises the exception:

SQL expression element expected, got <sqlalchemy.sql.selectable.Values object at 0x0000017717604AF0>. To create a column expression from a FROM clause row as a whole, use the .table_valued() method.

Using calling table_valued() does not produce the above query, generating intead

SELECT t.bar
FROM t, (VALUES (%(param_1)s), (%(param_2)s), (%(param_3)s)) AS x (bar)
WHERE x.bar = ANY (x)

this results in an error, at least on pg

Additional context
Since the current values is a from element, maybe this should be a pg only thing, that's not related to the global values?
The current compile basically works so it could be reused for also for this new construct

Have a nice day!

@CaselIT CaselIT added postgresql sql use case not really a feature or a bug; can be support for new DB features or user use cases not anticipated labels Apr 15, 2021
@zzzeek
Copy link
Member

zzzeek commented Apr 15, 2021

this probalby needs a new coercions role that is similar to IN impl. Also any_() all_() should accept any SELECT statement just like in_() does. Adding a method like .scalar() to the Values construct, with a datatype that's Tuple(<types>), would be a start.

@CaselIT CaselIT added this to the 1.4.x milestone Apr 15, 2021
@CaselIT
Copy link
Member Author

CaselIT commented Apr 15, 2021

Actually also the IN case does not work as expected. It's coerced to an hybrid between a tolumn valued and a subquery. I've updated the example above

if we want to use the same construct, and I'm not sure we should, I think .scalar should generate another type, that's not a from element.

Then I guess we can decide if in and any/all should coerce using this .scalar or raise

@zzzeek
Copy link
Member

zzzeek commented Apr 15, 2021

any_() / all_() should coerce to this .scalar() thing because that's what in_() does right now as well:

https://github.com/sqlalchemy/sqlalchemy/blob/master/lib/sqlalchemy/sql/coercions.py#L555

@CaselIT
Copy link
Member Author

CaselIT commented Apr 15, 2021

ok, but it does not really work that coercion in this case. I've corrected the sql above, but it renders this sql

SELECT bar
FROM t, (VALUES (:param_1), (:param_2), (:param_3))
WHERE bar IN (SELECT bar
FROM (VALUES (:param_4), (:param_5), (:param_6)))

@zzzeek
Copy link
Member

zzzeek commented Apr 15, 2021

we need a new coercion role to Do The Right Thing on this one.

@CaselIT
Copy link
Member Author

CaselIT commented Apr 15, 2021

I think I get the gist, I'll try giving a go after the two other regressions.

Values.scalar() will returns a new object that's called something like ScalarValue that is not a from element but a column/clause element. does it make sense?

@zzzeek
Copy link
Member

zzzeek commented Apr 15, 2021

yeah somehtiugn like. this one seemed like it didnt have any clear way to do it right now so if we get something up we can tweak it

@CaselIT
Copy link
Member Author

CaselIT commented Apr 15, 2021

I'm not sure this can be generalized.
On postgres at least this is very delicate: this query works

-- 1
SELECT bar
FROM t
WHERE bar IN (SELECT anon_2 FROM (VALUES (1), (2), (42)) as anon_1(anon_2));

SELECT bar
FROM t
WHERE bar = ANY(SELECT anon_2 FROM (VALUES (1), (2), (42)) as anon_1(anon_2)) -- or s/ANY/ALL/

this instead complain that it needs cast

-- 2
SELECT bar
FROM t
WHERE bar IN (SELECT anon_1 FROM (VALUES (1), (2), (42)) as anon_1);

SELECT bar
FROM t
WHERE bar = ANY(SELECT anon_1 FROM (VALUES (1), (2), (42)) as anon_1) -- or s/ANY/ALL/

any/all behave the same as in. so 1 ok, 2 not ok.

Calling scalar should instead render as:

SELECT bar
FROM t
WHERE bar IN (VALUES (1), (2), (42));

SELECT bar
FROM t
WHERE bar = ANY(VALUES (1), (2), (42)) -- or s/ANY/ALL/

@zzzeek
Copy link
Member

zzzeek commented Apr 15, 2021

but you mean to be using ALL/ANY in those sql examples right

@CaselIT
Copy link
Member Author

CaselIT commented Apr 15, 2021

they actually work the same with both. Updated examples above

@CaselIT CaselIT self-assigned this Aug 26, 2021
@CaselIT CaselIT added the public api change that require a stub package update label Sep 2, 2021
@zzzeek zzzeek modified the milestones: 1.4.x, 2.x.x Mar 28, 2022
@sqla-tester
Copy link
Collaborator

Federico Caselli has proposed a fix for this issue in the main branch:

Implement ScalarValue https://gerrit.sqlalchemy.org/c/sqlalchemy/sqlalchemy/+/4183

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
postgresql public api change that require a stub package update sql use case not really a feature or a bug; can be support for new DB features or user use cases not anticipated
Projects
None yet
Development

No branches or pull requests

3 participants