Skip to content

TypeAlias cannot be found in type_annotation_map #11370

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
RazerM opened this issue May 8, 2024 · 9 comments
Closed

TypeAlias cannot be found in type_annotation_map #11370

RazerM opened this issue May 8, 2024 · 9 comments
Labels
bug Something isn't working orm - annotated declarative issues with the new annotations-based declarative ORM approach
Milestone
2.1

Comments

@RazerM
Copy link
Contributor

RazerM commented May 8, 2024

Describe the bug

I have a TypeAlias that I'm trying to put in type_annotation_map but SQLAlchemy can't seem to find it.

(my naive understanding was that Json in the example would be looked up directly by hash in the annotation map)

Optional link from https://docs.sqlalchemy.org which documents the behavior that is expected

No response

SQLAlchemy Version in Use

2.0.30

DBAPI (i.e. the database driver)

psycopg2

Database Vendor and Major Version

PostgreSQL 12

Python Version

3.10

Operating system

macOS

To Reproduce

from typing import Annotated, Dict, List, Optional, TypeAlias, Union

from sqlalchemy.dialects.postgresql import JSONB
from sqlalchemy.orm import DeclarativeBase, Mapped, mapped_column

JsonPrimitive: TypeAlias = Union[str, int, float, bool, None]
JsonObject: TypeAlias = Dict[str, "Json"]
JsonArray: TypeAlias = List["Json"]
Json: TypeAlias = Union[JsonObject, JsonArray, JsonPrimitive]


class Base(DeclarativeBase):
    type_annotation_map = {
        Json: JSONB(none_as_null=True),
    }


class Model(Base):
    __tablename__ = "model"

    id: Mapped[int] = mapped_column(primary_key=True)
    data: Mapped[Optional[Json]]

Error

Traceback (most recent call last):
  File "/scratch_184.py", line 25, in <module>
    class Model(Base):
  File "/venv/lib/python3.10/site-packages/sqlalchemy/orm/decl_api.py", line 836, in __init_subclass__
    _as_declarative(cls._sa_registry, cls, cls.__dict__)
  File "/venv/lib/python3.10/site-packages/sqlalchemy/orm/decl_base.py", line 244, in _as_declarative
    return _MapperConfig.setup_mapping(registry, cls, dict_, None, {})
  File "/venv/lib/python3.10/site-packages/sqlalchemy/orm/decl_base.py", line 325, in setup_mapping
    return _ClassScanMapperConfig(
  File "/venv/lib/python3.10/site-packages/sqlalchemy/orm/decl_base.py", line 571, in __init__
    self._extract_mappable_attributes()
  File "/venv/lib/python3.10/site-packages/sqlalchemy/orm/decl_base.py", line 1541, in _extract_mappable_attributes
    value.declarative_scan(
  File "/venv/lib/python3.10/site-packages/sqlalchemy/orm/properties.py", line 709, in declarative_scan
    self._init_column_for_annotation(
  File "/venv/lib/python3.10/site-packages/sqlalchemy/orm/properties.py", line 880, in _init_column_for_annotation
    raise sa_exc.ArgumentError(
sqlalchemy.exc.ArgumentError: Could not locate SQLAlchemy Core type for Python type typing.Union[str, int, bool, float, typing.Dict[str, ForwardRef('Json')], typing.List[ForwardRef('Json')]] inside the 'data' attribute Mapped annotation

Additional context

There's still an error even if the TypeAlias is not self-referential:

JsonPrimitive: TypeAlias = Union[str, int, float, bool, None]
JsonObject: TypeAlias = Dict[str, object]
JsonArray: TypeAlias = List[object]
Json: TypeAlias = Union[JsonObject, JsonArray, JsonPrimitive]

but weirdly it works if JsonPrimitive doesn't have None:

JsonPrimitive: TypeAlias = Union[str, int, float, bool]
JsonObject: TypeAlias = Dict[str, object]
JsonArray: TypeAlias = List[object]
Json: TypeAlias = Union[JsonArray, str, int, float]

(both of these types aren't very useful, so I would specify the JSONB explicitly in mapped_column, but I thought it's worth mentioning what I'd tried).
@RazerM RazerM added the requires triage New issue that requires categorization label May 8, 2024
@zzzeek zzzeek added bug Something isn't working near-term release addition to the milestone which indicates this should be in a near-term release orm - annotated declarative issues with the new annotations-based declarative ORM approach and removed requires triage New issue that requires categorization labels May 8, 2024
@zzzeek
Copy link
Member

zzzeek commented May 8, 2024

we probably cant use Unions as keys, kind of an edge case

@zzzeek zzzeek added this to the 2.0.x milestone May 8, 2024
@CaselIT
Copy link
Member

CaselIT commented May 8, 2024 •
edited
Loading

I think the issue here is this:

JsonPrimitive: TypeAlias = Union[str, int, float, bool, None]
JsonObject: TypeAlias = Dict[str, object]
JsonArray: TypeAlias = List[object]
Json: TypeAlias = Union[JsonObject, JsonArray, JsonPrimitive]

print(Optional[Json] == Json)

This prints True. The repr is also the same

@zzzeek I think the current implementation removes the None from the union when searching for type.
It usually makes sense, but not in this case.

@zzzeek
Copy link
Member

zzzeek commented Aug 1, 2024

do we have a patch for this? near term label is for stuff we have a fix set up for. this one is in that intricate place w/ the type guessing etc

@zzzeek zzzeek added PRs (with tests!) welcome a fix or feature which is appropriate to be implemented by volunteers and removed near-term release addition to the milestone which indicates this should be in a near-term release labels Aug 1, 2024
@CaselIT
Copy link
Member

CaselIT commented Aug 1, 2024 •
edited
Loading

I'm not sure it can be fixed, since changing the current logic would change the current documented behaviour.

The main issue here is that we have an union that includes None in the column type, since this is a json column and there are two null supported: json-null and sql-null. This conflicts with the logic used to detect if a column is nullable, meaning that in the type map we look for the union-without-none.

Overall this case is undecidable, so I would flag it as won't fix, maybe adding that special case to the error message / linked error link.

RazerM added a commit to RazerM/sqlalchemy that referenced this issue Oct 1, 2024
@RazerM
Permit optional types in type annotation map
78bef6b 
Fixes sqlalchemy#11370
@RazerM RazerM mentioned this issue Oct 1, 2024
Closed
3 tasks
@sqla-tester
Copy link
Collaborator

Frazer McLean has proposed a fix for this issue in the main branch:

Permit optional types in type annotation map https://gerrit.sqlalchemy.org/c/sqlalchemy/sqlalchemy/+/5513

@zzzeek zzzeek mentioned this issue Oct 2, 2024
Closed
@zzzeek zzzeek modified the milestones: 2.0.x, 2.1 Oct 2, 2024
@zzzeek zzzeek removed the PRs (with tests!) welcome a fix or feature which is appropriate to be implemented by volunteers label Oct 2, 2024
@zzzeek
Copy link
Member

zzzeek commented Oct 2, 2024

this can't be cleanly fixed until we fix the architecture that's broken in #11944

RazerM added a commit to RazerM/sqlalchemy that referenced this issue Oct 3, 2024
@RazerM
Permit optional types in type annotation map
f226440 
Fixes sqlalchemy#11370
@sqla-tester
Copy link
Collaborator

Frazer McLean has proposed a fix for this issue in the main branch:

General improvement on annotated declarative https://gerrit.sqlalchemy.org/c/sqlalchemy/sqlalchemy/+/5513

@sqla-tester
Copy link
Collaborator

Frazer McLean has proposed a fix for this issue in the main branch:

Permit optional types in type annotation map https://gerrit.sqlalchemy.org/c/sqlalchemy/sqlalchemy/+/5513

@sqla-tester
Copy link
Collaborator

Frazer McLean has proposed a fix for this issue in the rel_2_0 branch:

Permit optional types in type annotation map https://gerrit.sqlalchemy.org/c/sqlalchemy/sqlalchemy/+/5600

sqlalchemy-bot pushed a commit that referenced this issue Dec 11, 2024
@RazerM @zzzeek
dont match partial types in type_annotation_map
Loading
Loading status checks…
8a45f13 
Fixed issue regarding ``Union`` types that would be present in the
:paramref:`_orm.registry.type_annotation_map` of a :class:`_orm.registry`
or declarative base class, where a ``Mapped[]`` element that included one
of the subtypes present in that ``Union`` would be matched to that entry,
potentially ignoring other entries that matched exactly.   The correct
behavior now takes place such that an entry should only match in
``type_annotation_map`` exactly, as a ``Union`` type is a self-contained
type. For example, an attribute with ``Mapped[float]`` would previously
match to a ``type_annotation_map`` entry ``Union[float, Decimal]``; this
will no longer match and will now only match to an entry that states
``float``. Pull request courtesy Frazer McLean.

Fixes #11370
Closes: #11942
Pull-request: #11942
Pull-request-sha: 21a3d19

Change-Id: I3467be00f8fa8bd011dd4805a77a3b80ff74a215
(cherry picked from commit 40c30ec)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug Something isn't working orm - annotated declarative issues with the new annotations-based declarative ORM approach
Projects
None yet
Milestone
2.1
Development

Successfully merging a pull request may close this issue.

Permit optional types in type annotation map RazerM/sqlalchemy
4 participants
@zzzeek @RazerM @sqla-tester @CaselIT