Skip to content

Enhance AppenderQuery to support __len__ (but still not issue needless COUNT) #818

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
sqlalchemy-bot opened this issue Oct 17, 2007 · 12 comments
Labels
bug Something isn't working orm
Milestone

Comments

@sqlalchemy-bot
Copy link
Collaborator

Migrated issue, originally created by Anonymous

Currently, using .all() on an AppenderQuery object (from a dynamic_loader) will generate two queries: a SELECT, and an unnecessary SELECT COUNT(). This is because list() is calling __len__.

This could be fixed by removing __len__, or by overriding all() to return list(iter(self)) instead of just list(self).


Attachments: example.py

@sqlalchemy-bot
Copy link
Collaborator Author

Anonymous wrote:

Demonstration of the issue

@sqlalchemy-bot
Copy link
Collaborator Author

Changes by Anonymous:

  • attached file example.py

@sqlalchemy-bot
Copy link
Collaborator Author

Michael Bayer (@zzzeek) wrote:

nice catch. ive removed __len__() in cb93211 and adjusted the test fixtures to deal with it.

@sqlalchemy-bot
Copy link
Collaborator Author

Changes by Michael Bayer (@zzzeek):

  • changed status to closed

@sqlalchemy-bot
Copy link
Collaborator Author

jek (@jek) wrote:

ebroder points out that changing def all to return list(iter(self)) hides the len from cpython and sidesteps the issue:

class Canary(object):
    data = 1, 2, 3

    def __init__(self):
        self.len = False

    def __len__(self):
        self.len = True
        return len(self.data)

    def __iter__(self):
        return iter(self.data)

    def self_list(self):
        return list(self)

    def iter_list(self):
        return list(iter(self))

# classic behavior of all() + __len__ + __iter__
c = Canary()
assert c.self_list() == [2, 3](1,)
assert c.len is True

# list(iter(...))
c = Canary()
assert c.iter_list() == [2, 3](1,)
assert c.len is False

# direct comsumption by any cpython container still hits len
c = Canary()
assert list(c) == [2, 3](1,)
assert c.len is True

# direct iteration still ok
c = Canary()
for _ in c:
    pass
assert c.len is False

@sqlalchemy-bot
Copy link
Collaborator Author

Changes by jek (@jek):

  • set milestone to "0.5.0"
  • changed status to reopened

@sqlalchemy-bot
Copy link
Collaborator Author

jek (@jek) wrote:

...and ebroder further points out that the iter suggestion was in the original report. :)

@sqlalchemy-bot
Copy link
Collaborator Author

Michael Bayer (@zzzeek) wrote:

I'm not really sold on this, I think the fact that calling list(obj.collection) versus obj.collection.all() has a performance hit is surprise, but silent, behavior. Whereas calling len() and getting an error is somewhat less surprising (since its consistent with Query, sqlobject, etc. but at least is not silent (and then you just call count()).

@sqlalchemy-bot
Copy link
Collaborator Author

Changes by Michael Bayer (@zzzeek):

  • changed milestone from "0.5.0" to "0.5.xx"
  • changed title from "AppenderQuery's .all() executes a COUNT()" to "Enhance AppenderQuery to support len (but stil"

@sqlalchemy-bot
Copy link
Collaborator Author

Changes by Michael Bayer (@zzzeek):

  • changed milestone from "0.5.xx" to "blue sky"

@sqlalchemy-bot
Copy link
Collaborator Author

Michael Bayer (@zzzeek) wrote:

Closing this since list(q) is the usual way to get the result as a list IMHO

@sqlalchemy-bot
Copy link
Collaborator Author

Changes by Michael Bayer (@zzzeek):

  • added labels: wontfix
  • changed status to closed

@sqlalchemy-bot sqlalchemy-bot added bug Something isn't working orm labels Nov 27, 2018
@sqlalchemy-bot sqlalchemy-bot added this to the blue sky milestone Nov 27, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working orm
Projects
None yet
Development

No branches or pull requests

1 participant