Skip to content

thread race in lazy loader _simple_lazy_clause + baked #4507

Closed
@zzzeek

Description

@zzzeek
Member

the baked query gets a hold of _simple_lazy_clause, if another thread re-runs it, the new state is incompatible. POC below.

import random
import threading
import time

from sqlalchemy import Column
from sqlalchemy import create_engine
from sqlalchemy import ForeignKey
from sqlalchemy import Integer
from sqlalchemy import String
from sqlalchemy.ext.declarative import declarative_base
from sqlalchemy.orm import relationship
from sqlalchemy.orm import Session

Base = declarative_base()


class A(Base):
    __tablename__ = 'a'

    id = Column(Integer, primary_key=True)
    data = Column(String(10))
    bs = relationship("B", ) #bake_queries=False)  # turn off baked to fix


class B(Base):
    __tablename__ = 'b'
    id = Column(Integer, primary_key=True)
    a_id = Column(ForeignKey("a.id"))
    data = Column(String(10))

e = create_engine("mysql://scott:tiger@localhost/test", echo="debug")
Base.metadata.create_all(e)

s = Session(e)

s.add(A(bs=[B()]))
s.commit()

def worker():
    s = Session(e)
    a1 = s.query(A).first()
    while True:
        time.sleep(.001)
        bs = a1.bs
        assert len(bs) == 1

        s.expire(a1, ['bs'])

def chaos():
    while True:
        time.sleep(random.random())
        try:
            del A.bs.property._lazy_strategy._simple_lazy_clause
        except AttributeError:
            pass

threads = [threading.Thread(target=chaos)]  # or remove chaos to fix
threads.extend(threading.Thread(target=worker) for i in range(20))

for t in threads:
    t.start()

for t in threads:
    t.join()

Activity

added this to the 1.2.x milestone on Feb 20, 2019
sqla-tester

sqla-tester commented on Feb 20, 2019

@sqla-tester
Collaborator

Mike Bayer has proposed a fix for this issue in the master branch:

Ensure _simple_lazy_clause bind names are fixed before cloning https://gerrit.sqlalchemy.org/1140

sqla-tester

sqla-tester commented on Feb 20, 2019

@sqla-tester
Collaborator

Mike Bayer has proposed a fix for this issue in the rel_1_2 branch:

Ensure _simple_lazy_clause bind names are fixed before cloning https://gerrit.sqlalchemy.org/1141

added a commit that references this issue on Feb 20, 2019
4d2bd4a
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Relationships

    None yet

      Development

      No branches or pull requests

        Participants

        @zzzeek@sqla-tester

        Issue actions

          thread race in lazy loader _simple_lazy_clause + baked · Issue #4507 · sqlalchemy/sqlalchemy