Skip to content

Exception while using asyncio extension with Python 3.6 and contextvars module #6166

Closed
@grossmj

Description

@grossmj

Describe the bug

I am running some tests using pytest and I got the following exception from sqlalchemy with Python 3.6 only when the contextvars module is installed: AttributeError: This Python interpreter does not support context variables

I understand Context Variables is a new feature introduced in Python 3.7 and then I noticed one of my project dependency, httpx, installed this backport of Context Variables: https://pypi.org/project/contextvars/

Expected behavior

I expect to run the script below using Python 3.6 with the contextvars module installed.

To Reproduce

Setup a virtual env and install the dependencies:

python3.6 -m venv python3.6-venv
source python3.6-venv/bin/activate
python3 -m pip install pytest==6.2.2 pytest-asyncio==0.14.0 sqlalchemy==1.4.4 aiosqlite===0.17.0 contextvars==2.4

Create a new script test.py containing this code:

import pytest
from sqlalchemy.ext.asyncio import AsyncSession, create_async_engine


@pytest.fixture
async def db_session():

    engine = create_async_engine(
        "sqlite+aiosqlite:///:memory:",
        connect_args={"check_same_thread": False},
        future=True
    )

    session = AsyncSession(engine)
    try:
        yield session
    finally:
        await session.close()


@pytest.mark.asyncio
async def test_db_session(db_session):

    assert db_session

Launch the test with python3 -m pytest test.py

Error

===================================================================================== test session starts ======================================================================================
platform linux -- Python 3.6.13, pytest-6.2.2, py-1.10.0, pluggy-0.13.1
rootdir: /home/grossmj/PycharmProjects/gns3-server, configfile: pytest.ini
plugins: asyncio-0.14.0, timeout-1.4.2
collected 1 item                                                                                                                                                                               

test.py .E                                                                                                                                                                               [100%]

============================================================================================ ERRORS ============================================================================================
_____________________________________________________________________________ ERROR at teardown of test_db_session _____________________________________________________________________________

    @pytest.fixture
    async def db_session():
    
        engine = create_async_engine(
            "sqlite+aiosqlite:///:memory:",
            connect_args={"check_same_thread": False},
            future=True
        )
    
        session = AsyncSession(engine)
        try:
            yield session
        finally:
>           await session.close()

test.py:19: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
python3.6-venv/lib/python3.6/site-packages/sqlalchemy/ext/asyncio/session.py:326: in close
    return await greenlet_spawn(self.sync_session.close)
python3.6-venv/lib/python3.6/site-packages/sqlalchemy/util/_concurrency_py3k.py:96: in greenlet_spawn
    context = _AsyncIoGreenlet(fn, greenlet.getcurrent())
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <_AsyncIoGreenlet object at 0x7fa40a7ba638 (otid=0x7fa40b5094c8) pending>, fn = <bound method Session.close of <sqlalchemy.orm.session.Session object at 0x7fa40c4cb5f8>>
driver = <greenlet.greenlet object at 0x7fa40b492768 (otid=0x7fa40c4fdf78) current active started main>

    def __init__(self, fn, driver):
        greenlet.greenlet.__init__(self, fn, driver)
        self.driver = driver
        if _copy_context is not None:
>           self.gr_context = _copy_context()
E           AttributeError: This Python interpreter does not support context variables

python3.6-venv/lib/python3.6/site-packages/sqlalchemy/util/_concurrency_py3k.py:32: AttributeError
-------------------------------------------------------------------------------------- Captured log setup --------------------------------------------------------------------------------------
DEBUG    asyncio:selector_events.py:54 Using selector: EpollSelector
=================================================================================== short test summary info ====================================================================================
ERROR test.py::test_db_session - AttributeError: This Python interpreter does not support context variables
================================================================================== 1 passed, 1 error in 0.17s ==================================================================================

Versions.

  • OS: Linux Ubuntu 20.04 LTS
  • Python: 3.6
  • SQLAlchemy: 1.4.4
  • Database: sqlite
  • DBAPI: aiosqlite

Activity

added
bugSomething isn't working
external library/application issuesa separate library / application that's not SQLAlchemy has a problem (dependent or dependee)
and removed
requires triageNew issue that requires categorization
on Mar 31, 2021
added this to the 1.4.x milestone on Mar 31, 2021
zzzeek

zzzeek commented on Mar 31, 2021

@zzzeek
Member

this issue stems from the PR for greenlet at python-greenlet/greenlet#198 . Since you are on a very old Python version you can work around this by pinning greenlet to 0.4.17.

Demonstration of the issue. Even though this uses the third party contextvars directly, greenlet's compatibility check is specific to the Python version in use so this only reproduces on an older Python

import contextvars
from greenlet import greenlet

def test1():
    gr1.gr_context = contextvars.copy_context()


gr1 = greenlet(test1)
gr1.switch()
$ .venv/bin/python test.py 
Traceback (most recent call last):
  File "test.py", line 9, in <module>
    gr1.switch()
  File "test.py", line 5, in test1
    gr1.gr_context = contextvars.copy_context()
AttributeError: This Python interpreter does not support context variables

for our end we'd need to do a py3 compatibility check.

added
easya one / two liner type of thing that anyone can do in short order. also see "fairly easy"
on Mar 31, 2021
zzzeek

zzzeek commented on Mar 31, 2021

@zzzeek
Member

patch:

diff --git a/lib/sqlalchemy/util/_concurrency_py3k.py b/lib/sqlalchemy/util/_concurrency_py3k.py
index 94f4705d8..cb7255727 100644
--- a/lib/sqlalchemy/util/_concurrency_py3k.py
+++ b/lib/sqlalchemy/util/_concurrency_py3k.py
@@ -6,19 +6,22 @@ from typing import Coroutine
 
 import greenlet
 
+from . import compat
 from .. import exc
 
-try:
-    from contextvars import copy_context as _copy_context
-
-    # If greenlet.gr_context is present in current version of greenlet,
-    # it will be set with a copy of the current context on creation.
-    # Refs: https://github.com/python-greenlet/greenlet/pull/198
-    getattr(greenlet.greenlet, "gr_context")
-except (ImportError, AttributeError):
+if compat.py37:
+    try:
+        from contextvars import copy_context as _copy_context
+
+        # If greenlet.gr_context is present in current version of greenlet,
+        # it will be set with a copy of the current context on creation.
+        # Refs: https://github.com/python-greenlet/greenlet/pull/198
+        getattr(greenlet.greenlet, "gr_context")
+    except (ImportError, AttributeError):
+        _copy_context = None
+else:
     _copy_context = None
 
-
 # implementation based on snaury gist at
 # https://gist.github.com/snaury/202bf4f22c41ca34e56297bae5f33fef
 # Issue for context: https://github.com/python-greenlet/greenlet/issues/173
grossmj

grossmj commented on Mar 31, 2021

@grossmj
Author

this issue stems from the PR for greenlet at python-greenlet/greenlet#198 . Since you are on a very old Python version you can work around this by pinning greenlet to 0.4.17.

Thanks for the work around, it solved my problem 👍

sqla-tester

sqla-tester commented on Apr 1, 2021

@sqla-tester
Collaborator

Federico Caselli has proposed a fix for this issue in the master branch:

Prevent loading contextvars in python 3.6 https://gerrit.sqlalchemy.org/c/sqlalchemy/sqlalchemy/+/2705

self-assigned this
on Apr 1, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

asynciobugSomething isn't workingeasya one / two liner type of thing that anyone can do in short order. also see "fairly easy"external library/application issuesa separate library / application that's not SQLAlchemy has a problem (dependent or dependee)

Type

No type

Projects

No projects

Relationships

None yet

    Development

    No branches or pull requests

      Participants

      @zzzeek@grossmj@sqla-tester@CaselIT

      Issue actions

        Exception while using asyncio extension with Python 3.6 and contextvars module · Issue #6166 · sqlalchemy/sqlalchemy