Skip to content

sqlite backend can't add datetimes before 1900 #968

Closed
@sqlalchemy-bot

Description

@sqlalchemy-bot
Collaborator

Migrated issue, originally created by Anonymous

In databases/sqlite.py, the DateTimeMixin class is converting datetime values to a string representation using their strftime() method.

Unfortunately, Python's strftime doesn't handle dates before 1900. This results in the below exception when you attempt to add such a date:

ValueError: year=1850 is before 1900; the datetime strftime() methods require year >= 1900

As a workaround, I changed this to convert value to a datetime (in case it was a date), and then used value.isoformat(" ") instead, which does seem to handle these dates, but I'm guessing this isn't a proper fix, as it looks like !format! is designed to be changable.


Attachments: sqlite.pre1900.20080916.patch | sqlite_test.py | sqlite_date_pre1900.patch

Activity

sqlalchemy-bot

sqlalchemy-bot commented on Feb 14, 2008

@sqlalchemy-bot
CollaboratorAuthor

Michael Bayer (@zzzeek) wrote:

the __format__ is not supremely important and we can move to isoformat() on this one - can you attach a patch ?

sqlalchemy-bot

sqlalchemy-bot commented on Feb 14, 2008

@sqlalchemy-bot
CollaboratorAuthor

Changes by Michael Bayer (@zzzeek):

  • set milestone to "0.4.xx"
sqlalchemy-bot

sqlalchemy-bot commented on Feb 15, 2008

@sqlalchemy-bot
CollaboratorAuthor

Anonymous wrote:

Deal with pre-1900 dates in sqlite

sqlalchemy-bot

sqlalchemy-bot commented on Feb 15, 2008

@sqlalchemy-bot
CollaboratorAuthor

Changes by Anonymous:

  • attached file sqlite_date_pre1900.patch
sqlalchemy-bot

sqlalchemy-bot commented on Feb 15, 2008

@sqlalchemy-bot
CollaboratorAuthor

Anonymous wrote:

OK. I've added a patch.

sqlalchemy-bot

sqlalchemy-bot commented on Sep 12, 2008

@sqlalchemy-bot
CollaboratorAuthor

Anonymous wrote:

Any ideas when this patch will get applied. We use a lot of pre 1900 dates with sqlite and it is a real pain for each developer to have to apply this patch (which definitely works btw!).

sqlalchemy-bot

sqlalchemy-bot commented on Sep 12, 2008

@sqlalchemy-bot
CollaboratorAuthor

Michael Bayer (@zzzeek) wrote:

sorry, this one fell off the radar....im gathering 6 months is too long huh ? (just kidding). I'll take a look at it now.

sqlalchemy-bot

sqlalchemy-bot commented on Sep 12, 2008

@sqlalchemy-bot
CollaboratorAuthor

Michael Bayer (@zzzeek) wrote:

There's problems with this patch, in that it doesn't maintain backwards compatibility with the existing approach, which has to remain identical throughout 0.4 (and also 0.5 where we changed the microseconds format). Subsequent versions of SQLA within a major version must work perfectly with existing SQLite databases created by a previous version.

If I pass a datetime.date to a DateTime type, I'll get "YYYY-MM-DD" due to the conditional, when currently I'd get "YYYY-MM-DD 00:00:00". Similar behavior for datetime.time objects. So we need to keep the output format nailed to the specific TypeEngine type, regardless of the type of object passed. It suggests that each type would have to marshal the given object into its own type (such as a time becomes a datetime as needed) before using isoformat() - or alternatively (and actually this is what I would probably do), we just re-implement strftime() with a regular expression that encodes numerically.

Keep in mind that you can continue to workaround this issue by just supplying your own DateTime types which use isoformat().

sqlalchemy-bot

sqlalchemy-bot commented on Sep 12, 2008

@sqlalchemy-bot
CollaboratorAuthor

Changes by Michael Bayer (@zzzeek):

  • added labels: high priority
sqlalchemy-bot

sqlalchemy-bot commented on Sep 12, 2008

@sqlalchemy-bot
CollaboratorAuthor

Michael Bayer (@zzzeek) wrote:

like, here's a start at a strftime replacement:

import datetime
import re

expr = re.compile(r'%(\w)')
repl = {
    "Y":lambda dt: "%4.4d" % dt.year,
    "m":lambda dt: "%2.2d" % dt.month,
    "d":lambda dt: "%2.2d" % dt.day,
    "H":lambda dt: "%2.2d" % dt.hour,
    "M":lambda dt: "%2.2d" % dt.minute,
    "S":lambda dt: "%2.2d" % dt.second,
    "c":lambda dt: '%06d' % dt.microsecond
}
legacy_repl = repl.copy()
legacy_repl['c']('c') = lambda dt: str(dt.microsecond)

def strftime(dt, format):
    return _strftime(dt, format, repl)

def legacy_strftime(dt, format):
    return _strftime(dt, format, legacy_repl)

def _strftime(dt, format, conversions):
    def go(m):
        return conversions[m.group(1)](m.group(1))(dt)
    return expr.sub(go, format)
    
format = "%Y-%m-%d %H:%M:%S.%c"

print strftime(datetime.datetime(2008, 12, 15, 10, 15, 27, 450), format)
print strftime(datetime.datetime(1890, 12, 15, 10, 15, 27, 450), format)

the above can be dropped in straight into 0.4 and 0.5, and have its own independent set of tests so that there's little impact on current code (mostly that we maintain the microsecond logic).

sqlalchemy-bot

sqlalchemy-bot commented on Sep 16, 2008

@sqlalchemy-bot
CollaboratorAuthor

Anonymous wrote:

New patch for pre1900 dates

sqlalchemy-bot

sqlalchemy-bot commented on Sep 16, 2008

@sqlalchemy-bot
CollaboratorAuthor

Changes by Anonymous:

  • attached file sqlite.pre1900.20080916.patch
sqlalchemy-bot

sqlalchemy-bot commented on Sep 16, 2008

@sqlalchemy-bot
CollaboratorAuthor

Anonymous wrote:

Tests for new date patch

sqlalchemy-bot

sqlalchemy-bot commented on Sep 16, 2008

@sqlalchemy-bot
CollaboratorAuthor

Changes by Anonymous:

  • attached file sqlite_test.py
sqlalchemy-bot

sqlalchemy-bot commented on Sep 16, 2008

@sqlalchemy-bot
CollaboratorAuthor

Anonymous wrote:

Commenter from 2008-09-12 again. I've taken your suggested strftime function, turned it into a class (with a few mods) and integrated it with the current (0.4.7p1) code where it works fine. I've attached the resulting patch along with a basic set of tests.

8 remaining items

Loading
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Relationships

    None yet

      Development

      No branches or pull requests

        Participants

        @sqlalchemy-bot

        Issue actions

          sqlite backend can't add datetimes before 1900 · Issue #968 · sqlalchemy/sqlalchemy