Small. Fast. Reliable.
Choose any three.
Status color codes: Active Bugs Active Issues Fixed Tested Deferred Closed
# Type Status Created By Subsys Changed Assigned Svr Pri Title  
Description
Remarks
 
2917 code active 2008 Feb anonymous   2008 Feb   4 4 Tcl interface - busy callback confusion script/procedure edit
In the Tcl interface the "busy" method doesn't work if a script is supplied instead of a procedure:

  %  package req sqlite3
  3.5.1
  % sqlite3 db hl7journal.db
  %  db busy {puts busy; expr 0}
  %  db eval {select count(*) from incoming}
  busy
  database is locked

Here the callback script is only invoked once, even though it returns 0. If we put this code in a procedure it works as desired/expected:

  % proc b args {puts busy; after 1000; expr 0}
  % db  busy b
  % db eval {select count(*) from incoming}
  busy
  busy
  busy
  ^C
2008-Feb-01 12:31:45 by anonymous:
After researching this a little it appears this happens because the busy callback is invoked with an extra argument. The extra argument leads to an error but that error is only visible through errorInfo, not the result.

I humbly suggest the following changes:

* mention the extra argument in the documentation of the Tcl interface

* forward the error from the busy callback to Tcl (replacing the "database is locked")

* enhance errorInfo to make the invokation of the busy callback apparent.

Currently, I'm getting this errorInfo:

  % db busy {puts hi; after 1000; return 0}
  %  db eval "select count(*) from incoming"
  hi
  database is locked
  %  set errorInfo
  bad option "0": must be -code, -errorcode, or -errorinfo
      while executing
  "return 0 0"
      invoked from within
  "db eval "select count(*) from incoming""

It would be nicer to get something like

  bad option "0": must be -code, -errorcode, or -errorinfo
      while executing
  "return 0 0"
      from busy callback
  "puts hi; after 1000; return 0 0"
      invoked from within
  "db eval "select count(*) from incoming""
 
2916 code active 2008 Feb anonymous   2008 Feb   1 1 sqlitedll-3_5_5.zip is older 3.5.4 binary edit
sqlitedll-3_5_5.zip in download section is same with old 3.5.4 binary.
2008-Feb-01 12:13:04 by anonymous:
Yes , I can confirm it
 
2228 new active 2007 Feb anonymous   2008 Feb   3 4 horizontal partitioning edit
I would love to see a feature in the next version of sqlite that includes horizontal table partitioning. I see it currently in MySQL Version 5.1 beta. It would be awesome to see it in SQLite
2008-Feb-01 02:46:29 by anonymous:
Well it would be great for us - won't need to emulate it. Any thoughts about that?

That's not only about large data sets - you can also make it for some sort of horizontal ATTACH of small separate pieces.

 
2915 build active 2008 Feb pweilbacher   2008 Feb   4 3 fix Makefile for platforms that need .exe extension edit
There are some targets that need a $(TEXE) added in Makefile.in. Otherwise a platform that needs a .exe extension cannot run e.g. tests.
 
2914 code active 2008 Jan anonymous   2008 Jan   3 3 ATTACH returns SQLITE_ERROR when it means SQLITE_BUSY edit
I'm seeing the same behavior as in #2096, with SQLite 3.5.4. ATTACH DATABASE fails with SQLITE_ERROR rather than SQLITE_BUSY when the database to be attached, or the main database of the connection being attached to, is EXCLUSIVE-locked by another database connection. For added confusion, sqlite3_errmsg() says "database is locked" when the ATTACH is done via sqlite3_exec(), but "SQL logic error or missing database" when the ATTACH is done via sqlite3_step().

As a result of this bug, it is difficult to distinguish between fatal and transient ATTACH errors, particularly when sqlite3_step() is used.

I am attaching a test program that demonstrates the problem.

 
2905 code active 2008 Jan anonymous   2008 Jan pweilbacher 2 2 mutex_os2.c - incorrect mutex implementation edit
The OS/2 version of sqlite3_mutex_alloc() is badly broken.

It creates named mutexes which, by design, are global rather than process-specific as intended. This might be minimally acceptable except that the function reuses the same name every time it attempts to create a SQLITE_MUTEX_FAST or SQLITE_MUTEX_RECURSIVE.

The result is that every call returns the exact same semaphore to every thread in every process using sqlite3. Once this mutex is owned by one process, other processes calling sqlite3_mutex_enter() will be blocked. Much the same is true for the static mutexes. Every process ends up using the exact same SQLITE_MUTEX_STATIC_MASTER, SQLITE_MUTEX_STATIC_MEM, etc.

There's another flaw that is fairly minor compared with the above: in an attempt to avoid concurrency when creating the static mutexes, this function uses an API call that is thoroughly deprecated.

The attached patch remedies all of these issues. Since the logic that protects the creation of the static mutexes may not be self-evident, here's an explanation:

The existence (or non-existence) of a given named mutex is itself a semaphore. If the isInit flag is false, the code attempts to create a mutex whose name is unique to that process. If the attempt is successful, there are two possibilities: (1) either the current thread is the first to reach this code & may proceed; (2) or while the current thread was making its preparations, another thread created the mutex, did the init, then closed the mutex. Testing isInit immediately after creating the mutex determines which possibility is valid.

If mutex creation fails due to a duplicate name, then another thread is currently performing the init. In this case, the current thread simply has to wait a while until the other thread is done & isInit becomes true.

Submitted by Rich Walsh (richATe-vertise.com)

 
2902 code active 2008 Jan anonymous   2008 Jan drh 3 3 Add watch support to SQLite edit
SQLite currently provides only TRIGGERs and the update_hook() as a way for applications to stay informed about changes to the database. But both of these alternatives do not provide enough details about the actual changes to the underlying database file(s).

We've prepared a patch for SQLite 3.5.x to allow applications to install a watch_hook into the database, that will be invoked everytime the database is changed with exact details about the change that was performed.

2008-Jan-22 16:06:59 by anonymous:
Great idea and nice job. This functionality is very useful.


2008-Jan-31 18:27:16 by anonymous:
Any chance to get this committed for the next release (i.e. 3.5.6)?


2008-Jan-31 19:38:35 by drh:
Unlikely, for two reasons:

  1. I am unconvinced that this patch solves a problem that needs solving. It is vitally important to a project like SQLite that we work to avoid clutter and cruft. That means that any change must have a compelling rational or else it is rejected.

  2. The patches are against version 3.5.4. There were many changes to the core for 3.5.5 and the patches no longer work.


2008-Jan-31 20:58:55 by anonymous:
We can (and will) port the changes to 3.5.5, so the second point will be done. First the first point, I'm not sure how many projects will actually need this functionality, but I guess there are quite a lot of projects that would benefit, and for the others, there's zero overhead due to this patch.


2008-Jan-31 21:21:30 by drh:
There is a lot of overhead for me because if I accept this patch, that means I have to maintain it forever. Most of the work is in maintenance, not coming up with the original patch.
 
2912 new active 2008 Jan anonymous   2008 Jan   5 4 merge join edit
Would it be possible to implement merge joins?
 
2903 code active 2008 Jan anonymous   2008 Jan   2 2 tclinstaller.tcl fails on path and permissions issue edit
When compiling using custom PREFIX, pointing to private directory, tclinstaller.tcl fails, because it tries to remove contents from /usr/share/tcl8.4/sqlite3.

./configure --prefix=/my/private/sqlite/sqlite-3.5.4
... # success
make
... # success
make install
...
tclsh ./tclinstaller.tcl 3.5
error deleting "/usr/share/tcl8.4/sqlite3": not owner
    while executing
"file delete -force $LIBDIR/sqlite3"
    (file "./tclinstaller.tcl" line 17)
make: *** [tcl_install] Error 1
I've found two work-arounds:

  1. If you run make install as root.
  2. If you use ./configure --disable-tcl
2008-Jan-28 17:47:39 by anonymous:
I also ran into this problem.

make install as root will end up copying files into the system's library directory and is almost certainly not what you want if you specified your own --prefix.

 
2911 code active 2008 Jan anonymous   2008 Jan   2 2 Adding parentheses to a FROM clause edit
Hi,

Parentheses in a FROM statement seem to mess with the ability to use table aliases in the "what" part. Here is an example:

Start SQLite:

    $ sqlite3 employee.db
    SQLite version 3.5.4
    Enter ".help" for instructions

create a couple of tables and populate them with test data:

    sqlite> create table person (id integer, name text, employerid integer);
    sqlite> create table employer (id integer, name text);
    sqlite> insert into person (id, name, employerid) values (1, "Dave", 1);
    sqlite> insert into employer (id, name) values (1, "ACME");

Run a simple query with no parentheses in the FROM statement:

    sqlite> select b.id from person as a inner join employer as b on a.employerid = b.id;
    1

Everything works as expected. Now, repeat that query with parentheses:

    sqlite> select b.id from (person as a inner join employer as b on a.employerid = b.id);
    SQL error: no such column: b.id

There you have it.

This may be related to ticket #1822, although that ticket deals with aliases and subqueries. This problem seems to be more fundamental.

Many thanks,

-- Dave

 
2909 build active 2008 Jan aswift   2008 Jan aswift 4 4 Omit the _XOPEN_SOURCE 500 define on Mac OS-X (check for __APPLE__) edit
sqliteInt.h attempts to avoid setting _XOPEN_SOURCE on MacOSX by checking if __DARWIN__ is defined, however on Leopard (as of Mac OS X 10.5) this test fails and should be enhanced by testing for both __DARWIN__ and __APPLE__

Building sqlite succeeds with _XOPEN_SOURCE defined, but it causes _POSIX_C_SOURCE to be defined which leads to failing to define F_FULLFSYNC which prevents use of the fullfsync pragma.

 
2613 code active 2007 Sep anonymous   2008 Jan drh 3 3 replace doesn't work with blobs containing \x0, otherwise it does edit
The replace expression function does not work with blobs in case of contained zero terminator character; but it does if there is not this special character included. I expected the function to work similar like substr with blob-safety in case of type is blob only. X'nnnn' is of type blob, so following example should have returned a blob type result X'0102FF0405' in the 2nd and 3rd line. How to get to this result?
SQLite version 3.4.2
Enter ".help" for instructions
sqlite> select hex(replace(X'0102030405',X'03',X'FF'));
0102FF0405
sqlite> select hex(replace(X'0102000405',X'00',X'FF'));

sqlite> select typeof(replace(X'0102000405',X'00',X'FF'));
null
sqlite>
2007-Sep-03 04:21:12 by anonymous:
Replace was designed to work with strings. However, working with blobs would be an interesting extension.


2007-Oct-18 06:13:10 by anonymous:
I've seen a similar situation where I can't reliably store stings with nulls in the middle of them as TEXT.

I can convert them to blobs, in which case length(...) works correctly. I if convert them back to strings, length(...) treats them as C-strings.

Is this the expected behavior? I notice the entire column is preserved even when it's has TEXT affinity, I can append data to it as a string, cast back to a blob and see everything (am I explaining this poorly?)

This all seems a bit counter intuitive in some ways. Perhaps strings shouldn't treat NULL characters as special?


2007-Oct-27 16:45:41 by anonymous:
Treatment of length operator is - as fas as I know - dependent on type:
As text it is the length number of UTF-8 characters and as blob it is the number of bytes. As long as all the UTF-8 characters out of the lower half ASCII char-set (127 of them), this is identical beside the fact of different 0-terminator interpretation.
To append is something different than using the replace operator. My suggestion would be to make the replace operator work with bytes (not UTF-8) in case of all 3 parameters are of type blob.
Another suggestion: the UTF aware functions are Private declared and not usable from within a loadable extension dll/so. This should be changed.


2008-Jan-28 19:36:39 by anonymous:
Will there come a solution for this with the next release? It is really not fair to handle a blob only like text which cannot contain a zero terminator. With this unique useful function a zero-containing blob could be formed into a normal text string without loosing the part behind the zero terminator. It would be really a step forward without too much effort.
 
2907 code active 2008 Jan anonymous   2008 Jan   1 1 Issues of sqlite3 with Windows Mobile 5/6 edit
hi. we are currently using sqlite3 for our mobile application. it has been running without a hitch on pocket pc 2003 and previous versions. come windows mobile 5 and 6 we have been getting errors, although not consistent yet. one example is 'EXCEPTION_DATATYPE_MISALIGNMENT'. another is 'SELECT STATMENTS TO THE LEFT AND RIGHT OF UNION ARE NOT EQUAL'. i was wondering if you have any known compatibility issues of your product with this version of windows mobile.

thanks in advance.

2008-Jan-28 13:26:26 by anonymous:
EXCEPTION_DATATYPE_MISALIGNMENT is thrown when you try to use and Odd pointer address. I wrote a custom allocator for WinCE/ARM platform, and I have to take care about memory alignment (I used to align at 2 bytes, and at that time it solved the problem)
 
2908 code active 2008 Jan anonymous   2008 Jan   3 3 Add support to examine whether statements modify the database edit
Currently there is no way to check whether a compiled statement will modify the database when being executed. Of course, there is the work-around of misusing the authorizer callback for this purpose, but this is kinda error prone and causes quite some overhead for such a simple purpose.
 
2898 code active 2008 Jan anonymous   2008 Jan   1 1 Latest CVS for 3.5.4 fails to build test1.c edit
gcc -pipe -O3 -g -Wall -DSQLITE_DISABLE_DIRSYNC=1 -I. -I../src -DNDEBUG -I/usr/include -DSQLITE_THREADSAFE=1 -DSQLITE_THREAD_OVERRIDE_LOCK=1 -DSQLITE_OMIT_LOAD_EXTENSION=1 -DTCLSH=1 -DSQLITE_TEST=1 -DSQLITE_CRASH_TEST=1 -DSQLITE_NO_SYNC=1 -DTEMP_STORE=1 -o .libs/testfixture ../src/attach.c ../src/btree.c ../src/build.c ../src/date.c ../src/expr.c ../src/func.c ../src/insert.c ../src/malloc.c ../src/os.c ../src/os_os2.c ../src/os_unix.c ../src/os_win.c ../src/pager.c ../src/pragma.c ../src/prepare.c ../src/printf.c ../src/select.c ../src/test1.c ../src/test2.c ../src/test3.c ../src/test4.c ../src/test5.c ../src/test6.c ../src/test7.c ../src/test8.c ../src/test9.c ../src/test_autoext.c ../src/test_async.c ../src/test_btree.c ../src/test_config.c ../src/test_hexio.c ../src/test_malloc.c ../src/test_md5.c ../src/test_onefile.c ../src/test_schema.c ../src/test_server.c ../src/test_tclvar.c ../src/test_thread.c ../src/tokenize.c ../src/utf.c ../src/util.c ../src/vdbe.c ../src/vdbeapi.c ../src/vdbeaux.c ../src/vdbemem.c ../src/where.c parse.c ../src/tclsqlite.c ./.libs/libsqlite3.so -L/usr/lib64 -ltcl8.4 -ldl -lpthread -lieee -lm -Wl,--rpath -Wl,/common/pkgs/sqlite-3.5.4.3/lib ../src/build.c: In function 'sqlite3RefillIndex': ../src/build.c:2275: warning: cast to pointer from integer of different size ../src/func.c: In function 'trimFunc': ../src/func.c:919: warning: cast from pointer to integer of different size ../src/func.c: In function 'sqlite3RegisterBuiltinFunctions': ../src/func.c:1464: warning: cast to pointer from integer of different size ../src/func.c:1483: warning: cast to pointer from integer of different size ../src/insert.c: In function 'sqlite3GenerateConstraintChecks': ../src/insert.c:1200: warning: cast to pointer from integer of different size ../src/insert.c:1034: warning: 'j2' may be used uninitialized in this function ../src/insert.c: In function 'sqlite3Insert': ../src/insert.c:373: warning: 'regFromSelect' may be used uninitialized in this function ../src/test1.c: In function 'test_collate_func': ../src/test1.c:2085: warning: cast from pointer to integer of different size ../src/test1.c: In function 'test_collate_needed_cb': ../src/test1.c:2209: warning: cast to pointer from integer of different size ../src/test1.c: In function 'alignmentCollFunc': ../src/test1.c:2258: warning: cast from pointer to integer of different size ../src/test1.c:2259: warning: cast from pointer to integer of different size ../src/test8.c: In function 'echoBestIndex': ../src/test8.c:722: warning: 'nRow' may be used uninitialized in this function ../src/vdbe.c: In function 'sqlite3VdbeExec': ../src/vdbe.c:502: warning: 'pOut' may be used uninitialized in this function ../src/vdbe.c:501: warning: 'pIn3' may be used uninitialized in this function ../src/vdbe.c:501: warning: 'pIn2' may be used uninitialized in this function ../src/vdbe.c:501: warning: 'pIn1' may be used uninitialized in this function ../src/vdbeaux.c: In function 'sqlite3VdbeChangeP4': ../src/vdbeaux.c:529: warning: cast from pointer to integer of different size ../src/vdbemem.c: In function 'sqlite3ValueText': ../src/vdbemem.c:911: warning: cast from pointer to integer of different size /tmp/ccsuOeus.o: In function `reset_prng_state': /build/work/sqlite-3.5.4.3/bld/../src/test1.c:4280: undefined reference to `sqlite3ResetPrngState' /tmp/ccsuOeus.o: In function `restore_prng_state': /build/work/sqlite-3.5.4.3/bld/../src/test1.c:4267: undefined reference to `sqlite3RestorePrngState' /tmp/ccsuOeus.o: In function `save_prng_state': /build/work/sqlite-3.5.4.3/bld/../src/test1.c:4254: undefined reference to `sqlite3SavePrngState' collect2: ld returned 1 exit status make: *** [testfixture] Error 1
2008-Jan-17 23:54:58 by anonymous:
Problem appears to be here in libsqlite.3.so.0.8.6 as shown by:

   nm -A .libs/libsqlite3.so.0.8.6 | grep sqlite3ResetPrngState

which shows no entry point. And:

   nm -A .libs/random.o  | grep sqlite3ResetPrngState

which also shows no entry point.


2008-Jan-17 23:56:55 by anonymous:
Ah... It appears -DSQLITE_TEST should be passed when building test1.c and left off when building prior to install.


2008-Jan-21 20:16:00 by anonymous:
In the makefile the right flag appears to be set, it's just not making it through to the compile for some reason.


2008-Jan-21 20:16:24 by anonymous:
Still fails the same based on today's cvs update.


2008-Jan-23 03:14:49 by anonymous:
This bug fixed as of latest cvs pull
 
2901 code active 2008 Jan anonymous   2008 Jan   3 3 ROLLBACK and COMMIT statements should not expire edit
Currently, whenever a statement changes the schema of the database, all prepared statements will be expired, no matter whether they actually need to be prepared again or not. This is especially problematic for ROLLBACK statements in a multi-statement transaction. Currently there is no way to guaranty that a multi-statement transaction can at least be rolled back in case of an error, because one has to (re)prepare the ROLLBACK statement to roll back the transaction, which can fail because of OOM (in a multi-threaded application).
 
2506 new active 2007 Jul anonymous   2008 Jan   3 2 New API to retrieve ROWID from SQLite3_stmt structire edit
Is it too much trouble to allow an API to retrieve ROWID for non-aggeregate queries directly from SQLite3_stmt structire? It would be very useful to create updatable non aggregate query results for situations when actually internal PK (ROWID) is not gived explicitly in SQL statement nor actual table's PK (if any).
SELECT queries that join two or more tables together would be a problem also.


2007-Jul-16 16:51:18 by anonymous:
It's more of a multi-step process. First you have to enumerate the open cursors on the sqlite3_stmt object. Then you need to resolve the table each cursor goes to, and then fetch the rowid for each active cursor. Of course this may get confusing when you've joined a table onto itself.


2008-Jan-19 10:32:59 by anonymous:
This is far to old active ticket. Is it in consideration to be implemented in the near future?
 
2899 code active 2008 Jan anonymous   2008 Jan   4 4 sqlite3_reset() after exec() takes > 100 ms to complete edit
I'm not entirely sure whether this is a bug or not... I'm not familiar enough with SQLite to know if this is way too unconventional, but I noticed today that running exec() in conjunction with a prepared statement really kills the performance of sqlite3_reset(), if it's called after exec():

// init sqlite3_prepare_v2( db,
"SELECT [file_id],[file_name],[file_mime],[file_type],"
"[file_size],[date_created],[date_accessed],[data] "
"FROM file_cache WHERE [file_id] = ? LIMIT 1;",
-1, &sqQueryStatement, &unused );

// query function -- called repeatedly (usually second or third run starts to cause big delays)

/* begin function */

sqlite3_reset( sqQueryStatement );
sqlite3_bind_int( sqQuerystatement, 1, 292 );
sqlite3_step( sqQueryStatement );

bla = sqlite3_column_int( sqQueryStatement, 0 );

/* ... */

char *erm;

sqlite3_exec( db, "UPDATE file_cache SET [date_accessed] = DATETIME( 'NOW', 'LOCALTIME' ) WHERE [file_id] = 292", NULL, NULL, &erm );

/* ... */

sqlite3_clear_bindings( sqQueryStatement );

/* end function */

If sqlite3_exec() is commented out, consecutive calls to the function run in less than a millisecond. With sqlite3_exec() included, sqlite3_reset() call in this function takes > 100 ms to complete.

Tested: Windows Vista Ultimate 32 bit, Visual Studio 2005 8.0.50727.876

 
2885 code active 2008 Jan anonymous   2008 Jan   4 4 (minor) fulltest failures edit
Minor noise in fulltest on a 64-bit machine (everything seems to work otherwise though):

  4 errors out of 61527 tests
  Failures on these tests: exclusive-ioerr-2.280.4
  exclusive-ioerr-2.281.4 exclusive-ioerr-2.282.4 incrvacuum-ioerr-1.31.4
  All memory allocations freed - no leaks

details:

  exclusive-ioerr-2.280.3... Ok
  exclusive-ioerr-2.280.4...
  Expected: [8ee59fe0b5bc391ecc5002539379b063]
       Got: [35e56178ad878809d4789c5797265a23]
  exclusive-ioerr-2.281.1... Ok
  exclusive-ioerr-2.281.2... Ok
  exclusive-ioerr-2.281.3... Ok
  exclusive-ioerr-2.281.4...
  Expected: [95762fb35ef83ddba65f681325346ef2]
       Got: [1c20c63d975ad85b395a5e7701d785c2]
  exclusive-ioerr-2.282.1... Ok
  exclusive-ioerr-2.282.2... Ok
  exclusive-ioerr-2.282.3... Ok
  exclusive-ioerr-2.282.4...
  Expected: [2c5ea7db8424f38d17cdff41056da0e0]
       Got: [c02841c40d3e6e0579922745bd9c0260]
  exclusive-ioerr-2.283.1... Ok

[...]

    incrvacuum-ioerr-1.31.4...
    Expected: [ea55042a449c3b4759730a882e8271a0]
	 Got: [9951814984696d0811cacbe862060af8]
    incrvacuum-ioerr-1.32.1... Ok
2008-Jan-19 00:56:30 by anonymous:
A test from 20 minutes ago passes cleanly. This could be closed.
 
2878 code active 2008 Jan anonymous   2008 Jan   1 1 Memory leaks with latest CVS [4693] edit
This SQL leaks memory with CVS [4693] :

  CREATE TABLE x(id integer primary key, a TEXT NULL);
  INSERT INTO x (a) VALUES ('first');
  CREATE TABLE tempx(id integer primary key, a TEXT NULL);
  INSERT INTO tempx (a) VALUES ('t-first');
  CREATE VIEW tv1 AS SELECT x.id, tx.id FROM x JOIN tempx tx ON tx.id=x.id;

One leak is caused by "CREATE TABLE tempx", a second one by "CREATE VIEW tv1".

The above SQL is a digest of select7.test, select7-2.1.

2008-Jan-14 17:51:11 by anonymous:
I have tested again with CVS [4711] and it does no longer show the original leaks.

I therefore consider this issue fixed and I will now close this ticket.


2008-Jan-14 23:56:34 by anonymous:
Doing a fulltest with -MSQLITE_MEMDEUG I see reports of memory leaks. I assume these are of little or minimal interest at present because of the amount of code flux.

If you do want details, please let me know (I'll recheck this ticket tomorrow I guess).


2008-Jan-15 08:23:56 by anonymous:
Thanks for the follow-up. I am not running the original test-suite but have have ported a great number of them to Delphi.

If you could just let me know which tests caused the leaks you fixed in [4712] I'd be more than glad the port these test as well and let you know my findings.


2008-Jan-19 00:56:17 by anonymous:
A test from 20 minutes ago passes cleanly. This could be closed.
 
2897 code active 2008 Jan anonymous   2008 Jan   1 1 String or BLOB exceed size limit edit
This error was shown after attemp to read script from SQLite 3.5.4 shell in order to recreate old DB. Details:

1. Database was created with SQLite 3.3.4. Around 20 standard fieds and one BLOB.

2. The only one existed table was dumped with shell of SQLite 3.5.4. SQL script seems to be coorrect.

3. Opened SQLite 3.5.4 and read script in new DB. The error "String or BLOB exceed size limit" are sown for several lines. Many records missing.

4. Attempted to dump table with shell of version 3.3.6 (have no more 3.3.4 shell) and read into new DB with 3.5.4 shell The same errors are shown.

The same steps was attempted with 3.3.6. shell only. All seems to be correct.

2008-Jan-17 20:23:25 by drh:
This size limit on BLOBs in SQLite version 3.5.4 is 1GB. How big is your blob, exactly?


2008-Jan-17 22:22:24 by anonymous:
BLOB in each record is no more than few MB. Mostly it is few KB (e-client and news application). Whole DB have around 200MB.


2008-Jan-18 02:28:11 by drh:
This issue is probably resolved by check-in [4636] , then.


2008-Jan-18 14:28:13 by anonymous:
If directive SQLITE_MAX_SQL_LENGTH is not defined it is set to 1,000,000 (10^6) in amalgamation code of 3.5.4.
 
2892 new active 2008 Jan anonymous   2008 Jan   5 5 There should be a way in the api to read more precise error message. edit
Currently all errors in sql queries issued via C,C++ api result in error code SQLITE_ERROR = 1 and message "SQL error or missing database".

That leaves user completely clueless what mistake in his sql he actually made.

Some evidence of confusion can be found here: http://bugs.php.net/bug.php?id=33117

2008-Jan-16 17:41:00 by drh:
SQLite gives detailed error information for SQL syntax or logic errors. (Try, for example, entering invalid SQL into the CLI.) I think perhaps that PHP is simply failing to to access those errors and is instead picking up some other error indication from someplace else.


2008-Jan-16 21:59:17 by anonymous:
Well I shall retest it, but I got the same error message upon a duplicate key in Delphi. Although I'm aware of an enhanced api to get the errormessage.


2008-Jan-17 00:05:58 by drh:
What error message do you get from the CLI?
 
2893 code active 2008 Jan anonymous   2008 Jan   1 1 incorrect integer range tests edit
recently a function that performs integer range tests was added to the cvs (check-in [4706] ), but if i am correct there is a problem in the return value of the function in the file vdbemem.c:

  static i64 doubleToInt64(double r){
    ...
    if( r<(double)minInt ){
      return minInt;
    }else if( r>(double)maxInt ){
      return minInt;     <-- is this correct, shouldn't it be maxInt?
    }else{
      return (i64)r;
    }
  }
2008-Jan-16 17:33:56 by drh:
See the remarks on ticket #2280. The code duplicates the behavior of the FPU on x86.


2008-Jan-16 18:21:28 by anonymous:
did you mean ticket #2880? didn't read that ticket before, but since there was no comment regarding that behavior in the function it seemed (to my eyes) that it was a mistake.

maybe adding a small comment in there would clarify this issue


2008-Jan-16 18:39:42 by anonymous:
Just because the double to int overflow behavior happens to be that way with GCC on x86, is it desirable?
 
2886 code active 2008 Jan anonymous   2008 Jan   3 3 testfixture: -fPIC needed when building extension(s) edit
(this fix/change is probably needed in older versions too, i meant to send this in earlier)

-fPIC is needed when building extensions (some platforms don't need this or don't care --- x86-64 does)

  diff --git a/test/loadext.test b/test/loadext.test
  index 81e152f..2a7fa2e 100644
  --- a/test/loadext.test
  +++ b/test/loadext.test
  @@ -64,7 +64,7 @@ if {![file exists $testextension]} {
     set srcdir [file dir $testdir]/src
     set testextsrc $srcdir/test_loadext.c
     if {[catch {
  -    exec gcc -Wall -I$srcdir -I. -g -shared $testextsrc -o $testextension
  +    exec gcc -Wall -fPIC -I$srcdir -I. -g -shared $testextsrc -o $testextension
     } msg]} {
       puts "Skipping loadext tests: Test extension not built..."
       puts $msg
 
2882 code active 2008 Jan anonymous   2008 Jan   3 3 fulltest failure: ./testfixture: wrong # args: should be "cksum db" edit
  exclusive-ioerr-2.2.1... Ok

  ./testfixture: wrong # args: should be "cksum db"
      while executing
  "ifcapable vacuum {
    do_ioerr_test ioerr-2 -cksum true -sqlprep {
      BEGIN;
      CREATE TABLE t1(a, b, c);
      INSERT INTO t1 VALUES(1, randstr(50,..."
      (file "../test/ioerr.test" line 58)
      invoked from within
  "source $testdir/ioerr.test"
      (file "../test/exclusive3.test" line 50)
      invoked from within
  "source $testfile"
      ("foreach" body line 5)
      invoked from within
  "foreach testfile [lsort -dictionary [glob $testdir/*.test]] {
      set tail [file tail $testfile]
      if {[lsearch -exact $EXCLUDE $tail]>=0} continue
  ..."
      ("for" body line 7)
      invoked from within
  "for {set Counter 0} {$Counter<$COUNT && $nErr==0} {incr Counter} {
    if {$Counter%2} {
      set ::SETUP_SQL {PRAGMA default_synchronous=off;}
    } else ..."
      (file "..//test/all.test" line 85)
2008-Jan-14 23:25:49 by anonymous:
The latest code seems to have fixed this. I would close this but I don't see how to do that.
 
2884 new active 2008 Jan anonymous   2008 Jan   4 4 Way to find out limits edit
The page http://www.sqlite.org/limits.html shows some limits that SQLite has. They are consts in the code, and defaults are listed, but there doesn't seem to be any way to find out what values a particular binary was built with.

For example, I've gotten SQLite as part of my OS or another program, and I don't know what value of SQLITE_MAX_COLUMN was used to compile this SQLite. I'm trying to track down a bug in my code, and if this value was too small, that would explain it. (In truth, it's probably not the cause, but I'd like to rule it out.)

I'd like a way to print the value of these limits at runtime. It doesn't have to have a stable API -- it would mostly be useful for debugging. If the command-line client had a command ".limits" that printed all of these values, that would be super.

 
489 new active 2003 Nov anonymous   2008 Jan   4 4 DLL exports suggestion edit
Just a suggestion:

I'm building SQLite using MS C++, and an easily maintained alternative to a .def file for the DLL exports is to incorporate the following into the sqlite.h header...


	#ifdef _MSC_VER
		#ifdef SQLITE_EXPORTS
			#define SQLITE_API __declspec(dllexport)
		#else
			#define SQLITE_API __declspec(dllimport)
		#endif
	#else
		#define SQLITE_API
	#endif

	// example function declaration
	SQLITE_API void sqlite_close(sqlite *);


SQLITE_EXPORTS is defined when building the library, but not when building client applications.

I don't know if other compilers have similar methods of defining library exports, but if so, this could possibly be extended to support them.

2008-Jan-11 01:58:27 by anonymous:
There is a good reason for adding this feature in some form other than merely avoiding the manual import library creation step.

When __declspec(dllimport) is used, it is a hint to MSVC++ to produce more efficient code. This allows the function in the DLL to be called in a single call. Otherwise there is an extra jmp (2 in Debug mode). So those that want maximum performance out of C and DLL combo should take note. I patched this into my copy of the source.

But the code given in the ticket assumes that the library is used as a DLL and not statically linked in which case dllimport shouldn't be used.

-- Bz


2008-Jan-11 13:28:27 by drh:
The prefix SQLITE_API appears in front of all interfaces in the amalgamation. So it seems like this problem could be solved by adding

    -DSQLITE_API=__declspec(dllexport)

to the compiler command line. No?


2008-Jan-12 07:16:44 by anonymous:
Yes -DSQLITE_API will do, and also it is necessary to define SQLITE_EXTERN because some data symbols:

sqlite3_version[]

sqlite3_temp_directory

sqlite3_io_trace

are forward declared with SQLITE_EXTERN but defined with SQLITE_API and these must match.

But like I mentioned, this is only half of the issue. The other half is the small performance boost from __declspec(dllimport). It would be nice to have SQLITE_API prefixes in sqlite3.h for users of the DLL. It's low priority, but I thought I would point this out.

Another thing to note is that using __declspec(dllexport) instead of DEF file along with the (non default) stdcall convention (callee cleans up the stack) will result in function names in DLL export table being mangled with numeric suffixes. If one links through the auto generated import lib file, this isn't an issue, but it affects looking up a function name with GetProcAddress(). So the DEF file (the original intent of the ticket before I hijacked it) has a benefit in this case. But to use the stdcall convention properly there would have to be yet another prefix for the benefit of clients using the default cdecl convention:

#define SQLITE_CALL

#define SQLITE_CALL __stdcall

SQLITE_API int SQLITE_CALL sqlite3_open( const char *filename, sqlite3 **ppDb );

The 2 prefixes is what MS does with their API functions. I assume there is some small performance boost from __stdcall.

-- Bz


2008-Jan-13 02:02:38 by sdwilsh:
See also Ticket #2448
 
2881 build active 2008 Jan anonymous   2008 Jan   1 1 Latest sqlite-3.5.4 build fail on latest Fedora 2.6.23.12-52.fc7 edit
Two test cases fail.

  io-4.1...
  Expected: [3]
     Got: [2]
  io-4.2.1... Ok
  io-4.2.2... Ok
  io-4.2.3...
  Expected: [3]
     Got: [2]
  io-4.3.1... Ok

Let me know how to run individual test cases and how this might be fixed.

Here's how I built sqlite using latest CVS. If something is wrong here, let me know and I'll rebuild/retest.

I'm building on latest Fedora fc7.

Thanks.

_______

  net1#uname -a
  Linux net1.coolsurf.com 2.6.23.12-52.fc7 #1 SMP Tue Dec 18 20:27:10 EST 2007 x86_64 x86_64 x86_64 GNU/Linux

  net1#build_sqlite
  mkdir -p /build/work/sqlite-3.5.4
  cd /build/work/sqlite-3.5.4
  unset CDPATH
  export CFLAGS='-pipe -O3 -g -DSQLITE_DISABLE_DIRSYNC=1 -Wall'
  rm -rf bld
  cvs -d :pserver:anonymous@www.sqlite.org:/sqlite -r update .
  mkdir bld
  cd bld
  ../configure --prefix=/common/pkgs/sqlite-3.5.4 --enable-tcl --with-tcl=/usr/lib64 --enable-threadsafe --enable-threads-override-locks
  make
  groupadd vuser || /bin/true
  useradd -M -g vuser -d /vhost/davidfavor.com/users/david -s /bin/zsh   david  || /bin/true
  useradd -M -g vuser -d /vhost/livefeast.com/users/yemiah -s /bin/zsh yemiah || /bin/true
  chown david:vuser -R ..
  su -c "make test" david
2008-Jan-11 17:18:52 by anonymous:
Same tests still fail with CVS of today around 11AM CST.


2008-Jan-11 17:41:55 by drh:
FWIW, both those test cases pass on SuSE 10.1. I do not understand why they are failing on Fedora. But in any event, the tests in question are verifying logic that implements an optimization that is not used on Fedora, ever. So the failures are of no consequence. If those are the only two tests that fail, then you can safely use the build for whatever it is you are trying to do.


2008-Jan-11 19:16:19 by anonymous:
Failures when 'make fulltest' built with CFLAGS of

  '-pipe -O3 -g -Wall -DSQLITE_DISABLE_DIRSYNC=1 -DSQLITE_MEMDEBUG'

   exclusive-malloc-1.transient.746...make: *** [fulltest]
      Segmentation fault

Failures when 'make fulltest' built with CFLAGS of

   '-pipe -O3 -g -Wall -DSQLITE_DISABLE_DIRSYNC=1'

   Skipping malloc tests: not compiled with -DSQLITE_MEMDEBUG...
   6 errors out of 61998 tests
   Failures on these tests: exclusive-ioerr-2.280.4
      exclusive-ioerr-2.281.4 exclusive-ioerr-2.282.4
      incrvacuum-ioerr-1.31.4 io-4.1 io-4.2.3
   All memory allocations freed - no leaks
   Maximum memory usage: 14376554 bytes

Pre 3.5.x builds work fine on Fedora. If you're open to debugging all these, I'd like to go through and resolve all these one by one, so Fedora has a clean build/fulltest.

Please let me know how to run each test individually and I'll try to figure out the problem with each.

Thanks.

 
2847 new active 2007 Dec anonymous   2008 Jan   5 4 Include major, minor, and patch version numbers in sqlite.h edit
Hi, I'm working on a project where sqlite is being compiled into a DLL. Currently, sqlite.h makes the version number available as both an x.y.z string and an integer value in the form of x*1000000 + y*1000 + z.

Unfortunately, neither of these options works particularly well when trying to create a resource file so that the DLL can display the proper version information within Windows. I've tried many different ways of disassembling the integer version number, but limitations in the resource compiler unfortunately prevent them from working.

As a result, I've been forced for the time being to define SQLITE_VERSION_MAJOR, SQLITE_VERSION_MINOR, and SQLITE_VERSION_PATCH with manually-given values of x, y, and z respectively in order to accomplish this task. It would be really nice if these could be generated automatically for sqlite.h when running configure in the same way that VERSION and VERSION_NUMBER are so that setting the values manually wouldn't be required for the future.

Would you be willing to do that? Thanks in advance.

2007-Dec-17 13:30:22 by anonymous:
You could invoke this awk script in your make file:
#
## extrvers.awk
## Extract verison parts from sqlite3.h
#
## Usage:
#  %GNU_AWK% -f extrvers.awk sqlite3.h >sqlite3.h.new
# 	rm / del sqlite3.h
# 	mv / ren sqlite3.h.new sqlite3.h
#
#
## Ignore any previous defines
/^#define SQLITE_VERSION_(MAJOR|MINOR|PATCH)/{
	next
}
## generate extra #define MAJOR/MINOR/PATH lines
/^#define[[:blank:]]+SQLITE_VERSION[[:blank:]]/{
	split(substr($3,2,length($3) - 2),tmp,".")
	print "#define SQLITE_VERSION_MAJOR " tmp[1]
	print "#define SQLITE_VERSION_MINOR " tmp[2]
	print "#define SQLITE_VERSION_PATCH " tmp[3]
}
## Repeat all other lines untouched
{
	print
}


2007-Dec-18 00:06:53 by anonymous:
Original poster here. Thanks for the useful script! I'm assuming you're granting a license for this (or a modified version of it) to be included in the source tree if the powers that be are willing to accept it? This would be for the Mozilla project.


2007-Dec-18 00:21:52 by drh:
First off, I didn't post the script. I don't know who "anonymous" is.

Secondly, if you are working for Mozilla, you will get much faster service if you identify yourself as such.


2007-Dec-18 00:39:38 by anonymous:
I just want to make sure I'm not running afoul of anybody by using their work without proper permission.

To whoever posted it, you can contact me at ryanvm [at] gmail [dot] com. Thanks again for help!


2007-Dec-26 19:06:27 by anonymous:
I'm the poster of the script. Of course I don't mind it being used. For the peace of mind of anyone using it: please prepend the code with:
# Copyright (C) 2007 by Kees Nuyt, Rotterdam, Netherlands
# The author of this code dedicates any and all copyright
# interest in this code to the public domain. I make this
# dedication for the benefit of the public at large and
# to the detriment of my heirs and successors. I intend
# this dedication to be an overt act of relinquishment in
# perpetuity of all present and future rights to this
# code under copyright law.
#

Cheers! "Kees Nuyt" <k [dot] nuyt [at] zonnet [dot] nl>


2007-Dec-27 02:32:44 by anonymous:
Declaring a copyright and putting it into the public domain is not compatible.


2008-Jan-11 04:55:22 by anonymous:
Original poster here: FYI, I ended up writing a Python script to parse sqlite3.h for what we needed. The Mozilla bug for the work was

You can see the final script here: http://mxr.mozilla.org/mozilla/source/db/sqlite3/src/sqlite-version.py

At this point, you can close the bug if you want. I'll leave that up to you to decide.

 
2879 code active 2008 Jan anonymous   2008 Jan anonymous 4 3 VACUUM enters temporary sequence numbers in sqlite_sequence edit
I have two TEMPORARY tables added to a database, that both contain an INTEGER PRIMARY KEY AUTOINCREMENT field. After issuing a VACUUM command, the maximum value of these fields is added to the sqlite_sequence table. And stay there after the connection closes.
 
2336 warn active 2007 May anonymous   2008 Jan   4 4 Compiler warnings on strcpy(), sprintf(), et al. edit
OpenBSD's gcc compiler reports link warnings concerning potential insecure use of certain functions that have a history of abuse, such as strcpy, strcat, sprintf. It would be nice if SQLite could use the more secure version when possible.

/home/achowe/Projects/org/sqlite/lib/libsqlite3.a(util.o)(.text+0x184): In function `sqlite3StrDup': : warning: strcpy() is almost always misused, please use strlcpy() /home/achowe/Projects/org/sqlite/lib/libsqlite3.a(os_unix.o)(.text+0x822): In function `sqlite3UnixTempFileName': : warning: sprintf() is often misused, please use snprintf()

There was no misuse of these functions. But we'll get rid of them to avoid the library dependency.

For those that think that the use of strcpy() is a security problem: if you will check the diffs for this check-in you will see that by avoiding the use of strcpy() we have made the code more obtuse and more likely to contain a security bug, not the other way around. But at least now the compiler will not give warnings about strcpy()...


2007-May-04 15:43:08 by anonymous:
Some are still left in fts1.c and fts2.c.


2008-Jan-07 10:54:43 by anonymous:
SQLite 3.5.4 now generates new warnings on OpenBSD concerning unsafe functions.

gcc -g -O2 -o lemon ./tool/lemon.c /tmp//cco18706.o(.text+0x14df): In function `ErrorMsg': tool/lemon.c:1327: warning: vsprintf() is often misused, please use vsnprintf() /tmp//cco18706.o(.text+0x1655): In function `handle_D_option': tool/lemon.c:1378: warning: strcpy() is almost always misused, please use strlcpy() /tmp//cco18706.o(.text+0x14a2): In function `ErrorMsg': tool/lemon.c:1319: warning: sprintf() is often misused, please use snprintf() /tmp//cco18706.o(.text+0x3918): In function `file_makename': tool/lemon.c:2678: warning: strcat() is almost always misused, please use strlcat()

 
2875 code active 2008 Jan anonymous   2008 Jan   3 2 LIKE does not work with lowercase swedish characters edit
Swedish letters &aring;,&auml;,&ouml; is not supported by the LIKE statement. When trying to perform a query like

SELECT * FROM table WHERE name LIKE "&aring;%"

we will not get a match for names starting on &Aring; (which is uppercase for &aring;).

To recreate:


SQLite version 3.5.4 Enter ".help" for instructions sqlite> CREATE TABLE TestingTable(Name varchar(20)); sqlite> INSERT INTO TestingTable values ('Sweden'); sqlite> INSERT INTO TestingTable values ('sweden'); sqlite> INSERT INTO TestingTable values ('&Aring;land'); sqlite> INSERT INTO TestingTable values ('&aring;land'); sqlite> SELECT * FROM TestingTable; Sweden sweden &Aring;land &aring;land sqlite> SELECT * FROM TestingTable WHERE Name LIKE "swe%"; Sweden sweden sqlite> SELECT * FROM TestingTable WHERE Name LIKE &aring;la%"; &aring;land
 
2853 new active 2007 Dec anonymous   2008 Jan   2 3 optimizer fails to use an index on MAX subquery edit
i have these 2 identical queries with an index on (place_id,visit_date), the second query is about 2x fast the the first, while i'd expect that the MAX is faster than a limited order by clause... It's like the index is mis-used with MAX

  SELECT h.id, h.url, h.title, h.rev_host, h.visit_count,
  (SELECT MAX(visit_date) FROM moz_historyvisits WHERE place_id = h.id
  AND visit_type NOT IN(0,4)),
  f.url, null, null
  FROM moz_places h
  LEFT OUTER JOIN moz_favicons f ON h.favicon_id = f.id
  WHERE h.id IN
  (SELECT DISTINCT p.id
          FROM moz_places p
          JOIN moz_historyvisits ON place_id = p.id
          WHERE hidden <> 1 AND visit_type NOT IN (0,4)
          ORDER BY visit_date DESC
          LIMIT 10)
  ORDER BY 6 DESC;

  SELECT h.id, h.url, h.title, h.rev_host, h.visit_count,
  (SELECT visit_date FROM moz_historyvisits WHERE place_id = h.id
   AND visit_type NOT IN(0,4) ORDER BY visit_date DESC LIMIT 1),
  f.url, null, null
  FROM moz_places h
  LEFT OUTER JOIN moz_favicons f ON h.favicon_id = f.id
  WHERE h.id IN
  (SELECT DISTINCT p.id
          FROM moz_places p
          JOIN moz_historyvisits ON place_id = p.id
          WHERE hidden <> 1 AND visit_type NOT IN (0,4)
          ORDER BY visit_date DESC
          LIMIT 10)
  ORDER BY 6 DESC;
2007-Dec-20 04:13:44 by anonymous:
Can you supply the schema of these tables and all related indexes?


2007-Dec-20 10:55:29 by danielk1977:
Correct. SQLite does not use the index to optimize the max() in the first query. But it does use it to optimize ORDER BY in the second.


2007-Dec-21 00:54:10 by anonymous:
the schema is that of mozilla firefox 3 places.sqlite, with an added index on moz_historyvisits(place_id,visit_date), this is extracted from a i bug i was working on. Do you need that i append the full schema here?

Will this be fixed to use the index with max or is that the expected behaviour?


2007-Dec-21 15:59:58 by drh:
We will be enhancing the min()/max() optimization to be able to cover the case you describe above. But this will take some time to get right. If you are in a hurry, it seems like modifying your SQL is the easiest way to go.

The current optimizer recognizes queries of the form:

    SELECT min(x) FROM table;
    SELECT max(x) FROM table;

And it causes special VDBE code to be generated for these cases. The presence of a WHERE clause defeats the current optimization. We need to modify the optimizer to recognize queries of the form:

    SELECT min(x) FROM table WHERE expr

And convert them into:

    SELECT x FROM table WHERE expr AND x NOT NULL
     ORDER BY x LIMIT 1

But we only want to do this optimization if the ORDER BY can be evaluated using an index. If we have to accumulate the entire result set, sort it, then take the largest entry, that is more work than just keeping track of the largest entry as we loop through the result set.

Notice the addition of the "x NOT NULL" term in the WHERE clause. This is critical to preserving correct semantics. The query optimizer currently does not know how to deal with a NOT NULL. It just evaluates all rows, tests them individually for NULL, and throws out those that match. This would work in most cases, but would slow down in a min() where there were many NULL entries. The current implemention of the min() optimization knows to skip over the NULL entries in a single operation. The WHERE generator part of the optimizer will need to be enhanced to recognize NOT NULL constraints and skip over them.

All of this is a substantial change. The min()/max() optimization function will be rewritten from scratch. Significant and non-trivial modifications will need to be made to the code that looks for indices and generates the WHERE clause constraints. There are many opportunities to make coding mistakes, so we will need to spend a lot of time in testing before we put these changes into production.

So, we will be working the problem. But do not expect an overnight fix.

I suppose that while we are redesigning the min/max optimizer, we might as well also fix it so that

   SELECT arbitrary_expr(max(x)) FROM table WHERE expr;

gets converted into

   SELECT arbitrary_expr(x) FROM table WHERE expr AND x NOT NULL
    ORDER BY x DESC LIMIT 1;


2007-Dec-31 06:41:03 by anonymous:
Regarding:

  SELECT min(x) from table WHERE expr

being converted to:

  SELECT x FROM table WHERE expr AND x NOT NULL ORDER BY x LIMIT 1

There's no need for the "x NOT NULL" condition considering NULL is returned by min() (or max for that matter) when no rows match.

  sqlite> .nullvalue <null>

  sqlite> select min(a) from (select 123 as a) where a=7;
  <null>

  sqlite> select min(a) from (select NULL as a) where a=7;
  <null>

Even with this in mind, you can see that the rewritten query still does not return the same result in this case:

  sqlite> select a from (select 123 as a) where a=7 order by 1 limit 1;
  -- no rows returned

Logic would have to be added to return a NULL row in the event the WHERE clause matches no rows.


2007-Dec-31 07:06:22 by anonymous:
Given:

  SELECT min(x) from table WHERE expr

If column x or the WHERE expr can make use of an index, then the min query should be converted to:

  SELECT x from (
    SELECT x FROM table WHERE expr
      ORDER BY x LIMIT 1
  ) UNION ALL SELECT NULL LIMIT 1

which ought to cover all the corner cases, even if the WHERE matches no rows.


2008-Jan-01 19:21:51 by anonymous:
This ticket is related to WHERE cost estimation in ticket #2857.

Given:

  create table stuff(a,b,c,d);
  insert into stuff values(1,2,3,4);
  create temp view v1 as select random()%100,
    random()%100, random()%1000, random()%10000
     from stuff x, stuff y;
  insert into stuff select * from v1;
  insert into stuff select * from v1;
  insert into stuff select * from v1;
  insert into stuff select * from v1;
  insert into stuff select * from v1;
  create index stuff_b on stuff(b);
  create index stuff_c on stuff(c);
  create index stuff_d on stuff(d);
  analyze;

In the following example the existing min() implementation which uses a full table scan:

  sqlite> explain query plan select min(c) from stuff where a=12345;
  0|0|TABLE stuff

  sqlite> select min(c) from stuff where a=12345;
  (null)
  CPU Time: user 1.388087 sys 0.216014

is faster than performing the select min->order by/limit transform which uses an index:

  sqlite> explain query plan select c from stuff where a=12345 order by 1 limit 1;
  0|0|TABLE stuff WITH INDEX stuff_c ORDER BY

  sqlite> explain query plan select c from (select c from stuff where a=12345 order by c limit 1) union all select null limit 1;
  0|0|TABLE stuff WITH INDEX stuff_c ORDER BY
  0|0|TABLE  AS sqlite_subquery_82F83F8_
  CPU Time: user 0.000000 sys 0.000000

  sqlite> select c from (select c from stuff where a=12345 order by c limit 1) union all select null limit 1;
  (null)
  CPU Time: user 15.880993 sys 15.400962

Note that a=12345 is always false in this data set.

Had a=23 been used instead, we can see that the transformed select would be much faster than the original min() select:

  sqlite> select min(c) from stuff where a=23;
  -999
  CPU Time: user 1.552097 sys 0.148009

  sqlite> select c from (select c from stuff where a=23 order by c limit 1) union all select null limit 1;
  -999
  CPU Time: user 0.004001 sys 0.000000

I don't think WHERE clause cost estimation can be done accurately without using a statistical method based on historical WHERE clause hit percentages.


2008-Jan-05 18:15:39 by anonymous:
I appreciate that you have more refinements pending, but this query in particular has regressed from sqlite 3.5.4 using the schema mentioned above:

  -- sqlite 3.5.4
  sqlite> select min(c) from stuff where a=12345;
  (null)
  CPU Time: user 1.532095 sys 0.132008

  -- as of Check-in [4687]
  sqlite> select min(c) from stuff where a=12345;
  (null)
  CPU Time: user 23.041440 sys 16.369023
 
2874 code active 2008 Jan anonymous   2008 Jan   1 1 THREADSAFE #define HAVE_LOCALTIME_R, HAVE_GMTIME_R in os_unix.c edit
The precompiled shared sqlite3 library for Linux on sqlite.org which appears to be built with pthread support is using localtime and gmtime which are not threadsafe.

For THREADSAFE builds could either configure be changed to detect the functions gmtime_r and localtime_r or change os_unix.c to explicitly #define HAVE_LOCALTIME_R and HAVE_GMTIME_R?

 
2873 code active 2008 Jan anonymous   2008 Jan   1 1 HAVE_USLEEP, HAVE_FDATASYNC=1 detected but not used by configure; make edit
I noticed that a couple of open source projects were not picking up usleep() for recent sqlite builds and used the coarser grained sleep() instead. Around 11 months ago something changed in sqlite's build process.

It seems that both -DHAVE_USLEEP=1, -DHAVE_FDATASYNC=1 and -DOS_UNIX=1 are detected correctly by configure but not used by the generated Makefile. As result, UNIX builds of sqlite3 via ./configure do not use usleep() and fdatasync() and do not define OS_UNIX.

I don't know whether the lack of fdatasync() versus the default fsync() affects anyone.

Please apply the following patch which corrects the problem with "./configure && make". Thank you.

Index: configure
===================================================================
RCS file: /sqlite/sqlite/configure,v
retrieving revision 1.45
diff -u -3 -p -r1.45 configure
--- configure	27 Nov 2007 14:50:07 -0000	1.45
+++ configure	5 Jan 2008 07:41:00 -0000
@@ -18520,9 +18520,9 @@ if test "$TARGET_EXEEXT" = ".exe"; then
     OS_UNIX=0
     OS_WIN=0
     OS_OS2=1
-    TARGET_CFLAGS="$TARGET_CFLAGS -DOS_OS2=1"
+    CFLAGS="$CFLAGS -DOS_OS2=1"
     if test "$ac_compiler_gnu" == "yes" ; then
-      TARGET_CFLAGS="$TARGET_CFLAGS -Zomf -Zexe -Zmap"
+      CFLAGS="$CFLAGS -Zomf -Zexe -Zmap"
       BUILD_CFLAGS="$BUILD_CFLAGS -Zomf -Zexe"
     fi
   else
@@ -18530,14 +18530,14 @@ if test "$TARGET_EXEEXT" = ".exe"; then
     OS_WIN=1
     OS_OS2=0
     tclsubdir=win
-    TARGET_CFLAGS="$TARGET_CFLAGS -DOS_WIN=1"
+    CFLAGS="$CFLAGS -DOS_WIN=1"
   fi
 else
   OS_UNIX=1
   OS_WIN=0
   OS_OS2=0
   tclsubdir=unix
-  TARGET_CFLAGS="$TARGET_CFLAGS -DOS_UNIX=1"
+  CFLAGS="$CFLAGS -DOS_UNIX=1"
 fi


@@ -19392,7 +19392,7 @@ fi
 echo "$as_me:$LINENO: result: $ac_cv_func_usleep" >&5
 echo "${ECHO_T}$ac_cv_func_usleep" >&6
 if test $ac_cv_func_usleep = yes; then
-  TARGET_CFLAGS="$TARGET_CFLAGS -DHAVE_USLEEP=1"
+  CFLAGS="$CFLAGS -DHAVE_USLEEP=1"
 fi


@@ -19491,7 +19491,7 @@ fi
 echo "$as_me:$LINENO: result: $ac_cv_func_fdatasync" >&5
 echo "${ECHO_T}$ac_cv_func_fdatasync" >&6
 if test $ac_cv_func_fdatasync = yes; then
-  TARGET_CFLAGS="$TARGET_CFLAGS -DHAVE_FDATASYNC=1"
+  CFLAGS="$CFLAGS -DHAVE_FDATASYNC=1"
 fi


Index: configure.ac
===================================================================
RCS file: /sqlite/sqlite/configure.ac,v
retrieving revision 1.31
diff -u -3 -p -r1.31 configure.ac
--- configure.ac	27 Nov 2007 14:50:07 -0000	1.31
+++ configure.ac	5 Jan 2008 07:41:00 -0000
@@ -310,9 +310,9 @@ if test "$TARGET_EXEEXT" = ".exe"; then
     OS_UNIX=0
     OS_WIN=0
     OS_OS2=1
-    TARGET_CFLAGS="$TARGET_CFLAGS -DOS_OS2=1"
+    CFLAGS="$CFLAGS -DOS_OS2=1"
     if test "$ac_compiler_gnu" == "yes" ; then
-      TARGET_CFLAGS="$TARGET_CFLAGS -Zomf -Zexe -Zmap"
+      CFLAGS="$CFLAGS -Zomf -Zexe -Zmap"
       BUILD_CFLAGS="$BUILD_CFLAGS -Zomf -Zexe"
     fi
   else
@@ -320,14 +320,14 @@ if test "$TARGET_EXEEXT" = ".exe"; then
     OS_WIN=1
     OS_OS2=0
     tclsubdir=win
-    TARGET_CFLAGS="$TARGET_CFLAGS -DOS_WIN=1"
+    CFLAGS="$CFLAGS -DOS_WIN=1"
   fi
 else
   OS_UNIX=1
   OS_WIN=0
   OS_OS2=0
   tclsubdir=unix
-  TARGET_CFLAGS="$TARGET_CFLAGS -DOS_UNIX=1"
+  CFLAGS="$CFLAGS -DOS_UNIX=1"
 fi

 AC_SUBST(BUILD_EXEEXT)
@@ -565,13 +565,13 @@ AC_SUBST(TARGET_DEBUG)
 #########
 # Figure out whether or not we have a "usleep()" function.
 #
-AC_CHECK_FUNC(usleep, [TARGET_CFLAGS="$TARGET_CFLAGS -DHAVE_USLEEP=1"])
+AC_CHECK_FUNC(usleep, [CFLAGS="$CFLAGS -DHAVE_USLEEP=1"])

 #--------------------------------------------------------------------
 # Redefine fdatasync as fsync on systems that lack fdatasync
 #--------------------------------------------------------------------

-AC_CHECK_FUNC(fdatasync, [TARGET_CFLAGS="$TARGET_CFLAGS -DHAVE_FDATASYNC=1"])
+AC_CHECK_FUNC(fdatasync, [CFLAGS="$CFLAGS -DHAVE_FDATASYNC=1"])

 #########
 # Generate the output files.
2008-Jan-05 08:24:29 by anonymous:
It appears that http://www.sqlite.org/sqlite3-3.5.4.bin.gz and http://www.sqlite.org/sqlite-3.5.4.so.gz use sleep and fsync even though usleep and fdatasync are available on Linux.

On the Linux man page, it claims that fdatasync is more efficient than fsync:

"Unfortunately, fsync() will always initiate two write operations: one for the newly written data and another one in order to update the modification time stored in the inode. If the modification time is not a part of the transaction concept fdatasync() can be used to avoid unnecessary inode disk write operations."

 
2872 code active 2008 Jan anonymous   2008 Jan   4 4 Some table scan operations could use an index for better data density edit
If a suitable index covers a column being during a table scan, it makes sense to use the index for IO not the table pages themselves for speed.

As (contrived) example:

  CREATE TABLE t1 ( c1 blob, c2 integer, c3 integer );
  CREATE INDEX i1 on t1(c2,c3);
  CREATE INDEX i2 on t1(c3);

T1 populated with 4096 rows, c1 being 64K random blobs (to make c2, c3 access slower in this case), c2 and c3 being small random integers.

Now:

  sqlite> select sum(c3) from t1;
  sum(c3)
  ----------
  519895

is very slow (several seconds). A table scan is done.

Forcing use of an index, in a way that I know all rows will be included:

  sqlite> select sum(c3) from t1 where c2<1000;
  sum(c3)
  ----------
  519895

This is instantaneous.

It seems to me that if a table scan has to be performance, it makes sense to grab the data from an index whenever possible. ideally the most densely packed index.

(BTW; this is contrived, I know putting the blob as the last column will greatly speed things up).

 
2869 new active 2008 Jan anonymous   2008 Jan   5 5 add "sqlite3_open16_v2" to the C API edit
I'm using UTF-16, and if the database file does not exist, "sqlite3_open16" will create a new one, but i wish it fails in such conditions. I notice that there's a "sqlite3_open_v2", but it doesn't support UTF-16, although i can implement a "sqlite3_open16_v2" myself, I think it should exists in the offical releases.
 
2867 code active 2008 Jan anonymous   2008 Jan   2 2 doesn't build on Cygwin - wrong sqlite3 exe suffix edit
The new Makefile used $(EXE), which doesn't seem to be defined (typo?)
2008-Jan-02 11:12:39 by anonymous:
Same on mingw:

Following patch fixes things:

--- sqlite-3.5.4/Makefile.in    Thu Dec 13 19:17:42 2007
+++ sqlite-3.5.4-mingw-fix/Makefile.in  Wed Jan  2 11:37:50 2008
@@ -322,7 +322,7 @@
                -o $@ $(TOP)/src/shell.c libsqlite3.la \
                $(LIBREADLINE) $(TLIBS)

-sqlite3$(EXE): $(TOP)/src/shell.c sqlite3.c sqlite3.h
+sqlite3$(TEXE):        $(TOP)/src/shell.c sqlite3.c sqlite3.h
        $(LTLINK) $(READLINE_FLAGS) -o $@                          \
                -DSQLITE_MAX_SQL_LENGTH=1000000000                  \
                -USQLITE_THREADSAFE -DSQLITE_THREADSAFE=0           \
@@ -577,7 +577,7 @@
          -e 's,$$,\\n",' \
          $(TOP)/tool/spaceanal.tcl >spaceanal_tcl.h
        $(LTLINK) -DTCLSH=2 -DSQLITE_TEST=1 $(TEMP_STORE)\
-                -o sqlite3_analyzer$(EXE) $(TESTSRC) $(TOP)/src/tclsqlite.c \
+                -o sqlite3_analyzer$(TEXE) $(TESTSRC) $(TOP)/src/tclsqlite.c \
                libtclsqlite3.la $(LIBTCL)
 
2866 build active 2008 Jan anonymous   2008 Jan   1 3 Problems building Windows native in cygwin/mingw environment edit
Trying to build Windows native version using the Cygwin build environment.

$ gcc -v Reading specs from /usr/lib/gcc/i686-pc-cygwin/3.4.4/specs Configured with: /usr/build/package/orig/test.respin/gcc-3.4.4-3/configure --verbose --prefix=/usr --exec-prefix=/usr --sysconfdir=/etc --libdir=/usr/lib --libexecdir=/usr/lib --mandir=/usr/share/man --infodir=/usr/share/info --enable-languages=c,ada,c++,d,f77,pascal,java,objc --enable-nls --without-included-gettext --enable-version-specific-runtime-libs --without-x --enable-libgcj --disable-java-awt --with-system-zlib --enable-interpreter --disable-libgcj-debug --enable-threads=posix --enable-java-gc=boehm --disable-win32-registry --enable-sjlj-exceptions --enable-hash-synchronization --enable-libstdcxx-debug Thread model: posix gcc version 3.4.4 (cygming special, gdc 0.12, using dmd 0.125) $

$ CFLAGS=-mno-cygwin ./configure --disable-tcl --enable-threadsafe

$ make

A) The make appears to build sqlite3.exe just fine, without errors or warnings. This binary does work from cmd.exe, BUT not from within the bash cygwin shell for some reason, unlike other Windows native binaries I've built.

Next...

$ make install

B) The

cc sqlite3.c -o sqlite3

fails to rebuild sqlite3.exe correctly with the -mno-cygwin option. The output follows:

rm -rf tsrc mkdir -p tsrc cp ./src/alter.c ./src/analyze.c ./src/attach.c ./src/auth.c ./src/btmutex.c ./src/btree.c ./src/btree.h ./src/build.c ./src/callback.c ./src/complete.c ./src/date.c ./src/delete.c ./src/expr.c ./src/func.c ./src/hash.c ./src/hash.h ./src/insert.c ./src/journal.c ./src/legacy.c ./src/loadext.c ./src/main.c ./src/malloc.c ./src/mem1.c ./src/mem2.c ./src/mem3.c ./src/mutex.c ./src/mutex_os2.c ./src/mutex_unix.c ./src/mutex_w32.c ./src/os.c ./src/os_unix.c ./src/os_win.c ./src/os_os2.c ./src/pager.c ./src/pager.h ./src/parse.y ./src/pragma.c ./src/prepare.c ./src/printf.c ./src/random.c ./src/select.c ./src/shell.c ./src/sqlite.h.in ./src/sqliteInt.h ./src/table.c ./src/tclsqlite.c ./src/tokenize.c ./src/trigger.c ./src/utf.c ./src/update.c ./src/util.c ./src/vacuum.c ./src/vdbe.c ./src/vdbe.h ./src/vdbeapi.c ./src/vdbeaux.c ./src/vdbeblob.c ./src/vdbefifo.c ./src/vdbemem.c ./src/vdbeInt.h ./src/vtab.c ./src/where.c ./ext/fts1/fts1.c ./ext/fts1/fts1.h ./ext/fts1/fts1_hash.c ./ext/fts1/fts1_hash.h ./ext/fts1/fts1_porter.c ./ext/fts1/fts1_tokenizer.h ./ext/fts1/fts1_tokenizer1.c sqlite3.h ./src/btree.h ./src/btreeInt.h ./src/hash.h ./src/sqliteLimit.h ./src/mutex.h opcodes.h ./src/os.h ./src/os_common.h ./src/sqlite3ext.h ./src/sqliteInt.h ./src/vdbe.h parse.h ./ext/fts1/fts1.h ./ext/fts1/fts1_hash.h ./ext/fts1/fts1_tokenizer.h ./src/vdbeInt.h tsrc cp: warning: source file `./src/btree.h' specified more than once cp: warning: source file `./src/hash.h' specified more than once cp: warning: source file `./src/sqliteInt.h' specified more than once cp: warning: source file `./src/vdbe.h' specified more than once cp: warning: source file `./ext/fts1/fts1.h' specified more than once cp: warning: source file `./ext/fts1/fts1_hash.h' specified more than once cp: warning: source file `./ext/fts1/fts1_tokenizer.h' specified more than once cp: warning: source file `./src/vdbeInt.h' specified more than once rm tsrc/sqlite.h.in tsrc/parse.y cp parse.c opcodes.c keywordhash.h tsrc tclsh ./tool/mksqlite3c.tcl cc sqlite3.c -o sqlite3 /usr/lib/gcc/i686-pc-cygwin/3.4.4/../../../libcygwin.a(libcmain.o):(.text+0xab): undefined reference to `_WinMain@16' collect2: ld returned 1 exit status make: *** [sqlite3] Error 1 $

 
2864 doc active 2007 Dec anonymous   2007 Dec   5 3 ext/fts3/README.txt edit
File ext/fts3/README.txt reads: This folder contains source code to the second full-text search [...]

Shouldn't that be: This folder contains source code to the third full-text search [...]

2007-Dec-30 18:27:08 by anonymous:
Oh, after Googl'ing a little bit, I found that fts3 really is fts2-with-rowid-fixed.

If both fts2 and fts3 are considered to be the "second full-text search extension for SQLite", the README files could maybe explain the situation.

 
2865 code active 2007 Dec anonymous   2007 Dec   1 2 FTS3 does not build with amalgamation in CVS edit
Grab the latest CVS sources, then run:

  ./configure
  make sqlite3.c

  grep sqlite3Fts3Init sqlite3.c

    extern int sqlite3Fts3Init(sqlite3*);
    rc = sqlite3Fts3Init(db);

If you compile sqlite3.c with -DSQLITE_ENABLE_FTS3, then sqlite3Fts3Init is unresolved.

For some reason, sqlite3Fts3Init and fts3.c was not included in the sqlite3.c amalg.

It used to work correctly in 3.5.4.

2007-Dec-30 18:17:57 by anonymous:
Nevermind, "make sqlite3.c" has never built with the fts3 sources in 3.5.4 or before.

You have to run ext/fts3/mkfts3amal.tcl


2007-Dec-30 18:20:56 by anonymous:
It seems that the sqlite3+fts3 amalg can only be built from main.mk, not Makefile.
 
2863 code active 2007 Dec anonymous   2007 Dec   2 3 test cast-3.14, cast-3.18 and cast-3.24 fail edit
  test cast-3.{14,18,24} fail on freebsd-6.3-PRERELEASE2:
  cast-3.14...^M
  Expected: [9223372036854774784]^M
       Got: [9223372036854773760]^M
  cast-3.18...^M
  Expected: [-9223372036854774784]^M
       Got: [-9223372036854773760]^M
  cast-3.24...^M
  Expected: [9223372036854774784]^M
       Got: [9223372036854773760]^M
  I used tcl8.4 from ports with no threads and here was the config line:
  ../sqlite-3.5.4/configure --prefix=/home/marc/local --with-tcl=/usr/local/lib/tcl8.4/

This was built on an ibm t30 laptop

 
2860 todo active 2007 Dec anonymous   2007 Dec   3 1 Database file fragmentation edit
Adding data in database file increases file fragmentation. for example my file which size is 1G, consists of 20000 pieces. (NTFS)

This happens because truncation of '-journal' file. I see some ways to reduce fragmentaion: 1. Increase database file size by greater pieces (not by PAGESIZE). 2. SQLite can save '-journal' file in another folder(logical disc). 3. Preallocation of database file(must increase INSERT speed).

 
2859 code active 2007 Dec anonymous   2007 Dec drh 3 2 Inconsistent column names with DISTINCT edit
Given the following SQL:

  CREATE TABLE foo(a,b);
  INSERT INTO foo (a, b) VALUES (1,2);

SQLite returns inconsistent column names when using the DISTINCT clause:

  SELECT DISTINCT foo.A, foo.B FROM foo;
  foo.A|foo.B
  1|2

  SELECT DISTINCT a, b FROM foo;
  a|b
  1|2

  SELECT DISTINCT * FROM foo;
  a|b
  1|2

  SELECT DISTINCT foo.* FROM foo;
  a|b
  1|2

Compared with SELECT without DISTINCT:

  SELECT foo.A, foo.B FROM foo;
  a|b
  1|2

  SELECT a, b FROM foo;
  a|b
  1|2

  SELECT * FROM foo;
  a|b
  1|2

  SELECT foo.* FROM foo;
  a|b
  1|2
 
2761 code active 2007 Nov anonymous   2007 Dec   3 3 CLI (shell.c) should be bundled with amalgamation edit
The CLI (shell.c) should be bundled with the amalgamation for database administrative purposes without downloading the matching shell.c from the full source tree.
I second that! Qt ships with the amalgamated source files, but we also ship shell.c, whch we have to retrieve from the non-amalgamated source files.


2007-Dec-26 15:20:04 by anonymous:
I also agree. It is inconvenient to retrieve the matching shell.c from the source tree.
 
2857 code active 2007 Dec anonymous   2007 Dec   2 2 GROUP BY cost estimate wrong with WHERE clause edit
There seems to be an issue with the sqlite cost heuristic with an INDEX present on GROUP BY with certain types of WHERE clauses.

Given the database formed by running these statements:

  create table stuff(a,b,c,d);
  insert into stuff values(1,2,3,4);
  create temp view v1 as select random()%100,
    random()%100, random()%1000, random()%10000
     from stuff x, stuff y;
  insert into stuff select * from v1;
  insert into stuff select * from v1;
  insert into stuff select * from v1;
  insert into stuff select * from v1;
  insert into stuff select * from v1;
  create index stuff_b on stuff(b);
  create index stuff_c on stuff(c);
  create index stuff_d on stuff(d);
  analyze;

Using sqlite.org's sqlite3-3.5.4.bin, this query takes 47 seconds:

  select c from stuff where a=23 group by c;

while this query takes just 2 seconds:

  select c from stuff where a=23 group by +c;

It is more efficient in this case to do a full table scan instead of using the INDEX on column c.

2007-Dec-23 23:14:06 by anonymous:
The queries above both run in a couple of seconds with this naive patch:

--- src/where.c 12 Dec 2007 17:42:53 -0000      1.266
+++ src/where.c 23 Dec 2007 22:48:37 -0000
@@ -1514,6 +1514,12 @@ static double bestIndex(
     flags = 0;
   }

+  if( pWC && pWC->nTerm>0 && pOrderBy ){
+    /* Reduce cost if both an ORDER/GROUP BY exists with a WHERE. */
+    cost /= 100; /* A very rough guess. */
+    WHERETRACE(("... WHERE + ORDER BY decreases cost to: %.9g\n", cost));
+  }
+
   /* If the table scan does not satisfy the ORDER BY clause, increase
   ** the cost by NlogN to cover the expense of sorting. */
   if( pOrderBy ){
But it has not been tested on queries with more than one table. Its logic could be flawed.


2007-Dec-24 00:09:00 by drh:
The complaint is centered around these two queries:

    /* 1 */  SELECT c FROM stuff WHERE a=23 GROUP BY c;
    /* 2 */  SELECT c FROM stuff WHERE a=23 GROUP BY +c;

Query 1 runs in about 40 seconds and query 2 in about 1.5 seconds on my macbook. But with the patch, both queries run in about 1.5 seconds. Fair enough. But now consider these two queries:

    /* 3 */  SELECT c FROM stuff WHERE a!=23 GROUP BY c;
    /* 4 */  SELECT c FROM stuff WHERE a!=23 GROUP BY +c;

In this case, query 3 runs in 42 seconds on an unpatched version of 3.5.4 and query 4 runs in about 109 seconds. So in cases where the WHERE clause is not particularly selective, the first version is faster than the second by a good margin. On a patched version of 3.5.4, both queries 3 and 4 run in about 110 seconds.

So it seems to me that the patch is robbing Peter to pay Paul. It makes ORDER BY queries with very selective WHERE clauses run faster but at the expense of making queries with unselective WHERE clauses running slower.

But notice this: in the current (unpatched) implementation, the programmer at least has the ability to select a different algorithm by the judicious placement of a "+" sign. After the patch, this is no longer possible. The patch forces the second algorithm to be used in all cases, even cases where it is slower.

It seems to me to be better to leave things as they are since the current approach at least allows the programmer to override SQLite's algorithm selection if SQLite chooses incorrectly.

The only way, it seems to me, to automatically choose the correct algorithm is to devise some test that will determine (in advance) whether or not the WHERE clause weeds out many or few rows from the result set. I'm thinking that determination is going to be very hard (or impossible) to do without first doing a full table scan.


2007-Dec-24 05:40:47 by anonymous:
It think it would be surprising to average users that adding an index (on column C in this case) may significantly decrease query performance for some queries. It was surprising to me, at least. In my opinion, a query being 20 times slower in a default bad guess situation is worse than a query only being 2.5 times slower with a default bad guess in a worst case scenario. It's a question of relative magnitude of the difference. This is why I think that the database should err on the side of the WHERE clause having a more selective bias.

(Side note: the query timings difference is less pronounced if you use PRAGMA temp_store=memory, in which case query 3 running on an unpatched 3.5.4 takes just 50% more time to run than query 4 on my machine.)

But you raise a good point in that if there's a wrong guess in the selectivity bias it would be nice to be able to manually override it.

How much do you hate this type of syntax that some other databases use?

  select c from stuff where a!=23 group by /*+stuff_c*/ c;

SQLite does not currently offer a way to pick a specific index. I think it would be quite useful.


2007-Dec-24 17:05:16 by anonymous:
Another option is to collect WHERE clause statistics in a table like

  create table sqlite_stat2(
    where_clause_md5 BLOB primary key,
    where_clause     TEXT,
    rows_examined    INT,
    rows_true        INT
  );

where the last 2 columns are cumulative for each query. The statistics option could be enabled/disabled via a PRAGMA sqlite_collect_statistics.

The where_clause column could be a string generated fairly easily from the walking the parse tree of the resolved Select statement's pWhere. This way the where_clause is normalized and a single query with many subselects could generate more than 1 where_clause, and different queries that happen to use the same normalized where clause would update the same entry in the stat2 table. where_clause normalization would strip off aliases and only refer to the original table and column names.

For example the 2 queries below:

  -- CREATE TABLE t1(a, b);
  -- CREATE TABLE t2(b, c);
  SELECT t1.a*c as AC, t2.b-a as BA FROM t1, t2 WHERE AC>BA;
  SELECT *, t1.a Foo FROM t2, t1 WHERE Foo*c > t2.b - t1.a;

would generate the same normalized where_clause string "(T1.A*T2.C)>(T2.B-T1.A)". The table information is already encoded within it.

The generated VDBE code would have to generate Mem counters that would be incremented by each WHERE test, and lazily updated at the end of transactions or periodically written to the stat2 table to minimize disk use, as this information is not critical.

One could also manually set the stat2 table with statistical values they would like their queries to use even if PRAGMA sqlite_collect_statistics=off;

Any time the schema is changed, the entire sqlite_stat2 table would be cleared.

 
2856 doc active 2007 Dec anonymous   2007 Dec anonymous 4 3 SQLite Documentation - Tcl API - Link broken edit
On the page "The Tcl Interface to the SQLite library", the link "enable load extension" does not work.
 
1648 new active 2006 Jan anonymous Shell 2007 Dec   4 3 meaningful error message: constraint failed edit
  create table emp( id text unique, sex text check( sex in 'm' or sex in 'f' );
  insert into emp values( '1','x' );
  SQL error: constraint failed

This error message could be better. If there are several constraints, which constraint failed? So I named the constraint

  create table emp( id text unique, sex text constraint chk_sex check( sex in 'm' or sex in 'f' );
  insert into emp values( '1','x' );
  SQL error: constraint failed

Still no joy . . . It would be nice if the error message were more specific.

2006-Jan-30 16:22:58 by anonymous:
actually my testing was better than my typing, I used:

  check (sex = 'm' or sex = 'f' )


2007-Oct-25 09:47:14 by anonymous:
This is a really big deal for me and for many others I suspect. If this is not a priority, could you at least throw out some hints about implementing it? I browsed through the code but can't seem to find where this would even go.


2007-Oct-25 10:10:36 by anonymous:
Hm, ok, the check constraints are stored in the table structure as a single expression which is the AND of all of them. This alone suggests that the task at hand is not simple...


2007-Oct-25 10:17:36 by anonymous:
Perhaps a new Check type could be created which could basically be Expr plus an extra pointer, which could then be used to make a list of them, similar to how the triggers seem to be stored. I'll keep snooping around, but I thought I'd post what I've found thus far in case anyone else looks at this.


2007-Oct-25 19:40:37 by anonymous:
I have attached a patch that implements this. I've only tested it lightly by hand. (The test suite failed to run and gave me some strange linking errors)


2007-Dec-20 11:38:03 by anonymous:
Although this is tagged as shell, the error message comes from the sqlite core.

My single biggest problem besides the lack of detail (some of my tables have 5 constraints) is that it also prevents me from localizing the error messages. If I have the constraint name then at least I can look it up in a translation table and tell non-english speakers something meaningful.

 
2508 code active 2007 Jul anonymous   2007 Dec   1 1 utf8ToUnicode() does not work on some WinCE devices edit
On some WinCE devices first call to MultiByteToWideChar() in utf8ToUnicode() always fails. Tried calling GetLastError() after it fails and it returns error code 87 -- ERROR_INVALID_PARAMETER.

To fix this had to change code page from CP_UTF8 to CP_ACP -- no idea why this works.

Original utf8ToUnicode()


static WCHAR *utf8ToUnicode(const char *zFilename) { int nChar; WCHAR *zWideFilename;

  nChar = MultiByteToWideChar(CP_UTF8, 0, zFilename, -1, NULL, 0);
  zWideFilename = sqliteMalloc( nChar*sizeof(zWideFilename[0]) );
  if( zWideFilename==0 ){
    return 0;
  }
  nChar = MultiByteToWideChar(CP_UTF8, 0, zFilename, -1, zWideFilename, nChar);
  if( nChar==0 ){
    sqliteFree(zWideFilename);
    zWideFilename = 0;
  }
  return zWideFilename;
}


Fixed utf8ToUnicode()
static WCHAR *utf8ToUnicode(const char *zFilename) { int nChar; WCHAR *zWideFilename;

  nChar = MultiByteToWideChar(CP_ACP, 0, zFilename, -1, NULL, 0);
  if( nChar == 0 )
  {
    DWORD dwError = GetLastError();
    OSTRACE2("MultiByteToWideChar() failed, last error: %d\n", dwError);
    return 0;
  }
  zWideFilename = sqliteMalloc( nChar*sizeof(zWideFilename[0]) );
  if( zWideFilename==0 ){
    return 0;
  }
  nChar = MultiByteToWideChar(CP_ACP, 0, zFilename, -1, zWideFilename, nChar);
  if( nChar==0 ){
    sqliteFree(zWideFilename);
    zWideFilename = 0;
  }
  return zWideFilename;
}


2007-Jul-17 23:56:10 by anonymous:
unicodeToUtf8() needs to be fixed the same way.

Before:


static char *unicodeToUtf8(const WCHAR *zWideFilename){ int nByte; char *zFilename;

  nByte = WideCharToMultiByte(CP_UTF8, 0, zWideFilename, -1, 0, 0, 0, 0);
  zFilename = sqliteMalloc( nByte );
  if( zFilename==0 ){
    return 0;
  }
  nByte = WideCharToMultiByte(CP_UTF8, 0, zWideFilename, -1, zFilename, nByte,
                              0, 0);
  if( nByte == 0 ){
    sqliteFree(zFilename);
    zFilename = 0;
  }
  return zFilename;
}


After:
static char *unicodeToUtf8(const WCHAR *zWideFilename){ int nByte; char *zFilename;

  nByte = WideCharToMultiByte(CP_ACP, 0, zWideFilename, -1, NULL, 0, NULL, NULL);
  if ( nByte == 0 )
  {
    DWORD dwError = GetLastError();
    OSTRACE2("WideCharToMultiByte() failed, last error = %d\n", dwError);
    return 0;
  }
  zFilename = sqliteMalloc( nByte );
  if( zFilename==0 ){
    return 0;
  }
  nByte = WideCharToMultiByte(CP_ACP, 0, zWideFilename, -1, zFilename, nByte,
                              0, 0);
  if( nByte == 0 ){
    sqliteFree(zFilename);
    zFilename = 0;
  }
  return zFilename;
}


Note that while original code with CP_UTF8 works on Windows and SOME WinCE devices, this modified code works well and Windows and all WinCE devices I've tested so far.


2007-Jul-18 16:01:21 by anonymous:
Why not using the conversions from SQLite internals ? It can change a UTF-16 to UTF-8 and vice-versa.

Or using UTF-16 variants in windows ce should be the best case.


2007-Aug-09 20:47:04 by anonymous:

  Why not using the conversions from SQLite internals ? It can change a UTF-16 to UTF-8 and vice-versa.

  Or using UTF-16 variants in windows ce should be the best case.

Not so simple. unicodeToUtf8() is used a lot internally regardless of what whether you use UTF-16 or UTF-8 yourself. For example, unicodeToUtf8() is used by sqlite3WinTempFileName() which is in turn used by sqlite3PagerOpentemp() -- I think you get the idea.


2007-Dec-20 00:29:33 by anonymous:
We've found that using CP_UTF8 fails on WinCE kernels that don't include SYSGEN_CORELOC (http://msdn2.microsoft.com/en-us/library/ms903883.aspx). To make the code handle any device it should be changed to:

  static WCHAR *utf8ToUnicode(const char *zFilename)
  {
    int nChar;
    WCHAR *zWideFilename;

    nChar = MultiByteToWideChar(CP_UTF8, 0, zFilename, -1, NULL, 0);
    if( nChar == 0 )
    {
      nChar = MultiByteToWideChar(CP_ACP, 0, zFilename, -1, NULL, 0);
      if( nChar == 0 )
      {
        DWORD dwError = GetLastError();
        OSTRACE2("MultiByteToWideChar() failed, last error: %d\n", dwError);
        return 0;
      }
    }
    zWideFilename = sqliteMalloc( nChar*sizeof(zWideFilename[0]) );
    if( zWideFilename==0 )
    {
      return 0;
    }
    nChar = MultiByteToWideChar(CP_UTF8, 0, zFilename, -1, zWideFilename, nChar);
    if( nChar==0 )
    {
      nChar = MultiByteToWideChar(CP_ACP, 0, zFilename, -1, zWideFilename, nChar);
      if( nChar==0 )
      {
        sqliteFree(zWideFilename);
        zWideFilename = 0;
      }
    }
    return zWideFilename;
  }
 
558 build active 2004 Jan anonymous   2007 Dec   4 4 Makefile.in should honor libdir and bindir edit
Please support non-standard installation layouts by honoring configure's --libdir and --bindir flags rather than hard-coding $(exec_prefix)/lib and $(exec_prefix)/bin. (For instance, the layout we often use on Solaris has parallel "lib" and "lib64" directories under a common prefix.)
2007-Dec-18 17:29:26 by anonymous:
Why is this ticket not solved? The patch is trivial and solves a real problem.

Thank you.


2007-Dec-18 17:54:46 by drh:
The patch does not apply to the current makefile. And I do not understand what the -libdir or -bindir options are for or what they are suppose to do so I do not know how to fix it.
 
368 code active 2003 Jun anonymous   2007 Dec   3 4 UPDATE trigger doesn't fire on INSERT OR REPLACE edit
After executing the following SQL, there will be nothing in table T2. I expect to see '1' there:

   CREATE TABLE T1 ( id, name );
   CREATE TABLE T2 ( id );
   CREATE TRIGGER T1A AFTER UPDATE ON T1
	BEGIN
	INSERT INTO T2 VALUES( new.id );
	END;

   INSERT INTO T1 VALUES (1, 'Hi');
   INSERT INTO T1 VALUES (2, 'There');
   INSERT OR REPLACE INTO T1 VALUES (1,'Me');
An INSERT trigger does fire on INSERT OR REPLACE if the item exists already -- I would expect an UPDATE trigger.


2004-Sep-21 17:15:37 by anonymous:
Still repros in 3.0.7 :-(


2007-Dec-17 21:45:03 by anonymous:
I would say that ON DELETE and ON INSERT better describes what really happens (and not ON UPDATE), because if there would be another columns in the '1' row, their values would not be preserved after INSERT OR REPLACE takes place, as the documentation of the ON REPLACE algorithm states: "When a UNIQUE constraint violation occurs, the pre-existing rows that are causing the constraint violation are removed prior to inserting or updating the current row". However, neither ON UPDATE nor ON DELETE trigger occurs, which still is a bug. Thank you.
 
916 new active 2004 Sep anonymous Unknown 2007 Dec   1 1 No delete notification for INSERT OR REPLACE edit
It would be nice if the "ON DELETE" trigger is called for the row substituted with a new one during REPLACE. Or, even better, one could add the OLD statement for the "ON INSERT" trigger and set it to point to the same row as NEW if a new row is inserted or to the deleting row if replace occurs. Thanks.
2007-Dec-17 21:36:40 by anonymous:
I have the same problem. My solution would be to stick with the documentation of the ON REPLACE algorithm: "When a UNIQUE constraint violation occurs, the pre-existing rows that are causing the constraint violation are removed prior to inserting or updating the current row". That is, to call ON DELETE trigger whenever rows are removed. Thank you, and keep going, you do wonderful job anyway.
 
2844 build active 2007 Dec anonymous   2007 Dec   4 1 lemon is being built without respecting LDFLAGS edit
lemon is being built without respecting LDFLAGS. I'm attaching a patch which fixes this bug.

In other words, why should we fix this? What problem is it causing?

2007-Dec-17 16:22:19 by drh:
Why is this important? What LDFLAGS settings might a user want to carry through into lemon?


2007-Dec-17 18:00:59 by anonymous:
> Why is this important?

It is considered to be be good practice to respect user's LDFLAGS. A user might want to have all executables and libraries built with identical LDFLAGS.

> What LDFLAGS settings might a user want to carry through into lemon?

A user might have LDFLAGS="-Wl,-O1,--hash-style=gnu,--sort-common"

You can read http://lwn.net/Articles/192082/.

Users can also use some other flags.

> In other words, why should we fix this? What problem is it causing?

It slightly increases the size of lemon executable and it slightly decreases performance.


2007-Dec-17 18:04:31 by drh:
lemon is used as an intermediate build tool in part of the SQLite build process. It is not a deliverable. If it runs a little slower or uses a little more memory, nobody cares. We only care if it gets the wrong answer.

Is it ever possible that the lack of LDFLAGS support might result in lemon getting the wrong answer?


2007-Dec-17 18:27:33 by anonymous:
Can you comment on Lemon bug in #2835? It produces 2 different sqlite3.c files depending on your malloc implementation.


2007-Dec-17 19:19:01 by anonymous:
> lemon is used as an intermediate build tool in part of the

> SQLite build process. It is not a deliverable. If it runs a

> little slower or uses a little more memory, nobody cares.

CFLAGS are respected when lemon is being built, so for consistency LDFLAGS also should be respected.

(The comment above was not created by me.)

 
2842 code active 2007 Dec anonymous   2007 Dec   1 1 .import does not recongnise NULL values edit
.import function fails to see NULL values in csv files as NULL values...instead they are treated as the string "NULL".

This is with .mode list and separator , But behaves similarly for .mode csv

Also if one outputs a table with NULL values to a file, then re-imports that file, again .import does not recognise the values as NULL, but as "NULL".

Everything here also applies to empty strings in files, e.g. instead of "NULL" using nothing...

This is a showstopper for us since we want to import a large amount of data with many tables containing NULL values. I can't see any valid reason for .import not to recognise the same syntax as the command line.

Note that something like:

sqlite3 my.db insert into MY_TABLE values (1,"foo","bar",NULL)

..works fine. It is just .import that appears to be broken.

2007-Dec-14 16:39:51 by rdc:
.import only inserts string values into database tables. If your column has a declared type that changes the columns affinity to numeric or integer, then those strings will be converted to numeric values by the SQLIte library.

The workaround is to simply insert a unique string where ever you want a NULL value, and then run an update that replaces those strings with real NULL values. If you inserted the string 'NULL' then do this after the .import

update t set field = null where field = 'NULL';

You will have to repeat this for each field in your table that might contain the 'NULL' string.

 
2841 todo active 2007 Dec anonymous   2007 Dec   1 1 The sqlite mailing list has become overrun by trolls edit
The sqlite mailing list is very useful. The S/N is at times a little high but nonetheless quite manageable.

Recently (see the DeviceSQL thread) it got really bad. Would moderation be unacceptable during these periods of time where people feel the need to protect their ego's?

The sqlite mailing list is primarily about sqlite (well, and lemon), not a marketing vector for other products? Surely they have their own lists and resources for that?

 
2721 code active 2007 Oct anonymous   2007 Dec   2 1 if db file is in a folder with non-ansi character some functions fail edit
If database file is located in directory with some non-ANSI characters (in my case with a Russian subdirectory c:\&#1052;&#1086;&#1080; &#1076;&#1086;&#1082;&#1091;&#1084;&#1077;&#1085;&#1090;&#1099;\Data_Jobs), or it's name is non-ansi. Some functions fail to execute sql.

For example (with defined UNICODE):

TCHAR sql[512];
_stprintf(sql, _T("INSERT INTO tab_SurveyedPoints (name, comment, code,")
		_T("coordinatetype, b, l, h, solutiontype, sigmah, sigmav)")
		_T(" VALUES ('%s','%s','%s',0,%lf,%lf,%lf,0,%lf,%lf);"),
		point.m_name.c_str(), point.m_description.c_str(), point.m_code.c_str(),
		point.m_coordinates.b, point.m_coordinates.l, point.m_coordinates.h,
		point.m_sigmah, point.m_sigmav);
int rc1 = sqlite3_prepare16(m_db, sqlfmt, -1, &stmt, (const void**)&pszTail);
rc != SQLITE_OK

But if I move the file to c:\My documents\Data_Jobs this works ok.

It's improbable behaviour, but I can't work around yet. Although, prepare() functions work ok as well in both cases.

Yuri Noyanov.

2007-Oct-11 19:33:34 by drh:
All string arguments to SQLite, and especially filename arguments, must be UTF-8 or UTF-16 (depending on the function). If you use string parameters which are not UTF-8 or UTF-16 (as appropriate) then the behavior of SQLite is undefined and probably not what you want.


2007-Oct-12 04:25:56 by anonymous:
but ALL programs to handle SQLite DBs (SQLIteBrowser, SQLite Control) fail to handle the files as well. Till I move the file to different directory !!!


2007-Oct-12 04:27:54 by anonymous:
Also I must note, that I CAN open the database, I CAN execute some SQLs with sqlite_prepare function OK. But sqlite_prepare16 FAILS if I just rename my database !!!


2007-Oct-12 04:31:46 by anonymous:
Also note to make my issue clearer:

sqlite_prepare16() with the same code either works OK either doesn't work. depends on database filename or folder path. The database is opened OK in both cases (I used utf8 conversion). sql_prepare() works ok in both cases.


2007-Oct-13 06:37:43 by anonymous:
That appears to be only with INSERT sql statement. Both SELECT and UPDATE work fine with sqlite_prepare16.
 
2831 new active 2007 Dec anonymous   2007 Dec   3 4 alter view edit
View can't be used after ALTER RENAME TO:

SQLite version 3.5.3
Enter ".help" for istructions
sqlite> create table t(a);
sqlite> create view v1 as select * from t;
sqlite> alter table v1 rename to v2;
sqlite> select * from v2;
SQL error: no such table: v2
sqlite> select * from v1;
SQL error: no such table: v1
sqlite> .schema
CREATE TABLE t(a);
CREATE VIEW v1 as select * from t;
sqlite> select * from sqlite_master;
table|t|t|2|CREATE TABLE t(a)
view|v1|v1|0|CREATE VIEW v1 as select * from t
This is a feature request, not a bug.


2007-Dec-11 18:40:17 by anonymous:
Notice that alter table doesn't return an error. After the command neither v1 nor v2 can be used.


2007-Dec-13 08:18:16 by danielk1977:
[4623] improves the situation by returning an error when the user attempts to rename a view.

One reason this feature (renaming views) is not a high priority is because a view can be dropped and recreated with a different name efficiently. This was not the case with tables.

 
2825 code active 2007 Dec anonymous   2007 Dec   3 3 FormatMessage (win32) should use extra flag and convert from Unicode edit
The call to FormatMessageA in the win32 source code needs to have the flags changed from: FORMAT_MESSAGE_FROM_SYSTEM to FORMAT_MESSAGE_FROM_SYSTEM | FORMAT_MESSAGE_IGNORE_INSERTS

This ensures that any system messages that expect arguments do not try to grab the argument from some random memory location.

ref: http://blogs.msdn.com/oldnewthing/archive/2007/11/28/6564257.aspx

2007-Dec-06 14:07:53 by anonymous:
I also noticed that the result is NOT converted to UTF-8.

FormatMessageA returns the text in the local ANSI codepage. FormatMessageW should be used on NT systems, and either result should be converted to the SQLite UTF-8 default.


2007-Dec-11 00:34:37 by anonymous:
to simplify what is meant even more...

http://www.sqlite.org/cvstrac/fileview?f=sqlite/src/os_win.c&v=1.118

Search for FormatMessageA (only 1 instance)

  -    FORMAT_MESSAGE_FROM_SYSTEM,
  +    FORMAT_MESSAGE_FROM_SYSTEM | FORMAT_MESSAGE_IGNORE_INSERTS,

No breakage, ensures that no crashes with some messages (e.g. filesystem errors). The encoding issue should be addressed separately.


2007-Dec-11 01:27:07 by anonymous:
The function should be changed to the following to correctly handle the conversion from Unicode/MBCS.

static void winDlError(sqlite3_vfs *pVfs, int nBuf, char *zBufOut){
  int error = GetLastError();
#if OS_WINCE
  if( error>0x7FFFFFF ){
    sqlite3_snprintf(nBuf, zBufOut, "OsError 0x%x", error);
  }else{
    sqlite3_snprintf(nBuf, zBufOut, "OsError %d", error);
  }
#else
  if( isNT() ){
    LPWSTR zWinTemp = NULL;
    DWORD dwLen = FormatMessageW(
      FORMAT_MESSAGE_ALLOCATE_BUFFER | FORMAT_MESSAGE_FROM_SYSTEM | FORMAT_MESSAGE_IGNORE_INSERTS,
      NULL,
      error,
      0,
      (LPWSTR) &zWinTemp,
      0,
      0
    );
    if (dwLen > 0) {
      char * zOut = unicodeToUtf8(zWinTemp);
      LocalFree(zWinTemp);
      sqlite3_snprintf(nBuf, zBufOut, "%s", zOut);
      free(zOut);
    }
  }else{
    LPSTR zWinTemp = NULL;
    DWORD dwLen = FormatMessageA(
      FORMAT_MESSAGE_ALLOCATE_BUFFER | FORMAT_MESSAGE_FROM_SYSTEM | FORMAT_MESSAGE_IGNORE_INSERTS,
      NULL,
      error,
      0,
      (LPSTR) &zWinTemp,
      0,
      0
    );
    if (dwLen > 0) {
      char * zOut = mbcsToUtf8(zWinTemp);
      LocalFree(zWinTemp);
      sqlite3_snprintf(nBuf, zBufOut, "%s", zOut);
      free(zOut);
    }
  }
#endif
}
 
2821 new active 2007 Dec anonymous   2007 Dec   3 4 hashtable indicies edit
It would be nice to implement non btree indices. I.e. CREATE INDEX ON table(rowid) AS HASH. Using a hashtable's O(1) properties, you could use the index for very quick lookups when one result is expected. This does have the tradeoff that a hashtable index has no ordering properties (can not be used for sorts or non-equality searching). However, it would be a huge win when you have 250,000 rowids in memory, and you want to go fetch another column in the database for each one of those rowids (SELECT * FROM table WHERE rowid=?).
2007-Dec-03 21:58:01 by anonymous:
For 250,000 rows I doubt you would see that much of an improvement (try it.)

You'll almost certainly find log_n is going to be fairly fast (especially for large n.)

I personally would prefer some sort of 'virtual' index though, that could be a hash or actually from a user-supplied function so that I can index large blobs by some function (i.e. a hash).

And yes, this would be an incompatible file-format change and it's not clear how to update an index when the function isn't loaded (i.e. db reopened with that function.) Perhaps mark the index as 'stale' and ignore it until the function loads then you can do the updates.

Of course this starts to get quite complicated.


2007-Dec-03 22:12:17 by anonymous:
Everything in sqlite depends on btree indexes. You're talking a major rewrite if you support hash-based or other indexing.
 
2814 code active 2007 Nov anonymous   2007 Dec   3 3 _XOPEN_SOURCE again edit
Ideally setting _XOPEN_SOURCE should be an opt-in detected by configure, rather than a hardcoded opt-out as it is now. I find you create more problems in setting it than just leaving it out on modern platforms.

Can you please give users the option of not defining _XOPEN_SOURCE at all?

+#ifndef SQLITE_DONT_DEFINE_XOPEN_SOURCE
 #if !defined(_XOPEN_SOURCE) && !defined(__DARWIN__) && SQLITE_THREADSAFE
 #  define _XOPEN_SOURCE 500  /* Needed to enable pthread recursive mutexes */
 #endif
+#endif
2007-Dec-01 09:23:15 by anonymous:
Also when using Python, it sets _XOPEN_SOURCE to 600. No idea what the 500 vs 600 difference is about.


2007-Dec-01 15:58:28 by anonymous:
I've used a couple of different Linux OSes and _XOPEN_SOURCE is not needed. Maybe it's for OSes more than 5 years old. Recursive mutexes are pretty much standard these days since the popularity of Java which uses them extensively.


2007-Dec-01 17:21:05 by drh:
See also tickets #2673, #2681, and #2741.


2007-Dec-02 02:08:26 by anonymous:
On Linux, PTHREAD_MUTEX_RECURSIVE is the same as PTHREAD_MUTEX_RECURSIVE_NP:

    PTHREAD_MUTEX_RECURSIVE = PTHREAD_MUTEX_RECURSIVE_NP,

Since PTHREAD_MUTEX_RECURSIVE_NP is always available, you could avoid defining _XOPEN_SOURCE and use this code instead:

-        pthread_mutexattr_settype(&recursiveAttr, PTHREAD_MUTEX_RECURSIVE);
+        pthread_mutexattr_settype(&recursiveAttr,
+#ifdef linux
+          PTHREAD_MUTEX_RECURSIVE_NP
+#else
+          PTHREAD_MUTEX_RECURSIVE
+#endif
+        );

2007-Dec-02 02:17:22 by anonymous:
A quick google search reveals how various projects deal with this recursive mutex declaration problem (in no particular order):

  • #define _XOPEN_SOURCE 500 and use PTHREAD_MUTEX_RECURSIVE
  • #define _XOPEN_SOURCE 600 and use PTHREAD_MUTEX_RECURSIVE
  • #define _GNU_SOURCE and use PTHREAD_MUTEX_RECURSIVE
  • don't define anything and use PTHREAD_MUTEX_RECURSIVE_NP on linux, and PTHREAD_MUTEX_RECURSIVE elsewhere.

Unfortunately, since PTHREAD_MUTEX_RECURSIVE is an enum on Linux, so you can't use the #ifdef PTHREAD_MUTEX_RECURSIVE compile-time technique.

 
2813 build active 2007 Nov anonymous   2007 Nov   1 1 compile error on Windows CE edit
environment: visual c++ 2005 window ce 6.0 customize sdk sqlite-amalgamation-3_5_3

I get error:

Error 27 error C2040: 'localtime' : 'tm *(const time_t *)' differs in levels of indirection from 'int ()' d:\SubProjects\Sqlite\sqlite3.c 18574

but if I add code in line 7095:

struct tm *__cdecl localtime(const time_t *t);

then Success!

 
2809 code active 2007 Nov anonymous   2007 Nov   1 1 PRAGMA collation_list shows unregistered collations edit
As presented on the mailing list:

Imagine that a SQLite3 database opened in a custom application with a registered a collation sequence named "unknown" has created the following table:

  CREATE TABLE a (b COLLATE unknown);

Now open this table in the default SQLite3 CLI. Up to here, everything works as expected.

Next issue "PRAGMA collation_list;" and notice that "unknown" lists next to the other registered collations, even though "unknown" is not registered with the default SQLite3 CLI:

  sqlite> PRAGMA collation_list;
  0|unknown
  1|NOCASE
  2|BINARY

Responses from the mailing list indicate that this is not the expected behaviour. "PRAGMA collation_list;" should list registered collations only.

2007-Nov-28 16:12:17 by anonymous:
I don't think this is a bug. If the CLI is not aware of the collation, it should not process the query that makes use of the collation because it would certainly be wrong if it simply ignored the collation. This is not unlike a user-registered SQL function that does not exist in the CLI. I would not expect or want the sqlite3 CLI to ignore the unknown function, nor would I want the CLI to process queries ignoring the custom collation.
 
2810 code active 2007 Nov anonymous   2007 Nov   1 1 Unregistered collation problems with simple subselects edit
As discussed on the mailing-list:

Imagine that a SQLite3 database opened in a custom application with a registered a collation sequence named "unknown" has created the following table:

  CREATE TABLE a (b COLLATE unknown);

Now open this table in the default SQLite3 CLI. Up to here, everything works as expected. Simple queries like "SELECT * FROM a;" work fine.

But subselects, in their most basic form and with no sorting or comparisons, result in an error:

  sqlite> INSERT INTO a VALUES ('one');

  sqlite> SELECT * FROM a, (SELECT * FROM a);
  SQL error: no such collation sequence: unknown

  sqlite> SELECT * FROM (SELECT * FROM a);
  SQL error: no such collation sequence: unknown

  sqlite> SELECT *, * FROM a;
  one|one

This is surprising because the collation sequence should not matter to the queries. In fact, the union without the subselect works just fine and without errors.

To demonstrate, here is the explain output of a table with a registered collation sequence. No mention of the collation name here:

  sqlite> CREATE TABLE b (b collate nocase);
  sqlite> EXPLAIN SELECT * FROM b, (SELECT * FROM b);
  0|Goto|0|17|
  1|Integer|0|0|
  2|OpenRead|0|3|
  3|SetNumColumns|0|1|
  4|Integer|0|0|
  5|OpenRead|2|3|
  6|SetNumColumns|2|1|
  7|Rewind|0|14|
  8|Rewind|2|13|
  9|Column|0|0|
  10|Column|2|0|
  11|Callback|2|0|
  12|Next|2|9|
  13|Next|0|8|
  14|Close|0|0|
  15|Close|2|0|
  16|Halt|0|0|
  17|Transaction|0|0|
  18|VerifyCookie|0|4|
  19|TableLock|0|3|b
  20|Goto|0|1|
  21|Noop|0|0|
 
2808 doc active 2007 Nov anonymous   2007 Nov   4 2 Documentation GIF images take up too much space edit
The GIF format used for the documentation images takes up too much space. I believe that this is in strict contrast to the "small" feature of SQLite.

Converting the GIFs to PNG images saves up to 181 KB (64%) of the documentation storage space:

  Documentation images as is (mostly GIF): 381 KB == 100%
  Documentation images as PNG:             292 KB ==  77%
  Documentation images as PNG 16 colors:   137 KB ==  36%

16 colors are more than plenty -- the diagrams show no visibility degradation. You might even cut it down to 8 colors to save even more ...

 
2223 code active 2007 Feb scouten   2007 Nov   3 3 pragma auto_vacuum doesn't survive .dump & reconstitute edit
When you run sqlite3 path/to/database .dump, it does not contain pragma auto_vacuum even if that option was chosen when creating the source database.
2007-Feb-08 18:13:27 by scouten:
We wonder if other pragmas are also not being propogated.


2007-Feb-08 18:53:42 by anonymous:
No pragmas are output from .dump. SQLite should have a .dump_with_pragmas command or equivalent.


2007-Nov-27 02:11:05 by anonymous:
auto_vacuum is especially important since you need to specify it before loading the tables in the dump; if you notice that it's missing after loading a dump, it's too late.

I'd also appreciate user_version surviving.

Would patches be welcome for these? I would be happy to contribute one: mail glasser@davidglasser.net.

 
2793 code active 2007 Nov anonymous   2007 Nov   3 3 fts3 lacks scoping edit
It would be nice if the fts3 symbols could optionally be made private/static as the rest of the sqlite3 library. Not sure why sqlite3_api becomes public when used with the amalgamation, for that matter.

  make TOP=`pwd` BCC=gcc TCC=gcc AR=ar RANLIB=echo NAWK=gawk -f \
    main.mk sqlite3.h sqlite3.c fts3amal.c
  cat fts3amal.c >> sqlite3.c
  gcc -DSQLITE_THREADSAFE -DSQLITE_API=static -DSQLITE_PRIVATE=static \
    -DSQLITE_EXTERN=static -DSQLITE_ENABLE_FTS3 -c sqlite3.c
  nm sqlite3.o | grep -v ' [trUbd] '

  00000004 C sqlite3_api
  00064da2 T sqlite3Fts3HashClear
  000652a4 T sqlite3Fts3HashFind
  00064d60 T sqlite3Fts3HashInit
  0006533b T sqlite3Fts3HashInsert
  00064b4c T sqlite3Fts3Init
  00066b34 T sqlite3Fts3InitHashTable
  000669bd T sqlite3Fts3PorterTokenizerModule
  0006702d T sqlite3Fts3SimpleTokenizerModule
 
285 build active 2003 Apr anonymous Unknown 2007 Nov   2 2 Configure doesn't honour LDFLAGS during build edit
Right now the configure script in the 2.8.0 tar.gz and CVS can't quite build a working SQLite when Fink is installed - it gets confused with finding readline.

(I've not been able to find the right places to add these changes).

* if LDFLAGS is set it is used during the check phase of configure but it isn't used during the build, ie. LDFLAGS doesn't get into the Makefile, this leads to the readline support being turned on but the libraries not being available at link time

* paths searched for readline.h should include /sw

 
2791 code active 2007 Nov anonymous   2007 Nov   1 1 Allow building FTS[123] as part of sqlite library with configure edit
See attached patch.
 
2787 build active 2007 Nov anonymous   2007 Nov   4 5 sqlite3.pc is not remade edit
the subject says it all, there is no make rule to rebuild the sqlite3.pc from the .in file. it is only possible by hand (./config.status sqlite3.pc)
2007-Nov-22 12:17:50 by drh:
I don't know what the sqlite3.pc file does. I certainly do not use it for any of my builds on Linux, Mac OSX, or windows. Why should I leave it in the source tree? Isn't the best solution to this problem to simply delete the file?


2007-Nov-22 17:14:27 by anonymous:
It is indirectly used by pkgconfig. Here's some info on pkgconfig:

http://pkg-config.freedesktop.org/wiki/


2007-Nov-23 15:22:32 by drh:
Could somebody who understands what the sqlite3.pc file is used for suggest a makefile rule for rebuilding it?
 
2770 code active 2007 Nov anonymous   2007 Nov   1 1 Problem with BLOB in 3.5.x ? edit
After I've switched from 3.3.18 to 3.5.2, selecting from table which contains BLOB LONGER THAN ABOUT 990 BYTES returns error "SQL logic error or missing database" after call to _sqlite3_step().

I'm using preprocessed sources downloaded from here. DEBUG build of preprocessed sources works correctly, problem is only in RELEASE build. I'm using VC6.0 to compile.

Any idea what could be wrong? Thank you!

Can you try to reproduce this with the sqlite shell tool? Thanks.

Large blobs work for me with both release and debug builds (not msvc though, gcc/linux).


2007-Nov-12 18:41:37 by anonymous:
sqlite3.exe provided here works with the database. Problem is only with release build (static library linked into test application). Here is test app which exits with "Error 1" in release build:

  int main(int argc, char* argv[])
  {
	int rc;
	sqlite3* db;
	sqlite3_stmt* stmt;

	rc = sqlite3_open("n2.db3", &db);

	rc = sqlite3_prepare(db, "CREATE TABLE [ttt] ([bbb] BLOB)", -1, &stmt, 0 );
	rc = sqlite3_step(stmt);
	rc = sqlite3_reset(stmt);

	char text[10000],query[20000];
	strnset(text,'a',sizeof(text)-1);
	sprintf(query,"insert into [ttt] values (?)");

	rc = sqlite3_prepare(db, query, -1, &stmt, 0 );

	rc = sqlite3_bind_blob(stmt,1,text,sizeof(text), SQLITE_TRANSIENT);

	rc = sqlite3_step(stmt);
	rc = sqlite3_reset(stmt);

	rc = sqlite3_prepare(db, "select * from ttt", -1, &stmt, 0 );

	rc = sqlite3_step(stmt);

	if (rc == SQLITE_ROW)
	{
		printf("%s: OK",sqlite3_column_text(stmt,1));
	}
	else
	if (rc == SQLITE_DONE)
	{
		printf("DONE");
	}
	else
	{
		printf("Error %d",rc);
	}

	return 0;
}


2007-Nov-12 18:56:24 by drh:
You should be using sqlite3_finalize() instead of sqlite3_reset(). You are leaking memory. Also, you should use sqlite3_prepare_v2() to avoid problems with changing schemas.

But even without those fixes, I cannot reproduce the problem on Linux.


2007-Nov-12 19:39:31 by anonymous:
Suggested fixes didn't help.

I've tried to debug it. It fails in btree.c, line 3056:

if( offset+amt > nKey+pCur->info.nData ){ /* Trying to read or write past the end of the data is an error */ return SQLITE_ERROR; }

there seems to be different values in release mode. My debugger does not show values of variables in release mode, so I can be wrong, but it seems in release offset is 5 and in debug it is 4. There can be something wrong with compilation, I'll try to figure this out tomorrow.

BTW compilation of static libraty in VC6.0 gives 185 warnings. I don't know if it is ok, it haven't caused problems in older sqlite


2007-Nov-13 08:47:28 by anonymous:
I've turned off "Maximize Speed" option - this is causing the problem. No optimizations and optimize for size seems to be working. But it still makes me nervous :(( I really don't need corrupted database and now I hope it won't slow down too much. Unfortunately old library does not implement replace function so I don't want to switch back. This could be warning to others, I'm using VC++ 6.0 SP 6.

Thank you for your time.


2007-Nov-22 17:20:31 by anonymous:
I have exactly the same problem here (win XP, vc6 SP2) when I link against my sqlite static or dynamic library in release.

I have also used boundschecker to check sqlite, and it detects many dangling pointers ! But the strange thing is that I cannot find why these pointers are dangling, here an example:

  In prepare.c@188
  pTab = sqlite3FindTable(db, zMasterName, db->aDb[iDb].zName);

Boundchecker say that zMasterName is a dangling pointer, previously released here:

  in build.c@711:
  void sqlite3StartTable(
    Parse *pParse,   /* Parser context */
    Token *pName1,   /* First part of the name of the table or view */
    Token *pName2,   /* Second part of the name of the table or view */
    int isTemp,      /* True if this is a TEMP table */
    int isView,      /* True if this is a VIEW */
    int isVirtual,   /* True if this is a VIRTUAL table */
    int noErr        /* Do nothing if table already exists */
  ){
  }

It does not make sens for me, maybe it a false positive from boundchecker, but it is weird.

I don't know if these "errors" are related to the "blob" bug in release mode. I will try to debug these error with some "printf" in release mode.

Note: The provided dll (the one from the sqlite site) does not have this "bug".


2007-Nov-22 18:27:35 by anonymous:
More info: It seems that there is a bug in the VC6 (SP6) compiler.

In btree.c, line 3056:

  if( offset+amt > nKey+pCur->info.nData ){
    /* Trying to read or write past the end of the data is an error */
    return SQLITE_ERROR;
  }

After adding some printf around, It seems that the "speed optimization" compilation flag of VC6 changes the code order in a way that the offset variable is miss incremented !!

Two remarks:

  • I've traced the calling function, sqlite3BtreeData, and the it call accessPayload with the good offset value
  • VC6 produces an internal error: "fatal error C1001: INTERNAL COMPILER ERROR" in the accessPayload function, if I try to access the offset value before this line:

  aPayload = pCur->info.pCell + pCur->info.nHeader;

A dirty workaround could be to change the code order or the local var usage. I'm trying ....

 
2776 warn active 2007 Nov anonymous   2007 Nov   5 1 mailing list edit
sqlite-users-digest doesn't work. It remembers registered addresses but doesn't send any emails.
 
2771 code active 2007 Nov anonymous   2007 Nov   4 4 Lemon: Generated parser needs stdlib.h (not in default template) edit
I tested a simple do-nothing parser just to get lemon output, and this doesn't compile (if warnings treated as errors) because of non declaration of the memset() function for the following statment:

memset(&yygotominor, 0, sizeof(yygotominor));

(added for the resolution of SQLite ticket #2172).

The lempar.c just include stdio.h, it would suffice to add stdlib.h to get the memset() declaration (even if all real parsers must include stdlib.h to get something really working).

2007-Nov-14 21:02:25 by anonymous:
Sorry, the needed header is string.h, not stdlib.h :-)
 
2766 code active 2007 Nov drh   2007 Nov   1 1 TCL transaction started from within a query does not commit edit
This is a problem with the TCL interface. Consider the following TCL script:

   file delete -force test.db test.db-journal
   sqlite3 db test.db
   db eval {
     CREATE TABLE t1(x,y);
     INSERT INTO t1 VALUES(1,2);
     CREATE TABLE t2(a,b);
     INSERT INTO t2 VALUES(8,9);
   }
   db eval {SELECT * FROM t1} {
     db transaction {
       db eval {UPDATE t2 SET a=a*2}
     }
   }

The [db transaction] statement starts a transaction and it is suppose to commit the tranaction at the end of the code block. But because the transaction started while a query was active, the tranaction is unable to commit. The TCL interface never commits the tranaction nor does it give any kind of error indication.

It is unclear if an error should be returned or if the commit should be deferred until outer query finishes.

If the code within the [db transaction] block throws an error, we really need the transaction to rollback right away. Perhaps there should be a new API that cancels all pending queries. Perhaps a call to sqlite3_interrupt() would suffice for this. Need to investigate further....

 
2417 new active 2007 Jun anonymous   2007 Nov drh 3 3 Idea for read write concurrency. edit
This is not a problem, but rather an idea on how to resolve the reader/writer concurrency issues encountered in sqlite.

The idea is to allow a reader and writer to work concurrently not blocking each other. Dual writers would of course block.

When a write occurs: 1. block level changes are made to the database file. 2. Pre-image of that change is written to the journal.

Readers: 1. File I/O on the main file would occur normally. 2. If the block encountered is "new" ie one that was written out by the writer. Then get the original block from the Journal file.

In order to determine "NEW" a change number could be put on each block. When a READ (select) begins it would first determine the starting global change number. (maybe on the master block?)

      When a write occurs it would read the Master blocks change number. (increment this in memory) and use write new blocks with the new value.  At commit. The Master block would be updated and the txn journal marked for purge if there are pending reads.

-- Drawbacks: Reading becomes dependent upon the txn journal.

-- Implementation of BLOCK level versioning may ultimately be a simpler approach. Idea would be for a seperate file conaining versioned blocks. This file could be accessed instead of the txn journal.

2007-Nov-08 15:12:00 by anonymous:
DRH: Also unaddressed in the proposal is how to locate a particular page within the journal file without having to do (performance killing) sequential scan of the possible very large file.

Resolution of page access to avoid sequential scans of Txn Journal.

When a writer is making the modification to a page first it writes the original page to the journal. At this point the journal file offset location is known. Save this offset in the "NEW" page being written into the database file. This implements a backwards chaining of pages into the txn journal.

The reader upon reading the db file page would recognize (see above) that the page is dirty. Acquire the txn journal offset from the dirty page, Read the page from the journal until the starting page is found. This would eliminate any sequential scanning, but may require more than one read request.

 
2763 build active 2007 Nov anonymous   2007 Nov   4 4 AIX build failure due to explicit _XOPEN_SOURCE definition edit
Our AIX machine has started having problems building the new sqlite 3.5.2 source. I'm no AIX expert, but I did some digging and I think it's due to sqliteInt.h defining _XOPEN_SOURCE without also defining _ALL_SOURCE.

AIX system header files usually define this themselves (in /usr/include/ standards.h) but only if _XOPEN_SOURCE was previously undefined.

Has anyone else come across this bug? I'm willing to believe it's our build system if not; we can work around it by setting _ALL_SOURCE on the command-line, but I thought it might be useful to raise a bug anyway.

See also tickets #2673, #2681, and #2741.

I do not have access to an AIX machine and so have no ability to debug this problem. If you have suggested patches we will consider them. Otherwise, there is not much we can do about this ticket.

 
2760 new active 2007 Nov anonymous   2007 Nov   5 4 request: sqlite3_unlink() to delete db files. edit
Hi!

Today i came across a use case where i would like client code to be able to delete an underlying sqlite3 db, but that code doesn't have immediate access to the file name of that db (without refactoring the db wrapper code).

An interesting feature addition would, IMO, be:

int sqlite3_unlink( sqlite3 * db, bool closeTheFile );

Unlinks the file associated with the given database. It does not alter the database in any way (thus is it a no-op on a :memory: database). The closeTheFile flag specifies whether the file handle associated with db should also be closed (and thus db must also be closed), or just unlinked (e.g., as temporary databases are unlinked right after creation but kept open).

After browsing through the VFS API a bit, i see that there is an xDelete function, but i'm not sure if its semantics require that the underlying file handle be closed. i don't see an extra xClose member of VFS, so i assume that xDelete also handles closing the file handle. If these were split into two features, sqlite3_unlink() could be implemented very easily.

:)

 
2756 new active 2007 Nov anonymous   2007 Nov   1 1 allow vacuum to change pragma setting instead of using existing ones edit
we've got databases created with page_size of 1k, and we'd like to change that setting to 4k. vacuum creates a temporary db, attach it to the current connection, creates the tables (based on what's in the old db), and then selects from the old db and inserts into the new one.

vacuum does exactly what we need (creating a new db from an old one), but it re-uses the existing pragmas for page size, auto vacuum and reserved page size.

from sqlite.c, see sqlite3RunVacuum()

  sqlite3BtreeSetPageSize(pTemp, sqlite3BtreeGetPageSize(pMain),
     sqlite3BtreeGetReserve(pMain));

dr hipp points out that the the operands to vacuum are unused. from sqlite.c:

sqlite3VdbeAddOp(v, OP_Vacuum, 0, 0);

one solution would be to allow the user to specify the page size, reserve page size, and autovacuum as optional params to vacuum.

he had an idea of using the signedness of the first operand to represent the autovacuum setting (since after a table is created, you can change the setting from auto to incremental, but you can't change it from none to auto (or none to incremental)

 
2755 code active 2007 Nov anonymous   2007 Nov   3 3 trace interfere with transaction Tcl interface edit
When using the transaction method of the Tcl interface to the SQLite with a registered "trace" function, the stack trace is lost in case an error occurs inside the transaction.

As an example I provide two outputs, the first one without a registered trace function and the second one with one (in which it cannot be seen where the exception cames from):


First:

   > ./a.tcl
vorher
BUMMM
    while executing
"a"
    invoked from within
"db transaction {
        puts "vorher"
        a
        puts "nachher"
    }"
    ("uplevel" body line 1)
    invoked from within
"uplevel 1 [list db transaction {
        puts "vorher"
        a
        puts "nachher"
    }]"
    (procedure "b" line 2)
    invoked from within
"b"
    (file "./a.tcl" line 28)


Second:

   > ./a.tcl
BEGIN
vorher
ROLLBACK

    while executing
"db transaction {
        puts "vorher"
        a
        puts "nachher"
    }"
    ("uplevel" body line 1)
    invoked from within
"uplevel 1 [list db transaction {
        puts "vorher"
        a
        puts "nachher"
    }]"
    (procedure "b" line 2)
    invoked from within
"b"
    (file "./a.tcl" line 28)

********

A scritp that demostrates this behaviour is attached. The only workaround is not to trace.

Thanks

 
2753 code active 2007 Nov anonymous   2007 Nov drh 3 3 Master journal files sometimes not deleted edit
In the 3.4.1 amalgamation, in vdbeCommit, the master journal file is created, and deleted at the end or if there is an error. But it looks like there is one case where it gets closed but not deleted. The code is:

    for(i=0; rc==SQLITE_OK && i<db->nDb; i++){
      Btree *pBt = db->aDb[i].pBt;
      if( pBt && sqlite3BtreeIsInTrans(pBt) ){
        rc = sqlite3BtreeCommitPhaseOne(pBt, zMaster);
      }
    }
    sqlite3OsClose(&master);
    if( rc!=SQLITE_OK ){
      sqliteFree(zMaster);
      return rc;
    }

It seems like that last bit should be:

    if( rc!=SQLITE_OK ){
      sqlite3OsDelete(zMaster);
      sqliteFree(zMaster);
      return rc;
    }
2007-Nov-01 19:26:28 by anonymous:
Should have read the comment above that code segment closer. Looks like this was by design. Trying to figure out what to do with the client that has hundreds of orphaned master journal files...
 
2716 new active 2007 Oct anonymous   2007 Oct   5 1 Create Clear Command edit
I want a command caled "clear" like in MySQL. This command should erase the screen and then put the sqlite pointer on top of the screen
2007-Oct-11 07:41:45 by anonymous:
How about a cookie instead?


2007-Oct-30 08:02:04 by anonymous:
Clearing the screen and moving the cursor are platform-dependent operations. On Unix they are not only platform-dependent, but also terminal-dependent. Thus such a feature does not really belong in the cross-platform and minimalistic sqlite3 shell (in my opinion).
 
2749 warn active 2007 Oct anonymous   2007 Oct   3 4 SQLITE_OMIT_FLOATING_POINT, int constants too large edit
Hiya!

i'm working from the 3.5.1 (CVS version) of the amalgamation. The code was pulled from CVS sometime around Oct 27th 2007.

Platform: Kubuntu Linux 7.04, i386-32, gcc 4.1.2.

The compiler output says it all:

  gcc -c -DSQLITE_OMIT_FLOATING_POINT sqlite3.c
  sqlite3.c: In function 'bestVirtualIndex':
  sqlite3.c:65531: warning: integer constant is too large for 'long' type
  sqlite3.c: In function 'bestIndex':
  sqlite3.c:65600: warning: integer constant is too large for 'long' type
  sqlite3.c: In function 'sqlite3WhereBegin':
  sqlite3.c:66223: warning: integer constant is too large for 'long' type
  sqlite3.c:66248: warning: integer constant is too large for 'long' type
  sqlite3.c:66254: warning: integer constant is too large for 'long' type

These warnings would seem to indicate potentially serious problems, though i admittedly have not investigated whether overflows are really fatal in the affected contexts.

 
2748 warn active 2007 Oct anonymous   2007 Oct   4 4 amalgamation: SQLITE_OMIT_ALTERTABLE warnings edit
Hiya!

i'm working from the 3.5.1 (CVS version) of the amalgamation. The code was pulled from CVS sometime around Oct 27th 2007.

The compiler output says it all:

  gcc -c -DSQLITE_OMIT_ALTERTABLE sqlite3.c
  sqlite3.c:6909: warning: 'sqlite3AlterRenameTable' used but never defined
  sqlite3.c:6916: warning: 'sqlite3AlterFinishAddColumn' used but never defined
  sqlite3.c:6917: warning: 'sqlite3AlterBeginAddColumn' used but never defined

The relevant code is:

  sed -ne '6909p;6916p;6917p' sqlite3.c
  SQLITE_PRIVATE void sqlite3AlterRenameTable(Parse*, SrcList*, Token*);
  SQLITE_PRIVATE void sqlite3AlterFinishAddColumn(Parse *, Token *);
  SQLITE_PRIVATE void sqlite3AlterBeginAddColumn(Parse *, SrcList *);

Platform: Kubuntu Linux 7.04, i386, gcc 4.1.2.

 
2392 code active 2007 May anonymous   2007 Oct   4 4 Reduce MemPage and PgHdr struct sizes. Better page memory utilization. edit
Patch to reduce sizeof(MemPage) below. It saves 8 bytes per cached page or in-memory page on Linux.

sizeof(MemPage) on Linux:

  original: 84
  patched:  76

Patched "make test" runs without regressions on Linux and Windows. Timings for "make test" (elapsed):

  original: 1:20.74
  patched:  1:20.22

Size of sqlite3.o when compiled from almalogmation with all sqlite features enabled with gcc flags -O3 -fomit-frame-pointer:

  original: 586976 bytes
  patched:  587880 bytes

Patched sqlite3.o is 904 bytes larger.

  Index: src/btreeInt.h
  ===================================================================
  RCS file: /sqlite/sqlite/src/btreeInt.h,v
  retrieving revision 1.4
  diff -u -3 -p -r1.4 btreeInt.h
  --- src/btreeInt.h      16 May 2007 17:28:43 -0000      1.4
  +++ src/btreeInt.h      30 May 2007 16:26:03 -0000
  @@ -269,15 +269,15 @@ typedef struct BtLock BtLock;
   */
   struct MemPage {
     u8 isInit;           /* True if previously initialized. MUST BE FIRST! */
  -  u8 idxShift;         /* True if Cell indices have changed */
     u8 nOverflow;        /* Number of overflow cell bodies in aCell[] */
  -  u8 intKey;           /* True if intkey flag is set */
  -  u8 leaf;             /* True if leaf flag is set */
  -  u8 zeroData;         /* True if table stores keys only */
  -  u8 leafData;         /* True if tables stores data on leaves only */
  -  u8 hasData;          /* True if this page stores data */
  -  u8 hdrOffset;        /* 100 for page 1.  0 otherwise */
  -  u8 childPtrSize;     /* 0 if leaf==1.  4 if leaf==0 */
  +  u8 hdrOffset:7;        /* 100 for page 1.  0 otherwise */
  +  u8 zeroData:1;         /* True if table stores keys only */
  +  u8 childPtrSize:3;     /* 0 if leaf==1.  4 if leaf==0 */
  +  u8 leaf:1;             /* True if leaf flag is set */
  +  u8 idxShift:1;         /* True if Cell indices have changed */
  +  u8 intKey:1;           /* True if intkey flag is set */
  +  u8 leafData:1;         /* True if tables stores data on leaves only */
  +  u8 hasData:1;          /* True if this page stores data */
     u16 maxLocal;        /* Copy of Btree.maxLocal or Btree.maxLeaf */
     u16 minLocal;        /* Copy of Btree.minLocal or Btree.minLeaf */
     u16 cellOffset;      /* Index in aData of first cell pointer */
2007-Sep-07 05:58:45 by anonymous:
Any word on applying this patch?

It saves 8 bytes per MemPage. For the default 2000 page cache it would save 16000 bytes. Larger page caches would save considerably more memory.


2007-Oct-26 21:08:03 by anonymous:
With this patch sizeof(PgHdr) is reduced by 4 bytes (from 48 bytes to 44 bytes) for gcc. No regressions. "make test" runs in the same amount of time.

Index: src/pager.c
===================================================================
RCS file: /sqlite/sqlite/src/pager.c,v
retrieving revision 1.393
diff -u -3 -p -r1.393 pager.c
--- src/pager.c 20 Oct 2007 13:17:55 -0000      1.393
+++ src/pager.c 26 Oct 2007 21:00:33 -0000
@@ -261,11 +261,11 @@ struct PgHdr {
   PgHdr *pNextHash, *pPrevHash;  /* Hash collision chain for PgHdr.pgno */
   PagerLruLink free;             /* Next and previous free pages */
   PgHdr *pNextAll;               /* A list of all pages */
-  u8 inJournal;                  /* TRUE if has been written to journal */
-  u8 dirty;                      /* TRUE if we need to write back changes */
-  u8 needSync;                   /* Sync journal before writing this page */
-  u8 alwaysRollback;             /* Disable DontRollback() for this page */
-  u8 needRead;                   /* Read content if PagerWrite() is called */
+  u8 inJournal:1;                /* TRUE if has been written to journal */
+  u8 dirty:1;                    /* TRUE if we need to write back changes */
+  u8 needSync:1;                 /* Sync journal before writing this page */
+  u8 alwaysRollback:1;           /* Disable DontRollback() for this page */
+  u8 needRead:1;                 /* Read content if PagerWrite() is called */
   short int nRef;                /* Number of users of this page */
   PgHdr *pDirty, *pPrevDirty;    /* Dirty pages */
 #ifdef SQLITE_ENABLE_MEMORY_MANAGEMENT

2007-Oct-26 21:20:09 by anonymous:
How about a compromise. What about some optional setting like?

  #ifndef OMIT_SQLITE_BITFIELDS
  # define SQLITE_BITFIELD(X) : X
  #else
  # define SQLITE_BITFIELD(X)
  #endif

...

  u8 inJournal SQLITE_BITFIELD(1); /* TRUE if has been written to journal */
 
2728 code active 2007 Oct anonymous   2007 Oct   4 4 Some indexes could contain pointers not the data edit
It would be nice if there was a class of index(-column) that would contain a reference to the data in (in the table) rather than a copy of the data.

This is useful where you have a need to index large strings or blobs and the size penalty of having 2+ copies ends up being very expensive.

Consider the cost of the implied index on:

create table t1 ( c1 BLOB UNIQUE);

and store 100s of 1MB objects. The raw file ends up being twice as large as you ideally would like it to be.

WHEN to use such indexes isn't entirely clear to me, not is the syntax for doing it in general, 30s of thought I can come up with:

  create table t1 ( c1 BLOB &UNIQUE);

and say

  create index idx1 on t2(c1, &c2, c3);

(meaning c2 would be done via reference, c1 & c3 as copies of the data). It's FAR from clear what other databases do here (I didn't look).

This would also be useful in cases where I have multiple indexes on tables like email (headers) and values which can be very large (100s of bytes in some cases (Subject:) and I might have 5 or more indexes, means a 100-byte column will take over 600 bytes (+ padding)). Allowing for references would be slower in some cases but faster in others because of the smaller footprint and much grater CPU cache utilization.

2007-Oct-14 15:27:12 by anonymous:
This enhacement has been discussed on the mailing list before, and would break sqlite3 file format compatability.


2007-Oct-15 02:42:02 by anonymous:
This is non standard sql.

Also, the random access to pages to check indexes constraints (unique indexes, also primary keys) will trash the entire database performance.

If you need small index disk usage, consider using hashes to your data keys. it&#180;s the best that you could do to solve your problem.


2007-Oct-18 06:07:58 by anonymous:
Using a hash is what I've tried doing in a couple of cases. It's not very ideal. For one thing is ordering is messed up.

I wonder about a virtual index concept, where you can define a function that takes the column value and returns something to actually store in the index?

I'm told oracle has this feature, so whilst it's non-standard there is some precedent (albeit in a very different space).

 
2736 build active 2007 Oct anonymous   2007 Oct   2 2 build problems on freebsd edit
on freebsd:

--disable-threads does not work. it is accepted as a valid option but no defs are added to the makefile -lgcc needs to be included in SHLIB_LD_LIBS pkgIndex.tcl is not built when -DSQLITE_THREADSAFE=0 is added manually, this causes the install target to fail.

2007-Oct-17 21:16:23 by anonymous:
I forgot to mention this is with the TEA version
 
2715 code active 2007 Oct anonymous   2007 Oct   1 1 no authorization needed to remove authorizer edit
there should be a new auth code created and the auth function should be consulted for permission for removal.
2007-Oct-10 01:08:48 by drh:
I'm assuming that this feature request comes from RockShox and that the development language is Tcl.

No. If your adversary has the ability to invoke the interface that removes an authorizer, then you system is already pwned.

What you really need is the ability to [interp alias] the eval method into a safe interpreter. That way you can:

  • Open the database in the main interpreter
  • Set up the authorizer in the main interpreter to invoke a script in the main interpreter
  • Set up the [interp alias] so that the safe interpreter can do [db eval ...] but not [db auth ...]

It seems like an "-interp" option on the "eval" method of the database connection object would likely be the right interface. Or perhaps there should be separate "safeeval" method. Either way, it has been years and years since I have done anything with safe interpreters so I will have to look into what needs to be done to make that happen.


2007-Oct-17 20:11:23 by anonymous:
ok i think i agree with that.

currently you cannot use an interp alias since the target command runs in the target interp and all your variables and commands are in the wrong scope. this means one needs to load sqlite again in the new interp, and sqlite will not load in a safe interp so a regular interp is required.

to be useful, a -interp flag would need to execute in the current scope of the interp and not the global scope.

 
2714 code active 2007 Oct anonymous   2007 Oct drh 3 4 Shell cannot import large files greater than 2 GB [patch] edit
If I issue an .import command within the SQLite shell and the file size is larger than 2 GB, the shell gives the error message "cannot open file: foo.txt".

Putting #define _FILE_OFFSET_BITS 64 before the #include statements in shell.c fixes this problem under Linux.

I tested this solution with a 2.6 GB file under Ubuntu Feisty Fawn 7.04 with Linux kernel 2.6.20-16-generic.

2007-Oct-12 18:48:35 by drh:
I'm not convinced SQLite should be able to import files larger than 2GiB. Does anybody really ever need to read more than 2GiB of SQL text? That is a lot of SQL, don't you think? Is this really a desirable feature? Can you not split it up into two or more smaller files?


2007-Oct-14 09:25:42 by anonymous:
I'm not 100% certain about this but:

  #define _FILE_OFFSET_BITS 64

isn't necessarily correct, better to tweak the makefile(s):

  CFLAGS += $(shell getconf LFS_CLFAGS)

(similar for LDFLAGS) should work on 32/64 bit machines and different platforms.

For shell.c I really wouldn't bother and think this should be left alone. Using .import for load more than 2GB makes me things the shell should be fleshed out and extended for this sort of abuse, handy as it is right now, it's not a full SQL shell but a minimalist shell and a useful example).


2007-Oct-16 14:08:09 by anonymous:
I am the original poster. I use the shell to import CSV files in the 3-4 GiB range on a daily basis. I don't see the point of splitting the files or writing my own importer when the shell does a perfectly fine job once large file support has been activated.

With all due respect, I don't understand the reluctance to enable a useful feature. Shouldn't the user decide whether it is reasonable for his application to import big files? I work in the health care industry processing insurance claims for several very large companies. There's nothing broken or wrong with my application -- big files are just the nature of the data I'm handling.

Neither of you has proposed any disadvantage to enabling LFS in the shell. If there are no drawbacks, why not turn it on? If the core library can open large database files, why shouldn't the shell be able to import large ones?


2007-Oct-16 16:24:22 by anonymous:
For reasons also unknown to me, the authors appear to regard the shell as a largely unused sample program. Based on my personal experience in talking to various sqlite library users, it is the primary administrative front-end for sqlite. In particular, doing backups and merging databases.
 
2729 doc active 2007 Oct anonymous   2007 Oct   1 1 Lemon: %fallback, %wildcard, and @X uncodumented edit
I noticed that the lemon documentation does not mention the %fallback and %wildcard directives. Both are in the code and are apparently doing useful work in SQLite's parse.y. Can other users benefit from them as well?

The symbol @X is also undocumented. From a source code comment I read that it "If the argument is of the form @X then substituted the token number of X, not the value of X". A short documentation example would help to understand where and how it can be useful to apply this syntax.

Are there other nice but undocumented Lemon goodies lacking documentation?

 
2725 code active 2007 Oct anonymous   2007 Oct   1 1 memory leak in sqlite3_open_v2() when it fails edit
only happens with

flags = SQLITE_OPEN_READWRITE;

and when

  res = sqlite3_open_v2(sourcename, &conn, flags, NULL);

seems to leak 674 bytes per call

2007-Oct-15 07:07:07 by danielk1977:
Are you calling sqlite3_close(conn) after the error occurs?

All calls to sqlite3_open_v2() need to be matched by a call to sqlite3_close(), even if an error occurs.

 
2701 new active 2007 Oct anonymous   2007 Oct   5 5 Make INSERT-ing multiple rows MySQL-compatible edit
SQLite syntax allows to insert only one row with
    insert into test (a, b, c) values (1, 2, 3);

MySQL allows to insert multiple with

    insert into test (a, b, c) values (1, 2, 3), (4, 5, 6), (7, 8, 9) -- etc

But SQLite is also capable of inserting multiple by using INSERT...SELECT:

    insert into test (a, b, c) select 1, 2, 3 union select 4, 5, 6 -- etc

It would be nice to make INSERT statement syntactically compatible with MySQL, allowing to insert multiple rows with VALUES clause.

It can be implemented by simply translating multiple 'VALUES ()()()' to 'select union' - no serious change required at all.

2007-Oct-08 21:45:05 by anonymous:
You mean "UNION ALL", not "UNION".

UNION would remove duplicate rows, and create an ephemeral table that you don't want because it's less efficient.

Your idea is a good one and could be implemented largely in the parser. The number of VDBE opcodes would be quite large for such a statement. I wonder if that would present a problem.


2007-Oct-14 08:12:11 by anonymous:
Patch implementing multi-row INSERT statements against 3.5.1 source tree:

http://www.mail-archive.com/sqlite-users%40sqlite.org/msg28337.html

 
2727 build active 2007 Oct anonymous   2007 Oct   4 4 building with -malign-double causes strange behavior edit
sqlite 3.5.1 amalgamation has problems when enabling a wide set of compiler features on gcc (GCC) 4.1.2 (Ubuntu 4.1.2-0ubuntu4) on linux/i686 /w glibc-2.5 strange behavior occurs.

typical strangeness is that SQLITE_FULL is returned from sqlite3_prepare_v2() (called immediately after a statement was created with sqlite3_mprintf). This happens even though there is a reasonable amount of disk space free (6G).

the compile line looks like:
cc -g -pg -O1 -march=i686 -msse2 -malign-double -m128bit-long-double -momit-leaf-frame-pointer -minline-all-stringops -D_XOPEN_SOURCE=520 '-DVERSION="0.10r276M"' -Isqlite/ -DSQLITE_OMIT_LOAD_EXTENSION -DTHREADSAFE=0 -DSQLITE_OMIT_EXPLAIN -DSQLITE_ENABLE_COLUMN_METADATA -c -o sqlite/sqlite3.o sqlite/sqlite3.c

and valgrind reports:
==15323== Use of uninitialised value of size 4
==15323== at 0x80585B7: insertElement (sqlite3.c:13072)
==15323== by 0x807366E: sqlite3HashInsert (sqlite3.c:13290)
==15323== by 0x809FE72: unixOpen (sqlite3.c:15403)
==15323== by 0x80579A7: sqlite3OsOpen (sqlite3.c:8210)
==15323== by 0x806B12F: sqlite3BtreeFactory (sqlite3.c:21317)
==15323== by 0x80767BA: openDatabase (sqlite3.c:71237)
==15323== by 0x8076BD1: sqlite3_open (sqlite3.c:71337)
==15323== by 0x804BEAE: db_init (dbmgr.c:81)
==15323== by 0x804CE64: main (main.c:59)

my dbmgr.c:81:db_init() is: res=sqlite3_open(filename, &system_db);

and system_db=NULL and filename="zomg.sqlite" (string literal). so the parameters seem normal. if I turn off -malign-double everything works fine. (this "bug" also seems to be on 3.4.1 amalgamation)

2007-Oct-14 04:35:16 by anonymous:
This is a compiler issue. -malign-double creates problems for most programs. What are you trying to accomplish?
 
2708 code active 2007 Oct anonymous   2007 Oct   4 2 SQL error:disk I/O error edit
I cross-compile sqlite to embedded Linux,but after I insert data to the table ,it failed.the warning is "SQL error:disk I/O error".
2007-Oct-09 05:12:28 by anonymous:

  Why do you think it is SQLite error ??


2007-Oct-09 05:46:06 by danielk1977:
We'll need a bit more data than that to figure this out. Did earlier SQLite versions work? Can you post the entire output of the compile process so that we can see if there are any clues there? Can you run strace so that we can see if there really is an IO error, or at least when SQLite believes there to be one?
 
2705 code active 2007 Oct anonymous   2007 Oct   4 4 testfixture unresolved externals with SQLITE_OMIT_GET_TABLE edit
Cannot build/run "make test" with -DSQLITE_OMIT_GET_TABLE due to testfixture link error:

In function `test_get_table_printf':
./src/test1.c:526: undefined reference to `sqlite3_get_table'
./src/test1.c:541: undefined reference to `sqlite3_free_table'
collect2: ld returned 1 exit status
make: *** [testfixture] Error 1
 
2704 code active 2007 Oct anonymous   2007 Oct   4 4 "make test" aborts before completion with SQLITE_OMIT_BLOB_LITERAL edit
When compiled with -DSQLITE_OMIT_BLOB_LITERAL make test aborts with this error:

substr-2.5.2... Ok
./testfixture: near "'61626364656667'": syntax error
    while executing
"db eval "
    DELETE FROM t1;
    INSERT INTO t1(b) VALUES(x'$hex')
  ""
    (procedure "subblob-test" line 2)
    invoked from within
"subblob-test 3.1 61626364656667 1 1 61"
    (file "./test/substr.test" line 86)
    invoked from within
"source $testfile"
    ("foreach" body line 5)
    invoked from within
"foreach testfile [lsort -dictionary [glob $testdir/*.test]] {
  set tail [file tail $testfile]
  if {[lsearch -exact $EXCLUDE $tail]>=0} continue
  if..."
    (file "./test/quick.test" line 93)
make: *** [test] Error 1
 
2703 code active 2007 Oct anonymous   2007 Oct   3 4 make test does not work with SQLITE_OMIT_FLOATING_POINT edit
make test cannot run with SQLITE_OMIT_FLOATING_POINT. Shouldn't these tests be skipped?

./src/test1.c:1255: warning: passing argument 3 of ‘Tcl_GetDouble’ from incompatible pointer type
./src/test1.c: In function ‘sqlite3_mprintf_scaled’:
./src/test1.c:1284: warning: passing argument 3 of ‘Tcl_GetDouble’ from incompatible pointer type
./src/test1.c: In function ‘test_bind_double’:
./src/test1.c:2607: warning: passing argument 3 of ‘Tcl_GetDoubleFromObj’ from incompatible pointer type
./src/tclsqlite.c: In function ‘tclSqlFunc’:
./src/tclsqlite.c:728: warning: passing argument 3 of ‘Tcl_GetDoubleFromObj’ from incompatible pointer type
./src/tclsqlite.c: In function ‘DbObjCmd’:
./src/tclsqlite.c:1609: warning: passing argument 3 of ‘Tcl_GetDoubleFromObj’ from incompatible pointer type

...

$ ./testfixture test/select1.test
select1-1.1... Ok
select1-1.2... Ok
select1-1.3... Ok
select1-1.4... Ok
select1-1.5... Ok
select1-1.6... Ok
select1-1.7... Ok
select1-1.8... Ok
select1-1.8.1... Ok
select1-1.8.2... Ok
select1-1.8.3... Ok
./testfixture: near ".": syntax error
    while executing
"db eval {INSERT INTO test2(r1,r2) VALUES(1.1,2.2)}"
    ("uplevel" body line 1)
    invoked from within
"uplevel [list $db eval $sql]"
    (procedure "execsql" line 3)
    invoked from within
"execsql {INSERT INTO test2(r1,r2) VALUES(1.1,2.2)}"
    (file "test/select1.test" line 69)
 
2684 code active 2007 Oct anonymous   2007 Oct   1 1 Accessing sqlite from an NT service will lock the complete databse. edit
Accessing sqlite from a NT service (application 1) will lock the complete database. Any other process trying to open an sqlite db (application 2) will get error "80004005 unable to lock database"

If application 1 runs as normal application, started by local user, this problem doesnt occur and both applications can open the db.

2007-Oct-02 15:48:05 by anonymous:
SQLite has no knowledge of Windows services. How do you propose to work around this Windows anachronism?


2007-Oct-02 17:20:38 by anonymous:
Suggesion: Try running the service in the same account as the other program that needs to access the database.

Anachronism? Service is just another word for daemon.

-knu-


2007-Oct-02 17:33:56 by anonymous:
Re: Anachronism, the OP suggested there was something fundamentally different about file access using a service. You've pointed out that it's just a file permissions issue.


2007-Oct-05 14:45:07 by drh:
Two points:

  1. The error message "80004005 unable to lock database" is not generated by SQLite. There must be some middleware someplace that is producing this message. The problem might be in that middleware and not in SQLite.

  2. None of the SQLite developers run windows. Consequently any fixes for this problem will need to come from the community. Please append patches to this ticket if you find a fix. Or close the ticket if you discover that the problem is outside of SQLite.
 
55 new active 2002 Jun anonymous   2007 Oct drh 5 3 instead of triggers for inserts (updates) on regular tablesxx edit
It would be very useful to be able to define a trigger that only executes the trigger code and will prevent the actual insert (or update) on regular tables (eg. specialised autoincrement fields). This could be done by allowing the same "instead of" syntax as used for views. Even better would be the possibility of return codes in a before trigger that can prevents the insert or raise an error. Ideally, this return code could be given conditionally (implementing foreign key and check like functionality).
test


2007-Oct-01 23:37:44 by anonymous:
create trigger syntax :

A special SQL function RAISE() may be used within a trigger-program, with the following syntax:

  raise-function ::= 	RAISE ( ABORT, error-message ) |

  RAISE ( FAIL, error-message ) |
  RAISE ( ROLLBACK, error-message ) |
  RAISE ( IGNORE )
 
2674 code active 2007 Sep anonymous   2007 Sep   3 4 NFS fails without lock manager edit
Problem:

SQLite fails entirely if the NFS lock manager is not running on the share hosting the DB -- even when all access to the DB is serialized.

Under these circumstances, it becomes difficult to create applications that are capable of running in a multitude of environments because restrictions are now imposed upon the storage location.


Reproduction:

  • setup an NFS share
  • disable the lock manager
  • attempt to perform any transactions on a db on that share
  • start the lock manager
  • perform the same operations


Request:

  • Make a change to the error handling, as necessary, to allow processes to access a DB over an NFS share without use of a lock manager.

  • Make a big bold flashing sign in the FAQ about this failure-mode. The wording of the current FAQ led us to believe that the transaction would go through, but protection from other processes was not guaranteed.


Shameless Fanmail:

Love the product!


Screenshot:

  dev-srs08 ~ # <cmd>
  [__db_init <file>.c:384] Statement:
  CREATE TABLE
  versions (
    id       INTEGER PRIMARY KEY AUTOINCREMENT,
    version  INTEGER,
    tbl      CHAR(256)
  );
  Failed with error:
  database is locked
  [main <file>.c:1096] Unable to init output DB: /mnt/nfs/o
  dev-srs08 ~ # /etc/init.d/nfslock start
  Starting NFS statd:                                        [  OK  ]
  dev-srs08 ~ # <cmd>
  dev-srs08 ~ #
2007-Sep-28 04:35:00 by anonymous:
Yet another reason to avoid using SQLite on remote shares.


2007-Sep-29 18:04:19 by anonymous:
Make a change to the error handling, as necessary, to allow processes to access a DB over an NFS share without use of a lock manager.

By design the SQLite library guarantees ACID. It can't provide that without file locks. In my opinion a non-ACID version would be a custom version, which you can (should) build yourself. The source is available, or there might already be a compilation option to accomplish that.

Make a big bold flashing sign in the FAQ about this failure-mode. The wording of the current FAQ led us to believe that the transaction would go through, but protection from other processes was not guaranteed.

FAQ (5) "When any process wants to write, it must lock the entire database file for the duration of its update." seems to be quite clear.

-knu-

 
2677 new active 2007 Sep anonymous   2007 Sep danielk1977 5 3 CREATE TRIGGER .. {databaseevent ON ... }+ for multiple events edit
I have to write triggers which react on either INSERT or UPDATE of a specific field in a table. In some SQL syntax descriptions I found a possibility to combine the trigger events for more than one condition but not in SQLite3. Isn't there a possibility to use one trigger definition for more than one event or didn't I find the trick? Do I have to copy the whole definition for each event; even if the body text is exactly the same?
Suggestion: On INSERT event use the OLD.fieldnames filled with NULL content to avoid problems with UPDATE event. - If not possible please explain why not for my better understanding. Thank you.
 
2671 code active 2007 Sep shess   2007 Sep shess 2 4 Fts field-based queries are not correctly case-insensitive. edit
  CREATE VIRTUAL TABLE t USING fts2(A, B);
  -- At the SQL level, things are case-insensitive:
  INSERT INTO t (A, b) VALUES ('Test', 'Columns');
  INSERT INTO T (a, B) VALUES ('Second', 'Test');
  -- Unfortunately, fts cannot do field-level queries:
  SELECT rowid FROM t WHERE t MATCH 'test';  -- works
  SELECT rowid FROM t WHERE b MATCH 'test';  -- works
  SELECT rowid FROM t WHERE t MATCH 'b:test';  -- no results
  SELECT rowid FROM t WHERE t MATCH 'B:test';  -- no results

It doesn't work because fts is keeping the column name as 'B', but the query parsing uses the results from the tokenizer, which are case-folded, and 'b' != 'B'.

I'm thinking on the solution. A quick fix would be to make the azColumn storage be lowercase, but the core problem is that field names probably shouldn't be run through the tokenizer in the first place.

 
2664 code active 2007 Sep danielk1977   2007 Sep   1 1 attaching the same db twice in shared-cache mode fails edit
The following SQL script can cause an assert() to fail in shared-cache mode.

  ATTACH 'db' AS aux1;
  ATTACH 'db' AS aux2;
  CREATE TABLE aux1.abc(a, b, c);
  CREATE TABLE aux2.abc(a, b, c);
See also #2653
 
2294 code active 2007 Apr anonymous   2007 Sep   2 1 segfault when destroying lock on WinCE with threads edit
DestroyLock emulation on WinCE platform releases the zDeleteOnClose file outside the mutex acquire section. This lead to frequent segfault when working with several databases concurrently.

Patch simply consists in moving code:

    if( pFile->zDeleteOnClose ){
      DeleteFileW(pFile->zDeleteOnClose);
      sqliteFree(pFile->zDeleteOnClose);
      pFile->zDeleteOnClose=NULL;
    }

from winClose() (os_win.c:980) to winceDestroyLock(), inside scope of winceMutexAcquire(pFile->hMutex):

    /* De-reference and close our copy of the shared memory handle */
    UnmapViewOfFile(pFile->shared);
    CloseHandle(pFile->hShared);

    + if( pFile->zDeleteOnClose ){
    +   DeleteFileW(pFile->zDeleteOnClose);
    +   sqliteFree(pFile->zDeleteOnClose);
    +   pFile->zDeleteOnClose=NULL;
    + }

    /* Done with the mutex */
    winceMutexRelease(pFile->hMutex);
    CloseHandle(pFile->hMutex);
    pFile->hMutex = NULL;
2007-Apr-13 00:21:33 by anonymous:
Check in [3836] fixes it for me. eTcl regression tests, related to running several sqlite database concurrently in several threads, are now passed while they were frequently segfaulting without it.

To help being confident with this blind commit, let's mention that exactly same patch has been introduced in eTcl built since a couple of monthes, to fix issue reported by WM2003 users, and all reported a correct fix. However, I did suggest the patch, so testing and feedback from others may help :-)

Also, note that [3836] has a typo, requesting feedback in ticket #2249 instead of #2294


2007-Sep-25 03:24:09 by anonymous:
The fix doesn't work and should be reverted. Temporary files do not create locks, so when they are closed and hMutex is null, the winceDestroyLock() file is never called and the temporary files are not cleaned up properly.
 
2652 code active 2007 Sep drh   2007 Sep   1 1 Aggregate function cannot be used from within a subquery edit
The following SQL fails:

   CREATE TABLE t1(x,y);
   INSERT INTO t1 VALUES(1,2);
   CREATE TABLE t2(z);
   INSERT INTO t2 VALUES(1);
   SELECT (SELECT y FROM t1 WHERE x=min(z)) FROM t2;

Problem reported on the mailing list.

2007-Sep-23 16:01:09 by anonymous:
Your syntax appears to be incorrect.

  SQLite v3.4.2
  CREATE TABLE t1(x,y);
  CREATE TABLE t2(z);
  INSERT INTO t1 VALUES(1,21);
  INSERT INTO t1 VALUES(2,22);
  INSERT INTO t1 VALUES(3,23);
  INSERT INTO t2 VALUES(3);
  INSERT INTO t2 VALUES(2);
  INSERT INTO t2 VALUES(1);

What you wanted to do:

  SELECT y FROM t1 WHERE x=(SELECT min(z) FROM t2);
  21    -- works as expected

What you did:

  SELECT (SELECT y FROM t1 WHERE x=min(z)) FROM t2;
  SQL error near line []: misuse of aggregate function min()
 
2539 code active 2007 Jul anonymous   2007 Sep   2 2 WinCE: Temporary etilqs_ files are not removed from temporary folder edit
Hi, when temporary etilqs_* files are created during SQLite work on Windows CE devices, they are not removed at all.

Temporary folder at CE devices: /Application Data/Volatile

I've research that it winClose(os_win.c) function has been changed at do not remove this file, assuming it to be removed at winceDestroyLock(os_win.c), so if no lock was happened then files will stay here forever.

Has fixed it in my local copy, with hope that it will be fixed when new cool versions of SQLite will be available.

My fix at os_win.c:

static int winClose(OsFile **pId){
  winFile *pFile;
  int rc = 1;
  if( pId && (pFile = (winFile*)*pId)!=0 ){
    int rc, cnt = 0;
    OSTRACE2("CLOSE %d\n", pFile->h);
    do{
      rc = CloseHandle(pFile->h);
    }while( rc==0 && cnt++ < MX_CLOSE_ATTEMPT && (Sleep(100), 1) );
#if OS_WINCE
    winceDestroyLock(pFile);
// fix begin
	if( pFile->zDeleteOnClose ){
		DeleteFileW(pFile->zDeleteOnClose);
		sqliteFree(pFile->zDeleteOnClose);
	}
// fix end
#endif
    OpenCounter(-1);
    sqliteFree(pFile);
    *pId = 0;
  }
  return rc ? SQLITE_OK : SQLITE_IOERR;
}
Thanks,

Fedor

2007-Jul-28 16:41:41 by anonymous:
The solution is to revert checkin 3836 and re-open ticket #2294.

Looking at the wince locking mechanism, the only time we ever use the zDeleteOnClose flag is when we've opened a database for exclusive access in sqlite3WinOpenExclusive. To save time and resources (and because its not necessary) we never bother creating a locking mechanism for exclusively-opened files. So pFile->hMutex is NULL when hitting winceDestroyLock(), and the file is never deleted.

Is it possible that the original poster of #2294 was trying to close the same connection on multiple threads at the same time?


2007-Jul-31 05:32:39 by anonymous:
This is actually a duplicate of #2533


2007-Sep-21 14:20:05 by anonymous:
So when the fix of [3836] was applied, the code to delete the file was only put in the section that is called when we have a mutex. I wonder, if the deletion of the file should also take place if there was no mutex. Works for me at least:

	static void winceDestroyLock(winFile *pFile){
	  if (pFile->hMutex){
	    /* Acquire the mutex */
	    winceMutexAcquire(pFile->hMutex);

	    /* The following blocks should probably assert in debug mode, but they
	       are to cleanup in case any locks remained open */
 	   if (pFile->local.nReaders){
 	     pFile->shared->nReaders --;
 	   }
 	   if (pFile->local.bReserved){
 	     pFile->shared->bReserved = FALSE;
 	   }
 	   if (pFile->local.bPending){
 	     pFile->shared->bPending = FALSE;
 	   }
 	   if (pFile->local.bExclusive){
 	     pFile->shared->bExclusive = FALSE;
 	   }

	    /* De-reference and close our copy of the shared memory handle */
	    UnmapViewOfFile(pFile->shared);
	    CloseHandle(pFile->hShared);

	*    if( pFile->zDeleteOnClose ){
 	*     DeleteFileW(pFile->zDeleteOnClose);
 	*     sqliteFree(pFile->zDeleteOnClose);
 	*     pFile->zDeleteOnClose = 0;
 	*   }

	    /* Done with the mutex */
	    winceMutexRelease(pFile->hMutex);
	    CloseHandle(pFile->hMutex);
	    pFile->hMutex = NULL;
	  }
	+ else
	+ {
	+   if( pFile->zDeleteOnClose ){
	+      DeleteFileW(pFile->zDeleteOnClose);
	+      sqliteFree(pFile->zDeleteOnClose);
	+      pFile->zDeleteOnClose = 0;
	+    }
	+  }
	}

The code marked with * was put there in

 
2653 code active 2007 Sep anonymous   2007 Sep   3 4 Exclusive Transactions do not work with a Database File attached twice edit
Regarding the docs, it is possible to attach the same database file multiple times.

After doing so, I wanted to begin an exclusive transaction. Unfortunately, this fails ("database is locked") and surprises me as I did not find any notice on this particular situation and possible side-effects neither in the attach nor in the transaction/locking documentation.

If this behaviour is seen as an error, It would be useful to have it error fixed, because if one reads a list of (possibly duplicate) database files but with unique identifiers, it would be helpful to use these defined identifiers when accessing the databases (and that in an exclusive transaction).

Real-world case: Each "module" uses its own database file. Some modules share a database file. So there is a list of module -> database file assignments. Now an update process gets some database update scripts from the modules. Every module wants its changes to be done in the right database, so it relies on having its own database attached with an unique identifier - the module's name. On the other hand, the update process need exclusive access to the databases and starts a transaction -- bummer.

 
2651 new active 2007 Sep anonymous   2007 Sep   5 4 Add support for overriding home directory location edit
Currently, the history file's and the rc file's location is hard-wired to $HOME. It would be nice if this could be overridden. One way is to look for a SQLITE_HOME environment variable that points to the location to use. This can be achieved by a simple addition to find_home_dir() in src/shell.c.
 
2649 new active 2007 Sep anonymous   2007 Sep   4 4 Add an "--enable-extensions" (default=no) to the configure script edit
The attached patch adds "--enable-extensions" to the configure script, but is disabled by default (because of the security considerations of having it enabled).
 
2645 build active 2007 Sep anonymous   2007 Sep   5 4 Conflict between tclConfig.sh and tclinstaller.tcl edit
I'm using a non-standard location for Tcl:

  --with-tcl=/home/scott/lib

The build process finds and uses the tclConfig.sh file from /home/scott/lib just fine, but tclinstaller.tcl is called without an explicit path and uses the system's tclsh instead of the one in /home/scott/bin and thus tries to install that portion of code into the system's area.

While I can change my path to resolve this, I think it makes more sense for tclinstaller.tcl to use the path information that's embedded in tclConfig.sh to be consistent.

/s.

 
2627 code active 2007 Sep anonymous   2007 Sep   3 2 Improper parsing of nested JOIN edit
SQLite has a problem with multiple nested JOINs. The only way to get it workig is to remove the surrounding brackets. Removing the brackets unfortunately do not work in other DB systems such as MS SQL, mysql etc.

This does not work:

Select ContactPhone.* From (ContactPhone LEFT OUTER JOIN ContactLocation ON ContactPhone.PHNLCT_ID = ContactLocation.LCT_ID) LEFT OUTER JOIN ContactItem ON ContactLocation.LCTITM_ID = ContactItem.ITM_ID

(It complains about LCT_ID or similar)

This works after removing the brackets:

Select ContactPhone.* From ContactPhone LEFT OUTER JOIN ContactLocation ON ContactPhone.PHNLCT_ID = ContactLocation.LCT_ID LEFT OUTER JOIN ContactItem ON ContactLocation.LCTITM_ID = ContactItem.ITM_ID

All other major DB systems require the surrounding brackets. Do you think it is possible to fix it? Apart from this little little SQLite is an awesome project.

Thank you Jakub Klos

2007-Sep-06 13:07:32 by anonymous:
I don't have access to MS SQL Server, but MySQL and Oracle have no issue with the query without parentheses:

create table x1(a int, b int);
create table x2(c int, d int);
create table x3(e int, f int);

mysql> select x1.* from x1 left join x2 on x1.a=x2.c left join x3 on x2.d=x3.e;
Empty set (0.00 sec)

2007-Sep-06 19:03:33 by anonymous:
MSSQL also has no problems without the parens. As a matter of fact, the only DB that I know of that requires them is MS Access (JET).


2007-Sep-06 20:11:31 by anonymous:
I guess he had no luck filing a JET bug.


2007-Sep-11 17:22:28 by anonymous:
True, MS access requires the parens but all other major DBs support the query syntax with the parens. So why SQLite does not like it? It should simply ignore them if possible. Thank you
 
2634 code active 2007 Sep anonymous   2007 Sep   3 3 .schema uses incorrect ORDER BY giving wrong dependency order edit
When the schema is exported, views are sorted by name instead of by dependency. If there are nested views, the schema may be invalid when used to re-create the database.

  sqlite3
  create table t ( f text );
  create view v2 as select f from t;
  create view v1 as select f from v2;
  .output test.txt
  .schema
  .exit

  sqlite3
  .read test.txt
  SQL error near line 2: no such table: main.v2
2007-Sep-07 15:33:06 by anonymous:
Use .dump instead as a workaround. Unlike .schema, .dump does not use ORDER BY in its queries on sqlite_master and it outputs its rows in order of entry.

SQLite version 3.5.0
Enter ".help" for instructions
sqlite> create table t ( f text );
sqlite> create view v2 as select f from t;
sqlite> create view v1 as select f from v2;
sqlite>
sqlite> .schema
CREATE TABLE t ( f text );
CREATE VIEW v1 as select f from v2;
CREATE VIEW v2 as select f from t;
sqlite>
sqlite> .dump
BEGIN TRANSACTION;
CREATE TABLE t ( f text );
CREATE VIEW v2 as select f from t;
CREATE VIEW v1 as select f from v2;
COMMIT;
Suggested patch:

Index: src/shell.c
===================================================================
RCS file: /sqlite/sqlite/src/shell.c,v
retrieving revision 1.167
diff -u -3 -p -r1.167 shell.c
--- src/shell.c 7 Sep 2007 01:12:32 -0000       1.167
+++ src/shell.c 7 Sep 2007 15:28:24 -0000
@@ -1411,8 +1411,7 @@ static int do_meta_command(char *zLine,
           "SELECT sql FROM "
           "  (SELECT * FROM sqlite_master UNION ALL"
           "   SELECT * FROM sqlite_temp_master) "
-          "WHERE tbl_name LIKE shellstatic() AND type!='meta' AND sql NOTNULL "
-          "ORDER BY substr(type,2,1), name",
+          "WHERE tbl_name LIKE shellstatic() AND type!='meta' AND sql NOTNULL",
           callback, &data, &zErrMsg);
         zShellStatic = 0;
       }
@@ -1421,8 +1420,7 @@ static int do_meta_command(char *zLine,
          "SELECT sql FROM "
          "  (SELECT * FROM sqlite_master UNION ALL"
          "   SELECT * FROM sqlite_temp_master) "
-         "WHERE type!='meta' AND sql NOTNULL AND name NOT LIKE 'sqlite_%'"
-         "ORDER BY substr(type,2,1), name",
+         "WHERE type!='meta' AND sql NOTNULL AND name NOT LIKE 'sqlite_%'",
          callback, &data, &zErrMsg
       );
     }
after patch:

SQLite version 3.5.0
Enter ".help" for instructions
sqlite>  create table t ( f text );
sqlite>   create view v2 as select f from t;
sqlite>   create view v1 as select f from v2;
sqlite> .schema
CREATE TABLE t ( f text );
CREATE VIEW v2 as select f from t;
CREATE VIEW v1 as select f from v2;
sqlite> .q
 
2629 doc active 2007 Sep anonymous   2007 Sep   4 4 typo in os_unix.c for nolock edit
Index: src/os_unix.c
===================================================================
RCS file: /sqlite/sqlite/src/os_unix.c,v
retrieving revision 1.165
diff -u -3 -p -r1.165 os_unix.c
--- src/os_unix.c       5 Sep 2007 13:56:32 -0000       1.165
+++ src/os_unix.c       6 Sep 2007 17:53:47 -0000
@@ -2126,7 +2126,7 @@ static const sqlite3_io_methods sqlite3D

 /*
 ** This vector defines all the methods that can operate on an sqlite3_file
-** for unix with dotlock style file locking.
+** for unix with nolock style file locking.
 */
 static const sqlite3_io_methods sqlite3NolockLockingUnixIoMethod = {
   1,                        /* iVersion */
 
2607 event active 2007 Aug anonymous   2007 Sep   2 1 Data loss, continuation to Re: [sqlite] how to flush database to disk? edit
See that mailing list.

The originator message :


I've just lost a couple of days' worth of data when my app crashed. (Well, the data wasn't a total loss thanks to backup plans, but the database itself essentially reverted to its state of 2 days ago.) This is despite my app doing a COMMIT after every modification of the DB.

It's acting as though all the changes were held in memory, or somehow journaled, and when the crash happened, the changes were all lost or rolled back. What I need is a way to force the database to save its data to disk while my app is running, so that in the event of a crash, I lose little or no data. How can I do this? I presume that closing the database would do the trick, but is there a less heavy-handed way?


The exact like that data loss occured at me too. Three times, non-repropucible regualrilly.

What common in these losses ?

{Editing+committing} in the main thread then {navigating, bof/eof checking, reading data} from within different threads then return to the main thread for {editing+committing}.

2007-Aug-31 19:38:26 by drh:
The COMMIT does not actually occur until you call sqlite3_reset() and/or sqlite3_finalize() on all your prepared statements. Any prepared statement that has not been reset or finalized is still running, is incomplete, and is thus still holding the transaction open. I'm guessing that you have an unreset and unfinialized statement in your application.

I wonder what would happen if we changed the definition of COMMIT so that it returned an error if there were active prepared statements. This is, technically, an incompatibility. But we are coming up on a release with several other minor incompatibilities, so now might be a good time to insert such a change.


2007-Aug-31 19:45:36 by drh:
I looked in the code, and it turns out we already do this. Perhaps the application is not checking the return code from the COMMIT to see that it is failing?


2007-Aug-31 20:40:47 by anonymous:
No error reports came from COMMIT. The data loss were noticed after UPDATE & COMMIT after massive reading of results of SQL addressing the same virtual (ATTACHed) tables, from within another thread.


2007-Aug-31 20:53:48 by anonymous:
Is it possible that a pending transaction survive app shutdown & then OS restart ? Is yes, then any DB error would cause rollback to the data on last BEGIN, isn't ?


2007-Aug-31 21:00:18 by anonymous:
But me committed each smallest change to dat,a then saw these refreshed data in the tables. And on the next day these data were present. Only reading (with full scrolling ) the affected virtual tables from within another thread then new editing then committing caused the loss.


2007-Sep-01 23:12:31 by anonymous:
You mention virtual tables. Their data is not maintained by the SQLite engine, but by your own module. If that doesn't implement ACID, you're out of luck.

By definition:
A virtual table is an interface to an external storage or computation engine that appears to be a table but does not actually store information in the database file.

references:
CreateVirtualTable
VirtualTables


2007-Sep-04 04:51:15 by anonymous:
Me was wrong. These were't true virtual tables. Me used a LEFT OUTER query to several tables residing in different ATTACHed databases.


2007-Sep-04 04:52:01 by anonymous:
were't => were not, above.
 
2604 new active 2007 Aug anonymous   2007 Aug   4 4 CREATE VIRTUAL TABLE does not allow IF NOT EXISTS edit
CREATE VIRTUAL TABLE vt IF NOT EXISTS; would help with development since creating a virtual table that exists returns error 1 - as do several "Real" errors.
 
2595 doc active 2007 Aug anonymous   2007 Aug   4 4 sqlite3_commit_hook doc typo edit
src/main.c:

  -** Register a function to be invoked when a transaction comments.
  +** Register a function to be invoked when a transaction commits.
 
2559 code active 2007 Aug anonymous   2007 Aug   4 4 "make clean" does not delete sqlite3.c and tsrc/ edit
Index: Makefile.in
===================================================================
RCS file: /sqlite/sqlite/Makefile.in,v
retrieving revision 1.179
diff -u -3 -p -r1.179 Makefile.in
--- Makefile.in 27 Aug 2007 23:38:43 -0000      1.179
+++ Makefile.in 28 Aug 2007 01:25:55 -0000
@@ -724,7 +724,7 @@ clean:
        rm -f testfixture$(TEXE) test.db
        rm -rf doc
        rm -f common.tcl
-       rm -f sqlite3.dll sqlite3.lib sqlite3.def
+       rm -rf sqlite3.dll sqlite3.lib sqlite3.def sqlite3.c tsrc

 distclean:     clean
        rm -f config.log config.status libtool Makefile config.h sqlite3.pc
 
2587 build active 2007 Aug anonymous   2007 Aug   3 4 Build problem when using the SQLITE_OMIT_FLOATING_POINT define. edit
I apologize in advance if the values I chose above are not appropriate.

If I define SQLITE_OMIT_FLOATING_POINT=1 and try to build a Windows DLL, I get two errors in loadext.c, line 116 and 192. "error C4028: formal para meter 3 different from declaration"

I believe you want to change the include order at the top of loadext.c from:

#include "sqlite3ext.h"

#include "sqliteInt.h"

to:

#include "sqliteInt.h"

#include "sqlite3ext.h"

Reversing the order of include fixes my build.

Yes, I know there is no real reason to disable floating point for the Windows DLL. I'm actually porting SqLite for use in an NT kernel mode driver and avoiding floating point operations will save a lot of time if I don't really need them and I don't. So I made sure this was a problem with a supported platform like the Windows DLL and griped about that instead of my insanity. ;-)

You can email questons to mspiegel@vipmail.com. If you want to discuss this over the phone, shoot me an email and I'll send you phone numbers.

 
2568 new active 2007 Aug anonymous   2007 Aug   3 3 TEMP_STORE is ignored in some cases edit
It seems that sometimes TEMP_STORE is ignored. I've tried to force SQLite to always use memory by setting TEMP_STORE=3, but some etilqs_* temp files are still being created. The call stack that's causing these file to be created is:

  sqlite3PagerOpentemp(OsFile * *)
  sqlite3PagerStmtBegin(Pager *)
  sqlite3BtreeBeginStmt(Btree *)
  sqlite3VdbeExec(Vdbe *)
  sqlite3Step(Vdbe *)
  sqlite3_step(sqlite3_stmt *)

It looks like the temp files are being used to store information for undoing earlier parts of a transaction if a later part fails. I'm assuming the fact this part of the code ignores TEMP_STORE is an over site?

2007-Aug-13 15:03:19 by drh:
The TEMP_STORE compile-time option only changes the storage for temporary database files. The statement journal is not a databaes file and thus does not come under the control of TEMP_STORE. There is currently no mechanism to force the statement journal into memory instead of onto disk.

I will reclassify this ticket as a "feature request".


2007-Aug-22 10:42:50 by anonymous:
Okay, thank you.
 
1242 code active 2005 May anonymous Shell 2007 Aug   3 4 EXPLAIN causes segmentation fault on OSX (and linux) edit
Under Mac OS X, EXPLAIN causes a segmentation fault:

  [jacob@046] ~$ sqlite3 foo.db
  SQLite version 3.2.1
  Enter ".help" for instructions
  sqlite> CREATE TABLE test (a int, b int);
  sqlite> EXPLAIN SELECT * FROM test;
  Segmentation fault

The crash dump follows:

    Host Name:      jacobian
    Date/Time:      2005-05-13 09:17:04.860 -0500
    OS Version:     10.4 (Build 8A428)
    Report Version: 3

    Command: sqlite3
    Path:    /usr/local/bin/sqlite3
    Parent:  bash [15421]

    Version: ??? (???)

    PID:    15544
    Thread: 0

    Exception:  EXC_BAD_ACCESS (0x0001)
    Codes:      KERN_INVALID_ADDRESS (0x0001) at 0x1400fffc

    Thread 0 Crashed:
    0   libSystem.B.dylib  	0x90003228 strlen + 8
    1   libsqlite3.0.dylib 	0x002387c8 sqlite3VdbeList + 284 (vdbeaux.c:609)
    2   libsqlite3.0.dylib 	0x002376e0 sqlite3_step + 312 (vdbeapi.c:207)
    3   libsqlite3.0.dylib 	0x0023e5d8 sqlite3_exec + 260 (legacy.c:82)
    4   sqlite3            	0x00005b64 process_input + 808 (shell.c:1504)
    5   sqlite3            	0x000062bc main + 1528 (shell.c:1790)
    6   sqlite3            	0x00001db4 _start + 348 (crt.c:272)
    7   sqlite3            	0x00001c54 start + 60

    Thread 0 crashed with PPC Thread State:
      srr0: 0x90003228 srr1: 0x0000d030                vrsave: 0x00000000
        cr: 0x22444428  xer: 0x00000006   lr: 0x002387c8  ctr: 0x90003220
        r0: 0x002387c8   r1: 0xbfffef40   r2: 0x00249a00   r3: 0x1400fffe
        r4: 0x00000028   r5: 0x00000000   r6: 0x00000001   r7: 0xffffffff
        r8: 0x00000001   r9: 0x1400fffc  r10: 0x00000086  r11: 0x00249180
       r12: 0x90003220  r13: 0x00000000  r14: 0x00000000  r15: 0x00000000
       r16: 0x00000000  r17: 0xbffff0f8  r18: 0x00000000  r19: 0xbffff17c
       r20: 0x00000000  r21: 0x000036d0  r22: 0x00303d90  r23: 0x00303d74
       r24: 0x01805700  r25: 0x01807e00  r26: 0x00000001  r27: 0x00000004
       r28: 0x01805640  r29: 0x01805600  r30: 0x01805200  r31: 0x002386bc

    Binary Images Description:
        0x1000 -     0x7fff sqlite3 	/usr/local/bin/sqlite3
      0x205000 -   0x248fff libsqlite3.0.dylib 	/usr/local/lib/libsqlite3.0.dylib
    0x8fe00000 - 0x8fe50fff dyld 43	/usr/lib/dyld
    0x90000000 - 0x901a6fff libSystem.B.dylib 	/usr/lib/libSystem.B.dylib
    0x901fe000 - 0x90202fff libmathCommon.A.dylib 	/usr/lib/system/libmathCommon.A.dylib
    0x91d33000 - 0x91d53fff libmx.A.dylib 	/usr/lib/libmx.A.dylib
    0x9680c000 - 0x9683afff libncurses.5.4.dylib 	/usr/lib/libncurses.5.4.dylib
    0x969a3000 - 0x969b9fff libedit.2.dylib 	/usr/lib/libedit.2.dylib
Happening to me as well on FC6 sqlite3 version 3.3.6


2007-Aug-21 17:09:34 by anonymous:
Try to upgrade to 3.4.2.
 
2582 doc active 2007 Aug anonymous   2007 Aug anonymous 5 5 documenation clarification edit
docs for topic `Set A Busy Timeout` int sqlite3_busy_timeout(sqlite3*, int ms); http://sqlite.org/capi3ref.html#sqlite3_busy_timeout

the wording "The handler will sleep multiple times until at least "ms" milliseconds of sleeping have been done" implies it will wait the total amount regardless of the lock status, it should perhaps indicate in the same sentence that it will exit early if the lock becomes available.

 
2580 code active 2007 Aug anonymous   2007 Aug anonymous 1 2 Can't open a query if text to search is Greek edit
for example:

SELECT * FROM mytable WHERE mycolumn LIKE '%some greek text%'

I get wrong results, using the 3.4.2 version. No problem instead using other earlier version. I tested only in Windows.

 
2567 build active 2007 Aug anonymous   2007 Aug   3 2 Build fails to install edit
I compile under MinGW with Msys. A build error occurs during 'make install'. After checking the makefile. The 'install' target depends on 'sqlite3', when it should be 'sqlite3$(TEXE)'.

The workaround is, after configure, edit makefile for target install, and replace 'sqlite3' with 'sqlite3${TEXE}' where needed. I did not have this problem with 3.3.17. I assume this could be fixed just by fixing the configure to produce correct makefile.

2007-Aug-12 04:41:12 by anonymous:
Do you have a patch?
 
2566 build active 2007 Aug anonymous   2007 Aug   2 1 fts2 broken after vacuum edit
Hi there, I'm testing your database and I'm having problems with fts2:
sqlite> select * from distB where distB match "MARIANO"; Assertion failed: *pData!='\0', file fts2amal.c, line 16790

This application has requested the Runtime to terminate it in an unusual way. Please contact the application's support team for more information.


Steps: 1) Create a new .db 2) Import data in new distA table 3) Import data in new distB table 4) Create a new distC virtual table (dts2) 5) insert into distC (rowid, f1, f2, f3) select rowid, f1, f2, f3 from DistB

Everything working like a charm until here!!! The fts2 works very well, but after

6) vacuum;

the fts seems broken... doing a select throws the error I paste at the post of the topic

If you want the .db file I can send it to you (607MB)

Thanks.-

 
2558 code active 2007 Aug anonymous   2007 Aug   2 3 Multiple JOIN USING() gives incorrect results edit
I'm having a problem joining multiple tables with USING. It appears to work, but the results are incorrect. Here is an example to illustrate the problem. I believe the three SELECT statements should be equivalent, but they produce three different results.

  .header on
  .mode column

  CREATE TABLE Main (pk INTEGER PRIMARY KEY, name VARCHAR);
  CREATE TABLE OptA (pk INTEGER PRIMARY KEY, alpha VARCHAR);
  CREATE TABLE OptB (pk INTEGER PRIMARY KEY, beta VARCHAR);

  INSERT INTO Main VALUES (1, 'One');
  INSERT INTO Main VALUES (2, 'Two');
  INSERT INTO Main VALUES (3, 'Three');
  INSERT INTO Main VALUES (4, 'Four');

  INSERT INTO OptA VALUES (1, 'Alpha1');
  INSERT INTO OptA VALUES (4, 'Alpha4');

  INSERT INTO OptB VALUES (2, 'Beta2');
  INSERT INTO OptB VALUES (4, 'Beta4');

  SELECT * FROM Main LEFT JOIN OptA USING (pk) LEFT JOIN OptB USING (pk);

  SELECT * FROM Main LEFT JOIN OptB USING (pk) LEFT JOIN OptA USING (pk);

  SELECT Main.pk, name, alpha, beta
    FROM Main LEFT JOIN OptA ON Main.pk = OptA.pk
              LEFT JOIN OptB ON Main.pk = OptB.pk;

Joining Main, OptA, and OptB omits Beta2:

  pk          name        alpha       beta
  ----------  ----------  ----------  ----------
  1           One         Alpha1
  2           Two
  3           Three
  4           Four        Alpha4      Beta4

Joining Main, OptB, and OptA omits Alpha1:

  pk          name        beta        alpha
  ----------  ----------  ----------  ----------
  1           One
  2           Two         Beta2
  3           Three
  4           Four        Beta4       Alpha4

Only by using ON instead of USING do we get the correct results:

  pk          name        alpha       beta
  ----------  ----------  ----------  ----------
  1           One         Alpha1
  2           Two                     Beta2
  3           Three
  4           Four        Alpha4      Beta4

I think this is basically the same issue as ticket #1637, but it's a more serious example. In that one, the query simply failed to compile. In this case, it seems to work, but gives you the wrong results.

I've also tried this script in PostgreSQL 8.0.13. All three queries give (the same) correct results.

2007-Aug-08 17:34:27 by anonymous:
The problem is that SQLite is transforming

  SELECT * FROM Main LEFT JOIN OptA USING (pk) LEFT JOIN OptB USING (pk);

into

  SELECT Main.pk, name, alpha, beta
    FROM Main LEFT JOIN OptA ON Main.pk = OptA.pk
              LEFT JOIN OptB ON OptA.pk = OptB.pk;

Here is a workaround to this bug that makes use of a subquery:

  select * from (SELECT * FROM Main LEFT JOIN OptA USING (pk))
    LEFT JOIN OptB USING (pk);

Conceivably all LEFT JOIN chains could be transformed into the above form, but that would decrease performance due to the intermediate result set of the subquery. Having it work without the subquery is tricky since sqlite must deduce that the last USING (pk) is equivalent to the first pk in the chain of joined tables, namely Main.pk, and not OptA.pk.

Joe Wilson

 
2555 new active 2007 Aug anonymous   2007 Aug   1 1 FTS index without original text edit
Is it possible to build FTS index without storing original text? I want to use fts index without features of snippets etc. I just want to find ID of the record not the content of indexed phrase. I suppose that the table myname_content stores this content. I have tried to update all columns of myname_content and set its values to &#8220;xyz&#8221; (without one column in which I store ID of the record). After this operation FTS search works good, but unfortunately the table isn&#8217;t smaller (I cant&#8217;t use vacuum on FTS tables). Is there any other way to have pure text indexes without source level changes?
 
2547 code active 2007 Aug danielk1977   2007 Aug   5 3 Changing db encoding of an attached db can confuse shared cache mode. edit
This is quite obscure, but in shared-cache mode:

  1) Open db A, attach empty db B.
  2) Using another connection from the same thread, set the
     encoding of B to be different from that of A. Add some
     data to B.
  3) Using the original connection, access database B. It assumes
     the encoding of A (and therefore mangling any text data).

The correct response is to return an error - "attached databases must use the same text encoding as main database".

 
2488 new active 2007 Jul anonymous   2007 Jul   5 4 autosize on column output mode in sqlite3 program edit
It would be nice if sqlite3 program has a autosizecolumn mode for displaying queries, because it truncates values, and to calculate and use .width size for each column is tedious.
2007-Jul-07 11:42:03 by drh:
In order to do this, we would have to either run the query twice or load the entire result set into memory. Otherwise, there would be no way to determine the longest element of each column. Neither approach seems attractive for large and complex queries.


2007-Jul-28 07:05:56 by anonymous:
every query should internally detect the longest column sizes and a new command should enable the user to set these values for any repetition or similar queries. At the end the queries run twice or more but only the first trial would have cause irritations on output. And this solution should be easy enough to implement it.
Even the core of sqlite could calculate the maximum length of each column and a new API function could make this available. It would be really nice to get such an enhancement!


2007-Jul-28 18:37:36 by anonymous:
while you don't move to next row, sqlite doesn't know the contents, so it will be impossible to do, only if you cache the text entirely in memory, but this is ugly, imagine a 1GB recordset into RAM...


have you tried .mode tabs ? here, i have correct column width
 
2545 code active 2007 Jul anonymous   2007 Jul   1 4 Group by returns table name with field name edit
imaginate a table:
create table test (
id INTEGER PRIMARY KEY,
name varchar(50) not null,
age integer not null
);

Then:

insert into test (name,age) values ('foo',22);
insert into test (name,age) values ('foo',23);
insert into test (name,age) values ('bar',22);
insert into test (name,age) values ('bar',35);
insert into test (name,age) values ('bar',72);

Now try this;

sqlite> .headers on
sqlite> select test.name, test.age from test order by name;
name|age
bar|22
bar|35
bar|72
foo|22
foo|23
sqlite> select test.name, test.age from test group by name;
test.name|test.age
bar|72
foo|23

You see ? if i use "GROUP BY", the field name contains tablename. Because i use "SELECT test.name" and not "SELECT name".

If i set an alias, i get alias, that's ok.

The trouble appears to be very high on Copix (http://wwW.copix.org). We create some DAO (Data Access Objects) automatically. The "groupBy" method doesn't works with SQLite...

Is this normal ? Mysql, PostgreSql, Oracle... doesn't need to create alias.

2007-Jul-31 15:54:52 by anonymous:
There's probably 4 other tickets reporting this. I don't think it will get fixed. The workaround is to use aliases (AS "whatever") for the selected columns.


2007-Jul-31 23:02:19 by anonymous:
Ok, we have created a special support for SQLite.

PS: I love this database :) Simple, nice, usefull, quick and easy

Regards

 
2543 code active 2007 Jul anonymous   2007 Jul   1 1 Chinese charset not support?? edit
when i create a table. the table name is " " (chinese) after this "alter table   add column aaa text null" error why??/ thank you
 
2540 code active 2007 Jul anonymous   2007 Jul   5 4 Display dlopen() errors for errors when loading modules on Unix system edit
If there is an error loading a module the message "unable to open shared library..." is displayed. In the Unix world the dlopen() error could be display to help diagnose the problem (e.g. missing external refs, etc). I have implemented it in our install of SQLite, I'm sure there is a Windows analog. Here is the patch for loadext.c.

- Chris - Christopher Hailey Sr. Software Engineer Maxim Systems

   ::::::::::::::
   loadext.c.patch
   ::::::::::::::
   *** ./src/loadext.c.orig        2007-07-22 00:11:35.000000000 -0700
   --- ./src/loadext.c     2007-07-22 00:12:07.000000000 -0700
   ***************
   *** 292,298 ****
       handle = sqlite3OsDlopen(zFile);
       if( handle==0 ){
         if( pzErrMsg ){
   !       *pzErrMsg = sqlite3_mprintf("unable to open shared library [%s]", zFile);
         }
         return SQLITE_ERROR;
       }
   --- 292,298 ----
       handle = sqlite3OsDlopen(zFile);
       if( handle==0 ){
         if( pzErrMsg ){
   !       *pzErrMsg = sqlite3_mprintf("unable to open shared library [%s]: %s", zFile,dlerror());
         }
         return SQLITE_ERROR;
 
2530 code active 2007 Jul anonymous   2007 Jul   2 3 Unable to write to windows share, even with exclusive lock edit
It has been mentioned that the file locking does not work on windows shared network drives (Samba or SMB drives from Windows or Linux). It seems that an exclusive lock should be a workaround for this problem if you need to write to a shared drive. Currently a more complicated locking is being attempted and failing on network drives.

With an exclusive lock, SQLite could resort to simply holding a open write or append enabled file handle to the database as a more primitive locking system that is more likely to work on network drive. No other process could open the database but that would be expected with an exclusive lock. The following case should then function:

  grudy@gamma:~$ mount | grep Files
  //winserver/FileDump on /mnt/Files type cifs (rw,mand,noexec,nosuid,nodev)
  grudy@gamma:~$ touch /mnt/Files/i_have_write_permissions.txt; rm /mnt/Files/i_have_write_permissions.txt
  grudy@gamma:~$ sqlite3 /mnt/Files/foo.sqlite
  SQLite version 3.3.17
  Enter ".help" for instructions
  sqlite> PRAGMA locking_mode = EXCLUSIVE;
  exclusive
  sqlite> create table bar (foobar);
  SQL error: database is locked
  sqlite>
 
2469 build active 2007 Jun anonymous   2007 Jul   1 1 test fails on Solaris edit
I have a problem running the test suite on Solaris 9. Build was done using gcc 4.2.0. The build completes without error but many tests fail. I've created my own minimal test that exhibits the problem:

set testdir [file dirname $argv0]
source $testdir/tester.tcl

db close
file delete -force test.db test.db-journal
sqlite db test.db

do_test tdb-1 {
  execsql {
    PRAGMA auto_vacuum = 1;
    BEGIN;
    CREATE TABLE t1(a, b);
  }
  execsql {
    COMMIT;
  }
} {}
integrity_check tdb-2

finish_test
When running this test I get the following output:

tdb-1... Ok
tdb-2...
Expected: [ok]
     Got: [{*** in database main ***
List of tree roots: 2nd reference to page 1
Page 3 is never used}]
Thread-specific data deallocated properly
1 errors out of 3 tests
Failures on these tests: tdb-2
This error happens on lots of, but not all, tests.

I'm happy to do whatever is necessary to help debug this.

Thanks, Tim.

2007-Jun-27 10:44:14 by anonymous:
Further to this, it appears to be related to gcc 4.2.0. It works fine with gcc 3.4.6.


2007-Jun-28 09:54:35 by anonymous:
Further more, it doesn't appear to be specific to Solaris. The same problem occurs on Linux with gcc 4.2.0.

So I guess the subject of this ticket should be changed to "build/test problems with gcc 4.2.0".

This is probably a significant problem - the build completes find but the resultant code is broken. People may not notice this until it's too late.


2007-Jun-28 12:24:05 by drh:
I installed gcc 4.2.0 on my SuSE linux i686 desktop and built test harnesses under three different configurations:

     gcc420 -g -O0 -Wall -fstrict-aliasing
     gcc420 -g -O3 -Wall
     gcc420 -g -O3 -fstrict-aliasing -fomit-frame-pointer

The first two configurations used separate source files. The third configuration was built using the amalgamation. I ran the "quick" test under all configurations. All tests ran to completion with no errors.


2007-Jun-28 13:22:20 by anonymous:
Two ideas:

1. Compile with gcc 4.2.0 using -O0 instead of -O2 and see what happens. Disable any other optimizations you may have.

2. Run truss with full read/write buffer display on the gcc 3.4.6 compiled testfixture running your simple test case and compare its output to the gcc 4.2.0 compiled test case.


2007-Jul-01 19:00:40 by anonymous:
I've done tests with optimisation, and this appears to tickle the problem.

With no optimisation, -O, -O0, -O1 and -03 it works.

With -O2 and -Os it's broken.

I was compiling with -O2 when I submitted the initial report.

Tim.


2007-Jul-01 19:54:54 by drh:
I can reproduce the problem now on Linux when compiling as follows:

    gcc420 -g -O2 -Wall


2007-Jul-01 21:50:42 by drh:
This appears to be a bug in GCC 4.3.0. A work-around is to compile with the -fno-tree-vrp option.

GCC appears to miscompile a single loop within the logic that implements the integrity_check PRAGMA. The code that gets miscompiled is in the file vdbe.c (lines numbers added):

   4308    for(j=0; j<nRoot; j++){
   4309      aRoot[j] = pTos[-j].u.i;
   4310    }
   4311    aRoot[j] = 0;

It is line 4309 that appears to be miscompiled. The pTos pointer points to the top of the stack in virtual machine. The loop attempts to load the array with integer values that have been pushed onto the stack.

Running this code in GDB and setting a breakpoint on line 4311, we get the following output:

    Breakpoint 1, sqlite3VdbeExec (p=0x80ca708) at ../sqlite/src/vdbe.c:4311
    4311      aRoot[j] = 0;
    (gdb) print nRoot
    $1 = 2
    (gdb) print aRoot[0]@2
    $2 = {1, 1}
    (gdb) print pTos[0].u.i
    $3 = 1
    (gdb) print pTos[-1].u.i
    $4 = 2

As you can see above, two values have been pushed onto the virtual machine stack. (nRoot==2). But instead of loading each value into aRoot[], the top-most value is loaded multiple times. aRoot[] ends up holding {1, 1} instead of the correct values of {1, 2}.

I will submit a bug report to GCC shortly...


2007-Jul-01 22:09:23 by drh:
Bug reported filed with GCC:

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=32575


2007-Jul-05 13:33:09 by anonymous:
Could the test case at the top of this ticket be checked in with a comment pointing to this ticket? Other people may be experiencing this problem and not realizing it.


2007-Jul-21 10:00:56 by anonymous:
I can confirm this is still broken with gcc 4.2.1 and sqlite 3.4.1.

Tim.


2007-Jul-21 12:53:40 by drh:
The bug is in GCC, not SQLite The work-around is to compile without optimization. See the comments above.


2007-Jul-21 14:10:43 by anonymous:
The problem is that this is a compiler in widespread use, and you'll likely see randomly corrupted databases in the wild as result of this.

Are you able to create a smaller test case so that GCC will still exhibit this bug? (Assuming it is verified to be a compiler bug).


2007-Jul-21 16:12:42 by anonymous:
Yup - I know it's a GCC bug. I just wanted to note that the recent gcc 4.2.1 release doesn't fix the issue.


2007-Jul-21 18:55:33 by anonymous:
Simple way to detect if your copy of sqlite3 is broken:

  echo "CREATE TABLE t(x); pragma integrity_check;" | ./sqlite3

If it is broken it will output:

  *** in database main ***
  List of tree roots: 2nd reference to page 1
  Page 2 is never used

If fine:

  ok


2007-Jul-21 19:20:07 by anonymous:
This patch works around the problem with gcc-4.2.1 and sqlite 3.4.1 and allows "make test" to run cleanly. I'd appreciate if you'd apply it as gcc 4.2.x is a widely distributed compiler, and the default sqlite3 ./configure build will result in this bug.

Joe

RCS file: /sqlite/sqlite/src/vdbe.c,v
retrieving revision 1.636
diff -u -3 -p -r1.636 vdbe.c
--- src/vdbe.c  1 Jul 2007 21:18:40 -0000       1.636
+++ src/vdbe.c  21 Jul 2007 19:10:13 -0000
@@ -4306,7 +4306,8 @@ case OP_IntegrityCk: {
   pnErr = &p->aMem[j];
   assert( (pnErr->flags & MEM_Int)!=0 );
   for(j=0; j<nRoot; j++){
-    aRoot[j] = pTos[-j].u.i;
+    /* See Ticket #2469. Was: aRoot[j] = pTos[-j].u.i; */
+    aRoot[j] = (pTos-j)->u.i;
   }
   aRoot[j] = 0;
   popStack(&pTos, nRoot);

2007-Jul-21 20:31:01 by anonymous:
Not verified, but these debug trace lines follow the same pattern and may be problematic under gcc 4.2.x -O2.

  src/vdbe.c:          fprintf(p->trace, " si:%lld", pTos[i].u.i);
  src/vdbe.c:          fprintf(p->trace, " i:%lld", pTos[i].u.i);

This line's code appears to be generated okay under -O2:

  src/vdbe.c:  nArg = pTos[-1].u.i;


2007-Jul-23 13:33:55 by anonymous:
Two other GCC 4.2.1 -O2 bug workarounds are shown below which may help in the search for the optimization bug.

Joe.

(1)

  int ZZZ = 0; // must be a global variable

...

   for(j=0; j<nRoot; j++){
     aRoot[j] = pTos[ZZZ-j].u.i;
   }

(2)

  int ZZZ = 0; // must be a global variable

...

   for(j=ZZZ; j<nRoot; j++){
     aRoot[j] = pTos[-j].u.i;
   }
 
2520 new active 2007 Jul anonymous   2007 Jul   4 1 User defined aggregate functions are not reentrant edit
When an aggregate function is defined using sqlite3_create_function, it is not possible to execute any sql statement inside the step part or the finalizer. This is due to the fact that aggregate functions are not reentrant.
2007-Jul-20 02:32:36 by anonymous:
related: Ticket #2242: sqlite3 authorizer is not reentrant
 
2517 code active 2007 Jul anonymous   2007 Jul dflam 1 1 exception on reading text in vista but not xp edit
My companies sqlite 3.1 db works perfectly on Win XP but when we moved to Vista (I'm using Vista 64)it is trowing an exception when I access a text field that contains this data:

'A/C Pressure Sensor, raw1 = A/C on, 0 = A/C off (A/C status determines which IACTx cell is used)'

Interestingly when I view data I've inserted using sqliteman3 it has unprintable characters added to it.

(A/C status determines which IACTx cell is used)9

If I define the field as Char[512] this artifact goes away. But reading your literature this isn't supposed to make a difference because everything is char.

I've changed the values in the error column, but the error seems to be depending on length rather than value.

Any help appreciatied!

Jim

2007-Jul-19 15:13:41 by drh:
We will be better able to help you with your problem on the SQLite mailing list. See http://www.sqlite.org/support.html for instructions on joining the mailing list.
 
2510 code active 2007 Jul anonymous   2007 Jul   1 1 Vacuum modified FTS2 rowids edit
VACUUM modifies FTS2 rowids. Here is the test:

  drop table if exists a;

  create virtual table a using fts2 (t);

  insert into a (t) values ('one');
  insert into a (t) values ('two');
  insert into a (t) values ('three');

  select rowid, * from a;

  delete from a where t = 'two';
  vacuum;

  select rowid, * from a;

Unfortunately there is no workaround since table a is auto-generated by the FTS2 module.

2007-Jul-17 14:05:58 by anonymous:
http://www.sqlite.org/cvstrac/chngview?cn=4157


2007-Jul-17 14:24:29 by anonymous:
Yes, this behavior has been recently documented, but there is no user workaround like PRIMARY KEY for FTS2 rowids. Therefore I consider this as a bug which should be fixed in fts2.c.


2007-Jul-17 14:55:57 by anonymous:
Should virtual tables be VACUUMable? What exactly is being vacuumed here - an internal table?


2007-Jul-17 16:34:55 by shess:
I agree, I think this is a bug. Rather severe, too, the entire fts system implicitely depends on rowids not changing, this means that vacuum will break fts tables (fts1 or fts2).

drop table if exists t;
create virtual table t using fts2;
insert into t (content) values ('This is a test');
insert into t (content) values ('This is a string');
insert into t (content) values ('That was a test');
insert into t (content) values ('A random string');
select content from t where t MATCH 'test';
delete from t where content = 'This is a string';
vacuum;
select content from t where t MATCH 'test';
The first select outputs 'This is a test' and 'That was a test'. The second select outputs 'This is a test', and 'A random string'.


2007-Jul-17 17:27:21 by anonymous:
This patch seems to address the FTS2 VACUUM problem and passes all fts2 tests.

It adds an INTEGER PRIMARY KEY docid column to the hidden %_content table.

Note: this new table format is not backwards compatible with existing FTS2 databases.

-Joe Wilson

Index: ext/fts2/fts2.c
===================================================================
RCS file: /sqlite/sqlite/ext/fts2/fts2.c,v
retrieving revision 1.40
diff -u -3 -p -r1.40 fts2.c
--- ext/fts2/fts2.c	2 Jul 2007 10:16:50 -0000	1.40
+++ ext/fts2/fts2.c	17 Jul 2007 17:19:49 -0000
@@ -1769,9 +1769,9 @@ typedef enum fulltext_statement {
 */
 static const char *const fulltext_zStatement[MAX_STMT] = {
   /* CONTENT_INSERT */ NULL,  /* generated in contentInsertStatement() */
-  /* CONTENT_SELECT */ "select * from %_content where rowid = ?",
+  /* CONTENT_SELECT */ "select * from %_content where docid = ?",
   /* CONTENT_UPDATE */ NULL,  /* generated in contentUpdateStatement() */
-  /* CONTENT_DELETE */ "delete from %_content where rowid = ?",
+  /* CONTENT_DELETE */ "delete from %_content where docid = ?",

   /* BLOCK_INSERT */ "insert into %_segments values (?)",
   /* BLOCK_SELECT */ "select block from %_segments where rowid = ?",
@@ -1860,14 +1860,14 @@ static struct fulltext_vtab *cursor_vtab
 static const sqlite3_module fts2Module;   /* forward declaration */

 /* Return a dynamically generated statement of the form
- *   insert into %_content (rowid, ...) values (?, ...)
+ *   insert into %_content (docid, ...) values (?, ...)
  */
 static const char *contentInsertStatement(fulltext_vtab *v){
   StringBuffer sb;
   int i;

   initStringBuffer(&sb);
-  append(&sb, "insert into %_content (rowid, ");
+  append(&sb, "insert into %_content (docid, ");
   appendList(&sb, v->nColumn, v->azContentColumn);
   append(&sb, ") values (?");
   for(i=0; i<v->nColumn; ++i)
@@ -1878,7 +1878,7 @@ static const char *contentInsertStatemen

 /* Return a dynamically generated statement of the form
  *   update %_content set [col_0] = ?, [col_1] = ?, ...
- *                    where rowid = ?
+ *                    where docid = ?
  */
 static const char *contentUpdateStatement(fulltext_vtab *v){
   StringBuffer sb;
@@ -1893,7 +1893,7 @@ static const char *contentUpdateStatemen
     append(&sb, v->azContentColumn[i]);
     append(&sb, " = ?");
   }
-  append(&sb, " where rowid = ?");
+  append(&sb, " where docid = ?");
   return stringBufferData(&sb);
 }

@@ -2027,15 +2027,15 @@ static int sql_step_leaf_statement(fullt
   return rc;
 }

-/* insert into %_content (rowid, ...) values ([rowid], [pValues]) */
-static int content_insert(fulltext_vtab *v, sqlite3_value *rowid,
+/* insert into %_content (docid, ...) values ([docid], [pValues]) */
+static int content_insert(fulltext_vtab *v, sqlite3_value *docid,
                           sqlite3_value **pValues){
   sqlite3_stmt *s;
   int i;
   int rc = sql_get_statement(v, CONTENT_INSERT_STMT, &s);
   if( rc!=SQLITE_OK ) return rc;

-  rc = sqlite3_bind_value(s, 1, rowid);
+  rc = sqlite3_bind_value(s, 1, docid);
   if( rc!=SQLITE_OK ) return rc;

   for(i=0; i<v->nColumn; ++i){
@@ -2047,7 +2047,7 @@ static int content_insert(fulltext_vtab
 }

 /* update %_content set col0 = pValues[0], col1 = pValues[1], ...
- *                  where rowid = [iRowid] */
+ *                  where docid = [iRowid] */
 static int content_update(fulltext_vtab *v, sqlite3_value **pValues,
                           sqlite_int64 iRowid){
   sqlite3_stmt *s;
@@ -2075,7 +2075,7 @@ static void freeStringArray(int nString,
   free((void *) pString);
 }

-/* select * from %_content where rowid = [iRow]
+/* select * from %_content where docid = [iRow]
  * The caller must delete the returned array and all strings in it.
  * null fields will be NULL in the returned array.
  *
@@ -2101,10 +2101,10 @@ static int content_select(fulltext_vtab

   values = (const char **) malloc(v->nColumn * sizeof(const char *));
   for(i=0; i<v->nColumn; ++i){
-    if( sqlite3_column_type(s, i)==SQLITE_NULL ){
+    if( sqlite3_column_type(s, i+1)==SQLITE_NULL ){
       values[i] = NULL;
     }else{
-      values[i] = string_dup((char*)sqlite3_column_text(s, i));
+      values[i] = string_dup((char*)sqlite3_column_text(s, i+1));
     }
   }

@@ -2120,7 +2120,7 @@ static int content_select(fulltext_vtab
   return rc;
 }

-/* delete from %_content where rowid = [iRow ] */
+/* delete from %_content where docid = [iRow ] */
 static int content_delete(fulltext_vtab *v, sqlite_int64 iRow){
   sqlite3_stmt *s;
   int rc = sql_get_statement(v, CONTENT_DELETE_STMT, &s);
@@ -2870,7 +2870,7 @@ static int fulltextCreate(sqlite3 *db, v
   if( rc!=SQLITE_OK ) return rc;

   initStringBuffer(&schema);
-  append(&schema, "CREATE TABLE %_content(");
+  append(&schema, "CREATE TABLE %_content(docid INTEGER PRIMARY KEY, ");
   appendList(&schema, spec.nColumn, spec.azContentColumn);
   append(&schema, ")");
   rc = sql_exec(db, spec.zDb, spec.zName, stringBufferData(&schema));
@@ -3731,8 +3731,8 @@ static int fulltextFilter(

   TRACE(("FTS2 Filter %p\n",pCursor));

-  zSql = sqlite3_mprintf("select rowid, * from %%_content %s",
-                          idxNum==QUERY_GENERIC ? "" : "where rowid=?");
+  zSql = sqlite3_mprintf("select * from %%_content %s",
+                          idxNum==QUERY_GENERIC ? "" : "where docid=?");
   sqlite3_finalize(c->pStmt);
   rc = sql_prepare(v->db, v->zDb, v->zName, &c->pStmt, zSql);
   sqlite3_free(zSql);

2007-Jul-18 00:13:56 by shess:
BTW, AFAICT this only happens for sqlite3.4. Older versions don't seem to have the problem.


2007-Jul-18 01:31:49 by anonymous:
The rowid changing after VACUUM predates 3.4.0...

SQLite version 3.3.7
Enter ".help" for instructions
sqlite> CREATE TABLE t(a);
sqlite> INSERT INTO "t" VALUES('one');
sqlite> INSERT INTO "t" VALUES('two');
sqlite> INSERT INTO "t" VALUES('three');
sqlite> select rowid, * from t;
1|one
2|two
3|three
sqlite> delete from t where a = 'one';
sqlite> select rowid, * from t;
2|two
3|three
sqlite> vacuum;
sqlite> select rowid, * from t;
1|two
2|three

SQLite version 3.2.0
Enter ".help" for instructions
sqlite> CREATE TABLE t(a);
sqlite> INSERT INTO "t" VALUES('one');
sqlite> INSERT INTO "t" VALUES('two');
sqlite> INSERT INTO "t" VALUES('three');
sqlite> select rowid, * from t;
1|one
2|two
3|three
sqlite> delete from t where a = 'one';
sqlite> select rowid, * from t;
2|two
3|three
sqlite> vacuum;
sqlite> select rowid, * from t;
1|two
2|three

2007-Jul-18 15:59:24 by anonymous:
As you may know, INTEGER PRIMARY KEY indexes are the ROWID, so I must supect they would change after a VACUUM.

The best workaround is to put docid as INTEGER, then adding a PRIMARY KEY index for the docid column.

 
2503 code active 2007 Jul anonymous   2007 Jul   3 4 sqlite3PagerReleaseMemory does not decrement page count edit
When cached pages are released by sqlite3PagerReleaseMemory the number of pages (pPager->nPage) is not decremented. This also subsequently affects the maximum value at pPager->nMaxPage.

This does not affect the operation of sqlite (but does upset my statistic gathering).

Although only tested with 3.3.8, the problem does not appear to have been corrected in version 3.4.0

Actually this will affect the handling of cache_size as the pager will think there are more pages cached than is the case, and may unnecessarily release some.


2007-Jul-18 15:54:29 by anonymous:
Do you have a simple test case that demonstrates this? Put a print statement in pager.c if necessary.
 
2512 code active 2007 Jul shess   2007 Jul   1 1 FTS virtual table name quoting problem edit
All table names should be quoted in the FTS module code.

with TRACE enabled in ext/fts2/fts2.c:

sqlite> create virtual table "a b c" using fts2 (t);
FTS2 Create
FTS2 sql: CREATE TABLE main.a b c_content(c0t)
SQL error: vtable constructor failed: a b c
2007-Jul-18 06:44:21 by anonymous:
A similar problem shows if a FTS column has the same name as the FTS table:

  CREATE VIRTUAL TABLE a USING fts2 (a);

Returns "vtable constructor failed: a.".

 
2511 code active 2007 Jul anonymous   2007 Jul drh 3 2 Inconsistent Pragma output edit
Pragma output is inconsistent when setting the value. Most do not generate any output and silently set the value, while others generate a singleton row with the set value. Here is a list of pragmas that generate output while setting the values:

sqlite> PRAGMA locking_mode = NORMAL; normal sqlite> PRAGMA max_page_count = 100000; 100000

The following do not generate any output upon query: PRAGMA case_sensitive_like; PRAGMA incremental_vacuum;

Sqlite was built from almagamation using the following configuration flags: --enable-threadsafe --disable-tcl --enable-tempstore

 
2509 code active 2007 Jul anonymous   2007 Jul   1 1 SQLITE_DATE edit
SELECT CAST(MyDate AS DATE), CAST(MyTime AS TIME) FROM MyData

I hope, it will result/return DATE, TIME. Please support to SQLITE_DATE and SQLITE_TIME. Thanks.

 
2498 code active 2007 Jul anonymous   2007 Jul   3 2 sqlite memory org on linux edit
(related ticket #2473)... he sample programme that I run(wrote) in tty1 and there I operate the command of ps at tty2, there seems two items from the programme of ps command. This error was not at the version 3.3.13 but now it is happening at sqlite versions although i change nothing from the programme, If I turn to old versions, there is seen only one item again. When I upgrade to version 3.3.13 or later, there is seen two items again Is it normal or there is any mistake? (excuse my poor english)
2007-Jul-11 16:44:22 by anonymous:
So you are seeing 2 processes instead of 1 on Linux? Linux 2.4 and earlier kernels show threads as seperate processes with unique process IDs. Is your program creating any threads?

The only place where SQLite creates threads is the function below - but it joins with the thread right away.

/*
** This procedure attempts to determine whether or not threads
** can override each others locks then sets the
** threadsOverrideEachOthersLocks variable appropriately.
*/
static void testThreadLockingBehavior(int fd_orig){
  int fd;
  struct threadTestData d[2];
  pthread_t t[2];

  fd = dup(fd_orig);
  if( fd<0 ) return;
  memset(d, 0, sizeof(d));
  d[0].fd = fd;
  d[0].lock.l_type = F_RDLCK;
  d[0].lock.l_len = 1;
  d[0].lock.l_start = 0;
  d[0].lock.l_whence = SEEK_SET;
  d[1] = d[0];
  d[1].lock.l_type = F_WRLCK;
  pthread_create(&t[0], 0, threadLockingTest, &d[0]);
  pthread_create(&t[1], 0, threadLockingTest, &d[1]);
  pthread_join(t[0], 0);
  pthread_join(t[1], 0);
  close(fd);
  threadsOverrideEachOthersLocks =  d[0].result==0 && d[1].result==0;
}
If you post a small C program demonstrating what you're seeing, someone may be able to offer a suggestion.


2007-Jul-11 16:47:10 by anonymous:
I suppose it's not inconceivable that the join failed. Perhaps these pthread_join calls' return codes should be examined for errors.


2007-Jul-11 18:53:36 by anonymous:
If you're playing games with tty's and you've got an early Linux 2.6 kernel, it's possible that processes are dying because of http://lkml.org/lkml/2004/10/21/119. It was, last I checked, fixed in 2.6.10.

The SIGHUP being generated might also interfer with a pthread_join(), although pthread_join() doesn't say anything about ever generating EINTR...

c.


2007-Jul-12 06:12:28 by anonymous:
my example program is very simple, i not use threading-multithreading structure... If I turn to old versions of sqlite, there is seen only one item again, when I upgrade to version 3.3.13 or later, there is seen two items again Is it.

note: /lib/libpthread.so.0 linked to /lib/libpthread-0.10.so (size 55468 byte)


2007-Jul-12 11:36:37 by anonymous:
Your description of the problem isn't clear enough, so the answers you're getting are just guesses.

You may have more luck by describing the problem (with as much detail as possible) in your native language and hoping someone in the SQLite community can add a translation. I know you're doing your best with the english you speak, but it's not working well enough for someone to help with your problem.

Adding code samples and command-line output would also help considerably, since that sort of this is mostly language independent.

 
2496 code active 2007 Jul anonymous   2007 Jul   5 4 "No such column" error should include table information edit
It'd be nice if the "no such column" error included the table/view that SQLite was searching for the column.

no such column: ChecklistID

Thanks,

Sam

 
2491 code active 2007 Jul anonymous   2007 Jul   1 1 Mingw Warnings w/ 3.4.0 Amalgamation edit
When compiling the 3.4.0 amalgamation sqlite3.c file w/ no defines, you get the following warnings:

sqlite3/sqlite3.c: In function `sqlite3BtreeFindCell':
sqlite3/sqlite3.c:23249: warning: unused variable `data'
sqlite3/sqlite3.c: In function `vxprintf':
sqlite3/sqlite3.c:8488: warning: 'xtype' might be used uninitialized in this function
sqlite3/sqlite3.c: In function `sqlite3BtreeOpen':
sqlite3/sqlite3.c:19488: warning: 'nameLen' might be used uninitialized in this function
sqlite3/sqlite3.c: In function `getOverflowPage':
sqlite3/sqlite3.c:25386: warning: 'rc' might be used uninitialized in this function
sqlite3/sqlite3.c: In function `sqlite3Select':
sqlite3/sqlite3.c:56300: warning: 'pEList' might be used uninitialized in this function
sqlite3/sqlite3.c:56301: warning: 'pTabList' might be used uninitialized in this function
sqlite3/sqlite3.c: At top level:
sqlite3/sqlite3.c:16020: warning: 'sqlite3GenericAllocationSize' defined but not used
sqlite3/sqlite3.c:6188: warning: 'sqlite3Utf16Substr' declared `static' but never defined
sqlite3/sqlite3.c:6307: warning: 'sqlite3Get2byte' declared `static' but never defined
sqlite3/sqlite3.c:6309: warning: 'sqlite3Put2byte' declared `static' but never defined
sqlite3/sqlite3.c:23248: warning: 'sqlite3BtreeFindCell' defined but not used
sqlite3/sqlite3.c:63547: warning: 'sqlite3ParserAlloc' defined but not used
sqlite3/sqlite3.c:63673: warning: 'sqlite3ParserFree' defined but not used
sqlite3/sqlite3.c:65286: warning: 'sqlite3Parser' defined but not used

I know the uninitialized warnings are false warnings but the defined functions that aren't used seem to be an error in building the amalgamation.

 
2487 code active 2007 Jul anonymous   2007 Jul   1 1 SQLite database locked error on NFS mounted home dir edit
I have a c program using the provided API. My home directory is NFS mounted, Im using SQLite 3.3.17. I open a new database using "sqlite3_open", then strcpy () a SQL command to create a table, and run "sqlite3_exec" with this string. I get a return code of 5=database locked. I then tried to manually (command line using sqlite3) create a table within a database in my home dir, that fails too.

   ===========
   x@y> sqlite3 db2
   SQLite version 3.3.17
   Enter ".help" for instructions
   sqlite> create table test (Lastname varchar);
   SQL error: database is locked
   sqlite>
   ==============

If I try this on my local machine (a Mac), it works fine, but I need it to work in my home directory mounted via NFS as that is where the output of our program goes

2007-Jul-06 19:04:15 by anonymous:
If you're using a Mac, compile sqlite with SQLITE_ENABLE_LOCKING_STYLE in os_unix.c


2007-Jul-07 11:51:10 by drh:
This is a problem with your NFS implementation - it does not appear to support posix advisory locking. There is nothing much that SQLite can do about this.

Anonymous above suggests making use of the dot-locking mechanism contributed by Apple. This might be an effective work-around. But remember that there is performance impact. Also remember that an SQLite database that uses dot-locking is subtly imcompatible with a standard SQLite database. The file format itself is the same, but if two processes try to access the database file at the same time and one uses dot-locks and the other uses posix advisory locks, you will end up with corruption.


2007-Jul-07 12:44:09 by anonymous:
It's very odd that Apple does not fix their Mac OSX POSIX locks for NFS given their resources.
 
2484 new active 2007 Jul anonymous   2007 Jul   5 4 Support for RETURNING edit
I was recently trying to get HTSQL (http://htsql.org) to work with SQLite, especially since it'd be nice to work out-of-the-box with Python. One of the hiccups was the lack of a RETURNING clause, this is especially important once you have auto-incremented keys. For example..

  INSERT INTO TABLE some_table (a_column) values ('value')
    RETURNING (serial_column);

This acts like a SELECT following the INSERT returning the requested columns on the affected rows. It is quite helpful for cases like UPDATE or DELETE when more than one row is affected. While this feature isn't critical for SQLite, it reduces client-side code significantly. Thank you for your kind consideration.

 
2377 build active 2007 May pweilbacher   2007 Jul   4 3 Allow easy DLL build on OS/2 edit
Makefile.in contains a target to build a DLL on Windows but unfortunately it doesn't work for OS/2. Current GCC versions use a calling convention that prepends underscores and these need to go into the .def file. To make a nice DLL header some extra lines in the .def file would be nice, too, that are probably incompatible with Windows linkers. Finally, to make the DLL usable we need to create an import library .lib.

As a nice-to-have feature I would like the DLL to be named after the VERSION but without the dot, as I expect DLLs from version 3.0.x to be imcompatible with 3.3.x or other future 3.x versions... I don't know a clever way to do that other than introducing a new variable into configure.ac and Makefile.in.

2007-Jul-03 23:42:20 by pweilbacher:
SQLite releases for OS/2 are now built from the amalgamation, so this is only useful for checks of the CVS code between releases.

Not sure if it still makes sense to check this in, but at least the patch can stay attached to the ticket for possible future reference.

 
2459 code active 2007 Jun anonymous   2007 Jun   4 4 changes() not reporting correctly after "failed" multiline INSERT edit
I have described this bug at http://www.php.net/manual/en/function.sqlite-exec.php#75962.
2007-Jun-25 23:37:22 by drh:
The bug reports says "If you run a multiline ... and there is a SQL error in any of the lines..." but in the accompanying example code, there are no errors. So I do not understand what the problem is.

I do not know what SQLite function the PDO method changes() is bound to. Let us assume that it is bound to sqlite3_changes(). In that case, it returns the number rows that changed in the most recent SQL statement - the last line of the multiline input. This is what SQLite is suppose to do. If you want to know the total number of changes across all the lines in your multiline input, then you have to call sqlite3_total_changes() before and after the multiline SQL and take the difference.


2007-Jun-26 00:10:08 by anonymous:
The error is in the $ins_query variable. The third INSERT is incorrectly spelled as "INSECT". This will cause an error. If you correct the spelling and run the code, changes() will return the integer result "3" to indicate that 3 records have been changed.


2007-Jun-26 00:29:56 by drh:
OK. I generated the following test case:

   CREATE TABLE t1(x INTEGER PRIMARY KEY, name CHAR(255));
   SELECT total_changes();
   INSERT INTO t1(name) VALUES('Ilia1');
   INSERT INTO t1(name) VALUES('Ilia2');
   INSECT INTO t1(name) VALUES('Ilia3');
   SELECT total_changes(), changes();
   SELECT * from t1;

Notice the changes of R->C in the last INSERT. This script appears to do the correct thing when handed in batch to sqlite3_exec() by the command-line shell.

   0
   SQL error near line 5: near "INSECT": syntax error
   2|1
   1|Ilia1
   2|Ilia2

Can you suggest revisions to this script that might induce the error? Do you know how PHP implements its "changes()" method? What is the "changes()" method suppose to show? Do you know that the "changes()" method is implemented correctly in PHP?


2007-Jun-26 05:52:57 by anonymous:
I am not using a command-line interface, and I am not using SQLite 3 (so would sqlite3_exec and my exec operate differently?). I am running these commands from PHP on a web server.

phpinfo() returns this information:

PHP Version: 5.1.6
System: SunOS snail 5.9 sun4u
Server API: Apache

I have created a working version of my initial example, as well as ported your example to my environment:

EDIT: I am aware that my port of your example produces a warning for "no such function". What I do not know is why; I was hoping that you could tell me if total_changes() is something supported only in a command line (TCL, C, C++, or other) environment, if it is not supported in SQLite 2.8.17, or if there is another reason you may be aware of.

EDIT2: I have provided an additional live example of what the server I'm using does with changes() when there is not an error. To clarify, if a multiline query contains no errors, changes() will show that multiple records have been changed, not just the one record affected by the last line of the statement. What I had expected is that, in this scenario, either changes() would report that there HAVE been rows changed, or that a multiline query with an error would not run, or that sqlite would undo the statements that DID execute in a multiline statement once it encountered an error. If there are ways that this CAN be done in SQLite 2.8.17, let me know. Otherwise, I will probably write up some logic in PHP to create the same functionality.


2007-Jun-27 00:25:10 by drh:
I did not notice before that you were talking about 2.8.17.

That version of SQLite is in maintenance mode. We will leave this ticket open for reference, but because 2.8.17 is so very very old and because this is not a serious bug, the problem will likely not be fixed anytime soon. Had this been a bug that was a security vulnerability or could lead to database corruption, we would look into it. But the policy with 2.8.17 is that minor things like this go unfixed. We feel that it is better to focus our efforts on the 3.x series.

You are encouraged to find a work-around. In this case, a good work-around seems to be to not enter invalid SQL...


2007-Jun-27 04:02:33 by anonymous:
Naturally, not entering invalid SQL would be a great idea, but I do not expect everyone to enter perfect SQL statements every time, so I want to try to anticipate possible errors and write in logic that would handle such errors. However, I will find some way to get this to work without changes() working as I desired it to.

Thank you for your help.


2007-Jun-27 11:04:11 by anonymous:
"What I had expected is that, in this scenario, either changes() would report that there HAVE been rows changed, or that a multiline query with an error would not run, or that sqlite would undo the statements that DID execute in a multiline statement once it encountered an error. If there are ways that this CAN be done in SQLite 2.8.17, let me know. Otherwise, I will probably write up some logic in PHP to create the same functionality."

I have determined that I can prevent a multiline query with an error from having its correct statements prior to the one with the error affecting the database by using "BEGIN TRANSACTION", "COMMIT TRANSACTION", and "ROLLBACK TRANSACTION" statements manually in combination with the queryExec method and PHP IF statements. There is a live example with source here:

I could also produce a way to calculate rows changed in such a transaction, by storing the number of records before and after the transaction and doing a simple subtraction. However, this would require me to not use ROLLBACK, so I do not see a benefit to writing such a function.


2007-Jun-30 07:45:12 by anonymous:
Oh, I just realized that I never mentioned what I was wanting this functionality for. I was writing an SQL command-line interface in PHP so that I could work with SQLite that was installed on a remote server. I needed a way to execute SQL commands or batch files without having to put it in a PHP page just to get an error and need to reupload everything.
 
2479 code active 2007 Jun anonymous   2007 Jun   1 1 WinCE regression on some systems. Any db open fails. edit
Because Windows CE is a modular system, meaning many parts of it can be optionally ommited by the system builder, some don't include the CP_UTF8 conversion algorithms for MultiByteToWideChar and family. I believe Windows 95 and early 98 systems can also lack this encoding if not updated with a later Internet Explorer version.

Solution is to just use the sqlite internal functions that already know how to do the same thing.

Attached is an untested patch to os_win.c (I don't have a windows machine nor a cross-compiler set up) to show where the problem is and a possible (sub-optimal) solution.

I believe the right thing to do would be to just drop the utf8ToUnicode and unicodeToUtf8 functions, add the sqlite3Utf8to16 equivalent to utf.c and use them instead.

~Nuno Lucas

2007-Jun-29 14:54:11 by anonymous:
The title is wrong. It should say "Any db open using the UTF-8 API", as using the open16 API will work.
 
2474 new active 2007 Jun anonymous   2007 Jun   5 4 Multiple-record comma-delineated INSERT command edit
I believe that both MySQL and DB2 support this feature. Instead of using separate commands for multiple INSERTS, you could use one command, and delineate the separate INSERT data with commas. Having support for this type of INSERT would make migrating MySQL or DB2 files to SQLite easier.

Regular INSERT method:

  INSERT INTO foo VALUES ('Title1',26,NULL);
  INSERT INTO foo VALUES ('Title2',24,NULL);
  INSERT INTO foo VALUES ('Title3',12,NULL);

Delineated INSERT method:

  INSERT INTO foo VALUES
  ('Title1',26,NULL),
  ('Title2',24,NULL),
  ('Title3',12,NULL);
2007-Jun-29 12:32:45 by anonymous:
From the parsing point of view this is a bit interesting. Imagine the multi-insert statement is 100,000 lines long. Do you parse the entire statement for correctness first and hold this entire parsed tree in memory? Or do you begin a transaction, and try to process each sub-insert row by row and rollback if there's any error? I'd think the latter would be better from both a time and memory point of view.

Actually, this multi-insert statement could be optimized to work around the sqlite slow bulk insert issue with multiple keys.

 
2476 todo active 2007 Jun anonymous   2007 Jun   4 3 SQLite3 ignores ORDER BY clause when performing SELECT ... GROUP BY edit
I found that sqlite3 ignores the ORDER BY clause when performing SELECT ... GROUP BY ... ORDER BY ...

Table schema:

  CREATE TABLE events (
    id integer not null primary key,
    title integer
  );

Data:

id title
1 hello
2 hello

Query: SELECT title, id FROM events GROUP BY title ORDER BY id ASC;

Result:

title id
hello 2

Expected result:

title id
hello 1

Note: I don't think this should even work in the first place, because id is not a grouped column, but MySQL and SQLite doesn't seem to have a problem with it. Oracle complains.

2007-Jun-29 07:46:00 by danielk1977:
In SQL, the sorting specified by the ORDER BY clause is performed (logically) after the grouping specified by the GROUP BY clause. In this case a single row - the ORDER BY clause is redundant.

So the problem is that SQLite and MySQL are implementing the non-standard SQL extension of allowing an expression that is neither an aggregate or a part of the GROUP BY clause in the result-set of the SELECT in a different way.

 
2473 warn active 2007 Jun anonymous   2007 Jun anonymous 3 2 sqlite memory org. on linux edit
i use sqlite on linux (2.4.35) and on FreePascal... i write an simple example code on linux_&_sqlite_&_freepascal. when working my example code on tty1, i run ps command on tty2; multiple process id showing about my example program... my example code is very simple... before v3.3.13 : no problem; but v3.3.17 and later is wrong... (excuse my poor english)
2007-Jun-28 13:10:10 by anonymous:
There is no bug mentioned in this ticket.


2007-Jun-28 14:09:36 by drh:
I think I see a hint of a bug report in the description where it says "before v3.3.13: no problem; but v3.3.17 and later is wrong". But that is not anything near enough information to isolate and fix the problem.

halukduman09: Please find a friend or coworker who speaks good English and ask that person to translate for you. We can only help you if we can understand what you are saying. Thanks.

 
2467 code active 2007 Jun anonymous   2007 Jun   4 4 changes() not reporting correctly after DELETE FROM table edit
Similar to the problem reported in #2459, this was originally reported by someone else in the PHP manual for sqlite_changes(). I looked into the description, and produced a live example of the bug here:

The original bug description is here:

EDIT: I found this "bug" noted on the site while browsing random documentation. It seems that this is, in fact, just how the method was designed. Please do correct me if I am wrong. Also, does specifying "WHERE 1" in the "DELETE FROM" statement cause it to delete records individually? Documentation noting this is here:

2007-Jun-27 20:32:31 by anonymous:
yeah. delete from [table_name] drops and reconstructs the table again.
 
2457 build active 2007 Jun anonymous   2007 Jun drh 1 2 Build fails with internal compiler error (Windows, MS VC++ 6.0) edit
I can't say much. Using the TEA tarball for sqlite 3.4.0 and trying to build this on a windows box using MS VC++ 6.0 the compilation aborts with an 'INTERNAL COMPILER ERROR'.

I will attach the log as a remark. There are lots of warnings as well, about double/int conversion, argument int size mismatches. Most worrying is a series of warnings with negative line numbers!. Given that I am actually not even sure if the location where it fails is correct.

... Yes, when I tried to exclude the reported line (25589, amalgamation) via -DSQLITE_OMIT_INCRBLOB the compiler still crashes, but now in line 25555 of the amalgamation.

Note: v3.3.17 builds fine (again tea tarball, amalgamation).

I wonder ... Is it possible to provide a TEA tarball without amalgamation ? I would like to see if that compiles, maybe the amalgamation has become so large that it is hitting some compiler limit.

2007-Jun-25 20:04:23 by anonymous:
Ok, reducing the size of the amalgamation by actually stripping out comments does not help. While the negative line numbers in the warnings go away, ditto the warnings about terminating line number generation, it still runs into the Internal Error.


2007-Jun-25 20:22:41 by anonymous:
Even a version of gcc chokes on the amalgamation when I compile with -g -O3. Try compiling without debug information. If that still fails, you have to build sqlite directly from the sources.


2007-Jun-25 21:16:07 by anonymous:
Which version of gcc is failing? It seems to work fine here:

  gcc -I. -g -O6 -c ./sqlite3.c
  gcc -I. -g -O6 -DHAVE_READLINE=1 -c ../src/shell.c
  gcc -I. -o amalg-sqlite -g ./shell.o ./sqlite3.o -ldl -lpthread -lreadline

works fine. It might be worth filing a gcc bug if you see something radically different with gcc.


2007-Jun-25 21:18:52 by anonymous:
AK Regarding GCC. We are here not using GCC on Windows, so the comment regarding 'gcc -O3' does not apply.


2007-Jun-25 21:20:47 by drh:
The comment above about being unable to compile using gcc -g -O3 is not from the originator of this problem report. I do not have any problem compiling with gcc -g -O3 here on SuSE Linux with gcc 4.1.0. Will the person who reports problems compiling the amalgamation with gcc -g -O3 please add details, such as the operating system and the version of gcc being used?


2007-Jun-25 21:22:31 by anonymous:
AK

Using the exploded sources the compiler error was still present, in the file btree.c. Same code line as in the amalgamation, just different line numbers.

So the thinking that this is a problem of the amalgamation is a red herring. It is something deeper.

Getting all the revisions of btree.c between releases 3.3.17 and 3.40, and bisecting I find that the trouble starts for me with revision 1.382 of that file. Some functions where replaced by macros.

Creating a revers patch and applying it to revision 3.88 (in release 3.4.0) gets this revisions to compile as well.

Which means I now have a workaround, sa a variant of the revers patch can be applied to the amalgamation as well.


2007-Jun-25 21:26:36 by drh:
Reversing patch [4015] results in a performance hit. I am unwilling to fold in a performance hit for all platforms in order to work around a bug in MSVC++ 6.0. Can anybody suggest a better fix?


2007-Jun-25 21:35:55 by anonymous:
AK

My specific built of MSVC++ 6.0 is

Microsoft (R) 32-bit C/C++ Optimizing Compiler Version 12.00.8168 for 80x86


2007-Jun-25 21:57:04 by anonymous:
Install Service pack 6 of Visual Studio 6.0. The KB article that I'm pretty sure covers this bug is:

http://support.microsoft.com/kb/890892


2007-Jun-25 22:16:32 by anonymous:
After hunting down and installing VC6 ServicePack 6 the compiler reports:

Microsoft (R) 32-bit C/C++ Optimizing Compiler Version 12.00.8804 for 80x86

So, it has changed. Even so, the internal compiler error is still present.


2007-Jun-25 22:33:38 by anonymous:
Reboot of the machine is no help.


2007-Jun-25 22:38:18 by anonymous:
VC 6.0 - what compile flags are you using?


2007-Jun-25 23:11:59 by anonymous:
I thought that cygwin gcc-3.4.4 fails with -O3 -g, but it's just 1800 warning messages. No error, as it turns out.

$ gcc -I`pwd` -O3 -g sqlite3.c src/shell.c -o sqlite3

  /cygdrive/c/tmp/cc2D7Vgb.s: Assembler messages:
  /cygdrive/c/tmp/cc2D7Vgb.s:30139: Warning: .stabn: description field '103ff' too big, try a different debug format
  /cygdrive/c/tmp/cc2D7Vgb.s:30145: Warning: .stabn: description field '103fa' too big, try a different debug format
  /cygdrive/c/tmp/cc2D7Vgb.s:30170: Warning: .stabn: description field '103fa' too big, try a different debug format
  /cygdrive/c/tmp/cc2D7Vgb.s:30174: Warning: .stabn: description field '103fc' too big, try a different debug format
  /cygdrive/c/tmp/cc2D7Vgb.s:30184: Warning: .stabn: description field '103fd' too big, try a different debug format
  /cygdrive/c/tmp/cc2D7Vgb.s:30190: Warning: .stabn: description field '103fe' too big, try a different debug format
  /cygdrive/c/tmp/cc2D7Vgb.s:30194: Warning: .stabn: description field
  ...1800 more lines of the same...


2007-Jun-25 23:25:41 by anonymous:
AK Compile Flags: -O2 -W2 -MD.

According to a poster on the sqlite ML the btree.c can be compiled using -Ow is instead of O2.

Went through the options and O1, O2, Ox, Og all run into the error, the others don't.


2007-Jun-26 02:00:23 by anonymous:
FYI, MinGW gcc version 3.4.2 also produces the same 1800 warnings when you compile with -O3 -g. Still produces an object file okay, it's just annoying. No warning when only -O3 used.

  gcc -c -I. -I.. -g -O3 sqlite3.c

  c:\tmp/ccuuaaaa.s: Assembler messages:
c:\tmp/ccuuaaaa.s:31474: Warning: .stabn: description field '1001c' too big, try a different debug format
c:\tmp/ccuuaaaa.s:31480: Warning: .stabn: description field '10017' too big, try a different debug format
c:\tmp/ccuuaaaa.s:31505: Warning: .stabn: description field '10017' too big, try a different debug format
c:\tmp/ccuuaaaa.s:31509: Warning: .stabn: description field '10019' too big, try a different debug format
c:\tmp/ccuuaaaa.s:31519: Warning: .stabn: description field '1001a' too big, try a different debug format
c:\tmp/ccuuaaaa.s:31525: Warning: .stabn: description field '1001b' too big, try a different debug format
c:\tmp/ccuuaaaa.s:31529: Warning: .stabn: description field '1001e' too big, try a different debug format
c:\tmp/ccuuaaaa.s:31615: Warning: .stabs: description field '10017' too big, try a different debug format
...1800 lines of this...
 
2456 new active 2007 Jun anonymous   2007 Jun   5 5 REQ: use index where applicable instead of full table scan edit
There are times when you need information from a table that's held entirely in an index. Would it not make sense to scan over the index in these cases as those are likely to be more densely packed and therefore faster and more cache friendly?

  sqlite> .sc
  CREATE TABLE t1 ( c1 integer, c2 text, c3 text );
  CREATE INDEX idx1 on t1(c1);

so here SELECT SUM(c1) from t1 could be satisfied by scanning over idx1, which might be a lot smaller & more dense than t1.

 
2448 new active 2007 Jun anonymous   2007 Jun   4 3 SQLITE needs to identify public exported symbols edit
A number of platforms allow source code tagging of functions which are meant to be public functions exported in a shared library (most notably windows and platforms that use GCC 4.0 and above).

This tagging is usually accomplished through some define placed in front of the function declaration and definition (SQLITE3_PUBLIC sqlite3_open() for example), and SQLITE3_PUBLIC is set to the proper declaration for the given platform. Example:

    windows it would be: __declspec(dllexport)
    GCC it would be: __attribute__ ((visibility("default")))

see http://gcc.gnu.org/wiki/Visibility for more information.

 
2443 new active 2007 Jun anonymous   2007 Jun   3 3 sqlite should return different exit codes for different errors edit
sqlite should return different exit codes for different errors reported. sqlite always returns exit code 0 or 1. It would be helpful to have different codes. I/O error, locked, interrupted, busy etc. should declare defined return codes. If sqlite is executed from a shell script it is difficult to handle plain text that could change or be reformatted in a later version.

Thanks.

2007-Jun-21 19:59:41 by anonymous:
Just run sqlite3 -batch -bail and grep for the error in the last line.
 
2442 doc active 2007 Jun anonymous   2007 Jun   1 4 mailing list link doesn't always exist in website header edit
The header at the website doesn't always show the "mailing list" link. It is shown on http://www.sqlite.org/contrib but not on the front page. Other problems exist, like the header graphic at http://www.sqlite.org/contrib doesn't link to the top page.

It would be best to have the menu consistent over all pages that it exists on, and to have the graphic link back to the top page.

 
2440 code active 2007 Jun rse   2007 Jun   2 2 pkg-config(1) script "sqlite.pc" does not provide Autoconf's LIBS edit
On some platforms it isn't sufficient to link a library just against "-lsqlite". For instance under Solaris one needs "-lsqlite -lrt" because of the use of "fdatasync()". The SQLite Autoconf glue already contains the necessary check for this in order to correctly build SQLite and especially link its sqlite(1). But this information is not passed through to the applications which use pkg-config(1) to build against SQLite. Possible fix from OpenPKG's "sqlite" package is following:

Index: sqlite3.pc.in
--- sqlite3.pc.in.orig  2004-07-19 06:25:47 +0200
+++ sqlite3.pc.in   2007-06-20 18:09:00 +0200
@@ -8,5 +8,5 @@
 Name: SQLite^M
 Description: SQL database engine^M
 Version: @VERSION@^M
-Libs: -L${libdir} -lsqlite3^M
+Libs: -L${libdir} -lsqlite3 @LIBS@^M
 Cflags: -I${includedir}^M
 
2438 new active 2007 Jun rse   2007 Jun   3 3 More easily allow the building of SQLite with FTS1 and FTS2 edit
I don't know what the intended way is to build SQLite with FTS1 and/or FTS2, but for the OpenPKG "sqlite" package I at least now use the following change -- as I was unable to find any other automated solution. I know that FTS1 and FTS2 are experimental extensions, but if it is too complicated for people to build SQLite with them, they certainly will never become non-experimental ;-) So I recommend to at least provide some build-time glue for them. In the OpenPKG "sqlite" package I now use the following patch which at least provides this glue (one still has to enable it, of course):

Index: Makefile.in
--- Makefile.in.orig    2007-06-14 22:54:38 +0200
+++ Makefile.in 2007-06-20 18:09:00 +0200
@@ -130,6 +130,18 @@
          vdbe.lo vdbeapi.lo vdbeaux.lo vdbeblob.lo vdbefifo.lo vdbemem.lo \
          where.lo utf.lo legacy.lo vtab.lo

+# FTS1 support
+ifdef FTS1
+TCC    += -DSQLITE_ENABLE_FTS1
+LIBOBJ += fts1.lo fts1_hash.lo fts1_porter.lo fts1_tokenizer1.lo
+endif
+
+# FTS2 support
+ifdef FTS2
+TCC    += -DSQLITE_ENABLE_FTS2
+LIBOBJ += fts2.lo fts2_hash.lo fts2_porter.lo fts2_tokenizer1.lo
+endif
+
 # All of the source code files.
 #
 SRC = \
@@ -498,6 +510,23 @@
        -o testfixture $(TESTSRC) $(TOP)/src/tclsqlite.c \
        libsqlite3.la $(LIBTCL)

+fts1.lo:   $(TOP)/ext/fts1/fts1.c $(HDR)
+   $(LTCOMPILE) -c $(TOP)/ext/fts1/fts1.c
+fts1_hash.lo:  $(TOP)/ext/fts1/fts1_hash.c $(HDR)
+   $(LTCOMPILE) -c $(TOP)/ext/fts1/fts1_hash.c
+fts1_porter.lo:    $(TOP)/ext/fts1/fts1_porter.c $(HDR)
+   $(LTCOMPILE) -c $(TOP)/ext/fts1/fts1_porter.c
+fts1_tokenizer1.lo:    $(TOP)/ext/fts1/fts1_tokenizer1.c $(HDR)
+   $(LTCOMPILE) -c $(TOP)/ext/fts1/fts1_tokenizer1.c
+
+fts2.lo:   $(TOP)/ext/fts2/fts2.c $(HDR)
+   $(LTCOMPILE) -c $(TOP)/ext/fts2/fts2.c
+fts2_hash.lo:  $(TOP)/ext/fts2/fts2_hash.c $(HDR)
+   $(LTCOMPILE) -c $(TOP)/ext/fts2/fts2_hash.c
+fts2_porter.lo:    $(TOP)/ext/fts2/fts2_porter.c $(HDR)
+   $(LTCOMPILE) -c $(TOP)/ext/fts2/fts2_porter.c
+fts2_tokenizer1.lo:    $(TOP)/ext/fts2/fts2_tokenizer1.c $(HDR)
+   $(LTCOMPILE) -c $(TOP)/ext/fts2/fts2_tokenizer1.c

 fulltest:  testfixture$(TEXE) sqlite3$(TEXE)
    ./testfixture $(TOP)/test/all.test
 
2427 doc active 2007 Jun anonymous   2007 Jun   4 3 HTML Tidy warnings on new capi3ref.html edit
HTML Tidy (http://tidy.sourceforge.net) produces a large number of warnings on the newly generated http://www.sqlite.org/capi3ref.html.

The warnings could cause problems with certain HTML browsers. It might be worth fixing the auto-generation script accordingly.

Here is the list:

  line 1 column 1 - Warning: missing <!DOCTYPE> declaration
  line 2 column 1 - Warning: inserting implicit <body>
  line 3 column 1 - Warning: discarding unexpected <body>
  line 336 column 1 - Warning: missing </a> before <h2>
  line 337 column 5 - Warning: inserting implicit <a>
  line 338 column 13 - Warning: inserting implicit <a>
  line 338 column 13 - Warning: missing </a> before <pre>
  line 352 column 1 - Warning: missing </a> before <h2>
  line 353 column 5 - Warning: inserting implicit <a>
  line 354 column 13 - Warning: inserting implicit <a>
  line 354 column 13 - Warning: missing </a> before <pre>
  line 368 column 1 - Warning: missing </a> before <h2>
  line 369 column 5 - Warning: inserting implicit <a>
  line 370 column 13 - Warning: inserting implicit <a>
  line 370 column 13 - Warning: missing </a> before <pre>
  line 380 column 1 - Warning: missing </a> before <h2>
  line 381 column 5 - Warning: inserting implicit <a>
  line 382 column 13 - Warning: inserting implicit <a>
  line 382 column 13 - Warning: missing </a> before <pre>
  line 398 column 6 - Warning: inserting implicit <p>
  line 402 column 1 - Warning: missing </a> before <h2>
  line 403 column 5 - Warning: inserting implicit <a>
  line 404 column 13 - Warning: inserting implicit <a>
  line 404 column 13 - Warning: missing </a> before <pre>
  line 415 column 1 - Warning: missing </a> before <h2>
  line 416 column 5 - Warning: inserting implicit <a>
  line 417 column 13 - Warning: inserting implicit <a>
  line 417 column 13 - Warning: missing </a> before <pre>
  line 434 column 1 - Warning: missing </a> before <h2>
  line 435 column 5 - Warning: inserting implicit <a>
  line 436 column 13 - Warning: inserting implicit <a>
  line 436 column 13 - Warning: missing </a> before <pre>
  line 454 column 1 - Warning: missing </a> before <h2>
  line 455 column 5 - Warning: inserting implicit <a>
  line 456 column 13 - Warning: inserting implicit <a>
  line 456 column 13 - Warning: missing </a> before <pre>
  line 473 column 1 - Warning: missing </a> before <h2>
  line 474 column 5 - Warning: inserting implicit <a>
  line 475 column 13 - Warning: inserting implicit <a>
  line 475 column 13 - Warning: missing </a> before <pre>
  line 485 column 1 - Warning: missing </a> before <h2>
  line 486 column 5 - Warning: inserting implicit <a>
  line 487 column 13 - Warning: inserting implicit <a>
  line 487 column 13 - Warning: missing </a> before <pre>
  line 504 column 1 - Warning: missing </a> before <h2>
  line 505 column 5 - Warning: inserting implicit <a>
  line 506 column 13 - Warning: inserting implicit <a>
  line 506 column 13 - Warning: missing </a> before <pre>
  line 515 column 1 - Warning: missing </a> before <h2>
  line 516 column 5 - Warning: inserting implicit <a>
  line 517 column 13 - Warning: inserting implicit <a>
  line 517 column 13 - Warning: missing </a> before <pre>
  line 525 column 1 - Warning: missing </a> before <h2>
  line 526 column 5 - Warning: inserting implicit <a>
  line 527 column 13 - Warning: inserting implicit <a>
  line 527 column 13 - Warning: missing </a> before <pre>
  line 544 column 7 - Warning: inserting implicit <p>
  line 554 column 1 - Warning: missing </a> before <h2>
  line 555 column 5 - Warning: inserting implicit <a>
  line 556 column 13 - Warning: inserting implicit <a>
  line 556 column 13 - Warning: missing </a> before <pre>
  line 569 column 1 - Warning: missing </a> before <h2>
  line 570 column 5 - Warning: inserting implicit <a>
  line 571 column 13 - Warning: inserting implicit <a>
  line 571 column 13 - Warning: missing </a> before <pre>
  line 589 column 1 - Warning: missing </a> before <h2>
  line 590 column 5 - Warning: inserting implicit <a>
  line 591 column 13 - Warning: inserting implicit <a>
  line 591 column 13 - Warning: missing </a> before <pre>
  line 648 column 1 - Warning: missing </a> before <h2>
  line 649 column 5 - Warning: inserting implicit <a>
  line 650 column 13 - Warning: inserting implicit <a>
  line 650 column 13 - Warning: missing </a> before <pre>
  line 666 column 1 - Warning: missing </a> before <h2>
  line 667 column 5 - Warning: inserting implicit <a>
  line 668 column 13 - Warning: inserting implicit <a>
  line 668 column 13 - Warning: missing </a> before <pre>
  line 694 column 1 - Warning: missing </a> before <h2>
  line 695 column 5 - Warning: inserting implicit <a>
  line 696 column 13 - Warning: inserting implicit <a>
  line 696 column 13 - Warning: missing </a> before <pre>
  line 707 column 1 - Warning: missing </a> before <h2>
  line 708 column 5 - Warning: inserting implicit <a>
  line 709 column 13 - Warning: inserting implicit <a>
  line 709 column 13 - Warning: missing </a> before <pre>
  line 722 column 1 - Warning: missing </a> before <h2>
  line 723 column 5 - Warning: inserting implicit <a>
  line 724 column 13 - Warning: inserting implicit <a>
  line 724 column 13 - Warning: missing </a> before <pre>
  line 735 column 1 - Warning: missing </a> before <h2>
  line 736 column 5 - Warning: inserting implicit <a>
  line 737 column 13 - Warning: inserting implicit <a>
  line 737 column 13 - Warning: missing </a> before <pre>
  line 749 column 1 - Warning: missing </a> before <h2>
  line 750 column 5 - Warning: inserting implicit <a>
  line 751 column 13 - Warning: inserting implicit <a>
  line 751 column 13 - Warning: missing </a> before <pre>
  line 764 column 1 - Warning: missing </a> before <h2>
  line 765 column 5 - Warning: inserting implicit <a>
  line 766 column 13 - Warning: inserting implicit <a>
  line 766 column 13 - Warning: missing </a> before <pre>
  line 796 column 1 - Warning: missing </a> before <h2>
  line 797 column 5 - Warning: inserting implicit <a>
  line 798 column 13 - Warning: inserting implicit <a>
  line 798 column 13 - Warning: missing </a> before <pre>
  line 837 column 1 - Warning: missing </a> before <h2>
  line 838 column 5 - Warning: inserting implicit <a>
  line 839 column 13 - Warning: inserting implicit <a>
  line 839 column 13 - Warning: missing </a> before <pre>
  line 855 column 1 - Warning: missing </a> before <h2>
  line 856 column 5 - Warning: inserting implicit <a>
  line 857 column 13 - Warning: inserting implicit <a>
  line 857 column 13 - Warning: missing </a> before <pre>
  line 876 column 1 - Warning: missing </a> before <h2>
  line 877 column 5 - Warning: inserting implicit <a>
  line 878 column 13 - Warning: inserting implicit <a>
  line 878 column 13 - Warning: missing </a> before <pre>
  line 889 column 1 - Warning: missing </a> before <h2>
  line 890 column 5 - Warning: inserting implicit <a>
  line 891 column 13 - Warning: inserting implicit <a>
  line 891 column 13 - Warning: missing </a> before <pre>
  line 907 column 1 - Warning: missing </a> before <h2>
  line 908 column 5 - Warning: inserting implicit <a>
  line 909 column 13 - Warning: inserting implicit <a>
  line 909 column 13 - Warning: missing </a> before <pre>
  line 928 column 1 - Warning: missing </a> before <h2>
  line 929 column 5 - Warning: inserting implicit <a>
  line 930 column 13 - Warning: inserting implicit <a>
  line 930 column 13 - Warning: missing </a> before <pre>
  line 948 column 1 - Warning: missing </a> before <h2>
  line 949 column 5 - Warning: inserting implicit <a>
  line 950 column 13 - Warning: inserting implicit <a>
  line 950 column 13 - Warning: missing </a> before <pre>
  line 976 column 1 - Warning: missing </a> before <h2>
  line 977 column 5 - Warning: inserting implicit <a>
  line 978 column 13 - Warning: inserting implicit <a>
  line 978 column 13 - Warning: missing </a> before <pre>
  line 989 column 1 - Warning: missing </a> before <h2>
  line 990 column 5 - Warning: inserting implicit <a>
  line 991 column 13 - Warning: inserting implicit <a>
  line 991 column 13 - Warning: missing </a> before <pre>
  line 1004 column 1 - Warning: missing </a> before <h2>
  line 1005 column 5 - Warning: inserting implicit <a>
  line 1006 column 13 - Warning: inserting implicit <a>
  line 1006 column 13 - Warning: missing </a> before <pre>
  line 1017 column 1 - Warning: missing </a> before <h2>
  line 1018 column 5 - Warning: inserting implicit <a>
  line 1019 column 13 - Warning: inserting implicit <a>
  line 1019 column 13 - Warning: missing </a> before <pre>
  line 1069 column 1 - Warning: missing </a> before <h2>
  line 1070 column 5 - Warning: inserting implicit <a>
  line 1071 column 13 - Warning: inserting implicit <a>
  line 1071 column 13 - Warning: missing </a> before <pre>
  line 1083 column 1 - Warning: missing </a> before <h2>
  line 1084 column 5 - Warning: inserting implicit <a>
  line 1085 column 13 - Warning: inserting implicit <a>
  line 1085 column 13 - Warning: missing </a> before <pre>
  line 1107 column 1 - Warning: missing </a> before <h2>
  line 1108 column 5 - Warning: inserting implicit <a>
  line 1109 column 13 - Warning: inserting implicit <a>
  line 1109 column 13 - Warning: missing </a> before <pre>
  line 1166 column 1 - Warning: missing </a> before <h2>
  line 1167 column 5 - Warning: inserting implicit <a>
  line 1168 column 13 - Warning: inserting implicit <a>
  line 1168 column 13 - Warning: missing </a> before <pre>
  line 1198 column 36 - Warning: discarding unexpected </p>
  line 1198 column 40 - Warning: using <br> in place of <p>
  line 1198 column 40 - Warning: replacing <p> by <br>
  line 1203 column 7 - Warning: inserting implicit <p>
  line 1215 column 7 - Warning: inserting implicit <p>
  line 1222 column 1 - Warning: missing </a> before <h2>
  line 1223 column 5 - Warning: inserting implicit <a>
  line 1224 column 13 - Warning: inserting implicit <a>
  line 1224 column 13 - Warning: missing </a> before <pre>
  line 1238 column 1 - Warning: missing </a> before <h2>
  line 1239 column 5 - Warning: inserting implicit <a>
  line 1240 column 13 - Warning: inserting implicit <a>
  line 1240 column 13 - Warning: missing </a> before <pre>
  line 1259 column 1 - Warning: missing </a> before <h2>
  line 1260 column 5 - Warning: inserting implicit <a>
  line 1261 column 13 - Warning: inserting implicit <a>
  line 1261 column 13 - Warning: missing </a> before <pre>
  line 1286 column 1 - Warning: missing </a> before <h2>
  line 1287 column 5 - Warning: inserting implicit <a>
  line 1288 column 13 - Warning: inserting implicit <a>
  line 1288 column 13 - Warning: missing </a> before <pre>
  line 1299 column 1 - Warning: missing </a> before <a>
  line 1300 column 1 - Warning: missing </a> before <a>
  line 1301 column 1 - Warning: missing </a> before <a>
  line 1302 column 1 - Warning: missing </a> before <a>
  line 1303 column 1 - Warning: missing </a> before <a>
  line 1304 column 1 - Warning: missing </a> before <a>
  line 1305 column 1 - Warning: missing </a> before <a>
  line 1306 column 1 - Warning: missing </a> before <a>
  line 1307 column 1 - Warning: missing </a> before <a>
  line 1308 column 1 - Warning: missing </a> before <a>
  line 1309 column 1 - Warning: missing </a> before <a>
  line 1310 column 1 - Warning: missing </a> before <a>
  line 1311 column 1 - Warning: missing </a> before <a>
  line 1312 column 1 - Warning: missing </a> before <a>
  line 1313 column 1 - Warning: missing </a> before <a>
  line 1314 column 1 - Warning: missing </a> before <a>
  line 1315 column 1 - Warning: missing </a> before <a>
  line 1316 column 1 - Warning: missing </a> before <a>
  line 1317 column 1 - Warning: missing </a> before <a>
  line 1318 column 1 - Warning: missing </a> before <a>
  line 1319 column 1 - Warning: missing </a> before <a>
  line 1320 column 1 - Warning: missing </a> before <a>
  line 1321 column 1 - Warning: missing </a> before <a>
  line 1322 column 1 - Warning: missing </a> before <a>
  line 1323 column 1 - Warning: missing </a> before <a>
  line 1324 column 1 - Warning: missing </a> before <a>
  line 1325 column 1 - Warning: missing </a> before <a>
  line 1326 column 1 - Warning: missing </a> before <a>
  line 1327 column 1 - Warning: missing </a> before <h2>
  line 1328 column 5 - Warning: inserting implicit <a>
  line 1329 column 13 - Warning: inserting implicit <a>
  line 1329 column 13 - Warning: missing </a> before <pre>
  line 1370 column 1 - Warning: missing </a> before <a>
  line 1371 column 1 - Warning: missing </a> before <a>
  line 1372 column 1 - Warning: missing </a> before <a>
  line 1373 column 1 - Warning: missing </a> before <a>
  line 1374 column 1 - Warning: missing </a> before <a>
  line 1375 column 1 - Warning: missing </a> before <a>
  line 1376 column 1 - Warning: missing </a> before <a>
  line 1377 column 1 - Warning: missing </a> before <a>
  line 1378 column 1 - Warning: missing </a> before <a>
  line 1379 column 1 - Warning: missing </a> before <a>
  line 1380 column 1 - Warning: missing </a> before <a>
  line 1381 column 1 - Warning: missing </a> before <a>
  line 1382 column 1 - Warning: missing </a> before <a>
  line 1383 column 1 - Warning: missing </a> before <a>
  line 1384 column 1 - Warning: missing </a> before <a>
  line 1385 column 1 - Warning: missing </a> before <a>
  line 1386 column 1 - Warning: missing </a> before <a>
  line 1387 column 1 - Warning: missing </a> before <a>
  line 1388 column 1 - Warning: missing </a> before <a>
  line 1389 column 1 - Warning: missing </a> before <a>
  line 1390 column 1 - Warning: missing </a> before <a>
  line 1391 column 1 - Warning: missing </a> before <a>
  line 1392 column 1 - Warning: missing </a> before <a>
  line 1393 column 1 - Warning: missing </a> before <a>
  line 1394 column 1 - Warning: missing </a> before <a>
  line 1395 column 1 - Warning: missing </a> before <a>
  line 1396 column 1 - Warning: missing </a> before <a>
  line 1397 column 1 - Warning: missing </a> before <a>
  line 1398 column 1 - Warning: missing </a> before <a>
  line 1399 column 1 - Warning: missing </a> before <a>
  line 1400 column 1 - Warning: missing </a> before <a>
  line 1401 column 1 - Warning: missing </a> before <h2>
  line 1402 column 5 - Warning: inserting implicit <a>
  line 1403 column 13 - Warning: inserting implicit <a>
  line 1403 column 13 - Warning: missing </a> before <pre>
  line 1455 column 1 - Warning: missing </a> before <a>
  line 1456 column 1 - Warning: missing </a> before <a>
  line 1457 column 1 - Warning: missing </a> before <a>
  line 1458 column 1 - Warning: missing </a> before <a>
  line 1459 column 1 - Warning: missing </a> before <a>
  line 1460 column 1 - Warning: missing </a> before <h2>
  line 1461 column 5 - Warning: inserting implicit <a>
  line 1462 column 13 - Warning: inserting implicit <a>
  line 1462 column 13 - Warning: missing </a> before <pre>
  line 1476 column 1 - Warning: missing </a> before <a>
  line 1477 column 1 - Warning: missing </a> before <a>
  line 1478 column 1 - Warning: missing </a> before <a>
  line 1479 column 1 - Warning: missing </a> before <h2>
  line 1480 column 5 - Warning: inserting implicit <a>
  line 1481 column 13 - Warning: inserting implicit <a>
  line 1481 column 13 - Warning: missing </a> before <pre>
  line 1501 column 6 - Warning: inserting implicit <p>
  line 1507 column 1 - Warning: missing </a> before <a>
  line 1508 column 1 - Warning: missing </a> before <h2>
  line 1509 column 5 - Warning: inserting implicit <a>
  line 1510 column 13 - Warning: inserting implicit <a>
  line 1510 column 13 - Warning: missing </a> before <pre>
  line 1523 column 1 - Warning: missing </a> before <a>
  line 1524 column 1 - Warning: missing </a> before <a>
  line 1525 column 1 - Warning: missing </a> before <a>
  line 1526 column 1 - Warning: missing </a> before <a>
  line 1527 column 1 - Warning: missing </a> before <a>
  line 1528 column 1 - Warning: missing </a> before <a>
  line 1529 column 1 - Warning: missing </a> before <a>
  line 1530 column 1 - Warning: missing </a> before <a>
  line 1531 column 1 - Warning: missing </a> before <a>
  line 1532 column 1 - Warning: missing </a> before <a>
  line 1533 column 1 - Warning: missing </a> before <h2>
  line 1534 column 5 - Warning: inserting implicit <a>
  line 1535 column 13 - Warning: inserting implicit <a>
  line 1535 column 13 - Warning: missing </a> before <pre>
  line 1569 column 1 - Warning: missing </a> before <a>
  line 1570 column 1 - Warning: missing </a> before <h2>
  line 1571 column 5 - Warning: inserting implicit <a>
  line 1572 column 13 - Warning: inserting implicit <a>
  line 1572 column 13 - Warning: missing </a> before <pre>
  line 1589 column 1 - Warning: missing </a> before <a>
  line 1590 column 1 - Warning: missing </a> before <h2>
  line 1591 column 5 - Warning: inserting implicit <a>
  line 1592 column 13 - Warning: inserting implicit <a>
  line 1592 column 13 - Warning: missing </a> before <pre>
  line 1615 column 1 - Warning: missing </a> before <a>
  line 1616 column 1 - Warning: missing </a> before <a>
  line 1617 column 1 - Warning: missing </a> before <a>
  line 1618 column 1 - Warning: missing </a> before <h2>
  line 1619 column 5 - Warning: inserting implicit <a>
  line 1620 column 13 - Warning: inserting implicit <a>
  line 1620 column 13 - Warning: missing </a> before <pre>
  line 1635 column 1 - Warning: missing </a> before <a>
  line 1636 column 1 - Warning: missing </a> before <a>
  line 1637 column 1 - Warning: missing </a> before <a>
  line 1638 column 1 - Warning: missing </a> before <a>
  line 1639 column 1 - Warning: missing </a> before <a>
  line 1640 column 1 - Warning: missing </a> before <a>
  line 1641 column 1 - Warning: missing </a> before <a>
  line 1642 column 1 - Warning: missing </a> before <a>
  line 1643 column 1 - Warning: missing </a> before <h2>
  line 1644 column 5 - Warning: inserting implicit <a>
  line 1645 column 13 - Warning: inserting implicit <a>
  line 1645 column 13 - Warning: missing </a> before <pre>
  line 1666 column 6 - Warning: inserting implicit <p>
  line 1711 column 1 - Warning: missing </a> before <a>
  line 1712 column 1 - Warning: missing </a> before <h2>
  line 1713 column 5 - Warning: inserting implicit <a>
  line 1714 column 13 - Warning: inserting implicit <a>
  line 1714 column 13 - Warning: missing </a> before <pre>
  line 1746 column 1 - Warning: missing </a> before <a>
  line 1747 column 1 - Warning: missing </a> before <a>
  line 1748 column 1 - Warning: missing </a> before <a>
  line 1749 column 1 - Warning: missing </a> before <a>
  line 1750 column 1 - Warning: missing </a> before <a>
  line 1751 column 1 - Warning: missing </a> before <a>
  line 1752 column 1 - Warning: missing </a> before <a>
  line 1753 column 1 - Warning: missing </a> before <a>
  line 1754 column 1 - Warning: missing </a> before <a>
  line 1755 column 1 - Warning: missing </a> before <h2>
  line 1756 column 5 - Warning: inserting implicit <a>
  line 1757 column 13 - Warning: inserting implicit <a>
  line 1757 column 13 - Warning: missing </a> before <pre>
  line 1805 column 49 - Warning: inserting implicit <p>
  line 1822 column 14 - Warning: inserting implicit <p>
  line 1833 column 41 - Warning: discarding unexpected </p>
  line 1833 column 45 - Warning: missing <li>
  line 1835 column 20 - Warning: discarding unexpected </p>
  line 1835 column 24 - Warning: missing <li>
  line 1838 column 6 - Warning: inserting implicit <p>
  line 1847 column 6 - Warning: inserting implicit <p>
  line 1855 column 1 - Warning: missing </a> before <a>
  line 1856 column 1 - Warning: missing </a> before <a>
  line 1857 column 1 - Warning: missing </a> before <a>
  line 1858 column 1 - Warning: missing </a> before <a>
  line 1859 column 1 - Warning: missing </a> before <a>
  line 1860 column 1 - Warning: missing </a> before <h2>
  line 1861 column 5 - Warning: inserting implicit <a>
  line 1862 column 13 - Warning: inserting implicit <a>
  line 1862 column 13 - Warning: missing </a> before <pre>
  line 1890 column 1 - Warning: missing </a> before <a>
  line 1891 column 1 - Warning: missing </a> before <h2>
  line 1892 column 5 - Warning: inserting implicit <a>
  line 1893 column 13 - Warning: inserting implicit <a>
  line 1893 column 13 - Warning: missing </a> before <pre>
  line 1916 column 1 - Warning: missing </a> before <a>
  line 1917 column 1 - Warning: missing </a> before <h2>
  line 1918 column 5 - Warning: inserting implicit <a>
  line 1919 column 13 - Warning: inserting implicit <a>
  line 1919 column 13 - Warning: missing </a> before <pre>
  line 1937 column 1 - Warning: missing </a> before <a>
  line 1938 column 1 - Warning: missing </a> before <h2>
  line 1939 column 5 - Warning: inserting implicit <a>
  line 1940 column 13 - Warning: inserting implicit <a>
  line 1940 column 13 - Warning: missing </a> before <pre>
  line 1958 column 1 - Warning: missing </a> before <a>
  line 1959 column 1 - Warning: missing </a> before <h2>
  line 1960 column 5 - Warning: inserting implicit <a>
  line 1961 column 13 - Warning: inserting implicit <a>
  line 1961 column 13 - Warning: missing </a> before <pre>
  line 1981 column 1 - Warning: missing </a> before <a>
  line 1982 column 1 - Warning: missing </a> before <a>
  line 1983 column 1 - Warning: missing </a> before <h2>
  line 1984 column 5 - Warning: inserting implicit <a>
  line 1985 column 13 - Warning: inserting implicit <a>
  line 1985 column 13 - Warning: missing </a> before <pre>
  line 2039 column 1 - Warning: missing </a> before <a>
  line 2040 column 1 - Warning: missing </a> before <h2>
  line 2041 column 5 - Warning: inserting implicit <a>
  line 2042 column 13 - Warning: inserting implicit <a>
  line 2042 column 13 - Warning: missing </a> before <pre>
  line 2108 column 1 - Warning: missing </a> before <a>
  line 2109 column 1 - Warning: missing </a> before <a>
  line 2110 column 1 - Warning: missing </a> before <h2>
  line 2111 column 5 - Warning: inserting implicit <a>
  line 2112 column 13 - Warning: inserting implicit <a>
  line 2112 column 13 - Warning: missing </a> before <pre>
  line 2139 column 1 - Warning: missing </a> before <a>
  line 2140 column 1 - Warning: missing </a> before <a>
  line 2141 column 1 - Warning: missing </a> before <h2>
  line 2142 column 5 - Warning: inserting implicit <a>
  line 2143 column 13 - Warning: inserting implicit <a>
  line 2143 column 13 - Warning: missing </a> before <pre>
  line 2157 column 1 - Warning: missing </a> before <a>
  line 2158 column 1 - Warning: missing </a> before <h2>
  line 2159 column 5 - Warning: inserting implicit <a>
  line 2160 column 13 - Warning: inserting implicit <a>
  line 2160 column 13 - Warning: missing </a> before <pre>
  line 2183 column 7 - Warning: inserting implicit <p>
  line 2183 column 39 - Warning: unescaped & or unknown entity "&azResult"
  line 2193 column 7 - Warning: inserting implicit <p>
  line 2204 column 1 - Warning: missing </a> before <a>
  line 2205 column 1 - Warning: missing </a> before <h2>
  line 2206 column 5 - Warning: inserting implicit <a>
  line 2207 column 13 - Warning: inserting implicit <a>
  line 2207 column 13 - Warning: missing </a> before <pre>
  line 2235 column 1 - Warning: missing </a> before <a>
  line 2236 column 1 - Warning: missing </a> before <h2>
  line 2237 column 5 - Warning: inserting implicit <a>
  line 2238 column 13 - Warning: inserting implicit <a>
  line 2238 column 13 - Warning: missing </a> before <pre>
  line 2257 column 1 - Warning: missing </a> before <a>
  line 2258 column 1 - Warning: missing </a> before <a>
  line 2259 column 1 - Warning: missing </a> before <h2>
  line 2260 column 5 - Warning: inserting implicit <a>
  line 2261 column 13 - Warning: inserting implicit <a>
  line 2261 column 13 - Warning: missing </a> before <pre>
  line 2298 column 20 - Warning: inserting implicit <p>
  line 2302 column 20 - Warning: inserting implicit <p>
  line 2305 column 20 - Warning: inserting implicit <p>
  line 2308 column 20 - Warning: inserting implicit <p>
  line 2317 column 20 - Warning: inserting implicit <p>
  line 2321 column 1 - Warning: missing </a> before <a>
  line 2322 column 1 - Warning: missing </a> before <h2>
  line 2323 column 5 - Warning: inserting implicit <a>
  line 2324 column 13 - Warning: inserting implicit <a>
  line 2324 column 13 - Warning: missing </a> before <pre>
  line 2352 column 1 - Warning: missing </a> before <a>
  line 2353 column 1 - Warning: missing </a> before <a>
  line 2354 column 1 - Warning: missing </a> before <a>
  line 2355 column 1 - Warning: missing </a> before <h2>
  line 2356 column 5 - Warning: inserting implicit <a>
  line 2357 column 13 - Warning: inserting implicit <a>
  line 2357 column 13 - Warning: missing </a> before <pre>
  line 2423 column 6 - Warning: discarding unexpected </p>
  line 2423 column 10 - Warning: missing <li>
  line 2435 column 1 - Warning: inserting implicit <p>
  line 2437 column 1 - Warning: missing </a> before <a>
  line 2438 column 1 - Warning: missing </a> before <h2>
  line 2439 column 5 - Warning: inserting implicit <a>
  line 2440 column 13 - Warning: inserting implicit <a>
  line 2440 column 13 - Warning: missing </a> before <pre>
  line 2457 column 1 - Warning: missing </a> before <a>
  line 2458 column 1 - Warning: missing </a> before <a>
  line 2459 column 1 - Warning: missing </a> before <a>
  line 2460 column 1 - Warning: missing </a> before <a>
  line 2461 column 1 - Warning: missing </a> before <a>
  line 2462 column 1 - Warning: missing </a> before <a>
  line 2463 column 1 - Warning: missing </a> before <a>
  line 2464 column 1 - Warning: missing </a> before <a>
  line 2465 column 1 - Warning: missing </a> before <a>
  line 2466 column 1 - Warning: missing </a> before <a>
  line 2467 column 1 - Warning: missing </a> before <a>
  line 2468 column 1 - Warning: missing </a> before <a>
  line 2469 column 1 - Warning: missing </a> before <a>
  line 2470 column 1 - Warning: missing </a> before <h2>
  line 2471 column 5 - Warning: inserting implicit <a>
  line 2472 column 13 - Warning: inserting implicit <a>
  line 2472 column 13 - Warning: missing </a> before <pre>
  line 2506 column 1 - Warning: missing </a> before <a>
  line 2507 column 1 - Warning: missing </a> before <a>
  line 2508 column 1 - Warning: missing </a> before <a>
  line 2509 column 1 - Warning: missing </a> before <a>
  line 2510 column 1 - Warning: missing </a> before <a>
  line 2511 column 1 - Warning: missing </a> before <a>
  line 2512 column 1 - Warning: missing </a> before <a>
  line 2513 column 1 - Warning: missing </a> before <a>
  line 2514 column 1 - Warning: missing </a> before <a>
  line 2515 column 1 - Warning: missing </a> before <a>
  line 2516 column 1 - Warning: missing </a> before <a>
  line 2517 column 1 - Warning: missing </a> before <h2>
  line 2518 column 5 - Warning: inserting implicit <a>
  line 2519 column 13 - Warning: inserting implicit <a>
  line 2519 column 13 - Warning: missing </a> before <pre>
  line 4 column 1 - Warning: <table> lacks "summary" attribute
  line 6 column 22 - Warning: <img> lacks "alt" attribute
  line 38 column 1 - Warning: <table> lacks "summary" attribute
  line 46 column 1 - Warning: <table> lacks "summary" attribute
  line 61 column 1 - Warning: <table> lacks "summary" attribute
  line 159 column 1 - Warning: <table> lacks "summary" attribute
  line 338 column 13 - Warning: <a> anchor "sqlite3" already defined
  line 354 column 13 - Warning: <a> anchor "sqlite3_blob" already defined
  line 370 column 13 - Warning: <a> anchor "sqlite3_context" already defined
  line 382 column 13 - Warning: <a> anchor "sqlite3_stmt" already defined
  line 404 column 13 - Warning: <a> anchor "sqlite3_value" already defined
  line 417 column 13 - Warning: <a> anchor "sqlite3_aggregate_context" already defined
  line 436 column 13 - Warning: <a> anchor "sqlite3_auto_extension" already defined
  line 456 column 13 - Warning: <a> anchor "sqlite3_bind_parameter_count" already defined
  line 475 column 13 - Warning: <a> anchor "sqlite3_bind_parameter_index" already defined
  line 487 column 13 - Warning: <a> anchor "sqlite3_bind_parameter_name" already defined
  line 506 column 13 - Warning: <a> anchor "sqlite3_blob_bytes" already defined
  line 517 column 13 - Warning: <a> anchor "sqlite3_blob_close" already defined
  line 527 column 13 - Warning: <a> anchor "sqlite3_blob_open" already defined
  line 556 column 13 - Warning: <a> anchor "sqlite3_blob_read" already defined
  line 571 column 13 - Warning: <a> anchor "sqlite3_blob_write" already defined
  line 591 column 13 - Warning: <a> anchor "sqlite3_busy_handler" already defined
  line 650 column 13 - Warning: <a> anchor "sqlite3_busy_timeout" already defined
  line 668 column 13 - Warning: <a> anchor "sqlite3_changes" already defined
  line 696 column 13 - Warning: <a> anchor "sqlite3_clear_bindings" already defined
  line 709 column 13 - Warning: <a> anchor "sqlite3_close" already defined
  line 724 column 13 - Warning: <a> anchor "sqlite3_column_count" already defined
  line 737 column 13 - Warning: <a> anchor "sqlite3_db_handle" already defined
  line 751 column 13 - Warning: <a> anchor "sqlite3_enable_load_extension" already defined
  line 766 column 13 - Warning: <a> anchor "sqlite3_enable_shared_cache" already defined
  line 798 column 13 - Warning: <a> anchor "sqlite3_exec" already defined
  line 839 column 13 - Warning: <a> anchor "sqlite3_extended_result_codes" already defined
  line 857 column 13 - Warning: <a> anchor "sqlite3_finalize" already defined
  line 878 column 13 - Warning: <a> anchor "sqlite3_get_autocommit" already defined
  line 891 column 13 - Warning: <a> anchor "sqlite3_interrupt" already defined
  line 909 column 13 - Warning: <a> anchor "sqlite3_last_insert_rowid" already defined
  line 930 column 13 - Warning: <a> anchor "sqlite3_load_extension" already defined
  line 950 column 13 - Warning: <a> anchor "sqlite3_progress_handler" already defined
  line 978 column 13 - Warning: <a> anchor "sqlite3_release_memory" already defined
  line 991 column 13 - Warning: <a> anchor "sqlite3_reset" already defined
  line 1006 column 13 - Warning: <a> anchor "sqlite3_reset_auto_extension" already defined
  line 1019 column 13 - Warning: <a> anchor "sqlite3_set_authorizer" already defined
  line 1071 column 13 - Warning: <a> anchor "sqlite3_sleep" already defined
  line 1085 column 13 - Warning: <a> anchor "sqlite3_soft_heap_limit" already defined
  line 1109 column 13 - Warning: <a> anchor "sqlite3_step" already defined
  line 1168 column 13 - Warning: <a> anchor "sqlite3_table_column_metadata" already defined
  line 1224 column 13 - Warning: <a> anchor "sqlite3_thread_cleanup" already defined
  line 1240 column 13 - Warning: <a> anchor "sqlite3_total_changes" already defined
  line 1261 column 13 - Warning: <a> anchor "sqlite3_update_hook" already defined
  line 1288 column 13 - Warning: <a> anchor "sqlite3_user_data" already defined
  line 1328 column 5 - Warning: <a> anchor "SQLITE_ABORT" already defined
  line 1329 column 13 - Warning: <a> anchor "SQLITE_ABORT" already defined
  line 1402 column 5 - Warning: <a> anchor "SQLITE_ALTER_TABLE" already defined
  line 1403 column 13 - Warning: <a> anchor "SQLITE_ALTER_TABLE" already defined
  line 1461 column 5 - Warning: <a> anchor "SQLITE_ANY" already defined
  line 1462 column 13 - Warning: <a> anchor "SQLITE_ANY" already defined
  line 1480 column 5 - Warning: <a> anchor "SQLITE_BLOB" already defined
  line 1481 column 13 - Warning: <a> anchor "SQLITE_BLOB" already defined
  line 1509 column 5 - Warning: <a> anchor "SQLITE_DENY" already defined
  line 1510 column 13 - Warning: <a> anchor "SQLITE_DENY" already defined
  line 1534 column 5 - Warning: <a> anchor "SQLITE_IOERR_BLOCKED" already defined
  line 1535 column 13 - Warning: <a> anchor "SQLITE_IOERR_BLOCKED" already defined
  line 1571 column 5 - Warning: <a> anchor "SQLITE_STATIC" already defined
  line 1572 column 13 - Warning: <a> anchor "SQLITE_STATIC" already defined
  line 1591 column 5 - Warning: <a> anchor "SQLITE_VERSION" already defined
  line 1592 column 13 - Warning: <a> anchor "SQLITE_VERSION" already defined
  line 1619 column 5 - Warning: <a> anchor "sqlite3_aggregate_count" already defined
  line 1620 column 13 - Warning: <a> anchor "sqlite3_aggregate_count" already defined
  line 1644 column 5 - Warning: <a> anchor "sqlite3_bind_blob" already defined
  line 1645 column 13 - Warning: <a> anchor "sqlite3_bind_blob" already defined
  line 1713 column 5 - Warning: <a> anchor "sqlite3_collation_needed" already defined
  line 1714 column 13 - Warning: <a> anchor "sqlite3_collation_needed" already defined
  line 1756 column 5 - Warning: <a> anchor "sqlite3_column_blob" already defined
  line 1757 column 13 - Warning: <a> anchor "sqlite3_column_blob" already defined
  line 1803 column 1 - Warning: <table> lacks "summary" attribute
  line 1861 column 5 - Warning: <a> anchor "sqlite3_column_database_name" already defined
  line 1862 column 13 - Warning: <a> anchor "sqlite3_column_database_name" already defined
  line 1892 column 5 - Warning: <a> anchor "sqlite3_column_decltype" already defined
  line 1893 column 13 - Warning: <a> anchor "sqlite3_column_decltype" already defined
  line 1918 column 5 - Warning: <a> anchor "sqlite3_column_name" already defined
  line 1919 column 13 - Warning: <a> anchor "sqlite3_column_name" already defined
  line 1939 column 5 - Warning: <a> anchor "sqlite3_commit_hook" already defined
  line 1940 column 13 - Warning: <a> anchor "sqlite3_commit_hook" already defined
  line 1960 column 5 - Warning: <a> anchor "sqlite3_complete" already defined
  line 1961 column 13 - Warning: <a> anchor "sqlite3_complete" already defined
  line 1984 column 5 - Warning: <a> anchor "sqlite3_create_collation" already defined
  line 1985 column 13 - Warning: <a> anchor "sqlite3_create_collation" already defined
  line 2041 column 5 - Warning: <a> anchor "sqlite3_create_function" already defined
  line 2042 column 13 - Warning: <a> anchor "sqlite3_create_function" already defined
  line 2111 column 5 - Warning: <a> anchor "sqlite3_errcode" already defined
  line 2112 column 13 - Warning: <a> anchor "sqlite3_errcode" already defined
  line 2142 column 5 - Warning: <a> anchor "sqlite3_free" already defined
  line 2143 column 13 - Warning: <a> anchor "sqlite3_free" already defined
  line 2159 column 5 - Warning: <a> anchor "sqlite3_free_table" already defined
  line 2160 column 13 - Warning: <a> anchor "sqlite3_free_table" already defined
  line 2206 column 5 - Warning: <a> anchor "sqlite3_get_auxdata" already defined
  line 2207 column 13 - Warning: <a> anchor "sqlite3_get_auxdata" already defined
  line 2237 column 5 - Warning: <a> anchor "sqlite3_libversion" already defined
  line 2238 column 13 - Warning: <a> anchor "sqlite3_libversion" already defined
  line 2260 column 5 - Warning: <a> anchor "sqlite3_mprintf" already defined
  line 2261 column 13 - Warning: <a> anchor "sqlite3_mprintf" already defined
  line 2323 column 5 - Warning: <a> anchor "sqlite3_open" already defined
  line 2324 column 13 - Warning: <a> anchor "sqlite3_open" already defined
  line 2356 column 5 - Warning: <a> anchor "sqlite3_prepare" already defined
  line 2357 column 13 - Warning: <a> anchor "sqlite3_prepare" already defined
  line 2439 column 5 - Warning: <a> anchor "sqlite3_profile" already defined
  line 2440 column 13 - Warning: <a> anchor "sqlite3_profile" already defined
  line 2471 column 5 - Warning: <a> anchor "sqlite3_result_blob" already defined
  line 2472 column 13 - Warning: <a> anchor "sqlite3_result_blob" already defined
  line 2518 column 5 - Warning: <a> anchor "sqlite3_value_blob" already defined
  line 2519 column 13 - Warning: <a> anchor "sqlite3_value_blob" already defined
  line 389 column 120 - Warning: trimming empty <p>
  line 398 column 6 - Warning: trimming empty <p>
  line 542 column 20 - Warning: trimming empty <p>
  line 544 column 7 - Warning: trimming empty <p>
  line 835 column 62 - Warning: trimming empty <p>
  line 1196 column 29 - Warning: trimming empty <p>
  line 1203 column 7 - Warning: trimming empty <p>
  line 1209 column 13 - Warning: trimming empty <p>
  line 1215 column 7 - Warning: trimming empty <p>
  line 1368 column 89 - Warning: trimming empty <p>
  line 1495 column 65 - Warning: trimming empty <p>
  line 1501 column 6 - Warning: trimming empty <p>
  line 1660 column 11 - Warning: trimming empty <p>
  line 1666 column 6 - Warning: trimming empty <p>
  line 1802 column 17 - Warning: trimming empty <p>
  line 1805 column 49 - Warning: trimming empty <p>
  line 1805 column 53 - Warning: trimming empty <p>
  line 1822 column 14 - Warning: trimming empty <p>
  line 1830 column 28 - Warning: trimming empty <p>
  line 1833 column 45 - Warning: trimming empty <p>
  line 1833 column 45 - Warning: trimming empty <li>
  line 1835 column 24 - Warning: trimming empty <p>
  line 1835 column 24 - Warning: trimming empty <li>
  line 1838 column 6 - Warning: trimming empty <p>
  line 1843 column 34 - Warning: trimming empty <p>
  line 1847 column 6 - Warning: trimming empty <p>
  line 2177 column 88 - Warning: trimming empty <p>
  line 2183 column 7 - Warning: trimming empty <p>
  line 2184 column 46 - Warning: trimming empty <p>
  line 2193 column 7 - Warning: trimming empty <p>
  line 2296 column 85 - Warning: trimming empty <p>
  line 2298 column 20 - Warning: trimming empty <p>
  line 2298 column 84 - Warning: trimming empty <p>
  line 2302 column 20 - Warning: trimming empty <p>
  line 2303 column 52 - Warning: trimming empty <p>
  line 2305 column 20 - Warning: trimming empty <p>
  line 2306 column 33 - Warning: trimming empty <p>
  line 2308 column 20 - Warning: trimming empty <p>
  line 2313 column 72 - Warning: trimming empty <p>
  line 2317 column 20 - Warning: trimming empty <p>
  line 2413 column 38 - Warning: trimming empty <p>
  line 2423 column 10 - Warning: trimming empty <p>
  line 2423 column 10 - Warning: trimming empty <li>
  line 2435 column 1 - Warning: trimming empty <p>
 
2419 build active 2007 Jun anonymous   2007 Jun   1 2 'CP_UTF8' : undeclared identifier when trying to build the dll edit
I am using Microsoft Visual C++ Development System (an old version - 4.0) on Windows XP, and trying to build the dll and associated lib file. The compiler chokes on line 15146 (and following) with an undeclared identifier 'CP_UTF8'.

I'm a newbie, so please be gentle if this is a stupid question. Thanks!

 
2414 code active 2007 Jun anonymous   2007 Jun   1 1 Unable t edit
I designed a tool in C# using the Sqlite.Net.dll and sqlite3.dll v 3.2.5. The call to sqlite3_step after the sqlite3_prepare function, causes a huge delay to return (~7MINS), sometimes it doesn't return at all. This happens when I'm using v3.2.5 but when I replace it with v 3.3.7, everything works normal. I have tried so many combination of things to get it to work on v3.2.5 (this is the version of the libraries the hardware uses), which includes setting the PRAGMA legacy_file_format to 1, but after every save it reverts back to 0. i will appreciate if I can get a response with any suggestions. Thanks in advance.
2007-Jun-13 15:58:00 by anonymous:
Please post the schema and the SELECT command that is slow under 3.2.5 and is fast under 3.3.7 so it can be reproduced using the sqlite3 commandline shell.
 
2409 code active 2007 Jun anonymous   2007 Jun drh 1 1 Database malformed with SQLite3.3.17 on WindowsXP edit
I encountered a problem with SQLite3.3.17 on Windows XP. Under certain situation, database file got seriously corrupted.

SQLite version: 3.3.17 Windows Binary Platform:Windows XP SP2(Japanese) Code wrtten in: Visual C++ 6.0

Here are the procedures to reproduce the problem:

1) Run a program SQLiteCrush.exe. This program updates 'test.db' repeatedly. Insert data to work table, copy them into items table, then delete records from work.

2) Open 'test.db' from sqlite3.exe.

3) Do '.read check.sql' repeatedly. check.sql is made from many lines of 'pragma integrity_check;'.

4) Keep doing 1 -3 for several minuites, and 'pragma integrity_check' starts to report something like "rowid 91667 missing from index sqlite_autoindex_link_1".

So far, I didn't see the database corrupted with SQLite 3.3.7. Also, without 3), the database was not corrupted. Instead of 'pragma integrity_check', issueing many select statements also make it currupted.

2007-Jun-12 02:51:26 by anonymous:
I did more tests to make it clear from which version it happens.

With 3.3.8, I couldn't reproduce the problem. With 3.3.9, I can reproduce the problem.

It seems there's some change between these two that causes the problem...


2007-Jun-12 03:06:47 by anonymous:
Can you try it against the latest version in CVS, or 3.3.17? A lot of code has changed since 3.3.9.


2007-Jun-12 05:00:50 by anonymous:
I Already tried 3.3.17. It happenes.

Where can I get the latest CVS precompiled binary for windows platform ?

tamagawa


2007-Jun-13 14:25:47 by drh:
The problem was introduced in SQLite version 3.3.0, specifically check-in [2848] . There are related problems that go back even further in time, we believe. The root of the problem is a logic error in my design of the pager layer. We are working on a fix now, as well as a set of test cases that will ensure that similar errors do not reappear in the future. I will also soon publish instructions on how to work around the problem in effected versions of SQLite.

The problem can be easily reproduced by running the script below using the "testfixture" program that we use for testing SQLite.

# Prepare the database:
#
file delete -force test.db test.db-journal
sqlite3 db test.db
db eval {
  CREATE TABLE t1 (
    x TEXT UNIQUE NOT NULL,
    y BLOB
  );
}

# Open a second connection to the database that will be
# used to lock the database file.
#
set DB2 [sqlite3 db2 test.db]
if {$DB2==""} {
  set DB2 [sqlite3_connection_pointer db2]
}

# A small cache will cause an early cache spill.
#
db eval {PRAGMA cache_size=10}

# Acquire read lock on the database file using the second connection.
#
set STMT [sqlite3_prepare $DB2 {SELECT rowid FROM sqlite_master} -1 TAIL]
sqlite3_step $STMT

# Insert a short record into the index (10 bytes) and a large record
# into the table (15K).  The index record goes in Ok, but during the
# insert into the table, SQLite attempts to upgrade to an EXCLUSIVE
# lock to do a cache flush. When this happens, the cache is left in
# an inconsistent state.
#
set zShort [string repeat 0123456789 1]
set zLong  [string repeat 0123456789 1500]
db eval {BEGIN}
set rc [catch { db eval {INSERT INTO t1 VALUES($zShort, $zLong)} } msg]
puts "rc=$rc msg=$msg"
sqlite3_finalize $STMT
db eval {COMMIT}
db close
sqlite3 db test.db
puts [db eval {PRAGMA integrity_check}]
 
2413 code active 2007 Jun anonymous   2007 Jun drh 1 1 1 bug and 2 suggestions in lemon edit
Hello, ...
Sorry for my english :-) and if i post this with Severity/Priority error.
I found some not serious bug and have some suggetions.
=============================================================================
BUG FIX:
lemon.c for Win32. It not found lempar.c - backslash-bug.
function:
PRIVATE char *pathsearch(argv0,name,modemask);

PATCH:

---- CUT --------------------------------------------------------------------

--- C:/lemon.c Wed Jun 13 15:02:37 2007
+++ D:/Den/Lemon/lemon.c Wed Jun 13 16:25:22 2007
@@ -2911,7 +2911,11 @@
c = *cp;
*cp = 0;
path = (char *)malloc( strlen(argv0) + strlen(name) + 2 );
- if( path ) sprintf(path,"%s/%s",argv0,name);
+ #ifdef __WIN32__
+ if( path ) sprintf(path,"%s\\%s",argv0,name);
+ #else
+ if( path ) sprintf(path,"%s/%s",argv0,name);
+ #endif
*cp = c;
}else{
extern char *getenv();
@@ -2920,11 +2924,19 @@
path = (char *)malloc( strlen(pathlist)+strlen(name)+2 );
if( path!=0 ){
while( *pathlist ){
- cp = strchr(pathlist,':');
+ #ifdef __WIN32__
+ cp = strchr(pathlist,';');
+ #else
+ cp = strchr(pathlist,':');
+ #endif
if( cp==0 ) cp = &pathlist[strlen(pathlist)];
c = *cp;
*cp = 0;
- sprintf(path,"%s/%s",pathlist,name);
+ #ifdef __WIN32__
+ sprintf(path,"%s\\%s",pathlist,name);
+ #else
+ sprintf(path,"%s/%s",pathlist,name);
+ #endif
*cp = c;
if( c==0 ) pathlist = "";
else pathlist = &cp[1];

---- CUT --------------------------------------------------------------------

=============================================================================
SUGGESTION 1:

Why we allocate parser with mallocProc parameter of ParseAlloc function
and free with freeProc of ParseFree function?

We do this because we want what parser is user-allocatable
with USER-DEFINED-MEMORY-ALOCATION-WAY but not with "malloc"/"free" from stdlib... am i right?

If so... why we still allocate memory for parser stack with "realloc" function?

It's bad for solutions where is no stdlib.

My suggestion is

FIRST WAY:

To add to yyParser struct 3 variables like

void *mem_alloc_fn;
void *mem_realloc_fn;
void *mem_free_fn;

and add 3 directives like

%memory_alloc
%memory_realloc
%memory_free

and if it declared - use it for allocating/free/reallocating memory in parser.

and
- void ParseAlloc(void *(*mallocProc)(size_t));
will now as void *ParseAlloc();
- void ParseFree(void *pParser, void (*freeProc)(void
));
will now as void ParseFree(void *pParser);

OR SECOND WAY (very simple):

To add to yyParser struct 1 variable like
void *mem_realloc_fn;

- void *ParseAlloc(void *(*mallocProc)(size_t));
will now as void *ParseAlloc(void *(*mallocProc)(size_t), void *(*reallocProc)(void *, size_t));

store reallocProc in mem_realloc_fn in yyParser

and in yyGrowStack something like this:
... yyGrowStack (...)
{
....
if(pParser->mem_realloc_fn != NULL)
{
pNew = pParser->mem_realloc_fn(p->yystack, newSize*sizeof(pNew[0]));
}
else
{
pNew = realloc(p->yystack, newSize*sizeof(pNew[0]));
}
....
}

and use it for reallocating memory in parser.

In this ways - memory allocating in parser is under FULL user control.

=============================================================================
SUGGESTION 2:

I build lemon with VC 8.0 with option /Wp64 (Detect 64-Bit Portability Issues)
and have warnings. Type int, size_t, pointer and unsigned long have diferent size on x32 and x64 platforms.

Can you fix type difference, please?

Only you can choice better way for this - type conversion OR change type of 'warning' variables.

WARNINGS:

d:\den\lemon\lemon.c(1331) : warning C4267: '=' : conversion from 'size_t' to 'int', possible loss of data
d:\den\lemon\lemon.c(1337) : warning C4267: '=' : conversion from 'size_t' to 'int', possible loss of data
d:\den\lemon\lemon.c(1455) : warning C4113: 'int (__cdecl )()' differs in parameter lists from 'int (__cdecl *)(const void *,const void *)'
d:\den\lemon\lemon.c(1578) : warning C4311: 'type cast' : pointer truncation from 'char *' to 'unsigned long'
d:\den\lemon\lemon.c(1578) : warning C4312: 'type cast' : conversion from 'unsigned long' to 'char **' of greater size
d:\den\lemon\lemon.c(1581) : warning C4311: 'type cast' : pointer truncation from 'char *' to 'unsigned long'
d:\den\lemon\lemon.c(1581) : warning C4312: 'type cast' : conversion from 'unsigned long' to 'char **' of greater size
d:\den\lemon\lemon.c(1586) : warning C4311: 'type cast' : pointer truncation from 'char *' to 'unsigned long'
d:\den\lemon\lemon.c(1586) : warning C4312: 'type cast' : conversion from 'unsigned long' to 'char **' of greater size
d:\den\lemon\lemon.c(1588) : warning C4311: 'type cast' : pointer truncation from 'char *' to 'unsigned long'
d:\den\lemon\lemon.c(1588) : warning C4312: 'type cast' : conversion from 'unsigned long' to 'char **' of greater size
d:\den\lemon\lemon.c(1590) : warning C4311: 'type cast' : pointer truncation from 'char *' to 'unsigned long'
d:\den\lemon\lemon.c(1590) : warning C4312: 'type cast' : conversion from 'unsigned long' to 'char **' of greater size
d:\den\lemon\lemon.c(1592) : warning C4311: 'type cast' : pointer truncation from 'char *' to 'unsigned long'
d:\den\lemon\lemon.c(1592) : warning C4312: 'type cast' : conversion from 'unsigned long' to 'char **' of greater size
d:\den\lemon\lemon.c(1595) : warning C4311: 'type cast' : pointer truncation from 'char *' to 'unsigned long'
d:\den\lemon\lemon.c(1595) : warning C4312: 'type cast' : conversion from 'unsigned long' to 'char **' of greater size
d:\den\lemon\lemon.c(1596) : warning C4311: 'type cast' : pointer truncation from 'char *' to 'unsigned long'
d:\den\lemon\lemon.c(1596) : warning C4312: 'type cast' : conversion from 'unsigned long' to 'char **' of greater size
d:\den\lemon\lemon.c(1624) : warning C4311: 'type cast' : pointer truncation from 'char **' to 'unsigned long'
d:\den\lemon\lemon.c(1624) : warning C4311: 'type cast' : pointer truncation from 'char *' to 'unsigned long'
d:\den\lemon\lemon.c(1628) : warning C4311: 'type cast' : pointer truncation from 'char *' to 'unsigned long'
d:\den\lemon\lemon.c(1628) : warning C4312: 'type cast' : conversion from 'unsigned long' to 'char **' of greater size
d:\den\lemon\lemon.c(1629) : warning C4311: 'type cast' : pointer truncation from 'char *' to 'unsigned long'
d:\den\lemon\lemon.c(1629) : warning C4312: 'type cast' : conversion from 'unsigned long' to 'char **' of greater size
d:\den\lemon\lemon.c(1658) : warning C4267: '=' : conversion from 'size_t' to 'int', possible loss of data
d:\den\lemon\lemon.c(1661) : warning C4267: '+=' : conversion from 'size_t' to 'int', possible loss of data
d:\den\lemon\lemon.c(1774) : warning C4311: 'type cast' : pointer truncation from 'char *' to 'unsigned long'
d:\den\lemon\lemon.c(1774) : warning C4311: 'type cast' : pointer truncation from 'char *' to 'unsigned long'
d:\den\lemon\lemon.c(1785) : warning C4311: 'type cast' : pointer truncation from 'char *' to 'unsigned long'
d:\den\lemon\lemon.c(1785) : warning C4311: 'type cast' : pointer truncation from 'char *' to 'unsigned long'
d:\den\lemon\lemon.c(1883) : warning C4267: '=' : conversion from 'size_t' to 'int', possible loss of data
d:\den\lemon\lemon.c(2722) : warning C4267: '=' : conversion from 'size_t' to 'int', possible loss of data
d:\den\lemon\lemon.c(3171) : warning C4267: '=' : conversion from 'size_t' to 'int', possible loss of data
d:\den\lemon\lemon.c(3173) : warning C4018: '>=' : signed/unsigned mismatch
d:\den\lemon\lemon.c(3184) : warning C4267: '+=' : conversion from 'size_t' to 'int', possible loss of data
d:\den\lemon\lemon.c(3340) : warning C4267: '=' : conversion from 'size_t' to 'int', possible loss of data
d:\den\lemon\lemon.c(3346) : warning C4267: '=' : conversion from 'size_t' to 'int', possible loss of data
d:\den\lemon\lemon.c(3542) : warning C4267: '=' : conversion from 'size_t' to 'int', possible loss of data

Ups ... drh, sorry - title change.
 
2411 new active 2007 Jun anonymous   2007 Jun   5 3 fts2 RFE - Separate index and storage edit
It is my understanding of the current implementations of the fts modules that you save your text into the chosen table.column and in the background an index is generated for use by a special search operator. IMHO this puts two distinct concepts together, namely the storage of the text to be indexed, and its actual indexing for full-text-search.

What I am asking for here is the ability to create an fts-index without having the system store the text itself. I.e. putting the text into the table.column extends the index, but will not save the text.

This allows several things not possible with the current implementations:

  1. Storage of the text outside of the database (fts-index joined to path names).
  2. Compressed storage of text in the database (fts-index joined to separate blob table).

While I currently have no real idea yet about usecases for the first item I do see the following possible applications for the last item:

  1. SCM systems which wish to allow fts over all versions of a file, without giving up delta-compression between revisions.
  2. Help files which allow fts despite being space-efficient due to compressed storage of the help pages (zlib).

In both cases the current implementations of fts would force the applications to choose between either space efficiency, or searchability.

2007-Jun-13 07:23:58 by anonymous:
This mailing list thread lists additional usage scenarios and arguments in favor of separating FTS index from text storage. It also gives some DB size savings statistics:
 
2408 code active 2007 Jun anonymous   2007 Jun   2 1 BLOBs not output correctly in .mode insert (shell.c - isnumber) edit
The method
static int isNumber(const char *z, int *realnum)

from shell.c is wrong.
Steps to reproduce:
1. Get the file www.smatei.3x.ro/project1.zip
2. extract project1.db from the zip file
3. execute
sqlite3 project1.db
sqlite>.mode insert
sqlite> select ID, Name, Color, Active, Priority, PrioritySource, IndexOrder, Language, 0 from Keywords;
INSERT INTO Keywords VALUES(42,#####,0,1,0,0,42,'',0);
INSERT INTO Keywords VALUES(41,'######',0,1,0,0,43,'',0);

If you look at the 42 item, the string next to 42 is not enclosed by the string delimiter '.
This is because the method isnumber returns that the string is number. It is not a number, it is a string in Hebrew (I inserted ### instead of the real strings).
A fix for this might be to set the first parameter unsigned char
static int isNumber(unsigned const char *z, int *realnum)
but I am not sure, because I haven't written C code for a long time.
If you have any more questions, please ask.
Best Regards,
Stefan

2007-Jun-11 14:46:03 by drh:
Please show me what you get from the following query:

    SELECT ID, quote(cast(Name AS BLOB)) FROM Keywords
     WHERE ID IN (41,42);

I need this information in order to track down the problem.


2007-Jun-11 14:50:33 by anonymous:
The output is

41|X'D791D7A8D799D799D7A7D793D790D7A0D7A1'
42|X'D7A1D798D7A4D7A1'


2007-Jun-11 16:29:46 by drh:
When I do this:

  CREATE TABLE t1(x);
  INSERT INTO t1 VALUES(cast(X'D791D7A8D799D799D7A7D793D790D7A0D7A1' as text));
  INSERT INTO t1 VALUES(cast(X'D7A1D798D7A4D7A1' AS text));
  .mode insert xyz
  SELECT * FROM t1;

I see the Hebrew characters, properly quoted. Can you suggest another way to reproduce the problem?


2007-Jun-11 16:34:55 by anonymous:

  SQLite version 3.3.17
  Enter ".help" for instructions
  sqlite> create table t1(b blob);
  sqlite> insert into t1 values(X'0102030405060708090a0dABCD');
  sqlite> .dump
  BEGIN TRANSACTION;
  CREATE TABLE t1(b blob);
  INSERT INTO "t1" VALUES(X'0102030405060708090A0DABCD');
  COMMIT;
  sqlite> .mode insert
  sqlite> select * from t1;
  INSERT INTO table VALUES('
  &#171;&#205;');
  sqlite>


2007-Jun-11 20:07:09 by anonymous:
If I recall correctly, sqlite3.exe assumes all I/O to and from the console is UTF-8. Windows consoles are either MBCS or UNICODE depending on the build settings. Sqlite3.exe was not compiled as UNICODE so anything inserted into a sqlite3 database will be inserted as MBCS. So if you insert into a database using a 3rd party application that does proper UTF-8, and then query it from the command-line, it will look wrong. Likewise, anything you insert from the command-line will look wrong when queried from an application that uses UTF-8.


2007-Jun-11 20:42:18 by anonymous:
.mode insert treats BLOBs as strings, which is a problem since it stops outputting at the first nil character.

  CREATE TABLE t1(a blob);
  INSERT INTO t1 VALUES(X'00000008090A0D0D0A00');
  .mode insert whatever
  select * from t1;


2007-Jun-12 07:39:12 by anonymous:
Hi,

My problem is not the way the string is displayed in the console. The application that reads the database reads it correctly. The problem is when I try to dump a certain table in a file using the following sequence:

.output "file.txt"
.mode insert whatever
select f1, f2, f3 from t1;

The result is a file that some Hebrew strings are not enclosed in ' (apostrophe). After debugging the code, I saw that the method isnumber does not return 0 for that string. It considers all the characters digits.

This problem occurs only on Windows. We tried this on Mac and Linux, and it worked fine.

Best Regards,
Stefan

 
2401 warn active 2007 Jun anonymous   2007 Jun   1 1 I can't send anything to the contrib page edit
I am trying to send a .zip file that is almost 3 meg in size to the contrib page.

I have a login and password.

When I try to upload the file I get a server error.

Is my file too large?

Thanks

Tony Scarpelli Computer Systems Specialist Maine Medical Center Portland Maine 207-662-4987

 
2398 code active 2007 Jun anonymous   2007 Jun   2 3 systemcalls should be restarted when they return EINTR edit
Introduction: I'm a developer for Alcatel-Lucent, where I program softswitches. I'm a bit of a code-correctness nazi, because I've programmed several systems that have to be very reliable (5 figures), or I've corrected them if they didn't. I was browsing thru the source code of SQLite, to become familiar with it, and to see how reliable it was. Not that we currently use it in a product, but you may never known.

The very first thing I noticed was that none of the systemcalls in os_unix.c was checked for EINTR. From experience, I know that a systemcall can be interrupted while inside the kernel, and return EINTR in errno. The correct reaction is to repeat the system call, just as if nothing happened. This is not an error, just a temporary situation which will be fixed the next time you call it again. One reason is that you call was interrupted by a higher priority interrupt (incoming IO for instance), or to avoid a deadlock in the kernel (returning your syscall will release any locks you might have). It's pretty rare in most usage, but I've seen it lots of times when testing a server under load.

Example of the solution for seekAndWrite (this only shows write, but pwrite is similar) :

   got = write(id->h, pBuf, cnt);

should be :

   do {
      got = write(id->h, pBuf, cnt);
   } while ( (got < 0) && (errno == EINTR) );

This has to be done for every systemcall that mentions it on its manpage (but please note, not every OS is the same). This includes many calls where everyone assumes that the call is safe to use, for instance close(). Almost nobody seems to realize that a close can fail, and often there's not even a check for the return code. But I can assure you, it can fail.

2007-Jun-05 13:39:08 by anonymous:
I prefer the current SQLite behavior NOT to retry in the event of EINTR. I've seen too many UNIX OSes with bugs related to EINTR for system calls within threads over the years, including the most popular ones. Most of these OS bugs have since been fixed, but it's still not worth the trouble in case you come across an unpatched platform. Getting an infinite loop of EINTR errnos in a tight loop due to an OS bug is no fun in a production application.

Technically, POSIX requires that you only read errno if -1 is returned by the system call. QNX is picky about this sort of thing.


2007-Jun-05 17:02:55 by anonymous:
Read my introduction again : I help make systems reliable in my daytime job. Ignoring a read- or write-error is not how you achieve 99.999% availability, especially not when it's easily correctable (a disk-full message or a I/O error due to a corrupt harddisk is another matter ofcourse).

>Technically, POSIX requires that you only read errno if -1 is returned by the system call. QNX is picky about this sort of thing.

well yes, that's what I mean.


2007-Jun-05 17:20:07 by anonymous:
All of SQLite's operations can also be retried if they fail.

SQLite has an extensive test suite for disk failures, malloc failures and many other types of failures that you've likely never considered. I tend to trust the judgement of the authors of SQLite more than an anonymous guy puffing his chest out in this ticket tracking system.


2007-Jun-05 22:24:09 by anonymous:
hi. instead of blaming in this ticket, the ticket author could place a patch like:
got = write(id->h, pBuf, cnt);
to do things like:
#ifdef SQLITE3_RETRY_EINTR_SYSCALLS
   do {
#endif // SQLITE3_RETRY_EINTR_SYSCALLS
      got = write(id->h, pBuf, cnt);
#ifdef SQLITE3_RETRY_EINTR_SYSCALLS
   } while ( (got < 0) && (errno == EINTR) );
#endif // SQLITE3_RETRY_EINTR_SYSCALLS
by leaving SQLITE3_RETRY_EINTR_SYSCALLS undefined, this should not affect the current code behaviour and should address the issues ticket author says and he can compile with those 'paranoic' checks. []'s


2007-Jun-05 22:25:06 by anonymous:
I did pass my name (visible for the developers), but I'm not logged in. That why it's shown as anonymous (just as you are).

Note that my co-workers have already decided it would be seen up as a negative point for this DBMS. At least for us, because it's a requirement for us to survive these errors. So if we ever are going to use SQLite, we're definitely going to use those changes.

Hey, I'm a code-review nazi, don't blame me. It's just my job :-)


2007-Jun-06 00:58:16 by anonymous:
I think that the retrying would destroy the function expected of sqlite3_interrupt.

Is there a problem in the following methods?

  • Re-execute the SQLite API.
  • Mask the signals while calling the API of SQLite.
  • Mask the signals in the threads that call the API of SQLite.


2007-Jun-06 01:04:47 by anonymous:
As long as #ifdef SQLITE3_RETRY_EINTR_SYSCALLS is not enabled by default, knock yourselves out.


2007-Jun-06 06:38:51 by anonymous:
sqlite3_interrupt() knows nothing about UNIX signals. It just sets a flag that sqlite's VDBE checks from time to time to see if it should gracefully abort an SQL operation.

shell.c happens to install an SIGINT signal handler that happens to call sqlite3_interrupt(), but that's unrelated. sqlite3_interrupt() could be called via any other means.


2007-Jun-06 10:47:27 by anonymous:
See also the SA_RESTART flag for sigaction(), which automatically will restart the systemcall instead of returning EINTR.

Unfortunately not available everywhere, and not very consistent either : see Linus reply at http://lkml.org/lkml/2005/7/23/119 . Applications should still be able to handle EINTR returns gracefully. They're not errors per se (that would be EIO or similar), you can safely restart the system call. I haven't seen endless loops yet, but it's true that on some OS you might want this solution. But you still have to deal with the error (which might cause a rollback in SQLite).


2007-Jun-06 14:51:14 by anonymous:
I would be in favor of optionally compiling SQLite with a #define SQLITE_USE_SA_RESTART or #define SQLITE_RETRY_ON_EINTR, where they are mutually exclusive and disabled by default. SQLITE_RETRY_ON_EINTR should have a maximum retry limit on buggy OSes, though.
 
2396 code active 2007 Jun anonymous   2007 Jun   1 3 -column output truncates characters from varchar field. edit
  CREATE TABLE unix (unix_id integer primary key, ip_address varchar(16) unique, status enum, updated timestamp);

  INSERT INTO unix VALUES (NULL,'172.26.242.92','in use',timestamp('now'));
  INSERT INTO unix VALUES (NULL,'172.26.242.129','in use',timestamp('now'));
  INSERT INTO unix VALUES (NULL,'172.26.242.131','in use',timestamp('now'));
  INSERT INTO unix VALUES (NULL,'172.26.242.132','in use',timestamp('now'));
  INSERT INTO unix VALUES (NULL,'172.26.242.136','in use',timestamp('now'));
  INSERT INTO unix VALUES (NULL,'172.26.242.213','in use',timestamp('now'));
  INSERT INTO unix VALUES (NULL,'10.193.33.7','in use',timestamp('now'));
  INSERT INTO unix VALUES (NULL,'10.193.32.239','in use',timestamp('now'));

Now: sqlite3 -column test.db "SELECT ip_address, status, updated FROM unix";

  172.26.242.92  in use      2007-05-31 22:11:39
  172.26.242.12  in use      2007-05-31 22:11:47
  172.26.242.13  in use      2007-05-31 22:11:51
  172.26.242.13  in use      2007-05-31 22:11:59
  172.26.242.13  in use      2007-05-31 22:12:04
  172.26.242.21  in use      2007-05-31 22:12:11
  10.193.33.7    in use      2007-05-31 23:27:09
  10.193.32.239  in use      2007-05-31 23:27:17

The output of the varchar column is always truncated regardless of which column it is placed in. Even if it's the only column.

  sqlite3 -column test.db "SELECT ip_address FROM unix";

  172.26.242.92
  172.26.242.12
  172.26.242.13
  172.26.242.13
  172.26.242.13
  172.26.242.21
  10.193.33.7
  10.193.32.239
2007-Jun-01 18:54:42 by anonymous:
Smaller test case:

  sqlite> CREATE TABLE t9(a);
  sqlite> INSERT INTO "t9" VALUES('a234567890');
  sqlite> INSERT INTO "t9" VALUES('a23456789012345');
  sqlite> select * from t9;
  a234567890
  a23456789012345
  sqlite> .mode column
  sqlite> select * from t9;
  a234567890
  a234567890
  sqlite> select * from t9 order by a desc;
  a23456789012345
  a234567890

.mode column seems to be remembering the first row's column lengths and using them as the maximum column lengths for subsequent rows.


2007-Jun-01 19:11:11 by anonymous:
.mode column could only work correctly if the outputting of the rows took place only after the maximum column width of each column for each row was determined in advance. This would require buffering alls rows in memory before any data rows would be output.

I recommend to remove MODE_Column functionality from shell.c completely, as it never worked correctly.


2007-Jun-05 14:54:30 by anonymous:
Column mode works fine for most applications. It bases it column widths on the width of the title or a minimum width of 10. If this is not suitable, you can set the column widths manually using the .width command in the shell. If you set your IP address column width to 16 or more everything will be fine.
 
154 code active 2002 Sep drh Pager 2007 Jun drh 3 3 Prohibit links on database files. edit
If a database file is aliased using either hard or symbolic links, it can happen that an aborted transaction will not roll back correctly.

Consider this scenario. The database file is named both a.db and a.db. Application one opens a.db and starts to make a change. This creates a journal file a.db-journal. But application one crashes without completing the transaction. Later, application two attempts to open the database as b.db. App two looks for a journal file to rollback, but it thinks the journal should be named b.db-journal. So it fails to see the a.db-journal that app one left and fails to rollback the transaction.

The only way I can think of to prevent this kind of thing it to refuse to open any database file that contains two or more hard links and to refuse to open a file through a symbolic link.

2004-Mar-16 20:46:17 by anonymous:
What if the journal file name wasn't based on the database name, but instead was based on the starting inode of the database file? For instance, "journal-10293" would be used if the starting inode for the associated database file was 10293.


2004-Mar-20 17:17:41 by anonymous:
Using Inode-numbers to solve this problem is a dangerous proposition, as disk defragmenters can alter the inode the db starts at in between a crash and a subsequent roll-back attempt.


2007-Jun-05 03:57:17 by anonymous:
On unix, you can use ftok() to solve this problem. It guarantees to return the same key for all paths to the same file, including symbolic and hard links.

I have to experience programming in Windows, but have no doubt a similar function call exists in that API.

 
2397 build active 2007 Jun anonymous   2007 Jun   2 4 make install failed on Cygwin edit
  On WinXP Cygwin platform, `make install' failed.
  There are at least two problems.  I could not resolve them.

    (1) Makefile defect.
    (2) Tcl on Cygwin is not compatible with other platforms.

  (1) can be fixed as follows:

     -install:        sqlite3$ libsqlite3.la sqlite3.h ${HAVE_TCL:1=tcl_install}
     +install:        sqlite3$(TEXE) libsqlite3.la sqlite3.h ${HAVE_TCL:1=tcl_install}

  (2) can not be fixed.  There are at least two Tcl problems.

     (a) On Cygwin, an empty string can not be set to the
         environment variable.  So tclinstaller.tcl line 9:

            set env(DESTDIR) ""

         this statement has no effect, and Tcl failed at line 10.

     (b) On Cygwin, [info sharedlibextension] returns ".dll".
         But `make all' generates static library only, libsqlite.dll
         does not exist.

     As a workaround, commenting out HAVE_TCL in Makefile did the trick.
 
2393 code active 2007 May anonymous   2007 May   4 4 pragma cache_size not accurate when set to less than 10 edit
It seems that the smallest the cache_size can be is 10 pages, despite what can be set and is reported by the cache_size and default_cache_size pragmas:

  /*
  ** Change the maximum number of in-memory pages that are allowed.
  */
  void sqlite3PagerSetCachesize(Pager *pPager, int mxPage){
    if( mxPage>10 ){
      pPager->mxPage = mxPage;
    }else{
      pPager->mxPage = 10;
    }
  }

...

  static int sqlite3BtreeSetCacheSize(Btree *p, int mxPage){
    BtShared *pBt = p->pBt;
    sqlite3PagerSetCachesize(pBt->pPager, mxPage);
    return SQLITE_OK;
  }

...

    if( sqlite3StrICmp(zLeft,"cache_size")==0 ){
      if( sqlite3ReadSchema(pParse) ) goto pragma_out;
      if( !zRight ){
        returnSingleInt(pParse, "cache_size", pDb->pSchema->cache_size);
      }else{
        int size = atoi(zRight);
        if( size<0 ) size = -size;
        pDb->pSchema->cache_size = size;
        sqlite3BtreeSetCacheSize(pDb->pBt, pDb->pSchema->cache_size);

...

    if( sqlite3StrICmp(zLeft,"default_cache_size")==0 ){
      static const VdbeOpList getCacheSize[] = {
        { OP_ReadCookie,  0, 2,        0},  /* 0 */
        { OP_AbsValue,    0, 0,        0},
        { OP_Dup,         0, 0,        0},
        { OP_Integer,     0, 0,        0},
        { OP_Ne,          0, 6,        0},
        { OP_Integer,     0, 0,        0},  /* 5 */
        { OP_Callback,    1, 0,        0},
      };
      int addr;
      if( sqlite3ReadSchema(pParse) ) goto pragma_out;
      if( !zRight ){
        sqlite3VdbeSetNumCols(v, 1);
        sqlite3VdbeSetColName(v, 0, COLNAME_NAME, "cache_size", P3_STATIC);
        addr = sqlite3VdbeAddOpList(v, ArraySize(getCacheSize), getCacheSize);
        sqlite3VdbeChangeP1(v, addr, iDb);
        sqlite3VdbeChangeP1(v, addr+5, SQLITE_DEFAULT_CACHE_SIZE);
      }else{
        int size = atoi(zRight);
        if( size<0 ) size = -size;
        sqlite3BeginWriteOperation(pParse, 0, iDb);
        sqlite3VdbeAddOp(v, OP_Integer, size, 0);
        sqlite3VdbeAddOp(v, OP_ReadCookie, iDb, 2);
        addr = sqlite3VdbeAddOp(v, OP_Integer, 0, 0);
        sqlite3VdbeAddOp(v, OP_Ge, 0, addr+3);
        sqlite3VdbeAddOp(v, OP_Negative, 0, 0);
        sqlite3VdbeAddOp(v, OP_SetCookie, iDb, 2);
        pDb->pSchema->cache_size = size;
        sqlite3BtreeSetCacheSize(pDb->pBt, pDb->pSchema->cache_size);

The information reported is not accurate:

  SQLite version 3.3.17
  Enter ".help" for instructions
  sqlite> pragma cache_size;
  2000
  sqlite> pragma default_cache_size;
  2000
  sqlite> pragma default_cache_size=3;
  sqlite> pragma cache_size=7;
  sqlite> pragma default_cache_size;
  3                                    (should be 10)
  sqlite> pragma cache_size;
  7                                    (should be 10)
 
2388 code active 2007 May anonymous   2007 May   1 3 ORDER BY fails on compound select edit
The query below was known to be working as late as 3.3.14, but in 3.3.17 it errors with "ORDER BY term number 1 does not match any result column".

Debugging with GDB shows that sqlite3NameFromToken(&pE->token) doesn't return anything, and pE->token looks pretty empty, so possibly this could be a parsing issue?

The query is:

  SELECT 	docs.guid, docs.info, conversations.subject, conversations.date
  FROM	conversations
  JOIN 	docs	ON 	docs.guid = conversations.guid
  WHERE 	conversations.guid IN (
  	SELECT	docs.guid
  	FROM 	conversations
  	JOIN 	docs 	ON 	docs.guid = conversations.guid
  	WHERE 	conversations.date >= (SELECT date FROM conversations WHERE guid = ?6)
  		AND 	docs.collection = ?1
  		AND 	((?2 == 0) OR (conversations.sources & ?2) !=   0)
  		AND 	((conversations.sources & ?3) == 0)
  		AND 	test_info_flags(docs.info, ?7, ?8)
  	LIMIT 	(?5 + 1)
  	)
  UNION
  SELECT 	docs.guid, docs.info, conversations.subject,   conversations.date
  FROM 	conversations
  JOIN 	docs   	ON 	docs.guid = conversations.guid
  WHERE 	conversations.guid IN (
  	SELECT 	docs.guid
  	FROM 	conversations
  	JOIN 	docs 	ON docs.guid = conversations.guid
  	WHERE 	conversations.date < (SELECT date FROM conversations WHERE guid = ?6)
  		AND 	docs.collection = ?1
  		AND 	((?2 == 0) OR (conversations.sources & ?2) != 0)
  		AND 	((conversations.sources & ?3) == 0)
  		AND 	test_info_flags(docs.info, ?7, ?8)
  	LIMIT ?4
  	)
  ORDER BY conversations.date DESC, docs.guid DESC;
2007-May-27 16:27:23 by anonymous:
2 workarounds:

  ORDER BY 4 DESC, 1 DESC;

or

  SELECT docs.guid             as "guid",
         docs.info             as "info",
         conversations.subject as "subject",
         conversations.date    as "date"
  ...
  ORDER BY date DESC, guid DESC;

Smaller test case:

  create table x1(a);

  select x1.a from x1 union select x1.a from x1 order by x1.a;
  -- SQL error: ORDER BY term number 1 does not match any result column
  -- error in latest 3.3.17+ CVS, but works in SQLite 3.2.2, 3.3.13

  select x1.a from x1 union select x1.a from x1 order by a;
  -- SQL error: ORDER BY term number 1 does not match any result column
  -- has never worked in previous versions

  select x1.a from x1 union select x1.a from x1 order by 1;
  -- ok in all versions

  select x1.a as a from x1 union select x1.a from x1 order by a;
  -- ok in all versions


2007-May-27 16:46:19 by anonymous:
Does this query work on other popular databases?

  create table x1(a integer);
  select x1.a from x1 union select x1.a from x1 order by x1.a;

I'd expect "order by a" to work on most databases, but not "order by x1.a" because the table name would have been lost by the UNION.

 
2383 code active 2007 May anonymous   2007 May   3 3 Inconsistent conversion of BLOB revisited edit
BLOBs of function results are converted as UTF-8 even if the encoding is UTF-16.

  $ ./sqlite3
  SQLite version 3.3.17
  Enter ".help" for instructions
  sqlite> pragma encoding = 'UTF-16BE';
  sqlite> select quote(cast(cast('ab' as blob) as text));             -- OK
  'ab'
  sqlite> select quote(cast(a as text)) from (select X'00610062' a);  -- OK
  'ab'
  sqlite> select quote(cast(X'00610062' as text));                    -- fixed by [3975]
  'ab'
  sqlite> select quote(cast(substr(X'110061006222', 2, 4) as text));  -- NG
  ''
  sqlite> select quote(cast(substr(X'11616222', 2, 2) as text));
  'ab'
  sqlite> select quote(upper(substr(X'11616222', 2, 2)));
  'AB'
 
2381 doc active 2007 May anonymous   2007 May   3 3 'VACUUM' aborts an active transaction. edit
VACUUM aborts an active transaction. I'm not sure if this is the intended behavior. I didn't see any documentation to this effect.

  sqlite> .sc
  CREATE TABLE bar ( c1 integer primary key not null );

  sqlite> select max(oid) from bar;
  count(oid)
  ----------
  240

  sqlite> begin;
  sqlite> delete from bar;
  sqlite> select max(oid) from bar;
  mcount(oid)
  ----------
  0

  sqlite> vacuum;
  SQL error: cannot VACUUM from within a transaction

  sqlite> select max(oid) from bar;
  count(oid)
  ----------
  240
 
2363 code active 2007 May anonymous   2007 May   3 3 Couldn't build 3.3.17 on Cygwin/Vista edit
I didn't see anything quite resembling this on a cursory scan thru the buglist; pardon my goof if it's a duplicate.

When I tried to build on Cygwin on Vista, things got to the install phase, and then the Makefile tried to execute

  cc sqlite3.c -o sqlite3

which didn't appear to be a defined rule in the Makefile. This then failed because of a lack of a main program, and the install failed.

After a quick scan through the Makefile, I decided to change the target for the install rule to

    install:	sqlite3$(TEXE) libsqlite3.la sqlite3.h ${HAVE_TCL:1=tcl_install}

which seemed to remedy the problem.

Suggested fix: modify Makefile as shown above.

2007-May-23 14:29:20 by anonymous:
Another vote for this
 
2378 code active 2007 May anonymous   2007 May   2 2 Quoted fields come back corrupted, using GROUP BY edit
Description:

When executing a query, where field names are quoted, and using GROUP BY, the field names are returned with quotes around.

SQLite version:

  SQLite support => enabled
  PECL Module version => 2.0-dev $Id: sqlite.c,v 1.166.2.13.2.7 2007/03/06 02:17:13 stas Exp $
  SQLite Library => 2.8.17
  SQLite Encoding => iso8859

Reproduce code

  <?php
  $db = sqlite_open(":memory:");
  sqlite_query($db, '
    CREATE TABLE test (
      id INTEGER PRIMARY KEY
    )
  ');
  sqlite_query($db, "INSERT INTO test (id) VALUES (1)");
  $result = sqlite_query($db, 'SELECT "id" FROM "test" GROUP BY "id"');
  var_dump(sqlite_fetch_array($result, SQLITE_ASSOC));

Expected result

  array(1) {
    ["id"]=>
    string(1) "1"
  }

Actual result

  array(1) {
    [""id""]=>
    string(1) "1"
  }
2007-May-21 18:14:27 by anonymous:
Corrupted is probably not the right term, but the fields are returned with quotes around them in sqlite3 as well when group by is used on a quoted column:

  SQLite version 3.3.17
  Enter ".help" for instructions
  sqlite> .header on
  sqlite> create table t1(a);
  sqlite> insert into t1 values(1);
  sqlite> select "a" from t1;
  a
  1
  sqlite> select "a" from t1 order by 1;
  a
  1
  sqlite> select "a" from t1 group by 1;
  "a"
  1
  sqlite> select "a" from t1 group by 1 order by 1;
  "a"
  1
  sqlite> select "a" from t1 group by a;
  "a"
  1
 
1597 new active 2006 Jan anonymous Unknown 2007 May   1 3 wish: support DROP COLUMN edit
Please - add support for "drop column". For GIS user, as I, the Sqlite3 is a great, lighweight, powerfull tool for mananaging spatial data attributes. However, the lack of "drop column" seriously hampers it's usefullnes and still forces many GIS folks to use eg. Postgres, which is much more hard to handle than Sqlite - especially for newbies.

Plese note that Sqlite3 support was recently added to Grass (http://grass.itc.it), the most powerfull FOSS GIS. There were even discussions about making Sqlite3 a default database driver for Grass - easy to use, powerfull and fast. Please see: http://search.gmane.org/?query=sqlite+default&email=&group=gmane.comp.gis.grass.devel&sort=relevance&DEFAULTOP=and&%3E=Next&xP=sqlite.default.&xFILTERS=Gcomp.gis.grass.devel---A Extending the alter table commands, "drop column" most of all, could incline Grass devs to do so even more. There already were sevaral requests from Grass users regarding "drop column" in Grass sqlite driver:

http://article.gmane.org/gmane.comp.gis.grass.user/11141 http://thread.gmane.org/gmane.comp.gis.grass.devel/9454 http://grass.itc.it/pipermail/grass5/2006-January/020764.html

Any chances?

Maciek

2006-Jan-10 22:51:16 by drh:
http://www.sqlite.org/faq.html#q13


2006-Oct-24 21:37:43 by anonymous:
Hope to see this feature too...


2006-Oct-24 21:44:53 by anonymous:
I did this in a way that I create a parses that undestand the drop column and modify column sintax, but they create a temporary table, copy data between old and new, then create the new table and copy data back. not efficient, but works.


2007-May-20 08:26:54 by anonymous:
Please give consideration to drop column. One of SQLite's most potent features is the tiny footprint embedding. Requiring a temporary table to perform simple structure alterations effectively halves the maximum safe size of a table with regards to its primary storage context.

There's a reason it comes up as often as it does, despite the points in the FAQ. The temporary table hack is frequently a serious limiting factor. It seems likely that this wouldn't be a huge issue for the codebase, considering that other table structural alterations (add column, most notably) are apparently workable.

Please?


2007-May-20 12:56:56 by anonymous:
What would you like DROP COLUMN to do?

Copy the old table to a new table without the column in question, or just leave the old table in place without reclaiming space and simply ignore the old column?

What about constraints on that column? If they are violated should the DROP COLUMN be ignored?

 
1924 new active 2006 Aug anonymous Parser 2007 May   2 3 optimize queries on unions, constant folding edit
Please see the attached patch that optimizes SELECTs on a compound subquery, or VIEWs containing UNIONS.

  pragma temp_store=memory;
  CREATE TABLE n1(a integer primary key);
  INSERT INTO "n1" VALUES(1);
  INSERT INTO "n1" VALUES(2);
  INSERT INTO "n1" VALUES(3);
  INSERT INTO "n1" VALUES(4);
  INSERT INTO "n1" VALUES(5);
  INSERT INTO "n1" VALUES(6);
  INSERT INTO "n1" VALUES(7);
  INSERT INTO "n1" VALUES(8);
  INSERT INTO "n1" VALUES(9);
  INSERT INTO "n1" VALUES(10);
  CREATE VIEW vu as select v3.a a, v5.a-v2.a*v7.a b from n1 v1,n1 v2,n1 v3,n1 v4,n1 v5,n1 v6,n1
v7;
  CREATE VIEW v2 as select * from vu union all select 7, 8;

  select count(*), avg(b) from v2 where a<3;

The above query takes 58 seconds in sqlite 3.3.7, using 136M of temp_store. With the patch, it takes just 12 seconds and uses 26M of temp_store.

The patch also performs 32 bit integer constant folding:

  sqlite> explain select 1*2+3-4%5/2|128;
  0|Goto|0|4|
  1|Integer|131|0|
  2|Callback|1|0|
  3|Halt|0|0|
  4|Goto|0|1|
  5|Noop|0|0|
2006-Aug-17 13:03:31 by anonymous:
TK_REM (the '%' operator) is not handled correctly in the patch. It should follow the logic of TK_SLASH and check 'right' against zero.

  +        case TK_REM:    { v = left %  right; break; }

...

  +        case TK_SLASH:  {
  +          if (right) {
  +            v = left / right;
  +          } else {
  +            return;
  +          }
  +          break;
  +        }


2006-Aug-17 15:50:20 by anonymous:
These 2 cases can overflow a 32 bit value. The calculation should be done in 64 bit int math, and if the result can fit into 32-bits, then fold it, otherwise return (similar to TK_SLASH).

  +        case TK_PLUS:   { v = left +  right; break; }
  +        case TK_STAR:   { v = left *  right; break; }

Or just handle all cases in 64-bit math.


2006-Aug-18 10:11:53 by anonymous:
Wouldn't TK_MINUS also be able to overflow 32-bit, just in the opposite direction, so to speak ? Example: -2000000000 - 2000000000


2006-Aug-18 13:51:05 by anonymous:
should you make

+ i64 left; + i64 right; + i64 v;

instead of

+ int left; + int right; + int v;

to avoid int32 overflows.


2006-Aug-18 16:27:21 by anonymous:
The attachment sqlite337-union-and-constants-opt-v2.diff.txt addresses all reported issues and passes "make test" without any regressions.


2006-Aug-18 19:41:20 by anonymous:
You're not assuming that "right" could be a i64 value in this last patch...


2006-Aug-19 13:41:14 by anonymous:
right is a 32 bit value.

  /*
  ** If the expression p codes a constant integer that is small enough
  ** to fit in a 32-bit integer, return 1 and put the value of the integer
  ** in *pValue.  If the expression is not an integer or if it is too big
  ** to fit in a signed 32-bit integer, return 0 and leave *pValue unchanged.
  */
  int sqlite3ExprIsInteger(Expr *p, int *pValue){


2007-May-16 20:09:38 by anonymous:
Updated patch as of May 16, 2007 CVS:

http://marc.info/?l=sqlite-users&m=117934558505665&w=2

http://marc.info/?l=sqlite-users&m=117934558505665&q=p3


2007-May-19 15:47:51 by anonymous:
Some improvements, new comments and a new test case.

http://www.mail-archive.com/sqlite-users%40sqlite.org/msg24859.html

http://marc.info/?l=sqlite-users&m=117958960408282&q=p3

 
2376 code active 2007 May anonymous   2007 May   3 3 -batch doesn't work soon enough edit
on Microsoft XP, given any.db and a batch file bug.bat containing

sqlite3 -batch -init bug.rc any.db "select 1;"

and bug.rc containing

.mode column

Running bug.bat gives the message "Loading resources from bug.rc"

I put in "-batch" so that the "Loading..." message would be suppressed, but it printed anyway.

 
2207 new active 2007 Jan anonymous   2007 May   3 4 "CREATE TABLE foo AS SELECT * FROM bar" doesn't copy constraints edit
(cut & paste from an email I sent to the list, though afaict it never appeared)

When creating a table using AS SELECT ... I noticed it didn't copy the constraints:

    SQLite version 3.3.8
    Enter ".help" for instructions
    sqlite> .schema
    CREATE TABLE bar (
            t INTEGER NOT NULL PRIMARY KEY,
            d INTEGER NOT NULL
    );
    sqlite> CREATE TABLE foo AS SELECT * FROM bar;
    sqlite> .schema
    CREATE TABLE foo(timestamp INTEGER,download INTEGER);
    CREATE TABLE bar (
            t INTEGER NOT NULL PRIMARY KEY,
            d INTEGER NOT NULL
    );

Is this expected behavior?

I find myself in a sitation where ideally I would like this to create a table copying contraints (so I can do some processing in dynamically created temporary tables).

I had a quick look over the documentation and it doesn't mention this either way.

2007-Jan-30 18:52:49 by drh:
I have a lot of code that depends "CREATE TABLE ... AS SELECT" not copying the constraints. Changing this so that constraints are copied would break my code - which is something I am disinclined to do.


2007-Jan-30 19:06:34 by anonymous:
Can we perhaps just document this then as the intended behavior then then close this bug out please?


2007-Jan-30 21:57:10 by anonymous:
How about a new feature:

CREATE TABLE foo WITH CONSTRAINTS AS SELECT * FROM bar

 
2374 code active 2007 May anonymous   2007 May   5 5 "ALTER TABLE foo RENAME TO bar" will quote bar (ie. 'bar') edit
Cosmetic issue.

When a table is renamed, its representation in the schema ends up being quoted. Whilst this is harmless it also seems to be unnecessary?

  sqlite> create table foo ( c1 integer );
  sqlite> .sc
  CREATE TABLE foo ( c1 integer );

  sqlite> alter table foo rename to bar;

  sqlite> .sc
  CREATE TABLE 'bar' ( c1 integer );

  sqlite> select oid,sql from sqlite_master;
  rowid       sql
  ----------  ---------------------------------
  1           CREATE TABLE 'bar' ( c1 integer )
 
2373 new active 2007 May anonymous   2007 May   4 4 "create table as explain <expr>" doesn't work edit
When using EXPLAIN to try to figure out why/when indices are being used on some tables and not all sometimes I get a lot of data returned.

It would be useful to be able to store this in a table so I can actually use SQL itself to examine the differences in the results I get.

 
2372 code active 2007 May anonymous   2007 May   4 4 sqlite3TestLockingStyle not used if SQLITE_FIXED_LOCKING_STYLE defined edit
static function sqlite3TestLockingStyle() is not used if SQLITE_FIXED_LOCKING_STYLE is defined. It should probably be wrapped in: #ifndef SQLITE_FIXED_LOCKING_STYLE
 
2371 code active 2007 May anonymous   2007 May   2 2 sqlite3_errcode() and sqlite3_errmsg() return unexpected results edit
The manual says that sqlite3_step() directly returns explicit error code if the query has been prepared with the new API sqlite3_prepare_v2().

Subsequently, the functions sqlite3_errcode() and sqlite3_errmsg() should return the correct appropriate error values as well, which they don't - instead something not matching the error is returned. One has to call sqlite3_reset() to get the correct values which should be unnecessary.

See the example output below :

  sqlite3_step: result rc = 19, errcode = 1, errmsg = SQL logic error or missing database
  sqlite3_reset: result rc = 19, errcode = 19, errmsg = PRIMARY KEY must be unique

Code for this example:

  #include <stdio.h>
  #include <stdlib.h>
  #include <unistd.h>
  #include <string.h>
  #include <sqlite3.h>

  int
  main(int argc, char **argv)
  {
      sqlite3 *db;
      int rc, errcode;
      sqlite3_stmt *stmt;
      const char *pztail;
      char *query;
      const char *errmsg;

      rc = sqlite3_open("test.db", &db);
      rc = sqlite3_exec(db, "create table t (x integer not null primary key)", NULL, NULL, NULL);
      rc = sqlite3_exec(db, "insert into t values(1)", NULL, NULL, NULL);
      query = "insert into t values(?)";
      rc = sqlite3_prepare_v2(db, query, strlen(query), &stmt, &pztail);
      if( rc!=SQLITE_OK ){
          fprintf(stderr, "SQL error: %d\n", rc);
          exit(1);
      }
      rc = sqlite3_bind_int(stmt, 1, 1);
      if( rc!=SQLITE_OK ){
          fprintf(stderr, "SQL error: %d\n", rc);
          exit(1);
      }
      rc = sqlite3_step(stmt);
      errcode = sqlite3_errcode(db);
      errmsg = sqlite3_errmsg(db);
      printf("sqlite3_step: result rc = %d, errcode = %d, errmsg = %s\n",
          rc, errcode, errmsg);
      rc = sqlite3_reset(stmt);
      errcode = sqlite3_errcode(db);
      errmsg = sqlite3_errmsg(db);
      printf("sqlite3_reset: result rc = %d, errcode = %d, errmsg = %s\n",
          rc, errcode, errmsg);
  }
 
2350 code active 2007 May anonymous   2007 May   1 1 Temp files not deleted on WinCE edit
When using SQLite 3.3.17 on WinCE device temp files end up being accumulated in Temp directory, eventually filling up device's open file quota (999?) and making further operation that requires creation of files impossible.

I have noticed that after a while all queries start failing on the device although if I copied the database to PC and tried the same query I had no problems with it. Started tracing through the code and noticed that sqlite3WinOpenExclusive() is consistently failing to create a file (CreateFileW() would always fail). It turned out that Temp directory was full of 0 length temp files -- after deleting all of them device started working again for a while.

Then started looking for the cause -- winceDestroyLock() never gets to execute DeleteFileW() and delete the file even though pFile->zDeleteOnClose is set because check on the top of the winceDestroyLock() function (if (pFile->hMutex)) always fails thanks to pFile->hMutex being 0.

sqlite3WinOpenReadWrite() calls winceCreateLock() on temporary file (zFilename) but sqlite3WinOpenExclusive() doesn't hence winceDestroyLock() cannot delete temporary file.

sqlite3PagerClose() contains this piece of commented out code:

  /* Temp files are automatically deleted by the OS
  ** if( pPager->tempFile ){
  **   sqlite3OsDelete(pPager->zFilename);
  ** }
  */

Comment is not true for WinCE as WinCE doesn't support FILE_ATTRIBUTE_TEMPORARY and FILE_FLAG_DELETE_ON_CLOSE flags. Perhaps it should be surrounded with #if OS_WINCE/#endif instead of being commented out?


2007-May-14 00:09:20 by anonymous:
Tried re-enabling that piece of code in sqlite3PagerClose() and it doesn't help.

Looks like even file handles for temporary files never get closed on WinCE so not only number of temporary file in Temp directory just keeps growing but also filed descriptor for each of them remains open, draining system resources (consistent with observed behavior of rapid and nasty system performance deterioration).

It appears that sqlite3VdbeFreeCursor() always exits immediately because pCx is always NULL and thus sqlite3BtreeClose() never gets called:

	void sqlite3VdbeFreeCursor(Vdbe *p, Cursor *pCx){
	  if( pCx==0 ){
	    return;
	  }
	  if( pCx->pCursor ){
	    sqlite3BtreeCloseCursor(pCx->pCursor);
	  }
	  if( pCx->pBt ){
	    sqlite3BtreeClose(pCx->pBt);
	  }

sqlite3BtreeClose() then calls sqlite3PagerClose() which probably explains why re-enabling that code doesn't help -- sqlite3PagerClose() itself never gets called.


2007-May-14 05:16:59 by anonymous:
Did some comparison between operation of the SQLite on WinXP and WinCE.

Comment in previous remark about sqlite3VdbeFreeCursor() always exiting immediately because pCx is always NULL is not correct. pCX is NULL most of the time but this is true for both WinXP and WinCE. WinXP has no problems getting rid of its temp files thanks to its support for FILE_ATTRIBUTE_TEMPORARY and FILE_FLAG_DELETE_ON_CLOSE flags. WinCE also does get its temp file descriptors closed (previous statement incorrect on that account too), it's just that its temp files do not get cleaned up automatically. Initial statement that this was caused because of winceDestroyLock() actually is correct.

Temp files are opened with sqlite3WinOpenExclusive() which DOESN'T use mutex (instead of with sqlite3WinOpenReadWrite() which DOES use mutex) so part of winClose() that was supposed to take care of deleting files after handle is closed (i.e. calling winceDestroyLock()) doesn't work:

	static int winClose(OsFile **pId){
	  winFile *pFile;
	  int rc = 1;
	  if( pId && (pFile = (winFile*)*pId)!=0 ){
	    int rc, cnt = 0;
	    OSTRACE2("CLOSE %d\n", pFile->h);
	    do{
	      rc = CloseHandle(pFile->h);
	    }while( rc==0 && cnt++ < MX_CLOSE_ATTEMPT && (Sleep(100), 1) );
	#if OS_WINCE
	    winceDestroyLock(pFile);
	#endif

OS_WINCE part should go something like this (although probably not exactly like this as this feels like an ugly hack):

	#if OS_WINCE
	    if (pFile->hMutex){
	      winceDestroyLock(pFile);
	    }else{
	      if( pFile->zDeleteOnClose ){
	        DeleteFileW(pFile->zDeleteOnClose);
	        sqliteFree(pFile->zDeleteOnClose);
	        pFile->zDeleteOnClose = 0;
	      }
	    }
	#endif


2007-May-14 05:20:33 by anonymous:
Tried the code from the previous remark and it does clean up Temp directory!

I also tried uncommenting that piece of code in sqlite3PagerClose() mentioned in the first remark and it looks like that temp files it tries to delete are never there. I'm guessing that that piece of code can remain commented out. Will keep an eye on it -- in case temp files still eventually start accumulating again even after the fix from previous remark is applied I'll reconsider.

 
2352 code active 2007 May anonymous   2007 May   5 3 timeout just 500 msec to soon edit
After upgrading from 3.3.12 to 3.3.17, the setting of a timeout behaves differently. It occurs exactly 500 msecs sooner. Of course assuming the database is locked and the timeout is set to a value larger than 500 msecs.
 
2351 secure active 2007 May anonymous   2007 May   2 2 UTF-16 convertor accepts malformed UTF-8 sequence edit
UTF encoding convertor accepts some invalid sequences of UTF-8.

  $ ./sqlite3 -version
  3.3.17
  $ cat test.sql
  select hex(a), hex(b), a == b from (select '<CE><B1>' a, '<D0><31>' b);
  pragma encoding = 'UTF-16';
  select hex(a), hex(b), a == b from (select '<CE><B1>' a, '<D0><31>' b);
  $ ./sqlite3 < test.sql
  CEB1|D031|0
  CEB1|CEB1|1

It's necessary to validate subsequent bytes.

 
2342 doc active 2007 May anonymous   2007 May   1 1 Document that prepared incremental vacuum needs multiple sqlite3_step edit
Unlike other PRAGMA commands, an sqlite3_prepare()ed PRAGMA incremental_vacuum(x); requires x calls to sqlite3_step to vacuum all x pages, or until sqlite3_step no longer returns SQLITE_ROW.

Maybe it is worth explicitly pointing this out in the documentation in order to avoid confusion?

 
2340 code active 2007 May anonymous   2007 May   3 3 "./configure && make testfixture" link problem edit
Until the (extremely welcome) sqlite3 blob I/O functionality was checked in, it was possible to run this command on a clean source tree:

  ./configure && make testfixture

But it now produces these unresolved externals:

  ./src/tclsqlite.c:297: warning: excess elements in struct initializer
  ./src/tclsqlite.c:297: warning: (near initialization for `IncrblobChannelType')
  /tmp/ccmyzR97.o: In function `test_blob_read':
  /sqlite/cvs/sqlite/./src/test1.c:1545: undefined reference to `_sqlite3_blob_read'
  /tmp/ccmyzR97.o: In function `test_blob_write':
  /sqlite/cvs/sqlite/./src/test1.c:1592: undefined reference to `_sqlite3_blob_write'
  /tmp/ccb5dcDK.o: In function `incrblobClose':
  /sqlite/cvs/sqlite/./src/tclsqlite.c:156: undefined reference to `_sqlite3_blob_close'
  /tmp/ccb5dcDK.o: In function `incrblobInput':
  /sqlite/cvs/sqlite/./src/tclsqlite.c:194: undefined reference to `_sqlite3_blob_bytes'
  /sqlite/cvs/sqlite/./src/tclsqlite.c:202: undefined reference to `_sqlite3_blob_read'
  /tmp/ccb5dcDK.o: In function `incrblobOutput':
  /sqlite/cvs/sqlite/./src/tclsqlite.c:226: undefined reference to `_sqlite3_blob_bytes'
  /sqlite/cvs/sqlite/./src/tclsqlite.c:235: undefined reference to `_sqlite3_blob_write'
  /tmp/ccb5dcDK.o: In function `incrblobSeek':
  /sqlite/cvs/sqlite/./src/tclsqlite.c:264: undefined reference to `_sqlite3_blob_bytes'
  /tmp/ccb5dcDK.o: In function `DbObjCmd':
  /sqlite/cvs/sqlite/./src/tclsqlite.c:322: undefined reference to `_sqlite3_blob_open'
  collect2: ld returned 1 exit status
  make: *** [testfixture.exe] Error 1

The 2 warnings related to src/tclsqlite.c:297 also seem to indicate a problem.

I'm using Tcl 8.4.

2007-May-06 21:16:01 by anonymous:
Workaround - hack the generated Makefile to use:

  # Compiler options needed for programs that use the TCL library.
  #
  TCC += -I/usr/local/include -DSQLITE_OMIT_INCRBLOB=1
 
2333 new active 2007 May anonymous   2007 May   5 4 support memory mapped files edit
Both Unix and Windows support memory mapped files, i.e. you can load file into memory and then operate on the memory while system reads and writes data from/to a file.

It would be useful if Sqlite could support this -- the user would load file to memory (map file) and then open a database with not a filename but handle to this memory-map.

Reason of this wish: I have read-only database which takes too much time for the first load. Any next load (not long from the first one) is quite fast. So I would like to map the entire database into memory and keep it resident (i.e. database would be in memory even my app quits). This way any load (except the first) would be fast. In other words it would be a permanent database (the whole database) cache.

2007-May-03 11:27:36 by drh:
The size of memory mapped files is limited by the address space of the underlying CPU. So on a 32-bit machine, databases larger than 4GiB are not accessible using memory mapped files.
 
2329 new active 2007 Apr anonymous   2007 Apr   5 4 add a feature to .dump : partial dumps edit
Hi,

I would like to request a feature related to the .dump command

introduction: ;-) I have a large database (few GB) and I only remove rows from it. since I forgot to use the pragma auto_vacuum, I am creating using .dump another database that has the triggers and pragmas I needed. okay. so, this is slow. probably mainly because of the millions of inserts it needs to perform.

the feature request: partial .dumps - that is, you specify how many rows or how many megabytes to dump. this should add a begin transaction and a commit at the end of each dumpfile, and enumerate them as well.

for example:

sqlite3 mylarge.db .partialdump --rows=40000 > dumpfile.sql<enter>

dumping.. please wait

    1. dumping first 40000 rows to file dumpfile.001.sql ....done
    2. dumping second 40000 rows to file dumpfile.002.sql ....done

etc..

hope this is obvious enough. if you need more info contact me. I know sqlite tries to be minimal and to the point, but this is a good feature and very handy. (dumping to text and then splitting can take too much space and then impractical) Thanks, Kobi

 
2328 code active 2007 Apr anonymous   2007 Apr   1 1 Makefile sqlite3.c target breakage for C++ edit
This is generated by "make sqlite3.c":

  #if 0
  extern "C" {
  #endif
2007-Apr-29 06:02:20 by anonymous:
If the entire sqlite3.c almalgomation is wrapped with:

  #ifdef __cplusplus
  extern "C" {
  #endif

  ...

  #ifdef __cplusplus
  }
  #endif

then it could be compiled with a C++ compiler.

Please close this ticket if you did not intend to have this capability.

 
2327 new active 2007 Apr anonymous   2007 Apr anonymous 2 1 "DELETE" operation makes memory rise edit
First declare a standard SQL script: delete from TableName where ....; Then calling repeatedly the sqlite3_exec() to process this "DELETE" operation. Surprisely the memory was rising fast, and couldn't be freed even the program exitted.
 
2322 code active 2007 Apr anonymous   2007 Apr   1 1 Windows error: datetime('2000-10-29 06:00:00','localtime') edit
NY time zone.

Windows (from http://sqlite.org/sqlite-3_3_17.zip)

  SELECT coalesce(datetime('2000-10-29 06:00:00','localtime'),'NULL');
  2000-10-29 02:00:00

Linux (from latest CVS, same TZ)

  SELECT coalesce(datetime('2000-10-29 06:00:00','localtime'),'NULL');
  2000-10-29 01:00:00

make test errors on Windows only:

  date-6.2...
  Expected: [{2000-10-29 01:00:00}]
       Got: [{2000-10-29 02:00:00}]

  date-6.3...
  Expected: [{2000-04-02 01:59:00}]
       Got: [{2000-04-02 02:59:00}]

  date-6.6...
  Expected: [{2000-10-29 07:00:00}]
       Got: [{2000-10-29 06:00:00}]

  date-6.7...
  Expected: [{2000-04-02 06:59:00}]
       Got: [{2000-04-02 05:59:00}]
2007-Apr-26 23:09:12 by anonymous:
Confirmed Windows bug on Windows 2000 in NY time zone with Y2K7DST OS patch.


2007-Apr-27 22:03:25 by drh:
Do I correctly understand the previous remark to say that this is confirmed to be a bug in Windows, not a bug in SQLite? It is identical code in SQLite for both operating systems, so I would certainly suspect that the problem is in windows and not in SQLite. But it would be nice to have confirmation of this before closing the ticket.


2007-Apr-28 03:19:06 by anonymous:
I meant to write "This erroneous SQLite datetime() output can also be seen on my Windows 2000 machine."

Does it work correctly for you under Windows XP or 2000 with the DST patch?

OS bug or not, it would be strange to not have the datetime() function correctly on a primary platform. If that were the case, it would be better to #ifdef it out of the Windows compile altogether.


2007-Apr-28 03:59:27 by anonymous:
This is a bigger mess than I thought.

http://groups.google.com/group/perl.perl5.porters/msg/e632557614474014?hl=en&

Key phrase:

"This API only provides the transition times according to the current DST rules. There is no database of historical transition times. That means that localtime() applied to previous years will use the new transition times even for old timestamps."

Please don't close this bug. Perhaps some industrious Windows programmer will have a correct solution for it one day. But in the meantime, Windows SQLite users should be aware of it.

 
2326 doc active 2007 Apr anonymous   2007 Apr a.rottmann 5 2 miss one word 'list' edit
in documentation sqlite3: A command-line access program for SQLite

The sqlite3 program is able to show the results of a query in eight different formats: "csv", "column", "html", "insert", "line", "tabs", and "tcl".

missed one format: "list"

it should be: The sqlite3 program is able to show the results of a query in eight different formats: "csv", "column", "html", "insert", "line", "list", "tabs", and "tcl".

 
2320 code active 2007 Apr anonymous   2007 Apr drh 1 1 sqlite3_open(sFN_with_umlaut) edit
Do it in a standard MS Visual Studio Project:

0. CString sFnWithUmlaut = "c:\\long path\\path with umlauts äÄ\\db";
1. call sqlite3_open(sFnWithUmlaut);
2. db cannot be opened, because the transformation functions utf8ToUnicode/unicodeToUtf8 work incorrect

Is there a way to correct this error on win32?
Is there a workaround?

For a solution thanks in advance...
2007-Apr-25 20:21:24 by anonymous:
The functions work correctly, but you are not using them in the correct way. The parameter to sqlite3_open function should be UTF8 string, but you are passing one that is specific to your code page.
 
2282 code active 2007 Apr anonymous   2007 Apr   3 4 Update on view with no matching trigger does not raise error edit
Attempting to update a view with no triggers properly fails with the error

  sqlite> update foo set key=key+1;
  SQL error: cannot modify foo because it is a view

However, if a single trigger is added that contains a WHEN clause, then UPDATE statements that do not satisfy that WHEN clause silently succeed without raising any error.

  sqlite> select 'Before:'; select * from foo; update foo set key=key+1; select 'After:'; select * from foo;
  Before:
  1|42|forty-two|42.0
  2|69|sixty-nine|69.0
  After:
  1|42|forty-two|42.0
  2|69|sixty-nine|69.0
2007-Apr-18 21:50:00 by anonymous:
Your desired behavior can be accomplished by changing your trigger to:

  create trigger foo_update instead of update on foo
  begin
	select raise(ABORT, 'invalid key')
	where old.key <> new.key;

	update foo_backing set num=new.num, str=new.str, float=new.float, dirty=1
		where key=new.key;
  end;


2007-Apr-24 20:26:46 by anonymous:
I have come up with a sample patch that partially fixes this problem -- specifically, it raises an error if any rows affected by an update/delete against a view are not caught by any triggers. It does not handle uncaught inserts, however, because I couldn't quite figure out how to make that work (much of the logic for updates and deletes is almost identical, whereas the code for inserts is quite different.)

This patch adds/updates 76 lines across 4 files. I disclaim all copyright to these 76 lines.

 
2313 build active 2007 Apr anonymous   2007 Apr   3 3 readline.h is not properly detected edit
This is actually an old issue i also had with 2.8.15.

configure says "checking for readline.h... no", but it really needs to look for readline/readline.h (or both?)

this is easily fixed with "--with-readline-inc=-I/path/to/include" although the actual syntax for this is a bit unusual/unintuitive) but such "fix" should not be needed as i had these in my environment:

CPPFLAGS="-I/path/to/include" CFLAGS="-I/path/to/include"

(The library was found by configure, thanks to my LDFLAGS environment setting which is similar to the above.)

 
2303 code active 2007 Apr anonymous   2007 Apr   1 1 Encrypted databases: No page cache due to problem in pagerSharedLock edit
With codec encryption enabled, pagerSharedLock always invalidates the page cache, even if no changes have occured since the cache was last valid and it would be safe to retain the cached pages. This in fact disables the newly improved page cache for encrypted databases and slows down performance.

The problem occurs because pagerSharedLock reads the change counter directly from the database file without codec decryption. Since the codec always encrypts full pages, the 4 bytes at offset 24 are read as encrypted data and do not match Pager->iChangeCount.

To solve, codecs would be required to store the 4 bytes at offset 24 of page 1 unencrypted. This would, however, render those 4 bytes vulnerable to attacks. It would therefore be more secure if pagerSharedLock could decrypt page one prior to extracting the change counter.

Check-in [3844] does not fix the problem to reset the cache if the codec is changed but the database file is not. The following procedure for opening an encrypted database no longer works with the improved page cache:

  • Open an encrypted database. Do not set a key yet as we (pretend to) believe that the database is not encrypted.

  • Access the DB for reading. This returns SQLITE_NOTADB, so we conclude that the DB is encrypted.

  • Attach the proper codec using sqlite3CodecAttach.

  • Access the DB again. Problem: This still returns SQLITE_NOTADB because the old page cache is still in use and is not reloaded.

The codec change is not detected because the pager checks the unencrypted DB file instead of the decrypted page. The file of course did not change, but the decrypted page did because of the new codec. The cache should therefore be cleared.

A workaround would be possible if sqlite3CodecAttach could reset the page cache. Unfortunately, the method to do so (pager_reset) is static to pager.c. It seems that there once was an external function sqlite3PagerReset (it is still defined in pager.h), but its implementation is unfortunately no longer available.

Could this be fixed in a way that pagerSharedLock checks the decrypted page 1 to see if the database has been modified or, alternatively, by reverting the static pager_reset back to the external sqlite3PagerReset?

 
2308 build active 2007 Apr anonymous   2007 Apr   4 3 make sqlite3.c recreates sqlite3.c even though nothing changed edit
When building the amalgamized sqlite3.c source file, make will recreate the sqlite3.c source file each time it's run. When using this as part of a larger build process, this is annoying, since it will result in unnecessary compilations.

The fix is to rename the makefile target target_source to tsrc to make sure make will be able to properly detect the dependencies. Below is a patch for Makefile.in that fixes this:

  --- Makefile.in 19 Apr 2007 10:20:59 -0000      1.167
  +++ Makefile.in 19 Apr 2007 11:08:50 -0000
  @@ -296,14 +296,14 @@
   # files are automatically generated.  This target takes care of
   # all that automatic generation.
   #
  -target_source: $(SRC) parse.c opcodes.c keywordhash.h $(VDBEHDR)
  +tsrc:  $(SRC) parse.c opcodes.c keywordhash.h $(VDBEHDR)
          rm -rf tsrc
          mkdir -p tsrc
          cp $(SRC) $(VDBEHDR) tsrc
          rm tsrc/sqlite.h.in tsrc/parse.y
          cp parse.c opcodes.c keywordhash.h tsrc

  -sqlite3.c:     target_source $(TOP)/tool/mksqlite3c.tcl
  +sqlite3.c:     tsrc $(TOP)/tool/mksqlite3c.tcl
          tclsh $(TOP)/tool/mksqlite3c.tcl

   # Rules to build the LEMON compiler generator
2007-Apr-20 06:01:23 by anonymous:
Make does not deal well with directories as dependencies (because their last modification time doesn't mean what Make thinks it means). It would be much better to use a stamp file.
 
2310 code active 2007 Apr anonymous   2007 Apr anonymous 4 4 Problem installing on AIX 5.3, ML5 after successful compile with xlc edit
After performing the suggested edits to the Makefile (from Tom Poindexter 2003-12-17):

     edit Makefile, change the TCC macro:

     TCC = xlc -q32 -qlonglong -D_LARGE_FILE=1 -D_LARGE_FILES=1 -DUSE_TCL_STUBS=1 -O2 -DOS_UNIX=1 -DOS_WIN=0 -DHAVE_USLEEP=1 -I. -I${TOP}/src

Version 3.3.15 compiled perfectly. However, a make install gave me this:

<root@tailinn 14:41:21>: /data/bld --> make install tclsh ../sqlite-3.3.15/tclinstaller.tcl 3.3 couldn't open ".libs/libtclsqlite3.so": no such file or directory while executing "open $LIBFILE" invoked from within "set in [open $LIBFILE]" (file "../sqlite-3.3.15/tclinstaller.tcl" line 23) make: 1254-004 The error code from the last command is 1.

I had to edit line 8 of ../sqlite-3-3-15/tclinstaller.tcl by adding a ".0" to the end. Then it installed perfectly.

2007-Apr-19 20:47:24 by anonymous:
Haven't used AIX or xLC for 15 years - does IBM still make UNIX machines?
 
2304 new active 2007 Apr anonymous   2007 Apr   1 1 resolve "databas is locked" problem under DEFERRED transaction edit
under DEFERRED transaction,

if there are multiple thread immediate execute writing operation after BEGIN statement,
sqlite will direct kick in "database is locked" exception,

but if you execute some reading operation before writing operation,
it works well,

could BEGIN statement acquire a shared lock and solve this problem?

{quote: relative mail archive http://www.mail-archive.com/sqlite-users@sqlite.org/msg21768.html }

 
2302 build active 2007 Apr anonymous   2007 Apr anonymous 4 4 sqlite3 does not honor configure --disable-threads anymore edit
In a non-threaded TCL build, the TEA configuration option --disable-threads is no longer honored. In version 3.3.12 this used to work:

  (test) 49 % packa req sqlite3
couldn't load file "/usr/local/tcl/8.5a5-1/lib/sqlite3.3.15/libsqlite3.3.15.so": /usr/local/tcl/8.5a5-1/lib/sqlite3.3.15/libsqlite3.3.15.so: undefined symbol: pthread_create
(test) 50 % packa req -exact sqlite3 3.3.12
3.3.12

In new file tclsqlite3.c, line 11734, threading is hard-coded with

  #define THREADSAFE 1

A workaround for non-threaded builds is to set this manually to

  #define THREADSAFE 0
2007-Apr-15 17:03:07 by anonymous:
I encountered the same problem and I agree that this change is problematic and should be reverted.


2007-Apr-15 18:03:05 by drh:
Why is it such a problem that the library is threadsafe? Just because it is threadsafe does not mean you are required to use threads, or anything like that. Everything continues to work normally in a single threaded application. There is no measurable performance impact. Why is it so important to you that the threading mutexes not be enabled?


2007-Apr-15 19:34:11 by anonymous:
In a shared library setting it's not such a big deal, but in a purely static binary it can pull in a fair bit of unwanted thread library code. Also, some embedded UNIX-like targets lack a pthreads implementation.

The autoconf default can be threadsafe instead of non-threadsafe. It would be nice if it respected the autoconf flag as it did before.

 
2299 build active 2007 Apr anonymous   2007 Apr   1 1 Cannot compile sqlite-3.3.15 on linux rhel edit
My platform: Linux 2.6.9-42.0.3.ELsmp #1 SMP Mon Sep 25 17:28:02 EDT 2006 i686 i686 i386 GNU/Linux

My gcc version: gcc (GCC) 3.4.6 20060404 (Red Hat 3.4.6-3)

My problem: I'm (ultimately) trying to get svntrac built and installed on this machine, but cannot compile the sqlite dependency.

If I follow the documented build procedure, namely: 1) Create sibling directory to source directory 2) Run ../sqlite-3.3.15/configure from build directory 3) Run make from build directory

I get build errors, mostly:

undefined reference to `__getreent'

2007-Apr-13 17:20:23 by anonymous:
gcc -g -O2 -o lemon ../sqlite-3.3.15/tool/lemon.c /tmp/ccOClNK1.o(.text+0x7c): In function `Action_new': ../sqlite-3.3.15/tool/lemon.c:344: undefined reference to `__getreent' /tmp/ccOClNK1.o(.text+0x153): In function `acttab_alloc': ../sqlite-3.3.15/tool/lemon.c:440: undefined reference to `__getreent' /tmp/ccOClNK1.o(.text+0x1ed): In function `acttab_action': ../sqlite-3.3.15/tool/lemon.c:455: undefined reference to `__getreent' /tmp/ccOClNK1.o(.text+0x223): In function `myassert': ../sqlite-3.3.15/tool/lemon.c:567: undefined reference to `__getreent' /tmp/ccOClNK1.o(.text+0x4e4): In function `acttab_insert': ../sqlite-3.3.15/tool/lemon.c:497: undefined reference to `__getreent' /tmp/ccOClNK1.o(.text+0x68f):../sqlite-3.3.15/tool/lemon.c:1362: more undefined references to `__getreent' follow /tmp/ccOClNK1.o(.text+0x280f): In function `tplt_xfer': ../sqlite-3.3.15/tool/lemon.c:2980: undefined reference to `__ctype_ptr' /tmp/ccOClNK1.o(.text+0x299a): In function `tplt_open': ../sqlite-3.3.15/tool/lemon.c:3026: undefined reference to `__getreent' /tmp/ccOClNK1.o(.text+0x2a54): In function `tplt_linedir': ../sqlite-3.3.15/tool/lemon.c:3042: undefined reference to `__getreent' /tmp/ccOClNK1.o(.text+0x2a5d):../sqlite-3.3.15/tool/lemon.c:3042: undefined reference to `__swbuf_r' /tmp/ccOClNK1.o(.text+0x2a8b):../sqlite-3.3.15/tool/lemon.c:3041: undefined reference to `__getreent' /tmp/ccOClNK1.o(.text+0x2a93):../sqlite-3.3.15/tool/lemon.c:3041: undefined reference to `__swbuf_r' /tmp/ccOClNK1.o(.text+0x2b68): In function `tplt_print': ../sqlite-3.3.15/tool/lemon.c:3061: undefined reference to `__getreent' /tmp/ccOClNK1.o(.text+0x2b71):../sqlite-3.3.15/tool/lemon.c:3061: undefined reference to `__swbuf_r' /tmp/ccOClNK1.o(.text+0x2ba6):../sqlite-3.3.15/tool/lemon.c:3065: undefined reference to `__getreent' /tmp/ccOClNK1.o(.text+0x2bae):../sqlite-3.3.15/tool/lemon.c:3065: undefined reference to `__swbuf_r' /tmp/ccOClNK1.o(.text+0x3196): In function `print_stack_union': ../sqlite-3.3.15/tool/lemon.c:3366: undefined reference to `__ctype_ptr' /tmp/ccOClNK1.o(.text+0x3319):../sqlite-3.3.15/tool/lemon.c:3387: undefined reference to `__getreent' /tmp/ccOClNK1.o(.text+0x3ae0): In function `translate_code': ../sqlite-3.3.15/tool/lemon.c:3207: undefined reference to `__ctype_ptr' /tmp/ccOClNK1.o(.text+0x447c): In function `ReportTable': ../sqlite-3.3.15/tool/lemon.c:3534: undefined reference to `__ctype_ptr' /tmp/ccOClNK1.o(.text+0x44a3):../sqlite-3.3.15/tool/lemon.c:3535: undefined reference to `__ctype_ptr' /tmp/ccOClNK1.o(.text+0x55d4):../sqlite-3.3.15/tool/lemon.c:3575: undefined reference to `__getreent' /tmp/ccOClNK1.o(.text+0x5800): In function `Symbol_new': ../sqlite-3.3.15/tool/lemon.c:4259: undefined reference to `__ctype_ptr' /tmp/ccOClNK1.o(.text+0x5ba3): In function `Parse': ../sqlite-3.3.15/tool/lemon.c:2500: undefined reference to `__ctype_ptr' /tmp/ccOClNK1.o(.text+0x5c2c):../sqlite-3.3.15/tool/lemon.c:2407: undefined reference to `__ctype_ptr' /tmp/ccOClNK1.o(.text+0x619b):../sqlite-3.3.15/tool/lemon.c:1997: undefined reference to `__ctype_ptr' /tmp/ccOClNK1.o(.text+0x61eb):../sqlite-3.3.15/tool/lemon.c:2201: undefined reference to `__ctype_ptr' /tmp/ccOClNK1.o(.text+0x628e):../sqlite-3.3.15/tool/lemon.c:2027: more undefined references to `__ctype_ptr' follow /tmp/ccOClNK1.o(.text+0x65b8): In function `Parse': ../sqlite-3.3.15/tool/lemon.c:2439: undefined reference to `__getreent' /tmp/ccOClNK1.o(.text+0x6603):../sqlite-3.3.15/tool/lemon.c:2415: undefined reference to `__ctype_ptr' /tmp/ccOClNK1.o(.text+0x6628):../sqlite-3.3.15/tool/lemon.c:2415: undefined reference to `__ctype_ptr' /tmp/ccOClNK1.o(.text+0x6baa):../sqlite-3.3.15/tool/lemon.c:2164: undefined reference to `__ctype_ptr' /tmp/ccOClNK1.o(.text+0x7bad): In function `main': ../sqlite-3.3.15/tool/lemon.c:1419: undefined reference to `__getreent' /tmp/ccOClNK1.o(.text+0x7c65):../sqlite-3.3.15/tool/lemon.c:1445: undefined reference to `__getreent' /tmp/ccOClNK1.o(.text+0x7ca0):../sqlite-3.3.15/tool/lemon.c:1425: undefined reference to `__getreent' /tmp/ccOClNK1.o(.text+0x7d4c):../sqlite-3.3.15/tool/lemon.c:1457: undefined reference to `__ctype_ptr' /tmp/ccOClNK1.o(.text+0x7e4b):../sqlite-3.3.15/tool/lemon.c:1514: undefined reference to `__getreent' collect2: ld returned 1 exit status make: *** [lemon] Error 1


2007-Apr-15 13:51:08 by anonymous:
I am not real familiar with Red Hat, but that looks to me like a case of your C library headers being out of sync with the library proper. There are several different ways that could happen; I would guess that the most probable is that you installed a version of GCC by hand, it copied some of the C library headers to a private directory (GCC tends to do this when you build it from source, unfortunately) and then you installed a new version of the C library from packages.


2007-Apr-15 13:54:33 by anonymous:
To be clearer, I think this is a local installation problem, not a bug in SQLite.
 
2301 build active 2007 Apr anonymous   2007 Apr   1 1 Latest cvs 3.3.15 fails lock4-1.3 test edit
  export CFLAGS=-O3
  ./configure --prefix=/usr/local
  make
  make test

  produces a single failure...

  lock4-1.2... Ok
  lock4-1.3...
  Error: database is locked
  lock4-999.1... Ok
2007-Apr-15 02:47:15 by anonymous:
which OS?


2007-Apr-15 11:31:46 by drh:
To amplify the previous comment, I observe that the test works fine for me on both Linux (SuSE 10.1) and Mac OS-X x86.
 
2297 code active 2007 Apr anonymous   2007 Apr drh 3 3 uninitialized var (with patch) edit
Warnings with amalgamation and NDEBUG.
2007-Apr-12 21:21:29 by drh:
I looked at the suggested changes and I didn't find any cases where it really was possible to use an uninitialized variable, at least not in a harmful way. Did I overlook something, or is this ticket just a request to silence compiler warnings?


2007-Apr-13 00:08:36 by anonymous:
vdbe.c with n, n64, payloadSize and payloadSize64
sqlite3BtreeKeySize,sqlite3BtreeLast return are not checked. You can not be sure the pointer passed as second argument will be init depending on the return of restoreOrClearCursorPosition (btree.c).

page.c with ro
Compiled with -DNDEBUG, the return of sqlite3OsOpenReadWrite is not checked before making a move with 'ro'.

For sContext.zAuthContext in delete.c/update.c, you're the one.

gcc (compiler in general) warnings are quite usefull, i don't think it's a good idea to ignore them and accumulate danger. Perhaps one day, one line in a subroutine will modify some tricky behavior and (re)raise a previous checked warning, making it completely normal and 'under control'.

 
2256 new active 2007 Feb anonymous   2007 Apr drh 5 1 Add POSITION() function (SQL-92 standard) edit
Hi!

Just voting for POSITION() function.

It's mighty useful when you need to modify a field in real time with SUBSTR() function (for instance, when 'start' parameter of SUBSTR() function needs to be variable according to POSITION() function).

Many thanks!

Regards.

2007-Mar-23 14:10:14 by anonymous:
Yes, this is a function which I often suffered from not being available.
 
2288 code active 2007 Apr anonymous   2007 Apr   4 2 FTS does not support REPLACE edit
Simple to replicate:

CREATE VIRTUAL TABLE fts_table USING fts2(text);

INSERT OR REPLACE INTO fts_table (rowid, text) VALUES (1, 'text1');

INSERT OR REPLACE INTO fts_table (rowid, text) VALUES (1, 'text2');

The first insert succeeds, the second fails. Also occurs with fts1.

2007-Apr-10 15:27:10 by anonymous:
http://www.mail-archive.com/sqlite-users%40sqlite.org/msg23865.html
 
2283 warn active 2007 Apr anonymous   2007 Apr   1 1 Compile warning by VCToolkit2003 edit
sqlite3.c D:\Ulti\MyApps\USQLite3\SQLite\sqlite3.c(5494) : warning C4244: '=' : conversion from 'double' to 'int', possible loss of data D:\Ulti\MyApps\USQLite3\SQLite\sqlite3.c(5495) : warning C4244: '=' : conversion from 'double' to 'int', possible loss of data D:\Ulti\MyApps\USQLite3\SQLite\sqlite3.c(5600) : warning C4244: '=' : conversion from 'double' to 'int', possible loss of data D:\Ulti\MyApps\USQLite3\SQLite\sqlite3.c(5601) : warning C4244: '=' : conversion from 'double' to 'int', possible loss of data D:\Ulti\MyApps\USQLite3\SQLite\sqlite3.c(5604) : warning C4244: '=' : conversion from 'double' to 'int', possible loss of data D:\Ulti\MyApps\USQLite3\SQLite\sqlite3.c(5605) : warning C4244: '=' : conversion from 'double' to 'int', possible loss of data D:\Ulti\MyApps\USQLite3\SQLite\sqlite3.c(5606) : warning C4244: '=' : conversion from 'double' to 'int', possible loss of data D:\Ulti\MyApps\USQLite3\SQLite\sqlite3.c(5607) : warning C4244: '=' : conversion from 'double' to 'int', possible loss of data D:\Ulti\MyApps\USQLite3\SQLite\sqlite3.c(5622) : warning C4244: '=' : conversion from 'double' to 'int', possible loss of data D:\Ulti\MyApps\USQLite3\SQLite\sqlite3.c(5623) : warning C4244: '=' : conversion from 'double' to 'int', possible loss of data D:\Ulti\MyApps\USQLite3\SQLite\sqlite3.c(5625) : warning C4244: '=' : conversion from 'double' to 'int', possible loss of data D:\Ulti\MyApps\USQLite3\SQLite\sqlite3.c(5668) : warning C4244: 'initializing' : conversion from 'double' to 'int', possible loss of data D:\Ulti\MyApps\USQLite3\SQLite\sqlite3.c(5674) : warning C4244: '=' : conversion from 'double' to 'time_t', possible loss of data D:\Ulti\MyApps\USQLite3\SQLite\sqlite3.c(5785) : warning C4244: '=' : conversion from 'double' to 'int', possible loss of data D:\Ulti\MyApps\USQLite3\SQLite\sqlite3.c(5791) : warning C4244: '=' : conversion from 'double' to 'int', possible loss of data D:\Ulti\MyApps\USQLite3\SQLite\sqlite3.c(5883) : warning C4244: '+=' : conversion from 'double' to 'int', possible loss of data D:\Ulti\MyApps\USQLite3\SQLite\sqlite3.c(5889) : warning C4244: '=' : conversion from 'double' to 'int', possible loss of data D:\Ulti\MyApps\USQLite3\SQLite\sqlite3.c(5895) : warning C4244: '+=' : conversion from 'double' to 'int', possible loss of data D:\Ulti\MyApps\USQLite3\SQLite\sqlite3.c(6104) : warning C4244: '=' : conversion from 'double' to 'int', possible loss of data D:\Ulti\MyApps\USQLite3\SQLite\sqlite3.c(9622) : warning C4244: '=' : conversion from 'u64' to 'unsigned char', possible loss of data D:\Ulti\MyApps\USQLite3\SQLite\sqlite3.c(9625) : warning C4244: '=' : conversion from 'u64' to 'unsigned char', possible loss of data D:\Ulti\MyApps\USQLite3\SQLite\sqlite3.c(9632) : warning C4244: '=' : conversion from 'u64' to 'u8', possible loss of data D:\Ulti\MyApps\USQLite3\SQLite\sqlite3.c(16030) : warning C4244: 'initializing' : conversion from 'i64' to 'LONG', possible loss of data D:\Ulti\MyApps\USQLite3\SQLite\sqlite3.c(16031) : warning C4244: 'initializing' : conversion from 'i64' to 'LONG', possible loss of data D:\Ulti\MyApps\USQLite3\SQLite\sqlite3.c(16071) : warning C4244: 'initializing' : conversion from 'i64' to 'LONG', possible loss of data D:\Ulti\MyApps\USQLite3\SQLite\sqlite3.c(16075) : warning C4244: 'function' : conversion from 'i64' to 'LONG', possible loss of data D:\Ulti\MyApps\USQLite3\SQLite\sqlite3.c(17312) : warning C4018: '<' : signed/unsigned mismatch D:\Ulti\MyApps\USQLite3\SQLite\sqlite3.c(17903) : warning C4244: 'function' : conversion from 'i64' to 'int', possible loss of data D:\Ulti\MyApps\USQLite3\SQLite\sqlite3.c(17908) : warning C4244: 'function' : conversion from 'i64' to 'int', possible loss of data D:\Ulti\MyApps\USQLite3\SQLite\sqlite3.c(18120) : warning C4244: '=' : conversion from 'i64' to 'u32', possible loss of data D:\Ulti\MyApps\USQLite3\SQLite\sqlite3.c(18128) : warning C4244: '=' : conversion from 'i64' to 'u32', possible loss of data D:\Ulti\MyApps\USQLite3\SQLite\sqlite3.c(18146) : warning C4018: '<' : signed/unsigned mismatch D:\Ulti\MyApps\USQLite3\SQLite\sqlite3.c(18264) : warning C4244: '=' : conversion from 'i64' to 'u32', possible loss of data D:\Ulti\MyApps\USQLite3\SQLite\sqlite3.c(18280) : warning C4244: '=' : conversion from 'i64' to 'u32', possible loss of data D:\Ulti\MyApps\USQLite3\SQLite\sqlite3.c(18668) : warning C4244: '=' : conversion from 'i64' to 'int', possible loss of data D:\Ulti\MyApps\USQLite3\SQLite\sqlite3.c(18674) : warning C4244: 'return' : conversion from 'i64' to 'int', possible loss of data D:\Ulti\MyApps\USQLite3\SQLite\sqlite3.c(18771) : warning C4018: '<=' : signed/unsigned mismatch D:\Ulti\MyApps\USQLite3\SQLite\sqlite3.c(19202) : warning C4018: '<=' : signed/unsigned mismatch D:\Ulti\MyApps\USQLite3\SQLite\sqlite3.c(20253) : warning C4018: '>' : signed/unsigned mismatch D:\Ulti\MyApps\USQLite3\SQLite\sqlite3.c(20470) : warning C4018: '<=' : signed/unsigned mismatch D:\Ulti\MyApps\USQLite3\SQLite\sqlite3.c(21671) : warning C4244: 'function' : conversion from 'i64' to 'int', possible loss of data D:\Ulti\MyApps\USQLite3\SQLite\sqlite3.c(21673) : warning C4244: 'function' : conversion from 'i64' to 'u32', possible loss of data D:\Ulti\MyApps\USQLite3\SQLite\sqlite3.c(23335) : warning C4018: '>' : signed/unsigned mismatch D:\Ulti\MyApps\USQLite3\SQLite\sqlite3.c(23335) : warning C4018: '<=' : signed/unsigned mismatch D:\Ulti\MyApps\USQLite3\SQLite\sqlite3.c(23977) : warning C4244: '=' : conversion from 'i64' to 'u32', possible loss of data D:\Ulti\MyApps\USQLite3\SQLite\sqlite3.c(23983) : warning C4018: '>' : signed/unsigned mismatch D:\Ulti\MyApps\USQLite3\SQLite\sqlite3.c(24117) : warning C4244: '=' : conversion from 'i64' to 'u32', possible loss of data D:\Ulti\MyApps\USQLite3\SQLite\sqlite3.c(24124) : warning C4018: '>' : signed/unsigned mismatch D:\Ulti\MyApps\USQLite3\SQLite\sqlite3.c(24437) : warning C4244: 'function' : conversion from 'i64' to 'int', possible loss of data D:\Ulti\MyApps\USQLite3\SQLite\sqlite3.c(24437) : warning C4244: 'function' : conversion from 'i64' to 'int', possible loss of data D:\Ulti\MyApps\USQLite3\SQLite\sqlite3.c(24439) : warning C4244: 'function' : conversion from 'i64' to 'int', possible loss of data D:\Ulti\MyApps\USQLite3\SQLite\sqlite3.c(24441) : warning C4244: 'function' : conversion from 'i64' to 'u32', possible loss of data D:\Ulti\MyApps\USQLite3\SQLite\sqlite3.c(24442) : warning C4244: 'function' : conversion from 'i64' to 'int', possible loss of data D:\Ulti\MyApps\USQLite3\SQLite\sqlite3.c(24442) : warning C4244: 'function' : conversion from 'i64' to 'int', possible loss of data D:\Ulti\MyApps\USQLite3\SQLite\sqlite3.c(24815) : warning C4018: '>' : signed/unsigned mismatch D:\Ulti\MyApps\USQLite3\SQLite\sqlite3.c(24976) : warning C4018: '>' : signed/unsigned mismatch D:\Ulti\MyApps\USQLite3\SQLite\sqlite3.c(25046) : warning C4244: '+=' : conversion from 'i64' to 'int', possible loss of data D:\Ulti\MyApps\USQLite3\SQLite\sqlite3.c(25048) : warning C4244: '=' : conversion from 'i64' to 'int', possible loss of data D:\Ulti\MyApps\USQLite3\SQLite\sqlite3.c(26605) : warning C4018: '>' : signed/unsigned mismatch D:\Ulti\MyApps\USQLite3\SQLite\sqlite3.c(27294) : warning C4244: '+=' : conversion from 'i64' to 'int', possible loss of data D:\Ulti\MyApps\USQLite3\SQLite\sqlite3.c(27821) : warning C4101: 'rc' : unreferenced local variable D:\Ulti\MyApps\USQLite3\SQLite\sqlite3.c(28087) : warning C4244: '=' : conversion from 'double' to 'i64', possible loss of data D:\Ulti\MyApps\USQLite3\SQLite\sqlite3.c(28325) : warning C4244: '=' : conversion from 'const i64' to 'double', possible loss of data D:\Ulti\MyApps\USQLite3\SQLite\sqlite3.c(28330) : warning C4244: '=' : conversion from 'const i64' to 'double', possible loss of data D:\Ulti\MyApps\USQLite3\SQLite\sqlite3.c(30422) : warning C4244: 'return' : conversion from 'i64' to 'u32', possible loss of data D:\Ulti\MyApps\USQLite3\SQLite\sqlite3.c(30479) : warning C4244: '=' : conversion from 'u64' to 'unsigned char', possible loss of data D:\Ulti\MyApps\USQLite3\SQLite\sqlite3.c(30644) : warning C4018: '>=' : signed/unsigned mismatch D:\Ulti\MyApps\USQLite3\SQLite\sqlite3.c(30646) : warning C4018: '>=' : signed/unsigned mismatch D:\Ulti\MyApps\USQLite3\SQLite\sqlite3.c(30671) : warning C4018: '<' : signed/unsigned mismatch D:\Ulti\MyApps\USQLite3\SQLite\sqlite3.c(30673) : warning C4018: '<' : signed/unsigned mismatch D:\Ulti\MyApps\USQLite3\SQLite\sqlite3.c(30717) : warning C4244: 'function' : conversion from 'i64' to 'int', possible loss of data D:\Ulti\MyApps\USQLite3\SQLite\sqlite3.c(30756) : warning C4244: 'function' : conversion from 'i64' to 'int', possible loss of data D:\Ulti\MyApps\USQLite3\SQLite\sqlite3.c(30860) : warning C4244: 'return' : conversion from 'i64' to 'int', possible loss of data D:\Ulti\MyApps\USQLite3\SQLite\sqlite3.c(31019) : warning C4244: '=' : conversion from 'double' to 'i64', possible loss of data D:\Ulti\MyApps\USQLite3\SQLite\sqlite3.c(31056) : warning C4244: '=' : conversion from 'double' to 'u64', possible loss of data D:\Ulti\MyApps\USQLite3\SQLite\sqlite3.c(33652) : warning C4244: '=' : conversion from 'i64' to 'u32', possible loss of data D:\Ulti\MyApps\USQLite3\SQLite\sqlite3.c(33718) : warning C4018: '>=' : signed/unsigned mismatch D:\Ulti\MyApps\USQLite3\SQLite\sqlite3.c(33736) : warning C4018: '<' : signed/unsigned mismatch D:\Ulti\MyApps\USQLite3\SQLite\sqlite3.c(33751) : warning C4018: '<' : signed/unsigned mismatch D:\Ulti\MyApps\USQLite3\SQLite\sqlite3.c(34173) : warning C4244: '=' : conversion from 'i64' to 'int', possible loss of data D:\Ulti\MyApps\USQLite3\SQLite\sqlite3.c(34177) : warning C4244: '=' : conversion from 'i64' to 'u8', possible loss of data D:\Ulti\MyApps\USQLite3\SQLite\sqlite3.c(34279) : warning C4244: '=' : conversion from 'i64' to 'int', possible loss of data D:\Ulti\MyApps\USQLite3\SQLite\sqlite3.c(34297) : warning C4244: '=' : conversion from 'i64' to 'int', possible loss of data D:\Ulti\MyApps\USQLite3\SQLite\sqlite3.c(35189) : warning C4244: '=' : conversion from 'i64' to 'u32', possible loss of data D:\Ulti\MyApps\USQLite3\SQLite\sqlite3.c(35869) : warning C4244: '=' : conversion from 'i64' to 'int', possible loss of data D:\Ulti\MyApps\USQLite3\SQLite\sqlite3.c(35875) : warning C4244: 'function' : conversion from 'i64' to 'int', possible loss of data D:\Ulti\MyApps\USQLite3\SQLite\sqlite3.c(36389) : warning C4244: '=' : conversion from 'i64' to 'int', possible loss of data D:\Ulti\MyApps\USQLite3\SQLite\sqlite3.c(36403) : warning C4244: 'function' : conversion from 'i64' to 'int', possible loss of data D:\Ulti\MyApps\USQLite3\SQLite\sqlite3.c(46396) : warning C4244: 'initializing' : conversion from 'i64' to 'int', possible loss of data D:\Ulti\MyApps\USQLite3\SQLite\sqlite3.c(46397) : warning C4244: 'initializing' : conversion from 'i64' to 'int', possible loss of data D:\Ulti\MyApps\USQLite3\SQLite\sqlite3.c(46398) : warning C4244: 'initializing' : conversion from 'i64' to 'int', possible loss of data D:\Ulti\MyApps\USQLite3\SQLite\sqlite3.c(56943) : warning C4005: 'ARRAYSIZE' : macro redefinition D:\Ulti\SDK\Include\WinNT.h(950) : see previous definition of 'ARRAYSIZE' D:\Ulti\MyApps\USQLite3\SQLite\sqlite3.c(58192) : warning C4244: '=' : conversion from 'u16' to 'unsigned char', possible loss of data USQLite3: 2 file(s) built in (0:04.04), 2022 msecs / file, duration = 4586 msecs
 
2280 new active 2007 Mar anonymous   2007 Mar   4 3 Check constraint failure message should include field name edit
When an insert or update fails due to a check constraint the error message only says that the insert/update failed. It would be nice if the error message included the field name related to the failed check constraint.

Thanks,

Sam

2007-Mar-31 00:53:55 by drh:
Duplicate of ticket #2258
 
930 new active 2004 Sep anonymous Unknown 2007 Mar drh 2 2 djgpp port for sqlite3 edit
hello,

attached is a diff to sqlite 3.0.7 for dos, using free djgpp.

i did it in the spirit of the isolated platform files, and the generic changes (such as creation of os_dependent functions) have been implemented for all platforms available.

as required, the following copyright applies:

The author or authors of this code dedicate any and all copyright interest in this code to the public domain. We make this dedication for the benefit of the public at large and to the detriment of our heirs and successors. We intend this dedication to be an overt act of relinquishment in perpetuity of all present and future rights this code under copyright law.

thanks for sqlite,

and enjoy coding,

alex

p.s. i see that the diff i provided got mangled, and i'm changing it for the original one. since i'm unable to delete the wrong attachment, i would urge you to use the second one.

alex

 
1802 build active 2006 May anonymous Unknown 2007 Mar   4 4 sqlite.pc needs -lrt on Solaris for fdatasync() edit
sqlite 3.3.x correctly checks for fdatasync() in -lrt during its configure, so it will make use of fdatasync() on Solaris.

The problem is that the sqlite.pc doesn't record that -lrt is needed when other packages wish to link with sqlite, so other packages will fail to link using the Libs: entry from sqlite.pc.

I think this could be fixed by just adding

  @TARGET_LIBS@

to the end of the Libs: line in sqlite.pc.in, since TARGET_LIBS should only contain any necessary additional libraries. The later call to AC_SUBST(TARGET_LIBS) will then handle getting any libraries listed in TARGET_LIBS substituted in.

I will submit a patch, if you wish, if you think that's the correct fix.

2007-Mar-30 05:37:44 by anonymous:
#1802, #2041 and #2270 are the same basic issue: librt should get linked into libsqlite3.so
 
2041 code active 2006 Oct anonymous   2007 Mar   4 4 sqlite3.pc on Solaris needs -lrt in "Libs:" entry edit
On Solaris, to get fdatasync(), it's necessary to link with -lrt (the realtime library). I reported that issue a while back, and sqlite's configure has since become smart enough to look for fdatasync() in -lrt, and some of the sqlite Makefile targets know to at $(TLIBS) to the link line, to pick up any addtional libraries (like -lrt) that may be needed.

The problem is that -lrt isn't added to the sqlite3.pc file that's generated, so other applications that use pkg-config and the sqlite3.pc to determine how to link against libsqlite3 don't know that they need to add -lrt.

One fairly easy fix for the problem is to modify sqlite3.pc.in to also include @TARGET_LIBS@ in the Libs: line. That way, if anything is added to TARGET_LIBS by configure, it's automatically substituted into both the Makefile and sqlite3.pc.

Still, I don't understand why libsqlite3 isn't directly linked against -lrt. It's the libsqlite.3.so.* that has the dependency on librt, not the sqlite3 shell, so I don't understand why $(TLIBS) is mentioned for the sqlite3 target instead of the libsqlite3.la target.

I can provide a patch that adds @TARGET_LIBS@ to the Libs line for sqlite3.pc, or I can provide a patch that adds $(TLIBS) to the link line for libsqlite3.la and removes it from sqlite3. I just need to know which the developers prefer.

Thanks!

Tim

2006-Nov-22 15:09:25 by anonymous:
I was having similar problems, but my build failures didn't occur until 'make test'. Unfortunately, <code>@TARGET_LIBS@</code> did not work for me... Thankfully, adding <code>$(TLIBS)</code> to the libsqlite3.la rule in the Makefile did.

Thank you for reporting your fix! You've helped me greatly!

-- Brett


2007-Mar-30 05:37:16 by anonymous:
#1802, #2041 and #2270 are the same basic issue: librt should get linked into libsqlite3.so
 
2270 build active 2007 Mar anonymous   2007 Mar   3 4 Build on Solaris 8 does not include other libs in .so build edit
Straightforward build on solaris: # uname -a SunOS ganga 5.8 Generic_108528-13 sun4u sparc SUNW,Sun-Blade-1000

    # ../sqlite-3.3.13/configure --enable-gcc
    # ../../tcl/8.4.14/bin/tclsh8.4
    % load  ./.libs/libtclsqlite3.so.0.8.6
    couldn't load file "./.libs/libtclsqlite3.so.0.8.6": ld.so.1: ../../tcl/8.4.14/bin/tclsh8.4: fatal: relocation error: file ./.libs/libtclsqlite3.so.0.8.6: symbol fdatasync: referenced symbol not found

Just rerun gcc for the libtclsqlite with the same options as what libtool printed and add the fdatasync library that configure found (which was -lrt):

    % exec gcc all-like-libtool -lrt

    % load  ./.libs/libtclsqlite3.so.0.8.6
    %

I don't know why the build calls the library .0.8.6 - but that's not holding me up - I just rename it to 3.so.3.13

2007-Mar-30 05:36:59 by anonymous:
#1802, #2041 and #2270 are the same basic issue: librt should get linked into libsqlite3.so
 
2136 new active 2007 Jan anonymous   2007 Mar   1 3 sqlite locking does not handle afs; patch included edit
Please see

http://www.mail-archive.com/sqlite-users%40sqlite.org/msg20672.html

2007-Jan-02 03:37:59 by anonymous:
According to this checkin, that code has already been added to SQLite:

http://www.sqlite.org/cvstrac/chngview?cn=3459


2007-Jan-16 00:08:58 by anonymous:
Nope, that's something different.


2007-Mar-30 00:39:05 by anonymous:
You're right. I should have looked at the link more closely.
 
2231 new active 2007 Feb anonymous   2007 Mar   4 4 shell should ignore leading whitespace for meta commands edit
The current shell.c doesn't ignore whitespace before meta commands. This is annoying since it ignores whitespace for sql and other commands and thus prevents indenting from being used. If a script is sent to someone like...

sqlite3 data.db3
  .header on
  create table ...
  .q
They can't just cut and paste the script into sqlite3 because of the whitespace.

Patch is ...

--- ..\sqlite-source-3_3_13\shell.c	2007-02-13 08:08:34.000000000 -0300
+++ shell.c	2007-02-15 04:18:05.726995200 -0300
@@ -934,22 +934,35 @@
   }
   return val;
 }

 /*
+** Determine if an input line begins with "." ignoring whitespace.
+*/
+static int is_meta_command(const char *z){
+    while( isspace((unsigned char)*z) ){ z++; }
+    return *z == '.';
+}
+
+/*
 ** If an input line begins with "." then invoke this routine to
 ** process that line.
 **
 ** Return 1 on error, 2 to exit, and 0 otherwise.
 */
 static int do_meta_command(char *zLine, struct callback_data *p){
-  int i = 1;
+  int i = 0;
   int nArg = 0;
   int n, c;
   int rc = 0;
   char *azArg[50];

+  /* Skip the "." prefix.
+  */
+  while(zLine[i] != '.'){ i++; }
+  ++i;
+
   /* Parse the input line into tokens.
   */
   while( zLine[i] && nArg<ArraySize(azArg) ){
     while( isspace((unsigned char)zLine[i]) ){ i++; }
     if( zLine[i]==0 ) break;
@@ -1560,11 +1573,11 @@
       seenInterrupt = 0;
     }
     lineno++;
     if( p->echoOn ) printf("%s\n", zLine);
     if( (zSql==0 || zSql[0]==0) && _all_whitespace(zLine) ) continue;
-    if( zLine && zLine[0]=='.' && nSql==0 ){
+    if( zLine && is_meta_command(zLine) && nSql==0 ){
       rc = do_meta_command(zLine, p);
       free(zLine);
       if( rc==2 ){
         break;
       }else if( rc ){
2007-Mar-01 21:53:33 by anonymous:
Any possibility of this going in? A simple change. Full patch. Brings meta command whitespace processing to the same functionality as SQL (i.e. ignore leading whitespace). Submitting patches to opensource projects is so frustrating when there is no feedback.
 
2241 code active 2007 Feb scouten   2007 Mar   3 3 pragma integrity_check no longer affects command-line tool's exit code edit
With the 3.3.5 build, if you did the following:

   sqlite3 /path/to/db "PRAGMA integrity_check;"

You seemed get a 1 result back on the command line if the check failed in any way (and 0 if it was "ok").

With 3.3.13, this doesn't seem to be the case, even if you specify the -bail command line parameter. (I also tried "PRAGMA integrity_check=1;", which is all I care about in this case.) I have a database with errors that returns 0 to the calling process (that is, the sqlite command line's main function returns 0).

We would like to see the 3.3.5 behavior return.

2007-Mar-29 18:32:54 by drh:
I pulled versions of SQLite going back to 2006-01-01 and I could not find one where the shell returned a non-zero exit code when PRAGMA integrity_check failed. Are you sure that you do not have a customized version of the 3.3.5 shell?
 
2233 new active 2007 Feb anonymous   2007 Mar   4 3 extend xBestIndex in vtables to carry the values for each contraint edit
It would be helpful if the xBestIndex method in virtual table carried the actual values for each constraint. This would allow clients of this call to more accurately set the estimatedCost of the query. The values are available in xFilter but the estimated cost has already been set and encoded by then.

Thanks...

At the time xBestIndex is called, the values are likely runtime variables or unbound host parameters and are thus unknown.
 
2258 new active 2007 Feb anonymous   2007 Mar   4 3 Include field name in check constraint failure message edit
When a check constraint on a column fails SQLite just responds with "constraint failed". It'd be nice if it included the column name for which constraint failed and possibly the value that failed to pass the constraint.

Thanks,

Sam

 
1778 new active 2006 Apr anonymous Parser 2007 Mar   1 3 stdev does not work, workaround not possible due to missing sqrt edit
It would be so great but it is too hard to use sqlite for any statistical purposes without having either an easy way of calculating standard deviation (stdev) nor any chance to create an ugly workaround because square root (sqrt) is also not working or missing. And even to square (sqr) values are resulting in long terms; but at least this is possible.
Is this because it is a lot of work to calculate the sum of values and sum of values squared at the same time during record traversal? Would this basic and standard function be not SQL conform?

Example:
select count(*), avg(Num), stdev(Num) from Population;

My workaround proposal is to use variance due to missing sqrt function:
select count(*), avg(Num) as mean,
(sum(Num*Num)/(count(*)-1)-sum(Num)*sum(Num)/count(*)/(count(*)-1)) as stdevsquared from Population;
Isn't this looking ugly and error prone? Can you imagine the performance if I have to calculate mean and stdev for 20 or more columns?
2006-Apr-20 14:27:31 by anonymous:
It's pretty simple to add your own custom functions to SQLite. Edit sqlite/src/func.c and take a search for sumStep and sumFinalize for an example.


2006-Apr-20 18:11:06 by anonymous:

  Index: src/func.c
  ===================================================================
  RCS file: /sqlite/sqlite/src/func.c,v
  retrieving revision 1.127
  diff -u -r1.127 func.c
  --- src/func.c        7 Apr 2006 13:26:43 -0000       1.127
  +++ src/func.c        20 Apr 2006 18:10:08 -0000
  @@ -20,7 +20,7 @@
   */
   #include "sqliteInt.h"
   #include <ctype.h>
  -/* #include <math.h> */
  +#include <math.h>
   #include <stdlib.h>
   #include <assert.h>
   #include "vdbeInt.h"
  @@ -114,6 +114,20 @@
   }

   /*
  +** Implementation of the sqrt() function
  +*/
  +static void sqrtFunc(sqlite3_context *context, int argc, sqlite3_value **argv){
  +  double rVal;
  +  assert( argc==1 );
  +  rVal = sqlite3_value_double(argv[0]);
  +  if( rVal<0.0 ) {
  +    sqlite3_result_error(context, "sqrt of negative", -1);
  +    return;
  +  }
  +  sqlite3_result_double(context, sqrt(rVal));
  +}
  +
  +/*
   ** Implementation of the abs() function
   */
   static void absFunc(sqlite3_context *context, int argc, sqlite3_value **argv){
  @@ -995,6 +1009,7 @@
   #ifndef SQLITE_OMIT_UTF16
       { "substr",             3, 0, SQLITE_UTF16LE, 0, sqlite3utf16Substr },
   #endif
  +    { "sqrt",               1, 0, SQLITE_UTF8,    0, sqrtFunc   },
       { "abs",                1, 0, SQLITE_UTF8,    0, absFunc    },
       { "round",              1, 0, SQLITE_UTF8,    0, roundFunc  },
       { "round",              2, 0, SQLITE_UTF8,    0, roundFunc  },


2006-Jun-03 22:30:12 by anonymous:
VAR() and STDEV() aggregate functions that match Excel:

  sqlite> select var(a), stdev(a) from (select 1 a union select 9 a);
  32.0|5.65685424949238

  Index: src/func.c
  ===================================================================
  RCS file: /sqlite/sqlite/src/func.c,v
  retrieving revision 1.128
  diff -u -3 -p -r1.128 func.c
  --- src/func.c        11 May 2006 13:25:39 -0000      1.128
  +++ src/func.c        3 Jun 2006 22:25:46 -0000
  @@ -20,7 +20,7 @@
   */
   #include "sqliteInt.h"
   #include <ctype.h>
  -/* #include <math.h> */
  +#include <math.h>
   #include <stdlib.h>
   #include <assert.h>
   #include "vdbeInt.h"
  @@ -820,6 +820,42 @@ static void test_error(
   }
   #endif /* SQLITE_TEST */

  +typedef struct VarCtx VarCtx;
  +struct VarCtx {
  +  i64 n;
  +  double sumw;
  +  double m;
  +  double t;
  +};
  +static void varStep(sqlite3_context *context, int argc, sqlite3_value **argv){
  +  VarCtx *p;
  +  double xi, q, temp, r;
  +  assert( argc==1 );
  +  p = sqlite3_aggregate_context(context, sizeof(*p));
  +  xi = sqlite3_value_double(argv[0]);
  +  q = xi - p->m;
  +  temp = p->sumw + 1;
  +  r = q / temp;
  +  p->m += r;
  +  p->t += r * p->sumw * q;
  +  p->sumw = temp;
  +  ++p->n;
  +}
  +static void varFinalize(sqlite3_context *context){
  +  VarCtx *p = sqlite3_aggregate_context(context, 0);
  +  if( p && p->n>1 ){
  +    double s2 = p->t * p->n / ((p->n - 1) * p->sumw);
  +    sqlite3_result_double(context, s2);
  +  }
  +}
  +static void stdevFinalize(sqlite3_context *context){
  +  VarCtx *p = sqlite3_aggregate_context(context, 0);
  +  if( p && p->n>1 ){
  +    double s2 = p->t * p->n / ((p->n - 1) * p->sumw);
  +    sqlite3_result_double(context, sqrt(s2));
  +  }
  +}
  +
   /*
   ** An instance of the following structure holds the context of a
   ** sum() or avg() aggregate computation.
  @@ -1026,6 +1062,8 @@ void sqlite3RegisterBuiltinFunctions(sql
       { "max",    1, 2, 1, minmaxStep,   minMaxFinalize },
       { "sum",    1, 0, 0, sumStep,      sumFinalize    },
       { "total",  1, 0, 0, sumStep,      totalFinalize    },
  +    { "var",    1, 0, 0, varStep,      varFinalize    },
  +    { "stdev",  1, 0, 0, varStep,      stdevFinalize  },
       { "avg",    1, 0, 0, sumStep,      avgFinalize    },
       { "count",  0, 0, 0, countStep,    countFinalize  },
       { "count",  1, 0, 0, countStep,    countFinalize  },


2007-Mar-29 08:18:34 by anonymous:
Is it still valid with 3.3.13 that sqrt function is not available from within SQLite? or stdev? Or does there exist anywhere in the www an extension library already (for windows and linux) to enhance SQLite 3.3.13 for mathematical/statistical functions? Or is it the standard that everybody has to develop her/his own extensions for basic functionality? I do not know if this is a bug/lack of function or if it's my fault to find the right way of usage. Please help! From my point of view some few statistical functions would increase the usability and value of the SQLite database very much.
 
2274 todo active 2007 Mar anonymous   2007 Mar drh 5 3 Sqlite segfaults consistently in FTS2 edit
Sqlite segfaults consistently in FTS2

sqlite> delete from mail where subject_='backup failed'; bt

  Program received signal EXC_BAD_ACCESS, Could not access memory.
  Reason: KERN_PROTECTION_FAILURE at address: 0x00000000
  0x00025726 in deleteTerms (v=0x404e20, pTerms=0xbfffe95c,   iRowid=10503) at ./ext/fts2/fts2.c:1418
1418      return string_dup_n(s, strlen(s));
  (gdb) bt
  #0  0x00025726 in deleteTerms (v=0x404e20, pTerms=0xbfffe95c, iRowid=10503) at ./ext/fts2/fts2.c:1418
  #1  0x00028c49 in fulltextUpdate (pVtab=0x404e20, nArg=1, ppArg=0x180e440, pRowid=0xbfffebf8) at ./ext/fts2/fts2.c:3678
  #2  0x000541df in sqlite3VdbeExec (p=0x180de00) at ./src/vdbe.c:4877
  #3  0x00012b46 in sqlite3_step (pStmt=0x180de00) at ./src/vdbeapi.c:236
#4  0x0001c392 in sqlite3_exec (db=0x400180, zSql=0x404850 "delete from mail where subject_='backup failed';", xCallback=0x3902 <callback>, pArg=0xbfffee94, pzErrMsg=0xbfffeddc) at ./src/legacy.c:78
#5  0x00006c86 in process_input (p=0xbfffee94, in=0x0) at ./src/shell.c:1649
#6  0x0000746f in main (argc=2, argv=0xbffff81c) at ./src/shell.c:1979
(gdb)
 
2133 build active 2006 Dec anonymous   2007 Mar   4 4 meta ticket for outstanding autoconf issues edit
Please add any outstanding autoconf issues to this ticket.

Ticket #2082: UNIX: configure script doesn't enable loading of extensions

Ticket #1906: HAVE_GMTIME_R and HAVE_LOCALTIME_R

Ticket #1814: Autoconf support for MacOSX univeral binaries

Ticket #2124: -ldl link issue

Ticket #1966: Add a --disable-readline option to configure

Ticket #2053: fdatasync unresolved symbol on solaris 10

Check-in [3709] : Add a new compile-time macro USE_PREAD64

 
151 new active 2002 Sep anonymous   2007 Mar   5 5 Add 'configure' options for locating readline headers and library edit
Just a feature request on the build system. If 'configure' offered compile-time options for specifying the location for readline's includes and headers, that would make it easier to link in readline support on systems where readline isn't installed in /usr/local.

It is common to see '--with-readline-includes' and '--with-readline-libs' as config options in other software distributions that use readline.

2007-Mar-20 23:11:10 by anonymous:
In the meantime, if you want to link against a readline in an unusual location, configure (it will configure without readline), then edit the resultant Makefile:

# Compiler options needed for programs that use the readline() library.
#
READLINE_FLAGS = -DHAVE_READLINE=1 -I/home/mjs/local/include

# The library that programs using readline() must link against.
#
LIBREADLINE = -L/home/mjs/local-linux/lib -lreadline -lncurses
 
2266 new active 2007 Mar anonymous   2007 Mar anonymous 3 3 Add support for Row_Number() Over edit
Row_Number() Over is a windowing function included in the SQL:2003 standard. I need it to be able to rank order several groups by size, using a single query. I know SQLite does not support SQL:2003, but it would be nice to have at least this one function supported.
2007-Mar-10 10:06:33 by anonymous:
You can easily write your own UDF using SQLite API. Here's an example if you need it right now:

typedef unsigned long long int rowNumberContext;
// aggregate step callback
void rowNumberStep(sqlite3_context *context, int argc, sqlite3_value **argv) {
  // initialize or get aggregate function context
  rowNumberContext *agg_context = (rowNumberContext*)sqlite3_aggregate_context(context, sizeof(rowNumberContext));
  (*agg_context)++;
}

// aggregate final callback
void rowNumberFinal(sqlite3_context *context) {
  sqlite3_result_int64(context, *((rowNumberContext *)sqlite3_aggregate_context(context, sizeof(rowNumberContext))));
}

// then just create function:
sqlite3_create_function(db_handle, "row_number", 0, SQLITE_ANY, null, null, rowNumberStep, rowNumberFinal);

I hope I get it right. This piece of sample code has NOT been tested! Use it at your own risk.


The code above is a good implementation of count(*), but not row_number().

I think Row_Number() is one of the SQL2003 windowing functions. Implementing these would require modifications to the parser, compiler and vdbe layers of sqlite. Not possible using current APIs.

At the current time queries that use Row_Number() will have to be rewritten to use temp tables as intermediate steps.

 
2254 new active 2007 Feb anonymous   2007 Feb   5 4 ATTACH support IF NOT ATTACHED statement edit
It'd be nice if ATTACH supported an IF NOT ATTACHED option as in

ATTACH IF NOT ATTACHED 'C:\db\log.dat' AS Logs

so there'd be no harm in issuing an attach statement multiple times (and no need to query the database list to see if a database is already attached).

 
2247 doc active 2007 Feb anonymous   2007 Feb   3 3 documentation of DEFUALT cluase in CREATE TABLE should be fixed edit
The documentation for the CREATE TABLE statement at http://www.sqlite.org/lang_createtable.html shows the syntax of the DEFAULT clause as

  DEFAULT value

and the description says that value can be NULL, a string constant, a number, or one of three keyword values; CURRENT_DATE, CURRENT_TIME, or CURRENT_TIMESTAMP.

This is incorrect since the DEFAULT clause also allows value to be an expression in brackets. The syntax should be changed to something like

  DEFAULT default_value

  default_value := value | ( expression )

and the description should say that some expressions are allowed. In particular functions can be used as the expression. It should also be clarified when the DEFAULT clause functions are evaluated, at the time the create statement executes, or at the time a record is added to the table.

Note that not all expressions are allowed. In particular the ( select_statement ) form produces an error saying the default value is not constant, even though the apparently non-constant function random() is accepted.

    sqlite> create table t2(id, b default ((select avg(a) from t1)));
    SQL error: default value of column [b] is not constant
    sqlite> create table t2(id, b default (random()));
    sqlite>

Tests using a default function julianday('now') as a default show that the function is evaluated at the time the record is inserted. If that is the case, why can the select expression above not be evaluated each time a record is inserted to generate a default value?

 
2246 doc active 2007 Feb anonymous   2007 Feb   5 4 SQL docs for CREATE TABLE should include FOREIGN KEY syntax edit
The documentation on supported SQL for CREATE TABLE should include syntax for foreign keys since foreign keys are parsed. Of course it should say that the foreign keys are not enforced, but since they are parsed having the syntax in the documentation would be appropriate.

http://sqlite.org/lang_createtable.html

Thanks,

Sam

 
2075 code active 2006 Nov anonymous   2007 Feb   3 3 Improve VACUUM speed and INDEX page locality edit
In testing several 100 Meg - 1 Gig databases (including the Monotone OpenEmbedded database) I found that changing the order of the SQL commands executed by VACUUM to create indexes after table inserts results in 15% faster VACUUM times, and up to 25% faster cold-file-cache queries when indexes are used. This patch effectively makes the pages of each index contiguous in the database file after a VACUUM, as opposed to being scattered throughout the pages of the table related to the index. Your results may vary, but I think this is a very safe change that can potentially boost average database performance.

  Index: src/vacuum.c
  ===================================================================
  RCS file: /sqlite/sqlite/src/vacuum.c,v
  retrieving revision 1.65
  diff -u -3 -p -r1.65 vacuum.c
  --- src/vacuum.c      18 Nov 2006 20:20:22 -0000      1.65
  +++ src/vacuum.c      20 Nov 2006 21:09:27 -0000
  @@ -143,14 +143,6 @@ int sqlite3RunVacuum(char **pzErrMsg, sq
         "   AND rootpage>0"
     );
     if( rc!=SQLITE_OK ) goto end_of_vacuum;
  -  rc = execExecSql(db,
  -      "SELECT 'CREATE INDEX vacuum_db.' || substr(sql,14,100000000)"
  -      "  FROM sqlite_master WHERE sql LIKE 'CREATE INDEX %' ");
  -  if( rc!=SQLITE_OK ) goto end_of_vacuum;
  -  rc = execExecSql(db,
  -      "SELECT 'CREATE UNIQUE INDEX vacuum_db.' || substr(sql,21,100000000) "
  -      "  FROM sqlite_master WHERE sql LIKE 'CREATE UNIQUE INDEX %'");
  -  if( rc!=SQLITE_OK ) goto end_of_vacuum;

     /* Loop through the tables in the main database. For each, do
     ** an "INSERT INTO vacuum_db.xxx SELECT * FROM xxx;" to copy
  @@ -162,10 +154,22 @@ int sqlite3RunVacuum(char **pzErrMsg, sq
         "FROM sqlite_master "
         "WHERE type = 'table' AND name!='sqlite_sequence' "
         "  AND rootpage>0"
  -
     );
     if( rc!=SQLITE_OK ) goto end_of_vacuum;

  +  /* Create indexes after the table inserts so that their pages
  +  ** will be contiguous resulting in (hopefully) fewer disk seeks.
  +  */
  +  rc = execExecSql(db,
  +      "SELECT 'CREATE UNIQUE INDEX vacuum_db.' || substr(sql,21,100000000) "
  +      "  FROM sqlite_master WHERE sql LIKE 'CREATE UNIQUE INDEX %'");
  +  if( rc!=SQLITE_OK ) goto end_of_vacuum;
  +
  +  rc = execExecSql(db,
  +      "SELECT 'CREATE INDEX vacuum_db.' || substr(sql,14,100000000)"
  +      "  FROM sqlite_master WHERE sql LIKE 'CREATE INDEX %' ");
  +  if( rc!=SQLITE_OK ) goto end_of_vacuum;
  +
     /* Copy over the sequence table
     */
     rc = execExecSql(db,
2007-Feb-11 00:49:50 by drh:
My alternative plan is to modify insert.c so that it recognizes the special case of

    INSERT INTO table1 SELECT * FROM table2;

when table1 and table2 have identical schemas, including all the same indices. When this special case is recognized, the generated bytecode will first transfer all table entries from table2 to table1, using a row by row transfer without decoding each row into its constituient columns. Then it will do the same for each index.

There will be two benefits here. First, when the above construct occurs during the course of a VACUUM, the table and each index, including intrisic indices associated with UNIQUE and PRIMARY KEY constraints, will be transferred separately so that all of there pages will be adjacent in the database file. The second benefit will occur when trying to load large quantities of data into an indexed table. Loading indexed data into a very large table is currently slow because the index entries are scattered haphazardly around in the file. But if data is first loaded into a smaller temporary table with the same schema, it can then be transferred to the main table using an INSERT statement such as the above in what amounts to a merge operation.


2007-Feb-11 06:58:36 by anonymous:
There's no question that your proposal will greatly improve VACUUM speed which relies on the "INSERT INTO table1 SELECT * from table2" construct. But would it be possible for you to relax the restriction on having identical indexes for table1 and table2? For that matter it would be nice if table2 could be any subselect or view. Then "REPLACE INTO table1 SELECT ...anything..." could also be optimized. Since you can detect that SQLite is doing a bulk insert anyway, it could generate code to make a temporary staging table with automatically generated identical indexes to table1 which could be periodically merged into table1 and truncated every X rows. X could be either set via pragma or be a function of the size of the page cache. The temporary staging table would be dropped after the bulk INSERT INTO ... SELECT. Every user inserting large volumes of data would have to perform this procedure anyway. Manually recreating all the indexes for a given temporary table to match the original table and performing the looping logic is cumbersome and error-prone. It would be very conveniant if SQLite were to do it on the user's behalf. This scheme could only work if there are no triggers on table1, of course.


2007-Feb-11 09:16:25 by drh:
My initial enhancement does nothing to preclude the more agressive enhancement described by anonymous. In order to avoid subtle bugs, and in view of my limited time available to work on this, I think it best to take the more conservative approach first and defer the more elaborate optimization suggested by anonymous until later.


2007-Feb-11 13:54:34 by anonymous:
It should be possible to identify contiguous blocks of individual "INSERT INTO table1 VALUES(...)" statements to the same table within a large transaction and perform the same proposed optimization as with "INSERT INTO table1 SELECT ...". This would require higher level coordination by the parser. Anytime a read operation (SELECT, UPDATE) occurs on such a table marked for bulk INSERT within the large transaction, its temp staging table could be merged into the INSERT destination table and the staging table truncated. The process could be repeated for the remainder of the transaction.

Such an optimization would be a huge benefit to SQLite users since they would need not know the idiosynchracies of the implementation of "INSERT INTO table1 SELECT ..." in order to have efficient table and index population.

Alternatively, if you wish to avoid the complexity of re-assembling and staging individual INSERT statements, it might be a good opportunity for SQLite to support the multi-row variant of the INSERT command:

  INSERT INTO table1 (a,b,c) VALUES(1,2,3), (4,5,6), (7,8,9);

Which is essentially a transform of:

  CREATE TEMP TABLE table1_staging (a,b,c);
  INSERT INTO table1_staging VALUES(1,2,3);
  INSERT INTO table1_staging VALUES(4,5,6);
  INSERT INTO table1_staging VALUES(7,8,9);
  INSERT INTO table1 SELECT * FROM table1_staging;
  -- TRUNCATE OR DROP table1_staging as necessary

which could use the same bulk staging optimization.


2007-Feb-13 02:42:41 by anonymous:
Any harm in checking in the simple patch above for the 3.3.13 release?


2007-Feb-13 12:51:47 by drh:
I have a much better fix standing by that I intend to check-in as soon as I get 3.3.13 out the door. I don't want this in 3.3.13 for stability reasons.


2007-Feb-18 23:07:08 by anonymous:
Some related analysis and an .import patch using a :memory: staging table with the "INSERT INTO table1 SELECT FROM table2" construct can be found here:

http://www.mail-archive.com/sqlite-users%40sqlite.org/msg22143.html

 
2238 new active 2007 Feb anonymous   2007 Feb   2 3 Streams as datbase edit
Would it be possible to allow the use of streams as a database source?
2007-Feb-18 03:56:46 by anonymous:
You'll have to be more precise in what you mean by that.

SQLite needs to be able to do random access to the database data (ie seek all over the place according to how it is laid out).

It also needs the ability to have a journal file alongside the database which is used when writing to do a rollback, or even for readers to know that a rollback needs to be done.

I am not aware of any 'streams' that meet those criteria.

 
2237 code active 2007 Feb anonymous   2007 Feb   4 4 Test suite regressions using Tcl 8.5 edit
Tcl 8.5, SQLite 3.3.13 CVS

  printf-1.7.6...
  Expected: [Three integers: (1000000) ( f4240) (3641100)]
       Got: [Three integers: ( 1000000) ( f4240) (3641100)]

  printf-1.8.6...
  Expected: [Three integers: (999999999) (3b9ac9ff) (7346544777)]
       Got: [Three integers: ( 999999999) (3b9ac9ff) (7346544777)]

  printf-1.9.7...
  Expected: [Three integers: (     0) (   0x0) (     0)]
       Got: [Three integers: (     0) (     0) (     0)]

  tcl-1.6...
  Expected: [1 {syntax error in expression "x*"}]
       Got: [1 {invalid bareword "x"
  in expression "x*";
  should be "$x" or "{x}" or "x(...)" or ...}]
 
2235 new active 2007 Feb anonymous   2007 Feb   4 3 Missing xml support in FTS2 for the snippet function edit
The snippet function may output invalid characters when used for an xml stream (like xhtml). Characters &, < and > need to be escaped (&amp, &lt; &gt;) in this context. The modification proposed is to add a boolean parameter to the snippet function to disable/enable the XML processing mode ; for example, given :

insert into poem (name, text) values ('test', 'Xml string with special < > & entities') ;

select snippet(poem, '<strong>', '</strong>', '...', 1) from poem where text match 'xml' ;

output should be:

<strong>Xml</strong> string with special &lt; &gt; &amp; entities

This modification does not affect the default behaviour of the snippet function. Patch included.

 
2060 code active 2006 Nov anonymous   2007 Feb   1 1 Table references enclosed in parenthesis become "invisible" edit
Hi,

I'm developing an RDF-based system, which translates queries from SPARQL into SQL. While trying to add support for SQLite (MySQL is already supported) I came across the following problem: when table references in a FROM clause are enclosed in parenthesis, they cannot be referenced from outside the parenthesized expression. For example, given the table definitions

  CREATE TABLE IF NOT EXISTS t1 (a, b);
  CREATE TABLE IF NOT EXISTS t2 (c, d);
  CREATE TABLE IF NOT EXISTS t3 (e, f);

The following queries all fail with "no such column" errors:

  SELECT t1.a, t3.f
  FROM (t1 CROSS JOIN t2 ON t1.b = t2.c) LEFT JOIN t3 ON t2.d = t3.e;

  SELECT t1.a, t3.f
  FROM t1 CROSS JOIN (t2 LEFT JOIN t3 ON t2.d = t3.e) ON t1.b = t2.c;

  SELECT t1.a, t2.d
  FROM (t1), (t2)
  WHERE t1.b = t2.c;

I'm not sure if it is always possible to reformulate the queries in such a way that the extra parenthesis aren't necessary, but I suspect that complex expressions involving joins may require them to achieve the intended semantics. In any case, my system would require large changes to be able to get rid of the parenthesized subjoins, so it would be nice if this problem could be fixed. :-)

2006-Nov-10 03:56:46 by anonymous:
For what it's worth, here's the parse trees of two similar queries ("SELECT t1.a, t2.d FROM t1, t2 WHERE t1.b = t2.c" and "SELECT t1.a, t2.d FROM (t1), (t2) WHERE t1.b = t2.c"), as well as one of the other more complicated join queries previously listed.

  SELECT t1.a, t2.d FROM t1, t2 WHERE t1.b = t2.c;
  Select {
    op: TK_SELECT
    isResolved: 1
    pSrc: {
      a[0]: {
        zName: t1
        iCursor: 0
        colUsed: 0x00000003
        pTab: t1
        jointype: JT_INNER
      }
      a[1]: {
        zName: t2
        iCursor: 1
        colUsed: 0x00000003
        pTab: t2
      }
    }
    pEList: {
      a[0]: {
        pExpr: {
          op: TK_COLUMN
          span: {t1.a}
          affinity: SQLITE_AFF_NONE
          iTable: 0
          iColumn: 0
          pTab: t1
        }
      }
      a[1]: {
        pExpr: {
          op: TK_COLUMN
          span: {t2.d}
          affinity: SQLITE_AFF_NONE
          iTable: 1
          iColumn: 1
          pTab: t2
        }
      }
    }
    pWhere: {
      op: TK_EQ
      span: {t1.b = t2.c}
      pLeft: {
        op: TK_COLUMN
        span: {t1.b}
        affinity: SQLITE_AFF_NONE
        iTable: 0
        iColumn: 1
        pTab: t1
      }
      pRight: {
        op: TK_COLUMN
        span: {t2.c}
        affinity: SQLITE_AFF_NONE
        iTable: 1
        iColumn: 0
        pTab: t2
      }
    }
  }

  SELECT t1.a, t2.d FROM (t1), (t2) WHERE t1.b = t2.c;
  Select {
    op: TK_SELECT
    isResolved: 1
    pSrc: {
      a[0]: {
        zAlias: sqlite_subquery_5C0A10_
        iCursor: 0
        pTab: sqlite_subquery_5C0A10_
        pSelect: {
          op: TK_SELECT
          isResolved: 1
          pSrc: {
            a[0]: {
              zName: t1
              iCursor: 1
              colUsed: 0x00000003
              pTab: t1
            }
          }
          pEList: {
            a[0]: {
              zName: a
              pExpr: {
                op: TK_COLUMN
                token: {a}
                span: {a}
                affinity: SQLITE_AFF_NONE
                iTable: 1
                iColumn: 0
                pTab: t1
              }
            }
            a[1]: {
              zName: b
              pExpr: {
                op: TK_COLUMN
                token: {b}
                span: {b}
                affinity: SQLITE_AFF_NONE
                iTable: 1
                iColumn: 1
                pTab: t1
              }
            }
          }
        }
        jointype: JT_INNER
      }
      a[1]: {
        zAlias: sqlite_subquery_5BE4F0_
        iCursor: 2
        pTab: sqlite_subquery_5BE4F0_
        pSelect: {
          op: TK_SELECT
          isResolved: 1
          pSrc: {
            a[0]: {
              zName: t2
              iCursor: 3
              colUsed: 0x00000003
              pTab: t2
            }
          }
          pEList: {
            a[0]: {
              zName: c
              pExpr: {
                op: TK_COLUMN
                token: {c}
                span: {c}
                affinity: SQLITE_AFF_NONE
                iTable: 3
                iColumn: 0
                pTab: t2
              }
            }
            a[1]: {
              zName: d
              pExpr: {
                op: TK_COLUMN
                token: {d}
                span: {d}
                affinity: SQLITE_AFF_NONE
                iTable: 3
                iColumn: 1
                pTab: t2
              }
            }
          }
        }
      }
    }
    pEList: {
      a[0]: {
        pExpr: {
          op: TK_COLUMN
          span: {t1.a}
          flags: EP_Resolved EP_Error
          iTable: -1
          iColumn: 0
        }
      }
      a[1]: {
        pExpr: {
          op: TK_DOT
          span: {t2.d}
          pLeft: {
            op: TK_ID
            token: {t2}
            span: {t2}
          }
          pRight: {
            op: TK_ID
            token: {d}
            span: {d}
          }
        }
      }
    }
    pWhere: {
      op: TK_EQ
      span: {t1.b = t2.c}
      pLeft: {
        op: TK_DOT
        span: {t1.b}
        pLeft: {
          op: TK_ID
          token: {t1}
          span: {t1}
        }
        pRight: {
          op: TK_ID
          token: {b}
          span: {b}
        }
      }
      pRight: {
        op: TK_DOT
        span: {t2.c}
        pLeft: {
          op: TK_ID
          token: {t2}
          span: {t2}
        }
        pRight: {
          op: TK_ID
          token: {c}
          span: {c}
        }
      }
    }
  }
  SQL error: no such column: t1.a

  SELECT t1.a, t3.f FROM (t1 CROSS JOIN t2 ON t1.b = t2.c) LEFT JOIN t3 ON t2.d = t3.e;
  Select {
    op: TK_SELECT
    isResolved: 1
    pSrc: {
      a[0]: {
        zAlias: sqlite_subquery_5BFA30_
        iCursor: 0
        pTab: sqlite_subquery_5BFA30_
        pSelect: {
          op: TK_SELECT
          isResolved: 1
          pSrc: {
            a[0]: {
              zName: t1
              iCursor: 1
              colUsed: 0x00000003
              pTab: t1
              jointype: JT_INNER JT_CROSS
            }
            a[1]: {
              zName: t2
              iCursor: 2
              colUsed: 0x00000003
              pTab: t2
            }
          }
          pEList: {
            a[0]: {
              zName: a
              pExpr: {
                op: TK_COLUMN
                span: {t1.a}
                affinity: SQLITE_AFF_NONE
                iTable: 1
                iColumn: 0
                pTab: t1
              }
            }
            a[1]: {
              zName: b
              pExpr: {
                op: TK_COLUMN
                span: {t1.b}
                affinity: SQLITE_AFF_NONE
                iTable: 1
                iColumn: 1
                pTab: t1
              }
            }
            a[2]: {
              zName: c
              pExpr: {
                op: TK_COLUMN
                span: {t2.c}
                affinity: SQLITE_AFF_NONE
                iTable: 2
                iColumn: 0
                pTab: t2
              }
            }
            a[3]: {
              zName: d
              pExpr: {
                op: TK_COLUMN
                span: {t2.d}
                affinity: SQLITE_AFF_NONE
                iTable: 2
                iColumn: 1
                pTab: t2
              }
            }
          }
          pWhere: {
            op: TK_EQ
            span: {t1.b = t2.c}
            flags: EP_FromJoin EP_Resolved
            iRightJoinTable: 2
            pLeft: {
              op: TK_COLUMN
              span: {t1.b}
              affinity: SQLITE_AFF_NONE
              flags: EP_FromJoin EP_Resolved
              iTable: 1
              iColumn: 1
              iRightJoinTable: 2
              pTab: t1
            }
            pRight: {
              op: TK_COLUMN
              span: {t2.c}
              affinity: SQLITE_AFF_NONE
              flags: EP_FromJoin EP_Resolved
              iTable: 2
              iColumn: 0
              iRightJoinTable: 2
              pTab: t2
            }
          }
        }
        jointype: JT_LEFT JT_OUTER
      }
      a[1]: {
        zName: t3
        iCursor: 3
        pTab: t3
      }
    }
    pEList: {
      a[0]: {
        pExpr: {
          op: TK_COLUMN
          span: {t1.a}
          flags: EP_Resolved EP_Error
          iTable: -1
          iColumn: 0
        }
      }
      a[1]: {
        pExpr: {
          op: TK_DOT
          span: {t3.f}
          pLeft: {
            op: TK_ID
            token: {t3}
            span: {t3}
          }
          pRight: {
            op: TK_ID
            token: {f}
            span: {f}
          }
        }
      }
    }
    pWhere: {
      op: TK_EQ
      span: {t2.d = t3.e}
      flags: EP_FromJoin
      iRightJoinTable: 3
      pLeft: {
        op: TK_DOT
        span: {t2.d}
        flags: EP_FromJoin
        iRightJoinTable: 3
        pLeft: {
          op: TK_ID
          token: {t2}
          span: {t2}
          flags: EP_FromJoin
          iRightJoinTable: 3
        }
        pRight: {
          op: TK_ID
          token: {d}
          span: {d}
          flags: EP_FromJoin
          iRightJoinTable: 3
        }
      }
      pRight: {
        op: TK_DOT
        span: {t3.e}
        flags: EP_FromJoin
        iRightJoinTable: 3
        pLeft: {
          op: TK_ID
          token: {t3}
          span: {t3}
          flags: EP_FromJoin
          iRightJoinTable: 3
        }
        pRight: {
          op: TK_ID
          token: {e}
          span: {e}
          flags: EP_FromJoin
          iRightJoinTable: 3
        }
      }
    }
  }
  SQL error: no such column: t1.a


2006-Nov-11 18:29:33 by anonymous:
The resolving bug appears to be that unique column names or column aliases are searched across all subqueries, but table names and table aliases are only searched at their current SELECT level only.

With this in mind, here are mechanical workarounds without using column aliases (assumes the column names in all joined tables are unique):

  SELECT a, f FROM (t1 CROSS JOIN t2 ON t1.b = t2.c) LEFT JOIN t3 ON d = e;

  SELECT t1.a, f FROM t1 CROSS JOIN (t2 LEFT JOIN t3 ON t2.d = t3.e) ON t1.b = c;

  SELECT a, d FROM (t1), (t2) WHERE b = c;

And here are mechanical workarounds using column aliases (assumes the column names are not unique between tables):

  SELECT t1.a, t3f FROM t1 CROSS JOIN (select t3.f t3f, t2.c t2c from t2 LEFT JOIN t3 ON t2.d = t3.e) ON t1.b = t2c;

  SELECT t1a, t3.f FROM (select t1.a t1a, t2.d t2d from t1 CROSS JOIN t2 ON t1.b = t2.c) LEFT JOIN t3 ON t2d = t3.e;

  SELECT t1a, t2d FROM (select t1.a t1a, t1.b t1b from t1), (select t2.c t2c, t2.d t2d from t2) WHERE t1b = t2c;

Notice that t3.f in the second query did not require an alias because the table "t3" was part of its immediate SELECT. You could make an alias for every column just in case, I just wanted to highlight the difference.


2007-Feb-13 15:40:31 by anonymous:
Fixing this issue would slow down SELECT parsing and column resolution for all queries (more specifically all prepared statements) due to the recursion required for column resolution. It would be easier to change your SQL code generator to accomodate SQLite. Just make aliases for every table at every subselect level and have the SELECT at any given level only work with the table aliases at that level.
 
2225 code active 2007 Feb scouten   2007 Feb   5 4 Request count of number of inserts and deletes from the free page list edit
As a rough indication of potential fragmentation in the database, we'd like to know how many pages have been added and removed from the free page list.
2007-Feb-10 19:24:13 by drh:
The issue of database fragmentation is also addressed by ticket #2075 (a ticket which I had overlooked prior to today.)
 
2224 new active 2007 Feb scouten   2007 Feb   4 4 Option to have one-bit "journal should exist" flag edit
Per discussion with DRH: Would it be possible to have a one-bit flag in the header page of the DB file that signals that there should be a journal file present. If you attempt to open a database with that flag set, but the journal file is not present, SQLite should fail to open the DB.
2007-Feb-09 13:47:44 by drh:
Here is the issue: An application that uses SQLite for persistence is receiving database corruption reports from the field. The developers believe that the corruption occurs after a power failure or other crash leaves a hot journal file and then the users manually deletes the hot journal thinking that it is some garbage file left over from the crash. If there is a "journal should exist" flag in the database file and no journal is seen, that would indicate that the journal has been deleted or renamed and that the database has been corrupted. If the application can detect this, it might be able to locate the deleted journal in the trashcan and recover from the user error.

Other ideas for addressing this problem:

  • Change the "-journal" extension on the journal files to something like "-do-not-delete".

  • Make the journal a hidden file. (The problem here is that if somebody goes to move the databaes file and the database has a hot journal, they would likely not know to move the journal too since it is not visible.)

  • Change permissions on the journal file so that it is read-only. This doesn't prevent the journal from being deleted by a determined user, but it might at least give them a clue that this is not a file to be deleted without at least due consideration.
 
2226 code active 2007 Feb scouten   2007 Feb   5 4 Report on unused indices (and tables?) edit
After exercising a particular database heavily, it would be nice to know which indices (and possibly tables) have not been used at all.
 
2222 build active 2007 Feb anonymous   2007 Feb   4 4 Type mismatch in fts2.c [5091] when compile as fastcall or stdcall edit
I noticed a problem when compiling with Borland TurboC using either the -pr or -ps switches and I have a fix. The error is in reference to termDataCmp being passed to qsort. By changing the declaration of termDataCmp from:

static int termDataCmp(const void *av, const void *bv){

to:

static int __cdecl termDataCmp(const void *av, const void *bv){

the problem was resolved. I'm not sure this is a bug, but I thought I should report it just in case someone else runs into it.

Kind Regards,

Tom Olson

 
2221 new active 2007 Feb anonymous   2007 Feb drh 3 4 Store blobs using inode-like lookup of pages rather than linked list edit
In a recent conversation, the matter of how BLOBs are stored came up. Currently, each page of BLOB data is in a linked list. By default each page is 1K so a very large BLOB may have many many pages.

The linked list becomes inefficient to find and update BLOBs.

DRH mentioned a thought to move to an inode style of page management for BLOBs. This would require updating the file format.

 
2220 new active 2007 Feb anonymous   2007 Feb drh 2 4 fsck for database files edit
The existing recovery strategies for dealing with a corrupted database are entirely manual and could be improved with a reasonable amount of effort. One possible way to mitigate the issue would be the creation of an fsck recovery mechanism. This would be an improved recovery from the current .dump support.
 
2219 code active 2007 Feb shess   2007 Feb shess 2 2 Creating an fts table in an attached database works wrong. edit
   ATTACH DATABASE 'test2.db' AS two;
   CREATE VIRTUAL TABLE two.t2 USING fts2(content);

will put t2_content, t2_segments, and t2_segdir in database 'main' rather than database 'two'. In some cases everything will appear to work, because the tables will be defaults for that name.

 
2218 code active 2007 Feb anonymous   2007 Feb   3 4 select columns from views with table prefix edit
I have a table and a view on the table, defined like this:

  create table mytable (mycolumn varchar);
  create view myview as select mytable.mycolumn from mytable;

Now

  select "mycolumn" from myview;

does work, but

  select mycolumn from myview;

gives "unknown column"!

 
2215 code active 2007 Feb anonymous   2007 Feb   1 2 error messages in virtual table are not propagated edit
I'm trying to return a customized error message in xBestIndex in my virtual table implementation. Rather than copying my implementation here the problem can be reproduced by changing the fulltextBestIndex method from fts1.

For example:

    /* Decide how to handle an SQL query. */
    static int fulltextBestIndex(sqlite3_vtab *pVTab, sqlite3_index_info *pInfo){
      int i;
      TRACE(("FTS1 BestIndex\n"));

      pVTab->zErrMsg = sqlite3_mprintf ("THIS IS AN ERROR MESSAGE");
      return SQLITE_ERROR;

      for(i=0; i<pInfo->nConstraint; ++i){
        const struct sqlite3_index_constraint *pConstraint;

If I run the following I do not see the error message reported when using the shell.

    $ ./sqlite3
    SQLite version 3.3.12
    Enter ".help" for instructions
    sqlite> create virtual table foo using fts1(name, address);
    sqlite> insert into foo (name, address) values ('amanda', '43 elm avenue');
    sqlite> select * from foo;
    SQL error: SQL logic error or missing database
 
2214 code active 2007 Feb anonymous   2007 Feb   5 4 lemon generates bad error messages for %destructor edit
When using incorrect syntax for %destructor, lemon generates a bad error message. When I wanted to use %default_destructor, but used %destructor instead:

  %destructor { ... }

I got this error message:

  Symbol name missing after 134583560estructor keyword

This is trivially fixed by replacing "Symbol name missing after %destructor keyword" with "Symbol name missing after %%destructor keyword" twice in lemon.c

 
2204 new active 2007 Jan anonymous   2007 Feb   5 3 Stable, documented metadata interface edit
In response to #2203, I'd like to suggest that a documented, stable means be added to SQLite3 by which consumers of the API may reliably query column metadata for a table, including the names of the columns, whether they are nullable or not, their types, and what their default values are. Given that, currently, the only way to get this data is via the undocumented table_info pragma, clients who want this data are at your mercy every time that pragma changes.

Thanks!

2007-Jan-30 00:07:28 by anonymous:
How about implementing the sql-standard information_schema?

I see something similar at http://www.sqlite.org/cvstrac/wiki?p=InformationSchema

The PostgreSQL equivalent: http://www.postgresql.org/docs/current/static/information-schema.html


2007-Jan-31 19:14:48 by anonymous:
Pragma table_info is documented at http://www.sqlite.org/pragma.html#schema


2007-Jan-31 19:31:48 by anonymous:
PRAGMAs are deficient because they cannot be used within SELECT statements or as sub-selects. This severely limits their usefulness in an SQL-only context. You have to use SQLite's API to make use of them.


2007-Feb-03 15:10:36 by anonymous:
Note that I was told by Richard himself that the table_info pragma is not considered a documented interface, and as such is fair game for incompatible changes in point releases (as we saw in 3.3.8). What I'm asking for in this ticket is an interface that is officially sanctioned and documented, and which (barring the occassional bug) can be guaranteed to remain stable (between point releases at the very least).
 
1126 new active 2005 Feb anonymous Unknown 2007 Jan drh 2 3 sqlite 2.8.16 port to djgpp edit
here is a diff to be applied on sqlite 2.8.16 to make it work with djgpp. some of the fixes are needed for general purpose, such as relative path handling, and bypass of history and readline wherever not functional.

dear drh, please apply this patch to mainstream sqlite. waiver of copyright in the patch itself.

best regards,

alex <alexbodn@012.net.il>

2005-Feb-16 14:14:56 by drh:
I applied these patches. But then the regression tests fail under unix. The patches much have broken something. No time to fix it now....


2005-Feb-17 11:46:57 by anonymous:
thanks for your time. i will try to compile on linux and compare results.


2005-Oct-11 14:58:05 by anonymous:
i have to appologize for the long time it took me to get to it. i have found the flaw in the patch that made the fulltest fail on unix.

now, that tests pass, please incorporate the diff in mainstream.

i will fix the ports for sqlite3 too.


2007-Jan-31 01:33:52 by anonymous:
it seems someone has accidentally changed the diff i've provided for an invalid binary file.
 
2203 code active 2007 Jan anonymous   2007 Jan   2 3 table_info pragma "default" column format changed? edit
Beginning with SQLite3 3.3.8, it looks like the format of the 'default' value returned by the table_info pragma has changed. Before, it used to be a bare string:

  dev:~> sqlite3
  SQLite version 3.3.7
  Enter ".help" for instructions
  sqlite> create table testings (a integer primary key, b string default 'Tester', c string default NULL);
  sqlite> pragma table_info(testings);
  0|a|integer|0||1
  1|b|string|0|Tester|0
  2|c|string|0||0

After 3.3.8, the 'defaults' column is now a SQL-quoted string:

  dev:~> sqlite3
  SQLite version 3.3.11
  Enter ".help" for instructions
  sqlite> create table testings (a integer primary key, b string default 'Tester', c string default NULL);
  sqlite> pragma table_info(testings);
  0|a|integer|0||1
  1|b|string|0|'Tester'|0
  2|c|string|0|NULL|0

Now, I think I do prefer the latter, where the default is a SQL-quoted string. However, this seems a rather significant change to make mid-stream, in a minor point release. It broke all Ruby on Rails applications that use sqlite3, for instance, because Rails reads that default value to determine how to default the value of each new record.

Was this intentional? Or is this a bug? I'd love to see this behavior reverted and saved for a release with a more significant release number.

2007-Jan-29 22:01:54 by anonymous:
One of your fellow Railers requested this change: Ticket #2078


2007-Jan-29 22:10:55 by drh:
See also ticket #1919 which might also be an issue here.


2007-Jan-29 22:33:19 by anonymous:
Anonymous, you make it sound as if anyone associated with Rails can make a request of the sqlite3 team and have it be automatically assumed to be sanctioned by the Rails core team. Whoever did the original request did not do so under the umbrella of Rails core. If that change was the one that resulted in this behavior, it most definitely should not have been recommended, and would not have been blessed by any of the core team.

At this point, though, I'm not interested in blame. I just want to see what can be done to make sqlite3 work with Rails again, preferably in a way that is backwards compatible with previous sqlite3 releases.


2007-Jan-30 05:20:56 by anonymous:
I agree the feature should be fixed due to backwards compatability, but Rails should try to accomodate both pragma variants since they are both "in the wild". You could base your decision on the sqlite version string, for instance.


2007-Jan-30 18:39:44 by anonymous:
Just FYI, there's another related ticket at the Rails trac at http://dev.rubyonrails.org/ticket/6523, and a Debian bug report with a patch at http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=397531.
 
2206 new active 2007 Jan anonymous   2007 Jan   5 4 Support for foreign key constraints in virtual tables edit
Please consider supporting parsing of foreign key constraints in the CREATE TABLE SQL text passed to sqlite3_declare_vtab().

Rationale: when a virtual table is used to implement something like a JOIN or VIEW on table(s) of the master SQLite database, PRAGMA foreign_key_list() on the virtual table(s) can provide information about key relationship between the virtual table(s) and the table(s) in the master SQLite database.

I've attached a patch which implements this feature, however I'm unsure about possible side effects.

 
2205 build active 2007 Jan anonymous   2007 Jan anonymous 1 1 Problem while using with tcl on ARM edit
I am using the sqlite-3.3.12. I have compiled this version for ARM and mandrake linux. On PC it is working fine. But on the Hand Held device with tcl, it produce error after creating the database file that "database disk image is malformed" while executing query for creating table. Another problem is that on executing sqlite3 executable on PC it shows version 3.3.11 But on executing sqlite3 executable on hand held it shows version 3.3.12 though these both executables were compiled from same source that is sqlite 3.3.12.
 
2202 new active 2007 Jan anonymous   2007 Jan   5 4 function request: sqlite3_attach() edit
Hello! A request for a new function, if i may:

  int sqlite3_attach( sqlite3 * host, sqlite3 * parasite, char const * dbname );

In case it's not obvious, this would be functionally identical to calling:

  ATTACH parasite as dbname;

from the host database.

Alternately, but probably less useful:

  int sqlite3_attach( sqlite3 * host, char const * parasite, char const * dbname );

to directly attach databases. This second option isn't so useful because we already have this feature via the ATTACH SQL command.

 
2200 new active 2007 Jan anonymous   2007 Jan   5 4 threadsafe status not reported in .pc file edit
Some application based on sqlite (for exampe Tracker indexing and searching tool) could need the threadsafe.

Currently there in no way to know the status of threadsafe of installed sqlite.

It could be good add a "threadsafe" variable in sqlite3.pc like

    libdir=${exec_prefix}/lib
    includedir=${prefix}/include
    threadsafe=yes

that developers could query using:

   $ pkg-config --variable=threadsafe sqlite3
   yes
 
2199 build active 2007 Jan anonymous   2007 Jan   1 1 configure doesn't find libreadline if its in uncommon place edit
configure doesn't find libreadline if its in uncommon place. I suggest to change configure to be able to deal with something like this: --with-readline=/path
 
2198 new active 2007 Jan anonymous   2007 Jan   5 4 API: opening only existing databases edit
It would be useful to enhance the sqlite3_open function so it would take 3 parameters -- the third would control whether the database should exist for function success.

In other words (the 3rd argument):
create_or_access_existing -- works in both cases
do_not_create_new_db -- fails if there is no such database file

It is extremely useful if the user wants only read from the db. In current API she/he gets an delayed error while trying to read the table in database.

 
2183 code active 2007 Jan anonymous   2007 Jan drh 1 1 OMIT_SHARED_CACHE: AV and crash with FTS2 INSERT edit
Given that SQLite is compiled with

  -DSQLITE_ENABLE_FTS2=1
  -DSQLITE_OMIT_SHARED_CACHE=1

the following code crashes after about 273 insertions with

  Access violation: Read of address 0x00000014

at

  btree.c, line 3451: if( pCur->idx>=pPage->nCell ){

Here is the code to reproduce:

  int main(int argc, char* argv[])
  {
     sqlite3_stmt *pStmt;
     int i;

     check( sqlite3_open( "test_fts2.db3", &db) );

     check( sqlite3_exec( db, "CREATE VIRTUAL TABLE FTS USING FTS2 (Content);", 0, 0, 0));
     check( sqlite3_exec( db, "BEGIN TRANSACTION;", 0, 0, 0));

     check( sqlite3_prepare( db, "INSERT INTO FTS (Content) VALUES ('Far out in the uncharted backwaters of the unfashionable end of the Western Spiral arm of the Galaxy lies a small unregarded yellow sun.');", -1, &pStmt, NULL));
     for( i = 1; i < 1000; i++) {
        printf( "%d\n", i);
        check( sqlite3_step( pStmt) );
        check( sqlite3_reset( pStmt) );
     }

     check( sqlite3_exec( db, "COMMIT;", 0, 0, 0));
     check( sqlite3_finalize( pStmt ));
     check( sqlite3_close( db ));

     printf ("Done");
     scanf ("*%s");

    return 0;
  }

Could this be related to ticket #2032?

 
2188 doc active 2007 Jan anonymous   2007 Jan   5 4 Doc bug in src/vdbe.c, should s/P1/P2/ in NotFound edit
  # diff
--- src/vdbe.c  2007-01-10 01:01:14.000000000 +1100
+++ src/vdbe_new.c      2007-01-24 16:34:38.139376872 +1100
@@ -2923,7 +2923,7 @@
 **
 ** The top of the stack holds a blob constructed by MakeRecord.  P1 is
 ** an index.  If no entry exists in P1 that matches the blob then jump
-** to P1.  If an entry does existing, fall through.  The cursor is left
+** to P2.  If an entry does existing, fall through.  The cursor is left
 ** pointing to the entry that matches.  The blob is popped from the stack.
 **
 ** The difference between this operation and Distinct is that
 
2187 todo active 2007 Jan anonymous   2007 Jan   3 4 RAISE in trigger not being caught by CONFLICT clause in calling SQL edit
When an exception is raised within a trigger, the on conflict clause on the calling SQL statement is not being invoked.

Note: The CASE statement in example is to demonstrate the error, a CHECK Constraint would have been more appropriate if this were a real world senario.

Sample SQL Below:


CREATE TABLE abc (a);

CREATE TRIGGER abc_insert BEFORE INSERT ON abc BEGIN
SELECT CASE WHEN (new.a > 2) THEN
RAISE(ABORT, 'error here')
END;
END;

BEGIN;
INSERT INTO abc VALUES (1);

-- This should raise error but not rollback
INSERT INTO abc VALUES (4);

-- This should raise error and rollback
INSERT OR ROLLBACK INTO abc VALUES (4);

-- Check to see if ROLLBACK performed (which it hasn't)
SELECT * FROM abc;

 
2185 new active 2007 Jan anonymous   2007 Jan   5 3 API access to opened database pathname - helpful for virtual tables edit
It would be helpful if there was an api to retrieve the pathname (or :memory:) to the opened database. I am implementing a virtual table and would like to open subsequent virtual table (flat files in the filesystem) in the same location that the DB was opened.
 
2182 code active 2007 Jan anonymous   2007 Jan   4 4 sqlite3BtreeGetMeta does not check the file format edit
sqlite3BtreeGetMeta reads database page 1 directly from the pager layer, skipping all of the format checks in lockBtree. Thus, if the very first query you do against a database is "PRAGMA user_version" and the database isn't valid (in particular, if it is an sqlite 2 database) you will get a garbage result rather than an SQLITE_NOTADB or SQLITE_CORRUPT error.

Demonstration:

  $ touch bug1.db                              # empty file
  $ dd if=/dev/zero of=bug2.db bs=100 count=1  # file header all zeroes
  $ sqlite2 bug3.db 'create table a(b);'       # old-format database

  $ sqlite3 bug1.db 'pragma user_version' ; echo $?
  0
  0
  $ sqlite3 bug2.db 'pragma user_version' ; echo $?
  0
  0
  $ sqlite3 bug3.db 'pragma user_version' ; echo $?
  1795162112
  0

Contrast the sensible behavior if you do a SELECT:

  $ sqlite3 bug1.db 'select * from a'; echo $?
  SQL error: no such table: a
  1
  $ sqlite3 bug2.db 'select * from a'; echo $?
  SQL error: file is encrypted or is not a database
  1
  $ sqlite3 bug3.db 'select * from a'; echo $?
  SQL error: file is encrypted or is not a database
  1
2007-Jan-21 22:05:14 by anonymous:
(Submitter:) Is there a reason why the file format check doesn't happen in sqlite3_open?
 
2181 new active 2007 Jan anonymous   2007 Jan   5 4 Generalize ON CONFLICT to failure trapping for all SQL statements edit
(This is a more general alternative to the feature request in #2180.)

It would be nice if clauses similar to ON CONFLICT were available for all supported statements, to specify error recovery behavior in all cases. For instance, one would like to be able to write things like

SELECT ... FROM foo ON ABSENT IGNORE;

with the effect that if there is no table 'foo', the SELECT simply returns zero rows. I am not sure what the complete set of conditions to recover from would be, but I think that ABSENT (table or column missing), EXISTS (something you're trying to create already exists), and NONEMPTY (to avoid deleting data unintentionally, per #2180) should cover most cases.

 
2180 new active 2007 Jan anonymous   2007 Jan   5 4 feature request: DROP TABLE IF [EXISTS AND] EMPTY edit
It would be useful to have a straightforward way to drop a table only if it contains no rows. Currently it is necessary to do this in application logic, by doing a dummy SELECT to find out if there's any data in the table (e.g. "SELECT 1 FROM table LIMIT 1" - this is the most efficient such query I can find). And of course one has to take care to handle errors in that SELECT (e.g. if the table doesn't exist).

I suggest DROP TABLE IF EMPTY, by analogy with the existing DROP TABLE IF EXISTS. Naturally, one would like to be able to combine the two, to drop an empty table but do nothing if the table exists or isn't empty.

2007-Jan-26 22:59:50 by anonymous:
To comment on that last statement:

"I suggest DROP TABLE IF EMPTY, by analogy with the existing DROP TABLE IF EXISTS. Naturally, one would like to be able to combine the two, to drop an empty table but do nothing if the table exists or isn't empty."

That sounds redundant: DROP TABLE IF EMPTY will not drop a table if it has any data, nor if the table does not exist. The question comes to mind, "when is there an error?" Is DROP TABLE IF EMPTY an error if the table is not empty? Or would we need DROP TABLE IF EXISTS AND IF EMPTY to ensure that we have clear success/failure paths in the case that we do DROP TABLE IF EMPTY for a table which we're not sure is there or not. If sounds like "IF EMPTY" should automatically imply "AND IF EXISTS", because a prerequisite of being empty is that the table has to exist.

IMO, a statement like DROP TABLE IF EMPTY is 100% sugar and should not produce any error code, similarly to DROP TABLE IF EXISTS (which does not produce an error if the table DOES exist, though we could rightfully argue that it should raise an error because its condition is not met).

----- sgb

 
2167 new active 2007 Jan anonymous   2007 Jan   1 3 add sqlite3_copy_bindings (parallel to sqlite3_transfer_bindings) edit
sqlite3_transfer_bindings( from, to ) does not leave the 'from' stmt in a usable mode. If I want to create a separate, independent copy of an sqlite3_stmt, I have to replicate the bindings.

I have created a modified version of sqlite3_transfer_bindings() which simply replaced sqlite3VdbeMemMove() with sqlite3VdbeMemCopy(), and named the function sqlite3_copy_bindings. I'm not an expert in Sqlite internals so I can't tell if there any issues with this.

 
2165 code active 2007 Jan anonymous   2007 Jan drh 4 3 pager performance and checksum edit
pager.c

Embedd the 2 byte pager_pagehash() result into the page, near the beginning of the page. Use the intire page to calculate the pager_pagehash exclusive of the two byte page_hash data embedded in the page. That way a simple xor as in CHECK_PAGE of the entire page including the 2 byte pager_pagehash is all that is needed to validate a page. Also you could include the "4 byte" random at the beginning and at the end... But that would be a bit of overkill. The sampling of only every 200 bytes is interesting.

Review change the SQLITE_CHECK_PAGES ifdef to a SQLITE_OMIT_PAGE_CHECK.. As on disk page validity is very important. Could this be integrated into a pragma setting?

2007-Jan-12 18:08:49 by anonymous:
uint16_t pg_chkval(Dpage *pg, uint32_t pg_size) { register int i; register uint16_t val = 0; register uint16_t *bw = (uint16_t *) pg;

  for(i= 0; i < pg_size;i=i+2  )
      val= val^ *bw++  ;

  return val;
}

uint16_t pg_calcval(Dpage *pg, uint32_t page_size) { int i; register uint16_t val = 0; register uint16_t *bw = (uint16_t *) pg;

  /* Scan up to location where chk val is stored */
  for(i= 0; i < 8;i=i+2  )
       val= val^ *bw++  ;

  val = val^ 0;
  bw ++ ;

  /* Now scan the tail of the block */
  for(i=10; i < page_size;i=i+2 )
      val= val^ *bw++  ;

  return val;
}
 
1722 new active 2006 Mar anonymous Unknown 2007 Jan   4 2 agregate sum() of strings edit
i'd like to have something like sum() i agregate functions but to work with strings. I'd like to that that function would concate strings similar to summing in sum()e.g: SELECT sum(name || ',')FROM names GROUP BY ..... etc... :) I've heard that something like this is in postgresql?
 
2148 doc active 2007 Jan anonymous   2007 Jan   5 5 3.3.9 changes.html addition edit
  - Fixed the ".dump" command in the command-line shell to show triggers and views again.
  + Fixed the ".dump" command in the command-line shell to show indexes, triggers and views again.
 
2140 code active 2007 Jan anonymous   2007 Jan   3 1 sqlite doesn't link to readline edit
sqlite relies on another library to link to libreadline, causing this error with LDFLAGS=-Wl,--as-needed:

gcc -O2 -march=i686 -pipe -DOS_UNIX=1 -DHAVE_USLEEP=1 -DHAVE_FDATASYNC=1 -I. -I./src -DNDEBUG -DTHREADSAFE=1 -DSQLITE_THREAD_OVERRIDE_LOCK=-1 -DSQLITE_OMIT_CURSOR -DHAVE_READLINE=1 -I/usr/include/readline -o .libs/sqlite3 ./src/shell.c ./.libs/libsqlite3.so -lpthread -lncurses /tmp/cclOD1M7.o: In function `process_input': shell.c:(.text+0x37a5): undefined reference to `readline' shell.c:(.text+0x37c0): undefined reference to `add_history' /tmp/cclOD1M7.o: In function `main': shell.c:(.text+0x3f01): undefined reference to `read_history' shell.c:(.text+0x3f1a): undefined reference to `stifle_history' shell.c:(.text+0x3f22): undefined reference to `write_history' collect2: ld returned 1 exit status

 
931 new active 2004 Sep anonymous   2006 Dec   5 4 Makefile for mingw on Win32, makes DLL, EXE and docs edit
This is a more compact and easier to understand makefile which will work out-of-the box (well, almost) for people using mingw32 on Windows platforms.

MSYS is not needed, but it does require gawk, sed and grep (well, and GNU make too).

2005-Aug-27 01:07:30 by anonymous:
Has anybody compiled successfully sqlite3 3.2.x with mingw? how?


2005-Aug-27 02:13:20 by anonymous:
From version 2.8.14 to 3.2.4 I have no problems.


2005-Aug-27 12:24:58 by anonymous:
1. installed msys -c:\msys
2. installed mingw - c:\msys\mingw
3. installed msys toolkit and mingw make. see http://mingw.org/download.shtml
4. run msys.
5. go to sqlite directory.
6. ./configure
7. make

So easy so I can't tell more.


2005-Aug-27 13:14:58 by drh:
The precompiled windows binaries on the SQLite website have always been generated using mingw running as a cross-compiler on a linux host.


2005-Aug-27 15:14:09 by anonymous:
Ok, I got it working. There is just one discrepancy - I end up with the following:

291840 sqlite3.dll
1280564 sqlite3.exe

Whereas the downloadable "precompiled" binaries are just:

248320 sqlite3.dll
330203 sqlite3.exe

Is there anything else I need to do to get the exact same byte-size? (this is 3.2.4)


2005-Aug-27 15:18:55 by anonymous:
Is it possible that you're compiling with readline support (which isn't present in the distributed binary due to licensing issues)?


2005-Aug-27 15:27:43 by anonymous:
I do not know how to enable/disable readline support. I have not changed or added any switch to the above. Besides, I think the .dll does not use readline.


2005-Dec-27 10:55:55 by anonymous:
see also BuildOnWindowsWithoutTcl


2006-Dec-28 19:18:55 by anonymous:
Does this also generate a .a for use in applications? Still needing the .dll that is. I don't need sqlite.exe at all...


2006-Dec-28 19:51:44 by anonymous:
I assume you mean .a for MinGW or Cygwin. Using Cygwin or MinGW/MSYS just download and untar the sqlite3 source and issue these commands:

  cd sqlite
  ./configure
  make sqlite3.dll

The .a file will be located in sqlite/.libs/libsqlite3.a

 
2132 code active 2006 Dec anonymous   2006 Dec   3 4 table_info, index_list, index_info & friends should be virtual tables edit
The following meta data should be accessible via read-only virtual tables in some sort of sqlite_meta_info table:

  PRAGMA database_list;
  PRAGMA foreign_key_list(table-name);
  PRAGMA index_info(index-name);
  PRAGMA index_list(table-name);
  PRAGMA table_info(table-name);

This would allow complex querying and joins of the meta data of the database. (Pragmas cannot be used as a subquery).

The underlying introspection mechanism can actually be PRAGMA-based for backwards compatability.

 
2131 code active 2006 Dec anonymous   2006 Dec   2 3 Add substring() function (Part of SQL 99) edit
sqlite> SELECT substring('foobar.class',-6,6) = '.class'; SQL error: no such function: substring sqlite> SELECT SUBSTRING('foobar.class',-6,6) = '.class'; SQL error: no such function: SUBSTRING sqlite> SELECT SUBSTR('foobar.class',-6,6) = '.class'; 1 sqlite> SELECT substring('foobar.class' FROM -6 FOR 6) = '.class'; SQL error: near "FROM": syntax error

Looking at: http://www.oreilly.com/catalog/sqlnut/chapter/ch04.html

SQL99 Syntax

SUBSTRING(extraction_string FROM starting_position [FOR length] [COLLATE collation_name])

It would be useful for sqlite to support this syntax too to make the SQL more portable.

2006-Dec-28 16:03:03 by anonymous:
sqlite has the substr() routine (func.c code):

    { "substr",             3, 0, SQLITE_UTF8,    0, substrFunc },
#ifndef SQLITE_OMIT_UTF16
    { "substr",             3, 0, SQLITE_UTF16LE, 0, sqlite3utf16Substr },
#endif
this could be 'aliased' to help you using the substring() SQL99 std just doing:

    { "substr",             3, 0, SQLITE_UTF8,    0, substrFunc },
    { "substring",          3, 0, SQLITE_UTF8,    0, substrFunc },
#ifndef SQLITE_OMIT_UTF16
    { "substr",             3, 0, SQLITE_UTF16LE, 0, sqlite3utf16Substr },
    { "substring",          3, 0, SQLITE_UTF16LE, 0, sqlite3utf16Substr },
#endif
 
2130 code active 2006 Dec anonymous   2006 Dec   4 4 replace "long long int" type with "sqlite_int64" defined in sqlite3.h edit
As the typedef "typedef long long int sqlite_int64" is available in sqlite3.h. Please ensure that remaining references to "long long" use this typedef (for 32bit compilers):

Searching for 'long long int'... sqlite3.h(89): typedef long long int sqlite_int64;

sqlite3.h(90): typedef unsigned long long int sqlite_uint64;

os_common.h(67):__inline__ unsigned long long int hwtime(void){

os_common.h(68): unsigned long long int x;

os_common.h(74):static unsigned long long int g_start;

sqliteInt.h(185):** cc '-DUINTPTR_TYPE=long long int'

vdbe.c(365):__inline__ unsigned long long int hwtime(void){

vdbe.c(366): unsigned long long int x; 8 occurrence(s) have been found.

thanks

 
2125 code new 2006 Dec anonymous   2006 Dec   4 4 SQLite strings/blobs limited to 1GB/2GB due to 'int' in api edit
In several places it is claimed that SQLite doesn't have arbitrary limits - eg http://www.tcl.tk/community/tcl2004/Papers/D.RichardHipp/drh.html

However the size of strings and blobs is limited due to using int in the APIs. Since int is 32 bit even on 64 bit platforms, that limits the size of individual strings to 2GB. For strings with a UTF16 database that means 1 billion characters. UTF8 strings are anywhere from 2 billion characters to 250 million worst case.

When integrating SQLite into 64 bit code, I have to verify the sizes of all strings and blobs being passed in to ensure they don't overflow the signed 32 bit int.

It would be nice if SQLite supported larger objects on 64 bit platforms. There may also be internal problems with the use of int everywhere. For example I could store a 1.5 billion byte UTF8 string and then request it back via the UTF16 api.

2006-Dec-21 23:47:22 by drh:
On a 64-bit platform "int" is 64-bits. So apparently you can store blobs and strings larger than 2GiB. But SQLite is not designed for this and you would do much better to store such massive objects in a separate file then store the filename in the database.


2006-Dec-22 08:37:46 by anonymous:
On a 64-bit platform "int" is 64-bits

This is not true for all platforms. On MS Platforms: "int" is always 32 bit.


2006-Dec-23 11:55:45 by anonymous:
I don't know of any 64 bit platform where int is 64 bits (ie ILP64). Here are the sizes on x64 Linux with gcc:

sizeof(int) = 4 sizeof(short) = 2 sizeof(long) = 8 sizeof(long long) = 8 sizeof(void*) = 8

There is a good list of what is supported on which platforms on Wikipedia. http://en.wikipedia.org/wiki/64-bit#64-bit_data_models Note "Most 64 bit compilers today use the LP64 model (including Solaris, AIX, HP, Linux, MacOS native compilers), Microsoft however decided to use the LLP64 model."

Note that this means it is impossible to use sizes larger than 2GB with SQLite using int in the API. And while I agree that SQLite isn't the ideal solution for this much data, you do claim no arbitrary limits! Additionally 64 bit code has to ensure that it never gives more than a signed int worth of data. I also suspect that wierd things would happen if too large values are given.

The usual way this is solved in other libraries is to use the size_t and ssize_t types which will always be sized appropriately for the platform. You could even define your own size types such as sqlite3_size_t (that is what Python did).


2006-Dec-25 12:25:57 by anonymous:
Actually I can categorically prove that the current SQLite code is broken and that bad things will happen. The front page of the site says "Sizes of strings and BLOBs limited only by available memory."

Using a simple example, I looked at the implementation of sqlite3_value_text which takes an int. Lets assume -1 is passed in. That calls bindText which calls sqlite3VdbeMemSetStr. That ultimately executes this code:

      if( n<0 ){
        pMem->n = strlen(z);

pMem->n is of integer type, ie signed 32 bits on all current platforms (32 bit and 64 bit). The memory being pointed to could be more than 2GB on a 64 bit platform. strlen's return type is size_t which is 64 bits on 64 bit platforms. Consequently the strlen returned will have the top 32 bits ignore in the code above and SQLite will store a negative number or a number smaller than the actual length of the string.

This means that data will be lost (positive truncation) or that wierd things will happen (negative truncation, several assert failures).

Note that there is the potential for problems even on 32 bit machines. For example a 1.1GB UTF8 string can be supplied and then be requested in UTF16 which will be 2.2GB and make the ints go negative.

I suggest one or more of the following fixes:

  • Remove the claim on the front page about only being limited by memory since this isn't true
  • Document that strings larger than 500MB (UTF8) should not be used and will result in undefined behaviour (worst case UTF8 to UTF16 expansion could be 4 bytes?)
  • Document that blobs larger than 2GB should not be used and will result in undefined behaviour
  • Switch to using size_t and ssize_t in the APIs. If using a signed type (eg ssize_t) for the string APIs then the 500MB limit still remains on 32 bit machines.
 
2128 code active 2006 Dec anonymous   2006 Dec   4 3 virtual table code doesn't verify type of rowid (calling xUpdate) edit
The virtual tables code doesn't verify the type of rowid when calling update. For example I used the following query:

UPDATE foo SET rowid='a string' WHERE 1

This results in a call to xUpdate with argv[0] equal the current rowid but argv[1] is 'a string'. While I'd be quite happy for rowids to be any SQLite type, the xRowid call only allows 64 bit integers. I believe SQLite should check the new rowid in a case like this is an integer and reject it, rather than calling xUpdate with the bad value. (I also just checked with rowid=3.4 and rowid=NULL and they get passed through as is as well)

A workaround is to document that the xUpdate method must check the new rowid is an integer type.

 
2127 code active 2006 Dec anonymous   2006 Dec   2 3 Virtual tables do not always free zErrmsg edit
The documentation for virtual tables and in particular the sqlite3_vtab structure says "The SQLite core will free and zero the content of zErrMsg when it delivers the error message text to the client application or when it destroys the virtual table."

The latter part does not appear to be true ("when it destroys the virtual table"). I can't find any code that does actually that. (eg vtab.c:496 definitely doesn't, nor does vtab.c:76)

Usually the former case happens. However some operations have their error codes ignored (eg xClose). This can result in the zErrMsg pointing to a message but no error code returned upstream (which would clear the message). Finally as far as I can tell the responsibility for freeing sqlite3_vtab is with the xDisconnect/xDestroy callbacks since the corresponding xCreate/xConnect callbacks allocated it. Consequently there is no way for SQLite to even access zErrmsg since it would be a member of a freed structure after xDisconnect/xDestroy returned.

 
2126 code active 2006 Dec anonymous   2006 Dec   3 1 Update hook not invoked when deleteing all rows from table edit
I was testing the update hook feature of SQLite and incidentally I noticed that hook is not invoked for "DELETE FROM" statement with no WHERE clause. Hook works well for "DELETE FROM ... WHERE ..." statement.

Steps to reproduce:

  1. Open database, setup update hook
  2. Execute:
    CREATE TABLE Test(Test INTEGER);
  3. Insert some data:
    INSERT INTO TEST (Test) VALUES (1); -- update hook invoked for INSERT
    INSERT INTO TEST (Test) VALUES (2); -- update hook invoked for INSERT
    INSERT INTO TEST (Test) VALUES (3); -- update hook invoked for INSERT
  4. Execute:
    DELETE FROM TEST; -- update hook IS NOT INVOKED(!) for each row.
2006-Dec-22 15:38:44 by drh:
Triggers don't work either. This is a feature not a bug. When you do "DELETE FROM table" with no WHERE clause, SQLite drops and recreates the table. Doing it this way have a huge speed boost.

If you really need the update hook to work add a "WHERE 1" to the end of the query.


2006-Dec-22 16:11:11 by anonymous:
I don't really need it, maybe other users. You should update the documentation of sqlite3_update_hook() and triggers mentioning this behaviour.

Another solution is to perform "DELETE FROM" as if it was with WHERE clause, if there is an update hook regitered.

 
2120 new active 2006 Dec anonymous   2006 Dec   5 4 Date Column support edit
1) date column declared as 'date' or 'datestr' or similar. 2) date real type is text, stored like 'yyyy-mm-dd'. 3) sqlite3_column_type() return SQLITE_DATE 4) min, max function return type SQLITE_DATE 5) (optional) new function: sqlite3_column_date() is same as sqlite3_column_text(); 6) modify return type of sqlite3_value_type() (add SQLITE_DATE) 7) (optional) new function sqlite3_value_date() same as sqlite3_value_text().
 
2112 code active 2006 Dec anonymous   2006 Dec   5 4 code compatibility with C++ edit
I need to compile SQLite with C++ compiler (GCC 3.4.5 from MingW). I also compiled the code as single translation unit, by including all relevant source files into single *.cpp file. To make it working I needed to make several dozens of small fixes.

    Problems I found:
  • 1. <vdbeInt.h> does not have guarding macros
  • 2. <os_common.h> does not have guarding macros
  • 3. parser.y used symbol"not" which is reserved in C++ and transformed to "!"
  • 4. pager.c redefines macros TRACE1 ... TRACE5 which are already defined elsewhere. With several files in one TU this clashes.
  • 5. Inner structures and name lookup. With structures like
      struct SrcList {
        ...
        struct SrcList_item {
        ...
    

C++ requires full qualification, e.g.

  struct SrcList::SrcList_item* p = ... // OK
  struct SrcList_item* p = ... // ERROR

  • 6. Code like
    char* p = sqlite3Malloc(...)
    

fails to compile without cast.

  • 7. There are few dozens of places where a signed value is compared with unsigned one and compiler emits a warning.

If some of the points above get eliminated than the porting should be much easier - it was few boring hours of change-try compile-fix cycle for me.

2006-Dec-19 16:42:35 by anonymous:
this software is written in C with a lemon grammar parser (parser.y), and it&#180;s compiled with GCC compiler suite (GCC, not G++). Also, the source code is not C++ compliant, and isn't structured for a single source file, so avoid reporting tickets like this in the trac, use mailing list to ask things, only report errors here.


2006-Dec-19 17:21:15 by anonymous:
It's fine to ask for reasonable features like making the code C++ clean. Just use a lower priority (4 or 5), as the original ticket poster did.
 
2110 build active 2006 Dec anonymous   2006 Dec   4 3 Non-optional linking with readline makes sqlite3 binary GPL edit
Currently, the sqlite3 binary is linked with libreadline support if it happens to be available at compile time. This may not always be desirable, because readline is licensed under GPL, and therefore the sqlite3 binary becomes GPL.

Solution: There ought to be a configure script parameter --disable-readline (or something similar) to allow creating non-GPL binaries.

2006-Dec-17 16:11:21 by anonymous:
Another solution would be to use editline instead, which is BSD licensed, from NetBSD project. Here is an autotool- and libtoolized port of it: <http://www.thrysoee.dk/editline/>.

It seems to even have a readline.h wrapper.

Check the links there for upstream sources and related projects.


2006-Dec-17 16:34:45 by anonymous:
This might work:

  env CFLAGS="-UHAVE_READLINE" ./configure


2006-Dec-17 16:55:55 by anonymous:
The -UHAVE_READLINE thing does not work because it is hardcoded into Makefile.in:

  # Compiler options needed for programs that use the readline() library.
  #
  READLINE_FLAGS = -DHAVE_READLINE=@TARGET_HAVE_READLINE@ @TARGET_READLINE_INC@

  # The library that programs using readline() must link against.
  #
  LIBREADLINE = @TARGET_READLINE_LIBS@

...

  sqlite3$(TEXE): $(TOP)/src/shell.c libsqlite3.la sqlite3.h
        $(LTLINK) $(READLINE_FLAGS) $(LIBPTHREAD) \
                -o $@ $(TOP)/src/shell.c libsqlite3.la \
                $(LIBREADLINE) $(TLIBS)

You have to manually edit the generated Makefile to remove READLINE_FLAGS and LIBREADLINE.

 
1946 code new 2006 Aug anonymous Unknown 2006 Dec   2 2 .read file fails on blob fields with end-of-file char edit
I've a table with a blob fields. I put there binary data that contains 0x1a (end of file) symbol. It's alright until i try to dump table to file and then trying to import that file.

sqlite3 my_db
>.output my_file
>.dump table_with_blob
>.exit
del my_db

sqlite3 my_db
>.read my_file

Fails with "Incomplete SQL: ..." SQL break before 0x1a char

I'm on windows. Possibly solving with opening file as binary file.

Sorry for my English

 
2106 doc active 2006 Dec anonymous   2006 Dec   4 4 FtsOne wiki page neglects to mention Porter stemming edit
http://www.sqlite.org/cvstrac/wiki?p=FtsOne claims that "the module does not perform stemming of any sort" when, in fact, FTS1 (in the 3.3.8 tarball and onwards) appears to fully support Porter stemming, using the same "tokenizer porter" syntax as FTS2.
 
2105 new active 2006 Dec anonymous   2006 Dec   5 5 sqlite3_exec does not return modified rows edit
SQLite does not provide a way to determine wether a statement has succeeded unless it is a select statement. For instance using the following sample database

  create table urls (url text unique)
  insert into urls(url) values('http://www.sqlite.org')
  insert into urls(url) values('http://www.google.com')

The following statement will execute the callback from sqlite3_exec which can be used to count the rows (becuase its deemed a query)

  select * from urls

But the following statement will not execute the callback and so there is no way of knowing wether the statement was succesful or not.

  delete from urls where url='http://news.com'

In fact, it is indistinguisable from the following statement...

  delete from urls where url='http://www.google.com'

Even using prepared statements and the sqlite3_step function does not allow the user to determine the success of a statement or count affected rows. There needs to be some way to determine the number of affected rows as a result of a particular statement, not just queries. One solution would be for sqlite3_exec to call the callback once for each affected row, and that the fields and values parameters be left blank for non queries. Of course, a better solution may be to add a new API call which can be used when SQLITE_DONE is returned to check for affected rows. This would allow the API to actually signal success or failure, and more importantly provide a way of counting the affected rows from a particular statement. Most other database API's provide a way to do this.

After submitting this bug, i saw other bug reports that were similar with the suggestion that sqlite3_changes can be used to determine the number of affected rows. So i would like to change this bug report to simply be a documentation request. Please add a link to sqlite3_changes and a description in the documentation for sqlite3_exec so that it is easier to find.
2006-Dec-11 01:06:07 by anonymous:
There is still one case which i cannot resolve even using the sqlite3_changes api call. That is the following statement.

  select * from urls where url='';

Since this select statement returns no results, the callback is not called, and when i call the sqlite3_changes routine it simply supplies the previous change (since select is not considdered a change). So how then would i determine wether a statement returned no results without knowing in advance wether it was a select? I suggest sqlite3_changes be modified to return 0 when the statement was a select statement (since this also makes sense, there were no changes). This would allow the detection of select statements which return no results.


2006-Dec-11 01:23:07 by anonymous:
A workaround for this problem is to always manually reset the change count before calling sqlite3_exec

  db->nChange = 0;

Unfortunately this is not available to the user, since the sqlite3 structure is not defined in the header. Perhaps executing a statement which is known to always have 0 changes may achieve the same effect until this can be added/fixed.


2006-Dec-11 19:36:38 by anonymous:
You should look at the empty_result_callbacks pragma documentation at http://www.sqlite.org/pragma.html You can use set this to 1 to get a callback even when your query returns no results.

The other pragma you may be interested in is the count_changes pragma. It can cause insert, delete, and update statements to return a single row with the number of changes when using the sqlite3_exec API.

 
2104 build active 2006 Dec anonymous   2006 Dec   2 3 manual link on Mac OS X fails due to common symbol edit
Attempting a manual link on OS X with fts1:

gcc -O -fPIC -dynamiclib -o mylib sqlite-3.3.8/*.o

Results in the error:

ld: common symbols not allowed with MH_DYLIB output format with the -multi_module option fts1.o definition of common _sqlite3_api (size 16) /usr/bin/libtool: internal link edit command failed

This error is described:

http://gcc.gnu.org/ml/gcc/2005-06/msg00199.html

And a fix:

  --- /tmp/sqlite-3.3.8/src/sqlite3ext.h  2006-09-23 21:28:30.000000000 +1000
  +++ sqlite3ext.h       2006-10-09 19:20:09.000000000 +1000
  @@ -276,7 +276,7 @@
  #define sqlite3_overload_function      sqlite3_api->overload_function
  #endif /* SQLITE_CORE */

  -#define SQLITE_EXTENSION_INIT1     const sqlite3_api_routines *sqlite3_api;
  +#define SQLITE_EXTENSION_INIT1     const sqlite3_api_routines *sqlite3_api = 0;
  #define SQLITE_EXTENSION_INIT2(v)  sqlite3_api = v;

  #endif /* _SQLITE3EXT_H_ */
 
2100 code active 2006 Dec anonymous   2006 Dec   1 1 Fixes for SQL lower() and upper() edit
As acknowledged in the documentation, the SQL lower() and upper() functions might not work correctly on UTF-8 characters. This bug might show if a country specific locale is used instead of the standard C locale. Under certain circumstances, SQL lower() or upper() can even corrupt the UTF-8 string into invalid UTF-8 if the tolower() and toupper() C functions convert character values starting from 0x80.

Below I propose implementations of lowerFunc() and upperFunc() which work correctly with UTF-8 characters, regardless of the implementation of the C library tolower() and toupper() functions. If these C functions are implemented to support high ASCII or even Unicode case conversion, the new SQL lower() and upper() will support them as well.

The proposed C implementation applies a technique also found in sqlite3VdbeMemTranslate() in utf.c and makes use of some macros contained in that unit. To avoid duplicating existing code, it could make sense to move lowerFunc() and lowerFunc() to utf.c, just as it has been done with sqlite3utf16Substr().

Finally, here is the code:

  /*
  ** Implementation of the upper() and lower() SQL functions.
  */
  static void upperFunc(sqlite3_context *context, int argc, sqlite3_value **argv){
    const unsigned char *zIn, *zInTerm;
    unsigned char *z, *zOut;
    int c, l;
    if( argc<1 || SQLITE_NULL==sqlite3_value_type(argv[0]) ) return;
    zIn = sqlite3_value_text(argv[0]);
    if( zIn==0 ) return;
    l = sqlite3_value_bytes(argv[0]);
    zInTerm = &zIn[l];
    /* When converting case, the maximum growth results from
    ** translating a 1-byte UTF-8 character to a 4-byte UTF-8 character.
    */
    zOut = sqliteMalloc( l * 4 );
    z = zOut;
     while( zIn<zInTerm ){
       READ_UTF8(zIn, c);
       c = toupper(c);
       WRITE_UTF8(z, c);
     }
    sqlite3_result_text(context, zOut, z-zOut, sqlite3FreeX);
  }

  static void lowerFunc(sqlite3_context *context, int argc, sqlite3_value **argv){
    const unsigned char *zIn, *zInTerm;
    unsigned char *z, *zOut;
    int c, l;
    if( argc<1 || SQLITE_NULL==sqlite3_value_type(argv[0]) ) return;
    zIn = sqlite3_value_text(argv[0]);
    if( zIn==0 ) return;
    l = sqlite3_value_bytes(argv[0]);
    zInTerm = &zIn[l];
    /* When converting case, the maximum growth results from
    ** translating a 1-byte UTF-8 character to a 4-byte UTF-8 character.
    */
    zOut = sqliteMalloc( l * 4 );
    z = zOut;
     while( zIn<zInTerm ){
       READ_UTF8(zIn, c);
       c = tolower(c);
       WRITE_UTF8(z, c);
     }
    sqlite3_result_text(context, zOut, z-zOut, sqlite3FreeX);
  }
2006-Dec-05 19:07:32 by anonymous:
I'm not sure this is an improvement. The argument to the ctype.h functions tolower(int C) and toupper(int C) is only defined when C is an integer in the range `EOF' to `255'. The compiler implementation may return garbage for values outside this range.


2006-Dec-06 16:17:23 by anonymous:
The integer in the range 'EOF' to '255' is exactly the problem with the existing implementation: This range will (and does, in some cases) modify UTF-8 bytes (not a character) starting from 128, which can lead to invalid UTF-8 sequences.

If the result of tolower() is indeed undefined for values greater than 255 in some C-implementations then I agree that adding this check to the new code will probably be a good thing:

  if( c < 256 ){
    c = tolower(c);
  }

If, on the other hand, tolower() just returns the character unchanged for values greater than 255, then this check is not necessary.

I want to emphasize once more that the proposed new SQL lower() and upper() implementations never return invalid UTF-8, whereas the current implementation sometimes does. This is, by itself, already a great improvement.


2006-Dec-07 15:29:07 by anonymous:
This modified patch does better than previous if SQLITE_UNICODE_UPPERLOWERFUNCS is defined at compile time, because it uses libc _wcsupr / _wcslwr routines.

  #ifdef SQLITE_UNICODE_UPPERLOWERFUNCS
    #define WCHAR_T_SIZE		sizeof(wchar_t)
    #if (WCHAR_T_SIZE == 2)
    #define MAXUPPERLOWERCHAR_AVAIL 	0x0000ffff
    #else // (WCHAR_T_SIZE == 4)
    #define MAXUPPERLOWERCHAR_AVAIL 	0x7fffffff
    #endif // (WCHAR_T_SIZE == 2)
    #define TOLOWERSQLFUNC(c) unicode_tolower
    #define TOUPPERSQLFUNC(c) unicode_toupper
    int unicode_tolower(const int c) {
      wchar_t buff [2];
      if (c > MAXUPPERLOWERCHAR_AVAIL)
        return c;
      buff[0] = (wchar_t) c;
      buff[1] = 0;
      _wcslwr(buff);
      return (int) buff[0];
    }
    int unicode_toupper(const int c) {
      wchar_t buff [2];
      if (c > MAXUPPERLOWERCHAR_AVAIL)
        return c;
      buff[0] = (wchar_t) c;
      buff[1] = 0;
      _wcsupr(buff);
      return (int) buff[0];
    }
  #else // SQLITE_UNICODE_UPPERLOWERFUNCS
    #define TOLOWERSQLFUNC(c) (c > 255 ? c : tolower(c))
    #define TOUPPERSQLFUNC(c) (c > 255 ? c : toupper(c))
  #endif // SQLITE_UNICODE_UPPERLOWERFUNCS

  /*
  ** Implementation of the upper() and lower() SQL functions.
  */
  static void upperFunc(sqlite3_context *context, int argc, sqlite3_value **argv){
    const unsigned char *zIn, *zInTerm;
    unsigned char *z, *zOut;
    int c, l;
    if( argc<1 || SQLITE_NULL==sqlite3_value_type(argv[0]) ) return;
    zIn = sqlite3_value_text(argv[0]);
    if( zIn==0 ) return;
    l = sqlite3_value_bytes(argv[0]);
    zInTerm = &zIn[l];
    /* When converting case, the maximum growth results from
    ** translating a 1-byte UTF-8 character to a 4-byte UTF-8 character.
    */
    zOut = sqliteMalloc( l * 4 );
    z = zOut;
     while( zIn<zInTerm ){
       READ_UTF8(zIn, c);
       c = TOUPPERSQLFUNC(c);
       WRITE_UTF8(z, c);
     }
    sqlite3_result_text(context, zOut, z-zOut, sqlite3FreeX);
  }

  static void lowerFunc(sqlite3_context *context, int argc, sqlite3_value **argv){
    const unsigned char *zIn, *zInTerm;
    unsigned char *z, *zOut;
    int c, l;
    if( argc<1 || SQLITE_NULL==sqlite3_value_type(argv[0]) ) return;
    zIn = sqlite3_value_text(argv[0]);
    if( zIn==0 ) return;
    l = sqlite3_value_bytes(argv[0]);
    zInTerm = &zIn[l];
    /* When converting case, the maximum growth results from
    ** translating a 1-byte UTF-8 character to a 4-byte UTF-8 character.
    */
    zOut = sqliteMalloc( l * 4 );
    z = zOut;
     while( zIn<zInTerm ){
       READ_UTF8(zIn, c);
       c = TOLOWERSQLFUNC(c);
       WRITE_UTF8(z, c);
     }
    sqlite3_result_text(context, zOut, z-zOut, sqlite3FreeX);
  }
 
2099 doc active 2006 Dec anonymous   2006 Dec   3 3 virtual table xDestroy/xDisconnect and cleanups edit
It isn't clear what happens if xDisconnect or xDestroy return an error. From the code for sqlite3VtabCallDestroy pTab->pVtab is not set to zero. Does that mean that Destroy/Disconnect can be called again (the table has been dropped so I don't see how)? If not, then the memory is table is memory leaked.

In my Python wrapper I have to decide what to do when the Python code returns an error. Should I free the Python objects when returning the error code, or leave them allocated. If they are left allocated, then will they be leaked or will there be another opportunity to free them?

2006-Dec-24 11:16:56 by anonymous:
From the code, the return code from xDisconnect is completely ignored (vtab.c:62)

xDisconnect is also unconditionally called whereas xDestroy is optionally called (vtab.c:496). If anything I would expect xDisconnect to be optional and xDestroy to be mandatory.

I still can't answer the original question in this ticket. Currently xDisconnect ignores the return value and therefore results in unconditional disconnection. xDestroy is ultimately called in OP_VDestroy in the vdbe but I can't tell what happens if an error is returned.

 
2098 warn active 2006 Dec anonymous   2006 Dec   1 1 php sqlite problem after commit transaction edit
please see php bugs

http://bugs.php.net/bug.php?id=39735&thanks=2

the error: COMMIT TRANSACTION; begin transaction; PRAGMA default_synchronous=NORMAL; PRAGMA auto_vacuum=1; PRAGMA synchronous=NORMAL; PRAGMA default_temp_store=MEMORY; PRAGMA temp_store=MEMORY; PRAGMA case_sensitive_like=0; PRAGMA encoding=\"UTF-8\"; COMMIT TRANSACTION; <-- here php(sqlite) crash

error message: php: ./src/pager.c:1237: syncJournal: Assertion `pPg->needSync==0' failed. Aborted

php 5.1.6

2006-Dec-04 23:41:22 by drh:
The code in question has been essentially unchanged for three years. This is the first report of this bug. Meanwhile, most of the rest of the world has moved on to use SQLite version 3.x.

Your work-around is to issued the pragmas separately, not inside a transaction.

We will keep this bug on file and work on it as time permits. But because the problem appears to be in 3-year-old code, does not lead to data loss or database corruption, does not impact any paid support customers, and because nobody has ever run into it before, this bug will be fixed at a very low priority. Please do not expect it to be fixed anytime soon.

 
2096 code active 2006 Dec anonymous   2006 Dec   3 3 ATTACH DATABASE returns SQLITE_ERROR when database is locked edit
From an email sent to DRH:

I am working on a problem surrounding the inability to ATTACH to a database file. The error text being returned is "database is locked", which should be SQLITE_BUSY, however, the error code being returned by sqlite3_exec is SQLITE_ERROR. Is sqlite3_exec wrong in returning SQLITE_ERROR rather than SQLITE_BUSY? I have some nagging feeling that I determined or read that the attachFunc function does not return a truly-relevant status code, but I can't see why offhand nor can I find any evidence to support that theory. If sqlite3_exec is doing the right thing, however, then the question becomes one of identifying when to retry the ATTACH statement; we're currently keying off SQLITE_BUSY or SQLITE_LOCKED, as appropriate, and I'd rather not be trying to trap errors based on error text.

 
2095 doc active 2006 Dec anonymous   2006 Dec   1 3 xFindFunction documentation incomplete edit
The documentation doesn't say when xFindFunction is called. It is obviously called the first time a function is needed, but is it called every time?

My underlying issue is that the ppArg return value is going to be dynamically allocated and I need to know when it can be freed (I have to free it since it will be a pointer to a reference counted object)

 
2093 code active 2006 Dec anonymous   2006 Dec   2 3 sqlite3_vtab_cursor doesn't have errMsg edit
The sqlite3_vtab_cursor structure doesn't have a zErrMsg pointer. Only the containing vtable does. This means that operations on cursor objects that have an error have to set the error on the vtable not the cursor. Unfortunately this means that there are race conditions since two different cursors on the same vtable could have errors at the same time. If the cursors are in different threads then a crash or worse can happen.
 
2090 build active 2006 Nov anonymous   2006 Nov   4 3 Test corrupt2.test fails: Solaris edit
While running 'make test', I had come across the following errors:

  ...
corrupt2-1.1... Ok
corrupt2-1.2...
Expected: [1 {file is encrypted or is not a database}]
     Got: [0 {table abc abc 2 {CREATE TABLE abc(a, b, c)}}]
corrupt2-1.3...
Expected: [1 {file is encrypted or is not a database}]
     Got: [0 {table abc abc 2 {CREATE TABLE abc(a, b, c)}}]
corrupt2-1.4...
Expected: [1 {database disk image is malformed}]
     Got: [0 {table abc abc 2 {CREATE TABLE abc(a, b, c)}}]
corrupt2-1.5...
Expected: [1 {database disk image is malformed}]
     Got: [0 {table abc abc 2 {CREATE TABLE abc(a, b, c)}}]
  ...

Turns out that SQLite was working fine, but TCL was not corrupting the database correctly (who would ever have thought I would want to?). Apparently the 'a' mode for opening a file in Solaris was resetting the position of a write to the end of a file before actually writing (this appears to be a point of contention on the TCL bug tracker). From the way the test is written, it appears that portions of the file were to be overwritten, instead of appending to the end of the file.

I will attach a patch to corrupt2.test after posting this message which, instead of attempting an overwrite, writes individual portions of the database file at a time, with requested strings inserted (technically, replacing) into the file at the requested offsets.

2006-Nov-29 23:37:05 by anonymous:
I should mention that this is SunOS 5.8, and TCL version 8.4.14.


2006-Dec-05 10:22:08 by anonymous:
I also get these errors (same tcl version but not SunOS). Have you tried a simpler patch by replacing the 'a' in the open calls by a 'r+'? This solved the problem for me.


2007-Jan-23 19:55:08 by anonymous:
That worked! Thank you so much!
 
2087 new active 2006 Nov anonymous   2006 Nov   3 3 Ability to add a check constraint via the alter table command edit
This is an improvement request.

Add the possibility to add a check constraint to an existing table via the alter table statement.

Example :

  CREATE TABLE a(x INTEGER, y INTEGER);
  INSERT INTO a VALUES (1,2);
  ALTER TABLE a ADD CHECK (y>0);

Actual result :

  SQL error: near "CHECK": syntax error
 
2046 code active 2006 Oct anonymous   2006 Nov shess 1 1 FTS1 - Error closing database due to unfinished statements edit
The following script causes an error in SQLite3.exe with FTS1. The error will surface only AFTER the script has finished AND you have typed .exit at the sqlite> prompt to quit SQLite3.

The problem seems that the SELECT statement is not properly finalized due to an internal error.

  -- The next line is for Windows only, please adopt it
  -- if running Linux or use a FTS1-enabled SQLite3 binary.
  select load_extension ('fts1.dll');

  CREATE TABLE Snippets(
    SnippetID INTEGER PRIMARY KEY,
    SnippetTitle TEXT,
    FtsID INTEGER);

  CREATE VIRTUAL TABLE SnippetsFts USING FTS1 (SnippetTitle, SnippetText);

  INSERT INTO Snippets (SnippetTitle) VALUES ('one');
  INSERT INTO Snippets (SnippetTitle) VALUES ('two');

  SELECT SnippetID FROM Snippets JOIN SnippetsFts ON FtsID = +SnippetsFts.RowID
    WHERE SnippetsFts MATCH 'one';

  -- After the script is done, type .exit at the prompt to close the database.
  --
  -- SQLite3 will close, but report the following error before doing so:
  --
  --   "error closing database: Unable to close due to unfinalised statements"
  --
  --  Does this qualify for a bug?

The script is also attached to this ticket.

2006-Nov-27 22:58:49 by shess:
Attached tighter version of the replication script, generated in isolating what mattered to the bug.
 
2089 code active 2006 Nov anonymous   2006 Nov   3 3 Decouple sqlite_int64 from other 64bit datatypes edit
Currently sqlite3 makes the (valid) assumption that sqlite_int64 (or i64, u64) is 64 bit wide, matches with Tcl_WideInt and has the same datasize (and byte order) than double. The following patch fixes this and allows sqlite_int64 to be any integral type, e.g. a 32bit int (with the limitations of the reduced datatype size).

The use case for this is for systems that do not support 64bit integers (e.g. lack of compiler feature, embedded system), db's of small data size, and systems without large file support. The patch allows compiling with -DSQLITE_INT64_TYPE=int -DSQLITE_32BIT_ROWID for such a system.

2006-Nov-29 01:13:07 by anonymous:
Hm, now I wanted to add the patch file but I don't get the formatting right without editing the file and removing empty lines. How am I supposed to add a patch file (created with diff -ru)?
 
2086 code active 2006 Nov anonymous   2006 Nov   5 4 Alias in update edit
Aliases at UPDATE clause doesn't work, i.e.:

UPDATE table1 t1 SET uid=(SELECT rowid FROM table2 WHERE uid=t1.uid AND data=t1.data);

Code whithout aliases work fine.

 
2084 code active 2006 Nov anonymous   2006 Nov   4 3 Add API function mapping column decl string to SQLite type edit
This is an API feature request.

It would be nice to be able to obtain the SQLite type (e.g. SQLITE_INTEGER) from the declared column type string as returned by sqlite3_column_decltype.

This was discussed briefly on the mailing list here: http://marc.10east.com/?l=sqlite-users&m=116422872301957&w=2

The function I have in mind is:

int sqlite3_decltype_to_type(const char *decl) { Token decl_token; char aff_type; int col_type;

    decl_token.z = decl;
    if( decl_token.z ){
        decl_token.n = strlen(decl_token.z);
        aff_type = sqlite3AffinityType(&decl_token);
        switch( aff_type ){
        case SQLITE_AFF_INTEGER:
            col_type = SQLITE_INTEGER;
            break;
        case SQLITE_AFF_NUMERIC: /* falls through */
        case SQLITE_AFF_REAL:
            col_type = SQLITE_FLOAT;
            break;
        case SQLITE_AFF_TEXT:
            col_type = SQLITE_TEXT;
            break;
        case SQLITE_AFF_NONE:
            col_type = SQLITE_BLOB;
            break;
        default:
            col_type = 0; /* unknown */
            break;
        }
    }
    return col_type;
}

If this seems agreeable, I would be willing to put together a real patch. However, I would need some guidance on where it should go. I'm not sure what should happen when no type can be determined.

2006-Nov-26 22:32:45 by anonymous:
According to the comment above the function sqlite3AffinityType: "If none of the substrings in the above table are found, SQLITE_AFF_NUMERIC is returned". The default condition in sqlite3_decltype_to_type will not be reached.


2006-Nov-26 23:04:23 by anonymous:
Thanks for pointing to that comment. Looks like SQLITE_AFF_NUMERIC is, for these purposes, unknown. So the case statement could be:

        switch( aff_type ){
        case SQLITE_AFF_INTEGER:
            col_type = SQLITE_INTEGER;
            break;
        case SQLITE_AFF_REAL:
            col_type = SQLITE_FLOAT;
            break;
        case SQLITE_AFF_TEXT:
            col_type = SQLITE_TEXT;
            break;
        case SQLITE_AFF_NONE:
            col_type = SQLITE_BLOB;
            break;
        case SQLITE_AFF_NUMERIC: /* falls through */
        default:
            col_type = 0; /* unknown */
            break;
        }


2006-Nov-27 02:43:06 by anonymous:
Your first function was correct, it just had some unreachable code. There's no unknown affinity, in the absence of a match the affinity is assumed to be numeric:

  int sqlite3_decltype_to_type(const char *decl) {
    int type = SQLITE_FLOAT;
    if( decl ){
        Token token;
        token.z = decl;
        token.n = strlen(token.z);
        switch( sqlite3AffinityType(&token) ){
        case SQLITE_AFF_INTEGER:
            type = SQLITE_INTEGER;
            break;
        case SQLITE_AFF_TEXT:
            type = SQLITE_TEXT;
            break;
        case SQLITE_AFF_NONE:
            type = SQLITE_BLOB;
            break;
        default:
            break;
        }
    }
    return type;
  }
 
2083 code active 2006 Nov anonymous   2006 Nov   4 4 Give more detailed extension loading error information with dlerror edit
When using loadable extensions. if dlopen returns an error then SQLite just gives a generic "unable to open shared library" message back. This makes it quite hard to diagnose problems.

I suggest that on Unix platforms you append %s/dlerror() to the message and on Windows append %d/GetLastError()

 
2082 code active 2006 Nov anonymous   2006 Nov   3 4 UNIX: configure script doesn't enable loading of extensions edit
The code in loadext.c:234 looks for HAVE_DLOPEN being #defined in order to enable library loading on Linux. However as best I can tell, the configure script never looks for dlopen. It does look for dlfcn.h. (Based on examining the output of configure and config.log)

Consequently extension loading isn't available on Unixen that do support it if you build using ./configure

Work around is to use this commmand:

   env CFLAGS="-DHAVE_DLOPEN" ./configure
2006-Nov-26 20:53:59 by drh:
The "autoconf" command is busted in SuSE 10.2, which is the OS I am currently running. So I am unable to rebuild configure after editing configure.ac. Until the autoconf problem is resolved, I am unable to address the request in this ticket. Sorry.


2006-Nov-26 22:36:48 by anonymous:
What happens when you upgrade to the latest version of autoconf for SuSE? I'm sure someone on the list could help you resolve this issue.


2006-Nov-27 07:05:27 by anonymous:
I am actually using Gentoo.

There is a trivial workaround as I noted so there is no need for a solution for 3.3.8. It would be nice to have it fixed for whatever version comes next so that I don't need to document the workaround.


2006-Nov-27 18:32:11 by anonymous:
Another open autoconf ticket:

Check-in [3397] : Add HAVE_GMTIME_R and HAVE_LOCALTIME_R flags and use them if defined. Unable to modify the configure script to test for gmtime_r and localtime_r, however, because on my SuSE 10.2 system, autoconf generates a configure script that does not work. Bummer. Ticket #1906

 
2081 code active 2006 Nov anonymous   2006 Nov doughenry 1 1 sqlite3_column_decltype throws exception, if selection is grouped edit
If I "group by" a selection over several columns I can't find out the orgin type of these columns using sqlite3_column_decltype(..). An exception is thrown.
2006-Nov-23 18:37:47 by anonymous:
You also get no decl type from a subselect. This goes to the typeless nature of SQLite - I don't think a type can even be derived in this case.
 
2080 code active 2006 Nov anonymous   2006 Nov   4 4 Tranfering large BLOB data not efficient edit
The current approach for tranfering BLOB data (sqlite3_bind_blob, sqlite3_column_blob) is not efficent for large BLOBs, since the whole BLOB data needs to be kept (multiple times?) in memory.

It would be nice to have (additional) methods for streaming the (large) BLOB data to/from the database. Alternatively we could have methods for transfering the BLOB data in chunks.

Same holds to some extend for large text fields.

2006-Dec-03 09:53:02 by anonymous:
What is your definition of large? (1MB, 100MB, 1GB?)

Note also that SQLite has an upper limit of 2GB on a field due to the use of signed int in the apis which is 32 bit even on 64 bit platforms. That will limit you to 2GB for blobs, 1GB for UTF-16 strings and somewhere inbetween for UTF-8 strings

 
2077 code active 2006 Nov anonymous   2006 Nov   2 1 Problems with using ASCII symbols 0x80 - 0xFF in database path edit
Platform: Windows.

The SQLite library and executable doesn't see database files that are placed into folders named using ASCII symbols with codes 0x80-0xFF. That symbols are used to represent language-specific symbols (for example, Russian). In result, database cannot be placed into folder with name in Russian language. This bug is "unstable": it doesn't appear in all cases. Below are logs from my experiments with this problem. In all cases the path I requested exists, and database file is placed there. I have noticed that problem depends on filename path and name lengths. =========================================================
// creating test database
E:\!DISTRIB\sqlite-3_3_7>sqlite3.exe test.sqb
SQLite version 3.3.7
Enter ".help" for instructions
sqlite> create table a(id int);
sqlite> insert into a values (1);
sqlite> ^C

E:\!DISTRIB\sqlite-3_3_7>copy test.sqb e:\test.sqb
'3'\'`'a'Z'b'`'S'Q'_'` 'f'Q'[']'`'S: 1. //This means that 1 file was copied

E:\!DISTRIB\sqlite-3_3_7>sqlite3 e:\test.sqb
SQLite version 3.3.7
Enter ".help" for instructions
sqlite> select * from a;
1
sqlite> ^C
// Works!

E:\!DISTRIB\sqlite-3_3_7>mkdir e:\'/

//Using ASCII symbol "'/" (0x8D) to represent cyrillic letter which can be entered in the command line by using Alt+(141) combination

E:\!DISTRIB\sqlite-3_3_7>copy test.sqb E:\'/\test.sqb
'3'\'`'a'Z'b'`'S'Q'_'` 'f'Q'[']'`'S: 1.

E:\!DISTRIB\sqlite-3_3_7>sqlite3 e:\'/\test.sqb
SQLite version 3.3.7
Enter ".help" for instructions
sqlite> select * from a;
1
sqlite> ^C
// That is works too!

E:\!DISTRIB\sqlite-3_3_7>mkdir E:\'/\1
E:\!DISTRIB\sqlite-3_3_7>copy test.sqb E:\'/\1\test.sqb
'3'\'`'a'Z'b'`'S'Q'_'` 'f'Q'[']'`'S: 1.
E:\!DISTRIB\sqlite-3_3_7>sqlite3 E:\'/\1\test.sqb
Unable to open database "E:\(T\1\test.sqb": unable to open database file
// Doesn't work, and writes the wrong symbol "(T" in place of "'/"! I've noticed that if we convert symbol "'/" from DOS encoding to Windows encoding and then write it in DOS encoding, then we'll get "(T".

E:\!DISTRIB\sqlite-3_3_7>copy test.sqb E:\'/\tst.sqb
'3'\'`'a'Z'b'`'S'Q'_'` 'f'Q'[']'`'S: 1.

E:\!DISTRIB\sqlite-3_3_7>sqlite3 E:\'/\tst.sqb
SQLite version 3.3.7
Enter ".help" for instructions
sqlite> select * from a;
SQL error: no such table: a
sqlite> ^C

// It seems to work, i don't get an error, but doesn't see the tables! =(
=================================
 
2076 code active 2006 Nov anonymous   2006 Nov a.rottmann 1 1 % exists as value in varchar edit
abnormal abend of client application (C++) when sqlite returns stream of data containing "%" value. Is % a special character?
2006-Nov-21 14:14:25 by anonymous:
% is not a special character. Can you post a small C program demonstrating the problem?
 
2074 code active 2006 Nov anonymous   2006 Nov   4 4 feature request: .dump with page_size edit
It would be useful for sqlite shell users to have a .dump command variant that would cause .dump to output the current database setting of "PRAGMA page_size;". Something similar to:

  sqlite> .dump2
  PRAGMA page_size=16364;
  ...rest of dump...

This way they can trivially preserve the page size when exporting/importing data from/to SQLite:

  sqlite3 old.db .dump2 | sqlite3 new.db

without resorting to non-portable shell gymnastics:

  (echo -n "PRAGMA page_size=" ; sqlite3 old.db "PRAGMA page_size;" ; echo ";" ; ./sqlite3.exe old.db .dump) | sqlite3 new.db

Perhaps other PRAGMA settings could also optionally be exported (legacy_file_format, cache size, etc).

 
2070 code active 2006 Nov anonymous   2006 Nov   4 4 No error for ambiguous result alias in WHERE clause edit
This SELECT should result in an error since 'x' is ambiguous:

  select a x, 2*a x from (select 3 a union select 4 a) where x>3;
  4|8

The current heuristic seems to be the first matching result set expression alias from left-to-right "wins".

2006-Nov-17 19:38:15 by anonymous:
In this test case, the right-most ambiguous expression wins:

  CREATE TABLE t1(a);
  INSERT INTO t1 VALUES(3);
  INSERT INTO t1 VALUES(4);
  INSERT INTO t1 VALUES(5);
  select a*2 a, a from t1 where a>4;
  10|5

It appears that table values take precedence over result set aliases in where clauses.

 
2068 code active 2006 Nov anonymous   2006 Nov   4 4 pkgIndex.tcl contains incomplete version number edit
The pkgIndex.tcl file generated by sqlite 3.3.8 contains the line:

  package ifneeded sqlite3 3.3 ...

Whereas the library actually provides version 3.3.8. In a 8.5 Tcl interpreter, this results in an error message when package require'd:

  attempt to provide package sqlite3 3.3 failed: package sqlite3 3.3.8 provided instead

The solution seems to be to adjust the Makefile.in tcl_install target to pass $(RELEASE) rather than $(VERSION) to the tclinstaller.tcl script.

 
2066 code active 2006 Nov anonymous   2006 Nov   2 2 Incorrect error message in the case of ENOLCK edit
If you're trying to open a sqlite database that is stored on a filesystem that doesn't support locking, then you'll get the error when you try to execute any commands on it:

    Error: file is encrypted or is not a database

If you run sqlite under strace, you see:

   read(0, ".schema\n.quit\n", 4096)       = 14
   fcntl64(3, F_SETLK64, {type=F_RDLCK, whence=SEEK_SET, start=1073741824, len=1}, 0xafa5cd70) = 0
   fcntl64(3, F_SETLK64, {type=F_RDLCK, whence=SEEK_SET, start=1073741826, len=510}, 0xafa5cd70) = 0
   fcntl64(3, F_SETLK64, {type=F_UNLCK, whence=SEEK_SET, start=1073741824, len=1}, 0xafa5cd70) = 0
   access("/mnt/www/zzz_old_sites/trac.db-journal", F_OK) = -1 ENOENT (No such file or directory)
   fstat64(3, {st_mode=S_IFREG|0644, st_size=584704, ...}) = 0
   _llseek(3, 0, [0], SEEK_SET)            = 0
   read(3, "** This file contains an SQLite "..., 1024) = 1024
   fcntl64(3, F_SETLK64, {type=F_UNLCK, whence=SEEK_SET, start=0, len=0}, 0xafa5cdd0) = -1 ENOLCK (No locks available)
   write(2, "Error: file is encrypted or is n"..., 46Error: file is encrypted or is not a database

Sqlite should really check the exact error code, and give a more helpful error (eg "Locking not available on this filesystem. Databases may only be stored on filesystems that support locking")

 
1992 code active 2006 Sep anonymous   2006 Nov shess 1 1 FTS1: Problems after dropping utility tables edit
There are problems if FTS1 utilities tables are dropped from a database. See following SQL for details.

  drop table if exists x;

  -- Create a FTS1 table.

  create virtual table x using fts1 ('content');

  -- Drop table x_content: Works fine, but should this be allowed?
  -- The same errors below also show if table x_term is dropped.

  drop table x_content;

  -- All attempts to access table x now result in errors,
  -- including dropping table x. There seems to be no way out
  -- except of recreating the database. All three commands below
  -- cause the same error, regardless if executed in sequence
  -- or individually:

  insert into x (content) values ('one two three'); -- Error!

  delete from x; -- Error!

  drop table x; -- Error!
Added "not exists" to allow dropping an fts table with corrupted backing. Allowing updates to such tables is unlikely to happen (not even clear what it would mean, in most cases!).
 
2063 code active 2006 Nov anonymous   2006 Nov   4 4 vtab_err.test fails if sqlite is compiled without -DSQLITE_MEMDEBUG edit
I noticed that when running `make fulltest', vtab_err.test fails with an error message like this one (repeated over and over) if sqlite has been compiled without the option -DSQLITE_MEMDEBUG

	vtab_err-2.1...
	Error: invalid command name "sqlite_malloc_fail"

altermalloc.test also has the same "sqlite_malloc_fail" command in it, but it doesn't cause an error because it skips the test if it detects that -DSQLITE_MEMDEBUG isn't available. I'll attach a patch that should fix it. The code is pretty much copied directly from altermalloc.test.

 
2062 code active 2006 Nov anonymous   2006 Nov   4 4 document 'pk' column of PRAGMA table_info() edit
Comment in pragma.c and sqlite.org/pragma.html does not mention the sixth column of PRAGMA table_info().
 
2061 code active 2006 Nov anonymous   2006 Nov anonymous 5 5 cleanup for quickstart.html edit
just compiled the C example from quickstart.html (gcc/glibc Debian SID) a small hint (a bit pea-counting): to avoid warnings either #include<stdlib.h> for the exit() function or (maybe better) use return instead of calling exit().
 
2059 code active 2006 Nov anonymous   2006 Nov   1 1 Still missing .DEF file from Windows 3.3.8 source code distribution edit
The file sqlite3.def is missing from the zip archive of sources used to build sqlite3 on Windows. Ticket number 2031 was closed with a remark that this file is generated during the build process. That is true if one is building on Linux with MinGW32 configured as a cross-compiler. If one were building using that method then I assume one would not be downloading the src.zip archive anyway.

My impression is that the src.zip archive is prepared once the build has been performed on Linux so Windows developers can directly build sqlite (and the generated files) without need of the other tools that the build process depends on. If this is accurate, then it would be very helpful if the src.zip archive could also include the sqlite3.def file. Without this file it is not possible for Windows developers to create a DLL from the src.zip archive.

Thanks

2006-Nov-09 20:05:23 by anonymous:
Works fine as is with MinGW ./configure && make sqlite3.exe
 
2057 code active 2006 Nov anonymous   2006 Nov   3 1 full_column_names when 2 or more tables are joined is not working edit
Version 2.8 has the behavior described in the documentation in respect to full_column_names when 2 or more tables are present with (table/alias).*, but 3.3.8 doesn't, mixing the pragmas "full_column_names" and "short_column_names" can only force to have full_column_names allways or never, some programs expect the behavior described in the documentation to remain working.
2006-Nov-08 20:10:13 by anonymous:
Version 3.3.3 as well has the same problem.


2006-Nov-09 09:34:52 by anonymous:
Changing the line 977 of select.c (3.3.8) from:

if( pEList->a[i].zName){

to:

if( pEList->a[i].zName && pTabList->nSrc==1){

with pragma short_column_names = 0 behaves like 2.8 series.

 
2051 code active 2006 Nov anonymous   2006 Nov   5 5 minor documentation bug edit
On the page http://www.sqlite.org/lang_attach.html you wrote:

If an attached table doesn't have a duplicate table name in the main database, it doesn't require a database name prefix. When a database is attached, all of its tables which don't have duplicate names become the default table of that name. Any tables of that name attached afterwards require the table prefix. If the default table of a given name is detached, then the last table of that name attached becomes the new default.

I think the right form should be: Any tables of that name attached afterwards require the database prefix. Am I right?

Thank you, Dim Zegebart

 
2048 code active 2006 Oct anonymous   2006 Oct drh 1 1 table_info on columns with no default value are returned as string edit
On line 486, noDflt is declared as
static const Token noDflt = { (unsigned char*)"", 0, 0 };

And on line 493:
if( pDflt->z ){
sqlite3VdbeOp3(v, OP_String8, 0, 0, (char*)pDflt->z, pDflt->n);
}else{
sqlite3VdbeAddOp(v, OP_Null, 0, 0);

So columns with no default value aren't being set to null because the (pDflt->z) condition is non-null.
 
2043 code active 2006 Oct anonymous   2006 Oct   1 1 Spaces in view statement edit
If you have a table defined with fields that contain spaces.

create table table1 ("field one", "field two", "field three");

Then you do a select

select "field one" from table1;

That works fine. However if you save it as a view

create view view_one as select "field one" from table1;

Then if you run a select on the view it fails.

select * from view_one;

 
1820 warn active 2006 May anonymous TclLib 2006 Oct anonymous 5 1 make: *** [tclsqlite.lo] Error 1 edit
unding ReHatlinux9.0

$ make .............. .............. ./libtool --mode=compile gcc -g -O2 -DOS_UNIX=1 -DHAVE_USLEEP=1 -DHAVE_FDATASYNC=1 -I. -I../sqlite-3.3.5/src -DNDEBUG -DTHREADSAFE=0 -DSQLITE_THREAD_OVERRIDE_LOCK=-1 -DSQLITE_OMIT_CURSOR -c ../sqlite-3.3.5/src/tclsqlite.c gcc -g -O2 -DOS_UNIX=1 -DHAVE_USLEEP=1 -DHAVE_FDATASYNC=1 -I. -I../sqlite-3.3.5/src -DNDEBUG -DTHREADSAFE=0 -DSQLITE_THREAD_OVERRIDE_LOCK=-1 -DSQLITE_OMIT_CURSOR -c ../sqlite-3.3.5/src/tclsqlite.c -fPIC -DPIC -o .libs/tclsqlite.o ../sqlite-3.3.5/src/tclsqlite.c: In function `DbUpdateHandler': ../sqlite-3.3.5/src/tclsqlite.c:333: warning: passing arg 3 of `Tcl_ListObjAppendElement' makes pointer from integer without a cast ../sqlite-3.3.5/src/tclsqlite.c: In function `tclSqlFunc': ../sqlite-3.3.5/src/tclsqlite.c:419: warning: passing arg 1 of `Tcl_NewByteArrayObj' discards qualifiers from pointer target type ../sqlite-3.3.5/src/tclsqlite.c:427: warning: assignment makes pointer from integer without a cast ../sqlite-3.3.5/src/tclsqlite.c:485: `Tcl_WideInt' undeclared (first use in this function) ../sqlite-3.3.5/src/tclsqlite.c:485: (Each undeclared identifier is reported only once ../sqlite-3.3.5/src/tclsqlite.c:485: for each function it appears in.) ../sqlite-3.3.5/src/tclsqlite.c:485: parse error before "v" ../sqlite-3.3.5/src/tclsqlite.c:486: `v' undeclared (first use in this function) ../sqlite-3.3.5/src/tclsqlite.c: In function `DbObjCmd': ../sqlite-3.3.5/src/tclsqlite.c:685: warning: passing arg 3 of `Tcl_GetIndexFromObj' from incompatible pointer type ../sqlite-3.3.5/src/tclsqlite.c:1309: warning: passing arg 2 of `Tcl_GetVar2Ex' discards qualifiers from pointer target type ../sqlite-3.3.5/src/tclsqlite.c:1331: `Tcl_WideInt' undeclared (first use in this function) ../sqlite-3.3.5/src/tclsqlite.c:1331: parse error before "v" ../sqlite-3.3.5/src/tclsqlite.c:1332: `v' undeclared (first use in this function) ../sqlite-3.3.5/src/tclsqlite.c:1382: warning: passing arg 1 of `Tcl_NewByteArrayObj' discards qualifiers from pointer target type ../sqlite-3.3.5/src/tclsqlite.c:1390: warning: assignment makes pointer from integer without a cast ../sqlite-3.3.5/src/tclsqlite.c:1838: warning: passing arg 3 of `Tcl_GetIndexFromObj' from incompatible pointer type ../sqlite-3.3.5/src/tclsqlite.c: In function `DbMain': ../sqlite-3.3.5/src/tclsqlite.c:2024: warning: passing arg 2 of `Tcl_CreateObjCommand' discards qualifiers from pointer target type make: *** [tclsqlite.lo] Error 1

To solve this error, install ActiveTCL and then: ../sqlite-3.3.8/configure --with-tcl=/usr/local/ActiveTcl/lib make
 
2037 code active 2006 Oct anonymous   2006 Oct   1 1 Sqlite3 can't use datafile in Chinese path with Win2000 and WindowsXP. edit
Sqlite3 can't use datafile in Chinese path with Win2000 and WindowsXP. This is a bug in os_win.c . My firend modify code to so , it work right.

/* ** Convert a UTF-8 string to UTF-32. Space to hold the returned string ** is obtained from sqliteMalloc. */ static WCHAR *utf8ToUnicode(const char *zFilename){ int nChar; WCHAR *zWideFilename;

  if( !isNT() ){
    return 0;
  }
  nChar = MultiByteToWideChar(CP_THREAD_ACP, MB_COMPOSITE, zFilename, -1, NULL, 0);
  zWideFilename = sqliteMalloc( nChar*sizeof(zWideFilename[0]) );
  if( zWideFilename==0 ){
    return 0;
  }
  nChar = MultiByteToWideChar(CP_THREAD_ACP, MB_COMPOSITE, zFilename, -1, zWideFilename, nChar);
  if( nChar==0 ){
    sqliteFree(zWideFilename);
    zWideFilename = 0;
  }
  return zWideFilename;
}

/* ** Convert UTF-32 to UTF-8. Space to hold the returned string is ** obtained from sqliteMalloc(). */ static char *unicodeToUtf8(const WCHAR *zWideFilename){ int nByte; char *zFilename;

  nByte = WideCharToMultiByte(CP_THREAD_ACP, WC_COMPOSITECHECK, zWideFilename, -1, 0, 0, 0, 0);
  zFilename = sqliteMalloc( nByte );
  if( zFilename==0 ){
    return 0;
  }
  nByte = WideCharToMultiByte(CP_THREAD_ACP, WC_COMPOSITECHECK, zWideFilename, -1, zFilename, nByte,
                              0, 0);
  if( nByte == 0 ){
    sqliteFree(zFilename);
    zFilename = 0;
  }
  return zFilename;
}
2006-Oct-20 10:26:46 by anonymous:
The proposed fix is completely wrong, but the bug exists nonetheless. The problem is that SQLite expects file names in UTF-8 encoding (and there is probably bug in your application too guessing from the proposed fix). While this works fine on NT systems where the UTF-8 encoding is converted to UTF-16 and passed to system wide-character APIs, the code path for non-NT systems (Win 9x) with ANSI-only APIs doesn't convert the UTF-8 file names into the ANSI code page which is expected by the system APIs.
 
2032 code active 2006 Oct anonymous   2006 Oct   1 1 AV in btree.c running FTS2 compiled with SQLITE_OMIT_SHARED_CACHE edit
If compiled with FTS2 support as well as SQLITE_OMIT_SHARED_CACHE=1, the sqlite console application causes an

  Access Violation: btree.c, line 3538: Read of address x00000014
    if( pCur->idx>=pPage->nCell ){

if the SQL (attatched) is executed.

I believe that this is a bug in btree.c, for the following reasons:

  • The AV does not show if the #ifndef SQLITE_OMIT_SHARED_CACHE (lines 3514 and 3525) are commentet out.

  • From my reading, all virtual tables use the extension API only and do not access the btree directly.
2006-Oct-25 06:30:43 by shess:
Note that the attached SQL has exactly 273 INSERT statements. 273==256+16+1, so this is kicking in at a merge point. Don't know how that's relevant, but it seems suspicious.


2006-Oct-25 16:31:34 by anonymous:
Many thanks for looking into this - it was driving me mad until I came up with the rather simple SQL to reproduce it.

I am not sure if the number of INSERTS is 100% the number needed to cause the problem, but the crash always happens after the exact same number of inserts. I did not count them but added roughly enough of them to cause the error.

Sidenote: I can also make FTS2 to crash at another point, which I thought was related to the sizeof() bug I also reported. But apprarently it is not. Unfortunately I can not provide a test case for this since I can reproduce it only after adding some 3000 or so copyrighted documents to an empty database. At the time of the crash the DB is about 250 MB in size. However, I will run a test after the next commits to FTS2.


2006-Oct-26 08:57:41 by anonymous:
My previious comments from yesterday seem to be invalidated by the latest checkins [3486] , [3488] and [3489] . Many thanks for those!

However, the problem with SQLITE_OMIT_SHARED_CACHE still persists.

 
2028 code active 2006 Oct anonymous   2006 Oct   4 2 FTS1: UNIQUE() expression and UPDATE command not working edit
I'm working with tables, containing around 1,4 million entries (1GB file size). To allow faster fulltext search I tried FTS1 now.

What I saw is: creating the virtual FTS1 table with one keyword "UNIQUE(code), reference, text, ..." I had the idea to have faster access to "code", because this entry is only one time existing in table. In my actual SQLITE table "UNIQUE" was good idea, because "UPDATE"ing of entries was much faster as without "UNIQUE" expression. Unfortunately, in that moemnt I use "UNIQUE" expression in fulltext table, the FTS1 table doesn't accept insertion of entries like "INSERT into receipe (code, reference, text) values ('4711', 'RefnotAvailable', 'Test');"

So I removed the "UNIQUE" keyword, knowing that later "UPDATE" command to modify entries will be slower.

So I built new table with additional FTS1 fulltext table. Then I tried to "UPDATE" one entry. In that moment the program stopped immediately working (WIN XP system), what means that the application stopped without comment and returned to desktop. I tried the same in SQLITE3.exe (command line program) but also that program suspended immediately after the UPDATE command (like "UPDATE Volltext SET code = '4710', reference = 'RefChanged', text = 'notext';"

That seems to me to be a bug.

By the way, creating fulltext table to search inside my whole database increased the filesize a lot (4 times). May be that is solved in FTS2? Last wish: Fulltext search like "foo*" to find "fool" and "foot" would be a really great improvement.

Best regards Ingo

2006-Oct-23 13:56:59 by anonymous:
Ooops, as I saw today, also "DELETE" statements are causing SQLITE to stop working (crash). Program returns to Desktop on WIN XP after DELETE command.
 
2027 code active 2006 Oct anonymous   2006 Oct   1 1 FTS: Phrase searches return Offsets for individual phrase words edit
With FTS (one as well as two), phrase searches return offsets for all individual words instead of the phrase as a whole, like in

  select name, ingredients from recipe where ingredients match '"broccoli cheese"';

Offsets() returns at least two matches for both individual words:

  • broccoli
  • cheese
 
2026 code active 2006 Oct anonymous   2006 Oct   4 5 \n in .output is not allowed even with quote edit
.output drive:\nabc.txt
.output e:\new0.txt
.output z:\new1.txt
.output "c:\new2.txt"

will result an error

omitting the \ will just put the file in the same folder to the sqlite3.exe (doesnt help)

solve it by replace \ by /

2006-Oct-15 21:28:55 by anonymous:
How about c:\\new.txt?


2006-Oct-16 11:49:58 by anonymous:
How about "c:/new.txt"?
 
2025 code active 2006 Oct anonymous   2006 Oct drh 5 5 Add pragama command to return loaded extension list edit
How about add a new pragama command to return loaded extension list?
 
2024 code active 2006 Oct anonymous   2006 Oct drh 5 5 Add If not exist syntax to Create Virtual Table edit
Parser Enhancement request: is it possible to add If not exist syntax to Create Virtual Table? all other create schema support such syntax.
 
2022 code active 2006 Oct anonymous   2006 Oct   1 1 .import command is not working edit
I have a windows system running version 3.3.6 and a linux system running 3.3.3 when I run .import catalog.csv TEMPDATA on the windows system, it works fine. On the linux system, no data gets imported. There are no error messages.

Is this a known issue in 3.3.3?

2006-Oct-14 01:15:07 by anonymous:
A sample SQL schema and a 3 line import file demonstrating the problem would be helpful.


2006-Nov-08 15:48:28 by anonymous:
Schema: CREATE TABLE Catalog ( UPC text , SKU text primary key , DESC text , PACK text , PRICE text , SIZE text );

test.csv contents 00000000103,103,EFFEM CHOCOLATE FUNSIZE 75PPK 1 X1EA,1,$155.94,1 EA 00000000152,414317,CLEARLIGHT SLUSH CUP 16OZ CDL16 1X50EA,1,$5.04,50 EA 00000000152,56880,CLEARLIGHT SLUSH CUP 16OZ CDL16 20X50EA,20,$96.31,50 EA

Command that does nothing: .import test.csv Catalog


2006-Nov-08 15:52:40 by anonymous:
Sorry, I'll try this again:

  Schema:
CREATE TABLE Catalog (
UPC text
, SKU text primary key
, DESC text
, PACK text
, PRICE text
, SIZE text );

  test.csv contents
00000000103,103,EFFEM CHOCOLATE FUNSIZE 75PPK 1 X1EA,1,$155.94,1 EA
00000000152,414317,CLEARLIGHT SLUSH CUP 16OZ CDL16 1X50EA,1,$5.04,50 EA
00000000152,56880,CLEARLIGHT SLUSH CUP 16OZ CDL16 20X50EA,20,$96.31,50 EA

  Command that does nothing:
.import test.csv Catalog
 
2019 code active 2006 Oct anonymous   2006 Oct   1 1 FTS1: Create table in transaction raises Out of Sequence error (21) edit
This error:

  SQL error: library routine called out of sequence

is caused if the following script is executed by the Windows version of the SQLite3 console application with .load fts1.dll extension. If it does not show immediately, it will eventually surface if the script is run multiple times.

The cause of the problem seems to be related to the transaction, the create virtual table as well as the amount of data inserted.

Finally, the script is attached.

 
2017 code active 2006 Oct anonymous   2006 Oct   1 1 DROP TABLE fails on FTS1 utility tables with certain OMIT_s defined edit
The following SQL fails when SQLite is compiled with the SQLITE_OMIT_ defines stated below:

  create virtual table foo using fts1 (content);
  drop table foo;
  create virtual table foo using fts1 (content);

Cause: The foo_content and foo_term tables are not deleted.

To verify, please define these SQLITE_OMIT_s:

  OPTS += -DSQLITE_OMIT_ALTERTABLE
  OPTS += -DSQLITE_OMIT_ANALYZE
  OPTS += -DSQLITE_OMIT_AUTHORIZATION
  OPTS += -DSQLITE_OMIT_AUTOINCREMENT
  OPTS += -DSQLITE_OMIT_AUTOVACUUM
  OPTS += -DSQLITE_OMIT_BETWEEN_OPTIMIZATION
  OPTS += -DSQLITE_OMIT_BLOB_LITERAL
  OPTS += -DSQLITE_OMIT_CAST
  OPTS += -DSQLITE_OMIT_CHECK
  OPTS += -DSQLITE_OMIT_COMPLETE
  OPTS += -DSQLITE_OMIT_COMPOUND_SELECT
  OPTS += -DSQLITE_OMIT_EXPLAIN
  OPTS += -DSQLITE_OMIT_FLAG_PRAGMAS
  OPTS += -DSQLITE_OMIT_FOREIGN_KEY
  OPTS += -DSQLITE_OMIT_GET_TABLE
  OPTS += -DSQLITE_OMIT_GLOBALRECOVER
  OPTS += -DSQLITE_OMIT_INTEGRITY_CHECK
  OPTS += -DSQLITE_OMIT_LIKE_OPTIMIZATION
  OPTS += -DSQLITE_OMIT_MEMORYDB
  OPTS += -DSQLITE_OMIT_OR_OPTIMIZATION
  OPTS += -DSQLITE_OMIT_ORIGIN_NAMES
  OPTS += -DSQLITE_OMIT_PAGER_PRAGMAS
  OPTS += -DSQLITE_OMIT_PROGRESS_CALLBACK
  OPTS += -DSQLITE_OMIT_QUICKBALANCE
  OPTS += -DSQLITE_OMIT_REINDEX
  OPTS += -DSQLITE_OMIT_SCHEMA_VERSION_PRAGMAS
  OPTS += -DSQLITE_OMIT_SHARED_CACHE
  OPTS += -DSQLITE_OMIT_SUBQUERY
  OPTS += -DSQLITE_OMIT_TCL_VARIABLE
  OPTS += -DSQLITE_OMIT_TEMPDB
  OPTS += -DSQLITE_OMIT_TRACE
  OPTS += -DSQLITE_OMIT_TRIGGER
  OPTS += -DSQLITE_OMIT_UTF16
  OPTS += -DSQLITE_OMIT_VACUUM
  OPTS += -DSQLITE_OMIT_VIEW

Without the SQLITE_OMIT_s, everything works just fine.

 
2015 code active 2006 Oct anonymous   2006 Oct anonymous 5 4 Enhancement Req: "EXPRn" PRAGMA for result set expression column names edit
I would like to propose a new PRAGMA command that could be set to control how expression column names are represented in result sets.

The current behavior appears to be that the expression that generated the column becomes the column name. For example, "SELECT COLUMN1, COLUMN 2, COLUMN1 + COLUMN2 FROM MYTABLE" yields:

[COLUMN1] | [COLUMN2] | [COLUMN1 + COLUMN2]

I propose a PRAGMA to remove the expression itself and replace it with 'EXPRn', where n is an ordinal based on the number of expression columns in the result set. (First expression is 0, second is 1, and so on):

[COLUMN1] | [COLUMN2] | [EXPR0]

2006-Oct-12 09:17:25 by anonymous:
Maybe you can just use

  SELECT COLUMN1, COLUMN 2, COLUMN1 + COLUMN2 AS EXPR0 FROM MYTABLE

instead.

 
2014 code active 2006 Oct anonymous   2006 Oct anonymous 4 3 Enhancement Req: CREATE [TEMP | TEMPORARY] VIRTUAL TABLE edit
Regarding the experimental VIRTUAL TABLE implementation, I believe it would of benefit to provide a "temp", or volatile construct when working with them.

-- From a SQL syntax perspective, adding an optional keyword "TEMP" to the declaration: CREATE [TEMP | TEMPORARY] VIRTUAL TABLE.

-- From a code perspective, I would envision this to invoke xCreate as it does now, but when the database is closed, the table is automatically dropped like any temp table, and xDestroy invoked rather than xDisconnect.

One sticky point I can picture is behavior when multiple opens exist to a single database from the same process space. Since virtual tables are already reference counted (in SQLite 3.3.8), perhaps the reference count could be made to span database handles and be bubbled up to the process level instead. That would allow the table to be CREATEd on one handle, CONNECTed on a second handle, then DISCONNECTed/DESTROYed based on the process-wide reference count.

I feel that there are numerous implementation possibilities for this. Having no option to auto-drop a virtual table can lead to stray module references, creating SQLite database files that cannot be properly utilized if the vtable module is not available.

Of course this can be implemented by the application calling DROP TABLE on it's own, but an embedded solution that takes care of it seems more 'proper' given the thought that goes into SQLite as a whole.

 
2013 code active 2006 Oct anonymous   2006 Oct drh 4 3 Autoincrement increments on failing INSERT OR IGNORE edit
  % package require sqlite3
  3.3.8
  % sqlite3 db ""
  % db eval "CREATE TABLE test (counter INTEGER PRIMARY KEY AUTOINCREMENT, value text NOT NULL UNIQUE)"
  % db eval "INSERT INTO test VALUES(4, 'hallo')"
  % db eval "SELECT * FROM sqlite_sequence"
  test 4
  % db eval "INSERT OR IGNORE INTO test(value) VALUES('hallo')"
  % db eval "SELECT * FROM sqlite_sequence"
  test 5
---> there has no dataset been inserted but the AUTOINCREMENT-counter is incremented

  % db eval "INSERT OR IGNORE INTO test VALUES(4, 'hallo')"
  % db eval "SELECT * FROM sqlite_sequence"
  test 5
---> right behavior: no inserted dataset and no incrementation

This maybe could be a problem if the "INSERT OR IGNORE" happens very often.

 
2012 code active 2006 Oct anonymous   2006 Oct   4 3 trigger4.test aborts "make test" on Windows edit
The failure to remove these files causes "make test" to abort without completing remaining tests:

  trigger4-99.9... Ok
  ./testfixture: error deleting "trigtest.db": permission denied
      while executing
  "file delete -force trigtest.db trigtest.db-journal"
      (file "test/trigger4.test" line 199)

fix:

  Index: test/trigger4.test
  ===================================================================
  RCS file: /sqlite/sqlite/test/trigger4.test,v
  retrieving revision 1.9
  diff -u -3 -p -r1.9 trigger4.test
  --- test/trigger4.test        4 Oct 2006 11:55:50 -0000       1.9
  +++ test/trigger4.test        9 Oct 2006 14:09:07 -0000
  @@ -195,6 +195,6 @@ do_test trigger4-7.2 {

   integrity_check trigger4-99.9

  -file delete -force trigtest.db trigtest.db-journal
  +catch {file delete -force trigtest.db trigtest.db-journal}

   finish_test
Not sure why this ticket was set to Fixed_in_3.0, but I can reproduce the "make test" abort on Windows.


2006-Oct-11 00:27:16 by drh:
I do not know why the resolution was set to "Fixed_In_3.0" either. It seems to have been set that why by the original submitter.

I will fix this eventually, but since it does not represent a real malfunction, it has a lower priority.

 
2011 code active 2006 Oct anonymous   2006 Oct   3 2 Escaping Porblem with .mode insert (double apostrophe) edit
select * from messages where message_id="74B23AAF-5FFD6BF2";

74B23AAF-5FFD6BF2|75|0|0|0|0|Europe talks, acts tough on Iran||http://www.ncr-iran.org/index.php?option=com_content&task=view&id=1052&Itemid=71|1140529235.0|By Gareth HardingThe United Press International, BRUSSELS -- Europeans are supposed to prefer soft to hard power, jaw-jaw to war-war and appeasement to confrontation. In short, in the words of neo-conservative scholar Robert Kagan: \'Americans are from Mars; Europeans are from Venus.\'

The ".mode insert / .output" file looks like this.

INSERT INTO messages VALUES('74B23AAF-5FFD6BF2',75,0,0,0,0,'Europe talks, acts tough on Iran','','http://www.ncr-iran.org/index.php?option=com_content&task=view&id=1052&Itemid=71',1140529235.0,'By Gareth HardingThe United Press International, BRUSSELS -- Europeans are supposed to prefer soft to hard power, jaw-jaw to war-war and appeasement to confrontation. In short, in the words of neo-conservative scholar Robert Kagan: \''Americans are from Mars; Europeans are from Venus.\''');

Now there are two apostrophe and the Escaping is broken.

 
2010 code active 2006 Oct anonymous   2006 Oct   3 3 Timeout ignored in Shared-Cache locking model edit
With shared cache enabled, the busy timeout seems to be ignored. SQLITE_BUSY comes immediately. This occurs at least for locking situations within one shared cache.

My server (if i may call the cache sharing thread that way) has its own timeout handling. But I thought that a small timeout in sqlite3 might help to distinguish locks from deadlocks.

This was reproduced with both Python wrappers. These just call sqlite3_enable_shared_cache and sqlite3_busy_timeout and then execute BEGIN IMMEDIATE from two connections.

2006-Oct-06 13:56:21 by anonymous:
Weird, I thought it's my fault, but I see exactly the same behaviour with the C# ADO.NET 2.0 wrapper w/ the shared cache patch.
 
2006 event active 2006 Sep anonymous   2006 Sep   1 1 Strange data in a table. edit
When dumping a database file, this is what I found:

CREATE TABLE TopSites ( XID INTEGER REFERENCES X(ID), YID INTEGER REFERENCES Y(ID), URLID INTEGER REFERENCES TopSitesURLs(ID));

INSERT INTO "TopSites" VALUES(-761955577, 5, 1322);
INSERT INTO "TopSites" VALUES(-761955577, 5, 1120);
INSERT INTO "TopSites" VALUES(-761955577, 5, 1323);
INSERT INTO "TopSites" VALUES(-761955577, 5, 1324);
....................................................... INSERT INTO "TopSites" VALUES(-761955577, 5, 1323);
INSERT INTO "TopSites" VALUES(-761955577, 5, 1324);
INSERT INTO "TopSites" VALUES(NULL, 'http://www.bnimanningham.com', NULL);
INSERT INTO "TopSites" VALUES(NULL, 'http://www.wellnesscareoncollins.com.au/Chiropractic-Articles.html', NULL);
INSERT INTO "TopSites" VALUES(NULL, 'http://www.healthyrisepharmacy.com', NULL);
INSERT INTO "TopSites" VALUES(NULL, 'http://www.alextechmelb.com/testimonials.html', NULL);
INSERT INTO "TopSites" VALUES(NULL, 'http://www.ecca.com.au/melbourne-contactus.html', NULL);
INSERT INTO "TopSites" VALUES(NULL, 'http://www.naturopathicwellness.com.au/additionaltherapies.htm', NULL);
INSERT INTO "TopSites" VALUES(NULL, 'http://www.rrr.org.au/sponsors.php', NULL);
INSERT INTO "TopSites" VALUES(NULL, 'http://www.caavic.asn.au/html/s02_article/show_article.asp?id=507&topic_id=-1&category_id=-1', NULL); INSERT INTO "TopSites" VALUES(NULL, 'http://www.cosmeticchoice.com.au/healing_nutrition.php?PHPSESSID=&PHPSESSID=d85928253b38f1bf88200022e7a93218', NULL);
INSERT INTO "TopSites" VALUES(NULL, 'http://www.coca.com.au/vic.htm', NULL);
INSERT INTO "TopSites" VALUES(NULL, 'http://www.embracechiropractic.com.au', NULL);
INSERT INTO "TopSites" VALUES(NULL, 'http://www.cooperchiro.com', NULL);

The database was created on: os: Mac OS X 10.4.6 jre: 1.5.0_06-64 sqlite: 3.3.4

The code for inserting into the database is:

  public static String GetTopSitesInsert(int aX, int aY, int aURLID)
{
return "INSERT OR ROLLBACK INTO TopSites (XID, YID, URLID) VALUES
(" + aX + ", " + aY + ", " + aURLID + ");";
}

I think that the last lines are from another table, or from another insert, as the java int could have never been a value like: http://www.bnimanningham.com

When trying to delete some rows from this table, sqlite threw "malformed database" exception and the java virtual machine crashed.

2006-Sep-29 12:23:31 by anonymous:
This is duplicate of #2005


2006-Sep-29 14:03:50 by drh:
I'm thinking this and #2005 represent a bug in whatever Java bindings the reporter is using.
 
2005 event active 2006 Sep anonymous   2006 Sep   1 1 Multiple rows with the same primary key, and null values in "not null" edit
This is what I have found when dumping a database file:

CREATE TABLE TopSitesURLs ( ID INTEGER PRIMARY KEY, URLText TEXT NOT NULL );

INSERT INTO "TopSitesURLs" VALUES(1, 'http://www.backinline.com.au');
INSERT INTO "TopSitesURLs" VALUES(2, 'http://www.wellnesscareoncollins.com.au');
INSERT INTO "TopSitesURLs" VALUES(3, 'http://www.oakleighdental.com.au/chirodontics.php');
INSERT INTO "TopSitesURLs" VALUES(4, 'http://bacinactionchiropractic.com');
INSERT INTO "TopSitesURLs" VALUES(5, 'http://melbourne.zpages.com.au/chiropractors');
INSERT INTO "TopSitesURLs" VALUES(6, 'http://myname.chiropractic.com.au');
INSERT INTO "TopSitesURLs" VALUES(7, 'http://www.melbournechiropractor.com');
INSERT INTO "TopSitesURLs" VALUES(8, 'http://www.chiroweb.net/us/fl_melbourne.html');
INSERT INTO "TopSitesURLs" VALUES(9, 'http://www.chiropractor.net.au/aridiskin.htm');
INSERT INTO "TopSitesURLs" VALUES(10, 'http://www.melbournechiropractic.com.au');
INSERT INTO "TopSitesURLs" VALUES(11, 'http://www.melbournechiropractor.com/index.php?page=privacy.php&pageID=-1');
INSERT INTO "TopSitesURLs" VALUES(12, 'http://www.vitaminstoday.com.au/chiropractor/index.php?page=grid');
INSERT INTO "TopSitesURLs" VALUES(13, 'http://www.goodechiro.com/index.asp');
INSERT INTO "TopSitesURLs" VALUES(14, 'http://www.goodechiro.com/FirstVisit.asp');
INSERT INTO "TopSitesURLs" VALUES(15, 'http://www.usenature.com/chirodirectory.htm');
INSERT INTO "TopSitesURLs" VALUES(16, 'http://www.melbournemeditationcentre.com.au/courses/teacher.htm');
INSERT INTO "TopSitesURLs" VALUES(17, 'http://www.yogatree.com.au/Therapies.htm');
................................................................................... INSERT INTO "TopSitesURLs" VALUES(6259, 'http://www.yarravillehealth.com.au/osteopath-melbourne.html');
INSERT INTO "TopSitesURLs" VALUES(6260, 'http://www.worldveganday.org.au/forum/viewtopic.php?p=4543&sid=465ea5c2e7452f6fd23470488e277781');
INSERT INTO "TopSitesURLs" VALUES(15, NULL);
INSERT INTO "TopSitesURLs" VALUES(15, NULL);
INSERT INTO "TopSitesURLs" VALUES(15, NULL);
INSERT INTO "TopSitesURLs" VALUES(15, NULL);
INSERT INTO "TopSitesURLs" VALUES(15, NULL);
INSERT INTO "TopSitesURLs" VALUES(15, NULL);
INSERT INTO "TopSitesURLs" VALUES(15, NULL);
INSERT INTO "TopSitesURLs" VALUES(15, NULL);
INSERT INTO "TopSitesURLs" VALUES(15, NULL);
INSERT INTO "TopSitesURLs" VALUES(15, NULL);
INSERT INTO "TopSitesURLs" VALUES(15, NULL);
INSERT INTO "TopSitesURLs" VALUES(15, NULL);
INSERT INTO "TopSitesURLs" VALUES(14, NULL);
INSERT INTO "TopSitesURLs" VALUES(15, NULL);
INSERT INTO "TopSitesURLs" VALUES(15, NULL);
INSERT INTO "TopSitesURLs" VALUES(15, NULL);
INSERT INTO "TopSitesURLs" VALUES(15, NULL);
INSERT INTO "TopSitesURLs" VALUES(15, NULL);
INSERT INTO "TopSitesURLs" VALUES(15, NULL);
INSERT INTO "TopSitesURLs" VALUES(15, NULL);
INSERT INTO "TopSitesURLs" VALUES(15, NULL);
INSERT INTO "TopSitesURLs" VALUES(15, NULL);
INSERT INTO "TopSitesURLs" VALUES(15, NULL);
INSERT INTO "TopSitesURLs" VALUES(15, NULL);
INSERT INTO "TopSitesURLs" VALUES(15, NULL);
INSERT INTO "TopSitesURLs" VALUES(15, NULL);
INSERT INTO "TopSitesURLs" VALUES(15, NULL);

The database was created on: os: Mac OS X 10.4.6 jre: 1.5.0_06-64 sqlite: 3.3.4

The primary key 15 is duplicated, and the "not null" field is null.

2006-Sep-29 12:03:17 by anonymous:
What tool (and parameters) did you use to dump the database?


2006-Sep-29 13:13:28 by drh:
When I run the SQL, I get lots of errors. And the resulting database does not contain any duplicate primary keys or NULLs in NOT NULL columns.

Can you attach the database that contains duplicate primary keys and NULLs in NOT NULL columns to this ticket so that I can see it?

jre==Java Runtime Engine? Are you using some kind of java binding to SQLite? If so, which one? Is SQLite in a separate DLL, or is your Java binding using a statically linked (and possibly modified and broken) version of SQLite?


2006-Oct-03 15:15:54 by anonymous:
I am using a java wrapper for sqlite:

http://www.ch-werner.de/javasqlite/overview-summary.html

I got the same problem again:

INSERT INTO "TopSitesURLs" VALUES(13023, 'http://costaricaretirementvacationproperties.com/index.php?op=show_listing&ShowOption=Condo Resales&option=cat'); INSERT INTO "TopSitesURLs" VALUES(13024, 'http://www.hot-tropics.com/costa-rica-links.html'); INSERT INTO "TopSitesURLs" VALUES(13025, 'http://southpacificrealestateservices.com/index.php?PHPSESSID=6b7a257fad5cbd886f09526a2cd59ed8'); INSERT INTO "TopSitesURLs" VALUES(9, NULL); INSERT INTO "TopSitesURLs" VALUES(9, NULL); INSERT INTO "TopSitesURLs" VALUES(9, NULL); INSERT INTO "TopSitesURLs" VALUES(9, NULL); INSERT INTO "TopSitesURLs" VALUES(9, NULL); INSERT INTO "TopSitesURLs" VALUES(9, NULL); INSERT INTO "TopSitesURLs" VALUES(9, NULL); INSERT INTO "TopSitesURLs" VALUES(9, NULL); INSERT INTO "TopSitesURLs" VALUES(9, NULL); INSERT INTO "TopSitesURLs" VALUES(9, NULL); INSERT INTO "TopSitesURLs" VALUES(9, NULL); INSERT INTO "TopSitesURLs" VALUES(9, NULL); INSERT INTO "TopSitesURLs" VALUES(9, NULL); INSERT INTO "TopSitesURLs" VALUES(9, NULL); INSERT INTO "TopSitesURLs" VALUES(9, NULL); INSERT INTO "TopSitesURLs" VALUES(9, NULL); INSERT INTO "TopSitesURLs" VALUES(9, NULL); INSERT INTO "TopSitesURLs" VALUES(9, NULL); INSERT INTO "TopSitesURLs" VALUES(9, NULL); INSERT INTO "TopSitesURLs" VALUES(9, NULL); INSERT INTO "TopSitesURLs" VALUES(9, NULL); INSERT INTO "TopSitesURLs" VALUES(9, NULL); INSERT INTO "TopSitesURLs" VALUES(9, NULL); INSERT INTO "TopSitesURLs" VALUES(9, NULL); INSERT INTO "TopSitesURLs" VALUES(9, NULL); INSERT INTO "TopSitesURLs" VALUES(9, NULL); INSERT INTO "TopSitesURLs" VALUES(9, NULL); INSERT INTO "TopSitesURLs" VALUES(9, NULL); INSERT INTO "TopSitesURLs" VALUES(9, NULL); INSERT INTO "TopSitesURLs" VALUES(9, NULL); INSERT INTO "TopSitesURLs" VALUES(9, NULL); INSERT INTO "TopSitesURLs" VALUES(9, NULL); INSERT INTO "TopSitesURLs" VALUES(9, NULL); INSERT INTO "TopSitesURLs" VALUES(9, NULL); INSERT INTO "TopSitesURLs" VALUES(9, NULL); INSERT INTO "TopSitesURLs" VALUES(9, NULL); INSERT INTO "TopSitesURLs" VALUES(9, NULL); INSERT INTO "TopSitesURLs" VALUES(9, NULL); INSERT INTO "TopSitesURLs" VALUES(9, NULL); INSERT INTO "TopSitesURLs" VALUES(9, NULL); INSERT INTO "TopSitesURLs" VALUES(9, NULL); INSERT INTO "TopSitesURLs" VALUES(9, NULL); INSERT INTO "TopSitesURLs" VALUES(9, NULL); INSERT INTO "TopSitesURLs" VALUES(9, NULL); INSERT INTO "TopSitesURLs" VALUES(9, NULL); INSERT INTO "TopSitesURLs" VALUES(9, NULL); INSERT INTO "TopSitesURLs" VALUES(9, NULL); INSERT INTO "TopSitesURLs" VALUES(9, NULL); INSERT INTO "TopSitesURLs" VALUES(9, NULL); INSERT INTO "TopSitesURLs" VALUES(9, NULL); INSERT INTO "TopSitesURLs" VALUES(9, NULL); INSERT INTO "TopSitesURLs" VALUES(9, NULL); INSERT INTO "TopSitesURLs" VALUES(9, NULL); INSERT INTO "TopSitesURLs" VALUES(9, NULL); INSERT INTO "TopSitesURLs" VALUES(9, NULL); INSERT INTO "TopSitesURLs" VALUES(9, NULL); INSERT INTO "TopSitesURLs" VALUES(9, NULL); INSERT INTO "TopSitesURLs" VALUES(9, NULL); INSERT INTO "TopSitesURLs" VALUES(9, NULL); INSERT INTO "TopSitesURLs" VALUES(9, NULL); INSERT INTO "TopSitesURLs" VALUES(9, NULL); INSERT INTO "TopSitesURLs" VALUES(9, NULL); INSERT INTO "TopSitesURLs" VALUES(9, NULL); INSERT INTO "TopSitesURLs" VALUES(9, NULL); INSERT INTO "TopSitesURLs" VALUES(9, NULL); INSERT INTO "TopSitesURLs" VALUES(9, NULL); INSERT INTO "TopSitesURLs" VALUES(9, NULL); INSERT INTO "TopSitesURLs" VALUES(9, NULL); INSERT INTO "TopSitesURLs" VALUES(9, NULL); INSERT INTO "TopSitesURLs" VALUES(9, NULL); INSERT INTO "TopSitesURLs" VALUES(9, NULL); INSERT INTO "TopSitesURLs" VALUES(9, NULL); INSERT INTO "TopSitesURLs" VALUES(9, NULL); INSERT INTO "TopSitesURLs" VALUES(9, NULL); INSERT INTO "TopSitesURLs" VALUES(13041, 'http://www.livingabroadincostarica.com'); INSERT INTO "TopSitesURLs" VALUES(13042, 'http://limitededitionsre.com/blog.html'); INSERT INTO "TopSitesURLs" VALUES(13043, 'http://www.officecenter.nosaranet.com/property.html');

You can see the index 13025, then up to 13041 you can see only 9.

How do I upload a database?


2006-Oct-03 15:17:33 by anonymous:
I have dumped the database with sqlite3.exe in command line: sqlite3.exe file.db .dump > file.sql


2006-Oct-03 17:44:00 by anonymous:

  How do I upload a database?

Use the Attach link near the top right. Note that attachment size is currently limited to 100KB.


2006-Oct-04 06:38:23 by anonymous:
An integrity check on this database looks like this:

C:\Documents and Settings\stefan matei\Desktop>sqlite3 project.db
SQLite version 3.3.4
Enter ".help" for instructions
sqlite> PRAGMA integrity_check;
*** in database main ***
On tree page 59 cell 10: 2nd reference to page 1077
On tree page 59 cell 10: Child page depth differs
On tree page 59 cell 11: Child page depth differs
On page 871 at right child: 2nd reference to page 1078
sqlite> .quit

When can this happen? Is there a fix for this (integrity fix or something)? I ask this because the database is perfectly readable. I assume that a tool can be done to check the tables and remove all the data that are not complying to the table definition.

 
2004 warn active 2006 Sep anonymous   2006 Sep   1 4 vtab.c:142: warning: pointer targets in initialization differ in signe edit
const unsigned char *z = pParse->sArg.z;

fix this warning but then this warning appears:

vtab.c:145: warning: pointer targets in passing argument 1 of 'sqlite3StrNDup' differ in signedness

which can be fixed with:

addModuleArgument(pParse->pNewTable, sqliteStrNDup((char *)z, n));

 
1822 code active 2006 May anonymous   2006 Sep   3 3 Table Alias together with Subquery seems not to work proper edit
SELECT * FROM auth AS a LEFT JOIN (SELECT tm.team FROM teammbs AS tm) AS tr ON a.ateam=tr.team;

Error message: No such colum tr.team

But if I run the sub-query itself, it works fine. Of course, this example can be expressed different, so no subquery required. But the complete expression looks like this:

SELECT a.auth, a.avalue FROM auth a LEFT JOIN (SELECT tm.member, tm.team FROM teammbs tm, team t WHERE tm.team=t.teamid AND (t._state<64 or (t._state>120 AND t._state<192)) AND (tm._state<64 or (tm._state>120 AND tm._state<192))) AS tr ON a.ateam=tr.team WHERE (a._state<64 or (a._state>120 AND a._state<192)) AND (a.auser='test' OR tr.member='test') ORDER BY a.auth;

It works fine with MySQL 5, and brings the same error on SQLite 3: No such column tr.team.

Any idea?

 
1998 build active 2006 Sep anonymous   2006 Sep   2 3 prefix option to configure ignored in tclinstaller.tcl edit
schliep@karlin:~/tmp/sqlite-3.3.7> configure --prefix=/some/dir ...

schliep@karlin:~/tmp/sqlite-3.3.7> make install tclsh ./tclinstaller.tcl 3.3 can't create directory "/usr/lib/tcl8.4/sqlite3": permission denied

After commenting out all the stuff in ./tclinstaller.tcl things work

 
1996 new active 2006 Sep anonymous Unknown 2006 Sep drh 2 3 Data type CHAR edit
An interface API for CHAR datatypes would really be helpful.

For example, often sql tables contain CHAR(1) datatypes or CHAR(10) types. There should be some mechanism for handling these types natively. ie: sqlite3_bind_char sqlite3_column_char sqlite3_result_char sqlite3_value_char

This will allow a more native implementation for CHAR datatypes, As it is, a single CHAR(1) must be first converted into a string (char[2]) and copied with a terminator. for CHAR types, not \000 termination is required. It is implied with the lenght.

Thanks...

 
1994 code active 2006 Sep anonymous Parser 2006 Sep   1 3 Columns from nested joins aren't properly propagated edit
When using this query:

    SELECT * FROM ROLE_ATTRIBUTE INNER JOIN (ROLE INNER JOIN PERSON ON ROLE.PERSON_ID=PERSON.ID) ON ROLE_ATTRIBUTE.PERSON_ID=ROLE.PERSON_ID AND ROLE_ATTRIBUTE.PROJECT_ID=ROLE.PROJECT_ID WHERE ((PERSON.FIRSTNAME = "bob"));

the parser fails with an error "no such column: ROLE.PROJECT_ID". It seems that doing an inner join with more than one subexpression doesn't work.

2006-Sep-25 22:41:52 by anonymous:
Your query will run without the brackets.

  SELECT *
  FROM PERSON P INNER JOIN ROLE_ATTRIBUTE RA
       ON P.ID = RA.PERSON_ID
       INNER JOIN ROLE R
       ON RA.PROJECT_ID = R.PROJECT_ID AND
          P.ID          = R.PERSON_ID
  WHERE P.FIRSTNAME = 'bob';


2006-Sep-25 23:03:28 by navaraf:
Hm, you're right. So actually the thing SQLite chokes on is the parenthesis syntax as JOIN parameter. I can try to modify the generator to produce the expanded form, but since the same code is used for MSSQL, MySQL and Oracle I still think it would be handy to allow it in SQLite too. Also it's not my code that generates these horrible expressions and I'd rather try to avoid modifying it.


2006-Sep-26 09:59:13 by anonymous:
I changed the title to correctly describe the problem. Also I found another thread on the mailing list that describes exactly the same problem: http://marc.10east.com/?t=115378699000001


2006-Sep-26 11:42:38 by navaraf:
I believe the "lookupName" function in src/expr.c should do recursion for ephemeral tables found in the pSrcList (at least those that were created as subqueries in the FROM clause of the SELECT statement).
 
1990 code active 2006 Sep anonymous   2006 Sep   1 1 sqlite3_close doesn't release always the file handle edit
I think that sqlite3_close behave strangly.

I use version 3.3.7 on Linux (Fedora Core 5).

What I do is to open a database, and start a transaction in it. Then, without ending the transaction, open again the database and simply close it. I found out, that the inner sqlite3_close return 0 (SQLITE_OK), but the file handle is not released. So if I do it too many times, I run out of file handles.

You are free to ask why I open and close that many times the same database while it is already in transaction. This is my mistake. Actually, it is already fixed. But I still wonder - shouldn't the sqlite3_close return other thing then just SQLITE_OK? Especially if the file handle is not released? If it did, I would find my mistake much earlier.

Here is my script that demonstrate it (you can use /usr/sbin/lsof in linux to see how many times the file is opened):

  #include <sqlite3.h>
  int main(int argc, char **argv) {
    sqlite3* db;
    sqlite3* db_inner;
    int rc;
    int i;
    system("rm -f open_many_test.db");

    rc = sqlite3_open("open_many_test.db", &db);
    sqlite3_exec(db, "begin", 0, 0, 0);
    sqlite3_stmt *pStmt;
    rc = sqlite3_prepare(db,
                         "create table a (id varchar)",
                         -1,
                         &pStmt,
                         0);
    rc = sqlite3_step(pStmt);
    sqlite3_finalize(pStmt);

    rc = sqlite3_prepare(db,
                         "insert into a values('bla')",
                         -1,
                         &pStmt,
                         0);
    rc = sqlite3_step(pStmt);
    sqlite3_finalize(pStmt);

    for (i = 0; i < 10000; i++) {
      rc = sqlite3_open("open_many_test.db", &db_inner);
      printf("sqlite3_open gives %d\n", rc);

      rc = sqlite3_close(db_inner);
      printf("sqlite3_close gives %d\n", rc);
    }

    sqlite3_exec(db, "commit", 0, 0, 0);
    rc = sqlite3_close(db);
  }
2006-Sep-23 15:29:46 by drh:
This behavior is intentional. It is there to work around bugs in the design of posix advistory locks. See ticket #561 and check-in [1171] .

Under posix, if you have the same file open multiple times and you close one of the file descriptors, all locks on that file for all file descriptors are cleared. To prevent this from occurring, SQLite defers closing file descriptors until all locks on the file have been released.

One possible work-around would be to reuse file descriptors that waiting to be closed for the next open, rather than creating a new file descriptor.


2006-Sep-23 15:35:21 by anonymous:
The inner call should to sqlite3_open() should simply fail in that case, rather than set up a condition where by a file descriptor is leaked (which no one wants). This is unfortunate because sqlite3_open()'s behavior would not be uniform across platforms.


2006-Sep-23 16:43:32 by anonymous:
SQLite should do a lookup via stat()'s st_dev/st_ino fields prior to open() and if found to be the same as an already opened database file, it should use the same (refcounted) file descriptor, eliminating the need for open() in this case.

...upon reflection, having two sqlite connections using the same file descriptor would be a bad thing. stat() could be used to decide if a fd pending close() is recyclable, though.


2006-Sep-23 18:17:34 by drh:
Two points:

  1. SQLite does not and has never leaked file descriptors. All file descriptors are eventually closed. The close is merely deferred until the pending transaction COMMITs.

  2. I will be taking a very caution and careful approach toward resolving this issue. The issue itself is minor (it has only just now been reported but the behavior has been there for 3 years) but the consequences of getting the fix wrong are severe (database corruption.) And there are abundant opportunities for getting the fix wrong.
 
1983 code active 2006 Sep anonymous   2006 Sep   2 2 I/O Error at a size of 4GB and auto_vacuum=1 edit
when i'm building a database with auto_vacuum=1 and page_size=8192, i get an I/O error at a size of about 4GB. All tables are still readable but then it isn't possible to insert any more data.

The table is filled with a column of BLOBs and some columns with numbers. I use the 3.3.7 binary with Windows 2000 Server.

 
1980 code active 2006 Sep drh   2006 Sep   1 1 Initializing FTS1 twice causes it to fail. edit
If you try to load the shared module twice, it causes the module to no longer work.
 
1975 new active 2006 Sep anonymous   2006 Sep   5 4 Request for sqlite3_table_column_metadata16 edit
It would be nice to have a sqlite3_table_column_metadata16() function as an UTF-16 version of the existing sqlite3_table_column_metadata().
 
1972 code active 2006 Sep anonymous   2006 Sep   2 4 segfault on empty query edit
SQLite 2.8.17 used in latest versions of PHP segfaults with empty query (i.e. " ", 1 whitespace).

PHP reproduce code: <?php $dbh = new PDO('sqlite2:pdo_sqlite2'); var_dump($dbh->query(" ")); ?>

GDB backrace:

  Program received signal SIGSEGV, Segmentation fault.
  [Switching to Thread 1077220512 (LWP 3909)]
  0x0814d227 in sqlite_step (pVm=0x0, pN=0x40364218, pazValue=0x40364210, pazColName=0x40364214) at /local/dev/php-src_5_2/ext/sqlite/libsqlite/src/vdbe.c:117
  117       if( p->magic!=VDBE_MAGIC_RUN ){
  (gdb) bt
  #0  0x0814d227 in sqlite_step (pVm=0x0, pN=0x40364218, pazValue=0x40364210, pazColName=0x40364214) at /local/dev/php-src_5_2/ext/sqlite/libsqlite/src/vdbe.c:117
  #1  0x0812556a in pdo_sqlite2_stmt_execute (stmt=0x40364094) at /local/dev/php-src_5_2/ext/sqlite/pdo_sqlite2.c:102
  #2  0x080bf4d5 in zim_PDO_query (ht=1, return_value=0x40363110, return_value_ptr=0x0, this_ptr=0x40363178, return_value_used=1)
    at /local/dev/php-src_5_2/ext/pdo/pdo_dbh.c:1033
  #3  0x0824c1d6 in zend_do_fcall_common_helper_SPEC (execute_data=0xbfffca90) at /local/dev/php-src_5_2/Zend/zend_vm_execute.h:200
  #4  0x0824c722 in ZEND_DO_FCALL_BY_NAME_SPEC_HANDLER (execute_data=0xbfffca90) at /local/dev/php-src_5_2/Zend/zend_vm_execute.h:322
  #5  0x0824bde9 in execute (op_array=0x403637e4) at /local/dev/php-src_5_2/Zend/zend_vm_execute.h:92
  #6  0x0822e66a in zend_execute_scripts (type=8, retval=0x0, file_count=3) at /local/dev/php-src_5_2/Zend/zend.c:1095
  #7  0x081e7993 in php_execute_script (primary_file=0xbfffef00) at /local/dev/php-src_5_2/main/main.c:1759
  #8  0x082933de in main (argc=2, argv=0xbfffefe4) at /local/dev/php-src_5_2/sapi/cli/php_cli.c:1102
  (gdb) f 0
  #0  0x0814d227 in sqlite_step (pVm=0x0, pN=0x40364218, pazValue=0x40364210, pazColName=0x40364214) at /local/dev/php-src_5_2/ext/sqlite/libsqlite/src/vdbe.c:117
  117       if( p->magic!=VDBE_MAGIC_RUN ){
  (gdb) p p
  $1 = (Vdbe *) 0x0

Proposed patch: http://cvs.php.net/viewvc.cgi/php-src/ext/sqlite/libsqlite/src/vdbe.c?r1=1.7.4.1&r2=1.7.4.1.2.1

2006-Sep-10 11:10:26 by drh:
PHP is calling sqlite_step() with a NULL sqlite_stmt pointer. This seems more like a bug in PHP than SQLite. I suggest patching PHP, somewhere in pdo_sqlite2.c I'm guessing, so that it checks to see if the statement pointer returned by sqlite_prepare() is NULL and skips the sqlite_step() and sqlite_finalize() calls if it is. The proposed patch above seems to confirm this strategy: The proposed patch would cause PHP to receive an SQLITE_MISUSE error. An SQLITE_MISUSE error indicates that the API is begin used incorrectly. The right fix, it seems to me, would be to use the API correctly.


2006-Sep-10 15:40:46 by anonymous:
"An SQLITE_MISUSE error indicates that the API is begin used incorrectly". I agree - and this simple NULL check would be the perfect place for SQLite to issue such a MISUSE error. Having SQLite recover from such a relatively common type of NULL input error would be beneficial to its users. SQLite already goes to great lengths to recover from out of memory situations. I don't see any reason not to add a simple "if NULL" check here to avoid crashing the the user's application.


2006-Sep-10 16:56:12 by drh:
I would quickly add such a change to SQLite version 3. And in fact I have already done so. See Ticket #870 and check-in [1906] . But we are talking about SQLite version 2, here, which is in maintenance mode and should not be used for new development.


2006-Sep-10 20:05:41 by anonymous:
>I suggest patching PHP, somewhere in pdo_sqlite2.c

That's possible too, but currently I see no reasons to do it.

>I'm guessing, so that it checks to see if the statement pointer returned by > sqlite_prepare() is NULL and skips the sqlite_step() and sqlite_finalize() calls if it is.

The statement pointer in this case is (not) returned by sqlite_compile() call. Here is the piece of code:

    ...
    S->einfo.errcode = sqlite_compile(S->H->db, stmt->active_query_string, &tail, &S->vm, &errmsg);
    if (S->einfo.errcode != SQLITE_OK) {
        pdo_sqlite2_error_stmt(errmsg, stmt);
        return 0;
    }

    S->done = 0;
    S->einfo.errcode = sqlite_step(S->vm, &S->ncols, &S->rowdata, &S->colnames);
    ...

From what I see, sqlite_compile() considers " " query as valid and doesn't return error, but in the same time the statement pointer remains NULL, which is clearly wrong. As I've said, it's easy to check if it's NULL or not in PHPs code, but I really think that the problem is in sqlite_compile().

 
1974 code active 2006 Sep anonymous Unknown 2006 Sep   1 1 column type not consistent in views edit
package require sqlite3

sqlite3 db test.db

db eval {

create table one ( size FLOAT );

create view two as select size from one; }

db eval {insert into one values(50.0)}

puts [db eval {select size from one}]

puts [db eval {select size from two}]

outputs:

50.0

50

 
1445 code active 2005 Sep anonymous   2006 Sep   3 3 Errors testing sqlite 3.2.6 (& v3.3.7) edit
  $ make test
  [...]
  conflict-6.0... Ok
  conflict-6.1... Ok
  conflict-6.2...
  Expected: [0 {7 6 9} 1 1]
       Got: [0 {7 6 9} 1 0]
  conflict-6.3...
  Expected: [0 {6 7 3 9} 1 1]
       Got: [0 {6 7 3 9} 1 0]
  conflict-6.4... Ok
  conflict-6.5... Ok
  conflict-6.6... Ok
  conflict-6.7...
  Expected: [0 {6 7 3 9} 1 1]
       Got: [0 {6 7 3 9} 1 0]
  conflict-6.8...
  Expected: [0 {7 6 9} 1 1]
       Got: [0 {7 6 9} 1 0]
  conflict-6.9...
  Expected: [0 {6 7 3 9} 1 1]
       Got: [0 {6 7 3 9} 1 0]
  conflict-6.10...
  Expected: [0 {7 6 9} 1 1]
       Got: [0 {7 6 9} 1 0]
  conflict-6.11...
  Expected: [0 {6 7 3 9} 1 1]
       Got: [0 {6 7 3 9} 1 0]
  conflict-6.12...
  Expected: [0 {6 7 3 9} 1 1]
       Got: [0 {6 7 3 9} 1 0]
  conflict-6.13...
  Expected: [0 {7 6 9} 1 1]
       Got: [0 {7 6 9} 1 0]
  conflict-6.14... Ok
  conflict-6.15... Ok
  conflict-6.16... Ok
  [...]
  date-3.12... Ok
  date-3.13... Ok
  date-3.14... Ok
  date-3.15... Ok
  date-3.16... Ok
  date-3.17... Ok
  /tmp/sqlite-3.2.6/.libs/lt-testfixture: invalid command name "clock"
      while executing
  "clock seconds"
      invoked from within
  "clock format [clock seconds] -format "%Y-%m-%d" -gmt 1"
      invoked from within
  "set now [clock format [clock seconds] -format "%Y-%m-%d" -gmt 1]"
      (file "./test/date.test" line 142)
      invoked from within
  "source $testfile"
      ("foreach" body line 4)
      invoked from within
  "foreach testfile [lsort -dictionary [glob $testdir/*.test]] {
    set tail [file tail $testfile]
    if {[lsearch -exact $EXCLUDE $tail]>=0} continue
    so..."
      (file "./test/quick.test" line 45)
  make: *** [test] Error 1
2005-Sep-19 23:03:56 by drh:
The test scripts do not (yet) work with Tcl 8.5. Use Tcl 8.4.


2005-Sep-20 01:59:42 by anonymous:
FYI, The conflict failures occur even when using tcl-8.4. The problem was reported on the mailing list:

  http://www.mail-archive.com/sqlite-users%40sqlite.org/msg10203.html

Curiously, the failures correspond exactly to the test cases that were changed by the following patch:

  http://www.sqlite.org/cvstrac/filediff?f=sqlite/test/conflict.test&v1=1.24&v2=1.25


2006-Aug-31 23:49:40 by anonymous:
building v337 on OSX 10.4.7 w/ TCL8.5 installed as Framework, 'make test' still fails w/:

  date-3.16... Ok
  date-3.17... Ok
  /usr/ports/sqlite-3.3.7/build/.libs/testfixture: invalid command name "clock"
      while executing
  "clock seconds"
      invoked from within
  "clock format [clock seconds] -format "%Y-%m-%d" -gmt 1"
      invoked from within
  "set now [clock format [clock seconds] -format "%Y-%m-%d" -gmt 1]"
      (file "../test/date.test" line 142)
      invoked from within
  "source $testfile"
      ("foreach" body line 4)
      invoked from within
  "foreach testfile [lsort -dictionary [glob $testdir/*.test]] {
    set tail [file tail $testfile]
    if {[lsearch -exact $EXCLUDE $tail]>=0} continue
    so..."
      (file "../test/quick.test" line 66)
  make: *** [test] Error 1

any resolution for this, other than revert to TCL 8.4?


2006-Sep-01 01:26:37 by anonymous:
SQLite under Cygwin fails all tests that involve integers larger than 32 bits. Sqlite produces the correct 64 bit values, but Tcl as distributed with Cygwin cannot grok 64 bit ints, so the comparisons fail. Would it be possible to change Sqlite's test harness to compare SQL results as strings rather than as integers? Then it would not matter if Tcl worked in 64 bit or not.


2006-Sep-01 15:50:48 by drh:
The test suite has been revised so that it now works with Tcl8.5. But, no, it is not practical to rewrite the tests to compare the results using strings instead of integers in order to work with the (broken) tcl implementation that comes with cygwin.


2006-Sep-06 02:39:24 by anonymous:

  updating to latest cvs-checkout to get the aforementioned fix for:

      date-3.17... Ok
      /usr/ports/sqlite-3.3.7/build/.libs/testfixture: invalid command name "clock"
          while executing

  i can verify that _that_ is now ok:

      ...
      date-3.14... Ok
      date-3.15... Ok
      date-3.16... Ok
      date-3.17... Ok
      date-4.1...
      Expected: [2006-09-01]
           Got: [2006-09-06]
      date-5.1... Ok
      date-5.2... Ok
      date-5.3... Ok
      ...

  but now, 'make test' fails next @:

      delete-8.4... Ok
      delete-8.5... Ok
      delete-8.6... Ok
      delete-8.7... Ok
      /usr/ports/sqlite-cvs/build/.libs/testfixture: error deleting "test.db": not owner
          while executing
      "file delete -force test.db"
          (file "../test/tester.tcl" line 62)
          invoked from within
      "source $testdir/tester.tcl"
          (file "../test/delete2.test" line 36)
          invoked from within
      "source $testfile"
          ("foreach" body line 4)
          invoked from within
      "foreach testfile [lsort -dictionary [glob $testdir/*.test]] {
        set tail [file tail $testfile]
        if {[lsearch -exact $EXCLUDE $tail]>=0} continue
        so..."
          (file "../test/quick.test" line 66)
      make: *** [test] Error 1


2006-Sep-06 11:11:19 by drh:
Run the build starting from an empty directory as a non-root user.


2006-Sep-06 13:27:18 by anonymous:
per INSTALL instructions, i did:

  cvs -d :pserver:anonymous@www.sqlite.org:/sqlite checkout -d sqlite-cvs sqlite
  cd /usr/ports/sqlite-cvs

  mkdir build
  cd build

  ../configure \
  ...

  make

  chown -R myuser:wheel /usr/ports/sqlite-cvs
  sudo -u myuser make test

and, as reported, the error was the result.


2006-Sep-30 21:43:45 by anonymous:
bump.

anyone?


2006-Sep-30 22:19:24 by anonymous:
If you don't happen to be testing on Linux/gcc or Windows/VC++ I find that the Tcl test results have more than a few failures. It is not always easy to discern which failures are due to some odd quirk of Tcl or whether it is a legitimate SQLite issue on a given platform. Be prepared to change test scripts and tinker with the code.
 
1961 build active 2006 Sep anonymous   2006 Sep   3 3 3.3.7 : wrong readline.h path in Makefile edit
We have readline.h installed in /usr/local/include/readline. In SQLite it is accessed with :

#include <readline/readline.h>

But unfortunatly in Makefile, READLINE flags contains :

-I /usr/local/include/readline

instead of

-I /usr/local/include

 
1960 code active 2006 Sep anonymous   2006 Sep   4 2 Issues with .import in sqlite.exe edit
I ran into two possible problems when using the .import operation in sqlite3:

- .import seems to be confused by NULLs; in the file NullTest.dat the null is at the end of the line

- .import chokes on empty field when importing to field of type: integer PRIMARY KEY AUTOINCREMENT

For example line like:

~2~3~4~5~6

Example:

Schema:

--Table with autoincrement

CREATE TABLE test1( id integer PRIMARY KEY AUTOINCREMENT, c1 integer NULL , c2 integer NULL , c3 text NULL, c4 text NULL, c5 text NULL );

-- Table with no autoincrement field

CREATE TABLE test2( id integer NULL, c1 integer NULL , c2 integer NULL , c3 text NULL, c4 text NULL, c5 text NULL );

.separator ~

.import NullTest.dat test1

.import NullTest.dat test2

.import NoNullTest.dat test2

I have short test files that I can email to the person who is looking at this.

 
1959 new active 2006 Sep anonymous   2006 Sep   4 3 Unblockable TEMP TABLES edit
TEMP TABLES locks the complete database as long as a prepared stmt is running at the main database. Temp Tables are in separate files... so I hope it can be changed without big problems.

The new driver for OpenOffice.org needs temp tables that won't lock the complete database because of cached resultsets.

It only can be emulate it with "attach a database, copy data, detach". But the problem is that the API of OOo needs to change the cached resultset. It isn't possible to add this without temporary tables. So the driver could use sqlite3_update_hook() to know when he needs to reload the resultset.

Thanks

 
1958 code active 2006 Sep anonymous   2006 Sep   4 4 some printf tests fail with Tcl 8.5a5, ok with Tcl 8.4 edit
Tcl 8.5a5:

  printf-1.7.6...
  Expected: [Three integers: (1000000) ( f4240) (3641100)]
       Got: [Three integers: ( 1000000) ( f4240) (3641100)]

  printf-1.8.6...
  Expected: [Three integers: (999999999) (3b9ac9ff) (7346544777)]
       Got: [Three integers: ( 999999999) (3b9ac9ff) (7346544777)]

  printf-1.9.7...
  Expected: [Three integers: (     0) (   0x0) (     0)]
       Got: [Three integers: (     0) (     0) (     0)]

Tcl 8.4:

  printf-1.7.6... Ok
  printf-1.8.6... Ok
  printf-1.9.7... Ok
2006-Sep-05 02:27:00 by anonymous:
This is not directly related to the ticket, but concerns the same test file... Why are these tests not run on windows? I thought sqlite3_mprintf() is platform independent.

  if {$::tcl_platform(platform)!="windows"} {

  set m 1
  foreach {a b} {1 1 5 5 10 10 10 5} {
    set n 1
    foreach x {0.001 1.0e-20 1.0 0.0 100.0 9.99999 -0.00543 -1.0 -99.99999} {
      do_test printf-2.$m.$n.1 [subst {
        sqlite3_mprintf_double {A double: %*.*f} $a $b $x
      }] [format {A double: %*.*f} $a $b $x]
      do_test printf-2.$m.$n.2 [subst {
        sqlite3_mprintf_double {A double: %*.*e} $a $b $x
      }] [format {A double: %*.*e} $a $b $x]
      do_test printf-2.$m.$n.3 [subst {
        sqlite3_mprintf_double {A double: %*.*g} $a $b $x
      }] [format {A double: %*.*g} $a $b $x]
      do_test printf-2.$m.$n.4 [subst {
        sqlite3_mprintf_double {A double: %d %d %g} $a $b $x
      }] [format {A double: %d %d %g} $a $b $x]
      do_test printf-2.$m.$n.5 [subst {
        sqlite3_mprintf_double {A double: %d %d %#g} $a $b $x
      }] [format {A double: %d %d %#g} $a $b $x]
      do_test printf-2.$m.$n.6 [subst {
        sqlite3_mprintf_double {A double: %d %d %010g} $a $b $x
      }] [format {A double: %d %d %010g} $a $b $x]
      incr n
    }
    incr m
  }

  }  ;# endif not windows
 
1953 code active 2006 Sep anonymous TclLib 2006 Sep   4 3 Fix for false 64-bit comparisons "make test" failures on Cygwin edit
The trivial patch below allows Cygwin to correctly pass all (two dozen or so) 64-bit integer-related tests in "make test". It does so by treating all 64-bit integer SQL results as strings. (Note: SQLite has always produced correct 64-bit integer results, it's just that the test harness on Cygwin produces false failures without this patch.) There is no impact to other platforms, and allows us unfortunate Windows users to be useful members of society.

  RCS file: /sqlite/sqlite/src/tclsqlite.c,v
  retrieving revision 1.172
  diff -u -r1.172 tclsqlite.c
  --- src/tclsqlite.c     31 Aug 2006 15:07:15 -0000      1.172
  +++ src/tclsqlite.c     1 Sep 2006 17:27:44 -0000
  @@ -432,7 +432,12 @@
             if( v>=-2147483647 && v<=2147483647 ){
               pVal = Tcl_NewIntObj(v);
             }else{
  +#ifndef __CYGWIN__
               pVal = Tcl_NewWideIntObj(v);
  +#else
  +            int bytes = sqlite3_value_bytes(pIn);
  +            pVal = Tcl_NewStringObj((char *)sqlite3_value_text(pIn), bytes);
  +#endif
             }
             break;
           }
  @@ -1420,7 +1425,11 @@
                 if( v>=-2147483647 && v<=2147483647 ){
                   pVal = Tcl_NewIntObj(v);
                 }else{
  +#ifndef __CYGWIN__
                   pVal = Tcl_NewWideIntObj(v);
  +#else
  +                pVal = dbTextToObj((char *)sqlite3_column_text(pStmt, i));
  +#endif
                 }
                 break;
               }

Example test failures before patch:

  $ ./testfixture.exe test/misc2.testmisc2-1.1... Ok
  misc2-1.2... Ok
  misc2-2.1... Ok
  misc2-2.2... Ok
  misc2-2.3... Ok
  misc2-3.1... Ok
  misc2-4.1...
  Expected: [4000000000]
       Got: [-294967296]
  misc2-4.2...
  Expected: [4000000000 2147483648]
       Got: [-294967296 -2147483648]
  misc2-4.3... Ok
  misc2-4.4...
  Expected: [1 2147483648 2147483647]
       Got: [1 -2147483648 2147483647]
  misc2-4.5...
  Expected: [1 4000000000 2147483648 2147483647]
       Got: [1 -294967296 -2147483648 2147483647]
  misc2-4.6...
  Expected: [1 2147483647 2147483648 4000000000]
       Got: [1 2147483647 -2147483648 -294967296]
  misc2-5.1... Ok
  misc2-6.1... Ok
  misc2-7.1... Ok
  misc2-7.2... Ok
  misc2-7.3... Ok
  misc2-7.4... Ok
  misc2-7.5... Ok
  misc2-7.6... Ok
  misc2-7.7... Ok
  misc2-7.8... Ok
  misc2-8.1... Ok
  misc2-9.1... Ok
  misc2-9.2... Ok
  misc2-9.3... Ok
  misc2-10.1... Ok
  Thread-specific data deallocated properly
  5 errors out of 28 tests
  Failures on these tests: misc2-4.1 misc2-4.2 misc2-4.4 misc2-4.5 misc2-4.6

After patch applied:

  $ ./testfixture.exe test/misc2.testmisc2-1.1... Ok
  misc2-1.2... Ok
  misc2-2.1... Ok
  misc2-2.2... Ok
  misc2-2.3... Ok
  misc2-3.1... Ok
  misc2-4.1... Ok
  misc2-4.2... Ok
  misc2-4.3... Ok
  misc2-4.4... Ok
  misc2-4.5... Ok
  misc2-4.6... Ok
  misc2-5.1... Ok
  misc2-6.1... Ok
  misc2-7.1... Ok
  misc2-7.2... Ok
  misc2-7.3... Ok
  misc2-7.4... Ok
  misc2-7.5... Ok
  misc2-7.6... Ok
  misc2-7.7... Ok
  misc2-7.8... Ok
  misc2-8.1... Ok
  misc2-9.1... Ok
  misc2-9.2... Ok
  misc2-9.3... Ok
  misc2-10.1... Ok
  Thread-specific data deallocated properly
  0 errors out of 28 tests
  Failures on these tests:

The only new regression on Cygwin is this test, which is expected:

  types3-2.3...
  Expected: [wideInt]
       Got: []
2006-Sep-01 18:55:25 by drh:
The TCL interface is more than just part of the test harness. A lot of people use the TCL interface as part of their applications.

I believe what this patch does is mask a real problem.

I would prefer to fix the underlying problem, not just treat the symptom.


2006-Sep-02 02:48:57 by anonymous:
I have no interest in fixing bugs in Tcl itself on Cygwin. I just want to reliably build and test SQLite. The proposed fix is purely pragmatic and is intended only for the test harness. Indeed, when dealing with testing only, the fix is not Cygwin-specific and would work on any platform. The test harness under stock Cygwin as it stands simply does not work for 64 bit values. When you see such a failure you assume that SQLite is in error. Perhaps a compromise can be made and the code fix in question can be wrapped in #ifdef SQLITE_TESTFIXTURE or equivalent instead of #ifdef __CYGWIN__. I would hate to see someone else waste any time on this trivially fixable issue.


2006-Sep-02 13:27:08 by drh:
Perhaps you could put an "if" statement in the test scripts that skipped over the tests that do not work if running under cygwin. You can probably figure out if you are running under cygwin by looking at elements of the tcl_platform array.


2006-Sep-02 13:48:56 by drh:
I retract my previous suggestion. I do not want such patches in the SQLite source tree.

I will resist any patches such as shown here because they are really hacks to work around a faulty Tcl build on Cygwin. The correct way to fix this is to fix the Tcl build for Cygwin. This is probably as simple as download a copy of Tcl and recompiling. I'm curious to know why the default Tcl build for Cygwin only supports 32-bit integers. Is there some problem with 64-bit integer support on Cygwin?

The patch shown in the Description section above is not good because it presumes that Cygwin will always be broken. I think a better assumption is that Cygwin will get fixed. And I do not want to cripple the TCL interface to work around a bug that is not related to SQLite and which might not exist on every system. That is so wrong.

I will be willing to put in a test that checks for the cygwin brokenness and prints a warning to the user. Perhaps something like this:

   if {"*[db eval {SELECT 10000000000}]*"!="*10000000000*"} {
      puts "*********** WARNING *************"
      puts "Your build of TCL only supports 32-bit integers."
      puts "This will cause many test failures.  To run these"
      puts "tests you must install a version of TCL that supports"
      puts "64-bit integers."
      exit
   }

The question is, does that test correctly detect the broken Cygwin? Since I have no ready access to a windows machine, I have no way of testing it.


2006-Sep-02 14:06:28 by anonymous:
Then you would have an on-going maintenance issue with future tests. If'ing out valid tests just masks the problem and defeats the purpose of having a test regression suite. If a test fails legitimately, it should be reported as such. But these particular 64-bit tests work correctly if the simple proposed patch to the test harness is checked in. There is nothing wrong with the tests themselves - just the test harness on certain platforms for which Tcl does support 64-bit integers for whatever reason. Is the purpose of the test suite to test SQLite or Tcl implementations?

I know that Cygwin is a considered a tier "C" platform for SQLite, but appreciate that from a Cygwin environment me and many others have reported at least couple of dozen non-platform-specific SQLite bugs over the past year. You probably have as many or more Cygwin users on the mailing list than Mac OSX users. Why put up artificial ideological roadblocks?


2006-Sep-02 14:10:35 by anonymous:
Please do not check the "Your build of TCL only supports 32-bit integers". It is couter-productive to exit when the great majority of tests will pass. Such a check will basically exclude stock Cygwin installs from testing SQLite. Given the choice between having a broken test harness and this TCL 32-bit check, it is more useful to have a broken test harness.


2006-Sep-04 01:25:00 by anonymous:
Cygwin Tcl 8.5 64-bit integer math bug report

Cygwin Tcl 8.5a5 64-bit integer math fix

 
1954 code active 2006 Sep anonymous Unknown 2006 Sep   1 1 Dual Core Processor Lockup edit
I seem to be seeing a problem with dual core processors in the the Open call is locking and does not release or throw an exception. It does not occur every time, but occurs around 50% of the time. I have not seen the problem on non dual core processors.
2006-Sep-02 21:06:38 by anonymous:
This ticket is way too vague to be actionable. What operating system? AMD or Intel? What specific version of SQLite? Was the library precompiled or did you compile it yourself?

Personally, I can report no errors or problems with dual-core CPU's on Windows XP using an AMD X2 4400+ dual-core CPU. Tested with both a 32-bit build and a 64-bit build of SQLite on x64 Windows.

 
725 new active 2004 May anonymous   2006 Aug anonymous 5 5 Implement ATTACH LIB comand edit
It would be nice to mantain some functions in separate library, and be able to reuse in command line sqlite or in other projects. Syntax should be something like:

ATTACH LIB "some_lib_name"

some_lib_name should export function like

RegisterSQLiteFunctions

 
1948 code active 2006 Aug anonymous Shell 2006 Aug   2 3 Double quotes are not escaped in csv mode edit
If text is exported using "csv" mode, double quotes in strings are not escaped. Generally double-quotes in a quoted field in CSV should be escaped by repeate. I.e., 'This is a "test".' could be about as "This is a ""test""."

This doesn't appear to be the behavior SQLite uses, so, in the meantime, I'll have to export my data using another method and then transform that data into CSV for my import script.

 
1947 code active 2006 Aug anonymous Shell 2006 Aug   3 3 ".mode insert" works bad with BLOBs edit
.mode insert displays BLOBs as strings, which isn't very good for embedded NULs.

Having output more like the one from .dump would be better, IMO.

  sqlite> select * from t;
  INSERT INTO table VALUES('');
  sqlite> .dump
  BEGIN TRANSACTION;
  CREATE TABLE t(f BLOB);
  INSERT INTO "t" VALUES(X'0041');
  COMMIT;
 
1938 build active 2006 Aug anonymous   2006 Aug   4 5 Cygwin compilation of v3 fails edit
make install leads to this:

[UTEKLIFEBOOK] ~/MyDocuments/dl/sqlite-3.3.7 > make install tclsh ./tclinstaller.tcl 3.3 can't read "env(DESTDIR)": no such variable

2006-Aug-25 13:24:19 by anonymous:
Looking at your log file you can see that sqlite3.exe was built successfully. Tcl is an optional component of SQLite. All you need is:

  ./configure && make sqlite3.exe

Or install TCL via Cygwin setup.exe.


2006-Sep-26 17:02:54 by anonymous:
Why not build with this?

  • ./configure --disable-tcl
  • make
  • make install

It works for me.

 
1942 new active 2006 Aug anonymous   2006 Aug   4 4 select without left join with the *= operator edit
  Enhancement request for support *=/=* operator in select queries.
  It's more easier to write SQL queries without left/right join by using *=/=* operator.

  Mentionned on Unsupported Wiki page :

http://www.sqlite.org/cvstrac/wiki?p=UnsupportedSql

    * 2006.03.06 : select without left join with the *= operator

      SELECT t1.code, t2.code
      FROM table1 t1, table2 t2
      WHERE t1.t2_ref_id *= t2.id

      This is Sybase ASE syntax. Use LEFT JOIN or RIGHT JOIN instead.
 
518 code active 2003 Dec anonymous Unknown 2006 Aug   3 4 PRIMARY KEY does not imply UNIQUE and NOT NULL edit
According to SQL92 and Dr. Hipp, PRIMARY KEY should imply both NOT NULL and UNIQUE. It does not. The following statements exhibit the problem:

        CREATE TABLE User (     Name VARCHAR (40),
                                UID INTEGER,
                                DeviceID VARCHAR (64),
                                PRIMARY KEY (Name, UID, DeviceID) );
        INSERT INTO User (Name,DeviceID) VALUES ('Michael','Test');
        INSERT INTO User (Name,DeviceID) VALUES ('Michael','Test');

Gets me two rows with ('Michael',NULL,'Test').

2005-Jan-03 01:46:01 by drh:
See http://www.sqlite.org/nulls.html. SQLite considers NULLs to be distinct. PostgreSQL, Oracle, MySQL, and Firebird all work this way. Informix and MS-SQL work differently.

Works as designed.


2006-Aug-26 07:29:58 by anonymous:

as far as I can see, the closing remark applies to the UNIQUEness of NULL values, not the NOT NULLness of PRIMARY KEYs (or part of that).

let's see what some engines do:

PostgreSQL:

biblioteca=> CREATE TABLE test (
    Name VARCHAR (40),
    UID INTEGER,
    DeviceID VARCHAR (64),
    PRIMARY KEY (Name, UID, DeviceID) );
NOTICE:  CREATE TABLE / PRIMARY KEY will create implicit index "test_pkey" for table "test"
CREATE TABLE
biblioteca=> insert into test(Name, DeviceID) values ('Michael', 'test');
ERROR:  null value in column "uid" violates not-null constraint

MySQL:

mysql> CREATE TABLE test (
    Name VARCHAR (40),
    UID INTEGER,
    DeviceID VARCHAR (64),
    PRIMARY KEY (Name, UID, DeviceID) );
Query OK, 0 rows affected (0.07 sec)

mysql> insert into test(Name, DeviceID) values ('Michael', 'test');
Query OK, 1 row affected (0.11 sec)

mysql> insert into test(Name, DeviceID) values ('Michael', 'test');
ERROR 1062 (23000): Duplicate entry 'Michael-0-test' for key 1
that is: it MySQL is also not allowing NULL primary keys and is silently replacing NULL values with 0.

Oracle:

CREATE TABLE test (
    Name VARCHAR (40),
    UniqueID INTEGER,
    DeviceID VARCHAR (64),
    PRIMARY KEY (Name, UniqueID, DeviceID) );
insert into test(Name, DeviceID) values ('Michael', 'test');
ORA-01400: impossibile inserire NULL in ("IBOTEST"."TEST"."UNIQUEID")
I hope no-one will disagree if I reopen the ticket.


2006-Aug-29 11:51:00 by anonymous:
behaviour with the patch:

  sqlite> CREATE TABLE User ( Name VARCHAR (40),
   ...>                       UID INTEGER,
   ...>                       DeviceID VARCHAR (64),
   ...>                       PRIMARY KEY (Name, UID, DeviceID) );
  sqlite> INSERT INTO User (Name,DeviceID) VALUES ('Michael','Test');
  SQL error: User.UID may not be NULL
  sqlite> INSERT INTO User (Name,DeviceID) VALUES ('Michael','Test');
  SQL error: User.UID may not be NULL


2006-Aug-29 13:10:37 by drh:
To my great surprise, I discovered that SQLite has allowed NULL values in PRIMARY KEY columns for a very long time. This is a bug. But the bug has been in the code for so long, that we fear fixing it might break legacy applications. So for now we have chosen to merely document the fact that the bug exists and to warn developers not to depend on it since it might be fixed at some unspecified point in the future.


2006-Aug-29 19:54:12 by anonymous:
Can you put a SQLITE_ENABLE_NOTNULL_PRIMARYKEYS #ifdef and apply this patch to the official tree? This may solve the problems with legacy code and solve the SQL issue
 
1941 code active 2006 Aug anonymous   2006 Aug   1 1 Unrevolved _sqlite3ExprCodeAndCache with SQLITE_OMIT_TRIGGER edit
If SQLITE_OMIT_TRIGGER is set, linker complains about an unresolved _sqlite3ExprCodeAndCache symbol.

sqlite3ExprCodeAndCache is defined in expr.c and wrapped with #ifndef SQLITE_OMIT_TRIGGER.

However, references in

  insert.c, line 536
  update.c, line 348 and 362

are not wrapped with #ifndef SQLITE_OMIT_TRIGGER.

I followed the suggestion quoted below (posted earlier to this list) without avail.

Is it safe (or even required?) to change sqliteInt.h to

  #ifndef SQLITE_OMIT_TRIGGER
  void sqlite3ExprCodeAndCache(Parse*, Expr*);
  #else
  # define sqlite3ExprCodeAndCache(A,B)
  #endif

In the mailing list, DRH argued that the above change will probably fail and suggested that a safer fix would be to remove the #ifndef SQLITE_OMIT_TRIGGER from around the sqlite3ExprCodeAndCache function.

2006-Oct-12 17:35:32 by anonymous:
The problem is still present in 3.3.8. Removing the

  #ifndef SQLITE_OMIT_TRIGGER

from around the

  sqlite3ExprCodeAndCache

function seems to fix it. Could you commit this?

 
1939 event active 2006 Aug anonymous Unknown 2006 Aug   2 1 SQLite hangs with WHERE condition in an SELECT with joined table edit
I have an database with 7 Tables. I wanted to make an SELECT in which i join the other Tables with an LEFT OUTER JOIN. That works but when i add an WHERE clause on an Joined Table my Application hangs. The sqlite3.exe hangs at the same query.

My Database: CREATE TABLE MMBundesland (ID INTEGER, Bezeichnung varchar(255), Kennung INTEGER); CREATE TABLE MMCoords (ID INTEGER, Laenge DOUBLE, Breite DOUBLE, System varchar(50)); CREATE TABLE MMKFZ (ID INTEGER, Bezeichnung varchar(255)); CREATE TABLE MMKategorie (ID INTEGER,Bezeichnung varchar(255)); CREATE TABLE MMKreis (ID INTEGER, Bezeichnung varchar(255), Kennung INTEGER); CREATE TABLE MMOrte (ID INTEGER, CoordID INTEGER, Name varchar(255), KategorieID INTEGER, KreisID INTEGER, BundeslandID INTEGER, KfzID INTEGER, DefPLZID INTEGER, Kennung INTEGER); CREATE TABLE MMPLZ (ID INTEGER, PLZ varchar(10), CoordID INTEGER, OrtID INTEGER);

The Query was:

SELECT o.ID, o.Name,p.PLZ as DefPLZ,k.Bezeichnung,b.Bezeichnung as Bundesland, p.ID AS PLZID FROM MMOrte o LEFT OUTER JOIN MMPLZ p ON o.DefPLZID = p.ID OR o.ID = p.OrtID LEFT OUTER JOIN MMKategorie k ON o.KategorieID = k.ID LEFT OUTER JOIN MMBundesland b ON o.BundeslandID = b.ID LEFT OUTER JOIN MMPLZ p2 ON o.ID = p2.OrtID WHERE (p.PLZ like '72141%') ORDER BY o.Name

Wenn I use: SELECT o.ID, o.Name,p.PLZ as DefPLZ,k.Bezeichnung,b.Bezeichnung as Bundesland, p.ID AS PLZID FROM MMPLZ p LEFT OUTER JOIN MMOrte o ON o.DefPLZID = p.ID OR o.ID = p.OrtID LEFT OUTER JOIN MMKategorie k ON o.KategorieID = k.ID LEFT OUTER JOIN MMBundesland b ON o.BundeslandID = b.ID LEFT OUTER JOIN MMPLZ p2 ON o.ID = p2.OrtID WHERE (p.PLZ like '72141%') ORDER BY o.Name

The Query Works.

It seems to work if i use in WHERE Clause one Column from the Table neer the FROM keyword.

The main Problem is that SQLite hangs... I will generate my Query so that it works...

2006-Aug-25 13:32:21 by anonymous:
"Hanging" is not very descriptive. Your query is working but now it either makes poor use of indexes or is doing full table scans, or possibly an unintended cross join.


2007-Jan-10 16:31:41 by anonymous:
I am seeing the same problem. It works on a limit of 1 and then fails on a limit of 2.


2007-Jan-10 16:49:09 by drh:
If you would like me to work on this, please send a reproducible test case.
 
1928 new active 2006 Aug anonymous Parser 2006 Aug   2 2 NULL rowid in views and subqueries edit
$ cat rowid_on_views.sql

  create table abc(a,b,c);
  insert into abc values(4,5,6);
  insert into abc values(3,2,1);
  create view vv as select * from abc;
  select 'select rowid, * from abc;';
  select rowid, * from abc;
  select 'select rowid, * from (select * from abc);';
  select rowid, * from (select * from abc);
  select 'select rowid, * from vv;';
  select rowid, * from vv;

$ ./sqlite337 < rowid_on_views.sql

  select rowid, * from abc;
  1|4|5|6
  2|3|2|1
  select rowid, * from (select * from abc);
  |4|5|6
  |3|2|1
  select rowid, * from vv;
  |4|5|6
  |3|2|1
2006-Aug-21 09:09:15 by anonymous:
The point is simply that a view or subquery does not (meaning "cannot") have a rowid - just contemplate the case where your view takes the max(..) or count(..) of something.

You would get the behaviour that you seem to expect when you declare your original table as

    create table abc(rowid integer primary key, a,b,c);


2006-Aug-21 14:35:04 by anonymous:
Not sure what you're talking about. Your example does not work.

Many databases support the concept of "row id's" on views. It is a very useful construct.

  -- working Oracle example

  create table abc(a integer, b integer ,c integer);
  insert into abc values(4,5,6);
  insert into abc values(3,2,1);
  create view abc_vu as select * from abc;
  select rownum, a, b, c  from (select * from abc_vu);

  ROWNUM	A	B	C
  1	4	5	6
  2	3	2	1

  -- another Oracle example
  select rownum, v1.*, v2.*  from (select * from abc_vu) v1, (select * from abc_vu) v2;

  ROWNUM	A	B	C	A	B	C

  1	4	5	6	4	5	6
  2	3	2	1	4	5	6
  3	4	5	6	3	2	1
  4	3	2	1	3	2	1


2006-Aug-22 05:32:26 by anonymous:
Hm, rowid (original ticket) and rownum (3rd remark) are very different things. I've tested the following with Oracle:

   create table abc(a integer, b integer ,c integer);
   insert into abc values(4,5,6);
   insert into abc values(3,2,1);
   insert into abc values(4,5,10);
   create view abc_vu as select max(a) as amax,b,count(*) as cnt from abc group by b;

Now

  SQL> select rownum, amax,b,cnt from (select * from abc_vu);

      ROWNUM	 AMAX	       B	CNT
  ---------- ---------- ---------- ----------
  	 1	    3	       2	  1
  	 2	    4	       5	  2

whereas

   SQL> select rowid, amax,b,cnt from (select * from abc_vu);
   select rowid, amax,b,cnt from (select * from abc_vu)
                                             *
   ERROR at line 1:
   ORA-01446: cannot select ROWID from view with DISTINCT, GROUP BY, etc.


2006-Aug-22 05:36:03 by anonymous:
PS: selecting rowid from views without "group by" is supported by Oracle:

  SQL> create view abc_vu2 as select * from abc;

  View created.

  SQL>  select rowid,a,b,c from (select * from abc_vu2);

  ROWID			    A	       B	  C
  ------------------ ---------- ---------- ----------
  AAADmaAAFAAAFNzAAA	    4	       5	  6
  AAADmaAAFAAAFNzAAB	    3	       2	  1
  AAADmaAAFAAAFNzAAC	    4	       5	 10
 
1935 event active 2006 Aug anonymous Parser 2006 Aug   4 4 GROUP BY and ORDER BY constant expressions valid? edit
I'm not sure what other databases do in this situation, or whether a constant expression should be deemed a constant in ORDER BY and GROUP BY and result in an error.

  CREATE TABLE t1(a,b);
  INSERT INTO "t1" VALUES(1, 2);
  INSERT INTO "t1" VALUES(1, 3);
  INSERT INTO "t1" VALUES(2, 3);
  INSERT INTO "t1" VALUES(2, 4);
  INSERT INTO "t1" VALUES(2, 5);

  -- as expected
  select a, b from t1 group by 0;
  SQL error: GROUP BY column number 0 out of range - should be between 1 and 2

  -- ???

  select a, b from t1 group by 0/1;
  2|5

  select a, b from t1 group by 0.0;
  2|5

  select a, b from t1 order by 13/2;
  1|2
  1|3
  2|3
  2|4
  2|5

  select a, b from t1 group by 3/9 order by 1/9, 7.0, 0.4-3;
  2|5
2006-Aug-26 07:26:06 by anonymous:
The integer is the column number.
 
1934 doc active 2006 Aug anonymous   2006 Aug   5 5 Documentation about default file format edit
Default file format has changed from 3.3.7.

This is mentionned here :

http://www.sqlite.org/changes.html and http://www.sqlite.org/formatchng.html

but not here :

http://www.sqlite.org/pragma.html (PRAGMA legacy_file_format)

 
1933 new active 2006 Aug anonymous   2006 Aug   5 4 Add different date's separators edit
A time string can be in any of the following formats:

   1. YYYY-MM-DD
   2. YYYY-MM-DD HH:MM
   3. YYYY-MM-DD HH:MM:SS
   4. YYYY-MM-DD HH:MM:SS.SSS
   5. YYYY-MM-DDTHH:MM
   6. YYYY-MM-DDTHH:MM:SS
   7. YYYY-MM-DDTHH:MM:SS.SSS
   8. HH:MM
   9. HH:MM:SS
  10. HH:MM:SS.SSS
  11. now
  12. DDDD.DDDD

Add possibility to have date's separators different of dash

Examples :

  YYYY.MM.DD HH:MM:SS
  YYYY/MM/DD HH:MM:SS
 
1907 new active 2006 Aug anonymous   2006 Aug   4 4 date and time types edit
I would like to have datetime types in sqlite (date, datetime, interval).

as far as I understood of sqlite, the declared type used in the create table statement is just a hint, and the data is stored with its own type description. this makes it impossible to store a date into any field, unless type checking, conversion and validation is performed by the client side.

implementing type checking and conversion on the client side would mean that the dynamic typing in sqlite goes lost for that applications.

2006-Aug-05 13:49:35 by anonymous:
Also, the way SQLite currently stores datetimes is not space efficient (stored as text) and there is no standard format between different SQLite applications. But I can't see a way to improve this situation without breaking backwards compatability.
 
1906 new active 2006 Aug anonymous Unknown 2006 Aug   1 2 date.c line 417 uses non-thread safe localtime() call edit
The date.c file uses a call to localtime() that is not threadsafe. Could it be replaced with localtime_r where supported.
2006-Aug-04 16:17:22 by anonymous:
Please apply this patch to make SQLite threadsafe:

  Index: configure.ac
  ===================================================================
  RCS file: /sqlite/sqlite/configure.ac,v
  retrieving revision 1.26
  diff -u -r1.26 configure.ac
  --- configure.ac      3 Jun 2006 18:02:18 -0000       1.26
  +++ configure.ac      4 Aug 2006 16:05:21 -0000
  @@ -669,6 +669,11 @@
   #
   AC_CHECK_FUNC(usleep, [TARGET_CFLAGS="$TARGET_CFLAGS -DHAVE_USLEEP=1"])

  +#########
  +# Figure out whether or not we have a "localtime_r()" function.
  +#
  +AC_CHECK_FUNC(localtime_r, [TARGET_CFLAGS="$TARGET_CFLAGS -DHAVE_LOCALTIME_R=1"])
  +
   #--------------------------------------------------------------------
   # Redefine fdatasync as fsync on systems that lack fdatasync
   #--------------------------------------------------------------------
  Index: src/date.c
  ===================================================================
  RCS file: /sqlite/sqlite/src/date.c,v
  retrieving revision 1.54
  diff -u -r1.54 date.c
  --- src/date.c        31 Jan 2006 20:49:13 -0000      1.54
  +++ src/date.c        4 Aug 2006 16:05:21 -0000
  @@ -393,7 +393,7 @@
   static double localtimeOffset(DateTime *p){
     DateTime x, y;
     time_t t;
  -  struct tm *pTm;
  +  struct tm tmLocaltime;
     x = *p;
     computeYMD_HMS(&x);
     if( x.Y<1971 || x.Y>=2038 ){
  @@ -411,15 +411,20 @@
     x.validJD = 0;
     computeJD(&x);
     t = (x.rJD-2440587.5)*86400.0 + 0.5;
  -  sqlite3OsEnterMutex();
  -  pTm = localtime(&t);
  -  y.Y = pTm->tm_year + 1900;
  -  y.M = pTm->tm_mon + 1;
  -  y.D = pTm->tm_mday;
  -  y.h = pTm->tm_hour;
  -  y.m = pTm->tm_min;
  -  y.s = pTm->tm_sec;
  -  sqlite3OsLeaveMutex();
  +//#define HAVE_LOCALTIME_R
  +#ifdef HAVE_LOCALTIME_R
  +  localtime_r(&t, &tmLocaltime);
  +#else
  +  /* sqlite3OsEnterMutex(); not needed due to HAVE_LOCALTIME_R */
  +  tmLocaltime = *localtime(&t);
  +  /* sqlite3OsLeaveMutex(); not needed due to HAVE_LOCALTIME_R */
  +#endif
  +  y.Y = tmLocaltime.tm_year + 1900;
  +  y.M = tmLocaltime.tm_mon + 1;
  +  y.D = tmLocaltime.tm_mday;
  +  y.h = tmLocaltime.tm_hour;
  +  y.m = tmLocaltime.tm_min;
  +  y.s = tmLocaltime.tm_sec;
     y.validYMD = 1;
     y.validHMS = 1;
     y.validJD = 0;


2006-Aug-04 17:48:38 by anonymous:
The same type of autoconfig and src/date.c code changes have to be done for the not-threadsafe gmtime() function. The current implementation (even with the mutex) is not threadsafe in all cases.
 
1893 code active 2006 Jul anonymous   2006 Aug   3 3 sqlite doesn't use indexes containing primary key in prim. key selects edit
I have table:

    CREATE TABLE IF NOT EXISTS 'customers' (
       'rowid'      INTEGER       PRIMARY KEY AUTOINCREMENT NOT NULL,
       'fname'      CHAR(40)      NOT NULL,
       'sname'      CHAR(40)      NOT NULL,
       'birthno'    CHAR(11)      NULL)

And index:

    CREATE UNIQUE INDEX IF NOT EXISTS 'idx_customers_sname'
       ON 'customers' (
       'sname'  ASC,
       'fname'  ASC,
       'rowid'  ASC
       );

Command

   SELECT * FROM customers ORDER BY sname ASC, fname ASC, rowid ASC;

doesn't use created index.

Command

   SELECT * FROM customers ORDER BY sname ASC, fname ASC;

uses index idx_customers_sname.

I think this is a bug, but maybe (i don't know), it is by desing. If I don't specify rowid in ORDER BY, is the resultset ordered by rowid anyway?

2006-Jul-24 16:02:29 by anonymous:
In SQL single quotes are used around string literals, and double quotes are used around identifiers where required to enclose keywords and/or embedded spaces. In your case no quotes are required at all because your table and column identifiers are continuos (i.e. do not contain embedded spaces) non-keyword names. If you are going to include unnecessary quotes then you should at least use the correct ones.

    CREATE TABLE IF NOT EXISTS "customers" (
       "rowid"      INTEGER       PRIMARY KEY AUTOINCREMENT NOT NULL,
       "fname"      CHAR(40)      NOT NULL,
       "sname"      CHAR(40)      NOT NULL,
       "birthno"    CHAR(11)      NULL);

    CREATE UNIQUE INDEX IF NOT EXISTS "idx_customers_sname" ON "customers" ( "sname" ASC, "fname" ASC, "rowid" ASC );

Aside from that, this does look like a bug. SQLite is doing an unnecessary sort for the first query, and correctly using the index for the second.

I suspected that it might be related to handling of the special column name rowid, but it does the same thing if rowid is replaced with a more generic name like id as shown below.

    SQLite version 3.3.6
    Enter ".help" for instructions
    sqlite> CREATE TABLE IF NOT EXISTS "customers" (
       ...>    "id"      INTEGER       PRIMARY KEY AUTOINCREMENT NOT NULL,
       ...>    "fname"      CHAR(40)      NOT NULL,
       ...>    "sname"      CHAR(40)      NOT NULL,
       ...>    "birthno"    CHAR(11)      NULL);
    sqlite>
    sqlite> CREATE UNIQUE INDEX IF NOT EXISTS "idx_customers_sname"
       ...>    ON "customers" ( "sname" ASC, "fname" ASC, "id" ASC );
    sqlite>
    sqlite>
    sqlite> explain query plan SELECT * FROM customers ORDER BY sname ASC, fname ASC
    , id ASC;
    0|0|TABLE customers
    sqlite> explain query plan SELECT * FROM customers ORDER BY sname ASC, fname ASC
    ;
    0|0|TABLE customers WITH INDEX idx_customers_sname ORDER BY
    sqlite>
    sqlite> .explain on
    sqlite>
    sqlite> explain SELECT * FROM customers ORDER BY sname ASC, fname ASC, id ASC;
    addr  opcode          p1          p2          p3
    ----  --------------  ----------  ----------  ---------------------------------
    0     OpenVirtual     1           5           keyinfo(3,BINARY,BINARY)
    1     Goto            0           34
    2     Integer         0           0
    3     OpenRead        0           2
    4     SetNumColumns   0           4
    5     Rewind          0           19
    6     Rowid           0           0
    7     Column          0           1
    8     Column          0           2
    9     Column          0           3
    10    MakeRecord      4           0
    11    Column          0           2
    12    Column          0           1
    13    Rowid           0           0
    14    Sequence        1           0
    15    Pull            4           0
    16    MakeRecord      5           0
    17    IdxInsert       1           0
    18    Next            0           6
    19    Close           0           0
    20    OpenPseudo      2           0
    21    SetNumColumns   2           4
    22    Sort            1           32
    23    Integer         1           0
    24    Column          1           4
    25    Insert          2           0
    26    Column          2           0
    27    Column          2           1
    28    Column          2           2
    29    Column          2           3
    30    Callback        4           0
    31    Next            1           23
    32    Close           2           0
    33    Halt            0           0
    34    Transaction     0           0
    35    VerifyCookie    0           2
    36    Goto            0           2
    37    Noop            0           0
    sqlite> explain SELECT * FROM customers ORDER BY sname ASC, fname ASC;
    addr  opcode          p1          p2          p3
    ----  --------------  ----------  ----------  ---------------------------------
    0     Noop            0           0
    1     Goto            0           21
    2     Integer         0           0
    3     OpenRead        0           2
    4     SetNumColumns   0           4
    5     Integer         0           0
    6     OpenRead        2           4           keyinfo(3,BINARY,BINARY)
    7     Rewind          2           18
    8     RowKey          2           0
    9     IdxIsNull       0           17
    10    IdxRowid        2           0
    11    MoveGe          0           0
    12    Rowid           0           0
    13    Column          0           1
    14    Column          0           2
    15    Column          0           3
    16    Callback        4           0
    17    Next            2           8
    18    Close           0           0
    19    Close           2           0
    20    Halt            0           0
    21    Transaction     0           0
    22    VerifyCookie    0           2
    23    Goto            0           2
    24    Noop            0           0
    sqlite>


2006-Aug-03 17:33:51 by anonymous:
Thank you for clarification of single and double quotes usage. I will drop the quotes completely since using double quotes is little bit annoying inside C string literals...

It seems that the problem with index and sorting on primary key is independent of primary key column name. In fact, previously I was using "id" :-) and the result was the same as you mentioned.

 
1905 new active 2006 Aug anonymous VDBE 2006 Aug   5 4 Get bound param as sqlite3_value? edit
It doesn't seem to be possible to get the values, that have been bound to a prepared statement, after binding them.

What's the use you ask?

In complex scenarios the code might do complex binds (binding a NULL or an int/text depending on application logic is a simple example). A wrapper around sqlite3_step can use the requested new function to iterate over all parameters and dump them to the debug log on errors.

Looking at the vdbe stuff, this seems to be a straight forward function, that will just return a pointer to vdbe->aVar[i-1] after range checking.

 
1902 new active 2006 Jul anonymous Parser 2006 Jul   5 3 allow alternate INSERT syntax using SET foo=bar edit
Please support the alternate INSERT statement syntax in your parser that several other database products (and possibly the SQL standard) supports where you can say things like:

  INSERT INTO foo SET bar='hello', baz='world';

The main part of the statement, which maps field names to values, then is a lot easier for users to work with, either manually or in SQL generators, and can be reused between INSERT and UPDATE statements.

This should be simple to add, and I strongly suggest having it in before the next release (eg, as part of 3.3.7).

Thank you. -- Darren Duncan

 
1901 code active 2006 Jul anonymous Unknown 2006 Jul adamd 2 2 problem in select request with a alias table edit
I have a table with 3 columns : c0, c1 and c2

My request is: select * from (select *, 'test' as new_col from table) as tmp inner join (select 'test' as new_col) as tmp1 on tmp.new_col = tmp1.new_col;

The column's name as a result of this request (sqlite 3-3.3.6) is: |tmp.table.c0|tmp.table.c1|tmp.table.c2|tmp.new_col|tmp1.new_col

In sqlite 3-3.2.7, the column's name is: |c0|c1|c2|collected|new_col|new_col

Before this version, my request ran on mysql, postgresql and sqlite. Now I don't have the possibility of using this request with the new sqlite version.

2006-Jul-31 10:11:59 by anonymous:
sorry in In sqlite 3-3.2.7, the column's name is: |c0|c1|c2|new_col|new_col


2007-Jan-08 14:52:43 by anonymous:
I had a similar problem with SQLite in PHP, see my bug report here: http://bugs.php.net/bug.php?id=40064
 
1900 code active 2006 Jul anonymous Unknown 2006 Jul a.rottmann 1 1 CURRENT_TIMESTAMP keyword not inserting UTC date in column edit
This is the schema for my table.

create table char (player varchar(64) NOT NULL default '~', name varchar(64) NOT NULL default '~', date timestamp NOT NULL default current_timestamp)

Whenever an insert is made to the table the column 'date' does get a UTC timestamp, it gets a string value 'current_timestamp'. Is my schema wrong?

2006-Jul-30 22:31:06 by anonymous:
*doesnt get a UTC timestap


2006-Jul-31 00:38:49 by anonymous:
Works fine for me. What's the exact syntax of your INSERT statement?
 
1898 doc active 2006 Jul anonymous   2006 Jul   5 4 sqlite3_progress_handler still marked experimental in documentation edit
According to DRH's posting on the sqlite-user mailing list, sqlite3_progress_handler is no longer experimental and the note in the documentation should be removed. Here's the ticket to track this issue...
 
1896 new active 2006 Jul drh   2006 Jul   1 1 All extensions to be statically linked edit
The new LoadableExtensions mechanism is great for loading new extensions out of external files. But we also need the ability to statically link extensions with an application and load them into database connections as needed.

One possible solution would be a new API that takes a fake filename and a function pointer. Any attempt to call sqlite3_load_extension() on that fake filename by any SQLite connection invokes the function pointer rather than actually opening a shared library with that filename.

2006-Jul-26 11:30:28 by anonymous:
In order to statically link extensions to the application, they'll all need to have unique entry function names. Besides meaning that extensions couldn't be statically linked without possible modification (I somehow suspect "sqlite3_extension_init" will be a popular name), it ensures that the extensions are uniquely identified in the application function namespace.

So I think it should just be necessary to have a function to register these function names and pointers, and sqlite3_load_extension() with a NULL zFile and appropriate zProc would have no trouble finding them in the "registry".

A trivial modification to the SQL load_extension() function to take just the function name would be a bonus.

c.


2006-Jul-26 11:39:59 by anonymous:
Something else which would be handy for statically linking extensions... It might be useful for a developer to directly call the extension entry function instead of formally registering it (i.e. they only want the extension available to a specific DB handle), in which case there should be a way to provide the sqlite3_api_routines structure to the entry function. Just making sure it's mentioned and formally documented in sqlite.h should be sufficient.
 
1895 doc active 2006 Jul anonymous   2006 Jul anonymous 4 3 IS operator not documented edit
Hello,

I tried to work with varchar fields having NULL values. However, I was not able to find the right operator to catch such values, and none of the operators mentionend in datatype3.html seemed to help. Finally I tried "is null" (or IS NOT NULL) and well, that seems to do what I want. But there are questions:

  • is this an intended feature?

  • if yes, can I rely on having it in the future versions?

  • if this is not an intended feature, what is the right way to match NULL values?

  • if that's just a documentation problem, will the documentation be updated?
2006-Jul-19 19:02:17 by anonymous:
It's part of the SQL standard. "IS" isn't really an operator; it's an optional token that's part of the NULL and NOT NULL predicates.
 
1890 code active 2006 Jul anonymous Unknown 2006 Jul   3 4 double quotes ("") in a query are ambiguous edit
sqlite> SELECT "uuid" FROM objects LIMIT 1;
b43c9cdc-0dc8-11db-9475-080020a846a9
sqlite> SELECT "uuidx" FROM objects LIMIT 1;
uuidx

The objects table has a column named uuid; it does not have a column named uuidx. The behaviour of "" depends on whether the contents are a valid column name, and I cannot see when this is desirable behaviour.

(I know that `` and '' are the right quotes to use, by the way - I'm just pointing out that "" can surprise people a lot and should probably be fixed or removed)

2006-Jul-13 13:39:37 by anonymous:
The current behaviour is an SQL standard thing, so SQLite implements it for the sake of compatibility.

Certainly most people agree with you that it's a bad thing.


2006-Jul-13 16:56:05 by anonymous:
Perhaps what we need then is a big fat warning in the manual :) - Peter


2006-Jul-17 00:06:41 by drh:
Please suggest a specific location in the documentation where I should put a warning about the use of " instead of ' and I will add it.


2006-Jul-27 10:30:12 by anonymous:
lang_expr.html seems like an obvious choice; perhaps also a sidenote on FAQ question 16 - Peter
 
1885 code active 2006 Jul anonymous Shell 2006 Jul   2 3 sqlite3 .mode insert and .dump do not list column names for selects edit
In sqlite3 .mode insert does not list column names for selects - it should. This makes dumping selected columns from tables when intending to add or delete columns problematic. .dump doesn't list column names either, IMHO it should. Consider

sqlite> .mode tabs
sqlite> select * from users;
ed 2006-07-05 52
sqlite> .mode insert
sqlite> select abs_tgt from users;
INSERT INTO table VALUES(52);
sqlite>

Obviously the workaround is to hand edit the output SQL

2006-Jul-11 10:20:08 by anonymous:
I've just noticed it doesn't include the table name in the INSERT statements either.
 
1884 code active 2006 Jul anonymous   2006 Jul   3 2 pragma table_info caches results from previous query edit
this problem is observed with pysqlite's latest windows build 2.3.2 and others. it does not occur on unix-based builds, which is why I suspect the issue is in sqlite, since pysqlite's code is platform-neutral.

if you get a result from a "pragma table_info()" call, and do not consume all the results, then a subsequent call to the same statement does not return up-to-date results, i.e. if the table had been dropped in between. it behaves as though the results of "pragma table_info" are globally cached somewhere, ignoring the fact that is was executed again.

this test program illustrates the problem:

	from pysqlite2 import dbapi2 as sqlite

	connection = sqlite.connect(':memory:')

	# check for a nonexistent table
	c = connection.execute("pragma table_info(users)")
	row = c.fetchone()
	assert row is None   # its good.

	# now create the table
	connection.execute("""
	create table users (
		foo VARCHAR(10),
		name VARCHAR(40)
	)
	""")

	# do the table_info pragma.  returns two rows
	c = connection.execute("pragma table_info(users)")

	# get the first row
	row = c.fetchone()
	print row

	# but then dont get the second, close out the cursor instead.

	#row2 = c.fetchone()  # uncomment to fully consume both rows, then it works

	c.close()
	c = None

	# rollback too.
	connection.rollback()

	# now drop the table
	connection.execute("DROP TABLE users")

	print "dropped"

	# now it should be gone, right? well it is, but the pragma
	# call starts off with the former result set
	c = connection.execute("pragma table_info(users)")
	row = c.fetchone()
	print row
	assert row is None  # fails.
 
1882 code active 2006 Jul anonymous   2006 Jul   1 1 Wrong algorithm of SQLITE_VERSION_NUMBER calculation edit
The sqlite3.h comment describing how numeric version number is calculated is as follows:

"The SQLITE_VERSION_NUMBER is an integer with the value (X*100000 + Y*1000 + Z). For example, for version "3.1.1beta", SQLITE_VERSION_NUMBER is set to 3001001."

But the value of SQLITE_VERSION_NUMBER is greater than the equation above suggests. The value X*100000 should be changed to X*1000000 (one milion).

 
1878 code active 2006 Jun anonymous CodeGen 2006 Jun   2 3 No index used when specifying alias name in ORDER BY clause edit
Using an alias name in the ORDER BY clause prevents indices from being used in the query for sorting purposes:

For this schema:

  CREATE TABLE t1 (c1, c2);
  CREATE TABLE t2 (c3, c4);
  CREATE INDEX t1_idx ON t1(c2);

the following select query:

  EXPLAIN QUERY PLAN
  SELECT t1.c2 AS col2, t2.c4 AS col4
    FROM t1 LEFT JOIN t2 ON t1.c1=t2.c3 ORDER BY t1.c2;

will indeed use index t1_idx:

  sqlite> EXPLAIN QUERY PLAN
          SELECT t1.c2 AS col2, t2.c4 AS col4
            FROM t1 LEFT JOIN t2 ON t1.c1=t2.c3
           ORDER BY t1.c2;
  0|0|TABLE t1 WITH INDEX t1_idx
  1|1|TABLE t2

However, when using the alias name col2 in the ORDER BY clause, the index won't be used:

  sqlite> EXPLAIN QUERY PLAN
          SELECT t1.c2 AS col2, t2.c4 AS col4
            FROM t1 LEFT JOIN t2 ON t1.c1=t2.c3
           ORDER BY col2;
  0|0|TABLE t1
  1|1|TABLE t2

IMHO, the same index should be used in both queries?

2006-Jun-30 13:54:10 by anonymous:
Not sure whether it's a different issue, but when using a second column in the ORDER BY clause, also no index will be used:

  sqlite> EXPLAIN QUERY PLAN
          SELECT t1.c2 AS col2, t2.c4 AS col4
            FROM t1 LEFT JOIN t2 ON t1.c1=t2.c3 ORDER BY t1.c2, t2.c4;
  0|0|TABLE t1
  1|1|TABLE t2

Personally, I'd expect sqlite to use the t1_idx index as well to fulfill the primary ordering?


2006-Jun-30 16:04:31 by anonymous:
As a workaround try "ORDER BY 1"


2006-Jul-03 08:41:01 by anonymous:
Sorry, I'm not sure how "ORDER BY 1" would be a workaround, when I really need the results to be sorted by table column data...

(I don't want to start a discussion in the bug tracker, so you're welcome to take any suggestions/answers to the sqlite-user mailing list, which I also monitor.)

2006-Jul-03 15:51:11 by anonymous:
I'm not the poster of previous comment, but ORDER BY (n) order by result column index. In your case, using ORDER BY 1, it will be ordered by the first column.


2006-Jul-04 07:33:30 by anonymous:
Thanks for the clarification. This would be a workaround for the first problem mentioned, but when sorting by two columns, still no index will be used, even if using ORDER BY 1,2


2006-Jul-04 21:34:24 by anonymous:
SQLite really needs a way to explicitly state which index(es) to use. Perhaps something similar to Oracle's comment hints.
 
1817 new active 2006 May anonymous   2006 Jun   1 2 Patch to enable SQLite again on OS/2 edit
As we urgently need OS/2 support to be able to build Mozilla applications (Firefox, SeaMonkey, Thunderbird) we have to activate it again in SQLite CVS.

Daniel Lee Kruse ported the two C files with some input from Andy Willis and me (Peter Weilbacher). I hope this is the right way to go about this.

The OS/2 changes from this and from follow-up ticket #1836 were checked in some time ago, so I am marking this fixed.
 
1874 new active 2006 Jun anonymous CodeGen 2006 Jun   2 3 IN is much slower than making separate queries edit
I have a 500,000-row table with the following schema:

	CREATE TABLE foo(bar INTEGER NOT NULL, baz INTEGER NOT NULL, biz INTEGER NULL, buzz INTEGER NULL);
	CREATE INDEX biz ON foo (bar, baz, biz);
	CREATE INDEX buzz ON foo (bar, baz, buzz);

I'm performing the query:

   SELECT * FROM foo WHERE bar IN (0,1) AND baz IN (0,1) AND (biz IN (0,1) OR buzz IN (0,1));

On both Apple's 3.1.3 and a stock 3.3.6 on Mac OS X 10.4.6 PowerPC, this query consistently takes 3 seconds to execute. However, if I unroll the query:

	SELECT * FROM foo WHERE bar=0 AND baz=0 AND biz=0;
	SELECT * FROM foo WHERE bar=0 AND baz=0 AND buzz=0;
	...and so on for the other values of bar and baz...

it takes 0.2 seconds.

I was able to reproduce this with the attached scripts by doing:

	./mkdb > mkdb.sql
	sqlite3 testdb < mkdb.sql
	time sqlite3 testdb < q1 > /dev/null
	time sqlite3 testdb < q2 > /dev/null

The database was recreated between the 3.1.3 and the 3.3.6 testing.

EXPLAIN on the IN query segfaults on both 3.1.3 and 3.3.6, otherwise I'd attach that output. I'll write up a separate bug for that :)

2006-Jun-27 20:52:35 by anonymous:
[3315] fixed the EXPLAIN crash, so I've attached EXPLAIN output for the IN query.


2006-Jun-27 22:01:31 by drh:
On SuSE Linux 10.0 running on a Dell Latitude D600 laptop and using the latest code from CVS, I'm getting times of 1.2s and 0.45s. If I create an additional index:

    CREATE INDEX i2 ON foo(bar,baz,biz,buzz);

then the time for the first query drops to 0.7s. Note, however, the q1 and q2 are very different queries. In particular q2 omits half the rows. The (rough) equivalent of q2 is this:

    SELECT * FROM foo
     WHERE bar IN (0,1)
         AND baz IN (0,1)
        AND (biz=0 or buzz=0);

If I modify q2 so that it includes the biz=1 and buzz=1 cases, its query time increases to 0.7s, the same as q1 with the added index.

Further note that q1 and q2 are still not exactly the same. Q2 includes multiple copies of rows where biz IN (0,1) AND buzz IN (0,1) where q1 only includes such lines once. There are only 120 such lines in the database, but it still a difference.

I will recast this ticket as a request for performance enhancements on queries using the IN operator on a fixed list of values.


2006-Jun-28 01:52:32 by drh:
Note to self: The expression +x IN (1,2,3,...) appears to be faster than +x=1 OR +x=2 OR ... when there are 6 or more terms. With 5 terms or fewer, a string of ORs is faster.
 
1872 code active 2006 Jun anonymous   2006 Jun   4 3 sqlite3_open doesn't support RFC1738 format for filename edit
sqlite3_open only supports UTF-8 encoding as a format for its filename argument (http://www.sqlite.org/capi3ref.html#sqlite3_open).

If your application receives a RFC1738 encoded URL for filename, that has to be UTF-8-encoded for use in SQLite. It would be nice if that could be instead passed directly to sqlite3_open.

Is RFC1738 URL decoding support planned for SQLite?

(RFC1738 link: http://www.cse.ohio-state.edu/cgi-bin/rfc/rfc1738.html)

 
1871 event active 2006 Jun anonymous   2006 Jun   2 3 VACUUM should not change 3.x file format edit
VACUUM should not "upgrade" the file format as it violates the principle of least astonishment. VACUUM upgrading the file format prevents users working with older versions of SQLite 3.x from sharing a common database file with users of more recent versions of the library. At the very least, if a version of SQLite can not produce the same version of the database file after VACUUM, it should do nothing, or perhaps return a warning.
2006-Jun-27 13:25:30 by drh:
What if a user wants to upgrade the file format so that they can take advantage of descending indices, for example? How should they accomplish that? Should they be forced to dump and restore the database?


2006-Jun-27 14:10:16 by anonymous:
Manually dumping the old database and restoring it with a more recent version of SQLite is reasonable given the incompatible nature of the change.


2006-Jun-27 15:30:32 by anonymous:
I agree. Can't SQLite do the equivalent of pragma legacy_file_format=1 on the new database created by the VACUUM command if the current file is in the old format?

The dump and restore operation is usable for updating the file format, but does require two installed versions of SQLite, one old and one new. Couldn't you add a new command or pragma that would do the format upgrade. Perhaps an optional upgrade argument to the VACUUM command that defualts to off could be used. If it is off the format of the new database is unchanged, if it is on, the format is upgraded. The VACUUM command doesn't seem like the obvious place tto look for a format upgrade option though. A new UPGRADE command would be more obvious, even if it is actually implemented by the same routines that do the VACUUM. It may be better to use something a little lower profile than a new command, perhaps a "PRAGMA upgrade_file_format" would be better. It would also allow future extension to provide a format version number that the database is to be upgraded to (ie PRAGMA upgrade_file_format=4).


2006-Jun-27 17:55:23 by anonymous:
it's a simple issue. since SQLite know the file format of database that is opened, on VACUUM it should create a file with the same version (such like a internal _sqlite3_open_ex() call that receive the file format that should be created. since SQLite can read/write those formats, there's no reason for doing a 'file format' upgrade.
 
1869 doc active 2006 Jun anonymous Unknown 2006 Jun anonymous 4 4 Website typo edit
http://www.sqlite.org/capi3ref.html#sqlite3_exec has this:

    "As an example, suppose the query result where this table:"

Instead of "where," "were" should have been used.

 
1867 code active 2006 Jun anonymous BTree 2006 Jun   1 3 Access Violation after set a new page_size edit
An access violation occured on W2K when I try to create a new table in the empty database. There was a following sequence of SQL commands

select count(*)==2 as cnt from sqlite_master where type='table' and tbl_name in ('tbl1', 'tbl2');

so if cnt is equal 0 then I execute command pragma page_size=4096; and then create a new table.

I gess that some of internal structures by this time have been initialized and so when I try to create new table the page_size is lower then needed.

we overwrite memory in the function zeroPage in instruction: memset(&data[hdr], 0, pBt->usableSize - hdr); Size of structure data less then pBt->usableSize

Below result after memset

0:000> dt MemPage 004c3cf0
+0x000 isInit : 0 ''
+0x001 idxShift : 0 ''
+0x002 nOverflow : 0 ''
+0x003 intKey : 0x1 ''
+0x004 leaf : 0x1 ''
+0x005 zeroData : 0 ''
+0x006 leafData : 0x1 ''
+0x007 hasData : 0 ''
+0x008 hdrOffset : 0 ''
+0x009 childPtrSize : 0 ''
+0x00a maxLocal : 0
+0x00c minLocal : 0
+0x00e cellOffset : 0
+0x010 idxParent : 0
+0x012 nFree : 0xf94
+0x014 nCell : 0
+0x018 aOvfl : [5] _OvflCell
+0x040 pBt : (null)
+0x044 aData : (null)
+0x048 pgno : 0
+0x04c pParent : (null)

0012ea50 10006861 004c3cf0 0000000d 00000064 dblited!decodeFlags+0x80 [D:\sqllite\sqlite-3.3.6\btree.c @ 1349]
0012ea70 10006710 004c3cf0 0000000d 004c3cf0 dblited!zeroPage+0xd0 [D:\sqllite\sqlite-3.3.6\btree.c @ 1466]
0012ea8c 10006215 002fd390 002fd390 00000000 dblited!newDatabase+0xf9 [D:\sqllite\sqlite-3.3.6\btree.c @ 2061]
0012eaa0 10052ba0 002f7c30 00000001 0012f0e4 dblited!sqlite3BtreeBeginTrans+0xd6 [D:\sqllite\sqlite-3.3.6\btree.c @ 2141]
0012f0a4 10057cf5 004c3d80 0012f13c 0012f478 dblited!sqlite3VdbeExec+0x2c6d [D:\sqllite\sqlite-3.3.6\vdbe.c @ 2386]
0012f0e4 00412801 004c3d80 0012f1d4 0012f478 dblited!sqlite3_step+0x1db [D:\sqllite\sqlite-3.3.6\vdbeapi.c @ 223]
 
1864 new active 2006 Jun anonymous   2006 Jun drh 3 3 List availabe SQL functions edit
Once LoadableExtensions are implemented it would be really nice to be able to get list of all available SQL functions and number[s] of arguments.
Mostly I'm thinking about a scenario where you'd load someone else's library from SQLite shell. Hopefully those will become common once infrastructure is in place.

I guess SQLite already keeps track of all available functions internally?

2006-Jun-21 19:48:07 by drh:
Perhaps this could be done using a virtual table.


2006-Jun-21 20:15:27 by anonymous:
Yes, this seems to be a really nice fit for virtual tables.

I lurked through the sources briefly, and it seems like all required info is already there, in struct sqlite3. Right?

 
1862 code active 2006 Jun anonymous TclLib 2006 Jun tclguy 1 1 SQLite cannot load/import data from file edit
I found the problem when I tried to load a data file into a table. To reproduce the problem, I got a mini testcase.

DATA FILE - test.dat


    1       0       0
    2       90000   0
    3       366000  0
---------------------------

Log from SQLite:


    khronos-yajun>sqlite3 test
    SQLite version 3.3.6
    Enter ".help" for instructions
    sqlite> create table test (id INT, x1 INT, x2 INT);
    sqlite> .import test.dat test
    test.dat line 1: expected 3 columns of data but found 1
    sqlite> .exit


The problem also exists when I use tcl wrapper (sql copy abort test test.dat).

I looked into the code in src/tclsqlite.c,

In Lines

   1045     nByte = strlen(zSql);
   1046     rc = sqlite3_prepare(pDb->db, zSql, 0, &pStmt, 0);
   1047     sqlite3_free(zSql);

Is the third argument of sqlite3_prepare supposed to be the length of zSql, hence nByte?

Also in lines

   1070     zSql[j++] = ')';
   1071     zSql[j] = 0;
   1072     rc = sqlite3_prepare(pDb->db, zSql, 0, &pStmt, 0);
   1073     free(zSql);

If I change these two places to reflect the length of zSql, I seem to succeed.

Yajun

2006-Sep-27 16:25:47 by anonymous:
This is a duplicate of #1797
 
1861 code active 2006 Jun anonymous Pager 2006 Jun   1 1 Problem in using Triggers and multithreading edit
I am using SQLite3 database with triggers . This database is used by my processing engine which is having 10 threads accessing the same database. Trigger is used to updata and insert records in a table and that very table is also updated by threads.

Processing engine crashes whenever a trigger updates or inserts a record in the table.

Can you tell me how to configure my existing engine to avoid crashing? Is it safe to use trigger?

 
1860 doc active 2006 Jun anonymous Pager 2006 Jun danielk1977 1 1 Problem in using SQLite3 with trigger and multithreading edit
I am using SQLite3 database with triggers . This database is used by my processing engine which is having 10 threads accessing the same database. Trigger is used to updata and insert records in a table and that very table is also updated by threads.

Processing engine crashes whenever a trigger updates or inserts a record in the table.

 
1858 doc active 2006 Jun anonymous   2006 Jun   5 5 Typo: Pearl -> Perl edit
sqlite/www/index.tcl 1.139

  s/Pearl/Perl/g
 
1857 code active 2006 Jun anonymous   2006 Jun   3 4 Can't use ISO 8601 dates with time zone designator 'Z' edit
Date-times of the format 'YYYY-MM-DDTHH:MM:SS.sssZ' do not return a valid date. e.g. datetime('2006-06-19T15:44:07.466940Z').

The 'Z' indicates UTC.
This is the format generated by 'svn log --xml'.

More details of the format at http://www.w3.org/TR/NOTE-datetime.

This 'svn diff' fixes the problem:

--- date.c      (revision 656)
+++ date.c      (working copy)
@@ -136,7 +136,7 @@
 **
 ** If the parse is successful, write the number of minutes
 ** of change in *pnMin and return 0.  If a parser error occurs,
-** return 0.
+** return 1.
 **
 ** A missing specifier is not considered an error.
 */
@@ -200,7 +200,10 @@
   p->h = h;
   p->m = m;
   p->s = s + ms;
-  if( parseTimezone(zDate, p) ) return 1;
+  if (*zDate == 'Z')
+       p->tz = 0;
+  else if( parseTimezone(zDate, p) )
+    return 1;
   p->validTZ = p->tz!=0;
   return 0;
 }

PS Should probably add a test case to 'sqlite/test/date.test'.

 
1856 code active 2006 Jun anonymous   2006 Jun   2 3 SQLITE_OMIT_UTF16 breaks 'make test' edit
When compiling sqlite 3.3.6 with -DSQLITE_OMIT_UTF16 and you say 'make test' it fails:

  make test
./libtool --mode=link gcc -g -O2 -DOS_UNIX=1 -DHAVE_USLEEP=1 -DHAVE_FDATASYNC=1 -I. -I./src -DSQLITE_DEBUG=2 -DSQLITE_MEMDEBUG=2 -DSQLITE_OMIT_UTF16  -I/usr/include -DTHREADSAFE=1 -DSQLITE_THREAD_OVERRIDE_LOCK=-1 -DSQLITE_OMIT_CURSOR -DTCLSH=1 -DSQLITE_TEST=1 -DSQLITE_CRASH_TEST=1 \
        -DTEMP_STORE=1 -o testfixture ./src/btree.c ./src/date.c ./src/func.c ./src/os.c ./src/os_unix.c ./src/os_win.c ./src/os_os2.c ./src/pager.c ./src/pragma.c ./src/printf.c ./src/test1.c ./src/test2.c ./src/test3.c ./src/test4.c ./src/test5.c ./src/test6.c ./src/test7.c ./src/test_async.c ./src/test_md5.c ./src/test_server.c ./src/utf.c ./src/util.c ./src/vdbe.c ./src/where.c ./src/tclsqlite.c \
        libsqlite3.la -L/usr/lib -ltcl8.4 -ldl  -lpthread -lieee -lm
gcc -g -O2 -DOS_UNIX=1 -DHAVE_USLEEP=1 -DHAVE_FDATASYNC=1 -I. -I./src -DSQLITE_DEBUG=2 -DSQLITE_MEMDEBUG=2 -DSQLITE_OMIT_UTF16 -I/usr/include -DTHREADSAFE=1 -DSQLITE_THREAD_OVERRIDE_LOCK=-1 -DSQLITE_OMIT_CURSOR -DTCLSH=1 -DSQLITE_TEST=1 -DSQLITE_CRASH_TEST=1 -DTEMP_STORE=1 -o .libs/testfixture ./src/btree.c ./src/date.c ./src/func.c ./src/os.c ./src/os_unix.c ./src/os_win.c ./src/os_os2.c ./src/pager.c ./src/pragma.c ./src/printf.c ./src/test1.c ./src/test2.c ./src/test3.c ./src/test4.c ./src/test5.c ./src/test6.c ./src/test7.c ./src/test_async.c ./src/test_md5.c ./src/test_server.c ./src/utf.c ./src/util.c ./src/vdbe.c ./src/where.c ./src/tclsqlite.c  ./.libs/libsqlite3.so -L/usr/lib -ltcl8.4 -ldl -lpthread -lieee -lm -Wl,--rpath -Wl,/home/cla/proj/caissadb/sqlite/sqlite/lib
./src/test1.c: In function 'Sqlitetest1_Init':
./src/test1.c:3742: error: 'unaligned_string_counter' undeclared (first use in this function)
./src/test1.c:3742: error: (Each undeclared identifier is reported only once
./src/test1.c:3742: error: for each function it appears in.)
make: *** [testfixture] Error 1

Maybe there is a '#ifndef SQLITE_OMIT_UTF16' / '#endif' needed around

  Tcl_LinkVar(interp, "unaligned_string_counter",
      (char*)&unaligned_string_counter, TCL_LINK_INT);

in Line 3742 in file src/test1.c?

Regards.

 
1853 code active 2006 Jun anonymous   2006 Jun   3 4 sqlite3_analyzer-3.3.4 integer overflows on large database edit
sqlite3_analyzer-3.3.4 reports senseless values (negative bytes and percents) for some fields of my 28 GB database:

Examples:

   Bytes of storage consumed............. 291456000
   Bytes of payload...................... -1382746251 -474.4%
   Average payload per entry............. -3.80

(I'll attached the full output separatly.)

I'm using the pre-built Linux binary of sqlite3_analyzer-3.3.4 from the sqlite homepage. My system is

cn@jehova:~/misc/jibladze> uname -a Linux jehova 2.6.13-15.10-smp #1 SMP Fri May 12 16:11:24 UTC 2006 x86_64 x86_64 x86_64 GNU/Linux

2006-Jun-18 17:28:20 by anonymous:
Here's the full output:

  cn@jehova:~> du -ah /playground/cn/yacop-data/gfr/steenrod-5-E-1Rfull
  28G     /playground/cn/yacop-data/gfr/steenrod-5-E-1Rfull
  cn@jehova:~> sqlite-analyzer /playground/cn/yacop-data/gfr/steenrod-5-E-1Rfull
  Analyzing table chartinfo...
  Analyzing table complog...
  Analyzing table fragments...
  Analyzing table generators...
  Analyzing table header...
  Analyzing table sqlite_master...
  Analyzing table worklog...
  Analyzing index sdegind of table fragments...
  Analyzing index sqlite_autoindex_header_1 of table header...
  Analyzing index sqlite_autoindex_worklog_1 of table worklog...
  /** Disk-Space Utilization Report For /playground/cn/yacop-data/gfr/steenrod-5-E-1Rfull
  *** As of 2006-Jun-18 18:55:01

  Page size in bytes.................... 1024
  Pages in the whole file (measured).... 28439973
  Pages in the whole file (calculated).. 28439972
  Pages that store data................. 28439972   100.000%
  Pages on the freelist (per header).... 0            0.0%
  Pages on the freelist (calculated).... 1            0.0%
  Pages of auto-vacuum overhead......... 0            0.0%
  Number of tables in the database...... 7
  Number of indices..................... 3
  Number of named indices............... 1
  Automatically generated indices....... 2
  Size of the file in bytes............. 29122532352
  Bytes of user payload stored.......... -1818729787  -6.2%

  *** Page counts for all tables with their indices ********************

  FRAGMENTS............................. 28409448    99.89%
  COMPLOG............................... 13559        0.048%
  CHARTINFO............................. 11151        0.039%
  GENERATORS............................ 5797         0.020%
  WORKLOG............................... 10           0.0%
  SQLITE_MASTER......................... 5            0.0%
  HEADER................................ 2            0.0%

  *** All tables and indices *******************************************

  Percentage of total database.......... 100.000%
  Number of entries..................... 728140955
  Bytes of storage consumed............. -942239744
  Bytes of payload...................... 1093495310 -116.1%
  Average payload per entry............. 1.50
  Average unused bytes per entry........ -2.73
  Average fanout........................ 88.00
  Maximum payload per entry............. 1157
  Entries that use overflow............. 351          0.0%
  Index pages used...................... 270372
  Primary pages used.................... 28169249
  Overflow pages used................... 351
  Total pages used...................... 28439972
  Unused bytes on index pages........... 32954241    11.9%
  Unused bytes on primary pages......... -2022053395 165.815%
  Unused bytes on overflow pages........ 37488       10.4%
  Unused bytes on all pages............. -1989061666 211.099%

  *** All tables *******************************************************

  Percentage of total database..........  84.3%
  Number of entries..................... 364670985
  Bytes of storage consumed............. -1233700864
  Bytes of payload...................... -1818727376 147.420%
  Average payload per entry............. -4.99
  Average unused bytes per entry........ 4.87
  Average fanout........................ 88.00
  Maximum payload per entry............. 1157
  Entries that use overflow............. 351          0.0%
  Index pages used...................... 270372
  Primary pages used.................... 23690315
  Overflow pages used................... 351
  Total pages used...................... 23961038
  Unused bytes on index pages........... 32954241    11.9%
  Unused bytes on primary pages......... 1742865277 -115.4%
  Unused bytes on overflow pages........ 37488       10.4%
  Unused bytes on all pages............. 1775857006 -143.9%

  *** All indices ******************************************************

  Percentage of total database..........  15.7%
  Number of entries..................... 363469970
  Bytes of storage consumed............. 291461120
  Bytes of payload...................... -1382744610 -474.4%
  Average payload per entry............. -3.80
  Average unused bytes per entry........ 1.46
  Maximum payload per entry............. 17
  Entries that use overflow............. 0            0.0%
  Primary pages used.................... 4478934
  Overflow pages used................... 0
  Total pages used...................... 4478934
  Unused bytes on primary pages......... 530048624  181.859%
  Unused bytes on overflow pages........ 0
  Unused bytes on all pages............. 530048624  181.859%

  *** Table CHARTINFO **************************************************

  Percentage of total database..........   0.039%
  Number of entries..................... 560603
  Bytes of storage consumed............. 11418624
  Bytes of payload...................... 7708145     67.5%
  Average payload per entry............. 13.75
  Average unused bytes per entry........ 0.31
  Average fanout........................ 99.00
  Maximum payload per entry............. 14
  Entries that use overflow............. 0            0.0%
  Index pages used...................... 112
  Primary pages used.................... 11039
  Overflow pages used................... 0
  Total pages used...................... 11151
  Unused bytes on index pages........... 14280       12.5%
  Unused bytes on primary pages......... 160371       1.4%
  Unused bytes on overflow pages........ 0
  Unused bytes on all pages............. 174651       1.5%

  *** Table COMPLOG ****************************************************

  Percentage of total database..........   0.048%
  Number of entries..................... 322413
  Bytes of storage consumed............. 13884416
  Bytes of payload...................... 11350473    81.7%
  Average payload per entry............. 35.20
  Average unused bytes per entry........ 1.20
  Average fanout........................ 98.00
  Maximum payload per entry............. 45
  Entries that use overflow............. 0            0.0%
  Index pages used...................... 138
  Primary pages used.................... 13421
  Overflow pages used................... 0
  Total pages used...................... 13559
  Unused bytes on index pages........... 19411       13.7%
  Unused bytes on primary pages......... 367295       2.7%
  Unused bytes on overflow pages........ 0
  Unused bytes on all pages............. 386706       2.8%

  *** Table FRAGMENTS and all its indices ******************************

  Percentage of total database..........  99.89%
  Number of entries..................... 726939530
  Bytes of storage consumed............. -973496320
  Bytes of payload...................... 1070540938 -110.0%
  Average payload per entry............. 1.47
  Average unused bytes per entry........ -2.74
  Average fanout........................ 88.00
  Maximum payload per entry............. 1157
  Entries that use overflow............. 351          0.0%
  Index pages used...................... 270061
  Primary pages used.................... 28139036
  Overflow pages used................... 351
  Total pages used...................... 28409448
  Unused bytes on index pages........... 32910337    11.9%
  Unused bytes on primary pages......... -2022636857 161.759%
  Unused bytes on overflow pages........ 37488       10.4%
  Unused bytes on all pages............. -1989689032 204.386%

  *** Table FRAGMENTS w/o any indices **********************************

  Percentage of total database..........  84.1%
  Number of entries..................... 363469765
  Bytes of storage consumed............. -1264952320
  Bytes of payload...................... -1841680107 145.593%
  Average payload per entry............. -5.07
  Average unused bytes per entry........ 4.88
  Average fanout........................ 88.00
  Maximum payload per entry............. 1157
  Entries that use overflow............. 351          0.0%
  Index pages used...................... 270061
  Primary pages used.................... 23660107
  Overflow pages used................... 351
  Total pages used...................... 23930519
  Unused bytes on index pages........... 32910337    11.9%
  Unused bytes on primary pages......... 1742284627 -113.0%
  Unused bytes on overflow pages........ 37488       10.4%
  Unused bytes on all pages............. 1775232452 -140.3%

  *** Indices of table FRAGMENTS ***************************************

  Percentage of total database..........  15.7%
  Number of entries..................... 363469765
  Bytes of storage consumed............. 291456000
  Bytes of payload...................... -1382746251 -474.4%
  Average payload per entry............. -3.80
  Average unused bytes per entry........ 1.46
  Maximum payload per entry............. 9
  Entries that use overflow............. 0            0.0%
  Primary pages used.................... 4478929
  Overflow pages used................... 0
  Total pages used...................... 4478929
  Unused bytes on primary pages......... 530045812  181.861%
  Unused bytes on overflow pages........ 0
  Unused bytes on all pages............. 530045812  181.861%

  *** Table GENERATORS *************************************************

  Percentage of total database..........   0.020%
  Number of entries..................... 317989
  Bytes of storage consumed............. 5936128
  Bytes of payload...................... 3889213     65.5%
  Average payload per entry............. 12.23
  Average unused bytes per entry........ 0.18
  Average fanout........................ 98.00
  Maximum payload per entry............. 14
  Entries that use overflow............. 0            0.0%
  Index pages used...................... 59
  Primary pages used.................... 5738
  Overflow pages used................... 0
  Total pages used...................... 5797
  Unused bytes on index pages........... 8350        13.8%
  Unused bytes on primary pages......... 49171        0.84%
  Unused bytes on overflow pages........ 0
  Unused bytes on all pages............. 57521        0.97%

  *** Table HEADER and all its indices *********************************

  Percentage of total database..........   0.0%
  Number of entries..................... 10
  Bytes of storage consumed............. 2048
  Bytes of payload...................... 187          9.1%
  Average payload per entry............. 18.70
  Average unused bytes per entry........ 180.70
  Maximum payload per entry............. 50
  Entries that use overflow............. 0            0.0%
  Primary pages used.................... 2
  Overflow pages used................... 0
  Total pages used...................... 2
  Unused bytes on primary pages......... 1807        88.2%
  Unused bytes on overflow pages........ 0
  Unused bytes on all pages............. 1807        88.2%

  *** Table HEADER w/o any indices *************************************

  Percentage of total database..........   0.0%
  Number of entries..................... 5
  Bytes of storage consumed............. 1024
  Bytes of payload...................... 124         12.1%
  Average payload per entry............. 24.80
  Average unused bytes per entry........ 173.80
  Maximum payload per entry............. 50
  Entries that use overflow............. 0            0.0%
  Primary pages used.................... 1
  Overflow pages used................... 0
  Total pages used...................... 1
  Unused bytes on primary pages......... 869         84.9%
  Unused bytes on overflow pages........ 0
  Unused bytes on all pages............. 869         84.9%

  *** Indices of table HEADER ******************************************

  Percentage of total database..........   0.0%
  Number of entries..................... 5
  Bytes of storage consumed............. 1024
  Bytes of payload...................... 63           6.2%
  Average payload per entry............. 12.60
  Average unused bytes per entry........ 187.60
  Maximum payload per entry............. 17
  Entries that use overflow............. 0            0.0%
  Primary pages used.................... 1
  Overflow pages used................... 0
  Total pages used...................... 1
  Unused bytes on primary pages......... 938         91.6%
  Unused bytes on overflow pages........ 0
  Unused bytes on all pages............. 938         91.6%

  *** Table SQLITE_MASTER **********************************************

  Percentage of total database..........   0.0%
  Number of entries..................... 10
  Bytes of storage consumed............. 5120
  Bytes of payload...................... 2411        47.1%
  Average payload per entry............. 241.10
  Average unused bytes per entry........ 249.70
  Average fanout........................ 4.00
  Maximum payload per entry............. 483
  Entries that use overflow............. 0            0.0%
  Index pages used...................... 1
  Primary pages used.................... 4
  Overflow pages used................... 0
  Total pages used...................... 5
  Unused bytes on index pages........... 891         87.0%
  Unused bytes on primary pages......... 1606        39.2%
  Unused bytes on overflow pages........ 0
  Unused bytes on all pages............. 2497        48.8%

  *** Table WORKLOG and all its indices ********************************

  Percentage of total database..........   0.0%
  Number of entries..................... 400
  Bytes of storage consumed............. 10240
  Bytes of payload...................... 3943        38.5%
  Average payload per entry............. 9.86
  Average unused bytes per entry........ 10.46
  Average fanout........................ 5.00
  Maximum payload per entry............. 13
  Entries that use overflow............. 0            0.0%
  Index pages used...................... 1
  Primary pages used.................... 9
  Overflow pages used................... 0
  Total pages used...................... 10
  Unused bytes on index pages........... 972         94.9%
  Unused bytes on primary pages......... 3212        34.9%
  Unused bytes on overflow pages........ 0
  Unused bytes on all pages............. 4184        40.9%

  *** Table WORKLOG w/o any indices ************************************

  Percentage of total database..........   0.0%
  Number of entries..................... 200
  Bytes of storage consumed............. 6144
  Bytes of payload...................... 2365        38.5%
  Average payload per entry............. 11.82
  Average unused bytes per entry........ 11.55
  Average fanout........................ 5.00
  Maximum payload per entry............. 13
  Entries that use overflow............. 0            0.0%
  Index pages used...................... 1
  Primary pages used.................... 5
  Overflow pages used................... 0
  Total pages used...................... 6
  Unused bytes on index pages........... 972         94.9%
  Unused bytes on primary pages......... 1338        26.1%
  Unused bytes on overflow pages........ 0
  Unused bytes on all pages............. 2310        37.6%

  *** Indices of table WORKLOG *****************************************

  Percentage of total database..........   0.0%
  Number of entries..................... 200
  Bytes of storage consumed............. 4096
  Bytes of payload...................... 1578        38.5%
  Average payload per entry............. 7.89
  Average unused bytes per entry........ 9.37
  Maximum payload per entry............. 9
  Entries that use overflow............. 0            0.0%
  Primary pages used.................... 4
  Overflow pages used................... 0
  Total pages used...................... 4
  Unused bytes on primary pages......... 1874        45.8%
  Unused bytes on overflow pages........ 0
  Unused bytes on all pages............. 1874        45.8%

  *** Definitions ******************************************************

  Page size in bytes

      The number of bytes in a single page of the database file.
      Usually 1024.

  Number of pages in the whole file

      The number of 1024-byte pages that go into forming the complete
      database

  Pages that store data

      The number of pages that store data, either as primary B*Tree pages or
      as overflow pages.  The number at the right is the data pages divided by
      the total number of pages in the file.

  Pages on the freelist

      The number of pages that are not currently in use but are reserved for
      future use.  The percentage at the right is the number of freelist pages
      divided by the total number of pages in the file.

  Pages of auto-vacuum overhead

      The number of pages that store data used by the database to facilitate
      auto-vacuum. This is zero for databases that do not support auto-vacuum.

  Number of tables in the database

      The number of tables in the database, including the SQLITE_MASTER table
      used to store schema information.

  Number of indices

      The total number of indices in the database.

  Number of named indices

      The number of indices created using an explicit CREATE INDEX statement.

  Automatically generated indices

      The number of indices used to implement PRIMARY KEY or UNIQUE constraints
      on tables.

  Size of the file in bytes

      The total amount of disk space used by the entire database files.

  Bytes of user payload stored

      The total number of bytes of user payload stored in the database. The
      schema information in the SQLITE_MASTER table is not counted when
      computing this number.  The percentage at the right shows the payload
      divided by the total file size.

  Percentage of total database

      The amount of the complete database file that is devoted to storing
      information described by this category.

  Number of entries

      The total number of B-Tree key/value pairs stored under this category.

  Bytes of storage consumed

      The total amount of disk space required to store all B-Tree entries
      under this category.  The is the total number of pages used times
      the pages size.

  Bytes of payload

      The amount of payload stored under this category.  Payload is the data
      part of table entries and the key part of index entries.  The percentage
      at the right is the bytes of payload divided by the bytes of storage
      consumed.

  Average payload per entry

      The average amount of payload on each entry.  This is just the bytes of
      payload divided by the number of entries.

  Average unused bytes per entry

      The average amount of free space remaining on all pages under this
      category on a per-entry basis.  This is the number of unused bytes on
      all pages divided by the number of entries.

  Maximum payload per entry

      The largest payload size of any entry.

  Entries that use overflow

      The number of entries that user one or more overflow pages.

  Total pages used

      This is the number of pages used to hold all information in the current
      category.  This is the sum of index, primary, and overflow pages.

  Index pages used

      This is the number of pages in a table B-tree that hold only key (rowid)
      information and no data.

  Primary pages used

      This is the number of B-tree pages that hold both key and data.

  Overflow pages used

      The total number of overflow pages used for this category.

  Unused bytes on index pages

      The total number of bytes of unused space on all index pages.  The
      percentage at the right is the number of unused bytes divided by the
      total number of bytes on index pages.

  Unused bytes on primary pages

      The total number of bytes of unused space on all primary pages.  The
      percentage at the right is the number of unused bytes divided by the
      total number of bytes on primary pages.

  Unused bytes on overflow pages

      The total number of bytes of unused space on all overflow pages.  The
      percentage at the right is the number of unused bytes divided by the
      total number of bytes on overflow pages.

  Unused bytes on all pages

      The total number of bytes of unused space on all primary and overflow
      pages.  The percentage at the right is the number of unused bytes
      divided by the total number of bytes.

  **********************************************************************
  The entire text of this report can be sourced into any SQL database
  engine for further analysis.  All of the text above is an SQL comment.
  The data used to generate this report follows:
  */
  BEGIN;
  CREATE TABLE space_used(
     name clob,        -- Name of a table or index in the database file
     tblname clob,     -- Name of associated table
     is_index boolean, -- TRUE if it is an index, false for a table
     nentry int,       -- Number of entries in the BTree
     leaf_entries int, -- Number of leaf entries
     payload int,      -- Total amount of data stored in this table or index
     ovfl_payload int, -- Total amount of data stored on overflow pages
     ovfl_cnt int,     -- Number of entries that use overflow
     mx_payload int,   -- Maximum payload size
     int_pages int,    -- Number of interior pages used
     leaf_pages int,   -- Number of leaf pages used
     ovfl_pages int,   -- Number of overflow pages used
     int_unused int,   -- Number of unused bytes on interior pages
     leaf_unused int,  -- Number of unused bytes on primary pages
     ovfl_unused int   -- Number of unused bytes on overflow pages
  );
  INSERT INTO space_used VALUES('chartinfo','chartinfo',0,571641,560603,7708145,0,0,14,112,11039,0,14280,160371,0);
  INSERT INTO space_used VALUES('complog','complog',0,335833,322413,11350473,0,0,45,138,13421,0,19411,367295,0);
  INSERT INTO space_used VALUES('fragments','fragments',0,387129871,363469765,19633156373,320532,351,1157,270061,23660107,351,32910337,1742284627,37488);
  INSERT INTO space_used VALUES('generators','generators',0,323726,317989,3889213,0,0,14,59,5738,0,8350,49171,0);
  INSERT INTO space_used VALUES('header','header',0,5,5,124,0,0,50,0,1,0,0,869,0);
  INSERT INTO space_used VALUES('sqlite_master','sqlite_master',0,13,10,2411,0,0,483,1,4,0,891,1606,0);
  INSERT INTO space_used VALUES('worklog','worklog',0,204,200,2365,0,0,13,1,5,0,972,1338,0);
  INSERT INTO space_used VALUES('sdegind','fragments',1,363469765,363469765,2912221045,0,0,9,0,4478929,0,0,530045812,0);
  INSERT INTO space_used VALUES('sqlite_autoindex_header_1','header',1,5,5,63,0,0,17,0,1,0,0,938,0);
  INSERT INTO space_used VALUES('sqlite_autoindex_worklog_1','worklog',1,200,200,1578,0,0,9,0,4,0,0,1874,0);
  COMMIT;
 
1851 code active 2006 Jun anonymous Unknown 2006 Jun   2 1 USE "ORDER BY" error on uClinux edit
when I use "ORDER BY" function in a "select" on uClinux 2.4.24, I get a error:"SQL error or missing database" but the same program run on windows or Linux OK.
2006-Jun-16 11:20:15 by drh:
This is certainly a strange error. Combined with #1850, it suggests a problem with your build, not a problem in SQLite.

I have no ability to use or run uCLinux. So if the error cannot be reproduced on a desktop system, there is not much I can do to address the problem. I am afraid you are on your own on this one.


2006-Jun-19 03:12:31 by anonymous:
I'm just wondering that SQLite 3.2.8 runs on this uClinux system OK but SQLite 3.3.5 is error.
 
1850 code active 2006 Jun anonymous Unknown 2006 Jun   2 1 NUMERIC data type ERROE when read on uClinux edit
I have update some data of a tables's NUMERIC TYPE column on Windows or Linux,but when I use "select *......." to read on uClinux 2.4.24,I get the wrong value,example:the date I've written is 12.5,but readback is 2.3534826093695e -18.5(use the sqlite3_column_text API). I tried to get the value used "sqlite3_column_double" API,but the result is also wrong;

But when I update some data with this column on uClinux,I can read the data right!

2006-Jun-16 01:53:24 by drh:
What CPU is this happening on?

SQLite assumes that floating point values are stored as IEEE 64-bit floats in the same byte order as a 64-bit integer. If your chip does not match this expectation, then floating point won't work.

 
1838 code active 2006 Jun anonymous   2006 Jun   5 5 test types3-1.3 fails on 64-bit Linux edit
I just compiled 3.3.6 on my 64-bit Linux system (OpenSuse 10, using a self-compiled Tcl 8.4.11) and got one failure from "make test":

  types3-1.3...
  Expected: [wideInt integer]
       Got: [int integer]

This seems to be an error in the test suite itself: Changing

    set V [expr {1+123456789012345}]

to

    set V [expr {wide(1+123456789012345)}]

gets rid of the failure.

 
1837 new active 2006 Jun anonymous Pager 2006 Jun   4 4 Deadlock detection would be best reported using explicit error code. edit
As discussed in email (*), an explicit deadlock error code might be beneficial for an application to detect and recover from deadlock. Deadlock is different to SQLITE_BUSY, and should be notified as such (IMHO).

(*) http://www.mail-archive.com/sqlite-users%40sqlite.org/msg15979.html

2006-Jun-12 11:23:49 by drh:
Changing the integer return code would be not be a backwards compatible change, so that approach must be rejected. But we will consider some mechanism to make it easier for programmers to figure out that they are dealing with deadlock and not a temporary delay.


2006-Jun-12 15:43:59 by anonymous:
Is there a fatal return code? If so, perhaps it could be overloaded with an additional function to determine if this 'fatal' error is actually a deadlock. Deadlocks are a common return code from almost every database I've ever programmed for. I am not sure how you can work around it otherwise.
 
1833 doc active 2006 Jun anonymous Unknown 2006 Jun drh 4 3 PRAGMA legacy_file_format not documented edit
[2922] introduces the legacy_file_format pragma, but it's not documented anywhere. At the very least, it should be mentioned in /sqlite/www/pragma.tcl, unless there's some better way to have a SQLite 3.3.5 (or so) generate databases usable by older versions of SQLite 3 (Debian stable, for example, ships with 3.2.1).
 
1816 code active 2006 May anonymous VDBE 2006 May   1 2 Database corruption with pragma auto_vacuum edit
We had a database created with PRAGMA auto_vacuum=1, that started returning the following message on a DELETE statement.

  SQL error: database disk image is malformed

Running the VACUUM command and running the same DELETE statement succeeds.

  Running PRAGMA integrity_check on the database (before the VACUUM command is issued) results in the following output:
sqlite> PRAGMA integrity_check;
*** in database main ***
Page 3393 is never used
Page 3398 is never used
Page 3400 is never used
Page 3401 is never used
Page 3402 is never used
Page 3405 is never used
Page 3406 is never used
sqlite> VACUUM;
sqlite> PRAGMA integrity_check;
ok

We tried as a temporary workaround, running PRAGMA integrity_check and, based on the result, deciding whether or not to run VACUUM, but this can consume too much time.

If needed, I can send a small database that exhibits this problem.

2006-May-22 21:45:47 by drh:
The database is probably not helpful. What I need to know is:

  • What sequence of SQL statements do you issue to cause this to occur?
  • What operating system you are using.
  • Is the application multi-threaded?
  • Is the problem reproducible?
  • Are you using a precompiled binary or did you compile it yourself?
  • Does the problem go away if you turn off autovacuum?


2006-May-22 22:11:09 by anonymous:
  • What sequence of SQL statements do you issue to cause this to occur? It is unknown exactly what all of the the statements are leading up to the corruption. I can send the possible statements via private e-mail.

  • What operating system you are using. Windows XP Professional w/ Service Pack 2.

  • Is the application multi-threaded? Yes.

  • Is the problem reproducible? The corruption happens on occasion -- so far it is not known to be easily reproducable in a finite number of steps.

  • Are you using a precompiled binary or did you compile it yourself? Self-compiled library. When we use the database in our application, it is contained in abstracted classes with concurrency control.

  • Does the problem go away if you turn off autovacuum? We have not seen database corruption if auto_vacuum is off when the database is initially created. Is it possible to turn off auto vacuum after the database tables have been created (no when using pragma auto_vacuum, according to the docs)?


2006-May-22 22:28:46 by anonymous:
Rather than relying on trial and error to reproduce the bug, one technique the bug reporter might try to reproduce the problem is to take a snapshot of the database when it is in a known good state and save it somewhere and then have every process that comes into contact with the database file log every SQLite command (and pragma) complete with millisecond-resolution timestamp and process/thread ID as follows:

  SELECT * FROM WHATEVER;         -- 2006-05-23 14:44:45.237 PID 345 Thread 0
  insert into blah values(3,4,5); -- 2006-05-23 14:50:15.345 PID 345 Thread 0
  update foo set v=5 where y>4;   -- 2006-05-23 15:05:12.930 PID 239 Thread 0

Should the problem happen again, each command could easily be replayed in an appropriate thread in the same order from the last known "good" state, greatly increasing the chances of repeating the bug. If repeating these commands does not lead to database corruption, it is fairly likely that the bug is in your multithreaded code, and not in SQLite.

Perhaps SQLite already has such a command tracing facility already. I don't know.


2006-May-22 22:42:04 by anonymous:

  sqlite3_trace();

It passes all the caller-generated SQL statements to a callback (although it doesn't fill in bindings).

It also outputs a lot of "internal" SQL statements (VACUUM, for example, is a collection of operations on a temp table), but you should be able to recognize that stuff as something your app would never generate.

 
1815 code active 2006 May anonymous Parser 2006 May   3 3 Support of W3C-DTF(ISO8601 subset) is incomplete edit
"Z" of a time zone is ignored.

Reference: http://www.w3.org/TR/NOTE-datetime

  CREATE table test(dt);
  INSERT INTO "test" VALUES('2006-05-20T01:10:20+00:00');
  INSERT INTO "test" VALUES('2006-05-20T01:10:20Z');
  INSERT INTO "test" VALUES('2006-05-20T10:10:20+09:00');
  SELECT datetime(dt) from test;
  2006-05-20 01:10:20

  2006-05-20 01:10:20
 
1814 new active 2006 May anonymous   2006 May   3 4 Autoconf support for MacOSX univeral binaries edit
SQLite is used widely on OS X. One problem for many OS X developers is preparing SQLite for universal binaries. I think there are a couple ways to solve this.

1. Add a new configure option --enable-macosx-universal to put the right compiler switches

2. Rework the configure makefiles so that you can override with the typical CFLAGS overrides as suggested by Apple in this article: http://developer.apple.com/documentation/Porting/Conceptual/PortingUnix/compiling/chapter_4_section_3.html

I've attached a patch that will enable universal binaries if you chose the --enable-macosx-universal option, are on -*-darwin os, and have disabled shared libraries. This patch may not be exactly how you would want to integrate but it should serve as a good starting point.

I've tested in a PPC powermac, PPC powerbook, and Intel Mac mini.


--- sqlite-3.3.5/configure.ac 2006-04-03 13:16:01.000000000 -0700
+++ sqlite-3.3.x/configure.ac 2006-05-18 16:42:08.000000000 -0700
@@ -661,6 +661,32 @@

AC_CHECK_FUNC(fdatasync, [TARGET_CFLAGS="$TARGET_CFLAGS -DHAVE_FDATASYNC=1"])

+##########
+# Mac OS X Universal Binary support
+#
+AC_MSG_CHECKING([whether building for macosx])
+case "${build}" in
+ *-*-darwin* )
+ AC_ARG_ENABLE(macosx-universal,
+ AC_HELP_STRING([--enable-macosx-universal],[Enable macosx universal binaries]),,enable_macosxuniversal=no)
+ AC_MSG_CHECKING([whether building Universal Binaries])
+ if test "$enable_macosxuniversal" = "no"; then
+ AC_MSG_RESULT([no])
+ else
+ AC_MSG_CHECKING([if shared libraries are disabled])
+ if test "$enable_shared" = "no"; then
+ TARGET_CFLAGS="$TARGET_CFLAGS -arch ppc -arch i386 -isysroot /Developer/SDKs/MacOSX10.4u.sdk"
+ AC_MSG_RESULT([yes (Universal Binaries enabled)])
+ else
+ AC_MSG_RESULT([no (Universal Binaries disabled)])
+ fi
+ fi
+ ;;
+ *)
+ AC_MSG_RESULT([no])
+esac
+
+
#########
# Put out accumulated miscellaneous LIBRARIES
#

 
1812 new active 2006 May anonymous   2006 May   4 1 alter table add column default current_timestamp edit
I wish I can alter my schema and add column with non-constant default. How difficult is this compared with constant default?
 
1811 doc active 2006 May anonymous   2006 May   5 3 how many open cursors are allowed for one application? edit
I would like to know, how many open cursors allowed at the same time.
 
1797 code active 2006 May anonymous TclLib 2006 May drh 1 1 COPY command doesn't work in tclsqlite 3.3.5 edit
The COPY command doesn't seem to work in the tcl sqlite lib. This same script and datafile works in version 3.2.7.

load ./lib/libtclsqlite[info sharedlibextension] sqlite MEMORY_DB :memory: MEMORY_DB onecolumn "PRAGMA empty_result_callbacks=1" puts [MEMORY_DB version] MEMORY_DB eval "create table xyz (col1,col2)" MEMORY_DB copy ignore win_pol /home/centadm/win_pol4.csv \t MEMORY_DB eval "select * from xyz" sqlite_array { puts "Here in the callback" foreach sqlite_value $sqlite_array(*) { puts "$sqlite_value $sqlite_array($sqlite_value)" } }

The data file win_pol4.csv consists of two columns, tab seperated. DATA1 DATA2

And the output: -bash-3.00$ tclsh test_sqlite.tcl

3.3.5

    while executing
"MEMORY_DB copy ignore win_pol /home/centadm/win_pol4.csv \t"
    (file "test_sqlite.tcl" line 5)
-bash-3.00$ pwd
/home/centadm
-bash-3.00$ ls -l /home/centadm/win_pol4.csv
-rw-r--r--  1 centadm centadm 12 May  5 14:21 /home/centadm/win_pol4.csv
-bash-3.00$ more /home/centadm/win_pol4.csv
DATA1   DATA2

A TCL Error is returned from the copy command, no message tho. I have used catch to capture the command and verified that there is no data going into the table.

Also, PRAGMA empty_result_callbacks=1 still doesn't seem to work in the tcllib. If you catch the COPY command above, you still never see the "Here in the callback" message.

2006-May-05 17:57:42 by anonymous:
Clarification:

The line MEMORY_DB copy ignore win_pol /home/centadm/win_pol4.csv \t

should read

MEMORY_DB copy ignore xyz /home/centadm/win_pol4.csv \t

However the result is the same:

-bash-3.00$ tclsh test_sqlite.tcl 3.3.5

    while executing
"MEMORY_DB copy ignore xyz /home/centadm/win_pol4.csv \t"
    (file "test_sqlite.tcl" line 7)
-bash-3.00$


2006-May-05 19:46:56 by anonymous:
I have narrowed it down to the code here in tclsqlite.c:

    zSql = sqlite3_mprintf("SELECT * FROM '%q'", zTable);
    if( zSql==0 ){
      Tcl_AppendResult(interp, "Error: no such table: ", zTable, 0);
      return TCL_ERROR;
    }
    nByte = strlen(zSql);
    rc = sqlite3_prepare(pDb->db, zSql, 0, &pStmt, 0);
    sqlite3_free(zSql);
    if( rc ){
      Tcl_AppendResult(interp, "Error: ", sqlite3_errmsg(pDb->db), 0);
      nCol = 0;
    }else{
      nCol = sqlite3_column_count(pStmt); <--- RETURNING 0 FOR COLUMN COUNT, HAVE VERIFIED TABLE HAS TWO COLUMNS
    }
    sqlite3_finalize(pStmt);
    if( nCol==0 ) {
      return TCL_ERROR;       <--- NO ERROR MESSAGE RETURNED
    }


2006-May-16 17:51:28 by anonymous:
I found the problem. The first sqlite3_prepare under DB_COPY should have -1 as it's third argument. When this was change from a 0 to -1 the copy command works in tclsqlite.

rc = sqlite3_prepare(pDb->db, zSql,0, &pStmt, 0);

should be

rc = sqlite3_prepare(pDb->db, zSql,-1, &pStmt, 0);


2006-May-16 18:01:11 by anonymous:
There is also another reference (the insert statement) to the prepare statement under DB_COPY that needs to change it's third argument from 0 to -1.


2006-Sep-27 16:24:53 by anonymous:
The same problem is present with version 3.3.7 over here. However, the indicated patch seem to work.
 
1810 new active 2006 May anonymous   2006 May   3 3 localtime() not threadsafe on UNIX edit
The following SQLite code from src/date.c is only guaranteed to function correctly with multiple threads on UNIX if the SQLite library is the only caller of localtime(). If an application that uses the SQLite library's date functions happens to call localtime() either directly or indirectly via another third party library, then localtime() can return a pointer to inconsistant data. localtime_r() should be use instead.

  sqlite3OsEnterMutex();
  pTm = localtime(&t);
  y.Y = pTm->tm_year + 1900;
  y.M = pTm->tm_mon + 1;
  y.D = pTm->tm_mday;
  y.h = pTm->tm_hour;
  y.m = pTm->tm_min;
  y.s = pTm->tm_sec;
  sqlite3OsLeaveMutex();

Windows and some versions of UNIX may use thread-local storage to make localtime() threadsafe. This is not the case with Linux or any other OS that uses GNU libc:

  /* Convert `time_t' to `struct tm' in local time zone.
     Copyright (C) 1991,92,93,95,96,97,98,2002 Free Software Foundation, Inc.
     This file is part of the GNU C Library.

     The GNU C Library is free software; you can redistribute it and/or
     modify it under the terms of the GNU Lesser General Public
     License as published by the Free Software Foundation; either
     version 2.1 of the License, or (at your option) any later version.

     The GNU C Library is distributed in the hope that it will be useful,
     but WITHOUT ANY WARRANTY; without even the implied warranty of
     MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
     Lesser General Public License for more details.

     You should have received a copy of the GNU Lesser General Public
     License along with the GNU C Library; if not, write to the Free
     Software Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA
     02111-1307 USA.  */

  #include <time.h>

  /* The C Standard says that localtime and gmtime return the same pointer.  */
  struct tm _tmbuf;

  /* Return the `struct tm' representation of *T in local time,
     using *TP to store the result.  */
  struct tm *
  __localtime_r (t, tp)
       const time_t *t;
       struct tm *tp;
  {
    return __tz_convert (t, 1, tp);
  }
  weak_alias (__localtime_r, localtime_r)

  /* Return the `struct tm' representation of *T in local time.  */
  struct tm *
  localtime (t)
       const time_t *t;
  {
    return __tz_convert (t, 1, &_tmbuf);
  }
  libc_hidden_def (localtime)

As an added benefit, you get better thread concurrency by getting rid of the sqlite3OsEnterMutex/sqlite3OsLeaveMutex when you use localtime_r.

2006-May-15 12:53:42 by drh:
All of this is pointed out already in the documentation. I will therefore change this ticket from a bug to an enhancement request.

Note that we have considered the use of localtime_r() in the past and rejected it since it will lead to a significant complication of the build process.

 
1809 code active 2006 May anonymous CodeGen 2006 May   1 3 Huge slowdown/increased memory use when using GROUP BY on big dataset edit
This seemingly nonsensical query is a greatly reduced test case taken from several queries I use with SQLite 3.2.1. The real example joins various huge tables and much more complicated views. I'd like to upgrade beyond SQLite 3.2.1, but this is a showstopper.

It takes 13 seconds to run on SQLite 3.2.1 and uses just 1.2M of memory. With 3.3.5+ from CVS it takes 185 seconds and uses 230M of memory.

  PRAGMA temp_store=MEMORY;

  CREATE TABLE n1(a integer primary key);
  INSERT INTO "n1" VALUES(1);
  INSERT INTO "n1" VALUES(2);
  INSERT INTO "n1" VALUES(3);
  INSERT INTO "n1" VALUES(4);
  INSERT INTO "n1" VALUES(5);
  INSERT INTO "n1" VALUES(6);
  INSERT INTO "n1" VALUES(7);
  INSERT INTO "n1" VALUES(8);
  INSERT INTO "n1" VALUES(9);
  INSERT INTO "n1" VALUES(10);
  INSERT INTO "n1" VALUES(11);
  INSERT INTO "n1" VALUES(12);
  INSERT INTO "n1" VALUES(13);
  INSERT INTO "n1" VALUES(14);
  INSERT INTO "n1" VALUES(15);
  CREATE VIEW vu as select v3.a a, v5.a-v2.a*v7.a b
                      from n1 v1,n1 v2,n1 v3,n1 v4,n1 v5,n1 v6,n1 v7;

  select a a, sum(b) T from vu where a=7 group by a;

It seems that SQLite 3.2.1 had a much more efficient GROUP BY algorithm that discarded unnecessary data as the view was traversed.

2006-May-13 03:01:28 by anonymous:
Seeing as this ticket concerns the GROUP BY statement it would make more sense to have an example like this:

  select a a, sum(b) T from vu where a<4 group by a;

But both queries exhibit the same slowdown and memory increase, in any event.


2006-May-13 15:09:39 by anonymous:
This GROUP BY slowdown/memory increase is not specific to VIEWs. I repeated the test against a comparably sized table with the same results. You'll see this effect for any SELECT operating on a large number of rows using GROUP BY.


2006-May-13 16:44:04 by anonymous:
The slowdown first appears in SQLite 3.2.6 in check-in [2662] .


2006-May-24 13:19:29 by anonymous:
Here's an example to show an actual real-life use of GROUP BY in SQLite <= 3.2.5... Imagine performing mathematical operations on every combination of rows in several large tables for statistical analysis. The GROUP BY algorithm change in 3.2.6 now makes using GROUP BY on huge cross joins not usable for this purpose because it creates an intermediate result set of the product of all cross joins - several times larger than the size of the (already huge) database itself. Indexing is not useful in this case because there is nothing to index by design. All table rows must be traversed.

Older versions of SQLite performed this operation extremely efficiently because grouping took place in the main traversal loop. I would think that the old algorithm could be used, but instead of keeping the intermediate results in memory, an index and a table in temp store could be used.

 
1804 code active 2006 May anonymous Unknown 2006 May   4 3 Inconsistent value type returned by SUM when using a GROUP BY edit
Using a schema with test table:

  CREATE TABLE Tbl1 (Key1 INTEGER, Num1 REAL)

And test data:

  INSERT INTO Tbl1 (Key1,Num1) VALUES (1,5.0)

The query:

  SELECT SUM(Tbl1.Num1) AS Num1Sum FROM Tbl1

Returns a column with the value type correctly reported as FLOAT (2).

However, the query:

  SELECT Tbl1.Key1, SUM(Tbl1.Num1) AS Num1Sum FROM Tbl1 GROUP BY Tbl1.Key1

Returns two columns with value types INT (1) and INT (1).

The SUM function is returning a different value type for these two queries when both should return FLOAT (2).

This problem does not occur when any SUMmed value is not a whole number in which case, both queries return a value type of FLOAT for the SUM column.

I have applied the patch from Check In 3169 (relating to #1726 and #1755) to select.c but this does not resolve the problem.

2006-May-10 09:34:11 by anonymous:
I should have added that this problem was seen in a Windows CE build running on Pocket PC 2003 and built using eMbedded Visual C++ 4.0.


2006-May-10 11:24:47 by anonymous:
I can confirm that exactly the same behaviour is exhibited when built under Windows XP (32-bit).


2006-May-10 12:40:21 by drh:
The answer you are getting back is exactly correct. Why do you care what its datatype is? If you don't like the datatype, cast it.


2006-May-10 13:42:14 by anonymous:
For maximum compatibility with other SQL databases, both

  SELECT SUM(field2) FROM table

and

  SELECT field1, SUM(field2) FROM table GROUP BY field1

should return the same data type for the SUM column. All other databases I have worked with do this. I understand that SQLite uses manifest typing but believe that it should be consistent.

The problem I have is that in my query function (which takes an SQL string and returns a page of results as a 2D array of objects), I don't know whether to use sqlite3_column_double or sqlite3_column_int because I don't know what the calling function requires this column to be returned as. I am currently using the sqlite3_column_decltype call to switch which sqlite3_column_* function I use (and falling back on sqlite3_column_type when the declared type is not known e.g. for aggregate functions like SUM). If the return type of SUM is unpredictable, my calling functions can't assume returned values will be of the same type as the field that is being SUMmed (as is the case with other SQL databases).

If you don't consider this to be a problem with SQLite, then I think my only option will be for calling functions to pass in an array of return types so that I always return objects of the correct type.

 
1801 new active 2006 May anonymous Pager 2006 May owensmk 5 4 Include Support for User-defined Lock Synchronization edit
SQLite uses a busy-wait model for locking. For high concurrency applications this can be become inefficient. I have written a patch for pager.c which introduces two hooks - lock() and unlock() - whereby the application can participate in locking/sychronization of connections. This can in some cases increase overall concurrency by an order of magnitude. The callback implementation is very similar to the busy handler.

All of the synchronization is implemented by the application. SQLite simply calls the hooks at the appropriate times, if they are registered. A full description of the patch is available at http://www.gintana.com/sqlite/.

 
1799 code active 2006 May anonymous Pager 2006 May   2 3 temp_store=MEMORY slower than FILE for large intermediate result sets edit
(This ticket was split off from #1790 because that ticket was becoming too broad.)

When temp_store=MEMORY it can negatively effect the performance of queries with large intermediate result sets generated from SELECTs of either file-based tables or memory-based tables. This is true when sufficient RAM is available to the SQLite process to completely hold the intermediate results in memory without swapping to disk.

In the example below, "big" is a file-based table in foo.db with 10 million rows. It was created with "create table big(x,y)".

  # unmodified stock SQLite built from May 5 2006 CVS (after check-in [3178])
  # compiled with default settings for SQLITE_DEFAULT_PAGE_SIZE and N_PG_HASH
  $ time ./may5-sqlite/sqlite3 foo.db "PRAGMA temp_store = MEMORY; select x, y from big order by y, x" >/dev/null
  real    13m23.828s
  user    13m18.452s
  sys     0m0.811s

  # SQLite built from May 5 2006 CVS, but compiled with proposed change of
  # SQLITE_DEFAULT_PAGE_SIZE set to 4096, and N_PG_HASH set to 16384
  $ time ./may5-sqlite-hash-opt/sqlite3 foo.db "PRAGMA temp_store = MEMORY; select x, y from big order by y, x" >/dev/null
  real    6m16.031s
  user    6m13.108s
  sys     0m0.811s

Compiling with SQLITE_DEFAULT_PAGE_SIZE = 1024, and N_PG_HASH = 32768 resulted in the same timing as the may5-sqlite-hash-opt test run above.

If temp_store=FILE (with default SQLite values for SQLITE_DEFAULT_PAGE_SIZE and N_PG_HASH), the timings are comparable to temp_store=MEMORY with SQLITE_DEFAULT_PAGE_SIZE=4096, and N_PG_HASH=16384.

Large intermediate results sets can cause SQLite to spend more than half of its CPU time in the function pager_lookup(). By increasing the value of N_PG_HASH and SQLITE_DEFAULT_PAGE_SIZE, the time spent in pager_lookup can be reduced to near zero, thus doubling performance in such cases.

    %   cumulative   self               self     total
   time   seconds   seconds     calls  ms/call  ms/call  name
   51.97    118.31   118.31 119658386     0.00     0.00  pager_lookup
    4.36    128.25     9.94   4000009     0.00     0.06  sqlite3VdbeExec
    3.06    135.21     6.96 315629923     0.00     0.00  parseCellPtr
    3.05    142.16     6.95 171797186     0.00     0.00  sqlite3VdbeRecordCompare
    2.67    148.22     6.07  12000005     0.00     0.01  sqlite3BtreeMoveto
    2.14    153.10     4.88 343594380     0.00     0.00  sqlite3VdbeSerialGet
    1.68    156.93     3.83 171797188     0.00     0.00  sqlite3MemCompare
    1.63    160.65     3.72  77995781     0.00     0.00  sqlite3pager_get
    1.60    164.29     3.65 169734946     0.00     0.00  sqlite3pager_unref
    1.58    167.88     3.59 654100795     0.00     0.00  get2byte
2006-May-07 18:37:50 by anonymous:
Timings on same Windows machine with check-in [3180] applied:

  # FILE
  $ time ./may7-sqlite/sqlite3 foo.db "PRAGMA temp_store = FILE; select x, y from big order by y, x" >/dev/null
  real    5m7.157s
  user    4m19.905s
  sys     0m20.827s

  # MEMORY
  $ time ./may7-sqlite/sqlite3 foo.db "PRAGMA temp_store = MEMORY; select x, y from big order by y, x" >/dev/null
  real    5m12.328s
  user    5m9.781s
  sys     0m0.984s

Much better.

temp_store=MEMORY is now competitive with FILE, although temp_store=FILE (when the OS is able to cache the file entirely in memory) is marginally faster.

I still think the MEMORY time can be reduced further by another 20 seconds judging by the sys time of 20.827s in the FILE test. The MEMORY subsystem of SQLite ought to have an advantage over the FILE subsystem because it does not incur any system call overhead. I'll see if a profile turns up anything obvious.

 
1790 code active 2006 May anonymous Pager 2006 May   3 3 :memory: performance difference between v2 and v3 edit
Please see the following link for details: http://www.mail-archive.com/sqlite-users%40sqlite.org/msg14937.html

Possible fix?

  RCS file: /sqlite/sqlite/src/pager.c,v
  retrieving revision 1.266
  diff -u -r1.266 pager.c
  --- pager.c     7 Apr 2006 13:54:47 -0000       1.266
  +++ pager.c     3 May 2006 19:02:17 -0000
  @@ -1663,7 +1663,7 @@
     pPager->memDb = memDb;
     pPager->readOnly = readOnly;
     /* pPager->needSync = 0; */
  -  pPager->noSync = pPager->tempFile || !useJournal;
  +  pPager->noSync = pPager->tempFile || !pPager->useJournal;
     pPager->fullSync = (pPager->noSync?0:1);
     /* pPager->pFirst = 0; */
     /* pPager->pFirstSynced = 0; */
2006-May-03 19:32:12 by drh:
The suggested change makes no difference in performance when I try it.


2006-May-03 21:41:24 by anonymous:
If transactions are not used, 85% of the time of this memory database benchmark is spent in pager_get_all_dirty_pages().

  Each sample counts as 0.01 seconds.
    %   cumulative   self              self     total
   time   seconds   seconds    calls  ms/call  ms/call  name
   85.25     31.20    31.20   100002     0.31     0.31  pager_get_all_dirty_pages
    1.39     31.71     0.51   100011     0.01     0.20  sqlite3VdbeExec
    1.17     32.14     0.43 10487713     0.00     0.00  parseCellPtr
    0.63     32.37     0.23 12943618     0.00     0.00  sqlite3VdbeSerialGet
    0.61     32.59     0.23  3432951     0.00     0.00  pager_lookup
    0.52     32.78     0.19  4849544     0.00     0.00  sqlite3VdbeRecordCompare
    0.44     32.95     0.16   400006     0.00     0.00  sqlite3BtreeMoveto
    0.41     33.09     0.15  2064924     0.00     0.00  sqlite3pager_get
    0.40     33.24     0.14  6471807     0.00     0.00  sqlite3MemCompare

                  0.06   31.25  100002/100002      sqlite3BtreeCommit [4]
  [5]     85.6    0.06   31.25  100002         sqlite3pager_commit [5]
                 31.20    0.00  100002/100002      pager_get_all_dirty_pages [6]
                  0.05    0.00  389365/389365      clearHistory [65]
  -----------------------------------------------
                 31.20    0.00  100002/100002      sqlite3pager_commit [5]
  [6]     85.2   31.20    0.00  100002         pager_get_all_dirty_pages [6]


2006-May-03 21:51:30 by anonymous:
Stats with BEGIN/COMMIT enabled:

  Each sample counts as 0.01 seconds.
    %   cumulative   self              self     total
   time   seconds   seconds    calls  ms/call  ms/call  name
   11.88      0.34     0.34  4849544     0.00     0.00  sqlite3VdbeRecordCompare
    8.16      0.56     0.23 10487713     0.00     0.00  parseCellPtr
    7.80      0.79     0.22 12943618     0.00     0.00  sqlite3VdbeSerialGet
    6.38      0.96     0.18   100013     0.00     0.03  sqlite3VdbeExec
    4.26      1.08     0.12    29816     0.00     0.02  balance_nonroot
    3.90      1.20     0.11  6471807     0.00     0.00  sqlite3MemCompare
    3.19      1.28     0.09  1964925     0.00     0.00  sqlite3pager_get
    3.19      1.38     0.09   400006     0.00     0.00  sqlite3BtreeMoveto
    2.84      1.46     0.08 19170231     0.00     0.00  get2byte
    2.66      1.53     0.07   700015     0.00     0.00  sqlite3VdbeSerialPut
    2.13      1.59     0.06   600993     0.00     0.00  sqlite3Malloc
    1.77      1.64     0.05  4400155     0.00     0.00  sqlite3pager_unref
    1.77      1.69     0.05  3332952     0.00     0.00  pager_lookup
    1.77      1.74     0.05  1418379     0.00     0.00  decodeFlags
    1.77      1.79     0.05  1332302     0.00     0.00  initPage
    1.60      1.83     0.04  5270826     0.00     0.00  findOverflowCell
    1.42      1.88     0.04 12181181     0.00     0.00  findCell
    1.42      1.92     0.04  4849549     0.00     0.00  fetchPayload
    1.42      1.96     0.04   359548     0.00     0.00  insertCell
    1.24      1.99     0.04  4896877     0.00     0.00  parseCell
    1.06      2.02     0.03  5284245     0.00     0.00  cellSizePtr
    1.06      2.05     0.03  3227291     0.00     0.00  binCollFunc
    1.06      2.08     0.03  2616113     0.00     0.00  _page_ref
    1.06      2.11     0.03  1368027     0.00     0.00  reparentPage
    1.06      2.14     0.03   934205     0.00     0.00  sqlite3GenericMalloc
    1.06      2.17     0.03   300010     0.00     0.00  sqlite3BtreeCursor
    0.89      2.19     0.03  2536689     0.00     0.00  get4byte
    0.89      2.22     0.03  1864920     0.00     0.00  getPage
...
    0.00      2.82     0.00        3     0.00     0.00  pager_get_all_dirty_pages

                  0.00    0.00       3/3           sqlite3BtreeCommit [116]
  [119]    0.0    0.00    0.00       3         sqlite3pager_commit [119]
                  0.00    0.00    6551/6551        clearHistory [118]
                  0.00    0.00       3/3           pager_get_all_dirty_pages [370]
  -----------------------------------------------
                  0.00    0.00       3/3           sqlite3pager_commit [119]
  [370]    0.0    0.00    0.00       3         pager_get_all_dirty_pages [370]


2006-May-03 22:27:35 by anonymous:
with the outer BEGIN/COMMIT disabled, the memory database benchmark stats:

  static PgHdr *pager_get_all_dirty_pages(Pager *pPager){
    // this point is reached 100,002 times
    PgHdr *p, *pList;
    pList = 0;
    for(p=pPager->pAll; p; p=p->pNextAll){
      // this point is reached 322,956,271 times
      if( p->dirty ){
        // this point is reached 389,365 times
        p->pDirty = pList;
        pList = p;
      }
    }
    return pList;
  }


2006-May-04 05:23:08 by anonymous:
This patch makes the test (with transaction) run 7% faster for gcc 3.4.4 with -O2. At -O3, gcc performs the inlining of these functions even without the inline hint, so this patch has no effect.

  RCS file: /sqlite/sqlite/src/btree.c,v
  retrieving revision 1.324
  diff -u -3 -p -r1.324 btree.c
  --- btree.c     4 Apr 2006 01:54:55 -0000       1.324
  +++ btree.c     4 May 2006 05:12:35 -0000
  @@ -439,17 +439,17 @@ static int checkReadLocks(BtShared*,Pgno
   /*
   ** Read or write a two- and four-byte big-endian integer values.
   */
  -static u32 get2byte(unsigned char *p){
  +inline static u32 get2byte(unsigned char *p){
     return (p[0]<<8) | p[1];
   }
  -static u32 get4byte(unsigned char *p){
  +inline static u32 get4byte(unsigned char *p){
     return (p[0]<<24) | (p[1]<<16) | (p[2]<<8) | p[3];
   }
  -static void put2byte(unsigned char *p, u32 v){
  +inline static void put2byte(unsigned char *p, u32 v){
     p[0] = v>>8;
     p[1] = v;
   }
  -static void put4byte(unsigned char *p, u32 v){
  +inline static void put4byte(unsigned char *p, u32 v){
     p[0] = v>>24;
     p[1] = v>>16;
     p[2] = v>>8;


2006-May-04 19:44:57 by anonymous:
I just want to confirm that a file database is faster than a memory database for 3.3.5+. Are these numbers correct? 43,478 inserts/second best case for file for 3.3.5+ and 40,000 inserts/second best case for memory? Even with the OS caching the entire database file entirely in RAM, this finding is quite surprising.

  Test    DB      IDX TX  RC      3.3.5+  3.3.5   2.8.17
  1       mem     n   y   1000000 40000   33333   76923
  2       mem     y   y   1000000 27027   22727   58824
  3       mem     n   n   1000000 35714   5263    83333
  4       mem     y   n   1000000 24390   2778    62500
  5       file    n   y   1000000 43478   35714   40000
  6       file    y   y   1000000 28571   24390   23256
  7       file    n   n   1000    11      11      13
  8       file    y   n   1000    9       10      13

http://www.sqlite.org/cvstrac/attach_get/256/sqlite_speed.txt


2006-May-04 20:19:18 by anonymous:
I'm seeing slightly different results. The memory database using a transaction is (slightly) faster than the file-based database using a transaction.

Timings on 3.3.5+ on Windows XP, gcc 3.4.4 -O3 -fomit-frame-pointer

        IDX  TX   # inserts   wall time  inserts/sec
        ---  ---  ---------  ----------  -----------
  mem   no   no     100,000        4.8s       20,833
  mem   no   yes    100,000        4.3s       23,255
  file  no   yes    100,000        4.7s       21,276
  file  no   no       1,000       99.8s           10

...things get worse for :memory: as you increase the number of inserts, while the file database numbers remain constant:

        IDX  TX   # inserts   wall time  inserts/sec
        ---  ---  ---------  ----------  -----------
  mem   no   yes  1,000,000       48.5s       20,638
  mem   no   yes  2,000,000      118.6s       16,863
  mem   no   yes  4,000,000      364.7s       10,967
  file  no   yes  1,000,000       46.8s       21,354
  file  no   yes  2,000,000       93.8s       21,321
  file  no   yes  4,000,000      187.5s       21,333

Do Linux users get similar results?

Considering I have 512K CPU L2 cache, I wonder if there's some CPU cache effect going on here with the way the :memory: db is allocated.


2006-May-04 21:35:07 by anonymous:
It seems there is some quadratic behavior in pager_lookup (latest CVS). 52% of the time is spent in that function. Profile data from :memory: db, TX on, no IDX, 4 million inserts:

  /*
  ** Find a page in the hash table given its page number.  Return
  ** a pointer to the page or NULL if not found.
  */
  static PgHdr *pager_lookup(Pager *pPager, Pgno pgno){
    PgHdr *p = pPager->aHash[pager_hash(pgno)];
    while( p && p->pgno!=pgno ){
      p = p->pNextHash;
    }
    return p;
  }

    %   cumulative   self               self     total
   time   seconds   seconds     calls  ms/call  ms/call  name
   51.97    118.31   118.31 119658386     0.00     0.00  pager_lookup
    4.36    128.25     9.94   4000009     0.00     0.06  sqlite3VdbeExec
    3.06    135.21     6.96 315629923     0.00     0.00  parseCellPtr
    3.05    142.16     6.95 171797186     0.00     0.00  sqlite3VdbeRecordCompare
    2.67    148.22     6.07  12000005     0.00     0.01  sqlite3BtreeMoveto
    2.14    153.10     4.88 343594380     0.00     0.00  sqlite3VdbeSerialGet
    1.68    156.93     3.83 171797188     0.00     0.00  sqlite3MemCompare
    1.63    160.65     3.72  77995781     0.00     0.00  sqlite3pager_get
    1.60    164.29     3.65 169734946     0.00     0.00  sqlite3pager_unref
    1.58    167.88     3.59 654100795     0.00     0.00  get2byte
    1.30    170.84     2.95    973877     0.00     0.07  balance_nonroot
    1.27    173.74     2.90  56939555     0.00     0.00  initPage
    1.24    176.56     2.83 171797188     0.00     0.00  binCollFunc
    0.93    178.69     2.12 386371475     0.00     0.00  findCell
    0.86    180.65     1.96  96207437     0.00     0.00  pageDestructor
    0.83    182.53     1.89  95976540     0.00     0.00  _page_ref
    0.80    184.36     1.83   2708031     0.00     0.00  assemblePage
    0.80    186.19     1.82  41662605     0.00     0.00  reparentPage
    0.74    187.88     1.70 171797188     0.00     0.00  fetchPayload
    0.73    189.55     1.67  73995778     0.00     0.00  getPage
    0.67    191.07     1.52  59647596     0.00     0.00  decodeFlags
    0.63    192.51     1.44 132945443     0.00     0.00  findOverflowCell
    0.62    193.93     1.41  40148167     0.00     0.00  sqlite3PutVarint
    0.59    195.27     1.34 134687272     0.00     0.00  releasePage
    0.59    196.62     1.34  73764879     0.00     0.00  getAndInitPage
    0.54    197.84     1.22   8000003     0.00     0.02  sqlite3BtreeInsert
    0.52    199.01     1.18  60000030     0.00     0.00  sqlite3VdbeSerialType
    0.52    200.19     1.18  24000011     0.00     0.00  moveToRoot
    0.49    201.30     1.10 179797130     0.00     0.00  getCellInfo
    0.43    202.28     0.98   9882306     0.00     0.00  insertCell
    0.42    203.22     0.94  47288132     0.00     0.00  moveToChild
    0.40    204.15     0.92 173434760     0.00     0.00  parseCell
    0.40    205.06     0.91  95806930     0.00     0.00  get4byte
    0.34    205.83     0.78  41662605     0.00     0.00  sqlite3pager_lookup
    0.33    206.57     0.74 165099370     0.00     0.00  sqlite3MallocFailed
    0.32    207.31     0.73  20000010     0.00     0.00  sqlite3VdbeSerialPut
    0.31    208.02     0.71   8000015     0.00     0.00  sqlite3VdbeHalt
    0.30    208.70     0.68  27052986     0.00     0.00  sqlite3GetVarint
    0.28    209.33     0.63 174637767     0.00     0.00  put2byte
    0.27    209.96     0.62   8000006     0.00     0.00  sqlite3BtreeCursor
    0.26    210.54     0.59   8148152     0.00     0.00  fillInCell
    0.25    211.12     0.57   3385610     0.00     0.01  reparentChildPages
    0.25    211.69     0.57  16000006     0.00     0.00  checkReadLocks
    0.22    212.19     0.51  48000093     0.00     0.00  sqlite3VdbeFreeCursor
    0.22    212.69     0.50 133898861     0.00     0.00  cellSizePtr
    0.22    213.19     0.50  24000010     0.00     0.00  popStack
    0.20    213.65     0.46  50076560     0.00     0.00  sqlite3pager_ref
    0.20    214.10     0.45                              pager_reset
    0.19    214.54     0.44   8000024     0.00     0.00  closeAllCursors
    0.19    214.97     0.42  12000024     0.00     0.00  sqlite3VdbeMemMakeWriteable
    0.18    215.38     0.41  32000052     0.00     0.00  sqlite3VdbeMemSetStr
    0.18    215.78     0.40  11616158     0.00     0.00  allocateSpace
    0.17    216.16     0.39   8000000     0.00     0.00  bindText
    0.16    216.51     0.35  25098767     0.00     0.00  sqlite3MallocRaw
    0.16    216.87     0.35   8000005     0.00     0.00  sqlite3BtreeCloseCursor
    0.15    217.22     0.34  45560699     0.00     0.00  sqlite3FreeX
    0.15    217.56     0.34  36000014     0.00     0.00  sqlite3VarintLen
    0.15    217.90     0.34  36000009     0.00     0.00  sqlite3VdbeMemShallowCopy
    0.14    218.22     0.33  47999969     0.00     0.00  sqlite3VdbeSerialTypeLen
    0.14    218.54     0.33   4000008     0.00     0.00  sqlite3VdbeMakeReady

  -----------------------------------------------
                 41.19    0.00 41662605/119658386     sqlite3pager_lookup [15]
                 77.12    0.00 77995781/119658386     sqlite3pager_get [8]
  [5]     52.0  118.31    0.00 119658386         pager_lookup [5]
  -----------------------------------------------
                  0.19    4.02 4000003/77995781     sqlite3BtreeGetMeta [28]
                  3.53   74.30 73995778/77995781     getPage [9]
  [8]     36.0    3.72   78.31 77995781         sqlite3pager_get [8]
                 77.12    0.00 77995781/119658386     pager_lookup [5]
                  1.12    0.00 56939550/95976540     _page_ref [40]
                  0.03    0.00  230897/230897      page_remove_from_stmt_list [139]
                  0.03    0.00  230897/230897      makeClean [138]
                  0.01    0.00  230897/461804      sqlite3pager_pagecount [150]
                  0.00    0.00  230897/25098767     sqlite3MallocRaw [58]
  -----------------------------------------------
                  1.82   44.94 41662605/41662605     reparentChildPages [13]
  [14]    20.5    1.82   44.94 41662605         reparentPage [14]
                  0.78   41.96 41662605/41662605     sqlite3pager_lookup [15]
                  2.21    0.00 41672966/131189801     sqlite3pager_unref <cycle 2> [31]
                  0.00    0.00   93099/50076560     sqlite3pager_ref [75]

  -----------------------------------------------
                  0.78   41.96 41662605/41662605     reparentPage [14]
  [15]    18.8    0.78   41.96 41662605         sqlite3pager_lookup [15]
                 41.19    0.00 41662605/119658386     pager_lookup [5]
                  0.77    0.00 39036990/95976540     _page_ref [40]
  -----------------------------------------------
                  0.77    0.00 39036990/95976540     sqlite3pager_lookup [15]
                  1.12    0.00 56939550/95976540     sqlite3pager_get [8]
  [40]     0.8    1.89    0.00 95976540         _page_ref [40]


2006-May-04 21:41:37 by anonymous:
I guess increasing this array size is in order:

  PgHdr *aHash[N_PG_HASH];    /* Hash table to map page number to PgHdr */

Too many hash collisions leading to growing linked lists in buckets.

Or perhaps pager_hash has to be replaced with a better hash function.


2006-May-04 22:04:47 by anonymous:
Increasing the size of N_PG_HASH to 8192 seems to help the "4 million insert in a transaction into a memory database" benchmark. It now runs in 203.5 seconds (19656 inserts/sec), as opposed to 364.7 seconds (10967 inserts/sec) previously. This is closer to the 187.5 seconds for the file-based database timing.


2006-May-04 22:13:16 by anonymous:
Increasing N_PG_HASH to 16384 yields 21,052 inserts/second for a 4 million insert single-transaction :memory: database no-index run. This is very close to the file database figure of 21,333 inserts/second.


2006-May-04 22:23:19 by anonymous:
Setting N_PG_HASH to 32768 yields 21,621 inserts/second in the 4M insert s in a single-transaction in a memory db test. This is marginally faster than the file based database timing. Increasing N_PG_HASH has diminishing returns after 16384.


2006-May-05 15:33:31 by anonymous:
You should get the same effect if you increase the page size instead of increasing the size of the hash table. With a larger page size there will be fewer pages to be managed by the hash table. This might be a better solution for many applications. A hash table with 32K entries occupies 128K of RAM, whether it is used or not.


2006-May-05 19:37:51 by anonymous:
128K of RAM when dealing with a 230M :memory: database is not terribly significant.

Here's the timings for various N_PG_HASH and SQLITE_DEFAULT_PAGE_SIZE values for 4 million inserts into a :memory: database in a single transaction:

  N_PG_HASH  SQLITE_DEFAULT_PAGE_SIZE  inserts/sec
  ---------  ------------------------  -----------
      16384                      4096       21,622
      32768                      1024       21,621
       8192                      8192       20,513
       4096                      4096       20,101
       4096                      8192       19,417
       2048                      4096       16,878
       2048                      8192       16,598
       2048                     16384       15,038
       2048                     32768       13,937
       2048                      1024       10,782

So it seems the default values of N_PG_HASH and SQLITE_DEFAULT_PAGE_SIZE should be raised.


2006-May-05 21:34:01 by anonymous:
My point was that most users do not have 230 MB memory databases, so having a large hash table which is fixed at that size may be a burden. 128K for the hash table is a lot if you only have 128K in your memmory database.

I agree that increasing these values would seem to provide a substantial performance increase at little cost. I would suggest using the 4K hash table and the 4K page size. These values are close to the current values. Many users have reported a general speed improvement using a page size of 4K which matches the value used by WinXP (and think many other Os's as well) for disk I/O blocks. These values nearly double the insert rate over the current default values. The fixed size hash table only takes twice the space.


2006-May-06 14:54:32 by anonymous:
Memory page speed should be as fast as possible as it effects the general performance of SQLite. Perhaps a static hash table is not the best data structure here. Don't temp tables and intermediate select results on file-based tables use memory-based pages? Making memory page speed as fast as possible will improve overall SQLite performance whether you are using a file or memory based database. For example, when ordering result sets from a file-based database select this routine is used to generate the code:

  static void pushOntoSorter(
    Parse *pParse,         /* Parser context */
    ExprList *pOrderBy,    /* The ORDER BY clause */
    Select *pSelect        /* The whole SELECT statement */
  ){
    Vdbe *v = pParse->pVdbe;
    sqlite3ExprCodeExprList(pParse, pOrderBy);
    sqlite3VdbeAddOp(v, OP_Sequence, pOrderBy->iECursor, 0);
    sqlite3VdbeAddOp(v, OP_Pull, pOrderBy->nExpr + 1, 0);
    sqlite3VdbeAddOp(v, OP_MakeRecord, pOrderBy->nExpr + 2, 0);
    sqlite3VdbeAddOp(v, OP_IdxInsert, pOrderBy->iECursor, 0);

For those of us who have very complicated nested sub-selects of file-based tables in many queries or even ORDER BYs on huge result sets, speeding up the memory page performance should be a performance win for SQLite in general.


2006-May-06 17:37:32 by anonymous:
The following test demonstrates that this memory page issue can greatly effect the performance of queries against file-based tables if temp_store is set to MEMORY.

"big" is a file-based table in foo.db with 10 million rows. It was created with "create table big(x,y)".

  # unmodified stock SQLite built from May 5 2006 CVS (after check-in [3178])
  # compiled with default settings for SQLITE_DEFAULT_PAGE_SIZE and N_PG_HASH
  $ time ./may5-sqlite/sqlite3 foo.db "PRAGMA temp_store = MEMORY; select x, y from big order by y, x" >/dev/null
  real    13m23.828s
  user    13m18.452s
  sys     0m0.811s

  # SQLite built from May 5 2006 CVS, but compiled with proposed change of
  # SQLITE_DEFAULT_PAGE_SIZE set to 4096, and N_PG_HASH set to 16384
  $ time ./may5-sqlite-hash-opt/sqlite3 foo.db "PRAGMA temp_store = MEMORY; select x, y from big order by y, x" >/dev/null
  real    6m16.031s
  user    6m13.108s
  sys     0m0.811s

This is not even what I would consider to be a big table.

I should mention that compiling with SQLITE_DEFAULT_PAGE_SIZE = 1024, and N_PG_HASH = 32768 resulted in same timing as the may5-sqlite-hash-opt test run above. A pretty good return for an extra 126K.


2006-May-08 04:07:25 by anonymous:
You now get 20,725 inserts/second as of the latest check-in [3180] for 4 million inserts into a :memory: database in a single transaction (using the default SQLITE_DEFAULT_PAGE_SIZE of 1K). This is nearly twice as fast as SQLite prior to the check-in [3180] (10,782 inserts/second). However, it is 4% slower than the best timing prior to [3180] when compiled with N_PG_HASH=32768 and SQLITE_DEFAULT_PAGE_SIZE=1024 which got 21,622 inserts/second (see table above). Increasing the size of SQLITE_DEFAULT_PAGE_SIZE with the latest CVS either has no effect or makes the memory insert benchmark timings slightly worse.
 
1798 doc active 2006 May anonymous   2006 May   5 4 preprocessed is misspelled on download page edit
On the download page you'll find the text "proprocessed".
 
1791 code active 2006 May anonymous Unknown 2006 May   1 1 Native threads support for BeOS edit
BeOS ports lacks native thread support. BeOS has very powerful but lightweight threading system, being throughout multithreaded, but it differs from posix-thread ideology, thus our pthreads implementation atm looks more like flacky workaround. Ideally will be to have separate implementation for thread-support, like for Win16/32 versions.

At the moment this problem caused bustage of BeOS Mozilla port, https://bugzilla.mozilla.org/show_bug.cgi?id=330340 nearest workaround might be pthreads usage, inspite its flackyness, but it also causes mess for Mozilla build/configure system, because for other parts in Mozilla we use nspr-threads, which, for BeOS, use native version

2006-Oct-27 05:48:51 by anonymous:
BeOS locking extensions (using native bthreads) have been written and are included in the SQLite3 built into Mozilla Firefox. Is there some process wherein these changes might be incorporated into the SQLite tree?


2006-Oct-27 12:48:11 by anonymous:
Follow the example of OS/2 and propose a patch against the latest SQLite CVS that has proper #ifdef's around BeOS code so it won't break other platforms. Since you're probably the only one interested in this patch, you'll have to do the diffing/merging/testing work yourself.


2006-Nov-07 03:55:36 by anonymous:
Thanks for the advice. We've completed updates to code so it works with the sqlite 3.3.8 patches proposed for Firefox. Current implementation has a parallel os-specific file (os_beos.c). However, with the latest round of locking enhancements to os_unix.c, we're now wondering if it makes more sense to simply enhance this file to support BeOS locking. (yes, we. surprisingly, there is more than one BeOS user left on the planet.) :)
 
1789 doc active 2006 May anonymous Unknown 2006 May drh 4 4 sqlite3_result_error() not adequately documented edit
Documentation for sqlite3_result_error() and friends says "operation of these routines is very similar to the operation of sqlite3_bind_blob() and its cousins", but none of the bind routines are really that similar. So it's not obvious from existing documentation whether the int argument is a string length or, say, a SQLITE_ error code, or a static/transient flag. A glance at /sqlite/src/func.c suggests that it's a static/transient flag, but it's not entirely clear (and if it is, why isn't the signature similar to sqlite3_result_text()?)

sqlite3.h and capi3.html should probably have a little more discussion about returning error situations from user-defined functions.

c.

 
1788 new active 2006 Apr anonymous   2006 Apr   5 5 memory allocator able to return with several allocated bloks at once edit
dlmalloc from Doug Lea (http://g.oswego.edu/dl/html/malloc.html) is public domain memory allocator written in C, replacing the malloc/free family. It is very fast (my microbenchmark had shown amortized allocation or deallocation time around 100 CPU cycles (Athlone XP), while default RTLs for VC++ or Borland were around 500x (really) slower, all three allocators were MT safe). dlmalloc is also used in Linux, I think, as a default allocator in libc.

dlmalloc has one more feature - it is able to allocate several blocks at once. This should be faster than chanin of malloc()s, has better cache locality and possible causes less of fragmentation.

Perhaps SQLite may employ such feature in some CPU bound place.

2006-Apr-29 21:50:53 by anonymous:
Developers are free to link with whatever malloc lib they want. The coding of SQLite itself does not require a change if you happen to use Lea's malloc or Hoard or whatever.


2006-May-01 23:56:15 by anonymous:
That's not my point (I do use dlmalloc). The library may take advantage of ability to allocate several independent memory block during single call to the allocator, expecting it to be faster, having better cache locality and less fragmentation than sequence of calls. Hypothetical effect depend on how much and where SQLite is allocator-bound.
 
1786 warn active 2006 Apr anonymous   2006 Apr   4 4 Yet another list of compiler warnings. edit
I compiled sqlite 3.3.5 from sqlite-source-3_3_5.zip using msvc 2005. Similar warnings show up with both gcc4 on mac (and I assume other platforms) and using sun studio on both solaris and linux. On msvc I set to the higest warning level, then disabled the following warnings which are in my opinion not real issues (4311;4312;4100;4210;4127). Most of the warnings follow the general form of int to smaller size int truncation without explicit cast, float or double sized type to int without explicit cast, or comparison of a signed to unsigned type. The following is my build log showing the warnings:

------ Rebuild All started: Project: sqlite, Configuration: Debug Win32 ------
Deleting intermediate and output files for project 'sqlite', configuration 'Debug|Win32'
Compiling...
alter.c
analyze.c
attach.c
c:\src\sqlite-source-3_3_5\attach.c(314) : warning C4267: 'function' : conversion from 'size_t' to 'int', possible loss of data
auth.c
btree.c
c:\src\sqlite-source-3_3_5\btree.c(449) : warning C4244: '=' : conversion from 'u32' to 'unsigned char', possible loss of data
c:\src\sqlite-source-3_3_5\btree.c(450) : warning C4244: '=' : conversion from 'u32' to 'unsigned char', possible loss of data
c:\src\sqlite-source-3_3_5\btree.c(453) : warning C4244: '=' : conversion from 'u32' to 'unsigned char', possible loss of data
c:\src\sqlite-source-3_3_5\btree.c(454) : warning C4244: '=' : conversion from 'u32' to 'unsigned char', possible loss of data
c:\src\sqlite-source-3_3_5\btree.c(455) : warning C4244: '=' : conversion from 'u32' to 'unsigned char', possible loss of data
c:\src\sqlite-source-3_3_5\btree.c(456) : warning C4244: '=' : conversion from 'u32' to 'unsigned char', possible loss of data
c:\src\sqlite-source-3_3_5\btree.c(538) : warning C4244: 'function' : conversion from 'i64' to 'int', possible loss of data
c:\src\sqlite-source-3_3_5\btree.c(540) : warning C4244: 'function' : conversion from 'i64' to 'u32', possible loss of data
c:\src\sqlite-source-3_3_5\btree.c(831) : warning C4389: '==' : signed/unsigned mismatch
c:\src\sqlite-source-3_3_5\btree.c(898) : warning C4018: '<' : signed/unsigned mismatch
c:\src\sqlite-source-3_3_5\btree.c(955) : warning C4244: '=' : conversion from 'int' to 'u16', possible loss of data
c:\src\sqlite-source-3_3_5\btree.c(961) : warning C4244: '=' : conversion from 'u32' to 'u16', possible loss of data
c:\src\sqlite-source-3_3_5\btree.c(967) : warning C4244: '=' : conversion from 'int' to 'u16', possible loss of data
c:\src\sqlite-source-3_3_5\btree.c(986) : warning C4244: '=' : conversion from 'int' to 'u16', possible loss of data
c:\src\sqlite-source-3_3_5\btree.c(988) : warning C4244: '=' : conversion from 'int' to 'u16', possible loss of data
c:\src\sqlite-source-3_3_5\btree.c(990) : warning C4244: '=' : conversion from 'int' to 'u16', possible loss of data
c:\src\sqlite-source-3_3_5\btree.c(1174) : warning C4389: '==' : signed/unsigned mismatch
c:\src\sqlite-source-3_3_5\btree.c(1226) : warning C4244: '-=' : conversion from 'int' to 'u16', possible loss of data
c:\src\sqlite-source-3_3_5\btree.c(1239) : warning C4244: '=' : conversion from 'int' to 'unsigned char', possible loss of data
c:\src\sqlite-source-3_3_5\btree.c(1303) : warning C4244: '+=' : conversion from 'int' to 'u16', possible loss of data
c:\src\sqlite-source-3_3_5\btree.c(1316) : warning C4244: '-=' : conversion from 'int' to 'unsigned char', possible loss of data
c:\src\sqlite-source-3_3_5\btree.c(1342) : warning C4244: '=' : conversion from 'int' to 'u8', possible loss of data
c:\src\sqlite-source-3_3_5\btree.c(1343) : warning C4244: '=' : conversion from 'int' to 'u8', possible loss of data
c:\src\sqlite-source-3_3_5\btree.c(1344) : warning C4244: '=' : conversion from 'int' to 'u8', possible loss of data
c:\src\sqlite-source-3_3_5\btree.c(1349) : warning C4244: '=' : conversion from 'int' to 'u16', possible loss of data
c:\src\sqlite-source-3_3_5\btree.c(1350) : warning C4244: '=' : conversion from 'int' to 'u16', possible loss of data
c:\src\sqlite-source-3_3_5\btree.c(1353) : warning C4244: '=' : conversion from 'int' to 'u16', possible loss of data
c:\src\sqlite-source-3_3_5\btree.c(1354) : warning C4244: '=' : conversion from 'int' to 'u16', possible loss of data
c:\src\sqlite-source-3_3_5\btree.c(1356) : warning C4244: '=' : conversion from 'int' to 'u8', possible loss of data
c:\src\sqlite-source-3_3_5\btree.c(1405) : warning C4244: '=' : conversion from 'int' to 'u16', possible loss of data
c:\src\sqlite-source-3_3_5\btree.c(1407) : warning C4244: '=' : conversion from 'u32' to 'u16', possible loss of data
c:\src\sqlite-source-3_3_5\btree.c(1435) : warning C4244: '=' : conversion from 'int' to 'u16', possible loss of data
c:\src\sqlite-source-3_3_5\btree.c(1460) : warning C4244: '=' : conversion from 'int' to 'unsigned char', possible loss of data
c:\src\sqlite-source-3_3_5\btree.c(1465) : warning C4244: '=' : conversion from 'int' to 'u16', possible loss of data
c:\src\sqlite-source-3_3_5\btree.c(1467) : warning C4244: '=' : conversion from 'int' to 'u8', possible loss of data
c:\src\sqlite-source-3_3_5\btree.c(1468) : warning C4244: '=' : conversion from 'int' to 'u16', possible loss of data
c:\src\sqlite-source-3_3_5\btree.c(1661) : warning C4244: '=' : conversion from 'int' to 'u8', possible loss of data
c:\src\sqlite-source-3_3_5\btree.c(1663) : warning C4244: '=' : conversion from 'u32' to 'u16', possible loss of data
c:\src\sqlite-source-3_3_5\btree.c(1692) : warning C4244: '=' : conversion from 'int' to 'u16', possible loss of data
c:\src\sqlite-source-3_3_5\btree.c(1865) : warning C4244: '=' : conversion from 'int' to 'u16', possible loss of data
c:\src\sqlite-source-3_3_5\btree.c(1867) : warning C4244: '=' : conversion from 'int' to 'u16', possible loss of data
c:\src\sqlite-source-3_3_5\btree.c(1949) : warning C4244: '=' : conversion from 'int' to 'u16', possible loss of data
c:\src\sqlite-source-3_3_5\btree.c(1950) : warning C4244: '=' : conversion from 'int' to 'u16', possible loss of data
c:\src\sqlite-source-3_3_5\btree.c(2056) : warning C4244: '=' : conversion from 'int' to 'unsigned char', possible loss of data
c:\src\sqlite-source-3_3_5\btree.c(2206) : warning C4244: '=' : conversion from 'int' to 'u8', possible loss of data
c:\src\sqlite-source-3_3_5\btree.c(2267) : warning C4244: '=' : conversion from 'int' to 'u8', possible loss of data
c:\src\sqlite-source-3_3_5\btree.c(2374) : warning C4389: '==' : signed/unsigned mismatch
c:\src\sqlite-source-3_3_5\btree.c(2397) : warning C4389: '==' : signed/unsigned mismatch
c:\src\sqlite-source-3_3_5\btree.c(2402) : warning C4018: '>' : signed/unsigned mismatch
c:\src\sqlite-source-3_3_5\btree.c(2402) : warning C4018: '<=' : signed/unsigned mismatch
c:\src\sqlite-source-3_3_5\btree.c(2405) : warning C4389: '==' : signed/unsigned mismatch
c:\src\sqlite-source-3_3_5\btree.c(2418) : warning C4389: '==' : signed/unsigned mismatch
c:\src\sqlite-source-3_3_5\btree.c(2475) : warning C4389: '!=' : signed/unsigned mismatch
c:\src\sqlite-source-3_3_5\btree.c(2814) : warning C4244: '=' : conversion from 'int' to 'u8', possible loss of data
c:\src\sqlite-source-3_3_5\btree.c(2990) : warning C4244: '=' : conversion from 'i64' to 'u32', possible loss of data
c:\src\sqlite-source-3_3_5\btree.c(2996) : warning C4018: '>' : signed/unsigned mismatch
c:\src\sqlite-source-3_3_5\btree.c(3129) : warning C4244: '=' : conversion from 'i64' to 'u32', possible loss of data
c:\src\sqlite-source-3_3_5\btree.c(3136) : warning C4018: '>' : signed/unsigned mismatch
c:\src\sqlite-source-3_3_5\btree.c(3184) : warning C4244: '=' : conversion from 'int' to 'u16', possible loss of data
c:\src\sqlite-source-3_3_5\btree.c(3448) : warning C4244: 'function' : conversion from 'i64' to 'int', possible loss of data
c:\src\sqlite-source-3_3_5\btree.c(3448) : warning C4244: 'function' : conversion from 'i64' to 'int', possible loss of data
c:\src\sqlite-source-3_3_5\btree.c(3450) : warning C4244: 'function' : conversion from 'i64' to 'int', possible loss of data
c:\src\sqlite-source-3_3_5\btree.c(3452) : warning C4244: 'function' : conversion from 'i64' to 'u32', possible loss of data
c:\src\sqlite-source-3_3_5\btree.c(3453) : warning C4244: 'function' : conversion from 'i64' to 'int', possible loss of data
c:\src\sqlite-source-3_3_5\btree.c(3453) : warning C4244: 'function' : conversion from 'i64' to 'int', possible loss of data
c:\src\sqlite-source-3_3_5\btree.c(3817) : warning C4018: '>' : signed/unsigned mismatch
c:\src\sqlite-source-3_3_5\btree.c(3854) : warning C4389: '!=' : signed/unsigned mismatch
c:\src\sqlite-source-3_3_5\btree.c(3859) : warning C4389: '!=' : signed/unsigned mismatch
c:\src\sqlite-source-3_3_5\btree.c(3869) : warning C4389: '!=' : signed/unsigned mismatch
c:\src\sqlite-source-3_3_5\btree.c(3970) : warning C4018: '>' : signed/unsigned mismatch
c:\src\sqlite-source-3_3_5\btree.c(4029) : warning C4389: '==' : signed/unsigned mismatch
c:\src\sqlite-source-3_3_5\btree.c(4038) : warning C4244: '+=' : conversion from 'i64' to 'int', possible loss of data
c:\src\sqlite-source-3_3_5\btree.c(4040) : warning C4244: '=' : conversion from 'i64' to 'int', possible loss of data

c:\src\sqlite-source-3_3_5\btree.c(4117) : warning C4244: '=' : conversion from 'int' to 'u16', possible loss of data
c:\src\sqlite-source-3_3_5\btree.c(4242) : warning C4244: '=' : conversion from 'int' to 'u16', possible loss of data
c:\src\sqlite-source-3_3_5\btree.c(4259) : warning C4018: '<=' : signed/unsigned mismatch
c:\src\sqlite-source-3_3_5\btree.c(4322) : warning C4244: '-=' : conversion from 'int' to 'u16', possible loss of data
c:\src\sqlite-source-3_3_5\btree.c(4331) : warning C4244: '=' : conversion from 'int' to 'u16', possible loss of data
c:\src\sqlite-source-3_3_5\btree.c(4611) : warning C4244: '=' : conversion from 'int' to 'u16', possible loss of data
c:\src\sqlite-source-3_3_5\btree.c(4695) : warning C4244: '=' : conversion from 'int' to 'u8', possible loss of data
c:\src\sqlite-source-3_3_5\btree.c(5070) : warning C4018: '<=' : signed/unsigned mismatch
c:\src\sqlite-source-3_3_5\btree.c(5499) : warning C4389: '==' : signed/unsigned mismatch
c:\src\sqlite-source-3_3_5\btree.c(5591) : warning C4018: '>' : signed/unsigned mismatch
c:\src\sqlite-source-3_3_5\btree.c(5725) : warning C4389: '==' : signed/unsigned mismatch
c:\src\sqlite-source-3_3_5\btree.c(5768) : warning C4389: '==' : signed/unsigned mismatch
c:\src\sqlite-source-3_3_5\btree.c(5774) : warning C4389: '!=' : signed/unsigned mismatch
c:\src\sqlite-source-3_3_5\btree.c(5895) : warning C4244: '=' : conversion from 'int' to 'u8', possible loss of data
c:\src\sqlite-source-3_3_5\btree.c(5896) : warning C4244: '=' : conversion from 'int' to 'u8', possible loss of data
c:\src\sqlite-source-3_3_5\btree.c(5897) : warning C4244: '=' : conversion from 'int' to 'u8', possible loss of data
c:\src\sqlite-source-3_3_5\btree.c(5898) : warning C4244: '=' : conversion from 'int' to 'u8', possible loss of data
c:\src\sqlite-source-3_3_5\btree.c(5899) : warning C4244: '=' : conversion from 'int' to 'u8', possible loss of data
c:\src\sqlite-source-3_3_5\btree.c(5905) : warning C4244: '=' : conversion from 'int' to 'u16', possible loss of data
c:\src\sqlite-source-3_3_5\btree.c(5924) : warning C4244: '+=' : conversion from 'i64' to 'int', possible loss of data
c:\src\sqlite-source-3_3_5\btree.c(5941) : warning C4244: '=' : conversion from 'u32' to 'u16', possible loss of data
c:\src\sqlite-source-3_3_5\btree.c(5948) : warning C4244: '=' : conversion from 'u32' to 'u16', possible loss of data
c:\src\sqlite-source-3_3_5\btree.c(5958) : warning C4244: '=' : conversion from 'u32' to 'u16', possible loss of data
c:\src\sqlite-source-3_3_5\btree.c(5962) : warning C4244: '=' : conversion from 'int' to 'u8', possible loss of data
c:\src\sqlite-source-3_3_5\btree.c(6281) : warning C4244: '+=' : conversion from 'i64' to 'int', possible loss of data
c:\src\sqlite-source-3_3_5\btree.c(6444) : warning C4389: '!=' : signed/unsigned mismatch

c:\src\sqlite-source-3_3_5\btree.c(6448) : warning C4389: '==' : signed/unsigned mismatch
build.c
c:\src\sqlite-source-3_3_5\build.c(35) : warning C4244: '=' : conversion from 'int' to 'u8', possible loss of data
c:\src\sqlite-source-3_3_5\build.c(183) : warning C4244: 'function' : conversion from '__w64 int' to 'int', possible loss of data
c:\src\sqlite-source-3_3_5\build.c(262) : warning C4267: 'function' : conversion from 'size_t' to 'int', possible loss of data
c:\src\sqlite-source-3_3_5\build.c(320) : warning C4267: 'function' : conversion from 'size_t' to 'int', possible loss of data
c:\src\sqlite-source-3_3_5\build.c(347) : warning C4267: 'function' : conversion from 'size_t' to 'int', possible loss of data
c:\src\sqlite-source-3_3_5\build.c(363) : warning C4267: '=' : conversion from 'size_t' to 'int', possible loss of data
c:\src\sqlite-source-3_3_5\build.c(522) : warning C4267: 'function' : conversion from 'size_t' to 'int', possible loss of data
c:\src\sqlite-source-3_3_5\build.c(552) : warning C4267: 'function' : conversion from 'size_t' to 'int', possible loss of data
c:\src\sqlite-source-3_3_5\build.c(556) : warning C4267: 'initializing' : conversion from 'size_t' to 'int', possible loss of data
c:\src\sqlite-source-3_3_5\build.c(620) : warning C4267: '=' : conversion from 'size_t' to 'int', possible loss of data
c:\src\sqlite-source-3_3_5\build.c(622) : warning C4389: '==' : signed/unsigned mismatch
c:\src\sqlite-source-3_3_5\build.c(955) : warning C4244: '=' : conversion from 'int' to 'u8', possible loss of data
c:\src\sqlite-source-3_3_5\build.c(1127) : warning C4244: '=' : conversion from 'int' to 'u8', possible loss of data
c:\src\sqlite-source-3_3_5\build.c(1128) : warning C4244: '=' : conversion from 'int' to 'u8', possible loss of data
c:\src\sqlite-source-3_3_5\build.c(1216) : warning C4267: '=' : conversion from 'size_t' to 'int', possible loss of data
c:\src\sqlite-source-3_3_5\build.c(1302) : warning C4267: '+=' : conversion from 'size_t' to 'int', possible loss of data
c:\src\sqlite-source-3_3_5\build.c(1319) : warning C4267: '=' : conversion from 'size_t' to 'int', possible loss of data
c:\src\sqlite-source-3_3_5\build.c(1324) : warning C4267: '+=' : conversion from 'size_t' to 'int', possible loss of data
c:\src\sqlite-source-3_3_5\build.c(1330) : warning C4267: '+=' : conversion from 'size_t' to 'int', possible loss of data
c:\src\sqlite-source-3_3_5\build.c(1481) : warning C4244: '=' : conversion from '__w64 int' to 'int', possible loss of data
c:\src\sqlite-source-3_3_5\build.c(1531) : warning C4267: 'function' : conversion from 'size_t' to 'int', possible loss of data
c:\src\sqlite-source-3_3_5\build.c(1538) : warning C4267: 'initializing' : conversion from 'size_t' to 'int', possible loss of data
c:\src\sqlite-source-3_3_5\build.c(1555) : warning C4244: '=' : conversion from '__w64 int' to 'int', possible loss of data
c:\src\sqlite-source-3_3_5\build.c(1624) : warning C4244: '=' : conversion from '__w64 int' to 'int', possible loss of data
c:\src\sqlite-source-3_3_5\build.c(2036) : warning C4267: '+=' : conversion from 'size_t' to 'int', possible loss of data
c:\src\sqlite-source-3_3_5\build.c(2073) : warning C4267: 'initializing' : conversion from 'size_t' to 'int', possible loss of data
c:\src\sqlite-source-3_3_5\build.c(2081) : warning C4244: '=' : conversion from 'int' to 'u8', possible loss of data
c:\src\sqlite-source-3_3_5\build.c(2082) : warning C4244: '=' : conversion from 'int' to 'u8', possible loss of data
c:\src\sqlite-source-3_3_5\build.c(2083) : warning C4244: '=' : conversion from 'int' to 'u8', possible loss of data
c:\src\sqlite-source-3_3_5\build.c(2109) : warning C4244: '=' : conversion from 'int' to 'u8', possible loss of data
c:\src\sqlite-source-3_3_5\build.c(2342) : warning C4267: '=' : conversion from 'size_t' to 'unsigned int', possible loss of data
c:\src\sqlite-source-3_3_5\build.c(2345) : warning C4244: '=' : conversion from 'int' to 'u8', possible loss of data
c:\src\sqlite-source-3_3_5\build.c(2354) : warning C4267: '+=' : conversion from 'size_t' to 'int', possible loss of data
c:\src\sqlite-source-3_3_5\build.c(2361) : warning C4267: '=' : conversion from 'size_t' to 'int', possible loss of data
c:\src\sqlite-source-3_3_5\build.c(2382) : warning C4244: '=' : conversion from 'int' to 'u8', possible loss of data
c:\src\sqlite-source-3_3_5\build.c(2383) : warning C4244: '=' : conversion from 'int' to 'u8', possible loss of data
c:\src\sqlite-source-3_3_5\build.c(2429) : warning C4244: '=' : conversion from 'int' to 'u8', possible loss of data
c:\src\sqlite-source-3_3_5\build.c(2490) : warning C4267: 'function' : conversion from 'size_t' to 'int', possible loss of data
c:\src\sqlite-source-3_3_5\build.c(2873) : warning C4244: '=' : conversion from 'int' to 'i16', possible loss of data
callback.c
c:\src\sqlite-source-3_3_5\callback.c(28) : warning C4267: '=' : conversion from 'size_t' to 'int', possible loss of data
c:\src\sqlite-source-3_3_5\callback.c(59) : warning C4267: 'initializing' : conversion from 'size_t' to 'int', possible loss of data
c:\src\sqlite-source-3_3_5\callback.c(160) : warning C4267: '=' : conversion from 'size_t' to 'int', possible loss of data
c:\src\sqlite-source-3_3_5\callback.c(301) : warning C4244: '=' : conversion from 'int' to 'i16', possible loss of data
complete.c
date.c
c:\src\sqlite-source-3_3_5\date.c(204) : warning C4244: '=' : conversion from 'int' to 'char', possible loss of data
c:\src\sqlite-source-3_3_5\date.c(233) : warning C4244: '=' : conversion from 'double' to 'int', possible loss of data
c:\src\sqlite-source-3_3_5\date.c(234) : warning C4244: '=' : conversion from 'double' to 'int', possible loss of data
c:\src\sqlite-source-3_3_5\date.c(339) : warning C4244: '=' : conversion from 'double' to 'int', possible loss of data
c:\src\sqlite-source-3_3_5\date.c(340) : warning C4244: '=' : conversion from 'double' to 'int', possible loss of data
c:\src\sqlite-source-3_3_5\date.c(343) : warning C4244: '=' : conversion from 'double' to 'int', possible loss of data
c:\src\sqlite-source-3_3_5\date.c(344) : warning C4244: '=' : conversion from 'double' to 'int', possible loss of data
c:\src\sqlite-source-3_3_5\date.c(345) : warning C4244: '=' : conversion from 'double' to 'int', possible loss of data
c:\src\sqlite-source-3_3_5\date.c(346) : warning C4244: '=' : conversion from 'double' to 'int', possible loss of data
c:\src\sqlite-source-3_3_5\date.c(360) : warning C4244: '=' : conversion from 'double' to 'int', possible loss of data
c:\src\sqlite-source-3_3_5\date.c(361) : warning C4244: '=' : conversion from 'double' to 'int', possible loss of data
c:\src\sqlite-source-3_3_5\date.c(363) : warning C4244: '=' : conversion from 'double' to 'int', possible loss of data
c:\src\sqlite-source-3_3_5\date.c(407) : warning C4244: 'initializing' : conversion from 'double' to 'int', possible loss of data
c:\src\sqlite-source-3_3_5\date.c(413) : warning C4244: '=' : conversion from 'double' to 'time_t', possible loss of data
c:\src\sqlite-source-3_3_5\date.c(459) : warning C4244: '=' : conversion from 'int' to 'char', possible loss of data
c:\src\sqlite-source-3_3_5\date.c(508) : warning C4244: '=' : conversion from 'double' to 'int', possible loss of data
c:\src\sqlite-source-3_3_5\date.c(514) : warning C4244: '=' : conversion from 'double' to 'int', possible loss of data
c:\src\sqlite-source-3_3_5\date.c(590) : warning C4267: '=' : conversion from 'size_t' to 'int', possible loss of data
c:\src\sqlite-source-3_3_5\date.c(606) : warning C4244: '+=' : conversion from 'double' to 'int', possible loss of data
c:\src\sqlite-source-3_3_5\date.c(612) : warning C4244: '=' : conversion from 'double' to 'int', possible loss of data
c:\src\sqlite-source-3_3_5\date.c(618) : warning C4244: '+=' : conversion from 'double' to 'int', possible loss of data
c:\src\sqlite-source-3_3_5\date.c(812) : warning C4244: 'initializing' : conversion from 'double' to 'int', possible loss of data
c:\src\sqlite-source-3_3_5\date.c(813) : warning C4244: 'initializing' : conversion from 'double' to 'int', possible loss of data
c:\src\sqlite-source-3_3_5\date.c(815) : warning C4267: '+=' : conversion from 'size_t' to 'int', possible loss of data
c:\src\sqlite-source-3_3_5\date.c(827) : warning C4244: '=' : conversion from 'double' to 'int', possible loss of data
c:\src\sqlite-source-3_3_5\date.c(839) : warning C4267: '+=' : conversion from 'size_t' to 'int', possible loss of data
c:\src\sqlite-source-3_3_5\date.c(844) : warning C4267: '+=' : conversion from 'size_t' to 'int', possible loss of data
c:\src\sqlite-source-3_3_5\date.c(848) : warning C4244: '=' : conversion from 'int' to 'char', possible loss of data
c:\src\sqlite-source-3_3_5\date.c(849) : warning C4267: '+=' : conversion from 'size_t' to 'int', possible loss of data
delete.c
c:\src\sqlite-source-3_3_5\delete.c(71) : warning C4244: 'function' : conversion from 'int' to 'u8', possible loss of data
c:\src\sqlite-source-3_3_5\delete.c(165) : warning C4244: '=' : conversion from 'int' to 'i16', possible loss of data
expr.c
c:\src\sqlite-source-3_3_5\expr.c(201) : warning C4244: '=' : conversion from 'int' to 'u8', possible loss of data
c:\src\sqlite-source-3_3_5\expr.c(346) : warning C4389: '==' : signed/unsigned mismatch
c:\src\sqlite-source-3_3_5\expr.c(1158) : warning C4244: 'function' : conversion from 'int' to 'u8', possible loss of data
c:\src\sqlite-source-3_3_5\expr.c(1160) : warning C4244: 'function' : conversion from 'int' to 'u8', possible loss of data
func.c
c:\src\sqlite-source-3_3_5\func.c(221) : warning C4244: '=' : conversion from 'int' to 'unsigned char', possible loss of data
c:\src\sqlite-source-3_3_5\func.c(234) : warning C4244: '=' : conversion from 'int' to 'unsigned char', possible loss of data
c:\src\sqlite-source-3_3_5\func.c(866) : warning C4244: 'initializing' : conversion from 'i64' to 'int', possible loss of data
c:\src\sqlite-source-3_3_5\func.c(867) : warning C4244: 'initializing' : conversion from 'i64' to 'int', possible loss of data
c:\src\sqlite-source-3_3_5\func.c(868) : warning C4244: 'initializing' : conversion from 'i64' to 'int', possible loss of data
c:\src\sqlite-source-3_3_5\func.c(869) : warning C4244: '=' : conversion from 'int' to 'u8', possible loss of data
c:\src\sqlite-source-3_3_5\func.c(1052) : warning C4267: 'function' : conversion from 'size_t' to 'int', possible loss of data
c:\src\sqlite-source-3_3_5\func.c(1074) : warning C4267: 'function' : conversion from 'size_t' to 'int', possible loss of data
c:\src\sqlite-source-3_3_5\func.c(1096) : warning C4267: 'function' : conversion from 'size_t' to 'int', possible loss of data
c:\src\sqlite-source-3_3_5\func.c(1098) : warning C4244: '=' : conversion from 'int' to 'u8', possible loss of data
hash.c
c:\src\sqlite-source-3_3_5\hash.c(35) : warning C4244: '=' : conversion from 'int' to 'char', possible loss of data
c:\src\sqlite-source-3_3_5\hash.c(39) : warning C4244: '=' : conversion from 'int' to 'char', possible loss of data
c:\src\sqlite-source-3_3_5\hash.c(105) : warning C4267: '=' : conversion from 'size_t' to 'int', possible loss of data
insert.c
c:\src\sqlite-source-3_3_5\insert.c(993) : warning C4267: '=' : conversion from 'size_t' to 'int', possible loss of data
c:\src\sqlite-source-3_3_5\insert.c(996) : warning C4267: '=' : conversion from 'size_t' to 'int', possible loss of data
legacy.c
main.c
c:\src\sqlite-source-3_3_5\main.c(413) : warning C4267: '=' : conversion from 'size_t' to 'int', possible loss of data
c:\src\sqlite-source-3_3_5\main.c(446) : warning C4244: 'function' : conversion from 'int' to 'u8', possible loss of data
c:\src\sqlite-source-3_3_5\main.c(458) : warning C4244: 'function' : conversion from 'int' to 'u8', possible loss of data
c:\src\sqlite-source-3_3_5\main.c(773) : warning C4267: 'function' : conversion from 'size_t' to 'int', possible loss of data
c:\src\sqlite-source-3_3_5\main.c(783) : warning C4267: 'function' : conversion from 'size_t' to 'int', possible loss of data
c:\src\sqlite-source-3_3_5\main.c(787) : warning C4244: '=' : conversion from 'int' to 'u8', possible loss of data
c:\src\sqlite-source-3_3_5\main.c(1102) : warning C4244: '=' : conversion from 'int' to 'u8', possible loss of data
opcodes.c
os.c
os_unix.c
os_win.c
c:\src\sqlite-source-3_3_5\os_win.c(781) : warning C4267: '=' : conversion from 'size_t' to 'int', possible loss of data
c:\src\sqlite-source-3_3_5\os_win.c(785) : warning C4267: '=' : conversion from 'size_t' to 'int', possible loss of data
c:\src\sqlite-source-3_3_5\os_win.c(873) : warning C4244: 'initializing' : conversion from 'i64' to 'LONG', possible loss of data
c:\src\sqlite-source-3_3_5\os_win.c(874) : warning C4244: 'initializing' : conversion from 'i64' to 'LONG', possible loss of data
c:\src\sqlite-source-3_3_5\os_win.c(880) : warning C4244: '=' : conversion from 'i64' to 'int', possible loss of data
c:\src\sqlite-source-3_3_5\os_win.c(915) : warning C4244: 'initializing' : conversion from 'i64' to 'LONG', possible loss of data
c:\src\sqlite-source-3_3_5\os_win.c(919) : warning C4244: 'function' : conversion from 'i64' to 'LONG', possible loss of data
c:\src\sqlite-source-3_3_5\os_win.c(959) : warning C4244: '=' : conversion from 'int' to 'short', possible loss of data
c:\src\sqlite-source-3_3_5\os_win.c(1135) : warning C4244: '=' : conversion from 'int' to 'unsigned char', possible loss of data
c:\src\sqlite-source-3_3_5\os_win.c(1199) : warning C4244: '=' : conversion from 'int' to 'unsigned char', possible loss of data
Generating Code...
c:\src\sqlite-source-3_3_5\os_win.c(856) : warning C4701: potentially uninitialized local variable 'wrote' used
c:\src\sqlite-source-3_3_5\btree.c(5424) : warning C4701: potentially uninitialized local variable 'pNext' used
c:\src\sqlite-source-3_3_5\btree.c(5424) : warning C4701: potentially uninitialized local variable 'szNext' used
Compiling...
pager.c
c:\src\sqlite-source-3_3_5\pager.c(424) : warning C4244: '=' : conversion from 'u32' to 'char', possible loss of data
c:\src\sqlite-source-3_3_5\pager.c(425) : warning C4244: '=' : conversion from 'u32' to 'char', possible loss of data
c:\src\sqlite-source-3_3_5\pager.c(426) : warning C4244: '=' : conversion from 'u32' to 'char', possible loss of data
c:\src\sqlite-source-3_3_5\pager.c(427) : warning C4244: '=' : conversion from 'u32' to 'char', possible loss of data
c:\src\sqlite-source-3_3_5\pager.c(469) : warning C4244: '=' : conversion from 'int' to 'u8', possible loss of data
c:\src\sqlite-source-3_3_5\pager.c(554) : warning C4018: '<' : signed/unsigned mismatch
c:\src\sqlite-source-3_3_5\pager.c(750) : warning C4267: '=' : conversion from 'size_t' to 'int', possible loss of data
c:\src\sqlite-source-3_3_5\pager.c(972) : warning C4389: '==' : signed/unsigned mismatch
c:\src\sqlite-source-3_3_5\pager.c(1075) : warning C4244: 'function' : conversion from 'i64' to 'int', possible loss of data
c:\src\sqlite-source-3_3_5\pager.c(1080) : warning C4244: 'function' : conversion from 'i64' to 'int', possible loss of data
c:\src\sqlite-source-3_3_5\pager.c(1289) : warning C4244: '=' : conversion from 'i64' to 'u32', possible loss of data
c:\src\sqlite-source-3_3_5\pager.c(1297) : warning C4389: '==' : signed/unsigned mismatch
c:\src\sqlite-source-3_3_5\pager.c(1307) : warning C4018: '<' : signed/unsigned mismatch
c:\src\sqlite-source-3_3_5\pager.c(1423) : warning C4244: '=' : conversion from 'i64' to 'u32', possible loss of data
c:\src\sqlite-source-3_3_5\pager.c(1440) : warning C4244: '=' : conversion from 'i64' to 'u32', possible loss of data
c:\src\sqlite-source-3_3_5\pager.c(1498) : warning C4244: '=' : conversion from 'int' to 'u8', possible loss of data
c:\src\sqlite-source-3_3_5\pager.c(1499) : warning C4244: '=' : conversion from 'int' to 'u8', possible loss of data
c:\src\sqlite-source-3_3_5\pager.c(1500) : warning C4244: '=' : conversion from 'int' to 'u8', possible loss of data
c:\src\sqlite-source-3_3_5\pager.c(1617) : warning C4267: '=' : conversion from 'size_t' to 'int', possible loss of data
c:\src\sqlite-source-3_3_5\pager.c(1647) : warning C4244: '=' : conversion from 'int' to 'u8', possible loss of data
c:\src\sqlite-source-3_3_5\pager.c(1648) : warning C4244: '=' : conversion from 'int' to 'u8', possible loss of data
c:\src\sqlite-source-3_3_5\pager.c(1662) : warning C4244: '=' : conversion from 'int' to 'u8', possible loss of data
c:\src\sqlite-source-3_3_5\pager.c(1663) : warning C4244: '=' : conversion from 'int' to 'u8', possible loss of data
c:\src\sqlite-source-3_3_5\pager.c(1664) : warning C4244: '=' : conversion from 'int' to 'u8', possible loss of data
c:\src\sqlite-source-3_3_5\pager.c(1666) : warning C4244: '=' : conversion from 'int' to 'u8', possible loss of data
c:\src\sqlite-source-3_3_5\pager.c(1799) : warning C4244: '=' : conversion from 'i64' to 'int', possible loss of data
c:\src\sqlite-source-3_3_5\pager.c(1805) : warning C4244: 'return' : conversion from 'i64' to 'int', possible loss of data
c:\src\sqlite-source-3_3_5\pager.c(1890) : warning C4018: '<=' : signed/unsigned mismatch
c:\src\sqlite-source-3_3_5\pager.c(1927) : warning C4244: '=' : conversion from 'int' to 'u8', possible loss of data
c:\src\sqlite-source-3_3_5\pager.c(2268) : warning C4018: '<=' : signed/unsigned mismatch
c:\src\sqlite-source-3_3_5\pager.c(2520) : warning C4389: '==' : signed/unsigned mismatch
c:\src\sqlite-source-3_3_5\pager.c(2637) : warning C4244: '=' : conversion from 'int' to 'u8', possible loss of data
c:\src\sqlite-source-3_3_5\pager.c(3009) : warning C4389: '!=' : signed/unsigned mismatch
c:\src\sqlite-source-3_3_5\pager.c(3628) : warning C4018: '<=' : signed/unsigned mismatch
c:\src\sqlite-source-3_3_5\pager.c(3629) : warning C4389: '!=' : signed/unsigned mismatch
parse.c
parse.c(1309) : warning C4244: '=' : conversion from 'int' to 'unsigned char', possible loss of data
parse.y(154) : warning C4244: '=' : conversion from '__w64 unsigned int' to 'unsigned int', possible loss of data
parse.y(229) : warning C4244: '=' : conversion from '__w64 int' to 'unsigned int', possible loss of data
parse.y(233) : warning C4244: '=' : conversion from '__w64 int' to 'unsigned int', possible loss of data
parse.y(379) : warning C4244: '=' : conversion from 'int' to 'u8', possible loss of data
parse.y(451) : warning C4244: '=' : conversion from 'int' to 'u8', possible loss of data
parse.y(532) : warning C4244: '=' : conversion from 'int' to 'u8', possible loss of data
parse.y(536) : warning C4244: '=' : conversion from 'int' to 'u8', possible loss of data
parse.y(871) : warning C4244: '=' : conversion from 'int' to 'u8', possible loss of data
parse.y(880) : warning C4244: '=' : conversion from 'int' to 'u8', possible loss of data
parse.y(919) : warning C4244: '=' : conversion from '__w64 unsigned int' to 'unsigned int', possible loss of data
parse.c(3271) : warning C4244: 'function' : conversion from 'int' to 'unsigned char', possible loss of data
parse.c(3282) : warning C4244: 'function' : conversion from 'int' to 'unsigned char', possible loss of data
pragma.c
c:\src\sqlite-source-3_3_5\pragma.c(49) : warning C4267: '=' : conversion from 'size_t' to 'int', possible loss of data
c:\src\sqlite-source-3_3_5\pragma.c(118) : warning C4244: '=' : conversion from 'int' to 'u8', possible loss of data
c:\src\sqlite-source-3_3_5\pragma.c(443) : warning C4244: '=' : conversion from 'int' to 'u8', possible loss of data
prepare.c
c:\src\sqlite-source-3_3_5\prepare.c(274) : warning C4244: '=' : conversion from 'int' to 'u8', possible loss of data
c:\src\sqlite-source-3_3_5\prepare.c(577) : warning C4244: 'function' : conversion from '__w64 int' to 'int', possible loss of data
printf.c
c:\src\sqlite-source-3_3_5\printf.c(413) : warning C4244: '=' : conversion from '__w64 int' to 'int', possible loss of data
c:\src\sqlite-source-3_3_5\printf.c(426) : warning C4244: '=' : conversion from '__w64 int' to 'int', possible loss of data
c:\src\sqlite-source-3_3_5\printf.c(493) : warning C4244: '=' : conversion from 'int' to 'etByte', possible loss of data
c:\src\sqlite-source-3_3_5\printf.c(503) : warning C4244: '=' : conversion from 'int' to 'char', possible loss of data
c:\src\sqlite-source-3_3_5\printf.c(517) : warning C4244: '=' : conversion from 'int' to 'char', possible loss of data
c:\src\sqlite-source-3_3_5\printf.c(540) : warning C4244: '=' : conversion from 'int' to 'char', possible loss of data
c:\src\sqlite-source-3_3_5\printf.c(543) : warning C4244: '=' : conversion from 'int' to 'char', possible loss of data
c:\src\sqlite-source-3_3_5\printf.c(544) : warning C4244: '=' : conversion from 'int' to 'char', possible loss of data
c:\src\sqlite-source-3_3_5\printf.c(551) : warning C4244: '=' : conversion from '__w64 int' to 'int', possible loss of data
c:\src\sqlite-source-3_3_5\printf.c(579) : warning C4244: '=' : conversion from 'int' to 'char', possible loss of data
c:\src\sqlite-source-3_3_5\printf.c(581) : warning C4244: '=' : conversion from 'int' to 'char', possible loss of data
c:\src\sqlite-source-3_3_5\printf.c(596) : warning C4267: '=' : conversion from 'size_t' to 'int', possible loss of data
c:\src\sqlite-source-3_3_5\printf.c(620) : warning C4244: '=' : conversion from 'int' to 'char', possible loss of data
c:\src\sqlite-source-3_3_5\printf.c(621) : warning C4244: '=' : conversion from 'int' to 'char', possible loss of data
c:\src\sqlite-source-3_3_5\printf.c(644) : warning C4267: 'function' : conversion from 'size_t' to 'int', possible loss of data
c:\src\sqlite-source-3_3_5\printf.c(647) : warning C4267: 'function' : conversion from 'size_t' to 'int', possible loss of data
random.c
c:\src\sqlite-source-3_3_5\random.c(68) : warning C4244: '=' : conversion from 'int' to 'unsigned char', possible loss of data
c:\src\sqlite-source-3_3_5\random.c(83) : warning C4244: '+=' : conversion from 'int' to 'unsigned char', possible loss of data
c:\src\sqlite-source-3_3_5\random.c(86) : warning C4244: '+=' : conversion from 'int' to 'unsigned char', possible loss of data
c:\src\sqlite-source-3_3_5\random.c(97) : warning C4244: '=' : conversion from 'int' to 'unsigned char', possible loss of data
select.c
c:\src\sqlite-source-3_3_5\select.c(69) : warning C4244: '=' : conversion from 'int' to 'u8', possible loss of data
c:\src\sqlite-source-3_3_5\select.c(183) : warning C4267: '=' : conversion from 'size_t' to 'unsigned int', possible loss of data
c:\src\sqlite-source-3_3_5\select.c(557) : warning C4244: 'initializing' : conversion from 'int' to 'char', possible loss of data
c:\src\sqlite-source-3_3_5\select.c(976) : warning C4267: 'function' : conversion from 'size_t' to 'int', possible loss of data
c:\src\sqlite-source-3_3_5\select.c(1004) : warning C4267: 'function' : conversion from 'size_t' to 'int', possible loss of data
c:\src\sqlite-source-3_3_5\select.c(1744) : warning C4244: '=' : conversion from 'int' to 'u8', possible loss of data
c:\src\sqlite-source-3_3_5\select.c(2231) : warning C4244: '=' : conversion from 'int' to 'u8', possible loss of data
shell.c
c:\src\sqlite-source-3_3_5\shell.c(387) : warning C4267: 'initializing' : conversion from 'size_t' to 'int', possible loss of data
c:\src\sqlite-source-3_3_5\shell.c(407) : warning C4267: '=' : conversion from 'size_t' to 'int', possible loss of data
c:\src\sqlite-source-3_3_5\shell.c(409) : warning C4267: '=' : conversion from 'size_t' to 'int', possible loss of data
c:\src\sqlite-source-3_3_5\shell.c(586) : warning C4267: 'initializing' : conversion from 'size_t' to 'int', possible loss of data
c:\src\sqlite-source-3_3_5\shell.c(587) : warning C4267: 'initializing' : conversion from 'size_t' to 'int', possible loss of data
c:\src\sqlite-source-3_3_5\shell.c(738) : warning C4267: 'initializing' : conversion from 'size_t' to 'int', possible loss of data
c:\src\sqlite-source-3_3_5\shell.c(838) : warning C4244: '=' : conversion from 'int' to 'char', possible loss of data
c:\src\sqlite-source-3_3_5\shell.c(880) : warning C4267: '=' : conversion from 'size_t' to 'int', possible loss of data
c:\src\sqlite-source-3_3_5\shell.c(941) : warning C4244: '=' : conversion from 'int' to 'char', possible loss of data
c:\src\sqlite-source-3_3_5\shell.c(961) : warning C4244: '=' : conversion from 'int' to 'char', possible loss of data
c:\src\sqlite-source-3_3_5\shell.c(1005) : warning C4244: '=' : conversion from 'int' to 'char', possible loss of data
c:\src\sqlite-source-3_3_5\shell.c(1035) : warning C4267: '=' : conversion from 'size_t' to 'int', possible loss of data
c:\src\sqlite-source-3_3_5\shell.c(1042) : warning C4267: '=' : conversion from 'size_t' to 'int', possible loss of data
c:\src\sqlite-source-3_3_5\shell.c(1056) : warning C4267: '=' : conversion from 'size_t' to 'int', possible loss of data
c:\src\sqlite-source-3_3_5\shell.c(1148) : warning C4267: 'initializing' : conversion from 'size_t' to 'int', possible loss of data
c:\src\sqlite-source-3_3_5\shell.c(1236) : warning C4244: '=' : conversion from 'int' to 'char', possible loss of data
c:\src\sqlite-source-3_3_5\shell.c(1353) : warning C4267: '=' : conversion from 'size_t' to 'int', possible loss of data
c:\src\sqlite-source-3_3_5\shell.c(1472) : warning C4267: '=' : conversion from 'size_t' to 'int', possible loss of data
c:\src\sqlite-source-3_3_5\shell.c(1477) : warning C4267: 'initializing' : conversion from 'size_t' to 'int', possible loss of data
c:\src\sqlite-source-3_3_5\shell.c(1698) : warning C4013: 'access' undefined; assuming extern returning int
table.c
tokenize.c
trigger.c
c:\src\sqlite-source-3_3_5\trigger.c(113) : warning C4267: 'function' : conversion from 'size_t' to 'int', possible loss of data
c:\src\sqlite-source-3_3_5\trigger.c(172) : warning C4244: '=' : conversion from 'int' to 'u8', possible loss of data
c:\src\sqlite-source-3_3_5\trigger.c(260) : warning C4267: 'function' : conversion from 'size_t' to 'int', possible loss of data
c:\src\sqlite-source-3_3_5\trigger.c(265) : warning C4267: '=' : conversion from 'size_t' to 'int', possible loss of data
c:\src\sqlite-source-3_3_5\trigger.c(454) : warning C4267: '=' : conversion from 'size_t' to 'int', possible loss of data
c:\src\sqlite-source-3_3_5\trigger.c(476) : warning C4267: 'initializing' : conversion from 'size_t' to 'int', possible loss of data
c:\src\sqlite-source-3_3_5\trigger.c(540) : warning C4267: 'initializing' : conversion from 'size_t' to 'int', possible loss of data
c:\src\sqlite-source-3_3_5\trigger.c(631) : warning C4267: '=' : conversion from 'size_t' to 'unsigned int', possible loss of data
update.c
c:\src\sqlite-source-3_3_5\update.c(155) : warning C4244: '=' : conversion from 'int' to 'i16', possible loss of data
utf.c
c:\src\sqlite-source-3_3_5\utf.c(326) : warning C4244: '=' : conversion from 'int' to 'unsigned char', possible loss of data
c:\src\sqlite-source-3_3_5\utf.c(326) : warning C4244: '=' : conversion from 'int' to 'unsigned char', possible loss of data
c:\src\sqlite-source-3_3_5\utf.c(326) : warning C4244: '=' : conversion from 'int' to 'unsigned char', possible loss of data
c:\src\sqlite-source-3_3_5\utf.c(326) : warning C4244: '=' : conversion from 'int' to 'unsigned char', possible loss of data
c:\src\sqlite-source-3_3_5\utf.c(326) : warning C4244: '=' : conversion from 'int' to 'unsigned char', possible loss of data
c:\src\sqlite-source-3_3_5\utf.c(326) : warning C4244: '=' : conversion from 'int' to 'unsigned char', possible loss of data
c:\src\sqlite-source-3_3_5\utf.c(333) : warning C4244: '=' : conversion from 'int' to 'unsigned char', possible loss of data
c:\src\sqlite-source-3_3_5\utf.c(333) : warning C4244: '=' : conversion from 'int' to 'unsigned char', possible loss of data
c:\src\sqlite-source-3_3_5\utf.c(333) : warning C4244: '=' : conversion from 'int' to 'unsigned char', possible loss of data
c:\src\sqlite-source-3_3_5\utf.c(333) : warning C4244: '=' : conversion from 'int' to 'unsigned char', possible loss of data
c:\src\sqlite-source-3_3_5\utf.c(333) : warning C4244: '=' : conversion from 'int' to 'unsigned char', possible loss of data
c:\src\sqlite-source-3_3_5\utf.c(333) : warning C4244: '=' : conversion from 'int' to 'unsigned char', possible loss of data
c:\src\sqlite-source-3_3_5\utf.c(336) : warning C4244: '=' : conversion from '__w64 int' to 'int', possible loss of data
c:\src\sqlite-source-3_3_5\utf.c(344) : warning C4244: '=' : conversion from 'int' to 'unsigned char', possible loss of data
c:\src\sqlite-source-3_3_5\utf.c(344) : warning C4244: '=' : conversion from 'int' to 'unsigned char', possible loss of data
c:\src\sqlite-source-3_3_5\utf.c(344) : warning C4244: '=' : conversion from 'int' to 'unsigned char', possible loss of data
c:\src\sqlite-source-3_3_5\utf.c(344) : warning C4244: '=' : conversion from 'int' to 'unsigned char', possible loss of data
c:\src\sqlite-source-3_3_5\utf.c(344) : warning C4244: '=' : conversion from 'int' to 'unsigned char', possible loss of data
c:\src\sqlite-source-3_3_5\utf.c(344) : warning C4244: '=' : conversion from 'int' to 'unsigned char', possible loss of data
c:\src\sqlite-source-3_3_5\utf.c(344) : warning C4244: '=' : conversion from 'int' to 'unsigned char', possible loss of data
c:\src\sqlite-source-3_3_5\utf.c(344) : warning C4244: '=' : conversion from 'int' to 'unsigned char', possible loss of data
c:\src\sqlite-source-3_3_5\utf.c(344) : warning C4244: '=' : conversion from 'int' to 'unsigned char', possible loss of data
c:\src\sqlite-source-3_3_5\utf.c(344) : warning C4244: '=' : conversion from 'int' to 'unsigned char', possible loss of data
c:\src\sqlite-source-3_3_5\utf.c(350) : warning C4244: '=' : conversion from 'int' to 'unsigned char', possible loss of data
c:\src\sqlite-source-3_3_5\utf.c(350) : warning C4244: '=' : conversion from 'int' to 'unsigned char', possible loss of data
c:\src\sqlite-source-3_3_5\utf.c(350) : warning C4244: '=' : conversion from 'int' to 'unsigned char', possible loss of data
c:\src\sqlite-source-3_3_5\utf.c(350) : warning C4244: '=' : conversion from 'int' to 'unsigned char', possible loss of data
c:\src\sqlite-source-3_3_5\utf.c(350) : warning C4244: '=' : conversion from 'int' to 'unsigned char', possible loss of data
c:\src\sqlite-source-3_3_5\utf.c(350) : warning C4244: '=' : conversion from 'int' to 'unsigned char', possible loss of data
c:\src\sqlite-source-3_3_5\utf.c(350) : warning C4244: '=' : conversion from 'int' to 'unsigned char', possible loss of data
c:\src\sqlite-source-3_3_5\utf.c(350) : warning C4244: '=' : conversion from 'int' to 'unsigned char', possible loss of data
c:\src\sqlite-source-3_3_5\utf.c(350) : warning C4244: '=' : conversion from 'int' to 'unsigned char', possible loss of data
c:\src\sqlite-source-3_3_5\utf.c(350) : warning C4244: '=' : conversion from 'int' to 'unsigned char', possible loss of data
c:\src\sqlite-source-3_3_5\utf.c(353) : warning C4244: '=' : conversion from '__w64 int' to 'int', possible loss of data
c:\src\sqlite-source-3_3_5\utf.c(501) : warning C4244: 'return' : conversion from '__w64 int' to 'int', possible loss of data
c:\src\sqlite-source-3_3_5\utf.c(549) : warning C4244: 'function' : conversion from '__w64 int' to 'int', possible loss of data
util.c
c:\src\sqlite-source-3_3_5\util.c(724) : warning C4267: 'function' : conversion from 'size_t' to 'int', possible loss of data
c:\src\sqlite-source-3_3_5\util.c(756) : warning C4267: '+=' : conversion from 'size_t' to 'int', possible loss of data
c:\src\sqlite-source-3_3_5\util.c(870) : warning C4244: '=' : conversion from 'int' to 'char', possible loss of data
c:\src\sqlite-source-3_3_5\util.c(1040) : warning C4244: '=' : conversion from 'long double' to 'double', possible loss of data
c:\src\sqlite-source-3_3_5\util.c(1041) : warning C4244: 'return' : conversion from '__w64 int' to 'int', possible loss of data
c:\src\sqlite-source-3_3_5\util.c(1226) : warning C4244: '=' : conversion from 'u64' to 'unsigned char', possible loss of data
c:\src\sqlite-source-3_3_5\util.c(1229) : warning C4244: '=' : conversion from 'u64' to 'unsigned char', possible loss of data
c:\src\sqlite-source-3_3_5\util.c(1236) : warning C4244: '=' : conversion from 'u64' to 'u8', possible loss of data
c:\src\sqlite-source-3_3_5\util.c(1355) : warning C4267: 'initializing' : conversion from 'size_t' to 'int', possible loss of data
c:\src\sqlite-source-3_3_5\util.c(1360) : warning C4244: '=' : conversion from 'int' to 'char', possible loss of data
vacuum.c
c:\src\sqlite-source-3_3_5\vacuum.c(131) : warning C4267: '=' : conversion from 'size_t' to 'int', possible loss of data
vdbe.c
c:\src\sqlite-source-3_3_5\vdbe.c(662) : warning C4267: '=' : conversion from 'size_t' to 'int', possible loss of data
c:\src\sqlite-source-3_3_5\vdbe.c(677) : warning C4267: '=' : conversion from 'size_t' to 'int', possible loss of data
c:\src\sqlite-source-3_3_5\vdbe.c(693) : warning C4267: '=' : conversion from 'size_t' to 'int', possible loss of data
c:\src\sqlite-source-3_3_5\vdbe.c(752) : warning C4267: '=' : conversion from 'size_t' to 'int', possible loss of data
c:\src\sqlite-source-3_3_5\vdbe.c(1601) : warning C4244: '=' : conversion from 'int' to 'char', possible loss of data
c:\src\sqlite-source-3_3_5\vdbe.c(1931) : warning C4244: '=' : conversion from 'i64' to 'u32', possible loss of data
c:\src\sqlite-source-3_3_5\vdbe.c(1957) : warning C4018: '<' : signed/unsigned mismatch
c:\src\sqlite-source-3_3_5\vdbe.c(1997) : warning C4018: '>=' : signed/unsigned mismatch
c:\src\sqlite-source-3_3_5\vdbe.c(2004) : warning C4018: '>=' : signed/unsigned mismatch
c:\src\sqlite-source-3_3_5\vdbe.c(2013) : warning C4018: '<' : signed/unsigned mismatch
c:\src\sqlite-source-3_3_5\vdbe.c(2028) : warning C4018: '<' : signed/unsigned mismatch
c:\src\sqlite-source-3_3_5\vdbe.c(2312) : warning C4244: 'initializing' : conversion from 'int' to 'u8', possible loss of data
c:\src\sqlite-source-3_3_5\vdbe.c(2313) : warning C4244: 'initializing' : conversion from 'int' to 'u8', possible loss of data
c:\src\sqlite-source-3_3_5\vdbe.c(2450) : warning C4244: '=' : conversion from 'i64' to 'int', possible loss of data
c:\src\sqlite-source-3_3_5\vdbe.c(2454) : warning C4244: '=' : conversion from 'i64' to 'u8', possible loss of data
c:\src\sqlite-source-3_3_5\vdbe.c(2554) : warning C4244: '=' : conversion from 'i64' to 'int', possible loss of data
c:\src\sqlite-source-3_3_5\vdbe.c(2572) : warning C4244: '=' : conversion from 'i64' to 'int', possible loss of data
c:\src\sqlite-source-3_3_5\vdbe.c(2614) : warning C4244: '=' : conversion from 'int' to 'Bool', possible loss of data
c:\src\sqlite-source-3_3_5\vdbe.c(2615) : warning C4244: '=' : conversion from 'int' to 'Bool', possible loss of data
c:\src\sqlite-source-3_3_5\vdbe.c(2628) : warning C4244: '=' : conversion from 'int' to 'Bool', possible loss of data
c:\src\sqlite-source-3_3_5\vdbe.c(2788) : warning C4244: '=' : conversion from 'int' to 'u8', possible loss of data
c:\src\sqlite-source-3_3_5\vdbe.c(2805) : warning C4244: '=' : conversion from 'int' to 'Bool', possible loss of data
c:\src\sqlite-source-3_3_5\vdbe.c(3064) : warning C4244: '=' : conversion from 'int' to 'Bool', possible loss of data
c:\src\sqlite-source-3_3_5\vdbe.c(3445) : warning C4244: '=' : conversion from 'i64' to 'u32', possible loss of data
c:\src\sqlite-source-3_3_5\vdbe.c(3546) : warning C4244: '=' : conversion from 'int' to 'Bool', possible loss of data
c:\src\sqlite-source-3_3_5\vdbe.c(3595) : warning C4244: '=' : conversion from 'int' to 'Bool', possible loss of data
c:\src\sqlite-source-3_3_5\vdbe.c(3601) : warning C4244: '=' : conversion from 'int' to 'Bool', possible loss of data
c:\src\sqlite-source-3_3_5\vdbe.c(3641) : warning C4244: '=' : conversion from 'int' to 'Bool', possible loss of data
c:\src\sqlite-source-3_3_5\vdbe.c(3809) : warning C4244: '=' : conversion from 'int' to 'u8', possible loss of data
c:\src\sqlite-source-3_3_5\vdbe.c(4118) : warning C4244: '=' : conversion from 'i64' to 'int', possible loss of data
c:\src\sqlite-source-3_3_5\vdbe.c(4131) : warning C4267: '=' : conversion from 'size_t' to 'int', possible loss of data
c:\src\sqlite-source-3_3_5\vdbe.c(4514) : warning C4244: 'initializing' : conversion from 'int' to 'u8', possible loss of data
vdbeapi.c
c:\src\sqlite-source-3_3_5\vdbeapi.c(55) : warning C4244: 'return' : conversion from 'i64' to 'int', possible loss of data
c:\src\sqlite-source-3_3_5\vdbeapi.c(201) : warning C4244: '=' : conversion from 'double' to 'i64', possible loss of data
c:\src\sqlite-source-3_3_5\vdbeapi.c(238) : warning C4244: '=' : conversion from 'double' to 'u64', possible loss of data
c:\src\sqlite-source-3_3_5\vdbeapi.c(650) : warning C4244: 'function' : conversion from 'int' to 'u8', possible loss of data
vdbeaux.c
c:\src\sqlite-source-3_3_5\vdbeaux.c(117) : warning C4244: '=' : conversion from 'int' to 'u8', possible loss of data
c:\src\sqlite-source-3_3_5\vdbeaux.c(482) : warning C4267: '=' : conversion from 'size_t' to 'int', possible loss of data
c:\src\sqlite-source-3_3_5\vdbeaux.c(527) : warning C4267: '=' : conversion from 'size_t' to 'int', possible loss of data
c:\src\sqlite-source-3_3_5\vdbeaux.c(531) : warning C4267: 'initializing' : conversion from 'size_t' to 'int', possible loss of data
c:\src\sqlite-source-3_3_5\vdbeaux.c(564) : warning C4018: '<=' : signed/unsigned mismatch
c:\src\sqlite-source-3_3_5\vdbeaux.c(659) : warning C4267: '=' : conversion from 'size_t' to 'int', possible loss of data
c:\src\sqlite-source-3_3_5\vdbeaux.c(676) : warning C4267: '=' : conversion from 'size_t' to 'int', possible loss of data
c:\src\sqlite-source-3_3_5\vdbeaux.c(862) : warning C4244: 'function' : conversion from '__w64 int' to 'int', possible loss of data
c:\src\sqlite-source-3_3_5\vdbeaux.c(1034) : warning C4267: 'function' : conversion from 'size_t' to 'int', possible loss of data
c:\src\sqlite-source-3_3_5\vdbeaux.c(1526) : warning C4244: '=' : conversion from 'int' to 'Bool', possible loss of data
c:\src\sqlite-source-3_3_5\vdbeaux.c(1595) : warning C4244: 'return' : conversion from 'i64' to 'u32', possible loss of data
c:\src\sqlite-source-3_3_5\vdbeaux.c(1651) : warning C4244: '=' : conversion from 'u64' to 'unsigned char', possible loss of data
c:\src\sqlite-source-3_3_5\vdbeaux.c(1813) : warning C4018: '>=' : signed/unsigned mismatch
c:\src\sqlite-source-3_3_5\vdbeaux.c(1815) : warning C4018: '>=' : signed/unsigned mismatch
c:\src\sqlite-source-3_3_5\vdbeaux.c(1841) : warning C4018: '<' : signed/unsigned mismatch
c:\src\sqlite-source-3_3_5\vdbeaux.c(1843) : warning C4018: '<' : signed/unsigned mismatch
c:\src\sqlite-source-3_3_5\vdbeaux.c(1887) : warning C4244: 'function' : conversion from 'i64' to 'int', possible loss of data
c:\src\sqlite-source-3_3_5\vdbeaux.c(1926) : warning C4244: 'function' : conversion from 'i64' to 'int', possible loss of data
vdbefifo.c
vdbemem.c
c:\src\sqlite-source-3_3_5\vdbemem.c(49) : warning C4244: 'function' : conversion from 'int' to 'u8', possible loss of data
c:\src\sqlite-source-3_3_5\vdbemem.c(194) : warning C4267: '=' : conversion from 'size_t' to 'int', possible loss of data
c:\src\sqlite-source-3_3_5\vdbemem.c(319) : warning C4244: '=' : conversion from 'double' to 'i64', possible loss of data
c:\src\sqlite-source-3_3_5\vdbemem.c(491) : warning C4267: '=' : conversion from 'size_t' to 'int', possible loss of data
c:\src\sqlite-source-3_3_5\vdbemem.c(557) : warning C4244: '=' : conversion from 'const i64' to 'double', possible loss of data
c:\src\sqlite-source-3_3_5\vdbemem.c(562) : warning C4244: '=' : conversion from 'const i64' to 'double', possible loss of data
c:\src\sqlite-source-3_3_5\vdbemem.c(740) : warning C4018: '<=' : signed/unsigned mismatch
c:\src\sqlite-source-3_3_5\vdbemem.c(859) : warning C4267: '=' : conversion from 'size_t' to 'int', possible loss of data
Generating Code...
c:\src\sqlite-source-3_3_5\select.c(3290) : warning C4701: potentially uninitialized local variable 'pTabList' used
c:\src\sqlite-source-3_3_5\select.c(3290) : warning C4701: potentially uninitialized local variable 'pEList' used
c:\src\sqlite-source-3_3_5\select.c(764) : warning C4701: potentially uninitialized local variable 'pseudoTab' used
c:\src\sqlite-source-3_3_5\printf.c(372) : warning C4701: potentially uninitialized local variable 'xtype' used
c:\src\sqlite-source-3_3_5\pager.c(1635) : warning C4701: potentially uninitialized local variable 'nameLen' used
Compiling...
where.c
c:\src\sqlite-source-3_3_5\where.c(227) : warning C4244: '=' : conversion from 'int' to 'u8', possible loss of data
c:\src\sqlite-source-3_3_5\where.c(581) : warning C4244: '=' : conversion from 'int' to 'i16', possible loss of data
c:\src\sqlite-source-3_3_5\where.c(582) : warning C4244: '=' : conversion from 'int' to 'i16', possible loss of data
c:\src\sqlite-source-3_3_5\where.c(583) : warning C4244: '=' : conversion from 'int' to 'u16', possible loss of data
c:\src\sqlite-source-3_3_5\where.c(594) : warning C4244: '=' : conversion from 'int' to 'i16', possible loss of data
c:\src\sqlite-source-3_3_5\where.c(604) : warning C4244: '=' : conversion from 'int' to 'i16', possible loss of data
c:\src\sqlite-source-3_3_5\where.c(605) : warning C4244: '=' : conversion from 'int' to 'i16', possible loss of data
c:\src\sqlite-source-3_3_5\where.c(608) : warning C4244: '=' : conversion from 'int' to 'u16', possible loss of data
c:\src\sqlite-source-3_3_5\where.c(630) : warning C4244: '=' : conversion from 'int' to 'i16', possible loss of data
c:\src\sqlite-source-3_3_5\where.c(702) : warning C4244: '=' : conversion from 'int' to 'i16', possible loss of data
c:\src\sqlite-source-3_3_5\where.c(743) : warning C4244: '=' : conversion from 'int' to 'i16', possible loss of data
c:\src\sqlite-sou