Many hyperlinks are disabled.
Use anonymous login
to enable hyperlinks.
Changes In Branch begin-concurrent-pnu-wal2 Excluding Merge-Ins
This is equivalent to a diff from a93ca38b43 to 5a22010c35
2023-02-02
| ||
20:27 | Instead of free()ing them, retain allocated page buffers on an internal per-connection reuse-list when a page-cache is reset. (check-in: 754f331614 user: dan tags: bedrock) | |
14:27 | Merge latest enhancments into this branch from branch wal2. (Leaf check-in: 5a22010c35 user: dan tags: begin-concurrent-pnu-wal2) | |
14:21 | Merge trunk enhancements into this branch. (check-in: c9b0180203 user: dan tags: wal2) | |
2023-01-12
| ||
15:44 | Fix a startup race condition that could occur if a wal file grows from 0 bytes while a BEGIN CONCURRENT transaction is running. Cherrypick of [c1feca4d]. (check-in: 2c14f3832e user: dan tags: begin-concurrent-pnu-wal2) | |
2019-03-08
| ||
15:30 | Merge all the latest enhancements from trunk. (check-in: 667cce3dce user: drh tags: begin-concurrent-pnu) | |
2018-12-18
| ||
18:01 | Merge the wal2 feature into this branch. (check-in: a2c618c0a4 user: dan tags: begin-concurrent-pnu-wal2) | |
17:47 | Merge latest begin-concurrent changes into this branch. (check-in: a93ca38b43 user: dan tags: begin-concurrent-pnu) | |
17:20 | Merge latest trunk changes into this branch. (check-in: 123cbb3312 user: dan tags: begin-concurrent) | |
2018-12-06
| ||
03:13 | Merge fixes from trunk. (check-in: f2083ee410 user: drh tags: begin-concurrent-pnu) | |
Added LICENSE.md.
> > > > > > | 1 2 3 4 5 6 | The author disclaims copyright to this source code. In place of a legal notice, here is a blessing: * May you do good and not evil. * May you find forgiveness for yourself and forgive others. * May you share freely, never taking more than you give. |
Changes to Makefile.in.
︙ | ︙ | |||
31 32 33 34 35 36 37 | CC = @CC@ CFLAGS = @CPPFLAGS@ @CFLAGS@ TCC = ${CC} ${CFLAGS} -I. -I${TOP}/src -I${TOP}/ext/rtree -I${TOP}/ext/icu TCC += -I${TOP}/ext/fts3 -I${TOP}/ext/async -I${TOP}/ext/session TCC += -I${TOP}/ext/userauth # Define this for the autoconf-based build, so that the code knows it can | | | 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 | CC = @CC@ CFLAGS = @CPPFLAGS@ @CFLAGS@ TCC = ${CC} ${CFLAGS} -I. -I${TOP}/src -I${TOP}/ext/rtree -I${TOP}/ext/icu TCC += -I${TOP}/ext/fts3 -I${TOP}/ext/async -I${TOP}/ext/session TCC += -I${TOP}/ext/userauth # Define this for the autoconf-based build, so that the code knows it can # include the generated sqlite_cfg.h # TCC += -D_HAVE_SQLITE_CONFIG_H -DBUILD_sqlite # Define -DNDEBUG to compile without debugging (i.e., for production usage) # Omitting the define will cause extra debugging code to be inserted and # includes extra comments when "EXPLAIN stmt" is used. # |
︙ | ︙ | |||
161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 | LTLINK = $(LIBTOOL) --mode=link $(TCC) $(LTCOMPILE_EXTRAS) @LDFLAGS@ $(LTLINK_EXTRAS) LTINSTALL = $(LIBTOOL) --mode=install $(INSTALL) # You should not have to change anything below this line ############################################################################### USE_AMALGAMATION = @USE_AMALGAMATION@ # Object files for the SQLite library (non-amalgamation). # LIBOBJS0 = alter.lo analyze.lo attach.lo auth.lo \ backup.lo bitvec.lo btmutex.lo btree.lo build.lo \ callback.lo complete.lo ctime.lo \ date.lo dbpage.lo dbstat.lo delete.lo \ expr.lo fault.lo fkey.lo \ fts3.lo fts3_aux.lo fts3_expr.lo fts3_hash.lo fts3_icu.lo \ fts3_porter.lo fts3_snippet.lo fts3_tokenizer.lo fts3_tokenizer1.lo \ fts3_tokenize_vtab.lo \ fts3_unicode.lo fts3_unicode2.lo fts3_write.lo \ fts5.lo \ func.lo global.lo hash.lo \ | > | | > | | 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 | LTLINK = $(LIBTOOL) --mode=link $(TCC) $(LTCOMPILE_EXTRAS) @LDFLAGS@ $(LTLINK_EXTRAS) LTINSTALL = $(LIBTOOL) --mode=install $(INSTALL) # You should not have to change anything below this line ############################################################################### USE_AMALGAMATION = @USE_AMALGAMATION@ AMALGAMATION_LINE_MACROS = @AMALGAMATION_LINE_MACROS@ # Object files for the SQLite library (non-amalgamation). # LIBOBJS0 = alter.lo analyze.lo attach.lo auth.lo \ backup.lo bitvec.lo btmutex.lo btree.lo build.lo \ callback.lo complete.lo ctime.lo \ date.lo dbpage.lo dbstat.lo delete.lo \ expr.lo fault.lo fkey.lo \ fts3.lo fts3_aux.lo fts3_expr.lo fts3_hash.lo fts3_icu.lo \ fts3_porter.lo fts3_snippet.lo fts3_tokenizer.lo fts3_tokenizer1.lo \ fts3_tokenize_vtab.lo \ fts3_unicode.lo fts3_unicode2.lo fts3_write.lo \ fts5.lo \ func.lo global.lo hash.lo \ icu.lo insert.lo json.lo legacy.lo loadext.lo \ main.lo malloc.lo mem0.lo mem1.lo mem2.lo mem3.lo mem5.lo \ memdb.lo memjournal.lo \ mutex.lo mutex_noop.lo mutex_unix.lo mutex_w32.lo \ notify.lo opcodes.lo os.lo os_kv.lo os_unix.lo os_win.lo \ pager.lo parse.lo pcache.lo pcache1.lo pragma.lo prepare.lo printf.lo \ random.lo resolve.lo rowset.lo rtree.lo \ sqlite3session.lo select.lo sqlite3rbu.lo status.lo stmt.lo \ table.lo threads.lo tokenize.lo treeview.lo trigger.lo \ update.lo userauth.lo upsert.lo util.lo vacuum.lo \ vdbe.lo vdbeapi.lo vdbeaux.lo vdbeblob.lo vdbemem.lo vdbesort.lo \ vdbetrace.lo vdbevtab.lo \ wal.lo walker.lo where.lo wherecode.lo whereexpr.lo \ window.lo utf.lo vtab.lo # Object files for the amalgamation. # LIBOBJS1 = sqlite3.lo # Determine the real value of LIBOBJ based on the 'configure' script |
︙ | ︙ | |||
228 229 230 231 232 233 234 235 236 237 238 239 240 241 | $(TOP)/src/fkey.c \ $(TOP)/src/func.c \ $(TOP)/src/global.c \ $(TOP)/src/hash.c \ $(TOP)/src/hash.h \ $(TOP)/src/hwtime.h \ $(TOP)/src/insert.c \ $(TOP)/src/legacy.c \ $(TOP)/src/loadext.c \ $(TOP)/src/main.c \ $(TOP)/src/malloc.c \ $(TOP)/src/mem0.c \ $(TOP)/src/mem1.c \ $(TOP)/src/mem2.c \ | > | 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 | $(TOP)/src/fkey.c \ $(TOP)/src/func.c \ $(TOP)/src/global.c \ $(TOP)/src/hash.c \ $(TOP)/src/hash.h \ $(TOP)/src/hwtime.h \ $(TOP)/src/insert.c \ $(TOP)/src/json.c \ $(TOP)/src/legacy.c \ $(TOP)/src/loadext.c \ $(TOP)/src/main.c \ $(TOP)/src/malloc.c \ $(TOP)/src/mem0.c \ $(TOP)/src/mem1.c \ $(TOP)/src/mem2.c \ |
︙ | ︙ | |||
250 251 252 253 254 255 256 257 258 259 260 261 262 263 | $(TOP)/src/mutex_unix.c \ $(TOP)/src/mutex_w32.c \ $(TOP)/src/notify.c \ $(TOP)/src/os.c \ $(TOP)/src/os.h \ $(TOP)/src/os_common.h \ $(TOP)/src/os_setup.h \ $(TOP)/src/os_unix.c \ $(TOP)/src/os_win.c \ $(TOP)/src/os_win.h \ $(TOP)/src/pager.c \ $(TOP)/src/pager.h \ $(TOP)/src/parse.y \ $(TOP)/src/pcache.c \ | > | 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 | $(TOP)/src/mutex_unix.c \ $(TOP)/src/mutex_w32.c \ $(TOP)/src/notify.c \ $(TOP)/src/os.c \ $(TOP)/src/os.h \ $(TOP)/src/os_common.h \ $(TOP)/src/os_setup.h \ $(TOP)/src/os_kv.c \ $(TOP)/src/os_unix.c \ $(TOP)/src/os_win.c \ $(TOP)/src/os_win.h \ $(TOP)/src/pager.c \ $(TOP)/src/pager.h \ $(TOP)/src/parse.y \ $(TOP)/src/pcache.c \ |
︙ | ︙ | |||
292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 | $(TOP)/src/vdbe.h \ $(TOP)/src/vdbeapi.c \ $(TOP)/src/vdbeaux.c \ $(TOP)/src/vdbeblob.c \ $(TOP)/src/vdbemem.c \ $(TOP)/src/vdbesort.c \ $(TOP)/src/vdbetrace.c \ $(TOP)/src/vdbeInt.h \ $(TOP)/src/vtab.c \ $(TOP)/src/vxworks.h \ $(TOP)/src/wal.c \ $(TOP)/src/wal.h \ $(TOP)/src/walker.c \ $(TOP)/src/where.c \ $(TOP)/src/wherecode.c \ $(TOP)/src/whereexpr.c \ $(TOP)/src/whereInt.h \ $(TOP)/src/window.c # Source code for extensions # SRC += \ | > < < < < < < < < < < < < < < < < < < | 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 | $(TOP)/src/vdbe.h \ $(TOP)/src/vdbeapi.c \ $(TOP)/src/vdbeaux.c \ $(TOP)/src/vdbeblob.c \ $(TOP)/src/vdbemem.c \ $(TOP)/src/vdbesort.c \ $(TOP)/src/vdbetrace.c \ $(TOP)/src/vdbevtab.c \ $(TOP)/src/vdbeInt.h \ $(TOP)/src/vtab.c \ $(TOP)/src/vxworks.h \ $(TOP)/src/wal.c \ $(TOP)/src/wal.h \ $(TOP)/src/walker.c \ $(TOP)/src/where.c \ $(TOP)/src/wherecode.c \ $(TOP)/src/whereexpr.c \ $(TOP)/src/whereInt.h \ $(TOP)/src/window.c # Source code for extensions # SRC += \ $(TOP)/ext/fts3/fts3.c \ $(TOP)/ext/fts3/fts3.h \ $(TOP)/ext/fts3/fts3Int.h \ $(TOP)/ext/fts3/fts3_aux.c \ $(TOP)/ext/fts3/fts3_expr.c \ $(TOP)/ext/fts3/fts3_hash.c \ $(TOP)/ext/fts3/fts3_hash.h \ |
︙ | ︙ | |||
359 360 361 362 363 364 365 | SRC += \ $(TOP)/ext/userauth/userauth.c \ $(TOP)/ext/userauth/sqlite3userauth.h SRC += \ $(TOP)/ext/rbu/sqlite3rbu.h \ $(TOP)/ext/rbu/sqlite3rbu.c SRC += \ | < | | 346 347 348 349 350 351 352 353 354 355 356 357 358 359 360 361 362 363 364 365 366 367 368 369 370 | SRC += \ $(TOP)/ext/userauth/userauth.c \ $(TOP)/ext/userauth/sqlite3userauth.h SRC += \ $(TOP)/ext/rbu/sqlite3rbu.h \ $(TOP)/ext/rbu/sqlite3rbu.c SRC += \ $(TOP)/ext/misc/stmt.c # Generated source code files # SRC += \ keywordhash.h \ opcodes.c \ opcodes.h \ parse.c \ parse.h \ sqlite_cfg.h \ shell.c \ sqlite3.h # Source code to the test files. # TESTSRC = \ $(TOP)/src/test1.c \ |
︙ | ︙ | |||
418 419 420 421 422 423 424 425 426 427 428 429 430 431 432 433 434 435 436 437 438 439 440 441 442 443 444 445 446 447 448 449 450 451 452 453 454 455 456 457 458 459 460 461 462 | $(TOP)/src/test_schema.c \ $(TOP)/src/test_server.c \ $(TOP)/src/test_superlock.c \ $(TOP)/src/test_syscall.c \ $(TOP)/src/test_tclsh.c \ $(TOP)/src/test_tclvar.c \ $(TOP)/src/test_thread.c \ $(TOP)/src/test_vfs.c \ $(TOP)/src/test_windirent.c \ $(TOP)/src/test_window.c \ $(TOP)/src/test_wsd.c \ $(TOP)/ext/fts3/fts3_term.c \ $(TOP)/ext/fts3/fts3_test.c \ $(TOP)/ext/session/test_session.c \ $(TOP)/ext/rbu/test_rbu.c # Statically linked extensions # TESTSRC += \ $(TOP)/ext/expert/sqlite3expert.c \ $(TOP)/ext/expert/test_expert.c \ $(TOP)/ext/misc/amatch.c \ $(TOP)/ext/misc/carray.c \ $(TOP)/ext/misc/closure.c \ $(TOP)/ext/misc/csv.c \ $(TOP)/ext/misc/eval.c \ $(TOP)/ext/misc/explain.c \ $(TOP)/ext/misc/fileio.c \ $(TOP)/ext/misc/fuzzer.c \ $(TOP)/ext/fts5/fts5_tcl.c \ $(TOP)/ext/fts5/fts5_test_mi.c \ $(TOP)/ext/fts5/fts5_test_tok.c \ $(TOP)/ext/misc/ieee754.c \ $(TOP)/ext/misc/mmapwarm.c \ $(TOP)/ext/misc/nextchar.c \ $(TOP)/ext/misc/normalize.c \ $(TOP)/ext/misc/percentile.c \ $(TOP)/ext/misc/regexp.c \ $(TOP)/ext/misc/remember.c \ $(TOP)/ext/misc/series.c \ $(TOP)/ext/misc/spellfix.c \ $(TOP)/ext/misc/totype.c \ $(TOP)/ext/misc/unionvtab.c \ $(TOP)/ext/misc/wholenumber.c \ $(TOP)/ext/misc/zipfile.c \ | > > > > > > > > > > > > | > | 404 405 406 407 408 409 410 411 412 413 414 415 416 417 418 419 420 421 422 423 424 425 426 427 428 429 430 431 432 433 434 435 436 437 438 439 440 441 442 443 444 445 446 447 448 449 450 451 452 453 454 455 456 457 458 459 460 461 462 463 464 465 466 467 468 469 | $(TOP)/src/test_schema.c \ $(TOP)/src/test_server.c \ $(TOP)/src/test_superlock.c \ $(TOP)/src/test_syscall.c \ $(TOP)/src/test_tclsh.c \ $(TOP)/src/test_tclvar.c \ $(TOP)/src/test_thread.c \ $(TOP)/src/test_vdbecov.c \ $(TOP)/src/test_vfs.c \ $(TOP)/src/test_windirent.c \ $(TOP)/src/test_window.c \ $(TOP)/src/test_wsd.c \ $(TOP)/ext/fts3/fts3_term.c \ $(TOP)/ext/fts3/fts3_test.c \ $(TOP)/ext/session/test_session.c \ $(TOP)/ext/session/sqlite3changebatch.c \ $(TOP)/ext/recover/sqlite3recover.c \ $(TOP)/ext/recover/dbdata.c \ $(TOP)/ext/recover/test_recover.c \ $(TOP)/ext/rbu/test_rbu.c # Statically linked extensions # TESTSRC += \ $(TOP)/ext/expert/sqlite3expert.c \ $(TOP)/ext/expert/test_expert.c \ $(TOP)/ext/misc/amatch.c \ $(TOP)/ext/misc/bgckpt.c \ $(TOP)/ext/misc/appendvfs.c \ $(TOP)/ext/misc/basexx.c \ $(TOP)/ext/misc/carray.c \ $(TOP)/ext/misc/cksumvfs.c \ $(TOP)/ext/misc/closure.c \ $(TOP)/ext/misc/csv.c \ $(TOP)/ext/misc/decimal.c \ $(TOP)/ext/misc/eval.c \ $(TOP)/ext/misc/explain.c \ $(TOP)/ext/misc/fileio.c \ $(TOP)/ext/misc/fuzzer.c \ $(TOP)/ext/fts5/fts5_tcl.c \ $(TOP)/ext/fts5/fts5_test_mi.c \ $(TOP)/ext/fts5/fts5_test_tok.c \ $(TOP)/ext/misc/ieee754.c \ $(TOP)/ext/misc/mmapwarm.c \ $(TOP)/ext/misc/nextchar.c \ $(TOP)/ext/misc/normalize.c \ $(TOP)/ext/misc/percentile.c \ $(TOP)/ext/misc/prefixes.c \ $(TOP)/ext/misc/qpvtab.c \ $(TOP)/ext/misc/regexp.c \ $(TOP)/ext/misc/remember.c \ $(TOP)/ext/misc/series.c \ $(TOP)/ext/misc/spellfix.c \ $(TOP)/ext/misc/totype.c \ $(TOP)/ext/misc/unionvtab.c \ $(TOP)/ext/misc/wholenumber.c \ $(TOP)/ext/misc/zipfile.c \ $(TOP)/ext/userauth/userauth.c \ $(TOP)/ext/rtree/test_rtreedoc.c # Source code to the library files needed by the test fixture # TESTSRC2 = \ $(TOP)/src/attach.c \ $(TOP)/src/backup.c \ $(TOP)/src/bitvec.c \ |
︙ | ︙ | |||
478 479 480 481 482 483 484 485 486 487 488 489 490 491 492 493 494 495 496 497 498 499 500 501 502 503 504 505 506 507 508 509 510 511 512 513 514 515 | $(TOP)/src/func.c \ $(TOP)/src/global.c \ $(TOP)/src/insert.c \ $(TOP)/src/wal.c \ $(TOP)/src/main.c \ $(TOP)/src/mem5.c \ $(TOP)/src/os.c \ $(TOP)/src/os_unix.c \ $(TOP)/src/os_win.c \ $(TOP)/src/pager.c \ $(TOP)/src/pragma.c \ $(TOP)/src/prepare.c \ $(TOP)/src/printf.c \ $(TOP)/src/random.c \ $(TOP)/src/pcache.c \ $(TOP)/src/pcache1.c \ $(TOP)/src/select.c \ $(TOP)/src/tokenize.c \ $(TOP)/src/utf.c \ $(TOP)/src/util.c \ $(TOP)/src/vdbeapi.c \ $(TOP)/src/vdbeaux.c \ $(TOP)/src/vdbe.c \ $(TOP)/src/vdbemem.c \ $(TOP)/src/vdbetrace.c \ $(TOP)/src/where.c \ $(TOP)/src/wherecode.c \ $(TOP)/src/whereexpr.c \ $(TOP)/src/window.c \ parse.c \ $(TOP)/ext/fts3/fts3.c \ $(TOP)/ext/fts3/fts3_aux.c \ $(TOP)/ext/fts3/fts3_expr.c \ $(TOP)/ext/fts3/fts3_term.c \ $(TOP)/ext/fts3/fts3_tokenizer.c \ $(TOP)/ext/fts3/fts3_write.c \ $(TOP)/ext/async/sqlite3async.c \ $(TOP)/ext/session/sqlite3session.c \ | > > > | > | 477 478 479 480 481 482 483 484 485 486 487 488 489 490 491 492 493 494 495 496 497 498 499 500 501 502 503 504 505 506 507 508 509 510 511 512 513 514 515 516 517 518 519 520 521 522 523 524 525 526 | $(TOP)/src/func.c \ $(TOP)/src/global.c \ $(TOP)/src/insert.c \ $(TOP)/src/wal.c \ $(TOP)/src/main.c \ $(TOP)/src/mem5.c \ $(TOP)/src/os.c \ $(TOP)/src/os_kv.c \ $(TOP)/src/os_unix.c \ $(TOP)/src/os_win.c \ $(TOP)/src/pager.c \ $(TOP)/src/pragma.c \ $(TOP)/src/prepare.c \ $(TOP)/src/printf.c \ $(TOP)/src/random.c \ $(TOP)/src/pcache.c \ $(TOP)/src/pcache1.c \ $(TOP)/src/select.c \ $(TOP)/src/tokenize.c \ $(TOP)/src/treeview.c \ $(TOP)/src/utf.c \ $(TOP)/src/util.c \ $(TOP)/src/vdbeapi.c \ $(TOP)/src/vdbeaux.c \ $(TOP)/src/vdbe.c \ $(TOP)/src/vdbemem.c \ $(TOP)/src/vdbetrace.c \ $(TOP)/src/vdbevtab.c \ $(TOP)/src/where.c \ $(TOP)/src/wherecode.c \ $(TOP)/src/whereexpr.c \ $(TOP)/src/window.c \ parse.c \ $(TOP)/ext/fts3/fts3.c \ $(TOP)/ext/fts3/fts3_aux.c \ $(TOP)/ext/fts3/fts3_expr.c \ $(TOP)/ext/fts3/fts3_term.c \ $(TOP)/ext/fts3/fts3_tokenizer.c \ $(TOP)/ext/fts3/fts3_write.c \ $(TOP)/ext/async/sqlite3async.c \ $(TOP)/ext/session/sqlite3session.c \ $(TOP)/ext/misc/stmt.c \ fts5.c # Header files used by all library source files. # HDR = \ $(TOP)/src/btree.h \ $(TOP)/src/btreeInt.h \ $(TOP)/src/hash.h \ |
︙ | ︙ | |||
538 539 540 541 542 543 544 | $(TOP)/src/sqlite3ext.h \ $(TOP)/src/sqliteInt.h \ $(TOP)/src/sqliteLimit.h \ $(TOP)/src/vdbe.h \ $(TOP)/src/vdbeInt.h \ $(TOP)/src/vxworks.h \ $(TOP)/src/whereInt.h \ | | < < < < < < < < | 541 542 543 544 545 546 547 548 549 550 551 552 553 554 555 556 557 558 559 | $(TOP)/src/sqlite3ext.h \ $(TOP)/src/sqliteInt.h \ $(TOP)/src/sqliteLimit.h \ $(TOP)/src/vdbe.h \ $(TOP)/src/vdbeInt.h \ $(TOP)/src/vxworks.h \ $(TOP)/src/whereInt.h \ sqlite_cfg.h # Header files used by extensions # EXTHDR += \ $(TOP)/ext/fts3/fts3.h \ $(TOP)/ext/fts3/fts3Int.h \ $(TOP)/ext/fts3/fts3_hash.h \ $(TOP)/ext/fts3/fts3_tokenizer.h EXTHDR += \ $(TOP)/ext/rtree/rtree.h \ $(TOP)/ext/rtree/geopoly.c |
︙ | ︙ | |||
584 585 586 587 588 589 590 | FUZZDATA = \ $(TOP)/test/fuzzdata1.db \ $(TOP)/test/fuzzdata2.db \ $(TOP)/test/fuzzdata3.db \ $(TOP)/test/fuzzdata4.db \ $(TOP)/test/fuzzdata5.db \ $(TOP)/test/fuzzdata6.db \ | | > | | | < | > > > | > > > > > > > | > > > > > > | 579 580 581 582 583 584 585 586 587 588 589 590 591 592 593 594 595 596 597 598 599 600 601 602 603 604 605 606 607 608 609 610 611 612 613 614 615 616 617 618 619 620 621 622 623 624 625 626 627 628 629 630 631 632 633 | FUZZDATA = \ $(TOP)/test/fuzzdata1.db \ $(TOP)/test/fuzzdata2.db \ $(TOP)/test/fuzzdata3.db \ $(TOP)/test/fuzzdata4.db \ $(TOP)/test/fuzzdata5.db \ $(TOP)/test/fuzzdata6.db \ $(TOP)/test/fuzzdata7.db \ $(TOP)/test/fuzzdata8.db # Standard options to testfixture # TESTOPTS = --verbose=file --output=test-out.txt # Extra compiler options for various shell tools # SHELL_OPT = -DSQLITE_ENABLE_FTS4 #SHELL_OPT += -DSQLITE_ENABLE_FTS5 SHELL_OPT += -DSQLITE_ENABLE_RTREE SHELL_OPT += -DSQLITE_ENABLE_EXPLAIN_COMMENTS SHELL_OPT += -DSQLITE_ENABLE_UNKNOWN_SQL_FUNCTION SHELL_OPT += -DSQLITE_ENABLE_STMTVTAB SHELL_OPT += -DSQLITE_ENABLE_DBPAGE_VTAB SHELL_OPT += -DSQLITE_ENABLE_DBSTAT_VTAB SHELL_OPT += -DSQLITE_ENABLE_BYTECODE_VTAB SHELL_OPT += -DSQLITE_ENABLE_OFFSET_SQL_FUNC FUZZERSHELL_OPT = FUZZCHECK_OPT += -I$(TOP)/test FUZZCHECK_OPT += -I$(TOP)/ext/recover FUZZCHECK_OPT += -DSQLITE_OMIT_LOAD_EXTENSION FUZZCHECK_OPT += -DSQLITE_ENABLE_MEMSYS5 -DSQLITE_OSS_FUZZ FUZZCHECK_OPT += -DSQLITE_MAX_MEMORY=50000000 FUZZCHECK_OPT += -DSQLITE_PRINTF_PRECISION_LIMIT=1000 FUZZCHECK_OPT += -DSQLITE_ENABLE_FTS4 FUZZCHECK_OPT += -DSQLITE_ENABLE_FTS3_PARENTHESIS FUZZCHECK_OPT += -DSQLITE_ENABLE_FTS5 FUZZCHECK_OPT += -DSQLITE_ENABLE_RTREE FUZZCHECK_OPT += -DSQLITE_ENABLE_GEOPOLY FUZZCHECK_OPT += -DSQLITE_ENABLE_DBSTAT_VTAB FUZZCHECK_OPT += -DSQLITE_ENABLE_BYTECODE_VTAB FUZZCHECK_SRC += $(TOP)/test/fuzzcheck.c FUZZCHECK_SRC += $(TOP)/test/ossfuzz.c FUZZCHECK_SRC += $(TOP)/test/fuzzinvariants.c FUZZCHECK_SRC += $(TOP)/ext/recover/dbdata.c FUZZCHECK_SRC += $(TOP)/ext/recover/sqlite3recover.c FUZZCHECK_SRC += $(TOP)/test/vt02.c DBFUZZ_OPT = ST_OPT = -DSQLITE_OS_KV_OPTIONAL # This is the default Makefile target. The objects listed here # are what get build when you type just "make" with no arguments. # all: sqlite3.h libsqlite3.la sqlite3$(TEXE) $(HAVE_TCL:1=libtclsqlite3.la) Makefile: $(TOP)/Makefile.in |
︙ | ︙ | |||
657 658 659 660 661 662 663 | sourcetest: srcck1$(BEXE) sqlite3.c ./srcck1 sqlite3.c fuzzershell$(TEXE): $(TOP)/tool/fuzzershell.c sqlite3.c sqlite3.h $(LTLINK) -o $@ $(FUZZERSHELL_OPT) \ $(TOP)/tool/fuzzershell.c sqlite3.c $(TLIBS) | | < > | > > > > > > > | | | > > > > > > > | 668 669 670 671 672 673 674 675 676 677 678 679 680 681 682 683 684 685 686 687 688 689 690 691 692 693 694 695 696 697 698 699 700 701 702 703 704 705 706 707 708 709 710 711 712 713 714 715 716 717 718 719 720 721 722 | sourcetest: srcck1$(BEXE) sqlite3.c ./srcck1 sqlite3.c fuzzershell$(TEXE): $(TOP)/tool/fuzzershell.c sqlite3.c sqlite3.h $(LTLINK) -o $@ $(FUZZERSHELL_OPT) \ $(TOP)/tool/fuzzershell.c sqlite3.c $(TLIBS) fuzzcheck$(TEXE): $(FUZZCHECK_SRC) sqlite3.c sqlite3.h $(FUZZCHECK_DEP) $(LTLINK) -o $@ $(FUZZCHECK_OPT) $(FUZZCHECK_SRC) sqlite3.c $(TLIBS) ossshell$(TEXE): $(TOP)/test/ossfuzz.c $(TOP)/test/ossshell.c sqlite3.c sqlite3.h $(LTLINK) -o $@ $(FUZZCHECK_OPT) $(TOP)/test/ossshell.c \ $(TOP)/test/ossfuzz.c sqlite3.c $(TLIBS) sessionfuzz$(TEXE): $(TOP)/test/sessionfuzz.c sqlite3.c sqlite3.h $(LTLINK) -o $@ $(TOP)/test/sessionfuzz.c $(TLIBS) dbfuzz$(TEXE): $(TOP)/test/dbfuzz.c sqlite3.c sqlite3.h $(LTLINK) -o $@ $(DBFUZZ_OPT) $(TOP)/test/dbfuzz.c sqlite3.c $(TLIBS) DBFUZZ2_OPTS = \ -DSQLITE_THREADSAFE=0 \ -DSQLITE_OMIT_LOAD_EXTENSION \ -DSQLITE_DEBUG \ -DSQLITE_ENABLE_DBSTAT_VTAB \ -DSQLITE_ENABLE_BYTECODE_VTAB \ -DSQLITE_ENABLE_RTREE \ -DSQLITE_ENABLE_FTS4 \ -DSQLITE_ENABLE_FTS5 dbfuzz2$(TEXE): $(TOP)/test/dbfuzz2.c sqlite3.c sqlite3.h $(CC) $(OPT_FEATURE_FLAGS) $(OPTS) -I. -g -O0 \ -DSTANDALONE -o dbfuzz2 \ $(DBFUZZ2_OPTS) $(TOP)/test/dbfuzz2.c sqlite3.c $(TLIBS) mkdir -p dbfuzz2-dir cp $(TOP)/test/dbfuzz2-seed* dbfuzz2-dir dbfuzz2-asan: $(TOP)/test/dbfuzz2.c sqlite3.c sqlite3.h clang-6.0 $(OPT_FEATURE_FLAGS) $(OPTS) -I. -g -O0 \ -fsanitize=fuzzer,undefined,address -o dbfuzz2-asan \ $(DBFUZZ2_OPTS) $(TOP)/test/dbfuzz2.c sqlite3.c $(TLIBS) mkdir -p dbfuzz2-dir cp $(TOP)/test/dbfuzz2-seed* dbfuzz2-dir dbfuzz2-msan: $(TOP)/test/dbfuzz2.c sqlite3.c sqlite3.h clang-6.0 $(OPT_FEATURE_FLAGS) $(OPTS) -I. -g -O0 \ -fsanitize=fuzzer,undefined,memory -o dbfuzz2-msan \ $(DBFUZZ2_OPTS) $(TOP)/test/dbfuzz2.c sqlite3.c $(TLIBS) mkdir -p dbfuzz2-dir cp $(TOP)/test/dbfuzz2-seed* dbfuzz2-dir mptester$(TEXE): sqlite3.lo $(TOP)/mptest/mptest.c $(LTLINK) -o $@ -I. $(TOP)/mptest/mptest.c sqlite3.lo \ $(TLIBS) -rpath "$(libdir)" |
︙ | ︙ | |||
722 723 724 725 726 727 728 | rm tsrc/sqlite.h.in tsrc/parse.y $(TCLSH_CMD) $(TOP)/tool/vdbe-compress.tcl $(OPTS) <tsrc/vdbe.c >vdbe.new mv vdbe.new tsrc/vdbe.c cp fts5.c fts5.h tsrc touch .target_source sqlite3.c: .target_source $(TOP)/tool/mksqlite3c.tcl | | > > > > > > > > > | 747 748 749 750 751 752 753 754 755 756 757 758 759 760 761 762 763 764 765 766 767 768 769 770 771 772 773 | rm tsrc/sqlite.h.in tsrc/parse.y $(TCLSH_CMD) $(TOP)/tool/vdbe-compress.tcl $(OPTS) <tsrc/vdbe.c >vdbe.new mv vdbe.new tsrc/vdbe.c cp fts5.c fts5.h tsrc touch .target_source sqlite3.c: .target_source $(TOP)/tool/mksqlite3c.tcl $(TCLSH_CMD) $(TOP)/tool/mksqlite3c.tcl $(AMALGAMATION_LINE_MACROS) cp tsrc/sqlite3ext.h . cp $(TOP)/ext/session/sqlite3session.h . sqlite3r.h: sqlite3.h $(TCLSH_CMD) $(TOP)/tool/mksqlite3h.tcl $(TOP) --enable-recover >sqlite3r.h sqlite3r.c: sqlite3.c sqlite3r.h cp $(TOP)/ext/recover/sqlite3recover.c tsrc/ cp $(TOP)/ext/recover/sqlite3recover.h tsrc/ cp $(TOP)/ext/recover/dbdata.c tsrc/ $(TCLSH_CMD) $(TOP)/tool/mksqlite3c.tcl --enable-recover $(AMALGAMATION_LINE_MACROS) sqlite3ext.h: .target_source cp tsrc/sqlite3ext.h . tclsqlite3.c: sqlite3.c echo '#ifndef USE_SYSTEM_SQLITE' >tclsqlite3.c cat sqlite3.c >>tclsqlite3.c echo '#endif /* USE_SYSTEM_SQLITE */' >>tclsqlite3.c |
︙ | ︙ | |||
837 838 839 840 841 842 843 844 845 846 847 848 849 850 | hash.lo: $(TOP)/src/hash.c $(HDR) $(LTCOMPILE) $(TEMP_STORE) -c $(TOP)/src/hash.c insert.lo: $(TOP)/src/insert.c $(HDR) $(LTCOMPILE) $(TEMP_STORE) -c $(TOP)/src/insert.c legacy.lo: $(TOP)/src/legacy.c $(HDR) $(LTCOMPILE) $(TEMP_STORE) -c $(TOP)/src/legacy.c loadext.lo: $(TOP)/src/loadext.c $(HDR) $(LTCOMPILE) $(TEMP_STORE) -c $(TOP)/src/loadext.c main.lo: $(TOP)/src/main.c $(HDR) | > > > | 871 872 873 874 875 876 877 878 879 880 881 882 883 884 885 886 887 | hash.lo: $(TOP)/src/hash.c $(HDR) $(LTCOMPILE) $(TEMP_STORE) -c $(TOP)/src/hash.c insert.lo: $(TOP)/src/insert.c $(HDR) $(LTCOMPILE) $(TEMP_STORE) -c $(TOP)/src/insert.c json.lo: $(TOP)/src/json.c $(HDR) $(LTCOMPILE) $(TEMP_STORE) -c $(TOP)/src/json.c legacy.lo: $(TOP)/src/legacy.c $(HDR) $(LTCOMPILE) $(TEMP_STORE) -c $(TOP)/src/legacy.c loadext.lo: $(TOP)/src/loadext.c $(HDR) $(LTCOMPILE) $(TEMP_STORE) -c $(TOP)/src/loadext.c main.lo: $(TOP)/src/main.c $(HDR) |
︙ | ︙ | |||
897 898 899 900 901 902 903 904 905 906 907 908 909 910 | pcache1.lo: $(TOP)/src/pcache1.c $(HDR) $(TOP)/src/pcache.h $(LTCOMPILE) $(TEMP_STORE) -c $(TOP)/src/pcache1.c os.lo: $(TOP)/src/os.c $(HDR) $(LTCOMPILE) $(TEMP_STORE) -c $(TOP)/src/os.c os_unix.lo: $(TOP)/src/os_unix.c $(HDR) $(LTCOMPILE) $(TEMP_STORE) -c $(TOP)/src/os_unix.c os_win.lo: $(TOP)/src/os_win.c $(HDR) $(LTCOMPILE) $(TEMP_STORE) -c $(TOP)/src/os_win.c pragma.lo: $(TOP)/src/pragma.c $(HDR) | > > > | 934 935 936 937 938 939 940 941 942 943 944 945 946 947 948 949 950 | pcache1.lo: $(TOP)/src/pcache1.c $(HDR) $(TOP)/src/pcache.h $(LTCOMPILE) $(TEMP_STORE) -c $(TOP)/src/pcache1.c os.lo: $(TOP)/src/os.c $(HDR) $(LTCOMPILE) $(TEMP_STORE) -c $(TOP)/src/os.c os_kv.lo: $(TOP)/src/os_kv.c $(HDR) $(LTCOMPILE) $(TEMP_STORE) -c $(TOP)/src/os_kv.c os_unix.lo: $(TOP)/src/os_unix.c $(HDR) $(LTCOMPILE) $(TEMP_STORE) -c $(TOP)/src/os_unix.c os_win.lo: $(TOP)/src/os_win.c $(HDR) $(LTCOMPILE) $(TEMP_STORE) -c $(TOP)/src/os_win.c pragma.lo: $(TOP)/src/pragma.c $(HDR) |
︙ | ︙ | |||
978 979 980 981 982 983 984 985 986 987 988 989 990 991 | vdbesort.lo: $(TOP)/src/vdbesort.c $(HDR) $(LTCOMPILE) $(TEMP_STORE) -c $(TOP)/src/vdbesort.c vdbetrace.lo: $(TOP)/src/vdbetrace.c $(HDR) $(LTCOMPILE) $(TEMP_STORE) -c $(TOP)/src/vdbetrace.c vtab.lo: $(TOP)/src/vtab.c $(HDR) $(LTCOMPILE) $(TEMP_STORE) -c $(TOP)/src/vtab.c wal.lo: $(TOP)/src/wal.c $(HDR) $(LTCOMPILE) $(TEMP_STORE) -c $(TOP)/src/wal.c walker.lo: $(TOP)/src/walker.c $(HDR) | > > > | 1018 1019 1020 1021 1022 1023 1024 1025 1026 1027 1028 1029 1030 1031 1032 1033 1034 | vdbesort.lo: $(TOP)/src/vdbesort.c $(HDR) $(LTCOMPILE) $(TEMP_STORE) -c $(TOP)/src/vdbesort.c vdbetrace.lo: $(TOP)/src/vdbetrace.c $(HDR) $(LTCOMPILE) $(TEMP_STORE) -c $(TOP)/src/vdbetrace.c vdbevtab.lo: $(TOP)/src/vdbevtab.c $(HDR) $(LTCOMPILE) $(TEMP_STORE) -c $(TOP)/src/vdbevtab.c vtab.lo: $(TOP)/src/vtab.c $(HDR) $(LTCOMPILE) $(TEMP_STORE) -c $(TOP)/src/vtab.c wal.lo: $(TOP)/src/wal.c $(HDR) $(LTCOMPILE) $(TEMP_STORE) -c $(TOP)/src/wal.c walker.lo: $(TOP)/src/walker.c $(HDR) |
︙ | ︙ | |||
1024 1025 1026 1027 1028 1029 1030 | opcodes.h: parse.h $(TOP)/src/vdbe.c $(TOP)/tool/mkopcodeh.tcl cat parse.h $(TOP)/src/vdbe.c | $(TCLSH_CMD) $(TOP)/tool/mkopcodeh.tcl >opcodes.h # Rules to build parse.c and parse.h - the outputs of lemon. # parse.h: parse.c | | < | < < > > > > > > > | > > > | > > > > > > > > < < < < < < < < < < < < < < < < < < | 1067 1068 1069 1070 1071 1072 1073 1074 1075 1076 1077 1078 1079 1080 1081 1082 1083 1084 1085 1086 1087 1088 1089 1090 1091 1092 1093 1094 1095 1096 1097 1098 1099 1100 1101 1102 1103 1104 1105 1106 1107 1108 1109 1110 1111 1112 1113 1114 1115 1116 1117 1118 1119 1120 1121 1122 1123 1124 1125 1126 1127 1128 1129 1130 1131 1132 1133 | opcodes.h: parse.h $(TOP)/src/vdbe.c $(TOP)/tool/mkopcodeh.tcl cat parse.h $(TOP)/src/vdbe.c | $(TCLSH_CMD) $(TOP)/tool/mkopcodeh.tcl >opcodes.h # Rules to build parse.c and parse.h - the outputs of lemon. # parse.h: parse.c parse.c: $(TOP)/src/parse.y lemon$(BEXE) cp $(TOP)/src/parse.y . ./lemon$(BEXE) $(OPT_FEATURE_FLAGS) $(OPTS) -S parse.y sqlite3.h: $(TOP)/src/sqlite.h.in $(TOP)/manifest mksourceid$(BEXE) $(TOP)/VERSION $(TCLSH_CMD) $(TOP)/tool/mksqlite3h.tcl $(TOP) >sqlite3.h sqlite3rc.h: $(TOP)/src/sqlite3.rc $(TOP)/VERSION echo '#ifndef SQLITE_RESOURCE_VERSION' >$@ echo -n '#define SQLITE_RESOURCE_VERSION ' >>$@ cat $(TOP)/VERSION | $(TCLSH_CMD) $(TOP)/tool/replace.tcl exact . , >>$@ echo '#endif' >>sqlite3rc.h keywordhash.h: $(TOP)/tool/mkkeywordhash.c $(BCC) -o mkkeywordhash$(BEXE) $(OPT_FEATURE_FLAGS) $(OPTS) $(TOP)/tool/mkkeywordhash.c ./mkkeywordhash$(BEXE) >keywordhash.h # Source files that go into making shell.c SHELL_SRC = \ $(TOP)/src/shell.c.in \ $(TOP)/ext/misc/appendvfs.c \ $(TOP)/ext/misc/completion.c \ $(TOP)/ext/misc/decimal.c \ $(TOP)/ext/misc/basexx.c \ $(TOP)/ext/misc/base64.c \ $(TOP)/ext/misc/base85.c \ $(TOP)/ext/misc/fileio.c \ $(TOP)/ext/misc/ieee754.c \ $(TOP)/ext/misc/regexp.c \ $(TOP)/ext/misc/series.c \ $(TOP)/ext/misc/shathree.c \ $(TOP)/ext/misc/sqlar.c \ $(TOP)/ext/misc/uint.c \ $(TOP)/ext/expert/sqlite3expert.c \ $(TOP)/ext/expert/sqlite3expert.h \ $(TOP)/ext/misc/zipfile.c \ $(TOP)/ext/misc/memtrace.c \ $(TOP)/ext/recover/dbdata.c \ $(TOP)/ext/recover/sqlite3recover.c \ $(TOP)/ext/recover/sqlite3recover.h \ $(TOP)/src/test_windirent.c shell.c: $(SHELL_SRC) $(TOP)/tool/mkshellc.tcl $(TCLSH_CMD) $(TOP)/tool/mkshellc.tcl >shell.c # Rules to build the extension objects. # icu.lo: $(TOP)/ext/icu/icu.c $(HDR) $(EXTHDR) $(LTCOMPILE) -DSQLITE_CORE -c $(TOP)/ext/icu/icu.c fts3.lo: $(TOP)/ext/fts3/fts3.c $(HDR) $(EXTHDR) $(LTCOMPILE) -DSQLITE_CORE -c $(TOP)/ext/fts3/fts3.c fts3_aux.lo: $(TOP)/ext/fts3/fts3_aux.c $(HDR) $(EXTHDR) $(LTCOMPILE) -DSQLITE_CORE -c $(TOP)/ext/fts3/fts3_aux.c fts3_expr.lo: $(TOP)/ext/fts3/fts3_expr.c $(HDR) $(EXTHDR) |
︙ | ︙ | |||
1122 1123 1124 1125 1126 1127 1128 | fts3_write.lo: $(TOP)/ext/fts3/fts3_write.c $(HDR) $(EXTHDR) $(LTCOMPILE) -DSQLITE_CORE -c $(TOP)/ext/fts3/fts3_write.c rtree.lo: $(TOP)/ext/rtree/rtree.c $(HDR) $(EXTHDR) $(LTCOMPILE) -DSQLITE_CORE -c $(TOP)/ext/rtree/rtree.c | | | < < < | 1162 1163 1164 1165 1166 1167 1168 1169 1170 1171 1172 1173 1174 1175 1176 1177 1178 1179 1180 1181 | fts3_write.lo: $(TOP)/ext/fts3/fts3_write.c $(HDR) $(EXTHDR) $(LTCOMPILE) -DSQLITE_CORE -c $(TOP)/ext/fts3/fts3_write.c rtree.lo: $(TOP)/ext/rtree/rtree.c $(HDR) $(EXTHDR) $(LTCOMPILE) -DSQLITE_CORE -c $(TOP)/ext/rtree/rtree.c userauth.lo: $(TOP)/ext/userauth/userauth.c $(HDR) $(EXTHDR) $(LTCOMPILE) -DSQLITE_CORE -c $(TOP)/ext/userauth/userauth.c sqlite3session.lo: $(TOP)/ext/session/sqlite3session.c $(HDR) $(EXTHDR) $(LTCOMPILE) -DSQLITE_CORE -c $(TOP)/ext/session/sqlite3session.c stmt.lo: $(TOP)/ext/misc/stmt.c $(LTCOMPILE) -DSQLITE_CORE -c $(TOP)/ext/misc/stmt.c # FTS5 things # FTS5_SRC = \ $(TOP)/ext/fts5/fts5.h \ |
︙ | ︙ | |||
1153 1154 1155 1156 1157 1158 1159 | fts5parse.c fts5parse.h \ $(TOP)/ext/fts5/fts5_storage.c \ $(TOP)/ext/fts5/fts5_tokenize.c \ $(TOP)/ext/fts5/fts5_unicode2.c \ $(TOP)/ext/fts5/fts5_varint.c \ $(TOP)/ext/fts5/fts5_vocab.c \ | | | | 1190 1191 1192 1193 1194 1195 1196 1197 1198 1199 1200 1201 1202 1203 1204 1205 1206 1207 | fts5parse.c fts5parse.h \ $(TOP)/ext/fts5/fts5_storage.c \ $(TOP)/ext/fts5/fts5_tokenize.c \ $(TOP)/ext/fts5/fts5_unicode2.c \ $(TOP)/ext/fts5/fts5_varint.c \ $(TOP)/ext/fts5/fts5_vocab.c \ fts5parse.c: $(TOP)/ext/fts5/fts5parse.y lemon$(BEXE) cp $(TOP)/ext/fts5/fts5parse.y . rm -f fts5parse.h ./lemon$(BEXE) $(OPTS) -S fts5parse.y fts5parse.h: fts5parse.c fts5.c: $(FTS5_SRC) $(TCLSH_CMD) $(TOP)/ext/fts5/tool/mkfts5c.tcl cp $(TOP)/ext/fts5/fts5.h . |
︙ | ︙ | |||
1186 1187 1188 1189 1190 1191 1192 | TESTFIXTURE_FLAGS += -DTCLSH_INIT_PROC=sqlite3TestInit TESTFIXTURE_FLAGS += -DSQLITE_SERVER=1 -DSQLITE_PRIVATE="" -DSQLITE_CORE TESTFIXTURE_FLAGS += -DBUILD_sqlite TESTFIXTURE_FLAGS += -DSQLITE_SERIES_CONSTRAINT_VERIFY=1 TESTFIXTURE_FLAGS += -DSQLITE_DEFAULT_PAGE_SIZE=1024 TESTFIXTURE_FLAGS += -DSQLITE_ENABLE_STMTVTAB TESTFIXTURE_FLAGS += -DSQLITE_ENABLE_DBPAGE_VTAB | | > > > > | < < < < | > > > > > > | > > > | 1223 1224 1225 1226 1227 1228 1229 1230 1231 1232 1233 1234 1235 1236 1237 1238 1239 1240 1241 1242 1243 1244 1245 1246 1247 1248 1249 1250 1251 1252 1253 1254 1255 1256 1257 1258 1259 1260 1261 1262 1263 1264 1265 1266 1267 1268 1269 1270 1271 1272 1273 1274 1275 1276 1277 1278 1279 1280 1281 1282 1283 1284 1285 1286 1287 1288 1289 1290 1291 1292 1293 1294 1295 1296 1297 1298 1299 1300 1301 1302 1303 1304 1305 1306 1307 1308 1309 1310 1311 1312 1313 | TESTFIXTURE_FLAGS += -DTCLSH_INIT_PROC=sqlite3TestInit TESTFIXTURE_FLAGS += -DSQLITE_SERVER=1 -DSQLITE_PRIVATE="" -DSQLITE_CORE TESTFIXTURE_FLAGS += -DBUILD_sqlite TESTFIXTURE_FLAGS += -DSQLITE_SERIES_CONSTRAINT_VERIFY=1 TESTFIXTURE_FLAGS += -DSQLITE_DEFAULT_PAGE_SIZE=1024 TESTFIXTURE_FLAGS += -DSQLITE_ENABLE_STMTVTAB TESTFIXTURE_FLAGS += -DSQLITE_ENABLE_DBPAGE_VTAB TESTFIXTURE_FLAGS += -DSQLITE_ENABLE_BYTECODE_VTAB TESTFIXTURE_FLAGS += -DSQLITE_CKSUMVFS_STATIC TESTFIXTURE_SRC0 = $(TESTSRC2) libsqlite3.la TESTFIXTURE_SRC1 = sqlite3.c TESTFIXTURE_SRC = $(TESTSRC) $(TOP)/src/tclsqlite.c TESTFIXTURE_SRC += $(TESTFIXTURE_SRC$(USE_AMALGAMATION)) testfixture$(TEXE): $(TESTFIXTURE_SRC) $(LTLINK) -DSQLITE_NO_SYNC=1 $(TEMP_STORE) $(TESTFIXTURE_FLAGS) \ -o $@ $(TESTFIXTURE_SRC) $(LIBTCL) $(TLIBS) coretestprogs: $(TESTPROGS) testprogs: coretestprogs srcck1$(BEXE) fuzzcheck$(TEXE) sessionfuzz$(TEXE) # A very detailed test running most or all test cases fulltest: alltest fuzztest # Run most or all tcl test cases alltest: $(TESTPROGS) ./testfixture$(TEXE) $(TOP)/test/all.test $(TESTOPTS) # Really really long testing soaktest: $(TESTPROGS) ./testfixture$(TEXE) $(TOP)/test/all.test -soak=1 $(TESTOPTS) # Do extra testing but not everything. fulltestonly: $(TESTPROGS) fuzztest ./testfixture$(TEXE) $(TOP)/test/full.test # Fuzz testing fuzztest: fuzzcheck$(TEXE) $(FUZZDATA) sessionfuzz$(TEXE) $(TOP)/test/sessionfuzz-data1.db ./fuzzcheck$(TEXE) $(FUZZDATA) ./sessionfuzz$(TEXE) run $(TOP)/test/sessionfuzz-data1.db valgrindfuzz: fuzzcheck$(TEXT) $(FUZZDATA) sessionfuzz$(TEXE) $(TOP)/test/sessionfuzz-data1.db valgrind ./fuzzcheck$(TEXE) --cell-size-check --limit-mem 10M $(FUZZDATA) valgrind ./sessionfuzz$(TEXE) run $(TOP)/test/sessionfuzz-data1.db # The veryquick.test TCL tests. # tcltest: ./testfixture$(TEXE) ./testfixture$(TEXE) $(TOP)/test/veryquick.test $(TESTOPTS) # Runs all the same tests cases as the "tcltest" target but uses # the testrunner.tcl script to run them in multiple cores # concurrently. testrunner: testfixture$(TEXE) ./testfixture$(TEXE) $(TOP)/test/testrunner.tcl # Minimal testing that runs in less than 3 minutes # quicktest: ./testfixture$(TEXE) ./testfixture$(TEXE) $(TOP)/test/extraquick.test $(TESTOPTS) # This is the common case. Run many tests that do not take too long, # including fuzzcheck, sqlite3_analyzer, and sqldiff tests. # test: fuzztest sourcetest $(TESTPROGS) tcltest # Run a test using valgrind. This can take a really long time # because valgrind is so much slower than a native machine. # valgrindtest: $(TESTPROGS) valgrindfuzz OMIT_MISUSE=1 valgrind -v ./testfixture$(TEXE) $(TOP)/test/permutations.test valgrind $(TESTOPTS) # A very fast test that checks basic sanity. The name comes from # the 60s-era electronics testing: "Turn it on and see if smoke # comes out." # smoketest: $(TESTPROGS) fuzzcheck$(TEXE) ./testfixture$(TEXE) $(TOP)/test/main.test $(TESTOPTS) shelltest: $(TESTPROGS) ./testfixture$(TEXT) $(TOP)/test/permutations.test shell sqlite3_analyzer.c: sqlite3.c $(TOP)/src/tclsqlite.c $(TOP)/tool/spaceanal.tcl $(TOP)/tool/mkccode.tcl $(TOP)/tool/sqlite3_analyzer.c.in $(TCLSH_CMD) $(TOP)/tool/mkccode.tcl $(TOP)/tool/sqlite3_analyzer.c.in >sqlite3_analyzer.c sqlite3_analyzer$(TEXE): sqlite3_analyzer.c $(LTLINK) sqlite3_analyzer.c -o $@ $(LIBTCL) $(TLIBS) sqltclsh.c: sqlite3.c $(TOP)/src/tclsqlite.c $(TOP)/tool/sqltclsh.tcl $(TOP)/ext/misc/appendvfs.c $(TOP)/tool/mkccode.tcl $(TOP)/tool/sqltclsh.c.in |
︙ | ︙ | |||
1308 1309 1310 1311 1312 1313 1314 | showwal$(TEXE): $(TOP)/tool/showwal.c sqlite3.lo $(LTLINK) -o $@ $(TOP)/tool/showwal.c sqlite3.lo $(TLIBS) showshm$(TEXE): $(TOP)/tool/showshm.c $(LTLINK) -o $@ $(TOP)/tool/showshm.c index_usage$(TEXE): $(TOP)/tool/index_usage.c sqlite3.lo | | | > > > | | > | | > > > | > | 1354 1355 1356 1357 1358 1359 1360 1361 1362 1363 1364 1365 1366 1367 1368 1369 1370 1371 1372 1373 1374 1375 1376 1377 1378 1379 1380 1381 1382 1383 1384 1385 1386 1387 1388 1389 1390 1391 1392 1393 1394 1395 1396 1397 1398 1399 1400 1401 1402 1403 1404 1405 1406 1407 1408 1409 1410 1411 1412 1413 1414 1415 1416 1417 1418 1419 1420 1421 1422 1423 1424 1425 1426 1427 1428 1429 1430 1431 1432 1433 1434 1435 1436 1437 1438 1439 1440 1441 1442 1443 1444 1445 1446 1447 1448 1449 1450 1451 1452 1453 1454 1455 1456 1457 1458 1459 1460 1461 1462 1463 1464 1465 1466 1467 1468 1469 1470 1471 1472 1473 1474 1475 1476 1477 | showwal$(TEXE): $(TOP)/tool/showwal.c sqlite3.lo $(LTLINK) -o $@ $(TOP)/tool/showwal.c sqlite3.lo $(TLIBS) showshm$(TEXE): $(TOP)/tool/showshm.c $(LTLINK) -o $@ $(TOP)/tool/showshm.c index_usage$(TEXE): $(TOP)/tool/index_usage.c sqlite3.lo $(LTLINK) $(SHELL_OPT) -o $@ $(TOP)/tool/index_usage.c sqlite3.lo $(TLIBS) changeset$(TEXE): $(TOP)/ext/session/changeset.c sqlite3.lo $(LTLINK) -o $@ $(TOP)/ext/session/changeset.c sqlite3.lo $(TLIBS) changesetfuzz$(TEXE): $(TOP)/ext/session/changesetfuzz.c sqlite3.lo $(LTLINK) -o $@ $(TOP)/ext/session/changesetfuzz.c sqlite3.lo $(TLIBS) rollback-test$(TEXE): $(TOP)/tool/rollback-test.c sqlite3.lo $(LTLINK) -o $@ $(TOP)/tool/rollback-test.c sqlite3.lo $(TLIBS) atrc$(TEXX): $(TOP)/test/atrc.c sqlite3.lo $(LTLINK) -o $@ $(TOP)/test/atrc.c sqlite3.lo $(TLIBS) LogEst$(TEXE): $(TOP)/tool/logest.c sqlite3.h $(LTLINK) -I. -o $@ $(TOP)/tool/logest.c wordcount$(TEXE): $(TOP)/test/wordcount.c sqlite3.lo $(LTLINK) -o $@ $(TOP)/test/wordcount.c sqlite3.lo $(TLIBS) speedtest1$(TEXE): $(TOP)/test/speedtest1.c sqlite3.c Makefile $(LTLINK) $(ST_OPT) -o $@ $(TOP)/test/speedtest1.c sqlite3.c $(TLIBS) startup$(TEXE): $(TOP)/test/startup.c sqlite3.c $(CC) -Os -g -DSQLITE_THREADSAFE=0 -o $@ $(TOP)/test/startup.c sqlite3.c $(TLIBS) KV_OPT += -DSQLITE_DIRECT_OVERFLOW_READ kvtest$(TEXE): $(TOP)/test/kvtest.c sqlite3.c $(LTLINK) $(KV_OPT) -o $@ $(TOP)/test/kvtest.c sqlite3.c $(TLIBS) rbu$(EXE): $(TOP)/ext/rbu/rbu.c $(TOP)/ext/rbu/sqlite3rbu.c sqlite3.lo $(LTLINK) -I. -o $@ $(TOP)/ext/rbu/rbu.c sqlite3.lo $(TLIBS) loadfts$(EXE): $(TOP)/tool/loadfts.c libsqlite3.la $(LTLINK) $(TOP)/tool/loadfts.c libsqlite3.la -o $@ $(TLIBS) # This target will fail if the SQLite amalgamation contains any exported # symbols that do not begin with "sqlite3_". It is run as part of the # releasetest.tcl script. # VALIDIDS=' sqlite3(changeset|changegroup|session)?_' checksymbols: sqlite3.o nm -g --defined-only sqlite3.o nm -g --defined-only sqlite3.o | egrep -v $(VALIDIDS); test $$? -ne 0 echo '0 errors out of 1 tests' # Build the amalgamation-autoconf package. The amalamgation-tarball target builds # a tarball named for the version number. Ex: sqlite-autoconf-3110000.tar.gz. # The snapshot-tarball target builds a tarball named by the SHA1 hash # amalgamation-tarball: sqlite3.c sqlite3rc.h TOP=$(TOP) sh $(TOP)/tool/mkautoconfamal.sh --normal snapshot-tarball: sqlite3.c sqlite3rc.h TOP=$(TOP) sh $(TOP)/tool/mkautoconfamal.sh --snapshot # The next two rules are used to support the "threadtest" target. Building # threadtest runs a few thread-safety tests that are implemented in C. This # target is invoked by the releasetest.tcl script. # THREADTEST3_SRC = $(TOP)/test/threadtest3.c \ $(TOP)/test/tt3_checkpoint.c \ $(TOP)/test/tt3_index.c \ $(TOP)/test/tt3_vacuum.c \ $(TOP)/test/tt3_stress.c \ $(TOP)/test/tt3_lookaside1.c threadtest3$(TEXE): sqlite3.lo $(THREADTEST3_SRC) $(LTLINK) $(TOP)/test/threadtest3.c $(TOP)/src/test_multiplex.c sqlite3.lo -o $@ $(TLIBS) threadtest: threadtest3$(TEXE) ./threadtest3$(TEXE) threadtest5: sqlite3.c $(TOP)/test/threadtest5.c $(LTLINK) $(TOP)/test/threadtest5.c sqlite3.c -o $@ $(TLIBS) releasetest: $(TCLSH_CMD) $(TOP)/test/releasetest.tcl # Standard install and cleanup targets # lib_install: libsqlite3.la $(INSTALL) -d $(DESTDIR)$(libdir) $(LTINSTALL) libsqlite3.la $(DESTDIR)$(libdir) install: sqlite3$(TEXE) lib_install sqlite3.h sqlite3.pc ${HAVE_TCL:1=tcl_install} $(INSTALL) -d $(DESTDIR)$(bindir) $(LTINSTALL) sqlite3$(TEXE) $(DESTDIR)$(bindir) $(INSTALL) -d $(DESTDIR)$(includedir) $(INSTALL) -m 0644 sqlite3.h $(DESTDIR)$(includedir) $(INSTALL) -m 0644 $(TOP)/src/sqlite3ext.h $(DESTDIR)$(includedir) $(INSTALL) -d $(DESTDIR)$(pkgconfigdir) $(INSTALL) -m 0644 sqlite3.pc $(DESTDIR)$(pkgconfigdir) pkgIndex.tcl: echo 'package ifneeded sqlite3 $(RELEASE) [list load [file join $$dir libtclsqlite3[info sharedlibextension]] sqlite3]' > $@ tcl_install: lib_install libtclsqlite3.la pkgIndex.tcl $(INSTALL) -d $(DESTDIR)$(TCLLIBDIR) $(LTINSTALL) libtclsqlite3.la $(DESTDIR)$(TCLLIBDIR) rm -f $(DESTDIR)$(TCLLIBDIR)/libtclsqlite3.la $(DESTDIR)$(TCLLIBDIR)/libtclsqlite3.a $(INSTALL) -m 0644 pkgIndex.tcl $(DESTDIR)$(TCLLIBDIR) clean: rm -f *.lo *.la *.o sqlite3$(TEXE) libsqlite3.la rm -f sqlite3.h opcodes.* rm -rf .libs .deps rm -f lemon$(BEXE) lempar.c parse.* sqlite*.tar.gz rm -f mkkeywordhash$(BEXE) keywordhash.h rm -f mksourceid$(BEXE) rm -f *.da *.bb *.bbg gmon.out rm -rf tsrc .target_source rm -f tclsqlite3$(TEXE) rm -f testfixture$(TEXE) test.db rm -f LogEst$(TEXE) fts3view$(TEXE) rollback-test$(TEXE) showdb$(TEXE) rm -f showjournal$(TEXE) showstat4$(TEXE) showwal$(TEXE) speedtest1$(TEXE) rm -f wordcount$(TEXE) changeset$(TEXE) |
︙ | ︙ | |||
1431 1432 1433 1434 1435 1436 1437 1438 1439 | rm -f rbu rbu.exe rm -f srcck1 srcck1.exe rm -f fuzzershell fuzzershell.exe rm -f fuzzcheck fuzzcheck.exe rm -f sqldiff sqldiff.exe rm -f dbhash dbhash.exe rm -f fts5.* fts5parse.* distclean: clean | > | > > > > > > > | 1485 1486 1487 1488 1489 1490 1491 1492 1493 1494 1495 1496 1497 1498 1499 1500 1501 1502 1503 1504 1505 1506 1507 1508 1509 1510 1511 1512 1513 1514 1515 1516 1517 1518 1519 1520 | rm -f rbu rbu.exe rm -f srcck1 srcck1.exe rm -f fuzzershell fuzzershell.exe rm -f fuzzcheck fuzzcheck.exe rm -f sqldiff sqldiff.exe rm -f dbhash dbhash.exe rm -f fts5.* fts5parse.* rm -f threadtest5 distclean: clean rm -f sqlite_cfg.h config.log config.status libtool Makefile sqlite3.pc \ $(TESTPROGS) # # Windows section # dll: sqlite3.dll REAL_LIBOBJ = $(LIBOBJ:%.lo=.libs/%.o) $(REAL_LIBOBJ): $(LIBOBJ) sqlite3.def: $(REAL_LIBOBJ) echo 'EXPORTS' >sqlite3.def nm $(REAL_LIBOBJ) | grep ' T ' | grep ' _sqlite3_' \ | sed 's/^.* _//' >>sqlite3.def sqlite3.dll: $(REAL_LIBOBJ) sqlite3.def $(TCC) -shared -o $@ sqlite3.def \ -Wl,"--strip-all" $(REAL_LIBOBJ) # # Fiddle app # fiddle: sqlite3.c shell.c make -C ext/wasm fiddle emcc_opt=-Os |
Changes to Makefile.linux-gcc.
︙ | ︙ | |||
15 16 17 18 19 20 21 | # that contains this "Makefile.in" and the "configure.in" script. # TOP = ../sqlite #### C Compiler and options for use in building executables that # will run on the platform that is doing the build. # | | | | | | < < | | < < | < < | < < | 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 | # that contains this "Makefile.in" and the "configure.in" script. # TOP = ../sqlite #### C Compiler and options for use in building executables that # will run on the platform that is doing the build. # BCC = gcc -g -O0 #BCC = /opt/ancic/bin/c89 -0 #### If the target operating system supports the "usleep()" system # call, then define the HAVE_USLEEP macro for all C modules. # #USLEEP = USLEEP = -DHAVE_USLEEP=1 #### If you want the SQLite library to be safe for use within a # multi-threaded program, then define the following macro # appropriately: # #THREADSAFE = -DTHREADSAFE=1 THREADSAFE = -DTHREADSAFE=0 #### Specify any extra linker options needed to make the library # thread safe # THREADLIB = -lpthread -lm -ldl #THREADLIB = #### Specify any extra libraries needed to access required functions. # #TLIBS = -lrt # fdatasync on Solaris 8 TLIBS = #### Leave SQLITE_DEBUG undefined for maximum speed. Use SQLITE_DEBUG=1 # to check for memory leaks. Use SQLITE_DEBUG=2 to print a log of all # malloc()s and free()s in order to track down memory leaks. # # SQLite uses some expensive assert() statements in the inner loop. # You can make the library go almost twice as fast if you compile # with -DNDEBUG=1 # OPTS += -DSQLITE_DEBUG=1 OPTS += -DSQLITE_ENABLE_WHERETRACE OPTS += -DSQLITE_ENABLE_SELECTTRACE #### The suffix to add to executable files. ".exe" for windows. # Nothing for unix. # #EXE = .exe EXE = #### C Compile and options for use in building executables that # will run on the target platform. This is usually the same # as BCC, unless you are cross-compiling. # TCC = gcc -O0 #TCC = gcc -g -O0 -Wall #TCC = gcc -g -O0 -Wall -fprofile-arcs -ftest-coverage #TCC = /opt/mingw/bin/i386-mingw32-gcc -O6 #TCC = /opt/ansic/bin/c89 -O +z -Wl,-a,archive #### Tools used to build a static library. # AR = ar cr #AR = /opt/mingw/bin/i386-mingw32-ar cr RANLIB = ranlib #RANLIB = /opt/mingw/bin/i386-mingw32-ranlib MKSHLIB = gcc -shared SO = so SHPREFIX = lib # SO = dll # SHPREFIX = #### Extra compiler options needed for programs that use the TCL library. # TCL_FLAGS = -I/home/drh/tcl/include/tcl8.6 #### Linker options needed to link against the TCL library. # #LIBTCL = -ltcl -lm -ldl LIBTCL = /home/drh/tcl/lib/libtcl8.6.a -lm -lpthread -ldl -lz #### Additional objects for SQLite library when TCL support is enabled. #TCLOBJ = TCLOBJ = tclsqlite.o #### Compiler options needed for programs that use the readline() library. # |
︙ | ︙ |
Changes to Makefile.msc.
︙ | ︙ | |||
69 70 71 72 73 74 75 | # If necessary, create a list of harmless compiler warnings to disable when # compiling the various tools. For the SQLite source code itself, warnings, # if any, will be disabled from within it. # !IFNDEF NO_WARN !IF $(USE_FULLWARN)!=0 NO_WARN = -wd4054 -wd4055 -wd4100 -wd4127 -wd4130 -wd4152 -wd4189 -wd4206 | | | 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 | # If necessary, create a list of harmless compiler warnings to disable when # compiling the various tools. For the SQLite source code itself, warnings, # if any, will be disabled from within it. # !IFNDEF NO_WARN !IF $(USE_FULLWARN)!=0 NO_WARN = -wd4054 -wd4055 -wd4100 -wd4127 -wd4130 -wd4152 -wd4189 -wd4206 NO_WARN = $(NO_WARN) -wd4210 -wd4232 -wd4244 -wd4305 -wd4306 -wd4702 -wd4706 !ENDIF !ENDIF # Set this non-0 to use the library paths and other options necessary for # Windows Phone 8.1. # !IFNDEF USE_WP81_OPTS |
︙ | ︙ | |||
230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 | # 5 == SQLITE_DEBUG_OS_TRACE: Enables output from the OSTRACE() macros. # 6 == SQLITE_ENABLE_IOTRACE: Enables output from the IOTRACE() macros. # !IFNDEF DEBUG DEBUG = 0 !ENDIF # Enable use of available compiler optimizations? Normally, this should be # non-zero. Setting this to zero, thus disabling all compiler optimizations, # can be useful for testing. # !IFNDEF OPTIMIZATIONS OPTIMIZATIONS = 2 !ENDIF # Set this to non-0 to enable support for the session extension. # !IFNDEF SESSION SESSION = 0 !ENDIF # Set the source code file to be used by executables and libraries when # they need the amalgamation. # !IFNDEF SQLITE3C !IF $(SPLIT_AMALGAMATION)!=0 SQLITE3C = sqlite3-all.c | > > > > > > > > > > > > > > > > | 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 | # 5 == SQLITE_DEBUG_OS_TRACE: Enables output from the OSTRACE() macros. # 6 == SQLITE_ENABLE_IOTRACE: Enables output from the IOTRACE() macros. # !IFNDEF DEBUG DEBUG = 0 !ENDIF # <<mark>> # By default, use --linemacros=1 argument to the mksqlite3c.tcl tool, which # is used to build the amalgamation. This can be turned off to ease debug # of the amalgamation away from the source tree. # !IFNDEF NO_LINEMACROS NO_LINEMACROS = 0 !ENDIF # <</mark>> # Enable use of available compiler optimizations? Normally, this should be # non-zero. Setting this to zero, thus disabling all compiler optimizations, # can be useful for testing. # !IFNDEF OPTIMIZATIONS OPTIMIZATIONS = 2 !ENDIF # Set this to non-0 to enable support for the session extension. # !IFNDEF SESSION SESSION = 0 !ENDIF # Set this to non-0 to enable support for the rbu extension. # !IFNDEF RBU RBU = 0 !ENDIF # Set the source code file to be used by executables and libraries when # they need the amalgamation. # !IFNDEF SQLITE3C !IF $(SPLIT_AMALGAMATION)!=0 SQLITE3C = sqlite3-all.c |
︙ | ︙ | |||
315 316 317 318 319 320 321 322 323 324 325 326 327 328 | !IFNDEF SQLITETCLH SQLITETCLH = sqlite_tcl.h !ENDIF !IFNDEF SQLITETCLDECLSH SQLITETCLDECLSH = sqlite_tclDecls.h !ENDIF # These are the additional targets that the targets that integrate with the # Tcl library should depend on when compiling, etc. # !IFNDEF SQLITE_TCL_DEP !IF $(USE_STDCALL)!=0 || $(FOR_WIN10)!=0 SQLITE_TCL_DEP = $(SQLITETCLDECLSH) $(SQLITETCLH) | > > > > > > > | 331 332 333 334 335 336 337 338 339 340 341 342 343 344 345 346 347 348 349 350 351 | !IFNDEF SQLITETCLH SQLITETCLH = sqlite_tcl.h !ENDIF !IFNDEF SQLITETCLDECLSH SQLITETCLDECLSH = sqlite_tclDecls.h !ENDIF # This is the name to use for the dynamic link library (DLL) containing the # Tcl bindings for SQLite. # !IFNDEF SQLITE3TCLDLL SQLITE3TCLDLL = tclsqlite3.dll !ENDIF # These are the additional targets that the targets that integrate with the # Tcl library should depend on when compiling, etc. # !IFNDEF SQLITE_TCL_DEP !IF $(USE_STDCALL)!=0 || $(FOR_WIN10)!=0 SQLITE_TCL_DEP = $(SQLITETCLDECLSH) $(SQLITETCLH) |
︙ | ︙ | |||
336 337 338 339 340 341 342 | # the Windows platform. # !IFNDEF OPT_FEATURE_FLAGS !IF $(MINIMAL_AMALGAMATION)==0 OPT_FEATURE_FLAGS = $(OPT_FEATURE_FLAGS) -DSQLITE_ENABLE_FTS3=1 OPT_FEATURE_FLAGS = $(OPT_FEATURE_FLAGS) -DSQLITE_ENABLE_RTREE=1 OPT_FEATURE_FLAGS = $(OPT_FEATURE_FLAGS) -DSQLITE_ENABLE_GEOPOLY=1 | < | < > > > > > > > > > > | 359 360 361 362 363 364 365 366 367 368 369 370 371 372 373 374 375 376 377 378 379 380 381 382 383 384 385 386 387 388 389 390 391 392 393 394 395 396 397 | # the Windows platform. # !IFNDEF OPT_FEATURE_FLAGS !IF $(MINIMAL_AMALGAMATION)==0 OPT_FEATURE_FLAGS = $(OPT_FEATURE_FLAGS) -DSQLITE_ENABLE_FTS3=1 OPT_FEATURE_FLAGS = $(OPT_FEATURE_FLAGS) -DSQLITE_ENABLE_RTREE=1 OPT_FEATURE_FLAGS = $(OPT_FEATURE_FLAGS) -DSQLITE_ENABLE_GEOPOLY=1 OPT_FEATURE_FLAGS = $(OPT_FEATURE_FLAGS) -DSQLITE_ENABLE_STMTVTAB=1 OPT_FEATURE_FLAGS = $(OPT_FEATURE_FLAGS) -DSQLITE_ENABLE_DBPAGE_VTAB=1 OPT_FEATURE_FLAGS = $(OPT_FEATURE_FLAGS) -DSQLITE_ENABLE_DBSTAT_VTAB=1 OPT_FEATURE_FLAGS = $(OPT_FEATURE_FLAGS) -DSQLITE_ENABLE_BYTECODE_VTAB=1 !ENDIF OPT_FEATURE_FLAGS = $(OPT_FEATURE_FLAGS) -DSQLITE_ENABLE_COLUMN_METADATA=1 !ENDIF # Should the session extension be enabled? If so, add compilation options # to enable it. # !IF $(SESSION)!=0 OPT_FEATURE_FLAGS = $(OPT_FEATURE_FLAGS) -DSQLITE_ENABLE_SESSION=1 OPT_FEATURE_FLAGS = $(OPT_FEATURE_FLAGS) -DSQLITE_ENABLE_PREUPDATE_HOOK=1 !ENDIF # Always enable math functions on Windows OPT_FEATURE_FLAGS = $(OPT_FEATURE_FLAGS) -DSQLITE_ENABLE_MATH_FUNCTIONS # Should the rbu extension be enabled? If so, add compilation options # to enable it. # !IF $(RBU)!=0 OPT_FEATURE_FLAGS = $(OPT_FEATURE_FLAGS) -DSQLITE_ENABLE_RBU=1 !ENDIF # These are the "extended" SQLite compilation options used when compiling for # the Windows 10 platform. # !IFNDEF EXT_FEATURE_FLAGS !IF $(FOR_WIN10)!=0 EXT_FEATURE_FLAGS = $(EXT_FEATURE_FLAGS) -DSQLITE_ENABLE_FTS4=1 |
︙ | ︙ | |||
491 492 493 494 495 496 497 | UCRTLIBPATH = $(UCRTLIBPATH:\\=\) # C compiler and options for use in building executables that # will run on the platform that is doing the build. # !IF $(USE_FULLWARN)!=0 | | | | 522 523 524 525 526 527 528 529 530 531 532 533 534 535 536 537 538 | UCRTLIBPATH = $(UCRTLIBPATH:\\=\) # C compiler and options for use in building executables that # will run on the platform that is doing the build. # !IF $(USE_FULLWARN)!=0 BCC = $(NCC) -nologo -W4 -Fd$*.pdb $(CCOPTS) $(BCCOPTS) !ELSE BCC = $(NCC) -nologo -W3 -Fd$*.pdb $(CCOPTS) $(BCCOPTS) !ENDIF # Check if assembly code listings should be generated for the source # code files to be compiled. # !IF $(USE_LISTINGS)!=0 BCC = $(BCC) -FAcs |
︙ | ︙ | |||
544 545 546 547 548 549 550 | # also be noted here that building any target with these "stdcall" options # will most likely fail if the Tcl library is also required. This is due # to how the Tcl library functions are declared and exported (i.e. without # an explicit calling convention, which results in "cdecl"). # !IF $(USE_STDCALL)!=0 || $(FOR_WIN10)!=0 !IF "$(PLATFORM)"=="x86" | | | | | | | | 575 576 577 578 579 580 581 582 583 584 585 586 587 588 589 590 591 592 593 594 595 596 597 598 599 | # also be noted here that building any target with these "stdcall" options # will most likely fail if the Tcl library is also required. This is due # to how the Tcl library functions are declared and exported (i.e. without # an explicit calling convention, which results in "cdecl"). # !IF $(USE_STDCALL)!=0 || $(FOR_WIN10)!=0 !IF "$(PLATFORM)"=="x86" CORE_CCONV_OPTS = -Gz -guard:cf -DSQLITE_CDECL=__cdecl -DSQLITE_APICALL=__stdcall -DSQLITE_CALLBACK=__stdcall -DSQLITE_SYSAPI=__stdcall SHELL_CCONV_OPTS = -Gz -guard:cf -DSQLITE_CDECL=__cdecl -DSQLITE_APICALL=__stdcall -DSQLITE_CALLBACK=__stdcall -DSQLITE_SYSAPI=__stdcall # <<mark>> TEST_CCONV_OPTS = -Gz -guard:cf -DSQLITE_CDECL=__cdecl -DSQLITE_APICALL=__stdcall -DSQLITE_CALLBACK=__stdcall -DSQLITE_SYSAPI=__stdcall -DINCLUDE_SQLITE_TCL_H=1 -DSQLITE_TCLAPI=__cdecl # <</mark>> !ELSE !IFNDEF PLATFORM CORE_CCONV_OPTS = -Gz -guard:cf -DSQLITE_CDECL=__cdecl -DSQLITE_APICALL=__stdcall -DSQLITE_CALLBACK=__stdcall -DSQLITE_SYSAPI=__stdcall SHELL_CCONV_OPTS = -Gz -guard:cf -DSQLITE_CDECL=__cdecl -DSQLITE_APICALL=__stdcall -DSQLITE_CALLBACK=__stdcall -DSQLITE_SYSAPI=__stdcall # <<mark>> TEST_CCONV_OPTS = -Gz -guard:cf -DSQLITE_CDECL=__cdecl -DSQLITE_APICALL=__stdcall -DSQLITE_CALLBACK=__stdcall -DSQLITE_SYSAPI=__stdcall -DINCLUDE_SQLITE_TCL_H=1 -DSQLITE_TCLAPI=__cdecl # <</mark>> !ELSE CORE_CCONV_OPTS = SHELL_CCONV_OPTS = # <<mark>> TEST_CCONV_OPTS = # <</mark>> |
︙ | ︙ | |||
752 753 754 755 756 757 758 | MKSQLITE3C_TOOL = $(TOP)\tool\mksqlite3c-noext.tcl !ELSE MKSQLITE3C_TOOL = $(TOP)\tool\mksqlite3c.tcl !ENDIF !ENDIF !IFNDEF MKSQLITE3C_ARGS | | | | | 783 784 785 786 787 788 789 790 791 792 793 794 795 796 797 798 799 800 | MKSQLITE3C_TOOL = $(TOP)\tool\mksqlite3c-noext.tcl !ELSE MKSQLITE3C_TOOL = $(TOP)\tool\mksqlite3c.tcl !ENDIF !ENDIF !IFNDEF MKSQLITE3C_ARGS !IF $(DEBUG)>1 && $(NO_LINEMACROS)==0 MKSQLITE3C_ARGS = --linemacros=1 !ELSE MKSQLITE3C_ARGS = --linemacros=0 !ENDIF !IF $(USE_STDCALL)!=0 || $(FOR_WIN10)!=0 MKSQLITE3C_ARGS = $(MKSQLITE3C_ARGS) --useapicall !ENDIF !ENDIF # The mksqlite3h.tcl script accepts some options on the command line. |
︙ | ︙ | |||
849 850 851 852 853 854 855 856 857 858 859 860 861 862 863 864 865 866 867 868 | # <<mark>> # The locations of the Tcl header and library files. Also, the library that # non-stubs enabled programs using Tcl must link against. These variables # (TCLINCDIR, TCLLIBDIR, and LIBTCL) may be overridden via the environment # prior to running nmake in order to match the actual installed location and # version on this machine. # !IFNDEF TCLDIR TCLDIR = $(TOP)\compat\tcl !ENDIF !IFNDEF TCLINCDIR TCLINCDIR = $(TCLDIR)\include !ENDIF !IFNDEF TCLLIBDIR TCLLIBDIR = $(TCLDIR)\lib !ENDIF !IFNDEF LIBTCL | > > > > > > > > | | | 880 881 882 883 884 885 886 887 888 889 890 891 892 893 894 895 896 897 898 899 900 901 902 903 904 905 906 907 908 909 910 911 912 913 914 915 916 917 918 919 | # <<mark>> # The locations of the Tcl header and library files. Also, the library that # non-stubs enabled programs using Tcl must link against. These variables # (TCLINCDIR, TCLLIBDIR, and LIBTCL) may be overridden via the environment # prior to running nmake in order to match the actual installed location and # version on this machine. # !IFNDEF TCLVERSION TCLVERSION = 86 !ENDIF !IFNDEF TCLSUFFIX TCLSUFFIX = !ENDIF !IFNDEF TCLDIR TCLDIR = $(TOP)\compat\tcl !ENDIF !IFNDEF TCLINCDIR TCLINCDIR = $(TCLDIR)\include !ENDIF !IFNDEF TCLLIBDIR TCLLIBDIR = $(TCLDIR)\lib !ENDIF !IFNDEF LIBTCL LIBTCL = tcl$(TCLVERSION)$(TCLSUFFIX).lib !ENDIF !IFNDEF LIBTCLSTUB LIBTCLSTUB = tclstub$(TCLVERSION)$(TCLSUFFIX).lib !ENDIF !IFNDEF LIBTCLPATH LIBTCLPATH = $(TCLDIR)\bin !ENDIF # The locations of the zlib header and library files. These variables |
︙ | ︙ | |||
1059 1060 1061 1062 1063 1064 1065 | RCC = $(RCC) -I$(ICUINCDIR) !ENDIF # <</mark>> # Command line prefixes for compiling code, compiling resources, # linking, etc. # | | > > > > > | 1098 1099 1100 1101 1102 1103 1104 1105 1106 1107 1108 1109 1110 1111 1112 1113 1114 1115 1116 1117 1118 1119 1120 1121 1122 1123 1124 1125 1126 1127 1128 1129 1130 1131 1132 1133 1134 | RCC = $(RCC) -I$(ICUINCDIR) !ENDIF # <</mark>> # Command line prefixes for compiling code, compiling resources, # linking, etc. # LTCOMPILE = $(TCC) -Fo$@ -Fd$*.pdb LTRCOMPILE = $(RCC) -r LTLIB = lib.exe LTLINK = $(TCC) -Fe$@ # If requested, link to the RPCRT4 library. # !IF $(USE_RPCRT4_LIB)!=0 LTLIBS = $(LTLIBS) rpcrt4.lib !ENDIF # If a platform was set, force the linker to target that. # Note that the vcvars*.bat family of batch files typically # set this for you. Otherwise, the linker will attempt # to deduce the binary type based on the object files. !IFDEF PLATFORM LTLINKOPTS = /NOLOGO /MACHINE:$(PLATFORM) LTLIBOPTS = /NOLOGO /MACHINE:$(PLATFORM) !ELSEIF "$(VISUALSTUDIOVERSION)"=="12.0" || \ "$(VISUALSTUDIOVERSION)"=="14.0" || \ "$(VISUALSTUDIOVERSION)"=="15.0" LTLINKOPTS = /NOLOGO /MACHINE:x86 LTLIBOPTS = /NOLOGO /MACHINE:x86 !ELSE LTLINKOPTS = /NOLOGO LTLIBOPTS = /NOLOGO !ENDIF # When compiling for use in the WinRT environment, the following # linker option must be used to mark the executable as runnable |
︙ | ︙ | |||
1199 1200 1201 1202 1203 1204 1205 | date.lo dbpage.lo dbstat.lo delete.lo \ expr.lo fault.lo fkey.lo \ fts3.lo fts3_aux.lo fts3_expr.lo fts3_hash.lo fts3_icu.lo \ fts3_porter.lo fts3_snippet.lo fts3_tokenizer.lo fts3_tokenizer1.lo \ fts3_tokenize_vtab.lo fts3_unicode.lo fts3_unicode2.lo fts3_write.lo \ fts5.lo \ func.lo global.lo hash.lo \ | | | | > | 1243 1244 1245 1246 1247 1248 1249 1250 1251 1252 1253 1254 1255 1256 1257 1258 1259 1260 1261 1262 1263 1264 1265 1266 1267 1268 1269 | date.lo dbpage.lo dbstat.lo delete.lo \ expr.lo fault.lo fkey.lo \ fts3.lo fts3_aux.lo fts3_expr.lo fts3_hash.lo fts3_icu.lo \ fts3_porter.lo fts3_snippet.lo fts3_tokenizer.lo fts3_tokenizer1.lo \ fts3_tokenize_vtab.lo fts3_unicode.lo fts3_unicode2.lo fts3_write.lo \ fts5.lo \ func.lo global.lo hash.lo \ icu.lo insert.lo json.lo legacy.lo loadext.lo \ main.lo malloc.lo mem0.lo mem1.lo mem2.lo mem3.lo mem5.lo \ memdb.lo memjournal.lo \ mutex.lo mutex_noop.lo mutex_unix.lo mutex_w32.lo \ notify.lo opcodes.lo os.lo os_kv.lo os_unix.lo os_win.lo \ pager.lo pcache.lo pcache1.lo pragma.lo prepare.lo printf.lo \ random.lo resolve.lo rowset.lo rtree.lo \ sqlite3session.lo select.lo sqlite3rbu.lo status.lo stmt.lo \ table.lo threads.lo tokenize.lo treeview.lo trigger.lo \ update.lo upsert.lo util.lo vacuum.lo \ vdbeapi.lo vdbeaux.lo vdbeblob.lo vdbemem.lo vdbesort.lo \ vdbetrace.lo vdbevtab.lo wal.lo walker.lo where.lo wherecode.lo \ whereexpr.lo \ window.lo utf.lo vtab.lo # <</mark>> # Object files for the amalgamation. # LIBOBJS1 = sqlite3.lo |
︙ | ︙ | |||
1265 1266 1267 1268 1269 1270 1271 1272 1273 1274 1275 1276 1277 1278 1279 1280 1281 1282 1283 1284 1285 1286 1287 1288 1289 1290 1291 1292 1293 1294 1295 | $(TOP)\src\expr.c \ $(TOP)\src\fault.c \ $(TOP)\src\fkey.c \ $(TOP)\src\func.c \ $(TOP)\src\global.c \ $(TOP)\src\hash.c \ $(TOP)\src\insert.c \ $(TOP)\src\legacy.c \ $(TOP)\src\loadext.c \ $(TOP)\src\main.c \ $(TOP)\src\malloc.c \ $(TOP)\src\mem0.c \ $(TOP)\src\mem1.c \ $(TOP)\src\mem2.c \ $(TOP)\src\mem3.c \ $(TOP)\src\mem5.c \ $(TOP)\src\memdb.c \ $(TOP)\src\memjournal.c \ $(TOP)\src\mutex.c \ $(TOP)\src\mutex_noop.c \ $(TOP)\src\mutex_unix.c \ $(TOP)\src\mutex_w32.c \ $(TOP)\src\notify.c \ $(TOP)\src\os.c \ $(TOP)\src\os_unix.c \ $(TOP)\src\os_win.c # Core source code files, part 2. # SRC01 = \ $(TOP)\src\pager.c \ | > > | 1310 1311 1312 1313 1314 1315 1316 1317 1318 1319 1320 1321 1322 1323 1324 1325 1326 1327 1328 1329 1330 1331 1332 1333 1334 1335 1336 1337 1338 1339 1340 1341 1342 | $(TOP)\src\expr.c \ $(TOP)\src\fault.c \ $(TOP)\src\fkey.c \ $(TOP)\src\func.c \ $(TOP)\src\global.c \ $(TOP)\src\hash.c \ $(TOP)\src\insert.c \ $(TOP)\src\json.c \ $(TOP)\src\legacy.c \ $(TOP)\src\loadext.c \ $(TOP)\src\main.c \ $(TOP)\src\malloc.c \ $(TOP)\src\mem0.c \ $(TOP)\src\mem1.c \ $(TOP)\src\mem2.c \ $(TOP)\src\mem3.c \ $(TOP)\src\mem5.c \ $(TOP)\src\memdb.c \ $(TOP)\src\memjournal.c \ $(TOP)\src\mutex.c \ $(TOP)\src\mutex_noop.c \ $(TOP)\src\mutex_unix.c \ $(TOP)\src\mutex_w32.c \ $(TOP)\src\notify.c \ $(TOP)\src\os.c \ $(TOP)\src\os_kv.c \ $(TOP)\src\os_unix.c \ $(TOP)\src\os_win.c # Core source code files, part 2. # SRC01 = \ $(TOP)\src\pager.c \ |
︙ | ︙ | |||
1317 1318 1319 1320 1321 1322 1323 1324 1325 1326 1327 1328 1329 1330 | $(TOP)\src\vdbe.c \ $(TOP)\src\vdbeapi.c \ $(TOP)\src\vdbeaux.c \ $(TOP)\src\vdbeblob.c \ $(TOP)\src\vdbemem.c \ $(TOP)\src\vdbesort.c \ $(TOP)\src\vdbetrace.c \ $(TOP)\src\vtab.c \ $(TOP)\src\wal.c \ $(TOP)\src\walker.c \ $(TOP)\src\where.c \ $(TOP)\src\wherecode.c \ $(TOP)\src\whereexpr.c \ $(TOP)\src\window.c | > | 1364 1365 1366 1367 1368 1369 1370 1371 1372 1373 1374 1375 1376 1377 1378 | $(TOP)\src\vdbe.c \ $(TOP)\src\vdbeapi.c \ $(TOP)\src\vdbeaux.c \ $(TOP)\src\vdbeblob.c \ $(TOP)\src\vdbemem.c \ $(TOP)\src\vdbesort.c \ $(TOP)\src\vdbetrace.c \ $(TOP)\src\vdbevtab.c \ $(TOP)\src\vtab.c \ $(TOP)\src\wal.c \ $(TOP)\src\walker.c \ $(TOP)\src\where.c \ $(TOP)\src\wherecode.c \ $(TOP)\src\whereexpr.c \ $(TOP)\src\window.c |
︙ | ︙ | |||
1360 1361 1362 1363 1364 1365 1366 | $(TOP)\src\sqliteLimit.h \ $(TOP)\src\vdbe.h \ $(TOP)\src\vdbeInt.h \ $(TOP)\src\vxworks.h \ $(TOP)\src\wal.h \ $(TOP)\src\whereInt.h | < < < < < < < < < < < < < < < < < < < < < < < < < | 1408 1409 1410 1411 1412 1413 1414 1415 1416 1417 1418 1419 1420 1421 1422 1423 1424 1425 1426 1427 1428 1429 1430 1431 1432 1433 1434 1435 1436 1437 1438 1439 1440 1441 1442 1443 | $(TOP)\src\sqliteLimit.h \ $(TOP)\src\vdbe.h \ $(TOP)\src\vdbeInt.h \ $(TOP)\src\vxworks.h \ $(TOP)\src\wal.h \ $(TOP)\src\whereInt.h # Extension source code files, part 2. # SRC07 = \ $(TOP)\ext\fts3\fts3.c \ $(TOP)\ext\fts3\fts3_aux.c \ $(TOP)\ext\fts3\fts3_expr.c \ $(TOP)\ext\fts3\fts3_hash.c \ $(TOP)\ext\fts3\fts3_icu.c \ $(TOP)\ext\fts3\fts3_porter.c \ $(TOP)\ext\fts3\fts3_snippet.c \ $(TOP)\ext\fts3\fts3_tokenizer.c \ $(TOP)\ext\fts3\fts3_tokenizer1.c \ $(TOP)\ext\fts3\fts3_tokenize_vtab.c \ $(TOP)\ext\fts3\fts3_unicode.c \ $(TOP)\ext\fts3\fts3_unicode2.c \ $(TOP)\ext\fts3\fts3_write.c \ $(TOP)\ext\icu\icu.c \ $(TOP)\ext\rtree\rtree.c \ $(TOP)\ext\session\sqlite3session.c \ $(TOP)\ext\rbu\sqlite3rbu.c \ $(TOP)\ext\misc\stmt.c # Extension header files, part 2. # SRC09 = \ $(TOP)\ext\fts3\fts3.h \ $(TOP)\ext\fts3\fts3Int.h \ $(TOP)\ext\fts3\fts3_hash.h \ $(TOP)\ext\fts3\fts3_tokenizer.h \ |
︙ | ︙ | |||
1447 1448 1449 1450 1451 1452 1453 | $(SQLITETCLDECLSH) !ELSE SRC12 = !ENDIF # All source code files. # | | | 1470 1471 1472 1473 1474 1475 1476 1477 1478 1479 1480 1481 1482 1483 1484 | $(SQLITETCLDECLSH) !ELSE SRC12 = !ENDIF # All source code files. # SRC = $(SRC00) $(SRC01) $(SRC03) $(SRC04) $(SRC05) $(SRC07) $(SRC09) $(SRC10) $(SRC11) $(SRC12) # Source code to the test files. # TESTSRC = \ $(TOP)\src\test1.c \ $(TOP)\src\test2.c \ $(TOP)\src\test3.c \ |
︙ | ︙ | |||
1493 1494 1495 1496 1497 1498 1499 1500 1501 1502 1503 1504 1505 1506 | $(TOP)\src\test_schema.c \ $(TOP)\src\test_server.c \ $(TOP)\src\test_superlock.c \ $(TOP)\src\test_syscall.c \ $(TOP)\src\test_tclsh.c \ $(TOP)\src\test_tclvar.c \ $(TOP)\src\test_thread.c \ $(TOP)\src\test_vfs.c \ $(TOP)\src\test_windirent.c \ $(TOP)\src\test_window.c \ $(TOP)\src\test_wsd.c \ $(TOP)\ext\fts3\fts3_term.c \ $(TOP)\ext\fts3\fts3_test.c \ $(TOP)\ext\rbu\test_rbu.c \ | > | > > > > > > > | > > > > > < | 1516 1517 1518 1519 1520 1521 1522 1523 1524 1525 1526 1527 1528 1529 1530 1531 1532 1533 1534 1535 1536 1537 1538 1539 1540 1541 1542 1543 1544 1545 1546 1547 1548 1549 1550 1551 1552 1553 1554 1555 1556 1557 1558 1559 1560 1561 1562 1563 1564 1565 1566 1567 1568 1569 1570 1571 1572 1573 1574 1575 1576 1577 1578 1579 1580 1581 1582 1583 1584 1585 1586 1587 1588 1589 1590 1591 1592 | $(TOP)\src\test_schema.c \ $(TOP)\src\test_server.c \ $(TOP)\src\test_superlock.c \ $(TOP)\src\test_syscall.c \ $(TOP)\src\test_tclsh.c \ $(TOP)\src\test_tclvar.c \ $(TOP)\src\test_thread.c \ $(TOP)\src\test_vdbecov.c \ $(TOP)\src\test_vfs.c \ $(TOP)\src\test_windirent.c \ $(TOP)\src\test_window.c \ $(TOP)\src\test_wsd.c \ $(TOP)\ext\fts3\fts3_term.c \ $(TOP)\ext\fts3\fts3_test.c \ $(TOP)\ext\rbu\test_rbu.c \ $(TOP)\ext\session\test_session.c # Statically linked extensions. # TESTEXT = \ $(TOP)\ext\expert\sqlite3expert.c \ $(TOP)\ext\expert\test_expert.c \ $(TOP)\ext\misc\amatch.c \ $(TOP)\ext\misc\appendvfs.c \ $(TOP)\ext\misc\basexx.c \ $(TOP)\ext\misc\bgckpt.c \ $(TOP)\ext\misc\carray.c \ $(TOP)\ext\misc\cksumvfs.c \ $(TOP)\ext\misc\closure.c \ $(TOP)\ext\misc\csv.c \ $(TOP)\ext\misc\decimal.c \ $(TOP)\ext\misc\eval.c \ $(TOP)\ext\misc\explain.c \ $(TOP)\ext\misc\fileio.c \ $(TOP)\ext\misc\fuzzer.c \ $(TOP)\ext\fts5\fts5_tcl.c \ $(TOP)\ext\fts5\fts5_test_mi.c \ $(TOP)\ext\fts5\fts5_test_tok.c \ $(TOP)\ext\misc\ieee754.c \ $(TOP)\ext\misc\mmapwarm.c \ $(TOP)\ext\misc\nextchar.c \ $(TOP)\ext\misc\normalize.c \ $(TOP)\ext\misc\percentile.c \ $(TOP)\ext\misc\prefixes.c \ $(TOP)\ext\misc\qpvtab.c \ $(TOP)\ext\misc\regexp.c \ $(TOP)\ext\misc\remember.c \ $(TOP)\ext\misc\series.c \ $(TOP)\ext\misc\spellfix.c \ $(TOP)\ext\misc\totype.c \ $(TOP)\ext\misc\unionvtab.c \ $(TOP)\ext\misc\wholenumber.c \ $(TOP)\ext\rtree\test_rtreedoc.c \ $(TOP)\ext\recover\sqlite3recover.c \ $(TOP)\ext\recover\test_recover.c \ $(TOP)\ext\recover\dbdata.c \ fts5.c # If use of zlib is enabled, add the "zipfile.c" source file. # !IF $(USE_ZLIB)!=0 TESTEXT = $(TESTEXT) $(TOP)\ext\misc\zipfile.c !ENDIF # Source code to the library files needed by the test fixture # (non-amalgamation) # TESTSRC2 = \ $(SRC00) \ $(SRC01) \ $(SRC07) \ $(SRC10) \ $(TOP)\ext\async\sqlite3async.c # Header files used by all library source files. # HDR = \ |
︙ | ︙ | |||
1579 1580 1581 1582 1583 1584 1585 | $(TOP)\src\vdbeInt.h \ $(TOP)\src\vxworks.h \ $(TOP)\src\whereInt.h # Header files used by extensions # EXTHDR = $(EXTHDR) \ | < < < < < < < < | 1614 1615 1616 1617 1618 1619 1620 1621 1622 1623 1624 1625 1626 1627 | $(TOP)\src\vdbeInt.h \ $(TOP)\src\vxworks.h \ $(TOP)\src\whereInt.h # Header files used by extensions # EXTHDR = $(EXTHDR) \ $(TOP)\ext\fts3\fts3.h \ $(TOP)\ext\fts3\fts3Int.h \ $(TOP)\ext\fts3\fts3_hash.h \ $(TOP)\ext\fts3\fts3_tokenizer.h EXTHDR = $(EXTHDR) \ $(TOP)\ext\rtree\rtree.h \ $(TOP)\ext\rtree\geopoly.c |
︙ | ︙ | |||
1621 1622 1623 1624 1625 1626 1627 | FUZZDATA = \ $(TOP)\test\fuzzdata1.db \ $(TOP)\test\fuzzdata2.db \ $(TOP)\test\fuzzdata3.db \ $(TOP)\test\fuzzdata4.db \ $(TOP)\test\fuzzdata5.db \ $(TOP)\test\fuzzdata6.db \ | | > < | | > > > > | > > > > > > > | > > > > > > | | 1648 1649 1650 1651 1652 1653 1654 1655 1656 1657 1658 1659 1660 1661 1662 1663 1664 1665 1666 1667 1668 1669 1670 1671 1672 1673 1674 1675 1676 1677 1678 1679 1680 1681 1682 1683 1684 1685 1686 1687 1688 1689 1690 1691 1692 1693 1694 1695 1696 1697 1698 1699 1700 1701 1702 1703 1704 1705 1706 1707 1708 1709 1710 1711 | FUZZDATA = \ $(TOP)\test\fuzzdata1.db \ $(TOP)\test\fuzzdata2.db \ $(TOP)\test\fuzzdata3.db \ $(TOP)\test\fuzzdata4.db \ $(TOP)\test\fuzzdata5.db \ $(TOP)\test\fuzzdata6.db \ $(TOP)\test\fuzzdata7.db \ $(TOP)\test\fuzzdata8.db # <</mark>> # Additional compiler options for the shell. These are only effective # when the shell is not being dynamically linked. # !IF $(DYNAMIC_SHELL)==0 && $(FOR_WIN10)==0 SHELL_COMPILE_OPTS = $(SHELL_COMPILE_OPTS) -DSQLITE_ENABLE_FTS4=1 SHELL_COMPILE_OPTS = $(SHELL_COMPILE_OPTS) -DSQLITE_ENABLE_EXPLAIN_COMMENTS=1 SHELL_COMPILE_OPTS = $(SHELL_COMPILE_OPTS) -DSQLITE_ENABLE_OFFSET_SQL_FUNC=1 !ENDIF # <<mark>> # Extra compiler options for various test tools. # MPTESTER_COMPILE_OPTS = -DSQLITE_ENABLE_FTS5 FUZZERSHELL_COMPILE_OPTS = FUZZCHECK_OPTS = $(FUZZCHECK_OPTS) -I$(TOP)\test -I$(TOP)\ext\recover FUZZCHECK_OPTS = $(FUZZCHECK_OPTS) -DSQLITE_ENABLE_MEMSYS5 FUZZCHECK_OPTS = $(FUZZCHECK_OPTS) -DSQLITE_OSS_FUZZ FUZZCHECK_OPTS = $(FUZZCHECK_OPTS) -DSQLITE_MAX_MEMORY=50000000 FUZZCHECK_OPTS = $(FUZZCHECK_OPTS) -DSQLITE_PRINTF_PRECISION_LIMIT=1000 FUZZCHECK_OPTS = $(FUZZCHECK_OPTS) -DSQLITE_OMIT_LOAD_EXTENSION FUZZCHECK_OPTS = $(FUZZCHECK_OPTS) -DSQLITE_ENABLE_FTS4 FUZZCHECK_OPTS = $(FUZZCHECK_OPTS) -DSQLITE_ENABLE_FTS5 FUZZCHECK_OPTS = $(FUZZCHECK_OPTS) -DSQLITE_ENABLE_RTREE FUZZCHECK_OPTS = $(FUZZCHECK_OPTS) -DSQLITE_ENABLE_GEOPOLY FUZZCHECK_OPTS = $(FUZZCHECK_OPTS) -DSQLITE_ENABLE_DBSTAT_VTAB FUZZCHECK_OPTS = $(FUZZCHECK_OPTS) -DSQLITE_ENABLE_BYTECODE_VTAB FUZZCHECK_SRC = $(FUZZCHECK_SRC) $(TOP)\test\fuzzcheck.c FUZZCHECK_SRC = $(FUZZCHECK_SRC) $(TOP)\test\ossfuzz.c FUZZCHECK_SRC = $(FUZZCHECK_SRC) $(TOP)\test\fuzzinvariants.c FUZZCHECK_SRC = $(FUZZCHECK_SRC) $(TOP)\test\vt02.c FUZZCHECK_SRC = $(FUZZCHECK_SRC) $(TOP)\ext\recover\dbdata.c FUZZCHECK_SRC = $(FUZZCHECK_SRC) $(TOP)\ext\recover\sqlite3recover.c OSSSHELL_SRC = $(TOP)\test\ossshell.c $(TOP)\test\ossfuzz.c DBFUZZ_COMPILE_OPTS = -DSQLITE_THREADSAFE=0 -DSQLITE_OMIT_LOAD_EXTENSION KV_COMPILE_OPTS = -DSQLITE_THREADSAFE=0 -DSQLITE_DIRECT_OVERFLOW_READ ST_COMPILE_OPTS = -DSQLITE_THREADSAFE=0 # Standard options to testfixture. # TESTOPTS = --verbose=file --output=test-out.txt # Extra targets for the "all" target that require Tcl. # !IF $(NO_TCL)==0 ALL_TCL_TARGETS = $(SQLITE3TCLDLL) !ELSE ALL_TCL_TARGETS = !ENDIF # <</mark>> # This is the default Makefile target. The objects listed here # are what get build when you type just "make" with no arguments. |
︙ | ︙ | |||
1685 1686 1687 1688 1689 1690 1691 | shell: $(SQLITE3EXE) # <<mark>> libsqlite3.lib: $(LIBOBJ) $(LTLIB) $(LTLIBOPTS) /OUT:$@ $(LIBOBJ) $(TLIBS) libtclsqlite3.lib: tclsqlite.lo libsqlite3.lib | | > > > > > > > > > > > > > > > | | 1729 1730 1731 1732 1733 1734 1735 1736 1737 1738 1739 1740 1741 1742 1743 1744 1745 1746 1747 1748 1749 1750 1751 1752 1753 1754 1755 1756 1757 1758 1759 1760 1761 1762 1763 1764 1765 1766 1767 1768 | shell: $(SQLITE3EXE) # <<mark>> libsqlite3.lib: $(LIBOBJ) $(LTLIB) $(LTLIBOPTS) /OUT:$@ $(LIBOBJ) $(TLIBS) libtclsqlite3.lib: tclsqlite.lo libsqlite3.lib $(LTLIB) $(LTLIBOPTS) $(TCLLIBPATHS) $(LTLIBPATHS) /OUT:$@ tclsqlite.lo libsqlite3.lib $(LIBTCLSTUB) $(TLIBS) tclsqlite3.def: tclsqlite.lo echo EXPORTS > tclsqlite3.def dumpbin /all tclsqlite.lo \ | $(TCLSH_CMD) $(TOP)\tool\replace.tcl include "^\s+/EXPORT:_?((?:Sqlite3|Tclsqlite3)_[^@]*)(?:@\d+)?$$" \1 \ | sort >> tclsqlite3.def pkgIndex.tcl: $(TOP)\VERSION for /F %%V in ('type "$(TOP)\VERSION"') do ( \ echo package ifneeded sqlite3 @version@ [list load [file join $$dir $(SQLITE3TCLDLL)] sqlite3] \ | $(TCLSH_CMD) $(TOP)\tool\replace.tcl exact @version@ %%V > pkgIndex.tcl \ ) $(SQLITE3TCLDLL): libtclsqlite3.lib $(LIBRESOBJS) tclsqlite3.def pkgIndex.tcl $(LD) $(LDFLAGS) $(LTLINKOPTS) $(LTLIBPATHS) /DLL /DEF:tclsqlite3.def /OUT:$@ libtclsqlite3.lib $(LIBRESOBJS) $(LTLIBS) $(TLIBS) # <</mark>> $(SQLITE3DLL): $(LIBOBJ) $(LIBRESOBJS) $(CORE_LINK_DEP) $(LD) $(LDFLAGS) $(LTLINKOPTS) $(LTLIBPATHS) /DLL $(CORE_LINK_OPTS) /OUT:$@ $(LIBOBJ) $(LIBRESOBJS) $(LTLIBS) $(TLIBS) # <<block2>> sqlite3.def: libsqlite3.lib echo EXPORTS > sqlite3.def dumpbin /all libsqlite3.lib \ | $(TCLSH_CMD) $(TOP)\tool\replace.tcl include "^\s+1 _?(sqlite3(?:session|changeset|changegroup|rebaser|rbu)?_[^@]*)(?:@\d+)?$$" \1 \ | sort >> sqlite3.def # <</block2>> $(SQLITE3EXE): shell.c $(SHELL_CORE_DEP) $(LIBRESOBJS) $(SHELL_CORE_SRC) $(SQLITE3H) $(LTLINK) $(SHELL_COMPILE_OPTS) $(READLINE_FLAGS) shell.c $(SHELL_CORE_SRC) \ /link $(SQLITE3EXEPDB) $(LDFLAGS) $(LTLINKOPTS) $(SHELL_LINK_OPTS) $(LTLIBPATHS) $(LIBRESOBJS) $(LIBREADLINE) $(LTLIBS) $(TLIBS) |
︙ | ︙ | |||
1726 1727 1728 1729 1730 1731 1732 | fuzzershell.exe: $(TOP)\tool\fuzzershell.c $(SQLITE3C) $(SQLITE3H) $(LTLINK) $(NO_WARN) $(FUZZERSHELL_COMPILE_OPTS) $(TOP)\tool\fuzzershell.c $(SQLITE3C) /link $(LDFLAGS) $(LTLINKOPTS) dbfuzz.exe: $(TOP)\test\dbfuzz.c $(SQLITE3C) $(SQLITE3H) $(LTLINK) $(NO_WARN) $(DBFUZZ_COMPILE_OPTS) $(TOP)\test\dbfuzz.c $(SQLITE3C) /link $(LDFLAGS) $(LTLINKOPTS) fuzzcheck.exe: $(FUZZCHECK_SRC) $(SQLITE3C) $(SQLITE3H) | | | | 1785 1786 1787 1788 1789 1790 1791 1792 1793 1794 1795 1796 1797 1798 1799 1800 1801 1802 | fuzzershell.exe: $(TOP)\tool\fuzzershell.c $(SQLITE3C) $(SQLITE3H) $(LTLINK) $(NO_WARN) $(FUZZERSHELL_COMPILE_OPTS) $(TOP)\tool\fuzzershell.c $(SQLITE3C) /link $(LDFLAGS) $(LTLINKOPTS) dbfuzz.exe: $(TOP)\test\dbfuzz.c $(SQLITE3C) $(SQLITE3H) $(LTLINK) $(NO_WARN) $(DBFUZZ_COMPILE_OPTS) $(TOP)\test\dbfuzz.c $(SQLITE3C) /link $(LDFLAGS) $(LTLINKOPTS) fuzzcheck.exe: $(FUZZCHECK_SRC) $(SQLITE3C) $(SQLITE3H) $(LTLINK) $(NO_WARN) $(FUZZCHECK_OPTS) $(FUZZCHECK_SRC) $(SQLITE3C) /link $(LDFLAGS) $(LTLINKOPTS) ossshell.exe: $(OSSSHELL_SRC) $(SQLITE3C) $(SQLITE3H) $(LTLINK) $(NO_WARN) $(FUZZCHECK_OPTS) $(OSSSHELL_SRC) $(SQLITE3C) /link $(LDFLAGS) $(LTLINKOPTS) sessionfuzz.exe: zlib $(TOP)\test\sessionfuzz.c $(SQLITE3C) $(SQLITE3H) $(LTLINK) $(NO_WARN) -I$(ZLIBINCDIR) $(TOP)\test\sessionfuzz.c /link $(LDFLAGS) $(LTLINKOPTS) /LIBPATH:$(ZLIBLIBDIR) $(ZLIBLIB) mptester.exe: $(TOP)\mptest\mptest.c $(SQLITE3C) $(SQLITE3H) $(LTLINK) $(NO_WARN) $(MPTESTER_COMPILE_OPTS) $(TOP)\mptest\mptest.c $(SQLITE3C) /link $(LDFLAGS) $(LTLINKOPTS) |
︙ | ︙ | |||
1765 1766 1767 1768 1769 1770 1771 | -rmdir /Q/S tsrc 2>NUL -mkdir tsrc for %i in ($(SRC00)) do copy /Y %i tsrc for %i in ($(SRC01)) do copy /Y %i tsrc for %i in ($(SRC03)) do copy /Y %i tsrc for %i in ($(SRC04)) do copy /Y %i tsrc for %i in ($(SRC05)) do copy /Y %i tsrc | < < > > | < | > | 1824 1825 1826 1827 1828 1829 1830 1831 1832 1833 1834 1835 1836 1837 1838 1839 1840 1841 1842 1843 1844 1845 1846 1847 1848 1849 1850 1851 1852 1853 1854 1855 1856 1857 1858 1859 1860 1861 1862 1863 1864 1865 1866 1867 1868 1869 | -rmdir /Q/S tsrc 2>NUL -mkdir tsrc for %i in ($(SRC00)) do copy /Y %i tsrc for %i in ($(SRC01)) do copy /Y %i tsrc for %i in ($(SRC03)) do copy /Y %i tsrc for %i in ($(SRC04)) do copy /Y %i tsrc for %i in ($(SRC05)) do copy /Y %i tsrc for %i in ($(SRC07)) do copy /Y %i tsrc for %i in ($(SRC09)) do copy /Y %i tsrc for %i in ($(SRC10)) do copy /Y %i tsrc for %i in ($(SRC11)) do copy /Y %i tsrc for %i in ($(SRC12)) do copy /Y %i tsrc copy /Y fts5.c tsrc copy /B tsrc\fts5.c +,, copy /Y fts5.h tsrc copy /B tsrc\fts5.h +,, del /Q tsrc\sqlite.h.in tsrc\parse.y 2>NUL $(TCLSH_CMD) $(TOP)\tool\vdbe-compress.tcl $(OPTS) < tsrc\vdbe.c > vdbe.new move vdbe.new tsrc\vdbe.c echo > .target_source sqlite3.c: .target_source sqlite3ext.h sqlite3session.h $(MKSQLITE3C_TOOL) $(TCLSH_CMD) $(MKSQLITE3C_TOOL) $(MKSQLITE3C_ARGS) sqlite3-all.c: sqlite3.c $(TOP)\tool\split-sqlite3c.tcl $(TCLSH_CMD) $(TOP)\tool\split-sqlite3c.tcl # <</mark>> # Rule to build the amalgamation # sqlite3.lo: $(SQLITE3C) $(LTCOMPILE) $(CORE_COMPILE_OPTS) -c $(SQLITE3C) # <<mark>> # Rules to build the LEMON compiler generator # lempar.c: $(TOP)\tool\lempar.c copy /Y $(TOP)\tool\lempar.c . copy /B lempar.c +,, lemon.exe: $(TOP)\tool\lemon.c lempar.c $(BCC) $(NO_WARN) -Daccess=_access \ -Fe$@ $(TOP)\tool\lemon.c /link $(LDFLAGS) $(NLTLINKOPTS) $(NLTLIBPATHS) # <<mark>> # Rules to build the source-id generator tool |
︙ | ︙ | |||
1825 1826 1827 1828 1829 1830 1831 | $(LTCOMPILE) $(CORE_COMPILE_OPTS) -c opcodes.c # <</mark>> # Rule to build the Win32 resources object file. # !IF $(USE_RC)!=0 # <<block1>> | | | 1884 1885 1886 1887 1888 1889 1890 1891 1892 1893 1894 1895 1896 1897 1898 | $(LTCOMPILE) $(CORE_COMPILE_OPTS) -c opcodes.c # <</mark>> # Rule to build the Win32 resources object file. # !IF $(USE_RC)!=0 # <<block1>> $(LIBRESOBJS): $(TOP)\src\sqlite3.rc $(SQLITE3H) $(TOP)\VERSION echo #ifndef SQLITE_RESOURCE_VERSION > sqlite3rc.h for /F %%V in ('type "$(TOP)\VERSION"') do ( \ echo #define SQLITE_RESOURCE_VERSION %%V \ | $(TCLSH_CMD) $(TOP)\tool\replace.tcl exact . ^, >> sqlite3rc.h \ ) echo #endif >> sqlite3rc.h $(LTRCOMPILE) -fo $(LIBRESOBJS) $(TOP)\src\sqlite3.rc |
︙ | ︙ | |||
1908 1909 1910 1911 1912 1913 1914 1915 1916 1917 1918 1919 1920 1921 | hash.lo: $(TOP)\src\hash.c $(HDR) $(LTCOMPILE) $(CORE_COMPILE_OPTS) -c $(TOP)\src\hash.c insert.lo: $(TOP)\src\insert.c $(HDR) $(LTCOMPILE) $(CORE_COMPILE_OPTS) -c $(TOP)\src\insert.c legacy.lo: $(TOP)\src\legacy.c $(HDR) $(LTCOMPILE) $(CORE_COMPILE_OPTS) -c $(TOP)\src\legacy.c loadext.lo: $(TOP)\src\loadext.c $(HDR) $(LTCOMPILE) $(CORE_COMPILE_OPTS) -c $(TOP)\src\loadext.c main.lo: $(TOP)\src\main.c $(HDR) | > > > | 1967 1968 1969 1970 1971 1972 1973 1974 1975 1976 1977 1978 1979 1980 1981 1982 1983 | hash.lo: $(TOP)\src\hash.c $(HDR) $(LTCOMPILE) $(CORE_COMPILE_OPTS) -c $(TOP)\src\hash.c insert.lo: $(TOP)\src\insert.c $(HDR) $(LTCOMPILE) $(CORE_COMPILE_OPTS) -c $(TOP)\src\insert.c json.lo: $(TOP)\src\json.c $(HDR) $(LTCOMPILE) $(CORE_COMPILE_OPTS) -c $(TOP)\src\json.c legacy.lo: $(TOP)\src\legacy.c $(HDR) $(LTCOMPILE) $(CORE_COMPILE_OPTS) -c $(TOP)\src\legacy.c loadext.lo: $(TOP)\src\loadext.c $(HDR) $(LTCOMPILE) $(CORE_COMPILE_OPTS) -c $(TOP)\src\loadext.c main.lo: $(TOP)\src\main.c $(HDR) |
︙ | ︙ | |||
1968 1969 1970 1971 1972 1973 1974 1975 1976 1977 1978 1979 1980 1981 | pcache1.lo: $(TOP)\src\pcache1.c $(HDR) $(TOP)\src\pcache.h $(LTCOMPILE) $(CORE_COMPILE_OPTS) -c $(TOP)\src\pcache1.c os.lo: $(TOP)\src\os.c $(HDR) $(LTCOMPILE) $(CORE_COMPILE_OPTS) -c $(TOP)\src\os.c os_unix.lo: $(TOP)\src\os_unix.c $(HDR) $(LTCOMPILE) $(CORE_COMPILE_OPTS) -c $(TOP)\src\os_unix.c os_win.lo: $(TOP)\src\os_win.c $(HDR) $(LTCOMPILE) $(CORE_COMPILE_OPTS) -c $(TOP)\src\os_win.c pragma.lo: $(TOP)\src\pragma.c $(HDR) | > > > | 2030 2031 2032 2033 2034 2035 2036 2037 2038 2039 2040 2041 2042 2043 2044 2045 2046 | pcache1.lo: $(TOP)\src\pcache1.c $(HDR) $(TOP)\src\pcache.h $(LTCOMPILE) $(CORE_COMPILE_OPTS) -c $(TOP)\src\pcache1.c os.lo: $(TOP)\src\os.c $(HDR) $(LTCOMPILE) $(CORE_COMPILE_OPTS) -c $(TOP)\src\os.c os_kv.lo: $(TOP)\src\os_kv.c $(HDR) $(LTCOMPILE) $(CORE_COMPILE_OPTS) -c $(TOP)\src\os_kv.c os_unix.lo: $(TOP)\src\os_unix.c $(HDR) $(LTCOMPILE) $(CORE_COMPILE_OPTS) -c $(TOP)\src\os_unix.c os_win.lo: $(TOP)\src\os_win.c $(HDR) $(LTCOMPILE) $(CORE_COMPILE_OPTS) -c $(TOP)\src\os_win.c pragma.lo: $(TOP)\src\pragma.c $(HDR) |
︙ | ︙ | |||
2049 2050 2051 2052 2053 2054 2055 2056 2057 2058 2059 2060 2061 2062 | vdbesort.lo: $(TOP)\src\vdbesort.c $(HDR) $(LTCOMPILE) $(CORE_COMPILE_OPTS) -c $(TOP)\src\vdbesort.c vdbetrace.lo: $(TOP)\src\vdbetrace.c $(HDR) $(LTCOMPILE) $(CORE_COMPILE_OPTS) -c $(TOP)\src\vdbetrace.c vtab.lo: $(TOP)\src\vtab.c $(HDR) $(LTCOMPILE) $(CORE_COMPILE_OPTS) -c $(TOP)\src\vtab.c wal.lo: $(TOP)\src\wal.c $(HDR) $(LTCOMPILE) $(CORE_COMPILE_OPTS) -c $(TOP)\src\wal.c walker.lo: $(TOP)\src\walker.c $(HDR) | > > > | 2114 2115 2116 2117 2118 2119 2120 2121 2122 2123 2124 2125 2126 2127 2128 2129 2130 | vdbesort.lo: $(TOP)\src\vdbesort.c $(HDR) $(LTCOMPILE) $(CORE_COMPILE_OPTS) -c $(TOP)\src\vdbesort.c vdbetrace.lo: $(TOP)\src\vdbetrace.c $(HDR) $(LTCOMPILE) $(CORE_COMPILE_OPTS) -c $(TOP)\src\vdbetrace.c vdbevtab.lo: $(TOP)\src\vdbevtab.c $(HDR) $(LTCOMPILE) $(CORE_COMPILE_OPTS) -c $(TOP)\src\vdbevtab.c vtab.lo: $(TOP)\src\vtab.c $(HDR) $(LTCOMPILE) $(CORE_COMPILE_OPTS) -c $(TOP)\src\vtab.c wal.lo: $(TOP)\src\wal.c $(HDR) $(LTCOMPILE) $(CORE_COMPILE_OPTS) -c $(TOP)\src\wal.c walker.lo: $(TOP)\src\walker.c $(HDR) |
︙ | ︙ | |||
2091 2092 2093 2094 2095 2096 2097 | opcodes.h: parse.h $(TOP)\src\vdbe.c $(TOP)\tool\mkopcodeh.tcl type parse.h $(TOP)\src\vdbe.c | $(TCLSH_CMD) $(TOP)\tool\mkopcodeh.tcl > opcodes.h # Rules to build parse.c and parse.h - the outputs of lemon. # parse.h: parse.c | | | > | < < > > > > > > | > > > > > > | > > > > < < < < < < < < < < < < < < < < < < | 2159 2160 2161 2162 2163 2164 2165 2166 2167 2168 2169 2170 2171 2172 2173 2174 2175 2176 2177 2178 2179 2180 2181 2182 2183 2184 2185 2186 2187 2188 2189 2190 2191 2192 2193 2194 2195 2196 2197 2198 2199 2200 2201 2202 2203 2204 2205 2206 2207 2208 2209 2210 2211 2212 2213 2214 2215 2216 2217 2218 2219 2220 2221 2222 2223 2224 2225 2226 2227 2228 2229 2230 2231 2232 2233 2234 2235 2236 2237 2238 2239 2240 2241 2242 | opcodes.h: parse.h $(TOP)\src\vdbe.c $(TOP)\tool\mkopcodeh.tcl type parse.h $(TOP)\src\vdbe.c | $(TCLSH_CMD) $(TOP)\tool\mkopcodeh.tcl > opcodes.h # Rules to build parse.c and parse.h - the outputs of lemon. # parse.h: parse.c parse.c: $(TOP)\src\parse.y lemon.exe del /Q parse.y parse.h parse.h.temp 2>NUL copy /Y $(TOP)\src\parse.y . copy /B parse.y +,, .\lemon.exe $(REQ_FEATURE_FLAGS) $(OPT_FEATURE_FLAGS) $(EXT_FEATURE_FLAGS) $(OPTS) -S parse.y $(SQLITE3H): $(TOP)\src\sqlite.h.in $(TOP)\manifest mksourceid.exe $(TOP)\VERSION $(TCLSH_CMD) $(TOP)\tool\mksqlite3h.tcl $(TOP:\=/) > $(SQLITE3H) $(MKSQLITE3H_ARGS) sqlite3ext.h: .target_source !IF $(USE_STDCALL)!=0 || $(FOR_WIN10)!=0 type tsrc\sqlite3ext.h | $(TCLSH_CMD) $(TOP)\tool\replace.tcl regsub "\(\*\)" "(SQLITE_CALLBACK *)" \ | $(TCLSH_CMD) $(TOP)\tool\replace.tcl regsub "\(\*" "(SQLITE_APICALL *" > sqlite3ext.h copy /Y sqlite3ext.h tsrc\sqlite3ext.h !ELSE copy /Y tsrc\sqlite3ext.h sqlite3ext.h copy /B sqlite3ext.h +,, !ENDIF sqlite3session.h: $(TOP)\ext\session\sqlite3session.h copy /Y $(TOP)\ext\session\sqlite3session.h . copy /B sqlite3session.h +,, mkkeywordhash.exe: $(TOP)\tool\mkkeywordhash.c $(BCC) $(NO_WARN) -Fe$@ $(REQ_FEATURE_FLAGS) $(OPT_FEATURE_FLAGS) $(EXT_FEATURE_FLAGS) $(OPTS) \ $(TOP)\tool\mkkeywordhash.c /link $(LDFLAGS) $(NLTLINKOPTS) $(NLTLIBPATHS) keywordhash.h: $(TOP)\tool\mkkeywordhash.c mkkeywordhash.exe .\mkkeywordhash.exe > keywordhash.h # Source files that go into making shell.c SHELL_SRC = \ $(TOP)\src\shell.c.in \ $(TOP)\ext\misc\appendvfs.c \ $(TOP)\ext\misc\completion.c \ $(TOP)\ext\misc\base64.c \ $(TOP)\ext\misc\base85.c \ $(TOP)\ext\misc\decimal.c \ $(TOP)\ext\misc\fileio.c \ $(TOP)\ext\misc\ieee754.c \ $(TOP)\ext\misc\regexp.c \ $(TOP)\ext\misc\series.c \ $(TOP)\ext\misc\shathree.c \ $(TOP)\ext\misc\uint.c \ $(TOP)\ext\expert\sqlite3expert.c \ $(TOP)\ext\expert\sqlite3expert.h \ $(TOP)\ext\misc\memtrace.c \ $(TOP)/ext/recover/dbdata.c \ $(TOP)/ext/recover/sqlite3recover.c \ $(TOP)/ext/recover/sqlite3recover.h \ $(TOP)\src\test_windirent.c # If use of zlib is enabled, add the "zipfile.c" source file. # !IF $(USE_ZLIB)!=0 SHELL_SRC = $(SHELL_SRC) $(TOP)\ext\misc\sqlar.c SHELL_SRC = $(SHELL_SRC) $(TOP)\ext\misc\zipfile.c !ENDIF shell.c: $(SHELL_SRC) $(TOP)\tool\mkshellc.tcl $(TCLSH_CMD) $(TOP)\tool\mkshellc.tcl > shell.c zlib: pushd $(ZLIBDIR) && $(MAKE) /f win32\Makefile.msc clean $(ZLIBLIB) && popd # Rules to build the extension objects. # icu.lo: $(TOP)\ext\icu\icu.c $(HDR) $(EXTHDR) $(LTCOMPILE) $(CORE_COMPILE_OPTS) $(NO_WARN) -DSQLITE_CORE -c $(TOP)\ext\icu\icu.c fts3.lo: $(TOP)\ext\fts3\fts3.c $(HDR) $(EXTHDR) $(LTCOMPILE) $(CORE_COMPILE_OPTS) $(NO_WARN) -DSQLITE_CORE -c $(TOP)\ext\fts3\fts3.c fts3_aux.lo: $(TOP)\ext\fts3\fts3_aux.c $(HDR) $(EXTHDR) $(LTCOMPILE) $(CORE_COMPILE_OPTS) $(NO_WARN) -DSQLITE_CORE -c $(TOP)\ext\fts3\fts3_aux.c fts3_expr.lo: $(TOP)\ext\fts3\fts3_expr.c $(HDR) $(EXTHDR) |
︙ | ︙ | |||
2203 2204 2205 2206 2207 2208 2209 | fts3_unicode2.lo: $(TOP)\ext\fts3\fts3_unicode2.c $(HDR) $(EXTHDR) $(LTCOMPILE) $(CORE_COMPILE_OPTS) $(NO_WARN) -DSQLITE_CORE -c $(TOP)\ext\fts3\fts3_unicode2.c fts3_write.lo: $(TOP)\ext\fts3\fts3_write.c $(HDR) $(EXTHDR) $(LTCOMPILE) $(CORE_COMPILE_OPTS) $(NO_WARN) -DSQLITE_CORE -c $(TOP)\ext\fts3\fts3_write.c | < < < | 2268 2269 2270 2271 2272 2273 2274 2275 2276 2277 2278 2279 2280 2281 | fts3_unicode2.lo: $(TOP)\ext\fts3\fts3_unicode2.c $(HDR) $(EXTHDR) $(LTCOMPILE) $(CORE_COMPILE_OPTS) $(NO_WARN) -DSQLITE_CORE -c $(TOP)\ext\fts3\fts3_unicode2.c fts3_write.lo: $(TOP)\ext\fts3\fts3_write.c $(HDR) $(EXTHDR) $(LTCOMPILE) $(CORE_COMPILE_OPTS) $(NO_WARN) -DSQLITE_CORE -c $(TOP)\ext\fts3\fts3_write.c stmt.lo: $(TOP)\ext\misc\stmt.c $(HDR) $(EXTHDR) $(LTCOMPILE) $(CORE_COMPILE_OPTS) $(NO_WARN) -DSQLITE_CORE -c $(TOP)\ext\misc\stmt.c rtree.lo: $(TOP)\ext\rtree\rtree.c $(HDR) $(EXTHDR) $(LTCOMPILE) $(CORE_COMPILE_OPTS) $(NO_WARN) -DSQLITE_CORE -c $(TOP)\ext\rtree\rtree.c sqlite3session.lo: $(TOP)\ext\session\sqlite3session.c $(HDR) $(EXTHDR) |
︙ | ︙ | |||
2253 2254 2255 2256 2257 2258 2259 | $(TOP)\ext\lsm1\lsm_tree.c \ $(TOP)\ext\lsm1\lsm_unix.c \ $(TOP)\ext\lsm1\lsm_varint.c \ $(TOP)\ext\lsm1\lsm_vtab.c \ $(TOP)\ext\lsm1\lsm_win32.c fts5parse.c: $(TOP)\ext\fts5\fts5parse.y lemon.exe | | > | | > | > | 2315 2316 2317 2318 2319 2320 2321 2322 2323 2324 2325 2326 2327 2328 2329 2330 2331 2332 2333 2334 2335 2336 2337 2338 2339 2340 2341 2342 2343 2344 | $(TOP)\ext\lsm1\lsm_tree.c \ $(TOP)\ext\lsm1\lsm_unix.c \ $(TOP)\ext\lsm1\lsm_varint.c \ $(TOP)\ext\lsm1\lsm_vtab.c \ $(TOP)\ext\lsm1\lsm_win32.c fts5parse.c: $(TOP)\ext\fts5\fts5parse.y lemon.exe copy /Y $(TOP)\ext\fts5\fts5parse.y . copy /B fts5parse.y +,, del /Q fts5parse.h 2>NUL .\lemon.exe $(REQ_FEATURE_FLAGS) $(OPT_FEATURE_FLAGS) $(EXT_FEATURE_FLAGS) $(OPTS) -S fts5parse.y fts5parse.h: fts5parse.c fts5.c: $(FTS5_SRC) $(TCLSH_CMD) $(TOP)\ext\fts5\tool\mkfts5c.tcl copy /Y $(TOP)\ext\fts5\fts5.h . copy /B fts5.h +,, lsm1.c: $(LSM1_SRC) $(TCLSH_CMD) $(TOP)\ext\lsm1\tool\mklsm1c.tcl copy /Y $(TOP)\ext\lsm1\lsm.h . copy /B lsm.h +,, fts5.lo: fts5.c $(HDR) $(EXTHDR) $(LTCOMPILE) $(CORE_COMPILE_OPTS) $(NO_WARN) -DSQLITE_CORE -c fts5.c fts5_ext.lo: fts5.c $(HDR) $(EXTHDR) $(LTCOMPILE) $(NO_WARN) -c fts5.c |
︙ | ︙ | |||
2293 2294 2295 2296 2297 2298 2299 | TESTFIXTURE_FLAGS = -DTCLSH_INIT_PROC=sqlite3TestInit -DSQLITE_TEST=1 -DSQLITE_CRASH_TEST=1 TESTFIXTURE_FLAGS = $(TESTFIXTURE_FLAGS) -DSQLITE_SERVER=1 -DSQLITE_PRIVATE="" TESTFIXTURE_FLAGS = $(TESTFIXTURE_FLAGS) -DSQLITE_CORE $(NO_WARN) TESTFIXTURE_FLAGS = $(TESTFIXTURE_FLAGS) -DSQLITE_SERIES_CONSTRAINT_VERIFY=1 TESTFIXTURE_FLAGS = $(TESTFIXTURE_FLAGS) -DSQLITE_DEFAULT_PAGE_SIZE=1024 TESTFIXTURE_FLAGS = $(TESTFIXTURE_FLAGS) -DSQLITE_ENABLE_STMTVTAB=1 TESTFIXTURE_FLAGS = $(TESTFIXTURE_FLAGS) -DSQLITE_ENABLE_DBPAGE_VTAB=1 | | | | 2358 2359 2360 2361 2362 2363 2364 2365 2366 2367 2368 2369 2370 2371 2372 2373 | TESTFIXTURE_FLAGS = -DTCLSH_INIT_PROC=sqlite3TestInit -DSQLITE_TEST=1 -DSQLITE_CRASH_TEST=1 TESTFIXTURE_FLAGS = $(TESTFIXTURE_FLAGS) -DSQLITE_SERVER=1 -DSQLITE_PRIVATE="" TESTFIXTURE_FLAGS = $(TESTFIXTURE_FLAGS) -DSQLITE_CORE $(NO_WARN) TESTFIXTURE_FLAGS = $(TESTFIXTURE_FLAGS) -DSQLITE_SERIES_CONSTRAINT_VERIFY=1 TESTFIXTURE_FLAGS = $(TESTFIXTURE_FLAGS) -DSQLITE_DEFAULT_PAGE_SIZE=1024 TESTFIXTURE_FLAGS = $(TESTFIXTURE_FLAGS) -DSQLITE_ENABLE_STMTVTAB=1 TESTFIXTURE_FLAGS = $(TESTFIXTURE_FLAGS) -DSQLITE_ENABLE_DBPAGE_VTAB=1 TESTFIXTURE_FLAGS = $(TESTFIXTURE_FLAGS) -DSQLITE_ENABLE_BYTECODE_VTAB=1 TESTFIXTURE_FLAGS = $(TESTFIXTURE_FLAGS) -DSQLITE_CKSUMVFS_STATIC=1 TESTFIXTURE_FLAGS = $(TESTFIXTURE_FLAGS) $(TEST_CCONV_OPTS) TESTFIXTURE_SRC0 = $(TESTEXT) $(TESTSRC2) TESTFIXTURE_SRC1 = $(TESTEXT) $(SQLITE3C) !IF $(USE_AMALGAMATION)==0 TESTFIXTURE_SRC = $(TESTSRC) $(TOP)\src\tclsqlite.c $(TESTFIXTURE_SRC0) !ELSE |
︙ | ︙ | |||
2339 2340 2341 2342 2343 2344 2345 | @set PATH=$(LIBTCLPATH);$(PATH) .\testfixture.exe $(TOP)\test\loadext.test $(TESTOPTS) coretestprogs: $(TESTPROGS) testprogs: coretestprogs srcck1.exe fuzzcheck.exe sessionfuzz.exe | > > | < < < | > > > > > > > > > > > > > | 2404 2405 2406 2407 2408 2409 2410 2411 2412 2413 2414 2415 2416 2417 2418 2419 2420 2421 2422 2423 2424 2425 2426 2427 2428 2429 2430 2431 2432 2433 2434 2435 2436 2437 2438 2439 2440 2441 2442 2443 2444 2445 2446 2447 2448 2449 2450 2451 2452 2453 2454 2455 2456 2457 2458 2459 2460 2461 2462 2463 2464 2465 2466 2467 2468 | @set PATH=$(LIBTCLPATH);$(PATH) .\testfixture.exe $(TOP)\test\loadext.test $(TESTOPTS) coretestprogs: $(TESTPROGS) testprogs: coretestprogs srcck1.exe fuzzcheck.exe sessionfuzz.exe fulltest: alltest fuzztest alltest: $(TESTPROGS) @set PATH=$(LIBTCLPATH);$(PATH) .\testfixture.exe $(TOP)\test\all.test $(TESTOPTS) soaktest: $(TESTPROGS) @set PATH=$(LIBTCLPATH);$(PATH) .\testfixture.exe $(TOP)\test\all.test -soak=1 $(TESTOPTS) fulltestonly: $(TESTPROGS) fuzztest @set PATH=$(LIBTCLPATH);$(PATH) .\testfixture.exe $(TOP)\test\full.test queryplantest: testfixture.exe shell @set PATH=$(LIBTCLPATH);$(PATH) .\testfixture.exe $(TOP)\test\permutations.test queryplanner $(TESTOPTS) fuzztest: fuzzcheck.exe .\fuzzcheck.exe $(FUZZDATA) # Minimal testing that runs in less than 3 minutes (on a fast machine) # quicktest: testfixture.exe sourcetest @set PATH=$(LIBTCLPATH);$(PATH) .\testfixture.exe $(TOP)\test\extraquick.test $(TESTOPTS) # This is the common case. Run many tests that do not take too long, # including fuzzcheck, sqlite3_analyzer, and sqldiff tests. # test: $(TESTPROGS) sourcetest fuzztest tcltest # The veryquick.test TCL tests. # tcltest: testfixture.exe @set PATH=$(LIBTCLPATH);$(PATH) .\testfixture.exe $(TOP)\test\veryquick.test $(TESTOPTS) # Runs all the same tests cases as the "tcltest" target but uses # the testrunner.tcl script to run them in multiple cores # concurrently. testrunner: testfixture.exe .\testfixture.exe $(TOP)\test\testrunner.tcl smoketest: $(TESTPROGS) @set PATH=$(LIBTCLPATH);$(PATH) .\testfixture.exe $(TOP)\test\main.test $(TESTOPTS) shelltest: $(TESTPROGS) .\testfixture.exe $(TOP)\test\permutations.test shell sqlite3_analyzer.c: $(SQLITE3C) $(SQLITE3H) $(TOP)\src\tclsqlite.c $(TOP)\tool\spaceanal.tcl $(TOP)\tool\mkccode.tcl $(TOP)\tool\sqlite3_analyzer.c.in $(SQLITE_TCL_DEP) $(TCLSH_CMD) $(TOP)\tool\mkccode.tcl $(TOP)\tool\sqlite3_analyzer.c.in > $@ sqlite3_analyzer.exe: sqlite3_analyzer.c $(LIBRESOBJS) $(LTLINK) $(NO_WARN) -DBUILD_sqlite -I$(TCLINCDIR) sqlite3_analyzer.c \ /link $(LDFLAGS) $(LTLINKOPTS) $(TCLLIBPATHS) $(LTLIBPATHS) $(LIBRESOBJS) $(TCLLIBS) $(LTLIBS) $(TLIBS) |
︙ | ︙ | |||
2489 2490 2491 2492 2493 2494 2495 2496 2497 2498 2499 2500 2501 2502 2503 2504 2505 2506 2507 2508 2509 2510 2511 2512 2513 2514 2515 2516 2517 2518 2519 2520 2521 2522 2523 2524 2525 2526 2527 2528 2529 2530 | $(LTLINK) $(NO_WARN) $(KV_COMPILE_OPTS) \ $(TOP)\test\kvtest.c $(SQLITE3C) /link $(LDFLAGS) $(LTLINKOPTS) rbu.exe: $(TOP)\ext\rbu\rbu.c $(TOP)\ext\rbu\sqlite3rbu.c $(SQLITE3C) $(SQLITE3H) $(LTLINK) $(NO_WARN) -DSQLITE_ENABLE_RBU \ $(TOP)\ext\rbu\rbu.c $(SQLITE3C) /link $(LDFLAGS) $(LTLINKOPTS) LSMDIR=$(TOP)\ext\lsm1 !INCLUDE $(LSMDIR)\Makefile.msc moreclean: clean del /Q $(SQLITE3C) $(SQLITE3H) 2>NUL # <</mark>> clean: del /Q *.exp *.lo *.ilk *.lib *.obj *.ncb *.pdb *.sdf *.suo 2>NUL del /Q *.bsc *.def *.cod *.da *.bb *.bbg *.vc gmon.out 2>NUL del /Q $(SQLITE3EXE) $(SQLITE3DLL) Replace.exe 2>NUL # <<mark>> del /Q opcodes.c opcodes.h 2>NUL del /Q lemon.* lempar.c parse.* 2>NUL del /Q mksourceid.* mkkeywordhash.* keywordhash.h 2>NUL del /Q notasharedlib.* 2>NUL -rmdir /Q/S .deps 2>NUL -rmdir /Q/S .libs 2>NUL -rmdir /Q/S tsrc 2>NUL del /Q .target_source 2>NUL del /Q tclsqlite3.exe $(SQLITETCLH) $(SQLITETCLDECLSH) 2>NUL del /Q lsm.dll lsmtest.exe 2>NUL del /Q testloadext.dll 2>NUL del /Q testfixture.exe test.db 2>NUL del /Q LogEst.exe fts3view.exe rollback-test.exe showdb.exe dbdump.exe 2>NUL del /Q changeset.exe 2>NUL del /Q showjournal.exe showstat4.exe showwal.exe speedtest1.exe 2>NUL del /Q mptester.exe wordcount.exe rbu.exe srcck1.exe 2>NUL del /Q sqlite3.c sqlite3-*.c sqlite3.h 2>NUL del /Q sqlite3rc.h 2>NUL del /Q shell.c sqlite3ext.h sqlite3session.h 2>NUL del /Q sqlite3_analyzer.exe sqlite3_analyzer.c 2>NUL del /Q sqlite-*-output.vsix 2>NUL del /Q fuzzershell.exe fuzzcheck.exe sqldiff.exe dbhash.exe 2>NUL del /Q sqltclsh.* 2>NUL | > > > > > > > > > > > > > | | 2566 2567 2568 2569 2570 2571 2572 2573 2574 2575 2576 2577 2578 2579 2580 2581 2582 2583 2584 2585 2586 2587 2588 2589 2590 2591 2592 2593 2594 2595 2596 2597 2598 2599 2600 2601 2602 2603 2604 2605 2606 2607 2608 2609 2610 2611 2612 2613 2614 2615 2616 2617 2618 2619 2620 2621 2622 2623 2624 2625 2626 | $(LTLINK) $(NO_WARN) $(KV_COMPILE_OPTS) \ $(TOP)\test\kvtest.c $(SQLITE3C) /link $(LDFLAGS) $(LTLINKOPTS) rbu.exe: $(TOP)\ext\rbu\rbu.c $(TOP)\ext\rbu\sqlite3rbu.c $(SQLITE3C) $(SQLITE3H) $(LTLINK) $(NO_WARN) -DSQLITE_ENABLE_RBU \ $(TOP)\ext\rbu\rbu.c $(SQLITE3C) /link $(LDFLAGS) $(LTLINKOPTS) THREADTEST3_SRC = \ $(TOP)\test\threadtest3.c \ $(TOP)\test\tt3_checkpoint.c \ $(TOP)\test\tt3_index.c \ $(TOP)\test\tt3_vacuum.c \ $(TOP)\test\tt3_stress.c \ $(TOP)\test\tt3_lookaside1.c threadtest3.exe: $(THREADTEST3_SRC) $(TOP)\src\test_multiplex.c $(SQLITE3C) $(SQLITE3H) $(LTLINK) $(NO_WARN) $(TOP)\test\threadtest3.c $(TOP)\src\test_multiplex.c $(SQLITE3C) /link $(LDFLAGS) $(LTLINKOPTS) LSMDIR=$(TOP)\ext\lsm1 !INCLUDE $(LSMDIR)\Makefile.msc moreclean: clean del /Q $(SQLITE3C) $(SQLITE3H) 2>NUL # <</mark>> clean: del /Q *.exp *.lo *.ilk *.lib *.obj *.ncb *.pdb *.sdf *.suo 2>NUL del /Q *.bsc *.def *.cod *.da *.bb *.bbg *.vc gmon.out 2>NUL del /Q $(SQLITE3EXE) $(SQLITE3DLL) Replace.exe 2>NUL # <<mark>> del /Q $(SQLITE3TCLDLL) pkgIndex.tcl 2>NUL del /Q opcodes.c opcodes.h 2>NUL del /Q lemon.* lempar.c parse.* 2>NUL del /Q mksourceid.* mkkeywordhash.* keywordhash.h 2>NUL del /Q notasharedlib.* 2>NUL -rmdir /Q/S .deps 2>NUL -rmdir /Q/S .libs 2>NUL -rmdir /Q/S tsrc 2>NUL del /Q .target_source 2>NUL del /Q tclsqlite3.exe $(SQLITETCLH) $(SQLITETCLDECLSH) 2>NUL del /Q lsm.dll lsmtest.exe 2>NUL del /Q atrc.exe changesetfuzz.exe dbtotxt.exe index_usage.exe 2>NUL del /Q testloadext.dll 2>NUL del /Q testfixture.exe test.db 2>NUL del /Q LogEst.exe fts3view.exe rollback-test.exe showdb.exe dbdump.exe 2>NUL del /Q changeset.exe 2>NUL del /Q showjournal.exe showstat4.exe showwal.exe speedtest1.exe 2>NUL del /Q mptester.exe wordcount.exe rbu.exe srcck1.exe 2>NUL del /Q sqlite3.c sqlite3-*.c sqlite3.h 2>NUL del /Q sqlite3rc.h 2>NUL del /Q shell.c sqlite3ext.h sqlite3session.h 2>NUL del /Q sqlite3_analyzer.exe sqlite3_analyzer.c 2>NUL del /Q sqlite-*-output.vsix 2>NUL del /Q fuzzershell.exe fuzzcheck.exe sqldiff.exe dbhash.exe 2>NUL del /Q sqltclsh.* 2>NUL del /Q dbfuzz.exe sessionfuzz.exe threadtest3.exe 2>NUL del /Q kvtest.exe ossshell.exe scrub.exe 2>NUL del /Q showshm.exe sqlite3_checker.* sqlite3_expert.exe 2>NUL del /Q fts5.* fts5parse.* 2>NUL del /Q lsm.h lsm1.c 2>NUL # <</mark>> |
Changes to README.md.
1 2 | <h1 align="center">SQLite Source Repository</h1> | | | > | > > > > > | > > > > | > > | > > > < < < | | | | | | | | | | | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 | <h1 align="center">SQLite Source Repository</h1> This repository contains the complete source code for the [SQLite database engine](https://sqlite.org/). Some test scripts are also included. However, many other test scripts and most of the documentation are managed separately. ## Version Control SQLite sources are managed using the [Fossil](https://www.fossil-scm.org/), a distributed version control system that was specifically designed and written to support SQLite development. The [Fossil repository](https://sqlite.org/src/timeline) contains the urtext. If you are reading this on GitHub or some other Git repository or service, then you are looking at a mirror. The names of check-ins and other artifacts in a Git mirror are different from the official names for those objects. The official names for check-ins are found in a footer on the check-in comment for authorized mirrors. The official check-in name can also be seen in the `manifest.uuid` file in the root of the tree. Always use the official name, not the Git-name, when communicating about an SQLite check-in. If you pulled your SQLite source code from a secondary source and want to verify its integrity, there are hints on how to do that in the [Verifying Code Authenticity](#vauth) section below. ## Obtaining The Code If you do not want to use Fossil, you can download tarballs or ZIP archives or [SQLite archives](https://sqlite.org/cli.html#sqlar) as follows: * Latest trunk check-in as [Tarball](https://www.sqlite.org/src/tarball/sqlite.tar.gz), [ZIP-archive](https://www.sqlite.org/src/zip/sqlite.zip), or [SQLite-archive](https://www.sqlite.org/src/sqlar/sqlite.sqlar). * Latest release as [Tarball](https://www.sqlite.org/src/tarball/sqlite.tar.gz?r=release), [ZIP-archive](https://www.sqlite.org/src/zip/sqlite.zip?r=release), or [SQLite-archive](https://www.sqlite.org/src/sqlar/sqlite.sqlar?r=release). * For other check-ins, substitute an appropriate branch name or tag or hash prefix in place of "release" in the URLs of the previous bullet. Or browse the [timeline](https://www.sqlite.org/src/timeline) to locate the check-in desired, click on its information page link, then click on the "Tarball" or "ZIP Archive" links on the information page. If you do want to use Fossil to check out the source tree, first install Fossil version 2.0 or later. (Source tarballs and precompiled binaries available [here](https://www.fossil-scm.org/fossil/uv/download.html). Fossil is a stand-alone program. To install, simply download or build the single executable file and put that file someplace on your $PATH.) Then run commands like this: mkdir -p ~/sqlite ~/Fossils cd ~/sqlite fossil clone https://www.sqlite.org/src ~/Fossils/sqlite.fossil fossil open ~/Fossils/sqlite.fossil After setting up a repository using the steps above, you can always update to the latest version using: fossil update trunk ;# latest trunk check-in fossil update release ;# latest official release Or type "fossil ui" to get a web-based user interface. ## Compiling for Unix-like systems First create a directory in which to place the build products. It is recommended, but not required, that the build directory be separate from the source directory. Cd into the build directory and then from the build directory run the configure script found at the root of the source tree. Then run "make". |
︙ | ︙ | |||
78 79 80 81 82 83 84 | The configure script uses autoconf 2.61 and libtool. If the configure script does not work out for you, there is a generic makefile named "Makefile.linux-gcc" in the top directory of the source tree that you can copy and edit to suit your needs. Comments on the generic makefile show what changes are needed. | | | | | | | | | 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 | The configure script uses autoconf 2.61 and libtool. If the configure script does not work out for you, there is a generic makefile named "Makefile.linux-gcc" in the top directory of the source tree that you can copy and edit to suit your needs. Comments on the generic makefile show what changes are needed. ## Using MSVC for Windows systems On Windows, all applicable build products can be compiled with MSVC. First open the command prompt window associated with the desired compiler version (e.g. "Developer Command Prompt for VS2013"). Next, use NMAKE with the provided "Makefile.msc" to build one of the supported targets. For example, from the parent directory of the source subtree named "sqlite": mkdir bld cd bld nmake /f ..\sqlite\Makefile.msc TOP=..\sqlite nmake /f ..\sqlite\Makefile.msc sqlite3.c TOP=..\sqlite nmake /f ..\sqlite\Makefile.msc sqlite3.dll TOP=..\sqlite nmake /f ..\sqlite\Makefile.msc sqlite3.exe TOP=..\sqlite nmake /f ..\sqlite\Makefile.msc test TOP=..\sqlite There are several build options that can be set via the NMAKE command line. For example, to build for WinRT, simply add "FOR_WINRT=1" argument to the "sqlite3.dll" command line above. When debugging into the SQLite code, adding the "DEBUG=1" argument to one of the above command lines is recommended. |
︙ | ︙ | |||
120 121 122 123 124 125 126 | [command-line shell](https://sqlite.org/cli.html) and the "tclsqlite.c" file which implements the [Tcl bindings](https://sqlite.org/tclsqlite.html) for SQLite. (Historical note: SQLite began as a Tcl extension and only later escaped to the wild as an independent library.) Test scripts and programs are found in the **test/** subdirectory. | | | 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 | [command-line shell](https://sqlite.org/cli.html) and the "tclsqlite.c" file which implements the [Tcl bindings](https://sqlite.org/tclsqlite.html) for SQLite. (Historical note: SQLite began as a Tcl extension and only later escaped to the wild as an independent library.) Test scripts and programs are found in the **test/** subdirectory. Additional test code is found in other source repositories. See [How SQLite Is Tested](http://www.sqlite.org/testing.html) for additional information. The **ext/** subdirectory contains code for extensions. The Full-text search engine is in **ext/fts3**. The R-Tree engine is in **ext/rtree**. The **ext/misc** subdirectory contains a number of smaller, single-file extensions, such as a REGEXP operator. |
︙ | ︙ | |||
154 155 156 157 158 159 160 | and is used to generate the SQLITE\_SOURCE\_ID macro. The VERSION file contains the current SQLite version number. The sqlite3.h header is really just a copy of src/sqlite.h.in with the source-id and version number inserted at just the right spots. Note that comment text in the sqlite3.h file is used to generate much of the SQLite API documentation. The Tcl scripts used to generate that documentation are in a separate source repository. | | < | < < | | 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 | and is used to generate the SQLITE\_SOURCE\_ID macro. The VERSION file contains the current SQLite version number. The sqlite3.h header is really just a copy of src/sqlite.h.in with the source-id and version number inserted at just the right spots. Note that comment text in the sqlite3.h file is used to generate much of the SQLite API documentation. The Tcl scripts used to generate that documentation are in a separate source repository. The SQL language parser is **parse.c** which is generated from a grammar in the src/parse.y file. The conversion of "parse.y" into "parse.c" is done by the [lemon](./doc/lemon.html) LALR(1) parser generator. The source code for lemon is at tool/lemon.c. Lemon uses the tool/lempar.c file as a template for generating its parser. Lemon also generates the **parse.h** header file, at the same time it generates parse.c. The **opcodes.h** header file contains macros that define the numbers corresponding to opcodes in the "VDBE" virtual machine. The opcodes.h file is generated by scanning the src/vdbe.c source file. The Tcl script at ./mkopcodeh.tcl does this scan and generates opcodes.h. A second Tcl script, ./mkopcodec.tcl, then scans opcodes.h to generate the **opcodes.c** source file, which contains a reverse mapping from opcode-number to opcode-name that is used for EXPLAIN output. The **keywordhash.h** header file contains the definition of a hash table that maps SQL language keywords (ex: "CREATE", "SELECT", "INDEX", etc.) into |
︙ | ︙ | |||
224 225 226 227 228 229 230 | (helping to understand how SQLite works include the [file format](http://www.sqlite.org/fileformat2.html) description, the [virtual machine](http://www.sqlite.org/opcode.html) that runs prepared statements, the description of [how transactions work](http://www.sqlite.org/atomiccommit.html), and the [overview of the query planner](http://www.sqlite.org/optoverview.html). | | | 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 | (helping to understand how SQLite works include the [file format](http://www.sqlite.org/fileformat2.html) description, the [virtual machine](http://www.sqlite.org/opcode.html) that runs prepared statements, the description of [how transactions work](http://www.sqlite.org/atomiccommit.html), and the [overview of the query planner](http://www.sqlite.org/optoverview.html). Years of effort have gone into optimizing SQLite, both for small size and high performance. And optimizations tend to result in complex code. So there is a lot of complexity in the current SQLite implementation. It will not be the easiest library in the world to hack. Key files: * **sqlite.h.in** - This file defines the public interface to the SQLite |
︙ | ︙ | |||
281 282 283 284 285 286 287 | is not part of the core SQLite library. But as most of the tests in this repository are written in Tcl, the Tcl language bindings are important. * **test*.c** - Files in the src/ folder that begin with "test" go into building the "testfixture.exe" program. The testfixture.exe program is an enhanced Tcl shell. The testfixture.exe program runs scripts in the test/ folder to validate the core SQLite code. The testfixture program | | | > > > > > > > > > > > > > > > > > | | 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 | is not part of the core SQLite library. But as most of the tests in this repository are written in Tcl, the Tcl language bindings are important. * **test*.c** - Files in the src/ folder that begin with "test" go into building the "testfixture.exe" program. The testfixture.exe program is an enhanced Tcl shell. The testfixture.exe program runs scripts in the test/ folder to validate the core SQLite code. The testfixture program (and some other test programs too) is built and run when you type "make test". * **ext/misc/json1.c** - This file implements the various JSON functions that are built into SQLite. There are many other source files. Each has a succinct header comment that describes its purpose and role within the larger system. <a name="vauth"></a> ## Verifying Code Authenticity The `manifest` file at the root directory of the source tree contains either a SHA3-256 hash (for newer files) or a SHA1 hash (for older files) for every source file in the repository. The name of the version of the entire source tree is just the SHA3-256 hash of the `manifest` file itself, possibly with the last line of that file omitted if the last line begins with "`# Remove this line`". The `manifest.uuid` file should contain the SHA3-256 hash of the `manifest` file. If all of the above hash comparisons are correct, then you can be confident that your source tree is authentic and unadulterated. The format of the `manifest` file should be mostly self-explanatory, but if you want details, they are available [here](https://fossil-scm.org/fossil/doc/trunk/www/fileformat.wiki#manifest). ## Contacts The main SQLite website is [http://www.sqlite.org/](http://www.sqlite.org/) with geographically distributed backups at [http://www2.sqlite.org/](http://www2.sqlite.org) and [http://www3.sqlite.org/](http://www3.sqlite.org). |
Changes to VERSION.
|
| | | 1 | 3.41.0 |
Changes to autoconf/Makefile.am.
︙ | ︙ | |||
9 10 11 12 13 14 15 | EXTRA_sqlite3_SOURCES = sqlite3.c sqlite3_LDADD = @EXTRA_SHELL_OBJ@ @READLINE_LIBS@ sqlite3_DEPENDENCIES = @EXTRA_SHELL_OBJ@ sqlite3_CFLAGS = $(AM_CFLAGS) -DSQLITE_ENABLE_EXPLAIN_COMMENTS -DSQLITE_ENABLE_DBPAGE_VTAB -DSQLITE_ENABLE_STMTVTAB -DSQLITE_ENABLE_DBSTAT_VTAB $(SHELL_CFLAGS) include_HEADERS = sqlite3.h sqlite3ext.h | | | 9 10 11 12 13 14 15 16 17 18 19 20 | EXTRA_sqlite3_SOURCES = sqlite3.c sqlite3_LDADD = @EXTRA_SHELL_OBJ@ @READLINE_LIBS@ sqlite3_DEPENDENCIES = @EXTRA_SHELL_OBJ@ sqlite3_CFLAGS = $(AM_CFLAGS) -DSQLITE_ENABLE_EXPLAIN_COMMENTS -DSQLITE_ENABLE_DBPAGE_VTAB -DSQLITE_ENABLE_STMTVTAB -DSQLITE_ENABLE_DBSTAT_VTAB $(SHELL_CFLAGS) include_HEADERS = sqlite3.h sqlite3ext.h EXTRA_DIST = sqlite3.1 tea Makefile.msc sqlite3.rc sqlite3rc.h README.txt Replace.cs Makefile.fallback pkgconfigdir = ${libdir}/pkgconfig pkgconfig_DATA = sqlite3.pc man_MANS = sqlite3.1 |
Changes to autoconf/Makefile.msc.
︙ | ︙ | |||
69 70 71 72 73 74 75 | # If necessary, create a list of harmless compiler warnings to disable when # compiling the various tools. For the SQLite source code itself, warnings, # if any, will be disabled from within it. # !IFNDEF NO_WARN !IF $(USE_FULLWARN)!=0 NO_WARN = -wd4054 -wd4055 -wd4100 -wd4127 -wd4130 -wd4152 -wd4189 -wd4206 | | | 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 | # If necessary, create a list of harmless compiler warnings to disable when # compiling the various tools. For the SQLite source code itself, warnings, # if any, will be disabled from within it. # !IFNDEF NO_WARN !IF $(USE_FULLWARN)!=0 NO_WARN = -wd4054 -wd4055 -wd4100 -wd4127 -wd4130 -wd4152 -wd4189 -wd4206 NO_WARN = $(NO_WARN) -wd4210 -wd4232 -wd4244 -wd4305 -wd4306 -wd4702 -wd4706 !ENDIF !ENDIF # Set this non-0 to use the library paths and other options necessary for # Windows Phone 8.1. # !IFNDEF USE_WP81_OPTS |
︙ | ︙ | |||
191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 | # 4 == SQLITE_WIN32_MALLOC_VALIDATE: Validate the Win32 native heap per call. # 5 == SQLITE_DEBUG_OS_TRACE: Enables output from the OSTRACE() macros. # 6 == SQLITE_ENABLE_IOTRACE: Enables output from the IOTRACE() macros. # !IFNDEF DEBUG DEBUG = 0 !ENDIF # Enable use of available compiler optimizations? Normally, this should be # non-zero. Setting this to zero, thus disabling all compiler optimizations, # can be useful for testing. # !IFNDEF OPTIMIZATIONS OPTIMIZATIONS = 2 !ENDIF # Set this to non-0 to enable support for the session extension. # !IFNDEF SESSION SESSION = 0 !ENDIF # Set the source code file to be used by executables and libraries when # they need the amalgamation. # !IFNDEF SQLITE3C !IF $(SPLIT_AMALGAMATION)!=0 SQLITE3C = sqlite3-all.c | > > > > > > > | 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 | # 4 == SQLITE_WIN32_MALLOC_VALIDATE: Validate the Win32 native heap per call. # 5 == SQLITE_DEBUG_OS_TRACE: Enables output from the OSTRACE() macros. # 6 == SQLITE_ENABLE_IOTRACE: Enables output from the IOTRACE() macros. # !IFNDEF DEBUG DEBUG = 0 !ENDIF # Enable use of available compiler optimizations? Normally, this should be # non-zero. Setting this to zero, thus disabling all compiler optimizations, # can be useful for testing. # !IFNDEF OPTIMIZATIONS OPTIMIZATIONS = 2 !ENDIF # Set this to non-0 to enable support for the session extension. # !IFNDEF SESSION SESSION = 0 !ENDIF # Set this to non-0 to enable support for the rbu extension. # !IFNDEF RBU RBU = 0 !ENDIF # Set the source code file to be used by executables and libraries when # they need the amalgamation. # !IFNDEF SQLITE3C !IF $(SPLIT_AMALGAMATION)!=0 SQLITE3C = sqlite3-all.c |
︙ | ︙ | |||
274 275 276 277 278 279 280 | # the Windows platform. # !IFNDEF OPT_FEATURE_FLAGS !IF $(MINIMAL_AMALGAMATION)==0 OPT_FEATURE_FLAGS = $(OPT_FEATURE_FLAGS) -DSQLITE_ENABLE_FTS3=1 OPT_FEATURE_FLAGS = $(OPT_FEATURE_FLAGS) -DSQLITE_ENABLE_RTREE=1 OPT_FEATURE_FLAGS = $(OPT_FEATURE_FLAGS) -DSQLITE_ENABLE_GEOPOLY=1 | < | < > > > > > > > > > > | 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 | # the Windows platform. # !IFNDEF OPT_FEATURE_FLAGS !IF $(MINIMAL_AMALGAMATION)==0 OPT_FEATURE_FLAGS = $(OPT_FEATURE_FLAGS) -DSQLITE_ENABLE_FTS3=1 OPT_FEATURE_FLAGS = $(OPT_FEATURE_FLAGS) -DSQLITE_ENABLE_RTREE=1 OPT_FEATURE_FLAGS = $(OPT_FEATURE_FLAGS) -DSQLITE_ENABLE_GEOPOLY=1 OPT_FEATURE_FLAGS = $(OPT_FEATURE_FLAGS) -DSQLITE_ENABLE_STMTVTAB=1 OPT_FEATURE_FLAGS = $(OPT_FEATURE_FLAGS) -DSQLITE_ENABLE_DBPAGE_VTAB=1 OPT_FEATURE_FLAGS = $(OPT_FEATURE_FLAGS) -DSQLITE_ENABLE_DBSTAT_VTAB=1 OPT_FEATURE_FLAGS = $(OPT_FEATURE_FLAGS) -DSQLITE_ENABLE_BYTECODE_VTAB=1 !ENDIF OPT_FEATURE_FLAGS = $(OPT_FEATURE_FLAGS) -DSQLITE_ENABLE_COLUMN_METADATA=1 !ENDIF # Should the session extension be enabled? If so, add compilation options # to enable it. # !IF $(SESSION)!=0 OPT_FEATURE_FLAGS = $(OPT_FEATURE_FLAGS) -DSQLITE_ENABLE_SESSION=1 OPT_FEATURE_FLAGS = $(OPT_FEATURE_FLAGS) -DSQLITE_ENABLE_PREUPDATE_HOOK=1 !ENDIF # Always enable math functions on Windows OPT_FEATURE_FLAGS = $(OPT_FEATURE_FLAGS) -DSQLITE_ENABLE_MATH_FUNCTIONS # Should the rbu extension be enabled? If so, add compilation options # to enable it. # !IF $(RBU)!=0 OPT_FEATURE_FLAGS = $(OPT_FEATURE_FLAGS) -DSQLITE_ENABLE_RBU=1 !ENDIF # These are the "extended" SQLite compilation options used when compiling for # the Windows 10 platform. # !IFNDEF EXT_FEATURE_FLAGS !IF $(FOR_WIN10)!=0 EXT_FEATURE_FLAGS = $(EXT_FEATURE_FLAGS) -DSQLITE_ENABLE_FTS4=1 |
︙ | ︙ | |||
429 430 431 432 433 434 435 | UCRTLIBPATH = $(UCRTLIBPATH:\\=\) # C compiler and options for use in building executables that # will run on the platform that is doing the build. # !IF $(USE_FULLWARN)!=0 | | | | 444 445 446 447 448 449 450 451 452 453 454 455 456 457 458 459 460 | UCRTLIBPATH = $(UCRTLIBPATH:\\=\) # C compiler and options for use in building executables that # will run on the platform that is doing the build. # !IF $(USE_FULLWARN)!=0 BCC = $(NCC) -nologo -W4 -Fd$*.pdb $(CCOPTS) $(BCCOPTS) !ELSE BCC = $(NCC) -nologo -W3 -Fd$*.pdb $(CCOPTS) $(BCCOPTS) !ENDIF # Check if assembly code listings should be generated for the source # code files to be compiled. # !IF $(USE_LISTINGS)!=0 BCC = $(BCC) -FAcs |
︙ | ︙ | |||
482 483 484 485 486 487 488 | # also be noted here that building any target with these "stdcall" options # will most likely fail if the Tcl library is also required. This is due # to how the Tcl library functions are declared and exported (i.e. without # an explicit calling convention, which results in "cdecl"). # !IF $(USE_STDCALL)!=0 || $(FOR_WIN10)!=0 !IF "$(PLATFORM)"=="x86" | | | | | | 497 498 499 500 501 502 503 504 505 506 507 508 509 510 511 512 513 514 515 516 | # also be noted here that building any target with these "stdcall" options # will most likely fail if the Tcl library is also required. This is due # to how the Tcl library functions are declared and exported (i.e. without # an explicit calling convention, which results in "cdecl"). # !IF $(USE_STDCALL)!=0 || $(FOR_WIN10)!=0 !IF "$(PLATFORM)"=="x86" CORE_CCONV_OPTS = -Gz -guard:cf -DSQLITE_CDECL=__cdecl -DSQLITE_APICALL=__stdcall -DSQLITE_CALLBACK=__stdcall -DSQLITE_SYSAPI=__stdcall SHELL_CCONV_OPTS = -Gz -guard:cf -DSQLITE_CDECL=__cdecl -DSQLITE_APICALL=__stdcall -DSQLITE_CALLBACK=__stdcall -DSQLITE_SYSAPI=__stdcall !ELSE !IFNDEF PLATFORM CORE_CCONV_OPTS = -Gz -guard:cf -DSQLITE_CDECL=__cdecl -DSQLITE_APICALL=__stdcall -DSQLITE_CALLBACK=__stdcall -DSQLITE_SYSAPI=__stdcall SHELL_CCONV_OPTS = -Gz -guard:cf -DSQLITE_CDECL=__cdecl -DSQLITE_APICALL=__stdcall -DSQLITE_CALLBACK=__stdcall -DSQLITE_SYSAPI=__stdcall !ELSE CORE_CCONV_OPTS = SHELL_CCONV_OPTS = !ENDIF !ENDIF !ELSE CORE_CCONV_OPTS = |
︙ | ︙ | |||
804 805 806 807 808 809 810 | BCC = $(BCC) -Zi !ENDIF # Command line prefixes for compiling code, compiling resources, # linking, etc. # | | > > > > > | 819 820 821 822 823 824 825 826 827 828 829 830 831 832 833 834 835 836 837 838 839 840 841 842 843 844 845 846 847 848 849 850 851 852 853 854 855 | BCC = $(BCC) -Zi !ENDIF # Command line prefixes for compiling code, compiling resources, # linking, etc. # LTCOMPILE = $(TCC) -Fo$@ -Fd$*.pdb LTRCOMPILE = $(RCC) -r LTLIB = lib.exe LTLINK = $(TCC) -Fe$@ # If requested, link to the RPCRT4 library. # !IF $(USE_RPCRT4_LIB)!=0 LTLIBS = $(LTLIBS) rpcrt4.lib !ENDIF # If a platform was set, force the linker to target that. # Note that the vcvars*.bat family of batch files typically # set this for you. Otherwise, the linker will attempt # to deduce the binary type based on the object files. !IFDEF PLATFORM LTLINKOPTS = /NOLOGO /MACHINE:$(PLATFORM) LTLIBOPTS = /NOLOGO /MACHINE:$(PLATFORM) !ELSEIF "$(VISUALSTUDIOVERSION)"=="12.0" || \ "$(VISUALSTUDIOVERSION)"=="14.0" || \ "$(VISUALSTUDIOVERSION)"=="15.0" LTLINKOPTS = /NOLOGO /MACHINE:x86 LTLIBOPTS = /NOLOGO /MACHINE:x86 !ELSE LTLINKOPTS = /NOLOGO LTLIBOPTS = /NOLOGO !ENDIF # When compiling for use in the WinRT environment, the following # linker option must be used to mark the executable as runnable |
︙ | ︙ | |||
934 935 936 937 938 939 940 | # Additional compiler options for the shell. These are only effective # when the shell is not being dynamically linked. # !IF $(DYNAMIC_SHELL)==0 && $(FOR_WIN10)==0 SHELL_COMPILE_OPTS = $(SHELL_COMPILE_OPTS) -DSQLITE_ENABLE_FTS4=1 SHELL_COMPILE_OPTS = $(SHELL_COMPILE_OPTS) -DSQLITE_ENABLE_EXPLAIN_COMMENTS=1 SHELL_COMPILE_OPTS = $(SHELL_COMPILE_OPTS) -DSQLITE_ENABLE_OFFSET_SQL_FUNC=1 | < | 954 955 956 957 958 959 960 961 962 963 964 965 966 967 | # Additional compiler options for the shell. These are only effective # when the shell is not being dynamically linked. # !IF $(DYNAMIC_SHELL)==0 && $(FOR_WIN10)==0 SHELL_COMPILE_OPTS = $(SHELL_COMPILE_OPTS) -DSQLITE_ENABLE_FTS4=1 SHELL_COMPILE_OPTS = $(SHELL_COMPILE_OPTS) -DSQLITE_ENABLE_EXPLAIN_COMMENTS=1 SHELL_COMPILE_OPTS = $(SHELL_COMPILE_OPTS) -DSQLITE_ENABLE_OFFSET_SQL_FUNC=1 !ENDIF # This is the default Makefile target. The objects listed here # are what get build when you type just "make" with no arguments. # core: dll shell |
︙ | ︙ | |||
969 970 971 972 973 974 975 | Replace.exe: $(CSC) /target:exe $(TOP)\Replace.cs sqlite3.def: Replace.exe $(LIBOBJ) echo EXPORTS > sqlite3.def dumpbin /all $(LIBOBJ) \ | | | 988 989 990 991 992 993 994 995 996 997 998 999 1000 1001 1002 | Replace.exe: $(CSC) /target:exe $(TOP)\Replace.cs sqlite3.def: Replace.exe $(LIBOBJ) echo EXPORTS > sqlite3.def dumpbin /all $(LIBOBJ) \ | .\Replace.exe "^\s+/EXPORT:_?(sqlite3(?:session|changeset|changegroup|rebaser|rbu)?_[^@,]*)(?:@\d+|,DATA)?$$" $$1 true \ | sort >> sqlite3.def $(SQLITE3EXE): shell.c $(SHELL_CORE_DEP) $(LIBRESOBJS) $(SHELL_CORE_SRC) $(SQLITE3H) $(LTLINK) $(SHELL_COMPILE_OPTS) $(READLINE_FLAGS) shell.c $(SHELL_CORE_SRC) \ /link $(SQLITE3EXEPDB) $(LDFLAGS) $(LTLINKOPTS) $(SHELL_LINK_OPTS) $(LTLIBPATHS) $(LIBRESOBJS) $(LIBREADLINE) $(LTLIBS) $(TLIBS) |
︙ | ︙ |
Changes to autoconf/README.txt.
︙ | ︙ | |||
101 102 103 104 105 106 107 | Additionally, preprocessor defines may be specified by using the OPTS macro on the NMAKE command line. However, not all possible preprocessor defines may be specified in this manner as some require the amalgamation to be built with them enabled (see http://www.sqlite.org/compile.html). For example, the following will work: | | | 101 102 103 104 105 106 107 108 109 110 111 112 113 | Additionally, preprocessor defines may be specified by using the OPTS macro on the NMAKE command line. However, not all possible preprocessor defines may be specified in this manner as some require the amalgamation to be built with them enabled (see http://www.sqlite.org/compile.html). For example, the following will work: "OPTS=-DSQLITE_ENABLE_STAT4=1 -DSQLITE_OMIT_JSON=1" However, the following will not compile unless the amalgamation was built with it enabled: "OPTS=-DSQLITE_ENABLE_UPDATE_DELETE_LIMIT=1" |
Changes to autoconf/configure.ac.
︙ | ︙ | |||
83 84 85 86 87 88 89 | #----------------------------------------------------------------------- # --enable-threadsafe # AC_ARG_ENABLE(threadsafe, [AS_HELP_STRING( [--enable-threadsafe], [build a thread-safe library [default=yes]])], [], [enable_threadsafe=yes]) | | > > | 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 | #----------------------------------------------------------------------- # --enable-threadsafe # AC_ARG_ENABLE(threadsafe, [AS_HELP_STRING( [--enable-threadsafe], [build a thread-safe library [default=yes]])], [], [enable_threadsafe=yes]) if test x"$enable_threadsafe" == "xno"; then BUILD_CFLAGS="$BUILD_CFLAGS -DSQLITE_THREADSAFE=0" else BUILD_CFLAGS="$BUILD_CFLAGS -D_REENTRANT=1 -DSQLITE_THREADSAFE=1" AC_SEARCH_LIBS(pthread_create, pthread) AC_SEARCH_LIBS(pthread_mutexattr_init, pthread) fi #----------------------------------------------------------------------- #----------------------------------------------------------------------- |
︙ | ︙ | |||
105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 | else BUILD_CFLAGS="$BUILD_CFLAGS -DSQLITE_OMIT_LOAD_EXTENSION=1" fi AC_MSG_CHECKING([for whether to support dynamic extensions]) AC_MSG_RESULT($enable_dynamic_extensions) #----------------------------------------------------------------------- #----------------------------------------------------------------------- # --enable-fts4 # AC_ARG_ENABLE(fts4, [AS_HELP_STRING( [--enable-fts4], [include fts4 support [default=yes]])], [], [enable_fts4=yes]) if test x"$enable_fts4" = "xyes"; then BUILD_CFLAGS="$BUILD_CFLAGS -DSQLITE_ENABLE_FTS4" fi #----------------------------------------------------------------------- #----------------------------------------------------------------------- # --enable-fts3 # AC_ARG_ENABLE(fts3, [AS_HELP_STRING( [--enable-fts3], [include fts3 support [default=no]])], [], []) if test x"$enable_fts3" = "xyes" -a x"$enable_fts4" = "xno"; then BUILD_CFLAGS="$BUILD_CFLAGS -DSQLITE_ENABLE_FTS3" fi #----------------------------------------------------------------------- #----------------------------------------------------------------------- # --enable-fts5 # AC_ARG_ENABLE(fts5, [AS_HELP_STRING( [--enable-fts5], [include fts5 support [default=yes]])], [], [enable_fts5=yes]) if test x"$enable_fts5" = "xyes"; then AC_SEARCH_LIBS(log, m) BUILD_CFLAGS="$BUILD_CFLAGS -DSQLITE_ENABLE_FTS5" | > > > > > > > > > > > > > > > > > > > > > > > > > > < < | < | < < < < < < > | > > > > > > > > > > > | 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 | else BUILD_CFLAGS="$BUILD_CFLAGS -DSQLITE_OMIT_LOAD_EXTENSION=1" fi AC_MSG_CHECKING([for whether to support dynamic extensions]) AC_MSG_RESULT($enable_dynamic_extensions) #----------------------------------------------------------------------- #----------------------------------------------------------------------- # --enable-math # AC_ARG_ENABLE(math, [AS_HELP_STRING( [--enable-math], [SQL math functions [default=yes]])], [], [enable_math=yes]) AC_MSG_CHECKING([SQL math functions]) if test x"$enable_math" = "xyes"; then BUILD_CFLAGS="$BUILD_CFLAGS -DSQLITE_ENABLE_MATH_FUNCTIONS" AC_MSG_RESULT([enabled]) AC_SEARCH_LIBS(ceil, m) else AC_MSG_RESULT([disabled]) fi #----------------------------------------------------------------------- #----------------------------------------------------------------------- # --enable-fts4 # AC_ARG_ENABLE(fts4, [AS_HELP_STRING( [--enable-fts4], [include fts4 support [default=yes]])], [], [enable_fts4=yes]) AC_MSG_CHECKING([FTS4 extension]) if test x"$enable_fts4" = "xyes"; then BUILD_CFLAGS="$BUILD_CFLAGS -DSQLITE_ENABLE_FTS4" AC_MSG_RESULT([enabled]) else AC_MSG_RESULT([disabled]) fi #----------------------------------------------------------------------- #----------------------------------------------------------------------- # --enable-fts3 # AC_ARG_ENABLE(fts3, [AS_HELP_STRING( [--enable-fts3], [include fts3 support [default=no]])], [], []) AC_MSG_CHECKING([FTS3 extension]) if test x"$enable_fts3" = "xyes" -a x"$enable_fts4" = "xno"; then BUILD_CFLAGS="$BUILD_CFLAGS -DSQLITE_ENABLE_FTS3" AC_MSG_RESULT([enabled]) else AC_MSG_RESULT([disabled]) fi #----------------------------------------------------------------------- #----------------------------------------------------------------------- # --enable-fts5 # AC_ARG_ENABLE(fts5, [AS_HELP_STRING( [--enable-fts5], [include fts5 support [default=yes]])], [], [enable_fts5=yes]) AC_MSG_CHECKING([FTS5 extension]) if test x"$enable_fts5" = "xyes"; then AC_MSG_RESULT([enabled]) AC_SEARCH_LIBS(log, m) BUILD_CFLAGS="$BUILD_CFLAGS -DSQLITE_ENABLE_FTS5" else AC_MSG_RESULT([disabled]) fi #----------------------------------------------------------------------- #----------------------------------------------------------------------- # --enable-rtree # AC_ARG_ENABLE(rtree, [AS_HELP_STRING( [--enable-rtree], [include rtree support [default=yes]])], [], [enable_rtree=yes]) AC_MSG_CHECKING([RTREE extension]) if test x"$enable_rtree" = "xyes"; then BUILD_CFLAGS="$BUILD_CFLAGS -DSQLITE_ENABLE_RTREE -DSQLITE_ENABLE_GEOPOLY" AC_MSG_RESULT([enabled]) else AC_MSG_RESULT([disabled]) fi #----------------------------------------------------------------------- #----------------------------------------------------------------------- # --enable-session # AC_ARG_ENABLE(session, [AS_HELP_STRING( [--enable-session], [enable the session extension [default=no]])], [], []) AC_MSG_CHECKING([Session extension]) if test x"$enable_session" = "xyes"; then BUILD_CFLAGS="$BUILD_CFLAGS -DSQLITE_ENABLE_SESSION -DSQLITE_ENABLE_PREUPDATE_HOOK" AC_MSG_RESULT([enabled]) else AC_MSG_RESULT([disabled]) fi #----------------------------------------------------------------------- #----------------------------------------------------------------------- # --enable-debug # AC_ARG_ENABLE(debug, [AS_HELP_STRING( [--enable-debug], [build with debugging features enabled [default=no]])], [], []) AC_MSG_CHECKING([Build type]) if test x"$enable_debug" = "xyes"; then BUILD_CFLAGS="$BUILD_CFLAGS -DSQLITE_DEBUG -DSQLITE_ENABLE_SELECTTRACE -DSQLITE_ENABLE_WHERETRACE" CFLAGS="-g -O0" AC_MSG_RESULT([debug]) else AC_MSG_RESULT([release]) fi #----------------------------------------------------------------------- #----------------------------------------------------------------------- # --enable-static-shell # AC_ARG_ENABLE(static-shell, [AS_HELP_STRING( |
︙ | ︙ |
Changes to autoconf/tea/Makefile.in.
1 2 3 4 5 6 7 8 9 10 11 12 13 | # Makefile.in -- # # This file is a Makefile for Sample TEA Extension. If it has the name # "Makefile.in" then it is a template for a Makefile; to generate the # actual Makefile, run "./configure", which is a configuration script # generated by the "autoconf" program (constructs like "@foo@" will get # replaced in the actual Makefile. # # Copyright (c) 1999 Scriptics Corporation. # Copyright (c) 2002-2005 ActiveState Corporation. # # See the file "license.terms" for information on usage and redistribution # of this file, and for a DISCLAIMER OF ALL WARRANTIES. | < < | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 | # Makefile.in -- # # This file is a Makefile for Sample TEA Extension. If it has the name # "Makefile.in" then it is a template for a Makefile; to generate the # actual Makefile, run "./configure", which is a configuration script # generated by the "autoconf" program (constructs like "@foo@" will get # replaced in the actual Makefile. # # Copyright (c) 1999 Scriptics Corporation. # Copyright (c) 2002-2005 ActiveState Corporation. # # See the file "license.terms" for information on usage and redistribution # of this file, and for a DISCLAIMER OF ALL WARRANTIES. #======================================================================== # Add additional lines to handle any additional AC_SUBST cases that # have been added in a customized configure script. #======================================================================== #SAMPLE_NEW_VAR = @SAMPLE_NEW_VAR@ |
︙ | ︙ | |||
56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 | #======================================================================== # "PKG_LIB_FILE" refers to the library (dynamic or static as per # configuration options) composed of the named objects. #======================================================================== PKG_LIB_FILE = @PKG_LIB_FILE@ PKG_STUB_LIB_FILE = @PKG_STUB_LIB_FILE@ lib_BINARIES = $(PKG_LIB_FILE) BINARIES = $(lib_BINARIES) SHELL = @SHELL@ srcdir = @srcdir@ prefix = @prefix@ exec_prefix = @exec_prefix@ bindir = @bindir@ libdir = @libdir@ datarootdir = @datarootdir@ datadir = @datadir@ mandir = @mandir@ | > > > > < | > | | > > > < < < < < | | < | > | < | | > > > > | > > > > > > > > | < | 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 | #======================================================================== # "PKG_LIB_FILE" refers to the library (dynamic or static as per # configuration options) composed of the named objects. #======================================================================== PKG_LIB_FILE = @PKG_LIB_FILE@ PKG_LIB_FILE8 = @PKG_LIB_FILE8@ PKG_LIB_FILE9 = @PKG_LIB_FILE9@ PKG_STUB_LIB_FILE = @PKG_STUB_LIB_FILE@ lib_BINARIES = $(PKG_LIB_FILE) BINARIES = $(lib_BINARIES) SHELL = @SHELL@ srcdir = @srcdir@ prefix = @prefix@ exec_prefix = @exec_prefix@ bindir = @bindir@ libdir = @libdir@ includedir = @includedir@ datarootdir = @datarootdir@ runstatedir = @runstatedir@ datadir = @datadir@ mandir = @mandir@ DESTDIR = PKG_DIR = $(PACKAGE_NAME)$(PACKAGE_VERSION) pkgdatadir = $(datadir)/$(PKG_DIR) pkglibdir = $(libdir)/$(PKG_DIR) pkgincludedir = $(includedir)/$(PKG_DIR) top_builddir = @abs_top_builddir@ INSTALL_OPTIONS = INSTALL = @INSTALL@ $(INSTALL_OPTIONS) INSTALL_DATA_DIR = @INSTALL_DATA_DIR@ INSTALL_DATA = @INSTALL_DATA@ INSTALL_PROGRAM = @INSTALL_PROGRAM@ INSTALL_SCRIPT = @INSTALL_SCRIPT@ INSTALL_LIBRARY = @INSTALL_LIBRARY@ PACKAGE_NAME = @PACKAGE_NAME@ PACKAGE_VERSION = @PACKAGE_VERSION@ CC = @CC@ CCLD = @CCLD@ CFLAGS_DEFAULT = @CFLAGS_DEFAULT@ CFLAGS_WARNING = @CFLAGS_WARNING@ EXEEXT = @EXEEXT@ LDFLAGS_DEFAULT = @LDFLAGS_DEFAULT@ MAKE_LIB = @MAKE_LIB@ MAKE_STUB_LIB = @MAKE_STUB_LIB@ OBJEXT = @OBJEXT@ RANLIB = @RANLIB@ RANLIB_STUB = @RANLIB_STUB@ SHLIB_CFLAGS = @SHLIB_CFLAGS@ SHLIB_LD = @SHLIB_LD@ SHLIB_LD_LIBS = @SHLIB_LD_LIBS@ STLIB_LD = @STLIB_LD@ #TCL_DEFS = @TCL_DEFS@ TCL_BIN_DIR = @TCL_BIN_DIR@ TCL_SRC_DIR = @TCL_SRC_DIR@ #TK_BIN_DIR = @TK_BIN_DIR@ #TK_SRC_DIR = @TK_SRC_DIR@ # Not used, but retained for reference of what libs Tcl required #TCL_LIBS = @TCL_LIBS@ #======================================================================== # TCLLIBPATH seeds the auto_path in Tcl's init.tcl so we can test our # package without installing. The other environment variables allow us # to test against an uninstalled Tcl. Add special env vars that you # require for testing here (like TCLX_LIBRARY). #======================================================================== EXTRA_PATH = $(top_builddir):$(TCL_BIN_DIR) #EXTRA_PATH = $(top_builddir):$(TCL_BIN_DIR):$(TK_BIN_DIR) TCLLIBPATH = $(top_builddir) TCLSH_ENV = TCL_LIBRARY=`@CYGPATH@ $(TCL_SRC_DIR)/library` PKG_ENV = @LD_LIBRARY_PATH_VAR@="$(EXTRA_PATH):$(@LD_LIBRARY_PATH_VAR@)" \ PATH="$(EXTRA_PATH):$(PATH)" \ TCLLIBPATH="$(TCLLIBPATH)" TCLSH_PROG = @TCLSH_PROG@ TCLSH = $(TCLSH_ENV) $(PKG_ENV) $(TCLSH_PROG) #WISH_ENV = TK_LIBRARY=`@CYGPATH@ $(TK_SRC_DIR)/library` #WISH_PROG = @WISH_PROG@ #WISH = $(TCLSH_ENV) $(WISH_ENV) $(PKG_ENV) $(WISH_PROG) SHARED_BUILD = @SHARED_BUILD@ INCLUDES = @PKG_INCLUDES@ @TCL_INCLUDES@ -I. -I$(srcdir)/.. #INCLUDES = @PKG_INCLUDES@ @TCL_INCLUDES@ @TK_INCLUDES@ @TK_XINCLUDES@ PKG_CFLAGS = @PKG_CFLAGS@ # TCL_DEFS is not strictly need here, but if you remove it, then you # must make sure that configure.ac checks for the necessary components # that your library may use. TCL_DEFS can actually be a problem if # you do not compile with a similar machine setup as the Tcl core was # compiled with. #DEFS = $(TCL_DEFS) @DEFS@ $(PKG_CFLAGS) DEFS = @DEFS@ $(PKG_CFLAGS) # Move pkgIndex.tcl to 'BINARIES' var if it is generated in the Makefile CONFIG_CLEAN_FILES = Makefile pkgIndex.tcl CLEANFILES = @CLEANFILES@ CPPFLAGS = @CPPFLAGS@ LIBS = @PKG_LIBS@ @LIBS@ AR = @AR@ CFLAGS = @CFLAGS@ LDFLAGS = @LDFLAGS@ LDFLAGS_DEFAULT = @LDFLAGS_DEFAULT@ COMPILE = $(CC) $(DEFS) $(INCLUDES) $(AM_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) \ $(CFLAGS_DEFAULT) $(CFLAGS_WARNING) $(SHLIB_CFLAGS) $(CFLAGS) GDB = gdb VALGRIND = valgrind VALGRINDARGS = --tool=memcheck --num-callers=8 --leak-resolution=high \ --leak-check=yes --show-reachable=yes -v .SUFFIXES: .c .$(OBJEXT) #======================================================================== # Start of user-definable TARGETS section #======================================================================== #======================================================================== # TEA TARGETS. Please note that the "libraries:" target refers to platform # independent files, and the "binaries:" target includes executable programs and # platform-dependent libraries. Modify these targets so that they install # the various pieces of your package. The make and install rules # for the BINARIES that you specified above have already been done. #======================================================================== all: binaries libraries doc #======================================================================== # The binaries target builds executable programs, Windows .dll's, unix # shared/static libraries, and any other platform-dependent files. # The list of targets to build for "binaries:" is specified at the top # of the Makefile, in the "BINARIES" variable. #======================================================================== binaries: $(BINARIES) libraries: #======================================================================== # Your doc target should differentiate from doc builds (by the developer) # and doc installs (see install-doc), which just install the docs on the # end user machine when building from source. #======================================================================== |
︙ | ︙ | |||
212 213 214 215 216 217 218 | #======================================================================== # This rule installs platform-independent files, such as header files. # The list=...; for p in $$list handles the empty list case x-platform. #======================================================================== install-libraries: libraries | | | | | < > > > | > > > > > > > > > > > | 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 | #======================================================================== # This rule installs platform-independent files, such as header files. # The list=...; for p in $$list handles the empty list case x-platform. #======================================================================== install-libraries: libraries @$(INSTALL_DATA_DIR) "$(DESTDIR)$(includedir)" @echo "Installing header files in $(DESTDIR)$(includedir)" @list='$(PKG_HEADERS)'; for i in $$list; do \ echo "Installing $(srcdir)/$$i" ; \ $(INSTALL_DATA) $(srcdir)/$$i "$(DESTDIR)$(includedir)" ; \ done; #======================================================================== # Install documentation. Unix manpages should go in the $(mandir) # directory. #======================================================================== install-doc: doc @$(INSTALL_DATA_DIR) "$(DESTDIR)$(mandir)/mann" @echo "Installing documentation in $(DESTDIR)$(mandir)" @list='$(srcdir)/doc/*.n'; for i in $$list; do \ echo "Installing $$i"; \ $(INSTALL_DATA) $$i "$(DESTDIR)$(mandir)/mann" ; \ done test: binaries libraries @echo "SQLite TEA distribution does not include tests" shell: binaries libraries @$(TCLSH) $(SCRIPT) gdb: $(TCLSH_ENV) $(PKG_ENV) $(GDB) $(TCLSH_PROG) $(SCRIPT) gdb-test: binaries libraries $(TCLSH_ENV) $(PKG_ENV) $(GDB) \ --args $(TCLSH_PROG) `@CYGPATH@ $(srcdir)/tests/all.tcl` \ $(TESTFLAGS) -singleproc 1 \ -load "package ifneeded $(PACKAGE_NAME) $(PACKAGE_VERSION) \ [list load `@CYGPATH@ $(PKG_LIB_FILE)` [string totitle $(PACKAGE_NAME)]]" valgrind: binaries libraries $(TCLSH_ENV) $(PKG_ENV) $(VALGRIND) $(VALGRINDARGS) $(TCLSH_PROG) \ `@CYGPATH@ $(srcdir)/tests/all.tcl` $(TESTFLAGS) valgrindshell: binaries libraries $(TCLSH_ENV) $(PKG_ENV) $(VALGRIND) $(VALGRINDARGS) $(TCLSH_PROG) $(SCRIPT) depend: #======================================================================== # $(PKG_LIB_FILE) should be listed as part of the BINARIES variable # mentioned above. That will ensure that this target is built when you # run "make binaries". |
︙ | ︙ | |||
279 280 281 282 283 284 285 | # $(COMPILE) -c `@CYGPATH@ $(srcdir)/generic/sample.c` -o $@ # # Setting the VPATH variable to a list of paths will cause the makefile # to look into these paths when resolving .c to .obj dependencies. # As necessary, add $(srcdir):$(srcdir)/compat:.... #======================================================================== | | > > > > > > > > > > > > > > | > > > | | | > > | | < | > > | < > | < < > | | > > > > | < < | | | | | | | | < < < < < < | < < | | | | | < < | | | > | 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340 341 342 343 344 345 346 347 348 349 350 351 352 353 354 355 356 357 358 359 360 361 362 363 364 365 366 367 368 369 370 371 372 373 374 375 376 377 378 379 380 381 382 383 384 385 386 387 388 389 390 391 392 393 394 395 396 397 398 399 400 401 402 403 404 405 406 407 408 409 410 411 412 413 414 415 416 417 418 419 420 421 422 423 424 425 426 427 428 429 430 431 432 433 434 435 436 437 438 439 440 441 442 443 444 445 446 447 448 449 450 451 452 453 454 455 456 457 458 459 460 461 462 463 464 465 466 467 468 469 470 471 472 473 474 475 | # $(COMPILE) -c `@CYGPATH@ $(srcdir)/generic/sample.c` -o $@ # # Setting the VPATH variable to a list of paths will cause the makefile # to look into these paths when resolving .c to .obj dependencies. # As necessary, add $(srcdir):$(srcdir)/compat:.... #======================================================================== VPATH = $(srcdir):$(srcdir)/generic:$(srcdir)/unix:$(srcdir)/win:$(srcdir)/macosx .c.@OBJEXT@: $(COMPILE) -c `@CYGPATH@ $<` -o $@ tclsample.@OBJEXT@: sampleUuid.h $(srcdir)/manifest.uuid: printf "git-" >$(srcdir)/manifest.uuid (cd $(srcdir); git rev-parse HEAD >>$(srcdir)/manifest.uuid || \ (printf "svn-r" >$(srcdir)/manifest.uuid ; \ svn info --show-item last-changed-revision >>$(srcdir)/manifest.uuid) || \ printf "unknown" >$(srcdir)/manifest.uuid) sampleUuid.h: $(srcdir)/manifest.uuid echo "#define SAMPLE_VERSION_UUID \\" >$@ cat $(srcdir)/manifest.uuid >>$@ echo "" >>$@ #======================================================================== # Distribution creation # You may need to tweak this target to make it work correctly. #======================================================================== #COMPRESS = tar cvf $(PKG_DIR).tar $(PKG_DIR); compress $(PKG_DIR).tar COMPRESS = tar zcvf $(PKG_DIR).tar.gz $(PKG_DIR) DIST_ROOT = /tmp/dist DIST_DIR = $(DIST_ROOT)/$(PKG_DIR) DIST_INSTALL_DATA = CPPROG='cp -p' $(INSTALL) -m 644 DIST_INSTALL_SCRIPT = CPPROG='cp -p' $(INSTALL) -m 755 dist-clean: rm -rf $(DIST_DIR) $(DIST_ROOT)/$(PKG_DIR).tar.* dist: dist-clean $(srcdir)/manifest.uuid $(INSTALL_DATA_DIR) $(DIST_DIR) # TEA files $(DIST_INSTALL_DATA) $(srcdir)/Makefile.in \ $(srcdir)/aclocal.m4 $(srcdir)/configure.ac \ $(DIST_DIR)/ $(DIST_INSTALL_SCRIPT) $(srcdir)/configure $(DIST_DIR)/ $(INSTALL_DATA_DIR) $(DIST_DIR)/tclconfig $(DIST_INSTALL_DATA) $(srcdir)/tclconfig/README.txt \ $(srcdir)/manifest.uuid \ $(srcdir)/tclconfig/tcl.m4 $(srcdir)/tclconfig/install-sh \ $(DIST_DIR)/tclconfig/ # Extension files $(DIST_INSTALL_DATA) \ $(srcdir)/ChangeLog \ $(srcdir)/README.sha \ $(srcdir)/license.terms \ $(srcdir)/README \ $(srcdir)/pkgIndex.tcl.in \ $(DIST_DIR)/ list='demos doc generic library macosx tests unix win'; \ for p in $$list; do \ if test -d $(srcdir)/$$p ; then \ $(INSTALL_DATA_DIR) $(DIST_DIR)/$$p; \ $(DIST_INSTALL_DATA) $(srcdir)/$$p/* $(DIST_DIR)/$$p/; \ fi; \ done (cd $(DIST_ROOT); $(COMPRESS);) #======================================================================== # End of user-definable section #======================================================================== #======================================================================== # Don't modify the file to clean here. Instead, set the "CLEANFILES" # variable in configure.ac #======================================================================== clean: -test -z "$(BINARIES)" || rm -f $(BINARIES) -rm -f *.$(OBJEXT) core *.core -test -z "$(CLEANFILES)" || rm -f $(CLEANFILES) distclean: clean -rm -f *.tab.c -rm -f $(CONFIG_CLEAN_FILES) -rm -f config.cache config.log config.status #======================================================================== # Install binary object libraries. On Windows this includes both .dll and # .lib files. Because the .lib files are not explicitly listed anywhere, # we need to deduce their existence from the .dll file of the same name. # Library files go into the lib directory. # In addition, this will generate the pkgIndex.tcl # file in the install location (assuming it can find a usable tclsh shell) # # You should not have to modify this target. #======================================================================== install-lib-binaries: binaries @$(INSTALL_DATA_DIR) "$(DESTDIR)$(pkglibdir)" @list='$(lib_BINARIES)'; for p in $$list; do \ if test -f $$p; then \ echo " $(INSTALL_LIBRARY) $$p $(DESTDIR)$(pkglibdir)/$$p"; \ $(INSTALL_LIBRARY) $$p "$(DESTDIR)$(pkglibdir)/$$p"; \ ext=`echo $$p|sed -e "s/.*\.//"`; \ if test "x$$ext" = "xdll"; then \ lib=`basename $$p|sed -e 's/.[^.]*$$//'`.lib; \ if test -f $$lib; then \ echo " $(INSTALL_DATA) $$lib $(DESTDIR)$(pkglibdir)/$$lib"; \ $(INSTALL_DATA) $$lib "$(DESTDIR)$(pkglibdir)/$$lib"; \ fi; \ fi; \ fi; \ done @list='$(PKG_TCL_SOURCES)'; for p in $$list; do \ if test -f $(srcdir)/$$p; then \ destp=`basename $$p`; \ echo " Install $$destp $(DESTDIR)$(pkglibdir)/$$destp"; \ $(INSTALL_DATA) $(srcdir)/$$p "$(DESTDIR)$(pkglibdir)/$$destp"; \ fi; \ done @if test "x$(SHARED_BUILD)" = "x1"; then \ echo " Install pkgIndex.tcl $(DESTDIR)$(pkglibdir)"; \ $(INSTALL_DATA) pkgIndex.tcl "$(DESTDIR)$(pkglibdir)"; \ fi #======================================================================== # Install binary executables (e.g. .exe files and dependent .dll files) # This is for files that must go in the bin directory (located next to # wish and tclsh), like dependent .dll files on Windows. # # You should not have to modify this target, except to define bin_BINARIES # above if necessary. #======================================================================== install-bin-binaries: binaries @$(INSTALL_DATA_DIR) "$(DESTDIR)$(bindir)" @list='$(bin_BINARIES)'; for p in $$list; do \ if test -f $$p; then \ echo " $(INSTALL_PROGRAM) $$p $(DESTDIR)$(bindir)/$$p"; \ $(INSTALL_PROGRAM) $$p "$(DESTDIR)$(bindir)/$$p"; \ fi; \ done Makefile: $(srcdir)/Makefile.in $(top_builddir)/config.status cd $(top_builddir) \ && CONFIG_FILES=$@ CONFIG_HEADERS= $(SHELL) ./config.status uninstall-binaries: list='$(lib_BINARIES)'; for p in $$list; do \ rm -f "$(DESTDIR)$(pkglibdir)/$$p"; \ done list='$(PKG_TCL_SOURCES)'; for p in $$list; do \ p=`basename $$p`; \ rm -f "$(DESTDIR)$(pkglibdir)/$$p"; \ done list='$(bin_BINARIES)'; for p in $$list; do \ rm -f "$(DESTDIR)$(bindir)/$$p"; \ done .PHONY: all binaries clean depend distclean doc install libraries test .PHONY: gdb gdb-test valgrind valgrindshell # Tell versions [3.59,3.63) of GNU make to not export all variables. # Otherwise a system limit (for SysV at least) may be exceeded. .NOEXPORT: |
Changes to autoconf/tea/configure.ac.
1 2 3 4 | #!/bin/bash -norc dnl This file is an input file used by the GNU "autoconf" program to dnl generate the file "configure", which is run during Tcl installation dnl to configure the system for the local environment. | | < < | > > | | | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 | #!/bin/bash -norc dnl This file is an input file used by the GNU "autoconf" program to dnl generate the file "configure", which is run during Tcl installation dnl to configure the system for the local environment. #----------------------------------------------------------------------- # Sample configure.ac for Tcl Extensions. The only places you should # need to modify this file are marked by the string __CHANGE__ #----------------------------------------------------------------------- #----------------------------------------------------------------------- # __CHANGE__ # Set your package name and version numbers here. # # This initializes the environment with PACKAGE_NAME and PACKAGE_VERSION # set as provided. These will also be added as -D defs in your Makefile # so you can encode the package version directly into the source files. # This will also define a special symbol for Windows (BUILD_<PACKAGE_NAME> # so that we create the export library with the dll. #----------------------------------------------------------------------- AC_INIT([sqlite],[3.40.0]) #-------------------------------------------------------------------- # Call TEA_INIT as the first TEA_ macro to set up initial vars. # This will define a ${TEA_PLATFORM} variable == "unix" or "windows" # as well as PKG_LIB_FILE and PKG_STUB_LIB_FILE. #-------------------------------------------------------------------- TEA_INIT() AC_CONFIG_AUX_DIR(tclconfig) #-------------------------------------------------------------------- # Load the tclConfig.sh file #-------------------------------------------------------------------- |
︙ | ︙ | |||
51 52 53 54 55 56 57 | #----------------------------------------------------------------------- TEA_PREFIX #----------------------------------------------------------------------- # Standard compiler checks. # This sets up CC by using the CC env var, or looks for gcc otherwise. | | | | > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 | #----------------------------------------------------------------------- TEA_PREFIX #----------------------------------------------------------------------- # Standard compiler checks. # This sets up CC by using the CC env var, or looks for gcc otherwise. # This also calls AC_PROG_CC and a few others to create the basic setup # necessary to compile executables. #----------------------------------------------------------------------- TEA_SETUP_COMPILER #----------------------------------------------------------------------- # __CHANGE__ # Specify the C source files to compile in TEA_ADD_SOURCES, # public headers that need to be installed in TEA_ADD_HEADERS, # stub library C source files to compile in TEA_ADD_STUB_SOURCES, # and runtime Tcl library files in TEA_ADD_TCL_SOURCES. # This defines PKG(_STUB)_SOURCES, PKG(_STUB)_OBJECTS, PKG_HEADERS # and PKG_TCL_SOURCES. #----------------------------------------------------------------------- TEA_ADD_SOURCES([tclsqlite3.c]) TEA_ADD_HEADERS([]) TEA_ADD_INCLUDES([]) TEA_ADD_LIBS([]) TEA_ADD_CFLAGS([-DSQLITE_ENABLE_FTS3=1]) TEA_ADD_CFLAGS([-DSQLITE_ENABLE_FTS4=1]) TEA_ADD_CFLAGS([-DSQLITE_ENABLE_FTS5=1]) TEA_ADD_CFLAGS([-DSQLITE_3_SUFFIX_ONLY=1]) TEA_ADD_CFLAGS([-DSQLITE_ENABLE_RTREE=1]) TEA_ADD_CFLAGS([-DSQLITE_ENABLE_GEOPOLY=1]) TEA_ADD_CFLAGS([-DSQLITE_ENABLE_MATH_FUNCTIONS=1]) TEA_ADD_CFLAGS([-DSQLITE_ENABLE_DESERIALIZE=1]) TEA_ADD_CFLAGS([-DSQLITE_ENABLE_DBPAGE_VTAB=1]) TEA_ADD_CFLAGS([-DSQLITE_ENABLE_BYTECODE_VTAB=1]) TEA_ADD_CFLAGS([-DSQLITE_ENABLE_DBSTAT_VTAB=1]) TEA_ADD_STUB_SOURCES([]) TEA_ADD_TCL_SOURCES([]) #-------------------------------------------------------------------- # The --with-system-sqlite causes the TCL bindings to SQLite to use # the system shared library for SQLite rather than statically linking # against its own private copy. This is dangerous and leads to # undersirable dependences and is not recommended. # Patchs from rmax. #-------------------------------------------------------------------- AC_ARG_WITH([system-sqlite], [AC_HELP_STRING([--with-system-sqlite], [use a system-supplied libsqlite3 instead of the bundled one])], [], [with_system_sqlite=no]) if test x$with_system_sqlite != xno; then AC_CHECK_HEADER([sqlite3.h], [AC_CHECK_LIB([sqlite3],[sqlite3_initialize], [AC_DEFINE(USE_SYSTEM_SQLITE) LIBS="$LIBS -lsqlite3"])]) fi #-------------------------------------------------------------------- # __CHANGE__ # # You can add more files to clean if your extension creates any extra # files by extending CLEANFILES. # Add pkgIndex.tcl if it is generated in the Makefile instead of ./configure # and change Makefile.in to move it from CONFIG_CLEAN_FILES to BINARIES var. # # A few miscellaneous platform-specific items: # TEA_ADD_* any platform specific compiler/build info here. #-------------------------------------------------------------------- #CLEANFILES="$CLEANFILES pkgIndex.tcl" if test "${TEA_PLATFORM}" = "windows" ; then # Ensure no empty if clauses : #TEA_ADD_SOURCES([win/winFile.c]) #TEA_ADD_INCLUDES([-I\"$(${CYGPATH} ${srcdir}/win)\"]) else # Ensure no empty else clauses : #TEA_ADD_SOURCES([unix/unixFile.c]) #TEA_ADD_LIBS([-lsuperfly]) fi #-------------------------------------------------------------------- # __CHANGE__ # Choose which headers you need. Extension authors should try very # hard to only rely on the Tcl public header files. Internal headers # contain private data structures and are subject to change without # notice. |
︙ | ︙ | |||
146 147 148 149 150 151 152 | TEA_CONFIG_CFLAGS #-------------------------------------------------------------------- # Set the default compiler switches based on the --enable-symbols option. #-------------------------------------------------------------------- TEA_ENABLE_SYMBOLS | < < < < < < < < < < < < < < < < < < < < < < | 179 180 181 182 183 184 185 186 187 188 189 190 191 192 | TEA_CONFIG_CFLAGS #-------------------------------------------------------------------- # Set the default compiler switches based on the --enable-symbols option. #-------------------------------------------------------------------- TEA_ENABLE_SYMBOLS #-------------------------------------------------------------------- # This macro generates a line to use when building a library. It # depends on values set by the TEA_ENABLE_SHARED, TEA_ENABLE_SYMBOLS, # and TEA_LOAD_TCLCONFIG macros above. #-------------------------------------------------------------------- |
︙ | ︙ | |||
189 190 191 192 193 194 195 | # a pkgIndex.tcl file or anything else at extension build time. #-------------------------------------------------------------------- TEA_PROG_TCLSH #TEA_PROG_WISH #-------------------------------------------------------------------- | > > | > > > > > | | | > > > > > > > > | 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 | # a pkgIndex.tcl file or anything else at extension build time. #-------------------------------------------------------------------- TEA_PROG_TCLSH #TEA_PROG_WISH #-------------------------------------------------------------------- # Setup a *Config.sh.in configuration file. #-------------------------------------------------------------------- #TEA_EXPORT_CONFIG([sample]) #AC_SUBST(SAMPLE_VAR) #-------------------------------------------------------------------- # Specify files to substitute AC variables in. You may alternatively # have a special pkgIndex.tcl.in or other files which require # substituting the AC variables in. Include these here. #-------------------------------------------------------------------- AC_CONFIG_FILES([Makefile pkgIndex.tcl]) #AC_CONFIG_FILES([sampleConfig.sh]) #-------------------------------------------------------------------- # Finally, substitute all of the various values into the files # specified with AC_CONFIG_FILES. #-------------------------------------------------------------------- AC_OUTPUT |
Changes to autoconf/tea/pkgIndex.tcl.in.
|
| | | > | < > > | | > | 1 2 3 4 5 6 7 8 9 10 | # -*- tcl -*- # Tcl package index file, version 1.1 # if {[package vsatisfies [package provide Tcl] 9.0-]} { package ifneeded sqlite3 @PACKAGE_VERSION@ \ [list load [file join $dir @PKG_LIB_FILE9@] sqlite3] } else { package ifneeded sqlite3 @PACKAGE_VERSION@ \ [list load [file join $dir @PKG_LIB_FILE8@] sqlite3] } |
Changes to autoconf/tea/tclconfig/tcl.m4.
1 2 3 4 5 6 7 8 9 10 11 | # tcl.m4 -- # # This file provides a set of autoconf macros to help TEA-enable # a Tcl extension. # # Copyright (c) 1999-2000 Ajuba Solutions. # Copyright (c) 2002-2005 ActiveState Corporation. # # See the file "license.terms" for information on usage and redistribution # of this file, and for a DISCLAIMER OF ALL WARRANTIES. | | < < < < > | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 | # tcl.m4 -- # # This file provides a set of autoconf macros to help TEA-enable # a Tcl extension. # # Copyright (c) 1999-2000 Ajuba Solutions. # Copyright (c) 2002-2005 ActiveState Corporation. # # See the file "license.terms" for information on usage and redistribution # of this file, and for a DISCLAIMER OF ALL WARRANTIES. AC_PREREQ([2.69]) # Possible values for key variables defined: # # TEA_WINDOWINGSYSTEM - win32 aqua x11 (mirrors 'tk windowingsystem') # TEA_PLATFORM - windows unix # TEA_TK_EXTENSION - True if this is a Tk extension # #------------------------------------------------------------------------ # TEA_PATH_TCLCONFIG -- # # Locate the tclConfig.sh file and perform a sanity check on # the Tcl compile flags |
︙ | ︙ | |||
49 50 51 52 53 54 55 | # the alternative search directory is invoked by --with-tcl # if test x"${no_tcl}" = x ; then # we reset no_tcl in case something fails here no_tcl=true AC_ARG_WITH(tcl, | | | | 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 | # the alternative search directory is invoked by --with-tcl # if test x"${no_tcl}" = x ; then # we reset no_tcl in case something fails here no_tcl=true AC_ARG_WITH(tcl, AS_HELP_STRING([--with-tcl], [directory containing tcl configuration (tclConfig.sh)]), [with_tclconfig="${withval}"]) AC_MSG_CHECKING([for Tcl configuration]) AC_CACHE_VAL(ac_cv_c_tclconfig,[ # First check to see if --with-tcl was specified. if test x"${with_tclconfig}" != x ; then case "${with_tclconfig}" in */tclConfig.sh ) |
︙ | ︙ | |||
103 104 105 106 107 108 109 | fi # on Darwin, check in Framework installation locations if test "`uname -s`" = "Darwin" -a x"${ac_cv_c_tclconfig}" = x ; then for i in `ls -d ~/Library/Frameworks 2>/dev/null` \ `ls -d /Library/Frameworks 2>/dev/null` \ `ls -d /Network/Library/Frameworks 2>/dev/null` \ | > > | | 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 | fi # on Darwin, check in Framework installation locations if test "`uname -s`" = "Darwin" -a x"${ac_cv_c_tclconfig}" = x ; then for i in `ls -d ~/Library/Frameworks 2>/dev/null` \ `ls -d /Library/Frameworks 2>/dev/null` \ `ls -d /Network/Library/Frameworks 2>/dev/null` \ `ls -d /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/Library/Frameworks/Tcl.framework 2>/dev/null` \ `ls -d /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/Network/Library/Frameworks/Tcl.framework 2>/dev/null` \ `ls -d /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/System/Library/Frameworks/Tcl.framework 2>/dev/null` \ ; do if test -f "$i/Tcl.framework/tclConfig.sh" ; then ac_cv_c_tclconfig="`(cd $i/Tcl.framework; pwd)`" break fi done fi |
︙ | ︙ | |||
132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 | # check in a few common install locations if test x"${ac_cv_c_tclconfig}" = x ; then for i in `ls -d ${libdir} 2>/dev/null` \ `ls -d ${exec_prefix}/lib 2>/dev/null` \ `ls -d ${prefix}/lib 2>/dev/null` \ `ls -d /usr/local/lib 2>/dev/null` \ `ls -d /usr/contrib/lib 2>/dev/null` \ `ls -d /usr/lib 2>/dev/null` \ `ls -d /usr/lib64 2>/dev/null` \ `ls -d /usr/lib/tcl8.6 2>/dev/null` \ `ls -d /usr/lib/tcl8.5 2>/dev/null` \ ; do if test -f "$i/tclConfig.sh" ; then ac_cv_c_tclconfig="`(cd $i; pwd)`" break fi done fi | > > > > > | 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 | # check in a few common install locations if test x"${ac_cv_c_tclconfig}" = x ; then for i in `ls -d ${libdir} 2>/dev/null` \ `ls -d ${exec_prefix}/lib 2>/dev/null` \ `ls -d ${prefix}/lib 2>/dev/null` \ `ls -d /usr/local/lib 2>/dev/null` \ `ls -d /usr/contrib/lib 2>/dev/null` \ `ls -d /usr/pkg/lib 2>/dev/null` \ `ls -d /usr/lib 2>/dev/null` \ `ls -d /usr/lib64 2>/dev/null` \ `ls -d /usr/lib/tcl8.6 2>/dev/null` \ `ls -d /usr/lib/tcl8.5 2>/dev/null` \ `ls -d /usr/local/lib/tcl8.6 2>/dev/null` \ `ls -d /usr/local/lib/tcl8.5 2>/dev/null` \ `ls -d /usr/local/lib/tcl/tcl8.6 2>/dev/null` \ `ls -d /usr/local/lib/tcl/tcl8.5 2>/dev/null` \ ; do if test -f "$i/tclConfig.sh" ; then ac_cv_c_tclconfig="`(cd $i; pwd)`" break fi done fi |
︙ | ︙ | |||
204 205 206 207 208 209 210 | # the alternative search directory is invoked by --with-tk # if test x"${no_tk}" = x ; then # we reset no_tk in case something fails here no_tk=true AC_ARG_WITH(tk, | | | | 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 | # the alternative search directory is invoked by --with-tk # if test x"${no_tk}" = x ; then # we reset no_tk in case something fails here no_tk=true AC_ARG_WITH(tk, AS_HELP_STRING([--with-tk], [directory containing tk configuration (tkConfig.sh)]), [with_tkconfig="${withval}"]) AC_MSG_CHECKING([for Tk configuration]) AC_CACHE_VAL(ac_cv_c_tkconfig,[ # First check to see if --with-tkconfig was specified. if test x"${with_tkconfig}" != x ; then case "${with_tkconfig}" in */tkConfig.sh ) |
︙ | ︙ | |||
258 259 260 261 262 263 264 | fi # on Darwin, check in Framework installation locations if test "`uname -s`" = "Darwin" -a x"${ac_cv_c_tkconfig}" = x ; then for i in `ls -d ~/Library/Frameworks 2>/dev/null` \ `ls -d /Library/Frameworks 2>/dev/null` \ `ls -d /Network/Library/Frameworks 2>/dev/null` \ | < > > > > > > > | 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 | fi # on Darwin, check in Framework installation locations if test "`uname -s`" = "Darwin" -a x"${ac_cv_c_tkconfig}" = x ; then for i in `ls -d ~/Library/Frameworks 2>/dev/null` \ `ls -d /Library/Frameworks 2>/dev/null` \ `ls -d /Network/Library/Frameworks 2>/dev/null` \ ; do if test -f "$i/Tk.framework/tkConfig.sh" ; then ac_cv_c_tkconfig="`(cd $i/Tk.framework; pwd)`" break fi done fi # check in a few common install locations if test x"${ac_cv_c_tkconfig}" = x ; then for i in `ls -d ${libdir} 2>/dev/null` \ `ls -d ${exec_prefix}/lib 2>/dev/null` \ `ls -d ${prefix}/lib 2>/dev/null` \ `ls -d /usr/local/lib 2>/dev/null` \ `ls -d /usr/contrib/lib 2>/dev/null` \ `ls -d /usr/pkg/lib 2>/dev/null` \ `ls -d /usr/lib/tk8.6 2>/dev/null` \ `ls -d /usr/lib/tk8.5 2>/dev/null` \ `ls -d /usr/lib 2>/dev/null` \ `ls -d /usr/lib64 2>/dev/null` \ `ls -d /usr/local/lib/tk8.6 2>/dev/null` \ `ls -d /usr/local/lib/tk8.5 2>/dev/null` \ `ls -d /usr/local/lib/tcl/tk8.6 2>/dev/null` \ `ls -d /usr/local/lib/tcl/tk8.5 2>/dev/null` \ ; do if test -f "$i/tkConfig.sh" ; then ac_cv_c_tkconfig="`(cd $i; pwd)`" break fi done fi |
︙ | ︙ | |||
344 345 346 347 348 349 350 351 352 353 354 355 356 357 358 359 360 361 362 | # # Results: # # Substitutes the following vars: # TCL_BIN_DIR # TCL_SRC_DIR # TCL_LIB_FILE #------------------------------------------------------------------------ AC_DEFUN([TEA_LOAD_TCLCONFIG], [ AC_MSG_CHECKING([for existence of ${TCL_BIN_DIR}/tclConfig.sh]) if test -f "${TCL_BIN_DIR}/tclConfig.sh" ; then AC_MSG_RESULT([loading]) . "${TCL_BIN_DIR}/tclConfig.sh" else AC_MSG_RESULT([could not find ${TCL_BIN_DIR}/tclConfig.sh]) fi | > > < < < < | 354 355 356 357 358 359 360 361 362 363 364 365 366 367 368 369 370 371 372 373 374 375 376 377 378 379 380 381 | # # Results: # # Substitutes the following vars: # TCL_BIN_DIR # TCL_SRC_DIR # TCL_LIB_FILE # TCL_ZIP_FILE # TCL_ZIPFS_SUPPORT #------------------------------------------------------------------------ AC_DEFUN([TEA_LOAD_TCLCONFIG], [ AC_MSG_CHECKING([for existence of ${TCL_BIN_DIR}/tclConfig.sh]) if test -f "${TCL_BIN_DIR}/tclConfig.sh" ; then AC_MSG_RESULT([loading]) . "${TCL_BIN_DIR}/tclConfig.sh" else AC_MSG_RESULT([could not find ${TCL_BIN_DIR}/tclConfig.sh]) fi # If the TCL_BIN_DIR is the build directory (not the install directory), # then set the common variable name to the value of the build variables. # For example, the variable TCL_LIB_SPEC will be set to the value # of TCL_BUILD_LIB_SPEC. An extension should make use of TCL_LIB_SPEC # instead of TCL_BUILD_LIB_SPEC since it will work with both an # installed and uninstalled version of Tcl. if test -f "${TCL_BIN_DIR}/Makefile" ; then |
︙ | ︙ | |||
393 394 395 396 397 398 399 | TCL_STUB_LIB_SPEC="-L`echo "${TCL_BIN_DIR}" | sed -e 's/ /\\\\ /g'` ${TCL_STUB_LIB_FLAG}" TCL_STUB_LIB_PATH="${TCL_BIN_DIR}/${TCL_STUB_LIB_FILE}" fi ;; esac fi | < < < < < < | > > | > > < > > | 401 402 403 404 405 406 407 408 409 410 411 412 413 414 415 416 417 418 419 420 421 422 423 424 425 426 427 428 429 430 431 432 433 434 435 436 437 438 439 440 441 | TCL_STUB_LIB_SPEC="-L`echo "${TCL_BIN_DIR}" | sed -e 's/ /\\\\ /g'` ${TCL_STUB_LIB_FLAG}" TCL_STUB_LIB_PATH="${TCL_BIN_DIR}/${TCL_STUB_LIB_FILE}" fi ;; esac fi AC_SUBST(TCL_VERSION) AC_SUBST(TCL_PATCH_LEVEL) AC_SUBST(TCL_BIN_DIR) AC_SUBST(TCL_SRC_DIR) AC_SUBST(TCL_LIB_FILE) AC_SUBST(TCL_LIB_FLAG) AC_SUBST(TCL_LIB_SPEC) AC_SUBST(TCL_STUB_LIB_FILE) AC_SUBST(TCL_STUB_LIB_FLAG) AC_SUBST(TCL_STUB_LIB_SPEC) AC_MSG_CHECKING([platform]) hold_cc=$CC; CC="$TCL_CC" AC_COMPILE_IFELSE([AC_LANG_PROGRAM([[]], [[ #ifdef _WIN32 #error win32 #endif ]])],[ # first test we've already retrieved platform (cross-compile), fallback to unix otherwise: TEA_PLATFORM="${TEA_PLATFORM-unix}" CYGPATH=echo ],[ TEA_PLATFORM="windows" AC_CHECK_PROG(CYGPATH, cygpath, cygpath -m, echo) ]) CC=$hold_cc AC_MSG_RESULT($TEA_PLATFORM) # The BUILD_$pkg is to define the correct extern storage class # handling when making this package AC_DEFINE_UNQUOTED(BUILD_${PACKAGE_NAME}, [], [Building extension source?]) |
︙ | ︙ | |||
469 470 471 472 473 474 475 | if test -f "${TK_BIN_DIR}/tkConfig.sh" ; then AC_MSG_RESULT([loading]) . "${TK_BIN_DIR}/tkConfig.sh" else AC_MSG_RESULT([could not find ${TK_BIN_DIR}/tkConfig.sh]) fi | < < < < | 476 477 478 479 480 481 482 483 484 485 486 487 488 489 | if test -f "${TK_BIN_DIR}/tkConfig.sh" ; then AC_MSG_RESULT([loading]) . "${TK_BIN_DIR}/tkConfig.sh" else AC_MSG_RESULT([could not find ${TK_BIN_DIR}/tkConfig.sh]) fi # If the TK_BIN_DIR is the build directory (not the install directory), # then set the common variable name to the value of the build variables. # For example, the variable TK_LIB_SPEC will be set to the value # of TK_BUILD_LIB_SPEC. An extension should make use of TK_LIB_SPEC # instead of TK_BUILD_LIB_SPEC since it will work with both an # installed and uninstalled version of Tcl. if test -f "${TK_BIN_DIR}/Makefile" ; then |
︙ | ︙ | |||
506 507 508 509 510 511 512 | TK_STUB_LIB_SPEC="-L` echo "${TK_BIN_DIR}" | sed -e 's/ /\\\\ /g'` ${TK_STUB_LIB_FLAG}" TK_STUB_LIB_PATH="${TK_BIN_DIR}/${TK_STUB_LIB_FILE}" fi ;; esac fi | < < < < < < | 509 510 511 512 513 514 515 516 517 518 519 520 521 522 | TK_STUB_LIB_SPEC="-L` echo "${TK_BIN_DIR}" | sed -e 's/ /\\\\ /g'` ${TK_STUB_LIB_FLAG}" TK_STUB_LIB_PATH="${TK_BIN_DIR}/${TK_STUB_LIB_FILE}" fi ;; esac fi # TEA specific: Ensure windowingsystem is defined if test "${TEA_PLATFORM}" = "unix" ; then case ${TK_DEFS} in *MAC_OSX_TK*) AC_DEFINE(MAC_OSX_TK, 1, [Are we building against Mac OS X TkAqua?]) TEA_WINDOWINGSYSTEM="aqua" ;; |
︙ | ︙ | |||
568 569 570 571 572 573 574 | #------------------------------------------------------------------------ AC_DEFUN([TEA_PROG_TCLSH], [ AC_MSG_CHECKING([for tclsh]) if test -f "${TCL_BIN_DIR}/Makefile" ; then # tclConfig.sh is in Tcl build directory if test "${TEA_PLATFORM}" = "windows"; then | > | > > > > > > > | | | 565 566 567 568 569 570 571 572 573 574 575 576 577 578 579 580 581 582 583 584 585 586 587 588 589 590 591 592 593 594 595 596 | #------------------------------------------------------------------------ AC_DEFUN([TEA_PROG_TCLSH], [ AC_MSG_CHECKING([for tclsh]) if test -f "${TCL_BIN_DIR}/Makefile" ; then # tclConfig.sh is in Tcl build directory if test "${TEA_PLATFORM}" = "windows"; then if test -f "${TCL_BIN_DIR}/tclsh${TCL_MAJOR_VERSION}${TCL_MINOR_VERSION}${EXEEXT}" ; then TCLSH_PROG="${TCL_BIN_DIR}/tclsh${TCL_MAJOR_VERSION}${TCL_MINOR_VERSION}${EXEEXT}" elif test -f "${TCL_BIN_DIR}/tclsh${TCL_MAJOR_VERSION}${TCL_MINOR_VERSION}s${EXEEXT}" ; then TCLSH_PROG="${TCL_BIN_DIR}/tclsh${TCL_MAJOR_VERSION}${TCL_MINOR_VERSION}s${EXEEXT}" elif test -f "${TCL_BIN_DIR}/tclsh${TCL_MAJOR_VERSION}${TCL_MINOR_VERSION}t${EXEEXT}" ; then TCLSH_PROG="${TCL_BIN_DIR}/tclsh${TCL_MAJOR_VERSION}${TCL_MINOR_VERSION}t${EXEEXT}" elif test -f "${TCL_BIN_DIR}/tclsh${TCL_MAJOR_VERSION}${TCL_MINOR_VERSION}st${EXEEXT}" ; then TCLSH_PROG="${TCL_BIN_DIR}/tclsh${TCL_MAJOR_VERSION}${TCL_MINOR_VERSION}st${EXEEXT}" fi else TCLSH_PROG="${TCL_BIN_DIR}/tclsh" fi else # tclConfig.sh is in install location if test "${TEA_PLATFORM}" = "windows"; then TCLSH_PROG="tclsh${TCL_MAJOR_VERSION}${TCL_MINOR_VERSION}${EXEEXT}" else TCLSH_PROG="tclsh${TCL_MAJOR_VERSION}.${TCL_MINOR_VERSION}" fi list="`ls -d ${TCL_BIN_DIR}/../bin 2>/dev/null` \ `ls -d ${TCL_BIN_DIR}/.. 2>/dev/null` \ `ls -d ${TCL_PREFIX}/bin 2>/dev/null`" for i in $list ; do if test -f "$i/${TCLSH_PROG}" ; then REAL_TCL_BIN_DIR="`cd "$i"; pwd`/" |
︙ | ︙ | |||
618 619 620 621 622 623 624 | #------------------------------------------------------------------------ AC_DEFUN([TEA_PROG_WISH], [ AC_MSG_CHECKING([for wish]) if test -f "${TK_BIN_DIR}/Makefile" ; then # tkConfig.sh is in Tk build directory if test "${TEA_PLATFORM}" = "windows"; then | > | > > > > > > > | | | 623 624 625 626 627 628 629 630 631 632 633 634 635 636 637 638 639 640 641 642 643 644 645 646 647 648 649 650 651 652 653 654 | #------------------------------------------------------------------------ AC_DEFUN([TEA_PROG_WISH], [ AC_MSG_CHECKING([for wish]) if test -f "${TK_BIN_DIR}/Makefile" ; then # tkConfig.sh is in Tk build directory if test "${TEA_PLATFORM}" = "windows"; then if test -f "${TK_BIN_DIR}/wish${TK_MAJOR_VERSION}${TK_MINOR_VERSION}${EXEEXT}" ; then WISH_PROG="${TK_BIN_DIR}/wish${TK_MAJOR_VERSION}${TK_MINOR_VERSION}${EXEEXT}" elif test -f "${TK_BIN_DIR}/wish${TK_MAJOR_VERSION}${TK_MINOR_VERSION}s${EXEEXT}" ; then WISH_PROG="${TK_BIN_DIR}/wish${TK_MAJOR_VERSION}${TK_MINOR_VERSION}$s{EXEEXT}" elif test -f "${TK_BIN_DIR}/wish${TK_MAJOR_VERSION}${TK_MINOR_VERSION}t${EXEEXT}" ; then WISH_PROG="${TK_BIN_DIR}/wish${TK_MAJOR_VERSION}${TK_MINOR_VERSION}t${EXEEXT}" elif test -f "${TK_BIN_DIR}/wish${TK_MAJOR_VERSION}${TK_MINOR_VERSION}st${EXEEXT}" ; then WISH_PROG="${TK_BIN_DIR}/wish${TK_MAJOR_VERSION}${TK_MINOR_VERSION}st${EXEEXT}" fi else WISH_PROG="${TK_BIN_DIR}/wish" fi else # tkConfig.sh is in install location if test "${TEA_PLATFORM}" = "windows"; then WISH_PROG="wish${TK_MAJOR_VERSION}${TK_MINOR_VERSION}${EXEEXT}" else WISH_PROG="wish${TK_MAJOR_VERSION}.${TK_MINOR_VERSION}" fi list="`ls -d ${TK_BIN_DIR}/../bin 2>/dev/null` \ `ls -d ${TK_BIN_DIR}/.. 2>/dev/null` \ `ls -d ${TK_PREFIX}/bin 2>/dev/null`" for i in $list ; do if test -f "$i/${WISH_PROG}" ; then REAL_TK_BIN_DIR="`cd "$i"; pwd`/" |
︙ | ︙ | |||
656 657 658 659 660 661 662 663 664 665 666 667 668 669 670 | # Arguments: # none # # Results: # # Adds the following arguments to configure: # --enable-shared=yes|no # # Defines the following vars: # STATIC_BUILD Used for building import/export libraries # on Windows. # # Sets the following vars: # SHARED_BUILD Value of 1 or 0 #------------------------------------------------------------------------ | > > > > > > < | | | | > > > > > > > > > > > > > | > | > > > > | > > > > > > > > > > | 669 670 671 672 673 674 675 676 677 678 679 680 681 682 683 684 685 686 687 688 689 690 691 692 693 694 695 696 697 698 699 700 701 702 703 704 705 706 707 708 709 710 711 712 713 714 715 716 717 718 719 720 721 722 723 724 725 726 727 728 729 730 731 732 733 734 735 736 737 738 739 740 741 742 743 744 745 746 747 | # Arguments: # none # # Results: # # Adds the following arguments to configure: # --enable-shared=yes|no # --enable-stubs=yes|no # # Defines the following vars: # STATIC_BUILD Used for building import/export libraries # on Windows. # # Sets the following vars: # SHARED_BUILD Value of 1 or 0 # STUBS_BUILD Value if 1 or 0 # USE_TCL_STUBS Value true: if SHARED_BUILD or --enable-stubs # USE_TCLOO_STUBS Value true: if SHARED_BUILD or --enable-stubs # USE_TK_STUBS Value true: if SHARED_BUILD or --enable-stubs # AND TEA_WINDOWING_SYSTEM != "" #------------------------------------------------------------------------ AC_DEFUN([TEA_ENABLE_SHARED], [ AC_MSG_CHECKING([how to build libraries]) AC_ARG_ENABLE(shared, AS_HELP_STRING([--enable-shared], [build and link with shared libraries (default: on)]), [shared_ok=$enableval], [shared_ok=yes]) if test "${enable_shared+set}" = set; then enableval="$enable_shared" shared_ok=$enableval else shared_ok=yes fi AC_ARG_ENABLE(stubs, AS_HELP_STRING([--enable-stubs], [build and link with stub libraries. Always true for shared builds (default: on)]), [stubs_ok=$enableval], [stubs_ok=yes]) if test "${enable_stubs+set}" = set; then enableval="$enable_stubs" stubs_ok=$enableval else stubs_ok=yes fi # Stubs are always enabled for shared builds if test "$shared_ok" = "yes" ; then AC_MSG_RESULT([shared]) SHARED_BUILD=1 STUBS_BUILD=1 else AC_MSG_RESULT([static]) SHARED_BUILD=0 AC_DEFINE(STATIC_BUILD, 1, [This a static build]) if test "$stubs_ok" = "yes" ; then STUBS_BUILD=1 else STUBS_BUILD=0 fi fi if test "${STUBS_BUILD}" = "1" ; then AC_DEFINE(USE_TCL_STUBS, 1, [Use Tcl stubs]) AC_DEFINE(USE_TCLOO_STUBS, 1, [Use TclOO stubs]) if test "${TEA_WINDOWINGSYSTEM}" != ""; then AC_DEFINE(USE_TK_STUBS, 1, [Use Tk stubs]) fi fi AC_SUBST(SHARED_BUILD) AC_SUBST(STUBS_BUILD) ]) #------------------------------------------------------------------------ # TEA_ENABLE_THREADS -- # # Specify if thread support should be enabled. If "yes" is specified # as an arg (optional), threads are enabled by default, "no" means |
︙ | ︙ | |||
724 725 726 727 728 729 730 | # TCL_THREADS # _REENTRANT # _THREAD_SAFE #------------------------------------------------------------------------ AC_DEFUN([TEA_ENABLE_THREADS], [ AC_ARG_ENABLE(threads, | | | | 770 771 772 773 774 775 776 777 778 779 780 781 782 783 784 785 | # TCL_THREADS # _REENTRANT # _THREAD_SAFE #------------------------------------------------------------------------ AC_DEFUN([TEA_ENABLE_THREADS], [ AC_ARG_ENABLE(threads, AS_HELP_STRING([--enable-threads], [build with threads (default: on)]), [tcl_ok=$enableval], [tcl_ok=yes]) if test "${enable_threads+set}" = set; then enableval="$enable_threads" tcl_ok=$enableval else tcl_ok=yes |
︙ | ︙ | |||
809 810 811 812 813 814 815 | *THREADS=1*) if test "${TCL_THREADS}" = "0"; then AC_MSG_WARN([ Building ${PACKAGE_NAME} without threads enabled, but building against Tcl that IS thread-enabled. It is recommended to use --enable-threads.]) fi ;; | < < < < < < < < | 855 856 857 858 859 860 861 862 863 864 865 866 867 868 | *THREADS=1*) if test "${TCL_THREADS}" = "0"; then AC_MSG_WARN([ Building ${PACKAGE_NAME} without threads enabled, but building against Tcl that IS thread-enabled. It is recommended to use --enable-threads.]) fi ;; esac AC_SUBST(TCL_THREADS) ]) #------------------------------------------------------------------------ # TEA_ENABLE_SYMBOLS -- # |
︙ | ︙ | |||
846 847 848 849 850 851 852 | # --enable-symbols # # Defines the following vars: # CFLAGS_DEFAULT Sets to $(CFLAGS_DEBUG) if true # Sets to "$(CFLAGS_OPTIMIZE) -DNDEBUG" if false # LDFLAGS_DEFAULT Sets to $(LDFLAGS_DEBUG) if true # Sets to $(LDFLAGS_OPTIMIZE) if false | < < | < > < < < < < | 884 885 886 887 888 889 890 891 892 893 894 895 896 897 898 899 900 901 902 903 904 905 906 907 908 909 910 911 912 913 914 915 916 917 918 919 920 921 | # --enable-symbols # # Defines the following vars: # CFLAGS_DEFAULT Sets to $(CFLAGS_DEBUG) if true # Sets to "$(CFLAGS_OPTIMIZE) -DNDEBUG" if false # LDFLAGS_DEFAULT Sets to $(LDFLAGS_DEBUG) if true # Sets to $(LDFLAGS_OPTIMIZE) if false #------------------------------------------------------------------------ AC_DEFUN([TEA_ENABLE_SYMBOLS], [ dnl TEA specific: Make sure we are initialized AC_REQUIRE([TEA_CONFIG_CFLAGS]) AC_MSG_CHECKING([for build with symbols]) AC_ARG_ENABLE(symbols, AS_HELP_STRING([--enable-symbols], [build with debugging symbols (default: off)]), [tcl_ok=$enableval], [tcl_ok=no]) if test "$tcl_ok" = "no"; then CFLAGS_DEFAULT="${CFLAGS_OPTIMIZE} -DNDEBUG" LDFLAGS_DEFAULT="${LDFLAGS_OPTIMIZE}" AC_MSG_RESULT([no]) AC_DEFINE(TCL_CFG_OPTIMIZED, 1, [Is this an optimized build?]) else CFLAGS_DEFAULT="${CFLAGS_DEBUG}" LDFLAGS_DEFAULT="${LDFLAGS_DEBUG}" if test "$tcl_ok" = "yes"; then AC_MSG_RESULT([yes (standard debugging)]) fi fi AC_SUBST(CFLAGS_DEFAULT) AC_SUBST(LDFLAGS_DEFAULT) if test "$tcl_ok" = "mem" -o "$tcl_ok" = "all"; then AC_DEFINE(TCL_MEM_DEBUG, 1, [Is memory debugging enabled?]) fi if test "$tcl_ok" != "yes" -a "$tcl_ok" != "no"; then if test "$tcl_ok" = "all"; then |
︙ | ︙ | |||
911 912 913 914 915 916 917 | # # Defines the following vars: # HAVE_LANGINFO Triggers use of nl_langinfo if defined. #------------------------------------------------------------------------ AC_DEFUN([TEA_ENABLE_LANGINFO], [ AC_ARG_ENABLE(langinfo, | | | | 942 943 944 945 946 947 948 949 950 951 952 953 954 955 956 957 958 959 960 961 962 963 964 965 966 967 | # # Defines the following vars: # HAVE_LANGINFO Triggers use of nl_langinfo if defined. #------------------------------------------------------------------------ AC_DEFUN([TEA_ENABLE_LANGINFO], [ AC_ARG_ENABLE(langinfo, AS_HELP_STRING([--enable-langinfo], [use nl_langinfo if possible to determine encoding at startup, otherwise use old heuristic (default: on)]), [langinfo_ok=$enableval], [langinfo_ok=yes]) HAVE_LANGINFO=0 if test "$langinfo_ok" = "yes"; then AC_CHECK_HEADER(langinfo.h,[langinfo_ok=yes],[langinfo_ok=no]) fi AC_MSG_CHECKING([whether to use nl_langinfo]) if test "$langinfo_ok" = "yes"; then AC_CACHE_VAL(tcl_cv_langinfo_h, [ AC_COMPILE_IFELSE([AC_LANG_PROGRAM([[#include <langinfo.h>]], [[nl_langinfo(CODESET);]])], [tcl_cv_langinfo_h=yes],[tcl_cv_langinfo_h=no])]) AC_MSG_RESULT([$tcl_cv_langinfo_h]) if test $tcl_cv_langinfo_h = yes; then AC_DEFINE(HAVE_LANGINFO, 1, [Do we have nl_langinfo()?]) fi else AC_MSG_RESULT([$langinfo_ok]) |
︙ | ︙ | |||
947 948 949 950 951 952 953 954 955 956 957 958 959 960 961 962 963 964 965 966 967 968 969 970 971 972 973 974 975 | # Arguments: # none # # Results: # Defines the following var: # # system - System/platform/version identification code. #-------------------------------------------------------------------- AC_DEFUN([TEA_CONFIG_SYSTEM], [ AC_CACHE_CHECK([system version], tcl_cv_sys_version, [ # TEA specific: if test "${TEA_PLATFORM}" = "windows" ; then tcl_cv_sys_version=windows else tcl_cv_sys_version=`uname -s`-`uname -r` if test "$?" -ne 0 ; then AC_MSG_WARN([can't find uname command]) tcl_cv_sys_version=unknown else if test "`uname -s`" = "AIX" ; then tcl_cv_sys_version=AIX-`uname -v`.`uname -r` fi fi fi ]) system=$tcl_cv_sys_version ]) | > > > > | 978 979 980 981 982 983 984 985 986 987 988 989 990 991 992 993 994 995 996 997 998 999 1000 1001 1002 1003 1004 1005 1006 1007 1008 1009 1010 | # Arguments: # none # # Results: # Defines the following var: # # system - System/platform/version identification code. # #-------------------------------------------------------------------- AC_DEFUN([TEA_CONFIG_SYSTEM], [ AC_CACHE_CHECK([system version], tcl_cv_sys_version, [ # TEA specific: if test "${TEA_PLATFORM}" = "windows" ; then tcl_cv_sys_version=windows else tcl_cv_sys_version=`uname -s`-`uname -r` if test "$?" -ne 0 ; then AC_MSG_WARN([can't find uname command]) tcl_cv_sys_version=unknown else if test "`uname -s`" = "AIX" ; then tcl_cv_sys_version=AIX-`uname -v`.`uname -r` fi if test "`uname -s`" = "NetBSD" -a -f /etc/debian_version ; then tcl_cv_sys_version=NetBSD-Debian fi fi fi ]) system=$tcl_cv_sys_version ]) |
︙ | ︙ | |||
1039 1040 1041 1042 1043 1044 1045 | dnl TEA specific: Make sure we are initialized AC_REQUIRE([TEA_INIT]) # Step 0.a: Enable 64 bit support? AC_MSG_CHECKING([if 64bit support is requested]) AC_ARG_ENABLE(64bit, | | | | | | | < < < < < < < < < < < | 1074 1075 1076 1077 1078 1079 1080 1081 1082 1083 1084 1085 1086 1087 1088 1089 1090 1091 1092 1093 1094 1095 1096 1097 1098 1099 1100 1101 1102 1103 1104 1105 1106 1107 1108 1109 1110 1111 1112 1113 1114 1115 1116 1117 1118 1119 1120 1121 1122 1123 1124 1125 1126 1127 1128 1129 1130 | dnl TEA specific: Make sure we are initialized AC_REQUIRE([TEA_INIT]) # Step 0.a: Enable 64 bit support? AC_MSG_CHECKING([if 64bit support is requested]) AC_ARG_ENABLE(64bit, AS_HELP_STRING([--enable-64bit], [enable 64bit support (default: off)]), [do64bit=$enableval], [do64bit=no]) AC_MSG_RESULT([$do64bit]) # Step 0.b: Enable Solaris 64 bit VIS support? AC_MSG_CHECKING([if 64bit Sparc VIS support is requested]) AC_ARG_ENABLE(64bit-vis, AS_HELP_STRING([--enable-64bit-vis], [enable 64bit Sparc VIS support (default: off)]), [do64bitVIS=$enableval], [do64bitVIS=no]) AC_MSG_RESULT([$do64bitVIS]) # Force 64bit on with VIS AS_IF([test "$do64bitVIS" = "yes"], [do64bit=yes]) # Step 0.c: Check if visibility support is available. Do this here so # that platform specific alternatives can be used below if this fails. AC_CACHE_CHECK([if compiler supports visibility "hidden"], tcl_cv_cc_visibility_hidden, [ hold_cflags=$CFLAGS; CFLAGS="$CFLAGS -Werror" AC_LINK_IFELSE([AC_LANG_PROGRAM([[ extern __attribute__((__visibility__("hidden"))) void f(void); void f(void) {}]], [[f();]])],[tcl_cv_cc_visibility_hidden=yes], [tcl_cv_cc_visibility_hidden=no]) CFLAGS=$hold_cflags]) AS_IF([test $tcl_cv_cc_visibility_hidden = yes], [ AC_DEFINE(MODULE_SCOPE, [extern __attribute__((__visibility__("hidden")))], [Compiler support for module scope symbols]) AC_DEFINE(HAVE_HIDDEN, [1], [Compiler support for module scope symbols]) ]) # Step 0.d: Disable -rpath support? AC_MSG_CHECKING([if rpath support is requested]) AC_ARG_ENABLE(rpath, AS_HELP_STRING([--disable-rpath], [disable rpath support (default: on)]), [doRpath=$enableval], [doRpath=yes]) AC_MSG_RESULT([$doRpath]) # Set the variable "system" to hold the name and version number # for the system. TEA_CONFIG_SYSTEM # Require ranlib early so we can override it in special cases below. |
︙ | ︙ | |||
1129 1130 1131 1132 1133 1134 1135 | ], [ CFLAGS_OPTIMIZE=-O CFLAGS_WARNING="" ]) AC_CHECK_TOOL(AR, ar) STLIB_LD='${AR} cr' LD_LIBRARY_PATH_VAR="LD_LIBRARY_PATH" | | < < < < < < < < < < < > > > < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < > > > > > > > > < | < | | | < < < < | < < < < < < < < < < < < < < < < < < < < < < | | | | | | > > > > > > > | | 1153 1154 1155 1156 1157 1158 1159 1160 1161 1162 1163 1164 1165 1166 1167 1168 1169 1170 1171 1172 1173 1174 1175 1176 1177 1178 1179 1180 1181 1182 1183 1184 1185 1186 1187 1188 1189 1190 1191 1192 1193 1194 1195 1196 1197 1198 1199 1200 1201 1202 1203 1204 1205 1206 1207 1208 1209 1210 1211 1212 1213 1214 1215 1216 1217 1218 1219 1220 1221 1222 1223 1224 1225 1226 1227 1228 1229 1230 1231 1232 1233 1234 1235 1236 1237 1238 1239 1240 1241 1242 1243 1244 1245 1246 1247 1248 1249 1250 1251 1252 1253 1254 1255 1256 1257 | ], [ CFLAGS_OPTIMIZE=-O CFLAGS_WARNING="" ]) AC_CHECK_TOOL(AR, ar) STLIB_LD='${AR} cr' LD_LIBRARY_PATH_VAR="LD_LIBRARY_PATH" AS_IF([test "x$SHLIB_VERSION" = x],[SHLIB_VERSION=""],[SHLIB_VERSION=".$SHLIB_VERSION"]) case $system in # TEA specific: windows) MACHINE="X86" if test "$do64bit" != "no" ; then case "$do64bit" in amd64|x64|yes) MACHINE="AMD64" ; # default to AMD64 64-bit build ;; arm64|aarch64) MACHINE="ARM64" ;; ia64) MACHINE="IA64" ;; esac fi if test "$GCC" != "yes" ; then if test "${SHARED_BUILD}" = "0" ; then runtime=-MT else runtime=-MD fi case "x`echo \${VisualStudioVersion}`" in x1[[4-9]]*) lflags="${lflags} -nodefaultlib:libucrt.lib" TEA_ADD_LIBS([ucrt.lib]) ;; *) ;; esac if test "$do64bit" != "no" ; then CC="cl.exe" RC="rc.exe" lflags="${lflags} -nologo -MACHINE:${MACHINE} " LINKBIN="link.exe" CFLAGS_DEBUG="-nologo -Zi -Od -W3 ${runtime}d" CFLAGS_OPTIMIZE="-nologo -O2 -W2 ${runtime}" # Avoid 'unresolved external symbol __security_cookie' # errors, c.f. http://support.microsoft.com/?id=894573 TEA_ADD_LIBS([bufferoverflowU.lib]) else RC="rc" lflags="${lflags} -nologo" LINKBIN="link" CFLAGS_DEBUG="-nologo -Z7 -Od -W3 -WX ${runtime}d" CFLAGS_OPTIMIZE="-nologo -O2 -W2 ${runtime}" fi fi if test "$GCC" = "yes"; then # mingw gcc mode AC_CHECK_TOOL(RC, windres) CFLAGS_DEBUG="-g" CFLAGS_OPTIMIZE="-O2 -fomit-frame-pointer" SHLIB_LD='${CC} -shared' UNSHARED_LIB_SUFFIX='${TCL_TRIM_DOTS}.a' LDFLAGS_CONSOLE="-wl,--subsystem,console ${lflags}" LDFLAGS_WINDOW="-wl,--subsystem,windows ${lflags}" AC_CACHE_CHECK(for cross-compile version of gcc, ac_cv_cross, AC_COMPILE_IFELSE([AC_LANG_PROGRAM([[ #ifdef _WIN32 #error cross-compiler #endif ]], [[]])], [ac_cv_cross=yes], [ac_cv_cross=no]) ) if test "$ac_cv_cross" = "yes"; then case "$do64bit" in amd64|x64|yes) CC="x86_64-w64-mingw32-${CC}" LD="x86_64-w64-mingw32-ld" AR="x86_64-w64-mingw32-ar" RANLIB="x86_64-w64-mingw32-ranlib" RC="x86_64-w64-mingw32-windres" ;; arm64|aarch64) CC="aarch64-w64-mingw32-clang" LD="aarch64-w64-mingw32-ld" AR="aarch64-w64-mingw32-ar" RANLIB="aarch64-w64-mingw32-ranlib" RC="aarch64-w64-mingw32-windres" ;; *) CC="i686-w64-mingw32-${CC}" LD="i686-w64-mingw32-ld" AR="i686-w64-mingw32-ar" RANLIB="i686-w64-mingw32-ranlib" RC="i686-w64-mingw32-windres" ;; esac fi |
︙ | ︙ | |||
1330 1331 1332 1333 1334 1335 1336 | # For information on what debugtype is most useful, see: # http://msdn.microsoft.com/library/en-us/dnvc60/html/gendepdebug.asp # and also # http://msdn2.microsoft.com/en-us/library/y0zzbyt4%28VS.80%29.aspx # This essentially turns it all on. LDFLAGS_DEBUG="-debug -debugtype:cv" LDFLAGS_OPTIMIZE="-release" | < < < < | | < | | 1265 1266 1267 1268 1269 1270 1271 1272 1273 1274 1275 1276 1277 1278 1279 1280 1281 1282 1283 1284 1285 1286 1287 1288 1289 | # For information on what debugtype is most useful, see: # http://msdn.microsoft.com/library/en-us/dnvc60/html/gendepdebug.asp # and also # http://msdn2.microsoft.com/en-us/library/y0zzbyt4%28VS.80%29.aspx # This essentially turns it all on. LDFLAGS_DEBUG="-debug -debugtype:cv" LDFLAGS_OPTIMIZE="-release" LDFLAGS_CONSOLE="-link -subsystem:console ${lflags}" LDFLAGS_WINDOW="-link -subsystem:windows ${lflags}" fi SHLIB_SUFFIX=".dll" SHARED_LIB_SUFFIX='${TCL_TRIM_DOTS}.dll' TCL_LIB_VERSIONS_OK=nodots ;; AIX-*) AS_IF([test "$GCC" != "yes"], [ # AIX requires the _r compiler when gcc isn't being used case "${CC}" in *_r|*_r\ *) # ok ... ;; *) # Make sure only first arg gets _r |
︙ | ︙ | |||
1382 1383 1384 1385 1386 1387 1388 | ]) ]) AS_IF([test "`uname -m`" = ia64], [ # AIX-5 uses ELF style dynamic libraries on IA-64, but not PPC SHLIB_LD="/usr/ccs/bin/ld -G -z text" AS_IF([test "$GCC" = yes], [ | | | | | > > > > > > > > > > > > > > > > | < < < < > > | | < < < < | > > > > > > > > > > > > > > > > > > > > > | | | 1312 1313 1314 1315 1316 1317 1318 1319 1320 1321 1322 1323 1324 1325 1326 1327 1328 1329 1330 1331 1332 1333 1334 1335 1336 1337 1338 1339 1340 1341 1342 1343 1344 1345 1346 1347 1348 1349 1350 1351 1352 1353 1354 1355 1356 1357 1358 1359 1360 1361 1362 1363 1364 1365 1366 1367 1368 1369 1370 1371 1372 1373 1374 1375 1376 1377 1378 1379 1380 1381 1382 1383 1384 1385 1386 1387 1388 1389 1390 1391 1392 1393 1394 1395 1396 1397 1398 1399 1400 1401 1402 1403 1404 1405 1406 1407 1408 1409 1410 1411 1412 1413 1414 1415 1416 1417 1418 1419 1420 1421 1422 1423 1424 1425 1426 1427 1428 1429 1430 1431 1432 1433 1434 1435 1436 1437 1438 1439 1440 1441 1442 1443 1444 1445 1446 1447 1448 1449 1450 1451 1452 1453 1454 1455 1456 1457 1458 1459 1460 1461 1462 1463 1464 1465 1466 1467 1468 1469 1470 1471 1472 | ]) ]) AS_IF([test "`uname -m`" = ia64], [ # AIX-5 uses ELF style dynamic libraries on IA-64, but not PPC SHLIB_LD="/usr/ccs/bin/ld -G -z text" AS_IF([test "$GCC" = yes], [ CC_SEARCH_FLAGS='"-Wl,-R,${LIB_RUNTIME_DIR}"' ], [ CC_SEARCH_FLAGS='"-R${LIB_RUNTIME_DIR}"' ]) LD_SEARCH_FLAGS='-R "${LIB_RUNTIME_DIR}"' ], [ AS_IF([test "$GCC" = yes], [ SHLIB_LD='${CC} -shared -Wl,-bexpall' ], [ SHLIB_LD="/bin/ld -bhalt:4 -bM:SRE -bexpall -H512 -T512 -bnoentry" LDFLAGS="$LDFLAGS -brtl" ]) SHLIB_LD="${SHLIB_LD} ${SHLIB_LD_FLAGS}" CC_SEARCH_FLAGS='"-L${LIB_RUNTIME_DIR}"' LD_SEARCH_FLAGS=${CC_SEARCH_FLAGS} ]) ;; BeOS*) SHLIB_CFLAGS="-fPIC" SHLIB_LD='${CC} -nostart' SHLIB_SUFFIX=".so" #----------------------------------------------------------- # Check for inet_ntoa in -lbind, for BeOS (which also needs # -lsocket, even if the network functions are in -lnet which # is always linked to, for compatibility. #----------------------------------------------------------- AC_CHECK_LIB(bind, inet_ntoa, [LIBS="$LIBS -lbind -lsocket"]) ;; BSD/OS-2.1*|BSD/OS-3*) SHLIB_CFLAGS="" SHLIB_LD="shlicc -r" SHLIB_SUFFIX=".so" CC_SEARCH_FLAGS="" LD_SEARCH_FLAGS="" ;; BSD/OS-4.*) SHLIB_CFLAGS="-export-dynamic -fPIC" SHLIB_LD='${CC} -shared' SHLIB_SUFFIX=".so" LDFLAGS="$LDFLAGS -export-dynamic" CC_SEARCH_FLAGS="" LD_SEARCH_FLAGS="" ;; CYGWIN_*) SHLIB_CFLAGS="" SHLIB_LD='${CC} -shared' SHLIB_SUFFIX=".dll" SHLIB_LD_LIBS="${SHLIB_LD_LIBS} -Wl,--out-implib,\$[@].a" EXEEXT=".exe" do64bit_ok=yes CC_SEARCH_FLAGS="" LD_SEARCH_FLAGS="" ;; dgux*) SHLIB_CFLAGS="-K PIC" SHLIB_LD='${CC} -G' SHLIB_LD_LIBS="" SHLIB_SUFFIX=".so" CC_SEARCH_FLAGS="" LD_SEARCH_FLAGS="" ;; Haiku*) LDFLAGS="$LDFLAGS -Wl,--export-dynamic" SHLIB_CFLAGS="-fPIC" SHLIB_SUFFIX=".so" SHLIB_LD='${CC} ${CFLAGS} ${LDFLAGS} -shared' AC_CHECK_LIB(network, inet_ntoa, [LIBS="$LIBS -lnetwork"]) ;; HP-UX-*.11.*) # Use updated header definitions where possible AC_DEFINE(_XOPEN_SOURCE_EXTENDED, 1, [Do we want to use the XOPEN network library?]) # TEA specific: Needed by Tcl, but not most extensions #AC_DEFINE(_XOPEN_SOURCE, 1, [Do we want to use the XOPEN network library?]) #LIBS="$LIBS -lxnet" # Use the XOPEN network library AS_IF([test "`uname -m`" = ia64], [ SHLIB_SUFFIX=".so" ], [ SHLIB_SUFFIX=".sl" ]) AC_CHECK_LIB(dld, shl_load, tcl_ok=yes, tcl_ok=no) AS_IF([test "$tcl_ok" = yes], [ SHLIB_CFLAGS="+z" SHLIB_LD="ld -b" LDFLAGS="$LDFLAGS -Wl,-E" CC_SEARCH_FLAGS='"-Wl,+s,+b,${LIB_RUNTIME_DIR}:."' LD_SEARCH_FLAGS='+s +b "${LIB_RUNTIME_DIR}:."' LD_LIBRARY_PATH_VAR="SHLIB_PATH" ]) AS_IF([test "$GCC" = yes], [ SHLIB_LD='${CC} -shared' LD_SEARCH_FLAGS=${CC_SEARCH_FLAGS} ], [ CFLAGS="$CFLAGS -z" ]) # Check to enable 64-bit flags for compiler/linker AS_IF([test "$do64bit" = "yes"], [ AS_IF([test "$GCC" = yes], [ case `${CC} -dumpmachine` in hppa64*) # 64-bit gcc in use. Fix flags for GNU ld. do64bit_ok=yes SHLIB_LD='${CC} -shared' AS_IF([test $doRpath = yes], [ CC_SEARCH_FLAGS='"-Wl,-rpath,${LIB_RUNTIME_DIR}"']) LD_SEARCH_FLAGS=${CC_SEARCH_FLAGS} ;; *) AC_MSG_WARN([64bit mode not supported with GCC on $system]) ;; esac ], [ do64bit_ok=yes CFLAGS="$CFLAGS +DD64" LDFLAGS_ARCH="+DD64" ]) ]) ;; HP-UX-*.08.*|HP-UX-*.09.*|HP-UX-*.10.*) SHLIB_SUFFIX=".sl" AC_CHECK_LIB(dld, shl_load, tcl_ok=yes, tcl_ok=no) AS_IF([test "$tcl_ok" = yes], [ SHLIB_CFLAGS="+z" SHLIB_LD="ld -b" SHLIB_LD_LIBS="" LDFLAGS="$LDFLAGS -Wl,-E" CC_SEARCH_FLAGS='"-Wl,+s,+b,${LIB_RUNTIME_DIR}:."' LD_SEARCH_FLAGS='+s +b "${LIB_RUNTIME_DIR}:."' LD_LIBRARY_PATH_VAR="SHLIB_PATH" ]) ;; IRIX-5.*) SHLIB_CFLAGS="" SHLIB_LD="ld -shared -rdata_shared" SHLIB_SUFFIX=".so" AC_LIBOBJ(mkstemp) AS_IF([test $doRpath = yes], [ CC_SEARCH_FLAGS='"-Wl,-rpath,${LIB_RUNTIME_DIR}"' LD_SEARCH_FLAGS='-rpath "${LIB_RUNTIME_DIR}"']) ;; IRIX-6.*) SHLIB_CFLAGS="" SHLIB_LD="ld -n32 -shared -rdata_shared" SHLIB_SUFFIX=".so" AS_IF([test $doRpath = yes], [ CC_SEARCH_FLAGS='"-Wl,-rpath,${LIB_RUNTIME_DIR}"' LD_SEARCH_FLAGS='-rpath "${LIB_RUNTIME_DIR}"']) AS_IF([test "$GCC" = yes], [ CFLAGS="$CFLAGS -mabi=n32" LDFLAGS="$LDFLAGS -mabi=n32" ], [ case $system in IRIX-6.3) # Use to build 6.2 compatible binaries on 6.3. |
︙ | ︙ | |||
1519 1520 1521 1522 1523 1524 1525 | ]) ;; IRIX64-6.*) SHLIB_CFLAGS="" SHLIB_LD="ld -n32 -shared -rdata_shared" SHLIB_SUFFIX=".so" AS_IF([test $doRpath = yes], [ | | | | | > > > > > > > > > > > | > | | 1480 1481 1482 1483 1484 1485 1486 1487 1488 1489 1490 1491 1492 1493 1494 1495 1496 1497 1498 1499 1500 1501 1502 1503 1504 1505 1506 1507 1508 1509 1510 1511 1512 1513 1514 1515 1516 1517 1518 1519 1520 1521 1522 1523 1524 1525 1526 1527 1528 1529 1530 1531 1532 1533 1534 1535 1536 1537 1538 1539 1540 | ]) ;; IRIX64-6.*) SHLIB_CFLAGS="" SHLIB_LD="ld -n32 -shared -rdata_shared" SHLIB_SUFFIX=".so" AS_IF([test $doRpath = yes], [ CC_SEARCH_FLAGS='"-Wl,-rpath,${LIB_RUNTIME_DIR}"' LD_SEARCH_FLAGS='-rpath "${LIB_RUNTIME_DIR}"']) # Check to enable 64-bit flags for compiler/linker AS_IF([test "$do64bit" = yes], [ AS_IF([test "$GCC" = yes], [ AC_MSG_WARN([64bit mode not supported by gcc]) ], [ do64bit_ok=yes SHLIB_LD="ld -64 -shared -rdata_shared" CFLAGS="$CFLAGS -64" LDFLAGS_ARCH="-64" ]) ]) ;; Linux*|GNU*|NetBSD-Debian|DragonFly-*|FreeBSD-*) SHLIB_CFLAGS="-fPIC" SHLIB_SUFFIX=".so" # TEA specific: CFLAGS_OPTIMIZE="-O2 -fomit-frame-pointer" # TEA specific: use LDFLAGS_DEFAULT instead of LDFLAGS SHLIB_LD='${CC} ${CFLAGS} ${LDFLAGS_DEFAULT} -shared' LDFLAGS="$LDFLAGS -Wl,--export-dynamic" case $system in DragonFly-*|FreeBSD-*) AS_IF([test "${TCL_THREADS}" = "1"], [ # The -pthread needs to go in the LDFLAGS, not LIBS LIBS=`echo $LIBS | sed s/-pthread//` CFLAGS="$CFLAGS $PTHREAD_CFLAGS" LDFLAGS="$LDFLAGS $PTHREAD_LIBS"]) ;; esac AS_IF([test $doRpath = yes], [ CC_SEARCH_FLAGS='"-Wl,-rpath,${LIB_RUNTIME_DIR}"']) LD_SEARCH_FLAGS=${CC_SEARCH_FLAGS} AS_IF([test "`uname -m`" = "alpha"], [CFLAGS="$CFLAGS -mieee"]) AS_IF([test $do64bit = yes], [ AC_CACHE_CHECK([if compiler accepts -m64 flag], tcl_cv_cc_m64, [ hold_cflags=$CFLAGS CFLAGS="$CFLAGS -m64" AC_LINK_IFELSE([AC_LANG_PROGRAM([[]], [[]])], [tcl_cv_cc_m64=yes],[tcl_cv_cc_m64=no]) CFLAGS=$hold_cflags]) AS_IF([test $tcl_cv_cc_m64 = yes], [ CFLAGS="$CFLAGS -m64" do64bit_ok=yes ]) ]) |
︙ | ︙ | |||
1576 1577 1578 1579 1580 1581 1582 | Lynx*) SHLIB_CFLAGS="-fPIC" SHLIB_SUFFIX=".so" CFLAGS_OPTIMIZE=-02 SHLIB_LD='${CC} -shared' LD_FLAGS="-Wl,--export-dynamic" AS_IF([test $doRpath = yes], [ | | | | < < | | > > | | | | | | | < < < < < < < | < < < | | | | < | | < | | | | < < < < < < < < < < < < < < < < < < < < < < < < < < < | | | | > | < < < < | | | 1549 1550 1551 1552 1553 1554 1555 1556 1557 1558 1559 1560 1561 1562 1563 1564 1565 1566 1567 1568 1569 1570 1571 1572 1573 1574 1575 1576 1577 1578 1579 1580 1581 1582 1583 1584 1585 1586 1587 1588 1589 1590 1591 1592 1593 1594 1595 1596 1597 1598 1599 1600 1601 1602 1603 1604 1605 1606 1607 1608 1609 1610 1611 1612 1613 1614 1615 1616 1617 1618 1619 1620 1621 1622 1623 1624 1625 1626 1627 1628 1629 1630 1631 1632 1633 1634 1635 1636 1637 1638 1639 1640 1641 1642 1643 1644 1645 1646 1647 1648 1649 1650 1651 1652 1653 1654 1655 1656 1657 1658 1659 1660 1661 1662 1663 1664 1665 1666 1667 1668 1669 1670 1671 1672 1673 1674 | Lynx*) SHLIB_CFLAGS="-fPIC" SHLIB_SUFFIX=".so" CFLAGS_OPTIMIZE=-02 SHLIB_LD='${CC} -shared' LD_FLAGS="-Wl,--export-dynamic" AS_IF([test $doRpath = yes], [ CC_SEARCH_FLAGS='"-Wl,-rpath,${LIB_RUNTIME_DIR}"' LD_SEARCH_FLAGS='"-Wl,-rpath,${LIB_RUNTIME_DIR}"']) ;; OpenBSD-*) arch=`arch -s` case "$arch" in alpha|sparc64) SHLIB_CFLAGS="-fPIC" ;; *) SHLIB_CFLAGS="-fpic" ;; esac SHLIB_LD='${CC} ${SHLIB_CFLAGS} -shared' SHLIB_SUFFIX=".so" AS_IF([test $doRpath = yes], [ CC_SEARCH_FLAGS='"-Wl,-rpath,${LIB_RUNTIME_DIR}"']) LD_SEARCH_FLAGS=${CC_SEARCH_FLAGS} SHARED_LIB_SUFFIX='${TCL_TRIM_DOTS}.so${SHLIB_VERSION}' LDFLAGS="$LDFLAGS -Wl,-export-dynamic" CFLAGS_OPTIMIZE="-O2" # On OpenBSD: Compile with -pthread # Don't link with -lpthread LIBS=`echo $LIBS | sed s/-lpthread//` CFLAGS="$CFLAGS -pthread" # OpenBSD doesn't do version numbers with dots. UNSHARED_LIB_SUFFIX='${TCL_TRIM_DOTS}.a' TCL_LIB_VERSIONS_OK=nodots ;; NetBSD-*) # NetBSD has ELF and can use 'cc -shared' to build shared libs SHLIB_CFLAGS="-fPIC" SHLIB_LD='${CC} ${SHLIB_CFLAGS} -shared' SHLIB_SUFFIX=".so" LDFLAGS="$LDFLAGS -export-dynamic" AS_IF([test $doRpath = yes], [ CC_SEARCH_FLAGS='"-Wl,-rpath,${LIB_RUNTIME_DIR}"']) LD_SEARCH_FLAGS=${CC_SEARCH_FLAGS} # The -pthread needs to go in the CFLAGS, not LIBS LIBS=`echo $LIBS | sed s/-pthread//` CFLAGS="$CFLAGS -pthread" LDFLAGS="$LDFLAGS -pthread" ;; Darwin-*) CFLAGS_OPTIMIZE="-Os" SHLIB_CFLAGS="-fno-common" # To avoid discrepancies between what headers configure sees during # preprocessing tests and compiling tests, move any -isysroot and # -mmacosx-version-min flags from CFLAGS to CPPFLAGS: CPPFLAGS="${CPPFLAGS} `echo " ${CFLAGS}" | \ awk 'BEGIN {FS=" +-";ORS=" "}; {for (i=2;i<=NF;i++) \ if ([$]i~/^(isysroot|mmacosx-version-min)/) print "-"[$]i}'`" CFLAGS="`echo " ${CFLAGS}" | \ awk 'BEGIN {FS=" +-";ORS=" "}; {for (i=2;i<=NF;i++) \ if (!([$]i~/^(isysroot|mmacosx-version-min)/)) print "-"[$]i}'`" AS_IF([test $do64bit = yes], [ case `arch` in ppc) AC_CACHE_CHECK([if compiler accepts -arch ppc64 flag], tcl_cv_cc_arch_ppc64, [ hold_cflags=$CFLAGS CFLAGS="$CFLAGS -arch ppc64 -mpowerpc64 -mcpu=G5" AC_LINK_IFELSE([AC_LANG_PROGRAM([[]], [[]])], [tcl_cv_cc_arch_ppc64=yes],[tcl_cv_cc_arch_ppc64=no]) CFLAGS=$hold_cflags]) AS_IF([test $tcl_cv_cc_arch_ppc64 = yes], [ CFLAGS="$CFLAGS -arch ppc64 -mpowerpc64 -mcpu=G5" do64bit_ok=yes ]);; i386) AC_CACHE_CHECK([if compiler accepts -arch x86_64 flag], tcl_cv_cc_arch_x86_64, [ hold_cflags=$CFLAGS CFLAGS="$CFLAGS -arch x86_64" AC_LINK_IFELSE([AC_LANG_PROGRAM([[]], [[]])], [tcl_cv_cc_arch_x86_64=yes],[tcl_cv_cc_arch_x86_64=no]) CFLAGS=$hold_cflags]) AS_IF([test $tcl_cv_cc_arch_x86_64 = yes], [ CFLAGS="$CFLAGS -arch x86_64" do64bit_ok=yes ]);; *) AC_MSG_WARN([Don't know how enable 64-bit on architecture `arch`]);; esac ], [ # Check for combined 32-bit and 64-bit fat build AS_IF([echo "$CFLAGS " |grep -E -q -- '-arch (ppc64|x86_64) ' \ && echo "$CFLAGS " |grep -E -q -- '-arch (ppc|i386) '], [ fat_32_64=yes]) ]) # TEA specific: use LDFLAGS_DEFAULT instead of LDFLAGS SHLIB_LD='${CC} -dynamiclib ${CFLAGS} ${LDFLAGS_DEFAULT}' AC_CACHE_CHECK([if ld accepts -single_module flag], tcl_cv_ld_single_module, [ hold_ldflags=$LDFLAGS LDFLAGS="$LDFLAGS -dynamiclib -Wl,-single_module" AC_LINK_IFELSE([AC_LANG_PROGRAM([[]], [[int i;]])], [tcl_cv_ld_single_module=yes],[tcl_cv_ld_single_module=no]) LDFLAGS=$hold_ldflags]) AS_IF([test $tcl_cv_ld_single_module = yes], [ SHLIB_LD="${SHLIB_LD} -Wl,-single_module" ]) # TEA specific: link shlib with current and compatibility version flags vers=`echo ${PACKAGE_VERSION} | sed -e 's/^\([[0-9]]\{1,5\}\)\(\(\.[[0-9]]\{1,3\}\)\{0,2\}\).*$/\1\2/p' -e d` SHLIB_LD="${SHLIB_LD} -current_version ${vers:-0} -compatibility_version ${vers:-0}" SHLIB_SUFFIX=".dylib" LDFLAGS="$LDFLAGS -headerpad_max_install_names" AC_CACHE_CHECK([if ld accepts -search_paths_first flag], tcl_cv_ld_search_paths_first, [ hold_ldflags=$LDFLAGS LDFLAGS="$LDFLAGS -Wl,-search_paths_first" AC_LINK_IFELSE([AC_LANG_PROGRAM([[]], [[int i;]])], [tcl_cv_ld_search_paths_first=yes],[tcl_cv_ld_search_paths_first=no]) LDFLAGS=$hold_ldflags]) AS_IF([test $tcl_cv_ld_search_paths_first = yes], [ LDFLAGS="$LDFLAGS -Wl,-search_paths_first" ]) AS_IF([test "$tcl_cv_cc_visibility_hidden" != yes], [ AC_DEFINE(MODULE_SCOPE, [__private_extern__], [Compiler support for module scope symbols]) |
︙ | ︙ | |||
1752 1753 1754 1755 1756 1757 1758 | AS_IF([test "${TEA_WINDOWINGSYSTEM}" = x11], [ AC_CACHE_CHECK([for 64-bit X11], tcl_cv_lib_x11_64, [ for v in CFLAGS CPPFLAGS LDFLAGS; do eval 'hold_'$v'="$'$v'";'$v'="`echo "$'$v' "|sed -e "s/-arch ppc / /g" -e "s/-arch i386 / /g"`"' done CPPFLAGS="$CPPFLAGS -I/usr/X11R6/include" LDFLAGS="$LDFLAGS -L/usr/X11R6/lib -lX11" | | | | | | 1683 1684 1685 1686 1687 1688 1689 1690 1691 1692 1693 1694 1695 1696 1697 1698 1699 1700 1701 1702 1703 1704 1705 1706 1707 1708 1709 1710 1711 | AS_IF([test "${TEA_WINDOWINGSYSTEM}" = x11], [ AC_CACHE_CHECK([for 64-bit X11], tcl_cv_lib_x11_64, [ for v in CFLAGS CPPFLAGS LDFLAGS; do eval 'hold_'$v'="$'$v'";'$v'="`echo "$'$v' "|sed -e "s/-arch ppc / /g" -e "s/-arch i386 / /g"`"' done CPPFLAGS="$CPPFLAGS -I/usr/X11R6/include" LDFLAGS="$LDFLAGS -L/usr/X11R6/lib -lX11" AC_LINK_IFELSE([AC_LANG_PROGRAM([[#include <X11/Xlib.h>]], [[XrmInitialize();]])], [tcl_cv_lib_x11_64=yes],[tcl_cv_lib_x11_64=no]) for v in CFLAGS CPPFLAGS LDFLAGS; do eval $v'="$hold_'$v'"' done]) ]) AS_IF([test "${TEA_WINDOWINGSYSTEM}" = aqua], [ AC_CACHE_CHECK([for 64-bit Tk], tcl_cv_lib_tk_64, [ for v in CFLAGS CPPFLAGS LDFLAGS; do eval 'hold_'$v'="$'$v'";'$v'="`echo "$'$v' "|sed -e "s/-arch ppc / /g" -e "s/-arch i386 / /g"`"' done CPPFLAGS="$CPPFLAGS -DUSE_TCL_STUBS=1 -DUSE_TK_STUBS=1 ${TCL_INCLUDES} ${TK_INCLUDES}" LDFLAGS="$LDFLAGS ${TCL_STUB_LIB_SPEC} ${TK_STUB_LIB_SPEC}" AC_LINK_IFELSE([AC_LANG_PROGRAM([[#include <tk.h>]], [[Tk_InitStubs(NULL, "", 0);]])], [tcl_cv_lib_tk_64=yes],[tcl_cv_lib_tk_64=no]) for v in CFLAGS CPPFLAGS LDFLAGS; do eval $v'="$hold_'$v'"' done]) ]) # remove 64-bit arch flags from CFLAGS et al. if configuration # does not support 64-bit. AS_IF([test "$tcl_cv_lib_tk_64" = no -o "$tcl_cv_lib_x11_64" = no], [ |
︙ | ︙ | |||
1795 1796 1797 1798 1799 1800 1801 | AS_IF([test "$SHARED_BUILD" = 1], [ SHLIB_LD='ld -shared -expect_unresolved "*"' ], [ SHLIB_LD='ld -non_shared -expect_unresolved "*"' ]) SHLIB_SUFFIX=".so" AS_IF([test $doRpath = yes], [ | | < | | | | | | | | < | 1726 1727 1728 1729 1730 1731 1732 1733 1734 1735 1736 1737 1738 1739 1740 1741 1742 1743 1744 1745 1746 1747 1748 1749 1750 1751 1752 | AS_IF([test "$SHARED_BUILD" = 1], [ SHLIB_LD='ld -shared -expect_unresolved "*"' ], [ SHLIB_LD='ld -non_shared -expect_unresolved "*"' ]) SHLIB_SUFFIX=".so" AS_IF([test $doRpath = yes], [ CC_SEARCH_FLAGS='"-Wl,-rpath,${LIB_RUNTIME_DIR}"' LD_SEARCH_FLAGS='-rpath ${LIB_RUNTIME_DIR}']) AS_IF([test "$GCC" = yes], [CFLAGS="$CFLAGS -mieee"], [ CFLAGS="$CFLAGS -DHAVE_TZSET -std1 -ieee"]) # see pthread_intro(3) for pthread support on osf1, k.furukawa CFLAGS="$CFLAGS -DHAVE_PTHREAD_ATTR_SETSTACKSIZE" CFLAGS="$CFLAGS -DTCL_THREAD_STACK_MIN=PTHREAD_STACK_MIN*64" LIBS=`echo $LIBS | sed s/-lpthreads//` AS_IF([test "$GCC" = yes], [ LIBS="$LIBS -lpthread -lmach -lexc" ], [ CFLAGS="$CFLAGS -pthread" LDFLAGS="$LDFLAGS -pthread" ]) ;; QNX-6*) # QNX RTP # This may work for all QNX, but it was only reported for v6. SHLIB_CFLAGS="-fPIC" SHLIB_LD="ld -Bshareable -x" |
︙ | ︙ | |||
1850 1851 1852 1853 1854 1855 1856 | AC_DEFINE(_POSIX_PTHREAD_SEMANTICS, 1, [Do we really want to follow the standard? Yes we do!]) SHLIB_CFLAGS="-KPIC" SHLIB_SUFFIX=".so" AS_IF([test "$GCC" = yes], [ SHLIB_LD='${CC} -shared' | | | | 1779 1780 1781 1782 1783 1784 1785 1786 1787 1788 1789 1790 1791 1792 1793 1794 1795 1796 1797 | AC_DEFINE(_POSIX_PTHREAD_SEMANTICS, 1, [Do we really want to follow the standard? Yes we do!]) SHLIB_CFLAGS="-KPIC" SHLIB_SUFFIX=".so" AS_IF([test "$GCC" = yes], [ SHLIB_LD='${CC} -shared' CC_SEARCH_FLAGS='"-Wl,-R,${LIB_RUNTIME_DIR}"' LD_SEARCH_FLAGS=${CC_SEARCH_FLAGS} ], [ SHLIB_LD="/usr/ccs/bin/ld -G -z text" CC_SEARCH_FLAGS='-R "${LIB_RUNTIME_DIR}"' LD_SEARCH_FLAGS=${CC_SEARCH_FLAGS} ]) ;; SunOS-5*) # Note: If _REENTRANT isn't defined, then Solaris # won't define thread-safe library routines. |
︙ | ︙ | |||
1920 1921 1922 1923 1924 1925 1926 | ]) ], [AC_MSG_WARN([64bit mode not supported for $arch])])]) ]) SHLIB_SUFFIX=".so" AS_IF([test "$GCC" = yes], [ SHLIB_LD='${CC} -shared' | | | 1849 1850 1851 1852 1853 1854 1855 1856 1857 1858 1859 1860 1861 1862 1863 | ]) ], [AC_MSG_WARN([64bit mode not supported for $arch])])]) ]) SHLIB_SUFFIX=".so" AS_IF([test "$GCC" = yes], [ SHLIB_LD='${CC} -shared' CC_SEARCH_FLAGS='"-Wl,-R,${LIB_RUNTIME_DIR}"' LD_SEARCH_FLAGS=${CC_SEARCH_FLAGS} AS_IF([test "$do64bit_ok" = yes], [ AS_IF([test "$arch" = "sparcv9 sparc"], [ # We need to specify -static-libgcc or we need to # add the path to the sparv9 libgcc. # JH: static-libgcc is necessary for core Tcl, but may # not be necessary for extensions. |
︙ | ︙ | |||
1947 1948 1949 1950 1951 1952 1953 | case $system in SunOS-5.[[1-9]][[0-9]]*) # TEA specific: use LDFLAGS_DEFAULT instead of LDFLAGS SHLIB_LD='${CC} -G -z text ${LDFLAGS_DEFAULT}';; *) SHLIB_LD='/usr/ccs/bin/ld -G -z text';; esac | | | > | | 1876 1877 1878 1879 1880 1881 1882 1883 1884 1885 1886 1887 1888 1889 1890 1891 1892 1893 1894 1895 1896 1897 1898 1899 1900 1901 1902 1903 1904 1905 | case $system in SunOS-5.[[1-9]][[0-9]]*) # TEA specific: use LDFLAGS_DEFAULT instead of LDFLAGS SHLIB_LD='${CC} -G -z text ${LDFLAGS_DEFAULT}';; *) SHLIB_LD='/usr/ccs/bin/ld -G -z text';; esac CC_SEARCH_FLAGS='"-Wl,-R,${LIB_RUNTIME_DIR}"' LD_SEARCH_FLAGS='-R "${LIB_RUNTIME_DIR}"' ]) ;; UNIX_SV* | UnixWare-5*) SHLIB_CFLAGS="-KPIC" SHLIB_LD='${CC} -G' SHLIB_LD_LIBS="" SHLIB_SUFFIX=".so" # Some UNIX_SV* systems (unixware 1.1.2 for example) have linkers # that don't grok the -Bexport option. Test that it does. AC_CACHE_CHECK([for ld accepts -Bexport flag], tcl_cv_ld_Bexport, [ hold_ldflags=$LDFLAGS LDFLAGS="$LDFLAGS -Wl,-Bexport" AC_LINK_IFELSE([AC_LANG_PROGRAM([[]], [[int i;]])], [tcl_cv_ld_Bexport=yes],[tcl_cv_ld_Bexport=no]) LDFLAGS=$hold_ldflags]) AS_IF([test $tcl_cv_ld_Bexport = yes], [ LDFLAGS="$LDFLAGS -Wl,-Bexport" ]) CC_SEARCH_FLAGS="" LD_SEARCH_FLAGS="" ;; |
︙ | ︙ | |||
1993 1994 1995 1996 1997 1998 1999 | # libraries to the right flags for gcc, instead of those for the # standard manufacturer compiler. AS_IF([test "$GCC" = yes], [ case $system in AIX-*) ;; BSD/OS*) ;; | | | | 1923 1924 1925 1926 1927 1928 1929 1930 1931 1932 1933 1934 1935 1936 1937 1938 1939 | # libraries to the right flags for gcc, instead of those for the # standard manufacturer compiler. AS_IF([test "$GCC" = yes], [ case $system in AIX-*) ;; BSD/OS*) ;; CYGWIN_*|MINGW32_*|MINGW64_*|MSYS_*) ;; IRIX*) ;; NetBSD-*|DragonFly-*|FreeBSD-*|OpenBSD-*) ;; Darwin-*) ;; SCO_SV-3.2*) ;; windows) ;; *) SHLIB_CFLAGS="-fPIC" ;; esac]) AS_IF([test "$tcl_cv_cc_visibility_hidden" != yes], [ |
︙ | ︙ | |||
2017 2018 2019 2020 2021 2022 2023 | AS_IF([test "$UNSHARED_LIB_SUFFIX" = ""], [ # TEA specific: use PACKAGE_VERSION instead of VERSION UNSHARED_LIB_SUFFIX='${PACKAGE_VERSION}.a']) if test "${GCC}" = "yes" -a ${SHLIB_SUFFIX} = ".dll"; then AC_CACHE_CHECK(for SEH support in compiler, tcl_cv_seh, | | | | | | | | | | | | | | | | | < | | | > > > > | 1947 1948 1949 1950 1951 1952 1953 1954 1955 1956 1957 1958 1959 1960 1961 1962 1963 1964 1965 1966 1967 1968 1969 1970 1971 1972 1973 1974 1975 1976 1977 1978 1979 1980 1981 1982 1983 1984 1985 1986 1987 1988 1989 1990 1991 1992 1993 1994 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013 2014 2015 2016 2017 2018 2019 2020 2021 2022 2023 2024 2025 2026 2027 2028 2029 2030 2031 2032 2033 2034 2035 2036 2037 2038 2039 2040 2041 2042 2043 2044 2045 2046 2047 2048 2049 2050 2051 2052 2053 2054 2055 2056 2057 2058 | AS_IF([test "$UNSHARED_LIB_SUFFIX" = ""], [ # TEA specific: use PACKAGE_VERSION instead of VERSION UNSHARED_LIB_SUFFIX='${PACKAGE_VERSION}.a']) if test "${GCC}" = "yes" -a ${SHLIB_SUFFIX} = ".dll"; then AC_CACHE_CHECK(for SEH support in compiler, tcl_cv_seh, AC_RUN_IFELSE([AC_LANG_SOURCE([[ #define WIN32_LEAN_AND_MEAN #include <windows.h> #undef WIN32_LEAN_AND_MEAN int main(int argc, char** argv) { int a, b = 0; __try { a = 666 / b; } __except (EXCEPTION_EXECUTE_HANDLER) { return 0; } return 1; } ]])], [tcl_cv_seh=yes], [tcl_cv_seh=no], [tcl_cv_seh=no]) ) if test "$tcl_cv_seh" = "no" ; then AC_DEFINE(HAVE_NO_SEH, 1, [Defined when mingw does not support SEH]) fi # # Check to see if the excpt.h include file provided contains the # definition for EXCEPTION_DISPOSITION; if not, which is the case # with Cygwin's version as of 2002-04-10, define it to be int, # sufficient for getting the current code to work. # AC_CACHE_CHECK(for EXCEPTION_DISPOSITION support in include files, tcl_cv_eh_disposition, AC_COMPILE_IFELSE([AC_LANG_PROGRAM([[ # define WIN32_LEAN_AND_MEAN # include <windows.h> # undef WIN32_LEAN_AND_MEAN ]], [[ EXCEPTION_DISPOSITION x; ]])], [tcl_cv_eh_disposition=yes], [tcl_cv_eh_disposition=no]) ) if test "$tcl_cv_eh_disposition" = "no" ; then AC_DEFINE(EXCEPTION_DISPOSITION, int, [Defined when cygwin/mingw does not support EXCEPTION DISPOSITION]) fi # Check to see if winnt.h defines CHAR, SHORT, and LONG # even if VOID has already been #defined. The win32api # used by mingw and cygwin is known to do this. AC_CACHE_CHECK(for winnt.h that ignores VOID define, tcl_cv_winnt_ignore_void, AC_COMPILE_IFELSE([AC_LANG_PROGRAM([[ #define VOID void #define WIN32_LEAN_AND_MEAN #include <windows.h> #undef WIN32_LEAN_AND_MEAN ]], [[ CHAR c; SHORT s; LONG l; ]])], [tcl_cv_winnt_ignore_void=yes], [tcl_cv_winnt_ignore_void=no]) ) if test "$tcl_cv_winnt_ignore_void" = "yes" ; then AC_DEFINE(HAVE_WINNT_IGNORE_VOID, 1, [Defined when cygwin/mingw ignores VOID define in winnt.h]) fi fi # See if the compiler supports casting to a union type. # This is used to stop gcc from printing a compiler # warning when initializing a union member. AC_CACHE_CHECK(for cast to union support, tcl_cv_cast_to_union, AC_COMPILE_IFELSE([AC_LANG_PROGRAM([[]], [[ union foo { int i; double d; }; union foo f = (union foo) (int) 0; ]])], [tcl_cv_cast_to_union=yes], [tcl_cv_cast_to_union=no]) ) if test "$tcl_cv_cast_to_union" = "yes"; then AC_DEFINE(HAVE_CAST_TO_UNION, 1, [Defined when compiler supports casting to union type.]) fi AC_CHECK_HEADER(stdbool.h, [AC_DEFINE(HAVE_STDBOOL_H, 1, [Do we have <stdbool.h>?])],) AC_SUBST(CFLAGS_DEBUG) AC_SUBST(CFLAGS_OPTIMIZE) AC_SUBST(CFLAGS_WARNING) AC_SUBST(LDFLAGS_DEBUG) AC_SUBST(LDFLAGS_OPTIMIZE) AC_SUBST(STLIB_LD) AC_SUBST(SHLIB_LD) AC_SUBST(SHLIB_LD_LIBS) AC_SUBST(SHLIB_CFLAGS) |
︙ | ︙ | |||
2151 2152 2153 2154 2155 2156 2157 | # USE_TERMIO # USE_SGTTY #-------------------------------------------------------------------- AC_DEFUN([TEA_SERIAL_PORT], [ AC_CHECK_HEADERS(sys/modem.h) AC_CACHE_CHECK([termios vs. termio vs. sgtty], tcl_cv_api_serial, [ | | | | | | | | | | | | | < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < | 2084 2085 2086 2087 2088 2089 2090 2091 2092 2093 2094 2095 2096 2097 2098 2099 2100 2101 2102 2103 2104 2105 2106 2107 2108 2109 2110 2111 2112 2113 2114 2115 2116 2117 2118 2119 2120 2121 2122 2123 2124 2125 2126 2127 2128 2129 2130 2131 2132 2133 2134 2135 2136 2137 2138 2139 2140 2141 2142 2143 2144 2145 2146 2147 2148 2149 2150 2151 2152 2153 2154 2155 2156 2157 2158 2159 2160 2161 2162 2163 2164 2165 2166 2167 2168 2169 2170 2171 2172 2173 2174 2175 2176 2177 2178 2179 2180 2181 2182 2183 2184 2185 2186 2187 2188 2189 2190 | # USE_TERMIO # USE_SGTTY #-------------------------------------------------------------------- AC_DEFUN([TEA_SERIAL_PORT], [ AC_CHECK_HEADERS(sys/modem.h) AC_CACHE_CHECK([termios vs. termio vs. sgtty], tcl_cv_api_serial, [ AC_RUN_IFELSE([AC_LANG_SOURCE([[ #include <termios.h> int main() { struct termios t; if (tcgetattr(0, &t) == 0) { cfsetospeed(&t, 0); t.c_cflag |= PARENB | PARODD | CSIZE | CSTOPB; return 0; } return 1; }]])],[tcl_cv_api_serial=termios],[tcl_cv_api_serial=no],[tcl_cv_api_serial=no]) if test $tcl_cv_api_serial = no ; then AC_RUN_IFELSE([AC_LANG_SOURCE([[ #include <termio.h> int main() { struct termio t; if (ioctl(0, TCGETA, &t) == 0) { t.c_cflag |= CBAUD | PARENB | PARODD | CSIZE | CSTOPB; return 0; } return 1; }]])],[tcl_cv_api_serial=termio],[tcl_cv_api_serial=no],[tcl_cv_api_serial=no]) fi if test $tcl_cv_api_serial = no ; then AC_RUN_IFELSE([AC_LANG_SOURCE([[ #include <sgtty.h> int main() { struct sgttyb t; if (ioctl(0, TIOCGETP, &t) == 0) { t.sg_ospeed = 0; t.sg_flags |= ODDP | EVENP | RAW; return 0; } return 1; }]])],[tcl_cv_api_serial=sgtty],[tcl_cv_api_serial=no],[tcl_cv_api_serial=no]) fi if test $tcl_cv_api_serial = no ; then AC_RUN_IFELSE([AC_LANG_SOURCE([[ #include <termios.h> #include <errno.h> int main() { struct termios t; if (tcgetattr(0, &t) == 0 || errno == ENOTTY || errno == ENXIO || errno == EINVAL) { cfsetospeed(&t, 0); t.c_cflag |= PARENB | PARODD | CSIZE | CSTOPB; return 0; } return 1; }]])],[tcl_cv_api_serial=termios],[tcl_cv_api_serial=no],[tcl_cv_api_serial=no]) fi if test $tcl_cv_api_serial = no; then AC_RUN_IFELSE([AC_LANG_SOURCE([[ #include <termio.h> #include <errno.h> int main() { struct termio t; if (ioctl(0, TCGETA, &t) == 0 || errno == ENOTTY || errno == ENXIO || errno == EINVAL) { t.c_cflag |= CBAUD | PARENB | PARODD | CSIZE | CSTOPB; return 0; } return 1; }]])],[tcl_cv_api_serial=termio],[tcl_cv_api_serial=no],[tcl_cv_api_serial=no]) fi if test $tcl_cv_api_serial = no; then AC_RUN_IFELSE([AC_LANG_SOURCE([[ #include <sgtty.h> #include <errno.h> int main() { struct sgttyb t; if (ioctl(0, TIOCGETP, &t) == 0 || errno == ENOTTY || errno == ENXIO || errno == EINVAL) { t.sg_ospeed = 0; t.sg_flags |= ODDP | EVENP | RAW; return 0; } return 1; }]])],[tcl_cv_api_serial=sgtty],[tcl_cv_api_serial=none],[tcl_cv_api_serial=none]) fi]) case $tcl_cv_api_serial in termios) AC_DEFINE(USE_TERMIOS, 1, [Use the termios API for serial lines]);; termio) AC_DEFINE(USE_TERMIO, 1, [Use the termio API for serial lines]);; sgtty) AC_DEFINE(USE_SGTTY, 1, [Use the sgtty API for serial lines]);; esac ]) #-------------------------------------------------------------------- # TEA_PATH_X # # Locate the X11 header files and the X11 library archive. Try # the ac_path_x macro first, but if it doesn't find the X stuff # (e.g. because there's no xmkmf program) then check through # a list of possible directories. Under some conditions the |
︙ | ︙ | |||
2370 2371 2372 2373 2374 2375 2376 | ]) AC_DEFUN([TEA_PATH_UNIX_X], [ AC_PATH_X not_really_there="" if test "$no_x" = ""; then if test "$x_includes" = ""; then | | | | 2212 2213 2214 2215 2216 2217 2218 2219 2220 2221 2222 2223 2224 2225 2226 2227 2228 2229 2230 2231 2232 2233 2234 2235 2236 | ]) AC_DEFUN([TEA_PATH_UNIX_X], [ AC_PATH_X not_really_there="" if test "$no_x" = ""; then if test "$x_includes" = ""; then AC_PREPROC_IFELSE([AC_LANG_SOURCE([[#include <X11/Xlib.h>]])],[],[not_really_there="yes"]) else if test ! -r $x_includes/X11/Xlib.h; then not_really_there="yes" fi fi fi if test "$no_x" = "yes" -o "$not_really_there" = "yes"; then AC_MSG_CHECKING([for X11 header files]) found_xincludes="no" AC_PREPROC_IFELSE([AC_LANG_SOURCE([[#include <X11/Xlib.h>]])],[found_xincludes="yes"],[found_xincludes="no"]) if test "$found_xincludes" = "no"; then dirs="/usr/unsupported/include /usr/local/include /usr/X386/include /usr/X11R6/include /usr/X11R5/include /usr/include/X11R5 /usr/include/X11R4 /usr/openwin/include /usr/X11/include /usr/sww/include" for i in $dirs ; do if test -r $i/X11/Xlib.h; then AC_MSG_RESULT([$i]) XINCLUDES=" -I$i" found_xincludes="yes" |
︙ | ︙ | |||
2486 2487 2488 2489 2490 2491 2492 2493 2494 2495 2496 2497 2498 2499 | # Results: # # Defines some of the following vars: # USE_DELTA_FOR_TZ # HAVE_TM_GMTOFF # HAVE_TM_TZADJ # HAVE_TIMEZONE_VAR #-------------------------------------------------------------------- AC_DEFUN([TEA_TIME_HANDLER], [ AC_CHECK_HEADERS(sys/time.h) AC_HEADER_TIME AC_STRUCT_TIMEZONE | > | | | > | | > | > | | | | > | | | | 2328 2329 2330 2331 2332 2333 2334 2335 2336 2337 2338 2339 2340 2341 2342 2343 2344 2345 2346 2347 2348 2349 2350 2351 2352 2353 2354 2355 2356 2357 2358 2359 2360 2361 2362 2363 2364 2365 2366 2367 2368 2369 2370 2371 2372 2373 2374 2375 2376 2377 2378 2379 2380 2381 2382 2383 2384 2385 2386 2387 2388 2389 2390 2391 | # Results: # # Defines some of the following vars: # USE_DELTA_FOR_TZ # HAVE_TM_GMTOFF # HAVE_TM_TZADJ # HAVE_TIMEZONE_VAR # #-------------------------------------------------------------------- AC_DEFUN([TEA_TIME_HANDLER], [ AC_CHECK_HEADERS(sys/time.h) AC_HEADER_TIME AC_STRUCT_TIMEZONE AC_CHECK_FUNCS(gmtime_r localtime_r mktime) AC_CACHE_CHECK([tm_tzadj in struct tm], tcl_cv_member_tm_tzadj, [ AC_COMPILE_IFELSE([AC_LANG_PROGRAM([[#include <time.h>]], [[struct tm tm; (void)tm.tm_tzadj;]])], [tcl_cv_member_tm_tzadj=yes], [tcl_cv_member_tm_tzadj=no])]) if test $tcl_cv_member_tm_tzadj = yes ; then AC_DEFINE(HAVE_TM_TZADJ, 1, [Should we use the tm_tzadj field of struct tm?]) fi AC_CACHE_CHECK([tm_gmtoff in struct tm], tcl_cv_member_tm_gmtoff, [ AC_COMPILE_IFELSE([AC_LANG_PROGRAM([[#include <time.h>]], [[struct tm tm; (void)tm.tm_gmtoff;]])], [tcl_cv_member_tm_gmtoff=yes], [tcl_cv_member_tm_gmtoff=no])]) if test $tcl_cv_member_tm_gmtoff = yes ; then AC_DEFINE(HAVE_TM_GMTOFF, 1, [Should we use the tm_gmtoff field of struct tm?]) fi # # Its important to include time.h in this check, as some systems # (like convex) have timezone functions, etc. # AC_CACHE_CHECK([long timezone variable], tcl_cv_timezone_long, [ AC_COMPILE_IFELSE([AC_LANG_PROGRAM([[#include <time.h> #include <stdlib.h>]], [[extern long timezone; timezone += 1; exit (0);]])], [tcl_cv_timezone_long=yes], [tcl_cv_timezone_long=no])]) if test $tcl_cv_timezone_long = yes ; then AC_DEFINE(HAVE_TIMEZONE_VAR, 1, [Should we use the global timezone variable?]) else # # On some systems (eg IRIX 6.2), timezone is a time_t and not a long. # AC_CACHE_CHECK([time_t timezone variable], tcl_cv_timezone_time, [ AC_COMPILE_IFELSE([AC_LANG_PROGRAM([[#include <time.h> #include <stdlib.h>]], [[extern time_t timezone; timezone += 1; exit (0);]])], [tcl_cv_timezone_time=yes], [tcl_cv_timezone_time=no])]) if test $tcl_cv_timezone_time = yes ; then AC_DEFINE(HAVE_TIMEZONE_VAR, 1, [Should we use the global timezone variable?]) fi fi ]) #-------------------------------------------------------------------- |
︙ | ︙ | |||
2560 2561 2562 2563 2564 2565 2566 | # strtod (=fixstrtod) #-------------------------------------------------------------------- AC_DEFUN([TEA_BUGGY_STRTOD], [ AC_CHECK_FUNC(strtod, tcl_strtod=1, tcl_strtod=0) if test "$tcl_strtod" = 1; then AC_CACHE_CHECK([for Solaris2.4/Tru64 strtod bugs], tcl_cv_strtod_buggy,[ | | > | | | | > > > > | < < < < < < < < < < > < < < | 2407 2408 2409 2410 2411 2412 2413 2414 2415 2416 2417 2418 2419 2420 2421 2422 2423 2424 2425 2426 2427 2428 2429 2430 2431 2432 2433 2434 2435 2436 2437 2438 2439 2440 2441 2442 2443 2444 2445 2446 2447 2448 2449 2450 2451 2452 2453 2454 2455 2456 2457 2458 2459 2460 2461 2462 2463 2464 2465 2466 2467 2468 2469 2470 2471 2472 2473 2474 2475 2476 2477 2478 | # strtod (=fixstrtod) #-------------------------------------------------------------------- AC_DEFUN([TEA_BUGGY_STRTOD], [ AC_CHECK_FUNC(strtod, tcl_strtod=1, tcl_strtod=0) if test "$tcl_strtod" = 1; then AC_CACHE_CHECK([for Solaris2.4/Tru64 strtod bugs], tcl_cv_strtod_buggy,[ AC_RUN_IFELSE([AC_LANG_SOURCE([[ #include <stdlib.h> extern double strtod(); int main() { char *infString="Inf", *nanString="NaN", *spaceString=" "; char *term; double value; value = strtod(infString, &term); if ((term != infString) && (term[-1] == 0)) { exit(1); } value = strtod(nanString, &term); if ((term != nanString) && (term[-1] == 0)) { exit(1); } value = strtod(spaceString, &term); if (term == (spaceString+1)) { exit(1); } exit(0); }]])], [tcl_cv_strtod_buggy=ok], [tcl_cv_strtod_buggy=buggy], [tcl_cv_strtod_buggy=buggy])]) if test "$tcl_cv_strtod_buggy" = buggy; then AC_LIBOBJ([fixstrtod]) USE_COMPAT=1 AC_DEFINE(strtod, fixstrtod, [Do we want to use the strtod() in compat?]) fi fi ]) #-------------------------------------------------------------------- # TEA_TCL_LINK_LIBS # # Search for the libraries needed to link the Tcl shell. # Things like the math library (-lm), socket stuff (-lsocket vs. # -lnsl), zlib (-lz) and libtommath (-ltommath) are dealt with here. # # Arguments: # None. # # Results: # # Might append to the following vars: # LIBS # MATH_LIBS # # Might define the following vars: # HAVE_NET_ERRNO_H # #-------------------------------------------------------------------- AC_DEFUN([TEA_TCL_LINK_LIBS], [ #-------------------------------------------------------------------- # On a few very rare systems, all of the libm.a stuff is # already in libc.a. Set compiler flags accordingly. #-------------------------------------------------------------------- AC_CHECK_FUNC(sin, MATH_LIBS="", MATH_LIBS="-lm") #-------------------------------------------------------------------- # Interactive UNIX requires -linet instead of -lsocket, plus it # needs net/errno.h to define the socket-related error codes. #-------------------------------------------------------------------- AC_CHECK_LIB(inet, main, [LIBS="$LIBS -linet"]) |
︙ | ︙ | |||
2666 2667 2668 2669 2670 2671 2672 | if test "$tcl_checkBoth" = 1; then tk_oldLibs=$LIBS LIBS="$LIBS -lsocket -lnsl" AC_CHECK_FUNC(accept, tcl_checkNsl=0, [LIBS=$tk_oldLibs]) fi AC_CHECK_FUNC(gethostbyname, , [AC_CHECK_LIB(nsl, gethostbyname, [LIBS="$LIBS -lnsl"])]) | | < | | < | < > > | < | | | | 2506 2507 2508 2509 2510 2511 2512 2513 2514 2515 2516 2517 2518 2519 2520 2521 2522 2523 2524 2525 2526 2527 2528 2529 2530 2531 2532 2533 2534 2535 2536 2537 2538 2539 2540 2541 2542 2543 2544 2545 2546 2547 2548 2549 2550 | if test "$tcl_checkBoth" = 1; then tk_oldLibs=$LIBS LIBS="$LIBS -lsocket -lnsl" AC_CHECK_FUNC(accept, tcl_checkNsl=0, [LIBS=$tk_oldLibs]) fi AC_CHECK_FUNC(gethostbyname, , [AC_CHECK_LIB(nsl, gethostbyname, [LIBS="$LIBS -lnsl"])]) AC_CHECK_FUNC(mp_log_u32, , [AC_CHECK_LIB(tommath, mp_log_u32, [LIBS="$LIBS -ltommath"])]) AC_CHECK_FUNC(deflateSetHeader, , [AC_CHECK_LIB(z, deflateSetHeader, [LIBS="$LIBS -lz"])]) ]) #-------------------------------------------------------------------- # TEA_TCL_EARLY_FLAGS # # Check for what flags are needed to be passed so the correct OS # features are available. # # Arguments: # None # # Results: # # Might define the following vars: # _ISOC99_SOURCE # _LARGEFILE64_SOURCE # _LARGEFILE_SOURCE64 # #-------------------------------------------------------------------- AC_DEFUN([TEA_TCL_EARLY_FLAG],[ AC_CACHE_VAL([tcl_cv_flag_]translit($1,[A-Z],[a-z]), AC_COMPILE_IFELSE([AC_LANG_PROGRAM([[$2]], [[$3]])], [tcl_cv_flag_]translit($1,[A-Z],[a-z])=no,[AC_COMPILE_IFELSE([AC_LANG_PROGRAM([[[#define ]$1[ 1 ]$2]], [[$3]])], [tcl_cv_flag_]translit($1,[A-Z],[a-z])=yes, [tcl_cv_flag_]translit($1,[A-Z],[a-z])=no)])) if test ["x${tcl_cv_flag_]translit($1,[A-Z],[a-z])[}" = "xyes"] ; then AC_DEFINE($1, 1, [Add the ]$1[ flag when building]) tcl_flags="$tcl_flags $1" fi ]) AC_DEFUN([TEA_TCL_EARLY_FLAGS],[ |
︙ | ︙ | |||
2734 2735 2736 2737 2738 2739 2740 | # None # # Results: # # Might define the following vars: # TCL_WIDE_INT_IS_LONG # TCL_WIDE_INT_TYPE | | > | | | | | | | | | | | > > > > > > > > > | | | | | | | 2572 2573 2574 2575 2576 2577 2578 2579 2580 2581 2582 2583 2584 2585 2586 2587 2588 2589 2590 2591 2592 2593 2594 2595 2596 2597 2598 2599 2600 2601 2602 2603 2604 2605 2606 2607 2608 2609 2610 2611 2612 2613 2614 2615 2616 2617 2618 2619 2620 2621 2622 2623 2624 2625 2626 2627 2628 2629 2630 2631 2632 2633 2634 2635 2636 2637 2638 2639 2640 2641 2642 2643 2644 2645 2646 2647 2648 2649 | # None # # Results: # # Might define the following vars: # TCL_WIDE_INT_IS_LONG # TCL_WIDE_INT_TYPE # HAVE_STRUCT_DIRENT64, HAVE_DIR64 # HAVE_STRUCT_STAT64 # HAVE_TYPE_OFF64_T # #-------------------------------------------------------------------- AC_DEFUN([TEA_TCL_64BIT_FLAGS], [ AC_MSG_CHECKING([for 64-bit integer type]) AC_CACHE_VAL(tcl_cv_type_64bit,[ tcl_cv_type_64bit=none # See if the compiler knows natively about __int64 AC_COMPILE_IFELSE([AC_LANG_PROGRAM([[]], [[__int64 value = (__int64) 0;]])], [tcl_type_64bit=__int64],[tcl_type_64bit="long long"]) # See if we could use long anyway Note that we substitute in the # type that is our current guess for a 64-bit type inside this check # program, so it should be modified only carefully... AC_COMPILE_IFELSE([AC_LANG_PROGRAM([[]], [[switch (0) { case 1: case (sizeof(${tcl_type_64bit})==sizeof(long)): ; }]])],[tcl_cv_type_64bit=${tcl_type_64bit}],[])]) if test "${tcl_cv_type_64bit}" = none ; then AC_DEFINE(TCL_WIDE_INT_IS_LONG, 1, [Do 'long' and 'long long' have the same size (64-bit)?]) AC_MSG_RESULT([yes]) elif test "${tcl_cv_type_64bit}" = "__int64" \ -a "${TEA_PLATFORM}" = "windows" ; then # TEA specific: We actually want to use the default tcl.h checks in # this case to handle both TCL_WIDE_INT_TYPE and TCL_LL_MODIFIER* AC_MSG_RESULT([using Tcl header defaults]) else AC_DEFINE_UNQUOTED(TCL_WIDE_INT_TYPE,${tcl_cv_type_64bit}, [What type should be used to define wide integers?]) AC_MSG_RESULT([${tcl_cv_type_64bit}]) # Now check for auxiliary declarations AC_CACHE_CHECK([for struct dirent64], tcl_cv_struct_dirent64,[ AC_COMPILE_IFELSE([AC_LANG_PROGRAM([[#include <sys/types.h> #include <dirent.h>]], [[struct dirent64 p;]])], [tcl_cv_struct_dirent64=yes],[tcl_cv_struct_dirent64=no])]) if test "x${tcl_cv_struct_dirent64}" = "xyes" ; then AC_DEFINE(HAVE_STRUCT_DIRENT64, 1, [Is 'struct dirent64' in <sys/types.h>?]) fi AC_CACHE_CHECK([for DIR64], tcl_cv_DIR64,[ AC_COMPILE_IFELSE([AC_LANG_PROGRAM([[#include <sys/types.h> #include <dirent.h>]], [[struct dirent64 *p; DIR64 d = opendir64("."); p = readdir64(d); rewinddir64(d); closedir64(d);]])], [tcl_cv_DIR64=yes], [tcl_cv_DIR64=no])]) if test "x${tcl_cv_DIR64}" = "xyes" ; then AC_DEFINE(HAVE_DIR64, 1, [Is 'DIR64' in <sys/types.h>?]) fi AC_CACHE_CHECK([for struct stat64], tcl_cv_struct_stat64,[ AC_COMPILE_IFELSE([AC_LANG_PROGRAM([[#include <sys/stat.h>]], [[struct stat64 p; ]])], [tcl_cv_struct_stat64=yes], [tcl_cv_struct_stat64=no])]) if test "x${tcl_cv_struct_stat64}" = "xyes" ; then AC_DEFINE(HAVE_STRUCT_STAT64, 1, [Is 'struct stat64' in <sys/stat.h>?]) fi AC_CHECK_FUNCS(open64 lseek64) AC_MSG_CHECKING([for off64_t]) AC_CACHE_VAL(tcl_cv_type_off64_t,[ AC_COMPILE_IFELSE([AC_LANG_PROGRAM([[#include <sys/types.h>]], [[off64_t offset; ]])], [tcl_cv_type_off64_t=yes], [tcl_cv_type_off64_t=no])]) dnl Define HAVE_TYPE_OFF64_T only when the off64_t type and the dnl functions lseek64 and open64 are defined. if test "x${tcl_cv_type_off64_t}" = "xyes" && \ test "x${ac_cv_func_lseek64}" = "xyes" && \ test "x${ac_cv_func_open64}" = "xyes" ; then AC_DEFINE(HAVE_TYPE_OFF64_T, 1, [Is off64_t in <sys/types.h>?]) AC_MSG_RESULT([yes]) |
︙ | ︙ | |||
2836 2837 2838 2839 2840 2841 2842 | # EXEEXT # Select the executable extension based on the host type. This # is a lightweight replacement for AC_EXEEXT that doesn't require # a compiler. #------------------------------------------------------------------------ AC_DEFUN([TEA_INIT], [ | < < | | | < < < < < < | < | | < | | 2684 2685 2686 2687 2688 2689 2690 2691 2692 2693 2694 2695 2696 2697 2698 2699 2700 2701 2702 2703 2704 2705 2706 2707 2708 2709 2710 2711 2712 2713 2714 2715 2716 2717 2718 2719 2720 2721 | # EXEEXT # Select the executable extension based on the host type. This # is a lightweight replacement for AC_EXEEXT that doesn't require # a compiler. #------------------------------------------------------------------------ AC_DEFUN([TEA_INIT], [ TEA_VERSION="3.13" AC_MSG_CHECKING([TEA configuration]) if test x"${PACKAGE_NAME}" = x ; then AC_MSG_ERROR([ The PACKAGE_NAME variable must be defined by your TEA configure.ac]) fi AC_MSG_RESULT([ok (TEA ${TEA_VERSION})]) # If the user did not set CFLAGS, set it now to keep macros # like AC_PROG_CC and AC_TRY_COMPILE from adding "-g -O2". if test "${CFLAGS+set}" != "set" ; then CFLAGS="" fi case "`uname -s`" in *win32*|*WIN32*|*MINGW32_*|*MINGW64_*|*MSYS_*) AC_CHECK_PROG(CYGPATH, cygpath, cygpath -m, echo) EXEEXT=".exe" TEA_PLATFORM="windows" ;; *CYGWIN_*) EXEEXT=".exe" # CYGPATH and TEA_PLATFORM are determined later in LOAD_TCLCONFIG ;; *) CYGPATH=echo # Maybe we are cross-compiling.... case ${host_alias} in *mingw32*) EXEEXT=".exe" |
︙ | ︙ | |||
2903 2904 2905 2906 2907 2908 2909 2910 2911 2912 2913 2914 2915 2916 2917 2918 2919 2920 2921 2922 2923 2924 2925 2926 2927 2928 | AC_MSG_NOTICE([configuring ${PACKAGE_NAME} ${PACKAGE_VERSION}]) AC_SUBST(EXEEXT) AC_SUBST(CYGPATH) # This package name must be replaced statically for AC_SUBST to work AC_SUBST(PKG_LIB_FILE) # Substitute STUB_LIB_FILE in case package creates a stub library too. AC_SUBST(PKG_STUB_LIB_FILE) # We AC_SUBST these here to ensure they are subst'ed, # in case the user doesn't call TEA_ADD_... AC_SUBST(PKG_STUB_SOURCES) AC_SUBST(PKG_STUB_OBJECTS) AC_SUBST(PKG_TCL_SOURCES) AC_SUBST(PKG_HEADERS) AC_SUBST(PKG_INCLUDES) AC_SUBST(PKG_LIBS) AC_SUBST(PKG_CFLAGS) ]) #------------------------------------------------------------------------ # TEA_ADD_SOURCES -- # # Specify one or more source files. Users should check for # the right platform before adding to their list. | > > > > > | 2741 2742 2743 2744 2745 2746 2747 2748 2749 2750 2751 2752 2753 2754 2755 2756 2757 2758 2759 2760 2761 2762 2763 2764 2765 2766 2767 2768 2769 2770 2771 | AC_MSG_NOTICE([configuring ${PACKAGE_NAME} ${PACKAGE_VERSION}]) AC_SUBST(EXEEXT) AC_SUBST(CYGPATH) # This package name must be replaced statically for AC_SUBST to work AC_SUBST(PKG_LIB_FILE) AC_SUBST(PKG_LIB_FILE8) AC_SUBST(PKG_LIB_FILE9) # Substitute STUB_LIB_FILE in case package creates a stub library too. AC_SUBST(PKG_STUB_LIB_FILE) # We AC_SUBST these here to ensure they are subst'ed, # in case the user doesn't call TEA_ADD_... AC_SUBST(PKG_STUB_SOURCES) AC_SUBST(PKG_STUB_OBJECTS) AC_SUBST(PKG_TCL_SOURCES) AC_SUBST(PKG_HEADERS) AC_SUBST(PKG_INCLUDES) AC_SUBST(PKG_LIBS) AC_SUBST(PKG_CFLAGS) # Configure the installer. TEA_INSTALLER ]) #------------------------------------------------------------------------ # TEA_ADD_SOURCES -- # # Specify one or more source files. Users should check for # the right platform before adding to their list. |
︙ | ︙ | |||
3107 3108 3109 3110 3111 3112 3113 | # PKG_LIBS #------------------------------------------------------------------------ AC_DEFUN([TEA_ADD_LIBS], [ vars="$@" for i in $vars; do if test "${TEA_PLATFORM}" = "windows" -a "$GCC" = "yes" ; then # Convert foo.lib to -lfoo for GCC. No-op if not *.lib | | | 2950 2951 2952 2953 2954 2955 2956 2957 2958 2959 2960 2961 2962 2963 2964 | # PKG_LIBS #------------------------------------------------------------------------ AC_DEFUN([TEA_ADD_LIBS], [ vars="$@" for i in $vars; do if test "${TEA_PLATFORM}" = "windows" -a "$GCC" = "yes" ; then # Convert foo.lib to -lfoo for GCC. No-op if not *.lib i=`echo "$i" | sed -e 's/^\([[^-]].*\)\.[[lL]][[iI]][[bB]][$]/-l\1/'` fi PKG_LIBS="$PKG_LIBS $i" done AC_SUBST(PKG_LIBS) ]) #------------------------------------------------------------------------ |
︙ | ︙ | |||
3190 3191 3192 3193 3194 3195 3196 | fi ]) #------------------------------------------------------------------------ # TEA_SETUP_COMPILER_CC -- # # Do compiler checks the way we want. This is just a replacement | | < < < < < < < < < | 3033 3034 3035 3036 3037 3038 3039 3040 3041 3042 3043 3044 3045 3046 3047 3048 3049 3050 3051 3052 3053 3054 3055 3056 3057 3058 3059 3060 3061 3062 | fi ]) #------------------------------------------------------------------------ # TEA_SETUP_COMPILER_CC -- # # Do compiler checks the way we want. This is just a replacement # for AC_PROG_CC in TEA configure.ac files to make them cleaner. # # Arguments: # none # # Results: # # Sets up CC var and other standard bits we need to make executables. #------------------------------------------------------------------------ AC_DEFUN([TEA_SETUP_COMPILER_CC], [ # Don't put any macros that use the compiler (e.g. AC_TRY_COMPILE) # in this macro, they need to go into TEA_SETUP_COMPILER instead. AC_PROG_CC AC_PROG_CPP #-------------------------------------------------------------------- # Checks to see if the make program sets the $MAKE variable. #-------------------------------------------------------------------- AC_PROG_MAKE_SET #-------------------------------------------------------------------- |
︙ | ︙ | |||
3261 3262 3263 3264 3265 3266 3267 | # It makes compiling go faster. (This is only a performance feature.) #------------------------------------------------------------------------ if test -z "$no_pipe" -a -n "$GCC"; then AC_CACHE_CHECK([if the compiler understands -pipe], tcl_cv_cc_pipe, [ hold_cflags=$CFLAGS; CFLAGS="$CFLAGS -pipe" | | < < < < < < < < | 3095 3096 3097 3098 3099 3100 3101 3102 3103 3104 3105 3106 3107 3108 3109 3110 3111 3112 3113 3114 3115 3116 3117 3118 3119 3120 | # It makes compiling go faster. (This is only a performance feature.) #------------------------------------------------------------------------ if test -z "$no_pipe" -a -n "$GCC"; then AC_CACHE_CHECK([if the compiler understands -pipe], tcl_cv_cc_pipe, [ hold_cflags=$CFLAGS; CFLAGS="$CFLAGS -pipe" AC_COMPILE_IFELSE([AC_LANG_PROGRAM([[]], [[]])],[tcl_cv_cc_pipe=yes],[tcl_cv_cc_pipe=no]) CFLAGS=$hold_cflags]) if test $tcl_cv_cc_pipe = yes; then CFLAGS="$CFLAGS -pipe" fi fi #-------------------------------------------------------------------- # Common compiler flag setup #-------------------------------------------------------------------- AC_C_BIGENDIAN ]) #------------------------------------------------------------------------ # TEA_MAKE_LIB -- # # Generate a line that can be used to build a shared/unshared library # in a platform independent manner. |
︙ | ︙ | |||
3311 3312 3313 3314 3315 3316 3317 | # VC_MANIFEST_EMBED_DLL Makefile rule for embedded VC manifest in DLL # VC_MANIFEST_EMBED_EXE Makefile rule for embedded VC manifest in EXE #------------------------------------------------------------------------ AC_DEFUN([TEA_MAKE_LIB], [ if test "${TEA_PLATFORM}" = "windows" -a "$GCC" != "yes"; then MAKE_STATIC_LIB="\${STLIB_LD} -out:\[$]@ \$(PKG_OBJECTS)" | | | > > > > > > > > > | < | > > > | > > | > > | | | 3137 3138 3139 3140 3141 3142 3143 3144 3145 3146 3147 3148 3149 3150 3151 3152 3153 3154 3155 3156 3157 3158 3159 3160 3161 3162 3163 3164 3165 3166 3167 3168 3169 3170 3171 3172 3173 3174 3175 3176 3177 3178 3179 3180 3181 3182 3183 3184 3185 3186 3187 3188 3189 3190 3191 3192 3193 3194 3195 3196 3197 3198 3199 3200 3201 3202 3203 3204 3205 3206 3207 3208 3209 3210 3211 3212 3213 3214 3215 3216 3217 3218 3219 3220 3221 3222 3223 3224 3225 3226 3227 3228 3229 3230 3231 3232 3233 3234 3235 3236 | # VC_MANIFEST_EMBED_DLL Makefile rule for embedded VC manifest in DLL # VC_MANIFEST_EMBED_EXE Makefile rule for embedded VC manifest in EXE #------------------------------------------------------------------------ AC_DEFUN([TEA_MAKE_LIB], [ if test "${TEA_PLATFORM}" = "windows" -a "$GCC" != "yes"; then MAKE_STATIC_LIB="\${STLIB_LD} -out:\[$]@ \$(PKG_OBJECTS)" MAKE_SHARED_LIB="\${SHLIB_LD} \${LDFLAGS} \${LDFLAGS_DEFAULT} -out:\[$]@ \$(PKG_OBJECTS) \${SHLIB_LD_LIBS}" AC_EGREP_CPP([manifest needed], [ #if defined(_MSC_VER) && _MSC_VER >= 1400 print("manifest needed") #endif ], [ # Could do a CHECK_PROG for mt, but should always be with MSVC8+ VC_MANIFEST_EMBED_DLL="if test -f \[$]@.manifest ; then mt.exe -nologo -manifest \[$]@.manifest -outputresource:\[$]@\;2 ; fi" VC_MANIFEST_EMBED_EXE="if test -f \[$]@.manifest ; then mt.exe -nologo -manifest \[$]@.manifest -outputresource:\[$]@\;1 ; fi" MAKE_SHARED_LIB="${MAKE_SHARED_LIB} ; ${VC_MANIFEST_EMBED_DLL}" TEA_ADD_CLEANFILES([*.manifest]) ]) MAKE_STUB_LIB="\${STLIB_LD} -nodefaultlib -out:\[$]@ \$(PKG_STUB_OBJECTS)" else MAKE_STATIC_LIB="\${STLIB_LD} \[$]@ \$(PKG_OBJECTS)" MAKE_SHARED_LIB="\${SHLIB_LD} \${LDFLAGS} \${LDFLAGS_DEFAULT} -o \[$]@ \$(PKG_OBJECTS) \${SHLIB_LD_LIBS}" MAKE_STUB_LIB="\${STLIB_LD} \[$]@ \$(PKG_STUB_OBJECTS)" fi if test "${SHARED_BUILD}" = "1" ; then MAKE_LIB="${MAKE_SHARED_LIB} " else MAKE_LIB="${MAKE_STATIC_LIB} " fi #-------------------------------------------------------------------- # Shared libraries and static libraries have different names. # Use the double eval to make sure any variables in the suffix is # substituted. (@@@ Might not be necessary anymore) #-------------------------------------------------------------------- PACKAGE_LIB_PREFIX8="${PACKAGE_LIB_PREFIX}" PACKAGE_LIB_PREFIX9="${PACKAGE_LIB_PREFIX}tcl9" if test "${TCL_MAJOR_VERSION}" -gt 8 ; then PACKAGE_LIB_PREFIX="${PACKAGE_LIB_PREFIX9}" else PACKAGE_LIB_PREFIX="${PACKAGE_LIB_PREFIX8}" fi if test "${TEA_PLATFORM}" = "windows" ; then if test "${SHARED_BUILD}" = "1" ; then # We force the unresolved linking of symbols that are really in # the private libraries of Tcl and Tk. if test x"${TK_BIN_DIR}" != x ; then SHLIB_LD_LIBS="${SHLIB_LD_LIBS} \"`${CYGPATH} ${TK_BIN_DIR}/${TK_STUB_LIB_FILE}`\"" fi SHLIB_LD_LIBS="${SHLIB_LD_LIBS} \"`${CYGPATH} ${TCL_BIN_DIR}/${TCL_STUB_LIB_FILE}`\"" if test "$GCC" = "yes"; then SHLIB_LD_LIBS="${SHLIB_LD_LIBS} -static-libgcc" fi eval eval "PKG_LIB_FILE8=${PACKAGE_LIB_PREFIX8}${PACKAGE_NAME}${SHARED_LIB_SUFFIX}" eval eval "PKG_LIB_FILE9=${PACKAGE_LIB_PREFIX9}${PACKAGE_NAME}${SHARED_LIB_SUFFIX}" eval eval "PKG_LIB_FILE=${PACKAGE_LIB_PREFIX}${PACKAGE_NAME}${SHARED_LIB_SUFFIX}" else if test "$GCC" = "yes"; then PACKAGE_LIB_PREFIX=lib${PACKAGE_LIB_PREFIX} fi eval eval "PKG_LIB_FILE8=${PACKAGE_LIB_PREFIX8}${PACKAGE_NAME}${UNSHARED_LIB_SUFFIX}" eval eval "PKG_LIB_FILE9=${PACKAGE_LIB_PREFIX9}${PACKAGE_NAME}${UNSHARED_LIB_SUFFIX}" eval eval "PKG_LIB_FILE=${PACKAGE_LIB_PREFIX}${PACKAGE_NAME}${UNSHARED_LIB_SUFFIX}" fi # Some packages build their own stubs libraries eval eval "PKG_STUB_LIB_FILE=${PACKAGE_LIB_PREFIX}${PACKAGE_NAME}stub${UNSHARED_LIB_SUFFIX}" if test "$GCC" = "yes"; then PKG_STUB_LIB_FILE=lib${PKG_STUB_LIB_FILE} fi # These aren't needed on Windows (either MSVC or gcc) RANLIB=: RANLIB_STUB=: else RANLIB_STUB="${RANLIB}" if test "${SHARED_BUILD}" = "1" ; then SHLIB_LD_LIBS="${SHLIB_LD_LIBS} ${TCL_STUB_LIB_SPEC}" if test x"${TK_BIN_DIR}" != x ; then SHLIB_LD_LIBS="${SHLIB_LD_LIBS} ${TK_STUB_LIB_SPEC}" fi eval eval "PKG_LIB_FILE8=lib${PACKAGE_LIB_PREFIX8}${PACKAGE_NAME}${SHARED_LIB_SUFFIX}" eval eval "PKG_LIB_FILE9=lib${PACKAGE_LIB_PREFIX9}${PACKAGE_NAME}${SHARED_LIB_SUFFIX}" eval eval "PKG_LIB_FILE=lib${PACKAGE_LIB_PREFIX}${PACKAGE_NAME}${SHARED_LIB_SUFFIX}" RANLIB=: else eval eval "PKG_LIB_FILE=lib${PACKAGE_LIB_PREFIX8}${PACKAGE_NAME}${UNSHARED_LIB_SUFFIX}" eval eval "PKG_LIB_FILE=lib${PACKAGE_LIB_PREFIX9}${PACKAGE_NAME}${UNSHARED_LIB_SUFFIX}" eval eval "PKG_LIB_FILE=lib${PACKAGE_LIB_PREFIX}${PACKAGE_NAME}${UNSHARED_LIB_SUFFIX}" fi # Some packages build their own stubs libraries eval eval "PKG_STUB_LIB_FILE=lib${PACKAGE_LIB_PREFIX8}${PACKAGE_NAME}stub${UNSHARED_LIB_SUFFIX}" fi # These are escaped so that only CFLAGS is picked up at configure time. # The other values will be substituted at make time. CFLAGS="${CFLAGS} \${CFLAGS_DEFAULT} \${CFLAGS_WARNING}" if test "${SHARED_BUILD}" = "1" ; then CFLAGS="${CFLAGS} \${SHLIB_CFLAGS}" |
︙ | ︙ | |||
3926 3927 3928 3929 3930 3931 3932 3933 3934 3935 3936 3937 3938 3939 | # check in a few common install locations if test x"${ac_cv_c_$1config}" = x ; then for i in `ls -d ${libdir} 2>/dev/null` \ `ls -d ${exec_prefix}/lib 2>/dev/null` \ `ls -d ${prefix}/lib 2>/dev/null` \ `ls -d /usr/local/lib 2>/dev/null` \ `ls -d /usr/contrib/lib 2>/dev/null` \ `ls -d /usr/lib 2>/dev/null` \ `ls -d /usr/lib64 2>/dev/null` \ ; do if test -f "$i/$1Config.sh" ; then ac_cv_c_$1config=`(cd $i; pwd)` break fi | > | 3767 3768 3769 3770 3771 3772 3773 3774 3775 3776 3777 3778 3779 3780 3781 | # check in a few common install locations if test x"${ac_cv_c_$1config}" = x ; then for i in `ls -d ${libdir} 2>/dev/null` \ `ls -d ${exec_prefix}/lib 2>/dev/null` \ `ls -d ${prefix}/lib 2>/dev/null` \ `ls -d /usr/local/lib 2>/dev/null` \ `ls -d /usr/contrib/lib 2>/dev/null` \ `ls -d /usr/pkg/lib 2>/dev/null` \ `ls -d /usr/lib 2>/dev/null` \ `ls -d /usr/lib64 2>/dev/null` \ ; do if test -f "$i/$1Config.sh" ; then ac_cv_c_$1config=`(cd $i; pwd)` break fi |
︙ | ︙ | |||
4067 4068 4069 4070 4071 4072 4073 | #-------------------------------------------------------------------- # These are for $1Config.sh #-------------------------------------------------------------------- # pkglibdir must be a fully qualified path and (not ${exec_prefix}/lib) eval pkglibdir="[$]{libdir}/$1${PACKAGE_VERSION}" if test "${TCL_LIB_VERSIONS_OK}" = "ok"; then | | | | | | | | | | | | | > > > > > > > > | > > > > > > | > > > > > | > > > > > > > | > > > > > > > > > > > | < < > > > > > | < < > > > > > < < < < | | > | > > | | | | < | | > > > > > > > > > > > > > | < | < < < < < < < < < < | | | | < | > > | | > | > > > > | > > > > > | < < | | < > > > > > > | 3909 3910 3911 3912 3913 3914 3915 3916 3917 3918 3919 3920 3921 3922 3923 3924 3925 3926 3927 3928 3929 3930 3931 3932 3933 3934 3935 3936 3937 3938 3939 3940 3941 3942 3943 3944 3945 3946 3947 3948 3949 3950 3951 3952 3953 3954 3955 3956 3957 3958 3959 3960 3961 3962 3963 3964 3965 3966 3967 3968 3969 3970 3971 3972 3973 3974 3975 3976 3977 3978 3979 3980 3981 3982 3983 3984 3985 3986 3987 3988 3989 3990 3991 3992 3993 3994 3995 3996 3997 3998 3999 4000 4001 4002 4003 4004 4005 4006 4007 4008 4009 4010 4011 4012 4013 4014 4015 4016 4017 4018 4019 4020 4021 4022 4023 4024 4025 4026 4027 4028 4029 4030 4031 4032 4033 4034 4035 4036 4037 4038 4039 4040 4041 4042 4043 4044 4045 4046 4047 4048 4049 4050 4051 4052 4053 4054 4055 4056 4057 4058 4059 4060 4061 4062 4063 4064 4065 4066 4067 | #-------------------------------------------------------------------- # These are for $1Config.sh #-------------------------------------------------------------------- # pkglibdir must be a fully qualified path and (not ${exec_prefix}/lib) eval pkglibdir="[$]{libdir}/$1${PACKAGE_VERSION}" if test "${TCL_LIB_VERSIONS_OK}" = "ok"; then eval $1_LIB_FLAG="-l$1${PACKAGE_VERSION}" eval $1_STUB_LIB_FLAG="-l$1stub${PACKAGE_VERSION}" else eval $1_LIB_FLAG="-l$1`echo ${PACKAGE_VERSION} | tr -d .`" eval $1_STUB_LIB_FLAG="-l$1stub`echo ${PACKAGE_VERSION} | tr -d .`" fi $1_BUILD_LIB_SPEC="-L`$CYGPATH $(pwd)` ${$1_LIB_FLAG}" $1_LIB_SPEC="-L`$CYGPATH ${pkglibdir}` ${$1_LIB_FLAG}" $1_BUILD_STUB_LIB_SPEC="-L`$CYGPATH $(pwd)` [$]{$1_STUB_LIB_FLAG}" $1_STUB_LIB_SPEC="-L`$CYGPATH ${pkglibdir}` [$]{$1_STUB_LIB_FLAG}" $1_BUILD_STUB_LIB_PATH="`$CYGPATH $(pwd)`/[$]{PKG_STUB_LIB_FILE}" $1_STUB_LIB_PATH="`$CYGPATH ${pkglibdir}`/[$]{PKG_STUB_LIB_FILE}" AC_SUBST($1_BUILD_LIB_SPEC) AC_SUBST($1_LIB_SPEC) AC_SUBST($1_BUILD_STUB_LIB_SPEC) AC_SUBST($1_STUB_LIB_SPEC) AC_SUBST($1_BUILD_STUB_LIB_PATH) AC_SUBST($1_STUB_LIB_PATH) AC_SUBST(MAJOR_VERSION) AC_SUBST(MINOR_VERSION) AC_SUBST(PATCHLEVEL) ]) #------------------------------------------------------------------------ # TEA_INSTALLER -- # # Configure the installer. # # Arguments: # none # # Results: # Substitutes the following vars: # INSTALL # INSTALL_DATA_DIR # INSTALL_DATA # INSTALL_PROGRAM # INSTALL_SCRIPT # INSTALL_LIBRARY #------------------------------------------------------------------------ AC_DEFUN([TEA_INSTALLER], [ INSTALL='$(SHELL) $(srcdir)/tclconfig/install-sh -c' INSTALL_DATA_DIR='${INSTALL} -d -m 755' INSTALL_DATA='${INSTALL} -m 644' INSTALL_PROGRAM='${INSTALL} -m 755' INSTALL_SCRIPT='${INSTALL} -m 755' TEA_CONFIG_SYSTEM case $system in HP-UX-*) INSTALL_LIBRARY='${INSTALL} -m 755' ;; *) INSTALL_LIBRARY='${INSTALL} -m 644' ;; esac AC_SUBST(INSTALL) AC_SUBST(INSTALL_DATA_DIR) AC_SUBST(INSTALL_DATA) AC_SUBST(INSTALL_PROGRAM) AC_SUBST(INSTALL_SCRIPT) AC_SUBST(INSTALL_LIBRARY) ]) ### # Tip 430 - ZipFS Modifications ### #------------------------------------------------------------------------ # TEA_ZIPFS_SUPPORT # Locate a zip encoder installed on the system path, or none. # # Arguments: # none # # Results: # Substitutes the following vars: # MACHER_PROG # ZIP_PROG # ZIP_PROG_OPTIONS # ZIP_PROG_VFSSEARCH # ZIP_INSTALL_OBJS #------------------------------------------------------------------------ AC_DEFUN([TEA_ZIPFS_SUPPORT], [ MACHER_PROG="" ZIP_PROG="" ZIP_PROG_OPTIONS="" ZIP_PROG_VFSSEARCH="" ZIP_INSTALL_OBJS="" AC_MSG_CHECKING([for macher]) AC_CACHE_VAL(ac_cv_path_macher, [ search_path=`echo ${PATH} | sed -e 's/:/ /g'` for dir in $search_path ; do for j in `ls -r $dir/macher 2> /dev/null` \ `ls -r $dir/macher 2> /dev/null` ; do if test x"$ac_cv_path_macher" = x ; then if test -f "$j" ; then ac_cv_path_macher=$j break fi fi done done ]) if test -f "$ac_cv_path_macher" ; then MACHER_PROG="$ac_cv_path_macher" AC_MSG_RESULT([$MACHER_PROG]) AC_MSG_RESULT([Found macher in environment]) fi AC_MSG_CHECKING([for zip]) AC_CACHE_VAL(ac_cv_path_zip, [ search_path=`echo ${PATH} | sed -e 's/:/ /g'` for dir in $search_path ; do for j in `ls -r $dir/zip 2> /dev/null` \ `ls -r $dir/zip 2> /dev/null` ; do if test x"$ac_cv_path_zip" = x ; then if test -f "$j" ; then ac_cv_path_zip=$j break fi fi done done ]) if test -f "$ac_cv_path_zip" ; then ZIP_PROG="$ac_cv_path_zip" AC_MSG_RESULT([$ZIP_PROG]) ZIP_PROG_OPTIONS="-rq" ZIP_PROG_VFSSEARCH="*" AC_MSG_RESULT([Found INFO Zip in environment]) # Use standard arguments for zip else # It is not an error if an installed version of Zip can't be located. # We can use the locally distributed minizip instead ZIP_PROG="./minizip${EXEEXT_FOR_BUILD}" ZIP_PROG_OPTIONS="-o -r" ZIP_PROG_VFSSEARCH="*" ZIP_INSTALL_OBJS="minizip${EXEEXT_FOR_BUILD}" AC_MSG_RESULT([No zip found on PATH. Building minizip]) fi AC_SUBST(MACHER_PROG) AC_SUBST(ZIP_PROG) AC_SUBST(ZIP_PROG_OPTIONS) AC_SUBST(ZIP_PROG_VFSSEARCH) AC_SUBST(ZIP_INSTALL_OBJS) ]) # Local Variables: # mode: autoconf # End: |
Changes to autoconf/tea/win/makefile.vc.
︙ | ︙ | |||
158 159 160 161 162 163 164 | # nmakehelp -V <file> <tag> will search the file for tag, skips until a # number and returns all character until a character not in [0-9.ab] # is read. !if [echo REM = This file is generated from Makefile.vc > versions.vc] !endif | | | < < < | > > > > > > > > > > > | | | | > > > > > > > > > > > | 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 | # nmakehelp -V <file> <tag> will search the file for tag, skips until a # number and returns all character until a character not in [0-9.ab] # is read. !if [echo REM = This file is generated from Makefile.vc > versions.vc] !endif # get project version from row "AC_INIT([sqlite], [3.x.y])" !if [echo DOTVERSION = \>> versions.vc] \ && [nmakehlp -V ..\configure.ac AC_INIT >> versions.vc] !endif !include "versions.vc" VERSION = $(DOTVERSION:.=) STUBPREFIX = $(PROJECT)stub #------------------------------------------------------------------------- # Target names and paths ( shouldn't need changing ) #------------------------------------------------------------------------- BINROOT = . ROOT = .. PRJIMPLIB = $(OUT_DIR)\$(PROJECT)$(VERSION)$(SUFX).lib PRJLIBNAME = $(PROJECT).$(EXT) PRJLIB = $(OUT_DIR)\$(PRJLIBNAME) PRJSTUBLIBNAME = $(STUBPREFIX)$(VERSION).lib PRJSTUBLIB = $(OUT_DIR)\$(PRJSTUBLIBNAME) ### Make sure we use backslash only. PRJ_INSTALL_DIR = $(_INSTALLDIR)\$(PROJECT)$(DOTVERSION) LIB_INSTALL_DIR = $(PRJ_INSTALL_DIR) BIN_INSTALL_DIR = $(PRJ_INSTALL_DIR) DOC_INSTALL_DIR = $(PRJ_INSTALL_DIR) SCRIPT_INSTALL_DIR = $(PRJ_INSTALL_DIR) INCLUDE_INSTALL_DIR = $(_TCLDIR)\include ### The following paths CANNOT have spaces in them. GENERICDIR = $(ROOT)\generic WINDIR = $(ROOT)\win LIBDIR = $(ROOT)\library DOCDIR = $(ROOT)\doc TOOLSDIR = $(ROOT)\tools COMPATDIR = $(ROOT)\compat ### Figure out where the primary source code file(s) is/are. !if exist("$(ROOT)\..\..\sqlite3.c") && exist("$(ROOT)\..\..\src\tclsqlite.c") SQL_INCLUDES = -I"$(ROOT)\..\.." SQLITE_SRCDIR = $(ROOT)\..\.. TCLSQLITE_SRCDIR = $(ROOT)\..\..\src DLLOBJS = $(TMP_DIR)\sqlite3.obj $(TMP_DIR)\tclsqlite.obj !else TCLSQLITE_SRCDIR = $(ROOT)\generic DLLOBJS = $(TMP_DIR)\tclsqlite3.obj !endif #--------------------------------------------------------------------- # Compile flags #--------------------------------------------------------------------- !if !$(DEBUG) !if $(OPTIMIZING) ### This cranks the optimization level to maximize speed cdebug = -O2 -Op -Gs !else cdebug = !endif !else if "$(MACHINE)" == "IA64" ### Warnings are too many, can't support warnings into errors. cdebug = -Z7 -Od -GZ !else cdebug = -Z7 -WX -Od -GZ !endif ### Declarations common to all compiler options cflags = -nologo -c -W3 -D_CRT_SECURE_NO_WARNINGS -YX -Fp$(TMP_DIR)^\ !if $(MSVCRT) !if $(DEBUG) crt = -MDd !else crt = -MD !endif !else !if $(DEBUG) crt = -MTd !else crt = -MT !endif !endif INCLUDES = $(SQL_INCLUDES) $(TCL_INCLUDES) -I"$(WINDIR)" \ -I"$(GENERICDIR)" -I"$(ROOT)\.." BASE_CLFAGS = $(cflags) $(cdebug) $(crt) $(INCLUDES) \ -DSQLITE_3_SUFFIX_ONLY=1 -DSQLITE_ENABLE_RTREE=1 \ -DSQLITE_ENABLE_FTS3=1 -DSQLITE_OMIT_DEPRECATED=1 \ -DSQLITE_ENABLE_FTS4=1 \ -DSQLITE_ENABLE_FTS5=1 \ -DSQLITE_3_SUFFIX_ONLY=1 \ -DSQLITE_ENABLE_RTREE=1 \ -DSQLITE_ENABLE_GEOPOLY=1 \ -DSQLITE_ENABLE_MATH_FUNCTIONS=1 \ -DSQLITE_ENABLE_DESERIALIZE=1 \ -DSQLITE_ENABLE_DBPAGE_VTAB=1 \ -DSQLITE_ENABLE_BYTECODE_VTAB=1 \ -DSQLITE_ENABLE_DBSTAT_VTAB=1 CON_CFLAGS = $(cflags) $(cdebug) $(crt) -DCONSOLE -DSQLITE_ENABLE_FTS3=1 TCL_CFLAGS = -DBUILD_sqlite -DUSE_TCL_STUBS \ -DPACKAGE_VERSION="\"$(DOTVERSION)\"" $(BASE_CLFAGS) \ $(OPTDEFINES) #--------------------------------------------------------------------- # Link flags |
︙ | ︙ | |||
337 338 339 340 341 342 343 | $(PRJSTUBLIB): $(PRJSTUBOBJS) $(lib32) -nologo -out:$@ $(PRJSTUBOBJS) #--------------------------------------------------------------------- # Implicit rules #--------------------------------------------------------------------- | | | | < | | < < > | | < < > | 356 357 358 359 360 361 362 363 364 365 366 367 368 369 370 371 372 373 374 375 376 377 378 379 380 | $(PRJSTUBLIB): $(PRJSTUBOBJS) $(lib32) -nologo -out:$@ $(PRJSTUBOBJS) #--------------------------------------------------------------------- # Implicit rules #--------------------------------------------------------------------- $(TMP_DIR)\sqlite3.obj: $(SQLITE_SRCDIR)\sqlite3.c $(cc32) $(TCL_CFLAGS) -DBUILD_$(PROJECT) -Fo$(TMP_DIR)\ \ -c $(SQLITE_SRCDIR)\sqlite3.c $(TMP_DIR)\tclsqlite.obj: $(TCLSQLITE_SRCDIR)\tclsqlite.c $(cc32) $(TCL_CFLAGS) -DBUILD_$(PROJECT) -Fo$(TMP_DIR)\ \ -c $(TCLSQLITE_SRCDIR)\tclsqlite.c $(TMP_DIR)\tclsqlite3.obj: $(TCLSQLITE_SRCDIR)\tclsqlite3.c $(cc32) $(TCL_CFLAGS) -DBUILD_$(PROJECT) -Fo$(TMP_DIR)\ \ -c $(TCLSQLITE_SRCDIR)\tclsqlite3.c {$(WINDIR)}.rc{$(TMP_DIR)}.res: $(rc32) -fo $@ -r -i "$(GENERICDIR)" -D__WIN32__ \ !if $(DEBUG) -d DEBUG \ !endif !if $(TCL_THREADS) |
︙ | ︙ |
Changes to autoconf/tea/win/nmakehlp.c.
︙ | ︙ | |||
10 11 12 13 14 15 16 | * See the file "license.terms" for information on usage and redistribution of * this file, and for a DISCLAIMER OF ALL WARRANTIES. * ---------------------------------------------------------------------------- */ #define _CRT_SECURE_NO_DEPRECATE #include <windows.h> | | < < < | < | > | | | > | 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 | * See the file "license.terms" for information on usage and redistribution of * this file, and for a DISCLAIMER OF ALL WARRANTIES. * ---------------------------------------------------------------------------- */ #define _CRT_SECURE_NO_DEPRECATE #include <windows.h> #ifdef _MSC_VER #pragma comment (lib, "user32.lib") #pragma comment (lib, "kernel32.lib") #endif #include <stdio.h> #include <math.h> /* * This library is required for x64 builds with _some_ versions of MSVC */ #if defined(_M_IA64) || defined(_M_AMD64) #if _MSC_VER >= 1400 && _MSC_VER < 1500 #pragma comment(lib, "bufferoverflowU") #endif #endif /* ISO hack for dumb VC++ */ #ifdef _MSC_VER #define snprintf _snprintf #endif /* protos */ static int CheckForCompilerFeature(const char *option); static int CheckForLinkerFeature(char **options, int count); static int IsIn(const char *string, const char *substring); static int SubstituteFile(const char *substs, const char *filename); static int QualifyPath(const char *path); static int LocateDependency(const char *keyfile); static const char *GetVersionFromFile(const char *filename, const char *match, int numdots); static DWORD WINAPI ReadFromPipe(LPVOID args); /* globals */ #define CHUNK 25 #define STATICBUFFERSIZE 1000 typedef struct { HANDLE pipe; char buffer[STATICBUFFERSIZE]; } pipeinfo; pipeinfo Out = {INVALID_HANDLE_VALUE, ""}; pipeinfo Err = {INVALID_HANDLE_VALUE, ""}; /* * exitcodes: 0 == no, 1 == yes, 2 == error */ int main( int argc, char *argv[]) { char msg[300]; DWORD dwWritten; int chars; const char *s; /* * Make sure children (cl.exe and link.exe) are kept quiet. */ SetErrorMode(SEM_FAILCRITICALERRORS | SEM_NOOPENFILEERRORBOX); |
︙ | ︙ | |||
98 99 100 101 102 103 104 | "exitcodes: 0 == no, 1 == yes, 2 == error\n", argv[0]); WriteFile(GetStdHandle(STD_ERROR_HANDLE), msg, chars, &dwWritten, NULL); return 2; } return CheckForCompilerFeature(argv[2]); case 'l': | | | | | 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 | "exitcodes: 0 == no, 1 == yes, 2 == error\n", argv[0]); WriteFile(GetStdHandle(STD_ERROR_HANDLE), msg, chars, &dwWritten, NULL); return 2; } return CheckForCompilerFeature(argv[2]); case 'l': if (argc < 3) { chars = snprintf(msg, sizeof(msg) - 1, "usage: %s -l <linker option> ?<mandatory option> ...?\n" "Tests for whether link.exe supports an option\n" "exitcodes: 0 == no, 1 == yes, 2 == error\n", argv[0]); WriteFile(GetStdHandle(STD_ERROR_HANDLE), msg, chars, &dwWritten, NULL); return 2; } return CheckForLinkerFeature(&argv[2], argc-2); case 'f': if (argc == 2) { chars = snprintf(msg, sizeof(msg) - 1, "usage: %s -f <string> <substring>\n" "Find a substring within another\n" "exitcodes: 0 == no, 1 == yes, 2 == error\n", argv[0]); WriteFile(GetStdHandle(STD_ERROR_HANDLE), msg, chars, |
︙ | ︙ | |||
149 150 151 152 153 154 155 | "Extract a version from a file:\n" "eg: pkgIndex.tcl \"package ifneeded http\"", argv[0]); WriteFile(GetStdHandle(STD_ERROR_HANDLE), msg, chars, &dwWritten, NULL); return 0; } | > > | | > > > > > > > > > > > > > > > | 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 | "Extract a version from a file:\n" "eg: pkgIndex.tcl \"package ifneeded http\"", argv[0]); WriteFile(GetStdHandle(STD_ERROR_HANDLE), msg, chars, &dwWritten, NULL); return 0; } s = GetVersionFromFile(argv[2], argv[3], *(argv[1]+2) - '0'); if (s && *s) { printf("%s\n", s); return 0; } else return 1; /* Version not found. Return non-0 exit code */ case 'Q': if (argc != 3) { chars = snprintf(msg, sizeof(msg) - 1, "usage: %s -Q path\n" "Emit the fully qualified path\n" "exitcodes: 0 == no, 1 == yes, 2 == error\n", argv[0]); WriteFile(GetStdHandle(STD_ERROR_HANDLE), msg, chars, &dwWritten, NULL); return 2; } return QualifyPath(argv[2]); case 'L': if (argc != 3) { chars = snprintf(msg, sizeof(msg) - 1, "usage: %s -L keypath\n" "Emit the fully qualified path of directory containing keypath\n" "exitcodes: 0 == success, 1 == not found, 2 == error\n", argv[0]); WriteFile(GetStdHandle(STD_ERROR_HANDLE), msg, chars, &dwWritten, NULL); return 2; } return LocateDependency(argv[2]); } } chars = snprintf(msg, sizeof(msg) - 1, "usage: %s -c|-f|-l|-Q|-s|-V ...\n" "This is a little helper app to equalize shell differences between WinNT and\n" "Win9x and get nmake.exe to accomplish its job.\n", argv[0]); |
︙ | ︙ | |||
256 257 258 259 260 261 262 | if (!ok) { DWORD err = GetLastError(); int chars = snprintf(msg, sizeof(msg) - 1, "Tried to launch: \"%s\", but got error [%u]: ", cmdline, err); FormatMessage(FORMAT_MESSAGE_FROM_SYSTEM|FORMAT_MESSAGE_IGNORE_INSERTS| | | | 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 | if (!ok) { DWORD err = GetLastError(); int chars = snprintf(msg, sizeof(msg) - 1, "Tried to launch: \"%s\", but got error [%u]: ", cmdline, err); FormatMessage(FORMAT_MESSAGE_FROM_SYSTEM|FORMAT_MESSAGE_IGNORE_INSERTS| FORMAT_MESSAGE_MAX_WIDTH_MASK, 0L, err, 0, (LPSTR)&msg[chars], (300-chars), 0); WriteFile(GetStdHandle(STD_ERROR_HANDLE), msg, lstrlen(msg), &err,NULL); return 2; } /* * Close our references to the write handles that have now been inherited. |
︙ | ︙ | |||
309 310 311 312 313 314 315 | || strstr(Err.buffer, "D9002") != NULL || strstr(Out.buffer, "D2021") != NULL || strstr(Err.buffer, "D2021") != NULL); } static int CheckForLinkerFeature( | | > > | | 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340 341 342 343 344 345 346 347 348 349 | || strstr(Err.buffer, "D9002") != NULL || strstr(Out.buffer, "D2021") != NULL || strstr(Err.buffer, "D2021") != NULL); } static int CheckForLinkerFeature( char **options, int count) { STARTUPINFO si; PROCESS_INFORMATION pi; SECURITY_ATTRIBUTES sa; DWORD threadID; char msg[300]; BOOL ok; HANDLE hProcess, h, pipeThreads[2]; int i; char cmdline[255]; hProcess = GetCurrentProcess(); ZeroMemory(&pi, sizeof(PROCESS_INFORMATION)); ZeroMemory(&si, sizeof(STARTUPINFO)); si.cb = sizeof(STARTUPINFO); si.dwFlags = STARTF_USESTDHANDLES; |
︙ | ︙ | |||
364 365 366 367 368 369 370 | lstrcpy(cmdline, "link.exe -nologo "); /* * Append our option for testing. */ | > | > > > | | 381 382 383 384 385 386 387 388 389 390 391 392 393 394 395 396 397 398 399 400 401 402 403 404 405 406 407 408 409 410 411 412 413 414 415 416 417 418 419 | lstrcpy(cmdline, "link.exe -nologo "); /* * Append our option for testing. */ for (i = 0; i < count; i++) { lstrcat(cmdline, " \""); lstrcat(cmdline, options[i]); lstrcat(cmdline, "\""); } ok = CreateProcess( NULL, /* Module name. */ cmdline, /* Command line. */ NULL, /* Process handle not inheritable. */ NULL, /* Thread handle not inheritable. */ TRUE, /* yes, inherit handles. */ DETACHED_PROCESS, /* No console for you. */ NULL, /* Use parent's environment block. */ NULL, /* Use parent's starting directory. */ &si, /* Pointer to STARTUPINFO structure. */ &pi); /* Pointer to PROCESS_INFORMATION structure. */ if (!ok) { DWORD err = GetLastError(); int chars = snprintf(msg, sizeof(msg) - 1, "Tried to launch: \"%s\", but got error [%u]: ", cmdline, err); FormatMessage(FORMAT_MESSAGE_FROM_SYSTEM|FORMAT_MESSAGE_IGNORE_INSERTS| FORMAT_MESSAGE_MAX_WIDTH_MASK, 0L, err, 0, (LPSTR)&msg[chars], (300-chars), 0); WriteFile(GetStdHandle(STD_ERROR_HANDLE), msg, lstrlen(msg), &err,NULL); return 2; } /* * Close our references to the write handles that have now been inherited. |
︙ | ︙ | |||
429 430 431 432 433 434 435 | /* * Look for the commandline warning code in the stderr stream. */ return !(strstr(Out.buffer, "LNK1117") != NULL || strstr(Err.buffer, "LNK1117") != NULL || strstr(Out.buffer, "LNK4044") != NULL || | | > > | 450 451 452 453 454 455 456 457 458 459 460 461 462 463 464 465 466 | /* * Look for the commandline warning code in the stderr stream. */ return !(strstr(Out.buffer, "LNK1117") != NULL || strstr(Err.buffer, "LNK1117") != NULL || strstr(Out.buffer, "LNK4044") != NULL || strstr(Err.buffer, "LNK4044") != NULL || strstr(Out.buffer, "LNK4224") != NULL || strstr(Err.buffer, "LNK4224") != NULL); } static DWORD WINAPI ReadFromPipe( LPVOID args) { pipeinfo *pi = (pipeinfo *) args; |
︙ | ︙ | |||
475 476 477 478 479 480 481 | * following the match where a version is anything acceptable to * package provide or package ifneeded. */ static const char * GetVersionFromFile( const char *filename, | | > < | | > | | > < | | | 498 499 500 501 502 503 504 505 506 507 508 509 510 511 512 513 514 515 516 517 518 519 520 521 522 523 524 525 526 527 528 529 530 531 532 533 534 535 536 537 538 539 540 541 542 543 544 545 546 547 548 549 | * following the match where a version is anything acceptable to * package provide or package ifneeded. */ static const char * GetVersionFromFile( const char *filename, const char *match, int numdots) { static char szBuffer[100]; char *szResult = NULL; FILE *fp = fopen(filename, "rt"); if (fp != NULL) { /* * Read data until we see our match string. */ while (fgets(szBuffer, sizeof(szBuffer), fp) != NULL) { LPSTR p, q; p = strstr(szBuffer, match); if (p != NULL) { /* * Skip to first digit after the match. */ p += strlen(match); while (*p && !isdigit((unsigned char)*p)) { ++p; } /* * Find ending whitespace. */ q = p; while (*q && (strchr("0123456789.ab", *q)) && (((!strchr(".ab", *q) && !strchr("ab", q[-1])) || --numdots))) { ++q; } *q = 0; szResult = p; break; } } fclose(fp); } return szResult; } |
︙ | ︙ | |||
534 535 536 537 538 539 540 | char * value; } list_item_t; /* insert a list item into the list (list may be null) */ static list_item_t * list_insert(list_item_t **listPtrPtr, const char *key, const char *value) { | | | 558 559 560 561 562 563 564 565 566 567 568 569 570 571 572 | char * value; } list_item_t; /* insert a list item into the list (list may be null) */ static list_item_t * list_insert(list_item_t **listPtrPtr, const char *key, const char *value) { list_item_t *itemPtr = (list_item_t *)malloc(sizeof(list_item_t)); if (itemPtr) { itemPtr->key = strdup(key); itemPtr->value = strdup(value); itemPtr->nextPtr = NULL; while(*listPtrPtr) { listPtrPtr = &(*listPtrPtr)->nextPtr; |
︙ | ︙ | |||
583 584 585 586 587 588 589 | */ static int SubstituteFile( const char *substitutions, const char *filename) { | < < | | | | | | | > > > > > > > > > > > > > > > > > > > > > > > > > > > > | > > > | > > > > > > > > | > > > > > > > > > > > > | > > > > > > > > > > > > > > | > > > > | > > > > > > > | > > > > > > > > > > > > > > > > > | > > > | | > > > | 607 608 609 610 611 612 613 614 615 616 617 618 619 620 621 622 623 624 625 626 627 628 629 630 631 632 633 634 635 636 637 638 639 640 641 642 643 644 645 646 647 648 649 650 651 652 653 654 655 656 657 658 659 660 661 662 663 664 665 666 667 668 669 670 671 672 673 674 675 676 677 678 679 680 681 682 683 684 685 686 687 688 689 690 691 692 693 694 695 696 697 698 699 700 701 702 703 704 705 706 707 708 709 710 711 712 713 714 715 716 717 718 719 720 721 722 723 724 725 726 727 728 729 730 731 732 733 734 735 736 737 738 739 740 741 742 743 744 745 746 747 748 749 750 751 752 753 754 755 756 757 758 759 760 761 762 763 764 765 766 767 768 769 770 771 772 773 774 775 776 777 778 779 780 781 782 783 784 785 786 787 788 789 790 791 792 793 794 795 796 797 798 799 800 801 802 803 804 805 806 807 808 809 810 811 812 813 814 815 | */ static int SubstituteFile( const char *substitutions, const char *filename) { static char szBuffer[1024], szCopy[1024]; list_item_t *substPtr = NULL; FILE *fp, *sp; fp = fopen(filename, "rt"); if (fp != NULL) { /* * Build a list of substutitions from the first filename */ sp = fopen(substitutions, "rt"); if (sp != NULL) { while (fgets(szBuffer, sizeof(szBuffer), sp) != NULL) { unsigned char *ks, *ke, *vs, *ve; ks = (unsigned char*)szBuffer; while (ks && *ks && isspace(*ks)) ++ks; ke = ks; while (ke && *ke && !isspace(*ke)) ++ke; vs = ke; while (vs && *vs && isspace(*vs)) ++vs; ve = vs; while (ve && *ve && !(*ve == '\r' || *ve == '\n')) ++ve; *ke = 0, *ve = 0; list_insert(&substPtr, (char*)ks, (char*)vs); } fclose(sp); } /* debug: dump the list */ #ifndef NDEBUG { int n = 0; list_item_t *p = NULL; for (p = substPtr; p != NULL; p = p->nextPtr, ++n) { fprintf(stderr, "% 3d '%s' => '%s'\n", n, p->key, p->value); } } #endif /* * Run the substitutions over each line of the input */ while (fgets(szBuffer, sizeof(szBuffer), fp) != NULL) { list_item_t *p = NULL; for (p = substPtr; p != NULL; p = p->nextPtr) { char *m = strstr(szBuffer, p->key); if (m) { char *cp, *op, *sp; cp = szCopy; op = szBuffer; while (op != m) *cp++ = *op++; sp = p->value; while (sp && *sp) *cp++ = *sp++; op += strlen(p->key); while (*op) *cp++ = *op++; *cp = 0; memcpy(szBuffer, szCopy, sizeof(szCopy)); } } printf("%s", szBuffer); } list_free(&substPtr); } fclose(fp); return 0; } BOOL FileExists(LPCTSTR szPath) { #ifndef INVALID_FILE_ATTRIBUTES #define INVALID_FILE_ATTRIBUTES ((DWORD)-1) #endif DWORD pathAttr = GetFileAttributes(szPath); return (pathAttr != INVALID_FILE_ATTRIBUTES && !(pathAttr & FILE_ATTRIBUTE_DIRECTORY)); } /* * QualifyPath -- * * This composes the current working directory with a provided path * and returns the fully qualified and normalized path. * Mostly needed to setup paths for testing. */ static int QualifyPath( const char *szPath) { char szCwd[MAX_PATH + 1]; GetFullPathName(szPath, sizeof(szCwd)-1, szCwd, NULL); printf("%s\n", szCwd); return 0; } /* * Implements LocateDependency for a single directory. See that command * for an explanation. * Returns 0 if found after printing the directory. * Returns 1 if not found but no errors. * Returns 2 on any kind of error * Basically, these are used as exit codes for the process. */ static int LocateDependencyHelper(const char *dir, const char *keypath) { HANDLE hSearch; char path[MAX_PATH+1]; size_t dirlen; int keylen, ret; WIN32_FIND_DATA finfo; if (dir == NULL || keypath == NULL) return 2; /* Have no real error reporting mechanism into nmake */ dirlen = strlen(dir); if ((dirlen + 3) > sizeof(path)) return 2; strncpy(path, dir, dirlen); strncpy(path+dirlen, "\\*", 3); /* Including terminating \0 */ keylen = strlen(keypath); #if 0 /* This function is not available in Visual C++ 6 */ /* * Use numerics 0 -> FindExInfoStandard, * 1 -> FindExSearchLimitToDirectories, * as these are not defined in Visual C++ 6 */ hSearch = FindFirstFileEx(path, 0, &finfo, 1, NULL, 0); #else hSearch = FindFirstFile(path, &finfo); #endif if (hSearch == INVALID_HANDLE_VALUE) return 1; /* Not found */ /* Loop through all subdirs checking if the keypath is under there */ ret = 1; /* Assume not found */ do { int sublen; /* * We need to check it is a directory despite the * FindExSearchLimitToDirectories in the above call. See SDK docs */ if ((finfo.dwFileAttributes & FILE_ATTRIBUTE_DIRECTORY) == 0) continue; sublen = strlen(finfo.cFileName); if ((dirlen+1+sublen+1+keylen+1) > sizeof(path)) continue; /* Path does not fit, assume not matched */ strncpy(path+dirlen+1, finfo.cFileName, sublen); path[dirlen+1+sublen] = '\\'; strncpy(path+dirlen+1+sublen+1, keypath, keylen+1); if (FileExists(path)) { /* Found a match, print to stdout */ path[dirlen+1+sublen] = '\0'; QualifyPath(path); ret = 0; break; } } while (FindNextFile(hSearch, &finfo)); FindClose(hSearch); return ret; } /* * LocateDependency -- * * Locates a dependency for a package. * keypath - a relative path within the package directory * that is used to confirm it is the correct directory. * The search path for the package directory is currently only * the parent and grandparent of the current working directory. * If found, the command prints * name_DIRPATH=<full path of located directory> * and returns 0. If not found, does not print anything and returns 1. */ static int LocateDependency(const char *keypath) { size_t i; int ret; static const char *paths[] = {"..", "..\\..", "..\\..\\.."}; for (i = 0; i < (sizeof(paths)/sizeof(paths[0])); ++i) { ret = LocateDependencyHelper(paths[i], keypath); if (ret == 0) return ret; } return ret; } /* * Local variables: * mode: c * c-basic-offset: 4 * fill-column: 78 * indent-tabs-mode: t * tab-width: 8 * End: */ |
Changes to config.guess.
1 2 | #! /bin/sh # Attempt to guess a canonical system name. | < | < | | | < < | | > | | < < > | < < > | < | < | | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 | #! /bin/sh # Attempt to guess a canonical system name. # Copyright 1992-2019 Free Software Foundation, Inc. timestamp='2019-05-28' # This file is free software; you can redistribute it and/or modify it # under the terms of the GNU General Public License as published by # the Free Software Foundation; either version 3 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, but # WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU # General Public License for more details. # # You should have received a copy of the GNU General Public License # along with this program; if not, see <https://www.gnu.org/licenses/>. # # As a special exception to the GNU General Public License, if you # distribute this file as part of a program that contains a # configuration script generated by Autoconf, you may include it under # the same distribution terms that you use for the rest of that # program. This Exception is an additional permission under section 7 # of the GNU General Public License, version 3 ("GPLv3"). # # Originally written by Per Bothner; maintained since 2000 by Ben Elliston. # # You can get the latest version of this script from: # https://git.savannah.gnu.org/gitweb/?p=config.git;a=blob_plain;f=config.guess # # Please send patches to <config-patches@gnu.org>. me=`echo "$0" | sed -e 's,.*/,,'` usage="\ Usage: $0 [OPTION] Output the configuration name of the system \`$me' is run on. Options: -h, --help print this help, then exit -t, --time-stamp print date of last modification, then exit -v, --version print version number, then exit Report bugs and patches to <config-patches@gnu.org>." version="\ GNU config.guess ($timestamp) Originally written by Per Bothner. Copyright 1992-2019 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE." help=" Try \`$me --help' for more information." |
︙ | ︙ | |||
87 88 89 90 91 92 93 | done if test $# != 0; then echo "$me: too many arguments$help" >&2 exit 1 fi | < < > > > > | < < | > | | | | | < | | | | | > | | | | | | | | | | > | | > > > > > | | | < < < > | | > | > > > > > | > > | | | > > | | > > > > > | | | > > > | | | > > > > > > > | | | > > > > > > > | > > > > | | | > > > | > > > > > > | | | | | | | | | | | | | | | | | > | < < < < < | < < | | | | | | | 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340 341 342 343 344 345 346 347 348 349 350 351 352 353 354 355 356 357 358 359 360 361 362 363 364 | done if test $# != 0; then echo "$me: too many arguments$help" >&2 exit 1 fi # CC_FOR_BUILD -- compiler used by this script. Note that the use of a # compiler to aid in system detection is discouraged as it requires # temporary files to be created and, as you can see below, it is a # headache to deal with in a portable fashion. # Historically, `CC_FOR_BUILD' used to be named `HOST_CC'. We still # use `HOST_CC' if defined, but it is deprecated. # Portable tmp directory creation inspired by the Autoconf team. tmp= # shellcheck disable=SC2172 trap 'test -z "$tmp" || rm -fr "$tmp"' 0 1 2 13 15 set_cc_for_build() { : "${TMPDIR=/tmp}" # shellcheck disable=SC2039 { tmp=`(umask 077 && mktemp -d "$TMPDIR/cgXXXXXX") 2>/dev/null` && test -n "$tmp" && test -d "$tmp" ; } || { test -n "$RANDOM" && tmp=$TMPDIR/cg$$-$RANDOM && (umask 077 && mkdir "$tmp" 2>/dev/null) ; } || { tmp=$TMPDIR/cg-$$ && (umask 077 && mkdir "$tmp" 2>/dev/null) && echo "Warning: creating insecure temp directory" >&2 ; } || { echo "$me: cannot create a temporary directory in $TMPDIR" >&2 ; exit 1 ; } dummy=$tmp/dummy case ${CC_FOR_BUILD-},${HOST_CC-},${CC-} in ,,) echo "int x;" > "$dummy.c" for driver in cc gcc c89 c99 ; do if ($driver -c -o "$dummy.o" "$dummy.c") >/dev/null 2>&1 ; then CC_FOR_BUILD="$driver" break fi done if test x"$CC_FOR_BUILD" = x ; then CC_FOR_BUILD=no_compiler_found fi ;; ,,*) CC_FOR_BUILD=$CC ;; ,*,*) CC_FOR_BUILD=$HOST_CC ;; esac } # This is needed to find uname on a Pyramid OSx when run in the BSD universe. # (ghazi@noc.rutgers.edu 1994-08-24) if test -f /.attbin/uname ; then PATH=$PATH:/.attbin ; export PATH fi UNAME_MACHINE=`(uname -m) 2>/dev/null` || UNAME_MACHINE=unknown UNAME_RELEASE=`(uname -r) 2>/dev/null` || UNAME_RELEASE=unknown UNAME_SYSTEM=`(uname -s) 2>/dev/null` || UNAME_SYSTEM=unknown UNAME_VERSION=`(uname -v) 2>/dev/null` || UNAME_VERSION=unknown case "$UNAME_SYSTEM" in Linux|GNU|GNU/*) # If the system lacks a compiler, then just pick glibc. # We could probably try harder. LIBC=gnu set_cc_for_build cat <<-EOF > "$dummy.c" #include <features.h> #if defined(__UCLIBC__) LIBC=uclibc #elif defined(__dietlibc__) LIBC=dietlibc #else LIBC=gnu #endif EOF eval "`$CC_FOR_BUILD -E "$dummy.c" 2>/dev/null | grep '^LIBC' | sed 's, ,,g'`" # If ldd exists, use it to detect musl libc. if command -v ldd >/dev/null && \ ldd --version 2>&1 | grep -q ^musl then LIBC=musl fi ;; esac # Note: order is significant - the case branches are not exclusive. case "$UNAME_MACHINE:$UNAME_SYSTEM:$UNAME_RELEASE:$UNAME_VERSION" in *:NetBSD:*:*) # NetBSD (nbsd) targets should (where applicable) match one or # more of the tuples: *-*-netbsdelf*, *-*-netbsdaout*, # *-*-netbsdecoff* and *-*-netbsd*. For targets that recently # switched to ELF, *-*-netbsd* would select the old # object file format. This provides both forward # compatibility and a consistent mechanism for selecting the # object file format. # # Note: NetBSD doesn't particularly care about the vendor # portion of the name. We always set it to "unknown". sysctl="sysctl -n hw.machine_arch" UNAME_MACHINE_ARCH=`(uname -p 2>/dev/null || \ "/sbin/$sysctl" 2>/dev/null || \ "/usr/sbin/$sysctl" 2>/dev/null || \ echo unknown)` case "$UNAME_MACHINE_ARCH" in armeb) machine=armeb-unknown ;; arm*) machine=arm-unknown ;; sh3el) machine=shl-unknown ;; sh3eb) machine=sh-unknown ;; sh5el) machine=sh5le-unknown ;; earmv*) arch=`echo "$UNAME_MACHINE_ARCH" | sed -e 's,^e\(armv[0-9]\).*$,\1,'` endian=`echo "$UNAME_MACHINE_ARCH" | sed -ne 's,^.*\(eb\)$,\1,p'` machine="${arch}${endian}"-unknown ;; *) machine="$UNAME_MACHINE_ARCH"-unknown ;; esac # The Operating System including object format, if it has switched # to ELF recently (or will in the future) and ABI. case "$UNAME_MACHINE_ARCH" in earm*) os=netbsdelf ;; arm*|i386|m68k|ns32k|sh3*|sparc|vax) set_cc_for_build if echo __ELF__ | $CC_FOR_BUILD -E - 2>/dev/null \ | grep -q __ELF__ then # Once all utilities can be ECOFF (netbsdecoff) or a.out (netbsdaout). # Return netbsd for either. FIX? os=netbsd else os=netbsdelf fi ;; *) os=netbsd ;; esac # Determine ABI tags. case "$UNAME_MACHINE_ARCH" in earm*) expr='s/^earmv[0-9]/-eabi/;s/eb$//' abi=`echo "$UNAME_MACHINE_ARCH" | sed -e "$expr"` ;; esac # The OS release # Debian GNU/NetBSD machines have a different userland, and # thus, need a distinct triplet. However, they do not need # kernel version information, so it can be replaced with a # suitable tag, in the style of linux-gnu. case "$UNAME_VERSION" in Debian*) release='-gnu' ;; *) release=`echo "$UNAME_RELEASE" | sed -e 's/[-_].*//' | cut -d. -f1,2` ;; esac # Since CPU_TYPE-MANUFACTURER-KERNEL-OPERATING_SYSTEM: # contains redundant information, the shorter form: # CPU_TYPE-MANUFACTURER-OPERATING_SYSTEM is used. echo "$machine-${os}${release}${abi-}" exit ;; *:Bitrig:*:*) UNAME_MACHINE_ARCH=`arch | sed 's/Bitrig.//'` echo "$UNAME_MACHINE_ARCH"-unknown-bitrig"$UNAME_RELEASE" exit ;; *:OpenBSD:*:*) UNAME_MACHINE_ARCH=`arch | sed 's/OpenBSD.//'` echo "$UNAME_MACHINE_ARCH"-unknown-openbsd"$UNAME_RELEASE" exit ;; *:LibertyBSD:*:*) UNAME_MACHINE_ARCH=`arch | sed 's/^.*BSD\.//'` echo "$UNAME_MACHINE_ARCH"-unknown-libertybsd"$UNAME_RELEASE" exit ;; *:MidnightBSD:*:*) echo "$UNAME_MACHINE"-unknown-midnightbsd"$UNAME_RELEASE" exit ;; *:ekkoBSD:*:*) echo "$UNAME_MACHINE"-unknown-ekkobsd"$UNAME_RELEASE" exit ;; *:SolidBSD:*:*) echo "$UNAME_MACHINE"-unknown-solidbsd"$UNAME_RELEASE" exit ;; macppc:MirBSD:*:*) echo powerpc-unknown-mirbsd"$UNAME_RELEASE" exit ;; *:MirBSD:*:*) echo "$UNAME_MACHINE"-unknown-mirbsd"$UNAME_RELEASE" exit ;; *:Sortix:*:*) echo "$UNAME_MACHINE"-unknown-sortix exit ;; *:Redox:*:*) echo "$UNAME_MACHINE"-unknown-redox exit ;; mips:OSF1:*.*) echo mips-dec-osf1 exit ;; alpha:OSF1:*:*) case $UNAME_RELEASE in *4.0) UNAME_RELEASE=`/usr/sbin/sizer -v | awk '{print $3}'` ;; *5.*) UNAME_RELEASE=`/usr/sbin/sizer -v | awk '{print $4}'` ;; esac # According to Compaq, /usr/sbin/psrinfo has been available on # OSF/1 and Tru64 systems produced since 1995. I hope that # covers most systems running today. This code pipes the CPU # types through head -n 1, so we only detect the type of CPU 0. ALPHA_CPU_TYPE=`/usr/sbin/psrinfo -v | sed -n -e 's/^ The alpha \(.*\) processor.*$/\1/p' | head -n 1` case "$ALPHA_CPU_TYPE" in "EV4 (21064)") UNAME_MACHINE=alpha ;; "EV4.5 (21064)") UNAME_MACHINE=alpha ;; "LCA4 (21066/21068)") UNAME_MACHINE=alpha ;; "EV5 (21164)") UNAME_MACHINE=alphaev5 ;; "EV5.6 (21164A)") UNAME_MACHINE=alphaev56 ;; "EV5.6 (21164PC)") UNAME_MACHINE=alphapca56 ;; "EV5.7 (21164PC)") UNAME_MACHINE=alphapca57 ;; "EV6 (21264)") UNAME_MACHINE=alphaev6 ;; "EV6.7 (21264A)") UNAME_MACHINE=alphaev67 ;; "EV6.8CB (21264C)") UNAME_MACHINE=alphaev68 ;; "EV6.8AL (21264B)") UNAME_MACHINE=alphaev68 ;; "EV6.8CX (21264D)") UNAME_MACHINE=alphaev68 ;; "EV6.9A (21264/EV69A)") UNAME_MACHINE=alphaev69 ;; "EV7 (21364)") UNAME_MACHINE=alphaev7 ;; "EV7.9 (21364A)") UNAME_MACHINE=alphaev79 ;; esac # A Pn.n version is a patched version. # A Vn.n version is a released version. # A Tn.n version is a released field test version. # A Xn.n version is an unreleased experimental baselevel. # 1.2 uses "1.2" for uname -r. echo "$UNAME_MACHINE"-dec-osf"`echo "$UNAME_RELEASE" | sed -e 's/^[PVTX]//' | tr ABCDEFGHIJKLMNOPQRSTUVWXYZ abcdefghijklmnopqrstuvwxyz`" # Reset EXIT trap before exiting to avoid spurious non-zero exit code. exitcode=$? trap '' 0 exit $exitcode ;; Amiga*:UNIX_System_V:4.0:*) echo m68k-unknown-sysv4 exit ;; *:[Aa]miga[Oo][Ss]:*:*) echo "$UNAME_MACHINE"-unknown-amigaos exit ;; *:[Mm]orph[Oo][Ss]:*:*) echo "$UNAME_MACHINE"-unknown-morphos exit ;; *:OS/390:*:*) echo i370-ibm-openedition exit ;; *:z/VM:*:*) echo s390-ibm-zvmoe exit ;; *:OS400:*:*) echo powerpc-ibm-os400 exit ;; arm:RISC*:1.[012]*:*|arm:riscix:1.[012]*:*) echo arm-acorn-riscix"$UNAME_RELEASE" exit ;; arm*:riscos:*:*|arm*:RISCOS:*:*) echo arm-unknown-riscos exit ;; SR2?01:HI-UX/MPP:*:* | SR8000:HI-UX/MPP:*:*) echo hppa1.1-hitachi-hiuxmpp exit ;; Pyramid*:OSx*:*:* | MIS*:OSx*:*:* | MIS*:SMP_DC-OSx*:*:*) # akee@wpdis03.wpafb.af.mil (Earle F. Ake) contributed MIS and NILE. |
︙ | ︙ | |||
337 338 339 340 341 342 343 344 | DRS?6000:unix:4.0:6*) echo sparc-icl-nx6 exit ;; DRS?6000:UNIX_SV:4.2*:7* | DRS?6000:isis:4.2*:7*) case `/usr/bin/uname -p` in sparc) echo sparc-icl-nx7; exit ;; esac ;; sun4H:SunOS:5.*:*) | > > > | | > > > > > > > > > > > > > > > > | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | 374 375 376 377 378 379 380 381 382 383 384 385 386 387 388 389 390 391 392 393 394 395 396 397 398 399 400 401 402 403 404 405 406 407 408 409 410 411 412 413 414 415 416 417 418 419 420 421 422 423 424 425 426 427 428 429 430 431 432 433 434 435 436 437 438 439 440 441 442 443 444 445 446 447 448 449 450 451 452 453 454 455 456 457 458 459 460 461 462 463 464 465 466 467 468 469 470 471 472 473 474 475 476 477 478 479 480 481 482 483 484 485 486 487 488 489 490 491 492 493 494 495 496 497 498 499 500 501 502 503 504 505 506 507 508 509 510 511 512 513 514 515 516 517 518 519 520 | DRS?6000:unix:4.0:6*) echo sparc-icl-nx6 exit ;; DRS?6000:UNIX_SV:4.2*:7* | DRS?6000:isis:4.2*:7*) case `/usr/bin/uname -p` in sparc) echo sparc-icl-nx7; exit ;; esac ;; s390x:SunOS:*:*) echo "$UNAME_MACHINE"-ibm-solaris2"`echo "$UNAME_RELEASE" | sed -e 's/[^.]*//'`" exit ;; sun4H:SunOS:5.*:*) echo sparc-hal-solaris2"`echo "$UNAME_RELEASE"|sed -e 's/[^.]*//'`" exit ;; sun4*:SunOS:5.*:* | tadpole*:SunOS:5.*:*) echo sparc-sun-solaris2"`echo "$UNAME_RELEASE" | sed -e 's/[^.]*//'`" exit ;; i86pc:AuroraUX:5.*:* | i86xen:AuroraUX:5.*:*) echo i386-pc-auroraux"$UNAME_RELEASE" exit ;; i86pc:SunOS:5.*:* | i86xen:SunOS:5.*:*) set_cc_for_build SUN_ARCH=i386 # If there is a compiler, see if it is configured for 64-bit objects. # Note that the Sun cc does not turn __LP64__ into 1 like gcc does. # This test works for both compilers. if [ "$CC_FOR_BUILD" != no_compiler_found ]; then if (echo '#ifdef __amd64'; echo IS_64BIT_ARCH; echo '#endif') | \ (CCOPTS="" $CC_FOR_BUILD -E - 2>/dev/null) | \ grep IS_64BIT_ARCH >/dev/null then SUN_ARCH=x86_64 fi fi echo "$SUN_ARCH"-pc-solaris2"`echo "$UNAME_RELEASE"|sed -e 's/[^.]*//'`" exit ;; sun4*:SunOS:6*:*) # According to config.sub, this is the proper way to canonicalize # SunOS6. Hard to guess exactly what SunOS6 will be like, but # it's likely to be more like Solaris than SunOS4. echo sparc-sun-solaris3"`echo "$UNAME_RELEASE"|sed -e 's/[^.]*//'`" exit ;; sun4*:SunOS:*:*) case "`/usr/bin/arch -k`" in Series*|S4*) UNAME_RELEASE=`uname -v` ;; esac # Japanese Language versions have a version number like `4.1.3-JL'. echo sparc-sun-sunos"`echo "$UNAME_RELEASE"|sed -e 's/-/_/'`" exit ;; sun3*:SunOS:*:*) echo m68k-sun-sunos"$UNAME_RELEASE" exit ;; sun*:*:4.2BSD:*) UNAME_RELEASE=`(sed 1q /etc/motd | awk '{print substr($5,1,3)}') 2>/dev/null` test "x$UNAME_RELEASE" = x && UNAME_RELEASE=3 case "`/bin/arch`" in sun3) echo m68k-sun-sunos"$UNAME_RELEASE" ;; sun4) echo sparc-sun-sunos"$UNAME_RELEASE" ;; esac exit ;; aushp:SunOS:*:*) echo sparc-auspex-sunos"$UNAME_RELEASE" exit ;; # The situation for MiNT is a little confusing. The machine name # can be virtually everything (everything which is not # "atarist" or "atariste" at least should have a processor # > m68000). The system name ranges from "MiNT" over "FreeMiNT" # to the lowercase version "mint" (or "freemint"). Finally # the system name "TOS" denotes a system which is actually not # MiNT. But MiNT is downward compatible to TOS, so this should # be no problem. atarist[e]:*MiNT:*:* | atarist[e]:*mint:*:* | atarist[e]:*TOS:*:*) echo m68k-atari-mint"$UNAME_RELEASE" exit ;; atari*:*MiNT:*:* | atari*:*mint:*:* | atarist[e]:*TOS:*:*) echo m68k-atari-mint"$UNAME_RELEASE" exit ;; *falcon*:*MiNT:*:* | *falcon*:*mint:*:* | *falcon*:*TOS:*:*) echo m68k-atari-mint"$UNAME_RELEASE" exit ;; milan*:*MiNT:*:* | milan*:*mint:*:* | *milan*:*TOS:*:*) echo m68k-milan-mint"$UNAME_RELEASE" exit ;; hades*:*MiNT:*:* | hades*:*mint:*:* | *hades*:*TOS:*:*) echo m68k-hades-mint"$UNAME_RELEASE" exit ;; *:*MiNT:*:* | *:*mint:*:* | *:*TOS:*:*) echo m68k-unknown-mint"$UNAME_RELEASE" exit ;; m68k:machten:*:*) echo m68k-apple-machten"$UNAME_RELEASE" exit ;; powerpc:machten:*:*) echo powerpc-apple-machten"$UNAME_RELEASE" exit ;; RISC*:Mach:*:*) echo mips-dec-mach_bsd4.3 exit ;; RISC*:ULTRIX:*:*) echo mips-dec-ultrix"$UNAME_RELEASE" exit ;; VAX*:ULTRIX*:*:*) echo vax-dec-ultrix"$UNAME_RELEASE" exit ;; 2020:CLIX:*:* | 2430:CLIX:*:*) echo clipper-intergraph-clix"$UNAME_RELEASE" exit ;; mips:*:*:UMIPS | mips:*:*:RISCos) set_cc_for_build sed 's/^ //' << EOF > "$dummy.c" #ifdef __cplusplus #include <stdio.h> /* for printf() prototype */ int main (int argc, char *argv[]) { #else int main (argc, argv) int argc; char *argv[]; { #endif #if defined (host_mips) && defined (MIPSEB) #if defined (SYSTYPE_SYSV) printf ("mips-mips-riscos%ssysv\\n", argv[1]); exit (0); #endif #if defined (SYSTYPE_SVR4) printf ("mips-mips-riscos%ssvr4\\n", argv[1]); exit (0); #endif #if defined (SYSTYPE_BSD43) || defined(SYSTYPE_BSD) printf ("mips-mips-riscos%sbsd\\n", argv[1]); exit (0); #endif #endif exit (-1); } EOF $CC_FOR_BUILD -o "$dummy" "$dummy.c" && dummyarg=`echo "$UNAME_RELEASE" | sed -n 's/\([0-9]*\).*/\1/p'` && SYSTEM_NAME=`"$dummy" "$dummyarg"` && { echo "$SYSTEM_NAME"; exit; } echo mips-mips-riscos"$UNAME_RELEASE" exit ;; Motorola:PowerMAX_OS:*:*) echo powerpc-motorola-powermax exit ;; Motorola:*:4.3:PL8-*) echo powerpc-harris-powermax exit ;; |
︙ | ︙ | |||
474 475 476 477 478 479 480 | m88k:*:4*:R4*) echo m88k-motorola-sysv4 exit ;; m88k:*:3*:R3*) echo m88k-motorola-sysv3 exit ;; AViiON:dgux:*:*) | | | | | | | | | | | | | | | | | | | | > | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | 530 531 532 533 534 535 536 537 538 539 540 541 542 543 544 545 546 547 548 549 550 551 552 553 554 555 556 557 558 559 560 561 562 563 564 565 566 567 568 569 570 571 572 573 574 575 576 577 578 579 580 581 582 583 584 585 586 587 588 589 590 591 592 593 594 595 596 597 598 599 600 601 602 603 604 605 606 607 608 609 610 611 612 613 614 615 616 617 618 619 620 621 622 623 624 625 626 627 628 629 630 631 632 633 634 635 636 637 638 639 640 641 642 643 644 645 646 647 648 649 650 651 652 653 654 655 656 657 658 659 660 661 662 663 664 665 666 667 668 669 670 671 672 673 674 675 676 677 678 679 680 681 682 683 684 685 686 687 688 689 690 691 692 693 694 695 696 697 698 699 700 701 702 703 704 705 706 707 708 709 710 711 712 713 714 715 716 717 718 719 720 721 722 723 724 725 726 727 728 729 730 731 732 733 734 735 736 737 738 739 | m88k:*:4*:R4*) echo m88k-motorola-sysv4 exit ;; m88k:*:3*:R3*) echo m88k-motorola-sysv3 exit ;; AViiON:dgux:*:*) # DG/UX returns AViiON for all architectures UNAME_PROCESSOR=`/usr/bin/uname -p` if [ "$UNAME_PROCESSOR" = mc88100 ] || [ "$UNAME_PROCESSOR" = mc88110 ] then if [ "$TARGET_BINARY_INTERFACE"x = m88kdguxelfx ] || \ [ "$TARGET_BINARY_INTERFACE"x = x ] then echo m88k-dg-dgux"$UNAME_RELEASE" else echo m88k-dg-dguxbcs"$UNAME_RELEASE" fi else echo i586-dg-dgux"$UNAME_RELEASE" fi exit ;; M88*:DolphinOS:*:*) # DolphinOS (SVR3) echo m88k-dolphin-sysv3 exit ;; M88*:*:R3*:*) # Delta 88k system running SVR3 echo m88k-motorola-sysv3 exit ;; XD88*:*:*:*) # Tektronix XD88 system running UTekV (SVR3) echo m88k-tektronix-sysv3 exit ;; Tek43[0-9][0-9]:UTek:*:*) # Tektronix 4300 system running UTek (BSD) echo m68k-tektronix-bsd exit ;; *:IRIX*:*:*) echo mips-sgi-irix"`echo "$UNAME_RELEASE"|sed -e 's/-/_/g'`" exit ;; ????????:AIX?:[12].1:2) # AIX 2.2.1 or AIX 2.1.1 is RT/PC AIX. echo romp-ibm-aix # uname -m gives an 8 hex-code CPU id exit ;; # Note that: echo "'`uname -s`'" gives 'AIX ' i*86:AIX:*:*) echo i386-ibm-aix exit ;; ia64:AIX:*:*) if [ -x /usr/bin/oslevel ] ; then IBM_REV=`/usr/bin/oslevel` else IBM_REV="$UNAME_VERSION.$UNAME_RELEASE" fi echo "$UNAME_MACHINE"-ibm-aix"$IBM_REV" exit ;; *:AIX:2:3) if grep bos325 /usr/include/stdio.h >/dev/null 2>&1; then set_cc_for_build sed 's/^ //' << EOF > "$dummy.c" #include <sys/systemcfg.h> main() { if (!__power_pc()) exit(1); puts("powerpc-ibm-aix3.2.5"); exit(0); } EOF if $CC_FOR_BUILD -o "$dummy" "$dummy.c" && SYSTEM_NAME=`"$dummy"` then echo "$SYSTEM_NAME" else echo rs6000-ibm-aix3.2.5 fi elif grep bos324 /usr/include/stdio.h >/dev/null 2>&1; then echo rs6000-ibm-aix3.2.4 else echo rs6000-ibm-aix3.2 fi exit ;; *:AIX:*:[4567]) IBM_CPU_ID=`/usr/sbin/lsdev -C -c processor -S available | sed 1q | awk '{ print $1 }'` if /usr/sbin/lsattr -El "$IBM_CPU_ID" | grep ' POWER' >/dev/null 2>&1; then IBM_ARCH=rs6000 else IBM_ARCH=powerpc fi if [ -x /usr/bin/lslpp ] ; then IBM_REV=`/usr/bin/lslpp -Lqc bos.rte.libc | awk -F: '{ print $3 }' | sed s/[0-9]*$/0/` else IBM_REV="$UNAME_VERSION.$UNAME_RELEASE" fi echo "$IBM_ARCH"-ibm-aix"$IBM_REV" exit ;; *:AIX:*:*) echo rs6000-ibm-aix exit ;; ibmrt:4.4BSD:*|romp-ibm:4.4BSD:*) echo romp-ibm-bsd4.4 exit ;; ibmrt:*BSD:*|romp-ibm:BSD:*) # covers RT/PC BSD and echo romp-ibm-bsd"$UNAME_RELEASE" # 4.3 with uname added to exit ;; # report: romp-ibm BSD 4.3 *:BOSX:*:*) echo rs6000-bull-bosx exit ;; DPX/2?00:B.O.S.:*:*) echo m68k-bull-sysv3 exit ;; 9000/[34]??:4.3bsd:1.*:*) echo m68k-hp-bsd exit ;; hp300:4.4BSD:*:* | 9000/[34]??:4.3bsd:2.*:*) echo m68k-hp-bsd4.4 exit ;; 9000/[34678]??:HP-UX:*:*) HPUX_REV=`echo "$UNAME_RELEASE"|sed -e 's/[^.]*.[0B]*//'` case "$UNAME_MACHINE" in 9000/31?) HP_ARCH=m68000 ;; 9000/[34]??) HP_ARCH=m68k ;; 9000/[678][0-9][0-9]) if [ -x /usr/bin/getconf ]; then sc_cpu_version=`/usr/bin/getconf SC_CPU_VERSION 2>/dev/null` sc_kernel_bits=`/usr/bin/getconf SC_KERNEL_BITS 2>/dev/null` case "$sc_cpu_version" in 523) HP_ARCH=hppa1.0 ;; # CPU_PA_RISC1_0 528) HP_ARCH=hppa1.1 ;; # CPU_PA_RISC1_1 532) # CPU_PA_RISC2_0 case "$sc_kernel_bits" in 32) HP_ARCH=hppa2.0n ;; 64) HP_ARCH=hppa2.0w ;; '') HP_ARCH=hppa2.0 ;; # HP-UX 10.20 esac ;; esac fi if [ "$HP_ARCH" = "" ]; then set_cc_for_build sed 's/^ //' << EOF > "$dummy.c" #define _HPUX_SOURCE #include <stdlib.h> #include <unistd.h> int main () { #if defined(_SC_KERNEL_BITS) long bits = sysconf(_SC_KERNEL_BITS); #endif long cpu = sysconf (_SC_CPU_VERSION); switch (cpu) { case CPU_PA_RISC1_0: puts ("hppa1.0"); break; case CPU_PA_RISC1_1: puts ("hppa1.1"); break; case CPU_PA_RISC2_0: #if defined(_SC_KERNEL_BITS) switch (bits) { case 64: puts ("hppa2.0w"); break; case 32: puts ("hppa2.0n"); break; default: puts ("hppa2.0"); break; } break; #else /* !defined(_SC_KERNEL_BITS) */ puts ("hppa2.0"); break; #endif default: puts ("hppa1.0"); break; } exit (0); } EOF (CCOPTS="" $CC_FOR_BUILD -o "$dummy" "$dummy.c" 2>/dev/null) && HP_ARCH=`"$dummy"` test -z "$HP_ARCH" && HP_ARCH=hppa fi ;; esac if [ "$HP_ARCH" = hppa2.0w ] then set_cc_for_build # hppa2.0w-hp-hpux* has a 64-bit kernel and a compiler generating # 32-bit code. hppa64-hp-hpux* has the same kernel and a compiler # generating 64-bit code. GNU and HP use different nomenclature: # # $ CC_FOR_BUILD=cc ./config.guess # => hppa2.0w-hp-hpux11.23 # $ CC_FOR_BUILD="cc +DA2.0w" ./config.guess # => hppa64-hp-hpux11.23 if echo __LP64__ | (CCOPTS="" $CC_FOR_BUILD -E - 2>/dev/null) | grep -q __LP64__ then HP_ARCH=hppa2.0w else HP_ARCH=hppa64 fi fi echo "$HP_ARCH"-hp-hpux"$HPUX_REV" exit ;; ia64:HP-UX:*:*) HPUX_REV=`echo "$UNAME_RELEASE"|sed -e 's/[^.]*.[0B]*//'` echo ia64-hp-hpux"$HPUX_REV" exit ;; 3050*:HI-UX:*:*) set_cc_for_build sed 's/^ //' << EOF > "$dummy.c" #include <unistd.h> int main () { long cpu = sysconf (_SC_CPU_VERSION); /* The order matters, because CPU_IS_HP_MC68K erroneously returns true for CPU_PA_RISC1_0. CPU_IS_PA_RISC returns correct |
︙ | ︙ | |||
693 694 695 696 697 698 699 | } else if (CPU_IS_HP_MC68K (cpu)) puts ("m68k-hitachi-hiuxwe2"); else puts ("unknown-hitachi-hiuxwe2"); exit (0); } EOF | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | > > > | > > > > > > > | < > | > | > > > | | < | | | | | | | < < > | | < < < < < | | | < < < | | | | | < < < < < < < < < < < < < < < | | | | < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < | < < < < < < | | | > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | > > > > > > > > > | | | | | > > > > > > > > > > > > | | | | > > > | | | | < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < | | | | | | | | | | | | | | | | | | | | | | | > > > | | | | | | | | > > > > > > > > > > | | | | | | | | | | | | | | | | > > > > > > | | | | | | > > > | | | > > > > > > > > > > > > > > > > > > > > > > > > | > > > > > > | | | > > > | | | > > > > > > | > | | | | | | | | > > > > > > > > > > > > | < < | | | | > > > > > > > > | | | < < < < < < < < | 750 751 752 753 754 755 756 757 758 759 760 761 762 763 764 765 766 767 768 769 770 771 772 773 774 775 776 777 778 779 780 781 782 783 784 785 786 787 788 789 790 791 792 793 794 795 796 797 798 799 800 801 802 803 804 805 806 807 808 809 810 811 812 813 814 815 816 817 818 819 820 821 822 823 824 825 826 827 828 829 830 831 832 833 834 835 836 837 838 839 840 841 842 843 844 845 846 847 848 849 850 851 852 853 854 855 856 857 858 859 860 861 862 863 864 865 866 867 868 869 870 871 872 873 874 875 876 877 878 879 880 881 882 883 884 885 886 887 888 889 890 891 892 893 894 895 896 897 898 899 900 901 902 903 904 905 906 907 908 909 910 911 912 913 914 915 916 917 918 919 920 921 922 923 924 925 926 927 928 929 930 931 932 933 934 935 936 937 938 939 940 941 942 943 944 945 946 947 948 949 950 951 952 953 954 955 956 957 958 959 960 961 962 963 964 965 966 967 968 969 970 971 972 973 974 975 976 977 978 979 980 981 982 983 984 985 986 987 988 989 990 991 992 993 994 995 996 997 998 999 1000 1001 1002 1003 1004 1005 1006 1007 1008 1009 1010 1011 1012 1013 1014 1015 1016 1017 1018 1019 1020 1021 1022 1023 1024 1025 1026 1027 1028 1029 1030 1031 1032 1033 1034 1035 1036 1037 1038 1039 1040 1041 1042 1043 1044 1045 1046 1047 1048 1049 1050 1051 1052 1053 1054 1055 1056 1057 1058 1059 1060 1061 1062 1063 1064 1065 1066 1067 1068 1069 1070 1071 1072 1073 1074 1075 1076 1077 1078 1079 1080 1081 1082 1083 1084 1085 1086 1087 1088 1089 1090 1091 1092 1093 1094 1095 1096 1097 1098 1099 1100 1101 1102 1103 1104 1105 1106 1107 1108 1109 1110 1111 1112 1113 1114 1115 1116 1117 1118 1119 1120 1121 1122 1123 1124 1125 1126 1127 1128 1129 1130 1131 1132 1133 1134 1135 1136 1137 1138 1139 1140 1141 1142 1143 1144 1145 1146 1147 1148 1149 1150 1151 1152 1153 1154 1155 1156 1157 1158 1159 1160 1161 1162 1163 1164 1165 1166 1167 1168 1169 1170 1171 1172 1173 1174 1175 1176 1177 1178 1179 1180 1181 1182 1183 1184 1185 1186 1187 1188 1189 1190 1191 1192 1193 1194 1195 1196 1197 1198 1199 1200 1201 1202 1203 1204 1205 1206 1207 1208 1209 1210 1211 1212 1213 1214 1215 1216 1217 1218 1219 1220 1221 1222 1223 1224 1225 1226 1227 1228 1229 1230 1231 1232 1233 1234 1235 1236 1237 1238 1239 1240 1241 1242 1243 1244 1245 1246 1247 1248 1249 1250 1251 1252 1253 1254 1255 1256 1257 1258 1259 1260 1261 1262 1263 1264 1265 1266 1267 1268 1269 1270 1271 1272 1273 1274 1275 1276 1277 1278 1279 1280 1281 1282 1283 1284 1285 1286 1287 1288 1289 1290 1291 1292 1293 1294 1295 1296 1297 1298 1299 1300 1301 1302 1303 1304 1305 1306 1307 1308 1309 1310 1311 1312 1313 1314 1315 1316 1317 1318 1319 1320 1321 1322 1323 1324 1325 1326 1327 1328 1329 1330 1331 1332 1333 1334 1335 1336 1337 1338 1339 1340 1341 1342 1343 1344 1345 1346 1347 1348 1349 1350 1351 1352 1353 1354 1355 1356 1357 1358 1359 1360 1361 1362 1363 1364 1365 1366 1367 1368 1369 1370 1371 1372 1373 1374 1375 1376 1377 1378 1379 1380 1381 1382 1383 1384 1385 1386 1387 1388 1389 1390 1391 1392 1393 1394 1395 1396 1397 1398 1399 1400 1401 1402 1403 1404 1405 1406 1407 1408 1409 1410 1411 1412 1413 1414 1415 1416 1417 1418 1419 1420 1421 1422 1423 1424 1425 1426 1427 1428 1429 1430 1431 1432 1433 1434 1435 1436 1437 1438 1439 1440 1441 1442 1443 1444 1445 1446 1447 1448 1449 1450 1451 1452 1453 1454 1455 1456 1457 1458 1459 1460 1461 1462 1463 1464 1465 1466 1467 1468 1469 1470 1471 1472 1473 1474 1475 1476 1477 1478 1479 1480 1481 1482 1483 1484 1485 1486 1487 1488 1489 1490 1491 1492 1493 1494 1495 1496 1497 1498 1499 1500 1501 1502 1503 | } else if (CPU_IS_HP_MC68K (cpu)) puts ("m68k-hitachi-hiuxwe2"); else puts ("unknown-hitachi-hiuxwe2"); exit (0); } EOF $CC_FOR_BUILD -o "$dummy" "$dummy.c" && SYSTEM_NAME=`"$dummy"` && { echo "$SYSTEM_NAME"; exit; } echo unknown-hitachi-hiuxwe2 exit ;; 9000/7??:4.3bsd:*:* | 9000/8?[79]:4.3bsd:*:*) echo hppa1.1-hp-bsd exit ;; 9000/8??:4.3bsd:*:*) echo hppa1.0-hp-bsd exit ;; *9??*:MPE/iX:*:* | *3000*:MPE/iX:*:*) echo hppa1.0-hp-mpeix exit ;; hp7??:OSF1:*:* | hp8?[79]:OSF1:*:*) echo hppa1.1-hp-osf exit ;; hp8??:OSF1:*:*) echo hppa1.0-hp-osf exit ;; i*86:OSF1:*:*) if [ -x /usr/sbin/sysversion ] ; then echo "$UNAME_MACHINE"-unknown-osf1mk else echo "$UNAME_MACHINE"-unknown-osf1 fi exit ;; parisc*:Lites*:*:*) echo hppa1.1-hp-lites exit ;; C1*:ConvexOS:*:* | convex:ConvexOS:C1*:*) echo c1-convex-bsd exit ;; C2*:ConvexOS:*:* | convex:ConvexOS:C2*:*) if getsysinfo -f scalar_acc then echo c32-convex-bsd else echo c2-convex-bsd fi exit ;; C34*:ConvexOS:*:* | convex:ConvexOS:C34*:*) echo c34-convex-bsd exit ;; C38*:ConvexOS:*:* | convex:ConvexOS:C38*:*) echo c38-convex-bsd exit ;; C4*:ConvexOS:*:* | convex:ConvexOS:C4*:*) echo c4-convex-bsd exit ;; CRAY*Y-MP:*:*:*) echo ymp-cray-unicos"$UNAME_RELEASE" | sed -e 's/\.[^.]*$/.X/' exit ;; CRAY*[A-Z]90:*:*:*) echo "$UNAME_MACHINE"-cray-unicos"$UNAME_RELEASE" \ | sed -e 's/CRAY.*\([A-Z]90\)/\1/' \ -e y/ABCDEFGHIJKLMNOPQRSTUVWXYZ/abcdefghijklmnopqrstuvwxyz/ \ -e 's/\.[^.]*$/.X/' exit ;; CRAY*TS:*:*:*) echo t90-cray-unicos"$UNAME_RELEASE" | sed -e 's/\.[^.]*$/.X/' exit ;; CRAY*T3E:*:*:*) echo alphaev5-cray-unicosmk"$UNAME_RELEASE" | sed -e 's/\.[^.]*$/.X/' exit ;; CRAY*SV1:*:*:*) echo sv1-cray-unicos"$UNAME_RELEASE" | sed -e 's/\.[^.]*$/.X/' exit ;; *:UNICOS/mp:*:*) echo craynv-cray-unicosmp"$UNAME_RELEASE" | sed -e 's/\.[^.]*$/.X/' exit ;; F30[01]:UNIX_System_V:*:* | F700:UNIX_System_V:*:*) FUJITSU_PROC=`uname -m | tr ABCDEFGHIJKLMNOPQRSTUVWXYZ abcdefghijklmnopqrstuvwxyz` FUJITSU_SYS=`uname -p | tr ABCDEFGHIJKLMNOPQRSTUVWXYZ abcdefghijklmnopqrstuvwxyz | sed -e 's/\///'` FUJITSU_REL=`echo "$UNAME_RELEASE" | sed -e 's/ /_/'` echo "${FUJITSU_PROC}-fujitsu-${FUJITSU_SYS}${FUJITSU_REL}" exit ;; 5000:UNIX_System_V:4.*:*) FUJITSU_SYS=`uname -p | tr ABCDEFGHIJKLMNOPQRSTUVWXYZ abcdefghijklmnopqrstuvwxyz | sed -e 's/\///'` FUJITSU_REL=`echo "$UNAME_RELEASE" | tr ABCDEFGHIJKLMNOPQRSTUVWXYZ abcdefghijklmnopqrstuvwxyz | sed -e 's/ /_/'` echo "sparc-fujitsu-${FUJITSU_SYS}${FUJITSU_REL}" exit ;; i*86:BSD/386:*:* | i*86:BSD/OS:*:* | *:Ascend\ Embedded/OS:*:*) echo "$UNAME_MACHINE"-pc-bsdi"$UNAME_RELEASE" exit ;; sparc*:BSD/OS:*:*) echo sparc-unknown-bsdi"$UNAME_RELEASE" exit ;; *:BSD/OS:*:*) echo "$UNAME_MACHINE"-unknown-bsdi"$UNAME_RELEASE" exit ;; arm:FreeBSD:*:*) UNAME_PROCESSOR=`uname -p` set_cc_for_build if echo __ARM_PCS_VFP | $CC_FOR_BUILD -E - 2>/dev/null \ | grep -q __ARM_PCS_VFP then echo "${UNAME_PROCESSOR}"-unknown-freebsd"`echo ${UNAME_RELEASE}|sed -e 's/[-(].*//'`"-gnueabi else echo "${UNAME_PROCESSOR}"-unknown-freebsd"`echo ${UNAME_RELEASE}|sed -e 's/[-(].*//'`"-gnueabihf fi exit ;; *:FreeBSD:*:*) UNAME_PROCESSOR=`/usr/bin/uname -p` case "$UNAME_PROCESSOR" in amd64) UNAME_PROCESSOR=x86_64 ;; i386) UNAME_PROCESSOR=i586 ;; esac echo "$UNAME_PROCESSOR"-unknown-freebsd"`echo "$UNAME_RELEASE"|sed -e 's/[-(].*//'`" exit ;; i*:CYGWIN*:*) echo "$UNAME_MACHINE"-pc-cygwin exit ;; *:MINGW64*:*) echo "$UNAME_MACHINE"-pc-mingw64 exit ;; *:MINGW*:*) echo "$UNAME_MACHINE"-pc-mingw32 exit ;; *:MSYS*:*) echo "$UNAME_MACHINE"-pc-msys exit ;; i*:PW*:*) echo "$UNAME_MACHINE"-pc-pw32 exit ;; *:Interix*:*) case "$UNAME_MACHINE" in x86) echo i586-pc-interix"$UNAME_RELEASE" exit ;; authenticamd | genuineintel | EM64T) echo x86_64-unknown-interix"$UNAME_RELEASE" exit ;; IA64) echo ia64-unknown-interix"$UNAME_RELEASE" exit ;; esac ;; i*:UWIN*:*) echo "$UNAME_MACHINE"-pc-uwin exit ;; amd64:CYGWIN*:*:* | x86_64:CYGWIN*:*:*) echo x86_64-pc-cygwin exit ;; prep*:SunOS:5.*:*) echo powerpcle-unknown-solaris2"`echo "$UNAME_RELEASE"|sed -e 's/[^.]*//'`" exit ;; *:GNU:*:*) # the GNU system echo "`echo "$UNAME_MACHINE"|sed -e 's,[-/].*$,,'`-unknown-$LIBC`echo "$UNAME_RELEASE"|sed -e 's,/.*$,,'`" exit ;; *:GNU/*:*:*) # other systems with GNU libc and userland echo "$UNAME_MACHINE-unknown-`echo "$UNAME_SYSTEM" | sed 's,^[^/]*/,,' | tr "[:upper:]" "[:lower:]"``echo "$UNAME_RELEASE"|sed -e 's/[-(].*//'`-$LIBC" exit ;; *:Minix:*:*) echo "$UNAME_MACHINE"-unknown-minix exit ;; aarch64:Linux:*:*) echo "$UNAME_MACHINE"-unknown-linux-"$LIBC" exit ;; aarch64_be:Linux:*:*) UNAME_MACHINE=aarch64_be echo "$UNAME_MACHINE"-unknown-linux-"$LIBC" exit ;; alpha:Linux:*:*) case `sed -n '/^cpu model/s/^.*: \(.*\)/\1/p' < /proc/cpuinfo` in EV5) UNAME_MACHINE=alphaev5 ;; EV56) UNAME_MACHINE=alphaev56 ;; PCA56) UNAME_MACHINE=alphapca56 ;; PCA57) UNAME_MACHINE=alphapca56 ;; EV6) UNAME_MACHINE=alphaev6 ;; EV67) UNAME_MACHINE=alphaev67 ;; EV68*) UNAME_MACHINE=alphaev68 ;; esac objdump --private-headers /bin/sh | grep -q ld.so.1 if test "$?" = 0 ; then LIBC=gnulibc1 ; fi echo "$UNAME_MACHINE"-unknown-linux-"$LIBC" exit ;; arc:Linux:*:* | arceb:Linux:*:*) echo "$UNAME_MACHINE"-unknown-linux-"$LIBC" exit ;; arm*:Linux:*:*) set_cc_for_build if echo __ARM_EABI__ | $CC_FOR_BUILD -E - 2>/dev/null \ | grep -q __ARM_EABI__ then echo "$UNAME_MACHINE"-unknown-linux-"$LIBC" else if echo __ARM_PCS_VFP | $CC_FOR_BUILD -E - 2>/dev/null \ | grep -q __ARM_PCS_VFP then echo "$UNAME_MACHINE"-unknown-linux-"$LIBC"eabi else echo "$UNAME_MACHINE"-unknown-linux-"$LIBC"eabihf fi fi exit ;; avr32*:Linux:*:*) echo "$UNAME_MACHINE"-unknown-linux-"$LIBC" exit ;; cris:Linux:*:*) echo "$UNAME_MACHINE"-axis-linux-"$LIBC" exit ;; crisv32:Linux:*:*) echo "$UNAME_MACHINE"-axis-linux-"$LIBC" exit ;; e2k:Linux:*:*) echo "$UNAME_MACHINE"-unknown-linux-"$LIBC" exit ;; frv:Linux:*:*) echo "$UNAME_MACHINE"-unknown-linux-"$LIBC" exit ;; hexagon:Linux:*:*) echo "$UNAME_MACHINE"-unknown-linux-"$LIBC" exit ;; i*86:Linux:*:*) echo "$UNAME_MACHINE"-pc-linux-"$LIBC" exit ;; ia64:Linux:*:*) echo "$UNAME_MACHINE"-unknown-linux-"$LIBC" exit ;; k1om:Linux:*:*) echo "$UNAME_MACHINE"-unknown-linux-"$LIBC" exit ;; m32r*:Linux:*:*) echo "$UNAME_MACHINE"-unknown-linux-"$LIBC" exit ;; m68*:Linux:*:*) echo "$UNAME_MACHINE"-unknown-linux-"$LIBC" exit ;; mips:Linux:*:* | mips64:Linux:*:*) set_cc_for_build IS_GLIBC=0 test x"${LIBC}" = xgnu && IS_GLIBC=1 sed 's/^ //' << EOF > "$dummy.c" #undef CPU #undef mips #undef mipsel #undef mips64 #undef mips64el #if ${IS_GLIBC} && defined(_ABI64) LIBCABI=gnuabi64 #else #if ${IS_GLIBC} && defined(_ABIN32) LIBCABI=gnuabin32 #else LIBCABI=${LIBC} #endif #endif #if ${IS_GLIBC} && defined(__mips64) && defined(__mips_isa_rev) && __mips_isa_rev>=6 CPU=mipsisa64r6 #else #if ${IS_GLIBC} && !defined(__mips64) && defined(__mips_isa_rev) && __mips_isa_rev>=6 CPU=mipsisa32r6 #else #if defined(__mips64) CPU=mips64 #else CPU=mips #endif #endif #endif #if defined(__MIPSEL__) || defined(__MIPSEL) || defined(_MIPSEL) || defined(MIPSEL) MIPS_ENDIAN=el #else #if defined(__MIPSEB__) || defined(__MIPSEB) || defined(_MIPSEB) || defined(MIPSEB) MIPS_ENDIAN= #else MIPS_ENDIAN= #endif #endif EOF eval "`$CC_FOR_BUILD -E "$dummy.c" 2>/dev/null | grep '^CPU\|^MIPS_ENDIAN\|^LIBCABI'`" test "x$CPU" != x && { echo "$CPU${MIPS_ENDIAN}-unknown-linux-$LIBCABI"; exit; } ;; mips64el:Linux:*:*) echo "$UNAME_MACHINE"-unknown-linux-"$LIBC" exit ;; openrisc*:Linux:*:*) echo or1k-unknown-linux-"$LIBC" exit ;; or32:Linux:*:* | or1k*:Linux:*:*) echo "$UNAME_MACHINE"-unknown-linux-"$LIBC" exit ;; padre:Linux:*:*) echo sparc-unknown-linux-"$LIBC" exit ;; parisc64:Linux:*:* | hppa64:Linux:*:*) echo hppa64-unknown-linux-"$LIBC" exit ;; parisc:Linux:*:* | hppa:Linux:*:*) # Look for CPU level case `grep '^cpu[^a-z]*:' /proc/cpuinfo 2>/dev/null | cut -d' ' -f2` in PA7*) echo hppa1.1-unknown-linux-"$LIBC" ;; PA8*) echo hppa2.0-unknown-linux-"$LIBC" ;; *) echo hppa-unknown-linux-"$LIBC" ;; esac exit ;; ppc64:Linux:*:*) echo powerpc64-unknown-linux-"$LIBC" exit ;; ppc:Linux:*:*) echo powerpc-unknown-linux-"$LIBC" exit ;; ppc64le:Linux:*:*) echo powerpc64le-unknown-linux-"$LIBC" exit ;; ppcle:Linux:*:*) echo powerpcle-unknown-linux-"$LIBC" exit ;; riscv32:Linux:*:* | riscv64:Linux:*:*) echo "$UNAME_MACHINE"-unknown-linux-"$LIBC" exit ;; s390:Linux:*:* | s390x:Linux:*:*) echo "$UNAME_MACHINE"-ibm-linux-"$LIBC" exit ;; sh64*:Linux:*:*) echo "$UNAME_MACHINE"-unknown-linux-"$LIBC" exit ;; sh*:Linux:*:*) echo "$UNAME_MACHINE"-unknown-linux-"$LIBC" exit ;; sparc:Linux:*:* | sparc64:Linux:*:*) echo "$UNAME_MACHINE"-unknown-linux-"$LIBC" exit ;; tile*:Linux:*:*) echo "$UNAME_MACHINE"-unknown-linux-"$LIBC" exit ;; vax:Linux:*:*) echo "$UNAME_MACHINE"-dec-linux-"$LIBC" exit ;; x86_64:Linux:*:*) echo "$UNAME_MACHINE"-pc-linux-"$LIBC" exit ;; xtensa*:Linux:*:*) echo "$UNAME_MACHINE"-unknown-linux-"$LIBC" exit ;; i*86:DYNIX/ptx:4*:*) # ptx 4.0 does uname -s correctly, with DYNIX/ptx in there. # earlier versions are messed up and put the nodename in both # sysname and nodename. echo i386-sequent-sysv4 exit ;; i*86:UNIX_SV:4.2MP:2.*) # Unixware is an offshoot of SVR4, but it has its own version # number series starting with 2... # I am not positive that other SVR4 systems won't match this, # I just have to hope. -- rms. # Use sysv4.2uw... so that sysv4* matches it. echo "$UNAME_MACHINE"-pc-sysv4.2uw"$UNAME_VERSION" exit ;; i*86:OS/2:*:*) # If we were able to find `uname', then EMX Unix compatibility # is probably installed. echo "$UNAME_MACHINE"-pc-os2-emx exit ;; i*86:XTS-300:*:STOP) echo "$UNAME_MACHINE"-unknown-stop exit ;; i*86:atheos:*:*) echo "$UNAME_MACHINE"-unknown-atheos exit ;; i*86:syllable:*:*) echo "$UNAME_MACHINE"-pc-syllable exit ;; i*86:LynxOS:2.*:* | i*86:LynxOS:3.[01]*:* | i*86:LynxOS:4.[02]*:*) echo i386-unknown-lynxos"$UNAME_RELEASE" exit ;; i*86:*DOS:*:*) echo "$UNAME_MACHINE"-pc-msdosdjgpp exit ;; i*86:*:4.*:*) UNAME_REL=`echo "$UNAME_RELEASE" | sed 's/\/MP$//'` if grep Novell /usr/include/link.h >/dev/null 2>/dev/null; then echo "$UNAME_MACHINE"-univel-sysv"$UNAME_REL" else echo "$UNAME_MACHINE"-pc-sysv"$UNAME_REL" fi exit ;; i*86:*:5:[678]*) # UnixWare 7.x, OpenUNIX and OpenServer 6. case `/bin/uname -X | grep "^Machine"` in *486*) UNAME_MACHINE=i486 ;; *Pentium) UNAME_MACHINE=i586 ;; *Pent*|*Celeron) UNAME_MACHINE=i686 ;; esac echo "$UNAME_MACHINE-unknown-sysv${UNAME_RELEASE}${UNAME_SYSTEM}${UNAME_VERSION}" exit ;; i*86:*:3.2:*) if test -f /usr/options/cb.name; then UNAME_REL=`sed -n 's/.*Version //p' </usr/options/cb.name` echo "$UNAME_MACHINE"-pc-isc"$UNAME_REL" elif /bin/uname -X 2>/dev/null >/dev/null ; then UNAME_REL=`(/bin/uname -X|grep Release|sed -e 's/.*= //')` (/bin/uname -X|grep i80486 >/dev/null) && UNAME_MACHINE=i486 (/bin/uname -X|grep '^Machine.*Pentium' >/dev/null) \ && UNAME_MACHINE=i586 (/bin/uname -X|grep '^Machine.*Pent *II' >/dev/null) \ && UNAME_MACHINE=i686 (/bin/uname -X|grep '^Machine.*Pentium Pro' >/dev/null) \ && UNAME_MACHINE=i686 echo "$UNAME_MACHINE"-pc-sco"$UNAME_REL" else echo "$UNAME_MACHINE"-pc-sysv32 fi exit ;; pc:*:*:*) # Left here for compatibility: # uname -m prints for DJGPP always 'pc', but it prints nothing about # the processor, so we play safe by assuming i586. # Note: whatever this is, it MUST be the same as what config.sub # prints for the "djgpp" host, or else GDB configure will decide that # this is a cross-build. echo i586-pc-msdosdjgpp exit ;; Intel:Mach:3*:*) echo i386-pc-mach3 exit ;; paragon:*:*:*) echo i860-intel-osf1 exit ;; i860:*:4.*:*) # i860-SVR4 if grep Stardent /usr/include/sys/uadmin.h >/dev/null 2>&1 ; then echo i860-stardent-sysv"$UNAME_RELEASE" # Stardent Vistra i860-SVR4 else # Add other i860-SVR4 vendors below as they are discovered. echo i860-unknown-sysv"$UNAME_RELEASE" # Unknown i860-SVR4 fi exit ;; mini*:CTIX:SYS*5:*) # "miniframe" echo m68010-convergent-sysv exit ;; mc68k:UNIX:SYSTEM5:3.51m) echo m68k-convergent-sysv exit ;; M680?0:D-NIX:5.3:*) echo m68k-diab-dnix exit ;; M68*:*:R3V[5678]*:*) test -r /sysV68 && { echo 'm68k-motorola-sysv'; exit; } ;; 3[345]??:*:4.0:3.0 | 3[34]??A:*:4.0:3.0 | 3[34]??,*:*:4.0:3.0 | 3[34]??/*:*:4.0:3.0 | 4400:*:4.0:3.0 | 4850:*:4.0:3.0 | SKA40:*:4.0:3.0 | SDS2:*:4.0:3.0 | SHG2:*:4.0:3.0 | S7501*:*:4.0:3.0) OS_REL='' test -r /etc/.relid \ && OS_REL=.`sed -n 's/[^ ]* [^ ]* \([0-9][0-9]\).*/\1/p' < /etc/.relid` /bin/uname -p 2>/dev/null | grep 86 >/dev/null \ && { echo i486-ncr-sysv4.3"$OS_REL"; exit; } /bin/uname -p 2>/dev/null | /bin/grep entium >/dev/null \ && { echo i586-ncr-sysv4.3"$OS_REL"; exit; } ;; 3[34]??:*:4.0:* | 3[34]??,*:*:4.0:*) /bin/uname -p 2>/dev/null | grep 86 >/dev/null \ && { echo i486-ncr-sysv4; exit; } ;; NCR*:*:4.2:* | MPRAS*:*:4.2:*) OS_REL='.3' test -r /etc/.relid \ && OS_REL=.`sed -n 's/[^ ]* [^ ]* \([0-9][0-9]\).*/\1/p' < /etc/.relid` /bin/uname -p 2>/dev/null | grep 86 >/dev/null \ && { echo i486-ncr-sysv4.3"$OS_REL"; exit; } /bin/uname -p 2>/dev/null | /bin/grep entium >/dev/null \ && { echo i586-ncr-sysv4.3"$OS_REL"; exit; } /bin/uname -p 2>/dev/null | /bin/grep pteron >/dev/null \ && { echo i586-ncr-sysv4.3"$OS_REL"; exit; } ;; m68*:LynxOS:2.*:* | m68*:LynxOS:3.0*:*) echo m68k-unknown-lynxos"$UNAME_RELEASE" exit ;; mc68030:UNIX_System_V:4.*:*) echo m68k-atari-sysv4 exit ;; TSUNAMI:LynxOS:2.*:*) echo sparc-unknown-lynxos"$UNAME_RELEASE" exit ;; rs6000:LynxOS:2.*:*) echo rs6000-unknown-lynxos"$UNAME_RELEASE" exit ;; PowerPC:LynxOS:2.*:* | PowerPC:LynxOS:3.[01]*:* | PowerPC:LynxOS:4.[02]*:*) echo powerpc-unknown-lynxos"$UNAME_RELEASE" exit ;; SM[BE]S:UNIX_SV:*:*) echo mips-dde-sysv"$UNAME_RELEASE" exit ;; RM*:ReliantUNIX-*:*:*) echo mips-sni-sysv4 exit ;; RM*:SINIX-*:*:*) echo mips-sni-sysv4 exit ;; *:SINIX-*:*:*) if uname -p 2>/dev/null >/dev/null ; then UNAME_MACHINE=`(uname -p) 2>/dev/null` echo "$UNAME_MACHINE"-sni-sysv4 else echo ns32k-sni-sysv fi exit ;; PENTIUM:*:4.0*:*) # Unisys `ClearPath HMP IX 4000' SVR4/MP effort # says <Richard.M.Bartel@ccMail.Census.GOV> echo i586-unisys-sysv4 exit ;; *:UNIX_System_V:4*:FTX*) # From Gerald Hewes <hewes@openmarket.com>. # How about differentiating between stratus architectures? -djm echo hppa1.1-stratus-sysv4 exit ;; *:*:*:FTX*) # From seanf@swdc.stratus.com. echo i860-stratus-sysv4 exit ;; i*86:VOS:*:*) # From Paul.Green@stratus.com. echo "$UNAME_MACHINE"-stratus-vos exit ;; *:VOS:*:*) # From Paul.Green@stratus.com. echo hppa1.1-stratus-vos exit ;; mc68*:A/UX:*:*) echo m68k-apple-aux"$UNAME_RELEASE" exit ;; news*:NEWS-OS:6*:*) echo mips-sony-newsos6 exit ;; R[34]000:*System_V*:*:* | R4000:UNIX_SYSV:*:* | R*000:UNIX_SV:*:*) if [ -d /usr/nec ]; then echo mips-nec-sysv"$UNAME_RELEASE" else echo mips-unknown-sysv"$UNAME_RELEASE" fi exit ;; BeBox:BeOS:*:*) # BeOS running on hardware made by Be, PPC only. echo powerpc-be-beos exit ;; BeMac:BeOS:*:*) # BeOS running on Mac or Mac clone, PPC only. echo powerpc-apple-beos exit ;; BePC:BeOS:*:*) # BeOS running on Intel PC compatible. echo i586-pc-beos exit ;; BePC:Haiku:*:*) # Haiku running on Intel PC compatible. echo i586-pc-haiku exit ;; x86_64:Haiku:*:*) echo x86_64-unknown-haiku exit ;; SX-4:SUPER-UX:*:*) echo sx4-nec-superux"$UNAME_RELEASE" exit ;; SX-5:SUPER-UX:*:*) echo sx5-nec-superux"$UNAME_RELEASE" exit ;; SX-6:SUPER-UX:*:*) echo sx6-nec-superux"$UNAME_RELEASE" exit ;; SX-7:SUPER-UX:*:*) echo sx7-nec-superux"$UNAME_RELEASE" exit ;; SX-8:SUPER-UX:*:*) echo sx8-nec-superux"$UNAME_RELEASE" exit ;; SX-8R:SUPER-UX:*:*) echo sx8r-nec-superux"$UNAME_RELEASE" exit ;; SX-ACE:SUPER-UX:*:*) echo sxace-nec-superux"$UNAME_RELEASE" exit ;; Power*:Rhapsody:*:*) echo powerpc-apple-rhapsody"$UNAME_RELEASE" exit ;; *:Rhapsody:*:*) echo "$UNAME_MACHINE"-apple-rhapsody"$UNAME_RELEASE" exit ;; *:Darwin:*:*) UNAME_PROCESSOR=`uname -p` case $UNAME_PROCESSOR in unknown) UNAME_PROCESSOR=powerpc ;; esac if command -v xcode-select > /dev/null 2> /dev/null && \ ! xcode-select --print-path > /dev/null 2> /dev/null ; then # Avoid executing cc if there is no toolchain installed as # cc will be a stub that puts up a graphical alert # prompting the user to install developer tools. CC_FOR_BUILD=no_compiler_found else set_cc_for_build fi if [ "$CC_FOR_BUILD" != no_compiler_found ]; then if (echo '#ifdef __LP64__'; echo IS_64BIT_ARCH; echo '#endif') | \ (CCOPTS="" $CC_FOR_BUILD -E - 2>/dev/null) | \ grep IS_64BIT_ARCH >/dev/null then case $UNAME_PROCESSOR in i386) UNAME_PROCESSOR=x86_64 ;; powerpc) UNAME_PROCESSOR=powerpc64 ;; esac fi # On 10.4-10.6 one might compile for PowerPC via gcc -arch ppc if (echo '#ifdef __POWERPC__'; echo IS_PPC; echo '#endif') | \ (CCOPTS="" $CC_FOR_BUILD -E - 2>/dev/null) | \ grep IS_PPC >/dev/null then UNAME_PROCESSOR=powerpc fi elif test "$UNAME_PROCESSOR" = i386 ; then # uname -m returns i386 or x86_64 UNAME_PROCESSOR=$UNAME_MACHINE fi echo "$UNAME_PROCESSOR"-apple-darwin"$UNAME_RELEASE" exit ;; *:procnto*:*:* | *:QNX:[0123456789]*:*) UNAME_PROCESSOR=`uname -p` if test "$UNAME_PROCESSOR" = x86; then UNAME_PROCESSOR=i386 UNAME_MACHINE=pc fi echo "$UNAME_PROCESSOR"-"$UNAME_MACHINE"-nto-qnx"$UNAME_RELEASE" exit ;; *:QNX:*:4*) echo i386-pc-qnx exit ;; NEO-*:NONSTOP_KERNEL:*:*) echo neo-tandem-nsk"$UNAME_RELEASE" exit ;; NSE-*:NONSTOP_KERNEL:*:*) echo nse-tandem-nsk"$UNAME_RELEASE" exit ;; NSR-*:NONSTOP_KERNEL:*:*) echo nsr-tandem-nsk"$UNAME_RELEASE" exit ;; NSV-*:NONSTOP_KERNEL:*:*) echo nsv-tandem-nsk"$UNAME_RELEASE" exit ;; NSX-*:NONSTOP_KERNEL:*:*) echo nsx-tandem-nsk"$UNAME_RELEASE" exit ;; *:NonStop-UX:*:*) echo mips-compaq-nonstopux exit ;; BS2000:POSIX*:*:*) echo bs2000-siemens-sysv exit ;; DS/*:UNIX_System_V:*:*) echo "$UNAME_MACHINE"-"$UNAME_SYSTEM"-"$UNAME_RELEASE" exit ;; *:Plan9:*:*) # "uname -m" is not consistent, so use $cputype instead. 386 # is converted to i386 for consistency with other x86 # operating systems. # shellcheck disable=SC2154 if test "$cputype" = 386; then UNAME_MACHINE=i386 else UNAME_MACHINE="$cputype" fi echo "$UNAME_MACHINE"-unknown-plan9 exit ;; *:TOPS-10:*:*) echo pdp10-unknown-tops10 exit ;; *:TENEX:*:*) echo pdp10-unknown-tenex exit ;; KS10:TOPS-20:*:* | KL10:TOPS-20:*:* | TYPE4:TOPS-20:*:*) echo pdp10-dec-tops20 exit ;; XKL-1:TOPS-20:*:* | TYPE5:TOPS-20:*:*) echo pdp10-xkl-tops20 exit ;; *:TOPS-20:*:*) echo pdp10-unknown-tops20 exit ;; *:ITS:*:*) echo pdp10-unknown-its exit ;; SEI:*:*:SEIUX) echo mips-sei-seiux"$UNAME_RELEASE" exit ;; *:DragonFly:*:*) echo "$UNAME_MACHINE"-unknown-dragonfly"`echo "$UNAME_RELEASE"|sed -e 's/[-(].*//'`" exit ;; *:*VMS:*:*) UNAME_MACHINE=`(uname -p) 2>/dev/null` case "$UNAME_MACHINE" in A*) echo alpha-dec-vms ; exit ;; I*) echo ia64-dec-vms ; exit ;; V*) echo vax-dec-vms ; exit ;; esac ;; *:XENIX:*:SysV) echo i386-pc-xenix exit ;; i*86:skyos:*:*) echo "$UNAME_MACHINE"-pc-skyos"`echo "$UNAME_RELEASE" | sed -e 's/ .*$//'`" exit ;; i*86:rdos:*:*) echo "$UNAME_MACHINE"-pc-rdos exit ;; i*86:AROS:*:*) echo "$UNAME_MACHINE"-pc-aros exit ;; x86_64:VMkernel:*:*) echo "$UNAME_MACHINE"-unknown-esx exit ;; amd64:Isilon\ OneFS:*:*) echo x86_64-unknown-onefs exit ;; *:Unleashed:*:*) echo "$UNAME_MACHINE"-unknown-unleashed"$UNAME_RELEASE" exit ;; esac # No uname command or uname output not recognized. set_cc_for_build cat > "$dummy.c" <<EOF #ifdef _SEQUENT_ #include <sys/types.h> #include <sys/utsname.h> #endif #if defined(ultrix) || defined(_ultrix) || defined(__ultrix) || defined(__ultrix__) #if defined (vax) || defined (__vax) || defined (__vax__) || defined(mips) || defined(__mips) || defined(__mips__) || defined(MIPS) || defined(__MIPS__) #include <signal.h> #if defined(_SIZE_T_) || defined(SIGLOST) #include <sys/utsname.h> #endif #endif #endif main () { #if defined (sony) #if defined (MIPSEB) /* BFD wants "bsd" instead of "newsos". Perhaps BFD should be changed, I don't know.... */ printf ("mips-sony-bsd\n"); exit (0); #else #include <sys/param.h> printf ("m68k-sony-newsos%s\n", #ifdef NEWSOS4 "4" #else "" #endif ); exit (0); #endif #endif #if defined (NeXT) #if !defined (__ARCHITECTURE__) #define __ARCHITECTURE__ "m68k" #endif int version; |
︙ | ︙ | |||
1406 1407 1408 1409 1410 1411 1412 | #endif #if defined (ns32000) printf ("ns32k-sequent-dynix\n"); exit (0); #endif #endif #if defined (_SEQUENT_) | | | < | | | | | | | < | | | | | | | | | | | | | | | | > > > > > | | > > > > > > > > > > > > | > | < > > > > < > | < < < > | < < < < < < < < < < < < < < < | < < | | | | | < | | | | | | | | | 1531 1532 1533 1534 1535 1536 1537 1538 1539 1540 1541 1542 1543 1544 1545 1546 1547 1548 1549 1550 1551 1552 1553 1554 1555 1556 1557 1558 1559 1560 1561 1562 1563 1564 1565 1566 1567 1568 1569 1570 1571 1572 1573 1574 1575 1576 1577 1578 1579 1580 1581 1582 1583 1584 1585 1586 1587 1588 1589 1590 1591 1592 1593 1594 1595 1596 1597 1598 1599 1600 1601 1602 1603 1604 1605 1606 1607 1608 1609 1610 1611 1612 1613 1614 1615 1616 1617 1618 1619 1620 1621 1622 1623 1624 1625 1626 1627 1628 1629 1630 1631 1632 1633 1634 1635 1636 1637 1638 1639 1640 1641 1642 1643 1644 1645 1646 1647 1648 1649 1650 1651 1652 1653 1654 1655 1656 1657 1658 1659 | #endif #if defined (ns32000) printf ("ns32k-sequent-dynix\n"); exit (0); #endif #endif #if defined (_SEQUENT_) struct utsname un; uname(&un); if (strncmp(un.version, "V2", 2) == 0) { printf ("i386-sequent-ptx2\n"); exit (0); } if (strncmp(un.version, "V1", 2) == 0) { /* XXX is V1 correct? */ printf ("i386-sequent-ptx1\n"); exit (0); } printf ("i386-sequent-ptx\n"); exit (0); #endif #if defined (vax) #if !defined (ultrix) #include <sys/param.h> #if defined (BSD) #if BSD == 43 printf ("vax-dec-bsd4.3\n"); exit (0); #else #if BSD == 199006 printf ("vax-dec-bsd4.3reno\n"); exit (0); #else printf ("vax-dec-bsd\n"); exit (0); #endif #endif #else printf ("vax-dec-bsd\n"); exit (0); #endif #else #if defined(_SIZE_T_) || defined(SIGLOST) struct utsname un; uname (&un); printf ("vax-dec-ultrix%s\n", un.release); exit (0); #else printf ("vax-dec-ultrix\n"); exit (0); #endif #endif #endif #if defined(ultrix) || defined(_ultrix) || defined(__ultrix) || defined(__ultrix__) #if defined(mips) || defined(__mips) || defined(__mips__) || defined(MIPS) || defined(__MIPS__) #if defined(_SIZE_T_) || defined(SIGLOST) struct utsname *un; uname (&un); printf ("mips-dec-ultrix%s\n", un.release); exit (0); #else printf ("mips-dec-ultrix\n"); exit (0); #endif #endif #endif #if defined (alliant) && defined (i860) printf ("i860-alliant-bsd\n"); exit (0); #endif exit (1); } EOF $CC_FOR_BUILD -o "$dummy" "$dummy.c" 2>/dev/null && SYSTEM_NAME=`$dummy` && { echo "$SYSTEM_NAME"; exit; } # Apollos put the system type in the environment. test -d /usr/apollo && { echo "$ISP-apollo-$SYSTYPE"; exit; } echo "$0: unable to guess system type" >&2 case "$UNAME_MACHINE:$UNAME_SYSTEM" in mips:Linux | mips64:Linux) # If we got here on MIPS GNU/Linux, output extra information. cat >&2 <<EOF NOTE: MIPS GNU/Linux systems require a C compiler to fully recognize the system type. Please install a C compiler and try again. EOF ;; esac cat >&2 <<EOF This script (version $timestamp), has failed to recognize the operating system you are using. If your script is old, overwrite *all* copies of config.guess and config.sub with the latest versions from: https://git.savannah.gnu.org/gitweb/?p=config.git;a=blob_plain;f=config.guess and https://git.savannah.gnu.org/gitweb/?p=config.git;a=blob_plain;f=config.sub If $0 has already been updated, send the following data and any information you think might be pertinent to config-patches@gnu.org to provide the necessary information to handle your system. config.guess timestamp = $timestamp uname -m = `(uname -m) 2>/dev/null || echo unknown` uname -r = `(uname -r) 2>/dev/null || echo unknown` uname -s = `(uname -s) 2>/dev/null || echo unknown` uname -v = `(uname -v) 2>/dev/null || echo unknown` /usr/bin/uname -p = `(/usr/bin/uname -p) 2>/dev/null` /bin/uname -X = `(/bin/uname -X) 2>/dev/null` hostinfo = `(hostinfo) 2>/dev/null` /bin/universe = `(/bin/universe) 2>/dev/null` /usr/bin/arch -k = `(/usr/bin/arch -k) 2>/dev/null` /bin/arch = `(/bin/arch) 2>/dev/null` /usr/bin/oslevel = `(/usr/bin/oslevel) 2>/dev/null` /usr/convex/getsysinfo = `(/usr/convex/getsysinfo) 2>/dev/null` UNAME_MACHINE = "$UNAME_MACHINE" UNAME_RELEASE = "$UNAME_RELEASE" UNAME_SYSTEM = "$UNAME_SYSTEM" UNAME_VERSION = "$UNAME_VERSION" EOF exit 1 # Local variables: # eval: (add-hook 'before-save-hook 'time-stamp) # time-stamp-start: "timestamp='" # time-stamp-format: "%:y-%02m-%02d" # time-stamp-end: "'" # End: |
Deleted config.h.in.
|
| < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < |
Changes to config.sub.
1 2 | #! /bin/sh # Configuration validation subroutine script. | < | < | < < < < | | | | | | | | < < | > > | < > > > | < | < | | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 | #! /bin/sh # Configuration validation subroutine script. # Copyright 1992-2019 Free Software Foundation, Inc. timestamp='2019-05-23' # This file is free software; you can redistribute it and/or modify it # under the terms of the GNU General Public License as published by # the Free Software Foundation; either version 3 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, but # WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU # General Public License for more details. # # You should have received a copy of the GNU General Public License # along with this program; if not, see <https://www.gnu.org/licenses/>. # # As a special exception to the GNU General Public License, if you # distribute this file as part of a program that contains a # configuration script generated by Autoconf, you may include it under # the same distribution terms that you use for the rest of that # program. This Exception is an additional permission under section 7 # of the GNU General Public License, version 3 ("GPLv3"). # Please send patches to <config-patches@gnu.org>. # # Configuration subroutine to validate and canonicalize a configuration type. # Supply the specified configuration type as an argument. # If it is invalid, we print an error message on stderr and exit with code 1. # Otherwise, we print the canonical config type on stdout and succeed. # You can get the latest version of this script from: # https://git.savannah.gnu.org/gitweb/?p=config.git;a=blob_plain;f=config.sub # This file is supposed to be the same for all GNU packages # and recognize all the CPU types, system types and aliases # that are meaningful with *any* GNU software. # Each package is responsible for reporting which valid configurations # it does not support. The user should be able to distinguish # a failure to support a valid configuration from a meaningless # configuration. # The goal of this file is to map all the various variations of a given # machine specification into a single specification in the form: # CPU_TYPE-MANUFACTURER-OPERATING_SYSTEM # or in some cases, the newer four-part form: # CPU_TYPE-MANUFACTURER-KERNEL-OPERATING_SYSTEM # It is wrong to echo any other type of specification. me=`echo "$0" | sed -e 's,.*/,,'` usage="\ Usage: $0 [OPTION] CPU-MFR-OPSYS or ALIAS Canonicalize a configuration name. Options: -h, --help print this help, then exit -t, --time-stamp print date of last modification, then exit -v, --version print version number, then exit Report bugs and patches to <config-patches@gnu.org>." version="\ GNU config.sub ($timestamp) Copyright 1992-2019 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE." help=" Try \`$me --help' for more information." |
︙ | ︙ | |||
91 92 93 94 95 96 97 | --help | --h* | -h ) echo "$usage"; exit ;; -- ) # Stop option processing shift; break ;; - ) # Use stdin as input. break ;; -* ) | | | > > > > > | > > > > > > > > > > > | > | | | > | > | > | > > | > | | > > > > > > > > > | > > > > > > > | | > > > > > > > > > > > > | | > > > > | > > > > > > > > > > > > > > > > > > > > > > > > > > > | | | < > > > | > > > | > > > > > > > | > | > > | < < | < < < | | > > > > > > | > | > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | > | > > | > | > > | | > > > > | > | > > | > > > > > | > > | > > > | > > | > | > > | > > > > > > > | > > | > | > > > > > > > | > > > > > | > > > > > > > > > > > > > > > > > > > > > > > > | > > > > > > > > > > > > > > > > > > | > > > | > > > > > > | > | | > | > > | > | | > | > > | > | > > > > > > > > > | | > > > > > > | > | > > > > | < > > > > | > | > > > > > > > > > > > > > > > > > > > > > > > > | < > > > > > > > > > > > > | > | > > | > > > | > > | > | > > > > > > > > > > > > > > > > | | > | > > > > > > > > > > | > > > > > > > > > > > | > > | > | > > > > | | > | > > | > > | > | | > | | | > > > > | > | > | > > > > | | > > | > > > | | > > > > > > > > > > > > > > > > > > > > > > > > > > | > > | > > > > | | | > > | | | > > > > | > > > > > > > > > > > > > > > | < < > > > | > > > | > > > > > > | > > > > | > > > | > | < < > > | > > > > | > > > > > > > > > | < > > > > | | | < > | | | < > | | > > | > > > > | > > > | > > > > > > > > > > > > > > > | > > > > > > > > > > > > > | > > > > > > > > > > > > > > > > > > > > > > > | > > > > | | > | > > > > > | > > > > > > > | > > | < | > > > > > | < | | < > > > > > > > > > > > > | < > > | > > > > > > > | < > > > > > > | < > | > | > > > > > | > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | > > < > > > > | | > > > > > > | > | < < > | > | > > > > > | | < < < < < < | < < < > > | < < < < < < | | | | | | | | | | < > | | | < > | | < < > | < < < | < < > | > > > > > | < < < < < < < | > | < < | < < < < < | | < > | > > > | | < < | > < | > > | | > | < | < < < < < < < | | < < > | < < < < | < < < < < < < < < < < < < < < < < < | < | < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < | < < < < < < < < < < < < < < < < < < < < < < | < | < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < | < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < | < < | < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < | < < | < < < | < < < < < < | < < < < < < < < < < < < < | | | | > > > > | < > > > | > > > > > > > | > > > > > | < > | < > | > < < | > > > > | | < < > | < | < > > | > | < | < | | | | | | | > | < > > > | > > > | < > | | > | < | < < > | < | | > | | > | < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < | < < < < < < < < < < < < < | < < < < < < < < < < < < < < < < < < < < < < < < | < < < < < < | < < < < < < < < < < < < < < < < < < < < < < < < < < < < | < < < < < < < < < < < < < < < < < < < < < < < < < < < < | < < < | < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < | < < < < < < < < < < < < < < < | < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < | < < < < < < < < < < < < < < | < < < < < < < < < < | < | | < | < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | | | | > > | | < > | | | | | | > > > > > > | | | < < < | | | > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | | | | | > | | | | | | | | | | | | | > | | | | | | | | | > > | | | > > > > > > > > > | > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > < < < < < < < < < | < < < < < < < < < < < < < < < < < < < < < < < < < | < < < < < < < < < < < < < < < < < < < < < < | < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < | < < < < < < < < < < < < < < < < < < < < | | | | | | | | | | | > > > > > > > > > > > > > > > > > > | | | | | | < < < | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | < < < | | | | | | | | | | | > > > | | < | | | | | | | | | | | | > > > | | | | | | | | | | < | | | 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340 341 342 343 344 345 346 347 348 349 350 351 352 353 354 355 356 357 358 359 360 361 362 363 364 365 366 367 368 369 370 371 372 373 374 375 376 377 378 379 380 381 382 383 384 385 386 387 388 389 390 391 392 393 394 395 396 397 398 399 400 401 402 403 404 405 406 407 408 409 410 411 412 413 414 415 416 417 418 419 420 421 422 423 424 425 426 427 428 429 430 431 432 433 434 435 436 437 438 439 440 441 442 443 444 445 446 447 448 449 450 451 452 453 454 455 456 457 458 459 460 461 462 463 464 465 466 467 468 469 470 471 472 473 474 475 476 477 478 479 480 481 482 483 484 485 486 487 488 489 490 491 492 493 494 495 496 497 498 499 500 501 502 503 504 505 506 507 508 509 510 511 512 513 514 515 516 517 518 519 520 521 522 523 524 525 526 527 528 529 530 531 532 533 534 535 536 537 538 539 540 541 542 543 544 545 546 547 548 549 550 551 552 553 554 555 556 557 558 559 560 561 562 563 564 565 566 567 568 569 570 571 572 573 574 575 576 577 578 579 580 581 582 583 584 585 586 587 588 589 590 591 592 593 594 595 596 597 598 599 600 601 602 603 604 605 606 607 608 609 610 611 612 613 614 615 616 617 618 619 620 621 622 623 624 625 626 627 628 629 630 631 632 633 634 635 636 637 638 639 640 641 642 643 644 645 646 647 648 649 650 651 652 653 654 655 656 657 658 659 660 661 662 663 664 665 666 667 668 669 670 671 672 673 674 675 676 677 678 679 680 681 682 683 684 685 686 687 688 689 690 691 692 693 694 695 696 697 698 699 700 701 702 703 704 705 706 707 708 709 710 711 712 713 714 715 716 717 718 719 720 721 722 723 724 725 726 727 728 729 730 731 732 733 734 735 736 737 738 739 740 741 742 743 744 745 746 747 748 749 750 751 752 753 754 755 756 757 758 759 760 761 762 763 764 765 766 767 768 769 770 771 772 773 774 775 776 777 778 779 780 781 782 783 784 785 786 787 788 789 790 791 792 793 794 795 796 797 798 799 800 801 802 803 804 805 806 807 808 809 810 811 812 813 814 815 816 817 818 819 820 821 822 823 824 825 826 827 828 829 830 831 832 833 834 835 836 837 838 839 840 841 842 843 844 845 846 847 848 849 850 851 852 853 854 855 856 857 858 859 860 861 862 863 864 865 866 867 868 869 870 871 872 873 874 875 876 877 878 879 880 881 882 883 884 885 886 887 888 889 890 891 892 893 894 895 896 897 898 899 900 901 902 903 904 905 906 907 908 909 910 911 912 913 914 915 916 917 918 919 920 921 922 923 924 925 926 927 928 929 930 931 932 933 934 935 936 937 938 939 940 941 942 943 944 945 946 947 948 949 950 951 952 953 954 955 956 957 958 959 960 961 962 963 964 965 966 967 968 969 970 971 972 973 974 975 976 977 978 979 980 981 982 983 984 985 986 987 988 989 990 991 992 993 994 995 996 997 998 999 1000 1001 1002 1003 1004 1005 1006 1007 1008 1009 1010 1011 1012 1013 1014 1015 1016 1017 1018 1019 1020 1021 1022 1023 1024 1025 1026 1027 1028 1029 1030 1031 1032 1033 1034 1035 1036 1037 1038 1039 1040 1041 1042 1043 1044 1045 1046 1047 1048 1049 1050 1051 1052 1053 1054 1055 1056 1057 1058 1059 1060 1061 1062 1063 1064 1065 1066 1067 1068 1069 1070 1071 1072 1073 1074 1075 1076 1077 1078 1079 1080 1081 1082 1083 1084 1085 1086 1087 1088 1089 1090 1091 1092 1093 1094 1095 1096 1097 1098 1099 1100 1101 1102 1103 1104 1105 1106 1107 1108 1109 1110 1111 1112 1113 1114 1115 1116 1117 1118 1119 1120 1121 1122 1123 1124 1125 1126 1127 1128 1129 1130 1131 1132 1133 1134 1135 1136 1137 1138 1139 1140 1141 1142 1143 1144 1145 1146 1147 1148 1149 1150 1151 1152 1153 1154 1155 1156 1157 1158 1159 1160 1161 1162 1163 1164 1165 1166 1167 1168 1169 1170 1171 1172 1173 1174 1175 1176 1177 1178 1179 1180 1181 1182 1183 1184 1185 1186 1187 1188 1189 1190 1191 1192 1193 1194 1195 1196 1197 1198 1199 1200 1201 1202 1203 1204 1205 1206 1207 1208 1209 1210 1211 1212 1213 1214 1215 1216 1217 1218 1219 1220 1221 1222 1223 1224 1225 1226 1227 1228 1229 1230 1231 1232 1233 1234 1235 1236 1237 1238 1239 1240 1241 1242 1243 1244 1245 1246 1247 1248 1249 1250 1251 1252 1253 1254 1255 1256 1257 1258 1259 1260 1261 1262 1263 1264 1265 1266 1267 1268 1269 1270 1271 1272 1273 1274 1275 1276 1277 1278 1279 1280 1281 1282 1283 1284 1285 1286 1287 1288 1289 1290 1291 1292 1293 1294 1295 1296 1297 1298 1299 1300 1301 1302 1303 1304 1305 1306 1307 1308 1309 1310 1311 1312 1313 1314 1315 1316 1317 1318 1319 1320 1321 1322 1323 1324 1325 1326 1327 1328 1329 1330 1331 1332 1333 1334 1335 1336 1337 1338 1339 1340 1341 1342 1343 1344 1345 1346 1347 1348 1349 1350 1351 1352 1353 1354 1355 1356 1357 1358 1359 1360 1361 1362 1363 1364 1365 1366 1367 1368 1369 1370 1371 1372 1373 1374 1375 1376 1377 1378 1379 1380 1381 1382 1383 1384 1385 1386 1387 1388 1389 1390 1391 1392 1393 1394 1395 1396 1397 1398 1399 1400 1401 1402 1403 1404 1405 1406 1407 1408 1409 1410 1411 1412 1413 1414 1415 1416 1417 1418 1419 1420 1421 1422 1423 1424 1425 1426 1427 1428 1429 1430 1431 1432 1433 1434 1435 1436 1437 1438 1439 1440 1441 1442 1443 1444 1445 1446 1447 1448 1449 1450 1451 1452 1453 1454 1455 1456 1457 1458 1459 1460 1461 1462 1463 1464 1465 1466 1467 1468 1469 1470 1471 1472 1473 1474 1475 1476 1477 1478 1479 1480 1481 1482 1483 1484 1485 1486 1487 1488 1489 1490 1491 1492 1493 1494 1495 1496 1497 1498 1499 1500 1501 1502 1503 1504 1505 1506 1507 1508 1509 1510 1511 1512 1513 1514 1515 1516 1517 1518 1519 1520 1521 1522 1523 1524 1525 1526 1527 1528 1529 1530 1531 1532 1533 1534 1535 1536 1537 1538 1539 1540 1541 1542 1543 1544 1545 1546 1547 1548 1549 1550 1551 1552 1553 1554 1555 1556 1557 1558 1559 1560 1561 1562 1563 1564 1565 1566 1567 1568 1569 1570 1571 1572 1573 1574 1575 1576 1577 1578 1579 1580 1581 1582 1583 1584 1585 1586 1587 1588 1589 1590 1591 1592 1593 1594 1595 1596 1597 1598 1599 1600 1601 1602 1603 1604 1605 1606 1607 1608 1609 1610 1611 1612 1613 1614 1615 1616 1617 1618 1619 1620 1621 1622 1623 1624 1625 1626 1627 1628 1629 1630 1631 1632 1633 1634 1635 1636 1637 1638 1639 1640 1641 1642 1643 1644 1645 1646 1647 1648 1649 1650 1651 1652 1653 1654 1655 1656 1657 1658 1659 1660 1661 1662 1663 1664 1665 1666 1667 1668 1669 1670 1671 1672 1673 1674 1675 1676 1677 1678 1679 1680 1681 1682 1683 1684 1685 1686 1687 1688 1689 1690 1691 1692 1693 1694 1695 1696 1697 1698 1699 1700 1701 1702 1703 1704 1705 1706 1707 1708 1709 1710 1711 1712 1713 1714 1715 1716 1717 1718 1719 1720 1721 1722 1723 1724 1725 1726 1727 1728 1729 1730 1731 1732 1733 1734 1735 1736 1737 1738 1739 1740 1741 1742 1743 1744 1745 1746 1747 1748 1749 1750 1751 1752 1753 1754 1755 1756 1757 1758 1759 1760 1761 1762 1763 1764 1765 1766 1767 1768 1769 1770 1771 1772 1773 1774 1775 1776 1777 1778 1779 1780 1781 1782 1783 1784 1785 1786 1787 1788 1789 1790 1791 1792 1793 1794 1795 1796 1797 1798 | --help | --h* | -h ) echo "$usage"; exit ;; -- ) # Stop option processing shift; break ;; - ) # Use stdin as input. break ;; -* ) echo "$me: invalid option $1$help" >&2 exit 1 ;; *local*) # First pass through any local machine types. echo "$1" exit ;; * ) break ;; esac done case $# in 0) echo "$me: missing argument$help" >&2 exit 1;; 1) ;; *) echo "$me: too many arguments$help" >&2 exit 1;; esac # Split fields of configuration type # shellcheck disable=SC2162 IFS="-" read field1 field2 field3 field4 <<EOF $1 EOF # Separate into logical components for further validation case $1 in *-*-*-*-*) echo Invalid configuration \`"$1"\': more than four components >&2 exit 1 ;; *-*-*-*) basic_machine=$field1-$field2 os=$field3-$field4 ;; *-*-*) # Ambiguous whether COMPANY is present, or skipped and KERNEL-OS is two # parts maybe_os=$field2-$field3 case $maybe_os in nto-qnx* | linux-gnu* | linux-android* | linux-dietlibc \ | linux-newlib* | linux-musl* | linux-uclibc* | uclinux-uclibc* \ | uclinux-gnu* | kfreebsd*-gnu* | knetbsd*-gnu* | netbsd*-gnu* \ | netbsd*-eabi* | kopensolaris*-gnu* | cloudabi*-eabi* \ | storm-chaos* | os2-emx* | rtmk-nova*) basic_machine=$field1 os=$maybe_os ;; android-linux) basic_machine=$field1-unknown os=linux-android ;; *) basic_machine=$field1-$field2 os=$field3 ;; esac ;; *-*) # A lone config we happen to match not fitting any pattern case $field1-$field2 in decstation-3100) basic_machine=mips-dec os= ;; *-*) # Second component is usually, but not always the OS case $field2 in # Prevent following clause from handling this valid os sun*os*) basic_machine=$field1 os=$field2 ;; # Manufacturers dec* | mips* | sequent* | encore* | pc533* | sgi* | sony* \ | att* | 7300* | 3300* | delta* | motorola* | sun[234]* \ | unicom* | ibm* | next | hp | isi* | apollo | altos* \ | convergent* | ncr* | news | 32* | 3600* | 3100* \ | hitachi* | c[123]* | convex* | sun | crds | omron* | dg \ | ultra | tti* | harris | dolphin | highlevel | gould \ | cbm | ns | masscomp | apple | axis | knuth | cray \ | microblaze* | sim | cisco \ | oki | wec | wrs | winbond) basic_machine=$field1-$field2 os= ;; *) basic_machine=$field1 os=$field2 ;; esac ;; esac ;; *) # Convert single-component short-hands not valid as part of # multi-component configurations. case $field1 in 386bsd) basic_machine=i386-pc os=bsd ;; a29khif) basic_machine=a29k-amd os=udi ;; adobe68k) basic_machine=m68010-adobe os=scout ;; alliant) basic_machine=fx80-alliant os= ;; altos | altos3068) basic_machine=m68k-altos os= ;; am29k) basic_machine=a29k-none os=bsd ;; amdahl) basic_machine=580-amdahl os=sysv ;; amiga) basic_machine=m68k-unknown os= ;; amigaos | amigados) basic_machine=m68k-unknown os=amigaos ;; amigaunix | amix) basic_machine=m68k-unknown os=sysv4 ;; apollo68) basic_machine=m68k-apollo os=sysv ;; apollo68bsd) basic_machine=m68k-apollo os=bsd ;; aros) basic_machine=i386-pc os=aros ;; aux) basic_machine=m68k-apple os=aux ;; balance) basic_machine=ns32k-sequent os=dynix ;; blackfin) basic_machine=bfin-unknown os=linux ;; cegcc) basic_machine=arm-unknown os=cegcc ;; convex-c1) basic_machine=c1-convex os=bsd ;; convex-c2) basic_machine=c2-convex os=bsd ;; convex-c32) basic_machine=c32-convex os=bsd ;; convex-c34) basic_machine=c34-convex os=bsd ;; convex-c38) basic_machine=c38-convex os=bsd ;; cray) basic_machine=j90-cray os=unicos ;; crds | unos) basic_machine=m68k-crds os= ;; da30) basic_machine=m68k-da30 os= ;; decstation | pmax | pmin | dec3100 | decstatn) basic_machine=mips-dec os= ;; delta88) basic_machine=m88k-motorola os=sysv3 ;; dicos) basic_machine=i686-pc os=dicos ;; djgpp) basic_machine=i586-pc os=msdosdjgpp ;; ebmon29k) basic_machine=a29k-amd os=ebmon ;; es1800 | OSE68k | ose68k | ose | OSE) basic_machine=m68k-ericsson os=ose ;; gmicro) basic_machine=tron-gmicro os=sysv ;; go32) basic_machine=i386-pc os=go32 ;; h8300hms) basic_machine=h8300-hitachi os=hms ;; h8300xray) basic_machine=h8300-hitachi os=xray ;; h8500hms) basic_machine=h8500-hitachi os=hms ;; harris) basic_machine=m88k-harris os=sysv3 ;; hp300) basic_machine=m68k-hp ;; hp300bsd) basic_machine=m68k-hp os=bsd ;; hp300hpux) basic_machine=m68k-hp os=hpux ;; hppaosf) basic_machine=hppa1.1-hp os=osf ;; hppro) basic_machine=hppa1.1-hp os=proelf ;; i386mach) basic_machine=i386-mach os=mach ;; vsta) basic_machine=i386-pc os=vsta ;; isi68 | isi) basic_machine=m68k-isi os=sysv ;; m68knommu) basic_machine=m68k-unknown os=linux ;; magnum | m3230) basic_machine=mips-mips os=sysv ;; merlin) basic_machine=ns32k-utek os=sysv ;; mingw64) basic_machine=x86_64-pc os=mingw64 ;; mingw32) basic_machine=i686-pc os=mingw32 ;; mingw32ce) basic_machine=arm-unknown os=mingw32ce ;; monitor) basic_machine=m68k-rom68k os=coff ;; morphos) basic_machine=powerpc-unknown os=morphos ;; moxiebox) basic_machine=moxie-unknown os=moxiebox ;; msdos) basic_machine=i386-pc os=msdos ;; msys) basic_machine=i686-pc os=msys ;; mvs) basic_machine=i370-ibm os=mvs ;; nacl) basic_machine=le32-unknown os=nacl ;; ncr3000) basic_machine=i486-ncr os=sysv4 ;; netbsd386) basic_machine=i386-pc os=netbsd ;; netwinder) basic_machine=armv4l-rebel os=linux ;; news | news700 | news800 | news900) basic_machine=m68k-sony os=newsos ;; news1000) basic_machine=m68030-sony os=newsos ;; necv70) basic_machine=v70-nec os=sysv ;; nh3000) basic_machine=m68k-harris os=cxux ;; nh[45]000) basic_machine=m88k-harris os=cxux ;; nindy960) basic_machine=i960-intel os=nindy ;; mon960) basic_machine=i960-intel os=mon960 ;; nonstopux) basic_machine=mips-compaq os=nonstopux ;; os400) basic_machine=powerpc-ibm os=os400 ;; OSE68000 | ose68000) basic_machine=m68000-ericsson os=ose ;; os68k) basic_machine=m68k-none os=os68k ;; paragon) basic_machine=i860-intel os=osf ;; parisc) basic_machine=hppa-unknown os=linux ;; pw32) basic_machine=i586-unknown os=pw32 ;; rdos | rdos64) basic_machine=x86_64-pc os=rdos ;; rdos32) basic_machine=i386-pc os=rdos ;; rom68k) basic_machine=m68k-rom68k os=coff ;; sa29200) basic_machine=a29k-amd os=udi ;; sei) basic_machine=mips-sei os=seiux ;; sequent) basic_machine=i386-sequent os= ;; sps7) basic_machine=m68k-bull os=sysv2 ;; st2000) basic_machine=m68k-tandem os= ;; stratus) basic_machine=i860-stratus os=sysv4 ;; sun2) basic_machine=m68000-sun os= ;; sun2os3) basic_machine=m68000-sun os=sunos3 ;; sun2os4) basic_machine=m68000-sun os=sunos4 ;; sun3) basic_machine=m68k-sun os= ;; sun3os3) basic_machine=m68k-sun os=sunos3 ;; sun3os4) basic_machine=m68k-sun os=sunos4 ;; sun4) basic_machine=sparc-sun os= ;; sun4os3) basic_machine=sparc-sun os=sunos3 ;; sun4os4) basic_machine=sparc-sun os=sunos4 ;; sun4sol2) basic_machine=sparc-sun os=solaris2 ;; sun386 | sun386i | roadrunner) basic_machine=i386-sun os= ;; sv1) basic_machine=sv1-cray os=unicos ;; symmetry) basic_machine=i386-sequent os=dynix ;; t3e) basic_machine=alphaev5-cray os=unicos ;; t90) basic_machine=t90-cray os=unicos ;; toad1) basic_machine=pdp10-xkl os=tops20 ;; tpf) basic_machine=s390x-ibm os=tpf ;; udi29k) basic_machine=a29k-amd os=udi ;; ultra3) basic_machine=a29k-nyu os=sym1 ;; v810 | necv810) basic_machine=v810-nec os=none ;; vaxv) basic_machine=vax-dec os=sysv ;; vms) basic_machine=vax-dec os=vms ;; vxworks960) basic_machine=i960-wrs os=vxworks ;; vxworks68) basic_machine=m68k-wrs os=vxworks ;; vxworks29k) basic_machine=a29k-wrs os=vxworks ;; xbox) basic_machine=i686-pc os=mingw32 ;; ymp) basic_machine=ymp-cray os=unicos ;; *) basic_machine=$1 os= ;; esac ;; esac # Decode 1-component or ad-hoc basic machines case $basic_machine in # Here we handle the default manufacturer of certain CPU types. It is in # some cases the only manufacturer, in others, it is the most popular. w89k) cpu=hppa1.1 vendor=winbond ;; op50n) cpu=hppa1.1 vendor=oki ;; op60c) cpu=hppa1.1 vendor=oki ;; ibm*) cpu=i370 vendor=ibm ;; orion105) cpu=clipper vendor=highlevel ;; mac | mpw | mac-mpw) cpu=m68k vendor=apple ;; pmac | pmac-mpw) cpu=powerpc vendor=apple ;; # Recognize the various machine names and aliases which stand # for a CPU type and a company and sometimes even an OS. 3b1 | 7300 | 7300-att | att-7300 | pc7300 | safari | unixpc) cpu=m68000 vendor=att ;; 3b*) cpu=we32k vendor=att ;; bluegene*) cpu=powerpc vendor=ibm os=cnk ;; decsystem10* | dec10*) cpu=pdp10 vendor=dec os=tops10 ;; decsystem20* | dec20*) cpu=pdp10 vendor=dec os=tops20 ;; delta | 3300 | motorola-3300 | motorola-delta \ | 3300-motorola | delta-motorola) cpu=m68k vendor=motorola ;; dpx2*) cpu=m68k vendor=bull os=sysv3 ;; encore | umax | mmax) cpu=ns32k vendor=encore ;; elxsi) cpu=elxsi vendor=elxsi os=${os:-bsd} ;; fx2800) cpu=i860 vendor=alliant ;; genix) cpu=ns32k vendor=ns ;; h3050r* | hiux*) cpu=hppa1.1 vendor=hitachi os=hiuxwe2 ;; hp3k9[0-9][0-9] | hp9[0-9][0-9]) cpu=hppa1.0 vendor=hp ;; hp9k2[0-9][0-9] | hp9k31[0-9]) cpu=m68000 vendor=hp ;; hp9k3[2-9][0-9]) cpu=m68k vendor=hp ;; hp9k6[0-9][0-9] | hp6[0-9][0-9]) cpu=hppa1.0 vendor=hp ;; hp9k7[0-79][0-9] | hp7[0-79][0-9]) cpu=hppa1.1 vendor=hp ;; hp9k78[0-9] | hp78[0-9]) # FIXME: really hppa2.0-hp cpu=hppa1.1 vendor=hp ;; hp9k8[67]1 | hp8[67]1 | hp9k80[24] | hp80[24] | hp9k8[78]9 | hp8[78]9 | hp9k893 | hp893) # FIXME: really hppa2.0-hp cpu=hppa1.1 vendor=hp ;; hp9k8[0-9][13679] | hp8[0-9][13679]) cpu=hppa1.1 vendor=hp ;; hp9k8[0-9][0-9] | hp8[0-9][0-9]) cpu=hppa1.0 vendor=hp ;; i*86v32) cpu=`echo "$1" | sed -e 's/86.*/86/'` vendor=pc os=sysv32 ;; i*86v4*) cpu=`echo "$1" | sed -e 's/86.*/86/'` vendor=pc os=sysv4 ;; i*86v) cpu=`echo "$1" | sed -e 's/86.*/86/'` vendor=pc os=sysv ;; i*86sol2) cpu=`echo "$1" | sed -e 's/86.*/86/'` vendor=pc os=solaris2 ;; j90 | j90-cray) cpu=j90 vendor=cray os=${os:-unicos} ;; iris | iris4d) cpu=mips vendor=sgi case $os in irix*) ;; *) os=irix4 ;; esac ;; miniframe) cpu=m68000 vendor=convergent ;; *mint | mint[0-9]* | *MiNT | *MiNT[0-9]*) cpu=m68k vendor=atari os=mint ;; news-3600 | risc-news) cpu=mips vendor=sony os=newsos ;; next | m*-next) cpu=m68k vendor=next case $os in openstep*) ;; nextstep*) ;; ns2*) os=nextstep2 ;; *) os=nextstep3 ;; esac ;; np1) cpu=np1 vendor=gould ;; op50n-* | op60c-*) cpu=hppa1.1 vendor=oki os=proelf ;; pa-hitachi) cpu=hppa1.1 vendor=hitachi os=hiuxwe2 ;; pbd) cpu=sparc vendor=tti ;; pbb) cpu=m68k vendor=tti ;; pc532) cpu=ns32k vendor=pc532 ;; pn) cpu=pn vendor=gould ;; power) cpu=power vendor=ibm ;; ps2) cpu=i386 vendor=ibm ;; rm[46]00) cpu=mips vendor=siemens ;; rtpc | rtpc-*) cpu=romp vendor=ibm ;; sde) cpu=mipsisa32 vendor=sde os=${os:-elf} ;; simso-wrs) cpu=sparclite vendor=wrs os=vxworks ;; tower | tower-32) cpu=m68k vendor=ncr ;; vpp*|vx|vx-*) cpu=f301 vendor=fujitsu ;; w65) cpu=w65 vendor=wdc ;; w89k-*) cpu=hppa1.1 vendor=winbond os=proelf ;; none) cpu=none vendor=none ;; leon|leon[3-9]) cpu=sparc vendor=$basic_machine ;; leon-*|leon[3-9]-*) cpu=sparc vendor=`echo "$basic_machine" | sed 's/-.*//'` ;; *-*) # shellcheck disable=SC2162 IFS="-" read cpu vendor <<EOF $basic_machine EOF ;; # We use `pc' rather than `unknown' # because (1) that's what they normally are, and # (2) the word "unknown" tends to confuse beginning users. i*86 | x86_64) cpu=$basic_machine vendor=pc ;; # These rules are duplicated from below for sake of the special case above; # i.e. things that normalized to x86 arches should also default to "pc" pc98) cpu=i386 vendor=pc ;; x64 | amd64) cpu=x86_64 vendor=pc ;; # Recognize the basic CPU types without company name. *) cpu=$basic_machine vendor=unknown ;; esac unset -v basic_machine # Decode basic machines in the full and proper CPU-Company form. case $cpu-$vendor in # Here we handle the default manufacturer of certain CPU types in canonical form. It is in # some cases the only manufacturer, in others, it is the most popular. craynv-unknown) vendor=cray os=${os:-unicosmp} ;; c90-unknown | c90-cray) vendor=cray os=${os:-unicos} ;; fx80-unknown) vendor=alliant ;; romp-unknown) vendor=ibm ;; mmix-unknown) vendor=knuth ;; microblaze-unknown | microblazeel-unknown) vendor=xilinx ;; rs6000-unknown) vendor=ibm ;; vax-unknown) vendor=dec ;; pdp11-unknown) vendor=dec ;; we32k-unknown) vendor=att ;; cydra-unknown) vendor=cydrome ;; i370-ibm*) vendor=ibm ;; orion-unknown) vendor=highlevel ;; xps-unknown | xps100-unknown) cpu=xps100 vendor=honeywell ;; # Here we normalize CPU types with a missing or matching vendor dpx20-unknown | dpx20-bull) cpu=rs6000 vendor=bull os=${os:-bosx} ;; # Here we normalize CPU types irrespective of the vendor amd64-*) cpu=x86_64 ;; blackfin-*) cpu=bfin os=linux ;; c54x-*) cpu=tic54x ;; c55x-*) cpu=tic55x ;; c6x-*) cpu=tic6x ;; e500v[12]-*) cpu=powerpc os=$os"spe" ;; mips3*-*) cpu=mips64 ;; ms1-*) cpu=mt ;; m68knommu-*) cpu=m68k os=linux ;; m9s12z-* | m68hcs12z-* | hcs12z-* | s12z-*) cpu=s12z ;; openrisc-*) cpu=or32 ;; parisc-*) cpu=hppa os=linux ;; pentium-* | p5-* | k5-* | k6-* | nexgen-* | viac3-*) cpu=i586 ;; pentiumpro-* | p6-* | 6x86-* | athlon-* | athalon_*-*) cpu=i686 ;; pentiumii-* | pentium2-* | pentiumiii-* | pentium3-*) cpu=i686 ;; pentium4-*) cpu=i786 ;; pc98-*) cpu=i386 ;; ppc-* | ppcbe-*) cpu=powerpc ;; ppcle-* | powerpclittle-*) cpu=powerpcle ;; ppc64-*) cpu=powerpc64 ;; ppc64le-* | powerpc64little-*) cpu=powerpc64le ;; sb1-*) cpu=mipsisa64sb1 ;; sb1el-*) cpu=mipsisa64sb1el ;; sh5e[lb]-*) cpu=`echo "$cpu" | sed 's/^\(sh.\)e\(.\)$/\1\2e/'` ;; spur-*) cpu=spur ;; strongarm-* | thumb-*) cpu=arm ;; tx39-*) cpu=mipstx39 ;; tx39el-*) cpu=mipstx39el ;; x64-*) cpu=x86_64 ;; xscale-* | xscalee[bl]-*) cpu=`echo "$cpu" | sed 's/^xscale/arm/'` ;; # Recognize the canonical CPU Types that limit and/or modify the # company names they are paired with. cr16-*) os=${os:-elf} ;; crisv32-* | etraxfs*-*) cpu=crisv32 vendor=axis ;; cris-* | etrax*-*) cpu=cris vendor=axis ;; crx-*) os=${os:-elf} ;; neo-tandem) cpu=neo vendor=tandem ;; nse-tandem) cpu=nse vendor=tandem ;; nsr-tandem) cpu=nsr vendor=tandem ;; nsv-tandem) cpu=nsv vendor=tandem ;; nsx-tandem) cpu=nsx vendor=tandem ;; s390-*) cpu=s390 vendor=ibm ;; s390x-*) cpu=s390x vendor=ibm ;; tile*-*) os=${os:-linux-gnu} ;; *) # Recognize the canonical CPU types that are allowed with any # company name. case $cpu in 1750a | 580 \ | a29k \ | aarch64 | aarch64_be \ | abacus \ | alpha | alphaev[4-8] | alphaev56 | alphaev6[78] \ | alpha64 | alpha64ev[4-8] | alpha64ev56 | alpha64ev6[78] \ | alphapca5[67] | alpha64pca5[67] \ | am33_2.0 \ | amdgcn \ | arc | arceb \ | arm | arm[lb]e | arme[lb] | armv* \ | avr | avr32 \ | asmjs \ | ba \ | be32 | be64 \ | bfin | bpf | bs2000 \ | c[123]* | c30 | [cjt]90 | c4x \ | c8051 | clipper | craynv | csky | cydra \ | d10v | d30v | dlx | dsp16xx \ | e2k | elxsi | epiphany \ | f30[01] | f700 | fido | fr30 | frv | ft32 | fx80 \ | h8300 | h8500 \ | hppa | hppa1.[01] | hppa2.0 | hppa2.0[nw] | hppa64 \ | hexagon \ | i370 | i*86 | i860 | i960 | ia16 | ia64 \ | ip2k | iq2000 \ | k1om \ | le32 | le64 \ | lm32 \ | m32c | m32r | m32rle \ | m5200 | m68000 | m680[012346]0 | m68360 | m683?2 | m68k \ | m6811 | m68hc11 | m6812 | m68hc12 | m68hcs12x \ | m88110 | m88k | maxq | mb | mcore | mep | metag \ | microblaze | microblazeel \ | mips | mipsbe | mipseb | mipsel | mipsle \ | mips16 \ | mips64 | mips64eb | mips64el \ | mips64octeon | mips64octeonel \ | mips64orion | mips64orionel \ | mips64r5900 | mips64r5900el \ | mips64vr | mips64vrel \ | mips64vr4100 | mips64vr4100el \ | mips64vr4300 | mips64vr4300el \ | mips64vr5000 | mips64vr5000el \ | mips64vr5900 | mips64vr5900el \ | mipsisa32 | mipsisa32el \ | mipsisa32r2 | mipsisa32r2el \ | mipsisa32r6 | mipsisa32r6el \ | mipsisa64 | mipsisa64el \ | mipsisa64r2 | mipsisa64r2el \ | mipsisa64r6 | mipsisa64r6el \ | mipsisa64sb1 | mipsisa64sb1el \ | mipsisa64sr71k | mipsisa64sr71kel \ | mipsr5900 | mipsr5900el \ | mipstx39 | mipstx39el \ | mmix \ | mn10200 | mn10300 \ | moxie \ | mt \ | msp430 \ | nds32 | nds32le | nds32be \ | nfp \ | nios | nios2 | nios2eb | nios2el \ | none | np1 | ns16k | ns32k | nvptx \ | open8 \ | or1k* \ | or32 \ | orion \ | picochip \ | pdp10 | pdp11 | pj | pjl | pn | power \ | powerpc | powerpc64 | powerpc64le | powerpcle | powerpcspe \ | pru \ | pyramid \ | riscv | riscv32 | riscv64 \ | rl78 | romp | rs6000 | rx \ | score \ | sh | shl \ | sh[1234] | sh[24]a | sh[24]ae[lb] | sh[23]e | she[lb] | sh[lb]e \ | sh[1234]e[lb] | sh[12345][lb]e | sh[23]ele | sh64 | sh64le \ | sparc | sparc64 | sparc64b | sparc64v | sparc86x | sparclet \ | sparclite \ | sparcv8 | sparcv9 | sparcv9b | sparcv9v | sv1 | sx* \ | spu \ | tahoe \ | tic30 | tic4x | tic54x | tic55x | tic6x | tic80 \ | tron \ | ubicom32 \ | v70 | v850 | v850e | v850e1 | v850es | v850e2 | v850e2v3 \ | vax \ | visium \ | w65 \ | wasm32 | wasm64 \ | we32k \ | x86 | x86_64 | xc16x | xgate | xps100 \ | xstormy16 | xtensa* \ | ymp \ | z8k | z80) ;; *) echo Invalid configuration \`"$1"\': machine \`"$cpu-$vendor"\' not recognized 1>&2 exit 1 ;; esac ;; esac # Here we canonicalize certain aliases for manufacturers. case $vendor in digital*) vendor=dec ;; commodore*) vendor=cbm ;; *) ;; esac # Decode manufacturer-specific aliases for certain operating systems. if [ x$os != x ] then case $os in # First match some system type aliases that might get confused # with valid system types. # solaris* is a basic system type, with this one exception. auroraux) os=auroraux ;; bluegene*) os=cnk ;; solaris1 | solaris1.*) os=`echo $os | sed -e 's|solaris1|sunos4|'` ;; solaris) os=solaris2 ;; unixware*) os=sysv4.2uw ;; gnu/linux*) os=`echo $os | sed -e 's|gnu/linux|linux-gnu|'` ;; # es1800 is here to avoid being matched by es* (a different OS) es1800*) os=ose ;; # Some version numbers need modification chorusos*) os=chorusos ;; isc) os=isc2.2 ;; sco6) os=sco5v6 ;; sco5) os=sco3.2v5 ;; sco4) os=sco3.2v4 ;; sco3.2.[4-9]*) os=`echo $os | sed -e 's/sco3.2./sco3.2v/'` ;; sco3.2v[4-9]* | sco5v6*) # Don't forget version if it is 3.2v4 or newer. ;; scout) # Don't match below ;; sco*) os=sco3.2v2 ;; psos*) os=psos ;; # Now accept the basic system types. # The portable systems comes first. # Each alternative MUST end in a * to match a version number. # sysv* is not here because it comes later, after sysvr4. gnu* | bsd* | mach* | minix* | genix* | ultrix* | irix* \ | *vms* | esix* | aix* | cnk* | sunos | sunos[34]*\ | hpux* | unos* | osf* | luna* | dgux* | auroraux* | solaris* \ | sym* | kopensolaris* | plan9* \ | amigaos* | amigados* | msdos* | newsos* | unicos* | aof* \ | aos* | aros* | cloudabi* | sortix* \ | nindy* | vxsim* | vxworks* | ebmon* | hms* | mvs* \ | clix* | riscos* | uniplus* | iris* | isc* | rtu* | xenix* \ | knetbsd* | mirbsd* | netbsd* \ | bitrig* | openbsd* | solidbsd* | libertybsd* \ | ekkobsd* | kfreebsd* | freebsd* | riscix* | lynxos* \ | bosx* | nextstep* | cxux* | aout* | elf* | oabi* \ | ptx* | coff* | ecoff* | winnt* | domain* | vsta* \ | udi* | eabi* | lites* | ieee* | go32* | aux* | hcos* \ | chorusrdb* | cegcc* | glidix* \ | cygwin* | msys* | pe* | moss* | proelf* | rtems* \ | midipix* | mingw32* | mingw64* | linux-gnu* | linux-android* \ | linux-newlib* | linux-musl* | linux-uclibc* \ | uxpv* | beos* | mpeix* | udk* | moxiebox* \ | interix* | uwin* | mks* | rhapsody* | darwin* \ | openstep* | oskit* | conix* | pw32* | nonstopux* \ | storm-chaos* | tops10* | tenex* | tops20* | its* \ | os2* | vos* | palmos* | uclinux* | nucleus* \ | morphos* | superux* | rtmk* | windiss* \ | powermax* | dnix* | nx6 | nx7 | sei* | dragonfly* \ | skyos* | haiku* | rdos* | toppers* | drops* | es* \ | onefs* | tirtos* | phoenix* | fuchsia* | redox* | bme* \ | midnightbsd* | amdhsa* | unleashed* | emscripten* | wasi*) # Remember, each alternative MUST END IN *, to match a version number. ;; qnx*) case $cpu in x86 | i*86) ;; *) os=nto-$os ;; esac ;; hiux*) os=hiuxwe2 ;; nto-qnx*) ;; nto*) os=`echo $os | sed -e 's|nto|nto-qnx|'` ;; sim | xray | os68k* | v88r* \ | windows* | osx | abug | netware* | os9* \ | macos* | mpw* | magic* | mmixware* | mon960* | lnews*) ;; linux-dietlibc) os=linux-dietlibc ;; linux*) os=`echo $os | sed -e 's|linux|linux-gnu|'` ;; lynx*178) os=lynxos178 ;; lynx*5) os=lynxos5 ;; lynx*) os=lynxos ;; mac*) os=`echo "$os" | sed -e 's|mac|macos|'` ;; opened*) os=openedition ;; os400*) os=os400 ;; sunos5*) os=`echo "$os" | sed -e 's|sunos5|solaris2|'` ;; sunos6*) os=`echo "$os" | sed -e 's|sunos6|solaris3|'` ;; wince*) os=wince ;; utek*) os=bsd ;; dynix*) os=bsd ;; acis*) os=aos ;; atheos*) os=atheos ;; syllable*) os=syllable ;; 386bsd) os=bsd ;; ctix* | uts*) os=sysv ;; nova*) os=rtmk-nova ;; ns2) os=nextstep2 ;; nsk*) os=nsk ;; # Preserve the version number of sinix5. sinix5.*) os=`echo $os | sed -e 's|sinix|sysv|'` ;; sinix*) os=sysv4 ;; tpf*) os=tpf ;; triton*) os=sysv3 ;; oss*) os=sysv3 ;; svr4*) os=sysv4 ;; svr3) os=sysv3 ;; sysvr4) os=sysv4 ;; # This must come after sysvr4. sysv*) ;; ose*) os=ose ;; *mint | mint[0-9]* | *MiNT | MiNT[0-9]*) os=mint ;; zvmoe) os=zvmoe ;; dicos*) os=dicos ;; pikeos*) # Until real need of OS specific support for # particular features comes up, bare metal # configurations are quite functional. case $cpu in arm*) os=eabi ;; *) os=elf ;; esac ;; nacl*) ;; ios) ;; none) ;; *-eabi) ;; *) echo Invalid configuration \`"$1"\': system \`"$os"\' not recognized 1>&2 exit 1 ;; esac else # Here we handle the default operating systems that come with various machines. # The value should be what the vendor currently ships out the door with their # machine or put another way, the most popular os provided with the machine. # Note that if you're going to try to match "-MANUFACTURER" here (say, # "-sun"), then you have to tell the case statement up towards the top # that MANUFACTURER isn't an operating system. Otherwise, code above # will signal an error saying that MANUFACTURER isn't an operating # system, and we'll never get to this point. case $cpu-$vendor in score-*) os=elf ;; spu-*) os=elf ;; *-acorn) os=riscix1.2 ;; arm*-rebel) os=linux ;; arm*-semi) os=aout ;; c4x-* | tic4x-*) os=coff ;; c8051-*) os=elf ;; clipper-intergraph) os=clix ;; hexagon-*) os=elf ;; tic54x-*) os=coff ;; tic55x-*) os=coff ;; tic6x-*) os=coff ;; # This must come before the *-dec entry. pdp10-*) os=tops20 ;; pdp11-*) os=none ;; *-dec | vax-*) os=ultrix4.2 ;; m68*-apollo) os=domain ;; i386-sun) os=sunos4.0.2 ;; m68000-sun) os=sunos3 ;; m68*-cisco) os=aout ;; mep-*) os=elf ;; mips*-cisco) os=elf ;; mips*-*) os=elf ;; or32-*) os=coff ;; *-tti) # must be before sparc entry or we get the wrong os. os=sysv3 ;; sparc-* | *-sun) os=sunos4.1.1 ;; pru-*) os=elf ;; *-be) os=beos ;; *-ibm) os=aix ;; *-knuth) os=mmixware ;; *-wec) os=proelf ;; *-winbond) os=proelf ;; *-oki) os=proelf ;; *-hp) os=hpux ;; *-hitachi) os=hiux ;; i860-* | *-att | *-ncr | *-altos | *-motorola | *-convergent) os=sysv ;; *-cbm) os=amigaos ;; *-dg) os=dgux ;; *-dolphin) os=sysv3 ;; m68k-ccur) os=rtu ;; m88k-omron*) os=luna ;; *-next) os=nextstep ;; *-sequent) os=ptx ;; *-crds) os=unos ;; *-ns) os=genix ;; i370-*) os=mvs ;; *-gould) os=sysv ;; *-highlevel) os=bsd ;; *-encore) os=bsd ;; *-sgi) os=irix ;; *-siemens) os=sysv4 ;; *-masscomp) os=rtu ;; f30[01]-fujitsu | f700-fujitsu) os=uxpv ;; *-rom68k) os=coff ;; *-*bug) os=coff ;; *-apple) os=macos ;; *-atari*) os=mint ;; *-wrs) os=vxworks ;; *) os=none ;; esac fi # Here we handle the case where we know the os, and the CPU type, but not the # manufacturer. We pick the logical manufacturer. case $vendor in unknown) case $os in riscix*) vendor=acorn ;; sunos*) vendor=sun ;; cnk*|-aix*) vendor=ibm ;; beos*) vendor=be ;; hpux*) vendor=hp ;; mpeix*) vendor=hp ;; hiux*) vendor=hitachi ;; unos*) vendor=crds ;; dgux*) vendor=dg ;; luna*) vendor=omron ;; genix*) vendor=ns ;; clix*) vendor=intergraph ;; mvs* | opened*) vendor=ibm ;; os400*) vendor=ibm ;; ptx*) vendor=sequent ;; tpf*) vendor=ibm ;; vxsim* | vxworks* | windiss*) vendor=wrs ;; aux*) vendor=apple ;; hms*) vendor=hitachi ;; mpw* | macos*) vendor=apple ;; *mint | mint[0-9]* | *MiNT | MiNT[0-9]*) vendor=atari ;; vos*) vendor=stratus ;; esac ;; esac echo "$cpu-$vendor-$os" exit # Local variables: # eval: (add-hook 'before-save-hook 'time-stamp) # time-stamp-start: "timestamp='" # time-stamp-format: "%:y-%02m-%02d" # time-stamp-end: "'" # End: |
Changes to configure.
1 2 | #! /bin/sh # Guess values for system-dependent variables and create Makefiles. | | | 1 2 3 4 5 6 7 8 9 10 | #! /bin/sh # Guess values for system-dependent variables and create Makefiles. # Generated by GNU Autoconf 2.69 for sqlite 3.41.0. # # # Copyright (C) 1992-1996, 1998-2012 Free Software Foundation, Inc. # # # This configure script is free software; the Free Software Foundation # gives unlimited permission to copy, distribute and modify it. |
︙ | ︙ | |||
722 723 724 725 726 727 728 | subdirs= MFLAGS= MAKEFLAGS= # Identity of this package. PACKAGE_NAME='sqlite' PACKAGE_TARNAME='sqlite' | | | | 722 723 724 725 726 727 728 729 730 731 732 733 734 735 736 737 | subdirs= MFLAGS= MAKEFLAGS= # Identity of this package. PACKAGE_NAME='sqlite' PACKAGE_TARNAME='sqlite' PACKAGE_VERSION='3.41.0' PACKAGE_STRING='sqlite 3.41.0' PACKAGE_BUGREPORT='' PACKAGE_URL='' # Factoring default headers for most tests. ac_includes_default="\ #include <stdio.h> #ifdef HAVE_SYS_TYPES_H |
︙ | ︙ | |||
766 767 768 769 770 771 772 773 774 775 776 777 778 779 | #ifdef HAVE_UNISTD_H # include <unistd.h> #endif" ac_subst_vars='LTLIBOBJS LIBOBJS BUILD_CFLAGS USE_GCOV OPT_FEATURE_FLAGS HAVE_ZLIB USE_AMALGAMATION TARGET_DEBUG TARGET_HAVE_EDITLINE TARGET_HAVE_READLINE | > | 766 767 768 769 770 771 772 773 774 775 776 777 778 779 780 | #ifdef HAVE_UNISTD_H # include <unistd.h> #endif" ac_subst_vars='LTLIBOBJS LIBOBJS BUILD_CFLAGS AMALGAMATION_LINE_MACROS USE_GCOV OPT_FEATURE_FLAGS HAVE_ZLIB USE_AMALGAMATION TARGET_DEBUG TARGET_HAVE_EDITLINE TARGET_HAVE_READLINE |
︙ | ︙ | |||
795 796 797 798 799 800 801 | SQLITE_OS_WIN SQLITE_OS_UNIX BUILD_EXEEXT TEMP_STORE ALLOWRELEASE SQLITE_THREADSAFE BUILD_CC | < | 796 797 798 799 800 801 802 803 804 805 806 807 808 809 | SQLITE_OS_WIN SQLITE_OS_UNIX BUILD_EXEEXT TEMP_STORE ALLOWRELEASE SQLITE_THREADSAFE BUILD_CC RELEASE VERSION program_prefix TCLLIBDIR TCLSH_CMD INSTALL_DATA INSTALL_SCRIPT |
︙ | ︙ | |||
900 901 902 903 904 905 906 907 908 909 910 911 | enable_editline enable_readline with_readline_lib with_readline_inc enable_debug enable_amalgamation enable_load_extension enable_memsys5 enable_memsys3 enable_fts3 enable_fts4 enable_fts5 | > > > < | 900 901 902 903 904 905 906 907 908 909 910 911 912 913 914 915 916 917 918 919 920 921 | enable_editline enable_readline with_readline_lib with_readline_inc enable_debug enable_amalgamation enable_load_extension enable_math enable_json enable_all enable_memsys5 enable_memsys3 enable_fts3 enable_fts4 enable_fts5 enable_update_limit enable_geopoly enable_rtree enable_session enable_gcov ' ac_precious_vars='build_alias |
︙ | ︙ | |||
1462 1463 1464 1465 1466 1467 1468 | # # Report the --help message. # if test "$ac_init_help" = "long"; then # Omit some internal or obsolete options to make the list less imposing. # This message is too long to be a string in the A/UX 3.1 sh. cat <<_ACEOF | | | 1464 1465 1466 1467 1468 1469 1470 1471 1472 1473 1474 1475 1476 1477 1478 | # # Report the --help message. # if test "$ac_init_help" = "long"; then # Omit some internal or obsolete options to make the list less imposing. # This message is too long to be a string in the A/UX 3.1 sh. cat <<_ACEOF \`configure' configures sqlite 3.41.0 to adapt to many kinds of systems. Usage: $0 [OPTION]... [VAR=VALUE]... To assign environment variables (e.g., CC, CFLAGS...), specify them as VAR=VALUE. See below for descriptions of some of the useful variables. Defaults for the options are specified in brackets. |
︙ | ︙ | |||
1527 1528 1529 1530 1531 1532 1533 | --build=BUILD configure for building on BUILD [guessed] --host=HOST cross-compile to build programs to run on HOST [BUILD] _ACEOF fi if test -n "$ac_init_help"; then case $ac_init_help in | | | 1529 1530 1531 1532 1533 1534 1535 1536 1537 1538 1539 1540 1541 1542 1543 | --build=BUILD configure for building on BUILD [guessed] --host=HOST cross-compile to build programs to run on HOST [BUILD] _ACEOF fi if test -n "$ac_init_help"; then case $ac_init_help in short | recursive ) echo "Configuration of sqlite 3.41.0:";; esac cat <<\_ACEOF Optional Features: --disable-option-checking ignore unrecognized --enable/--with options --disable-FEATURE do not include FEATURE (same as --enable-FEATURE=no) --enable-FEATURE[=ARG] include FEATURE [ARG=yes] |
︙ | ︙ | |||
1553 1554 1555 1556 1557 1558 1559 1560 1561 1562 1563 1564 | --enable-editline enable BSD editline support --disable-readline disable readline support --enable-debug enable debugging & verbose explain --disable-amalgamation Disable the amalgamation and instead build all files separately --disable-load-extension Disable loading of external extensions --enable-memsys5 Enable MEMSYS5 --enable-memsys3 Enable MEMSYS3 --enable-fts3 Enable the FTS3 extension --enable-fts4 Enable the FTS4 extension --enable-fts5 Enable the FTS5 extension | > > > < | 1555 1556 1557 1558 1559 1560 1561 1562 1563 1564 1565 1566 1567 1568 1569 1570 1571 1572 1573 1574 1575 1576 | --enable-editline enable BSD editline support --disable-readline disable readline support --enable-debug enable debugging & verbose explain --disable-amalgamation Disable the amalgamation and instead build all files separately --disable-load-extension Disable loading of external extensions --disable-math Disable math functions --disable-json Disable JSON functions --enable-all Enable FTS4, FTS5, Geopoly, RTree, Sessions --enable-memsys5 Enable MEMSYS5 --enable-memsys3 Enable MEMSYS3 --enable-fts3 Enable the FTS3 extension --enable-fts4 Enable the FTS4 extension --enable-fts5 Enable the FTS5 extension --enable-update-limit Enable the UPDATE/DELETE LIMIT clause --enable-geopoly Enable the GEOPOLY extension --enable-rtree Enable the RTREE extension --enable-session Enable the SESSION extension --enable-gcov Enable coverage testing using gcov Optional Packages: |
︙ | ︙ | |||
1653 1654 1655 1656 1657 1658 1659 | cd "$ac_pwd" || { ac_status=$?; break; } done fi test -n "$ac_init_help" && exit $ac_status if $ac_init_version; then cat <<\_ACEOF | | | 1657 1658 1659 1660 1661 1662 1663 1664 1665 1666 1667 1668 1669 1670 1671 | cd "$ac_pwd" || { ac_status=$?; break; } done fi test -n "$ac_init_help" && exit $ac_status if $ac_init_version; then cat <<\_ACEOF sqlite configure 3.41.0 generated by GNU Autoconf 2.69 Copyright (C) 2012 Free Software Foundation, Inc. This configure script is free software; the Free Software Foundation gives unlimited permission to copy, distribute and modify it. _ACEOF exit |
︙ | ︙ | |||
2072 2073 2074 2075 2076 2077 2078 | eval $as_lineno_stack; ${as_lineno_stack:+:} unset as_lineno } # ac_fn_c_check_header_mongrel cat >config.log <<_ACEOF This file contains any messages produced by compilers while running configure, to aid debugging if configure makes a mistake. | | | 2076 2077 2078 2079 2080 2081 2082 2083 2084 2085 2086 2087 2088 2089 2090 | eval $as_lineno_stack; ${as_lineno_stack:+:} unset as_lineno } # ac_fn_c_check_header_mongrel cat >config.log <<_ACEOF This file contains any messages produced by compilers while running configure, to aid debugging if configure makes a mistake. It was created by sqlite $as_me 3.41.0, which was generated by GNU Autoconf 2.69. Invocation command line was $ $0 $@ _ACEOF exec 5>>config.log { |
︙ | ︙ | |||
3930 3931 3932 3933 3934 3935 3936 | { $as_echo "$as_me:${as_lineno-$LINENO}: checking the name lister ($NM) interface" >&5 $as_echo_n "checking the name lister ($NM) interface... " >&6; } if ${lt_cv_nm_interface+:} false; then : $as_echo_n "(cached) " >&6 else lt_cv_nm_interface="BSD nm" echo "int some_variable = 0;" > conftest.$ac_ext | | | | | 3934 3935 3936 3937 3938 3939 3940 3941 3942 3943 3944 3945 3946 3947 3948 3949 3950 3951 3952 3953 3954 | { $as_echo "$as_me:${as_lineno-$LINENO}: checking the name lister ($NM) interface" >&5 $as_echo_n "checking the name lister ($NM) interface... " >&6; } if ${lt_cv_nm_interface+:} false; then : $as_echo_n "(cached) " >&6 else lt_cv_nm_interface="BSD nm" echo "int some_variable = 0;" > conftest.$ac_ext (eval echo "\"\$as_me:3941: $ac_compile\"" >&5) (eval "$ac_compile" 2>conftest.err) cat conftest.err >&5 (eval echo "\"\$as_me:3944: $NM \\\"conftest.$ac_objext\\\"\"" >&5) (eval "$NM \"conftest.$ac_objext\"" 2>conftest.err > conftest.out) cat conftest.err >&5 (eval echo "\"\$as_me:3947: output\"" >&5) cat conftest.out >&5 if $GREP 'External.*some_variable' conftest.out > /dev/null; then lt_cv_nm_interface="MS dumpbin" fi rm -f conftest* fi { $as_echo "$as_me:${as_lineno-$LINENO}: result: $lt_cv_nm_interface" >&5 |
︙ | ︙ | |||
5142 5143 5144 5145 5146 5147 5148 | ;; esac fi rm -rf conftest* ;; *-*-irix6*) # Find out which ABI we are using. | | | 5146 5147 5148 5149 5150 5151 5152 5153 5154 5155 5156 5157 5158 5159 5160 | ;; esac fi rm -rf conftest* ;; *-*-irix6*) # Find out which ABI we are using. echo '#line 5153 "configure"' > conftest.$ac_ext if { { eval echo "\"\$as_me\":${as_lineno-$LINENO}: \"$ac_compile\""; } >&5 (eval $ac_compile) 2>&5 ac_status=$? $as_echo "$as_me:${as_lineno-$LINENO}: \$? = $ac_status" >&5 test $ac_status = 0; }; then if test "$lt_cv_prog_gnu_ld" = yes; then case `/usr/bin/file conftest.$ac_objext` in |
︙ | ︙ | |||
6667 6668 6669 6670 6671 6672 6673 | # Note that $ac_compile itself does not contain backslashes and begins # with a dollar sign (not a hyphen), so the echo should work correctly. # The option is referenced via a variable to avoid confusing sed. lt_compile=`echo "$ac_compile" | $SED \ -e 's:.*FLAGS}\{0,1\} :&$lt_compiler_flag :; t' \ -e 's: [^ ]*conftest\.: $lt_compiler_flag&:; t' \ -e 's:$: $lt_compiler_flag:'` | | | | 6671 6672 6673 6674 6675 6676 6677 6678 6679 6680 6681 6682 6683 6684 6685 6686 6687 6688 6689 | # Note that $ac_compile itself does not contain backslashes and begins # with a dollar sign (not a hyphen), so the echo should work correctly. # The option is referenced via a variable to avoid confusing sed. lt_compile=`echo "$ac_compile" | $SED \ -e 's:.*FLAGS}\{0,1\} :&$lt_compiler_flag :; t' \ -e 's: [^ ]*conftest\.: $lt_compiler_flag&:; t' \ -e 's:$: $lt_compiler_flag:'` (eval echo "\"\$as_me:6678: $lt_compile\"" >&5) (eval "$lt_compile" 2>conftest.err) ac_status=$? cat conftest.err >&5 echo "$as_me:6682: \$? = $ac_status" >&5 if (exit $ac_status) && test -s "$ac_outfile"; then # The compiler can only warn and ignore the option if not recognized # So say no if there are warnings other than the usual output. $ECHO "X$_lt_compiler_boilerplate" | $Xsed -e '/^$/d' >conftest.exp $SED '/^$/d; /^ *+/d' conftest.err >conftest.er2 if test ! -s conftest.er2 || diff conftest.exp conftest.er2 >/dev/null; then lt_cv_prog_compiler_rtti_exceptions=yes |
︙ | ︙ | |||
7006 7007 7008 7009 7010 7011 7012 | # Note that $ac_compile itself does not contain backslashes and begins # with a dollar sign (not a hyphen), so the echo should work correctly. # The option is referenced via a variable to avoid confusing sed. lt_compile=`echo "$ac_compile" | $SED \ -e 's:.*FLAGS}\{0,1\} :&$lt_compiler_flag :; t' \ -e 's: [^ ]*conftest\.: $lt_compiler_flag&:; t' \ -e 's:$: $lt_compiler_flag:'` | | | | 7010 7011 7012 7013 7014 7015 7016 7017 7018 7019 7020 7021 7022 7023 7024 7025 7026 7027 7028 | # Note that $ac_compile itself does not contain backslashes and begins # with a dollar sign (not a hyphen), so the echo should work correctly. # The option is referenced via a variable to avoid confusing sed. lt_compile=`echo "$ac_compile" | $SED \ -e 's:.*FLAGS}\{0,1\} :&$lt_compiler_flag :; t' \ -e 's: [^ ]*conftest\.: $lt_compiler_flag&:; t' \ -e 's:$: $lt_compiler_flag:'` (eval echo "\"\$as_me:7017: $lt_compile\"" >&5) (eval "$lt_compile" 2>conftest.err) ac_status=$? cat conftest.err >&5 echo "$as_me:7021: \$? = $ac_status" >&5 if (exit $ac_status) && test -s "$ac_outfile"; then # The compiler can only warn and ignore the option if not recognized # So say no if there are warnings other than the usual output. $ECHO "X$_lt_compiler_boilerplate" | $Xsed -e '/^$/d' >conftest.exp $SED '/^$/d; /^ *+/d' conftest.err >conftest.er2 if test ! -s conftest.er2 || diff conftest.exp conftest.er2 >/dev/null; then lt_cv_prog_compiler_pic_works=yes |
︙ | ︙ | |||
7111 7112 7113 7114 7115 7116 7117 | # (2) before a word containing "conftest.", or (3) at the end. # Note that $ac_compile itself does not contain backslashes and begins # with a dollar sign (not a hyphen), so the echo should work correctly. lt_compile=`echo "$ac_compile" | $SED \ -e 's:.*FLAGS}\{0,1\} :&$lt_compiler_flag :; t' \ -e 's: [^ ]*conftest\.: $lt_compiler_flag&:; t' \ -e 's:$: $lt_compiler_flag:'` | | | | 7115 7116 7117 7118 7119 7120 7121 7122 7123 7124 7125 7126 7127 7128 7129 7130 7131 7132 7133 | # (2) before a word containing "conftest.", or (3) at the end. # Note that $ac_compile itself does not contain backslashes and begins # with a dollar sign (not a hyphen), so the echo should work correctly. lt_compile=`echo "$ac_compile" | $SED \ -e 's:.*FLAGS}\{0,1\} :&$lt_compiler_flag :; t' \ -e 's: [^ ]*conftest\.: $lt_compiler_flag&:; t' \ -e 's:$: $lt_compiler_flag:'` (eval echo "\"\$as_me:7122: $lt_compile\"" >&5) (eval "$lt_compile" 2>out/conftest.err) ac_status=$? cat out/conftest.err >&5 echo "$as_me:7126: \$? = $ac_status" >&5 if (exit $ac_status) && test -s out/conftest2.$ac_objext then # The compiler can only warn and ignore the option if not recognized # So say no if there are warnings $ECHO "X$_lt_compiler_boilerplate" | $Xsed -e '/^$/d' > out/conftest.exp $SED '/^$/d; /^ *+/d' out/conftest.err >out/conftest.er2 if test ! -s out/conftest.er2 || diff out/conftest.exp out/conftest.er2 >/dev/null; then |
︙ | ︙ | |||
7166 7167 7168 7169 7170 7171 7172 | # (2) before a word containing "conftest.", or (3) at the end. # Note that $ac_compile itself does not contain backslashes and begins # with a dollar sign (not a hyphen), so the echo should work correctly. lt_compile=`echo "$ac_compile" | $SED \ -e 's:.*FLAGS}\{0,1\} :&$lt_compiler_flag :; t' \ -e 's: [^ ]*conftest\.: $lt_compiler_flag&:; t' \ -e 's:$: $lt_compiler_flag:'` | | | | 7170 7171 7172 7173 7174 7175 7176 7177 7178 7179 7180 7181 7182 7183 7184 7185 7186 7187 7188 | # (2) before a word containing "conftest.", or (3) at the end. # Note that $ac_compile itself does not contain backslashes and begins # with a dollar sign (not a hyphen), so the echo should work correctly. lt_compile=`echo "$ac_compile" | $SED \ -e 's:.*FLAGS}\{0,1\} :&$lt_compiler_flag :; t' \ -e 's: [^ ]*conftest\.: $lt_compiler_flag&:; t' \ -e 's:$: $lt_compiler_flag:'` (eval echo "\"\$as_me:7177: $lt_compile\"" >&5) (eval "$lt_compile" 2>out/conftest.err) ac_status=$? cat out/conftest.err >&5 echo "$as_me:7181: \$? = $ac_status" >&5 if (exit $ac_status) && test -s out/conftest2.$ac_objext then # The compiler can only warn and ignore the option if not recognized # So say no if there are warnings $ECHO "X$_lt_compiler_boilerplate" | $Xsed -e '/^$/d' > out/conftest.exp $SED '/^$/d; /^ *+/d' out/conftest.err >out/conftest.er2 if test ! -s out/conftest.er2 || diff out/conftest.exp out/conftest.er2 >/dev/null; then |
︙ | ︙ | |||
9546 9547 9548 9549 9550 9551 9552 | else if test "$cross_compiling" = yes; then : lt_cv_dlopen_self=cross else lt_dlunknown=0; lt_dlno_uscore=1; lt_dlneed_uscore=2 lt_status=$lt_dlunknown cat > conftest.$ac_ext <<_LT_EOF | | | 9550 9551 9552 9553 9554 9555 9556 9557 9558 9559 9560 9561 9562 9563 9564 | else if test "$cross_compiling" = yes; then : lt_cv_dlopen_self=cross else lt_dlunknown=0; lt_dlno_uscore=1; lt_dlneed_uscore=2 lt_status=$lt_dlunknown cat > conftest.$ac_ext <<_LT_EOF #line 9557 "configure" #include "confdefs.h" #if HAVE_DLFCN_H #include <dlfcn.h> #endif #include <stdio.h> |
︙ | ︙ | |||
9642 9643 9644 9645 9646 9647 9648 | else if test "$cross_compiling" = yes; then : lt_cv_dlopen_self_static=cross else lt_dlunknown=0; lt_dlno_uscore=1; lt_dlneed_uscore=2 lt_status=$lt_dlunknown cat > conftest.$ac_ext <<_LT_EOF | | | 9646 9647 9648 9649 9650 9651 9652 9653 9654 9655 9656 9657 9658 9659 9660 | else if test "$cross_compiling" = yes; then : lt_cv_dlopen_self_static=cross else lt_dlunknown=0; lt_dlno_uscore=1; lt_dlneed_uscore=2 lt_status=$lt_dlunknown cat > conftest.$ac_ext <<_LT_EOF #line 9653 "configure" #include "confdefs.h" #if HAVE_DLFCN_H #include <dlfcn.h> #endif #include <stdio.h> |
︙ | ︙ | |||
10359 10360 10361 10362 10363 10364 10365 | fi if test "x${TCLLIBDIR+set}" != "xset" ; then TCLLIBDIR='$(libdir)' for i in `echo 'puts stdout $auto_path' | ${TCLSH_CMD}` ; do | > | | > | 10363 10364 10365 10366 10367 10368 10369 10370 10371 10372 10373 10374 10375 10376 10377 10378 10379 10380 | fi if test "x${TCLLIBDIR+set}" != "xset" ; then TCLLIBDIR='$(libdir)' for i in `echo 'puts stdout $auto_path' | ${TCLSH_CMD}` ; do if test -d $i ; then TCLLIBDIR=$i break fi done TCLLIBDIR="${TCLLIBDIR}/sqlite3" fi ######### # Set up an appropriate program prefix |
︙ | ︙ | |||
10382 10383 10384 10385 10386 10387 10388 | { $as_echo "$as_me:${as_lineno-$LINENO}: Version set to $VERSION" >&5 $as_echo "$as_me: Version set to $VERSION" >&6;} RELEASE=`cat $srcdir/VERSION` { $as_echo "$as_me:${as_lineno-$LINENO}: Release set to $RELEASE" >&5 $as_echo "$as_me: Release set to $RELEASE" >&6;} | < < < < < < | 10388 10389 10390 10391 10392 10393 10394 10395 10396 10397 10398 10399 10400 10401 | { $as_echo "$as_me:${as_lineno-$LINENO}: Version set to $VERSION" >&5 $as_echo "$as_me: Version set to $VERSION" >&6;} RELEASE=`cat $srcdir/VERSION` { $as_echo "$as_me:${as_lineno-$LINENO}: Release set to $RELEASE" >&5 $as_echo "$as_me: Release set to $RELEASE" >&6;} ######### # Locate a compiler for the build machine. This compiler should # generate command-line programs that run on the build machine. # if test x"$cross_compiling" = xno; then BUILD_CC=$CC |
︙ | ︙ | |||
11248 11249 11250 11251 11252 11253 11254 11255 11256 11257 11258 11259 11260 11261 11262 11263 11264 11265 11266 11267 11268 | ######### # check for debug enabled # Check whether --enable-debug was given. if test "${enable_debug+set}" = set; then : enableval=$enable_debug; fi if test "${enable_debug}" = "yes" ; then TARGET_DEBUG="-DSQLITE_DEBUG=1 -DSQLITE_ENABLE_SELECTTRACE -DSQLITE_ENABLE_WHERETRACE -O0" else TARGET_DEBUG="-DNDEBUG" fi ######### # See whether we should use the amalgamation to build # Check whether --enable-amalgamation was given. if test "${enable_amalgamation+set}" = set; then : enableval=$enable_amalgamation; fi | > > > > > > | | 11248 11249 11250 11251 11252 11253 11254 11255 11256 11257 11258 11259 11260 11261 11262 11263 11264 11265 11266 11267 11268 11269 11270 11271 11272 11273 11274 11275 11276 11277 11278 11279 11280 11281 11282 | ######### # check for debug enabled # Check whether --enable-debug was given. if test "${enable_debug+set}" = set; then : enableval=$enable_debug; fi { $as_echo "$as_me:${as_lineno-$LINENO}: checking build type" >&5 $as_echo_n "checking build type... " >&6; } if test "${enable_debug}" = "yes" ; then TARGET_DEBUG="-DSQLITE_DEBUG=1 -DSQLITE_ENABLE_SELECTTRACE -DSQLITE_ENABLE_WHERETRACE -O0" { $as_echo "$as_me:${as_lineno-$LINENO}: result: debug" >&5 $as_echo "debug" >&6; } else TARGET_DEBUG="-DNDEBUG" { $as_echo "$as_me:${as_lineno-$LINENO}: result: release" >&5 $as_echo "release" >&6; } fi ######### # See whether we should use the amalgamation to build # Check whether --enable-amalgamation was given. if test "${enable_amalgamation+set}" = set; then : enableval=$enable_amalgamation; fi if test "${enable_amalgamation}" = "no" ; then USE_AMALGAMATION=0 fi ######### # Look for zlib. Only needed by extensions and by the sqlite3.exe shell for ac_header in zlib.h |
︙ | ︙ | |||
11411 11412 11413 11414 11415 11416 11417 11418 11419 11420 11421 11422 11423 11424 | test "$ac_res" = "none required" || LIBS="$ac_res $LIBS" fi else OPT_FEATURE_FLAGS="-DSQLITE_OMIT_LOAD_EXTENSION=1" fi ########## # Do we want to support memsys3 and/or memsys5 # # Check whether --enable-memsys5 was given. if test "${enable_memsys5+set}" = set; then : enableval=$enable_memsys5; | > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | 11417 11418 11419 11420 11421 11422 11423 11424 11425 11426 11427 11428 11429 11430 11431 11432 11433 11434 11435 11436 11437 11438 11439 11440 11441 11442 11443 11444 11445 11446 11447 11448 11449 11450 11451 11452 11453 11454 11455 11456 11457 11458 11459 11460 11461 11462 11463 11464 11465 11466 11467 11468 11469 11470 11471 11472 11473 11474 11475 11476 11477 11478 11479 11480 11481 11482 11483 11484 11485 11486 11487 11488 11489 11490 11491 11492 11493 11494 11495 11496 11497 11498 11499 11500 11501 11502 11503 11504 11505 11506 11507 11508 11509 11510 11511 11512 11513 11514 11515 11516 11517 11518 11519 11520 11521 11522 11523 11524 11525 11526 11527 11528 11529 11530 11531 11532 11533 | test "$ac_res" = "none required" || LIBS="$ac_res $LIBS" fi else OPT_FEATURE_FLAGS="-DSQLITE_OMIT_LOAD_EXTENSION=1" fi ########## # Do we want to support math functions # # Check whether --enable-math was given. if test "${enable_math+set}" = set; then : enableval=$enable_math; fi { $as_echo "$as_me:${as_lineno-$LINENO}: checking whether to support math functions" >&5 $as_echo_n "checking whether to support math functions... " >&6; } if test "$enable_math" = "no"; then { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 $as_echo "no" >&6; } else { $as_echo "$as_me:${as_lineno-$LINENO}: result: yes" >&5 $as_echo "yes" >&6; } OPT_FEATURE_FLAGS="${OPT_FEATURE_FLAGS} -DSQLITE_ENABLE_MATH_FUNCTIONS" { $as_echo "$as_me:${as_lineno-$LINENO}: checking for library containing ceil" >&5 $as_echo_n "checking for library containing ceil... " >&6; } if ${ac_cv_search_ceil+:} false; then : $as_echo_n "(cached) " >&6 else ac_func_search_save_LIBS=$LIBS cat confdefs.h - <<_ACEOF >conftest.$ac_ext /* end confdefs.h. */ /* Override any GCC internal prototype to avoid an error. Use char because int might match the return type of a GCC builtin and then its argument prototype would still apply. */ #ifdef __cplusplus extern "C" #endif char ceil (); int main () { return ceil (); ; return 0; } _ACEOF for ac_lib in '' m; do if test -z "$ac_lib"; then ac_res="none required" else ac_res=-l$ac_lib LIBS="-l$ac_lib $ac_func_search_save_LIBS" fi if ac_fn_c_try_link "$LINENO"; then : ac_cv_search_ceil=$ac_res fi rm -f core conftest.err conftest.$ac_objext \ conftest$ac_exeext if ${ac_cv_search_ceil+:} false; then : break fi done if ${ac_cv_search_ceil+:} false; then : else ac_cv_search_ceil=no fi rm conftest.$ac_ext LIBS=$ac_func_search_save_LIBS fi { $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_cv_search_ceil" >&5 $as_echo "$ac_cv_search_ceil" >&6; } ac_res=$ac_cv_search_ceil if test "$ac_res" != no; then : test "$ac_res" = "none required" || LIBS="$ac_res $LIBS" fi fi ########## # Do we want to support JSON functions # # Check whether --enable-json was given. if test "${enable_json+set}" = set; then : enableval=$enable_json; fi { $as_echo "$as_me:${as_lineno-$LINENO}: checking whether to support JSON functions" >&5 $as_echo_n "checking whether to support JSON functions... " >&6; } if test "$enable_json" = "no"; then { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 $as_echo "no" >&6; } OPT_FEATURE_FLAGS="${OPT_FEATURE_FLAGS} -DSQLITE_OMIT_JSON" else { $as_echo "$as_me:${as_lineno-$LINENO}: result: yes" >&5 $as_echo "yes" >&6; } fi ######## # The --enable-all argument is short-hand to enable # multiple extensions. # Check whether --enable-all was given. if test "${enable_all+set}" = set; then : enableval=$enable_all; fi ########## # Do we want to support memsys3 and/or memsys5 # # Check whether --enable-memsys5 was given. if test "${enable_memsys5+set}" = set; then : enableval=$enable_memsys5; |
︙ | ︙ | |||
11453 11454 11455 11456 11457 11458 11459 11460 11461 11462 11463 11464 11465 11466 11467 | ######### # See whether we should enable Full Text Search extensions # Check whether --enable-fts3 was given. if test "${enable_fts3+set}" = set; then : enableval=$enable_fts3; fi if test "${enable_fts3}" = "yes" ; then OPT_FEATURE_FLAGS="${OPT_FEATURE_FLAGS} -DSQLITE_ENABLE_FTS3" fi # Check whether --enable-fts4 was given. if test "${enable_fts4+set}" = set; then : enableval=$enable_fts4; fi | > > > > > > > > > | > > | 11562 11563 11564 11565 11566 11567 11568 11569 11570 11571 11572 11573 11574 11575 11576 11577 11578 11579 11580 11581 11582 11583 11584 11585 11586 11587 11588 11589 11590 11591 11592 11593 11594 11595 | ######### # See whether we should enable Full Text Search extensions # Check whether --enable-fts3 was given. if test "${enable_fts3+set}" = set; then : enableval=$enable_fts3; fi { $as_echo "$as_me:${as_lineno-$LINENO}: checking whether to support FTS3" >&5 $as_echo_n "checking whether to support FTS3... " >&6; } if test "${enable_fts3}" = "yes" ; then OPT_FEATURE_FLAGS="${OPT_FEATURE_FLAGS} -DSQLITE_ENABLE_FTS3" { $as_echo "$as_me:${as_lineno-$LINENO}: result: yes" >&5 $as_echo "yes" >&6; } else { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 $as_echo "no" >&6; } fi # Check whether --enable-fts4 was given. if test "${enable_fts4+set}" = set; then : enableval=$enable_fts4; fi { $as_echo "$as_me:${as_lineno-$LINENO}: checking whether to support FTS4" >&5 $as_echo_n "checking whether to support FTS4... " >&6; } if test "${enable_fts4}" = "yes" -o "${enable_all}" = "yes" ; then { $as_echo "$as_me:${as_lineno-$LINENO}: result: yes" >&5 $as_echo "yes" >&6; } OPT_FEATURE_FLAGS="${OPT_FEATURE_FLAGS} -DSQLITE_ENABLE_FTS4" { $as_echo "$as_me:${as_lineno-$LINENO}: checking for library containing log" >&5 $as_echo_n "checking for library containing log... " >&6; } if ${ac_cv_search_log+:} false; then : $as_echo_n "(cached) " >&6 else ac_func_search_save_LIBS=$LIBS |
︙ | ︙ | |||
11519 11520 11521 11522 11523 11524 11525 11526 11527 11528 11529 11530 11531 | $as_echo "$ac_cv_search_log" >&6; } ac_res=$ac_cv_search_log if test "$ac_res" != no; then : test "$ac_res" = "none required" || LIBS="$ac_res $LIBS" fi fi # Check whether --enable-fts5 was given. if test "${enable_fts5+set}" = set; then : enableval=$enable_fts5; fi | > > > > > | > > | 11639 11640 11641 11642 11643 11644 11645 11646 11647 11648 11649 11650 11651 11652 11653 11654 11655 11656 11657 11658 11659 11660 11661 11662 11663 11664 11665 11666 | $as_echo "$ac_cv_search_log" >&6; } ac_res=$ac_cv_search_log if test "$ac_res" != no; then : test "$ac_res" = "none required" || LIBS="$ac_res $LIBS" fi else { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 $as_echo "no" >&6; } fi # Check whether --enable-fts5 was given. if test "${enable_fts5+set}" = set; then : enableval=$enable_fts5; fi { $as_echo "$as_me:${as_lineno-$LINENO}: checking whether to support FTS5" >&5 $as_echo_n "checking whether to support FTS5... " >&6; } if test "${enable_fts5}" = "yes" -o "${enable_all}" = "yes" ; then { $as_echo "$as_me:${as_lineno-$LINENO}: result: yes" >&5 $as_echo "yes" >&6; } OPT_FEATURE_FLAGS="${OPT_FEATURE_FLAGS} -DSQLITE_ENABLE_FTS5" { $as_echo "$as_me:${as_lineno-$LINENO}: checking for library containing log" >&5 $as_echo_n "checking for library containing log... " >&6; } if ${ac_cv_search_log+:} false; then : $as_echo_n "(cached) " >&6 else ac_func_search_save_LIBS=$LIBS |
︙ | ︙ | |||
11583 11584 11585 11586 11587 11588 11589 | $as_echo "$ac_cv_search_log" >&6; } ac_res=$ac_cv_search_log if test "$ac_res" != no; then : test "$ac_res" = "none required" || LIBS="$ac_res $LIBS" fi | < | < < < < < < | < < > > > | > > > > > > > | > > > > > > > > > > > > > > | > > > > > | 11710 11711 11712 11713 11714 11715 11716 11717 11718 11719 11720 11721 11722 11723 11724 11725 11726 11727 11728 11729 11730 11731 11732 11733 11734 11735 11736 11737 11738 11739 11740 11741 11742 11743 11744 11745 11746 11747 11748 11749 11750 11751 11752 11753 11754 11755 11756 11757 11758 11759 11760 11761 11762 11763 11764 11765 11766 11767 11768 11769 11770 11771 11772 11773 11774 11775 11776 11777 11778 11779 11780 11781 11782 11783 11784 11785 11786 11787 11788 11789 11790 11791 11792 11793 11794 11795 11796 11797 11798 11799 11800 11801 11802 11803 | $as_echo "$ac_cv_search_log" >&6; } ac_res=$ac_cv_search_log if test "$ac_res" != no; then : test "$ac_res" = "none required" || LIBS="$ac_res $LIBS" fi else { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 $as_echo "no" >&6; } fi ######### # See whether we should enable the LIMIT clause on UPDATE and DELETE # statements. # Check whether --enable-update-limit was given. if test "${enable_update_limit+set}" = set; then : enableval=$enable_update_limit; fi { $as_echo "$as_me:${as_lineno-$LINENO}: checking whether to support LIMIT on UPDATE and DELETE statements" >&5 $as_echo_n "checking whether to support LIMIT on UPDATE and DELETE statements... " >&6; } if test "${enable_update_limit}" = "yes" ; then OPT_FEATURE_FLAGS="${OPT_FEATURE_FLAGS} -DSQLITE_ENABLE_UPDATE_DELETE_LIMIT" { $as_echo "$as_me:${as_lineno-$LINENO}: result: yes" >&5 $as_echo "yes" >&6; } else { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 $as_echo "no" >&6; } fi ######### # See whether we should enable GEOPOLY # Check whether --enable-geopoly was given. if test "${enable_geopoly+set}" = set; then : enableval=$enable_geopoly; enable_geopoly=yes else enable_geopoly=no fi { $as_echo "$as_me:${as_lineno-$LINENO}: checking whether to support GEOPOLY" >&5 $as_echo_n "checking whether to support GEOPOLY... " >&6; } if test "${enable_geopoly}" = "yes" -o "${enable_all}" = "yes" ; then OPT_FEATURE_FLAGS="${OPT_FEATURE_FLAGS} -DSQLITE_ENABLE_GEOPOLY" enable_rtree=yes { $as_echo "$as_me:${as_lineno-$LINENO}: result: yes" >&5 $as_echo "yes" >&6; } else { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 $as_echo "no" >&6; } fi ######### # See whether we should enable RTREE # Check whether --enable-rtree was given. if test "${enable_rtree+set}" = set; then : enableval=$enable_rtree; fi { $as_echo "$as_me:${as_lineno-$LINENO}: checking whether to support RTREE" >&5 $as_echo_n "checking whether to support RTREE... " >&6; } if test "${enable_rtree}" = "yes" ; then OPT_FEATURE_FLAGS="${OPT_FEATURE_FLAGS} -DSQLITE_ENABLE_RTREE" { $as_echo "$as_me:${as_lineno-$LINENO}: result: yes" >&5 $as_echo "yes" >&6; } else { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 $as_echo "no" >&6; } fi ######### # See whether we should enable the SESSION extension # Check whether --enable-session was given. if test "${enable_session+set}" = set; then : enableval=$enable_session; fi { $as_echo "$as_me:${as_lineno-$LINENO}: checking whether to support SESSION" >&5 $as_echo_n "checking whether to support SESSION... " >&6; } if test "${enable_session}" = "yes" -o "${enable_all}" = "yes" ; then OPT_FEATURE_FLAGS="${OPT_FEATURE_FLAGS} -DSQLITE_ENABLE_SESSION" OPT_FEATURE_FLAGS="${OPT_FEATURE_FLAGS} -DSQLITE_ENABLE_PREUPDATE_HOOK" { $as_echo "$as_me:${as_lineno-$LINENO}: result: yes" >&5 $as_echo "yes" >&6; } else { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 $as_echo "no" >&6; } fi ######### # attempt to duplicate any OMITS and ENABLES into the ${OPT_FEATURE_FLAGS} parameter for option in $CFLAGS $CPPFLAGS do case $option in |
︙ | ︙ | |||
11710 11711 11712 11713 11714 11715 11716 11717 11718 11719 | if test "${use_gcov}" = "yes" ; then USE_GCOV=1 else USE_GCOV=0 fi ######### # Output the config header | > > > > > > > > > > > | | 11857 11858 11859 11860 11861 11862 11863 11864 11865 11866 11867 11868 11869 11870 11871 11872 11873 11874 11875 11876 11877 11878 11879 11880 11881 11882 11883 11884 11885 | if test "${use_gcov}" = "yes" ; then USE_GCOV=1 else USE_GCOV=0 fi ######### # Enable/disabled amalagamation line macros ######## AMALGAMATION_LINE_MACROS=--linemacros=0 if test "${amalgamation_line_macros}" = "yes" ; then AMALGAMATION_LINE_MACROS=--linemacros=1 fi if test "${amalgamation_line_macros}" = "no" ; then AMALGAMATION_LINE_MACROS=--linemacros=0 fi ######### # Output the config header ac_config_headers="$ac_config_headers sqlite_cfg.h" ######### # Generate the output files. # ac_config_files="$ac_config_files Makefile sqlite3.pc" |
︙ | ︙ | |||
12228 12229 12230 12231 12232 12233 12234 | test $as_write_fail = 0 && chmod +x $CONFIG_STATUS || ac_write_fail=1 cat >>$CONFIG_STATUS <<\_ACEOF || ac_write_fail=1 # Save the log message, to keep $0 and so on meaningful, and to # report actual input values of CONFIG_FILES etc. instead of their # values after options handling. ac_log=" | | | 12386 12387 12388 12389 12390 12391 12392 12393 12394 12395 12396 12397 12398 12399 12400 | test $as_write_fail = 0 && chmod +x $CONFIG_STATUS || ac_write_fail=1 cat >>$CONFIG_STATUS <<\_ACEOF || ac_write_fail=1 # Save the log message, to keep $0 and so on meaningful, and to # report actual input values of CONFIG_FILES etc. instead of their # values after options handling. ac_log=" This file was extended by sqlite $as_me 3.41.0, which was generated by GNU Autoconf 2.69. Invocation command line was CONFIG_FILES = $CONFIG_FILES CONFIG_HEADERS = $CONFIG_HEADERS CONFIG_LINKS = $CONFIG_LINKS CONFIG_COMMANDS = $CONFIG_COMMANDS $ $0 $@ |
︙ | ︙ | |||
12294 12295 12296 12297 12298 12299 12300 | Report bugs to the package provider." _ACEOF cat >>$CONFIG_STATUS <<_ACEOF || ac_write_fail=1 ac_cs_config="`$as_echo "$ac_configure_args" | sed 's/^ //; s/[\\""\`\$]/\\\\&/g'`" ac_cs_version="\\ | | | 12452 12453 12454 12455 12456 12457 12458 12459 12460 12461 12462 12463 12464 12465 12466 | Report bugs to the package provider." _ACEOF cat >>$CONFIG_STATUS <<_ACEOF || ac_write_fail=1 ac_cs_config="`$as_echo "$ac_configure_args" | sed 's/^ //; s/[\\""\`\$]/\\\\&/g'`" ac_cs_version="\\ sqlite config.status 3.41.0 configured by $0, generated by GNU Autoconf 2.69, with options \\"\$ac_cs_config\\" Copyright (C) 2012 Free Software Foundation, Inc. This config.status script is free software; the Free Software Foundation gives unlimited permission to copy, distribute and modify it." |
︙ | ︙ | |||
12676 12677 12678 12679 12680 12681 12682 | cat >>$CONFIG_STATUS <<\_ACEOF || ac_write_fail=1 # Handling of arguments. for ac_config_target in $ac_config_targets do case $ac_config_target in "libtool") CONFIG_COMMANDS="$CONFIG_COMMANDS libtool" ;; | | | 12834 12835 12836 12837 12838 12839 12840 12841 12842 12843 12844 12845 12846 12847 12848 | cat >>$CONFIG_STATUS <<\_ACEOF || ac_write_fail=1 # Handling of arguments. for ac_config_target in $ac_config_targets do case $ac_config_target in "libtool") CONFIG_COMMANDS="$CONFIG_COMMANDS libtool" ;; "sqlite_cfg.h") CONFIG_HEADERS="$CONFIG_HEADERS sqlite_cfg.h" ;; "Makefile") CONFIG_FILES="$CONFIG_FILES Makefile" ;; "sqlite3.pc") CONFIG_FILES="$CONFIG_FILES sqlite3.pc" ;; *) as_fn_error $? "invalid argument: \`$ac_config_target'" "$LINENO" 5;; esac done |
︙ | ︙ |
Changes to configure.ac.
︙ | ︙ | |||
130 131 132 133 134 135 136 | fi AC_SUBST(TCLSH_CMD) AC_ARG_VAR([TCLLIBDIR], [Where to install tcl plugin]) if test "x${TCLLIBDIR+set}" != "xset" ; then TCLLIBDIR='$(libdir)' for i in `echo 'puts stdout $auto_path' | ${TCLSH_CMD}` ; do | > | | > < < < < < | 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 | fi AC_SUBST(TCLSH_CMD) AC_ARG_VAR([TCLLIBDIR], [Where to install tcl plugin]) if test "x${TCLLIBDIR+set}" != "xset" ; then TCLLIBDIR='$(libdir)' for i in `echo 'puts stdout $auto_path' | ${TCLSH_CMD}` ; do if test -d $i ; then TCLLIBDIR=$i break fi done TCLLIBDIR="${TCLLIBDIR}/sqlite3" fi ######### # Set up an appropriate program prefix # if test "$program_prefix" = "NONE"; then program_prefix="" fi AC_SUBST(program_prefix) VERSION=[`cat $srcdir/VERSION | sed 's/^\([0-9]*\.*[0-9]*\).*/\1/'`] AC_MSG_NOTICE(Version set to $VERSION) AC_SUBST(VERSION) RELEASE=`cat $srcdir/VERSION` AC_MSG_NOTICE(Release set to $RELEASE) AC_SUBST(RELEASE) ######### # Locate a compiler for the build machine. This compiler should # generate command-line programs that run on the build machine. # if test x"$cross_compiling" = xno; then BUILD_CC=$CC |
︙ | ︙ | |||
554 555 556 557 558 559 560 561 562 563 564 565 566 567 568 569 570 571 | # that use "fdatasync()" function. # AC_SEARCH_LIBS(fdatasync, [rt]) ######### # check for debug enabled AC_ARG_ENABLE(debug, AC_HELP_STRING([--enable-debug],[enable debugging & verbose explain])) if test "${enable_debug}" = "yes" ; then TARGET_DEBUG="-DSQLITE_DEBUG=1 -DSQLITE_ENABLE_SELECTTRACE -DSQLITE_ENABLE_WHERETRACE -O0" else TARGET_DEBUG="-DNDEBUG" fi AC_SUBST(TARGET_DEBUG) ######### # See whether we should use the amalgamation to build AC_ARG_ENABLE(amalgamation, AC_HELP_STRING([--disable-amalgamation], [Disable the amalgamation and instead build all files separately])) | > > > | | 551 552 553 554 555 556 557 558 559 560 561 562 563 564 565 566 567 568 569 570 571 572 573 574 575 576 577 578 579 | # that use "fdatasync()" function. # AC_SEARCH_LIBS(fdatasync, [rt]) ######### # check for debug enabled AC_ARG_ENABLE(debug, AC_HELP_STRING([--enable-debug],[enable debugging & verbose explain])) AC_MSG_CHECKING([build type]) if test "${enable_debug}" = "yes" ; then TARGET_DEBUG="-DSQLITE_DEBUG=1 -DSQLITE_ENABLE_SELECTTRACE -DSQLITE_ENABLE_WHERETRACE -O0" AC_MSG_RESULT([debug]) else TARGET_DEBUG="-DNDEBUG" AC_MSG_RESULT([release]) fi AC_SUBST(TARGET_DEBUG) ######### # See whether we should use the amalgamation to build AC_ARG_ENABLE(amalgamation, AC_HELP_STRING([--disable-amalgamation], [Disable the amalgamation and instead build all files separately])) if test "${enable_amalgamation}" = "no" ; then USE_AMALGAMATION=0 fi AC_SUBST(USE_AMALGAMATION) ######### # Look for zlib. Only needed by extensions and by the sqlite3.exe shell AC_CHECK_HEADERS(zlib.h) |
︙ | ︙ | |||
587 588 589 590 591 592 593 594 595 596 597 598 599 600 | if test "${enable_load_extension}" = "yes" ; then OPT_FEATURE_FLAGS="" AC_SEARCH_LIBS(dlopen, dl) else OPT_FEATURE_FLAGS="-DSQLITE_OMIT_LOAD_EXTENSION=1" fi ########## # Do we want to support memsys3 and/or memsys5 # AC_ARG_ENABLE(memsys5, AC_HELP_STRING([--enable-memsys5],[Enable MEMSYS5])) AC_MSG_CHECKING([whether to support MEMSYS5]) if test "${enable_memsys5}" = "yes"; then | > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | 587 588 589 590 591 592 593 594 595 596 597 598 599 600 601 602 603 604 605 606 607 608 609 610 611 612 613 614 615 616 617 618 619 620 621 622 623 624 625 626 627 628 629 630 631 632 633 | if test "${enable_load_extension}" = "yes" ; then OPT_FEATURE_FLAGS="" AC_SEARCH_LIBS(dlopen, dl) else OPT_FEATURE_FLAGS="-DSQLITE_OMIT_LOAD_EXTENSION=1" fi ########## # Do we want to support math functions # AC_ARG_ENABLE(math, AC_HELP_STRING([--disable-math],[Disable math functions])) AC_MSG_CHECKING([whether to support math functions]) if test "$enable_math" = "no"; then AC_MSG_RESULT([no]) else AC_MSG_RESULT([yes]) OPT_FEATURE_FLAGS="${OPT_FEATURE_FLAGS} -DSQLITE_ENABLE_MATH_FUNCTIONS" AC_SEARCH_LIBS(ceil, m) fi ########## # Do we want to support JSON functions # AC_ARG_ENABLE(json, AC_HELP_STRING([--disable-json],[Disable JSON functions])) AC_MSG_CHECKING([whether to support JSON functions]) if test "$enable_json" = "no"; then AC_MSG_RESULT([no]) OPT_FEATURE_FLAGS="${OPT_FEATURE_FLAGS} -DSQLITE_OMIT_JSON" else AC_MSG_RESULT([yes]) fi ######## # The --enable-all argument is short-hand to enable # multiple extensions. AC_ARG_ENABLE(all, AC_HELP_STRING([--enable-all], [Enable FTS4, FTS5, Geopoly, RTree, Sessions])) ########## # Do we want to support memsys3 and/or memsys5 # AC_ARG_ENABLE(memsys5, AC_HELP_STRING([--enable-memsys5],[Enable MEMSYS5])) AC_MSG_CHECKING([whether to support MEMSYS5]) if test "${enable_memsys5}" = "yes"; then |
︙ | ︙ | |||
613 614 615 616 617 618 619 620 621 622 623 624 | AC_MSG_RESULT([no]) fi ######### # See whether we should enable Full Text Search extensions AC_ARG_ENABLE(fts3, AC_HELP_STRING([--enable-fts3], [Enable the FTS3 extension])) if test "${enable_fts3}" = "yes" ; then OPT_FEATURE_FLAGS="${OPT_FEATURE_FLAGS} -DSQLITE_ENABLE_FTS3" fi AC_ARG_ENABLE(fts4, AC_HELP_STRING([--enable-fts4], [Enable the FTS4 extension])) | > > > > > | > > > > | > < | < | < < < > | > > > > | > > > > > > > > | > > > | 646 647 648 649 650 651 652 653 654 655 656 657 658 659 660 661 662 663 664 665 666 667 668 669 670 671 672 673 674 675 676 677 678 679 680 681 682 683 684 685 686 687 688 689 690 691 692 693 694 695 696 697 698 699 700 701 702 703 704 705 706 707 708 709 710 711 712 713 714 715 716 717 718 719 720 721 722 723 724 725 726 727 728 729 730 731 732 733 734 735 736 737 | AC_MSG_RESULT([no]) fi ######### # See whether we should enable Full Text Search extensions AC_ARG_ENABLE(fts3, AC_HELP_STRING([--enable-fts3], [Enable the FTS3 extension])) AC_MSG_CHECKING([whether to support FTS3]) if test "${enable_fts3}" = "yes" ; then OPT_FEATURE_FLAGS="${OPT_FEATURE_FLAGS} -DSQLITE_ENABLE_FTS3" AC_MSG_RESULT([yes]) else AC_MSG_RESULT([no]) fi AC_ARG_ENABLE(fts4, AC_HELP_STRING([--enable-fts4], [Enable the FTS4 extension])) AC_MSG_CHECKING([whether to support FTS4]) if test "${enable_fts4}" = "yes" -o "${enable_all}" = "yes" ; then AC_MSG_RESULT([yes]) OPT_FEATURE_FLAGS="${OPT_FEATURE_FLAGS} -DSQLITE_ENABLE_FTS4" AC_SEARCH_LIBS([log],[m]) else AC_MSG_RESULT([no]) fi AC_ARG_ENABLE(fts5, AC_HELP_STRING([--enable-fts5], [Enable the FTS5 extension])) AC_MSG_CHECKING([whether to support FTS5]) if test "${enable_fts5}" = "yes" -o "${enable_all}" = "yes" ; then AC_MSG_RESULT([yes]) OPT_FEATURE_FLAGS="${OPT_FEATURE_FLAGS} -DSQLITE_ENABLE_FTS5" AC_SEARCH_LIBS([log],[m]) else AC_MSG_RESULT([no]) fi ######### # See whether we should enable the LIMIT clause on UPDATE and DELETE # statements. AC_ARG_ENABLE(update-limit, AC_HELP_STRING([--enable-update-limit], [Enable the UPDATE/DELETE LIMIT clause])) AC_MSG_CHECKING([whether to support LIMIT on UPDATE and DELETE statements]) if test "${enable_update_limit}" = "yes" ; then OPT_FEATURE_FLAGS="${OPT_FEATURE_FLAGS} -DSQLITE_ENABLE_UPDATE_DELETE_LIMIT" AC_MSG_RESULT([yes]) else AC_MSG_RESULT([no]) fi ######### # See whether we should enable GEOPOLY AC_ARG_ENABLE(geopoly, AC_HELP_STRING([--enable-geopoly], [Enable the GEOPOLY extension]), [enable_geopoly=yes],[enable_geopoly=no]) AC_MSG_CHECKING([whether to support GEOPOLY]) if test "${enable_geopoly}" = "yes" -o "${enable_all}" = "yes" ; then OPT_FEATURE_FLAGS="${OPT_FEATURE_FLAGS} -DSQLITE_ENABLE_GEOPOLY" enable_rtree=yes AC_MSG_RESULT([yes]) else AC_MSG_RESULT([no]) fi ######### # See whether we should enable RTREE AC_ARG_ENABLE(rtree, AC_HELP_STRING([--enable-rtree], [Enable the RTREE extension])) AC_MSG_CHECKING([whether to support RTREE]) if test "${enable_rtree}" = "yes" ; then OPT_FEATURE_FLAGS="${OPT_FEATURE_FLAGS} -DSQLITE_ENABLE_RTREE" AC_MSG_RESULT([yes]) else AC_MSG_RESULT([no]) fi ######### # See whether we should enable the SESSION extension AC_ARG_ENABLE(session, AC_HELP_STRING([--enable-session], [Enable the SESSION extension])) AC_MSG_CHECKING([whether to support SESSION]) if test "${enable_session}" = "yes" -o "${enable_all}" = "yes" ; then OPT_FEATURE_FLAGS="${OPT_FEATURE_FLAGS} -DSQLITE_ENABLE_SESSION" OPT_FEATURE_FLAGS="${OPT_FEATURE_FLAGS} -DSQLITE_ENABLE_PREUPDATE_HOOK" AC_MSG_RESULT([yes]) else AC_MSG_RESULT([no]) fi ######### # attempt to duplicate any OMITS and ENABLES into the ${OPT_FEATURE_FLAGS} parameter for option in $CFLAGS $CPPFLAGS do case $option in |
︙ | ︙ | |||
734 735 736 737 738 739 740 741 742 743 | if test "${use_gcov}" = "yes" ; then USE_GCOV=1 else USE_GCOV=0 fi AC_SUBST(USE_GCOV) ######### # Output the config header | > > > > > > > > > > > | | 788 789 790 791 792 793 794 795 796 797 798 799 800 801 802 803 804 805 806 807 808 809 810 811 812 813 814 815 816 817 818 | if test "${use_gcov}" = "yes" ; then USE_GCOV=1 else USE_GCOV=0 fi AC_SUBST(USE_GCOV) ######### # Enable/disabled amalagamation line macros ######## AMALGAMATION_LINE_MACROS=--linemacros=0 if test "${amalgamation_line_macros}" = "yes" ; then AMALGAMATION_LINE_MACROS=--linemacros=1 fi if test "${amalgamation_line_macros}" = "no" ; then AMALGAMATION_LINE_MACROS=--linemacros=0 fi AC_SUBST(AMALGAMATION_LINE_MACROS) ######### # Output the config header AC_CONFIG_HEADERS(sqlite_cfg.h) ######### # Generate the output files. # AC_SUBST(BUILD_CFLAGS) AC_OUTPUT([ Makefile sqlite3.pc ]) |
Added doc/begin_concurrent.md.
> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 | Begin Concurrent ================ ## Overview Usually, SQLite allows at most one writer to proceed concurrently. The BEGIN CONCURRENT enhancement allows multiple writers to process write transactions simultanously if the database is in "wal" or "wal2" mode, although the system still serializes COMMIT commands. When a write-transaction is opened with "BEGIN CONCURRENT", actually locking the database is deferred until a COMMIT is executed. This means that any number of transactions started with BEGIN CONCURRENT may proceed concurrently. The system uses optimistic page-level-locking to prevent conflicting concurrent transactions from being committed. When a BEGIN CONCURRENT transaction is committed, the system checks whether or not any of the database pages that the transaction has read have been modified since the BEGIN CONCURRENT was opened. In other words - it asks if the transaction being committed operates on a different set of data than all other concurrently executing transactions. If the answer is "yes, this transaction did not read or modify any data modified by any concurrent transaction", then the transaction is committed as normal. Otherwise, if the transaction does conflict, it cannot be committed and an SQLITE_BUSY_SNAPSHOT error is returned. At this point, all the client can do is ROLLBACK the transaction. If SQLITE_BUSY_SNAPSHOT is returned, messages are output via the sqlite3_log mechanism indicating the page and table or index on which the conflict occurred. This can be useful when optimizing concurrency. ## Application Programming Notes In order to serialize COMMIT processing, SQLite takes a lock on the database as part of each COMMIT command and releases it before returning. At most one writer may hold this lock at any one time. If a writer cannot obtain the lock, it uses SQLite's busy-handler to pause and retry for a while: <a href=https://www.sqlite.org/c3ref/busy_handler.html> https://www.sqlite.org/c3ref/busy_handler.html </a> If there is significant contention for the writer lock, this mechanism can be inefficient. In this case it is better for the application to use a mutex or some other mechanism that supports blocking to ensure that at most one writer is attempting to COMMIT a BEGIN CONCURRENT transaction at a time. This is usually easier if all writers are part of the same operating system process. If all database clients (readers and writers) are located in the same OS process, and if that OS is a Unix variant, then it can be more efficient to the built-in VFS "unix-excl" instead of the default "unix". This is because it uses more efficient locking primitives. The key to maximizing concurrency using BEGIN CONCURRENT is to ensure that there are a large number of non-conflicting transactions. In SQLite, each table and each index is stored as a separate b-tree, each of which is distributed over a discrete set of database pages. This means that: * Two transactions that write to different sets of tables never conflict, and that * Two transactions that write to the same tables or indexes only conflict if the values of the keys (either primary keys or indexed rows) are fairly close together. For example, given a large table with the schema: <pre> CREATE TABLE t1(a INTEGER PRIMARY KEY, b BLOB);</pre> writing two rows with adjacent values for "a" probably will cause a conflict (as the two keys are stored on the same page), but writing two rows with vastly different values for "a" will not (as the keys will likly be stored on different pages). Note that, in SQLite, if values are not explicitly supplied for an INTEGER PRIMARY KEY, as for example in: > INSERT INTO t1(b) VALUES(<blob-value>); then monotonically increasing values are assigned automatically. This is terrible for concurrency, as it all but ensures that all new rows are added to the same database page. In such situations, it is better to explicitly assign random values to INTEGER PRIMARY KEY fields. This problem also comes up for non-WITHOUT ROWID tables that do not have an explicit INTEGER PRIMARY KEY column. In these cases each table has an implicit INTEGER PRIMARY KEY column that is assigned increasing values, leading to the same problem as omitting to assign a value to an explicit INTEGER PRIMARY KEY column. For both explicit and implicit INTEGER PRIMARY KEYs, it is possible to have SQLite assign values at random (instead of the monotonically increasing values) by writing a row with a rowid equal to the largest possible signed 64-bit integer to the table. For example: INSERT INTO t1(a) VALUES(9223372036854775807); Applications should take care not to malfunction due to the presence of such rows. The nature of some types of indexes, for example indexes on timestamp fields, can also cause problems (as concurrent transactions may assign similar timestamps that will be stored on the same db page to new records). In these cases the database schema may need to be rethought to increase the concurrency provided by page-level-locking. |
Added doc/json-enhancements.md.
> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 | # JSON Functions Enhancements (2022) This document summaries enhancements to the SQLite JSON support added in early 2022. ## 1.0 Change summary: 1. New **->** and **->>** operators that work like MySQL and PostgreSQL (PG). 2. JSON functions are built-in rather than being an extension. They are included by default, but can be omitted using the -DSQLITE_OMIT_JSON compile-time option. ## 2.0 New operators **->** and **->>** The SQLite language adds two new binary operators **->** and **->>**. Both operators are similar to json_extract(). The left operand is JSON and the right operand is a JSON path expression (possibly abbreviated for compatibility with PG - see below). So they are similar to a two-argument call to json_extract(). The difference between -> and ->> (and json_extract()) is as follows: * The -> operator always returns JSON. * The ->> operator converts the answer into a primitive SQL datatype such as TEXT, INTEGER, REAL, or NULL. If a JSON object or array is selected, that object or array is rendered as text. If a JSON value is selected, that value is converted into its corresponding SQL type * The json_extract() interface returns JSON when a JSON object or array is selected, or a primitive SQL datatype when a JSON value is selected. This is different from MySQL, in which json_extract() always returns JSON, but the difference is retained because it has worked that way for 6 years and changing it now would likely break a lot of legacy code. In MySQL and PG, the ->> operator always returns TEXT (or NULL) and never INTEGER or REAL. This is due to limitations in the type handling capabilities of those systems. In MySQL and PG, the result type a function or operator may only depend on the type of its arguments, never the value of its arguments. But the underlying JSON type depends on the value of the JSON path expression, not the type of the JSON path expression (which is always TEXT). Hence, the result type of ->> in MySQL and PG is unable to vary according to the type of the JSON value being extracted. The type system in SQLite is more general. Functions in SQLite are able to return different datatypes depending on the value of their arguments. So the ->> operator in SQLite is able to return TEXT, INTEGER, REAL, or NULL depending on the JSON type of the value being extracted. This means that the behavior of the ->> is slightly different in SQLite versus MySQL and PG in that it will sometimes return INTEGER and REAL values, depending on its inputs. It is possible to implement the ->> operator in SQLite so that it always operates exactly like MySQL and PG and always returns TEXT or NULL, but I have been unable to think of any situations where returning the actual JSON value this would cause problems, so I'm including the enhanced functionality in SQLite. The table below attempts to summarize the differences between the -> and ->> operators and the json_extract() function, for SQLite, MySQL, and PG. JSON values are shown using their SQL text representation but in a bold font. <table border=1 cellpadding=5 cellspacing=0> <tr><th>JSON<th>PATH<th>-> operator<br>(all)<th>->> operator<br>(MySQL/PG) <th>->> operator<br>(SQLite)<th>json_extract()<br>(SQLite) <tr><td> **'{"a":123}'** <td>'$.a'<td> **'123'** <td> '123' <td> 123 <td> 123 <tr><td> **'{"a":4.5}'** <td>'$.a'<td> **'4.5'** <td> '4.5' <td> 4.5 <td> 4.5 <tr><td> **'{"a":"xyz"}'** <td>'$.a'<td> **'"xyz"'** <td> 'xyz' <td> 'xyz' <td> 'xyz' <tr><td> **'{"a":null}'** <td>'$.a'<td> **'null'** <td> NULL <td> NULL <td> NULL <tr><td> **'{"a":[6,7,8]}'** <td>'$.a'<td> **'[6,7,8]'** <td> '[6,7,8]' <td> '[6,7,8]' <td> **'[6,7,8]'** <tr><td> **'{"a":{"x":9}}'** <td>'$.a'<td> **'{"x":9}'** <td> '{"x":9}' <td> '{"x":9}' <td> **'{"x":9}'** <tr><td> **'{"b":999}'** <td>'$.a'<td> NULL <td> NULL <td> NULL <td> NULL </table> Important points about the table above: * The -> operator always returns either JSON or NULL. * The ->> operator never returns JSON. It always returns TEXT or NULL, or in the case of SQLite, INTEGER or REAL. * The MySQL json_extract() function works exactly the same as the MySQL -> operator. * The SQLite json_extract() operator works like -> for JSON objects and arrays, and like ->> for JSON values. * The -> operator works the same for all systems. * The only difference in ->> between SQLite and other systems is that when the JSON value is numeric, SQLite returns a numeric SQL value, whereas the other systems return a text representation of the numeric value. ### 2.1 Abbreviated JSON path expressions for PG compatibility The table above always shows the full JSON path expression: '$.a'. But PG does not accept this syntax. PG only allows a single JSON object label name or a single integer array index. In order to provide compatibility with PG, The -> and ->> operators in SQLite are extended to also support a JSON object label or an integer array index for the right-hand side operand, in addition to a full JSON path expression. Thus, a -> or ->> operator that works on MySQL will work in SQLite. And a -> or ->> operator that works in PG will work in SQLite. But because SQLite supports the union of the disjoint capabilities of MySQL and PG, there will always be -> and ->> operators that work in SQLite that do not work in one of MySQL and PG. This is an unavoidable consequence of the different syntax for -> and ->> in MySQL and PG. In the following table, assume that "value1" is a JSON object and "value2" is a JSON array. <table border=1 cellpadding=5 cellspacing=0> <tr><th>SQL expression <th>Works in MySQL?<th>Works in PG?<th>Works in SQLite <tr><td>value1->'$.a' <td> yes <td> no <td> yes <tr><td>value1->'a' <td> no <td> yes <td> yes <tr><td>value2->'$[2]' <td> yes <td> no <td> yes <tr><td>value2->2 <td> no <td> yes <td> yes </table> The abbreviated JSON path expressions only work for the -> and ->> operators in SQLite. The json_extract() function, and all other built-in SQLite JSON functions, continue to require complete JSON path expressions for their PATH arguments. ## 3.0 JSON moved into the core The JSON interface is now moved into the SQLite core. When originally written in 2015, the JSON functions were an extension that could be optionally included at compile-time, or loaded at run-time. The implementation was in a source file named ext/misc/json1.c in the source tree. JSON functions were only compiled in if the -DSQLITE_ENABLE_JSON1 compile-time option was used. After these enhancements, the JSON functions are now built-ins. The source file that implements the JSON functions is moved to src/json.c. No special compile-time options are needed to load JSON into the build. Instead, there is a new -DSQLITE_OMIT_JSON compile-time option to leave them out. |
Changes to doc/lemon.html.
1 2 3 4 | <html> <head> <title>The Lemon Parser Generator</title> </head> | | > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | | > | | | | | > | | | | | > | | | > | | | | | > > > > | > | | | | | | | | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 | <html> <head> <title>The Lemon Parser Generator</title> </head> <body> <a id="main"></a> <h1 align='center'>The Lemon Parser Generator</h1> <p>Lemon is an LALR(1) parser generator for C. It does the same job as "bison" and "yacc". But Lemon is not a bison or yacc clone. Lemon uses a different grammar syntax which is designed to reduce the number of coding errors. Lemon also uses a parsing engine that is faster than yacc and bison and which is both reentrant and threadsafe. (Update: Since the previous sentence was written, bison has also been updated so that it too can generate a reentrant and threadsafe parser.) Lemon also implements features that can be used to eliminate resource leaks, making it suitable for use in long-running programs such as graphical user interfaces or embedded controllers.</p> <p>This document is an introduction to the Lemon parser generator.</p> <a id="toc"></a> <h2>1.0 Table of Contents</h2> <ul> <li><a href="#main">Introduction</a> <li><a href="#toc">1.0 Table of Contents</a> <li><a href="#secnot">2.0 Security Notes</a><br> <li><a href="#optheory">3.0 Theory of Operation</a> <ul> <li><a href="#options">3.1 Command Line Options</a> <li><a href="#interface">3.2 The Parser Interface</a> <ul> <li><a href="#onstack">3.2.1 Allocating The Parse Object On Stack</a> <li><a href="#ifsum">3.2.2 Interface Summary</a> </ul> <li><a href="#yaccdiff">3.3 Differences With YACC and BISON</a> <li><a href="#build">3.4 Building The "lemon" Or "lemon.exe" Executable</a> </ul> <li><a href="#syntax">4.0 Input File Syntax</a> <ul> <li><a href="#tnt">4.1 Terminals and Nonterminals</a> <li><a href="#rules">4.2 Grammar Rules</a> <li><a href="#precrules">4.3 Precedence Rules</a> <li><a href="#special">4.4 Special Directives</a> </ul> <li><a href="#errors">5.0 Error Processing</a> <li><a href="#history">6.0 History of Lemon</a> <li><a href="#copyright">7.0 Copyright</a> </ul> <a id="secnot"></a> <h2>2.0 Security Note</h2> <p>The language parser code created by Lemon is very robust and is well-suited for use in internet-facing applications that need to safely process maliciously crafted inputs.</p> <p>The "lemon.exe" command-line tool itself works great when given a valid input grammar file and almost always gives helpful error messages for malformed inputs. However, it is possible for a malicious user to craft a grammar file that will cause lemon.exe to crash. We do not see this as a problem, as lemon.exe is not intended to be used with hostile inputs. To summarize:</p> <ul> <li>Parser code generated by lemon → Robust and secure <li>The "lemon.exe" command line tool itself → Not so much </ul> <a id="optheory"></a> <h2>3.0 Theory of Operation</h2> <p>Lemon is computer program that translates a context free grammar (CFG) for a particular language into C code that implements a parser for that language. The Lemon program has two inputs:</p> <ul> <li>The grammar specification. <li>A parser template file. </ul> <p>Typically, only the grammar specification is supplied by the programmer. Lemon comes with a default parser template ("<a href="https://sqlite.org/src/file/tool/lempar.c">lempar.c</a>") that works fine for most applications. But the user is free to substitute a different parser template if desired.</p> <p>Depending on command-line options, Lemon will generate up to three output files.</p> <ul> <li>C code to implement a parser for the input grammar. <li>A header file defining an integer ID for each terminal symbol (or "token"). <li>An information file that describes the states of the generated parser automaton. </ul> <p>By default, all three of these output files are generated. The header file is suppressed if the "-m" command-line option is used and the report file is omitted when "-q" is selected.</p> <p>The grammar specification file uses a ".y" suffix, by convention. In the examples used in this document, we'll assume the name of the grammar file is "gram.y". A typical use of Lemon would be the following command:</p> <pre> lemon gram.y </pre> <p>This command will generate three output files named "gram.c", "gram.h" and "gram.out". The first is C code to implement the parser. The second is the header file that defines numerical values for all terminal symbols, and the last is the report that explains the states used by the parser automaton.</p> <a id="options"></a> <h3>3.1 Command Line Options</h3> <p>The behavior of Lemon can be modified using command-line options. You can obtain a list of the available command-line options together with a brief explanation of what each does by typing</p> <pre> lemon "-?" </pre> <p>As of this writing, the following command-line options are supported:</p> <ul> <li><b>-b</b> Show only the basis for each parser state in the report file. <li><b>-c</b> Do not compress the generated action tables. The parser will be a little larger and slower, but it will detect syntax errors sooner. <li><b>-d</b><i>directory</i> Write all output files into <i>directory</i>. Normally, output files are written into the directory that contains the input grammar file. <li><b>-D<i>name</i></b> Define C preprocessor macro <i>name</i>. This macro is usable by "<tt><a href='#pifdef'>%ifdef</a></tt>", "<tt><a href='#pifdef'>%ifndef</a></tt>", and "<tt><a href="#pifdef">%if</a></tt> lines in the grammar file. <li><b>-E</b> Run the "%if" preprocessor step only and print the revised grammar file. <li><b>-g</b> Do not generate a parser. Instead write the input grammar to standard output with all comments, actions, and other extraneous text removed. <li><b>-l</b> Omit "#line" directives in the generated parser C code. <li><b>-m</b> Cause the output C source code to be compatible with the "makeheaders" program. <li><b>-p</b> Display all conflicts that are resolved by <a href='#precrules'>precedence rules</a>. <li><b>-q</b> Suppress generation of the report file. <li><b>-r</b> Do not sort or renumber the parser states as part of optimization. <li><b>-s</b> Show parser statistics before exiting. <li><b>-T<i>file</i></b> Use <i>file</i> as the template for the generated C-code parser implementation. <li><b>-x</b> Print the Lemon version number. </ul> <a id="interface"></a> <h3>3.2 The Parser Interface</h3> <p>Lemon doesn't generate a complete, working program. It only generates a few subroutines that implement a parser. This section describes the interface to those subroutines. It is up to the programmer to call these subroutines in an appropriate way in order to produce a complete system.</p> <p>Before a program begins using a Lemon-generated parser, the program must first create the parser. A new parser is created as follows:</p> <pre> void *pParser = ParseAlloc( malloc ); </pre> <p>The ParseAlloc() routine allocates and initializes a new parser and returns a pointer to it. The actual data structure used to represent a parser is opaque — its internal structure is not visible or usable by the calling routine. For this reason, the ParseAlloc() routine returns a pointer to void rather than a pointer to some particular structure. The sole argument to the ParseAlloc() routine is a pointer to the subroutine used to allocate memory. Typically this means malloc().</p> <p>After a program is finished using a parser, it can reclaim all memory allocated by that parser by calling</p> <pre> ParseFree(pParser, free); </pre> <p>The first argument is the same pointer returned by ParseAlloc(). The second argument is a pointer to the function used to release bulk memory back to the system.</p> <p>After a parser has been allocated using ParseAlloc(), the programmer must supply the parser with a sequence of tokens (terminal symbols) to be parsed. This is accomplished by calling the following function once for each token:<p> <pre> Parse(pParser, hTokenID, sTokenData, pArg); </pre> <p>The first argument to the Parse() routine is the pointer returned by ParseAlloc(). The second argument is a small positive integer that tells the parser the type of the next token in the data stream. There is one token type for each terminal symbol in the grammar. The gram.h file generated by Lemon contains #define statements that map symbolic terminal symbol names into appropriate integer values. A value of 0 for the second argument is a special flag to the |
︙ | ︙ | |||
191 192 193 194 195 196 197 | the Parse() function will have a fourth parameter that can be of any type chosen by the programmer. The parser doesn't do anything with this argument except to pass it through to action routines. This is a convenient mechanism for passing state information down to the action routines without having to use global variables.</p> <p>A typical use of a Lemon parser might look something like the | | | | | | | | > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | | | | > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | | | | | > | | 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340 341 342 343 344 345 346 347 348 349 350 351 352 353 354 355 356 357 358 359 360 361 362 363 364 365 366 367 368 369 370 371 372 373 374 375 376 377 378 379 380 381 382 383 384 385 386 387 388 389 390 391 392 393 394 395 396 397 398 399 400 401 402 403 404 405 406 407 408 409 410 411 412 413 414 415 416 417 418 419 420 421 422 423 424 425 426 427 428 429 430 431 432 433 434 435 436 437 438 439 440 441 442 443 | the Parse() function will have a fourth parameter that can be of any type chosen by the programmer. The parser doesn't do anything with this argument except to pass it through to action routines. This is a convenient mechanism for passing state information down to the action routines without having to use global variables.</p> <p>A typical use of a Lemon parser might look something like the following:</p> <pre> 1 ParseTree *ParseFile(const char *zFilename){ 2 Tokenizer *pTokenizer; 3 void *pParser; 4 Token sToken; 5 int hTokenId; 6 ParserState sState; 7 8 pTokenizer = TokenizerCreate(zFilename); 9 pParser = ParseAlloc( malloc ); 10 InitParserState(&sState); 11 while( GetNextToken(pTokenizer, &hTokenId, &sToken) ){ 12 Parse(pParser, hTokenId, sToken, &sState); 13 } 14 Parse(pParser, 0, sToken, &sState); 15 ParseFree(pParser, free ); 16 TokenizerFree(pTokenizer); 17 return sState.treeRoot; 18 } </pre> <p>This example shows a user-written routine that parses a file of text and returns a pointer to the parse tree. (All error-handling code is omitted from this example to keep it simple.) We assume the existence of some kind of tokenizer which is created using TokenizerCreate() on line 8 and deleted by TokenizerFree() on line 16. The GetNextToken() function on line 11 retrieves the next token from the input file and puts its type in the integer variable hTokenId. The sToken variable is assumed to be some kind of structure that contains details about each token, such as its complete text, what line it occurs on, etc.</p> <p>This example also assumes the existence of a structure of type ParserState that holds state information about a particular parse. An instance of such a structure is created on line 6 and initialized on line 10. A pointer to this structure is passed into the Parse() routine as the optional 4th argument. The action routine specified by the grammar for the parser can use the ParserState structure to hold whatever information is useful and appropriate. In the example, we note that the treeRoot field of the ParserState structure is left pointing to the root of the parse tree.</p> <p>The core of this example as it relates to Lemon is as follows:</p> <pre> ParseFile(){ pParser = ParseAlloc( malloc ); while( GetNextToken(pTokenizer,&hTokenId, &sToken) ){ Parse(pParser, hTokenId, sToken); } Parse(pParser, 0, sToken); ParseFree(pParser, free ); } </pre> <p>Basically, what a program has to do to use a Lemon-generated parser is first create the parser, then send it lots of tokens obtained by tokenizing an input source. When the end of input is reached, the Parse() routine should be called one last time with a token type of 0. This step is necessary to inform the parser that the end of input has been reached. Finally, we reclaim memory used by the parser by calling ParseFree().</p> <p>There is one other interface routine that should be mentioned before we move on. The ParseTrace() function can be used to generate debugging output from the parser. A prototype for this routine is as follows:</p> <pre> ParseTrace(FILE *stream, char *zPrefix); </pre> <p>After this routine is called, a short (one-line) message is written to the designated output stream every time the parser changes states or calls an action routine. Each such message is prefaced using the text given by zPrefix. This debugging output can be turned off by calling ParseTrace() again with a first argument of NULL (0).</p> <a id="onstack"></a> <h4>3.2.1 Allocating The Parse Object On Stack</h4> <p>If all calls to the Parse() interface are made from within <a href="#pcode"><tt>%code</tt> directives</a>, then the parse object can be allocated from the stack rather than from the heap. These are the steps: <ul> <li> Declare a local variable of type "yyParser" <li> Initialize the variable using ParseInit() <li> Pass a pointer to the variable in calls ot Parse() <li> Deallocate substructure in the parse variable using ParseFinalize(). </ul> <p>The following code illustrates how this is done: <pre> ParseFile(){ yyParser x; ParseInit( &x ); while( GetNextToken(pTokenizer,&hTokenId, &sToken) ){ Parse(&x, hTokenId, sToken); } Parse(&x, 0, sToken); ParseFinalize( &x ); } </pre> <a id="ifsum"></a> <h4>3.2.2 Interface Summary</h4> <p>Here is a quick overview of the C-language interface to a Lemon-generated parser:</p> <blockquote><pre> void *ParseAlloc( (void*(*malloc)(size_t) ); void ParseFree(void *pParser, (void(*free)(void*) ); void Parse(void *pParser, int tokenCode, ParseTOKENTYPE token, ...); void ParseTrace(FILE *stream, char *zPrefix); </pre></blockquote> <p>Notes:</p> <ul> <li> Use the <a href="#pname"><tt>%name</tt> directive</a> to change the "Parse" prefix names of the procedures in the interface. <li> Use the <a href="#token_type"><tt>%token_type</tt> directive</a> to define the "ParseTOKENTYPE" type. <li> Use the <a href="#extraarg"><tt>%extra_argument</tt> directive</a> to specify the type and name of the 4th parameter to the Parse() function. </ul> <a id="yaccdiff"></a> <h3>3.3 Differences With YACC and BISON</h3> <p>Programmers who have previously used the yacc or bison parser generator will notice several important differences between yacc and/or bison and Lemon.</p> <ul> <li>In yacc and bison, the parser calls the tokenizer. In Lemon, the tokenizer calls the parser. <li>Lemon uses no global variables. Yacc and bison use global variables to pass information between the tokenizer and parser. <li>Lemon allows multiple parsers to be running simultaneously. Yacc and bison do not. </ul> <p>These differences may cause some initial confusion for programmers with prior yacc and bison experience. But after years of experience using Lemon, I firmly believe that the Lemon way of doing things is better.</p> <p><i>Updated as of 2016-02-16:</i> The text above was written in the 1990s. We are told that Bison has lately been enhanced to support the tokenizer-calls-parser paradigm used by Lemon, eliminating the need for global variables.</p> <a id="build"><a> <h3>3.4 Building The "lemon" or "lemon.exe" Executable</h3> <p>The "lemon" or "lemon.exe" program is built from a single file of C-code named "<a href="https://sqlite.org/src/tool/lemon.c">lemon.c</a>". The Lemon source code is generic C89 code that uses no unusual or non-standard libraries. Any reasonable C compiler should suffice to compile the lemon program. A command-line like the following will usually work:</p> <blockquote><pre> cc -o lemon lemon.c </pre></blockquote <p>On Windows machines with Visual C++ installed, bring up a "VS20<i>NN</i> x64 Native Tools Command Prompt" window and enter: <blockquote><pre> cl lemon.c </pre></blockquote> <p>Compiling Lemon really is that simple. Additional compiler options such as "-O2" or "-g" or "-Wall" can be added if desired, but they are not necessary.</p> <a id="syntax"></a> <h2>4.0 Input File Syntax</h2> <p>The main purpose of the grammar specification file for Lemon is to define the grammar for the parser. But the input file also specifies additional information Lemon requires to do its job. Most of the work in using Lemon is in writing an appropriate grammar file.</p> <p>The grammar file for Lemon is, for the most part, a free format. It does not have sections or divisions like yacc or bison. Any declaration can occur at any point in the file. Lemon ignores whitespace (except where it is needed to separate tokens), and it honors the same commenting conventions as C and C++.</p> <a id="tnt"></a> <h3>4.1 Terminals and Nonterminals</h3> <p>A terminal symbol (token) is any string of alphanumeric and/or underscore characters that begins with an uppercase letter. A terminal can contain lowercase letters after the first character, but the usual convention is to make terminals all uppercase. A nonterminal, on the other hand, is any string of alphanumeric |
︙ | ︙ | |||
330 331 332 333 334 335 336 | <p>Yacc and bison allow terminal symbols to have either alphanumeric names or to be individual characters included in single quotes, like this: ')' or '$'. Lemon does not allow this alternative form for terminal symbols. With Lemon, all symbols, terminals and nonterminals, must have alphanumeric names.</p> | > | | < | < | | | | | | | | | | | | | | | | | | | | | | | | | > | | > > > | | | > > > | | | | | | | | | | | | 454 455 456 457 458 459 460 461 462 463 464 465 466 467 468 469 470 471 472 473 474 475 476 477 478 479 480 481 482 483 484 485 486 487 488 489 490 491 492 493 494 495 496 497 498 499 500 501 502 503 504 505 506 507 508 509 510 511 512 513 514 515 516 517 518 519 520 521 522 523 524 525 526 527 528 529 530 531 532 533 534 535 536 537 538 539 540 541 542 543 544 545 546 547 548 549 550 551 552 553 554 555 556 557 558 559 560 561 562 563 564 565 566 567 568 569 570 571 572 573 574 575 576 577 578 579 580 581 582 583 584 585 586 587 588 589 590 591 592 593 594 595 596 597 598 599 600 601 602 603 604 605 606 607 608 609 610 611 612 613 614 615 616 617 618 619 620 621 622 623 624 625 626 627 628 629 630 631 632 633 634 635 636 637 638 639 640 641 642 643 644 645 646 647 648 649 650 651 652 653 654 655 656 657 658 659 660 661 662 663 664 665 666 667 668 669 670 671 672 673 674 675 676 677 678 679 680 681 682 683 684 685 686 687 688 689 690 691 692 693 694 695 696 697 698 699 700 701 702 703 704 705 706 707 708 709 710 711 712 713 714 715 716 717 718 719 720 721 722 723 724 725 726 727 728 729 730 731 732 733 734 735 736 737 738 739 740 741 742 743 744 745 746 747 748 749 750 751 752 753 754 755 756 757 758 759 760 761 762 763 764 765 766 767 768 769 770 771 772 773 774 775 776 777 778 779 780 781 | <p>Yacc and bison allow terminal symbols to have either alphanumeric names or to be individual characters included in single quotes, like this: ')' or '$'. Lemon does not allow this alternative form for terminal symbols. With Lemon, all symbols, terminals and nonterminals, must have alphanumeric names.</p> <a id="rules"></a> <h3>4.2 Grammar Rules</h3> <p>The main component of a Lemon grammar file is a sequence of grammar rules. Each grammar rule consists of a nonterminal symbol followed by the special symbol "::=" and then a list of terminals and/or nonterminals. The rule is terminated by a period. The list of terminals and nonterminals on the right-hand side of the rule can be empty. Rules can occur in any order, except that the left-hand side of the first rule is assumed to be the start symbol for the grammar (unless specified otherwise using the <tt><a href='#start_symbol'>%start_symbol</a></tt> directive described below.) A typical sequence of grammar rules might look something like this:</p> <pre> expr ::= expr PLUS expr. expr ::= expr TIMES expr. expr ::= LPAREN expr RPAREN. expr ::= VALUE. </pre> <p>There is one non-terminal in this example, "expr", and five terminal symbols or tokens: "PLUS", "TIMES", "LPAREN", "RPAREN" and "VALUE".</p> <p>Like yacc and bison, Lemon allows the grammar to specify a block of C code that will be executed whenever a grammar rule is reduced by the parser. In Lemon, this action is specified by putting the C code (contained within curly braces <tt>{...}</tt>) immediately after the period that closes the rule. For example:</p> <pre> expr ::= expr PLUS expr. { printf("Doing an addition...\n"); } </pre> <p>In order to be useful, grammar actions must normally be linked to their associated grammar rules. In yacc and bison, this is accomplished by embedding a "$$" in the action to stand for the value of the left-hand side of the rule and symbols "$1", "$2", and so forth to stand for the value of the terminal or nonterminal at position 1, 2 and so forth on the right-hand side of the rule. This idea is very powerful, but it is also very error-prone. The single most common source of errors in a yacc or bison grammar is to miscount the number of symbols on the right-hand side of a grammar rule and say "$7" when you really mean "$8".</p> <p>Lemon avoids the need to count grammar symbols by assigning symbolic names to each symbol in a grammar rule and then using those symbolic names in the action. In yacc or bison, one would write this:</p> <pre> expr -> expr PLUS expr { $$ = $1 + $3; }; </pre> <p>But in Lemon, the same rule becomes the following:</p> <pre> expr(A) ::= expr(B) PLUS expr(C). { A = B+C; } </pre> <p>In the Lemon rule, any symbol in parentheses after a grammar rule symbol becomes a place holder for that symbol in the grammar rule. This place holder can then be used in the associated C action to stand for the value of that symbol.</p> <p>The Lemon notation for linking a grammar rule with its reduce action is superior to yacc/bison on several counts. First, as mentioned above, the Lemon method avoids the need to count grammar symbols. Secondly, if a terminal or nonterminal in a Lemon grammar rule includes a linking symbol in parentheses but that linking symbol is not actually used in the reduce action, then an error message is generated. For example, the rule</p> <pre> expr(A) ::= expr(B) PLUS expr(C). { A = B; } </pre> <p>will generate an error because the linking symbol "C" is used in the grammar rule but not in the reduce action.</p> <p>The Lemon notation for linking grammar rules to reduce actions also facilitates the use of destructors for reclaiming memory allocated by the values of terminals and nonterminals on the right-hand side of a rule.</p> <a id='precrules'></a> <h3>4.3 Precedence Rules</h3> <p>Lemon resolves parsing ambiguities in exactly the same way as yacc and bison. A shift-reduce conflict is resolved in favor of the shift, and a reduce-reduce conflict is resolved by reducing whichever rule comes first in the grammar file.</p> <p>Just like in yacc and bison, Lemon allows a measure of control over the resolution of parsing conflicts using precedence rules. A precedence value can be assigned to any terminal symbol using the <tt><a href='#pleft'>%left</a></tt>, <tt><a href='#pright'>%right</a></tt> or <tt><a href='#pnonassoc'>%nonassoc</a></tt> directives. Terminal symbols mentioned in earlier directives have a lower precedence than terminal symbols mentioned in later directives. For example:</p> <pre> %left AND. %left OR. %nonassoc EQ NE GT GE LT LE. %left PLUS MINUS. %left TIMES DIVIDE MOD. %right EXP NOT. </pre> <p>In the preceding sequence of directives, the AND operator is defined to have the lowest precedence. The OR operator is one precedence level higher. And so forth. Hence, the grammar would attempt to group the ambiguous expression</p> <pre> a AND b OR c </pre> <p>like this</p> <pre> a AND (b OR c). </pre> <p>The associativity (left, right or nonassoc) is used to determine the grouping when the precedence is the same. AND is left-associative in our example, so</p> <pre> a AND b AND c </pre> <p>is parsed like this</p> <pre> (a AND b) AND c. </pre> <p>The EXP operator is right-associative, though, so</p> <pre> a EXP b EXP c </pre> <p>is parsed like this</p> <pre> a EXP (b EXP c). </pre> <p>The nonassoc precedence is used for non-associative operators. So</p> <pre> a EQ b EQ c </pre> <p>is an error.</p> <p>The precedence of non-terminals is transferred to rules as follows: The precedence of a grammar rule is equal to the precedence of the left-most terminal symbol in the rule for which a precedence is defined. This is normally what you want, but in those cases where you want the precedence of a grammar rule to be something different, you can specify an alternative precedence symbol by putting the symbol in square braces after the period at the end of the rule and before any C-code. For example:</p> <pre> expr = MINUS expr. [NOT] </pre> <p>This rule has a precedence equal to that of the NOT symbol, not the MINUS symbol as would have been the case by default.</p> <p>With the knowledge of how precedence is assigned to terminal symbols and individual grammar rules, we can now explain precisely how parsing conflicts are resolved in Lemon. Shift-reduce conflicts are resolved as follows:</p> <ul> <li> If either the token to be shifted or the rule to be reduced lacks precedence information, then resolve in favor of the shift, but report a parsing conflict. <li> If the precedence of the token to be shifted is greater than the precedence of the rule to reduce, then resolve in favor of the shift. No parsing conflict is reported. <li> If the precedence of the token to be shifted is less than the precedence of the rule to reduce, then resolve in favor of the reduce action. No parsing conflict is reported. <li> If the precedences are the same and the shift token is right-associative, then resolve in favor of the shift. No parsing conflict is reported. <li> If the precedences are the same and the shift token is left-associative, then resolve in favor of the reduce. No parsing conflict is reported. <li> Otherwise, resolve the conflict by doing the shift, and report a parsing conflict. </ul> <p>Reduce-reduce conflicts are resolved this way:</p> <ul> <li> If either reduce rule lacks precedence information, then resolve in favor of the rule that appears first in the grammar, and report a parsing conflict. <li> If both rules have precedence and the precedence is different, then resolve the dispute in favor of the rule with the highest precedence, and do not report a conflict. <li> Otherwise, resolve the conflict by reducing by the rule that appears first in the grammar, and report a parsing conflict. </ul> <a id="special"></a> <h3>4.4 Special Directives</h3> <p>The input grammar to Lemon consists of grammar rules and special directives. We've described all the grammar rules, so now we'll talk about the special directives.</p> <p>Directives in Lemon can occur in any order. You can put them before the grammar rules, or after the grammar rules, or in the midst of the grammar rules. It doesn't matter. The relative order of directives used to assign precedence to terminals is important, but other than that, the order of directives in Lemon is arbitrary.</p> <p>Lemon supports the following special directives:</p> <ul> <li><tt><a href='#pcode'>%code</a></tt> <li><tt><a href='#default_destructor'>%default_destructor</a></tt> <li><tt><a href='#default_type'>%default_type</a></tt> <li><tt><a href='#destructor'>%destructor</a></tt> <li><tt><a href='#pifdef'>%else</a></tt> <li><tt><a href='#pifdef'>%endif</a></tt> <li><tt><a href='#extraarg'>%extra_argument</a></tt> <li><tt><a href='#pfallback'>%fallback</a></tt> <li><tt><a href='#pifdef'>%if</a></tt> <li><tt><a href='#pifdef'>%ifdef</a></tt> <li><tt><a href='#pifdef'>%ifndef</a></tt> <li><tt><a href='#pinclude'>%include</a></tt> <li><tt><a href='#pleft'>%left</a></tt> <li><tt><a href='#pname'>%name</a></tt> <li><tt><a href='#pnonassoc'>%nonassoc</a></tt> <li><tt><a href='#parse_accept'>%parse_accept</a></tt> <li><tt><a href='#parse_failure'>%parse_failure</a></tt> <li><tt><a href='#pright'>%right</a></tt> <li><tt><a href='#stack_overflow'>%stack_overflow</a></tt> <li><tt><a href='#stack_size'>%stack_size</a></tt> <li><tt><a href='#start_symbol'>%start_symbol</a></tt> <li><tt><a href='#syntax_error'>%syntax_error</a></tt> <li><tt><a href='#token'>%token</a></tt> <li><tt><a href='#token_class'>%token_class</a></tt> <li><tt><a href='#token_destructor'>%token_destructor</a></tt> <li><tt><a href='#token_prefix'>%token_prefix</a></tt> <li><tt><a href='#token_type'>%token_type</a></tt> <li><tt><a href='#ptype'>%type</a></tt> <li><tt><a href='#pwildcard'>%wildcard</a></tt> </ul> <p>Each of these directives will be described separately in the following sections:</p> <a id='pcode'></a> <h4>4.4.1 The <tt>%code</tt> directive</h4> <p>The <tt>%code</tt> directive is used to specify additional C code that is added to the end of the main output file. This is similar to the <tt><a href='#pinclude'>%include</a></tt> directive except that <tt>%include</tt> is inserted at the beginning of the main output file.</p> <p><tt>%code</tt> is typically used to include some action routines or perhaps a tokenizer or even the "main()" function as part of the output file.</p> <p>There can be multiple <tt>%code</tt> directives. The arguments of all <tt>%code</tt> directives are concatenated.</p> <a id='default_destructor'></a> <h4>4.4.2 The <tt>%default_destructor</tt> directive</h4> <p>The <tt>%default_destructor</tt> directive specifies a destructor to use for non-terminals that do not have their own destructor specified by a separate <tt>%destructor</tt> directive. See the documentation on the <tt><a href='#destructor'>%destructor</a></tt> directive below for additional information.</p> <p>In some grammars, many different non-terminal symbols have the same data type and hence the same destructor. This directive is a convenient way to specify the same destructor for all those non-terminals using a single statement.</p> <a id='default_type'></a> <h4>4.4.3 The <tt>%default_type</tt> directive</h4> <p>The <tt>%default_type</tt> directive specifies the data type of non-terminal symbols that do not have their own data type defined using a separate <tt><a href='#ptype'>%type</a></tt> directive.</p> <a id='destructor'></a> <h4>4.4.4 The <tt>%destructor</tt> directive</h4> <p>The <tt>%destructor</tt> directive is used to specify a destructor for a non-terminal symbol. (See also the <tt><a href='#token_destructor'>%token_destructor</a></tt> directive which is used to specify a destructor for terminal symbols.)</p> <p>A non-terminal's destructor is called to dispose of the non-terminal's value whenever the non-terminal is popped from the stack. This includes all of the following circumstances:</p> <ul> <li> When a rule reduces and the value of a non-terminal on the right-hand side is not linked to C code. <li> When the stack is popped during error processing. <li> When the ParseFree() function runs. </ul> <p>The destructor can do whatever it wants with the value of the non-terminal, but its design is to deallocate memory or other resources held by that non-terminal.</p> <p>Consider an example:</p> <pre> %type nt {void*} %destructor nt { free($$); } nt(A) ::= ID NUM. { A = malloc( 100 ); } </pre> <p>This example is a bit contrived, but it serves to illustrate how destructors work. The example shows a non-terminal named "nt" that holds values of type "void*". When the rule for an "nt" reduces, it sets the value of the non-terminal to space obtained from malloc(). Later, when the nt non-terminal is popped from the stack, the destructor will fire and call free() on this malloced space, thus avoiding a memory leak. (Note that the symbol "$$" in the destructor code is replaced |
︙ | ︙ | |||
660 661 662 663 664 665 666 | larger structure, and we don't want to destroy it, which is why the destructor is not called in this circumstance.</p> <p>Destructors help avoid memory leaks by automatically freeing allocated objects when they go out of scope. To do the same using yacc or bison is much more difficult.</p> | | | | | | | | | | | | | | | | | | | | | > | | | > > > > > > > > > > > | | < | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | > > > > > > > > > > > > > > > > > > > > > > > | | | | | | | | | | | | | | | | | | | | | | | 790 791 792 793 794 795 796 797 798 799 800 801 802 803 804 805 806 807 808 809 810 811 812 813 814 815 816 817 818 819 820 821 822 823 824 825 826 827 828 829 830 831 832 833 834 835 836 837 838 839 840 841 842 843 844 845 846 847 848 849 850 851 852 853 854 855 856 857 858 859 860 861 862 863 864 865 866 867 868 869 870 871 872 873 874 875 876 877 878 879 880 881 882 883 884 885 886 887 888 889 890 891 892 893 894 895 896 897 898 899 900 901 902 903 904 905 906 907 908 909 910 911 912 913 914 915 916 917 918 919 920 921 922 923 924 925 926 927 928 929 930 931 932 933 934 935 936 937 938 939 940 941 942 943 944 945 946 947 948 949 950 951 952 953 954 955 956 957 958 959 960 961 962 963 964 965 966 967 968 969 970 971 972 973 974 975 976 977 978 979 980 981 982 983 984 985 986 987 988 989 990 991 992 993 994 995 996 997 998 999 1000 1001 1002 1003 1004 1005 1006 1007 1008 1009 1010 1011 1012 1013 1014 1015 1016 1017 1018 1019 1020 1021 1022 1023 1024 1025 1026 1027 1028 1029 1030 1031 1032 1033 1034 1035 1036 1037 1038 1039 1040 1041 1042 1043 1044 1045 1046 1047 1048 1049 1050 1051 1052 1053 1054 1055 1056 1057 1058 1059 1060 1061 1062 1063 1064 1065 1066 1067 1068 1069 1070 1071 1072 1073 1074 1075 1076 1077 1078 1079 1080 1081 1082 1083 1084 1085 1086 1087 1088 1089 1090 1091 1092 1093 1094 1095 1096 1097 1098 1099 1100 1101 1102 1103 1104 1105 1106 1107 1108 1109 1110 1111 1112 1113 1114 1115 1116 1117 1118 1119 1120 1121 1122 1123 1124 1125 1126 1127 1128 1129 1130 1131 1132 1133 1134 1135 1136 1137 1138 1139 1140 1141 1142 1143 1144 1145 1146 1147 1148 1149 1150 1151 1152 1153 1154 1155 1156 1157 1158 1159 1160 1161 1162 1163 1164 1165 1166 1167 1168 1169 1170 1171 1172 1173 1174 1175 1176 1177 1178 1179 1180 1181 1182 1183 1184 1185 1186 1187 1188 1189 1190 1191 1192 1193 1194 1195 1196 1197 1198 1199 1200 1201 1202 1203 1204 1205 1206 1207 1208 1209 1210 1211 | larger structure, and we don't want to destroy it, which is why the destructor is not called in this circumstance.</p> <p>Destructors help avoid memory leaks by automatically freeing allocated objects when they go out of scope. To do the same using yacc or bison is much more difficult.</p> <a id='extraarg'></a> <h4>4.4.5 The <tt>%extra_argument</tt> directive</h4> <p>The <tt>%extra_argument</tt> directive instructs Lemon to add a 4th parameter to the parameter list of the Parse() function it generates. Lemon doesn't do anything itself with this extra argument, but it does make the argument available to C-code action routines, destructors, and so forth. For example, if the grammar file contains:</p> <pre> %extra_argument { MyStruct *pAbc } </pre> <p>Then the Parse() function generated will have an 4th parameter of type "MyStruct*" and all action routines will have access to a variable named "pAbc" that is the value of the 4th parameter in the most recent call to Parse().</p> <p>The <tt>%extra_context</tt> directive works the same except that it is passed in on the ParseAlloc() or ParseInit() routines instead of on Parse().</p> <a id='extractx'></a> <h4>4.4.6 The <tt>%extra_context</tt> directive</h4> <p>The <tt>%extra_context</tt> directive instructs Lemon to add a 2nd parameter to the parameter list of the ParseAlloc() and ParseInit() functions. Lemon doesn't do anything itself with these extra argument, but it does store the value make it available to C-code action routines, destructors, and so forth. For example, if the grammar file contains:</p> <pre> %extra_context { MyStruct *pAbc } </pre> <p>Then the ParseAlloc() and ParseInit() functions will have an 2nd parameter of type "MyStruct*" and all action routines will have access to a variable named "pAbc" that is the value of that 2nd parameter.</p> <p>The <tt>%extra_argument</tt> directive works the same except that it is passed in on the Parse() routine instead of on ParseAlloc()/ParseInit().</p> <a id='pfallback'></a> <h4>4.4.7 The <tt>%fallback</tt> directive</h4> <p>The <tt>%fallback</tt> directive specifies an alternative meaning for one or more tokens. The alternative meaning is tried if the original token would have generated a syntax error.</p> <p>The <tt>%fallback</tt> directive was added to support robust parsing of SQL syntax in <a href='https://www.sqlite.org/'>SQLite</a>. The SQL language contains a large assortment of keywords, each of which appears as a different token to the language parser. SQL contains so many keywords that it can be difficult for programmers to keep up with them all. Programmers will, therefore, sometimes mistakenly use an obscure language keyword for an identifier. The <tt>%fallback</tt> directive provides a mechanism to tell the parser: "If you are unable to parse this keyword, try treating it as an identifier instead."</p> <p>The syntax of <tt>%fallback</tt> is as follows:</p> <blockquote> <tt>%fallback</tt> <i>ID</i> <i>TOKEN...</i> <b>.</b> </blockquote></p> <p>In words, the <tt>%fallback</tt> directive is followed by a list of token names terminated by a period. The first token name is the fallback token — the token to which all the other tokens fall back to. The second and subsequent arguments are tokens which fall back to the token identified by the first argument.</p> <a id='pifdef'></a> <h4>4.4.8 The <tt>%if</tt> directive and its friends</h4> <p>The <tt>%if</tt>, <tt>%ifdef</tt>, <tt>%ifndef</tt>, <tt>%else</tt>, and <tt>%endif</tt> directives are similar to #if, #ifdef, #ifndef, #else, and #endif in the C-preprocessor, just not as general. Each of these directives must begin at the left margin. No whitespace is allowed between the "%" and the directive name.</p> <p>Grammar text in between "<tt>%ifdef MACRO</tt>" and the next nested "<tt>%endif</tt>" is ignored unless the "-DMACRO" command-line option is used. Grammar text betwen "<tt>%ifndef MACRO</tt>" and the next nested "<tt>%endif</tt>" is included except when the "-DMACRO" command-line option is used.<p> <p>The text in between "<tt>%if</tt> <i>CONDITIONAL</i>" and its corresponding <tt>%endif</tt> is included only if <i>CONDITIONAL</i> is true. The CONDITION is one or more macro names, optionally connected using the "||" and "&&" binary operators, the "!" unary operator, and grouped using balanced parentheses. Each term is true if the corresponding macro exists, and false if it does not exist.</p> <p>An optional "<tt>%else</tt>" directive can occur anywhere in between a <tt>%ifdef</tt>, <tt>%ifndef</tt>, or <tt>%if</tt> directive and its corresponding <tt>%endif</tt>.</p> <p>Note that the argument to <tt>%ifdef</tt> and <tt>%ifndef</tt> is intended to be a single preprocessor symbol name, not a general expression. Use the "<tt>%if</tt>" directive for general expressions.</p> <a id='pinclude'></a> <h4>4.4.9 The <tt>%include</tt> directive</h4> <p>The <tt>%include</tt> directive specifies C code that is included at the top of the generated parser. You can include any text you want — the Lemon parser generator copies it blindly. If you have multiple <tt>%include</tt> directives in your grammar file, their values are concatenated so that all <tt>%include</tt> code ultimately appears near the top of the generated parser, in the same order as it appeared in the grammar.</p> <p>The <tt>%include</tt> directive is very handy for getting some extra #include preprocessor statements at the beginning of the generated parser. For example:</p> <pre> %include {#include <unistd.h>} </pre> <p>This might be needed, for example, if some of the C actions in the grammar call functions that are prototyped in unistd.h.</p> <p>Use the <tt><a href="#pcode">%code</a></tt> directive to add code to the end of the generated parser.</p> <a id='pleft'></a> <h4>4.4.10 The <tt>%left</tt> directive</h4> The <tt>%left</tt> directive is used (along with the <tt><a href='#pright'>%right</a></tt> and <tt><a href='#pnonassoc'>%nonassoc</a></tt> directives) to declare precedences of terminal symbols. Every terminal symbol whose name appears after a <tt>%left</tt> directive but before the next period (".") is given the same left-associative precedence value. Subsequent <tt>%left</tt> directives have higher precedence. For example:</p> <pre> %left AND. %left OR. %nonassoc EQ NE GT GE LT LE. %left PLUS MINUS. %left TIMES DIVIDE MOD. %right EXP NOT. </pre> <p>Note the period that terminates each <tt>%left</tt>, <tt>%right</tt> or <tt>%nonassoc</tt> directive.</p> <p>LALR(1) grammars can get into a situation where they require a large amount of stack space if you make heavy use or right-associative operators. For this reason, it is recommended that you use <tt>%left</tt> rather than <tt>%right</tt> whenever possible.</p> <a id='pname'></a> <h4>4.4.11 The <tt>%name</tt> directive</h4> <p>By default, the functions generated by Lemon all begin with the five-character string "Parse". You can change this string to something different using the <tt>%name</tt> directive. For instance:</p> <pre> %name Abcde </pre> <p>Putting this directive in the grammar file will cause Lemon to generate functions named</p> <ul> <li> AbcdeAlloc(), <li> AbcdeFree(), <li> AbcdeTrace(), and <li> Abcde(). </ul> </p>The <tt>%name</tt> directive allows you to generate two or more different parsers and link them all into the same executable.</p> <a id='pnonassoc'></a> <h4>4.4.12 The <tt>%nonassoc</tt> directive</h4> <p>This directive is used to assign non-associative precedence to one or more terminal symbols. See the section on <a href='#precrules'>precedence rules</a> or on the <tt><a href='#pleft'>%left</a></tt> directive for additional information.</p> <a id='parse_accept'></a> <h4>4.4.13 The <tt>%parse_accept</tt> directive</h4> <p>The <tt>%parse_accept</tt> directive specifies a block of C code that is executed whenever the parser accepts its input string. To "accept" an input string means that the parser was able to process all tokens without error.</p> <p>For example:</p> <pre> %parse_accept { printf("parsing complete!\n"); } </pre> <a id='parse_failure'></a> <h4>4.4.14 The <tt>%parse_failure</tt> directive</h4> <p>The <tt>%parse_failure</tt> directive specifies a block of C code that is executed whenever the parser fails complete. This code is not executed until the parser has tried and failed to resolve an input error using is usual error recovery strategy. The routine is only invoked when parsing is unable to continue.</p> <pre> %parse_failure { fprintf(stderr,"Giving up. Parser is hopelessly lost...\n"); } </pre> <a id='pright'></a> <h4>4.4.15 The <tt>%right</tt> directive</h4> <p>This directive is used to assign right-associative precedence to one or more terminal symbols. See the section on <a href='#precrules'>precedence rules</a> or on the <a href='#pleft'>%left</a> directive for additional information.</p> <a id='stack_overflow'></a> <h4>4.4.16 The <tt>%stack_overflow</tt> directive</h4> <p>The <tt>%stack_overflow</tt> directive specifies a block of C code that is executed if the parser's internal stack ever overflows. Typically this just prints an error message. After a stack overflow, the parser will be unable to continue and must be reset.</p> <pre> %stack_overflow { fprintf(stderr,"Giving up. Parser stack overflow\n"); } </pre> <p>You can help prevent parser stack overflows by avoiding the use of right recursion and right-precedence operators in your grammar. Use left recursion and and left-precedence operators instead to encourage rules to reduce sooner and keep the stack size down. For example, do rules like this:</p> <pre> list ::= list element. // left-recursion. Good! list ::= . </pre> <p>Not like this:</p> <pre> list ::= element list. // right-recursion. Bad! list ::= . </pre> <a id='stack_size'></a> <h4>4.4.17 The <tt>%stack_size</tt> directive</h4> <p>If stack overflow is a problem and you can't resolve the trouble by using left-recursion, then you might want to increase the size of the parser's stack using this directive. Put an positive integer after the <tt>%stack_size</tt> directive and Lemon will generate a parse with a stack of the requested size. The default value is 100.</p> <pre> %stack_size 2000 </pre> <a id='start_symbol'></a> <h4>4.4.18 The <tt>%start_symbol</tt> directive</h4> <p>By default, the start symbol for the grammar that Lemon generates is the first non-terminal that appears in the grammar file. But you can choose a different start symbol using the <tt>%start_symbol</tt> directive.</p> <pre> %start_symbol prog </pre> <a id='syntax_error'></a> <h4>4.4.19 The <tt>%syntax_error</tt> directive</h4> <p>See <a href='#errors'>Error Processing</a>.</p> <a id='token'></a> <h4>4.4.20 The <tt>%token</tt> directive</h4> <p>Tokens are normally created automatically, the first time they are used. Any identifier that begins with an upper-case letter is a token. <p>Sometimes it is useful to declare tokens in advance, however. The integer values assigned to each token determined by the order in which the tokens are seen. So by declaring tokens in advance, it is possible to cause some tokens to have low-numbered values, which might be desirable in some grammers, or to have sequential values assigned to a sequence of related tokens. For this reason, the %token directive is provided to declare tokens in advance. The syntax is as follows: <blockquote> <tt>%token</tt> <i>TOKEN</i> <i>TOKEN...</i> <b>.</b> </blockquote></p> <p>The %token directive is followed by zero or more token symbols and terminated by a single ".". Each token named is created if it does not already exist. Tokens are created in order. <a id='token_class'></a> <h4>4.4.21 The <tt>%token_class</tt> directive</h4> <p>Undocumented. Appears to be related to the MULTITERMINAL concept. <a href='http://sqlite.org/src/fdiff?v1=796930d5fc2036c7&v2=624b24c5dc048e09&sbs=0'>Implementation</a>.</p> <a id='token_destructor'></a> <h4>4.4.22 The <tt>%token_destructor</tt> directive</h4> <p>The <tt>%destructor</tt> directive assigns a destructor to a non-terminal symbol. (See the description of the <tt><a href='%destructor'>%destructor</a></tt> directive above.) The <tt>%token_destructor</tt> directive does the same thing for all terminal symbols.</p> <p>Unlike non-terminal symbols, which may each have a different data type for their values, terminals all use the same data type (defined by the <tt><a href='#token_type'>%token_type</a></tt> directive) and so they use a common destructor. Other than that, the token destructor works just like the non-terminal destructors.</p> <a id='token_prefix'></a> <h4>4.4.23 The <tt>%token_prefix</tt> directive</h4> <p>Lemon generates #defines that assign small integer constants to each terminal symbol in the grammar. If desired, Lemon will add a prefix specified by this directive to each of the #defines it generates.</p> <p>So if the default output of Lemon looked like this:</p> <pre> #define AND 1 #define MINUS 2 #define OR 3 #define PLUS 4 </pre> <p>You can insert a statement into the grammar like this:</p> <pre> %token_prefix TOKEN_ </pre> <p>to cause Lemon to produce these symbols instead:</p> <pre> #define TOKEN_AND 1 #define TOKEN_MINUS 2 #define TOKEN_OR 3 #define TOKEN_PLUS 4 </pre> <a id='token_type'></a><a id='ptype'></a> <h4>4.4.24 The <tt>%token_type</tt> and <tt>%type</tt> directives</h4> <p>These directives are used to specify the data types for values on the parser's stack associated with terminal and non-terminal symbols. The values of all terminal symbols must be of the same type. This turns out to be the same data type as the 3rd parameter to the Parse() function generated by Lemon. Typically, you will make the value of a terminal symbol be a pointer to some kind of token structure. Like this:</p> <pre> %token_type {Token*} </pre> <p>If the data type of terminals is not specified, the default value is "void*".</p> <p>Non-terminal symbols can each have their own data types. Typically the data type of a non-terminal is a pointer to the root of a parse tree structure that contains all information about that non-terminal. For example:</p> <pre> %type expr {Expr*} </pre> <p>Each entry on the parser's stack is actually a union containing instances of all data types for every non-terminal and terminal symbol. Lemon will automatically use the correct element of this union depending on what the corresponding non-terminal or terminal symbol is. But the grammar designer should keep in mind that the size of the union will be the size of its largest element. So if you have a single non-terminal whose data type requires 1K of storage, then your 100 entry parser stack will require 100K of heap space. If you are willing and able to pay that price, fine. You just need to know.</p> <a id='pwildcard'></a> <h4>4.4.25 The <tt>%wildcard</tt> directive</h4> <p>The <tt>%wildcard</tt> directive is followed by a single token name and a period. This directive specifies that the identified token should match any input token.</p> <p>When the generated parser has the choice of matching an input against the wildcard token and some other token, the other token is always used. The wildcard token is only matched if there are no alternatives.</p> <a id='errors'></a> <h2>5.0 Error Processing</h2> <p>After extensive experimentation over several years, it has been discovered that the error recovery strategy used by yacc is about as good as it gets. And so that is what Lemon uses.</p> <p>When a Lemon-generated parser encounters a syntax error, it first invokes the code specified by the <tt>%syntax_error</tt> directive, if |
︙ | ︙ | |||
1055 1056 1057 1058 1059 1060 1061 1062 1063 1064 | <p>If the parser pops its stack until the stack is empty, and it still is unable to shift the error symbol, then the <tt><a href='#parse_failure'>%parse_failure</a></tt> routine is invoked and the parser resets itself to its start state, ready to begin parsing a new file. This is what will happen at the very first syntax error, of course, if there are no instances of the "error" non-terminal in your grammar.</p> </body> </html> | > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | 1219 1220 1221 1222 1223 1224 1225 1226 1227 1228 1229 1230 1231 1232 1233 1234 1235 1236 1237 1238 1239 1240 1241 1242 1243 1244 1245 1246 1247 1248 1249 1250 1251 1252 1253 1254 1255 1256 1257 1258 1259 1260 1261 1262 1263 1264 | <p>If the parser pops its stack until the stack is empty, and it still is unable to shift the error symbol, then the <tt><a href='#parse_failure'>%parse_failure</a></tt> routine is invoked and the parser resets itself to its start state, ready to begin parsing a new file. This is what will happen at the very first syntax error, of course, if there are no instances of the "error" non-terminal in your grammar.</p> <a id='history'></a> <h2>6.0 History of Lemon</h2> <p>Lemon was originally written by Richard Hipp sometime in the late 1980s on a Sun4 Workstation using K&R C. There was a companion LL(1) parser generator program named "Lime", the source code to which as been lost.</p> <p>The lemon.c source file was originally many separate files that were compiled together to generate the "lemon" executable. Sometime in the 1990s, the individual source code files were combined together into the current single large "lemon.c" source file. You can still see traces of original filenames in the code.</p> <p>Since 2001, Lemon has been part of the <a href="https://sqlite.org/">SQLite project</a> and the source code to Lemon has been managed as a part of the <a href="https://sqlite.org/src">SQLite source tree</a> in the following files:</p> <ul> <li> <a href="https://sqlite.org/src/file/tool/lemon.c">tool/lemon.c</a> <li> <a href="https://sqlite.org/src/file/tool/lempar.c">tool/lempar.c</a> <li> <a href="https://sqlite.org/src/file/doc/lemon.html">doc/lemon.html</a> </ul> <a id="copyright"></a> <h2>7.0 Copyright</h2> <p>All of the source code to Lemon, including the template parser file "lempar.c" and this documentation file ("lemon.html") are in the public domain. You can use the code for any purpose and without attribution.</p> <p>The code comes with no warranty. If it breaks, you get to keep both pieces.</p> </body> </html> |
Added doc/trusted-schema.md.
> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 | # The new-security-options branch ## The problem that the [new-security-options](/timeline?r=new-security-options) branch tries to solve An attacker might modify the schema of an SQLite database by adding structures that cause code to run when some other application opens and reads the database. For example, the attacker might replace a table definition with a view. Or the attacker might add triggers to tables or views, or add new CHECK constraints or generated columns or indexes with expressions in the index list or in the WHERE clause. If the added features invoke SQL functions or virtual tables with side effects, that might cause harm to the system if run by a high-privilege victim. Or, the added features might exfiltrate information if the database is read by a high-privilege victim. The changes in this branch strive to make it easier for high-privilege applications to safely read SQLite database files that might have been maliciously corrupted by an attacker. ## Overview of changes in [new-security-options](/timeline?r=new-security-options) The basic idea is to tag every SQL function and virtual table with one of three risk levels: 1. Innocuous 2. Normal 3. Direct-Only Innocuous functions/vtabs are safe and can be used at any time. Direct-only elements, in contrast, might have cause side-effects and should only be used from top-level SQL, not from within triggers or views nor in elements of the schema such as CHECK constraint, DEFAULT values, generated columns, index expressions, or in the WHERE clause of a partial index that are potentially under the control of an attacker. Normal elements behave like Innocuous if TRUSTED\_SCHEMA=on and behave like direct-only if TRUSTED\_SCHEMA=off. Application-defined functions and virtual tables go in as Normal unless the application takes deliberate steps to change the risk level. For backwards compatibility, the default is TRUSTED\_SCHEMA=on. Documentation will be updated to recommend applications turn TRUSTED\_SCHEMA to off. An innocuous function or virtual table is one that can only read content from the database file in which it resides, and can only alter the database in which it resides. Most SQL functions are innocuous. For example, there is no harm in an attacker running the abs() function. Direct-only elements that have side-effects that go outside the database file in which it lives, or return information from outside of the database file. Examples of direct-only elements include: 1. The fts3\_tokenizer() function 2. The writefile() function 3. The readfile() function 4. The zipvfs virtual table 5. The csv virtual table We do not want an attacker to be able to add these kinds of things to the database schema and possibly trick a high-privilege application from performing any of these actions. Therefore, functions and vtabs with side-effects are marked as Direct-Only. Legacy applications might add other risky functions or vtabs. Those will go in as "Normal" by default. For optimal security, we want those risky app-defined functions and vtabs to be direct-only, but making that the default might break some legacy applications. Hence, all app-defined functions and vtabs go in as Normal, but the application can switch them over to "Direct-Only" behavior using a single pragma. The restrictions on the use of functions and virtual tables do not apply to TEMP. A TEMP VIEW or a TEMP TRIGGER can use any valid SQL function or virtual table. The idea is that TEMP views and triggers must be directly created by the application and are thus under the control of the application. TEMP views and triggers cannot be created by an attacker who corrupts the schema of a persistent database file. Hence TEMP views and triggers are safe. ## Specific changes 1. New sqlite3\_db\_config() option SQLITE\_DBCONFIG\_TRUSTED\_SCHEMA for turning TRUSTED\_SCHEMA on and off. It defaults to ON. 2. Compile-time option -DSQLITE\_TRUSTED\_SCHEMA=0 causes the default TRUSTED\_SCHEMA setting to be off. 3. New pragma "PRAGMA trusted\_schema=(ON\|OFF);". This provides access to the TRUSTED_SCHEMA setting for application coded using scripting languages or other secondary languages where they are unable to make calls to sqlite3\_db\_config(). 4. New options for the "enc" parameter to sqlite3\_create\_function() and its kin: <ol type="a"> <li> _SQLITE\_INNOCUOUS_ → tags the new functions as Innocuous <li> _SQLITE\_DIRECTONLY_ → tags the new functions as Direct-Only </ol> 5. New options to sqlite3\_vtab\_config(): <ol type="a"> <li> _SQLITE\_VTAB\_INNOCUOUS_ → tags the vtab as Innocuous <li> _SQLITE\_VTAB\_DIRECTONLY_ → tags the vtab as Direct-Only </ol> 6. Change many of the functions and virtual tables in the SQLite source tree to use one of the tags above. 7. Enhanced PRAGMA function\_list and virtual-table "pragma\_function\_list" with additional columns. The columns now are: <ul> <li> _name_ → Name of the function <li> _builtin_ → 1 for built-in functions. 0 otherwise. <li> _type_ → 's'=Scalar, 'a'=Aggregate, 'w'=Window <li> _enc_ → 'utf8', 'utf16le', or 'utf16be' <li> _narg_ → number of argument <li> _flags_ → Bitmask of SQLITE\_INNOCUOUS, SQLITE\_DIRECTONLY, SQLITE\_DETERMINISTIC, SQLITE\_SUBTYPE, and SQLITE\_FUNC\_INTERNAL flags. </ul> <p>The last four columns are new. 8. The function\_list PRAGMA now also shows all entries for each function. So, for example, if a function can take either 2 or 3 arguments, there are separate rows for the 2-argument and 3-argument versions of the function. ## Additional Notes The function_list enhancements allow the application to query the set of SQL functions that meet various criteria. For example, to see all SQL functions that are never allowed to be used in the schema or in trigger or views: ~~~ SELECT DISTINCT name FROM pragma_function_list WHERE (flags & 0x80000)!=0 ORDER BY name; ~~~ Doing the same is not possible for virtual tables, as a virtual table might be Innocuous, Normal, or Direct-Only depending on the arguments passed into the xConnect method. |
Added doc/vdbesort-memory.md.
> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 | 20-11-2020 # Memory Allocation In vdbesort.c Memory allocation is slightly different depending on: * whether or not SQLITE_CONFIG_SMALL_MALLOC is set, and * whether or not worker threads are enabled. ## SQLITE_CONFIG_SMALL_MALLOC=0 Assuming SQLITE_CONFIG_SMALL_MALLOC is not set, keys passed to the sorter are added to an in-memory buffer. This buffer is grown using sqlite3Realloc() as required it reaches the size configured for the main pager cache using "PRAGMA cache_size". i.e. if the user has executed "PRAGMA main.cache_size = -2048", then this buffer is allowed to grow up to 2MB in size. Once the buffer has grown to its threshold, keys are sorted and written to a temp file. If worker threads are not enabled, this is the only significant allocation the sorter module makes. After keys are sorted and flushed out to the temp file, the buffer is reused to accumulate the next batch of keys. If worker threads are available, then the buffer is passed to a worker thread to sort and flush once it is full, and a new buffer allocated to allow the main thread to continue to accumulate keys. Buffers are reused once they have been flushed, so in this case at most (nWorker+1) buffers are allocated and used, where nWorker is the number of configured worker threads. There are no other significant users of heap memory in the sorter module. Once sorted buffers of keys have been flushed to disk, they are read back either by mapping the file (via sqlite3_file.xFetch()) or else read back in one page at a time. All buffers are allocated by the main thread. A sorter object is associated with a single database connection, to which it holds a pointer. ## SQLITE_CONFIG_SMALL_MALLOC=1 This case is similar to the above, except that instead of accumulating multiple keys in a single large buffer, sqlite3VdbeSorterWrite() stores keys in a regular heap-memory linked list (one allocation per element). List elements are freed as they are flushed to disk, either by the main thread or by a worker thread. Each time a key is added the sorter (and an allocation made), sqlite3HeapNearlyFull() is called. If it returns true, the current list of keys is flushed to a temporary file, even if it has not yet reached the size threshold. |
Added doc/wal-lock.md.
> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 | # Wal-Mode Blocking Locks On some Unix-like systems, SQLite may be configured to use POSIX blocking locks by: * building the library with SQLITE\_ENABLE\_SETLK\_TIMEOUT defined, and * configuring a timeout in ms using the sqlite3\_busy\_timeout() API. Blocking locks may be advantageous as (a) waiting database clients do not need to continuously poll the database lock, and (b) using blocking locks facilitates transfer of OS priority between processes when a high priority process is blocked by a lower priority one. Only read/write clients use blocking locks. Clients that have read-only access to the \*-shm file nevery use blocking locks. Threads or processes that access a single database at a time never deadlock as a result of blocking database locks. But it is of course possible for threads that lock multiple databases simultaneously to do so. In most cases the OS will detect the deadlock and return an error. ## Wal Recovery Wal database "recovery" is a process required when the number of connected database clients changes from zero to one. In this case, a client is considered to connect to the database when it first reads data from it. Before recovery commences, an exclusive WRITER lock is taken. Without blocking locks, if two clients attempt recovery simultaneously, one fails to obtain the WRITER lock and either invokes the busy-handler callback or returns SQLITE\_BUSY to the user. With blocking locks configured, the second client blocks on the WRITER lock. ## Database Readers Usually, read-only are not blocked by any other database clients, so they have no need of blocking locks. If a read-only transaction is being opened on a snapshot, the CHECKPOINTER lock is required briefly as part of opening the transaction (to check that a checkpointer is not currently overwriting the snapshot being opened). A blocking lock is used to obtain the CHECKPOINTER lock in this case. A snapshot opener may therefore block on and transfer priority to a checkpointer in some cases. ## Database Writers A database writer must obtain the exclusive WRITER lock. It uses a blocking lock to do so if any of the following are true: * the transaction is an implicit one consisting of a single DML or DDL statement, or * the transaction is opened using BEGIN IMMEDIATE or BEGIN EXCLUSIVE, or * the first SQL statement executed following the BEGIN command is a DML or DDL statement (not a read-only statement like a SELECT). In other words, in all cases except when an open read-transaction is upgraded to a write-transaction. In that case a non-blocking lock is used. ## Database Checkpointers Database checkpointers takes the following locks, in order: * The exclusive CHECKPOINTER lock. * The exclusive WRITER lock (FULL, RESTART and TRUNCATE only). * Exclusive lock on read-mark slots 1-N. These are immediately released after being taken. * Exclusive lock on read-mark 0. * Exclusive lock on read-mark slots 1-N again. These are immediately released after being taken (RESTART and TRUNCATE only). All of the above use blocking locks. ## Summary With blocking locks configured, the only cases in which clients should see an SQLITE\_BUSY error are: * if the OS does not grant a blocking lock before the configured timeout expires, and * when an open read-transaction is upgraded to a write-transaction. In all other cases the blocking locks implementation should prevent clients from having to handle SQLITE\_BUSY errors and facilitate appropriate transfer of priorities between competing clients. Clients that lock multiple databases simultaneously must be wary of deadlock. |
Added doc/wal2.md.
> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 | Wal2 Mode Notes =============== ## Activating/Deactivating Wal2 Mode "Wal2" mode is very similar to "wal" mode. To change a database to wal2 mode, use the command: > PRAGMA journal_mode = wal2; It is not possible to change a database directly from "wal" mode to "wal2" mode. Instead, it must first be changed to rollback mode. So, to change a wal mode database to wal2 mode, the following two commands may be used: > PRAGMA journal_mode = delete; PRAGMA journal_mode = wal2; A database in wal2 mode may only be accessed by versions of SQLite compiled from this branch. Attempting to use any other version of SQLite results in an SQLITE_NOTADB error. A wal2 mode database may be changed back to rollback mode (making it accessible by all versions of SQLite) using: > PRAGMA journal_mode = delete; ## The Advantage of Wal2 Mode In legacy wal mode, when a writer writes data to the database, it doesn't modify the database file directly. Instead, it appends new data to the "<database>-wal" file. Readers read data from both the original database file and the "<database>-wal" file. At some point, data is copied from the "<database>-wal" file into the database file, after which the wal file can be deleted or overwritten. Copying data from the wal file into the database file is called a "checkpoint", and may be done explictly (either by "PRAGMA wal_checkpoint" or sqlite3_wal_checkpoint_v2()), or automatically (by configuring "PRAGMA wal_autocheckpoint" - this is the default). Checkpointers do not block writers, and writers do not block checkpointers. However, if a writer writes to the database while a checkpoint is ongoing, then the new data is appended to the end of the wal file. This means that, even following the checkpoint, the wal file cannot be overwritten or deleted, and so all subsequent transactions must also be appended to the wal file. The work of the checkpointer is not wasted - SQLite remembers which parts of the wal file have already been copied into the db file so that the next checkpoint does not have to do so again - but it does mean that the wal file may grow indefinitely if the checkpointer never gets a chance to finish without a writer appending to the wal file. There are also circumstances in which long-running readers may prevent a checkpointer from checkpointing the entire wal file - also causing the wal file to grow indefinitely in a busy system. Wal2 mode does not have this problem. In wal2 mode, wal files do not grow indefinitely even if the checkpointer never has a chance to finish uninterrupted. In wal2 mode, the system uses two wal files instead of one. The files are named "<database>-wal" and "<database>-wal2", where "<database>" is of course the name of the database file. When data is written to the database, the writer begins by appending the new data to the first wal file. Once the first wal file has grown large enough, writers switch to appending data to the second wal file. At this point the first wal file can be checkpointed (after which it can be overwritten). Then, once the second wal file has grown large enough and the first wal file has been checkpointed, writers switch back to the first wal file. And so on. ## Application Programming From the point of view of the user, the main differences between wal and wal2 mode are to do with checkpointing: * In wal mode, a checkpoint may be attempted at any time. In wal2 mode, the checkpointer has to wait until writers have switched to the "other" wal file before a checkpoint can take place. * In wal mode, the wal-hook (callback registered using sqlite3_wal_hook()) is invoked after a transaction is committed with the total number of pages in the wal file as an argument. In wal2 mode, the argument is either the total number of uncheckpointed pages in both wal files, or - if the "other" wal file is empty or already checkpointed - 0. Clients are recommended to use the same strategies for checkpointing wal2 mode databases as for wal databases - by registering a wal-hook using sqlite3_wal_hook() and attempting a checkpoint when the parameter exceeds a certain threshold. However, it should be noted that although the wal-hook is invoked after each transaction is committed to disk and database locks released, it is still invoked from within the sqlite3_step() call used to execute the "COMMIT" command. In BEGIN CONCURRENT systems, where the "COMMIT" is often protected by an application mutex, this may reduce concurrency. In such systems, instead of executing a checkpoint from within the wal-hook, a thread might defer this action until after the application mutex has been released. |
Changes to ext/async/sqlite3async.c.
︙ | ︙ | |||
1700 1701 1702 1703 1704 1705 1706 | break; } va_end(ap); return rc; } #endif /* !defined(SQLITE_CORE) || defined(SQLITE_ENABLE_ASYNCIO) */ | < | 1700 1701 1702 1703 1704 1705 1706 | break; } va_end(ap); return rc; } #endif /* !defined(SQLITE_CORE) || defined(SQLITE_ENABLE_ASYNCIO) */ |
Changes to ext/async/sqlite3async.h.
︙ | ︙ | |||
216 217 218 219 220 221 222 | #define SQLITEASYNC_HALT_NOW 1 /* Halt as soon as possible */ #define SQLITEASYNC_HALT_IDLE 2 /* Halt when write-queue is empty */ #ifdef __cplusplus } /* End of the 'extern "C"' block */ #endif #endif /* ifndef __SQLITEASYNC_H_ */ | < | 216 217 218 219 220 221 222 | #define SQLITEASYNC_HALT_NOW 1 /* Halt as soon as possible */ #define SQLITEASYNC_HALT_IDLE 2 /* Halt when write-queue is empty */ #ifdef __cplusplus } /* End of the 'extern "C"' block */ #endif #endif /* ifndef __SQLITEASYNC_H_ */ |
Changes to ext/expert/expert1.test.
︙ | ︙ | |||
23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 | source $testdir/tester.tcl set testprefix expert1 if {[info commands sqlite3_expert_new]==""} { finish_test return } set CLI [test_binary_name sqlite3] set CMD [test_binary_name sqlite3_expert] proc squish {txt} { regsub -all {[[:space:]]+} $txt { } } proc do_setup_rec_test {tn setup sql res} { reset_db db eval $setup uplevel [list do_rec_test $tn $sql $res] } foreach {tn setup} { 1 { if {![file executable $CMD]} { continue } | > > > > | 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 | source $testdir/tester.tcl set testprefix expert1 if {[info commands sqlite3_expert_new]==""} { finish_test return } set CLI [test_binary_name sqlite3] set CMD [test_binary_name sqlite3_expert] proc squish {txt} { regsub -all {[[:space:]]+} $txt { } } proc do_setup_rec_test {tn setup sql res} { reset_db if {[info exists ::set_main_db_name]} { dbconfig_maindbname_icecube db } db eval $setup uplevel [list do_rec_test $tn $sql $res] } foreach {tn setup} { 1 { if {![file executable $CMD]} { continue } |
︙ | ︙ | |||
72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 | $expert destroy set tst [subst -nocommands {set {} [squish [join {$result}]]}] uplevel [list do_test $tn $tst [string trim [squish $res]]] } } 3 { if {![file executable $CLI]} { continue } proc do_rec_test {tn sql res} { set res [squish [string trim $res]] set tst [subst -nocommands { squish [string trim [exec $::CLI test.db ".expert" {$sql;}]] }] uplevel [list do_test $tn $tst $res] } } } { eval $setup do_setup_rec_test $tn.1 { CREATE TABLE t1(a, b, c) } { SELECT * FROM t1 } { (no new indexes) | > > > > | | | | | > | > | | | | | | | | | | | | > | > | | | | | | | | > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | | | | < | 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340 341 342 343 344 345 346 347 348 349 350 351 352 353 354 355 356 357 358 359 360 361 362 363 364 365 366 367 368 369 370 371 372 373 374 375 376 377 378 379 380 381 382 383 384 385 386 387 388 389 390 391 392 393 394 395 396 397 398 399 400 401 402 403 404 405 406 407 408 409 410 411 412 413 414 415 416 417 418 419 420 421 422 423 424 425 426 427 428 429 430 431 432 433 434 435 436 437 438 439 440 441 442 443 444 445 446 447 448 449 450 451 452 453 454 455 456 457 458 459 460 461 462 463 464 465 466 467 | $expert destroy set tst [subst -nocommands {set {} [squish [join {$result}]]}] uplevel [list do_test $tn $tst [string trim [squish $res]]] } } 3 { if {[info commands sqlite3_expert_new]==""} { continue } set ::set_main_db_name 1 } 4 { if {![file executable $CLI]} { continue } proc do_rec_test {tn sql res} { set res [squish [string trim $res]] set tst [subst -nocommands { squish [string trim [exec $::CLI test.db ".expert" {$sql;}]] }] uplevel [list do_test $tn $tst $res] } } } { eval $setup do_setup_rec_test $tn.1 { CREATE TABLE t1(a, b, c) } { SELECT * FROM t1 } { (no new indexes) SCAN t1 } do_setup_rec_test $tn.2 { CREATE TABLE t1(a, b, c); } { SELECT * FROM t1 WHERE b>?; } { CREATE INDEX t1_idx_00000062 ON t1(b); SEARCH t1 USING INDEX t1_idx_00000062 (b>?) } do_setup_rec_test $tn.3 { CREATE TABLE t1(a, b, c); } { SELECT * FROM t1 WHERE b COLLATE nocase BETWEEN ? AND ? } { CREATE INDEX t1_idx_3e094c27 ON t1(b COLLATE NOCASE); SEARCH t1 USING INDEX t1_idx_3e094c27 (b>? AND b<?) } do_setup_rec_test $tn.4 { CREATE TABLE t1(a, b, c); } { SELECT a FROM t1 ORDER BY b; } { CREATE INDEX t1_idx_00000062 ON t1(b); SCAN t1 USING INDEX t1_idx_00000062 } do_setup_rec_test $tn.5 { CREATE TABLE t1(a, b, c); } { SELECT a FROM t1 WHERE a=? ORDER BY b; } { CREATE INDEX t1_idx_000123a7 ON t1(a, b); SEARCH t1 USING COVERING INDEX t1_idx_000123a7 (a=?) } if 0 { do_setup_rec_test $tn.6 { CREATE TABLE t1(a, b, c); } { SELECT min(a) FROM t1 } { CREATE INDEX t1_idx_00000061 ON t1(a); SEARCH t1 USING COVERING INDEX t1_idx_00000061 } } do_setup_rec_test $tn.7 { CREATE TABLE t1(a, b, c); } { SELECT * FROM t1 ORDER BY a, b, c; } { CREATE INDEX t1_idx_033e95fe ON t1(a, b, c); SCAN t1 USING COVERING INDEX t1_idx_033e95fe } #do_setup_rec_test $tn.1.8 { # CREATE TABLE t1(a, b, c); #} { # SELECT * FROM t1 ORDER BY a ASC, b COLLATE nocase DESC, c ASC; #} { # CREATE INDEX t1_idx_5be6e222 ON t1(a, b COLLATE NOCASE DESC, c); # 0|0|0|SCAN t1 USING COVERING INDEX t1_idx_5be6e222 #} do_setup_rec_test $tn.8.1 { CREATE TABLE t1(a COLLATE NOCase, b, c); } { SELECT * FROM t1 WHERE a=? } { CREATE INDEX t1_idx_00000061 ON t1(a); SEARCH t1 USING INDEX t1_idx_00000061 (a=?) } do_setup_rec_test $tn.8.2 { CREATE TABLE t1(a, b COLLATE nocase, c); } { SELECT * FROM t1 ORDER BY a ASC, b DESC, c ASC; } { CREATE INDEX t1_idx_5cb97285 ON t1(a, b DESC, c); SCAN t1 USING COVERING INDEX t1_idx_5cb97285 } # Tables with names that require quotes. # do_setup_rec_test $tn.9.1 { CREATE TABLE "t t"(a, b, c); } { SELECT * FROM "t t" WHERE a=? } { CREATE INDEX "t t_idx_00000061" ON "t t"(a); SEARCH t t USING INDEX t t_idx_00000061 (a=?) } do_setup_rec_test $tn.9.2 { CREATE TABLE "t t"(a, b, c); } { SELECT * FROM "t t" WHERE b BETWEEN ? AND ? } { CREATE INDEX "t t_idx_00000062" ON "t t"(b); SEARCH t t USING INDEX t t_idx_00000062 (b>? AND b<?) } # Columns with names that require quotes. # do_setup_rec_test $tn.10.1 { CREATE TABLE t3(a, "b b", c); } { SELECT * FROM t3 WHERE "b b" = ? } { CREATE INDEX t3_idx_00050c52 ON t3('b b'); SEARCH t3 USING INDEX t3_idx_00050c52 (b b=?) } do_setup_rec_test $tn.10.2 { CREATE TABLE t3(a, "b b", c); } { SELECT * FROM t3 ORDER BY "b b" } { CREATE INDEX t3_idx_00050c52 ON t3('b b'); SCAN t3 USING INDEX t3_idx_00050c52 } # Transitive constraints # do_setup_rec_test $tn.11.1 { CREATE TABLE t5(a, b); CREATE TABLE t6(c, d); } { SELECT * FROM t5, t6 WHERE a=? AND b=c AND c=? } { CREATE INDEX t5_idx_000123a7 ON t5(a, b); CREATE INDEX t6_idx_00000063 ON t6(c); SEARCH t6 USING INDEX t6_idx_00000063 (c=?) SEARCH t5 USING COVERING INDEX t5_idx_000123a7 (a=? AND b=?) } # OR terms. # do_setup_rec_test $tn.12.1 { CREATE TABLE t7(a, b); } { SELECT * FROM t7 WHERE a=? OR b=? } { CREATE INDEX t7_idx_00000062 ON t7(b); CREATE INDEX t7_idx_00000061 ON t7(a); MULTI-INDEX OR INDEX 1 SEARCH t7 USING INDEX t7_idx_00000061 (a=?) INDEX 2 SEARCH t7 USING INDEX t7_idx_00000062 (b=?) } # rowid terms. # do_setup_rec_test $tn.13.1 { CREATE TABLE t8(a, b); } { SELECT * FROM t8 WHERE rowid=? } { (no new indexes) SEARCH t8 USING INTEGER PRIMARY KEY (rowid=?) } do_setup_rec_test $tn.13.2 { CREATE TABLE t8(a, b); } { SELECT * FROM t8 ORDER BY rowid } { (no new indexes) SCAN t8 } do_setup_rec_test $tn.13.3 { CREATE TABLE t8(a, b); } { SELECT * FROM t8 WHERE a=? ORDER BY rowid } { CREATE INDEX t8_idx_00000061 ON t8(a); SEARCH t8 USING INDEX t8_idx_00000061 (a=?) } # Triggers # do_setup_rec_test $tn.14 { CREATE TABLE t9(a, b, c); CREATE TABLE t10(a, b, c); CREATE TRIGGER t9t AFTER INSERT ON t9 BEGIN UPDATE t10 SET a=new.a WHERE b = new.b; END; } { INSERT INTO t9 VALUES(?, ?, ?); } { CREATE INDEX t10_idx_00000062 ON t10(b); SEARCH t10 USING INDEX t10_idx_00000062 (b=?) } do_setup_rec_test $tn.15 { CREATE TABLE t1(a, b); CREATE TABLE t2(c, d); WITH s(i) AS ( VALUES(1) UNION ALL SELECT i+1 FROM s WHERE i<100) INSERT INTO t1 SELECT (i-1)/50, (i-1)/20 FROM s; WITH s(i) AS ( VALUES(1) UNION ALL SELECT i+1 FROM s WHERE i<100) INSERT INTO t2 SELECT (i-1)/20, (i-1)/5 FROM s; } { SELECT * FROM t2, t1 WHERE b=? AND d=? AND t2.rowid=t1.rowid } { CREATE INDEX t2_idx_00000064 ON t2(d); SEARCH t2 USING INDEX t2_idx_00000064 (d=?) SEARCH t1 USING INTEGER PRIMARY KEY (rowid=?) } do_setup_rec_test $tn.16 { CREATE TABLE t1(a, b); } { SELECT * FROM t1 WHERE b IS NOT NULL; } { (no new indexes) SCAN t1 } do_setup_rec_test $tn.17.1 { CREATE TABLE example (A INTEGER, B INTEGER, C INTEGER, PRIMARY KEY (A,B)); } { SELECT * FROM example WHERE a=? } { (no new indexes) SEARCH example USING INDEX sqlite_autoindex_example_1 (A=?) } do_setup_rec_test $tn.17.2 { CREATE TABLE example (A INTEGER, B INTEGER, C INTEGER, PRIMARY KEY (A,B)); } { SELECT * FROM example WHERE b=? } { CREATE INDEX example_idx_00000042 ON example(B); SEARCH example USING INDEX example_idx_00000042 (B=?) } do_setup_rec_test $tn.17.3 { CREATE TABLE example (A INTEGER, B INTEGER, C INTEGER, PRIMARY KEY (A,B)); } { SELECT * FROM example WHERE a=? AND b=? } { (no new indexes) SEARCH example USING INDEX sqlite_autoindex_example_1 (A=? AND B=?) } do_setup_rec_test $tn.17.4 { CREATE TABLE example (A INTEGER, B INTEGER, C INTEGER, PRIMARY KEY (A,B)); } { SELECT * FROM example WHERE a=? AND b>? } { (no new indexes) SEARCH example USING INDEX sqlite_autoindex_example_1 (A=? AND B>?) } do_setup_rec_test $tn.17.5 { CREATE TABLE example (A INTEGER, B INTEGER, C INTEGER, PRIMARY KEY (A,B)); } { SELECT * FROM example WHERE a>? AND b=? } { CREATE INDEX example_idx_0000cb3f ON example(B, A); SEARCH example USING INDEX example_idx_0000cb3f (B=? AND A>?) } do_setup_rec_test $tn.18.0 { CREATE TABLE SomeObject ( a INTEGER PRIMARY KEY, x TEXT GENERATED ALWAYS AS(HEX(a)) VIRTUAL ); } { SELECT x FROM SomeObject; } { (no new indexes) SCAN SomeObject } do_setup_rec_test $tn.18.1 { CREATE TABLE SomeObject ( a INTEGER PRIMARY KEY, x TEXT GENERATED ALWAYS AS(HEX(a)) VIRTUAL ); } { SELECT * FROM SomeObject WHERE x=?; } { CREATE INDEX SomeObject_idx_00000078 ON SomeObject(x); SEARCH SomeObject USING COVERING INDEX SomeObject_idx_00000078 (x=?) } do_setup_rec_test $tn.19.0 { CREATE TABLE t1("index"); } { SELECT * FROM t1 ORDER BY "index"; } { CREATE INDEX t1_idx_01a7214e ON t1('index'); SCAN t1 USING COVERING INDEX t1_idx_01a7214e } } proc do_candidates_test {tn sql res} { set res [squish [string trim $res]] set expert [sqlite3_expert_new db] $expert sql $sql $expert analyze set candidates [squish [string trim [$expert report 0 candidates]]] $expert destroy uplevel [list do_test $tn [list set {} $candidates] $res] } reset_db do_execsql_test 5.0 { CREATE TABLE t1(a, b); CREATE TABLE t2(c, d); WITH s(i) AS ( VALUES(1) UNION ALL SELECT i+1 FROM s WHERE i<100) INSERT INTO t1 SELECT (i-1)/50, (i-1)/20 FROM s; WITH s(i) AS ( VALUES(1) UNION ALL SELECT i+1 FROM s WHERE i<100) INSERT INTO t2 SELECT (i-1)/20, (i-1)/5 FROM s; } do_candidates_test 5.1 { SELECT * FROM t1,t2 WHERE (b=? OR a=?) AND (c=? OR d=?) } { CREATE INDEX t1_idx_00000062 ON t1(b); -- stat1: 100 20 CREATE INDEX t1_idx_00000061 ON t1(a); -- stat1: 100 50 CREATE INDEX t2_idx_00000063 ON t2(c); -- stat1: 100 20 CREATE INDEX t2_idx_00000064 ON t2(d); -- stat1: 100 5 } do_candidates_test 5.2 { SELECT * FROM t1,t2 WHERE a=? AND b=? AND c=? AND d=? } { CREATE INDEX t1_idx_000123a7 ON t1(a, b); -- stat1: 100 50 17 CREATE INDEX t2_idx_0001295b ON t2(c, d); -- stat1: 100 20 5 } do_execsql_test 5.3 { CREATE INDEX t1_idx_00000061 ON t1(a); -- stat1: 100 50 CREATE INDEX t1_idx_00000062 ON t1(b); -- stat1: 100 20 CREATE INDEX t1_idx_000123a7 ON t1(a, b); -- stat1: 100 50 16 CREATE INDEX t2_idx_00000063 ON t2(c); -- stat1: 100 20 CREATE INDEX t2_idx_00000064 ON t2(d); -- stat1: 100 5 CREATE INDEX t2_idx_0001295b ON t2(c, d); -- stat1: 100 20 5 ANALYZE; SELECT * FROM sqlite_stat1 ORDER BY 1, 2; } { t1 t1_idx_00000061 {100 50} t1 t1_idx_00000062 {100 20} t1 t1_idx_000123a7 {100 50 17} t2 t2_idx_00000063 {100 20} t2 t2_idx_00000064 {100 5} t2 t2_idx_0001295b {100 20 5} } finish_test |
Changes to ext/expert/sqlite3expert.c.
︙ | ︙ | |||
10 11 12 13 14 15 16 17 18 19 20 21 22 23 | ** ************************************************************************* */ #include "sqlite3expert.h" #include <assert.h> #include <string.h> #include <stdio.h> #ifndef SQLITE_OMIT_VIRTUALTABLE typedef sqlite3_int64 i64; typedef sqlite3_uint64 u64; typedef struct IdxColumn IdxColumn; | > > > > > > > > > > > > > > > > > | 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 | ** ************************************************************************* */ #include "sqlite3expert.h" #include <assert.h> #include <string.h> #include <stdio.h> #if !defined(SQLITE_AMALGAMATION) #if defined(SQLITE_COVERAGE_TEST) || defined(SQLITE_MUTATION_TEST) # define SQLITE_OMIT_AUXILIARY_SAFETY_CHECKS 1 #endif #if defined(SQLITE_OMIT_AUXILIARY_SAFETY_CHECKS) # define ALWAYS(X) (1) # define NEVER(X) (0) #elif !defined(NDEBUG) # define ALWAYS(X) ((X)?1:(assert(0),0)) # define NEVER(X) ((X)?(assert(0),1):0) #else # define ALWAYS(X) (X) # define NEVER(X) (X) #endif #endif /* !defined(SQLITE_AMALGAMATION) */ #ifndef SQLITE_OMIT_VIRTUALTABLE typedef sqlite3_int64 i64; typedef sqlite3_uint64 u64; typedef struct IdxColumn IdxColumn; |
︙ | ︙ | |||
676 677 678 679 680 681 682 | sqlite3 *db, /* Database connection to read details from */ const char *zTab, /* Table name */ IdxTable **ppOut, /* OUT: New object (if successful) */ char **pzErrmsg /* OUT: Error message (if not) */ ){ sqlite3_stmt *p1 = 0; int nCol = 0; | | | > > > > > | > > > > > | > | > > > > | | | > | | | | | 693 694 695 696 697 698 699 700 701 702 703 704 705 706 707 708 709 710 711 712 713 714 715 716 717 718 719 720 721 722 723 724 725 726 727 728 729 730 731 732 733 734 735 736 737 738 739 740 741 742 743 744 745 746 747 748 749 750 751 752 753 754 755 756 757 758 759 760 761 762 763 764 765 766 767 768 769 770 771 772 773 774 775 776 777 778 779 780 | sqlite3 *db, /* Database connection to read details from */ const char *zTab, /* Table name */ IdxTable **ppOut, /* OUT: New object (if successful) */ char **pzErrmsg /* OUT: Error message (if not) */ ){ sqlite3_stmt *p1 = 0; int nCol = 0; int nTab; int nByte; IdxTable *pNew = 0; int rc, rc2; char *pCsr = 0; int nPk = 0; *ppOut = 0; if( zTab==0 ) return SQLITE_ERROR; nTab = STRLEN(zTab); nByte = sizeof(IdxTable) + nTab + 1; rc = idxPrintfPrepareStmt(db, &p1, pzErrmsg, "PRAGMA table_xinfo=%Q", zTab); while( rc==SQLITE_OK && SQLITE_ROW==sqlite3_step(p1) ){ const char *zCol = (const char*)sqlite3_column_text(p1, 1); const char *zColSeq = 0; if( zCol==0 ){ rc = SQLITE_ERROR; break; } nByte += 1 + STRLEN(zCol); rc = sqlite3_table_column_metadata( db, "main", zTab, zCol, 0, &zColSeq, 0, 0, 0 ); if( zColSeq==0 ) zColSeq = "binary"; nByte += 1 + STRLEN(zColSeq); nCol++; nPk += (sqlite3_column_int(p1, 5)>0); } rc2 = sqlite3_reset(p1); if( rc==SQLITE_OK ) rc = rc2; nByte += sizeof(IdxColumn) * nCol; if( rc==SQLITE_OK ){ pNew = idxMalloc(&rc, nByte); } if( rc==SQLITE_OK ){ pNew->aCol = (IdxColumn*)&pNew[1]; pNew->nCol = nCol; pCsr = (char*)&pNew->aCol[nCol]; } nCol = 0; while( rc==SQLITE_OK && SQLITE_ROW==sqlite3_step(p1) ){ const char *zCol = (const char*)sqlite3_column_text(p1, 1); const char *zColSeq = 0; int nCopy; if( zCol==0 ) continue; nCopy = STRLEN(zCol) + 1; pNew->aCol[nCol].zName = pCsr; pNew->aCol[nCol].iPk = (sqlite3_column_int(p1, 5)==1 && nPk==1); memcpy(pCsr, zCol, nCopy); pCsr += nCopy; rc = sqlite3_table_column_metadata( db, "main", zTab, zCol, 0, &zColSeq, 0, 0, 0 ); if( rc==SQLITE_OK ){ if( zColSeq==0 ) zColSeq = "binary"; nCopy = STRLEN(zColSeq) + 1; pNew->aCol[nCol].zColl = pCsr; memcpy(pCsr, zColSeq, nCopy); pCsr += nCopy; } nCol++; } idxFinalize(&rc, p1); if( rc!=SQLITE_OK ){ sqlite3_free(pNew); pNew = 0; }else if( ALWAYS(pNew!=0) ){ pNew->zName = pCsr; if( ALWAYS(pNew->zName!=0) ) memcpy(pNew->zName, zTab, nTab+1); } *ppOut = pNew; return rc; } /* |
︙ | ︙ | |||
783 784 785 786 787 788 789 790 791 792 793 794 795 796 | /* ** Return true if zId must be quoted in order to use it as an SQL ** identifier, or false otherwise. */ static int idxIdentifierRequiresQuotes(const char *zId){ int i; for(i=0; zId[i]; i++){ if( !(zId[i]=='_') && !(zId[i]>='0' && zId[i]<='9') && !(zId[i]>='a' && zId[i]<='z') && !(zId[i]>='A' && zId[i]<='Z') ){ return 1; | > > > > | 816 817 818 819 820 821 822 823 824 825 826 827 828 829 830 831 832 833 | /* ** Return true if zId must be quoted in order to use it as an SQL ** identifier, or false otherwise. */ static int idxIdentifierRequiresQuotes(const char *zId){ int i; int nId = STRLEN(zId); if( sqlite3_keyword_check(zId, nId) ) return 1; for(i=0; zId[i]; i++){ if( !(zId[i]=='_') && !(zId[i]>='0' && zId[i]<='9') && !(zId[i]>='a' && zId[i]<='z') && !(zId[i]>='A' && zId[i]<='Z') ){ return 1; |
︙ | ︙ | |||
859 860 861 862 863 864 865 866 867 868 869 870 871 872 | rc = idxPrintfPrepareStmt(dbm, &pIdxList, 0, "PRAGMA index_list=%Q", zTbl); while( rc==SQLITE_OK && sqlite3_step(pIdxList)==SQLITE_ROW ){ int bMatch = 1; IdxConstraint *pT = pTail; sqlite3_stmt *pInfo = 0; const char *zIdx = (const char*)sqlite3_column_text(pIdxList, 1); /* Zero the IdxConstraint.bFlag values in the pEq list */ for(pIter=pEq; pIter; pIter=pIter->pLink) pIter->bFlag = 0; rc = idxPrintfPrepareStmt(dbm, &pInfo, 0, "PRAGMA index_xInfo=%Q", zIdx); while( rc==SQLITE_OK && sqlite3_step(pInfo)==SQLITE_ROW ){ int iIdx = sqlite3_column_int(pInfo, 0); | > | 896 897 898 899 900 901 902 903 904 905 906 907 908 909 910 | rc = idxPrintfPrepareStmt(dbm, &pIdxList, 0, "PRAGMA index_list=%Q", zTbl); while( rc==SQLITE_OK && sqlite3_step(pIdxList)==SQLITE_ROW ){ int bMatch = 1; IdxConstraint *pT = pTail; sqlite3_stmt *pInfo = 0; const char *zIdx = (const char*)sqlite3_column_text(pIdxList, 1); if( zIdx==0 ) continue; /* Zero the IdxConstraint.bFlag values in the pEq list */ for(pIter=pEq; pIter; pIter=pIter->pLink) pIter->bFlag = 0; rc = idxPrintfPrepareStmt(dbm, &pInfo, 0, "PRAGMA index_xInfo=%Q", zIdx); while( rc==SQLITE_OK && sqlite3_step(pInfo)==SQLITE_ROW ){ int iIdx = sqlite3_column_int(pInfo, 0); |
︙ | ︙ | |||
903 904 905 906 907 908 909 910 911 912 913 914 915 916 | } } idxFinalize(&rc, pIdxList); *pRc = rc; return 0; } static int idxCreateFromCons( sqlite3expert *p, IdxScan *pScan, IdxConstraint *pEq, IdxConstraint *pTail ){ | > > > > > > > > > > > > > | 941 942 943 944 945 946 947 948 949 950 951 952 953 954 955 956 957 958 959 960 961 962 963 964 965 966 967 | } } idxFinalize(&rc, pIdxList); *pRc = rc; return 0; } /* Callback for sqlite3_exec() with query with leading count(*) column. * The first argument is expected to be an int*, referent to be incremented * if that leading column is not exactly '0'. */ static int countNonzeros(void* pCount, int nc, char* azResults[], char* azColumns[]){ (void)azColumns; /* Suppress unused parameter warning */ if( nc>0 && (azResults[0][0]!='0' || azResults[0][1]!=0) ){ *((int *)pCount) += 1; } return 0; } static int idxCreateFromCons( sqlite3expert *p, IdxScan *pScan, IdxConstraint *pEq, IdxConstraint *pTail ){ |
︙ | ︙ | |||
930 931 932 933 934 935 936 | for(pCons=pTail; pCons; pCons=pCons->pLink){ zCols = idxAppendColDefn(&rc, zCols, pTab, pCons); } if( rc==SQLITE_OK ){ /* Hash the list of columns to come up with a name for the index */ const char *zTable = pScan->pTab->zName; | > | > > | > | | | > | | > > > > > > > > > > > > > > > > > > | | > > > | > | 981 982 983 984 985 986 987 988 989 990 991 992 993 994 995 996 997 998 999 1000 1001 1002 1003 1004 1005 1006 1007 1008 1009 1010 1011 1012 1013 1014 1015 1016 1017 1018 1019 1020 1021 1022 1023 1024 1025 1026 1027 1028 1029 1030 1031 1032 1033 1034 1035 1036 1037 1038 1039 1040 1041 | for(pCons=pTail; pCons; pCons=pCons->pLink){ zCols = idxAppendColDefn(&rc, zCols, pTab, pCons); } if( rc==SQLITE_OK ){ /* Hash the list of columns to come up with a name for the index */ const char *zTable = pScan->pTab->zName; int quoteTable = idxIdentifierRequiresQuotes(zTable); char *zName = 0; /* Index name */ int collisions = 0; do{ int i; char *zFind; for(i=0; zCols[i]; i++){ h += ((h<<3) + zCols[i]); } sqlite3_free(zName); zName = sqlite3_mprintf("%s_idx_%08x", zTable, h); if( zName==0 ) break; /* Is is unique among table, view and index names? */ zFmt = "SELECT count(*) FROM sqlite_schema WHERE name=%Q" " AND type in ('index','table','view')"; zFind = sqlite3_mprintf(zFmt, zName); i = 0; rc = sqlite3_exec(dbm, zFind, countNonzeros, &i, 0); assert(rc==SQLITE_OK); sqlite3_free(zFind); if( i==0 ){ collisions = 0; break; } ++collisions; }while( collisions<50 && zName!=0 ); if( collisions ){ /* This return means "Gave up trying to find a unique index name." */ rc = SQLITE_BUSY_TIMEOUT; }else if( zName==0 ){ rc = SQLITE_NOMEM; }else{ if( quoteTable ){ zFmt = "CREATE INDEX \"%w\" ON \"%w\"(%s)"; }else{ zFmt = "CREATE INDEX %s ON %s(%s)"; } zIdx = sqlite3_mprintf(zFmt, zName, zTable, zCols); if( !zIdx ){ rc = SQLITE_NOMEM; }else{ rc = sqlite3_exec(dbm, zIdx, 0, 0, p->pzErrmsg); if( rc!=SQLITE_OK ){ rc = SQLITE_BUSY_TIMEOUT; }else{ idxHashAdd(&rc, &p->hIdx, zName, zIdx); } } sqlite3_free(zName); sqlite3_free(zIdx); } } sqlite3_free(zCols); |
︙ | ︙ | |||
1101 1102 1103 1104 1105 1106 1107 | /* ** This function is called after candidate indexes have been created. It ** runs all the queries to see which indexes they prefer, and populates ** IdxStatement.zIdx and IdxStatement.zEQP with the results. */ | | | 1179 1180 1181 1182 1183 1184 1185 1186 1187 1188 1189 1190 1191 1192 1193 | /* ** This function is called after candidate indexes have been created. It ** runs all the queries to see which indexes they prefer, and populates ** IdxStatement.zIdx and IdxStatement.zEQP with the results. */ static int idxFindIndexes( sqlite3expert *p, char **pzErr /* OUT: Error message (sqlite3_malloc) */ ){ IdxStatement *pStmt; sqlite3 *dbm = p->dbm; int rc = SQLITE_OK; |
︙ | ︙ | |||
1124 1125 1126 1127 1128 1129 1130 | "EXPLAIN QUERY PLAN %s", pStmt->zSql ); while( rc==SQLITE_OK && sqlite3_step(pExplain)==SQLITE_ROW ){ /* int iId = sqlite3_column_int(pExplain, 0); */ /* int iParent = sqlite3_column_int(pExplain, 1); */ /* int iNotUsed = sqlite3_column_int(pExplain, 2); */ const char *zDetail = (const char*)sqlite3_column_text(pExplain, 3); | | > > > | > | > | 1202 1203 1204 1205 1206 1207 1208 1209 1210 1211 1212 1213 1214 1215 1216 1217 1218 1219 1220 1221 1222 1223 1224 1225 1226 1227 1228 | "EXPLAIN QUERY PLAN %s", pStmt->zSql ); while( rc==SQLITE_OK && sqlite3_step(pExplain)==SQLITE_ROW ){ /* int iId = sqlite3_column_int(pExplain, 0); */ /* int iParent = sqlite3_column_int(pExplain, 1); */ /* int iNotUsed = sqlite3_column_int(pExplain, 2); */ const char *zDetail = (const char*)sqlite3_column_text(pExplain, 3); int nDetail; int i; if( !zDetail ) continue; nDetail = STRLEN(zDetail); for(i=0; i<nDetail; i++){ const char *zIdx = 0; if( i+13<nDetail && memcmp(&zDetail[i], " USING INDEX ", 13)==0 ){ zIdx = &zDetail[i+13]; }else if( i+22<nDetail && memcmp(&zDetail[i], " USING COVERING INDEX ", 22)==0 ){ zIdx = &zDetail[i+22]; } if( zIdx ){ const char *zSql; int nIdx = 0; while( zIdx[nIdx]!='\0' && (zIdx[nIdx]!=' ' || zIdx[nIdx+1]!='(') ){ nIdx++; |
︙ | ︙ | |||
1214 1215 1216 1217 1218 1219 1220 | char **pzErr ){ static const char *zInt = UNIQUE_TABLE_NAME; static const char *zDrop = "DROP TABLE " UNIQUE_TABLE_NAME; IdxTable *pTab = pWrite->pTab; const char *zTab = pTab->zName; const char *zSql = | | > | 1297 1298 1299 1300 1301 1302 1303 1304 1305 1306 1307 1308 1309 1310 1311 1312 1313 1314 1315 1316 1317 1318 1319 1320 1321 1322 | char **pzErr ){ static const char *zInt = UNIQUE_TABLE_NAME; static const char *zDrop = "DROP TABLE " UNIQUE_TABLE_NAME; IdxTable *pTab = pWrite->pTab; const char *zTab = pTab->zName; const char *zSql = "SELECT 'CREATE TEMP' || substr(sql, 7) FROM sqlite_schema " "WHERE tbl_name = %Q AND type IN ('table', 'trigger') " "ORDER BY type;"; sqlite3_stmt *pSelect = 0; int rc = SQLITE_OK; char *zWrite = 0; /* Create the table and its triggers in the temp schema */ rc = idxPrintfPrepareStmt(p->db, &pSelect, pzErr, zSql, zTab, zTab); while( rc==SQLITE_OK && SQLITE_ROW==sqlite3_step(pSelect) ){ const char *zCreate = (const char*)sqlite3_column_text(pSelect, 0); if( zCreate==0 ) continue; rc = sqlite3_exec(p->dbv, zCreate, 0, 0, pzErr); } idxFinalize(&rc, pSelect); /* Rename the table in the temp schema to zInt */ if( rc==SQLITE_OK ){ char *z = sqlite3_mprintf("ALTER TABLE temp.%Q RENAME TO %Q", zTab, zInt); |
︙ | ︙ | |||
1314 1315 1316 1317 1318 1319 1320 | /* For each table in the main db schema: ** ** 1) Add an entry to the p->pTable list, and ** 2) Create the equivalent virtual table in dbv. */ rc = idxPrepareStmt(p->db, &pSchema, pzErrmsg, | | | | > | | 1398 1399 1400 1401 1402 1403 1404 1405 1406 1407 1408 1409 1410 1411 1412 1413 1414 1415 1416 1417 1418 1419 1420 1421 1422 1423 1424 1425 1426 1427 | /* For each table in the main db schema: ** ** 1) Add an entry to the p->pTable list, and ** 2) Create the equivalent virtual table in dbv. */ rc = idxPrepareStmt(p->db, &pSchema, pzErrmsg, "SELECT type, name, sql, 1 FROM sqlite_schema " "WHERE type IN ('table','view') AND name NOT LIKE 'sqlite_%%' " " UNION ALL " "SELECT type, name, sql, 2 FROM sqlite_schema " "WHERE type = 'trigger'" " AND tbl_name IN(SELECT name FROM sqlite_schema WHERE type = 'view') " "ORDER BY 4, 1" ); while( rc==SQLITE_OK && SQLITE_ROW==sqlite3_step(pSchema) ){ const char *zType = (const char*)sqlite3_column_text(pSchema, 0); const char *zName = (const char*)sqlite3_column_text(pSchema, 1); const char *zSql = (const char*)sqlite3_column_text(pSchema, 2); if( zType==0 || zName==0 ) continue; if( zType[0]=='v' || zType[1]=='r' ){ if( zSql ) rc = sqlite3_exec(p->dbv, zSql, 0, 0, pzErrmsg); }else{ IdxTable *pTab; rc = idxGetTableInfo(p->db, zName, &pTab, pzErrmsg); if( rc==SQLITE_OK ){ int i; char *zInner = 0; char *zOuter = 0; |
︙ | ︙ | |||
1465 1466 1467 1468 1469 1470 1471 1472 1473 1474 1475 1476 1477 1478 1479 1480 1481 1482 | case SQLITE_FLOAT: pSlot->rVal = sqlite3_value_double(argv[1]); break; case SQLITE_BLOB: case SQLITE_TEXT: { int nByte = sqlite3_value_bytes(argv[1]); if( nByte>pSlot->nByte ){ char *zNew = (char*)sqlite3_realloc(pSlot->z, nByte*2); if( zNew==0 ){ sqlite3_result_error_nomem(pCtx); return; } pSlot->nByte = nByte*2; pSlot->z = zNew; } pSlot->n = nByte; if( pSlot->eType==SQLITE_BLOB ){ | > | > | > | | 1550 1551 1552 1553 1554 1555 1556 1557 1558 1559 1560 1561 1562 1563 1564 1565 1566 1567 1568 1569 1570 1571 1572 1573 1574 1575 1576 1577 1578 1579 1580 1581 1582 1583 1584 1585 1586 1587 1588 1589 1590 1591 | case SQLITE_FLOAT: pSlot->rVal = sqlite3_value_double(argv[1]); break; case SQLITE_BLOB: case SQLITE_TEXT: { int nByte = sqlite3_value_bytes(argv[1]); const void *pData = 0; if( nByte>pSlot->nByte ){ char *zNew = (char*)sqlite3_realloc(pSlot->z, nByte*2); if( zNew==0 ){ sqlite3_result_error_nomem(pCtx); return; } pSlot->nByte = nByte*2; pSlot->z = zNew; } pSlot->n = nByte; if( pSlot->eType==SQLITE_BLOB ){ pData = sqlite3_value_blob(argv[1]); if( pData ) memcpy(pSlot->z, pData, nByte); }else{ pData = sqlite3_value_text(argv[1]); memcpy(pSlot->z, pData, nByte); } break; } } } static int idxLargestIndex(sqlite3 *db, int *pnMax, char **pzErr){ int rc = SQLITE_OK; const char *zMax = "SELECT max(i.seqno) FROM " " sqlite_schema AS s, " " pragma_index_list(s.name) AS l, " " pragma_index_info(l.name) AS i " "WHERE s.type = 'table'"; sqlite3_stmt *pMax = 0; *pnMax = 0; rc = idxPrepareStmt(db, &pMax, pzErr, zMax); |
︙ | ︙ | |||
1642 1643 1644 1645 1646 1647 1648 | i64 iPrev = -100000; sqlite3_stmt *pAllIndex = 0; sqlite3_stmt *pIndexXInfo = 0; sqlite3_stmt *pWrite = 0; const char *zAllIndex = "SELECT s.rowid, s.name, l.name FROM " | | | 1730 1731 1732 1733 1734 1735 1736 1737 1738 1739 1740 1741 1742 1743 1744 | i64 iPrev = -100000; sqlite3_stmt *pAllIndex = 0; sqlite3_stmt *pIndexXInfo = 0; sqlite3_stmt *pWrite = 0; const char *zAllIndex = "SELECT s.rowid, s.name, l.name FROM " " sqlite_schema AS s, " " pragma_index_list(s.name) AS l " "WHERE s.type = 'table'"; const char *zIndexXInfo = "SELECT name, coll FROM pragma_index_xinfo(?) WHERE key"; const char *zWrite = "INSERT INTO sqlite_stat1 VALUES(?, ?, ?)"; /* If iSample==0, no sqlite_stat1 data is required. */ |
︙ | ︙ | |||
1689 1690 1691 1692 1693 1694 1695 1696 1697 1698 1699 1700 1701 1702 | rc = idxPrepareStmt(p->dbm, &pWrite, pzErr, zWrite); } while( rc==SQLITE_OK && SQLITE_ROW==sqlite3_step(pAllIndex) ){ i64 iRowid = sqlite3_column_int64(pAllIndex, 0); const char *zTab = (const char*)sqlite3_column_text(pAllIndex, 1); const char *zIdx = (const char*)sqlite3_column_text(pAllIndex, 2); if( p->iSample<100 && iPrev!=iRowid ){ samplectx.target = (double)p->iSample / 100.0; samplectx.iTarget = p->iSample; samplectx.nRow = 0.0; samplectx.nRet = 0.0; rc = idxBuildSampleTable(p, zTab); if( rc!=SQLITE_OK ) break; | > | 1777 1778 1779 1780 1781 1782 1783 1784 1785 1786 1787 1788 1789 1790 1791 | rc = idxPrepareStmt(p->dbm, &pWrite, pzErr, zWrite); } while( rc==SQLITE_OK && SQLITE_ROW==sqlite3_step(pAllIndex) ){ i64 iRowid = sqlite3_column_int64(pAllIndex, 0); const char *zTab = (const char*)sqlite3_column_text(pAllIndex, 1); const char *zIdx = (const char*)sqlite3_column_text(pAllIndex, 2); if( zTab==0 || zIdx==0 ) continue; if( p->iSample<100 && iPrev!=iRowid ){ samplectx.target = (double)p->iSample / 100.0; samplectx.iTarget = p->iSample; samplectx.nRow = 0.0; samplectx.nRet = 0.0; rc = idxBuildSampleTable(p, zTab); if( rc!=SQLITE_OK ) break; |
︙ | ︙ | |||
1710 1711 1712 1713 1714 1715 1716 | ); } idxFinalize(&rc, pAllIndex); idxFinalize(&rc, pIndexXInfo); idxFinalize(&rc, pWrite); | > | | | | | > | | 1799 1800 1801 1802 1803 1804 1805 1806 1807 1808 1809 1810 1811 1812 1813 1814 1815 1816 1817 1818 1819 1820 1821 | ); } idxFinalize(&rc, pAllIndex); idxFinalize(&rc, pIndexXInfo); idxFinalize(&rc, pWrite); if( pCtx ){ for(i=0; i<pCtx->nSlot; i++){ sqlite3_free(pCtx->aSlot[i].z); } sqlite3_free(pCtx); } if( rc==SQLITE_OK ){ rc = sqlite3_exec(p->dbm, "ANALYZE sqlite_schema", 0, 0, 0); } sqlite3_exec(p->db, "DROP TABLE IF EXISTS temp."UNIQUE_TABLE_NAME,0,0,0); return rc; } /* |
︙ | ︙ | |||
1753 1754 1755 1756 1757 1758 1759 | sqlite3_db_config(pNew->dbm, SQLITE_DBCONFIG_TRIGGER_EQP, 1, (int*)0); } } /* Copy the entire schema of database [db] into [dbm]. */ if( rc==SQLITE_OK ){ | | | | | 1844 1845 1846 1847 1848 1849 1850 1851 1852 1853 1854 1855 1856 1857 1858 1859 1860 1861 1862 1863 1864 1865 | sqlite3_db_config(pNew->dbm, SQLITE_DBCONFIG_TRIGGER_EQP, 1, (int*)0); } } /* Copy the entire schema of database [db] into [dbm]. */ if( rc==SQLITE_OK ){ sqlite3_stmt *pSql = 0; rc = idxPrintfPrepareStmt(pNew->db, &pSql, pzErrmsg, "SELECT sql FROM sqlite_schema WHERE name NOT LIKE 'sqlite_%%'" " AND sql NOT LIKE 'CREATE VIRTUAL %%'" ); while( rc==SQLITE_OK && SQLITE_ROW==sqlite3_step(pSql) ){ const char *zSql = (const char*)sqlite3_column_text(pSql, 0); if( zSql ) rc = sqlite3_exec(pNew->dbm, zSql, 0, 0, pzErrmsg); } idxFinalize(&rc, pSql); } /* Create the vtab schema */ if( rc==SQLITE_OK ){ rc = idxCreateVtabSchema(pNew, pzErrmsg); |
︙ | ︙ | |||
1866 1867 1868 1869 1870 1871 1872 1873 1874 1875 1876 1877 1878 1879 | /* Do trigger processing to collect any extra IdxScan structures */ rc = idxProcessTriggers(p, pzErr); /* Create candidate indexes within the in-memory database file */ if( rc==SQLITE_OK ){ rc = idxCreateCandidates(p); } /* Generate the stat1 data */ if( rc==SQLITE_OK ){ rc = idxPopulateStat1(p, pzErr); } | > > > > | 1957 1958 1959 1960 1961 1962 1963 1964 1965 1966 1967 1968 1969 1970 1971 1972 1973 1974 | /* Do trigger processing to collect any extra IdxScan structures */ rc = idxProcessTriggers(p, pzErr); /* Create candidate indexes within the in-memory database file */ if( rc==SQLITE_OK ){ rc = idxCreateCandidates(p); }else if ( rc==SQLITE_BUSY_TIMEOUT ){ if( pzErr ) *pzErr = sqlite3_mprintf("Cannot find a unique index name to propose."); return rc; } /* Generate the stat1 data */ if( rc==SQLITE_OK ){ rc = idxPopulateStat1(p, pzErr); } |
︙ | ︙ | |||
1946 1947 1948 1949 1950 1951 1952 | idxWriteFree(p->pWrite); idxHashClear(&p->hIdx); sqlite3_free(p->zCandidates); sqlite3_free(p); } } | | | 2041 2042 2043 2044 2045 2046 2047 2048 | idxWriteFree(p->pWrite); idxHashClear(&p->hIdx); sqlite3_free(p->zCandidates); sqlite3_free(p); } } #endif /* ifndef SQLITE_OMIT_VIRTUALTABLE */ |
Changes to ext/expert/sqlite3expert.h.
1 2 3 4 5 6 7 8 9 10 11 12 | /* ** 2017 April 07 ** ** The author disclaims copyright to this source code. In place of ** a legal notice, here is a blessing: ** ** May you do good and not evil. ** May you find forgiveness for yourself and forgive others. ** May you share freely, never taking more than you give. ** ************************************************************************* */ | | | | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 | /* ** 2017 April 07 ** ** The author disclaims copyright to this source code. In place of ** a legal notice, here is a blessing: ** ** May you do good and not evil. ** May you find forgiveness for yourself and forgive others. ** May you share freely, never taking more than you give. ** ************************************************************************* */ #if !defined(SQLITEEXPERT_H) #define SQLITEEXPERT_H 1 #include "sqlite3.h" typedef struct sqlite3expert sqlite3expert; /* ** Create a new sqlite3expert object. ** |
︙ | ︙ | |||
161 162 163 164 165 166 167 | /* ** Free an (sqlite3expert*) handle and all associated resources. There ** should be one call to this function for each successful call to ** sqlite3-expert_new(). */ void sqlite3_expert_destroy(sqlite3expert*); | | | 161 162 163 164 165 166 167 168 | /* ** Free an (sqlite3expert*) handle and all associated resources. There ** should be one call to this function for each successful call to ** sqlite3-expert_new(). */ void sqlite3_expert_destroy(sqlite3expert*); #endif /* !defined(SQLITEEXPERT_H) */ |
Deleted ext/fts1/README.txt.
|
| < < |
Deleted ext/fts1/ft_hash.c.
|
| < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < |
Deleted ext/fts1/ft_hash.h.
|
| < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < |
Deleted ext/fts1/fts1.c.
|
| < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < |
Deleted ext/fts1/fts1.h.
|
| < < < < < < < < < < < |
Deleted ext/fts1/fts1_hash.c.
|
| < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < |
Deleted ext/fts1/fts1_hash.h.
|
| < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < |
Deleted ext/fts1/fts1_porter.c.
|
| < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < |
Deleted ext/fts1/fts1_tokenizer.h.
|
| < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < |
Deleted ext/fts1/fts1_tokenizer1.c.
|
| < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < |
Deleted ext/fts1/fulltext.c.
|
| < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < |
Deleted ext/fts1/fulltext.h.
|
| < < < < < < < < < < < |
Deleted ext/fts1/simple_tokenizer.c.
|
| < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < |
Deleted ext/fts1/tokenizer.h.
|
| < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < |
Deleted ext/fts2/README.tokenizers.
|
| < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < |
Deleted ext/fts2/README.txt.
|
| < < < < |
Deleted ext/fts2/fts2.c.
|
| < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < |
Deleted ext/fts2/fts2.h.
|
| < < < < < < < < < < < < < < < < < < < < < < < < < < |
Deleted ext/fts2/fts2_hash.c.
|
| < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < |
Deleted ext/fts2/fts2_hash.h.
|
| < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < |
Deleted ext/fts2/fts2_icu.c.
|
| < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < |
Deleted ext/fts2/fts2_porter.c.
|
| < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < |
Deleted ext/fts2/fts2_tokenizer.c.
|
| < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < |
Deleted ext/fts2/fts2_tokenizer.h.
|
| < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < |
Deleted ext/fts2/fts2_tokenizer1.c.
|
| < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < |
Deleted ext/fts2/mkfts2amal.tcl.
|
| < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < |
Changes to ext/fts3/README.content.
︙ | ︙ | |||
170 171 172 173 174 175 176 | INSERT INTO t3(t3) VALUES('rebuild'); This command may also be used with ordinary FTS4 tables, although it may only be useful if the full-text index has somehow become corrupt. It is an error to attempt to rebuild the full-text index maintained by a contentless FTS4 table. | < < | 170 171 172 173 174 175 176 | INSERT INTO t3(t3) VALUES('rebuild'); This command may also be used with ordinary FTS4 tables, although it may only be useful if the full-text index has somehow become corrupt. It is an error to attempt to rebuild the full-text index maintained by a contentless FTS4 table. |
Changes to ext/fts3/README.tokenizers.
︙ | ︙ | |||
48 49 50 51 52 53 54 | returned. If only one argument is passed, a pointer to the tokenizer implementation currently registered as <tokenizer-name> is returned, encoded as a blob. Or, if no such tokenizer exists, an SQL exception (error) is raised. SECURITY: If the fts3 extension is used in an environment where potentially malicious users may execute arbitrary SQL (i.e. gears), they should be | | | > > | 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 | returned. If only one argument is passed, a pointer to the tokenizer implementation currently registered as <tokenizer-name> is returned, encoded as a blob. Or, if no such tokenizer exists, an SQL exception (error) is raised. SECURITY: If the fts3 extension is used in an environment where potentially malicious users may execute arbitrary SQL (i.e. gears), they should be prevented from invoking the fts3_tokenizer() function. The fts3_tokenizer() function is disabled by default. It is only enabled by SQLITE_DBCONFIG_ENABLE_FTS3_TOKENIZER. Do not enable it in security sensitive environments. See "Sample code" below for an example of calling the fts3_tokenizer() function from C code. 3. ICU Library Tokenizers If this extension is compiled with the SQLITE_ENABLE_ICU pre-processor |
︙ | ︙ |
Changes to ext/fts3/fts3.c.
︙ | ︙ | |||
303 304 305 306 307 308 309 310 311 312 313 314 315 | #include <stdarg.h> #include "fts3.h" #ifndef SQLITE_CORE # include "sqlite3ext.h" SQLITE_EXTENSION_INIT1 #endif static int fts3EvalNext(Fts3Cursor *pCsr); static int fts3EvalStart(Fts3Cursor *pCsr); static int fts3TermSegReaderCursor( Fts3Cursor *, const char *, int, int, Fts3MultiSegReader **); | > > > > > > < > > > > > > | | < < | < < < < < | > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | | | | | | | 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340 341 342 343 344 345 346 347 348 349 350 351 352 353 354 355 356 357 358 359 360 361 362 363 364 365 366 367 368 369 370 371 372 373 374 375 376 377 378 379 380 381 382 383 384 385 386 387 388 389 390 391 392 393 394 395 396 397 398 399 400 401 402 403 404 405 406 407 408 409 410 411 412 413 414 415 416 417 418 419 420 421 422 423 424 425 426 427 428 429 430 431 432 433 434 435 436 437 438 439 | #include <stdarg.h> #include "fts3.h" #ifndef SQLITE_CORE # include "sqlite3ext.h" SQLITE_EXTENSION_INIT1 #endif typedef struct Fts3HashWrapper Fts3HashWrapper; struct Fts3HashWrapper { Fts3Hash hash; /* Hash table */ int nRef; /* Number of pointers to this object */ }; static int fts3EvalNext(Fts3Cursor *pCsr); static int fts3EvalStart(Fts3Cursor *pCsr); static int fts3TermSegReaderCursor( Fts3Cursor *, const char *, int, int, Fts3MultiSegReader **); /* ** This variable is set to false when running tests for which the on disk ** structures should not be corrupt. Otherwise, true. If it is false, extra ** assert() conditions in the fts3 code are activated - conditions that are ** only true if it is guaranteed that the fts3 database is not corrupt. */ #ifdef SQLITE_DEBUG int sqlite3_fts3_may_be_corrupt = 1; #endif /* ** Write a 64-bit variable-length integer to memory starting at p[0]. ** The length of data written will be between 1 and FTS3_VARINT_MAX bytes. ** The number of bytes written is returned. */ int sqlite3Fts3PutVarint(char *p, sqlite_int64 v){ unsigned char *q = (unsigned char *) p; sqlite_uint64 vu = v; do{ *q++ = (unsigned char) ((vu & 0x7f) | 0x80); vu >>= 7; }while( vu!=0 ); q[-1] &= 0x7f; /* turn off high bit in final byte */ assert( q - (unsigned char *)p <= FTS3_VARINT_MAX ); return (int) (q - (unsigned char *)p); } #define GETVARINT_STEP(v, ptr, shift, mask1, mask2, var, ret) \ v = (v & mask1) | ( (*(const unsigned char*)(ptr++)) << shift ); \ if( (v & mask2)==0 ){ var = v; return ret; } #define GETVARINT_INIT(v, ptr, shift, mask1, mask2, var, ret) \ v = (*ptr++); \ if( (v & mask2)==0 ){ var = v; return ret; } int sqlite3Fts3GetVarintU(const char *pBuf, sqlite_uint64 *v){ const unsigned char *p = (const unsigned char*)pBuf; const unsigned char *pStart = p; u32 a; u64 b; int shift; GETVARINT_INIT(a, p, 0, 0x00, 0x80, *v, 1); GETVARINT_STEP(a, p, 7, 0x7F, 0x4000, *v, 2); GETVARINT_STEP(a, p, 14, 0x3FFF, 0x200000, *v, 3); GETVARINT_STEP(a, p, 21, 0x1FFFFF, 0x10000000, *v, 4); b = (a & 0x0FFFFFFF ); for(shift=28; shift<=63; shift+=7){ u64 c = *p++; b += (c&0x7F) << shift; if( (c & 0x80)==0 ) break; } *v = b; return (int)(p - pStart); } /* ** Read a 64-bit variable-length integer from memory starting at p[0]. ** Return the number of bytes read, or 0 on error. ** The value is stored in *v. */ int sqlite3Fts3GetVarint(const char *pBuf, sqlite_int64 *v){ return sqlite3Fts3GetVarintU(pBuf, (sqlite3_uint64*)v); } /* ** Read a 64-bit variable-length integer from memory starting at p[0] and ** not extending past pEnd[-1]. ** Return the number of bytes read, or 0 on error. ** The value is stored in *v. */ int sqlite3Fts3GetVarintBounded( const char *pBuf, const char *pEnd, sqlite_int64 *v ){ const unsigned char *p = (const unsigned char*)pBuf; const unsigned char *pStart = p; const unsigned char *pX = (const unsigned char*)pEnd; u64 b = 0; int shift; for(shift=0; shift<=63; shift+=7){ u64 c = p<pX ? *p : 0; p++; b += (c&0x7F) << shift; if( (c & 0x80)==0 ) break; } *v = b; return (int)(p - pStart); } /* ** Similar to sqlite3Fts3GetVarint(), except that the output is truncated to ** a non-negative 32-bit integer before it is returned. */ int sqlite3Fts3GetVarint32(const char *p, int *pi){ const unsigned char *ptr = (const unsigned char*)p; u32 a; #ifndef fts3GetVarint32 GETVARINT_INIT(a, ptr, 0, 0x00, 0x80, *pi, 1); #else a = (*ptr++); assert( a & 0x80 ); #endif GETVARINT_STEP(a, ptr, 7, 0x7F, 0x4000, *pi, 2); GETVARINT_STEP(a, ptr, 14, 0x3FFF, 0x200000, *pi, 3); GETVARINT_STEP(a, ptr, 21, 0x1FFFFF, 0x10000000, *pi, 4); a = (a & 0x0FFFFFFF ); *pi = (int)(a | ((u32)(*ptr & 0x07) << 28)); assert( 0==(a & 0x80000000) ); assert( *pi>=0 ); return 5; } /* ** Return the number of bytes required to encode v as a varint |
︙ | ︙ | |||
556 557 558 559 560 561 562 | static int fts3DestroyMethod(sqlite3_vtab *pVtab){ Fts3Table *p = (Fts3Table *)pVtab; int rc = SQLITE_OK; /* Return code */ const char *zDb = p->zDb; /* Name of database (e.g. "main", "temp") */ sqlite3 *db = p->db; /* Database handle */ /* Drop the shadow tables */ | < | < | | | | > > > > > > > | 596 597 598 599 600 601 602 603 604 605 606 607 608 609 610 611 612 613 614 615 616 617 618 619 620 621 | static int fts3DestroyMethod(sqlite3_vtab *pVtab){ Fts3Table *p = (Fts3Table *)pVtab; int rc = SQLITE_OK; /* Return code */ const char *zDb = p->zDb; /* Name of database (e.g. "main", "temp") */ sqlite3 *db = p->db; /* Database handle */ /* Drop the shadow tables */ fts3DbExec(&rc, db, "DROP TABLE IF EXISTS %Q.'%q_segments';" "DROP TABLE IF EXISTS %Q.'%q_segdir';" "DROP TABLE IF EXISTS %Q.'%q_docsize';" "DROP TABLE IF EXISTS %Q.'%q_stat';" "%s DROP TABLE IF EXISTS %Q.'%q_content';", zDb, p->zName, zDb, p->zName, zDb, p->zName, zDb, p->zName, (p->zContentTbl ? "--" : ""), zDb,p->zName ); /* If everything has worked, invoke fts3DisconnectMethod() to free the ** memory associated with the Fts3Table structure and return SQLITE_OK. ** Otherwise, return an SQLite error code. */ return (rc==SQLITE_OK ? fts3DisconnectMethod(pVtab) : rc); } |
︙ | ︙ | |||
794 795 796 797 798 799 800 | ** fts3QuoteId("un \"zip\"") -> "un \"\"zip\"\"" ** ** The pointer returned points to memory obtained from sqlite3_malloc(). It ** is the callers responsibility to call sqlite3_free() to release this ** memory. */ static char *fts3QuoteId(char const *zInput){ | | | | 839 840 841 842 843 844 845 846 847 848 849 850 851 852 853 854 855 856 | ** fts3QuoteId("un \"zip\"") -> "un \"\"zip\"\"" ** ** The pointer returned points to memory obtained from sqlite3_malloc(). It ** is the callers responsibility to call sqlite3_free() to release this ** memory. */ static char *fts3QuoteId(char const *zInput){ sqlite3_int64 nRet; char *zRet; nRet = 2 + (int)strlen(zInput)*2 + 1; zRet = sqlite3_malloc64(nRet); if( zRet ){ int i; char *z = zRet; *(z++) = '"'; for(i=0; zInput[i]; i++){ if( zInput[i]=='"' ) *(z++) = '"'; *(z++) = zInput[i]; |
︙ | ︙ | |||
913 914 915 916 917 918 919 920 921 922 923 924 925 926 927 928 929 930 931 932 933 934 935 | } if( p->zLanguageid ){ fts3Appendf(pRc, &zRet, ", ?"); } sqlite3_free(zFree); return zRet; } /* ** This function interprets the string at (*pp) as a non-negative integer ** value. It reads the integer and sets *pnOut to the value read, then ** sets *pp to point to the byte immediately following the last byte of ** the integer value. ** ** Only decimal digits ('0'..'9') may be part of an integer value. ** ** If *pp does not being with a decimal digit SQLITE_ERROR is returned and ** the output value undefined. Otherwise SQLITE_OK is returned. ** ** This function is used when parsing the "prefix=" FTS4 parameter. */ static int fts3GobbleInt(const char **pp, int *pnOut){ const int MAX_NPREFIX = 10000000; | > > > > > > > > > > > > > > > > < | < | | | < | > > < | | 958 959 960 961 962 963 964 965 966 967 968 969 970 971 972 973 974 975 976 977 978 979 980 981 982 983 984 985 986 987 988 989 990 991 992 993 994 995 996 997 998 999 1000 1001 1002 1003 1004 1005 1006 1007 1008 1009 1010 1011 1012 1013 1014 | } if( p->zLanguageid ){ fts3Appendf(pRc, &zRet, ", ?"); } sqlite3_free(zFree); return zRet; } /* ** Buffer z contains a positive integer value encoded as utf-8 text. ** Decode this value and store it in *pnOut, returning the number of bytes ** consumed. If an overflow error occurs return a negative value. */ int sqlite3Fts3ReadInt(const char *z, int *pnOut){ u64 iVal = 0; int i; for(i=0; z[i]>='0' && z[i]<='9'; i++){ iVal = iVal*10 + (z[i] - '0'); if( iVal>0x7FFFFFFF ) return -1; } *pnOut = (int)iVal; return i; } /* ** This function interprets the string at (*pp) as a non-negative integer ** value. It reads the integer and sets *pnOut to the value read, then ** sets *pp to point to the byte immediately following the last byte of ** the integer value. ** ** Only decimal digits ('0'..'9') may be part of an integer value. ** ** If *pp does not being with a decimal digit SQLITE_ERROR is returned and ** the output value undefined. Otherwise SQLITE_OK is returned. ** ** This function is used when parsing the "prefix=" FTS4 parameter. */ static int fts3GobbleInt(const char **pp, int *pnOut){ const int MAX_NPREFIX = 10000000; int nInt = 0; /* Output value */ int nByte; nByte = sqlite3Fts3ReadInt(*pp, &nInt); if( nInt>MAX_NPREFIX ){ nInt = 0; } if( nByte==0 ){ return SQLITE_ERROR; } *pnOut = nInt; *pp += nByte; return SQLITE_OK; } /* ** This function is called to allocate an array of Fts3Index structures ** representing the indexes maintained by the current FTS table. FTS tables ** always maintain the main "terms" index, but may also maintain one or |
︙ | ︙ | |||
978 979 980 981 982 983 984 | const char *p; nIndex++; for(p=zParam; *p; p++){ if( *p==',' ) nIndex++; } } | | | 1037 1038 1039 1040 1041 1042 1043 1044 1045 1046 1047 1048 1049 1050 1051 | const char *p; nIndex++; for(p=zParam; *p; p++){ if( *p==',' ) nIndex++; } } aIndex = sqlite3_malloc64(sizeof(struct Fts3Index) * nIndex); *apIndex = aIndex; if( !aIndex ){ return SQLITE_NOMEM; } memset(aIndex, 0, sizeof(struct Fts3Index) * nIndex); if( zParam ){ |
︙ | ︙ | |||
1057 1058 1059 1060 1061 1062 1063 | sqlite3Fts3ErrMsg(pzErr, "%s", sqlite3_errmsg(db)); } } sqlite3_free(zSql); if( rc==SQLITE_OK ){ const char **azCol; /* Output array */ | | | | | 1116 1117 1118 1119 1120 1121 1122 1123 1124 1125 1126 1127 1128 1129 1130 1131 1132 1133 1134 1135 1136 1137 1138 1139 1140 1141 1142 1143 1144 | sqlite3Fts3ErrMsg(pzErr, "%s", sqlite3_errmsg(db)); } } sqlite3_free(zSql); if( rc==SQLITE_OK ){ const char **azCol; /* Output array */ sqlite3_int64 nStr = 0; /* Size of all column names (incl. 0x00) */ int nCol; /* Number of table columns */ int i; /* Used to iterate through columns */ /* Loop through the returned columns. Set nStr to the number of bytes of ** space required to store a copy of each column name, including the ** nul-terminator byte. */ nCol = sqlite3_column_count(pStmt); for(i=0; i<nCol; i++){ const char *zCol = sqlite3_column_name(pStmt, i); nStr += strlen(zCol) + 1; } /* Allocate and populate the array to return. */ azCol = (const char **)sqlite3_malloc64(sizeof(char *) * nCol + nStr); if( azCol==0 ){ rc = SQLITE_NOMEM; }else{ char *p = (char *)&azCol[nCol]; for(i=0; i<nCol; i++){ const char *zCol = sqlite3_column_name(pStmt, i); int n = (int)strlen(zCol)+1; |
︙ | ︙ | |||
1115 1116 1117 1118 1119 1120 1121 | sqlite3 *db, /* The SQLite database connection */ void *pAux, /* Hash table containing tokenizers */ int argc, /* Number of elements in argv array */ const char * const *argv, /* xCreate/xConnect argument array */ sqlite3_vtab **ppVTab, /* Write the resulting vtab structure here */ char **pzErr /* Write any error message here */ ){ | | | | 1174 1175 1176 1177 1178 1179 1180 1181 1182 1183 1184 1185 1186 1187 1188 1189 1190 1191 1192 | sqlite3 *db, /* The SQLite database connection */ void *pAux, /* Hash table containing tokenizers */ int argc, /* Number of elements in argv array */ const char * const *argv, /* xCreate/xConnect argument array */ sqlite3_vtab **ppVTab, /* Write the resulting vtab structure here */ char **pzErr /* Write any error message here */ ){ Fts3Hash *pHash = &((Fts3HashWrapper*)pAux)->hash; Fts3Table *p = 0; /* Pointer to allocated vtab */ int rc = SQLITE_OK; /* Return code */ int i; /* Iterator variable */ sqlite3_int64 nByte; /* Size of allocation used for *p */ int iCol; /* Column index */ int nString = 0; /* Bytes required to hold all column names */ int nCol = 0; /* Number of columns in the FTS table */ char *zCsr; /* Space for holding column names */ int nDb; /* Bytes required to hold database name */ int nName; /* Bytes required to hold table name */ int isFts4 = (argv[0][3]=='4'); /* True for FTS4, false for FTS3 */ |
︙ | ︙ | |||
1153 1154 1155 1156 1157 1158 1159 | || (sqlite3_strnicmp(argv[0], "fts3", 4)==0 && !isFts4) ); nDb = (int)strlen(argv[1]) + 1; nName = (int)strlen(argv[2]) + 1; nByte = sizeof(const char *) * (argc-2); | | | | 1212 1213 1214 1215 1216 1217 1218 1219 1220 1221 1222 1223 1224 1225 1226 1227 1228 1229 | || (sqlite3_strnicmp(argv[0], "fts3", 4)==0 && !isFts4) ); nDb = (int)strlen(argv[1]) + 1; nName = (int)strlen(argv[2]) + 1; nByte = sizeof(const char *) * (argc-2); aCol = (const char **)sqlite3_malloc64(nByte); if( aCol ){ memset((void*)aCol, 0, nByte); azNotindexed = (char **)sqlite3_malloc64(nByte); } if( azNotindexed ){ memset(azNotindexed, 0, nByte); } if( !aCol || !azNotindexed ){ rc = SQLITE_NOMEM; goto fts3_init_out; |
︙ | ︙ | |||
1351 1352 1353 1354 1355 1356 1357 | nByte = sizeof(Fts3Table) + /* Fts3Table */ nCol * sizeof(char *) + /* azColumn */ nIndex * sizeof(struct Fts3Index) + /* aIndex */ nCol * sizeof(u8) + /* abNotindexed */ nName + /* zName */ nDb + /* zDb */ nString; /* Space for azColumn strings */ | | | 1410 1411 1412 1413 1414 1415 1416 1417 1418 1419 1420 1421 1422 1423 1424 | nByte = sizeof(Fts3Table) + /* Fts3Table */ nCol * sizeof(char *) + /* azColumn */ nIndex * sizeof(struct Fts3Index) + /* aIndex */ nCol * sizeof(u8) + /* abNotindexed */ nName + /* zName */ nDb + /* zDb */ nString; /* Space for azColumn strings */ p = (Fts3Table*)sqlite3_malloc64(nByte); if( p==0 ){ rc = SQLITE_NOMEM; goto fts3_init_out; } memset(p, 0, nByte); p->db = db; p->nColumn = nCol; |
︙ | ︙ | |||
1455 1456 1457 1458 1459 1460 1461 1462 1463 1464 1465 1466 1467 1468 | p->bHasStat = 2; } /* Figure out the page-size for the database. This is required in order to ** estimate the cost of loading large doclists from the database. */ fts3DatabasePageSize(&rc, p); p->nNodeSize = p->nPgsz-35; /* Declare the table schema to SQLite. */ fts3DeclareVtab(&rc, p); fts3_init_out: sqlite3_free(zPrefix); sqlite3_free(aIndex); | > > > > | 1514 1515 1516 1517 1518 1519 1520 1521 1522 1523 1524 1525 1526 1527 1528 1529 1530 1531 | p->bHasStat = 2; } /* Figure out the page-size for the database. This is required in order to ** estimate the cost of loading large doclists from the database. */ fts3DatabasePageSize(&rc, p); p->nNodeSize = p->nPgsz-35; #if defined(SQLITE_DEBUG)||defined(SQLITE_TEST) p->nMergeCount = FTS3_MERGE_COUNT; #endif /* Declare the table schema to SQLite. */ fts3DeclareVtab(&rc, p); fts3_init_out: sqlite3_free(zPrefix); sqlite3_free(aIndex); |
︙ | ︙ | |||
1550 1551 1552 1553 1554 1555 1556 1557 1558 1559 1560 1561 1562 1563 | int i; /* Iterator variable */ int iCons = -1; /* Index of constraint to use */ int iLangidCons = -1; /* Index of langid=x constraint, if present */ int iDocidGe = -1; /* Index of docid>=x constraint, if present */ int iDocidLe = -1; /* Index of docid<=x constraint, if present */ int iIdx; /* By default use a full table scan. This is an expensive option, ** so search through the constraints to see if a more efficient ** strategy is possible. */ pInfo->idxNum = FTS3_FULLSCAN_SEARCH; pInfo->estimatedCost = 5000000; | > > > > | 1613 1614 1615 1616 1617 1618 1619 1620 1621 1622 1623 1624 1625 1626 1627 1628 1629 1630 | int i; /* Iterator variable */ int iCons = -1; /* Index of constraint to use */ int iLangidCons = -1; /* Index of langid=x constraint, if present */ int iDocidGe = -1; /* Index of docid>=x constraint, if present */ int iDocidLe = -1; /* Index of docid<=x constraint, if present */ int iIdx; if( p->bLock ){ return SQLITE_ERROR; } /* By default use a full table scan. This is an expensive option, ** so search through the constraints to see if a more efficient ** strategy is possible. */ pInfo->idxNum = FTS3_FULLSCAN_SEARCH; pInfo->estimatedCost = 5000000; |
︙ | ︙ | |||
1749 1750 1751 1752 1753 1754 1755 | char *zSql; if( p->pSeekStmt ){ pCsr->pStmt = p->pSeekStmt; p->pSeekStmt = 0; }else{ zSql = sqlite3_mprintf("SELECT %s WHERE rowid = ?", p->zReadExprlist); if( !zSql ) return SQLITE_NOMEM; | > > | > > > > > > | 1816 1817 1818 1819 1820 1821 1822 1823 1824 1825 1826 1827 1828 1829 1830 1831 1832 1833 1834 1835 1836 1837 1838 1839 1840 1841 1842 1843 1844 1845 1846 1847 1848 1849 1850 1851 1852 1853 1854 1855 1856 1857 1858 1859 1860 | char *zSql; if( p->pSeekStmt ){ pCsr->pStmt = p->pSeekStmt; p->pSeekStmt = 0; }else{ zSql = sqlite3_mprintf("SELECT %s WHERE rowid = ?", p->zReadExprlist); if( !zSql ) return SQLITE_NOMEM; p->bLock++; rc = sqlite3_prepare_v3( p->db, zSql,-1,SQLITE_PREPARE_PERSISTENT,&pCsr->pStmt,0 ); p->bLock--; sqlite3_free(zSql); } if( rc==SQLITE_OK ) pCsr->bSeekStmt = 1; } return rc; } /* ** Position the pCsr->pStmt statement so that it is on the row ** of the %_content table that contains the last match. Return ** SQLITE_OK on success. */ static int fts3CursorSeek(sqlite3_context *pContext, Fts3Cursor *pCsr){ int rc = SQLITE_OK; if( pCsr->isRequireSeek ){ rc = fts3CursorSeekStmt(pCsr); if( rc==SQLITE_OK ){ Fts3Table *pTab = (Fts3Table*)pCsr->base.pVtab; pTab->bLock++; sqlite3_bind_int64(pCsr->pStmt, 1, pCsr->iPrevId); pCsr->isRequireSeek = 0; if( SQLITE_ROW==sqlite3_step(pCsr->pStmt) ){ pTab->bLock--; return SQLITE_OK; }else{ pTab->bLock--; rc = sqlite3_reset(pCsr->pStmt); if( rc==SQLITE_OK && ((Fts3Table *)pCsr->base.pVtab)->zContentTbl==0 ){ /* If no row was found and no error has occurred, then the %_content ** table is missing a row that is present in the full-text index. ** The data structures are corrupt. */ rc = FTS_CORRUPT_VTAB; pCsr->isEof = 1; |
︙ | ︙ | |||
1819 1820 1821 1822 1823 1824 1825 | ){ int rc = SQLITE_OK; /* Return code */ const char *zCsr = zNode; /* Cursor to iterate through node */ const char *zEnd = &zCsr[nNode];/* End of interior node buffer */ char *zBuffer = 0; /* Buffer to load terms into */ i64 nAlloc = 0; /* Size of allocated buffer */ int isFirstTerm = 1; /* True when processing first term on page */ | | > | | < > > > > | | 1894 1895 1896 1897 1898 1899 1900 1901 1902 1903 1904 1905 1906 1907 1908 1909 1910 1911 1912 1913 1914 1915 1916 1917 1918 1919 1920 1921 1922 1923 1924 1925 1926 1927 1928 1929 1930 1931 1932 1933 1934 1935 1936 1937 1938 1939 1940 1941 1942 1943 1944 1945 1946 1947 1948 | ){ int rc = SQLITE_OK; /* Return code */ const char *zCsr = zNode; /* Cursor to iterate through node */ const char *zEnd = &zCsr[nNode];/* End of interior node buffer */ char *zBuffer = 0; /* Buffer to load terms into */ i64 nAlloc = 0; /* Size of allocated buffer */ int isFirstTerm = 1; /* True when processing first term on page */ u64 iChild; /* Block id of child node to descend to */ int nBuffer = 0; /* Total term size */ /* Skip over the 'height' varint that occurs at the start of every ** interior node. Then load the blockid of the left-child of the b-tree ** node into variable iChild. ** ** Even if the data structure on disk is corrupted, this (reading two ** varints from the buffer) does not risk an overread. If zNode is a ** root node, then the buffer comes from a SELECT statement. SQLite does ** not make this guarantee explicitly, but in practice there are always ** either more than 20 bytes of allocated space following the nNode bytes of ** contents, or two zero bytes. Or, if the node is read from the %_segments ** table, then there are always 20 bytes of zeroed padding following the ** nNode bytes of content (see sqlite3Fts3ReadBlock() for details). */ zCsr += sqlite3Fts3GetVarintU(zCsr, &iChild); zCsr += sqlite3Fts3GetVarintU(zCsr, &iChild); if( zCsr>zEnd ){ return FTS_CORRUPT_VTAB; } while( zCsr<zEnd && (piFirst || piLast) ){ int cmp; /* memcmp() result */ int nSuffix; /* Size of term suffix */ int nPrefix = 0; /* Size of term prefix */ /* Load the next term on the node into zBuffer. Use realloc() to expand ** the size of zBuffer if required. */ if( !isFirstTerm ){ zCsr += fts3GetVarint32(zCsr, &nPrefix); if( nPrefix>nBuffer ){ rc = FTS_CORRUPT_VTAB; goto finish_scan; } } isFirstTerm = 0; zCsr += fts3GetVarint32(zCsr, &nSuffix); assert( nPrefix>=0 && nSuffix>=0 ); if( nPrefix>zCsr-zNode || nSuffix>zEnd-zCsr || nSuffix==0 ){ rc = FTS_CORRUPT_VTAB; goto finish_scan; } if( (i64)nPrefix+nSuffix>nAlloc ){ char *zNew; nAlloc = ((i64)nPrefix+nSuffix) * 2; zNew = (char *)sqlite3_realloc64(zBuffer, nAlloc); |
︙ | ︙ | |||
1885 1886 1887 1888 1889 1890 1891 | ** iChild. ** ** If the interior node term is larger than the specified term, then ** the tree headed by iChild may contain the specified term. */ cmp = memcmp(zTerm, zBuffer, (nBuffer>nTerm ? nTerm : nBuffer)); if( piFirst && (cmp<0 || (cmp==0 && nBuffer>nTerm)) ){ | | | | | | 1964 1965 1966 1967 1968 1969 1970 1971 1972 1973 1974 1975 1976 1977 1978 1979 1980 1981 1982 1983 1984 1985 1986 1987 1988 1989 1990 1991 | ** iChild. ** ** If the interior node term is larger than the specified term, then ** the tree headed by iChild may contain the specified term. */ cmp = memcmp(zTerm, zBuffer, (nBuffer>nTerm ? nTerm : nBuffer)); if( piFirst && (cmp<0 || (cmp==0 && nBuffer>nTerm)) ){ *piFirst = (i64)iChild; piFirst = 0; } if( piLast && cmp<0 ){ *piLast = (i64)iChild; piLast = 0; } iChild++; }; if( piFirst ) *piFirst = (i64)iChild; if( piLast ) *piLast = (i64)iChild; finish_scan: sqlite3_free(zBuffer); return rc; } |
︙ | ︙ | |||
1943 1944 1945 1946 1947 1948 1949 | int rc = SQLITE_OK; /* Return code */ int iHeight; /* Height of this node in tree */ assert( piLeaf || piLeaf2 ); fts3GetVarint32(zNode, &iHeight); rc = fts3ScanInteriorNode(zTerm, nTerm, zNode, nNode, piLeaf, piLeaf2); | | > > > > > | > | | 2022 2023 2024 2025 2026 2027 2028 2029 2030 2031 2032 2033 2034 2035 2036 2037 2038 2039 2040 2041 2042 2043 2044 2045 2046 2047 2048 2049 2050 2051 2052 2053 2054 2055 2056 2057 2058 2059 2060 2061 2062 2063 2064 2065 2066 2067 2068 2069 2070 2071 2072 2073 2074 2075 2076 2077 2078 2079 | int rc = SQLITE_OK; /* Return code */ int iHeight; /* Height of this node in tree */ assert( piLeaf || piLeaf2 ); fts3GetVarint32(zNode, &iHeight); rc = fts3ScanInteriorNode(zTerm, nTerm, zNode, nNode, piLeaf, piLeaf2); assert_fts3_nc( !piLeaf2 || !piLeaf || rc!=SQLITE_OK || (*piLeaf<=*piLeaf2) ); if( rc==SQLITE_OK && iHeight>1 ){ char *zBlob = 0; /* Blob read from %_segments table */ int nBlob = 0; /* Size of zBlob in bytes */ if( piLeaf && piLeaf2 && (*piLeaf!=*piLeaf2) ){ rc = sqlite3Fts3ReadBlock(p, *piLeaf, &zBlob, &nBlob, 0); if( rc==SQLITE_OK ){ rc = fts3SelectLeaf(p, zTerm, nTerm, zBlob, nBlob, piLeaf, 0); } sqlite3_free(zBlob); piLeaf = 0; zBlob = 0; } if( rc==SQLITE_OK ){ rc = sqlite3Fts3ReadBlock(p, piLeaf?*piLeaf:*piLeaf2, &zBlob, &nBlob, 0); } if( rc==SQLITE_OK ){ int iNewHeight = 0; fts3GetVarint32(zBlob, &iNewHeight); if( iNewHeight>=iHeight ){ rc = FTS_CORRUPT_VTAB; }else{ rc = fts3SelectLeaf(p, zTerm, nTerm, zBlob, nBlob, piLeaf, piLeaf2); } } sqlite3_free(zBlob); } return rc; } /* ** This function is used to create delta-encoded serialized lists of FTS3 ** varints. Each call to this function appends a single varint to a list. */ static void fts3PutDeltaVarint( char **pp, /* IN/OUT: Output pointer */ sqlite3_int64 *piPrev, /* IN/OUT: Previous value written to list */ sqlite3_int64 iVal /* Write this value to the list */ ){ assert_fts3_nc( iVal-*piPrev > 0 || (*piPrev==0 && iVal==0) ); *pp += sqlite3Fts3PutVarint(*pp, iVal-*piPrev); *piPrev = iVal; } /* ** When this function is called, *ppPoslist is assumed to point to the ** start of a position-list. After it returns, *ppPoslist points to the |
︙ | ︙ | |||
2068 2069 2070 2071 2072 2073 2074 | p += n; *pp = p; } *ppPoslist = pEnd; } /* | | | > | | 2153 2154 2155 2156 2157 2158 2159 2160 2161 2162 2163 2164 2165 2166 2167 2168 2169 2170 2171 | p += n; *pp = p; } *ppPoslist = pEnd; } /* ** Value used to signify the end of an position-list. This must be ** as large or larger than any value that might appear on the ** position-list, even a position list that has been corrupted. */ #define POSITION_LIST_END LARGEST_INT64 /* ** This function is used to help parse position-lists. When this function is ** called, *pp may point to the start of the next varint in the position-list ** being parsed, or it may point to 1 byte past the end of the position-list ** (in which case **pp will be a terminator bytes POS_END (0) or ** (1)). |
︙ | ︙ | |||
2096 2097 2098 2099 2100 2101 2102 | ** the next position. */ static void fts3ReadNextPos( char **pp, /* IN/OUT: Pointer into position-list buffer */ sqlite3_int64 *pi /* IN/OUT: Value read from position-list */ ){ if( (**pp)&0xFE ){ | > | > | 2182 2183 2184 2185 2186 2187 2188 2189 2190 2191 2192 2193 2194 2195 2196 2197 2198 | ** the next position. */ static void fts3ReadNextPos( char **pp, /* IN/OUT: Pointer into position-list buffer */ sqlite3_int64 *pi /* IN/OUT: Value read from position-list */ ){ if( (**pp)&0xFE ){ int iVal; *pp += fts3GetVarint32((*pp), &iVal); *pi += iVal; *pi -= 2; }else{ *pi = POSITION_LIST_END; } } /* |
︙ | ︙ | |||
2130 2131 2132 2133 2134 2135 2136 | /* ** Compute the union of two position lists. The output written ** into *pp contains all positions of both *pp1 and *pp2 in sorted ** order and with any duplicates removed. All pointers are ** updated appropriately. The caller is responsible for insuring ** that there is enough space in *pp to hold the complete output. */ | | | > > > | | > > > | | 2218 2219 2220 2221 2222 2223 2224 2225 2226 2227 2228 2229 2230 2231 2232 2233 2234 2235 2236 2237 2238 2239 2240 2241 2242 2243 2244 2245 2246 2247 2248 2249 2250 2251 2252 2253 2254 2255 2256 | /* ** Compute the union of two position lists. The output written ** into *pp contains all positions of both *pp1 and *pp2 in sorted ** order and with any duplicates removed. All pointers are ** updated appropriately. The caller is responsible for insuring ** that there is enough space in *pp to hold the complete output. */ static int fts3PoslistMerge( char **pp, /* Output buffer */ char **pp1, /* Left input list */ char **pp2 /* Right input list */ ){ char *p = *pp; char *p1 = *pp1; char *p2 = *pp2; while( *p1 || *p2 ){ int iCol1; /* The current column index in pp1 */ int iCol2; /* The current column index in pp2 */ if( *p1==POS_COLUMN ){ fts3GetVarint32(&p1[1], &iCol1); if( iCol1==0 ) return FTS_CORRUPT_VTAB; } else if( *p1==POS_END ) iCol1 = 0x7fffffff; else iCol1 = 0; if( *p2==POS_COLUMN ){ fts3GetVarint32(&p2[1], &iCol2); if( iCol2==0 ) return FTS_CORRUPT_VTAB; } else if( *p2==POS_END ) iCol2 = 0x7fffffff; else iCol2 = 0; if( iCol1==iCol2 ){ sqlite3_int64 i1 = 0; /* Last position from pp1 */ sqlite3_int64 i2 = 0; /* Last position from pp2 */ sqlite3_int64 iPrev = 0; int n = fts3PutColNumber(&p, iCol1); |
︙ | ︙ | |||
2170 2171 2172 2173 2174 2175 2176 2177 2178 2179 2180 2181 2182 2183 | ** POS_END (0) or POS_COLUMN (1). The following block merges the two lists ** and writes the results to buffer p. p is left pointing to the byte ** after the list written. No terminator (POS_END or POS_COLUMN) is ** written to the output. */ fts3GetDeltaVarint(&p1, &i1); fts3GetDeltaVarint(&p2, &i2); do { fts3PutDeltaVarint(&p, &iPrev, (i1<i2) ? i1 : i2); iPrev -= 2; if( i1==i2 ){ fts3ReadNextPos(&p1, &i1); fts3ReadNextPos(&p2, &i2); }else if( i1<i2 ){ | > > > | 2264 2265 2266 2267 2268 2269 2270 2271 2272 2273 2274 2275 2276 2277 2278 2279 2280 | ** POS_END (0) or POS_COLUMN (1). The following block merges the two lists ** and writes the results to buffer p. p is left pointing to the byte ** after the list written. No terminator (POS_END or POS_COLUMN) is ** written to the output. */ fts3GetDeltaVarint(&p1, &i1); fts3GetDeltaVarint(&p2, &i2); if( i1<2 || i2<2 ){ break; } do { fts3PutDeltaVarint(&p, &iPrev, (i1<i2) ? i1 : i2); iPrev -= 2; if( i1==i2 ){ fts3ReadNextPos(&p1, &i1); fts3ReadNextPos(&p2, &i2); }else if( i1<i2 ){ |
︙ | ︙ | |||
2195 2196 2197 2198 2199 2200 2201 2202 2203 2204 2205 2206 2207 2208 | } } *p++ = POS_END; *pp = p; *pp1 = p1 + 1; *pp2 = p2 + 1; } /* ** This function is used to merge two position lists into one. When it is ** called, *pp1 and *pp2 must both point to position lists. A position-list is ** the part of a doclist that follows each document id. For example, if a row ** contains: | > | 2292 2293 2294 2295 2296 2297 2298 2299 2300 2301 2302 2303 2304 2305 2306 | } } *p++ = POS_END; *pp = p; *pp1 = p1 + 1; *pp2 = p2 + 1; return SQLITE_OK; } /* ** This function is used to merge two position lists into one. When it is ** called, *pp1 and *pp2 must both point to position lists. A position-list is ** the part of a doclist that follows each document id. For example, if a row ** contains: |
︙ | ︙ | |||
2237 2238 2239 2240 2241 2242 2243 | char *p2 = *pp2; int iCol1 = 0; int iCol2 = 0; /* Never set both isSaveLeft and isExact for the same invocation. */ assert( isSaveLeft==0 || isExact==0 ); | | | 2335 2336 2337 2338 2339 2340 2341 2342 2343 2344 2345 2346 2347 2348 2349 | char *p2 = *pp2; int iCol1 = 0; int iCol2 = 0; /* Never set both isSaveLeft and isExact for the same invocation. */ assert( isSaveLeft==0 || isExact==0 ); assert_fts3_nc( p!=0 && *p1!=0 && *p2!=0 ); if( *p1==POS_COLUMN ){ p1++; p1 += fts3GetVarint32(p1, &iCol1); } if( *p2==POS_COLUMN ){ p2++; p2 += fts3GetVarint32(p2, &iCol2); |
︙ | ︙ | |||
2259 2260 2261 2262 2263 2264 2265 | sqlite3_int64 iPos2 = 0; if( iCol1 ){ *p++ = POS_COLUMN; p += sqlite3Fts3PutVarint(p, iCol1); } | < < > | 2357 2358 2359 2360 2361 2362 2363 2364 2365 2366 2367 2368 2369 2370 2371 2372 2373 | sqlite3_int64 iPos2 = 0; if( iCol1 ){ *p++ = POS_COLUMN; p += sqlite3Fts3PutVarint(p, iCol1); } fts3GetDeltaVarint(&p1, &iPos1); iPos1 -= 2; fts3GetDeltaVarint(&p2, &iPos2); iPos2 -= 2; if( iPos1<0 || iPos2<0 ) break; while( 1 ){ if( iPos2==iPos1+nToken || (isExact==0 && iPos2>iPos1 && iPos2<=iPos1+nToken) ){ sqlite3_int64 iSave; iSave = isSaveLeft ? iPos1 : iPos2; |
︙ | ︙ | |||
2411 2412 2413 2414 2415 2416 2417 | char *pEnd, /* End of buffer */ int bDescIdx, /* True if docids are descending */ sqlite3_int64 *pVal /* IN/OUT: Integer value */ ){ if( *pp>=pEnd ){ *pp = 0; }else{ | | | | | | 2508 2509 2510 2511 2512 2513 2514 2515 2516 2517 2518 2519 2520 2521 2522 2523 2524 2525 2526 2527 | char *pEnd, /* End of buffer */ int bDescIdx, /* True if docids are descending */ sqlite3_int64 *pVal /* IN/OUT: Integer value */ ){ if( *pp>=pEnd ){ *pp = 0; }else{ u64 iVal; *pp += sqlite3Fts3GetVarintU(*pp, &iVal); if( bDescIdx ){ *pVal = (i64)((u64)*pVal - iVal); }else{ *pVal = (i64)((u64)*pVal + iVal); } } } /* ** This function is used to write a single varint to a buffer. The varint ** is written to *pp. Before returning, *pp is set to point 1 byte past the |
︙ | ︙ | |||
2443 2444 2445 2446 2447 2448 2449 | static void fts3PutDeltaVarint3( char **pp, /* IN/OUT: Output pointer */ int bDescIdx, /* True for descending docids */ sqlite3_int64 *piPrev, /* IN/OUT: Previous value written to list */ int *pbFirst, /* IN/OUT: True after first int written */ sqlite3_int64 iVal /* Write this value to the list */ ){ | | > | > | | > | | 2540 2541 2542 2543 2544 2545 2546 2547 2548 2549 2550 2551 2552 2553 2554 2555 2556 2557 2558 2559 2560 2561 2562 2563 2564 2565 2566 2567 2568 2569 2570 2571 2572 2573 2574 2575 2576 2577 2578 2579 2580 | static void fts3PutDeltaVarint3( char **pp, /* IN/OUT: Output pointer */ int bDescIdx, /* True for descending docids */ sqlite3_int64 *piPrev, /* IN/OUT: Previous value written to list */ int *pbFirst, /* IN/OUT: True after first int written */ sqlite3_int64 iVal /* Write this value to the list */ ){ sqlite3_uint64 iWrite; if( bDescIdx==0 || *pbFirst==0 ){ assert_fts3_nc( *pbFirst==0 || iVal>=*piPrev ); iWrite = (u64)iVal - (u64)*piPrev; }else{ assert_fts3_nc( *piPrev>=iVal ); iWrite = (u64)*piPrev - (u64)iVal; } assert( *pbFirst || *piPrev==0 ); assert_fts3_nc( *pbFirst==0 || iWrite>0 ); *pp += sqlite3Fts3PutVarint(*pp, iWrite); *piPrev = iVal; *pbFirst = 1; } /* ** This macro is used by various functions that merge doclists. The two ** arguments are 64-bit docid values. If the value of the stack variable ** bDescDoclist is 0 when this macro is invoked, then it returns (i1-i2). ** Otherwise, (i2-i1). ** ** Using this makes it easier to write code that can merge doclists that are ** sorted in either ascending or descending order. */ /* #define DOCID_CMP(i1, i2) ((bDescDoclist?-1:1) * (i64)((u64)i1-i2)) */ #define DOCID_CMP(i1, i2) ((bDescDoclist?-1:1) * (i1>i2?1:((i1==i2)?0:-1))) /* ** This function does an "OR" merge of two doclists (output contains all ** positions contained in either argument doclist). If the docids in the ** input doclists are sorted in ascending order, parameter bDescDoclist ** should be false. If they are sorted in ascending order, it should be ** passed a non-zero value. |
︙ | ︙ | |||
2488 2489 2490 2491 2492 2493 2494 2495 2496 2497 2498 2499 2500 2501 | */ static int fts3DoclistOrMerge( int bDescDoclist, /* True if arguments are desc */ char *a1, int n1, /* First doclist */ char *a2, int n2, /* Second doclist */ char **paOut, int *pnOut /* OUT: Malloc'd doclist */ ){ sqlite3_int64 i1 = 0; sqlite3_int64 i2 = 0; sqlite3_int64 iPrev = 0; char *pEnd1 = &a1[n1]; char *pEnd2 = &a2[n2]; char *p1 = a1; char *p2 = a2; | > | 2588 2589 2590 2591 2592 2593 2594 2595 2596 2597 2598 2599 2600 2601 2602 | */ static int fts3DoclistOrMerge( int bDescDoclist, /* True if arguments are desc */ char *a1, int n1, /* First doclist */ char *a2, int n2, /* Second doclist */ char **paOut, int *pnOut /* OUT: Malloc'd doclist */ ){ int rc = SQLITE_OK; sqlite3_int64 i1 = 0; sqlite3_int64 i2 = 0; sqlite3_int64 iPrev = 0; char *pEnd1 = &a1[n1]; char *pEnd2 = &a2[n2]; char *p1 = a1; char *p2 = a2; |
︙ | ︙ | |||
2531 2532 2533 2534 2535 2536 2537 | ** The space required to store the output is therefore the sum of the ** sizes of the two inputs, plus enough space for exactly one of the input ** docids to grow. ** ** A symetric argument may be made if the doclists are in descending ** order. */ | | | > | > | > > > > > > > > < | | 2632 2633 2634 2635 2636 2637 2638 2639 2640 2641 2642 2643 2644 2645 2646 2647 2648 2649 2650 2651 2652 2653 2654 2655 2656 2657 2658 2659 2660 2661 2662 2663 2664 2665 2666 2667 2668 2669 2670 2671 2672 2673 2674 2675 2676 2677 2678 2679 2680 2681 2682 2683 | ** The space required to store the output is therefore the sum of the ** sizes of the two inputs, plus enough space for exactly one of the input ** docids to grow. ** ** A symetric argument may be made if the doclists are in descending ** order. */ aOut = sqlite3_malloc64((i64)n1+n2+FTS3_VARINT_MAX-1+FTS3_BUFFER_PADDING); if( !aOut ) return SQLITE_NOMEM; p = aOut; fts3GetDeltaVarint3(&p1, pEnd1, 0, &i1); fts3GetDeltaVarint3(&p2, pEnd2, 0, &i2); while( p1 || p2 ){ sqlite3_int64 iDiff = DOCID_CMP(i1, i2); if( p2 && p1 && iDiff==0 ){ fts3PutDeltaVarint3(&p, bDescDoclist, &iPrev, &bFirstOut, i1); rc = fts3PoslistMerge(&p, &p1, &p2); if( rc ) break; fts3GetDeltaVarint3(&p1, pEnd1, bDescDoclist, &i1); fts3GetDeltaVarint3(&p2, pEnd2, bDescDoclist, &i2); }else if( !p2 || (p1 && iDiff<0) ){ fts3PutDeltaVarint3(&p, bDescDoclist, &iPrev, &bFirstOut, i1); fts3PoslistCopy(&p, &p1); fts3GetDeltaVarint3(&p1, pEnd1, bDescDoclist, &i1); }else{ fts3PutDeltaVarint3(&p, bDescDoclist, &iPrev, &bFirstOut, i2); fts3PoslistCopy(&p, &p2); fts3GetDeltaVarint3(&p2, pEnd2, bDescDoclist, &i2); } assert( (p-aOut)<=((p1?(p1-a1):n1)+(p2?(p2-a2):n2)+FTS3_VARINT_MAX-1) ); } if( rc!=SQLITE_OK ){ sqlite3_free(aOut); p = aOut = 0; }else{ assert( (p-aOut)<=n1+n2+FTS3_VARINT_MAX-1 ); memset(&aOut[(p-aOut)], 0, FTS3_BUFFER_PADDING); } *paOut = aOut; *pnOut = (int)(p-aOut); return rc; } /* ** This function does a "phrase" merge of two doclists. In a phrase merge, ** the output contains a copy of each position from the right-hand input ** doclist for which there is a position in the left-hand input doclist ** exactly nDist tokens before it. |
︙ | ︙ | |||
2594 2595 2596 2597 2598 2599 2600 | char *p2 = aRight; char *p; int bFirstOut = 0; char *aOut; assert( nDist>0 ); if( bDescDoclist ){ | | | 2704 2705 2706 2707 2708 2709 2710 2711 2712 2713 2714 2715 2716 2717 2718 | char *p2 = aRight; char *p; int bFirstOut = 0; char *aOut; assert( nDist>0 ); if( bDescDoclist ){ aOut = sqlite3_malloc64((sqlite3_int64)*pnRight + FTS3_VARINT_MAX); if( aOut==0 ) return SQLITE_NOMEM; }else{ aOut = aRight; } p = aOut; fts3GetDeltaVarint3(&p1, pEnd1, 0, &i1); |
︙ | ︙ | |||
2774 2775 2776 2777 2778 2779 2780 | ** not true for order=DESC. For example, a doclist containing (1, -1) ** may be smaller than (-1), as in the first example the -1 may be stored ** as a single-byte delta, whereas in the second it must be stored as a ** FTS3_VARINT_MAX byte varint. ** ** Similar padding is added in the fts3DoclistOrMerge() function. */ | | > | 2884 2885 2886 2887 2888 2889 2890 2891 2892 2893 2894 2895 2896 2897 2898 2899 2900 2901 2902 | ** not true for order=DESC. For example, a doclist containing (1, -1) ** may be smaller than (-1), as in the first example the -1 may be stored ** as a single-byte delta, whereas in the second it must be stored as a ** FTS3_VARINT_MAX byte varint. ** ** Similar padding is added in the fts3DoclistOrMerge() function. */ pTS->aaOutput[0] = sqlite3_malloc64((i64)nDoclist + FTS3_VARINT_MAX + 1); pTS->anOutput[0] = nDoclist; if( pTS->aaOutput[0] ){ memcpy(pTS->aaOutput[0], aDoclist, nDoclist); memset(&pTS->aaOutput[0][nDoclist], 0, FTS3_VARINT_MAX); }else{ return SQLITE_NOMEM; } }else{ char *aMerge = aDoclist; int nMerge = nDoclist; int iOut; |
︙ | ︙ | |||
2829 2830 2831 2832 2833 2834 2835 | */ static int fts3SegReaderCursorAppend( Fts3MultiSegReader *pCsr, Fts3SegReader *pNew ){ if( (pCsr->nSegment%16)==0 ){ Fts3SegReader **apNew; | | | | 2940 2941 2942 2943 2944 2945 2946 2947 2948 2949 2950 2951 2952 2953 2954 2955 | */ static int fts3SegReaderCursorAppend( Fts3MultiSegReader *pCsr, Fts3SegReader *pNew ){ if( (pCsr->nSegment%16)==0 ){ Fts3SegReader **apNew; sqlite3_int64 nByte = (pCsr->nSegment + 16)*sizeof(Fts3SegReader*); apNew = (Fts3SegReader **)sqlite3_realloc64(pCsr->apSegment, nByte); if( !apNew ){ sqlite3Fts3SegReaderFree(pNew); return SQLITE_NOMEM; } pCsr->apSegment = apNew; } pCsr->apSegment[pCsr->nSegment++] = pNew; |
︙ | ︙ | |||
2869 2870 2871 2872 2873 2874 2875 | /* If iLevel is less than 0 and this is not a scan, include a seg-reader ** for the pending-terms. If this is a scan, then this call must be being ** made by an fts4aux module, not an FTS table. In this case calling ** Fts3SegReaderPending might segfault, as the data structures used by ** fts4aux are not completely populated. So it's easiest to filter these ** calls out here. */ | | | 2980 2981 2982 2983 2984 2985 2986 2987 2988 2989 2990 2991 2992 2993 2994 | /* If iLevel is less than 0 and this is not a scan, include a seg-reader ** for the pending-terms. If this is a scan, then this call must be being ** made by an fts4aux module, not an FTS table. In this case calling ** Fts3SegReaderPending might segfault, as the data structures used by ** fts4aux are not completely populated. So it's easiest to filter these ** calls out here. */ if( iLevel<0 && p->aIndex && p->iPrevLangid==iLangid ){ Fts3SegReader *pSeg = 0; rc = sqlite3Fts3SegReaderPending(p, iIndex, zTerm, nTerm, isPrefix||isScan, &pSeg); if( rc==SQLITE_OK && pSeg ){ rc = fts3SegReaderCursorAppend(pCsr, pSeg); } } |
︙ | ︙ | |||
2894 2895 2896 2897 2898 2899 2900 | sqlite3_int64 iLeavesEndBlock = sqlite3_column_int64(pStmt, 2); sqlite3_int64 iEndBlock = sqlite3_column_int64(pStmt, 3); int nRoot = sqlite3_column_bytes(pStmt, 4); char const *zRoot = sqlite3_column_blob(pStmt, 4); /* If zTerm is not NULL, and this segment is not stored entirely on its ** root node, the range of leaves scanned can be reduced. Do this. */ | | | 3005 3006 3007 3008 3009 3010 3011 3012 3013 3014 3015 3016 3017 3018 3019 | sqlite3_int64 iLeavesEndBlock = sqlite3_column_int64(pStmt, 2); sqlite3_int64 iEndBlock = sqlite3_column_int64(pStmt, 3); int nRoot = sqlite3_column_bytes(pStmt, 4); char const *zRoot = sqlite3_column_blob(pStmt, 4); /* If zTerm is not NULL, and this segment is not stored entirely on its ** root node, the range of leaves scanned can be reduced. Do this. */ if( iStartBlock && zTerm && zRoot ){ sqlite3_int64 *pi = (isPrefix ? &iLeavesEndBlock : 0); rc = fts3SelectLeaf(p, zTerm, nTerm, zRoot, nRoot, &iStartBlock, pi); if( rc!=SQLITE_OK ) goto finished; if( isPrefix==0 && isScan==0 ) iLeavesEndBlock = iStartBlock; } rc = sqlite3Fts3SegReaderNew(pCsr->nSegment+1, |
︙ | ︙ | |||
3132 3133 3134 3135 3136 3137 3138 3139 3140 3141 3142 3143 3144 3145 3146 3147 3148 3149 3150 3151 3152 | ** even if we reach end-of-file. The fts3EofMethod() will be called ** subsequently to determine whether or not an EOF was hit. */ static int fts3NextMethod(sqlite3_vtab_cursor *pCursor){ int rc; Fts3Cursor *pCsr = (Fts3Cursor *)pCursor; if( pCsr->eSearch==FTS3_DOCID_SEARCH || pCsr->eSearch==FTS3_FULLSCAN_SEARCH ){ if( SQLITE_ROW!=sqlite3_step(pCsr->pStmt) ){ pCsr->isEof = 1; rc = sqlite3_reset(pCsr->pStmt); }else{ pCsr->iPrevId = sqlite3_column_int64(pCsr->pStmt, 0); rc = SQLITE_OK; } }else{ rc = fts3EvalNext((Fts3Cursor *)pCursor); } assert( ((Fts3Table *)pCsr->base.pVtab)->pSegments==0 ); return rc; } | > > > < < < < < < < < < < < < | 3243 3244 3245 3246 3247 3248 3249 3250 3251 3252 3253 3254 3255 3256 3257 3258 3259 3260 3261 3262 3263 3264 3265 3266 3267 3268 3269 3270 3271 3272 3273 | ** even if we reach end-of-file. The fts3EofMethod() will be called ** subsequently to determine whether or not an EOF was hit. */ static int fts3NextMethod(sqlite3_vtab_cursor *pCursor){ int rc; Fts3Cursor *pCsr = (Fts3Cursor *)pCursor; if( pCsr->eSearch==FTS3_DOCID_SEARCH || pCsr->eSearch==FTS3_FULLSCAN_SEARCH ){ Fts3Table *pTab = (Fts3Table*)pCursor->pVtab; pTab->bLock++; if( SQLITE_ROW!=sqlite3_step(pCsr->pStmt) ){ pCsr->isEof = 1; rc = sqlite3_reset(pCsr->pStmt); }else{ pCsr->iPrevId = sqlite3_column_int64(pCsr->pStmt, 0); rc = SQLITE_OK; } pTab->bLock--; }else{ rc = fts3EvalNext((Fts3Cursor *)pCursor); } assert( ((Fts3Table *)pCsr->base.pVtab)->pSegments==0 ); return rc; } /* ** If the numeric type of argument pVal is "integer", then return it ** converted to a 64-bit signed integer. Otherwise, return a copy of ** the second parameter, iDefault. */ static sqlite3_int64 fts3DocidRange(sqlite3_value *pVal, i64 iDefault){ if( pVal ){ |
︙ | ︙ | |||
3210 3211 3212 3213 3214 3215 3216 3217 3218 3219 3220 3221 3222 3223 | sqlite3_value *pLangid = 0; /* The "langid = ?" constraint, if any */ sqlite3_value *pDocidGe = 0; /* The "docid >= ?" constraint, if any */ sqlite3_value *pDocidLe = 0; /* The "docid <= ?" constraint, if any */ int iIdx; UNUSED_PARAMETER(idxStr); UNUSED_PARAMETER(nVal); eSearch = (idxNum & 0x0000FFFF); assert( eSearch>=0 && eSearch<=(FTS3_FULLTEXT_SEARCH+p->nColumn) ); assert( p->pSegments==0 ); /* Collect arguments into local variables */ iIdx = 0; | > > > > | 3312 3313 3314 3315 3316 3317 3318 3319 3320 3321 3322 3323 3324 3325 3326 3327 3328 3329 | sqlite3_value *pLangid = 0; /* The "langid = ?" constraint, if any */ sqlite3_value *pDocidGe = 0; /* The "docid >= ?" constraint, if any */ sqlite3_value *pDocidLe = 0; /* The "docid <= ?" constraint, if any */ int iIdx; UNUSED_PARAMETER(idxStr); UNUSED_PARAMETER(nVal); if( p->bLock ){ return SQLITE_ERROR; } eSearch = (idxNum & 0x0000FFFF); assert( eSearch>=0 && eSearch<=(FTS3_FULLTEXT_SEARCH+p->nColumn) ); assert( p->pSegments==0 ); /* Collect arguments into local variables */ iIdx = 0; |
︙ | ︙ | |||
3282 3283 3284 3285 3286 3287 3288 | ); }else{ zSql = sqlite3_mprintf("SELECT %s ORDER BY rowid %s", p->zReadExprlist, (pCsr->bDesc ? "DESC" : "ASC") ); } if( zSql ){ | > > | > > | 3388 3389 3390 3391 3392 3393 3394 3395 3396 3397 3398 3399 3400 3401 3402 3403 3404 3405 3406 | ); }else{ zSql = sqlite3_mprintf("SELECT %s ORDER BY rowid %s", p->zReadExprlist, (pCsr->bDesc ? "DESC" : "ASC") ); } if( zSql ){ p->bLock++; rc = sqlite3_prepare_v3( p->db,zSql,-1,SQLITE_PREPARE_PERSISTENT,&pCsr->pStmt,0 ); p->bLock--; sqlite3_free(zSql); }else{ rc = SQLITE_NOMEM; } }else if( eSearch==FTS3_DOCID_SEARCH ){ rc = fts3CursorSeekStmt(pCsr); if( rc==SQLITE_OK ){ |
︙ | ︙ | |||
3366 3367 3368 3369 3370 3371 3372 | sqlite3_result_int64(pCtx, pCsr->iLangid); break; }else if( p->zLanguageid==0 ){ sqlite3_result_int(pCtx, 0); break; }else{ iCol = p->nColumn; | | | 3476 3477 3478 3479 3480 3481 3482 3483 3484 3485 3486 3487 3488 3489 3490 | sqlite3_result_int64(pCtx, pCsr->iLangid); break; }else if( p->zLanguageid==0 ){ sqlite3_result_int(pCtx, 0); break; }else{ iCol = p->nColumn; /* no break */ deliberate_fall_through } default: /* A user column. Or, if this is a full-table scan, possibly the ** language-id column. Seek the cursor. */ rc = fts3CursorSeek(0, pCsr); if( rc==SQLITE_OK && sqlite3_data_count(pCsr->pStmt)-1>iCol ){ |
︙ | ︙ | |||
3473 3474 3475 3476 3477 3478 3479 3480 3481 3482 3483 | } /* ** Implementation of xBegin() method. */ static int fts3BeginMethod(sqlite3_vtab *pVtab){ Fts3Table *p = (Fts3Table*)pVtab; UNUSED_PARAMETER(pVtab); assert( p->pSegments==0 ); assert( p->nPendingData==0 ); assert( p->inTransaction!=1 ); | > > > > > | | < > > | | 3583 3584 3585 3586 3587 3588 3589 3590 3591 3592 3593 3594 3595 3596 3597 3598 3599 3600 3601 3602 3603 3604 3605 3606 3607 3608 3609 3610 | } /* ** Implementation of xBegin() method. */ static int fts3BeginMethod(sqlite3_vtab *pVtab){ Fts3Table *p = (Fts3Table*)pVtab; int rc; UNUSED_PARAMETER(pVtab); assert( p->pSegments==0 ); assert( p->nPendingData==0 ); assert( p->inTransaction!=1 ); p->nLeafAdd = 0; rc = fts3SetHasStat(p); #ifdef SQLITE_DEBUG if( rc==SQLITE_OK ){ p->inTransaction = 1; p->mxSavepoint = -1; } #endif return rc; } /* ** Implementation of xCommit() method. This is a no-op. The contents of ** the pending-terms hash-table have already been flushed into the database ** by fts3SyncMethod(). */ |
︙ | ︙ | |||
3609 3610 3611 3612 3613 3614 3615 3616 3617 3618 3619 3620 3621 3622 3623 3624 3625 | "wrong number of arguments to function snippet()", -1); return; } if( fts3FunctionArg(pContext, "snippet", apVal[0], &pCsr) ) return; switch( nVal ){ case 6: nToken = sqlite3_value_int(apVal[5]); case 5: iCol = sqlite3_value_int(apVal[4]); case 4: zEllipsis = (const char*)sqlite3_value_text(apVal[3]); case 3: zEnd = (const char*)sqlite3_value_text(apVal[2]); case 2: zStart = (const char*)sqlite3_value_text(apVal[1]); } if( !zEllipsis || !zEnd || !zStart ){ sqlite3_result_error_nomem(pContext); }else if( nToken==0 ){ sqlite3_result_text(pContext, "", -1, SQLITE_STATIC); }else if( SQLITE_OK==fts3CursorSeek(pContext, pCsr) ){ | > > > > | 3725 3726 3727 3728 3729 3730 3731 3732 3733 3734 3735 3736 3737 3738 3739 3740 3741 3742 3743 3744 3745 | "wrong number of arguments to function snippet()", -1); return; } if( fts3FunctionArg(pContext, "snippet", apVal[0], &pCsr) ) return; switch( nVal ){ case 6: nToken = sqlite3_value_int(apVal[5]); /* no break */ deliberate_fall_through case 5: iCol = sqlite3_value_int(apVal[4]); /* no break */ deliberate_fall_through case 4: zEllipsis = (const char*)sqlite3_value_text(apVal[3]); /* no break */ deliberate_fall_through case 3: zEnd = (const char*)sqlite3_value_text(apVal[2]); /* no break */ deliberate_fall_through case 2: zStart = (const char*)sqlite3_value_text(apVal[1]); } if( !zEllipsis || !zEnd || !zStart ){ sqlite3_result_error_nomem(pContext); }else if( nToken==0 ){ sqlite3_result_text(pContext, "", -1, SQLITE_STATIC); }else if( SQLITE_OK==fts3CursorSeek(pContext, pCsr) ){ |
︙ | ︙ | |||
3836 3837 3838 3839 3840 3841 3842 | ** ** Discard the contents of the pending terms table. */ static int fts3RollbackToMethod(sqlite3_vtab *pVtab, int iSavepoint){ Fts3Table *p = (Fts3Table*)pVtab; UNUSED_PARAMETER(iSavepoint); assert( p->inTransaction ); | < | 3956 3957 3958 3959 3960 3961 3962 3963 3964 3965 3966 3967 3968 3969 | ** ** Discard the contents of the pending terms table. */ static int fts3RollbackToMethod(sqlite3_vtab *pVtab, int iSavepoint){ Fts3Table *p = (Fts3Table*)pVtab; UNUSED_PARAMETER(iSavepoint); assert( p->inTransaction ); TESTONLY( p->mxSavepoint = iSavepoint ); sqlite3Fts3PendingTermsClear(p); return SQLITE_OK; } /* ** Return true if zName is the extension on one of the shadow tables used |
︙ | ︙ | |||
3890 3891 3892 3893 3894 3895 3896 | /* ** This function is registered as the module destructor (called when an ** FTS3 enabled database connection is closed). It frees the memory ** allocated for the tokenizer hash table. */ static void hashDestroy(void *p){ | | > > | | > | 4009 4010 4011 4012 4013 4014 4015 4016 4017 4018 4019 4020 4021 4022 4023 4024 4025 4026 4027 4028 | /* ** This function is registered as the module destructor (called when an ** FTS3 enabled database connection is closed). It frees the memory ** allocated for the tokenizer hash table. */ static void hashDestroy(void *p){ Fts3HashWrapper *pHash = (Fts3HashWrapper *)p; pHash->nRef--; if( pHash->nRef<=0 ){ sqlite3Fts3HashClear(&pHash->hash); sqlite3_free(pHash); } } /* ** The fts3 built-in tokenizers - "simple", "porter" and "icu"- are ** implemented in files fts3_tokenizer1.c, fts3_porter.c and fts3_icu.c ** respectively. The following three forward declarations are for functions ** declared in these files used to retrieve the respective implementations. |
︙ | ︙ | |||
3922 3923 3924 3925 3926 3927 3928 | ** Initialize the fts3 extension. If this extension is built as part ** of the sqlite library, then this function is called directly by ** SQLite. If fts3 is built as a dynamically loadable extension, this ** function is called by the sqlite3_extension_init() entry point. */ int sqlite3Fts3Init(sqlite3 *db){ int rc = SQLITE_OK; | | | 4044 4045 4046 4047 4048 4049 4050 4051 4052 4053 4054 4055 4056 4057 4058 | ** Initialize the fts3 extension. If this extension is built as part ** of the sqlite library, then this function is called directly by ** SQLite. If fts3 is built as a dynamically loadable extension, this ** function is called by the sqlite3_extension_init() entry point. */ int sqlite3Fts3Init(sqlite3 *db){ int rc = SQLITE_OK; Fts3HashWrapper *pHash = 0; const sqlite3_tokenizer_module *pSimple = 0; const sqlite3_tokenizer_module *pPorter = 0; #ifndef SQLITE_DISABLE_FTS3_UNICODE const sqlite3_tokenizer_module *pUnicode = 0; #endif #ifdef SQLITE_ENABLE_ICU |
︙ | ︙ | |||
3950 3951 3952 3953 3954 3955 3956 | rc = sqlite3Fts3InitAux(db); if( rc!=SQLITE_OK ) return rc; sqlite3Fts3SimpleTokenizerModule(&pSimple); sqlite3Fts3PorterTokenizerModule(&pPorter); /* Allocate and initialize the hash-table used to store tokenizers. */ | | | > | | | | | | > > | > | | | 4072 4073 4074 4075 4076 4077 4078 4079 4080 4081 4082 4083 4084 4085 4086 4087 4088 4089 4090 4091 4092 4093 4094 4095 4096 4097 4098 4099 4100 4101 4102 4103 4104 4105 4106 4107 4108 4109 4110 4111 4112 4113 4114 4115 4116 4117 4118 4119 4120 4121 4122 4123 4124 4125 4126 4127 4128 4129 4130 4131 4132 4133 4134 4135 4136 4137 4138 4139 4140 4141 4142 4143 4144 4145 4146 4147 4148 4149 | rc = sqlite3Fts3InitAux(db); if( rc!=SQLITE_OK ) return rc; sqlite3Fts3SimpleTokenizerModule(&pSimple); sqlite3Fts3PorterTokenizerModule(&pPorter); /* Allocate and initialize the hash-table used to store tokenizers. */ pHash = sqlite3_malloc(sizeof(Fts3HashWrapper)); if( !pHash ){ rc = SQLITE_NOMEM; }else{ sqlite3Fts3HashInit(&pHash->hash, FTS3_HASH_STRING, 1); pHash->nRef = 0; } /* Load the built-in tokenizers into the hash table */ if( rc==SQLITE_OK ){ if( sqlite3Fts3HashInsert(&pHash->hash, "simple", 7, (void *)pSimple) || sqlite3Fts3HashInsert(&pHash->hash, "porter", 7, (void *)pPorter) #ifndef SQLITE_DISABLE_FTS3_UNICODE || sqlite3Fts3HashInsert(&pHash->hash, "unicode61", 10, (void *)pUnicode) #endif #ifdef SQLITE_ENABLE_ICU || (pIcu && sqlite3Fts3HashInsert(&pHash->hash, "icu", 4, (void *)pIcu)) #endif ){ rc = SQLITE_NOMEM; } } #ifdef SQLITE_TEST if( rc==SQLITE_OK ){ rc = sqlite3Fts3ExprInitTestInterface(db, &pHash->hash); } #endif /* Create the virtual table wrapper around the hash-table and overload ** the four scalar functions. If this is successful, register the ** module with sqlite. */ if( SQLITE_OK==rc && SQLITE_OK==(rc=sqlite3Fts3InitHashTable(db,&pHash->hash,"fts3_tokenizer")) && SQLITE_OK==(rc = sqlite3_overload_function(db, "snippet", -1)) && SQLITE_OK==(rc = sqlite3_overload_function(db, "offsets", 1)) && SQLITE_OK==(rc = sqlite3_overload_function(db, "matchinfo", 1)) && SQLITE_OK==(rc = sqlite3_overload_function(db, "matchinfo", 2)) && SQLITE_OK==(rc = sqlite3_overload_function(db, "optimize", 1)) ){ pHash->nRef++; rc = sqlite3_create_module_v2( db, "fts3", &fts3Module, (void *)pHash, hashDestroy ); if( rc==SQLITE_OK ){ pHash->nRef++; rc = sqlite3_create_module_v2( db, "fts4", &fts3Module, (void *)pHash, hashDestroy ); } if( rc==SQLITE_OK ){ pHash->nRef++; rc = sqlite3Fts3InitTok(db, (void *)pHash, hashDestroy); } return rc; } /* An error has occurred. Delete the hash table and return the error code. */ assert( rc!=SQLITE_OK ); if( pHash ){ sqlite3Fts3HashClear(&pHash->hash); sqlite3_free(pHash); } return rc; } /* ** Allocate an Fts3MultiSegReader for each token in the expression headed |
︙ | ︙ | |||
4178 4179 4180 4181 4182 4183 4184 | ** SQLITE_OK is returned if no error occurs, otherwise an SQLite error code. */ static int fts3EvalDeferredPhrase(Fts3Cursor *pCsr, Fts3Phrase *pPhrase){ int iToken; /* Used to iterate through phrase tokens */ char *aPoslist = 0; /* Position list for deferred tokens */ int nPoslist = 0; /* Number of bytes in aPoslist */ int iPrev = -1; /* Token number of previous deferred token */ | < | > > | 4304 4305 4306 4307 4308 4309 4310 4311 4312 4313 4314 4315 4316 4317 4318 4319 4320 4321 4322 4323 4324 4325 4326 4327 4328 4329 4330 4331 4332 4333 4334 4335 4336 4337 4338 4339 4340 4341 4342 4343 4344 4345 4346 4347 4348 4349 4350 4351 4352 4353 | ** SQLITE_OK is returned if no error occurs, otherwise an SQLite error code. */ static int fts3EvalDeferredPhrase(Fts3Cursor *pCsr, Fts3Phrase *pPhrase){ int iToken; /* Used to iterate through phrase tokens */ char *aPoslist = 0; /* Position list for deferred tokens */ int nPoslist = 0; /* Number of bytes in aPoslist */ int iPrev = -1; /* Token number of previous deferred token */ char *aFree = (pPhrase->doclist.bFreeList ? pPhrase->doclist.pList : 0); for(iToken=0; iToken<pPhrase->nToken; iToken++){ Fts3PhraseToken *pToken = &pPhrase->aToken[iToken]; Fts3DeferredToken *pDeferred = pToken->pDeferred; if( pDeferred ){ char *pList; int nList; int rc = sqlite3Fts3DeferredTokenList(pDeferred, &pList, &nList); if( rc!=SQLITE_OK ) return rc; if( pList==0 ){ sqlite3_free(aPoslist); sqlite3_free(aFree); pPhrase->doclist.pList = 0; pPhrase->doclist.nList = 0; return SQLITE_OK; }else if( aPoslist==0 ){ aPoslist = pList; nPoslist = nList; }else{ char *aOut = pList; char *p1 = aPoslist; char *p2 = aOut; assert( iPrev>=0 ); fts3PoslistPhraseMerge(&aOut, iToken-iPrev, 0, 1, &p1, &p2); sqlite3_free(aPoslist); aPoslist = pList; nPoslist = (int)(aOut - aPoslist); if( nPoslist==0 ){ sqlite3_free(aPoslist); sqlite3_free(aFree); pPhrase->doclist.pList = 0; pPhrase->doclist.nList = 0; return SQLITE_OK; } } iPrev = iToken; } |
︙ | ︙ | |||
4245 4246 4247 4248 4249 4250 4251 | nDistance = nMaxUndeferred - iPrev; }else{ p1 = pPhrase->doclist.pList; p2 = aPoslist; nDistance = iPrev - nMaxUndeferred; } | | > > | 4372 4373 4374 4375 4376 4377 4378 4379 4380 4381 4382 4383 4384 4385 4386 4387 4388 4389 4390 4391 4392 4393 4394 4395 4396 4397 4398 4399 4400 4401 4402 4403 4404 4405 4406 | nDistance = nMaxUndeferred - iPrev; }else{ p1 = pPhrase->doclist.pList; p2 = aPoslist; nDistance = iPrev - nMaxUndeferred; } aOut = (char *)sqlite3Fts3MallocZero(nPoslist+FTS3_BUFFER_PADDING); if( !aOut ){ sqlite3_free(aPoslist); return SQLITE_NOMEM; } pPhrase->doclist.pList = aOut; assert( p1 && p2 ); if( fts3PoslistPhraseMerge(&aOut, nDistance, 0, 1, &p1, &p2) ){ pPhrase->doclist.bFreeList = 1; pPhrase->doclist.nList = (int)(aOut - pPhrase->doclist.pList); }else{ sqlite3_free(aOut); pPhrase->doclist.pList = 0; pPhrase->doclist.nList = 0; } sqlite3_free(aPoslist); } } if( pPhrase->doclist.pList!=aFree ) sqlite3_free(aFree); return SQLITE_OK; } #endif /* SQLITE_DISABLE_FTS4_DEFERRED */ /* ** Maximum number of tokens a phrase may have to be considered for the ** incremental doclists strategy. |
︙ | ︙ | |||
4300 4301 4302 4303 4304 4305 4306 | ** scanned in forward order, and the phrase consists of ** MAX_INCR_PHRASE_TOKENS or fewer tokens, none of which are are "^first" ** tokens or prefix tokens that cannot use a prefix-index. */ int bHaveIncr = 0; int bIncrOk = (bOptOk && pCsr->bDesc==pTab->bDescIdx && p->nToken<=MAX_INCR_PHRASE_TOKENS && p->nToken>0 | | | 4429 4430 4431 4432 4433 4434 4435 4436 4437 4438 4439 4440 4441 4442 4443 | ** scanned in forward order, and the phrase consists of ** MAX_INCR_PHRASE_TOKENS or fewer tokens, none of which are are "^first" ** tokens or prefix tokens that cannot use a prefix-index. */ int bHaveIncr = 0; int bIncrOk = (bOptOk && pCsr->bDesc==pTab->bDescIdx && p->nToken<=MAX_INCR_PHRASE_TOKENS && p->nToken>0 #if defined(SQLITE_DEBUG) || defined(SQLITE_TEST) && pTab->bNoIncrDoclist==0 #endif ); for(i=0; bIncrOk==1 && i<p->nToken; i++){ Fts3PhraseToken *pToken = &p->aToken[i]; if( pToken->bFirst || (pToken->pSegcsr!=0 && !pToken->pSegcsr->bLookup) ){ bIncrOk = 0; |
︙ | ︙ | |||
4356 4357 4358 4359 4360 4361 4362 | int *pnList, /* OUT: List length pointer */ u8 *pbEof /* OUT: End-of-file flag */ ){ char *p = *ppIter; assert( nDoclist>0 ); assert( *pbEof==0 ); | | | 4485 4486 4487 4488 4489 4490 4491 4492 4493 4494 4495 4496 4497 4498 4499 | int *pnList, /* OUT: List length pointer */ u8 *pbEof /* OUT: End-of-file flag */ ){ char *p = *ppIter; assert( nDoclist>0 ); assert( *pbEof==0 ); assert_fts3_nc( p || *piDocid==0 ); assert( !p || (p>aDoclist && p<&aDoclist[nDoclist]) ); if( p==0 ){ sqlite3_int64 iDocid = 0; char *pNext = 0; char *pDocid = aDoclist; char *pEnd = &aDoclist[nDoclist]; |
︙ | ︙ | |||
4411 4412 4413 4414 4415 4416 4417 | sqlite3_int64 *piDocid, /* IN/OUT: Docid pointer */ u8 *pbEof /* OUT: End-of-file flag */ ){ char *p = *ppIter; assert( nDoclist>0 ); assert( *pbEof==0 ); | | | 4540 4541 4542 4543 4544 4545 4546 4547 4548 4549 4550 4551 4552 4553 4554 | sqlite3_int64 *piDocid, /* IN/OUT: Docid pointer */ u8 *pbEof /* OUT: End-of-file flag */ ){ char *p = *ppIter; assert( nDoclist>0 ); assert( *pbEof==0 ); assert_fts3_nc( p || *piDocid==0 ); assert( !p || (p>=aDoclist && p<=&aDoclist[nDoclist]) ); if( p==0 ){ p = aDoclist; p += sqlite3Fts3GetVarint(p, piDocid); }else{ fts3PoslistCopy(0, &p); |
︙ | ︙ | |||
4442 4443 4444 4445 4446 4447 4448 | */ static void fts3EvalDlPhraseNext( Fts3Table *pTab, Fts3Doclist *pDL, u8 *pbEof ){ char *pIter; /* Used to iterate through aAll */ | | > | | 4571 4572 4573 4574 4575 4576 4577 4578 4579 4580 4581 4582 4583 4584 4585 4586 4587 4588 4589 4590 4591 4592 4593 4594 | */ static void fts3EvalDlPhraseNext( Fts3Table *pTab, Fts3Doclist *pDL, u8 *pbEof ){ char *pIter; /* Used to iterate through aAll */ char *pEnd; /* 1 byte past end of aAll */ if( pDL->pNextDocid ){ pIter = pDL->pNextDocid; assert( pDL->aAll!=0 || pIter==0 ); }else{ pIter = pDL->aAll; } if( pIter==0 || pIter>=(pEnd = pDL->aAll + pDL->nAll) ){ /* We have already reached the end of this doclist. EOF. */ *pbEof = 1; }else{ sqlite3_int64 iDelta; pIter += sqlite3Fts3GetVarint(pIter, &iDelta); if( pTab->bDescIdx==0 || pDL->pNextDocid==0 ){ pDL->iDocid += iDelta; |
︙ | ︙ | |||
4611 4612 4613 4614 4615 4616 4617 | } } /* Check if the current entries really are a phrase match */ if( bEof==0 ){ int nList = 0; int nByte = a[p->nToken-1].nList; | | > | 4741 4742 4743 4744 4745 4746 4747 4748 4749 4750 4751 4752 4753 4754 4755 4756 4757 4758 | } } /* Check if the current entries really are a phrase match */ if( bEof==0 ){ int nList = 0; int nByte = a[p->nToken-1].nList; char *aDoclist = sqlite3_malloc64((i64)nByte+FTS3_BUFFER_PADDING); if( !aDoclist ) return SQLITE_NOMEM; memcpy(aDoclist, a[p->nToken-1].pList, nByte+1); memset(&aDoclist[nByte], 0, FTS3_BUFFER_PADDING); for(i=0; i<(p->nToken-1); i++){ if( a[i].bIgnore==0 ){ char *pL = a[i].pList; char *pR = aDoclist; char *pOut = aDoclist; int nDist = p->nToken-1-i; |
︙ | ︙ | |||
4821 4822 4823 4824 4825 4826 4827 | sqlite3_int64 nByte = 0; const char *pEnd; const char *a; rc = sqlite3Fts3SelectDoctotal(p, &pStmt); if( rc!=SQLITE_OK ) return rc; a = sqlite3_column_blob(pStmt, 0); | > | < | | | | > | 4952 4953 4954 4955 4956 4957 4958 4959 4960 4961 4962 4963 4964 4965 4966 4967 4968 4969 4970 4971 4972 | sqlite3_int64 nByte = 0; const char *pEnd; const char *a; rc = sqlite3Fts3SelectDoctotal(p, &pStmt); if( rc!=SQLITE_OK ) return rc; a = sqlite3_column_blob(pStmt, 0); testcase( a==0 ); /* If %_stat.value set to X'' */ if( a ){ pEnd = &a[sqlite3_column_bytes(pStmt, 0)]; a += sqlite3Fts3GetVarintBounded(a, pEnd, &nDoc); while( a<pEnd ){ a += sqlite3Fts3GetVarintBounded(a, pEnd, &nByte); } } if( nDoc==0 || nByte==0 ){ sqlite3_reset(pStmt); return FTS_CORRUPT_VTAB; } pCsr->nDoc = nDoc; |
︙ | ︙ | |||
5003 5004 5005 5006 5007 5008 5009 | /* Allocate a MultiSegReader for each token in the expression. */ fts3EvalAllocateReaders(pCsr, pCsr->pExpr, &nToken, &nOr, &rc); /* Determine which, if any, tokens in the expression should be deferred. */ #ifndef SQLITE_DISABLE_FTS4_DEFERRED if( rc==SQLITE_OK && nToken>1 && pTab->bFts4 ){ Fts3TokenAndCost *aTC; | < | < > | 5135 5136 5137 5138 5139 5140 5141 5142 5143 5144 5145 5146 5147 5148 5149 5150 5151 5152 5153 5154 5155 5156 5157 | /* Allocate a MultiSegReader for each token in the expression. */ fts3EvalAllocateReaders(pCsr, pCsr->pExpr, &nToken, &nOr, &rc); /* Determine which, if any, tokens in the expression should be deferred. */ #ifndef SQLITE_DISABLE_FTS4_DEFERRED if( rc==SQLITE_OK && nToken>1 && pTab->bFts4 ){ Fts3TokenAndCost *aTC; aTC = (Fts3TokenAndCost *)sqlite3_malloc64( sizeof(Fts3TokenAndCost) * nToken + sizeof(Fts3Expr *) * nOr * 2 ); if( !aTC ){ rc = SQLITE_NOMEM; }else{ Fts3Expr **apOr = (Fts3Expr **)&aTC[nToken]; int ii; Fts3TokenAndCost *pTC = aTC; Fts3Expr **ppOr = apOr; fts3EvalTokenCosts(pCsr, 0, pCsr->pExpr, &pTC, &ppOr, &rc); nToken = (int)(pTC-aTC); nOr = (int)(ppOr-apOr); |
︙ | ︙ | |||
5058 5059 5060 5061 5062 5063 5064 | ** the phrase object passed as the fifth argument according to a NEAR ** condition. For example: ** ** abc NEAR/5 "def ghi" ** ** Parameter nNear is passed the NEAR distance of the expression (5 in ** the example above). When this function is called, *paPoslist points to | | | 5189 5190 5191 5192 5193 5194 5195 5196 5197 5198 5199 5200 5201 5202 5203 | ** the phrase object passed as the fifth argument according to a NEAR ** condition. For example: ** ** abc NEAR/5 "def ghi" ** ** Parameter nNear is passed the NEAR distance of the expression (5 in ** the example above). When this function is called, *paPoslist points to ** the position list, and *pnToken is the number of phrase tokens in the ** phrase on the other side of the NEAR operator to pPhrase. For example, ** if pPhrase refers to the "def ghi" phrase, then *paPoslist points to ** the position list associated with phrase "abc". ** ** All positions in the pPhrase position list that are not sufficiently ** close to a position in the *paPoslist position list are removed. If this ** leaves 0 positions, zero is returned. Otherwise, non-zero. |
︙ | ︙ | |||
5093 5094 5095 5096 5097 5098 5099 | p2 = pOut = pPhrase->doclist.pList; res = fts3PoslistNearMerge( &pOut, aTmp, nParam1, nParam2, paPoslist, &p2 ); if( res ){ nNew = (int)(pOut - pPhrase->doclist.pList) - 1; | > > | < | | > | 5224 5225 5226 5227 5228 5229 5230 5231 5232 5233 5234 5235 5236 5237 5238 5239 5240 5241 5242 5243 | p2 = pOut = pPhrase->doclist.pList; res = fts3PoslistNearMerge( &pOut, aTmp, nParam1, nParam2, paPoslist, &p2 ); if( res ){ nNew = (int)(pOut - pPhrase->doclist.pList) - 1; assert_fts3_nc( nNew<=pPhrase->doclist.nList && nNew>0 ); if( nNew>=0 && nNew<=pPhrase->doclist.nList ){ assert( pPhrase->doclist.pList[nNew]=='\0' ); memset(&pPhrase->doclist.pList[nNew], 0, pPhrase->doclist.nList - nNew); pPhrase->doclist.nList = nNew; } *paPoslist = pPhrase->doclist.pList; *pnToken = pPhrase->nToken; } return res; } |
︙ | ︙ | |||
5150 5151 5152 5153 5154 5155 5156 | ** really a match, taking into account deferred tokens and NEAR operators. */ static void fts3EvalNextRow( Fts3Cursor *pCsr, /* FTS Cursor handle */ Fts3Expr *pExpr, /* Expr. to advance to next matching row */ int *pRc /* IN/OUT: Error code */ ){ | | < | 5283 5284 5285 5286 5287 5288 5289 5290 5291 5292 5293 5294 5295 5296 5297 5298 | ** really a match, taking into account deferred tokens and NEAR operators. */ static void fts3EvalNextRow( Fts3Cursor *pCsr, /* FTS Cursor handle */ Fts3Expr *pExpr, /* Expr. to advance to next matching row */ int *pRc /* IN/OUT: Error code */ ){ if( *pRc==SQLITE_OK && pExpr->bEof==0 ){ int bDescDoclist = pCsr->bDesc; /* Used by DOCID_CMP() macro */ pExpr->bStart = 1; switch( pExpr->eType ){ case FTSQUERY_NEAR: case FTSQUERY_AND: { Fts3Expr *pLeft = pExpr->pLeft; Fts3Expr *pRight = pExpr->pRight; |
︙ | ︙ | |||
5205 5206 5207 5208 5209 5210 5211 5212 5213 5214 5215 5216 5217 5218 5219 5220 5221 | if( pLeft->pPhrase && pLeft->pPhrase->doclist.aAll ){ Fts3Doclist *pDl = &pLeft->pPhrase->doclist; while( *pRc==SQLITE_OK && pLeft->bEof==0 ){ memset(pDl->pList, 0, pDl->nList); fts3EvalNextRow(pCsr, pLeft, pRc); } } } } break; } case FTSQUERY_OR: { Fts3Expr *pLeft = pExpr->pLeft; Fts3Expr *pRight = pExpr->pRight; sqlite3_int64 iCmp = DOCID_CMP(pLeft->iDocid, pRight->iDocid); | > | | | 5337 5338 5339 5340 5341 5342 5343 5344 5345 5346 5347 5348 5349 5350 5351 5352 5353 5354 5355 5356 5357 5358 5359 5360 5361 5362 5363 | if( pLeft->pPhrase && pLeft->pPhrase->doclist.aAll ){ Fts3Doclist *pDl = &pLeft->pPhrase->doclist; while( *pRc==SQLITE_OK && pLeft->bEof==0 ){ memset(pDl->pList, 0, pDl->nList); fts3EvalNextRow(pCsr, pLeft, pRc); } } pRight->bEof = pLeft->bEof = 1; } } break; } case FTSQUERY_OR: { Fts3Expr *pLeft = pExpr->pLeft; Fts3Expr *pRight = pExpr->pRight; sqlite3_int64 iCmp = DOCID_CMP(pLeft->iDocid, pRight->iDocid); assert_fts3_nc( pLeft->bStart || pLeft->iDocid==pRight->iDocid ); assert_fts3_nc( pRight->bStart || pLeft->iDocid==pRight->iDocid ); if( pRight->bEof || (pLeft->bEof==0 && iCmp<0) ){ fts3EvalNextRow(pCsr, pLeft, pRc); }else if( pLeft->bEof || iCmp>0 ){ fts3EvalNextRow(pCsr, pRight, pRc); }else{ fts3EvalNextRow(pCsr, pLeft, pRc); |
︙ | ︙ | |||
5315 5316 5317 5318 5319 5320 5321 | ** no exceptions to this - it's the way the parser in fts3_expr.c works. */ if( *pRc==SQLITE_OK && pExpr->eType==FTSQUERY_NEAR && (pExpr->pParent==0 || pExpr->pParent->eType!=FTSQUERY_NEAR) ){ Fts3Expr *p; | | | | 5448 5449 5450 5451 5452 5453 5454 5455 5456 5457 5458 5459 5460 5461 5462 5463 5464 5465 5466 5467 5468 5469 5470 5471 | ** no exceptions to this - it's the way the parser in fts3_expr.c works. */ if( *pRc==SQLITE_OK && pExpr->eType==FTSQUERY_NEAR && (pExpr->pParent==0 || pExpr->pParent->eType!=FTSQUERY_NEAR) ){ Fts3Expr *p; sqlite3_int64 nTmp = 0; /* Bytes of temp space */ char *aTmp; /* Temp space for PoslistNearMerge() */ /* Allocate temporary working space. */ for(p=pExpr; p->pLeft; p=p->pLeft){ assert( p->pRight->pPhrase->doclist.nList>0 ); nTmp += p->pRight->pPhrase->doclist.nList; } nTmp += p->pPhrase->doclist.nList; aTmp = sqlite3_malloc64(nTmp*2); if( !aTmp ){ *pRc = SQLITE_NOMEM; res = 0; }else{ char *aPoslist = p->pPhrase->doclist.pList; int nToken = p->pPhrase->nToken; |
︙ | ︙ | |||
5433 5434 5435 5436 5437 5438 5439 | fts3EvalTestExpr(pCsr, pExpr->pLeft, pRc) && !fts3EvalTestExpr(pCsr, pExpr->pRight, pRc) ); break; default: { #ifndef SQLITE_DISABLE_FTS4_DEFERRED | | | | < > | > > | 5566 5567 5568 5569 5570 5571 5572 5573 5574 5575 5576 5577 5578 5579 5580 5581 5582 5583 5584 5585 5586 5587 5588 5589 5590 5591 5592 5593 5594 5595 5596 | fts3EvalTestExpr(pCsr, pExpr->pLeft, pRc) && !fts3EvalTestExpr(pCsr, pExpr->pRight, pRc) ); break; default: { #ifndef SQLITE_DISABLE_FTS4_DEFERRED if( pCsr->pDeferred && (pExpr->bDeferred || ( pExpr->iDocid==pCsr->iPrevId && pExpr->pPhrase->doclist.pList ))){ Fts3Phrase *pPhrase = pExpr->pPhrase; if( pExpr->bDeferred ){ fts3EvalInvalidatePoslist(pPhrase); } *pRc = fts3EvalDeferredPhrase(pCsr, pPhrase); bHit = (pPhrase->doclist.pList!=0); pExpr->iDocid = pCsr->iPrevId; }else #endif { bHit = ( pExpr->bEof==0 && pExpr->iDocid==pCsr->iPrevId && pExpr->pPhrase->doclist.nList>0 ); } break; } } } return bHit; } |
︙ | ︙ | |||
5594 5595 5596 5597 5598 5599 5600 | ** After allocating the Fts3Expr.aMI[] array for each phrase in the ** expression rooted at pExpr, the cursor iterates through all rows matched ** by pExpr, calling this function for each row. This function increments ** the values in Fts3Expr.aMI[] according to the position-list currently ** found in Fts3Expr.pPhrase->doclist.pList for each of the phrase ** expression nodes. */ | | < < > | | | > > > > > > > > > > > > > > > > | 5729 5730 5731 5732 5733 5734 5735 5736 5737 5738 5739 5740 5741 5742 5743 5744 5745 5746 5747 5748 5749 5750 5751 5752 5753 5754 5755 5756 5757 5758 5759 5760 5761 5762 5763 5764 5765 5766 5767 5768 5769 5770 5771 5772 5773 5774 5775 5776 5777 5778 5779 5780 5781 5782 5783 5784 5785 5786 5787 | ** After allocating the Fts3Expr.aMI[] array for each phrase in the ** expression rooted at pExpr, the cursor iterates through all rows matched ** by pExpr, calling this function for each row. This function increments ** the values in Fts3Expr.aMI[] according to the position-list currently ** found in Fts3Expr.pPhrase->doclist.pList for each of the phrase ** expression nodes. */ static void fts3EvalUpdateCounts(Fts3Expr *pExpr, int nCol){ if( pExpr ){ Fts3Phrase *pPhrase = pExpr->pPhrase; if( pPhrase && pPhrase->doclist.pList ){ int iCol = 0; char *p = pPhrase->doclist.pList; do{ u8 c = 0; int iCnt = 0; while( 0xFE & (*p | c) ){ if( (c&0x80)==0 ) iCnt++; c = *p++ & 0x80; } /* aMI[iCol*3 + 1] = Number of occurrences ** aMI[iCol*3 + 2] = Number of rows containing at least one instance */ pExpr->aMI[iCol*3 + 1] += iCnt; pExpr->aMI[iCol*3 + 2] += (iCnt>0); if( *p==0x00 ) break; p++; p += fts3GetVarint32(p, &iCol); }while( iCol<nCol ); } fts3EvalUpdateCounts(pExpr->pLeft, nCol); fts3EvalUpdateCounts(pExpr->pRight, nCol); } } /* ** This is an sqlite3Fts3ExprIterate() callback. If the Fts3Expr.aMI[] array ** has not yet been allocated, allocate and zero it. Otherwise, just zero ** it. */ static int fts3AllocateMSI(Fts3Expr *pExpr, int iPhrase, void *pCtx){ Fts3Table *pTab = (Fts3Table*)pCtx; UNUSED_PARAMETER(iPhrase); if( pExpr->aMI==0 ){ pExpr->aMI = (u32 *)sqlite3_malloc64(pTab->nColumn * 3 * sizeof(u32)); if( pExpr->aMI==0 ) return SQLITE_NOMEM; } memset(pExpr->aMI, 0, pTab->nColumn * 3 * sizeof(u32)); return SQLITE_OK; } /* ** Expression pExpr must be of type FTSQUERY_PHRASE. ** ** If it is not already allocated and populated, this function allocates and ** populates the Fts3Expr.aMI[] array for expression pExpr. If pExpr is part |
︙ | ︙ | |||
5647 5648 5649 5650 5651 5652 5653 | ){ int rc = SQLITE_OK; /* Return code */ assert( pExpr->eType==FTSQUERY_PHRASE ); if( pExpr->aMI==0 ){ Fts3Table *pTab = (Fts3Table *)pCsr->base.pVtab; Fts3Expr *pRoot; /* Root of NEAR expression */ | < | > > < | < < < < < | | 5797 5798 5799 5800 5801 5802 5803 5804 5805 5806 5807 5808 5809 5810 5811 5812 5813 5814 5815 5816 5817 5818 5819 5820 5821 5822 5823 5824 5825 5826 5827 5828 5829 | ){ int rc = SQLITE_OK; /* Return code */ assert( pExpr->eType==FTSQUERY_PHRASE ); if( pExpr->aMI==0 ){ Fts3Table *pTab = (Fts3Table *)pCsr->base.pVtab; Fts3Expr *pRoot; /* Root of NEAR expression */ sqlite3_int64 iPrevId = pCsr->iPrevId; sqlite3_int64 iDocid; u8 bEof; /* Find the root of the NEAR expression */ pRoot = pExpr; while( pRoot->pParent && (pRoot->pParent->eType==FTSQUERY_NEAR || pRoot->bDeferred) ){ pRoot = pRoot->pParent; } iDocid = pRoot->iDocid; bEof = pRoot->bEof; assert( pRoot->bStart ); /* Allocate space for the aMSI[] array of each FTSQUERY_PHRASE node */ rc = sqlite3Fts3ExprIterate(pRoot, fts3AllocateMSI, (void*)pTab); if( rc!=SQLITE_OK ) return rc; fts3EvalRestart(pCsr, pRoot, &rc); while( pCsr->isEof==0 && rc==SQLITE_OK ){ do { /* Ensure the %_content statement is reset. */ if( pCsr->isRequireSeek==0 ) sqlite3_reset(pCsr->pStmt); |
︙ | ︙ | |||
5692 5693 5694 5695 5696 5697 5698 | pCsr->iPrevId = pRoot->iDocid; }while( pCsr->isEof==0 && pRoot->eType==FTSQUERY_NEAR && sqlite3Fts3EvalTestDeferred(pCsr, &rc) ); if( rc==SQLITE_OK && pCsr->isEof==0 ){ | | | > | 5837 5838 5839 5840 5841 5842 5843 5844 5845 5846 5847 5848 5849 5850 5851 5852 5853 5854 5855 5856 5857 5858 5859 5860 5861 5862 5863 5864 5865 5866 5867 5868 5869 5870 5871 | pCsr->iPrevId = pRoot->iDocid; }while( pCsr->isEof==0 && pRoot->eType==FTSQUERY_NEAR && sqlite3Fts3EvalTestDeferred(pCsr, &rc) ); if( rc==SQLITE_OK && pCsr->isEof==0 ){ fts3EvalUpdateCounts(pRoot, pTab->nColumn); } } pCsr->isEof = 0; pCsr->iPrevId = iPrevId; if( bEof ){ pRoot->bEof = bEof; }else{ /* Caution: pRoot may iterate through docids in ascending or descending ** order. For this reason, even though it seems more defensive, the ** do loop can not be written: ** ** do {...} while( pRoot->iDocid<iDocid && rc==SQLITE_OK ); */ fts3EvalRestart(pCsr, pRoot, &rc); do { fts3EvalNextRow(pCsr, pRoot, &rc); assert_fts3_nc( pRoot->bEof==0 ); if( pRoot->bEof ) rc = FTS_CORRUPT_VTAB; }while( pRoot->iDocid!=iDocid && rc==SQLITE_OK ); } } return rc; } /* |
︙ | ︙ | |||
5825 5826 5827 5828 5829 5830 5831 5832 5833 5834 5835 5836 5837 5838 5839 5840 5841 5842 5843 5844 5845 5846 5847 5848 5849 5850 | if( iDocid!=pCsr->iPrevId || pExpr->bEof ){ int rc = SQLITE_OK; int bDescDoclist = pTab->bDescIdx; /* For DOCID_CMP macro */ int bOr = 0; u8 bTreeEof = 0; Fts3Expr *p; /* Used to iterate from pExpr to root */ Fts3Expr *pNear; /* Most senior NEAR ancestor (or pExpr) */ int bMatch; /* Check if this phrase descends from an OR expression node. If not, ** return NULL. Otherwise, the entry that corresponds to docid ** pCsr->iPrevId may lie earlier in the doclist buffer. Or, if the ** tree that the node is part of has been marked as EOF, but the node ** itself is not EOF, then it may point to an earlier entry. */ pNear = pExpr; for(p=pExpr->pParent; p; p=p->pParent){ if( p->eType==FTSQUERY_OR ) bOr = 1; if( p->eType==FTSQUERY_NEAR ) pNear = p; if( p->bEof ) bTreeEof = 1; } if( bOr==0 ) return SQLITE_OK; /* This is the descendent of an OR node. In this case we cannot use ** an incremental phrase. Load the entire doclist for the phrase ** into memory in this case. */ if( pPhrase->bIncr ){ | > > > > > > | | | | | > > | > | | | 5971 5972 5973 5974 5975 5976 5977 5978 5979 5980 5981 5982 5983 5984 5985 5986 5987 5988 5989 5990 5991 5992 5993 5994 5995 5996 5997 5998 5999 6000 6001 6002 6003 6004 6005 6006 6007 6008 6009 6010 6011 6012 6013 6014 6015 6016 6017 6018 6019 6020 6021 6022 6023 | if( iDocid!=pCsr->iPrevId || pExpr->bEof ){ int rc = SQLITE_OK; int bDescDoclist = pTab->bDescIdx; /* For DOCID_CMP macro */ int bOr = 0; u8 bTreeEof = 0; Fts3Expr *p; /* Used to iterate from pExpr to root */ Fts3Expr *pNear; /* Most senior NEAR ancestor (or pExpr) */ Fts3Expr *pRun; /* Closest non-deferred ancestor of pNear */ int bMatch; /* Check if this phrase descends from an OR expression node. If not, ** return NULL. Otherwise, the entry that corresponds to docid ** pCsr->iPrevId may lie earlier in the doclist buffer. Or, if the ** tree that the node is part of has been marked as EOF, but the node ** itself is not EOF, then it may point to an earlier entry. */ pNear = pExpr; for(p=pExpr->pParent; p; p=p->pParent){ if( p->eType==FTSQUERY_OR ) bOr = 1; if( p->eType==FTSQUERY_NEAR ) pNear = p; if( p->bEof ) bTreeEof = 1; } if( bOr==0 ) return SQLITE_OK; pRun = pNear; while( pRun->bDeferred ){ assert( pRun->pParent ); pRun = pRun->pParent; } /* This is the descendent of an OR node. In this case we cannot use ** an incremental phrase. Load the entire doclist for the phrase ** into memory in this case. */ if( pPhrase->bIncr ){ int bEofSave = pRun->bEof; fts3EvalRestart(pCsr, pRun, &rc); while( rc==SQLITE_OK && !pRun->bEof ){ fts3EvalNextRow(pCsr, pRun, &rc); if( bEofSave==0 && pRun->iDocid==iDocid ) break; } assert( rc!=SQLITE_OK || pPhrase->bIncr==0 ); if( rc==SQLITE_OK && pRun->bEof!=bEofSave ){ rc = FTS_CORRUPT_VTAB; } } if( bTreeEof ){ while( rc==SQLITE_OK && !pRun->bEof ){ fts3EvalNextRow(pCsr, pRun, &rc); } } if( rc!=SQLITE_OK ) return rc; bMatch = 1; for(p=pNear; p; p=p->pLeft){ u8 bEof = 0; |
︙ | ︙ |
Changes to ext/fts3/fts3Int.h.
︙ | ︙ | |||
91 92 93 94 95 96 97 98 99 100 101 102 103 104 | /* ** Maximum length of a varint encoded integer. The varint format is different ** from that used by SQLite, so the maximum length is 10, not 9. */ #define FTS3_VARINT_MAX 10 /* ** FTS4 virtual tables may maintain multiple indexes - one index of all terms ** in the document set and zero or more prefix indexes. All indexes are stored ** as one or more b+-trees in the %_segments and %_segdir tables. ** ** It is possible to determine which index a b+-tree belongs to based on the ** value stored in the "%_segdir.level" column. Given this value L, the index | > > | 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 | /* ** Maximum length of a varint encoded integer. The varint format is different ** from that used by SQLite, so the maximum length is 10, not 9. */ #define FTS3_VARINT_MAX 10 #define FTS3_BUFFER_PADDING 8 /* ** FTS4 virtual tables may maintain multiple indexes - one index of all terms ** in the document set and zero or more prefix indexes. All indexes are stored ** as one or more b+-trees in the %_segments and %_segdir tables. ** ** It is possible to determine which index a b+-tree belongs to based on the ** value stored in the "%_segdir.level" column. Given this value L, the index |
︙ | ︙ | |||
123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 | /* ** Terminator values for position-lists and column-lists. */ #define POS_COLUMN (1) /* Column-list terminator */ #define POS_END (0) /* Position-list terminator */ /* ** This section provides definitions to allow the ** FTS3 extension to be compiled outside of the ** amalgamation. */ #ifndef SQLITE_AMALGAMATION /* ** Macros indicating that conditional expressions are always true or ** false. */ | > > > > > > > > > > > > | > > > | | | | | < < | | | 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 | /* ** Terminator values for position-lists and column-lists. */ #define POS_COLUMN (1) /* Column-list terminator */ #define POS_END (0) /* Position-list terminator */ /* ** The assert_fts3_nc() macro is similar to the assert() macro, except that it ** is used for assert() conditions that are true only if it can be ** guranteed that the database is not corrupt. */ #ifdef SQLITE_DEBUG extern int sqlite3_fts3_may_be_corrupt; # define assert_fts3_nc(x) assert(sqlite3_fts3_may_be_corrupt || (x)) #else # define assert_fts3_nc(x) assert(x) #endif /* ** This section provides definitions to allow the ** FTS3 extension to be compiled outside of the ** amalgamation. */ #ifndef SQLITE_AMALGAMATION /* ** Macros indicating that conditional expressions are always true or ** false. */ #if defined(SQLITE_COVERAGE_TEST) || defined(SQLITE_MUTATION_TEST) # define SQLITE_OMIT_AUXILIARY_SAFETY_CHECKS 1 #endif #if defined(SQLITE_OMIT_AUXILIARY_SAFETY_CHECKS) # define ALWAYS(X) (1) # define NEVER(X) (0) #elif !defined(NDEBUG) # define ALWAYS(X) ((X)?1:(assert(0),0)) # define NEVER(X) ((X)?(assert(0),1):0) #else # define ALWAYS(X) (X) # define NEVER(X) (X) #endif /* ** Internal types used by SQLite. */ typedef unsigned char u8; /* 1-byte (or larger) unsigned integer */ typedef short int i16; /* 2-byte (or larger) signed integer */ |
︙ | ︙ | |||
177 178 179 180 181 182 183 184 185 186 187 188 189 190 | ** within testcase() and assert() macros. */ #if defined(SQLITE_DEBUG) || defined(SQLITE_COVERAGE_TEST) # define TESTONLY(X) X #else # define TESTONLY(X) #endif #endif /* SQLITE_AMALGAMATION */ #ifdef SQLITE_DEBUG int sqlite3Fts3Corrupt(void); # define FTS_CORRUPT_VTAB sqlite3Fts3Corrupt() #else | > > > > > | 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 | ** within testcase() and assert() macros. */ #if defined(SQLITE_DEBUG) || defined(SQLITE_COVERAGE_TEST) # define TESTONLY(X) X #else # define TESTONLY(X) #endif #define LARGEST_INT64 (0xffffffff|(((i64)0x7fffffff)<<32)) #define SMALLEST_INT64 (((i64)-1) - LARGEST_INT64) #define deliberate_fall_through #endif /* SQLITE_AMALGAMATION */ #ifdef SQLITE_DEBUG int sqlite3Fts3Corrupt(void); # define FTS_CORRUPT_VTAB sqlite3Fts3Corrupt() #else |
︙ | ︙ | |||
221 222 223 224 225 226 227 228 229 230 231 232 233 234 | char **azColumn; /* column names. malloced */ u8 *abNotindexed; /* True for 'notindexed' columns */ sqlite3_tokenizer *pTokenizer; /* tokenizer for inserts and queries */ char *zContentTbl; /* content=xxx option, or NULL */ char *zLanguageid; /* languageid=xxx option, or NULL */ int nAutoincrmerge; /* Value configured by 'automerge' */ u32 nLeafAdd; /* Number of leaf blocks added this trans */ /* Precompiled statements used by the implementation. Each of these ** statements is run and reset within a single virtual table API call. */ sqlite3_stmt *aStmt[40]; sqlite3_stmt *pSeekStmt; /* Cache for fts3CursorSeekStmt() */ | > | 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 | char **azColumn; /* column names. malloced */ u8 *abNotindexed; /* True for 'notindexed' columns */ sqlite3_tokenizer *pTokenizer; /* tokenizer for inserts and queries */ char *zContentTbl; /* content=xxx option, or NULL */ char *zLanguageid; /* languageid=xxx option, or NULL */ int nAutoincrmerge; /* Value configured by 'automerge' */ u32 nLeafAdd; /* Number of leaf blocks added this trans */ int bLock; /* Used to prevent recursive content= tbls */ /* Precompiled statements used by the implementation. Each of these ** statements is run and reset within a single virtual table API call. */ sqlite3_stmt *aStmt[40]; sqlite3_stmt *pSeekStmt; /* Cache for fts3CursorSeekStmt() */ |
︙ | ︙ | |||
279 280 281 282 283 284 285 | ** values do not contribute to FTS functionality; they are used for ** verifying the operation of the SQLite core. */ int inTransaction; /* True after xBegin but before xCommit/xRollback */ int mxSavepoint; /* Largest valid xSavepoint integer */ #endif | | > > > > > > > > > > | 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 | ** values do not contribute to FTS functionality; they are used for ** verifying the operation of the SQLite core. */ int inTransaction; /* True after xBegin but before xCommit/xRollback */ int mxSavepoint; /* Largest valid xSavepoint integer */ #endif #if defined(SQLITE_DEBUG) || defined(SQLITE_TEST) /* True to disable the incremental doclist optimization. This is controled ** by special insert command 'test-no-incr-doclist'. */ int bNoIncrDoclist; /* Number of segments in a level */ int nMergeCount; #endif }; /* Macro to find the number of segments to merge */ #if defined(SQLITE_DEBUG) || defined(SQLITE_TEST) # define MergeCount(P) ((P)->nMergeCount) #else # define MergeCount(P) FTS3_MERGE_COUNT #endif /* ** When the core wants to read from the virtual table, it creates a ** virtual table cursor (an instance of the following structure) using ** the xOpen method. Cursors are destroyed using the xClose method. */ struct Fts3Cursor { |
︙ | ︙ | |||
523 524 525 526 527 528 529 | struct Fts3MultiSegReader { /* Used internally by sqlite3Fts3SegReaderXXX() calls */ Fts3SegReader **apSegment; /* Array of Fts3SegReader objects */ int nSegment; /* Size of apSegment array */ int nAdvance; /* How many seg-readers to advance */ Fts3SegFilter *pFilter; /* Pointer to filter object */ char *aBuffer; /* Buffer to merge doclists in */ | | | 554 555 556 557 558 559 560 561 562 563 564 565 566 567 568 | struct Fts3MultiSegReader { /* Used internally by sqlite3Fts3SegReaderXXX() calls */ Fts3SegReader **apSegment; /* Array of Fts3SegReader objects */ int nSegment; /* Size of apSegment array */ int nAdvance; /* How many seg-readers to advance */ Fts3SegFilter *pFilter; /* Pointer to filter object */ char *aBuffer; /* Buffer to merge doclists in */ i64 nBuffer; /* Allocated size of aBuffer[] in bytes */ int iColFilter; /* If >=0, filter for this column */ int bRestart; /* Used by fts3.c only. */ int nCost; /* Cost of running iterator */ int bLookup; /* True if a lookup of a single entry. */ |
︙ | ︙ | |||
549 550 551 552 553 554 555 556 557 558 559 560 561 562 563 564 565 566 567 568 569 570 | (*(u8*)(p)&0x80) ? sqlite3Fts3GetVarint32(p, piVal) : (*piVal=*(u8*)(p), 1) \ ) /* fts3.c */ void sqlite3Fts3ErrMsg(char**,const char*,...); int sqlite3Fts3PutVarint(char *, sqlite3_int64); int sqlite3Fts3GetVarint(const char *, sqlite_int64 *); int sqlite3Fts3GetVarint32(const char *, int *); int sqlite3Fts3VarintLen(sqlite3_uint64); void sqlite3Fts3Dequote(char *); void sqlite3Fts3DoclistPrev(int,char*,int,char**,sqlite3_int64*,int*,u8*); int sqlite3Fts3EvalPhraseStats(Fts3Cursor *, Fts3Expr *, u32 *); int sqlite3Fts3FirstFilter(sqlite3_int64, char *, int, char *); void sqlite3Fts3CreateStatTable(int*, Fts3Table*); int sqlite3Fts3EvalTestDeferred(Fts3Cursor *pCsr, int *pRc); /* fts3_tokenizer.c */ const char *sqlite3Fts3NextToken(const char *, int *); int sqlite3Fts3InitHashTable(sqlite3 *, Fts3Hash *, const char *); int sqlite3Fts3InitTokenizer(Fts3Hash *pHash, const char *, sqlite3_tokenizer **, char ** ); | > > > | 580 581 582 583 584 585 586 587 588 589 590 591 592 593 594 595 596 597 598 599 600 601 602 603 604 | (*(u8*)(p)&0x80) ? sqlite3Fts3GetVarint32(p, piVal) : (*piVal=*(u8*)(p), 1) \ ) /* fts3.c */ void sqlite3Fts3ErrMsg(char**,const char*,...); int sqlite3Fts3PutVarint(char *, sqlite3_int64); int sqlite3Fts3GetVarint(const char *, sqlite_int64 *); int sqlite3Fts3GetVarintU(const char *, sqlite_uint64 *); int sqlite3Fts3GetVarintBounded(const char*,const char*,sqlite3_int64*); int sqlite3Fts3GetVarint32(const char *, int *); int sqlite3Fts3VarintLen(sqlite3_uint64); void sqlite3Fts3Dequote(char *); void sqlite3Fts3DoclistPrev(int,char*,int,char**,sqlite3_int64*,int*,u8*); int sqlite3Fts3EvalPhraseStats(Fts3Cursor *, Fts3Expr *, u32 *); int sqlite3Fts3FirstFilter(sqlite3_int64, char *, int, char *); void sqlite3Fts3CreateStatTable(int*, Fts3Table*); int sqlite3Fts3EvalTestDeferred(Fts3Cursor *pCsr, int *pRc); int sqlite3Fts3ReadInt(const char *z, int *pnOut); /* fts3_tokenizer.c */ const char *sqlite3Fts3NextToken(const char *, int *); int sqlite3Fts3InitHashTable(sqlite3 *, Fts3Hash *, const char *); int sqlite3Fts3InitTokenizer(Fts3Hash *pHash, const char *, sqlite3_tokenizer **, char ** ); |
︙ | ︙ | |||
583 584 585 586 587 588 589 590 591 592 593 594 595 596 597 598 599 600 601 602 603 604 605 606 607 608 | char **, int, int, int, const char *, int, Fts3Expr **, char ** ); void sqlite3Fts3ExprFree(Fts3Expr *); #ifdef SQLITE_TEST int sqlite3Fts3ExprInitTestInterface(sqlite3 *db, Fts3Hash*); int sqlite3Fts3InitTerm(sqlite3 *db); #endif int sqlite3Fts3OpenTokenizer(sqlite3_tokenizer *, int, const char *, int, sqlite3_tokenizer_cursor ** ); /* fts3_aux.c */ int sqlite3Fts3InitAux(sqlite3 *db); void sqlite3Fts3EvalPhraseCleanup(Fts3Phrase *); int sqlite3Fts3MsrIncrStart( Fts3Table*, Fts3MultiSegReader*, int, const char*, int); int sqlite3Fts3MsrIncrNext( Fts3Table *, Fts3MultiSegReader *, sqlite3_int64 *, char **, int *); int sqlite3Fts3EvalPhrasePoslist(Fts3Cursor *, Fts3Expr *, int iCol, char **); int sqlite3Fts3MsrOvfl(Fts3Cursor *, Fts3MultiSegReader *, int *); int sqlite3Fts3MsrIncrRestart(Fts3MultiSegReader *pCsr); /* fts3_tokenize_vtab.c */ | > | > > | 617 618 619 620 621 622 623 624 625 626 627 628 629 630 631 632 633 634 635 636 637 638 639 640 641 642 643 644 645 646 647 648 649 650 651 652 653 654 655 656 | char **, int, int, int, const char *, int, Fts3Expr **, char ** ); void sqlite3Fts3ExprFree(Fts3Expr *); #ifdef SQLITE_TEST int sqlite3Fts3ExprInitTestInterface(sqlite3 *db, Fts3Hash*); int sqlite3Fts3InitTerm(sqlite3 *db); #endif void *sqlite3Fts3MallocZero(i64 nByte); int sqlite3Fts3OpenTokenizer(sqlite3_tokenizer *, int, const char *, int, sqlite3_tokenizer_cursor ** ); /* fts3_aux.c */ int sqlite3Fts3InitAux(sqlite3 *db); void sqlite3Fts3EvalPhraseCleanup(Fts3Phrase *); int sqlite3Fts3MsrIncrStart( Fts3Table*, Fts3MultiSegReader*, int, const char*, int); int sqlite3Fts3MsrIncrNext( Fts3Table *, Fts3MultiSegReader *, sqlite3_int64 *, char **, int *); int sqlite3Fts3EvalPhrasePoslist(Fts3Cursor *, Fts3Expr *, int iCol, char **); int sqlite3Fts3MsrOvfl(Fts3Cursor *, Fts3MultiSegReader *, int *); int sqlite3Fts3MsrIncrRestart(Fts3MultiSegReader *pCsr); /* fts3_tokenize_vtab.c */ int sqlite3Fts3InitTok(sqlite3*, Fts3Hash *, void(*xDestroy)(void*)); /* fts3_unicode2.c (functions generated by parsing unicode text files) */ #ifndef SQLITE_DISABLE_FTS3_UNICODE int sqlite3FtsUnicodeFold(int, int); int sqlite3FtsUnicodeIsalnum(int); int sqlite3FtsUnicodeIsdiacritic(int); #endif int sqlite3Fts3ExprIterate(Fts3Expr*, int (*x)(Fts3Expr*,int,void*), void*); #endif /* !SQLITE_CORE || SQLITE_ENABLE_FTS3 */ #endif /* _FTSINT_H */ |
Changes to ext/fts3/fts3_aux.c.
︙ | ︙ | |||
62 63 64 65 66 67 68 | sqlite3_vtab **ppVtab, /* OUT: New sqlite3_vtab object */ char **pzErr /* OUT: sqlite3_malloc'd error message */ ){ char const *zDb; /* Name of database (e.g. "main") */ char const *zFts3; /* Name of fts3 table */ int nDb; /* Result of strlen(zDb) */ int nFts3; /* Result of strlen(zFts3) */ | | | 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 | sqlite3_vtab **ppVtab, /* OUT: New sqlite3_vtab object */ char **pzErr /* OUT: sqlite3_malloc'd error message */ ){ char const *zDb; /* Name of database (e.g. "main") */ char const *zFts3; /* Name of fts3 table */ int nDb; /* Result of strlen(zDb) */ int nFts3; /* Result of strlen(zFts3) */ sqlite3_int64 nByte; /* Bytes of space to allocate here */ int rc; /* value returned by declare_vtab() */ Fts3auxTable *p; /* Virtual table object to return */ UNUSED_PARAMETER(pUnused); /* The user should invoke this in one of two forms: ** |
︙ | ︙ | |||
94 95 96 97 98 99 100 | } nFts3 = (int)strlen(zFts3); rc = sqlite3_declare_vtab(db, FTS3_AUX_SCHEMA); if( rc!=SQLITE_OK ) return rc; nByte = sizeof(Fts3auxTable) + sizeof(Fts3Table) + nDb + nFts3 + 2; | | | 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 | } nFts3 = (int)strlen(zFts3); rc = sqlite3_declare_vtab(db, FTS3_AUX_SCHEMA); if( rc!=SQLITE_OK ) return rc; nByte = sizeof(Fts3auxTable) + sizeof(Fts3Table) + nDb + nFts3 + 2; p = (Fts3auxTable *)sqlite3_malloc64(nByte); if( !p ) return SQLITE_NOMEM; memset(p, 0, nByte); p->pFts3Tab = (Fts3Table *)&p[1]; p->pFts3Tab->zDb = (char *)&p->pFts3Tab[1]; p->pFts3Tab->zName = &p->pFts3Tab->zDb[nDb+1]; p->pFts3Tab->db = db; |
︙ | ︙ | |||
244 245 246 247 248 249 250 | sqlite3_free(pCsr); return SQLITE_OK; } static int fts3auxGrowStatArray(Fts3auxCursor *pCsr, int nSize){ if( nSize>pCsr->nStat ){ struct Fts3auxColstats *aNew; | | | 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 | sqlite3_free(pCsr); return SQLITE_OK; } static int fts3auxGrowStatArray(Fts3auxCursor *pCsr, int nSize){ if( nSize>pCsr->nStat ){ struct Fts3auxColstats *aNew; aNew = (struct Fts3auxColstats *)sqlite3_realloc64(pCsr->aStat, sizeof(struct Fts3auxColstats) * nSize ); if( aNew==0 ) return SQLITE_NOMEM; memset(&aNew[pCsr->nStat], 0, sizeof(struct Fts3auxColstats) * (nSize - pCsr->nStat) ); pCsr->aStat = aNew; |
︙ | ︙ | |||
293 294 295 296 297 298 299 300 301 302 303 304 305 306 | return SQLITE_OK; } } if( fts3auxGrowStatArray(pCsr, 2) ) return SQLITE_NOMEM; memset(pCsr->aStat, 0, sizeof(struct Fts3auxColstats) * pCsr->nStat); iCol = 0; while( i<nDoclist ){ sqlite3_int64 v = 0; i += sqlite3Fts3GetVarint(&aDoclist[i], &v); switch( eState ){ /* State 0. In this state the integer just read was a docid. */ | > | 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 | return SQLITE_OK; } } if( fts3auxGrowStatArray(pCsr, 2) ) return SQLITE_NOMEM; memset(pCsr->aStat, 0, sizeof(struct Fts3auxColstats) * pCsr->nStat); iCol = 0; rc = SQLITE_OK; while( i<nDoclist ){ sqlite3_int64 v = 0; i += sqlite3Fts3GetVarint(&aDoclist[i], &v); switch( eState ){ /* State 0. In this state the integer just read was a docid. */ |
︙ | ︙ | |||
336 337 338 339 340 341 342 343 344 345 346 347 348 349 350 | pCsr->aStat[0].nOcc++; } break; /* State 3. The integer just read is a column number. */ default: assert( eState==3 ); iCol = (int)v; if( fts3auxGrowStatArray(pCsr, iCol+2) ) return SQLITE_NOMEM; pCsr->aStat[iCol+1].nDoc++; eState = 2; break; } } pCsr->iCol = 0; | > > > > < | 337 338 339 340 341 342 343 344 345 346 347 348 349 350 351 352 353 354 355 356 357 358 359 360 361 362 | pCsr->aStat[0].nOcc++; } break; /* State 3. The integer just read is a column number. */ default: assert( eState==3 ); iCol = (int)v; if( iCol<1 ){ rc = SQLITE_CORRUPT_VTAB; break; } if( fts3auxGrowStatArray(pCsr, iCol+2) ) return SQLITE_NOMEM; pCsr->aStat[iCol+1].nDoc++; eState = 2; break; } } pCsr->iCol = 0; }else{ pCsr->isEof = 1; } return rc; } /* |
︙ | ︙ | |||
402 403 404 405 406 407 408 409 410 411 412 413 414 415 416 417 418 | } /* In case this cursor is being reused, close and zero it. */ testcase(pCsr->filter.zTerm); sqlite3Fts3SegReaderFinish(&pCsr->csr); sqlite3_free((void *)pCsr->filter.zTerm); sqlite3_free(pCsr->aStat); memset(&pCsr->csr, 0, ((u8*)&pCsr[1]) - (u8*)&pCsr->csr); pCsr->filter.flags = FTS3_SEGMENT_REQUIRE_POS|FTS3_SEGMENT_IGNORE_EMPTY; if( isScan ) pCsr->filter.flags |= FTS3_SEGMENT_SCAN; if( iEq>=0 || iGe>=0 ){ const unsigned char *zStr = sqlite3_value_text(apVal[0]); assert( (iEq==0 && iGe==-1) || (iEq==-1 && iGe==0) ); if( zStr ){ pCsr->filter.zTerm = sqlite3_mprintf("%s", zStr); | > < > < > | 406 407 408 409 410 411 412 413 414 415 416 417 418 419 420 421 422 423 424 425 426 427 428 429 430 431 432 433 434 435 436 437 438 439 | } /* In case this cursor is being reused, close and zero it. */ testcase(pCsr->filter.zTerm); sqlite3Fts3SegReaderFinish(&pCsr->csr); sqlite3_free((void *)pCsr->filter.zTerm); sqlite3_free(pCsr->aStat); sqlite3_free(pCsr->zStop); memset(&pCsr->csr, 0, ((u8*)&pCsr[1]) - (u8*)&pCsr->csr); pCsr->filter.flags = FTS3_SEGMENT_REQUIRE_POS|FTS3_SEGMENT_IGNORE_EMPTY; if( isScan ) pCsr->filter.flags |= FTS3_SEGMENT_SCAN; if( iEq>=0 || iGe>=0 ){ const unsigned char *zStr = sqlite3_value_text(apVal[0]); assert( (iEq==0 && iGe==-1) || (iEq==-1 && iGe==0) ); if( zStr ){ pCsr->filter.zTerm = sqlite3_mprintf("%s", zStr); if( pCsr->filter.zTerm==0 ) return SQLITE_NOMEM; pCsr->filter.nTerm = (int)strlen(pCsr->filter.zTerm); } } if( iLe>=0 ){ pCsr->zStop = sqlite3_mprintf("%s", sqlite3_value_text(apVal[iLe])); if( pCsr->zStop==0 ) return SQLITE_NOMEM; pCsr->nStop = (int)strlen(pCsr->zStop); } if( iLangid>=0 ){ iLangVal = sqlite3_value_int(apVal[iLangid]); /* If the user specified a negative value for the languageid, use zero ** instead. This works, as the "languageid=?" constraint will also |
︙ | ︙ |
Changes to ext/fts3/fts3_expr.c.
︙ | ︙ | |||
118 119 120 121 122 123 124 | } /* ** Allocate nByte bytes of memory using sqlite3_malloc(). If successful, ** zero the memory before returning a pointer to it. If unsuccessful, ** return NULL. */ | | | | 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 | } /* ** Allocate nByte bytes of memory using sqlite3_malloc(). If successful, ** zero the memory before returning a pointer to it. If unsuccessful, ** return NULL. */ void *sqlite3Fts3MallocZero(sqlite3_int64 nByte){ void *pRet = sqlite3_malloc64(nByte); if( pRet ) memset(pRet, 0, nByte); return pRet; } int sqlite3Fts3OpenTokenizer( sqlite3_tokenizer *pTokenizer, int iLangid, |
︙ | ︙ | |||
194 195 196 197 198 199 200 | } *pnConsumed = i; rc = sqlite3Fts3OpenTokenizer(pTokenizer, pParse->iLangid, z, i, &pCursor); if( rc==SQLITE_OK ){ const char *zToken; int nToken = 0, iStart = 0, iEnd = 0, iPosition = 0; | | | | 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 | } *pnConsumed = i; rc = sqlite3Fts3OpenTokenizer(pTokenizer, pParse->iLangid, z, i, &pCursor); if( rc==SQLITE_OK ){ const char *zToken; int nToken = 0, iStart = 0, iEnd = 0, iPosition = 0; sqlite3_int64 nByte; /* total space to allocate */ rc = pModule->xNext(pCursor, &zToken, &nToken, &iStart, &iEnd, &iPosition); if( rc==SQLITE_OK ){ nByte = sizeof(Fts3Expr) + sizeof(Fts3Phrase) + nToken; pRet = (Fts3Expr *)sqlite3Fts3MallocZero(nByte); if( !pRet ){ rc = SQLITE_NOMEM; }else{ pRet->eType = FTSQUERY_PHRASE; pRet->pPhrase = (Fts3Phrase *)&pRet[1]; pRet->pPhrase->nToken = 1; pRet->pPhrase->iColumn = iCol; |
︙ | ︙ | |||
248 249 250 251 252 253 254 | } /* ** Enlarge a memory allocation. If an out-of-memory allocation occurs, ** then free the old allocation. */ | | | | 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 | } /* ** Enlarge a memory allocation. If an out-of-memory allocation occurs, ** then free the old allocation. */ static void *fts3ReallocOrFree(void *pOrig, sqlite3_int64 nNew){ void *pRet = sqlite3_realloc64(pOrig, nNew); if( !pRet ){ sqlite3_free(pOrig); } return pRet; } /* |
︙ | ︙ | |||
442 443 444 445 446 447 448 | int nKey = pKey->n; char cNext; /* If this is a "NEAR" keyword, check for an explicit nearness. */ if( pKey->eType==FTSQUERY_NEAR ){ assert( nKey==4 ); if( zInput[4]=='/' && zInput[5]>='0' && zInput[5]<='9' ){ | < < | < | | 442 443 444 445 446 447 448 449 450 451 452 453 454 455 456 457 458 459 460 461 462 463 464 465 466 467 468 | int nKey = pKey->n; char cNext; /* If this is a "NEAR" keyword, check for an explicit nearness. */ if( pKey->eType==FTSQUERY_NEAR ){ assert( nKey==4 ); if( zInput[4]=='/' && zInput[5]>='0' && zInput[5]<='9' ){ nKey += 1+sqlite3Fts3ReadInt(&zInput[nKey+1], &nNear); } } /* At this point this is probably a keyword. But for that to be true, ** the next byte must contain either whitespace, an open or close ** parenthesis, a quote character, or EOF. */ cNext = zInput[nKey]; if( fts3isspace(cNext) || cNext=='"' || cNext=='(' || cNext==')' || cNext==0 ){ pRet = (Fts3Expr *)sqlite3Fts3MallocZero(sizeof(Fts3Expr)); if( !pRet ){ return SQLITE_NOMEM; } pRet->eType = pKey->eType; pRet->nNear = nNear; *ppExpr = pRet; *pnConsumed = (int)((zInput - z) + nKey); |
︙ | ︙ | |||
492 493 494 495 496 497 498 499 | return getNextString(pParse, &zInput[1], ii-1, ppExpr); } if( sqlite3_fts3_enable_parentheses ){ if( *zInput=='(' ){ int nConsumed = 0; pParse->nNest++; rc = fts3ExprParse(pParse, zInput+1, nInput-1, ppExpr, &nConsumed); | > > > > > < | 489 490 491 492 493 494 495 496 497 498 499 500 501 502 503 504 505 506 507 508 | return getNextString(pParse, &zInput[1], ii-1, ppExpr); } if( sqlite3_fts3_enable_parentheses ){ if( *zInput=='(' ){ int nConsumed = 0; pParse->nNest++; #if !defined(SQLITE_MAX_EXPR_DEPTH) if( pParse->nNest>1000 ) return SQLITE_ERROR; #elif SQLITE_MAX_EXPR_DEPTH>0 if( pParse->nNest>SQLITE_MAX_EXPR_DEPTH ) return SQLITE_ERROR; #endif rc = fts3ExprParse(pParse, zInput+1, nInput-1, ppExpr, &nConsumed); *pnConsumed = (int)(zInput - z) + 1 + nConsumed; return rc; }else if( *zInput==')' ){ pParse->nNest--; *pnConsumed = (int)((zInput - z) + 1); *ppExpr = 0; return SQLITE_DONE; |
︙ | ︙ | |||
632 633 634 635 636 637 638 | if( p ){ int isPhrase; if( !sqlite3_fts3_enable_parentheses && p->eType==FTSQUERY_PHRASE && pParse->isNot ){ /* Create an implicit NOT operator. */ | | | 633 634 635 636 637 638 639 640 641 642 643 644 645 646 647 | if( p ){ int isPhrase; if( !sqlite3_fts3_enable_parentheses && p->eType==FTSQUERY_PHRASE && pParse->isNot ){ /* Create an implicit NOT operator. */ Fts3Expr *pNot = sqlite3Fts3MallocZero(sizeof(Fts3Expr)); if( !pNot ){ sqlite3Fts3ExprFree(p); rc = SQLITE_NOMEM; goto exprparse_out; } pNot->eType = FTSQUERY_NOT; pNot->pRight = p; |
︙ | ︙ | |||
666 667 668 669 670 671 672 | goto exprparse_out; } if( isPhrase && !isRequirePhrase ){ /* Insert an implicit AND operator. */ Fts3Expr *pAnd; assert( pRet && pPrev ); | | | 667 668 669 670 671 672 673 674 675 676 677 678 679 680 681 | goto exprparse_out; } if( isPhrase && !isRequirePhrase ){ /* Insert an implicit AND operator. */ Fts3Expr *pAnd; assert( pRet && pPrev ); pAnd = sqlite3Fts3MallocZero(sizeof(Fts3Expr)); if( !pAnd ){ sqlite3Fts3ExprFree(p); rc = SQLITE_NOMEM; goto exprparse_out; } pAnd->eType = FTSQUERY_AND; insertBinaryOperator(&pRet, pPrev, pAnd); |
︙ | ︙ | |||
792 793 794 795 796 797 798 | if( nMaxDepth==0 ){ rc = SQLITE_ERROR; } if( rc==SQLITE_OK ){ if( (eType==FTSQUERY_AND || eType==FTSQUERY_OR) ){ Fts3Expr **apLeaf; | | | 793 794 795 796 797 798 799 800 801 802 803 804 805 806 807 | if( nMaxDepth==0 ){ rc = SQLITE_ERROR; } if( rc==SQLITE_OK ){ if( (eType==FTSQUERY_AND || eType==FTSQUERY_OR) ){ Fts3Expr **apLeaf; apLeaf = (Fts3Expr **)sqlite3_malloc64(sizeof(Fts3Expr *) * nMaxDepth); if( 0==apLeaf ){ rc = SQLITE_NOMEM; }else{ memset(apLeaf, 0, sizeof(Fts3Expr *) * nMaxDepth); } if( rc==SQLITE_OK ){ |
︙ | ︙ | |||
1212 1213 1214 1215 1216 1217 1218 | sqlite3_free(zErr); return; } zExpr = (const char *)sqlite3_value_text(argv[1]); nExpr = sqlite3_value_bytes(argv[1]); nCol = argc-2; | | | 1213 1214 1215 1216 1217 1218 1219 1220 1221 1222 1223 1224 1225 1226 1227 | sqlite3_free(zErr); return; } zExpr = (const char *)sqlite3_value_text(argv[1]); nExpr = sqlite3_value_bytes(argv[1]); nCol = argc-2; azCol = (char **)sqlite3_malloc64(nCol*sizeof(char *)); if( !azCol ){ sqlite3_result_error_nomem(context); goto exprtest_out; } for(ii=0; ii<nCol; ii++){ azCol[ii] = (char *)sqlite3_value_text(argv[ii+2]); } |
︙ | ︙ |
Changes to ext/fts3/fts3_hash.c.
︙ | ︙ | |||
31 32 33 34 35 36 37 | #include <string.h> #include "fts3_hash.h" /* ** Malloc and Free functions */ | | | | 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 | #include <string.h> #include "fts3_hash.h" /* ** Malloc and Free functions */ static void *fts3HashMalloc(sqlite3_int64 n){ void *p = sqlite3_malloc64(n); if( p ){ memset(p, 0, n); } return p; } static void fts3HashFree(void *p){ sqlite3_free(p); |
︙ | ︙ |
Changes to ext/fts3/fts3_icu.c.
︙ | ︙ | |||
56 57 58 59 60 61 62 | ){ IcuTokenizer *p; int n = 0; if( argc>0 ){ n = strlen(argv[0])+1; } | | | 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 | ){ IcuTokenizer *p; int n = 0; if( argc>0 ){ n = strlen(argv[0])+1; } p = (IcuTokenizer *)sqlite3_malloc64(sizeof(IcuTokenizer)+n); if( !p ){ return SQLITE_NOMEM; } memset(p, 0, sizeof(IcuTokenizer)); if( n ){ p->zLocale = (char *)&p[1]; |
︙ | ︙ | |||
113 114 115 116 117 118 119 | if( zInput==0 ){ nInput = 0; zInput = ""; }else if( nInput<0 ){ nInput = strlen(zInput); } nChar = nInput+1; | | | 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 | if( zInput==0 ){ nInput = 0; zInput = ""; }else if( nInput<0 ){ nInput = strlen(zInput); } nChar = nInput+1; pCsr = (IcuCursor *)sqlite3_malloc64( sizeof(IcuCursor) + /* IcuCursor */ ((nChar+3)&~3) * sizeof(UChar) + /* IcuCursor.aChar[] */ (nChar+1) * sizeof(int) /* IcuCursor.aOffset[] */ ); if( !pCsr ){ return SQLITE_NOMEM; } |
︙ | ︙ |
Changes to ext/fts3/fts3_porter.c.
︙ | ︙ | |||
617 618 619 620 621 622 623 | } if( c->iOffset>iStartOffset ){ int n = c->iOffset-iStartOffset; if( n>c->nAllocated ){ char *pNew; c->nAllocated = n+20; | | | 617 618 619 620 621 622 623 624 625 626 627 628 629 630 631 | } if( c->iOffset>iStartOffset ){ int n = c->iOffset-iStartOffset; if( n>c->nAllocated ){ char *pNew; c->nAllocated = n+20; pNew = sqlite3_realloc64(c->zToken, c->nAllocated); if( !pNew ) return SQLITE_NOMEM; c->zToken = pNew; } porter_stemmer(&z[iStartOffset], n, c->zToken, pnBytes); *pzToken = c->zToken; *piStartOffset = iStartOffset; *piEndOffset = c->iOffset; |
︙ | ︙ |
Changes to ext/fts3/fts3_snippet.c.
︙ | ︙ | |||
13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 | #include "fts3Int.h" #if !defined(SQLITE_CORE) || defined(SQLITE_ENABLE_FTS3) #include <string.h> #include <assert.h> /* ** Characters that may appear in the second argument to matchinfo(). */ #define FTS3_MATCHINFO_NPHRASE 'p' /* 1 value */ #define FTS3_MATCHINFO_NCOL 'c' /* 1 value */ #define FTS3_MATCHINFO_NDOC 'n' /* 1 value */ #define FTS3_MATCHINFO_AVGLENGTH 'a' /* nCol values */ #define FTS3_MATCHINFO_LENGTH 'l' /* nCol values */ #define FTS3_MATCHINFO_LCS 's' /* nCol values */ #define FTS3_MATCHINFO_HITS 'x' /* 3*nCol*nPhrase values */ #define FTS3_MATCHINFO_LHITS 'y' /* nCol*nPhrase values */ #define FTS3_MATCHINFO_LHITS_BM 'b' /* nCol*nPhrase values */ /* ** The default value for the second argument to matchinfo(). */ #define FTS3_MATCHINFO_DEFAULT "pcx" /* | > > > > | | 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 | #include "fts3Int.h" #if !defined(SQLITE_CORE) || defined(SQLITE_ENABLE_FTS3) #include <string.h> #include <assert.h> #ifndef SQLITE_AMALGAMATION typedef sqlite3_int64 i64; #endif /* ** Characters that may appear in the second argument to matchinfo(). */ #define FTS3_MATCHINFO_NPHRASE 'p' /* 1 value */ #define FTS3_MATCHINFO_NCOL 'c' /* 1 value */ #define FTS3_MATCHINFO_NDOC 'n' /* 1 value */ #define FTS3_MATCHINFO_AVGLENGTH 'a' /* nCol values */ #define FTS3_MATCHINFO_LENGTH 'l' /* nCol values */ #define FTS3_MATCHINFO_LCS 's' /* nCol values */ #define FTS3_MATCHINFO_HITS 'x' /* 3*nCol*nPhrase values */ #define FTS3_MATCHINFO_LHITS 'y' /* nCol*nPhrase values */ #define FTS3_MATCHINFO_LHITS_BM 'b' /* nCol*nPhrase values */ /* ** The default value for the second argument to matchinfo(). */ #define FTS3_MATCHINFO_DEFAULT "pcx" /* ** Used as an sqlite3Fts3ExprIterate() context when loading phrase doclists to ** Fts3Expr.aDoclist[]/nDoclist. */ typedef struct LoadDoclistCtx LoadDoclistCtx; struct LoadDoclistCtx { Fts3Cursor *pCsr; /* FTS3 Cursor */ int nPhrase; /* Number of phrases seen so far */ int nToken; /* Number of tokens seen so far */ |
︙ | ︙ | |||
63 64 65 66 67 68 69 | SnippetPhrase *aPhrase; /* Array of size nPhrase */ int iCurrent; /* First token of current snippet */ }; struct SnippetPhrase { int nToken; /* Number of tokens in phrase */ char *pList; /* Pointer to start of phrase position list */ | | | | | 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 | SnippetPhrase *aPhrase; /* Array of size nPhrase */ int iCurrent; /* First token of current snippet */ }; struct SnippetPhrase { int nToken; /* Number of tokens in phrase */ char *pList; /* Pointer to start of phrase position list */ i64 iHead; /* Next value in position list */ char *pHead; /* Position list data following iHead */ i64 iTail; /* Next value in trailing position list */ char *pTail; /* Position list data following iTail */ }; struct SnippetFragment { int iCol; /* Column snippet is extracted from */ int iPos; /* Index of first token in snippet */ u64 covered; /* Mask of query phrases covered */ u64 hlmask; /* Mask of snippet terms to highlight */ }; /* ** This type is used as an sqlite3Fts3ExprIterate() context object while ** accumulating the data returned by the matchinfo() function. */ typedef struct MatchInfo MatchInfo; struct MatchInfo { Fts3Cursor *pCursor; /* FTS3 Cursor */ int nCol; /* Number of columns in table */ int nPhrase; /* Number of matchable phrases in query */ |
︙ | ︙ | |||
124 125 126 127 128 129 130 | /************************************************************************* ** Start of MatchinfoBuffer code. */ /* ** Allocate a two-slot MatchinfoBuffer object. */ | | > | | | < | > | | 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 | /************************************************************************* ** Start of MatchinfoBuffer code. */ /* ** Allocate a two-slot MatchinfoBuffer object. */ static MatchinfoBuffer *fts3MIBufferNew(size_t nElem, const char *zMatchinfo){ MatchinfoBuffer *pRet; sqlite3_int64 nByte = sizeof(u32) * (2*(sqlite3_int64)nElem + 1) + sizeof(MatchinfoBuffer); sqlite3_int64 nStr = strlen(zMatchinfo); pRet = sqlite3Fts3MallocZero(nByte + nStr+1); if( pRet ){ pRet->aMatchinfo[0] = (u8*)(&pRet->aMatchinfo[1]) - (u8*)pRet; pRet->aMatchinfo[1+nElem] = pRet->aMatchinfo[0] + sizeof(u32)*((int)nElem+1); pRet->nElem = (int)nElem; pRet->zMatchinfo = ((char*)pRet) + nByte; memcpy(pRet->zMatchinfo, zMatchinfo, nStr+1); pRet->aRef[0] = 1; } return pRet; } |
︙ | ︙ | |||
174 175 176 177 178 179 180 | xRet = fts3MIBufferFree; } else if( p->aRef[2]==0 ){ p->aRef[2] = 1; aOut = &p->aMatchinfo[p->nElem+2]; xRet = fts3MIBufferFree; }else{ | | | 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 | xRet = fts3MIBufferFree; } else if( p->aRef[2]==0 ){ p->aRef[2] = 1; aOut = &p->aMatchinfo[p->nElem+2]; xRet = fts3MIBufferFree; }else{ aOut = (u32*)sqlite3_malloc64(p->nElem * sizeof(u32)); if( aOut ){ xRet = sqlite3_free; if( p->bGlobal ) memcpy(aOut, &p->aMatchinfo[1], p->nElem*sizeof(u32)); } } *paOut = aOut; |
︙ | ︙ | |||
228 229 230 231 232 233 234 | ** are encoded. ** ** When this function is called, *pp points to the start of an element of ** the list. *piPos contains the value of the previous entry in the list. ** After it returns, *piPos contains the value of the next element of the ** list and *pp is advanced to the following varint. */ | | | | 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 | ** are encoded. ** ** When this function is called, *pp points to the start of an element of ** the list. *piPos contains the value of the previous entry in the list. ** After it returns, *piPos contains the value of the next element of the ** list and *pp is advanced to the following varint. */ static void fts3GetDeltaPosition(char **pp, i64 *piPos){ int iVal; *pp += fts3GetVarint32(*pp, &iVal); *piPos += (iVal-2); } /* ** Helper function for sqlite3Fts3ExprIterate() (see below). */ static int fts3ExprIterate2( Fts3Expr *pExpr, /* Expression to iterate phrases of */ int *piPhrase, /* Pointer to phrase counter */ int (*x)(Fts3Expr*,int,void*), /* Callback function to invoke for phrases */ void *pCtx /* Second argument to pass to callback */ ){ |
︙ | ︙ | |||
269 270 271 272 273 274 275 | ** For each phrase node found, the supplied callback function is invoked. ** ** If the callback function returns anything other than SQLITE_OK, ** the iteration is abandoned and the error code returned immediately. ** Otherwise, SQLITE_OK is returned after a callback has been made for ** all eligible phrase nodes. */ | | < | | | 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 | ** For each phrase node found, the supplied callback function is invoked. ** ** If the callback function returns anything other than SQLITE_OK, ** the iteration is abandoned and the error code returned immediately. ** Otherwise, SQLITE_OK is returned after a callback has been made for ** all eligible phrase nodes. */ int sqlite3Fts3ExprIterate( Fts3Expr *pExpr, /* Expression to iterate phrases of */ int (*x)(Fts3Expr*,int,void*), /* Callback function to invoke for phrases */ void *pCtx /* Second argument to pass to callback */ ){ int iPhrase = 0; /* Variable used as the phrase counter */ return fts3ExprIterate2(pExpr, &iPhrase, x, pCtx); } /* ** This is an sqlite3Fts3ExprIterate() callback used while loading the ** doclists for each phrase into Fts3Expr.aDoclist[]/nDoclist. See also ** fts3ExprLoadDoclists(). */ static int fts3ExprLoadDoclistsCb(Fts3Expr *pExpr, int iPhrase, void *ctx){ int rc = SQLITE_OK; Fts3Phrase *pPhrase = pExpr->pPhrase; LoadDoclistCtx *p = (LoadDoclistCtx *)ctx; |
︙ | ︙ | |||
313 314 315 316 317 318 319 | */ static int fts3ExprLoadDoclists( Fts3Cursor *pCsr, /* Fts3 cursor for current query */ int *pnPhrase, /* OUT: Number of phrases in query */ int *pnToken /* OUT: Number of tokens in query */ ){ int rc; /* Return Code */ | | | | | | | 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340 341 342 343 344 345 346 347 348 349 350 351 352 353 354 355 356 357 358 | */ static int fts3ExprLoadDoclists( Fts3Cursor *pCsr, /* Fts3 cursor for current query */ int *pnPhrase, /* OUT: Number of phrases in query */ int *pnToken /* OUT: Number of tokens in query */ ){ int rc; /* Return Code */ LoadDoclistCtx sCtx = {0,0,0}; /* Context for sqlite3Fts3ExprIterate() */ sCtx.pCsr = pCsr; rc = sqlite3Fts3ExprIterate(pCsr->pExpr,fts3ExprLoadDoclistsCb,(void*)&sCtx); if( pnPhrase ) *pnPhrase = sCtx.nPhrase; if( pnToken ) *pnToken = sCtx.nToken; return rc; } static int fts3ExprPhraseCountCb(Fts3Expr *pExpr, int iPhrase, void *ctx){ (*(int *)ctx)++; pExpr->iPhrase = iPhrase; return SQLITE_OK; } static int fts3ExprPhraseCount(Fts3Expr *pExpr){ int nPhrase = 0; (void)sqlite3Fts3ExprIterate(pExpr, fts3ExprPhraseCountCb, (void *)&nPhrase); return nPhrase; } /* ** Advance the position list iterator specified by the first two ** arguments so that it points to the first element with a value greater ** than or equal to parameter iNext. */ static void fts3SnippetAdvance(char **ppIter, i64 *piIter, int iNext){ char *pIter = *ppIter; if( pIter ){ i64 iIter = *piIter; while( iIter<iNext ){ if( 0==(*pIter & 0xFE) ){ iIter = -1; pIter = 0; break; } |
︙ | ︙ | |||
423 424 425 426 427 428 429 | u64 mCover = 0; /* Mask of phrases covered by this snippet */ u64 mHighlight = 0; /* Mask of tokens to highlight in snippet */ for(i=0; i<pIter->nPhrase; i++){ SnippetPhrase *pPhrase = &pIter->aPhrase[i]; if( pPhrase->pTail ){ char *pCsr = pPhrase->pTail; | | | | | > | 427 428 429 430 431 432 433 434 435 436 437 438 439 440 441 442 443 444 445 446 447 448 | u64 mCover = 0; /* Mask of phrases covered by this snippet */ u64 mHighlight = 0; /* Mask of tokens to highlight in snippet */ for(i=0; i<pIter->nPhrase; i++){ SnippetPhrase *pPhrase = &pIter->aPhrase[i]; if( pPhrase->pTail ){ char *pCsr = pPhrase->pTail; i64 iCsr = pPhrase->iTail; while( iCsr<(iStart+pIter->nSnippet) && iCsr>=iStart ){ int j; u64 mPhrase = (u64)1 << (i%64); u64 mPos = (u64)1 << (iCsr - iStart); assert( iCsr>=iStart && (iCsr - iStart)<=64 ); assert( i>=0 ); if( (mCover|mCovered)&mPhrase ){ iScore++; }else{ iScore += 1000; } mCover |= mPhrase; |
︙ | ︙ | |||
455 456 457 458 459 460 461 | *piToken = iStart; *piScore = iScore; *pmCover = mCover; *pmHighlight = mHighlight; } /* | | | > | | > > | | | | > | 460 461 462 463 464 465 466 467 468 469 470 471 472 473 474 475 476 477 478 479 480 481 482 483 484 485 486 487 488 489 490 491 492 493 494 495 496 497 498 | *piToken = iStart; *piScore = iScore; *pmCover = mCover; *pmHighlight = mHighlight; } /* ** This function is an sqlite3Fts3ExprIterate() callback used by ** fts3BestSnippet(). Each invocation populates an element of the ** SnippetIter.aPhrase[] array. */ static int fts3SnippetFindPositions(Fts3Expr *pExpr, int iPhrase, void *ctx){ SnippetIter *p = (SnippetIter *)ctx; SnippetPhrase *pPhrase = &p->aPhrase[iPhrase]; char *pCsr; int rc; pPhrase->nToken = pExpr->pPhrase->nToken; rc = sqlite3Fts3EvalPhrasePoslist(p->pCsr, pExpr, p->iCol, &pCsr); assert( rc==SQLITE_OK || pCsr==0 ); if( pCsr ){ i64 iFirst = 0; pPhrase->pList = pCsr; fts3GetDeltaPosition(&pCsr, &iFirst); if( iFirst<0 ){ rc = FTS_CORRUPT_VTAB; }else{ pPhrase->pHead = pCsr; pPhrase->pTail = pCsr; pPhrase->iHead = iFirst; pPhrase->iTail = iFirst; } }else{ assert( rc!=SQLITE_OK || ( pPhrase->pList==0 && pPhrase->pHead==0 && pPhrase->pTail==0 )); } return rc; |
︙ | ︙ | |||
512 513 514 515 516 517 518 | u64 *pmSeen, /* IN/OUT: Mask of phrases seen */ SnippetFragment *pFragment, /* OUT: Best snippet found */ int *piScore /* OUT: Score of snippet pFragment */ ){ int rc; /* Return Code */ int nList; /* Number of phrases in expression */ SnippetIter sIter; /* Iterates through snippet candidates */ | | | < > | > | | 521 522 523 524 525 526 527 528 529 530 531 532 533 534 535 536 537 538 539 540 541 542 543 544 545 546 547 548 549 550 551 552 553 554 555 556 557 558 559 560 561 562 563 564 565 566 567 568 569 570 571 572 573 574 | u64 *pmSeen, /* IN/OUT: Mask of phrases seen */ SnippetFragment *pFragment, /* OUT: Best snippet found */ int *piScore /* OUT: Score of snippet pFragment */ ){ int rc; /* Return Code */ int nList; /* Number of phrases in expression */ SnippetIter sIter; /* Iterates through snippet candidates */ sqlite3_int64 nByte; /* Number of bytes of space to allocate */ int iBestScore = -1; /* Best snippet score found so far */ int i; /* Loop counter */ memset(&sIter, 0, sizeof(sIter)); /* Iterate through the phrases in the expression to count them. The same ** callback makes sure the doclists are loaded for each phrase. */ rc = fts3ExprLoadDoclists(pCsr, &nList, 0); if( rc!=SQLITE_OK ){ return rc; } /* Now that it is known how many phrases there are, allocate and zero ** the required space using malloc(). */ nByte = sizeof(SnippetPhrase) * nList; sIter.aPhrase = (SnippetPhrase *)sqlite3Fts3MallocZero(nByte); if( !sIter.aPhrase ){ return SQLITE_NOMEM; } /* Initialize the contents of the SnippetIter object. Then iterate through ** the set of phrases in the expression to populate the aPhrase[] array. */ sIter.pCsr = pCsr; sIter.iCol = iCol; sIter.nSnippet = nSnippet; sIter.nPhrase = nList; sIter.iCurrent = -1; rc = sqlite3Fts3ExprIterate( pCsr->pExpr, fts3SnippetFindPositions, (void*)&sIter ); if( rc==SQLITE_OK ){ /* Set the *pmSeen output variable. */ for(i=0; i<nList; i++){ if( sIter.aPhrase[i].pHead ){ *pmSeen |= (u64)1 << (i%64); } } /* Loop through all candidate snippets. Store the best snippet in ** *pFragment. Store its associated 'score' in iBestScore. */ pFragment->iCol = iCol; |
︙ | ︙ | |||
600 601 602 603 604 605 606 | } /* If there is insufficient space allocated at StrBuffer.z, use realloc() ** to grow the buffer until so that it is big enough to accomadate the ** appended data. */ if( pStr->n+nAppend+1>=pStr->nAlloc ){ | | | | 610 611 612 613 614 615 616 617 618 619 620 621 622 623 624 625 | } /* If there is insufficient space allocated at StrBuffer.z, use realloc() ** to grow the buffer until so that it is big enough to accomadate the ** appended data. */ if( pStr->n+nAppend+1>=pStr->nAlloc ){ sqlite3_int64 nAlloc = pStr->nAlloc+(sqlite3_int64)nAppend+100; char *zNew = sqlite3_realloc64(pStr->z, nAlloc); if( !zNew ){ return SQLITE_NOMEM; } pStr->z = zNew; pStr->nAlloc = nAlloc; } assert( pStr->z!=0 && (pStr->nAlloc >= pStr->n+nAppend+1) ); |
︙ | ︙ | |||
656 657 658 659 660 661 662 663 664 665 666 667 668 669 | if( hlmask ){ int nLeft; /* Tokens to the left of first highlight */ int nRight; /* Tokens to the right of last highlight */ int nDesired; /* Ideal number of tokens to shift forward */ for(nLeft=0; !(hlmask & ((u64)1 << nLeft)); nLeft++); for(nRight=0; !(hlmask & ((u64)1 << (nSnippet-1-nRight))); nRight++); nDesired = (nLeft-nRight)/2; /* Ideally, the start of the snippet should be pushed forward in the ** document nDesired tokens. This block checks if there are actually ** nDesired tokens to the right of the snippet. If so, *piPos and ** *pHlMask are updated to shift the snippet nDesired tokens to the ** right. Otherwise, the snippet is shifted by the number of tokens | > | 666 667 668 669 670 671 672 673 674 675 676 677 678 679 680 | if( hlmask ){ int nLeft; /* Tokens to the left of first highlight */ int nRight; /* Tokens to the right of last highlight */ int nDesired; /* Ideal number of tokens to shift forward */ for(nLeft=0; !(hlmask & ((u64)1 << nLeft)); nLeft++); for(nRight=0; !(hlmask & ((u64)1 << (nSnippet-1-nRight))); nRight++); assert( (nSnippet-1-nRight)<=63 && (nSnippet-1-nRight)>=0 ); nDesired = (nLeft-nRight)/2; /* Ideally, the start of the snippet should be pushed forward in the ** document nDesired tokens. This block checks if there are actually ** nDesired tokens to the right of the snippet. If so, *piPos and ** *pHlMask are updated to shift the snippet nDesired tokens to the ** right. Otherwise, the snippet is shifted by the number of tokens |
︙ | ︙ | |||
848 849 850 851 852 853 854 | *ppCollist = pEnd; return nEntry; } /* ** This function gathers 'y' or 'b' data for a single phrase. */ | | | > > | > | | | > | | | | 859 860 861 862 863 864 865 866 867 868 869 870 871 872 873 874 875 876 877 878 879 880 881 882 883 884 885 886 887 888 889 890 891 892 893 894 895 896 897 898 899 900 901 902 903 904 905 906 907 908 909 910 911 912 913 914 915 916 917 918 919 920 921 922 923 924 925 926 927 928 929 930 931 932 | *ppCollist = pEnd; return nEntry; } /* ** This function gathers 'y' or 'b' data for a single phrase. */ static int fts3ExprLHits( Fts3Expr *pExpr, /* Phrase expression node */ MatchInfo *p /* Matchinfo context */ ){ Fts3Table *pTab = (Fts3Table *)p->pCursor->base.pVtab; int iStart; Fts3Phrase *pPhrase = pExpr->pPhrase; char *pIter = pPhrase->doclist.pList; int iCol = 0; assert( p->flag==FTS3_MATCHINFO_LHITS_BM || p->flag==FTS3_MATCHINFO_LHITS ); if( p->flag==FTS3_MATCHINFO_LHITS ){ iStart = pExpr->iPhrase * p->nCol; }else{ iStart = pExpr->iPhrase * ((p->nCol + 31) / 32); } if( pIter ) while( 1 ){ int nHit = fts3ColumnlistCount(&pIter); if( (pPhrase->iColumn>=pTab->nColumn || pPhrase->iColumn==iCol) ){ if( p->flag==FTS3_MATCHINFO_LHITS ){ p->aMatchinfo[iStart + iCol] = (u32)nHit; }else if( nHit ){ p->aMatchinfo[iStart + (iCol+1)/32] |= (1 << (iCol&0x1F)); } } assert( *pIter==0x00 || *pIter==0x01 ); if( *pIter!=0x01 ) break; pIter++; pIter += fts3GetVarint32(pIter, &iCol); if( iCol>=p->nCol ) return FTS_CORRUPT_VTAB; } return SQLITE_OK; } /* ** Gather the results for matchinfo directives 'y' and 'b'. */ static int fts3ExprLHitGather( Fts3Expr *pExpr, MatchInfo *p ){ int rc = SQLITE_OK; assert( (pExpr->pLeft==0)==(pExpr->pRight==0) ); if( pExpr->bEof==0 && pExpr->iDocid==p->pCursor->iPrevId ){ if( pExpr->pLeft ){ rc = fts3ExprLHitGather(pExpr->pLeft, p); if( rc==SQLITE_OK ) rc = fts3ExprLHitGather(pExpr->pRight, p); }else{ rc = fts3ExprLHits(pExpr, p); } } return rc; } /* ** sqlite3Fts3ExprIterate() callback used to collect the "global" matchinfo ** stats for a single query. ** ** sqlite3Fts3ExprIterate() callback to load the 'global' elements of a ** FTS3_MATCHINFO_HITS matchinfo array. The global stats are those elements ** of the matchinfo array that are constant for all rows returned by the ** current query. ** ** Argument pCtx is actually a pointer to a struct of type MatchInfo. This ** function populates Matchinfo.aMatchinfo[] as follows: ** |
︙ | ︙ | |||
938 939 940 941 942 943 944 | MatchInfo *p = (MatchInfo *)pCtx; return sqlite3Fts3EvalPhraseStats( p->pCursor, pExpr, &p->aMatchinfo[3*iPhrase*p->nCol] ); } /* | | | 953 954 955 956 957 958 959 960 961 962 963 964 965 966 967 | MatchInfo *p = (MatchInfo *)pCtx; return sqlite3Fts3EvalPhraseStats( p->pCursor, pExpr, &p->aMatchinfo[3*iPhrase*p->nCol] ); } /* ** sqlite3Fts3ExprIterate() callback used to collect the "local" part of the ** FTS3_MATCHINFO_HITS array. The local stats are those elements of the ** array that are different for each row returned by the query. */ static int fts3ExprLocalHitsCb( Fts3Expr *pExpr, /* Phrase expression node */ int iPhrase, /* Phrase number */ void *pCtx /* Pointer to MatchInfo structure */ |
︙ | ︙ | |||
986 987 988 989 990 991 992 | ){ return SQLITE_OK; } sqlite3Fts3ErrMsg(pzErr, "unrecognized matchinfo request: %c", cArg); return SQLITE_ERROR; } | | | | 1001 1002 1003 1004 1005 1006 1007 1008 1009 1010 1011 1012 1013 1014 1015 1016 | ){ return SQLITE_OK; } sqlite3Fts3ErrMsg(pzErr, "unrecognized matchinfo request: %c", cArg); return SQLITE_ERROR; } static size_t fts3MatchinfoSize(MatchInfo *pInfo, char cArg){ size_t nVal; /* Number of integers output by cArg */ switch( cArg ){ case FTS3_MATCHINFO_NDOC: case FTS3_MATCHINFO_NPHRASE: case FTS3_MATCHINFO_NCOL: nVal = 1; break; |
︙ | ︙ | |||
1023 1024 1025 1026 1027 1028 1029 | return nVal; } static int fts3MatchinfoSelectDoctotal( Fts3Table *pTab, sqlite3_stmt **ppStmt, sqlite3_int64 *pnDoc, | | > > > > > > > > > | > | > | > | 1038 1039 1040 1041 1042 1043 1044 1045 1046 1047 1048 1049 1050 1051 1052 1053 1054 1055 1056 1057 1058 1059 1060 1061 1062 1063 1064 1065 1066 1067 1068 1069 1070 1071 1072 1073 1074 1075 1076 1077 1078 1079 1080 1081 1082 | return nVal; } static int fts3MatchinfoSelectDoctotal( Fts3Table *pTab, sqlite3_stmt **ppStmt, sqlite3_int64 *pnDoc, const char **paLen, const char **ppEnd ){ sqlite3_stmt *pStmt; const char *a; const char *pEnd; sqlite3_int64 nDoc; int n; if( !*ppStmt ){ int rc = sqlite3Fts3SelectDoctotal(pTab, ppStmt); if( rc!=SQLITE_OK ) return rc; } pStmt = *ppStmt; assert( sqlite3_data_count(pStmt)==1 ); n = sqlite3_column_bytes(pStmt, 0); a = sqlite3_column_blob(pStmt, 0); if( a==0 ){ return FTS_CORRUPT_VTAB; } pEnd = a + n; a += sqlite3Fts3GetVarintBounded(a, pEnd, &nDoc); if( nDoc<=0 || a>pEnd ){ return FTS_CORRUPT_VTAB; } *pnDoc = nDoc; if( paLen ) *paLen = a; if( ppEnd ) *ppEnd = pEnd; return SQLITE_OK; } /* ** An instance of the following structure is used to store state while ** iterating through a multi-column position-list corresponding to the ** hits for a single phrase on a single row in order to calculate the |
︙ | ︙ | |||
1081 1082 1083 1084 1085 1086 1087 | /* ** Advance the iterator passed as an argument to the next position. Return ** 1 if the iterator is at EOF or if it now points to the start of the ** position list for the next column. */ static int fts3LcsIteratorAdvance(LcsIterator *pIter){ | | > > | 1108 1109 1110 1111 1112 1113 1114 1115 1116 1117 1118 1119 1120 1121 1122 1123 1124 1125 1126 1127 | /* ** Advance the iterator passed as an argument to the next position. Return ** 1 if the iterator is at EOF or if it now points to the start of the ** position list for the next column. */ static int fts3LcsIteratorAdvance(LcsIterator *pIter){ char *pRead; sqlite3_int64 iRead; int rc = 0; if( NEVER(pIter==0) ) return 1; pRead = pIter->pRead; pRead += sqlite3Fts3GetVarint(pRead, &iRead); if( iRead==0 || iRead==1 ){ pRead = 0; rc = 1; }else{ pIter->iPos += (int)(iRead-2); } |
︙ | ︙ | |||
1113 1114 1115 1116 1117 1118 1119 1120 1121 1122 1123 | ** undefined. */ static int fts3MatchinfoLcs(Fts3Cursor *pCsr, MatchInfo *pInfo){ LcsIterator *aIter; int i; int iCol; int nToken = 0; /* Allocate and populate the array of LcsIterator objects. The array ** contains one element for each matchable phrase in the query. **/ | > | < | < | | > > > > | 1142 1143 1144 1145 1146 1147 1148 1149 1150 1151 1152 1153 1154 1155 1156 1157 1158 1159 1160 1161 1162 1163 1164 1165 1166 1167 1168 1169 1170 1171 1172 1173 1174 1175 1176 1177 1178 1179 1180 1181 1182 1183 1184 1185 | ** undefined. */ static int fts3MatchinfoLcs(Fts3Cursor *pCsr, MatchInfo *pInfo){ LcsIterator *aIter; int i; int iCol; int nToken = 0; int rc = SQLITE_OK; /* Allocate and populate the array of LcsIterator objects. The array ** contains one element for each matchable phrase in the query. **/ aIter = sqlite3Fts3MallocZero(sizeof(LcsIterator) * pCsr->nPhrase); if( !aIter ) return SQLITE_NOMEM; (void)sqlite3Fts3ExprIterate(pCsr->pExpr, fts3MatchinfoLcsCb, (void*)aIter); for(i=0; i<pInfo->nPhrase; i++){ LcsIterator *pIter = &aIter[i]; nToken -= pIter->pExpr->pPhrase->nToken; pIter->iPosOffset = nToken; } for(iCol=0; iCol<pInfo->nCol; iCol++){ int nLcs = 0; /* LCS value for this column */ int nLive = 0; /* Number of iterators in aIter not at EOF */ for(i=0; i<pInfo->nPhrase; i++){ LcsIterator *pIt = &aIter[i]; rc = sqlite3Fts3EvalPhrasePoslist(pCsr, pIt->pExpr, iCol, &pIt->pRead); if( rc!=SQLITE_OK ) goto matchinfo_lcs_out; if( pIt->pRead ){ pIt->iPos = pIt->iPosOffset; fts3LcsIteratorAdvance(pIt); if( pIt->pRead==0 ){ rc = FTS_CORRUPT_VTAB; goto matchinfo_lcs_out; } nLive++; } } while( nLive>0 ){ LcsIterator *pAdv = 0; /* The iterator to advance by one position */ int nThisLcs = 0; /* LCS for the current iterator positions */ |
︙ | ︙ | |||
1171 1172 1173 1174 1175 1176 1177 1178 | } if( fts3LcsIteratorAdvance(pAdv) ) nLive--; } pInfo->aMatchinfo[iCol] = nLcs; } sqlite3_free(aIter); | > | | 1203 1204 1205 1206 1207 1208 1209 1210 1211 1212 1213 1214 1215 1216 1217 1218 1219 | } if( fts3LcsIteratorAdvance(pAdv) ) nLive--; } pInfo->aMatchinfo[iCol] = nLcs; } matchinfo_lcs_out: sqlite3_free(aIter); return rc; } /* ** Populate the buffer pInfo->aMatchinfo[] with an array of integers to ** be returned by the matchinfo() function. Argument zArg contains the ** format string passed as the second argument to matchinfo (or the ** default value "pcx" if no second argument was specified). The format |
︙ | ︙ | |||
1217 1218 1219 1220 1221 1222 1223 | case FTS3_MATCHINFO_NCOL: if( bGlobal ) pInfo->aMatchinfo[0] = pInfo->nCol; break; case FTS3_MATCHINFO_NDOC: if( bGlobal ){ sqlite3_int64 nDoc = 0; | | > | > > > > > | > > > > | | | | | | 1250 1251 1252 1253 1254 1255 1256 1257 1258 1259 1260 1261 1262 1263 1264 1265 1266 1267 1268 1269 1270 1271 1272 1273 1274 1275 1276 1277 1278 1279 1280 1281 1282 1283 1284 1285 1286 1287 1288 1289 1290 1291 1292 1293 1294 1295 1296 1297 1298 1299 1300 1301 1302 1303 1304 1305 1306 1307 1308 1309 1310 1311 1312 1313 1314 1315 1316 1317 1318 1319 1320 1321 1322 1323 1324 1325 1326 1327 1328 1329 1330 1331 1332 1333 1334 1335 1336 1337 1338 1339 1340 1341 1342 1343 1344 | case FTS3_MATCHINFO_NCOL: if( bGlobal ) pInfo->aMatchinfo[0] = pInfo->nCol; break; case FTS3_MATCHINFO_NDOC: if( bGlobal ){ sqlite3_int64 nDoc = 0; rc = fts3MatchinfoSelectDoctotal(pTab, &pSelect, &nDoc, 0, 0); pInfo->aMatchinfo[0] = (u32)nDoc; } break; case FTS3_MATCHINFO_AVGLENGTH: if( bGlobal ){ sqlite3_int64 nDoc; /* Number of rows in table */ const char *a; /* Aggregate column length array */ const char *pEnd; /* First byte past end of length array */ rc = fts3MatchinfoSelectDoctotal(pTab, &pSelect, &nDoc, &a, &pEnd); if( rc==SQLITE_OK ){ int iCol; for(iCol=0; iCol<pInfo->nCol; iCol++){ u32 iVal; sqlite3_int64 nToken; a += sqlite3Fts3GetVarint(a, &nToken); if( a>pEnd ){ rc = SQLITE_CORRUPT_VTAB; break; } iVal = (u32)(((u32)(nToken&0xffffffff)+nDoc/2)/nDoc); pInfo->aMatchinfo[iCol] = iVal; } } } break; case FTS3_MATCHINFO_LENGTH: { sqlite3_stmt *pSelectDocsize = 0; rc = sqlite3Fts3SelectDocsize(pTab, pCsr->iPrevId, &pSelectDocsize); if( rc==SQLITE_OK ){ int iCol; const char *a = sqlite3_column_blob(pSelectDocsize, 0); const char *pEnd = a + sqlite3_column_bytes(pSelectDocsize, 0); for(iCol=0; iCol<pInfo->nCol; iCol++){ sqlite3_int64 nToken; a += sqlite3Fts3GetVarintBounded(a, pEnd, &nToken); if( a>pEnd ){ rc = SQLITE_CORRUPT_VTAB; break; } pInfo->aMatchinfo[iCol] = (u32)nToken; } } sqlite3_reset(pSelectDocsize); break; } case FTS3_MATCHINFO_LCS: rc = fts3ExprLoadDoclists(pCsr, 0, 0); if( rc==SQLITE_OK ){ rc = fts3MatchinfoLcs(pCsr, pInfo); } break; case FTS3_MATCHINFO_LHITS_BM: case FTS3_MATCHINFO_LHITS: { size_t nZero = fts3MatchinfoSize(pInfo, zArg[i]) * sizeof(u32); memset(pInfo->aMatchinfo, 0, nZero); rc = fts3ExprLHitGather(pCsr->pExpr, pInfo); break; } default: { Fts3Expr *pExpr; assert( zArg[i]==FTS3_MATCHINFO_HITS ); pExpr = pCsr->pExpr; rc = fts3ExprLoadDoclists(pCsr, 0, 0); if( rc!=SQLITE_OK ) break; if( bGlobal ){ if( pCsr->pDeferred ){ rc = fts3MatchinfoSelectDoctotal(pTab, &pSelect, &pInfo->nDoc,0,0); if( rc!=SQLITE_OK ) break; } rc = sqlite3Fts3ExprIterate(pExpr, fts3ExprGlobalHitsCb,(void*)pInfo); sqlite3Fts3EvalTestDeferred(pCsr, &rc); if( rc!=SQLITE_OK ) break; } (void)sqlite3Fts3ExprIterate(pExpr, fts3ExprLocalHitsCb,(void*)pInfo); break; } } pInfo->aMatchinfo += fts3MatchinfoSize(pInfo, zArg[i]); } |
︙ | ︙ | |||
1335 1336 1337 1338 1339 1340 1341 | /* If Fts3Cursor.pMIBuffer is NULL, then this is the first time the ** matchinfo function has been called for this query. In this case ** allocate the array used to accumulate the matchinfo data and ** initialize those elements that are constant for every row. */ if( pCsr->pMIBuffer==0 ){ | | | 1378 1379 1380 1381 1382 1383 1384 1385 1386 1387 1388 1389 1390 1391 1392 | /* If Fts3Cursor.pMIBuffer is NULL, then this is the first time the ** matchinfo function has been called for this query. In this case ** allocate the array used to accumulate the matchinfo data and ** initialize those elements that are constant for every row. */ if( pCsr->pMIBuffer==0 ){ size_t nMatchinfo = 0; /* Number of u32 elements in match-info */ int i; /* Used to iterate through zArg */ /* Determine the number of phrases in the query */ pCsr->nPhrase = fts3ExprPhraseCount(pCsr->pExpr); sInfo.nPhrase = pCsr->nPhrase; /* Determine the number of integers in the buffer returned by this call. */ |
︙ | ︙ | |||
1419 1420 1421 1422 1423 1424 1425 1426 1427 1428 1429 1430 1431 1432 | SnippetFragment aSnippet[4]; /* Maximum of 4 fragments per snippet */ int nFToken = -1; /* Number of tokens in each fragment */ if( !pCsr->pExpr ){ sqlite3_result_text(pCtx, "", 0, SQLITE_STATIC); return; } for(nSnippet=1; 1; nSnippet++){ int iSnip; /* Loop counter 0..nSnippet-1 */ u64 mCovered = 0; /* Bitmask of phrases covered by snippet */ u64 mSeen = 0; /* Bitmask of phrases seen by BestSnippet() */ | > > > > | 1462 1463 1464 1465 1466 1467 1468 1469 1470 1471 1472 1473 1474 1475 1476 1477 1478 1479 | SnippetFragment aSnippet[4]; /* Maximum of 4 fragments per snippet */ int nFToken = -1; /* Number of tokens in each fragment */ if( !pCsr->pExpr ){ sqlite3_result_text(pCtx, "", 0, SQLITE_STATIC); return; } /* Limit the snippet length to 64 tokens. */ if( nToken<-64 ) nToken = -64; if( nToken>+64 ) nToken = +64; for(nSnippet=1; 1; nSnippet++){ int iSnip; /* Loop counter 0..nSnippet-1 */ u64 mCovered = 0; /* Bitmask of phrases covered by snippet */ u64 mSeen = 0; /* Bitmask of phrases seen by BestSnippet() */ |
︙ | ︙ | |||
1493 1494 1495 1496 1497 1498 1499 | typedef struct TermOffset TermOffset; typedef struct TermOffsetCtx TermOffsetCtx; struct TermOffset { char *pList; /* Position-list */ | | | | | | | 1540 1541 1542 1543 1544 1545 1546 1547 1548 1549 1550 1551 1552 1553 1554 1555 1556 1557 1558 1559 1560 1561 1562 1563 1564 1565 1566 1567 1568 1569 1570 1571 1572 1573 1574 1575 1576 1577 1578 1579 1580 1581 1582 | typedef struct TermOffset TermOffset; typedef struct TermOffsetCtx TermOffsetCtx; struct TermOffset { char *pList; /* Position-list */ i64 iPos; /* Position just read from pList */ i64 iOff; /* Offset of this term from read positions */ }; struct TermOffsetCtx { Fts3Cursor *pCsr; int iCol; /* Column of table to populate aTerm for */ int iTerm; sqlite3_int64 iDocid; TermOffset *aTerm; }; /* ** This function is an sqlite3Fts3ExprIterate() callback used by sqlite3Fts3Offsets(). */ static int fts3ExprTermOffsetInit(Fts3Expr *pExpr, int iPhrase, void *ctx){ TermOffsetCtx *p = (TermOffsetCtx *)ctx; int nTerm; /* Number of tokens in phrase */ int iTerm; /* For looping through nTerm phrase terms */ char *pList; /* Pointer to position list for phrase */ i64 iPos = 0; /* First position in position-list */ int rc; UNUSED_PARAMETER(iPhrase); rc = sqlite3Fts3EvalPhrasePoslist(p->pCsr, pExpr, p->iCol, &pList); nTerm = pExpr->pPhrase->nToken; if( pList ){ fts3GetDeltaPosition(&pList, &iPos); assert_fts3_nc( iPos>=0 ); } for(iTerm=0; iTerm<nTerm; iTerm++){ TermOffset *pT = &p->aTerm[p->iTerm++]; pT->iOff = nTerm-iTerm-1; pT->pList = pList; pT->iPos = iPos; |
︙ | ︙ | |||
1562 1563 1564 1565 1566 1567 1568 | assert( pCsr->isRequireSeek==0 ); /* Count the number of terms in the query */ rc = fts3ExprLoadDoclists(pCsr, 0, &nToken); if( rc!=SQLITE_OK ) goto offsets_out; /* Allocate the array of TermOffset iterators. */ | | | | < > | > > | 1609 1610 1611 1612 1613 1614 1615 1616 1617 1618 1619 1620 1621 1622 1623 1624 1625 1626 1627 1628 1629 1630 1631 1632 1633 1634 1635 1636 1637 1638 1639 1640 1641 1642 1643 1644 1645 1646 1647 1648 1649 1650 1651 1652 | assert( pCsr->isRequireSeek==0 ); /* Count the number of terms in the query */ rc = fts3ExprLoadDoclists(pCsr, 0, &nToken); if( rc!=SQLITE_OK ) goto offsets_out; /* Allocate the array of TermOffset iterators. */ sCtx.aTerm = (TermOffset *)sqlite3Fts3MallocZero(sizeof(TermOffset)*nToken); if( 0==sCtx.aTerm ){ rc = SQLITE_NOMEM; goto offsets_out; } sCtx.iDocid = pCsr->iPrevId; sCtx.pCsr = pCsr; /* Loop through the table columns, appending offset information to ** string-buffer res for each column. */ for(iCol=0; iCol<pTab->nColumn; iCol++){ sqlite3_tokenizer_cursor *pC; /* Tokenizer cursor */ const char *ZDUMMY; /* Dummy argument used with xNext() */ int NDUMMY = 0; /* Dummy argument used with xNext() */ int iStart = 0; int iEnd = 0; int iCurrent = 0; const char *zDoc; int nDoc; /* Initialize the contents of sCtx.aTerm[] for column iCol. This ** operation may fail if the database contains corrupt records. */ sCtx.iCol = iCol; sCtx.iTerm = 0; rc = sqlite3Fts3ExprIterate( pCsr->pExpr, fts3ExprTermOffsetInit, (void*)&sCtx ); if( rc!=SQLITE_OK ) goto offsets_out; /* Retreive the text stored in column iCol. If an SQL NULL is stored ** in column iCol, jump immediately to the next iteration of the loop. ** If an OOM occurs while retrieving the data (this can happen if SQLite ** needs to transform the data from utf-16 to utf-8), return SQLITE_NOMEM ** to the caller. */ |
︙ | ︙ | |||
1631 1632 1633 1634 1635 1636 1637 | } } if( !pTerm ){ /* All offsets for this column have been gathered. */ rc = SQLITE_DONE; }else{ | | | 1680 1681 1682 1683 1684 1685 1686 1687 1688 1689 1690 1691 1692 1693 1694 | } } if( !pTerm ){ /* All offsets for this column have been gathered. */ rc = SQLITE_DONE; }else{ assert_fts3_nc( iCurrent<=iMinPos ); if( 0==(0xFE&*pTerm->pList) ){ pTerm->pList = 0; }else{ fts3GetDeltaPosition(&pTerm->pList, &pTerm->iPos); } while( rc==SQLITE_OK && iCurrent<iMinPos ){ rc = pMod->xNext(pC, &ZDUMMY, &NDUMMY, &iStart, &iEnd, &iCurrent); |
︙ | ︙ |
Changes to ext/fts3/fts3_term.c.
︙ | ︙ | |||
64 65 66 67 68 69 70 | sqlite3_vtab **ppVtab, /* OUT: New sqlite3_vtab object */ char **pzErr /* OUT: sqlite3_malloc'd error message */ ){ char const *zDb; /* Name of database (e.g. "main") */ char const *zFts3; /* Name of fts3 table */ int nDb; /* Result of strlen(zDb) */ int nFts3; /* Result of strlen(zFts3) */ | | | | 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 | sqlite3_vtab **ppVtab, /* OUT: New sqlite3_vtab object */ char **pzErr /* OUT: sqlite3_malloc'd error message */ ){ char const *zDb; /* Name of database (e.g. "main") */ char const *zFts3; /* Name of fts3 table */ int nDb; /* Result of strlen(zDb) */ int nFts3; /* Result of strlen(zFts3) */ sqlite3_int64 nByte; /* Bytes of space to allocate here */ int rc; /* value returned by declare_vtab() */ Fts3termTable *p; /* Virtual table object to return */ int iIndex = 0; UNUSED_PARAMETER(pCtx); if( argc==5 ){ iIndex = atoi(argv[4]); argc--; } |
︙ | ︙ | |||
92 93 94 95 96 97 98 | zFts3 = argv[3]; nFts3 = (int)strlen(zFts3); rc = sqlite3_declare_vtab(db, FTS3_TERMS_SCHEMA); if( rc!=SQLITE_OK ) return rc; nByte = sizeof(Fts3termTable) + sizeof(Fts3Table) + nDb + nFts3 + 2; | | | | 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 | zFts3 = argv[3]; nFts3 = (int)strlen(zFts3); rc = sqlite3_declare_vtab(db, FTS3_TERMS_SCHEMA); if( rc!=SQLITE_OK ) return rc; nByte = sizeof(Fts3termTable) + sizeof(Fts3Table) + nDb + nFts3 + 2; p = (Fts3termTable *)sqlite3_malloc64(nByte); if( !p ) return SQLITE_NOMEM; memset(p, 0, (size_t)nByte); p->pFts3Tab = (Fts3Table *)&p[1]; p->pFts3Tab->zDb = (char *)&p->pFts3Tab[1]; p->pFts3Tab->zName = &p->pFts3Tab->zDb[nDb+1]; p->pFts3Tab->db = db; p->pFts3Tab->nIndex = iIndex+1; p->iIndex = iIndex; |
︙ | ︙ |
Changes to ext/fts3/fts3_test.c.
︙ | ︙ | |||
444 445 446 447 448 449 450 | while( p<pEnd && testIsTokenChar(*p)==0 ) p++; if( p==pEnd ){ rc = SQLITE_DONE; }else{ /* Advance to the end of the token */ const char *pToken = p; | | | | | | 444 445 446 447 448 449 450 451 452 453 454 455 456 457 458 459 460 461 462 463 464 465 466 467 468 469 470 471 472 473 474 475 476 477 478 479 480 481 | while( p<pEnd && testIsTokenChar(*p)==0 ) p++; if( p==pEnd ){ rc = SQLITE_DONE; }else{ /* Advance to the end of the token */ const char *pToken = p; sqlite3_int64 nToken; while( p<pEnd && testIsTokenChar(*p) ) p++; nToken = (sqlite3_int64)(p-pToken); /* Copy the token into the buffer */ if( nToken>pCsr->nBuffer ){ sqlite3_free(pCsr->aBuffer); pCsr->aBuffer = sqlite3_malloc64(nToken); } if( pCsr->aBuffer==0 ){ rc = SQLITE_NOMEM; }else{ int i; if( pCsr->iLangid & 0x00000001 ){ for(i=0; i<nToken; i++) pCsr->aBuffer[i] = pToken[i]; }else{ for(i=0; i<nToken; i++) pCsr->aBuffer[i] = (char)testTolower(pToken[i]); } pCsr->iToken++; pCsr->iInput = (int)(p - pCsr->aInput); *ppToken = pCsr->aBuffer; *pnBytes = (int)nToken; *piStartOffset = (int)(pToken - pCsr->aInput); *piEndOffset = (int)(p - pCsr->aInput); *piPosition = pCsr->iToken; } } return rc; |
︙ | ︙ | |||
569 570 571 572 573 574 575 576 577 578 579 580 581 582 583 584 | UNUSED_PARAMETER(clientData); return TCL_OK; } /* ** End of tokenizer code. **************************************************************************/ int Sqlitetestfts3_Init(Tcl_Interp *interp){ Tcl_CreateObjCommand(interp, "fts3_near_match", fts3_near_match_cmd, 0, 0); Tcl_CreateObjCommand(interp, "fts3_configure_incr_load", fts3_configure_incr_load_cmd, 0, 0 ); Tcl_CreateObjCommand( interp, "fts3_test_tokenizer", fts3_test_tokenizer_cmd, 0, 0 ); | > > > > > > > > > > > > > > > > > > > > > > > > > > > > > < > > > | 569 570 571 572 573 574 575 576 577 578 579 580 581 582 583 584 585 586 587 588 589 590 591 592 593 594 595 596 597 598 599 600 601 602 603 604 605 606 607 608 609 610 611 612 613 614 615 616 617 618 619 620 621 622 623 | UNUSED_PARAMETER(clientData); return TCL_OK; } /* ** End of tokenizer code. **************************************************************************/ /* ** sqlite3_fts3_may_be_corrupt BOOLEAN ** ** Set or clear the global "may-be-corrupt" flag. Return the old value. */ static int SQLITE_TCLAPI fts3_may_be_corrupt( void * clientData, Tcl_Interp *interp, int objc, Tcl_Obj *CONST objv[] ){ #ifdef SQLITE_DEBUG int bOld = sqlite3_fts3_may_be_corrupt; if( objc!=2 && objc!=1 ){ Tcl_WrongNumArgs(interp, 1, objv, "?BOOLEAN?"); return TCL_ERROR; } if( objc==2 ){ int bNew; if( Tcl_GetBooleanFromObj(interp, objv[1], &bNew) ) return TCL_ERROR; sqlite3_fts3_may_be_corrupt = bNew; } Tcl_SetObjResult(interp, Tcl_NewIntObj(bOld)); #endif return TCL_OK; } int Sqlitetestfts3_Init(Tcl_Interp *interp){ Tcl_CreateObjCommand(interp, "fts3_near_match", fts3_near_match_cmd, 0, 0); Tcl_CreateObjCommand(interp, "fts3_configure_incr_load", fts3_configure_incr_load_cmd, 0, 0 ); Tcl_CreateObjCommand( interp, "fts3_test_tokenizer", fts3_test_tokenizer_cmd, 0, 0 ); Tcl_CreateObjCommand( interp, "fts3_test_varint", fts3_test_varint_cmd, 0, 0 ); Tcl_CreateObjCommand( interp, "sqlite3_fts3_may_be_corrupt", fts3_may_be_corrupt, 0, 0 ); return TCL_OK; } #endif /* SQLITE_ENABLE_FTS3 || SQLITE_ENABLE_FTS4 */ #endif /* ifdef SQLITE_TEST */ |
Changes to ext/fts3/fts3_tokenize_vtab.c.
︙ | ︙ | |||
118 119 120 121 122 123 124 | int nByte = 0; char **azDequote; for(i=0; i<argc; i++){ nByte += (int)(strlen(argv[i]) + 1); } | | | 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 | int nByte = 0; char **azDequote; for(i=0; i<argc; i++){ nByte += (int)(strlen(argv[i]) + 1); } *pazDequote = azDequote = sqlite3_malloc64(sizeof(char *)*argc + nByte); if( azDequote==0 ){ rc = SQLITE_NOMEM; }else{ char *pSpace = (char *)&azDequote[argc]; for(i=0; i<argc; i++){ int n = (int)strlen(argv[i]); azDequote[i] = pSpace; |
︙ | ︙ | |||
184 185 186 187 188 189 190 | zModule = azDequote[0]; } rc = fts3tokQueryTokenizer((Fts3Hash*)pHash, zModule, &pMod, pzErr); } assert( (rc==SQLITE_OK)==(pMod!=0) ); if( rc==SQLITE_OK ){ | | > | 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 | zModule = azDequote[0]; } rc = fts3tokQueryTokenizer((Fts3Hash*)pHash, zModule, &pMod, pzErr); } assert( (rc==SQLITE_OK)==(pMod!=0) ); if( rc==SQLITE_OK ){ const char * const *azArg = 0; if( nDequote>1 ) azArg = (const char * const *)&azDequote[1]; rc = pMod->xCreate((nDequote>1 ? nDequote-1 : 0), azArg, &pTok); } if( rc==SQLITE_OK ){ pTab = (Fts3tokTable *)sqlite3_malloc(sizeof(Fts3tokTable)); if( pTab==0 ){ rc = SQLITE_NOMEM; |
︙ | ︙ | |||
342 343 344 345 346 347 348 | UNUSED_PARAMETER(idxStr); UNUSED_PARAMETER(nVal); fts3tokResetCursor(pCsr); if( idxNum==1 ){ const char *zByte = (const char *)sqlite3_value_text(apVal[0]); int nByte = sqlite3_value_bytes(apVal[0]); | | | | 343 344 345 346 347 348 349 350 351 352 353 354 355 356 357 358 359 360 361 | UNUSED_PARAMETER(idxStr); UNUSED_PARAMETER(nVal); fts3tokResetCursor(pCsr); if( idxNum==1 ){ const char *zByte = (const char *)sqlite3_value_text(apVal[0]); int nByte = sqlite3_value_bytes(apVal[0]); pCsr->zInput = sqlite3_malloc64(nByte+1); if( pCsr->zInput==0 ){ rc = SQLITE_NOMEM; }else{ if( nByte>0 ) memcpy(pCsr->zInput, zByte, nByte); pCsr->zInput[nByte] = 0; rc = pTab->pMod->xOpen(pTab->pTok, pCsr->zInput, nByte, &pCsr->pCsr); if( rc==SQLITE_OK ){ pCsr->pCsr->pTokenizer = pTab->pTok; } } } |
︙ | ︙ | |||
415 416 417 418 419 420 421 | return SQLITE_OK; } /* ** Register the fts3tok module with database connection db. Return SQLITE_OK ** if successful or an error code if sqlite3_create_module() fails. */ | | | 416 417 418 419 420 421 422 423 424 425 426 427 428 429 430 | return SQLITE_OK; } /* ** Register the fts3tok module with database connection db. Return SQLITE_OK ** if successful or an error code if sqlite3_create_module() fails. */ int sqlite3Fts3InitTok(sqlite3 *db, Fts3Hash *pHash, void(*xDestroy)(void*)){ static const sqlite3_module fts3tok_module = { 0, /* iVersion */ fts3tokConnectMethod, /* xCreate */ fts3tokConnectMethod, /* xConnect */ fts3tokBestIndexMethod, /* xBestIndex */ fts3tokDisconnectMethod, /* xDisconnect */ fts3tokDisconnectMethod, /* xDestroy */ |
︙ | ︙ | |||
444 445 446 447 448 449 450 | 0, /* xSavepoint */ 0, /* xRelease */ 0, /* xRollbackTo */ 0 /* xShadowName */ }; int rc; /* Return code */ | | > > | 445 446 447 448 449 450 451 452 453 454 455 456 457 458 | 0, /* xSavepoint */ 0, /* xRelease */ 0, /* xRollbackTo */ 0 /* xShadowName */ }; int rc; /* Return code */ rc = sqlite3_create_module_v2( db, "fts3tokenize", &fts3tok_module, (void*)pHash, xDestroy ); return rc; } #endif /* !defined(SQLITE_CORE) || defined(SQLITE_ENABLE_FTS3) */ |
Changes to ext/fts3/fts3_tokenizer.c.
︙ | ︙ | |||
75 76 77 78 79 80 81 | pHash = (Fts3Hash *)sqlite3_user_data(context); zName = sqlite3_value_text(argv[0]); nName = sqlite3_value_bytes(argv[0])+1; if( argc==2 ){ | | | 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 | pHash = (Fts3Hash *)sqlite3_user_data(context); zName = sqlite3_value_text(argv[0]); nName = sqlite3_value_bytes(argv[0])+1; if( argc==2 ){ if( fts3TokenizerEnabled(context) || sqlite3_value_frombind(argv[1]) ){ void *pOld; int n = sqlite3_value_bytes(argv[1]); if( zName==0 || n!=sizeof(pPtr) ){ sqlite3_result_error(context, "argument type mismatch", -1); return; } pPtr = *(void **)sqlite3_value_blob(argv[1]); |
︙ | ︙ | |||
102 103 104 105 106 107 108 | if( !pPtr ){ char *zErr = sqlite3_mprintf("unknown tokenizer: %s", zName); sqlite3_result_error(context, zErr, -1); sqlite3_free(zErr); return; } } | > | > | 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 | if( !pPtr ){ char *zErr = sqlite3_mprintf("unknown tokenizer: %s", zName); sqlite3_result_error(context, zErr, -1); sqlite3_free(zErr); return; } } if( fts3TokenizerEnabled(context) || sqlite3_value_frombind(argv[0]) ){ sqlite3_result_blob(context, (void *)&pPtr, sizeof(pPtr), SQLITE_TRANSIENT); } } int sqlite3Fts3IsIdChar(char c){ static const char isFtsIdChar[] = { 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, /* 0x */ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, /* 1x */ 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, /* 2x */ |
︙ | ︙ | |||
190 191 192 193 194 195 196 | sqlite3Fts3ErrMsg(pzErr, "unknown tokenizer: %s", z); rc = SQLITE_ERROR; }else{ char const **aArg = 0; int iArg = 0; z = &z[n+1]; while( z<zEnd && (NULL!=(z = (char *)sqlite3Fts3NextToken(z, &n))) ){ | | | | 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 | sqlite3Fts3ErrMsg(pzErr, "unknown tokenizer: %s", z); rc = SQLITE_ERROR; }else{ char const **aArg = 0; int iArg = 0; z = &z[n+1]; while( z<zEnd && (NULL!=(z = (char *)sqlite3Fts3NextToken(z, &n))) ){ sqlite3_int64 nNew = sizeof(char *)*(iArg+1); char const **aNew = (const char **)sqlite3_realloc64((void *)aArg, nNew); if( !aNew ){ sqlite3_free(zCopy); sqlite3_free((void *)aArg); return SQLITE_NOMEM; } aArg = aNew; aArg[iArg++] = z; |
︙ | ︙ | |||
384 385 386 387 388 389 390 | rc = sqlite3_prepare_v2(db, zSql, -1, &pStmt, 0); if( rc!=SQLITE_OK ){ return rc; } sqlite3_bind_text(pStmt, 1, zName, -1, SQLITE_STATIC); if( SQLITE_ROW==sqlite3_step(pStmt) ){ | | > > | 386 387 388 389 390 391 392 393 394 395 396 397 398 399 400 401 402 | rc = sqlite3_prepare_v2(db, zSql, -1, &pStmt, 0); if( rc!=SQLITE_OK ){ return rc; } sqlite3_bind_text(pStmt, 1, zName, -1, SQLITE_STATIC); if( SQLITE_ROW==sqlite3_step(pStmt) ){ if( sqlite3_column_type(pStmt, 0)==SQLITE_BLOB && sqlite3_column_bytes(pStmt, 0)==sizeof(*pp) ){ memcpy((void *)pp, sqlite3_column_blob(pStmt, 0), sizeof(*pp)); } } return sqlite3_finalize(pStmt); } |
︙ | ︙ | |||
473 474 475 476 477 478 479 | int sqlite3Fts3InitHashTable( sqlite3 *db, Fts3Hash *pHash, const char *zName ){ int rc = SQLITE_OK; void *p = (void *)pHash; | | | 477 478 479 480 481 482 483 484 485 486 487 488 489 490 491 | int sqlite3Fts3InitHashTable( sqlite3 *db, Fts3Hash *pHash, const char *zName ){ int rc = SQLITE_OK; void *p = (void *)pHash; const int any = SQLITE_UTF8|SQLITE_DIRECTONLY; #ifdef SQLITE_TEST char *zTest = 0; char *zTest2 = 0; void *pdb = (void *)db; zTest = sqlite3_mprintf("%s_test", zName); zTest2 = sqlite3_mprintf("%s_internal_test", zName); |
︙ | ︙ |
Changes to ext/fts3/fts3_tokenizer1.c.
︙ | ︙ | |||
181 182 183 184 185 186 187 | } if( c->iOffset>iStartOffset ){ int i, n = c->iOffset-iStartOffset; if( n>c->nTokenAllocated ){ char *pNew; c->nTokenAllocated = n+20; | | | 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 | } if( c->iOffset>iStartOffset ){ int i, n = c->iOffset-iStartOffset; if( n>c->nTokenAllocated ){ char *pNew; c->nTokenAllocated = n+20; pNew = sqlite3_realloc64(c->pToken, c->nTokenAllocated); if( !pNew ) return SQLITE_NOMEM; c->pToken = pNew; } for(i=0; i<n; i++){ /* TODO(shess) This needs expansion to handle UTF-8 ** case-insensitivity. */ |
︙ | ︙ |
Changes to ext/fts3/fts3_unicode.c.
︙ | ︙ | |||
151 152 153 154 155 156 157 | } } if( nEntry ){ int *aNew; /* New aiException[] array */ int nNew; /* Number of valid entries in array aNew[] */ | | | 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 | } } if( nEntry ){ int *aNew; /* New aiException[] array */ int nNew; /* Number of valid entries in array aNew[] */ aNew = sqlite3_realloc64(p->aiException,(p->nException+nEntry)*sizeof(int)); if( aNew==0 ) return SQLITE_NOMEM; nNew = p->nException; z = (const unsigned char *)zIn; while( z<zTerm ){ READ_UTF8(z, zTerm, iCode); if( sqlite3FtsUnicodeIsalnum((int)iCode)!=bAlnum |
︙ | ︙ | |||
281 282 283 284 285 286 287 288 289 290 291 292 293 294 | return SQLITE_NOMEM; } memset(pCsr, 0, sizeof(unicode_cursor)); pCsr->aInput = (const unsigned char *)aInput; if( aInput==0 ){ pCsr->nInput = 0; }else if( nInput<0 ){ pCsr->nInput = (int)strlen(aInput); }else{ pCsr->nInput = nInput; } *pp = &pCsr->base; | > | 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 | return SQLITE_NOMEM; } memset(pCsr, 0, sizeof(unicode_cursor)); pCsr->aInput = (const unsigned char *)aInput; if( aInput==0 ){ pCsr->nInput = 0; pCsr->aInput = (const unsigned char*)""; }else if( nInput<0 ){ pCsr->nInput = (int)strlen(aInput); }else{ pCsr->nInput = nInput; } *pp = &pCsr->base; |
︙ | ︙ | |||
340 341 342 343 344 345 346 | zOut = pCsr->zToken; do { int iOut; /* Grow the output buffer if required. */ if( (zOut-pCsr->zToken)>=(pCsr->nAlloc-4) ){ | | | 341 342 343 344 345 346 347 348 349 350 351 352 353 354 355 | zOut = pCsr->zToken; do { int iOut; /* Grow the output buffer if required. */ if( (zOut-pCsr->zToken)>=(pCsr->nAlloc-4) ){ char *zNew = sqlite3_realloc64(pCsr->zToken, pCsr->nAlloc+64); if( !zNew ) return SQLITE_NOMEM; zOut = &zNew[zOut - pCsr->zToken]; pCsr->zToken = zNew; pCsr->nAlloc += 64; } /* Write the folded case of the last character read to the output */ |
︙ | ︙ |
Changes to ext/fts3/fts3_unicode2.c.
1 | /* | | | 1 2 3 4 5 6 7 8 9 | /* ** 2012-05-25 ** ** The author disclaims copyright to this source code. In place of ** a legal notice, here is a blessing: ** ** May you do good and not evil. ** May you find forgiveness for yourself and forgive others. ** May you share freely, never taking more than you give. |
︙ | ︙ | |||
174 175 176 177 178 179 180 | 61726, 61784, 61800, 61816, 61836, 61880, 61896, 61914, 61948, 61998, 62062, 62122, 62154, 62184, 62200, 62218, 62252, 62302, 62364, 62410, 62442, 62478, 62536, 62554, 62584, 62604, 62640, 62648, 62656, 62664, 62730, 62766, 62830, 62890, 62924, 62974, 63032, 63050, 63082, 63118, 63182, 63242, 63274, 63310, 63368, 63390, }; | | | | | 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 | 61726, 61784, 61800, 61816, 61836, 61880, 61896, 61914, 61948, 61998, 62062, 62122, 62154, 62184, 62200, 62218, 62252, 62302, 62364, 62410, 62442, 62478, 62536, 62554, 62584, 62604, 62640, 62648, 62656, 62664, 62730, 62766, 62830, 62890, 62924, 62974, 63032, 63050, 63082, 63118, 63182, 63242, 63274, 63310, 63368, 63390, }; #define HIBIT ((unsigned char)0x80) unsigned char aChar[] = { '\0', 'a', 'c', 'e', 'i', 'n', 'o', 'u', 'y', 'y', 'a', 'c', 'd', 'e', 'e', 'g', 'h', 'i', 'j', 'k', 'l', 'n', 'o', 'r', 's', 't', 'u', 'u', 'w', 'y', 'z', 'o', 'u', 'a', 'i', 'o', 'u', 'u'|HIBIT, 'a'|HIBIT, 'g', 'k', 'o', 'o'|HIBIT, 'j', 'g', 'n', 'a'|HIBIT, 'a', 'e', 'i', 'o', 'r', 'u', 's', 't', 'h', 'a', 'e', 'o'|HIBIT, 'o', 'o'|HIBIT, 'y', '\0', '\0', '\0', '\0', '\0', '\0', '\0', '\0', 'a', 'b', 'c'|HIBIT, 'd', 'd', 'e'|HIBIT, 'e', 'e'|HIBIT, 'f', 'g', 'h', 'h', 'i', 'i'|HIBIT, 'k', 'l', 'l'|HIBIT, 'l', 'm', 'n', 'o'|HIBIT, 'p', 'r', 'r'|HIBIT, 'r', 's', 's'|HIBIT, 't', 'u', 'u'|HIBIT, 'v', 'w', 'w', 'x', 'y', 'z', 'h', 't', 'w', 'y', 'a', 'a'|HIBIT, 'a'|HIBIT, 'a'|HIBIT, 'e', 'e'|HIBIT, 'e'|HIBIT, 'i', 'o', 'o'|HIBIT, 'o'|HIBIT, 'o'|HIBIT, 'u', 'u'|HIBIT, 'u'|HIBIT, 'y', }; unsigned int key = (((unsigned int)c)<<3) | 0x00000007; int iRes = 0; int iHi = sizeof(aDia)/sizeof(aDia[0]) - 1; int iLo = 0; while( iHi>=iLo ){ |
︙ | ︙ | |||
227 228 229 230 231 232 233 | ** is a diacritical modifier character. */ int sqlite3FtsUnicodeIsdiacritic(int c){ unsigned int mask0 = 0x08029FDF; unsigned int mask1 = 0x000361F8; if( c<768 || c>817 ) return 0; return (c < 768+32) ? | | | | 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 | ** is a diacritical modifier character. */ int sqlite3FtsUnicodeIsdiacritic(int c){ unsigned int mask0 = 0x08029FDF; unsigned int mask1 = 0x000361F8; if( c<768 || c>817 ) return 0; return (c < 768+32) ? (mask0 & ((unsigned int)1 << (c-768))) : (mask1 & ((unsigned int)1 << (c-768-32))); } /* ** Interpret the argument as a unicode codepoint. If the codepoint ** is an upper case character that has a lower case equivalent, ** return the codepoint corresponding to the lower case version. |
︙ | ︙ |
Changes to ext/fts3/fts3_write.c.
︙ | ︙ | |||
19 20 21 22 23 24 25 | #include "fts3Int.h" #if !defined(SQLITE_CORE) || defined(SQLITE_ENABLE_FTS3) #include <string.h> #include <assert.h> #include <stdlib.h> | | | 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 | #include "fts3Int.h" #if !defined(SQLITE_CORE) || defined(SQLITE_ENABLE_FTS3) #include <string.h> #include <assert.h> #include <stdlib.h> #include <stdio.h> #define FTS_MAX_APPENDABLE_HEIGHT 16 /* ** When full-text index nodes are loaded from disk, the buffer that they ** are loaded into has the following number of bytes of padding at the end ** of it. i.e. if a full-text index node is 900 bytes in size, then a buffer |
︙ | ︙ | |||
63 64 65 66 67 68 69 | # define FTS3_NODE_CHUNK_THRESHOLD test_fts3_node_chunk_threshold #else # define FTS3_NODE_CHUNKSIZE (4*1024) # define FTS3_NODE_CHUNK_THRESHOLD (FTS3_NODE_CHUNKSIZE*4) #endif /* | | | 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 | # define FTS3_NODE_CHUNK_THRESHOLD test_fts3_node_chunk_threshold #else # define FTS3_NODE_CHUNKSIZE (4*1024) # define FTS3_NODE_CHUNK_THRESHOLD (FTS3_NODE_CHUNKSIZE*4) #endif /* ** The values that may be meaningfully bound to the :1 parameter in ** statements SQL_REPLACE_STAT and SQL_SELECT_STAT. */ #define FTS_STAT_DOCTOTAL 0 #define FTS_STAT_INCRMERGEHINT 1 #define FTS_STAT_AUTOINCRMERGE 2 /* |
︙ | ︙ | |||
331 332 333 334 335 336 337 | /* This statement is used to determine which level to read the input from ** when performing an incremental merge. It returns the absolute level number ** of the oldest level in the db that contains at least ? segments. Or, ** if no level in the FTS index contains more than ? segments, the statement ** returns zero rows. */ /* 28 */ "SELECT level, count(*) AS cnt FROM %Q.'%q_segdir' " " GROUP BY level HAVING cnt>=?" | | | > > | 331 332 333 334 335 336 337 338 339 340 341 342 343 344 345 346 347 348 349 350 351 352 353 | /* This statement is used to determine which level to read the input from ** when performing an incremental merge. It returns the absolute level number ** of the oldest level in the db that contains at least ? segments. Or, ** if no level in the FTS index contains more than ? segments, the statement ** returns zero rows. */ /* 28 */ "SELECT level, count(*) AS cnt FROM %Q.'%q_segdir' " " GROUP BY level HAVING cnt>=?" " ORDER BY (level %% 1024) ASC, 2 DESC LIMIT 1", /* Estimate the upper limit on the number of leaf nodes in a new segment ** created by merging the oldest :2 segments from absolute level :1. See ** function sqlite3Fts3Incrmerge() for details. */ /* 29 */ "SELECT 2 * total(1 + leaves_end_block - start_block) " " FROM (SELECT * FROM %Q.'%q_segdir' " " WHERE level = ? ORDER BY idx ASC LIMIT ?" " )", /* SQL_DELETE_SEGDIR_ENTRY ** Delete the %_segdir entry on absolute level :1 with index :2. */ /* 30 */ "DELETE FROM %Q.'%q_segdir' WHERE level = ? AND idx = ?", /* SQL_SHIFT_SEGDIR_ENTRY ** Modify the idx value for the segment with idx=:3 on absolute level :2 |
︙ | ︙ | |||
392 393 394 395 396 397 398 399 400 401 402 403 404 405 406 407 408 409 | sqlite3_stmt *pStmt; assert( SizeofArray(azSql)==SizeofArray(p->aStmt) ); assert( eStmt<SizeofArray(azSql) && eStmt>=0 ); pStmt = p->aStmt[eStmt]; if( !pStmt ){ char *zSql; if( eStmt==SQL_CONTENT_INSERT ){ zSql = sqlite3_mprintf(azSql[eStmt], p->zDb, p->zName, p->zWriteExprlist); }else if( eStmt==SQL_SELECT_CONTENT_BY_ROWID ){ zSql = sqlite3_mprintf(azSql[eStmt], p->zReadExprlist); }else{ zSql = sqlite3_mprintf(azSql[eStmt], p->zDb, p->zName); } if( !zSql ){ rc = SQLITE_NOMEM; }else{ | > > | < | 394 395 396 397 398 399 400 401 402 403 404 405 406 407 408 409 410 411 412 413 414 415 416 417 418 419 420 421 | sqlite3_stmt *pStmt; assert( SizeofArray(azSql)==SizeofArray(p->aStmt) ); assert( eStmt<SizeofArray(azSql) && eStmt>=0 ); pStmt = p->aStmt[eStmt]; if( !pStmt ){ int f = SQLITE_PREPARE_PERSISTENT|SQLITE_PREPARE_NO_VTAB; char *zSql; if( eStmt==SQL_CONTENT_INSERT ){ zSql = sqlite3_mprintf(azSql[eStmt], p->zDb, p->zName, p->zWriteExprlist); }else if( eStmt==SQL_SELECT_CONTENT_BY_ROWID ){ f &= ~SQLITE_PREPARE_NO_VTAB; zSql = sqlite3_mprintf(azSql[eStmt], p->zReadExprlist); }else{ zSql = sqlite3_mprintf(azSql[eStmt], p->zDb, p->zName); } if( !zSql ){ rc = SQLITE_NOMEM; }else{ rc = sqlite3_prepare_v3(p->db, zSql, -1, f, &pStmt, NULL); sqlite3_free(zSql); assert( rc==SQLITE_OK || pStmt==0 ); p->aStmt[eStmt] = pStmt; } } if( apVal ){ int i; |
︙ | ︙ | |||
562 563 564 565 566 567 568 | static sqlite3_int64 getAbsoluteLevel( Fts3Table *p, /* FTS3 table handle */ int iLangid, /* Language id */ int iIndex, /* Index in p->aIndex[] */ int iLevel /* Level of segments */ ){ sqlite3_int64 iBase; /* First absolute level for iLangid/iIndex */ | | | 565 566 567 568 569 570 571 572 573 574 575 576 577 578 579 | static sqlite3_int64 getAbsoluteLevel( Fts3Table *p, /* FTS3 table handle */ int iLangid, /* Language id */ int iIndex, /* Index in p->aIndex[] */ int iLevel /* Level of segments */ ){ sqlite3_int64 iBase; /* First absolute level for iLangid/iIndex */ assert_fts3_nc( iLangid>=0 ); assert( p->nIndex>0 ); assert( iIndex>=0 && iIndex<p->nIndex ); iBase = ((sqlite3_int64)iLangid * p->nIndex + iIndex) * FTS3_SEGDIR_MAXLEVEL; return iBase + iLevel; } |
︙ | ︙ | |||
642 643 644 645 646 647 648 | PendingList **pp, /* IN/OUT: Pointer to PendingList struct */ sqlite3_int64 i /* Value to append to data */ ){ PendingList *p = *pp; /* Allocate or grow the PendingList as required. */ if( !p ){ | | | | | | 645 646 647 648 649 650 651 652 653 654 655 656 657 658 659 660 661 662 663 664 665 666 667 668 669 670 671 672 673 674 675 | PendingList **pp, /* IN/OUT: Pointer to PendingList struct */ sqlite3_int64 i /* Value to append to data */ ){ PendingList *p = *pp; /* Allocate or grow the PendingList as required. */ if( !p ){ p = sqlite3_malloc64(sizeof(*p) + 100); if( !p ){ return SQLITE_NOMEM; } p->nSpace = 100; p->aData = (char *)&p[1]; p->nData = 0; } else if( p->nData+FTS3_VARINT_MAX+1>p->nSpace ){ i64 nNew = p->nSpace * 2; p = sqlite3_realloc64(p, sizeof(*p) + nNew); if( !p ){ sqlite3_free(*pp); *pp = 0; return SQLITE_NOMEM; } p->nSpace = (int)nNew; p->aData = (char *)&p[1]; } /* Append the new serialized varint to the end of the list. */ p->nData += sqlite3Fts3PutVarint(&p->aData[p->nData], i); p->aData[p->nData] = '\0'; *pp = p; |
︙ | ︙ | |||
691 692 693 694 695 696 697 | ){ PendingList *p = *pp; int rc = SQLITE_OK; assert( !p || p->iLastDocid<=iDocid ); if( !p || p->iLastDocid!=iDocid ){ | | | 694 695 696 697 698 699 700 701 702 703 704 705 706 707 708 | ){ PendingList *p = *pp; int rc = SQLITE_OK; assert( !p || p->iLastDocid<=iDocid ); if( !p || p->iLastDocid!=iDocid ){ u64 iDelta = (u64)iDocid - (u64)(p ? p->iLastDocid : 0); if( p ){ assert( p->nData<p->nSpace ); assert( p->aData[p->nData]==0 ); p->nData++; } if( SQLITE_OK!=(rc = fts3PendingListAppendVarint(&p, iDelta)) ){ goto pendinglistappend_out; |
︙ | ︙ | |||
1148 1149 1150 1151 1152 1153 1154 | if( rc==SQLITE_OK ){ /* If iNext is FTS3_MERGE_COUNT, indicating that level iLevel is already ** full, merge all segments in level iLevel into a single iLevel+1 ** segment and allocate (newly freed) index 0 at level iLevel. Otherwise, ** if iNext is less than FTS3_MERGE_COUNT, allocate index iNext. */ | | | 1151 1152 1153 1154 1155 1156 1157 1158 1159 1160 1161 1162 1163 1164 1165 | if( rc==SQLITE_OK ){ /* If iNext is FTS3_MERGE_COUNT, indicating that level iLevel is already ** full, merge all segments in level iLevel into a single iLevel+1 ** segment and allocate (newly freed) index 0 at level iLevel. Otherwise, ** if iNext is less than FTS3_MERGE_COUNT, allocate index iNext. */ if( iNext>=MergeCount(p) ){ fts3LogMerge(16, getAbsoluteLevel(p, iLangid, iIndex, iLevel)); rc = fts3SegmentMerge(p, iLangid, iIndex, iLevel); *piIdx = 0; }else{ *piIdx = iNext; } } |
︙ | ︙ | |||
1215 1216 1217 1218 1219 1220 1221 | ); } if( rc==SQLITE_OK ){ int nByte = sqlite3_blob_bytes(p->pSegments); *pnBlob = nByte; if( paBlob ){ | | > > | 1218 1219 1220 1221 1222 1223 1224 1225 1226 1227 1228 1229 1230 1231 1232 1233 1234 1235 1236 1237 1238 1239 1240 1241 1242 1243 1244 1245 1246 1247 1248 1249 1250 | ); } if( rc==SQLITE_OK ){ int nByte = sqlite3_blob_bytes(p->pSegments); *pnBlob = nByte; if( paBlob ){ char *aByte = sqlite3_malloc64((i64)nByte + FTS3_NODE_PADDING); if( !aByte ){ rc = SQLITE_NOMEM; }else{ if( pnLoad && nByte>(FTS3_NODE_CHUNK_THRESHOLD) ){ nByte = FTS3_NODE_CHUNKSIZE; *pnLoad = nByte; } rc = sqlite3_blob_read(p->pSegments, aByte, nByte, 0); memset(&aByte[nByte], 0, FTS3_NODE_PADDING); if( rc!=SQLITE_OK ){ sqlite3_free(aByte); aByte = 0; } } *paBlob = aByte; } }else if( rc==SQLITE_ERROR ){ rc = FTS_CORRUPT_VTAB; } return rc; } /* ** Close the blob handle at p->pSegments, if it is open. See comments above |
︙ | ︙ | |||
1326 1327 1328 1329 1330 1331 1332 | Fts3HashElem *pElem = *(pReader->ppNextElem); sqlite3_free(pReader->aNode); pReader->aNode = 0; if( pElem ){ char *aCopy; PendingList *pList = (PendingList *)fts3HashData(pElem); int nCopy = pList->nData+1; | > > > > | > > > > > | > | > | > | 1331 1332 1333 1334 1335 1336 1337 1338 1339 1340 1341 1342 1343 1344 1345 1346 1347 1348 1349 1350 1351 1352 1353 1354 1355 1356 1357 1358 1359 1360 1361 1362 1363 1364 1365 1366 1367 1368 1369 1370 1371 1372 1373 1374 | Fts3HashElem *pElem = *(pReader->ppNextElem); sqlite3_free(pReader->aNode); pReader->aNode = 0; if( pElem ){ char *aCopy; PendingList *pList = (PendingList *)fts3HashData(pElem); int nCopy = pList->nData+1; int nTerm = fts3HashKeysize(pElem); if( (nTerm+1)>pReader->nTermAlloc ){ sqlite3_free(pReader->zTerm); pReader->zTerm = (char*)sqlite3_malloc64(((i64)nTerm+1)*2); if( !pReader->zTerm ) return SQLITE_NOMEM; pReader->nTermAlloc = (nTerm+1)*2; } memcpy(pReader->zTerm, fts3HashKey(pElem), nTerm); pReader->zTerm[nTerm] = '\0'; pReader->nTerm = nTerm; aCopy = (char*)sqlite3_malloc64(nCopy); if( !aCopy ) return SQLITE_NOMEM; memcpy(aCopy, pList->aData, nCopy); pReader->nNode = pReader->nDoclist = nCopy; pReader->aNode = pReader->aDoclist = aCopy; pReader->ppNextElem++; assert( pReader->aNode ); } return SQLITE_OK; } fts3SegReaderSetEof(pReader); /* If iCurrentBlock>=iLeafEndBlock, this is an EOF condition. All leaf ** blocks have already been traversed. */ #ifdef CORRUPT_DB assert( pReader->iCurrentBlock<=pReader->iLeafEndBlock || CORRUPT_DB ); #endif if( pReader->iCurrentBlock>=pReader->iLeafEndBlock ){ return SQLITE_OK; } rc = sqlite3Fts3ReadBlock( p, ++pReader->iCurrentBlock, &pReader->aNode, &pReader->nNode, (bIncr ? &pReader->nPopulate : 0) |
︙ | ︙ | |||
1372 1373 1374 1375 1376 1377 1378 | /* Because of the FTS3_NODE_PADDING bytes of padding, the following is ** safe (no risk of overread) even if the node data is corrupted. */ pNext += fts3GetVarint32(pNext, &nPrefix); pNext += fts3GetVarint32(pNext, &nSuffix); if( nSuffix<=0 || (&pReader->aNode[pReader->nNode] - pNext)<nSuffix | | | 1389 1390 1391 1392 1393 1394 1395 1396 1397 1398 1399 1400 1401 1402 1403 | /* Because of the FTS3_NODE_PADDING bytes of padding, the following is ** safe (no risk of overread) even if the node data is corrupted. */ pNext += fts3GetVarint32(pNext, &nPrefix); pNext += fts3GetVarint32(pNext, &nSuffix); if( nSuffix<=0 || (&pReader->aNode[pReader->nNode] - pNext)<nSuffix || nPrefix>pReader->nTerm ){ return FTS_CORRUPT_VTAB; } /* Both nPrefix and nSuffix were read by fts3GetVarint32() and so are ** between 0 and 0x7FFFFFFF. But the sum of the two may cause integer ** overflow - hence the (i64) casts. */ |
︙ | ︙ | |||
1404 1405 1406 1407 1408 1409 1410 | pReader->aDoclist = pNext; pReader->pOffsetList = 0; /* Check that the doclist does not appear to extend past the end of the ** b-tree node. And that the final byte of the doclist is 0x00. If either ** of these statements is untrue, then the data structure is corrupt. */ | | > | 1421 1422 1423 1424 1425 1426 1427 1428 1429 1430 1431 1432 1433 1434 1435 1436 1437 | pReader->aDoclist = pNext; pReader->pOffsetList = 0; /* Check that the doclist does not appear to extend past the end of the ** b-tree node. And that the final byte of the doclist is 0x00. If either ** of these statements is untrue, then the data structure is corrupt. */ if( pReader->nDoclist > pReader->nNode-(pReader->aDoclist-pReader->aNode) || (pReader->nPopulate==0 && pReader->aDoclist[pReader->nDoclist-1]) || pReader->nDoclist==0 ){ return FTS_CORRUPT_VTAB; } return SQLITE_OK; } /* |
︙ | ︙ | |||
1522 1523 1524 1525 1526 1527 1528 | ** returning. */ if( p>=pEnd ){ pReader->pOffsetList = 0; }else{ rc = fts3SegReaderRequire(pReader, p, FTS3_VARINT_MAX); if( rc==SQLITE_OK ){ | | | | | | | 1540 1541 1542 1543 1544 1545 1546 1547 1548 1549 1550 1551 1552 1553 1554 1555 1556 1557 1558 1559 1560 1561 1562 1563 1564 1565 | ** returning. */ if( p>=pEnd ){ pReader->pOffsetList = 0; }else{ rc = fts3SegReaderRequire(pReader, p, FTS3_VARINT_MAX); if( rc==SQLITE_OK ){ u64 iDelta; pReader->pOffsetList = p + sqlite3Fts3GetVarintU(p, &iDelta); if( pTab->bDescIdx ){ pReader->iDocid = (i64)((u64)pReader->iDocid - iDelta); }else{ pReader->iDocid = (i64)((u64)pReader->iDocid + iDelta); } } } } return rc; } int sqlite3Fts3MsrOvfl( Fts3Cursor *pCsr, Fts3MultiSegReader *pMsr, int *pnOvfl |
︙ | ︙ | |||
1577 1578 1579 1580 1581 1582 1583 | /* ** Free all allocations associated with the iterator passed as the ** second argument. */ void sqlite3Fts3SegReaderFree(Fts3SegReader *pReader){ if( pReader ){ | < | < | 1595 1596 1597 1598 1599 1600 1601 1602 1603 1604 1605 1606 1607 1608 1609 | /* ** Free all allocations associated with the iterator passed as the ** second argument. */ void sqlite3Fts3SegReaderFree(Fts3SegReader *pReader){ if( pReader ){ sqlite3_free(pReader->zTerm); if( !fts3SegReaderIsRootOnly(pReader) ){ sqlite3_free(pReader->aNode); } sqlite3_blob_close(pReader->pBlob); } sqlite3_free(pReader); } |
︙ | ︙ | |||
1604 1605 1606 1607 1608 1609 1610 | const char *zRoot, /* Buffer containing root node */ int nRoot, /* Size of buffer containing root node */ Fts3SegReader **ppReader /* OUT: Allocated Fts3SegReader */ ){ Fts3SegReader *pReader; /* Newly allocated SegReader object */ int nExtra = 0; /* Bytes to allocate segment root node */ | > > > > | > | | | 1620 1621 1622 1623 1624 1625 1626 1627 1628 1629 1630 1631 1632 1633 1634 1635 1636 1637 1638 1639 1640 1641 1642 1643 1644 1645 1646 1647 1648 1649 1650 1651 1652 1653 1654 1655 1656 1657 1658 1659 1660 | const char *zRoot, /* Buffer containing root node */ int nRoot, /* Size of buffer containing root node */ Fts3SegReader **ppReader /* OUT: Allocated Fts3SegReader */ ){ Fts3SegReader *pReader; /* Newly allocated SegReader object */ int nExtra = 0; /* Bytes to allocate segment root node */ assert( zRoot!=0 || nRoot==0 ); #ifdef CORRUPT_DB assert( zRoot!=0 || CORRUPT_DB ); #endif if( iStartLeaf==0 ){ if( iEndLeaf!=0 ) return FTS_CORRUPT_VTAB; nExtra = nRoot + FTS3_NODE_PADDING; } pReader = (Fts3SegReader *)sqlite3_malloc64(sizeof(Fts3SegReader) + nExtra); if( !pReader ){ return SQLITE_NOMEM; } memset(pReader, 0, sizeof(Fts3SegReader)); pReader->iIdx = iAge; pReader->bLookup = bLookup!=0; pReader->iStartBlock = iStartLeaf; pReader->iLeafEndBlock = iEndLeaf; pReader->iEndBlock = iEndBlock; if( nExtra ){ /* The entire segment is stored in the root node. */ pReader->aNode = (char *)&pReader[1]; pReader->rootOnly = 1; pReader->nNode = nRoot; if( nRoot ) memcpy(pReader->aNode, zRoot, nRoot); memset(&pReader->aNode[nRoot], 0, FTS3_NODE_PADDING); }else{ pReader->iCurrentBlock = iStartLeaf-1; } *ppReader = pReader; return SQLITE_OK; } |
︙ | ︙ | |||
1701 1702 1703 1704 1705 1706 1707 | for(pE=fts3HashFirst(pHash); pE; pE=fts3HashNext(pE)){ char *zKey = (char *)fts3HashKey(pE); int nKey = fts3HashKeysize(pE); if( nTerm==0 || (nKey>=nTerm && 0==memcmp(zKey, zTerm, nTerm)) ){ if( nElem==nAlloc ){ Fts3HashElem **aElem2; nAlloc += 16; | | | 1722 1723 1724 1725 1726 1727 1728 1729 1730 1731 1732 1733 1734 1735 1736 | for(pE=fts3HashFirst(pHash); pE; pE=fts3HashNext(pE)){ char *zKey = (char *)fts3HashKey(pE); int nKey = fts3HashKeysize(pE); if( nTerm==0 || (nKey>=nTerm && 0==memcmp(zKey, zTerm, nTerm)) ){ if( nElem==nAlloc ){ Fts3HashElem **aElem2; nAlloc += 16; aElem2 = (Fts3HashElem **)sqlite3_realloc64( aElem, nAlloc*sizeof(Fts3HashElem *) ); if( !aElem2 ){ rc = SQLITE_NOMEM; nElem = 0; break; } |
︙ | ︙ | |||
1740 1741 1742 1743 1744 1745 1746 | if( pE ){ aElem = &pE; nElem = 1; } } if( nElem>0 ){ | > | | | 1761 1762 1763 1764 1765 1766 1767 1768 1769 1770 1771 1772 1773 1774 1775 1776 1777 | if( pE ){ aElem = &pE; nElem = 1; } } if( nElem>0 ){ sqlite3_int64 nByte; nByte = sizeof(Fts3SegReader) + (nElem+1)*sizeof(Fts3HashElem *); pReader = (Fts3SegReader *)sqlite3_malloc64(nByte); if( !pReader ){ rc = SQLITE_NOMEM; }else{ memset(pReader, 0, nByte); pReader->iIdx = 0x7FFFFFFF; pReader->ppNextElem = (Fts3HashElem **)&pReader[1]; memcpy(pReader->ppNextElem, aElem, nElem*sizeof(Fts3HashElem *)); |
︙ | ︙ | |||
1789 1790 1791 1792 1793 1794 1795 | } }else{ rc = (pLhs->aNode==0) - (pRhs->aNode==0); } if( rc==0 ){ rc = pRhs->iIdx - pLhs->iIdx; } | | | 1811 1812 1813 1814 1815 1816 1817 1818 1819 1820 1821 1822 1823 1824 1825 | } }else{ rc = (pLhs->aNode==0) - (pRhs->aNode==0); } if( rc==0 ){ rc = pRhs->iIdx - pLhs->iIdx; } assert_fts3_nc( rc!=0 ); return rc; } /* ** A different comparison function for SegReader structures. In this ** version, it is assumed that each SegReader points to an entry in ** a doclist for identical terms. Comparison is made as follows: |
︙ | ︙ | |||
1985 1986 1987 1988 1989 1990 1991 | static int fts3PrefixCompress( const char *zPrev, /* Buffer containing previous term */ int nPrev, /* Size of buffer zPrev in bytes */ const char *zNext, /* Buffer containing next term */ int nNext /* Size of buffer zNext in bytes */ ){ int n; | < | > | 2007 2008 2009 2010 2011 2012 2013 2014 2015 2016 2017 2018 2019 2020 2021 2022 | static int fts3PrefixCompress( const char *zPrev, /* Buffer containing previous term */ int nPrev, /* Size of buffer zPrev in bytes */ const char *zNext, /* Buffer containing next term */ int nNext /* Size of buffer zNext in bytes */ ){ int n; for(n=0; n<nPrev && n<nNext && zPrev[n]==zNext[n]; n++); assert_fts3_nc( n<nNext ); return n; } /* ** Add term zTerm to the SegmentNode. It is guaranteed that zTerm is larger ** (according to memcmp) than the previous term. */ |
︙ | ︙ | |||
2016 2017 2018 2019 2020 2021 2022 2023 2024 2025 2026 2027 2028 2029 2030 2031 2032 2033 2034 2035 | int nData = pTree->nData; /* Current size of node in bytes */ int nReq = nData; /* Required space after adding zTerm */ int nPrefix; /* Number of bytes of prefix compression */ int nSuffix; /* Suffix length */ nPrefix = fts3PrefixCompress(pTree->zTerm, pTree->nTerm, zTerm, nTerm); nSuffix = nTerm-nPrefix; nReq += sqlite3Fts3VarintLen(nPrefix)+sqlite3Fts3VarintLen(nSuffix)+nSuffix; if( nReq<=p->nNodeSize || !pTree->zTerm ){ if( nReq>p->nNodeSize ){ /* An unusual case: this is the first term to be added to the node ** and the static node buffer (p->nNodeSize bytes) is not large ** enough. Use a separately malloced buffer instead This wastes ** p->nNodeSize bytes, but since this scenario only comes about when ** the database contain two terms that share a prefix of almost 2KB, ** this is not expected to be a serious problem. */ assert( pTree->aData==(char *)&pTree[1] ); | > > > > > | | | 2038 2039 2040 2041 2042 2043 2044 2045 2046 2047 2048 2049 2050 2051 2052 2053 2054 2055 2056 2057 2058 2059 2060 2061 2062 2063 2064 2065 2066 2067 2068 2069 2070 2071 2072 2073 2074 2075 2076 2077 2078 2079 2080 2081 2082 2083 2084 2085 2086 2087 2088 | int nData = pTree->nData; /* Current size of node in bytes */ int nReq = nData; /* Required space after adding zTerm */ int nPrefix; /* Number of bytes of prefix compression */ int nSuffix; /* Suffix length */ nPrefix = fts3PrefixCompress(pTree->zTerm, pTree->nTerm, zTerm, nTerm); nSuffix = nTerm-nPrefix; /* If nSuffix is zero or less, then zTerm/nTerm must be a prefix of ** pWriter->zTerm/pWriter->nTerm. i.e. must be equal to or less than when ** compared with BINARY collation. This indicates corruption. */ if( nSuffix<=0 ) return FTS_CORRUPT_VTAB; nReq += sqlite3Fts3VarintLen(nPrefix)+sqlite3Fts3VarintLen(nSuffix)+nSuffix; if( nReq<=p->nNodeSize || !pTree->zTerm ){ if( nReq>p->nNodeSize ){ /* An unusual case: this is the first term to be added to the node ** and the static node buffer (p->nNodeSize bytes) is not large ** enough. Use a separately malloced buffer instead This wastes ** p->nNodeSize bytes, but since this scenario only comes about when ** the database contain two terms that share a prefix of almost 2KB, ** this is not expected to be a serious problem. */ assert( pTree->aData==(char *)&pTree[1] ); pTree->aData = (char *)sqlite3_malloc64(nReq); if( !pTree->aData ){ return SQLITE_NOMEM; } } if( pTree->zTerm ){ /* There is no prefix-length field for first term in a node */ nData += sqlite3Fts3PutVarint(&pTree->aData[nData], nPrefix); } nData += sqlite3Fts3PutVarint(&pTree->aData[nData], nSuffix); memcpy(&pTree->aData[nData], &zTerm[nPrefix], nSuffix); pTree->nData = nData + nSuffix; pTree->nEntry++; if( isCopyTerm ){ if( pTree->nMalloc<nTerm ){ char *zNew = sqlite3_realloc64(pTree->zMalloc, (i64)nTerm*2); if( !zNew ){ return SQLITE_NOMEM; } pTree->nMalloc = nTerm*2; pTree->zMalloc = zNew; } pTree->zTerm = pTree->zMalloc; |
︙ | ︙ | |||
2073 2074 2075 2076 2077 2078 2079 | ** current node. Create a new node (a right-sibling of the current node). ** If this is the first node in the tree, the term is added to it. ** ** Otherwise, the term is not added to the new node, it is left empty for ** now. Instead, the term is inserted into the parent of pTree. If pTree ** has no parent, one is created here. */ | | | 2100 2101 2102 2103 2104 2105 2106 2107 2108 2109 2110 2111 2112 2113 2114 | ** current node. Create a new node (a right-sibling of the current node). ** If this is the first node in the tree, the term is added to it. ** ** Otherwise, the term is not added to the new node, it is left empty for ** now. Instead, the term is inserted into the parent of pTree. If pTree ** has no parent, one is created here. */ pNew = (SegmentNode *)sqlite3_malloc64(sizeof(SegmentNode) + p->nNodeSize); if( !pNew ){ return SQLITE_NOMEM; } memset(pNew, 0, sizeof(SegmentNode)); pNew->nData = 1 + FTS3_VARINT_MAX; pNew->aData = (char *)&pNew[1]; |
︙ | ︙ | |||
2211 2212 2213 2214 2215 2216 2217 | const char *zTerm, /* Pointer to buffer containing term */ int nTerm, /* Size of term in bytes */ const char *aDoclist, /* Pointer to buffer containing doclist */ int nDoclist /* Size of doclist in bytes */ ){ int nPrefix; /* Size of term prefix in bytes */ int nSuffix; /* Size of term suffix in bytes */ | | | | > > > > > > | 2238 2239 2240 2241 2242 2243 2244 2245 2246 2247 2248 2249 2250 2251 2252 2253 2254 2255 2256 2257 2258 2259 2260 2261 2262 2263 2264 2265 2266 2267 2268 2269 2270 2271 2272 2273 2274 2275 2276 2277 2278 2279 2280 2281 2282 2283 2284 2285 2286 2287 2288 2289 2290 2291 2292 2293 2294 2295 2296 2297 2298 2299 2300 2301 2302 | const char *zTerm, /* Pointer to buffer containing term */ int nTerm, /* Size of term in bytes */ const char *aDoclist, /* Pointer to buffer containing doclist */ int nDoclist /* Size of doclist in bytes */ ){ int nPrefix; /* Size of term prefix in bytes */ int nSuffix; /* Size of term suffix in bytes */ i64 nReq; /* Number of bytes required on leaf page */ int nData; SegmentWriter *pWriter = *ppWriter; if( !pWriter ){ int rc; sqlite3_stmt *pStmt; /* Allocate the SegmentWriter structure */ pWriter = (SegmentWriter *)sqlite3_malloc64(sizeof(SegmentWriter)); if( !pWriter ) return SQLITE_NOMEM; memset(pWriter, 0, sizeof(SegmentWriter)); *ppWriter = pWriter; /* Allocate a buffer in which to accumulate data */ pWriter->aData = (char *)sqlite3_malloc64(p->nNodeSize); if( !pWriter->aData ) return SQLITE_NOMEM; pWriter->nSize = p->nNodeSize; /* Find the next free blockid in the %_segments table */ rc = fts3SqlStmt(p, SQL_NEXT_SEGMENTS_ID, &pStmt, 0); if( rc!=SQLITE_OK ) return rc; if( SQLITE_ROW==sqlite3_step(pStmt) ){ pWriter->iFree = sqlite3_column_int64(pStmt, 0); pWriter->iFirst = pWriter->iFree; } rc = sqlite3_reset(pStmt); if( rc!=SQLITE_OK ) return rc; } nData = pWriter->nData; nPrefix = fts3PrefixCompress(pWriter->zTerm, pWriter->nTerm, zTerm, nTerm); nSuffix = nTerm-nPrefix; /* If nSuffix is zero or less, then zTerm/nTerm must be a prefix of ** pWriter->zTerm/pWriter->nTerm. i.e. must be equal to or less than when ** compared with BINARY collation. This indicates corruption. */ if( nSuffix<=0 ) return FTS_CORRUPT_VTAB; /* Figure out how many bytes are required by this new entry */ nReq = sqlite3Fts3VarintLen(nPrefix) + /* varint containing prefix size */ sqlite3Fts3VarintLen(nSuffix) + /* varint containing suffix size */ nSuffix + /* Term suffix */ sqlite3Fts3VarintLen(nDoclist) + /* Size of doclist */ nDoclist; /* Doclist data */ if( nData>0 && nData+nReq>p->nNodeSize ){ int rc; /* The current leaf node is full. Write it out to the database. */ if( pWriter->iFree==LARGEST_INT64 ) return FTS_CORRUPT_VTAB; rc = fts3WriteSegment(p, pWriter->iFree++, pWriter->aData, nData); if( rc!=SQLITE_OK ) return rc; p->nLeafAdd++; /* Add the current term to the interior node tree. The term added to ** the interior tree must: ** |
︙ | ︙ | |||
2295 2296 2297 2298 2299 2300 2301 | /* Increase the total number of bytes written to account for the new entry. */ pWriter->nLeafData += nReq; /* If the buffer currently allocated is too small for this entry, realloc ** the buffer to make it large enough. */ if( nReq>pWriter->nSize ){ | | > > | > | 2328 2329 2330 2331 2332 2333 2334 2335 2336 2337 2338 2339 2340 2341 2342 2343 2344 2345 2346 2347 2348 2349 2350 2351 2352 2353 2354 2355 2356 2357 2358 2359 2360 2361 2362 2363 2364 2365 2366 2367 2368 2369 2370 2371 2372 2373 2374 2375 2376 | /* Increase the total number of bytes written to account for the new entry. */ pWriter->nLeafData += nReq; /* If the buffer currently allocated is too small for this entry, realloc ** the buffer to make it large enough. */ if( nReq>pWriter->nSize ){ char *aNew = sqlite3_realloc64(pWriter->aData, nReq); if( !aNew ) return SQLITE_NOMEM; pWriter->aData = aNew; pWriter->nSize = nReq; } assert( nData+nReq<=pWriter->nSize ); /* Append the prefix-compressed term and doclist to the buffer. */ nData += sqlite3Fts3PutVarint(&pWriter->aData[nData], nPrefix); nData += sqlite3Fts3PutVarint(&pWriter->aData[nData], nSuffix); assert( nSuffix>0 ); memcpy(&pWriter->aData[nData], &zTerm[nPrefix], nSuffix); nData += nSuffix; nData += sqlite3Fts3PutVarint(&pWriter->aData[nData], nDoclist); assert( nDoclist>0 ); memcpy(&pWriter->aData[nData], aDoclist, nDoclist); pWriter->nData = nData + nDoclist; /* Save the current term so that it can be used to prefix-compress the next. ** If the isCopyTerm parameter is true, then the buffer pointed to by ** zTerm is transient, so take a copy of the term data. Otherwise, just ** store a copy of the pointer. */ if( isCopyTerm ){ if( nTerm>pWriter->nMalloc ){ char *zNew = sqlite3_realloc64(pWriter->zMalloc, (i64)nTerm*2); if( !zNew ){ return SQLITE_NOMEM; } pWriter->nMalloc = nTerm*2; pWriter->zMalloc = zNew; pWriter->zTerm = zNew; } assert( pWriter->zTerm==pWriter->zMalloc ); assert( nTerm>0 ); memcpy(pWriter->zTerm, zTerm, nTerm); }else{ pWriter->zTerm = (char *)zTerm; } pWriter->nTerm = nTerm; return SQLITE_OK; |
︙ | ︙ | |||
2473 2474 2475 2476 2477 2478 2479 | ** (1024 is actually the value of macro FTS3_SEGDIR_PREFIXLEVEL_STR). */ sqlite3_stmt *pStmt; int rc = fts3SqlStmt(p, SQL_SELECT_SEGDIR_MAX_LEVEL, &pStmt, 0); if( rc!=SQLITE_OK ) return rc; sqlite3_bind_int64(pStmt, 1, iAbsLevel+1); sqlite3_bind_int64(pStmt, 2, | | | 2509 2510 2511 2512 2513 2514 2515 2516 2517 2518 2519 2520 2521 2522 2523 | ** (1024 is actually the value of macro FTS3_SEGDIR_PREFIXLEVEL_STR). */ sqlite3_stmt *pStmt; int rc = fts3SqlStmt(p, SQL_SELECT_SEGDIR_MAX_LEVEL, &pStmt, 0); if( rc!=SQLITE_OK ) return rc; sqlite3_bind_int64(pStmt, 1, iAbsLevel+1); sqlite3_bind_int64(pStmt, 2, (((u64)iAbsLevel/FTS3_SEGDIR_MAXLEVEL)+1) * FTS3_SEGDIR_MAXLEVEL ); *pbMax = 0; if( SQLITE_ROW==sqlite3_step(pStmt) ){ *pbMax = sqlite3_column_type(pStmt, 0)==SQLITE_NULL; } return sqlite3_reset(pStmt); |
︙ | ︙ | |||
2601 2602 2603 2604 2605 2606 2607 | if( iCol==iCurrent ){ nList = (int)(p - pList); break; } nList -= (int)(p - pList); pList = p; | | | | | > | 2637 2638 2639 2640 2641 2642 2643 2644 2645 2646 2647 2648 2649 2650 2651 2652 2653 2654 2655 2656 2657 2658 2659 2660 2661 2662 2663 2664 2665 2666 2667 2668 2669 2670 2671 2672 2673 2674 2675 2676 2677 2678 2679 2680 2681 2682 2683 2684 2685 | if( iCol==iCurrent ){ nList = (int)(p - pList); break; } nList -= (int)(p - pList); pList = p; if( nList<=0 ){ break; } p = &pList[1]; p += fts3GetVarint32(p, &iCurrent); } if( bZero && (pEnd - &pList[nList])>0){ memset(&pList[nList], 0, pEnd - &pList[nList]); } *ppList = pList; *pnList = nList; } /* ** Cache data in the Fts3MultiSegReader.aBuffer[] buffer (overwriting any ** existing data). Grow the buffer if required. ** ** If successful, return SQLITE_OK. Otherwise, if an OOM error is encountered ** trying to resize the buffer, return SQLITE_NOMEM. */ static int fts3MsrBufferData( Fts3MultiSegReader *pMsr, /* Multi-segment-reader handle */ char *pList, i64 nList ){ if( nList>pMsr->nBuffer ){ char *pNew; pMsr->nBuffer = nList*2; pNew = (char *)sqlite3_realloc64(pMsr->aBuffer, pMsr->nBuffer); if( !pNew ) return SQLITE_NOMEM; pMsr->aBuffer = pNew; } assert( nList>0 ); memcpy(pMsr->aBuffer, pList, nList); return SQLITE_OK; } int sqlite3Fts3MsrIncrNext( Fts3Table *p, /* Virtual table handle */ Fts3MultiSegReader *pMsr, /* Multi-segment-reader handle */ |
︙ | ︙ | |||
2685 2686 2687 2688 2689 2690 2691 | rc = fts3SegReaderNextDocid(p, apSegment[j], 0, 0); j++; } if( rc!=SQLITE_OK ) return rc; fts3SegReaderSort(pMsr->apSegment, nMerge, j, xCmp); if( nList>0 && fts3SegReaderIsPending(apSegment[0]) ){ | | | 2722 2723 2724 2725 2726 2727 2728 2729 2730 2731 2732 2733 2734 2735 2736 | rc = fts3SegReaderNextDocid(p, apSegment[j], 0, 0); j++; } if( rc!=SQLITE_OK ) return rc; fts3SegReaderSort(pMsr->apSegment, nMerge, j, xCmp); if( nList>0 && fts3SegReaderIsPending(apSegment[0]) ){ rc = fts3MsrBufferData(pMsr, pList, (i64)nList+1); if( rc!=SQLITE_OK ) return rc; assert( (pMsr->aBuffer[nList] & 0xFE)==0x00 ); pList = pMsr->aBuffer; } if( pMsr->iColFilter>=0 ){ fts3ColumnFilter(pMsr->iColFilter, 1, &pList, &nList); |
︙ | ︙ | |||
2822 2823 2824 2825 2826 2827 2828 2829 2830 2831 2832 2833 2834 2835 | pCsr->apSegment[i]->nOffsetList = 0; pCsr->apSegment[i]->iDocid = 0; } return SQLITE_OK; } int sqlite3Fts3SegReaderStep( Fts3Table *p, /* Virtual table handle */ Fts3MultiSegReader *pCsr /* Cursor object */ ){ int rc = SQLITE_OK; | > > > > > > > > > > > > > | 2859 2860 2861 2862 2863 2864 2865 2866 2867 2868 2869 2870 2871 2872 2873 2874 2875 2876 2877 2878 2879 2880 2881 2882 2883 2884 2885 | pCsr->apSegment[i]->nOffsetList = 0; pCsr->apSegment[i]->iDocid = 0; } return SQLITE_OK; } static int fts3GrowSegReaderBuffer(Fts3MultiSegReader *pCsr, i64 nReq){ if( nReq>pCsr->nBuffer ){ char *aNew; pCsr->nBuffer = nReq*2; aNew = sqlite3_realloc64(pCsr->aBuffer, pCsr->nBuffer); if( !aNew ){ return SQLITE_NOMEM; } pCsr->aBuffer = aNew; } return SQLITE_OK; } int sqlite3Fts3SegReaderStep( Fts3Table *p, /* Virtual table handle */ Fts3MultiSegReader *pCsr /* Cursor object */ ){ int rc = SQLITE_OK; |
︙ | ︙ | |||
2904 2905 2906 2907 2908 2909 2910 | if( nMerge==1 && !isIgnoreEmpty && !isFirst && (p->bDescIdx==0 || fts3SegReaderIsPending(apSegment[0])==0) ){ pCsr->nDoclist = apSegment[0]->nDoclist; if( fts3SegReaderIsPending(apSegment[0]) ){ | | > | 2954 2955 2956 2957 2958 2959 2960 2961 2962 2963 2964 2965 2966 2967 2968 2969 | if( nMerge==1 && !isIgnoreEmpty && !isFirst && (p->bDescIdx==0 || fts3SegReaderIsPending(apSegment[0])==0) ){ pCsr->nDoclist = apSegment[0]->nDoclist; if( fts3SegReaderIsPending(apSegment[0]) ){ rc = fts3MsrBufferData(pCsr, apSegment[0]->aDoclist, (i64)pCsr->nDoclist); pCsr->aDoclist = pCsr->aBuffer; }else{ pCsr->aDoclist = apSegment[0]->aDoclist; } if( rc==SQLITE_OK ) rc = SQLITE_ROW; }else{ int nDoclist = 0; /* Size of doclist */ |
︙ | ︙ | |||
2948 2949 2950 2951 2952 2953 2954 | if( !isIgnoreEmpty || nList>0 ){ /* Calculate the 'docid' delta value to write into the merged ** doclist. */ sqlite3_int64 iDelta; if( p->bDescIdx && nDoclist>0 ){ | > | > | < < < < < < < < | | < > > | 2999 3000 3001 3002 3003 3004 3005 3006 3007 3008 3009 3010 3011 3012 3013 3014 3015 3016 3017 3018 3019 3020 3021 3022 3023 3024 | if( !isIgnoreEmpty || nList>0 ){ /* Calculate the 'docid' delta value to write into the merged ** doclist. */ sqlite3_int64 iDelta; if( p->bDescIdx && nDoclist>0 ){ if( iPrev<=iDocid ) return FTS_CORRUPT_VTAB; iDelta = (i64)((u64)iPrev - (u64)iDocid); }else{ if( nDoclist>0 && iPrev>=iDocid ) return FTS_CORRUPT_VTAB; iDelta = (i64)((u64)iDocid - (u64)iPrev); } nByte = sqlite3Fts3VarintLen(iDelta) + (isRequirePos?nList+1:0); rc = fts3GrowSegReaderBuffer(pCsr, (i64)nByte+nDoclist+FTS3_NODE_PADDING); if( rc ) return rc; if( isFirst ){ char *a = &pCsr->aBuffer[nDoclist]; int nWrite; nWrite = sqlite3Fts3FirstFilter(iDelta, pList, nList, a); if( nWrite ){ |
︙ | ︙ | |||
2989 2990 2991 2992 2993 2994 2995 2996 2997 2998 2999 3000 3001 3002 | } } } fts3SegReaderSort(apSegment, nMerge, j, xCmp); } if( nDoclist>0 ){ pCsr->aDoclist = pCsr->aBuffer; pCsr->nDoclist = nDoclist; rc = SQLITE_ROW; } } pCsr->nAdvance = nMerge; }while( rc==SQLITE_OK ); | > > > | 3035 3036 3037 3038 3039 3040 3041 3042 3043 3044 3045 3046 3047 3048 3049 3050 3051 | } } } fts3SegReaderSort(apSegment, nMerge, j, xCmp); } if( nDoclist>0 ){ rc = fts3GrowSegReaderBuffer(pCsr, (i64)nDoclist+FTS3_NODE_PADDING); if( rc ) return rc; memset(&pCsr->aBuffer[nDoclist], 0, FTS3_NODE_PADDING); pCsr->aDoclist = pCsr->aBuffer; pCsr->nDoclist = nDoclist; rc = SQLITE_ROW; } } pCsr->nAdvance = nMerge; }while( rc==SQLITE_OK ); |
︙ | ︙ | |||
3038 3039 3040 3041 3042 3043 3044 | i64 *piEndBlock, i64 *pnByte ){ const unsigned char *zText = sqlite3_column_text(pStmt, iCol); if( zText ){ int i; int iMul = 1; | | | | | 3087 3088 3089 3090 3091 3092 3093 3094 3095 3096 3097 3098 3099 3100 3101 3102 3103 3104 3105 3106 3107 3108 3109 3110 3111 3112 3113 3114 3115 | i64 *piEndBlock, i64 *pnByte ){ const unsigned char *zText = sqlite3_column_text(pStmt, iCol); if( zText ){ int i; int iMul = 1; u64 iVal = 0; for(i=0; zText[i]>='0' && zText[i]<='9'; i++){ iVal = iVal*10 + (zText[i] - '0'); } *piEndBlock = (i64)iVal; while( zText[i]==' ' ) i++; iVal = 0; if( zText[i]=='-' ){ i++; iMul = -1; } for(/* no-op */; zText[i]>='0' && zText[i]<='9'; i++){ iVal = iVal*10 + (zText[i] - '0'); } *pnByte = ((i64)iVal * (i64)iMul); } } /* ** A segment of size nByte bytes has just been written to absolute level ** iAbsLevel. Promote any segments that should be promoted as a result. |
︙ | ︙ | |||
3218 3219 3220 3221 3222 3223 3224 | iNewLevel = getAbsoluteLevel(p, iLangid, iIndex, iLevel+1); rc = fts3AllocateSegdirIdx(p, iLangid, iIndex, iLevel+1, &iIdx); bIgnoreEmpty = (iLevel!=FTS3_SEGCURSOR_PENDING) && (iNewLevel>iMaxLevel); } if( rc!=SQLITE_OK ) goto finished; assert( csr.nSegment>0 ); | | > | > | | 3267 3268 3269 3270 3271 3272 3273 3274 3275 3276 3277 3278 3279 3280 3281 3282 3283 3284 3285 3286 3287 3288 3289 3290 3291 3292 3293 3294 3295 3296 3297 3298 | iNewLevel = getAbsoluteLevel(p, iLangid, iIndex, iLevel+1); rc = fts3AllocateSegdirIdx(p, iLangid, iIndex, iLevel+1, &iIdx); bIgnoreEmpty = (iLevel!=FTS3_SEGCURSOR_PENDING) && (iNewLevel>iMaxLevel); } if( rc!=SQLITE_OK ) goto finished; assert( csr.nSegment>0 ); assert_fts3_nc( iNewLevel>=getAbsoluteLevel(p, iLangid, iIndex, 0) ); assert_fts3_nc( iNewLevel<getAbsoluteLevel(p, iLangid, iIndex,FTS3_SEGDIR_MAXLEVEL) ); memset(&filter, 0, sizeof(Fts3SegFilter)); filter.flags = FTS3_SEGMENT_REQUIRE_POS; filter.flags |= (bIgnoreEmpty ? FTS3_SEGMENT_IGNORE_EMPTY : 0); rc = sqlite3Fts3SegReaderStart(p, &csr, &filter); while( SQLITE_OK==rc ){ rc = sqlite3Fts3SegReaderStep(p, &csr); if( rc!=SQLITE_ROW ) break; rc = fts3SegWriterAdd(p, &pWriter, 1, csr.zTerm, csr.nTerm, csr.aDoclist, csr.nDoclist); } if( rc!=SQLITE_OK ) goto finished; assert_fts3_nc( pWriter || bIgnoreEmpty ); if( iLevel!=FTS3_SEGCURSOR_PENDING ){ rc = fts3DeleteSegdir( p, iLangid, iIndex, iLevel, csr.apSegment, csr.nSegment ); if( rc!=SQLITE_OK ) goto finished; } |
︙ | ︙ | |||
3318 3319 3320 3321 3322 3323 3324 | */ static void fts3DecodeIntArray( int N, /* The number of integers to decode */ u32 *a, /* Write the integer values */ const char *zBuf, /* The BLOB containing the varints */ int nBuf /* size of the BLOB */ ){ | | | > | | | < | | > > | | 3369 3370 3371 3372 3373 3374 3375 3376 3377 3378 3379 3380 3381 3382 3383 3384 3385 3386 3387 3388 3389 3390 3391 3392 3393 3394 3395 3396 3397 3398 3399 3400 3401 3402 3403 3404 3405 3406 3407 3408 3409 3410 3411 | */ static void fts3DecodeIntArray( int N, /* The number of integers to decode */ u32 *a, /* Write the integer values */ const char *zBuf, /* The BLOB containing the varints */ int nBuf /* size of the BLOB */ ){ int i = 0; if( nBuf && (zBuf[nBuf-1]&0x80)==0 ){ int j; for(i=j=0; i<N && j<nBuf; i++){ sqlite3_int64 x; j += sqlite3Fts3GetVarint(&zBuf[j], &x); a[i] = (u32)(x & 0xffffffff); } } while( i<N ) a[i++] = 0; } /* ** Insert the sizes (in tokens) for each column of the document ** with docid equal to p->iPrevDocid. The sizes are encoded as ** a blob of varints. */ static void fts3InsertDocsize( int *pRC, /* Result code */ Fts3Table *p, /* Table into which to insert */ u32 *aSz /* Sizes of each column, in tokens */ ){ char *pBlob; /* The BLOB encoding of the document size */ int nBlob; /* Number of bytes in the BLOB */ sqlite3_stmt *pStmt; /* Statement used to insert the encoding */ int rc; /* Result code from subfunctions */ if( *pRC ) return; pBlob = sqlite3_malloc64( 10*(sqlite3_int64)p->nColumn ); if( pBlob==0 ){ *pRC = SQLITE_NOMEM; return; } fts3EncodeIntArray(p->nColumn, aSz, pBlob, &nBlob); rc = fts3SqlStmt(p, SQL_REPLACE_DOCSIZE, &pStmt, 0); if( rc ){ |
︙ | ︙ | |||
3394 3395 3396 3397 3398 3399 3400 | sqlite3_stmt *pStmt; /* Statement for reading and writing */ int i; /* Loop counter */ int rc; /* Result code from subfunctions */ const int nStat = p->nColumn+2; if( *pRC ) return; | | | 3447 3448 3449 3450 3451 3452 3453 3454 3455 3456 3457 3458 3459 3460 3461 | sqlite3_stmt *pStmt; /* Statement for reading and writing */ int i; /* Loop counter */ int rc; /* Result code from subfunctions */ const int nStat = p->nColumn+2; if( *pRC ) return; a = sqlite3_malloc64( (sizeof(u32)+10)*(sqlite3_int64)nStat ); if( a==0 ){ *pRC = SQLITE_NOMEM; return; } pBlob = (char*)&a[nStat]; rc = fts3SqlStmt(p, SQL_SELECT_STAT, &pStmt, 0); if( rc ){ |
︙ | ︙ | |||
3458 3459 3460 3461 3462 3463 3464 | ** iIndex/iLangid combination. */ static int fts3DoOptimize(Fts3Table *p, int bReturnDone){ int bSeenDone = 0; int rc; sqlite3_stmt *pAllLangid = 0; | > > | > < | 3511 3512 3513 3514 3515 3516 3517 3518 3519 3520 3521 3522 3523 3524 3525 3526 3527 3528 3529 3530 3531 3532 3533 3534 3535 3536 3537 3538 3539 3540 3541 3542 3543 3544 3545 3546 3547 3548 | ** iIndex/iLangid combination. */ static int fts3DoOptimize(Fts3Table *p, int bReturnDone){ int bSeenDone = 0; int rc; sqlite3_stmt *pAllLangid = 0; rc = sqlite3Fts3PendingTermsFlush(p); if( rc==SQLITE_OK ){ rc = fts3SqlStmt(p, SQL_SELECT_ALL_LANGID, &pAllLangid, 0); } if( rc==SQLITE_OK ){ int rc2; sqlite3_bind_int(pAllLangid, 1, p->iPrevLangid); sqlite3_bind_int(pAllLangid, 2, p->nIndex); while( sqlite3_step(pAllLangid)==SQLITE_ROW ){ int i; int iLangid = sqlite3_column_int(pAllLangid, 0); for(i=0; rc==SQLITE_OK && i<p->nIndex; i++){ rc = fts3SegmentMerge(p, iLangid, i, FTS3_SEGCURSOR_ALL); if( rc==SQLITE_DONE ){ bSeenDone = 1; rc = SQLITE_OK; } } } rc2 = sqlite3_reset(pAllLangid); if( rc==SQLITE_OK ) rc = rc2; } sqlite3Fts3SegmentsClose(p); return (rc==SQLITE_OK && bReturnDone && bSeenDone) ? SQLITE_DONE : rc; } /* ** This function is called when the user executes the following statement: ** |
︙ | ︙ | |||
3515 3516 3517 3518 3519 3520 3521 | rc = SQLITE_NOMEM; }else{ rc = sqlite3_prepare_v2(p->db, zSql, -1, &pStmt, 0); sqlite3_free(zSql); } if( rc==SQLITE_OK ){ | | | | 3570 3571 3572 3573 3574 3575 3576 3577 3578 3579 3580 3581 3582 3583 3584 3585 | rc = SQLITE_NOMEM; }else{ rc = sqlite3_prepare_v2(p->db, zSql, -1, &pStmt, 0); sqlite3_free(zSql); } if( rc==SQLITE_OK ){ sqlite3_int64 nByte = sizeof(u32) * ((sqlite3_int64)p->nColumn+1)*3; aSz = (u32 *)sqlite3_malloc64(nByte); if( aSz==0 ){ rc = SQLITE_NOMEM; }else{ memset(aSz, 0, nByte); aSzIns = &aSz[p->nColumn+1]; aSzDel = &aSzIns[p->nColumn+1]; } |
︙ | ︙ | |||
3582 3583 3584 3585 3586 3587 3588 | Fts3Table *p, /* FTS3 table handle */ sqlite3_int64 iAbsLevel, /* Absolute level to open */ int nSeg, /* Number of segments to merge */ Fts3MultiSegReader *pCsr /* Cursor object to populate */ ){ int rc; /* Return Code */ sqlite3_stmt *pStmt = 0; /* Statement used to read %_segdir entry */ | | | | 3637 3638 3639 3640 3641 3642 3643 3644 3645 3646 3647 3648 3649 3650 3651 3652 3653 3654 3655 3656 | Fts3Table *p, /* FTS3 table handle */ sqlite3_int64 iAbsLevel, /* Absolute level to open */ int nSeg, /* Number of segments to merge */ Fts3MultiSegReader *pCsr /* Cursor object to populate */ ){ int rc; /* Return Code */ sqlite3_stmt *pStmt = 0; /* Statement used to read %_segdir entry */ sqlite3_int64 nByte; /* Bytes allocated at pCsr->apSegment[] */ /* Allocate space for the Fts3MultiSegReader.aCsr[] array */ memset(pCsr, 0, sizeof(*pCsr)); nByte = sizeof(Fts3SegReader *) * nSeg; pCsr->apSegment = (Fts3SegReader **)sqlite3_malloc64(nByte); if( pCsr->apSegment==0 ){ rc = SQLITE_NOMEM; }else{ memset(pCsr->apSegment, 0, nByte); rc = fts3SqlStmt(p, SQL_SELECT_LEVEL, &pStmt, 0); } |
︙ | ︙ | |||
3693 3694 3695 3696 3697 3698 3699 | ** If an OOM error occurs, set *pRc to SQLITE_NOMEM and leave pBlob->a ** unmodified. Otherwise, if the allocation succeeds, update pBlob->nAlloc ** to reflect the new size of the pBlob->a[] buffer. */ static void blobGrowBuffer(Blob *pBlob, int nMin, int *pRc){ if( *pRc==SQLITE_OK && nMin>pBlob->nAlloc ){ int nAlloc = nMin; | | | 3748 3749 3750 3751 3752 3753 3754 3755 3756 3757 3758 3759 3760 3761 3762 | ** If an OOM error occurs, set *pRc to SQLITE_NOMEM and leave pBlob->a ** unmodified. Otherwise, if the allocation succeeds, update pBlob->nAlloc ** to reflect the new size of the pBlob->a[] buffer. */ static void blobGrowBuffer(Blob *pBlob, int nMin, int *pRc){ if( *pRc==SQLITE_OK && nMin>pBlob->nAlloc ){ int nAlloc = nMin; char *a = (char *)sqlite3_realloc64(pBlob->a, nAlloc); if( a ){ pBlob->nAlloc = nAlloc; pBlob->a = a; }else{ *pRc = SQLITE_NOMEM; } } |
︙ | ︙ | |||
3730 3731 3732 3733 3734 3735 3736 | p->aNode = 0; }else{ if( bFirst==0 ){ p->iOff += fts3GetVarint32(&p->aNode[p->iOff], &nPrefix); } p->iOff += fts3GetVarint32(&p->aNode[p->iOff], &nSuffix); | | | | | | | 3785 3786 3787 3788 3789 3790 3791 3792 3793 3794 3795 3796 3797 3798 3799 3800 3801 3802 3803 3804 3805 3806 3807 3808 3809 3810 3811 3812 3813 3814 3815 3816 3817 3818 | p->aNode = 0; }else{ if( bFirst==0 ){ p->iOff += fts3GetVarint32(&p->aNode[p->iOff], &nPrefix); } p->iOff += fts3GetVarint32(&p->aNode[p->iOff], &nSuffix); if( nPrefix>p->term.n || nSuffix>p->nNode-p->iOff || nSuffix==0 ){ return FTS_CORRUPT_VTAB; } blobGrowBuffer(&p->term, nPrefix+nSuffix, &rc); if( rc==SQLITE_OK && ALWAYS(p->term.a!=0) ){ memcpy(&p->term.a[nPrefix], &p->aNode[p->iOff], nSuffix); p->term.n = nPrefix+nSuffix; p->iOff += nSuffix; if( p->iChild==0 ){ p->iOff += fts3GetVarint32(&p->aNode[p->iOff], &p->nDoclist); if( (p->nNode-p->iOff)<p->nDoclist ){ return FTS_CORRUPT_VTAB; } p->aDoclist = &p->aNode[p->iOff]; p->iOff += p->nDoclist; } } } assert_fts3_nc( p->iOff<=p->nNode ); return rc; } /* ** Release all dynamic resources held by node-reader object *p. */ static void nodeReaderRelease(NodeReader *p){ |
︙ | ︙ | |||
3773 3774 3775 3776 3777 3778 3779 | */ static int nodeReaderInit(NodeReader *p, const char *aNode, int nNode){ memset(p, 0, sizeof(NodeReader)); p->aNode = aNode; p->nNode = nNode; /* Figure out if this is a leaf or an internal node. */ | | | | 3828 3829 3830 3831 3832 3833 3834 3835 3836 3837 3838 3839 3840 3841 3842 3843 3844 3845 3846 3847 3848 3849 | */ static int nodeReaderInit(NodeReader *p, const char *aNode, int nNode){ memset(p, 0, sizeof(NodeReader)); p->aNode = aNode; p->nNode = nNode; /* Figure out if this is a leaf or an internal node. */ if( aNode && aNode[0] ){ /* An internal node. */ p->iOff = 1 + sqlite3Fts3GetVarint(&p->aNode[1], &p->iChild); }else{ p->iOff = 1; } return aNode ? nodeReaderNext(p) : SQLITE_OK; } /* ** This function is called while writing an FTS segment each time a leaf o ** node is finished and written to disk. The key (zTerm/nTerm) is guaranteed ** to be greater than the largest key on the node just written, but smaller ** than or equal to the first key that will be written to the next leaf |
︙ | ︙ | |||
3817 3818 3819 3820 3821 3822 3823 3824 3825 3826 3827 3828 3829 3830 | /* Figure out how much space the key will consume if it is written to ** the current node of layer iLayer. Due to the prefix compression, ** the space required changes depending on which node the key is to ** be added to. */ nPrefix = fts3PrefixCompress(pNode->key.a, pNode->key.n, zTerm, nTerm); nSuffix = nTerm - nPrefix; nSpace = sqlite3Fts3VarintLen(nPrefix); nSpace += sqlite3Fts3VarintLen(nSuffix) + nSuffix; if( pNode->key.n==0 || (pNode->block.n + nSpace)<=p->nNodeSize ){ /* If the current node of layer iLayer contains zero keys, or if adding ** the key to it will not cause it to grow to larger than nNodeSize ** bytes in size, write the key here. */ | > | 3872 3873 3874 3875 3876 3877 3878 3879 3880 3881 3882 3883 3884 3885 3886 | /* Figure out how much space the key will consume if it is written to ** the current node of layer iLayer. Due to the prefix compression, ** the space required changes depending on which node the key is to ** be added to. */ nPrefix = fts3PrefixCompress(pNode->key.a, pNode->key.n, zTerm, nTerm); nSuffix = nTerm - nPrefix; if(nSuffix<=0 ) return FTS_CORRUPT_VTAB; nSpace = sqlite3Fts3VarintLen(nPrefix); nSpace += sqlite3Fts3VarintLen(nSuffix) + nSuffix; if( pNode->key.n==0 || (pNode->block.n + nSpace)<=p->nNodeSize ){ /* If the current node of layer iLayer contains zero keys, or if adding ** the key to it will not cause it to grow to larger than nNodeSize ** bytes in size, write the key here. */ |
︙ | ︙ | |||
3841 3842 3843 3844 3845 3846 3847 3848 3849 3850 3851 3852 3853 3854 | blobGrowBuffer(&pNode->key, nTerm, &rc); if( rc==SQLITE_OK ){ if( pNode->key.n ){ pBlk->n += sqlite3Fts3PutVarint(&pBlk->a[pBlk->n], nPrefix); } pBlk->n += sqlite3Fts3PutVarint(&pBlk->a[pBlk->n], nSuffix); memcpy(&pBlk->a[pBlk->n], &zTerm[nPrefix], nSuffix); pBlk->n += nSuffix; memcpy(pNode->key.a, zTerm, nTerm); pNode->key.n = nTerm; } }else{ | > > | 3897 3898 3899 3900 3901 3902 3903 3904 3905 3906 3907 3908 3909 3910 3911 3912 | blobGrowBuffer(&pNode->key, nTerm, &rc); if( rc==SQLITE_OK ){ if( pNode->key.n ){ pBlk->n += sqlite3Fts3PutVarint(&pBlk->a[pBlk->n], nPrefix); } pBlk->n += sqlite3Fts3PutVarint(&pBlk->a[pBlk->n], nSuffix); assert( nPrefix+nSuffix<=nTerm ); assert( nPrefix>=0 ); memcpy(&pBlk->a[pBlk->n], &zTerm[nPrefix], nSuffix); pBlk->n += nSuffix; memcpy(pNode->key.a, zTerm, nTerm); pNode->key.n = nTerm; } }else{ |
︙ | ︙ | |||
3910 3911 3912 3913 3914 3915 3916 | int bFirst = (pPrev->n==0); /* True if this is the first term written */ int nPrefix; /* Size of term prefix in bytes */ int nSuffix; /* Size of term suffix in bytes */ /* Node must have already been started. There must be a doclist for a ** leaf node, and there must not be a doclist for an internal node. */ assert( pNode->n>0 ); | | > | 3968 3969 3970 3971 3972 3973 3974 3975 3976 3977 3978 3979 3980 3981 3982 3983 3984 3985 3986 3987 3988 3989 | int bFirst = (pPrev->n==0); /* True if this is the first term written */ int nPrefix; /* Size of term prefix in bytes */ int nSuffix; /* Size of term suffix in bytes */ /* Node must have already been started. There must be a doclist for a ** leaf node, and there must not be a doclist for an internal node. */ assert( pNode->n>0 ); assert_fts3_nc( (pNode->a[0]=='\0')==(aDoclist!=0) ); blobGrowBuffer(pPrev, nTerm, &rc); if( rc!=SQLITE_OK ) return rc; nPrefix = fts3PrefixCompress(pPrev->a, pPrev->n, zTerm, nTerm); nSuffix = nTerm - nPrefix; if( nSuffix<=0 ) return FTS_CORRUPT_VTAB; memcpy(pPrev->a, zTerm, nTerm); pPrev->n = nTerm; if( bFirst==0 ){ pNode->n += sqlite3Fts3PutVarint(&pNode->a[pNode->n], nPrefix); } pNode->n += sqlite3Fts3PutVarint(&pNode->a[pNode->n], nSuffix); |
︙ | ︙ | |||
3962 3963 3964 3965 3966 3967 3968 3969 3970 3971 3972 3973 3974 3975 | int nPrefix; /* Size of prefix shared with previous term */ int nSuffix; /* Size of suffix (nTerm - nPrefix) */ NodeWriter *pLeaf; /* Object used to write leaf nodes */ pLeaf = &pWriter->aNodeWriter[0]; nPrefix = fts3PrefixCompress(pLeaf->key.a, pLeaf->key.n, zTerm, nTerm); nSuffix = nTerm - nPrefix; nSpace = sqlite3Fts3VarintLen(nPrefix); nSpace += sqlite3Fts3VarintLen(nSuffix) + nSuffix; nSpace += sqlite3Fts3VarintLen(nDoclist) + nDoclist; /* If the current block is not empty, and if adding this term/doclist ** to the current block would make it larger than Fts3Table.nNodeSize | > | 4021 4022 4023 4024 4025 4026 4027 4028 4029 4030 4031 4032 4033 4034 4035 | int nPrefix; /* Size of prefix shared with previous term */ int nSuffix; /* Size of suffix (nTerm - nPrefix) */ NodeWriter *pLeaf; /* Object used to write leaf nodes */ pLeaf = &pWriter->aNodeWriter[0]; nPrefix = fts3PrefixCompress(pLeaf->key.a, pLeaf->key.n, zTerm, nTerm); nSuffix = nTerm - nPrefix; if(nSuffix<=0 ) return FTS_CORRUPT_VTAB; nSpace = sqlite3Fts3VarintLen(nPrefix); nSpace += sqlite3Fts3VarintLen(nSuffix) + nSuffix; nSpace += sqlite3Fts3VarintLen(nDoclist) + nDoclist; /* If the current block is not empty, and if adding this term/doclist ** to the current block would make it larger than Fts3Table.nNodeSize |
︙ | ︙ | |||
4126 4127 4128 4129 4130 4131 4132 | static int fts3TermCmp( const char *zLhs, int nLhs, /* LHS of comparison */ const char *zRhs, int nRhs /* RHS of comparison */ ){ int nCmp = MIN(nLhs, nRhs); int res; | > | > > > | 4186 4187 4188 4189 4190 4191 4192 4193 4194 4195 4196 4197 4198 4199 4200 4201 4202 4203 4204 | static int fts3TermCmp( const char *zLhs, int nLhs, /* LHS of comparison */ const char *zRhs, int nRhs /* RHS of comparison */ ){ int nCmp = MIN(nLhs, nRhs); int res; if( nCmp && ALWAYS(zLhs) && ALWAYS(zRhs) ){ res = memcmp(zLhs, zRhs, nCmp); }else{ res = 0; } if( res==0 ) res = nLhs - nRhs; return res; } /* |
︙ | ︙ | |||
4210 4211 4212 4213 4214 4215 4216 4217 4218 4219 4220 4221 4222 4223 | fts3ReadEndBlockField(pSelect, 3, &iEnd, &pWriter->nLeafData); if( pWriter->nLeafData<0 ){ pWriter->nLeafData = pWriter->nLeafData * -1; } pWriter->bNoLeafData = (pWriter->nLeafData==0); nRoot = sqlite3_column_bytes(pSelect, 4); aRoot = sqlite3_column_blob(pSelect, 4); }else{ return sqlite3_reset(pSelect); } /* Check for the zero-length marker in the %_segments table */ rc = fts3IsAppendable(p, iEnd, &bAppendable); | > > > > | 4274 4275 4276 4277 4278 4279 4280 4281 4282 4283 4284 4285 4286 4287 4288 4289 4290 4291 | fts3ReadEndBlockField(pSelect, 3, &iEnd, &pWriter->nLeafData); if( pWriter->nLeafData<0 ){ pWriter->nLeafData = pWriter->nLeafData * -1; } pWriter->bNoLeafData = (pWriter->nLeafData==0); nRoot = sqlite3_column_bytes(pSelect, 4); aRoot = sqlite3_column_blob(pSelect, 4); if( aRoot==0 ){ sqlite3_reset(pSelect); return nRoot ? SQLITE_NOMEM : FTS_CORRUPT_VTAB; } }else{ return sqlite3_reset(pSelect); } /* Check for the zero-length marker in the %_segments table */ rc = fts3IsAppendable(p, iEnd, &bAppendable); |
︙ | ︙ | |||
4245 4246 4247 4248 4249 4250 4251 4252 4253 4254 4255 4256 4257 4258 4259 4260 4261 4262 4263 4264 | if( rc==SQLITE_OK && bAppendable ){ /* It is possible to append to this segment. Set up the IncrmergeWriter ** object to do so. */ int i; int nHeight = (int)aRoot[0]; NodeWriter *pNode; pWriter->nLeafEst = (int)((iEnd - iStart) + 1)/FTS_MAX_APPENDABLE_HEIGHT; pWriter->iStart = iStart; pWriter->iEnd = iEnd; pWriter->iAbsLevel = iAbsLevel; pWriter->iIdx = iIdx; for(i=nHeight+1; i<FTS_MAX_APPENDABLE_HEIGHT; i++){ pWriter->aNodeWriter[i].iBlock = pWriter->iStart + i*pWriter->nLeafEst; } pNode = &pWriter->aNodeWriter[nHeight]; pNode->iBlock = pWriter->iStart + pWriter->nLeafEst*nHeight; | > > > > | > > > > | | | | > > | > | | | | | | | | > > | | | > | | > | 4313 4314 4315 4316 4317 4318 4319 4320 4321 4322 4323 4324 4325 4326 4327 4328 4329 4330 4331 4332 4333 4334 4335 4336 4337 4338 4339 4340 4341 4342 4343 4344 4345 4346 4347 4348 4349 4350 4351 4352 4353 4354 4355 4356 4357 4358 4359 4360 4361 4362 4363 4364 4365 4366 4367 4368 4369 4370 4371 4372 4373 4374 4375 4376 4377 4378 4379 4380 4381 4382 | if( rc==SQLITE_OK && bAppendable ){ /* It is possible to append to this segment. Set up the IncrmergeWriter ** object to do so. */ int i; int nHeight = (int)aRoot[0]; NodeWriter *pNode; if( nHeight<1 || nHeight>=FTS_MAX_APPENDABLE_HEIGHT ){ sqlite3_reset(pSelect); return FTS_CORRUPT_VTAB; } pWriter->nLeafEst = (int)((iEnd - iStart) + 1)/FTS_MAX_APPENDABLE_HEIGHT; pWriter->iStart = iStart; pWriter->iEnd = iEnd; pWriter->iAbsLevel = iAbsLevel; pWriter->iIdx = iIdx; for(i=nHeight+1; i<FTS_MAX_APPENDABLE_HEIGHT; i++){ pWriter->aNodeWriter[i].iBlock = pWriter->iStart + i*pWriter->nLeafEst; } pNode = &pWriter->aNodeWriter[nHeight]; pNode->iBlock = pWriter->iStart + pWriter->nLeafEst*nHeight; blobGrowBuffer(&pNode->block, MAX(nRoot, p->nNodeSize)+FTS3_NODE_PADDING, &rc ); if( rc==SQLITE_OK ){ memcpy(pNode->block.a, aRoot, nRoot); pNode->block.n = nRoot; memset(&pNode->block.a[nRoot], 0, FTS3_NODE_PADDING); } for(i=nHeight; i>=0 && rc==SQLITE_OK; i--){ NodeReader reader; pNode = &pWriter->aNodeWriter[i]; if( pNode->block.a){ rc = nodeReaderInit(&reader, pNode->block.a, pNode->block.n); while( reader.aNode && rc==SQLITE_OK ) rc = nodeReaderNext(&reader); blobGrowBuffer(&pNode->key, reader.term.n, &rc); if( rc==SQLITE_OK ){ assert_fts3_nc( reader.term.n>0 || reader.aNode==0 ); if( reader.term.n>0 ){ memcpy(pNode->key.a, reader.term.a, reader.term.n); } pNode->key.n = reader.term.n; if( i>0 ){ char *aBlock = 0; int nBlock = 0; pNode = &pWriter->aNodeWriter[i-1]; pNode->iBlock = reader.iChild; rc = sqlite3Fts3ReadBlock(p, reader.iChild, &aBlock, &nBlock,0); blobGrowBuffer(&pNode->block, MAX(nBlock, p->nNodeSize)+FTS3_NODE_PADDING, &rc ); if( rc==SQLITE_OK ){ memcpy(pNode->block.a, aBlock, nBlock); pNode->block.n = nBlock; memset(&pNode->block.a[nBlock], 0, FTS3_NODE_PADDING); } sqlite3_free(aBlock); } } } nodeReaderRelease(&reader); } } rc2 = sqlite3_reset(pSelect); |
︙ | ︙ | |||
4462 4463 4464 4465 4466 4467 4468 | if( rc==SQLITE_OK ){ int rc2; sqlite3_bind_int64(pSelect, 1, iAbsLevel); while( SQLITE_ROW==sqlite3_step(pSelect) ){ if( nIdx>=nAlloc ){ int *aNew; nAlloc += 16; | | | 4545 4546 4547 4548 4549 4550 4551 4552 4553 4554 4555 4556 4557 4558 4559 | if( rc==SQLITE_OK ){ int rc2; sqlite3_bind_int64(pSelect, 1, iAbsLevel); while( SQLITE_ROW==sqlite3_step(pSelect) ){ if( nIdx>=nAlloc ){ int *aNew; nAlloc += 16; aNew = sqlite3_realloc64(aIdx, nAlloc*sizeof(int)); if( !aNew ){ rc = SQLITE_NOMEM; break; } aIdx = aNew; } aIdx[nIdx++] = sqlite3_column_int(pSelect, 0); |
︙ | ︙ | |||
4528 4529 4530 4531 4532 4533 4534 | const char *zTerm, /* Omit all terms smaller than this */ int nTerm, /* Size of zTerm in bytes */ sqlite3_int64 *piBlock /* OUT: Block number in next layer down */ ){ NodeReader reader; /* Reader object */ Blob prev = {0, 0, 0}; /* Previous term written to new node */ int rc = SQLITE_OK; /* Return code */ | > > > | | 4611 4612 4613 4614 4615 4616 4617 4618 4619 4620 4621 4622 4623 4624 4625 4626 4627 4628 | const char *zTerm, /* Omit all terms smaller than this */ int nTerm, /* Size of zTerm in bytes */ sqlite3_int64 *piBlock /* OUT: Block number in next layer down */ ){ NodeReader reader; /* Reader object */ Blob prev = {0, 0, 0}; /* Previous term written to new node */ int rc = SQLITE_OK; /* Return code */ int bLeaf; /* True for a leaf node */ if( nNode<1 ) return FTS_CORRUPT_VTAB; bLeaf = aNode[0]=='\0'; /* Allocate required output space */ blobGrowBuffer(pNew, nNode, &rc); if( rc!=SQLITE_OK ) return rc; pNew->n = 0; /* Populate new node buffer */ |
︙ | ︙ | |||
4748 4749 4750 4751 4752 4753 4754 | sqlite3_bind_int(pSelect, 1, FTS_STAT_INCRMERGEHINT); if( SQLITE_ROW==sqlite3_step(pSelect) ){ const char *aHint = sqlite3_column_blob(pSelect, 0); int nHint = sqlite3_column_bytes(pSelect, 0); if( aHint ){ blobGrowBuffer(pHint, nHint, &rc); if( rc==SQLITE_OK ){ | | | 4834 4835 4836 4837 4838 4839 4840 4841 4842 4843 4844 4845 4846 4847 4848 | sqlite3_bind_int(pSelect, 1, FTS_STAT_INCRMERGEHINT); if( SQLITE_ROW==sqlite3_step(pSelect) ){ const char *aHint = sqlite3_column_blob(pSelect, 0); int nHint = sqlite3_column_bytes(pSelect, 0); if( aHint ){ blobGrowBuffer(pHint, nHint, &rc); if( rc==SQLITE_OK ){ if( ALWAYS(pHint->a!=0) ) memcpy(pHint->a, aHint, nHint); pHint->n = nHint; } } } rc2 = sqlite3_reset(pSelect); if( rc==SQLITE_OK ) rc = rc2; } |
︙ | ︙ | |||
4794 4795 4796 4797 4798 4799 4800 | ** If no error occurs, return SQLITE_OK. If the hint blob in *pHint does ** not contain at least two valid varints, return SQLITE_CORRUPT_VTAB. */ static int fts3IncrmergeHintPop(Blob *pHint, i64 *piAbsLevel, int *pnInput){ const int nHint = pHint->n; int i; | | > > > > | 4880 4881 4882 4883 4884 4885 4886 4887 4888 4889 4890 4891 4892 4893 4894 4895 4896 4897 4898 4899 4900 4901 4902 4903 4904 | ** If no error occurs, return SQLITE_OK. If the hint blob in *pHint does ** not contain at least two valid varints, return SQLITE_CORRUPT_VTAB. */ static int fts3IncrmergeHintPop(Blob *pHint, i64 *piAbsLevel, int *pnInput){ const int nHint = pHint->n; int i; i = pHint->n-1; if( (pHint->a[i] & 0x80) ) return FTS_CORRUPT_VTAB; while( i>0 && (pHint->a[i-1] & 0x80) ) i--; if( i==0 ) return FTS_CORRUPT_VTAB; i--; while( i>0 && (pHint->a[i-1] & 0x80) ) i--; pHint->n = i; i += sqlite3Fts3GetVarint(&pHint->a[i], piAbsLevel); i += fts3GetVarint32(&pHint->a[i], pnInput); assert( i<=nHint ); if( i!=nHint ) return FTS_CORRUPT_VTAB; return SQLITE_OK; } /* |
︙ | ︙ | |||
4829 4830 4831 4832 4833 4834 4835 | int nSeg = 0; /* Number of input segments */ sqlite3_int64 iAbsLevel = 0; /* Absolute level number to work on */ Blob hint = {0, 0, 0}; /* Hint read from %_stat table */ int bDirtyHint = 0; /* True if blob 'hint' has been modified */ /* Allocate space for the cursor, filter and writer objects */ const int nAlloc = sizeof(*pCsr) + sizeof(*pFilter) + sizeof(*pWriter); | | | 4919 4920 4921 4922 4923 4924 4925 4926 4927 4928 4929 4930 4931 4932 4933 | int nSeg = 0; /* Number of input segments */ sqlite3_int64 iAbsLevel = 0; /* Absolute level number to work on */ Blob hint = {0, 0, 0}; /* Hint read from %_stat table */ int bDirtyHint = 0; /* True if blob 'hint' has been modified */ /* Allocate space for the cursor, filter and writer objects */ const int nAlloc = sizeof(*pCsr) + sizeof(*pFilter) + sizeof(*pWriter); pWriter = (IncrmergeWriter *)sqlite3_malloc64(nAlloc); if( !pWriter ) return SQLITE_NOMEM; pFilter = (Fts3SegFilter *)&pWriter[1]; pCsr = (Fts3MultiSegReader *)&pFilter[1]; rc = fts3IncrmergeHintLoad(p, &hint); while( rc==SQLITE_OK && nRem>0 ){ const i64 nMod = FTS3_SEGDIR_MAXLEVEL * p->nIndex; |
︙ | ︙ | |||
4870 4871 4872 4873 4874 4875 4876 4877 | if( rc==SQLITE_OK && hint.n ){ int nHint = hint.n; sqlite3_int64 iHintAbsLevel = 0; /* Hint level */ int nHintSeg = 0; /* Hint number of segments */ rc = fts3IncrmergeHintPop(&hint, &iHintAbsLevel, &nHintSeg); if( nSeg<0 || (iAbsLevel % nMod) >= (iHintAbsLevel % nMod) ){ iAbsLevel = iHintAbsLevel; | > > > > > > | > > > > > | > | 4960 4961 4962 4963 4964 4965 4966 4967 4968 4969 4970 4971 4972 4973 4974 4975 4976 4977 4978 4979 4980 4981 4982 4983 4984 4985 4986 4987 4988 4989 4990 4991 4992 4993 4994 4995 4996 4997 4998 4999 5000 | if( rc==SQLITE_OK && hint.n ){ int nHint = hint.n; sqlite3_int64 iHintAbsLevel = 0; /* Hint level */ int nHintSeg = 0; /* Hint number of segments */ rc = fts3IncrmergeHintPop(&hint, &iHintAbsLevel, &nHintSeg); if( nSeg<0 || (iAbsLevel % nMod) >= (iHintAbsLevel % nMod) ){ /* Based on the scan in the block above, it is known that there ** are no levels with a relative level smaller than that of ** iAbsLevel with more than nSeg segments, or if nSeg is -1, ** no levels with more than nMin segments. Use this to limit the ** value of nHintSeg to avoid a large memory allocation in case the ** merge-hint is corrupt*/ iAbsLevel = iHintAbsLevel; nSeg = MIN(MAX(nMin,nSeg), nHintSeg); bUseHint = 1; bDirtyHint = 1; }else{ /* This undoes the effect of the HintPop() above - so that no entry ** is removed from the hint blob. */ hint.n = nHint; } } /* If nSeg is less that zero, then there is no level with at least ** nMin segments and no hint in the %_stat table. No work to do. ** Exit early in this case. */ if( nSeg<=0 ) break; assert( nMod<=0x7FFFFFFF ); if( iAbsLevel<0 || iAbsLevel>(nMod<<32) ){ rc = FTS_CORRUPT_VTAB; break; } /* Open a cursor to iterate through the contents of the oldest nSeg ** indexes of absolute level iAbsLevel. If this cursor is opened using ** the 'hint' parameters, it is possible that there are less than nSeg ** segments available in level iAbsLevel. In this case, no work is ** done on iAbsLevel - fall through to the next iteration of the loop ** to start work on some other level. */ |
︙ | ︙ | |||
4912 4913 4914 4915 4916 4917 4918 | } if( rc==SQLITE_OK ){ rc = fts3IncrmergeCsr(p, iAbsLevel, nSeg, pCsr); } if( SQLITE_OK==rc && pCsr->nSegment==nSeg && SQLITE_OK==(rc = sqlite3Fts3SegReaderStart(p, pCsr, pFilter)) | < > > > > > > > > > | | | | | > | 5014 5015 5016 5017 5018 5019 5020 5021 5022 5023 5024 5025 5026 5027 5028 5029 5030 5031 5032 5033 5034 5035 5036 5037 5038 5039 5040 5041 5042 5043 5044 5045 5046 5047 5048 5049 5050 5051 5052 5053 | } if( rc==SQLITE_OK ){ rc = fts3IncrmergeCsr(p, iAbsLevel, nSeg, pCsr); } if( SQLITE_OK==rc && pCsr->nSegment==nSeg && SQLITE_OK==(rc = sqlite3Fts3SegReaderStart(p, pCsr, pFilter)) ){ int bEmpty = 0; rc = sqlite3Fts3SegReaderStep(p, pCsr); if( rc==SQLITE_OK ){ bEmpty = 1; }else if( rc!=SQLITE_ROW ){ sqlite3Fts3SegReaderFinish(pCsr); break; } if( bUseHint && iIdx>0 ){ const char *zKey = pCsr->zTerm; int nKey = pCsr->nTerm; rc = fts3IncrmergeLoad(p, iAbsLevel, iIdx-1, zKey, nKey, pWriter); }else{ rc = fts3IncrmergeWriter(p, iAbsLevel, iIdx, pCsr, pWriter); } if( rc==SQLITE_OK && pWriter->nLeafEst ){ fts3LogMerge(nSeg, iAbsLevel); if( bEmpty==0 ){ do { rc = fts3IncrmergeAppend(p, pWriter, pCsr); if( rc==SQLITE_OK ) rc = sqlite3Fts3SegReaderStep(p, pCsr); if( pWriter->nWork>=nRem && rc==SQLITE_ROW ) rc = SQLITE_OK; }while( rc==SQLITE_ROW ); } /* Update or delete the input segments */ if( rc==SQLITE_OK ){ nRem -= (1 + pWriter->nWork); rc = fts3IncrmergeChomp(p, iAbsLevel, pCsr, &nSeg); if( nSeg!=0 ){ bDirtyHint = 1; |
︙ | ︙ | |||
4993 4994 4995 4996 4997 4998 4999 | ** before it will be selected for a merge, respectively. */ static int fts3DoIncrmerge( Fts3Table *p, /* FTS3 table handle */ const char *zParam /* Nul-terminated string containing "A,B" */ ){ int rc; | | | 5104 5105 5106 5107 5108 5109 5110 5111 5112 5113 5114 5115 5116 5117 5118 | ** before it will be selected for a merge, respectively. */ static int fts3DoIncrmerge( Fts3Table *p, /* FTS3 table handle */ const char *zParam /* Nul-terminated string containing "A,B" */ ){ int rc; int nMin = (MergeCount(p) / 2); int nMerge = 0; const char *z = zParam; /* Read the first integer value */ nMerge = fts3Getint(&z); /* If the first integer value is followed by a ',', read the second |
︙ | ︙ | |||
5038 5039 5040 5041 5042 5043 5044 | static int fts3DoAutoincrmerge( Fts3Table *p, /* FTS3 table handle */ const char *zParam /* Nul-terminated string containing boolean */ ){ int rc = SQLITE_OK; sqlite3_stmt *pStmt = 0; p->nAutoincrmerge = fts3Getint(&zParam); | | | 5149 5150 5151 5152 5153 5154 5155 5156 5157 5158 5159 5160 5161 5162 5163 | static int fts3DoAutoincrmerge( Fts3Table *p, /* FTS3 table handle */ const char *zParam /* Nul-terminated string containing boolean */ ){ int rc = SQLITE_OK; sqlite3_stmt *pStmt = 0; p->nAutoincrmerge = fts3Getint(&zParam); if( p->nAutoincrmerge==1 || p->nAutoincrmerge>MergeCount(p) ){ p->nAutoincrmerge = 8; } if( !p->bHasStat ){ assert( p->bFts4==0 ); sqlite3Fts3CreateStatTable(&rc, p); if( rc ) return rc; } |
︙ | ︙ | |||
5121 5122 5123 5124 5125 5126 5127 | if( rc==SQLITE_OK ){ while( SQLITE_ROW==(rc = sqlite3Fts3SegReaderStep(p, &csr)) ){ char *pCsr = csr.aDoclist; char *pEnd = &pCsr[csr.nDoclist]; i64 iDocid = 0; i64 iCol = 0; | | | | | > | > > > | 5232 5233 5234 5235 5236 5237 5238 5239 5240 5241 5242 5243 5244 5245 5246 5247 5248 5249 5250 5251 5252 5253 5254 5255 5256 5257 5258 5259 5260 5261 5262 5263 5264 | if( rc==SQLITE_OK ){ while( SQLITE_ROW==(rc = sqlite3Fts3SegReaderStep(p, &csr)) ){ char *pCsr = csr.aDoclist; char *pEnd = &pCsr[csr.nDoclist]; i64 iDocid = 0; i64 iCol = 0; u64 iPos = 0; pCsr += sqlite3Fts3GetVarint(pCsr, &iDocid); while( pCsr<pEnd ){ u64 iVal = 0; pCsr += sqlite3Fts3GetVarintU(pCsr, &iVal); if( pCsr<pEnd ){ if( iVal==0 || iVal==1 ){ iCol = 0; iPos = 0; if( iVal ){ pCsr += sqlite3Fts3GetVarint(pCsr, &iCol); }else{ pCsr += sqlite3Fts3GetVarintU(pCsr, &iVal); if( p->bDescIdx ){ iDocid = (i64)((u64)iDocid - iVal); }else{ iDocid = (i64)((u64)iDocid + iVal); } } }else{ iPos += (iVal - 2); cksum = cksum ^ fts3ChecksumEntry( csr.zTerm, csr.nTerm, iLangid, iIndex, iDocid, (int)iCol, (int)iPos ); |
︙ | ︙ | |||
5208 5209 5210 5211 5212 5213 5214 | i64 iDocid = sqlite3_column_int64(pStmt, 0); int iLang = langidFromSelect(p, pStmt); int iCol; for(iCol=0; rc==SQLITE_OK && iCol<p->nColumn; iCol++){ if( p->abNotindexed[iCol]==0 ){ const char *zText = (const char *)sqlite3_column_text(pStmt, iCol+1); | < | | 5323 5324 5325 5326 5327 5328 5329 5330 5331 5332 5333 5334 5335 5336 5337 5338 5339 | i64 iDocid = sqlite3_column_int64(pStmt, 0); int iLang = langidFromSelect(p, pStmt); int iCol; for(iCol=0; rc==SQLITE_OK && iCol<p->nColumn; iCol++){ if( p->abNotindexed[iCol]==0 ){ const char *zText = (const char *)sqlite3_column_text(pStmt, iCol+1); sqlite3_tokenizer_cursor *pT = 0; rc = sqlite3Fts3OpenTokenizer(p->pTokenizer, iLang, zText, -1, &pT); while( rc==SQLITE_OK ){ char const *zToken; /* Buffer containing token */ int nToken = 0; /* Number of bytes in token */ int iDum1 = 0, iDum2 = 0; /* Dummy variables */ int iPos = 0; /* Position of token in zText */ rc = pModule->xNext(pT, &zToken, &nToken, &iDum1, &iDum2, &iPos); |
︙ | ︙ | |||
5296 5297 5298 5299 5300 5301 5302 | ** ** "INSERT INTO tbl(tbl) VALUES(<expr>)" ** ** Argument pVal contains the result of <expr>. Currently the only ** meaningful value to insert is the text 'optimize'. */ static int fts3SpecialInsert(Fts3Table *p, sqlite3_value *pVal){ | | | > > | | > | | | > | | | | < | > > | | > | | 5410 5411 5412 5413 5414 5415 5416 5417 5418 5419 5420 5421 5422 5423 5424 5425 5426 5427 5428 5429 5430 5431 5432 5433 5434 5435 5436 5437 5438 5439 5440 5441 5442 5443 5444 5445 5446 5447 5448 5449 5450 5451 5452 5453 5454 5455 5456 5457 5458 5459 5460 | ** ** "INSERT INTO tbl(tbl) VALUES(<expr>)" ** ** Argument pVal contains the result of <expr>. Currently the only ** meaningful value to insert is the text 'optimize'. */ static int fts3SpecialInsert(Fts3Table *p, sqlite3_value *pVal){ int rc = SQLITE_ERROR; /* Return Code */ const char *zVal = (const char *)sqlite3_value_text(pVal); int nVal = sqlite3_value_bytes(pVal); if( !zVal ){ return SQLITE_NOMEM; }else if( nVal==8 && 0==sqlite3_strnicmp(zVal, "optimize", 8) ){ rc = fts3DoOptimize(p, 0); }else if( nVal==7 && 0==sqlite3_strnicmp(zVal, "rebuild", 7) ){ rc = fts3DoRebuild(p); }else if( nVal==15 && 0==sqlite3_strnicmp(zVal, "integrity-check", 15) ){ rc = fts3DoIntegrityCheck(p); }else if( nVal>6 && 0==sqlite3_strnicmp(zVal, "merge=", 6) ){ rc = fts3DoIncrmerge(p, &zVal[6]); }else if( nVal>10 && 0==sqlite3_strnicmp(zVal, "automerge=", 10) ){ rc = fts3DoAutoincrmerge(p, &zVal[10]); #if defined(SQLITE_DEBUG) || defined(SQLITE_TEST) }else{ int v; if( nVal>9 && 0==sqlite3_strnicmp(zVal, "nodesize=", 9) ){ v = atoi(&zVal[9]); if( v>=24 && v<=p->nPgsz-35 ) p->nNodeSize = v; rc = SQLITE_OK; }else if( nVal>11 && 0==sqlite3_strnicmp(zVal, "maxpending=", 9) ){ v = atoi(&zVal[11]); if( v>=64 && v<=FTS3_MAX_PENDING_DATA ) p->nMaxPendingData = v; rc = SQLITE_OK; }else if( nVal>21 && 0==sqlite3_strnicmp(zVal,"test-no-incr-doclist=",21) ){ p->bNoIncrDoclist = atoi(&zVal[21]); rc = SQLITE_OK; }else if( nVal>11 && 0==sqlite3_strnicmp(zVal,"mergecount=",11) ){ v = atoi(&zVal[11]); if( v>=4 && v<=FTS3_MERGE_COUNT && (v&1)==0 ) p->nMergeCount = v; rc = SQLITE_OK; } #endif } return rc; } #ifndef SQLITE_DISABLE_FTS4_DEFERRED /* ** Delete all cached deferred doclists. Deferred doclists are cached ** (allocated) by the sqlite3Fts3CacheDeferredDoclists() function. |
︙ | ︙ | |||
5435 5436 5437 5438 5439 5440 5441 | *ppData = 0; *pnData = 0; if( p->pList==0 ){ return SQLITE_OK; } | | | | 5555 5556 5557 5558 5559 5560 5561 5562 5563 5564 5565 5566 5567 5568 5569 5570 5571 5572 5573 5574 5575 5576 5577 5578 5579 5580 5581 5582 5583 5584 5585 5586 5587 5588 5589 | *ppData = 0; *pnData = 0; if( p->pList==0 ){ return SQLITE_OK; } pRet = (char *)sqlite3_malloc64(p->pList->nData); if( !pRet ) return SQLITE_NOMEM; nSkip = sqlite3Fts3GetVarint(p->pList->aData, &dummy); *pnData = p->pList->nData - nSkip; *ppData = pRet; memcpy(pRet, &p->pList->aData[nSkip], *pnData); return SQLITE_OK; } /* ** Add an entry for token pToken to the pCsr->pDeferred list. */ int sqlite3Fts3DeferToken( Fts3Cursor *pCsr, /* Fts3 table cursor */ Fts3PhraseToken *pToken, /* Token to defer */ int iCol /* Column that token must appear in (or -1) */ ){ Fts3DeferredToken *pDeferred; pDeferred = sqlite3_malloc64(sizeof(*pDeferred)); if( !pDeferred ){ return SQLITE_NOMEM; } memset(pDeferred, 0, sizeof(*pDeferred)); pDeferred->pToken = pToken; pDeferred->pNext = pCsr->pDeferred; pDeferred->iCol = iCol; |
︙ | ︙ | |||
5567 5568 5569 5570 5571 5572 5573 | if( nArg>1 && sqlite3_value_int(apVal[2 + p->nColumn + 2])<0 ){ rc = SQLITE_CONSTRAINT; goto update_out; } /* Allocate space to hold the change in document sizes */ | | | 5687 5688 5689 5690 5691 5692 5693 5694 5695 5696 5697 5698 5699 5700 5701 | if( nArg>1 && sqlite3_value_int(apVal[2 + p->nColumn + 2])<0 ){ rc = SQLITE_CONSTRAINT; goto update_out; } /* Allocate space to hold the change in document sizes */ aSzDel = sqlite3_malloc64(sizeof(aSzDel[0])*((sqlite3_int64)p->nColumn+1)*2); if( aSzDel==0 ){ rc = SQLITE_NOMEM; goto update_out; } aSzIns = &aSzDel[p->nColumn+1]; memset(aSzDel, 0, sizeof(aSzDel[0])*(p->nColumn+1)*2); |
︙ | ︙ |
Deleted ext/fts3/mkfts3amal.tcl.
|
| < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < |
Changes to ext/fts3/tool/fts3view.c.
︙ | ︙ | |||
89 90 91 92 93 94 95 | /* ** Show the table schema */ static void showSchema(sqlite3 *db, const char *zTab){ sqlite3_stmt *pStmt; pStmt = prepare(db, | | | 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 | /* ** Show the table schema */ static void showSchema(sqlite3 *db, const char *zTab){ sqlite3_stmt *pStmt; pStmt = prepare(db, "SELECT sql FROM sqlite_schema" " WHERE name LIKE '%q%%'" " ORDER BY 1", zTab); while( sqlite3_step(pStmt)==SQLITE_ROW ){ printf("%s;\n", sqlite3_column_text(pStmt, 0)); } sqlite3_finalize(pStmt); |
︙ | ︙ | |||
827 828 829 830 831 832 833 | fprintf(stderr, "Cannot open %s\n", argv[1]); exit(1); } if( argc==2 ){ sqlite3_stmt *pStmt; int cnt = 0; pStmt = prepare(db, "SELECT b.sql" | | | 827 828 829 830 831 832 833 834 835 836 837 838 839 840 841 | fprintf(stderr, "Cannot open %s\n", argv[1]); exit(1); } if( argc==2 ){ sqlite3_stmt *pStmt; int cnt = 0; pStmt = prepare(db, "SELECT b.sql" " FROM sqlite_schema a, sqlite_schema b" " WHERE a.name GLOB '*_segdir'" " AND b.name=substr(a.name,1,length(a.name)-7)" " ORDER BY 1"); while( sqlite3_step(pStmt)==SQLITE_ROW ){ cnt++; printf("%s;\n", sqlite3_column_text(pStmt, 0)); } |
︙ | ︙ |
Changes to ext/fts3/unicode/mkunicode.tcl.
︙ | ︙ | |||
59 60 61 62 63 64 65 | incr i puts -nonewline [format "%5d" [expr ($iCode<<3) + $nRange-1]] puts -nonewline ", " } puts "" puts " \};" | > | | | | 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 | incr i puts -nonewline [format "%5d" [expr ($iCode<<3) + $nRange-1]] puts -nonewline ", " } puts "" puts " \};" puts "#define HIBIT ((unsigned char)0x80)" puts " unsigned char aChar\[\] = \{" puts -nonewline " '\\0', " set i 1 foreach c $aChar f $aFlag { if { $f } { set str "'$c'|HIBIT, " } else { set str "'$c', " } if {$c == ""} { set str "'\\0', " } if {($i % 6)==0} {puts "" ; puts -nonewline " " } incr i puts -nonewline "$str" } |
︙ | ︙ | |||
130 131 132 133 134 135 136 | puts "*/" puts "int ${zFunc}\(int c)\{" puts " unsigned int mask0 = [format "0x%08X" $i1];" puts " unsigned int mask1 = [format "0x%08X" $i2];" puts " if( c<$iFirst || c>$iLast ) return 0;" puts " return (c < $iFirst+32) ?" | | | | 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 | puts "*/" puts "int ${zFunc}\(int c)\{" puts " unsigned int mask0 = [format "0x%08X" $i1];" puts " unsigned int mask1 = [format "0x%08X" $i2];" puts " if( c<$iFirst || c>$iLast ) return 0;" puts " return (c < $iFirst+32) ?" puts " (mask0 & ((unsigned int)1 << (c-$iFirst))) :" puts " (mask1 & ((unsigned int)1 << (c-$iFirst-32)));" puts "\}" } #------------------------------------------------------------------------- proc an_load_separator_ranges {} { |
︙ | ︙ | |||
695 696 697 698 699 700 701 | set aMapArray [intarray $aMap] set aDataArray [intarray $aData] puts [code { static u16 aFts5UnicodeBlock[] = {$aBlockArray}; static u16 aFts5UnicodeMap[] = {$aMapArray}; static u16 aFts5UnicodeData[] = {$aDataArray}; | | | 696 697 698 699 700 701 702 703 704 705 706 707 708 709 710 | set aMapArray [intarray $aMap] set aDataArray [intarray $aData] puts [code { static u16 aFts5UnicodeBlock[] = {$aBlockArray}; static u16 aFts5UnicodeMap[] = {$aMapArray}; static u16 aFts5UnicodeData[] = {$aDataArray}; int sqlite3Fts5UnicodeCategory(u32 iCode) { int iRes = -1; int iHi; int iLo; int ret; u16 iKey; if( iCode>=(1<<20) ){ |
︙ | ︙ | |||
733 734 735 736 737 738 739 | void sqlite3Fts5UnicodeAscii(u8 *aArray, u8 *aAscii){ int i = 0; int iTbl = 0; while( i<128 ){ int bToken = aArray[ aFts5UnicodeData[iTbl] & 0x1F ]; int n = (aFts5UnicodeData[iTbl] >> 5) + i; for(; i<128 && i<n; i++){ | | > | 734 735 736 737 738 739 740 741 742 743 744 745 746 747 748 749 750 751 752 | void sqlite3Fts5UnicodeAscii(u8 *aArray, u8 *aAscii){ int i = 0; int iTbl = 0; while( i<128 ){ int bToken = aArray[ aFts5UnicodeData[iTbl] & 0x1F ]; int n = (aFts5UnicodeData[iTbl] >> 5) + i; for(; i<128 && i<n; i++){ aAscii[i] = (u8)bToken; } iTbl++; } aAscii[0] = 0; /* 0x00 is never a token character */ } }] } proc print_test_categories {lMap} { set lCP [list] |
︙ | ︙ | |||
778 779 780 781 782 783 784 | if( aCP[iCP].iCode==i ){ sqlite3Fts5UnicodeCatParse(aCP[iCP].zCat, aArray); iCP++; }else{ aArray[0] = 1; } | | | 780 781 782 783 784 785 786 787 788 789 790 791 792 793 794 | if( aCP[iCP].iCode==i ){ sqlite3Fts5UnicodeCatParse(aCP[iCP].zCat, aArray); iCP++; }else{ aArray[0] = 1; } c = sqlite3Fts5UnicodeCategory((u32)i); if( aArray[c]==0 ){ *piCode = i; return 1; } } return 0; |
︙ | ︙ | |||
833 834 835 836 837 838 839 | puts "\}" } proc print_fileheader {} { puts [string trim { /* | | | 835 836 837 838 839 840 841 842 843 844 845 846 847 848 849 | puts "\}" } proc print_fileheader {} { puts [string trim { /* ** 2012-05-25 ** ** The author disclaims copyright to this source code. In place of ** a legal notice, here is a blessing: ** ** May you do good and not evil. ** May you find forgiveness for yourself and forgive others. ** May you share freely, never taking more than you give. |
︙ | ︙ |
Changes to ext/fts5/fts5.h.
︙ | ︙ | |||
116 117 118 119 120 121 122 | ** Query for the details of phrase match iIdx within the current row. ** Phrase matches are numbered starting from zero, so the iIdx argument ** should be greater than or equal to zero and smaller than the value ** output by xInstCount(). ** ** Usually, output parameter *piPhrase is set to the phrase number, *piCol ** to the column in which it occurs and *piOff the token offset of the | < < < < | | | 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 | ** Query for the details of phrase match iIdx within the current row. ** Phrase matches are numbered starting from zero, so the iIdx argument ** should be greater than or equal to zero and smaller than the value ** output by xInstCount(). ** ** Usually, output parameter *piPhrase is set to the phrase number, *piCol ** to the column in which it occurs and *piOff the token offset of the ** first token of the phrase. Returns SQLITE_OK if successful, or an error ** code (i.e. SQLITE_NOMEM) if an error occurs. ** ** This API can be quite slow if used with an FTS5 table created with the ** "detail=none" or "detail=column" option. ** ** xRowid: ** Returns the rowid of the current row. ** |
︙ | ︙ | |||
159 160 161 162 163 164 165 | ** If the query runs to completion without incident, SQLITE_OK is returned. ** Or, if some error occurs before the query completes or is aborted by ** the callback, an SQLite error code is returned. ** ** ** xSetAuxdata(pFts5, pAux, xDelete) ** | | | | | 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 | ** If the query runs to completion without incident, SQLITE_OK is returned. ** Or, if some error occurs before the query completes or is aborted by ** the callback, an SQLite error code is returned. ** ** ** xSetAuxdata(pFts5, pAux, xDelete) ** ** Save the pointer passed as the second argument as the extension function's ** "auxiliary data". The pointer may then be retrieved by the current or any ** future invocation of the same fts5 extension function made as part of ** the same MATCH query using the xGetAuxdata() API. ** ** Each extension function is allocated a single auxiliary data slot for ** each FTS query (MATCH expression). If the extension function is invoked ** more than once for a single FTS query, then all invocations share a ** single auxiliary data context. ** ** If there is already an auxiliary data pointer when this function is ** invoked, then it is replaced by the new pointer. If an xDelete callback ** was specified along with the original pointer, it is invoked at this ** point. ** ** The xDelete callback, if one is specified, is also invoked on the ** auxiliary data pointer after the FTS5 query has finished. ** ** If an error (e.g. an OOM condition) occurs within this function, ** the auxiliary data is set to NULL and an error code returned. If the ** xDelete parameter was not NULL, it is invoked on the auxiliary data ** pointer before returning. ** ** ** xGetAuxdata(pFts5, bClear) ** |
︙ | ︙ | |||
401 402 403 404 405 406 407 | ** of "first place" within the document set, but not alternative forms ** such as "1st place". In some applications, it would be better to match ** all instances of "first place" or "1st place" regardless of which form ** the user specified in the MATCH query text. ** ** There are several ways to approach this in FTS5: ** | | | | 397 398 399 400 401 402 403 404 405 406 407 408 409 410 411 412 | ** of "first place" within the document set, but not alternative forms ** such as "1st place". In some applications, it would be better to match ** all instances of "first place" or "1st place" regardless of which form ** the user specified in the MATCH query text. ** ** There are several ways to approach this in FTS5: ** ** <ol><li> By mapping all synonyms to a single token. In this case, using ** the above example, this means that the tokenizer returns the ** same token for inputs "first" and "1st". Say that token is in ** fact "first", so that when the user inserts the document "I won ** 1st place" entries are added to the index for tokens "i", "won", ** "first" and "place". If the user then queries for '1st + place', ** the tokenizer substitutes "first" for "1st" and the query works ** as expected. ** |
︙ | ︙ |
Changes to ext/fts5/fts5Int.h.
︙ | ︙ | |||
31 32 33 34 35 36 37 | typedef sqlite3_uint64 u64; #ifndef ArraySize # define ArraySize(x) ((int)(sizeof(x) / sizeof(x[0]))) #endif #define testcase(x) | > > > > > | | > > > > > > > | 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 | typedef sqlite3_uint64 u64; #ifndef ArraySize # define ArraySize(x) ((int)(sizeof(x) / sizeof(x[0]))) #endif #define testcase(x) #if defined(SQLITE_COVERAGE_TEST) || defined(SQLITE_MUTATION_TEST) # define SQLITE_OMIT_AUXILIARY_SAFETY_CHECKS 1 #endif #if defined(SQLITE_OMIT_AUXILIARY_SAFETY_CHECKS) # define ALWAYS(X) (1) # define NEVER(X) (0) #elif !defined(NDEBUG) # define ALWAYS(X) ((X)?1:(assert(0),0)) # define NEVER(X) ((X)?(assert(0),1):0) #else # define ALWAYS(X) (X) # define NEVER(X) (X) #endif #define MIN(x,y) (((x) < (y)) ? (x) : (y)) #define MAX(x,y) (((x) > (y)) ? (x) : (y)) /* ** Constants for the largest and smallest possible 64-bit signed integers. */ |
︙ | ︙ | |||
57 58 59 60 61 62 63 64 65 66 67 68 69 70 | /* ** Maximum number of prefix indexes on single FTS5 table. This must be ** less than 32. If it is set to anything large than that, an #error ** directive in fts5_index.c will cause the build to fail. */ #define FTS5_MAX_PREFIX_INDEXES 31 #define FTS5_DEFAULT_NEARDIST 10 #define FTS5_DEFAULT_RANK "bm25" /* Name of rank and rowid columns */ #define FTS5_RANK_NAME "rank" #define FTS5_ROWID_NAME "rowid" | > > > > > | 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 | /* ** Maximum number of prefix indexes on single FTS5 table. This must be ** less than 32. If it is set to anything large than that, an #error ** directive in fts5_index.c will cause the build to fail. */ #define FTS5_MAX_PREFIX_INDEXES 31 /* ** Maximum segments permitted in a single index */ #define FTS5_MAX_SEGMENT 2000 #define FTS5_DEFAULT_NEARDIST 10 #define FTS5_DEFAULT_RANK "bm25" /* Name of rank and rowid columns */ #define FTS5_RANK_NAME "rank" #define FTS5_ROWID_NAME "rowid" |
︙ | ︙ | |||
83 84 85 86 87 88 89 90 91 92 93 94 95 96 | #ifdef SQLITE_DEBUG extern int sqlite3_fts5_may_be_corrupt; # define assert_nc(x) assert(sqlite3_fts5_may_be_corrupt || (x)) #else # define assert_nc(x) assert(x) #endif /* Mark a function parameter as unused, to suppress nuisance compiler ** warnings. */ #ifndef UNUSED_PARAM # define UNUSED_PARAM(X) (void)(X) #endif #ifndef UNUSED_PARAM2 | > > > > > > | 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 | #ifdef SQLITE_DEBUG extern int sqlite3_fts5_may_be_corrupt; # define assert_nc(x) assert(sqlite3_fts5_may_be_corrupt || (x)) #else # define assert_nc(x) assert(x) #endif /* ** A version of memcmp() that does not cause asan errors if one of the pointer ** parameters is NULL and the number of bytes to compare is zero. */ #define fts5Memcmp(s1, s2, n) ((n)<=0 ? 0 : memcmp((s1), (s2), (n))) /* Mark a function parameter as unused, to suppress nuisance compiler ** warnings. */ #ifndef UNUSED_PARAM # define UNUSED_PARAM(X) (void)(X) #endif #ifndef UNUSED_PARAM2 |
︙ | ︙ | |||
168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 | char *zContent; /* content table */ char *zContentRowid; /* "content_rowid=" option value */ int bColumnsize; /* "columnsize=" option value (dflt==1) */ int eDetail; /* FTS5_DETAIL_XXX value */ char *zContentExprlist; Fts5Tokenizer *pTok; fts5_tokenizer *pTokApi; /* Values loaded from the %_config table */ int iCookie; /* Incremented when %_config is modified */ int pgsz; /* Approximate page size used in %_data */ int nAutomerge; /* 'automerge' setting */ int nCrisisMerge; /* Maximum allowed segments per level */ int nUsermerge; /* 'usermerge' setting */ int nHashSize; /* Bytes of memory for in-memory hash */ char *zRank; /* Name of rank function */ char *zRankArgs; /* Arguments to rank function */ /* If non-NULL, points to sqlite3_vtab.base.zErrmsg. Often NULL. */ char **pzErrmsg; #ifdef SQLITE_DEBUG int bPrefixIndex; /* True to use prefix-indexes */ #endif }; /* Current expected value of %_config table 'version' field */ | > > | | | | | > > | 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 | char *zContent; /* content table */ char *zContentRowid; /* "content_rowid=" option value */ int bColumnsize; /* "columnsize=" option value (dflt==1) */ int eDetail; /* FTS5_DETAIL_XXX value */ char *zContentExprlist; Fts5Tokenizer *pTok; fts5_tokenizer *pTokApi; int bLock; /* True when table is preparing statement */ int ePattern; /* FTS_PATTERN_XXX constant */ /* Values loaded from the %_config table */ int iCookie; /* Incremented when %_config is modified */ int pgsz; /* Approximate page size used in %_data */ int nAutomerge; /* 'automerge' setting */ int nCrisisMerge; /* Maximum allowed segments per level */ int nUsermerge; /* 'usermerge' setting */ int nHashSize; /* Bytes of memory for in-memory hash */ char *zRank; /* Name of rank function */ char *zRankArgs; /* Arguments to rank function */ /* If non-NULL, points to sqlite3_vtab.base.zErrmsg. Often NULL. */ char **pzErrmsg; #ifdef SQLITE_DEBUG int bPrefixIndex; /* True to use prefix-indexes */ #endif }; /* Current expected value of %_config table 'version' field */ #define FTS5_CURRENT_VERSION 4 #define FTS5_CONTENT_NORMAL 0 #define FTS5_CONTENT_NONE 1 #define FTS5_CONTENT_EXTERNAL 2 #define FTS5_DETAIL_FULL 0 #define FTS5_DETAIL_NONE 1 #define FTS5_DETAIL_COLUMNS 2 #define FTS5_PATTERN_NONE 0 #define FTS5_PATTERN_LIKE 65 /* matches SQLITE_INDEX_CONSTRAINT_LIKE */ #define FTS5_PATTERN_GLOB 66 /* matches SQLITE_INDEX_CONSTRAINT_GLOB */ int sqlite3Fts5ConfigParse( Fts5Global*, sqlite3*, int, const char **, Fts5Config**, char** ); void sqlite3Fts5ConfigFree(Fts5Config*); int sqlite3Fts5ConfigDeclareVtab(Fts5Config *pConfig); |
︙ | ︙ | |||
255 256 257 258 259 260 261 | void sqlite3Fts5BufferZero(Fts5Buffer*); void sqlite3Fts5BufferSet(int*, Fts5Buffer*, int, const u8*); void sqlite3Fts5BufferAppendPrintf(int *, Fts5Buffer*, char *zFmt, ...); char *sqlite3Fts5Mprintf(int *pRc, const char *zFmt, ...); #define fts5BufferZero(x) sqlite3Fts5BufferZero(x) | | | | 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 | void sqlite3Fts5BufferZero(Fts5Buffer*); void sqlite3Fts5BufferSet(int*, Fts5Buffer*, int, const u8*); void sqlite3Fts5BufferAppendPrintf(int *, Fts5Buffer*, char *zFmt, ...); char *sqlite3Fts5Mprintf(int *pRc, const char *zFmt, ...); #define fts5BufferZero(x) sqlite3Fts5BufferZero(x) #define fts5BufferAppendVarint(a,b,c) sqlite3Fts5BufferAppendVarint(a,b,(i64)c) #define fts5BufferFree(a) sqlite3Fts5BufferFree(a) #define fts5BufferAppendBlob(a,b,c,d) sqlite3Fts5BufferAppendBlob(a,b,c,d) #define fts5BufferSet(a,b,c,d) sqlite3Fts5BufferSet(a,b,c,d) #define fts5BufferGrow(pRc,pBuf,nn) ( \ (u32)((pBuf)->n) + (u32)(nn) <= (u32)((pBuf)->nSpace) ? 0 : \ sqlite3Fts5BufferSize((pRc),(pBuf),(nn)+(pBuf)->n) \ ) /* Write and decode big-endian 32-bit integer values */ void sqlite3Fts5Put32(u8*, int); int sqlite3Fts5Get32(const u8*); #define FTS5_POS2COLUMN(iPos) (int)(iPos >> 32) #define FTS5_POS2OFFSET(iPos) (int)(iPos & 0x7FFFFFFF) typedef struct Fts5PoslistReader Fts5PoslistReader; struct Fts5PoslistReader { /* Variables used only by sqlite3Fts5PoslistIterXXX() functions. */ const u8 *a; /* Position list to iterate through */ int n; /* Size of buffer at a[] in bytes */ int i; /* Current offset in a[] */ |
︙ | ︙ | |||
305 306 307 308 309 310 311 | int sqlite3Fts5PoslistNext64( const u8 *a, int n, /* Buffer containing poslist */ int *pi, /* IN/OUT: Offset within a[] */ i64 *piOff /* IN/OUT: Current offset */ ); /* Malloc utility */ | | | 332 333 334 335 336 337 338 339 340 341 342 343 344 345 346 | int sqlite3Fts5PoslistNext64( const u8 *a, int n, /* Buffer containing poslist */ int *pi, /* IN/OUT: Offset within a[] */ i64 *piOff /* IN/OUT: Current offset */ ); /* Malloc utility */ void *sqlite3Fts5MallocZero(int *pRc, sqlite3_int64 nByte); char *sqlite3Fts5Strndup(int *pRc, const char *pIn, int nIn); /* Character set tests (like isspace(), isalpha() etc.) */ int sqlite3Fts5IsBareword(char t); /* Bucket of terms object used by the integrity-check in offsets=0 mode. */ |
︙ | ︙ | |||
406 407 408 409 410 411 412 413 414 415 416 417 418 419 420 421 422 423 424 | int sqlite3Fts5IterNextFrom(Fts5IndexIter*, i64 iMatch); /* ** Close an iterator opened by sqlite3Fts5IndexQuery(). */ void sqlite3Fts5IterClose(Fts5IndexIter*); /* ** This interface is used by the fts5vocab module. */ const char *sqlite3Fts5IterTerm(Fts5IndexIter*, int*); int sqlite3Fts5IterNextScan(Fts5IndexIter*); /* ** Insert or remove data to or from the index. Each time a document is ** added to or removed from the index, this function is called one or more ** times. ** | > > > > > > > > | 433 434 435 436 437 438 439 440 441 442 443 444 445 446 447 448 449 450 451 452 453 454 455 456 457 458 459 | int sqlite3Fts5IterNextFrom(Fts5IndexIter*, i64 iMatch); /* ** Close an iterator opened by sqlite3Fts5IndexQuery(). */ void sqlite3Fts5IterClose(Fts5IndexIter*); /* ** Close the reader blob handle, if it is open. */ void sqlite3Fts5IndexCloseReader(Fts5Index*); /* ** This interface is used by the fts5vocab module. */ const char *sqlite3Fts5IterTerm(Fts5IndexIter*, int*); int sqlite3Fts5IterNextScan(Fts5IndexIter*); void *sqlite3Fts5StructureRef(Fts5Index*); void sqlite3Fts5StructureRelease(void*); int sqlite3Fts5StructureTest(Fts5Index*, void*); /* ** Insert or remove data to or from the index. Each time a document is ** added to or removed from the index, this function is called one or more ** times. ** |
︙ | ︙ | |||
463 464 465 466 467 468 469 | */ int sqlite3Fts5IndexGetAverages(Fts5Index *p, i64 *pnRow, i64 *anSize); int sqlite3Fts5IndexSetAverages(Fts5Index *p, const u8*, int); /* ** Functions called by the storage module as part of integrity-check. */ | | | 498 499 500 501 502 503 504 505 506 507 508 509 510 511 512 | */ int sqlite3Fts5IndexGetAverages(Fts5Index *p, i64 *pnRow, i64 *anSize); int sqlite3Fts5IndexSetAverages(Fts5Index *p, const u8*, int); /* ** Functions called by the storage module as part of integrity-check. */ int sqlite3Fts5IndexIntegrityCheck(Fts5Index*, u64 cksum, int bUseCksum); /* ** Called during virtual module initialization to register UDF ** fts5_decode() with SQLite */ int sqlite3Fts5IndexInit(sqlite3*); |
︙ | ︙ | |||
516 517 518 519 520 521 522 | /* ** End of interface to code in fts5_varint.c. **************************************************************************/ /************************************************************************** | | > > > > > > > > > > | < | > > | 551 552 553 554 555 556 557 558 559 560 561 562 563 564 565 566 567 568 569 570 571 572 573 574 575 576 577 578 579 580 581 582 583 584 585 586 587 588 | /* ** End of interface to code in fts5_varint.c. **************************************************************************/ /************************************************************************** ** Interface to code in fts5_main.c. */ /* ** Virtual-table object. */ typedef struct Fts5Table Fts5Table; struct Fts5Table { sqlite3_vtab base; /* Base class used by SQLite core */ Fts5Config *pConfig; /* Virtual table configuration */ Fts5Index *pIndex; /* Full-text index */ }; int sqlite3Fts5GetTokenizer( Fts5Global*, const char **azArg, int nArg, Fts5Config*, char **pzErr ); Fts5Table *sqlite3Fts5TableFromCsrid(Fts5Global*, i64); int sqlite3Fts5FlushToDisk(Fts5Table*); /* ** End of interface to code in fts5.c. **************************************************************************/ /************************************************************************** ** Interface to code in fts5_hash.c. |
︙ | ︙ | |||
561 562 563 564 565 566 567 568 | /* ** Empty (but do not delete) a hash table. */ void sqlite3Fts5HashClear(Fts5Hash*); int sqlite3Fts5HashQuery( Fts5Hash*, /* Hash table to query */ const char *pTerm, int nTerm, /* Query term */ | > | | 607 608 609 610 611 612 613 614 615 616 617 618 619 620 621 622 623 | /* ** Empty (but do not delete) a hash table. */ void sqlite3Fts5HashClear(Fts5Hash*); int sqlite3Fts5HashQuery( Fts5Hash*, /* Hash table to query */ int nPre, const char *pTerm, int nTerm, /* Query term */ void **ppObj, /* OUT: Pointer to doclist for pTerm */ int *pnDoclist /* OUT: Size of doclist in bytes */ ); int sqlite3Fts5HashScanInit( Fts5Hash*, /* Hash table to query */ const char *pTerm, int nTerm /* Query prefix */ ); |
︙ | ︙ | |||
605 606 607 608 609 610 611 | int sqlite3Fts5DropAll(Fts5Config*); int sqlite3Fts5CreateTable(Fts5Config*, const char*, const char*, int, char **); int sqlite3Fts5StorageDelete(Fts5Storage *p, i64, sqlite3_value**); int sqlite3Fts5StorageContentInsert(Fts5Storage *p, sqlite3_value**, i64*); int sqlite3Fts5StorageIndexInsert(Fts5Storage *p, sqlite3_value**, i64); | | | 652 653 654 655 656 657 658 659 660 661 662 663 664 665 666 | int sqlite3Fts5DropAll(Fts5Config*); int sqlite3Fts5CreateTable(Fts5Config*, const char*, const char*, int, char **); int sqlite3Fts5StorageDelete(Fts5Storage *p, i64, sqlite3_value**); int sqlite3Fts5StorageContentInsert(Fts5Storage *p, sqlite3_value**, i64*); int sqlite3Fts5StorageIndexInsert(Fts5Storage *p, sqlite3_value**, i64); int sqlite3Fts5StorageIntegrity(Fts5Storage *p, int iArg); int sqlite3Fts5StorageStmt(Fts5Storage *p, int eStmt, sqlite3_stmt**, char**); void sqlite3Fts5StorageStmtRelease(Fts5Storage *p, int eStmt, sqlite3_stmt*); int sqlite3Fts5StorageDocsize(Fts5Storage *p, i64 iRowid, int *aCol); int sqlite3Fts5StorageSize(Fts5Storage *p, int iCol, i64 *pnAvg); int sqlite3Fts5StorageRowCount(Fts5Storage *p, i64 *pnRow); |
︙ | ︙ | |||
650 651 652 653 654 655 656 657 658 659 660 661 662 663 664 665 666 667 668 669 670 671 672 673 674 675 676 677 678 679 680 681 682 683 684 | const char *p; /* Token text (not NULL terminated) */ int n; /* Size of buffer p in bytes */ }; /* Parse a MATCH expression. */ int sqlite3Fts5ExprNew( Fts5Config *pConfig, int iCol, /* Column on LHS of MATCH operator */ const char *zExpr, Fts5Expr **ppNew, char **pzErr ); /* ** for(rc = sqlite3Fts5ExprFirst(pExpr, pIdx, bDesc); ** rc==SQLITE_OK && 0==sqlite3Fts5ExprEof(pExpr); ** rc = sqlite3Fts5ExprNext(pExpr) ** ){ ** // The document with rowid iRowid matches the expression! ** i64 iRowid = sqlite3Fts5ExprRowid(pExpr); ** } */ int sqlite3Fts5ExprFirst(Fts5Expr*, Fts5Index *pIdx, i64 iMin, int bDesc); int sqlite3Fts5ExprNext(Fts5Expr*, i64 iMax); int sqlite3Fts5ExprEof(Fts5Expr*); i64 sqlite3Fts5ExprRowid(Fts5Expr*); void sqlite3Fts5ExprFree(Fts5Expr*); /* Called during startup to register a UDF with SQLite */ int sqlite3Fts5ExprInit(Fts5Global*, sqlite3*); int sqlite3Fts5ExprPhraseCount(Fts5Expr*); int sqlite3Fts5ExprPhraseSize(Fts5Expr*, int iPhrase); int sqlite3Fts5ExprPoslist(Fts5Expr*, int, const u8 **); | > > > > > > > > > | 697 698 699 700 701 702 703 704 705 706 707 708 709 710 711 712 713 714 715 716 717 718 719 720 721 722 723 724 725 726 727 728 729 730 731 732 733 734 735 736 737 738 739 740 | const char *p; /* Token text (not NULL terminated) */ int n; /* Size of buffer p in bytes */ }; /* Parse a MATCH expression. */ int sqlite3Fts5ExprNew( Fts5Config *pConfig, int bPhraseToAnd, int iCol, /* Column on LHS of MATCH operator */ const char *zExpr, Fts5Expr **ppNew, char **pzErr ); int sqlite3Fts5ExprPattern( Fts5Config *pConfig, int bGlob, int iCol, const char *zText, Fts5Expr **pp ); /* ** for(rc = sqlite3Fts5ExprFirst(pExpr, pIdx, bDesc); ** rc==SQLITE_OK && 0==sqlite3Fts5ExprEof(pExpr); ** rc = sqlite3Fts5ExprNext(pExpr) ** ){ ** // The document with rowid iRowid matches the expression! ** i64 iRowid = sqlite3Fts5ExprRowid(pExpr); ** } */ int sqlite3Fts5ExprFirst(Fts5Expr*, Fts5Index *pIdx, i64 iMin, int bDesc); int sqlite3Fts5ExprNext(Fts5Expr*, i64 iMax); int sqlite3Fts5ExprEof(Fts5Expr*); i64 sqlite3Fts5ExprRowid(Fts5Expr*); void sqlite3Fts5ExprFree(Fts5Expr*); int sqlite3Fts5ExprAnd(Fts5Expr **pp1, Fts5Expr *p2); /* Called during startup to register a UDF with SQLite */ int sqlite3Fts5ExprInit(Fts5Global*, sqlite3*); int sqlite3Fts5ExprPhraseCount(Fts5Expr*); int sqlite3Fts5ExprPhraseSize(Fts5Expr*, int iPhrase); int sqlite3Fts5ExprPoslist(Fts5Expr*, int, const u8 **); |
︙ | ︙ | |||
762 763 764 765 766 767 768 769 770 771 772 773 774 775 | **************************************************************************/ /************************************************************************** ** Interface to code in fts5_tokenizer.c. */ int sqlite3Fts5TokenizerInit(fts5_api*); /* ** End of interface to code in fts5_tokenizer.c. **************************************************************************/ /************************************************************************** ** Interface to code in fts5_vocab.c. */ | > > > > | 818 819 820 821 822 823 824 825 826 827 828 829 830 831 832 833 834 835 | **************************************************************************/ /************************************************************************** ** Interface to code in fts5_tokenizer.c. */ int sqlite3Fts5TokenizerInit(fts5_api*); int sqlite3Fts5TokenizerPattern( int (*xCreate)(void*, const char**, int, Fts5Tokenizer**), Fts5Tokenizer *pTok ); /* ** End of interface to code in fts5_tokenizer.c. **************************************************************************/ /************************************************************************** ** Interface to code in fts5_vocab.c. */ |
︙ | ︙ | |||
784 785 786 787 788 789 790 | /************************************************************************** ** Interface to automatically generated code in fts5_unicode2.c. */ int sqlite3Fts5UnicodeIsdiacritic(int c); int sqlite3Fts5UnicodeFold(int c, int bRemoveDiacritic); int sqlite3Fts5UnicodeCatParse(const char*, u8*); | | | 844 845 846 847 848 849 850 851 852 853 854 855 856 857 | /************************************************************************** ** Interface to automatically generated code in fts5_unicode2.c. */ int sqlite3Fts5UnicodeIsdiacritic(int c); int sqlite3Fts5UnicodeFold(int c, int bRemoveDiacritic); int sqlite3Fts5UnicodeCatParse(const char*, u8*); int sqlite3Fts5UnicodeCategory(u32 iCode); void sqlite3Fts5UnicodeAscii(u8*, u8*); /* ** End of interface to code in fts5_unicode2.c. **************************************************************************/ #endif |
Changes to ext/fts5/fts5_aux.c.
︙ | ︙ | |||
132 133 134 135 136 137 138 | ** *pRc is set to an error code before returning. */ static void fts5HighlightAppend( int *pRc, HighlightContext *p, const char *z, int n ){ | | | 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 | ** *pRc is set to an error code before returning. */ static void fts5HighlightAppend( int *pRc, HighlightContext *p, const char *z, int n ){ if( *pRc==SQLITE_OK && z ){ if( n<0 ) n = (int)strlen(z); p->zOut = sqlite3_mprintf("%z%.*s", p->zOut, n, z); if( p->zOut==0 ) *pRc = SQLITE_NOMEM; } } /* |
︙ | ︙ | |||
264 265 266 267 268 269 270 | ** error occurs. */ static int fts5SentenceFinderAdd(Fts5SFinder *p, int iAdd){ if( p->nFirstAlloc==p->nFirst ){ int nNew = p->nFirstAlloc ? p->nFirstAlloc*2 : 64; int *aNew; | | | 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 | ** error occurs. */ static int fts5SentenceFinderAdd(Fts5SFinder *p, int iAdd){ if( p->nFirstAlloc==p->nFirst ){ int nNew = p->nFirstAlloc ? p->nFirstAlloc*2 : 64; int *aNew; aNew = (int*)sqlite3_realloc64(p->aFirst, nNew*sizeof(int)); if( aNew==0 ) return SQLITE_NOMEM; p->aFirst = aNew; p->nFirstAlloc = nNew; } p->aFirst[p->nFirst++] = iAdd; return SQLITE_OK; } |
︙ | ︙ | |||
331 332 333 334 335 336 337 338 339 340 341 | int ip = 0; int ic = 0; int iOff = 0; int iFirst = -1; int nInst; int nScore = 0; int iLast = 0; rc = pApi->xInstCount(pFts, &nInst); for(i=0; i<nInst && rc==SQLITE_OK; i++){ rc = pApi->xInst(pFts, i, &ip, &ic, &iOff); | > | | | | 331 332 333 334 335 336 337 338 339 340 341 342 343 344 345 346 347 348 349 350 351 352 353 354 355 356 357 358 359 360 361 362 363 | int ip = 0; int ic = 0; int iOff = 0; int iFirst = -1; int nInst; int nScore = 0; int iLast = 0; sqlite3_int64 iEnd = (sqlite3_int64)iPos + nToken; rc = pApi->xInstCount(pFts, &nInst); for(i=0; i<nInst && rc==SQLITE_OK; i++){ rc = pApi->xInst(pFts, i, &ip, &ic, &iOff); if( rc==SQLITE_OK && ic==iCol && iOff>=iPos && iOff<iEnd ){ nScore += (aSeen[ip] ? 1 : 1000); aSeen[ip] = 1; if( iFirst<0 ) iFirst = iOff; iLast = iOff + pApi->xPhraseSize(pFts, ip); } } *pnScore = nScore; if( piPos ){ sqlite3_int64 iAdj = iFirst - (nToken - (iLast-iFirst)) / 2; if( (iAdj+nToken)>nDocsize ) iAdj = nDocsize - nToken; if( iAdj<0 ) iAdj = 0; *piPos = (int)iAdj; } return rc; } /* ** Return the value in pVal interpreted as utf-8 text. Except, if pVal |
︙ | ︙ | |||
438 439 440 441 442 443 444 | for(ii=0; rc==SQLITE_OK && ii<nInst; ii++){ int ip, ic, io; int iAdj; int nScore; int jj; rc = pApi->xInst(pFts, ii, &ip, &ic, &io); | > > | | 439 440 441 442 443 444 445 446 447 448 449 450 451 452 453 454 455 | for(ii=0; rc==SQLITE_OK && ii<nInst; ii++){ int ip, ic, io; int iAdj; int nScore; int jj; rc = pApi->xInst(pFts, ii, &ip, &ic, &io); if( ic!=i ) continue; if( io>nDocsize ) rc = FTS5_CORRUPT; if( rc!=SQLITE_OK ) continue; memset(aSeen, 0, nPhrase); rc = fts5SnippetScore(pApi, pFts, nDocsize, aSeen, i, io, nToken, &nScore, &iAdj ); if( rc==SQLITE_OK && nScore>nBestScore ){ nBestScore = nScore; iBestCol = i; |
︙ | ︙ | |||
559 560 561 562 563 564 565 | const Fts5ExtensionApi *pApi, Fts5Context *pFts, Fts5Bm25Data **ppData /* OUT: bm25-data object for this query */ ){ int rc = SQLITE_OK; /* Return code */ Fts5Bm25Data *p; /* Object to return */ | | | | | > | 562 563 564 565 566 567 568 569 570 571 572 573 574 575 576 577 578 579 580 581 582 583 584 585 586 587 588 589 590 591 592 593 594 595 596 597 598 599 | const Fts5ExtensionApi *pApi, Fts5Context *pFts, Fts5Bm25Data **ppData /* OUT: bm25-data object for this query */ ){ int rc = SQLITE_OK; /* Return code */ Fts5Bm25Data *p; /* Object to return */ p = (Fts5Bm25Data*)pApi->xGetAuxdata(pFts, 0); if( p==0 ){ int nPhrase; /* Number of phrases in query */ sqlite3_int64 nRow = 0; /* Number of rows in table */ sqlite3_int64 nToken = 0; /* Number of tokens in table */ sqlite3_int64 nByte; /* Bytes of space to allocate */ int i; /* Allocate the Fts5Bm25Data object */ nPhrase = pApi->xPhraseCount(pFts); nByte = sizeof(Fts5Bm25Data) + nPhrase*2*sizeof(double); p = (Fts5Bm25Data*)sqlite3_malloc64(nByte); if( p==0 ){ rc = SQLITE_NOMEM; }else{ memset(p, 0, (size_t)nByte); p->nPhrase = nPhrase; p->aIDF = (double*)&p[1]; p->aFreq = &p->aIDF[nPhrase]; } /* Calculate the average document length for this FTS5 table */ if( rc==SQLITE_OK ) rc = pApi->xRowCount(pFts, &nRow); assert( rc!=SQLITE_OK || nRow>0 ); if( rc==SQLITE_OK ) rc = pApi->xColumnTotalSize(pFts, -1, &nToken); if( rc==SQLITE_OK ) p->avgdl = (double)nToken / (double)nRow; /* Calculate an IDF for each phrase in the query */ for(i=0; rc==SQLITE_OK && i<nPhrase; i++){ sqlite3_int64 nHit = 0; rc = pApi->xQueryPhrase(pFts, i, (void*)&nHit, fts5CountCb); |
︙ | ︙ | |||
632 633 634 635 636 637 638 | Fts5Context *pFts, /* First arg to pass to pApi functions */ sqlite3_context *pCtx, /* Context for returning result/error */ int nVal, /* Number of values in apVal[] array */ sqlite3_value **apVal /* Array of trailing arguments */ ){ const double k1 = 1.2; /* Constant "k1" from BM25 formula */ const double b = 0.75; /* Constant "b" from BM25 formula */ | | | 636 637 638 639 640 641 642 643 644 645 646 647 648 649 650 | Fts5Context *pFts, /* First arg to pass to pApi functions */ sqlite3_context *pCtx, /* Context for returning result/error */ int nVal, /* Number of values in apVal[] array */ sqlite3_value **apVal /* Array of trailing arguments */ ){ const double k1 = 1.2; /* Constant "k1" from BM25 formula */ const double b = 0.75; /* Constant "b" from BM25 formula */ int rc; /* Error code */ double score = 0.0; /* SQL function return value */ Fts5Bm25Data *pData; /* Values allocated/calculated once only */ int i; /* Iterator variable */ int nInst = 0; /* Value returned by xInstCount() */ double D = 0.0; /* Total number of tokens in row */ double *aFreq = 0; /* Array of phrase freq. for current row */ |
︙ | ︙ | |||
664 665 666 667 668 669 670 | /* Figure out the total size of the current row in tokens. */ if( rc==SQLITE_OK ){ int nTok; rc = pApi->xColumnSize(pFts, -1, &nTok); D = (double)nTok; } | | > > | | | | | | < < < < | 668 669 670 671 672 673 674 675 676 677 678 679 680 681 682 683 684 685 686 687 688 689 690 | /* Figure out the total size of the current row in tokens. */ if( rc==SQLITE_OK ){ int nTok; rc = pApi->xColumnSize(pFts, -1, &nTok); D = (double)nTok; } /* Determine and return the BM25 score for the current row. Or, if an ** error has occurred, throw an exception. */ if( rc==SQLITE_OK ){ for(i=0; i<pData->nPhrase; i++){ score += pData->aIDF[i] * ( ( aFreq[i] * (k1 + 1.0) ) / ( aFreq[i] + k1 * (1 - b + b * D / pData->avgdl) ) ); } sqlite3_result_double(pCtx, -1.0 * score); }else{ sqlite3_result_error_code(pCtx, rc); } } int sqlite3Fts5AuxInit(fts5_api *pApi){ |
︙ | ︙ | |||
706 707 708 709 710 711 712 | aBuiltin[i].xFunc, aBuiltin[i].xDestroy ); } return rc; } | < < | 708 709 710 711 712 713 714 | aBuiltin[i].xFunc, aBuiltin[i].xDestroy ); } return rc; } |
Changes to ext/fts5/fts5_buffer.c.
︙ | ︙ | |||
13 14 15 16 17 18 19 | #include "fts5Int.h" int sqlite3Fts5BufferSize(int *pRc, Fts5Buffer *pBuf, u32 nByte){ if( (u32)pBuf->nSpace<nByte ){ | | | | | 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 | #include "fts5Int.h" int sqlite3Fts5BufferSize(int *pRc, Fts5Buffer *pBuf, u32 nByte){ if( (u32)pBuf->nSpace<nByte ){ u64 nNew = pBuf->nSpace ? pBuf->nSpace : 64; u8 *pNew; while( nNew<nByte ){ nNew = nNew * 2; } pNew = sqlite3_realloc64(pBuf->p, nNew); if( pNew==0 ){ *pRc = SQLITE_NOMEM; return 1; }else{ pBuf->nSpace = (int)nNew; pBuf->p = pNew; } } return 0; } |
︙ | ︙ | |||
48 49 50 51 52 53 54 | aBuf[0] = (iVal>>24) & 0x00FF; aBuf[1] = (iVal>>16) & 0x00FF; aBuf[2] = (iVal>> 8) & 0x00FF; aBuf[3] = (iVal>> 0) & 0x00FF; } int sqlite3Fts5Get32(const u8 *aBuf){ | | < | 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 | aBuf[0] = (iVal>>24) & 0x00FF; aBuf[1] = (iVal>>16) & 0x00FF; aBuf[2] = (iVal>> 8) & 0x00FF; aBuf[3] = (iVal>> 0) & 0x00FF; } int sqlite3Fts5Get32(const u8 *aBuf){ return (int)((((u32)aBuf[0])<<24) + (aBuf[1]<<16) + (aBuf[2]<<8) + aBuf[3]); } /* ** Append buffer nData/pData to buffer pBuf. If an OOM error occurs, set ** the error code in p. If an error has already occurred when this function ** is called, it is a no-op. */ void sqlite3Fts5BufferAppendBlob( int *pRc, Fts5Buffer *pBuf, u32 nData, const u8 *pData ){ if( nData ){ if( fts5BufferGrow(pRc, pBuf, nData) ) return; memcpy(&pBuf->p[pBuf->n], pData, nData); pBuf->n += nData; } } |
︙ | ︙ | |||
172 173 174 175 176 177 178 | int i = *pi; if( i>=n ){ /* EOF */ *piOff = -1; return 1; }else{ i64 iOff = *piOff; | | | > > > > > > > > > | | > > > > | 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 | int i = *pi; if( i>=n ){ /* EOF */ *piOff = -1; return 1; }else{ i64 iOff = *piOff; u32 iVal; fts5FastGetVarint32(a, i, iVal); if( iVal<=1 ){ if( iVal==0 ){ *pi = i; return 0; } fts5FastGetVarint32(a, i, iVal); iOff = ((i64)iVal) << 32; assert( iOff>=0 ); fts5FastGetVarint32(a, i, iVal); if( iVal<2 ){ /* This is a corrupt record. So stop parsing it here. */ *piOff = -1; return 1; } *piOff = iOff + ((iVal-2) & 0x7FFFFFFF); }else{ *piOff = (iOff & (i64)0x7FFFFFFF<<32)+((iOff + (iVal-2)) & 0x7FFFFFFF); } *pi = i; assert_nc( *piOff>=iOff ); return 0; } } /* ** Advance the iterator object passed as the only argument. Return true |
︙ | ︙ | |||
219 220 221 222 223 224 225 | ** to iPos before returning. */ void sqlite3Fts5PoslistSafeAppend( Fts5Buffer *pBuf, i64 *piPrev, i64 iPos ){ | > | | | | | | | | > | | | | 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 | ** to iPos before returning. */ void sqlite3Fts5PoslistSafeAppend( Fts5Buffer *pBuf, i64 *piPrev, i64 iPos ){ if( iPos>=*piPrev ){ static const i64 colmask = ((i64)(0x7FFFFFFF)) << 32; if( (iPos & colmask) != (*piPrev & colmask) ){ pBuf->p[pBuf->n++] = 1; pBuf->n += sqlite3Fts5PutVarint(&pBuf->p[pBuf->n], (iPos>>32)); *piPrev = (iPos & colmask); } pBuf->n += sqlite3Fts5PutVarint(&pBuf->p[pBuf->n], (iPos-*piPrev)+2); *piPrev = iPos; } } int sqlite3Fts5PoslistWriterAppend( Fts5Buffer *pBuf, Fts5PoslistWriter *pWriter, i64 iPos ){ int rc = 0; /* Initialized only to suppress erroneous warning from Clang */ if( fts5BufferGrow(&rc, pBuf, 5+5+5) ) return rc; sqlite3Fts5PoslistSafeAppend(pBuf, &pWriter->iPrev, iPos); return SQLITE_OK; } void *sqlite3Fts5MallocZero(int *pRc, sqlite3_int64 nByte){ void *pRet = 0; if( *pRc==SQLITE_OK ){ pRet = sqlite3_malloc64(nByte); if( pRet==0 ){ if( nByte>0 ) *pRc = SQLITE_NOMEM; }else{ memset(pRet, 0, (size_t)nByte); } } return pRet; } /* ** Return a nul-terminated copy of the string indicated by pIn. If nIn |
︙ | ︙ |
Changes to ext/fts5/fts5_config.c.
︙ | ︙ | |||
19 20 21 22 23 24 25 | #define FTS5_DEFAULT_PAGE_SIZE 4050 #define FTS5_DEFAULT_AUTOMERGE 4 #define FTS5_DEFAULT_USERMERGE 4 #define FTS5_DEFAULT_CRISISMERGE 16 #define FTS5_DEFAULT_HASHSIZE (1024*1024) /* Maximum allowed page size */ | | | 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 | #define FTS5_DEFAULT_PAGE_SIZE 4050 #define FTS5_DEFAULT_AUTOMERGE 4 #define FTS5_DEFAULT_USERMERGE 4 #define FTS5_DEFAULT_CRISISMERGE 16 #define FTS5_DEFAULT_HASHSIZE (1024*1024) /* Maximum allowed page size */ #define FTS5_MAX_PAGE_SIZE (64*1024) static int fts5_iswhitespace(char x){ return (x==' '); } static int fts5_isopenquote(char x){ return (x=='"' || x=='\'' || x=='[' || x=='`'); |
︙ | ︙ | |||
146 147 148 149 150 151 152 | int iOut = 0; q = z[0]; /* Set stack variable q to the close-quote character */ assert( q=='[' || q=='\'' || q=='"' || q=='`' ); if( q=='[' ) q = ']'; | | | 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 | int iOut = 0; q = z[0]; /* Set stack variable q to the close-quote character */ assert( q=='[' || q=='\'' || q=='"' || q=='`' ); if( q=='[' ) q = ']'; while( z[iIn] ){ if( z[iIn]==q ){ if( z[iIn+1]!=q ){ /* Character iIn was the close quote. */ iIn++; break; }else{ /* Character iIn and iIn+1 form an escaped quote character. Skip |
︙ | ︙ | |||
291 292 293 294 295 296 297 | } assert( pConfig->nPrefix<=FTS5_MAX_PREFIX_INDEXES ); return rc; } if( sqlite3_strnicmp("tokenize", zCmd, nCmd)==0 ){ const char *p = (const char*)zArg; | | | 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 | } assert( pConfig->nPrefix<=FTS5_MAX_PREFIX_INDEXES ); return rc; } if( sqlite3_strnicmp("tokenize", zCmd, nCmd)==0 ){ const char *p = (const char*)zArg; sqlite3_int64 nArg = strlen(zArg) + 1; char **azArg = sqlite3Fts5MallocZero(&rc, sizeof(char*) * nArg); char *pDel = sqlite3Fts5MallocZero(&rc, nArg * 2); char *pSpace = pDel; if( azArg && pSpace ){ if( pConfig->pTok ){ *pzErr = sqlite3_mprintf("multiple tokenize=... directives"); |
︙ | ︙ | |||
321 322 323 324 325 326 327 | } } if( p==0 ){ *pzErr = sqlite3_mprintf("parse error in tokenize directive"); rc = SQLITE_ERROR; }else{ rc = sqlite3Fts5GetTokenizer(pGlobal, | | | 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 | } } if( p==0 ){ *pzErr = sqlite3_mprintf("parse error in tokenize directive"); rc = SQLITE_ERROR; }else{ rc = sqlite3Fts5GetTokenizer(pGlobal, (const char**)azArg, (int)nArg, pConfig, pzErr ); } } } sqlite3_free(azArg); |
︙ | ︙ | |||
393 394 395 396 397 398 399 | /* ** Allocate an instance of the default tokenizer ("simple") at ** Fts5Config.pTokenizer. Return SQLITE_OK if successful, or an SQLite error ** code if an error occurs. */ static int fts5ConfigDefaultTokenizer(Fts5Global *pGlobal, Fts5Config *pConfig){ assert( pConfig->pTok==0 && pConfig->pTokApi==0 ); | | < < | 393 394 395 396 397 398 399 400 401 402 403 404 405 406 407 | /* ** Allocate an instance of the default tokenizer ("simple") at ** Fts5Config.pTokenizer. Return SQLITE_OK if successful, or an SQLite error ** code if an error occurs. */ static int fts5ConfigDefaultTokenizer(Fts5Global *pGlobal, Fts5Config *pConfig){ assert( pConfig->pTok==0 && pConfig->pTokApi==0 ); return sqlite3Fts5GetTokenizer(pGlobal, 0, 0, pConfig, 0); } /* ** Gobble up the first bareword or quoted word from the input buffer zIn. ** Return a pointer to the character immediately following the last in ** the gobbled word if successful, or a NULL pointer otherwise (failed ** to find close-quote character). |
︙ | ︙ | |||
421 422 423 424 425 426 427 | int *pRc, /* IN/OUT: Error code */ const char *zIn, /* Buffer to gobble string/bareword from */ char **pzOut, /* OUT: malloc'd buffer containing str/bw */ int *pbQuoted /* OUT: Set to true if dequoting required */ ){ const char *zRet = 0; | | | | | 419 420 421 422 423 424 425 426 427 428 429 430 431 432 433 434 435 436 437 438 439 440 441 442 443 | int *pRc, /* IN/OUT: Error code */ const char *zIn, /* Buffer to gobble string/bareword from */ char **pzOut, /* OUT: malloc'd buffer containing str/bw */ int *pbQuoted /* OUT: Set to true if dequoting required */ ){ const char *zRet = 0; sqlite3_int64 nIn = strlen(zIn); char *zOut = sqlite3_malloc64(nIn+1); assert( *pRc==SQLITE_OK ); *pbQuoted = 0; *pzOut = 0; if( zOut==0 ){ *pRc = SQLITE_NOMEM; }else{ memcpy(zOut, zIn, (size_t)(nIn+1)); if( fts5_isopenquote(zOut[0]) ){ int ii = fts5Dequote(zOut); zRet = &zIn[ii]; *pbQuoted = 1; }else{ zRet = fts5ConfigSkipBareword(zIn); if( zRet ){ |
︙ | ︙ | |||
525 526 527 528 529 530 531 | const char **azArg, /* Array of nArg CREATE VIRTUAL TABLE args */ Fts5Config **ppOut, /* OUT: Results of parse */ char **pzErr /* OUT: Error message */ ){ int rc = SQLITE_OK; /* Return code */ Fts5Config *pRet; /* New object to return */ int i; | | | | 523 524 525 526 527 528 529 530 531 532 533 534 535 536 537 538 539 540 541 542 543 544 545 546 547 | const char **azArg, /* Array of nArg CREATE VIRTUAL TABLE args */ Fts5Config **ppOut, /* OUT: Results of parse */ char **pzErr /* OUT: Error message */ ){ int rc = SQLITE_OK; /* Return code */ Fts5Config *pRet; /* New object to return */ int i; sqlite3_int64 nByte; *ppOut = pRet = (Fts5Config*)sqlite3_malloc(sizeof(Fts5Config)); if( pRet==0 ) return SQLITE_NOMEM; memset(pRet, 0, sizeof(Fts5Config)); pRet->db = db; pRet->iCookie = -1; nByte = nArg * (sizeof(char*) + sizeof(u8)); pRet->azCol = (char**)sqlite3Fts5MallocZero(&rc, nByte); pRet->abUnindexed = pRet->azCol ? (u8*)&pRet->azCol[nArg] : 0; pRet->zDb = sqlite3Fts5Strndup(&rc, azArg[1], -1); pRet->zName = sqlite3Fts5Strndup(&rc, azArg[2], -1); pRet->bColumnsize = 1; pRet->eDetail = FTS5_DETAIL_FULL; #ifdef SQLITE_DEBUG pRet->bPrefixIndex = 1; #endif |
︙ | ︙ | |||
560 561 562 563 564 565 566 567 568 569 570 571 572 573 574 575 576 577 578 579 580 581 582 | int bOption = 0; int bMustBeCol = 0; z = fts5ConfigGobbleWord(&rc, zOrig, &zOne, &bMustBeCol); z = fts5ConfigSkipWhitespace(z); if( z && *z=='=' ){ bOption = 1; z++; if( bMustBeCol ) z = 0; } z = fts5ConfigSkipWhitespace(z); if( z && z[0] ){ int bDummy; z = fts5ConfigGobbleWord(&rc, z, &zTwo, &bDummy); if( z && z[0] ) z = 0; } if( rc==SQLITE_OK ){ if( z==0 ){ *pzErr = sqlite3_mprintf("parse error in \"%s\"", zOrig); rc = SQLITE_ERROR; }else{ if( bOption ){ | > | > > > > | 558 559 560 561 562 563 564 565 566 567 568 569 570 571 572 573 574 575 576 577 578 579 580 581 582 583 584 585 586 587 588 589 590 591 592 593 | int bOption = 0; int bMustBeCol = 0; z = fts5ConfigGobbleWord(&rc, zOrig, &zOne, &bMustBeCol); z = fts5ConfigSkipWhitespace(z); if( z && *z=='=' ){ bOption = 1; assert( zOne!=0 ); z++; if( bMustBeCol ) z = 0; } z = fts5ConfigSkipWhitespace(z); if( z && z[0] ){ int bDummy; z = fts5ConfigGobbleWord(&rc, z, &zTwo, &bDummy); if( z && z[0] ) z = 0; } if( rc==SQLITE_OK ){ if( z==0 ){ *pzErr = sqlite3_mprintf("parse error in \"%s\"", zOrig); rc = SQLITE_ERROR; }else{ if( bOption ){ rc = fts5ConfigParseSpecial(pGlobal, pRet, ALWAYS(zOne)?zOne:"", zTwo?zTwo:"", pzErr ); }else{ rc = fts5ConfigParseColumn(pRet, zOne, zTwo, pzErr); zOne = 0; } } } |
︙ | ︙ | |||
679 680 681 682 683 684 685 | ); assert( zSql || rc==SQLITE_NOMEM ); if( zSql ){ rc = sqlite3_declare_vtab(pConfig->db, zSql); sqlite3_free(zSql); } | | | 682 683 684 685 686 687 688 689 690 691 692 693 694 695 696 | ); assert( zSql || rc==SQLITE_NOMEM ); if( zSql ){ rc = sqlite3_declare_vtab(pConfig->db, zSql); sqlite3_free(zSql); } return rc; } /* ** Tokenize the text passed via the second and third arguments. ** ** The callback is invoked once for each token in the input text. The |
︙ | ︙ | |||
824 825 826 827 828 829 830 | int rc = SQLITE_OK; if( 0==sqlite3_stricmp(zKey, "pgsz") ){ int pgsz = 0; if( SQLITE_INTEGER==sqlite3_value_numeric_type(pVal) ){ pgsz = sqlite3_value_int(pVal); } | | | 827 828 829 830 831 832 833 834 835 836 837 838 839 840 841 | int rc = SQLITE_OK; if( 0==sqlite3_stricmp(zKey, "pgsz") ){ int pgsz = 0; if( SQLITE_INTEGER==sqlite3_value_numeric_type(pVal) ){ pgsz = sqlite3_value_int(pVal); } if( pgsz<32 || pgsz>FTS5_MAX_PAGE_SIZE ){ *pbBadkey = 1; }else{ pConfig->pgsz = pgsz; } } else if( 0==sqlite3_stricmp(zKey, "hashsize") ){ |
︙ | ︙ | |||
877 878 879 880 881 882 883 884 885 886 887 888 889 890 | if( SQLITE_INTEGER==sqlite3_value_numeric_type(pVal) ){ nCrisisMerge = sqlite3_value_int(pVal); } if( nCrisisMerge<0 ){ *pbBadkey = 1; }else{ if( nCrisisMerge<=1 ) nCrisisMerge = FTS5_DEFAULT_CRISISMERGE; pConfig->nCrisisMerge = nCrisisMerge; } } else if( 0==sqlite3_stricmp(zKey, "rank") ){ const char *zIn = (const char*)sqlite3_value_text(pVal); char *zRank; | > | 880 881 882 883 884 885 886 887 888 889 890 891 892 893 894 | if( SQLITE_INTEGER==sqlite3_value_numeric_type(pVal) ){ nCrisisMerge = sqlite3_value_int(pVal); } if( nCrisisMerge<0 ){ *pbBadkey = 1; }else{ if( nCrisisMerge<=1 ) nCrisisMerge = FTS5_DEFAULT_CRISISMERGE; if( nCrisisMerge>=FTS5_MAX_SEGMENT ) nCrisisMerge = FTS5_MAX_SEGMENT-1; pConfig->nCrisisMerge = nCrisisMerge; } } else if( 0==sqlite3_stricmp(zKey, "rank") ){ const char *zIn = (const char*)sqlite3_value_text(pVal); char *zRank; |
︙ | ︙ |
Changes to ext/fts5/fts5_expr.c.
︙ | ︙ | |||
124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 | struct Fts5Parse { Fts5Config *pConfig; char *zErr; int rc; int nPhrase; /* Size of apPhrase array */ Fts5ExprPhrase **apPhrase; /* Array of all phrases */ Fts5ExprNode *pExpr; /* Result of a successful parse */ }; void sqlite3Fts5ParseError(Fts5Parse *pParse, const char *zFmt, ...){ va_list ap; va_start(ap, zFmt); if( pParse->rc==SQLITE_OK ){ pParse->zErr = sqlite3_vmprintf(zFmt, ap); pParse->rc = SQLITE_ERROR; } va_end(ap); } static int fts5ExprIsspace(char t){ | > > | 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 | struct Fts5Parse { Fts5Config *pConfig; char *zErr; int rc; int nPhrase; /* Size of apPhrase array */ Fts5ExprPhrase **apPhrase; /* Array of all phrases */ Fts5ExprNode *pExpr; /* Result of a successful parse */ int bPhraseToAnd; /* Convert "a+b" to "a AND b" */ }; void sqlite3Fts5ParseError(Fts5Parse *pParse, const char *zFmt, ...){ va_list ap; va_start(ap, zFmt); if( pParse->rc==SQLITE_OK ){ assert( pParse->zErr==0 ); pParse->zErr = sqlite3_vmprintf(zFmt, ap); pParse->rc = SQLITE_ERROR; } va_end(ap); } static int fts5ExprIsspace(char t){ |
︙ | ︙ | |||
207 208 209 210 211 212 213 | } } *pz = &pToken->p[pToken->n]; return tok; } | | > > | 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 | } } *pz = &pToken->p[pToken->n]; return tok; } static void *fts5ParseAlloc(u64 t){ return sqlite3_malloc64((sqlite3_int64)t);} static void fts5ParseFree(void *p){ sqlite3_free(p); } int sqlite3Fts5ExprNew( Fts5Config *pConfig, /* FTS5 Configuration */ int bPhraseToAnd, int iCol, const char *zExpr, /* Expression text */ Fts5Expr **ppNew, char **pzErr ){ Fts5Parse sParse; Fts5Token token; const char *z = zExpr; int t; /* Next token type */ void *pEngine; Fts5Expr *pNew; *ppNew = 0; *pzErr = 0; memset(&sParse, 0, sizeof(sParse)); sParse.bPhraseToAnd = bPhraseToAnd; pEngine = sqlite3Fts5ParserAlloc(fts5ParseAlloc); if( pEngine==0 ){ return SQLITE_NOMEM; } sParse.pConfig = pConfig; do { t = fts5ExprGetToken(&sParse, &z, &token); sqlite3Fts5Parser(pEngine, t, token, &sParse); |
︙ | ︙ | |||
269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 | }else{ pNew->pRoot = sParse.pExpr; } pNew->pIndex = 0; pNew->pConfig = pConfig; pNew->apExprPhrase = sParse.apPhrase; pNew->nPhrase = sParse.nPhrase; sParse.apPhrase = 0; } }else{ sqlite3Fts5ParseNodeFree(sParse.pExpr); } sqlite3_free(sParse.apPhrase); *pzErr = sParse.zErr; return sParse.rc; } /* ** Free the expression node object passed as the only argument. */ void sqlite3Fts5ParseNodeFree(Fts5ExprNode *p){ if( p ){ int i; | > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340 341 342 343 344 345 346 347 348 349 350 351 352 353 354 355 356 357 358 359 360 361 362 363 364 365 366 367 368 369 370 371 372 | }else{ pNew->pRoot = sParse.pExpr; } pNew->pIndex = 0; pNew->pConfig = pConfig; pNew->apExprPhrase = sParse.apPhrase; pNew->nPhrase = sParse.nPhrase; pNew->bDesc = 0; sParse.apPhrase = 0; } }else{ sqlite3Fts5ParseNodeFree(sParse.pExpr); } sqlite3_free(sParse.apPhrase); *pzErr = sParse.zErr; return sParse.rc; } /* ** This function is only called when using the special 'trigram' tokenizer. ** Argument zText contains the text of a LIKE or GLOB pattern matched ** against column iCol. This function creates and compiles an FTS5 MATCH ** expression that will match a superset of the rows matched by the LIKE or ** GLOB. If successful, SQLITE_OK is returned. Otherwise, an SQLite error ** code. */ int sqlite3Fts5ExprPattern( Fts5Config *pConfig, int bGlob, int iCol, const char *zText, Fts5Expr **pp ){ i64 nText = strlen(zText); char *zExpr = (char*)sqlite3_malloc64(nText*4 + 1); int rc = SQLITE_OK; if( zExpr==0 ){ rc = SQLITE_NOMEM; }else{ char aSpec[3]; int iOut = 0; int i = 0; int iFirst = 0; if( bGlob==0 ){ aSpec[0] = '_'; aSpec[1] = '%'; aSpec[2] = 0; }else{ aSpec[0] = '*'; aSpec[1] = '?'; aSpec[2] = '['; } while( i<=nText ){ if( i==nText || zText[i]==aSpec[0] || zText[i]==aSpec[1] || zText[i]==aSpec[2] ){ if( i-iFirst>=3 ){ int jj; zExpr[iOut++] = '"'; for(jj=iFirst; jj<i; jj++){ zExpr[iOut++] = zText[jj]; if( zText[jj]=='"' ) zExpr[iOut++] = '"'; } zExpr[iOut++] = '"'; zExpr[iOut++] = ' '; } if( zText[i]==aSpec[2] ){ i += 2; if( zText[i-1]=='^' ) i++; while( i<nText && zText[i]!=']' ) i++; } iFirst = i+1; } i++; } if( iOut>0 ){ int bAnd = 0; if( pConfig->eDetail!=FTS5_DETAIL_FULL ){ bAnd = 1; if( pConfig->eDetail==FTS5_DETAIL_NONE ){ iCol = pConfig->nCol; } } zExpr[iOut] = '\0'; rc = sqlite3Fts5ExprNew(pConfig, bAnd, iCol, zExpr, pp,pConfig->pzErrmsg); }else{ *pp = 0; } sqlite3_free(zExpr); } return rc; } /* ** Free the expression node object passed as the only argument. */ void sqlite3Fts5ParseNodeFree(Fts5ExprNode *p){ if( p ){ int i; |
︙ | ︙ | |||
304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 | void sqlite3Fts5ExprFree(Fts5Expr *p){ if( p ){ sqlite3Fts5ParseNodeFree(p->pRoot); sqlite3_free(p->apExprPhrase); sqlite3_free(p); } } /* ** Argument pTerm must be a synonym iterator. Return the current rowid ** that it points to. */ static i64 fts5ExprSynonymRowid(Fts5ExprTerm *pTerm, int bDesc, int *pbEof){ i64 iRet = 0; int bRetValid = 0; Fts5ExprTerm *p; assert( pTerm->pSynonym ); assert( bDesc==0 || bDesc==1 ); for(p=pTerm; p; p=p->pSynonym){ if( 0==sqlite3Fts5IterEof(p->pIter) ){ i64 iRowid = p->pIter->iRowid; if( bRetValid==0 || (bDesc!=(iRowid<iRet)) ){ iRet = iRowid; | > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | 384 385 386 387 388 389 390 391 392 393 394 395 396 397 398 399 400 401 402 403 404 405 406 407 408 409 410 411 412 413 414 415 416 417 418 419 420 421 422 423 424 425 426 427 428 429 430 431 432 433 434 435 436 437 438 439 440 441 442 443 444 | void sqlite3Fts5ExprFree(Fts5Expr *p){ if( p ){ sqlite3Fts5ParseNodeFree(p->pRoot); sqlite3_free(p->apExprPhrase); sqlite3_free(p); } } int sqlite3Fts5ExprAnd(Fts5Expr **pp1, Fts5Expr *p2){ Fts5Parse sParse; memset(&sParse, 0, sizeof(sParse)); if( *pp1 ){ Fts5Expr *p1 = *pp1; int nPhrase = p1->nPhrase + p2->nPhrase; p1->pRoot = sqlite3Fts5ParseNode(&sParse, FTS5_AND, p1->pRoot, p2->pRoot,0); p2->pRoot = 0; if( sParse.rc==SQLITE_OK ){ Fts5ExprPhrase **ap = (Fts5ExprPhrase**)sqlite3_realloc( p1->apExprPhrase, nPhrase * sizeof(Fts5ExprPhrase*) ); if( ap==0 ){ sParse.rc = SQLITE_NOMEM; }else{ int i; memmove(&ap[p2->nPhrase], ap, p1->nPhrase*sizeof(Fts5ExprPhrase*)); for(i=0; i<p2->nPhrase; i++){ ap[i] = p2->apExprPhrase[i]; } p1->nPhrase = nPhrase; p1->apExprPhrase = ap; } } sqlite3_free(p2->apExprPhrase); sqlite3_free(p2); }else{ *pp1 = p2; } return sParse.rc; } /* ** Argument pTerm must be a synonym iterator. Return the current rowid ** that it points to. */ static i64 fts5ExprSynonymRowid(Fts5ExprTerm *pTerm, int bDesc, int *pbEof){ i64 iRet = 0; int bRetValid = 0; Fts5ExprTerm *p; assert( pTerm ); assert( pTerm->pSynonym ); assert( bDesc==0 || bDesc==1 ); for(p=pTerm; p; p=p->pSynonym){ if( 0==sqlite3Fts5IterEof(p->pIter) ){ i64 iRowid = p->pIter->iRowid; if( bRetValid==0 || (bDesc!=(iRowid<iRet)) ){ iRet = iRowid; |
︙ | ︙ | |||
352 353 354 355 356 357 358 | assert( pTerm->pSynonym ); for(p=pTerm; p; p=p->pSynonym){ Fts5IndexIter *pIter = p->pIter; if( sqlite3Fts5IterEof(pIter)==0 && pIter->iRowid==iRowid ){ if( pIter->nData==0 ) continue; if( nIter==nAlloc ){ | | | | 469 470 471 472 473 474 475 476 477 478 479 480 481 482 483 484 | assert( pTerm->pSynonym ); for(p=pTerm; p; p=p->pSynonym){ Fts5IndexIter *pIter = p->pIter; if( sqlite3Fts5IterEof(pIter)==0 && pIter->iRowid==iRowid ){ if( pIter->nData==0 ) continue; if( nIter==nAlloc ){ sqlite3_int64 nByte = sizeof(Fts5PoslistReader) * nAlloc * 2; Fts5PoslistReader *aNew = (Fts5PoslistReader*)sqlite3_malloc64(nByte); if( aNew==0 ){ rc = SQLITE_NOMEM; goto synonym_poslist_out; } memcpy(aNew, aIter, sizeof(Fts5PoslistReader) * nIter); nAlloc = nAlloc*2; if( aIter!=aStatic ) sqlite3_free(aIter); |
︙ | ︙ | |||
433 434 435 436 437 438 439 | int bFirst = pPhrase->aTerm[0].bFirst; fts5BufferZero(&pPhrase->poslist); /* If the aStatic[] array is not large enough, allocate a large array ** using sqlite3_malloc(). This approach could be improved upon. */ if( pPhrase->nTerm>ArraySize(aStatic) ){ | | | | 550 551 552 553 554 555 556 557 558 559 560 561 562 563 564 565 | int bFirst = pPhrase->aTerm[0].bFirst; fts5BufferZero(&pPhrase->poslist); /* If the aStatic[] array is not large enough, allocate a large array ** using sqlite3_malloc(). This approach could be improved upon. */ if( pPhrase->nTerm>ArraySize(aStatic) ){ sqlite3_int64 nByte = sizeof(Fts5PoslistReader) * pPhrase->nTerm; aIter = (Fts5PoslistReader*)sqlite3_malloc64(nByte); if( !aIter ) return SQLITE_NOMEM; } memset(aIter, 0, sizeof(Fts5PoslistReader) * pPhrase->nTerm); /* Initialize a term iterator for each term in the phrase */ for(i=0; i<pPhrase->nTerm; i++){ Fts5ExprTerm *pTerm = &pPhrase->aTerm[i]; |
︙ | ︙ | |||
568 569 570 571 572 573 574 | int bMatch; assert( pNear->nPhrase>1 ); /* If the aStatic[] array is not large enough, allocate a large array ** using sqlite3_malloc(). This approach could be improved upon. */ if( pNear->nPhrase>ArraySize(aStatic) ){ | | | 685 686 687 688 689 690 691 692 693 694 695 696 697 698 699 | int bMatch; assert( pNear->nPhrase>1 ); /* If the aStatic[] array is not large enough, allocate a large array ** using sqlite3_malloc(). This approach could be improved upon. */ if( pNear->nPhrase>ArraySize(aStatic) ){ sqlite3_int64 nByte = sizeof(Fts5NearTrimmer) * pNear->nPhrase; a = (Fts5NearTrimmer*)sqlite3Fts5MallocZero(&rc, nByte); }else{ memset(aStatic, 0, sizeof(aStatic)); } if( rc!=SQLITE_OK ){ *pRc = rc; return 0; |
︙ | ︙ | |||
1381 1382 1383 1384 1385 1386 1387 | && 0==pRoot->bEof && fts5RowidCmp(p, pRoot->iRowid, iFirst)<0 ){ rc = fts5ExprNodeNext(p, pRoot, 1, iFirst); } /* If the iterator is not at a real match, skip forward until it is. */ | | | | 1498 1499 1500 1501 1502 1503 1504 1505 1506 1507 1508 1509 1510 1511 1512 1513 | && 0==pRoot->bEof && fts5RowidCmp(p, pRoot->iRowid, iFirst)<0 ){ rc = fts5ExprNodeNext(p, pRoot, 1, iFirst); } /* If the iterator is not at a real match, skip forward until it is. */ while( pRoot->bNomatch && rc==SQLITE_OK ){ assert( pRoot->bEof==0 ); rc = fts5ExprNodeNext(p, pRoot, 0, 0); } return rc; } /* ** Move to the next document |
︙ | ︙ | |||
1477 1478 1479 1480 1481 1482 1483 | Fts5ExprNearset *pRet = 0; if( pParse->rc==SQLITE_OK ){ if( pPhrase==0 ){ return pNear; } if( pNear==0 ){ | > | | | | > | > > > | 1594 1595 1596 1597 1598 1599 1600 1601 1602 1603 1604 1605 1606 1607 1608 1609 1610 1611 1612 1613 1614 1615 1616 1617 1618 1619 1620 1621 1622 1623 1624 1625 1626 1627 1628 1629 1630 1631 1632 1633 1634 1635 1636 1637 1638 1639 | Fts5ExprNearset *pRet = 0; if( pParse->rc==SQLITE_OK ){ if( pPhrase==0 ){ return pNear; } if( pNear==0 ){ sqlite3_int64 nByte; nByte = sizeof(Fts5ExprNearset) + SZALLOC * sizeof(Fts5ExprPhrase*); pRet = sqlite3_malloc64(nByte); if( pRet==0 ){ pParse->rc = SQLITE_NOMEM; }else{ memset(pRet, 0, (size_t)nByte); } }else if( (pNear->nPhrase % SZALLOC)==0 ){ int nNew = pNear->nPhrase + SZALLOC; sqlite3_int64 nByte; nByte = sizeof(Fts5ExprNearset) + nNew * sizeof(Fts5ExprPhrase*); pRet = (Fts5ExprNearset*)sqlite3_realloc64(pNear, nByte); if( pRet==0 ){ pParse->rc = SQLITE_NOMEM; } }else{ pRet = pNear; } } if( pRet==0 ){ assert( pParse->rc!=SQLITE_OK ); sqlite3Fts5ParseNearsetFree(pNear); sqlite3Fts5ParsePhraseFree(pPhrase); }else{ if( pRet->nPhrase>0 ){ Fts5ExprPhrase *pLast = pRet->apPhrase[pRet->nPhrase-1]; assert( pParse!=0 ); assert( pParse->apPhrase!=0 ); assert( pParse->nPhrase>=2 ); assert( pLast==pParse->apPhrase[pParse->nPhrase-2] ); if( pPhrase->nTerm==0 ){ fts5ExprPhraseFree(pPhrase); pRet->nPhrase--; pParse->nPhrase--; pPhrase = pLast; }else if( pLast->nTerm==0 ){ |
︙ | ︙ | |||
1552 1553 1554 1555 1556 1557 1558 | /* If an error has already occurred, this is a no-op */ if( pCtx->rc!=SQLITE_OK ) return pCtx->rc; if( nToken>FTS5_MAX_TOKEN_SIZE ) nToken = FTS5_MAX_TOKEN_SIZE; if( pPhrase && pPhrase->nTerm>0 && (tflags & FTS5_TOKEN_COLOCATED) ){ Fts5ExprTerm *pSyn; | | | | | | 1674 1675 1676 1677 1678 1679 1680 1681 1682 1683 1684 1685 1686 1687 1688 1689 1690 1691 1692 1693 1694 1695 1696 1697 1698 1699 1700 1701 1702 1703 1704 1705 | /* If an error has already occurred, this is a no-op */ if( pCtx->rc!=SQLITE_OK ) return pCtx->rc; if( nToken>FTS5_MAX_TOKEN_SIZE ) nToken = FTS5_MAX_TOKEN_SIZE; if( pPhrase && pPhrase->nTerm>0 && (tflags & FTS5_TOKEN_COLOCATED) ){ Fts5ExprTerm *pSyn; sqlite3_int64 nByte = sizeof(Fts5ExprTerm) + sizeof(Fts5Buffer) + nToken+1; pSyn = (Fts5ExprTerm*)sqlite3_malloc64(nByte); if( pSyn==0 ){ rc = SQLITE_NOMEM; }else{ memset(pSyn, 0, (size_t)nByte); pSyn->zTerm = ((char*)pSyn) + sizeof(Fts5ExprTerm) + sizeof(Fts5Buffer); memcpy(pSyn->zTerm, pToken, nToken); pSyn->pSynonym = pPhrase->aTerm[pPhrase->nTerm-1].pSynonym; pPhrase->aTerm[pPhrase->nTerm-1].pSynonym = pSyn; } }else{ Fts5ExprTerm *pTerm; if( pPhrase==0 || (pPhrase->nTerm % SZALLOC)==0 ){ Fts5ExprPhrase *pNew; int nNew = SZALLOC + (pPhrase ? pPhrase->nTerm : 0); pNew = (Fts5ExprPhrase*)sqlite3_realloc64(pPhrase, sizeof(Fts5ExprPhrase) + sizeof(Fts5ExprTerm) * nNew ); if( pNew==0 ){ rc = SQLITE_NOMEM; }else{ if( pPhrase==0 ) memset(pNew, 0, sizeof(Fts5ExprPhrase)); pCtx->pPhrase = pPhrase = pNew; |
︙ | ︙ | |||
1618 1619 1620 1621 1622 1623 1624 1625 1626 1627 1628 1629 1630 1631 | } } void sqlite3Fts5ParseFinished(Fts5Parse *pParse, Fts5ExprNode *p){ assert( pParse->pExpr==0 ); pParse->pExpr = p; } /* ** This function is called by the parser to process a string token. The ** string may or may not be quoted. In any case it is tokenized and a ** phrase object consisting of all tokens returned. */ Fts5ExprPhrase *sqlite3Fts5ParseTerm( | > > > > > > > > > > > > > > | 1740 1741 1742 1743 1744 1745 1746 1747 1748 1749 1750 1751 1752 1753 1754 1755 1756 1757 1758 1759 1760 1761 1762 1763 1764 1765 1766 1767 | } } void sqlite3Fts5ParseFinished(Fts5Parse *pParse, Fts5ExprNode *p){ assert( pParse->pExpr==0 ); pParse->pExpr = p; } static int parseGrowPhraseArray(Fts5Parse *pParse){ if( (pParse->nPhrase % 8)==0 ){ sqlite3_int64 nByte = sizeof(Fts5ExprPhrase*) * (pParse->nPhrase + 8); Fts5ExprPhrase **apNew; apNew = (Fts5ExprPhrase**)sqlite3_realloc64(pParse->apPhrase, nByte); if( apNew==0 ){ pParse->rc = SQLITE_NOMEM; return SQLITE_NOMEM; } pParse->apPhrase = apNew; } return SQLITE_OK; } /* ** This function is called by the parser to process a string token. The ** string may or may not be quoted. In any case it is tokenized and a ** phrase object consisting of all tokens returned. */ Fts5ExprPhrase *sqlite3Fts5ParseTerm( |
︙ | ︙ | |||
1654 1655 1656 1657 1658 1659 1660 | if( rc || (rc = sCtx.rc) ){ pParse->rc = rc; fts5ExprPhraseFree(sCtx.pPhrase); sCtx.pPhrase = 0; }else{ if( pAppend==0 ){ | < < | < < < | | < < | 1790 1791 1792 1793 1794 1795 1796 1797 1798 1799 1800 1801 1802 1803 1804 1805 1806 | if( rc || (rc = sCtx.rc) ){ pParse->rc = rc; fts5ExprPhraseFree(sCtx.pPhrase); sCtx.pPhrase = 0; }else{ if( pAppend==0 ){ if( parseGrowPhraseArray(pParse) ){ fts5ExprPhraseFree(sCtx.pPhrase); return 0; } pParse->nPhrase++; } if( sCtx.pPhrase==0 ){ /* This happens when parsing a token or quoted phrase that contains ** no token characters at all. (e.g ... MATCH '""'). */ |
︙ | ︙ | |||
1712 1713 1714 1715 1716 1717 1718 | if( rc==SQLITE_OK ){ pNew->pRoot->pNear = (Fts5ExprNearset*)sqlite3Fts5MallocZero(&rc, sizeof(Fts5ExprNearset) + sizeof(Fts5ExprPhrase*)); } if( rc==SQLITE_OK ){ Fts5Colset *pColsetOrig = pOrig->pNode->pNear->pColset; if( pColsetOrig ){ | > > | | | | 1841 1842 1843 1844 1845 1846 1847 1848 1849 1850 1851 1852 1853 1854 1855 1856 1857 1858 1859 1860 | if( rc==SQLITE_OK ){ pNew->pRoot->pNear = (Fts5ExprNearset*)sqlite3Fts5MallocZero(&rc, sizeof(Fts5ExprNearset) + sizeof(Fts5ExprPhrase*)); } if( rc==SQLITE_OK ){ Fts5Colset *pColsetOrig = pOrig->pNode->pNear->pColset; if( pColsetOrig ){ sqlite3_int64 nByte; Fts5Colset *pColset; nByte = sizeof(Fts5Colset) + (pColsetOrig->nCol-1) * sizeof(int); pColset = (Fts5Colset*)sqlite3Fts5MallocZero(&rc, nByte); if( pColset ){ memcpy(pColset, pColsetOrig, (size_t)nByte); } pNew->pRoot->pNear->pColset = pColset; } } if( pOrig->nTerm ){ int i; /* Used to iterate through phrase terms */ |
︙ | ︙ | |||
1743 1744 1745 1746 1747 1748 1749 | } }else{ /* This happens when parsing a token or quoted phrase that contains ** no token characters at all. (e.g ... MATCH '""'). */ sCtx.pPhrase = sqlite3Fts5MallocZero(&rc, sizeof(Fts5ExprPhrase)); } | | | 1874 1875 1876 1877 1878 1879 1880 1881 1882 1883 1884 1885 1886 1887 1888 | } }else{ /* This happens when parsing a token or quoted phrase that contains ** no token characters at all. (e.g ... MATCH '""'). */ sCtx.pPhrase = sqlite3Fts5MallocZero(&rc, sizeof(Fts5ExprPhrase)); } if( rc==SQLITE_OK && ALWAYS(sCtx.pPhrase) ){ /* All the allocations succeeded. Put the expression object together. */ pNew->pIndex = pExpr->pIndex; pNew->pConfig = pExpr->pConfig; pNew->nPhrase = 1; pNew->apExprPhrase[0] = sCtx.pPhrase; pNew->pRoot->pNear->apPhrase[0] = sCtx.pPhrase; pNew->pRoot->pNear->nPhrase = 1; |
︙ | ︙ | |||
1833 1834 1835 1836 1837 1838 1839 | ){ int nCol = p ? p->nCol : 0; /* Num. columns already in colset object */ Fts5Colset *pNew; /* New colset object to return */ assert( pParse->rc==SQLITE_OK ); assert( iCol>=0 && iCol<pParse->pConfig->nCol ); | | | 1964 1965 1966 1967 1968 1969 1970 1971 1972 1973 1974 1975 1976 1977 1978 | ){ int nCol = p ? p->nCol : 0; /* Num. columns already in colset object */ Fts5Colset *pNew; /* New colset object to return */ assert( pParse->rc==SQLITE_OK ); assert( iCol>=0 && iCol<pParse->pConfig->nCol ); pNew = sqlite3_realloc64(p, sizeof(Fts5Colset) + sizeof(int)*nCol); if( pNew==0 ){ pParse->rc = SQLITE_NOMEM; }else{ int *aiCol = pNew->aiCol; int i, j; for(i=0; i<nCol; i++){ if( aiCol[i]==iCol ) return pNew; |
︙ | ︙ | |||
1929 1930 1931 1932 1933 1934 1935 | ** Otherwise, a copy of (*pOrig) is made into memory obtained from ** sqlite3Fts5MallocZero() and a pointer to it returned. If the allocation ** fails, (*pRc) is set to SQLITE_NOMEM and NULL is returned. */ static Fts5Colset *fts5CloneColset(int *pRc, Fts5Colset *pOrig){ Fts5Colset *pRet; if( pOrig ){ | | | | 2060 2061 2062 2063 2064 2065 2066 2067 2068 2069 2070 2071 2072 2073 2074 2075 2076 2077 | ** Otherwise, a copy of (*pOrig) is made into memory obtained from ** sqlite3Fts5MallocZero() and a pointer to it returned. If the allocation ** fails, (*pRc) is set to SQLITE_NOMEM and NULL is returned. */ static Fts5Colset *fts5CloneColset(int *pRc, Fts5Colset *pOrig){ Fts5Colset *pRet; if( pOrig ){ sqlite3_int64 nByte = sizeof(Fts5Colset) + (pOrig->nCol-1) * sizeof(int); pRet = (Fts5Colset*)sqlite3Fts5MallocZero(pRc, nByte); if( pRet ){ memcpy(pRet, pOrig, (size_t)nByte); } }else{ pRet = 0; } return pRet; } |
︙ | ︙ | |||
2014 2015 2016 2017 2018 2019 2020 | void sqlite3Fts5ParseSetColset( Fts5Parse *pParse, Fts5ExprNode *pExpr, Fts5Colset *pColset ){ Fts5Colset *pFree = pColset; if( pParse->pConfig->eDetail==FTS5_DETAIL_NONE ){ | < | | | 2145 2146 2147 2148 2149 2150 2151 2152 2153 2154 2155 2156 2157 2158 2159 2160 | void sqlite3Fts5ParseSetColset( Fts5Parse *pParse, Fts5ExprNode *pExpr, Fts5Colset *pColset ){ Fts5Colset *pFree = pColset; if( pParse->pConfig->eDetail==FTS5_DETAIL_NONE ){ sqlite3Fts5ParseError(pParse, "fts5: column queries are not supported (detail=none)" ); }else{ fts5ParseSetColset(pParse, pExpr, pColset, &pFree); } sqlite3_free(pFree); } |
︙ | ︙ | |||
2067 2068 2069 2070 2071 2072 2073 2074 2075 2076 2077 2078 2079 2080 2081 2082 2083 2084 2085 2086 2087 2088 2089 | memcpy(&p->apChild[p->nChild], pSub->apChild, nByte); p->nChild += pSub->nChild; sqlite3_free(pSub); }else{ p->apChild[p->nChild++] = pSub; } } /* ** Allocate and return a new expression object. If anything goes wrong (i.e. ** OOM error), leave an error code in pParse and return NULL. */ Fts5ExprNode *sqlite3Fts5ParseNode( Fts5Parse *pParse, /* Parse context */ int eType, /* FTS5_STRING, AND, OR or NOT */ Fts5ExprNode *pLeft, /* Left hand child expression */ Fts5ExprNode *pRight, /* Right hand child expression */ Fts5ExprNearset *pNear /* For STRING expressions, the near cluster */ ){ Fts5ExprNode *pRet = 0; if( pParse->rc==SQLITE_OK ){ int nChild = 0; /* Number of children of returned node */ | > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | > > > > > > | | | | | | | | | | | | | | | | | | | | | | | | | | | | < < < | | | | | | | | | | | > | 2197 2198 2199 2200 2201 2202 2203 2204 2205 2206 2207 2208 2209 2210 2211 2212 2213 2214 2215 2216 2217 2218 2219 2220 2221 2222 2223 2224 2225 2226 2227 2228 2229 2230 2231 2232 2233 2234 2235 2236 2237 2238 2239 2240 2241 2242 2243 2244 2245 2246 2247 2248 2249 2250 2251 2252 2253 2254 2255 2256 2257 2258 2259 2260 2261 2262 2263 2264 2265 2266 2267 2268 2269 2270 2271 2272 2273 2274 2275 2276 2277 2278 2279 2280 2281 2282 2283 2284 2285 2286 2287 2288 2289 2290 2291 2292 2293 2294 2295 2296 2297 2298 2299 2300 2301 2302 2303 2304 2305 2306 2307 2308 2309 2310 2311 2312 2313 2314 2315 2316 2317 2318 2319 2320 2321 2322 2323 2324 2325 2326 2327 2328 2329 2330 2331 2332 2333 2334 2335 2336 2337 2338 2339 2340 2341 2342 2343 2344 2345 | memcpy(&p->apChild[p->nChild], pSub->apChild, nByte); p->nChild += pSub->nChild; sqlite3_free(pSub); }else{ p->apChild[p->nChild++] = pSub; } } /* ** This function is used when parsing LIKE or GLOB patterns against ** trigram indexes that specify either detail=column or detail=none. ** It converts a phrase: ** ** abc + def + ghi ** ** into an AND tree: ** ** abc AND def AND ghi */ static Fts5ExprNode *fts5ParsePhraseToAnd( Fts5Parse *pParse, Fts5ExprNearset *pNear ){ int nTerm = pNear->apPhrase[0]->nTerm; int ii; int nByte; Fts5ExprNode *pRet; assert( pNear->nPhrase==1 ); assert( pParse->bPhraseToAnd ); nByte = sizeof(Fts5ExprNode) + nTerm*sizeof(Fts5ExprNode*); pRet = (Fts5ExprNode*)sqlite3Fts5MallocZero(&pParse->rc, nByte); if( pRet ){ pRet->eType = FTS5_AND; pRet->nChild = nTerm; fts5ExprAssignXNext(pRet); pParse->nPhrase--; for(ii=0; ii<nTerm; ii++){ Fts5ExprPhrase *pPhrase = (Fts5ExprPhrase*)sqlite3Fts5MallocZero( &pParse->rc, sizeof(Fts5ExprPhrase) ); if( pPhrase ){ if( parseGrowPhraseArray(pParse) ){ fts5ExprPhraseFree(pPhrase); }else{ pParse->apPhrase[pParse->nPhrase++] = pPhrase; pPhrase->nTerm = 1; pPhrase->aTerm[0].zTerm = sqlite3Fts5Strndup( &pParse->rc, pNear->apPhrase[0]->aTerm[ii].zTerm, -1 ); pRet->apChild[ii] = sqlite3Fts5ParseNode(pParse, FTS5_STRING, 0, 0, sqlite3Fts5ParseNearset(pParse, 0, pPhrase) ); } } } if( pParse->rc ){ sqlite3Fts5ParseNodeFree(pRet); pRet = 0; }else{ sqlite3Fts5ParseNearsetFree(pNear); } } return pRet; } /* ** Allocate and return a new expression object. If anything goes wrong (i.e. ** OOM error), leave an error code in pParse and return NULL. */ Fts5ExprNode *sqlite3Fts5ParseNode( Fts5Parse *pParse, /* Parse context */ int eType, /* FTS5_STRING, AND, OR or NOT */ Fts5ExprNode *pLeft, /* Left hand child expression */ Fts5ExprNode *pRight, /* Right hand child expression */ Fts5ExprNearset *pNear /* For STRING expressions, the near cluster */ ){ Fts5ExprNode *pRet = 0; if( pParse->rc==SQLITE_OK ){ int nChild = 0; /* Number of children of returned node */ sqlite3_int64 nByte; /* Bytes of space to allocate for this node */ assert( (eType!=FTS5_STRING && !pNear) || (eType==FTS5_STRING && !pLeft && !pRight) ); if( eType==FTS5_STRING && pNear==0 ) return 0; if( eType!=FTS5_STRING && pLeft==0 ) return pRight; if( eType!=FTS5_STRING && pRight==0 ) return pLeft; if( eType==FTS5_STRING && pParse->bPhraseToAnd && pNear->apPhrase[0]->nTerm>1 ){ pRet = fts5ParsePhraseToAnd(pParse, pNear); }else{ if( eType==FTS5_NOT ){ nChild = 2; }else if( eType==FTS5_AND || eType==FTS5_OR ){ nChild = 2; if( pLeft->eType==eType ) nChild += pLeft->nChild-1; if( pRight->eType==eType ) nChild += pRight->nChild-1; } nByte = sizeof(Fts5ExprNode) + sizeof(Fts5ExprNode*)*(nChild-1); pRet = (Fts5ExprNode*)sqlite3Fts5MallocZero(&pParse->rc, nByte); if( pRet ){ pRet->eType = eType; pRet->pNear = pNear; fts5ExprAssignXNext(pRet); if( eType==FTS5_STRING ){ int iPhrase; for(iPhrase=0; iPhrase<pNear->nPhrase; iPhrase++){ pNear->apPhrase[iPhrase]->pNode = pRet; if( pNear->apPhrase[iPhrase]->nTerm==0 ){ pRet->xNext = 0; pRet->eType = FTS5_EOF; } } if( pParse->pConfig->eDetail!=FTS5_DETAIL_FULL ){ Fts5ExprPhrase *pPhrase = pNear->apPhrase[0]; if( pNear->nPhrase!=1 || pPhrase->nTerm>1 || (pPhrase->nTerm>0 && pPhrase->aTerm[0].bFirst) ){ sqlite3Fts5ParseError(pParse, "fts5: %s queries are not supported (detail!=full)", pNear->nPhrase==1 ? "phrase": "NEAR" ); sqlite3_free(pRet); pRet = 0; } } }else{ fts5ExprAddChildren(pRet, pLeft); fts5ExprAddChildren(pRet, pRight); } } } } if( pRet==0 ){ assert( pParse->rc!=SQLITE_OK ); sqlite3Fts5ParseNodeFree(pLeft); |
︙ | ︙ | |||
2214 2215 2216 2217 2218 2219 2220 2221 | pRet = sqlite3Fts5ParseNode(pParse, FTS5_AND, pLeft, pRight, 0); } } return pRet; } static char *fts5ExprTermPrint(Fts5ExprTerm *pTerm){ | > | | | 2409 2410 2411 2412 2413 2414 2415 2416 2417 2418 2419 2420 2421 2422 2423 2424 2425 2426 2427 2428 2429 2430 2431 2432 2433 | pRet = sqlite3Fts5ParseNode(pParse, FTS5_AND, pLeft, pRight, 0); } } return pRet; } #ifdef SQLITE_TEST static char *fts5ExprTermPrint(Fts5ExprTerm *pTerm){ sqlite3_int64 nByte = 0; Fts5ExprTerm *p; char *zQuoted; /* Determine the maximum amount of space required. */ for(p=pTerm; p; p=p->pSynonym){ nByte += (int)strlen(pTerm->zTerm) * 2 + 3 + 2; } zQuoted = sqlite3_malloc64(nByte); if( zQuoted ){ int i = 0; for(p=pTerm; p; p=p->pSynonym){ char *zIn = p->zTerm; zQuoted[i++] = '"'; while( *zIn ){ |
︙ | ︙ | |||
2357 2358 2359 2360 2361 2362 2363 | }else if( pExpr->eType==FTS5_STRING || pExpr->eType==FTS5_TERM ){ Fts5ExprNearset *pNear = pExpr->pNear; int i; int iTerm; if( pNear->pColset ){ | > | > > > > > > > | > | 2553 2554 2555 2556 2557 2558 2559 2560 2561 2562 2563 2564 2565 2566 2567 2568 2569 2570 2571 2572 2573 2574 2575 2576 2577 | }else if( pExpr->eType==FTS5_STRING || pExpr->eType==FTS5_TERM ){ Fts5ExprNearset *pNear = pExpr->pNear; int i; int iTerm; if( pNear->pColset ){ int ii; Fts5Colset *pColset = pNear->pColset; if( pColset->nCol>1 ) zRet = fts5PrintfAppend(zRet, "{"); for(ii=0; ii<pColset->nCol; ii++){ zRet = fts5PrintfAppend(zRet, "%s%s", pConfig->azCol[pColset->aiCol[ii]], ii==pColset->nCol-1 ? "" : " " ); } if( zRet ){ zRet = fts5PrintfAppend(zRet, "%s : ", pColset->nCol>1 ? "}" : ""); } if( zRet==0 ) return 0; } if( pNear->nPhrase>1 ){ zRet = fts5PrintfAppend(zRet, "NEAR("); if( zRet==0 ) return 0; } |
︙ | ︙ | |||
2463 2464 2465 2466 2467 2468 2469 | if( bTcl && nArg>1 ){ zNearsetCmd = (const char*)sqlite3_value_text(apVal[1]); iArg = 2; } nConfig = 3 + (nArg-iArg); | | | > > | | 2668 2669 2670 2671 2672 2673 2674 2675 2676 2677 2678 2679 2680 2681 2682 2683 2684 2685 2686 2687 2688 2689 2690 2691 2692 2693 2694 2695 2696 2697 2698 2699 2700 | if( bTcl && nArg>1 ){ zNearsetCmd = (const char*)sqlite3_value_text(apVal[1]); iArg = 2; } nConfig = 3 + (nArg-iArg); azConfig = (const char**)sqlite3_malloc64(sizeof(char*) * nConfig); if( azConfig==0 ){ sqlite3_result_error_nomem(pCtx); return; } azConfig[0] = 0; azConfig[1] = "main"; azConfig[2] = "tbl"; for(i=3; iArg<nArg; iArg++){ const char *z = (const char*)sqlite3_value_text(apVal[iArg]); azConfig[i++] = (z ? z : ""); } zExpr = (const char*)sqlite3_value_text(apVal[0]); if( zExpr==0 ) zExpr = ""; rc = sqlite3Fts5ConfigParse(pGlobal, db, nConfig, azConfig, &pConfig, &zErr); if( rc==SQLITE_OK ){ rc = sqlite3Fts5ExprNew(pConfig, 0, pConfig->nCol, zExpr, &pExpr, &zErr); } if( rc==SQLITE_OK ){ char *zText; if( pExpr->pRoot->xNext==0 ){ zText = sqlite3_mprintf(""); }else if( bTcl ){ zText = fts5ExprPrintTcl(pConfig, zNearsetCmd, pExpr->pRoot); |
︙ | ︙ | |||
2549 2550 2551 2552 2553 2554 2555 | return; } memset(aArr, 0, sizeof(aArr)); sqlite3Fts5UnicodeCatParse("L*", aArr); sqlite3Fts5UnicodeCatParse("N*", aArr); sqlite3Fts5UnicodeCatParse("Co", aArr); iCode = sqlite3_value_int(apVal[0]); | | > > > > > > | 2756 2757 2758 2759 2760 2761 2762 2763 2764 2765 2766 2767 2768 2769 2770 2771 2772 2773 2774 2775 2776 2777 2778 2779 2780 2781 2782 2783 2784 2785 2786 2787 2788 2789 2790 2791 2792 2793 2794 2795 2796 2797 2798 2799 2800 2801 2802 2803 2804 2805 2806 2807 2808 2809 2810 2811 2812 2813 2814 2815 2816 2817 2818 | return; } memset(aArr, 0, sizeof(aArr)); sqlite3Fts5UnicodeCatParse("L*", aArr); sqlite3Fts5UnicodeCatParse("N*", aArr); sqlite3Fts5UnicodeCatParse("Co", aArr); iCode = sqlite3_value_int(apVal[0]); sqlite3_result_int(pCtx, aArr[sqlite3Fts5UnicodeCategory((u32)iCode)]); } static void fts5ExprFold( sqlite3_context *pCtx, /* Function call context */ int nArg, /* Number of args */ sqlite3_value **apVal /* Function arguments */ ){ if( nArg!=1 && nArg!=2 ){ sqlite3_result_error(pCtx, "wrong number of arguments to function fts5_fold", -1 ); }else{ int iCode; int bRemoveDiacritics = 0; iCode = sqlite3_value_int(apVal[0]); if( nArg==2 ) bRemoveDiacritics = sqlite3_value_int(apVal[1]); sqlite3_result_int(pCtx, sqlite3Fts5UnicodeFold(iCode, bRemoveDiacritics)); } } #endif /* ifdef SQLITE_TEST */ /* ** This is called during initialization to register the fts5_expr() scalar ** UDF with the SQLite handle passed as the only argument. */ int sqlite3Fts5ExprInit(Fts5Global *pGlobal, sqlite3 *db){ #ifdef SQLITE_TEST struct Fts5ExprFunc { const char *z; void (*x)(sqlite3_context*,int,sqlite3_value**); } aFunc[] = { { "fts5_expr", fts5ExprFunctionHr }, { "fts5_expr_tcl", fts5ExprFunctionTcl }, { "fts5_isalnum", fts5ExprIsAlnum }, { "fts5_fold", fts5ExprFold }, }; int i; int rc = SQLITE_OK; void *pCtx = (void*)pGlobal; for(i=0; rc==SQLITE_OK && i<ArraySize(aFunc); i++){ struct Fts5ExprFunc *p = &aFunc[i]; rc = sqlite3_create_function(db, p->z, -1, SQLITE_UTF8, pCtx, p->x, 0, 0); } #else int rc = SQLITE_OK; UNUSED_PARAM2(pGlobal,db); #endif /* Avoid warnings indicating that sqlite3Fts5ParserTrace() and ** sqlite3Fts5ParserFallback() are unused */ #ifndef NDEBUG (void)sqlite3Fts5ParserTrace; #endif (void)sqlite3Fts5ParserFallback; |
︙ | ︙ | |||
2642 2643 2644 2645 2646 2647 2648 2649 2650 | struct Fts5PoslistPopulator { Fts5PoslistWriter writer; int bOk; /* True if ok to populate */ int bMiss; }; Fts5PoslistPopulator *sqlite3Fts5ExprClearPoslists(Fts5Expr *pExpr, int bLive){ Fts5PoslistPopulator *pRet; | > > > > > > > > > | | | 2855 2856 2857 2858 2859 2860 2861 2862 2863 2864 2865 2866 2867 2868 2869 2870 2871 2872 2873 2874 2875 2876 2877 2878 2879 2880 2881 2882 2883 2884 2885 2886 2887 | struct Fts5PoslistPopulator { Fts5PoslistWriter writer; int bOk; /* True if ok to populate */ int bMiss; }; /* ** Clear the position lists associated with all phrases in the expression ** passed as the first argument. Argument bLive is true if the expression ** might be pointing to a real entry, otherwise it has just been reset. ** ** At present this function is only used for detail=col and detail=none ** fts5 tables. This implies that all phrases must be at most 1 token ** in size, as phrase matches are not supported without detail=full. */ Fts5PoslistPopulator *sqlite3Fts5ExprClearPoslists(Fts5Expr *pExpr, int bLive){ Fts5PoslistPopulator *pRet; pRet = sqlite3_malloc64(sizeof(Fts5PoslistPopulator)*pExpr->nPhrase); if( pRet ){ int i; memset(pRet, 0, sizeof(Fts5PoslistPopulator)*pExpr->nPhrase); for(i=0; i<pExpr->nPhrase; i++){ Fts5Buffer *pBuf = &pExpr->apExprPhrase[i]->poslist; Fts5ExprNode *pNode = pExpr->apExprPhrase[i]->pNode; assert( pExpr->apExprPhrase[i]->nTerm<=1 ); if( bLive && (pBuf->n==0 || pNode->iRowid!=pExpr->pRoot->iRowid || pNode->bEof) ){ pRet[i].bMiss = 1; }else{ pBuf->n = 0; } |
︙ | ︙ | |||
2843 2844 2845 2846 2847 2848 2849 | }else{ *ppCollist = 0; *pnCollist = 0; } return rc; } | < | 3065 3066 3067 3068 3069 3070 3071 | }else{ *ppCollist = 0; *pnCollist = 0; } return rc; } |
Changes to ext/fts5/fts5_hash.c.
︙ | ︙ | |||
86 87 88 89 90 91 92 | int rc = SQLITE_OK; Fts5Hash *pNew; *ppNew = pNew = (Fts5Hash*)sqlite3_malloc(sizeof(Fts5Hash)); if( pNew==0 ){ rc = SQLITE_NOMEM; }else{ | | | | | 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 | int rc = SQLITE_OK; Fts5Hash *pNew; *ppNew = pNew = (Fts5Hash*)sqlite3_malloc(sizeof(Fts5Hash)); if( pNew==0 ){ rc = SQLITE_NOMEM; }else{ sqlite3_int64 nByte; memset(pNew, 0, sizeof(Fts5Hash)); pNew->pnByte = pnByte; pNew->eDetail = pConfig->eDetail; pNew->nSlot = 1024; nByte = sizeof(Fts5HashEntry*) * pNew->nSlot; pNew->aSlot = (Fts5HashEntry**)sqlite3_malloc64(nByte); if( pNew->aSlot==0 ){ sqlite3_free(pNew); *ppNew = 0; rc = SQLITE_NOMEM; }else{ memset(pNew->aSlot, 0, (size_t)nByte); } } return rc; } /* ** Free a hash table object. |
︙ | ︙ | |||
161 162 163 164 165 166 167 | */ static int fts5HashResize(Fts5Hash *pHash){ int nNew = pHash->nSlot*2; int i; Fts5HashEntry **apNew; Fts5HashEntry **apOld = pHash->aSlot; | | | 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 | */ static int fts5HashResize(Fts5Hash *pHash){ int nNew = pHash->nSlot*2; int i; Fts5HashEntry **apNew; Fts5HashEntry **apOld = pHash->aSlot; apNew = (Fts5HashEntry**)sqlite3_malloc64(nNew*sizeof(Fts5HashEntry*)); if( !apNew ) return SQLITE_NOMEM; memset(apNew, 0, nNew*sizeof(Fts5HashEntry*)); for(i=0; i<pHash->nSlot; i++){ while( apOld[i] ){ unsigned int iHash; Fts5HashEntry *p = apOld[i]; |
︙ | ︙ | |||
183 184 185 186 187 188 189 | sqlite3_free(apOld); pHash->nSlot = nNew; pHash->aSlot = apNew; return SQLITE_OK; } | | > > > > > | > | | | | | > > | | | > | > > | 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 | sqlite3_free(apOld); pHash->nSlot = nNew; pHash->aSlot = apNew; return SQLITE_OK; } static int fts5HashAddPoslistSize( Fts5Hash *pHash, Fts5HashEntry *p, Fts5HashEntry *p2 ){ int nRet = 0; if( p->iSzPoslist ){ u8 *pPtr = p2 ? (u8*)p2 : (u8*)p; int nData = p->nData; if( pHash->eDetail==FTS5_DETAIL_NONE ){ assert( nData==p->iSzPoslist ); if( p->bDel ){ pPtr[nData++] = 0x00; if( p->bContent ){ pPtr[nData++] = 0x00; } } }else{ int nSz = (nData - p->iSzPoslist - 1); /* Size in bytes */ int nPos = nSz*2 + p->bDel; /* Value of nPos field */ assert( p->bDel==0 || p->bDel==1 ); if( nPos<=127 ){ pPtr[p->iSzPoslist] = (u8)nPos; }else{ int nByte = sqlite3Fts5GetVarintLen((u32)nPos); memmove(&pPtr[p->iSzPoslist + nByte], &pPtr[p->iSzPoslist + 1], nSz); sqlite3Fts5PutVarint(&pPtr[p->iSzPoslist], nPos); nData += (nByte-1); } } nRet = nData - p->nData; if( p2==0 ){ p->iSzPoslist = 0; p->bDel = 0; p->bContent = 0; p->nData = nData; } } return nRet; } /* ** Add an entry to the in-memory hash table. The key is the concatenation ** of bByte and (pToken/nToken). The value is (iRowid/iCol/iPos). ** ** (bByte || pToken) -> (iRowid,iCol,iPos) |
︙ | ︙ | |||
255 256 257 258 259 260 261 | } } /* If an existing hash entry cannot be found, create a new one. */ if( p==0 ){ /* Figure out how much space to allocate */ char *zKey; | | | | | 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 | } } /* If an existing hash entry cannot be found, create a new one. */ if( p==0 ){ /* Figure out how much space to allocate */ char *zKey; sqlite3_int64 nByte = sizeof(Fts5HashEntry) + (nToken+1) + 1 + 64; if( nByte<128 ) nByte = 128; /* Grow the Fts5Hash.aSlot[] array if necessary. */ if( (pHash->nEntry*2)>=pHash->nSlot ){ int rc = fts5HashResize(pHash); if( rc!=SQLITE_OK ) return rc; iHash = fts5HashKey2(pHash->nSlot, (u8)bByte, (const u8*)pToken, nToken); } /* Allocate new Fts5HashEntry and add it to the hash table. */ p = (Fts5HashEntry*)sqlite3_malloc64(nByte); if( !p ) return SQLITE_NOMEM; memset(p, 0, sizeof(Fts5HashEntry)); p->nAlloc = (int)nByte; zKey = fts5EntryKey(p); zKey[0] = bByte; memcpy(&zKey[1], pToken, nToken); assert( iHash==fts5HashKey(pHash->nSlot, (u8*)zKey, nToken+1) ); p->nKey = nToken; zKey[nToken+1] = '\0'; p->nData = nToken+1 + 1 + sizeof(Fts5HashEntry); |
︙ | ︙ | |||
291 292 293 294 295 296 297 | p->iSzPoslist = p->nData; if( pHash->eDetail!=FTS5_DETAIL_NONE ){ p->nData += 1; p->iCol = (pHash->eDetail==FTS5_DETAIL_FULL ? 0 : -1); } | < | | | > | | | | 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340 341 342 343 344 345 346 347 348 349 350 351 352 353 354 355 356 357 358 359 360 361 362 363 364 365 366 | p->iSzPoslist = p->nData; if( pHash->eDetail!=FTS5_DETAIL_NONE ){ p->nData += 1; p->iCol = (pHash->eDetail==FTS5_DETAIL_FULL ? 0 : -1); } }else{ /* Appending to an existing hash-entry. Check that there is enough ** space to append the largest possible new entry. Worst case scenario ** is: ** ** + 9 bytes for a new rowid, ** + 4 byte reserved for the "poslist size" varint. ** + 1 byte for a "new column" byte, ** + 3 bytes for a new column number (16-bit max) as a varint, ** + 5 bytes for the new position offset (32-bit max). */ if( (p->nAlloc - p->nData) < (9 + 4 + 1 + 3 + 5) ){ sqlite3_int64 nNew = p->nAlloc * 2; Fts5HashEntry *pNew; Fts5HashEntry **pp; pNew = (Fts5HashEntry*)sqlite3_realloc64(p, nNew); if( pNew==0 ) return SQLITE_NOMEM; pNew->nAlloc = (int)nNew; for(pp=&pHash->aSlot[iHash]; *pp!=p; pp=&(*pp)->pHashNext); *pp = pNew; p = pNew; } nIncr -= p->nData; } assert( (p->nAlloc - p->nData) >= (9 + 4 + 1 + 3 + 5) ); pPtr = (u8*)p; /* If this is a new rowid, append the 4-byte size field for the previous ** entry, and the new rowid for this entry. */ if( iRowid!=p->iRowid ){ u64 iDiff = (u64)iRowid - (u64)p->iRowid; fts5HashAddPoslistSize(pHash, p, 0); p->nData += sqlite3Fts5PutVarint(&pPtr[p->nData], iDiff); p->iRowid = iRowid; bNew = 1; p->iSzPoslist = p->nData; if( pHash->eDetail!=FTS5_DETAIL_NONE ){ p->nData += 1; p->iCol = (pHash->eDetail==FTS5_DETAIL_FULL ? 0 : -1); p->iPos = 0; } } if( iCol>=0 ){ if( pHash->eDetail==FTS5_DETAIL_NONE ){ p->bContent = 1; }else{ /* Append a new column value, if necessary */ assert_nc( iCol>=p->iCol ); if( iCol!=p->iCol ){ if( pHash->eDetail==FTS5_DETAIL_FULL ){ pPtr[p->nData++] = 0x01; p->nData += sqlite3Fts5PutVarint(&pPtr[p->nData], iCol); p->iCol = (i16)iCol; p->iPos = 0; }else{ |
︙ | ︙ | |||
434 435 436 437 438 439 440 | const int nMergeSlot = 32; Fts5HashEntry **ap; Fts5HashEntry *pList; int iSlot; int i; *ppSorted = 0; | | > | > | 445 446 447 448 449 450 451 452 453 454 455 456 457 458 459 460 461 462 463 464 465 466 467 468 | const int nMergeSlot = 32; Fts5HashEntry **ap; Fts5HashEntry *pList; int iSlot; int i; *ppSorted = 0; ap = sqlite3_malloc64(sizeof(Fts5HashEntry*) * nMergeSlot); if( !ap ) return SQLITE_NOMEM; memset(ap, 0, sizeof(Fts5HashEntry*) * nMergeSlot); for(iSlot=0; iSlot<pHash->nSlot; iSlot++){ Fts5HashEntry *pIter; for(pIter=pHash->aSlot[iSlot]; pIter; pIter=pIter->pHashNext){ if( pTerm==0 || (pIter->nKey+1>=nTerm && 0==memcmp(fts5EntryKey(pIter), pTerm, nTerm)) ){ Fts5HashEntry *pEntry = pIter; pEntry->pScanNext = 0; for(i=0; ap[i]; i++){ pEntry = fts5HashEntryMerge(pEntry, ap[i]); ap[i] = 0; } ap[i] = pEntry; |
︙ | ︙ | |||
469 470 471 472 473 474 475 476 | } /* ** Query the hash table for a doclist associated with term pTerm/nTerm. */ int sqlite3Fts5HashQuery( Fts5Hash *pHash, /* Hash table to query */ const char *pTerm, int nTerm, /* Query term */ | > | > | > > > > > > | | > | > > | | 482 483 484 485 486 487 488 489 490 491 492 493 494 495 496 497 498 499 500 501 502 503 504 505 506 507 508 509 510 511 512 513 514 515 516 517 518 519 520 521 522 523 524 525 | } /* ** Query the hash table for a doclist associated with term pTerm/nTerm. */ int sqlite3Fts5HashQuery( Fts5Hash *pHash, /* Hash table to query */ int nPre, const char *pTerm, int nTerm, /* Query term */ void **ppOut, /* OUT: Pointer to new object */ int *pnDoclist /* OUT: Size of doclist in bytes */ ){ unsigned int iHash = fts5HashKey(pHash->nSlot, (const u8*)pTerm, nTerm); char *zKey = 0; Fts5HashEntry *p; for(p=pHash->aSlot[iHash]; p; p=p->pHashNext){ zKey = fts5EntryKey(p); assert( p->nKey+1==(int)strlen(zKey) ); if( nTerm==p->nKey+1 && memcmp(zKey, pTerm, nTerm)==0 ) break; } if( p ){ int nHashPre = sizeof(Fts5HashEntry) + nTerm + 1; int nList = p->nData - nHashPre; u8 *pRet = (u8*)(*ppOut = sqlite3_malloc64(nPre + nList + 10)); if( pRet ){ Fts5HashEntry *pFaux = (Fts5HashEntry*)&pRet[nPre-nHashPre]; memcpy(&pRet[nPre], &((u8*)p)[nHashPre], nList); nList += fts5HashAddPoslistSize(pHash, p, pFaux); *pnDoclist = nList; }else{ *pnDoclist = 0; return SQLITE_NOMEM; } }else{ *ppOut = 0; *pnDoclist = 0; } return SQLITE_OK; } int sqlite3Fts5HashScanInit( |
︙ | ︙ | |||
520 521 522 523 524 525 526 | const u8 **ppDoclist, /* OUT: pointer to doclist */ int *pnDoclist /* OUT: size of doclist in bytes */ ){ Fts5HashEntry *p; if( (p = pHash->pScan) ){ char *zKey = fts5EntryKey(p); int nTerm = (int)strlen(zKey); | | < | 544 545 546 547 548 549 550 551 552 553 554 555 556 557 558 559 560 | const u8 **ppDoclist, /* OUT: pointer to doclist */ int *pnDoclist /* OUT: size of doclist in bytes */ ){ Fts5HashEntry *p; if( (p = pHash->pScan) ){ char *zKey = fts5EntryKey(p); int nTerm = (int)strlen(zKey); fts5HashAddPoslistSize(pHash, p, 0); *pzTerm = zKey; *ppDoclist = (const u8*)&zKey[nTerm+1]; *pnDoclist = p->nData - (sizeof(Fts5HashEntry) + nTerm + 1); }else{ *pzTerm = 0; *ppDoclist = 0; *pnDoclist = 0; } } |
Changes to ext/fts5/fts5_index.c.
︙ | ︙ | |||
235 236 237 238 239 240 241 | ((i64)(height) << (FTS5_DATA_PAGE_B)) + \ ((i64)(pgno)) \ ) #define FTS5_SEGMENT_ROWID(segid, pgno) fts5_dri(segid, 0, 0, pgno) #define FTS5_DLIDX_ROWID(segid, height, pgno) fts5_dri(segid, 1, height, pgno) | < < < < < | 235 236 237 238 239 240 241 242 243 244 245 246 247 248 | ((i64)(height) << (FTS5_DATA_PAGE_B)) + \ ((i64)(pgno)) \ ) #define FTS5_SEGMENT_ROWID(segid, pgno) fts5_dri(segid, 0, 0, pgno) #define FTS5_DLIDX_ROWID(segid, height, pgno) fts5_dri(segid, 1, height, pgno) #ifdef SQLITE_DEBUG int sqlite3Fts5Corrupt() { return SQLITE_CORRUPT_VTAB; } #endif /* ** Each time a blob is read from the %_data table, it is padded with this |
︙ | ︙ | |||
297 298 299 300 301 302 303 | int rc; /* Current error code */ /* State used by the fts5DataXXX() functions. */ sqlite3_blob *pReader; /* RO incr-blob open on %_data table */ sqlite3_stmt *pWriter; /* "INSERT ... %_data VALUES(?,?)" */ sqlite3_stmt *pDeleter; /* "DELETE FROM %_data ... id>=? AND id<=?" */ sqlite3_stmt *pIdxWriter; /* "INSERT ... %_idx VALUES(?,?,?,?)" */ | | | 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 | int rc; /* Current error code */ /* State used by the fts5DataXXX() functions. */ sqlite3_blob *pReader; /* RO incr-blob open on %_data table */ sqlite3_stmt *pWriter; /* "INSERT ... %_data VALUES(?,?)" */ sqlite3_stmt *pDeleter; /* "DELETE FROM %_data ... id>=? AND id<=?" */ sqlite3_stmt *pIdxWriter; /* "INSERT ... %_idx VALUES(?,?,?,?)" */ sqlite3_stmt *pIdxDeleter; /* "DELETE FROM %_idx WHERE segid=?" */ sqlite3_stmt *pIdxSelect; int nRead; /* Total number of blocks read */ sqlite3_stmt *pDataVersion; i64 iStructVersion; /* data_version when pStruct read */ Fts5Structure *pStruct; /* Current db structure (or NULL) */ }; |
︙ | ︙ | |||
432 433 434 435 436 437 438 | */ struct Fts5SegIter { Fts5StructureSegment *pSeg; /* Segment to iterate through */ int flags; /* Mask of configuration flags */ int iLeafPgno; /* Current leaf page number */ Fts5Data *pLeaf; /* Current leaf data */ Fts5Data *pNextLeaf; /* Leaf page (iLeafPgno+1) */ | | | 427 428 429 430 431 432 433 434 435 436 437 438 439 440 441 | */ struct Fts5SegIter { Fts5StructureSegment *pSeg; /* Segment to iterate through */ int flags; /* Mask of configuration flags */ int iLeafPgno; /* Current leaf page number */ Fts5Data *pLeaf; /* Current leaf data */ Fts5Data *pNextLeaf; /* Leaf page (iLeafPgno+1) */ i64 iLeafOffset; /* Byte offset within current leaf */ /* Next method */ void (*xNext)(Fts5Index*, Fts5SegIter*, int*); /* The page and offset from which the current term was read. The offset ** is the offset of the first rowid in the current doclist. */ int iTermLeafPgno; |
︙ | ︙ | |||
508 509 510 511 512 513 514 | ** Used by sqlite3Fts5IterPoslist() when the poslist needs to be buffered. ** There is no way to tell if this is populated or not. */ struct Fts5Iter { Fts5IndexIter base; /* Base class containing output vars */ Fts5Index *pIndex; /* Index that owns this iterator */ | < | 503 504 505 506 507 508 509 510 511 512 513 514 515 516 | ** Used by sqlite3Fts5IterPoslist() when the poslist needs to be buffered. ** There is no way to tell if this is populated or not. */ struct Fts5Iter { Fts5IndexIter base; /* Base class containing output vars */ Fts5Index *pIndex; /* Index that owns this iterator */ Fts5Buffer poslist; /* Buffer containing current poslist */ Fts5Colset *pColset; /* Restrict matches to these columns */ /* Invoked to set output variables. */ void (*xSetOutputs)(Fts5Iter*, Fts5SegIter*); int nSeg; /* Size of aSeg[] array */ |
︙ | ︙ | |||
569 570 571 572 573 574 575 | /* ** Allocate and return a buffer at least nByte bytes in size. ** ** If an OOM error is encountered, return NULL and set the error code in ** the Fts5Index handle passed as the first argument. */ | | | 563 564 565 566 567 568 569 570 571 572 573 574 575 576 577 | /* ** Allocate and return a buffer at least nByte bytes in size. ** ** If an OOM error is encountered, return NULL and set the error code in ** the Fts5Index handle passed as the first argument. */ static void *fts5IdxMalloc(Fts5Index *p, sqlite3_int64 nByte){ return sqlite3Fts5MallocZero(&p->rc, nByte); } /* ** Compare the contents of the pLeft buffer with the pRight/nRight blob. ** ** Return -ve if pLeft is smaller than pRight, 0 if they are equal or |
︙ | ︙ | |||
602 603 604 605 606 607 608 | ** ** Return -ve if pLeft is smaller than pRight, 0 if they are equal or ** +ve if pRight is smaller than pLeft. In other words: ** ** res = *pLeft - *pRight */ static int fts5BufferCompare(Fts5Buffer *pLeft, Fts5Buffer *pRight){ | > | > > | | | 596 597 598 599 600 601 602 603 604 605 606 607 608 609 610 611 612 613 614 615 616 617 618 619 620 621 622 623 624 625 626 627 | ** ** Return -ve if pLeft is smaller than pRight, 0 if they are equal or ** +ve if pRight is smaller than pLeft. In other words: ** ** res = *pLeft - *pRight */ static int fts5BufferCompare(Fts5Buffer *pLeft, Fts5Buffer *pRight){ int nCmp, res; nCmp = MIN(pLeft->n, pRight->n); assert( nCmp<=0 || pLeft->p!=0 ); assert( nCmp<=0 || pRight->p!=0 ); res = fts5Memcmp(pLeft->p, pRight->p, nCmp); return (res==0 ? (pLeft->n - pRight->n) : res); } static int fts5LeafFirstTermOff(Fts5Data *pLeaf){ int ret; fts5GetVarint32(&pLeaf->p[pLeaf->szLeaf], ret); return ret; } /* ** Close the read-only blob handle, if it is open. */ void sqlite3Fts5IndexCloseReader(Fts5Index *p){ if( p->pReader ){ sqlite3_blob *pReader = p->pReader; p->pReader = 0; sqlite3_blob_close(pReader); } } |
︙ | ︙ | |||
645 646 647 648 649 650 651 | ** is required. */ sqlite3_blob *pBlob = p->pReader; p->pReader = 0; rc = sqlite3_blob_reopen(pBlob, iRowid); assert( p->pReader==0 ); p->pReader = pBlob; if( rc!=SQLITE_OK ){ | | | 642 643 644 645 646 647 648 649 650 651 652 653 654 655 656 | ** is required. */ sqlite3_blob *pBlob = p->pReader; p->pReader = 0; rc = sqlite3_blob_reopen(pBlob, iRowid); assert( p->pReader==0 ); p->pReader = pBlob; if( rc!=SQLITE_OK ){ sqlite3Fts5IndexCloseReader(p); } if( rc==SQLITE_ABORT ) rc = SQLITE_OK; } /* If the blob handle is not open at this point, open it and seek ** to the requested entry. */ if( p->pReader==0 && rc==SQLITE_OK ){ |
︙ | ︙ | |||
669 670 671 672 673 674 675 | ** table, missing row, non-blob/text in block column - indicate ** backing store corruption. */ if( rc==SQLITE_ERROR ) rc = FTS5_CORRUPT; if( rc==SQLITE_OK ){ u8 *aOut = 0; /* Read blob data into this buffer */ int nByte = sqlite3_blob_bytes(p->pReader); | | | > > > | | > | 666 667 668 669 670 671 672 673 674 675 676 677 678 679 680 681 682 683 684 685 686 687 688 689 690 691 692 693 694 695 696 697 698 699 700 701 702 703 704 705 706 707 708 709 710 711 712 713 714 715 716 717 718 719 720 721 722 723 724 725 726 727 728 729 730 731 732 733 734 735 736 737 738 739 740 | ** table, missing row, non-blob/text in block column - indicate ** backing store corruption. */ if( rc==SQLITE_ERROR ) rc = FTS5_CORRUPT; if( rc==SQLITE_OK ){ u8 *aOut = 0; /* Read blob data into this buffer */ int nByte = sqlite3_blob_bytes(p->pReader); sqlite3_int64 nAlloc = sizeof(Fts5Data) + nByte + FTS5_DATA_PADDING; pRet = (Fts5Data*)sqlite3_malloc64(nAlloc); if( pRet ){ pRet->nn = nByte; aOut = pRet->p = (u8*)&pRet[1]; }else{ rc = SQLITE_NOMEM; } if( rc==SQLITE_OK ){ rc = sqlite3_blob_read(p->pReader, aOut, nByte, 0); } if( rc!=SQLITE_OK ){ sqlite3_free(pRet); pRet = 0; }else{ /* TODO1: Fix this */ pRet->p[nByte] = 0x00; pRet->p[nByte+1] = 0x00; pRet->szLeaf = fts5GetU16(&pRet->p[2]); } } p->rc = rc; p->nRead++; } assert( (pRet==0)==(p->rc!=SQLITE_OK) ); return pRet; } /* ** Release a reference to data record returned by an earlier call to ** fts5DataRead(). */ static void fts5DataRelease(Fts5Data *pData){ sqlite3_free(pData); } static Fts5Data *fts5LeafRead(Fts5Index *p, i64 iRowid){ Fts5Data *pRet = fts5DataRead(p, iRowid); if( pRet ){ if( pRet->nn<4 || pRet->szLeaf>pRet->nn ){ p->rc = FTS5_CORRUPT; fts5DataRelease(pRet); pRet = 0; } } return pRet; } static int fts5IndexPrepareStmt( Fts5Index *p, sqlite3_stmt **ppStmt, char *zSql ){ if( p->rc==SQLITE_OK ){ if( zSql ){ p->rc = sqlite3_prepare_v3(p->pConfig->db, zSql, -1, SQLITE_PREPARE_PERSISTENT|SQLITE_PREPARE_NO_VTAB, ppStmt, 0); }else{ p->rc = SQLITE_NOMEM; } } sqlite3_free(zSql); return p->rc; } |
︙ | ︙ | |||
766 767 768 769 770 771 772 | ** ** DELETE FROM %_data WHERE id BETWEEN $iFirst AND $iLast */ static void fts5DataDelete(Fts5Index *p, i64 iFirst, i64 iLast){ if( p->rc!=SQLITE_OK ) return; if( p->pDeleter==0 ){ | < < < < < | < < < < < < | 767 768 769 770 771 772 773 774 775 776 777 778 779 780 781 782 783 784 785 786 | ** ** DELETE FROM %_data WHERE id BETWEEN $iFirst AND $iLast */ static void fts5DataDelete(Fts5Index *p, i64 iFirst, i64 iLast){ if( p->rc!=SQLITE_OK ) return; if( p->pDeleter==0 ){ Fts5Config *pConfig = p->pConfig; char *zSql = sqlite3_mprintf( "DELETE FROM '%q'.'%q_data' WHERE id>=? AND id<=?", pConfig->zDb, pConfig->zName ); if( fts5IndexPrepareStmt(p, &p->pDeleter, zSql) ) return; } sqlite3_bind_int64(p->pDeleter, 1, iFirst); sqlite3_bind_int64(p->pDeleter, 2, iLast); sqlite3_step(p->pDeleter); p->rc = sqlite3_reset(p->pDeleter); } |
︙ | ︙ | |||
830 831 832 833 834 835 836 837 838 839 840 841 842 843 | sqlite3_free(pStruct); } } static void fts5StructureRef(Fts5Structure *pStruct){ pStruct->nRef++; } /* ** Deserialize and return the structure record currently stored in serialized ** form within buffer pData/nData. ** ** The Fts5Structure.aLevel[] and each Fts5StructureLevel.aSeg[] array ** are over-allocated by one slot. This allows the structure contents | > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | 820 821 822 823 824 825 826 827 828 829 830 831 832 833 834 835 836 837 838 839 840 841 842 843 844 845 846 847 848 849 850 851 852 853 854 855 856 857 858 859 860 861 862 863 864 865 866 867 868 869 870 871 872 873 874 875 876 877 878 879 880 881 882 883 884 885 | sqlite3_free(pStruct); } } static void fts5StructureRef(Fts5Structure *pStruct){ pStruct->nRef++; } void *sqlite3Fts5StructureRef(Fts5Index *p){ fts5StructureRef(p->pStruct); return (void*)p->pStruct; } void sqlite3Fts5StructureRelease(void *p){ if( p ){ fts5StructureRelease((Fts5Structure*)p); } } int sqlite3Fts5StructureTest(Fts5Index *p, void *pStruct){ if( p->pStruct!=(Fts5Structure*)pStruct ){ return SQLITE_ABORT; } return SQLITE_OK; } /* ** Ensure that structure object (*pp) is writable. ** ** This function is a no-op if (*pRc) is not SQLITE_OK when it is called. If ** an error occurs, (*pRc) is set to an SQLite error code before returning. */ static void fts5StructureMakeWritable(int *pRc, Fts5Structure **pp){ Fts5Structure *p = *pp; if( *pRc==SQLITE_OK && p->nRef>1 ){ i64 nByte = sizeof(Fts5Structure)+(p->nLevel-1)*sizeof(Fts5StructureLevel); Fts5Structure *pNew; pNew = (Fts5Structure*)sqlite3Fts5MallocZero(pRc, nByte); if( pNew ){ int i; memcpy(pNew, p, nByte); for(i=0; i<p->nLevel; i++) pNew->aLevel[i].aSeg = 0; for(i=0; i<p->nLevel; i++){ Fts5StructureLevel *pLvl = &pNew->aLevel[i]; nByte = sizeof(Fts5StructureSegment) * pNew->aLevel[i].nSeg; pLvl->aSeg = (Fts5StructureSegment*)sqlite3Fts5MallocZero(pRc, nByte); if( pLvl->aSeg==0 ){ for(i=0; i<p->nLevel; i++){ sqlite3_free(pNew->aLevel[i].aSeg); } sqlite3_free(pNew); return; } memcpy(pLvl->aSeg, p->aLevel[i].aSeg, nByte); } p->nRef--; pNew->nRef = 1; } *pp = pNew; } } /* ** Deserialize and return the structure record currently stored in serialized ** form within buffer pData/nData. ** ** The Fts5Structure.aLevel[] and each Fts5StructureLevel.aSeg[] array ** are over-allocated by one slot. This allows the structure contents |
︙ | ︙ | |||
854 855 856 857 858 859 860 | Fts5Structure **ppOut /* OUT: Deserialized object */ ){ int rc = SQLITE_OK; int i = 0; int iLvl; int nLevel = 0; int nSegment = 0; | | > > > > > | 896 897 898 899 900 901 902 903 904 905 906 907 908 909 910 911 912 913 914 915 916 917 918 919 920 921 922 923 924 925 | Fts5Structure **ppOut /* OUT: Deserialized object */ ){ int rc = SQLITE_OK; int i = 0; int iLvl; int nLevel = 0; int nSegment = 0; sqlite3_int64 nByte; /* Bytes of space to allocate at pRet */ Fts5Structure *pRet = 0; /* Structure object to return */ /* Grab the cookie value */ if( piCookie ) *piCookie = sqlite3Fts5Get32(pData); i = 4; /* Read the total number of levels and segments from the start of the ** structure record. */ i += fts5GetVarint32(&pData[i], nLevel); i += fts5GetVarint32(&pData[i], nSegment); if( nLevel>FTS5_MAX_SEGMENT || nLevel<0 || nSegment>FTS5_MAX_SEGMENT || nSegment<0 ){ return FTS5_CORRUPT; } nByte = ( sizeof(Fts5Structure) + /* Main structure */ sizeof(Fts5StructureLevel) * (nLevel-1) /* aLevel[] array */ ); pRet = (Fts5Structure*)sqlite3Fts5MallocZero(&rc, nByte); if( pRet ){ |
︙ | ︙ | |||
887 888 889 890 891 892 893 | int iSeg; if( i>=nData ){ rc = FTS5_CORRUPT; }else{ i += fts5GetVarint32(&pData[i], pLvl->nMerge); i += fts5GetVarint32(&pData[i], nTotal); | | > > | | | > > > | | > > | > > > > | > | | | 934 935 936 937 938 939 940 941 942 943 944 945 946 947 948 949 950 951 952 953 954 955 956 957 958 959 960 961 962 963 964 965 966 967 968 969 970 971 972 973 974 975 976 977 978 979 980 981 982 983 984 985 986 987 988 989 990 991 992 993 994 995 996 997 998 999 1000 1001 | int iSeg; if( i>=nData ){ rc = FTS5_CORRUPT; }else{ i += fts5GetVarint32(&pData[i], pLvl->nMerge); i += fts5GetVarint32(&pData[i], nTotal); if( nTotal<pLvl->nMerge ) rc = FTS5_CORRUPT; pLvl->aSeg = (Fts5StructureSegment*)sqlite3Fts5MallocZero(&rc, nTotal * sizeof(Fts5StructureSegment) ); nSegment -= nTotal; } if( rc==SQLITE_OK ){ pLvl->nSeg = nTotal; for(iSeg=0; iSeg<nTotal; iSeg++){ Fts5StructureSegment *pSeg = &pLvl->aSeg[iSeg]; if( i>=nData ){ rc = FTS5_CORRUPT; break; } i += fts5GetVarint32(&pData[i], pSeg->iSegid); i += fts5GetVarint32(&pData[i], pSeg->pgnoFirst); i += fts5GetVarint32(&pData[i], pSeg->pgnoLast); if( pSeg->pgnoLast<pSeg->pgnoFirst ){ rc = FTS5_CORRUPT; break; } } if( iLvl>0 && pLvl[-1].nMerge && nTotal==0 ) rc = FTS5_CORRUPT; if( iLvl==nLevel-1 && pLvl->nMerge ) rc = FTS5_CORRUPT; } } if( nSegment!=0 && rc==SQLITE_OK ) rc = FTS5_CORRUPT; if( rc!=SQLITE_OK ){ fts5StructureRelease(pRet); pRet = 0; } } *ppOut = pRet; return rc; } /* ** Add a level to the Fts5Structure.aLevel[] array of structure object ** (*ppStruct). */ static void fts5StructureAddLevel(int *pRc, Fts5Structure **ppStruct){ fts5StructureMakeWritable(pRc, ppStruct); if( *pRc==SQLITE_OK ){ Fts5Structure *pStruct = *ppStruct; int nLevel = pStruct->nLevel; sqlite3_int64 nByte = ( sizeof(Fts5Structure) + /* Main structure */ sizeof(Fts5StructureLevel) * (nLevel+1) /* aLevel[] array */ ); pStruct = sqlite3_realloc64(pStruct, nByte); if( pStruct ){ memset(&pStruct->aLevel[nLevel], 0, sizeof(Fts5StructureLevel)); pStruct->nLevel++; *ppStruct = pStruct; }else{ *pRc = SQLITE_NOMEM; } |
︙ | ︙ | |||
953 954 955 956 957 958 959 | int iLvl, int nExtra, int bInsert ){ if( *pRc==SQLITE_OK ){ Fts5StructureLevel *pLvl = &pStruct->aLevel[iLvl]; Fts5StructureSegment *aNew; | | | | 1012 1013 1014 1015 1016 1017 1018 1019 1020 1021 1022 1023 1024 1025 1026 1027 1028 1029 | int iLvl, int nExtra, int bInsert ){ if( *pRc==SQLITE_OK ){ Fts5StructureLevel *pLvl = &pStruct->aLevel[iLvl]; Fts5StructureSegment *aNew; sqlite3_int64 nByte; nByte = (pLvl->nSeg + nExtra) * sizeof(Fts5StructureSegment); aNew = sqlite3_realloc64(pLvl->aSeg, nByte); if( aNew ){ if( bInsert==0 ){ memset(&aNew[pLvl->nSeg], 0, sizeof(Fts5StructureSegment) * nExtra); }else{ int nMove = pLvl->nSeg * sizeof(Fts5StructureSegment); memmove(&aNew[nExtra], aNew, nMove); memset(aNew, 0, sizeof(Fts5StructureSegment) * nExtra); |
︙ | ︙ | |||
983 984 985 986 987 988 989 | Fts5Data *pData; pData = fts5DataRead(p, FTS5_STRUCTURE_ROWID); if( p->rc==SQLITE_OK ){ /* TODO: Do we need this if the leaf-index is appended? Probably... */ memset(&pData->p[pData->nn], 0, FTS5_DATA_PADDING); p->rc = fts5StructureDecode(pData->p, pData->nn, &iCookie, &pRet); | | | 1042 1043 1044 1045 1046 1047 1048 1049 1050 1051 1052 1053 1054 1055 1056 | Fts5Data *pData; pData = fts5DataRead(p, FTS5_STRUCTURE_ROWID); if( p->rc==SQLITE_OK ){ /* TODO: Do we need this if the leaf-index is appended? Probably... */ memset(&pData->p[pData->nn], 0, FTS5_DATA_PADDING); p->rc = fts5StructureDecode(pData->p, pData->nn, &iCookie, &pRet); if( p->rc==SQLITE_OK && (pConfig->pgsz==0 || pConfig->iCookie!=iCookie) ){ p->rc = sqlite3Fts5ConfigLoad(pConfig, iCookie); } fts5DataRelease(pData); if( p->rc!=SQLITE_OK ){ fts5StructureRelease(pRet); pRet = 0; } |
︙ | ︙ | |||
1470 1471 1472 1473 1474 1475 1476 | int iLeafPg /* Leaf page number to load dlidx for */ ){ Fts5DlidxIter *pIter = 0; int i; int bDone = 0; for(i=0; p->rc==SQLITE_OK && bDone==0; i++){ | | | | 1529 1530 1531 1532 1533 1534 1535 1536 1537 1538 1539 1540 1541 1542 1543 1544 1545 1546 | int iLeafPg /* Leaf page number to load dlidx for */ ){ Fts5DlidxIter *pIter = 0; int i; int bDone = 0; for(i=0; p->rc==SQLITE_OK && bDone==0; i++){ sqlite3_int64 nByte = sizeof(Fts5DlidxIter) + i * sizeof(Fts5DlidxLvl); Fts5DlidxIter *pNew; pNew = (Fts5DlidxIter*)sqlite3_realloc64(pIter, nByte); if( pNew==0 ){ p->rc = SQLITE_NOMEM; }else{ i64 iRowid = FTS5_DLIDX_ROWID(iSegid, i, iLeafPg); Fts5DlidxLvl *pLvl = &pNew->aLvl[i]; pIter = pNew; memset(pLvl, 0, sizeof(Fts5DlidxLvl)); |
︙ | ︙ | |||
1606 1607 1608 1609 1610 1611 1612 | } pIter->iLeafOffset = iOff; } } static void fts5SegIterLoadRowid(Fts5Index *p, Fts5SegIter *pIter){ u8 *a = pIter->pLeaf->p; /* Buffer to read data from */ | | | 1665 1666 1667 1668 1669 1670 1671 1672 1673 1674 1675 1676 1677 1678 1679 | } pIter->iLeafOffset = iOff; } } static void fts5SegIterLoadRowid(Fts5Index *p, Fts5SegIter *pIter){ u8 *a = pIter->pLeaf->p; /* Buffer to read data from */ i64 iOff = pIter->iLeafOffset; ASSERT_SZLEAF_OK(pIter->pLeaf); if( iOff>=pIter->pLeaf->szLeaf ){ fts5SegIterNextPage(p, pIter); if( pIter->pLeaf==0 ){ if( p->rc==SQLITE_OK ) p->rc = FTS5_CORRUPT; return; |
︙ | ︙ | |||
1639 1640 1641 1642 1643 1644 1645 | ** ** accordingly and leaves (Fts5SegIter.iLeafOffset) set to the content of ** the first position list. The position list belonging to document ** (Fts5SegIter.iRowid). */ static void fts5SegIterLoadTerm(Fts5Index *p, Fts5SegIter *pIter, int nKeep){ u8 *a = pIter->pLeaf->p; /* Buffer to read data from */ | | | > | 1698 1699 1700 1701 1702 1703 1704 1705 1706 1707 1708 1709 1710 1711 1712 1713 1714 1715 1716 1717 1718 1719 1720 1721 1722 | ** ** accordingly and leaves (Fts5SegIter.iLeafOffset) set to the content of ** the first position list. The position list belonging to document ** (Fts5SegIter.iRowid). */ static void fts5SegIterLoadTerm(Fts5Index *p, Fts5SegIter *pIter, int nKeep){ u8 *a = pIter->pLeaf->p; /* Buffer to read data from */ i64 iOff = pIter->iLeafOffset; /* Offset to read at */ int nNew; /* Bytes of new data */ iOff += fts5GetVarint32(&a[iOff], nNew); if( iOff+nNew>pIter->pLeaf->szLeaf || nKeep>pIter->term.n || nNew==0 ){ p->rc = FTS5_CORRUPT; return; } pIter->term.n = nKeep; fts5BufferAppendBlob(&p->rc, &pIter->term, nNew, &a[iOff]); assert( pIter->term.n<=pIter->term.nSpace ); iOff += nNew; pIter->iTermLeafOffset = iOff; pIter->iTermLeafPgno = pIter->iLeafPgno; pIter->iLeafOffset = iOff; if( pIter->iPgidxOff>=pIter->pLeaf->nn ){ pIter->iEndofDoclist = pIter->pLeaf->nn+1; |
︙ | ︙ | |||
1712 1713 1714 1715 1716 1717 1718 1719 | pIter->pSeg = pSeg; pIter->iLeafPgno = pSeg->pgnoFirst-1; fts5SegIterNextPage(p, pIter); } if( p->rc==SQLITE_OK ){ pIter->iLeafOffset = 4; assert_nc( pIter->pLeaf->nn>4 ); | > | | 1772 1773 1774 1775 1776 1777 1778 1779 1780 1781 1782 1783 1784 1785 1786 1787 1788 | pIter->pSeg = pSeg; pIter->iLeafPgno = pSeg->pgnoFirst-1; fts5SegIterNextPage(p, pIter); } if( p->rc==SQLITE_OK ){ pIter->iLeafOffset = 4; assert( pIter->pLeaf!=0 ); assert_nc( pIter->pLeaf->nn>4 ); assert_nc( fts5LeafFirstTermOff(pIter->pLeaf)==4 ); pIter->iPgidxOff = pIter->pLeaf->szLeaf+1; fts5SegIterLoadTerm(p, pIter, 0); fts5SegIterLoadNPos(p, pIter); } } /* |
︙ | ︙ | |||
1748 1749 1750 1751 1752 1753 1754 | if( n>pIter->iEndofDoclist ){ n = pIter->iEndofDoclist; } ASSERT_SZLEAF_OK(pIter->pLeaf); while( 1 ){ | | | | | 1809 1810 1811 1812 1813 1814 1815 1816 1817 1818 1819 1820 1821 1822 1823 1824 1825 1826 1827 1828 1829 1830 1831 1832 1833 1834 1835 1836 1837 1838 1839 1840 1841 1842 1843 1844 | if( n>pIter->iEndofDoclist ){ n = pIter->iEndofDoclist; } ASSERT_SZLEAF_OK(pIter->pLeaf); while( 1 ){ u64 iDelta = 0; if( eDetail==FTS5_DETAIL_NONE ){ /* todo */ if( i<n && a[i]==0 ){ i++; if( i<n && a[i]==0 ) i++; } }else{ int nPos; int bDummy; i += fts5GetPoslistSize(&a[i], &nPos, &bDummy); i += nPos; } if( i>=n ) break; i += fts5GetVarint(&a[i], &iDelta); pIter->iRowid += iDelta; /* If necessary, grow the pIter->aRowidOffset[] array. */ if( iRowidOffset>=pIter->nRowidOffset ){ int nNew = pIter->nRowidOffset + 8; int *aNew = (int*)sqlite3_realloc64(pIter->aRowidOffset,nNew*sizeof(int)); if( aNew==0 ){ p->rc = SQLITE_NOMEM; break; } pIter->aRowidOffset = aNew; pIter->nRowidOffset = nNew; } |
︙ | ︙ | |||
1814 1815 1816 1817 1818 1819 1820 | pIter->pLeaf = pNew; pIter->iLeafOffset = pIter->iTermLeafOffset; } }else{ int iRowidOff; iRowidOff = fts5LeafFirstRowidOff(pNew); if( iRowidOff ){ | > > > | | > | 1875 1876 1877 1878 1879 1880 1881 1882 1883 1884 1885 1886 1887 1888 1889 1890 1891 1892 1893 1894 | pIter->pLeaf = pNew; pIter->iLeafOffset = pIter->iTermLeafOffset; } }else{ int iRowidOff; iRowidOff = fts5LeafFirstRowidOff(pNew); if( iRowidOff ){ if( iRowidOff>=pNew->szLeaf ){ p->rc = FTS5_CORRUPT; }else{ pIter->pLeaf = pNew; pIter->iLeafOffset = iRowidOff; } } } if( pIter->pLeaf ){ u8 *a = &pIter->pLeaf->p[pIter->iLeafOffset]; pIter->iLeafOffset += fts5GetVarint(a, (u64*)&pIter->iRowid); break; |
︙ | ︙ | |||
1862 1863 1864 1865 1866 1867 1868 | assert( pIter->flags & FTS5_SEGITER_REVERSE ); assert( pIter->pNextLeaf==0 ); UNUSED_PARAM(pbUnused); if( pIter->iRowidOffset>0 ){ u8 *a = pIter->pLeaf->p; int iOff; | | | | 1927 1928 1929 1930 1931 1932 1933 1934 1935 1936 1937 1938 1939 1940 1941 1942 1943 1944 1945 1946 1947 1948 1949 1950 | assert( pIter->flags & FTS5_SEGITER_REVERSE ); assert( pIter->pNextLeaf==0 ); UNUSED_PARAM(pbUnused); if( pIter->iRowidOffset>0 ){ u8 *a = pIter->pLeaf->p; int iOff; u64 iDelta; pIter->iRowidOffset--; pIter->iLeafOffset = pIter->aRowidOffset[pIter->iRowidOffset]; fts5SegIterLoadNPos(p, pIter); iOff = pIter->iLeafOffset; if( p->pConfig->eDetail!=FTS5_DETAIL_NONE ){ iOff += pIter->nPos; } fts5GetVarint(&a[iOff], &iDelta); pIter->iRowid -= iDelta; }else{ fts5SegIterReverseNewPage(p, pIter); } } /* |
︙ | ︙ | |||
2064 2065 2066 2067 2068 2069 2070 | fts5SegIterLoadTerm(p, pIter, nKeep); fts5SegIterLoadNPos(p, pIter); if( pbNewTerm ) *pbNewTerm = 1; } }else{ /* The following could be done by calling fts5SegIterLoadNPos(). But ** this block is particularly performance critical, so equivalent | | < < < < < | | 2129 2130 2131 2132 2133 2134 2135 2136 2137 2138 2139 2140 2141 2142 2143 2144 2145 | fts5SegIterLoadTerm(p, pIter, nKeep); fts5SegIterLoadNPos(p, pIter); if( pbNewTerm ) *pbNewTerm = 1; } }else{ /* The following could be done by calling fts5SegIterLoadNPos(). But ** this block is particularly performance critical, so equivalent ** code is inlined. */ int nSz; assert_nc( pIter->iLeafOffset<=pIter->pLeaf->nn ); fts5FastGetVarint32(pIter->pLeaf->p, pIter->iLeafOffset, nSz); pIter->bDel = (nSz & 0x0001); pIter->nPos = nSz>>1; assert_nc( pIter->nPos>=0 ); } } } |
︙ | ︙ | |||
2100 2101 2102 2103 2104 2105 2106 | Fts5DlidxIter *pDlidx = pIter->pDlidx; Fts5Data *pLast = 0; int pgnoLast = 0; if( pDlidx ){ int iSegid = pIter->pSeg->iSegid; pgnoLast = fts5DlidxIterPgno(pDlidx); | | | 2160 2161 2162 2163 2164 2165 2166 2167 2168 2169 2170 2171 2172 2173 2174 | Fts5DlidxIter *pDlidx = pIter->pDlidx; Fts5Data *pLast = 0; int pgnoLast = 0; if( pDlidx ){ int iSegid = pIter->pSeg->iSegid; pgnoLast = fts5DlidxIterPgno(pDlidx); pLast = fts5LeafRead(p, FTS5_SEGMENT_ROWID(iSegid, pgnoLast)); }else{ Fts5Data *pLeaf = pIter->pLeaf; /* Current leaf data */ /* Currently, Fts5SegIter.iLeafOffset points to the first byte of ** position-list content for the current rowid. Back it up so that it ** points to the start of the position-list size field. */ int iPoslist; |
︙ | ︙ | |||
2127 2128 2129 2130 2131 2132 2133 | int pgno; Fts5StructureSegment *pSeg = pIter->pSeg; /* The last rowid in the doclist may not be on the current page. Search ** forward to find the page containing the last rowid. */ for(pgno=pIter->iLeafPgno+1; !p->rc && pgno<=pSeg->pgnoLast; pgno++){ i64 iAbs = FTS5_SEGMENT_ROWID(pSeg->iSegid, pgno); | | | 2187 2188 2189 2190 2191 2192 2193 2194 2195 2196 2197 2198 2199 2200 2201 | int pgno; Fts5StructureSegment *pSeg = pIter->pSeg; /* The last rowid in the doclist may not be on the current page. Search ** forward to find the page containing the last rowid. */ for(pgno=pIter->iLeafPgno+1; !p->rc && pgno<=pSeg->pgnoLast; pgno++){ i64 iAbs = FTS5_SEGMENT_ROWID(pSeg->iSegid, pgno); Fts5Data *pNew = fts5LeafRead(p, iAbs); if( pNew ){ int iRowid, bTermless; iRowid = fts5LeafFirstRowidOff(pNew); bTermless = fts5LeafIsTermless(pNew); if( iRowid ){ SWAPVAL(Fts5Data*, pNew, pLast); pgnoLast = pgno; |
︙ | ︙ | |||
2158 2159 2160 2161 2162 2163 2164 2165 2166 2167 2168 2169 2170 2171 2172 | */ if( pLast ){ int iOff; fts5DataRelease(pIter->pLeaf); pIter->pLeaf = pLast; pIter->iLeafPgno = pgnoLast; iOff = fts5LeafFirstRowidOff(pLast); iOff += fts5GetVarint(&pLast->p[iOff], (u64*)&pIter->iRowid); pIter->iLeafOffset = iOff; if( fts5LeafIsTermless(pLast) ){ pIter->iEndofDoclist = pLast->nn+1; }else{ pIter->iEndofDoclist = fts5LeafFirstTermOff(pLast); } | > > > > < | 2218 2219 2220 2221 2222 2223 2224 2225 2226 2227 2228 2229 2230 2231 2232 2233 2234 2235 2236 2237 2238 2239 2240 2241 2242 2243 | */ if( pLast ){ int iOff; fts5DataRelease(pIter->pLeaf); pIter->pLeaf = pLast; pIter->iLeafPgno = pgnoLast; iOff = fts5LeafFirstRowidOff(pLast); if( iOff>pLast->szLeaf ){ p->rc = FTS5_CORRUPT; return; } iOff += fts5GetVarint(&pLast->p[iOff], (u64*)&pIter->iRowid); pIter->iLeafOffset = iOff; if( fts5LeafIsTermless(pLast) ){ pIter->iEndofDoclist = pLast->nn+1; }else{ pIter->iEndofDoclist = fts5LeafFirstTermOff(pLast); } } fts5SegIterReverseInitPage(p, pIter); } /* ** Iterator pIter currently points to the first rowid of a doclist. |
︙ | ︙ | |||
2218 2219 2220 2221 2222 2223 2224 | */ static void fts5LeafSeek( Fts5Index *p, /* Leave any error code here */ int bGe, /* True for a >= search */ Fts5SegIter *pIter, /* Iterator to seek */ const u8 *pTerm, int nTerm /* Term to search for */ ){ | | < | | | | | | | | | | | | 2281 2282 2283 2284 2285 2286 2287 2288 2289 2290 2291 2292 2293 2294 2295 2296 2297 2298 2299 2300 2301 2302 2303 2304 2305 2306 2307 2308 2309 2310 2311 2312 2313 2314 2315 2316 2317 2318 2319 2320 2321 2322 2323 2324 2325 2326 2327 2328 2329 2330 2331 2332 2333 2334 | */ static void fts5LeafSeek( Fts5Index *p, /* Leave any error code here */ int bGe, /* True for a >= search */ Fts5SegIter *pIter, /* Iterator to seek */ const u8 *pTerm, int nTerm /* Term to search for */ ){ u32 iOff; const u8 *a = pIter->pLeaf->p; u32 n = (u32)pIter->pLeaf->nn; u32 nMatch = 0; u32 nKeep = 0; u32 nNew = 0; u32 iTermOff; u32 iPgidx; /* Current offset in pgidx */ int bEndOfPage = 0; assert( p->rc==SQLITE_OK ); iPgidx = (u32)pIter->pLeaf->szLeaf; iPgidx += fts5GetVarint32(&a[iPgidx], iTermOff); iOff = iTermOff; if( iOff>n ){ p->rc = FTS5_CORRUPT; return; } while( 1 ){ /* Figure out how many new bytes are in this term */ fts5FastGetVarint32(a, iOff, nNew); if( nKeep<nMatch ){ goto search_failed; } assert( nKeep>=nMatch ); if( nKeep==nMatch ){ u32 nCmp; u32 i; nCmp = (u32)MIN(nNew, nTerm-nMatch); for(i=0; i<nCmp; i++){ if( a[iOff+i]!=pTerm[nMatch+i] ) break; } nMatch += i; if( (u32)nTerm==nMatch ){ if( i==nNew ){ goto search_success; }else{ goto search_failed; } }else if( i<nNew && a[iOff+i]>pTerm[nMatch] ){ goto search_failed; |
︙ | ︙ | |||
2298 2299 2300 2301 2302 2303 2304 | return; }else if( bEndOfPage ){ do { fts5SegIterNextPage(p, pIter); if( pIter->pLeaf==0 ) return; a = pIter->pLeaf->p; if( fts5LeafIsTermless(pIter->pLeaf)==0 ){ | | | > | > > > | | 2360 2361 2362 2363 2364 2365 2366 2367 2368 2369 2370 2371 2372 2373 2374 2375 2376 2377 2378 2379 2380 2381 2382 2383 2384 2385 2386 2387 2388 2389 2390 2391 2392 2393 2394 | return; }else if( bEndOfPage ){ do { fts5SegIterNextPage(p, pIter); if( pIter->pLeaf==0 ) return; a = pIter->pLeaf->p; if( fts5LeafIsTermless(pIter->pLeaf)==0 ){ iPgidx = (u32)pIter->pLeaf->szLeaf; iPgidx += fts5GetVarint32(&pIter->pLeaf->p[iPgidx], iOff); if( iOff<4 || (i64)iOff>=pIter->pLeaf->szLeaf ){ p->rc = FTS5_CORRUPT; return; }else{ nKeep = 0; iTermOff = iOff; n = (u32)pIter->pLeaf->nn; iOff += fts5GetVarint32(&a[iOff], nNew); break; } } }while( 1 ); } search_success: if( (i64)iOff+nNew>n || nNew<1 ){ p->rc = FTS5_CORRUPT; return; } pIter->iLeafOffset = iOff + nNew; pIter->iTermLeafOffset = pIter->iLeafOffset; pIter->iTermLeafPgno = pIter->iLeafPgno; fts5BufferSet(&p->rc, &pIter->term, nKeep, pTerm); fts5BufferAppendBlob(&p->rc, &pIter->term, nNew, &a[iOff]); |
︙ | ︙ | |||
2422 2423 2424 2425 2426 2427 2428 | ** ** 1) an error has occurred, or ** 2) the iterator points to EOF, or ** 3) the iterator points to an entry with term (pTerm/nTerm), or ** 4) the FTS5INDEX_QUERY_SCAN flag was set and the iterator points ** to an entry with a term greater than or equal to (pTerm/nTerm). */ | | < > > > > > > > > > < | > > > > > > | < < < < | 2488 2489 2490 2491 2492 2493 2494 2495 2496 2497 2498 2499 2500 2501 2502 2503 2504 2505 2506 2507 2508 2509 2510 2511 2512 2513 2514 2515 2516 2517 2518 2519 2520 2521 2522 2523 2524 2525 2526 2527 2528 2529 2530 2531 2532 2533 2534 2535 2536 2537 2538 2539 2540 2541 2542 2543 2544 2545 2546 2547 2548 2549 2550 2551 2552 2553 2554 2555 2556 | ** ** 1) an error has occurred, or ** 2) the iterator points to EOF, or ** 3) the iterator points to an entry with term (pTerm/nTerm), or ** 4) the FTS5INDEX_QUERY_SCAN flag was set and the iterator points ** to an entry with a term greater than or equal to (pTerm/nTerm). */ assert_nc( p->rc!=SQLITE_OK /* 1 */ || pIter->pLeaf==0 /* 2 */ || fts5BufferCompareBlob(&pIter->term, pTerm, nTerm)==0 /* 3 */ || (bGe && fts5BufferCompareBlob(&pIter->term, pTerm, nTerm)>0) /* 4 */ ); } /* ** Initialize the object pIter to point to term pTerm/nTerm within the ** in-memory hash table. If there is no such term in the hash-table, the ** iterator is set to EOF. ** ** If an error occurs, Fts5Index.rc is set to an appropriate error code. If ** an error has already occurred when this function is called, it is a no-op. */ static void fts5SegIterHashInit( Fts5Index *p, /* FTS5 backend */ const u8 *pTerm, int nTerm, /* Term to seek to */ int flags, /* Mask of FTS5INDEX_XXX flags */ Fts5SegIter *pIter /* Object to populate */ ){ int nList = 0; const u8 *z = 0; int n = 0; Fts5Data *pLeaf = 0; assert( p->pHash ); assert( p->rc==SQLITE_OK ); if( pTerm==0 || (flags & FTS5INDEX_QUERY_SCAN) ){ const u8 *pList = 0; p->rc = sqlite3Fts5HashScanInit(p->pHash, (const char*)pTerm, nTerm); sqlite3Fts5HashScanEntry(p->pHash, (const char**)&z, &pList, &nList); n = (z ? (int)strlen((const char*)z) : 0); if( pList ){ pLeaf = fts5IdxMalloc(p, sizeof(Fts5Data)); if( pLeaf ){ pLeaf->p = (u8*)pList; } } }else{ p->rc = sqlite3Fts5HashQuery(p->pHash, sizeof(Fts5Data), (const char*)pTerm, nTerm, (void**)&pLeaf, &nList ); if( pLeaf ){ pLeaf->p = (u8*)&pLeaf[1]; } z = pTerm; n = nTerm; pIter->flags |= FTS5_SEGITER_ONETERM; } if( pLeaf ){ sqlite3Fts5BufferSet(&p->rc, &pIter->term, n, z); pLeaf->nn = pLeaf->szLeaf = nList; pIter->pLeaf = pLeaf; pIter->iLeafOffset = fts5GetVarint(pLeaf->p, (u64*)&pIter->iRowid); pIter->iEndofDoclist = pLeaf->nn; if( flags & FTS5INDEX_QUERY_DESC ){ pIter->flags |= FTS5_SEGITER_REVERSE; |
︙ | ︙ | |||
2520 2521 2522 2523 2524 2525 2526 | if( p1->pLeaf || p2->pLeaf ){ if( p1->pLeaf==0 ){ assert( pRes->iFirst==i2 ); }else if( p2->pLeaf==0 ){ assert( pRes->iFirst==i1 ); }else{ int nMin = MIN(p1->term.n, p2->term.n); | | | 2595 2596 2597 2598 2599 2600 2601 2602 2603 2604 2605 2606 2607 2608 2609 | if( p1->pLeaf || p2->pLeaf ){ if( p1->pLeaf==0 ){ assert( pRes->iFirst==i2 ); }else if( p2->pLeaf==0 ){ assert( pRes->iFirst==i1 ); }else{ int nMin = MIN(p1->term.n, p2->term.n); int res = fts5Memcmp(p1->term.p, p2->term.p, nMin); if( res==0 ) res = p1->term.n - p2->term.n; if( res==0 ){ assert( pRes->bTermEq==1 ); assert( p1->iRowid!=p2->iRowid ); res = ((p1->iRowid > p2->iRowid)==pIter->bRev) ? -1 : 1; }else{ |
︙ | ︙ | |||
2620 2621 2622 2623 2624 2625 2626 | if( p1->pLeaf==0 ){ /* If p1 is at EOF */ iRes = i2; }else if( p2->pLeaf==0 ){ /* If p2 is at EOF */ iRes = i1; }else{ int res = fts5BufferCompare(&p1->term, &p2->term); if( res==0 ){ | | | | 2695 2696 2697 2698 2699 2700 2701 2702 2703 2704 2705 2706 2707 2708 2709 2710 | if( p1->pLeaf==0 ){ /* If p1 is at EOF */ iRes = i2; }else if( p2->pLeaf==0 ){ /* If p2 is at EOF */ iRes = i1; }else{ int res = fts5BufferCompare(&p1->term, &p2->term); if( res==0 ){ assert_nc( i2>i1 ); assert_nc( i2!=0 ); pRes->bTermEq = 1; if( p1->iRowid==p2->iRowid ){ p1->bDel = p2->bDel; return i2; } res = ((p1->iRowid > p2->iRowid)==pIter->bRev) ? -1 : +1; } |
︙ | ︙ | |||
2661 2662 2663 2664 2665 2666 2667 | }else{ fts5DataRelease(pIter->pNextLeaf); pIter->pNextLeaf = 0; pIter->iLeafPgno = iLeafPgno-1; fts5SegIterNextPage(p, pIter); assert( p->rc!=SQLITE_OK || pIter->iLeafPgno==iLeafPgno ); | | | 2736 2737 2738 2739 2740 2741 2742 2743 2744 2745 2746 2747 2748 2749 2750 | }else{ fts5DataRelease(pIter->pNextLeaf); pIter->pNextLeaf = 0; pIter->iLeafPgno = iLeafPgno-1; fts5SegIterNextPage(p, pIter); assert( p->rc!=SQLITE_OK || pIter->iLeafPgno==iLeafPgno ); if( p->rc==SQLITE_OK && ALWAYS(pIter->pLeaf!=0) ){ int iOff; u8 *a = pIter->pLeaf->p; int n = pIter->pLeaf->szLeaf; iOff = fts5LeafFirstRowidOff(pIter->pLeaf); if( iOff<4 || iOff>=n ){ p->rc = FTS5_CORRUPT; |
︙ | ︙ | |||
2743 2744 2745 2746 2747 2748 2749 | */ static void fts5MultiIterFree(Fts5Iter *pIter){ if( pIter ){ int i; for(i=0; i<pIter->nSeg; i++){ fts5SegIterClear(&pIter->aSeg[i]); } | < | 2818 2819 2820 2821 2822 2823 2824 2825 2826 2827 2828 2829 2830 2831 | */ static void fts5MultiIterFree(Fts5Iter *pIter){ if( pIter ){ int i; for(i=0; i<pIter->nSeg; i++){ fts5SegIterClear(&pIter->aSeg[i]); } fts5BufferFree(&pIter->poslist); sqlite3_free(pIter); } } static void fts5MultiIterAdvanced( Fts5Index *p, /* FTS5 backend to iterate within */ |
︙ | ︙ | |||
3051 3052 3053 3054 3055 3056 3057 | int nRem = pSeg->nPos; /* Number of bytes still to come */ Fts5Data *pData = 0; u8 *pChunk = &pSeg->pLeaf->p[pSeg->iLeafOffset]; int nChunk = MIN(nRem, pSeg->pLeaf->szLeaf - pSeg->iLeafOffset); int pgno = pSeg->iLeafPgno; int pgnoSave = 0; | | > > > | 3125 3126 3127 3128 3129 3130 3131 3132 3133 3134 3135 3136 3137 3138 3139 3140 3141 3142 3143 3144 3145 3146 3147 3148 3149 3150 3151 3152 3153 3154 | int nRem = pSeg->nPos; /* Number of bytes still to come */ Fts5Data *pData = 0; u8 *pChunk = &pSeg->pLeaf->p[pSeg->iLeafOffset]; int nChunk = MIN(nRem, pSeg->pLeaf->szLeaf - pSeg->iLeafOffset); int pgno = pSeg->iLeafPgno; int pgnoSave = 0; /* This function does not work with detail=none databases. */ assert( p->pConfig->eDetail!=FTS5_DETAIL_NONE ); if( (pSeg->flags & FTS5_SEGITER_REVERSE)==0 ){ pgnoSave = pgno+1; } while( 1 ){ xChunk(p, pCtx, pChunk, nChunk); nRem -= nChunk; fts5DataRelease(pData); if( nRem<=0 ){ break; }else if( pSeg->pSeg==0 ){ p->rc = FTS5_CORRUPT; return; }else{ pgno++; pData = fts5LeafRead(p, FTS5_SEGMENT_ROWID(pSeg->pSeg->iSegid, pgno)); if( pData==0 ) break; pChunk = &pData->p[4]; nChunk = MIN(nRem, pData->szLeaf - 4); if( pgno==pgnoSave ){ |
︙ | ︙ | |||
3091 3092 3093 3094 3095 3096 3097 | */ static void fts5SegiterPoslist( Fts5Index *p, Fts5SegIter *pSeg, Fts5Colset *pColset, Fts5Buffer *pBuf ){ | > > | > > > | 3168 3169 3170 3171 3172 3173 3174 3175 3176 3177 3178 3179 3180 3181 3182 3183 3184 3185 3186 3187 | */ static void fts5SegiterPoslist( Fts5Index *p, Fts5SegIter *pSeg, Fts5Colset *pColset, Fts5Buffer *pBuf ){ assert( pBuf!=0 ); assert( pSeg!=0 ); if( 0==fts5BufferGrow(&p->rc, pBuf, pSeg->nPos+FTS5_DATA_ZERO_PADDING) ){ assert( pBuf->p!=0 ); assert( pBuf->nSpace >= pBuf->n+pSeg->nPos+FTS5_DATA_ZERO_PADDING ); memset(&pBuf->p[pBuf->n+pSeg->nPos], 0, FTS5_DATA_ZERO_PADDING); if( pColset==0 ){ fts5ChunkIterate(p, pSeg, (void*)pBuf, fts5PoslistCallback); }else{ if( p->pConfig->eDetail==FTS5_DETAIL_FULL ){ PoslistCallbackCtx sCtx; sCtx.pBuf = pBuf; sCtx.pColset = pColset; |
︙ | ︙ | |||
3114 3115 3116 3117 3118 3119 3120 | fts5ChunkIterate(p, pSeg, (void*)&sCtx, fts5PoslistOffsetsCallback); } } } } /* | | | | | < < < < < < < < < | | < < < < < | < < < < < < < < < < < < | < < < < < | | < | | > > > | > | > > > > > > > | > > > > > | > > > > > > | | > > > > | > > > > | > > > > > | > > > | 3196 3197 3198 3199 3200 3201 3202 3203 3204 3205 3206 3207 3208 3209 3210 3211 3212 3213 3214 3215 3216 3217 3218 3219 3220 3221 3222 3223 3224 3225 3226 3227 3228 3229 3230 3231 3232 3233 3234 3235 3236 3237 3238 3239 3240 3241 3242 3243 3244 3245 3246 3247 3248 3249 3250 3251 3252 3253 3254 3255 3256 3257 3258 3259 3260 3261 3262 3263 3264 3265 3266 3267 3268 3269 3270 3271 3272 3273 3274 3275 | fts5ChunkIterate(p, pSeg, (void*)&sCtx, fts5PoslistOffsetsCallback); } } } } /* ** Parameter pPos points to a buffer containing a position list, size nPos. ** This function filters it according to pColset (which must be non-NULL) ** and sets pIter->base.pData/nData to point to the new position list. ** If memory is required for the new position list, use buffer pIter->poslist. ** Or, if the new position list is a contiguous subset of the input, set ** pIter->base.pData/nData to point directly to it. ** ** This function is a no-op if *pRc is other than SQLITE_OK when it is ** called. If an OOM error is encountered, *pRc is set to SQLITE_NOMEM ** before returning. */ static void fts5IndexExtractColset( int *pRc, Fts5Colset *pColset, /* Colset to filter on */ const u8 *pPos, int nPos, /* Position list */ Fts5Iter *pIter ){ if( *pRc==SQLITE_OK ){ const u8 *p = pPos; const u8 *aCopy = p; const u8 *pEnd = &p[nPos]; /* One byte past end of position list */ int i = 0; int iCurrent = 0; if( pColset->nCol>1 && sqlite3Fts5BufferSize(pRc, &pIter->poslist, nPos) ){ return; } while( 1 ){ while( pColset->aiCol[i]<iCurrent ){ i++; if( i==pColset->nCol ){ pIter->base.pData = pIter->poslist.p; pIter->base.nData = pIter->poslist.n; return; } } /* Advance pointer p until it points to pEnd or an 0x01 byte that is ** not part of a varint */ while( p<pEnd && *p!=0x01 ){ while( *p++ & 0x80 ); } if( pColset->aiCol[i]==iCurrent ){ if( pColset->nCol==1 ){ pIter->base.pData = aCopy; pIter->base.nData = p-aCopy; return; } fts5BufferSafeAppendBlob(&pIter->poslist, aCopy, p-aCopy); } if( p>=pEnd ){ pIter->base.pData = pIter->poslist.p; pIter->base.nData = pIter->poslist.n; return; } aCopy = p++; iCurrent = *p++; if( iCurrent & 0x80 ){ p--; p += fts5GetVarint32(p, iCurrent); } } } } /* ** xSetOutputs callback used by detail=none tables. */ static void fts5IterSetOutputs_None(Fts5Iter *pIter, Fts5SegIter *pSeg){ assert( pIter->pIndex->pConfig->eDetail==FTS5_DETAIL_NONE ); |
︙ | ︙ | |||
3293 3294 3295 3296 3297 3298 3299 | assert( pIter->pIndex->pConfig->eDetail==FTS5_DETAIL_FULL ); assert( pColset ); if( pSeg->iLeafOffset+pSeg->nPos<=pSeg->pLeaf->szLeaf ){ /* All data is stored on the current page. Populate the output ** variables to point into the body of the page object. */ const u8 *a = &pSeg->pLeaf->p[pSeg->iLeafOffset]; | < < < < | | | < < < > | 3381 3382 3383 3384 3385 3386 3387 3388 3389 3390 3391 3392 3393 3394 3395 3396 3397 3398 3399 3400 3401 3402 3403 3404 3405 3406 3407 3408 3409 3410 | assert( pIter->pIndex->pConfig->eDetail==FTS5_DETAIL_FULL ); assert( pColset ); if( pSeg->iLeafOffset+pSeg->nPos<=pSeg->pLeaf->szLeaf ){ /* All data is stored on the current page. Populate the output ** variables to point into the body of the page object. */ const u8 *a = &pSeg->pLeaf->p[pSeg->iLeafOffset]; int *pRc = &pIter->pIndex->rc; fts5BufferZero(&pIter->poslist); fts5IndexExtractColset(pRc, pColset, a, pSeg->nPos, pIter); }else{ /* The data is distributed over two or more pages. Copy it into the ** Fts5Iter.poslist buffer and then set the output pointer to point ** to this buffer. */ fts5BufferZero(&pIter->poslist); fts5SegiterPoslist(pIter->pIndex, pSeg, pColset, &pIter->poslist); pIter->base.pData = pIter->poslist.p; pIter->base.nData = pIter->poslist.n; } } static void fts5IterSetOutputCb(int *pRc, Fts5Iter *pIter){ assert( pIter!=0 || (*pRc)!=SQLITE_OK ); if( *pRc==SQLITE_OK ){ Fts5Config *pConfig = pIter->pIndex->pConfig; if( pConfig->eDetail==FTS5_DETAIL_NONE ){ pIter->xSetOutputs = fts5IterSetOutputs_None; } else if( pIter->pColset==0 ){ |
︙ | ︙ | |||
3386 3387 3388 3389 3390 3391 3392 | nSeg = pStruct->nSegment; nSeg += (p->pHash ? 1 : 0); }else{ nSeg = MIN(pStruct->aLevel[iLevel].nSeg, nSegment); } } *ppOut = pNew = fts5MultiIterAlloc(p, nSeg); | | > > > < < | 3468 3469 3470 3471 3472 3473 3474 3475 3476 3477 3478 3479 3480 3481 3482 3483 3484 3485 3486 3487 3488 | nSeg = pStruct->nSegment; nSeg += (p->pHash ? 1 : 0); }else{ nSeg = MIN(pStruct->aLevel[iLevel].nSeg, nSegment); } } *ppOut = pNew = fts5MultiIterAlloc(p, nSeg); if( pNew==0 ){ assert( p->rc!=SQLITE_OK ); goto fts5MultiIterNew_post_check; } pNew->bRev = (0!=(flags & FTS5INDEX_QUERY_DESC)); pNew->bSkipEmpty = (0!=(flags & FTS5INDEX_QUERY_SKIPEMPTY)); pNew->pColset = pColset; if( (flags & FTS5INDEX_QUERY_NOOUTPUT)==0 ){ fts5IterSetOutputCb(&p->rc, pNew); } /* Initialize each of the component segment iterators. */ if( p->rc==SQLITE_OK ){ if( iLevel<0 ){ |
︙ | ︙ | |||
3452 3453 3454 3455 3456 3457 3458 3459 3460 3461 3462 3463 3464 3465 | pNew->xSetOutputs(pNew, pSeg); } }else{ fts5MultiIterFree(pNew); *ppOut = 0; } } /* ** Create an Fts5Iter that iterates through the doclist provided ** as the second argument. */ static void fts5MultiIterNew2( | > > > > | 3535 3536 3537 3538 3539 3540 3541 3542 3543 3544 3545 3546 3547 3548 3549 3550 3551 3552 | pNew->xSetOutputs(pNew, pSeg); } }else{ fts5MultiIterFree(pNew); *ppOut = 0; } fts5MultiIterNew_post_check: assert( (*ppOut)!=0 || p->rc!=SQLITE_OK ); return; } /* ** Create an Fts5Iter that iterates through the doclist provided ** as the second argument. */ static void fts5MultiIterNew2( |
︙ | ︙ | |||
3499 3500 3501 3502 3503 3504 3505 | } /* ** Return true if the iterator is at EOF or if an error has occurred. ** False otherwise. */ static int fts5MultiIterEof(Fts5Index *p, Fts5Iter *pIter){ | > | | 3586 3587 3588 3589 3590 3591 3592 3593 3594 3595 3596 3597 3598 3599 3600 3601 | } /* ** Return true if the iterator is at EOF or if an error has occurred. ** False otherwise. */ static int fts5MultiIterEof(Fts5Index *p, Fts5Iter *pIter){ assert( pIter!=0 || p->rc!=SQLITE_OK ); assert( p->rc!=SQLITE_OK || (pIter->aSeg[ pIter->aFirst[1].iFirst ].pLeaf==0)==pIter->base.bEof ); return (p->rc || pIter->base.bEof); } /* ** Return the rowid of the entry that the iterator currently points |
︙ | ︙ | |||
3569 3570 3571 3572 3573 3574 3575 | int iLvl, iSeg; int i; u32 mask; memset(aUsed, 0, sizeof(aUsed)); for(iLvl=0; iLvl<pStruct->nLevel; iLvl++){ for(iSeg=0; iSeg<pStruct->aLevel[iLvl].nSeg; iSeg++){ int iId = pStruct->aLevel[iLvl].aSeg[iSeg].iSegid; | | | | | | | | 3657 3658 3659 3660 3661 3662 3663 3664 3665 3666 3667 3668 3669 3670 3671 3672 3673 3674 3675 3676 3677 3678 3679 3680 3681 3682 3683 3684 3685 3686 3687 3688 3689 3690 3691 3692 3693 3694 3695 3696 | int iLvl, iSeg; int i; u32 mask; memset(aUsed, 0, sizeof(aUsed)); for(iLvl=0; iLvl<pStruct->nLevel; iLvl++){ for(iSeg=0; iSeg<pStruct->aLevel[iLvl].nSeg; iSeg++){ int iId = pStruct->aLevel[iLvl].aSeg[iSeg].iSegid; if( iId<=FTS5_MAX_SEGMENT && iId>0 ){ aUsed[(iId-1) / 32] |= (u32)1 << ((iId-1) % 32); } } } for(i=0; aUsed[i]==0xFFFFFFFF; i++); mask = aUsed[i]; for(iSegid=0; mask & ((u32)1 << iSegid); iSegid++); iSegid += 1 + i*32; #ifdef SQLITE_DEBUG for(iLvl=0; iLvl<pStruct->nLevel; iLvl++){ for(iSeg=0; iSeg<pStruct->aLevel[iLvl].nSeg; iSeg++){ assert_nc( iSegid!=pStruct->aLevel[iLvl].aSeg[iSeg].iSegid ); } } assert_nc( iSegid>0 && iSegid<=FTS5_MAX_SEGMENT ); { sqlite3_stmt *pIdxSelect = fts5IdxSelectStmt(p); if( p->rc==SQLITE_OK ){ u8 aBlob[2] = {0xff, 0xff}; sqlite3_bind_int(pIdxSelect, 1, iSegid); sqlite3_bind_blob(pIdxSelect, 2, aBlob, 2, SQLITE_STATIC); assert_nc( sqlite3_step(pIdxSelect)!=SQLITE_ROW ); p->rc = sqlite3_reset(pIdxSelect); sqlite3_bind_null(pIdxSelect, 2); } } #endif } } |
︙ | ︙ | |||
3664 3665 3666 3667 3668 3669 3670 | */ static int fts5WriteDlidxGrow( Fts5Index *p, Fts5SegWriter *pWriter, int nLvl ){ if( p->rc==SQLITE_OK && nLvl>=pWriter->nDlidx ){ | | | | 3752 3753 3754 3755 3756 3757 3758 3759 3760 3761 3762 3763 3764 3765 3766 3767 3768 3769 3770 3771 3772 | */ static int fts5WriteDlidxGrow( Fts5Index *p, Fts5SegWriter *pWriter, int nLvl ){ if( p->rc==SQLITE_OK && nLvl>=pWriter->nDlidx ){ Fts5DlidxWriter *aDlidx = (Fts5DlidxWriter*)sqlite3_realloc64( pWriter->aDlidx, sizeof(Fts5DlidxWriter) * nLvl ); if( aDlidx==0 ){ p->rc = SQLITE_NOMEM; }else{ size_t nByte = sizeof(Fts5DlidxWriter) * (nLvl - pWriter->nDlidx); memset(&aDlidx[pWriter->nDlidx], 0, nByte); pWriter->aDlidx = aDlidx; pWriter->nDlidx = nLvl; } } return p->rc; } |
︙ | ︙ | |||
3743 3744 3745 3746 3747 3748 3749 | */ static void fts5WriteBtreeTerm( Fts5Index *p, /* FTS5 backend object */ Fts5SegWriter *pWriter, /* Writer object */ int nTerm, const u8 *pTerm /* First term on new page */ ){ fts5WriteFlushBtree(p, pWriter); | > | | > | 3831 3832 3833 3834 3835 3836 3837 3838 3839 3840 3841 3842 3843 3844 3845 3846 3847 3848 | */ static void fts5WriteBtreeTerm( Fts5Index *p, /* FTS5 backend object */ Fts5SegWriter *pWriter, /* Writer object */ int nTerm, const u8 *pTerm /* First term on new page */ ){ fts5WriteFlushBtree(p, pWriter); if( p->rc==SQLITE_OK ){ fts5BufferSet(&p->rc, &pWriter->btterm, nTerm, pTerm); pWriter->iBtPage = pWriter->writer.pgno; } } /* ** This function is called when flushing a leaf page that contains no ** terms at all to disk. */ static void fts5WriteBtreeNoTerm( |
︙ | ︙ | |||
3895 3896 3897 3898 3899 3900 3901 3902 3903 3904 3905 3906 3907 3908 3909 3910 3911 3912 3913 3914 3915 3916 3917 | Fts5Index *p, Fts5SegWriter *pWriter, int nTerm, const u8 *pTerm ){ int nPrefix; /* Bytes of prefix compression for term */ Fts5PageWriter *pPage = &pWriter->writer; Fts5Buffer *pPgidx = &pWriter->writer.pgidx; assert( p->rc==SQLITE_OK ); assert( pPage->buf.n>=4 ); assert( pPage->buf.n>4 || pWriter->bFirstTermInPage ); /* If the current leaf page is full, flush it to disk. */ if( (pPage->buf.n + pPgidx->n + nTerm + 2)>=p->pConfig->pgsz ){ if( pPage->buf.n>4 ){ fts5WriteFlushLeaf(p, pWriter); } fts5BufferGrow(&p->rc, &pPage->buf, nTerm+FTS5_DATA_PADDING); } /* TODO1: Updating pgidx here. */ pPgidx->n += sqlite3Fts5PutVarint( &pPgidx->p[pPgidx->n], pPage->buf.n - pPage->iPrevPgidx | > > | 3985 3986 3987 3988 3989 3990 3991 3992 3993 3994 3995 3996 3997 3998 3999 4000 4001 4002 4003 4004 4005 4006 4007 4008 4009 | Fts5Index *p, Fts5SegWriter *pWriter, int nTerm, const u8 *pTerm ){ int nPrefix; /* Bytes of prefix compression for term */ Fts5PageWriter *pPage = &pWriter->writer; Fts5Buffer *pPgidx = &pWriter->writer.pgidx; int nMin = MIN(pPage->term.n, nTerm); assert( p->rc==SQLITE_OK ); assert( pPage->buf.n>=4 ); assert( pPage->buf.n>4 || pWriter->bFirstTermInPage ); /* If the current leaf page is full, flush it to disk. */ if( (pPage->buf.n + pPgidx->n + nTerm + 2)>=p->pConfig->pgsz ){ if( pPage->buf.n>4 ){ fts5WriteFlushLeaf(p, pWriter); if( p->rc!=SQLITE_OK ) return; } fts5BufferGrow(&p->rc, &pPage->buf, nTerm+FTS5_DATA_PADDING); } /* TODO1: Updating pgidx here. */ pPgidx->n += sqlite3Fts5PutVarint( &pPgidx->p[pPgidx->n], pPage->buf.n - pPage->iPrevPgidx |
︙ | ︙ | |||
3936 3937 3938 3939 3940 3941 3942 | ** Usually, the previous term is available in pPage->term. The exception ** is if this is the first term written in an incremental-merge step. ** In this case the previous term is not available, so just write a ** copy of (pTerm/nTerm) into the parent node. This is slightly ** inefficient, but still correct. */ int n = nTerm; if( pPage->term.n ){ | | > | | 4028 4029 4030 4031 4032 4033 4034 4035 4036 4037 4038 4039 4040 4041 4042 4043 4044 4045 4046 4047 4048 4049 | ** Usually, the previous term is available in pPage->term. The exception ** is if this is the first term written in an incremental-merge step. ** In this case the previous term is not available, so just write a ** copy of (pTerm/nTerm) into the parent node. This is slightly ** inefficient, but still correct. */ int n = nTerm; if( pPage->term.n ){ n = 1 + fts5PrefixCompress(nMin, pPage->term.p, pTerm); } fts5WriteBtreeTerm(p, pWriter, n, pTerm); if( p->rc!=SQLITE_OK ) return; pPage = &pWriter->writer; } }else{ nPrefix = fts5PrefixCompress(nMin, pPage->term.p, pTerm); fts5BufferAppendVarint(&p->rc, &pPage->buf, nPrefix); } /* Append the number of bytes of new data, then the term data itself ** to the page. */ fts5BufferAppendVarint(&p->rc, &pPage->buf, nTerm - nPrefix); fts5BufferAppendBlob(&p->rc, &pPage->buf, nTerm - nPrefix, &pTerm[nPrefix]); |
︙ | ︙ | |||
3989 3990 3991 3992 3993 3994 3995 | fts5WriteDlidxAppend(p, pWriter, iRowid); } /* Write the rowid. */ if( pWriter->bFirstRowidInDoclist || pWriter->bFirstRowidInPage ){ fts5BufferAppendVarint(&p->rc, &pPage->buf, iRowid); }else{ | | | > > | 4082 4083 4084 4085 4086 4087 4088 4089 4090 4091 4092 4093 4094 4095 4096 4097 4098 4099 | fts5WriteDlidxAppend(p, pWriter, iRowid); } /* Write the rowid. */ if( pWriter->bFirstRowidInDoclist || pWriter->bFirstRowidInPage ){ fts5BufferAppendVarint(&p->rc, &pPage->buf, iRowid); }else{ assert_nc( p->rc || iRowid>pWriter->iPrevRowid ); fts5BufferAppendVarint(&p->rc, &pPage->buf, (u64)iRowid - (u64)pWriter->iPrevRowid ); } pWriter->iPrevRowid = iRowid; pWriter->bFirstRowidInDoclist = 0; pWriter->bFirstRowidInPage = 0; } } |
︙ | ︙ | |||
4111 4112 4113 4114 4115 4116 4117 | ** incremental merge operation. This function is called if the incremental ** merge step has finished but the input has not been completely exhausted. */ static void fts5TrimSegments(Fts5Index *p, Fts5Iter *pIter){ int i; Fts5Buffer buf; memset(&buf, 0, sizeof(Fts5Buffer)); | | | > > > > > > > | | | | | | | | | | | | | | > | | | | | | | < | | | > > | 4206 4207 4208 4209 4210 4211 4212 4213 4214 4215 4216 4217 4218 4219 4220 4221 4222 4223 4224 4225 4226 4227 4228 4229 4230 4231 4232 4233 4234 4235 4236 4237 4238 4239 4240 4241 4242 4243 4244 4245 4246 4247 4248 4249 4250 4251 4252 4253 4254 4255 4256 4257 4258 4259 4260 4261 4262 4263 4264 4265 4266 4267 4268 4269 4270 4271 4272 4273 4274 4275 | ** incremental merge operation. This function is called if the incremental ** merge step has finished but the input has not been completely exhausted. */ static void fts5TrimSegments(Fts5Index *p, Fts5Iter *pIter){ int i; Fts5Buffer buf; memset(&buf, 0, sizeof(Fts5Buffer)); for(i=0; i<pIter->nSeg && p->rc==SQLITE_OK; i++){ Fts5SegIter *pSeg = &pIter->aSeg[i]; if( pSeg->pSeg==0 ){ /* no-op */ }else if( pSeg->pLeaf==0 ){ /* All keys from this input segment have been transfered to the output. ** Set both the first and last page-numbers to 0 to indicate that the ** segment is now empty. */ pSeg->pSeg->pgnoLast = 0; pSeg->pSeg->pgnoFirst = 0; }else{ int iOff = pSeg->iTermLeafOffset; /* Offset on new first leaf page */ i64 iLeafRowid; Fts5Data *pData; int iId = pSeg->pSeg->iSegid; u8 aHdr[4] = {0x00, 0x00, 0x00, 0x00}; iLeafRowid = FTS5_SEGMENT_ROWID(iId, pSeg->iTermLeafPgno); pData = fts5LeafRead(p, iLeafRowid); if( pData ){ if( iOff>pData->szLeaf ){ /* This can occur if the pages that the segments occupy overlap - if ** a single page has been assigned to more than one segment. In ** this case a prior iteration of this loop may have corrupted the ** segment currently being trimmed. */ p->rc = FTS5_CORRUPT; }else{ fts5BufferZero(&buf); fts5BufferGrow(&p->rc, &buf, pData->nn); fts5BufferAppendBlob(&p->rc, &buf, sizeof(aHdr), aHdr); fts5BufferAppendVarint(&p->rc, &buf, pSeg->term.n); fts5BufferAppendBlob(&p->rc, &buf, pSeg->term.n, pSeg->term.p); fts5BufferAppendBlob(&p->rc, &buf, pData->szLeaf-iOff,&pData->p[iOff]); if( p->rc==SQLITE_OK ){ /* Set the szLeaf field */ fts5PutU16(&buf.p[2], (u16)buf.n); } /* Set up the new page-index array */ fts5BufferAppendVarint(&p->rc, &buf, 4); if( pSeg->iLeafPgno==pSeg->iTermLeafPgno && pSeg->iEndofDoclist<pData->szLeaf && pSeg->iPgidxOff<=pData->nn ){ int nDiff = pData->szLeaf - pSeg->iEndofDoclist; fts5BufferAppendVarint(&p->rc, &buf, buf.n - 1 - nDiff - 4); fts5BufferAppendBlob(&p->rc, &buf, pData->nn - pSeg->iPgidxOff, &pData->p[pSeg->iPgidxOff] ); } pSeg->pSeg->pgnoFirst = pSeg->iTermLeafPgno; fts5DataDelete(p, FTS5_SEGMENT_ROWID(iId, 1), iLeafRowid); fts5DataWrite(p, iLeafRowid, buf.p, buf.n); } fts5DataRelease(pData); } } } fts5BufferFree(&buf); } static void fts5MergeChunkCallback( |
︙ | ︙ | |||
4249 4250 4251 4252 4253 4254 4255 | ){ Fts5SegIter *pSegIter = &pIter->aSeg[ pIter->aFirst[1].iFirst ]; int nPos; /* position-list size field value */ int nTerm; const u8 *pTerm; pTerm = fts5MultiIterTerm(pIter, &nTerm); | | | 4353 4354 4355 4356 4357 4358 4359 4360 4361 4362 4363 4364 4365 4366 4367 | ){ Fts5SegIter *pSegIter = &pIter->aSeg[ pIter->aFirst[1].iFirst ]; int nPos; /* position-list size field value */ int nTerm; const u8 *pTerm; pTerm = fts5MultiIterTerm(pIter, &nTerm); if( nTerm!=term.n || fts5Memcmp(pTerm, term.p, nTerm) ){ if( pnRem && writer.nLeafWritten>nRem ){ break; } fts5BufferSet(&p->rc, &term, nTerm, pTerm); bTermWritten =0; } |
︙ | ︙ | |||
4289 4290 4291 4292 4293 4294 4295 4296 4297 4298 4299 4300 4301 4302 | } } /* Flush the last leaf page to disk. Set the output segment b-tree height ** and last leaf page number at the same time. */ fts5WriteFinish(p, &writer, &pSeg->pgnoLast); if( fts5MultiIterEof(p, pIter) ){ int i; /* Remove the redundant segments from the %_data table */ for(i=0; i<nInput; i++){ fts5DataRemoveSegment(p, pLvl->aSeg[i].iSegid); } | > | 4393 4394 4395 4396 4397 4398 4399 4400 4401 4402 4403 4404 4405 4406 4407 | } } /* Flush the last leaf page to disk. Set the output segment b-tree height ** and last leaf page number at the same time. */ fts5WriteFinish(p, &writer, &pSeg->pgnoLast); assert( pIter!=0 || p->rc!=SQLITE_OK ); if( fts5MultiIterEof(p, pIter) ){ int i; /* Remove the redundant segments from the %_data table */ for(i=0; i<nInput; i++){ fts5DataRemoveSegment(p, pLvl->aSeg[i].iSegid); } |
︙ | ︙ | |||
4389 4390 4391 4392 4393 4394 4395 | ** already occurred, this function is a no-op. */ static void fts5IndexAutomerge( Fts5Index *p, /* FTS5 backend object */ Fts5Structure **ppStruct, /* IN/OUT: Current structure of index */ int nLeaf /* Number of output leaves just written */ ){ | | | 4494 4495 4496 4497 4498 4499 4500 4501 4502 4503 4504 4505 4506 4507 4508 | ** already occurred, this function is a no-op. */ static void fts5IndexAutomerge( Fts5Index *p, /* FTS5 backend object */ Fts5Structure **ppStruct, /* IN/OUT: Current structure of index */ int nLeaf /* Number of output leaves just written */ ){ if( p->rc==SQLITE_OK && p->pConfig->nAutomerge>0 && ALWAYS((*ppStruct)!=0) ){ Fts5Structure *pStruct = *ppStruct; u64 nWrite; /* Initial value of write-counter */ int nWork; /* Number of work-quanta to perform */ int nRem; /* Number of leaf pages left to write */ /* Update the write-counter. While doing so, set nWork. */ nWrite = pStruct->nWriteCounter; |
︙ | ︙ | |||
4504 4505 4506 4507 4508 4509 4510 4511 4512 4513 4514 4515 4516 4517 | const char *zTerm; /* Buffer containing term */ const u8 *pDoclist; /* Pointer to doclist for this term */ int nDoclist; /* Size of doclist in bytes */ /* Write the term for this entry to disk. */ sqlite3Fts5HashScanEntry(pHash, &zTerm, &pDoclist, &nDoclist); fts5WriteAppendTerm(p, &writer, (int)strlen(zTerm), (const u8*)zTerm); assert( writer.bFirstRowidInPage==0 ); if( pgsz>=(pBuf->n + pPgidx->n + nDoclist + 1) ){ /* The entire doclist will fit on the current leaf. */ fts5BufferSafeAppendBlob(pBuf, pDoclist, nDoclist); }else{ i64 iRowid = 0; | > | | > | 4609 4610 4611 4612 4613 4614 4615 4616 4617 4618 4619 4620 4621 4622 4623 4624 4625 4626 4627 4628 4629 4630 4631 4632 4633 4634 4635 4636 4637 4638 4639 4640 4641 4642 4643 4644 4645 4646 | const char *zTerm; /* Buffer containing term */ const u8 *pDoclist; /* Pointer to doclist for this term */ int nDoclist; /* Size of doclist in bytes */ /* Write the term for this entry to disk. */ sqlite3Fts5HashScanEntry(pHash, &zTerm, &pDoclist, &nDoclist); fts5WriteAppendTerm(p, &writer, (int)strlen(zTerm), (const u8*)zTerm); if( p->rc!=SQLITE_OK ) break; assert( writer.bFirstRowidInPage==0 ); if( pgsz>=(pBuf->n + pPgidx->n + nDoclist + 1) ){ /* The entire doclist will fit on the current leaf. */ fts5BufferSafeAppendBlob(pBuf, pDoclist, nDoclist); }else{ i64 iRowid = 0; u64 iDelta = 0; int iOff = 0; /* The entire doclist will not fit on this leaf. The following ** loop iterates through the poslists that make up the current ** doclist. */ while( p->rc==SQLITE_OK && iOff<nDoclist ){ iOff += fts5GetVarint(&pDoclist[iOff], &iDelta); iRowid += iDelta; if( writer.bFirstRowidInPage ){ fts5PutU16(&pBuf->p[0], (u16)pBuf->n); /* first rowid on page */ pBuf->n += sqlite3Fts5PutVarint(&pBuf->p[pBuf->n], iRowid); writer.bFirstRowidInPage = 0; fts5WriteDlidxAppend(p, &writer, iRowid); if( p->rc!=SQLITE_OK ) break; }else{ pBuf->n += sqlite3Fts5PutVarint(&pBuf->p[pBuf->n], iDelta); } assert( pBuf->n<=pBuf->nSpace ); if( eDetail==FTS5_DETAIL_NONE ){ if( iOff<nDoclist && pDoclist[iOff]==0 ){ |
︙ | ︙ | |||
4583 4584 4585 4586 4587 4588 4589 | } } } /* TODO2: Doclist terminator written here. */ /* pBuf->p[pBuf->n++] = '\0'; */ assert( pBuf->n<=pBuf->nSpace ); | | | 4690 4691 4692 4693 4694 4695 4696 4697 4698 4699 4700 4701 4702 4703 4704 | } } } /* TODO2: Doclist terminator written here. */ /* pBuf->p[pBuf->n++] = '\0'; */ assert( pBuf->n<=pBuf->nSpace ); if( p->rc==SQLITE_OK ) sqlite3Fts5HashScanNext(pHash); } sqlite3Fts5HashClear(pHash); fts5WriteFinish(p, &writer, &pgnoLast); /* Update the Fts5Structure. It is written back to the database by the ** fts5StructureRelease() call below. */ if( pStruct->nLevel==0 ){ |
︙ | ︙ | |||
4627 4628 4629 4630 4631 4632 4633 | } static Fts5Structure *fts5IndexOptimizeStruct( Fts5Index *p, Fts5Structure *pStruct ){ Fts5Structure *pNew = 0; | | | 4734 4735 4736 4737 4738 4739 4740 4741 4742 4743 4744 4745 4746 4747 4748 | } static Fts5Structure *fts5IndexOptimizeStruct( Fts5Index *p, Fts5Structure *pStruct ){ Fts5Structure *pNew = 0; sqlite3_int64 nByte = sizeof(Fts5Structure); int nSeg = pStruct->nSegment; int i; /* Figure out if this structure requires optimization. A structure does ** not require optimization if either: ** ** + it consists of fewer than two segments, or |
︙ | ︙ | |||
4742 4743 4744 4745 4746 4747 4748 | fts5StructureRelease(pStruct); } return fts5IndexReturn(p); } static void fts5AppendRowid( Fts5Index *p, | | | > | > | > > > > | | | > | | | | | 4849 4850 4851 4852 4853 4854 4855 4856 4857 4858 4859 4860 4861 4862 4863 4864 4865 4866 4867 4868 4869 4870 4871 4872 4873 4874 4875 4876 4877 4878 4879 4880 4881 4882 4883 4884 4885 4886 4887 4888 4889 4890 4891 4892 4893 4894 4895 4896 4897 4898 4899 4900 4901 4902 4903 4904 4905 4906 4907 4908 4909 4910 4911 4912 4913 4914 4915 4916 4917 4918 4919 4920 4921 4922 4923 4924 4925 4926 4927 4928 4929 4930 4931 4932 4933 4934 4935 4936 4937 4938 4939 4940 4941 4942 4943 4944 4945 4946 4947 4948 4949 4950 4951 | fts5StructureRelease(pStruct); } return fts5IndexReturn(p); } static void fts5AppendRowid( Fts5Index *p, u64 iDelta, Fts5Iter *pUnused, Fts5Buffer *pBuf ){ UNUSED_PARAM(pUnused); fts5BufferAppendVarint(&p->rc, pBuf, iDelta); } static void fts5AppendPoslist( Fts5Index *p, u64 iDelta, Fts5Iter *pMulti, Fts5Buffer *pBuf ){ int nData = pMulti->base.nData; int nByte = nData + 9 + 9 + FTS5_DATA_ZERO_PADDING; assert( nData>0 ); if( p->rc==SQLITE_OK && 0==fts5BufferGrow(&p->rc, pBuf, nByte) ){ fts5BufferSafeAppendVarint(pBuf, iDelta); fts5BufferSafeAppendVarint(pBuf, nData*2); fts5BufferSafeAppendBlob(pBuf, pMulti->base.pData, nData); memset(&pBuf->p[pBuf->n], 0, FTS5_DATA_ZERO_PADDING); } } static void fts5DoclistIterNext(Fts5DoclistIter *pIter){ u8 *p = pIter->aPoslist + pIter->nSize + pIter->nPoslist; assert( pIter->aPoslist || (p==0 && pIter->aPoslist==0) ); if( p>=pIter->aEof ){ pIter->aPoslist = 0; }else{ i64 iDelta; p += fts5GetVarint(p, (u64*)&iDelta); pIter->iRowid += iDelta; /* Read position list size */ if( p[0] & 0x80 ){ int nPos; pIter->nSize = fts5GetVarint32(p, nPos); pIter->nPoslist = (nPos>>1); }else{ pIter->nPoslist = ((int)(p[0])) >> 1; pIter->nSize = 1; } pIter->aPoslist = p; if( &pIter->aPoslist[pIter->nPoslist]>pIter->aEof ){ pIter->aPoslist = 0; } } } static void fts5DoclistIterInit( Fts5Buffer *pBuf, Fts5DoclistIter *pIter ){ memset(pIter, 0, sizeof(*pIter)); if( pBuf->n>0 ){ pIter->aPoslist = pBuf->p; pIter->aEof = &pBuf->p[pBuf->n]; fts5DoclistIterNext(pIter); } } #if 0 /* ** Append a doclist to buffer pBuf. ** ** This function assumes that space within the buffer has already been ** allocated. */ static void fts5MergeAppendDocid( Fts5Buffer *pBuf, /* Buffer to write to */ i64 *piLastRowid, /* IN/OUT: Previous rowid written (if any) */ i64 iRowid /* Rowid to append */ ){ assert( pBuf->n!=0 || (*piLastRowid)==0 ); fts5BufferSafeAppendVarint(pBuf, iRowid - *piLastRowid); *piLastRowid = iRowid; } #endif #define fts5MergeAppendDocid(pBuf, iLastRowid, iRowid) { \ assert( (pBuf)->n!=0 || (iLastRowid)==0 ); \ fts5BufferSafeAppendVarint((pBuf), (u64)(iRowid) - (u64)(iLastRowid)); \ (iLastRowid) = (iRowid); \ } /* ** Swap the contents of buffer *p1 with that of *p2. */ static void fts5BufferSwap(Fts5Buffer *p1, Fts5Buffer *p2){ Fts5Buffer tmp = *p1; |
︙ | ︙ | |||
4853 4854 4855 4856 4857 4858 4859 | /* ** This is the equivalent of fts5MergePrefixLists() for detail=none mode. ** In this case the buffers consist of a delta-encoded list of rowids only. */ static void fts5MergeRowidLists( Fts5Index *p, /* FTS5 backend object */ Fts5Buffer *p1, /* First list to merge */ | > | | > > > | 4967 4968 4969 4970 4971 4972 4973 4974 4975 4976 4977 4978 4979 4980 4981 4982 4983 4984 4985 4986 4987 4988 4989 4990 4991 4992 4993 4994 | /* ** This is the equivalent of fts5MergePrefixLists() for detail=none mode. ** In this case the buffers consist of a delta-encoded list of rowids only. */ static void fts5MergeRowidLists( Fts5Index *p, /* FTS5 backend object */ Fts5Buffer *p1, /* First list to merge */ int nBuf, /* Number of entries in apBuf[] */ Fts5Buffer *aBuf /* Array of other lists to merge into p1 */ ){ int i1 = 0; int i2 = 0; i64 iRowid1 = 0; i64 iRowid2 = 0; i64 iOut = 0; Fts5Buffer *p2 = &aBuf[0]; Fts5Buffer out; (void)nBuf; memset(&out, 0, sizeof(out)); assert( nBuf==1 ); sqlite3Fts5BufferSize(&p->rc, &out, p1->n + p2->n); if( p->rc ) return; fts5NextRowid(p1, &i1, &iRowid1); fts5NextRowid(p2, &i2, &iRowid2); while( i1>=0 || i2>=0 ){ if( i1>=0 && (i2<0 || iRowid1<iRowid2) ){ |
︙ | ︙ | |||
4888 4889 4890 4891 4892 4893 4894 4895 | fts5NextRowid(p2, &i2, &iRowid2); } } fts5BufferSwap(&out, p1); fts5BufferFree(&out); } | > > > > > | > > | > > > > > > > > > > > > > | > > | > > > > > > > | > > | | > > > > | < > > > > | | < > | | > > > > > > > > > > > > > > > > | | | | | | > > > > > > | < < | < < | < < < | | | > > | < | | < > | < | > | | | | > | | > | < < | < < < < < < < > > > | | > > > | < | < < | > > > > > > | | < > | | | < | > > > | | | < < | > | < > > | | | | | > | | < < | | > | | | | | < | | > | | > | > > > < < < | < < | | < > > > | | > | | > > > > > > > > > > > > > > > > > > > > > > > > > | > | | | | > > > | > | > | > | > | > | > | | 5006 5007 5008 5009 5010 5011 5012 5013 5014 5015 5016 5017 5018 5019 5020 5021 5022 5023 5024 5025 5026 5027 5028 5029 5030 5031 5032 5033 5034 5035 5036 5037 5038 5039 5040 5041 5042 5043 5044 5045 5046 5047 5048 5049 5050 5051 5052 5053 5054 5055 5056 5057 5058 5059 5060 5061 5062 5063 5064 5065 5066 5067 5068 5069 5070 5071 5072 5073 5074 5075 5076 5077 5078 5079 5080 5081 5082 5083 5084 5085 5086 5087 5088 5089 5090 5091 5092 5093 5094 5095 5096 5097 5098 5099 5100 5101 5102 5103 5104 5105 5106 5107 5108 5109 5110 5111 5112 5113 5114 5115 5116 5117 5118 5119 5120 5121 5122 5123 5124 5125 5126 5127 5128 5129 5130 5131 5132 5133 5134 5135 5136 5137 5138 5139 5140 5141 5142 5143 5144 5145 5146 5147 5148 5149 5150 5151 5152 5153 5154 5155 5156 5157 5158 5159 5160 5161 5162 5163 5164 5165 5166 5167 5168 5169 5170 5171 5172 5173 5174 5175 5176 5177 5178 5179 5180 5181 5182 5183 5184 5185 5186 5187 5188 5189 5190 5191 5192 5193 5194 5195 5196 5197 5198 5199 5200 5201 5202 5203 5204 5205 5206 5207 5208 5209 5210 5211 5212 5213 5214 5215 5216 5217 5218 5219 5220 5221 5222 5223 5224 5225 5226 5227 5228 5229 5230 5231 5232 5233 5234 5235 5236 5237 5238 5239 5240 5241 5242 5243 5244 5245 5246 5247 5248 5249 5250 5251 5252 5253 5254 5255 5256 5257 5258 5259 5260 5261 5262 5263 5264 5265 5266 5267 5268 5269 5270 5271 5272 5273 5274 5275 5276 5277 5278 5279 5280 5281 5282 5283 5284 5285 5286 5287 5288 5289 5290 5291 5292 5293 5294 5295 5296 5297 5298 5299 5300 5301 5302 5303 5304 5305 5306 5307 5308 5309 5310 5311 5312 5313 5314 5315 5316 5317 5318 5319 5320 5321 5322 5323 5324 5325 | fts5NextRowid(p2, &i2, &iRowid2); } } fts5BufferSwap(&out, p1); fts5BufferFree(&out); } typedef struct PrefixMerger PrefixMerger; struct PrefixMerger { Fts5DoclistIter iter; /* Doclist iterator */ i64 iPos; /* For iterating through a position list */ int iOff; u8 *aPos; PrefixMerger *pNext; /* Next in docid/poslist order */ }; static void fts5PrefixMergerInsertByRowid( PrefixMerger **ppHead, PrefixMerger *p ){ if( p->iter.aPoslist ){ PrefixMerger **pp = ppHead; while( *pp && p->iter.iRowid>(*pp)->iter.iRowid ){ pp = &(*pp)->pNext; } p->pNext = *pp; *pp = p; } } static void fts5PrefixMergerInsertByPosition( PrefixMerger **ppHead, PrefixMerger *p ){ if( p->iPos>=0 ){ PrefixMerger **pp = ppHead; while( *pp && p->iPos>(*pp)->iPos ){ pp = &(*pp)->pNext; } p->pNext = *pp; *pp = p; } } /* ** Array aBuf[] contains nBuf doclists. These are all merged in with the ** doclist in buffer p1. */ static void fts5MergePrefixLists( Fts5Index *p, /* FTS5 backend object */ Fts5Buffer *p1, /* First list to merge */ int nBuf, /* Number of buffers in array aBuf[] */ Fts5Buffer *aBuf /* Other lists to merge in */ ){ #define fts5PrefixMergerNextPosition(p) \ sqlite3Fts5PoslistNext64((p)->aPos,(p)->iter.nPoslist,&(p)->iOff,&(p)->iPos) #define FTS5_MERGE_NLIST 16 PrefixMerger aMerger[FTS5_MERGE_NLIST]; PrefixMerger *pHead = 0; int i; int nOut = 0; Fts5Buffer out = {0, 0, 0}; Fts5Buffer tmp = {0, 0, 0}; i64 iLastRowid = 0; /* Initialize a doclist-iterator for each input buffer. Arrange them in ** a linked-list starting at pHead in ascending order of rowid. Avoid ** linking any iterators already at EOF into the linked list at all. */ assert( nBuf+1<=(int)(sizeof(aMerger)/sizeof(aMerger[0])) ); memset(aMerger, 0, sizeof(PrefixMerger)*(nBuf+1)); pHead = &aMerger[nBuf]; fts5DoclistIterInit(p1, &pHead->iter); for(i=0; i<nBuf; i++){ fts5DoclistIterInit(&aBuf[i], &aMerger[i].iter); fts5PrefixMergerInsertByRowid(&pHead, &aMerger[i]); nOut += aBuf[i].n; } if( nOut==0 ) return; nOut += p1->n + 9 + 10*nBuf; /* The maximum size of the output is equal to the sum of the ** input sizes + 1 varint (9 bytes). The extra varint is because if the ** first rowid in one input is a large negative number, and the first in ** the other a non-negative number, the delta for the non-negative ** number will be larger on disk than the literal integer value ** was. ** ** Or, if the input position-lists are corrupt, then the output might ** include up to (nBuf+1) extra 10-byte positions created by interpreting -1 ** (the value PoslistNext64() uses for EOF) as a position and appending ** it to the output. This can happen at most once for each input ** position-list, hence (nBuf+1) 10 byte paddings. */ if( sqlite3Fts5BufferSize(&p->rc, &out, nOut) ) return; while( pHead ){ fts5MergeAppendDocid(&out, iLastRowid, pHead->iter.iRowid); if( pHead->pNext && iLastRowid==pHead->pNext->iter.iRowid ){ /* Merge data from two or more poslists */ i64 iPrev = 0; int nTmp = FTS5_DATA_ZERO_PADDING; int nMerge = 0; PrefixMerger *pSave = pHead; PrefixMerger *pThis = 0; int nTail = 0; pHead = 0; while( pSave && pSave->iter.iRowid==iLastRowid ){ PrefixMerger *pNext = pSave->pNext; pSave->iOff = 0; pSave->iPos = 0; pSave->aPos = &pSave->iter.aPoslist[pSave->iter.nSize]; fts5PrefixMergerNextPosition(pSave); nTmp += pSave->iter.nPoslist + 10; nMerge++; fts5PrefixMergerInsertByPosition(&pHead, pSave); pSave = pNext; } if( pHead==0 || pHead->pNext==0 ){ p->rc = FTS5_CORRUPT; break; } /* See the earlier comment in this function for an explanation of why ** corrupt input position lists might cause the output to consume ** at most nMerge*10 bytes of unexpected space. */ if( sqlite3Fts5BufferSize(&p->rc, &tmp, nTmp+nMerge*10) ){ break; } fts5BufferZero(&tmp); pThis = pHead; pHead = pThis->pNext; sqlite3Fts5PoslistSafeAppend(&tmp, &iPrev, pThis->iPos); fts5PrefixMergerNextPosition(pThis); fts5PrefixMergerInsertByPosition(&pHead, pThis); while( pHead->pNext ){ pThis = pHead; if( pThis->iPos!=iPrev ){ sqlite3Fts5PoslistSafeAppend(&tmp, &iPrev, pThis->iPos); } fts5PrefixMergerNextPosition(pThis); pHead = pThis->pNext; fts5PrefixMergerInsertByPosition(&pHead, pThis); } if( pHead->iPos!=iPrev ){ sqlite3Fts5PoslistSafeAppend(&tmp, &iPrev, pHead->iPos); } nTail = pHead->iter.nPoslist - pHead->iOff; /* WRITEPOSLISTSIZE */ assert_nc( tmp.n+nTail<=nTmp ); assert( tmp.n+nTail<=nTmp+nMerge*10 ); if( tmp.n+nTail>nTmp-FTS5_DATA_ZERO_PADDING ){ if( p->rc==SQLITE_OK ) p->rc = FTS5_CORRUPT; break; } fts5BufferSafeAppendVarint(&out, (tmp.n+nTail) * 2); fts5BufferSafeAppendBlob(&out, tmp.p, tmp.n); if( nTail>0 ){ fts5BufferSafeAppendBlob(&out, &pHead->aPos[pHead->iOff], nTail); } pHead = pSave; for(i=0; i<nBuf+1; i++){ PrefixMerger *pX = &aMerger[i]; if( pX->iter.aPoslist && pX->iter.iRowid==iLastRowid ){ fts5DoclistIterNext(&pX->iter); fts5PrefixMergerInsertByRowid(&pHead, pX); } } }else{ /* Copy poslist from pHead to output */ PrefixMerger *pThis = pHead; Fts5DoclistIter *pI = &pThis->iter; fts5BufferSafeAppendBlob(&out, pI->aPoslist, pI->nPoslist+pI->nSize); fts5DoclistIterNext(pI); pHead = pThis->pNext; fts5PrefixMergerInsertByRowid(&pHead, pThis); } } fts5BufferFree(p1); fts5BufferFree(&tmp); memset(&out.p[out.n], 0, FTS5_DATA_ZERO_PADDING); *p1 = out; } static void fts5SetupPrefixIter( Fts5Index *p, /* Index to read from */ int bDesc, /* True for "ORDER BY rowid DESC" */ int iIdx, /* Index to scan for data */ u8 *pToken, /* Buffer containing prefix to match */ int nToken, /* Size of buffer pToken in bytes */ Fts5Colset *pColset, /* Restrict matches to these columns */ Fts5Iter **ppIter /* OUT: New iterator */ ){ Fts5Structure *pStruct; Fts5Buffer *aBuf; int nBuf = 32; int nMerge = 1; void (*xMerge)(Fts5Index*, Fts5Buffer*, int, Fts5Buffer*); void (*xAppend)(Fts5Index*, u64, Fts5Iter*, Fts5Buffer*); if( p->pConfig->eDetail==FTS5_DETAIL_NONE ){ xMerge = fts5MergeRowidLists; xAppend = fts5AppendRowid; }else{ nMerge = FTS5_MERGE_NLIST-1; nBuf = nMerge*8; /* Sufficient to merge (16^8)==(2^32) lists */ xMerge = fts5MergePrefixLists; xAppend = fts5AppendPoslist; } aBuf = (Fts5Buffer*)fts5IdxMalloc(p, sizeof(Fts5Buffer)*nBuf); pStruct = fts5StructureRead(p); if( aBuf && pStruct ){ const int flags = FTS5INDEX_QUERY_SCAN | FTS5INDEX_QUERY_SKIPEMPTY | FTS5INDEX_QUERY_NOOUTPUT; int i; i64 iLastRowid = 0; Fts5Iter *p1 = 0; /* Iterator used to gather data from index */ Fts5Data *pData; Fts5Buffer doclist; int bNewTerm = 1; memset(&doclist, 0, sizeof(doclist)); if( iIdx!=0 ){ int dummy = 0; const int f2 = FTS5INDEX_QUERY_SKIPEMPTY|FTS5INDEX_QUERY_NOOUTPUT; pToken[0] = FTS5_MAIN_PREFIX; fts5MultiIterNew(p, pStruct, f2, pColset, pToken, nToken, -1, 0, &p1); fts5IterSetOutputCb(&p->rc, p1); for(; fts5MultiIterEof(p, p1)==0; fts5MultiIterNext2(p, p1, &dummy) ){ Fts5SegIter *pSeg = &p1->aSeg[ p1->aFirst[1].iFirst ]; p1->xSetOutputs(p1, pSeg); if( p1->base.nData ){ xAppend(p, (u64)p1->base.iRowid-(u64)iLastRowid, p1, &doclist); iLastRowid = p1->base.iRowid; } } fts5MultiIterFree(p1); } pToken[0] = FTS5_MAIN_PREFIX + iIdx; fts5MultiIterNew(p, pStruct, flags, pColset, pToken, nToken, -1, 0, &p1); fts5IterSetOutputCb(&p->rc, p1); for( /* no-op */ ; fts5MultiIterEof(p, p1)==0; fts5MultiIterNext2(p, p1, &bNewTerm) ){ Fts5SegIter *pSeg = &p1->aSeg[ p1->aFirst[1].iFirst ]; int nTerm = pSeg->term.n; const u8 *pTerm = pSeg->term.p; p1->xSetOutputs(p1, pSeg); assert_nc( memcmp(pToken, pTerm, MIN(nToken, nTerm))<=0 ); if( bNewTerm ){ if( nTerm<nToken || memcmp(pToken, pTerm, nToken) ) break; } if( p1->base.nData==0 ) continue; if( p1->base.iRowid<=iLastRowid && doclist.n>0 ){ for(i=0; p->rc==SQLITE_OK && doclist.n; i++){ int i1 = i*nMerge; int iStore; assert( i1+nMerge<=nBuf ); for(iStore=i1; iStore<i1+nMerge; iStore++){ if( aBuf[iStore].n==0 ){ fts5BufferSwap(&doclist, &aBuf[iStore]); fts5BufferZero(&doclist); break; } } if( iStore==i1+nMerge ){ xMerge(p, &doclist, nMerge, &aBuf[i1]); for(iStore=i1; iStore<i1+nMerge; iStore++){ fts5BufferZero(&aBuf[iStore]); } } } iLastRowid = 0; } xAppend(p, (u64)p1->base.iRowid-(u64)iLastRowid, p1, &doclist); iLastRowid = p1->base.iRowid; } assert( (nBuf%nMerge)==0 ); for(i=0; i<nBuf; i+=nMerge){ int iFree; if( p->rc==SQLITE_OK ){ xMerge(p, &doclist, nMerge, &aBuf[i]); } for(iFree=i; iFree<i+nMerge; iFree++){ fts5BufferFree(&aBuf[iFree]); } } fts5MultiIterFree(p1); pData = fts5IdxMalloc(p, sizeof(Fts5Data)+doclist.n+FTS5_DATA_ZERO_PADDING); if( pData ){ pData->p = (u8*)&pData[1]; pData->nn = pData->szLeaf = doclist.n; if( doclist.n ) memcpy(pData->p, doclist.p, doclist.n); fts5MultiIterNew2(p, pData, bDesc, ppIter); } fts5BufferFree(&doclist); |
︙ | ︙ | |||
5144 5145 5146 5147 5148 5149 5150 | /* ** Commit data to disk. */ int sqlite3Fts5IndexSync(Fts5Index *p){ assert( p->rc==SQLITE_OK ); fts5IndexFlush(p); | | | > | 5357 5358 5359 5360 5361 5362 5363 5364 5365 5366 5367 5368 5369 5370 5371 5372 5373 5374 5375 5376 5377 5378 5379 5380 5381 5382 5383 5384 5385 5386 5387 5388 5389 5390 5391 5392 5393 5394 5395 5396 5397 | /* ** Commit data to disk. */ int sqlite3Fts5IndexSync(Fts5Index *p){ assert( p->rc==SQLITE_OK ); fts5IndexFlush(p); sqlite3Fts5IndexCloseReader(p); return fts5IndexReturn(p); } /* ** Discard any data stored in the in-memory hash tables. Do not write it ** to the database. Additionally, assume that the contents of the %_data ** table may have changed on disk. So any in-memory caches of %_data ** records must be invalidated. */ int sqlite3Fts5IndexRollback(Fts5Index *p){ sqlite3Fts5IndexCloseReader(p); fts5IndexDiscardData(p); fts5StructureInvalidate(p); /* assert( p->rc==SQLITE_OK ); */ return SQLITE_OK; } /* ** The %_data table is completely empty when this function is called. This ** function populates it with the initial structure objects for each index, ** and the initial version of the "averages" record (a zero-byte blob). */ int sqlite3Fts5IndexReinit(Fts5Index *p){ Fts5Structure s; fts5StructureInvalidate(p); fts5IndexDiscardData(p); memset(&s, 0, sizeof(Fts5Structure)); fts5DataWrite(p, FTS5_AVERAGES_ROWID, (const u8*)"", 0); fts5StructureWrite(p, &s); return fts5IndexReturn(p); } /* |
︙ | ︙ | |||
5257 5258 5259 5260 5261 5262 5263 5264 5265 | int nChar ){ int n = 0; int i; for(i=0; i<nChar; i++){ if( n>=nByte ) return 0; /* Input contains fewer than nChar chars */ if( (unsigned char)p[n++]>=0xc0 ){ while( (p[n] & 0xc0)==0x80 ){ n++; | > | > > > | 5471 5472 5473 5474 5475 5476 5477 5478 5479 5480 5481 5482 5483 5484 5485 5486 5487 5488 5489 5490 5491 | int nChar ){ int n = 0; int i; for(i=0; i<nChar; i++){ if( n>=nByte ) return 0; /* Input contains fewer than nChar chars */ if( (unsigned char)p[n++]>=0xc0 ){ if( n>=nByte ) return 0; while( (p[n] & 0xc0)==0x80 ){ n++; if( n>=nByte ){ if( i+1==nChar ) break; return 0; } } } } return n; } /* |
︙ | ︙ | |||
5344 5345 5346 5347 5348 5349 5350 | Fts5Buffer buf = {0, 0, 0}; /* If the QUERY_SCAN flag is set, all other flags must be clear. */ assert( (flags & FTS5INDEX_QUERY_SCAN)==0 || flags==FTS5INDEX_QUERY_SCAN ); if( sqlite3Fts5BufferSize(&p->rc, &buf, nToken+1)==0 ){ int iIdx = 0; /* Index to search */ | > | | > > < | > | > > | | | | > | | 5562 5563 5564 5565 5566 5567 5568 5569 5570 5571 5572 5573 5574 5575 5576 5577 5578 5579 5580 5581 5582 5583 5584 5585 5586 5587 5588 5589 5590 5591 5592 5593 5594 5595 5596 5597 5598 5599 5600 5601 5602 5603 5604 5605 5606 5607 5608 5609 5610 5611 5612 5613 5614 5615 5616 5617 5618 5619 5620 5621 5622 5623 5624 5625 5626 5627 5628 5629 5630 5631 5632 5633 | Fts5Buffer buf = {0, 0, 0}; /* If the QUERY_SCAN flag is set, all other flags must be clear. */ assert( (flags & FTS5INDEX_QUERY_SCAN)==0 || flags==FTS5INDEX_QUERY_SCAN ); if( sqlite3Fts5BufferSize(&p->rc, &buf, nToken+1)==0 ){ int iIdx = 0; /* Index to search */ int iPrefixIdx = 0; /* +1 prefix index */ if( nToken>0 ) memcpy(&buf.p[1], pToken, nToken); /* Figure out which index to search and set iIdx accordingly. If this ** is a prefix query for which there is no prefix index, set iIdx to ** greater than pConfig->nPrefix to indicate that the query will be ** satisfied by scanning multiple terms in the main index. ** ** If the QUERY_TEST_NOIDX flag was specified, then this must be a ** prefix-query. Instead of using a prefix-index (if one exists), ** evaluate the prefix query using the main FTS index. This is used ** for internal sanity checking by the integrity-check in debug ** mode only. */ #ifdef SQLITE_DEBUG if( pConfig->bPrefixIndex==0 || (flags & FTS5INDEX_QUERY_TEST_NOIDX) ){ assert( flags & FTS5INDEX_QUERY_PREFIX ); iIdx = 1+pConfig->nPrefix; }else #endif if( flags & FTS5INDEX_QUERY_PREFIX ){ int nChar = fts5IndexCharlen(pToken, nToken); for(iIdx=1; iIdx<=pConfig->nPrefix; iIdx++){ int nIdxChar = pConfig->aPrefix[iIdx-1]; if( nIdxChar==nChar ) break; if( nIdxChar==nChar+1 ) iPrefixIdx = iIdx; } } if( iIdx<=pConfig->nPrefix ){ /* Straight index lookup */ Fts5Structure *pStruct = fts5StructureRead(p); buf.p[0] = (u8)(FTS5_MAIN_PREFIX + iIdx); if( pStruct ){ fts5MultiIterNew(p, pStruct, flags | FTS5INDEX_QUERY_SKIPEMPTY, pColset, buf.p, nToken+1, -1, 0, &pRet ); fts5StructureRelease(pStruct); } }else{ /* Scan multiple terms in the main index */ int bDesc = (flags & FTS5INDEX_QUERY_DESC)!=0; fts5SetupPrefixIter(p, bDesc, iPrefixIdx, buf.p, nToken+1, pColset,&pRet); if( pRet==0 ){ assert( p->rc!=SQLITE_OK ); }else{ assert( pRet->pColset==0 ); fts5IterSetOutputCb(&p->rc, pRet); if( p->rc==SQLITE_OK ){ Fts5SegIter *pSeg = &pRet->aSeg[pRet->aFirst[1].iFirst]; if( pSeg->pLeaf ) pRet->xSetOutputs(pRet, pSeg); } } } if( p->rc ){ sqlite3Fts5IterClose((Fts5IndexIter*)pRet); pRet = 0; sqlite3Fts5IndexCloseReader(p); } *ppIter = (Fts5IndexIter*)pRet; sqlite3Fts5BufferFree(&buf); } return fts5IndexReturn(p); } |
︙ | ︙ | |||
5456 5457 5458 5459 5460 5461 5462 5463 | /* ** Return the current term. */ const char *sqlite3Fts5IterTerm(Fts5IndexIter *pIndexIter, int *pn){ int n; const char *z = (const char*)fts5MultiIterTerm((Fts5Iter*)pIndexIter, &n); *pn = n-1; | > | | | 5680 5681 5682 5683 5684 5685 5686 5687 5688 5689 5690 5691 5692 5693 5694 5695 5696 5697 5698 5699 5700 5701 5702 5703 5704 5705 5706 5707 | /* ** Return the current term. */ const char *sqlite3Fts5IterTerm(Fts5IndexIter *pIndexIter, int *pn){ int n; const char *z = (const char*)fts5MultiIterTerm((Fts5Iter*)pIndexIter, &n); assert_nc( z || n<=1 ); *pn = n-1; return (z ? &z[1] : 0); } /* ** Close an iterator opened by an earlier call to sqlite3Fts5IndexQuery(). */ void sqlite3Fts5IterClose(Fts5IndexIter *pIndexIter){ if( pIndexIter ){ Fts5Iter *pIter = (Fts5Iter*)pIndexIter; Fts5Index *pIndex = pIter->pIndex; fts5MultiIterFree(pIter); sqlite3Fts5IndexCloseReader(pIndex); } } /* ** Read and decode the "averages" record from the database. ** ** Parameter anSize must point to an array of size nCol, where nCol is |
︙ | ︙ | |||
5635 5636 5637 5638 5639 5640 5641 | u64 *pCksum /* IN/OUT: Checksum value */ ){ int eDetail = p->pConfig->eDetail; u64 cksum = *pCksum; Fts5IndexIter *pIter = 0; int rc = sqlite3Fts5IndexQuery(p, z, n, flags, 0, &pIter); | | | 5860 5861 5862 5863 5864 5865 5866 5867 5868 5869 5870 5871 5872 5873 5874 | u64 *pCksum /* IN/OUT: Checksum value */ ){ int eDetail = p->pConfig->eDetail; u64 cksum = *pCksum; Fts5IndexIter *pIter = 0; int rc = sqlite3Fts5IndexQuery(p, z, n, flags, 0, &pIter); while( rc==SQLITE_OK && ALWAYS(pIter!=0) && 0==sqlite3Fts5IterEof(pIter) ){ i64 rowid = pIter->iRowid; if( eDetail==FTS5_DETAIL_NONE ){ cksum ^= sqlite3Fts5IndexEntryCksum(rowid, 0, 0, iIdx, z, n); }else{ Fts5PoslistReader sReader; for(sqlite3Fts5PoslistReaderInit(pIter->pData, pIter->nData, &sReader); |
︙ | ︙ | |||
5661 5662 5663 5664 5665 5666 5667 5668 5669 5670 5671 5672 5673 5674 | } sqlite3Fts5IterClose(pIter); *pCksum = cksum; return rc; } /* ** This function is also purely an internal test. It does not contribute to ** FTS functionality, or even the integrity-check, in any way. */ static void fts5TestTerm( Fts5Index *p, | > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | 5886 5887 5888 5889 5890 5891 5892 5893 5894 5895 5896 5897 5898 5899 5900 5901 5902 5903 5904 5905 5906 5907 5908 5909 5910 5911 5912 5913 5914 5915 5916 5917 5918 5919 5920 5921 5922 5923 5924 5925 5926 5927 5928 5929 5930 | } sqlite3Fts5IterClose(pIter); *pCksum = cksum; return rc; } /* ** Check if buffer z[], size n bytes, contains as series of valid utf-8 ** encoded codepoints. If so, return 0. Otherwise, if the buffer does not ** contain valid utf-8, return non-zero. */ static int fts5TestUtf8(const char *z, int n){ int i = 0; assert_nc( n>0 ); while( i<n ){ if( (z[i] & 0x80)==0x00 ){ i++; }else if( (z[i] & 0xE0)==0xC0 ){ if( i+1>=n || (z[i+1] & 0xC0)!=0x80 ) return 1; i += 2; }else if( (z[i] & 0xF0)==0xE0 ){ if( i+2>=n || (z[i+1] & 0xC0)!=0x80 || (z[i+2] & 0xC0)!=0x80 ) return 1; i += 3; }else if( (z[i] & 0xF8)==0xF0 ){ if( i+3>=n || (z[i+1] & 0xC0)!=0x80 || (z[i+2] & 0xC0)!=0x80 ) return 1; if( (z[i+2] & 0xC0)!=0x80 ) return 1; i += 3; }else{ return 1; } } return 0; } /* ** This function is also purely an internal test. It does not contribute to ** FTS functionality, or even the integrity-check, in any way. */ static void fts5TestTerm( Fts5Index *p, |
︙ | ︙ | |||
5701 5702 5703 5704 5705 5706 5707 | /* If this is a prefix query, check that the results returned if the ** the index is disabled are the same. In both ASC and DESC order. ** ** This check may only be performed if the hash table is empty. This ** is because the hash table only supports a single scan query at ** a time, and the multi-iter loop from which this function is called | | > > > > > > | | 5957 5958 5959 5960 5961 5962 5963 5964 5965 5966 5967 5968 5969 5970 5971 5972 5973 5974 5975 5976 5977 5978 | /* If this is a prefix query, check that the results returned if the ** the index is disabled are the same. In both ASC and DESC order. ** ** This check may only be performed if the hash table is empty. This ** is because the hash table only supports a single scan query at ** a time, and the multi-iter loop from which this function is called ** is already performing such a scan. ** ** Also only do this if buffer zTerm contains nTerm bytes of valid ** utf-8. Otherwise, the last part of the buffer contents might contain ** a non-utf-8 sequence that happens to be a prefix of a valid utf-8 ** character stored in the main fts index, which will cause the ** test to fail. */ if( p->nPendingData==0 && 0==fts5TestUtf8(zTerm, nTerm) ){ if( iIdx>0 && rc==SQLITE_OK ){ int f = flags|FTS5INDEX_QUERY_TEST_NOIDX; ck2 = 0; rc = fts5QueryCksum(p, iIdx, zTerm, nTerm, f, &ck2); if( rc==SQLITE_OK && ck1!=ck2 ) rc = FTS5_CORRUPT; } if( iIdx>0 && rc==SQLITE_OK ){ |
︙ | ︙ | |||
5825 5826 5827 5828 5829 5830 5831 | int rc2; int iIdxPrevLeaf = pSeg->pgnoFirst-1; int iDlidxPrevLeaf = pSeg->pgnoLast; if( pSeg->pgnoFirst==0 ) return; fts5IndexPrepareStmt(p, &pStmt, sqlite3_mprintf( | | > > < | 6087 6088 6089 6090 6091 6092 6093 6094 6095 6096 6097 6098 6099 6100 6101 6102 6103 6104 6105 6106 6107 6108 6109 6110 6111 6112 | int rc2; int iIdxPrevLeaf = pSeg->pgnoFirst-1; int iDlidxPrevLeaf = pSeg->pgnoLast; if( pSeg->pgnoFirst==0 ) return; fts5IndexPrepareStmt(p, &pStmt, sqlite3_mprintf( "SELECT segid, term, (pgno>>1), (pgno&1) FROM %Q.'%q_idx' WHERE segid=%d " "ORDER BY 1, 2", pConfig->zDb, pConfig->zName, pSeg->iSegid )); /* Iterate through the b-tree hierarchy. */ while( p->rc==SQLITE_OK && SQLITE_ROW==sqlite3_step(pStmt) ){ i64 iRow; /* Rowid for this leaf */ Fts5Data *pLeaf; /* Data for this leaf */ const char *zIdxTerm = (const char*)sqlite3_column_blob(pStmt, 1); int nIdxTerm = sqlite3_column_bytes(pStmt, 1); int iIdxLeaf = sqlite3_column_int(pStmt, 2); int bIdxDlidx = sqlite3_column_int(pStmt, 3); /* If the leaf in question has already been trimmed from the segment, ** ignore this b-tree entry. Otherwise, load it into memory. */ if( iIdxLeaf<pSeg->pgnoFirst ) continue; iRow = FTS5_SEGMENT_ROWID(pSeg->iSegid, iIdxLeaf); |
︙ | ︙ | |||
5860 5861 5862 5863 5864 5865 5866 | int iOff; /* Offset of first term on leaf */ int iRowidOff; /* Offset of first rowid on leaf */ int nTerm; /* Size of term on leaf in bytes */ int res; /* Comparison of term and split-key */ iOff = fts5LeafFirstTermOff(pLeaf); iRowidOff = fts5LeafFirstRowidOff(pLeaf); | | | | 6123 6124 6125 6126 6127 6128 6129 6130 6131 6132 6133 6134 6135 6136 6137 6138 6139 6140 6141 | int iOff; /* Offset of first term on leaf */ int iRowidOff; /* Offset of first rowid on leaf */ int nTerm; /* Size of term on leaf in bytes */ int res; /* Comparison of term and split-key */ iOff = fts5LeafFirstTermOff(pLeaf); iRowidOff = fts5LeafFirstRowidOff(pLeaf); if( iRowidOff>=iOff || iOff>=pLeaf->szLeaf ){ p->rc = FTS5_CORRUPT; }else{ iOff += fts5GetVarint32(&pLeaf->p[iOff], nTerm); res = fts5Memcmp(&pLeaf->p[iOff], zIdxTerm, MIN(nTerm, nIdxTerm)); if( res==0 ) res = nTerm - nIdxTerm; if( res<0 ) p->rc = FTS5_CORRUPT; } fts5IntegrityCheckPgidx(p, pLeaf); } fts5DataRelease(pLeaf); |
︙ | ︙ | |||
5956 5957 5958 5959 5960 5961 5962 | ** as calculated by sqlite3Fts5IndexEntryCksum() is cksum. ** ** Return SQLITE_CORRUPT if any of the internal checks fail, or if the ** checksum does not match. Return SQLITE_OK if all checks pass without ** error, or some other SQLite error code if another error (e.g. OOM) ** occurs. */ | | > > > > | > < < | | | | < | 6219 6220 6221 6222 6223 6224 6225 6226 6227 6228 6229 6230 6231 6232 6233 6234 6235 6236 6237 6238 6239 6240 6241 6242 6243 6244 6245 6246 6247 6248 6249 6250 6251 6252 6253 6254 6255 6256 6257 6258 6259 | ** as calculated by sqlite3Fts5IndexEntryCksum() is cksum. ** ** Return SQLITE_CORRUPT if any of the internal checks fail, or if the ** checksum does not match. Return SQLITE_OK if all checks pass without ** error, or some other SQLite error code if another error (e.g. OOM) ** occurs. */ int sqlite3Fts5IndexIntegrityCheck(Fts5Index *p, u64 cksum, int bUseCksum){ int eDetail = p->pConfig->eDetail; u64 cksum2 = 0; /* Checksum based on contents of indexes */ Fts5Buffer poslist = {0,0,0}; /* Buffer used to hold a poslist */ Fts5Iter *pIter; /* Used to iterate through entire index */ Fts5Structure *pStruct; /* Index structure */ int iLvl, iSeg; #ifdef SQLITE_DEBUG /* Used by extra internal tests only run if NDEBUG is not defined */ u64 cksum3 = 0; /* Checksum based on contents of indexes */ Fts5Buffer term = {0,0,0}; /* Buffer used to hold most recent term */ #endif const int flags = FTS5INDEX_QUERY_NOOUTPUT; /* Load the FTS index structure */ pStruct = fts5StructureRead(p); if( pStruct==0 ){ assert( p->rc!=SQLITE_OK ); return fts5IndexReturn(p); } /* Check that the internal nodes of each segment match the leaves */ for(iLvl=0; iLvl<pStruct->nLevel; iLvl++){ for(iSeg=0; iSeg<pStruct->aLevel[iLvl].nSeg; iSeg++){ Fts5StructureSegment *pSeg = &pStruct->aLevel[iLvl].aSeg[iSeg]; fts5IndexIntegrityCheckSegment(p, pSeg); } } /* The cksum argument passed to this function is a checksum calculated ** based on all expected entries in the FTS index (including prefix index ** entries). This block checks that a checksum calculated based on the ** actual contents of FTS index is identical. |
︙ | ︙ | |||
6009 6010 6011 6012 6013 6014 6015 6016 6017 6018 6019 6020 6021 6022 6023 6024 6025 6026 6027 6028 6029 6030 6031 6032 6033 | i64 iPos = 0; /* Position read from poslist */ int iOff = 0; /* Offset within poslist */ i64 iRowid = fts5MultiIterRowid(pIter); char *z = (char*)fts5MultiIterTerm(pIter, &n); /* If this is a new term, query for it. Update cksum3 with the results. */ fts5TestTerm(p, &term, z, n, cksum2, &cksum3); if( eDetail==FTS5_DETAIL_NONE ){ if( 0==fts5MultiIterIsEmpty(p, pIter) ){ cksum2 ^= sqlite3Fts5IndexEntryCksum(iRowid, 0, 0, -1, z, n); } }else{ poslist.n = 0; fts5SegiterPoslist(p, &pIter->aSeg[pIter->aFirst[1].iFirst], 0, &poslist); while( 0==sqlite3Fts5PoslistNext64(poslist.p, poslist.n, &iOff, &iPos) ){ int iCol = FTS5_POS2COLUMN(iPos); int iTokOff = FTS5_POS2OFFSET(iPos); cksum2 ^= sqlite3Fts5IndexEntryCksum(iRowid, iCol, iTokOff, -1, z, n); } } } fts5TestTerm(p, &term, 0, 0, cksum2, &cksum3); fts5MultiIterFree(pIter); | > > | > | 6274 6275 6276 6277 6278 6279 6280 6281 6282 6283 6284 6285 6286 6287 6288 6289 6290 6291 6292 6293 6294 6295 6296 6297 6298 6299 6300 6301 6302 6303 6304 6305 6306 6307 6308 6309 6310 6311 6312 6313 6314 6315 6316 6317 6318 6319 6320 6321 6322 6323 6324 | i64 iPos = 0; /* Position read from poslist */ int iOff = 0; /* Offset within poslist */ i64 iRowid = fts5MultiIterRowid(pIter); char *z = (char*)fts5MultiIterTerm(pIter, &n); /* If this is a new term, query for it. Update cksum3 with the results. */ fts5TestTerm(p, &term, z, n, cksum2, &cksum3); if( p->rc ) break; if( eDetail==FTS5_DETAIL_NONE ){ if( 0==fts5MultiIterIsEmpty(p, pIter) ){ cksum2 ^= sqlite3Fts5IndexEntryCksum(iRowid, 0, 0, -1, z, n); } }else{ poslist.n = 0; fts5SegiterPoslist(p, &pIter->aSeg[pIter->aFirst[1].iFirst], 0, &poslist); fts5BufferAppendBlob(&p->rc, &poslist, 4, (const u8*)"\0\0\0\0"); while( 0==sqlite3Fts5PoslistNext64(poslist.p, poslist.n, &iOff, &iPos) ){ int iCol = FTS5_POS2COLUMN(iPos); int iTokOff = FTS5_POS2OFFSET(iPos); cksum2 ^= sqlite3Fts5IndexEntryCksum(iRowid, iCol, iTokOff, -1, z, n); } } } fts5TestTerm(p, &term, 0, 0, cksum2, &cksum3); fts5MultiIterFree(pIter); if( p->rc==SQLITE_OK && bUseCksum && cksum!=cksum2 ) p->rc = FTS5_CORRUPT; fts5StructureRelease(pStruct); #ifdef SQLITE_DEBUG fts5BufferFree(&term); #endif fts5BufferFree(&poslist); return fts5IndexReturn(p); } /************************************************************************* ************************************************************************** ** Below this point is the implementation of the fts5_decode() scalar ** function only. */ #ifdef SQLITE_TEST /* ** Decode a segment-data rowid from the %_data table. This function is ** the opposite of macro FTS5_SEGMENT_ROWID(). */ static void fts5DecodeRowid( i64 iRowid, /* Rowid from %_data table */ int *piSegid, /* OUT: Segment id */ |
︙ | ︙ | |||
6065 6066 6067 6068 6069 6070 6071 6072 6073 6074 6075 6076 6077 6078 6079 6080 6081 6082 6083 6084 6085 6086 6087 6088 6089 6090 6091 6092 6093 6094 6095 6096 6097 | iRowid >>= FTS5_DATA_HEIGHT_B; *pbDlidx = (int)(iRowid & 0x0001); iRowid >>= FTS5_DATA_DLI_B; *piSegid = (int)(iRowid & (((i64)1 << FTS5_DATA_ID_B) - 1)); } static void fts5DebugRowid(int *pRc, Fts5Buffer *pBuf, i64 iKey){ int iSegid, iHeight, iPgno, bDlidx; /* Rowid compenents */ fts5DecodeRowid(iKey, &iSegid, &bDlidx, &iHeight, &iPgno); if( iSegid==0 ){ if( iKey==FTS5_AVERAGES_ROWID ){ sqlite3Fts5BufferAppendPrintf(pRc, pBuf, "{averages} "); }else{ sqlite3Fts5BufferAppendPrintf(pRc, pBuf, "{structure}"); } } else{ sqlite3Fts5BufferAppendPrintf(pRc, pBuf, "{%ssegid=%d h=%d pgno=%d}", bDlidx ? "dlidx " : "", iSegid, iHeight, iPgno ); } } static void fts5DebugStructure( int *pRc, /* IN/OUT: error code */ Fts5Buffer *pBuf, Fts5Structure *p ){ int iLvl, iSeg; /* Iterate through levels, segments */ | > > > > | 6333 6334 6335 6336 6337 6338 6339 6340 6341 6342 6343 6344 6345 6346 6347 6348 6349 6350 6351 6352 6353 6354 6355 6356 6357 6358 6359 6360 6361 6362 6363 6364 6365 6366 6367 6368 6369 | iRowid >>= FTS5_DATA_HEIGHT_B; *pbDlidx = (int)(iRowid & 0x0001); iRowid >>= FTS5_DATA_DLI_B; *piSegid = (int)(iRowid & (((i64)1 << FTS5_DATA_ID_B) - 1)); } #endif /* SQLITE_TEST */ #ifdef SQLITE_TEST static void fts5DebugRowid(int *pRc, Fts5Buffer *pBuf, i64 iKey){ int iSegid, iHeight, iPgno, bDlidx; /* Rowid compenents */ fts5DecodeRowid(iKey, &iSegid, &bDlidx, &iHeight, &iPgno); if( iSegid==0 ){ if( iKey==FTS5_AVERAGES_ROWID ){ sqlite3Fts5BufferAppendPrintf(pRc, pBuf, "{averages} "); }else{ sqlite3Fts5BufferAppendPrintf(pRc, pBuf, "{structure}"); } } else{ sqlite3Fts5BufferAppendPrintf(pRc, pBuf, "{%ssegid=%d h=%d pgno=%d}", bDlidx ? "dlidx " : "", iSegid, iHeight, iPgno ); } } #endif /* SQLITE_TEST */ #ifdef SQLITE_TEST static void fts5DebugStructure( int *pRc, /* IN/OUT: error code */ Fts5Buffer *pBuf, Fts5Structure *p ){ int iLvl, iSeg; /* Iterate through levels, segments */ |
︙ | ︙ | |||
6105 6106 6107 6108 6109 6110 6111 6112 6113 6114 6115 6116 6117 6118 6119 | sqlite3Fts5BufferAppendPrintf(pRc, pBuf, " {id=%d leaves=%d..%d}", pSeg->iSegid, pSeg->pgnoFirst, pSeg->pgnoLast ); } sqlite3Fts5BufferAppendPrintf(pRc, pBuf, "}"); } } /* ** This is part of the fts5_decode() debugging aid. ** ** Arguments pBlob/nBlob contain a serialized Fts5Structure object. This ** function appends a human-readable representation of the same object ** to the buffer passed as the second argument. */ | > > | 6377 6378 6379 6380 6381 6382 6383 6384 6385 6386 6387 6388 6389 6390 6391 6392 6393 | sqlite3Fts5BufferAppendPrintf(pRc, pBuf, " {id=%d leaves=%d..%d}", pSeg->iSegid, pSeg->pgnoFirst, pSeg->pgnoLast ); } sqlite3Fts5BufferAppendPrintf(pRc, pBuf, "}"); } } #endif /* SQLITE_TEST */ #ifdef SQLITE_TEST /* ** This is part of the fts5_decode() debugging aid. ** ** Arguments pBlob/nBlob contain a serialized Fts5Structure object. This ** function appends a human-readable representation of the same object ** to the buffer passed as the second argument. */ |
︙ | ︙ | |||
6130 6131 6132 6133 6134 6135 6136 6137 6138 6139 6140 6141 6142 6143 6144 | *pRc = rc; return; } fts5DebugStructure(pRc, pBuf, p); fts5StructureRelease(p); } /* ** This is part of the fts5_decode() debugging aid. ** ** Arguments pBlob/nBlob contain an "averages" record. This function ** appends a human-readable representation of record to the buffer passed ** as the second argument. */ | > > | 6404 6405 6406 6407 6408 6409 6410 6411 6412 6413 6414 6415 6416 6417 6418 6419 6420 | *pRc = rc; return; } fts5DebugStructure(pRc, pBuf, p); fts5StructureRelease(p); } #endif /* SQLITE_TEST */ #ifdef SQLITE_TEST /* ** This is part of the fts5_decode() debugging aid. ** ** Arguments pBlob/nBlob contain an "averages" record. This function ** appends a human-readable representation of record to the buffer passed ** as the second argument. */ |
︙ | ︙ | |||
6153 6154 6155 6156 6157 6158 6159 6160 6161 6162 6163 6164 6165 6166 6167 6168 6169 6170 6171 6172 6173 6174 6175 6176 6177 6178 6179 6180 6181 6182 6183 6184 | while( i<nBlob ){ u64 iVal; i += sqlite3Fts5GetVarint(&pBlob[i], &iVal); sqlite3Fts5BufferAppendPrintf(pRc, pBuf, "%s%d", zSpace, (int)iVal); zSpace = " "; } } /* ** Buffer (a/n) is assumed to contain a list of serialized varints. Read ** each varint and append its string representation to buffer pBuf. Return ** after either the input buffer is exhausted or a 0 value is read. ** ** The return value is the number of bytes read from the input buffer. */ static int fts5DecodePoslist(int *pRc, Fts5Buffer *pBuf, const u8 *a, int n){ int iOff = 0; while( iOff<n ){ int iVal; iOff += fts5GetVarint32(&a[iOff], iVal); sqlite3Fts5BufferAppendPrintf(pRc, pBuf, " %d", iVal); } return iOff; } /* ** The start of buffer (a/n) contains the start of a doclist. The doclist ** may or may not finish within the buffer. This function appends a text ** representation of the part of the doclist that is present to buffer ** pBuf. ** ** The return value is the number of bytes read from the input buffer. | > > > > | 6429 6430 6431 6432 6433 6434 6435 6436 6437 6438 6439 6440 6441 6442 6443 6444 6445 6446 6447 6448 6449 6450 6451 6452 6453 6454 6455 6456 6457 6458 6459 6460 6461 6462 6463 6464 | while( i<nBlob ){ u64 iVal; i += sqlite3Fts5GetVarint(&pBlob[i], &iVal); sqlite3Fts5BufferAppendPrintf(pRc, pBuf, "%s%d", zSpace, (int)iVal); zSpace = " "; } } #endif /* SQLITE_TEST */ #ifdef SQLITE_TEST /* ** Buffer (a/n) is assumed to contain a list of serialized varints. Read ** each varint and append its string representation to buffer pBuf. Return ** after either the input buffer is exhausted or a 0 value is read. ** ** The return value is the number of bytes read from the input buffer. */ static int fts5DecodePoslist(int *pRc, Fts5Buffer *pBuf, const u8 *a, int n){ int iOff = 0; while( iOff<n ){ int iVal; iOff += fts5GetVarint32(&a[iOff], iVal); sqlite3Fts5BufferAppendPrintf(pRc, pBuf, " %d", iVal); } return iOff; } #endif /* SQLITE_TEST */ #ifdef SQLITE_TEST /* ** The start of buffer (a/n) contains the start of a doclist. The doclist ** may or may not finish within the buffer. This function appends a text ** representation of the part of the doclist that is present to buffer ** pBuf. ** ** The return value is the number of bytes read from the input buffer. |
︙ | ︙ | |||
6203 6204 6205 6206 6207 6208 6209 6210 6211 6212 6213 6214 6215 6216 6217 | iDocid += iDelta; sqlite3Fts5BufferAppendPrintf(pRc, pBuf, " id=%lld", iDocid); } } return iOff; } /* ** This function is part of the fts5_decode() debugging function. It is ** only ever used with detail=none tables. ** ** Buffer (pData/nData) contains a doclist in the format used by detail=none ** tables. This function appends a human-readable version of that list to ** buffer pBuf. | > > | 6483 6484 6485 6486 6487 6488 6489 6490 6491 6492 6493 6494 6495 6496 6497 6498 6499 | iDocid += iDelta; sqlite3Fts5BufferAppendPrintf(pRc, pBuf, " id=%lld", iDocid); } } return iOff; } #endif /* SQLITE_TEST */ #ifdef SQLITE_TEST /* ** This function is part of the fts5_decode() debugging function. It is ** only ever used with detail=none tables. ** ** Buffer (pData/nData) contains a doclist in the format used by detail=none ** tables. This function appends a human-readable version of that list to ** buffer pBuf. |
︙ | ︙ | |||
6244 6245 6246 6247 6248 6249 6250 6251 6252 6253 6254 6255 6256 6257 6258 6259 6260 6261 6262 6263 6264 6265 | zApp = "*"; } } sqlite3Fts5BufferAppendPrintf(pRc, pBuf, " %lld%s", iRowid, zApp); } } /* ** The implementation of user-defined scalar function fts5_decode(). */ static void fts5DecodeFunction( sqlite3_context *pCtx, /* Function call context */ int nArg, /* Number of args (always 2) */ sqlite3_value **apVal /* Function arguments */ ){ i64 iRowid; /* Rowid for record being decoded */ int iSegid,iHeight,iPgno,bDlidx;/* Rowid components */ const u8 *aBlob; int n; /* Record to decode */ u8 *a = 0; Fts5Buffer s; /* Build up text to return here */ int rc = SQLITE_OK; /* Return code */ | > > | | < | 6526 6527 6528 6529 6530 6531 6532 6533 6534 6535 6536 6537 6538 6539 6540 6541 6542 6543 6544 6545 6546 6547 6548 6549 6550 6551 6552 6553 6554 6555 6556 6557 6558 6559 6560 6561 6562 6563 6564 6565 6566 6567 6568 6569 6570 6571 6572 6573 | zApp = "*"; } } sqlite3Fts5BufferAppendPrintf(pRc, pBuf, " %lld%s", iRowid, zApp); } } #endif /* SQLITE_TEST */ #ifdef SQLITE_TEST /* ** The implementation of user-defined scalar function fts5_decode(). */ static void fts5DecodeFunction( sqlite3_context *pCtx, /* Function call context */ int nArg, /* Number of args (always 2) */ sqlite3_value **apVal /* Function arguments */ ){ i64 iRowid; /* Rowid for record being decoded */ int iSegid,iHeight,iPgno,bDlidx;/* Rowid components */ const u8 *aBlob; int n; /* Record to decode */ u8 *a = 0; Fts5Buffer s; /* Build up text to return here */ int rc = SQLITE_OK; /* Return code */ sqlite3_int64 nSpace = 0; int eDetailNone = (sqlite3_user_data(pCtx)!=0); assert( nArg==2 ); UNUSED_PARAM(nArg); memset(&s, 0, sizeof(Fts5Buffer)); iRowid = sqlite3_value_int64(apVal[0]); /* Make a copy of the second argument (a blob) in aBlob[]. The aBlob[] ** copy is followed by FTS5_DATA_ZERO_PADDING 0x00 bytes, which prevents ** buffer overreads even if the record is corrupt. */ n = sqlite3_value_bytes(apVal[1]); aBlob = sqlite3_value_blob(apVal[1]); nSpace = n + FTS5_DATA_ZERO_PADDING; a = (u8*)sqlite3Fts5MallocZero(&rc, nSpace); if( a==0 ) goto decode_out; if( n>0 ) memcpy(a, aBlob, n); fts5DecodeRowid(iRowid, &iSegid, &bDlidx, &iHeight, &iPgno); fts5DebugRowid(&rc, &s, iRowid); if( bDlidx ){ Fts5Data dlidx; Fts5DlidxLvl lvl; |
︙ | ︙ | |||
6371 6372 6373 6374 6375 6376 6377 6378 6379 6380 6381 6382 6383 6384 6385 6386 6387 6388 6389 6390 6391 6392 6393 6394 6395 | sqlite3Fts5BufferSet(&rc, &s, 7, (const u8*)"corrupt"); goto decode_out; }else{ iRowidOff = fts5GetU16(&a[0]); iPgidxOff = szLeaf = fts5GetU16(&a[2]); if( iPgidxOff<n ){ fts5GetVarint32(&a[iPgidxOff], iTermOff); } } /* Decode the position list tail at the start of the page */ if( iRowidOff!=0 ){ iOff = iRowidOff; }else if( iTermOff!=0 ){ iOff = iTermOff; }else{ iOff = szLeaf; } fts5DecodePoslist(&rc, &s, &a[4], iOff-4); /* Decode any more doclist data that appears on the page before the ** first term. */ nDoclist = (iTermOff ? iTermOff : szLeaf) - iOff; fts5DecodeDoclist(&rc, &s, &a[iOff], nDoclist); | > > > > > > > > > > > | > > > > > > > > > > > > | 6654 6655 6656 6657 6658 6659 6660 6661 6662 6663 6664 6665 6666 6667 6668 6669 6670 6671 6672 6673 6674 6675 6676 6677 6678 6679 6680 6681 6682 6683 6684 6685 6686 6687 6688 6689 6690 6691 6692 6693 6694 6695 6696 6697 6698 6699 6700 6701 6702 6703 6704 6705 6706 6707 6708 6709 6710 6711 6712 6713 6714 6715 6716 6717 6718 6719 6720 6721 6722 6723 6724 6725 6726 6727 6728 6729 | sqlite3Fts5BufferSet(&rc, &s, 7, (const u8*)"corrupt"); goto decode_out; }else{ iRowidOff = fts5GetU16(&a[0]); iPgidxOff = szLeaf = fts5GetU16(&a[2]); if( iPgidxOff<n ){ fts5GetVarint32(&a[iPgidxOff], iTermOff); }else if( iPgidxOff>n ){ rc = FTS5_CORRUPT; goto decode_out; } } /* Decode the position list tail at the start of the page */ if( iRowidOff!=0 ){ iOff = iRowidOff; }else if( iTermOff!=0 ){ iOff = iTermOff; }else{ iOff = szLeaf; } if( iOff>n ){ rc = FTS5_CORRUPT; goto decode_out; } fts5DecodePoslist(&rc, &s, &a[4], iOff-4); /* Decode any more doclist data that appears on the page before the ** first term. */ nDoclist = (iTermOff ? iTermOff : szLeaf) - iOff; if( nDoclist+iOff>n ){ rc = FTS5_CORRUPT; goto decode_out; } fts5DecodeDoclist(&rc, &s, &a[iOff], nDoclist); while( iPgidxOff<n && rc==SQLITE_OK ){ int bFirst = (iPgidxOff==szLeaf); /* True for first term on page */ int nByte; /* Bytes of data */ int iEnd; iPgidxOff += fts5GetVarint32(&a[iPgidxOff], nByte); iPgidxPrev += nByte; iOff = iPgidxPrev; if( iPgidxOff<n ){ fts5GetVarint32(&a[iPgidxOff], nByte); iEnd = iPgidxPrev + nByte; }else{ iEnd = szLeaf; } if( iEnd>szLeaf ){ rc = FTS5_CORRUPT; break; } if( bFirst==0 ){ iOff += fts5GetVarint32(&a[iOff], nByte); if( nByte>term.n ){ rc = FTS5_CORRUPT; break; } term.n = nByte; } iOff += fts5GetVarint32(&a[iOff], nByte); if( iOff+nByte>n ){ rc = FTS5_CORRUPT; break; } fts5BufferAppendBlob(&rc, &term, nByte, &a[iOff]); iOff += nByte; sqlite3Fts5BufferAppendPrintf( &rc, &s, " term=%.*s", term.n, (const char*)term.p ); iOff += fts5DecodeDoclist(&rc, &s, &a[iOff], iEnd-iOff); |
︙ | ︙ | |||
6431 6432 6433 6434 6435 6436 6437 6438 6439 6440 6441 6442 6443 6444 6445 | if( rc==SQLITE_OK ){ sqlite3_result_text(pCtx, (const char*)s.p, s.n, SQLITE_TRANSIENT); }else{ sqlite3_result_error_code(pCtx, rc); } fts5BufferFree(&s); } /* ** The implementation of user-defined scalar function fts5_rowid(). */ static void fts5RowidFunction( sqlite3_context *pCtx, /* Function call context */ int nArg, /* Number of args (always 2) */ sqlite3_value **apVal /* Function arguments */ | > > | 6737 6738 6739 6740 6741 6742 6743 6744 6745 6746 6747 6748 6749 6750 6751 6752 6753 | if( rc==SQLITE_OK ){ sqlite3_result_text(pCtx, (const char*)s.p, s.n, SQLITE_TRANSIENT); }else{ sqlite3_result_error_code(pCtx, rc); } fts5BufferFree(&s); } #endif /* SQLITE_TEST */ #ifdef SQLITE_TEST /* ** The implementation of user-defined scalar function fts5_rowid(). */ static void fts5RowidFunction( sqlite3_context *pCtx, /* Function call context */ int nArg, /* Number of args (always 2) */ sqlite3_value **apVal /* Function arguments */ |
︙ | ︙ | |||
6465 6466 6467 6468 6469 6470 6471 6472 6473 6474 6475 6476 6477 6478 6479 6480 6481 6482 6483 6484 6485 6486 6487 6488 6489 6490 6491 6492 6493 6494 6495 6496 6497 6498 6499 6500 6501 6502 6503 6504 6505 6506 6507 6508 | }else{ sqlite3_result_error(pCtx, "first arg to fts5_rowid() must be 'segment'" , -1 ); } } } /* ** This is called as part of registering the FTS5 module with database ** connection db. It registers several user-defined scalar functions useful ** with FTS5. ** ** If successful, SQLITE_OK is returned. If an error occurs, some other ** SQLite error code is returned instead. */ int sqlite3Fts5IndexInit(sqlite3 *db){ int rc = sqlite3_create_function( db, "fts5_decode", 2, SQLITE_UTF8, 0, fts5DecodeFunction, 0, 0 ); if( rc==SQLITE_OK ){ rc = sqlite3_create_function( db, "fts5_decode_none", 2, SQLITE_UTF8, (void*)db, fts5DecodeFunction, 0, 0 ); } if( rc==SQLITE_OK ){ rc = sqlite3_create_function( db, "fts5_rowid", -1, SQLITE_UTF8, 0, fts5RowidFunction, 0, 0 ); } return rc; } int sqlite3Fts5IndexReset(Fts5Index *p){ assert( p->pStruct==0 || p->iStructVersion!=0 ); if( fts5IndexDataVersion(p)!=p->iStructVersion ){ fts5StructureInvalidate(p); } return fts5IndexReturn(p); } | > > > > > > | 6773 6774 6775 6776 6777 6778 6779 6780 6781 6782 6783 6784 6785 6786 6787 6788 6789 6790 6791 6792 6793 6794 6795 6796 6797 6798 6799 6800 6801 6802 6803 6804 6805 6806 6807 6808 6809 6810 6811 6812 6813 6814 6815 6816 6817 6818 6819 6820 6821 6822 | }else{ sqlite3_result_error(pCtx, "first arg to fts5_rowid() must be 'segment'" , -1 ); } } } #endif /* SQLITE_TEST */ /* ** This is called as part of registering the FTS5 module with database ** connection db. It registers several user-defined scalar functions useful ** with FTS5. ** ** If successful, SQLITE_OK is returned. If an error occurs, some other ** SQLite error code is returned instead. */ int sqlite3Fts5IndexInit(sqlite3 *db){ #ifdef SQLITE_TEST int rc = sqlite3_create_function( db, "fts5_decode", 2, SQLITE_UTF8, 0, fts5DecodeFunction, 0, 0 ); if( rc==SQLITE_OK ){ rc = sqlite3_create_function( db, "fts5_decode_none", 2, SQLITE_UTF8, (void*)db, fts5DecodeFunction, 0, 0 ); } if( rc==SQLITE_OK ){ rc = sqlite3_create_function( db, "fts5_rowid", -1, SQLITE_UTF8, 0, fts5RowidFunction, 0, 0 ); } return rc; #else return SQLITE_OK; UNUSED_PARAM(db); #endif } int sqlite3Fts5IndexReset(Fts5Index *p){ assert( p->pStruct==0 || p->iStructVersion!=0 ); if( fts5IndexDataVersion(p)!=p->iStructVersion ){ fts5StructureInvalidate(p); } return fts5IndexReturn(p); } |
Changes to ext/fts5/fts5_main.c.
︙ | ︙ | |||
18 19 20 21 22 23 24 25 26 27 28 29 30 | /* ** This variable is set to false when running tests for which the on disk ** structures should not be corrupt. Otherwise, true. If it is false, extra ** assert() conditions in the fts5 code are activated - conditions that are ** only true if it is guaranteed that the fts5 database is not corrupt. */ int sqlite3_fts5_may_be_corrupt = 1; typedef struct Fts5Auxdata Fts5Auxdata; typedef struct Fts5Auxiliary Fts5Auxiliary; typedef struct Fts5Cursor Fts5Cursor; | > > | | | 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 | /* ** This variable is set to false when running tests for which the on disk ** structures should not be corrupt. Otherwise, true. If it is false, extra ** assert() conditions in the fts5 code are activated - conditions that are ** only true if it is guaranteed that the fts5 database is not corrupt. */ #ifdef SQLITE_DEBUG int sqlite3_fts5_may_be_corrupt = 1; #endif typedef struct Fts5Auxdata Fts5Auxdata; typedef struct Fts5Auxiliary Fts5Auxiliary; typedef struct Fts5Cursor Fts5Cursor; typedef struct Fts5FullTable Fts5FullTable; typedef struct Fts5Sorter Fts5Sorter; typedef struct Fts5TokenizerModule Fts5TokenizerModule; /* ** NOTES ON TRANSACTIONS: ** ** SQLite invokes the following virtual table methods as transactions are ** opened and closed by the user: |
︙ | ︙ | |||
106 107 108 109 110 111 112 | char *zName; /* Name of tokenizer */ void *pUserData; /* User pointer passed to xCreate() */ fts5_tokenizer x; /* Tokenizer functions */ void (*xDestroy)(void*); /* Destructor function */ Fts5TokenizerModule *pNext; /* Next registered tokenizer module */ }; | < < < | < < | | 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 | char *zName; /* Name of tokenizer */ void *pUserData; /* User pointer passed to xCreate() */ fts5_tokenizer x; /* Tokenizer functions */ void (*xDestroy)(void*); /* Destructor function */ Fts5TokenizerModule *pNext; /* Next registered tokenizer module */ }; struct Fts5FullTable { Fts5Table p; /* Public class members from fts5Int.h */ Fts5Storage *pStorage; /* Document store */ Fts5Global *pGlobal; /* Global (connection wide) data */ Fts5Cursor *pSortCsr; /* Sort data from this cursor */ #ifdef SQLITE_DEBUG struct Fts5TransactionState ts; #endif }; |
︙ | ︙ | |||
250 251 252 253 254 255 256 | #define FTS5_BEGIN 1 #define FTS5_SYNC 2 #define FTS5_COMMIT 3 #define FTS5_ROLLBACK 4 #define FTS5_SAVEPOINT 5 #define FTS5_RELEASE 6 #define FTS5_ROLLBACKTO 7 | | | | | | | > > > | | | | | | | | | 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340 341 342 343 344 345 346 347 | #define FTS5_BEGIN 1 #define FTS5_SYNC 2 #define FTS5_COMMIT 3 #define FTS5_ROLLBACK 4 #define FTS5_SAVEPOINT 5 #define FTS5_RELEASE 6 #define FTS5_ROLLBACKTO 7 static void fts5CheckTransactionState(Fts5FullTable *p, int op, int iSavepoint){ switch( op ){ case FTS5_BEGIN: assert( p->ts.eState==0 ); p->ts.eState = 1; p->ts.iSavepoint = -1; break; case FTS5_SYNC: assert( p->ts.eState==1 || p->ts.eState==2 ); p->ts.eState = 2; break; case FTS5_COMMIT: assert( p->ts.eState==2 ); p->ts.eState = 0; break; case FTS5_ROLLBACK: assert( p->ts.eState==1 || p->ts.eState==2 || p->ts.eState==0 ); p->ts.eState = 0; break; case FTS5_SAVEPOINT: assert( p->ts.eState>=1 ); assert( iSavepoint>=0 ); assert( iSavepoint>=p->ts.iSavepoint ); p->ts.iSavepoint = iSavepoint; break; case FTS5_RELEASE: assert( p->ts.eState>=1 ); assert( iSavepoint>=0 ); assert( iSavepoint<=p->ts.iSavepoint ); p->ts.iSavepoint = iSavepoint-1; break; case FTS5_ROLLBACKTO: assert( p->ts.eState>=1 ); assert( iSavepoint>=-1 ); /* The following assert() can fail if another vtab strikes an error ** within an xSavepoint() call then SQLite calls xRollbackTo() - without ** having called xSavepoint() on this vtab. */ /* assert( iSavepoint<=p->ts.iSavepoint ); */ p->ts.iSavepoint = iSavepoint; break; } } #else # define fts5CheckTransactionState(x,y,z) #endif /* ** Return true if pTab is a contentless table. */ static int fts5IsContentless(Fts5FullTable *pTab){ return pTab->p.pConfig->eContent==FTS5_CONTENT_NONE; } /* ** Delete a virtual table handle allocated by fts5InitVtab(). */ static void fts5FreeVtab(Fts5FullTable *pTab){ if( pTab ){ sqlite3Fts5IndexClose(pTab->p.pIndex); sqlite3Fts5StorageClose(pTab->pStorage); sqlite3Fts5ConfigFree(pTab->p.pConfig); sqlite3_free(pTab); } } /* ** The xDisconnect() virtual table method. */ static int fts5DisconnectMethod(sqlite3_vtab *pVtab){ fts5FreeVtab((Fts5FullTable*)pVtab); return SQLITE_OK; } /* ** The xDestroy() virtual table method. */ static int fts5DestroyMethod(sqlite3_vtab *pVtab){ Fts5Table *pTab = (Fts5Table*)pVtab; int rc = sqlite3Fts5DropAll(pTab->pConfig); if( rc==SQLITE_OK ){ fts5FreeVtab((Fts5FullTable*)pVtab); } return rc; } /* ** This function is the implementation of both the xConnect and xCreate ** methods of the FTS3 virtual table. |
︙ | ︙ | |||
362 363 364 365 366 367 368 | sqlite3_vtab **ppVTab, /* Write the resulting vtab structure here */ char **pzErr /* Write any error message here */ ){ Fts5Global *pGlobal = (Fts5Global*)pAux; const char **azConfig = (const char**)argv; int rc = SQLITE_OK; /* Return code */ Fts5Config *pConfig = 0; /* Results of parsing argc/argv */ | | | | | | | | | 362 363 364 365 366 367 368 369 370 371 372 373 374 375 376 377 378 379 380 381 382 383 384 385 386 387 388 389 390 391 392 393 394 395 396 397 398 399 400 401 402 403 404 405 406 407 408 409 410 411 | sqlite3_vtab **ppVTab, /* Write the resulting vtab structure here */ char **pzErr /* Write any error message here */ ){ Fts5Global *pGlobal = (Fts5Global*)pAux; const char **azConfig = (const char**)argv; int rc = SQLITE_OK; /* Return code */ Fts5Config *pConfig = 0; /* Results of parsing argc/argv */ Fts5FullTable *pTab = 0; /* New virtual table object */ /* Allocate the new vtab object and parse the configuration */ pTab = (Fts5FullTable*)sqlite3Fts5MallocZero(&rc, sizeof(Fts5FullTable)); if( rc==SQLITE_OK ){ rc = sqlite3Fts5ConfigParse(pGlobal, db, argc, azConfig, &pConfig, pzErr); assert( (rc==SQLITE_OK && *pzErr==0) || pConfig==0 ); } if( rc==SQLITE_OK ){ pTab->p.pConfig = pConfig; pTab->pGlobal = pGlobal; } /* Open the index sub-system */ if( rc==SQLITE_OK ){ rc = sqlite3Fts5IndexOpen(pConfig, bCreate, &pTab->p.pIndex, pzErr); } /* Open the storage sub-system */ if( rc==SQLITE_OK ){ rc = sqlite3Fts5StorageOpen( pConfig, pTab->p.pIndex, bCreate, &pTab->pStorage, pzErr ); } /* Call sqlite3_declare_vtab() */ if( rc==SQLITE_OK ){ rc = sqlite3Fts5ConfigDeclareVtab(pConfig); } /* Load the initial configuration */ if( rc==SQLITE_OK ){ assert( pConfig->pzErrmsg==0 ); pConfig->pzErrmsg = pzErr; rc = sqlite3Fts5IndexLoadConfig(pTab->p.pIndex); sqlite3Fts5IndexRollback(pTab->p.pIndex); pConfig->pzErrmsg = 0; } if( rc!=SQLITE_OK ){ fts5FreeVtab(pTab); pTab = 0; }else if( bCreate ){ |
︙ | ︙ | |||
461 462 463 464 465 466 467 468 469 470 471 472 | if( sqlite3_libversion_number()>=3008012 ) #endif { pIdxInfo->idxFlags |= SQLITE_INDEX_SCAN_UNIQUE; } #endif } /* ** Implementation of the xBestIndex method for FTS5 tables. Within the ** WHERE constraint, it searches for the following: ** | > > > > > > > > > > > > > > > > > | > | | | | > > > > > > > > > > > > > > > > > > > > > > > | 461 462 463 464 465 466 467 468 469 470 471 472 473 474 475 476 477 478 479 480 481 482 483 484 485 486 487 488 489 490 491 492 493 494 495 496 497 498 499 500 501 502 503 504 505 506 507 508 509 510 511 512 513 514 515 516 517 518 519 520 521 522 523 524 525 526 527 528 529 530 | if( sqlite3_libversion_number()>=3008012 ) #endif { pIdxInfo->idxFlags |= SQLITE_INDEX_SCAN_UNIQUE; } #endif } static int fts5UsePatternMatch( Fts5Config *pConfig, struct sqlite3_index_constraint *p ){ assert( FTS5_PATTERN_GLOB==SQLITE_INDEX_CONSTRAINT_GLOB ); assert( FTS5_PATTERN_LIKE==SQLITE_INDEX_CONSTRAINT_LIKE ); if( pConfig->ePattern==FTS5_PATTERN_GLOB && p->op==FTS5_PATTERN_GLOB ){ return 1; } if( pConfig->ePattern==FTS5_PATTERN_LIKE && (p->op==FTS5_PATTERN_LIKE || p->op==FTS5_PATTERN_GLOB) ){ return 1; } return 0; } /* ** Implementation of the xBestIndex method for FTS5 tables. Within the ** WHERE constraint, it searches for the following: ** ** 1. A MATCH constraint against the table column. ** 2. A MATCH constraint against the "rank" column. ** 3. A MATCH constraint against some other column. ** 4. An == constraint against the rowid column. ** 5. A < or <= constraint against the rowid column. ** 6. A > or >= constraint against the rowid column. ** ** Within the ORDER BY, the following are supported: ** ** 5. ORDER BY rank [ASC|DESC] ** 6. ORDER BY rowid [ASC|DESC] ** ** Information for the xFilter call is passed via both the idxNum and ** idxStr variables. Specifically, idxNum is a bitmask of the following ** flags used to encode the ORDER BY clause: ** ** FTS5_BI_ORDER_RANK ** FTS5_BI_ORDER_ROWID ** FTS5_BI_ORDER_DESC ** ** idxStr is used to encode data from the WHERE clause. For each argument ** passed to the xFilter method, the following is appended to idxStr: ** ** Match against table column: "m" ** Match against rank column: "r" ** Match against other column: "M<column-number>" ** LIKE against other column: "L<column-number>" ** GLOB against other column: "G<column-number>" ** Equality constraint against the rowid: "=" ** A < or <= against the rowid: "<" ** A > or >= against the rowid: ">" ** ** This function ensures that there is at most one "r" or "=". And that if ** there exists an "=" then there is no "<" or ">". ** ** Costs are assigned as follows: ** ** a) If an unusable MATCH operator is present in the WHERE clause, the ** cost is unconditionally set to 1e50 (a really big number). ** ** a) If a MATCH operator is present, the cost depends on the other |
︙ | ︙ | |||
504 505 506 507 508 509 510 | ** Costs are not modified by the ORDER BY clause. */ static int fts5BestIndexMethod(sqlite3_vtab *pVTab, sqlite3_index_info *pInfo){ Fts5Table *pTab = (Fts5Table*)pVTab; Fts5Config *pConfig = pTab->pConfig; const int nCol = pConfig->nCol; int idxFlags = 0; /* Parameter passed through to xFilter() */ | < < | < < | < | < < < < < < < < < < < | | | > > | > > > > > > | > > > > > < | | | < < < > > > > > > > > > > > > > | > > > | > > > > > > > > > > > > > > | > | | | > > > > > | > > > | > > > | < | | | | | | | | < < < < < < < < < < | | | | | | | 545 546 547 548 549 550 551 552 553 554 555 556 557 558 559 560 561 562 563 564 565 566 567 568 569 570 571 572 573 574 575 576 577 578 579 580 581 582 583 584 585 586 587 588 589 590 591 592 593 594 595 596 597 598 599 600 601 602 603 604 605 606 607 608 609 610 611 612 613 614 615 616 617 618 619 620 621 622 623 624 625 626 627 628 629 630 631 632 633 634 635 636 637 638 639 640 641 642 643 644 645 646 647 648 649 650 651 652 653 654 655 656 657 658 659 660 661 662 663 664 665 666 667 668 669 670 671 672 673 674 675 676 677 678 679 680 681 682 683 684 685 686 687 688 689 690 691 692 693 694 695 696 697 698 699 700 701 702 703 704 705 706 707 708 709 710 711 712 713 | ** Costs are not modified by the ORDER BY clause. */ static int fts5BestIndexMethod(sqlite3_vtab *pVTab, sqlite3_index_info *pInfo){ Fts5Table *pTab = (Fts5Table*)pVTab; Fts5Config *pConfig = pTab->pConfig; const int nCol = pConfig->nCol; int idxFlags = 0; /* Parameter passed through to xFilter() */ int i; char *idxStr; int iIdxStr = 0; int iCons = 0; int bSeenEq = 0; int bSeenGt = 0; int bSeenLt = 0; int bSeenMatch = 0; int bSeenRank = 0; assert( SQLITE_INDEX_CONSTRAINT_EQ<SQLITE_INDEX_CONSTRAINT_MATCH ); assert( SQLITE_INDEX_CONSTRAINT_GT<SQLITE_INDEX_CONSTRAINT_MATCH ); assert( SQLITE_INDEX_CONSTRAINT_LE<SQLITE_INDEX_CONSTRAINT_MATCH ); assert( SQLITE_INDEX_CONSTRAINT_GE<SQLITE_INDEX_CONSTRAINT_MATCH ); assert( SQLITE_INDEX_CONSTRAINT_LE<SQLITE_INDEX_CONSTRAINT_MATCH ); if( pConfig->bLock ){ pTab->base.zErrMsg = sqlite3_mprintf( "recursively defined fts5 content table" ); return SQLITE_ERROR; } idxStr = (char*)sqlite3_malloc(pInfo->nConstraint * 8 + 1); if( idxStr==0 ) return SQLITE_NOMEM; pInfo->idxStr = idxStr; pInfo->needToFreeIdxStr = 1; for(i=0; i<pInfo->nConstraint; i++){ struct sqlite3_index_constraint *p = &pInfo->aConstraint[i]; int iCol = p->iColumn; if( p->op==SQLITE_INDEX_CONSTRAINT_MATCH || (p->op==SQLITE_INDEX_CONSTRAINT_EQ && iCol>=nCol) ){ /* A MATCH operator or equivalent */ if( p->usable==0 || iCol<0 ){ /* As there exists an unusable MATCH constraint this is an ** unusable plan. Set a prohibitively high cost. */ pInfo->estimatedCost = 1e50; assert( iIdxStr < pInfo->nConstraint*6 + 1 ); idxStr[iIdxStr] = 0; return SQLITE_OK; }else{ if( iCol==nCol+1 ){ if( bSeenRank ) continue; idxStr[iIdxStr++] = 'r'; bSeenRank = 1; }else if( iCol>=0 ){ bSeenMatch = 1; idxStr[iIdxStr++] = 'M'; sqlite3_snprintf(6, &idxStr[iIdxStr], "%d", iCol); idxStr += strlen(&idxStr[iIdxStr]); assert( idxStr[iIdxStr]=='\0' ); } pInfo->aConstraintUsage[i].argvIndex = ++iCons; pInfo->aConstraintUsage[i].omit = 1; } }else if( p->usable ){ if( iCol>=0 && iCol<nCol && fts5UsePatternMatch(pConfig, p) ){ assert( p->op==FTS5_PATTERN_LIKE || p->op==FTS5_PATTERN_GLOB ); idxStr[iIdxStr++] = p->op==FTS5_PATTERN_LIKE ? 'L' : 'G'; sqlite3_snprintf(6, &idxStr[iIdxStr], "%d", iCol); idxStr += strlen(&idxStr[iIdxStr]); pInfo->aConstraintUsage[i].argvIndex = ++iCons; assert( idxStr[iIdxStr]=='\0' ); }else if( bSeenEq==0 && p->op==SQLITE_INDEX_CONSTRAINT_EQ && iCol<0 ){ idxStr[iIdxStr++] = '='; bSeenEq = 1; pInfo->aConstraintUsage[i].argvIndex = ++iCons; } } } if( bSeenEq==0 ){ for(i=0; i<pInfo->nConstraint; i++){ struct sqlite3_index_constraint *p = &pInfo->aConstraint[i]; if( p->iColumn<0 && p->usable ){ int op = p->op; if( op==SQLITE_INDEX_CONSTRAINT_LT || op==SQLITE_INDEX_CONSTRAINT_LE ){ if( bSeenLt ) continue; idxStr[iIdxStr++] = '<'; pInfo->aConstraintUsage[i].argvIndex = ++iCons; bSeenLt = 1; }else if( op==SQLITE_INDEX_CONSTRAINT_GT || op==SQLITE_INDEX_CONSTRAINT_GE ){ if( bSeenGt ) continue; idxStr[iIdxStr++] = '>'; pInfo->aConstraintUsage[i].argvIndex = ++iCons; bSeenGt = 1; } } } } idxStr[iIdxStr] = '\0'; /* Set idxFlags flags for the ORDER BY clause */ if( pInfo->nOrderBy==1 ){ int iSort = pInfo->aOrderBy[0].iColumn; if( iSort==(pConfig->nCol+1) && bSeenMatch ){ idxFlags |= FTS5_BI_ORDER_RANK; }else if( iSort==-1 ){ idxFlags |= FTS5_BI_ORDER_ROWID; } if( BitFlagTest(idxFlags, FTS5_BI_ORDER_RANK|FTS5_BI_ORDER_ROWID) ){ pInfo->orderByConsumed = 1; if( pInfo->aOrderBy[0].desc ){ idxFlags |= FTS5_BI_ORDER_DESC; } } } /* Calculate the estimated cost based on the flags set in idxFlags. */ if( bSeenEq ){ pInfo->estimatedCost = bSeenMatch ? 100.0 : 10.0; if( bSeenMatch==0 ) fts5SetUniqueFlag(pInfo); }else if( bSeenLt && bSeenGt ){ pInfo->estimatedCost = bSeenMatch ? 500.0 : 250000.0; }else if( bSeenLt || bSeenGt ){ pInfo->estimatedCost = bSeenMatch ? 750.0 : 750000.0; }else{ pInfo->estimatedCost = bSeenMatch ? 1000.0 : 1000000.0; } pInfo->idxNum = idxFlags; return SQLITE_OK; } static int fts5NewTransaction(Fts5FullTable *pTab){ Fts5Cursor *pCsr; for(pCsr=pTab->pGlobal->pCsr; pCsr; pCsr=pCsr->pNext){ if( pCsr->base.pVtab==(sqlite3_vtab*)pTab ) return SQLITE_OK; } return sqlite3Fts5StorageReset(pTab->pStorage); } /* ** Implementation of xOpen method. */ static int fts5OpenMethod(sqlite3_vtab *pVTab, sqlite3_vtab_cursor **ppCsr){ Fts5FullTable *pTab = (Fts5FullTable*)pVTab; Fts5Config *pConfig = pTab->p.pConfig; Fts5Cursor *pCsr = 0; /* New cursor object */ sqlite3_int64 nByte; /* Bytes of space to allocate */ int rc; /* Return code */ rc = fts5NewTransaction(pTab); if( rc==SQLITE_OK ){ nByte = sizeof(Fts5Cursor) + pConfig->nCol * sizeof(int); pCsr = (Fts5Cursor*)sqlite3_malloc64(nByte); if( pCsr ){ Fts5Global *pGlobal = pTab->pGlobal; memset(pCsr, 0, (size_t)nByte); pCsr->aColumnSize = (int*)&pCsr[1]; pCsr->pNext = pGlobal->pCsr; pGlobal->pCsr = pCsr; pCsr->iCsrId = ++pGlobal->iNextId; }else{ rc = SQLITE_NOMEM; } |
︙ | ︙ | |||
669 670 671 672 673 674 675 | | FTS5CSR_REQUIRE_DOCSIZE | FTS5CSR_REQUIRE_INST | FTS5CSR_REQUIRE_POSLIST ); } static void fts5FreeCursorComponents(Fts5Cursor *pCsr){ | | | 734 735 736 737 738 739 740 741 742 743 744 745 746 747 748 | | FTS5CSR_REQUIRE_DOCSIZE | FTS5CSR_REQUIRE_INST | FTS5CSR_REQUIRE_POSLIST ); } static void fts5FreeCursorComponents(Fts5Cursor *pCsr){ Fts5FullTable *pTab = (Fts5FullTable*)(pCsr->base.pVtab); Fts5Auxdata *pData; Fts5Auxdata *pNext; sqlite3_free(pCsr->aInstIter); sqlite3_free(pCsr->aInst); if( pCsr->pStmt ){ int eStmt = fts5StmtType(pCsr); |
︙ | ︙ | |||
703 704 705 706 707 708 709 710 711 712 713 714 715 716 717 718 719 | sqlite3_free(pCsr->apRankArg); if( CsrFlagTest(pCsr, FTS5CSR_FREE_ZRANK) ){ sqlite3_free(pCsr->zRank); sqlite3_free(pCsr->zRankArgs); } memset(&pCsr->ePlan, 0, sizeof(Fts5Cursor) - ((u8*)&pCsr->ePlan - (u8*)pCsr)); } /* ** Close the cursor. For additional information see the documentation ** on the xClose method of the virtual table interface. */ static int fts5CloseMethod(sqlite3_vtab_cursor *pCursor){ if( pCursor ){ | > | | | 768 769 770 771 772 773 774 775 776 777 778 779 780 781 782 783 784 785 786 787 788 789 790 791 792 793 794 795 796 797 798 799 800 801 802 803 804 805 806 807 808 809 810 811 812 813 814 | sqlite3_free(pCsr->apRankArg); if( CsrFlagTest(pCsr, FTS5CSR_FREE_ZRANK) ){ sqlite3_free(pCsr->zRank); sqlite3_free(pCsr->zRankArgs); } sqlite3Fts5IndexCloseReader(pTab->p.pIndex); memset(&pCsr->ePlan, 0, sizeof(Fts5Cursor) - ((u8*)&pCsr->ePlan - (u8*)pCsr)); } /* ** Close the cursor. For additional information see the documentation ** on the xClose method of the virtual table interface. */ static int fts5CloseMethod(sqlite3_vtab_cursor *pCursor){ if( pCursor ){ Fts5FullTable *pTab = (Fts5FullTable*)(pCursor->pVtab); Fts5Cursor *pCsr = (Fts5Cursor*)pCursor; Fts5Cursor **pp; fts5FreeCursorComponents(pCsr); /* Remove the cursor from the Fts5Global.pCsr list */ for(pp=&pTab->pGlobal->pCsr; (*pp)!=pCsr; pp=&(*pp)->pNext); *pp = pCsr->pNext; sqlite3_free(pCsr); } return SQLITE_OK; } static int fts5SorterNext(Fts5Cursor *pCsr){ Fts5Sorter *pSorter = pCsr->pSorter; int rc; rc = sqlite3_step(pSorter->pStmt); if( rc==SQLITE_DONE ){ rc = SQLITE_OK; CsrFlagSet(pCsr, FTS5CSR_EOF|FTS5CSR_REQUIRE_CONTENT); }else if( rc==SQLITE_ROW ){ const u8 *a; const u8 *aBlob; int nBlob; int i; int iOff = 0; rc = SQLITE_OK; |
︙ | ︙ | |||
770 771 772 773 774 775 776 | } /* ** Set the FTS5CSR_REQUIRE_RESEEK flag on all FTS5_PLAN_MATCH cursors ** open on table pTab. */ | | | 836 837 838 839 840 841 842 843 844 845 846 847 848 849 850 | } /* ** Set the FTS5CSR_REQUIRE_RESEEK flag on all FTS5_PLAN_MATCH cursors ** open on table pTab. */ static void fts5TripCursors(Fts5FullTable *pTab){ Fts5Cursor *pCsr; for(pCsr=pTab->pGlobal->pCsr; pCsr; pCsr=pCsr->pNext){ if( pCsr->ePlan==FTS5_PLAN_MATCH && pCsr->base.pVtab==(sqlite3_vtab*)pTab ){ CsrFlagSet(pCsr, FTS5CSR_REQUIRE_RESEEK); } |
︙ | ︙ | |||
797 798 799 800 801 802 803 | ** Return SQLITE_OK if successful or if no reseek was required, or an ** error code if an error occurred. */ static int fts5CursorReseek(Fts5Cursor *pCsr, int *pbSkip){ int rc = SQLITE_OK; assert( *pbSkip==0 ); if( CsrFlagTest(pCsr, FTS5CSR_REQUIRE_RESEEK) ){ | | | | 863 864 865 866 867 868 869 870 871 872 873 874 875 876 877 878 879 880 881 | ** Return SQLITE_OK if successful or if no reseek was required, or an ** error code if an error occurred. */ static int fts5CursorReseek(Fts5Cursor *pCsr, int *pbSkip){ int rc = SQLITE_OK; assert( *pbSkip==0 ); if( CsrFlagTest(pCsr, FTS5CSR_REQUIRE_RESEEK) ){ Fts5FullTable *pTab = (Fts5FullTable*)(pCsr->base.pVtab); int bDesc = pCsr->bDesc; i64 iRowid = sqlite3Fts5ExprRowid(pCsr->pExpr); rc = sqlite3Fts5ExprFirst(pCsr->pExpr, pTab->p.pIndex, iRowid, bDesc); if( rc==SQLITE_OK && iRowid!=sqlite3Fts5ExprRowid(pCsr->pExpr) ){ *pbSkip = 1; } CsrFlagClear(pCsr, FTS5CSR_REQUIRE_RESEEK); fts5CsrNewrow(pCsr); if( sqlite3Fts5ExprEof(pCsr->pExpr) ){ |
︙ | ︙ | |||
853 854 855 856 857 858 859 | } case FTS5_PLAN_SORTED_MATCH: { rc = fts5SorterNext(pCsr); break; } | | > > > > > > > > > | 919 920 921 922 923 924 925 926 927 928 929 930 931 932 933 934 935 936 937 938 939 940 941 942 943 944 945 946 947 948 949 950 | } case FTS5_PLAN_SORTED_MATCH: { rc = fts5SorterNext(pCsr); break; } default: { Fts5Config *pConfig = ((Fts5Table*)pCursor->pVtab)->pConfig; pConfig->bLock++; rc = sqlite3_step(pCsr->pStmt); pConfig->bLock--; if( rc!=SQLITE_ROW ){ CsrFlagSet(pCsr, FTS5CSR_EOF); rc = sqlite3_reset(pCsr->pStmt); if( rc!=SQLITE_OK ){ pCursor->pVtab->zErrMsg = sqlite3_mprintf( "%s", sqlite3_errmsg(pConfig->db) ); } }else{ rc = SQLITE_OK; } break; } } } return rc; } |
︙ | ︙ | |||
898 899 900 901 902 903 904 | } va_end(ap); *ppStmt = pRet; return rc; } | | > > > > | | | | | | 973 974 975 976 977 978 979 980 981 982 983 984 985 986 987 988 989 990 991 992 993 994 995 996 997 998 999 1000 1001 1002 1003 1004 1005 1006 1007 1008 1009 1010 1011 1012 1013 1014 1015 | } va_end(ap); *ppStmt = pRet; return rc; } static int fts5CursorFirstSorted( Fts5FullTable *pTab, Fts5Cursor *pCsr, int bDesc ){ Fts5Config *pConfig = pTab->p.pConfig; Fts5Sorter *pSorter; int nPhrase; sqlite3_int64 nByte; int rc; const char *zRank = pCsr->zRank; const char *zRankArgs = pCsr->zRankArgs; nPhrase = sqlite3Fts5ExprPhraseCount(pCsr->pExpr); nByte = sizeof(Fts5Sorter) + sizeof(int) * (nPhrase-1); pSorter = (Fts5Sorter*)sqlite3_malloc64(nByte); if( pSorter==0 ) return SQLITE_NOMEM; memset(pSorter, 0, (size_t)nByte); pSorter->nIdx = nPhrase; /* TODO: It would be better to have some system for reusing statement ** handles here, rather than preparing a new one for each query. But that ** is not possible as SQLite reference counts the virtual table objects. ** And since the statement required here reads from this very virtual ** table, saving it creates a circular reference. ** ** If SQLite a built-in statement cache, this wouldn't be a problem. */ rc = fts5PrepareStatement(&pSorter->pStmt, pConfig, "SELECT rowid, rank FROM %Q.%Q ORDER BY %s(\"%w\"%s%s) %s", pConfig->zDb, pConfig->zName, zRank, pConfig->zName, (zRankArgs ? ", " : ""), (zRankArgs ? zRankArgs : ""), bDesc ? "DESC" : "ASC" ); pCsr->pSorter = pSorter; |
︙ | ︙ | |||
946 947 948 949 950 951 952 | sqlite3_free(pSorter); pCsr->pSorter = 0; } return rc; } | | | | | | | | | | | | | | 1025 1026 1027 1028 1029 1030 1031 1032 1033 1034 1035 1036 1037 1038 1039 1040 1041 1042 1043 1044 1045 1046 1047 1048 1049 1050 1051 1052 1053 1054 1055 1056 1057 1058 1059 1060 1061 1062 1063 1064 1065 1066 1067 1068 1069 1070 1071 1072 1073 1074 1075 1076 1077 1078 1079 1080 1081 1082 1083 1084 1085 1086 1087 1088 1089 1090 1091 1092 1093 1094 1095 1096 1097 1098 1099 1100 1101 1102 1103 1104 1105 1106 1107 1108 1109 1110 1111 1112 1113 1114 1115 1116 1117 1118 1119 1120 1121 | sqlite3_free(pSorter); pCsr->pSorter = 0; } return rc; } static int fts5CursorFirst(Fts5FullTable *pTab, Fts5Cursor *pCsr, int bDesc){ int rc; Fts5Expr *pExpr = pCsr->pExpr; rc = sqlite3Fts5ExprFirst(pExpr, pTab->p.pIndex, pCsr->iFirstRowid, bDesc); if( sqlite3Fts5ExprEof(pExpr) ){ CsrFlagSet(pCsr, FTS5CSR_EOF); } fts5CsrNewrow(pCsr); return rc; } /* ** Process a "special" query. A special query is identified as one with a ** MATCH expression that begins with a '*' character. The remainder of ** the text passed to the MATCH operator are used as the special query ** parameters. */ static int fts5SpecialMatch( Fts5FullTable *pTab, Fts5Cursor *pCsr, const char *zQuery ){ int rc = SQLITE_OK; /* Return code */ const char *z = zQuery; /* Special query text */ int n; /* Number of bytes in text at z */ while( z[0]==' ' ) z++; for(n=0; z[n] && z[n]!=' '; n++); assert( pTab->p.base.zErrMsg==0 ); pCsr->ePlan = FTS5_PLAN_SPECIAL; if( n==5 && 0==sqlite3_strnicmp("reads", z, n) ){ pCsr->iSpecial = sqlite3Fts5IndexReads(pTab->p.pIndex); } else if( n==2 && 0==sqlite3_strnicmp("id", z, n) ){ pCsr->iSpecial = pCsr->iCsrId; } else{ /* An unrecognized directive. Return an error message. */ pTab->p.base.zErrMsg = sqlite3_mprintf("unknown special query: %.*s", n, z); rc = SQLITE_ERROR; } return rc; } /* ** Search for an auxiliary function named zName that can be used with table ** pTab. If one is found, return a pointer to the corresponding Fts5Auxiliary ** structure. Otherwise, if no such function exists, return NULL. */ static Fts5Auxiliary *fts5FindAuxiliary(Fts5FullTable *pTab, const char *zName){ Fts5Auxiliary *pAux; for(pAux=pTab->pGlobal->pAux; pAux; pAux=pAux->pNext){ if( sqlite3_stricmp(zName, pAux->zFunc)==0 ) return pAux; } /* No function of the specified name was found. Return 0. */ return 0; } static int fts5FindRankFunction(Fts5Cursor *pCsr){ Fts5FullTable *pTab = (Fts5FullTable*)(pCsr->base.pVtab); Fts5Config *pConfig = pTab->p.pConfig; int rc = SQLITE_OK; Fts5Auxiliary *pAux = 0; const char *zRank = pCsr->zRank; const char *zRankArgs = pCsr->zRankArgs; if( zRankArgs ){ char *zSql = sqlite3Fts5Mprintf(&rc, "SELECT %s", zRankArgs); if( zSql ){ sqlite3_stmt *pStmt = 0; rc = sqlite3_prepare_v3(pConfig->db, zSql, -1, SQLITE_PREPARE_PERSISTENT, &pStmt, 0); sqlite3_free(zSql); assert( rc==SQLITE_OK || pCsr->pRankArgStmt==0 ); if( rc==SQLITE_OK ){ if( SQLITE_ROW==sqlite3_step(pStmt) ){ sqlite3_int64 nByte; pCsr->nRankArg = sqlite3_column_count(pStmt); nByte = sizeof(sqlite3_value*)*pCsr->nRankArg; pCsr->apRankArg = (sqlite3_value**)sqlite3Fts5MallocZero(&rc, nByte); if( rc==SQLITE_OK ){ int i; for(i=0; i<pCsr->nRankArg; i++){ pCsr->apRankArg[i] = sqlite3_column_value(pStmt, i); |
︙ | ︙ | |||
1050 1051 1052 1053 1054 1055 1056 | } } } if( rc==SQLITE_OK ){ pAux = fts5FindAuxiliary(pTab, zRank); if( pAux==0 ){ | | | | 1129 1130 1131 1132 1133 1134 1135 1136 1137 1138 1139 1140 1141 1142 1143 1144 | } } } if( rc==SQLITE_OK ){ pAux = fts5FindAuxiliary(pTab, zRank); if( pAux==0 ){ assert( pTab->p.base.zErrMsg==0 ); pTab->p.base.zErrMsg = sqlite3_mprintf("no such function: %s", zRank); rc = SQLITE_ERROR; } } pCsr->pRank = pAux; return rc; } |
︙ | ︙ | |||
1122 1123 1124 1125 1126 1127 1128 | ** 1. Full-text search using a MATCH operator. ** 2. A by-rowid lookup. ** 3. A full-table scan. */ static int fts5FilterMethod( sqlite3_vtab_cursor *pCursor, /* The cursor used for this query */ int idxNum, /* Strategy index */ | | | | < < > > > < > > > | > > > | | | < | | | < | > | > > > > > > > | > > > > > > > > > > > > > > > | > > > > > > | > | > > > > > > > > > > > > > > > > > > | > > > > | 1201 1202 1203 1204 1205 1206 1207 1208 1209 1210 1211 1212 1213 1214 1215 1216 1217 1218 1219 1220 1221 1222 1223 1224 1225 1226 1227 1228 1229 1230 1231 1232 1233 1234 1235 1236 1237 1238 1239 1240 1241 1242 1243 1244 1245 1246 1247 1248 1249 1250 1251 1252 1253 1254 1255 1256 1257 1258 1259 1260 1261 1262 1263 1264 1265 1266 1267 1268 1269 1270 1271 1272 1273 1274 1275 1276 1277 1278 1279 1280 1281 1282 1283 1284 1285 1286 1287 1288 1289 1290 1291 1292 1293 1294 1295 1296 1297 1298 1299 1300 1301 1302 1303 1304 1305 1306 1307 1308 1309 1310 1311 1312 1313 1314 1315 1316 1317 1318 1319 1320 | ** 1. Full-text search using a MATCH operator. ** 2. A by-rowid lookup. ** 3. A full-table scan. */ static int fts5FilterMethod( sqlite3_vtab_cursor *pCursor, /* The cursor used for this query */ int idxNum, /* Strategy index */ const char *idxStr, /* Unused */ int nVal, /* Number of elements in apVal */ sqlite3_value **apVal /* Arguments for the indexing scheme */ ){ Fts5FullTable *pTab = (Fts5FullTable*)(pCursor->pVtab); Fts5Config *pConfig = pTab->p.pConfig; Fts5Cursor *pCsr = (Fts5Cursor*)pCursor; int rc = SQLITE_OK; /* Error code */ int bDesc; /* True if ORDER BY [rank|rowid] DESC */ int bOrderByRank; /* True if ORDER BY rank */ sqlite3_value *pRank = 0; /* rank MATCH ? expression (or NULL) */ sqlite3_value *pRowidEq = 0; /* rowid = ? expression (or NULL) */ sqlite3_value *pRowidLe = 0; /* rowid <= ? expression (or NULL) */ sqlite3_value *pRowidGe = 0; /* rowid >= ? expression (or NULL) */ int iCol; /* Column on LHS of MATCH operator */ char **pzErrmsg = pConfig->pzErrmsg; int i; int iIdxStr = 0; Fts5Expr *pExpr = 0; if( pConfig->bLock ){ pTab->p.base.zErrMsg = sqlite3_mprintf( "recursively defined fts5 content table" ); return SQLITE_ERROR; } if( pCsr->ePlan ){ fts5FreeCursorComponents(pCsr); memset(&pCsr->ePlan, 0, sizeof(Fts5Cursor) - ((u8*)&pCsr->ePlan-(u8*)pCsr)); } assert( pCsr->pStmt==0 ); assert( pCsr->pExpr==0 ); assert( pCsr->csrflags==0 ); assert( pCsr->pRank==0 ); assert( pCsr->zRank==0 ); assert( pCsr->zRankArgs==0 ); assert( pTab->pSortCsr==0 || nVal==0 ); assert( pzErrmsg==0 || pzErrmsg==&pTab->p.base.zErrMsg ); pConfig->pzErrmsg = &pTab->p.base.zErrMsg; /* Decode the arguments passed through to this function. */ for(i=0; i<nVal; i++){ switch( idxStr[iIdxStr++] ){ case 'r': pRank = apVal[i]; break; case 'M': { const char *zText = (const char*)sqlite3_value_text(apVal[i]); if( zText==0 ) zText = ""; iCol = 0; do{ iCol = iCol*10 + (idxStr[iIdxStr]-'0'); iIdxStr++; }while( idxStr[iIdxStr]>='0' && idxStr[iIdxStr]<='9' ); if( zText[0]=='*' ){ /* The user has issued a query of the form "MATCH '*...'". This ** indicates that the MATCH expression is not a full text query, ** but a request for an internal parameter. */ rc = fts5SpecialMatch(pTab, pCsr, &zText[1]); goto filter_out; }else{ char **pzErr = &pTab->p.base.zErrMsg; rc = sqlite3Fts5ExprNew(pConfig, 0, iCol, zText, &pExpr, pzErr); if( rc==SQLITE_OK ){ rc = sqlite3Fts5ExprAnd(&pCsr->pExpr, pExpr); pExpr = 0; } if( rc!=SQLITE_OK ) goto filter_out; } break; } case 'L': case 'G': { int bGlob = (idxStr[iIdxStr-1]=='G'); const char *zText = (const char*)sqlite3_value_text(apVal[i]); iCol = 0; do{ iCol = iCol*10 + (idxStr[iIdxStr]-'0'); iIdxStr++; }while( idxStr[iIdxStr]>='0' && idxStr[iIdxStr]<='9' ); if( zText ){ rc = sqlite3Fts5ExprPattern(pConfig, bGlob, iCol, zText, &pExpr); } if( rc==SQLITE_OK ){ rc = sqlite3Fts5ExprAnd(&pCsr->pExpr, pExpr); pExpr = 0; } if( rc!=SQLITE_OK ) goto filter_out; break; } case '=': pRowidEq = apVal[i]; break; case '<': pRowidLe = apVal[i]; break; default: assert( idxStr[iIdxStr-1]=='>' ); pRowidGe = apVal[i]; break; } } bOrderByRank = ((idxNum & FTS5_BI_ORDER_RANK) ? 1 : 0); pCsr->bDesc = bDesc = ((idxNum & FTS5_BI_ORDER_DESC) ? 1 : 0); /* Set the cursor upper and lower rowid limits. Only some strategies ** actually use them. This is ok, as the xBestIndex() method leaves the ** sqlite3_index_constraint.omit flag clear for range constraints ** on the rowid field. */ |
︙ | ︙ | |||
1198 1199 1200 1201 1202 1203 1204 | /* If pSortCsr is non-NULL, then this call is being made as part of ** processing for a "... MATCH <expr> ORDER BY rank" query (ePlan is ** set to FTS5_PLAN_SORTED_MATCH). pSortCsr is the cursor that will ** return results to the user for this query. The current cursor ** (pCursor) is used to execute the query issued by function ** fts5CursorFirstSorted() above. */ assert( pRowidEq==0 && pRowidLe==0 && pRowidGe==0 && pRank==0 ); | | | < < < < < < < < < < < < | | | | | | < < | > | | > > | 1333 1334 1335 1336 1337 1338 1339 1340 1341 1342 1343 1344 1345 1346 1347 1348 1349 1350 1351 1352 1353 1354 1355 1356 1357 1358 1359 1360 1361 1362 1363 1364 1365 1366 1367 1368 1369 1370 1371 1372 1373 1374 1375 1376 1377 1378 1379 1380 1381 1382 1383 1384 1385 1386 1387 1388 1389 1390 1391 1392 1393 1394 1395 1396 | /* If pSortCsr is non-NULL, then this call is being made as part of ** processing for a "... MATCH <expr> ORDER BY rank" query (ePlan is ** set to FTS5_PLAN_SORTED_MATCH). pSortCsr is the cursor that will ** return results to the user for this query. The current cursor ** (pCursor) is used to execute the query issued by function ** fts5CursorFirstSorted() above. */ assert( pRowidEq==0 && pRowidLe==0 && pRowidGe==0 && pRank==0 ); assert( nVal==0 && bOrderByRank==0 && bDesc==0 ); assert( pCsr->iLastRowid==LARGEST_INT64 ); assert( pCsr->iFirstRowid==SMALLEST_INT64 ); if( pTab->pSortCsr->bDesc ){ pCsr->iLastRowid = pTab->pSortCsr->iFirstRowid; pCsr->iFirstRowid = pTab->pSortCsr->iLastRowid; }else{ pCsr->iLastRowid = pTab->pSortCsr->iLastRowid; pCsr->iFirstRowid = pTab->pSortCsr->iFirstRowid; } pCsr->ePlan = FTS5_PLAN_SOURCE; pCsr->pExpr = pTab->pSortCsr->pExpr; rc = fts5CursorFirst(pTab, pCsr, bDesc); }else if( pCsr->pExpr ){ rc = fts5CursorParseRank(pConfig, pCsr, pRank); if( rc==SQLITE_OK ){ if( bOrderByRank ){ pCsr->ePlan = FTS5_PLAN_SORTED_MATCH; rc = fts5CursorFirstSorted(pTab, pCsr, bDesc); }else{ pCsr->ePlan = FTS5_PLAN_MATCH; rc = fts5CursorFirst(pTab, pCsr, bDesc); } } }else if( pConfig->zContent==0 ){ *pConfig->pzErrmsg = sqlite3_mprintf( "%s: table does not support scanning", pConfig->zName ); rc = SQLITE_ERROR; }else{ /* This is either a full-table scan (ePlan==FTS5_PLAN_SCAN) or a lookup ** by rowid (ePlan==FTS5_PLAN_ROWID). */ pCsr->ePlan = (pRowidEq ? FTS5_PLAN_ROWID : FTS5_PLAN_SCAN); rc = sqlite3Fts5StorageStmt( pTab->pStorage, fts5StmtType(pCsr), &pCsr->pStmt, &pTab->p.base.zErrMsg ); if( rc==SQLITE_OK ){ if( pRowidEq!=0 ){ assert( pCsr->ePlan==FTS5_PLAN_ROWID ); sqlite3_bind_value(pCsr->pStmt, 1, pRowidEq); }else{ sqlite3_bind_int64(pCsr->pStmt, 1, pCsr->iFirstRowid); sqlite3_bind_int64(pCsr->pStmt, 2, pCsr->iLastRowid); } rc = fts5NextMethod(pCursor); } } filter_out: sqlite3Fts5ExprFree(pExpr); pConfig->pzErrmsg = pzErrmsg; return rc; } /* ** This is the xEof method of the virtual table. SQLite calls this ** routine to find out if it has reached the end of a result set. |
︙ | ︙ | |||
1329 1330 1331 1332 1333 1334 1335 | ** be left in sqlite3_vtab.zErrMsg. */ static int fts5SeekCursor(Fts5Cursor *pCsr, int bErrormsg){ int rc = SQLITE_OK; /* If the cursor does not yet have a statement handle, obtain one now. */ if( pCsr->pStmt==0 ){ | | | | > > > > > > > | | | | | | 1453 1454 1455 1456 1457 1458 1459 1460 1461 1462 1463 1464 1465 1466 1467 1468 1469 1470 1471 1472 1473 1474 1475 1476 1477 1478 1479 1480 1481 1482 1483 1484 1485 1486 1487 1488 1489 1490 1491 1492 1493 1494 1495 1496 1497 1498 1499 1500 1501 1502 1503 1504 1505 1506 1507 1508 1509 1510 1511 1512 1513 1514 1515 1516 1517 1518 1519 1520 1521 1522 1523 1524 1525 1526 1527 1528 1529 | ** be left in sqlite3_vtab.zErrMsg. */ static int fts5SeekCursor(Fts5Cursor *pCsr, int bErrormsg){ int rc = SQLITE_OK; /* If the cursor does not yet have a statement handle, obtain one now. */ if( pCsr->pStmt==0 ){ Fts5FullTable *pTab = (Fts5FullTable*)(pCsr->base.pVtab); int eStmt = fts5StmtType(pCsr); rc = sqlite3Fts5StorageStmt( pTab->pStorage, eStmt, &pCsr->pStmt, (bErrormsg?&pTab->p.base.zErrMsg:0) ); assert( rc!=SQLITE_OK || pTab->p.base.zErrMsg==0 ); assert( CsrFlagTest(pCsr, FTS5CSR_REQUIRE_CONTENT) ); } if( rc==SQLITE_OK && CsrFlagTest(pCsr, FTS5CSR_REQUIRE_CONTENT) ){ Fts5Table *pTab = (Fts5Table*)(pCsr->base.pVtab); assert( pCsr->pExpr ); sqlite3_reset(pCsr->pStmt); sqlite3_bind_int64(pCsr->pStmt, 1, fts5CursorRowid(pCsr)); pTab->pConfig->bLock++; rc = sqlite3_step(pCsr->pStmt); pTab->pConfig->bLock--; if( rc==SQLITE_ROW ){ rc = SQLITE_OK; CsrFlagClear(pCsr, FTS5CSR_REQUIRE_CONTENT); }else{ rc = sqlite3_reset(pCsr->pStmt); if( rc==SQLITE_OK ){ rc = FTS5_CORRUPT; }else if( pTab->pConfig->pzErrmsg ){ *pTab->pConfig->pzErrmsg = sqlite3_mprintf( "%s", sqlite3_errmsg(pTab->pConfig->db) ); } } } return rc; } static void fts5SetVtabError(Fts5FullTable *p, const char *zFormat, ...){ va_list ap; /* ... printf arguments */ va_start(ap, zFormat); assert( p->p.base.zErrMsg==0 ); p->p.base.zErrMsg = sqlite3_vmprintf(zFormat, ap); va_end(ap); } /* ** This function is called to handle an FTS INSERT command. In other words, ** an INSERT statement of the form: ** ** INSERT INTO fts(fts) VALUES($pCmd) ** INSERT INTO fts(fts, rank) VALUES($pCmd, $pVal) ** ** Argument pVal is the value assigned to column "fts" by the INSERT ** statement. This function returns SQLITE_OK if successful, or an SQLite ** error code if an error occurs. ** ** The commands implemented by this function are documented in the "Special ** INSERT Directives" section of the documentation. It should be updated if ** more commands are added to this function. */ static int fts5SpecialInsert( Fts5FullTable *pTab, /* Fts5 table object */ const char *zCmd, /* Text inserted into table-name column */ sqlite3_value *pVal /* Value inserted into rank column */ ){ Fts5Config *pConfig = pTab->p.pConfig; int rc = SQLITE_OK; int bError = 0; if( 0==sqlite3_stricmp("delete-all", zCmd) ){ if( pConfig->eContent==FTS5_CONTENT_NORMAL ){ fts5SetVtabError(pTab, "'delete-all' may only be used with a " |
︙ | ︙ | |||
1413 1414 1415 1416 1417 1418 1419 | } }else if( 0==sqlite3_stricmp("optimize", zCmd) ){ rc = sqlite3Fts5StorageOptimize(pTab->pStorage); }else if( 0==sqlite3_stricmp("merge", zCmd) ){ int nMerge = sqlite3_value_int(pVal); rc = sqlite3Fts5StorageMerge(pTab->pStorage, nMerge); }else if( 0==sqlite3_stricmp("integrity-check", zCmd) ){ | > | | | | | | 1544 1545 1546 1547 1548 1549 1550 1551 1552 1553 1554 1555 1556 1557 1558 1559 1560 1561 1562 1563 1564 1565 1566 1567 1568 1569 1570 1571 1572 1573 1574 1575 1576 1577 1578 1579 1580 1581 1582 1583 1584 1585 1586 1587 1588 1589 1590 1591 1592 1593 1594 1595 | } }else if( 0==sqlite3_stricmp("optimize", zCmd) ){ rc = sqlite3Fts5StorageOptimize(pTab->pStorage); }else if( 0==sqlite3_stricmp("merge", zCmd) ){ int nMerge = sqlite3_value_int(pVal); rc = sqlite3Fts5StorageMerge(pTab->pStorage, nMerge); }else if( 0==sqlite3_stricmp("integrity-check", zCmd) ){ int iArg = sqlite3_value_int(pVal); rc = sqlite3Fts5StorageIntegrity(pTab->pStorage, iArg); #ifdef SQLITE_DEBUG }else if( 0==sqlite3_stricmp("prefix-index", zCmd) ){ pConfig->bPrefixIndex = sqlite3_value_int(pVal); #endif }else{ rc = sqlite3Fts5IndexLoadConfig(pTab->p.pIndex); if( rc==SQLITE_OK ){ rc = sqlite3Fts5ConfigSetValue(pTab->p.pConfig, zCmd, pVal, &bError); } if( rc==SQLITE_OK ){ if( bError ){ rc = SQLITE_ERROR; }else{ rc = sqlite3Fts5StorageConfigValue(pTab->pStorage, zCmd, pVal, 0); } } } return rc; } static int fts5SpecialDelete( Fts5FullTable *pTab, sqlite3_value **apVal ){ int rc = SQLITE_OK; int eType1 = sqlite3_value_type(apVal[1]); if( eType1==SQLITE_INTEGER ){ sqlite3_int64 iDel = sqlite3_value_int64(apVal[1]); rc = sqlite3Fts5StorageDelete(pTab->pStorage, iDel, &apVal[2]); } return rc; } static void fts5StorageInsert( int *pRc, Fts5FullTable *pTab, sqlite3_value **apVal, i64 *piRowid ){ int rc = *pRc; if( rc==SQLITE_OK ){ rc = sqlite3Fts5StorageContentInsert(pTab->pStorage, apVal, piRowid); } |
︙ | ︙ | |||
1483 1484 1485 1486 1487 1488 1489 | */ static int fts5UpdateMethod( sqlite3_vtab *pVtab, /* Virtual table handle */ int nArg, /* Size of argument array */ sqlite3_value **apVal, /* Array of arguments */ sqlite_int64 *pRowid /* OUT: The affected (or effected) rowid */ ){ | | | | < | | | | | 1615 1616 1617 1618 1619 1620 1621 1622 1623 1624 1625 1626 1627 1628 1629 1630 1631 1632 1633 1634 1635 1636 1637 1638 1639 1640 1641 1642 1643 | */ static int fts5UpdateMethod( sqlite3_vtab *pVtab, /* Virtual table handle */ int nArg, /* Size of argument array */ sqlite3_value **apVal, /* Array of arguments */ sqlite_int64 *pRowid /* OUT: The affected (or effected) rowid */ ){ Fts5FullTable *pTab = (Fts5FullTable*)pVtab; Fts5Config *pConfig = pTab->p.pConfig; int eType0; /* value_type() of apVal[0] */ int rc = SQLITE_OK; /* Return code */ /* A transaction must be open when this is called. */ assert( pTab->ts.eState==1 || pTab->ts.eState==2 ); assert( pVtab->zErrMsg==0 ); assert( nArg==1 || nArg==(2+pConfig->nCol+2) ); assert( sqlite3_value_type(apVal[0])==SQLITE_INTEGER || sqlite3_value_type(apVal[0])==SQLITE_NULL ); assert( pTab->p.pConfig->pzErrmsg==0 ); pTab->p.pConfig->pzErrmsg = &pTab->p.base.zErrMsg; /* Put any active cursors into REQUIRE_SEEK state. */ fts5TripCursors(pTab); eType0 = sqlite3_value_type(apVal[0]); if( eType0==SQLITE_NULL && sqlite3_value_type(apVal[2+pConfig->nCol])!=SQLITE_NULL |
︙ | ︙ | |||
1539 1540 1541 1542 1543 1544 1545 | assert( eType0==SQLITE_INTEGER || eType0==SQLITE_NULL ); assert( nArg!=1 || eType0==SQLITE_INTEGER ); /* Filter out attempts to run UPDATE or DELETE on contentless tables. ** This is not suported. */ if( eType0==SQLITE_INTEGER && fts5IsContentless(pTab) ){ | | | > > > > > > > | | | < < | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | > > | | | | | | | | | 1670 1671 1672 1673 1674 1675 1676 1677 1678 1679 1680 1681 1682 1683 1684 1685 1686 1687 1688 1689 1690 1691 1692 1693 1694 1695 1696 1697 1698 1699 1700 1701 1702 1703 1704 1705 1706 1707 1708 1709 1710 1711 1712 1713 1714 1715 1716 1717 1718 1719 1720 1721 1722 1723 1724 1725 1726 1727 1728 1729 1730 1731 1732 1733 1734 1735 1736 1737 1738 1739 1740 1741 1742 1743 1744 1745 1746 1747 1748 1749 1750 1751 1752 1753 1754 1755 1756 1757 1758 1759 1760 1761 1762 1763 1764 1765 1766 1767 1768 1769 1770 1771 1772 1773 1774 1775 1776 1777 1778 1779 1780 1781 1782 1783 1784 1785 1786 1787 | assert( eType0==SQLITE_INTEGER || eType0==SQLITE_NULL ); assert( nArg!=1 || eType0==SQLITE_INTEGER ); /* Filter out attempts to run UPDATE or DELETE on contentless tables. ** This is not suported. */ if( eType0==SQLITE_INTEGER && fts5IsContentless(pTab) ){ pTab->p.base.zErrMsg = sqlite3_mprintf( "cannot %s contentless fts5 table: %s", (nArg>1 ? "UPDATE" : "DELETE from"), pConfig->zName ); rc = SQLITE_ERROR; } /* DELETE */ else if( nArg==1 ){ i64 iDel = sqlite3_value_int64(apVal[0]); /* Rowid to delete */ rc = sqlite3Fts5StorageDelete(pTab->pStorage, iDel, 0); } /* INSERT or UPDATE */ else{ int eType1 = sqlite3_value_numeric_type(apVal[1]); if( eType1!=SQLITE_INTEGER && eType1!=SQLITE_NULL ){ rc = SQLITE_MISMATCH; } else if( eType0!=SQLITE_INTEGER ){ /* If this is a REPLACE, first remove the current entry (if any) */ if( eConflict==SQLITE_REPLACE && eType1==SQLITE_INTEGER ){ i64 iNew = sqlite3_value_int64(apVal[1]); /* Rowid to delete */ rc = sqlite3Fts5StorageDelete(pTab->pStorage, iNew, 0); } fts5StorageInsert(&rc, pTab, apVal, pRowid); } /* UPDATE */ else{ i64 iOld = sqlite3_value_int64(apVal[0]); /* Old rowid */ i64 iNew = sqlite3_value_int64(apVal[1]); /* New rowid */ if( eType1==SQLITE_INTEGER && iOld!=iNew ){ if( eConflict==SQLITE_REPLACE ){ rc = sqlite3Fts5StorageDelete(pTab->pStorage, iOld, 0); if( rc==SQLITE_OK ){ rc = sqlite3Fts5StorageDelete(pTab->pStorage, iNew, 0); } fts5StorageInsert(&rc, pTab, apVal, pRowid); }else{ rc = sqlite3Fts5StorageContentInsert(pTab->pStorage, apVal, pRowid); if( rc==SQLITE_OK ){ rc = sqlite3Fts5StorageDelete(pTab->pStorage, iOld, 0); } if( rc==SQLITE_OK ){ rc = sqlite3Fts5StorageIndexInsert(pTab->pStorage, apVal,*pRowid); } } }else{ rc = sqlite3Fts5StorageDelete(pTab->pStorage, iOld, 0); fts5StorageInsert(&rc, pTab, apVal, pRowid); } } } } sqlite3Fts5IndexCloseReader(pTab->p.pIndex); pTab->p.pConfig->pzErrmsg = 0; return rc; } /* ** Implementation of xSync() method. */ static int fts5SyncMethod(sqlite3_vtab *pVtab){ int rc; Fts5FullTable *pTab = (Fts5FullTable*)pVtab; fts5CheckTransactionState(pTab, FTS5_SYNC, 0); pTab->p.pConfig->pzErrmsg = &pTab->p.base.zErrMsg; fts5TripCursors(pTab); rc = sqlite3Fts5StorageSync(pTab->pStorage); pTab->p.pConfig->pzErrmsg = 0; return rc; } /* ** Implementation of xBegin() method. */ static int fts5BeginMethod(sqlite3_vtab *pVtab){ fts5CheckTransactionState((Fts5FullTable*)pVtab, FTS5_BEGIN, 0); fts5NewTransaction((Fts5FullTable*)pVtab); return SQLITE_OK; } /* ** Implementation of xCommit() method. This is a no-op. The contents of ** the pending-terms hash-table have already been flushed into the database ** by fts5SyncMethod(). */ static int fts5CommitMethod(sqlite3_vtab *pVtab){ UNUSED_PARAM(pVtab); /* Call below is a no-op for NDEBUG builds */ fts5CheckTransactionState((Fts5FullTable*)pVtab, FTS5_COMMIT, 0); return SQLITE_OK; } /* ** Implementation of xRollback(). Discard the contents of the pending-terms ** hash-table. Any changes made to the database are reverted by SQLite. */ static int fts5RollbackMethod(sqlite3_vtab *pVtab){ int rc; Fts5FullTable *pTab = (Fts5FullTable*)pVtab; fts5CheckTransactionState(pTab, FTS5_ROLLBACK, 0); rc = sqlite3Fts5StorageRollback(pTab->pStorage); return rc; } static int fts5CsrPoslist(Fts5Cursor*, int, const u8**, int*); |
︙ | ︙ | |||
1659 1660 1661 1662 1663 1664 1665 | static int fts5ApiColumnTotalSize( Fts5Context *pCtx, int iCol, sqlite3_int64 *pnToken ){ Fts5Cursor *pCsr = (Fts5Cursor*)pCtx; | | | | 1797 1798 1799 1800 1801 1802 1803 1804 1805 1806 1807 1808 1809 1810 1811 1812 1813 1814 1815 1816 1817 | static int fts5ApiColumnTotalSize( Fts5Context *pCtx, int iCol, sqlite3_int64 *pnToken ){ Fts5Cursor *pCsr = (Fts5Cursor*)pCtx; Fts5FullTable *pTab = (Fts5FullTable*)(pCsr->base.pVtab); return sqlite3Fts5StorageSize(pTab->pStorage, iCol, pnToken); } static int fts5ApiRowCount(Fts5Context *pCtx, i64 *pnRow){ Fts5Cursor *pCsr = (Fts5Cursor*)pCtx; Fts5FullTable *pTab = (Fts5FullTable*)(pCsr->base.pVtab); return sqlite3Fts5StorageRowCount(pTab->pStorage, pnRow); } static int fts5ApiTokenize( Fts5Context *pCtx, const char *pText, int nText, void *pUserData, |
︙ | ︙ | |||
1700 1701 1702 1703 1704 1705 1706 | Fts5Context *pCtx, int iCol, const char **pz, int *pn ){ int rc = SQLITE_OK; Fts5Cursor *pCsr = (Fts5Cursor*)pCtx; | | > > | 1838 1839 1840 1841 1842 1843 1844 1845 1846 1847 1848 1849 1850 1851 1852 1853 1854 | Fts5Context *pCtx, int iCol, const char **pz, int *pn ){ int rc = SQLITE_OK; Fts5Cursor *pCsr = (Fts5Cursor*)pCtx; if( fts5IsContentless((Fts5FullTable*)(pCsr->base.pVtab)) || pCsr->ePlan==FTS5_PLAN_SPECIAL ){ *pz = 0; *pn = 0; }else{ rc = fts5SeekCursor(pCsr, 0); if( rc==SQLITE_OK ){ *pz = (const char*)sqlite3_column_text(pCsr->pStmt, iCol+1); *pn = sqlite3_column_bytes(pCsr->pStmt, iCol+1); |
︙ | ︙ | |||
1769 1770 1771 1772 1773 1774 1775 1776 1777 1778 | ** correctly for the current view. Return SQLITE_OK if successful, or an ** SQLite error code otherwise. */ static int fts5CacheInstArray(Fts5Cursor *pCsr){ int rc = SQLITE_OK; Fts5PoslistReader *aIter; /* One iterator for each phrase */ int nIter; /* Number of iterators/phrases */ nIter = sqlite3Fts5ExprPhraseCount(pCsr->pExpr); if( pCsr->aInstIter==0 ){ | > | | 1909 1910 1911 1912 1913 1914 1915 1916 1917 1918 1919 1920 1921 1922 1923 1924 1925 1926 1927 | ** correctly for the current view. Return SQLITE_OK if successful, or an ** SQLite error code otherwise. */ static int fts5CacheInstArray(Fts5Cursor *pCsr){ int rc = SQLITE_OK; Fts5PoslistReader *aIter; /* One iterator for each phrase */ int nIter; /* Number of iterators/phrases */ int nCol = ((Fts5Table*)pCsr->base.pVtab)->pConfig->nCol; nIter = sqlite3Fts5ExprPhraseCount(pCsr->pExpr); if( pCsr->aInstIter==0 ){ sqlite3_int64 nByte = sizeof(Fts5PoslistReader) * nIter; pCsr->aInstIter = (Fts5PoslistReader*)sqlite3Fts5MallocZero(&rc, nByte); } aIter = pCsr->aInstIter; if( aIter ){ int nInst = 0; /* Number instances seen so far */ int i; |
︙ | ︙ | |||
1806 1807 1808 1809 1810 1811 1812 | iBest = i; } } if( iBest<0 ) break; nInst++; if( nInst>=pCsr->nInstAlloc ){ | | | | > > > > > > | 1947 1948 1949 1950 1951 1952 1953 1954 1955 1956 1957 1958 1959 1960 1961 1962 1963 1964 1965 1966 1967 1968 1969 1970 1971 1972 1973 1974 1975 1976 1977 1978 1979 1980 1981 1982 | iBest = i; } } if( iBest<0 ) break; nInst++; if( nInst>=pCsr->nInstAlloc ){ int nNewSize = pCsr->nInstAlloc ? pCsr->nInstAlloc*2 : 32; aInst = (int*)sqlite3_realloc64( pCsr->aInst, nNewSize*sizeof(int)*3 ); if( aInst ){ pCsr->aInst = aInst; pCsr->nInstAlloc = nNewSize; }else{ nInst--; rc = SQLITE_NOMEM; break; } } aInst = &pCsr->aInst[3 * (nInst-1)]; aInst[0] = iBest; aInst[1] = FTS5_POS2COLUMN(aIter[iBest].iPos); aInst[2] = FTS5_POS2OFFSET(aIter[iBest].iPos); if( aInst[1]<0 || aInst[1]>=nCol ){ rc = FTS5_CORRUPT; break; } sqlite3Fts5PoslistReaderNext(&aIter[iBest]); } } pCsr->nInstCount = nInst; CsrFlagClear(pCsr, FTS5CSR_REQUIRE_INST); } |
︙ | ︙ | |||
1894 1895 1896 1897 1898 1899 1900 | (*pCnt)++; } return SQLITE_OK; } static int fts5ApiColumnSize(Fts5Context *pCtx, int iCol, int *pnToken){ Fts5Cursor *pCsr = (Fts5Cursor*)pCtx; | | | | 2041 2042 2043 2044 2045 2046 2047 2048 2049 2050 2051 2052 2053 2054 2055 2056 | (*pCnt)++; } return SQLITE_OK; } static int fts5ApiColumnSize(Fts5Context *pCtx, int iCol, int *pnToken){ Fts5Cursor *pCsr = (Fts5Cursor*)pCtx; Fts5FullTable *pTab = (Fts5FullTable*)(pCsr->base.pVtab); Fts5Config *pConfig = pTab->p.pConfig; int rc = SQLITE_OK; if( CsrFlagTest(pCsr, FTS5CSR_REQUIRE_DOCSIZE) ){ if( pConfig->bColumnsize ){ i64 iRowid = fts5CursorRowid(pCsr); rc = sqlite3Fts5StorageDocsize(pTab->pStorage, iRowid, pCsr->aColumnSize); }else if( pConfig->zContent==0 ){ |
︙ | ︙ | |||
2032 2033 2034 2035 2036 2037 2038 | Fts5PhraseIter *pIter, int *piCol, int *piOff ){ Fts5Cursor *pCsr = (Fts5Cursor*)pCtx; int n; int rc = fts5CsrPoslist(pCsr, iPhrase, &pIter->a, &n); if( rc==SQLITE_OK ){ | > | | 2179 2180 2181 2182 2183 2184 2185 2186 2187 2188 2189 2190 2191 2192 2193 2194 | Fts5PhraseIter *pIter, int *piCol, int *piOff ){ Fts5Cursor *pCsr = (Fts5Cursor*)pCtx; int n; int rc = fts5CsrPoslist(pCsr, iPhrase, &pIter->a, &n); if( rc==SQLITE_OK ){ assert( pIter->a || n==0 ); pIter->b = (pIter->a ? &pIter->a[n] : 0); *piCol = 0; *piOff = 0; fts5ApiPhraseNext(pCtx, pIter, piCol, piOff); } return rc; } |
︙ | ︙ | |||
2091 2092 2093 2094 2095 2096 2097 | int i1 = (iPhrase==0 ? 0 : pSorter->aIdx[iPhrase-1]); n = pSorter->aIdx[iPhrase] - i1; pIter->a = &pSorter->aPoslist[i1]; }else{ rc = sqlite3Fts5ExprPhraseCollist(pCsr->pExpr, iPhrase, &pIter->a, &n); } if( rc==SQLITE_OK ){ | > | > | | 2239 2240 2241 2242 2243 2244 2245 2246 2247 2248 2249 2250 2251 2252 2253 2254 2255 2256 2257 2258 2259 2260 2261 2262 2263 | int i1 = (iPhrase==0 ? 0 : pSorter->aIdx[iPhrase-1]); n = pSorter->aIdx[iPhrase] - i1; pIter->a = &pSorter->aPoslist[i1]; }else{ rc = sqlite3Fts5ExprPhraseCollist(pCsr->pExpr, iPhrase, &pIter->a, &n); } if( rc==SQLITE_OK ){ assert( pIter->a || n==0 ); pIter->b = (pIter->a ? &pIter->a[n] : 0); *piCol = 0; fts5ApiPhraseNextColumn(pCtx, pIter, piCol); } }else{ int n; rc = fts5CsrPoslist(pCsr, iPhrase, &pIter->a, &n); if( rc==SQLITE_OK ){ assert( pIter->a || n==0 ); pIter->b = (pIter->a ? &pIter->a[n] : 0); if( n<=0 ){ *piCol = -1; }else if( pIter->a[0]==0x01 ){ pIter->a += 1 + fts5GetVarint32(&pIter->a[1], *piCol); }else{ *piCol = 0; } |
︙ | ︙ | |||
2151 2152 2153 2154 2155 2156 2157 | static int fts5ApiQueryPhrase( Fts5Context *pCtx, int iPhrase, void *pUserData, int(*xCallback)(const Fts5ExtensionApi*, Fts5Context*, void*) ){ Fts5Cursor *pCsr = (Fts5Cursor*)pCtx; | | | 2301 2302 2303 2304 2305 2306 2307 2308 2309 2310 2311 2312 2313 2314 2315 | static int fts5ApiQueryPhrase( Fts5Context *pCtx, int iPhrase, void *pUserData, int(*xCallback)(const Fts5ExtensionApi*, Fts5Context*, void*) ){ Fts5Cursor *pCsr = (Fts5Cursor*)pCtx; Fts5FullTable *pTab = (Fts5FullTable*)(pCsr->base.pVtab); int rc; Fts5Cursor *pNew = 0; rc = fts5OpenMethod(pCsr->base.pVtab, (sqlite3_vtab_cursor**)&pNew); if( rc==SQLITE_OK ){ pNew->ePlan = FTS5_PLAN_MATCH; pNew->iFirstRowid = SMALLEST_INT64; |
︙ | ︙ | |||
2217 2218 2219 2220 2221 2222 2223 | i64 iCsrId; assert( argc>=1 ); pAux = (Fts5Auxiliary*)sqlite3_user_data(context); iCsrId = sqlite3_value_int64(argv[0]); pCsr = fts5CursorFromCsrid(pAux->pGlobal, iCsrId); | | | < < < | | < < < > | < | | | 2367 2368 2369 2370 2371 2372 2373 2374 2375 2376 2377 2378 2379 2380 2381 2382 2383 2384 2385 2386 2387 2388 2389 2390 2391 2392 2393 2394 2395 2396 2397 2398 2399 2400 2401 2402 2403 2404 | i64 iCsrId; assert( argc>=1 ); pAux = (Fts5Auxiliary*)sqlite3_user_data(context); iCsrId = sqlite3_value_int64(argv[0]); pCsr = fts5CursorFromCsrid(pAux->pGlobal, iCsrId); if( pCsr==0 || pCsr->ePlan==0 ){ char *zErr = sqlite3_mprintf("no such cursor: %lld", iCsrId); sqlite3_result_error(context, zErr, -1); sqlite3_free(zErr); }else{ fts5ApiInvoke(pAux, pCsr, context, argc-1, &argv[1]); } } /* ** Given cursor id iId, return a pointer to the corresponding Fts5Table ** object. Or NULL If the cursor id does not exist. */ Fts5Table *sqlite3Fts5TableFromCsrid( Fts5Global *pGlobal, /* FTS5 global context for db handle */ i64 iCsrId /* Id of cursor to find */ ){ Fts5Cursor *pCsr; pCsr = fts5CursorFromCsrid(pGlobal, iCsrId); if( pCsr ){ return (Fts5Table*)pCsr->base.pVtab; } return 0; } /* ** Return a "position-list blob" corresponding to the current position of ** cursor pCsr via sqlite3_result_blob(). A position-list blob contains ** the current position-list for each phrase in the query associated with ** cursor pCsr. |
︙ | ︙ | |||
2326 2327 2328 2329 2330 2331 2332 | ** the row that the supplied cursor currently points to. */ static int fts5ColumnMethod( sqlite3_vtab_cursor *pCursor, /* Cursor to retrieve value from */ sqlite3_context *pCtx, /* Context for sqlite3_result_xxx() calls */ int iCol /* Index of column to read value from */ ){ | | | | 2470 2471 2472 2473 2474 2475 2476 2477 2478 2479 2480 2481 2482 2483 2484 2485 | ** the row that the supplied cursor currently points to. */ static int fts5ColumnMethod( sqlite3_vtab_cursor *pCursor, /* Cursor to retrieve value from */ sqlite3_context *pCtx, /* Context for sqlite3_result_xxx() calls */ int iCol /* Index of column to read value from */ ){ Fts5FullTable *pTab = (Fts5FullTable*)(pCursor->pVtab); Fts5Config *pConfig = pTab->p.pConfig; Fts5Cursor *pCsr = (Fts5Cursor*)pCursor; int rc = SQLITE_OK; assert( CsrFlagTest(pCsr, FTS5CSR_EOF)==0 ); if( pCsr->ePlan==FTS5_PLAN_SPECIAL ){ if( iCol==pConfig->nCol ){ |
︙ | ︙ | |||
2359 2360 2361 2362 2363 2364 2365 2366 2367 2368 2369 2370 2371 2372 2373 2374 2375 2376 2377 2378 2379 2380 2381 2382 2383 2384 2385 | || pCsr->ePlan==FTS5_PLAN_SORTED_MATCH ){ if( pCsr->pRank || SQLITE_OK==(rc = fts5FindRankFunction(pCsr)) ){ fts5ApiInvoke(pCsr->pRank, pCsr, pCtx, pCsr->nRankArg, pCsr->apRankArg); } } }else if( !fts5IsContentless(pTab) ){ rc = fts5SeekCursor(pCsr, 1); if( rc==SQLITE_OK ){ sqlite3_result_value(pCtx, sqlite3_column_value(pCsr->pStmt, iCol+1)); } } return rc; } /* ** This routine implements the xFindFunction method for the FTS3 ** virtual table. */ static int fts5FindFunctionMethod( sqlite3_vtab *pVtab, /* Virtual table handle */ int nUnused, /* Number of SQL function arguments */ const char *zName, /* Name of SQL function */ void (**pxFunc)(sqlite3_context*,int,sqlite3_value**), /* OUT: Result */ void **ppArg /* OUT: User data for *pxFunc */ ){ | > > | | 2503 2504 2505 2506 2507 2508 2509 2510 2511 2512 2513 2514 2515 2516 2517 2518 2519 2520 2521 2522 2523 2524 2525 2526 2527 2528 2529 2530 2531 2532 2533 2534 2535 2536 2537 2538 2539 | || pCsr->ePlan==FTS5_PLAN_SORTED_MATCH ){ if( pCsr->pRank || SQLITE_OK==(rc = fts5FindRankFunction(pCsr)) ){ fts5ApiInvoke(pCsr->pRank, pCsr, pCtx, pCsr->nRankArg, pCsr->apRankArg); } } }else if( !fts5IsContentless(pTab) ){ pConfig->pzErrmsg = &pTab->p.base.zErrMsg; rc = fts5SeekCursor(pCsr, 1); if( rc==SQLITE_OK ){ sqlite3_result_value(pCtx, sqlite3_column_value(pCsr->pStmt, iCol+1)); } pConfig->pzErrmsg = 0; } return rc; } /* ** This routine implements the xFindFunction method for the FTS3 ** virtual table. */ static int fts5FindFunctionMethod( sqlite3_vtab *pVtab, /* Virtual table handle */ int nUnused, /* Number of SQL function arguments */ const char *zName, /* Name of SQL function */ void (**pxFunc)(sqlite3_context*,int,sqlite3_value**), /* OUT: Result */ void **ppArg /* OUT: User data for *pxFunc */ ){ Fts5FullTable *pTab = (Fts5FullTable*)pVtab; Fts5Auxiliary *pAux; UNUSED_PARAM(nUnused); pAux = fts5FindAuxiliary(pTab, zName); if( pAux ){ *pxFunc = fts5ApiCallback; *ppArg = (void*)pAux; |
︙ | ︙ | |||
2401 2402 2403 2404 2405 2406 2407 | /* ** Implementation of FTS5 xRename method. Rename an fts5 table. */ static int fts5RenameMethod( sqlite3_vtab *pVtab, /* Virtual table handle */ const char *zName /* New name of table */ ){ | | > > > > > < | < | < | < | | | | | | | | 2547 2548 2549 2550 2551 2552 2553 2554 2555 2556 2557 2558 2559 2560 2561 2562 2563 2564 2565 2566 2567 2568 2569 2570 2571 2572 2573 2574 2575 2576 2577 2578 2579 2580 2581 2582 2583 2584 2585 2586 2587 2588 2589 2590 2591 2592 2593 2594 2595 2596 2597 2598 2599 2600 2601 2602 2603 2604 2605 2606 2607 2608 2609 2610 2611 2612 2613 2614 2615 2616 2617 2618 2619 2620 2621 2622 2623 2624 2625 2626 | /* ** Implementation of FTS5 xRename method. Rename an fts5 table. */ static int fts5RenameMethod( sqlite3_vtab *pVtab, /* Virtual table handle */ const char *zName /* New name of table */ ){ Fts5FullTable *pTab = (Fts5FullTable*)pVtab; return sqlite3Fts5StorageRename(pTab->pStorage, zName); } int sqlite3Fts5FlushToDisk(Fts5Table *pTab){ fts5TripCursors((Fts5FullTable*)pTab); return sqlite3Fts5StorageSync(((Fts5FullTable*)pTab)->pStorage); } /* ** The xSavepoint() method. ** ** Flush the contents of the pending-terms table to disk. */ static int fts5SavepointMethod(sqlite3_vtab *pVtab, int iSavepoint){ UNUSED_PARAM(iSavepoint); /* Call below is a no-op for NDEBUG builds */ fts5CheckTransactionState((Fts5FullTable*)pVtab, FTS5_SAVEPOINT, iSavepoint); return sqlite3Fts5FlushToDisk((Fts5Table*)pVtab); } /* ** The xRelease() method. ** ** This is a no-op. */ static int fts5ReleaseMethod(sqlite3_vtab *pVtab, int iSavepoint){ UNUSED_PARAM(iSavepoint); /* Call below is a no-op for NDEBUG builds */ fts5CheckTransactionState((Fts5FullTable*)pVtab, FTS5_RELEASE, iSavepoint); return sqlite3Fts5FlushToDisk((Fts5Table*)pVtab); } /* ** The xRollbackTo() method. ** ** Discard the contents of the pending terms table. */ static int fts5RollbackToMethod(sqlite3_vtab *pVtab, int iSavepoint){ Fts5FullTable *pTab = (Fts5FullTable*)pVtab; UNUSED_PARAM(iSavepoint); /* Call below is a no-op for NDEBUG builds */ fts5CheckTransactionState(pTab, FTS5_ROLLBACKTO, iSavepoint); fts5TripCursors(pTab); return sqlite3Fts5StorageRollback(pTab->pStorage); } /* ** Register a new auxiliary function with global context pGlobal. */ static int fts5CreateAux( fts5_api *pApi, /* Global context (one per db handle) */ const char *zName, /* Name of new function */ void *pUserData, /* User data for aux. function */ fts5_extension_function xFunc, /* Aux. function implementation */ void(*xDestroy)(void*) /* Destructor for pUserData */ ){ Fts5Global *pGlobal = (Fts5Global*)pApi; int rc = sqlite3_overload_function(pGlobal->db, zName, -1); if( rc==SQLITE_OK ){ Fts5Auxiliary *pAux; sqlite3_int64 nName; /* Size of zName in bytes, including \0 */ sqlite3_int64 nByte; /* Bytes of space to allocate */ nName = strlen(zName) + 1; nByte = sizeof(Fts5Auxiliary) + nName; pAux = (Fts5Auxiliary*)sqlite3_malloc64(nByte); if( pAux ){ memset(pAux, 0, (size_t)nByte); pAux->zFunc = (char*)&pAux[1]; memcpy(pAux->zFunc, zName, nName); pAux->pGlobal = pGlobal; pAux->pUserData = pUserData; pAux->xFunc = xFunc; pAux->xDestroy = xDestroy; pAux->pNext = pGlobal->pAux; |
︙ | ︙ | |||
2495 2496 2497 2498 2499 2500 2501 | const char *zName, /* Name of new function */ void *pUserData, /* User data for aux. function */ fts5_tokenizer *pTokenizer, /* Tokenizer implementation */ void(*xDestroy)(void*) /* Destructor for pUserData */ ){ Fts5Global *pGlobal = (Fts5Global*)pApi; Fts5TokenizerModule *pNew; | | | | | | | 2642 2643 2644 2645 2646 2647 2648 2649 2650 2651 2652 2653 2654 2655 2656 2657 2658 2659 2660 2661 2662 2663 2664 | const char *zName, /* Name of new function */ void *pUserData, /* User data for aux. function */ fts5_tokenizer *pTokenizer, /* Tokenizer implementation */ void(*xDestroy)(void*) /* Destructor for pUserData */ ){ Fts5Global *pGlobal = (Fts5Global*)pApi; Fts5TokenizerModule *pNew; sqlite3_int64 nName; /* Size of zName and its \0 terminator */ sqlite3_int64 nByte; /* Bytes of space to allocate */ int rc = SQLITE_OK; nName = strlen(zName) + 1; nByte = sizeof(Fts5TokenizerModule) + nName; pNew = (Fts5TokenizerModule*)sqlite3_malloc64(nByte); if( pNew ){ memset(pNew, 0, (size_t)nByte); pNew->zName = (char*)&pNew[1]; memcpy(pNew->zName, zName, nName); pNew->pUserData = pUserData; pNew->x = *pTokenizer; pNew->xDestroy = xDestroy; pNew->pNext = pGlobal->pTok; pGlobal->pTok = pNew; |
︙ | ︙ | |||
2567 2568 2569 2570 2571 2572 2573 | return rc; } int sqlite3Fts5GetTokenizer( Fts5Global *pGlobal, const char **azArg, int nArg, | | < | > > | | | > > > > | | | 2714 2715 2716 2717 2718 2719 2720 2721 2722 2723 2724 2725 2726 2727 2728 2729 2730 2731 2732 2733 2734 2735 2736 2737 2738 2739 2740 2741 2742 2743 2744 2745 2746 2747 2748 2749 2750 2751 2752 2753 2754 2755 | return rc; } int sqlite3Fts5GetTokenizer( Fts5Global *pGlobal, const char **azArg, int nArg, Fts5Config *pConfig, char **pzErr ){ Fts5TokenizerModule *pMod; int rc = SQLITE_OK; pMod = fts5LocateTokenizer(pGlobal, nArg==0 ? 0 : azArg[0]); if( pMod==0 ){ assert( nArg>0 ); rc = SQLITE_ERROR; *pzErr = sqlite3_mprintf("no such tokenizer: %s", azArg[0]); }else{ rc = pMod->x.xCreate( pMod->pUserData, (azArg?&azArg[1]:0), (nArg?nArg-1:0), &pConfig->pTok ); pConfig->pTokApi = &pMod->x; if( rc!=SQLITE_OK ){ if( pzErr ) *pzErr = sqlite3_mprintf("error in tokenizer constructor"); }else{ pConfig->ePattern = sqlite3Fts5TokenizerPattern( pMod->x.xCreate, pConfig->pTok ); } } if( rc!=SQLITE_OK ){ pConfig->pTokApi = 0; pConfig->pTok = 0; } return rc; } static void fts5ModuleDestroy(void *pCtx){ Fts5TokenizerModule *pTok, *pNextTok; |
︙ | ︙ | |||
2711 2712 2713 2714 2715 2716 2717 | if( rc==SQLITE_OK ){ rc = sqlite3_create_function( db, "fts5", 1, SQLITE_UTF8, p, fts5Fts5Func, 0, 0 ); } if( rc==SQLITE_OK ){ rc = sqlite3_create_function( | | > > | 2863 2864 2865 2866 2867 2868 2869 2870 2871 2872 2873 2874 2875 2876 2877 2878 2879 | if( rc==SQLITE_OK ){ rc = sqlite3_create_function( db, "fts5", 1, SQLITE_UTF8, p, fts5Fts5Func, 0, 0 ); } if( rc==SQLITE_OK ){ rc = sqlite3_create_function( db, "fts5_source_id", 0, SQLITE_UTF8|SQLITE_DETERMINISTIC|SQLITE_INNOCUOUS, p, fts5SourceIdFunc, 0, 0 ); } } /* If SQLITE_FTS5_ENABLE_TEST_MI is defined, assume that the file ** fts5_test_mi.c is compiled and linked into the executable. And call ** its entry point to enable the matchinfo() demo. */ |
︙ | ︙ |
Changes to ext/fts5/fts5_storage.c.
︙ | ︙ | |||
111 112 113 114 115 116 117 | case FTS5_STMT_INSERT_CONTENT: case FTS5_STMT_REPLACE_CONTENT: { int nCol = pC->nCol + 1; char *zBind; int i; | | > > > | < > | 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 | case FTS5_STMT_INSERT_CONTENT: case FTS5_STMT_REPLACE_CONTENT: { int nCol = pC->nCol + 1; char *zBind; int i; zBind = sqlite3_malloc64(1 + nCol*2); if( zBind ){ for(i=0; i<nCol; i++){ zBind[i*2] = '?'; zBind[i*2 + 1] = ','; } zBind[i*2-1] = '\0'; zSql = sqlite3_mprintf(azStmt[eStmt], pC->zDb, pC->zName, zBind); sqlite3_free(zBind); } break; } default: zSql = sqlite3_mprintf(azStmt[eStmt], pC->zDb, pC->zName); break; } if( zSql==0 ){ rc = SQLITE_NOMEM; }else{ int f = SQLITE_PREPARE_PERSISTENT; if( eStmt>FTS5_STMT_LOOKUP ) f |= SQLITE_PREPARE_NO_VTAB; p->pConfig->bLock++; rc = sqlite3_prepare_v3(pC->db, zSql, -1, f, &p->aStmt[eStmt], 0); p->pConfig->bLock--; sqlite3_free(zSql); if( rc!=SQLITE_OK && pzErrMsg ){ *pzErrMsg = sqlite3_mprintf("%s", sqlite3_errmsg(pC->db)); } } } |
︙ | ︙ | |||
277 278 279 280 281 282 283 | Fts5Index *pIndex, int bCreate, Fts5Storage **pp, char **pzErr /* OUT: Error message */ ){ int rc = SQLITE_OK; Fts5Storage *p; /* New object */ | | | | | | 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 | Fts5Index *pIndex, int bCreate, Fts5Storage **pp, char **pzErr /* OUT: Error message */ ){ int rc = SQLITE_OK; Fts5Storage *p; /* New object */ sqlite3_int64 nByte; /* Bytes of space to allocate */ nByte = sizeof(Fts5Storage) /* Fts5Storage object */ + pConfig->nCol * sizeof(i64); /* Fts5Storage.aTotalSize[] */ *pp = p = (Fts5Storage*)sqlite3_malloc64(nByte); if( !p ) return SQLITE_NOMEM; memset(p, 0, (size_t)nByte); p->aTotalSize = (i64*)&p[1]; p->pConfig = pConfig; p->pIndex = pIndex; if( bCreate ){ if( pConfig->eContent==FTS5_CONTENT_NORMAL ){ int nDefn = 32 + pConfig->nCol*10; char *zDefn = sqlite3_malloc64(32 + (sqlite3_int64)pConfig->nCol * 10); if( zDefn==0 ){ rc = SQLITE_NOMEM; }else{ int i; int iOff; sqlite3_snprintf(nDefn, zDefn, "id INTEGER PRIMARY KEY"); iOff = (int)strlen(zDefn); |
︙ | ︙ | |||
410 411 412 413 414 415 416 417 418 419 | ctx.pStorage = p; ctx.iCol = -1; rc = sqlite3Fts5IndexBeginWrite(p->pIndex, 1, iDel); for(iCol=1; rc==SQLITE_OK && iCol<=pConfig->nCol; iCol++){ if( pConfig->abUnindexed[iCol-1]==0 ){ const char *zText; int nText; if( pSeek ){ zText = (const char*)sqlite3_column_text(pSeek, iCol); nText = sqlite3_column_bytes(pSeek, iCol); | > > | > > > > | | > > > > | > | 413 414 415 416 417 418 419 420 421 422 423 424 425 426 427 428 429 430 431 432 433 434 435 436 437 438 439 440 441 442 443 444 445 446 447 448 449 450 451 452 | ctx.pStorage = p; ctx.iCol = -1; rc = sqlite3Fts5IndexBeginWrite(p->pIndex, 1, iDel); for(iCol=1; rc==SQLITE_OK && iCol<=pConfig->nCol; iCol++){ if( pConfig->abUnindexed[iCol-1]==0 ){ const char *zText; int nText; assert( pSeek==0 || apVal==0 ); assert( pSeek!=0 || apVal!=0 ); if( pSeek ){ zText = (const char*)sqlite3_column_text(pSeek, iCol); nText = sqlite3_column_bytes(pSeek, iCol); }else if( ALWAYS(apVal) ){ zText = (const char*)sqlite3_value_text(apVal[iCol-1]); nText = sqlite3_value_bytes(apVal[iCol-1]); }else{ continue; } ctx.szCol = 0; rc = sqlite3Fts5Tokenize(pConfig, FTS5_TOKENIZE_DOCUMENT, zText, nText, (void*)&ctx, fts5StorageInsertCallback ); p->aTotalSize[iCol-1] -= (i64)ctx.szCol; if( p->aTotalSize[iCol-1]<0 ){ rc = FTS5_CORRUPT; } } } if( rc==SQLITE_OK && p->nTotalRow<1 ){ rc = FTS5_CORRUPT; }else{ p->nTotalRow--; } rc2 = sqlite3_reset(pSeek); if( rc==SQLITE_OK ) rc = rc2; return rc; } |
︙ | ︙ | |||
552 553 554 555 556 557 558 559 560 561 562 563 564 565 | /* ** Delete all entries in the FTS5 index. */ int sqlite3Fts5StorageDeleteAll(Fts5Storage *p){ Fts5Config *pConfig = p->pConfig; int rc; /* Delete the contents of the %_data and %_docsize tables. */ rc = fts5ExecPrintf(pConfig->db, 0, "DELETE FROM %Q.'%q_data';" "DELETE FROM %Q.'%q_idx';", pConfig->zDb, pConfig->zName, pConfig->zDb, pConfig->zName | > > | 566 567 568 569 570 571 572 573 574 575 576 577 578 579 580 581 | /* ** Delete all entries in the FTS5 index. */ int sqlite3Fts5StorageDeleteAll(Fts5Storage *p){ Fts5Config *pConfig = p->pConfig; int rc; p->bTotalsValid = 0; /* Delete the contents of the %_data and %_docsize tables. */ rc = fts5ExecPrintf(pConfig->db, 0, "DELETE FROM %Q.'%q_data';" "DELETE FROM %Q.'%q_idx';", pConfig->zDb, pConfig->zName, pConfig->zDb, pConfig->zName |
︙ | ︙ | |||
583 584 585 586 587 588 589 | } int sqlite3Fts5StorageRebuild(Fts5Storage *p){ Fts5Buffer buf = {0,0,0}; Fts5Config *pConfig = p->pConfig; sqlite3_stmt *pScan = 0; Fts5InsertCtx ctx; | | > > < < > > > | 599 600 601 602 603 604 605 606 607 608 609 610 611 612 613 614 615 616 617 618 619 620 621 622 623 624 625 626 627 628 629 630 631 632 633 634 635 636 637 638 639 640 641 642 643 644 645 646 647 648 649 650 651 652 653 654 | } int sqlite3Fts5StorageRebuild(Fts5Storage *p){ Fts5Buffer buf = {0,0,0}; Fts5Config *pConfig = p->pConfig; sqlite3_stmt *pScan = 0; Fts5InsertCtx ctx; int rc, rc2; memset(&ctx, 0, sizeof(Fts5InsertCtx)); ctx.pStorage = p; rc = sqlite3Fts5StorageDeleteAll(p); if( rc==SQLITE_OK ){ rc = fts5StorageLoadTotals(p, 1); } if( rc==SQLITE_OK ){ rc = fts5StorageGetStmt(p, FTS5_STMT_SCAN, &pScan, 0); } while( rc==SQLITE_OK && SQLITE_ROW==sqlite3_step(pScan) ){ i64 iRowid = sqlite3_column_int64(pScan, 0); sqlite3Fts5BufferZero(&buf); rc = sqlite3Fts5IndexBeginWrite(p->pIndex, 0, iRowid); for(ctx.iCol=0; rc==SQLITE_OK && ctx.iCol<pConfig->nCol; ctx.iCol++){ ctx.szCol = 0; if( pConfig->abUnindexed[ctx.iCol]==0 ){ const char *zText = (const char*)sqlite3_column_text(pScan, ctx.iCol+1); int nText = sqlite3_column_bytes(pScan, ctx.iCol+1); rc = sqlite3Fts5Tokenize(pConfig, FTS5_TOKENIZE_DOCUMENT, zText, nText, (void*)&ctx, fts5StorageInsertCallback ); } sqlite3Fts5BufferAppendVarint(&rc, &buf, ctx.szCol); p->aTotalSize[ctx.iCol] += (i64)ctx.szCol; } p->nTotalRow++; if( rc==SQLITE_OK ){ rc = fts5StorageInsertDocsize(p, iRowid, &buf); } } sqlite3_free(buf.p); rc2 = sqlite3_reset(pScan); if( rc==SQLITE_OK ) rc = rc2; /* Write the averages record */ if( rc==SQLITE_OK ){ rc = fts5StorageSaveTotals(p); } return rc; } |
︙ | ︙ | |||
727 728 729 730 731 732 733 734 735 | if( rc==SQLITE_OK ){ rc = sqlite3Fts5IndexBeginWrite(p->pIndex, 0, iRowid); } for(ctx.iCol=0; rc==SQLITE_OK && ctx.iCol<pConfig->nCol; ctx.iCol++){ ctx.szCol = 0; if( pConfig->abUnindexed[ctx.iCol]==0 ){ rc = sqlite3Fts5Tokenize(pConfig, FTS5_TOKENIZE_DOCUMENT, | > > < < > | 746 747 748 749 750 751 752 753 754 755 756 757 758 759 760 761 762 763 764 | if( rc==SQLITE_OK ){ rc = sqlite3Fts5IndexBeginWrite(p->pIndex, 0, iRowid); } for(ctx.iCol=0; rc==SQLITE_OK && ctx.iCol<pConfig->nCol; ctx.iCol++){ ctx.szCol = 0; if( pConfig->abUnindexed[ctx.iCol]==0 ){ const char *zText = (const char*)sqlite3_value_text(apVal[ctx.iCol+2]); int nText = sqlite3_value_bytes(apVal[ctx.iCol+2]); rc = sqlite3Fts5Tokenize(pConfig, FTS5_TOKENIZE_DOCUMENT, zText, nText, (void*)&ctx, fts5StorageInsertCallback ); } sqlite3Fts5BufferAppendVarint(&rc, &buf, ctx.szCol); p->aTotalSize[ctx.iCol] += (i64)ctx.szCol; } |
︙ | ︙ | |||
861 862 863 864 865 866 867 | /* ** Check that the contents of the FTS index match that of the %_content ** table. Return SQLITE_OK if they do, or SQLITE_CORRUPT if not. Return ** some other SQLite error code if an error occurs while attempting to ** determine this. */ | | | > | > > > > | | | | | | | | | | | | | | | | | | | | | | | > > | | | < | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | > | | 881 882 883 884 885 886 887 888 889 890 891 892 893 894 895 896 897 898 899 900 901 902 903 904 905 906 907 908 909 910 911 912 913 914 915 916 917 918 919 920 921 922 923 924 925 926 927 928 929 930 931 932 933 934 935 936 937 938 939 940 941 942 943 944 945 946 947 948 949 950 951 952 953 954 955 956 957 958 959 960 961 962 963 964 965 966 967 968 969 970 971 972 973 974 975 976 977 978 979 980 981 982 983 984 985 986 987 988 989 990 991 992 | /* ** Check that the contents of the FTS index match that of the %_content ** table. Return SQLITE_OK if they do, or SQLITE_CORRUPT if not. Return ** some other SQLite error code if an error occurs while attempting to ** determine this. */ int sqlite3Fts5StorageIntegrity(Fts5Storage *p, int iArg){ Fts5Config *pConfig = p->pConfig; int rc = SQLITE_OK; /* Return code */ int *aColSize; /* Array of size pConfig->nCol */ i64 *aTotalSize; /* Array of size pConfig->nCol */ Fts5IntegrityCtx ctx; sqlite3_stmt *pScan; int bUseCksum; memset(&ctx, 0, sizeof(Fts5IntegrityCtx)); ctx.pConfig = p->pConfig; aTotalSize = (i64*)sqlite3_malloc64(pConfig->nCol*(sizeof(int)+sizeof(i64))); if( !aTotalSize ) return SQLITE_NOMEM; aColSize = (int*)&aTotalSize[pConfig->nCol]; memset(aTotalSize, 0, sizeof(i64) * pConfig->nCol); bUseCksum = (pConfig->eContent==FTS5_CONTENT_NORMAL || (pConfig->eContent==FTS5_CONTENT_EXTERNAL && iArg) ); if( bUseCksum ){ /* Generate the expected index checksum based on the contents of the ** %_content table. This block stores the checksum in ctx.cksum. */ rc = fts5StorageGetStmt(p, FTS5_STMT_SCAN, &pScan, 0); if( rc==SQLITE_OK ){ int rc2; while( SQLITE_ROW==sqlite3_step(pScan) ){ int i; ctx.iRowid = sqlite3_column_int64(pScan, 0); ctx.szCol = 0; if( pConfig->bColumnsize ){ rc = sqlite3Fts5StorageDocsize(p, ctx.iRowid, aColSize); } if( rc==SQLITE_OK && pConfig->eDetail==FTS5_DETAIL_NONE ){ rc = sqlite3Fts5TermsetNew(&ctx.pTermset); } for(i=0; rc==SQLITE_OK && i<pConfig->nCol; i++){ if( pConfig->abUnindexed[i] ) continue; ctx.iCol = i; ctx.szCol = 0; if( pConfig->eDetail==FTS5_DETAIL_COLUMNS ){ rc = sqlite3Fts5TermsetNew(&ctx.pTermset); } if( rc==SQLITE_OK ){ const char *zText = (const char*)sqlite3_column_text(pScan, i+1); int nText = sqlite3_column_bytes(pScan, i+1); rc = sqlite3Fts5Tokenize(pConfig, FTS5_TOKENIZE_DOCUMENT, zText, nText, (void*)&ctx, fts5StorageIntegrityCallback ); } if( rc==SQLITE_OK && pConfig->bColumnsize && ctx.szCol!=aColSize[i] ){ rc = FTS5_CORRUPT; } aTotalSize[i] += ctx.szCol; if( pConfig->eDetail==FTS5_DETAIL_COLUMNS ){ sqlite3Fts5TermsetFree(ctx.pTermset); ctx.pTermset = 0; } } sqlite3Fts5TermsetFree(ctx.pTermset); ctx.pTermset = 0; if( rc!=SQLITE_OK ) break; } rc2 = sqlite3_reset(pScan); if( rc==SQLITE_OK ) rc = rc2; } /* Test that the "totals" (sometimes called "averages") record looks Ok */ if( rc==SQLITE_OK ){ int i; rc = fts5StorageLoadTotals(p, 0); for(i=0; rc==SQLITE_OK && i<pConfig->nCol; i++){ if( p->aTotalSize[i]!=aTotalSize[i] ) rc = FTS5_CORRUPT; } } /* Check that the %_docsize and %_content tables contain the expected ** number of rows. */ if( rc==SQLITE_OK && pConfig->eContent==FTS5_CONTENT_NORMAL ){ i64 nRow = 0; rc = fts5StorageCount(p, "content", &nRow); if( rc==SQLITE_OK && nRow!=p->nTotalRow ) rc = FTS5_CORRUPT; } if( rc==SQLITE_OK && pConfig->bColumnsize ){ i64 nRow = 0; rc = fts5StorageCount(p, "docsize", &nRow); if( rc==SQLITE_OK && nRow!=p->nTotalRow ) rc = FTS5_CORRUPT; } } /* Pass the expected checksum down to the FTS index module. It will ** verify, amongst other things, that it matches the checksum generated by ** inspecting the index itself. */ if( rc==SQLITE_OK ){ rc = sqlite3Fts5IndexIntegrityCheck(p->pIndex, ctx.cksum, bUseCksum); } sqlite3_free(aTotalSize); return rc; } /* |
︙ | ︙ | |||
1031 1032 1033 1034 1035 1036 1037 | int sqlite3Fts5StorageDocsize(Fts5Storage *p, i64 iRowid, int *aCol){ int nCol = p->pConfig->nCol; /* Number of user columns in table */ sqlite3_stmt *pLookup = 0; /* Statement to query %_docsize */ int rc; /* Return Code */ assert( p->pConfig->bColumnsize ); rc = fts5StorageGetStmt(p, FTS5_STMT_LOOKUP_DOCSIZE, &pLookup, 0); | | > > > | 1058 1059 1060 1061 1062 1063 1064 1065 1066 1067 1068 1069 1070 1071 1072 1073 1074 1075 1076 1077 1078 1079 1080 1081 1082 1083 1084 1085 1086 1087 1088 | int sqlite3Fts5StorageDocsize(Fts5Storage *p, i64 iRowid, int *aCol){ int nCol = p->pConfig->nCol; /* Number of user columns in table */ sqlite3_stmt *pLookup = 0; /* Statement to query %_docsize */ int rc; /* Return Code */ assert( p->pConfig->bColumnsize ); rc = fts5StorageGetStmt(p, FTS5_STMT_LOOKUP_DOCSIZE, &pLookup, 0); if( pLookup ){ int bCorrupt = 1; assert( rc==SQLITE_OK ); sqlite3_bind_int64(pLookup, 1, iRowid); if( SQLITE_ROW==sqlite3_step(pLookup) ){ const u8 *aBlob = sqlite3_column_blob(pLookup, 0); int nBlob = sqlite3_column_bytes(pLookup, 0); if( 0==fts5StorageDecodeSizeArray(aCol, nCol, aBlob, nBlob) ){ bCorrupt = 0; } } rc = sqlite3_reset(pLookup); if( bCorrupt && rc==SQLITE_OK ){ rc = FTS5_CORRUPT; } }else{ assert( rc!=SQLITE_OK ); } return rc; } int sqlite3Fts5StorageSize(Fts5Storage *p, int iCol, i64 *pnToken){ int rc = fts5StorageLoadTotals(p, 0); |
︙ | ︙ | |||
1071 1072 1073 1074 1075 1076 1077 1078 1079 1080 1081 1082 1083 1084 1085 | } return rc; } int sqlite3Fts5StorageRowCount(Fts5Storage *p, i64 *pnRow){ int rc = fts5StorageLoadTotals(p, 0); if( rc==SQLITE_OK ){ *pnRow = p->nTotalRow; } return rc; } /* ** Flush any data currently held in-memory to disk. */ | > > > > > > | 1101 1102 1103 1104 1105 1106 1107 1108 1109 1110 1111 1112 1113 1114 1115 1116 1117 1118 1119 1120 1121 | } return rc; } int sqlite3Fts5StorageRowCount(Fts5Storage *p, i64 *pnRow){ int rc = fts5StorageLoadTotals(p, 0); if( rc==SQLITE_OK ){ /* nTotalRow being zero does not necessarily indicate a corrupt ** database - it might be that the FTS5 table really does contain zero ** rows. However this function is only called from the xRowCount() API, ** and there is no way for that API to be invoked if the table contains ** no rows. Hence the FTS5_CORRUPT return. */ *pnRow = p->nTotalRow; if( p->nTotalRow<=0 ) rc = FTS5_CORRUPT; } return rc; } /* ** Flush any data currently held in-memory to disk. */ |
︙ | ︙ |
Changes to ext/fts5/fts5_tcl.c.
︙ | ︙ | |||
25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 | #ifdef SQLITE_ENABLE_FTS5 #include "fts5.h" #include <string.h> #include <assert.h> extern int sqlite3_fts5_may_be_corrupt; extern int sqlite3Fts5TestRegisterMatchinfo(sqlite3*); extern int sqlite3Fts5TestRegisterTok(sqlite3*, fts5_api*); /************************************************************************* ** This is a copy of the first part of the SqliteDb structure in ** tclsqlite.c. We need it here so that the get_sqlite_pointer routine ** can extract the sqlite3* pointer from an existing Tcl SQLite | > > | 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 | #ifdef SQLITE_ENABLE_FTS5 #include "fts5.h" #include <string.h> #include <assert.h> #ifdef SQLITE_DEBUG extern int sqlite3_fts5_may_be_corrupt; #endif extern int sqlite3Fts5TestRegisterMatchinfo(sqlite3*); extern int sqlite3Fts5TestRegisterTok(sqlite3*, fts5_api*); /************************************************************************* ** This is a copy of the first part of the SqliteDb structure in ** tclsqlite.c. We need it here so that the get_sqlite_pointer routine ** can extract the sqlite3* pointer from an existing Tcl SQLite |
︙ | ︙ | |||
1007 1008 1009 1010 1011 1012 1013 1014 1015 1016 1017 1018 1019 1020 1021 1022 1023 1024 1025 1026 1027 1028 1029 1030 1031 1032 1033 | */ static int SQLITE_TCLAPI f5tMayBeCorrupt( void * clientData, Tcl_Interp *interp, int objc, Tcl_Obj *CONST objv[] ){ int bOld = sqlite3_fts5_may_be_corrupt; if( objc!=2 && objc!=1 ){ Tcl_WrongNumArgs(interp, 1, objv, "?BOOLEAN?"); return TCL_ERROR; } if( objc==2 ){ int bNew; if( Tcl_GetBooleanFromObj(interp, objv[1], &bNew) ) return TCL_ERROR; sqlite3_fts5_may_be_corrupt = bNew; } Tcl_SetObjResult(interp, Tcl_NewIntObj(bOld)); return TCL_OK; } static unsigned int f5t_fts5HashKey(int nSlot, const char *p, int n){ int i; unsigned int h = 13; | > > | 1009 1010 1011 1012 1013 1014 1015 1016 1017 1018 1019 1020 1021 1022 1023 1024 1025 1026 1027 1028 1029 1030 1031 1032 1033 1034 1035 1036 1037 | */ static int SQLITE_TCLAPI f5tMayBeCorrupt( void * clientData, Tcl_Interp *interp, int objc, Tcl_Obj *CONST objv[] ){ #ifdef SQLITE_DEBUG int bOld = sqlite3_fts5_may_be_corrupt; if( objc!=2 && objc!=1 ){ Tcl_WrongNumArgs(interp, 1, objv, "?BOOLEAN?"); return TCL_ERROR; } if( objc==2 ){ int bNew; if( Tcl_GetBooleanFromObj(interp, objv[1], &bNew) ) return TCL_ERROR; sqlite3_fts5_may_be_corrupt = bNew; } Tcl_SetObjResult(interp, Tcl_NewIntObj(bOld)); #endif return TCL_OK; } static unsigned int f5t_fts5HashKey(int nSlot, const char *p, int n){ int i; unsigned int h = 13; |
︙ | ︙ |
Changes to ext/fts5/fts5_test_mi.c.
︙ | ︙ | |||
305 306 307 308 309 310 311 | const char *zArg /* Matchinfo flag string */ ){ Fts5MatchinfoCtx *p; int nCol; int nPhrase; int i; int nInt; | | | | 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340 | const char *zArg /* Matchinfo flag string */ ){ Fts5MatchinfoCtx *p; int nCol; int nPhrase; int i; int nInt; sqlite3_int64 nByte; int rc; nCol = pApi->xColumnCount(pFts); nPhrase = pApi->xPhraseCount(pFts); nInt = 0; for(i=0; zArg[i]; i++){ int n = fts5MatchinfoFlagsize(nCol, nPhrase, zArg[i]); if( n<0 ){ char *zErr = sqlite3_mprintf("unrecognized matchinfo flag: %c", zArg[i]); sqlite3_result_error(pCtx, zErr, -1); sqlite3_free(zErr); return 0; } nInt += n; } nByte = sizeof(Fts5MatchinfoCtx) /* The struct itself */ + sizeof(u32) * nInt /* The p->aRet[] array */ + (i+1); /* The p->zArg string */ p = (Fts5MatchinfoCtx*)sqlite3_malloc64(nByte); if( p==0 ){ sqlite3_result_error_nomem(pCtx); return 0; } memset(p, 0, nByte); p->nCol = nCol; |
︙ | ︙ |
Changes to ext/fts5/fts5_test_tok.c.
︙ | ︙ | |||
133 134 135 136 137 138 139 | int nByte = 0; char **azDequote; for(i=0; i<argc; i++){ nByte += (int)(strlen(argv[i]) + 1); } | | | 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 | int nByte = 0; char **azDequote; for(i=0; i<argc; i++){ nByte += (int)(strlen(argv[i]) + 1); } *pazDequote = azDequote = sqlite3_malloc64(sizeof(char *)*argc + nByte); if( azDequote==0 ){ rc = SQLITE_NOMEM; }else{ char *pSpace = (char *)&azDequote[argc]; for(i=0; i<argc; i++){ int n = (int)strlen(argv[i]); azDequote[i] = pSpace; |
︙ | ︙ | |||
207 208 209 210 211 212 213 | const char *zModule = 0; if( nDequote>0 ){ zModule = azDequote[0]; } rc = pApi->xFindTokenizer(pApi, zModule, &pTokCtx, &pTab->tok); if( rc==SQLITE_OK ){ | | | 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 | const char *zModule = 0; if( nDequote>0 ){ zModule = azDequote[0]; } rc = pApi->xFindTokenizer(pApi, zModule, &pTokCtx, &pTab->tok); if( rc==SQLITE_OK ){ const char **azArg = (nDequote>1 ? (const char **)&azDequote[1] : 0); int nArg = nDequote>0 ? nDequote-1 : 0; rc = pTab->tok.xCreate(pTokCtx, azArg, nArg, &pTab->pTok); } } if( rc!=SQLITE_OK ){ sqlite3_free(pTab); |
︙ | ︙ | |||
331 332 333 334 335 336 337 | ){ Fts5tokCursor *pCsr = (Fts5tokCursor*)pCtx; Fts5tokRow *pRow; if( (pCsr->nRow & (pCsr->nRow-1))==0 ){ int nNew = pCsr->nRow ? pCsr->nRow*2 : 32; Fts5tokRow *aNew; | | | 331 332 333 334 335 336 337 338 339 340 341 342 343 344 345 | ){ Fts5tokCursor *pCsr = (Fts5tokCursor*)pCtx; Fts5tokRow *pRow; if( (pCsr->nRow & (pCsr->nRow-1))==0 ){ int nNew = pCsr->nRow ? pCsr->nRow*2 : 32; Fts5tokRow *aNew; aNew = (Fts5tokRow*)sqlite3_realloc64(pCsr->aRow, nNew*sizeof(Fts5tokRow)); if( aNew==0 ) return SQLITE_NOMEM; memset(&aNew[pCsr->nRow], 0, sizeof(Fts5tokRow)*(nNew-pCsr->nRow)); pCsr->aRow = aNew; } pRow = &pCsr->aRow[pCsr->nRow]; pRow->iStart = iStart; |
︙ | ︙ | |||
374 375 376 377 378 379 380 | if( idxNum==1 ){ const char *zByte = (const char *)sqlite3_value_text(apVal[0]); int nByte = sqlite3_value_bytes(apVal[0]); pCsr->zInput = sqlite3_malloc(nByte+1); if( pCsr->zInput==0 ){ rc = SQLITE_NOMEM; }else{ | | | 374 375 376 377 378 379 380 381 382 383 384 385 386 387 388 | if( idxNum==1 ){ const char *zByte = (const char *)sqlite3_value_text(apVal[0]); int nByte = sqlite3_value_bytes(apVal[0]); pCsr->zInput = sqlite3_malloc(nByte+1); if( pCsr->zInput==0 ){ rc = SQLITE_NOMEM; }else{ if( nByte>0 ) memcpy(pCsr->zInput, zByte, nByte); pCsr->zInput[nByte] = 0; rc = pTab->tok.xTokenize( pTab->pTok, (void*)pCsr, 0, zByte, nByte, fts5tokCb ); } } |
︙ | ︙ |
Changes to ext/fts5/fts5_tokenize.c.
︙ | ︙ | |||
148 149 150 151 152 153 154 | ie++; } /* Fold to lower case */ nByte = ie-is; if( nByte>nFold ){ if( pFold!=aFold ) sqlite3_free(pFold); | | | 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 | ie++; } /* Fold to lower case */ nByte = ie-is; if( nByte>nFold ){ if( pFold!=aFold ) sqlite3_free(pFold); pFold = sqlite3_malloc64((sqlite3_int64)nByte*2); if( pFold==0 ){ rc = SQLITE_NOMEM; break; } nFold = nByte*2; } asciiFold(pFold, &pText[is], nByte); |
︙ | ︙ | |||
252 253 254 255 256 257 258 | int bTokenChars /* 1 for 'tokenchars', 0 for 'separators' */ ){ int rc = SQLITE_OK; int n = (int)strlen(z); int *aNew; if( n>0 ){ | | > | | | 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 | int bTokenChars /* 1 for 'tokenchars', 0 for 'separators' */ ){ int rc = SQLITE_OK; int n = (int)strlen(z); int *aNew; if( n>0 ){ aNew = (int*)sqlite3_realloc64(p->aiException, (n+p->nException)*sizeof(int)); if( aNew ){ int nNew = p->nException; const unsigned char *zCsr = (const unsigned char*)z; const unsigned char *zTerm = (const unsigned char*)&z[n]; while( zCsr<zTerm ){ u32 iCode; int bToken; READ_UTF8(zCsr, zTerm, iCode); if( iCode<128 ){ p->aTokenChar[iCode] = (unsigned char)bTokenChars; }else{ bToken = p->aCategory[sqlite3Fts5UnicodeCategory(iCode)]; assert( (bToken==0 || bToken==1) ); assert( (bTokenChars==0 || bTokenChars==1) ); if( bToken!=bTokenChars && sqlite3Fts5UnicodeIsdiacritic(iCode)==0 ){ int i; for(i=0; i<nNew; i++){ if( (u32)aNew[i]>iCode ) break; } memmove(&aNew[i+1], &aNew[i], (nNew-i)*sizeof(int)); aNew[i] = iCode; nNew++; } } } |
︙ | ︙ | |||
364 365 366 367 368 369 370 | if( p ){ const char *zCat = "L* N* Co"; int i; memset(p, 0, sizeof(Unicode61Tokenizer)); p->eRemoveDiacritic = FTS5_REMOVE_DIACRITICS_SIMPLE; p->nFold = 64; | | | 365 366 367 368 369 370 371 372 373 374 375 376 377 378 379 | if( p ){ const char *zCat = "L* N* Co"; int i; memset(p, 0, sizeof(Unicode61Tokenizer)); p->eRemoveDiacritic = FTS5_REMOVE_DIACRITICS_SIMPLE; p->nFold = 64; p->aFold = sqlite3_malloc64(p->nFold * sizeof(char)); if( p->aFold==0 ){ rc = SQLITE_NOMEM; } /* Search for a "categories" argument */ for(i=0; rc==SQLITE_OK && i<nArg; i+=2){ if( 0==sqlite3_stricmp(azArg[i], "categories") ){ |
︙ | ︙ | |||
425 426 427 428 429 430 431 | /* ** Return true if, for the purposes of tokenizing with the tokenizer ** passed as the first argument, codepoint iCode is considered a token ** character (not a separator). */ static int fts5UnicodeIsAlnum(Unicode61Tokenizer *p, int iCode){ return ( | | | 426 427 428 429 430 431 432 433 434 435 436 437 438 439 440 | /* ** Return true if, for the purposes of tokenizing with the tokenizer ** passed as the first argument, codepoint iCode is considered a token ** character (not a separator). */ static int fts5UnicodeIsAlnum(Unicode61Tokenizer *p, int iCode){ return ( p->aCategory[sqlite3Fts5UnicodeCategory((u32)iCode)] ^ fts5UnicodeIsException(p, iCode) ); } static int fts5UnicodeTokenize( Fts5Tokenizer *pTokenizer, void *pCtx, |
︙ | ︙ | |||
454 455 456 457 458 459 460 | const char *pEnd = &aFold[nFold-6]; UNUSED_PARAM(iUnused); /* Each iteration of this loop gobbles up a contiguous run of separators, ** then the next token. */ while( rc==SQLITE_OK ){ | | | 455 456 457 458 459 460 461 462 463 464 465 466 467 468 469 | const char *pEnd = &aFold[nFold-6]; UNUSED_PARAM(iUnused); /* Each iteration of this loop gobbles up a contiguous run of separators, ** then the next token. */ while( rc==SQLITE_OK ){ u32 iCode; /* non-ASCII codepoint read from input */ char *zOut = aFold; int is; int ie; /* Skip any separator characters. */ while( 1 ){ if( zCsr>=zTerm ) goto tokenize_done; |
︙ | ︙ | |||
486 487 488 489 490 491 492 | /* Run through the tokenchars. Fold them into the output buffer along ** the way. */ while( zCsr<zTerm ){ /* Grow the output buffer so that there is sufficient space to fit the ** largest possible utf-8 character. */ if( zOut>pEnd ){ | | | 487 488 489 490 491 492 493 494 495 496 497 498 499 500 501 | /* Run through the tokenchars. Fold them into the output buffer along ** the way. */ while( zCsr<zTerm ){ /* Grow the output buffer so that there is sufficient space to fit the ** largest possible utf-8 character. */ if( zOut>pEnd ){ aFold = sqlite3_malloc64((sqlite3_int64)nFold*2); if( aFold==0 ){ rc = SQLITE_NOMEM; goto tokenize_done; } zOut = &aFold[zOut - p->aFold]; memcpy(aFold, p->aFold, nFold); sqlite3_free(p->aFold); |
︙ | ︙ | |||
1252 1253 1254 1255 1256 1257 1258 1259 1260 1261 1262 1263 1264 1265 1266 1267 1268 1269 1270 1271 1272 1273 1274 1275 1276 1277 1278 1279 1280 1281 1282 1283 1284 1285 1286 | sCtx.xToken = xToken; sCtx.pCtx = pCtx; sCtx.aBuf = p->aBuf; return p->tokenizer.xTokenize( p->pTokenizer, (void*)&sCtx, flags, pText, nText, fts5PorterCb ); } /* ** Register all built-in tokenizers with FTS5. */ int sqlite3Fts5TokenizerInit(fts5_api *pApi){ struct BuiltinTokenizer { const char *zName; fts5_tokenizer x; } aBuiltin[] = { { "unicode61", {fts5UnicodeCreate, fts5UnicodeDelete, fts5UnicodeTokenize}}, { "ascii", {fts5AsciiCreate, fts5AsciiDelete, fts5AsciiTokenize }}, { "porter", {fts5PorterCreate, fts5PorterDelete, fts5PorterTokenize }}, }; int rc = SQLITE_OK; /* Return code */ int i; /* To iterate through builtin functions */ for(i=0; rc==SQLITE_OK && i<ArraySize(aBuiltin); i++){ rc = pApi->xCreateTokenizer(pApi, aBuiltin[i].zName, (void*)pApi, &aBuiltin[i].x, 0 ); } return rc; } | > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > < < | 1253 1254 1255 1256 1257 1258 1259 1260 1261 1262 1263 1264 1265 1266 1267 1268 1269 1270 1271 1272 1273 1274 1275 1276 1277 1278 1279 1280 1281 1282 1283 1284 1285 1286 1287 1288 1289 1290 1291 1292 1293 1294 1295 1296 1297 1298 1299 1300 1301 1302 1303 1304 1305 1306 1307 1308 1309 1310 1311 1312 1313 1314 1315 1316 1317 1318 1319 1320 1321 1322 1323 1324 1325 1326 1327 1328 1329 1330 1331 1332 1333 1334 1335 1336 1337 1338 1339 1340 1341 1342 1343 1344 1345 1346 1347 1348 1349 1350 1351 1352 1353 1354 1355 1356 1357 1358 1359 1360 1361 1362 1363 1364 1365 1366 1367 1368 1369 1370 1371 1372 1373 1374 1375 1376 1377 1378 1379 1380 1381 1382 1383 1384 1385 1386 1387 1388 1389 1390 1391 1392 1393 1394 1395 1396 1397 1398 1399 1400 1401 1402 1403 1404 1405 1406 1407 1408 1409 1410 1411 1412 1413 1414 1415 | sCtx.xToken = xToken; sCtx.pCtx = pCtx; sCtx.aBuf = p->aBuf; return p->tokenizer.xTokenize( p->pTokenizer, (void*)&sCtx, flags, pText, nText, fts5PorterCb ); } /************************************************************************** ** Start of trigram implementation. */ typedef struct TrigramTokenizer TrigramTokenizer; struct TrigramTokenizer { int bFold; /* True to fold to lower-case */ }; /* ** Free a trigram tokenizer. */ static void fts5TriDelete(Fts5Tokenizer *p){ sqlite3_free(p); } /* ** Allocate a trigram tokenizer. */ static int fts5TriCreate( void *pUnused, const char **azArg, int nArg, Fts5Tokenizer **ppOut ){ int rc = SQLITE_OK; TrigramTokenizer *pNew = (TrigramTokenizer*)sqlite3_malloc(sizeof(*pNew)); UNUSED_PARAM(pUnused); if( pNew==0 ){ rc = SQLITE_NOMEM; }else{ int i; pNew->bFold = 1; for(i=0; rc==SQLITE_OK && i<nArg; i+=2){ const char *zArg = azArg[i+1]; if( 0==sqlite3_stricmp(azArg[i], "case_sensitive") ){ if( (zArg[0]!='0' && zArg[0]!='1') || zArg[1] ){ rc = SQLITE_ERROR; }else{ pNew->bFold = (zArg[0]=='0'); } }else{ rc = SQLITE_ERROR; } } if( rc!=SQLITE_OK ){ fts5TriDelete((Fts5Tokenizer*)pNew); pNew = 0; } } *ppOut = (Fts5Tokenizer*)pNew; return rc; } /* ** Trigram tokenizer tokenize routine. */ static int fts5TriTokenize( Fts5Tokenizer *pTok, void *pCtx, int unusedFlags, const char *pText, int nText, int (*xToken)(void*, int, const char*, int, int, int) ){ TrigramTokenizer *p = (TrigramTokenizer*)pTok; int rc = SQLITE_OK; char aBuf[32]; const unsigned char *zIn = (const unsigned char*)pText; const unsigned char *zEof = &zIn[nText]; u32 iCode; UNUSED_PARAM(unusedFlags); while( 1 ){ char *zOut = aBuf; int iStart = zIn - (const unsigned char*)pText; const unsigned char *zNext; READ_UTF8(zIn, zEof, iCode); if( iCode==0 ) break; zNext = zIn; if( zIn<zEof ){ if( p->bFold ) iCode = sqlite3Fts5UnicodeFold(iCode, 0); WRITE_UTF8(zOut, iCode); READ_UTF8(zIn, zEof, iCode); if( iCode==0 ) break; }else{ break; } if( zIn<zEof ){ if( p->bFold ) iCode = sqlite3Fts5UnicodeFold(iCode, 0); WRITE_UTF8(zOut, iCode); READ_UTF8(zIn, zEof, iCode); if( iCode==0 ) break; if( p->bFold ) iCode = sqlite3Fts5UnicodeFold(iCode, 0); WRITE_UTF8(zOut, iCode); }else{ break; } rc = xToken(pCtx, 0, aBuf, zOut-aBuf, iStart, iStart + zOut-aBuf); if( rc!=SQLITE_OK ) break; zIn = zNext; } return rc; } /* ** Argument xCreate is a pointer to a constructor function for a tokenizer. ** pTok is a tokenizer previously created using the same method. This function ** returns one of FTS5_PATTERN_NONE, FTS5_PATTERN_LIKE or FTS5_PATTERN_GLOB ** indicating the style of pattern matching that the tokenizer can support. ** In practice, this is: ** ** "trigram" tokenizer, case_sensitive=1 - FTS5_PATTERN_GLOB ** "trigram" tokenizer, case_sensitive=0 (the default) - FTS5_PATTERN_LIKE ** all other tokenizers - FTS5_PATTERN_NONE */ int sqlite3Fts5TokenizerPattern( int (*xCreate)(void*, const char**, int, Fts5Tokenizer**), Fts5Tokenizer *pTok ){ if( xCreate==fts5TriCreate ){ TrigramTokenizer *p = (TrigramTokenizer*)pTok; return p->bFold ? FTS5_PATTERN_LIKE : FTS5_PATTERN_GLOB; } return FTS5_PATTERN_NONE; } /* ** Register all built-in tokenizers with FTS5. */ int sqlite3Fts5TokenizerInit(fts5_api *pApi){ struct BuiltinTokenizer { const char *zName; fts5_tokenizer x; } aBuiltin[] = { { "unicode61", {fts5UnicodeCreate, fts5UnicodeDelete, fts5UnicodeTokenize}}, { "ascii", {fts5AsciiCreate, fts5AsciiDelete, fts5AsciiTokenize }}, { "porter", {fts5PorterCreate, fts5PorterDelete, fts5PorterTokenize }}, { "trigram", {fts5TriCreate, fts5TriDelete, fts5TriTokenize}}, }; int rc = SQLITE_OK; /* Return code */ int i; /* To iterate through builtin functions */ for(i=0; rc==SQLITE_OK && i<ArraySize(aBuiltin); i++){ rc = pApi->xCreateTokenizer(pApi, aBuiltin[i].zName, (void*)pApi, &aBuiltin[i].x, 0 ); } return rc; } |
Changes to ext/fts5/fts5_unicode2.c.
1 | /* | | | 1 2 3 4 5 6 7 8 9 | /* ** 2012-05-25 ** ** The author disclaims copyright to this source code. In place of ** a legal notice, here is a blessing: ** ** May you do good and not evil. ** May you find forgiveness for yourself and forgive others. ** May you share freely, never taking more than you give. |
︙ | ︙ | |||
43 44 45 46 47 48 49 | 61726, 61784, 61800, 61816, 61836, 61880, 61896, 61914, 61948, 61998, 62062, 62122, 62154, 62184, 62200, 62218, 62252, 62302, 62364, 62410, 62442, 62478, 62536, 62554, 62584, 62604, 62640, 62648, 62656, 62664, 62730, 62766, 62830, 62890, 62924, 62974, 63032, 63050, 63082, 63118, 63182, 63242, 63274, 63310, 63368, 63390, }; | | | | | 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 | 61726, 61784, 61800, 61816, 61836, 61880, 61896, 61914, 61948, 61998, 62062, 62122, 62154, 62184, 62200, 62218, 62252, 62302, 62364, 62410, 62442, 62478, 62536, 62554, 62584, 62604, 62640, 62648, 62656, 62664, 62730, 62766, 62830, 62890, 62924, 62974, 63032, 63050, 63082, 63118, 63182, 63242, 63274, 63310, 63368, 63390, }; #define HIBIT ((unsigned char)0x80) unsigned char aChar[] = { '\0', 'a', 'c', 'e', 'i', 'n', 'o', 'u', 'y', 'y', 'a', 'c', 'd', 'e', 'e', 'g', 'h', 'i', 'j', 'k', 'l', 'n', 'o', 'r', 's', 't', 'u', 'u', 'w', 'y', 'z', 'o', 'u', 'a', 'i', 'o', 'u', 'u'|HIBIT, 'a'|HIBIT, 'g', 'k', 'o', 'o'|HIBIT, 'j', 'g', 'n', 'a'|HIBIT, 'a', 'e', 'i', 'o', 'r', 'u', 's', 't', 'h', 'a', 'e', 'o'|HIBIT, 'o', 'o'|HIBIT, 'y', '\0', '\0', '\0', '\0', '\0', '\0', '\0', '\0', 'a', 'b', 'c'|HIBIT, 'd', 'd', 'e'|HIBIT, 'e', 'e'|HIBIT, 'f', 'g', 'h', 'h', 'i', 'i'|HIBIT, 'k', 'l', 'l'|HIBIT, 'l', 'm', 'n', 'o'|HIBIT, 'p', 'r', 'r'|HIBIT, 'r', 's', 's'|HIBIT, 't', 'u', 'u'|HIBIT, 'v', 'w', 'w', 'x', 'y', 'z', 'h', 't', 'w', 'y', 'a', 'a'|HIBIT, 'a'|HIBIT, 'a'|HIBIT, 'e', 'e'|HIBIT, 'e'|HIBIT, 'i', 'o', 'o'|HIBIT, 'o'|HIBIT, 'o'|HIBIT, 'u', 'u'|HIBIT, 'u'|HIBIT, 'y', }; unsigned int key = (((unsigned int)c)<<3) | 0x00000007; int iRes = 0; int iHi = sizeof(aDia)/sizeof(aDia[0]) - 1; int iLo = 0; while( iHi>=iLo ){ |
︙ | ︙ | |||
96 97 98 99 100 101 102 | ** is a diacritical modifier character. */ int sqlite3Fts5UnicodeIsdiacritic(int c){ unsigned int mask0 = 0x08029FDF; unsigned int mask1 = 0x000361F8; if( c<768 || c>817 ) return 0; return (c < 768+32) ? | | | | 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 | ** is a diacritical modifier character. */ int sqlite3Fts5UnicodeIsdiacritic(int c){ unsigned int mask0 = 0x08029FDF; unsigned int mask1 = 0x000361F8; if( c<768 || c>817 ) return 0; return (c < 768+32) ? (mask0 & ((unsigned int)1 << (c-768))) : (mask1 & ((unsigned int)1 << (c-768-32))); } /* ** Interpret the argument as a unicode codepoint. If the codepoint ** is an upper case character that has a lower case equivalent, ** return the codepoint corresponding to the lower case version. |
︙ | ︙ | |||
244 245 246 247 248 249 250 251 252 253 254 255 256 257 | else if( c>=66560 && c<66600 ){ ret = c + 40; } return ret; } int sqlite3Fts5UnicodeCatParse(const char *zCat, u8 *aArray){ aArray[0] = 1; switch( zCat[0] ){ case 'C': switch( zCat[1] ){ case 'c': aArray[1] = 1; break; | > | 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 | else if( c>=66560 && c<66600 ){ ret = c + 40; } return ret; } int sqlite3Fts5UnicodeCatParse(const char *zCat, u8 *aArray){ aArray[0] = 1; switch( zCat[0] ){ case 'C': switch( zCat[1] ){ case 'c': aArray[1] = 1; break; |
︙ | ︙ | |||
726 727 728 729 730 731 732 | 89, 1434, 3226, 506, 474, 506, 506, 367, 1018, 1946, 1402, 954, 1402, 314, 90, 1082, 218, 2266, 666, 1210, 186, 570, 2042, 58, 5850, 154, 2010, 154, 794, 2266, 378, 2266, 3738, 39, 39, 39, 39, 39, 39, 17351, 34, 3074, 7692, 63, 63, }; | | | 727 728 729 730 731 732 733 734 735 736 737 738 739 740 741 | 89, 1434, 3226, 506, 474, 506, 506, 367, 1018, 1946, 1402, 954, 1402, 314, 90, 1082, 218, 2266, 666, 1210, 186, 570, 2042, 58, 5850, 154, 2010, 154, 794, 2266, 378, 2266, 3738, 39, 39, 39, 39, 39, 39, 17351, 34, 3074, 7692, 63, 63, }; int sqlite3Fts5UnicodeCategory(u32 iCode) { int iRes = -1; int iHi; int iLo; int ret; u16 iKey; if( iCode>=(1<<20) ){ |
︙ | ︙ | |||
764 765 766 767 768 769 770 | void sqlite3Fts5UnicodeAscii(u8 *aArray, u8 *aAscii){ int i = 0; int iTbl = 0; while( i<128 ){ int bToken = aArray[ aFts5UnicodeData[iTbl] & 0x1F ]; int n = (aFts5UnicodeData[iTbl] >> 5) + i; for(; i<128 && i<n; i++){ | | > > | 765 766 767 768 769 770 771 772 773 774 775 776 777 778 | void sqlite3Fts5UnicodeAscii(u8 *aArray, u8 *aAscii){ int i = 0; int iTbl = 0; while( i<128 ){ int bToken = aArray[ aFts5UnicodeData[iTbl] & 0x1F ]; int n = (aFts5UnicodeData[iTbl] >> 5) + i; for(; i<128 && i<n; i++){ aAscii[i] = (u8)bToken; } iTbl++; } aAscii[0] = 0; /* 0x00 is never a token character */ } |
Changes to ext/fts5/fts5_varint.c.
︙ | ︙ | |||
72 73 74 75 76 77 78 | ** routine. */ { u64 v64; u8 n; p -= 2; n = sqlite3Fts5GetVarint(p, &v64); | | | 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 | ** routine. */ { u64 v64; u8 n; p -= 2; n = sqlite3Fts5GetVarint(p, &v64); *v = ((u32)v64) & 0x7FFFFFFF; assert( n>3 && n<=9 ); return n; } } /* |
︙ | ︙ | |||
338 339 340 341 342 343 344 | #endif assert( iVal>=(1 << 7) ); if( iVal<(1 << 14) ) return 2; if( iVal<(1 << 21) ) return 3; if( iVal<(1 << 28) ) return 4; return 5; } | < | 338 339 340 341 342 343 344 | #endif assert( iVal>=(1 << 7) ); if( iVal<(1 << 14) ) return 2; if( iVal<(1 << 21) ) return 3; if( iVal<(1 << 28) ) return 4; return 5; } |
Changes to ext/fts5/fts5_vocab.c.
︙ | ︙ | |||
46 47 48 49 50 51 52 53 54 55 56 57 | struct Fts5VocabTable { sqlite3_vtab base; char *zFts5Tbl; /* Name of fts5 table */ char *zFts5Db; /* Db containing fts5 table */ sqlite3 *db; /* Database handle */ Fts5Global *pGlobal; /* FTS5 global object for this database */ int eType; /* FTS5_VOCAB_COL, ROW or INSTANCE */ }; struct Fts5VocabCursor { sqlite3_vtab_cursor base; sqlite3_stmt *pStmt; /* Statement holding lock on pIndex */ | > | > < | 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 | struct Fts5VocabTable { sqlite3_vtab base; char *zFts5Tbl; /* Name of fts5 table */ char *zFts5Db; /* Db containing fts5 table */ sqlite3 *db; /* Database handle */ Fts5Global *pGlobal; /* FTS5 global object for this database */ int eType; /* FTS5_VOCAB_COL, ROW or INSTANCE */ unsigned bBusy; /* True if busy */ }; struct Fts5VocabCursor { sqlite3_vtab_cursor base; sqlite3_stmt *pStmt; /* Statement holding lock on pIndex */ Fts5Table *pFts5; /* Associated FTS5 table */ int bEof; /* True if this cursor is at EOF */ Fts5IndexIter *pIter; /* Term/rowid iterator object */ void *pStruct; /* From sqlite3Fts5StructureRef() */ int nLeTerm; /* Size of zLeTerm in bytes */ char *zLeTerm; /* (term <= $zLeTerm) paramater, or NULL */ /* These are used by 'col' tables only */ int iCol; i64 *aCnt; i64 *aDoc; /* Output values used by all tables. */ i64 rowid; /* This table's current rowid value */ Fts5Buffer term; /* Current value of 'term' column */ |
︙ | ︙ | |||
323 324 325 326 327 328 329 | ** Implementation of xOpen method. */ static int fts5VocabOpenMethod( sqlite3_vtab *pVTab, sqlite3_vtab_cursor **ppCsr ){ Fts5VocabTable *pTab = (Fts5VocabTable*)pVTab; | | < > > > > > > > | > | > | | | | | | | > > > | | < | > > > | 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340 341 342 343 344 345 346 347 348 349 350 351 352 353 354 355 356 357 358 359 360 361 362 363 364 365 366 367 368 369 370 371 372 373 374 375 376 377 378 379 380 381 382 383 384 385 386 387 388 389 390 391 392 393 394 395 396 397 398 399 400 401 402 403 404 405 406 407 408 409 410 | ** Implementation of xOpen method. */ static int fts5VocabOpenMethod( sqlite3_vtab *pVTab, sqlite3_vtab_cursor **ppCsr ){ Fts5VocabTable *pTab = (Fts5VocabTable*)pVTab; Fts5Table *pFts5 = 0; Fts5VocabCursor *pCsr = 0; int rc = SQLITE_OK; sqlite3_stmt *pStmt = 0; char *zSql = 0; if( pTab->bBusy ){ pVTab->zErrMsg = sqlite3_mprintf( "recursive definition for %s.%s", pTab->zFts5Db, pTab->zFts5Tbl ); return SQLITE_ERROR; } zSql = sqlite3Fts5Mprintf(&rc, "SELECT t.%Q FROM %Q.%Q AS t WHERE t.%Q MATCH '*id'", pTab->zFts5Tbl, pTab->zFts5Db, pTab->zFts5Tbl, pTab->zFts5Tbl ); if( zSql ){ rc = sqlite3_prepare_v2(pTab->db, zSql, -1, &pStmt, 0); } sqlite3_free(zSql); assert( rc==SQLITE_OK || pStmt==0 ); if( rc==SQLITE_ERROR ) rc = SQLITE_OK; pTab->bBusy = 1; if( pStmt && sqlite3_step(pStmt)==SQLITE_ROW ){ i64 iId = sqlite3_column_int64(pStmt, 0); pFts5 = sqlite3Fts5TableFromCsrid(pTab->pGlobal, iId); } pTab->bBusy = 0; if( rc==SQLITE_OK ){ if( pFts5==0 ){ rc = sqlite3_finalize(pStmt); pStmt = 0; if( rc==SQLITE_OK ){ pVTab->zErrMsg = sqlite3_mprintf( "no such fts5 table: %s.%s", pTab->zFts5Db, pTab->zFts5Tbl ); rc = SQLITE_ERROR; } }else{ rc = sqlite3Fts5FlushToDisk(pFts5); } } if( rc==SQLITE_OK ){ i64 nByte = pFts5->pConfig->nCol * sizeof(i64)*2 + sizeof(Fts5VocabCursor); pCsr = (Fts5VocabCursor*)sqlite3Fts5MallocZero(&rc, nByte); } if( pCsr ){ pCsr->pFts5 = pFts5; pCsr->pStmt = pStmt; pCsr->aCnt = (i64*)&pCsr[1]; pCsr->aDoc = &pCsr->aCnt[pFts5->pConfig->nCol]; }else{ sqlite3_finalize(pStmt); } *ppCsr = (sqlite3_vtab_cursor*)pCsr; return rc; } static void fts5VocabResetCursor(Fts5VocabCursor *pCsr){ pCsr->rowid = 0; sqlite3Fts5IterClose(pCsr->pIter); sqlite3Fts5StructureRelease(pCsr->pStruct); pCsr->pStruct = 0; pCsr->pIter = 0; sqlite3_free(pCsr->zLeTerm); pCsr->nLeTerm = -1; pCsr->zLeTerm = 0; pCsr->bEof = 0; } /* ** Close the cursor. For additional information see the documentation ** on the xClose method of the virtual table interface. */ static int fts5VocabCloseMethod(sqlite3_vtab_cursor *pCursor){ |
︙ | ︙ | |||
421 422 423 424 425 426 427 | sqlite3Fts5BufferSet(&rc, &pCsr->term, nTerm, (const u8*)zTerm); } return rc; } static int fts5VocabInstanceNext(Fts5VocabCursor *pCsr){ | | | 435 436 437 438 439 440 441 442 443 444 445 446 447 448 449 | sqlite3Fts5BufferSet(&rc, &pCsr->term, nTerm, (const u8*)zTerm); } return rc; } static int fts5VocabInstanceNext(Fts5VocabCursor *pCsr){ int eDetail = pCsr->pFts5->pConfig->eDetail; int rc = SQLITE_OK; Fts5IndexIter *pIter = pCsr->pIter; i64 *pp = &pCsr->iInstPos; int *po = &pCsr->iInstOff; assert( sqlite3Fts5IterEof(pIter)==0 ); assert( pCsr->bEof==0 ); |
︙ | ︙ | |||
455 456 457 458 459 460 461 | /* ** Advance the cursor to the next row in the table. */ static int fts5VocabNextMethod(sqlite3_vtab_cursor *pCursor){ Fts5VocabCursor *pCsr = (Fts5VocabCursor*)pCursor; Fts5VocabTable *pTab = (Fts5VocabTable*)pCursor->pVtab; | < | > > > > | | 469 470 471 472 473 474 475 476 477 478 479 480 481 482 483 484 485 486 487 488 489 490 491 492 493 494 495 496 497 498 499 500 501 502 503 504 505 506 507 508 509 510 511 512 513 514 515 516 517 518 519 520 521 522 523 524 525 | /* ** Advance the cursor to the next row in the table. */ static int fts5VocabNextMethod(sqlite3_vtab_cursor *pCursor){ Fts5VocabCursor *pCsr = (Fts5VocabCursor*)pCursor; Fts5VocabTable *pTab = (Fts5VocabTable*)pCursor->pVtab; int nCol = pCsr->pFts5->pConfig->nCol; int rc; rc = sqlite3Fts5StructureTest(pCsr->pFts5->pIndex, pCsr->pStruct); if( rc!=SQLITE_OK ) return rc; pCsr->rowid++; if( pTab->eType==FTS5_VOCAB_INSTANCE ){ return fts5VocabInstanceNext(pCsr); } if( pTab->eType==FTS5_VOCAB_COL ){ for(pCsr->iCol++; pCsr->iCol<nCol; pCsr->iCol++){ if( pCsr->aDoc[pCsr->iCol] ) break; } } if( pTab->eType!=FTS5_VOCAB_COL || pCsr->iCol>=nCol ){ if( sqlite3Fts5IterEof(pCsr->pIter) ){ pCsr->bEof = 1; }else{ const char *zTerm; int nTerm; zTerm = sqlite3Fts5IterTerm(pCsr->pIter, &nTerm); assert( nTerm>=0 ); if( pCsr->nLeTerm>=0 ){ int nCmp = MIN(nTerm, pCsr->nLeTerm); int bCmp = memcmp(pCsr->zLeTerm, zTerm, nCmp); if( bCmp<0 || (bCmp==0 && pCsr->nLeTerm<nTerm) ){ pCsr->bEof = 1; return SQLITE_OK; } } sqlite3Fts5BufferSet(&rc, &pCsr->term, nTerm, (const u8*)zTerm); memset(pCsr->aCnt, 0, nCol * sizeof(i64)); memset(pCsr->aDoc, 0, nCol * sizeof(i64)); pCsr->iCol = 0; assert( pTab->eType==FTS5_VOCAB_COL || pTab->eType==FTS5_VOCAB_ROW ); while( rc==SQLITE_OK ){ int eDetail = pCsr->pFts5->pConfig->eDetail; const u8 *pPos; int nPos; /* Position list */ i64 iPos = 0; /* 64-bit position read from poslist */ int iOff = 0; /* Current offset within position list */ pPos = pCsr->pIter->pData; nPos = pCsr->pIter->nData; |
︙ | ︙ | |||
517 518 519 520 521 522 523 | break; case FTS5_VOCAB_COL: if( eDetail==FTS5_DETAIL_FULL ){ int iCol = -1; while( 0==sqlite3Fts5PoslistNext64(pPos, nPos, &iOff, &iPos) ){ int ii = FTS5_POS2COLUMN(iPos); | < > | 534 535 536 537 538 539 540 541 542 543 544 545 546 547 548 549 550 551 552 553 554 555 556 | break; case FTS5_VOCAB_COL: if( eDetail==FTS5_DETAIL_FULL ){ int iCol = -1; while( 0==sqlite3Fts5PoslistNext64(pPos, nPos, &iOff, &iPos) ){ int ii = FTS5_POS2COLUMN(iPos); if( iCol!=ii ){ if( ii>=nCol ){ rc = FTS5_CORRUPT; break; } pCsr->aDoc[ii]++; iCol = ii; } pCsr->aCnt[ii]++; } }else if( eDetail==FTS5_DETAIL_COLUMNS ){ while( 0==sqlite3Fts5PoslistNext64(pPos, nPos, &iOff,&iPos) ){ assert_nc( iPos>=0 && iPos<nCol ); if( iPos>=nCol ){ rc = FTS5_CORRUPT; break; |
︙ | ︙ | |||
554 555 556 557 558 559 560 | if( rc==SQLITE_OK ){ rc = sqlite3Fts5IterNextScan(pCsr->pIter); } if( pTab->eType==FTS5_VOCAB_INSTANCE ) break; if( rc==SQLITE_OK ){ zTerm = sqlite3Fts5IterTerm(pCsr->pIter, &nTerm); | | > > | | > > | 571 572 573 574 575 576 577 578 579 580 581 582 583 584 585 586 587 588 589 590 591 592 593 594 595 596 597 598 599 600 | if( rc==SQLITE_OK ){ rc = sqlite3Fts5IterNextScan(pCsr->pIter); } if( pTab->eType==FTS5_VOCAB_INSTANCE ) break; if( rc==SQLITE_OK ){ zTerm = sqlite3Fts5IterTerm(pCsr->pIter, &nTerm); if( nTerm!=pCsr->term.n || (nTerm>0 && memcmp(zTerm, pCsr->term.p, nTerm)) ){ break; } if( sqlite3Fts5IterEof(pCsr->pIter) ) break; } } } } if( rc==SQLITE_OK && pCsr->bEof==0 && pTab->eType==FTS5_VOCAB_COL ){ for(/* noop */; pCsr->iCol<nCol && pCsr->aDoc[pCsr->iCol]==0; pCsr->iCol++); if( pCsr->iCol==nCol ){ rc = FTS5_CORRUPT; } } return rc; } /* ** This is the xFilter implementation for the virtual table. */ |
︙ | ︙ | |||
612 613 614 615 616 617 618 619 620 621 622 623 624 625 626 627 628 629 | }else{ if( pGe ){ zTerm = (const char *)sqlite3_value_text(pGe); nTerm = sqlite3_value_bytes(pGe); } if( pLe ){ const char *zCopy = (const char *)sqlite3_value_text(pLe); pCsr->nLeTerm = sqlite3_value_bytes(pLe); pCsr->zLeTerm = sqlite3_malloc(pCsr->nLeTerm+1); if( pCsr->zLeTerm==0 ){ rc = SQLITE_NOMEM; }else{ memcpy(pCsr->zLeTerm, zCopy, pCsr->nLeTerm+1); } } } if( rc==SQLITE_OK ){ | > > | > > > | < | > | 633 634 635 636 637 638 639 640 641 642 643 644 645 646 647 648 649 650 651 652 653 654 655 656 657 658 659 660 661 662 663 664 665 666 667 668 669 670 | }else{ if( pGe ){ zTerm = (const char *)sqlite3_value_text(pGe); nTerm = sqlite3_value_bytes(pGe); } if( pLe ){ const char *zCopy = (const char *)sqlite3_value_text(pLe); if( zCopy==0 ) zCopy = ""; pCsr->nLeTerm = sqlite3_value_bytes(pLe); pCsr->zLeTerm = sqlite3_malloc(pCsr->nLeTerm+1); if( pCsr->zLeTerm==0 ){ rc = SQLITE_NOMEM; }else{ memcpy(pCsr->zLeTerm, zCopy, pCsr->nLeTerm+1); } } } if( rc==SQLITE_OK ){ Fts5Index *pIndex = pCsr->pFts5->pIndex; rc = sqlite3Fts5IndexQuery(pIndex, zTerm, nTerm, f, 0, &pCsr->pIter); if( rc==SQLITE_OK ){ pCsr->pStruct = sqlite3Fts5StructureRef(pIndex); } } if( rc==SQLITE_OK && eType==FTS5_VOCAB_INSTANCE ){ rc = fts5VocabInstanceNewTerm(pCsr); } if( rc==SQLITE_OK && !pCsr->bEof && (eType!=FTS5_VOCAB_INSTANCE || pCsr->pFts5->pConfig->eDetail!=FTS5_DETAIL_NONE) ){ rc = fts5VocabNextMethod(pCursor); } return rc; } |
︙ | ︙ | |||
653 654 655 656 657 658 659 | static int fts5VocabColumnMethod( sqlite3_vtab_cursor *pCursor, /* Cursor to retrieve value from */ sqlite3_context *pCtx, /* Context for sqlite3_result_xxx() calls */ int iCol /* Index of column to read value from */ ){ Fts5VocabCursor *pCsr = (Fts5VocabCursor*)pCursor; | | | | 679 680 681 682 683 684 685 686 687 688 689 690 691 692 693 694 695 696 697 698 699 700 701 702 703 704 705 | static int fts5VocabColumnMethod( sqlite3_vtab_cursor *pCursor, /* Cursor to retrieve value from */ sqlite3_context *pCtx, /* Context for sqlite3_result_xxx() calls */ int iCol /* Index of column to read value from */ ){ Fts5VocabCursor *pCsr = (Fts5VocabCursor*)pCursor; int eDetail = pCsr->pFts5->pConfig->eDetail; int eType = ((Fts5VocabTable*)(pCursor->pVtab))->eType; i64 iVal = 0; if( iCol==0 ){ sqlite3_result_text( pCtx, (const char*)pCsr->term.p, pCsr->term.n, SQLITE_TRANSIENT ); }else if( eType==FTS5_VOCAB_COL ){ assert( iCol==1 || iCol==2 || iCol==3 ); if( iCol==1 ){ if( eDetail!=FTS5_DETAIL_NONE ){ const char *z = pCsr->pFts5->pConfig->azCol[pCsr->iCol]; sqlite3_result_text(pCtx, z, -1, SQLITE_STATIC); } }else if( iCol==2 ){ iVal = pCsr->aDoc[pCsr->iCol]; }else{ iVal = pCsr->aCnt[pCsr->iCol]; } |
︙ | ︙ | |||
693 694 695 696 697 698 699 | case 2: { int ii = -1; if( eDetail==FTS5_DETAIL_FULL ){ ii = FTS5_POS2COLUMN(pCsr->iInstPos); }else if( eDetail==FTS5_DETAIL_COLUMNS ){ ii = (int)pCsr->iInstPos; } | | | | 719 720 721 722 723 724 725 726 727 728 729 730 731 732 733 734 | case 2: { int ii = -1; if( eDetail==FTS5_DETAIL_FULL ){ ii = FTS5_POS2COLUMN(pCsr->iInstPos); }else if( eDetail==FTS5_DETAIL_COLUMNS ){ ii = (int)pCsr->iInstPos; } if( ii>=0 && ii<pCsr->pFts5->pConfig->nCol ){ const char *z = pCsr->pFts5->pConfig->azCol[ii]; sqlite3_result_text(pCtx, z, -1, SQLITE_STATIC); } break; } default: { assert( iCol==3 ); if( eDetail==FTS5_DETAIL_FULL ){ |
︙ | ︙ |
Changes to ext/fts5/test/fts5aa.test.
︙ | ︙ | |||
34 35 36 37 38 39 40 | t1_docsize {CREATE TABLE 't1_docsize'(id INTEGER PRIMARY KEY, sz BLOB)} t1_config {CREATE TABLE 't1_config'(k PRIMARY KEY, v) WITHOUT ROWID} } do_execsql_test 1.1 { DROP TABLE t1; SELECT name, sql FROM sqlite_master; | | < | 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 | t1_docsize {CREATE TABLE 't1_docsize'(id INTEGER PRIMARY KEY, sz BLOB)} t1_config {CREATE TABLE 't1_config'(k PRIMARY KEY, v) WITHOUT ROWID} } do_execsql_test 1.1 { DROP TABLE t1; SELECT name, sql FROM sqlite_master; } {} #------------------------------------------------------------------------- # do_execsql_test 2.0 { CREATE VIRTUAL TABLE t1 USING fts5(x, y, detail=%DETAIL%); } |
︙ | ︙ | |||
424 425 426 427 428 429 430 | INSERT INTO n1 VALUES('a b c d'); } proc funk {} { db eval { UPDATE n1_config SET v=50 WHERE k='version' } set fd [db incrblob main n1_data block 10] fconfigure $fd -encoding binary -translation binary | | | 423 424 425 426 427 428 429 430 431 432 433 434 435 436 437 | INSERT INTO n1 VALUES('a b c d'); } proc funk {} { db eval { UPDATE n1_config SET v=50 WHERE k='version' } set fd [db incrblob main n1_data block 10] fconfigure $fd -encoding binary -translation binary # puts -nonewline $fd "\x44\x45" close $fd } db func funk funk # This test case corrupts the structure record within the first invocation # of function funk(). Which used to cause the bm25() function to throw an # exception. But since bm25() can now used the cached structure record, |
︙ | ︙ | |||
599 600 601 602 603 604 605 606 607 608 609 610 | } do_execsql_test 23.1 { SELECT * FROM t11, t10 WHERE t11.x = t10.x AND t10.rowid IS NULL; } do_execsql_test 23.2 { SELECT * FROM t11, t10 WHERE t10.rowid IS NULL; } } expand_all_sql db finish_test | > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | 598 599 600 601 602 603 604 605 606 607 608 609 610 611 612 613 614 615 616 617 618 619 620 621 622 623 624 625 626 627 628 629 630 631 632 633 634 635 636 637 638 639 640 641 | } do_execsql_test 23.1 { SELECT * FROM t11, t10 WHERE t11.x = t10.x AND t10.rowid IS NULL; } do_execsql_test 23.2 { SELECT * FROM t11, t10 WHERE t10.rowid IS NULL; } #------------------------------------------------------------------------- do_execsql_test 24.0 { CREATE VIRTUAL TABLE t12 USING fts5(x, detail=%DETAIL%); INSERT INTO t12 VALUES('aaaa'); } do_execsql_test 24.1 { BEGIN; DELETE FROM t12 WHERE rowid=1; SELECT * FROM t12('aaaa'); INSERT INTO t12 VALUES('aaaa'); END; } do_execsql_test 24.2 { INSERT INTO t12(t12) VALUES('integrity-check'); } do_execsql_test 24.3 { SELECT * FROM t12('aaaa'); } {aaaa} #------------------------------------------------------------------------- do_execsql_test 25.0 { CREATE VIRTUAL TABLE t13 USING fts5(x, detail=%DETAIL%); } do_execsql_test 25.1 { BEGIN; INSERT INTO t13 VALUES('AAAA'); SELECT * FROM t13('BBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBB*'); END; } } expand_all_sql db finish_test |
Changes to ext/fts5/test/fts5af.test.
︙ | ︙ | |||
161 162 163 164 165 166 167 | INSERT INTO x1 VALUES('xyz', '1 2 3 4 5 6 7 8 9 10 11 12 13'); SELECT snippet(x1, 1, '[', ']', '...', 5) FROM x1('xyz'); } { {1 2 3 4 5...} } do_execsql_test 5.0 { | | > > > > > > | 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 | INSERT INTO x1 VALUES('xyz', '1 2 3 4 5 6 7 8 9 10 11 12 13'); SELECT snippet(x1, 1, '[', ']', '...', 5) FROM x1('xyz'); } { {1 2 3 4 5...} } do_execsql_test 5.0 { CREATE VIRTUAL TABLE p1 USING fts5(a, b, detail=%DETAIL%); INSERT INTO p1 VALUES( 'x a a a a a a a a a a', 'a a a a a a a a a a a a a a a a a a a x' ); } do_execsql_test 5.1 { SELECT snippet(p1, 0, '[', ']', '...', 6) FROM p1('x'); } {{[x] a a a a a...}} do_execsql_test 5.2 { SELECT snippet(p1, 0, '[', ']', NULL, 6) FROM p1('x'); } {{[x] a a a a a}} do_execsql_test 5.3 { SELECT snippet(p1, 0, NULL, ']', '...', 6) FROM p1('x'); } {{x] a a a a a...}} do_execsql_test 5.4 { SELECT snippet(p1, 0, '[', NULL, '...', 6) FROM p1('x'); } {{[x a a a a a...}} do_execsql_test 5.5 { SELECT snippet(p1, 0, '[', NULL, '...', 6) FROM p1('x OR ""'); } {{[x a a a a a...}} do_execsql_test 5.6 { SELECT snippet(p1, 0, '[', NULL, '...', 6) FROM p1('x OR ' || x'DB'); } {{[x a a a a a...}} } ;# foreach_detail_mode finish_test |
Changes to ext/fts5/test/fts5ah.test.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 | # 2014 June 17 # # The author disclaims copyright to this source code. In place of # a legal notice, here is a blessing: # # May you do good and not evil. # May you find forgiveness for yourself and forgive others. # May you share freely, never taking more than you give. # #************************************************************************* # This file implements regression tests for SQLite library. The # focus of this script is testing the FTS5 module. # source [file join [file dirname [info script]] fts5_common.tcl] set testprefix fts5ah # If SQLITE_ENABLE_FTS5 is defined, omit this file. ifcapable !fts5 { finish_test | > | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 | # 2014 June 17 # # The author disclaims copyright to this source code. In place of # a legal notice, here is a blessing: # # May you do good and not evil. # May you find forgiveness for yourself and forgive others. # May you share freely, never taking more than you give. # #************************************************************************* # This file implements regression tests for SQLite library. The # focus of this script is testing the FTS5 module. # # TESTRUNNER: slow source [file join [file dirname [info script]] fts5_common.tcl] set testprefix fts5ah # If SQLITE_ENABLE_FTS5 is defined, omit this file. ifcapable !fts5 { finish_test |
︙ | ︙ |
Changes to ext/fts5/test/fts5ai.test.
︙ | ︙ | |||
47 48 49 50 51 52 53 54 55 56 57 | ROLLBACK TO one; COMMIT; } do_execsql_test 1.2 { INSERT INTO t1(t1) VALUES('integrity-check'); } } finish_test | > > > > > > | 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 | ROLLBACK TO one; COMMIT; } do_execsql_test 1.2 { INSERT INTO t1(t1) VALUES('integrity-check'); } do_execsql_test 1.3 { SAVEPOINT one; INSERT INTO t1 VALUES('v w x'); ROLLBACK TO one; } } finish_test |
Changes to ext/fts5/test/fts5ak.test.
︙ | ︙ | |||
139 140 141 142 143 144 145 146 147 148 149 | -- '[a b c d e]' SELECT highlight(ft, 0, '[', ']') FROM ft WHERE ft MATCH 'a+b+c AND c+d+e'; } { {[a b c] x [c d e]} {[a b c] [c d e]} {[a b c d e]} } } finish_test | > > > > > > > > | 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 | -- '[a b c d e]' SELECT highlight(ft, 0, '[', ']') FROM ft WHERE ft MATCH 'a+b+c AND c+d+e'; } { {[a b c] x [c d e]} {[a b c] [c d e]} {[a b c d e]} } do_execsql_test 3.2 { SELECT highlight(ft, 0, NULL, NULL) FROM ft WHERE ft MATCH 'a+b+c AND c+d+e'; } { {a b c x c d e} {a b c c d e} {a b c d e} } } finish_test |
Changes to ext/fts5/test/fts5aux.test.
︙ | ︙ | |||
271 272 273 274 275 276 277 | do_execsql_test 9.3 { SELECT rowid FROM t1('b:a AND b:b') ORDER BY rank; } { 9 10 } | > > | > > > > > > > > > | > > > > > > > > > > > > > > > > > > > > | 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 | do_execsql_test 9.3 { SELECT rowid FROM t1('b:a AND b:b') ORDER BY rank; } { 9 10 } #------------------------------------------------------------------------- # Test that aux. functions may not be used in aggregate queries. # reset_db do_execsql_test 10.0 { CREATE VIRTUAL TABLE t1 USING fts5(x, y, z); INSERT INTO t1 VALUES('a', 'one two', 1); INSERT INTO t1 VALUES('b', 'two three', 2); INSERT INTO t1 VALUES('c', 'three four', 1); INSERT INTO t1 VALUES('d', 'four five', 2); INSERT INTO t1 VALUES('e', 'five six', 1); INSERT INTO t1 VALUES('f', 'six seven', 2); } proc firstcol {cmd} { $cmd xColumnText 0 } sqlite3_fts5_create_function db firstcol firstcol do_execsql_test 10.1.1 { SELECT firstcol(t1) FROM t1 } {a b c d e f} do_execsql_test 10.1.2 { SELECT group_concat(x, '.') FROM t1 } {a.b.c.d.e.f} do_catchsql_test 10.1.3 { SELECT group_concat(firstcol(t1), '.') FROM t1 } {1 {unable to use function firstcol in the requested context}} do_catchsql_test 10.1.4 { SELECT group_concat(firstcol(t1), '.') FROM t1 GROUP BY rowid } {1 {unable to use function firstcol in the requested context}} finish_test |
Added ext/fts5/test/fts5circref.test.
> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 | # 2018 Dec 22 # # The author disclaims copyright to this source code. In place of # a legal notice, here is a blessing: # # May you do good and not evil. # May you find forgiveness for yourself and forgive others. # May you share freely, never taking more than you give. # #************************************************************************* # This file implements regression tests for SQLite library. The # focus of this script is testing the FTS5 module. # source [file join [file dirname [info script]] fts5_common.tcl] set testprefix fts5circref # If SQLITE_ENABLE_FTS5 is not defined, omit this file. ifcapable !fts5 { finish_test return } do_execsql_test 1.0 { CREATE VIRTUAL TABLE tt USING fts5(a); SELECT name FROM sqlite_master ORDER BY 1; } { tt tt_config tt_content tt_data tt_docsize tt_idx } db_save_and_close foreach {tn schema sql} { 1 { CREATE TRIGGER tr1 AFTER INSERT ON tt_config BEGIN SELECT * FROM tt; END; } { INSERT INTO tt(tt, rank) VALUES('usermerge', 4); } 2 { CREATE TRIGGER tr1 AFTER INSERT ON tt_docsize BEGIN SELECT * FROM tt; END; } { INSERT INTO tt(a) VALUES('one two three'); } 3 { CREATE TRIGGER tr1 AFTER INSERT ON tt_content BEGIN SELECT * FROM tt; END; } { INSERT INTO tt(a) VALUES('one two three'); } 4 { CREATE TRIGGER tr1 AFTER INSERT ON tt_data BEGIN SELECT * FROM tt; END; } { INSERT INTO tt(a) VALUES('one two three'); } 5 { CREATE TRIGGER tr1 AFTER INSERT ON tt_idx BEGIN SELECT * FROM tt; END; } { INSERT INTO tt(a) VALUES('one two three'); } } { db_restore_and_reopen do_execsql_test 1.1.$tn.1 $schema do_catchsql_test 1.1.$tn.2 $sql {1 {SQL logic error}} db close } finish_test |
Changes to ext/fts5/test/fts5colset.test.
︙ | ︙ | |||
77 78 79 80 81 82 83 84 85 86 | " $res } do_catchsql_test 4.1 { SELECT * FROM t1 WHERE rowid MATCH 'a' } {1 {unable to use function MATCH in the requested context}} } finish_test | > > > > > > > > > > > > > > > > > > > | 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 | " $res } do_catchsql_test 4.1 { SELECT * FROM t1 WHERE rowid MATCH 'a' } {1 {unable to use function MATCH in the requested context}} } #------------------------------------------------------------------------- # Confirm that the expression parser creates the same expression tree # for: # # {a b} : (abc AND def) # -{c d} : (abc AND def) # # Assuming that the table columns are (a, b, c, d). # do_execsql_test 5.1 { SELECT fts5_expr('abcd AND cdef'); } {{"abcd" AND "cdef"}} do_execsql_test 5.2 { SELECT fts5_expr('{a b} : (abcd AND cdef)', 'a', 'b', 'c', 'd'); } {{{a b} : "abcd" AND {a b} : "cdef"}} do_execsql_test 5.3 { SELECT fts5_expr('-{c d} : (abcd AND cdef)', 'a', 'b', 'c', 'd'); } {{{a b} : "abcd" AND {a b} : "cdef"}} finish_test |
Added ext/fts5/test/fts5concurrent.test.
> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 | # 2022 May 09 # # The author disclaims copyright to this source code. In place of # a legal notice, here is a blessing: # # May you do good and not evil. # May you find forgiveness for yourself and forgive others. # May you share freely, never taking more than you give. # #************************************************************************* # This file implements regression tests for SQLite library. The # focus of this script is testing the FTS5 module. # source [file join [file dirname [info script]] fts5_common.tcl] set testprefix fts5concurrent # If SQLITE_ENABLE_FTS5 is not defined, omit this file. ifcapable !fts5 { finish_test return } do_execsql_test 1.0 { CREATE VIRTUAL TABLE ft USING fts5(line, tokenize=trigram); } do_execsql_test 1.1 { BEGIN CONCURRENT; INSERT INTO ft VALUES( hex(randomblob(50)) ); COMMIT } {} do_execsql_test 1.2 { BEGIN CONCURRENT; WITH s(i) AS ( SELECT 1 UNION ALL SELECT i+1 FROM s WHERE i<50 ) INSERT INTO ft SELECT hex(randomblob(50)) FROM s; WITH s(i) AS ( SELECT 1 UNION ALL SELECT i+1 FROM s WHERE i<50 ) INSERT INTO ft SELECT hex(randomblob(50)) FROM s; WITH s(i) AS ( SELECT 1 UNION ALL SELECT i+1 FROM s WHERE i<50 ) INSERT INTO ft SELECT hex(randomblob(50)) FROM s; COMMIT; } finish_test |
Changes to ext/fts5/test/fts5content.test.
︙ | ︙ | |||
249 250 251 252 253 254 255 | SELECT name FROM sqlite_master; } {xx xx_data xx_idx xx_docsize xx_config} do_execsql_test 6.2 { DROP TABLE xx; SELECT name FROM sqlite_master; } {} | > > | > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 | SELECT name FROM sqlite_master; } {xx xx_data xx_idx xx_docsize xx_config} do_execsql_test 6.2 { DROP TABLE xx; SELECT name FROM sqlite_master; } {} #--------------------------------------------------------------------------- # Check that an fts5 table cannot be its own content table. # reset_db do_execsql_test 7.1.1 { CREATE VIRTUAL TABLE t1 USING fts5(a, c=t1 ); INSERT INTO t1( a ) VALUES('abc'); } do_catchsql_test 7.1.2 { SELECT * FROM t1; } {1 {recursively defined fts5 content table}} do_catchsql_test 7.1.3 { SELECT * FROM t1('abc'); } {1 {recursively defined fts5 content table}} do_catchsql_test 7.1.4 { SELECT count(*) FROM t1; } {1 {recursively defined fts5 content table}} do_catchsql_test 7.1.5 { SELECT * FROM t1('abc') ORDER BY rank; } {1 {recursively defined fts5 content table}} reset_db do_execsql_test 7.2.1 { CREATE VIRTUAL TABLE t1 USING fts5(a, c=t2 ); CREATE VIRTUAL TABLE t2 USING fts5(a, c=t1 ); INSERT INTO t1( a ) VALUES('abc'); } do_catchsql_test 7.2.2 { SELECT * FROM t1; } {1 {recursively defined fts5 content table}} do_catchsql_test 7.2.3 { SELECT * FROM t1('abc'); } {1 {recursively defined fts5 content table}} do_catchsql_test 7.2.4 { SELECT count(*) FROM t1; } {1 {recursively defined fts5 content table}} do_catchsql_test 7.2.5 { SELECT * FROM t1('abc') ORDER BY rank; } {1 {recursively defined fts5 content table}} finish_test |
Changes to ext/fts5/test/fts5corrupt3.test.
more than 10,000 changes
Added ext/fts5/test/fts5corrupt4.test.
> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 | # 2019 May 16 # # The author disclaims copyright to this source code. In place of # a legal notice, here is a blessing: # # May you do good and not evil. # May you find forgiveness for yourself and forgive others. # May you share freely, never taking more than you give. # #*********************************************************************** # # source [file join [file dirname [info script]] fts5_common.tcl] set testprefix fts5corrupt4 # If SQLITE_ENABLE_FTS5 is defined, omit this file. ifcapable !fts5 { finish_test return } sqlite3_fts5_may_be_corrupt 1 do_execsql_test 1.0 { CREATE VIRTUAL TABLE ttt USING fts5(a, b); INSERT INTO ttt VALUES('e ee eee e ee eee e ee eee', 'eee ee e e e ee eee ee ee'); INSERT INTO ttt SELECT a||a, b||b FROM ttt; INSERT INTO ttt SELECT a||a, b||b FROM ttt; } expr srand(1) proc mutate {blob i} { set o [expr {$i % [string length $blob]}] set a [string range $blob 0 $o-1] set b [string range $blob $o+1 end] set v [expr int(rand()*255) - 127] return "$a[binary format c $v]$b" } db func mutate mutate for {set j 1000} {$j <= 5000} {incr j 1000} { do_test 1.$j { for {set i 0} {$i < 1000} {incr i} { execsql { BEGIN; UPDATE ttt_data SET block = mutate(block, $i) WHERE id>10; } foreach sql { {SELECT snippet(ttt, -1, '.', '..', '[', ']'), * FROM ttt('e*')} {SELECT snippet(ttt, -1, '.', '..', '[', ']'), * FROM ttt('e* NOT ee*')} } { catch { execsql $sql } } execsql ROLLBACK } } {} } sqlite3_fts5_may_be_corrupt 0 finish_test |
Added ext/fts5/test/fts5corrupt5.test.
> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340 341 342 343 344 345 346 347 348 349 350 351 352 353 354 355 356 357 358 359 360 361 362 363 364 365 366 367 368 369 370 371 372 373 374 375 376 377 378 379 380 381 382 383 384 385 386 387 388 389 390 391 392 393 394 395 396 397 398 399 400 401 402 403 404 405 406 407 408 409 410 411 412 413 414 415 416 417 418 419 420 421 422 423 424 425 426 427 428 429 430 431 432 433 434 435 436 437 438 439 440 441 442 443 444 445 446 447 448 449 450 451 452 453 454 455 456 457 458 459 460 461 462 463 464 465 466 467 468 469 470 471 472 473 474 475 476 477 478 479 480 481 482 483 484 485 486 487 488 489 490 491 492 493 494 495 496 497 498 499 500 501 502 503 504 505 506 507 508 509 510 511 512 513 514 515 516 517 518 519 520 521 522 523 524 525 526 527 528 529 530 531 532 533 534 535 536 537 538 539 540 541 542 543 544 545 546 547 548 549 550 551 552 553 554 555 556 557 558 559 560 561 562 563 564 565 566 567 568 569 570 571 572 573 574 575 576 577 578 579 580 581 582 583 584 585 586 587 588 589 590 591 592 593 594 595 596 597 598 599 600 601 602 603 604 605 606 607 608 609 610 611 612 613 614 615 616 617 618 619 620 621 622 623 624 625 626 627 628 629 630 631 632 633 634 635 636 637 638 639 640 641 642 643 644 645 646 647 648 649 650 651 652 653 654 655 656 657 658 659 660 661 662 663 664 665 666 667 668 669 670 671 672 673 674 675 676 677 678 679 680 681 682 683 684 685 686 687 688 689 690 691 692 693 694 695 696 697 698 699 700 701 702 703 704 705 706 707 708 709 710 711 712 713 714 715 716 717 718 719 720 721 722 723 724 725 726 727 728 729 730 731 732 733 734 735 736 737 738 739 740 741 742 743 744 745 746 747 748 749 750 751 752 753 754 755 756 757 758 759 760 761 762 763 764 765 766 767 768 769 770 771 772 773 774 775 776 777 778 779 780 781 782 783 784 785 786 787 788 789 790 791 792 793 794 795 796 797 798 | # 2015 Apr 24 # # The author disclaims copyright to this source code. In place of # a legal notice, here is a blessing: # # May you do good and not evil. # May you find forgiveness for yourself and forgive others. # May you share freely, never taking more than you give. # #*********************************************************************** # # This file tests that FTS5 handles corrupt databases (i.e. internal # inconsistencies in the backing tables) correctly. In this case # "correctly" means without crashing. # source [file join [file dirname [info script]] fts5_common.tcl] set testprefix fts5corrupt3 # If SQLITE_ENABLE_FTS5 is defined, omit this file. ifcapable !fts5 { finish_test return } sqlite3_fts5_may_be_corrupt 1 database_may_be_corrupt #------------------------------------------------------------------------- # dbsqlfuzz crash-0f47112aa7520cf08c6a835a88fdff8c2a32a188 # reset_db do_test 1.0 { sqlite3 db {} db deserialize [decode_hexdb { .open --hexdb | size 24576 pagesize 4096 filename crash-0f47112aa7520c.db | page 1 offset 0 | 0: 53 51 4c 69 74 65 20 66 6f 72 6d 61 74 20 33 00 SQLite format 3. | 16: 10 00 01 01 00 40 20 20 00 00 00 00 00 00 00 00 .....@ ........ | 96: 00 00 00 00 0d 00 00 00 06 0e 0f 00 0f aa 0f 53 ...............S | 112: 0e e8 0e 8b 0e 33 0e 0f 00 00 00 00 00 00 00 00 .....3.......... | 3584: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 22 ................ | 3600: 06 06 17 11 11 01 31 74 61 62 6c 65 62 62 62 62 ......1tablebbbb | 3616: 06 43 52 45 41 54 45 20 54 41 42 4c 45 20 62 62 .CREATE TABLE bb | 3632: 28 61 29 56 05 06 17 1f 1f 01 7d 74 61 62 6c 65 (a)V.......table | 3648: 74 31 5f 63 6f 6e 66 69 67 74 31 5f 63 6f 6e 66 t1_configt1_conf | 3664: 69 67 05 43 52 45 41 54 45 20 54 41 42 4c 45 20 ig.CREATE TABLE | 3680: 27 74 31 5f 63 6f 6e 66 69 67 27 28 6b 20 50 52 't1_config'(k PR | 3696: 49 4d 41 52 59 20 4b 45 59 2c 20 76 29 20 57 49 IMARY KEY, v) WI | 3712: 54 48 4f 55 54 20 52 4f 57 49 44 5b 04 07 17 21 THOUT ROWID[...! | 3728: 21 01 81 01 74 61 62 6c 65 74 31 5f 64 6f 63 73 !...tablet1_docs | 3744: 69 7a 65 74 31 5f 64 6f 63 73 69 7a 65 04 43 52 izet1_docsize.CR | 3760: 45 41 54 45 20 54 41 42 4c 45 20 27 74 31 5f 64 EATE TABLE 't1_d | 3776: 6f 63 73 69 7a 65 27 28 69 64 20 49 4e 54 45 47 ocsize'(id INTEG | 3792: 45 52 20 50 52 49 4d 41 52 59 20 4b 45 59 2c 20 ER PRIMARY KEY, | 3808: 73 7a 20 42 4c 4f 42 29 69 03 07 17 19 19 01 81 sz BLOB)i....... | 3824: 2d 74 61 62 6c 65 74 31 5f 69 64 78 74 31 5f 69 -tablet1_idxt1_i | 3840: 64 78 03 43 52 45 41 54 45 20 54 41 42 4c 45 20 dx.CREATE TABLE | 3856: 27 74 31 5f 69 64 78 27 28 73 65 67 69 64 2c 20 't1_idx'(segid, | 3872: 74 65 72 6d 2c 20 70 67 6e 6f 2c 20 50 52 49 4d term, pgno, PRIM | 3888: 41 52 59 20 4b 45 59 28 73 65 67 69 64 2c 20 74 ARY KEY(segid, t | 3904: 65 72 6d 29 29 20 57 49 54 48 4f 55 54 20 52 4f erm)) WITHOUT RO | 3920: 57 49 44 55 02 07 17 1b 1b 01 81 01 74 61 62 6c WIDU........tabl | 3936: 65 74 31 5f 64 61 74 61 74 31 5f 64 61 74 61 02 et1_datat1_data. | 3952: 43 52 45 41 54 45 20 54 41 42 4c 45 20 27 74 31 CREATE TABLE 't1 | 3968: 5f 64 61 74 61 27 28 69 64 20 49 4e 54 45 47 45 _data'(id INTEGE | 3984: 52 20 50 52 49 4d 41 52 59 20 4b 45 59 2c 20 62 R PRIMARY KEY, b | 4000: 6c 6f 63 6b 20 42 4c 4f 42 29 54 01 07 17 11 11 lock BLOB)T..... | 4016: 08 81 15 74 61 62 6c 65 74 31 74 31 43 52 45 41 ...tablet1t1CREA | 4032: 54 45 20 56 49 52 54 55 41 4c 20 54 41 42 4c 45 TE VIRTUAL TABLE | 4048: 20 74 31 20 55 53 49 4e 47 20 66 74 73 35 28 61 t1 USING fts5(a | 4064: 2c 62 2c 70 72 65 66 69 78 3d 22 31 2c 32 2c 33 ,b,prefix=.1,2,3 | 4080: 2c 34 22 2c 20 63 6f 6e 74 65 6e 74 3d 22 22 29 ,4., content=..) | page 2 offset 4096 | 0: 0d 0b 6a 00 37 09 4c 02 0f e7 09 4c 0f c6 0f a4 ..j.7.L....L.... | 16: 0f 88 0f 6d 0f 4b 0f 2c 0f 0e 0e ec 0e cd 0e ad ...m.K.,........ | 32: 0e 8e 0e 6c 0e 4b 0e 29 0e 08 0d e6 0d c4 0d b5 ...l.K.)........ | 48: 0d 97 0d 76 0d 54 0d 31 0d 15 0c f3 0c d3 0c b5 ...v.T.1........ | 64: 0c 95 0c 73 0c 54 0c 32 0c 10 0b ee 0b cc 0b b0 ...s.T.2........ | 80: 0b 8d 0b 7e 0b 48 0b 2e 0b 0b 0a ef 00 00 00 00 ...~.H.......... | 2368: 00 00 00 00 00 00 00 00 00 00 00 00 15 0a 03 00 ................ | 2384: 30 00 00 00 01 01 03 35 00 03 01 11 12 02 01 12 0......5........ | 2400: 03 01 11 1c 8c 80 80 80 80 10 03 00 3e 00 00 00 ............>... | 2416: 17 01 05 05 34 74 61 62 6c 03 02 03 01 04 77 68 ....4tabl.....wh | 2432: 65 72 03 02 06 09 1b 8c 80 80 80 80 0f 03 00 3c er.............< | 2448: 00 00 00 16 05 34 66 74 73 34 03 02 02 01 04 6e .....4fts4.....n | 2464: 75 6d 62 03 06 01 04 09 1b 8c 80 80 80 80 0e 03 umb............. | 2480: 00 3c 00 00 00 16 04 33 74 68 65 03 06 01 01 04 .<.....3the..... | 2496: 01 03 77 68 65 03 02 04 04 0a 1b 8c 80 80 80 80 ..whe........... | 2512: 0d 03 00 3c 00 00 00 16 04 33 6e 75 6d 03 06 01 ...<.....3num... | 2528: 01 05 01 03 74 61 62 03 02 03 04 0a 19 8c 80 80 ....tab......... | 2544: 80 80 0c 03 00 38 00 00 00 14 03 32 77 68 03 02 .....8.....2wh.. | 2560: 04 00 04 33 66 74 73 03 02 02 04 07 18 8c 80 80 ...3fts......... | 2576: 80 80 0b 03 00 36 00 00 00 13 03 32 74 61 03 02 .....6.....2ta.. | 2592: 03 02 01 68 03 06 01 01 04 04 07 1b 8c 80 80 80 ...h............ | 2608: 80 0a 03 00 3c 00 00 00 16 03 32 6e 75 03 06 01 ....<.....2nu... | 2624: 01 05 01 02 6f 66 03 06 01 01 06 04 09 19 8c 80 ....of.......... | 2640: 80 80 80 09 03 00 38 00 00 00 14 03 32 66 74 03 ......8.....2ft. | 2656: 02 02 01 02 69 73 03 06 01 01 03 04 07 18 8c 80 ....is.......... | 2672: 80 80 80 08 03 00 36 00 00 00 13 02 31 74 03 08 ......6.....1t.. | 2688: 03 01 01 04 01 01 77 03 02 04 04 09 1a 8c 80 80 ......w......... | 2704: 80 80 07 03 00 3a 00 00 00 15 02 31 6e 03 08 01 .....:.....1n... | 2720: 01 02 05 01 00 6f 03 06 01 01 06 04 09 18 8c 80 .....o.......... | 2736: 80 80 80 06 03 00 36 00 00 00 03 04 02 31 66 03 ......6......1f. | 2752: 02 02 01 01 69 03 06 01 01 03 04 f6 1c 8c 80 80 ....i........... | 2768: 80 80 05 03 00 3e 00 00 00 17 04 30 74 68 65 03 .....>.....0the. | 2784: f6 01 01 04 01 05 77 68 65 72 65 03 02 04 0a 15 ......where..... | 2800: 8c 80 80 80 80 04 03 00 30 00 00 00 11 01 01 06 ........0....... | 2816: 06 30 74 61 62 6c 65 0f 42 03 07 1c 8c 81 80 80 .0table.B....... | 2832: 80 03 03 00 3e 00 00 00 17 07 30 6e 75 6d 62 65 ....>.....0numbe | 2848: 72 03 06 01 01 05 01 02 6f 66 03 06 04 0d 13 8c r.......of...... | 2864: 80 80 80 80 02 03 00 2c 00 00 00 0f 01 01 03 02 .......,........ | 2880: 30 6e 03 06 01 01 02 07 1b 8c 80 80 80 80 01 03 0n.............. | 2896: 00 3c 00 00 00 16 08 30 66 74 73 34 61 75 78 03 .<.....0fts4aux. | 2912: 02 02 01 02 69 73 03 06 04 0c 00 00 00 14 2a 00 ....is........*. | 2928: 00 00 01 01 02 24 00 02 01 01 12 02 01 12 08 88 .....$.......... | 2944: 80 80 80 80 12 03 00 16 00 00 00 05 02 1c 88 80 ................ | 2960: 80 80 80 11 03 00 3e 00 00 00 17 05 34 72 6f 77 ......>.....4row | 2976: 73 02 06 01 01 05 01 04 74 68 65 72 02 02 04 0b s.......ther.... | 2992: 15 88 80 80 80 80 10 03 00 30 00 00 00 11 02 01 .........0...... | 3008: 01 07 05 34 62 65 74 77 02 02 04 08 1b 88 80 80 ...4betw........ | 3024: 80 80 0f 03 00 3c 00 00 00 16 04 04 33 72 6f 77 .....<......3row | 3040: 02 06 01 01 05 01 03 74 68 65 02 08 05 0a 1b 88 .......the...... | 3056: 80 80 80 80 0e 03 00 3c 00 00 00 16 01 01 02 04 .......<........ | 3072: 33 61 72 65 02 02 03 01 03 62 65 74 02 02 07 08 3are.....bet.... | 3088: 1b 88 80 80 80 80 0d 03 00 3c 00 00 00 16 13 32 .........<.....2 | 3104: 74 68 02 08 02 01 01 07 00 04 33 61 6e 64 02 06 th........3and.. | 3120: 04 0a 1b 88 80 80 80 80 0c 03 00 3c 00 00 00 16 ...........<.... | 3136: 03 32 69 6e 02 06 01 01 06 01 02 72 6f 02 06 01 .2in.......ro... | 3152: 01 05 04 09 18 88 80 80 80 80 0b 03 00 36 00 00 .............6.. | 3168: 00 13 02 03 32 61 72 02 02 03 01 02 62 65 02 02 ....2ar.....be.. | 3184: 04 05 07 1b 88 80 80 80 80 0a 03 00 3c 00 9e 00 ............<... | 3200: 16 02 31 74 02 08 02 01 01 07 00 03 32 61 6e 02 ..1t........2an. | 3216: 06 01 01 04 09 19 88 80 80 80 80 09 03 00 38 00 ..............8. | 3232: 00 00 14 02 31 6e 02 06 01 01 03 01 01 72 02 06 ....1n.......r.. | 3248: 01 01 05 04 08 17 88 80 80 80 80 08 03 00 34 00 ..............4. | 3264: 00 00 12 02 31 62 02 02 04 01 01 69 02 06 01 01 ....1b.....i.... | 3280: 06 04 06 19 88 80 80 80 80 07 03 00 38 00 00 00 ............8... | 3296: 14 04 02 31 32 02 02 05 01 01 61 02 08 03 01 01 ...12.....a..... | 3312: 02 05 06 1b 88 80 80 80 80 06 03 00 3c 00 00 00 ............<... | 3328: 16 06 30 74 68 65 72 65 02 02 02 00 02 31 31 02 ..0there.....11. | 3344: 06 01 01 04 0a 15 88 80 80 80 80 05 03 00 30 00 ..............0. | 3360: 00 00 11 01 01 05 04 30 74 68 65 02 06 01 01 07 .......0the..... | 3376: 07 1c 88 80 80 80 80 04 03 00 3e 00 00 00 17 01 ..........>..... | 3392: 01 06 02 30 6e 02 06 01 01 03 01 04 72 6f 77 73 ...0n.......rows | 3408: 02 06 07 08 1b 88 80 80 80 80 03 03 00 3c 00 00 .............<.. | 3424: 00 16 08 30 62 65 74 77 65 65 6e 02 02 04 01 02 ...0between..... | 3440: 69 6e 02 06 04 0c 1a 88 80 80 80 80 02 03 00 3a in.............: | 3456: 00 00 00 15 04 30 61 6e 64 02 06 01 01 02 02 02 .....0and....... | 3472: 72 65 02 02 03 04 0a 17 88 80 80 80 80 01 03 00 re.............. | 3488: 34 00 00 00 12 02 30 31 02 06 01 01 04 01 01 32 4.....01.......2 | 3504: 02 02 05 04 08 08 84 80 80 80 80 12 03 00 16 00 ................ | 3520: 00 00 05 04 1b 84 80 80 80 80 11 03 00 3c 00 00 .............<.. | 3536: 00 16 05 34 74 61 62 6c 01 06 01 01 05 02 03 65 ...4tabl.......e | 3552: 72 6d 01 02 04 0b 1b 84 80 80 80 80 10 03 00 3c rm.............< | 3568: 00 00 00 16 05 34 65 61 63 68 01 02 03 01 04 70 .....4each.....p | 3584: 72 65 73 01 02 05 04 08 1a 84 80 80 80 80 0f 03 res............. | 3600: 00 3a 00 00 00 15 04 33 74 65 72 01 02 04 02 02 .:.....3ter..... | 3616: 68 65 01 06 01 01 03 04 08 1b 84 80 80 80 80 0e he.............. | 3632: 03 00 3c 00 00 00 16 04 33 80 72 65 01 02 05 01 ..<.....3.re.... | 3648: 03 74 61 62 01 06 01 01 05 04 08 1a 84 80 80 80 .tab............ | 3664: 80 0d 03 00 3a 00 00 00 15 04 33 66 6f 72 01 02 ....:.....3for.. | 3680: 02 02 02 74 73 01 06 01 01 04 04 08 1b 84 80 80 ...ts........... | 3696: 80 80 0c 03 00 3c 00 00 00 17 03 32 74 68 01 06 .....<.....2th.. | 3712: 01 01 03 00 04 33 65 61 63 01 02 03 04 09 18 84 .....3eac....... | 3728: 80 80 80 80 0b 03 00 36 00 00 00 13 03 32 74 61 .......6.....2ta | 3744: 01 06 01 01 05 02 01 65 01 02 04 04 09 19 84 80 .......e........ | 3760: 80 80 80 0a 03 00 38 00 00 00 14 03 32 69 6e 01 ......8.....2in. | 3776: 06 01 01 02 01 02 70 72 01 02 05 04 09 18 84 80 ......pr........ | 3792: 80 80 80 09 03 00 36 00 00 00 13 03 32 66 6f 01 ......6.....2fo. | 3808: 02 02 02 01 74 01 06 01 01 04 04 07 1b 84 80 80 ....t........... | 3824: 80 80 08 03 00 3c 00 00 00 16 02 31 74 01 0a 04 .....<.....1t... | 3840: 01 01 03 04 00 03 32 65 61 01 02 03 04 0a 17 84 ......2ea....... | 3856: 80 80 80 80 07 03 00 34 00 00 00 12 02 31 69 01 .......4.....1i. | 3872: 06 01 01 02 01 01 70 01 02 05 04 08 18 84 80 80 ......p......... | 3888: 80 80 06 03 00 36 00 00 00 13 02 31 65 01 02 03 .....6.....1e... | 3904: 01 01 66 01 08 02 01 01 04 04 06 1b 84 80 80 80 ..f............. | 3920: 80 05 03 00 3c 00 00 00 16 05 30 74 65 72 6d 01 ....<.....0term. | 3936: 02 04 02 02 68 65 01 06 01 01 03 04 09 14 84 80 ....he.......... | 3952: 80 80 80 04 03 00 2e 00 00 00 10 06 30 64 61 62 ............0dab | 3968: 6c 65 01 06 01 01 05 04 15 84 80 80 80 80 03 03 le.............. | 3984: 00 30 00 00 00 11 02 08 30 70 72 65 73 65 6e 74 .0......0present | 4000: 01 02 05 05 1b 84 80 80 80 80 02 03 00 3c 00 00 .............<.. | 4016: 00 16 04 30 66 74 73 01 06 01 01 04 01 02 69 6e ...0fts.......in | 4032: 01 06 01 01 04 0a 1a 84 80 80 80 80 01 03 00 3a ...............: | 4048: 00 00 00 15 05 30 65 61 63 68 01 02 03 01 13 66 .....0each.....f | 4064: 6f 72 01 02 02 04 09 06 01 03 00 12 03 0b 0f 00 or.............. | 4080: 00 08 8c 80 80 80 80 11 03 00 16 00 00 00 05 04 ................ | page 3 offset 8192 | 0: 0a 00 00 00 32 0e 4f 00 0f fa 0f f1 0f e9 0f e1 ....2.O......... | 16: 0f d8 0f d1 0f c9 0f c1 0f b9 0f b1 0f a9 0f a0 ................ | 32: 0f 98 0f 90 0f 87 0f 80 0f 78 0f 71 0f 68 0f 5f .........x.q.h._ | 48: 0f 56 0f 4d 0f 41 0f 38 0f 2f 0f 26 0f 1d 0f 13 .V.M.A.8./.&.... | 64: 0f 0a 0f 01 0e f7 0e ee 0e e6 0e dd 0e d6 0e cd ................ | 80: 0e c3 0e ba 0e 00 00 00 00 00 00 00 00 00 00 00 ................ | 3648: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 08 ................ | 3664: 04 01 10 01 03 34 74 20 07 04 01 0e 01 03 34 1e .....4t ......4. | 3680: 09 04 01 12 01 03 33 74 68 1c 08 04 01 10 01 03 ......3th....... | 3696: 33 6e 1a 08 04 01 10 01 03 32 77 18 08 04 01 10 3n.......2w..... | 3712: 01 03 32 74 16 08 04 01 10 01 03 32 6e 14 07 04 ..2t.......2n... | 3728: 01 0e 01 03 32 12 08 04 01 10 01 03 31 74 10 08 ....2.......1t.. | 3744: 04 01 10 01 03 31 6e 0e 07 04 01 0e 01 03 31 0c .....1n.......1. | 3760: 09 04 01 12 01 03 30 74 68 0a 08 04 01 10 01 03 ......0th....... | 3776: 30 74 08 09 04 01 12 01 03 30 6e 75 06 08 04 01 0t.......0nu.... | 3792: 10 01 03 30 6e 04 06 04 01 0c 01 03 02 08 04 01 ...0n........... | 3808: 10 01 02 34 72 22 07 04 01 0e 01 02 34 20 08 04 ...4r.......4 .. | 3824: 01 10 01 02 33 72 1e 09 04 01 12 01 02 33 61 72 ....3r.......3ar | 3840: 1c 08 04 01 10 01 02 32 74 1a 08 04 01 10 01 02 .......2t....... | 3856: 32 69 18 09 04 01 12 01 02 32 61 72 16 08 04 01 2i.......2ar.... | 3872: 10 01 02 31 74 14 08 04 01 10 01 02 31 6e 12 08 ...1t.......1n.. | 3888: 04 01 10 01 02 31 62 10 08 04 01 10 01 02 31 32 .....1b.......12 | 3904: 0e 0b 04 01 16 01 02 30 74 68 65 72 0c 08 04 01 .......0ther.... | 3920: 10 01 02 30 74 0a 08 04 01 10 01 02 30 6e 08 08 ...0t.......0n.. | 3936: 14 01 10 01 02 30 62 06 08 04 01 10 01 02 30 61 .....0b.......0a | 3952: 04 06 04 01 0c 01 02 02 07 04 09 10 01 34 74 22 .............4t. | 3968: 06 04 09 0e 01 34 20 08 04 09 12 01 33 74 65 1e .....4 .....3te. | 3984: 07 04 09 10 01 33 70 1c 07 04 09 10 01 33 66 1a .....3p......3f. | 4000: 08 04 09 12 01 32 74 68 18 07 04 09 10 01 32 74 .....2th......2t | 4016: 16 01 64 09 10 01 32 69 14 07 04 09 10 01 32 66 ..d...2i......2f | 4032: 12 07 04 09 10 01 31 74 10 07 04 09 10 01 31 69 ......1t......1i | 4048: 0e 06 04 09 0e 01 31 0c 08 04 09 12 01 30 74 65 ......1......0te | 4064: 0a 06 04 09 10 01 30 74 08 07 04 09 10 01 30 70 ......0t......0p | 4080: 06 08 04 09 12 00 00 00 00 00 00 00 00 00 00 00 ................ | page 4 offset 12288 | 4064: 00 00 00 00 00 00 00 00 00 00 00 05 03 03 00 10 ................ | 4080: 03 05 05 02 03 00 10 04 06 05 01 03 00 10 04 04 ................ | page 5 offset 16384 | 0: 0a 00 00 00 02 0f eb 00 0f eb 0f f4 00 00 00 00 ................ | 4064: 00 00 00 00 00 00 00 00 00 00 00 08 03 15 01 70 ...............p | 4080: 67 73 7a 18 0b 03 1b 01 76 65 72 73 69 6f 6e 04 gsz.....version. | page 6 offset 20480 | 0: 0d 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ | 4080: 00 00 03 03 02 01 03 03 02 02 01 00 00 00 00 00 ................ | end crash-0f47112aa7520c.db }] } {} do_catchsql_test 1.1 { SELECT * FROM t1('R*') WHERE (a,b)<=(current_date,0) ORDER BY rowid DESC; } {1 {database disk image is malformed}} #------------------------------------------------------------------------- # reset_db do_test 2.0 { sqlite3 db {} db deserialize [decode_hexdb { .open --hexdb | size 24576 pagesize 4096 filename sql047467.txt.db | page 1 offset 0 | 0: 53 51 4c 69 74 65 20 66 6f 72 6d 61 74 20 33 00 SQLite format 3. | 16: 10 00 01 01 00 40 20 20 00 00 00 00 00 00 00 00 .....@ ........ | 96: 00 00 00 00 0d 00 00 00 06 0e 0f 00 0f aa 0f 53 ...............S | 112: 0e e8 0e 8b 0e 33 0e 0f 01 00 00 00 00 00 00 00 .....3.......... | 3584: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 22 ................ | 3600: 06 06 17 11 11 01 31 74 61 62 6c 65 62 62 62 62 ......1tablebbbb | 3616: 06 43 52 45 41 54 45 20 54 41 42 4c 45 20 62 62 .CREATE TABLE bb | 3632: 28 61 29 56 05 06 17 1f 1f 01 7d 74 61 62 6c 65 (a)V.......table | 3648: 74 31 5f 63 6f 6e 66 69 67 74 31 5f 63 6f 6e 66 t1_configt1_conf | 3664: 69 67 05 43 52 45 41 54 45 20 54 41 42 4c 45 20 ig.CREATE TABLE | 3680: 27 74 31 5f 63 6f 6e 66 69 67 27 28 6b 20 50 52 't1_config'(k PR | 3696: 49 4d 41 52 59 20 4b 45 59 2c 20 76 29 20 57 49 IMARY KEY, v) WI | 3712: 54 48 4f 55 54 20 52 4f 57 49 44 5b 04 07 17 21 THOUT ROWID[...! | 3728: 21 01 81 01 74 61 62 6c 65 74 31 5f 64 6f 63 73 !...tablet1_docs | 3744: 69 7a 65 74 31 5f 64 6f 63 73 69 7a 65 04 43 52 izet1_docsize.CR | 3760: 45 41 54 45 20 54 41 42 4c 45 20 27 74 31 5f 64 EATE TABLE 't1_d | 3776: 6f 63 73 69 7a 65 27 28 69 64 20 49 4e 54 45 47 ocsize'(id INTEG | 3792: 45 52 20 50 52 49 4d 41 52 59 20 4b 45 59 2c 20 ER PRIMARY KEY, | 3808: 73 7a 20 42 4c 4f 42 29 69 03 07 17 19 19 01 81 sz BLOB)i....... | 3824: 2d 74 61 62 6c 65 74 31 5f 69 64 78 74 31 5f 69 -tablet1_idxt1_i | 3840: 64 78 03 43 52 45 41 54 45 20 54 41 42 4c 45 20 dx.CREATE TABLE | 3856: 27 74 31 5f 69 64 78 27 28 73 65 67 69 64 2c 20 't1_idx'(segid, | 3872: 74 65 72 6d 2c 20 70 67 6e 6f 2c 20 50 52 49 4d term, pgno, PRIM | 3888: 41 52 59 20 4b 45 59 28 73 65 67 69 64 2c 20 74 ARY KEY(segid, t | 3904: 65 72 6d 29 29 20 57 49 54 48 4f 55 54 20 52 4f erm)) WITHOUT RO | 3920: 57 49 44 55 02 07 17 1b 1b 01 81 01 74 61 62 6c WIDU........tabl | 3936: 65 74 31 5f 64 61 74 61 74 31 5f 64 61 74 61 02 et1_datat1_data. | 3952: 43 52 45 41 54 45 20 54 41 42 4c 45 20 27 74 31 CREATE TABLE 't1 | 3968: 5f 64 61 74 61 27 28 69 64 20 49 4e 54 45 47 45 _data'(id INTEGE | 3984: 52 20 50 52 49 4d 41 52 59 20 4b 45 59 2c 20 62 R PRIMARY KEY, b | 4000: 6c 6f 63 6b 20 42 4c 4f 42 29 54 01 07 17 11 11 lock BLOB)T..... | 4016: 08 81 15 74 61 62 6c 65 74 31 74 31 43 52 45 41 ...tablet1t1CREA | 4032: 54 45 20 56 49 52 54 55 41 4c 20 54 41 42 4c 45 TE VIRTUAL TABLE | 4048: 20 74 31 20 55 53 49 4e 47 20 66 74 73 35 28 61 t1 USING fts5(a | 4064: 2c 62 2c 70 72 65 66 69 78 3d 22 31 2c 32 2c 33 ,b,prefix=.1,2,3 | 4080: 2c 34 22 2c 20 63 6f 6e 74 65 6e 74 3d 22 22 29 ,4., content=..) | page 2 offset 4096 | 0: 0d 0b 6a 00 37 09 4c 02 0f e7 09 4c 0f c6 0f a4 ..j.7.L....L.... | 16: 0f 88 0f 6d 0f 4b 0f 2c 0f 0e 0e ec 0e cd 0e ad ...m.K.,........ | 32: 0e 8e 0e 6c 0e 4b 0e 29 0e 08 0d e6 0d c4 0d b5 ...l.K.)........ | 48: 0d 97 0d 76 0d 54 0d 31 0d 15 0c f3 0c d3 0c b5 ...v.T.1........ | 64: 0c 95 0c 73 0c 54 0c 32 0c 10 0b ee 0b cc 0b b0 ...s.T.2........ | 80: 0b 8d 0b 7e 0b 48 0b 2e 0b 0b 0a ef 00 00 00 00 ...~.H.......... | 2368: 00 00 00 00 00 00 00 00 00 00 00 00 15 0a 03 00 ................ | 2384: 30 00 00 00 01 01 03 35 00 03 01 11 12 02 01 12 0......5........ | 2400: 03 01 11 1c 8c 80 80 80 80 10 03 00 3e 00 00 00 ............>... | 2416: 17 01 05 05 34 74 61 62 6c 03 02 03 01 04 77 68 ....4tabl.....wh | 2432: 65 72 03 02 06 09 1b 8c 80 80 80 80 0f 03 00 3c er.............< | 2448: 00 00 00 16 05 34 66 74 73 34 03 02 02 01 04 6e .....4fts4.....n | 2464: 75 6d 62 03 06 01 04 09 1b 8c 80 80 80 80 0e 03 umb............. | 2480: 00 3b ff f0 00 16 04 33 74 68 65 03 06 01 01 04 .;.....3the..... | 2496: 01 03 77 68 65 03 02 04 04 0a 1b 8c 80 80 80 80 ..whe........... | 2512: 0d 03 00 3c 00 00 00 16 04 33 6e 75 6d 03 06 01 ...<.....3num... | 2528: 01 05 01 03 74 61 62 03 02 03 04 0a 19 8c 80 80 ....tab......... | 2544: 80 80 0c 03 00 38 00 00 00 14 03 32 77 68 03 02 .....8.....2wh.. | 2560: 04 00 04 33 66 74 73 03 02 02 04 07 18 8c 80 80 ...3fts......... | 2576: 80 80 0b 03 00 36 00 00 00 13 03 32 74 61 03 02 .....6.....2ta.. | 2592: 03 02 01 68 03 06 01 01 04 04 07 1b 8c 80 80 80 ...h............ | 2608: 80 0a 03 00 3c 00 00 00 16 03 32 6e 75 03 06 01 ....<.....2nu... | 2624: 01 05 01 02 6f 66 03 06 01 01 06 04 09 19 8c 80 ....of.......... | 2640: 80 80 80 09 03 00 38 00 00 00 14 03 32 66 74 03 ......8.....2ft. | 2656: 02 02 01 02 69 73 03 06 01 01 03 04 07 18 8c 80 ....is.......... | 2672: 80 80 80 08 03 00 36 00 00 00 13 02 31 74 03 08 ......6.....1t.. | 2688: 03 01 01 04 01 01 77 03 02 04 04 09 1a 8c 80 80 ......w......... | 2704: 80 80 07 03 00 3a 00 00 00 15 02 31 6e 03 08 01 .....:.....1n... | 2720: 01 02 05 01 00 6f 03 06 01 01 06 14 09 18 8c 80 .....o.......... | 2736: 80 80 80 06 03 00 36 00 00 00 03 04 02 31 66 03 ......6......1f. | 2752: 02 02 01 01 69 03 06 01 01 03 04 f6 1c 8c 80 80 ....i........... | 2768: 80 80 05 03 00 3e 00 00 00 17 04 30 74 68 65 03 .....>.....0the. | 2784: f6 01 01 04 01 05 77 68 65 72 65 03 02 04 0a 15 ......where..... | 2800: 8c 80 80 80 80 04 03 00 30 00 00 00 11 01 01 06 ........0....... | 2816: 06 30 74 61 62 6c 65 0f 42 03 07 1c 8c 81 80 80 .0table.B....... | 2832: 80 03 03 00 3e 00 00 00 17 07 30 6e 75 6d 62 65 ....>.....0numbe | 2848: 72 03 06 01 01 05 01 02 6f 66 03 06 04 0d 13 8c r.......of...... | 2864: 80 80 80 80 02 03 00 2c 00 00 00 0f 01 01 03 02 .......,........ | 2880: 30 6e 03 06 01 01 02 07 1b 8c 80 80 80 80 01 03 0n.............. | 2896: 00 3c 00 00 00 16 08 30 66 74 73 34 61 75 78 03 .<.....0fts4aux. | 2912: 02 02 01 02 69 73 03 06 04 0c 00 00 00 14 2a 00 ....is........*. | 2928: 00 00 01 01 02 24 00 02 01 01 12 02 01 12 08 88 .....$.......... | 2944: 80 80 80 80 12 03 00 16 00 00 00 05 02 1c 88 80 ................ | 2960: 80 80 80 11 03 00 3e 00 00 00 17 05 34 72 6f 77 ......>.....4row | 2976: 73 02 06 01 01 05 01 04 74 68 65 72 02 02 04 0b s.......ther.... | 2992: 15 88 80 80 80 80 10 03 00 30 00 00 00 11 02 01 .........0...... | 3008: 01 07 05 34 62 65 74 77 02 02 04 08 1b 88 80 80 ...4betw........ | 3024: 80 80 0f 03 00 3c 00 00 00 16 04 04 33 72 6f 77 .....<......3row | 3040: 02 06 01 01 05 01 03 74 68 65 02 08 05 0a 1b 88 .......the...... | 3056: 80 80 80 80 0e 03 00 3c 00 00 00 16 01 01 02 04 .......<........ | 3072: 33 61 72 65 02 02 03 01 03 62 65 74 02 02 07 08 3are.....bet.... | 3088: 1b 88 80 80 80 80 0d 03 00 3c 00 00 00 16 13 32 .........<.....2 | 3104: 74 68 02 08 02 01 01 07 00 04 33 61 6e 64 02 06 th........3and.. | 3120: 04 0a 1b 88 80 80 80 80 0c 03 00 3c 00 00 00 16 ...........<.... | 3136: 03 32 69 6e 02 06 01 01 06 01 02 72 6f 02 06 01 .2in.......ro... | 3152: 01 05 04 09 18 88 80 80 80 80 0b 03 00 36 00 00 .............6.. | 3168: 00 13 02 03 32 61 72 02 02 03 01 02 62 65 02 02 ....2ar.....be.. | 3184: 04 05 07 1b 88 80 80 80 80 0a 03 00 3c 00 94 50 ............<..P | 3200: 16 02 31 74 02 08 02 01 01 07 00 03 32 61 6e 02 ..1t........2an. | 3216: 06 01 01 04 09 19 88 80 80 80 80 09 03 00 38 00 ..............8. | 3232: 00 00 14 02 31 6e 02 06 01 01 03 01 01 72 02 06 ....1n.......r.. | 3248: 01 01 05 04 08 17 88 80 80 80 80 08 03 00 34 00 ..............4. | 3264: 00 00 12 02 31 62 02 02 04 01 01 69 02 06 01 01 ....1b.....i.... | 3280: 06 04 06 19 88 80 80 80 80 07 03 00 38 00 00 00 ............8... | 3296: 14 04 02 31 32 02 02 05 01 01 61 02 08 03 01 01 ...12.....a..... | 3312: 02 05 06 1b 88 80 80 80 80 06 03 00 3c 00 00 00 ............<... | 3328: 16 06 30 74 68 65 72 65 02 02 02 00 02 31 31 02 ..0there.....11. | 3344: 06 01 01 04 0a 15 88 80 80 80 80 05 03 00 30 00 ..............0. | 3360: 00 00 11 01 01 05 04 30 74 68 65 02 06 01 01 07 .......0the..... | 3376: 07 1c 88 80 80 80 80 04 03 00 3e 00 00 00 17 01 ..........>..... | 3392: 01 06 02 30 6e 02 06 01 01 03 01 04 72 6f 77 73 ...0n.......rows | 3408: 02 06 07 08 1b 88 80 80 80 80 03 03 00 3c 00 00 .............<.. | 3424: 00 16 08 30 62 65 74 77 65 65 6e 02 02 04 01 02 ...0between..... | 3440: 69 6e 02 06 04 0c 1a 88 80 80 80 80 02 03 00 3a in.............: | 3456: 00 00 00 15 04 30 61 6e 64 02 06 01 01 02 02 02 .....0and....... | 3472: 72 65 02 02 03 04 0a 17 88 80 80 80 80 01 03 00 re.............. | 3488: 34 00 00 00 12 02 30 31 02 06 01 01 04 01 01 32 4.....01.......2 | 3504: 02 02 05 04 08 08 84 80 80 80 80 12 03 00 16 00 ................ | 3520: 00 00 05 04 1b 84 80 80 80 80 11 03 00 3c 00 00 .............<.. | 3536: 00 16 05 34 74 61 62 6c 01 06 01 01 05 02 03 65 ...4tabl.......e | 3552: 72 6d 01 02 04 0b 1b 84 80 80 80 80 10 03 00 3c rm.............< | 3568: 00 00 00 16 05 34 65 61 63 68 01 02 03 01 04 70 .....4each.....p | 3584: 72 65 73 01 02 05 04 08 1a 84 80 80 80 80 0f 03 res............. | 3600: 00 3a 00 00 00 15 04 33 74 65 72 01 02 04 02 02 .:.....3ter..... | 3616: 68 65 01 06 01 01 03 04 08 1b 84 80 80 80 80 0e he.............. | 3632: 03 00 3c 00 00 00 16 04 33 80 72 65 01 02 05 01 ..<.....3.re.... | 3648: 03 74 61 62 01 06 01 01 05 04 08 1a 84 80 80 80 .tab............ | 3664: 80 0d 03 00 3a 00 00 00 15 04 33 66 6f 72 01 02 ....:.....3for.. | 3680: 02 02 02 74 73 01 06 01 01 04 04 08 1b 84 80 80 ...ts........... | 3696: 80 80 0c 03 00 3c 00 00 00 17 03 32 74 68 01 06 .....<.....2th.. | 3712: 01 01 03 00 04 33 65 61 63 01 02 03 04 09 18 84 .....3eac....... | 3728: 80 80 80 80 0b 03 00 36 00 00 00 13 03 32 74 61 .......6.....2ta | 3744: 01 06 01 01 05 02 01 65 01 02 04 04 09 19 84 80 .......e........ | 3760: 80 80 80 0a 03 00 38 00 00 00 14 03 32 69 6e 01 ......8.....2in. | 3776: 06 01 01 02 01 02 70 72 01 02 05 04 09 18 84 80 ......pr........ | 3792: 80 80 80 09 03 00 36 00 00 00 13 03 32 66 6f 01 ......6.....2fo. | 3808: 02 02 02 01 74 01 06 01 01 04 04 07 1b 84 80 80 ....t........... | 3824: 80 80 08 03 00 3c 00 00 00 16 02 31 74 01 0a 04 .....<.....1t... | 3840: 01 01 03 04 00 03 32 65 61 01 02 03 04 0a 17 84 ......2ea....... | 3856: 80 80 80 80 07 03 00 34 00 00 00 12 02 31 69 01 .......4.....1i. | 3872: 06 01 01 02 01 01 70 01 02 05 04 08 18 84 80 80 ......p......... | 3888: 80 80 06 03 00 36 00 00 00 13 02 31 65 01 02 03 .....6.....1e... | 3904: 01 01 66 01 08 02 01 01 04 04 06 1b 84 80 80 80 ..f............. | 3920: 80 05 03 00 3c 00 00 00 16 05 30 74 65 72 6d 01 ....<.....0term. | 3936: 02 04 02 02 68 65 01 06 01 01 03 04 09 14 84 80 ....he.......... | 3952: 80 80 80 04 03 00 2e 00 00 00 10 06 30 64 61 62 ............0dab | 3968: 6c 65 01 06 01 01 05 04 15 84 80 80 80 80 03 03 le.............. | 3984: 00 30 00 00 00 11 02 08 30 70 72 65 73 65 6e 74 .0......0present | 4000: 01 02 05 05 1b 84 80 80 80 80 02 03 00 3c 00 00 .............<.. | 4016: 00 16 04 30 66 74 73 01 06 01 01 04 01 02 69 6e ...0fts.......in | 4032: 01 06 01 01 04 0a 1a 84 80 80 80 80 01 03 00 3a ...............: | 4048: 00 00 00 15 05 30 65 61 63 68 01 02 03 01 13 66 .....0each.....f | 4064: 6f 72 01 02 02 04 09 06 01 03 00 12 03 0b 0f 00 or.............. | 4080: 00 08 8c 80 80 80 80 11 03 00 16 00 00 00 05 04 ................ | page 3 offset 8192 | 0: 0a 00 00 00 32 0e 4f 00 0f fa 0f f1 0f e9 0f e1 ....2.O......... | 16: 0f d8 0f d1 0f c9 0f c1 0f b9 0f b1 0f a9 0f a0 ................ | 32: 0f 98 0f 90 0f 87 0f 80 0f 78 0f 71 0f 68 0f 5f .........x.q.h._ | 48: 0f 56 0f 4d 0f 41 0f 38 0f 2f 0f 26 0f 1d 0f 13 .V.M.A.8./.&.... | 64: 0f 0a 0f 01 0e f7 0e ee 0e e6 0e dd 0e d6 0e cd ................ | 80: 0e c3 0e ba 0e 00 00 00 00 00 00 00 00 00 00 00 ................ | 3648: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 08 ................ | 3664: 04 01 10 01 03 34 74 20 07 04 01 0e 01 03 34 1e .....4t ......4. | 3680: 09 04 01 12 01 03 33 74 68 1c 08 04 01 10 01 03 ......3th....... | 3696: 33 6e 1a 08 04 01 10 01 03 32 77 18 08 04 01 10 3n.......2w..... | 3712: 01 03 32 74 16 08 04 01 10 01 03 32 6e 14 07 04 ..2t.......2n... | 3728: 01 0e 01 03 32 12 08 04 01 10 01 03 31 74 10 08 ....2.......1t.. | 3744: 04 01 10 01 03 31 6e 0e 07 04 01 0e 01 03 31 0c .....1n.......1. | 3760: 09 04 01 12 01 03 30 74 68 0a 08 04 01 10 01 03 ......0th....... | 3776: 30 74 08 09 04 01 12 01 03 30 6e 75 06 08 04 01 0t.......0nu.... | 3792: 10 01 03 30 6e 04 06 04 01 0c 01 03 02 08 04 01 ...0n........... | 3808: 10 01 02 34 72 22 07 04 01 0e 01 02 34 20 08 04 ...4r.......4 .. | 3824: 01 10 01 02 33 72 1e 09 04 01 12 01 02 33 61 72 ....3r.......3ar | 3840: 1c 08 04 01 10 01 02 32 74 1a 08 04 01 10 01 02 .......2t....... | 3856: 32 69 18 09 04 01 12 01 02 32 61 72 16 08 04 01 2i.......2ar.... | 3872: 10 01 02 31 74 14 08 04 01 10 01 02 31 6e 12 08 ...1t.......1n.. | 3888: 04 01 10 01 02 31 62 10 08 04 01 10 01 02 31 32 .....1b.......12 | 3904: 0e 0b 04 01 16 01 02 30 74 68 65 72 0c 08 04 01 .......0ther.... | 3920: 10 01 02 30 74 0a 08 04 01 10 01 02 30 6e 08 08 ...0t.......0n.. | 3936: 14 01 10 01 02 30 62 06 08 04 01 10 01 02 30 61 .....0b.......0a | 3952: 04 06 04 01 0c 01 02 02 07 04 09 10 01 34 74 22 .............4t. | 3968: 06 04 09 0e 01 34 20 08 04 09 12 01 33 74 65 1e .....4 .....3te. | 3984: 07 04 09 10 01 33 70 1c 07 04 09 10 01 33 66 1a .....3p......3f. | 4000: 08 04 09 12 01 32 74 68 18 07 04 09 10 01 32 74 .....2th......2t | 4016: 16 01 64 09 10 01 32 69 14 07 04 09 10 01 32 66 ..d...2i......2f | 4032: 12 07 04 09 10 01 31 74 10 07 04 09 10 01 31 69 ......1t......1i | 4048: 0e 06 04 09 0e 01 31 0c 08 04 09 12 01 30 74 65 ......1......0te | 4064: 0a 06 04 09 10 01 30 74 08 07 04 09 10 01 30 70 ......0t......0p | 4080: 06 08 04 09 12 00 00 00 00 00 00 00 00 00 00 00 ................ | page 4 offset 12288 | 4064: 00 00 00 00 00 00 00 00 00 00 00 05 03 03 00 10 ................ | 4080: 03 05 05 02 03 00 10 04 06 05 01 03 00 10 04 04 ................ | page 5 offset 16384 | 0: 0a 00 00 00 02 0f eb 00 0f eb 0f f4 00 00 00 00 ................ | 4064: 00 00 00 00 00 00 00 00 00 00 00 08 03 15 01 70 ...............p | 4080: 67 73 7a 18 0b 03 1b 01 76 65 72 73 69 6f 6e 04 gsz.....version. | page 6 offset 20480 | 0: 0d 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ | 4080: 00 00 03 03 02 01 03 03 02 02 01 00 00 00 00 00 ................ | end sql047467.txt.db }]} {} do_catchsql_test 2.1 { SELECT * FROM t1('R*R*R*R*') WHERE (a,b)<=(current_date,0) ORDER BY rowid DESC; } {1 {database disk image is malformed}} #------------------------------------------------------------------------- reset_db do_test 3.0 { sqlite3 db {} db deserialize [decode_hexdb { .open --hexdb | size 32768 pagesize 4096 filename crash-c69fcaceff1e50.db | page 1 offset 0 | 0: 53 51 4c 69 74 65 20 66 6f 72 6d 61 74 20 33 00 SQLite format 3. | 16: 10 00 01 01 00 40 20 20 00 00 00 00 00 00 00 08 .....@ ........ | 32: 00 00 00 02 00 00 00 01 00 00 00 09 00 00 00 04 ................ | 96: 00 00 00 00 0d 0f c7 00 07 0d 92 00 0f 8d 0f 36 ...............6 | 112: 0e cb 0e 6b 0e 0e 0d b6 0d 92 0d 92 00 00 00 00 ...k............ | 3472: 00 00 22 08 06 17 11 11 01 31 74 61 62 6c 65 74 .........1tablet | 3488: 32 74 32 08 43 52 45 41 54 45 20 54 41 42 4c 45 2t2.CREATE TABLE | 3504: 20 74 32 28 78 29 56 07 06 17 1f 1f 01 7d 74 61 t2(x)V.......ta | 3520: 62 6c 65 74 31 5f 63 6f 6e 66 69 67 74 31 5f 63 blet1_configt1_c | 3536: 6f 6e 66 69 67 07 43 52 45 41 54 45 20 54 41 42 onfig.CREATE TAB | 3552: 4c 45 20 27 74 31 5f 63 6f 6e 66 69 67 27 28 6b LE 't1_config'(k | 3568: 20 50 52 49 4d 41 52 59 20 4b 45 59 2c 20 76 29 PRIMARY KEY, v) | 3584: 20 57 49 54 48 4f 55 54 20 52 4f 57 49 44 5b 06 WITHOUT ROWID[. | 3600: 07 17 21 21 01 81 01 74 61 62 6c 65 74 31 5f 64 ..!!...tablet1_d | 3616: 6f 63 73 69 7a 65 74 31 5f 64 6f 63 73 69 7a 65 ocsizet1_docsize | 3632: 06 43 52 45 41 54 45 20 54 41 42 4c 45 20 27 74 .CREATE TABLE 't | 3648: 31 5f 64 6f 63 73 69 7a 65 27 28 69 64 20 49 4e 1_docsize'(id IN | 3664: 54 45 47 45 52 20 50 52 49 4d 41 52 59 20 4b 45 TEGER PRIMARY KE | 3680: 59 2c 20 73 7a 20 42 4c 4f 42 29 5e 05 07 17 21 Y, sz BLOB)^...! | 3696: 21 01 81 07 74 61 62 6c 65 74 31 5f 63 6f 6e 74 !...tablet1_cont | 3712: 65 6e 74 74 31 5f 63 6f 6e 74 65 6e 74 05 43 52 entt1_content.CR | 3728: 45 41 54 45 20 54 41 42 4c 45 20 27 74 31 5f 63 EATE TABLE 't1_c | 3744: 6f 6e 74 65 6e 74 27 28 69 64 20 49 4e 54 45 47 ontent'(id INTEG | 3760: 45 52 20 50 52 49 4d 41 52 59 20 4b 45 59 2c 20 ER PRIMARY KEY, | 3776: 63 30 2c 20 63 31 2c 20 63 32 29 69 04 07 17 19 c0, c1, c2)i.... | 3792: 19 01 81 2d 74 61 62 6c 65 74 31 5f 69 64 78 74 ...-tablet1_idxt | 3808: 31 5f 69 64 78 04 43 52 45 41 54 45 20 54 41 42 1_idx.CREATE TAB | 3824: 4c 45 20 27 74 31 5f 69 64 78 27 28 73 65 67 69 LE 't1_idx'(segi | 3840: 64 2c 20 74 65 72 6d 2c 20 70 67 6e 6f 2c 20 50 d, term, pgno, P | 3856: 52 49 4d 41 52 59 20 4b 45 59 28 73 65 67 69 64 RIMARY KEY(segid | 3872: 2c 20 74 65 72 6d 29 29 20 57 49 54 48 4f 55 54 , term)) WITHOUT | 3888: 20 52 4f 57 49 44 55 03 07 17 1b 1b 01 81 01 74 ROWIDU........t | 3904: 61 62 6c 65 74 31 5f 64 61 74 61 74 31 5f 64 61 ablet1_datat1_da | 3920: 74 61 03 43 52 45 41 54 45 20 54 41 42 4c 45 20 ta.CREATE TABLE | 3936: 27 74 31 5f 64 61 74 61 27 28 69 64 20 49 4e 54 't1_data'(id INT | 3952: 45 47 45 52 20 50 52 49 4d 41 52 59 20 4b 45 59 EGER PRIMARY KEY | 3968: 2c 20 62 6c 6f 63 6b 20 42 4c 4f 42 29 38 02 06 , block BLOB)8.. | 3984: 17 11 11 08 5f 74 61 62 6c 65 74 31 74 31 43 52 ...._tablet1t1CR | 4000: 45 41 54 45 20 56 49 52 54 55 41 4c 20 54 41 42 EATE VIRTUAL TAB | 4016: 4c 45 20 74 31 20 55 53 49 4e 47 20 66 74 73 35 LE t1 USING fts5 | 4032: 28 61 2c 62 2c 63 29 00 00 00 00 00 00 00 00 00 (a,b,c)......... | page 3 offset 8192 | 0: 0d 00 00 00 03 0c 94 00 0f e6 0f ef 0c 94 00 00 ................ | 3216: 00 00 00 00 86 4a 84 80 80 80 80 01 04 00 8d 18 .....J.......... | 3232: 00 00 01 bb 02 30 30 01 02 06 01 02 06 01 02 06 .....00......... | 3248: 1f 02 03 01 02 03 01 02 03 01 08 32 30 31 36 30 ...........20160 | 3264: 36 30 39 01 02 07 01 02 07 01 02 07 01 01 34 01 609...........4. | 3280: 02 05 01 02 05 01 02 05 01 01 35 01 02 04 01 02 ..........5..... | 3296: 04 01 02 04 02 07 30 30 30 30 30 30 30 1c 02 04 ......0000000... | 3312: 01 02 04 01 02 04 01 06 62 69 6e 61 72 79 03 06 ........binary.. | 3328: 01 02 02 03 06 01 02 02 03 06 01 02 02 03 06 01 ................ | 3344: 02 02 03 06 01 02 02 03 06 01 02 02 03 06 01 02 ................ | 3360: 02 03 06 01 02 02 03 06 01 02 02 03 06 01 02 02 ................ | 3376: 03 06 01 02 02 03 06 01 02 02 01 08 63 6f 6d 70 ............comp | 3392: 69 6c 65 72 01 02 02 01 02 02 01 02 02 01 06 64 iler...........d | 3408: 62 73 74 61 74 07 02 03 01 02 03 01 02 03 02 04 bstat........... | 3424: 65 62 75 67 04 02 02 01 02 02 01 02 02 01 06 65 ebug...........e | 3440: 6b b1 62 6c 65 07 02 02 01 02 02 01 02 02 01 02 k.ble........... | 3456: 02 01 02 02 01 02 02 01 02 02 01 02 02 01 02 02 ................ | 3472: 01 02 02 01 02 02 05 02 02 01 02 02 01 02 02 01 ................ | 3488: 02 02 01 02 02 01 02 02 01 02 02 01 02 02 01 02 ................ | 3504: 02 01 02 02 02 08 78 74 65 6e 73 69 6f 6e 1f 02 ......xtension.. | 3520: 04 01 02 04 01 02 04 01 04 66 74 73 34 1a 02 03 .........fts4... | 3536: 01 02 03 01 02 03 04 01 35 0d 02 03 01 02 03 01 ........5....... | 3552: 02 03 01 03 67 63 63 01 02 03 01 02 03 01 02 03 ....gcc......... | 3568: 02 06 65 6f 70 6f 6c 79 10 02 03 01 02 03 01 02 ..eopoly........ | 3584: 03 01 05 6a 73 6f 6e 31 13 02 03 01 02 03 01 02 ...json1........ | 3600: 03 01 04 6c 6f 61 64 1f 02 03 01 02 03 01 02 03 ...load......... | 3616: 01 03 6d 61 78 1c 02 02 01 02 02 01 02 02 02 05 ..max........... | 3632: 65 6d 6f 72 79 1c 02 03 01 02 03 01 02 03 04 04 emory........... | 3648: 73 79 73 35 16 02 03 01 02 03 01 02 03 01 06 6e sys5...........n | 3664: 6f 63 61 73 65 02 06 01 02 02 03 06 01 02 02 03 ocase........... | 3680: 06 01 02 02 03 06 01 02 02 03 06 01 02 02 03 06 ................ | 3696: 01 02 02 03 06 01 02 02 03 06 01 02 02 03 06 01 ................ | 3712: 02 02 03 06 01 02 02 03 06 01 02 02 03 06 01 02 ................ | 3728: 02 01 04 6f 6d 69 74 1f 02 02 01 02 02 01 02 01 ...omit......... | 3744: ff ff ff ff ff ff ff ff ff ff ff ff ff ff f1 02 ................ | 3760: 58 81 96 4d 01 06 01 02 02 03 06 01 02 02 03 06 X..M............ | 3776: 01 02 02 03 06 01 02 02 03 06 01 02 02 03 06 01 ................ | 3792: 02 02 03 06 01 02 02 03 06 01 02 02 03 06 01 02 ................ | 3808: 02 03 06 01 02 02 03 06 01 02 02 03 06 01 02 02 ................ | 3824: 01 0a 74 68 72 65 61 64 73 61 66 65 22 02 02 01 ..threadsafe.... | 3840: 02 02 01 02 02 01 04 76 74 61 62 07 02 04 01 02 .......vtab..... | 3856: 04 01 02 04 01 01 78 01 06 01 01 02 01 06 01 01 ......x......... | 3872: 02 01 06 01 c6 02 01 06 01 01 02 01 06 01 01 02 ................ | 3888: 01 06 01 01 02 01 06 01 01 02 01 06 01 01 02 01 ................ | 3904: 06 01 01 02 00 f6 01 01 02 01 06 01 01 02 01 06 ................ | 3920: 01 01 02 01 06 01 01 02 01 06 01 01 02 01 06 01 ................ | 3936: 01 02 01 06 01 01 02 01 06 01 01 02 01 06 01 01 ................ | 3952: 02 01 06 01 01 02 01 06 01 01 02 01 06 01 01 02 ................ | 3968: 01 06 01 01 02 01 06 01 01 02 01 06 01 01 02 01 ................ | 3984: 06 01 01 02 01 06 01 01 02 01 06 01 01 02 01 06 ................ | 4000: 01 01 02 01 06 01 01 02 01 06 01 01 02 01 06 01 ................ | 4016: 01 02 01 06 01 01 02 01 06 01 01 02 01 06 01 01 ................ | 4032: 02 01 06 01 01 02 01 06 01 01 02 04 15 13 0c 0c ................ | 4048: 12 44 13 11 0f 47 13 0f 0c 0e 11 10 0f 0e 10 0f .D...G.......... | 4064: 44 0f 10 40 15 0f 07 01 03 00 14 24 5a 24 24 0f D..@.......$Z$$. | 4080: 0a 03 00 24 00 00 00 00 01 01 01 00 01 01 01 01 ...$............ | page 4 offset 12288 | 0: 0a 00 00 00 01 0f fa 00 0f fa 00 00 00 00 00 00 ................ | 4080: 00 00 00 00 00 00 00 00 00 00 05 04 09 0c 01 02 ................ | page 7 offset 24576 | 0: 0a 00 00 00 01 0f f4 00 0f f4 00 00 00 00 00 00 ................ | 4080: 00 00 00 00 0b 03 1b 01 76 65 72 73 69 6f 6e 04 ........version. | end crash-c69fcaceff1e50.db }]} {} do_catchsql_test 3.1 { UPDATE t1 SET b=quote(zeroblob(200)) WHERE a MATCH 'thra*T'; } {1 {database disk image is malformed}} #------------------------------------------------------------------------- reset_db do_test 4.0 { sqlite3 db {} db deserialize [decode_hexdb { .open --hexdb | size 24576 pagesize 4096 filename crash-ef6738247b1344.db | page 1 offset 0 | 0: 53 51 4c 69 74 65 20 66 6f 72 6d 61 74 20 33 00 SQLite format 3. | 16: 10 00 01 01 00 40 20 20 00 10 00 06 40 00 00 06 .....@ ....@... | 32: 00 00 00 00 00 00 00 00 00 00 00 06 00 00 00 00 ................ | 96: 00 00 00 00 0d 00 00 00 06 0e 0f 00 0f aa 0f 53 ...............S | 112: 0e e8 0e 8b 0e 33 0e 0f 00 00 00 00 00 00 00 00 .....3.......... | 3584: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 22 ................ | 3600: 06 06 17 11 11 01 31 74 61 62 6c 65 62 62 62 62 ......1tablebbbb | 3616: 06 43 52 45 41 54 45 20 54 41 42 4c 45 20 62 62 .CREATE TABLE bb | 3632: 28 61 29 56 05 06 17 1f 1f 01 7d 74 61 62 6c 65 (a)V.......table | 3648: 74 31 5f 63 6f 6e 66 69 67 74 31 5f 63 6f 6e 66 t1_configt1_conf | 3664: 69 67 05 43 52 45 41 54 45 20 54 41 42 4c 45 20 ig.CREATE TABLE | 3680: 27 74 31 5f 63 6f 6e 66 69 67 27 28 6b 20 50 52 't1_config'(k PR | 3696: 49 4d 41 52 59 20 4b 45 59 2c 20 76 29 20 57 49 IMARY KEY, v) WI | 3712: 54 48 4f 55 54 20 52 4f 57 49 44 5b 04 07 17 21 THOUT ROWID[...! | 3728: 21 01 81 01 74 61 62 6c 65 74 31 5f 64 6f 63 73 !...tablet1_docs | 3744: 69 7a 65 74 31 5f 64 6f 63 73 69 7a 65 04 43 52 izet1_docsize.CR | 3760: 45 41 54 45 20 54 41 42 4c 45 20 27 74 31 5f 64 EATE TABLE 't1_d | 3776: 6f 63 73 69 7a 65 27 28 69 64 20 49 4e 54 45 47 ocsize'(id INTEG | 3792: 45 52 20 50 52 49 4d 41 52 59 20 4b 45 59 2c 20 ER PRIMARY KEY, | 3808: 73 7a 20 42 4c 4f 42 29 69 03 07 17 19 19 01 81 sz BLOB)i....... | 3824: 2d 74 61 62 6c 65 74 31 5f 69 64 78 74 31 5f 69 -tablet1_idxt1_i | 3840: 64 78 03 43 52 45 41 54 45 20 54 41 42 4c 45 20 dx.CREATE TABLE | 3856: 27 74 31 5f 69 64 78 27 28 73 65 67 69 64 2c 20 't1_idx'(segid, | 3872: 74 65 72 6d 2c 20 6f 67 6e 6f 2c 20 50 52 49 4d term, ogno, PRIM | 3888: 41 52 59 20 4b 45 59 28 73 65 67 69 64 2c 20 74 ARY KEY(segid, t | 3904: 65 72 6d 29 29 20 57 49 54 48 4f 55 54 20 52 4f erm)) WITHOUT RO | 3920: 57 49 44 55 35 07 17 1b 1b 01 81 01 74 61 62 6c WIDU5.......tabl | 3936: 65 74 31 5f 64 61 74 61 74 31 5f 64 61 74 61 02 et1_datat1_data. | 3952: 43 52 45 41 54 45 20 54 41 42 4c 45 20 27 74 31 CREATE TABLE 't1 | 3968: 5f 64 61 74 61 27 28 69 64 20 49 4e 54 45 47 45 _data'(id INTEGE | 3984: 52 20 50 52 49 4d 41 52 59 20 4b 45 59 2c 20 62 R PRIMARY KEY, b | 4000: 6c 6f 63 6b 20 42 4c 4f 42 29 54 01 07 17 11 11 lock BLOB)T..... | 4016: 08 81 15 74 61 62 6c 65 74 31 74 31 43 52 45 41 ...tablet1t1CREA | 4032: 54 45 20 56 49 52 54 55 41 4c 20 54 41 42 4c 45 TE VIRTUAL TABLE | 4048: 20 74 31 20 55 53 49 4e 47 20 66 74 73 35 28 61 t1 USING fts5(a | 4064: 2c 62 2c 70 72 65 66 69 78 3d 22 31 2c 32 2c 33 ,b,prefix=.1,2,3 | 4080: 2c 34 22 2c 20 63 6f 6e 74 65 6e 74 3d 22 22 29 ,4., content=..) | page 2 offset 4096 | 0: 0d 0b 6a 00 37 09 4c 02 0f e7 09 4c 0f c6 0f a4 ..j.7.L....L.... | 16: 0f 88 0f 6d 0f 4b 0f 2c 0f 0e 0e ec 0e cd 0e ad ...m.K.,........ | 32: 0e 8e 0e 6c 0e 4b 0e 29 0e 08 0d e6 0d c4 0d b5 ...l.K.)........ | 48: 0d 97 0d 76 0d 54 0d 31 0d 15 0c f3 0c d3 0c b5 ...v.T.1........ | 64: 0c 95 0c 73 0c 54 0c 32 0c 10 0b ee 0b cc 0b b0 ...s.T.2........ | 80: 0b 8d 0b 7e 0b 48 0b 2e 0b 0b 0a ef 0a cc 0a 00 ...~.H.......... | 2368: 00 00 00 00 00 00 00 00 00 00 00 00 15 0a 03 00 ................ | 2384: 30 00 00 00 01 01 03 35 00 03 01 01 12 02 01 12 0......5........ | 2400: 03 01 11 1c 8c 80 80 80 80 10 02 9c 3e 00 00 00 ............>... | 2416: 17 01 05 05 34 74 61 62 6c 03 02 03 01 04 77 68 ....4tabl.....wh | 2432: 65 72 03 02 06 09 1b 8c 80 80 80 80 0f 03 00 3c er.............< | 2448: 00 00 00 16 05 34 66 74 73 34 03 02 02 01 04 6e .....4fts4.....n | 2464: 75 6d 62 03 06 01 04 09 1b 8c 80 80 80 80 0e 03 umb............. | 2480: 00 3c 00 00 00 16 04 33 74 68 65 03 06 01 01 04 .<.....3the..... | 2496: 01 03 77 68 65 03 02 04 04 0a 1b 8c 80 80 80 80 ..whe........... | 2512: 0d 03 00 3c 00 00 00 16 04 33 6e 75 6d 03 06 01 ...<.....3num... | 2528: 01 05 01 03 74 61 62 03 02 03 04 0a 19 8c 80 80 ....tab......... | 2544: 80 80 0c 03 00 38 00 00 00 14 03 32 77 68 03 02 .....8.....2wh.. | 2560: 04 00 04 33 66 74 73 03 12 02 04 07 18 8c 80 80 ...3fts......... | 2576: 80 80 0b 03 00 36 00 00 00 13 03 32 74 61 03 02 .....6.....2ta.. | 2592: 03 02 01 68 03 06 01 01 04 04 07 1b 8c 80 80 80 ...h............ | 2608: 80 0a 03 00 3c 00 00 00 16 03 32 6e 75 03 06 01 ....<.....2nu... | 2624: 01 05 01 02 6f 66 03 06 01 01 06 04 09 19 8c 80 ....of.......... | 2640: 82 d0 80 09 03 00 38 00 00 00 14 03 32 66 74 03 ......8.....2ft. | 2656: 02 02 01 02 69 73 03 06 01 01 03 04 07 18 8c 80 ....is.......... | 2672: 80 80 80 08 03 00 36 00 00 00 13 02 31 74 03 08 ......6.....1t.. | 2688: 03 01 01 04 01 01 77 03 02 04 04 09 1a 8c 80 80 ......w......... | 2704: 80 80 07 03 00 3a 00 00 00 15 02 31 6e 03 08 01 .....:.....1n... | 2720: 01 02 05 01 01 6f 03 06 01 01 06 04 09 18 8c 80 .....o.......... | 2736: 81 80 80 06 03 00 36 00 00 00 13 04 12 31 66 03 ......6......1f. | 2752: 02 02 01 01 69 03 06 01 01 03 05 06 1c 8c 80 80 ....i........... | 2768: 80 80 05 03 00 3e 00 00 00 17 04 30 74 68 65 03 .....>.....0the. | 2784: 06 01 01 04 01 05 77 68 65 72 65 03 02 04 0a 15 ......where..... | 2800: 8c 80 65 80 80 04 03 00 30 00 00 00 11 01 01 06 ..e.....0....... | 2816: 06 30 74 61 62 6c 65 03 02 03 07 1c 8c 80 80 80 .0table......... | 2832: 80 03 03 00 3e 00 00 00 17 07 30 6e 75 6d 62 65 ....>.....0numbe | 2848: 72 03 06 01 01 05 01 02 6f 66 03 06 04 0d 13 8c r.......of...... | 2864: 80 80 80 80 02 03 00 2c 00 00 00 0f fc 01 03 02 .......,........ | 2880: 30 6e 03 06 01 00 f2 07 1b 8c 80 80 80 80 01 03 0n.............. | 2896: 00 3c 00 00 00 16 08 30 66 74 73 34 61 75 78 03 .<.....0fts4aux. | 2912: 03 02 01 02 69 73 03 06 04 0c 00 00 00 18 ea 00 ....is.......... | 2928: 00 00 01 01 02 24 00 02 01 01 12 02 01 12 08 88 .....$.......... | 2944: 80 80 80 80 12 03 00 12 10 00 00 05 02 1c 88 80 ................ | 2960: 80 80 80 11 03 00 3e 00 00 00 17 05 34 72 6f 77 ......>.....4row | 2976: 73 02 06 01 01 05 01 04 74 68 65 72 02 02 04 0b s.......ther.... | 2992: 15 88 80 80 80 80 10 03 00 2f ff ff f0 11 02 01 ........./...... | 3008: 01 07 05 34 62 65 74 77 02 02 04 08 1b 88 80 80 ...4betw........ | 3024: 80 80 0f cf 00 3c 00 00 00 16 04 04 33 72 6f 77 .....<......3row | 3040: 02 06 01 01 05 01 03 74 68 65 02 08 05 0a 1b 88 .......the...... | 3056: 80 80 80 80 0d 03 00 3c 00 00 00 16 00 01 02 04 .......<........ | 3072: 33 61 72 65 02 02 03 01 03 62 65 74 02 02 07 08 3are.....bet.... | 3088: 1b 88 80 80 80 80 0d 03 00 3c 00 00 00 16 03 32 .........<.....2 | 3104: 74 68 02 08 02 01 01 07 00 04 33 61 6e 64 02 06 th........3and.. | 3120: 04 0a 1b 88 80 80 80 80 0c 03 00 3c 00 00 00 16 ...........<.... | 3136: 03 32 69 6e 02 06 01 01 06 01 02 72 6f 02 06 01 .2in.......ro... | 3152: 01 05 04 09 18 88 80 80 80 80 0b 03 00 36 00 00 .............6.. | 3168: 00 13 02 03 32 61 72 02 02 03 01 02 62 65 02 02 ....2ar.....be.. | 3184: 04 05 07 1b 88 80 80 80 80 0a 03 00 3c 00 00 00 ............<... | 3200: 16 12 31 74 02 08 02 01 01 07 00 03 32 61 6e 02 ..1t........2an. | 3216: 06 01 01 04 09 19 88 80 80 80 80 09 03 00 38 00 ..............8. | 3232: 00 00 14 02 31 6e 02 06 01 01 03 01 01 72 02 06 ....1n.......r.. | 3248: 01 01 05 04 08 17 78 80 80 80 80 08 03 00 34 10 ......x.......4. | 3264: 01 00 12 02 31 62 02 02 04 01 01 69 02 06 01 01 ....1b.....i.... | 3280: 06 04 06 19 88 80 80 80 80 07 03 00 38 00 00 00 ............8... | 3296: 14 04 02 31 32 02 02 05 01 01 61 02 08 03 01 01 ...12.....a..... | 3312: 02 05 06 1b 88 80 80 80 80 06 03 00 3c 00 00 00 ............<... | 3328: 16 06 30 74 68 65 72 65 e7 02 02 00 02 31 31 02 ..0there.....11. | 3344: 06 01 01 04 0a 15 88 80 80 80 80 05 03 00 30 00 ..............0. | 3360: 00 00 11 01 01 05 e5 30 74 68 65 02 06 01 01 07 .......0the..... | 3376: 07 1c 88 80 80 80 80 04 03 00 3e 00 00 00 17 01 ..........>..... | 3392: 01 06 02 30 6e 02 06 01 01 03 01 04 72 6f 77 73 ...0n.......rows | 3408: 02 06 07 08 1b 88 80 80 80 80 03 03 00 3c 00 00 .............<.. | 3424: 00 16 08 30 62 65 74 77 65 65 6e 02 02 04 01 02 ...0between..... | 3440: 69 6e 02 06 04 0c 1a 88 80 80 80 80 02 03 00 3a in.............: | 3456: 00 00 00 15 04 30 61 6e 64 02 06 01 01 02 02 02 .....0and....... | 3472: 72 65 02 02 03 04 0a 17 88 80 80 80 80 01 03 00 re.............. | 3488: 34 01 00 00 12 02 30 31 02 06 01 01 04 01 01 32 4.....01.......2 | 3504: 02 02 05 04 08 08 84 80 80 80 80 12 03 00 16 00 ................ | 3520: 00 00 05 04 1b 84 80 80 80 80 11 03 00 3c 00 00 .............<.. | 3536: 00 16 05 34 74 61 62 6c 01 06 01 01 05 02 03 65 ...4tabl.......e | 3552: 72 6d 01 02 04 0b 1b 84 80 80 80 80 10 03 00 3c rm.............< | 3568: 00 00 00 16 05 34 65 61 63 68 01 02 02 01 04 70 .....4each.....p | 3584: 72 65 73 01 02 05 04 09 1a 84 80 80 80 80 0f 03 res............. | 3600: 00 3a 00 00 00 15 04 33 74 65 72 01 02 04 02 02 .:.....3ter..... | 3616: 68 65 01 06 01 01 03 04 08 1b 84 80 80 80 80 0e he.............. | 3632: 03 00 3c 00 00 00 16 04 33 70 72 65 01 02 05 01 ..<.....3pre.... | 3648: 03 74 61 62 01 06 01 01 05 04 08 1a 84 80 80 80 .tab............ | 3664: 80 0d 03 0d 1a 00 00 00 15 04 33 66 6e 72 01 02 ..........3fnr.. | 3680: 02 02 02 74 73 01 06 01 01 04 04 08 1b 84 80 80 ...ts........... | 3696: 80 80 0c 03 00 3c 00 00 00 16 03 32 74 68 01 06 .....<.....2th.. | 3712: 01 01 03 00 04 33 65 61 63 01 02 03 04 09 18 84 .....3eac....... | 3728: 80 80 80 80 0b 03 00 36 00 00 00 13 03 32 74 61 .......6.....2ta | 3744: 01 06 01 01 05 02 01 65 01 02 04 04 09 19 84 80 .......e........ | 3760: 80 80 80 0a 03 00 38 00 00 00 14 03 30 c9 6e 01 ......8.....0.n. | 3776: 06 01 01 02 01 02 70 72 01 02 05 04 09 18 84 80 ......pr........ | 3792: 80 80 80 09 03 00 36 00 00 00 13 03 32 66 6f 01 ......6.....2fo. | 3808: 02 0b e2 01 74 01 06 01 01 04 04 07 1b 84 80 80 ....t........... | 3824: 86 f0 08 03 00 3c 00 00 00 16 02 31 74 01 0a 04 .....<.....1t... | 3840: 01 01 03 04 00 03 32 65 61 01 02 03 04 0a 17 84 ......2ea....... | 3856: 80 80 80 80 07 03 00 34 00 00 00 12 02 31 69 01 .......4.....1i. | 3872: 06 01 01 02 01 01 70 01 02 05 04 08 18 84 80 80 ......p......... | 3888: 80 80 06 03 00 36 00 00 00 13 02 31 65 01 02 03 .....6.....1e... | 3904: 01 01 66 01 08 02 01 01 04 04 06 1b 84 80 80 80 ..f............. | 3920: 80 05 03 00 3c 00 00 00 16 05 30 74 65 72 6d 01 ....<.....0term. | 3936: 02 04 02 02 68 65 01 06 01 01 03 04 09 14 84 80 ....he.......... | 3952: 80 80 80 04 03 00 2e 00 00 00 10 06 30 74 61 62 ............0tab | 3968: 6c 65 01 06 01 01 05 04 15 84 7f 80 80 80 03 03 le.............. | 3984: 00 30 00 00 00 11 02 08 30 70 72 65 73 65 6e 74 .0......0present | 4000: 01 02 05 05 1b 84 80 22 80 80 02 03 00 3c 00 00 .............<.. | 4016: 00 16 04 30 66 74 73 01 06 01 01 04 01 02 69 6e ...0fts.......in | 4032: 01 06 01 01 04 0a 1a 84 80 80 80 80 01 03 00 3a ...............: | 4048: 00 00 00 15 05 30 65 61 63 68 01 02 03 01 03 66 .....0each.....f | 4064: 6f 72 01 02 02 04 09 06 01 03 00 12 03 0b 0f 00 or.............. | 4080: 00 08 8c 80 80 80 80 11 03 00 16 00 00 00 05 04 ................ | page 3 offset 8192 | 0: 0a 00 00 00 32 0e 4f 00 0f fa 0f f1 0f e9 0f e1 ....2.O......... | 16: 0f d8 0f d1 0f c9 0f c1 0f b9 0f b1 0f a9 0f a0 ................ | 32: 0f 98 0f 90 0f 87 0f 80 0f 78 0f 71 0f 68 0f 5f .........x.q.h._ | 48: 0f 56 0f 00 00 00 00 00 00 00 00 00 00 00 00 00 .V.............. | 3392: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 08 ................ | 3408: 04 01 10 01 03 34 74 20 07 04 01 0e 01 03 34 1e .....4t ......4. | 3424: 09 04 01 12 01 03 33 74 68 1c 08 04 01 10 01 03 ......3th....... | 3440: 34 6e 1a 08 04 01 10 01 03 32 67 18 08 04 01 10 4n.......2g..... | 3456: 01 03 32 74 16 08 04 01 10 01 03 32 6e 14 07 04 ..2t.......2n... | 3472: 01 0e 01 03 32 12 08 04 01 10 01 03 31 74 10 08 ....2.......1t.. | 3488: 04 01 10 01 03 31 6e 0e 07 04 01 0e 01 03 31 0c .....1n.......1. | 3504: 09 04 01 12 01 03 30 74 68 0a 08 04 01 10 01 03 ......0th....... | 3520: 30 74 08 09 04 01 12 01 03 30 6e 75 06 08 04 01 0t.......0nu.... | 3536: 10 01 03 30 6e 04 06 04 01 0c 01 03 02 08 04 01 ...0n........... | 3552: 10 01 02 34 72 22 07 04 01 0e 01 02 34 20 08 04 ...4r.......4 .. | 3568: 01 10 01 02 33 72 1e 09 04 01 12 01 02 33 61 72 ....3r.......3ar | 3584: 1c 18 04 01 10 01 02 32 74 1a 08 04 01 10 01 02 .......2t....... | 3600: 32 69 18 09 04 01 12 01 02 32 61 72 16 08 04 01 2i.......2ar.... | 3616: 10 01 02 31 74 2a 08 04 01 0f 01 02 31 6e 12 0a ...1t*......1n.. | 3632: d4 01 10 01 02 31 62 10 08 04 01 10 01 02 31 32 .....1b.......12 | 3648: 0e 0b 04 01 16 01 02 30 74 68 65 72 0c 08 04 01 .......0ther.... | 3664: 10 01 02 30 74 0a 08 04 01 10 01 02 30 6e 08 08 ...0t.......0n.. | 3680: 04 01 10 01 02 30 62 06 08 04 01 10 01 02 30 61 .....0b.......0a | 3696: 04 06 03 f1 0c 01 02 02 07 04 09 10 01 34 74 22 .............4t. | 3712: 06 04 09 0e 01 34 20 08 04 09 12 01 33 74 65 1e .....4 .....3te. | 3728: 07 04 09 10 01 33 70 1c 07 04 09 10 01 33 66 1a .....3p......3f. | 3744: 08 04 09 12 01 32 74 68 18 07 04 09 10 01 32 74 .....2th......2t | 3760: 16 07 04 09 10 01 32 69 14 07 04 09 10 01 32 66 ......2i......2f | 3776: 12 07 04 09 10 01 31 74 10 07 04 09 10 01 31 69 ......1t......1i | 3792: 0e 06 04 09 0e 01 31 0c 08 04 09 12 01 30 74 65 ......1......0te | 3808: 0a 07 04 09 10 01 30 74 08 07 04 09 10 01 30 70 ......0t......0p | 3824: 06 08 04 09 1e 61 30 66 74 04 05 00 00 00 00 00 .....a0ft....... | page 4 offset 12288 | 4064: 00 00 00 00 00 00 00 00 00 00 00 05 03 03 00 10 ................ | 4080: 03 05 05 02 03 00 10 04 06 05 01 03 00 10 04 04 ................ | page 5 offset 16384 | 0: 0a 00 00 00 02 0f eb 00 0f eb 0f f4 00 00 00 00 ................ | 4064: 00 00 00 00 00 00 00 00 00 00 00 08 03 15 01 70 ...............p | 4080: 67 73 7a 18 0b 03 1b 01 76 65 72 73 69 6f 6e 04 gsz.....version. | end crash-ef6738247b1344.db }]} {} do_catchsql_test 4.1 { BEGIN; REPLACE INTO t1(rowid,b,a,rowid) VALUES(200,1,2,3); } {1 {database disk image is malformed}} do_catchsql_test 4.2 { INSERT INTO t1(t1) VALUES('delete-all'); } {1 {database disk image is malformed}} do_catchsql_test 4.3 { REPLACE INTO t1(rowid,b,rowid,a) VALUES(200,1,2,3); } {1 {database disk image is malformed}} do_catchsql_test 4.4 { REPLACE INTO t1(rowid,b,a,rowid) VALUES(0,1,2,3); } {1 {database disk image is malformed}} do_catchsql_test 4.5 { REPLACE INTO t1(rowid,a,b,rowid) VALUES(200,1,2,3); } {1 {database disk image is malformed}} sqlite3_fts5_may_be_corrupt 0 finish_test |
Added ext/fts5/test/fts5corrupt6.test.
> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 | # 2015 Apr 24 # # The author disclaims copyright to this source code. In place of # a legal notice, here is a blessing: # # May you do good and not evil. # May you find forgiveness for yourself and forgive others. # May you share freely, never taking more than you give. # #*********************************************************************** # # This file tests that FTS5 handles corrupt databases (i.e. internal # inconsistencies in the backing tables) correctly. In this case # "correctly" means without crashing. # source [file join [file dirname [info script]] fts5_common.tcl] set testprefix fts5corrupt6 # If SQLITE_ENABLE_FTS5 is defined, omit this file. ifcapable !fts5 { finish_test return } sqlite3_fts5_may_be_corrupt 1 database_may_be_corrupt proc editblock {block} { binary format Sa* 20000 [string range $block 2 end] } db func editblock editblock do_execsql_test 1.0 { CREATE VIRTUAL TABLE ft USING fts5(abc, def); WITH a(i) AS ( SELECT 1 UNION ALL SELECT i+1 FROM a WHERE i<1000 ) INSERT INTO ft SELECT 'abc abc abc abc abc abc abc abc abc abc', 'def def def def def def def def def def' FROM a; UPDATE ft_data SET block = editblock(block) WHERE id=( SELECT id FROM ft_data ORDER BY id LIMIT 1 OFFSET 5 ); } do_catchsql_test 1.1 { SELECT rowid FROM ft('def') ORDER BY rowid DESC LIMIT 1 OFFSET 9999; } {1 {database disk image is malformed}} sqlite3_fts5_may_be_corrupt 0 finish_test |
Changes to ext/fts5/test/fts5delete.test.
︙ | ︙ | |||
45 46 47 48 49 50 51 52 53 | do_test 1.2 { execsql { INSERT INTO t1(t1, rank) VALUES('usermerge', 2); } for {set i 0} {$i < 5} {incr i} { execsql { INSERT INTO t1(t1, rank) VALUES('merge', 1) } execsql { INSERT INTO t1(t1) VALUES('integrity-check') } } } {} finish_test | > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 | do_test 1.2 { execsql { INSERT INTO t1(t1, rank) VALUES('usermerge', 2); } for {set i 0} {$i < 5} {incr i} { execsql { INSERT INTO t1(t1, rank) VALUES('merge', 1) } execsql { INSERT INTO t1(t1) VALUES('integrity-check') } } } {} #------------------------------------------------------------------------- reset_db do_execsql_test 2.0 { CREATE TABLE test ( id INTEGER PRIMARY KEY, name TEXT, value TEXT ); CREATE VIRTUAL TABLE test_idx USING fts5( name, content=test, content_rowid=id ); } do_catchsql_test 2.1 { INSERT INTO test_idx (test_idx, rowid, name) VALUES('delete', 1, 'quick'); } {1 {database disk image is malformed}} do_catchsql_test 2.2 { INSERT INTO test_idx(rowid, name) VALUES(123, 'one one one'); INSERT INTO test_idx (test_idx, rowid, name) VALUES('delete', 123, 'one'); INSERT INTO test_idx (test_idx, rowid, name) VALUES('delete', 123, 'one'); } {1 {database disk image is malformed}} do_execsql_test 2.3 { DROP TABLE test_idx; CREATE VIRTUAL TABLE test_idx USING fts5( name, content=test, content_rowid=id ); INSERT INTO test_idx(rowid, name) VALUES(123, 'one one one'); INSERT INTO test_idx(rowid, name) VALUES(124, 'two two two'); INSERT INTO test_idx(rowid, name) VALUES(125, 'two two two'); INSERT INTO test_idx (test_idx, rowid, name) VALUES('delete', 123, 'one'); INSERT INTO test_idx (test_idx, rowid, name) VALUES('delete', 123, 'one'); INSERT INTO test_idx (test_idx, rowid, name) VALUES('delete', 123, 'one'); } do_catchsql_test 2.4 { SELECT rowid FROM test_idx WHERE test_idx MATCH 'two' ORDER BY rank; } {1 {database disk image is malformed}} #------------------------------------------------------------------------- reset_db do_execsql_test 3.0 { CREATE VIRTUAL TABLE tx USING fts5(a, b, c, d, content=); INSERT INTO tx(rowid, a, c) VALUES(1, 'abc def', 'a b c'); INSERT INTO tx(rowid, a, c) VALUES(5, 'a b c', 'a b d def'); } do_execsql_test 3.1 { INSERT INTO tx(tx, rowid, a, b, c, d) VALUES('delete', 5, 'a b c', NULL, 'a b d def', NULL); } do_execsql_test 3.2 { INSERT INTO tx(tx) VALUES('integrity-check'); } do_execsql_test 3.3 { INSERT INTO tx(tx, rowid, a, b, c, d) VALUES('delete', 1, 'abc def', NULL, 'a b c', NULL); } do_execsql_test 3.4 { INSERT INTO tx(tx) VALUES('integrity-check'); } finish_test |
Changes to ext/fts5/test/fts5detail.test.
︙ | ︙ | |||
208 209 210 211 212 213 214 215 216 217 218 219 220 221 | INSERT INTO t4 VALUES('1 2 3', '4 5 6', '7 8 9'); } do_catchsql_test 4.1 { SELECT * FROM t4('a:a') } {1 {fts5: column queries are not supported (detail=none)}} #------------------------------------------------------------------------- # Test that for the same content detail=none uses less space than # detail=col, and that detail=col uses less space than detail=full # reset_db do_test 5.1 { foreach {tbl detail} {t1 none t2 col t3 full} { | > > > > | 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 | INSERT INTO t4 VALUES('1 2 3', '4 5 6', '7 8 9'); } do_catchsql_test 4.1 { SELECT * FROM t4('a:a') } {1 {fts5: column queries are not supported (detail=none)}} do_catchsql_test 4.2 { SELECT * FROM t4('a:a &') } {1 {fts5: syntax error near "&"}} #------------------------------------------------------------------------- # Test that for the same content detail=none uses less space than # detail=col, and that detail=col uses less space than detail=full # reset_db do_test 5.1 { foreach {tbl detail} {t1 none t2 col t3 full} { |
︙ | ︙ |
Changes to ext/fts5/test/fts5doclist.test.
︙ | ︙ | |||
38 39 40 41 42 43 44 45 46 | INSERT INTO ccc(x899) SELECT rnddoc(500) FROM ii; } do_execsql_test 1.2 { INSERT INTO ccc(ccc) VALUES('integrity-check'); } finish_test | > > > > > > > > > > > > > > > > > > > > > | 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 | INSERT INTO ccc(x899) SELECT rnddoc(500) FROM ii; } do_execsql_test 1.2 { INSERT INTO ccc(ccc) VALUES('integrity-check'); } #------------------------------------------------------------------------- # reset_db do_execsql_test 2.1 { CREATE VIRTUAL TABLE tx USING fts5(x); } set doc [string repeat "abc " 5000] do_execsql_test 2.2 { BEGIN; INSERT INTO tx(rowid, x) VALUES(-9000000000000000000, $doc); INSERT INTO tx(rowid, x) VALUES(9000000000000000000, $doc); COMMIT; } do_execsql_test 2.3 { SELECT rowid FROM tx('abc'); } { -9000000000000000000 9000000000000000000 } finish_test |
Changes to ext/fts5/test/fts5eb.test.
︙ | ︙ | |||
55 56 57 58 59 60 61 | do_execsql_test 1.$tn {SELECT fts5_expr($expr)} [list $res] } do_catchsql_test 2.1 { SELECT fts5_expr() } {1 {wrong number of arguments to function fts5_expr}} | | > > > > > > > > > > > > > > > > > | | 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 | do_execsql_test 1.$tn {SELECT fts5_expr($expr)} [list $res] } do_catchsql_test 2.1 { SELECT fts5_expr() } {1 {wrong number of arguments to function fts5_expr}} do_catchsql_test 2.2 { SELECT fts5_expr_tcl() } {1 {wrong number of arguments to function fts5_expr_tcl}} do_catchsql_test 2.3 { SELECT fts5_expr('') } {1 {fts5: syntax error near ""}} do_catchsql_test 2.4 { SELECT fts5_expr(NULL) } {1 {fts5: syntax error near ""}} do_catchsql_test 2.5 { SELECT fts5_expr(NULL, NULL) } {1 {parse error in ""}} for {set i 0} {$i < 255} {incr i} { do_test 2.6.$i { lindex [catchsql {sELECT fts5_expr(NULL, char($i));}] 0 } 1 } do_execsql_test 3.0 { CREATE VIRTUAL TABLE e1 USING fts5(text, tokenize = 'porter unicode61'); INSERT INTO e1 VALUES ('just a few words with a / inside'); } do_execsql_test 3.1 { SELECT rowid, bm25(e1) FROM e1 WHERE e1 MATCH '"just"' ORDER BY rank; } {1 -1e-06} do_execsql_test 3.2 { SELECT rowid FROM e1 WHERE e1 MATCH '"/" OR "just"' } 1 |
︙ | ︙ |
Changes to ext/fts5/test/fts5fault4.test.
︙ | ︙ | |||
12 13 14 15 16 17 18 | # This file is focused on OOM errors. # source [file join [file dirname [info script]] fts5_common.tcl] source $testdir/malloc_common.tcl set testprefix fts5fault4 | | > > > > | 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 | # This file is focused on OOM errors. # source [file join [file dirname [info script]] fts5_common.tcl] source $testdir/malloc_common.tcl set testprefix fts5fault4 # If SQLITE_ENABLE_FTS5 is not defined, omit this file. ifcapable !fts5 { finish_test return } set ::TMPDBERROR [list 1 \ {unable to open a temporary database file for storing temporary tables} ] #------------------------------------------------------------------------- # An OOM while dropping an fts5 table. # db func rnddoc fts5_rnddoc do_test 1.0 { execsql { CREATE VIRTUAL TABLE xx USING fts5(x) } |
︙ | ︙ | |||
387 388 389 390 391 392 393 | faultsim_save_and_close do_faultsim_test 14.1 -faults oom-t* -prep { faultsim_restore_and_reopen db eval { SELECT * FROM "tbl one" } } -body { db eval { ALTER TABLE "tbl one" RENAME TO "tbl two" } } -test { | | | 391 392 393 394 395 396 397 398 399 400 401 | faultsim_save_and_close do_faultsim_test 14.1 -faults oom-t* -prep { faultsim_restore_and_reopen db eval { SELECT * FROM "tbl one" } } -body { db eval { ALTER TABLE "tbl one" RENAME TO "tbl two" } } -test { faultsim_test_result {0 {}} $::TMPDBERROR } finish_test |
Changes to ext/fts5/test/fts5faultB.test.
︙ | ︙ | |||
142 143 144 145 146 147 148 149 150 151 | INSERT INTO t1 VALUES('b c d a'); -- 4 } do_faultsim_test 5.1 -faults oom* -body { execsql { SELECT rowid FROM t1('^a OR ^b') } } -test { faultsim_test_result {0 {1 4}} } finish_test | > > > > > > > > > > > > > > > > > > > > > > | 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 | INSERT INTO t1 VALUES('b c d a'); -- 4 } do_faultsim_test 5.1 -faults oom* -body { execsql { SELECT rowid FROM t1('^a OR ^b') } } -test { faultsim_test_result {0 {1 4}} } #------------------------------------------------------------------------- # Test OOM injection in a query with two MATCH expressions # reset_db do_execsql_test 6.0 { CREATE VIRTUAL TABLE t1 USING fts5(a); INSERT INTO t1 VALUES('a b c d'); -- 1 INSERT INTO t1 VALUES('d a b c'); -- 2 INSERT INTO t1 VALUES('c d a b'); -- 3 INSERT INTO t1 VALUES('b c d a'); -- 4 } do_faultsim_test 6.1 -faults oom* -body { execsql { SELECT rowid FROM t1 WHERE t1 MATCH 'a' AND t1 MATCH 'b' } } -test { faultsim_test_result {0 {1 2 3 4}} } do_faultsim_test 6.2 -faults oom* -body { execsql { SELECT rowid FROM t1 WHERE t1 MATCH 'a OR b' AND t1 MATCH 'c OR d' } } -test { faultsim_test_result {0 {1 2 3 4}} } finish_test |
Changes to ext/fts5/test/fts5faultD.test.
︙ | ︙ | |||
10 11 12 13 14 15 16 | #************************************************************************* # # This file is focused on OOM errors. # source [file join [file dirname [info script]] fts5_common.tcl] source $testdir/malloc_common.tcl | | | 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 | #************************************************************************* # # This file is focused on OOM errors. # source [file join [file dirname [info script]] fts5_common.tcl] source $testdir/malloc_common.tcl set testprefix fts5faultD # If SQLITE_ENABLE_FTS3 is defined, omit this file. ifcapable !fts5 { finish_test return } |
︙ | ︙ |
Added ext/fts5/test/fts5faultE.test.
> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 | # 2016 February 2 # # The author disclaims copyright to this source code. In place of # a legal notice, here is a blessing: # # May you do good and not evil. # May you find forgiveness for yourself and forgive others. # May you share freely, never taking more than you give. # #************************************************************************* # # This file is focused on OOM errors. # source [file join [file dirname [info script]] fts5_common.tcl] source $testdir/malloc_common.tcl set testprefix fts5faultE # If SQLITE_ENABLE_FTS5 is not defined, omit this file. ifcapable !fts5 { finish_test return } faultsim_save_and_close do_faultsim_test 1 -prep { faultsim_restore_and_reopen } -body { execsql { CREATE VIRTUAL TABLE t1 USING fts5(x, y, tokenize=trigram) } } -test { faultsim_test_result {0 {}} {1 {vtable constructor failed: t1}} } reset_db do_execsql_test 2.0 { CREATE VIRTUAL TABLE t1 USING fts5(x, y, tokenize=trigram); } faultsim_save_and_close do_faultsim_test 2 -prep { faultsim_restore_and_reopen } -body { execsql { INSERT INTO t1 VALUES('abcdefghijklmnopqrstuvwxyz', NULL); SELECT count(*) FROM t1 WHERE x LIKE '%mnop%' AND t1 MATCH 'jkl'; } } -test { faultsim_test_result {0 1} {1 {vtable constructor failed: t1}} } reset_db do_execsql_test 3.0 { CREATE VIRTUAL TABLE t1 USING fts5(x, y, tokenize=trigram, detail=none); INSERT INTO t1 VALUES('abcdefghijklmnopqrstuvwxyz', NULL); } faultsim_save_and_close do_faultsim_test 3 -prep { faultsim_restore_and_reopen } -body { execsql { SELECT count(*) FROM t1 WHERE x LIKE '%mnopqrs%' AND t1 MATCH 'abc' } } -test { faultsim_test_result {0 1} {1 {vtable constructor failed: t1}} } finish_test |
Changes to ext/fts5/test/fts5full.test.
︙ | ︙ | |||
32 33 34 35 36 37 38 | db func rnddoc fts5_rnddoc do_test 1.1 { list [catch { for {set i 0} {$i < 2500} {incr i} { execsql { INSERT INTO x8 VALUES( rnddoc(5) ); } } } msg] $msg | | | 32 33 34 35 36 37 38 39 40 41 42 | db func rnddoc fts5_rnddoc do_test 1.1 { list [catch { for {set i 0} {$i < 2500} {incr i} { execsql { INSERT INTO x8 VALUES( rnddoc(5) ); } } } msg] $msg } {0 {}} finish_test |
Changes to ext/fts5/test/fts5hash.test.
︙ | ︙ | |||
108 109 110 111 112 113 114 | INSERT INTO eee(eee) VALUES('integrity-check'); } #----------------------------------------------------------------------- # Add a small and very large token with the same hash value to an # empty table. At one point this would provoke an asan error. # | | > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 | INSERT INTO eee(eee) VALUES('integrity-check'); } #----------------------------------------------------------------------- # Add a small and very large token with the same hash value to an # empty table. At one point this would provoke an asan error. # do_test 1.5 { set big [string repeat 12345 40] set hash [sqlite3_fts5_token_hash 1024 $big] while {1} { set small [random_token] if {[sqlite3_fts5_token_hash 1024 $small]==$hash} break } execsql { CREATE VIRTUAL TABLE t2 USING fts5(x, detail=%DETAIL%) } execsql { INSERT INTO t2 VALUES($small || ' ' || $big); } } {} } ;# foreach_detail_mode #------------------------------------------------------------------------- reset_db do_execsql_test 2.1 { CREATE VIRTUAL TABLE t1 USING fts5(x); INSERT INTO t1(t1, rank) VALUES('hashsize', 1024); INSERT INTO t1(t1, rank) VALUES('automerge', 0); INSERT INTO t1(t1, rank) VALUES('crisismerge', 1000); } do_execsql_test 2.2 { BEGIN; INSERT INTO t1 VALUES('abc def ghi'); SELECT count(*) FROM t1_data; } {2} do_execsql_test 2.3 { WITH s(i) AS ( SELECT 1 UNION ALL SELECT i+1 FROM s WHERE i<1024 ) INSERT INTO t1 SELECT 'abc def ghi' FROM s; SELECT (SELECT count(*) FROM t1_data) > 10; } {1} do_execsql_test 2.4 { COMMIT; DROP TABLE t1; CREATE VIRTUAL TABLE t1 USING fts5(x); INSERT INTO t1(t1, rank) VALUES('hashsize', 1024); INSERT INTO t1(t1, rank) VALUES('automerge', 0); INSERT INTO t1(t1, rank) VALUES('crisismerge', 1000); WITH s(i) AS ( SELECT 1 UNION ALL SELECT i+1 FROM s WHERE i<1024 ) INSERT INTO t1 SELECT 'abc' || i || ' def' || i || ' ghi' || i FROM s; SELECT (SELECT count(*) FROM t1_data) > 100; } {1} finish_test |
Changes to ext/fts5/test/fts5integrity.test.
︙ | ︙ | |||
184 185 186 187 188 189 190 | } { do_execsql_test 6.$tn.1 { DROP TABLE IF EXISTS hh; CREATE VIRTUAL TABLE hh USING fts5(y); INSERT INTO hh(hh, rank) VALUES('pgsz', $pgsz); WITH s(i) AS (SELECT 0 UNION ALL SELECT i+1 FROM s WHERE i<999) | | | > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 | } { do_execsql_test 6.$tn.1 { DROP TABLE IF EXISTS hh; CREATE VIRTUAL TABLE hh USING fts5(y); INSERT INTO hh(hh, rank) VALUES('pgsz', $pgsz); WITH s(i) AS (SELECT 0 UNION ALL SELECT i+1 FROM s WHERE i<999) INSERT INTO hh SELECT printf('%.3d%.3d%.3d %.3d%.3d%.3d',i,i,i,i+1,i+1,i+1) FROM s; WITH s(i) AS (SELECT 0 UNION ALL SELECT i+1 FROM s WHERE i<999) INSERT INTO hh SELECT printf('%.3d%.3d%.3d %.3d%.3d%.3d',i,i,i,i+1,i+1,i+1) FROM s; INSERT INTO hh(hh) VALUES('optimize'); } do_test 6.$tn.2 { set ok 0 for {set i 0} {$i < 1000} {incr i} { set T [format %.3d%.3d%.3d $i $i $i] set res [db eval { SELECT rowid FROM hh($T) ORDER BY rowid ASC }] set res2 [db eval { SELECT rowid FROM hh($T) ORDER BY rowid DESC }] if {$res == [lsort -integer $res2]} { incr ok } } set ok } {1000} } #------------------------------------------------------------------------- # reset_db do_execsql_test 7.0 { PRAGMA encoding = 'UTF-16'; CREATE VIRTUAL TABLE vt0 USING fts5(c0); INSERT INTO vt0 VALUES (x'46f0'); SELECT quote(c0) FROM vt0; } {X'46F0'} do_execsql_test 7.1 { INSERT INTO vt0(vt0) VALUES('integrity-check'); } do_execsql_test 7.2 { INSERT INTO vt0(vt0) VALUES('rebuild'); } do_execsql_test 7.3 { INSERT INTO vt0(vt0) VALUES('integrity-check'); } do_execsql_test 7.4 { UPDATE vt0 SET c0=''; } do_execsql_test 7.5 { INSERT INTO vt0(vt0) VALUES('integrity-check'); } #------------------------------------------------------------------------- # Ticket 7a458c2a5f4 # reset_db do_execsql_test 8.0 { PRAGMA locking_mode = EXCLUSIVE; PRAGMA journal_mode = PERSIST; CREATE VIRTUAL TABLE vt0 USING fts5(c0); } {exclusive persist} do_execsql_test 8.1 { PRAGMA data_version } {1} do_execsql_test 8.2 { INSERT INTO vt0(vt0) VALUES('integrity-check'); PRAGMA data_version; } {1} do_execsql_test 8.1 { INSERT INTO vt0(vt0, rank) VALUES('usermerge', 2); } #------------------------------------------------------------------------- # Ticket [771fe617] # reset_db do_execsql_test 9.0 { PRAGMA encoding = 'UTF16'; CREATE VIRTUAL TABLE vt0 USING fts5(c0); } #explain_i { SELECT quote(SUBSTR(x'37', 0)); } #execsql { PRAGMA vdbe_trace = 1 } do_execsql_test 9.1.1 { SELECT quote(SUBSTR(x'37', 0)); } {X'37'} do_execsql_test 9.1.2 { SELECT quote(x'37'); } {X'37'} do_execsql_test 9.2 { INSERT INTO vt0 VALUES (SUBSTR(x'37', 0)); -- INSERT INTO vt0 VALUES (x'37'); } do_execsql_test 9.3 { INSERT INTO vt0(vt0) VALUES('integrity-check'); } #------------------------------------------------------------------------- reset_db do_execsql_test 10.0 { CREATE TABLE t1(i INTEGER PRIMARY KEY, a, b); CREATE VIRTUAL TABLE vt0 USING fts5(a, b, content=t1); INSERT INTO vt0(rowid, a, b) VALUES(1, 'abc', 'def'); } do_catchsql_test 10.1 { INSERT INTO vt0(vt0) VALUES('integrity-check'); } {0 {}} do_catchsql_test 10.2 { INSERT INTO vt0(vt0, rank) VALUES('integrity-check', 0); } {0 {}} do_catchsql_test 10.3 { INSERT INTO vt0(vt0, rank) VALUES('integrity-check', 1); } {1 {database disk image is malformed}} do_catchsql_test 10.3 { INSERT INTO t1 VALUES(1, 'abc', 'def'); INSERT INTO vt0(vt0, rank) VALUES('integrity-check', 1); } {0 {}} do_execsql_test 10.4 { CREATE VIRTUAL TABLE vt1 USING fts5(a, b, content=); INSERT INTO vt1(rowid, a, b) VALUES(1, 'abc', 'def'); } do_catchsql_test 10.5.1 { INSERT INTO vt0(vt0, rank) VALUES('integrity-check', 0); } {0 {}} do_catchsql_test 10.5.2 { INSERT INTO vt0(vt0, rank) VALUES('integrity-check', 1); } {0 {}} do_catchsql_test 10.5.3 { INSERT INTO vt0(vt0) VALUES('integrity-check'); } {0 {}} finish_test |
Added ext/fts5/test/fts5interrupt.test.
> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 | # 2019 Jan 4 # # The author disclaims copyright to this source code. In place of # a legal notice, here is a blessing: # # May you do good and not evil. # May you find forgiveness for yourself and forgive others. # May you share freely, never taking more than you give. # #************************************************************************* # This file implements regression tests for SQLite library. The # focus of this script is testing the FTS5 module. # source [file join [file dirname [info script]] fts5_common.tcl] set testprefix fts5interrupt # If SQLITE_ENABLE_FTS5 is not defined, omit this file. ifcapable !fts5 { finish_test return } do_execsql_test 1.0 { CREATE VIRTUAL TABLE t1 USING fts5(a); INSERT INTO t1(t1, rank) VALUES('pgsz', 40); } db_save_and_close proc progress_handler {args} { incr ::progress_handler_delay -1 if {$::progress_handler_delay<=0} { return 1 } return 0 } foreach {tn sql} { 1 { INSERT INTO t1(rowid, a) VALUES(0, 'z z z z') } 2 { COMMIT } } { set bDone 0 for {set i 1} {$bDone==0} {incr i} { do_test 1.$tn.$i { db_restore_and_reopen execsql { BEGIN; INSERT INTO t1(rowid, a) VALUES(1, 'a b c d'); INSERT INTO t1(rowid, a) VALUES(2, 'd e f g'); INSERT INTO t1(rowid, a) VALUES(3, 'h i j k'); INSERT INTO t1(rowid, a) VALUES(4, 'l m n o'); } set ::progress_handler_delay $i db progress 1 progress_handler set res [catchsql $sql] db close if {$res=="0 {}"} { set bDone 1 } else { if {$res!="1 interrupted"} { error "got: $res" } } set {} {} } {} } } finish_test |
Changes to ext/fts5/test/fts5matchinfo.test.
︙ | ︙ | |||
486 487 488 489 490 491 492 493 494 | CREATE VIRTUAL TABLE x1 USING fts5(z); INSERT INTO x1 VALUES('a b c a b c a b c'); } {} do_catchsql_test 14.2 { SELECT matchinfo(x1, 'd') FROM x1('a b c'); } {1 {unrecognized matchinfo flag: d}} finish_test | > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | 486 487 488 489 490 491 492 493 494 495 496 497 498 499 500 501 502 503 504 505 506 507 508 509 510 511 512 513 514 515 516 517 518 519 520 521 522 523 | CREATE VIRTUAL TABLE x1 USING fts5(z); INSERT INTO x1 VALUES('a b c a b c a b c'); } {} do_catchsql_test 14.2 { SELECT matchinfo(x1, 'd') FROM x1('a b c'); } {1 {unrecognized matchinfo flag: d}} #------------------------------------------------------------------------- # Test using matchinfo() and similar on a non-full-text query # do_execsql_test 15.0 { CREATE VIRTUAL TABLE t1 USING fts5(x, y); INSERT INTO t1 VALUES('a', 'b'); INSERT INTO t1 VALUES('c', 'd'); } if {$tcl_platform(byteOrder)=="littleEndian"} { set res {X'02000000'} } else { set res {X'00000002'} } do_execsql_test 15.1 { SELECT quote(matchinfo(t1, 'n')) FROM t1 LIMIT 1; } $res do_execsql_test 15.2 { DELETE FROM t1_content WHERE rowid=1; SELECT quote(matchinfo(t1, 'n')) FROM t1 LIMIT 1; } $res fts5_aux_test_functions db do_execsql_test 15.3 { SELECT fts5_test_all(t1) FROM t1 LIMIT 1; } { {columnsize {0 0} columntext {c d} columntotalsize {2 2} poslist {} tokenize {c d} rowcount 2} } finish_test |
Added ext/fts5/test/fts5misc.test.
> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340 341 342 343 344 345 346 347 348 349 350 351 352 353 354 355 356 357 358 359 360 361 362 363 364 365 366 367 368 369 370 371 372 373 374 375 376 377 378 379 380 381 382 383 384 385 386 387 388 389 390 391 392 393 394 395 396 397 398 399 400 401 402 403 404 405 406 407 408 409 410 411 412 413 414 415 416 417 418 419 420 421 422 423 424 425 426 427 428 429 430 431 432 433 434 435 436 437 438 439 440 441 442 443 444 445 446 447 | # 2019 September 02 # # The author disclaims copyright to this source code. In place of # a legal notice, here is a blessing: # # May you do good and not evil. # May you find forgiveness for yourself and forgive others. # May you share freely, never taking more than you give. # #************************************************************************* # This file implements regression tests for SQLite library. The # focus of this script is testing the FTS5 module. # source [file join [file dirname [info script]] fts5_common.tcl] set testprefix fts5misc # If SQLITE_ENABLE_FTS5 is not defined, omit this file. ifcapable !fts5 { finish_test return } do_execsql_test 1.0 { CREATE VIRTUAL TABLE t1 USING fts5(a); } do_catchsql_test 1.1.1 { SELECT highlight(t1, 4, '<b>', '</b>') FROM t1('*'); } {1 {unknown special query: }} do_catchsql_test 1.1.2 { SELECT a FROM t1 WHERE rank = (SELECT highlight(t1, 4, '<b>', '</b>') FROM t1('*')); } {1 {unknown special query: }} do_catchsql_test 1.2.1 { SELECT highlight(t1, 4, '<b>', '</b>') FROM t1('*id'); } {0 {{}}} do_catchsql_test 1.2.2 { SELECT a FROM t1 WHERE rank = (SELECT highlight(t1, 4, '<b>', '</b>') FROM t1('*id')); } {0 {}} do_catchsql_test 1.3.1 { SELECT highlight(t1, 4, '<b>', '</b>') FROM t1('*reads'); } {1 {no such cursor: 1}} do_catchsql_test 1.3.2 { SELECT a FROM t1 WHERE rank = (SELECT highlight(t1, 4, '<b>', '</b>') FROM t1('*reads')); } {1 {no such cursor: 1}} db close sqlite3 db test.db do_catchsql_test 1.3.3 { SELECT a FROM t1 WHERE rank = (SELECT highlight(t1, 4, '<b>', '</b>') FROM t1('*reads')); } {1 {no such cursor: 1}} #------------------------------------------------------------------------- reset_db do_execsql_test 2.0 { CREATE TABLE t0(c0); CREATE VIRTUAL TABLE vt0 USING fts5(c0); } do_execsql_test 2.1.1 { BEGIN TRANSACTION; INSERT INTO vt0(c0) VALUES ('xyz'); } do_execsql_test 2.1.2 { ALTER TABLE t0 ADD COLUMN c5; } do_execsql_test 2.1.3 { INSERT INTO vt0(vt0) VALUES('integrity-check'); } do_execsql_test 2.1.4 { INSERT INTO vt0(c0) VALUES ('abc'); COMMIT } do_execsql_test 2.1.5 { INSERT INTO vt0(vt0) VALUES('integrity-check'); } reset_db do_execsql_test 2.2.1 { CREATE TABLE t0(c0); CREATE VIRTUAL TABLE vt0 USING fts5(c0); BEGIN TRANSACTION; INSERT INTO vt0(c0) VALUES ('xyz'); } breakpoint do_execsql_test 2.2.2 { ALTER TABLE t0 RENAME TO t1; } do_execsql_test 2.2.3 { INSERT INTO vt0(vt0) VALUES('integrity-check'); } do_execsql_test 2.2.4 { INSERT INTO vt0(c0) VALUES ('abc'); COMMIT; } do_execsql_test 2.2.5 { INSERT INTO vt0(vt0) VALUES('integrity-check'); } #------------------------------------------------------------------------- reset_db do_execsql_test 3.0 { CREATE VIRTUAL TABLE vt0 USING fts5(a); PRAGMA reverse_unordered_selects = true; INSERT INTO vt0 VALUES('365062398'), (0), (0); INSERT INTO vt0(vt0, rank) VALUES('pgsz', '38'); } do_execsql_test 3.1 { UPDATE vt0 SET a = 399905135; -- unexpected: database disk image is malformed } do_execsql_test 3.2 { INSERT INTO vt0(vt0) VALUES('integrity-check'); } #------------------------------------------------------------------------- reset_db do_execsql_test 4.0 { CREATE VIRTUAL TABLE vt0 USING fts5(c0); INSERT INTO vt0(c0) VALUES ('xyz'); } do_execsql_test 4.1 { BEGIN; INSERT INTO vt0(c0) VALUES ('abc'); INSERT INTO vt0(vt0) VALUES('rebuild'); COMMIT; } do_execsql_test 4.2 { INSERT INTO vt0(vt0) VALUES('integrity-check'); } do_execsql_test 4.3 { BEGIN; INSERT INTO vt0(vt0) VALUES('rebuild'); INSERT INTO vt0(vt0) VALUES('rebuild'); COMMIT; } do_execsql_test 4.4 { INSERT INTO vt0(vt0) VALUES('integrity-check'); } #------------------------------------------------------------------------- # Ticket [81a7f7b9]. # reset_db do_execsql_test 5.0 { CREATE VIRTUAL TABLE vt0 USING fts5(c0, c1); INSERT INTO vt0(vt0, rank) VALUES('pgsz', '65536'); WITH s(i) AS ( SELECT 1 UNION ALL SELECT i+1 FROM s WHERE i<1236 ) INSERT INTO vt0(c0) SELECT '0' FROM s; } {} do_execsql_test 5.1 { UPDATE vt0 SET c1 = 'T,D&p^y/7#3*v<b<4j7|f'; } do_execsql_test 5.2 { INSERT INTO vt0(vt0) VALUES('integrity-check'); } do_catchsql_test 5.3 { INSERT INTO vt0(vt0, rank) VALUES('pgsz', '65537'); } {1 {SQL logic error}} #------------------------------------------------------------------------- # Ticket [d392017c]. # reset_db do_execsql_test 6.0 { CREATE VIRTUAL TABLE vt0 USING fts5(c0); WITH s(i) AS ( SELECT 1 UNION ALL SELECT i+1 FROM s WHERE i<10000 ) INSERT INTO vt0(c0) SELECT '0' FROM s; INSERT INTO vt0(vt0, rank) VALUES('crisismerge', 2000); INSERT INTO vt0(vt0, rank) VALUES('automerge', 0); } {} do_execsql_test 6.1 { INSERT INTO vt0(vt0) VALUES('rebuild'); } #------------------------------------------------------------------------- # reset_db do_execsql_test 7.0 { CREATE VIRTUAL TABLE t1 USING fts5(x); INSERT INTO t1(rowid, x) VALUES(1, 'hello world'); INSERT INTO t1(rowid, x) VALUES(2, 'well said'); INSERT INTO t1(rowid, x) VALUES(3, 'hello said'); INSERT INTO t1(rowid, x) VALUES(4, 'well world'); CREATE TABLE t2 (a, b); INSERT INTO t2 VALUES(1, 'hello'); INSERT INTO t2 VALUES(2, 'world'); INSERT INTO t2 VALUES(3, 'said'); INSERT INTO t2 VALUES(4, 'hello'); } do_execsql_test 7.1 { SELECT rowid FROM t1 WHERE (rowid, x) IN (SELECT a, b FROM t2); } do_execsql_test 7.2 { SELECT rowid FROM t1 WHERE rowid=2 AND t1 = 'hello'; } #------------------------------------------------------------------------- # reset_db do_execsql_test 8.0 { CREATE VIRTUAL TABLE vt0 USING fts5(c0, tokenize = "ascii", prefix = 1); INSERT INTO vt0(c0) VALUES (x'd1'); } do_execsql_test 8.1 { INSERT INTO vt0(vt0) VALUES('integrity-check'); } #------------------------------------------------------------------------- # reset_db do_execsql_test 9.0 { CREATE VIRTUAL TABLE t1 using FTS5(mailcontent); insert into t1(rowid, mailcontent) values (-4764623217061966105, 'we are going to upgrade'), (8324454597464624651, 'we are going to upgrade'); } do_execsql_test 9.1 { INSERT INTO t1(t1) VALUES('integrity-check'); } do_execsql_test 9.2 { SELECT rowid FROM t1('upgrade'); } { -4764623217061966105 8324454597464624651 } #------------------------------------------------------------------------- # reset_db do_execsql_test 10.0 { CREATE VIRTUAL TABLE vt1 USING fts5(c1, c2, prefix = 1, tokenize = "ascii"); INSERT INTO vt1 VALUES (x'e4', '䔬'); } do_execsql_test 10.1 { SELECT quote(CAST(c1 AS blob)), quote(CAST(c2 AS blob)) FROM vt1 } {X'E4' X'E494AC'} do_execsql_test 10.2 { INSERT INTO vt1(vt1) VALUES('integrity-check'); } #------------------------------------------------------------------------- # reset_db do_execsql_test 11.0 { CREATE VIRTUAL TABLE vt0 USING fts5( c0, prefix = 71, tokenize = "porter ascii", prefix = 9 ); } {} do_execsql_test 11.1 { BEGIN; INSERT INTO vt0(c0) VALUES (x'e8'); } do_execsql_test 11.2 { INSERT INTO vt0(vt0) VALUES('integrity-check'); } #------------------------------------------------------------------------- # Ticket [752fdbf6] # reset_db do_execsql_test 11.0 { PRAGMA encoding = 'UTF-16'; CREATE VIRTUAL TABLE vt0 USING fts5(c0, c1); INSERT INTO vt0(vt0, rank) VALUES('pgsz', '37'); INSERT INTO vt0(c0, c1) VALUES (0.66077, 1957391816); } do_execsql_test 11.1 { INSERT INTO vt0(vt0) VALUES('integrity-check'); } #------------------------------------------------------------------------- # Ticket [7c0e06b16] # do_execsql_test 12.0 { CREATE TABLE t1(a, b, rank); INSERT INTO t1 VALUES('a', 'hello', ''); INSERT INTO t1 VALUES('b', 'world', ''); CREATE VIRTUAL TABLE ft USING fts5(a); INSERT INTO ft VALUES('b'); INSERT INTO ft VALUES('y'); CREATE TABLE t2(x, y, ft); INSERT INTO t2 VALUES(1, 2, 'x'); INSERT INTO t2 VALUES(3, 4, 'b'); } do_execsql_test 12.1 { SELECT * FROM t1 NATURAL JOIN ft WHERE ft MATCH('b') } {b world {}} do_execsql_test 12.2 { SELECT * FROM ft NATURAL JOIN t1 WHERE ft MATCH('b') } {b world {}} do_execsql_test 12.3 { SELECT * FROM t2 JOIN ft USING (ft) } {3 4 b b} #------------------------------------------------------------------------- # Forum post https://sqlite.org/forum/forumpost/21127c1160 # reset_db sqlite3_db_config db DEFENSIVE 1 do_execsql_test 13.0 { CREATE TABLE a (id INTEGER PRIMARY KEY, name TEXT); CREATE VIRTUAL TABLE b USING fts5(name); CREATE TRIGGER a_trigger AFTER INSERT ON a BEGIN INSERT INTO b (name) VALUES ('foo'); END; } do_test 13.1 { set ::STMT [ sqlite3_prepare db "INSERT INTO a VALUES (1, 'foo') RETURNING id;" -1 dummy ] sqlite3_step $::STMT } {SQLITE_ROW} do_test 13.2 { sqlite3_finalize $::STMT } {SQLITE_OK} do_test 13.3 { sqlite3_errmsg db } {not an error} #------------------------------------------------------------------------- reset_db db close sqlite3 db test.db -uri 1 do_execsql_test 14.0 { PRAGMA locking_mode=EXCLUSIVE; BEGIN; ATTACH 'file:/one?vfs=memdb' AS aux1; ATTACH 'file:/one?vfs=memdb' AS aux2; CREATE VIRTUAL TABLE t1 USING fts5(x); } {exclusive} do_catchsql_test 14.1 { ANALYZE; } {1 {database is locked}} do_catchsql_test 14.2 { COMMIT; } {1 {database is locked}} do_catchsql_test 14.3 { COMMIT; } {1 {database is locked}} do_catchsql_test 14.4 { ROLLBACK; } {0 {}} #------------------------------------------------------------------------- reset_db sqlite3 db2 test.db do_execsql_test 15.0 { CREATE TABLE t1(a, b); BEGIN; SELECT * FROM t1; } do_execsql_test -db db2 15.1 { BEGIN; CREATE VIRTUAL TABLE x1 USING fts5(y); } do_test 15.2 { list [catch { db2 eval COMMIT } msg] $msg } {1 {database is locked}} do_execsql_test -db db2 15.3 { SAVEPOINT one; } {} do_execsql_test 15.4 END do_test 15.4 { list [catch { db2 eval COMMIT } msg] $msg } {0 {}} #------------------------------------------------------------------------- reset_db forcedelete test.db2 sqlite3 db2 test.db do_execsql_test 16.0 { ATTACH 'test.db2' AS aux; CREATE TABLE aux.t2(x,y); INSERT INTO t2 VALUES(1, 2); CREATE VIRTUAL TABLE x1 USING fts5(a); BEGIN; INSERT INTO x1 VALUES('abc'); INSERT INTO t2 VALUES(3, 4); } do_execsql_test -db db2 16.1 { ATTACH 'test.db2' AS aux; BEGIN; SELECT * FROM t2 } {1 2} do_catchsql_test 16.2 { COMMIT; } {1 {database is locked}} do_execsql_test 16.3 { INSERT INTO x1 VALUES('def'); } do_execsql_test -db db2 16.4 { END } do_execsql_test 16.5 { COMMIT } do_execsql_test -db db2 16.6 { SELECT * FROM x1 } {abc def} finish_test |
Added ext/fts5/test/fts5multi.test.
> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 | # 2014 September 13 # # The author disclaims copyright to this source code. In place of # a legal notice, here is a blessing: # # May you do good and not evil. # May you find forgiveness for yourself and forgive others. # May you share freely, never taking more than you give. # #************************************************************************* # This file implements regression tests for SQLite library. The # focus of this script is testing the FTS5 module. # source [file join [file dirname [info script]] fts5_common.tcl] set testprefix fts5multi # If SQLITE_ENABLE_FTS5 is not defined, omit this file. ifcapable !fts5 { finish_test return } fts5_aux_test_functions db do_execsql_test 1.0 { CREATE VIRTUAL TABLE t1 USING fts5(a, b, c); INSERT INTO t1 VALUES('gg bb bb' ,'gg ff gg' ,'ii ii'); INSERT INTO t1 VALUES('dd dd hh kk','jj' ,'aa'); INSERT INTO t1 VALUES('kk gg ee' ,'hh cc' ,'hh jj aa cc'); INSERT INTO t1 VALUES('hh' ,'bb jj cc' ,'kk ii'); INSERT INTO t1 VALUES('kk dd kk ii','aa ee aa' ,'ee'); INSERT INTO t1 VALUES('ee' ,'ff gg kk aa','ee ff ee'); INSERT INTO t1 VALUES('ff jj' ,'gg ee' ,'kk ee gg kk'); INSERT INTO t1 VALUES('ff ee dd hh','kk ee' ,'gg dd'); INSERT INTO t1 VALUES('bb' ,'aa' ,'bb aa'); INSERT INTO t1 VALUES('hh cc bb' ,'ff bb' ,'cc'); INSERT INTO t1 VALUES('jj' ,'ff dd bb aa','dd dd ff ff'); INSERT INTO t1 VALUES('ff dd gg dd','gg aa bb ff','cc'); INSERT INTO t1 VALUES('ff aa cc jj','kk' ,'ii dd'); INSERT INTO t1 VALUES('jj dd' ,'cc' ,'ii hh ee aa'); INSERT INTO t1 VALUES('ff ii hh' ,'dd' ,'gg'); INSERT INTO t1 VALUES('ff dd gg hh','hh' ,'ff dd'); INSERT INTO t1 VALUES('cc cc' ,'ff dd ff' ,'bb'); INSERT INTO t1 VALUES('ii' ,'bb ii' ,'jj kk'); INSERT INTO t1 VALUES('ff hh' ,'hh bb' ,'bb dd ee'); INSERT INTO t1 VALUES('jj kk' ,'jj' ,'gg ff cc'); INSERT INTO t1 VALUES('dd kk' ,'ii gg' ,'dd'); INSERT INTO t1 VALUES('cc' ,'aa ff' ,'ii'); INSERT INTO t1 VALUES('bb ff bb ii','bb kk bb aa','hh ff ii dd'); INSERT INTO t1 VALUES('aa' ,'ee bb jj jj','dd'); INSERT INTO t1 VALUES('kk dd cc' ,'aa jj' ,'ee aa ff'); INSERT INTO t1 VALUES('aa gg aa' ,'jj' ,'ii kk hh gg'); INSERT INTO t1 VALUES('ff hh aa' ,'jj ii' ,'hh dd bb jj'); INSERT INTO t1 VALUES('hh' ,'aa gg kk' ,'bb ee'); INSERT INTO t1 VALUES('bb' ,'ee' ,'gg'); INSERT INTO t1 VALUES('dd kk' ,'kk bb aa' ,'ee'); } foreach {tn c1 e1 c2 e2} { 1 t1 aa t1 bb 2 a aa b bb 3 a "aa OR bb OR cc" b "jj OR ii OR hh" 4 t1 "aa AND bb" t1 "cc" 5 c "kk" b "aa OR bb OR cc OR dd OR ee" } { if {$c1=="t1"} { set lhs "( $e1 )" } else { set lhs "$c1 : ( $e1 )" } if {$c2=="t1"} { set rhs "( $e2 )" } else { set rhs "$c2 : ( $e2 )" } set q1 "t1 MATCH '($lhs) AND ($rhs)'" set q2 "$c1 MATCH '$e1' AND $c2 MATCH '$e2'" set ret [execsql "SELECT rowid FROM t1 WHERE $q1"] set N [llength $ret] do_execsql_test 1.$tn.1.($N) "SELECT rowid FROM t1 WHERE $q2" $ret set ret [execsql "SELECT fts5_test_poslist(t1) FROM t1 WHERE $q1"] do_execsql_test 1.$tn.2.($N) " SELECT fts5_test_poslist(t1) FROM t1 WHERE $q2 " $ret } do_catchsql_test 2.1.1 { SELECT rowid FROM t1 WHERE t1 MATCH '(NOT' AND t1 MATCH 'aa bb'; } {1 {fts5: syntax error near "NOT"}} do_catchsql_test 2.1.2 { SELECT rowid FROM t1 WHERE t1 MATCH 'aa bb' AND t1 MATCH '(NOT'; } {1 {fts5: syntax error near "NOT"}} finish_test |
Changes to ext/fts5/test/fts5plan.test.
︙ | ︙ | |||
26 27 28 29 30 31 32 | CREATE VIRTUAL TABLE f1 USING fts5(ff); } do_eqp_test 1.1 { SELECT * FROM t1, f1 WHERE f1 MATCH t1.x } { QUERY PLAN | | | | | | | | | 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 | CREATE VIRTUAL TABLE f1 USING fts5(ff); } do_eqp_test 1.1 { SELECT * FROM t1, f1 WHERE f1 MATCH t1.x } { QUERY PLAN |--SCAN t1 `--SCAN f1 VIRTUAL TABLE INDEX 0:M1 } do_eqp_test 1.2 { SELECT * FROM t1, f1 WHERE f1 > t1.x } { QUERY PLAN |--SCAN f1 VIRTUAL TABLE INDEX 0: `--SCAN t1 } do_eqp_test 1.3 { SELECT * FROM f1 WHERE f1 MATCH ? ORDER BY ff } { QUERY PLAN |--SCAN f1 VIRTUAL TABLE INDEX 0:M1 `--USE TEMP B-TREE FOR ORDER BY } do_eqp_test 1.4 { SELECT * FROM f1 ORDER BY rank } { QUERY PLAN |--SCAN f1 VIRTUAL TABLE INDEX 0: `--USE TEMP B-TREE FOR ORDER BY } do_eqp_test 1.5 { SELECT * FROM f1 WHERE rank MATCH ? } {SCAN f1 VIRTUAL TABLE INDEX 0:r} finish_test |
Added ext/fts5/test/fts5prefix2.test.
> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 | # 2020 Dec 3 # # The author disclaims copyright to this source code. In place of # a legal notice, here is a blessing: # # May you do good and not evil. # May you find forgiveness for yourself and forgive others. # May you share freely, never taking more than you give. # #*********************************************************************** # # This file contains tests focused on prefix indexes. # source [file join [file dirname [info script]] fts5_common.tcl] set testprefix fts5prefix2 # If SQLITE_ENABLE_FTS5 is defined, omit this file. ifcapable !fts5 { finish_test return } foreach p {3 2 1} { reset_db do_execsql_test 1.$p.0 " CREATE VIRTUAL TABLE t1 USING fts5(xyz, prefix=$p); " do_execsql_test 1.$p.1 { INSERT INTO t1 VALUES ('May you do good and not evil.'), ('May you find forgiveness for yourself and forgive others.'), ('May you share freely, never taking more than you give f.'); } do_execsql_test 1.$p.2 { SELECT highlight(t1, 0, '[', ']') FROM t1('f*'); } { {May you [find] [forgiveness] [for] yourself and [forgive] others.} {May you share [freely], never taking more than you give [f].} } } do_execsql_test 2.0 { CREATE VIRTUAL TABLE t2 USING fts5(one, prefix=3); INSERT INTO t2 VALUES('top'); INSERT INTO t2 VALUES('to'); INSERT INTO t2 VALUES('tommy'); } do_execsql_test 2.1 { SELECT * FROM t2('to*'); } {top to tommy} finish_test |
Changes to ext/fts5/test/fts5rank.test.
︙ | ︙ | |||
157 158 159 160 161 162 163 164 165 | ) INSERT INTO ttt SELECT 'word ' || i FROM s; } do_execsql_test 5.1 { SELECT rowid FROM ttt('word') WHERE rowid BETWEEN 30 AND 40 ORDER BY rank; } {30 31 32 33 34 35 36 37 38 39 40} finish_test | > > > > > > > > > > > > > > > > > > | 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 | ) INSERT INTO ttt SELECT 'word ' || i FROM s; } do_execsql_test 5.1 { SELECT rowid FROM ttt('word') WHERE rowid BETWEEN 30 AND 40 ORDER BY rank; } {30 31 32 33 34 35 36 37 38 39 40} #------------------------------------------------------------------------- reset_db do_execsql_test 6.0 { CREATE VIRTUAL TABLE "My.Table" USING fts5(Text); INSERT INTO "My.Table" VALUES ('hello this is a test'); INSERT INTO "My.Table" VALUES ('of trying to order by'); INSERT INTO "My.Table" VALUES ('rank on an fts5 table'); INSERT INTO "My.Table" VALUES ('that have periods in'); INSERT INTO "My.Table" VALUES ('the table names.'); INSERT INTO "My.Table" VALUES ('table table table'); } do_execsql_test 6.1 { SELECT * FROM "My.Table" WHERE Text MATCH 'table' ORDER BY rank; } { {table table table} {the table names.} {rank on an fts5 table} } finish_test |
Added ext/fts5/test/fts5savepoint.test.
> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 | # 2019 Dec 26 # # The author disclaims copyright to this source code. In place of # a legal notice, here is a blessing: # # May you do good and not evil. # May you find forgiveness for yourself and forgive others. # May you share freely, never taking more than you give. # #*********************************************************************** # source [file join [file dirname [info script]] fts5_common.tcl] set testprefix fts5savepoint # If SQLITE_ENABLE_FTS5 is defined, omit this file. ifcapable !fts5 { finish_test return } do_execsql_test 1.0 { CREATE VIRTUAL TABLE ft USING fts5(c); BEGIN; SAVEPOINT one; INSERT INTO ft VALUES('a'); SAVEPOINT two; INSERT INTO ft VALUES('b'); RELEASE two; SAVEPOINT four; INSERT INTO ft VALUES('c'); RELEASE four; SAVEPOINT three; INSERT INTO ft VALUES('d'); ROLLBACK TO three; COMMIT; SELECT * FROM ft } {a b c} reset_db do_catchsql_test 2.0 { CREATE VIRTUAL TABLE ft1 USING fts5(c); CREATE VIRTUAL TABLE ft2 USING fts5(c); DROP TABLE ft2_idx; BEGIN; INSERT INTO ft2 VALUES('a'); INSERT INTO ft1 VALUES('a'); SAVEPOINT two; INSERT INTO ft1 VALUES('b'); COMMIT; } {1 {SQL logic error}} reset_db ifcapable fts3 { do_execsql_test 3.0 { CREATE VIRTUAL TABLE vt0 USING fts5(c0); CREATE VIRTUAL TABLE vt1 USING fts4(c0); INSERT INTO vt1(c0) VALUES(0); } do_execsql_test 3.1 { BEGIN; UPDATE vt1 SET c0 = 0; INSERT INTO vt1(c0) VALUES (0), (0); UPDATE vt0 SET c0 = 0; INSERT INTO vt1(c0) VALUES (0); UPDATE vt1 SET c0 = 0; INSERT INTO vt1(vt1) VALUES('automerge=1'); UPDATE vt1 SET c0 = 0; } do_catchsql_test 3.2 { DROP TABLE vt1; } {1 {SQL logic error}} do_execsql_test 3.3 { SAVEPOINT x; INSERT INTO vt0 VALUES('x'); COMMIT; INSERT INTO vt0(vt0) VALUES('integrity-check'); } } finish_test |
Changes to ext/fts5/test/fts5simple.test.
︙ | ︙ | |||
463 464 465 466 467 468 469 470 | } {11111 11112} do_execsql_test 21.3 { DELETE FROM x1 WHERE rowid=11111; INSERT INTO x1(x1) VALUES('integrity-check'); SELECT rowid FROM x1($doc); } {11112} finish_test | > > > > > > > > > > > > > | 463 464 465 466 467 468 469 470 471 472 473 474 475 476 477 478 479 480 481 482 483 | } {11111 11112} do_execsql_test 21.3 { DELETE FROM x1 WHERE rowid=11111; INSERT INTO x1(x1) VALUES('integrity-check'); SELECT rowid FROM x1($doc); } {11112} #------------------------------------------------------------------------- reset_db do_execsql_test 22.0 { CREATE VIRTUAL TABLE x1 USING fts5(x); INSERT INTO x1(x) VALUES('a b c'); INSERT INTO x1(x) VALUES('x y z'); INSERT INTO x1(x) VALUES('c b a'); INSERT INTO x1(x) VALUES('z y x'); } do_catchsql_test 22.1 {SELECT * FROM x1('')} {1 {fts5: syntax error near ""}} do_catchsql_test 22.2 {SELECT * FROM x1(NULL)} {1 {fts5: syntax error near ""}} finish_test |
Changes to ext/fts5/test/fts5tok1.test.
︙ | ︙ | |||
106 107 108 109 110 111 112 113 114 115 | CREATE VIRTUAL TABLE tX USING fts5tokenize(nosuchtokenizer); } {1 {vtable constructor failed: tX}} do_catchsql_test 2.1 { CREATE VIRTUAL TABLE t4 USING fts5tokenize; SELECT * FROM t4; } {1 {SQL logic error}} finish_test | > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 | CREATE VIRTUAL TABLE tX USING fts5tokenize(nosuchtokenizer); } {1 {vtable constructor failed: tX}} do_catchsql_test 2.1 { CREATE VIRTUAL TABLE t4 USING fts5tokenize; SELECT * FROM t4; } {1 {SQL logic error}} #------------------------------------------------------------------------- # Embedded 0x00 characters. # reset_db do_execsql_test 3.1.0 { CREATE VIRTUAL TABLE t1 USING fts5(z); CREATE VIRTUAL TABLE tt USING fts5vocab(t1, 'instance'); INSERT INTO t1 VALUES('abc' || char(0) || 'def'); SELECT * FROM tt; } { abc 1 z 0 def 1 z 1 } do_execsql_test 3.1.1 { SELECT hex(z) FROM t1; } {61626300646566} do_execsql_test 3.1.2 { INSERT INTO t1(t1) VALUES('integrity-check'); } {} do_execsql_test 3.2.0 { CREATE VIRTUAL TABLE t2 USING fts5(z, tokenize="unicode61 categories 'L* N* Co Cc'" ); CREATE VIRTUAL TABLE tu USING fts5vocab(t2, 'instance'); INSERT INTO t2 VALUES('abc' || char(0) || 'def'); SELECT * FROM tu; } { abc 1 z 0 def 1 z 1 } do_execsql_test 3.2.1 { SELECT hex(z) FROM t1; } {61626300646566} do_execsql_test 3.2.2 { INSERT INTO t1(t1) VALUES('integrity-check'); } {} finish_test |
Added ext/fts5/test/fts5trigram.test.
> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 | # 2020 September 30 # # The author disclaims copyright to this source code. In place of # a legal notice, here is a blessing: # # May you do good and not evil. # May you find forgiveness for yourself and forgive others. # May you share freely, never taking more than you give. # #************************************************************************* # # Tests for the fts5 "trigram" tokenizer. # source [file join [file dirname [info script]] fts5_common.tcl] ifcapable !fts5 { finish_test ; return } set ::testprefix fts5trigram do_execsql_test 1.0 { CREATE VIRTUAL TABLE t1 USING fts5(y, tokenize=trigram); INSERT INTO t1 VALUES('abcdefghijklm'); INSERT INTO t1 VALUES('กรุงเทพมหานคร'); } foreach {tn s res} { 1 abc "(abc)defghijklm" 2 defgh "abc(defgh)ijklm" 3 abcdefghijklm "(abcdefghijklm)" 4 กรุ "(กรุ)งเทพมหานคร" 5 งเทพมห "กรุ(งเทพมห)านคร" 6 กรุงเทพมหานคร "(กรุงเทพมหานคร)" 7 Abc "(abc)defghijklm" 8 deFgh "abc(defgh)ijklm" 9 aBcdefGhijKlm "(abcdefghijklm)" } { do_execsql_test 1.1.$tn { SELECT highlight(t1, 0, '(', ')') FROM t1($s) } $res } do_execsql_test 1.2.0 { SELECT fts5_expr('ABCD', 'tokenize=trigram') } {{"abc" + "bcd"}} do_execsql_test 1.2.1 { SELECT * FROM t1 WHERE y LIKE ? ESCAPE 'a' } foreach {tn like res} { 1 {%cDef%} 1 2 {cDef%} {} 3 {%f%} 1 4 {%f_h%} 1 5 {%f_g%} {} 6 {abc%klm} 1 7 {ABCDEFG%} 1 8 {%รุงเ%} 2 } { do_execsql_test 1.3.$tn { SELECT rowid FROM t1 WHERE y LIKE $like } $res } #------------------------------------------------------------------------- reset_db do_execsql_test 2.0 { CREATE VIRTUAL TABLE t1 USING fts5(y, tokenize="trigram case_sensitive 1"); INSERT INTO t1 VALUES('abcdefghijklm'); INSERT INTO t1 VALUES('กรุงเทพมหานคร'); } foreach {tn s res} { 1 abc "(abc)defghijklm" 2 defgh "abc(defgh)ijklm" 3 abcdefghijklm "(abcdefghijklm)" 4 กรุ "(กรุ)งเทพมหานคร" 5 งเทพมห "กรุ(งเทพมห)านคร" 6 กรุงเทพมหานคร "(กรุงเทพมหานคร)" 7 Abc "" 8 deFgh "" 9 aBcdefGhijKlm "" } { do_execsql_test 2.1.$tn { SELECT highlight(t1, 0, '(', ')') FROM t1($s) } $res } foreach {tn like res} { 1 {%cDef%} 1 2 {cDef%} {} 3 {%f%} 1 4 {%f_h%} 1 5 {%f_g%} {} 6 {abc%klm} 1 7 {ABCDEFG%} 1 8 {%รุงเ%} 2 } { do_execsql_test 2.2.$tn { SELECT rowid FROM t1 WHERE y LIKE $like } $res } foreach {tn like res} { 1 {*cdef*} 1 2 {cdef*} {} 3 {*f*} 1 4 {*f?h*} 1 5 {*f?g*} {} 6 {abc*klm} 1 7 {abcdefg*} 1 8 {*รุงเ*} 2 9 {abc[d]efg*} 1 10 {abc[]d]efg*} 1 11 {abc[^]d]efg*} {} 12 {abc[^]XYZ]efg*} 1 } { do_execsql_test 2.3.$tn { SELECT rowid FROM t1 WHERE y GLOB $like } $res } do_execsql_test 2.3.null.1 { SELECT rowid FROM t1 WHERE y LIKE NULL } #------------------------------------------------------------------------- reset_db do_catchsql_test 3.1 { CREATE VIRTUAL TABLE ttt USING fts5(c, tokenize="trigram case_sensitive 2"); } {1 {error in tokenizer constructor}} do_catchsql_test 3.2 { CREATE VIRTUAL TABLE ttt USING fts5(c, tokenize="trigram case_sensitive 11"); } {1 {error in tokenizer constructor}} do_catchsql_test 3.3 { CREATE VIRTUAL TABLE ttt USING fts5(c, "tokenize=trigram case_sensitive 1"); } {0 {}} #------------------------------------------------------------------------- reset_db do_execsql_test 4.0 { CREATE VIRTUAL TABLE t0 USING fts5(b, tokenize = "trigram"); } do_execsql_test 4.1 { INSERT INTO t0 VALUES (x'000b01'); } do_execsql_test 4.2 { INSERT INTO t0(t0) VALUES('integrity-check'); } #------------------------------------------------------------------------- reset_db foreach_detail_mode $::testprefix { foreach {ci} {0 1} { reset_db do_execsql_test 5.cs=$ci.0.1 " CREATE VIRTUAL TABLE t1 USING fts5( y, tokenize=\"trigram case_sensitive $ci\", detail=%DETAIL% ); " do_execsql_test 5.cs=$ci.0.2 { INSERT INTO t1 VALUES('abcdefghijklm'); INSERT INTO t1 VALUES('กรุงเทพมหานคร'); } foreach {tn like res} { 1 {%cDef%} 1 2 {cDef%} {} 3 {%f%} 1 4 {%f_h%} 1 5 {%f_g%} {} 6 {abc%klm} 1 7 {ABCDEFG%} 1 8 {%รุงเ%} 2 } { do_execsql_test 5.cs=$ci.1.$tn { SELECT rowid FROM t1 WHERE y LIKE $like } $res } } } do_execsql_test 6.0 { CREATE VIRTUAL TABLE ci0 USING fts5(x, tokenize="trigram"); CREATE VIRTUAL TABLE ci1 USING fts5(x, tokenize="trigram case_sensitive 1"); } # LIKE and GLOB both work with case-insensitive tokenizers. Only GLOB works # with case-sensitive. do_eqp_test 6.1 { SELECT * FROM ci0 WHERE x LIKE ? } {VIRTUAL TABLE INDEX 0:L0} do_eqp_test 6.2 { SELECT * FROM ci0 WHERE x GLOB ? } {VIRTUAL TABLE INDEX 0:G0} do_eqp_test 6.3 { SELECT * FROM ci1 WHERE x LIKE ? } {{SCAN ci1 VIRTUAL TABLE INDEX 0:}} do_eqp_test 6.4 { SELECT * FROM ci1 WHERE x GLOB ? } {VIRTUAL TABLE INDEX 0:G0} finish_test |
Added ext/fts5/test/fts5ubsan.test.
> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 | # 2022 August 9 # # The author disclaims copyright to this source code. In place of # a legal notice, here is a blessing: # # May you do good and not evil. # May you find forgiveness for yourself and forgive others. # May you share freely, never taking more than you give. # #*********************************************************************** # # This test is focused on edge cases that cause ubsan errors. # source [file join [file dirname [info script]] fts5_common.tcl] set testprefix fts5ubsan # If SQLITE_ENABLE_FTS5 is defined, omit this file. ifcapable !fts5 { finish_test return } do_execsql_test 1.0 { CREATE VIRTUAL TABLE x1 USING fts5(x); } set BIG 9000000000000000000 set SMALL -9000000000000000000 do_execsql_test 1.1 { BEGIN; INSERT INTO x1 (rowid, x) VALUES($BIG, 'aaa aba acc'); INSERT INTO x1 (rowid, x) VALUES($SMALL, 'aaa abc acb'); COMMIT; } do_execsql_test 1.2 { SELECT rowid, x FROM x1('ab*'); } [list $SMALL {aaa abc acb} $BIG {aaa aba acc}] do_execsql_test 1.3 { SELECT rowid, x FROM x1('ac*'); } [list $SMALL {aaa abc acb} $BIG {aaa aba acc}] reset_db do_execsql_test 2.0 { CREATE VIRTUAL TABLE x1 USING fts5(x); } do_execsql_test 2.1 { INSERT INTO x1 (rowid, x) VALUES($BIG, 'aaa aba acc'); INSERT INTO x1 (rowid, x) VALUES($SMALL, 'aaa abc acb'); } do_execsql_test 2.2 { INSERT INTO x1 (x1) VALUES('optimize'); } finish_test |
Changes to ext/fts5/test/fts5unicode3.test.
︙ | ︙ | |||
17 18 19 20 21 22 23 | # If SQLITE_ENABLE_FTS5 is defined, omit this file. ifcapable !fts5 { finish_test return } proc fts3_unicode_path {file} { | | | 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 | # If SQLITE_ENABLE_FTS5 is defined, omit this file. ifcapable !fts5 { finish_test return } proc fts3_unicode_path {file} { file join .. [file dirname [info script]] .. .. fts3 unicode $file } source [fts3_unicode_path parseunicode.tcl] set testprefix fts5unicode3 set CF [fts3_unicode_path CaseFolding.txt] set UD [fts3_unicode_path UnicodeData.txt] |
︙ | ︙ |
Changes to ext/fts5/test/fts5update.test.
︙ | ︙ | |||
110 111 112 113 114 115 116 117 118 119 | for {set i 0} {$i < 1000} {incr i} { execsql { UPDATE x2 SET x=x WHERE rowid=2 } } } {} do_execsql_test 2.2.integrity { INSERT INTO x2(x2) VALUES('integrity-check'); } } finish_test | > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 | for {set i 0} {$i < 1000} {incr i} { execsql { UPDATE x2 SET x=x WHERE rowid=2 } } } {} do_execsql_test 2.2.integrity { INSERT INTO x2(x2) VALUES('integrity-check'); } #------------------------------------------------------------------------- # do_execsql_test 3.0 { CREATE VIRTUAL TABLE x3 USING fts5(x, detail=%DETAIL%); INSERT INTO x3 VALUES('one'); INSERT INTO x3 VALUES('two'); INSERT INTO x3 VALUES('one'); INSERT INTO x3 VALUES('two'); INSERT INTO x3 VALUES('one'); } do_test 3.1 { db eval { SELECT * FROM x3('one') } { db eval { INSERT INTO x3(x3) VALUES('optimize'); } } } {} do_execsql_test 4.0 { CREATE VIRTUAL TABLE x4 USING fts5(a, detail=%DETAIL%); INSERT INTO x4 VALUES('one two three'); INSERT INTO x4(rowid, a) VALUES('2', 'one two three'); INSERT INTO x4(rowid, a) VALUES('3.0', 'one two three'); } do_catchsql_test 4.1 { INSERT INTO x4(rowid, a) VALUES('four', 'one two three'); } {1 {datatype mismatch}} do_catchsql_test 4.2 { UPDATE x4 SET rowid = 'four' WHERE rowid=1; } {1 {datatype mismatch}} } reset_db do_catchsql_test 4.0 { CREATE VIRTUAL TABLE t1 USING fts5(a,b,c); } {0 {}} do_catchsql_test 4.1 { DELETE FROM t1 WHERE t1 MATCH 'f*'; } {0 {}} finish_test |
Changes to ext/fts5/test/fts5vocab.test.
︙ | ︙ | |||
75 76 77 78 79 80 81 | } { 0 term {} 0 {} 0 1 col {} 0 {} 0 2 doc {} 0 {} 0 3 cnt {} 0 {} 0 } | | | | 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 | } { 0 term {} 0 {} 0 1 col {} 0 {} 0 2 doc {} 0 {} 0 3 cnt {} 0 {} 0 } do_execsql_test 1.2.1 { SELECT * FROM v1 } {} do_execsql_test 1.2.2 { SELECT * FROM v2 } {} do_execsql_test 1.3 { INSERT INTO t1 VALUES('x y z'); INSERT INTO t1 VALUES('x x x'); } do_execsql_test 1.4.1 { |
︙ | ︙ | |||
475 476 477 478 479 480 481 | expr [lsearch $e2 SorterSort]<0 } 1 do_test 9.6 { set e2 [db eval { EXPLAIN SELECT * FROM rrr ORDER BY term DESC }] expr [lsearch $e2 SorterSort]<0 } 0 | > > > > > | > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | 475 476 477 478 479 480 481 482 483 484 485 486 487 488 489 490 491 492 493 494 495 496 497 498 499 500 501 502 503 504 505 506 507 508 509 510 511 512 513 514 515 516 517 518 519 520 521 522 523 524 525 526 527 528 529 530 531 532 533 534 535 536 537 538 539 540 541 542 543 544 545 546 547 548 549 550 551 552 553 554 555 556 557 | expr [lsearch $e2 SorterSort]<0 } 1 do_test 9.6 { set e2 [db eval { EXPLAIN SELECT * FROM rrr ORDER BY term DESC }] expr [lsearch $e2 SorterSort]<0 } 0 #------------------------------------------------------------------------- do_execsql_test 10.0 { CREATE VIRTUAL TABLE ft USING fts5(a, b, c); CREATE VIRTUAL TABLE t2 USING fts5vocab('ft','row'); CREATE VIRTUAL TABLE t3 USING fts5vocab('ft','row'); } do_execsql_test 10.1 { BEGIN; INSERT INTO ft(b) VALUES('x y'); } do_execsql_test 10.2 { SELECT t2.term FROM t2; } {x y} do_execsql_test 10.3 { SELECT t2.term, t3.term FROM t2, t3; } {x x x y y x y y} do_execsql_test 10.4 { COMMIT; } do_execsql_test 10.5 { BEGIN; INSERT INTO ft(a) VALUES('1 2 3'); INSERT INTO ft(a) VALUES('4 5 6'); INSERT INTO ft(a) VALUES('1 2 3'); INSERT INTO ft(a) VALUES('4 5 6'); INSERT INTO ft(a) VALUES('1 2 3'); INSERT INTO ft(a) VALUES('4 5 6'); } do_test 10.6 { set res [list] db eval { SELECT rowid FROM ft('4') } x { db eval { SELECT * FROM t2 } lappend res $x(rowid) } db eval COMMIT set res } {3 5 7} do_execsql_test 10.6.1 { SELECT * FROM t2 WHERE term<NULL; } do_execsql_test 10.6.2 { SELECT * FROM t2 WHERE term>NULL; } do_execsql_test 10.6.3 { SELECT * FROM t2 WHERE term=NULL; } do_execsql_test 10.7.1 { SELECT * FROM t2 WHERE term<?; } do_execsql_test 10.7.2 { SELECT * FROM t2 WHERE term>?; } do_execsql_test 10.7.3 { SELECT * FROM t2 WHERE term=?; } # 2020-02-16 Detect recursively define fts5vocab() tables. # Error found by dbsqlfuzz. # reset_db do_execsql_test 11.100 { CREATE VIRTUAL TABLE t3 USING fts5vocab(rowid , 'col'); CREATE VIRTUAL TABLE rowid USING fts5vocab(rowid , 'instance'); } {} do_catchsql_test 11.110 { SELECT rowid+1,rowid, * FROM t3 WHERE null>rowid ; } {1 {SQL logic error}} finish_test |
Changes to ext/fts5/test/fts5vocab2.test.
︙ | ︙ | |||
76 77 78 79 80 81 82 | four 2 b 0 three 2 a 0 } do_execsql_test 1.5 { DELETE FROM t1; SELECT * FROM v1; | | < | 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 | four 2 b 0 three 2 a 0 } do_execsql_test 1.5 { DELETE FROM t1; SELECT * FROM v1; } {} #------------------------------------------------------------------------- # do_execsql_test 2.0 { DROP TABLE IF EXISTS t1; DROP TABLE IF EXISTS v1; |
︙ | ︙ | |||
139 140 141 142 143 144 145 | four 2 b {} three 2 a {} } do_execsql_test 2.5 { DELETE FROM t1; SELECT * FROM v1; | | < | 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 | four 2 b {} three 2 a {} } do_execsql_test 2.5 { DELETE FROM t1; SELECT * FROM v1; } {} #------------------------------------------------------------------------- # do_execsql_test 3.0 { DROP TABLE IF EXISTS t1; DROP TABLE IF EXISTS v1; |
︙ | ︙ | |||
198 199 200 201 202 203 204 | four 2 {} {} three 2 {} {} } do_execsql_test 3.5 { DELETE FROM t1; SELECT * FROM v1; | | | > | > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 | four 2 {} {} three 2 {} {} } do_execsql_test 3.5 { DELETE FROM t1; SELECT * FROM v1; } {} #------------------------------------------------------------------------- # reset_db do_execsql_test 4.0 { CREATE VIRTUAL TABLE v1 USING fts5vocab(nosuchtable, col); } do_catchsql_test 4.1 { SELECT * FROM v1 WHERE term=='nosuchterm'; } {1 {no such fts5 table: main.nosuchtable}} do_execsql_test 4.2.1 { CREATE TABLE nosuchtable(nosuchtable, y, z); } do_catchsql_test 4.2.2 { SELECT * FROM v1 WHERE term=='nosuchterm'; } {1 {no such fts5 table: main.nosuchtable}} ifcapable fts3 { do_execsql_test 4.3.1 { DROP TABLE nosuchtable; CREATE VIRTUAL TABLE nosuchtable USING fts3(a, b); } {} do_catchsql_test 4.3.2 { SELECT * FROM v1 WHERE term=='nosuchterm'; } {1 {no such fts5 table: main.nosuchtable}} do_catchsql_test 4.3.3 { INSERT INTO nosuchtable VALUES('id', '*id'); SELECT * FROM v1 WHERE term=='nosuchterm'; } {1 {no such fts5 table: main.nosuchtable}} } #------------------------------------------------------------------------- # Check that the fts5 table cannot be written while there are vocab # cursors open. reset_db do_execsql_test 5.0 { CREATE VIRTUAL TABLE t1 USING fts5(a); CREATE VIRTUAL TABLE v1 USING fts5vocab(t1, instance); INSERT INTO t1 VALUES('one'), ('two'), ('three'), ('four'); } do_test 5.1 { list [catch { db eval { SELECT * FROM v1 } { db eval {INSERT INTO t1 VALUES('five')} } } msg] $msg } {1 {query aborted}} do_execsql_test 5.2 { SELECT * FROM t1 } {one two three four five} #------------------------------------------------------------------------- # Check that the fts5 table cannot be written while there are vocab # cursors open. reset_db do_execsql_test 5.0 { CREATE VIRTUAL TABLE t1 USING fts5(a); CREATE VIRTUAL TABLE v1 USING fts5vocab(t1, instance); WITH s(i) AS ( VALUES(1) UNION ALL SELECT i+1 FROM s WHERE i<10000 ) INSERT INTO t1 SELECT 'State Emergency Service (SES), Rural Fire Service (RFS) and Volunteers' FROM s; } do_catchsql_test 5.1 { INSERT INTO t1 SELECT rowid FROM v1 } {1 {query aborted}} do_catchsql_test 5.2 { DELETE FROM t1 WHERE rowid>100; INSERT INTO t1 SELECT randomblob(3000) FROM v1 } {1 {query aborted}} finish_test |
Changes to ext/fts5/tool/fts5txt2db.tcl.
︙ | ︙ | |||
8 9 10 11 12 13 14 15 16 17 18 19 20 21 | # May you find forgiveness for yourself and forgive others. # May you share freely, never taking more than you give. # proc process_cmdline {} { cmdline::process ::A $::argv { {fts5 "use fts5 (this is the default)"} {fts4 "use fts4"} {colsize "10 10 10" "list of column sizes"} {tblname "t1" "table name to create"} {detail "full" "Fts5 detail mode to use"} {repeat 1 "Load each file this many times"} {prefix "" "Fts prefix= option"} {trans 1 "True to use a transaction"} database | > | 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 | # May you find forgiveness for yourself and forgive others. # May you share freely, never taking more than you give. # proc process_cmdline {} { cmdline::process ::A $::argv { {fts5 "use fts5 (this is the default)"} {fts4 "use fts4"} {trigram "Use tokenize=trigram"} {colsize "10 10 10" "list of column sizes"} {tblname "t1" "table name to create"} {detail "full" "Fts5 detail mode to use"} {repeat 1 "Load each file this many times"} {prefix "" "Fts prefix= option"} {trans 1 "True to use a transaction"} database |
︙ | ︙ | |||
171 172 173 174 175 176 177 178 179 180 181 182 183 184 | set nCol [llength $A(colsize)] set cols [lrange $cols 0 [expr $nCol-1]] set sql "CREATE VIRTUAL TABLE IF NOT EXISTS $A(tblname) USING $A(fts) (" append sql [join $cols ,] if {$A(fts)=="fts5"} { append sql ",detail=$A(detail)" } append sql ", prefix='$A(prefix)');" db eval $sql return $cols } # Return a list of tokens from the named file. | > | 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 | set nCol [llength $A(colsize)] set cols [lrange $cols 0 [expr $nCol-1]] set sql "CREATE VIRTUAL TABLE IF NOT EXISTS $A(tblname) USING $A(fts) (" append sql [join $cols ,] if {$A(fts)=="fts5"} { append sql ",detail=$A(detail)" } if {$A(trigram)} { append sql ",tokenize=trigram" } append sql ", prefix='$A(prefix)');" db eval $sql return $cols } # Return a list of tokens from the named file. |
︙ | ︙ |
Changes to ext/fts5/tool/mkfts5c.tcl.
︙ | ︙ | |||
56 57 58 59 60 61 62 | # proc fts5_source_id {zDir} { set top [file dirname [file dirname $zDir]] set uuid [string trim [readfile [file join $top manifest.uuid]]] set L [split [readfile [file join $top manifest]]] set date [lindex $L [expr [lsearch -exact $L D]+1]] | > | | 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 | # proc fts5_source_id {zDir} { set top [file dirname [file dirname $zDir]] set uuid [string trim [readfile [file join $top manifest.uuid]]] set L [split [readfile [file join $top manifest]]] set date [lindex $L [expr [lsearch -exact $L D]+1]] set idx [expr {[string last . $date]-1}] set date [string range $date 0 $idx] set date [string map {T { }} $date] return "fts5: $date $uuid" } proc fts5c_init {zOut} { global G |
︙ | ︙ |
Changes to ext/icu/README.txt.
︙ | ︙ | |||
112 113 114 115 116 117 118 | 2 COMPILATION AND USAGE The easiest way to compile and use the ICU extension is to build and use it as a dynamically loadable SQLite extension. To do this using gcc on *nix: | > | | 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 | 2 COMPILATION AND USAGE The easiest way to compile and use the ICU extension is to build and use it as a dynamically loadable SQLite extension. To do this using gcc on *nix: gcc -fPIC -shared icu.c `pkg-config --libs --cflags icu-uc icu-io` \ -o libSqliteIcu.so You may need to add "-I" flags so that gcc can find sqlite3ext.h and sqlite3.h. The resulting shared lib, libSqliteIcu.so, may be loaded into sqlite in the same way as any other dynamically loadable extension. |
︙ | ︙ | |||
148 149 150 151 152 153 154 | problem and uses the same solution. However, since the ICU extension code does not include the SQLite file "limits.h", modifying the default value therein does not affect the ICU extension. The default value of SQLITE_MAX_LIKE_PATTERN_LENGTH used by the ICU extension LIKE operator is 50000, defined in source file "icu.c". | | | < < < | < < | < | 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 | problem and uses the same solution. However, since the ICU extension code does not include the SQLite file "limits.h", modifying the default value therein does not affect the ICU extension. The default value of SQLITE_MAX_LIKE_PATTERN_LENGTH used by the ICU extension LIKE operator is 50000, defined in source file "icu.c". 3.3 Collation Sequence Security Internally, SQLite assumes that indices stored in database files are sorted according to the collation sequence indicated by the SQL schema. Changing the definition of a collation sequence after an index has been built is therefore equivalent to database corruption. The SQLite library is well tested for robustness in the fact of database corruption. Database corruption may well lead to incorrect answers, but should not cause memory errors. |
Changes to ext/icu/icu.c.
︙ | ︙ | |||
139 140 141 142 143 144 145 | /* There are now 4 possibilities: ** ** 1. uPattern is an unescaped match-all character "%", ** 2. uPattern is an unescaped match-one character "_", ** 3. uPattern is an unescaped escape character, or ** 4. uPattern is to be handled as an ordinary character */ | | | 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 | /* There are now 4 possibilities: ** ** 1. uPattern is an unescaped match-all character "%", ** 2. uPattern is an unescaped match-one character "_", ** 3. uPattern is an unescaped escape character, or ** 4. uPattern is to be handled as an ordinary character */ if( uPattern==MATCH_ALL && !prevEscape && uPattern!=(uint32_t)uEsc ){ /* Case 1. */ uint8_t c; /* Skip any MATCH_ALL or MATCH_ONE characters that follow a ** MATCH_ALL. For each MATCH_ONE, skip one character in the ** test string. */ |
︙ | ︙ | |||
165 166 167 168 169 170 171 | if( icuLikeCompare(zPattern, zString, uEsc) ){ return 1; } SQLITE_ICU_SKIP_UTF8(zString); } return 0; | | | | 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 | if( icuLikeCompare(zPattern, zString, uEsc) ){ return 1; } SQLITE_ICU_SKIP_UTF8(zString); } return 0; }else if( uPattern==MATCH_ONE && !prevEscape && uPattern!=(uint32_t)uEsc ){ /* Case 2. */ if( *zString==0 ) return 0; SQLITE_ICU_SKIP_UTF8(zString); }else if( uPattern==(uint32_t)uEsc && !prevEscape ){ /* Case 3. */ prevEscape = 1; }else{ /* Case 4. */ uint32_t uString; SQLITE_ICU_READ_UTF8(zString, uString); |
︙ | ︙ | |||
295 296 297 298 299 300 301 | if( !zPattern ){ return; } pExpr = uregex_open(zPattern, -1, 0, 0, &status); if( U_SUCCESS(status) ){ sqlite3_set_auxdata(p, 0, pExpr, icuRegexpDelete); | | > | | 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 | if( !zPattern ){ return; } pExpr = uregex_open(zPattern, -1, 0, 0, &status); if( U_SUCCESS(status) ){ sqlite3_set_auxdata(p, 0, pExpr, icuRegexpDelete); pExpr = sqlite3_get_auxdata(p, 0); } if( !pExpr ){ icuFunctionError(p, "uregex_open", status); return; } } /* Configure the text that the regular expression operates on. */ uregex_setText(pExpr, zString, -1, &status); |
︙ | ︙ | |||
495 496 497 498 499 500 501 502 503 504 | } } /* ** Register the ICU extension functions with database db. */ int sqlite3IcuInit(sqlite3 *db){ static const struct IcuScalar { const char *zName; /* Function name */ unsigned char nArg; /* Number of arguments */ | > | | | | | | | | | | | | | | 496 497 498 499 500 501 502 503 504 505 506 507 508 509 510 511 512 513 514 515 516 517 518 519 520 521 522 523 524 525 526 527 528 529 530 | } } /* ** Register the ICU extension functions with database db. */ int sqlite3IcuInit(sqlite3 *db){ # define SQLITEICU_EXTRAFLAGS (SQLITE_DETERMINISTIC|SQLITE_INNOCUOUS) static const struct IcuScalar { const char *zName; /* Function name */ unsigned char nArg; /* Number of arguments */ unsigned int enc; /* Optimal text encoding */ unsigned char iContext; /* sqlite3_user_data() context */ void (*xFunc)(sqlite3_context*,int,sqlite3_value**); } scalars[] = { {"icu_load_collation",2,SQLITE_UTF8|SQLITE_DIRECTONLY,1, icuLoadCollation}, #if !defined(SQLITE_CORE) || defined(SQLITE_ENABLE_ICU) {"regexp", 2, SQLITE_ANY|SQLITEICU_EXTRAFLAGS, 0, icuRegexpFunc}, {"lower", 1, SQLITE_UTF16|SQLITEICU_EXTRAFLAGS, 0, icuCaseFunc16}, {"lower", 2, SQLITE_UTF16|SQLITEICU_EXTRAFLAGS, 0, icuCaseFunc16}, {"upper", 1, SQLITE_UTF16|SQLITEICU_EXTRAFLAGS, 1, icuCaseFunc16}, {"upper", 2, SQLITE_UTF16|SQLITEICU_EXTRAFLAGS, 1, icuCaseFunc16}, {"lower", 1, SQLITE_UTF8|SQLITEICU_EXTRAFLAGS, 0, icuCaseFunc16}, {"lower", 2, SQLITE_UTF8|SQLITEICU_EXTRAFLAGS, 0, icuCaseFunc16}, {"upper", 1, SQLITE_UTF8|SQLITEICU_EXTRAFLAGS, 1, icuCaseFunc16}, {"upper", 2, SQLITE_UTF8|SQLITEICU_EXTRAFLAGS, 1, icuCaseFunc16}, {"like", 2, SQLITE_UTF8|SQLITEICU_EXTRAFLAGS, 0, icuLikeFunc}, {"like", 3, SQLITE_UTF8|SQLITEICU_EXTRAFLAGS, 0, icuLikeFunc}, #endif /* !defined(SQLITE_CORE) || defined(SQLITE_ENABLE_ICU) */ }; int rc = SQLITE_OK; int i; for(i=0; rc==SQLITE_OK && i<(int)(sizeof(scalars)/sizeof(scalars[0])); i++){ const struct IcuScalar *p = &scalars[i]; |
︙ | ︙ |
Changes to ext/icu/sqliteicu.h.
︙ | ︙ | |||
20 21 22 23 24 25 26 | #endif /* __cplusplus */ int sqlite3IcuInit(sqlite3 *db); #ifdef __cplusplus } /* extern "C" */ #endif /* __cplusplus */ | < | 20 21 22 23 24 25 26 | #endif /* __cplusplus */ int sqlite3IcuInit(sqlite3 *db); #ifdef __cplusplus } /* extern "C" */ #endif /* __cplusplus */ |
Changes to ext/lsm1/Makefile.
︙ | ︙ | |||
39 40 41 42 43 44 45 | $(LSMDIR)/lsm-test/lsmtest_main.c $(LSMDIR)/lsm-test/lsmtest_mem.c \ $(LSMDIR)/lsm-test/lsmtest_tdb.c $(LSMDIR)/lsm-test/lsmtest_tdb3.c \ $(LSMDIR)/lsm-test/lsmtest_util.c $(LSMDIR)/lsm-test/lsmtest_win32.c # all: lsm.so | | | | 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 | $(LSMDIR)/lsm-test/lsmtest_main.c $(LSMDIR)/lsm-test/lsmtest_mem.c \ $(LSMDIR)/lsm-test/lsmtest_tdb.c $(LSMDIR)/lsm-test/lsmtest_tdb3.c \ $(LSMDIR)/lsm-test/lsmtest_util.c $(LSMDIR)/lsm-test/lsmtest_win32.c # all: lsm.so LSMOPTS += -fPIC -DLSM_MUTEX_PTHREADS=1 -I$(LSMDIR) -DHAVE_ZLIB lsm.so: $(LSMOBJ) $(TCCX) -shared -fPIC -o lsm.so $(LSMOBJ) %.o: $(LSMDIR)/%.c $(LSMHDR) sqlite3.h $(TCCX) $(LSMOPTS) -c $< lsmtest$(EXE): $(LSMOBJ) $(LSMTESTSRC) $(LSMTESTHDR) sqlite3.o # $(TCPPX) -c $(TOP)/lsm-test/lsmtest_tdb2.cc $(TCCX) $(LSMOPTS) $(LSMTESTSRC) $(LSMOBJ) sqlite3.o -o lsmtest$(EXE) $(THREADLIB) -lz |
Changes to ext/lsm1/lsm-test/lsmtest1.c.
︙ | ︙ | |||
650 651 652 653 654 655 656 | char *zName = getName3(zSystem, &aTest[i]); if( testCaseBegin(pRc, zPattern, "%s", zName) ){ doDataTest3(zSystem, &aTest[i], pRc); } testFree(zName); } } | < < | 650 651 652 653 654 655 656 | char *zName = getName3(zSystem, &aTest[i]); if( testCaseBegin(pRc, zPattern, "%s", zName) ){ doDataTest3(zSystem, &aTest[i], pRc); } testFree(zName); } } |
Changes to ext/lsm1/lsm-test/lsmtest8.c.
︙ | ︙ | |||
318 319 320 321 322 323 324 | if( testCaseBegin(pRc, zPattern, p->zName) ){ p->xFunc(pRc); testCaseFinish(*pRc); } } } | < < | 318 319 320 321 322 323 324 | if( testCaseBegin(pRc, zPattern, p->zName) ){ p->xFunc(pRc); testCaseFinish(*pRc); } } } |
Changes to ext/lsm1/lsm-test/lsmtest9.c.
︙ | ︙ | |||
134 135 136 137 138 139 140 | char *zName = getName4(zSystem, &aTest[i]); if( testCaseBegin(pRc, zPattern, "%s", zName) ){ doDataTest4(zSystem, &aTest[i], pRc); } testFree(zName); } } | < < < | 134 135 136 137 138 139 140 | char *zName = getName4(zSystem, &aTest[i]); if( testCaseBegin(pRc, zPattern, "%s", zName) ){ doDataTest4(zSystem, &aTest[i], pRc); } testFree(zName); } } |
Changes to ext/lsm1/lsm-test/lsmtest_bt.c.
︙ | ︙ | |||
65 66 67 68 69 70 71 | return -4; } printf("%s\n", (char*)buf.output.p); sqlite4_buffer_clear(&buf.output); return 0; } | < < < < | 65 66 67 68 69 70 71 | return -4; } printf("%s\n", (char*)buf.output.p); sqlite4_buffer_clear(&buf.output); return 0; } |
Changes to ext/lsm1/lsm-test/lsmtest_tdb.c.
︙ | ︙ | |||
549 550 551 552 553 554 555 | /* iLevel==0 is a no-op */ if( iLevel==0 ) return 0; /* If there are no transactions at all open, open a read transaction. */ if( pDb->nOpenTrans==0 ){ int rc = sqlite3_exec(pDb->db, | | | 549 550 551 552 553 554 555 556 557 558 559 560 561 562 563 | /* iLevel==0 is a no-op */ if( iLevel==0 ) return 0; /* If there are no transactions at all open, open a read transaction. */ if( pDb->nOpenTrans==0 ){ int rc = sqlite3_exec(pDb->db, "BEGIN; SELECT * FROM sqlite_schema LIMIT 1;" , 0, 0, 0 ); if( rc!=0 ) return rc; pDb->nOpenTrans = 1; } /* Open any required write transactions */ for(i=pDb->nOpenTrans; i<iLevel; i++){ |
︙ | ︙ |
Changes to ext/lsm1/lsm-test/lsmtest_tdb2.cc.
︙ | ︙ | |||
363 364 365 366 367 368 369 | } } return rc; } #endif /* HAVE_MDB */ | < | 363 364 365 366 367 368 369 | } } return rc; } #endif /* HAVE_MDB */ |
Changes to ext/lsm1/lsm-test/lsmtest_tdb4.c.
︙ | ︙ | |||
974 975 976 977 978 979 980 | } return rc; } /* ** End of background checkpointer. *************************************************************************/ | < < | 974 975 976 977 978 979 980 | } return rc; } /* ** End of background checkpointer. *************************************************************************/ |
Changes to ext/lsm1/lsmInt.h.
︙ | ︙ | |||
40 41 42 43 44 45 46 47 48 49 50 51 52 53 | # endif #else # ifndef LSM_DEBUG # define LSM_DEBUG # endif #endif /* ** Default values for various data structure parameters. These may be ** overridden by calls to lsm_config(). */ #define LSM_DFLT_PAGE_SIZE (4 * 1024) #define LSM_DFLT_BLOCK_SIZE (1 * 1024 * 1024) #define LSM_DFLT_AUTOFLUSH (1 * 1024 * 1024) | > > | 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 | # endif #else # ifndef LSM_DEBUG # define LSM_DEBUG # endif #endif /* #define LSM_DEBUG_EXPENSIVE 1 */ /* ** Default values for various data structure parameters. These may be ** overridden by calls to lsm_config(). */ #define LSM_DFLT_PAGE_SIZE (4 * 1024) #define LSM_DFLT_BLOCK_SIZE (1 * 1024 * 1024) #define LSM_DFLT_AUTOFLUSH (1 * 1024 * 1024) |
︙ | ︙ | |||
401 402 403 404 405 406 407 | u32 aSnapshot[LSM_META_PAGE_SIZE / sizeof(u32)]; }; struct Segment { LsmPgno iFirst; /* First page of this run */ LsmPgno iLastPg; /* Last page of this run */ LsmPgno iRoot; /* Root page number (if any) */ | | | 403 404 405 406 407 408 409 410 411 412 413 414 415 416 417 | u32 aSnapshot[LSM_META_PAGE_SIZE / sizeof(u32)]; }; struct Segment { LsmPgno iFirst; /* First page of this run */ LsmPgno iLastPg; /* Last page of this run */ LsmPgno iRoot; /* Root page number (if any) */ LsmPgno nSize; /* Size of this run in pages */ Redirect *pRedirect; /* Block redirects (or NULL) */ }; /* ** iSplitTopic/pSplitKey/nSplitKey: ** If nRight>0, this buffer contains a copy of the largest key that has |
︙ | ︙ | |||
848 849 850 851 852 853 854 855 856 857 858 859 860 861 | /* ** Functions from "lsm_varint.c". */ int lsmVarintPut32(u8 *, int); int lsmVarintGet32(u8 *, int *); int lsmVarintPut64(u8 *aData, i64 iVal); int lsmVarintGet64(const u8 *aData, i64 *piVal); int lsmVarintLen32(int); int lsmVarintSize(u8 c); /* ** Functions from file "main.c". */ | > > | 850 851 852 853 854 855 856 857 858 859 860 861 862 863 864 865 | /* ** Functions from "lsm_varint.c". */ int lsmVarintPut32(u8 *, int); int lsmVarintGet32(u8 *, int *); int lsmVarintPut64(u8 *aData, i64 iVal); int lsmVarintGet64(const u8 *aData, i64 *piVal); int lsmVarintLen64(i64); int lsmVarintLen32(int); int lsmVarintSize(u8 c); /* ** Functions from file "main.c". */ |
︙ | ︙ |
Changes to ext/lsm1/lsm_ckpt.c.
︙ | ︙ | |||
507 508 509 510 511 512 513 | Segment *pSegment /* Populate this structure */ ){ assert( pSegment->iFirst==0 && pSegment->iLastPg==0 ); assert( pSegment->nSize==0 && pSegment->iRoot==0 ); pSegment->iFirst = ckptGobble64(aIn, piIn); pSegment->iLastPg = ckptGobble64(aIn, piIn); pSegment->iRoot = ckptGobble64(aIn, piIn); | | | 507 508 509 510 511 512 513 514 515 516 517 518 519 520 521 | Segment *pSegment /* Populate this structure */ ){ assert( pSegment->iFirst==0 && pSegment->iLastPg==0 ); assert( pSegment->nSize==0 && pSegment->iRoot==0 ); pSegment->iFirst = ckptGobble64(aIn, piIn); pSegment->iLastPg = ckptGobble64(aIn, piIn); pSegment->iRoot = ckptGobble64(aIn, piIn); pSegment->nSize = ckptGobble64(aIn, piIn); assert( pSegment->iFirst ); } static int ckptSetupMerge(lsm_db *pDb, u32 *aInt, int *piIn, Level *pLevel){ Merge *pMerge; /* Allocated Merge object */ int nInput; /* Number of input segments in merge */ int iIn = *piIn; /* Next value to read from aInt[] */ |
︙ | ︙ |
Changes to ext/lsm1/lsm_file.c.
︙ | ︙ | |||
211 212 213 214 215 216 217 | struct FileSystem { lsm_db *pDb; /* Database handle that owns this object */ lsm_env *pEnv; /* Environment pointer */ char *zDb; /* Database file name */ char *zLog; /* Database file name */ int nMetasize; /* Size of meta pages in bytes */ int nMetaRwSize; /* Read/written size of meta pages in bytes */ | | | | 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 | struct FileSystem { lsm_db *pDb; /* Database handle that owns this object */ lsm_env *pEnv; /* Environment pointer */ char *zDb; /* Database file name */ char *zLog; /* Database file name */ int nMetasize; /* Size of meta pages in bytes */ int nMetaRwSize; /* Read/written size of meta pages in bytes */ i64 nPagesize; /* Database page-size in bytes */ i64 nBlocksize; /* Database block-size in bytes */ /* r/w file descriptors for both files. */ LsmFile *pLsmFile; /* Used after lsm_close() to link into list */ lsm_file *fdDb; /* Database file */ lsm_file *fdLog; /* Log file */ int szSector; /* Database file sector size */ |
︙ | ︙ | |||
885 886 887 888 889 890 891 | if( pFS->pCompress ){ if( iBlock==1 ){ iPg = pFS->nMetasize * 2 + 4; }else{ iPg = pFS->nBlocksize * (LsmPgno)(iBlock-1) + 4; } }else{ | | | 885 886 887 888 889 890 891 892 893 894 895 896 897 898 899 | if( pFS->pCompress ){ if( iBlock==1 ){ iPg = pFS->nMetasize * 2 + 4; }else{ iPg = pFS->nBlocksize * (LsmPgno)(iBlock-1) + 4; } }else{ const i64 nPagePerBlock = (pFS->nBlocksize / pFS->nPagesize); if( iBlock==1 ){ iPg = 1 + ((pFS->nMetasize*2 + pFS->nPagesize - 1) / pFS->nPagesize); }else{ iPg = 1 + (iBlock-1) * nPagePerBlock; } } return iPg; |
︙ | ︙ | |||
1127 1128 1129 1130 1131 1132 1133 | ** This function is never called in compressed database mode. */ static void fsGrowMapping( FileSystem *pFS, /* File system object */ i64 iSz, /* Minimum size to extend mapping to */ int *pRc /* IN/OUT: Error code */ ){ | < | 1127 1128 1129 1130 1131 1132 1133 1134 1135 1136 1137 1138 1139 1140 | ** This function is never called in compressed database mode. */ static void fsGrowMapping( FileSystem *pFS, /* File system object */ i64 iSz, /* Minimum size to extend mapping to */ int *pRc /* IN/OUT: Error code */ ){ assert( PAGE_HASPREV==4 ); if( *pRc==LSM_OK && iSz>pFS->nMap ){ int rc; u8 *aOld = pFS->pMap; rc = lsmEnvRemap(pFS->pEnv, pFS->fdDb, iSz, &pFS->pMap, &pFS->nMap); if( rc==LSM_OK && pFS->pMap!=aOld ){ |
︙ | ︙ | |||
1868 1869 1870 1871 1872 1873 1874 | int iBlk; assert( pRun->nSize>0 ); assert( 0==fsSegmentRedirects(pFS, pRun) ); assert( nPgno>0 && 0==fsPageRedirects(pFS, pRun, aPgno[0]) ); iBlk = fsPageToBlock(pFS, pRun->iFirst); | | | | | 1867 1868 1869 1870 1871 1872 1873 1874 1875 1876 1877 1878 1879 1880 1881 1882 1883 1884 1885 1886 1887 1888 1889 1890 1891 1892 1893 1894 1895 1896 1897 1898 | int iBlk; assert( pRun->nSize>0 ); assert( 0==fsSegmentRedirects(pFS, pRun) ); assert( nPgno>0 && 0==fsPageRedirects(pFS, pRun, aPgno[0]) ); iBlk = fsPageToBlock(pFS, pRun->iFirst); pRun->nSize += (pRun->iFirst - fsFirstPageOnBlock(pFS, iBlk)); while( rc==LSM_OK ){ int iNext = 0; LsmPgno iFirst = firstOnBlock(pFS, iBlk, aPgno, nPgno); if( iFirst ){ pRun->iFirst = iFirst; break; } rc = fsBlockNext(pFS, pRun, iBlk, &iNext); if( rc==LSM_OK ) rc = fsFreeBlock(pFS, pSnapshot, pRun, iBlk); pRun->nSize -= ( 1 + fsLastPageOnBlock(pFS, iBlk) - fsFirstPageOnBlock(pFS, iBlk) ); iBlk = iNext; } pRun->nSize -= (pRun->iFirst - fsFirstPageOnBlock(pFS, iBlk)); assert( pRun->nSize>0 ); } /* ** This function is only used in compressed database mode. ** ** Argument iPg is the page number (byte offset) of a page within segment |
︙ | ︙ |
Changes to ext/lsm1/lsm_main.c.
︙ | ︙ | |||
428 429 430 431 432 433 434 | } va_end(ap); return rc; } void lsmAppendSegmentList(LsmString *pStr, char *zPre, Segment *pSeg){ | | | 428 429 430 431 432 433 434 435 436 437 438 439 440 441 442 | } va_end(ap); return rc; } void lsmAppendSegmentList(LsmString *pStr, char *zPre, Segment *pSeg){ lsmStringAppendf(pStr, "%s{%lld %lld %lld %lld}", zPre, pSeg->iFirst, pSeg->iLastPg, pSeg->iRoot, pSeg->nSize ); } static int infoGetWorker(lsm_db *pDb, Snapshot **pp, int *pbUnlock){ int rc = LSM_OK; |
︙ | ︙ |
Changes to ext/lsm1/lsm_shared.c.
︙ | ︙ | |||
1301 1302 1303 1304 1305 1306 1307 1308 1309 1310 1311 1312 1313 1314 | if( rc==LSM_OK ){ rc = lsmShmCacheChunks(db, 1); if( rc==LSM_OK ){ db->pShmhdr = (ShmHeader *)db->apShm[0]; } } } if( rc==LSM_OK ){ rc = lsmBeginReadTrans(db); } } return rc; | > > > > > > > > > > > > > > > > > > | 1301 1302 1303 1304 1305 1306 1307 1308 1309 1310 1311 1312 1313 1314 1315 1316 1317 1318 1319 1320 1321 1322 1323 1324 1325 1326 1327 1328 1329 1330 1331 1332 | if( rc==LSM_OK ){ rc = lsmShmCacheChunks(db, 1); if( rc==LSM_OK ){ db->pShmhdr = (ShmHeader *)db->apShm[0]; } } } /* In 'lsm_open()' we don't update the page and block sizes in the ** Filesystem for 'readonly' connection. Because member 'db->pShmhdr' is a ** nullpointer, this prevents loading a checkpoint. Now that the system is ** live this member should be set. So we can update both values in ** the Filesystem. ** ** Configure the file-system connection with the page-size and block-size ** of this database. Even if the database file is zero bytes in size ** on disk, these values have been set in shared-memory by now, and so ** are guaranteed not to change during the lifetime of this connection. */ if( LSM_OK==rc && 0==lsmCheckpointClientCacheOk(db) && LSM_OK==(rc=lsmCheckpointLoad(db, 0)) ){ lsmFsSetPageSize(db->pFS, lsmCheckpointPgsz(db->aSnapshot)); lsmFsSetBlockSize(db->pFS, lsmCheckpointBlksz(db->aSnapshot)); } if( rc==LSM_OK ){ rc = lsmBeginReadTrans(db); } } return rc; |
︙ | ︙ |
Changes to ext/lsm1/lsm_sorted.c.
︙ | ︙ | |||
521 522 523 524 525 526 527 | Page *pPg, /* Page to read from */ int iCell, /* Index of cell on page to read */ int *piTopic, /* OUT: Topic associated with this key */ int *pnKey, /* OUT: Size of key in bytes */ LsmBlob *pBlob /* If required, use this for dynamic memory */ ){ u8 *pKey; | | | | | 521 522 523 524 525 526 527 528 529 530 531 532 533 534 535 536 537 538 539 540 541 542 543 544 545 546 547 548 549 550 | Page *pPg, /* Page to read from */ int iCell, /* Index of cell on page to read */ int *piTopic, /* OUT: Topic associated with this key */ int *pnKey, /* OUT: Size of key in bytes */ LsmBlob *pBlob /* If required, use this for dynamic memory */ ){ u8 *pKey; i64 nDummy; int eType; u8 *aData; int nData; aData = fsPageData(pPg, &nData); assert( !(pageGetFlags(aData, nData) & SEGMENT_BTREE_FLAG) ); assert( iCell<pageGetNRec(aData, nData) ); pKey = pageGetCell(aData, nData, iCell); eType = *pKey++; pKey += lsmVarintGet64(pKey, &nDummy); pKey += lsmVarintGet32(pKey, pnKey); if( rtIsWrite(eType) ){ pKey += lsmVarintGet64(pKey, &nDummy); } *piTopic = rtTopic(eType); sortedReadData(pSeg, pPg, pKey-aData, *pnKey, (void **)&pKey, pBlob); return pKey; } |
︙ | ︙ | |||
653 654 655 656 657 658 659 | ); pCsr->eType |= LSM_SEPARATOR; } return rc; } | | | | | 653 654 655 656 657 658 659 660 661 662 663 664 665 666 667 668 669 670 671 672 673 674 | ); pCsr->eType |= LSM_SEPARATOR; } return rc; } static LsmPgno btreeCursorPtr(u8 *aData, int nData, int iCell){ int nCell; nCell = pageGetNRec(aData, nData); if( iCell>=nCell ){ return pageGetPtr(aData, nData); } return pageGetRecordPtr(aData, nData, iCell); } static int btreeCursorNext(BtreeCursor *pCsr){ int rc = LSM_OK; BtreePg *pPg = &pCsr->aPg[pCsr->iPg]; int nCell; |
︙ | ︙ | |||
747 748 749 750 751 752 753 | } static int btreeCursorFirst(BtreeCursor *pCsr){ int rc; Page *pPg = 0; FileSystem *pFS = pCsr->pFS; | | | 747 748 749 750 751 752 753 754 755 756 757 758 759 760 761 | } static int btreeCursorFirst(BtreeCursor *pCsr){ int rc; Page *pPg = 0; FileSystem *pFS = pCsr->pFS; LsmPgno iPg = pCsr->pSeg->iRoot; do { rc = lsmFsDbPageGet(pFS, pCsr->pSeg, iPg, &pPg); assert( (rc==LSM_OK)==(pPg!=0) ); if( rc==LSM_OK ){ u8 *aData; int nData; |
︙ | ︙ | |||
775 776 777 778 779 780 781 | } } if( rc==LSM_OK ){ assert( pCsr->aPg[pCsr->nDepth].iCell==0 ); pCsr->aPg[pCsr->nDepth].pPage = pPg; pCsr->nDepth++; | | | 775 776 777 778 779 780 781 782 783 784 785 786 787 788 789 | } } if( rc==LSM_OK ){ assert( pCsr->aPg[pCsr->nDepth].iCell==0 ); pCsr->aPg[pCsr->nDepth].pPage = pPg; pCsr->nDepth++; iPg = pageGetRecordPtr(aData, nData, 0); } } }while( rc==LSM_OK ); lsmFsPageRelease(pPg); pCsr->iPg = pCsr->nDepth-1; if( rc==LSM_OK && pCsr->nDepth ){ |
︙ | ︙ | |||
867 868 869 870 871 872 873 | /* Populate any other aPg[] array entries */ if( rc==LSM_OK && nDepth>1 ){ LsmBlob blob = {0,0,0}; void *pSeek; int nSeek; int iTopicSeek; int iPg = 0; | | | 867 868 869 870 871 872 873 874 875 876 877 878 879 880 881 | /* Populate any other aPg[] array entries */ if( rc==LSM_OK && nDepth>1 ){ LsmBlob blob = {0,0,0}; void *pSeek; int nSeek; int iTopicSeek; int iPg = 0; LsmPgno iLoad = pSeg->iRoot; Page *pPg = pCsr->aPg[nDepth-1].pPage; if( pageObjGetNRec(pPg)==0 ){ /* This can happen when pPg is the right-most leaf in the b-tree. ** In this case, set the iTopicSeek/pSeek/nSeek key to a value ** greater than any real key. */ assert( iCell==-1 ); |
︙ | ︙ | |||
899 900 901 902 903 904 905 | int iMin; int iMax; int iCell2; aData = fsPageData(pPg2, &nData); assert( (pageGetFlags(aData, nData) & SEGMENT_BTREE_FLAG) ); | | | 899 900 901 902 903 904 905 906 907 908 909 910 911 912 913 | int iMin; int iMax; int iCell2; aData = fsPageData(pPg2, &nData); assert( (pageGetFlags(aData, nData) & SEGMENT_BTREE_FLAG) ); iLoad = pageGetPtr(aData, nData); iCell2 = pageGetNRec(aData, nData); iMax = iCell2-1; iMin = 0; while( iMax>=iMin ){ int iTry = (iMin+iMax)/2; void *pKey; int nKey; /* Key for cell iTry */ |
︙ | ︙ | |||
922 923 924 925 926 927 928 | res = sortedKeyCompare( xCmp, iTopicSeek, pSeek, nSeek, iTopic, pKey, nKey ); assert( res!=0 ); if( res<0 ){ | | | 922 923 924 925 926 927 928 929 930 931 932 933 934 935 936 | res = sortedKeyCompare( xCmp, iTopicSeek, pSeek, nSeek, iTopic, pKey, nKey ); assert( res!=0 ); if( res<0 ){ iLoad = iPtr; iCell2 = iTry; iMax = iTry-1; }else{ iMin = iTry+1; } } |
︙ | ︙ | |||
1009 1010 1011 1012 1013 1014 1015 | /* ** Load a new page into the SegmentPtr object pPtr. */ static int segmentPtrLoadPage( FileSystem *pFS, SegmentPtr *pPtr, /* Load page into this SegmentPtr object */ | | | 1009 1010 1011 1012 1013 1014 1015 1016 1017 1018 1019 1020 1021 1022 1023 | /* ** Load a new page into the SegmentPtr object pPtr. */ static int segmentPtrLoadPage( FileSystem *pFS, SegmentPtr *pPtr, /* Load page into this SegmentPtr object */ LsmPgno iNew /* Page number of new page */ ){ Page *pPg = 0; /* The new page */ int rc; /* Return Code */ rc = lsmFsDbPageGet(pFS, pPtr->pSeg, iNew, &pPg); assert( rc==LSM_OK || pPg==0 ); segmentPtrSetPage(pPtr, pPg); |
︙ | ︙ | |||
1629 1630 1631 1632 1633 1634 1635 | static int segmentPtrSeek( MultiCursor *pCsr, /* Cursor context */ SegmentPtr *pPtr, /* Pointer to seek */ int iTopic, /* Key topic to seek to */ void *pKey, int nKey, /* Key to seek to */ int eSeek, /* Search bias - see above */ | | | 1629 1630 1631 1632 1633 1634 1635 1636 1637 1638 1639 1640 1641 1642 1643 | static int segmentPtrSeek( MultiCursor *pCsr, /* Cursor context */ SegmentPtr *pPtr, /* Pointer to seek */ int iTopic, /* Key topic to seek to */ void *pKey, int nKey, /* Key to seek to */ int eSeek, /* Search bias - see above */ LsmPgno *piPtr, /* OUT: FC pointer */ int *pbStop ){ int (*xCmp)(void *, int, void *, int) = pCsr->pDb->xCmp; int res = 0; /* Result of comparison operation */ int rc = LSM_OK; int iMin; int iMax; |
︙ | ︙ | |||
1755 1756 1757 1758 1759 1760 1761 | ){ assert( eSeek!=LSM_SEEK_EQ ); rc = segmentPtrAdvance(pCsr, pPtr, eSeek==LSM_SEEK_LE); } } assert( rc!=LSM_OK || assertSeekResult(pCsr,pPtr,iTopic,pKey,nKey,eSeek) ); | | | | | 1755 1756 1757 1758 1759 1760 1761 1762 1763 1764 1765 1766 1767 1768 1769 1770 1771 1772 1773 1774 1775 1776 1777 1778 1779 1780 1781 1782 1783 1784 1785 1786 1787 | ){ assert( eSeek!=LSM_SEEK_EQ ); rc = segmentPtrAdvance(pCsr, pPtr, eSeek==LSM_SEEK_LE); } } assert( rc!=LSM_OK || assertSeekResult(pCsr,pPtr,iTopic,pKey,nKey,eSeek) ); *piPtr = iPtrOut; return rc; } static int seekInBtree( MultiCursor *pCsr, /* Multi-cursor object */ Segment *pSeg, /* Seek within this segment */ int iTopic, void *pKey, int nKey, /* Key to seek to */ LsmPgno *aPg, /* OUT: Page numbers */ Page **ppPg /* OUT: Leaf (sorted-run) page reference */ ){ int i = 0; int rc; LsmPgno iPg; Page *pPg = 0; LsmBlob blob = {0, 0, 0}; iPg = pSeg->iRoot; do { LsmPgno *piFirst = 0; if( aPg ){ aPg[i++] = iPg; piFirst = &aPg[i]; } |
︙ | ︙ | |||
1795 1796 1797 1798 1799 1800 1801 | int nRec; int flags; aData = fsPageData(pPg, &nData); flags = pageGetFlags(aData, nData); if( (flags & SEGMENT_BTREE_FLAG)==0 ) break; | | | 1795 1796 1797 1798 1799 1800 1801 1802 1803 1804 1805 1806 1807 1808 1809 | int nRec; int flags; aData = fsPageData(pPg, &nData); flags = pageGetFlags(aData, nData); if( (flags & SEGMENT_BTREE_FLAG)==0 ) break; iPg = pageGetPtr(aData, nData); nRec = pageGetNRec(aData, nData); iMin = 0; iMax = nRec-1; while( iMax>=iMin ){ int iTry = (iMin+iMax)/2; void *pKeyT; int nKeyT; /* Key for cell iTry */ |
︙ | ︙ | |||
1821 1822 1823 1824 1825 1826 1827 | i++; } res = sortedKeyCompare( pCsr->pDb->xCmp, iTopic, pKey, nKey, iTopicT, pKeyT, nKeyT ); if( res<0 ){ | | | 1821 1822 1823 1824 1825 1826 1827 1828 1829 1830 1831 1832 1833 1834 1835 | i++; } res = sortedKeyCompare( pCsr->pDb->xCmp, iTopic, pKey, nKey, iTopicT, pKeyT, nKeyT ); if( res<0 ){ iPg = iPtr; iMax = iTry-1; }else{ iMin = iTry+1; } } lsmFsPageRelease(pPg); pPg = 0; |
︙ | ︙ | |||
1847 1848 1849 1850 1851 1852 1853 | } static int seekInSegment( MultiCursor *pCsr, SegmentPtr *pPtr, int iTopic, void *pKey, int nKey, | | | | | | 1847 1848 1849 1850 1851 1852 1853 1854 1855 1856 1857 1858 1859 1860 1861 1862 1863 1864 1865 1866 1867 1868 1869 1870 1871 1872 1873 1874 1875 1876 | } static int seekInSegment( MultiCursor *pCsr, SegmentPtr *pPtr, int iTopic, void *pKey, int nKey, LsmPgno iPg, /* Page to search */ int eSeek, /* Search bias - see above */ LsmPgno *piPtr, /* OUT: FC pointer */ int *pbStop /* OUT: Stop search flag */ ){ LsmPgno iPtr = iPg; int rc = LSM_OK; if( pPtr->pSeg->iRoot ){ Page *pPg; assert( pPtr->pSeg->iRoot!=0 ); rc = seekInBtree(pCsr, pPtr->pSeg, iTopic, pKey, nKey, 0, &pPg); if( rc==LSM_OK ) segmentPtrSetPage(pPtr, pPg); }else{ if( iPtr==0 ){ iPtr = pPtr->pSeg->iFirst; } if( rc==LSM_OK ){ rc = segmentPtrLoadPage(pCsr->pDb->pFS, pPtr, iPtr); } } if( rc==LSM_OK ){ |
︙ | ︙ | |||
1900 1901 1902 1903 1904 1905 1906 | int iTopic, /* Key topic to search for */ void *pKey, int nKey, /* Key to search for */ LsmPgno *piPgno, /* IN/OUT: fraction cascade pointer (or 0) */ int *pbStop /* OUT: See above */ ){ Level *pLvl = aPtr[0].pLevel; /* Level to seek within */ int rc = LSM_OK; /* Return code */ | | | | | | 1900 1901 1902 1903 1904 1905 1906 1907 1908 1909 1910 1911 1912 1913 1914 1915 1916 1917 1918 1919 1920 1921 1922 1923 1924 1925 1926 1927 1928 1929 1930 1931 1932 1933 1934 1935 1936 1937 1938 1939 1940 1941 1942 1943 1944 1945 1946 1947 1948 1949 | int iTopic, /* Key topic to search for */ void *pKey, int nKey, /* Key to search for */ LsmPgno *piPgno, /* IN/OUT: fraction cascade pointer (or 0) */ int *pbStop /* OUT: See above */ ){ Level *pLvl = aPtr[0].pLevel; /* Level to seek within */ int rc = LSM_OK; /* Return code */ LsmPgno iOut = 0; /* Pointer to return to caller */ int res = -1; /* Result of xCmp(pKey, split) */ int nRhs = pLvl->nRight; /* Number of right-hand-side segments */ int bStop = 0; /* If this is a composite level (one currently undergoing an incremental ** merge), figure out if the search key is larger or smaller than the ** levels split-key. */ if( nRhs ){ res = sortedKeyCompare(pCsr->pDb->xCmp, iTopic, pKey, nKey, pLvl->iSplitTopic, pLvl->pSplitKey, pLvl->nSplitKey ); } /* If (res<0), then key pKey/nKey is smaller than the split-key (or this ** is not a composite level and there is no split-key). Search the ** left-hand-side of the level in this case. */ if( res<0 ){ int i; LsmPgno iPtr = 0; if( nRhs==0 ) iPtr = *piPgno; rc = seekInSegment( pCsr, &aPtr[0], iTopic, pKey, nKey, iPtr, eSeek, &iOut, &bStop ); if( rc==LSM_OK && nRhs>0 && eSeek==LSM_SEEK_GE && aPtr[0].pPg==0 ){ res = 0; } for(i=1; i<=nRhs; i++){ segmentPtrReset(&aPtr[i], LSM_SEGMENTPTR_FREE_THRESHOLD); } } if( res>=0 ){ int bHit = 0; /* True if at least one rhs is not EOF */ LsmPgno iPtr = *piPgno; int i; segmentPtrReset(&aPtr[0], LSM_SEGMENTPTR_FREE_THRESHOLD); for(i=1; rc==LSM_OK && i<=nRhs && bStop==0; i++){ SegmentPtr *pPtr = &aPtr[i]; iOut = 0; rc = seekInSegment( pCsr, pPtr, iTopic, pKey, nKey, iPtr, eSeek, &iOut, &bStop |
︙ | ︙ | |||
3357 3358 3359 3360 3361 3362 3363 3364 3365 3366 3367 3368 3369 3370 3371 3372 | ** space). */ static int mergeWorkerPageOffset(u8 *aData, int nData){ int nRec; int iOff; int nKey; int eType; nRec = lsmGetU16(&aData[SEGMENT_NRECORD_OFFSET(nData)]); iOff = lsmGetU16(&aData[SEGMENT_CELLPTR_OFFSET(nData, nRec-1)]); eType = aData[iOff++]; assert( eType==0 || eType==(LSM_SYSTEMKEY|LSM_SEPARATOR) || eType==(LSM_SEPARATOR) ); | > > | | 3357 3358 3359 3360 3361 3362 3363 3364 3365 3366 3367 3368 3369 3370 3371 3372 3373 3374 3375 3376 3377 3378 3379 3380 3381 3382 | ** space). */ static int mergeWorkerPageOffset(u8 *aData, int nData){ int nRec; int iOff; int nKey; int eType; i64 nDummy; nRec = lsmGetU16(&aData[SEGMENT_NRECORD_OFFSET(nData)]); iOff = lsmGetU16(&aData[SEGMENT_CELLPTR_OFFSET(nData, nRec-1)]); eType = aData[iOff++]; assert( eType==0 || eType==(LSM_SYSTEMKEY|LSM_SEPARATOR) || eType==(LSM_SEPARATOR) ); iOff += lsmVarintGet64(&aData[iOff], &nDummy); iOff += lsmVarintGet32(&aData[iOff], &nKey); return iOff + (eType ? nKey : 0); } /* ** Following a checkpoint operation, database pages that are part of the |
︙ | ︙ | |||
3476 3477 3478 3479 3480 3481 3482 | p = &pMW->hier; if( p->apHier==0 && pSeg->iRoot!=0 ){ FileSystem *pFS = pMW->pDb->pFS; lsm_env *pEnv = pMW->pDb->pEnv; Page **apHier = 0; int nHier = 0; | | | 3478 3479 3480 3481 3482 3483 3484 3485 3486 3487 3488 3489 3490 3491 3492 | p = &pMW->hier; if( p->apHier==0 && pSeg->iRoot!=0 ){ FileSystem *pFS = pMW->pDb->pFS; lsm_env *pEnv = pMW->pDb->pEnv; Page **apHier = 0; int nHier = 0; LsmPgno iPg = pSeg->iRoot; do { Page *pPg = 0; u8 *aData; int nData; int flags; |
︙ | ︙ | |||
3502 3503 3504 3505 3506 3507 3508 | break; } apHier = apNew; memmove(&apHier[1], &apHier[0], sizeof(Page *) * nHier); nHier++; apHier[0] = pPg; | | | 3504 3505 3506 3507 3508 3509 3510 3511 3512 3513 3514 3515 3516 3517 3518 | break; } apHier = apNew; memmove(&apHier[1], &apHier[0], sizeof(Page *) * nHier); nHier++; apHier[0] = pPg; iPg = pageGetPtr(aData, nData); }else{ lsmFsPageRelease(pPg); break; } }while( 1 ); if( rc==LSM_OK ){ |
︙ | ︙ | |||
3621 3622 3623 3624 3625 3626 3627 | /* If the key will fit on this page, break out of the loop here. ** The new entry will be written to page apHier[iLevel]. */ pOld = p->apHier[iLevel]; assert( lsmFsPageWritable(pOld) ); aData = fsPageData(pOld, &nData); if( eType==0 ){ | | | > | 3623 3624 3625 3626 3627 3628 3629 3630 3631 3632 3633 3634 3635 3636 3637 3638 3639 3640 3641 | /* If the key will fit on this page, break out of the loop here. ** The new entry will be written to page apHier[iLevel]. */ pOld = p->apHier[iLevel]; assert( lsmFsPageWritable(pOld) ); aData = fsPageData(pOld, &nData); if( eType==0 ){ nByte = 2 + 1 + lsmVarintLen64(iPtr) + lsmVarintLen64(iKeyPg); }else{ nByte = 2 + 1 + lsmVarintLen64(iPtr) + lsmVarintLen32(nKey) + nKey; } nRec = pageGetNRec(aData, nData); nFree = SEGMENT_EOF(nData, nRec) - mergeWorkerPageOffset(aData, nData); if( nByte<=nFree ) break; /* Otherwise, this page is full. Set the right-hand-child pointer ** to iPtr and release it. */ lsmPutU64(&aData[SEGMENT_POINTER_OFFSET(nData)], iPtr); |
︙ | ︙ | |||
3668 3669 3670 3671 3672 3673 3674 | aData = fsPageData(p->apHier[iLevel], &nData); iOff = mergeWorkerPageOffset(aData, nData); nRec = pageGetNRec(aData, nData); lsmPutU16(&aData[SEGMENT_CELLPTR_OFFSET(nData, nRec)], (u16)iOff); lsmPutU16(&aData[SEGMENT_NRECORD_OFFSET(nData)], (u16)(nRec+1)); if( eType==0 ){ aData[iOff++] = 0x00; | | | | | 3671 3672 3673 3674 3675 3676 3677 3678 3679 3680 3681 3682 3683 3684 3685 3686 3687 3688 3689 | aData = fsPageData(p->apHier[iLevel], &nData); iOff = mergeWorkerPageOffset(aData, nData); nRec = pageGetNRec(aData, nData); lsmPutU16(&aData[SEGMENT_CELLPTR_OFFSET(nData, nRec)], (u16)iOff); lsmPutU16(&aData[SEGMENT_NRECORD_OFFSET(nData)], (u16)(nRec+1)); if( eType==0 ){ aData[iOff++] = 0x00; iOff += lsmVarintPut64(&aData[iOff], iPtr); iOff += lsmVarintPut64(&aData[iOff], iKeyPg); }else{ aData[iOff++] = eType; iOff += lsmVarintPut64(&aData[iOff], iPtr); iOff += lsmVarintPut32(&aData[iOff], nKey); memcpy(&aData[iOff], pKey, nKey); } return rc; } |
︙ | ︙ | |||
3868 3869 3870 3871 3872 3873 3874 | ** array. Otherwise, the main array. ** ** This function is used to write the blobs of data for keys and values. */ static int mergeWorkerData( MergeWorker *pMW, /* Merge worker object */ int bSep, /* True to write to separators run */ | | | 3871 3872 3873 3874 3875 3876 3877 3878 3879 3880 3881 3882 3883 3884 3885 | ** array. Otherwise, the main array. ** ** This function is used to write the blobs of data for keys and values. */ static int mergeWorkerData( MergeWorker *pMW, /* Merge worker object */ int bSep, /* True to write to separators run */ LsmPgno iFPtr, /* Footer ptr for new pages */ u8 *aWrite, /* Write data from this buffer */ int nWrite /* Size of aWrite[] in bytes */ ){ int rc = LSM_OK; /* Return code */ int nRem = nWrite; /* Number of bytes still to write */ while( rc==LSM_OK && nRem>0 ){ |
︙ | ︙ | |||
3912 3913 3914 3915 3916 3917 3918 | ** The MergeWorker passed as the only argument is working to merge two or ** more existing segments together (not to flush an in-memory tree). It ** has not yet written the first key to the first page of the output. */ static int mergeWorkerFirstPage(MergeWorker *pMW){ int rc = LSM_OK; /* Return code */ Page *pPg = 0; /* First page of run pSeg */ | | | | | | | | | | | | | 3915 3916 3917 3918 3919 3920 3921 3922 3923 3924 3925 3926 3927 3928 3929 3930 3931 3932 3933 3934 3935 3936 3937 3938 3939 3940 3941 3942 3943 3944 3945 3946 3947 3948 3949 3950 3951 3952 3953 3954 3955 3956 3957 3958 3959 3960 3961 3962 3963 3964 3965 3966 3967 3968 3969 3970 3971 3972 3973 3974 3975 3976 3977 3978 3979 3980 3981 3982 3983 3984 3985 3986 3987 3988 3989 3990 3991 3992 3993 3994 3995 3996 3997 3998 3999 4000 4001 4002 4003 4004 4005 4006 4007 4008 4009 4010 4011 4012 4013 4014 4015 4016 4017 4018 4019 4020 4021 4022 4023 | ** The MergeWorker passed as the only argument is working to merge two or ** more existing segments together (not to flush an in-memory tree). It ** has not yet written the first key to the first page of the output. */ static int mergeWorkerFirstPage(MergeWorker *pMW){ int rc = LSM_OK; /* Return code */ Page *pPg = 0; /* First page of run pSeg */ LsmPgno iFPtr = 0; /* Pointer value read from footer of pPg */ MultiCursor *pCsr = pMW->pCsr; assert( pMW->pPage==0 ); if( pCsr->pBtCsr ){ rc = LSM_OK; iFPtr = pMW->pLevel->pNext->lhs.iFirst; }else if( pCsr->nPtr>0 ){ Segment *pSeg; pSeg = pCsr->aPtr[pCsr->nPtr-1].pSeg; rc = lsmFsDbPageGet(pMW->pDb->pFS, pSeg, pSeg->iFirst, &pPg); if( rc==LSM_OK ){ u8 *aData; /* Buffer for page pPg */ int nData; /* Size of aData[] in bytes */ aData = fsPageData(pPg, &nData); iFPtr = pageGetPtr(aData, nData); lsmFsPageRelease(pPg); } } if( rc==LSM_OK ){ rc = mergeWorkerNextPage(pMW, iFPtr); if( pCsr->pPrevMergePtr ) *pCsr->pPrevMergePtr = iFPtr; pMW->aSave[0].bStore = 1; } return rc; } static int mergeWorkerWrite( MergeWorker *pMW, /* Merge worker object to write into */ int eType, /* One of SORTED_SEPARATOR, WRITE or DELETE */ void *pKey, int nKey, /* Key value */ void *pVal, int nVal, /* Value value */ LsmPgno iPtr /* Absolute value of page pointer, or 0 */ ){ int rc = LSM_OK; /* Return code */ Merge *pMerge; /* Persistent part of level merge state */ int nHdr; /* Space required for this record header */ Page *pPg; /* Page to write to */ u8 *aData; /* Data buffer for page pWriter->pPage */ int nData = 0; /* Size of buffer aData[] in bytes */ int nRec = 0; /* Number of records on page pPg */ LsmPgno iFPtr = 0; /* Value of pointer in footer of pPg */ LsmPgno iRPtr = 0; /* Value of pointer written into record */ int iOff = 0; /* Current write offset within page pPg */ Segment *pSeg; /* Segment being written */ int flags = 0; /* If != 0, flags value for page footer */ int bFirst = 0; /* True for first key of output run */ pMerge = pMW->pLevel->pMerge; pSeg = &pMW->pLevel->lhs; if( pSeg->iFirst==0 && pMW->pPage==0 ){ rc = mergeWorkerFirstPage(pMW); bFirst = 1; } pPg = pMW->pPage; if( pPg ){ aData = fsPageData(pPg, &nData); nRec = pageGetNRec(aData, nData); iFPtr = pageGetPtr(aData, nData); iRPtr = iPtr ? (iPtr - iFPtr) : 0; } /* Figure out how much space is required by the new record. The space ** required is divided into two sections: the header and the body. The ** header consists of the intial varint fields. The body are the blobs ** of data that correspond to the key and value data. The entire header ** must be stored on the page. The body may overflow onto the next and ** subsequent pages. ** ** The header space is: ** ** 1) record type - 1 byte. ** 2) Page-pointer-offset - 1 varint ** 3) Key size - 1 varint ** 4) Value size - 1 varint (only if LSM_INSERT flag is set) */ if( rc==LSM_OK ){ nHdr = 1 + lsmVarintLen64(iRPtr) + lsmVarintLen32(nKey); if( rtIsWrite(eType) ) nHdr += lsmVarintLen32(nVal); /* If the entire header will not fit on page pPg, or if page pPg is ** marked read-only, advance to the next page of the output run. */ iOff = pMerge->iOutputOff; if( iOff<0 || pPg==0 || iOff+nHdr > SEGMENT_EOF(nData, nRec+1) ){ if( iOff>=0 && pPg ){ /* Zero any free space on the page */ assert( aData ); memset(&aData[iOff], 0, SEGMENT_EOF(nData, nRec)-iOff); } iFPtr = *pMW->pCsr->pPrevMergePtr; iRPtr = iPtr ? (iPtr - iFPtr) : 0; iOff = 0; nRec = 0; rc = mergeWorkerNextPage(pMW, iFPtr); pPg = pMW->pPage; } } |
︙ | ︙ | |||
4045 4046 4047 4048 4049 4050 4051 | /* Update the page footer. */ lsmPutU16(&aData[SEGMENT_NRECORD_OFFSET(nData)], (u16)(nRec+1)); lsmPutU16(&aData[SEGMENT_CELLPTR_OFFSET(nData, nRec)], (u16)iOff); if( flags ) lsmPutU16(&aData[SEGMENT_FLAGS_OFFSET(nData)], (u16)flags); /* Write the entry header into the current page. */ aData[iOff++] = (u8)eType; /* 1 */ | | | 4048 4049 4050 4051 4052 4053 4054 4055 4056 4057 4058 4059 4060 4061 4062 | /* Update the page footer. */ lsmPutU16(&aData[SEGMENT_NRECORD_OFFSET(nData)], (u16)(nRec+1)); lsmPutU16(&aData[SEGMENT_CELLPTR_OFFSET(nData, nRec)], (u16)iOff); if( flags ) lsmPutU16(&aData[SEGMENT_FLAGS_OFFSET(nData)], (u16)flags); /* Write the entry header into the current page. */ aData[iOff++] = (u8)eType; /* 1 */ iOff += lsmVarintPut64(&aData[iOff], iRPtr); /* 2 */ iOff += lsmVarintPut32(&aData[iOff], nKey); /* 3 */ if( rtIsWrite(eType) ) iOff += lsmVarintPut32(&aData[iOff], nVal); /* 4 */ pMerge->iOutputOff = iOff; /* Write the key and data into the segment. */ assert( iFPtr==pageGetPtr(aData, nData) ); rc = mergeWorkerData(pMW, 0, iFPtr+iRPtr, pKey, nKey); |
︙ | ︙ | |||
4279 4280 4281 4282 4283 4284 4285 | rc = multiCursorGetVal(pCsr, iVal, &pVal, &nVal); if( pVal && rc==LSM_OK ){ assert( nVal>=0 ); rc = sortedBlobSet(pDb->pEnv, &pCsr->val, pVal, nVal); pVal = pCsr->val.pData; } if( rc==LSM_OK ){ | | | 4282 4283 4284 4285 4286 4287 4288 4289 4290 4291 4292 4293 4294 4295 4296 | rc = multiCursorGetVal(pCsr, iVal, &pVal, &nVal); if( pVal && rc==LSM_OK ){ assert( nVal>=0 ); rc = sortedBlobSet(pDb->pEnv, &pCsr->val, pVal, nVal); pVal = pCsr->val.pData; } if( rc==LSM_OK ){ rc = mergeWorkerWrite(pMW, eType, pKey, nKey, pVal, nVal, iPtr); } } } /* Advance the cursor to the next input record (assuming one exists). */ assert( lsmMCursorValid(pMW->pCsr) ); if( rc==LSM_OK ) rc = lsmMCursorNext(pMW->pCsr); |
︙ | ︙ | |||
4600 4601 4602 4603 4604 4605 4606 | int i; for(i=0; rc==LSM_OK && i<pCsr->nPtr; i++){ MergeInput *pInput = &pMerge->aInput[i]; if( pInput->iPg ){ SegmentPtr *pPtr; assert( pCsr->aPtr[i].pPg==0 ); pPtr = &pCsr->aPtr[i]; | | | 4603 4604 4605 4606 4607 4608 4609 4610 4611 4612 4613 4614 4615 4616 4617 | int i; for(i=0; rc==LSM_OK && i<pCsr->nPtr; i++){ MergeInput *pInput = &pMerge->aInput[i]; if( pInput->iPg ){ SegmentPtr *pPtr; assert( pCsr->aPtr[i].pPg==0 ); pPtr = &pCsr->aPtr[i]; rc = segmentPtrLoadPage(pDb->pFS, pPtr, pInput->iPg); if( rc==LSM_OK && pPtr->nCell>0 ){ rc = segmentPtrLoadCell(pPtr, pInput->iCell); } } } if( rc==LSM_OK && pCsr->pBtCsr ){ |
︙ | ︙ | |||
5465 5466 5467 5468 5469 5470 5471 | /* ** Return a string representation of the segment passed as the only argument. ** Space for the returned string is allocated using lsmMalloc(), and should ** be freed by the caller using lsmFree(). */ static char *segToString(lsm_env *pEnv, Segment *pSeg, int nMin){ | | | | | 5468 5469 5470 5471 5472 5473 5474 5475 5476 5477 5478 5479 5480 5481 5482 5483 5484 5485 5486 5487 5488 5489 5490 5491 5492 5493 5494 5495 5496 | /* ** Return a string representation of the segment passed as the only argument. ** Space for the returned string is allocated using lsmMalloc(), and should ** be freed by the caller using lsmFree(). */ static char *segToString(lsm_env *pEnv, Segment *pSeg, int nMin){ LsmPgno nSize = pSeg->nSize; LsmPgno iRoot = pSeg->iRoot; LsmPgno iFirst = pSeg->iFirst; LsmPgno iLast = pSeg->iLastPg; char *z; char *z1; char *z2; int nPad; z1 = lsmMallocPrintf(pEnv, "%d.%d", iFirst, iLast); if( iRoot ){ z2 = lsmMallocPrintf(pEnv, "root=%lld", iRoot); }else{ z2 = lsmMallocPrintf(pEnv, "size=%lld", nSize); } nPad = nMin - 2 - strlen(z1) - 1 - strlen(z2); nPad = LSM_MAX(0, nPad); if( iRoot ){ z = lsmMallocPrintf(pEnv, "/%s %*s%s\\", z1, nPad, "", z2); |
︙ | ︙ | |||
5532 5533 5534 5535 5536 5537 5538 | void sortedDumpPage(lsm_db *pDb, Segment *pRun, Page *pPg, int bVals){ LsmBlob blob = {0, 0, 0}; /* LsmBlob used for keys */ LsmString s; int i; int nRec; | | | | | | | 5535 5536 5537 5538 5539 5540 5541 5542 5543 5544 5545 5546 5547 5548 5549 5550 5551 5552 5553 5554 5555 5556 5557 5558 5559 5560 5561 5562 5563 5564 5565 5566 5567 5568 5569 5570 5571 5572 5573 5574 5575 5576 5577 | void sortedDumpPage(lsm_db *pDb, Segment *pRun, Page *pPg, int bVals){ LsmBlob blob = {0, 0, 0}; /* LsmBlob used for keys */ LsmString s; int i; int nRec; LsmPgno iPtr; int flags; u8 *aData; int nData; aData = fsPageData(pPg, &nData); nRec = pageGetNRec(aData, nData); iPtr = pageGetPtr(aData, nData); flags = pageGetFlags(aData, nData); lsmStringInit(&s, pDb->pEnv); lsmStringAppendf(&s,"nCell=%d iPtr=%lld flags=%d {", nRec, iPtr, flags); if( flags&SEGMENT_BTREE_FLAG ) iPtr = 0; for(i=0; i<nRec; i++){ Page *pRef = 0; /* Pointer to page iRef */ int iChar; u8 *aKey; int nKey = 0; /* Key */ u8 *aVal = 0; int nVal = 0; /* Value */ int iTopic; u8 *aCell; i64 iPgPtr; int eType; aCell = pageGetCell(aData, nData, i); eType = *aCell++; assert( (flags & SEGMENT_BTREE_FLAG) || eType!=0 ); aCell += lsmVarintGet64(aCell, &iPgPtr); if( eType==0 ){ LsmPgno iRef; /* Page number of referenced page */ aCell += lsmVarintGet64(aCell, &iRef); lsmFsDbPageGet(pDb->pFS, pRun, iRef, &pRef); aKey = pageGetKey(pRun, pRef, 0, &iTopic, &nKey, &blob); }else{ |
︙ | ︙ | |||
5586 5587 5588 5589 5590 5591 5592 | if( nVal>0 && bVals ){ lsmStringAppendf(&s, "##"); for(iChar=0; iChar<nVal; iChar++){ lsmStringAppendf(&s, "%c", isalnum(aVal[iChar]) ? aVal[iChar] : '.'); } } | | | 5589 5590 5591 5592 5593 5594 5595 5596 5597 5598 5599 5600 5601 5602 5603 | if( nVal>0 && bVals ){ lsmStringAppendf(&s, "##"); for(iChar=0; iChar<nVal; iChar++){ lsmStringAppendf(&s, "%c", isalnum(aVal[iChar]) ? aVal[iChar] : '.'); } } lsmStringAppendf(&s, " %lld", iPgPtr+iPtr); lsmFsPageRelease(pRef); } lsmStringAppend(&s, "}", 1); lsmLogMessage(pDb, LSM_OK, " Page %d: %s", lsmFsPageNumber(pPg), s.z); lsmStringClear(&s); |
︙ | ︙ | |||
5716 5717 5718 5719 5720 5721 5722 | } if( rc==LSM_OK ){ LsmBlob blob = {0, 0, 0, 0}; int nKeyWidth = 0; LsmString str; int nRec; | | | | | 5719 5720 5721 5722 5723 5724 5725 5726 5727 5728 5729 5730 5731 5732 5733 5734 5735 5736 5737 5738 5739 5740 5741 5742 5743 5744 5745 5746 | } if( rc==LSM_OK ){ LsmBlob blob = {0, 0, 0, 0}; int nKeyWidth = 0; LsmString str; int nRec; LsmPgno iPtr; int flags2; int iCell; u8 *aData; int nData; /* Page data and size thereof */ aData = fsPageData(pPg, &nData); nRec = pageGetNRec(aData, nData); iPtr = pageGetPtr(aData, nData); flags2 = pageGetFlags(aData, nData); lsmStringInit(&str, pDb->pEnv); lsmStringAppendf(&str, "Page : %lld (%d bytes)\n", iPg, nData); lsmStringAppendf(&str, "nRec : %d\n", nRec); lsmStringAppendf(&str, "iPtr : %lld\n", iPtr); lsmStringAppendf(&str, "flags: %04x\n", flags2); lsmStringAppendf(&str, "\n"); for(iCell=0; iCell<nRec; iCell++){ int nKey; infoCellDump( pDb, pSeg, bIndirect, pPg, iCell, 0, 0, 0, &nKey, 0, 0, &blob |
︙ | ︙ |
Changes to ext/lsm1/lsm_unix.c.
︙ | ︙ | |||
224 225 226 227 228 229 230 231 232 233 234 235 236 237 | if( iSz<iMin ){ iSz = ((iMin + nIncrSz-1) / nIncrSz) * nIncrSz; prc = ftruncate(p->fd, iSz); if( prc!=0 ) return LSM_IOERR_BKPT; } p->pMap = mmap(0, iSz, PROT_READ|PROT_WRITE, MAP_SHARED, p->fd, 0); p->nMap = iSz; } *ppOut = p->pMap; *pnOut = p->nMap; return LSM_OK; } | > > > > | 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 | if( iSz<iMin ){ iSz = ((iMin + nIncrSz-1) / nIncrSz) * nIncrSz; prc = ftruncate(p->fd, iSz); if( prc!=0 ) return LSM_IOERR_BKPT; } p->pMap = mmap(0, iSz, PROT_READ|PROT_WRITE, MAP_SHARED, p->fd, 0); if( p->pMap==MAP_FAILED ){ p->pMap = 0; return LSM_IOERR_BKPT; } p->nMap = iSz; } *ppOut = p->pMap; *pnOut = p->nMap; return LSM_OK; } |
︙ | ︙ | |||
409 410 411 412 413 414 415 | p->nShm = nNew; } if( p->apShm[iChunk]==0 ){ p->apShm[iChunk] = mmap(0, LSM_SHM_CHUNK_SIZE, PROT_READ|PROT_WRITE, MAP_SHARED, p->shmfd, iChunk*LSM_SHM_CHUNK_SIZE ); | | > > > | 413 414 415 416 417 418 419 420 421 422 423 424 425 426 427 428 429 430 | p->nShm = nNew; } if( p->apShm[iChunk]==0 ){ p->apShm[iChunk] = mmap(0, LSM_SHM_CHUNK_SIZE, PROT_READ|PROT_WRITE, MAP_SHARED, p->shmfd, iChunk*LSM_SHM_CHUNK_SIZE ); if( p->apShm[iChunk]==MAP_FAILED ){ p->apShm[iChunk] = 0; return LSM_IOERR_BKPT; } } *ppShm = p->apShm[iChunk]; return LSM_OK; } static void lsmPosixOsShmBarrier(void){ |
︙ | ︙ |
Changes to ext/lsm1/lsm_varint.c.
︙ | ︙ | |||
180 181 182 183 184 185 186 187 188 189 190 191 192 193 | return ret; } int lsmVarintLen32(int n){ u8 aData[9]; return lsmVarintPut32(aData, n); } /* ** The argument is the first byte of a varint. This function returns the ** total number of bytes in the entire varint (including the first byte). */ int lsmVarintSize(u8 c){ if( c<241 ) return 1; | > > > > > | 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 | return ret; } int lsmVarintLen32(int n){ u8 aData[9]; return lsmVarintPut32(aData, n); } int lsmVarintLen64(i64 n){ u8 aData[9]; return lsmVarintPut64(aData, n); } /* ** The argument is the first byte of a varint. This function returns the ** total number of bytes in the entire varint (including the first byte). */ int lsmVarintSize(u8 c){ if( c<241 ) return 1; |
︙ | ︙ |
Changes to ext/lsm1/lsm_vtab.c.
︙ | ︙ | |||
22 23 24 25 26 27 28 | ** ** The keytype must be one of: UINT, TEXT, BLOB. All keys must be of that ** one type. "UINT" means unsigned integer. The values may be of any ** SQLite datatype: BLOB, TEXT, INTEGER, FLOAT, or NULL. ** ** The virtual table contains read-only hidden columns: ** | | | 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 | ** ** The keytype must be one of: UINT, TEXT, BLOB. All keys must be of that ** one type. "UINT" means unsigned integer. The values may be of any ** SQLite datatype: BLOB, TEXT, INTEGER, FLOAT, or NULL. ** ** The virtual table contains read-only hidden columns: ** ** lsm1_key A BLOB which is the raw LSM key. If the "keytype" ** is BLOB or TEXT then this column is exactly the ** same as the key. For the UINT keytype, this column ** will be a variable-length integer encoding of the key. ** ** lsm1_value A BLOB which is the raw LSM value. All of the value ** columns are packed into this BLOB using the encoding ** described below. |
︙ | ︙ | |||
838 839 840 841 842 843 844 | int argIdx = -1; /* Index of the key== constraint, or -1 if none */ int iIdx2 = -1; /* The index of the second key */ int omit1 = 0; int omit2 = 0; const struct sqlite3_index_constraint *pConstraint; pConstraint = pIdxInfo->aConstraint; | | | 838 839 840 841 842 843 844 845 846 847 848 849 850 851 852 | int argIdx = -1; /* Index of the key== constraint, or -1 if none */ int iIdx2 = -1; /* The index of the second key */ int omit1 = 0; int omit2 = 0; const struct sqlite3_index_constraint *pConstraint; pConstraint = pIdxInfo->aConstraint; for(i=0; i<pIdxInfo->nConstraint; i++, pConstraint++){ if( pConstraint->usable==0 ) continue; if( pConstraint->iColumn!=0 ) continue; switch( pConstraint->op ){ case SQLITE_INDEX_CONSTRAINT_EQ: { if( idxNum>0 ){ argIdx = i; iIdx2 = -1; |
︙ | ︙ |
Changes to ext/lsm1/test/lsm1_simple.test.
︙ | ︙ | |||
84 85 86 87 88 89 90 91 92 93 | INSERT INTO x1(a,b,c,d) VALUES(15, 11, 22, 33),(8,'banjo',x'333231',NULL), (12,NULL,3.25,-559281390); SELECT quote(a), quote(b), quote(c), quote(d), '|' FROM x1; } {'12' NULL 3.25 -559281390 | '15' 11 22 33 | '8' 'banjo' X'333231' NULL |} do_execsql_test 211 { SELECT quote(a), quote(lsm1_key), quote(lsm1_value), '|' FROM x1; } {'12' X'3132' X'05320000000000000A401FFB42ABE9DB' | '15' X'3135' X'4284C6' | '8' X'38' X'2162616E6A6F1633323105' |} finish_test | > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 | INSERT INTO x1(a,b,c,d) VALUES(15, 11, 22, 33),(8,'banjo',x'333231',NULL), (12,NULL,3.25,-559281390); SELECT quote(a), quote(b), quote(c), quote(d), '|' FROM x1; } {'12' NULL 3.25 -559281390 | '15' 11 22 33 | '8' 'banjo' X'333231' NULL |} do_execsql_test 211 { SELECT quote(a), quote(lsm1_key), quote(lsm1_value), '|' FROM x1; } {'12' X'3132' X'05320000000000000A401FFB42ABE9DB' | '15' X'3135' X'4284C6' | '8' X'38' X'2162616E6A6F1633323105' |} do_execsql_test 212 { SELECT quote(a), quote(lsm1_key), quote(lsm1_value) FROM x1 WHERE a='12'; } {'12' X'3132' X'05320000000000000A401FFB42ABE9DB'} #------------------------------------------------------------------------- reset_db forcedelete testlsm.db load_lsm1_vtab db do_execsql_test 300 { CREATE VIRTUAL TABLE x1 USING lsm1(testlsm.db,a,TEXT,b,c,d); } do_eqp_test 310 { SELECT * FROM x1 WHERE a=? } {SCAN TABLE x1 VIRTUAL TABLE INDEX 0:} do_eqp_test 320 { SELECT * FROM x1 WHERE a>? } {SCAN TABLE x1 VIRTUAL TABLE INDEX 2:} do_eqp_test 330 { SELECT * FROM x1 WHERE a<? } {SCAN TABLE x1 VIRTUAL TABLE INDEX 3:} do_eqp_test 340 { SELECT * FROM x1 WHERE a BETWEEN ? AND ? } {SCAN TABLE x1 VIRTUAL TABLE INDEX 1:} #------------------------------------------------------------------------- reset_db forcedelete testlsm.db load_lsm1_vtab db do_execsql_test 400 { CREATE VIRTUAL TABLE x1 USING lsm1(testlsm.db,a,TEXT,b); INSERT INTO x1 VALUES('one', 1); INSERT INTO x1 VALUES('two', 2); INSERT INTO x1 VALUES('three', 3); INSERT INTO x1 VALUES('four', 4); INSERT INTO x1 VALUES('five', 5); } do_execsql_test 410 { SELECT b FROM x1 WHERE a = 'two' } {2} do_execsql_test 411 { SELECT b FROM x1 WHERE a = 'one' } {1} do_execsql_test 412 { SELECT b FROM x1 WHERE a = 'five' } {5} do_execsql_test 420 { SELECT b FROM x1 WHERE a BETWEEN 'one' AND 'three'; } {1 3} do_execsql_test 421 { SELECT b FROM x1 WHERE a BETWEEN 'five' AND 'two'; } {5 4 1 3 2} do_execsql_test 421 { SELECT b FROM x1 WHERE a > 'five'; } {4 1 3 2} do_execsql_test 421 { SELECT b FROM x1 WHERE a <= 'three'; } {3 1 4 5} finish_test |
Changes to ext/misc/amatch.c.
︙ | ︙ | |||
615 616 617 618 619 620 621 | if( strcmp(zFrom,"")==0 && strcmp(zTo,"?")==0 ){ if( p->rIns==0 || p->rIns>rCost ) p->rIns = rCost; }else if( strcmp(zFrom,"?")==0 && strcmp(zTo,"")==0 ){ if( p->rDel==0 || p->rDel>rCost ) p->rDel = rCost; }else { | | | 615 616 617 618 619 620 621 622 623 624 625 626 627 628 629 | if( strcmp(zFrom,"")==0 && strcmp(zTo,"?")==0 ){ if( p->rIns==0 || p->rIns>rCost ) p->rIns = rCost; }else if( strcmp(zFrom,"?")==0 && strcmp(zTo,"")==0 ){ if( p->rDel==0 || p->rDel>rCost ) p->rDel = rCost; }else { pRule = sqlite3_malloc64( sizeof(*pRule) + nFrom + nTo ); if( pRule==0 ){ rc = SQLITE_NOMEM; }else{ memset(pRule, 0, sizeof(*pRule)); pRule->zFrom = &pRule->zTo[nTo+1]; pRule->nFrom = (amatch_len)nFrom; memcpy(pRule->zFrom, zFrom, nFrom+1); |
︙ | ︙ | |||
734 735 736 737 738 739 740 | ** ** "abc" becomes abc ** 'xyz' becomes xyz ** [pqr] becomes pqr ** `mno` becomes mno */ static char *amatchDequote(const char *zIn){ | | | | | | 734 735 736 737 738 739 740 741 742 743 744 745 746 747 748 749 750 751 752 753 754 755 756 757 | ** ** "abc" becomes abc ** 'xyz' becomes xyz ** [pqr] becomes pqr ** `mno` becomes mno */ static char *amatchDequote(const char *zIn){ sqlite3_int64 nIn; /* Size of input string, in bytes */ char *zOut; /* Output (dequoted) string */ nIn = strlen(zIn); zOut = sqlite3_malloc64(nIn+1); if( zOut ){ char q = zIn[0]; /* Quote character (if any ) */ if( q!='[' && q!= '\'' && q!='"' && q!='`' ){ memcpy(zOut, zIn, (size_t)(nIn+1)); }else{ int iOut = 0; /* Index of next byte to write to output */ int iIn; /* Index of next byte to read from input */ if( q=='[' ) q = ']'; for(iIn=1; iIn<nIn; iIn++){ if( zIn[iIn]==q ) iIn++; |
︙ | ︙ | |||
896 897 898 899 900 901 902 903 904 905 906 907 908 909 | if( pNew->zCostTab==0 ){ *pzErr = sqlite3_mprintf("no edit_distances table specified"); rc = SQLITE_ERROR; }else{ rc = amatchLoadRules(db, pNew, pzErr); } if( rc==SQLITE_OK ){ rc = sqlite3_declare_vtab(db, "CREATE TABLE x(word,distance,language," "command HIDDEN,nword HIDDEN)" ); #define AMATCH_COL_WORD 0 #define AMATCH_COL_DISTANCE 1 #define AMATCH_COL_LANGUAGE 2 | > | 896 897 898 899 900 901 902 903 904 905 906 907 908 909 910 | if( pNew->zCostTab==0 ){ *pzErr = sqlite3_mprintf("no edit_distances table specified"); rc = SQLITE_ERROR; }else{ rc = amatchLoadRules(db, pNew, pzErr); } if( rc==SQLITE_OK ){ sqlite3_vtab_config(db, SQLITE_VTAB_INNOCUOUS); rc = sqlite3_declare_vtab(db, "CREATE TABLE x(word,distance,language," "command HIDDEN,nword HIDDEN)" ); #define AMATCH_COL_WORD 0 #define AMATCH_COL_DISTANCE 1 #define AMATCH_COL_LANGUAGE 2 |
︙ | ︙ | |||
1065 1066 1067 1068 1069 1070 1071 | pWord->rCost, pWord->zWord, pWord->zCost); #endif pOther = amatchAvlInsert(&pCur->pCost, &pWord->sCost); assert( pOther==0 ); (void)pOther; } return; } | | | 1066 1067 1068 1069 1070 1071 1072 1073 1074 1075 1076 1077 1078 1079 1080 | pWord->rCost, pWord->zWord, pWord->zCost); #endif pOther = amatchAvlInsert(&pCur->pCost, &pWord->sCost); assert( pOther==0 ); (void)pOther; } return; } pWord = sqlite3_malloc64( sizeof(*pWord) + nBase + nTail - 1 ); if( pWord==0 ) return; memset(pWord, 0, sizeof(*pWord)); pWord->rCost = rCost; pWord->iSeq = pCur->nWord++; amatchWriteCost(pWord); pWord->nMatch = (short)nMatch; pWord->pNext = pCur->pAllWords; |
︙ | ︙ |
Changes to ext/misc/appendvfs.c.
︙ | ︙ | |||
10 11 12 13 14 15 16 | ** ****************************************************************************** ** ** This file implements a VFS shim that allows an SQLite database to be ** appended onto the end of some other file, such as an executable. ** ** A special record must appear at the end of the file that identifies the | | > | | | | | < < < | < | | > > | | | | | | | | > | | > > > > > > > > > | > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | | | > | 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 | ** ****************************************************************************** ** ** This file implements a VFS shim that allows an SQLite database to be ** appended onto the end of some other file, such as an executable. ** ** A special record must appear at the end of the file that identifies the ** file as an appended database and provides the offset to the first page ** of the exposed content. (Or, it is the length of the content prefix.) ** For best performance page 1 should be located at a disk page boundary, ** though that is not required. ** ** When opening a database using this VFS, the connection might treat ** the file as an ordinary SQLite database, or it might treat it as a ** database appended onto some other file. The decision is made by ** applying the following rules in order: ** ** (1) An empty file is an ordinary database. ** ** (2) If the file ends with the appendvfs trailer string ** "Start-Of-SQLite3-NNNNNNNN" that file is an appended database. ** ** (3) If the file begins with the standard SQLite prefix string ** "SQLite format 3", that file is an ordinary database. ** ** (4) If none of the above apply and the SQLITE_OPEN_CREATE flag is ** set, then a new database is appended to the already existing file. ** ** (5) Otherwise, SQLITE_CANTOPEN is returned. ** ** To avoid unnecessary complications with the PENDING_BYTE, the size of ** the file containing the database is limited to 1GiB. (1073741824 bytes) ** This VFS will not read or write past the 1GiB mark. This restriction ** might be lifted in future versions. For now, if you need a larger ** database, then keep it in a separate file. ** ** If the file being opened is a plain database (not an appended one), then ** this shim is a pass-through into the default underlying VFS. (rule 3) **/ #include "sqlite3ext.h" SQLITE_EXTENSION_INIT1 #include <string.h> #include <assert.h> /* The append mark at the end of the database is: ** ** Start-Of-SQLite3-NNNNNNNN ** 123456789 123456789 12345 ** ** The NNNNNNNN represents a 64-bit big-endian unsigned integer which is ** the offset to page 1, and also the length of the prefix content. */ #define APND_MARK_PREFIX "Start-Of-SQLite3-" #define APND_MARK_PREFIX_SZ 17 #define APND_MARK_FOS_SZ 8 #define APND_MARK_SIZE (APND_MARK_PREFIX_SZ+APND_MARK_FOS_SZ) /* ** Maximum size of the combined prefix + database + append-mark. This ** must be less than 0x40000000 to avoid locking issues on Windows. */ #define APND_MAX_SIZE (0x40000000) /* ** Try to align the database to an even multiple of APND_ROUNDUP bytes. */ #ifndef APND_ROUNDUP #define APND_ROUNDUP 4096 #endif #define APND_ALIGN_MASK ((sqlite3_int64)(APND_ROUNDUP-1)) #define APND_START_ROUNDUP(fsz) (((fsz)+APND_ALIGN_MASK) & ~APND_ALIGN_MASK) /* ** Forward declaration of objects used by this utility */ typedef struct sqlite3_vfs ApndVfs; typedef struct ApndFile ApndFile; /* Access to a lower-level VFS that (might) implement dynamic loading, ** access to randomness, etc. */ #define ORIGVFS(p) ((sqlite3_vfs*)((p)->pAppData)) #define ORIGFILE(p) ((sqlite3_file*)(((ApndFile*)(p))+1)) /* An open appendvfs file ** ** An instance of this structure describes the appended database file. ** A separate sqlite3_file object is always appended. The appended ** sqlite3_file object (which can be accessed using ORIGFILE()) describes ** the entire file, including the prefix, the database, and the ** append-mark. ** ** The structure of an AppendVFS database is like this: ** ** +-------------+---------+----------+-------------+ ** | prefix-file | padding | database | append-mark | ** +-------------+---------+----------+-------------+ ** ^ ^ ** | | ** iPgOne iMark ** ** ** "prefix file" - file onto which the database has been appended. ** "padding" - zero or more bytes inserted so that "database" ** starts on an APND_ROUNDUP boundary ** "database" - The SQLite database file ** "append-mark" - The 25-byte "Start-Of-SQLite3-NNNNNNNN" that indicates ** the offset from the start of prefix-file to the start ** of "database". ** ** The size of the database is iMark - iPgOne. ** ** The NNNNNNNN in the "Start-Of-SQLite3-NNNNNNNN" suffix is the value ** of iPgOne stored as a big-ending 64-bit integer. ** ** iMark will be the size of the underlying file minus 25 (APND_MARKSIZE). ** Or, iMark is -1 to indicate that it has not yet been written. */ struct ApndFile { sqlite3_file base; /* Subclass. MUST BE FIRST! */ sqlite3_int64 iPgOne; /* Offset to the start of the database */ sqlite3_int64 iMark; /* Offset of the append mark. -1 if unwritten */ /* Always followed by another sqlite3_file that describes the whole file */ }; /* ** Methods for ApndFile */ static int apndClose(sqlite3_file*); static int apndRead(sqlite3_file*, void*, int iAmt, sqlite3_int64 iOfst); |
︙ | ︙ | |||
174 175 176 177 178 179 180 | apndShmLock, /* xShmLock */ apndShmBarrier, /* xShmBarrier */ apndShmUnmap, /* xShmUnmap */ apndFetch, /* xFetch */ apndUnfetch /* xUnfetch */ }; | < < | | | > > | > > | > > > > > < > | > > > | > > > < | > > < | < < < < < < | | > | < < | < | < < < > | < > | > | < < < < | < | | 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 | apndShmLock, /* xShmLock */ apndShmBarrier, /* xShmBarrier */ apndShmUnmap, /* xShmUnmap */ apndFetch, /* xFetch */ apndUnfetch /* xUnfetch */ }; /* ** Close an apnd-file. */ static int apndClose(sqlite3_file *pFile){ pFile = ORIGFILE(pFile); return pFile->pMethods->xClose(pFile); } /* ** Read data from an apnd-file. */ static int apndRead( sqlite3_file *pFile, void *zBuf, int iAmt, sqlite_int64 iOfst ){ ApndFile *paf = (ApndFile *)pFile; pFile = ORIGFILE(pFile); return pFile->pMethods->xRead(pFile, zBuf, iAmt, paf->iPgOne+iOfst); } /* ** Add the append-mark onto what should become the end of the file. * If and only if this succeeds, internal ApndFile.iMark is updated. * Parameter iWriteEnd is the appendvfs-relative offset of the new mark. */ static int apndWriteMark( ApndFile *paf, sqlite3_file *pFile, sqlite_int64 iWriteEnd ){ sqlite_int64 iPgOne = paf->iPgOne; unsigned char a[APND_MARK_SIZE]; int i = APND_MARK_FOS_SZ; int rc; assert(pFile == ORIGFILE(paf)); memcpy(a, APND_MARK_PREFIX, APND_MARK_PREFIX_SZ); while( --i >= 0 ){ a[APND_MARK_PREFIX_SZ+i] = (unsigned char)(iPgOne & 0xff); iPgOne >>= 8; } iWriteEnd += paf->iPgOne; if( SQLITE_OK==(rc = pFile->pMethods->xWrite (pFile, a, APND_MARK_SIZE, iWriteEnd)) ){ paf->iMark = iWriteEnd; } return rc; } /* ** Write data to an apnd-file. */ static int apndWrite( sqlite3_file *pFile, const void *zBuf, int iAmt, sqlite_int64 iOfst ){ ApndFile *paf = (ApndFile *)pFile; sqlite_int64 iWriteEnd = iOfst + iAmt; if( iWriteEnd>=APND_MAX_SIZE ) return SQLITE_FULL; pFile = ORIGFILE(pFile); /* If append-mark is absent or will be overwritten, write it. */ if( paf->iMark < 0 || paf->iPgOne + iWriteEnd > paf->iMark ){ int rc = apndWriteMark(paf, pFile, iWriteEnd); if( SQLITE_OK!=rc ) return rc; } return pFile->pMethods->xWrite(pFile, zBuf, iAmt, paf->iPgOne+iOfst); } /* ** Truncate an apnd-file. */ static int apndTruncate(sqlite3_file *pFile, sqlite_int64 size){ ApndFile *paf = (ApndFile *)pFile; pFile = ORIGFILE(pFile); /* The append mark goes out first so truncate failure does not lose it. */ if( SQLITE_OK!=apndWriteMark(paf, pFile, size) ) return SQLITE_IOERR; /* Truncate underlying file just past append mark */ return pFile->pMethods->xTruncate(pFile, paf->iMark+APND_MARK_SIZE); } /* ** Sync an apnd-file. */ static int apndSync(sqlite3_file *pFile, int flags){ pFile = ORIGFILE(pFile); return pFile->pMethods->xSync(pFile, flags); } /* ** Return the current file-size of an apnd-file. ** If the append mark is not yet there, the file-size is 0. */ static int apndFileSize(sqlite3_file *pFile, sqlite_int64 *pSize){ ApndFile *paf = (ApndFile *)pFile; *pSize = ( paf->iMark >= 0 )? (paf->iMark - paf->iPgOne) : 0; return SQLITE_OK; } /* ** Lock an apnd-file. */ static int apndLock(sqlite3_file *pFile, int eLock){ pFile = ORIGFILE(pFile); |
︙ | ︙ | |||
304 305 306 307 308 309 310 | return pFile->pMethods->xCheckReservedLock(pFile, pResOut); } /* ** File control method. For custom operations on an apnd-file. */ static int apndFileControl(sqlite3_file *pFile, int op, void *pArg){ | | > | | 346 347 348 349 350 351 352 353 354 355 356 357 358 359 360 361 362 363 364 365 366 | return pFile->pMethods->xCheckReservedLock(pFile, pResOut); } /* ** File control method. For custom operations on an apnd-file. */ static int apndFileControl(sqlite3_file *pFile, int op, void *pArg){ ApndFile *paf = (ApndFile *)pFile; int rc; pFile = ORIGFILE(pFile); if( op==SQLITE_FCNTL_SIZE_HINT ) *(sqlite3_int64*)pArg += paf->iPgOne; rc = pFile->pMethods->xFileControl(pFile, op, pArg); if( rc==SQLITE_OK && op==SQLITE_FCNTL_VFSNAME ){ *(char**)pArg = sqlite3_mprintf("apnd(%lld)/%z", paf->iPgOne,*(char**)pArg); } return rc; } /* ** Return the sector-size in bytes for an apnd-file. */ |
︙ | ︙ | |||
368 369 370 371 372 373 374 375 376 377 378 379 380 381 382 383 384 385 386 | static int apndFetch( sqlite3_file *pFile, sqlite3_int64 iOfst, int iAmt, void **pp ){ ApndFile *p = (ApndFile *)pFile; pFile = ORIGFILE(pFile); return pFile->pMethods->xFetch(pFile, iOfst+p->iPgOne, iAmt, pp); } /* Release a memory-mapped page */ static int apndUnfetch(sqlite3_file *pFile, sqlite3_int64 iOfst, void *pPage){ ApndFile *p = (ApndFile *)pFile; pFile = ORIGFILE(pFile); return pFile->pMethods->xUnfetch(pFile, iOfst+p->iPgOne, pPage); } /* | > > > < < < < < < < < < < < < < | | > > > > > | | | > | > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | | | | | < > > > > | < | < | > > | > > | | > > | < > | > > > | < | | > | > > > > > > > > < < < > > > > > > > > > > < < < | 411 412 413 414 415 416 417 418 419 420 421 422 423 424 425 426 427 428 429 430 431 432 433 434 435 436 437 438 439 440 441 442 443 444 445 446 447 448 449 450 451 452 453 454 455 456 457 458 459 460 461 462 463 464 465 466 467 468 469 470 471 472 473 474 475 476 477 478 479 480 481 482 483 484 485 486 487 488 489 490 491 492 493 494 495 496 497 498 499 500 501 502 503 504 505 506 507 508 509 510 511 512 513 514 515 516 517 518 519 520 521 522 523 524 525 526 527 528 529 530 531 532 533 534 535 536 537 538 539 540 541 542 543 544 545 546 547 548 549 550 551 552 553 554 555 556 557 558 559 560 561 562 563 564 565 566 567 568 569 570 571 572 573 574 575 576 577 578 579 580 581 582 583 584 585 586 587 | static int apndFetch( sqlite3_file *pFile, sqlite3_int64 iOfst, int iAmt, void **pp ){ ApndFile *p = (ApndFile *)pFile; if( p->iMark < 0 || iOfst+iAmt > p->iMark ){ return SQLITE_IOERR; /* Cannot read what is not yet there. */ } pFile = ORIGFILE(pFile); return pFile->pMethods->xFetch(pFile, iOfst+p->iPgOne, iAmt, pp); } /* Release a memory-mapped page */ static int apndUnfetch(sqlite3_file *pFile, sqlite3_int64 iOfst, void *pPage){ ApndFile *p = (ApndFile *)pFile; pFile = ORIGFILE(pFile); return pFile->pMethods->xUnfetch(pFile, iOfst+p->iPgOne, pPage); } /* ** Try to read the append-mark off the end of a file. Return the ** start of the appended database if the append-mark is present. ** If there is no valid append-mark, return -1; ** ** An append-mark is only valid if the NNNNNNNN start-of-database offset ** indicates that the appended database contains at least one page. The ** start-of-database value must be a multiple of 512. */ static sqlite3_int64 apndReadMark(sqlite3_int64 sz, sqlite3_file *pFile){ int rc, i; sqlite3_int64 iMark; int msbs = 8 * (APND_MARK_FOS_SZ-1); unsigned char a[APND_MARK_SIZE]; if( APND_MARK_SIZE!=(sz & 0x1ff) ) return -1; rc = pFile->pMethods->xRead(pFile, a, APND_MARK_SIZE, sz-APND_MARK_SIZE); if( rc ) return -1; if( memcmp(a, APND_MARK_PREFIX, APND_MARK_PREFIX_SZ)!=0 ) return -1; iMark = ((sqlite3_int64)(a[APND_MARK_PREFIX_SZ] & 0x7f)) << msbs; for(i=1; i<8; i++){ msbs -= 8; iMark |= (sqlite3_int64)a[APND_MARK_PREFIX_SZ+i]<<msbs; } if( iMark > (sz - APND_MARK_SIZE - 512) ) return -1; if( iMark & 0x1ff ) return -1; return iMark; } static const char apvfsSqliteHdr[] = "SQLite format 3"; /* ** Check to see if the file is an appendvfs SQLite database file. ** Return true iff it is such. Parameter sz is the file's size. */ static int apndIsAppendvfsDatabase(sqlite3_int64 sz, sqlite3_file *pFile){ int rc; char zHdr[16]; sqlite3_int64 iMark = apndReadMark(sz, pFile); if( iMark>=0 ){ /* If file has the correct end-marker, the expected odd size, and the ** SQLite DB type marker where the end-marker puts it, then it ** is an appendvfs database. */ rc = pFile->pMethods->xRead(pFile, zHdr, sizeof(zHdr), iMark); if( SQLITE_OK==rc && memcmp(zHdr, apvfsSqliteHdr, sizeof(zHdr))==0 && (sz & 0x1ff) == APND_MARK_SIZE && sz>=512+APND_MARK_SIZE ){ return 1; /* It's an appendvfs database */ } } return 0; } /* ** Check to see if the file is an ordinary SQLite database file. ** Return true iff so. Parameter sz is the file's size. */ static int apndIsOrdinaryDatabaseFile(sqlite3_int64 sz, sqlite3_file *pFile){ char zHdr[16]; if( apndIsAppendvfsDatabase(sz, pFile) /* rule 2 */ || (sz & 0x1ff) != 0 || SQLITE_OK!=pFile->pMethods->xRead(pFile, zHdr, sizeof(zHdr), 0) || memcmp(zHdr, apvfsSqliteHdr, sizeof(zHdr))!=0 ){ return 0; }else{ return 1; } } /* ** Open an apnd file handle. */ static int apndOpen( sqlite3_vfs *pApndVfs, const char *zName, sqlite3_file *pFile, int flags, int *pOutFlags ){ ApndFile *pApndFile = (ApndFile*)pFile; sqlite3_file *pBaseFile = ORIGFILE(pFile); sqlite3_vfs *pBaseVfs = ORIGVFS(pApndVfs); int rc; sqlite3_int64 sz = 0; if( (flags & SQLITE_OPEN_MAIN_DB)==0 ){ /* The appendvfs is not to be used for transient or temporary databases. ** Just use the base VFS open to initialize the given file object and ** open the underlying file. (Appendvfs is then unused for this file.) */ return pBaseVfs->xOpen(pBaseVfs, zName, pFile, flags, pOutFlags); } memset(pApndFile, 0, sizeof(ApndFile)); pFile->pMethods = &apnd_io_methods; pApndFile->iMark = -1; /* Append mark not yet written */ rc = pBaseVfs->xOpen(pBaseVfs, zName, pBaseFile, flags, pOutFlags); if( rc==SQLITE_OK ){ rc = pBaseFile->pMethods->xFileSize(pBaseFile, &sz); if( rc ){ pBaseFile->pMethods->xClose(pBaseFile); } } if( rc ){ pFile->pMethods = 0; return rc; } if( apndIsOrdinaryDatabaseFile(sz, pBaseFile) ){ /* The file being opened appears to be just an ordinary DB. Copy ** the base dispatch-table so this instance mimics the base VFS. */ memmove(pApndFile, pBaseFile, pBaseVfs->szOsFile); return SQLITE_OK; } pApndFile->iPgOne = apndReadMark(sz, pFile); if( pApndFile->iPgOne>=0 ){ pApndFile->iMark = sz - APND_MARK_SIZE; /* Append mark found */ return SQLITE_OK; } if( (flags & SQLITE_OPEN_CREATE)==0 ){ pBaseFile->pMethods->xClose(pBaseFile); rc = SQLITE_CANTOPEN; pFile->pMethods = 0; }else{ /* Round newly added appendvfs location to #define'd page boundary. ** Note that nothing has yet been written to the underlying file. ** The append mark will be written along with first content write. ** Until then, paf->iMark value indicates it is not yet written. */ pApndFile->iPgOne = APND_START_ROUNDUP(sz); } return rc; } /* ** Delete an apnd file. ** For an appendvfs, this could mean delete the appendvfs portion, ** leaving the appendee as it was before it gained an appendvfs. ** For now, this code deletes the underlying file too. */ static int apndDelete(sqlite3_vfs *pVfs, const char *zPath, int dirSync){ return ORIGVFS(pVfs)->xDelete(ORIGVFS(pVfs), zPath, dirSync); } /* ** All other VFS methods are pass-thrus. */ static int apndAccess( sqlite3_vfs *pVfs, const char *zPath, int flags, int *pResOut ){ return ORIGVFS(pVfs)->xAccess(ORIGVFS(pVfs), zPath, flags, pResOut); |
︙ | ︙ | |||
547 548 549 550 551 552 553 554 555 556 557 558 559 560 561 562 563 564 565 | ){ int rc = SQLITE_OK; sqlite3_vfs *pOrig; SQLITE_EXTENSION_INIT2(pApi); (void)pzErrMsg; (void)db; pOrig = sqlite3_vfs_find(0); apnd_vfs.iVersion = pOrig->iVersion; apnd_vfs.pAppData = pOrig; apnd_vfs.szOsFile = pOrig->szOsFile + sizeof(ApndFile); rc = sqlite3_vfs_register(&apnd_vfs, 0); #ifdef APPENDVFS_TEST if( rc==SQLITE_OK ){ rc = sqlite3_auto_extension((void(*)(void))apndvfsRegister); } #endif if( rc==SQLITE_OK ) rc = SQLITE_OK_LOAD_PERMANENTLY; return rc; } | > | 653 654 655 656 657 658 659 660 661 662 663 664 665 666 667 668 669 670 671 672 | ){ int rc = SQLITE_OK; sqlite3_vfs *pOrig; SQLITE_EXTENSION_INIT2(pApi); (void)pzErrMsg; (void)db; pOrig = sqlite3_vfs_find(0); if( pOrig==0 ) return SQLITE_ERROR; apnd_vfs.iVersion = pOrig->iVersion; apnd_vfs.pAppData = pOrig; apnd_vfs.szOsFile = pOrig->szOsFile + sizeof(ApndFile); rc = sqlite3_vfs_register(&apnd_vfs, 0); #ifdef APPENDVFS_TEST if( rc==SQLITE_OK ){ rc = sqlite3_auto_extension((void(*)(void))apndvfsRegister); } #endif if( rc==SQLITE_OK ) rc = SQLITE_OK_LOAD_PERMANENTLY; return rc; } |
Added ext/misc/base64.c.
> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 | /* ** 2022-11-18 ** ** The author disclaims copyright to this source code. In place of ** a legal notice, here is a blessing: ** ** May you do good and not evil. ** May you find forgiveness for yourself and forgive others. ** May you share freely, never taking more than you give. ** ************************************************************************* ** ** This is a SQLite extension for converting in either direction ** between a (binary) blob and base64 text. Base64 can transit a ** sane USASCII channel unmolested. It also plays nicely in CSV or ** written as TCL brace-enclosed literals or SQL string literals, ** and can be used unmodified in XML-like documents. ** ** This is an independent implementation of conversions specified in ** RFC 4648, done on the above date by the author (Larry Brasfield) ** who thereby has the right to put this into the public domain. ** ** The conversions meet RFC 4648 requirements, provided that this ** C source specifies that line-feeds are included in the encoded ** data to limit visible line lengths to 72 characters and to ** terminate any encoded blob having non-zero length. ** ** Length limitations are not imposed except that the runtime ** SQLite string or blob length limits are respected. Otherwise, ** any length binary sequence can be represented and recovered. ** Generated base64 sequences, with their line-feeds included, ** can be concatenated; the result converted back to binary will ** be the concatenation of the represented binary sequences. ** ** This SQLite3 extension creates a function, base64(x), which ** either: converts text x containing base64 to a returned blob; ** or converts a blob x to returned text containing base64. An ** error will be thrown for other input argument types. ** ** This code relies on UTF-8 encoding only with respect to the ** meaning of the first 128 (7-bit) codes matching that of USASCII. ** It will fail miserably if somehow made to try to convert EBCDIC. ** Because it is table-driven, it could be enhanced to handle that, ** but the world and SQLite have moved on from that anachronism. ** ** To build the extension: ** Set shell variable SQDIR=<your favorite SQLite checkout directory> ** *Nix: gcc -O2 -shared -I$SQDIR -fPIC -o base64.so base64.c ** OSX: gcc -O2 -dynamiclib -fPIC -I$SQDIR -o base64.dylib base64.c ** Win32: gcc -O2 -shared -I%SQDIR% -o base64.dll base64.c ** Win32: cl /Os -I%SQDIR% base64.c -link -dll -out:base64.dll */ #include <assert.h> #include "sqlite3ext.h" #ifndef deliberate_fall_through /* Quiet some compilers about some of our intentional code. */ # if GCC_VERSION>=7000000 # define deliberate_fall_through __attribute__((fallthrough)); # else # define deliberate_fall_through # endif #endif SQLITE_EXTENSION_INIT1; #define PC 0x80 /* pad character */ #define WS 0x81 /* whitespace */ #define ND 0x82 /* Not above or digit-value */ #define PAD_CHAR '=' #ifndef U8_TYPEDEF typedef unsigned char u8; #define U8_TYPEDEF #endif static const u8 b64DigitValues[128] = { /* HT LF VT FF CR */ ND,ND,ND,ND, ND,ND,ND,ND, ND,WS,WS,WS, WS,WS,ND,ND, /* US */ ND,ND,ND,ND, ND,ND,ND,ND, ND,ND,ND,ND, ND,ND,ND,ND, /*sp + / */ WS,ND,ND,ND, ND,ND,ND,ND, ND,ND,ND,62, ND,ND,ND,63, /* 0 1 5 9 = */ 52,53,54,55, 56,57,58,59, 60,61,ND,ND, ND,PC,ND,ND, /* A O */ ND, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9,10, 11,12,13,14, /* P Z */ 15,16,17,18, 19,20,21,22, 23,24,25,ND, ND,ND,ND,ND, /* a o */ ND,26,27,28, 29,30,31,32, 33,34,35,36, 37,38,39,40, /* p z */ 41,42,43,44, 45,46,47,48, 49,50,51,ND, ND,ND,ND,ND }; static const char b64Numerals[64+1] = "ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789+/"; #define BX_DV_PROTO(c) \ ((((u8)(c))<0x80)? (u8)(b64DigitValues[(u8)(c)]) : 0x80) #define IS_BX_DIGIT(bdp) (((u8)(bdp))<0x80) #define IS_BX_WS(bdp) ((bdp)==WS) #define IS_BX_PAD(bdp) ((bdp)==PC) #define BX_NUMERAL(dv) (b64Numerals[(u8)(dv)]) /* Width of base64 lines. Should be an integer multiple of 4. */ #define B64_DARK_MAX 72 /* Encode a byte buffer into base64 text with linefeeds appended to limit ** encoded group lengths to B64_DARK_MAX or to terminate the last group. */ static char* toBase64( u8 *pIn, int nbIn, char *pOut ){ int nCol = 0; while( nbIn >= 3 ){ /* Do the bit-shuffle, exploiting unsigned input to avoid masking. */ pOut[0] = BX_NUMERAL(pIn[0]>>2); pOut[1] = BX_NUMERAL(((pIn[0]<<4)|(pIn[1]>>4))&0x3f); pOut[2] = BX_NUMERAL(((pIn[1]&0xf)<<2)|(pIn[2]>>6)); pOut[3] = BX_NUMERAL(pIn[2]&0x3f); pOut += 4; nbIn -= 3; pIn += 3; if( (nCol += 4)>=B64_DARK_MAX || nbIn<=0 ){ *pOut++ = '\n'; nCol = 0; } } if( nbIn > 0 ){ signed char nco = nbIn+1; int nbe; unsigned long qv = *pIn++; for( nbe=1; nbe<3; ++nbe ){ qv <<= 8; if( nbe<nbIn ) qv |= *pIn++; } for( nbe=3; nbe>=0; --nbe ){ char ce = (nbe<nco)? BX_NUMERAL((u8)(qv & 0x3f)) : PAD_CHAR; qv >>= 6; pOut[nbe] = ce; } pOut += 4; *pOut++ = '\n'; } *pOut = 0; return pOut; } /* Skip over text which is not base64 numeral(s). */ static char * skipNonB64( char *s ){ char c; while( (c = *s) && !IS_BX_DIGIT(BX_DV_PROTO(c)) ) ++s; return s; } /* Decode base64 text into a byte buffer. */ static u8* fromBase64( char *pIn, int ncIn, u8 *pOut ){ if( ncIn>0 && pIn[ncIn-1]=='\n' ) --ncIn; while( ncIn>0 && *pIn!=PAD_CHAR ){ static signed char nboi[] = { 0, 0, 1, 2, 3 }; char *pUse = skipNonB64(pIn); unsigned long qv = 0L; int nti, nbo, nac; ncIn -= (pUse - pIn); pIn = pUse; nti = (ncIn>4)? 4 : ncIn; ncIn -= nti; nbo = nboi[nti]; if( nbo==0 ) break; for( nac=0; nac<4; ++nac ){ char c = (nac<nti)? *pIn++ : b64Numerals[0]; u8 bdp = BX_DV_PROTO(c); switch( bdp ){ case ND: /* Treat dark non-digits as pad, but they terminate decode too. */ ncIn = 0; deliberate_fall_through; case WS: /* Treat whitespace as pad and terminate this group.*/ nti = nac; deliberate_fall_through; case PC: bdp = 0; --nbo; deliberate_fall_through; default: /* bdp is the digit value. */ qv = qv<<6 | bdp; break; } } switch( nbo ){ case 3: pOut[2] = (qv) & 0xff; case 2: pOut[1] = (qv>>8) & 0xff; case 1: pOut[0] = (qv>>16) & 0xff; } pOut += nbo; } return pOut; } /* This function does the work for the SQLite base64(x) UDF. */ static void base64(sqlite3_context *context, int na, sqlite3_value *av[]){ int nb, nc, nv = sqlite3_value_bytes(av[0]); int nvMax = sqlite3_limit(sqlite3_context_db_handle(context), SQLITE_LIMIT_LENGTH, -1); char *cBuf; u8 *bBuf; assert(na==1); switch( sqlite3_value_type(av[0]) ){ case SQLITE_BLOB: nb = nv; nc = 4*(nv+2/3); /* quads needed */ nc += (nc+(B64_DARK_MAX-1))/B64_DARK_MAX + 1; /* LFs and a 0-terminator */ if( nvMax < nc ){ sqlite3_result_error(context, "blob expanded to base64 too big", -1); return; } cBuf = sqlite3_malloc(nc); if( !cBuf ) goto memFail; bBuf = (u8*)sqlite3_value_blob(av[0]); nc = (int)(toBase64(bBuf, nb, cBuf) - cBuf); sqlite3_result_text(context, cBuf, nc, sqlite3_free); break; case SQLITE_TEXT: nc = nv; nb = 3*((nv+3)/4); /* may overestimate due to LF and padding */ if( nvMax < nb ){ sqlite3_result_error(context, "blob from base64 may be too big", -1); return; }else if( nb<1 ){ nb = 1; } bBuf = sqlite3_malloc(nb); if( !bBuf ) goto memFail; cBuf = (char *)sqlite3_value_text(av[0]); nb = (int)(fromBase64(cBuf, nc, bBuf) - bBuf); sqlite3_result_blob(context, bBuf, nb, sqlite3_free); break; default: sqlite3_result_error(context, "base64 accepts only blob or text", -1); return; } return; memFail: sqlite3_result_error(context, "base64 OOM", -1); } /* ** Establish linkage to running SQLite library. */ #ifndef SQLITE_SHELL_EXTFUNCS #ifdef _WIN32 __declspec(dllexport) #endif int sqlite3_base_init #else static int sqlite3_base64_init #endif (sqlite3 *db, char **pzErr, const sqlite3_api_routines *pApi){ SQLITE_EXTENSION_INIT2(pApi); (void)pzErr; return sqlite3_create_function (db, "base64", 1, SQLITE_DETERMINISTIC|SQLITE_INNOCUOUS|SQLITE_DIRECTONLY|SQLITE_UTF8, 0, base64, 0, 0); } /* ** Define some macros to allow this extension to be built into the shell ** conveniently, in conjunction with use of SQLITE_SHELL_EXTFUNCS. This ** allows shell.c, as distributed, to have this extension built in. */ #define BASE64_INIT(db) sqlite3_base64_init(db, 0, 0) #define BASE64_EXPOSE(db, pzErr) /* Not needed, ..._init() does this. */ |
Added ext/misc/base85.c.
> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340 341 342 343 344 345 346 347 348 349 350 351 352 353 354 355 356 357 358 359 360 361 362 363 364 365 366 367 368 369 370 371 372 373 374 375 376 377 378 379 380 381 382 383 384 385 386 387 388 389 390 391 392 393 394 395 396 397 398 399 400 401 402 403 404 405 406 407 408 409 410 411 412 413 414 415 416 417 418 419 420 421 422 423 424 425 426 427 428 429 430 431 432 433 434 435 436 | /* ** 2022-11-16 ** ** The author disclaims copyright to this source code. In place of ** a legal notice, here is a blessing: ** ** May you do good and not evil. ** May you find forgiveness for yourself and forgive others. ** May you share freely, never taking more than you give. ** ************************************************************************* ** ** This is a utility for converting binary to base85 or vice-versa. ** It can be built as a standalone program or an SQLite3 extension. ** ** Much like base64 representations, base85 can be sent through a ** sane USASCII channel unmolested. It also plays nicely in CSV or ** written as TCL brace-enclosed literals or SQL string literals. ** It is not suited for unmodified use in XML-like documents. ** ** The encoding used resembles Ascii85, but was devised by the author ** (Larry Brasfield) before Mozilla, Adobe, ZMODEM or other Ascii85 ** variant sources existed, in the 1984 timeframe on a VAX mainframe. ** Further, this is an independent implementation of a base85 system. ** Hence, the author has rightfully put this into the public domain. ** ** Base85 numerals are taken from the set of 7-bit USASCII codes, ** excluding control characters and Space ! " ' ( ) { | } ~ Del ** in code order representing digit values 0 to 84 (base 10.) ** ** Groups of 4 bytes, interpreted as big-endian 32-bit values, ** are represented as 5-digit base85 numbers with MS to LS digit ** order. Groups of 1-3 bytes are represented with 2-4 digits, ** still big-endian but 8-24 bit values. (Using big-endian yields ** the simplest transition to byte groups smaller than 4 bytes. ** These byte groups can also be considered base-256 numbers.) ** Groups of 0 bytes are represented with 0 digits and vice-versa. ** No pad characters are used; Encoded base85 numeral sequence ** (aka "group") length maps 1-to-1 to the decoded binary length. ** ** Any character not in the base85 numeral set delimits groups. ** When base85 is streamed or stored in containers of indefinite ** size, newline is used to separate it into sub-sequences of no ** more than 80 digits so that fgets() can be used to read it. ** ** Length limitations are not imposed except that the runtime ** SQLite string or blob length limits are respected. Otherwise, ** any length binary sequence can be represented and recovered. ** Base85 sequences can be concatenated by separating them with ** a non-base85 character; the conversion to binary will then ** be the concatenation of the represented binary sequences. ** The standalone program either converts base85 on stdin to create ** a binary file or converts a binary file to base85 on stdout. ** Read or make it blurt its help for invocation details. ** ** The SQLite3 extension creates a function, base85(x), which will ** either convert text base85 to a blob or a blob to text base85 ** and return the result (or throw an error for other types.) ** Unless built with OMIT_BASE85_CHECKER defined, it also creates a ** function, is_base85(t), which returns 1 iff the text t contains ** nothing other than base85 numerals and whitespace, or 0 otherwise. ** ** To build the extension: ** Set shell variable SQDIR=<your favorite SQLite checkout directory> ** and variable OPTS to -DOMIT_BASE85_CHECKER if is_base85() unwanted. ** *Nix: gcc -O2 -shared -I$SQDIR $OPTS -fPIC -o base85.so base85.c ** OSX: gcc -O2 -dynamiclib -fPIC -I$SQDIR $OPTS -o base85.dylib base85.c ** Win32: gcc -O2 -shared -I%SQDIR% %OPTS% -o base85.dll base85.c ** Win32: cl /Os -I%SQDIR% %OPTS% base85.c -link -dll -out:base85.dll ** ** To build the standalone program, define PP symbol BASE85_STANDALONE. Eg. ** *Nix or OSX: gcc -O2 -DBASE85_STANDALONE base85.c -o base85 ** Win32: gcc -O2 -DBASE85_STANDALONE -o base85.exe base85.c ** Win32: cl /Os /MD -DBASE85_STANDALONE base85.c */ #include <stdio.h> #include <memory.h> #include <string.h> #include <assert.h> #ifndef OMIT_BASE85_CHECKER # include <ctype.h> #endif #ifndef BASE85_STANDALONE # include "sqlite3ext.h" SQLITE_EXTENSION_INIT1; #else # ifdef _WIN32 # include <io.h> # include <fcntl.h> # else # define setmode(fd,m) # endif static char *zHelp = "Usage: base85 <dirFlag> <binFile>\n" " <dirFlag> is either -r to read or -w to write <binFile>,\n" " content to be converted to/from base85 on stdout/stdin.\n" " <binFile> names a binary file to be rendered or created.\n" " Or, the name '-' refers to the stdin or stdout stream.\n" ; static void sayHelp(){ printf("%s", zHelp); } #endif #ifndef U8_TYPEDEF typedef unsigned char u8; #define U8_TYPEDEF #endif /* Classify c according to interval within USASCII set w.r.t. base85 * Values of 1 and 3 are base85 numerals. Values of 0, 2, or 4 are not. */ #define B85_CLASS( c ) (((c)>='#')+((c)>'&')+((c)>='*')+((c)>'z')) /* Provide digitValue to b85Numeral offset as a function of above class. */ static u8 b85_cOffset[] = { 0, '#', 0, '*'-4, 0 }; #define B85_DNOS( c ) b85_cOffset[B85_CLASS(c)] /* Say whether c is a base85 numeral. */ #define IS_B85( c ) (B85_CLASS(c) & 1) #if 0 /* Not used, */ static u8 base85DigitValue( char c ){ u8 dv = (u8)(c - '#'); if( dv>87 ) return 0xff; return (dv > 3)? dv-3 : dv; } #endif /* Width of base64 lines. Should be an integer multiple of 5. */ #define B85_DARK_MAX 80 static char * skipNonB85( char *s ){ char c; while( (c = *s) && !IS_B85(c) ) ++s; return s; } /* Convert small integer, known to be in 0..84 inclusive, to base85 numeral. * Do not use the macro form with argument expression having a side-effect.*/ #if 0 static char base85Numeral( u8 b ){ return (b < 4)? (char)(b + '#') : (char)(b - 4 + '*'); } #else # define base85Numeral( dn )\ ((char)(((dn) < 4)? (char)((dn) + '#') : (char)((dn) - 4 + '*'))) #endif static char *putcs(char *pc, char *s){ char c; while( (c = *s++)!=0 ) *pc++ = c; return pc; } /* Encode a byte buffer into base85 text. If pSep!=0, it's a C string ** to be appended to encoded groups to limit their length to B85_DARK_MAX ** or to terminate the last group (to aid concatenation.) */ static char* toBase85( u8 *pIn, int nbIn, char *pOut, char *pSep ){ int nCol = 0; while( nbIn >= 4 ){ int nco = 5; unsigned long qbv = (((unsigned long)pIn[0])<<24) | (pIn[1]<<16) | (pIn[2]<<8) | pIn[3]; while( nco > 0 ){ unsigned nqv = (unsigned)(qbv/85UL); unsigned char dv = qbv - 85UL*nqv; qbv = nqv; pOut[--nco] = base85Numeral(dv); } nbIn -= 4; pIn += 4; pOut += 5; if( pSep && (nCol += 5)>=B85_DARK_MAX ){ pOut = putcs(pOut, pSep); nCol = 0; } } if( nbIn > 0 ){ int nco = nbIn + 1; unsigned long qv = *pIn++; int nbe = 1; while( nbe++ < nbIn ){ qv = (qv<<8) | *pIn++; } nCol += nco; while( nco > 0 ){ u8 dv = (u8)(qv % 85); qv /= 85; pOut[--nco] = base85Numeral(dv); } pOut += (nbIn+1); } if( pSep && nCol>0 ) pOut = putcs(pOut, pSep); *pOut = 0; return pOut; } /* Decode base85 text into a byte buffer. */ static u8* fromBase85( char *pIn, int ncIn, u8 *pOut ){ if( ncIn>0 && pIn[ncIn-1]=='\n' ) --ncIn; while( ncIn>0 ){ static signed char nboi[] = { 0, 0, 1, 2, 3, 4 }; char *pUse = skipNonB85(pIn); unsigned long qv = 0L; int nti, nbo; ncIn -= (pUse - pIn); pIn = pUse; nti = (ncIn>5)? 5 : ncIn; nbo = nboi[nti]; if( nbo==0 ) break; while( nti>0 ){ char c = *pIn++; u8 cdo = B85_DNOS(c); --ncIn; if( cdo==0 ) break; qv = 85 * qv + (c - cdo); --nti; } nbo -= nti; /* Adjust for early (non-digit) end of group. */ switch( nbo ){ case 4: *pOut++ = (qv >> 24)&0xff; case 3: *pOut++ = (qv >> 16)&0xff; case 2: *pOut++ = (qv >> 8)&0xff; case 1: *pOut++ = qv&0xff; case 0: break; } } return pOut; } #ifndef OMIT_BASE85_CHECKER /* Say whether input char sequence is all (base85 and/or whitespace).*/ static int allBase85( char *p, int len ){ char c; while( len-- > 0 && (c = *p++) != 0 ){ if( !IS_B85(c) && !isspace(c) ) return 0; } return 1; } #endif #ifndef BASE85_STANDALONE # ifndef OMIT_BASE85_CHECKER /* This function does the work for the SQLite is_base85(t) UDF. */ static void is_base85(sqlite3_context *context, int na, sqlite3_value *av[]){ assert(na==1); switch( sqlite3_value_type(av[0]) ){ case SQLITE_TEXT: { int rv = allBase85( (char *)sqlite3_value_text(av[0]), sqlite3_value_bytes(av[0]) ); sqlite3_result_int(context, rv); } break; case SQLITE_NULL: sqlite3_result_null(context); break; default: sqlite3_result_error(context, "is_base85 accepts only text or NULL", -1); return; } } # endif /* This function does the work for the SQLite base85(x) UDF. */ static void base85(sqlite3_context *context, int na, sqlite3_value *av[]){ int nb, nc, nv = sqlite3_value_bytes(av[0]); int nvMax = sqlite3_limit(sqlite3_context_db_handle(context), SQLITE_LIMIT_LENGTH, -1); char *cBuf; u8 *bBuf; assert(na==1); switch( sqlite3_value_type(av[0]) ){ case SQLITE_BLOB: nb = nv; /* ulongs tail newlines tailenc+nul*/ nc = 5*(nv/4) + nv%4 + nv/64+1 + 2; if( nvMax < nc ){ sqlite3_result_error(context, "blob expanded to base85 too big", -1); return; } cBuf = sqlite3_malloc(nc); if( !cBuf ) goto memFail; bBuf = (u8*)sqlite3_value_blob(av[0]); nc = (int)(toBase85(bBuf, nb, cBuf, "\n") - cBuf); sqlite3_result_text(context, cBuf, nc, sqlite3_free); break; case SQLITE_TEXT: nc = nv; nb = 4*(nv/5) + nv%5; /* may overestimate */ if( nvMax < nb ){ sqlite3_result_error(context, "blob from base85 may be too big", -1); return; }else if( nb<1 ){ nb = 1; } bBuf = sqlite3_malloc(nb); if( !bBuf ) goto memFail; cBuf = (char *)sqlite3_value_text(av[0]); nb = (int)(fromBase85(cBuf, nc, bBuf) - bBuf); sqlite3_result_blob(context, bBuf, nb, sqlite3_free); break; default: sqlite3_result_error(context, "base85 accepts only blob or text.", -1); return; } return; memFail: sqlite3_result_error(context, "base85 OOM", -1); } /* ** Establish linkage to running SQLite library. */ #ifndef SQLITE_SHELL_EXTFUNCS #ifdef _WIN32 __declspec(dllexport) #endif int sqlite3_base_init #else static int sqlite3_base85_init #endif (sqlite3 *db, char **pzErr, const sqlite3_api_routines *pApi){ SQLITE_EXTENSION_INIT2(pApi); (void)pzErr; # ifndef OMIT_BASE85_CHECKER { int rc = sqlite3_create_function (db, "is_base85", 1, SQLITE_DETERMINISTIC|SQLITE_INNOCUOUS|SQLITE_UTF8, 0, is_base85, 0, 0); if( rc!=SQLITE_OK ) return rc; } # endif return sqlite3_create_function (db, "base85", 1, SQLITE_DETERMINISTIC|SQLITE_INNOCUOUS|SQLITE_DIRECTONLY|SQLITE_UTF8, 0, base85, 0, 0); } /* ** Define some macros to allow this extension to be built into the shell ** conveniently, in conjunction with use of SQLITE_SHELL_EXTFUNCS. This ** allows shell.c, as distributed, to have this extension built in. */ # define BASE85_INIT(db) sqlite3_base85_init(db, 0, 0) # define BASE85_EXPOSE(db, pzErr) /* Not needed, ..._init() does this. */ #else /* standalone program */ int main(int na, char *av[]){ int cin; int rc = 0; u8 bBuf[4*(B85_DARK_MAX/5)]; char cBuf[5*(sizeof(bBuf)/4)+2]; size_t nio; # ifndef OMIT_BASE85_CHECKER int b85Clean = 1; # endif char rw; FILE *fb = 0, *foc = 0; char fmode[3] = "xb"; if( na < 3 || av[1][0]!='-' || (rw = av[1][1])==0 || (rw!='r' && rw!='w') ){ sayHelp(); return 0; } fmode[0] = rw; if( av[2][0]=='-' && av[2][1]==0 ){ switch( rw ){ case 'r': fb = stdin; setmode(fileno(stdin), O_BINARY); break; case 'w': fb = stdout; setmode(fileno(stdout), O_BINARY); break; } }else{ fb = fopen(av[2], fmode); foc = fb; } if( !fb ){ fprintf(stderr, "Cannot open %s for %c\n", av[2], rw); rc = 1; }else{ switch( rw ){ case 'r': while( (nio = fread( bBuf, 1, sizeof(bBuf), fb))>0 ){ toBase85( bBuf, (int)nio, cBuf, 0 ); fprintf(stdout, "%s\n", cBuf); } break; case 'w': while( 0 != fgets(cBuf, sizeof(cBuf), stdin) ){ int nc = strlen(cBuf); size_t nbo = fromBase85( cBuf, nc, bBuf ) - bBuf; if( 1 != fwrite(bBuf, nbo, 1, fb) ) rc = 1; # ifndef OMIT_BASE85_CHECKER b85Clean &= allBase85( cBuf, nc ); # endif } break; default: sayHelp(); rc = 1; } if( foc ) fclose(foc); } # ifndef OMIT_BASE85_CHECKER if( !b85Clean ){ fprintf(stderr, "Base85 input had non-base85 dark or control content.\n"); } # endif return rc; } #endif |
Added ext/misc/basexx.c.
> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 | /* ** 2022-11-20 ** ** The author disclaims copyright to this source code. In place of ** a legal notice, here is a blessing: ** ** May you do good and not evil. ** May you find forgiveness for yourself and forgive others. ** May you share freely, never taking more than you give. ** ************************************************************************* ** ** This source allows multiple SQLite extensions to be either: combined ** into a single runtime-loadable library; or built into the SQLite shell ** using a preprocessing convention set by src/shell.c.in (and shell.c). ** ** Presently, it combines the base64.c and base85.c extensions. However, ** it can be used as a template for other combinations. ** ** Example usages: ** ** - Build a runtime-loadable extension from SQLite checkout directory: ** *Nix, OSX: gcc -O2 -shared -I. -fPIC -o basexx.so ext/misc/basexx.c ** Win32: cl /Os -I. ext/misc/basexx.c -link -dll -out:basexx.dll ** ** - Incorporate as built-in in sqlite3 shell: ** *Nix, OSX with gcc on a like platform: ** export mop1=-DSQLITE_SHELL_EXTSRC=ext/misc/basexx.c ** export mop2=-DSQLITE_SHELL_EXTFUNCS=BASEXX ** make sqlite3 "OPTS=$mop1 $mop2" ** Win32 with Microsoft toolset on Windows: ** set mop1=-DSQLITE_SHELL_EXTSRC=ext/misc/basexx.c ** set mop2=-DSQLITE_SHELL_EXTFUNCS=BASEXX ** set mops="OPTS=%mop1% %mop2%" ** nmake -f Makefile.msc sqlite3.exe %mops% */ #ifndef SQLITE_SHELL_EXTFUNCS /* Guard for #include as built-in extension. */ # include "sqlite3ext.h" SQLITE_EXTENSION_INIT1; #endif static void init_api_ptr(const sqlite3_api_routines *pApi){ SQLITE_EXTENSION_INIT2(pApi); } #undef SQLITE_EXTENSION_INIT1 #define SQLITE_EXTENSION_INIT1 /* */ #undef SQLITE_EXTENSION_INIT2 #define SQLITE_EXTENSION_INIT2(v) (void)v typedef unsigned char u8; #define U8_TYPEDEF /* These next 2 undef's are only needed because the entry point names * collide when formulated per the rules stated for loadable extension * entry point names that will be deduced from the file basenames. */ #undef sqlite3_base_init #define sqlite3_base_init sqlite3_base64_init #include "base64.c" #undef sqlite3_base_init #define sqlite3_base_init sqlite3_base85_init #include "base85.c" #ifdef _WIN32 __declspec(dllexport) #endif int sqlite3_basexx_init(sqlite3 *db, char **pzErr, const sqlite3_api_routines *pApi){ init_api_ptr(pApi); int rc1 = BASE64_INIT(db); int rc2 = BASE85_INIT(db); if( rc1==SQLITE_OK && rc2==SQLITE_OK ){ BASE64_EXPOSE(db, pzErr); BASE64_EXPOSE(db, pzErr); return SQLITE_OK; }else{ return SQLITE_ERROR; } } # define BASEXX_INIT(db) sqlite3_basexx_init(db, 0, 0) # define BASEXX_EXPOSE(db, pzErr) /* Not needed, ..._init() does this. */ |
Added ext/misc/bgckpt.c.
> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 | /* ** 2017-10-11 ** ** The author disclaims copyright to this source code. In place of ** a legal notice, here is a blessing: ** ** May you do good and not evil. ** May you find forgiveness for yourself and forgive others. ** May you share freely, never taking more than you give. ** ****************************************************************************** ** */ #if !defined(SQLITE_TEST) || defined(SQLITE_OS_UNIX) #include "sqlite3.h" #include <string.h> #include <pthread.h> /* ** API declarations. */ typedef struct Checkpointer Checkpointer; int sqlite3_bgckpt_create(const char *zFilename, Checkpointer **pp); int sqlite3_bgckpt_checkpoint(Checkpointer *p, int bBlock); void sqlite3_bgckpt_destroy(Checkpointer *p); struct Checkpointer { sqlite3 *db; /* Database handle */ pthread_t thread; /* Background thread */ pthread_mutex_t mutex; pthread_cond_t cond; int rc; /* Error from "PRAGMA wal_checkpoint" */ int bCkpt; /* True if checkpoint requested */ int bExit; /* True if exit requested */ }; static void *bgckptThreadMain(void *pCtx){ int rc = SQLITE_OK; Checkpointer *p = (Checkpointer*)pCtx; while( rc==SQLITE_OK ){ int bExit; pthread_mutex_lock(&p->mutex); if( p->bCkpt==0 && p->bExit==0 ){ pthread_cond_wait(&p->cond, &p->mutex); } p->bCkpt = 0; bExit = p->bExit; pthread_mutex_unlock(&p->mutex); if( bExit ) break; rc = sqlite3_exec(p->db, "PRAGMA wal_checkpoint", 0, 0, 0); if( rc==SQLITE_BUSY ){ rc = SQLITE_OK; } } pthread_mutex_lock(&p->mutex); p->rc = rc; pthread_mutex_unlock(&p->mutex); return 0; } void sqlite3_bgckpt_destroy(Checkpointer *p){ if( p ){ void *ret = 0; /* Signal the background thread to exit */ pthread_mutex_lock(&p->mutex); p->bExit = 1; pthread_cond_broadcast(&p->cond); pthread_mutex_unlock(&p->mutex); pthread_join(p->thread, &ret); sqlite3_close(p->db); sqlite3_free(p); } } int sqlite3_bgckpt_create(const char *zFilename, Checkpointer **pp){ Checkpointer *pNew = 0; int rc; pNew = (Checkpointer*)sqlite3_malloc(sizeof(Checkpointer)); if( pNew==0 ){ rc = SQLITE_NOMEM; }else{ memset(pNew, 0, sizeof(Checkpointer)); rc = sqlite3_open(zFilename, &pNew->db); } if( rc==SQLITE_OK ){ pthread_mutex_init(&pNew->mutex, 0); pthread_cond_init(&pNew->cond, 0); pthread_create(&pNew->thread, 0, bgckptThreadMain, (void*)pNew); } if( rc!=SQLITE_OK ){ sqlite3_bgckpt_destroy(pNew); pNew = 0; } *pp = pNew; return rc; } int sqlite3_bgckpt_checkpoint(Checkpointer *p, int bBlock){ int rc; pthread_mutex_lock(&p->mutex); rc = p->rc; if( rc==SQLITE_OK ){ p->bCkpt = 1; pthread_cond_broadcast(&p->cond); } pthread_mutex_unlock(&p->mutex); return rc; } #ifdef SQLITE_TEST #if defined(INCLUDE_SQLITE_TCL_H) # include "sqlite_tcl.h" #else # include "tcl.h" # ifndef SQLITE_TCLAPI # define SQLITE_TCLAPI # endif #endif const char *sqlite3ErrName(int rc); static void SQLITE_TCLAPI bgckpt_del(void * clientData){ Checkpointer *pCkpt = (Checkpointer*)clientData; sqlite3_bgckpt_destroy(pCkpt); } /* ** Tclcmd: $ckpt SUBCMD ... */ static int SQLITE_TCLAPI bgckpt_obj_cmd( void * clientData, Tcl_Interp *interp, int objc, Tcl_Obj *CONST objv[] ){ Checkpointer *pCkpt = (Checkpointer*)clientData; const char *aCmd[] = { "checkpoint", "destroy", 0 }; int iCmd; if( objc<2 ){ Tcl_WrongNumArgs(interp, 1, objv, "SUBCMD ..."); return TCL_ERROR; } if( Tcl_GetIndexFromObj(interp, objv[1], aCmd, "sub-command", 0, &iCmd) ){ return TCL_ERROR; } switch( iCmd ){ case 0: { int rc; int bBlock = 0; if( objc>3 ){ Tcl_WrongNumArgs(interp, 2, objv, "?BLOCKING?"); return TCL_ERROR; } if( objc==3 && Tcl_GetBooleanFromObj(interp, objv[2], &bBlock) ){ return TCL_ERROR; } rc = sqlite3_bgckpt_checkpoint(pCkpt, bBlock); if( rc!=SQLITE_OK ){ Tcl_SetObjResult(interp, Tcl_NewStringObj(sqlite3ErrName(rc), -1)); return TCL_ERROR; } break; } case 1: { Tcl_DeleteCommand(interp, Tcl_GetString(objv[0])); break; } } return TCL_OK; } /* ** Tclcmd: bgckpt CMDNAME FILENAME */ static int SQLITE_TCLAPI bgckpt_cmd( void * clientData, Tcl_Interp *interp, int objc, Tcl_Obj *CONST objv[] ){ const char *zCmd; const char *zFilename; int rc; Checkpointer *pCkpt; if( objc!=3 ){ Tcl_WrongNumArgs(interp, 1, objv, "CMDNAME FILENAME"); return TCL_ERROR; } zCmd = Tcl_GetString(objv[1]); zFilename = Tcl_GetString(objv[2]); rc = sqlite3_bgckpt_create(zFilename, &pCkpt); if( rc!=SQLITE_OK ){ Tcl_SetObjResult(interp, Tcl_NewStringObj(sqlite3ErrName(rc), -1)); return TCL_ERROR; } Tcl_CreateObjCommand(interp, zCmd, bgckpt_obj_cmd, (void*)pCkpt, bgckpt_del); Tcl_SetObjResult(interp, objv[1]); return TCL_OK; } int Bgckpt_Init(Tcl_Interp *interp){ Tcl_CreateObjCommand(interp, "bgckpt", bgckpt_cmd, 0, 0); return TCL_OK; } #endif /* SQLITE_TEST */ #else #if defined(INCLUDE_SQLITE_TCL_H) # include "sqlite_tcl.h" #else # include "tcl.h" # ifndef SQLITE_TCLAPI # define SQLITE_TCLAPI # endif #endif int Bgckpt_Init(Tcl_Interp *interp){ return TCL_OK; } #endif |
Added ext/misc/blobio.c.
> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 | /* ** 2019-03-30 ** ** The author disclaims copyright to this source code. In place of ** a legal notice, here is a blessing: ** ** May you do good and not evil. ** May you find forgiveness for yourself and forgive others. ** May you share freely, never taking more than you give. ** ****************************************************************************** ** ** An SQL function that uses the incremental BLOB I/O mechanism of SQLite ** to read or write part of a blob. This is intended for debugging use ** in the CLI. ** ** readblob(SCHEMA,TABLE,COLUMN,ROWID,OFFSET,N) ** ** Returns N bytes of the blob starting at OFFSET. ** ** writeblob(SCHEMA,TABLE,COLUMN,ROWID,OFFSET,NEWDATA) ** ** NEWDATA must be a blob. The content of NEWDATA overwrites the ** existing BLOB data at SCHEMA.TABLE.COLUMN for row ROWID beginning ** at OFFSET bytes into the blob. */ #include "sqlite3ext.h" SQLITE_EXTENSION_INIT1 #include <assert.h> #include <string.h> static void readblobFunc( sqlite3_context *context, int argc, sqlite3_value **argv ){ sqlite3_blob *pBlob = 0; const char *zSchema; const char *zTable; const char *zColumn; sqlite3_int64 iRowid; int iOfst; unsigned char *aData; int nData; sqlite3 *db; int rc; zSchema = (const char*)sqlite3_value_text(argv[0]); zTable = (const char*)sqlite3_value_text(argv[1]); if( zTable==0 ){ sqlite3_result_error(context, "bad table name", -1); return; } zColumn = (const char*)sqlite3_value_text(argv[2]); if( zTable==0 ){ sqlite3_result_error(context, "bad column name", -1); return; } iRowid = sqlite3_value_int64(argv[3]); iOfst = sqlite3_value_int(argv[4]); nData = sqlite3_value_int(argv[5]); if( nData<=0 ) return; aData = sqlite3_malloc64( nData+1 ); if( aData==0 ){ sqlite3_result_error_nomem(context); return; } db = sqlite3_context_db_handle(context); rc = sqlite3_blob_open(db, zSchema, zTable, zColumn, iRowid, 0, &pBlob); if( rc ){ sqlite3_free(aData); sqlite3_result_error(context, "cannot open BLOB pointer", -1); return; } rc = sqlite3_blob_read(pBlob, aData, nData, iOfst); sqlite3_blob_close(pBlob); if( rc ){ sqlite3_free(aData); sqlite3_result_error(context, "BLOB read failed", -1); }else{ sqlite3_result_blob(context, aData, nData, sqlite3_free); } } static void writeblobFunc( sqlite3_context *context, int argc, sqlite3_value **argv ){ sqlite3_blob *pBlob = 0; const char *zSchema; const char *zTable; const char *zColumn; sqlite3_int64 iRowid; int iOfst; unsigned char *aData; int nData; sqlite3 *db; int rc; zSchema = (const char*)sqlite3_value_text(argv[0]); zTable = (const char*)sqlite3_value_text(argv[1]); if( zTable==0 ){ sqlite3_result_error(context, "bad table name", -1); return; } zColumn = (const char*)sqlite3_value_text(argv[2]); if( zTable==0 ){ sqlite3_result_error(context, "bad column name", -1); return; } iRowid = sqlite3_value_int64(argv[3]); iOfst = sqlite3_value_int(argv[4]); if( sqlite3_value_type(argv[5])!=SQLITE_BLOB ){ sqlite3_result_error(context, "6th argument must be a BLOB", -1); return; } nData = sqlite3_value_bytes(argv[5]); aData = (unsigned char *)sqlite3_value_blob(argv[5]); db = sqlite3_context_db_handle(context); rc = sqlite3_blob_open(db, zSchema, zTable, zColumn, iRowid, 1, &pBlob); if( rc ){ sqlite3_result_error(context, "cannot open BLOB pointer", -1); return; } rc = sqlite3_blob_write(pBlob, aData, nData, iOfst); sqlite3_blob_close(pBlob); if( rc ){ sqlite3_result_error(context, "BLOB write failed", -1); } } #ifdef _WIN32 __declspec(dllexport) #endif int sqlite3_blobio_init( sqlite3 *db, char **pzErrMsg, const sqlite3_api_routines *pApi ){ int rc = SQLITE_OK; SQLITE_EXTENSION_INIT2(pApi); (void)pzErrMsg; /* Unused parameter */ rc = sqlite3_create_function(db, "readblob", 6, SQLITE_UTF8, 0, readblobFunc, 0, 0); if( rc==SQLITE_OK ){ rc = sqlite3_create_function(db, "writeblob", 6, SQLITE_UTF8, 0, writeblobFunc, 0, 0); } return rc; } |
Changes to ext/misc/btreeinfo.c.
︙ | ︙ | |||
17 18 19 20 21 22 23 | ** The schema is like this: ** ** CREATE TABLE sqlite_btreeinfo( ** type TEXT, -- "table" or "index" ** name TEXT, -- Name of table or index for this btree. ** tbl_name TEXT, -- Associated table ** rootpage INT, -- The root page of the btree | | | | | | 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 | ** The schema is like this: ** ** CREATE TABLE sqlite_btreeinfo( ** type TEXT, -- "table" or "index" ** name TEXT, -- Name of table or index for this btree. ** tbl_name TEXT, -- Associated table ** rootpage INT, -- The root page of the btree ** sql TEXT, -- SQL for this btree - from sqlite_schema ** hasRowid BOOLEAN, -- True if the btree has a rowid ** nEntry INT, -- Estimated number of entries ** nPage INT, -- Estimated number of pages ** depth INT, -- Depth of the btree ** szPage INT, -- Size of each page in bytes ** zSchema TEXT HIDDEN -- The schema to which this btree belongs ** ); ** ** The first 5 fields are taken directly from the sqlite_schema table. ** Considering only the first 5 fields, the only difference between ** this virtual table and the sqlite_schema table is that this virtual ** table omits all entries that have a 0 or NULL rowid - in other words ** it omits triggers and views. ** ** The value added by this table comes in the next 5 fields. ** ** Note that nEntry and nPage are *estimated*. They are computed doing ** a single search from the root to a leaf, counting the number of cells |
︙ | ︙ | |||
84 85 86 87 88 89 90 | /* Forward declarations */ typedef struct BinfoTable BinfoTable; typedef struct BinfoCursor BinfoCursor; /* A cursor for the sqlite_btreeinfo table */ struct BinfoCursor { sqlite3_vtab_cursor base; /* Base class. Must be first */ | | | 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 | /* Forward declarations */ typedef struct BinfoTable BinfoTable; typedef struct BinfoCursor BinfoCursor; /* A cursor for the sqlite_btreeinfo table */ struct BinfoCursor { sqlite3_vtab_cursor base; /* Base class. Must be first */ sqlite3_stmt *pStmt; /* Query against sqlite_schema */ int rc; /* Result of previous sqlite_step() call */ int hasRowid; /* hasRowid value. Negative if unknown. */ sqlite3_int64 nEntry; /* nEntry value */ int nPage; /* nPage value */ int depth; /* depth value */ int szPage; /* size of a btree page. 0 if unknown */ char *zSchema; /* Schema being interrogated */ |
︙ | ︙ | |||
238 239 240 241 242 243 244 | sqlite3_free(pCsr->zSchema); if( idxNum==1 && sqlite3_value_type(argv[0])!=SQLITE_NULL ){ pCsr->zSchema = sqlite3_mprintf("%s", sqlite3_value_text(argv[0])); }else{ pCsr->zSchema = sqlite3_mprintf("main"); } zSql = sqlite3_mprintf( | | | | 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 | sqlite3_free(pCsr->zSchema); if( idxNum==1 && sqlite3_value_type(argv[0])!=SQLITE_NULL ){ pCsr->zSchema = sqlite3_mprintf("%s", sqlite3_value_text(argv[0])); }else{ pCsr->zSchema = sqlite3_mprintf("main"); } zSql = sqlite3_mprintf( "SELECT 0, 'table','sqlite_schema','sqlite_schema',1,NULL " "UNION ALL " "SELECT rowid, type, name, tbl_name, rootpage, sql" " FROM \"%w\".sqlite_schema WHERE rootpage>=1", pCsr->zSchema); sqlite3_finalize(pCsr->pStmt); pCsr->pStmt = 0; pCsr->hasRowid = -1; rc = sqlite3_prepare_v2(pTab->db, zSql, -1, &pCsr->pStmt, 0); sqlite3_free(zSql); if( rc==SQLITE_OK ){ |
︙ | ︙ |
Changes to ext/misc/carray.c.
︙ | ︙ | |||
20 21 22 23 24 25 26 | ** at the address $ptr. $ptr is a pointer to the array of integers. ** The pointer value must be assigned to $ptr using the ** sqlite3_bind_pointer() interface with a pointer type of "carray". ** For example: ** ** static int aX[] = { 53, 9, 17, 2231, 4, 99 }; ** int i = sqlite3_bind_parameter_index(pStmt, "$ptr"); | | | | 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 | ** at the address $ptr. $ptr is a pointer to the array of integers. ** The pointer value must be assigned to $ptr using the ** sqlite3_bind_pointer() interface with a pointer type of "carray". ** For example: ** ** static int aX[] = { 53, 9, 17, 2231, 4, 99 }; ** int i = sqlite3_bind_parameter_index(pStmt, "$ptr"); ** sqlite3_bind_pointer(pStmt, i, aX, "carray", 0); ** ** There is an optional third parameter to determine the datatype of ** the C-language array. Allowed values of the third parameter are ** 'int32', 'int64', 'double', 'char*', 'struct iovec'. Example: ** ** SELECT * FROM carray($ptr,10,'char*'); ** ** The default value of the third parameter is 'int32'. ** ** HOW IT WORKS ** |
︙ | ︙ | |||
52 53 54 55 56 57 58 59 60 61 62 | ** as the number of elements in the array. The virtual table steps through ** the array, element by element. */ #include "sqlite3ext.h" SQLITE_EXTENSION_INIT1 #include <assert.h> #include <string.h> #ifndef SQLITE_OMIT_VIRTUALTABLE /* | > > > > > > > > > > > > > > > > > > > > > > > > > > > | | | < < < > > > | > > > > | 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 | ** as the number of elements in the array. The virtual table steps through ** the array, element by element. */ #include "sqlite3ext.h" SQLITE_EXTENSION_INIT1 #include <assert.h> #include <string.h> #ifdef _WIN32 struct iovec { void *iov_base; size_t iov_len; }; #else # include <sys/uio.h> #endif /* Allowed values for the mFlags parameter to sqlite3_carray_bind(). ** Must exactly match the definitions in carray.h. */ #ifndef CARRAY_INT32 # define CARRAY_INT32 0 /* Data is 32-bit signed integers */ # define CARRAY_INT64 1 /* Data is 64-bit signed integers */ # define CARRAY_DOUBLE 2 /* Data is doubles */ # define CARRAY_TEXT 3 /* Data is char* */ # define CARRAY_BLOB 4 /* Data is struct iovec* */ #endif #ifndef SQLITE_API # ifdef _WIN32 # define SQLITE_API __declspec(dllexport) # else # define SQLITE_API # endif #endif #ifndef SQLITE_OMIT_VIRTUALTABLE /* ** Names of allowed datatypes */ static const char *azType[] = { "int32", "int64", "double", "char*", "struct iovec" }; /* ** Structure used to hold the sqlite3_carray_bind() information */ typedef struct carray_bind carray_bind; struct carray_bind { void *aData; /* The data */ int nData; /* Number of elements */ int mFlags; /* Control flags */ void (*xDel)(void*); /* Destructor for aData */ }; /* carray_cursor is a subclass of sqlite3_vtab_cursor which will ** serve as the underlying representation of a cursor that scans ** over rows of the result */ typedef struct carray_cursor carray_cursor; |
︙ | ︙ | |||
199 200 201 202 203 204 205 206 207 208 209 210 211 212 | return SQLITE_OK; } case CARRAY_TEXT: { const char **p = (const char**)pCur->pPtr; sqlite3_result_text(ctx, p[pCur->iRowid-1], -1, SQLITE_TRANSIENT); return SQLITE_OK; } } } } sqlite3_result_int64(ctx, x); return SQLITE_OK; } | > > > > > > | 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 | return SQLITE_OK; } case CARRAY_TEXT: { const char **p = (const char**)pCur->pPtr; sqlite3_result_text(ctx, p[pCur->iRowid-1], -1, SQLITE_TRANSIENT); return SQLITE_OK; } case CARRAY_BLOB: { const struct iovec *p = (struct iovec*)pCur->pPtr; sqlite3_result_blob(ctx, p[pCur->iRowid-1].iov_base, (int)p[pCur->iRowid-1].iov_len, SQLITE_TRANSIENT); return SQLITE_OK; } } } } sqlite3_result_int64(ctx, x); return SQLITE_OK; } |
︙ | ︙ | |||
235 236 237 238 239 240 241 | */ static int carrayFilter( sqlite3_vtab_cursor *pVtabCursor, int idxNum, const char *idxStr, int argc, sqlite3_value **argv ){ carray_cursor *pCur = (carray_cursor *)pVtabCursor; | > > | > > > > > > > > > > | | | | | | | | | | | | | | | | | | | < < > > > > > > | > | > | | 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340 341 342 | */ static int carrayFilter( sqlite3_vtab_cursor *pVtabCursor, int idxNum, const char *idxStr, int argc, sqlite3_value **argv ){ carray_cursor *pCur = (carray_cursor *)pVtabCursor; pCur->pPtr = 0; pCur->iCnt = 0; switch( idxNum ){ case 1: { carray_bind *pBind = sqlite3_value_pointer(argv[0], "carray-bind"); if( pBind==0 ) break; pCur->pPtr = pBind->aData; pCur->iCnt = pBind->nData; pCur->eType = pBind->mFlags & 0x07; break; } case 2: case 3: { pCur->pPtr = sqlite3_value_pointer(argv[0], "carray"); pCur->iCnt = pCur->pPtr ? sqlite3_value_int64(argv[1]) : 0; if( idxNum<3 ){ pCur->eType = CARRAY_INT32; }else{ unsigned char i; const char *zType = (const char*)sqlite3_value_text(argv[2]); for(i=0; i<sizeof(azType)/sizeof(azType[0]); i++){ if( sqlite3_stricmp(zType, azType[i])==0 ) break; } if( i>=sizeof(azType)/sizeof(azType[0]) ){ pVtabCursor->pVtab->zErrMsg = sqlite3_mprintf( "unknown datatype: %Q", zType); return SQLITE_ERROR; }else{ pCur->eType = i; } } break; } } pCur->iRowid = 1; return SQLITE_OK; } /* ** SQLite will invoke this method one or more times while planning a query ** that uses the carray virtual table. This routine needs to create ** a query plan for each invocation and compute an estimated cost for that ** plan. ** ** In this implementation idxNum is used to represent the ** query plan. idxStr is unused. ** ** idxNum is: ** ** 1 If only the pointer= constraint exists. In this case, the ** parameter must be bound using sqlite3_carray_bind(). ** ** 2 if the pointer= and count= constraints exist. ** ** 3 if the ctype= constraint also exists. ** ** idxNum is 0 otherwise and carray becomes an empty table. */ static int carrayBestIndex( sqlite3_vtab *tab, sqlite3_index_info *pIdxInfo ){ int i; /* Loop over constraints */ int ptrIdx = -1; /* Index of the pointer= constraint, or -1 if none */ |
︙ | ︙ | |||
301 302 303 304 305 306 307 | cntIdx = i; break; case CARRAY_COLUMN_CTYPE: ctypeIdx = i; break; } } | | < < > > > > | | | | | > | 356 357 358 359 360 361 362 363 364 365 366 367 368 369 370 371 372 373 374 375 376 377 378 379 380 381 382 383 384 | cntIdx = i; break; case CARRAY_COLUMN_CTYPE: ctypeIdx = i; break; } } if( ptrIdx>=0 ){ pIdxInfo->aConstraintUsage[ptrIdx].argvIndex = 1; pIdxInfo->aConstraintUsage[ptrIdx].omit = 1; pIdxInfo->estimatedCost = (double)1; pIdxInfo->estimatedRows = 100; pIdxInfo->idxNum = 1; if( cntIdx>=0 ){ pIdxInfo->aConstraintUsage[cntIdx].argvIndex = 2; pIdxInfo->aConstraintUsage[cntIdx].omit = 1; pIdxInfo->idxNum = 2; if( ctypeIdx>=0 ){ pIdxInfo->aConstraintUsage[ctypeIdx].argvIndex = 3; pIdxInfo->aConstraintUsage[ctypeIdx].omit = 1; pIdxInfo->idxNum = 3; } } }else{ pIdxInfo->estimatedCost = (double)2147483647; pIdxInfo->estimatedRows = 2147483647; pIdxInfo->idxNum = 0; } return SQLITE_OK; |
︙ | ︙ | |||
348 349 350 351 352 353 354 355 356 357 358 359 360 361 | 0, /* xBegin */ 0, /* xSync */ 0, /* xCommit */ 0, /* xRollback */ 0, /* xFindMethod */ 0, /* xRename */ }; /* ** For testing purpose in the TCL test harness, we need a method for ** setting the pointer value. The inttoptr(X) SQL function accomplishes ** this. Tcl script will bind an integer to X and the inttoptr() SQL ** function will use sqlite3_result_pointer() to convert that integer into ** a pointer. | > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | 406 407 408 409 410 411 412 413 414 415 416 417 418 419 420 421 422 423 424 425 426 427 428 429 430 431 432 433 434 435 436 437 438 439 440 441 442 443 444 445 446 447 448 449 450 451 452 453 454 455 456 457 458 459 460 461 462 463 464 465 466 467 468 469 470 471 472 473 474 475 476 477 478 479 480 481 482 483 484 485 486 487 488 489 490 491 492 493 494 495 496 497 498 499 500 501 502 503 504 505 506 507 508 509 510 511 512 513 514 | 0, /* xBegin */ 0, /* xSync */ 0, /* xCommit */ 0, /* xRollback */ 0, /* xFindMethod */ 0, /* xRename */ }; /* ** Destructor for the carray_bind object */ static void carrayBindDel(void *pPtr){ carray_bind *p = (carray_bind*)pPtr; if( p->xDel!=SQLITE_STATIC ){ p->xDel(p->aData); } sqlite3_free(p); } /* ** Invoke this interface in order to bind to the single-argument ** version of CARRAY(). */ SQLITE_API int sqlite3_carray_bind( sqlite3_stmt *pStmt, int idx, void *aData, int nData, int mFlags, void (*xDestroy)(void*) ){ carray_bind *pNew; int i; pNew = sqlite3_malloc64(sizeof(*pNew)); if( pNew==0 ){ if( xDestroy!=SQLITE_STATIC && xDestroy!=SQLITE_TRANSIENT ){ xDestroy(aData); } return SQLITE_NOMEM; } pNew->nData = nData; pNew->mFlags = mFlags; if( xDestroy==SQLITE_TRANSIENT ){ sqlite3_int64 sz = nData; switch( mFlags & 0x07 ){ case CARRAY_INT32: sz *= 4; break; case CARRAY_INT64: sz *= 8; break; case CARRAY_DOUBLE: sz *= 8; break; case CARRAY_TEXT: sz *= sizeof(char*); break; case CARRAY_BLOB: sz *= sizeof(struct iovec); break; } if( (mFlags & 0x07)==CARRAY_TEXT ){ for(i=0; i<nData; i++){ const char *z = ((char**)aData)[i]; if( z ) sz += strlen(z) + 1; } }else if( (mFlags & 0x07)==CARRAY_BLOB ){ for(i=0; i<nData; i++){ sz += ((struct iovec*)aData)[i].iov_len; } } pNew->aData = sqlite3_malloc64( sz ); if( pNew->aData==0 ){ sqlite3_free(pNew); return SQLITE_NOMEM; } if( (mFlags & 0x07)==CARRAY_TEXT ){ char **az = (char**)pNew->aData; char *z = (char*)&az[nData]; for(i=0; i<nData; i++){ const char *zData = ((char**)aData)[i]; sqlite3_int64 n; if( zData==0 ){ az[i] = 0; continue; } az[i] = z; n = strlen(zData); memcpy(z, zData, n+1); z += n+1; } }else if( (mFlags & 0x07)==CARRAY_BLOB ){ struct iovec *p = (struct iovec*)pNew->aData; unsigned char *z = (unsigned char*)&p[nData]; for(i=0; i<nData; i++){ size_t n = ((struct iovec*)aData)[i].iov_len; p[i].iov_len = n; p[i].iov_base = z; z += n; memcpy(p[i].iov_base, ((struct iovec*)aData)[i].iov_base, n); } }else{ memcpy(pNew->aData, aData, sz); } pNew->xDel = sqlite3_free; }else{ pNew->aData = aData; pNew->xDel = xDestroy; } return sqlite3_bind_pointer(pStmt, idx, pNew, "carray-bind", carrayBindDel); } /* ** For testing purpose in the TCL test harness, we need a method for ** setting the pointer value. The inttoptr(X) SQL function accomplishes ** this. Tcl script will bind an integer to X and the inttoptr() SQL ** function will use sqlite3_result_pointer() to convert that integer into ** a pointer. |
︙ | ︙ | |||
379 380 381 382 383 384 385 | } sqlite3_result_pointer(context, p, "carray", 0); } #endif /* SQLITE_TEST */ #endif /* SQLITE_OMIT_VIRTUALTABLE */ | < < < | | 532 533 534 535 536 537 538 539 540 541 542 543 544 545 546 | } sqlite3_result_pointer(context, p, "carray", 0); } #endif /* SQLITE_TEST */ #endif /* SQLITE_OMIT_VIRTUALTABLE */ SQLITE_API int sqlite3_carray_init( sqlite3 *db, char **pzErrMsg, const sqlite3_api_routines *pApi ){ int rc = SQLITE_OK; SQLITE_EXTENSION_INIT2(pApi); #ifndef SQLITE_OMIT_VIRTUALTABLE |
︙ | ︙ |
Added ext/misc/carray.h.
> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 | /* ** 2020-11-17 ** ** The author disclaims copyright to this source code. In place of ** a legal notice, here is a blessing: ** ** May you do good and not evil. ** May you find forgiveness for yourself and forgive others. ** May you share freely, never taking more than you give. ** ************************************************************************* ** ** Interface definitions for the CARRAY table-valued function ** extension. */ #ifndef _CARRAY_H #define _CARRAY_H #include "sqlite3.h" /* Required for error code definitions */ #ifdef __cplusplus extern "C" { #endif /* Use this interface to bind an array to the single-argument version ** of CARRAY(). */ SQLITE_API int sqlite3_carray_bind( sqlite3_stmt *pStmt, /* Statement to be bound */ int i, /* Parameter index */ void *aData, /* Pointer to array data */ int nData, /* Number of data elements */ int mFlags, /* CARRAY flags */ void (*xDel)(void*) /* Destructgor for aData*/ ); /* Allowed values for the mFlags parameter to sqlite3_carray_bind(). */ #define CARRAY_INT32 0 /* Data is 32-bit signed integers */ #define CARRAY_INT64 1 /* Data is 64-bit signed integers */ #define CARRAY_DOUBLE 2 /* Data is doubles */ #define CARRAY_TEXT 3 /* Data is char* */ #define CARRAY_BLOB 4 /* Data is struct iovec */ #ifdef __cplusplus } /* end of the 'extern "C"' block */ #endif #endif /* ifndef _CARRAY_H */ |
Added ext/misc/cksumvfs.c.
> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340 341 342 343 344 345 346 347 348 349 350 351 352 353 354 355 356 357 358 359 360 361 362 363 364 365 366 367 368 369 370 371 372 373 374 375 376 377 378 379 380 381 382 383 384 385 386 387 388 389 390 391 392 393 394 395 396 397 398 399 400 401 402 403 404 405 406 407 408 409 410 411 412 413 414 415 416 417 418 419 420 421 422 423 424 425 426 427 428 429 430 431 432 433 434 435 436 437 438 439 440 441 442 443 444 445 446 447 448 449 450 451 452 453 454 455 456 457 458 459 460 461 462 463 464 465 466 467 468 469 470 471 472 473 474 475 476 477 478 479 480 481 482 483 484 485 486 487 488 489 490 491 492 493 494 495 496 497 498 499 500 501 502 503 504 505 506 507 508 509 510 511 512 513 514 515 516 517 518 519 520 521 522 523 524 525 526 527 528 529 530 531 532 533 534 535 536 537 538 539 540 541 542 543 544 545 546 547 548 549 550 551 552 553 554 555 556 557 558 559 560 561 562 563 564 565 566 567 568 569 570 571 572 573 574 575 576 577 578 579 580 581 582 583 584 585 586 587 588 589 590 591 592 593 594 595 596 597 598 599 600 601 602 603 604 605 606 607 608 609 610 611 612 613 614 615 616 617 618 619 620 621 622 623 624 625 626 627 628 629 630 631 632 633 634 635 636 637 638 639 640 641 642 643 644 645 646 647 648 649 650 651 652 653 654 655 656 657 658 659 660 661 662 663 664 665 666 667 668 669 670 671 672 673 674 675 676 677 678 679 680 681 682 683 684 685 686 687 688 689 690 691 692 693 694 695 696 697 698 699 700 701 702 703 704 705 706 707 708 709 710 711 712 713 714 715 716 717 718 719 720 721 722 723 724 725 726 727 728 729 730 731 732 733 734 735 736 737 738 739 740 741 742 743 744 745 746 747 748 749 750 751 752 753 754 755 756 757 758 759 760 761 762 763 764 765 766 767 768 769 770 771 772 773 774 775 776 777 778 779 780 781 782 783 784 785 786 787 788 789 790 791 792 793 794 795 796 797 798 799 800 801 802 803 804 805 806 807 808 809 810 811 812 813 814 815 816 817 818 819 820 821 822 823 824 825 826 827 828 829 830 831 832 833 834 835 836 837 838 839 840 841 842 843 844 845 846 847 848 849 850 851 852 853 854 855 856 857 858 859 860 861 862 863 864 865 866 867 868 869 870 871 872 873 874 875 876 877 878 879 880 | /* ** 2020-04-20 ** ** The author disclaims copyright to this source code. In place of ** a legal notice, here is a blessing: ** ** May you do good and not evil. ** May you find forgiveness for yourself and forgive others. ** May you share freely, never taking more than you give. ** ****************************************************************************** ** ** This file implements a VFS shim that writes a checksum on each page ** of an SQLite database file. When reading pages, the checksum is verified ** and an error is raised if the checksum is incorrect. ** ** COMPILING ** ** This extension requires SQLite 3.32.0 or later. It uses the ** sqlite3_database_file_object() interface which was added in ** version 3.32.0, so it will not link with an earlier version of ** SQLite. ** ** To build this extension as a separately loaded shared library or ** DLL, use compiler command-lines similar to the following: ** ** (linux) gcc -fPIC -shared cksumvfs.c -o cksumvfs.so ** (mac) clang -fPIC -dynamiclib cksumvfs.c -o cksumvfs.dylib ** (windows) cl cksumvfs.c -link -dll -out:cksumvfs.dll ** ** You may want to add additional compiler options, of course, ** according to the needs of your project. ** ** If you want to statically link this extension with your product, ** then compile it like any other C-language module but add the ** "-DSQLITE_CKSUMVFS_STATIC" option so that this module knows that ** it is being statically linked rather than dynamically linked ** ** LOADING ** ** To load this extension as a shared library, you first have to ** bring up a dummy SQLite database connection to use as the argument ** to the sqlite3_load_extension() API call. Then you invoke the ** sqlite3_load_extension() API and shutdown the dummy database ** connection. All subsequent database connections that are opened ** will include this extension. For example: ** ** sqlite3 *db; ** sqlite3_open(":memory:", &db); ** sqlite3_load_extension(db, "./cksumvfs"); ** sqlite3_close(db); ** ** If this extension is compiled with -DSQLITE_CKSUMVFS_STATIC and ** statically linked against the application, initialize it using ** a single API call as follows: ** ** sqlite3_register_cksumvfs(); ** ** Cksumvfs is a VFS Shim. When loaded, "cksmvfs" becomes the new ** default VFS and it uses the prior default VFS as the next VFS ** down in the stack. This is normally what you want. However, in ** complex situations where multiple VFS shims are being loaded, ** it might be important to ensure that cksumvfs is loaded in the ** correct order so that it sequences itself into the default VFS ** Shim stack in the right order. ** ** USING ** ** Open database connections using the sqlite3_open() or ** sqlite3_open_v2() interfaces, as normal. Ordinary database files ** (without a checksum) will operate normally. Databases with ** checksums will return an SQLITE_IOERR_DATA error if a page is ** encountered that contains an invalid checksum. ** ** Checksumming only works on databases that have a reserve-bytes ** value of exactly 8. The default value for reserve-bytes is 0. ** Hence, newly created database files will omit the checksum by ** default. To create a database that includes a checksum, change ** the reserve-bytes value to 8 by runing: ** ** int n = 8; ** sqlite3_file_control(db, 0, SQLITE_FCNTL_RESERVE_BYTES, &n); ** ** If you do this immediately after creating a new database file, ** before anything else has been written into the file, then that ** might be all that you need to do. Otherwise, the API call ** above should be followed by: ** ** sqlite3_exec(db, "VACUUM", 0, 0, 0); ** ** It never hurts to run the VACUUM, even if you don't need it. ** If the database is in WAL mode, you should shutdown and ** reopen all database connections before continuing. ** ** From the CLI, use the ".filectrl reserve_bytes 8" command, ** followed by "VACUUM;". ** ** Note that SQLite allows the number of reserve-bytes to be ** increased but not decreased. So if a database file already ** has a reserve-bytes value greater than 8, there is no way to ** activate checksumming on that database, other than to dump ** and restore the database file. Note also that other extensions ** might also make use of the reserve-bytes. Checksumming will ** be incompatible with those other extensions. ** ** VERIFICATION OF CHECKSUMS ** ** If any checksum is incorrect, the "PRAGMA quick_check" command ** will find it. To verify that checksums are actually enabled ** and running, use the following query: ** ** SELECT count(*), verify_checksum(data) ** FROM sqlite_dbpage ** GROUP BY 2; ** ** There are three possible outputs form the verify_checksum() ** function: 1, 0, and NULL. 1 is returned if the checksum is ** correct. 0 is returned if the checksum is incorrect. NULL ** is returned if the page is unreadable. If checksumming is ** enabled, the read will fail if the checksum is wrong, so the ** usual result from verify_checksum() on a bad checksum is NULL. ** ** If everything is OK, the query above should return a single ** row where the second column is 1. Any other result indicates ** either that there is a checksum error, or checksum validation ** is disabled. ** ** CONTROLLING CHECKSUM VERIFICATION ** ** The cksumvfs extension implements a new PRAGMA statement that can ** be used to disable, re-enable, or query the status of checksum ** verification: ** ** PRAGMA checksum_verification; -- query status ** PRAGMA checksum_verification=OFF; -- disable verification ** PRAGMA checksum_verification=ON; -- re-enable verification ** ** The "checksum_verification" pragma will return "1" (true) or "0" ** (false) if checksum verification is enabled or disabled, respectively. ** "Verification" in this context means the feature that causes ** SQLITE_IOERR_DATA errors if a checksum mismatch is detected while ** reading. Checksums are always kept up-to-date as long as the ** reserve-bytes value of the database is 8, regardless of the setting ** of this pragma. Checksum verification can be disabled (for example) ** to do forensic analysis of a database that has previously reported ** a checksum error. ** ** The "checksum_verification" pragma will always respond with "0" if ** the database file does not have a reserve-bytes value of 8. The ** pragma will return no rows at all if the cksumvfs extension is ** not loaded. ** ** IMPLEMENTATION NOTES ** ** The checksum is stored in the last 8 bytes of each page. This ** module only operates if the "bytes of reserved space on each page" ** value at offset 20 the SQLite database header is exactly 8. If ** the reserved-space value is not 8, this module is a no-op. */ #if defined(SQLITE_AMALGAMATION) && !defined(SQLITE_CKSUMVFS_STATIC) # define SQLITE_CKSUMVFS_STATIC #endif #ifdef SQLITE_CKSUMVFS_STATIC # include "sqlite3.h" #else # include "sqlite3ext.h" SQLITE_EXTENSION_INIT1 #endif #include <string.h> #include <assert.h> /* ** Forward declaration of objects used by this utility */ typedef struct sqlite3_vfs CksmVfs; typedef struct CksmFile CksmFile; /* ** Useful datatype abbreviations */ #if !defined(SQLITE_AMALGAMATION) typedef unsigned char u8; typedef unsigned int u32; #endif /* Access to a lower-level VFS that (might) implement dynamic loading, ** access to randomness, etc. */ #define ORIGVFS(p) ((sqlite3_vfs*)((p)->pAppData)) #define ORIGFILE(p) ((sqlite3_file*)(((CksmFile*)(p))+1)) /* An open file */ struct CksmFile { sqlite3_file base; /* IO methods */ const char *zFName; /* Original name of the file */ char computeCksm; /* True to compute checksums. ** Always true if reserve size is 8. */ char verifyCksm; /* True to verify checksums */ char isWal; /* True if processing a WAL file */ char inCkpt; /* Currently doing a checkpoint */ CksmFile *pPartner; /* Ptr from WAL to main-db, or from main-db to WAL */ }; /* ** Methods for CksmFile */ static int cksmClose(sqlite3_file*); static int cksmRead(sqlite3_file*, void*, int iAmt, sqlite3_int64 iOfst); static int cksmWrite(sqlite3_file*,const void*,int iAmt, sqlite3_int64 iOfst); static int cksmTruncate(sqlite3_file*, sqlite3_int64 size); static int cksmSync(sqlite3_file*, int flags); static int cksmFileSize(sqlite3_file*, sqlite3_int64 *pSize); static int cksmLock(sqlite3_file*, int); static int cksmUnlock(sqlite3_file*, int); static int cksmCheckReservedLock(sqlite3_file*, int *pResOut); static int cksmFileControl(sqlite3_file*, int op, void *pArg); static int cksmSectorSize(sqlite3_file*); static int cksmDeviceCharacteristics(sqlite3_file*); static int cksmShmMap(sqlite3_file*, int iPg, int pgsz, int, void volatile**); static int cksmShmLock(sqlite3_file*, int offset, int n, int flags); static void cksmShmBarrier(sqlite3_file*); static int cksmShmUnmap(sqlite3_file*, int deleteFlag); static int cksmFetch(sqlite3_file*, sqlite3_int64 iOfst, int iAmt, void **pp); static int cksmUnfetch(sqlite3_file*, sqlite3_int64 iOfst, void *p); /* ** Methods for CksmVfs */ static int cksmOpen(sqlite3_vfs*, const char *, sqlite3_file*, int , int *); static int cksmDelete(sqlite3_vfs*, const char *zName, int syncDir); static int cksmAccess(sqlite3_vfs*, const char *zName, int flags, int *); static int cksmFullPathname(sqlite3_vfs*, const char *zName, int, char *zOut); static void *cksmDlOpen(sqlite3_vfs*, const char *zFilename); static void cksmDlError(sqlite3_vfs*, int nByte, char *zErrMsg); static void (*cksmDlSym(sqlite3_vfs *pVfs, void *p, const char*zSym))(void); static void cksmDlClose(sqlite3_vfs*, void*); static int cksmRandomness(sqlite3_vfs*, int nByte, char *zOut); static int cksmSleep(sqlite3_vfs*, int microseconds); static int cksmCurrentTime(sqlite3_vfs*, double*); static int cksmGetLastError(sqlite3_vfs*, int, char *); static int cksmCurrentTimeInt64(sqlite3_vfs*, sqlite3_int64*); static int cksmSetSystemCall(sqlite3_vfs*, const char*,sqlite3_syscall_ptr); static sqlite3_syscall_ptr cksmGetSystemCall(sqlite3_vfs*, const char *z); static const char *cksmNextSystemCall(sqlite3_vfs*, const char *zName); static sqlite3_vfs cksm_vfs = { 3, /* iVersion (set when registered) */ 0, /* szOsFile (set when registered) */ 1024, /* mxPathname */ 0, /* pNext */ "cksmvfs", /* zName */ 0, /* pAppData (set when registered) */ cksmOpen, /* xOpen */ cksmDelete, /* xDelete */ cksmAccess, /* xAccess */ cksmFullPathname, /* xFullPathname */ cksmDlOpen, /* xDlOpen */ cksmDlError, /* xDlError */ cksmDlSym, /* xDlSym */ cksmDlClose, /* xDlClose */ cksmRandomness, /* xRandomness */ cksmSleep, /* xSleep */ cksmCurrentTime, /* xCurrentTime */ cksmGetLastError, /* xGetLastError */ cksmCurrentTimeInt64, /* xCurrentTimeInt64 */ cksmSetSystemCall, /* xSetSystemCall */ cksmGetSystemCall, /* xGetSystemCall */ cksmNextSystemCall /* xNextSystemCall */ }; static const sqlite3_io_methods cksm_io_methods = { 3, /* iVersion */ cksmClose, /* xClose */ cksmRead, /* xRead */ cksmWrite, /* xWrite */ cksmTruncate, /* xTruncate */ cksmSync, /* xSync */ cksmFileSize, /* xFileSize */ cksmLock, /* xLock */ cksmUnlock, /* xUnlock */ cksmCheckReservedLock, /* xCheckReservedLock */ cksmFileControl, /* xFileControl */ cksmSectorSize, /* xSectorSize */ cksmDeviceCharacteristics, /* xDeviceCharacteristics */ cksmShmMap, /* xShmMap */ cksmShmLock, /* xShmLock */ cksmShmBarrier, /* xShmBarrier */ cksmShmUnmap, /* xShmUnmap */ cksmFetch, /* xFetch */ cksmUnfetch /* xUnfetch */ }; /* Do byte swapping on a unsigned 32-bit integer */ #define BYTESWAP32(x) ( \ (((x)&0x000000FF)<<24) + (((x)&0x0000FF00)<<8) \ + (((x)&0x00FF0000)>>8) + (((x)&0xFF000000)>>24) \ ) /* Compute a checksum on a buffer */ static void cksmCompute( u8 *a, /* Content to be checksummed */ int nByte, /* Bytes of content in a[]. Must be a multiple of 8. */ u8 *aOut /* OUT: Final 8-byte checksum value output */ ){ u32 s1 = 0, s2 = 0; u32 *aData = (u32*)a; u32 *aEnd = (u32*)&a[nByte]; u32 x = 1; assert( nByte>=8 ); assert( (nByte&0x00000007)==0 ); assert( nByte<=65536 ); if( 1 == *(u8*)&x ){ /* Little-endian */ do { s1 += *aData++ + s2; s2 += *aData++ + s1; }while( aData<aEnd ); }else{ /* Big-endian */ do { s1 += BYTESWAP32(aData[0]) + s2; s2 += BYTESWAP32(aData[1]) + s1; aData += 2; }while( aData<aEnd ); s1 = BYTESWAP32(s1); s2 = BYTESWAP32(s2); } memcpy(aOut, &s1, 4); memcpy(aOut+4, &s2, 4); } /* ** SQL function: verify_checksum(BLOB) ** ** Return 0 or 1 if the checksum is invalid or valid. Or return ** NULL if the input is not a BLOB that is the right size for a ** database page. */ static void cksmVerifyFunc( sqlite3_context *context, int argc, sqlite3_value **argv ){ int nByte; u8 *data; u8 cksum[8]; data = (u8*)sqlite3_value_blob(argv[0]); if( data==0 ) return; if( sqlite3_value_type(argv[0])!=SQLITE_BLOB ) return; nByte = sqlite3_value_bytes(argv[0]); if( nByte<512 || nByte>65536 || (nByte & (nByte-1))!=0 ) return; cksmCompute(data, nByte-8, cksum); sqlite3_result_int(context, memcmp(data+nByte-8,cksum,8)==0); } #ifdef SQLITE_CKSUMVFS_INIT_FUNCNAME /* ** SQL function: initialize_cksumvfs(SCHEMANAME) ** ** This SQL functions (whose name is actually determined at compile-time ** by the value of the SQLITE_CKSUMVFS_INIT_FUNCNAME macro) invokes: ** ** sqlite3_file_control(db, SCHEMANAME, SQLITE_FCNTL_RESERVE_BYTE, &n); ** ** In order to set the reserve bytes value to 8, so that cksumvfs will ** operation. This feature is provided (if and only if the ** SQLITE_CKSUMVFS_INIT_FUNCNAME compile-time option is set to a string ** which is the name of the SQL function) so as to provide the ability ** to invoke the file-control in programming languages that lack ** direct access to the sqlite3_file_control() interface (ex: Java). ** ** This interface is undocumented, apart from this comment. Usage ** example: ** ** 1. Compile with -DSQLITE_CKSUMVFS_INIT_FUNCNAME="ckvfs_init" ** 2. Run: "SELECT cksum_init('main'); VACUUM;" */ static void cksmInitFunc( sqlite3_context *context, int argc, sqlite3_value **argv ){ int nByte = 8; const char *zSchemaName = (const char*)sqlite3_value_text(argv[0]); sqlite3 *db = sqlite3_context_db_handle(context); sqlite3_file_control(db, zSchemaName, SQLITE_FCNTL_RESERVE_BYTES, &nByte); /* Return NULL */ } #endif /* SQLITE_CKSUMBFS_INIT_FUNCNAME */ /* ** Close a cksm-file. */ static int cksmClose(sqlite3_file *pFile){ CksmFile *p = (CksmFile *)pFile; if( p->pPartner ){ assert( p->pPartner->pPartner==p ); p->pPartner->pPartner = 0; p->pPartner = 0; } pFile = ORIGFILE(pFile); return pFile->pMethods->xClose(pFile); } /* ** Set the computeCkSm and verifyCksm flags, if they need to be ** changed. */ static void cksmSetFlags(CksmFile *p, int hasCorrectReserveSize){ if( hasCorrectReserveSize!=p->computeCksm ){ p->computeCksm = p->verifyCksm = hasCorrectReserveSize; if( p->pPartner ){ p->pPartner->verifyCksm = hasCorrectReserveSize; p->pPartner->computeCksm = hasCorrectReserveSize; } } } /* ** Read data from a cksm-file. */ static int cksmRead( sqlite3_file *pFile, void *zBuf, int iAmt, sqlite_int64 iOfst ){ int rc; CksmFile *p = (CksmFile *)pFile; pFile = ORIGFILE(pFile); rc = pFile->pMethods->xRead(pFile, zBuf, iAmt, iOfst); if( rc==SQLITE_OK ){ if( iOfst==0 && iAmt>=100 && ( memcmp(zBuf,"SQLite format 3",16)==0 || memcmp(zBuf,"ZV-",3)==0 )){ u8 *d = (u8*)zBuf; char hasCorrectReserveSize = (d[20]==8); cksmSetFlags(p, hasCorrectReserveSize); } /* Verify the checksum if ** (1) the size indicates that we are dealing with a complete ** database page ** (2) checksum verification is enabled ** (3) we are not in the middle of checkpoint */ if( iAmt>=512 /* (1) */ && p->verifyCksm /* (2) */ && !p->inCkpt /* (3) */ ){ u8 cksum[8]; cksmCompute((u8*)zBuf, iAmt-8, cksum); if( memcmp((u8*)zBuf+iAmt-8, cksum, 8)!=0 ){ sqlite3_log(SQLITE_IOERR_DATA, "checksum fault offset %lld of \"%s\"", iOfst, p->zFName); rc = SQLITE_IOERR_DATA; } } } return rc; } /* ** Write data to a cksm-file. */ static int cksmWrite( sqlite3_file *pFile, const void *zBuf, int iAmt, sqlite_int64 iOfst ){ CksmFile *p = (CksmFile *)pFile; pFile = ORIGFILE(pFile); if( iOfst==0 && iAmt>=100 && ( memcmp(zBuf,"SQLite format 3",16)==0 || memcmp(zBuf,"ZV-",3)==0 )){ u8 *d = (u8*)zBuf; char hasCorrectReserveSize = (d[20]==8); cksmSetFlags(p, hasCorrectReserveSize); } /* If the write size is appropriate for a database page and if ** checksums where ever enabled, then it will be safe to compute ** the checksums. The reserve byte size might have increased, but ** it will never decrease. And because it cannot decrease, the ** checksum will not overwrite anything. */ if( iAmt>=512 && p->computeCksm && !p->inCkpt ){ cksmCompute((u8*)zBuf, iAmt-8, ((u8*)zBuf)+iAmt-8); } return pFile->pMethods->xWrite(pFile, zBuf, iAmt, iOfst); } /* ** Truncate a cksm-file. */ static int cksmTruncate(sqlite3_file *pFile, sqlite_int64 size){ pFile = ORIGFILE(pFile); return pFile->pMethods->xTruncate(pFile, size); } /* ** Sync a cksm-file. */ static int cksmSync(sqlite3_file *pFile, int flags){ pFile = ORIGFILE(pFile); return pFile->pMethods->xSync(pFile, flags); } /* ** Return the current file-size of a cksm-file. */ static int cksmFileSize(sqlite3_file *pFile, sqlite_int64 *pSize){ CksmFile *p = (CksmFile *)pFile; pFile = ORIGFILE(p); return pFile->pMethods->xFileSize(pFile, pSize); } /* ** Lock a cksm-file. */ static int cksmLock(sqlite3_file *pFile, int eLock){ pFile = ORIGFILE(pFile); return pFile->pMethods->xLock(pFile, eLock); } /* ** Unlock a cksm-file. */ static int cksmUnlock(sqlite3_file *pFile, int eLock){ pFile = ORIGFILE(pFile); return pFile->pMethods->xUnlock(pFile, eLock); } /* ** Check if another file-handle holds a RESERVED lock on a cksm-file. */ static int cksmCheckReservedLock(sqlite3_file *pFile, int *pResOut){ pFile = ORIGFILE(pFile); return pFile->pMethods->xCheckReservedLock(pFile, pResOut); } /* ** File control method. For custom operations on a cksm-file. */ static int cksmFileControl(sqlite3_file *pFile, int op, void *pArg){ int rc; CksmFile *p = (CksmFile*)pFile; pFile = ORIGFILE(pFile); if( op==SQLITE_FCNTL_PRAGMA ){ char **azArg = (char**)pArg; assert( azArg[1]!=0 ); if( sqlite3_stricmp(azArg[1],"checksum_verification")==0 ){ char *zArg = azArg[2]; if( zArg!=0 ){ if( (zArg[0]>='1' && zArg[0]<='9') || sqlite3_strlike("enable%",zArg,0)==0 || sqlite3_stricmp("yes",zArg)==0 || sqlite3_stricmp("on",zArg)==0 ){ p->verifyCksm = p->computeCksm; }else{ p->verifyCksm = 0; } if( p->pPartner ) p->pPartner->verifyCksm = p->verifyCksm; } azArg[0] = sqlite3_mprintf("%d",p->verifyCksm); return SQLITE_OK; }else if( p->computeCksm && azArg[2]!=0 && sqlite3_stricmp(azArg[1], "page_size")==0 ){ /* Do not allow page size changes on a checksum database */ return SQLITE_OK; } }else if( op==SQLITE_FCNTL_CKPT_START || op==SQLITE_FCNTL_CKPT_DONE ){ p->inCkpt = op==SQLITE_FCNTL_CKPT_START; if( p->pPartner ) p->pPartner->inCkpt = p->inCkpt; }else if( op==SQLITE_FCNTL_CKSM_FILE ){ /* This VFS needs to obtain a pointer to the corresponding database ** file handle from within xOpen() calls to open wal files. To do this, ** it uses the sqlite3_database_file_object() API to obtain a pointer ** to the file-handle used by SQLite to access the db file. This is ** fine if cksmvfs happens to be the top-level VFS, but not if there ** are one or more wrapper VFS. To handle this case, this file-control ** is used to extract the cksmvfs file-handle from any wrapper file ** handle. */ sqlite3_file **ppFile = (sqlite3_file**)pArg; *ppFile = (sqlite3_file*)p; return SQLITE_OK; } rc = pFile->pMethods->xFileControl(pFile, op, pArg); if( rc==SQLITE_OK && op==SQLITE_FCNTL_VFSNAME ){ *(char**)pArg = sqlite3_mprintf("cksm/%z", *(char**)pArg); } return rc; } /* ** Return the sector-size in bytes for a cksm-file. */ static int cksmSectorSize(sqlite3_file *pFile){ pFile = ORIGFILE(pFile); return pFile->pMethods->xSectorSize(pFile); } /* ** Return the device characteristic flags supported by a cksm-file. */ static int cksmDeviceCharacteristics(sqlite3_file *pFile){ pFile = ORIGFILE(pFile); return pFile->pMethods->xDeviceCharacteristics(pFile); } /* Create a shared memory file mapping */ static int cksmShmMap( sqlite3_file *pFile, int iPg, int pgsz, int bExtend, void volatile **pp ){ pFile = ORIGFILE(pFile); return pFile->pMethods->xShmMap(pFile,iPg,pgsz,bExtend,pp); } /* Perform locking on a shared-memory segment */ static int cksmShmLock(sqlite3_file *pFile, int offset, int n, int flags){ pFile = ORIGFILE(pFile); return pFile->pMethods->xShmLock(pFile,offset,n,flags); } /* Memory barrier operation on shared memory */ static void cksmShmBarrier(sqlite3_file *pFile){ pFile = ORIGFILE(pFile); pFile->pMethods->xShmBarrier(pFile); } /* Unmap a shared memory segment */ static int cksmShmUnmap(sqlite3_file *pFile, int deleteFlag){ pFile = ORIGFILE(pFile); return pFile->pMethods->xShmUnmap(pFile,deleteFlag); } /* Fetch a page of a memory-mapped file */ static int cksmFetch( sqlite3_file *pFile, sqlite3_int64 iOfst, int iAmt, void **pp ){ CksmFile *p = (CksmFile *)pFile; if( p->computeCksm ){ *pp = 0; return SQLITE_OK; } pFile = ORIGFILE(pFile); if( pFile->pMethods->iVersion>2 && pFile->pMethods->xFetch ){ return pFile->pMethods->xFetch(pFile, iOfst, iAmt, pp); } *pp = 0; return SQLITE_OK; } /* Release a memory-mapped page */ static int cksmUnfetch(sqlite3_file *pFile, sqlite3_int64 iOfst, void *pPage){ pFile = ORIGFILE(pFile); if( pFile->pMethods->iVersion>2 && pFile->pMethods->xUnfetch ){ return pFile->pMethods->xUnfetch(pFile, iOfst, pPage); } return SQLITE_OK; } /* ** Open a cksm file handle. */ static int cksmOpen( sqlite3_vfs *pVfs, const char *zName, sqlite3_file *pFile, int flags, int *pOutFlags ){ CksmFile *p; sqlite3_file *pSubFile; sqlite3_vfs *pSubVfs; int rc; pSubVfs = ORIGVFS(pVfs); if( (flags & (SQLITE_OPEN_MAIN_DB|SQLITE_OPEN_WAL))==0 ){ return pSubVfs->xOpen(pSubVfs, zName, pFile, flags, pOutFlags); } p = (CksmFile*)pFile; memset(p, 0, sizeof(*p)); pSubFile = ORIGFILE(pFile); pFile->pMethods = &cksm_io_methods; rc = pSubVfs->xOpen(pSubVfs, zName, pSubFile, flags, pOutFlags); if( rc ) goto cksm_open_done; if( flags & SQLITE_OPEN_WAL ){ sqlite3_file *pDb = sqlite3_database_file_object(zName); rc = pDb->pMethods->xFileControl(pDb, SQLITE_FCNTL_CKSM_FILE, (void*)&pDb); assert( rc==SQLITE_OK ); p->pPartner = (CksmFile*)pDb; assert( p->pPartner->pPartner==0 ); p->pPartner->pPartner = p; p->isWal = 1; p->computeCksm = p->pPartner->computeCksm; }else{ p->isWal = 0; p->computeCksm = 0; } p->zFName = zName; cksm_open_done: if( rc ) pFile->pMethods = 0; return rc; } /* ** All other VFS methods are pass-thrus. */ static int cksmDelete(sqlite3_vfs *pVfs, const char *zPath, int dirSync){ return ORIGVFS(pVfs)->xDelete(ORIGVFS(pVfs), zPath, dirSync); } static int cksmAccess( sqlite3_vfs *pVfs, const char *zPath, int flags, int *pResOut ){ return ORIGVFS(pVfs)->xAccess(ORIGVFS(pVfs), zPath, flags, pResOut); } static int cksmFullPathname( sqlite3_vfs *pVfs, const char *zPath, int nOut, char *zOut ){ return ORIGVFS(pVfs)->xFullPathname(ORIGVFS(pVfs),zPath,nOut,zOut); } static void *cksmDlOpen(sqlite3_vfs *pVfs, const char *zPath){ return ORIGVFS(pVfs)->xDlOpen(ORIGVFS(pVfs), zPath); } static void cksmDlError(sqlite3_vfs *pVfs, int nByte, char *zErrMsg){ ORIGVFS(pVfs)->xDlError(ORIGVFS(pVfs), nByte, zErrMsg); } static void (*cksmDlSym(sqlite3_vfs *pVfs, void *p, const char *zSym))(void){ return ORIGVFS(pVfs)->xDlSym(ORIGVFS(pVfs), p, zSym); } static void cksmDlClose(sqlite3_vfs *pVfs, void *pHandle){ ORIGVFS(pVfs)->xDlClose(ORIGVFS(pVfs), pHandle); } static int cksmRandomness(sqlite3_vfs *pVfs, int nByte, char *zBufOut){ return ORIGVFS(pVfs)->xRandomness(ORIGVFS(pVfs), nByte, zBufOut); } static int cksmSleep(sqlite3_vfs *pVfs, int nMicro){ return ORIGVFS(pVfs)->xSleep(ORIGVFS(pVfs), nMicro); } static int cksmCurrentTime(sqlite3_vfs *pVfs, double *pTimeOut){ return ORIGVFS(pVfs)->xCurrentTime(ORIGVFS(pVfs), pTimeOut); } static int cksmGetLastError(sqlite3_vfs *pVfs, int a, char *b){ return ORIGVFS(pVfs)->xGetLastError(ORIGVFS(pVfs), a, b); } static int cksmCurrentTimeInt64(sqlite3_vfs *pVfs, sqlite3_int64 *p){ sqlite3_vfs *pOrig = ORIGVFS(pVfs); int rc; assert( pOrig->iVersion>=2 ); if( pOrig->xCurrentTimeInt64 ){ rc = pOrig->xCurrentTimeInt64(pOrig, p); }else{ double r; rc = pOrig->xCurrentTime(pOrig, &r); *p = (sqlite3_int64)(r*86400000.0); } return rc; } static int cksmSetSystemCall( sqlite3_vfs *pVfs, const char *zName, sqlite3_syscall_ptr pCall ){ return ORIGVFS(pVfs)->xSetSystemCall(ORIGVFS(pVfs),zName,pCall); } static sqlite3_syscall_ptr cksmGetSystemCall( sqlite3_vfs *pVfs, const char *zName ){ return ORIGVFS(pVfs)->xGetSystemCall(ORIGVFS(pVfs),zName); } static const char *cksmNextSystemCall(sqlite3_vfs *pVfs, const char *zName){ return ORIGVFS(pVfs)->xNextSystemCall(ORIGVFS(pVfs), zName); } /* Register the verify_checksum() SQL function. */ static int cksmRegisterFunc( sqlite3 *db, char **pzErrMsg, const sqlite3_api_routines *pApi ){ int rc; if( db==0 ) return SQLITE_OK; rc = sqlite3_create_function(db, "verify_checksum", 1, SQLITE_UTF8|SQLITE_INNOCUOUS|SQLITE_DETERMINISTIC, 0, cksmVerifyFunc, 0, 0); #ifdef SQLITE_CKSUMVFS_INIT_FUNCNAME (void)sqlite3_create_function(db, SQLITE_CKSUMVFS_INIT_FUNCNAME, 1, SQLITE_UTF8|SQLITE_DIRECTONLY, 0, cksmInitFunc, 0, 0); #endif return rc; } /* ** Register the cksum VFS as the default VFS for the system. ** Also make arrangements to automatically register the "verify_checksum()" ** SQL function on each new database connection. */ static int cksmRegisterVfs(void){ int rc = SQLITE_OK; sqlite3_vfs *pOrig; if( sqlite3_vfs_find("cksmvfs")!=0 ) return SQLITE_OK; pOrig = sqlite3_vfs_find(0); if( pOrig==0 ) return SQLITE_ERROR; cksm_vfs.iVersion = pOrig->iVersion; cksm_vfs.pAppData = pOrig; cksm_vfs.szOsFile = pOrig->szOsFile + sizeof(CksmFile); rc = sqlite3_vfs_register(&cksm_vfs, 1); if( rc==SQLITE_OK ){ rc = sqlite3_auto_extension((void(*)(void))cksmRegisterFunc); } return rc; } #if defined(SQLITE_CKSUMVFS_STATIC) /* This variant of the initializer runs when the extension is ** statically linked. */ int sqlite3_register_cksumvfs(const char *NotUsed){ (void)NotUsed; return cksmRegisterVfs(); } int sqlite3_unregister_cksumvfs(void){ if( sqlite3_vfs_find("cksmvfs") ){ sqlite3_vfs_unregister(&cksm_vfs); sqlite3_cancel_auto_extension((void(*)(void))cksmRegisterFunc); } return SQLITE_OK; } #endif /* defined(SQLITE_CKSUMVFS_STATIC */ #if !defined(SQLITE_CKSUMVFS_STATIC) /* This variant of the initializer function is used when the ** extension is shared library to be loaded at run-time. */ #ifdef _WIN32 __declspec(dllexport) #endif /* ** This routine is called by sqlite3_load_extension() when the ** extension is first loaded. ***/ int sqlite3_cksumvfs_init( sqlite3 *db, char **pzErrMsg, const sqlite3_api_routines *pApi ){ int rc; SQLITE_EXTENSION_INIT2(pApi); (void)pzErrMsg; /* not used */ rc = cksmRegisterFunc(db, 0, 0); if( rc==SQLITE_OK ){ rc = cksmRegisterVfs(); } if( rc==SQLITE_OK ) rc = SQLITE_OK_LOAD_PERMANENTLY; return rc; } #endif /* !defined(SQLITE_CKSUMVFS_STATIC) */ |
Changes to ext/misc/closure.c.
︙ | ︙ | |||
418 419 420 421 422 423 424 | ** ** "abc" becomes abc ** 'xyz' becomes xyz ** [pqr] becomes pqr ** `mno` becomes mno */ static char *closureDequote(const char *zIn){ | | | | | | 418 419 420 421 422 423 424 425 426 427 428 429 430 431 432 433 434 435 436 437 438 439 440 441 | ** ** "abc" becomes abc ** 'xyz' becomes xyz ** [pqr] becomes pqr ** `mno` becomes mno */ static char *closureDequote(const char *zIn){ sqlite3_int64 nIn; /* Size of input string, in bytes */ char *zOut; /* Output (dequoted) string */ nIn = strlen(zIn); zOut = sqlite3_malloc64(nIn+1); if( zOut ){ char q = zIn[0]; /* Quote character (if any ) */ if( q!='[' && q!= '\'' && q!='"' && q!='`' ){ memcpy(zOut, zIn, (size_t)(nIn+1)); }else{ int iOut = 0; /* Index of next byte to write to output */ int iIn; /* Index of next byte to read from input */ if( q=='[' ) q = ']'; for(iIn=1; iIn<nIn; iIn++){ if( zIn[iIn]==q ) iIn++; |
︙ | ︙ |
Changes to ext/misc/completion.c.
︙ | ︙ | |||
114 115 116 117 118 119 120 121 122 123 124 125 126 127 | /* Column numbers */ #define COMPLETION_COLUMN_CANDIDATE 0 /* Suggested completion of the input */ #define COMPLETION_COLUMN_PREFIX 1 /* Prefix of the word to be completed */ #define COMPLETION_COLUMN_WHOLELINE 2 /* Entire line seen so far */ #define COMPLETION_COLUMN_PHASE 3 /* ePhase - used for debugging only */ rc = sqlite3_declare_vtab(db, "CREATE TABLE x(" " candidate TEXT," " prefix TEXT HIDDEN," " wholeline TEXT HIDDEN," " phase INT HIDDEN" /* Used for debugging only */ ")"); | > | 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 | /* Column numbers */ #define COMPLETION_COLUMN_CANDIDATE 0 /* Suggested completion of the input */ #define COMPLETION_COLUMN_PREFIX 1 /* Prefix of the word to be completed */ #define COMPLETION_COLUMN_WHOLELINE 2 /* Entire line seen so far */ #define COMPLETION_COLUMN_PHASE 3 /* ePhase - used for debugging only */ sqlite3_vtab_config(db, SQLITE_VTAB_INNOCUOUS); rc = sqlite3_declare_vtab(db, "CREATE TABLE x(" " candidate TEXT," " prefix TEXT HIDDEN," " wholeline TEXT HIDDEN," " phase INT HIDDEN" /* Used for debugging only */ ")"); |
︙ | ︙ | |||
221 222 223 224 225 226 227 | char *zSql = 0; const char *zSep = ""; sqlite3_prepare_v2(pCur->db, "PRAGMA database_list", -1, &pS2, 0); while( sqlite3_step(pS2)==SQLITE_ROW ){ const char *zDb = (const char*)sqlite3_column_text(pS2, 1); zSql = sqlite3_mprintf( "%z%s" | | | 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 | char *zSql = 0; const char *zSep = ""; sqlite3_prepare_v2(pCur->db, "PRAGMA database_list", -1, &pS2, 0); while( sqlite3_step(pS2)==SQLITE_ROW ){ const char *zDb = (const char*)sqlite3_column_text(pS2, 1); zSql = sqlite3_mprintf( "%z%s" "SELECT name FROM \"%w\".sqlite_schema", zSql, zSep, zDb ); if( zSql==0 ) return SQLITE_NOMEM; zSep = " UNION "; } sqlite3_finalize(pS2); sqlite3_prepare_v2(pCur->db, zSql, -1, &pCur->pStmt, 0); |
︙ | ︙ | |||
245 246 247 248 249 250 251 | char *zSql = 0; const char *zSep = ""; sqlite3_prepare_v2(pCur->db, "PRAGMA database_list", -1, &pS2, 0); while( sqlite3_step(pS2)==SQLITE_ROW ){ const char *zDb = (const char*)sqlite3_column_text(pS2, 1); zSql = sqlite3_mprintf( "%z%s" | | | 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 | char *zSql = 0; const char *zSep = ""; sqlite3_prepare_v2(pCur->db, "PRAGMA database_list", -1, &pS2, 0); while( sqlite3_step(pS2)==SQLITE_ROW ){ const char *zDb = (const char*)sqlite3_column_text(pS2, 1); zSql = sqlite3_mprintf( "%z%s" "SELECT pti.name FROM \"%w\".sqlite_schema AS sm" " JOIN pragma_table_info(sm.name,%Q) AS pti" " WHERE sm.type='table'", zSql, zSep, zDb, zDb ); if( zSql==0 ) return SQLITE_NOMEM; zSep = " UNION "; } |
︙ | ︙ |
Changes to ext/misc/compress.c.
︙ | ︙ | |||
115 116 117 118 119 120 121 | sqlite3 *db, char **pzErrMsg, const sqlite3_api_routines *pApi ){ int rc = SQLITE_OK; SQLITE_EXTENSION_INIT2(pApi); (void)pzErrMsg; /* Unused parameter */ | | > | | > | | 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 | sqlite3 *db, char **pzErrMsg, const sqlite3_api_routines *pApi ){ int rc = SQLITE_OK; SQLITE_EXTENSION_INIT2(pApi); (void)pzErrMsg; /* Unused parameter */ rc = sqlite3_create_function(db, "compress", 1, SQLITE_UTF8 | SQLITE_INNOCUOUS, 0, compressFunc, 0, 0); if( rc==SQLITE_OK ){ rc = sqlite3_create_function(db, "uncompress", 1, SQLITE_UTF8 | SQLITE_INNOCUOUS | SQLITE_DETERMINISTIC, 0, uncompressFunc, 0, 0); } return rc; } |
Changes to ext/misc/csv.c.
︙ | ︙ | |||
276 277 278 279 280 281 282 283 284 285 286 287 288 289 | } if( c=='\n' ){ p->nLine++; if( p->n>0 && p->z[p->n-1]=='\r' ) p->n--; } p->cTerm = (char)c; } if( p->z ) p->z[p->n] = 0; p->bNotFirst = 1; return p->z; } /* Forward references to the various virtual table methods implemented | > | 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 | } if( c=='\n' ){ p->nLine++; if( p->n>0 && p->z[p->n-1]=='\r' ) p->n--; } p->cTerm = (char)c; } assert( p->z==0 || p->n<p->nAlloc ); if( p->z ) p->z[p->n] = 0; p->bNotFirst = 1; return p->z; } /* Forward references to the various virtual table methods implemented |
︙ | ︙ | |||
617 618 619 620 621 622 623 | pNew->tstFlags = tstFlags; #endif if( bHeader!=1 ){ pNew->iStart = 0; }else if( pNew->zData ){ pNew->iStart = (int)sRdr.iIn; }else{ | | > > > > > > > > > | 618 619 620 621 622 623 624 625 626 627 628 629 630 631 632 633 634 635 636 637 638 639 640 641 642 643 644 645 646 647 648 649 650 651 | pNew->tstFlags = tstFlags; #endif if( bHeader!=1 ){ pNew->iStart = 0; }else if( pNew->zData ){ pNew->iStart = (int)sRdr.iIn; }else{ pNew->iStart = (int)(ftell(sRdr.in) - sRdr.nIn + sRdr.iIn); } csv_reader_reset(&sRdr); rc = sqlite3_declare_vtab(db, CSV_SCHEMA); if( rc ){ csv_errmsg(&sRdr, "bad schema: '%s' - %s", CSV_SCHEMA, sqlite3_errmsg(db)); goto csvtab_connect_error; } for(i=0; i<sizeof(azPValue)/sizeof(azPValue[0]); i++){ sqlite3_free(azPValue[i]); } /* Rationale for DIRECTONLY: ** An attacker who controls a database schema could use this vtab ** to exfiltrate sensitive data from other files in the filesystem. ** And, recommended practice is to put all CSV virtual tables in the ** TEMP namespace, so they should still be usable from within TEMP ** views, so there shouldn't be a serious loss of functionality by ** prohibiting the use of this vtab from persistent triggers and views. */ sqlite3_vtab_config(db, SQLITE_VTAB_DIRECTONLY); return SQLITE_OK; csvtab_connect_oom: rc = SQLITE_NOMEM; csv_errmsg(&sRdr, "out of memory"); csvtab_connect_error: |
︙ | ︙ | |||
737 738 739 740 741 742 743 | pCur->azVal[i] = zNew; pCur->aLen[i] = pCur->rdr.n+1; } memcpy(pCur->azVal[i], z, pCur->rdr.n+1); i++; } }while( pCur->rdr.cTerm==',' ); | | | 747 748 749 750 751 752 753 754 755 756 757 758 759 760 761 | pCur->azVal[i] = zNew; pCur->aLen[i] = pCur->rdr.n+1; } memcpy(pCur->azVal[i], z, pCur->rdr.n+1); i++; } }while( pCur->rdr.cTerm==',' ); if( z==0 && i==0 ){ pCur->iRowid = -1; }else{ pCur->iRowid++; while( i<pTab->nCol ){ sqlite3_free(pCur->azVal[i]); pCur->azVal[i] = 0; pCur->aLen[i] = 0; |
︙ | ︙ | |||
763 764 765 766 767 768 769 | sqlite3_vtab_cursor *cur, /* The cursor */ sqlite3_context *ctx, /* First argument to sqlite3_result_...() */ int i /* Which column to return */ ){ CsvCursor *pCur = (CsvCursor*)cur; CsvTable *pTab = (CsvTable*)cur->pVtab; if( i>=0 && i<pTab->nCol && pCur->azVal[i]!=0 ){ | | | 773 774 775 776 777 778 779 780 781 782 783 784 785 786 787 | sqlite3_vtab_cursor *cur, /* The cursor */ sqlite3_context *ctx, /* First argument to sqlite3_result_...() */ int i /* Which column to return */ ){ CsvCursor *pCur = (CsvCursor*)cur; CsvTable *pTab = (CsvTable*)cur->pVtab; if( i>=0 && i<pTab->nCol && pCur->azVal[i]!=0 ){ sqlite3_result_text(ctx, pCur->azVal[i], -1, SQLITE_TRANSIENT); } return SQLITE_OK; } /* ** Return the rowid for the current row. */ |
︙ | ︙ | |||
798 799 800 801 802 803 804 805 806 807 808 809 810 811 | sqlite3_vtab_cursor *pVtabCursor, int idxNum, const char *idxStr, int argc, sqlite3_value **argv ){ CsvCursor *pCur = (CsvCursor*)pVtabCursor; CsvTable *pTab = (CsvTable*)pVtabCursor->pVtab; pCur->iRowid = 0; if( pCur->rdr.in==0 ){ assert( pCur->rdr.zIn==pTab->zData ); assert( pTab->iStart>=0 ); assert( (size_t)pTab->iStart<=pCur->rdr.nIn ); pCur->rdr.iIn = pTab->iStart; }else{ fseek(pCur->rdr.in, pTab->iStart, SEEK_SET); | > > > > > > | 808 809 810 811 812 813 814 815 816 817 818 819 820 821 822 823 824 825 826 827 | sqlite3_vtab_cursor *pVtabCursor, int idxNum, const char *idxStr, int argc, sqlite3_value **argv ){ CsvCursor *pCur = (CsvCursor*)pVtabCursor; CsvTable *pTab = (CsvTable*)pVtabCursor->pVtab; pCur->iRowid = 0; /* Ensure the field buffer is always allocated. Otherwise, if the ** first field is zero bytes in size, this may be mistaken for an OOM ** error in csvtabNext(). */ if( csv_append(&pCur->rdr, 0) ) return SQLITE_NOMEM; if( pCur->rdr.in==0 ){ assert( pCur->rdr.zIn==pTab->zData ); assert( pTab->iStart>=0 ); assert( (size_t)pTab->iStart<=pCur->rdr.nIn ); pCur->rdr.iIn = pTab->iStart; }else{ fseek(pCur->rdr.in, pTab->iStart, SEEK_SET); |
︙ | ︙ | |||
928 929 930 931 932 933 934 | ** connection. */ int sqlite3_csv_init( sqlite3 *db, char **pzErrMsg, const sqlite3_api_routines *pApi ){ | | | 944 945 946 947 948 949 950 951 952 953 954 955 956 957 958 959 960 961 962 963 964 | ** connection. */ int sqlite3_csv_init( sqlite3 *db, char **pzErrMsg, const sqlite3_api_routines *pApi ){ #ifndef SQLITE_OMIT_VIRTUALTABLE int rc; SQLITE_EXTENSION_INIT2(pApi); rc = sqlite3_create_module(db, "csv", &CsvModule, 0); #ifdef SQLITE_TEST if( rc==SQLITE_OK ){ rc = sqlite3_create_module(db, "csv_wr", &CsvModuleFauxWrite, 0); } #endif return rc; #else return SQLITE_OK; #endif } |
Changes to ext/misc/dbdump.c.
︙ | ︙ | |||
191 192 193 194 195 196 197 | rc = sqlite3_prepare_v2(p->db, zSql, -1, &pStmt, 0); sqlite3_free(zSql); if( rc ) return 0; while( sqlite3_step(pStmt)==SQLITE_ROW ){ if( nCol>=nAlloc-2 ){ char **azNew; nAlloc = nAlloc*2 + nCol + 10; | | | 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 | rc = sqlite3_prepare_v2(p->db, zSql, -1, &pStmt, 0); sqlite3_free(zSql); if( rc ) return 0; while( sqlite3_step(pStmt)==SQLITE_ROW ){ if( nCol>=nAlloc-2 ){ char **azNew; nAlloc = nAlloc*2 + nCol + 10; azNew = sqlite3_realloc64(azCol, nAlloc*sizeof(azCol[0])); if( azNew==0 ) goto col_oom; azCol = azNew; azCol[0] = 0; } azCol[++nCol] = sqlite3_mprintf("%s", sqlite3_column_text(pStmt, 1)); if( azCol[nCol]==0 ) goto col_oom; if( sqlite3_column_int(pStmt, 5) ){ |
︙ | ︙ | |||
391 392 393 394 395 396 397 | zTable = azArg[0]; zType = azArg[1]; zSql = azArg[2]; if( strcmp(zTable, "sqlite_sequence")==0 ){ p->xCallback("DELETE FROM sqlite_sequence;\n", p->pArg); }else if( sqlite3_strglob("sqlite_stat?", zTable)==0 ){ | | | | 391 392 393 394 395 396 397 398 399 400 401 402 403 404 405 406 407 408 409 410 411 412 413 414 | zTable = azArg[0]; zType = azArg[1]; zSql = azArg[2]; if( strcmp(zTable, "sqlite_sequence")==0 ){ p->xCallback("DELETE FROM sqlite_sequence;\n", p->pArg); }else if( sqlite3_strglob("sqlite_stat?", zTable)==0 ){ p->xCallback("ANALYZE sqlite_schema;\n", p->pArg); }else if( strncmp(zTable, "sqlite_", 7)==0 ){ return 0; }else if( strncmp(zSql, "CREATE VIRTUAL TABLE", 20)==0 ){ if( !p->writableSchema ){ p->xCallback("PRAGMA writable_schema=ON;\n", p->pArg); p->writableSchema = 1; } output_formatted(p, "INSERT INTO sqlite_schema(type,name,tbl_name,rootpage,sql)" "VALUES('table','%q','%q',0,'%q');", zTable, zTable, zSql); return 0; }else{ if( sqlite3_strglob("CREATE TABLE ['\"]*", zSql)==0 ){ p->xCallback("CREATE TABLE IF NOT EXISTS ", p->pArg); p->xCallback(zSql+13, p->pArg); |
︙ | ︙ | |||
642 643 644 645 646 647 648 | if( x.rc ) return x.rc; x.db = db; x.xCallback = xCallback; x.pArg = pArg; xCallback("PRAGMA foreign_keys=OFF;\nBEGIN TRANSACTION;\n", pArg); if( zTable==0 ){ run_schema_dump_query(&x, | | | | | | | 642 643 644 645 646 647 648 649 650 651 652 653 654 655 656 657 658 659 660 661 662 663 664 665 666 667 668 669 670 671 672 673 674 675 676 | if( x.rc ) return x.rc; x.db = db; x.xCallback = xCallback; x.pArg = pArg; xCallback("PRAGMA foreign_keys=OFF;\nBEGIN TRANSACTION;\n", pArg); if( zTable==0 ){ run_schema_dump_query(&x, "SELECT name, type, sql FROM \"%w\".sqlite_schema " "WHERE sql NOT NULL AND type=='table' AND name!='sqlite_sequence'", zSchema ); run_schema_dump_query(&x, "SELECT name, type, sql FROM \"%w\".sqlite_schema " "WHERE name=='sqlite_sequence'", zSchema ); output_sql_from_query(&x, "SELECT sql FROM sqlite_schema " "WHERE sql NOT NULL AND type IN ('index','trigger','view')", 0 ); }else{ run_schema_dump_query(&x, "SELECT name, type, sql FROM \"%w\".sqlite_schema " "WHERE tbl_name=%Q COLLATE nocase AND type=='table'" " AND sql NOT NULL", zSchema, zTable ); output_sql_from_query(&x, "SELECT sql FROM \"%w\".sqlite_schema " "WHERE sql NOT NULL" " AND type IN ('index','trigger','view')" " AND tbl_name=%Q COLLATE nocase", zSchema, zTable ); } if( x.writableSchema ){ |
︙ | ︙ |
Added ext/misc/decimal.c.
> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340 341 342 343 344 345 346 347 348 349 350 351 352 353 354 355 356 357 358 359 360 361 362 363 364 365 366 367 368 369 370 371 372 373 374 375 376 377 378 379 380 381 382 383 384 385 386 387 388 389 390 391 392 393 394 395 396 397 398 399 400 401 402 403 404 405 406 407 408 409 410 411 412 413 414 415 416 417 418 419 420 421 422 423 424 425 426 427 428 429 430 431 432 433 434 435 436 437 438 439 440 441 442 443 444 445 446 447 448 449 450 451 452 453 454 455 456 457 458 459 460 461 462 463 464 465 466 467 468 469 470 471 472 473 474 475 476 477 478 479 480 481 482 483 484 485 486 487 488 489 490 491 492 493 494 495 496 497 498 499 500 501 502 503 504 505 506 507 508 509 510 511 512 513 514 515 516 517 518 519 520 521 522 523 524 525 526 527 528 529 530 531 532 533 534 535 536 537 538 539 540 541 542 543 544 545 546 547 548 549 550 551 552 553 554 555 556 557 558 559 560 561 562 563 564 565 566 567 568 569 570 571 572 573 574 575 576 577 578 579 580 581 582 583 584 585 586 587 588 589 590 591 592 593 594 595 596 597 598 599 600 601 602 603 604 605 606 607 608 609 610 611 612 613 614 615 616 617 618 619 620 621 622 623 624 625 626 627 628 629 630 631 632 633 634 635 | /* ** 2020-06-22 ** ** The author disclaims copyright to this source code. In place of ** a legal notice, here is a blessing: ** ** May you do good and not evil. ** May you find forgiveness for yourself and forgive others. ** May you share freely, never taking more than you give. ** ****************************************************************************** ** ** Routines to implement arbitrary-precision decimal math. ** ** The focus here is on simplicity and correctness, not performance. */ #include "sqlite3ext.h" SQLITE_EXTENSION_INIT1 #include <assert.h> #include <string.h> #include <ctype.h> #include <stdlib.h> /* Mark a function parameter as unused, to suppress nuisance compiler ** warnings. */ #ifndef UNUSED_PARAMETER # define UNUSED_PARAMETER(X) (void)(X) #endif /* A decimal object */ typedef struct Decimal Decimal; struct Decimal { char sign; /* 0 for positive, 1 for negative */ char oom; /* True if an OOM is encountered */ char isNull; /* True if holds a NULL rather than a number */ char isInit; /* True upon initialization */ int nDigit; /* Total number of digits */ int nFrac; /* Number of digits to the right of the decimal point */ signed char *a; /* Array of digits. Most significant first. */ }; /* ** Release memory held by a Decimal, but do not free the object itself. */ static void decimal_clear(Decimal *p){ sqlite3_free(p->a); } /* ** Destroy a Decimal object */ static void decimal_free(Decimal *p){ if( p ){ decimal_clear(p); sqlite3_free(p); } } /* ** Allocate a new Decimal object. Initialize it to the number given ** by the input string. */ static Decimal *decimal_new( sqlite3_context *pCtx, sqlite3_value *pIn, int nAlt, const unsigned char *zAlt ){ Decimal *p; int n, i; const unsigned char *zIn; int iExp = 0; p = sqlite3_malloc( sizeof(*p) ); if( p==0 ) goto new_no_mem; p->sign = 0; p->oom = 0; p->isInit = 1; p->isNull = 0; p->nDigit = 0; p->nFrac = 0; if( zAlt ){ n = nAlt, zIn = zAlt; }else{ if( sqlite3_value_type(pIn)==SQLITE_NULL ){ p->a = 0; p->isNull = 1; return p; } n = sqlite3_value_bytes(pIn); zIn = sqlite3_value_text(pIn); } p->a = sqlite3_malloc64( n+1 ); if( p->a==0 ) goto new_no_mem; for(i=0; isspace(zIn[i]); i++){} if( zIn[i]=='-' ){ p->sign = 1; i++; }else if( zIn[i]=='+' ){ i++; } while( i<n && zIn[i]=='0' ) i++; while( i<n ){ char c = zIn[i]; if( c>='0' && c<='9' ){ p->a[p->nDigit++] = c - '0'; }else if( c=='.' ){ p->nFrac = p->nDigit + 1; }else if( c=='e' || c=='E' ){ int j = i+1; int neg = 0; if( j>=n ) break; if( zIn[j]=='-' ){ neg = 1; j++; }else if( zIn[j]=='+' ){ j++; } while( j<n && iExp<1000000 ){ if( zIn[j]>='0' && zIn[j]<='9' ){ iExp = iExp*10 + zIn[j] - '0'; } j++; } if( neg ) iExp = -iExp; break; } i++; } if( p->nFrac ){ p->nFrac = p->nDigit - (p->nFrac - 1); } if( iExp>0 ){ if( p->nFrac>0 ){ if( iExp<=p->nFrac ){ p->nFrac -= iExp; iExp = 0; }else{ iExp -= p->nFrac; p->nFrac = 0; } } if( iExp>0 ){ p->a = sqlite3_realloc64(p->a, p->nDigit + iExp + 1 ); if( p->a==0 ) goto new_no_mem; memset(p->a+p->nDigit, 0, iExp); p->nDigit += iExp; } }else if( iExp<0 ){ int nExtra; iExp = -iExp; nExtra = p->nDigit - p->nFrac - 1; if( nExtra ){ if( nExtra>=iExp ){ p->nFrac += iExp; iExp = 0; }else{ iExp -= nExtra; p->nFrac = p->nDigit - 1; } } if( iExp>0 ){ p->a = sqlite3_realloc64(p->a, p->nDigit + iExp + 1 ); if( p->a==0 ) goto new_no_mem; memmove(p->a+iExp, p->a, p->nDigit); memset(p->a, 0, iExp); p->nDigit += iExp; p->nFrac += iExp; } } return p; new_no_mem: if( pCtx ) sqlite3_result_error_nomem(pCtx); sqlite3_free(p); return 0; } /* ** Make the given Decimal the result. */ static void decimal_result(sqlite3_context *pCtx, Decimal *p){ char *z; int i, j; int n; if( p==0 || p->oom ){ sqlite3_result_error_nomem(pCtx); return; } if( p->isNull ){ sqlite3_result_null(pCtx); return; } z = sqlite3_malloc( p->nDigit+4 ); if( z==0 ){ sqlite3_result_error_nomem(pCtx); return; } i = 0; if( p->nDigit==0 || (p->nDigit==1 && p->a[0]==0) ){ p->sign = 0; } if( p->sign ){ z[0] = '-'; i = 1; } n = p->nDigit - p->nFrac; if( n<=0 ){ z[i++] = '0'; } j = 0; while( n>1 && p->a[j]==0 ){ j++; n--; } while( n>0 ){ z[i++] = p->a[j] + '0'; j++; n--; } if( p->nFrac ){ z[i++] = '.'; do{ z[i++] = p->a[j] + '0'; j++; }while( j<p->nDigit ); } z[i] = 0; sqlite3_result_text(pCtx, z, i, sqlite3_free); } /* ** SQL Function: decimal(X) ** ** Convert input X into decimal and then back into text */ static void decimalFunc( sqlite3_context *context, int argc, sqlite3_value **argv ){ Decimal *p = decimal_new(context, argv[0], 0, 0); UNUSED_PARAMETER(argc); decimal_result(context, p); decimal_free(p); } /* ** Compare to Decimal objects. Return negative, 0, or positive if the ** first object is less than, equal to, or greater than the second. ** ** Preconditions for this routine: ** ** pA!=0 ** pA->isNull==0 ** pB!=0 ** pB->isNull==0 */ static int decimal_cmp(const Decimal *pA, const Decimal *pB){ int nASig, nBSig, rc, n; if( pA->sign!=pB->sign ){ return pA->sign ? -1 : +1; } if( pA->sign ){ const Decimal *pTemp = pA; pA = pB; pB = pTemp; } nASig = pA->nDigit - pA->nFrac; nBSig = pB->nDigit - pB->nFrac; if( nASig!=nBSig ){ return nASig - nBSig; } n = pA->nDigit; if( n>pB->nDigit ) n = pB->nDigit; rc = memcmp(pA->a, pB->a, n); if( rc==0 ){ rc = pA->nDigit - pB->nDigit; } return rc; } /* ** SQL Function: decimal_cmp(X, Y) ** ** Return negative, zero, or positive if X is less then, equal to, or ** greater than Y. */ static void decimalCmpFunc( sqlite3_context *context, int argc, sqlite3_value **argv ){ Decimal *pA = 0, *pB = 0; int rc; UNUSED_PARAMETER(argc); pA = decimal_new(context, argv[0], 0, 0); if( pA==0 || pA->isNull ) goto cmp_done; pB = decimal_new(context, argv[1], 0, 0); if( pB==0 || pB->isNull ) goto cmp_done; rc = decimal_cmp(pA, pB); if( rc<0 ) rc = -1; else if( rc>0 ) rc = +1; sqlite3_result_int(context, rc); cmp_done: decimal_free(pA); decimal_free(pB); } /* ** Expand the Decimal so that it has a least nDigit digits and nFrac ** digits to the right of the decimal point. */ static void decimal_expand(Decimal *p, int nDigit, int nFrac){ int nAddSig; int nAddFrac; if( p==0 ) return; nAddFrac = nFrac - p->nFrac; nAddSig = (nDigit - p->nDigit) - nAddFrac; if( nAddFrac==0 && nAddSig==0 ) return; p->a = sqlite3_realloc64(p->a, nDigit+1); if( p->a==0 ){ p->oom = 1; return; } if( nAddSig ){ memmove(p->a+nAddSig, p->a, p->nDigit); memset(p->a, 0, nAddSig); p->nDigit += nAddSig; } if( nAddFrac ){ memset(p->a+p->nDigit, 0, nAddFrac); p->nDigit += nAddFrac; p->nFrac += nAddFrac; } } /* ** Add the value pB into pA. ** ** Both pA and pB might become denormalized by this routine. */ static void decimal_add(Decimal *pA, Decimal *pB){ int nSig, nFrac, nDigit; int i, rc; if( pA==0 ){ return; } if( pA->oom || pB==0 || pB->oom ){ pA->oom = 1; return; } if( pA->isNull || pB->isNull ){ pA->isNull = 1; return; } nSig = pA->nDigit - pA->nFrac; if( nSig && pA->a[0]==0 ) nSig--; if( nSig<pB->nDigit-pB->nFrac ){ nSig = pB->nDigit - pB->nFrac; } nFrac = pA->nFrac; if( nFrac<pB->nFrac ) nFrac = pB->nFrac; nDigit = nSig + nFrac + 1; decimal_expand(pA, nDigit, nFrac); decimal_expand(pB, nDigit, nFrac); if( pA->oom || pB->oom ){ pA->oom = 1; }else{ if( pA->sign==pB->sign ){ int carry = 0; for(i=nDigit-1; i>=0; i--){ int x = pA->a[i] + pB->a[i] + carry; if( x>=10 ){ carry = 1; pA->a[i] = x - 10; }else{ carry = 0; pA->a[i] = x; } } }else{ signed char *aA, *aB; int borrow = 0; rc = memcmp(pA->a, pB->a, nDigit); if( rc<0 ){ aA = pB->a; aB = pA->a; pA->sign = !pA->sign; }else{ aA = pA->a; aB = pB->a; } for(i=nDigit-1; i>=0; i--){ int x = aA[i] - aB[i] - borrow; if( x<0 ){ pA->a[i] = x+10; borrow = 1; }else{ pA->a[i] = x; borrow = 0; } } } } } /* ** Compare text in decimal order. */ static int decimalCollFunc( void *notUsed, int nKey1, const void *pKey1, int nKey2, const void *pKey2 ){ const unsigned char *zA = (const unsigned char*)pKey1; const unsigned char *zB = (const unsigned char*)pKey2; Decimal *pA = decimal_new(0, 0, nKey1, zA); Decimal *pB = decimal_new(0, 0, nKey2, zB); int rc; UNUSED_PARAMETER(notUsed); if( pA==0 || pB==0 ){ rc = 0; }else{ rc = decimal_cmp(pA, pB); } decimal_free(pA); decimal_free(pB); return rc; } /* ** SQL Function: decimal_add(X, Y) ** decimal_sub(X, Y) ** ** Return the sum or difference of X and Y. */ static void decimalAddFunc( sqlite3_context *context, int argc, sqlite3_value **argv ){ Decimal *pA = decimal_new(context, argv[0], 0, 0); Decimal *pB = decimal_new(context, argv[1], 0, 0); UNUSED_PARAMETER(argc); decimal_add(pA, pB); decimal_result(context, pA); decimal_free(pA); decimal_free(pB); } static void decimalSubFunc( sqlite3_context *context, int argc, sqlite3_value **argv ){ Decimal *pA = decimal_new(context, argv[0], 0, 0); Decimal *pB = decimal_new(context, argv[1], 0, 0); UNUSED_PARAMETER(argc); if( pB ){ pB->sign = !pB->sign; decimal_add(pA, pB); decimal_result(context, pA); } decimal_free(pA); decimal_free(pB); } /* Aggregate funcion: decimal_sum(X) ** ** Works like sum() except that it uses decimal arithmetic for unlimited ** precision. */ static void decimalSumStep( sqlite3_context *context, int argc, sqlite3_value **argv ){ Decimal *p; Decimal *pArg; UNUSED_PARAMETER(argc); p = sqlite3_aggregate_context(context, sizeof(*p)); if( p==0 ) return; if( !p->isInit ){ p->isInit = 1; p->a = sqlite3_malloc(2); if( p->a==0 ){ p->oom = 1; }else{ p->a[0] = 0; } p->nDigit = 1; p->nFrac = 0; } if( sqlite3_value_type(argv[0])==SQLITE_NULL ) return; pArg = decimal_new(context, argv[0], 0, 0); decimal_add(p, pArg); decimal_free(pArg); } static void decimalSumInverse( sqlite3_context *context, int argc, sqlite3_value **argv ){ Decimal *p; Decimal *pArg; UNUSED_PARAMETER(argc); p = sqlite3_aggregate_context(context, sizeof(*p)); if( p==0 ) return; if( sqlite3_value_type(argv[0])==SQLITE_NULL ) return; pArg = decimal_new(context, argv[0], 0, 0); if( pArg ) pArg->sign = !pArg->sign; decimal_add(p, pArg); decimal_free(pArg); } static void decimalSumValue(sqlite3_context *context){ Decimal *p = sqlite3_aggregate_context(context, 0); if( p==0 ) return; decimal_result(context, p); } static void decimalSumFinalize(sqlite3_context *context){ Decimal *p = sqlite3_aggregate_context(context, 0); if( p==0 ) return; decimal_result(context, p); decimal_clear(p); } /* ** SQL Function: decimal_mul(X, Y) ** ** Return the product of X and Y. ** ** All significant digits after the decimal point are retained. ** Trailing zeros after the decimal point are omitted as long as ** the number of digits after the decimal point is no less than ** either the number of digits in either input. */ static void decimalMulFunc( sqlite3_context *context, int argc, sqlite3_value **argv ){ Decimal *pA = decimal_new(context, argv[0], 0, 0); Decimal *pB = decimal_new(context, argv[1], 0, 0); signed char *acc = 0; int i, j, k; int minFrac; UNUSED_PARAMETER(argc); if( pA==0 || pA->oom || pA->isNull || pB==0 || pB->oom || pB->isNull ){ goto mul_end; } acc = sqlite3_malloc64( pA->nDigit + pB->nDigit + 2 ); if( acc==0 ){ sqlite3_result_error_nomem(context); goto mul_end; } memset(acc, 0, pA->nDigit + pB->nDigit + 2); minFrac = pA->nFrac; if( pB->nFrac<minFrac ) minFrac = pB->nFrac; for(i=pA->nDigit-1; i>=0; i--){ signed char f = pA->a[i]; int carry = 0, x; for(j=pB->nDigit-1, k=i+j+3; j>=0; j--, k--){ x = acc[k] + f*pB->a[j] + carry; acc[k] = x%10; carry = x/10; } x = acc[k] + carry; acc[k] = x%10; acc[k-1] += x/10; } sqlite3_free(pA->a); pA->a = acc; acc = 0; pA->nDigit += pB->nDigit + 2; pA->nFrac += pB->nFrac; pA->sign ^= pB->sign; while( pA->nFrac>minFrac && pA->a[pA->nDigit-1]==0 ){ pA->nFrac--; pA->nDigit--; } decimal_result(context, pA); mul_end: sqlite3_free(acc); decimal_free(pA); decimal_free(pB); } #ifdef _WIN32 __declspec(dllexport) #endif int sqlite3_decimal_init( sqlite3 *db, char **pzErrMsg, const sqlite3_api_routines *pApi ){ int rc = SQLITE_OK; static const struct { const char *zFuncName; int nArg; void (*xFunc)(sqlite3_context*,int,sqlite3_value**); } aFunc[] = { { "decimal", 1, decimalFunc }, { "decimal_cmp", 2, decimalCmpFunc }, { "decimal_add", 2, decimalAddFunc }, { "decimal_sub", 2, decimalSubFunc }, { "decimal_mul", 2, decimalMulFunc }, }; unsigned int i; (void)pzErrMsg; /* Unused parameter */ SQLITE_EXTENSION_INIT2(pApi); for(i=0; i<(int)(sizeof(aFunc)/sizeof(aFunc[0])) && rc==SQLITE_OK; i++){ rc = sqlite3_create_function(db, aFunc[i].zFuncName, aFunc[i].nArg, SQLITE_UTF8|SQLITE_INNOCUOUS|SQLITE_DETERMINISTIC, 0, aFunc[i].xFunc, 0, 0); } if( rc==SQLITE_OK ){ rc = sqlite3_create_window_function(db, "decimal_sum", 1, SQLITE_UTF8|SQLITE_INNOCUOUS|SQLITE_DETERMINISTIC, 0, decimalSumStep, decimalSumFinalize, decimalSumValue, decimalSumInverse, 0); } if( rc==SQLITE_OK ){ rc = sqlite3_create_collation(db, "decimal", SQLITE_UTF8, 0, decimalCollFunc); } return rc; } |
Changes to ext/misc/eval.c.
︙ | ︙ | |||
40 41 42 43 44 45 46 | size_t sz = strlen(z); if( (sqlite3_int64)sz+p->nUsed+p->szSep+1 > p->nAlloc ){ char *zNew; p->nAlloc = p->nAlloc*2 + sz + p->szSep + 1; /* Using sqlite3_realloc64() would be better, but it is a recent ** addition and will cause a segfault if loaded by an older version ** of SQLite. */ | | | 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 | size_t sz = strlen(z); if( (sqlite3_int64)sz+p->nUsed+p->szSep+1 > p->nAlloc ){ char *zNew; p->nAlloc = p->nAlloc*2 + sz + p->szSep + 1; /* Using sqlite3_realloc64() would be better, but it is a recent ** addition and will cause a segfault if loaded by an older version ** of SQLite. */ zNew = p->nAlloc<=0x7fffffff ? sqlite3_realloc64(p->z, p->nAlloc) : 0; if( zNew==0 ){ sqlite3_free(p->z); memset(p, 0, sizeof(*p)); return 1; } p->z = zNew; } |
︙ | ︙ | |||
109 110 111 112 113 114 115 | sqlite3 *db, char **pzErrMsg, const sqlite3_api_routines *pApi ){ int rc = SQLITE_OK; SQLITE_EXTENSION_INIT2(pApi); (void)pzErrMsg; /* Unused parameter */ | | > | > | 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 | sqlite3 *db, char **pzErrMsg, const sqlite3_api_routines *pApi ){ int rc = SQLITE_OK; SQLITE_EXTENSION_INIT2(pApi); (void)pzErrMsg; /* Unused parameter */ rc = sqlite3_create_function(db, "eval", 1, SQLITE_UTF8|SQLITE_DIRECTONLY, 0, sqlEvalFunc, 0, 0); if( rc==SQLITE_OK ){ rc = sqlite3_create_function(db, "eval", 2, SQLITE_UTF8|SQLITE_DIRECTONLY, 0, sqlEvalFunc, 0, 0); } return rc; } |
Changes to ext/misc/explain.c.
︙ | ︙ | |||
12 13 14 15 16 17 18 | ** ** This file demonstrates an eponymous virtual table that returns the ** EXPLAIN output from an SQL statement. ** ** Usage example: ** ** .load ./explain | | | 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 | ** ** This file demonstrates an eponymous virtual table that returns the ** EXPLAIN output from an SQL statement. ** ** Usage example: ** ** .load ./explain ** SELECT p2 FROM explain('SELECT * FROM sqlite_schema') ** WHERE opcode='OpenRead'; ** ** This module was originally written to help simplify SQLite testing, ** by providing an easier means of verifying certain patterns in the ** generated bytecode. */ #if !defined(SQLITEINT_H) |
︙ | ︙ |
Changes to ext/misc/fileio.c.
︙ | ︙ | |||
68 69 70 71 72 73 74 75 76 77 78 79 80 81 | ** symlink, a text value containing the text of the link. For a ** directory, NULL. ** ** If a non-NULL value is specified for the optional $dir parameter and ** $path is a relative path, then $path is interpreted relative to $dir. ** And the paths returned in the "name" column of the table are also ** relative to directory $dir. */ #include "sqlite3ext.h" SQLITE_EXTENSION_INIT1 #include <stdio.h> #include <string.h> #include <assert.h> | > > > > > | 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 | ** symlink, a text value containing the text of the link. For a ** directory, NULL. ** ** If a non-NULL value is specified for the optional $dir parameter and ** $path is a relative path, then $path is interpreted relative to $dir. ** And the paths returned in the "name" column of the table are also ** relative to directory $dir. ** ** Notes on building this extension for Windows: ** Unless linked statically with the SQLite library, a preprocessor ** symbol, FILEIO_WIN32_DLL, must be #define'd to create a stand-alone ** DLL form of this extension for WIN32. See its use below for details. */ #include "sqlite3ext.h" SQLITE_EXTENSION_INIT1 #include <stdio.h> #include <string.h> #include <assert.h> |
︙ | ︙ | |||
117 118 119 120 121 122 123 | #define FSDIR_COLUMN_DATA 3 /* File content */ #define FSDIR_COLUMN_PATH 4 /* Path to top of search */ #define FSDIR_COLUMN_DIR 5 /* Path is relative to this directory */ /* ** Set the result stored by context ctx to a blob containing the | | > > > > > > > | > > | > > > > > > > > > > | | > > > > > | > | 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 | #define FSDIR_COLUMN_DATA 3 /* File content */ #define FSDIR_COLUMN_PATH 4 /* Path to top of search */ #define FSDIR_COLUMN_DIR 5 /* Path is relative to this directory */ /* ** Set the result stored by context ctx to a blob containing the ** contents of file zName. Or, leave the result unchanged (NULL) ** if the file does not exist or is unreadable. ** ** If the file exceeds the SQLite blob size limit, through an ** SQLITE_TOOBIG error. ** ** Throw an SQLITE_IOERR if there are difficulties pulling the file ** off of disk. */ static void readFileContents(sqlite3_context *ctx, const char *zName){ FILE *in; sqlite3_int64 nIn; void *pBuf; sqlite3 *db; int mxBlob; in = fopen(zName, "rb"); if( in==0 ){ /* File does not exist or is unreadable. Leave the result set to NULL. */ return; } fseek(in, 0, SEEK_END); nIn = ftell(in); rewind(in); db = sqlite3_context_db_handle(ctx); mxBlob = sqlite3_limit(db, SQLITE_LIMIT_LENGTH, -1); if( nIn>mxBlob ){ sqlite3_result_error_code(ctx, SQLITE_TOOBIG); fclose(in); return; } pBuf = sqlite3_malloc64( nIn ? nIn : 1 ); if( pBuf==0 ){ sqlite3_result_error_nomem(ctx); fclose(in); return; } if( nIn==(sqlite3_int64)fread(pBuf, 1, (size_t)nIn, in) ){ sqlite3_result_blob64(ctx, pBuf, nIn, sqlite3_free); }else{ sqlite3_result_error_code(ctx, SQLITE_IOERR); sqlite3_free(pBuf); } fclose(in); } /* ** Implementation of the "readfile(X)" SQL function. The entire content |
︙ | ︙ | |||
196 197 198 199 200 201 202 203 204 205 206 207 208 209 | fileIntervals.LowPart = pFileTime->dwLowDateTime; fileIntervals.HighPart = pFileTime->dwHighDateTime; return (fileIntervals.QuadPart - epochIntervals.QuadPart) / 10000000; } /* ** This function attempts to normalize the time values found in the stat() ** buffer to UTC. This is necessary on Win32, where the runtime library ** appears to return these values as local times. */ static void statTimesToUtc( const char *zPath, | > > > > > > > > > > > > > > > > | 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 | fileIntervals.LowPart = pFileTime->dwLowDateTime; fileIntervals.HighPart = pFileTime->dwHighDateTime; return (fileIntervals.QuadPart - epochIntervals.QuadPart) / 10000000; } #if defined(FILEIO_WIN32_DLL) && (defined(_WIN32) || defined(WIN32)) # /* To allow a standalone DLL, use this next replacement function: */ # undef sqlite3_win32_utf8_to_unicode # define sqlite3_win32_utf8_to_unicode utf8_to_utf16 # LPWSTR utf8_to_utf16(const char *z){ int nAllot = MultiByteToWideChar(CP_UTF8, 0, z, -1, NULL, 0); LPWSTR rv = sqlite3_malloc(nAllot * sizeof(WCHAR)); if( rv!=0 && 0 < MultiByteToWideChar(CP_UTF8, 0, z, -1, rv, nAllot) ) return rv; sqlite3_free(rv); return 0; } #endif /* ** This function attempts to normalize the time values found in the stat() ** buffer to UTC. This is necessary on Win32, where the runtime library ** appears to return these values as local times. */ static void statTimesToUtc( const char *zPath, |
︙ | ︙ | |||
264 265 266 267 268 269 270 | #endif } /* ** Argument zFile is the name of a file that will be created and/or written ** by SQL function writefile(). This function ensures that the directory ** zFile will be written to exists, creating it if required. The permissions | | > | < | | 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340 341 342 343 344 345 346 347 348 349 350 351 352 353 | #endif } /* ** Argument zFile is the name of a file that will be created and/or written ** by SQL function writefile(). This function ensures that the directory ** zFile will be written to exists, creating it if required. The permissions ** for any path components created by this function are set in accordance ** with the current umask. ** ** If an OOM condition is encountered, SQLITE_NOMEM is returned. Otherwise, ** SQLITE_OK is returned if the directory is successfully created, or ** SQLITE_ERROR otherwise. */ static int makeDirectory( const char *zFile ){ char *zCopy = sqlite3_mprintf("%s", zFile); int rc = SQLITE_OK; if( zCopy==0 ){ rc = SQLITE_NOMEM; }else{ int nCopy = (int)strlen(zCopy); int i = 1; while( rc==SQLITE_OK ){ struct stat sStat; int rc2; for(; zCopy[i]!='/' && i<nCopy; i++); if( i==nCopy ) break; zCopy[i] = '\0'; rc2 = fileStat(zCopy, &sStat); if( rc2!=0 ){ if( mkdir(zCopy, 0777) ) rc = SQLITE_ERROR; }else{ if( !S_ISDIR(sStat.st_mode) ) rc = SQLITE_ERROR; } zCopy[i] = '/'; i++; } |
︙ | ︙ | |||
318 319 320 321 322 323 324 325 326 327 | static int writeFile( sqlite3_context *pCtx, /* Context to return bytes written in */ const char *zFile, /* File to write */ sqlite3_value *pData, /* Data to write */ mode_t mode, /* MODE parameter passed to writefile() */ sqlite3_int64 mtime /* MTIME parameter (or -1 to not set time) */ ){ #if !defined(_WIN32) && !defined(WIN32) if( S_ISLNK(mode) ){ const char *zTo = (const char*)sqlite3_value_text(pData); | > | | 364 365 366 367 368 369 370 371 372 373 374 375 376 377 378 379 380 381 382 | static int writeFile( sqlite3_context *pCtx, /* Context to return bytes written in */ const char *zFile, /* File to write */ sqlite3_value *pData, /* Data to write */ mode_t mode, /* MODE parameter passed to writefile() */ sqlite3_int64 mtime /* MTIME parameter (or -1 to not set time) */ ){ if( zFile==0 ) return 1; #if !defined(_WIN32) && !defined(WIN32) if( S_ISLNK(mode) ){ const char *zTo = (const char*)sqlite3_value_text(pData); if( zTo==0 || symlink(zTo, zFile)<0 ) return 1; }else #endif { if( S_ISDIR(mode) ){ if( mkdir(zFile, mode) ){ /* The mkdir() call to create the directory failed. This might not ** be an error though - if there is already a directory at the same |
︙ | ︙ | |||
365 366 367 368 369 370 371 372 373 374 375 376 377 378 | if( rc ) return 2; sqlite3_result_int64(pCtx, nWrite); } } if( mtime>=0 ){ #if defined(_WIN32) /* Windows */ FILETIME lastAccess; FILETIME lastWrite; SYSTEMTIME currentTime; LONGLONG intervals; HANDLE hFile; LPWSTR zUnicodeName; | > | 412 413 414 415 416 417 418 419 420 421 422 423 424 425 426 | if( rc ) return 2; sqlite3_result_int64(pCtx, nWrite); } } if( mtime>=0 ){ #if defined(_WIN32) #if !SQLITE_OS_WINRT /* Windows */ FILETIME lastAccess; FILETIME lastWrite; SYSTEMTIME currentTime; LONGLONG intervals; HANDLE hFile; LPWSTR zUnicodeName; |
︙ | ︙ | |||
395 396 397 398 399 400 401 402 403 404 405 406 407 408 | if( hFile!=INVALID_HANDLE_VALUE ){ BOOL bResult = SetFileTime(hFile, NULL, &lastAccess, &lastWrite); CloseHandle(hFile); return !bResult; }else{ return 1; } #elif defined(AT_FDCWD) && 0 /* utimensat() is not universally available */ /* Recent unix */ struct timespec times[2]; times[0].tv_nsec = times[1].tv_nsec = 0; times[0].tv_sec = time(0); times[1].tv_sec = mtime; if( utimensat(AT_FDCWD, zFile, times, AT_SYMLINK_NOFOLLOW) ){ | > | 443 444 445 446 447 448 449 450 451 452 453 454 455 456 457 | if( hFile!=INVALID_HANDLE_VALUE ){ BOOL bResult = SetFileTime(hFile, NULL, &lastAccess, &lastWrite); CloseHandle(hFile); return !bResult; }else{ return 1; } #endif #elif defined(AT_FDCWD) && 0 /* utimensat() is not universally available */ /* Recent unix */ struct timespec times[2]; times[0].tv_nsec = times[1].tv_nsec = 0; times[0].tv_sec = time(0); times[1].tv_sec = mtime; if( utimensat(AT_FDCWD, zFile, times, AT_SYMLINK_NOFOLLOW) ){ |
︙ | ︙ | |||
451 452 453 454 455 456 457 | } if( argc==4 ){ mtime = sqlite3_value_int64(argv[3]); } res = writeFile(context, zFile, argv[1], mode, mtime); if( res==1 && errno==ENOENT ){ | | | 500 501 502 503 504 505 506 507 508 509 510 511 512 513 514 | } if( argc==4 ){ mtime = sqlite3_value_int64(argv[3]); } res = writeFile(context, zFile, argv[1], mode, mtime); if( res==1 && errno==ENOENT ){ if( makeDirectory(zFile)==SQLITE_OK ){ res = writeFile(context, zFile, argv[1], mode, mtime); } } if( argc>2 && res!=0 ){ if( S_ISLNK(mode) ){ ctxErrorMsg(context, "failed to create symlink: %s", zFile); |
︙ | ︙ | |||
556 557 558 559 560 561 562 563 564 565 566 567 568 569 | (void)argv; (void)pzErr; rc = sqlite3_declare_vtab(db, "CREATE TABLE x" FSDIR_SCHEMA); if( rc==SQLITE_OK ){ pNew = (fsdir_tab*)sqlite3_malloc( sizeof(*pNew) ); if( pNew==0 ) return SQLITE_NOMEM; memset(pNew, 0, sizeof(*pNew)); } *ppVtab = (sqlite3_vtab*)pNew; return rc; } /* ** This method is the destructor for fsdir vtab objects. | > | 605 606 607 608 609 610 611 612 613 614 615 616 617 618 619 | (void)argv; (void)pzErr; rc = sqlite3_declare_vtab(db, "CREATE TABLE x" FSDIR_SCHEMA); if( rc==SQLITE_OK ){ pNew = (fsdir_tab*)sqlite3_malloc( sizeof(*pNew) ); if( pNew==0 ) return SQLITE_NOMEM; memset(pNew, 0, sizeof(*pNew)); sqlite3_vtab_config(db, SQLITE_VTAB_DIRECTONLY); } *ppVtab = (sqlite3_vtab*)pNew; return rc; } /* ** This method is the destructor for fsdir vtab objects. |
︙ | ︙ | |||
642 643 644 645 646 647 648 | pCur->iRowid++; if( S_ISDIR(m) ){ /* Descend into this directory */ int iNew = pCur->iLvl + 1; FsdirLevel *pLvl; if( iNew>=pCur->nLvl ){ int nNew = iNew+1; | | | | 692 693 694 695 696 697 698 699 700 701 702 703 704 705 706 707 | pCur->iRowid++; if( S_ISDIR(m) ){ /* Descend into this directory */ int iNew = pCur->iLvl + 1; FsdirLevel *pLvl; if( iNew>=pCur->nLvl ){ int nNew = iNew+1; sqlite3_int64 nByte = nNew*sizeof(FsdirLevel); FsdirLevel *aNew = (FsdirLevel*)sqlite3_realloc64(pCur->aLvl, nByte); if( aNew==0 ) return SQLITE_NOMEM; memset(&aNew[pCur->nLvl], 0, sizeof(FsdirLevel)*(nNew-pCur->nLvl)); pCur->aLvl = aNew; pCur->nLvl = nNew; } pCur->iLvl = iNew; pLvl = &pCur->aLvl[iNew]; |
︙ | ︙ | |||
723 724 725 726 727 728 729 | mode_t m = pCur->sStat.st_mode; if( S_ISDIR(m) ){ sqlite3_result_null(ctx); #if !defined(_WIN32) && !defined(WIN32) }else if( S_ISLNK(m) ){ char aStatic[64]; char *aBuf = aStatic; | | | | 773 774 775 776 777 778 779 780 781 782 783 784 785 786 787 788 789 790 791 792 793 794 795 | mode_t m = pCur->sStat.st_mode; if( S_ISDIR(m) ){ sqlite3_result_null(ctx); #if !defined(_WIN32) && !defined(WIN32) }else if( S_ISLNK(m) ){ char aStatic[64]; char *aBuf = aStatic; sqlite3_int64 nBuf = 64; int n; while( 1 ){ n = readlink(pCur->zPath, aBuf, nBuf); if( n<nBuf ) break; if( aBuf!=aStatic ) sqlite3_free(aBuf); nBuf = nBuf*2; aBuf = sqlite3_malloc64(nBuf); if( aBuf==0 ){ sqlite3_result_error_nomem(ctx); return SQLITE_NOMEM; } } sqlite3_result_text(ctx, aBuf, n, SQLITE_TRANSIENT); |
︙ | ︙ | |||
949 950 951 952 953 954 955 | sqlite3 *db, char **pzErrMsg, const sqlite3_api_routines *pApi ){ int rc = SQLITE_OK; SQLITE_EXTENSION_INIT2(pApi); (void)pzErrMsg; /* Unused parameter */ | | > | > > > > > > > > > | 999 1000 1001 1002 1003 1004 1005 1006 1007 1008 1009 1010 1011 1012 1013 1014 1015 1016 1017 1018 1019 1020 1021 1022 1023 1024 1025 1026 1027 1028 1029 1030 | sqlite3 *db, char **pzErrMsg, const sqlite3_api_routines *pApi ){ int rc = SQLITE_OK; SQLITE_EXTENSION_INIT2(pApi); (void)pzErrMsg; /* Unused parameter */ rc = sqlite3_create_function(db, "readfile", 1, SQLITE_UTF8|SQLITE_DIRECTONLY, 0, readfileFunc, 0, 0); if( rc==SQLITE_OK ){ rc = sqlite3_create_function(db, "writefile", -1, SQLITE_UTF8|SQLITE_DIRECTONLY, 0, writefileFunc, 0, 0); } if( rc==SQLITE_OK ){ rc = sqlite3_create_function(db, "lsmode", 1, SQLITE_UTF8, 0, lsModeFunc, 0, 0); } if( rc==SQLITE_OK ){ rc = fsdirRegister(db); } return rc; } #if defined(FILEIO_WIN32_DLL) && (defined(_WIN32) || defined(WIN32)) /* To allow a standalone DLL, make test_windirent.c use the same * redefined SQLite API calls as the above extension code does. * Just pull in this .c to accomplish this. As a beneficial side * effect, this extension becomes a single translation unit. */ # include "test_windirent.c" #endif |
Added ext/misc/fossildelta.c.
> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340 341 342 343 344 345 346 347 348 349 350 351 352 353 354 355 356 357 358 359 360 361 362 363 364 365 366 367 368 369 370 371 372 373 374 375 376 377 378 379 380 381 382 383 384 385 386 387 388 389 390 391 392 393 394 395 396 397 398 399 400 401 402 403 404 405 406 407 408 409 410 411 412 413 414 415 416 417 418 419 420 421 422 423 424 425 426 427 428 429 430 431 432 433 434 435 436 437 438 439 440 441 442 443 444 445 446 447 448 449 450 451 452 453 454 455 456 457 458 459 460 461 462 463 464 465 466 467 468 469 470 471 472 473 474 475 476 477 478 479 480 481 482 483 484 485 486 487 488 489 490 491 492 493 494 495 496 497 498 499 500 501 502 503 504 505 506 507 508 509 510 511 512 513 514 515 516 517 518 519 520 521 522 523 524 525 526 527 528 529 530 531 532 533 534 535 536 537 538 539 540 541 542 543 544 545 546 547 548 549 550 551 552 553 554 555 556 557 558 559 560 561 562 563 564 565 566 567 568 569 570 571 572 573 574 575 576 577 578 579 580 581 582 583 584 585 586 587 588 589 590 591 592 593 594 595 596 597 598 599 600 601 602 603 604 605 606 607 608 609 610 611 612 613 614 615 616 617 618 619 620 621 622 623 624 625 626 627 628 629 630 631 632 633 634 635 636 637 638 639 640 641 642 643 644 645 646 647 648 649 650 651 652 653 654 655 656 657 658 659 660 661 662 663 664 665 666 667 668 669 670 671 672 673 674 675 676 677 678 679 680 681 682 683 684 685 686 687 688 689 690 691 692 693 694 695 696 697 698 699 700 701 702 703 704 705 706 707 708 709 710 711 712 713 714 715 716 717 718 719 720 721 722 723 724 725 726 727 728 729 730 731 732 733 734 735 736 737 738 739 740 741 742 743 744 745 746 747 748 749 750 751 752 753 754 755 756 757 758 759 760 761 762 763 764 765 766 767 768 769 770 771 772 773 774 775 776 777 778 779 780 781 782 783 784 785 786 787 788 789 790 791 792 793 794 795 796 797 798 799 800 801 802 803 804 805 806 807 808 809 810 811 812 813 814 815 816 817 818 819 820 821 822 823 824 825 826 827 828 829 830 831 832 833 834 835 836 837 838 839 840 841 842 843 844 845 846 847 848 849 850 851 852 853 854 855 856 857 858 859 860 861 862 863 864 865 866 867 868 869 870 871 872 873 874 875 876 877 878 879 880 881 882 883 884 885 886 887 888 889 890 891 892 893 894 895 896 897 898 899 900 901 902 903 904 905 906 907 908 909 910 911 912 913 914 915 916 917 918 919 920 921 922 923 924 925 926 927 928 929 930 931 932 933 934 935 936 937 938 939 940 941 942 943 944 945 946 947 948 949 950 951 952 953 954 955 956 957 958 959 960 961 962 963 964 965 966 967 968 969 970 971 972 973 974 975 976 977 978 979 980 981 982 983 984 985 986 987 988 989 990 991 992 993 994 995 996 997 998 999 1000 1001 1002 1003 1004 1005 1006 1007 1008 1009 1010 1011 1012 1013 1014 1015 1016 1017 1018 1019 1020 1021 1022 1023 1024 1025 1026 1027 1028 1029 1030 1031 1032 1033 1034 1035 1036 1037 1038 1039 1040 1041 1042 1043 1044 1045 1046 1047 1048 1049 1050 1051 1052 1053 1054 1055 1056 1057 1058 1059 1060 1061 1062 1063 1064 1065 1066 1067 1068 1069 1070 1071 1072 1073 1074 1075 1076 1077 1078 1079 1080 1081 1082 1083 1084 1085 1086 1087 1088 1089 1090 1091 1092 | /* ** 2019-02-19 ** ** The author disclaims copyright to this source code. In place of ** a legal notice, here is a blessing: ** ** May you do good and not evil. ** May you find forgiveness for yourself and forgive others. ** May you share freely, never taking more than you give. ** ****************************************************************************** ** ** This SQLite extension implements the delta functions used by the RBU ** extension. Three scalar functions and one table-valued function are ** implemented here: ** ** delta_apply(X,D) -- apply delta D to file X and return the result ** delta_create(X,Y) -- compute and return a delta that carries X into Y ** delta_output_size(D) -- blob size in bytes output from applying delta D ** delta_parse(D) -- returns rows describing delta D ** ** The delta format is the Fossil delta format, described in a comment ** on the delete_create() function implementation below, and also at ** ** https://www.fossil-scm.org/fossil/doc/trunk/www/delta_format.wiki ** ** This delta format is used by the RBU extension, which is the main ** reason that these routines are included in the extension library. ** RBU does not use this extension directly. Rather, this extension is ** provided as a convenience to developers who want to analyze RBU files ** that contain deltas. */ #include <string.h> #include <assert.h> #include <stdlib.h> #include "sqlite3ext.h" SQLITE_EXTENSION_INIT1 #ifndef SQLITE_AMALGAMATION /* ** The "u32" type must be an unsigned 32-bit integer. Adjust this */ typedef unsigned int u32; /* ** Must be a 16-bit value */ typedef short int s16; typedef unsigned short int u16; #endif /* SQLITE_AMALGAMATION */ /* ** The width of a hash window in bytes. The algorithm only works if this ** is a power of 2. */ #define NHASH 16 /* ** The current state of the rolling hash. ** ** z[] holds the values that have been hashed. z[] is a circular buffer. ** z[i] is the first entry and z[(i+NHASH-1)%NHASH] is the last entry of ** the window. ** ** Hash.a is the sum of all elements of hash.z[]. Hash.b is a weighted ** sum. Hash.b is z[i]*NHASH + z[i+1]*(NHASH-1) + ... + z[i+NHASH-1]*1. ** (Each index for z[] should be module NHASH, of course. The %NHASH operator ** is omitted in the prior expression for brevity.) */ typedef struct hash hash; struct hash { u16 a, b; /* Hash values */ u16 i; /* Start of the hash window */ char z[NHASH]; /* The values that have been hashed */ }; /* ** Initialize the rolling hash using the first NHASH characters of z[] */ static void hash_init(hash *pHash, const char *z){ u16 a, b, i; a = b = z[0]; for(i=1; i<NHASH; i++){ a += z[i]; b += a; } memcpy(pHash->z, z, NHASH); pHash->a = a & 0xffff; pHash->b = b & 0xffff; pHash->i = 0; } /* ** Advance the rolling hash by a single character "c" */ static void hash_next(hash *pHash, int c){ u16 old = pHash->z[pHash->i]; pHash->z[pHash->i] = c; pHash->i = (pHash->i+1)&(NHASH-1); pHash->a = pHash->a - old + c; pHash->b = pHash->b - NHASH*old + pHash->a; } /* ** Return a 32-bit hash value */ static u32 hash_32bit(hash *pHash){ return (pHash->a & 0xffff) | (((u32)(pHash->b & 0xffff))<<16); } /* ** Compute a hash on NHASH bytes. ** ** This routine is intended to be equivalent to: ** hash h; ** hash_init(&h, zInput); ** return hash_32bit(&h); */ static u32 hash_once(const char *z){ u16 a, b, i; a = b = z[0]; for(i=1; i<NHASH; i++){ a += z[i]; b += a; } return a | (((u32)b)<<16); } /* ** Write an base-64 integer into the given buffer. */ static void putInt(unsigned int v, char **pz){ static const char zDigits[] = "0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZ_abcdefghijklmnopqrstuvwxyz~"; /* 123456789 123456789 123456789 123456789 123456789 123456789 123 */ int i, j; char zBuf[20]; if( v==0 ){ *(*pz)++ = '0'; return; } for(i=0; v>0; i++, v>>=6){ zBuf[i] = zDigits[v&0x3f]; } for(j=i-1; j>=0; j--){ *(*pz)++ = zBuf[j]; } } /* ** Read bytes from *pz and convert them into a positive integer. When ** finished, leave *pz pointing to the first character past the end of ** the integer. The *pLen parameter holds the length of the string ** in *pz and is decremented once for each character in the integer. */ static unsigned int deltaGetInt(const char **pz, int *pLen){ static const signed char zValue[] = { -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, -1, -1, -1, -1, -1, -1, -1, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, -1, -1, -1, -1, 36, -1, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, -1, -1, -1, 63, -1, }; unsigned int v = 0; int c; unsigned char *z = (unsigned char*)*pz; unsigned char *zStart = z; while( (c = zValue[0x7f&*(z++)])>=0 ){ v = (v<<6) + c; } z--; *pLen -= z - zStart; *pz = (char*)z; return v; } /* ** Return the number digits in the base-64 representation of a positive integer */ static int digit_count(int v){ unsigned int i, x; for(i=1, x=64; v>=x; i++, x <<= 6){} return i; } #ifdef __GNUC__ # define GCC_VERSION (__GNUC__*1000000+__GNUC_MINOR__*1000+__GNUC_PATCHLEVEL__) #else # define GCC_VERSION 0 #endif /* ** Compute a 32-bit big-endian checksum on the N-byte buffer. If the ** buffer is not a multiple of 4 bytes length, compute the sum that would ** have occurred if the buffer was padded with zeros to the next multiple ** of four bytes. */ static unsigned int checksum(const char *zIn, size_t N){ static const int byteOrderTest = 1; const unsigned char *z = (const unsigned char *)zIn; const unsigned char *zEnd = (const unsigned char*)&zIn[N&~3]; unsigned sum = 0; assert( (z - (const unsigned char*)0)%4==0 ); /* Four-byte alignment */ if( 0==*(char*)&byteOrderTest ){ /* This is a big-endian machine */ while( z<zEnd ){ sum += *(unsigned*)z; z += 4; } }else{ /* A little-endian machine */ #if GCC_VERSION>=4003000 while( z<zEnd ){ sum += __builtin_bswap32(*(unsigned*)z); z += 4; } #elif defined(_MSC_VER) && _MSC_VER>=1300 while( z<zEnd ){ sum += _byteswap_ulong(*(unsigned*)z); z += 4; } #else unsigned sum0 = 0; unsigned sum1 = 0; unsigned sum2 = 0; while(N >= 16){ sum0 += ((unsigned)z[0] + z[4] + z[8] + z[12]); sum1 += ((unsigned)z[1] + z[5] + z[9] + z[13]); sum2 += ((unsigned)z[2] + z[6] + z[10]+ z[14]); sum += ((unsigned)z[3] + z[7] + z[11]+ z[15]); z += 16; N -= 16; } while(N >= 4){ sum0 += z[0]; sum1 += z[1]; sum2 += z[2]; sum += z[3]; z += 4; N -= 4; } sum += (sum2 << 8) + (sum1 << 16) + (sum0 << 24); #endif } switch(N&3){ case 3: sum += (z[2] << 8); case 2: sum += (z[1] << 16); case 1: sum += (z[0] << 24); default: ; } return sum; } /* ** Create a new delta. ** ** The delta is written into a preallocated buffer, zDelta, which ** should be at least 60 bytes longer than the target file, zOut. ** The delta string will be NUL-terminated, but it might also contain ** embedded NUL characters if either the zSrc or zOut files are ** binary. This function returns the length of the delta string ** in bytes, excluding the final NUL terminator character. ** ** Output Format: ** ** The delta begins with a base64 number followed by a newline. This ** number is the number of bytes in the TARGET file. Thus, given a ** delta file z, a program can compute the size of the output file ** simply by reading the first line and decoding the base-64 number ** found there. The delta_output_size() routine does exactly this. ** ** After the initial size number, the delta consists of a series of ** literal text segments and commands to copy from the SOURCE file. ** A copy command looks like this: ** ** NNN@MMM, ** ** where NNN is the number of bytes to be copied and MMM is the offset ** into the source file of the first byte (both base-64). If NNN is 0 ** it means copy the rest of the input file. Literal text is like this: ** ** NNN:TTTTT ** ** where NNN is the number of bytes of text (base-64) and TTTTT is the text. ** ** The last term is of the form ** ** NNN; ** ** In this case, NNN is a 32-bit bigendian checksum of the output file ** that can be used to verify that the delta applied correctly. All ** numbers are in base-64. ** ** Pure text files generate a pure text delta. Binary files generate a ** delta that may contain some binary data. ** ** Algorithm: ** ** The encoder first builds a hash table to help it find matching ** patterns in the source file. 16-byte chunks of the source file ** sampled at evenly spaced intervals are used to populate the hash ** table. ** ** Next we begin scanning the target file using a sliding 16-byte ** window. The hash of the 16-byte window in the target is used to ** search for a matching section in the source file. When a match ** is found, a copy command is added to the delta. An effort is ** made to extend the matching section to regions that come before ** and after the 16-byte hash window. A copy command is only issued ** if the result would use less space that just quoting the text ** literally. Literal text is added to the delta for sections that ** do not match or which can not be encoded efficiently using copy ** commands. */ static int delta_create( const char *zSrc, /* The source or pattern file */ unsigned int lenSrc, /* Length of the source file */ const char *zOut, /* The target file */ unsigned int lenOut, /* Length of the target file */ char *zDelta /* Write the delta into this buffer */ ){ int i, base; char *zOrigDelta = zDelta; hash h; int nHash; /* Number of hash table entries */ int *landmark; /* Primary hash table */ int *collide; /* Collision chain */ int lastRead = -1; /* Last byte of zSrc read by a COPY command */ /* Add the target file size to the beginning of the delta */ putInt(lenOut, &zDelta); *(zDelta++) = '\n'; /* If the source file is very small, it means that we have no ** chance of ever doing a copy command. Just output a single ** literal segment for the entire target and exit. */ if( lenSrc<=NHASH ){ putInt(lenOut, &zDelta); *(zDelta++) = ':'; memcpy(zDelta, zOut, lenOut); zDelta += lenOut; putInt(checksum(zOut, lenOut), &zDelta); *(zDelta++) = ';'; return zDelta - zOrigDelta; } /* Compute the hash table used to locate matching sections in the ** source file. */ nHash = lenSrc/NHASH; collide = sqlite3_malloc64( (sqlite3_int64)nHash*2*sizeof(int) ); memset(collide, -1, nHash*2*sizeof(int)); landmark = &collide[nHash]; for(i=0; i<lenSrc-NHASH; i+=NHASH){ int hv = hash_once(&zSrc[i]) % nHash; collide[i/NHASH] = landmark[hv]; landmark[hv] = i/NHASH; } /* Begin scanning the target file and generating copy commands and ** literal sections of the delta. */ base = 0; /* We have already generated everything before zOut[base] */ while( base+NHASH<lenOut ){ int iSrc, iBlock; unsigned int bestCnt, bestOfst=0, bestLitsz=0; hash_init(&h, &zOut[base]); i = 0; /* Trying to match a landmark against zOut[base+i] */ bestCnt = 0; while( 1 ){ int hv; int limit = 250; hv = hash_32bit(&h) % nHash; iBlock = landmark[hv]; while( iBlock>=0 && (limit--)>0 ){ /* ** The hash window has identified a potential match against ** landmark block iBlock. But we need to investigate further. ** ** Look for a region in zOut that matches zSrc. Anchor the search ** at zSrc[iSrc] and zOut[base+i]. Do not include anything prior to ** zOut[base] or after zOut[outLen] nor anything after zSrc[srcLen]. ** ** Set cnt equal to the length of the match and set ofst so that ** zSrc[ofst] is the first element of the match. litsz is the number ** of characters between zOut[base] and the beginning of the match. ** sz will be the overhead (in bytes) needed to encode the copy ** command. Only generate copy command if the overhead of the ** copy command is less than the amount of literal text to be copied. */ int cnt, ofst, litsz; int j, k, x, y; int sz; int limitX; /* Beginning at iSrc, match forwards as far as we can. j counts ** the number of characters that match */ iSrc = iBlock*NHASH; y = base+i; limitX = ( lenSrc-iSrc <= lenOut-y ) ? lenSrc : iSrc + lenOut - y; for(x=iSrc; x<limitX; x++, y++){ if( zSrc[x]!=zOut[y] ) break; } j = x - iSrc - 1; /* Beginning at iSrc-1, match backwards as far as we can. k counts ** the number of characters that match */ for(k=1; k<iSrc && k<=i; k++){ if( zSrc[iSrc-k]!=zOut[base+i-k] ) break; } k--; /* Compute the offset and size of the matching region */ ofst = iSrc-k; cnt = j+k+1; litsz = i-k; /* Number of bytes of literal text before the copy */ /* sz will hold the number of bytes needed to encode the "insert" ** command and the copy command, not counting the "insert" text */ sz = digit_count(i-k)+digit_count(cnt)+digit_count(ofst)+3; if( cnt>=sz && cnt>bestCnt ){ /* Remember this match only if it is the best so far and it ** does not increase the file size */ bestCnt = cnt; bestOfst = iSrc-k; bestLitsz = litsz; } /* Check the next matching block */ iBlock = collide[iBlock]; } /* We have a copy command that does not cause the delta to be larger ** than a literal insert. So add the copy command to the delta. */ if( bestCnt>0 ){ if( bestLitsz>0 ){ /* Add an insert command before the copy */ putInt(bestLitsz,&zDelta); *(zDelta++) = ':'; memcpy(zDelta, &zOut[base], bestLitsz); zDelta += bestLitsz; base += bestLitsz; } base += bestCnt; putInt(bestCnt, &zDelta); *(zDelta++) = '@'; putInt(bestOfst, &zDelta); *(zDelta++) = ','; if( bestOfst + bestCnt -1 > lastRead ){ lastRead = bestOfst + bestCnt - 1; } bestCnt = 0; break; } /* If we reach this point, it means no match is found so far */ if( base+i+NHASH>=lenOut ){ /* We have reached the end of the file and have not found any ** matches. Do an "insert" for everything that does not match */ putInt(lenOut-base, &zDelta); *(zDelta++) = ':'; memcpy(zDelta, &zOut[base], lenOut-base); zDelta += lenOut-base; base = lenOut; break; } /* Advance the hash by one character. Keep looking for a match */ hash_next(&h, zOut[base+i+NHASH]); i++; } } /* Output a final "insert" record to get all the text at the end of ** the file that does not match anything in the source file. */ if( base<lenOut ){ putInt(lenOut-base, &zDelta); *(zDelta++) = ':'; memcpy(zDelta, &zOut[base], lenOut-base); zDelta += lenOut-base; } /* Output the final checksum record. */ putInt(checksum(zOut, lenOut), &zDelta); *(zDelta++) = ';'; sqlite3_free(collide); return zDelta - zOrigDelta; } /* ** Return the size (in bytes) of the output from applying ** a delta. ** ** This routine is provided so that an procedure that is able ** to call delta_apply() can learn how much space is required ** for the output and hence allocate nor more space that is really ** needed. */ static int delta_output_size(const char *zDelta, int lenDelta){ int size; size = deltaGetInt(&zDelta, &lenDelta); if( *zDelta!='\n' ){ /* ERROR: size integer not terminated by "\n" */ return -1; } return size; } /* ** Apply a delta. ** ** The output buffer should be big enough to hold the whole output ** file and a NUL terminator at the end. The delta_output_size() ** routine will determine this size for you. ** ** The delta string should be null-terminated. But the delta string ** may contain embedded NUL characters (if the input and output are ** binary files) so we also have to pass in the length of the delta in ** the lenDelta parameter. ** ** This function returns the size of the output file in bytes (excluding ** the final NUL terminator character). Except, if the delta string is ** malformed or intended for use with a source file other than zSrc, ** then this routine returns -1. ** ** Refer to the delta_create() documentation above for a description ** of the delta file format. */ static int delta_apply( const char *zSrc, /* The source or pattern file */ int lenSrc, /* Length of the source file */ const char *zDelta, /* Delta to apply to the pattern */ int lenDelta, /* Length of the delta */ char *zOut /* Write the output into this preallocated buffer */ ){ unsigned int limit; unsigned int total = 0; #ifdef FOSSIL_ENABLE_DELTA_CKSUM_TEST char *zOrigOut = zOut; #endif limit = deltaGetInt(&zDelta, &lenDelta); if( *zDelta!='\n' ){ /* ERROR: size integer not terminated by "\n" */ return -1; } zDelta++; lenDelta--; while( *zDelta && lenDelta>0 ){ unsigned int cnt, ofst; cnt = deltaGetInt(&zDelta, &lenDelta); switch( zDelta[0] ){ case '@': { zDelta++; lenDelta--; ofst = deltaGetInt(&zDelta, &lenDelta); if( lenDelta>0 && zDelta[0]!=',' ){ /* ERROR: copy command not terminated by ',' */ return -1; } zDelta++; lenDelta--; total += cnt; if( total>limit ){ /* ERROR: copy exceeds output file size */ return -1; } if( ofst+cnt > lenSrc ){ /* ERROR: copy extends past end of input */ return -1; } memcpy(zOut, &zSrc[ofst], cnt); zOut += cnt; break; } case ':': { zDelta++; lenDelta--; total += cnt; if( total>limit ){ /* ERROR: insert command gives an output larger than predicted */ return -1; } if( cnt>lenDelta ){ /* ERROR: insert count exceeds size of delta */ return -1; } memcpy(zOut, zDelta, cnt); zOut += cnt; zDelta += cnt; lenDelta -= cnt; break; } case ';': { zDelta++; lenDelta--; zOut[0] = 0; #ifdef FOSSIL_ENABLE_DELTA_CKSUM_TEST if( cnt!=checksum(zOrigOut, total) ){ /* ERROR: bad checksum */ return -1; } #endif if( total!=limit ){ /* ERROR: generated size does not match predicted size */ return -1; } return total; } default: { /* ERROR: unknown delta operator */ return -1; } } } /* ERROR: unterminated delta */ return -1; } /* ** SQL functions: delta_create(X,Y) ** ** Return a delta for carrying X into Y. */ static void deltaCreateFunc( sqlite3_context *context, int argc, sqlite3_value **argv ){ const char *aOrig; int nOrig; /* old blob */ const char *aNew; int nNew; /* new blob */ char *aOut; int nOut; /* output delta */ assert( argc==2 ); if( sqlite3_value_type(argv[0])==SQLITE_NULL ) return; if( sqlite3_value_type(argv[1])==SQLITE_NULL ) return; nOrig = sqlite3_value_bytes(argv[0]); aOrig = (const char*)sqlite3_value_blob(argv[0]); nNew = sqlite3_value_bytes(argv[1]); aNew = (const char*)sqlite3_value_blob(argv[1]); aOut = sqlite3_malloc64(nNew+70); if( aOut==0 ){ sqlite3_result_error_nomem(context); }else{ nOut = delta_create(aOrig, nOrig, aNew, nNew, aOut); if( nOut<0 ){ sqlite3_free(aOut); sqlite3_result_error(context, "cannot create fossil delta", -1); }else{ sqlite3_result_blob(context, aOut, nOut, sqlite3_free); } } } /* ** SQL functions: delta_apply(X,D) ** ** Return the result of applying delta D to input X. */ static void deltaApplyFunc( sqlite3_context *context, int argc, sqlite3_value **argv ){ const char *aOrig; int nOrig; /* The X input */ const char *aDelta; int nDelta; /* The input delta (D) */ char *aOut; int nOut, nOut2; /* The output */ assert( argc==2 ); if( sqlite3_value_type(argv[0])==SQLITE_NULL ) return; if( sqlite3_value_type(argv[1])==SQLITE_NULL ) return; nOrig = sqlite3_value_bytes(argv[0]); aOrig = (const char*)sqlite3_value_blob(argv[0]); nDelta = sqlite3_value_bytes(argv[1]); aDelta = (const char*)sqlite3_value_blob(argv[1]); /* Figure out the size of the output */ nOut = delta_output_size(aDelta, nDelta); if( nOut<0 ){ sqlite3_result_error(context, "corrupt fossil delta", -1); return; } aOut = sqlite3_malloc64((sqlite3_int64)nOut+1); if( aOut==0 ){ sqlite3_result_error_nomem(context); }else{ nOut2 = delta_apply(aOrig, nOrig, aDelta, nDelta, aOut); if( nOut2!=nOut ){ sqlite3_free(aOut); sqlite3_result_error(context, "corrupt fossil delta", -1); }else{ sqlite3_result_blob(context, aOut, nOut, sqlite3_free); } } } /* ** SQL functions: delta_output_size(D) ** ** Return the size of the output that results from applying delta D. */ static void deltaOutputSizeFunc( sqlite3_context *context, int argc, sqlite3_value **argv ){ const char *aDelta; int nDelta; /* The input delta (D) */ int nOut; /* Size of output */ assert( argc==1 ); if( sqlite3_value_type(argv[0])==SQLITE_NULL ) return; nDelta = sqlite3_value_bytes(argv[0]); aDelta = (const char*)sqlite3_value_blob(argv[0]); /* Figure out the size of the output */ nOut = delta_output_size(aDelta, nDelta); if( nOut<0 ){ sqlite3_result_error(context, "corrupt fossil delta", -1); return; }else{ sqlite3_result_int(context, nOut); } } /***************************************************************************** ** Table-valued SQL function: delta_parse(DELTA) ** ** Schema: ** ** CREATE TABLE delta_parse( ** op TEXT, ** a1 INT, ** a2 ANY, ** delta HIDDEN BLOB ** ); ** ** Given an input DELTA, this function parses the delta and returns ** rows for each entry in the delta. The op column has one of the ** values SIZE, COPY, INSERT, CHECKSUM, ERROR. ** ** Assuming no errors, the first row has op='SIZE'. a1 is the size of ** the output in bytes and a2 is NULL. ** ** After the initial SIZE row, there are zero or more 'COPY' and/or 'INSERT' ** rows. A COPY row means content is copied from the source into the ** output. Column a1 is the number of bytes to copy and a2 is the offset ** into source from which to begin copying. An INSERT row means to ** insert text into the output stream. Column a1 is the number of bytes ** to insert and column is a BLOB that contains the text to be inserted. ** ** The last row of a well-formed delta will have an op value of 'CHECKSUM'. ** The a1 column will be the value of the checksum and a2 will be NULL. ** ** If the input delta is not well-formed, then a row with an op value ** of 'ERROR' is returned. The a1 value of the ERROR row is the offset ** into the delta where the error was encountered and a2 is NULL. */ typedef struct deltaparsevtab_vtab deltaparsevtab_vtab; typedef struct deltaparsevtab_cursor deltaparsevtab_cursor; struct deltaparsevtab_vtab { sqlite3_vtab base; /* Base class - must be first */ /* No additional information needed */ }; struct deltaparsevtab_cursor { sqlite3_vtab_cursor base; /* Base class - must be first */ char *aDelta; /* The delta being parsed */ int nDelta; /* Number of bytes in the delta */ int iCursor; /* Current cursor location */ int eOp; /* Name of current operator */ unsigned int a1, a2; /* Arguments to current operator */ int iNext; /* Next cursor value */ }; /* Operator names: */ static const char *azOp[] = { "SIZE", "COPY", "INSERT", "CHECKSUM", "ERROR", "EOF" }; #define DELTAPARSE_OP_SIZE 0 #define DELTAPARSE_OP_COPY 1 #define DELTAPARSE_OP_INSERT 2 #define DELTAPARSE_OP_CHECKSUM 3 #define DELTAPARSE_OP_ERROR 4 #define DELTAPARSE_OP_EOF 5 /* ** The deltaparsevtabConnect() method is invoked to create a new ** deltaparse virtual table. ** ** Think of this routine as the constructor for deltaparsevtab_vtab objects. ** ** All this routine needs to do is: ** ** (1) Allocate the deltaparsevtab_vtab object and initialize all fields. ** ** (2) Tell SQLite (via the sqlite3_declare_vtab() interface) what the ** result set of queries against the virtual table will look like. */ static int deltaparsevtabConnect( sqlite3 *db, void *pAux, int argc, const char *const*argv, sqlite3_vtab **ppVtab, char **pzErr ){ deltaparsevtab_vtab *pNew; int rc; rc = sqlite3_declare_vtab(db, "CREATE TABLE x(op,a1,a2,delta HIDDEN)" ); /* For convenience, define symbolic names for the index to each column. */ #define DELTAPARSEVTAB_OP 0 #define DELTAPARSEVTAB_A1 1 #define DELTAPARSEVTAB_A2 2 #define DELTAPARSEVTAB_DELTA 3 if( rc==SQLITE_OK ){ pNew = sqlite3_malloc64( sizeof(*pNew) ); *ppVtab = (sqlite3_vtab*)pNew; if( pNew==0 ) return SQLITE_NOMEM; memset(pNew, 0, sizeof(*pNew)); sqlite3_vtab_config(db, SQLITE_VTAB_INNOCUOUS); } return rc; } /* ** This method is the destructor for deltaparsevtab_vtab objects. */ static int deltaparsevtabDisconnect(sqlite3_vtab *pVtab){ deltaparsevtab_vtab *p = (deltaparsevtab_vtab*)pVtab; sqlite3_free(p); return SQLITE_OK; } /* ** Constructor for a new deltaparsevtab_cursor object. */ static int deltaparsevtabOpen(sqlite3_vtab *p, sqlite3_vtab_cursor **ppCursor){ deltaparsevtab_cursor *pCur; pCur = sqlite3_malloc( sizeof(*pCur) ); if( pCur==0 ) return SQLITE_NOMEM; memset(pCur, 0, sizeof(*pCur)); *ppCursor = &pCur->base; return SQLITE_OK; } /* ** Destructor for a deltaparsevtab_cursor. */ static int deltaparsevtabClose(sqlite3_vtab_cursor *cur){ deltaparsevtab_cursor *pCur = (deltaparsevtab_cursor*)cur; sqlite3_free(pCur->aDelta); sqlite3_free(pCur); return SQLITE_OK; } /* ** Advance a deltaparsevtab_cursor to its next row of output. */ static int deltaparsevtabNext(sqlite3_vtab_cursor *cur){ deltaparsevtab_cursor *pCur = (deltaparsevtab_cursor*)cur; const char *z; int i = 0; pCur->iCursor = pCur->iNext; z = pCur->aDelta + pCur->iCursor; pCur->a1 = deltaGetInt(&z, &i); switch( z[0] ){ case '@': { z++; pCur->a2 = deltaGetInt(&z, &i); pCur->eOp = DELTAPARSE_OP_COPY; pCur->iNext = (int)(&z[1] - pCur->aDelta); break; } case ':': { z++; pCur->a2 = (unsigned int)(z - pCur->aDelta); pCur->eOp = DELTAPARSE_OP_INSERT; pCur->iNext = (int)(&z[pCur->a1] - pCur->aDelta); break; } case ';': { pCur->eOp = DELTAPARSE_OP_CHECKSUM; pCur->iNext = pCur->nDelta; break; } default: { if( pCur->iNext==pCur->nDelta ){ pCur->eOp = DELTAPARSE_OP_EOF; }else{ pCur->eOp = DELTAPARSE_OP_ERROR; pCur->iNext = pCur->nDelta; } break; } } return SQLITE_OK; } /* ** Return values of columns for the row at which the deltaparsevtab_cursor ** is currently pointing. */ static int deltaparsevtabColumn( sqlite3_vtab_cursor *cur, /* The cursor */ sqlite3_context *ctx, /* First argument to sqlite3_result_...() */ int i /* Which column to return */ ){ deltaparsevtab_cursor *pCur = (deltaparsevtab_cursor*)cur; switch( i ){ case DELTAPARSEVTAB_OP: { sqlite3_result_text(ctx, azOp[pCur->eOp], -1, SQLITE_STATIC); break; } case DELTAPARSEVTAB_A1: { sqlite3_result_int(ctx, pCur->a1); break; } case DELTAPARSEVTAB_A2: { if( pCur->eOp==DELTAPARSE_OP_COPY ){ sqlite3_result_int(ctx, pCur->a2); }else if( pCur->eOp==DELTAPARSE_OP_INSERT ){ sqlite3_result_blob(ctx, pCur->aDelta+pCur->a2, pCur->a1, SQLITE_TRANSIENT); } break; } case DELTAPARSEVTAB_DELTA: { sqlite3_result_blob(ctx, pCur->aDelta, pCur->nDelta, SQLITE_TRANSIENT); break; } } return SQLITE_OK; } /* ** Return the rowid for the current row. In this implementation, the ** rowid is the same as the output value. */ static int deltaparsevtabRowid(sqlite3_vtab_cursor *cur, sqlite_int64 *pRowid){ deltaparsevtab_cursor *pCur = (deltaparsevtab_cursor*)cur; *pRowid = pCur->iCursor; return SQLITE_OK; } /* ** Return TRUE if the cursor has been moved off of the last ** row of output. */ static int deltaparsevtabEof(sqlite3_vtab_cursor *cur){ deltaparsevtab_cursor *pCur = (deltaparsevtab_cursor*)cur; return pCur->eOp==DELTAPARSE_OP_EOF; } /* ** This method is called to "rewind" the deltaparsevtab_cursor object back ** to the first row of output. This method is always called at least ** once prior to any call to deltaparsevtabColumn() or deltaparsevtabRowid() or ** deltaparsevtabEof(). */ static int deltaparsevtabFilter( sqlite3_vtab_cursor *pVtabCursor, int idxNum, const char *idxStr, int argc, sqlite3_value **argv ){ deltaparsevtab_cursor *pCur = (deltaparsevtab_cursor *)pVtabCursor; const char *a; int i = 0; pCur->eOp = DELTAPARSE_OP_ERROR; if( idxNum!=1 ){ return SQLITE_OK; } pCur->nDelta = sqlite3_value_bytes(argv[0]); a = (const char*)sqlite3_value_blob(argv[0]); if( pCur->nDelta==0 || a==0 ){ return SQLITE_OK; } pCur->aDelta = sqlite3_malloc64( pCur->nDelta+1 ); if( pCur->aDelta==0 ){ pCur->nDelta = 0; return SQLITE_NOMEM; } memcpy(pCur->aDelta, a, pCur->nDelta); pCur->aDelta[pCur->nDelta] = 0; a = pCur->aDelta; pCur->eOp = DELTAPARSE_OP_SIZE; pCur->a1 = deltaGetInt(&a, &i); if( a[0]!='\n' ){ pCur->eOp = DELTAPARSE_OP_ERROR; pCur->a1 = pCur->a2 = 0; pCur->iNext = pCur->nDelta; return SQLITE_OK; } a++; pCur->iNext = (unsigned int)(a - pCur->aDelta); return SQLITE_OK; } /* ** SQLite will invoke this method one or more times while planning a query ** that uses the virtual table. This routine needs to create ** a query plan for each invocation and compute an estimated cost for that ** plan. */ static int deltaparsevtabBestIndex( sqlite3_vtab *tab, sqlite3_index_info *pIdxInfo ){ int i; for(i=0; i<pIdxInfo->nConstraint; i++){ if( pIdxInfo->aConstraint[i].iColumn != DELTAPARSEVTAB_DELTA ) continue; if( pIdxInfo->aConstraint[i].usable==0 ) continue; if( pIdxInfo->aConstraint[i].op!=SQLITE_INDEX_CONSTRAINT_EQ ) continue; pIdxInfo->aConstraintUsage[i].argvIndex = 1; pIdxInfo->aConstraintUsage[i].omit = 1; pIdxInfo->estimatedCost = (double)1; pIdxInfo->estimatedRows = 10; pIdxInfo->idxNum = 1; return SQLITE_OK; } pIdxInfo->idxNum = 0; pIdxInfo->estimatedCost = (double)0x7fffffff; pIdxInfo->estimatedRows = 0x7fffffff; return SQLITE_CONSTRAINT; } /* ** This following structure defines all the methods for the ** virtual table. */ static sqlite3_module deltaparsevtabModule = { /* iVersion */ 0, /* xCreate */ 0, /* xConnect */ deltaparsevtabConnect, /* xBestIndex */ deltaparsevtabBestIndex, /* xDisconnect */ deltaparsevtabDisconnect, /* xDestroy */ 0, /* xOpen */ deltaparsevtabOpen, /* xClose */ deltaparsevtabClose, /* xFilter */ deltaparsevtabFilter, /* xNext */ deltaparsevtabNext, /* xEof */ deltaparsevtabEof, /* xColumn */ deltaparsevtabColumn, /* xRowid */ deltaparsevtabRowid, /* xUpdate */ 0, /* xBegin */ 0, /* xSync */ 0, /* xCommit */ 0, /* xRollback */ 0, /* xFindMethod */ 0, /* xRename */ 0, /* xSavepoint */ 0, /* xRelease */ 0, /* xRollbackTo */ 0, /* xShadowName */ 0 }; #ifdef _WIN32 __declspec(dllexport) #endif int sqlite3_fossildelta_init( sqlite3 *db, char **pzErrMsg, const sqlite3_api_routines *pApi ){ static const int enc = SQLITE_UTF8|SQLITE_INNOCUOUS; int rc = SQLITE_OK; SQLITE_EXTENSION_INIT2(pApi); (void)pzErrMsg; /* Unused parameter */ rc = sqlite3_create_function(db, "delta_create", 2, enc, 0, deltaCreateFunc, 0, 0); if( rc==SQLITE_OK ){ rc = sqlite3_create_function(db, "delta_apply", 2, enc, 0, deltaApplyFunc, 0, 0); } if( rc==SQLITE_OK ){ rc = sqlite3_create_function(db, "delta_output_size", 1, enc, 0, deltaOutputSizeFunc, 0, 0); } if( rc==SQLITE_OK ){ rc = sqlite3_create_module(db, "delta_parse", &deltaparsevtabModule, 0); } return rc; } |
Changes to ext/misc/fuzzer.c.
︙ | ︙ | |||
333 334 335 336 337 338 339 | if( iRuleset<0 || iRuleset>FUZZER_MX_RULEID ){ *pzErr = sqlite3_mprintf("%s: ruleset must be between 0 and %d", p->zClassName, FUZZER_MX_RULEID ); rc = SQLITE_ERROR; }else{ | | | 333 334 335 336 337 338 339 340 341 342 343 344 345 346 347 | if( iRuleset<0 || iRuleset>FUZZER_MX_RULEID ){ *pzErr = sqlite3_mprintf("%s: ruleset must be between 0 and %d", p->zClassName, FUZZER_MX_RULEID ); rc = SQLITE_ERROR; }else{ pRule = sqlite3_malloc64( sizeof(*pRule) + nFrom + nTo ); if( pRule==0 ){ rc = SQLITE_NOMEM; }else{ memset(pRule, 0, sizeof(*pRule)); pRule->zFrom = pRule->zTo; pRule->zFrom += nTo + 1; pRule->nFrom = (fuzzer_len)nFrom; |
︙ | ︙ | |||
443 444 445 446 447 448 449 | ** ** "abc" becomes abc ** 'xyz' becomes xyz ** [pqr] becomes pqr ** `mno` becomes mno */ static char *fuzzerDequote(const char *zIn){ | | | | | | 443 444 445 446 447 448 449 450 451 452 453 454 455 456 457 458 459 460 461 462 463 464 465 466 | ** ** "abc" becomes abc ** 'xyz' becomes xyz ** [pqr] becomes pqr ** `mno` becomes mno */ static char *fuzzerDequote(const char *zIn){ sqlite3_int64 nIn; /* Size of input string, in bytes */ char *zOut; /* Output (dequoted) string */ nIn = strlen(zIn); zOut = sqlite3_malloc64(nIn+1); if( zOut ){ char q = zIn[0]; /* Quote character (if any ) */ if( q!='[' && q!= '\'' && q!='"' && q!='`' ){ memcpy(zOut, zIn, (size_t)(nIn+1)); }else{ int iOut = 0; /* Index of next byte to write to output */ int iIn; /* Index of next byte to read from input */ if( q=='[' ) q = ']'; for(iIn=1; iIn<nIn; iIn++){ if( zIn[iIn]==q ) iIn++; |
︙ | ︙ | |||
509 510 511 512 513 514 515 | if( argc!=4 ){ *pzErr = sqlite3_mprintf( "%s: wrong number of CREATE VIRTUAL TABLE arguments", zModule ); rc = SQLITE_ERROR; }else{ | | | | | > > | 509 510 511 512 513 514 515 516 517 518 519 520 521 522 523 524 525 526 527 528 529 530 531 532 533 534 535 536 537 538 539 540 541 542 543 544 545 546 547 548 549 550 551 | if( argc!=4 ){ *pzErr = sqlite3_mprintf( "%s: wrong number of CREATE VIRTUAL TABLE arguments", zModule ); rc = SQLITE_ERROR; }else{ sqlite3_int64 nModule; /* Length of zModule, in bytes */ nModule = strlen(zModule); pNew = sqlite3_malloc64( sizeof(*pNew) + nModule + 1); if( pNew==0 ){ rc = SQLITE_NOMEM; }else{ char *zTab; /* Dequoted name of fuzzer data table */ memset(pNew, 0, sizeof(*pNew)); pNew->zClassName = (char*)&pNew[1]; memcpy(pNew->zClassName, zModule, (size_t)(nModule+1)); zTab = fuzzerDequote(argv[3]); if( zTab==0 ){ rc = SQLITE_NOMEM; }else{ rc = fuzzerLoadRules(db, pNew, zDb, zTab, pzErr); sqlite3_free(zTab); } if( rc==SQLITE_OK ){ rc = sqlite3_declare_vtab(db, "CREATE TABLE x(word,distance,ruleset)"); } if( rc!=SQLITE_OK ){ fuzzerDisconnect((sqlite3_vtab *)pNew); pNew = 0; }else{ sqlite3_vtab_config(db, SQLITE_VTAB_INNOCUOUS); } } } *ppVtab = (sqlite3_vtab *)pNew; return rc; } |
︙ | ︙ | |||
868 869 870 871 872 873 874 | const char *zWord, fuzzer_cost rBaseCost ){ fuzzer_stem *pNew; fuzzer_rule *pRule; unsigned int h; | | | 870 871 872 873 874 875 876 877 878 879 880 881 882 883 884 | const char *zWord, fuzzer_cost rBaseCost ){ fuzzer_stem *pNew; fuzzer_rule *pRule; unsigned int h; pNew = sqlite3_malloc64( sizeof(*pNew) + strlen(zWord) + 1 ); if( pNew==0 ) return 0; memset(pNew, 0, sizeof(*pNew)); pNew->zBasis = (char*)&pNew[1]; pNew->nBasis = (fuzzer_len)strlen(zWord); memcpy(pNew->zBasis, zWord, pNew->nBasis+1); pRule = pCur->pVtab->pRule; while( fuzzerSkipRule(pRule, pNew, pCur->iRuleset) ){ |
︙ | ︙ |
Changes to ext/misc/ieee754.c.
︙ | ︙ | |||
22 23 24 25 26 27 28 | ** ** In the second form, Y and Z are integers which are the mantissa and ** base-2 exponent of a new floating point number. The function returns ** a floating-point value equal to Y*pow(2,Z). ** ** Examples: ** | | | | | > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | > > > > > > > > > > | > > > > | > > > | | | > > > > > > > > | > > > > > > > > | | > > > > > > > | > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | < | < > | | 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 | ** ** In the second form, Y and Z are integers which are the mantissa and ** base-2 exponent of a new floating point number. The function returns ** a floating-point value equal to Y*pow(2,Z). ** ** Examples: ** ** ieee754(2.0) -> 'ieee754(2,0)' ** ieee754(45.25) -> 'ieee754(181,-2)' ** ieee754(2, 0) -> 2.0 ** ieee754(181, -2) -> 45.25 ** ** Two additional functions break apart the one-argument ieee754() ** result into separate integer values: ** ** ieee754_mantissa(45.25) -> 181 ** ieee754_exponent(45.25) -> -2 ** ** These functions convert binary64 numbers into blobs and back again. ** ** ieee754_from_blob(x'3ff0000000000000') -> 1.0 ** ieee754_to_blob(1.0) -> x'3ff0000000000000' ** ** In all single-argument functions, if the argument is an 8-byte blob ** then that blob is interpreted as a big-endian binary64 value. ** ** ** EXACT DECIMAL REPRESENTATION OF BINARY64 VALUES ** ----------------------------------------------- ** ** This extension in combination with the separate 'decimal' extension ** can be used to compute the exact decimal representation of binary64 ** values. To begin, first compute a table of exponent values: ** ** CREATE TABLE pow2(x INTEGER PRIMARY KEY, v TEXT); ** WITH RECURSIVE c(x,v) AS ( ** VALUES(0,'1') ** UNION ALL ** SELECT x+1, decimal_mul(v,'2') FROM c WHERE x+1<=971 ** ) INSERT INTO pow2(x,v) SELECT x, v FROM c; ** WITH RECURSIVE c(x,v) AS ( ** VALUES(-1,'0.5') ** UNION ALL ** SELECT x-1, decimal_mul(v,'0.5') FROM c WHERE x-1>=-1075 ** ) INSERT INTO pow2(x,v) SELECT x, v FROM c; ** ** Then, to compute the exact decimal representation of a floating ** point value (the value 47.49 is used in the example) do: ** ** WITH c(n) AS (VALUES(47.49)) ** ---------------^^^^^---- Replace with whatever you want ** SELECT decimal_mul(ieee754_mantissa(c.n),pow2.v) ** FROM pow2, c WHERE pow2.x=ieee754_exponent(c.n); ** ** Here is a query to show various boundry values for the binary64 ** number format: ** ** WITH c(name,bin) AS (VALUES ** ('minimum positive value', x'0000000000000001'), ** ('maximum subnormal value', x'000fffffffffffff'), ** ('mininum positive nornal value', x'0010000000000000'), ** ('maximum value', x'7fefffffffffffff')) ** SELECT c.name, decimal_mul(ieee754_mantissa(c.bin),pow2.v) ** FROM pow2, c WHERE pow2.x=ieee754_exponent(c.bin); ** */ #include "sqlite3ext.h" SQLITE_EXTENSION_INIT1 #include <assert.h> #include <string.h> /* Mark a function parameter as unused, to suppress nuisance compiler ** warnings. */ #ifndef UNUSED_PARAMETER # define UNUSED_PARAMETER(X) (void)(X) #endif /* ** Implementation of the ieee754() function */ static void ieee754func( sqlite3_context *context, int argc, sqlite3_value **argv ){ if( argc==1 ){ sqlite3_int64 m, a; double r; int e; int isNeg; char zResult[100]; assert( sizeof(m)==sizeof(r) ); if( sqlite3_value_type(argv[0])==SQLITE_BLOB && sqlite3_value_bytes(argv[0])==sizeof(r) ){ const unsigned char *x = sqlite3_value_blob(argv[0]); unsigned int i; sqlite3_uint64 v = 0; for(i=0; i<sizeof(r); i++){ v = (v<<8) | x[i]; } memcpy(&r, &v, sizeof(r)); }else{ r = sqlite3_value_double(argv[0]); } if( r<0.0 ){ isNeg = 1; r = -r; }else{ isNeg = 0; } memcpy(&a,&r,sizeof(a)); if( a==0 ){ e = 0; m = 0; }else{ e = a>>52; m = a & ((((sqlite3_int64)1)<<52)-1); if( e==0 ){ m <<= 1; }else{ m |= ((sqlite3_int64)1)<<52; } while( e<1075 && m>0 && (m&1)==0 ){ m >>= 1; e++; } if( isNeg ) m = -m; } switch( *(int*)sqlite3_user_data(context) ){ case 0: sqlite3_snprintf(sizeof(zResult), zResult, "ieee754(%lld,%d)", m, e-1075); sqlite3_result_text(context, zResult, -1, SQLITE_TRANSIENT); break; case 1: sqlite3_result_int64(context, m); break; case 2: sqlite3_result_int(context, e-1075); break; } }else{ sqlite3_int64 m, e, a; double r; int isNeg = 0; m = sqlite3_value_int64(argv[0]); e = sqlite3_value_int64(argv[1]); /* Limit the range of e. Ticket 22dea1cfdb9151e4 2021-03-02 */ if( e>10000 ){ e = 10000; }else if( e<-10000 ){ e = -10000; } if( m<0 ){ isNeg = 1; m = -m; if( m<0 ) return; }else if( m==0 && e>-1000 && e<1000 ){ sqlite3_result_double(context, 0.0); return; } while( (m>>32)&0xffe00000 ){ m >>= 1; e++; } while( m!=0 && ((m>>32)&0xfff00000)==0 ){ m <<= 1; e--; } e += 1075; if( e<=0 ){ /* Subnormal */ if( 1-e >= 64 ){ m = 0; }else{ m >>= 1-e; } e = 0; }else if( e>0x7ff ){ e = 0x7ff; } a = m & ((((sqlite3_int64)1)<<52)-1); a |= e<<52; if( isNeg ) a |= ((sqlite3_uint64)1)<<63; memcpy(&r, &a, sizeof(r)); sqlite3_result_double(context, r); } } /* ** Functions to convert between blobs and floats. */ static void ieee754func_from_blob( sqlite3_context *context, int argc, sqlite3_value **argv ){ UNUSED_PARAMETER(argc); if( sqlite3_value_type(argv[0])==SQLITE_BLOB && sqlite3_value_bytes(argv[0])==sizeof(double) ){ double r; const unsigned char *x = sqlite3_value_blob(argv[0]); unsigned int i; sqlite3_uint64 v = 0; for(i=0; i<sizeof(r); i++){ v = (v<<8) | x[i]; } memcpy(&r, &v, sizeof(r)); sqlite3_result_double(context, r); } } static void ieee754func_to_blob( sqlite3_context *context, int argc, sqlite3_value **argv ){ UNUSED_PARAMETER(argc); if( sqlite3_value_type(argv[0])==SQLITE_FLOAT || sqlite3_value_type(argv[0])==SQLITE_INTEGER ){ double r = sqlite3_value_double(argv[0]); sqlite3_uint64 v; unsigned char a[sizeof(r)]; unsigned int i; memcpy(&v, &r, sizeof(r)); for(i=1; i<=sizeof(r); i++){ a[sizeof(r)-i] = v&0xff; v >>= 8; } sqlite3_result_blob(context, a, sizeof(r), SQLITE_TRANSIENT); } } #ifdef _WIN32 __declspec(dllexport) #endif int sqlite3_ieee_init( sqlite3 *db, char **pzErrMsg, const sqlite3_api_routines *pApi ){ static const struct { char *zFName; int nArg; int iAux; void (*xFunc)(sqlite3_context*,int,sqlite3_value**); } aFunc[] = { { "ieee754", 1, 0, ieee754func }, { "ieee754", 2, 0, ieee754func }, { "ieee754_mantissa", 1, 1, ieee754func }, { "ieee754_exponent", 1, 2, ieee754func }, { "ieee754_to_blob", 1, 0, ieee754func_to_blob }, { "ieee754_from_blob", 1, 0, ieee754func_from_blob }, }; unsigned int i; int rc = SQLITE_OK; SQLITE_EXTENSION_INIT2(pApi); (void)pzErrMsg; /* Unused parameter */ for(i=0; i<sizeof(aFunc)/sizeof(aFunc[0]) && rc==SQLITE_OK; i++){ rc = sqlite3_create_function(db, aFunc[i].zFName, aFunc[i].nArg, SQLITE_UTF8|SQLITE_INNOCUOUS, (void*)&aFunc[i].iAux, aFunc[i].xFunc, 0, 0); } return rc; } |
Deleted ext/misc/json1.c.
|
| < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < |
Changes to ext/misc/memstat.c.
︙ | ︙ | |||
139 140 141 142 143 144 145 | rc = sqlite3_prepare_v2(pCur->db, "PRAGMA database_list", -1, &pStmt, 0); if( rc ){ sqlite3_finalize(pStmt); return rc; } while( sqlite3_step(pStmt)==SQLITE_ROW ){ char **az, *z; | | | 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 | rc = sqlite3_prepare_v2(pCur->db, "PRAGMA database_list", -1, &pStmt, 0); if( rc ){ sqlite3_finalize(pStmt); return rc; } while( sqlite3_step(pStmt)==SQLITE_ROW ){ char **az, *z; az = sqlite3_realloc64(pCur->azDb, sizeof(char*)*(pCur->nDb+1)); if( az==0 ){ memstatClearSchema(pCur); return SQLITE_NOMEM; } pCur->azDb = az; z = sqlite3_mprintf("%s", sqlite3_column_text(pStmt, 1)); if( z==0 ){ |
︙ | ︙ |
Added ext/misc/memtrace.c.
> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 | /* ** 2019-01-21 ** ** The author disclaims copyright to this source code. In place of ** a legal notice, here is a blessing: ** ** May you do good and not evil. ** May you find forgiveness for yourself and forgive others. ** May you share freely, never taking more than you give. ** ************************************************************************* ** ** This file implements an extension that uses the SQLITE_CONFIG_MALLOC ** mechanism to add a tracing layer on top of SQLite. If this extension ** is registered prior to sqlite3_initialize(), it will cause all memory ** allocation activities to be logged on standard output, or to some other ** FILE specified by the initializer. ** ** This file needs to be compiled into the application that uses it. ** ** This extension is used to implement the --memtrace option of the ** command-line shell. */ #include <assert.h> #include <string.h> #include <stdio.h> /* The original memory allocation routines */ static sqlite3_mem_methods memtraceBase; static FILE *memtraceOut; /* Methods that trace memory allocations */ static void *memtraceMalloc(int n){ if( memtraceOut ){ fprintf(memtraceOut, "MEMTRACE: allocate %d bytes\n", memtraceBase.xRoundup(n)); } return memtraceBase.xMalloc(n); } static void memtraceFree(void *p){ if( p==0 ) return; if( memtraceOut ){ fprintf(memtraceOut, "MEMTRACE: free %d bytes\n", memtraceBase.xSize(p)); } memtraceBase.xFree(p); } static void *memtraceRealloc(void *p, int n){ if( p==0 ) return memtraceMalloc(n); if( n==0 ){ memtraceFree(p); return 0; } if( memtraceOut ){ fprintf(memtraceOut, "MEMTRACE: resize %d -> %d bytes\n", memtraceBase.xSize(p), memtraceBase.xRoundup(n)); } return memtraceBase.xRealloc(p, n); } static int memtraceSize(void *p){ return memtraceBase.xSize(p); } static int memtraceRoundup(int n){ return memtraceBase.xRoundup(n); } static int memtraceInit(void *p){ return memtraceBase.xInit(p); } static void memtraceShutdown(void *p){ memtraceBase.xShutdown(p); } /* The substitute memory allocator */ static sqlite3_mem_methods ersaztMethods = { memtraceMalloc, memtraceFree, memtraceRealloc, memtraceSize, memtraceRoundup, memtraceInit, memtraceShutdown, 0 }; /* Begin tracing memory allocations to out. */ int sqlite3MemTraceActivate(FILE *out){ int rc = SQLITE_OK; if( memtraceBase.xMalloc==0 ){ rc = sqlite3_config(SQLITE_CONFIG_GETMALLOC, &memtraceBase); if( rc==SQLITE_OK ){ rc = sqlite3_config(SQLITE_CONFIG_MALLOC, &ersaztMethods); } } memtraceOut = out; return rc; } /* Deactivate memory tracing */ int sqlite3MemTraceDeactivate(void){ int rc = SQLITE_OK; if( memtraceBase.xMalloc!=0 ){ rc = sqlite3_config(SQLITE_CONFIG_MALLOC, &memtraceBase); if( rc==SQLITE_OK ){ memset(&memtraceBase, 0, sizeof(memtraceBase)); } } memtraceOut = 0; return rc; } |
Changes to ext/misc/memvfs.c.
︙ | ︙ | |||
555 556 557 558 559 560 561 562 563 564 565 566 567 568 | sqlite3 *db, char **pzErrMsg, const sqlite3_api_routines *pApi ){ int rc = SQLITE_OK; SQLITE_EXTENSION_INIT2(pApi); mem_vfs.pAppData = sqlite3_vfs_find(0); mem_vfs.szOsFile = sizeof(MemFile); rc = sqlite3_vfs_register(&mem_vfs, 1); #ifdef MEMVFS_TEST if( rc==SQLITE_OK ){ rc = sqlite3_auto_extension((void(*)(void))memvfsRegister); } if( rc==SQLITE_OK ){ | > | 555 556 557 558 559 560 561 562 563 564 565 566 567 568 569 | sqlite3 *db, char **pzErrMsg, const sqlite3_api_routines *pApi ){ int rc = SQLITE_OK; SQLITE_EXTENSION_INIT2(pApi); mem_vfs.pAppData = sqlite3_vfs_find(0); if( mem_vfs.pAppData==0 ) return SQLITE_ERROR; mem_vfs.szOsFile = sizeof(MemFile); rc = sqlite3_vfs_register(&mem_vfs, 1); #ifdef MEMVFS_TEST if( rc==SQLITE_OK ){ rc = sqlite3_auto_extension((void(*)(void))memvfsRegister); } if( rc==SQLITE_OK ){ |
︙ | ︙ |
Changes to ext/misc/mmapwarm.c.
︙ | ︙ | |||
39 40 41 42 43 44 45 | char *zSql = 0; int pgsz = 0; int nTotal = 0; if( 0==sqlite3_get_autocommit(db) ) return SQLITE_MISUSE; /* Open a read-only transaction on the file in question */ | | | 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 | char *zSql = 0; int pgsz = 0; int nTotal = 0; if( 0==sqlite3_get_autocommit(db) ) return SQLITE_MISUSE; /* Open a read-only transaction on the file in question */ zSql = sqlite3_mprintf("BEGIN; SELECT * FROM %s%q%ssqlite_schema", (zDb ? "'" : ""), (zDb ? zDb : ""), (zDb ? "'." : "") ); if( zSql==0 ) return SQLITE_NOMEM; rc = sqlite3_exec(db, zSql, 0, 0, 0); sqlite3_free(zSql); /* Find the SQLite page size of the file */ |
︙ | ︙ | |||
101 102 103 104 105 106 107 | rc2 = sqlite3_exec(db, "END", 0, 0, 0); if( rc==SQLITE_OK ) rc = rc2; } return rc; } | < | 101 102 103 104 105 106 107 | rc2 = sqlite3_exec(db, "END", 0, 0, 0); if( rc==SQLITE_OK ) rc = rc2; } return rc; } |
Changes to ext/misc/nextchar.c.
︙ | ︙ | |||
81 82 83 84 85 86 87 | int i; for(i=0; i<p->nUsed; i++){ if( p->aResult[i]==c ) return; } if( p->nUsed+1 > p->nAlloc ){ unsigned int *aNew; int n = p->nAlloc*2 + 30; | | | 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 | int i; for(i=0; i<p->nUsed; i++){ if( p->aResult[i]==c ) return; } if( p->nUsed+1 > p->nAlloc ){ unsigned int *aNew; int n = p->nAlloc*2 + 30; aNew = sqlite3_realloc64(p->aResult, n*sizeof(unsigned int)); if( aNew==0 ){ p->mallocFailed = 1; return; }else{ p->aResult = aNew; p->nAlloc = n; } |
︙ | ︙ | |||
265 266 267 268 269 270 271 | return; } findNextChars(&c); if( c.mallocFailed ){ sqlite3_result_error_nomem(context); }else{ unsigned char *pRes; | | | 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 | return; } findNextChars(&c); if( c.mallocFailed ){ sqlite3_result_error_nomem(context); }else{ unsigned char *pRes; pRes = sqlite3_malloc64( c.nUsed*4 + 1 ); if( pRes==0 ){ sqlite3_result_error_nomem(context); }else{ int i; int n = 0; for(i=0; i<c.nUsed; i++){ n += writeUtf8(pRes+n, c.aResult[i]); |
︙ | ︙ | |||
293 294 295 296 297 298 299 | sqlite3 *db, char **pzErrMsg, const sqlite3_api_routines *pApi ){ int rc = SQLITE_OK; SQLITE_EXTENSION_INIT2(pApi); (void)pzErrMsg; /* Unused parameter */ | | > | > | > | 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 | sqlite3 *db, char **pzErrMsg, const sqlite3_api_routines *pApi ){ int rc = SQLITE_OK; SQLITE_EXTENSION_INIT2(pApi); (void)pzErrMsg; /* Unused parameter */ rc = sqlite3_create_function(db, "next_char", 3, SQLITE_UTF8|SQLITE_INNOCUOUS, 0, nextCharFunc, 0, 0); if( rc==SQLITE_OK ){ rc = sqlite3_create_function(db, "next_char", 4, SQLITE_UTF8|SQLITE_INNOCUOUS, 0, nextCharFunc, 0, 0); } if( rc==SQLITE_OK ){ rc = sqlite3_create_function(db, "next_char", 5, SQLITE_UTF8|SQLITE_INNOCUOUS, 0, nextCharFunc, 0, 0); } return rc; } |
Added ext/misc/noop.c.
> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 | /* ** 2020-01-08 ** ** The author disclaims copyright to this source code. In place of ** a legal notice, here is a blessing: ** ** May you do good and not evil. ** May you find forgiveness for yourself and forgive others. ** May you share freely, never taking more than you give. ** ****************************************************************************** ** ** This SQLite extension implements a noop() function used for testing. ** ** Variants: ** ** noop(X) The default. Deterministic. ** noop_i(X) Deterministic and innocuous. ** noop_do(X) Deterministic and direct-only. ** noop_nd(X) Non-deterministic. */ #include "sqlite3ext.h" SQLITE_EXTENSION_INIT1 #include <assert.h> #include <string.h> /* ** Implementation of the noop() function. ** ** The function returns its argument, unchanged. */ static void noopfunc( sqlite3_context *context, int argc, sqlite3_value **argv ){ assert( argc==1 ); sqlite3_result_value(context, argv[0]); } #ifdef _WIN32 __declspec(dllexport) #endif int sqlite3_noop_init( sqlite3 *db, char **pzErrMsg, const sqlite3_api_routines *pApi ){ int rc = SQLITE_OK; SQLITE_EXTENSION_INIT2(pApi); (void)pzErrMsg; /* Unused parameter */ rc = sqlite3_create_function(db, "noop", 1, SQLITE_UTF8 | SQLITE_DETERMINISTIC, 0, noopfunc, 0, 0); if( rc ) return rc; rc = sqlite3_create_function(db, "noop_i", 1, SQLITE_UTF8 | SQLITE_DETERMINISTIC | SQLITE_INNOCUOUS, 0, noopfunc, 0, 0); if( rc ) return rc; rc = sqlite3_create_function(db, "noop_do", 1, SQLITE_UTF8 | SQLITE_DETERMINISTIC | SQLITE_DIRECTONLY, 0, noopfunc, 0, 0); if( rc ) return rc; rc = sqlite3_create_function(db, "noop_nd", 1, SQLITE_UTF8, 0, noopfunc, 0, 0); return rc; } |
Changes to ext/misc/normalize.c.
︙ | ︙ | |||
282 283 284 285 286 287 288 289 290 291 292 293 294 295 | #define TK_ILLEGAL TK_ERROR #define TK_DOT TK_PUNCT #define TK_INTEGER TK_LITERAL #define TK_FLOAT TK_LITERAL #define TK_VARIABLE TK_LITERAL #define TK_BLOB TK_LITERAL /* ** Return the length (in bytes) of the token that begins at z[0]. ** Store the token type in *tokenType before returning. */ static int sqlite3GetToken(const unsigned char *z, int *tokenType){ int i, c; switch( aiClass[*z] ){ /* Switch on the character-class of the first byte | > > > > > > > | 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 | #define TK_ILLEGAL TK_ERROR #define TK_DOT TK_PUNCT #define TK_INTEGER TK_LITERAL #define TK_FLOAT TK_LITERAL #define TK_VARIABLE TK_LITERAL #define TK_BLOB TK_LITERAL /* Disable nuisence warnings about case fall-through */ #if !defined(deliberate_fall_through) && defined(__GCC__) && __GCC__>=7 # define deliberate_fall_through __attribute__((fallthrough)); #else # define deliberate_fall_through #endif /* ** Return the length (in bytes) of the token that begins at z[0]. ** Store the token type in *tokenType before returning. */ static int sqlite3GetToken(const unsigned char *z, int *tokenType){ int i, c; switch( aiClass[*z] ){ /* Switch on the character-class of the first byte |
︙ | ︙ | |||
432 433 434 435 436 437 438 439 440 441 442 443 444 445 | case CC_DOT: { if( !sqlite3Isdigit(z[1]) ){ *tokenType = TK_DOT; return 1; } /* If the next character is a digit, this is a floating point ** number that begins with ".". Fall thru into the next case */ } case CC_DIGIT: { *tokenType = TK_INTEGER; if( z[0]=='0' && (z[1]=='x' || z[1]=='X') && sqlite3Isxdigit(z[2]) ){ for(i=3; sqlite3Isxdigit(z[i]); i++){} return i; } | > | 439 440 441 442 443 444 445 446 447 448 449 450 451 452 453 | case CC_DOT: { if( !sqlite3Isdigit(z[1]) ){ *tokenType = TK_DOT; return 1; } /* If the next character is a digit, this is a floating point ** number that begins with ".". Fall thru into the next case */ /* no break */ deliberate_fall_through } case CC_DIGIT: { *tokenType = TK_INTEGER; if( z[0]=='0' && (z[1]=='x' || z[1]=='X') && sqlite3Isxdigit(z[2]) ){ for(i=3; sqlite3Isxdigit(z[i]); i++){} return i; } |
︙ | ︙ | |||
524 525 526 527 528 529 530 531 532 533 534 535 536 537 | while( z[i] && z[i]!='\'' ){ i++; } } if( z[i] ) i++; return i; } /* If it is not a BLOB literal, then it must be an ID, since no ** SQL keywords start with the letter 'x'. Fall through */ } case CC_ID: { i = 1; break; } default: { *tokenType = TK_ILLEGAL; | > | 532 533 534 535 536 537 538 539 540 541 542 543 544 545 546 | while( z[i] && z[i]!='\'' ){ i++; } } if( z[i] ) i++; return i; } /* If it is not a BLOB literal, then it must be an ID, since no ** SQL keywords start with the letter 'x'. Fall through */ /* no break */ deliberate_fall_through } case CC_ID: { i = 1; break; } default: { *tokenType = TK_ILLEGAL; |
︙ | ︙ |
Changes to ext/misc/percentile.c.
︙ | ︙ | |||
104 105 106 107 108 109 110 | int eType; double y; assert( argc==2 ); /* Requirement 3: P must be a number between 0 and 100 */ eType = sqlite3_value_numeric_type(argv[1]); rPct = sqlite3_value_double(argv[1]); | | | | 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 | int eType; double y; assert( argc==2 ); /* Requirement 3: P must be a number between 0 and 100 */ eType = sqlite3_value_numeric_type(argv[1]); rPct = sqlite3_value_double(argv[1]); if( (eType!=SQLITE_INTEGER && eType!=SQLITE_FLOAT) || rPct<0.0 || rPct>100.0 ){ sqlite3_result_error(pCtx, "2nd argument to percentile() is not " "a number between 0.0 and 100.0", -1); return; } /* Allocate the session context. */ p = (Percentile*)sqlite3_aggregate_context(pCtx, sizeof(*p)); |
︙ | ︙ | |||
147 148 149 150 151 152 153 | sqlite3_result_error(pCtx, "Inf input to percentile()", -1); return; } /* Allocate and store the Y */ if( p->nUsed>=p->nAlloc ){ unsigned n = p->nAlloc*2 + 250; | | | 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 | sqlite3_result_error(pCtx, "Inf input to percentile()", -1); return; } /* Allocate and store the Y */ if( p->nUsed>=p->nAlloc ){ unsigned n = p->nAlloc*2 + 250; double *a = sqlite3_realloc64(p->a, sizeof(double)*n); if( a==0 ){ sqlite3_free(p->a); memset(p, 0, sizeof(*p)); sqlite3_result_error_nomem(pCtx); return; } p->nAlloc = n; |
︙ | ︙ | |||
209 210 211 212 213 214 215 | sqlite3 *db, char **pzErrMsg, const sqlite3_api_routines *pApi ){ int rc = SQLITE_OK; SQLITE_EXTENSION_INIT2(pApi); (void)pzErrMsg; /* Unused parameter */ | | > | 209 210 211 212 213 214 215 216 217 218 219 220 | sqlite3 *db, char **pzErrMsg, const sqlite3_api_routines *pApi ){ int rc = SQLITE_OK; SQLITE_EXTENSION_INIT2(pApi); (void)pzErrMsg; /* Unused parameter */ rc = sqlite3_create_function(db, "percentile", 2, SQLITE_UTF8|SQLITE_INNOCUOUS, 0, 0, percentStep, percentFinal); return rc; } |
Added ext/misc/prefixes.c.
> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 | /* ** 2018-04-19 ** ** The author disclaims copyright to this source code. In place of ** a legal notice, here is a blessing: ** ** May you do good and not evil. ** May you find forgiveness for yourself and forgive others. ** May you share freely, never taking more than you give. ** ************************************************************************* ** ** This file implements a table-valued function: ** ** prefixes('abcdefg') ** ** The function has a single (non-HIDDEN) column named prefix that takes ** on all prefixes of the string in its argument, including an empty string ** and the input string itself. The order of prefixes is from longest ** to shortest. */ #if !defined(SQLITE_CORE) || !defined(SQLITE_OMIT_VIRTUALTABLE) #if !defined(SQLITEINT_H) #include "sqlite3ext.h" #endif SQLITE_EXTENSION_INIT1 #include <string.h> #include <assert.h> /* prefixes_vtab is a subclass of sqlite3_vtab which is ** underlying representation of the virtual table */ typedef struct prefixes_vtab prefixes_vtab; struct prefixes_vtab { sqlite3_vtab base; /* Base class - must be first */ /* No additional fields are necessary */ }; /* prefixes_cursor is a subclass of sqlite3_vtab_cursor which will ** serve as the underlying representation of a cursor that scans ** over rows of the result */ typedef struct prefixes_cursor prefixes_cursor; struct prefixes_cursor { sqlite3_vtab_cursor base; /* Base class - must be first */ sqlite3_int64 iRowid; /* The rowid */ char *zStr; /* Original string to be prefixed */ int nStr; /* Length of the string in bytes */ }; /* ** The prefixesConnect() method is invoked to create a new ** template virtual table. ** ** Think of this routine as the constructor for prefixes_vtab objects. ** ** All this routine needs to do is: ** ** (1) Allocate the prefixes_vtab object and initialize all fields. ** ** (2) Tell SQLite (via the sqlite3_declare_vtab() interface) what the ** result set of queries against the virtual table will look like. */ static int prefixesConnect( sqlite3 *db, void *pAux, int argc, const char *const*argv, sqlite3_vtab **ppVtab, char **pzErr ){ prefixes_vtab *pNew; int rc; rc = sqlite3_declare_vtab(db, "CREATE TABLE prefixes(prefix TEXT, original_string TEXT HIDDEN)" ); if( rc==SQLITE_OK ){ pNew = sqlite3_malloc( sizeof(*pNew) ); *ppVtab = (sqlite3_vtab*)pNew; if( pNew==0 ) return SQLITE_NOMEM; memset(pNew, 0, sizeof(*pNew)); sqlite3_vtab_config(db, SQLITE_VTAB_INNOCUOUS); } return rc; } /* ** This method is the destructor for prefixes_vtab objects. */ static int prefixesDisconnect(sqlite3_vtab *pVtab){ prefixes_vtab *p = (prefixes_vtab*)pVtab; sqlite3_free(p); return SQLITE_OK; } /* ** Constructor for a new prefixes_cursor object. */ static int prefixesOpen(sqlite3_vtab *p, sqlite3_vtab_cursor **ppCursor){ prefixes_cursor *pCur; pCur = sqlite3_malloc( sizeof(*pCur) ); if( pCur==0 ) return SQLITE_NOMEM; memset(pCur, 0, sizeof(*pCur)); *ppCursor = &pCur->base; return SQLITE_OK; } /* ** Destructor for a prefixes_cursor. */ static int prefixesClose(sqlite3_vtab_cursor *cur){ prefixes_cursor *pCur = (prefixes_cursor*)cur; sqlite3_free(pCur->zStr); sqlite3_free(pCur); return SQLITE_OK; } /* ** Advance a prefixes_cursor to its next row of output. */ static int prefixesNext(sqlite3_vtab_cursor *cur){ prefixes_cursor *pCur = (prefixes_cursor*)cur; pCur->iRowid++; return SQLITE_OK; } /* ** Return values of columns for the row at which the prefixes_cursor ** is currently pointing. */ static int prefixesColumn( sqlite3_vtab_cursor *cur, /* The cursor */ sqlite3_context *ctx, /* First argument to sqlite3_result_...() */ int i /* Which column to return */ ){ prefixes_cursor *pCur = (prefixes_cursor*)cur; switch( i ){ case 0: sqlite3_result_text(ctx, pCur->zStr, pCur->nStr - (int)pCur->iRowid, 0); break; default: sqlite3_result_text(ctx, pCur->zStr, pCur->nStr, 0); break; } return SQLITE_OK; } /* ** Return the rowid for the current row. In this implementation, the ** rowid is the same as the output value. */ static int prefixesRowid(sqlite3_vtab_cursor *cur, sqlite_int64 *pRowid){ prefixes_cursor *pCur = (prefixes_cursor*)cur; *pRowid = pCur->iRowid; return SQLITE_OK; } /* ** Return TRUE if the cursor has been moved off of the last ** row of output. */ static int prefixesEof(sqlite3_vtab_cursor *cur){ prefixes_cursor *pCur = (prefixes_cursor*)cur; return pCur->iRowid>pCur->nStr; } /* ** This method is called to "rewind" the prefixes_cursor object back ** to the first row of output. This method is always called at least ** once prior to any call to prefixesColumn() or prefixesRowid() or ** prefixesEof(). */ static int prefixesFilter( sqlite3_vtab_cursor *pVtabCursor, int idxNum, const char *idxStr, int argc, sqlite3_value **argv ){ prefixes_cursor *pCur = (prefixes_cursor *)pVtabCursor; sqlite3_free(pCur->zStr); if( argc>0 ){ pCur->zStr = sqlite3_mprintf("%s", sqlite3_value_text(argv[0])); pCur->nStr = pCur->zStr ? (int)strlen(pCur->zStr) : 0; }else{ pCur->zStr = 0; pCur->nStr = 0; } pCur->iRowid = 0; return SQLITE_OK; } /* ** SQLite will invoke this method one or more times while planning a query ** that uses the virtual table. This routine needs to create ** a query plan for each invocation and compute an estimated cost for that ** plan. */ static int prefixesBestIndex( sqlite3_vtab *tab, sqlite3_index_info *pIdxInfo ){ /* Search for a usable equality constraint against column 1 ** (original_string) and use it if at all possible */ int i; const struct sqlite3_index_constraint *p; for(i=0, p=pIdxInfo->aConstraint; i<pIdxInfo->nConstraint; i++, p++){ if( p->iColumn!=1 ) continue; if( p->op!=SQLITE_INDEX_CONSTRAINT_EQ ) continue; if( !p->usable ) continue; pIdxInfo->aConstraintUsage[i].argvIndex = 1; pIdxInfo->aConstraintUsage[i].omit = 1; pIdxInfo->estimatedCost = (double)10; pIdxInfo->estimatedRows = 10; return SQLITE_OK; } pIdxInfo->estimatedCost = (double)1000000000; pIdxInfo->estimatedRows = 1000000000; return SQLITE_OK; } /* ** This following structure defines all the methods for the ** virtual table. */ static sqlite3_module prefixesModule = { /* iVersion */ 0, /* xCreate */ 0, /* xConnect */ prefixesConnect, /* xBestIndex */ prefixesBestIndex, /* xDisconnect */ prefixesDisconnect, /* xDestroy */ 0, /* xOpen */ prefixesOpen, /* xClose */ prefixesClose, /* xFilter */ prefixesFilter, /* xNext */ prefixesNext, /* xEof */ prefixesEof, /* xColumn */ prefixesColumn, /* xRowid */ prefixesRowid, /* xUpdate */ 0, /* xBegin */ 0, /* xSync */ 0, /* xCommit */ 0, /* xRollback */ 0, /* xFindMethod */ 0, /* xRename */ 0, /* xSavepoint */ 0, /* xRelease */ 0, /* xRollbackTo */ 0, /* xShadowName */ 0 }; /* ** This is a copy of the SQLITE_SKIP_UTF8(zIn) macro in sqliteInt.h. ** ** Assuming zIn points to the first byte of a UTF-8 character, ** advance zIn to point to the first byte of the next UTF-8 character. */ #define PREFIX_SKIP_UTF8(zIn) { \ if( (*(zIn++))>=0xc0 ){ \ while( (*zIn & 0xc0)==0x80 ){ zIn++; } \ } \ } /* ** Implementation of function prefix_length(). This function accepts two ** strings as arguments and returns the length in characters (not bytes), ** of the longest prefix shared by the two strings. For example: ** ** prefix_length('abcdxxx', 'abcyy') == 3 ** prefix_length('abcdxxx', 'bcyyy') == 0 ** prefix_length('abcdxxx', 'ab') == 2 ** prefix_length('ab', 'abcd') == 2 ** ** This function assumes the input is well-formed utf-8. If it is not, ** it is possible for this function to return -1. */ static void prefixLengthFunc( sqlite3_context *ctx, int nVal, sqlite3_value **apVal ){ int nByte; /* Number of bytes to compare */ int nRet = 0; /* Return value */ const unsigned char *zL = sqlite3_value_text(apVal[0]); const unsigned char *zR = sqlite3_value_text(apVal[1]); int nL = sqlite3_value_bytes(apVal[0]); int nR = sqlite3_value_bytes(apVal[1]); int i; nByte = (nL > nR ? nL : nR); for(i=0; i<nByte; i++){ if( zL[i]!=zR[i] ) break; if( (zL[i] & 0xC0)!=0x80 ) nRet++; } if( (zL[i] & 0xC0)==0x80 ) nRet--; sqlite3_result_int(ctx, nRet); } #ifdef _WIN32 __declspec(dllexport) #endif int sqlite3_prefixes_init( sqlite3 *db, char **pzErrMsg, const sqlite3_api_routines *pApi ){ int rc = SQLITE_OK; SQLITE_EXTENSION_INIT2(pApi); rc = sqlite3_create_module(db, "prefixes", &prefixesModule, 0); if( rc==SQLITE_OK ){ rc = sqlite3_create_function( db, "prefix_length", 2, SQLITE_UTF8, 0, prefixLengthFunc, 0, 0 ); } return rc; } #endif /* !defined(SQLITE_CORE) || !defined(SQLITE_OMIT_VIRTUALTABLE) */ |
Added ext/misc/qpvtab.c.
> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340 341 342 343 344 345 346 347 348 349 350 351 352 353 354 355 356 357 358 359 360 361 362 363 364 365 366 367 368 369 370 371 372 373 374 375 376 377 378 379 380 381 382 383 384 385 386 387 388 389 390 391 392 393 394 395 396 397 398 399 400 401 402 403 404 405 406 407 408 409 410 411 412 413 414 415 416 417 418 419 420 421 422 423 424 425 426 427 428 429 430 431 432 433 434 435 436 437 438 439 440 441 442 443 444 445 446 447 448 449 450 451 452 453 454 455 456 457 458 459 460 461 | /* ** 2022-01-19 ** ** The author disclaims copyright to this source code. In place of ** a legal notice, here is a blessing: ** ** May you do good and not evil. ** May you find forgiveness for yourself and forgive others. ** May you share freely, never taking more than you give. ** ************************************************************************* ** ** This file implements a virtual-table that returns information about ** how the query planner called the xBestIndex method. This virtual table ** is intended for testing and debugging only. ** ** The schema of the virtual table is this: ** ** CREATE TABLE qpvtab( ** vn TEXT, -- Name of an sqlite3_index_info field ** ix INTEGER, -- Array index or value ** cn TEXT, -- Column name ** op INTEGER, -- operator ** ux BOOLEAN, -- "usable" field ** rhs TEXT, -- sqlite3_vtab_rhs_value() ** ** a, b, c, d, e, -- Extra columns to attach constraints to ** ** flags INTEGER HIDDEN -- control flags ** ); ** ** The virtual table returns a description of the sqlite3_index_info object ** that was provided to the (successful) xBestIndex method. There is one ** row in the result table for each field in the sqlite3_index_info object. ** ** The values of the "a" through "e" columns are one of: ** ** 1. TEXT - the same as the column name ** 2. INTEGER - 1 for "a", 2 for "b", and so forth ** ** Option 1 is the default behavior. 2 is use if there is a usable ** constraint on "flags" with an integer right-hand side that where the ** value of the right-hand side has its 0x001 bit set. ** ** All constraints on columns "a" through "e" are marked as "omit". ** ** If there is a usable constraint on "flags" that has a RHS value that ** is an integer and that integer has its 0x02 bit set, then the ** orderByConsumed flag is set. ** ** FLAGS SUMMARY: ** ** 0x001 Columns 'a' through 'e' have INT values ** 0x002 orderByConsumed is set ** 0x004 OFFSET and LIMIT have omit set ** ** COMPILE: ** ** gcc -Wall -g -shared -fPIC -I. qpvtab.c -o qqvtab.so ** ** EXAMPLE USAGE: ** ** .load ./qpvtab ** SELECT rowid, *, flags FROM qpvtab(102) ** WHERE a=19 ** AND b BETWEEN 4.5 and 'hello' ** AND c<>x'aabbcc' ** ORDER BY d, e DESC; */ #if !defined(SQLITEINT_H) #include "sqlite3ext.h" #endif SQLITE_EXTENSION_INIT1 #include <string.h> #include <assert.h> #include <stdlib.h> #if !defined(SQLITE_OMIT_VIRTUALTABLE) /* qpvtab_vtab is a subclass of sqlite3_vtab which is ** underlying representation of the virtual table */ typedef struct qpvtab_vtab qpvtab_vtab; struct qpvtab_vtab { sqlite3_vtab base; /* Base class - must be first */ }; /* qpvtab_cursor is a subclass of sqlite3_vtab_cursor which will ** serve as the underlying representation of a cursor that scans ** over rows of the result */ typedef struct qpvtab_cursor qpvtab_cursor; struct qpvtab_cursor { sqlite3_vtab_cursor base; /* Base class - must be first */ sqlite3_int64 iRowid; /* The rowid */ const char *zData; /* Data to return */ int nData; /* Number of bytes of data */ int flags; /* Flags value */ }; /* ** Names of columns */ static const char *azColname[] = { "vn", "ix", "cn", "op", "ux", "rhs", "a", "b", "c", "d", "e", "flags", "" }; /* ** The qpvtabConnect() method is invoked to create a new ** qpvtab virtual table. */ static int qpvtabConnect( sqlite3 *db, void *pAux, int argc, const char *const*argv, sqlite3_vtab **ppVtab, char **pzErr ){ qpvtab_vtab *pNew; int rc; rc = sqlite3_declare_vtab(db, "CREATE TABLE x(" " vn TEXT," " ix INT," " cn TEXT," " op INT," " ux BOOLEAN," " rhs TEXT," " a, b, c, d, e," " flags INT HIDDEN)" ); #define QPVTAB_VN 0 #define QPVTAB_IX 1 #define QPVTAB_CN 2 #define QPVTAB_OP 3 #define QPVTAB_UX 4 #define QPVTAB_RHS 5 #define QPVTAB_A 6 #define QPVTAB_B 7 #define QPVTAB_C 8 #define QPVTAB_D 9 #define QPVTAB_E 10 #define QPVTAB_FLAGS 11 #define QPVTAB_NONE 12 if( rc==SQLITE_OK ){ pNew = sqlite3_malloc( sizeof(*pNew) ); *ppVtab = (sqlite3_vtab*)pNew; if( pNew==0 ) return SQLITE_NOMEM; memset(pNew, 0, sizeof(*pNew)); } return rc; } /* ** This method is the destructor for qpvtab_vtab objects. */ static int qpvtabDisconnect(sqlite3_vtab *pVtab){ qpvtab_vtab *p = (qpvtab_vtab*)pVtab; sqlite3_free(p); return SQLITE_OK; } /* ** Constructor for a new qpvtab_cursor object. */ static int qpvtabOpen(sqlite3_vtab *p, sqlite3_vtab_cursor **ppCursor){ qpvtab_cursor *pCur; pCur = sqlite3_malloc( sizeof(*pCur) ); if( pCur==0 ) return SQLITE_NOMEM; memset(pCur, 0, sizeof(*pCur)); *ppCursor = &pCur->base; return SQLITE_OK; } /* ** Destructor for a qpvtab_cursor. */ static int qpvtabClose(sqlite3_vtab_cursor *cur){ qpvtab_cursor *pCur = (qpvtab_cursor*)cur; sqlite3_free(pCur); return SQLITE_OK; } /* ** Advance a qpvtab_cursor to its next row of output. */ static int qpvtabNext(sqlite3_vtab_cursor *cur){ qpvtab_cursor *pCur = (qpvtab_cursor*)cur; if( pCur->iRowid<pCur->nData ){ const char *z = &pCur->zData[pCur->iRowid]; const char *zEnd = strchr(z, '\n'); if( zEnd ) zEnd++; pCur->iRowid = (int)(zEnd - pCur->zData); } return SQLITE_OK; } /* ** Return values of columns for the row at which the qpvtab_cursor ** is currently pointing. */ static int qpvtabColumn( sqlite3_vtab_cursor *cur, /* The cursor */ sqlite3_context *ctx, /* First argument to sqlite3_result_...() */ int i /* Which column to return */ ){ qpvtab_cursor *pCur = (qpvtab_cursor*)cur; if( i>=QPVTAB_VN && i<=QPVTAB_RHS && pCur->iRowid<pCur->nData ){ const char *z = &pCur->zData[pCur->iRowid]; const char *zEnd; int j; j = QPVTAB_VN; while(1){ zEnd = strchr(z, j==QPVTAB_RHS ? '\n' : ','); if( j==i || zEnd==0 ) break; z = zEnd+1; j++; } if( zEnd==z ){ sqlite3_result_null(ctx); }else if( i==QPVTAB_IX || i==QPVTAB_OP || i==QPVTAB_UX ){ sqlite3_result_int(ctx, atoi(z)); }else{ sqlite3_result_text64(ctx, z, zEnd-z, SQLITE_TRANSIENT, SQLITE_UTF8); } }else if( i>=QPVTAB_A && i<=QPVTAB_E ){ if( pCur->flags & 0x001 ){ sqlite3_result_int(ctx, i-QPVTAB_A+1); }else{ char x = 'a'+i-QPVTAB_A; sqlite3_result_text64(ctx, &x, 1, SQLITE_TRANSIENT, SQLITE_UTF8); } }else if( i==QPVTAB_FLAGS ){ sqlite3_result_int(ctx, pCur->flags); } return SQLITE_OK; } /* ** Return the rowid for the current row. In this implementation, the ** rowid is the same as the output value. */ static int qpvtabRowid(sqlite3_vtab_cursor *cur, sqlite_int64 *pRowid){ qpvtab_cursor *pCur = (qpvtab_cursor*)cur; *pRowid = pCur->iRowid; return SQLITE_OK; } /* ** Return TRUE if the cursor has been moved off of the last ** row of output. */ static int qpvtabEof(sqlite3_vtab_cursor *cur){ qpvtab_cursor *pCur = (qpvtab_cursor*)cur; return pCur->iRowid>=pCur->nData; } /* ** This method is called to "rewind" the qpvtab_cursor object back ** to the first row of output. This method is always called at least ** once prior to any call to qpvtabColumn() or qpvtabRowid() or ** qpvtabEof(). */ static int qpvtabFilter( sqlite3_vtab_cursor *pVtabCursor, int idxNum, const char *idxStr, int argc, sqlite3_value **argv ){ qpvtab_cursor *pCur = (qpvtab_cursor *)pVtabCursor; pCur->iRowid = 0; pCur->zData = idxStr; pCur->nData = (int)strlen(idxStr); pCur->flags = idxNum; return SQLITE_OK; } /* ** Append the text of a value to pStr */ static void qpvtabStrAppendValue( sqlite3_str *pStr, sqlite3_value *pVal ){ switch( sqlite3_value_type(pVal) ){ case SQLITE_NULL: sqlite3_str_appendf(pStr, "NULL"); break; case SQLITE_INTEGER: sqlite3_str_appendf(pStr, "%lld", sqlite3_value_int64(pVal)); break; case SQLITE_FLOAT: sqlite3_str_appendf(pStr, "%!f", sqlite3_value_double(pVal)); break; case SQLITE_TEXT: { int i; const char *a = (const char*)sqlite3_value_text(pVal); int n = sqlite3_value_bytes(pVal); sqlite3_str_append(pStr, "'", 1); for(i=0; i<n; i++){ char c = a[i]; if( c=='\n' ) c = ' '; sqlite3_str_append(pStr, &c, 1); if( c=='\'' ) sqlite3_str_append(pStr, &c, 1); } sqlite3_str_append(pStr, "'", 1); break; } case SQLITE_BLOB: { int i; const unsigned char *a = sqlite3_value_blob(pVal); int n = sqlite3_value_bytes(pVal); sqlite3_str_append(pStr, "x'", 2); for(i=0; i<n; i++){ sqlite3_str_appendf(pStr, "%02x", a[i]); } sqlite3_str_append(pStr, "'", 1); break; } } } /* ** SQLite will invoke this method one or more times while planning a query ** that uses the virtual table. This routine needs to create ** a query plan for each invocation and compute an estimated cost for that ** plan. */ static int qpvtabBestIndex( sqlite3_vtab *tab, sqlite3_index_info *pIdxInfo ){ sqlite3_str *pStr = sqlite3_str_new(0); int i, k = 0; int rc; sqlite3_str_appendf(pStr, "nConstraint,%d,,,,\n", pIdxInfo->nConstraint); for(i=0; i<pIdxInfo->nConstraint; i++){ sqlite3_value *pVal; int iCol = pIdxInfo->aConstraint[i].iColumn; int op = pIdxInfo->aConstraint[i].op; if( iCol==QPVTAB_FLAGS && pIdxInfo->aConstraint[i].usable ){ pVal = 0; rc = sqlite3_vtab_rhs_value(pIdxInfo, i, &pVal); assert( rc==SQLITE_OK || pVal==0 ); if( pVal ){ pIdxInfo->idxNum = sqlite3_value_int(pVal); if( pIdxInfo->idxNum & 0x002 ) pIdxInfo->orderByConsumed = 1; } } if( op==SQLITE_INDEX_CONSTRAINT_LIMIT || op==SQLITE_INDEX_CONSTRAINT_OFFSET ){ iCol = QPVTAB_NONE; } sqlite3_str_appendf(pStr,"aConstraint,%d,%s,%d,%d,", i, azColname[iCol], op, pIdxInfo->aConstraint[i].usable); pVal = 0; rc = sqlite3_vtab_rhs_value(pIdxInfo, i, &pVal); assert( rc==SQLITE_OK || pVal==0 ); if( pVal ){ qpvtabStrAppendValue(pStr, pVal); } sqlite3_str_append(pStr, "\n", 1); } for(i=0; i<pIdxInfo->nConstraint; i++){ int iCol = pIdxInfo->aConstraint[i].iColumn; int op = pIdxInfo->aConstraint[i].op; if( op==SQLITE_INDEX_CONSTRAINT_LIMIT || op==SQLITE_INDEX_CONSTRAINT_OFFSET ){ iCol = QPVTAB_NONE; } if( iCol>=QPVTAB_A && pIdxInfo->aConstraint[i].usable ){ pIdxInfo->aConstraintUsage[i].argvIndex = ++k; if( iCol<=QPVTAB_FLAGS || (pIdxInfo->idxNum & 0x004)!=0 ){ pIdxInfo->aConstraintUsage[i].omit = 1; } } } sqlite3_str_appendf(pStr, "nOrderBy,%d,,,,\n", pIdxInfo->nOrderBy); for(i=0; i<pIdxInfo->nOrderBy; i++){ int iCol = pIdxInfo->aOrderBy[i].iColumn; sqlite3_str_appendf(pStr, "aOrderBy,%d,%s,%d,,\n",i, iCol>=0 ? azColname[iCol] : "rowid", pIdxInfo->aOrderBy[i].desc ); } sqlite3_str_appendf(pStr, "sqlite3_vtab_distinct,%d,,,,\n", sqlite3_vtab_distinct(pIdxInfo)); sqlite3_str_appendf(pStr, "idxFlags,%d,,,,\n", pIdxInfo->idxFlags); sqlite3_str_appendf(pStr, "colUsed,%d,,,,\n", (int)pIdxInfo->colUsed); pIdxInfo->estimatedCost = (double)10; pIdxInfo->estimatedRows = 10; sqlite3_str_appendf(pStr, "idxNum,%d,,,,\n", pIdxInfo->idxNum); sqlite3_str_appendf(pStr, "orderByConsumed,%d,,,,\n", pIdxInfo->orderByConsumed); pIdxInfo->idxStr = sqlite3_str_finish(pStr); pIdxInfo->needToFreeIdxStr = 1; return SQLITE_OK; } /* ** This following structure defines all the methods for the ** virtual table. */ static sqlite3_module qpvtabModule = { /* iVersion */ 0, /* xCreate */ 0, /* xConnect */ qpvtabConnect, /* xBestIndex */ qpvtabBestIndex, /* xDisconnect */ qpvtabDisconnect, /* xDestroy */ 0, /* xOpen */ qpvtabOpen, /* xClose */ qpvtabClose, /* xFilter */ qpvtabFilter, /* xNext */ qpvtabNext, /* xEof */ qpvtabEof, /* xColumn */ qpvtabColumn, /* xRowid */ qpvtabRowid, /* xUpdate */ 0, /* xBegin */ 0, /* xSync */ 0, /* xCommit */ 0, /* xRollback */ 0, /* xFindMethod */ 0, /* xRename */ 0, /* xSavepoint */ 0, /* xRelease */ 0, /* xRollbackTo */ 0, /* xShadowName */ 0 }; #endif /* SQLITE_OMIT_VIRTUALTABLE */ #ifdef _WIN32 __declspec(dllexport) #endif int sqlite3_qpvtab_init( sqlite3 *db, char **pzErrMsg, const sqlite3_api_routines *pApi ){ int rc = SQLITE_OK; SQLITE_EXTENSION_INIT2(pApi); #ifndef SQLITE_OMIT_VIRTUALTABLE rc = sqlite3_create_module(db, "qpvtab", &qpvtabModule, 0); #endif return rc; } |
Changes to ext/misc/regexp.c.
︙ | ︙ | |||
68 69 70 71 72 73 74 75 76 77 78 79 80 81 | */ #define re_match sqlite3re_match #define re_compile sqlite3re_compile #define re_free sqlite3re_free /* The end-of-input character */ #define RE_EOF 0 /* End of input */ /* The NFA is implemented as sequence of opcodes taken from the following ** set. Each opcode has a single integer argument. */ #define RE_OP_MATCH 1 /* Match the one character in the argument */ #define RE_OP_ANY 2 /* Match any one character. (Implements ".") */ #define RE_OP_ANYSTAR 3 /* Special optimized version of .* */ | > | 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 | */ #define re_match sqlite3re_match #define re_compile sqlite3re_compile #define re_free sqlite3re_free /* The end-of-input character */ #define RE_EOF 0 /* End of input */ #define RE_START 0xfffffff /* Start of input - larger than an UTF-8 */ /* The NFA is implemented as sequence of opcodes taken from the following ** set. Each opcode has a single integer argument. */ #define RE_OP_MATCH 1 /* Match the one character in the argument */ #define RE_OP_ANY 2 /* Match any one character. (Implements ".") */ #define RE_OP_ANYSTAR 3 /* Special optimized version of .* */ |
︙ | ︙ | |||
89 90 91 92 93 94 95 96 97 98 99 100 101 102 | #define RE_OP_WORD 11 /* Perl word character [A-Za-z0-9_] */ #define RE_OP_NOTWORD 12 /* Not a perl word character */ #define RE_OP_DIGIT 13 /* digit: [0-9] */ #define RE_OP_NOTDIGIT 14 /* Not a digit */ #define RE_OP_SPACE 15 /* space: [ \t\n\r\v\f] */ #define RE_OP_NOTSPACE 16 /* Not a digit */ #define RE_OP_BOUNDARY 17 /* Boundary between word and non-word */ /* Each opcode is a "state" in the NFA */ typedef unsigned short ReStateNumber; /* Because this is an NFA and not a DFA, multiple states can be active at ** once. An instance of the following object records all active states in ** the NFA. The implementation is optimized for the common case where the | > > > > > > > > > > > > > > > > > > > > > > > > > > > | 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 | #define RE_OP_WORD 11 /* Perl word character [A-Za-z0-9_] */ #define RE_OP_NOTWORD 12 /* Not a perl word character */ #define RE_OP_DIGIT 13 /* digit: [0-9] */ #define RE_OP_NOTDIGIT 14 /* Not a digit */ #define RE_OP_SPACE 15 /* space: [ \t\n\r\v\f] */ #define RE_OP_NOTSPACE 16 /* Not a digit */ #define RE_OP_BOUNDARY 17 /* Boundary between word and non-word */ #define RE_OP_ATSTART 18 /* Currently at the start of the string */ #if defined(SQLITE_DEBUG) /* Opcode names used for symbolic debugging */ static const char *ReOpName[] = { "EOF", "MATCH", "ANY", "ANYSTAR", "FORK", "GOTO", "ACCEPT", "CC_INC", "CC_EXC", "CC_VALUE", "CC_RANGE", "WORD", "NOTWORD", "DIGIT", "NOTDIGIT", "SPACE", "NOTSPACE", "BOUNDARY", "ATSTART", }; #endif /* SQLITE_DEBUG */ /* Each opcode is a "state" in the NFA */ typedef unsigned short ReStateNumber; /* Because this is an NFA and not a DFA, multiple states can be active at ** once. An instance of the following object records all active states in ** the NFA. The implementation is optimized for the common case where the |
︙ | ︙ | |||
123 124 125 126 127 128 129 | struct ReCompiled { ReInput sIn; /* Regular expression text */ const char *zErr; /* Error message to return */ char *aOp; /* Operators for the virtual machine */ int *aArg; /* Arguments to each operator */ unsigned (*xNextChar)(ReInput*); /* Next character function */ unsigned char zInit[12]; /* Initial text to match */ | | | 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 | struct ReCompiled { ReInput sIn; /* Regular expression text */ const char *zErr; /* Error message to return */ char *aOp; /* Operators for the virtual machine */ int *aArg; /* Arguments to each operator */ unsigned (*xNextChar)(ReInput*); /* Next character function */ unsigned char zInit[12]; /* Initial text to match */ int nInit; /* Number of bytes in zInit */ unsigned nState; /* Number of entries in aOp[] and aArg[] */ unsigned nAlloc; /* Slots allocated for aOp[] and aArg[] */ }; /* Add a state to the given state set if it is not already there */ static void re_add_state(ReStateSet *pSet, int newState){ unsigned i; |
︙ | ︙ | |||
152 153 154 155 156 157 158 | if( (c&0xe0)==0xc0 && p->i<p->mx && (p->z[p->i]&0xc0)==0x80 ){ c = (c&0x1f)<<6 | (p->z[p->i++]&0x3f); if( c<0x80 ) c = 0xfffd; }else if( (c&0xf0)==0xe0 && p->i+1<p->mx && (p->z[p->i]&0xc0)==0x80 && (p->z[p->i+1]&0xc0)==0x80 ){ c = (c&0x0f)<<12 | ((p->z[p->i]&0x3f)<<6) | (p->z[p->i+1]&0x3f); p->i += 2; | | | | 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 | if( (c&0xe0)==0xc0 && p->i<p->mx && (p->z[p->i]&0xc0)==0x80 ){ c = (c&0x1f)<<6 | (p->z[p->i++]&0x3f); if( c<0x80 ) c = 0xfffd; }else if( (c&0xf0)==0xe0 && p->i+1<p->mx && (p->z[p->i]&0xc0)==0x80 && (p->z[p->i+1]&0xc0)==0x80 ){ c = (c&0x0f)<<12 | ((p->z[p->i]&0x3f)<<6) | (p->z[p->i+1]&0x3f); p->i += 2; if( c<=0x7ff || (c>=0xd800 && c<=0xdfff) ) c = 0xfffd; }else if( (c&0xf8)==0xf0 && p->i+2<p->mx && (p->z[p->i]&0xc0)==0x80 && (p->z[p->i+1]&0xc0)==0x80 && (p->z[p->i+2]&0xc0)==0x80 ){ c = (c&0x07)<<18 | ((p->z[p->i]&0x3f)<<12) | ((p->z[p->i+1]&0x3f)<<6) | (p->z[p->i+2]&0x3f); p->i += 3; if( c<=0xffff || c>0x10ffff ) c = 0xfffd; }else{ c = 0xfffd; |
︙ | ︙ | |||
196 197 198 199 200 201 202 | */ static int re_match(ReCompiled *pRe, const unsigned char *zIn, int nIn){ ReStateSet aStateSet[2], *pThis, *pNext; ReStateNumber aSpace[100]; ReStateNumber *pToFree; unsigned int i = 0; unsigned int iSwap = 0; | | > | > > > > | | | | | 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 | */ static int re_match(ReCompiled *pRe, const unsigned char *zIn, int nIn){ ReStateSet aStateSet[2], *pThis, *pNext; ReStateNumber aSpace[100]; ReStateNumber *pToFree; unsigned int i = 0; unsigned int iSwap = 0; int c = RE_START; int cPrev = 0; int rc = 0; ReInput in; in.z = zIn; in.i = 0; in.mx = nIn>=0 ? nIn : (int)strlen((char const*)zIn); /* Look for the initial prefix match, if there is one. */ if( pRe->nInit ){ unsigned char x = pRe->zInit[0]; while( in.i+pRe->nInit<=in.mx && (zIn[in.i]!=x || strncmp((const char*)zIn+in.i, (const char*)pRe->zInit, pRe->nInit)!=0) ){ in.i++; } if( in.i+pRe->nInit>in.mx ) return 0; c = RE_START-1; } if( pRe->nState<=(sizeof(aSpace)/(sizeof(aSpace[0])*2)) ){ pToFree = 0; aStateSet[0].aState = aSpace; }else{ pToFree = sqlite3_malloc64( sizeof(ReStateNumber)*2*pRe->nState ); if( pToFree==0 ) return -1; aStateSet[0].aState = pToFree; } aStateSet[1].aState = &aStateSet[0].aState[pRe->nState]; pNext = &aStateSet[1]; pNext->nState = 0; re_add_state(pNext, 0); while( c!=RE_EOF && pNext->nState>0 ){ cPrev = c; c = pRe->xNextChar(&in); pThis = pNext; pNext = &aStateSet[iSwap]; iSwap = 1 - iSwap; pNext->nState = 0; for(i=0; i<pThis->nState; i++){ int x = pThis->aState[i]; switch( pRe->aOp[x] ){ case RE_OP_MATCH: { if( pRe->aArg[x]==c ) re_add_state(pNext, x+1); break; } case RE_OP_ATSTART: { if( cPrev==RE_START ) re_add_state(pThis, x+1); break; } case RE_OP_ANY: { if( c!=0 ) re_add_state(pNext, x+1); break; } case RE_OP_WORD: { if( re_word_char(c) ) re_add_state(pNext, x+1); break; } case RE_OP_NOTWORD: { if( !re_word_char(c) && c!=0 ) re_add_state(pNext, x+1); break; } case RE_OP_DIGIT: { if( re_digit_char(c) ) re_add_state(pNext, x+1); break; } case RE_OP_NOTDIGIT: { if( !re_digit_char(c) && c!=0 ) re_add_state(pNext, x+1); break; } case RE_OP_SPACE: { if( re_space_char(c) ) re_add_state(pNext, x+1); break; } case RE_OP_NOTSPACE: { if( !re_space_char(c) && c!=0 ) re_add_state(pNext, x+1); break; } case RE_OP_BOUNDARY: { if( re_word_char(c)!=re_word_char(cPrev) ) re_add_state(pThis, x+1); break; } case RE_OP_ANYSTAR: { |
︙ | ︙ | |||
293 294 295 296 297 298 299 | re_add_state(pThis, x+pRe->aArg[x]); break; } case RE_OP_ACCEPT: { rc = 1; goto re_match_end; } | | > > > | | > > | | | | 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340 341 342 343 344 345 346 347 348 349 350 351 352 353 354 355 356 357 358 359 360 361 362 363 364 365 366 367 368 369 370 371 372 373 374 375 376 377 378 379 380 381 382 383 384 385 386 387 388 | re_add_state(pThis, x+pRe->aArg[x]); break; } case RE_OP_ACCEPT: { rc = 1; goto re_match_end; } case RE_OP_CC_EXC: { if( c==0 ) break; /* fall-through */ goto re_op_cc_inc; } case RE_OP_CC_INC: re_op_cc_inc: { int j = 1; int n = pRe->aArg[x]; int hit = 0; for(j=1; j>0 && j<n; j++){ if( pRe->aOp[x+j]==RE_OP_CC_VALUE ){ if( pRe->aArg[x+j]==c ){ hit = 1; j = -1; } }else{ if( pRe->aArg[x+j]<=c && pRe->aArg[x+j+1]>=c ){ hit = 1; j = -1; }else{ j++; } } } if( pRe->aOp[x]==RE_OP_CC_EXC ) hit = !hit; if( hit ) re_add_state(pNext, x+n); break; } } } } for(i=0; i<pNext->nState; i++){ int x = pNext->aState[i]; while( pRe->aOp[x]==RE_OP_GOTO ) x += pRe->aArg[x]; if( pRe->aOp[x]==RE_OP_ACCEPT ){ rc = 1; break; } } re_match_end: sqlite3_free(pToFree); return rc; } /* Resize the opcode and argument arrays for an RE under construction. */ static int re_resize(ReCompiled *p, int N){ char *aOp; int *aArg; aOp = sqlite3_realloc64(p->aOp, N*sizeof(p->aOp[0])); if( aOp==0 ) return 1; p->aOp = aOp; aArg = sqlite3_realloc64(p->aArg, N*sizeof(p->aArg[0])); if( aArg==0 ) return 1; p->aArg = aArg; p->nAlloc = N; return 0; } /* Insert a new opcode and argument into an RE under construction. The |
︙ | ︙ | |||
476 477 478 479 480 481 482 | int iStart; unsigned c; const char *zErr; while( (c = p->xNextChar(&p->sIn))!=0 ){ iStart = p->nState; switch( c ){ case '|': | < | > > > > > > > > > | 514 515 516 517 518 519 520 521 522 523 524 525 526 527 528 529 530 531 532 533 534 535 536 537 538 539 540 541 542 543 544 545 546 547 548 549 550 551 552 553 554 555 556 557 558 559 560 561 562 563 564 565 566 567 568 569 570 571 572 573 574 575 576 577 578 579 580 581 582 583 584 585 586 587 588 589 590 | int iStart; unsigned c; const char *zErr; while( (c = p->xNextChar(&p->sIn))!=0 ){ iStart = p->nState; switch( c ){ case '|': case ')': { p->sIn.i--; return 0; } case '(': { zErr = re_subcompile_re(p); if( zErr ) return zErr; if( rePeek(p)!=')' ) return "unmatched '('"; p->sIn.i++; break; } case '.': { if( rePeek(p)=='*' ){ re_append(p, RE_OP_ANYSTAR, 0); p->sIn.i++; }else{ re_append(p, RE_OP_ANY, 0); } break; } case '*': { if( iPrev<0 ) return "'*' without operand"; re_insert(p, iPrev, RE_OP_GOTO, p->nState - iPrev + 1); re_append(p, RE_OP_FORK, iPrev - p->nState + 1); break; } case '+': { if( iPrev<0 ) return "'+' without operand"; re_append(p, RE_OP_FORK, iPrev - p->nState); break; } case '?': { if( iPrev<0 ) return "'?' without operand"; re_insert(p, iPrev, RE_OP_FORK, p->nState - iPrev+1); break; } case '$': { re_append(p, RE_OP_MATCH, RE_EOF); break; } case '^': { re_append(p, RE_OP_ATSTART, 0); break; } case '{': { int m = 0, n = 0; int sz, j; if( iPrev<0 ) return "'{m,n}' without operand"; while( (c=rePeek(p))>='0' && c<='9' ){ m = m*10 + c - '0'; p->sIn.i++; } n = m; if( c==',' ){ p->sIn.i++; n = 0; while( (c=rePeek(p))>='0' && c<='9' ){ n = n*10 + c-'0'; p->sIn.i++; } } if( c!='}' ) return "unmatched '{'"; if( n>0 && n<m ) return "n less than m in '{m,n}'"; p->sIn.i++; sz = p->nState - iPrev; if( m==0 ){ if( n==0 ) return "both m and n are zero in '{m,n}'"; re_insert(p, iPrev, RE_OP_FORK, sz+1); iPrev++; n--; }else{ for(j=1; j<m; j++) re_copy(p, iPrev, sz); } for(j=m; j<n; j++){ re_append(p, RE_OP_FORK, sz+1); re_copy(p, iPrev, sz); |
︙ | ︙ | |||
606 607 608 609 610 611 612 | return 0; } /* Free and reclaim all the memory used by a previously compiled ** regular expression. Applications should invoke this routine once ** for every call to re_compile() to avoid memory leaks. */ | | | | 652 653 654 655 656 657 658 659 660 661 662 663 664 665 666 667 668 669 670 671 672 673 674 675 676 677 678 679 680 | return 0; } /* Free and reclaim all the memory used by a previously compiled ** regular expression. Applications should invoke this routine once ** for every call to re_compile() to avoid memory leaks. */ static void re_free(ReCompiled *pRe){ if( pRe ){ sqlite3_free(pRe->aOp); sqlite3_free(pRe->aArg); sqlite3_free(pRe); } } /* ** Compile a textual regular expression in zIn[] into a compiled regular ** expression suitable for us by re_match() and return a pointer to the ** compiled regular expression in *ppRe. Return NULL on success or an ** error message if something goes wrong. */ static const char *re_compile(ReCompiled **ppRe, const char *zIn, int noCase){ ReCompiled *pRe; const char *zErr; int i, j; *ppRe = 0; pRe = sqlite3_malloc( sizeof(*pRe) ); if( pRe==0 ){ |
︙ | ︙ | |||
649 650 651 652 653 654 655 | pRe->sIn.i = 0; pRe->sIn.mx = (int)strlen(zIn); zErr = re_subcompile_re(pRe); if( zErr ){ re_free(pRe); return zErr; } | < < < < | | | | | | | | 695 696 697 698 699 700 701 702 703 704 705 706 707 708 709 710 711 712 713 714 715 716 717 718 719 720 721 722 723 724 725 726 727 728 729 730 731 732 733 734 | pRe->sIn.i = 0; pRe->sIn.mx = (int)strlen(zIn); zErr = re_subcompile_re(pRe); if( zErr ){ re_free(pRe); return zErr; } if( pRe->sIn.i>=pRe->sIn.mx ){ re_append(pRe, RE_OP_ACCEPT, 0); *ppRe = pRe; }else{ re_free(pRe); return "unrecognized character"; } /* The following is a performance optimization. If the regex begins with ** ".*" (if the input regex lacks an initial "^") and afterwards there are ** one or more matching characters, enter those matching characters into ** zInit[]. The re_match() routine can then search ahead in the input ** string looking for the initial match without having to run the whole ** regex engine over the string. Do not worry about trying to match ** unicode characters beyond plane 0 - those are very rare and this is ** just an optimization. */ if( pRe->aOp[0]==RE_OP_ANYSTAR && !noCase ){ for(j=0, i=1; j<(int)sizeof(pRe->zInit)-2 && pRe->aOp[i]==RE_OP_MATCH; i++){ unsigned x = pRe->aArg[i]; if( x<=0x7f ){ pRe->zInit[j++] = (unsigned char)x; }else if( x<=0x7ff ){ pRe->zInit[j++] = (unsigned char)(0xc0 | (x>>6)); pRe->zInit[j++] = 0x80 | (x&0x3f); }else if( x<=0xffff ){ pRe->zInit[j++] = (unsigned char)(0xe0 | (x>>12)); pRe->zInit[j++] = 0x80 | ((x>>6)&0x3f); pRe->zInit[j++] = 0x80 | (x&0x3f); }else{ break; } } if( j>0 && pRe->zInit[j-1]==0 ) j--; |
︙ | ︙ | |||
701 702 703 704 705 706 707 | ** pattern and the second argument is the string. So, the SQL statements: ** ** A REGEXP B ** ** is implemented as regexp(B,A). */ static void re_sql_func( | | | > | > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | > | > > > > > > > > > > > > > > | 743 744 745 746 747 748 749 750 751 752 753 754 755 756 757 758 759 760 761 762 763 764 765 766 767 768 769 770 771 772 773 774 775 776 777 778 779 780 781 782 783 784 785 786 787 788 789 790 791 792 793 794 795 796 797 798 799 800 801 802 803 804 805 806 807 808 809 810 811 812 813 814 815 816 817 818 819 820 821 822 823 824 825 826 827 828 829 830 831 832 833 834 835 836 837 838 839 840 841 842 843 844 845 846 847 848 849 850 851 852 853 854 855 856 857 858 859 860 861 862 863 864 865 866 867 868 869 870 871 872 873 874 875 876 877 878 879 880 881 | ** pattern and the second argument is the string. So, the SQL statements: ** ** A REGEXP B ** ** is implemented as regexp(B,A). */ static void re_sql_func( sqlite3_context *context, int argc, sqlite3_value **argv ){ ReCompiled *pRe; /* Compiled regular expression */ const char *zPattern; /* The regular expression */ const unsigned char *zStr;/* String being searched */ const char *zErr; /* Compile error message */ int setAux = 0; /* True to invoke sqlite3_set_auxdata() */ (void)argc; /* Unused */ pRe = sqlite3_get_auxdata(context, 0); if( pRe==0 ){ zPattern = (const char*)sqlite3_value_text(argv[0]); if( zPattern==0 ) return; zErr = re_compile(&pRe, zPattern, sqlite3_user_data(context)!=0); if( zErr ){ re_free(pRe); sqlite3_result_error(context, zErr, -1); return; } if( pRe==0 ){ sqlite3_result_error_nomem(context); return; } setAux = 1; } zStr = (const unsigned char*)sqlite3_value_text(argv[1]); if( zStr!=0 ){ sqlite3_result_int(context, re_match(pRe, zStr, -1)); } if( setAux ){ sqlite3_set_auxdata(context, 0, pRe, (void(*)(void*))re_free); } } #if defined(SQLITE_DEBUG) /* ** This function is used for testing and debugging only. It is only available ** if the SQLITE_DEBUG compile-time option is used. ** ** Compile a regular expression and then convert the compiled expression into ** text and return that text. */ static void re_bytecode_func( sqlite3_context *context, int argc, sqlite3_value **argv ){ const char *zPattern; const char *zErr; ReCompiled *pRe; sqlite3_str *pStr; int i; int n; char *z; (void)argc; zPattern = (const char*)sqlite3_value_text(argv[0]); if( zPattern==0 ) return; zErr = re_compile(&pRe, zPattern, sqlite3_user_data(context)!=0); if( zErr ){ re_free(pRe); sqlite3_result_error(context, zErr, -1); return; } if( pRe==0 ){ sqlite3_result_error_nomem(context); return; } pStr = sqlite3_str_new(0); if( pStr==0 ) goto re_bytecode_func_err; if( pRe->nInit>0 ){ sqlite3_str_appendf(pStr, "INIT "); for(i=0; i<pRe->nInit; i++){ sqlite3_str_appendf(pStr, "%02x", pRe->zInit[i]); } sqlite3_str_appendf(pStr, "\n"); } for(i=0; (unsigned)i<pRe->nState; i++){ sqlite3_str_appendf(pStr, "%-8s %4d\n", ReOpName[(unsigned char)pRe->aOp[i]], pRe->aArg[i]); } n = sqlite3_str_length(pStr); z = sqlite3_str_finish(pStr); if( n==0 ){ sqlite3_free(z); }else{ sqlite3_result_text(context, z, n-1, sqlite3_free); } re_bytecode_func_err: re_free(pRe); } #endif /* SQLITE_DEBUG */ /* ** Invoke this routine to register the regexp() function with the ** SQLite database connection. */ #ifdef _WIN32 __declspec(dllexport) #endif int sqlite3_regexp_init( sqlite3 *db, char **pzErrMsg, const sqlite3_api_routines *pApi ){ int rc = SQLITE_OK; SQLITE_EXTENSION_INIT2(pApi); (void)pzErrMsg; /* Unused */ rc = sqlite3_create_function(db, "regexp", 2, SQLITE_UTF8|SQLITE_INNOCUOUS|SQLITE_DETERMINISTIC, 0, re_sql_func, 0, 0); if( rc==SQLITE_OK ){ /* The regexpi(PATTERN,STRING) function is a case-insensitive version ** of regexp(PATTERN,STRING). */ rc = sqlite3_create_function(db, "regexpi", 2, SQLITE_UTF8|SQLITE_INNOCUOUS|SQLITE_DETERMINISTIC, (void*)db, re_sql_func, 0, 0); #if defined(SQLITE_DEBUG) if( rc==SQLITE_OK ){ rc = sqlite3_create_function(db, "regexp_bytecode", 1, SQLITE_UTF8|SQLITE_INNOCUOUS|SQLITE_DETERMINISTIC, 0, re_bytecode_func, 0, 0); } #endif /* SQLITE_DEBUG */ } return rc; } |
Changes to ext/misc/rot13.c.
︙ | ︙ | |||
101 102 103 104 105 106 107 | sqlite3 *db, char **pzErrMsg, const sqlite3_api_routines *pApi ){ int rc = SQLITE_OK; SQLITE_EXTENSION_INIT2(pApi); (void)pzErrMsg; /* Unused parameter */ | | > | | 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 | sqlite3 *db, char **pzErrMsg, const sqlite3_api_routines *pApi ){ int rc = SQLITE_OK; SQLITE_EXTENSION_INIT2(pApi); (void)pzErrMsg; /* Unused parameter */ rc = sqlite3_create_function(db, "rot13", 1, SQLITE_UTF8|SQLITE_INNOCUOUS|SQLITE_DETERMINISTIC, 0, rot13func, 0, 0); if( rc==SQLITE_OK ){ rc = sqlite3_create_collation(db, "rot13", SQLITE_UTF8, 0, rot13CollFunc); } return rc; } |
Changes to ext/misc/scrub.c.
︙ | ︙ | |||
162 163 164 165 166 167 168 | SQLITE_OPEN_READWRITE | SQLITE_OPEN_URI | SQLITE_OPEN_PRIVATECACHE, 0); if( p->rcErr ){ scrubBackupErr(p, "cannot open source database: %s", sqlite3_errmsg(p->dbSrc)); return; } | | | 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 | SQLITE_OPEN_READWRITE | SQLITE_OPEN_URI | SQLITE_OPEN_PRIVATECACHE, 0); if( p->rcErr ){ scrubBackupErr(p, "cannot open source database: %s", sqlite3_errmsg(p->dbSrc)); return; } p->rcErr = sqlite3_exec(p->dbSrc, "SELECT 1 FROM sqlite_schema; BEGIN;", 0, 0, 0); if( p->rcErr ){ scrubBackupErr(p, "cannot start a read transaction on the source database: %s", sqlite3_errmsg(p->dbSrc)); return; } |
︙ | ︙ | |||
531 532 533 534 535 536 537 | /* Copy ptrmap pages */ n = scrubBackupInt32(&s.page1[52]); if( n ) scrubBackupPtrmap(&s); /* Copy all of the btrees */ scrubBackupBtree(&s, 1, 0); pStmt = scrubBackupPrepare(&s, s.dbSrc, | | | 531 532 533 534 535 536 537 538 539 540 541 542 543 544 545 | /* Copy ptrmap pages */ n = scrubBackupInt32(&s.page1[52]); if( n ) scrubBackupPtrmap(&s); /* Copy all of the btrees */ scrubBackupBtree(&s, 1, 0); pStmt = scrubBackupPrepare(&s, s.dbSrc, "SELECT rootpage FROM sqlite_schema WHERE coalesce(rootpage,0)>0"); if( pStmt==0 ) goto scrub_abort; while( sqlite3_step(pStmt)==SQLITE_ROW ){ i = (u32)sqlite3_column_int(pStmt, 0); scrubBackupBtree(&s, i, 0); } sqlite3_finalize(pStmt); |
︙ | ︙ |
Changes to ext/misc/series.c.
︙ | ︙ | |||
102 103 104 105 106 107 108 | ** (1) Allocate the series_vtab object and initialize all fields. ** ** (2) Tell SQLite (via the sqlite3_declare_vtab() interface) what the ** result set of queries against generate_series will look like. */ static int seriesConnect( sqlite3 *db, | | | | > > > > > | > | 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 | ** (1) Allocate the series_vtab object and initialize all fields. ** ** (2) Tell SQLite (via the sqlite3_declare_vtab() interface) what the ** result set of queries against generate_series will look like. */ static int seriesConnect( sqlite3 *db, void *pUnused, int argcUnused, const char *const*argvUnused, sqlite3_vtab **ppVtab, char **pzErrUnused ){ sqlite3_vtab *pNew; int rc; /* Column numbers */ #define SERIES_COLUMN_VALUE 0 #define SERIES_COLUMN_START 1 #define SERIES_COLUMN_STOP 2 #define SERIES_COLUMN_STEP 3 (void)pUnused; (void)argcUnused; (void)argvUnused; (void)pzErrUnused; rc = sqlite3_declare_vtab(db, "CREATE TABLE x(value,start hidden,stop hidden,step hidden)"); if( rc==SQLITE_OK ){ pNew = *ppVtab = sqlite3_malloc( sizeof(*pNew) ); if( pNew==0 ) return SQLITE_NOMEM; memset(pNew, 0, sizeof(*pNew)); sqlite3_vtab_config(db, SQLITE_VTAB_INNOCUOUS); } return rc; } /* ** This method is the destructor for series_cursor objects. */ static int seriesDisconnect(sqlite3_vtab *pVtab){ sqlite3_free(pVtab); return SQLITE_OK; } /* ** Constructor for a new series_cursor object. */ static int seriesOpen(sqlite3_vtab *pUnused, sqlite3_vtab_cursor **ppCursor){ series_cursor *pCur; (void)pUnused; pCur = sqlite3_malloc( sizeof(*pCur) ); if( pCur==0 ) return SQLITE_NOMEM; memset(pCur, 0, sizeof(*pCur)); *ppCursor = &pCur->base; return SQLITE_OK; } |
︙ | ︙ | |||
237 238 239 240 241 242 243 | ** is a bitmask showing which constraints are available: ** ** 1: start=VALUE ** 2: stop=VALUE ** 4: step=VALUE ** ** Also, if bit 8 is set, that means that the series should be output | | > | > | > > > > > | 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 | ** is a bitmask showing which constraints are available: ** ** 1: start=VALUE ** 2: stop=VALUE ** 4: step=VALUE ** ** Also, if bit 8 is set, that means that the series should be output ** in descending order rather than in ascending order. If bit 16 is ** set, then output must appear in ascending order. ** ** This routine should initialize the cursor and position it so that it ** is pointing at the first row, or pointing off the end of the table ** (so that seriesEof() will return true) if the table is empty. */ static int seriesFilter( sqlite3_vtab_cursor *pVtabCursor, int idxNum, const char *idxStrUnused, int argc, sqlite3_value **argv ){ series_cursor *pCur = (series_cursor *)pVtabCursor; int i = 0; (void)idxStrUnused; if( idxNum & 1 ){ pCur->mnValue = sqlite3_value_int64(argv[i++]); }else{ pCur->mnValue = 0; } if( idxNum & 2 ){ pCur->mxValue = sqlite3_value_int64(argv[i++]); }else{ pCur->mxValue = 0xffffffff; } if( idxNum & 4 ){ pCur->iStep = sqlite3_value_int64(argv[i++]); if( pCur->iStep==0 ){ pCur->iStep = 1; }else if( pCur->iStep<0 ){ pCur->iStep = -pCur->iStep; if( (idxNum & 16)==0 ) idxNum |= 8; } }else{ pCur->iStep = 1; } for(i=0; i<argc; i++){ if( sqlite3_value_type(argv[i])==SQLITE_NULL ){ /* If any of the constraints have a NULL value, then return no rows. ** See ticket https://www.sqlite.org/src/info/fac496b61722daf2 */ |
︙ | ︙ | |||
306 307 308 309 310 311 312 | ** ** (1) start = $value -- constraint exists ** (2) stop = $value -- constraint exists ** (4) step = $value -- constraint exists ** (8) output in descending order */ static int seriesBestIndex( | | > > > > > > > > > > > > > > > | | > > > > | 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340 341 342 343 344 345 346 347 348 349 350 351 352 353 354 355 356 357 358 359 360 361 362 363 364 365 366 367 368 369 370 371 372 373 374 375 376 377 378 379 380 381 382 383 384 385 386 387 388 389 390 391 392 393 394 395 396 397 398 399 400 401 | ** ** (1) start = $value -- constraint exists ** (2) stop = $value -- constraint exists ** (4) step = $value -- constraint exists ** (8) output in descending order */ static int seriesBestIndex( sqlite3_vtab *pVTab, sqlite3_index_info *pIdxInfo ){ int i, j; /* Loop over constraints */ int idxNum = 0; /* The query plan bitmask */ int bStartSeen = 0; /* EQ constraint seen on the START column */ int unusableMask = 0; /* Mask of unusable constraints */ int nArg = 0; /* Number of arguments that seriesFilter() expects */ int aIdx[3]; /* Constraints on start, stop, and step */ const struct sqlite3_index_constraint *pConstraint; /* This implementation assumes that the start, stop, and step columns ** are the last three columns in the virtual table. */ assert( SERIES_COLUMN_STOP == SERIES_COLUMN_START+1 ); assert( SERIES_COLUMN_STEP == SERIES_COLUMN_START+2 ); aIdx[0] = aIdx[1] = aIdx[2] = -1; pConstraint = pIdxInfo->aConstraint; for(i=0; i<pIdxInfo->nConstraint; i++, pConstraint++){ int iCol; /* 0 for start, 1 for stop, 2 for step */ int iMask; /* bitmask for those column */ if( pConstraint->iColumn<SERIES_COLUMN_START ) continue; iCol = pConstraint->iColumn - SERIES_COLUMN_START; assert( iCol>=0 && iCol<=2 ); iMask = 1 << iCol; if( iCol==0 ) bStartSeen = 1; if( pConstraint->usable==0 ){ unusableMask |= iMask; continue; }else if( pConstraint->op==SQLITE_INDEX_CONSTRAINT_EQ ){ idxNum |= iMask; aIdx[iCol] = i; } } for(i=0; i<3; i++){ if( (j = aIdx[i])>=0 ){ pIdxInfo->aConstraintUsage[j].argvIndex = ++nArg; pIdxInfo->aConstraintUsage[j].omit = !SQLITE_SERIES_CONSTRAINT_VERIFY; } } /* The current generate_column() implementation requires at least one ** argument (the START value). Legacy versions assumed START=0 if the ** first argument was omitted. Compile with -DZERO_ARGUMENT_GENERATE_SERIES ** to obtain the legacy behavior */ #ifndef ZERO_ARGUMENT_GENERATE_SERIES if( !bStartSeen ){ sqlite3_free(pVTab->zErrMsg); pVTab->zErrMsg = sqlite3_mprintf( "first argument to \"generate_series()\" missing or unusable"); return SQLITE_ERROR; } #endif if( (unusableMask & ~idxNum)!=0 ){ /* The start, stop, and step columns are inputs. Therefore if there ** are unusable constraints on any of start, stop, or step then ** this plan is unusable */ return SQLITE_CONSTRAINT; } if( (idxNum & 3)==3 ){ /* Both start= and stop= boundaries are available. This is the ** the preferred case */ pIdxInfo->estimatedCost = (double)(2 - ((idxNum&4)!=0)); pIdxInfo->estimatedRows = 1000; if( pIdxInfo->nOrderBy>=1 && pIdxInfo->aOrderBy[0].iColumn==0 ){ if( pIdxInfo->aOrderBy[0].desc ){ idxNum |= 8; }else{ idxNum |= 16; } pIdxInfo->orderByConsumed = 1; } }else{ /* If either boundary is missing, we have to generate a huge span ** of numbers. Make this case very expensive so that the query ** planner will work hard to avoid it. */ pIdxInfo->estimatedRows = 2147483647; |
︙ | ︙ | |||
393 394 395 396 397 398 399 400 401 402 403 404 405 406 407 408 409 410 411 412 413 414 | 0, /* xUpdate */ 0, /* xBegin */ 0, /* xSync */ 0, /* xCommit */ 0, /* xRollback */ 0, /* xFindMethod */ 0, /* xRename */ }; #endif /* SQLITE_OMIT_VIRTUALTABLE */ #ifdef _WIN32 __declspec(dllexport) #endif int sqlite3_series_init( sqlite3 *db, char **pzErrMsg, const sqlite3_api_routines *pApi ){ int rc = SQLITE_OK; SQLITE_EXTENSION_INIT2(pApi); #ifndef SQLITE_OMIT_VIRTUALTABLE | > > > > | | 425 426 427 428 429 430 431 432 433 434 435 436 437 438 439 440 441 442 443 444 445 446 447 448 449 450 451 452 453 454 455 456 457 458 459 | 0, /* xUpdate */ 0, /* xBegin */ 0, /* xSync */ 0, /* xCommit */ 0, /* xRollback */ 0, /* xFindMethod */ 0, /* xRename */ 0, /* xSavepoint */ 0, /* xRelease */ 0, /* xRollbackTo */ 0 /* xShadowName */ }; #endif /* SQLITE_OMIT_VIRTUALTABLE */ #ifdef _WIN32 __declspec(dllexport) #endif int sqlite3_series_init( sqlite3 *db, char **pzErrMsg, const sqlite3_api_routines *pApi ){ int rc = SQLITE_OK; SQLITE_EXTENSION_INIT2(pApi); #ifndef SQLITE_OMIT_VIRTUALTABLE if( sqlite3_libversion_number()<3008012 && pzErrMsg!=0 ){ *pzErrMsg = sqlite3_mprintf( "generate_series() requires SQLite 3.8.12 or later"); return SQLITE_ERROR; } rc = sqlite3_create_module(db, "generate_series", &seriesModule, 0); #endif return rc; } |
Changes to ext/misc/sha1.c.
︙ | ︙ | |||
35 36 37 38 39 40 41 | typedef struct SHA1Context SHA1Context; struct SHA1Context { unsigned int state[5]; unsigned int count[2]; unsigned char buffer[64]; }; | < < < < < < < < < < < < < < < < | 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 | typedef struct SHA1Context SHA1Context; struct SHA1Context { unsigned int state[5]; unsigned int count[2]; unsigned char buffer[64]; }; #define SHA_ROT(x,l,r) ((x) << (l) | (x) >> (r)) #define rol(x,k) SHA_ROT(x,k,32-(k)) #define ror(x,k) SHA_ROT(x,32-(k),k) #define blk0le(i) (block[i] = (ror(block[i],8)&0xFF00FF00) \ |(rol(block[i],8)&0x00FF00FF)) #define blk0be(i) block[i] #define blk(i) (block[i&15] = rol(block[(i+13)&15]^block[(i+8)&15] \ ^block[(i+2)&15]^block[i&15],1)) |
︙ | ︙ | |||
83 84 85 86 87 88 89 | z+=(((w|x)&y)|(w&x))+blk(i)+0x8F1BBCDC+rol(v,5);w=ror(w,2); #define R4(v,w,x,y,z,i) \ z+=(w^x^y)+blk(i)+0xCA62C1D6+rol(v,5);w=ror(w,2); /* * Hash a single 512-bit block. This is the core of the algorithm. */ | | | 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 | z+=(((w|x)&y)|(w&x))+blk(i)+0x8F1BBCDC+rol(v,5);w=ror(w,2); #define R4(v,w,x,y,z,i) \ z+=(w^x^y)+blk(i)+0xCA62C1D6+rol(v,5);w=ror(w,2); /* * Hash a single 512-bit block. This is the core of the algorithm. */ static void SHA1Transform(unsigned int state[5], const unsigned char buffer[64]){ unsigned int qq[5]; /* a, b, c, d, e; */ static int one = 1; unsigned int block[16]; memcpy(block, buffer, 64); memcpy(qq,state,5*sizeof(unsigned int)); #define a qq[0] |
︙ | ︙ | |||
393 394 395 396 397 398 399 | sqlite3 *db, char **pzErrMsg, const sqlite3_api_routines *pApi ){ int rc = SQLITE_OK; SQLITE_EXTENSION_INIT2(pApi); (void)pzErrMsg; /* Unused parameter */ | | > | | > | 377 378 379 380 381 382 383 384 385 386 387 388 389 390 391 392 393 | sqlite3 *db, char **pzErrMsg, const sqlite3_api_routines *pApi ){ int rc = SQLITE_OK; SQLITE_EXTENSION_INIT2(pApi); (void)pzErrMsg; /* Unused parameter */ rc = sqlite3_create_function(db, "sha1", 1, SQLITE_UTF8 | SQLITE_INNOCUOUS | SQLITE_DETERMINISTIC, 0, sha1Func, 0, 0); if( rc==SQLITE_OK ){ rc = sqlite3_create_function(db, "sha1_query", 1, SQLITE_UTF8|SQLITE_DIRECTONLY, 0, sha1QueryFunc, 0, 0); } return rc; } |
Changes to ext/misc/shathree.c.
︙ | ︙ | |||
15 16 17 18 19 20 21 | ** ** sha3(X,SIZE) ** sha3_query(Y,SIZE) ** ** The sha3(X) function computes the SHA3 hash of the input X, or NULL if ** X is NULL. ** | | > > > | 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 | ** ** sha3(X,SIZE) ** sha3_query(Y,SIZE) ** ** The sha3(X) function computes the SHA3 hash of the input X, or NULL if ** X is NULL. ** ** The sha3_query(Y) function evaluates all queries in the SQL statements of Y ** and returns a hash of their results. ** ** The SIZE argument is optional. If omitted, the SHA3-256 hash algorithm ** is used. If SIZE is included it must be one of the integers 224, 256, ** 384, or 512, to determine SHA3 hash variant that is computed. */ #include "sqlite3ext.h" SQLITE_EXTENSION_INIT1 #include <assert.h> #include <string.h> #include <stdarg.h> #ifndef SQLITE_AMALGAMATION typedef sqlite3_uint64 u64; #endif /* SQLITE_AMALGAMATION */ /****************************************************************************** ** The Hash Engine */ /* ** Macros to determine whether the machine is big or little endian, ** and whether or not that determination is run-time or compile-time. |
︙ | ︙ | |||
429 430 431 432 433 434 435 436 437 438 439 440 441 442 | */ static void SHA3Update( SHA3Context *p, const unsigned char *aData, unsigned int nData ){ unsigned int i = 0; #if SHA3_BYTEORDER==1234 if( (p->nLoaded % 8)==0 && ((aData - (const unsigned char*)0)&7)==0 ){ for(; i+7<nData; i+=8){ p->u.s[p->nLoaded/8] ^= *(u64*)&aData[i]; p->nLoaded += 8; if( p->nLoaded>=p->nRate ){ KeccakF1600Step(p); | > | 432 433 434 435 436 437 438 439 440 441 442 443 444 445 446 | */ static void SHA3Update( SHA3Context *p, const unsigned char *aData, unsigned int nData ){ unsigned int i = 0; if( aData==0 ) return; #if SHA3_BYTEORDER==1234 if( (p->nLoaded % 8)==0 && ((aData - (const unsigned char*)0)&7)==0 ){ for(; i+7<nData; i+=8){ p->u.s[p->nLoaded/8] ^= *(u64*)&aData[i]; p->nLoaded += 8; if( p->nLoaded>=p->nRate ){ KeccakF1600Step(p); |
︙ | ︙ | |||
523 524 525 526 527 528 529 | } sqlite3_result_blob(context, SHA3Final(&cx), iSize/8, SQLITE_TRANSIENT); } /* Compute a string using sqlite3_vsnprintf() with a maximum length ** of 50 bytes and add it to the hash. */ | | | 527 528 529 530 531 532 533 534 535 536 537 538 539 540 541 | } sqlite3_result_blob(context, SHA3Final(&cx), iSize/8, SQLITE_TRANSIENT); } /* Compute a string using sqlite3_vsnprintf() with a maximum length ** of 50 bytes and add it to the hash. */ static void sha3_step_vformat( SHA3Context *p, /* Add content to this context */ const char *zFormat, ... ){ va_list ap; int n; char zBuf[50]; |
︙ | ︙ | |||
617 618 619 620 621 622 623 | sqlite3_finalize(pStmt); sqlite3_result_error(context, zMsg, -1); sqlite3_free(zMsg); return; } nCol = sqlite3_column_count(pStmt); z = sqlite3_sql(pStmt); | > | | | > | 621 622 623 624 625 626 627 628 629 630 631 632 633 634 635 636 637 638 639 | sqlite3_finalize(pStmt); sqlite3_result_error(context, zMsg, -1); sqlite3_free(zMsg); return; } nCol = sqlite3_column_count(pStmt); z = sqlite3_sql(pStmt); if( z ){ n = (int)strlen(z); sha3_step_vformat(&cx,"S%d:",n); SHA3Update(&cx,(unsigned char*)z,n); } /* Compute a hash over the result of the query */ while( SQLITE_ROW==sqlite3_step(pStmt) ){ SHA3Update(&cx,(const unsigned char*)"R",1); for(i=0; i<nCol; i++){ switch( sqlite3_column_type(pStmt,i) ){ case SQLITE_NULL: { |
︙ | ︙ | |||
661 662 663 664 665 666 667 | x[0] = 'F'; SHA3Update(&cx,x,9); break; } case SQLITE_TEXT: { int n2 = sqlite3_column_bytes(pStmt, i); const unsigned char *z2 = sqlite3_column_text(pStmt, i); | | | | 667 668 669 670 671 672 673 674 675 676 677 678 679 680 681 682 683 684 685 686 687 688 | x[0] = 'F'; SHA3Update(&cx,x,9); break; } case SQLITE_TEXT: { int n2 = sqlite3_column_bytes(pStmt, i); const unsigned char *z2 = sqlite3_column_text(pStmt, i); sha3_step_vformat(&cx,"T%d:",n2); SHA3Update(&cx, z2, n2); break; } case SQLITE_BLOB: { int n2 = sqlite3_column_bytes(pStmt, i); const unsigned char *z2 = sqlite3_column_blob(pStmt, i); sha3_step_vformat(&cx,"B%d:",n2); SHA3Update(&cx, z2, n2); break; } } } } sqlite3_finalize(pStmt); |
︙ | ︙ | |||
692 693 694 695 696 697 698 | sqlite3 *db, char **pzErrMsg, const sqlite3_api_routines *pApi ){ int rc = SQLITE_OK; SQLITE_EXTENSION_INIT2(pApi); (void)pzErrMsg; /* Unused parameter */ | | > | | > | | > | | > | | 698 699 700 701 702 703 704 705 706 707 708 709 710 711 712 713 714 715 716 717 718 719 720 721 722 723 724 | sqlite3 *db, char **pzErrMsg, const sqlite3_api_routines *pApi ){ int rc = SQLITE_OK; SQLITE_EXTENSION_INIT2(pApi); (void)pzErrMsg; /* Unused parameter */ rc = sqlite3_create_function(db, "sha3", 1, SQLITE_UTF8 | SQLITE_INNOCUOUS | SQLITE_DETERMINISTIC, 0, sha3Func, 0, 0); if( rc==SQLITE_OK ){ rc = sqlite3_create_function(db, "sha3", 2, SQLITE_UTF8 | SQLITE_INNOCUOUS | SQLITE_DETERMINISTIC, 0, sha3Func, 0, 0); } if( rc==SQLITE_OK ){ rc = sqlite3_create_function(db, "sha3_query", 1, SQLITE_UTF8 | SQLITE_DIRECTONLY, 0, sha3QueryFunc, 0, 0); } if( rc==SQLITE_OK ){ rc = sqlite3_create_function(db, "sha3_query", 2, SQLITE_UTF8 | SQLITE_DIRECTONLY, 0, sha3QueryFunc, 0, 0); } return rc; } |
Changes to ext/misc/spellfix.c.
︙ | ︙ | |||
2065 2066 2067 2068 2069 2070 2071 2072 2073 2074 2075 2076 2077 2078 | pNew->zDbName = (char*)&pNew[1]; memcpy(pNew->zDbName, zDbName, nDbName+1); pNew->zTableName = sqlite3_mprintf("%s", zTableName); pNew->db = db; if( pNew->zTableName==0 ){ rc = SQLITE_NOMEM; }else{ rc = sqlite3_declare_vtab(db, "CREATE TABLE x(word,rank,distance,langid, " "score, matchlen, phonehash HIDDEN, " "top HIDDEN, scope HIDDEN, srchcnt HIDDEN, " "soundslike HIDDEN, command HIDDEN)" ); #define SPELLFIX_COL_WORD 0 | > | 2065 2066 2067 2068 2069 2070 2071 2072 2073 2074 2075 2076 2077 2078 2079 | pNew->zDbName = (char*)&pNew[1]; memcpy(pNew->zDbName, zDbName, nDbName+1); pNew->zTableName = sqlite3_mprintf("%s", zTableName); pNew->db = db; if( pNew->zTableName==0 ){ rc = SQLITE_NOMEM; }else{ sqlite3_vtab_config(db, SQLITE_VTAB_INNOCUOUS); rc = sqlite3_declare_vtab(db, "CREATE TABLE x(word,rank,distance,langid, " "score, matchlen, phonehash HIDDEN, " "top HIDDEN, scope HIDDEN, srchcnt HIDDEN, " "soundslike HIDDEN, command HIDDEN)" ); #define SPELLFIX_COL_WORD 0 |
︙ | ︙ |
Changes to ext/misc/sqlar.c.
︙ | ︙ | |||
13 14 15 16 17 18 19 20 21 22 23 24 25 26 | ** Utility functions sqlar_compress() and sqlar_uncompress(). Useful ** for working with sqlar archives and used by the shell tool's built-in ** sqlar support. */ #include "sqlite3ext.h" SQLITE_EXTENSION_INIT1 #include <zlib.h> /* ** Implementation of the "sqlar_compress(X)" SQL function. ** ** If the type of X is SQLITE_BLOB, and compressing that blob using ** zlib utility function compress() yields a smaller blob, return the ** compressed blob. Otherwise, return a copy of X. | > | 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 | ** Utility functions sqlar_compress() and sqlar_uncompress(). Useful ** for working with sqlar archives and used by the shell tool's built-in ** sqlar support. */ #include "sqlite3ext.h" SQLITE_EXTENSION_INIT1 #include <zlib.h> #include <assert.h> /* ** Implementation of the "sqlar_compress(X)" SQL function. ** ** If the type of X is SQLITE_BLOB, and compressing that blob using ** zlib utility function compress() yields a smaller blob, return the ** compressed blob. Otherwise, return a copy of X. |
︙ | ︙ | |||
86 87 88 89 90 91 92 | sz = sqlite3_value_int(argv[1]); if( sz<=0 || sz==(nData = sqlite3_value_bytes(argv[0])) ){ sqlite3_result_value(context, argv[0]); }else{ const Bytef *pData= sqlite3_value_blob(argv[0]); Bytef *pOut = sqlite3_malloc(sz); | > > | < | > | > | 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 | sz = sqlite3_value_int(argv[1]); if( sz<=0 || sz==(nData = sqlite3_value_bytes(argv[0])) ){ sqlite3_result_value(context, argv[0]); }else{ const Bytef *pData= sqlite3_value_blob(argv[0]); Bytef *pOut = sqlite3_malloc(sz); if( pOut==0 ){ sqlite3_result_error_nomem(context); }else if( Z_OK!=uncompress(pOut, &sz, pData, nData) ){ sqlite3_result_error(context, "error in uncompress()", -1); }else{ sqlite3_result_blob(context, pOut, sz, SQLITE_TRANSIENT); } sqlite3_free(pOut); } } #ifdef _WIN32 __declspec(dllexport) #endif int sqlite3_sqlar_init( sqlite3 *db, char **pzErrMsg, const sqlite3_api_routines *pApi ){ int rc = SQLITE_OK; SQLITE_EXTENSION_INIT2(pApi); (void)pzErrMsg; /* Unused parameter */ rc = sqlite3_create_function(db, "sqlar_compress", 1, SQLITE_UTF8|SQLITE_INNOCUOUS, 0, sqlarCompressFunc, 0, 0); if( rc==SQLITE_OK ){ rc = sqlite3_create_function(db, "sqlar_uncompress", 2, SQLITE_UTF8|SQLITE_INNOCUOUS, 0, sqlarUncompressFunc, 0, 0); } return rc; } |
Changes to ext/misc/stmt.c.
︙ | ︙ | |||
26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 | #endif SQLITE_EXTENSION_INIT1 #include <assert.h> #include <string.h> #ifndef SQLITE_OMIT_VIRTUALTABLE /* stmt_vtab is a subclass of sqlite3_vtab which will ** serve as the underlying representation of a stmt virtual table */ typedef struct stmt_vtab stmt_vtab; struct stmt_vtab { sqlite3_vtab base; /* Base class - must be first */ sqlite3 *db; /* Database connection for this stmt vtab */ }; /* stmt_cursor is a subclass of sqlite3_vtab_cursor which will ** serve as the underlying representation of a cursor that scans ** over rows of the result */ typedef struct stmt_cursor stmt_cursor; struct stmt_cursor { sqlite3_vtab_cursor base; /* Base class - must be first */ sqlite3 *db; /* Database connection for this cursor */ | > > > > > > > > > > < | | 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 | #endif SQLITE_EXTENSION_INIT1 #include <assert.h> #include <string.h> #ifndef SQLITE_OMIT_VIRTUALTABLE #define STMT_NUM_INTEGER_COLUMN 10 typedef struct StmtRow StmtRow; struct StmtRow { sqlite3_int64 iRowid; /* Rowid value */ char *zSql; /* column "sql" */ int aCol[STMT_NUM_INTEGER_COLUMN+1]; /* all other column values */ StmtRow *pNext; /* Next row to return */ }; /* stmt_vtab is a subclass of sqlite3_vtab which will ** serve as the underlying representation of a stmt virtual table */ typedef struct stmt_vtab stmt_vtab; struct stmt_vtab { sqlite3_vtab base; /* Base class - must be first */ sqlite3 *db; /* Database connection for this stmt vtab */ }; /* stmt_cursor is a subclass of sqlite3_vtab_cursor which will ** serve as the underlying representation of a cursor that scans ** over rows of the result */ typedef struct stmt_cursor stmt_cursor; struct stmt_cursor { sqlite3_vtab_cursor base; /* Base class - must be first */ sqlite3 *db; /* Database connection for this cursor */ StmtRow *pRow; /* Current row */ }; /* ** The stmtConnect() method is invoked to create a new ** stmt_vtab that describes the stmt virtual table. ** ** Think of this routine as the constructor for stmt_vtab objects. |
︙ | ︙ | |||
84 85 86 87 88 89 90 91 92 93 94 | #define STMT_COLUMN_NAIDX 6 /* SQLITE_STMTSTATUS_AUTOINDEX */ #define STMT_COLUMN_NSTEP 7 /* SQLITE_STMTSTATUS_VM_STEP */ #define STMT_COLUMN_REPREP 8 /* SQLITE_STMTSTATUS_REPREPARE */ #define STMT_COLUMN_RUN 9 /* SQLITE_STMTSTATUS_RUN */ #define STMT_COLUMN_MEM 10 /* SQLITE_STMTSTATUS_MEMUSED */ rc = sqlite3_declare_vtab(db, "CREATE TABLE x(sql,ncol,ro,busy,nscan,nsort,naidx,nstep," "reprep,run,mem)"); if( rc==SQLITE_OK ){ | > > > > | | 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 | #define STMT_COLUMN_NAIDX 6 /* SQLITE_STMTSTATUS_AUTOINDEX */ #define STMT_COLUMN_NSTEP 7 /* SQLITE_STMTSTATUS_VM_STEP */ #define STMT_COLUMN_REPREP 8 /* SQLITE_STMTSTATUS_REPREPARE */ #define STMT_COLUMN_RUN 9 /* SQLITE_STMTSTATUS_RUN */ #define STMT_COLUMN_MEM 10 /* SQLITE_STMTSTATUS_MEMUSED */ (void)pAux; (void)argc; (void)argv; (void)pzErr; rc = sqlite3_declare_vtab(db, "CREATE TABLE x(sql,ncol,ro,busy,nscan,nsort,naidx,nstep," "reprep,run,mem)"); if( rc==SQLITE_OK ){ pNew = sqlite3_malloc64( sizeof(*pNew) ); *ppVtab = (sqlite3_vtab*)pNew; if( pNew==0 ) return SQLITE_NOMEM; memset(pNew, 0, sizeof(*pNew)); pNew->db = db; } return rc; } |
︙ | ︙ | |||
110 111 112 113 114 115 116 | } /* ** Constructor for a new stmt_cursor object. */ static int stmtOpen(sqlite3_vtab *p, sqlite3_vtab_cursor **ppCursor){ stmt_cursor *pCur; | | > > > > > > > > > > > > | | | | | < | < < < < < < < < < | < < < < < < < < < < < < < < < < < | | > > > > > > > > > | > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | > > > | > > | 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 | } /* ** Constructor for a new stmt_cursor object. */ static int stmtOpen(sqlite3_vtab *p, sqlite3_vtab_cursor **ppCursor){ stmt_cursor *pCur; pCur = sqlite3_malloc64( sizeof(*pCur) ); if( pCur==0 ) return SQLITE_NOMEM; memset(pCur, 0, sizeof(*pCur)); pCur->db = ((stmt_vtab*)p)->db; *ppCursor = &pCur->base; return SQLITE_OK; } static void stmtCsrReset(stmt_cursor *pCur){ StmtRow *pRow = 0; StmtRow *pNext = 0; for(pRow=pCur->pRow; pRow; pRow=pNext){ pNext = pRow->pNext; sqlite3_free(pRow); } pCur->pRow = 0; } /* ** Destructor for a stmt_cursor. */ static int stmtClose(sqlite3_vtab_cursor *cur){ stmtCsrReset((stmt_cursor*)cur); sqlite3_free(cur); return SQLITE_OK; } /* ** Advance a stmt_cursor to its next row of output. */ static int stmtNext(sqlite3_vtab_cursor *cur){ stmt_cursor *pCur = (stmt_cursor*)cur; StmtRow *pNext = pCur->pRow->pNext; sqlite3_free(pCur->pRow); pCur->pRow = pNext; return SQLITE_OK; } /* ** Return values of columns for the row at which the stmt_cursor ** is currently pointing. */ static int stmtColumn( sqlite3_vtab_cursor *cur, /* The cursor */ sqlite3_context *ctx, /* First argument to sqlite3_result_...() */ int i /* Which column to return */ ){ stmt_cursor *pCur = (stmt_cursor*)cur; StmtRow *pRow = pCur->pRow; if( i==STMT_COLUMN_SQL ){ sqlite3_result_text(ctx, pRow->zSql, -1, SQLITE_TRANSIENT); }else{ sqlite3_result_int(ctx, pRow->aCol[i]); } return SQLITE_OK; } /* ** Return the rowid for the current row. In this implementation, the ** rowid is the same as the output value. */ static int stmtRowid(sqlite3_vtab_cursor *cur, sqlite_int64 *pRowid){ stmt_cursor *pCur = (stmt_cursor*)cur; *pRowid = pCur->pRow->iRowid; return SQLITE_OK; } /* ** Return TRUE if the cursor has been moved off of the last ** row of output. */ static int stmtEof(sqlite3_vtab_cursor *cur){ stmt_cursor *pCur = (stmt_cursor*)cur; return pCur->pRow==0; } /* ** This method is called to "rewind" the stmt_cursor object back ** to the first row of output. This method is always called at least ** once prior to any call to stmtColumn() or stmtRowid() or ** stmtEof(). */ static int stmtFilter( sqlite3_vtab_cursor *pVtabCursor, int idxNum, const char *idxStr, int argc, sqlite3_value **argv ){ stmt_cursor *pCur = (stmt_cursor *)pVtabCursor; sqlite3_stmt *p = 0; sqlite3_int64 iRowid = 1; StmtRow **ppRow = 0; (void)idxNum; (void)idxStr; (void)argc; (void)argv; stmtCsrReset(pCur); ppRow = &pCur->pRow; for(p=sqlite3_next_stmt(pCur->db, 0); p; p=sqlite3_next_stmt(pCur->db, p)){ const char *zSql = sqlite3_sql(p); sqlite3_int64 nSql = zSql ? strlen(zSql)+1 : 0; StmtRow *pNew = (StmtRow*)sqlite3_malloc64(sizeof(StmtRow) + nSql); if( pNew==0 ) return SQLITE_NOMEM; memset(pNew, 0, sizeof(StmtRow)); if( zSql ){ pNew->zSql = (char*)&pNew[1]; memcpy(pNew->zSql, zSql, nSql); } pNew->aCol[STMT_COLUMN_NCOL] = sqlite3_column_count(p); pNew->aCol[STMT_COLUMN_RO] = sqlite3_stmt_readonly(p); pNew->aCol[STMT_COLUMN_BUSY] = sqlite3_stmt_busy(p); pNew->aCol[STMT_COLUMN_NSCAN] = sqlite3_stmt_status( p, SQLITE_STMTSTATUS_FULLSCAN_STEP, 0 ); pNew->aCol[STMT_COLUMN_NSORT] = sqlite3_stmt_status( p, SQLITE_STMTSTATUS_SORT, 0 ); pNew->aCol[STMT_COLUMN_NAIDX] = sqlite3_stmt_status( p, SQLITE_STMTSTATUS_AUTOINDEX, 0 ); pNew->aCol[STMT_COLUMN_NSTEP] = sqlite3_stmt_status( p, SQLITE_STMTSTATUS_VM_STEP, 0 ); pNew->aCol[STMT_COLUMN_REPREP] = sqlite3_stmt_status( p, SQLITE_STMTSTATUS_REPREPARE, 0 ); pNew->aCol[STMT_COLUMN_RUN] = sqlite3_stmt_status( p, SQLITE_STMTSTATUS_RUN, 0 ); pNew->aCol[STMT_COLUMN_MEM] = sqlite3_stmt_status( p, SQLITE_STMTSTATUS_MEMUSED, 0 ); pNew->iRowid = iRowid++; *ppRow = pNew; ppRow = &pNew->pNext; } return SQLITE_OK; } /* ** SQLite will invoke this method one or more times while planning a query ** that uses the stmt virtual table. This routine needs to create ** a query plan for each invocation and compute an estimated cost for that ** plan. */ static int stmtBestIndex( sqlite3_vtab *tab, sqlite3_index_info *pIdxInfo ){ (void)tab; pIdxInfo->estimatedCost = (double)500; pIdxInfo->estimatedRows = 500; return SQLITE_OK; } /* ** This following structure defines all the methods for the |
︙ | ︙ |
Changes to ext/misc/totype.c.
︙ | ︙ | |||
498 499 500 501 502 503 504 | sqlite3 *db, char **pzErrMsg, const sqlite3_api_routines *pApi ){ int rc = SQLITE_OK; SQLITE_EXTENSION_INIT2(pApi); (void)pzErrMsg; /* Unused parameter */ | | > | | > | | 498 499 500 501 502 503 504 505 506 507 508 509 510 511 512 513 514 | sqlite3 *db, char **pzErrMsg, const sqlite3_api_routines *pApi ){ int rc = SQLITE_OK; SQLITE_EXTENSION_INIT2(pApi); (void)pzErrMsg; /* Unused parameter */ rc = sqlite3_create_function(db, "tointeger", 1, SQLITE_UTF8 | SQLITE_DETERMINISTIC | SQLITE_INNOCUOUS, 0, tointegerFunc, 0, 0); if( rc==SQLITE_OK ){ rc = sqlite3_create_function(db, "toreal", 1, SQLITE_UTF8 | SQLITE_DETERMINISTIC | SQLITE_INNOCUOUS, 0, torealFunc, 0, 0); } return rc; } |
Added ext/misc/uint.c.
> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 | /* ** 2020-04-14 ** ** The author disclaims copyright to this source code. In place of ** a legal notice, here is a blessing: ** ** May you do good and not evil. ** May you find forgiveness for yourself and forgive others. ** May you share freely, never taking more than you give. ** ****************************************************************************** ** ** This SQLite extension implements the UINT collating sequence. ** ** UINT works like BINARY for text, except that embedded strings ** of digits compare in numeric order. ** ** * Leading zeros are handled properly, in the sense that ** they do not mess of the maginitude comparison of embedded ** strings of digits. "x00123y" is equal to "x123y". ** ** * Only unsigned integers are recognized. Plus and minus ** signs are ignored. Decimal points and exponential notation ** are ignored. ** ** * Embedded integers can be of arbitrary length. Comparison ** is *not* limited integers that can be expressed as a ** 64-bit machine integer. */ #include "sqlite3ext.h" SQLITE_EXTENSION_INIT1 #include <assert.h> #include <string.h> #include <ctype.h> /* ** Compare text in lexicographic order, except strings of digits ** compare in numeric order. */ static int uintCollFunc( void *notUsed, int nKey1, const void *pKey1, int nKey2, const void *pKey2 ){ const unsigned char *zA = (const unsigned char*)pKey1; const unsigned char *zB = (const unsigned char*)pKey2; int i=0, j=0, x; (void)notUsed; while( i<nKey1 && j<nKey2 ){ x = zA[i] - zB[j]; if( isdigit(zA[i]) ){ int k; if( !isdigit(zB[j]) ) return x; while( i<nKey1 && zA[i]=='0' ){ i++; } while( j<nKey2 && zB[j]=='0' ){ j++; } k = 0; while( i+k<nKey1 && isdigit(zA[i+k]) && j+k<nKey2 && isdigit(zB[j+k]) ){ k++; } if( i+k<nKey1 && isdigit(zA[i+k]) ){ return +1; }else if( j+k<nKey2 && isdigit(zB[j+k]) ){ return -1; }else{ x = memcmp(zA+i, zB+j, k); if( x ) return x; i += k; j += k; } }else if( x ){ return x; }else{ i++; j++; } } return (nKey1 - i) - (nKey2 - j); } #ifdef _WIN32 __declspec(dllexport) #endif int sqlite3_uint_init( sqlite3 *db, char **pzErrMsg, const sqlite3_api_routines *pApi ){ SQLITE_EXTENSION_INIT2(pApi); (void)pzErrMsg; /* Unused parameter */ return sqlite3_create_collation(db, "uint", SQLITE_UTF8, 0, uintCollFunc); } |
Changes to ext/misc/unionvtab.c.
︙ | ︙ | |||
246 247 248 249 250 251 252 | /* ** If *pRc is other than SQLITE_OK when this function is called, it ** always returns NULL. Otherwise, it attempts to allocate and return ** a pointer to nByte bytes of zeroed memory. If the memory allocation ** is attempted but fails, NULL is returned and *pRc is set to ** SQLITE_NOMEM. */ | | | | | | | 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 | /* ** If *pRc is other than SQLITE_OK when this function is called, it ** always returns NULL. Otherwise, it attempts to allocate and return ** a pointer to nByte bytes of zeroed memory. If the memory allocation ** is attempted but fails, NULL is returned and *pRc is set to ** SQLITE_NOMEM. */ static void *unionMalloc(int *pRc, sqlite3_int64 nByte){ void *pRet; assert( nByte>0 ); if( *pRc==SQLITE_OK ){ pRet = sqlite3_malloc64(nByte); if( pRet ){ memset(pRet, 0, (size_t)nByte); }else{ *pRc = SQLITE_NOMEM; } }else{ pRet = 0; } return pRet; } /* ** If *pRc is other than SQLITE_OK when this function is called, it ** always returns NULL. Otherwise, it attempts to allocate and return ** a copy of the nul-terminated string passed as the second argument. ** If the allocation is attempted but fails, NULL is returned and *pRc is ** set to SQLITE_NOMEM. */ static char *unionStrdup(int *pRc, const char *zIn){ char *zRet = 0; if( zIn ){ sqlite3_int64 nByte = strlen(zIn) + 1; zRet = unionMalloc(pRc, nByte); if( zRet ){ memcpy(zRet, zIn, (size_t)nByte); } } return zRet; } /* ** If the first character of the string passed as the only argument to this |
︙ | ︙ | |||
935 936 937 938 939 940 941 | sqlite3_int64 iMin = sqlite3_column_int64(pStmt, 2); sqlite3_int64 iMax = sqlite3_column_int64(pStmt, 3); UnionSrc *pSrc; /* Grow the pTab->aSrc[] array if required. */ if( nAlloc<=pTab->nSrc ){ int nNew = nAlloc ? nAlloc*2 : 8; | | | 935 936 937 938 939 940 941 942 943 944 945 946 947 948 949 | sqlite3_int64 iMin = sqlite3_column_int64(pStmt, 2); sqlite3_int64 iMax = sqlite3_column_int64(pStmt, 3); UnionSrc *pSrc; /* Grow the pTab->aSrc[] array if required. */ if( nAlloc<=pTab->nSrc ){ int nNew = nAlloc ? nAlloc*2 : 8; UnionSrc *aNew = (UnionSrc*)sqlite3_realloc64( pTab->aSrc, nNew*sizeof(UnionSrc) ); if( aNew==0 ){ rc = SQLITE_NOMEM; break; }else{ memset(&aNew[pTab->nSrc], 0, (nNew-pTab->nSrc)*sizeof(UnionSrc)); |
︙ | ︙ |
Added ext/misc/urifuncs.c.
> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 | /* ** 2020-01-11 ** ** The author disclaims copyright to this source code. In place of ** a legal notice, here is a blessing: ** ** May you do good and not evil. ** May you find forgiveness for yourself and forgive others. ** May you share freely, never taking more than you give. ** ****************************************************************************** ** ** This SQLite extension implements various SQL functions used to access ** the following SQLite C-language APIs: ** ** sqlite3_uri_parameter() ** sqlite3_uri_boolean() ** sqlite3_uri_int64() ** sqlite3_uri_key() ** sqlite3_filename_database() ** sqlite3_filename_journal() ** sqlite3_filename_wal() ** sqlite3_db_filename() ** ** These SQL functions are for testing and demonstration purposes only. ** ** */ #include "sqlite3ext.h" SQLITE_EXTENSION_INIT1 #include <assert.h> #include <string.h> /* ** SQL function: sqlite3_db_filename(SCHEMA) ** ** Return the filename corresponding to SCHEMA. */ static void func_db_filename( sqlite3_context *context, int argc, sqlite3_value **argv ){ const char *zSchema = (const char*)sqlite3_value_text(argv[0]); sqlite3 *db = sqlite3_context_db_handle(context); const char *zFile = sqlite3_db_filename(db, zSchema); sqlite3_result_text(context, zFile, -1, SQLITE_TRANSIENT); } /* ** SQL function: sqlite3_uri_parameter(SCHEMA,NAME) ** ** Return the value of the NAME query parameter to the database for SCHEMA */ static void func_uri_parameter( sqlite3_context *context, int argc, sqlite3_value **argv ){ const char *zSchema = (const char*)sqlite3_value_text(argv[0]); sqlite3 *db = sqlite3_context_db_handle(context); const char *zName = (const char*)sqlite3_value_text(argv[1]); const char *zFile = sqlite3_db_filename(db, zSchema); const char *zRes = sqlite3_uri_parameter(zFile, zName); sqlite3_result_text(context, zRes, -1, SQLITE_TRANSIENT); } /* ** SQL function: sqlite3_uri_boolean(SCHEMA,NAME,DEFAULT) ** ** Return the boolean value of the NAME query parameter to ** the database for SCHEMA */ static void func_uri_boolean( sqlite3_context *context, int argc, sqlite3_value **argv ){ const char *zSchema = (const char*)sqlite3_value_text(argv[0]); sqlite3 *db = sqlite3_context_db_handle(context); const char *zName = (const char*)sqlite3_value_text(argv[1]); const char *zFile = sqlite3_db_filename(db, zSchema); int iDflt = sqlite3_value_int(argv[2]); int iRes = sqlite3_uri_boolean(zFile, zName, iDflt); sqlite3_result_int(context, iRes); } /* ** SQL function: sqlite3_uri_key(SCHEMA,N) ** ** Return the name of the Nth query parameter */ static void func_uri_key( sqlite3_context *context, int argc, sqlite3_value **argv ){ const char *zSchema = (const char*)sqlite3_value_text(argv[0]); sqlite3 *db = sqlite3_context_db_handle(context); int N = sqlite3_value_int(argv[1]); const char *zFile = sqlite3_db_filename(db, zSchema); const char *zRes = sqlite3_uri_key(zFile, N); sqlite3_result_text(context, zRes, -1, SQLITE_TRANSIENT); } /* ** SQL function: sqlite3_uri_int64(SCHEMA,NAME,DEFAULT) ** ** Return the int64 value of the NAME query parameter to ** the database for SCHEMA */ static void func_uri_int64( sqlite3_context *context, int argc, sqlite3_value **argv ){ const char *zSchema = (const char*)sqlite3_value_text(argv[0]); sqlite3 *db = sqlite3_context_db_handle(context); const char *zName = (const char*)sqlite3_value_text(argv[1]); const char *zFile = sqlite3_db_filename(db, zSchema); sqlite3_int64 iDflt = sqlite3_value_int64(argv[2]); sqlite3_int64 iRes = sqlite3_uri_int64(zFile, zName, iDflt); sqlite3_result_int64(context, iRes); } /* ** SQL function: sqlite3_filename_database(SCHEMA) ** ** Return the database filename for SCHEMA */ static void func_filename_database( sqlite3_context *context, int argc, sqlite3_value **argv ){ const char *zSchema = (const char*)sqlite3_value_text(argv[0]); sqlite3 *db = sqlite3_context_db_handle(context); const char *zFile = sqlite3_db_filename(db, zSchema); const char *zRes = zFile ? sqlite3_filename_database(zFile) : 0; sqlite3_result_text(context, zRes, -1, SQLITE_TRANSIENT); } /* ** SQL function: sqlite3_filename_journal(SCHEMA) ** ** Return the rollback journal filename for SCHEMA */ static void func_filename_journal( sqlite3_context *context, int argc, sqlite3_value **argv ){ const char *zSchema = (const char*)sqlite3_value_text(argv[0]); sqlite3 *db = sqlite3_context_db_handle(context); const char *zFile = sqlite3_db_filename(db, zSchema); const char *zRes = zFile ? sqlite3_filename_journal(zFile) : 0; sqlite3_result_text(context, zRes, -1, SQLITE_TRANSIENT); } /* ** SQL function: sqlite3_filename_wal(SCHEMA) ** ** Return the WAL filename for SCHEMA */ static void func_filename_wal( sqlite3_context *context, int argc, sqlite3_value **argv ){ const char *zSchema = (const char*)sqlite3_value_text(argv[0]); sqlite3 *db = sqlite3_context_db_handle(context); const char *zFile = sqlite3_db_filename(db, zSchema); const char *zRes = zFile ? sqlite3_filename_wal(zFile) : 0; sqlite3_result_text(context, zRes, -1, SQLITE_TRANSIENT); } #ifdef _WIN32 __declspec(dllexport) #endif int sqlite3_urifuncs_init( sqlite3 *db, char **pzErrMsg, const sqlite3_api_routines *pApi ){ static const struct { const char *zFuncName; int nArg; void (*xFunc)(sqlite3_context*,int,sqlite3_value**); } aFunc[] = { { "sqlite3_db_filename", 1, func_db_filename }, { "sqlite3_uri_parameter", 2, func_uri_parameter }, { "sqlite3_uri_boolean", 3, func_uri_boolean }, { "sqlite3_uri_int64", 3, func_uri_int64 }, { "sqlite3_uri_key", 2, func_uri_key }, { "sqlite3_filename_database", 1, func_filename_database }, { "sqlite3_filename_journal", 1, func_filename_journal }, { "sqlite3_filename_wal", 1, func_filename_wal }, }; int rc = SQLITE_OK; int i; SQLITE_EXTENSION_INIT2(pApi); (void)pzErrMsg; /* Unused parameter */ for(i=0; rc==SQLITE_OK && i<sizeof(aFunc)/sizeof(aFunc[0]); i++){ rc = sqlite3_create_function(db, aFunc[i].zFuncName, aFunc[i].nArg, SQLITE_UTF8, 0, aFunc[i].xFunc, 0, 0); } return rc; } |
Added ext/misc/uuid.c.
> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 | /* ** 2019-10-23 ** ** The author disclaims copyright to this source code. In place of ** a legal notice, here is a blessing: ** ** May you do good and not evil. ** May you find forgiveness for yourself and forgive others. ** May you share freely, never taking more than you give. ** ****************************************************************************** ** ** This SQLite extension implements functions that handling RFC-4122 UUIDs ** Three SQL functions are implemented: ** ** uuid() - generate a version 4 UUID as a string ** uuid_str(X) - convert a UUID X into a well-formed UUID string ** uuid_blob(X) - convert a UUID X into a 16-byte blob ** ** The output from uuid() and uuid_str(X) are always well-formed RFC-4122 ** UUID strings in this format: ** ** xxxxxxxx-xxxx-Mxxx-Nxxx-xxxxxxxxxxxx ** ** All of the 'x', 'M', and 'N' values are lower-case hexadecimal digits. ** The M digit indicates the "version". For uuid()-generated UUIDs, the ** version is always "4" (a random UUID). The upper three bits of N digit ** are the "variant". This library only supports variant 1 (indicated ** by values of N between '8' and 'b') as those are overwhelming the most ** common. Other variants are for legacy compatibility only. ** ** The output of uuid_blob(X) is always a 16-byte blob. The UUID input ** string is converted in network byte order (big-endian) in accordance ** with RFC-4122 specifications for variant-1 UUIDs. Note that network ** byte order is *always* used, even if the input self-identifies as a ** variant-2 UUID. ** ** The input X to the uuid_str() and uuid_blob() functions can be either ** a string or a BLOB. If it is a BLOB it must be exactly 16 bytes in ** length or else a NULL is returned. If the input is a string it must ** consist of 32 hexadecimal digits, upper or lower case, optionally ** surrounded by {...} and with optional "-" characters interposed in the ** middle. The flexibility of input is inspired by the PostgreSQL ** implementation of UUID functions that accept in all of the following ** formats: ** ** A0EEBC99-9C0B-4EF8-BB6D-6BB9BD380A11 ** {a0eebc99-9c0b-4ef8-bb6d-6bb9bd380a11} ** a0eebc999c0b4ef8bb6d6bb9bd380a11 ** a0ee-bc99-9c0b-4ef8-bb6d-6bb9-bd38-0a11 ** {a0eebc99-9c0b4ef8-bb6d6bb9-bd380a11} ** ** If any of the above inputs are passed into uuid_str(), the output will ** always be in the canonical RFC-4122 format: ** ** a0eebc99-9c0b-4ef8-bb6d-6bb9bd380a11 ** ** If the X input string has too few or too many digits or contains ** stray characters other than {, }, or -, then NULL is returned. */ #include "sqlite3ext.h" SQLITE_EXTENSION_INIT1 #include <assert.h> #include <string.h> #include <ctype.h> #if !defined(SQLITE_ASCII) && !defined(SQLITE_EBCDIC) # define SQLITE_ASCII 1 #endif /* ** Translate a single byte of Hex into an integer. ** This routine only works if h really is a valid hexadecimal ** character: 0..9a..fA..F */ static unsigned char sqlite3UuidHexToInt(int h){ assert( (h>='0' && h<='9') || (h>='a' && h<='f') || (h>='A' && h<='F') ); #ifdef SQLITE_ASCII h += 9*(1&(h>>6)); #endif #ifdef SQLITE_EBCDIC h += 9*(1&~(h>>4)); #endif return (unsigned char)(h & 0xf); } /* ** Convert a 16-byte BLOB into a well-formed RFC-4122 UUID. The output ** buffer zStr should be at least 37 bytes in length. The output will ** be zero-terminated. */ static void sqlite3UuidBlobToStr( const unsigned char *aBlob, /* Input blob */ unsigned char *zStr /* Write the answer here */ ){ static const char zDigits[] = "0123456789abcdef"; int i, k; unsigned char x; k = 0; for(i=0, k=0x550; i<16; i++, k=k>>1){ if( k&1 ){ zStr[0] = '-'; zStr++; } x = aBlob[i]; zStr[0] = zDigits[x>>4]; zStr[1] = zDigits[x&0xf]; zStr += 2; } *zStr = 0; } /* ** Attempt to parse a zero-terminated input string zStr into a binary ** UUID. Return 0 on success, or non-zero if the input string is not ** parsable. */ static int sqlite3UuidStrToBlob( const unsigned char *zStr, /* Input string */ unsigned char *aBlob /* Write results here */ ){ int i; if( zStr[0]=='{' ) zStr++; for(i=0; i<16; i++){ if( zStr[0]=='-' ) zStr++; if( isxdigit(zStr[0]) && isxdigit(zStr[1]) ){ aBlob[i] = (sqlite3UuidHexToInt(zStr[0])<<4) + sqlite3UuidHexToInt(zStr[1]); zStr += 2; }else{ return 1; } } if( zStr[0]=='}' ) zStr++; return zStr[0]!=0; } /* ** Render sqlite3_value pIn as a 16-byte UUID blob. Return a pointer ** to the blob, or NULL if the input is not well-formed. */ static const unsigned char *sqlite3UuidInputToBlob( sqlite3_value *pIn, /* Input text */ unsigned char *pBuf /* output buffer */ ){ switch( sqlite3_value_type(pIn) ){ case SQLITE_TEXT: { const unsigned char *z = sqlite3_value_text(pIn); if( sqlite3UuidStrToBlob(z, pBuf) ) return 0; return pBuf; } case SQLITE_BLOB: { int n = sqlite3_value_bytes(pIn); return n==16 ? sqlite3_value_blob(pIn) : 0; } default: { return 0; } } } /* Implementation of uuid() */ static void sqlite3UuidFunc( sqlite3_context *context, int argc, sqlite3_value **argv ){ unsigned char aBlob[16]; unsigned char zStr[37]; (void)argc; (void)argv; sqlite3_randomness(16, aBlob); aBlob[6] = (aBlob[6]&0x0f) + 0x40; aBlob[8] = (aBlob[8]&0x3f) + 0x80; sqlite3UuidBlobToStr(aBlob, zStr); sqlite3_result_text(context, (char*)zStr, 36, SQLITE_TRANSIENT); } /* Implementation of uuid_str() */ static void sqlite3UuidStrFunc( sqlite3_context *context, int argc, sqlite3_value **argv ){ unsigned char aBlob[16]; unsigned char zStr[37]; const unsigned char *pBlob; (void)argc; pBlob = sqlite3UuidInputToBlob(argv[0], aBlob); if( pBlob==0 ) return; sqlite3UuidBlobToStr(pBlob, zStr); sqlite3_result_text(context, (char*)zStr, 36, SQLITE_TRANSIENT); } /* Implementation of uuid_blob() */ static void sqlite3UuidBlobFunc( sqlite3_context *context, int argc, sqlite3_value **argv ){ unsigned char aBlob[16]; const unsigned char *pBlob; (void)argc; pBlob = sqlite3UuidInputToBlob(argv[0], aBlob); if( pBlob==0 ) return; sqlite3_result_blob(context, pBlob, 16, SQLITE_TRANSIENT); } #ifdef _WIN32 __declspec(dllexport) #endif int sqlite3_uuid_init( sqlite3 *db, char **pzErrMsg, const sqlite3_api_routines *pApi ){ int rc = SQLITE_OK; SQLITE_EXTENSION_INIT2(pApi); (void)pzErrMsg; /* Unused parameter */ rc = sqlite3_create_function(db, "uuid", 0, SQLITE_UTF8|SQLITE_INNOCUOUS, 0, sqlite3UuidFunc, 0, 0); if( rc==SQLITE_OK ){ rc = sqlite3_create_function(db, "uuid_str", 1, SQLITE_UTF8|SQLITE_INNOCUOUS|SQLITE_DETERMINISTIC, 0, sqlite3UuidStrFunc, 0, 0); } if( rc==SQLITE_OK ){ rc = sqlite3_create_function(db, "uuid_blob", 1, SQLITE_UTF8|SQLITE_INNOCUOUS|SQLITE_DETERMINISTIC, 0, sqlite3UuidBlobFunc, 0, 0); } return rc; } |
Changes to ext/misc/vfslog.c.
︙ | ︙ | |||
272 273 274 275 276 277 278 | if( nName>8 && strcmp(zFilename+nName-8,"-journal")==0 ){ nName -= 8; isJournal = 1; }else if( nName>12 && sqlite3_strglob("-mj??????9??", zFilename+nName-12)==0 ){ return 0; /* Do not log master journal files */ } | | | 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 | if( nName>8 && strcmp(zFilename+nName-8,"-journal")==0 ){ nName -= 8; isJournal = 1; }else if( nName>12 && sqlite3_strglob("-mj??????9??", zFilename+nName-12)==0 ){ return 0; /* Do not log master journal files */ } pTemp = sqlite3_malloc64( sizeof(*pLog)*2 + nName + 60 ); if( pTemp==0 ) return 0; pMutex = sqlite3_mutex_alloc(SQLITE_MUTEX_STATIC_MASTER); sqlite3_mutex_enter(pMutex); for(pLog=allLogs; pLog; pLog=pLog->pNext){ if( pLog->nFilename==nName && !memcmp(pLog->zFilename, zFilename, nName) ){ break; } |
︙ | ︙ | |||
750 751 752 753 754 755 756 757 758 759 | } /* ** Register debugvfs as the default VFS for this process. */ int sqlite3_register_vfslog(const char *zArg){ vlog_vfs.pVfs = sqlite3_vfs_find(0); vlog_vfs.base.szOsFile = sizeof(VLogFile) + vlog_vfs.pVfs->szOsFile; return sqlite3_vfs_register(&vlog_vfs.base, 1); } | > | 750 751 752 753 754 755 756 757 758 759 760 | } /* ** Register debugvfs as the default VFS for this process. */ int sqlite3_register_vfslog(const char *zArg){ vlog_vfs.pVfs = sqlite3_vfs_find(0); if( vlog_vfs.pVfs==0 ) return SQLITE_ERROR; vlog_vfs.base.szOsFile = sizeof(VLogFile) + vlog_vfs.pVfs->szOsFile; return sqlite3_vfs_register(&vlog_vfs.base, 1); } |
Changes to ext/misc/vfsstat.c.
︙ | ︙ | |||
779 780 781 782 783 784 785 | /* ** This routine is an sqlite3_auto_extension() callback, invoked to register ** the vfsstat virtual table for all new database connections. */ static int vstatRegister( sqlite3 *db, | | | | 779 780 781 782 783 784 785 786 787 788 789 790 791 792 793 794 | /* ** This routine is an sqlite3_auto_extension() callback, invoked to register ** the vfsstat virtual table for all new database connections. */ static int vstatRegister( sqlite3 *db, char **pzErrMsg, const sqlite3_api_routines *pThunk ){ return sqlite3_create_module(db, "vfsstat", &VfsStatModule, 0); } #ifdef _WIN32 __declspec(dllexport) #endif |
︙ | ︙ | |||
802 803 804 805 806 807 808 809 810 811 | sqlite3 *db, char **pzErrMsg, const sqlite3_api_routines *pApi ){ int rc = SQLITE_OK; SQLITE_EXTENSION_INIT2(pApi); vstat_vfs.pVfs = sqlite3_vfs_find(0); vstat_vfs.base.szOsFile = sizeof(VStatFile) + vstat_vfs.pVfs->szOsFile; rc = sqlite3_vfs_register(&vstat_vfs.base, 1); if( rc==SQLITE_OK ){ | > > > | > | 802 803 804 805 806 807 808 809 810 811 812 813 814 815 816 817 818 819 820 | sqlite3 *db, char **pzErrMsg, const sqlite3_api_routines *pApi ){ int rc = SQLITE_OK; SQLITE_EXTENSION_INIT2(pApi); vstat_vfs.pVfs = sqlite3_vfs_find(0); if( vstat_vfs.pVfs==0 ) return SQLITE_ERROR; vstat_vfs.base.szOsFile = sizeof(VStatFile) + vstat_vfs.pVfs->szOsFile; rc = sqlite3_vfs_register(&vstat_vfs.base, 1); if( rc==SQLITE_OK ){ rc = vstatRegister(db, pzErrMsg, pApi); if( rc==SQLITE_OK ){ rc = sqlite3_auto_extension((void(*)(void))vstatRegister); } } if( rc==SQLITE_OK ) rc = SQLITE_OK_LOAD_PERMANENTLY; return rc; } |
Changes to ext/misc/wholenumber.c.
︙ | ︙ | |||
46 47 48 49 50 51 52 53 54 55 56 57 58 59 | sqlite3_vtab **ppVtab, char **pzErr ){ sqlite3_vtab *pNew; pNew = *ppVtab = sqlite3_malloc( sizeof(*pNew) ); if( pNew==0 ) return SQLITE_NOMEM; sqlite3_declare_vtab(db, "CREATE TABLE x(value)"); memset(pNew, 0, sizeof(*pNew)); return SQLITE_OK; } /* Note that for this virtual table, the xCreate and xConnect ** methods are identical. */ static int wholenumberDisconnect(sqlite3_vtab *pVtab){ | > | 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 | sqlite3_vtab **ppVtab, char **pzErr ){ sqlite3_vtab *pNew; pNew = *ppVtab = sqlite3_malloc( sizeof(*pNew) ); if( pNew==0 ) return SQLITE_NOMEM; sqlite3_declare_vtab(db, "CREATE TABLE x(value)"); sqlite3_vtab_config(db, SQLITE_VTAB_INNOCUOUS); memset(pNew, 0, sizeof(*pNew)); return SQLITE_OK; } /* Note that for this virtual table, the xCreate and xConnect ** methods are identical. */ static int wholenumberDisconnect(sqlite3_vtab *pVtab){ |
︙ | ︙ | |||
215 216 217 218 219 220 221 | } if( pIdxInfo->nOrderBy==1 && pIdxInfo->aOrderBy[0].desc==0 ){ pIdxInfo->orderByConsumed = 1; } if( (idxNum & 12)==0 ){ | | | 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 | } if( pIdxInfo->nOrderBy==1 && pIdxInfo->aOrderBy[0].desc==0 ){ pIdxInfo->orderByConsumed = 1; } if( (idxNum & 12)==0 ){ pIdxInfo->estimatedCost = 1e99; }else if( (idxNum & 3)==0 ){ pIdxInfo->estimatedCost = (double)5; }else{ pIdxInfo->estimatedCost = (double)1; } return SQLITE_OK; } |
︙ | ︙ |
Changes to ext/misc/zipfile.c.
︙ | ︙ | |||
32 33 34 35 36 37 38 39 40 | #include <zlib.h> #ifndef SQLITE_OMIT_VIRTUALTABLE #ifndef SQLITE_AMALGAMATION typedef sqlite3_int64 i64; typedef unsigned char u8; | > > > > > > > > > > > > > > | | > > > | 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 | #include <zlib.h> #ifndef SQLITE_OMIT_VIRTUALTABLE #ifndef SQLITE_AMALGAMATION #ifndef UINT32_TYPE # ifdef HAVE_UINT32_T # define UINT32_TYPE uint32_t # else # define UINT32_TYPE unsigned int # endif #endif #ifndef UINT16_TYPE # ifdef HAVE_UINT16_T # define UINT16_TYPE uint16_t # else # define UINT16_TYPE unsigned short int # endif #endif typedef sqlite3_int64 i64; typedef unsigned char u8; typedef UINT32_TYPE u32; /* 4-byte unsigned integer */ typedef UINT16_TYPE u16; /* 2-byte unsigned integer */ #define MIN(a,b) ((a)<(b) ? (a) : (b)) #if defined(SQLITE_COVERAGE_TEST) || defined(SQLITE_MUTATION_TEST) # define SQLITE_OMIT_AUXILIARY_SAFETY_CHECKS 1 #endif #if defined(SQLITE_OMIT_AUXILIARY_SAFETY_CHECKS) # define ALWAYS(X) (1) # define NEVER(X) (0) #elif !defined(NDEBUG) # define ALWAYS(X) ((X)?1:(assert(0),0)) # define NEVER(X) ((X)?(assert(0),1):0) #else # define ALWAYS(X) (X) |
︙ | ︙ | |||
331 332 333 334 335 336 337 338 339 340 341 342 343 344 | char **pzErr ){ int nByte = sizeof(ZipfileTab) + ZIPFILE_BUFFER_SIZE; int nFile = 0; const char *zFile = 0; ZipfileTab *pNew = 0; int rc; /* If the table name is not "zipfile", require that the argument be ** specified. This stops zipfile tables from being created as: ** ** CREATE VIRTUAL TABLE zzz USING zipfile(); ** ** It does not prevent: | > | 348 349 350 351 352 353 354 355 356 357 358 359 360 361 362 | char **pzErr ){ int nByte = sizeof(ZipfileTab) + ZIPFILE_BUFFER_SIZE; int nFile = 0; const char *zFile = 0; ZipfileTab *pNew = 0; int rc; (void)pAux; /* If the table name is not "zipfile", require that the argument be ** specified. This stops zipfile tables from being created as: ** ** CREATE VIRTUAL TABLE zzz USING zipfile(); ** ** It does not prevent: |
︙ | ︙ | |||
354 355 356 357 358 359 360 | if( argc>3 ){ zFile = argv[3]; nFile = (int)strlen(zFile)+1; } rc = sqlite3_declare_vtab(db, ZIPFILE_SCHEMA); if( rc==SQLITE_OK ){ | | > | 372 373 374 375 376 377 378 379 380 381 382 383 384 385 386 387 388 389 390 391 392 393 394 395 396 397 | if( argc>3 ){ zFile = argv[3]; nFile = (int)strlen(zFile)+1; } rc = sqlite3_declare_vtab(db, ZIPFILE_SCHEMA); if( rc==SQLITE_OK ){ pNew = (ZipfileTab*)sqlite3_malloc64((sqlite3_int64)nByte+nFile); if( pNew==0 ) return SQLITE_NOMEM; memset(pNew, 0, nByte+nFile); pNew->db = db; pNew->aBuffer = (u8*)&pNew[1]; if( zFile ){ pNew->zFile = (char*)&pNew->aBuffer[ZIPFILE_BUFFER_SIZE]; memcpy(pNew->zFile, zFile, nFile); zipfileDequote(pNew->zFile); } } sqlite3_vtab_config(db, SQLITE_VTAB_DIRECTONLY); *ppVtab = (sqlite3_vtab*)pNew; return rc; } /* ** Free the ZipfileEntry structure indicated by the only argument. */ |
︙ | ︙ | |||
518 519 520 521 522 523 524 | } static int zipfileAppendData( ZipfileTab *pTab, const u8 *aWrite, int nWrite ){ | > | | | | | | | | > > | 537 538 539 540 541 542 543 544 545 546 547 548 549 550 551 552 553 554 555 556 557 558 559 560 561 562 563 564 565 566 567 568 569 570 571 572 573 574 575 | } static int zipfileAppendData( ZipfileTab *pTab, const u8 *aWrite, int nWrite ){ if( nWrite>0 ){ size_t n = nWrite; fseek(pTab->pWriteFd, (long)pTab->szCurrent, SEEK_SET); n = fwrite(aWrite, 1, nWrite, pTab->pWriteFd); if( (int)n!=nWrite ){ pTab->base.zErrMsg = sqlite3_mprintf("error in fwrite()"); return SQLITE_ERROR; } pTab->szCurrent += nWrite; } return SQLITE_OK; } /* ** Read and return a 16-bit little-endian unsigned integer from buffer aBuf. */ static u16 zipfileGetU16(const u8 *aBuf){ return (aBuf[1] << 8) + aBuf[0]; } /* ** Read and return a 32-bit little-endian unsigned integer from buffer aBuf. */ static u32 zipfileGetU32(const u8 *aBuf){ if( aBuf==0 ) return 0; return ((u32)(aBuf[3]) << 24) + ((u32)(aBuf[2]) << 16) + ((u32)(aBuf[1]) << 8) + ((u32)(aBuf[0]) << 0); } /* |
︙ | ︙ | |||
699 700 701 702 703 704 705 | ** Bits 00-04: day ** Bits 05-08: month (1-12) ** Bits 09-15: years from 1980 ** ** https://msdn.microsoft.com/en-us/library/9kkf9tah.aspx */ static u32 zipfileMtime(ZipfileCDS *pCDS){ | > > | | | < < | | | < < < < < | | | < | | < < | < < | < > | | 721 722 723 724 725 726 727 728 729 730 731 732 733 734 735 736 737 738 739 740 741 742 743 744 745 746 747 748 749 750 751 752 | ** Bits 00-04: day ** Bits 05-08: month (1-12) ** Bits 09-15: years from 1980 ** ** https://msdn.microsoft.com/en-us/library/9kkf9tah.aspx */ static u32 zipfileMtime(ZipfileCDS *pCDS){ int Y,M,D,X1,X2,A,B,sec,min,hr; i64 JDsec; Y = (1980 + ((pCDS->mDate >> 9) & 0x7F)); M = ((pCDS->mDate >> 5) & 0x0F); D = (pCDS->mDate & 0x1F); sec = (pCDS->mTime & 0x1F)*2; min = (pCDS->mTime >> 5) & 0x3F; hr = (pCDS->mTime >> 11) & 0x1F; if( M<=2 ){ Y--; M += 12; } X1 = 36525*(Y+4716)/100; X2 = 306001*(M+1)/10000; A = Y/100; B = 2 - A + (A/4); JDsec = (i64)((X1 + X2 + D + B - 1524.5)*86400) + hr*3600 + min*60 + sec; return (u32)(JDsec - (i64)24405875*(i64)8640); } /* ** The opposite of zipfileMtime(). This function populates the mTime and ** mDate fields of the CDS structure passed as the first argument according ** to the UNIX timestamp value passed as the second. */ |
︙ | ︙ | |||
793 794 795 796 797 798 799 800 801 802 803 804 805 806 807 808 | FILE *pFile, /* If aBlob==0, read from this file */ i64 iOff, /* Offset of CDS record */ ZipfileEntry **ppEntry /* OUT: Pointer to new object */ ){ u8 *aRead; char **pzErr = &pTab->base.zErrMsg; int rc = SQLITE_OK; if( aBlob==0 ){ aRead = pTab->aBuffer; rc = zipfileReadData(pFile, aRead, ZIPFILE_CDS_FIXED_SZ, iOff, pzErr); }else{ aRead = (u8*)&aBlob[iOff]; } if( rc==SQLITE_OK ){ | > | | | 805 806 807 808 809 810 811 812 813 814 815 816 817 818 819 820 821 822 823 824 825 826 827 828 829 830 831 832 833 834 835 836 837 838 839 840 841 | FILE *pFile, /* If aBlob==0, read from this file */ i64 iOff, /* Offset of CDS record */ ZipfileEntry **ppEntry /* OUT: Pointer to new object */ ){ u8 *aRead; char **pzErr = &pTab->base.zErrMsg; int rc = SQLITE_OK; (void)nBlob; if( aBlob==0 ){ aRead = pTab->aBuffer; rc = zipfileReadData(pFile, aRead, ZIPFILE_CDS_FIXED_SZ, iOff, pzErr); }else{ aRead = (u8*)&aBlob[iOff]; } if( rc==SQLITE_OK ){ sqlite3_int64 nAlloc; ZipfileEntry *pNew; int nFile = zipfileGetU16(&aRead[ZIPFILE_CDS_NFILE_OFF]); int nExtra = zipfileGetU16(&aRead[ZIPFILE_CDS_NFILE_OFF+2]); nExtra += zipfileGetU16(&aRead[ZIPFILE_CDS_NFILE_OFF+4]); nAlloc = sizeof(ZipfileEntry) + nExtra; if( aBlob ){ nAlloc += zipfileGetU32(&aRead[ZIPFILE_CDS_SZCOMPRESSED_OFF]); } pNew = (ZipfileEntry*)sqlite3_malloc64(nAlloc); if( pNew==0 ){ rc = SQLITE_NOMEM; }else{ memset(pNew, 0, sizeof(ZipfileEntry)); rc = zipfileReadCDS(aRead, &pNew->cds); if( rc!=SQLITE_OK ){ *pzErr = sqlite3_mprintf("failed to read CDS at offset %lld", iOff); |
︙ | ︙ | |||
852 853 854 855 856 857 858 | ZipfileLFH lfh; if( pFile ){ rc = zipfileReadData(pFile, aRead, szFix, pNew->cds.iOffset, pzErr); }else{ aRead = (u8*)&aBlob[pNew->cds.iOffset]; } | | | 865 866 867 868 869 870 871 872 873 874 875 876 877 878 879 | ZipfileLFH lfh; if( pFile ){ rc = zipfileReadData(pFile, aRead, szFix, pNew->cds.iOffset, pzErr); }else{ aRead = (u8*)&aBlob[pNew->cds.iOffset]; } if( rc==SQLITE_OK ) rc = zipfileReadLFH(aRead, &lfh); if( rc==SQLITE_OK ){ pNew->iDataOff = pNew->cds.iOffset + ZIPFILE_LFH_FIXED_SZ; pNew->iDataOff += lfh.nFile + lfh.nExtra; if( aBlob && pNew->cds.szCompressed ){ pNew->aData = &pNew->aExtra[nExtra]; memcpy(pNew->aData, &aBlob[pNew->iDataOff], pNew->cds.szCompressed); } |
︙ | ︙ | |||
977 978 979 980 981 982 983 | ** case. */ static int zipfileDeflate( const u8 *aIn, int nIn, /* Input */ u8 **ppOut, int *pnOut, /* Output */ char **pzErr /* OUT: Error message */ ){ | > | > | > > > > > | < < < < < < < | 990 991 992 993 994 995 996 997 998 999 1000 1001 1002 1003 1004 1005 1006 1007 1008 1009 1010 1011 1012 1013 1014 1015 1016 1017 1018 1019 1020 1021 1022 | ** case. */ static int zipfileDeflate( const u8 *aIn, int nIn, /* Input */ u8 **ppOut, int *pnOut, /* Output */ char **pzErr /* OUT: Error message */ ){ int rc = SQLITE_OK; sqlite3_int64 nAlloc; z_stream str; u8 *aOut; memset(&str, 0, sizeof(str)); str.next_in = (Bytef*)aIn; str.avail_in = nIn; deflateInit2(&str, 9, Z_DEFLATED, -15, 8, Z_DEFAULT_STRATEGY); nAlloc = deflateBound(&str, nIn); aOut = (u8*)sqlite3_malloc64(nAlloc); if( aOut==0 ){ rc = SQLITE_NOMEM; }else{ int res; str.next_out = aOut; str.avail_out = nAlloc; res = deflate(&str, Z_FINISH); if( res==Z_STREAM_END ){ *ppOut = aOut; *pnOut = (int)str.total_out; }else{ sqlite3_free(aOut); *pzErr = sqlite3_mprintf("zipfile: deflate() error"); rc = SQLITE_ERROR; |
︙ | ︙ | |||
1054 1055 1056 1057 1058 1059 1060 | int szFinal = pCDS->szUncompressed; if( szFinal>0 ){ u8 *aBuf; u8 *aFree = 0; if( pCsr->pCurrent->aData ){ aBuf = pCsr->pCurrent->aData; }else{ | | | 1067 1068 1069 1070 1071 1072 1073 1074 1075 1076 1077 1078 1079 1080 1081 | int szFinal = pCDS->szUncompressed; if( szFinal>0 ){ u8 *aBuf; u8 *aFree = 0; if( pCsr->pCurrent->aData ){ aBuf = pCsr->pCurrent->aData; }else{ aBuf = aFree = sqlite3_malloc64(sz); if( aBuf==0 ){ rc = SQLITE_NOMEM; }else{ FILE *pFile = pCsr->pFile; if( pFile==0 ){ pFile = ((ZipfileTab*)(pCsr->base.pVtab))->pWriteFd; } |
︙ | ︙ | |||
1128 1129 1130 1131 1132 1133 1134 1135 1136 1137 1138 1139 1140 | FILE *pFile, /* Read from this file if aBlob==0 */ ZipfileEOCD *pEOCD /* Object to populate */ ){ u8 *aRead = pTab->aBuffer; /* Temporary buffer */ int nRead; /* Bytes to read from file */ int rc = SQLITE_OK; if( aBlob==0 ){ i64 iOff; /* Offset to read from */ i64 szFile; /* Total size of file in bytes */ fseek(pFile, 0, SEEK_END); szFile = (i64)ftell(pFile); if( szFile==0 ){ | > < | 1141 1142 1143 1144 1145 1146 1147 1148 1149 1150 1151 1152 1153 1154 1155 1156 1157 1158 1159 1160 1161 | FILE *pFile, /* Read from this file if aBlob==0 */ ZipfileEOCD *pEOCD /* Object to populate */ ){ u8 *aRead = pTab->aBuffer; /* Temporary buffer */ int nRead; /* Bytes to read from file */ int rc = SQLITE_OK; memset(pEOCD, 0, sizeof(ZipfileEOCD)); if( aBlob==0 ){ i64 iOff; /* Offset to read from */ i64 szFile; /* Total size of file in bytes */ fseek(pFile, 0, SEEK_END); szFile = (i64)ftell(pFile); if( szFile==0 ){ return SQLITE_OK; } nRead = (int)(MIN(szFile, ZIPFILE_BUFFER_SIZE)); iOff = szFile - nRead; rc = zipfileReadData(pFile, aRead, nRead, iOff, &pTab->base.zErrMsg); }else{ nRead = (int)(MIN(nBlob, ZIPFILE_BUFFER_SIZE)); |
︙ | ︙ | |||
1238 1239 1240 1241 1242 1243 1244 1245 1246 1247 1248 1249 1250 1251 1252 1253 1254 1255 1256 1257 1258 1259 1260 1261 1262 1263 1264 1265 1266 | int argc, sqlite3_value **argv ){ ZipfileTab *pTab = (ZipfileTab*)cur->pVtab; ZipfileCsr *pCsr = (ZipfileCsr*)cur; const char *zFile = 0; /* Zip file to scan */ int rc = SQLITE_OK; /* Return Code */ int bInMemory = 0; /* True for an in-memory zipfile */ zipfileResetCursor(pCsr); if( pTab->zFile ){ zFile = pTab->zFile; }else if( idxNum==0 ){ zipfileCursorErr(pCsr, "zipfile() function requires an argument"); return SQLITE_ERROR; }else if( sqlite3_value_type(argv[0])==SQLITE_BLOB ){ const u8 *aBlob = (const u8*)sqlite3_value_blob(argv[0]); int nBlob = sqlite3_value_bytes(argv[0]); assert( pTab->pFirstEntry==0 ); rc = zipfileLoadDirectory(pTab, aBlob, nBlob); pCsr->pFreeEntry = pTab->pFirstEntry; pTab->pFirstEntry = pTab->pLastEntry = 0; if( rc!=SQLITE_OK ) return rc; bInMemory = 1; }else{ zFile = (const char*)sqlite3_value_text(argv[0]); } if( 0==pTab->pWriteFd && 0==bInMemory ){ | > > > > > > > > | | 1251 1252 1253 1254 1255 1256 1257 1258 1259 1260 1261 1262 1263 1264 1265 1266 1267 1268 1269 1270 1271 1272 1273 1274 1275 1276 1277 1278 1279 1280 1281 1282 1283 1284 1285 1286 1287 1288 1289 1290 1291 1292 1293 1294 1295 | int argc, sqlite3_value **argv ){ ZipfileTab *pTab = (ZipfileTab*)cur->pVtab; ZipfileCsr *pCsr = (ZipfileCsr*)cur; const char *zFile = 0; /* Zip file to scan */ int rc = SQLITE_OK; /* Return Code */ int bInMemory = 0; /* True for an in-memory zipfile */ (void)idxStr; (void)argc; zipfileResetCursor(pCsr); if( pTab->zFile ){ zFile = pTab->zFile; }else if( idxNum==0 ){ zipfileCursorErr(pCsr, "zipfile() function requires an argument"); return SQLITE_ERROR; }else if( sqlite3_value_type(argv[0])==SQLITE_BLOB ){ static const u8 aEmptyBlob = 0; const u8 *aBlob = (const u8*)sqlite3_value_blob(argv[0]); int nBlob = sqlite3_value_bytes(argv[0]); assert( pTab->pFirstEntry==0 ); if( aBlob==0 ){ aBlob = &aEmptyBlob; nBlob = 0; } rc = zipfileLoadDirectory(pTab, aBlob, nBlob); pCsr->pFreeEntry = pTab->pFirstEntry; pTab->pFirstEntry = pTab->pLastEntry = 0; if( rc!=SQLITE_OK ) return rc; bInMemory = 1; }else{ zFile = (const char*)sqlite3_value_text(argv[0]); } if( 0==pTab->pWriteFd && 0==bInMemory ){ pCsr->pFile = zFile ? fopen(zFile, "rb") : 0; if( pCsr->pFile==0 ){ zipfileCursorErr(pCsr, "cannot open file: %s", zFile); rc = SQLITE_ERROR; }else{ rc = zipfileReadEOCD(pTab, 0, 0, pCsr->pFile, &pCsr->eocd); if( rc==SQLITE_OK ){ if( pCsr->eocd.nEntry==0 ){ |
︙ | ︙ | |||
1294 1295 1296 1297 1298 1299 1300 1301 1302 1303 1304 1305 1306 1307 1308 1309 1310 1311 1312 1313 | static int zipfileBestIndex( sqlite3_vtab *tab, sqlite3_index_info *pIdxInfo ){ int i; int idx = -1; int unusable = 0; for(i=0; i<pIdxInfo->nConstraint; i++){ const struct sqlite3_index_constraint *pCons = &pIdxInfo->aConstraint[i]; if( pCons->iColumn!=ZIPFILE_F_COLUMN_IDX ) continue; if( pCons->usable==0 ){ unusable = 1; }else if( pCons->op==SQLITE_INDEX_CONSTRAINT_EQ ){ idx = i; } } if( idx>=0 ){ pIdxInfo->aConstraintUsage[idx].argvIndex = 1; pIdxInfo->aConstraintUsage[idx].omit = 1; | > > < | 1315 1316 1317 1318 1319 1320 1321 1322 1323 1324 1325 1326 1327 1328 1329 1330 1331 1332 1333 1334 1335 1336 1337 1338 1339 1340 1341 1342 1343 | static int zipfileBestIndex( sqlite3_vtab *tab, sqlite3_index_info *pIdxInfo ){ int i; int idx = -1; int unusable = 0; (void)tab; for(i=0; i<pIdxInfo->nConstraint; i++){ const struct sqlite3_index_constraint *pCons = &pIdxInfo->aConstraint[i]; if( pCons->iColumn!=ZIPFILE_F_COLUMN_IDX ) continue; if( pCons->usable==0 ){ unusable = 1; }else if( pCons->op==SQLITE_INDEX_CONSTRAINT_EQ ){ idx = i; } } pIdxInfo->estimatedCost = 1000.0; if( idx>=0 ){ pIdxInfo->aConstraintUsage[idx].argvIndex = 1; pIdxInfo->aConstraintUsage[idx].omit = 1; pIdxInfo->idxNum = 1; }else if( unusable ){ return SQLITE_CONSTRAINT; } return SQLITE_OK; } |
︙ | ︙ | |||
1429 1430 1431 1432 1433 1434 1435 | /* ** Both (const char*) arguments point to nul-terminated strings. Argument ** nB is the value of strlen(zB). This function returns 0 if the strings are ** identical, ignoring any trailing '/' character in either path. */ static int zipfileComparePath(const char *zA, const char *zB, int nB){ int nA = (int)strlen(zA); | | | > > > > | 1451 1452 1453 1454 1455 1456 1457 1458 1459 1460 1461 1462 1463 1464 1465 1466 1467 1468 1469 1470 1471 1472 1473 1474 1475 1476 1477 1478 1479 | /* ** Both (const char*) arguments point to nul-terminated strings. Argument ** nB is the value of strlen(zB). This function returns 0 if the strings are ** identical, ignoring any trailing '/' character in either path. */ static int zipfileComparePath(const char *zA, const char *zB, int nB){ int nA = (int)strlen(zA); if( nA>0 && zA[nA-1]=='/' ) nA--; if( nB>0 && zB[nB-1]=='/' ) nB--; if( nA==nB && memcmp(zA, zB, nA)==0 ) return 0; return 1; } static int zipfileBegin(sqlite3_vtab *pVtab){ ZipfileTab *pTab = (ZipfileTab*)pVtab; int rc = SQLITE_OK; assert( pTab->pWriteFd==0 ); if( pTab->zFile==0 || pTab->zFile[0]==0 ){ pTab->base.zErrMsg = sqlite3_mprintf("zipfile: missing filename"); return SQLITE_ERROR; } /* Open a write fd on the file. Also load the entire central directory ** structure into memory. During the transaction any new file data is ** appended to the archive file, but the central directory is accumulated ** in main-memory until the transaction is committed. */ pTab->pWriteFd = fopen(pTab->zFile, "ab+"); if( pTab->pWriteFd==0 ){ |
︙ | ︙ | |||
1471 1472 1473 1474 1475 1476 1477 1478 1479 1480 1481 1482 1483 1484 | /* ** Return the current time as a 32-bit timestamp in UNIX epoch format (like ** time(2)). */ static u32 zipfileTime(void){ sqlite3_vfs *pVfs = sqlite3_vfs_find(0); u32 ret; if( pVfs->iVersion>=2 && pVfs->xCurrentTimeInt64 ){ i64 ms; pVfs->xCurrentTimeInt64(pVfs, &ms); ret = (u32)((ms/1000) - ((i64)24405875 * 8640)); }else{ double day; pVfs->xCurrentTime(pVfs, &day); | > | 1497 1498 1499 1500 1501 1502 1503 1504 1505 1506 1507 1508 1509 1510 1511 | /* ** Return the current time as a 32-bit timestamp in UNIX epoch format (like ** time(2)). */ static u32 zipfileTime(void){ sqlite3_vfs *pVfs = sqlite3_vfs_find(0); u32 ret; if( pVfs==0 ) return 0; if( pVfs->iVersion>=2 && pVfs->xCurrentTimeInt64 ){ i64 ms; pVfs->xCurrentTimeInt64(pVfs, &ms); ret = (u32)((ms/1000) - ((i64)24405875 * 8640)); }else{ double day; pVfs->xCurrentTime(pVfs, &day); |
︙ | ︙ | |||
1538 1539 1540 1541 1542 1543 1544 1545 1546 1547 1548 1549 1550 1551 | u8 *pFree = 0; /* Free this */ char *zFree = 0; /* Also free this */ ZipfileEntry *pOld = 0; ZipfileEntry *pOld2 = 0; int bUpdate = 0; /* True for an update that modifies "name" */ int bIsDir = 0; u32 iCrc32 = 0; if( pTab->pWriteFd==0 ){ rc = zipfileBegin(pVtab); if( rc!=SQLITE_OK ) return rc; } /* If this is a DELETE or UPDATE, find the archive entry to delete. */ | > > | 1565 1566 1567 1568 1569 1570 1571 1572 1573 1574 1575 1576 1577 1578 1579 1580 | u8 *pFree = 0; /* Free this */ char *zFree = 0; /* Also free this */ ZipfileEntry *pOld = 0; ZipfileEntry *pOld2 = 0; int bUpdate = 0; /* True for an update that modifies "name" */ int bIsDir = 0; u32 iCrc32 = 0; (void)pRowid; if( pTab->pWriteFd==0 ){ rc = zipfileBegin(pVtab); if( rc!=SQLITE_OK ) return rc; } /* If this is a DELETE or UPDATE, find the archive entry to delete. */ |
︙ | ︙ | |||
1614 1615 1616 1617 1618 1619 1620 1621 1622 1623 1624 1625 1626 1627 1628 1629 | if( rc==SQLITE_OK ){ rc = zipfileGetMode(apVal[3], bIsDir, &mode, &pTab->base.zErrMsg); } if( rc==SQLITE_OK ){ zPath = (const char*)sqlite3_value_text(apVal[2]); nPath = (int)strlen(zPath); mTime = zipfileGetTime(apVal[4]); } if( rc==SQLITE_OK && bIsDir ){ /* For a directory, check that the last character in the path is a ** '/'. This appears to be required for compatibility with info-zip ** (the unzip command on unix). It does not create directories ** otherwise. */ | > | < > > | > > > | 1643 1644 1645 1646 1647 1648 1649 1650 1651 1652 1653 1654 1655 1656 1657 1658 1659 1660 1661 1662 1663 1664 1665 1666 1667 1668 1669 1670 1671 1672 1673 1674 1675 | if( rc==SQLITE_OK ){ rc = zipfileGetMode(apVal[3], bIsDir, &mode, &pTab->base.zErrMsg); } if( rc==SQLITE_OK ){ zPath = (const char*)sqlite3_value_text(apVal[2]); if( zPath==0 ) zPath = ""; nPath = (int)strlen(zPath); mTime = zipfileGetTime(apVal[4]); } if( rc==SQLITE_OK && bIsDir ){ /* For a directory, check that the last character in the path is a ** '/'. This appears to be required for compatibility with info-zip ** (the unzip command on unix). It does not create directories ** otherwise. */ if( nPath<=0 || zPath[nPath-1]!='/' ){ zFree = sqlite3_mprintf("%s/", zPath); zPath = (const char*)zFree; if( zFree==0 ){ rc = SQLITE_NOMEM; nPath = 0; }else{ nPath = (int)strlen(zPath); } } } /* Check that we're not inserting a duplicate entry -OR- updating an ** entry with a path, thereby making it into a duplicate. */ if( (pOld==0 || bUpdate) && rc==SQLITE_OK ){ ZipfileEntry *p; |
︙ | ︙ | |||
1868 1869 1870 1871 1872 1873 1874 1875 1876 1877 1878 1879 1880 1881 | static int zipfileFindFunction( sqlite3_vtab *pVtab, /* Virtual table handle */ int nArg, /* Number of SQL function arguments */ const char *zName, /* Name of SQL function */ void (**pxFunc)(sqlite3_context*,int,sqlite3_value**), /* OUT: Result */ void **ppArg /* OUT: User data for *pxFunc */ ){ if( sqlite3_stricmp("zipfile_cds", zName)==0 ){ *pxFunc = zipfileFunctionCds; *ppArg = (void*)pVtab; return 1; } return 0; } | > | 1902 1903 1904 1905 1906 1907 1908 1909 1910 1911 1912 1913 1914 1915 1916 | static int zipfileFindFunction( sqlite3_vtab *pVtab, /* Virtual table handle */ int nArg, /* Number of SQL function arguments */ const char *zName, /* Name of SQL function */ void (**pxFunc)(sqlite3_context*,int,sqlite3_value**), /* OUT: Result */ void **ppArg /* OUT: User data for *pxFunc */ ){ (void)nArg; if( sqlite3_stricmp("zipfile_cds", zName)==0 ){ *pxFunc = zipfileFunctionCds; *ppArg = (void*)pVtab; return 1; } return 0; } |
︙ | ︙ | |||
1893 1894 1895 1896 1897 1898 1899 | ZipfileBuffer body; ZipfileBuffer cds; }; static int zipfileBufferGrow(ZipfileBuffer *pBuf, int nByte){ if( pBuf->n+nByte>pBuf->nAlloc ){ u8 *aNew; | | | | | | 1928 1929 1930 1931 1932 1933 1934 1935 1936 1937 1938 1939 1940 1941 1942 1943 1944 1945 1946 1947 1948 1949 1950 1951 1952 1953 1954 1955 1956 1957 1958 1959 1960 1961 1962 | ZipfileBuffer body; ZipfileBuffer cds; }; static int zipfileBufferGrow(ZipfileBuffer *pBuf, int nByte){ if( pBuf->n+nByte>pBuf->nAlloc ){ u8 *aNew; sqlite3_int64 nNew = pBuf->n ? pBuf->n*2 : 512; int nReq = pBuf->n + nByte; while( nNew<nReq ) nNew = nNew*2; aNew = sqlite3_realloc64(pBuf->a, nNew); if( aNew==0 ) return SQLITE_NOMEM; pBuf->a = aNew; pBuf->nAlloc = (int)nNew; } return SQLITE_OK; } /* ** xStep() callback for the zipfile() aggregate. This can be called in ** any of the following ways: ** ** SELECT zipfile(name,data) ... ** SELECT zipfile(name,mode,mtime,data) ... ** SELECT zipfile(name,mode,mtime,data,method) ... */ static void zipfileStep(sqlite3_context *pCtx, int nVal, sqlite3_value **apVal){ ZipfileCtx *p; /* Aggregate function context */ ZipfileEntry e; /* New entry to add to zip archive */ sqlite3_value *pName = 0; sqlite3_value *pMode = 0; sqlite3_value *pMtime = 0; sqlite3_value *pData = 0; |
︙ | ︙ | |||
2020 2021 2022 2023 2024 2025 2026 | /* Decode the "mtime" argument. */ e.mUnixTime = zipfileGetTime(pMtime); /* If this is a directory entry, ensure that there is exactly one '/' ** at the end of the path. Or, if this is not a directory and the path ** ends in '/' it is an error. */ if( bIsDir==0 ){ | | | < > | 2055 2056 2057 2058 2059 2060 2061 2062 2063 2064 2065 2066 2067 2068 2069 2070 2071 2072 2073 2074 2075 2076 2077 2078 2079 2080 2081 | /* Decode the "mtime" argument. */ e.mUnixTime = zipfileGetTime(pMtime); /* If this is a directory entry, ensure that there is exactly one '/' ** at the end of the path. Or, if this is not a directory and the path ** ends in '/' it is an error. */ if( bIsDir==0 ){ if( nName>0 && zName[nName-1]=='/' ){ zErr = sqlite3_mprintf("non-directory name must not end with /"); rc = SQLITE_ERROR; goto zipfile_step_out; } }else{ if( nName==0 || zName[nName-1]!='/' ){ zName = zFree = sqlite3_mprintf("%s/", zName); if( zName==0 ){ rc = SQLITE_NOMEM; goto zipfile_step_out; } nName = (int)strlen(zName); }else{ while( nName>1 && zName[nName-2]=='/' ) nName--; } } /* Assemble the ZipfileEntry object for the new zip archive entry */ e.cds.iVersionMadeBy = ZIPFILE_NEWENTRY_MADEBY; |
︙ | ︙ | |||
2088 2089 2090 2091 2092 2093 2094 | } sqlite3_free(zErr); } /* ** xFinalize() callback for zipfile aggregate function. */ | | | | | | 2123 2124 2125 2126 2127 2128 2129 2130 2131 2132 2133 2134 2135 2136 2137 2138 2139 2140 2141 2142 2143 2144 2145 2146 2147 2148 2149 2150 2151 2152 2153 2154 2155 2156 2157 2158 2159 2160 | } sqlite3_free(zErr); } /* ** xFinalize() callback for zipfile aggregate function. */ static void zipfileFinal(sqlite3_context *pCtx){ ZipfileCtx *p; ZipfileEOCD eocd; sqlite3_int64 nZip; u8 *aZip; p = (ZipfileCtx*)sqlite3_aggregate_context(pCtx, sizeof(ZipfileCtx)); if( p==0 ) return; if( p->nEntry>0 ){ memset(&eocd, 0, sizeof(eocd)); eocd.nEntry = (u16)p->nEntry; eocd.nEntryTotal = (u16)p->nEntry; eocd.nSize = p->cds.n; eocd.iOffset = p->body.n; nZip = p->body.n + p->cds.n + ZIPFILE_EOCD_FIXED_SZ; aZip = (u8*)sqlite3_malloc64(nZip); if( aZip==0 ){ sqlite3_result_error_nomem(pCtx); }else{ memcpy(aZip, p->body.a, p->body.n); memcpy(&aZip[p->body.n], p->cds.a, p->cds.n); zipfileSerializeEOCD(&eocd, &aZip[p->body.n + p->cds.n]); sqlite3_result_blob(pCtx, aZip, (int)nZip, zipfileFree); } } sqlite3_free(p->body.a); sqlite3_free(p->cds.a); } |
︙ | ︙ | |||
2145 2146 2147 2148 2149 2150 2151 2152 2153 2154 2155 2156 2157 2158 2159 2160 2161 2162 2163 2164 2165 2166 2167 | zipfileUpdate, /* xUpdate */ zipfileBegin, /* xBegin */ 0, /* xSync */ zipfileCommit, /* xCommit */ zipfileRollback, /* xRollback */ zipfileFindFunction, /* xFindMethod */ 0, /* xRename */ }; int rc = sqlite3_create_module(db, "zipfile" , &zipfileModule, 0); if( rc==SQLITE_OK ) rc = sqlite3_overload_function(db, "zipfile_cds", -1); if( rc==SQLITE_OK ){ rc = sqlite3_create_function(db, "zipfile", -1, SQLITE_UTF8, 0, 0, zipfileStep, zipfileFinal ); } return rc; } #else /* SQLITE_OMIT_VIRTUALTABLE */ # define zipfileRegister(x) SQLITE_OK #endif #ifdef _WIN32 | > > > > > > > > | 2180 2181 2182 2183 2184 2185 2186 2187 2188 2189 2190 2191 2192 2193 2194 2195 2196 2197 2198 2199 2200 2201 2202 2203 2204 2205 2206 2207 2208 2209 2210 | zipfileUpdate, /* xUpdate */ zipfileBegin, /* xBegin */ 0, /* xSync */ zipfileCommit, /* xCommit */ zipfileRollback, /* xRollback */ zipfileFindFunction, /* xFindMethod */ 0, /* xRename */ 0, /* xSavepoint */ 0, /* xRelease */ 0, /* xRollback */ 0 /* xShadowName */ }; int rc = sqlite3_create_module(db, "zipfile" , &zipfileModule, 0); if( rc==SQLITE_OK ) rc = sqlite3_overload_function(db, "zipfile_cds", -1); if( rc==SQLITE_OK ){ rc = sqlite3_create_function(db, "zipfile", -1, SQLITE_UTF8, 0, 0, zipfileStep, zipfileFinal ); } assert( sizeof(i64)==8 ); assert( sizeof(u32)==4 ); assert( sizeof(u16)==2 ); assert( sizeof(u8)==1 ); return rc; } #else /* SQLITE_OMIT_VIRTUALTABLE */ # define zipfileRegister(x) SQLITE_OK #endif #ifdef _WIN32 |
︙ | ︙ |
Changes to ext/rbu/rbu.c.
︙ | ︙ | |||
52 53 54 55 56 57 58 | "\n" , zArgv0); exit(1); } void report_default_vfs(){ sqlite3_vfs *pVfs = sqlite3_vfs_find(0); | | | 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 | "\n" , zArgv0); exit(1); } void report_default_vfs(){ sqlite3_vfs *pVfs = sqlite3_vfs_find(0); fprintf(stdout, "default vfs is \"%s\"\n", pVfs ? pVfs->zName : "NULL"); } void report_rbu_vfs(sqlite3rbu *pRbu){ sqlite3 *db = sqlite3rbu_db(pRbu, 0); if( db ){ char *zName = 0; sqlite3_file_control(db, "main", SQLITE_FCNTL_VFSNAME, &zName); |
︙ | ︙ | |||
178 179 180 181 182 183 184 185 186 187 188 | fprintf(stdout, "%s", zBuf); break; default: fprintf(stderr, "error=%d: %s\n", rc, zErrmsg); break; } sqlite3_free(zErrmsg); return (rc==SQLITE_OK || rc==SQLITE_DONE) ? 0 : 1; } | > > > > > > > | 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 | fprintf(stdout, "%s", zBuf); break; default: fprintf(stderr, "error=%d: %s\n", rc, zErrmsg); break; } if( nStatStep>0 ){ sqlite3_int64 nUsed; sqlite3_int64 nHighwater; sqlite3_status64(SQLITE_STATUS_MEMORY_USED, &nUsed, &nHighwater, 0); fprintf(stdout, "memory used=%lld highwater=%lld\n", nUsed, nHighwater); } sqlite3_free(zErrmsg); return (rc==SQLITE_OK || rc==SQLITE_DONE) ? 0 : 1; } |
Changes to ext/rbu/rbu1.test.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 | # 2014 August 30 # # The author disclaims copyright to this source code. In place of # a legal notice, here is a blessing: # # May you do good and not evil. # May you find forgiveness for yourself and forgive others. # May you share freely, never taking more than you give. # #*********************************************************************** # source [file join [file dirname [info script]] rbu_common.tcl] set ::testprefix rbu1 db close sqlite3_shutdown sqlite3_config_uri 1 # Create a simple RBU database. That expects to write to a table: | > | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 | # 2014 August 30 # # The author disclaims copyright to this source code. In place of # a legal notice, here is a blessing: # # May you do good and not evil. # May you find forgiveness for yourself and forgive others. # May you share freely, never taking more than you give. # #*********************************************************************** # source [file join [file dirname [info script]] rbu_common.tcl] ifcapable !rbu { finish_test ; return } set ::testprefix rbu1 db close sqlite3_shutdown sqlite3_config_uri 1 # Create a simple RBU database. That expects to write to a table: |
︙ | ︙ | |||
128 129 130 131 132 133 134 135 136 137 138 139 140 141 | foreach {tn3 create_vfs destroy_vfs} { 1 {} {} 2 { sqlite3rbu_create_vfs -default myrbu "" } { sqlite3rbu_destroy_vfs myrbu } } { eval $create_vfs foreach {tn2 cmd} { 1 run_rbu 2 step_rbu 3 step_rbu_uri 4 step_rbu_state | > > > > > | 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 | foreach {tn3 create_vfs destroy_vfs} { 1 {} {} 2 { sqlite3rbu_create_vfs -default myrbu "" } { sqlite3rbu_destroy_vfs myrbu } 3 { sqlite3_register_cksumvfs } { sqlite3_unregister_cksumvfs } } { eval $create_vfs foreach {tn2 cmd} { 1 run_rbu 2 step_rbu 3 step_rbu_uri 4 step_rbu_state |
︙ | ︙ |
Changes to ext/rbu/rbu10.test.
1 2 3 4 5 6 7 8 9 10 11 12 | # 2014 August 30 # # The author disclaims copyright to this source code. In place of # a legal notice, here is a blessing: # # May you do good and not evil. # May you find forgiveness for yourself and forgive others. # May you share freely, never taking more than you give. # #*********************************************************************** # | < | < | < | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 | # 2014 August 30 # # The author disclaims copyright to this source code. In place of # a legal notice, here is a blessing: # # May you do good and not evil. # May you find forgiveness for yourself and forgive others. # May you share freely, never taking more than you give. # #*********************************************************************** # source [file join [file dirname [info script]] rbu_common.tcl] ifcapable !rbu { finish_test ; return } set ::testprefix rbu10 #-------------------------------------------------------------------- # Test that UPDATE commands work even if the input columns are in a # different order to the output columns. # do_execsql_test 1.0 { CREATE TABLE t1(a INTEGER PRIMARY KEY, b, c); |
︙ | ︙ | |||
46 47 48 49 50 51 52 | #-------------------------------------------------------------------- # Test that the hidden languageid column of an fts4 table can be # written. # ifcapable fts3 { do_execsql_test 2.0 { | | | 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 | #-------------------------------------------------------------------- # Test that the hidden languageid column of an fts4 table can be # written. # ifcapable fts3 { do_execsql_test 2.0 { create virtual TABLE ft USING fts4(a, b, languageid='langid'); } do_test 2.1 { apply_rbu { CREATE TABLE data_ft(a, b, rbu_rowid, langid, rbu_control); INSERT INTO data_ft VALUES('a', 'b', 22, 1, 0); -- insert INSERT INTO data_ft VALUES('a', 'b', 23, 10, 0); -- insert INSERT INTO data_ft VALUES('a', 'b', 24, 100, 0); -- insert |
︙ | ︙ |
Changes to ext/rbu/rbu11.test.
1 2 3 4 5 6 7 8 9 10 11 12 | # 2015 February 16 # # The author disclaims copyright to this source code. In place of # a legal notice, here is a blessing: # # May you do good and not evil. # May you find forgiveness for yourself and forgive others. # May you share freely, never taking more than you give. # #*********************************************************************** # | < | < | | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 | # 2015 February 16 # # The author disclaims copyright to this source code. In place of # a legal notice, here is a blessing: # # May you do good and not evil. # May you find forgiveness for yourself and forgive others. # May you share freely, never taking more than you give. # #*********************************************************************** # source [file join [file dirname [info script]] rbu_common.tcl] ifcapable !rbu { finish_test ; return } set ::testprefix rbu11 #-------------------------------------------------------------------- # Test that the xAccess() method of an rbu vfs handles queries other # than SQLITE_ACCESS_EXISTS correctly. The test code below causes # SQLite to call xAccess(SQLITE_ACCESS_READWRITE) on the directory |
︙ | ︙ |
Changes to ext/rbu/rbu12.test.
1 2 3 4 5 6 7 8 9 10 11 12 | # 2015 February 16 # # The author disclaims copyright to this source code. In place of # a legal notice, here is a blessing: # # May you do good and not evil. # May you find forgiveness for yourself and forgive others. # May you share freely, never taking more than you give. # #*********************************************************************** # | < | < | | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 | # 2015 February 16 # # The author disclaims copyright to this source code. In place of # a legal notice, here is a blessing: # # May you do good and not evil. # May you find forgiveness for yourself and forgive others. # May you share freely, never taking more than you give. # #*********************************************************************** # source [file join [file dirname [info script]] rbu_common.tcl] ifcapable !rbu { finish_test ; return } source $testdir/lock_common.tcl set ::testprefix rbu12 set setup_sql { DROP TABLE IF EXISTS xx; DROP TABLE IF EXISTS xy; CREATE TABLE xx(a, b, c PRIMARY KEY); |
︙ | ︙ |
Changes to ext/rbu/rbu13.test.
︙ | ︙ | |||
9 10 11 12 13 14 15 | # #*********************************************************************** # # Test an RBU update that features lots of different rbu_control strings # for UPDATE statements. This tests RBU's internal UPDATE statement cache. # | < | < | | 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 | # #*********************************************************************** # # Test an RBU update that features lots of different rbu_control strings # for UPDATE statements. This tests RBU's internal UPDATE statement cache. # source [file join [file dirname [info script]] rbu_common.tcl] ifcapable !rbu { finish_test ; return } source $testdir/lock_common.tcl set ::testprefix rbu13 do_execsql_test 1.0 { CREATE TABLE t1(a PRIMARY KEY, b, c, d, e, f, g, h); WITH ii(i) AS (SELECT 0 UNION ALL SELECT i+1 FROM ii WHERE i<127) INSERT INTO t1 SELECT i, 0, 0, 0, 0, 0, 0, 0 FROM ii; |
︙ | ︙ |
Changes to ext/rbu/rbu14.test.
︙ | ︙ | |||
9 10 11 12 13 14 15 | # #*********************************************************************** # # Test that an RBU data_xxx table may be a view instead of a regular # table. # | < | < | | 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 | # #*********************************************************************** # # Test that an RBU data_xxx table may be a view instead of a regular # table. # source [file join [file dirname [info script]] rbu_common.tcl] ifcapable !rbu { finish_test ; return } source $testdir/lock_common.tcl set ::testprefix rbu14 foreach {tn schema} { 1 { CREATE TABLE t1(a PRIMARY KEY, b, c); |
︙ | ︙ |
Changes to ext/rbu/rbu3.test.
1 2 3 4 5 6 7 8 9 10 11 12 | # 2014 August 30 # # The author disclaims copyright to this source code. In place of # a legal notice, here is a blessing: # # May you do good and not evil. # May you find forgiveness for yourself and forgive others. # May you share freely, never taking more than you give. # #*********************************************************************** # | < | < | | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 | # 2014 August 30 # # The author disclaims copyright to this source code. In place of # a legal notice, here is a blessing: # # May you do good and not evil. # May you find forgiveness for yourself and forgive others. # May you share freely, never taking more than you give. # #*********************************************************************** # source [file join [file dirname [info script]] rbu_common.tcl] ifcapable !rbu { finish_test ; return } set ::testprefix rbu3 # Run the RBU in file $rbu on target database $target until completion. # proc run_rbu {target rbu} { sqlite3rbu rbu $target $rbu |
︙ | ︙ |
Changes to ext/rbu/rbu5.test.
︙ | ︙ | |||
9 10 11 12 13 14 15 16 17 18 19 20 21 22 | # #*********************************************************************** # # Test some properties of the pager_rbu_mode and rbu_mode pragmas. # source [file join [file dirname [info script]] rbu_common.tcl] set ::testprefix rbu5 # Return a list of the primary key columns for table $tbl in the database # opened by database handle $db. # proc pkcols {db tbl} { | > | 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 | # #*********************************************************************** # # Test some properties of the pager_rbu_mode and rbu_mode pragmas. # source [file join [file dirname [info script]] rbu_common.tcl] ifcapable !rbu { finish_test ; return } set ::testprefix rbu5 # Return a list of the primary key columns for table $tbl in the database # opened by database handle $db. # proc pkcols {db tbl} { |
︙ | ︙ |
Changes to ext/rbu/rbu6.test.
︙ | ︙ | |||
9 10 11 12 13 14 15 | # #*********************************************************************** # # This file contains tests for the RBU module. Specifically, it tests the # outcome of some other client writing to the database while an RBU update # is being applied. | < | < | | 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 | # #*********************************************************************** # # This file contains tests for the RBU module. Specifically, it tests the # outcome of some other client writing to the database while an RBU update # is being applied. source [file join [file dirname [info script]] rbu_common.tcl] ifcapable !rbu { finish_test ; return } set ::testprefix rbu6 proc setup_test {} { reset_db execsql { CREATE TABLE t1(a INTEGER PRIMARY KEY, b UNIQUE); CREATE TABLE t2(a INTEGER PRIMARY KEY, b UNIQUE); |
︙ | ︙ |
Changes to ext/rbu/rbu7.test.
︙ | ︙ | |||
9 10 11 12 13 14 15 | # #*********************************************************************** # # This file contains tests for the RBU module. # | < | < | | 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 | # #*********************************************************************** # # This file contains tests for the RBU module. # source [file join [file dirname [info script]] rbu_common.tcl] ifcapable !rbu { finish_test ; return } set ::testprefix rbu7 # Test index: # # 1.*: That affinities are correctly applied to values within the # RBU database. # |
︙ | ︙ |
Changes to ext/rbu/rbu8.test.
︙ | ︙ | |||
8 9 10 11 12 13 14 | # May you share freely, never taking more than you give. # #*********************************************************************** # # Test the rbu_delta() feature. # | < | < | | 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 | # May you share freely, never taking more than you give. # #*********************************************************************** # # Test the rbu_delta() feature. # source [file join [file dirname [info script]] rbu_common.tcl] ifcapable !rbu { finish_test ; return } set ::testprefix rbu8 do_execsql_test 1.0 { CREATE TABLE t1(x, y PRIMARY KEY, z); INSERT INTO t1 VALUES(NULL, 1, 'one'); INSERT INTO t1 VALUES(NULL, 2, 'two'); INSERT INTO t1 VALUES(NULL, 3, 'three'); |
︙ | ︙ |
Changes to ext/rbu/rbu9.test.
︙ | ︙ | |||
8 9 10 11 12 13 14 | # May you share freely, never taking more than you give. # #*********************************************************************** # # Test RBU with virtual tables. And tables with no PRIMARY KEY declarations. # | < | < | | 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 | # May you share freely, never taking more than you give. # #*********************************************************************** # # Test RBU with virtual tables. And tables with no PRIMARY KEY declarations. # source [file join [file dirname [info script]] rbu_common.tcl] ifcapable !rbu { finish_test ; return } set ::testprefix rbu9 ifcapable !fts3 { finish_test return } |
︙ | ︙ |
Changes to ext/rbu/rbuA.test.
︙ | ︙ | |||
10 11 12 13 14 15 16 | #*********************************************************************** # # This file contains tests for the RBU module. More specifically, it # contains tests to ensure that it is an error to attempt to update # a wal mode database via RBU. # | < | < | | 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 | #*********************************************************************** # # This file contains tests for the RBU module. More specifically, it # contains tests to ensure that it is an error to attempt to update # a wal mode database via RBU. # source [file join [file dirname [info script]] rbu_common.tcl] ifcapable !rbu { finish_test ; return } set ::testprefix rbuA set db_sql { CREATE TABLE t1(a PRIMARY KEY, b, c); } set rbu_sql { CREATE TABLE data_t1(a, b, c, rbu_control); |
︙ | ︙ |
Changes to ext/rbu/rbuB.test.
︙ | ︙ | |||
8 9 10 11 12 13 14 15 16 17 18 19 20 21 | # May you share freely, never taking more than you give. # #*********************************************************************** # # source [file join [file dirname [info script]] rbu_common.tcl] set ::testprefix rbuB db close sqlite3_shutdown test_sqlite3_log xLog reset_db | > | 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 | # May you share freely, never taking more than you give. # #*********************************************************************** # # source [file join [file dirname [info script]] rbu_common.tcl] ifcapable !rbu { finish_test ; return } set ::testprefix rbuB db close sqlite3_shutdown test_sqlite3_log xLog reset_db |
︙ | ︙ | |||
44 45 46 47 48 49 50 | proc xLog {err msg} { lappend ::errlog $err } do_test 1.2 { run_rbu test.db rbu.db } {SQLITE_DONE} do_test 1.3 { set ::errlog | | | 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 | proc xLog {err msg} { lappend ::errlog $err } do_test 1.2 { run_rbu test.db rbu.db } {SQLITE_DONE} do_test 1.3 { set ::errlog } {SQLITE_NOTICE_RECOVER_WAL SQLITE_NOTICE_RBU} do_execsql_test 1.4 { SELECT * FROM t1 } {1 2 3 4 5 6 7 8 9} db close sqlite3_shutdown test_sqlite3_log sqlite3_initialize finish_test |
Changes to ext/rbu/rbuC.test.
︙ | ︙ | |||
9 10 11 12 13 14 15 16 17 18 19 20 21 22 | # #*********************************************************************** # Tests for RBU focused on the REPLACE operation (rbu_control column # contains integer value 2). # source [file join [file dirname [info script]] rbu_common.tcl] set ::testprefix rbuC #------------------------------------------------------------------------- # This test is actually of an UPDATE directive. Just to establish that # these work with UNIQUE indexes before preceding to REPLACE. # do_execsql_test 1.0 { | > | 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 | # #*********************************************************************** # Tests for RBU focused on the REPLACE operation (rbu_control column # contains integer value 2). # source [file join [file dirname [info script]] rbu_common.tcl] ifcapable !rbu { finish_test ; return } set ::testprefix rbuC #------------------------------------------------------------------------- # This test is actually of an UPDATE directive. Just to establish that # these work with UNIQUE indexes before preceding to REPLACE. # do_execsql_test 1.0 { |
︙ | ︙ |
Changes to ext/rbu/rbu_common.tcl.
︙ | ︙ | |||
85 86 87 88 89 90 91 | } set rc } proc do_rbu_vacuum_test {tn step {statedb state.db}} { forcedelete $statedb if {$statedb=="" && $step==1} breakpoint | | | | | | 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 | } set rc } proc do_rbu_vacuum_test {tn step {statedb state.db}} { forcedelete $statedb if {$statedb=="" && $step==1} breakpoint uplevel [list do_test $tn.1 [string map [list %state% $statedb %step% $step] { if {%step%==0} { sqlite3rbu_vacuum rbu test.db {%state%}} while 1 { if {%step%==1} { sqlite3rbu_vacuum rbu test.db {%state%}} set state [rbu state] check_prestep_state test.db $state set rc [rbu step] check_poststep_state $rc test.db $state if {$rc!="SQLITE_OK"} break if {%step%==1} { rbu close } } rbu close }] {SQLITE_DONE}] uplevel [list do_execsql_test $tn.2 { PRAGMA integrity_check } ok] |
︙ | ︙ |
Added ext/rbu/rbubusy.test.
> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 | # 2014 August 30 # # The author disclaims copyright to this source code. In place of # a legal notice, here is a blessing: # # May you do good and not evil. # May you find forgiveness for yourself and forgive others. # May you share freely, never taking more than you give. # #*********************************************************************** # # source [file join [file dirname [info script]] rbu_common.tcl] ifcapable !rbu { finish_test ; return } set ::testprefix rbubusy db close sqlite3_shutdown test_sqlite3_log xLog reset_db set db_sql { CREATE TABLE t1(a PRIMARY KEY, b, c); INSERT INTO t1 VALUES(1000, 2000, 3000); } set rbu_sql { CREATE TABLE data_t1(a, b, c, rbu_control); INSERT INTO data_t1 VALUES(1, 2, 3, 0); INSERT INTO data_t1 VALUES(4, 5, 6, 0); INSERT INTO data_t1 VALUES(7, 8, 9, 0); } do_test 1.1 { forcedelete rbu.db sqlite3 rbu rbu.db rbu eval $rbu_sql rbu close db eval $db_sql } {} do_execsql_test 1.2 { BEGIN; SELECT * FROM t1 } {1000 2000 3000} do_test 1.3 { sqlite3rbu rbu test.db rbu.db rbu step } {SQLITE_OK} do_test 1.4 { while 1 { set rc [rbu step] if {$rc!="SQLITE_OK"} break } set rc } {SQLITE_BUSY} do_test 1.5 { rbu step } {SQLITE_BUSY} do_test 1.6 { db eval COMMIT rbu step } {SQLITE_BUSY} catch { rbu close } do_test 1.7 { sqlite3rbu rbu test.db rbu.db while 1 { set rc [rbu step] if {$rc!="SQLITE_OK"} break } set rc } {SQLITE_DONE} rbu close db close sqlite3_shutdown test_sqlite3_log sqlite3_initialize finish_test |
Changes to ext/rbu/rbucollate.test.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 | # 2018 March 22 # # The author disclaims copyright to this source code. In place of # a legal notice, here is a blessing: # # May you do good and not evil. # May you find forgiveness for yourself and forgive others. # May you share freely, never taking more than you give. # #*********************************************************************** # source [file join [file dirname [info script]] rbu_common.tcl] set ::testprefix rbucollate ifcapable !icu_collations { finish_test return } | > | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 | # 2018 March 22 # # The author disclaims copyright to this source code. In place of # a legal notice, here is a blessing: # # May you do good and not evil. # May you find forgiveness for yourself and forgive others. # May you share freely, never taking more than you give. # #*********************************************************************** # source [file join [file dirname [info script]] rbu_common.tcl] ifcapable !rbu { finish_test ; return } set ::testprefix rbucollate ifcapable !icu_collations { finish_test return } |
︙ | ︙ |
Changes to ext/rbu/rbucrash.test.
1 2 3 4 5 6 7 8 9 10 11 12 | # 2014 October 22 # # The author disclaims copyright to this source code. In place of # a legal notice, here is a blessing: # # May you do good and not evil. # May you find forgiveness for yourself and forgive others. # May you share freely, never taking more than you give. # #*********************************************************************** # | < | < | | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 | # 2014 October 22 # # The author disclaims copyright to this source code. In place of # a legal notice, here is a blessing: # # May you do good and not evil. # May you find forgiveness for yourself and forgive others. # May you share freely, never taking more than you give. # #*********************************************************************** # source [file join [file dirname [info script]] rbu_common.tcl] ifcapable !rbu { finish_test ; return } set ::testprefix rbucrash db close forcedelete test.db-oal rbu.db sqlite3_shutdown sqlite3_config_uri 1 reset_db |
︙ | ︙ |
Changes to ext/rbu/rbucrash2.test.
1 2 3 4 5 6 7 8 9 10 11 12 | # 2017 March 02 # # The author disclaims copyright to this source code. In place of # a legal notice, here is a blessing: # # May you do good and not evil. # May you find forgiveness for yourself and forgive others. # May you share freely, never taking more than you give. # #*********************************************************************** # | < | < | | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 | # 2017 March 02 # # The author disclaims copyright to this source code. In place of # a legal notice, here is a blessing: # # May you do good and not evil. # May you find forgiveness for yourself and forgive others. # May you share freely, never taking more than you give. # #*********************************************************************** # source [file join [file dirname [info script]] rbu_common.tcl] ifcapable !rbu { finish_test ; return } set ::testprefix rbucrash2 db close forcedelete test.db-oal rbu.db sqlite3_shutdown sqlite3_config_uri 1 reset_db |
︙ | ︙ |
Changes to ext/rbu/rbudiff.test.
︙ | ︙ | |||
8 9 10 11 12 13 14 | # May you share freely, never taking more than you give. # #*********************************************************************** # # Tests for the [sqldiff --rbu] command. # # | < < | | > | 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 | # May you share freely, never taking more than you give. # #*********************************************************************** # # Tests for the [sqldiff --rbu] command. # # source [file join [file dirname [info script]] rbu_common.tcl] ifcapable !rbu { finish_test ; return } set testprefix rbudiff set PROG [test_find_sqldiff] db close proc get_rbudiff_sql {db1 db2} { exec $::PROG --rbu $db1 $db2 |
︙ | ︙ | |||
260 261 262 263 264 265 266 267 268 269 270 271 272 273 | } 3 { creAte virTUal tablE t1 USING FTs5(c); INSERT INTO t1 VALUES('a b c'); INSERT INTO t1 VALUES('a b c'); } { DELETE FROM t1 WHERE rowid = 1; INSERT INTO t1 VALUES('a b c'); } } { forcedelete test.db test.db2 | > > > > > > > > | 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 | } 3 { creAte virTUal tablE t1 USING FTs5(c); INSERT INTO t1 VALUES('a b c'); INSERT INTO t1 VALUES('a b c'); } { DELETE FROM t1 WHERE rowid = 1; INSERT INTO t1 VALUES('a b c'); } 4 { creAte virTUal tablE t1 USING FTs5(c); INSERT INTO t1 VALUES('a b c'); INSERT INTO t1 VALUES('a b c'); } { DELETE FROM t1 WHERE rowid = 1; INSERT INTO t1 VALUES('a b c'); } } { forcedelete test.db test.db2 |
︙ | ︙ |
Changes to ext/rbu/rbudor.test.
︙ | ︙ | |||
9 10 11 12 13 14 15 | # #*********************************************************************** # # This test file focuses on interactions between RBU and the feature # enabled by SQLITE_DIRECT_OVERFLOW_READ - Direct Overflow Read. # | < | < | | 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 | # #*********************************************************************** # # This test file focuses on interactions between RBU and the feature # enabled by SQLITE_DIRECT_OVERFLOW_READ - Direct Overflow Read. # source [file join [file dirname [info script]] rbu_common.tcl] ifcapable !rbu { finish_test ; return } set ::testprefix rbudor set bigA [string repeat a 5000] set bigB [string repeat b 5000] do_execsql_test 1.0 { PRAGMA page_size = 1024; CREATE TABLE t1(a INTEGER PRIMARY KEY, b BLOB); |
︙ | ︙ |
Added ext/rbu/rbuexlock.test.
> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 | # 2021 November 06 # # The author disclaims copyright to this source code. In place of # a legal notice, here is a blessing: # # May you do good and not evil. # May you find forgiveness for yourself and forgive others. # May you share freely, never taking more than you give. # #*********************************************************************** # source [file join [file dirname [info script]] rbu_common.tcl] ifcapable !rbu { finish_test ; return } set ::testprefix rbuexlock db close sqlite3_shutdown sqlite3_config_uri 1 # Create a simple RBU database. That expects to write to a table: # # CREATE TABLE t1(a INTEGER PRIMARY KEY, b, c); # proc create_rbu {filename} { forcedelete $filename sqlite3 rbu1 $filename rbu1 eval { CREATE TABLE data_t1(a, b, c, rbu_control); INSERT INTO data_t1 VALUES(10, random(), random(), 0); INSERT INTO data_t1 VALUES(20, random(), random(), 0); INSERT INTO data_t1 VALUES(30, random(), random(), 0); INSERT INTO data_t1 VALUES(40, random(), random(), 0); INSERT INTO data_t1 VALUES(50, random(), random(), 0); INSERT INTO data_t1 VALUES(60, random(), random(), 0); INSERT INTO data_t1 VALUES(70, random(), random(), 0); INSERT INTO data_t1 VALUES(80, random(), random(), 0); } rbu1 close return $filename } reset_db do_execsql_test 1.0 { CREATE TABLE t1(a PRIMARY KEY, b INT, c INT); CREATE INDEX t1b ON t1(b); CREATE INDEX t1c ON t1(c); INSERT INTO t1 VALUES(1, 2, 3); } create_rbu rbu1.db do_test 1.1.0 { sqlite3rbu rbu file:test.db?rbu_exclusive_checkpoint=1 rbu1.db rbu step } SQLITE_OK do_catchsql_test 1.1.1 { SELECT * FROM t1 } {0 {1 2 3}} do_test 1.2.0 { for {set ii 0} {$ii < 10} {incr ii} { rbu step } rbu step } SQLITE_OK do_catchsql_test 1.2.1 { SELECT * FROM t1 } {0 {1 2 3}} do_test 1.2.2 { db eval {PRAGMA journal_mode} } {delete} do_test 1.3.0 { while {[file exists test.db-wal]==0} { rbu step } } {} do_catchsql_test 1.3.1 { SELECT * FROM t1 } {1 {database is locked}} do_test 1.3.2 { db eval {PRAGMA journal_mode} } {delete} do_test 1.4.0 { rbu step } SQLITE_OK do_catchsql_test 1.4.1 { SELECT * FROM t1 } {1 {database is locked}} do_test 1.4.2 { db eval {PRAGMA journal_mode} } {delete} rbu close do_test 1.5.0 { file exists test.db-wal } {1} do_test 1.5.1 { sqlite3rbu rbu file:test.db?rbu_exclusive_checkpoint=1 rbu1.db file exists test.db-wal } 1 do_catchsql_test 1.5.2 { SELECT * FROM t1 } {1 {database is locked}} do_test 1.5.2 { db eval {PRAGMA journal_mode} } {delete} do_test 1.6.0 { rbu step } SQLITE_OK do_catchsql_test 1.6.1 { SELECT * FROM t1 } {1 {database is locked}} do_test 1.6.2 { db eval {PRAGMA journal_mode} } {delete} do_test 1.7.0 { while {[rbu step]=="SQLITE_OK"} {} rbu close } SQLITE_DONE do_catchsql_test 1.7.2 { SELECT count(*) FROM t1 } {0 9} do_test 1.7.2 { db eval {PRAGMA journal_mode} } {delete} reset_db do_execsql_test 2.0 { CREATE TABLE t1(a PRIMARY KEY, b INT, c INT); CREATE INDEX t1b ON t1(b); CREATE INDEX t1c ON t1(c); INSERT INTO t1 VALUES(1, 2, 3); } create_rbu rbu1.db do_test 2.1.0 { sqlite3rbu rbu file:test.db?rbu_exclusive_checkpoint=0 rbu1.db rbu step } SQLITE_OK do_catchsql_test 2.1.1 { SELECT * FROM t1 } {0 {1 2 3}} do_test 2.2.0 { for {set ii 0} {$ii < 10} {incr ii} { rbu step } rbu step } SQLITE_OK do_catchsql_test 2.2.1 { SELECT * FROM t1 } {0 {1 2 3}} do_test 2.3.0 { while {[file exists test.db-wal]==0} { rbu step } } {} do_test 2.3.1 { llength [db eval {SELECT * FROM t1}] } {27} do_test 2.3.2 { db eval {PRAGMA journal_mode} } {wal} do_test 2.4.0 { rbu step } SQLITE_OK do_test 2.4.1 { llength [db eval {SELECT * FROM t1}] } {27} do_test 2.4.2 { db eval {PRAGMA journal_mode} } {wal} rbu close do_test 2.5.0 { db eval {PRAGMA journal_mode} } {wal} do_execsql_test 2.5.1 { DELETE FROM t1; } {} create_rbu rbu1.db do_test 3.1.0 { sqlite3rbu rbu file:test.db?rbu_exclusive_checkpoint=0 rbu1.db rbu step } SQLITE_ERROR do_test 3.1.1 { set rc [catch {rbu close} msg] lappend rc $msg } {1 {SQLITE_ERROR - cannot update wal mode database}} db eval {PRAGMA journal_mode=DELETE} create_rbu rbu1.db do_test 3.2.0 { sqlite3rbu rbu file:test.db?rbu_exclusive_checkpoint=0 rbu1.db rbu step } SQLITE_OK do_test 3.3.1 { set rc [catch {rbu close} msg] lappend rc $msg } {0 SQLITE_OK} db close create_rbu rbu1.db do_test 3.4.0 { sqlite3rbu rbu file:test.db?rbu_exclusive_checkpoint=0 rbu1.db rbu step } SQLITE_OK rbu close finish_test |
Added ext/rbu/rbuexpr.test.
> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 | # 2014 August 30 # # The author disclaims copyright to this source code. In place of # a legal notice, here is a blessing: # # May you do good and not evil. # May you find forgiveness for yourself and forgive others. # May you share freely, never taking more than you give. # #*********************************************************************** # source [file join [file dirname [info script]] rbu_common.tcl] ifcapable !rbu { finish_test ; return } set ::testprefix rbuexpr db close sqlite3_shutdown sqlite3_config_uri 1 sqlite3 db test.db do_execsql_test 1.0 { CREATE TABLE t1(a, b, c PRIMARY KEY); CREATE INDEX i1 ON t1(a, null, b+1); CREATE INDEX i2 ON t1(a+1, b+1, c+1); INSERT INTO t1 VALUES(1, 2, 3); INSERT INTO t1 VALUES(4, 5, 6); INSERT INTO t1 VALUES(7, 8, 9); INSERT INTO t1 VALUES(10, 11, 12); PRAGMA integrity_check; } {ok} forcedelete rbu.db sqlite3 db2 rbu.db do_execsql_test -db db2 1.1 { CREATE TABLE data_t1(a, b, c, rbu_control); INSERT INTO data_t1 VALUES(13, 14, 15, 0); INSERT INTO data_t1 VALUES(NULL, NULL, 6, 1); INSERT INTO data_t1 VALUES(NULL, 'three', 3, '.x.'); } db2 close db close do_test 1.2 { run_rbu test.db rbu.db } {SQLITE_DONE} sqlite3 db test.db do_execsql_test 1.3 { SELECT * FROM t1 WHERE a=4; } integrity_check 1.4 #------------------------------------------------------------------------- # reset_db do_execsql_test 2.0 { CREATE TABLE t1(c1, c2, c3, i INTEGER PRIMARY KEY); INSERT INTO t1 VALUES('one', 'one', 'one', 1); INSERT INTO t1 VALUES('two', 'two', 'two', 2); INSERT INTO t1 VALUES('three', 'three', 'three', 3); INSERT INTO t1 VALUES('four', 'four', 'four', 4); CREATE INDEX i1 ON t1( substr(c1, 1, 2) ); CREATE INDEX i2 ON t1( c1 || c2 || c3 ); CREATE INDEX i3 ON t1( length(c1) + length(c2) - 1, c3||i ); } forcedelete rbu.db sqlite3 db2 rbu.db do_execsql_test -db db2 2.1 { CREATE TABLE data_t1(c1, c2, c3, i, rbu_control); INSERT INTO data_t1 VALUES(NULL, NULL, NULL, 2, 1); INSERT INTO data_t1 VALUES('thirty', NULL, NULL, 3, 'xx..'); INSERT INTO data_t1 VALUES('five', 'five', 'five', 5, 0); } db2 close db close do_test 2.2 { run_rbu test.db rbu.db } {SQLITE_DONE} sqlite3 db test.db integrity_check 2.3 finish_test |
Changes to ext/rbu/rbufault.test.
1 2 3 4 5 6 7 8 9 10 11 12 | # 2014 October 22 # # The author disclaims copyright to this source code. In place of # a legal notice, here is a blessing: # # May you do good and not evil. # May you find forgiveness for yourself and forgive others. # May you share freely, never taking more than you give. # #*********************************************************************** # | < | < | | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 | # 2014 October 22 # # The author disclaims copyright to this source code. In place of # a legal notice, here is a blessing: # # May you do good and not evil. # May you find forgiveness for yourself and forgive others. # May you share freely, never taking more than you give. # #*********************************************************************** # source [file join [file dirname [info script]] rbu_common.tcl] ifcapable !rbu { finish_test ; return } source $testdir/malloc_common.tcl set ::testprefix rbufault proc copy_if_exists {src target} { if {[file exists $src]} { forcecopy $src $target } else { |
︙ | ︙ |
Changes to ext/rbu/rbufault2.test.
1 2 3 4 5 6 7 8 9 10 11 12 | # 2014 October 22 # # The author disclaims copyright to this source code. In place of # a legal notice, here is a blessing: # # May you do good and not evil. # May you find forgiveness for yourself and forgive others. # May you share freely, never taking more than you give. # #*********************************************************************** # | < | < | | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 | # 2014 October 22 # # The author disclaims copyright to this source code. In place of # a legal notice, here is a blessing: # # May you do good and not evil. # May you find forgiveness for yourself and forgive others. # May you share freely, never taking more than you give. # #*********************************************************************** # source [file join [file dirname [info script]] rbu_common.tcl] ifcapable !rbu { finish_test ; return } source $testdir/malloc_common.tcl set ::testprefix rbufault2 forcedelete rbu.db do_execsql_test 1.0 { CREATE TABLE target(x UNIQUE, y, z, PRIMARY KEY(y)); INSERT INTO target VALUES(1, 2, 3); |
︙ | ︙ | |||
48 49 50 51 52 53 54 55 56 57 | {1 SQLITE_CONSTRAINT} \ {1 SQLITE_NOMEM} \ {1 {SQLITE_NOMEM - unable to open a temporary database file for storing temporary tables}} \ {1 {SQLITE_NOMEM - out of memory}} } finish_test | > > > > > > > > > | 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 | {1 SQLITE_CONSTRAINT} \ {1 SQLITE_NOMEM} \ {1 {SQLITE_NOMEM - unable to open a temporary database file for storing temporary tables}} \ {1 {SQLITE_NOMEM - out of memory}} } sqlite3rbu_create_vfs -default rbu "" sqlite3 db test.db set ::vfsname [file_control_vfsname db] do_faultsim_test 2 -faults oom* -prep { } -body { file_control_vfsname db } db close sqlite3rbu_destroy_vfs rbu finish_test |
Changes to ext/rbu/rbufault3.test.
︙ | ︙ | |||
9 10 11 12 13 14 15 16 17 18 19 20 21 22 | # #*********************************************************************** # # This file contains fault injection tests for RBU vacuum operations. # source [file join [file dirname [info script]] rbu_common.tcl] source $testdir/malloc_common.tcl set ::testprefix rbufault3 foreach {fault errlist} { oom-* { {1 SQLITE_NOMEM} {1 SQLITE_IOERR_NOMEM} | > | 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 | # #*********************************************************************** # # This file contains fault injection tests for RBU vacuum operations. # source [file join [file dirname [info script]] rbu_common.tcl] ifcapable !rbu { finish_test ; return } source $testdir/malloc_common.tcl set ::testprefix rbufault3 foreach {fault errlist} { oom-* { {1 SQLITE_NOMEM} {1 SQLITE_IOERR_NOMEM} |
︙ | ︙ | |||
79 80 81 82 83 84 85 | sqlite3rbu_vacuum rbu test.db test.db2 rbu step rbu close faultsim_save_and_close do_faultsim_test 3 -faults $fault -prep { faultsim_restore_and_reopen | < < | 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 | sqlite3rbu_vacuum rbu test.db test.db2 rbu step rbu close faultsim_save_and_close do_faultsim_test 3 -faults $fault -prep { faultsim_restore_and_reopen } -body { sqlite3rbu_vacuum rbu test.db test.db2 rbu step rbu close } -test { eval [list faultsim_test_result {0 SQLITE_OK} {*}$::errlist] } } finish_test |
Changes to ext/rbu/rbufault4.test.
1 2 3 4 5 6 7 8 9 10 11 12 | # 2014 October 22 # # The author disclaims copyright to this source code. In place of # a legal notice, here is a blessing: # # May you do good and not evil. # May you find forgiveness for yourself and forgive others. # May you share freely, never taking more than you give. # #*********************************************************************** # | < | < | | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 | # 2014 October 22 # # The author disclaims copyright to this source code. In place of # a legal notice, here is a blessing: # # May you do good and not evil. # May you find forgiveness for yourself and forgive others. # May you share freely, never taking more than you give. # #*********************************************************************** # source [file join [file dirname [info script]] rbu_common.tcl] ifcapable !rbu { finish_test ; return } source $testdir/malloc_common.tcl set ::testprefix rbufault4 for {set tn 1} {1} {incr tn} { reset_db do_execsql_test 1.0 { CREATE TABLE t1(a INTEGER PRIMARY KEY, b, c); |
︙ | ︙ |
Changes to ext/rbu/rbufts.test.
︙ | ︙ | |||
9 10 11 12 13 14 15 | # #*********************************************************************** # # This file contains tests for the RBU module. More specifically, it # contains tests to ensure that RBU works with FTS tables. # | < | < | | 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 | # #*********************************************************************** # # This file contains tests for the RBU module. More specifically, it # contains tests to ensure that RBU works with FTS tables. # source [file join [file dirname [info script]] rbu_common.tcl] ifcapable !rbu { finish_test ; return } set ::testprefix rbufts ifcapable !fts3 { finish_test return } |
︙ | ︙ |
Added ext/rbu/rbumisc.test.
> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 | # 2014 August 30 # # The author disclaims copyright to this source code. In place of # a legal notice, here is a blessing: # # May you do good and not evil. # May you find forgiveness for yourself and forgive others. # May you share freely, never taking more than you give. # #*********************************************************************** # source [file join [file dirname [info script]] rbu_common.tcl] ifcapable !rbu { finish_test ; return } set ::testprefix rbumisc db close sqlite3_shutdown sqlite3_config_uri 1 reset_db proc populate_rbu_db {} { forcedelete rbu.db sqlite3 rbu rbu.db rbu eval { CREATE TABLE data_x1(a, b, c, rbu_control); INSERT INTO data_x1 VALUES(1, 1, 1, 0); INSERT INTO data_x1 VALUES(2, 2, 2, 0); CREATE TABLE dat(a, b, c, rbu_control); CREATE TABLE "data x1"(a, b, c, rbu_control); CREATE TABLE datax1(a, b, c, rbu_control); CREATE TABLE data_(a, b, c, rbu_control); INSERT INTO "data x1" VALUES(3, 3, 3, 0); INSERT INTO datax1 VALUES(3, 3, 3, 0); INSERT INTO data_ VALUES(3, 3, 3, 0); INSERT INTO dat VALUES(3, 3, 3, 0); } rbu close } #------------------------------------------------------------------------- # Ensure that RBU is not confused by oddly named tables in an RBU # database. # do_execsql_test 1.0 { CREATE TABLE x1(a, b, c INTEGER PRIMARY KEY); } do_test 1.1 { populate_rbu_db } {} do_test 1.2 { step_rbu test.db rbu.db db eval { SELECT * FROM x1 } } {1 1 1 2 2 2} do_test 1.3 { db eval { DELETE FROM x1 } sqlite3 rbu rbu.db rbu eval { DELETE FROM rbu_state } rbu close step_rbu test.db rbu.db db eval { SELECT * FROM x1 } } {1 1 1 2 2 2} do_test 1.4 { db eval { DELETE FROM x1 } populate_rbu_db sqlite3rbu rbu test.db rbu.db rbu step rbu step rbu close forcecopy test.db-oal test.db-wal sqlite3rbu rbu test.db rbu.db rbu step list [catch { rbu close } msg] $msg } {1 {SQLITE_ERROR - cannot update wal mode database}} #------------------------------------------------------------------------- # Test the effect of a wal file appearing after the target database has # been opened, but before it has been locked. # catch { db close } testvfs tvfs -default 1 for {set N 1} {$N < 10} {incr N} { reset_db populate_rbu_db do_execsql_test 2.$N.0 { CREATE TABLE x1(a, b, c INTEGER PRIMARY KEY); } set nAccessCnt 0 do_test 2.$N.1 { sqlite3rbu rbu test.db rbu.db rbu step rbu step rbu close } {SQLITE_OK} tvfs script xAccess tvfs filter xAccess set nAccessCnt 0 proc xAccess {method file args} { global nAccessCnt if {[file tail $file]=="test.db-wal"} { incr nAccessCnt -1 if {$nAccessCnt==0} { set fd [open test.db-wal w] puts -nonewline $fd [string repeat 0 2000] close $fd } } return SQLITE_OK } foreach r { {1 {SQLITE_ERROR - cannot update wal mode database}} {0 SQLITE_OK} {1 {SQLITE_CANTOPEN - unable to open database file}} } { set RES($r) 1 } do_test 2.$N.2 { set ::nAccessCnt $N set res [list [catch { sqlite3rbu rbu test.db rbu.db rbu step rbu close } msg ] $msg] set RES($res) } {1} catch {rbu close} } catch {db close} catch {tvfs delete} #------------------------------------------------------------------------- testvfs tvfs -default 1 reset_db populate_rbu_db do_execsql_test 3.0 { CREATE TABLE x1(a, b, c INTEGER PRIMARY KEY); } tvfs script xFileControl tvfs filter xFileControl proc xFileControl {method file verb args} { if {$verb=="ZIPVFS" && [info exists ::zipvfs_filecontrol]} { return $::zipvfs_filecontrol } return "SQLITE_NOTFOUND" } breakpoint foreach {tn ret err} { 1 SQLITE_OK 0 2 SQLITE_ERROR 1 3 SQLITE_NOTFOUND 0 4 SQLITE_OMIT 1 } { set ::zipvfs_filecontrol $ret do_test 3.$tn.1 { catch { sqlite3rbu rbu test.db rbu.db rbu step rbu close } } $err } catch {db close} catch {tvfs delete} #------------------------------------------------------------------------- finish_test |
Changes to ext/rbu/rbumulti.test.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 | # 2018 January 11 # # The author disclaims copyright to this source code. In place of # a legal notice, here is a blessing: # # May you do good and not evil. # May you find forgiveness for yourself and forgive others. # May you share freely, never taking more than you give. # #*********************************************************************** # # This file contains tests of multiple RBU operations running # concurrently within the same process. # source [file join [file dirname [info script]] rbu_common.tcl] set ::testprefix rbumulti db close sqlite3_shutdown sqlite3_config_uri 1 autoinstall_test_functions | > > > | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 | # 2018 January 11 # # The author disclaims copyright to this source code. In place of # a legal notice, here is a blessing: # # May you do good and not evil. # May you find forgiveness for yourself and forgive others. # May you share freely, never taking more than you give. # #*********************************************************************** # # TESTRUNNER: slow # # This file contains tests of multiple RBU operations running # concurrently within the same process. # source [file join [file dirname [info script]] rbu_common.tcl] ifcapable !rbu { finish_test ; return } set ::testprefix rbumulti db close sqlite3_shutdown sqlite3_config_uri 1 autoinstall_test_functions |
︙ | ︙ |
Added ext/rbu/rbupartial.test.
> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 | # 2019 April 11 # # The author disclaims copyright to this source code. In place of # a legal notice, here is a blessing: # # May you do good and not evil. # May you find forgiveness for yourself and forgive others. # May you share freely, never taking more than you give. # #*********************************************************************** # source [file join [file dirname [info script]] rbu_common.tcl] ifcapable !rbu { finish_test ; return } set ::testprefix rbupartial db close sqlite3_shutdown sqlite3_config_uri 1 foreach {tn without_rowid a b c d} { 1 "" a b c d 2 "WITHOUT ROWID" aaa bbb ccc ddd 3 "WITHOUT ROWID" "\"hello\"" {"one'two"} {[c]} ddd 4 "WITHOUT ROWID" {`a b`} {"one'two"} {[c c c]} ddd 5 "" a b c {"d""d"} 6 "" {'one''two'} b {"c""c"} {"d""d"} } { eval [string map [list \ %WITHOUT_ROWID% $without_rowid %A% $a %B% $b %C% $c %D% $d ] { reset_db do_execsql_test $tn.1.0 { CREATE TABLE t1(%A% PRIMARY KEY, %B%, %C%, %D%) %WITHOUT_ROWID% ; CREATE INDEX i1b ON t1(%B%); CREATE INDEX i1b2 ON t1(%B%) WHERE %C%<5; CREATE INDEX i1b3 ON t1(%B%) WHERE %C%>=5; CREATE INDEX i1c ON t1(%C%); CREATE INDEX i1c2 ON t1(%C%) WHERE %C% IS NULL; CREATE INDEX i1c3 ON t1(%C%) WHERE %C% IS NOT NULL; CREATE INDEX i1c4 ON t1(%C%) WHERE %D% < 'd'; CREATE INDEX i1c5 ON t1( %C% -- for (c = ... expressions ) WHERE %D% < 'd'; CREATE INDEX i1c6 ON t1( %C% /* Again, for (c=... expr */, %D% ) WHERE %D% < 'd'; CREATE INDEX i1c7 ON t1( %C% /* As before, for (c=... "expr */) WHERE %D% < 'd'; } do_execsql_test $tn.1.1 { INSERT INTO t1 VALUES(0, NULL, NULL, 'a'); INSERT INTO t1 VALUES(1, 2, 3, 'b'); INSERT INTO t1 VALUES(4, 5, 6, 'c'); INSERT INTO t1 VALUES(7, 8, 9, 'd'); } forcedelete rbu.db do_test $tn.1.2 { sqlite3 rbu rbu.db rbu eval { CREATE TABLE data_t1(%A%, %B%, %C%, %D%, rbu_control); INSERT INTO data_t1 VALUES(10, 11, 12, 'e', 0); INSERT INTO data_t1 VALUES(13, 14, NULL, 'f', 0); INSERT INTO data_t1 VALUES(0, NULL, NULL, NULL, 1); INSERT INTO data_t1 VALUES(4, NULL, NULL, NULL, 1); INSERT INTO data_t1 VALUES(7, NULL, 4, NULL, '..x.'); INSERT INTO data_t1 VALUES(1, 10, NULL, NULL, '.xx.'); } rbu close } {} do_test $tn.1.3 { run_rbu test.db rbu.db execsql { PRAGMA integrity_check } } {ok} do_execsql_test $tn.1.4 { SELECT * FROM t1 ORDER BY %A%; } { 1 10 {} b 7 8 4 d 10 11 12 e 13 14 {} f } set step 0 do_rbu_vacuum_test $tn.1.5 0 do_test $tn.1.6 { execsql { PRAGMA integrity_check } } {ok} }] } finish_test |
Added ext/rbu/rbupass.test.
> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 | # 2023 January 13 # # The author disclaims copyright to this source code. In place of # a legal notice, here is a blessing: # # May you do good and not evil. # May you find forgiveness for yourself and forgive others. # May you share freely, never taking more than you give. # #*********************************************************************** # source [file join [file dirname [info script]] rbu_common.tcl] ifcapable !rbu { finish_test ; return } set ::testprefix rbupass if {[info commands register_demovfs]==""} { finish_test return } db close sqlite3_shutdown sqlite3_config_uri 1 register_demovfs sqlite3rbu_create_vfs myvfs demo sqlite3 db file:test.db?vfs=myvfs do_execsql_test 1.0 { CREATE TABLE t1(a INTEGER PRIMARY KEY, b); INSERT INTO t1 VALUES(1, 2); SELECT * FROM t1; } {1 2} do_execsql_test 1.1 { PRAGMA journal_mode = wal; } {delete} do_execsql_test 1.2 { SELECT * FROM t1; } {1 2} do_test 1.3 { forcedelete rbu.db sqlite3 rbu rbu.db rbu eval { CREATE TABLE data_t1(a, b, rbu_control); INSERT INTO data_t1 VALUES(2, 4, 0); } rbu close } {} do_test 1.4 { sqlite3rbu rbu test.db rbu.db } {rbu} do_test 1.5 { rbu step } {SQLITE_CANTOPEN} do_test 1.6 { list [catch { rbu close } msg] $msg } {1 {SQLITE_CANTOPEN - unable to open database file}} do_test 1.7 { sqlite3rbu_vacuum rbu test.db } {rbu} do_test 1.8 { rbu step catch { rbu close } } {1} do_execsql_test 1.9 { SELECT * FROM t1; } {1 2} db close sqlite3rbu_destroy_vfs myvfs unregister_demovfs sqlite3_shutdown finish_test |
Changes to ext/rbu/rbuprogress.test.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 | # 2016 March 18 # # The author disclaims copyright to this source code. In place of # a legal notice, here is a blessing: # # May you do good and not evil. # May you find forgiveness for yourself and forgive others. # May you share freely, never taking more than you give. # #*********************************************************************** # source [file join [file dirname [info script]] rbu_common.tcl] set ::testprefix rbuprogress proc create_db_file {filename sql} { forcedelete $filename sqlite3 tmpdb $filename tmpdb eval $sql | > | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 | # 2016 March 18 # # The author disclaims copyright to this source code. In place of # a legal notice, here is a blessing: # # May you do good and not evil. # May you find forgiveness for yourself and forgive others. # May you share freely, never taking more than you give. # #*********************************************************************** # source [file join [file dirname [info script]] rbu_common.tcl] ifcapable !rbu { finish_test ; return } set ::testprefix rbuprogress proc create_db_file {filename sql} { forcedelete $filename sqlite3 tmpdb $filename tmpdb eval $sql |
︙ | ︙ | |||
409 410 411 412 413 414 415 416 | set R(nopk) $r1 set R(vtab) $r2 do_sp_test 5.$tn.$bReopen.$tn2.1 $bReopen test.db rbu.db $R($tn) } } } | > > > | > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | 410 411 412 413 414 415 416 417 418 419 420 421 422 423 424 425 426 427 428 429 430 431 432 433 434 435 436 437 438 439 440 441 442 443 444 445 446 447 448 449 450 451 | set R(nopk) $r1 set R(vtab) $r2 do_sp_test 5.$tn.$bReopen.$tn2.1 $bReopen test.db rbu.db $R($tn) } } } #------------------------------------------------------------------------- # Test that sqlite3_bp_progress() works with an RBU vacuum if there # is an rbu_count table in the db being vacuumed. # reset_db do_execsql_test 6.0 { CREATE TABLE t1(a, b, c); CREATE INDEX i1 ON t1(a); CREATE INDEX i2 ON t1(b); WITH s(i) AS ( SELECT 1 UNION ALL SELECT i+1 FROM s WHERE i<100 ) INSERT INTO t1 SELECT i, i, i FROM s; CREATE TABLE rbu_count(tbl TEXT PRIMARY KEY, cnt INTEGER) WITHOUT ROWID; INSERT INTO rbu_count VALUES('t1', (SELECT count(*) FROM t1)); INSERT INTO rbu_count VALUES('rbu_count', 2); } forcedelete state.db do_test 6.1 { set maxA 0 set maxB 0 sqlite3rbu_vacuum rbu test.db state.db while {[rbu step]=="SQLITE_OK"} { foreach {a b} [rbu bp_progress] { if {$a > $maxA} { set maxA $a } if {$b > $maxB} { set maxB $b } } } list [rbu close] $maxA $maxB } {SQLITE_DONE 10000 10000} finish_test |
Added ext/rbu/rburename.test.
> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 | # 2022 November 07 # # The author disclaims copyright to this source code. In place of # a legal notice, here is a blessing: # # May you do good and not evil. # May you find forgiveness for yourself and forgive others. # May you share freely, never taking more than you give. # #*********************************************************************** # # source [file join [file dirname [info script]] rbu_common.tcl] ifcapable !rbu { finish_test ; return } set ::testprefix rburename do_execsql_test 1.0 { CREATE TABLE t1(a, b); INSERT INTO t1 VALUES(1, 2); INSERT INTO t1 VALUES(3, 4); INSERT INTO t1 VALUES(5, 6); } forcedelete test.db-vacuum proc my_rename {old new} { lappend ::my_rename_calls [list [file tail $old] [file tail $new]] file rename $old $new } do_test 1.1 { sqlite3rbu_vacuum rbu test.db rbu rename_handler my_rename while {[rbu step]=="SQLITE_OK"} {} rbu close } SQLITE_DONE do_test 1.2 { set ::my_rename_calls } {{test.db-oal test.db-wal}} proc my_rename {old new} { error "something went wrong" } do_test 1.3 { sqlite3rbu_vacuum rbu test.db rbu rename_handler my_rename while {[rbu step]=="SQLITE_OK"} {} list [catch { rbu close } msg] $msg } {1 SQLITE_IOERR} finish_test |
Changes to ext/rbu/rburesume.test.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 | # 2017 January 13 # # The author disclaims copyright to this source code. In place of # a legal notice, here is a blessing: # # May you do good and not evil. # May you find forgiveness for yourself and forgive others. # May you share freely, never taking more than you give. # #*********************************************************************** # # This file contains tests for resumption of RBU operations in the # case where the previous RBU process crashed. # source [file join [file dirname [info script]] rbu_common.tcl] set ::testprefix rburesume forcedelete test.db-shm test.db-oal do_execsql_test 1.0 { CREATE TABLE t1(a PRIMARY KEY, b, c); CREATE INDEX t1a ON t1(a); CREATE INDEX t1b ON t1(b); | > > > | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 | # 2017 January 13 # # The author disclaims copyright to this source code. In place of # a legal notice, here is a blessing: # # May you do good and not evil. # May you find forgiveness for yourself and forgive others. # May you share freely, never taking more than you give. # #*********************************************************************** # # TESTRUNNER: slow # # This file contains tests for resumption of RBU operations in the # case where the previous RBU process crashed. # source [file join [file dirname [info script]] rbu_common.tcl] ifcapable !rbu { finish_test ; return } set ::testprefix rburesume forcedelete test.db-shm test.db-oal do_execsql_test 1.0 { CREATE TABLE t1(a PRIMARY KEY, b, c); CREATE INDEX t1a ON t1(a); CREATE INDEX t1b ON t1(b); |
︙ | ︙ |
Changes to ext/rbu/rbusave.test.
1 2 3 4 5 6 7 8 9 10 11 12 | # 2015 August 14 # # The author disclaims copyright to this source code. In place of # a legal notice, here is a blessing: # # May you do good and not evil. # May you find forgiveness for yourself and forgive others. # May you share freely, never taking more than you give. # #*********************************************************************** # | < | < | | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 | # 2015 August 14 # # The author disclaims copyright to this source code. In place of # a legal notice, here is a blessing: # # May you do good and not evil. # May you find forgiveness for yourself and forgive others. # May you share freely, never taking more than you give. # #*********************************************************************** # source [file join [file dirname [info script]] rbu_common.tcl] ifcapable !rbu { finish_test ; return } set ::testprefix rbusave do_execsql_test 1.0 { CREATE TABLE t1(a PRIMARY KEY, b, c) WITHOUT ROWID; CREATE TABLE t2(a INTEGER PRIMARY KEY, b, c); CREATE INDEX i1 ON t1(b); CREATE INDEX i2 ON t2(c, b); |
︙ | ︙ |
Changes to ext/rbu/rbusplit.test.
︙ | ︙ | |||
8 9 10 11 12 13 14 15 16 17 18 19 20 21 | # May you share freely, never taking more than you give. # #*********************************************************************** # # source [file join [file dirname [info script]] rbu_common.tcl] set ::testprefix rbusplit db close sqlite3_shutdown sqlite3_config_uri 1 autoinstall_test_functions | > | 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 | # May you share freely, never taking more than you give. # #*********************************************************************** # # source [file join [file dirname [info script]] rbu_common.tcl] ifcapable !rbu { finish_test ; return } set ::testprefix rbusplit db close sqlite3_shutdown sqlite3_config_uri 1 autoinstall_test_functions |
︙ | ︙ |
Changes to ext/rbu/rbutemplimit.test.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 | # 2014 August 30 # # The author disclaims copyright to this source code. In place of # a legal notice, here is a blessing: # # May you do good and not evil. # May you find forgiveness for yourself and forgive others. # May you share freely, never taking more than you give. # #*********************************************************************** # source [file join [file dirname [info script]] rbu_common.tcl] set ::testprefix rbutemplimit db close sqlite3_shutdown sqlite3_config_uri 1 proc setup_databases {} { | > > | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 | # 2014 August 30 # # The author disclaims copyright to this source code. In place of # a legal notice, here is a blessing: # # May you do good and not evil. # May you find forgiveness for yourself and forgive others. # May you share freely, never taking more than you give. # #*********************************************************************** # # TESTRUNNER: slow source [file join [file dirname [info script]] rbu_common.tcl] ifcapable !rbu { finish_test ; return } set ::testprefix rbutemplimit db close sqlite3_shutdown sqlite3_config_uri 1 proc setup_databases {} { |
︙ | ︙ | |||
61 62 63 64 65 66 67 68 69 70 71 72 73 74 | } proc step_rbu_cachesize {target rbu stepsize cachesize temp_limit} { set res "" while 1 { sqlite3rbu rbu $target $rbu rbu temp_size_limit $temp_limit sqlite3_exec_nr [rbu db 1] "PRAGMA cache_size = $cachesize" for {set i 0} {$i < $stepsize} {incr i} { set rc [rbu step] set ::A([rbu temp_size]) 1 if {$rc!="SQLITE_OK"} break } set res [list [catch {rbu close} msg] $msg] | > | 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 | } proc step_rbu_cachesize {target rbu stepsize cachesize temp_limit} { set res "" while 1 { sqlite3rbu rbu $target $rbu rbu temp_size_limit $temp_limit if { [rbu temp_size_limit -1]!=$temp_limit } { error "round trip problem!" } sqlite3_exec_nr [rbu db 1] "PRAGMA cache_size = $cachesize" for {set i 0} {$i < $stepsize} {incr i} { set rc [rbu step] set ::A([rbu temp_size]) 1 if {$rc!="SQLITE_OK"} break } set res [list [catch {rbu close} msg] $msg] |
︙ | ︙ |
Changes to ext/rbu/rbuvacuum.test.
︙ | ︙ | |||
11 12 13 14 15 16 17 18 19 20 21 22 23 24 | # # This file contains tests for the RBU module. More specifically, it # contains tests to ensure that the sqlite3rbu_vacuum() API works as # expected. # source [file join [file dirname [info script]] rbu_common.tcl] set ::testprefix rbuvacuum foreach step {0 1} { set ::testprefix rbuvacuum-step=$step reset_db | > | 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 | # # This file contains tests for the RBU module. More specifically, it # contains tests to ensure that the sqlite3rbu_vacuum() API works as # expected. # source [file join [file dirname [info script]] rbu_common.tcl] ifcapable !rbu { finish_test ; return } set ::testprefix rbuvacuum foreach step {0 1} { set ::testprefix rbuvacuum-step=$step reset_db |
︙ | ︙ |
Changes to ext/rbu/rbuvacuum2.test.
︙ | ︙ | |||
11 12 13 14 15 16 17 18 19 20 21 22 23 24 | # # This file contains tests for the RBU module. More specifically, it # contains tests to ensure that the sqlite3rbu_vacuum() API works as # expected. # source [file join [file dirname [info script]] rbu_common.tcl] foreach {step} {0 1} { foreach {ttt state} { s state.db t test.db-vacuum n {} } { set ::testprefix rbuvacuum2-$step$ttt | > | 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 | # # This file contains tests for the RBU module. More specifically, it # contains tests to ensure that the sqlite3rbu_vacuum() API works as # expected. # source [file join [file dirname [info script]] rbu_common.tcl] ifcapable !rbu { finish_test ; return } foreach {step} {0 1} { foreach {ttt state} { s state.db t test.db-vacuum n {} } { set ::testprefix rbuvacuum2-$step$ttt |
︙ | ︙ | |||
194 195 196 197 198 199 200 | do_test 5.$tn.1 { file attributes test.db -permissions $perm sqlite3rbu_vacuum rbu test.db rbu step } {SQLITE_OK} do_test 5.$tn.2 { file exists test.db-vacuum } 1 | > > > > | | 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 | do_test 5.$tn.1 { file attributes test.db -permissions $perm sqlite3rbu_vacuum rbu test.db rbu step } {SQLITE_OK} do_test 5.$tn.2 { file exists test.db-vacuum } 1 # The result pattern might be 00xxx or 0oxxx depending on which # version of TCL is being used. So make perm2 into a regexp that # will match either regsub {^00} $perm {0.} perm2 do_test 5.$tn.3 { file attributes test.db-vacuum -permissions} /$perm2/ rbu close } } #------------------------------------------------------------------------- # Test the outcome of some other connection running a checkpoint while # the incremental checkpoint is suspended. |
︙ | ︙ | |||
219 220 221 222 223 224 225 | do_test 6.1 { sqlite3rbu_vacuum rbu test.db test.db2 while {[rbu state]!="checkpoint"} { rbu step } rbu close } {SQLITE_OK} | | | | > | | 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 | do_test 6.1 { sqlite3rbu_vacuum rbu test.db test.db2 while {[rbu state]!="checkpoint"} { rbu step } rbu close } {SQLITE_OK} do_test 6.2 { execsql { SELECT 1 FROM sqlite_master LIMIT 1 } execsql { PRAGMA wal_checkpoint } execsql { SELECT 1 FROM sqlite_master LIMIT 1 } } {1} do_test 6.3 { sqlite3rbu_vacuum rbu test.db test.db2 while {[rbu step]!="SQLITE_DONE"} { rbu step } rbu close execsql { PRAGMA integrity_check } } {ok} |
︙ | ︙ |
Added ext/rbu/rbuvacuum3.test.
> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 | # 2019 Jan 3 # # The author disclaims copyright to this source code. In place of # a legal notice, here is a blessing: # # May you do good and not evil. # May you find forgiveness for yourself and forgive others. # May you share freely, never taking more than you give. # #*********************************************************************** # # This file contains tests for the RBU module. More specifically, it # contains tests to ensure that the sqlite3rbu_vacuum() API works as # expected. # source [file join [file dirname [info script]] rbu_common.tcl] ifcapable !rbu { finish_test ; return } set testprefix rbuvacuum3 do_execsql_test 1.0 { CREATE TABLE t1(a PRIMARY KEY, b, c); CREATE INDEX i1b ON t1(b); CREATE INDEX i1c ON t1(c); WITH s(i) AS ( VALUES(1) UNION ALL SELECT i+1 FROM s WHERE i<100 ) INSERT INTO t1 SELECT i, randomblob(100), randomblob(100) FROM s; } forcedelete state.db do_test 1.1 { sqlite3rbu_vacuum rbu test.db state.db while {1} { set rc [rbu step] if {$rc!="SQLITE_OK"} break rbu savestate } rbu close } {SQLITE_DONE} do_test 1.2 { sqlite3rbu_vacuum rbu test.db state.db while {1} { set rc [rbu step] if {$rc!="SQLITE_OK"} break rbu savestate } rbu close } {SQLITE_DONE} do_test 1.3 { while {1} { sqlite3rbu_vacuum rbu test.db state.db set rc [rbu step] if {$rc!="SQLITE_OK"} break rbu savestate rbu close } rbu close } {SQLITE_DONE} finish_test |
Added ext/rbu/rbuvacuum4.test.
> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 | # 2019 Jan 3 # # The author disclaims copyright to this source code. In place of # a legal notice, here is a blessing: # # May you do good and not evil. # May you find forgiveness for yourself and forgive others. # May you share freely, never taking more than you give. # #*********************************************************************** # # This file contains tests for the RBU module. More specifically, it # contains tests to ensure that the sqlite3rbu_vacuum() API works as # expected. # source [file join [file dirname [info script]] rbu_common.tcl] ifcapable !rbu { finish_test ; return } set testprefix rbuvacuum4 set step 1 do_execsql_test 1.0 { CREATE TABLE t1(a PRIMARY KEY, b, c) WITHOUT ROWID; INSERT INTO t1 VALUES(1, 2, 3); INSERT INTO t1 VALUES(4, 5, 6); INSERT INTO t1 VALUES(7, 8, 9); } do_rbu_vacuum_test 1.1 1 #------------------------------------------------------------------------- reset_db do_execsql_test 2.0 { CREATE TABLE t1(a, b, c, PRIMARY KEY(a, b, c)) WITHOUT ROWID; INSERT INTO t1 VALUES(1, 2, 3); INSERT INTO t1 VALUES(4, 5, 6); INSERT INTO t1 VALUES(7, 8, 9); } do_rbu_vacuum_test 2.1 1 do_execsql_test 2.2 { SELECT * FROM t1; } {1 2 3 4 5 6 7 8 9} #------------------------------------------------------------------------- reset_db do_execsql_test 3.0 { CREATE TABLE t1(a INTEGER PRIMARY KEY, b, c); CREATE INDEX i1 oN t1(b, c); INSERT INTO t1 VALUES(1, 2, 3); INSERT INTO t1 VALUES(4, 5, 6); INSERT INTO t1 VALUES(7, 8, 9); CREATE TABLE t2(a, b, c INTEGER, PRIMARY KEY(c)); CREATE INDEX i2 oN t2(b, a); INSERT INTO t2 VALUES('a', 'b', -1); INSERT INTO t2 VALUES('c', 'd', -2); INSERT INTO t2 VALUES('e', 'f', -3); } do_rbu_vacuum_test 3.1 1 do_execsql_test 3.2 { SELECT * FROM t1; SELECT * FROM t2; } {1 2 3 4 5 6 7 8 9 e f -3 c d -2 a b -1} #------------------------------------------------------------------------- reset_db do_execsql_test 4.0 { CREATE TABLE x1(a, b, c, d, PRIMARY KEY(c, b)) WITHOUT ROWID; INSERT INTO x1 VALUES(1, 1, 1, 1); INSERT INTO x1 VALUES(1, 1, 2, 1); INSERT INTO x1 VALUES(1, 2, 2, 1); INSERT INTO x1 VALUES(NULL, 2, 3, NULL); INSERT INTO x1 VALUES(NULL, 2, 4, NULL); INSERT INTO x1 VALUES(NULL, 2, 5, NULL); CREATE INDEX x1ad ON x1(d, a); CREATE INDEX x1null ON x1(d, a) WHERE d>15; } do_rbu_vacuum_test 4.1.1 1 do_execsql_test 4.2 { SELECT count(*) fROM x1 } 6 do_rbu_vacuum_test 4.1.2 0 #------------------------------------------------------------------------- reset_db do_execsql_test 5.0 { CREATE TABLE "a b c"(a, "b b" PRIMARY KEY, "c c"); CREATE INDEX abc1 ON "a b c"(a, "c c"); INSERT INTO "a b c" VALUES(NULL, 'a', NULL); INSERT INTO "a b c" VALUES(NULL, 'b', NULL); INSERT INTO "a b c" VALUES(NULL, 'c', NULL); INSERT INTO "a b c" VALUES(1, 2, 3); INSERT INTO "a b c" VALUES(3, 9, 1); INSERT INTO "a b c" VALUES('aaa', 'bbb', 'ccc'); CREATE INDEX abc2 ON "a b c"("c c" DESC, a); CREATE TABLE x(a); INSERT INTO x VALUES('a'), ('b'), ('d'); CREATE UNIQUE INDEX y ON x(a); } do_rbu_vacuum_test 5.1 1 finish_test |
Changes to ext/rbu/sqlite3rbu.c.
︙ | ︙ | |||
105 106 107 108 109 110 111 112 113 114 115 116 117 118 | /* ** Swap two objects of type TYPE. */ #if !defined(SQLITE_AMALGAMATION) # define SWAP(TYPE,A,B) {TYPE t=A; A=B; B=t;} #endif /* ** The rbu_state table is used to save the state of a partially applied ** update so that it can be resumed later. The table consists of integer ** keys mapped to values as follows: ** ** RBU_STATE_STAGE: | > > > > > > > | 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 | /* ** Swap two objects of type TYPE. */ #if !defined(SQLITE_AMALGAMATION) # define SWAP(TYPE,A,B) {TYPE t=A; A=B; B=t;} #endif /* ** Name of the URI option that causes RBU to take an exclusive lock as ** part of the incremental checkpoint operation. */ #define RBU_EXCLUSIVE_CHECKPOINT "rbu_exclusive_checkpoint" /* ** The rbu_state table is used to save the state of a partially applied ** update so that it can be resumed later. The table consists of integer ** keys mapped to values as follows: ** ** RBU_STATE_STAGE: |
︙ | ︙ | |||
178 179 180 181 182 183 184 185 186 187 188 189 190 191 | #define RBU_CREATE_STATE \ "CREATE TABLE IF NOT EXISTS %s.rbu_state(k INTEGER PRIMARY KEY, v)" typedef struct RbuFrame RbuFrame; typedef struct RbuObjIter RbuObjIter; typedef struct RbuState RbuState; typedef struct rbu_vfs rbu_vfs; typedef struct rbu_file rbu_file; typedef struct RbuUpdateStmt RbuUpdateStmt; #if !defined(SQLITE_AMALGAMATION) typedef unsigned int u32; typedef unsigned short u16; | > | 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 | #define RBU_CREATE_STATE \ "CREATE TABLE IF NOT EXISTS %s.rbu_state(k INTEGER PRIMARY KEY, v)" typedef struct RbuFrame RbuFrame; typedef struct RbuObjIter RbuObjIter; typedef struct RbuState RbuState; typedef struct RbuSpan RbuSpan; typedef struct rbu_vfs rbu_vfs; typedef struct rbu_file rbu_file; typedef struct RbuUpdateStmt RbuUpdateStmt; #if !defined(SQLITE_AMALGAMATION) typedef unsigned int u32; typedef unsigned short u16; |
︙ | ︙ | |||
222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 | struct RbuUpdateStmt { char *zMask; /* Copy of update mask used with pUpdate */ sqlite3_stmt *pUpdate; /* Last update statement (or NULL) */ RbuUpdateStmt *pNext; }; /* ** An iterator of this type is used to iterate through all objects in ** the target database that require updating. For each such table, the ** iterator visits, in order: ** ** * the table itself, ** * each index of the table (zero or more points to visit), and ** * a special "cleanup table" state. ** ** abIndexed: ** If the table has no indexes on it, abIndexed is set to NULL. Otherwise, ** it points to an array of flags nTblCol elements in size. The flag is ** set for each column that is either a part of the PK or a part of an ** index. Or clear otherwise. ** */ struct RbuObjIter { sqlite3_stmt *pTblIter; /* Iterate through tables */ sqlite3_stmt *pIdxIter; /* Index iterator */ int nTblCol; /* Size of azTblCol[] array */ char **azTblCol; /* Array of unquoted target column names */ | > > > > > > > > > > | 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 | struct RbuUpdateStmt { char *zMask; /* Copy of update mask used with pUpdate */ sqlite3_stmt *pUpdate; /* Last update statement (or NULL) */ RbuUpdateStmt *pNext; }; struct RbuSpan { const char *zSpan; int nSpan; }; /* ** An iterator of this type is used to iterate through all objects in ** the target database that require updating. For each such table, the ** iterator visits, in order: ** ** * the table itself, ** * each index of the table (zero or more points to visit), and ** * a special "cleanup table" state. ** ** abIndexed: ** If the table has no indexes on it, abIndexed is set to NULL. Otherwise, ** it points to an array of flags nTblCol elements in size. The flag is ** set for each column that is either a part of the PK or a part of an ** index. Or clear otherwise. ** ** If there are one or more partial indexes on the table, all fields of ** this array set set to 1. This is because in that case, the module has ** no way to tell which fields will be required to add and remove entries ** from the partial indexes. ** */ struct RbuObjIter { sqlite3_stmt *pTblIter; /* Iterate through tables */ sqlite3_stmt *pIdxIter; /* Index iterator */ int nTblCol; /* Size of azTblCol[] array */ char **azTblCol; /* Array of unquoted target column names */ |
︙ | ︙ | |||
266 267 268 269 270 271 272 273 274 275 276 277 278 279 | /* Statements created by rbuObjIterPrepareAll() */ int nCol; /* Number of columns in current object */ sqlite3_stmt *pSelect; /* Source data */ sqlite3_stmt *pInsert; /* Statement for INSERT operations */ sqlite3_stmt *pDelete; /* Statement for DELETE ops */ sqlite3_stmt *pTmpInsert; /* Insert into rbu_tmp_$zDataTbl */ /* Last UPDATE used (for PK b-tree updates only), or NULL. */ RbuUpdateStmt *pRbuUpdate; }; /* ** Values for RbuObjIter.eType | > > > | 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 | /* Statements created by rbuObjIterPrepareAll() */ int nCol; /* Number of columns in current object */ sqlite3_stmt *pSelect; /* Source data */ sqlite3_stmt *pInsert; /* Statement for INSERT operations */ sqlite3_stmt *pDelete; /* Statement for DELETE ops */ sqlite3_stmt *pTmpInsert; /* Insert into rbu_tmp_$zDataTbl */ int nIdxCol; RbuSpan *aIdxCol; char *zIdxSql; /* Last UPDATE used (for PK b-tree updates only), or NULL. */ RbuUpdateStmt *pRbuUpdate; }; /* ** Values for RbuObjIter.eType |
︙ | ︙ | |||
368 369 370 371 372 373 374 375 376 377 378 379 380 381 | int nProgress; /* Rows processed for all objects */ RbuObjIter objiter; /* Iterator for skipping through tbl/idx */ const char *zVfsName; /* Name of automatically created rbu vfs */ rbu_file *pTargetFd; /* File handle open on target db */ int nPagePerSector; /* Pages per sector for pTargetFd */ i64 iOalSz; i64 nPhaseOneStep; /* The following state variables are used as part of the incremental ** checkpoint stage (eStage==RBU_STAGE_CKPT). See comments surrounding ** function rbuSetupCheckpoint() for details. */ u32 iMaxFrame; /* Largest iWalFrame value in aFrame[] */ u32 mLock; int nFrame; /* Entries in aFrame[] array */ | > > | 389 390 391 392 393 394 395 396 397 398 399 400 401 402 403 404 | int nProgress; /* Rows processed for all objects */ RbuObjIter objiter; /* Iterator for skipping through tbl/idx */ const char *zVfsName; /* Name of automatically created rbu vfs */ rbu_file *pTargetFd; /* File handle open on target db */ int nPagePerSector; /* Pages per sector for pTargetFd */ i64 iOalSz; i64 nPhaseOneStep; void *pRenameArg; int (*xRename)(void*, const char*, const char*); /* The following state variables are used as part of the incremental ** checkpoint stage (eStage==RBU_STAGE_CKPT). See comments surrounding ** function rbuSetupCheckpoint() for details. */ u32 iMaxFrame; /* Largest iWalFrame value in aFrame[] */ u32 mLock; int nFrame; /* Entries in aFrame[] array */ |
︙ | ︙ | |||
680 681 682 683 684 685 686 687 688 689 690 691 692 693 | aOut = sqlite3_malloc(nOut+1); if( aOut==0 ){ sqlite3_result_error_nomem(context); }else{ nOut2 = rbuDeltaApply(aOrig, nOrig, aDelta, nDelta, aOut); if( nOut2!=nOut ){ sqlite3_result_error(context, "corrupt fossil delta", -1); }else{ sqlite3_result_blob(context, aOut, nOut, sqlite3_free); } } } | > | 703 704 705 706 707 708 709 710 711 712 713 714 715 716 717 | aOut = sqlite3_malloc(nOut+1); if( aOut==0 ){ sqlite3_result_error_nomem(context); }else{ nOut2 = rbuDeltaApply(aOrig, nOrig, aDelta, nDelta, aOut); if( nOut2!=nOut ){ sqlite3_free(aOut); sqlite3_result_error(context, "corrupt fossil delta", -1); }else{ sqlite3_result_blob(context, aOut, nOut, sqlite3_free); } } } |
︙ | ︙ | |||
799 800 801 802 803 804 805 806 807 808 809 810 811 812 813 814 815 816 817 818 819 | pUp = pIter->pRbuUpdate; while( pUp ){ RbuUpdateStmt *pTmp = pUp->pNext; sqlite3_finalize(pUp->pUpdate); sqlite3_free(pUp); pUp = pTmp; } pIter->pSelect = 0; pIter->pInsert = 0; pIter->pDelete = 0; pIter->pRbuUpdate = 0; pIter->pTmpInsert = 0; pIter->nCol = 0; } /* ** Clean up any resources allocated as part of the iterator object passed ** as the only argument. */ static void rbuObjIterFinalize(RbuObjIter *pIter){ | > > > > > | 823 824 825 826 827 828 829 830 831 832 833 834 835 836 837 838 839 840 841 842 843 844 845 846 847 848 | pUp = pIter->pRbuUpdate; while( pUp ){ RbuUpdateStmt *pTmp = pUp->pNext; sqlite3_finalize(pUp->pUpdate); sqlite3_free(pUp); pUp = pTmp; } sqlite3_free(pIter->aIdxCol); sqlite3_free(pIter->zIdxSql); pIter->pSelect = 0; pIter->pInsert = 0; pIter->pDelete = 0; pIter->pRbuUpdate = 0; pIter->pTmpInsert = 0; pIter->nCol = 0; pIter->nIdxCol = 0; pIter->aIdxCol = 0; pIter->zIdxSql = 0; } /* ** Clean up any resources allocated as part of the iterator object passed ** as the only argument. */ static void rbuObjIterFinalize(RbuObjIter *pIter){ |
︙ | ︙ | |||
920 921 922 923 924 925 926 927 928 929 930 931 932 933 | sqlite3rbu *p = sqlite3_user_data(pCtx); const char *zIn; assert( argc==1 || argc==2 ); zIn = (const char*)sqlite3_value_text(argv[0]); if( zIn ){ if( rbuIsVacuum(p) ){ if( argc==1 || 0==sqlite3_value_int(argv[1]) ){ sqlite3_result_text(pCtx, zIn, -1, SQLITE_STATIC); } }else{ if( strlen(zIn)>4 && memcmp("data", zIn, 4)==0 ){ int i; for(i=4; zIn[i]>='0' && zIn[i]<='9'; i++); | > | 949 950 951 952 953 954 955 956 957 958 959 960 961 962 963 | sqlite3rbu *p = sqlite3_user_data(pCtx); const char *zIn; assert( argc==1 || argc==2 ); zIn = (const char*)sqlite3_value_text(argv[0]); if( zIn ){ if( rbuIsVacuum(p) ){ assert( argc==2 || argc==1 ); if( argc==1 || 0==sqlite3_value_int(argv[1]) ){ sqlite3_result_text(pCtx, zIn, -1, SQLITE_STATIC); } }else{ if( strlen(zIn)>4 && memcmp("data", zIn, 4)==0 ){ int i; for(i=4; zIn[i]>='0' && zIn[i]<='9'; i++); |
︙ | ︙ | |||
950 951 952 953 954 955 956 | static int rbuObjIterFirst(sqlite3rbu *p, RbuObjIter *pIter){ int rc; memset(pIter, 0, sizeof(RbuObjIter)); rc = prepareFreeAndCollectError(p->dbRbu, &pIter->pTblIter, &p->zErrmsg, sqlite3_mprintf( "SELECT rbu_target_name(name, type='view') AS target, name " | | | | 980 981 982 983 984 985 986 987 988 989 990 991 992 993 994 995 996 997 998 999 1000 1001 1002 1003 | static int rbuObjIterFirst(sqlite3rbu *p, RbuObjIter *pIter){ int rc; memset(pIter, 0, sizeof(RbuObjIter)); rc = prepareFreeAndCollectError(p->dbRbu, &pIter->pTblIter, &p->zErrmsg, sqlite3_mprintf( "SELECT rbu_target_name(name, type='view') AS target, name " "FROM sqlite_schema " "WHERE type IN ('table', 'view') AND target IS NOT NULL " " %s " "ORDER BY name" , rbuIsVacuum(p) ? "AND rootpage!=0 AND rootpage IS NOT NULL" : "")); if( rc==SQLITE_OK ){ rc = prepareAndCollectError(p->dbMain, &pIter->pIdxIter, &p->zErrmsg, "SELECT name, rootpage, sql IS NULL OR substr(8, 6)=='UNIQUE' " " FROM main.sqlite_schema " " WHERE type='index' AND tbl_name = ?" ); } pIter->bCleanup = 1; p->rc = rc; return rbuObjIterNext(p, pIter); |
︙ | ︙ | |||
1030 1031 1032 1033 1034 1035 1036 | ** ** If an error (i.e. an OOM condition) occurs, return NULL and leave an ** error code in the rbu handle passed as the first argument. Or, if an ** error has already occurred when this function is called, return NULL ** immediately without attempting the allocation or modifying the stored ** error code. */ | | | | 1060 1061 1062 1063 1064 1065 1066 1067 1068 1069 1070 1071 1072 1073 1074 1075 1076 1077 1078 1079 1080 1081 1082 1083 1084 1085 1086 1087 1088 1089 1090 1091 1092 1093 1094 1095 | ** ** If an error (i.e. an OOM condition) occurs, return NULL and leave an ** error code in the rbu handle passed as the first argument. Or, if an ** error has already occurred when this function is called, return NULL ** immediately without attempting the allocation or modifying the stored ** error code. */ static void *rbuMalloc(sqlite3rbu *p, sqlite3_int64 nByte){ void *pRet = 0; if( p->rc==SQLITE_OK ){ assert( nByte>0 ); pRet = sqlite3_malloc64(nByte); if( pRet==0 ){ p->rc = SQLITE_NOMEM; }else{ memset(pRet, 0, nByte); } } return pRet; } /* ** Allocate and zero the pIter->azTblCol[] and abTblPk[] arrays so that ** there is room for at least nCol elements. If an OOM occurs, store an ** error code in the RBU handle passed as the first argument. */ static void rbuAllocateIterArrays(sqlite3rbu *p, RbuObjIter *pIter, int nCol){ sqlite3_int64 nByte = (2*sizeof(char*) + sizeof(int) + 3*sizeof(u8)) * nCol; char **azNew; azNew = (char**)rbuMalloc(p, nByte); if( azNew ){ pIter->azTblCol = azNew; pIter->azTblType = &azNew[nCol]; pIter->aiSrcOrder = (int*)&pIter->azTblType[nCol]; |
︙ | ︙ | |||
1078 1079 1080 1081 1082 1083 1084 | ** If an OOM condition is encountered when attempting to allocate memory, ** output variable (*pRc) is set to SQLITE_NOMEM before returning. Otherwise, ** if the allocation succeeds, (*pRc) is left unchanged. */ static char *rbuStrndup(const char *zStr, int *pRc){ char *zRet = 0; | | | | | | | | | > | 1108 1109 1110 1111 1112 1113 1114 1115 1116 1117 1118 1119 1120 1121 1122 1123 1124 1125 1126 1127 1128 1129 1130 | ** If an OOM condition is encountered when attempting to allocate memory, ** output variable (*pRc) is set to SQLITE_NOMEM before returning. Otherwise, ** if the allocation succeeds, (*pRc) is left unchanged. */ static char *rbuStrndup(const char *zStr, int *pRc){ char *zRet = 0; if( *pRc==SQLITE_OK ){ if( zStr ){ size_t nCopy = strlen(zStr) + 1; zRet = (char*)sqlite3_malloc64(nCopy); if( zRet ){ memcpy(zRet, zStr, nCopy); }else{ *pRc = SQLITE_NOMEM; } } } return zRet; } /* |
︙ | ︙ | |||
1130 1131 1132 1133 1134 1135 1136 | ** (i.e. unless *peType is set to 3), then *piPk is set to zero. Or, ** if the table does have an external primary key index, then *piPk ** is set to the root page number of the primary key index before ** returning. ** ** ALGORITHM: ** | | | | | | > > | | 1161 1162 1163 1164 1165 1166 1167 1168 1169 1170 1171 1172 1173 1174 1175 1176 1177 1178 1179 1180 1181 1182 1183 1184 1185 1186 1187 1188 1189 1190 1191 1192 1193 1194 1195 1196 1197 1198 1199 1200 1201 1202 1203 1204 1205 1206 1207 1208 1209 1210 1211 1212 1213 1214 1215 1216 | ** (i.e. unless *peType is set to 3), then *piPk is set to zero. Or, ** if the table does have an external primary key index, then *piPk ** is set to the root page number of the primary key index before ** returning. ** ** ALGORITHM: ** ** if( no entry exists in sqlite_schema ){ ** return RBU_PK_NOTABLE ** }else if( sql for the entry starts with "CREATE VIRTUAL" ){ ** return RBU_PK_VTAB ** }else if( "PRAGMA index_list()" for the table contains a "pk" index ){ ** if( the index that is the pk exists in sqlite_schema ){ ** *piPK = rootpage of that index. ** return RBU_PK_EXTERNAL ** }else{ ** return RBU_PK_WITHOUT_ROWID ** } ** }else if( "PRAGMA table_info()" lists one or more "pk" columns ){ ** return RBU_PK_IPK ** }else{ ** return RBU_PK_NONE ** } */ static void rbuTableType( sqlite3rbu *p, const char *zTab, int *peType, int *piTnum, int *piPk ){ /* ** 0) SELECT count(*) FROM sqlite_schema where name=%Q AND IsVirtual(%Q) ** 1) PRAGMA index_list = ? ** 2) SELECT count(*) FROM sqlite_schema where name=%Q ** 3) PRAGMA table_info = ? */ sqlite3_stmt *aStmt[4] = {0, 0, 0, 0}; *peType = RBU_PK_NOTABLE; *piPk = 0; assert( p->rc==SQLITE_OK ); p->rc = prepareFreeAndCollectError(p->dbMain, &aStmt[0], &p->zErrmsg, sqlite3_mprintf( "SELECT " " (sql COLLATE nocase BETWEEN 'CREATE VIRTUAL' AND 'CREATE VIRTUAM')," " rootpage" " FROM sqlite_schema" " WHERE name=%Q", zTab )); if( p->rc!=SQLITE_OK || sqlite3_step(aStmt[0])!=SQLITE_ROW ){ /* Either an error, or no such table. */ goto rbuTableType_end; } if( sqlite3_column_int(aStmt[0], 0) ){ |
︙ | ︙ | |||
1192 1193 1194 1195 1196 1197 1198 | if( p->rc ) goto rbuTableType_end; while( sqlite3_step(aStmt[1])==SQLITE_ROW ){ const u8 *zOrig = sqlite3_column_text(aStmt[1], 3); const u8 *zIdx = sqlite3_column_text(aStmt[1], 1); if( zOrig && zIdx && zOrig[0]=='p' ){ p->rc = prepareFreeAndCollectError(p->dbMain, &aStmt[2], &p->zErrmsg, sqlite3_mprintf( | | | 1225 1226 1227 1228 1229 1230 1231 1232 1233 1234 1235 1236 1237 1238 1239 | if( p->rc ) goto rbuTableType_end; while( sqlite3_step(aStmt[1])==SQLITE_ROW ){ const u8 *zOrig = sqlite3_column_text(aStmt[1], 3); const u8 *zIdx = sqlite3_column_text(aStmt[1], 1); if( zOrig && zIdx && zOrig[0]=='p' ){ p->rc = prepareFreeAndCollectError(p->dbMain, &aStmt[2], &p->zErrmsg, sqlite3_mprintf( "SELECT rootpage FROM sqlite_schema WHERE name = %Q", zIdx )); if( p->rc==SQLITE_OK ){ if( sqlite3_step(aStmt[2])==SQLITE_ROW ){ *piPk = sqlite3_column_int(aStmt[2], 0); *peType = RBU_PK_EXTERNAL; }else{ *peType = RBU_PK_WITHOUT_ROWID; |
︙ | ︙ | |||
1245 1246 1247 1248 1249 1250 1251 1252 1253 1254 1255 1256 1257 1258 1259 1260 1261 1262 1263 1264 1265 1266 | sqlite3_mprintf("PRAGMA main.index_list = %Q", pIter->zTbl) ); } pIter->nIndex = 0; while( p->rc==SQLITE_OK && SQLITE_ROW==sqlite3_step(pList) ){ const char *zIdx = (const char*)sqlite3_column_text(pList, 1); sqlite3_stmt *pXInfo = 0; if( zIdx==0 ) break; p->rc = prepareFreeAndCollectError(p->dbMain, &pXInfo, &p->zErrmsg, sqlite3_mprintf("PRAGMA main.index_xinfo = %Q", zIdx) ); while( p->rc==SQLITE_OK && SQLITE_ROW==sqlite3_step(pXInfo) ){ int iCid = sqlite3_column_int(pXInfo, 1); if( iCid>=0 ) pIter->abIndexed[iCid] = 1; } rbuFinalize(p, pXInfo); bIndex = 1; pIter->nIndex++; } if( pIter->eType==RBU_PK_WITHOUT_ROWID ){ | > > > > > > > | 1278 1279 1280 1281 1282 1283 1284 1285 1286 1287 1288 1289 1290 1291 1292 1293 1294 1295 1296 1297 1298 1299 1300 1301 1302 1303 1304 1305 1306 | sqlite3_mprintf("PRAGMA main.index_list = %Q", pIter->zTbl) ); } pIter->nIndex = 0; while( p->rc==SQLITE_OK && SQLITE_ROW==sqlite3_step(pList) ){ const char *zIdx = (const char*)sqlite3_column_text(pList, 1); int bPartial = sqlite3_column_int(pList, 4); sqlite3_stmt *pXInfo = 0; if( zIdx==0 ) break; if( bPartial ){ memset(pIter->abIndexed, 0x01, sizeof(u8)*pIter->nTblCol); } p->rc = prepareFreeAndCollectError(p->dbMain, &pXInfo, &p->zErrmsg, sqlite3_mprintf("PRAGMA main.index_xinfo = %Q", zIdx) ); while( p->rc==SQLITE_OK && SQLITE_ROW==sqlite3_step(pXInfo) ){ int iCid = sqlite3_column_int(pXInfo, 1); if( iCid>=0 ) pIter->abIndexed[iCid] = 1; if( iCid==-2 ){ memset(pIter->abIndexed, 0x01, sizeof(u8)*pIter->nTblCol); } } rbuFinalize(p, pXInfo); bIndex = 1; pIter->nIndex++; } if( pIter->eType==RBU_PK_WITHOUT_ROWID ){ |
︙ | ︙ | |||
1367 1368 1369 1370 1371 1372 1373 | if( i!=iOrder ){ SWAP(int, pIter->aiSrcOrder[i], pIter->aiSrcOrder[iOrder]); SWAP(char*, pIter->azTblCol[i], pIter->azTblCol[iOrder]); } pIter->azTblType[iOrder] = rbuStrndup(zType, &p->rc); | > | | 1407 1408 1409 1410 1411 1412 1413 1414 1415 1416 1417 1418 1419 1420 1421 1422 | if( i!=iOrder ){ SWAP(int, pIter->aiSrcOrder[i], pIter->aiSrcOrder[iOrder]); SWAP(char*, pIter->azTblCol[i], pIter->azTblCol[iOrder]); } pIter->azTblType[iOrder] = rbuStrndup(zType, &p->rc); assert( iPk>=0 ); pIter->abTblPk[iOrder] = (u8)iPk; pIter->abNotNull[iOrder] = (u8)bNotNull || (iPk!=0); iOrder++; } } rbuFinalize(p, pStmt); rbuObjIterCacheIndexedCols(p, pIter); |
︙ | ︙ | |||
1401 1402 1403 1404 1405 1406 1407 1408 1409 1410 1411 1412 1413 1414 | for(i=0; i<pIter->nTblCol; i++){ const char *z = pIter->azTblCol[i]; zList = rbuMPrintf(p, "%z%s\"%w\"", zList, zSep, z); zSep = ", "; } return zList; } /* ** This function is used to create a SELECT list (the list of SQL ** expressions that follows a SELECT keyword) for a SELECT statement ** used to read from an data_xxx or rbu_tmp_xxx table while updating the ** index object currently indicated by the iterator object passed as the ** second argument. A "PRAGMA index_xinfo = <idxname>" statement is used | > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | 1442 1443 1444 1445 1446 1447 1448 1449 1450 1451 1452 1453 1454 1455 1456 1457 1458 1459 1460 1461 1462 1463 1464 1465 1466 1467 1468 1469 1470 1471 1472 1473 1474 1475 1476 1477 1478 1479 1480 1481 1482 1483 1484 1485 1486 1487 1488 1489 1490 1491 1492 1493 1494 1495 1496 1497 1498 1499 1500 1501 1502 1503 1504 1505 1506 1507 1508 1509 1510 1511 1512 1513 1514 1515 1516 1517 1518 1519 1520 1521 1522 1523 1524 1525 1526 1527 1528 1529 1530 1531 1532 1533 1534 1535 1536 1537 1538 1539 1540 1541 1542 1543 1544 1545 1546 1547 1548 1549 1550 1551 1552 1553 1554 1555 1556 1557 1558 1559 1560 1561 1562 1563 1564 1565 1566 1567 1568 1569 1570 1571 1572 1573 1574 1575 1576 1577 1578 1579 1580 1581 1582 1583 1584 1585 1586 1587 1588 1589 1590 1591 1592 1593 1594 1595 1596 1597 1598 1599 1600 1601 1602 1603 1604 1605 1606 1607 1608 1609 1610 1611 1612 1613 1614 1615 1616 1617 1618 1619 1620 1621 1622 1623 1624 1625 1626 1627 1628 1629 1630 1631 1632 1633 1634 1635 1636 1637 1638 1639 1640 1641 1642 1643 1644 1645 1646 1647 1648 1649 1650 1651 1652 1653 1654 1655 1656 1657 1658 1659 1660 1661 1662 1663 1664 | for(i=0; i<pIter->nTblCol; i++){ const char *z = pIter->azTblCol[i]; zList = rbuMPrintf(p, "%z%s\"%w\"", zList, zSep, z); zSep = ", "; } return zList; } /* ** Return a comma separated list of the quoted PRIMARY KEY column names, ** in order, for the current table. Before each column name, add the text ** zPre. After each column name, add the zPost text. Use zSeparator as ** the separator text (usually ", "). */ static char *rbuObjIterGetPkList( sqlite3rbu *p, /* RBU object */ RbuObjIter *pIter, /* Object iterator for column names */ const char *zPre, /* Before each quoted column name */ const char *zSeparator, /* Separator to use between columns */ const char *zPost /* After each quoted column name */ ){ int iPk = 1; char *zRet = 0; const char *zSep = ""; while( 1 ){ int i; for(i=0; i<pIter->nTblCol; i++){ if( (int)pIter->abTblPk[i]==iPk ){ const char *zCol = pIter->azTblCol[i]; zRet = rbuMPrintf(p, "%z%s%s\"%w\"%s", zRet, zSep, zPre, zCol, zPost); zSep = zSeparator; break; } } if( i==pIter->nTblCol ) break; iPk++; } return zRet; } /* ** This function is called as part of restarting an RBU vacuum within ** stage 1 of the process (while the *-oal file is being built) while ** updating a table (not an index). The table may be a rowid table or ** a WITHOUT ROWID table. It queries the target database to find the ** largest key that has already been written to the target table and ** constructs a WHERE clause that can be used to extract the remaining ** rows from the source table. For a rowid table, the WHERE clause ** is of the form: ** ** "WHERE _rowid_ > ?" ** ** and for WITHOUT ROWID tables: ** ** "WHERE (key1, key2) > (?, ?)" ** ** Instead of "?" placeholders, the actual WHERE clauses created by ** this function contain literal SQL values. */ static char *rbuVacuumTableStart( sqlite3rbu *p, /* RBU handle */ RbuObjIter *pIter, /* RBU iterator object */ int bRowid, /* True for a rowid table */ const char *zWrite /* Target table name prefix */ ){ sqlite3_stmt *pMax = 0; char *zRet = 0; if( bRowid ){ p->rc = prepareFreeAndCollectError(p->dbMain, &pMax, &p->zErrmsg, sqlite3_mprintf( "SELECT max(_rowid_) FROM \"%s%w\"", zWrite, pIter->zTbl ) ); if( p->rc==SQLITE_OK && SQLITE_ROW==sqlite3_step(pMax) ){ sqlite3_int64 iMax = sqlite3_column_int64(pMax, 0); zRet = rbuMPrintf(p, " WHERE _rowid_ > %lld ", iMax); } rbuFinalize(p, pMax); }else{ char *zOrder = rbuObjIterGetPkList(p, pIter, "", ", ", " DESC"); char *zSelect = rbuObjIterGetPkList(p, pIter, "quote(", "||','||", ")"); char *zList = rbuObjIterGetPkList(p, pIter, "", ", ", ""); if( p->rc==SQLITE_OK ){ p->rc = prepareFreeAndCollectError(p->dbMain, &pMax, &p->zErrmsg, sqlite3_mprintf( "SELECT %s FROM \"%s%w\" ORDER BY %s LIMIT 1", zSelect, zWrite, pIter->zTbl, zOrder ) ); if( p->rc==SQLITE_OK && SQLITE_ROW==sqlite3_step(pMax) ){ const char *zVal = (const char*)sqlite3_column_text(pMax, 0); zRet = rbuMPrintf(p, " WHERE (%s) > (%s) ", zList, zVal); } rbuFinalize(p, pMax); } sqlite3_free(zOrder); sqlite3_free(zSelect); sqlite3_free(zList); } return zRet; } /* ** This function is called as part of restating an RBU vacuum when the ** current operation is writing content to an index. If possible, it ** queries the target index b-tree for the largest key already written to ** it, then composes and returns an expression that can be used in a WHERE ** clause to select the remaining required rows from the source table. ** It is only possible to return such an expression if: ** ** * The index contains no DESC columns, and ** * The last key written to the index before the operation was ** suspended does not contain any NULL values. ** ** The expression is of the form: ** ** (index-field1, index-field2, ...) > (?, ?, ...) ** ** except that the "?" placeholders are replaced with literal values. ** ** If the expression cannot be created, NULL is returned. In this case, ** the caller has to use an OFFSET clause to extract only the required ** rows from the sourct table, just as it does for an RBU update operation. */ static char *rbuVacuumIndexStart( sqlite3rbu *p, /* RBU handle */ RbuObjIter *pIter /* RBU iterator object */ ){ char *zOrder = 0; char *zLhs = 0; char *zSelect = 0; char *zVector = 0; char *zRet = 0; int bFailed = 0; const char *zSep = ""; int iCol = 0; sqlite3_stmt *pXInfo = 0; p->rc = prepareFreeAndCollectError(p->dbMain, &pXInfo, &p->zErrmsg, sqlite3_mprintf("PRAGMA main.index_xinfo = %Q", pIter->zIdx) ); while( p->rc==SQLITE_OK && SQLITE_ROW==sqlite3_step(pXInfo) ){ int iCid = sqlite3_column_int(pXInfo, 1); const char *zCollate = (const char*)sqlite3_column_text(pXInfo, 4); const char *zCol; if( sqlite3_column_int(pXInfo, 3) ){ bFailed = 1; break; } if( iCid<0 ){ if( pIter->eType==RBU_PK_IPK ){ int i; for(i=0; pIter->abTblPk[i]==0; i++); assert( i<pIter->nTblCol ); zCol = pIter->azTblCol[i]; }else{ zCol = "_rowid_"; } }else{ zCol = pIter->azTblCol[iCid]; } zLhs = rbuMPrintf(p, "%z%s \"%w\" COLLATE %Q", zLhs, zSep, zCol, zCollate ); zOrder = rbuMPrintf(p, "%z%s \"rbu_imp_%d%w\" COLLATE %Q DESC", zOrder, zSep, iCol, zCol, zCollate ); zSelect = rbuMPrintf(p, "%z%s quote(\"rbu_imp_%d%w\")", zSelect, zSep, iCol, zCol ); zSep = ", "; iCol++; } rbuFinalize(p, pXInfo); if( bFailed ) goto index_start_out; if( p->rc==SQLITE_OK ){ sqlite3_stmt *pSel = 0; p->rc = prepareFreeAndCollectError(p->dbMain, &pSel, &p->zErrmsg, sqlite3_mprintf("SELECT %s FROM \"rbu_imp_%w\" ORDER BY %s LIMIT 1", zSelect, pIter->zTbl, zOrder ) ); if( p->rc==SQLITE_OK && SQLITE_ROW==sqlite3_step(pSel) ){ zSep = ""; for(iCol=0; iCol<pIter->nCol; iCol++){ const char *zQuoted = (const char*)sqlite3_column_text(pSel, iCol); if( zQuoted==0 ){ p->rc = SQLITE_NOMEM; }else if( zQuoted[0]=='N' ){ bFailed = 1; break; } zVector = rbuMPrintf(p, "%z%s%s", zVector, zSep, zQuoted); zSep = ", "; } if( !bFailed ){ zRet = rbuMPrintf(p, "(%s) > (%s)", zLhs, zVector); } } rbuFinalize(p, pSel); } index_start_out: sqlite3_free(zOrder); sqlite3_free(zSelect); sqlite3_free(zVector); sqlite3_free(zLhs); return zRet; } /* ** This function is used to create a SELECT list (the list of SQL ** expressions that follows a SELECT keyword) for a SELECT statement ** used to read from an data_xxx or rbu_tmp_xxx table while updating the ** index object currently indicated by the iterator object passed as the ** second argument. A "PRAGMA index_xinfo = <idxname>" statement is used |
︙ | ︙ | |||
1456 1457 1458 1459 1460 1461 1462 | ); } while( rc==SQLITE_OK && SQLITE_ROW==sqlite3_step(pXInfo) ){ int iCid = sqlite3_column_int(pXInfo, 1); int bDesc = sqlite3_column_int(pXInfo, 3); const char *zCollate = (const char*)sqlite3_column_text(pXInfo, 4); | | > > > > > > > | | | | | | | | | | | | | | | | | | > | | | 1706 1707 1708 1709 1710 1711 1712 1713 1714 1715 1716 1717 1718 1719 1720 1721 1722 1723 1724 1725 1726 1727 1728 1729 1730 1731 1732 1733 1734 1735 1736 1737 1738 1739 1740 1741 1742 1743 1744 1745 1746 1747 1748 1749 1750 | ); } while( rc==SQLITE_OK && SQLITE_ROW==sqlite3_step(pXInfo) ){ int iCid = sqlite3_column_int(pXInfo, 1); int bDesc = sqlite3_column_int(pXInfo, 3); const char *zCollate = (const char*)sqlite3_column_text(pXInfo, 4); const char *zCol = 0; const char *zType; if( iCid==-2 ){ int iSeq = sqlite3_column_int(pXInfo, 0); zRet = sqlite3_mprintf("%z%s(%.*s) COLLATE %Q", zRet, zCom, pIter->aIdxCol[iSeq].nSpan, pIter->aIdxCol[iSeq].zSpan, zCollate ); zType = ""; }else { if( iCid<0 ){ /* An integer primary key. If the table has an explicit IPK, use ** its name. Otherwise, use "rbu_rowid". */ if( pIter->eType==RBU_PK_IPK ){ int i; for(i=0; pIter->abTblPk[i]==0; i++); assert( i<pIter->nTblCol ); zCol = pIter->azTblCol[i]; }else if( rbuIsVacuum(p) ){ zCol = "_rowid_"; }else{ zCol = "rbu_rowid"; } zType = "INTEGER"; }else{ zCol = pIter->azTblCol[iCid]; zType = pIter->azTblType[iCid]; } zRet = sqlite3_mprintf("%z%s\"%w\" COLLATE %Q", zRet, zCom,zCol,zCollate); } if( pIter->bUnique==0 || sqlite3_column_int(pXInfo, 5) ){ const char *zOrder = (bDesc ? " DESC" : ""); zImpPK = sqlite3_mprintf("%z%s\"rbu_imp_%d%w\"%s", zImpPK, zCom, nBind, zCol, zOrder ); } zImpCols = sqlite3_mprintf("%z%s\"rbu_imp_%d%w\" %s COLLATE %Q", |
︙ | ︙ | |||
1691 1692 1693 1694 1695 1696 1697 | ** string, an error code is left in the rbu handle passed as the first ** argument and NULL is returned. Or, if an error has already occurred ** when this function is called, NULL is returned immediately, without ** attempting the allocation or modifying the stored error code. */ static char *rbuObjIterGetBindlist(sqlite3rbu *p, int nBind){ char *zRet = 0; | | | 1949 1950 1951 1952 1953 1954 1955 1956 1957 1958 1959 1960 1961 1962 1963 | ** string, an error code is left in the rbu handle passed as the first ** argument and NULL is returned. Or, if an error has already occurred ** when this function is called, NULL is returned immediately, without ** attempting the allocation or modifying the stored error code. */ static char *rbuObjIterGetBindlist(sqlite3rbu *p, int nBind){ char *zRet = 0; sqlite3_int64 nByte = 2*(sqlite3_int64)nBind + 1; zRet = (char*)rbuMalloc(p, nByte); if( zRet ){ int i; for(i=0; i<nBind; i++){ zRet[i*2] = '?'; zRet[i*2+1] = (i+1==nBind) ? '\0' : ','; |
︙ | ︙ | |||
1789 1790 1791 1792 1793 1794 1795 | char *zCols = 0; /* Used to build up list of table cols */ char *zPk = 0; /* Used to build up table PK declaration */ /* Figure out the name of the primary key index for the current table. ** This is needed for the argument to "PRAGMA index_xinfo". Set ** zIdx to point to a nul-terminated string containing this name. */ p->rc = prepareAndCollectError(p->dbMain, &pQuery, &p->zErrmsg, | | | 2047 2048 2049 2050 2051 2052 2053 2054 2055 2056 2057 2058 2059 2060 2061 | char *zCols = 0; /* Used to build up list of table cols */ char *zPk = 0; /* Used to build up table PK declaration */ /* Figure out the name of the primary key index for the current table. ** This is needed for the argument to "PRAGMA index_xinfo". Set ** zIdx to point to a nul-terminated string containing this name. */ p->rc = prepareAndCollectError(p->dbMain, &pQuery, &p->zErrmsg, "SELECT name FROM sqlite_schema WHERE rootpage = ?" ); if( p->rc==SQLITE_OK ){ sqlite3_bind_int(pQuery, 1, tnum); if( SQLITE_ROW==sqlite3_step(pQuery) ){ zIdx = (const char*)sqlite3_column_text(pQuery, 0); } } |
︙ | ︙ | |||
1952 1953 1954 1955 1956 1957 1958 1959 1960 1961 1962 1963 1964 1965 | rc = sqlite3_reset(p->objiter.pTmpInsert); } if( rc!=SQLITE_OK ){ sqlite3_result_error_code(pCtx, rc); } } /* ** Ensure that the SQLite statement handles required to update the ** target database object currently indicated by the iterator passed ** as the second argument are available. */ static int rbuObjIterPrepareAll( | > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | 2210 2211 2212 2213 2214 2215 2216 2217 2218 2219 2220 2221 2222 2223 2224 2225 2226 2227 2228 2229 2230 2231 2232 2233 2234 2235 2236 2237 2238 2239 2240 2241 2242 2243 2244 2245 2246 2247 2248 2249 2250 2251 2252 2253 2254 2255 2256 2257 2258 2259 2260 2261 2262 2263 2264 2265 2266 2267 2268 2269 2270 2271 2272 2273 2274 2275 2276 2277 2278 2279 2280 2281 2282 2283 2284 2285 2286 2287 2288 2289 2290 2291 2292 2293 2294 2295 2296 2297 2298 2299 2300 2301 2302 2303 2304 2305 2306 2307 2308 2309 2310 2311 2312 2313 2314 2315 2316 2317 2318 | rc = sqlite3_reset(p->objiter.pTmpInsert); } if( rc!=SQLITE_OK ){ sqlite3_result_error_code(pCtx, rc); } } static char *rbuObjIterGetIndexWhere(sqlite3rbu *p, RbuObjIter *pIter){ sqlite3_stmt *pStmt = 0; int rc = p->rc; char *zRet = 0; assert( pIter->zIdxSql==0 && pIter->nIdxCol==0 && pIter->aIdxCol==0 ); if( rc==SQLITE_OK ){ rc = prepareAndCollectError(p->dbMain, &pStmt, &p->zErrmsg, "SELECT trim(sql) FROM sqlite_schema WHERE type='index' AND name=?" ); } if( rc==SQLITE_OK ){ int rc2; rc = sqlite3_bind_text(pStmt, 1, pIter->zIdx, -1, SQLITE_STATIC); if( rc==SQLITE_OK && SQLITE_ROW==sqlite3_step(pStmt) ){ char *zSql = (char*)sqlite3_column_text(pStmt, 0); if( zSql ){ pIter->zIdxSql = zSql = rbuStrndup(zSql, &rc); } if( zSql ){ int nParen = 0; /* Number of open parenthesis */ int i; int iIdxCol = 0; int nIdxAlloc = 0; for(i=0; zSql[i]; i++){ char c = zSql[i]; /* If necessary, grow the pIter->aIdxCol[] array */ if( iIdxCol==nIdxAlloc ){ RbuSpan *aIdxCol = (RbuSpan*)sqlite3_realloc( pIter->aIdxCol, (nIdxAlloc+16)*sizeof(RbuSpan) ); if( aIdxCol==0 ){ rc = SQLITE_NOMEM; break; } pIter->aIdxCol = aIdxCol; nIdxAlloc += 16; } if( c=='(' ){ if( nParen==0 ){ assert( iIdxCol==0 ); pIter->aIdxCol[0].zSpan = &zSql[i+1]; } nParen++; } else if( c==')' ){ nParen--; if( nParen==0 ){ int nSpan = &zSql[i] - pIter->aIdxCol[iIdxCol].zSpan; pIter->aIdxCol[iIdxCol++].nSpan = nSpan; i++; break; } }else if( c==',' && nParen==1 ){ int nSpan = &zSql[i] - pIter->aIdxCol[iIdxCol].zSpan; pIter->aIdxCol[iIdxCol++].nSpan = nSpan; pIter->aIdxCol[iIdxCol].zSpan = &zSql[i+1]; }else if( c=='"' || c=='\'' || c=='`' ){ for(i++; 1; i++){ if( zSql[i]==c ){ if( zSql[i+1]!=c ) break; i++; } } }else if( c=='[' ){ for(i++; 1; i++){ if( zSql[i]==']' ) break; } }else if( c=='-' && zSql[i+1]=='-' ){ for(i=i+2; zSql[i] && zSql[i]!='\n'; i++); if( zSql[i]=='\0' ) break; }else if( c=='/' && zSql[i+1]=='*' ){ for(i=i+2; zSql[i] && (zSql[i]!='*' || zSql[i+1]!='/'); i++); if( zSql[i]=='\0' ) break; i++; } } if( zSql[i] ){ zRet = rbuStrndup(&zSql[i], &rc); } pIter->nIdxCol = iIdxCol; } } rc2 = sqlite3_finalize(pStmt); if( rc==SQLITE_OK ) rc = rc2; } p->rc = rc; return zRet; } /* ** Ensure that the SQLite statement handles required to update the ** target database object currently indicated by the iterator passed ** as the second argument are available. */ static int rbuObjIterPrepareAll( |
︙ | ︙ | |||
1982 1983 1984 1985 1986 1987 1988 1989 1990 1991 1992 1993 1994 1995 1996 1997 1998 | if( zIdx ){ const char *zTbl = pIter->zTbl; char *zImposterCols = 0; /* Columns for imposter table */ char *zImposterPK = 0; /* Primary key declaration for imposter */ char *zWhere = 0; /* WHERE clause on PK columns */ char *zBind = 0; int nBind = 0; assert( pIter->eType!=RBU_PK_VTAB ); zCollist = rbuObjIterGetIndexCols( p, pIter, &zImposterCols, &zImposterPK, &zWhere, &nBind ); zBind = rbuObjIterGetBindlist(p, nBind); /* Create the imposter table used to write to this index. */ sqlite3_test_control(SQLITE_TESTCTRL_IMPOSTER, p->dbMain, "main", 0, 1); | > > | 2335 2336 2337 2338 2339 2340 2341 2342 2343 2344 2345 2346 2347 2348 2349 2350 2351 2352 2353 | if( zIdx ){ const char *zTbl = pIter->zTbl; char *zImposterCols = 0; /* Columns for imposter table */ char *zImposterPK = 0; /* Primary key declaration for imposter */ char *zWhere = 0; /* WHERE clause on PK columns */ char *zBind = 0; char *zPart = 0; int nBind = 0; assert( pIter->eType!=RBU_PK_VTAB ); zPart = rbuObjIterGetIndexWhere(p, pIter); zCollist = rbuObjIterGetIndexCols( p, pIter, &zImposterCols, &zImposterPK, &zWhere, &nBind ); zBind = rbuObjIterGetBindlist(p, nBind); /* Create the imposter table used to write to this index. */ sqlite3_test_control(SQLITE_TESTCTRL_IMPOSTER, p->dbMain, "main", 0, 1); |
︙ | ︙ | |||
2020 2021 2022 2023 2024 2025 2026 2027 | ); } /* Create the SELECT statement to read keys in sorted order */ if( p->rc==SQLITE_OK ){ char *zSql; if( rbuIsVacuum(p) ){ zSql = sqlite3_mprintf( | > > > > > > > > > | > > > | | | | | > > > | > > > > | 2375 2376 2377 2378 2379 2380 2381 2382 2383 2384 2385 2386 2387 2388 2389 2390 2391 2392 2393 2394 2395 2396 2397 2398 2399 2400 2401 2402 2403 2404 2405 2406 2407 2408 2409 2410 2411 2412 2413 2414 2415 2416 2417 2418 2419 2420 2421 2422 2423 2424 2425 2426 2427 2428 2429 2430 2431 2432 2433 2434 2435 2436 2437 2438 2439 2440 | ); } /* Create the SELECT statement to read keys in sorted order */ if( p->rc==SQLITE_OK ){ char *zSql; if( rbuIsVacuum(p) ){ char *zStart = 0; if( nOffset ){ zStart = rbuVacuumIndexStart(p, pIter); if( zStart ){ sqlite3_free(zLimit); zLimit = 0; } } zSql = sqlite3_mprintf( "SELECT %s, 0 AS rbu_control FROM '%q' %s %s %s ORDER BY %s%s", zCollist, pIter->zDataTbl, zPart, (zStart ? (zPart ? "AND" : "WHERE") : ""), zStart, zCollist, zLimit ); sqlite3_free(zStart); }else if( pIter->eType==RBU_PK_EXTERNAL || pIter->eType==RBU_PK_NONE ){ zSql = sqlite3_mprintf( "SELECT %s, rbu_control FROM %s.'rbu_tmp_%q' %s ORDER BY %s%s", zCollist, p->zStateDb, pIter->zDataTbl, zPart, zCollist, zLimit ); }else{ zSql = sqlite3_mprintf( "SELECT %s, rbu_control FROM %s.'rbu_tmp_%q' %s " "UNION ALL " "SELECT %s, rbu_control FROM '%q' " "%s %s typeof(rbu_control)='integer' AND rbu_control!=1 " "ORDER BY %s%s", zCollist, p->zStateDb, pIter->zDataTbl, zPart, zCollist, pIter->zDataTbl, zPart, (zPart ? "AND" : "WHERE"), zCollist, zLimit ); } if( p->rc==SQLITE_OK ){ p->rc = prepareFreeAndCollectError(p->dbRbu,&pIter->pSelect,pz,zSql); }else{ sqlite3_free(zSql); } } sqlite3_free(zImposterCols); sqlite3_free(zImposterPK); sqlite3_free(zWhere); sqlite3_free(zBind); sqlite3_free(zPart); }else{ int bRbuRowid = (pIter->eType==RBU_PK_VTAB) ||(pIter->eType==RBU_PK_NONE) ||(pIter->eType==RBU_PK_EXTERNAL && rbuIsVacuum(p)); const char *zTbl = pIter->zTbl; /* Table this step applies to */ const char *zWrite; /* Imposter table name */ |
︙ | ︙ | |||
2145 2146 2147 2148 2149 2150 2151 2152 2153 2154 | rbuObjIterPrepareTmpInsert(p, pIter, zCollist, zRbuRowid); } /* Create the SELECT statement to read keys from data_xxx */ if( p->rc==SQLITE_OK ){ const char *zRbuRowid = ""; if( bRbuRowid ){ zRbuRowid = rbuIsVacuum(p) ? ",_rowid_ " : ",rbu_rowid"; } | > > > > > > > > > > > > > > > > > > > | | | | | | | > > | | > > > | 2519 2520 2521 2522 2523 2524 2525 2526 2527 2528 2529 2530 2531 2532 2533 2534 2535 2536 2537 2538 2539 2540 2541 2542 2543 2544 2545 2546 2547 2548 2549 2550 2551 2552 2553 2554 2555 2556 2557 2558 2559 2560 2561 2562 2563 2564 2565 2566 2567 2568 | rbuObjIterPrepareTmpInsert(p, pIter, zCollist, zRbuRowid); } /* Create the SELECT statement to read keys from data_xxx */ if( p->rc==SQLITE_OK ){ const char *zRbuRowid = ""; char *zStart = 0; char *zOrder = 0; if( bRbuRowid ){ zRbuRowid = rbuIsVacuum(p) ? ",_rowid_ " : ",rbu_rowid"; } if( rbuIsVacuum(p) ){ if( nOffset ){ zStart = rbuVacuumTableStart(p, pIter, bRbuRowid, zWrite); if( zStart ){ sqlite3_free(zLimit); zLimit = 0; } } if( bRbuRowid ){ zOrder = rbuMPrintf(p, "_rowid_"); }else{ zOrder = rbuObjIterGetPkList(p, pIter, "", ", ", ""); } } if( p->rc==SQLITE_OK ){ p->rc = prepareFreeAndCollectError(p->dbRbu, &pIter->pSelect, pz, sqlite3_mprintf( "SELECT %s,%s rbu_control%s FROM '%q'%s %s %s %s", zCollist, (rbuIsVacuum(p) ? "0 AS " : ""), zRbuRowid, pIter->zDataTbl, (zStart ? zStart : ""), (zOrder ? "ORDER BY" : ""), zOrder, zLimit ) ); } sqlite3_free(zStart); sqlite3_free(zOrder); } sqlite3_free(zWhere); sqlite3_free(zOldlist); sqlite3_free(zNewlist); sqlite3_free(zBindings); } |
︙ | ︙ | |||
2336 2337 2338 2339 2340 2341 2342 | break; case RBU_STATE_COOKIE: pRet->iCookie = (u32)sqlite3_column_int64(pStmt, 1); break; case RBU_STATE_OALSZ: | | | 2734 2735 2736 2737 2738 2739 2740 2741 2742 2743 2744 2745 2746 2747 2748 | break; case RBU_STATE_COOKIE: pRet->iCookie = (u32)sqlite3_column_int64(pStmt, 1); break; case RBU_STATE_OALSZ: pRet->iOalSz = sqlite3_column_int64(pStmt, 1); break; case RBU_STATE_PHASEONESTEP: pRet->nPhaseOneStep = sqlite3_column_int64(pStmt, 1); break; case RBU_STATE_DATATBL: |
︙ | ︙ | |||
2363 2364 2365 2366 2367 2368 2369 2370 | return pRet; } /* ** Open the database handle and attach the RBU database as "rbu". If an ** error occurs, leave an error code and message in the RBU handle. */ | > > > > | > > | | 2761 2762 2763 2764 2765 2766 2767 2768 2769 2770 2771 2772 2773 2774 2775 2776 2777 2778 2779 2780 2781 2782 2783 2784 2785 2786 2787 2788 2789 2790 2791 2792 2793 | return pRet; } /* ** Open the database handle and attach the RBU database as "rbu". If an ** error occurs, leave an error code and message in the RBU handle. ** ** If argument dbMain is not NULL, then it is a database handle already ** open on the target database. Use this handle instead of opening a new ** one. */ static void rbuOpenDatabase(sqlite3rbu *p, sqlite3 *dbMain, int *pbRetry){ assert( p->rc || (p->dbMain==0 && p->dbRbu==0) ); assert( p->rc || rbuIsVacuum(p) || p->zTarget!=0 ); assert( dbMain==0 || rbuIsVacuum(p)==0 ); /* Open the RBU database */ p->dbRbu = rbuOpenDbhandle(p, p->zRbu, 1); p->dbMain = dbMain; if( p->rc==SQLITE_OK && rbuIsVacuum(p) ){ sqlite3_file_control(p->dbRbu, "main", SQLITE_FCNTL_RBUCNT, (void*)p); if( p->zState==0 ){ const char *zFile = sqlite3_db_filename(p->dbRbu, "main"); p->zState = rbuMPrintf(p, "file:///%s-vacuum?modeof=%s", zFile, zFile); } } /* If using separate RBU and state databases, attach the state database to ** the RBU db handle now. */ if( p->zState ){ rbuMPrintfExec(p, p->dbRbu, "ATTACH %Q AS stat", p->zState); |
︙ | ︙ | |||
2404 2405 2406 2407 2408 2409 2410 | #if 0 if( rbuIsVacuum(p) ){ if( p->rc==SQLITE_OK ){ int rc2; int bOk = 0; sqlite3_stmt *pCnt = 0; p->rc = prepareAndCollectError(p->dbRbu, &pCnt, &p->zErrmsg, | | | 2808 2809 2810 2811 2812 2813 2814 2815 2816 2817 2818 2819 2820 2821 2822 | #if 0 if( rbuIsVacuum(p) ){ if( p->rc==SQLITE_OK ){ int rc2; int bOk = 0; sqlite3_stmt *pCnt = 0; p->rc = prepareAndCollectError(p->dbRbu, &pCnt, &p->zErrmsg, "SELECT count(*) FROM stat.sqlite_schema" ); if( p->rc==SQLITE_OK && sqlite3_step(pCnt)==SQLITE_ROW && 1==sqlite3_column_int(pCnt, 0) ){ bOk = 1; } |
︙ | ︙ | |||
2508 2509 2510 2511 2512 2513 2514 | "rbu_target_name", -1, SQLITE_UTF8, (void*)p, rbuTargetNameFunc, 0, 0 ); } if( p->rc==SQLITE_OK ){ p->rc = sqlite3_file_control(p->dbMain, "main", SQLITE_FCNTL_RBU, (void*)p); } | | | 2912 2913 2914 2915 2916 2917 2918 2919 2920 2921 2922 2923 2924 2925 2926 | "rbu_target_name", -1, SQLITE_UTF8, (void*)p, rbuTargetNameFunc, 0, 0 ); } if( p->rc==SQLITE_OK ){ p->rc = sqlite3_file_control(p->dbMain, "main", SQLITE_FCNTL_RBU, (void*)p); } rbuMPrintfExec(p, p->dbMain, "SELECT * FROM sqlite_schema"); /* Mark the database file just opened as an RBU target database. If ** this call returns SQLITE_NOTFOUND, then the RBU vfs is not in use. ** This is an error. */ if( p->rc==SQLITE_OK ){ p->rc = sqlite3_file_control(p->dbMain, "main", SQLITE_FCNTL_RBU, (void*)p); } |
︙ | ︙ | |||
2601 2602 2603 2604 2605 2606 2607 | /* If pState is NULL, then the wal file may not have been opened and ** recovered. Running a read-statement here to ensure that doing so ** does not interfere with the "capture" process below. */ if( pState==0 ){ p->eStage = 0; if( p->rc==SQLITE_OK ){ | | | 3005 3006 3007 3008 3009 3010 3011 3012 3013 3014 3015 3016 3017 3018 3019 | /* If pState is NULL, then the wal file may not have been opened and ** recovered. Running a read-statement here to ensure that doing so ** does not interfere with the "capture" process below. */ if( pState==0 ){ p->eStage = 0; if( p->rc==SQLITE_OK ){ p->rc = sqlite3_exec(p->dbMain, "SELECT * FROM sqlite_schema", 0, 0, 0); } } /* Assuming no error has occurred, run a "restart" checkpoint with the ** sqlite3rbu.eStage variable set to CAPTURE. This turns on the following ** special behaviour in the rbu VFS: ** |
︙ | ︙ | |||
2623 2624 2625 2626 2627 2628 2629 | ** array. ** ** * Calls to xShmLock(UNLOCK) to release the exclusive shm WRITER, ** READ0 and CHECKPOINT locks taken as part of the checkpoint are ** no-ops. These locks will not be released until the connection ** is closed. ** | | | | | 3027 3028 3029 3030 3031 3032 3033 3034 3035 3036 3037 3038 3039 3040 3041 3042 3043 3044 3045 3046 3047 3048 3049 3050 3051 3052 3053 3054 3055 | ** array. ** ** * Calls to xShmLock(UNLOCK) to release the exclusive shm WRITER, ** READ0 and CHECKPOINT locks taken as part of the checkpoint are ** no-ops. These locks will not be released until the connection ** is closed. ** ** * Attempting to xSync() the database file causes an SQLITE_NOTICE ** error. ** ** As a result, unless an error (i.e. OOM or SQLITE_BUSY) occurs, the ** checkpoint below fails with SQLITE_NOTICE, and leaves the aFrame[] ** array populated with a set of (frame -> page) mappings. Because the ** WRITER, CHECKPOINT and READ0 locks are still held, it is safe to copy ** data from the wal file into the database file according to the ** contents of aFrame[]. */ if( p->rc==SQLITE_OK ){ int rc2; p->eStage = RBU_STAGE_CAPTURE; rc2 = sqlite3_exec(p->dbMain, "PRAGMA main.wal_checkpoint=restart", 0, 0,0); if( rc2!=SQLITE_NOTICE ) p->rc = rc2; } if( p->rc==SQLITE_OK && p->nFrame>0 ){ p->eStage = RBU_STAGE_CKPT; p->nStep = (pState ? pState->nRow : 0); p->aBuf = rbuMalloc(p, p->pgsz); p->iWalCksum = rbuShmChecksum(p); |
︙ | ︙ | |||
2683 2684 2685 2686 2687 2688 2689 | */ static int rbuCaptureWalRead(sqlite3rbu *pRbu, i64 iOff, int iAmt){ const u32 mReq = (1<<WAL_LOCK_WRITE)|(1<<WAL_LOCK_CKPT)|(1<<WAL_LOCK_READ0); u32 iFrame; if( pRbu->mLock!=mReq ){ pRbu->rc = SQLITE_BUSY; | | | 3087 3088 3089 3090 3091 3092 3093 3094 3095 3096 3097 3098 3099 3100 3101 | */ static int rbuCaptureWalRead(sqlite3rbu *pRbu, i64 iOff, int iAmt){ const u32 mReq = (1<<WAL_LOCK_WRITE)|(1<<WAL_LOCK_CKPT)|(1<<WAL_LOCK_READ0); u32 iFrame; if( pRbu->mLock!=mReq ){ pRbu->rc = SQLITE_BUSY; return SQLITE_NOTICE_RBU; } pRbu->pgsz = iAmt; if( pRbu->nFrame==pRbu->nFrameAlloc ){ int nNew = (pRbu->nFrameAlloc ? pRbu->nFrameAlloc : 64) * 2; RbuFrame *aNew; aNew = (RbuFrame*)sqlite3_realloc64(pRbu->aFrame, nNew * sizeof(RbuFrame)); |
︙ | ︙ | |||
2735 2736 2737 2738 2739 2740 2741 | iOff = (i64)(pFrame->iDbPage-1) * p->pgsz; p->rc = pDb->pMethods->xWrite(pDb, p->aBuf, p->pgsz, iOff); } /* | | > | > | > | > | | | | > > > > > > > > > > > > | 3139 3140 3141 3142 3143 3144 3145 3146 3147 3148 3149 3150 3151 3152 3153 3154 3155 3156 3157 3158 3159 3160 3161 3162 3163 3164 3165 3166 3167 3168 3169 3170 3171 3172 3173 3174 3175 3176 3177 | iOff = (i64)(pFrame->iDbPage-1) * p->pgsz; p->rc = pDb->pMethods->xWrite(pDb, p->aBuf, p->pgsz, iOff); } /* ** Take an EXCLUSIVE lock on the database file. Return SQLITE_OK if ** successful, or an SQLite error code otherwise. */ static int rbuLockDatabase(sqlite3 *db){ int rc = SQLITE_OK; sqlite3_file *fd = 0; sqlite3_file_control(db, "main", SQLITE_FCNTL_FILE_POINTER, &fd); if( fd->pMethods ){ rc = fd->pMethods->xLock(fd, SQLITE_LOCK_SHARED); if( rc==SQLITE_OK ){ rc = fd->pMethods->xLock(fd, SQLITE_LOCK_EXCLUSIVE); } } return rc; } /* ** Return true if the database handle passed as the only argument ** was opened with the rbu_exclusive_checkpoint=1 URI parameter ** specified. Or false otherwise. */ static int rbuExclusiveCheckpoint(sqlite3 *db){ const char *zUri = sqlite3_db_filename(db, 0); return sqlite3_uri_boolean(zUri, RBU_EXCLUSIVE_CHECKPOINT, 0); } #if defined(_WIN32_WCE) static LPWSTR rbuWinUtf8ToUnicode(const char *zFilename){ int nChar; LPWSTR zWideFilename; |
︙ | ︙ | |||
2801 2802 2803 2804 2805 2806 2807 | /* Move the *-oal file to *-wal. At this point connection p->db is ** holding a SHARED lock on the target database file (because it is ** in WAL mode). So no other connection may be writing the db. ** ** In order to ensure that there are no database readers, an EXCLUSIVE ** lock is obtained here before the *-oal is moved to *-wal. */ | | < | | | | | | | | < < < < | < | < < < | < | | < < < | < < | > | | < | > > > > | | > | | | < | 3221 3222 3223 3224 3225 3226 3227 3228 3229 3230 3231 3232 3233 3234 3235 3236 3237 3238 3239 3240 3241 3242 3243 3244 3245 3246 3247 3248 3249 3250 3251 3252 3253 3254 3255 3256 3257 3258 3259 3260 3261 3262 3263 3264 3265 3266 | /* Move the *-oal file to *-wal. At this point connection p->db is ** holding a SHARED lock on the target database file (because it is ** in WAL mode). So no other connection may be writing the db. ** ** In order to ensure that there are no database readers, an EXCLUSIVE ** lock is obtained here before the *-oal is moved to *-wal. */ sqlite3 *dbMain = 0; rbuFileSuffix3(zBase, zWal); rbuFileSuffix3(zBase, zOal); /* Re-open the databases. */ rbuObjIterFinalize(&p->objiter); sqlite3_close(p->dbRbu); sqlite3_close(p->dbMain); p->dbMain = 0; p->dbRbu = 0; dbMain = rbuOpenDbhandle(p, p->zTarget, 1); if( dbMain ){ assert( p->rc==SQLITE_OK ); p->rc = rbuLockDatabase(dbMain); } if( p->rc==SQLITE_OK ){ p->rc = p->xRename(p->pRenameArg, zOal, zWal); } if( p->rc!=SQLITE_OK || rbuIsVacuum(p) || rbuExclusiveCheckpoint(dbMain)==0 ){ sqlite3_close(dbMain); dbMain = 0; } if( p->rc==SQLITE_OK ){ rbuOpenDatabase(p, dbMain, 0); rbuSetupCheckpoint(p, 0); } } sqlite3_free(zWal); sqlite3_free(zOal); } |
︙ | ︙ | |||
3192 3193 3194 3195 3196 3197 3198 | sqlite3_stmt *pSql = 0; sqlite3_stmt *pInsert = 0; assert( rbuIsVacuum(p) ); p->rc = sqlite3_exec(p->dbMain, "PRAGMA writable_schema=1", 0,0, &p->zErrmsg); if( p->rc==SQLITE_OK ){ p->rc = prepareAndCollectError(p->dbRbu, &pSql, &p->zErrmsg, | | | | | 3601 3602 3603 3604 3605 3606 3607 3608 3609 3610 3611 3612 3613 3614 3615 3616 3617 3618 3619 3620 3621 3622 3623 3624 3625 3626 3627 3628 3629 3630 3631 3632 3633 3634 3635 3636 | sqlite3_stmt *pSql = 0; sqlite3_stmt *pInsert = 0; assert( rbuIsVacuum(p) ); p->rc = sqlite3_exec(p->dbMain, "PRAGMA writable_schema=1", 0,0, &p->zErrmsg); if( p->rc==SQLITE_OK ){ p->rc = prepareAndCollectError(p->dbRbu, &pSql, &p->zErrmsg, "SELECT sql FROM sqlite_schema WHERE sql!='' AND rootpage!=0" " AND name!='sqlite_sequence' " " ORDER BY type DESC" ); } while( p->rc==SQLITE_OK && sqlite3_step(pSql)==SQLITE_ROW ){ const char *zSql = (const char*)sqlite3_column_text(pSql, 0); p->rc = sqlite3_exec(p->dbMain, zSql, 0, 0, &p->zErrmsg); } rbuFinalize(p, pSql); if( p->rc!=SQLITE_OK ) return; if( p->rc==SQLITE_OK ){ p->rc = prepareAndCollectError(p->dbRbu, &pSql, &p->zErrmsg, "SELECT * FROM sqlite_schema WHERE rootpage=0 OR rootpage IS NULL" ); } if( p->rc==SQLITE_OK ){ p->rc = prepareAndCollectError(p->dbMain, &pInsert, &p->zErrmsg, "INSERT INTO sqlite_schema VALUES(?,?,?,?,?)" ); } while( p->rc==SQLITE_OK && sqlite3_step(pSql)==SQLITE_ROW ){ int i; for(i=0; i<5; i++){ sqlite3_bind_value(pInsert, i+1, sqlite3_column_value(pSql, i)); |
︙ | ︙ | |||
3417 3418 3419 3420 3421 3422 3423 | ** If there is a "*-oal" file in the file-system corresponding to the ** target database in the file-system, delete it. If an error occurs, ** leave an error code and error message in the rbu handle. */ static void rbuDeleteOalFile(sqlite3rbu *p){ char *zOal = rbuMPrintf(p, "%s-oal", p->zTarget); if( zOal ){ | | > | 3826 3827 3828 3829 3830 3831 3832 3833 3834 3835 3836 3837 3838 3839 3840 3841 | ** If there is a "*-oal" file in the file-system corresponding to the ** target database in the file-system, delete it. If an error occurs, ** leave an error code and error message in the rbu handle. */ static void rbuDeleteOalFile(sqlite3rbu *p){ char *zOal = rbuMPrintf(p, "%s-oal", p->zTarget); if( zOal ){ sqlite3_vfs *pVfs = 0; sqlite3_file_control(p->dbMain, "main", SQLITE_FCNTL_VFS_POINTER, &pVfs); assert( pVfs && p->rc==SQLITE_OK && p->zErrmsg==0 ); pVfs->xDelete(pVfs, zOal, 0); sqlite3_free(zOal); } } /* |
︙ | ︙ | |||
3471 3472 3473 3474 3475 3476 3477 3478 3479 3480 | int nVal, sqlite3_value **apVal ){ sqlite3rbu *p = (sqlite3rbu*)sqlite3_user_data(pCtx); sqlite3_stmt *pStmt = 0; char *zErrmsg = 0; int rc; assert( nVal==1 ); | > | | | | 3881 3882 3883 3884 3885 3886 3887 3888 3889 3890 3891 3892 3893 3894 3895 3896 3897 3898 3899 3900 3901 3902 3903 3904 3905 3906 3907 3908 3909 3910 3911 3912 3913 3914 | int nVal, sqlite3_value **apVal ){ sqlite3rbu *p = (sqlite3rbu*)sqlite3_user_data(pCtx); sqlite3_stmt *pStmt = 0; char *zErrmsg = 0; int rc; sqlite3 *db = (rbuIsVacuum(p) ? p->dbRbu : p->dbMain); assert( nVal==1 ); rc = prepareFreeAndCollectError(db, &pStmt, &zErrmsg, sqlite3_mprintf("SELECT count(*) FROM sqlite_schema " "WHERE type='index' AND tbl_name = %Q", sqlite3_value_text(apVal[0])) ); if( rc!=SQLITE_OK ){ sqlite3_result_error(pCtx, zErrmsg, -1); }else{ int nIndex = 0; if( SQLITE_ROW==sqlite3_step(pStmt) ){ nIndex = sqlite3_column_int(pStmt, 0); } rc = sqlite3_finalize(pStmt); if( rc==SQLITE_OK ){ sqlite3_result_int(pCtx, nIndex); }else{ sqlite3_result_error(pCtx, sqlite3_errmsg(db), -1); } } sqlite3_free(zErrmsg); } /* |
︙ | ︙ | |||
3526 3527 3528 3529 3530 3531 3532 | "rbu_index_cnt", 1, SQLITE_UTF8, (void*)p, rbuIndexCntFunc, 0, 0 ); /* Check for the rbu_count table. If it does not exist, or if an error ** occurs, nPhaseOneStep will be left set to -1. */ if( p->rc==SQLITE_OK ){ p->rc = prepareAndCollectError(p->dbRbu, &pStmt, &p->zErrmsg, | | | 3937 3938 3939 3940 3941 3942 3943 3944 3945 3946 3947 3948 3949 3950 3951 | "rbu_index_cnt", 1, SQLITE_UTF8, (void*)p, rbuIndexCntFunc, 0, 0 ); /* Check for the rbu_count table. If it does not exist, or if an error ** occurs, nPhaseOneStep will be left set to -1. */ if( p->rc==SQLITE_OK ){ p->rc = prepareAndCollectError(p->dbRbu, &pStmt, &p->zErrmsg, "SELECT 1 FROM sqlite_schema WHERE tbl_name = 'rbu_count'" ); } if( p->rc==SQLITE_OK ){ if( SQLITE_ROW==sqlite3_step(pStmt) ){ bExists = 1; } p->rc = sqlite3_finalize(pStmt); |
︙ | ︙ | |||
3568 3569 3570 3571 3572 3573 3574 3575 3576 3577 3578 3579 3580 3581 | p = (sqlite3rbu*)sqlite3_malloc64(nByte); if( p ){ RbuState *pState = 0; /* Create the custom VFS. */ memset(p, 0, sizeof(sqlite3rbu)); rbuCreateVfs(p); /* Open the target, RBU and state databases */ if( p->rc==SQLITE_OK ){ char *pCsr = (char*)&p[1]; int bRetry = 0; if( zTarget ){ | > | 3979 3980 3981 3982 3983 3984 3985 3986 3987 3988 3989 3990 3991 3992 3993 | p = (sqlite3rbu*)sqlite3_malloc64(nByte); if( p ){ RbuState *pState = 0; /* Create the custom VFS. */ memset(p, 0, sizeof(sqlite3rbu)); sqlite3rbu_rename_handler(p, 0, 0); rbuCreateVfs(p); /* Open the target, RBU and state databases */ if( p->rc==SQLITE_OK ){ char *pCsr = (char*)&p[1]; int bRetry = 0; if( zTarget ){ |
︙ | ︙ | |||
3593 3594 3595 3596 3597 3598 3599 | /* If the first attempt to open the database file fails and the bRetry ** flag it set, this means that the db was not opened because it seemed ** to be a wal-mode db. But, this may have happened due to an earlier ** RBU vacuum operation leaving an old wal file in the directory. ** If this is the case, it will have been checkpointed and deleted ** when the handle was closed and a second attempt to open the ** database may succeed. */ | | | | 4005 4006 4007 4008 4009 4010 4011 4012 4013 4014 4015 4016 4017 4018 4019 4020 4021 | /* If the first attempt to open the database file fails and the bRetry ** flag it set, this means that the db was not opened because it seemed ** to be a wal-mode db. But, this may have happened due to an earlier ** RBU vacuum operation leaving an old wal file in the directory. ** If this is the case, it will have been checkpointed and deleted ** when the handle was closed and a second attempt to open the ** database may succeed. */ rbuOpenDatabase(p, 0, &bRetry); if( bRetry ){ rbuOpenDatabase(p, 0, 0); } } if( p->rc==SQLITE_OK ){ pState = rbuLoadState(p); assert( pState || p->rc!=SQLITE_OK ); if( p->rc==SQLITE_OK ){ |
︙ | ︙ | |||
3690 3691 3692 3693 3694 3695 3696 3697 3698 3699 3700 3701 3702 3703 | if( p->rc==SQLITE_OK ){ rbuSetupOal(p, pState); } } }else if( p->eStage==RBU_STAGE_MOVE ){ /* no-op */ }else if( p->eStage==RBU_STAGE_CKPT ){ rbuSetupCheckpoint(p, pState); }else if( p->eStage==RBU_STAGE_DONE ){ p->rc = SQLITE_DONE; }else{ p->rc = SQLITE_CORRUPT; } } | > > > > > > > > | 4102 4103 4104 4105 4106 4107 4108 4109 4110 4111 4112 4113 4114 4115 4116 4117 4118 4119 4120 4121 4122 4123 | if( p->rc==SQLITE_OK ){ rbuSetupOal(p, pState); } } }else if( p->eStage==RBU_STAGE_MOVE ){ /* no-op */ }else if( p->eStage==RBU_STAGE_CKPT ){ if( !rbuIsVacuum(p) && rbuExclusiveCheckpoint(p->dbMain) ){ /* If the rbu_exclusive_checkpoint=1 URI parameter was specified ** and an incremental checkpoint is being resumed, attempt an ** exclusive lock on the db file. If this fails, so be it. */ p->eStage = RBU_STAGE_DONE; rbuLockDatabase(p->dbMain); p->eStage = RBU_STAGE_CKPT; } rbuSetupCheckpoint(p, pState); }else if( p->eStage==RBU_STAGE_DONE ){ p->rc = SQLITE_DONE; }else{ p->rc = SQLITE_CORRUPT; } } |
︙ | ︙ | |||
3727 3728 3729 3730 3731 3732 3733 | */ sqlite3rbu *sqlite3rbu_open( const char *zTarget, const char *zRbu, const char *zState ){ if( zTarget==0 || zRbu==0 ){ return rbuMisuseError(); } | < | 4147 4148 4149 4150 4151 4152 4153 4154 4155 4156 4157 4158 4159 4160 | */ sqlite3rbu *sqlite3rbu_open( const char *zTarget, const char *zRbu, const char *zState ){ if( zTarget==0 || zRbu==0 ){ return rbuMisuseError(); } return openRbuHandle(zTarget, zRbu, zState); } /* ** Open a handle to begin or resume an RBU VACUUM operation. */ sqlite3rbu *sqlite3rbu_vacuum( |
︙ | ︙ | |||
3941 3942 3943 3944 3945 3946 3947 | p->rc = rc; rbuSaveState(p, p->eStage); rc = p->rc; if( p->eStage==RBU_STAGE_OAL ){ assert( rc!=SQLITE_DONE ); if( rc==SQLITE_OK ) rc = sqlite3_exec(p->dbRbu, "COMMIT", 0, 0, 0); | | > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | 4360 4361 4362 4363 4364 4365 4366 4367 4368 4369 4370 4371 4372 4373 4374 4375 4376 4377 4378 4379 4380 4381 4382 4383 4384 4385 4386 4387 4388 4389 4390 4391 4392 4393 4394 4395 4396 4397 4398 4399 4400 4401 4402 4403 4404 4405 4406 4407 4408 4409 4410 4411 4412 4413 4414 4415 4416 4417 4418 4419 4420 4421 4422 4423 4424 4425 4426 4427 4428 4429 4430 4431 | p->rc = rc; rbuSaveState(p, p->eStage); rc = p->rc; if( p->eStage==RBU_STAGE_OAL ){ assert( rc!=SQLITE_DONE ); if( rc==SQLITE_OK ) rc = sqlite3_exec(p->dbRbu, "COMMIT", 0, 0, 0); if( rc==SQLITE_OK ){ const char *zBegin = rbuIsVacuum(p) ? "BEGIN" : "BEGIN IMMEDIATE"; rc = sqlite3_exec(p->dbRbu, zBegin, 0, 0, 0); } if( rc==SQLITE_OK ) rc = sqlite3_exec(p->dbMain, "BEGIN IMMEDIATE", 0, 0,0); } p->rc = rc; return rc; } /* ** Default xRename callback for RBU. */ static int xDefaultRename(void *pArg, const char *zOld, const char *zNew){ int rc = SQLITE_OK; #if defined(_WIN32_WCE) { LPWSTR zWideOld; LPWSTR zWideNew; zWideOld = rbuWinUtf8ToUnicode(zOld); if( zWideOld ){ zWideNew = rbuWinUtf8ToUnicode(zNew); if( zWideNew ){ if( MoveFileW(zWideOld, zWideNew) ){ rc = SQLITE_OK; }else{ rc = SQLITE_IOERR; } sqlite3_free(zWideNew); }else{ rc = SQLITE_IOERR_NOMEM; } sqlite3_free(zWideOld); }else{ rc = SQLITE_IOERR_NOMEM; } } #else rc = rename(zOld, zNew) ? SQLITE_IOERR : SQLITE_OK; #endif return rc; } void sqlite3rbu_rename_handler( sqlite3rbu *pRbu, void *pArg, int (*xRename)(void *pArg, const char *zOld, const char *zNew) ){ if( xRename ){ pRbu->xRename = xRename; pRbu->pRenameArg = pArg; }else{ pRbu->xRename = xDefaultRename; pRbu->pRenameArg = 0; } } /************************************************************************** ** Beginning of RBU VFS shim methods. The VFS shim modifies the behaviour ** of a standard VFS in the following ways: ** ** 1. Whenever the first page of a main database file is read or ** written, the value of the change-counter cookie is stored in |
︙ | ︙ | |||
4005 4006 4007 4008 4009 4010 4011 | ** all xWrite() calls on the target database file perform no IO. ** Instead the frame and page numbers that would be read and written ** are recorded. Additionally, successful attempts to obtain exclusive ** xShmLock() WRITER, CHECKPOINTER and READ0 locks on the target ** database file are recorded. xShmLock() calls to unlock the same ** locks are no-ops (so that once obtained, these locks are never ** relinquished). Finally, calls to xSync() on the target database | | | 4475 4476 4477 4478 4479 4480 4481 4482 4483 4484 4485 4486 4487 4488 4489 | ** all xWrite() calls on the target database file perform no IO. ** Instead the frame and page numbers that would be read and written ** are recorded. Additionally, successful attempts to obtain exclusive ** xShmLock() WRITER, CHECKPOINTER and READ0 locks on the target ** database file are recorded. xShmLock() calls to unlock the same ** locks are no-ops (so that once obtained, these locks are never ** relinquished). Finally, calls to xSync() on the target database ** file fail with SQLITE_NOTICE errors. */ static void rbuUnlockShm(rbu_file *p){ assert( p->openFlags & SQLITE_OPEN_MAIN_DB ); if( p->pRbu ){ int (*xShmLock)(sqlite3_file*,int,int,int) = p->pReal->pMethods->xShmLock; int i; |
︙ | ︙ | |||
4114 4115 4116 4117 4118 4119 4120 4121 4122 | sqlite3_free(p->apShm[i]); } sqlite3_free(p->apShm); p->apShm = 0; sqlite3_free(p->zDel); if( p->openFlags & SQLITE_OPEN_MAIN_DB ){ rbuMainlistRemove(p); rbuUnlockShm(p); | > > | > | 4584 4585 4586 4587 4588 4589 4590 4591 4592 4593 4594 4595 4596 4597 4598 4599 4600 4601 4602 4603 | sqlite3_free(p->apShm[i]); } sqlite3_free(p->apShm); p->apShm = 0; sqlite3_free(p->zDel); if( p->openFlags & SQLITE_OPEN_MAIN_DB ){ const sqlite3_io_methods *pMeth = p->pReal->pMethods; rbuMainlistRemove(p); rbuUnlockShm(p); if( pMeth->iVersion>1 && pMeth->xShmUnmap ){ pMeth->xShmUnmap(p->pReal, 0); } } else if( (p->openFlags & SQLITE_OPEN_DELETEONCLOSE) && p->pRbu ){ rbuUpdateTempSize(p, 0); } assert( p->pMainNext==0 && p->pRbuVfs->pMain!=p ); /* Close the underlying file handle */ |
︙ | ︙ | |||
4284 4285 4286 4287 4288 4289 4290 | /* ** Sync an rbuVfs-file. */ static int rbuVfsSync(sqlite3_file *pFile, int flags){ rbu_file *p = (rbu_file *)pFile; if( p->pRbu && p->pRbu->eStage==RBU_STAGE_CAPTURE ){ if( p->openFlags & SQLITE_OPEN_MAIN_DB ){ | | | 4757 4758 4759 4760 4761 4762 4763 4764 4765 4766 4767 4768 4769 4770 4771 | /* ** Sync an rbuVfs-file. */ static int rbuVfsSync(sqlite3_file *pFile, int flags){ rbu_file *p = (rbu_file *)pFile; if( p->pRbu && p->pRbu->eStage==RBU_STAGE_CAPTURE ){ if( p->openFlags & SQLITE_OPEN_MAIN_DB ){ return SQLITE_NOTICE_RBU; } return SQLITE_OK; } return p->pReal->pMethods->xSync(p->pReal, flags); } /* |
︙ | ︙ | |||
4380 4381 4382 4383 4384 4385 4386 | rc = xControl(p->pReal, SQLITE_FCNTL_ZIPVFS, &dummy); if( rc==SQLITE_OK ){ rc = SQLITE_ERROR; pRbu->zErrmsg = sqlite3_mprintf("rbu/zipvfs setup error"); }else if( rc==SQLITE_NOTFOUND ){ pRbu->pTargetFd = p; p->pRbu = pRbu; | < | < | 4853 4854 4855 4856 4857 4858 4859 4860 4861 4862 4863 4864 4865 4866 4867 | rc = xControl(p->pReal, SQLITE_FCNTL_ZIPVFS, &dummy); if( rc==SQLITE_OK ){ rc = SQLITE_ERROR; pRbu->zErrmsg = sqlite3_mprintf("rbu/zipvfs setup error"); }else if( rc==SQLITE_NOTFOUND ){ pRbu->pTargetFd = p; p->pRbu = pRbu; rbuMainlistAdd(p); if( p->pWalFd ) p->pWalFd->pRbu = pRbu; rc = SQLITE_OK; } } return rc; } else if( op==SQLITE_FCNTL_RBUCNT ){ |
︙ | ︙ | |||
4437 4438 4439 4440 4441 4442 4443 | int rc = SQLITE_OK; #ifdef SQLITE_AMALGAMATION assert( WAL_CKPT_LOCK==1 ); #endif assert( p->openFlags & (SQLITE_OPEN_MAIN_DB|SQLITE_OPEN_TEMP_DB) ); | > > | > > | < < < > > < | < < < | | 4908 4909 4910 4911 4912 4913 4914 4915 4916 4917 4918 4919 4920 4921 4922 4923 4924 4925 4926 4927 4928 4929 4930 4931 4932 4933 4934 4935 4936 4937 4938 4939 | int rc = SQLITE_OK; #ifdef SQLITE_AMALGAMATION assert( WAL_CKPT_LOCK==1 ); #endif assert( p->openFlags & (SQLITE_OPEN_MAIN_DB|SQLITE_OPEN_TEMP_DB) ); if( pRbu && ( pRbu->eStage==RBU_STAGE_OAL || pRbu->eStage==RBU_STAGE_MOVE || pRbu->eStage==RBU_STAGE_DONE )){ /* Prevent SQLite from taking a shm-lock on the target file when it ** is supplying heap memory to the upper layer in place of *-shm ** segments. */ if( ofst==WAL_LOCK_CKPT && n==1 ) rc = SQLITE_BUSY; }else{ int bCapture = 0; if( pRbu && pRbu->eStage==RBU_STAGE_CAPTURE ){ bCapture = 1; } if( bCapture==0 || 0==(flags & SQLITE_SHM_UNLOCK) ){ rc = p->pReal->pMethods->xShmLock(p->pReal, ofst, n, flags); if( bCapture && rc==SQLITE_OK ){ pRbu->mLock |= ((1<<n) - 1) << ofst; } } } return rc; } |
︙ | ︙ | |||
4481 4482 4483 4484 4485 4486 4487 | int rc = SQLITE_OK; int eStage = (p->pRbu ? p->pRbu->eStage : 0); /* If not in RBU_STAGE_OAL, allow this call to pass through. Or, if this ** rbu is in the RBU_STAGE_OAL state, use heap memory for *-shm space ** instead of a file on disk. */ assert( p->openFlags & (SQLITE_OPEN_MAIN_DB|SQLITE_OPEN_TEMP_DB) ); | | < | | > > > > > > | | | | | | | | < | | 4951 4952 4953 4954 4955 4956 4957 4958 4959 4960 4961 4962 4963 4964 4965 4966 4967 4968 4969 4970 4971 4972 4973 4974 4975 4976 4977 4978 4979 4980 4981 4982 | int rc = SQLITE_OK; int eStage = (p->pRbu ? p->pRbu->eStage : 0); /* If not in RBU_STAGE_OAL, allow this call to pass through. Or, if this ** rbu is in the RBU_STAGE_OAL state, use heap memory for *-shm space ** instead of a file on disk. */ assert( p->openFlags & (SQLITE_OPEN_MAIN_DB|SQLITE_OPEN_TEMP_DB) ); if( eStage==RBU_STAGE_OAL ){ sqlite3_int64 nByte = (iRegion+1) * sizeof(char*); char **apNew = (char**)sqlite3_realloc64(p->apShm, nByte); /* This is an RBU connection that uses its own heap memory for the ** pages of the *-shm file. Since no other process can have run ** recovery, the connection must request *-shm pages in order ** from start to finish. */ assert( iRegion==p->nShm ); if( apNew==0 ){ rc = SQLITE_NOMEM; }else{ memset(&apNew[p->nShm], 0, sizeof(char*) * (1 + iRegion - p->nShm)); p->apShm = apNew; p->nShm = iRegion+1; } if( rc==SQLITE_OK ){ char *pNew = (char*)sqlite3_malloc64(szRegion); if( pNew==0 ){ rc = SQLITE_NOMEM; }else{ memset(pNew, 0, szRegion); p->apShm[iRegion] = pNew; } |
︙ | ︙ | |||
4544 4545 4546 4547 4548 4549 4550 | /* Release the checkpointer and writer locks */ rbuUnlockShm(p); rc = p->pReal->pMethods->xShmUnmap(p->pReal, delFlag); } return rc; } | < < < < < < < < < < < < < < < < < < < < < < < < < < < | 5018 5019 5020 5021 5022 5023 5024 5025 5026 5027 5028 5029 5030 5031 | /* Release the checkpointer and writer locks */ rbuUnlockShm(p); rc = p->pReal->pMethods->xShmUnmap(p->pReal, delFlag); } return rc; } /* ** Open an rbu file handle. */ static int rbuVfsOpen( sqlite3_vfs *pVfs, const char *zName, sqlite3_file *pFile, |
︙ | ︙ | |||
4601 4602 4603 4604 4605 4606 4607 4608 4609 4610 4611 4612 4613 4614 4615 4616 4617 4618 4619 4620 4621 4622 4623 4624 4625 | rbuVfsDeviceCharacteristics, /* xDeviceCharacteristics */ rbuVfsShmMap, /* xShmMap */ rbuVfsShmLock, /* xShmLock */ rbuVfsShmBarrier, /* xShmBarrier */ rbuVfsShmUnmap, /* xShmUnmap */ 0, 0 /* xFetch, xUnfetch */ }; rbu_vfs *pRbuVfs = (rbu_vfs*)pVfs; sqlite3_vfs *pRealVfs = pRbuVfs->pRealVfs; rbu_file *pFd = (rbu_file *)pFile; int rc = SQLITE_OK; const char *zOpen = zName; int oflags = flags; memset(pFd, 0, sizeof(rbu_file)); pFd->pReal = (sqlite3_file*)&pFd[1]; pFd->pRbuVfs = pRbuVfs; pFd->openFlags = flags; if( zName ){ if( flags & SQLITE_OPEN_MAIN_DB ){ /* A main database has just been opened. The following block sets ** (pFd->zWal) to point to a buffer owned by SQLite that contains ** the name of the *-wal file this db connection will use. SQLite ** happens to pass a pointer to this buffer when using xAccess() ** or xOpen() to operate on the *-wal file. */ | > > > > > > > > > > > > > > > > > > > | | < < < < | < | < > | < < < | < < < < < < | 5048 5049 5050 5051 5052 5053 5054 5055 5056 5057 5058 5059 5060 5061 5062 5063 5064 5065 5066 5067 5068 5069 5070 5071 5072 5073 5074 5075 5076 5077 5078 5079 5080 5081 5082 5083 5084 5085 5086 5087 5088 5089 5090 5091 5092 5093 5094 5095 5096 5097 5098 5099 5100 5101 5102 5103 5104 5105 5106 5107 5108 5109 5110 5111 5112 | rbuVfsDeviceCharacteristics, /* xDeviceCharacteristics */ rbuVfsShmMap, /* xShmMap */ rbuVfsShmLock, /* xShmLock */ rbuVfsShmBarrier, /* xShmBarrier */ rbuVfsShmUnmap, /* xShmUnmap */ 0, 0 /* xFetch, xUnfetch */ }; static sqlite3_io_methods rbuvfs_io_methods1 = { 1, /* iVersion */ rbuVfsClose, /* xClose */ rbuVfsRead, /* xRead */ rbuVfsWrite, /* xWrite */ rbuVfsTruncate, /* xTruncate */ rbuVfsSync, /* xSync */ rbuVfsFileSize, /* xFileSize */ rbuVfsLock, /* xLock */ rbuVfsUnlock, /* xUnlock */ rbuVfsCheckReservedLock, /* xCheckReservedLock */ rbuVfsFileControl, /* xFileControl */ rbuVfsSectorSize, /* xSectorSize */ rbuVfsDeviceCharacteristics, /* xDeviceCharacteristics */ 0, 0, 0, 0, 0, 0 }; rbu_vfs *pRbuVfs = (rbu_vfs*)pVfs; sqlite3_vfs *pRealVfs = pRbuVfs->pRealVfs; rbu_file *pFd = (rbu_file *)pFile; int rc = SQLITE_OK; const char *zOpen = zName; int oflags = flags; memset(pFd, 0, sizeof(rbu_file)); pFd->pReal = (sqlite3_file*)&pFd[1]; pFd->pRbuVfs = pRbuVfs; pFd->openFlags = flags; if( zName ){ if( flags & SQLITE_OPEN_MAIN_DB ){ /* A main database has just been opened. The following block sets ** (pFd->zWal) to point to a buffer owned by SQLite that contains ** the name of the *-wal file this db connection will use. SQLite ** happens to pass a pointer to this buffer when using xAccess() ** or xOpen() to operate on the *-wal file. */ pFd->zWal = sqlite3_filename_wal(zName); } else if( flags & SQLITE_OPEN_WAL ){ rbu_file *pDb = rbuFindMaindb(pRbuVfs, zName, 0); if( pDb ){ if( pDb->pRbu && pDb->pRbu->eStage==RBU_STAGE_OAL ){ /* This call is to open a *-wal file. Intead, open the *-oal. */ size_t nOpen; if( rbuIsVacuum(pDb->pRbu) ){ zOpen = sqlite3_db_filename(pDb->pRbu->dbRbu, "main"); zOpen = sqlite3_filename_wal(zOpen); } nOpen = strlen(zOpen); ((char*)zOpen)[nOpen-3] = 'o'; pFd->pRbu = pDb->pRbu; } pDb->pWalFd = pFd; } } }else{ pFd->pRbu = pRbuVfs->pRbu; |
︙ | ︙ | |||
4669 4670 4671 4672 4673 4674 4675 4676 4677 4678 | zOpen = 0; } if( rc==SQLITE_OK ){ rc = pRealVfs->xOpen(pRealVfs, zOpen, pFd->pReal, oflags, pOutFlags); } if( pFd->pReal->pMethods ){ /* The xOpen() operation has succeeded. Set the sqlite3_file.pMethods ** pointer and, if the file is a main database file, link it into the ** mutex protected linked list of all such files. */ | > > > > | > | 5121 5122 5123 5124 5125 5126 5127 5128 5129 5130 5131 5132 5133 5134 5135 5136 5137 5138 5139 5140 5141 5142 5143 | zOpen = 0; } if( rc==SQLITE_OK ){ rc = pRealVfs->xOpen(pRealVfs, zOpen, pFd->pReal, oflags, pOutFlags); } if( pFd->pReal->pMethods ){ const sqlite3_io_methods *pMeth = pFd->pReal->pMethods; /* The xOpen() operation has succeeded. Set the sqlite3_file.pMethods ** pointer and, if the file is a main database file, link it into the ** mutex protected linked list of all such files. */ if( pMeth->iVersion<2 || pMeth->xShmLock==0 ){ pFile->pMethods = &rbuvfs_io_methods1; }else{ pFile->pMethods = &rbuvfs_io_methods; } if( flags & SQLITE_OPEN_MAIN_DB ){ rbuMainlistAdd(pFd); } }else{ sqlite3_free(pFd->zDel); } |
︙ | ︙ | |||
4723 4724 4725 4726 4727 4728 4729 | ** b) if the *-wal file does not exist, claim that it does anyway, ** causing SQLite to call xOpen() to open it. This call will also ** be intercepted (see the rbuVfsOpen() function) and the *-oal ** file opened instead. */ if( rc==SQLITE_OK && flags==SQLITE_ACCESS_EXISTS ){ rbu_file *pDb = rbuFindMaindb(pRbuVfs, zPath, 1); | | > | 5180 5181 5182 5183 5184 5185 5186 5187 5188 5189 5190 5191 5192 5193 5194 5195 | ** b) if the *-wal file does not exist, claim that it does anyway, ** causing SQLite to call xOpen() to open it. This call will also ** be intercepted (see the rbuVfsOpen() function) and the *-oal ** file opened instead. */ if( rc==SQLITE_OK && flags==SQLITE_ACCESS_EXISTS ){ rbu_file *pDb = rbuFindMaindb(pRbuVfs, zPath, 1); if( pDb && pDb->pRbu->eStage==RBU_STAGE_OAL ){ assert( pDb->pRbu ); if( *pResOut ){ rc = SQLITE_CANTOPEN; }else{ sqlite3_int64 sz = 0; rc = rbuVfsFileSize(&pDb->base, &sz); *pResOut = (sz>0); } |
︙ | ︙ |
Changes to ext/rbu/sqlite3rbu.h.
1 2 3 4 5 6 7 8 9 10 11 12 | /* ** 2014 August 30 ** ** The author disclaims copyright to this source code. In place of ** a legal notice, here is a blessing: ** ** May you do good and not evil. ** May you find forgiveness for yourself and forgive others. ** May you share freely, never taking more than you give. ** ************************************************************************* ** | | | | | | | | | | | | | | | | | | | | | | | | | | | | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 | /* ** 2014 August 30 ** ** The author disclaims copyright to this source code. In place of ** a legal notice, here is a blessing: ** ** May you do good and not evil. ** May you find forgiveness for yourself and forgive others. ** May you share freely, never taking more than you give. ** ************************************************************************* ** ** This file contains the public interface for the RBU extension. */ /* ** SUMMARY ** ** Writing a transaction containing a large number of operations on ** b-tree indexes that are collectively larger than the available cache ** memory can be very inefficient. ** ** The problem is that in order to update a b-tree, the leaf page (at least) ** containing the entry being inserted or deleted must be modified. If the ** working set of leaves is larger than the available cache memory, then a ** single leaf that is modified more than once as part of the transaction ** may be loaded from or written to the persistent media multiple times. ** Additionally, because the index updates are likely to be applied in ** random order, access to pages within the database is also likely to be in ** random order, which is itself quite inefficient. ** ** One way to improve the situation is to sort the operations on each index ** by index key before applying them to the b-tree. This leads to an IO ** pattern that resembles a single linear scan through the index b-tree, ** and all but guarantees each modified leaf page is loaded and stored ** exactly once. SQLite uses this trick to improve the performance of ** CREATE INDEX commands. This extension allows it to be used to improve ** the performance of large transactions on existing databases. ** ** Additionally, this extension allows the work involved in writing the ** large transaction to be broken down into sub-transactions performed ** sequentially by separate processes. This is useful if the system cannot ** guarantee that a single update process will run for long enough to apply ** the entire update, for example because the update is being applied on a ** mobile device that is frequently rebooted. Even after the writer process ** has committed one or more sub-transactions, other database clients continue ** to read from the original database snapshot. In other words, partially ** applied transactions are not visible to other clients. ** ** "RBU" stands for "Resumable Bulk Update". As in a large database update ** transmitted via a wireless network to a mobile device. A transaction ** applied using this extension is hence refered to as an "RBU update". ** ** ** LIMITATIONS ** ** An "RBU update" transaction is subject to the following limitations: ** ** * The transaction must consist of INSERT, UPDATE and DELETE operations ** only. ** ** * INSERT statements may not use any default values. ** ** * UPDATE and DELETE statements must identify their target rows by ** non-NULL PRIMARY KEY values. Rows with NULL values stored in PRIMARY ** KEY fields may not be updated or deleted. If the table being written ** has no PRIMARY KEY, affected rows must be identified by rowid. ** ** * UPDATE statements may not modify PRIMARY KEY columns. ** ** * No triggers will be fired. ** ** * No foreign key violations are detected or reported. ** ** * CHECK constraints are not enforced. ** ** * No constraint handling mode except for "OR ROLLBACK" is supported. ** ** ** PREPARATION ** ** An "RBU update" is stored as a separate SQLite database. A database ** containing an RBU update is an "RBU database". For each table in the ** target database to be updated, the RBU database should contain a table ** named "data_<target name>" containing the same set of columns as the ** target table, and one more - "rbu_control". The data_% table should ** have no PRIMARY KEY or UNIQUE constraints, but each column should have ** the same type as the corresponding column in the target database. ** The "rbu_control" column should have no type at all. For example, if ** the target database contains: ** ** CREATE TABLE t1(a INTEGER PRIMARY KEY, b TEXT, c UNIQUE); ** ** Then the RBU database should contain: ** ** CREATE TABLE data_t1(a INTEGER, b TEXT, c, rbu_control); ** ** The order of the columns in the data_% table does not matter. ** ** Instead of a regular table, the RBU database may also contain virtual ** tables or views named using the data_<target> naming scheme. ** ** Instead of the plain data_<target> naming scheme, RBU database tables ** may also be named data<integer>_<target>, where <integer> is any sequence ** of zero or more numeric characters (0-9). This can be significant because ** tables within the RBU database are always processed in order sorted by ** name. By judicious selection of the <integer> portion of the names ** of the RBU tables the user can therefore control the order in which they ** are processed. This can be useful, for example, to ensure that "external ** content" FTS4 tables are updated before their underlying content tables. ** ** If the target database table is a virtual table or a table that has no ** PRIMARY KEY declaration, the data_% table must also contain a column ** named "rbu_rowid". This column is mapped to the table's implicit primary ** key column - "rowid". Virtual tables for which the "rowid" column does ** not function like a primary key value cannot be updated using RBU. For ** example, if the target db contains either of the following: ** ** CREATE VIRTUAL TABLE x1 USING fts3(a, b); ** CREATE TABLE x1(a, b) ** ** then the RBU database should contain: ** |
︙ | ︙ | |||
132 133 134 135 136 137 138 | ** CREATE VIRTUAL TABLE ft1 USING fts4(a, b, languageid='langid'); ** ** Either of the following input table schemas may be used: ** ** CREATE TABLE data_ft1(a, b, langid, rbu_rowid, rbu_control); ** CREATE TABLE data_ft1(a, b, rbu_rowid, rbu_control); ** | | | | | | | | | | | | | | | | | | | 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 | ** CREATE VIRTUAL TABLE ft1 USING fts4(a, b, languageid='langid'); ** ** Either of the following input table schemas may be used: ** ** CREATE TABLE data_ft1(a, b, langid, rbu_rowid, rbu_control); ** CREATE TABLE data_ft1(a, b, rbu_rowid, rbu_control); ** ** For each row to INSERT into the target database as part of the RBU ** update, the corresponding data_% table should contain a single record ** with the "rbu_control" column set to contain integer value 0. The ** other columns should be set to the values that make up the new record ** to insert. ** ** If the target database table has an INTEGER PRIMARY KEY, it is not ** possible to insert a NULL value into the IPK column. Attempting to ** do so results in an SQLITE_MISMATCH error. ** ** For each row to DELETE from the target database as part of the RBU ** update, the corresponding data_% table should contain a single record ** with the "rbu_control" column set to contain integer value 1. The ** real primary key values of the row to delete should be stored in the ** corresponding columns of the data_% table. The values stored in the ** other columns are not used. ** ** For each row to UPDATE from the target database as part of the RBU ** update, the corresponding data_% table should contain a single record ** with the "rbu_control" column set to contain a value of type text. ** The real primary key values identifying the row to update should be ** stored in the corresponding columns of the data_% table row, as should ** the new values of all columns being update. The text value in the ** "rbu_control" column must contain the same number of characters as ** there are columns in the target database table, and must consist entirely ** of 'x' and '.' characters (or in some special cases 'd' - see below). For ** each column that is being updated, the corresponding character is set to ** 'x'. For those that remain as they are, the corresponding character of the ** rbu_control value should be set to '.'. For example, given the tables ** above, the update statement: ** ** UPDATE t1 SET c = 'usa' WHERE a = 4; ** ** is represented by the data_t1 row created by: ** ** INSERT INTO data_t1(a, b, c, rbu_control) VALUES(4, NULL, 'usa', '..x'); ** ** Instead of an 'x' character, characters of the rbu_control value specified ** for UPDATEs may also be set to 'd'. In this case, instead of updating the ** target table with the value stored in the corresponding data_% column, the ** user-defined SQL function "rbu_delta()" is invoked and the result stored in ** the target table column. rbu_delta() is invoked with two arguments - the ** original value currently stored in the target table column and the ** value specified in the data_xxx table. ** ** For example, this row: ** ** INSERT INTO data_t1(a, b, c, rbu_control) VALUES(4, NULL, 'usa', '..d'); ** ** is similar to an UPDATE statement such as: ** ** UPDATE t1 SET c = rbu_delta(c, 'usa') WHERE a = 4; ** ** Finally, if an 'f' character appears in place of a 'd' or 's' in an ** ota_control string, the contents of the data_xxx table column is assumed ** to be a "fossil delta" - a patch to be applied to a blob value in the ** format used by the fossil source-code management system. In this case ** the existing value within the target database table must be of type BLOB. ** It is replaced by the result of applying the specified fossil delta to ** itself. ** ** If the target database table is a virtual table or a table with no PRIMARY ** KEY, the rbu_control value should not include a character corresponding ** to the rbu_rowid value. For example, this: ** ** INSERT INTO data_ft1(a, b, rbu_rowid, rbu_control) ** VALUES(NULL, 'usa', 12, '.x'); ** ** causes a result similar to: ** ** UPDATE ft1 SET b = 'usa' WHERE rowid = 12; ** ** The data_xxx tables themselves should have no PRIMARY KEY declarations. |
︙ | ︙ | |||
539 540 541 542 543 544 545 546 547 548 549 550 551 552 | #define SQLITE_RBU_STATE_OAL 1 #define SQLITE_RBU_STATE_MOVE 2 #define SQLITE_RBU_STATE_CHECKPOINT 3 #define SQLITE_RBU_STATE_DONE 4 #define SQLITE_RBU_STATE_ERROR 5 SQLITE_API int sqlite3rbu_state(sqlite3rbu *pRbu); /* ** Create an RBU VFS named zName that accesses the underlying file-system ** via existing VFS zParent. Or, if the zParent parameter is passed NULL, ** then the new RBU VFS uses the default system VFS to access the file-system. ** The new object is registered as a non-default VFS with SQLite before ** returning. | > > > > > > > > > > > > > > > > > > > > > > > > > > > > | 539 540 541 542 543 544 545 546 547 548 549 550 551 552 553 554 555 556 557 558 559 560 561 562 563 564 565 566 567 568 569 570 571 572 573 574 575 576 577 578 579 580 | #define SQLITE_RBU_STATE_OAL 1 #define SQLITE_RBU_STATE_MOVE 2 #define SQLITE_RBU_STATE_CHECKPOINT 3 #define SQLITE_RBU_STATE_DONE 4 #define SQLITE_RBU_STATE_ERROR 5 SQLITE_API int sqlite3rbu_state(sqlite3rbu *pRbu); /* ** As part of applying an RBU update or performing an RBU vacuum operation, ** the system must at one point move the *-oal file to the equivalent *-wal ** path. Normally, it does this by invoking POSIX function rename(2) directly. ** Except on WINCE platforms, where it uses win32 API MoveFileW(). This ** function may be used to register a callback that the RBU module will invoke ** instead of one of these APIs. ** ** If a callback is registered with an RBU handle, it invokes it instead ** of rename(2) when it needs to move a file within the file-system. The ** first argument passed to the xRename() callback is a copy of the second ** argument (pArg) passed to this function. The second is the full path ** to the file to move and the third the full path to which it should be ** moved. The callback function should return SQLITE_OK to indicate ** success. If an error occurs, it should return an SQLite error code. ** In this case the RBU operation will be abandoned and the error returned ** to the RBU user. ** ** Passing a NULL pointer in place of the xRename argument to this function ** restores the default behaviour. */ SQLITE_API void sqlite3rbu_rename_handler( sqlite3rbu *pRbu, void *pArg, int (*xRename)(void *pArg, const char *zOld, const char *zNew) ); /* ** Create an RBU VFS named zName that accesses the underlying file-system ** via existing VFS zParent. Or, if the zParent parameter is passed NULL, ** then the new RBU VFS uses the default system VFS to access the file-system. ** The new object is registered as a non-default VFS with SQLite before ** returning. |
︙ | ︙ |
Changes to ext/rbu/test_rbu.c.
︙ | ︙ | |||
22 23 24 25 26 27 28 29 30 31 32 33 34 35 | #else # include "tcl.h" # ifndef SQLITE_TCLAPI # define SQLITE_TCLAPI # endif #endif #include <assert.h> /* From main.c */ extern const char *sqlite3ErrName(int); extern int sqlite3TestMakePointerStr(Tcl_Interp*, char*, void*); void test_rbu_delta(sqlite3_context *pCtx, int nArg, sqlite3_value **apVal){ Tcl_Interp *interp = (Tcl_Interp*)sqlite3_user_data(pCtx); | > > > > > > > > | 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 | #else # include "tcl.h" # ifndef SQLITE_TCLAPI # define SQLITE_TCLAPI # endif #endif #include <assert.h> #include <string.h> typedef struct TestRbu TestRbu; struct TestRbu { sqlite3rbu *pRbu; Tcl_Interp *interp; Tcl_Obj *xRename; }; /* From main.c */ extern const char *sqlite3ErrName(int); extern int sqlite3TestMakePointerStr(Tcl_Interp*, char*, void*); void test_rbu_delta(sqlite3_context *pCtx, int nArg, sqlite3_value **apVal){ Tcl_Interp *interp = (Tcl_Interp*)sqlite3_user_data(pCtx); |
︙ | ︙ | |||
51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 | }else{ Tcl_BackgroundError(interp); } Tcl_DecrRefCount(pScript); } static int SQLITE_TCLAPI test_sqlite3rbu_cmd( ClientData clientData, Tcl_Interp *interp, int objc, Tcl_Obj *CONST objv[] ){ int ret = TCL_OK; | > > > > > > > > > > > > > > > | > | 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 | }else{ Tcl_BackgroundError(interp); } Tcl_DecrRefCount(pScript); } static int xRenameCallback(void *pArg, const char *zOld, const char *zNew){ int rc = SQLITE_OK; TestRbu *pTest = (TestRbu*)pArg; Tcl_Obj *pEval = Tcl_DuplicateObj(pTest->xRename); Tcl_IncrRefCount(pEval); Tcl_ListObjAppendElement(pTest->interp, pEval, Tcl_NewStringObj(zOld, -1)); Tcl_ListObjAppendElement(pTest->interp, pEval, Tcl_NewStringObj(zNew, -1)); rc = Tcl_EvalObjEx(pTest->interp, pEval, TCL_GLOBAL_ONLY); Tcl_DecrRefCount(pEval); return rc ? SQLITE_IOERR : SQLITE_OK; } static int SQLITE_TCLAPI test_sqlite3rbu_cmd( ClientData clientData, Tcl_Interp *interp, int objc, Tcl_Obj *CONST objv[] ){ int ret = TCL_OK; TestRbu *pTest = (TestRbu*)clientData; sqlite3rbu *pRbu = pTest->pRbu; struct RbuCmd { const char *zName; int nArg; const char *zUsage; } aCmd[] = { {"step", 2, ""}, /* 0 */ {"close", 2, ""}, /* 1 */ {"create_rbu_delta", 2, ""}, /* 2 */ {"savestate", 2, ""}, /* 3 */ {"dbMain_eval", 3, "SQL"}, /* 4 */ {"bp_progress", 2, ""}, /* 5 */ {"db", 3, "RBU"}, /* 6 */ {"state", 2, ""}, /* 7 */ {"progress", 2, ""}, /* 8 */ {"close_no_error", 2, ""}, /* 9 */ {"temp_size_limit", 3, "LIMIT"}, /* 10 */ {"temp_size", 2, ""}, /* 11 */ {"dbRbu_eval", 3, "SQL"}, /* 12 */ {"rename_handler", 3, "SCRIPT"},/* 13 */ {0,0,0} }; int iCmd; if( objc<2 ){ Tcl_WrongNumArgs(interp, 1, objv, "METHOD"); return TCL_ERROR; |
︙ | ︙ | |||
123 124 125 126 127 128 129 130 131 132 133 134 135 136 | Tcl_SetObjResult(interp, Tcl_NewStringObj(sqlite3ErrName(rc), -1)); if( zErrmsg ){ Tcl_AppendResult(interp, " - ", zErrmsg, 0); sqlite3_free(zErrmsg); } ret = TCL_ERROR; } break; } case 2: /* create_rbu_delta */ { sqlite3 *db = sqlite3rbu_db(pRbu, 0); int rc = sqlite3_create_function( db, "rbu_delta", -1, SQLITE_UTF8, (void*)interp, test_rbu_delta, 0, 0 | > > | 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 | Tcl_SetObjResult(interp, Tcl_NewStringObj(sqlite3ErrName(rc), -1)); if( zErrmsg ){ Tcl_AppendResult(interp, " - ", zErrmsg, 0); sqlite3_free(zErrmsg); } ret = TCL_ERROR; } if( pTest->xRename ) Tcl_DecrRefCount(pTest->xRename); ckfree(pTest); break; } case 2: /* create_rbu_delta */ { sqlite3 *db = sqlite3rbu_db(pRbu, 0); int rc = sqlite3_create_function( db, "rbu_delta", -1, SQLITE_UTF8, (void*)interp, test_rbu_delta, 0, 0 |
︙ | ︙ | |||
209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 | break; } case 11: /* temp_size */ { sqlite3_int64 sz = sqlite3rbu_temp_size(pRbu); Tcl_SetObjResult(interp, Tcl_NewWideIntObj(sz)); break; } default: /* seems unlikely */ assert( !"cannot happen" ); break; } return ret; } /* ** Tclcmd: sqlite3rbu CMD <target-db> <rbu-db> ?<state-db>? */ static int SQLITE_TCLAPI test_sqlite3rbu( ClientData clientData, Tcl_Interp *interp, | > > > > > > > > > > > > > > > > > > > > > > > > > | 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 | break; } case 11: /* temp_size */ { sqlite3_int64 sz = sqlite3rbu_temp_size(pRbu); Tcl_SetObjResult(interp, Tcl_NewWideIntObj(sz)); break; } case 13: /* rename_handler */ { Tcl_Obj *pScript = objv[2]; assert( !sqlite3_stricmp(aCmd[13].zName, "rename_handler") ); if( Tcl_GetCharLength(pScript)==0 ){ sqlite3rbu_rename_handler(pRbu, 0, 0); }else{ pTest->xRename = Tcl_DuplicateObj(pScript); Tcl_IncrRefCount(pTest->xRename); sqlite3rbu_rename_handler(pRbu, pTest, xRenameCallback); } break; } default: /* seems unlikely */ assert( !"cannot happen" ); break; } return ret; } static void createRbuWrapper( Tcl_Interp *interp, const char *zCmd, sqlite3rbu *pRbu ){ TestRbu *pTest = (TestRbu*)ckalloc(sizeof(TestRbu)); memset(pTest, 0, sizeof(TestRbu)); pTest->pRbu = pRbu; pTest->interp = interp; Tcl_CreateObjCommand(interp, zCmd, test_sqlite3rbu_cmd, (ClientData)pTest, 0); } /* ** Tclcmd: sqlite3rbu CMD <target-db> <rbu-db> ?<state-db>? */ static int SQLITE_TCLAPI test_sqlite3rbu( ClientData clientData, Tcl_Interp *interp, |
︙ | ︙ | |||
243 244 245 246 247 248 249 | } zCmd = Tcl_GetString(objv[1]); zTarget = Tcl_GetString(objv[2]); zRbu = Tcl_GetString(objv[3]); if( objc==5 ) zStateDb = Tcl_GetString(objv[4]); pRbu = sqlite3rbu_open(zTarget, zRbu, zStateDb); | | | 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 | } zCmd = Tcl_GetString(objv[1]); zTarget = Tcl_GetString(objv[2]); zRbu = Tcl_GetString(objv[3]); if( objc==5 ) zStateDb = Tcl_GetString(objv[4]); pRbu = sqlite3rbu_open(zTarget, zRbu, zStateDb); createRbuWrapper(interp, zCmd, pRbu); Tcl_SetObjResult(interp, objv[1]); return TCL_OK; } /* ** Tclcmd: sqlite3rbu_vacuum CMD <target-db> <state-db> */ |
︙ | ︙ | |||
272 273 274 275 276 277 278 | } zCmd = Tcl_GetString(objv[1]); zTarget = Tcl_GetString(objv[2]); if( objc==4 ) zStateDb = Tcl_GetString(objv[3]); if( zStateDb && zStateDb[0]=='\0' ) zStateDb = 0; pRbu = sqlite3rbu_vacuum(zTarget, zStateDb); | | | 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 | } zCmd = Tcl_GetString(objv[1]); zTarget = Tcl_GetString(objv[2]); if( objc==4 ) zStateDb = Tcl_GetString(objv[3]); if( zStateDb && zStateDb[0]=='\0' ) zStateDb = 0; pRbu = sqlite3rbu_vacuum(zTarget, zStateDb); createRbuWrapper(interp, zCmd, pRbu); Tcl_SetObjResult(interp, objv[1]); return TCL_OK; } /* ** Tclcmd: sqlite3rbu_create_vfs ?-default? NAME PARENT */ |
︙ | ︙ |
Added ext/recover/dbdata.c.
> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340 341 342 343 344 345 346 347 348 349 350 351 352 353 354 355 356 357 358 359 360 361 362 363 364 365 366 367 368 369 370 371 372 373 374 375 376 377 378 379 380 381 382 383 384 385 386 387 388 389 390 391 392 393 394 395 396 397 398 399 400 401 402 403 404 405 406 407 408 409 410 411 412 413 414 415 416 417 418 419 420 421 422 423 424 425 426 427 428 429 430 431 432 433 434 435 436 437 438 439 440 441 442 443 444 445 446 447 448 449 450 451 452 453 454 455 456 457 458 459 460 461 462 463 464 465 466 467 468 469 470 471 472 473 474 475 476 477 478 479 480 481 482 483 484 485 486 487 488 489 490 491 492 493 494 495 496 497 498 499 500 501 502 503 504 505 506 507 508 509 510 511 512 513 514 515 516 517 518 519 520 521 522 523 524 525 526 527 528 529 530 531 532 533 534 535 536 537 538 539 540 541 542 543 544 545 546 547 548 549 550 551 552 553 554 555 556 557 558 559 560 561 562 563 564 565 566 567 568 569 570 571 572 573 574 575 576 577 578 579 580 581 582 583 584 585 586 587 588 589 590 591 592 593 594 595 596 597 598 599 600 601 602 603 604 605 606 607 608 609 610 611 612 613 614 615 616 617 618 619 620 621 622 623 624 625 626 627 628 629 630 631 632 633 634 635 636 637 638 639 640 641 642 643 644 645 646 647 648 649 650 651 652 653 654 655 656 657 658 659 660 661 662 663 664 665 666 667 668 669 670 671 672 673 674 675 676 677 678 679 680 681 682 683 684 685 686 687 688 689 690 691 692 693 694 695 696 697 698 699 700 701 702 703 704 705 706 707 708 709 710 711 712 713 714 715 716 717 718 719 720 721 722 723 724 725 726 727 728 729 730 731 732 733 734 735 736 737 738 739 740 741 742 743 744 745 746 747 748 749 750 751 752 753 754 755 756 757 758 759 760 761 762 763 764 765 766 767 768 769 770 771 772 773 774 775 776 777 778 779 780 781 782 783 784 785 786 787 788 789 790 791 792 793 794 795 796 797 798 799 800 801 802 803 804 805 806 807 808 809 810 811 812 813 814 815 816 817 818 819 820 821 822 823 824 825 826 827 828 829 830 831 832 833 834 835 836 837 838 839 840 841 842 843 844 845 846 847 848 849 850 851 852 853 854 855 856 857 858 859 860 861 862 863 864 865 866 867 868 869 870 871 872 873 874 875 876 877 878 879 880 881 882 883 884 885 886 887 888 889 890 891 892 893 894 895 896 897 898 899 900 901 902 903 904 905 906 907 908 909 910 911 912 913 914 915 916 917 918 919 920 921 922 923 924 925 926 927 928 929 930 931 932 933 934 935 936 937 938 939 940 941 942 943 944 945 946 947 948 | /* ** 2019-04-17 ** ** The author disclaims copyright to this source code. In place of ** a legal notice, here is a blessing: ** ** May you do good and not evil. ** May you find forgiveness for yourself and forgive others. ** May you share freely, never taking more than you give. ** ****************************************************************************** ** ** This file contains an implementation of two eponymous virtual tables, ** "sqlite_dbdata" and "sqlite_dbptr". Both modules require that the ** "sqlite_dbpage" eponymous virtual table be available. ** ** SQLITE_DBDATA: ** sqlite_dbdata is used to extract data directly from a database b-tree ** page and its associated overflow pages, bypassing the b-tree layer. ** The table schema is equivalent to: ** ** CREATE TABLE sqlite_dbdata( ** pgno INTEGER, ** cell INTEGER, ** field INTEGER, ** value ANY, ** schema TEXT HIDDEN ** ); ** ** IMPORTANT: THE VIRTUAL TABLE SCHEMA ABOVE IS SUBJECT TO CHANGE. IN THE ** FUTURE NEW NON-HIDDEN COLUMNS MAY BE ADDED BETWEEN "value" AND ** "schema". ** ** Each page of the database is inspected. If it cannot be interpreted as ** a b-tree page, or if it is a b-tree page containing 0 entries, the ** sqlite_dbdata table contains no rows for that page. Otherwise, the ** table contains one row for each field in the record associated with ** each cell on the page. For intkey b-trees, the key value is stored in ** field -1. ** ** For example, for the database: ** ** CREATE TABLE t1(a, b); -- root page is page 2 ** INSERT INTO t1(rowid, a, b) VALUES(5, 'v', 'five'); ** INSERT INTO t1(rowid, a, b) VALUES(10, 'x', 'ten'); ** ** the sqlite_dbdata table contains, as well as from entries related to ** page 1, content equivalent to: ** ** INSERT INTO sqlite_dbdata(pgno, cell, field, value) VALUES ** (2, 0, -1, 5 ), ** (2, 0, 0, 'v' ), ** (2, 0, 1, 'five'), ** (2, 1, -1, 10 ), ** (2, 1, 0, 'x' ), ** (2, 1, 1, 'ten' ); ** ** If database corruption is encountered, this module does not report an ** error. Instead, it attempts to extract as much data as possible and ** ignores the corruption. ** ** SQLITE_DBPTR: ** The sqlite_dbptr table has the following schema: ** ** CREATE TABLE sqlite_dbptr( ** pgno INTEGER, ** child INTEGER, ** schema TEXT HIDDEN ** ); ** ** It contains one entry for each b-tree pointer between a parent and ** child page in the database. */ #if !defined(SQLITEINT_H) #include "sqlite3ext.h" typedef unsigned char u8; typedef unsigned int u32; #endif SQLITE_EXTENSION_INIT1 #include <string.h> #include <assert.h> #ifndef SQLITE_OMIT_VIRTUALTABLE #define DBDATA_PADDING_BYTES 100 typedef struct DbdataTable DbdataTable; typedef struct DbdataCursor DbdataCursor; /* Cursor object */ struct DbdataCursor { sqlite3_vtab_cursor base; /* Base class. Must be first */ sqlite3_stmt *pStmt; /* For fetching database pages */ int iPgno; /* Current page number */ u8 *aPage; /* Buffer containing page */ int nPage; /* Size of aPage[] in bytes */ int nCell; /* Number of cells on aPage[] */ int iCell; /* Current cell number */ int bOnePage; /* True to stop after one page */ int szDb; sqlite3_int64 iRowid; /* Only for the sqlite_dbdata table */ u8 *pRec; /* Buffer containing current record */ sqlite3_int64 nRec; /* Size of pRec[] in bytes */ sqlite3_int64 nHdr; /* Size of header in bytes */ int iField; /* Current field number */ u8 *pHdrPtr; u8 *pPtr; u32 enc; /* Text encoding */ sqlite3_int64 iIntkey; /* Integer key value */ }; /* Table object */ struct DbdataTable { sqlite3_vtab base; /* Base class. Must be first */ sqlite3 *db; /* The database connection */ sqlite3_stmt *pStmt; /* For fetching database pages */ int bPtr; /* True for sqlite3_dbptr table */ }; /* Column and schema definitions for sqlite_dbdata */ #define DBDATA_COLUMN_PGNO 0 #define DBDATA_COLUMN_CELL 1 #define DBDATA_COLUMN_FIELD 2 #define DBDATA_COLUMN_VALUE 3 #define DBDATA_COLUMN_SCHEMA 4 #define DBDATA_SCHEMA \ "CREATE TABLE x(" \ " pgno INTEGER," \ " cell INTEGER," \ " field INTEGER," \ " value ANY," \ " schema TEXT HIDDEN" \ ")" /* Column and schema definitions for sqlite_dbptr */ #define DBPTR_COLUMN_PGNO 0 #define DBPTR_COLUMN_CHILD 1 #define DBPTR_COLUMN_SCHEMA 2 #define DBPTR_SCHEMA \ "CREATE TABLE x(" \ " pgno INTEGER," \ " child INTEGER," \ " schema TEXT HIDDEN" \ ")" /* ** Connect to an sqlite_dbdata (pAux==0) or sqlite_dbptr (pAux!=0) virtual ** table. */ static int dbdataConnect( sqlite3 *db, void *pAux, int argc, const char *const*argv, sqlite3_vtab **ppVtab, char **pzErr ){ DbdataTable *pTab = 0; int rc = sqlite3_declare_vtab(db, pAux ? DBPTR_SCHEMA : DBDATA_SCHEMA); (void)argc; (void)argv; (void)pzErr; if( rc==SQLITE_OK ){ pTab = (DbdataTable*)sqlite3_malloc64(sizeof(DbdataTable)); if( pTab==0 ){ rc = SQLITE_NOMEM; }else{ memset(pTab, 0, sizeof(DbdataTable)); pTab->db = db; pTab->bPtr = (pAux!=0); } } *ppVtab = (sqlite3_vtab*)pTab; return rc; } /* ** Disconnect from or destroy a sqlite_dbdata or sqlite_dbptr virtual table. */ static int dbdataDisconnect(sqlite3_vtab *pVtab){ DbdataTable *pTab = (DbdataTable*)pVtab; if( pTab ){ sqlite3_finalize(pTab->pStmt); sqlite3_free(pVtab); } return SQLITE_OK; } /* ** This function interprets two types of constraints: ** ** schema=? ** pgno=? ** ** If neither are present, idxNum is set to 0. If schema=? is present, ** the 0x01 bit in idxNum is set. If pgno=? is present, the 0x02 bit ** in idxNum is set. ** ** If both parameters are present, schema is in position 0 and pgno in ** position 1. */ static int dbdataBestIndex(sqlite3_vtab *tab, sqlite3_index_info *pIdx){ DbdataTable *pTab = (DbdataTable*)tab; int i; int iSchema = -1; int iPgno = -1; int colSchema = (pTab->bPtr ? DBPTR_COLUMN_SCHEMA : DBDATA_COLUMN_SCHEMA); for(i=0; i<pIdx->nConstraint; i++){ struct sqlite3_index_constraint *p = &pIdx->aConstraint[i]; if( p->op==SQLITE_INDEX_CONSTRAINT_EQ ){ if( p->iColumn==colSchema ){ if( p->usable==0 ) return SQLITE_CONSTRAINT; iSchema = i; } if( p->iColumn==DBDATA_COLUMN_PGNO && p->usable ){ iPgno = i; } } } if( iSchema>=0 ){ pIdx->aConstraintUsage[iSchema].argvIndex = 1; pIdx->aConstraintUsage[iSchema].omit = 1; } if( iPgno>=0 ){ pIdx->aConstraintUsage[iPgno].argvIndex = 1 + (iSchema>=0); pIdx->aConstraintUsage[iPgno].omit = 1; pIdx->estimatedCost = 100; pIdx->estimatedRows = 50; if( pTab->bPtr==0 && pIdx->nOrderBy && pIdx->aOrderBy[0].desc==0 ){ int iCol = pIdx->aOrderBy[0].iColumn; if( pIdx->nOrderBy==1 ){ pIdx->orderByConsumed = (iCol==0 || iCol==1); }else if( pIdx->nOrderBy==2 && pIdx->aOrderBy[1].desc==0 && iCol==0 ){ pIdx->orderByConsumed = (pIdx->aOrderBy[1].iColumn==1); } } }else{ pIdx->estimatedCost = 100000000; pIdx->estimatedRows = 1000000000; } pIdx->idxNum = (iSchema>=0 ? 0x01 : 0x00) | (iPgno>=0 ? 0x02 : 0x00); return SQLITE_OK; } /* ** Open a new sqlite_dbdata or sqlite_dbptr cursor. */ static int dbdataOpen(sqlite3_vtab *pVTab, sqlite3_vtab_cursor **ppCursor){ DbdataCursor *pCsr; pCsr = (DbdataCursor*)sqlite3_malloc64(sizeof(DbdataCursor)); if( pCsr==0 ){ return SQLITE_NOMEM; }else{ memset(pCsr, 0, sizeof(DbdataCursor)); pCsr->base.pVtab = pVTab; } *ppCursor = (sqlite3_vtab_cursor *)pCsr; return SQLITE_OK; } /* ** Restore a cursor object to the state it was in when first allocated ** by dbdataOpen(). */ static void dbdataResetCursor(DbdataCursor *pCsr){ DbdataTable *pTab = (DbdataTable*)(pCsr->base.pVtab); if( pTab->pStmt==0 ){ pTab->pStmt = pCsr->pStmt; }else{ sqlite3_finalize(pCsr->pStmt); } pCsr->pStmt = 0; pCsr->iPgno = 1; pCsr->iCell = 0; pCsr->iField = 0; pCsr->bOnePage = 0; sqlite3_free(pCsr->aPage); sqlite3_free(pCsr->pRec); pCsr->pRec = 0; pCsr->aPage = 0; } /* ** Close an sqlite_dbdata or sqlite_dbptr cursor. */ static int dbdataClose(sqlite3_vtab_cursor *pCursor){ DbdataCursor *pCsr = (DbdataCursor*)pCursor; dbdataResetCursor(pCsr); sqlite3_free(pCsr); return SQLITE_OK; } /* ** Utility methods to decode 16 and 32-bit big-endian unsigned integers. */ static u32 get_uint16(unsigned char *a){ return (a[0]<<8)|a[1]; } static u32 get_uint32(unsigned char *a){ return ((u32)a[0]<<24) | ((u32)a[1]<<16) | ((u32)a[2]<<8) | ((u32)a[3]); } /* ** Load page pgno from the database via the sqlite_dbpage virtual table. ** If successful, set (*ppPage) to point to a buffer containing the page ** data, (*pnPage) to the size of that buffer in bytes and return ** SQLITE_OK. In this case it is the responsibility of the caller to ** eventually free the buffer using sqlite3_free(). ** ** Or, if an error occurs, set both (*ppPage) and (*pnPage) to 0 and ** return an SQLite error code. */ static int dbdataLoadPage( DbdataCursor *pCsr, /* Cursor object */ u32 pgno, /* Page number of page to load */ u8 **ppPage, /* OUT: pointer to page buffer */ int *pnPage /* OUT: Size of (*ppPage) in bytes */ ){ int rc2; int rc = SQLITE_OK; sqlite3_stmt *pStmt = pCsr->pStmt; *ppPage = 0; *pnPage = 0; if( pgno>0 ){ sqlite3_bind_int64(pStmt, 2, pgno); if( SQLITE_ROW==sqlite3_step(pStmt) ){ int nCopy = sqlite3_column_bytes(pStmt, 0); if( nCopy>0 ){ u8 *pPage; pPage = (u8*)sqlite3_malloc64(nCopy + DBDATA_PADDING_BYTES); if( pPage==0 ){ rc = SQLITE_NOMEM; }else{ const u8 *pCopy = sqlite3_column_blob(pStmt, 0); memcpy(pPage, pCopy, nCopy); memset(&pPage[nCopy], 0, DBDATA_PADDING_BYTES); } *ppPage = pPage; *pnPage = nCopy; } } rc2 = sqlite3_reset(pStmt); if( rc==SQLITE_OK ) rc = rc2; } return rc; } /* ** Read a varint. Put the value in *pVal and return the number of bytes. */ static int dbdataGetVarint(const u8 *z, sqlite3_int64 *pVal){ sqlite3_uint64 u = 0; int i; for(i=0; i<8; i++){ u = (u<<7) + (z[i]&0x7f); if( (z[i]&0x80)==0 ){ *pVal = (sqlite3_int64)u; return i+1; } } u = (u<<8) + (z[i]&0xff); *pVal = (sqlite3_int64)u; return 9; } /* ** Like dbdataGetVarint(), but set the output to 0 if it is less than 0 ** or greater than 0xFFFFFFFF. This can be used for all varints in an ** SQLite database except for key values in intkey tables. */ static int dbdataGetVarintU32(const u8 *z, sqlite3_int64 *pVal){ sqlite3_int64 val; int nRet = dbdataGetVarint(z, &val); if( val<0 || val>0xFFFFFFFF ) val = 0; *pVal = val; return nRet; } /* ** Return the number of bytes of space used by an SQLite value of type ** eType. */ static int dbdataValueBytes(int eType){ switch( eType ){ case 0: case 8: case 9: case 10: case 11: return 0; case 1: return 1; case 2: return 2; case 3: return 3; case 4: return 4; case 5: return 6; case 6: case 7: return 8; default: if( eType>0 ){ return ((eType-12) / 2); } return 0; } } /* ** Load a value of type eType from buffer pData and use it to set the ** result of context object pCtx. */ static void dbdataValue( sqlite3_context *pCtx, u32 enc, int eType, u8 *pData, sqlite3_int64 nData ){ if( eType>=0 && dbdataValueBytes(eType)<=nData ){ switch( eType ){ case 0: case 10: case 11: sqlite3_result_null(pCtx); break; case 8: sqlite3_result_int(pCtx, 0); break; case 9: sqlite3_result_int(pCtx, 1); break; case 1: case 2: case 3: case 4: case 5: case 6: case 7: { sqlite3_uint64 v = (signed char)pData[0]; pData++; switch( eType ){ case 7: case 6: v = (v<<16) + (pData[0]<<8) + pData[1]; pData += 2; case 5: v = (v<<16) + (pData[0]<<8) + pData[1]; pData += 2; case 4: v = (v<<8) + pData[0]; pData++; case 3: v = (v<<8) + pData[0]; pData++; case 2: v = (v<<8) + pData[0]; pData++; } if( eType==7 ){ double r; memcpy(&r, &v, sizeof(r)); sqlite3_result_double(pCtx, r); }else{ sqlite3_result_int64(pCtx, (sqlite3_int64)v); } break; } default: { int n = ((eType-12) / 2); if( eType % 2 ){ switch( enc ){ #ifndef SQLITE_OMIT_UTF16 case SQLITE_UTF16BE: sqlite3_result_text16be(pCtx, (void*)pData, n, SQLITE_TRANSIENT); break; case SQLITE_UTF16LE: sqlite3_result_text16le(pCtx, (void*)pData, n, SQLITE_TRANSIENT); break; #endif default: sqlite3_result_text(pCtx, (char*)pData, n, SQLITE_TRANSIENT); break; } }else{ sqlite3_result_blob(pCtx, pData, n, SQLITE_TRANSIENT); } } } } } /* ** Move an sqlite_dbdata or sqlite_dbptr cursor to the next entry. */ static int dbdataNext(sqlite3_vtab_cursor *pCursor){ DbdataCursor *pCsr = (DbdataCursor*)pCursor; DbdataTable *pTab = (DbdataTable*)pCursor->pVtab; pCsr->iRowid++; while( 1 ){ int rc; int iOff = (pCsr->iPgno==1 ? 100 : 0); int bNextPage = 0; if( pCsr->aPage==0 ){ while( 1 ){ if( pCsr->bOnePage==0 && pCsr->iPgno>pCsr->szDb ) return SQLITE_OK; rc = dbdataLoadPage(pCsr, pCsr->iPgno, &pCsr->aPage, &pCsr->nPage); if( rc!=SQLITE_OK ) return rc; if( pCsr->aPage ) break; if( pCsr->bOnePage ) return SQLITE_OK; pCsr->iPgno++; } pCsr->iCell = pTab->bPtr ? -2 : 0; pCsr->nCell = get_uint16(&pCsr->aPage[iOff+3]); } if( pTab->bPtr ){ if( pCsr->aPage[iOff]!=0x02 && pCsr->aPage[iOff]!=0x05 ){ pCsr->iCell = pCsr->nCell; } pCsr->iCell++; if( pCsr->iCell>=pCsr->nCell ){ sqlite3_free(pCsr->aPage); pCsr->aPage = 0; if( pCsr->bOnePage ) return SQLITE_OK; pCsr->iPgno++; }else{ return SQLITE_OK; } }else{ /* If there is no record loaded, load it now. */ if( pCsr->pRec==0 ){ int bHasRowid = 0; int nPointer = 0; sqlite3_int64 nPayload = 0; sqlite3_int64 nHdr = 0; int iHdr; int U, X; int nLocal; switch( pCsr->aPage[iOff] ){ case 0x02: nPointer = 4; break; case 0x0a: break; case 0x0d: bHasRowid = 1; break; default: /* This is not a b-tree page with records on it. Continue. */ pCsr->iCell = pCsr->nCell; break; } if( pCsr->iCell>=pCsr->nCell ){ bNextPage = 1; }else{ iOff += 8 + nPointer + pCsr->iCell*2; if( iOff>pCsr->nPage ){ bNextPage = 1; }else{ iOff = get_uint16(&pCsr->aPage[iOff]); } /* For an interior node cell, skip past the child-page number */ iOff += nPointer; /* Load the "byte of payload including overflow" field */ if( bNextPage || iOff>pCsr->nPage ){ bNextPage = 1; }else{ iOff += dbdataGetVarintU32(&pCsr->aPage[iOff], &nPayload); } /* If this is a leaf intkey cell, load the rowid */ if( bHasRowid && !bNextPage && iOff<pCsr->nPage ){ iOff += dbdataGetVarint(&pCsr->aPage[iOff], &pCsr->iIntkey); } /* Figure out how much data to read from the local page */ U = pCsr->nPage; if( bHasRowid ){ X = U-35; }else{ X = ((U-12)*64/255)-23; } if( nPayload<=X ){ nLocal = nPayload; }else{ int M, K; M = ((U-12)*32/255)-23; K = M+((nPayload-M)%(U-4)); if( K<=X ){ nLocal = K; }else{ nLocal = M; } } if( bNextPage || nLocal+iOff>pCsr->nPage ){ bNextPage = 1; }else{ /* Allocate space for payload. And a bit more to catch small buffer ** overruns caused by attempting to read a varint or similar from ** near the end of a corrupt record. */ pCsr->pRec = (u8*)sqlite3_malloc64(nPayload+DBDATA_PADDING_BYTES); if( pCsr->pRec==0 ) return SQLITE_NOMEM; memset(pCsr->pRec, 0, nPayload+DBDATA_PADDING_BYTES); pCsr->nRec = nPayload; /* Load the nLocal bytes of payload */ memcpy(pCsr->pRec, &pCsr->aPage[iOff], nLocal); iOff += nLocal; /* Load content from overflow pages */ if( nPayload>nLocal ){ sqlite3_int64 nRem = nPayload - nLocal; u32 pgnoOvfl = get_uint32(&pCsr->aPage[iOff]); while( nRem>0 ){ u8 *aOvfl = 0; int nOvfl = 0; int nCopy; rc = dbdataLoadPage(pCsr, pgnoOvfl, &aOvfl, &nOvfl); assert( rc!=SQLITE_OK || aOvfl==0 || nOvfl==pCsr->nPage ); if( rc!=SQLITE_OK ) return rc; if( aOvfl==0 ) break; nCopy = U-4; if( nCopy>nRem ) nCopy = nRem; memcpy(&pCsr->pRec[nPayload-nRem], &aOvfl[4], nCopy); nRem -= nCopy; pgnoOvfl = get_uint32(aOvfl); sqlite3_free(aOvfl); } } iHdr = dbdataGetVarintU32(pCsr->pRec, &nHdr); if( nHdr>nPayload ) nHdr = 0; pCsr->nHdr = nHdr; pCsr->pHdrPtr = &pCsr->pRec[iHdr]; pCsr->pPtr = &pCsr->pRec[pCsr->nHdr]; pCsr->iField = (bHasRowid ? -1 : 0); } } }else{ pCsr->iField++; if( pCsr->iField>0 ){ sqlite3_int64 iType; if( pCsr->pHdrPtr>&pCsr->pRec[pCsr->nRec] ){ bNextPage = 1; }else{ pCsr->pHdrPtr += dbdataGetVarintU32(pCsr->pHdrPtr, &iType); pCsr->pPtr += dbdataValueBytes(iType); } } } if( bNextPage ){ sqlite3_free(pCsr->aPage); sqlite3_free(pCsr->pRec); pCsr->aPage = 0; pCsr->pRec = 0; if( pCsr->bOnePage ) return SQLITE_OK; pCsr->iPgno++; }else{ if( pCsr->iField<0 || pCsr->pHdrPtr<&pCsr->pRec[pCsr->nHdr] ){ return SQLITE_OK; } /* Advance to the next cell. The next iteration of the loop will load ** the record and so on. */ sqlite3_free(pCsr->pRec); pCsr->pRec = 0; pCsr->iCell++; } } } assert( !"can't get here" ); return SQLITE_OK; } /* ** Return true if the cursor is at EOF. */ static int dbdataEof(sqlite3_vtab_cursor *pCursor){ DbdataCursor *pCsr = (DbdataCursor*)pCursor; return pCsr->aPage==0; } /* ** Return true if nul-terminated string zSchema ends in "()". Or false ** otherwise. */ static int dbdataIsFunction(const char *zSchema){ size_t n = strlen(zSchema); if( n>2 && zSchema[n-2]=='(' && zSchema[n-1]==')' ){ return (int)n-2; } return 0; } /* ** Determine the size in pages of database zSchema (where zSchema is ** "main", "temp" or the name of an attached database) and set ** pCsr->szDb accordingly. If successful, return SQLITE_OK. Otherwise, ** an SQLite error code. */ static int dbdataDbsize(DbdataCursor *pCsr, const char *zSchema){ DbdataTable *pTab = (DbdataTable*)pCsr->base.pVtab; char *zSql = 0; int rc, rc2; int nFunc = 0; sqlite3_stmt *pStmt = 0; if( (nFunc = dbdataIsFunction(zSchema))>0 ){ zSql = sqlite3_mprintf("SELECT %.*s(0)", nFunc, zSchema); }else{ zSql = sqlite3_mprintf("PRAGMA %Q.page_count", zSchema); } if( zSql==0 ) return SQLITE_NOMEM; rc = sqlite3_prepare_v2(pTab->db, zSql, -1, &pStmt, 0); sqlite3_free(zSql); if( rc==SQLITE_OK && sqlite3_step(pStmt)==SQLITE_ROW ){ pCsr->szDb = sqlite3_column_int(pStmt, 0); } rc2 = sqlite3_finalize(pStmt); if( rc==SQLITE_OK ) rc = rc2; return rc; } /* ** Attempt to figure out the encoding of the database by retrieving page 1 ** and inspecting the header field. If successful, set the pCsr->enc variable ** and return SQLITE_OK. Otherwise, return an SQLite error code. */ static int dbdataGetEncoding(DbdataCursor *pCsr){ int rc = SQLITE_OK; int nPg1 = 0; u8 *aPg1 = 0; rc = dbdataLoadPage(pCsr, 1, &aPg1, &nPg1); assert( rc!=SQLITE_OK || nPg1==0 || nPg1>=512 ); if( rc==SQLITE_OK && nPg1>0 ){ pCsr->enc = get_uint32(&aPg1[56]); } sqlite3_free(aPg1); return rc; } /* ** xFilter method for sqlite_dbdata and sqlite_dbptr. */ static int dbdataFilter( sqlite3_vtab_cursor *pCursor, int idxNum, const char *idxStr, int argc, sqlite3_value **argv ){ DbdataCursor *pCsr = (DbdataCursor*)pCursor; DbdataTable *pTab = (DbdataTable*)pCursor->pVtab; int rc = SQLITE_OK; const char *zSchema = "main"; (void)idxStr; (void)argc; dbdataResetCursor(pCsr); assert( pCsr->iPgno==1 ); if( idxNum & 0x01 ){ zSchema = (const char*)sqlite3_value_text(argv[0]); if( zSchema==0 ) zSchema = ""; } if( idxNum & 0x02 ){ pCsr->iPgno = sqlite3_value_int(argv[(idxNum & 0x01)]); pCsr->bOnePage = 1; }else{ rc = dbdataDbsize(pCsr, zSchema); } if( rc==SQLITE_OK ){ int nFunc = 0; if( pTab->pStmt ){ pCsr->pStmt = pTab->pStmt; pTab->pStmt = 0; }else if( (nFunc = dbdataIsFunction(zSchema))>0 ){ char *zSql = sqlite3_mprintf("SELECT %.*s(?2)", nFunc, zSchema); if( zSql==0 ){ rc = SQLITE_NOMEM; }else{ rc = sqlite3_prepare_v2(pTab->db, zSql, -1, &pCsr->pStmt, 0); sqlite3_free(zSql); } }else{ rc = sqlite3_prepare_v2(pTab->db, "SELECT data FROM sqlite_dbpage(?) WHERE pgno=?", -1, &pCsr->pStmt, 0 ); } } if( rc==SQLITE_OK ){ rc = sqlite3_bind_text(pCsr->pStmt, 1, zSchema, -1, SQLITE_TRANSIENT); }else{ pTab->base.zErrMsg = sqlite3_mprintf("%s", sqlite3_errmsg(pTab->db)); } /* Try to determine the encoding of the db by inspecting the header ** field on page 1. */ if( rc==SQLITE_OK ){ rc = dbdataGetEncoding(pCsr); } if( rc==SQLITE_OK ){ rc = dbdataNext(pCursor); } return rc; } /* ** Return a column for the sqlite_dbdata or sqlite_dbptr table. */ static int dbdataColumn( sqlite3_vtab_cursor *pCursor, sqlite3_context *ctx, int i ){ DbdataCursor *pCsr = (DbdataCursor*)pCursor; DbdataTable *pTab = (DbdataTable*)pCursor->pVtab; if( pTab->bPtr ){ switch( i ){ case DBPTR_COLUMN_PGNO: sqlite3_result_int64(ctx, pCsr->iPgno); break; case DBPTR_COLUMN_CHILD: { int iOff = pCsr->iPgno==1 ? 100 : 0; if( pCsr->iCell<0 ){ iOff += 8; }else{ iOff += 12 + pCsr->iCell*2; if( iOff>pCsr->nPage ) return SQLITE_OK; iOff = get_uint16(&pCsr->aPage[iOff]); } if( iOff<=pCsr->nPage ){ sqlite3_result_int64(ctx, get_uint32(&pCsr->aPage[iOff])); } break; } } }else{ switch( i ){ case DBDATA_COLUMN_PGNO: sqlite3_result_int64(ctx, pCsr->iPgno); break; case DBDATA_COLUMN_CELL: sqlite3_result_int(ctx, pCsr->iCell); break; case DBDATA_COLUMN_FIELD: sqlite3_result_int(ctx, pCsr->iField); break; case DBDATA_COLUMN_VALUE: { if( pCsr->iField<0 ){ sqlite3_result_int64(ctx, pCsr->iIntkey); }else if( &pCsr->pRec[pCsr->nRec] >= pCsr->pPtr ){ sqlite3_int64 iType; dbdataGetVarintU32(pCsr->pHdrPtr, &iType); dbdataValue( ctx, pCsr->enc, iType, pCsr->pPtr, &pCsr->pRec[pCsr->nRec] - pCsr->pPtr ); } break; } } } return SQLITE_OK; } /* ** Return the rowid for an sqlite_dbdata or sqlite_dptr table. */ static int dbdataRowid(sqlite3_vtab_cursor *pCursor, sqlite_int64 *pRowid){ DbdataCursor *pCsr = (DbdataCursor*)pCursor; *pRowid = pCsr->iRowid; return SQLITE_OK; } /* ** Invoke this routine to register the "sqlite_dbdata" virtual table module */ static int sqlite3DbdataRegister(sqlite3 *db){ static sqlite3_module dbdata_module = { 0, /* iVersion */ 0, /* xCreate */ dbdataConnect, /* xConnect */ dbdataBestIndex, /* xBestIndex */ dbdataDisconnect, /* xDisconnect */ 0, /* xDestroy */ dbdataOpen, /* xOpen - open a cursor */ dbdataClose, /* xClose - close a cursor */ dbdataFilter, /* xFilter - configure scan constraints */ dbdataNext, /* xNext - advance a cursor */ dbdataEof, /* xEof - check for end of scan */ dbdataColumn, /* xColumn - read data */ dbdataRowid, /* xRowid - read data */ 0, /* xUpdate */ 0, /* xBegin */ 0, /* xSync */ 0, /* xCommit */ 0, /* xRollback */ 0, /* xFindMethod */ 0, /* xRename */ 0, /* xSavepoint */ 0, /* xRelease */ 0, /* xRollbackTo */ 0 /* xShadowName */ }; int rc = sqlite3_create_module(db, "sqlite_dbdata", &dbdata_module, 0); if( rc==SQLITE_OK ){ rc = sqlite3_create_module(db, "sqlite_dbptr", &dbdata_module, (void*)1); } return rc; } #ifdef _WIN32 __declspec(dllexport) #endif int sqlite3_dbdata_init( sqlite3 *db, char **pzErrMsg, const sqlite3_api_routines *pApi ){ SQLITE_EXTENSION_INIT2(pApi); (void)pzErrMsg; return sqlite3DbdataRegister(db); } #endif /* ifndef SQLITE_OMIT_VIRTUALTABLE */ |
Added ext/recover/recover1.test.
> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 | # 2022 August 28 # # The author disclaims copyright to this source code. In place of # a legal notice, here is a blessing: # # May you do good and not evil. # May you find forgiveness for yourself and forgive others. # May you share freely, never taking more than you give. # #*********************************************************************** # source [file join [file dirname [info script]] recover_common.tcl] set testprefix recover1 proc compare_result {db1 db2 sql} { set r1 [$db1 eval $sql] set r2 [$db2 eval $sql] if {$r1 != $r2} { puts "r1: $r1" puts "r2: $r2" error "mismatch for $sql" } return "" } proc compare_dbs {db1 db2} { compare_result $db1 $db2 "SELECT sql FROM sqlite_master ORDER BY 1" foreach tbl [$db1 eval {SELECT name FROM sqlite_master WHERE type='table'}] { compare_result $db1 $db2 "SELECT * FROM $tbl" } compare_result $db1 $db2 "PRAGMA page_size" compare_result $db1 $db2 "PRAGMA auto_vacuum" compare_result $db1 $db2 "PRAGMA encoding" compare_result $db1 $db2 "PRAGMA user_version" compare_result $db1 $db2 "PRAGMA application_id" } proc do_recover_test {tn} { forcedelete test.db2 forcedelete rstate.db uplevel [list do_test $tn.1 { set R [sqlite3_recover_init db main test.db2] $R config testdb rstate.db $R run $R finish } {}] sqlite3 db2 test.db2 uplevel [list do_test $tn.2 [list compare_dbs db db2] {}] db2 close forcedelete test.db2 forcedelete rstate.db uplevel [list do_test $tn.3 { set ::sqlhook [list] set R [sqlite3_recover_init_sql db main my_sql_hook] $R config testdb rstate.db $R config rowids 1 $R run $R finish } {}] sqlite3 db2 test.db2 execsql [join $::sqlhook ";"] db2 db2 close sqlite3 db2 test.db2 uplevel [list do_test $tn.4 [list compare_dbs db db2] {}] db2 close } proc my_sql_hook {sql} { lappend ::sqlhook $sql return 0 } do_execsql_test 1.0 { CREATE TABLE t1(a INTEGER PRIMARY KEY, b); CREATE TABLE t2(a INTEGER PRIMARY KEY, b) WITHOUT ROWID; WITH s(i) AS ( SELECT 1 UNION ALL SELECT i+1 FROM s WHERE i<10 ) INSERT INTO t1 SELECT i*2, hex(randomblob(250)) FROM s; INSERT INTO t2 SELECT * FROM t1; } do_recover_test 1 do_execsql_test 2.0 { ALTER TABLE t1 ADD COLUMN c DEFAULT 'xyz' } do_recover_test 2 do_execsql_test 3.0 { CREATE INDEX i1 ON t1(c); } do_recover_test 3 do_execsql_test 4.0 { CREATE VIEW v1 AS SELECT * FROM t2; } do_recover_test 4 do_execsql_test 5.0 { CREATE UNIQUE INDEX i2 ON t1(c, b); } do_recover_test 5 #-------------------------------------------------------------------------- # reset_db do_execsql_test 6.0 { CREATE TABLE t1( a INTEGER PRIMARY KEY, b INT, c TEXT, d INT GENERATED ALWAYS AS (a*abs(b)) VIRTUAL, e TEXT GENERATED ALWAYS AS (substr(c,b,b+1)) STORED, f TEXT GENERATED ALWAYS AS (substr(c,b,b+1)) STORED ); INSERT INTO t1 VALUES(1, 2, 'hello world'); } do_recover_test 6 do_execsql_test 7.0 { CREATE TABLE t2(i, j GENERATED ALWAYS AS (i+1) STORED, k); INSERT INTO t2 VALUES(10, 'ten'); } do_execsql_test 7.1 { SELECT * FROM t2 } {10 11 ten} do_recover_test 7.2 #-------------------------------------------------------------------------- # reset_db do_execsql_test 8.0 { CREATE TABLE x1(a INTEGER PRIMARY KEY AUTOINCREMENT, b, c); WITH s(i) AS ( SELECT 1 UNION ALL SELECT i+1 FROM s WHERE i<2 ) INSERT INTO x1(b, c) SELECT hex(randomblob(100)), hex(randomblob(100)) FROM s; CREATE INDEX x1b ON x1(b); CREATE INDEX x1cb ON x1(c, b); DELETE FROM x1 WHERE a>50; ANALYZE; } do_recover_test 8 #------------------------------------------------------------------------- reset_db ifcapable fts5 { do_execsql_test 9.1 { CREATE VIRTUAL TABLE ft5 USING fts5(a, b); INSERT INTO ft5 VALUES('hello', 'world'); } do_recover_test 9 } #------------------------------------------------------------------------- reset_db do_execsql_test 10.1 { CREATE TABLE x1(a PRIMARY KEY, str TEXT) WITHOUT ROWID; INSERT INTO x1 VALUES(1, ' \nhello\012world(\n0)(\n1) '); INSERT INTO x1 VALUES(2, ' \nhello '); } do_execsql_test 10.2 " INSERT INTO x1 VALUES(3, '\012hello there\015world'); INSERT INTO x1 VALUES(4, '\015hello there\015world'); " do_recover_test 10 #------------------------------------------------------------------------- reset_db do_execsql_test 11.1 { PRAGMA page_size = 4096; PRAGMA encoding='utf16'; PRAGMA auto_vacuum = 2; PRAGMA user_version = 45; PRAGMA application_id = 22; CREATE TABLE u1(u, v); INSERT INTO u1 VALUES('edvin marton', 'bond'); INSERT INTO u1 VALUES(1, 4.0); } do_execsql_test 11.1a { PRAGMA auto_vacuum; } {2} do_recover_test 11 do_test 12.1 { set R [sqlite3_recover_init db "" test.db2] $R config lostandfound "" $R config invalid xyz } {12} do_test 12.2 { $R run $R run } {0} do_test 12.3 { $R finish } {} #------------------------------------------------------------------------- reset_db file_control_reservebytes db 16 do_execsql_test 12.1 { PRAGMA auto_vacuum = 2; PRAGMA user_version = 45; PRAGMA application_id = 22; CREATE TABLE u1(u, v); CREATE UNIQUE INDEX i1 ON u1(u, v); INSERT INTO u1 VALUES(1, 2), (3, 4); CREATE TABLE u2(u, v); CREATE UNIQUE INDEX i2 ON u1(u, v); INSERT INTO u2 VALUES(hex(randomblob(500)), hex(randomblob(1000))); INSERT INTO u2 VALUES(hex(randomblob(500)), hex(randomblob(1000))); INSERT INTO u2 VALUES(hex(randomblob(500)), hex(randomblob(1000))); INSERT INTO u2 VALUES(hex(randomblob(50000)), hex(randomblob(20000))); } do_recover_test 12 #------------------------------------------------------------------------- reset_db sqlite3 db "" do_recover_test 13 do_execsql_test 14.1 { PRAGMA auto_vacuum = 2; PRAGMA user_version = 45; PRAGMA application_id = 22; CREATE TABLE u1(u, v); CREATE UNIQUE INDEX i1 ON u1(u, v); INSERT INTO u1 VALUES(1, 2), (3, 4); CREATE TABLE u2(u, v); CREATE UNIQUE INDEX i2 ON u1(u, v); INSERT INTO u2 VALUES(hex(randomblob(500)), hex(randomblob(1000))); INSERT INTO u2 VALUES(hex(randomblob(500)), hex(randomblob(1000))); INSERT INTO u2 VALUES(hex(randomblob(500)), hex(randomblob(1000))); INSERT INTO u2 VALUES(hex(randomblob(50000)), hex(randomblob(20000))); } do_recover_test 14 #------------------------------------------------------------------------- reset_db execsql { PRAGMA journal_mode=OFF; PRAGMA mmap_size=10; } do_execsql_test 15.1 { CREATE TABLE t1(x); } {} do_recover_test 15 #------------------------------------------------------------------------- reset_db do_execsql_test 16.1 { PRAGMA journal_mode = wal; CREATE TABLE t1(x); INSERT INTO t1 VALUES(1), (2), (3); } {wal} do_test 16.2 { set R [sqlite3_recover_init db main test.db2] $R run $R finish } {} do_execsql_test 16.3 { SELECT * FROM t1; } {1 2 3} do_execsql_test 16.4 { BEGIN; SELECT * FROM t1; } {1 2 3} do_test 16.5 { set R [sqlite3_recover_init db main test.db2] $R run list [catch { $R finish } msg] $msg } {1 {cannot start a transaction within a transaction}} do_execsql_test 16.6 { SELECT * FROM t1; } {1 2 3} do_execsql_test 16.7 { INSERT INTO t1 VALUES(4); } do_test 16.8 { set R [sqlite3_recover_init db main test.db2] $R run list [catch { $R finish } msg] $msg } {1 {cannot start a transaction within a transaction}} do_execsql_test 16.9 { SELECT * FROM t1; COMMIT; } {1 2 3 4} finish_test |
Added ext/recover/recover_common.tcl.
> > > > > > > > > > > > > > | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 | if {![info exists testdir]} { set testdir [file join [file dirname [info script]] .. .. test] } source $testdir/tester.tcl if {[info commands sqlite3_recover_init]==""} { finish_test return -code return } |
Added ext/recover/recoverclobber.test.
> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 | # 2019 April 23 # # The author disclaims copyright to this source code. In place of # a legal notice, here is a blessing: # # May you do good and not evil. # May you find forgiveness for yourself and forgive others. # May you share freely, never taking more than you give. # #*********************************************************************** # # Tests for the SQLITE_RECOVER_ROWIDS option. # source [file join [file dirname [info script]] recover_common.tcl] set testprefix recoverclobber proc recover {db output} { set R [sqlite3_recover_init db main test.db2] $R run $R finish } forcedelete test.db2 do_execsql_test 1.0 { ATTACH 'test.db2' AS aux; CREATE TABLE aux.x1(x, one); INSERT INTO x1 VALUES(1, 'one'), (2, 'two'), (3, 'three'); CREATE TABLE t1(a, b); INSERT INTO t1 VALUES(1, 1), (2, 2), (3, 3), (4, 4); DETACH aux; } breakpoint do_test 1.1 { recover db test.db2 } {} do_execsql_test 1.2 { ATTACH 'test.db2' AS aux; SELECT * FROM aux.t1; } {1 1 2 2 3 3 4 4} do_catchsql_test 1.3 { SELECT * FROM aux.x1; } {1 {no such table: aux.x1}} finish_test |
Added ext/recover/recovercorrupt.test.
> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 | # 2022 August 28 # # The author disclaims copyright to this source code. In place of # a legal notice, here is a blessing: # # May you do good and not evil. # May you find forgiveness for yourself and forgive others. # May you share freely, never taking more than you give. # #*********************************************************************** # source [file join [file dirname [info script]] recover_common.tcl] set testprefix recovercorrupt database_may_be_corrupt do_execsql_test 1.0 { PRAGMA page_size = 512; CREATE TABLE t1(a INTEGER PRIMARY KEY, b, c); INSERT INTO t1 VALUES(1, 2, 3); INSERT INTO t1 VALUES(2, hex(randomblob(100)), randomblob(200)); CREATE INDEX i1 ON t1(b, c); CREATE TABLE t2(a PRIMARY KEY, b, c) WITHOUT ROWID; INSERT INTO t2 VALUES(1, 2, 3); INSERT INTO t2 VALUES(2, hex(randomblob(100)), randomblob(200)); ANALYZE; PRAGMA writable_schema = 1; DELETE FROM sqlite_schema WHERE name='t2'; } do_test 1.1 { expr [file size test.db]>3072 } {1} proc toggle_bit {blob bit} { set byte [expr {$bit / 8}] set bit [expr {$bit & 0x0F}] binary scan $blob a${byte}ca* A x B set x [expr {$x ^ (1 << $bit)}] binary format a*ca* $A $x $B } db_save_and_close for {set ii 0} {$ii < 10000} {incr ii} { db_restore_and_reopen db func toggle_bit toggle_bit set bitsperpage [expr 512*8] set pg [expr {($ii / $bitsperpage)+1}] set byte [expr {$ii % $bitsperpage}] db eval { UPDATE sqlite_dbpage SET data = toggle_bit(data, $byte) WHERE pgno=$pg } set R [sqlite3_recover_init db main test.db2] $R config lostandfound lost_and_found $R run do_test 1.2.$ii { $R finish } {} } finish_test |
Added ext/recover/recovercorrupt2.test.
> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 | # 2022 August 28 # # The author disclaims copyright to this source code. In place of # a legal notice, here is a blessing: # # May you do good and not evil. # May you find forgiveness for yourself and forgive others. # May you share freely, never taking more than you give. # #*********************************************************************** # source [file join [file dirname [info script]] recover_common.tcl] set testprefix recovercorrupt2 do_execsql_test 1.0 { PRAGMA page_size = 512; CREATE TABLE t1(a INTEGER PRIMARY KEY, b, c); INSERT INTO t1 VALUES(1, 2, 3); INSERT INTO t1 VALUES(2, hex(randomblob(100)), randomblob(200)); CREATE INDEX i1 ON t1(b, c); CREATE TABLE t2(a PRIMARY KEY, b, c) WITHOUT ROWID; INSERT INTO t2 VALUES(1, 2, 3); INSERT INTO t2 VALUES(2, hex(randomblob(100)), randomblob(200)); ANALYZE; PRAGMA writable_schema = 1; UPDATE sqlite_schema SET sql = 'CREATE INDEX i1 ON o(world)' WHERE name='i1'; DELETE FROM sqlite_schema WHERE name='sqlite_stat4'; } do_test 1.1 { set R [sqlite3_recover_init db main test.db2] $R run $R finish } {} sqlite3 db2 test.db2 do_execsql_test -db db2 1.2 { SELECT sql FROM sqlite_schema } { {CREATE TABLE t1(a INTEGER PRIMARY KEY, b, c)} {CREATE TABLE t2(a PRIMARY KEY, b, c) WITHOUT ROWID} {CREATE TABLE sqlite_stat1(tbl,idx,stat)} } db2 close do_execsql_test 1.3 { PRAGMA writable_schema = 1; UPDATE sqlite_schema SET sql = 'CREATE TABLE t2 syntax error!' WHERE name='t2'; } do_test 1.4 { set R [sqlite3_recover_init db main test.db2] $R run $R finish } {} sqlite3 db2 test.db2 do_execsql_test -db db2 1.5 { SELECT sql FROM sqlite_schema } { {CREATE TABLE t1(a INTEGER PRIMARY KEY, b, c)} {CREATE TABLE sqlite_stat1(tbl,idx,stat)} } db2 close #------------------------------------------------------------------------- # reset_db do_test 2.0 { sqlite3 db {} db deserialize [decode_hexdb { | size 8192 pagesize 4096 filename x3.db | page 1 offset 0 | 0: 53 51 4c 69 74 65 20 66 6f 72 6d 61 74 20 33 00 SQLite format 3. | 16: 10 00 01 01 00 40 20 20 00 00 00 02 00 00 00 02 .....@ ........ | 32: 00 00 00 00 00 00 00 00 00 00 00 01 00 00 00 04 ................ | 48: 00 00 00 00 00 00 00 00 00 00 00 01 00 00 00 00 ................ | 80: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 02 ................ | 96: 00 2e 63 00 0d 00 00 00 01 0f d8 00 0f d8 00 00 ..c............. | 4048: 00 00 00 00 00 00 00 00 26 01 06 17 11 11 01 39 ........&......9 | 4064: 74 61 62 6c 65 74 31 74 31 02 43 52 45 41 54 45 tablet1t1.CREATE | 4080: 20 54 41 42 4c 45 20 74 31 28 61 2c 62 2c 63 29 TABLE t1(a,b,c) | page 2 offset 4096 | 0: 0d 00 00 00 01 0f ce 00 0f ce 00 00 00 00 00 00 ................ | 4032: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ff ff ..............(. | 4048: ff ff ff ff ff ff ff 28 04 27 25 23 61 61 61 61 .........'%#aaaa | 4064: 61 61 61 61 61 61 61 61 61 62 62 62 62 62 62 62 aaaaaaaaabbbbbbb | 4080: 62 62 62 62 62 63 63 63 63 63 63 63 63 63 63 63 bbbbbccccccccccc | end x3.db }]} {} do_test 2.1 { set R [sqlite3_recover_init db main test.db2] $R run $R finish } {} sqlite3 db2 test.db2 do_execsql_test -db db2 2.2 { SELECT sql FROM sqlite_schema } { {CREATE TABLE t1(a,b,c)} } do_execsql_test -db db2 2.3 { SELECT * FROM t1 } {} db2 close #------------------------------------------------------------------------- # reset_db do_test 3.0 { sqlite3 db {} db deserialize [decode_hexdb { .open --hexdb | size 4096 pagesize 1024 filename corrupt032.txt.db | page 1 offset 0 | 0: 53 51 4c 69 74 65 20 66 6f 72 6d 61 74 20 33 00 SQLite format 3. | 16: 04 00 01 01 08 40 20 20 00 00 00 02 00 00 00 03 .....@ ........ | 32: 00 00 00 00 00 00 00 00 00 00 00 01 00 00 00 04 ................ | 48: 00 00 00 00 00 00 00 00 00 00 00 01 00 00 00 00 ................ | 80: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 02 ................ | 96: 00 2e 24 80 0d 00 00 00 01 03 d4 00 03 d4 00 00 ..$............. | 976: 00 00 00 00 22 01 06 17 11 11 01 31 74 61 62 6c ...........1tabl | 992: 65 74 31 74 31 02 43 52 45 41 54 45 20 54 41 42 et1t1.CREATE TAB | 1008: 4c 45 20 74 31 28 78 29 00 00 00 00 00 00 00 00 LE t1(x)........ | page 2 offset 1024 | 0: 0d 00 00 00 01 02 06 00 02 06 00 00 00 00 00 00 ................ | 512: 00 00 00 00 00 00 8b 60 01 03 97 46 00 00 00 00 .......`...F.... | 1008: 00 00 00 00 00 00 00 03 00 00 00 00 00 00 00 00 ................ | end corrupt032.txt.db }]} {} do_test 3.1 { set R [sqlite3_recover_init db main test.db2] $R run $R finish } {} #------------------------------------------------------------------------- # reset_db do_test 4.0 { sqlite3 db {} db deserialize [decode_hexdb { .open --hexdb | size 4096 pagesize 4096 filename crash-00f2d3627f1b43.db | page 1 offset 0 | 0: 53 51 4c 69 74 65 20 66 6f 72 6d 61 74 20 33 00 SQLite format 3. | 16: 00 01 01 02 00 40 20 20 01 00 ff 00 42 01 10 01 .....@ ....B... | 32: ef 00 00 87 00 ff ff ff f0 01 01 10 ff ff 00 00 ................ | end crash-00f2d3627f1b43.db }]} {} do_test 4.1 { set R [sqlite3_recover_init db main test.db2] catch { $R run } list [catch { $R finish } msg] $msg } {1 {unable to open database file}} #------------------------------------------------------------------------- # reset_db do_test 5.0 { sqlite3 db {} db deserialize [decode_hexdb { .open --hexdb | size 16384 pagesize 4096 filename crash-7b75760a4c5f15.db | page 1 offset 0 | 0: 53 51 4c 69 74 65 20 66 6f 72 6d 61 74 20 33 00 SQLite format 3. | 16: 10 00 01 01 00 40 20 20 00 00 00 00 00 00 00 04 .....@ ........ | 32: 00 00 00 00 00 00 00 00 00 00 00 03 00 00 00 00 ................ | 96: 00 00 00 00 0d 00 00 00 03 0f 4e 00 0f bc 0f 90 ..........N..... | 112: 0f 4e 00 00 00 00 00 00 00 00 00 00 00 00 00 00 .N.............. | 3904: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 40 03 ..............@. | 3920: 06 17 11 11 01 6d 74 61 62 6c 65 74 32 74 32 04 .....mtablet2t2. | 3936: 43 52 45 41 54 45 20 54 41 42 4c 45 20 74 32 28 CREATE TABLE t2( | 3952: 78 2c 79 2c 7a 20 50 52 49 4d 41 52 59 20 4b 45 x,y,z PRIMARY KE | 3968: 59 29 20 57 49 54 48 4f 55 54 20 52 4f 57 49 44 Y) WITHOUT ROWID | 3984: 2a 02 06 17 13 11 01 3f 69 6e 64 65 78 74 31 61 *......?indext1a | 4000: 74 31 03 43 52 45 41 54 45 20 49 4e 44 45 58 20 t1.CREATE INDEX | 4016: 74 31 61 20 4f 4e 20 74 31 28 61 29 42 01 06 17 t1a ON t1(a)B... | 4032: 11 11 01 71 74 61 62 6c 65 74 31 74 31 02 43 52 ...qtablet1t1.CR | 4048: 45 41 54 45 20 54 41 42 4c 45 20 74 31 28 61 20 EATE TABLE t1(a | 4064: 49 4e 54 2c 62 20 54 45 58 54 2c 63 20 42 4c 4f INT,b TEXT,c BLO | 4080: 42 2c 64 20 52 45 41 4c 29 20 53 54 52 49 43 54 B,d REAL) STRICT | page 2 offset 4096 | 0: 0d 00 00 00 14 0c ae 00 0f df 0f bd 0f 9a 0f 76 ...............v | 16: 0f 51 0f 2b 0f 04 0e dc 0e b3 0e 89 0e 5e 0e 32 .Q.+.........^.2 | 32: 0e 05 0d 1a 0d a8 0d 78 0d 47 0d 15 0c e2 00 00 .......x.G...... | 3232: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 32 14 ..............2. | 3248: 05 06 3f 34 07 15 f4 c9 23 af e2 b3 b6 61 62 63 ..?4....#....abc | 3264: 30 32 30 78 79 7a 01 00 00 00 00 00 00 00 00 00 020xyz.......... | 3280: 00 00 00 00 00 00 00 00 00 00 c3 b0 96 7e fb 4e .............~.N | 3296: c5 4c 31 13 05 06 1f 32 07 dd f2 2a a5 7e b2 4d .L1....2...*.~.M | 3312: 82 61 62 63 30 31 39 78 79 7a 01 00 00 00 00 00 .abc019xyz...... | 3328: 00 00 00 00 00 00 00 00 00 00 00 00 00 c3 a3 d6 ................ | 3344: e9 f1 c2 fd f3 30 12 05 06 1f 30 07 8f 8f f5 c4 .....0....0..... | 3360: 35 b6 7f 8d 61 62 63 30 31 38 00 00 00 00 00 00 5...abc018...... | 3376: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 43 ...............C | 3392: b2 13 1f 9d 56 8a 47 21 b1 05 06 1f 2e 07 7f 46 ....V.G!.......F | 3408: 91 03 3f 97 fb f7 61 62 63 30 00 00 00 00 00 00 ..?...abc0...... | 3440: c3 bb d8 96 86 c2 e8 2b 2e 10 05 06 1f 2c 07 6d .......+.....,.m | 3456: 85 7b ce d0 32 d2 54 61 62 63 30 00 00 00 00 00 ....2.Tabc0..... | 3488: 43 a1 eb 44 14 dc 03 7b 2d 0f 05 06 1f 2a 07 d9 C..D....-....*.. | 3504: ab ec bf 34 51 70 f3 61 62 63 30 31 35 78 79 7a ...4Qp.abc015xyz | 3520: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 c3 ................ | 3536: b6 3d f4 46 b1 6a af 2c 0e 05 06 1f 28 07 36 75 .=.F.j.,....(.6u | 3552: e9 a2 bd 05 04 ea 61 62 63 30 31 34 78 79 7a 00 ......abc014xyz. | 3568: 00 00 00 00 00 00 00 00 00 00 00 00 00 c3 ab 23 ...............# | 3584: a7 6a 34 ca f8 2b 0d 05 06 1f 26 07 48 45 ab e0 .j4..+....&.HE.. | 3600: 8c 7c ff 0c 61 62 63 30 31 33 78 79 7a 00 00 00 .|..abc013xyz... | 3616: 00 00 00 00 0d d0 00 00 00 00 43 b8 d3 93 f4 92 ..........C..... | 3632: 5b 7a 2a 0c 05 06 1f 24 07 be 6d 1e db 61 5d 80 [z*....$..m..a]. | 3648: 9f 61 62 63 30 31 32 78 79 7a 00 00 00 00 00 00 .abc012xyz...... | 3664: 00 00 00 00 00 00 43 b5 a1 a4 af 7b c6 60 29 0b ......C......`). | 3680: 05 06 1f 22 07 6e a2 a3 64 68 d4 a6 bd 61 62 63 .....n..dh...abc | 3696: 30 31 31 78 79 7a 00 00 00 00 00 00 00 00 00 00 011xyz.......... | 3712: 00 c3 c4 1e ff 0f fc e6 ff 28 0a 05 06 1f 20 07 .........(.... . | 3728: 50 f9 4a bb a5 7a 1e ca 61 62 63 30 31 30 78 79 P.J..z..abc010xy | 3744: 7a 00 00 00 00 00 00 00 00 00 00 c3 a7 90 ed d9 z............... | 3760: 5c 2c d5 27 09 05 06 1f 1e 07 90 8e 1d d9 1c 3a .,.'...........: | 3776: e8 c1 61 62 63 30 30 39 78 79 7a 00 00 00 00 00 ..abc009xyz..... | 3792: 00 00 00 00 43 a7 97 87 cf b0 ff 79 26 08 05 06 ....C......y&... | 3808: 1f 1c 07 86 65 f6 7c 50 7a 2c 76 61 62 63 30 30 ....e.|Pz,vabc00 | 3824: 38 78 79 7a 00 00 00 00 00 00 00 00 c3 b0 e3 4c 8xyz...........L | 3840: 4f d3 41 b5 25 07 05 06 1f 1a 07 8b 20 e5 68 11 O.A.%....... .h. | 3856: 13 55 87 61 62 63 30 30 37 78 79 7a 00 00 00 00 .U.abc007xyz.... | 3872: 00 00 00 c3 b6 a3 74 f1 9c 33 f8 24 06 05 06 1f ......t..3.$.... | 3888: 18 07 97 3c bc 34 49 94 54 ab 61 62 63 30 30 36 ...<.4I.T.abc006 | 3904: 78 79 7a 00 00 00 00 00 00 c3 88 00 c2 ca 4c 4d xyz...........LM | 3920: d3 23 05 05 06 1f 16 07 59 37 11 10 e9 e5 3d d5 .#......Y7....=. | 3936: 61 62 63 30 30 35 78 79 7a 00 00 00 00 00 c3 c0 abc005xyz....... | 3952: 15 12 67 ed 4b 79 22 04 05 06 1f 14 07 93 39 01 ..g.Ky........9. | 3968: 7f b8 c7 99 58 61 62 63 30 30 34 78 79 7a 00 00 ....Xabc004xyz.. | 3984: 09 c0 43 bf e0 e7 6d 70 fd 61 21 03 05 06 1f 12 ..C...mp.a!..... | 4000: 07 b6 df 8d 8b 27 08 22 5a 61 62 63 30 30 33 78 .....'..Zabc003x | 4016: 79 7a 00 00 00 c3 c7 ea 0f dc dd 32 22 20 02 05 yz.........2. .. | 4032: 06 1f 10 07 2f a6 da 71 df 66 b3 b5 61 62 63 30 ..../..q.f..abc0 | 4048: 30 32 78 79 7a 00 00 c3 ce d9 8d e9 ec 20 45 1f 02xyz........ E. | 4064: 01 05 06 1f 0e 07 5a 47 53 20 3b 48 8f c0 61 62 ......ZGS ;H..ab | 4080: 63 30 30 31 78 79 7a 00 c3 c9 e6 81 f8 d9 24 04 c001xyz.......$. | page 3 offset 8192 | 0: 0a 00 00 00 14 0e fd 00 0f f3 0f e6 0f d9 0f cc ................ | 16: 0f bf 0f b2 0f a5 0f 98 0f 8b 0f 7e 0f 71 0f 64 ...........~.q.d | 32: 0f 57 0f 4a 0f 3d 0f 30 0f 24 00 00 00 00 00 00 .W.J.=.0.$...... | 3824: 00 00 00 00 00 00 00 00 00 00 00 00 00 0c 03 06 ................ | 3840: 01 7f 46 91 03 3f 97 fb f7 11 0c 03 06 01 6e a2 ..F..?........n. | 3856: a3 64 68 d4 a6 bd 0b 0c 03 06 01 6d 85 7b ce d0 .dh........m.... | 3872: 32 d2 54 10 0b 03 06 09 5a 47 53 20 3b 48 8f c0 2.T.....ZGS ;H.. | 3888: 0c 03 06 01 59 37 11 10 e9 e5 3d d5 05 0c 03 06 ....Y7....=..... | 3904: 01 50 f9 4a bb a5 7a 1e ca 0a 0c 03 06 01 48 45 .P.J..z.......HE | 3920: ab e0 8c 7c ff 0c 0d 0c 03 06 01 36 75 e9 a2 bd ...|.......6u... | 3936: 05 04 ea 0e 0c 03 06 01 2f a6 da 71 df 66 b3 b5 ......../..q.f.. | 3952: 02 0c 03 06 01 15 f4 c9 23 af e2 b3 b6 14 0c 03 ........#....... | 3968: 06 01 dd f2 2a a5 7e b2 4d 82 13 0c 03 06 01 d9 ....*.~.M....... | 3984: ab ec bf 34 51 70 f3 0f 0c 03 06 01 be 6d 1e db ...4Qp.......m.. | 4000: 61 5d 80 9f 0c 0c 03 06 01 b6 df 8d 8b 27 08 22 a]...........'.. | 4016: 5a 03 0c 03 06 01 97 3c bc 34 49 94 54 ab 06 0c Z......<.4I.T... | 4032: 03 06 01 93 39 01 7f b8 c7 99 58 04 0c 03 06 01 ....9.....X..... | 4048: 90 8e 1d d9 1c 3a e8 c1 09 0c 03 06 01 8f 8f f5 .....:.......... | 4064: c4 35 b6 7f 8d 12 0c 03 06 01 8b 20 e5 68 11 13 .5......... .h.. | 4080: 55 87 07 0c 03 06 01 86 65 f6 7c 50 7a 2b 06 08 U.......e.|Pz+.. | page 4 offset 12288 | 0: 0a 00 00 00 14 0f 62 00 0f 7a 0f a1 0f c9 0f d9 ......b..z...... | 16: 0f 81 0f d1 0f f1 0f f9 0f e1 0f 89 0e 6a 0f c1 .............j.. | 32: 0f 91 0f 99 0f b9 0f 72 0f 62 0f e9 0f b1 0f a9 .......r.b...... | 3936: 00 00 07 04 01 01 01 11 0e 9e 07 04 01 01 01 0b ................ | 3952: 31 16 07 04 01 01 01 10 37 36 06 04 09 01 01 ab 1.......76...... | 3968: 58 07 04 01 01 01 05 1c 28 07 04 01 01 01 0a 10 X.......(....... | 3984: cf 07 04 01 01 01 0d b2 e3 07 04 01 01 01 0e d3 ................ | 4000: f2 07 04 01 01 01 02 41 ad 07 04 01 01 01 14 3e .......A.......> | 4016: 22 07 04 01 01 01 13 27 45 07 04 01 01 01 0f ad .......'E....... | 4032: dd 07 04 01 01 01 0c 2e a1 07 04 01 01 01 03 df ................ | 4048: e1 07 04 01 01 01 06 59 a7 07 04 01 01 01 04 27 .......Y.......' | 4064: bd 07 04 01 01 01 09 d0 e0 07 04 01 01 01 12 39 ...............9 | 4080: 4f 07 04 01 01 01 07 c4 11 06 04 00 00 00 00 00 O............... | end crash-7b75760a4c5f15.db }]} {} do_test 5.1 { set R [sqlite3_recover_init db main test.db2] catch { $R run } list [catch { $R finish } msg] $msg } {0 {}} finish_test |
Added ext/recover/recoverfault.test.
> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 | # 2022 August 28 # # The author disclaims copyright to this source code. In place of # a legal notice, here is a blessing: # # May you do good and not evil. # May you find forgiveness for yourself and forgive others. # May you share freely, never taking more than you give. # #*********************************************************************** # source [file join [file dirname [info script]] recover_common.tcl] set testprefix recoverfault #-------------------------------------------------------------------------- proc compare_result {db1 db2 sql} { set r1 [$db1 eval $sql] set r2 [$db2 eval $sql] if {$r1 != $r2} { puts "r1: $r1" puts "r2: $r2" error "mismatch for $sql" } return "" } proc compare_dbs {db1 db2} { compare_result $db1 $db2 "SELECT sql FROM sqlite_master ORDER BY 1" foreach tbl [$db1 eval {SELECT name FROM sqlite_master WHERE type='table'}] { compare_result $db1 $db2 "SELECT * FROM $tbl" } } #-------------------------------------------------------------------------- do_execsql_test 1.0 { CREATE TABLE t1(a INTEGER PRIMARY KEY, b, c); INSERT INTO t1 VALUES(1, 2, 3); INSERT INTO t1 VALUES(2, hex(randomblob(1000)), randomblob(2000)); CREATE INDEX i1 ON t1(b, c); ANALYZE; } faultsim_save_and_close do_faultsim_test 1 -faults oom* -prep { catch { db2 close } faultsim_restore_and_reopen } -body { set R [sqlite3_recover_init db main test.db2] $R run $R finish } -test { faultsim_test_result {0 {}} {1 {}} if {$testrc==0} { sqlite3 db2 test.db2 compare_dbs db db2 db2 close } } faultsim_restore_and_reopen do_execsql_test 2.0 { CREATE TABLE t2(a INTEGER PRIMARY KEY, b, c); INSERT INTO t2 VALUES(1, 2, 3); INSERT INTO t2 VALUES(2, hex(randomblob(1000)), hex(randomblob(2000))); PRAGMA writable_schema = 1; DELETE FROM sqlite_schema WHERE name='t2'; } faultsim_save_and_close do_faultsim_test 2 -faults oom* -prep { faultsim_restore_and_reopen } -body { set R [sqlite3_recover_init db main test.db2] $R config lostandfound lost_and_found $R run $R finish } -test { faultsim_test_result {0 {}} {1 {}} } finish_test |
Added ext/recover/recoverfault2.test.
> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 | # 2022 August 28 # # The author disclaims copyright to this source code. In place of # a legal notice, here is a blessing: # # May you do good and not evil. # May you find forgiveness for yourself and forgive others. # May you share freely, never taking more than you give. # #*********************************************************************** # source [file join [file dirname [info script]] recover_common.tcl] set testprefix recoverfault2 #-------------------------------------------------------------------------- proc compare_result {db1 db2 sql} { set r1 [$db1 eval $sql] set r2 [$db2 eval $sql] if {$r1 != $r2} { puts "r1: $r1" puts "r2: $r2" error "mismatch for $sql" } return "" } proc compare_dbs {db1 db2} { compare_result $db1 $db2 "SELECT sql FROM sqlite_master ORDER BY 1" foreach tbl [$db1 eval {SELECT name FROM sqlite_master WHERE type='table'}] { compare_result $db1 $db2 "SELECT * FROM $tbl" } } #-------------------------------------------------------------------------- do_execsql_test 1.0 " CREATE TABLE t1(a INTEGER PRIMARY KEY, b); INSERT INTO t1 VALUES(2, '\012hello\015world\012today\n'); " faultsim_save_and_close proc my_sql_hook {sql} { lappend ::lSql $sql return 0 } do_faultsim_test 1 -faults oom* -prep { catch { db2 close } faultsim_restore_and_reopen set ::lSql [list] } -body { set R [sqlite3_recover_init_sql db main my_sql_hook] $R run $R finish } -test { faultsim_test_result {0 {}} {1 {}} if {$testrc==0} { sqlite3 db2 "" db2 eval [join $::lSql ";"] compare_dbs db db2 db2 close } } ifcapable utf16 { reset_db do_execsql_test 2.0 " PRAGMA encoding='utf-16'; CREATE TABLE t1(a INTEGER PRIMARY KEY, b); INSERT INTO t1 VALUES(2, '\012hello\015world\012today\n'); " faultsim_save_and_close proc my_sql_hook {sql} { lappend ::lSql $sql return 0 } do_faultsim_test 2 -faults oom-t* -prep { catch { db2 close } faultsim_restore_and_reopen set ::lSql [list] } -body { set R [sqlite3_recover_init_sql db main my_sql_hook] $R run $R finish } -test { faultsim_test_result {0 {}} {1 {}} if {$testrc==0} { sqlite3 db2 "" db2 eval [join $::lSql ";"] compare_dbs db db2 db2 close } } } finish_test |
Added ext/recover/recoverold.test.
> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 | # 2019 April 23 # # The author disclaims copyright to this source code. In place of # a legal notice, here is a blessing: # # May you do good and not evil. # May you find forgiveness for yourself and forgive others. # May you share freely, never taking more than you give. # #*********************************************************************** # # source [file join [file dirname [info script]] recover_common.tcl] set testprefix recoverold proc compare_result {db1 db2 sql} { set r1 [$db1 eval $sql] set r2 [$db2 eval $sql] if {$r1 != $r2} { puts "sql: $sql" puts "r1: $r1" puts "r2: $r2" error "mismatch for $sql" } return "" } proc compare_dbs {db1 db2} { compare_result $db1 $db2 "SELECT sql FROM sqlite_master ORDER BY 1" foreach tbl [$db1 eval {SELECT name FROM sqlite_master WHERE type='table'}] { compare_result $db1 $db2 "SELECT * FROM $tbl" } } proc do_recover_test {tn {tsql {}} {res {}}} { forcedelete test.db2 forcedelete rstate.db set R [sqlite3_recover_init db main test.db2] $R config lostandfound lost_and_found $R run $R finish sqlite3 db2 test.db2 if {$tsql==""} { uplevel [list do_test $tn.1 [list compare_dbs db db2] {}] } else { uplevel [list do_execsql_test -db db2 $tn.1 $tsql $res] } db2 close forcedelete test.db2 forcedelete rstate.db set ::sqlhook [list] set R [sqlite3_recover_init_sql db main my_sql_hook] $R config lostandfound lost_and_found $R run $R finish sqlite3 db2 test.db2 db2 eval [join $::sqlhook ";"] db cache flush if {$tsql==""} { compare_dbs db db2 uplevel [list do_test $tn.sql [list compare_dbs db db2] {}] } else { uplevel [list do_execsql_test -db db2 $tn.sql $tsql $res] } db2 close } proc my_sql_hook {sql} { lappend ::sqlhook $sql return 0 } set doc { hello world } do_execsql_test 1.1.1 { CREATE TABLE t1(a INTEGER PRIMARY KEY, b, c); INSERT INTO t1 VALUES(1, 4, X'1234567800'); INSERT INTO t1 VALUES(2, 'test', 8.1); INSERT INTO t1 VALUES(3, $doc, 8.4); } do_recover_test 1.1.2 do_execsql_test 1.2.1 " DELETE FROM t1; INSERT INTO t1 VALUES(13, 'hello\r\nworld', 13); " do_recover_test 1.2.2 do_execsql_test 1.3.1 " CREATE TABLE t2(i INTEGER PRIMARY KEY AUTOINCREMENT, b, c); INSERT INTO t2 VALUES(NULL, 1, 2); INSERT INTO t2 VALUES(NULL, 3, 4); INSERT INTO t2 VALUES(NULL, 5, 6); CREATE TABLE t3(i INTEGER PRIMARY KEY AUTOINCREMENT, b, c); INSERT INTO t3 VALUES(NULL, 1, 2); INSERT INTO t3 VALUES(NULL, 3, 4); INSERT INTO t3 VALUES(NULL, 5, 6); DELETE FROM t2; " do_recover_test 1.3.2 #------------------------------------------------------------------------- reset_db do_execsql_test 2.1.0 { PRAGMA auto_vacuum = 0; CREATE TABLE t1(a, b, c, PRIMARY KEY(b, c)) WITHOUT ROWID; INSERT INTO t1 VALUES(1, 2, 3); INSERT INTO t1 VALUES(4, 5, 6); INSERT INTO t1 VALUES(7, 8, 9); } do_recover_test 2.1.1 do_execsql_test 2.2.0 { PRAGMA writable_schema = 1; DELETE FROM sqlite_master WHERE name='t1'; } do_recover_test 2.2.1 { SELECT name FROM sqlite_master } {lost_and_found} do_execsql_test 2.3.0 { CREATE TABLE lost_and_found(a, b, c); } do_recover_test 2.3.1 { SELECT name FROM sqlite_master } {lost_and_found lost_and_found_0} do_execsql_test 2.4.0 { CREATE TABLE lost_and_found_0(a, b, c); } do_recover_test 2.4.1 { SELECT name FROM sqlite_master; SELECT * FROM lost_and_found_1; } {lost_and_found lost_and_found_0 lost_and_found_1 2 2 3 {} 2 3 1 2 2 3 {} 5 6 4 2 2 3 {} 8 9 7 } do_execsql_test 2.5 { CREATE TABLE x1(a, b, c); WITH s(i) AS ( SELECT 1 UNION ALL SELECT i+1 FROM s WHERE i<100 ) INSERT INTO x1 SELECT i, i, hex(randomblob(500)) FROM s; DROP TABLE x1; } do_recover_test 2.5.1 { SELECT name FROM sqlite_master; SELECT * FROM lost_and_found_1; } {lost_and_found lost_and_found_0 lost_and_found_1 2 2 3 {} 2 3 1 2 2 3 {} 5 6 4 2 2 3 {} 8 9 7 } ifcapable !secure_delete { do_test 2.6 { forcedelete test.db2 set R [sqlite3_recover_init db main test.db2] $R config lostandfound lost_and_found $R config freelistcorrupt 1 $R run $R finish sqlite3 db2 test.db2 execsql { SELECT count(*) FROM lost_and_found_1; } db2 } {103} db2 close } #------------------------------------------------------------------------- breakpoint reset_db do_recover_test 3.0 finish_test |
Added ext/recover/recoverpgsz.test.
> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 | # 2022 October 14 # # The author disclaims copyright to this source code. In place of # a legal notice, here is a blessing: # # May you do good and not evil. # May you find forgiveness for yourself and forgive others. # May you share freely, never taking more than you give. # #*********************************************************************** # source [file join [file dirname [info script]] recover_common.tcl] db close sqlite3_test_control_pending_byte 0x1000000 set testprefix recoverpgsz foreach {pgsz bOverflow} { 512 0 1024 0 2048 0 4096 0 8192 0 16384 0 32768 0 65536 0 512 1 1024 1 2048 1 4096 1 8192 1 16384 1 32768 1 65536 1 } { reset_db execsql "PRAGMA page_size = $pgsz" execsql "PRAGMA auto_vacuum = 0" do_execsql_test 1.$pgsz.$bOverflow.1 { CREATE TABLE t1(a, b, c); CREATE INDEX i1 ON t1(b, a, c); INSERT INTO t1(a, b) VALUES(1, 2), (3, 4), (5, 6); DELETE FROM t1 WHERE a=3; } if {$bOverflow} { do_execsql_test 1.$pgsz.$bOverflow.1a { UPDATE t1 SET c = randomblob(100000); } } db close set fd [open test.db] fconfigure $fd -encoding binary -translation binary seek $fd $pgsz set pg1 [read $fd $pgsz] set pg2 [read $fd $pgsz] close $fd set fd2 [open test.db2 w] fconfigure $fd2 -encoding binary -translation binary seek $fd2 $pgsz puts -nonewline $fd2 $pg1 close $fd2 sqlite3 db2 test.db2 do_test 1.$pgsz.$bOverflow.2 { set R [sqlite3_recover_init db2 main test.db3] $R run $R finish } {} sqlite3 db3 test.db3 do_test 1.$pgsz.$bOverflow.3 { db3 eval { SELECT * FROM sqlite_schema } db3 eval { PRAGMA page_size } } $pgsz db2 close db3 close forcedelete test.db3 forcedelete test.db2 set fd2 [open test.db2 w] fconfigure $fd2 -encoding binary -translation binary seek $fd2 $pgsz puts -nonewline $fd2 $pg2 close $fd2 sqlite3 db2 test.db2 do_test 1.$pgsz.$bOverflow.4 { set R [sqlite3_recover_init db2 main test.db3] $R run $R finish } {} sqlite3 db3 test.db3 do_test 1.$pgsz.$bOverflow.5 { db3 eval { SELECT * FROM sqlite_schema } db3 eval { PRAGMA page_size } } $pgsz db2 close db3 close } finish_test |
Added ext/recover/recoverrowid.test.
> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 | # 2022 September 07 # # The author disclaims copyright to this source code. In place of # a legal notice, here is a blessing: # # May you do good and not evil. # May you find forgiveness for yourself and forgive others. # May you share freely, never taking more than you give. # #*********************************************************************** # # Tests for the SQLITE_RECOVER_ROWIDS option. # source [file join [file dirname [info script]] recover_common.tcl] set testprefix recoverrowid proc recover {db bRowids output} { forcedelete $output set R [sqlite3_recover_init db main test.db2] $R config rowids $bRowids $R run $R finish } do_execsql_test 1.0 { CREATE TABLE t1(a, b); INSERT INTO t1 VALUES(1, 1), (2, 2), (3, 3), (4, 4); DELETE FROM t1 WHERE a IN (1, 3); } do_test 1.1 { recover db 0 test.db2 sqlite3 db2 test.db2 execsql { SELECT rowid, a, b FROM t1 ORDER BY rowid} db2 } {1 2 2 2 4 4} do_test 1.2 { db2 close recover db 1 test.db2 sqlite3 db2 test.db2 execsql { SELECT rowid, a, b FROM t1 ORDER BY rowid} db2 } {2 2 2 4 4 4} db2 close finish_test |
Added ext/recover/recoverslowidx.test.
> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 | # 2022 September 25 # # The author disclaims copyright to this source code. In place of # a legal notice, here is a blessing: # # May you do good and not evil. # May you find forgiveness for yourself and forgive others. # May you share freely, never taking more than you give. # #*********************************************************************** # # Tests for the SQLITE_RECOVER_SLOWINDEXES option. # source [file join [file dirname [info script]] recover_common.tcl] set testprefix recoverslowidx do_execsql_test 1.0 { PRAGMA auto_vacuum = 0; CREATE TABLE t1(a, b); CREATE INDEX i1 ON t1(a); INSERT INTO t1 VALUES(1, 1), (2, 2), (3, 3), (4, 4); } proc my_sql_hook {sql} { lappend ::lSql $sql return 0 } do_test 1.1 { set lSql [list] set R [sqlite3_recover_init_sql db main my_sql_hook] while {[$R step]==0} { } $R finish } {} do_test 1.2 { set lSql } [list {*}{ {BEGIN} {PRAGMA writable_schema = on} {PRAGMA encoding = 'UTF-8'} {PRAGMA page_size = '1024'} {PRAGMA auto_vacuum = '0'} {PRAGMA user_version = '0'} {PRAGMA application_id = '0'} {CREATE TABLE t1(a, b)} {INSERT OR IGNORE INTO 't1'(_rowid_, 'a', 'b') VALUES (1, 1, 1)} {INSERT OR IGNORE INTO 't1'(_rowid_, 'a', 'b') VALUES (2, 2, 2)} {INSERT OR IGNORE INTO 't1'(_rowid_, 'a', 'b') VALUES (3, 3, 3)} {INSERT OR IGNORE INTO 't1'(_rowid_, 'a', 'b') VALUES (4, 4, 4)} {CREATE INDEX i1 ON t1(a)} {PRAGMA writable_schema = off} {COMMIT} }] do_test 1.3 { set lSql [list] set R [sqlite3_recover_init_sql db main my_sql_hook] $R config slowindexes 1 while {[$R step]==0} { } $R finish } {} do_test 1.4 { set lSql } [list {*}{ {BEGIN} {PRAGMA writable_schema = on} {PRAGMA encoding = 'UTF-8'} {PRAGMA page_size = '1024'} {PRAGMA auto_vacuum = '0'} {PRAGMA user_version = '0'} {PRAGMA application_id = '0'} {CREATE TABLE t1(a, b)} {CREATE INDEX i1 ON t1(a)} {INSERT OR IGNORE INTO 't1'(_rowid_, 'a', 'b') VALUES (1, 1, 1)} {INSERT OR IGNORE INTO 't1'(_rowid_, 'a', 'b') VALUES (2, 2, 2)} {INSERT OR IGNORE INTO 't1'(_rowid_, 'a', 'b') VALUES (3, 3, 3)} {INSERT OR IGNORE INTO 't1'(_rowid_, 'a', 'b') VALUES (4, 4, 4)} {PRAGMA writable_schema = off} {COMMIT} }] finish_test |
Added ext/recover/recoversql.test.
> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 | # 2022 September 13 # # The author disclaims copyright to this source code. In place of # a legal notice, here is a blessing: # # May you do good and not evil. # May you find forgiveness for yourself and forgive others. # May you share freely, never taking more than you give. # #*********************************************************************** # # source [file join [file dirname [info script]] recover_common.tcl] set testprefix recoversql do_execsql_test 1.0 { CREATE TABLE "x.1" (x, y); INSERT INTO "x.1" VALUES(1, 1), (2, 2), (3, 3); CREATE INDEX "i.1" ON "x.1"(y, x); } proc sql_hook {sql} { incr ::iSqlHook if {$::iSqlHook==$::sql_hook_cnt} { return 4 } return 0 } do_test 1.1 { set ::sql_hook_cnt -1 set ::iSqlHook 0 set R [sqlite3_recover_init_sql db main sql_hook] $R run $R finish } {} set nSqlCall $iSqlHook for {set ii 1} {$ii<$nSqlCall} {incr ii} { set iSqlHook 0 set sql_hook_cnt $ii do_test 1.$ii.a { set R [sqlite3_recover_init_sql db main sql_hook] $R run } {1} do_test 1.$ii.b { list [catch { $R finish } msg] $msg } {1 {callback returned an error - 4}} } finish_test |
Added ext/recover/sqlite3recover.c.
> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340 341 342 343 344 345 346 347 348 349 350 351 352 353 354 355 356 357 358 359 360 361 362 363 364 365 366 367 368 369 370 371 372 373 374 375 376 377 378 379 380 381 382 383 384 385 386 387 388 389 390 391 392 393 394 395 396 397 398 399 400 401 402 403 404 405 406 407 408 409 410 411 412 413 414 415 416 417 418 419 420 421 422 423 424 425 426 427 428 429 430 431 432 433 434 435 436 437 438 439 440 441 442 443 444 445 446 447 448 449 450 451 452 453 454 455 456 457 458 459 460 461 462 463 464 465 466 467 468 469 470 471 472 473 474 475 476 477 478 479 480 481 482 483 484 485 486 487 488 489 490 491 492 493 494 495 496 497 498 499 500 501 502 503 504 505 506 507 508 509 510 511 512 513 514 515 516 517 518 519 520 521 522 523 524 525 526 527 528 529 530 531 532 533 534 535 536 537 538 539 540 541 542 543 544 545 546 547 548 549 550 551 552 553 554 555 556 557 558 559 560 561 562 563 564 565 566 567 568 569 570 571 572 573 574 575 576 577 578 579 580 581 582 583 584 585 586 587 588 589 590 591 592 593 594 595 596 597 598 599 600 601 602 603 604 605 606 607 608 609 610 611 612 613 614 615 616 617 618 619 620 621 622 623 624 625 626 627 628 629 630 631 632 633 634 635 636 637 638 639 640 641 642 643 644 645 646 647 648 649 650 651 652 653 654 655 656 657 658 659 660 661 662 663 664 665 666 667 668 669 670 671 672 673 674 675 676 677 678 679 680 681 682 683 684 685 686 687 688 689 690 691 692 693 694 695 696 697 698 699 700 701 702 703 704 705 706 707 708 709 710 711 712 713 714 715 716 717 718 719 720 721 722 723 724 725 726 727 728 729 730 731 732 733 734 735 736 737 738 739 740 741 742 743 744 745 746 747 748 749 750 751 752 753 754 755 756 757 758 759 760 761 762 763 764 765 766 767 768 769 770 771 772 773 774 775 776 777 778 779 780 781 782 783 784 785 786 787 788 789 790 791 792 793 794 795 796 797 798 799 800 801 802 803 804 805 806 807 808 809 810 811 812 813 814 815 816 817 818 819 820 821 822 823 824 825 826 827 828 829 830 831 832 833 834 835 836 837 838 839 840 841 842 843 844 845 846 847 848 849 850 851 852 853 854 855 856 857 858 859 860 861 862 863 864 865 866 867 868 869 870 871 872 873 874 875 876 877 878 879 880 881 882 883 884 885 886 887 888 889 890 891 892 893 894 895 896 897 898 899 900 901 902 903 904 905 906 907 908 909 910 911 912 913 914 915 916 917 918 919 920 921 922 923 924 925 926 927 928 929 930 931 932 933 934 935 936 937 938 939 940 941 942 943 944 945 946 947 948 949 950 951 952 953 954 955 956 957 958 959 960 961 962 963 964 965 966 967 968 969 970 971 972 973 974 975 976 977 978 979 980 981 982 983 984 985 986 987 988 989 990 991 992 993 994 995 996 997 998 999 1000 1001 1002 1003 1004 1005 1006 1007 1008 1009 1010 1011 1012 1013 1014 1015 1016 1017 1018 1019 1020 1021 1022 1023 1024 1025 1026 1027 1028 1029 1030 1031 1032 1033 1034 1035 1036 1037 1038 1039 1040 1041 1042 1043 1044 1045 1046 1047 1048 1049 1050 1051 1052 1053 1054 1055 1056 1057 1058 1059 1060 1061 1062 1063 1064 1065 1066 1067 1068 1069 1070 1071 1072 1073 1074 1075 1076 1077 1078 1079 1080 1081 1082 1083 1084 1085 1086 1087 1088 1089 1090 1091 1092 1093 1094 1095 1096 1097 1098 1099 1100 1101 1102 1103 1104 1105 1106 1107 1108 1109 1110 1111 1112 1113 1114 1115 1116 1117 1118 1119 1120 1121 1122 1123 1124 1125 1126 1127 1128 1129 1130 1131 1132 1133 1134 1135 1136 1137 1138 1139 1140 1141 1142 1143 1144 1145 1146 1147 1148 1149 1150 1151 1152 1153 1154 1155 1156 1157 1158 1159 1160 1161 1162 1163 1164 1165 1166 1167 1168 1169 1170 1171 1172 1173 1174 1175 1176 1177 1178 1179 1180 1181 1182 1183 1184 1185 1186 1187 1188 1189 1190 1191 1192 1193 1194 1195 1196 1197 1198 1199 1200 1201 1202 1203 1204 1205 1206 1207 1208 1209 1210 1211 1212 1213 1214 1215 1216 1217 1218 1219 1220 1221 1222 1223 1224 1225 1226 1227 1228 1229 1230 1231 1232 1233 1234 1235 1236 1237 1238 1239 1240 1241 1242 1243 1244 1245 1246 1247 1248 1249 1250 1251 1252 1253 1254 1255 1256 1257 1258 1259 1260 1261 1262 1263 1264 1265 1266 1267 1268 1269 1270 1271 1272 1273 1274 1275 1276 1277 1278 1279 1280 1281 1282 1283 1284 1285 1286 1287 1288 1289 1290 1291 1292 1293 1294 1295 1296 1297 1298 1299 1300 1301 1302 1303 1304 1305 1306 1307 1308 1309 1310 1311 1312 1313 1314 1315 1316 1317 1318 1319 1320 1321 1322 1323 1324 1325 1326 1327 1328 1329 1330 1331 1332 1333 1334 1335 1336 1337 1338 1339 1340 1341 1342 1343 1344 1345 1346 1347 1348 1349 1350 1351 1352 1353 1354 1355 1356 1357 1358 1359 1360 1361 1362 1363 1364 1365 1366 1367 1368 1369 1370 1371 1372 1373 1374 1375 1376 1377 1378 1379 1380 1381 1382 1383 1384 1385 1386 1387 1388 1389 1390 1391 1392 1393 1394 1395 1396 1397 1398 1399 1400 1401 1402 1403 1404 1405 1406 1407 1408 1409 1410 1411 1412 1413 1414 1415 1416 1417 1418 1419 1420 1421 1422 1423 1424 1425 1426 1427 1428 1429 1430 1431 1432 1433 1434 1435 1436 1437 1438 1439 1440 1441 1442 1443 1444 1445 1446 1447 1448 1449 1450 1451 1452 1453 1454 1455 1456 1457 1458 1459 1460 1461 1462 1463 1464 1465 1466 1467 1468 1469 1470 1471 1472 1473 1474 1475 1476 1477 1478 1479 1480 1481 1482 1483 1484 1485 1486 1487 1488 1489 1490 1491 1492 1493 1494 1495 1496 1497 1498 1499 1500 1501 1502 1503 1504 1505 1506 1507 1508 1509 1510 1511 1512 1513 1514 1515 1516 1517 1518 1519 1520 1521 1522 1523 1524 1525 1526 1527 1528 1529 1530 1531 1532 1533 1534 1535 1536 1537 1538 1539 1540 1541 1542 1543 1544 1545 1546 1547 1548 1549 1550 1551 1552 1553 1554 1555 1556 1557 1558 1559 1560 1561 1562 1563 1564 1565 1566 1567 1568 1569 1570 1571 1572 1573 1574 1575 1576 1577 1578 1579 1580 1581 1582 1583 1584 1585 1586 1587 1588 1589 1590 1591 1592 1593 1594 1595 1596 1597 1598 1599 1600 1601 1602 1603 1604 1605 1606 1607 1608 1609 1610 1611 1612 1613 1614 1615 1616 1617 1618 1619 1620 1621 1622 1623 1624 1625 1626 1627 1628 1629 1630 1631 1632 1633 1634 1635 1636 1637 1638 1639 1640 1641 1642 1643 1644 1645 1646 1647 1648 1649 1650 1651 1652 1653 1654 1655 1656 1657 1658 1659 1660 1661 1662 1663 1664 1665 1666 1667 1668 1669 1670 1671 1672 1673 1674 1675 1676 1677 1678 1679 1680 1681 1682 1683 1684 1685 1686 1687 1688 1689 1690 1691 1692 1693 1694 1695 1696 1697 1698 1699 1700 1701 1702 1703 1704 1705 1706 1707 1708 1709 1710 1711 1712 1713 1714 1715 1716 1717 1718 1719 1720 1721 1722 1723 1724 1725 1726 1727 1728 1729 1730 1731 1732 1733 1734 1735 1736 1737 1738 1739 1740 1741 1742 1743 1744 1745 1746 1747 1748 1749 1750 1751 1752 1753 1754 1755 1756 1757 1758 1759 1760 1761 1762 1763 1764 1765 1766 1767 1768 1769 1770 1771 1772 1773 1774 1775 1776 1777 1778 1779 1780 1781 1782 1783 1784 1785 1786 1787 1788 1789 1790 1791 1792 1793 1794 1795 1796 1797 1798 1799 1800 1801 1802 1803 1804 1805 1806 1807 1808 1809 1810 1811 1812 1813 1814 1815 1816 1817 1818 1819 1820 1821 1822 1823 1824 1825 1826 1827 1828 1829 1830 1831 1832 1833 1834 1835 1836 1837 1838 1839 1840 1841 1842 1843 1844 1845 1846 1847 1848 1849 1850 1851 1852 1853 1854 1855 1856 1857 1858 1859 1860 1861 1862 1863 1864 1865 1866 1867 1868 1869 1870 1871 1872 1873 1874 1875 1876 1877 1878 1879 1880 1881 1882 1883 1884 1885 1886 1887 1888 1889 1890 1891 1892 1893 1894 1895 1896 1897 1898 1899 1900 1901 1902 1903 1904 1905 1906 1907 1908 1909 1910 1911 1912 1913 1914 1915 1916 1917 1918 1919 1920 1921 1922 1923 1924 1925 1926 1927 1928 1929 1930 1931 1932 1933 1934 1935 1936 1937 1938 1939 1940 1941 1942 1943 1944 1945 1946 1947 1948 1949 1950 1951 1952 1953 1954 1955 1956 1957 1958 1959 1960 1961 1962 1963 1964 1965 1966 1967 1968 1969 1970 1971 1972 1973 1974 1975 1976 1977 1978 1979 1980 1981 1982 1983 1984 1985 1986 1987 1988 1989 1990 1991 1992 1993 1994 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013 2014 2015 2016 2017 2018 2019 2020 2021 2022 2023 2024 2025 2026 2027 2028 2029 2030 2031 2032 2033 2034 2035 2036 2037 2038 2039 2040 2041 2042 2043 2044 2045 2046 2047 2048 2049 2050 2051 2052 2053 2054 2055 2056 2057 2058 2059 2060 2061 2062 2063 2064 2065 2066 2067 2068 2069 2070 2071 2072 2073 2074 2075 2076 2077 2078 2079 2080 2081 2082 2083 2084 2085 2086 2087 2088 2089 2090 2091 2092 2093 2094 2095 2096 2097 2098 2099 2100 2101 2102 2103 2104 2105 2106 2107 2108 2109 2110 2111 2112 2113 2114 2115 2116 2117 2118 2119 2120 2121 2122 2123 2124 2125 2126 2127 2128 2129 2130 2131 2132 2133 2134 2135 2136 2137 2138 2139 2140 2141 2142 2143 2144 2145 2146 2147 2148 2149 2150 2151 2152 2153 2154 2155 2156 2157 2158 2159 2160 2161 2162 2163 2164 2165 2166 2167 2168 2169 2170 2171 2172 2173 2174 2175 2176 2177 2178 2179 2180 2181 2182 2183 2184 2185 2186 2187 2188 2189 2190 2191 2192 2193 2194 2195 2196 2197 2198 2199 2200 2201 2202 2203 2204 2205 2206 2207 2208 2209 2210 2211 2212 2213 2214 2215 2216 2217 2218 2219 2220 2221 2222 2223 2224 2225 2226 2227 2228 2229 2230 2231 2232 2233 2234 2235 2236 2237 2238 2239 2240 2241 2242 2243 2244 2245 2246 2247 2248 2249 2250 2251 2252 2253 2254 2255 2256 2257 2258 2259 2260 2261 2262 2263 2264 2265 2266 2267 2268 2269 2270 2271 2272 2273 2274 2275 2276 2277 2278 2279 2280 2281 2282 2283 2284 2285 2286 2287 2288 2289 2290 2291 2292 2293 2294 2295 2296 2297 2298 2299 2300 2301 2302 2303 2304 2305 2306 2307 2308 2309 2310 2311 2312 2313 2314 2315 2316 2317 2318 2319 2320 2321 2322 2323 2324 2325 2326 2327 2328 2329 2330 2331 2332 2333 2334 2335 2336 2337 2338 2339 2340 2341 2342 2343 2344 2345 2346 2347 2348 2349 2350 2351 2352 2353 2354 2355 2356 2357 2358 2359 2360 2361 2362 2363 2364 2365 2366 2367 2368 2369 2370 2371 2372 2373 2374 2375 2376 2377 2378 2379 2380 2381 2382 2383 2384 2385 2386 2387 2388 2389 2390 2391 2392 2393 2394 2395 2396 2397 2398 2399 2400 2401 2402 2403 2404 2405 2406 2407 2408 2409 2410 2411 2412 2413 2414 2415 2416 2417 2418 2419 2420 2421 2422 2423 2424 2425 2426 2427 2428 2429 2430 2431 2432 2433 2434 2435 2436 2437 2438 2439 2440 2441 2442 2443 2444 2445 2446 2447 2448 2449 2450 2451 2452 2453 2454 2455 2456 2457 2458 2459 2460 2461 2462 2463 2464 2465 2466 2467 2468 2469 2470 2471 2472 2473 2474 2475 2476 2477 2478 2479 2480 2481 2482 2483 2484 2485 2486 2487 2488 2489 2490 2491 2492 2493 2494 2495 2496 2497 2498 2499 2500 2501 2502 2503 2504 2505 2506 2507 2508 2509 2510 2511 2512 2513 2514 2515 2516 2517 2518 2519 2520 2521 2522 2523 2524 2525 2526 2527 2528 2529 2530 2531 2532 2533 2534 2535 2536 2537 2538 2539 2540 2541 2542 2543 2544 2545 2546 2547 2548 2549 2550 2551 2552 2553 2554 2555 2556 2557 2558 2559 2560 2561 2562 2563 2564 2565 2566 2567 2568 2569 2570 2571 2572 2573 2574 2575 2576 2577 2578 2579 2580 2581 2582 2583 2584 2585 2586 2587 2588 2589 2590 2591 2592 2593 2594 2595 2596 2597 2598 2599 2600 2601 2602 2603 2604 2605 2606 2607 2608 2609 2610 2611 2612 2613 2614 2615 2616 2617 2618 2619 2620 2621 2622 2623 2624 2625 2626 2627 2628 2629 2630 2631 2632 2633 2634 2635 2636 2637 2638 2639 2640 2641 2642 2643 2644 2645 2646 2647 2648 2649 2650 2651 2652 2653 2654 2655 2656 2657 2658 2659 2660 2661 2662 2663 2664 2665 2666 2667 2668 2669 2670 2671 2672 2673 2674 2675 2676 2677 2678 2679 2680 2681 2682 2683 2684 2685 2686 2687 2688 2689 2690 2691 2692 2693 2694 2695 2696 2697 2698 2699 2700 2701 2702 2703 2704 2705 2706 2707 2708 2709 2710 2711 2712 2713 2714 2715 2716 2717 2718 2719 2720 2721 2722 2723 2724 2725 2726 2727 2728 2729 2730 2731 2732 2733 2734 2735 2736 2737 2738 2739 2740 2741 2742 2743 2744 2745 2746 2747 2748 2749 2750 2751 2752 2753 2754 2755 2756 2757 2758 2759 2760 2761 2762 2763 2764 2765 2766 2767 2768 2769 2770 2771 2772 2773 2774 2775 2776 2777 2778 2779 2780 2781 2782 2783 2784 2785 2786 2787 2788 2789 2790 2791 2792 2793 2794 2795 2796 2797 2798 2799 2800 2801 2802 2803 2804 2805 2806 2807 2808 2809 2810 2811 2812 2813 2814 2815 2816 2817 2818 2819 2820 2821 2822 2823 2824 2825 2826 2827 2828 2829 2830 2831 2832 2833 2834 2835 2836 2837 2838 2839 2840 2841 2842 2843 2844 2845 2846 2847 2848 2849 2850 2851 2852 2853 2854 2855 2856 2857 2858 2859 2860 2861 2862 2863 2864 2865 2866 2867 2868 2869 2870 | /* ** 2022-08-27 ** ** The author disclaims copyright to this source code. In place of ** a legal notice, here is a blessing: ** ** May you do good and not evil. ** May you find forgiveness for yourself and forgive others. ** May you share freely, never taking more than you give. ** ************************************************************************* ** */ #include "sqlite3recover.h" #include <assert.h> #include <string.h> #ifndef SQLITE_OMIT_VIRTUALTABLE /* ** Declaration for public API function in file dbdata.c. This may be called ** with NULL as the final two arguments to register the sqlite_dbptr and ** sqlite_dbdata virtual tables with a database handle. */ #ifdef _WIN32 __declspec(dllexport) #endif int sqlite3_dbdata_init(sqlite3*, char**, const sqlite3_api_routines*); typedef unsigned int u32; typedef unsigned char u8; typedef sqlite3_int64 i64; typedef struct RecoverTable RecoverTable; typedef struct RecoverColumn RecoverColumn; /* ** When recovering rows of data that can be associated with table ** definitions recovered from the sqlite_schema table, each table is ** represented by an instance of the following object. ** ** iRoot: ** The root page in the original database. Not necessarily (and usually ** not) the same in the recovered database. ** ** zTab: ** Name of the table. ** ** nCol/aCol[]: ** aCol[] is an array of nCol columns. In the order in which they appear ** in the table. ** ** bIntkey: ** Set to true for intkey tables, false for WITHOUT ROWID. ** ** iRowidBind: ** Each column in the aCol[] array has associated with it the index of ** the bind parameter its values will be bound to in the INSERT statement ** used to construct the output database. If the table does has a rowid ** but not an INTEGER PRIMARY KEY column, then iRowidBind contains the ** index of the bind paramater to which the rowid value should be bound. ** Otherwise, it contains -1. If the table does contain an INTEGER PRIMARY ** KEY column, then the rowid value should be bound to the index associated ** with the column. ** ** pNext: ** All RecoverTable objects used by the recovery operation are allocated ** and populated as part of creating the recovered database schema in ** the output database, before any non-schema data are recovered. They ** are then stored in a singly-linked list linked by this variable beginning ** at sqlite3_recover.pTblList. */ struct RecoverTable { u32 iRoot; /* Root page in original database */ char *zTab; /* Name of table */ int nCol; /* Number of columns in table */ RecoverColumn *aCol; /* Array of columns */ int bIntkey; /* True for intkey, false for without rowid */ int iRowidBind; /* If >0, bind rowid to INSERT here */ RecoverTable *pNext; }; /* ** Each database column is represented by an instance of the following object ** stored in the RecoverTable.aCol[] array of the associated table. ** ** iField: ** The index of the associated field within database records. Or -1 if ** there is no associated field (e.g. for virtual generated columns). ** ** iBind: ** The bind index of the INSERT statement to bind this columns values ** to. Or 0 if there is no such index (iff (iField<0)). ** ** bIPK: ** True if this is the INTEGER PRIMARY KEY column. ** ** zCol: ** Name of column. ** ** eHidden: ** A RECOVER_EHIDDEN_* constant value (see below for interpretation of each). */ struct RecoverColumn { int iField; /* Field in record on disk */ int iBind; /* Binding to use in INSERT */ int bIPK; /* True for IPK column */ char *zCol; int eHidden; }; #define RECOVER_EHIDDEN_NONE 0 /* Normal database column */ #define RECOVER_EHIDDEN_HIDDEN 1 /* Column is __HIDDEN__ */ #define RECOVER_EHIDDEN_VIRTUAL 2 /* Virtual generated column */ #define RECOVER_EHIDDEN_STORED 3 /* Stored generated column */ /* ** Bitmap object used to track pages in the input database. Allocated ** and manipulated only by the following functions: ** ** recoverBitmapAlloc() ** recoverBitmapFree() ** recoverBitmapSet() ** recoverBitmapQuery() ** ** nPg: ** Largest page number that may be stored in the bitmap. The range ** of valid keys is 1 to nPg, inclusive. ** ** aElem[]: ** Array large enough to contain a bit for each key. For key value ** iKey, the associated bit is the bit (iKey%32) of aElem[iKey/32]. ** In other words, the following is true if bit iKey is set, or ** false if it is clear: ** ** (aElem[iKey/32] & (1 << (iKey%32))) ? 1 : 0 */ typedef struct RecoverBitmap RecoverBitmap; struct RecoverBitmap { i64 nPg; /* Size of bitmap */ u32 aElem[1]; /* Array of 32-bit bitmasks */ }; /* ** State variables (part of the sqlite3_recover structure) used while ** recovering data for tables identified in the recovered schema (state ** RECOVER_STATE_WRITING). */ typedef struct RecoverStateW1 RecoverStateW1; struct RecoverStateW1 { sqlite3_stmt *pTbls; sqlite3_stmt *pSel; sqlite3_stmt *pInsert; int nInsert; RecoverTable *pTab; /* Table currently being written */ int nMax; /* Max column count in any schema table */ sqlite3_value **apVal; /* Array of nMax values */ int nVal; /* Number of valid entries in apVal[] */ int bHaveRowid; i64 iRowid; i64 iPrevPage; int iPrevCell; }; /* ** State variables (part of the sqlite3_recover structure) used while ** recovering data destined for the lost and found table (states ** RECOVER_STATE_LOSTANDFOUND[123]). */ typedef struct RecoverStateLAF RecoverStateLAF; struct RecoverStateLAF { RecoverBitmap *pUsed; i64 nPg; /* Size of db in pages */ sqlite3_stmt *pAllAndParent; sqlite3_stmt *pMapInsert; sqlite3_stmt *pMaxField; sqlite3_stmt *pUsedPages; sqlite3_stmt *pFindRoot; sqlite3_stmt *pInsert; /* INSERT INTO lost_and_found ... */ sqlite3_stmt *pAllPage; sqlite3_stmt *pPageData; sqlite3_value **apVal; int nMaxField; }; /* ** Main recover handle structure. */ struct sqlite3_recover { /* Copies of sqlite3_recover_init[_sql]() parameters */ sqlite3 *dbIn; /* Input database */ char *zDb; /* Name of input db ("main" etc.) */ char *zUri; /* URI for output database */ void *pSqlCtx; /* SQL callback context */ int (*xSql)(void*,const char*); /* Pointer to SQL callback function */ /* Values configured by sqlite3_recover_config() */ char *zStateDb; /* State database to use (or NULL) */ char *zLostAndFound; /* Name of lost-and-found table (or NULL) */ int bFreelistCorrupt; /* SQLITE_RECOVER_FREELIST_CORRUPT setting */ int bRecoverRowid; /* SQLITE_RECOVER_ROWIDS setting */ int bSlowIndexes; /* SQLITE_RECOVER_SLOWINDEXES setting */ int pgsz; int detected_pgsz; int nReserve; u8 *pPage1Disk; u8 *pPage1Cache; /* Error code and error message */ int errCode; /* For sqlite3_recover_errcode() */ char *zErrMsg; /* For sqlite3_recover_errmsg() */ int eState; int bCloseTransaction; /* Variables used with eState==RECOVER_STATE_WRITING */ RecoverStateW1 w1; /* Variables used with states RECOVER_STATE_LOSTANDFOUND[123] */ RecoverStateLAF laf; /* Fields used within sqlite3_recover_run() */ sqlite3 *dbOut; /* Output database */ sqlite3_stmt *pGetPage; /* SELECT against input db sqlite_dbdata */ RecoverTable *pTblList; /* List of tables recovered from schema */ }; /* ** The various states in which an sqlite3_recover object may exist: ** ** RECOVER_STATE_INIT: ** The object is initially created in this state. sqlite3_recover_step() ** has yet to be called. This is the only state in which it is permitted ** to call sqlite3_recover_config(). ** ** RECOVER_STATE_WRITING: ** ** RECOVER_STATE_LOSTANDFOUND1: ** State to populate the bitmap of pages used by other tables or the ** database freelist. ** ** RECOVER_STATE_LOSTANDFOUND2: ** Populate the recovery.map table - used to figure out a "root" page ** for each lost page from in the database from which records are ** extracted. ** ** RECOVER_STATE_LOSTANDFOUND3: ** Populate the lost-and-found table itself. */ #define RECOVER_STATE_INIT 0 #define RECOVER_STATE_WRITING 1 #define RECOVER_STATE_LOSTANDFOUND1 2 #define RECOVER_STATE_LOSTANDFOUND2 3 #define RECOVER_STATE_LOSTANDFOUND3 4 #define RECOVER_STATE_SCHEMA2 5 #define RECOVER_STATE_DONE 6 /* ** Global variables used by this extension. */ typedef struct RecoverGlobal RecoverGlobal; struct RecoverGlobal { const sqlite3_io_methods *pMethods; sqlite3_recover *p; }; static RecoverGlobal recover_g; /* ** Use this static SQLite mutex to protect the globals during the ** first call to sqlite3_recover_step(). */ #define RECOVER_MUTEX_ID SQLITE_MUTEX_STATIC_APP2 /* ** Default value for SQLITE_RECOVER_ROWIDS (sqlite3_recover.bRecoverRowid). */ #define RECOVER_ROWID_DEFAULT 1 /* ** Mutex handling: ** ** recoverEnterMutex() - Enter the recovery mutex ** recoverLeaveMutex() - Leave the recovery mutex ** recoverAssertMutexHeld() - Assert that the recovery mutex is held */ #if defined(SQLITE_THREADSAFE) && SQLITE_THREADSAFE==0 # define recoverEnterMutex() # define recoverLeaveMutex() #else static void recoverEnterMutex(void){ sqlite3_mutex_enter(sqlite3_mutex_alloc(RECOVER_MUTEX_ID)); } static void recoverLeaveMutex(void){ sqlite3_mutex_leave(sqlite3_mutex_alloc(RECOVER_MUTEX_ID)); } #endif #if SQLITE_THREADSAFE+0>=1 && defined(SQLITE_DEBUG) static void recoverAssertMutexHeld(void){ assert( sqlite3_mutex_held(sqlite3_mutex_alloc(RECOVER_MUTEX_ID)) ); } #else # define recoverAssertMutexHeld() #endif /* ** Like strlen(). But handles NULL pointer arguments. */ static int recoverStrlen(const char *zStr){ if( zStr==0 ) return 0; return (int)(strlen(zStr)&0x7fffffff); } /* ** This function is a no-op if the recover handle passed as the first ** argument already contains an error (if p->errCode!=SQLITE_OK). ** ** Otherwise, an attempt is made to allocate, zero and return a buffer nByte ** bytes in size. If successful, a pointer to the new buffer is returned. Or, ** if an OOM error occurs, NULL is returned and the handle error code ** (p->errCode) set to SQLITE_NOMEM. */ static void *recoverMalloc(sqlite3_recover *p, i64 nByte){ void *pRet = 0; assert( nByte>0 ); if( p->errCode==SQLITE_OK ){ pRet = sqlite3_malloc64(nByte); if( pRet ){ memset(pRet, 0, nByte); }else{ p->errCode = SQLITE_NOMEM; } } return pRet; } /* ** Set the error code and error message for the recover handle passed as ** the first argument. The error code is set to the value of parameter ** errCode. ** ** Parameter zFmt must be a printf() style formatting string. The handle ** error message is set to the result of using any trailing arguments for ** parameter substitutions in the formatting string. ** ** For example: ** ** recoverError(p, SQLITE_ERROR, "no such table: %s", zTablename); */ static int recoverError( sqlite3_recover *p, int errCode, const char *zFmt, ... ){ char *z = 0; va_list ap; va_start(ap, zFmt); if( zFmt ){ z = sqlite3_vmprintf(zFmt, ap); va_end(ap); } sqlite3_free(p->zErrMsg); p->zErrMsg = z; p->errCode = errCode; return errCode; } /* ** This function is a no-op if p->errCode is initially other than SQLITE_OK. ** In this case it returns NULL. ** ** Otherwise, an attempt is made to allocate and return a bitmap object ** large enough to store a bit for all page numbers between 1 and nPg, ** inclusive. The bitmap is initially zeroed. */ static RecoverBitmap *recoverBitmapAlloc(sqlite3_recover *p, i64 nPg){ int nElem = (nPg+1+31) / 32; int nByte = sizeof(RecoverBitmap) + nElem*sizeof(u32); RecoverBitmap *pRet = (RecoverBitmap*)recoverMalloc(p, nByte); if( pRet ){ pRet->nPg = nPg; } return pRet; } /* ** Free a bitmap object allocated by recoverBitmapAlloc(). */ static void recoverBitmapFree(RecoverBitmap *pMap){ sqlite3_free(pMap); } /* ** Set the bit associated with page iPg in bitvec pMap. */ static void recoverBitmapSet(RecoverBitmap *pMap, i64 iPg){ if( iPg<=pMap->nPg ){ int iElem = (iPg / 32); int iBit = (iPg % 32); pMap->aElem[iElem] |= (((u32)1) << iBit); } } /* ** Query bitmap object pMap for the state of the bit associated with page ** iPg. Return 1 if it is set, or 0 otherwise. */ static int recoverBitmapQuery(RecoverBitmap *pMap, i64 iPg){ int ret = 1; if( iPg<=pMap->nPg && iPg>0 ){ int iElem = (iPg / 32); int iBit = (iPg % 32); ret = (pMap->aElem[iElem] & (((u32)1) << iBit)) ? 1 : 0; } return ret; } /* ** Set the recover handle error to the error code and message returned by ** calling sqlite3_errcode() and sqlite3_errmsg(), respectively, on database ** handle db. */ static int recoverDbError(sqlite3_recover *p, sqlite3 *db){ return recoverError(p, sqlite3_errcode(db), "%s", sqlite3_errmsg(db)); } /* ** This function is a no-op if recover handle p already contains an error ** (if p->errCode!=SQLITE_OK). ** ** Otherwise, it attempts to prepare the SQL statement in zSql against ** database handle db. If successful, the statement handle is returned. ** Or, if an error occurs, NULL is returned and an error left in the ** recover handle. */ static sqlite3_stmt *recoverPrepare( sqlite3_recover *p, sqlite3 *db, const char *zSql ){ sqlite3_stmt *pStmt = 0; if( p->errCode==SQLITE_OK ){ if( sqlite3_prepare_v2(db, zSql, -1, &pStmt, 0) ){ recoverDbError(p, db); } } return pStmt; } /* ** This function is a no-op if recover handle p already contains an error ** (if p->errCode!=SQLITE_OK). ** ** Otherwise, argument zFmt is used as a printf() style format string, ** along with any trailing arguments, to create an SQL statement. This ** SQL statement is prepared against database handle db and, if successful, ** the statment handle returned. Or, if an error occurs - either during ** the printf() formatting or when preparing the resulting SQL - an ** error code and message are left in the recover handle. */ static sqlite3_stmt *recoverPreparePrintf( sqlite3_recover *p, sqlite3 *db, const char *zFmt, ... ){ sqlite3_stmt *pStmt = 0; if( p->errCode==SQLITE_OK ){ va_list ap; char *z; va_start(ap, zFmt); z = sqlite3_vmprintf(zFmt, ap); va_end(ap); if( z==0 ){ p->errCode = SQLITE_NOMEM; }else{ pStmt = recoverPrepare(p, db, z); sqlite3_free(z); } } return pStmt; } /* ** Reset SQLite statement handle pStmt. If the call to sqlite3_reset() ** indicates that an error occurred, and there is not already an error ** in the recover handle passed as the first argument, set the error ** code and error message appropriately. ** ** This function returns a copy of the statement handle pointer passed ** as the second argument. */ static sqlite3_stmt *recoverReset(sqlite3_recover *p, sqlite3_stmt *pStmt){ int rc = sqlite3_reset(pStmt); if( rc!=SQLITE_OK && rc!=SQLITE_CONSTRAINT && p->errCode==SQLITE_OK ){ recoverDbError(p, sqlite3_db_handle(pStmt)); } return pStmt; } /* ** Finalize SQLite statement handle pStmt. If the call to sqlite3_reset() ** indicates that an error occurred, and there is not already an error ** in the recover handle passed as the first argument, set the error ** code and error message appropriately. */ static void recoverFinalize(sqlite3_recover *p, sqlite3_stmt *pStmt){ sqlite3 *db = sqlite3_db_handle(pStmt); int rc = sqlite3_finalize(pStmt); if( rc!=SQLITE_OK && p->errCode==SQLITE_OK ){ recoverDbError(p, db); } } /* ** This function is a no-op if recover handle p already contains an error ** (if p->errCode!=SQLITE_OK). A copy of p->errCode is returned in this ** case. ** ** Otherwise, execute SQL script zSql. If successful, return SQLITE_OK. ** Or, if an error occurs, leave an error code and message in the recover ** handle and return a copy of the error code. */ static int recoverExec(sqlite3_recover *p, sqlite3 *db, const char *zSql){ if( p->errCode==SQLITE_OK ){ int rc = sqlite3_exec(db, zSql, 0, 0, 0); if( rc ){ recoverDbError(p, db); } } return p->errCode; } /* ** Bind the value pVal to parameter iBind of statement pStmt. Leave an ** error in the recover handle passed as the first argument if an error ** (e.g. an OOM) occurs. */ static void recoverBindValue( sqlite3_recover *p, sqlite3_stmt *pStmt, int iBind, sqlite3_value *pVal ){ if( p->errCode==SQLITE_OK ){ int rc = sqlite3_bind_value(pStmt, iBind, pVal); if( rc ) recoverError(p, rc, 0); } } /* ** This function is a no-op if recover handle p already contains an error ** (if p->errCode!=SQLITE_OK). NULL is returned in this case. ** ** Otherwise, an attempt is made to interpret zFmt as a printf() style ** formatting string and the result of using the trailing arguments for ** parameter substitution with it written into a buffer obtained from ** sqlite3_malloc(). If successful, a pointer to the buffer is returned. ** It is the responsibility of the caller to eventually free the buffer ** using sqlite3_free(). ** ** Or, if an error occurs, an error code and message is left in the recover ** handle and NULL returned. */ static char *recoverMPrintf(sqlite3_recover *p, const char *zFmt, ...){ va_list ap; char *z; va_start(ap, zFmt); z = sqlite3_vmprintf(zFmt, ap); va_end(ap); if( p->errCode==SQLITE_OK ){ if( z==0 ) p->errCode = SQLITE_NOMEM; }else{ sqlite3_free(z); z = 0; } return z; } /* ** This function is a no-op if recover handle p already contains an error ** (if p->errCode!=SQLITE_OK). Zero is returned in this case. ** ** Otherwise, execute "PRAGMA page_count" against the input database. If ** successful, return the integer result. Or, if an error occurs, leave an ** error code and error message in the sqlite3_recover handle and return ** zero. */ static i64 recoverPageCount(sqlite3_recover *p){ i64 nPg = 0; if( p->errCode==SQLITE_OK ){ sqlite3_stmt *pStmt = 0; pStmt = recoverPreparePrintf(p, p->dbIn, "PRAGMA %Q.page_count", p->zDb); if( pStmt ){ sqlite3_step(pStmt); nPg = sqlite3_column_int64(pStmt, 0); } recoverFinalize(p, pStmt); } return nPg; } /* ** Implementation of SQL scalar function "read_i32". The first argument to ** this function must be a blob. The second a non-negative integer. This ** function reads and returns a 32-bit big-endian integer from byte ** offset (4*<arg2>) of the blob. ** ** SELECT read_i32(<blob>, <idx>) */ static void recoverReadI32( sqlite3_context *context, int argc, sqlite3_value **argv ){ const unsigned char *pBlob; int nBlob; int iInt; assert( argc==2 ); nBlob = sqlite3_value_bytes(argv[0]); pBlob = (const unsigned char*)sqlite3_value_blob(argv[0]); iInt = sqlite3_value_int(argv[1]) & 0xFFFF; if( (iInt+1)*4<=nBlob ){ const unsigned char *a = &pBlob[iInt*4]; i64 iVal = ((i64)a[0]<<24) + ((i64)a[1]<<16) + ((i64)a[2]<< 8) + ((i64)a[3]<< 0); sqlite3_result_int64(context, iVal); } } /* ** Implementation of SQL scalar function "page_is_used". This function ** is used as part of the procedure for locating orphan rows for the ** lost-and-found table, and it depends on those routines having populated ** the sqlite3_recover.laf.pUsed variable. ** ** The only argument to this function is a page-number. It returns true ** if the page has already been used somehow during data recovery, or false ** otherwise. ** ** SELECT page_is_used(<pgno>); */ static void recoverPageIsUsed( sqlite3_context *pCtx, int nArg, sqlite3_value **apArg ){ sqlite3_recover *p = (sqlite3_recover*)sqlite3_user_data(pCtx); i64 pgno = sqlite3_value_int64(apArg[0]); assert( nArg==1 ); sqlite3_result_int(pCtx, recoverBitmapQuery(p->laf.pUsed, pgno)); } /* ** The implementation of a user-defined SQL function invoked by the ** sqlite_dbdata and sqlite_dbptr virtual table modules to access pages ** of the database being recovered. ** ** This function always takes a single integer argument. If the argument ** is zero, then the value returned is the number of pages in the db being ** recovered. If the argument is greater than zero, it is a page number. ** The value returned in this case is an SQL blob containing the data for ** the identified page of the db being recovered. e.g. ** ** SELECT getpage(0); -- return number of pages in db ** SELECT getpage(4); -- return page 4 of db as a blob of data */ static void recoverGetPage( sqlite3_context *pCtx, int nArg, sqlite3_value **apArg ){ sqlite3_recover *p = (sqlite3_recover*)sqlite3_user_data(pCtx); i64 pgno = sqlite3_value_int64(apArg[0]); sqlite3_stmt *pStmt = 0; assert( nArg==1 ); if( pgno==0 ){ i64 nPg = recoverPageCount(p); sqlite3_result_int64(pCtx, nPg); return; }else{ if( p->pGetPage==0 ){ pStmt = p->pGetPage = recoverPreparePrintf( p, p->dbIn, "SELECT data FROM sqlite_dbpage(%Q) WHERE pgno=?", p->zDb ); }else if( p->errCode==SQLITE_OK ){ pStmt = p->pGetPage; } if( pStmt ){ sqlite3_bind_int64(pStmt, 1, pgno); if( SQLITE_ROW==sqlite3_step(pStmt) ){ const u8 *aPg; int nPg; assert( p->errCode==SQLITE_OK ); aPg = sqlite3_column_blob(pStmt, 0); nPg = sqlite3_column_bytes(pStmt, 0); if( pgno==1 && nPg==p->pgsz && 0==memcmp(p->pPage1Cache, aPg, nPg) ){ aPg = p->pPage1Disk; } sqlite3_result_blob(pCtx, aPg, nPg-p->nReserve, SQLITE_TRANSIENT); } recoverReset(p, pStmt); } } if( p->errCode ){ if( p->zErrMsg ) sqlite3_result_error(pCtx, p->zErrMsg, -1); sqlite3_result_error_code(pCtx, p->errCode); } } /* ** Find a string that is not found anywhere in z[]. Return a pointer ** to that string. ** ** Try to use zA and zB first. If both of those are already found in z[] ** then make up some string and store it in the buffer zBuf. */ static const char *recoverUnusedString( const char *z, /* Result must not appear anywhere in z */ const char *zA, const char *zB, /* Try these first */ char *zBuf /* Space to store a generated string */ ){ unsigned i = 0; if( strstr(z, zA)==0 ) return zA; if( strstr(z, zB)==0 ) return zB; do{ sqlite3_snprintf(20,zBuf,"(%s%u)", zA, i++); }while( strstr(z,zBuf)!=0 ); return zBuf; } /* ** Implementation of scalar SQL function "escape_crnl". The argument passed to ** this function is the output of built-in function quote(). If the first ** character of the input is "'", indicating that the value passed to quote() ** was a text value, then this function searches the input for "\n" and "\r" ** characters and adds a wrapper similar to the following: ** ** replace(replace(<input>, '\n', char(10), '\r', char(13)); ** ** Or, if the first character of the input is not "'", then a copy of the input ** is returned. */ static void recoverEscapeCrnl( sqlite3_context *context, int argc, sqlite3_value **argv ){ const char *zText = (const char*)sqlite3_value_text(argv[0]); (void)argc; if( zText && zText[0]=='\'' ){ int nText = sqlite3_value_bytes(argv[0]); int i; char zBuf1[20]; char zBuf2[20]; const char *zNL = 0; const char *zCR = 0; int nCR = 0; int nNL = 0; for(i=0; zText[i]; i++){ if( zNL==0 && zText[i]=='\n' ){ zNL = recoverUnusedString(zText, "\\n", "\\012", zBuf1); nNL = (int)strlen(zNL); } if( zCR==0 && zText[i]=='\r' ){ zCR = recoverUnusedString(zText, "\\r", "\\015", zBuf2); nCR = (int)strlen(zCR); } } if( zNL || zCR ){ int iOut = 0; i64 nMax = (nNL > nCR) ? nNL : nCR; i64 nAlloc = nMax * nText + (nMax+64)*2; char *zOut = (char*)sqlite3_malloc64(nAlloc); if( zOut==0 ){ sqlite3_result_error_nomem(context); return; } if( zNL && zCR ){ memcpy(&zOut[iOut], "replace(replace(", 16); iOut += 16; }else{ memcpy(&zOut[iOut], "replace(", 8); iOut += 8; } for(i=0; zText[i]; i++){ if( zText[i]=='\n' ){ memcpy(&zOut[iOut], zNL, nNL); iOut += nNL; }else if( zText[i]=='\r' ){ memcpy(&zOut[iOut], zCR, nCR); iOut += nCR; }else{ zOut[iOut] = zText[i]; iOut++; } } if( zNL ){ memcpy(&zOut[iOut], ",'", 2); iOut += 2; memcpy(&zOut[iOut], zNL, nNL); iOut += nNL; memcpy(&zOut[iOut], "', char(10))", 12); iOut += 12; } if( zCR ){ memcpy(&zOut[iOut], ",'", 2); iOut += 2; memcpy(&zOut[iOut], zCR, nCR); iOut += nCR; memcpy(&zOut[iOut], "', char(13))", 12); iOut += 12; } sqlite3_result_text(context, zOut, iOut, SQLITE_TRANSIENT); sqlite3_free(zOut); return; } } sqlite3_result_value(context, argv[0]); } /* ** This function is a no-op if recover handle p already contains an error ** (if p->errCode!=SQLITE_OK). A copy of the error code is returned in ** this case. ** ** Otherwise, attempt to populate temporary table "recovery.schema" with the ** parts of the database schema that can be extracted from the input database. ** ** If no error occurs, SQLITE_OK is returned. Otherwise, an error code ** and error message are left in the recover handle and a copy of the ** error code returned. It is not considered an error if part of all of ** the database schema cannot be recovered due to corruption. */ static int recoverCacheSchema(sqlite3_recover *p){ return recoverExec(p, p->dbOut, "WITH RECURSIVE pages(p) AS (" " SELECT 1" " UNION" " SELECT child FROM sqlite_dbptr('getpage()'), pages WHERE pgno=p" ")" "INSERT INTO recovery.schema SELECT" " max(CASE WHEN field=0 THEN value ELSE NULL END)," " max(CASE WHEN field=1 THEN value ELSE NULL END)," " max(CASE WHEN field=2 THEN value ELSE NULL END)," " max(CASE WHEN field=3 THEN value ELSE NULL END)," " max(CASE WHEN field=4 THEN value ELSE NULL END)" "FROM sqlite_dbdata('getpage()') WHERE pgno IN (" " SELECT p FROM pages" ") GROUP BY pgno, cell" ); } /* ** If this recover handle is not in SQL callback mode (i.e. was not created ** using sqlite3_recover_init_sql()) of if an error has already occurred, ** this function is a no-op. Otherwise, issue a callback with SQL statement ** zSql as the parameter. ** ** If the callback returns non-zero, set the recover handle error code to ** the value returned (so that the caller will abandon processing). */ static void recoverSqlCallback(sqlite3_recover *p, const char *zSql){ if( p->errCode==SQLITE_OK && p->xSql ){ int res = p->xSql(p->pSqlCtx, zSql); if( res ){ recoverError(p, SQLITE_ERROR, "callback returned an error - %d", res); } } } /* ** Transfer the following settings from the input database to the output ** database: ** ** + page-size, ** + auto-vacuum settings, ** + database encoding, ** + user-version (PRAGMA user_version), and ** + application-id (PRAGMA application_id), and */ static void recoverTransferSettings(sqlite3_recover *p){ const char *aPragma[] = { "encoding", "page_size", "auto_vacuum", "user_version", "application_id" }; int ii; /* Truncate the output database to 0 pages in size. This is done by ** opening a new, empty, temp db, then using the backup API to clobber ** any existing output db with a copy of it. */ if( p->errCode==SQLITE_OK ){ sqlite3 *db2 = 0; int rc = sqlite3_open("", &db2); if( rc!=SQLITE_OK ){ recoverDbError(p, db2); return; } for(ii=0; ii<(int)(sizeof(aPragma)/sizeof(aPragma[0])); ii++){ const char *zPrag = aPragma[ii]; sqlite3_stmt *p1 = 0; p1 = recoverPreparePrintf(p, p->dbIn, "PRAGMA %Q.%s", p->zDb, zPrag); if( p->errCode==SQLITE_OK && sqlite3_step(p1)==SQLITE_ROW ){ const char *zArg = (const char*)sqlite3_column_text(p1, 0); char *z2 = recoverMPrintf(p, "PRAGMA %s = %Q", zPrag, zArg); recoverSqlCallback(p, z2); recoverExec(p, db2, z2); sqlite3_free(z2); if( zArg==0 ){ recoverError(p, SQLITE_NOMEM, 0); } } recoverFinalize(p, p1); } recoverExec(p, db2, "CREATE TABLE t1(a); DROP TABLE t1;"); if( p->errCode==SQLITE_OK ){ sqlite3 *db = p->dbOut; sqlite3_backup *pBackup = sqlite3_backup_init(db, "main", db2, "main"); if( pBackup ){ sqlite3_backup_step(pBackup, -1); p->errCode = sqlite3_backup_finish(pBackup); }else{ recoverDbError(p, db); } } sqlite3_close(db2); } } /* ** This function is a no-op if recover handle p already contains an error ** (if p->errCode!=SQLITE_OK). A copy of the error code is returned in ** this case. ** ** Otherwise, an attempt is made to open the output database, attach ** and create the schema of the temporary database used to store ** intermediate data, and to register all required user functions and ** virtual table modules with the output handle. ** ** If no error occurs, SQLITE_OK is returned. Otherwise, an error code ** and error message are left in the recover handle and a copy of the ** error code returned. */ static int recoverOpenOutput(sqlite3_recover *p){ struct Func { const char *zName; int nArg; void (*xFunc)(sqlite3_context*,int,sqlite3_value **); } aFunc[] = { { "getpage", 1, recoverGetPage }, { "page_is_used", 1, recoverPageIsUsed }, { "read_i32", 2, recoverReadI32 }, { "escape_crnl", 1, recoverEscapeCrnl }, }; const int flags = SQLITE_OPEN_URI|SQLITE_OPEN_CREATE|SQLITE_OPEN_READWRITE; sqlite3 *db = 0; /* New database handle */ int ii; /* For iterating through aFunc[] */ assert( p->dbOut==0 ); if( sqlite3_open_v2(p->zUri, &db, flags, 0) ){ recoverDbError(p, db); } /* Register the sqlite_dbdata and sqlite_dbptr virtual table modules. ** These two are registered with the output database handle - this ** module depends on the input handle supporting the sqlite_dbpage ** virtual table only. */ if( p->errCode==SQLITE_OK ){ p->errCode = sqlite3_dbdata_init(db, 0, 0); } /* Register the custom user-functions with the output handle. */ for(ii=0; p->errCode==SQLITE_OK && ii<(int)(sizeof(aFunc)/sizeof(aFunc[0])); ii++){ p->errCode = sqlite3_create_function(db, aFunc[ii].zName, aFunc[ii].nArg, SQLITE_UTF8, (void*)p, aFunc[ii].xFunc, 0, 0 ); } p->dbOut = db; return p->errCode; } /* ** Attach the auxiliary database 'recovery' to the output database handle. ** This temporary database is used during the recovery process and then ** discarded. */ static void recoverOpenRecovery(sqlite3_recover *p){ char *zSql = recoverMPrintf(p, "ATTACH %Q AS recovery;", p->zStateDb); recoverExec(p, p->dbOut, zSql); recoverExec(p, p->dbOut, "PRAGMA writable_schema = 1;" "CREATE TABLE recovery.map(pgno INTEGER PRIMARY KEY, parent INT);" "CREATE TABLE recovery.schema(type, name, tbl_name, rootpage, sql);" ); sqlite3_free(zSql); } /* ** This function is a no-op if recover handle p already contains an error ** (if p->errCode!=SQLITE_OK). ** ** Otherwise, argument zName must be the name of a table that has just been ** created in the output database. This function queries the output db ** for the schema of said table, and creates a RecoverTable object to ** store the schema in memory. The new RecoverTable object is linked into ** the list at sqlite3_recover.pTblList. ** ** Parameter iRoot must be the root page of table zName in the INPUT ** database. */ static void recoverAddTable( sqlite3_recover *p, const char *zName, /* Name of table created in output db */ i64 iRoot /* Root page of same table in INPUT db */ ){ sqlite3_stmt *pStmt = recoverPreparePrintf(p, p->dbOut, "PRAGMA table_xinfo(%Q)", zName ); if( pStmt ){ int iPk = -1; int iBind = 1; RecoverTable *pNew = 0; int nCol = 0; int nName = recoverStrlen(zName); int nByte = 0; while( sqlite3_step(pStmt)==SQLITE_ROW ){ nCol++; nByte += (sqlite3_column_bytes(pStmt, 1)+1); } nByte += sizeof(RecoverTable) + nCol*sizeof(RecoverColumn) + nName+1; recoverReset(p, pStmt); pNew = recoverMalloc(p, nByte); if( pNew ){ int i = 0; int iField = 0; char *csr = 0; pNew->aCol = (RecoverColumn*)&pNew[1]; pNew->zTab = csr = (char*)&pNew->aCol[nCol]; pNew->nCol = nCol; pNew->iRoot = iRoot; memcpy(csr, zName, nName); csr += nName+1; for(i=0; sqlite3_step(pStmt)==SQLITE_ROW; i++){ int iPKF = sqlite3_column_int(pStmt, 5); int n = sqlite3_column_bytes(pStmt, 1); const char *z = (const char*)sqlite3_column_text(pStmt, 1); const char *zType = (const char*)sqlite3_column_text(pStmt, 2); int eHidden = sqlite3_column_int(pStmt, 6); if( iPk==-1 && iPKF==1 && !sqlite3_stricmp("integer", zType) ) iPk = i; if( iPKF>1 ) iPk = -2; pNew->aCol[i].zCol = csr; pNew->aCol[i].eHidden = eHidden; if( eHidden==RECOVER_EHIDDEN_VIRTUAL ){ pNew->aCol[i].iField = -1; }else{ pNew->aCol[i].iField = iField++; } if( eHidden!=RECOVER_EHIDDEN_VIRTUAL && eHidden!=RECOVER_EHIDDEN_STORED ){ pNew->aCol[i].iBind = iBind++; } memcpy(csr, z, n); csr += (n+1); } pNew->pNext = p->pTblList; p->pTblList = pNew; pNew->bIntkey = 1; } recoverFinalize(p, pStmt); pStmt = recoverPreparePrintf(p, p->dbOut, "PRAGMA index_xinfo(%Q)", zName); while( pStmt && sqlite3_step(pStmt)==SQLITE_ROW ){ int iField = sqlite3_column_int(pStmt, 0); int iCol = sqlite3_column_int(pStmt, 1); assert( iField<pNew->nCol && iCol<pNew->nCol ); pNew->aCol[iCol].iField = iField; pNew->bIntkey = 0; iPk = -2; } recoverFinalize(p, pStmt); if( p->errCode==SQLITE_OK ){ if( iPk>=0 ){ pNew->aCol[iPk].bIPK = 1; }else if( pNew->bIntkey ){ pNew->iRowidBind = iBind++; } } } } /* ** This function is called after recoverCacheSchema() has cached those parts ** of the input database schema that could be recovered in temporary table ** "recovery.schema". This function creates in the output database copies ** of all parts of that schema that must be created before the tables can ** be populated. Specifically, this means: ** ** * all tables that are not VIRTUAL, and ** * UNIQUE indexes. ** ** If the recovery handle uses SQL callbacks, then callbacks containing ** the associated "CREATE TABLE" and "CREATE INDEX" statements are made. ** ** Additionally, records are added to the sqlite_schema table of the ** output database for any VIRTUAL tables. The CREATE VIRTUAL TABLE ** records are written directly to sqlite_schema, not actually executed. ** If the handle is in SQL callback mode, then callbacks are invoked ** with equivalent SQL statements. */ static int recoverWriteSchema1(sqlite3_recover *p){ sqlite3_stmt *pSelect = 0; sqlite3_stmt *pTblname = 0; pSelect = recoverPrepare(p, p->dbOut, "WITH dbschema(rootpage, name, sql, tbl, isVirtual, isIndex) AS (" " SELECT rootpage, name, sql, " " type='table', " " sql LIKE 'create virtual%'," " (type='index' AND (sql LIKE '%unique%' OR ?1))" " FROM recovery.schema" ")" "SELECT rootpage, tbl, isVirtual, name, sql" " FROM dbschema " " WHERE tbl OR isIndex" " ORDER BY tbl DESC, name=='sqlite_sequence' DESC" ); pTblname = recoverPrepare(p, p->dbOut, "SELECT name FROM sqlite_schema " "WHERE type='table' ORDER BY rowid DESC LIMIT 1" ); if( pSelect ){ sqlite3_bind_int(pSelect, 1, p->bSlowIndexes); while( sqlite3_step(pSelect)==SQLITE_ROW ){ i64 iRoot = sqlite3_column_int64(pSelect, 0); int bTable = sqlite3_column_int(pSelect, 1); int bVirtual = sqlite3_column_int(pSelect, 2); const char *zName = (const char*)sqlite3_column_text(pSelect, 3); const char *zSql = (const char*)sqlite3_column_text(pSelect, 4); char *zFree = 0; int rc = SQLITE_OK; if( bVirtual ){ zSql = (const char*)(zFree = recoverMPrintf(p, "INSERT INTO sqlite_schema VALUES('table', %Q, %Q, 0, %Q)", zName, zName, zSql )); } rc = sqlite3_exec(p->dbOut, zSql, 0, 0, 0); if( rc==SQLITE_OK ){ recoverSqlCallback(p, zSql); if( bTable && !bVirtual ){ if( SQLITE_ROW==sqlite3_step(pTblname) ){ const char *zTbl = (const char*)sqlite3_column_text(pTblname, 0); recoverAddTable(p, zTbl, iRoot); } recoverReset(p, pTblname); } }else if( rc!=SQLITE_ERROR ){ recoverDbError(p, p->dbOut); } sqlite3_free(zFree); } } recoverFinalize(p, pSelect); recoverFinalize(p, pTblname); return p->errCode; } /* ** This function is called after the output database has been populated. It ** adds all recovered schema elements that were not created in the output ** database by recoverWriteSchema1() - everything except for tables and ** UNIQUE indexes. Specifically: ** ** * views, ** * triggers, ** * non-UNIQUE indexes. ** ** If the recover handle is in SQL callback mode, then equivalent callbacks ** are issued to create the schema elements. */ static int recoverWriteSchema2(sqlite3_recover *p){ sqlite3_stmt *pSelect = 0; pSelect = recoverPrepare(p, p->dbOut, p->bSlowIndexes ? "SELECT rootpage, sql FROM recovery.schema " " WHERE type!='table' AND type!='index'" : "SELECT rootpage, sql FROM recovery.schema " " WHERE type!='table' AND (type!='index' OR sql NOT LIKE '%unique%')" ); if( pSelect ){ while( sqlite3_step(pSelect)==SQLITE_ROW ){ const char *zSql = (const char*)sqlite3_column_text(pSelect, 1); int rc = sqlite3_exec(p->dbOut, zSql, 0, 0, 0); if( rc==SQLITE_OK ){ recoverSqlCallback(p, zSql); }else if( rc!=SQLITE_ERROR ){ recoverDbError(p, p->dbOut); } } } recoverFinalize(p, pSelect); return p->errCode; } /* ** This function is a no-op if recover handle p already contains an error ** (if p->errCode!=SQLITE_OK). In this case it returns NULL. ** ** Otherwise, if the recover handle is configured to create an output ** database (was created by sqlite3_recover_init()), then this function ** prepares and returns an SQL statement to INSERT a new record into table ** pTab, assuming the first nField fields of a record extracted from disk ** are valid. ** ** For example, if table pTab is: ** ** CREATE TABLE name(a, b GENERATED ALWAYS AS (a+1) STORED, c, d, e); ** ** And nField is 4, then the SQL statement prepared and returned is: ** ** INSERT INTO (a, c, d) VALUES (?1, ?2, ?3); ** ** In this case even though 4 values were extracted from the input db, ** only 3 are written to the output, as the generated STORED column ** cannot be written. ** ** If the recover handle is in SQL callback mode, then the SQL statement ** prepared is such that evaluating it returns a single row containing ** a single text value - itself an SQL statement similar to the above, ** except with SQL literals in place of the variables. For example: ** ** SELECT 'INSERT INTO (a, c, d) VALUES (' ** || quote(?1) || ', ' ** || quote(?2) || ', ' ** || quote(?3) || ')'; ** ** In either case, it is the responsibility of the caller to eventually ** free the statement handle using sqlite3_finalize(). */ static sqlite3_stmt *recoverInsertStmt( sqlite3_recover *p, RecoverTable *pTab, int nField ){ sqlite3_stmt *pRet = 0; const char *zSep = ""; const char *zSqlSep = ""; char *zSql = 0; char *zFinal = 0; char *zBind = 0; int ii; int bSql = p->xSql ? 1 : 0; if( nField<=0 ) return 0; assert( nField<=pTab->nCol ); zSql = recoverMPrintf(p, "INSERT OR IGNORE INTO %Q(", pTab->zTab); if( pTab->iRowidBind ){ assert( pTab->bIntkey ); zSql = recoverMPrintf(p, "%z_rowid_", zSql); if( bSql ){ zBind = recoverMPrintf(p, "%zquote(?%d)", zBind, pTab->iRowidBind); }else{ zBind = recoverMPrintf(p, "%z?%d", zBind, pTab->iRowidBind); } zSqlSep = "||', '||"; zSep = ", "; } for(ii=0; ii<nField; ii++){ int eHidden = pTab->aCol[ii].eHidden; if( eHidden!=RECOVER_EHIDDEN_VIRTUAL && eHidden!=RECOVER_EHIDDEN_STORED ){ assert( pTab->aCol[ii].iField>=0 && pTab->aCol[ii].iBind>=1 ); zSql = recoverMPrintf(p, "%z%s%Q", zSql, zSep, pTab->aCol[ii].zCol); if( bSql ){ zBind = recoverMPrintf(p, "%z%sescape_crnl(quote(?%d))", zBind, zSqlSep, pTab->aCol[ii].iBind ); zSqlSep = "||', '||"; }else{ zBind = recoverMPrintf(p, "%z%s?%d", zBind, zSep, pTab->aCol[ii].iBind); } zSep = ", "; } } if( bSql ){ zFinal = recoverMPrintf(p, "SELECT %Q || ') VALUES (' || %s || ')'", zSql, zBind ); }else{ zFinal = recoverMPrintf(p, "%s) VALUES (%s)", zSql, zBind); } pRet = recoverPrepare(p, p->dbOut, zFinal); sqlite3_free(zSql); sqlite3_free(zBind); sqlite3_free(zFinal); return pRet; } /* ** Search the list of RecoverTable objects at p->pTblList for one that ** has root page iRoot in the input database. If such an object is found, ** return a pointer to it. Otherwise, return NULL. */ static RecoverTable *recoverFindTable(sqlite3_recover *p, u32 iRoot){ RecoverTable *pRet = 0; for(pRet=p->pTblList; pRet && pRet->iRoot!=iRoot; pRet=pRet->pNext); return pRet; } /* ** This function attempts to create a lost and found table within the ** output db. If successful, it returns a pointer to a buffer containing ** the name of the new table. It is the responsibility of the caller to ** eventually free this buffer using sqlite3_free(). ** ** If an error occurs, NULL is returned and an error code and error ** message left in the recover handle. */ static char *recoverLostAndFoundCreate( sqlite3_recover *p, /* Recover object */ int nField /* Number of column fields in new table */ ){ char *zTbl = 0; sqlite3_stmt *pProbe = 0; int ii = 0; pProbe = recoverPrepare(p, p->dbOut, "SELECT 1 FROM sqlite_schema WHERE name=?" ); for(ii=-1; zTbl==0 && p->errCode==SQLITE_OK && ii<1000; ii++){ int bFail = 0; if( ii<0 ){ zTbl = recoverMPrintf(p, "%s", p->zLostAndFound); }else{ zTbl = recoverMPrintf(p, "%s_%d", p->zLostAndFound, ii); } if( p->errCode==SQLITE_OK ){ sqlite3_bind_text(pProbe, 1, zTbl, -1, SQLITE_STATIC); if( SQLITE_ROW==sqlite3_step(pProbe) ){ bFail = 1; } recoverReset(p, pProbe); } if( bFail ){ sqlite3_clear_bindings(pProbe); sqlite3_free(zTbl); zTbl = 0; } } recoverFinalize(p, pProbe); if( zTbl ){ const char *zSep = 0; char *zField = 0; char *zSql = 0; zSep = "rootpgno INTEGER, pgno INTEGER, nfield INTEGER, id INTEGER, "; for(ii=0; p->errCode==SQLITE_OK && ii<nField; ii++){ zField = recoverMPrintf(p, "%z%sc%d", zField, zSep, ii); zSep = ", "; } zSql = recoverMPrintf(p, "CREATE TABLE %s(%s)", zTbl, zField); sqlite3_free(zField); recoverExec(p, p->dbOut, zSql); recoverSqlCallback(p, zSql); sqlite3_free(zSql); }else if( p->errCode==SQLITE_OK ){ recoverError( p, SQLITE_ERROR, "failed to create %s output table", p->zLostAndFound ); } return zTbl; } /* ** Synthesize and prepare an INSERT statement to write to the lost_and_found ** table in the output database. The name of the table is zTab, and it has ** nField c* fields. */ static sqlite3_stmt *recoverLostAndFoundInsert( sqlite3_recover *p, const char *zTab, int nField ){ int nTotal = nField + 4; int ii; char *zBind = 0; sqlite3_stmt *pRet = 0; if( p->xSql==0 ){ for(ii=0; ii<nTotal; ii++){ zBind = recoverMPrintf(p, "%z%s?", zBind, zBind?", ":"", ii); } pRet = recoverPreparePrintf( p, p->dbOut, "INSERT INTO %s VALUES(%s)", zTab, zBind ); }else{ const char *zSep = ""; for(ii=0; ii<nTotal; ii++){ zBind = recoverMPrintf(p, "%z%squote(?)", zBind, zSep); zSep = "|| ', ' ||"; } pRet = recoverPreparePrintf( p, p->dbOut, "SELECT 'INSERT INTO %s VALUES(' || %s || ')'", zTab, zBind ); } sqlite3_free(zBind); return pRet; } /* ** Input database page iPg contains data that will be written to the ** lost-and-found table of the output database. This function attempts ** to identify the root page of the tree that page iPg belonged to. ** If successful, it sets output variable (*piRoot) to the page number ** of the root page and returns SQLITE_OK. Otherwise, if an error occurs, ** an SQLite error code is returned and the final value of *piRoot ** undefined. */ static int recoverLostAndFoundFindRoot( sqlite3_recover *p, i64 iPg, i64 *piRoot ){ RecoverStateLAF *pLaf = &p->laf; if( pLaf->pFindRoot==0 ){ pLaf->pFindRoot = recoverPrepare(p, p->dbOut, "WITH RECURSIVE p(pgno) AS (" " SELECT ?" " UNION" " SELECT parent FROM recovery.map AS m, p WHERE m.pgno=p.pgno" ") " "SELECT p.pgno FROM p, recovery.map m WHERE m.pgno=p.pgno " " AND m.parent IS NULL" ); } if( p->errCode==SQLITE_OK ){ sqlite3_bind_int64(pLaf->pFindRoot, 1, iPg); if( sqlite3_step(pLaf->pFindRoot)==SQLITE_ROW ){ *piRoot = sqlite3_column_int64(pLaf->pFindRoot, 0); }else{ *piRoot = iPg; } recoverReset(p, pLaf->pFindRoot); } return p->errCode; } /* ** Recover data from page iPage of the input database and write it to ** the lost-and-found table in the output database. */ static void recoverLostAndFoundOnePage(sqlite3_recover *p, i64 iPage){ RecoverStateLAF *pLaf = &p->laf; sqlite3_value **apVal = pLaf->apVal; sqlite3_stmt *pPageData = pLaf->pPageData; sqlite3_stmt *pInsert = pLaf->pInsert; int nVal = -1; int iPrevCell = 0; i64 iRoot = 0; int bHaveRowid = 0; i64 iRowid = 0; int ii = 0; if( recoverLostAndFoundFindRoot(p, iPage, &iRoot) ) return; sqlite3_bind_int64(pPageData, 1, iPage); while( p->errCode==SQLITE_OK && SQLITE_ROW==sqlite3_step(pPageData) ){ int iCell = sqlite3_column_int64(pPageData, 0); int iField = sqlite3_column_int64(pPageData, 1); if( iPrevCell!=iCell && nVal>=0 ){ /* Insert the new row */ sqlite3_bind_int64(pInsert, 1, iRoot); /* rootpgno */ sqlite3_bind_int64(pInsert, 2, iPage); /* pgno */ sqlite3_bind_int(pInsert, 3, nVal); /* nfield */ if( bHaveRowid ){ sqlite3_bind_int64(pInsert, 4, iRowid); /* id */ } for(ii=0; ii<nVal; ii++){ recoverBindValue(p, pInsert, 5+ii, apVal[ii]); } if( sqlite3_step(pInsert)==SQLITE_ROW ){ recoverSqlCallback(p, (const char*)sqlite3_column_text(pInsert, 0)); } recoverReset(p, pInsert); /* Discard the accumulated row data */ for(ii=0; ii<nVal; ii++){ sqlite3_value_free(apVal[ii]); apVal[ii] = 0; } sqlite3_clear_bindings(pInsert); bHaveRowid = 0; nVal = -1; } if( iCell<0 ) break; if( iField<0 ){ assert( nVal==-1 ); iRowid = sqlite3_column_int64(pPageData, 2); bHaveRowid = 1; nVal = 0; }else if( iField<pLaf->nMaxField ){ sqlite3_value *pVal = sqlite3_column_value(pPageData, 2); apVal[iField] = sqlite3_value_dup(pVal); assert( iField==nVal || (nVal==-1 && iField==0) ); nVal = iField+1; if( apVal[iField]==0 ){ recoverError(p, SQLITE_NOMEM, 0); } } iPrevCell = iCell; } recoverReset(p, pPageData); for(ii=0; ii<nVal; ii++){ sqlite3_value_free(apVal[ii]); apVal[ii] = 0; } } /* ** Perform one step (sqlite3_recover_step()) of work for the connection ** passed as the only argument, which is guaranteed to be in ** RECOVER_STATE_LOSTANDFOUND3 state - during which the lost-and-found ** table of the output database is populated with recovered data that can ** not be assigned to any recovered schema object. */ static int recoverLostAndFound3Step(sqlite3_recover *p){ RecoverStateLAF *pLaf = &p->laf; if( p->errCode==SQLITE_OK ){ if( pLaf->pInsert==0 ){ return SQLITE_DONE; }else{ if( p->errCode==SQLITE_OK ){ int res = sqlite3_step(pLaf->pAllPage); if( res==SQLITE_ROW ){ i64 iPage = sqlite3_column_int64(pLaf->pAllPage, 0); if( recoverBitmapQuery(pLaf->pUsed, iPage)==0 ){ recoverLostAndFoundOnePage(p, iPage); } }else{ recoverReset(p, pLaf->pAllPage); return SQLITE_DONE; } } } } return SQLITE_OK; } /* ** Initialize resources required in RECOVER_STATE_LOSTANDFOUND3 ** state - during which the lost-and-found table of the output database ** is populated with recovered data that can not be assigned to any ** recovered schema object. */ static void recoverLostAndFound3Init(sqlite3_recover *p){ RecoverStateLAF *pLaf = &p->laf; if( pLaf->nMaxField>0 ){ char *zTab = 0; /* Name of lost_and_found table */ zTab = recoverLostAndFoundCreate(p, pLaf->nMaxField); pLaf->pInsert = recoverLostAndFoundInsert(p, zTab, pLaf->nMaxField); sqlite3_free(zTab); pLaf->pAllPage = recoverPreparePrintf(p, p->dbOut, "WITH RECURSIVE seq(ii) AS (" " SELECT 1 UNION ALL SELECT ii+1 FROM seq WHERE ii<%lld" ")" "SELECT ii FROM seq" , p->laf.nPg ); pLaf->pPageData = recoverPrepare(p, p->dbOut, "SELECT cell, field, value " "FROM sqlite_dbdata('getpage()') d WHERE d.pgno=? " "UNION ALL " "SELECT -1, -1, -1" ); pLaf->apVal = (sqlite3_value**)recoverMalloc(p, pLaf->nMaxField*sizeof(sqlite3_value*) ); } } /* ** Initialize resources required in RECOVER_STATE_WRITING state - during which ** tables recovered from the schema of the input database are populated with ** recovered data. */ static int recoverWriteDataInit(sqlite3_recover *p){ RecoverStateW1 *p1 = &p->w1; RecoverTable *pTbl = 0; int nByte = 0; /* Figure out the maximum number of columns for any table in the schema */ assert( p1->nMax==0 ); for(pTbl=p->pTblList; pTbl; pTbl=pTbl->pNext){ if( pTbl->nCol>p1->nMax ) p1->nMax = pTbl->nCol; } /* Allocate an array of (sqlite3_value*) in which to accumulate the values ** that will be written to the output database in a single row. */ nByte = sizeof(sqlite3_value*) * (p1->nMax+1); p1->apVal = (sqlite3_value**)recoverMalloc(p, nByte); if( p1->apVal==0 ) return p->errCode; /* Prepare the SELECT to loop through schema tables (pTbls) and the SELECT ** to loop through cells that appear to belong to a single table (pSel). */ p1->pTbls = recoverPrepare(p, p->dbOut, "SELECT rootpage FROM recovery.schema " " WHERE type='table' AND (sql NOT LIKE 'create virtual%')" " ORDER BY (tbl_name='sqlite_sequence') ASC" ); p1->pSel = recoverPrepare(p, p->dbOut, "WITH RECURSIVE pages(page) AS (" " SELECT ?1" " UNION" " SELECT child FROM sqlite_dbptr('getpage()'), pages " " WHERE pgno=page" ") " "SELECT page, cell, field, value " "FROM sqlite_dbdata('getpage()') d, pages p WHERE p.page=d.pgno " "UNION ALL " "SELECT 0, 0, 0, 0" ); return p->errCode; } /* ** Clean up resources allocated by recoverWriteDataInit() (stuff in ** sqlite3_recover.w1). */ static void recoverWriteDataCleanup(sqlite3_recover *p){ RecoverStateW1 *p1 = &p->w1; int ii; for(ii=0; ii<p1->nVal; ii++){ sqlite3_value_free(p1->apVal[ii]); } sqlite3_free(p1->apVal); recoverFinalize(p, p1->pInsert); recoverFinalize(p, p1->pTbls); recoverFinalize(p, p1->pSel); memset(p1, 0, sizeof(*p1)); } /* ** Perform one step (sqlite3_recover_step()) of work for the connection ** passed as the only argument, which is guaranteed to be in ** RECOVER_STATE_WRITING state - during which tables recovered from the ** schema of the input database are populated with recovered data. */ static int recoverWriteDataStep(sqlite3_recover *p){ RecoverStateW1 *p1 = &p->w1; sqlite3_stmt *pSel = p1->pSel; sqlite3_value **apVal = p1->apVal; if( p->errCode==SQLITE_OK && p1->pTab==0 ){ if( sqlite3_step(p1->pTbls)==SQLITE_ROW ){ i64 iRoot = sqlite3_column_int64(p1->pTbls, 0); p1->pTab = recoverFindTable(p, iRoot); recoverFinalize(p, p1->pInsert); p1->pInsert = 0; /* If this table is unknown, return early. The caller will invoke this ** function again and it will move on to the next table. */ if( p1->pTab==0 ) return p->errCode; /* If this is the sqlite_sequence table, delete any rows added by ** earlier INSERT statements on tables with AUTOINCREMENT primary ** keys before recovering its contents. The p1->pTbls SELECT statement ** is rigged to deliver "sqlite_sequence" last of all, so we don't ** worry about it being modified after it is recovered. */ if( sqlite3_stricmp("sqlite_sequence", p1->pTab->zTab)==0 ){ recoverExec(p, p->dbOut, "DELETE FROM sqlite_sequence"); recoverSqlCallback(p, "DELETE FROM sqlite_sequence"); } /* Bind the root page of this table within the original database to ** SELECT statement p1->pSel. The SELECT statement will then iterate ** through cells that look like they belong to table pTab. */ sqlite3_bind_int64(pSel, 1, iRoot); p1->nVal = 0; p1->bHaveRowid = 0; p1->iPrevPage = -1; p1->iPrevCell = -1; }else{ return SQLITE_DONE; } } assert( p->errCode!=SQLITE_OK || p1->pTab ); if( p->errCode==SQLITE_OK && sqlite3_step(pSel)==SQLITE_ROW ){ RecoverTable *pTab = p1->pTab; i64 iPage = sqlite3_column_int64(pSel, 0); int iCell = sqlite3_column_int(pSel, 1); int iField = sqlite3_column_int(pSel, 2); sqlite3_value *pVal = sqlite3_column_value(pSel, 3); int bNewCell = (p1->iPrevPage!=iPage || p1->iPrevCell!=iCell); assert( bNewCell==0 || (iField==-1 || iField==0) ); assert( bNewCell || iField==p1->nVal || p1->nVal==pTab->nCol ); if( bNewCell ){ int ii = 0; if( p1->nVal>=0 ){ if( p1->pInsert==0 || p1->nVal!=p1->nInsert ){ recoverFinalize(p, p1->pInsert); p1->pInsert = recoverInsertStmt(p, pTab, p1->nVal); p1->nInsert = p1->nVal; } if( p1->nVal>0 ){ sqlite3_stmt *pInsert = p1->pInsert; for(ii=0; ii<pTab->nCol; ii++){ RecoverColumn *pCol = &pTab->aCol[ii]; int iBind = pCol->iBind; if( iBind>0 ){ if( pCol->bIPK ){ sqlite3_bind_int64(pInsert, iBind, p1->iRowid); }else if( pCol->iField<p1->nVal ){ recoverBindValue(p, pInsert, iBind, apVal[pCol->iField]); } } } if( p->bRecoverRowid && pTab->iRowidBind>0 && p1->bHaveRowid ){ sqlite3_bind_int64(pInsert, pTab->iRowidBind, p1->iRowid); } if( SQLITE_ROW==sqlite3_step(pInsert) ){ const char *z = (const char*)sqlite3_column_text(pInsert, 0); recoverSqlCallback(p, z); } recoverReset(p, pInsert); assert( p->errCode || pInsert ); if( pInsert ) sqlite3_clear_bindings(pInsert); } } for(ii=0; ii<p1->nVal; ii++){ sqlite3_value_free(apVal[ii]); apVal[ii] = 0; } p1->nVal = -1; p1->bHaveRowid = 0; } if( iPage!=0 ){ if( iField<0 ){ p1->iRowid = sqlite3_column_int64(pSel, 3); assert( p1->nVal==-1 ); p1->nVal = 0; p1->bHaveRowid = 1; }else if( iField<pTab->nCol ){ assert( apVal[iField]==0 ); apVal[iField] = sqlite3_value_dup( pVal ); if( apVal[iField]==0 ){ recoverError(p, SQLITE_NOMEM, 0); } p1->nVal = iField+1; } p1->iPrevCell = iCell; p1->iPrevPage = iPage; } }else{ recoverReset(p, pSel); p1->pTab = 0; } return p->errCode; } /* ** Initialize resources required by sqlite3_recover_step() in ** RECOVER_STATE_LOSTANDFOUND1 state - during which the set of pages not ** already allocated to a recovered schema element is determined. */ static void recoverLostAndFound1Init(sqlite3_recover *p){ RecoverStateLAF *pLaf = &p->laf; sqlite3_stmt *pStmt = 0; assert( p->laf.pUsed==0 ); pLaf->nPg = recoverPageCount(p); pLaf->pUsed = recoverBitmapAlloc(p, pLaf->nPg); /* Prepare a statement to iterate through all pages that are part of any tree ** in the recoverable part of the input database schema to the bitmap. And, ** if !p->bFreelistCorrupt, add all pages that appear to be part of the ** freelist. */ pStmt = recoverPrepare( p, p->dbOut, "WITH trunk(pgno) AS (" " SELECT read_i32(getpage(1), 8) AS x WHERE x>0" " UNION" " SELECT read_i32(getpage(trunk.pgno), 0) AS x FROM trunk WHERE x>0" ")," "trunkdata(pgno, data) AS (" " SELECT pgno, getpage(pgno) FROM trunk" ")," "freelist(data, n, freepgno) AS (" " SELECT data, min(16384, read_i32(data, 1)-1), pgno FROM trunkdata" " UNION ALL" " SELECT data, n-1, read_i32(data, 2+n) FROM freelist WHERE n>=0" ")," "" "roots(r) AS (" " SELECT 1 UNION ALL" " SELECT rootpage FROM recovery.schema WHERE rootpage>0" ")," "used(page) AS (" " SELECT r FROM roots" " UNION" " SELECT child FROM sqlite_dbptr('getpage()'), used " " WHERE pgno=page" ") " "SELECT page FROM used" " UNION ALL " "SELECT freepgno FROM freelist WHERE NOT ?" ); if( pStmt ) sqlite3_bind_int(pStmt, 1, p->bFreelistCorrupt); pLaf->pUsedPages = pStmt; } /* ** Perform one step (sqlite3_recover_step()) of work for the connection ** passed as the only argument, which is guaranteed to be in ** RECOVER_STATE_LOSTANDFOUND1 state - during which the set of pages not ** already allocated to a recovered schema element is determined. */ static int recoverLostAndFound1Step(sqlite3_recover *p){ RecoverStateLAF *pLaf = &p->laf; int rc = p->errCode; if( rc==SQLITE_OK ){ rc = sqlite3_step(pLaf->pUsedPages); if( rc==SQLITE_ROW ){ i64 iPg = sqlite3_column_int64(pLaf->pUsedPages, 0); recoverBitmapSet(pLaf->pUsed, iPg); rc = SQLITE_OK; }else{ recoverFinalize(p, pLaf->pUsedPages); pLaf->pUsedPages = 0; } } return rc; } /* ** Initialize resources required by RECOVER_STATE_LOSTANDFOUND2 ** state - during which the pages identified in RECOVER_STATE_LOSTANDFOUND1 ** are sorted into sets that likely belonged to the same database tree. */ static void recoverLostAndFound2Init(sqlite3_recover *p){ RecoverStateLAF *pLaf = &p->laf; assert( p->laf.pAllAndParent==0 ); assert( p->laf.pMapInsert==0 ); assert( p->laf.pMaxField==0 ); assert( p->laf.nMaxField==0 ); pLaf->pMapInsert = recoverPrepare(p, p->dbOut, "INSERT OR IGNORE INTO recovery.map(pgno, parent) VALUES(?, ?)" ); pLaf->pAllAndParent = recoverPreparePrintf(p, p->dbOut, "WITH RECURSIVE seq(ii) AS (" " SELECT 1 UNION ALL SELECT ii+1 FROM seq WHERE ii<%lld" ")" "SELECT pgno, child FROM sqlite_dbptr('getpage()') " " UNION ALL " "SELECT NULL, ii FROM seq", p->laf.nPg ); pLaf->pMaxField = recoverPreparePrintf(p, p->dbOut, "SELECT max(field)+1 FROM sqlite_dbdata('getpage') WHERE pgno = ?" ); } /* ** Perform one step (sqlite3_recover_step()) of work for the connection ** passed as the only argument, which is guaranteed to be in ** RECOVER_STATE_LOSTANDFOUND2 state - during which the pages identified ** in RECOVER_STATE_LOSTANDFOUND1 are sorted into sets that likely belonged ** to the same database tree. */ static int recoverLostAndFound2Step(sqlite3_recover *p){ RecoverStateLAF *pLaf = &p->laf; if( p->errCode==SQLITE_OK ){ int res = sqlite3_step(pLaf->pAllAndParent); if( res==SQLITE_ROW ){ i64 iChild = sqlite3_column_int(pLaf->pAllAndParent, 1); if( recoverBitmapQuery(pLaf->pUsed, iChild)==0 ){ sqlite3_bind_int64(pLaf->pMapInsert, 1, iChild); sqlite3_bind_value(pLaf->pMapInsert, 2, sqlite3_column_value(pLaf->pAllAndParent, 0) ); sqlite3_step(pLaf->pMapInsert); recoverReset(p, pLaf->pMapInsert); sqlite3_bind_int64(pLaf->pMaxField, 1, iChild); if( SQLITE_ROW==sqlite3_step(pLaf->pMaxField) ){ int nMax = sqlite3_column_int(pLaf->pMaxField, 0); if( nMax>pLaf->nMaxField ) pLaf->nMaxField = nMax; } recoverReset(p, pLaf->pMaxField); } }else{ recoverFinalize(p, pLaf->pAllAndParent); pLaf->pAllAndParent =0; return SQLITE_DONE; } } return p->errCode; } /* ** Free all resources allocated as part of sqlite3_recover_step() calls ** in one of the RECOVER_STATE_LOSTANDFOUND[123] states. */ static void recoverLostAndFoundCleanup(sqlite3_recover *p){ recoverBitmapFree(p->laf.pUsed); p->laf.pUsed = 0; sqlite3_finalize(p->laf.pUsedPages); sqlite3_finalize(p->laf.pAllAndParent); sqlite3_finalize(p->laf.pMapInsert); sqlite3_finalize(p->laf.pMaxField); sqlite3_finalize(p->laf.pFindRoot); sqlite3_finalize(p->laf.pInsert); sqlite3_finalize(p->laf.pAllPage); sqlite3_finalize(p->laf.pPageData); p->laf.pUsedPages = 0; p->laf.pAllAndParent = 0; p->laf.pMapInsert = 0; p->laf.pMaxField = 0; p->laf.pFindRoot = 0; p->laf.pInsert = 0; p->laf.pAllPage = 0; p->laf.pPageData = 0; sqlite3_free(p->laf.apVal); p->laf.apVal = 0; } /* ** Free all resources allocated as part of sqlite3_recover_step() calls. */ static void recoverFinalCleanup(sqlite3_recover *p){ RecoverTable *pTab = 0; RecoverTable *pNext = 0; recoverWriteDataCleanup(p); recoverLostAndFoundCleanup(p); for(pTab=p->pTblList; pTab; pTab=pNext){ pNext = pTab->pNext; sqlite3_free(pTab); } p->pTblList = 0; sqlite3_finalize(p->pGetPage); p->pGetPage = 0; sqlite3_file_control(p->dbIn, p->zDb, SQLITE_FCNTL_RESET_CACHE, 0); { #ifndef NDEBUG int res = #endif sqlite3_close(p->dbOut); assert( res==SQLITE_OK ); } p->dbOut = 0; } /* ** Decode and return an unsigned 16-bit big-endian integer value from ** buffer a[]. */ static u32 recoverGetU16(const u8 *a){ return (((u32)a[0])<<8) + ((u32)a[1]); } /* ** Decode and return an unsigned 32-bit big-endian integer value from ** buffer a[]. */ static u32 recoverGetU32(const u8 *a){ return (((u32)a[0])<<24) + (((u32)a[1])<<16) + (((u32)a[2])<<8) + ((u32)a[3]); } /* ** Decode an SQLite varint from buffer a[]. Write the decoded value to (*pVal) ** and return the number of bytes consumed. */ static int recoverGetVarint(const u8 *a, i64 *pVal){ sqlite3_uint64 u = 0; int i; for(i=0; i<8; i++){ u = (u<<7) + (a[i]&0x7f); if( (a[i]&0x80)==0 ){ *pVal = (sqlite3_int64)u; return i+1; } } u = (u<<8) + (a[i]&0xff); *pVal = (sqlite3_int64)u; return 9; } /* ** The second argument points to a buffer n bytes in size. If this buffer ** or a prefix thereof appears to contain a well-formed SQLite b-tree page, ** return the page-size in bytes. Otherwise, if the buffer does not ** appear to contain a well-formed b-tree page, return 0. */ static int recoverIsValidPage(u8 *aTmp, const u8 *a, int n){ u8 *aUsed = aTmp; int nFrag = 0; int nActual = 0; int iFree = 0; int nCell = 0; /* Number of cells on page */ int iCellOff = 0; /* Offset of cell array in page */ int iContent = 0; int eType = 0; int ii = 0; eType = (int)a[0]; if( eType!=0x02 && eType!=0x05 && eType!=0x0A && eType!=0x0D ) return 0; iFree = (int)recoverGetU16(&a[1]); nCell = (int)recoverGetU16(&a[3]); iContent = (int)recoverGetU16(&a[5]); if( iContent==0 ) iContent = 65536; nFrag = (int)a[7]; if( iContent>n ) return 0; memset(aUsed, 0, n); memset(aUsed, 0xFF, iContent); /* Follow the free-list. This is the same format for all b-tree pages. */ if( iFree && iFree<=iContent ) return 0; while( iFree ){ int iNext = 0; int nByte = 0; if( iFree>(n-4) ) return 0; iNext = recoverGetU16(&a[iFree]); nByte = recoverGetU16(&a[iFree+2]); if( iFree+nByte>n ) return 0; if( iNext && iNext<iFree+nByte ) return 0; memset(&aUsed[iFree], 0xFF, nByte); iFree = iNext; } /* Run through the cells */ if( eType==0x02 || eType==0x05 ){ iCellOff = 12; }else{ iCellOff = 8; } if( (iCellOff + 2*nCell)>iContent ) return 0; for(ii=0; ii<nCell; ii++){ int iByte; i64 nPayload = 0; int nByte = 0; int iOff = recoverGetU16(&a[iCellOff + 2*ii]); if( iOff<iContent || iOff>n ){ return 0; } if( eType==0x05 || eType==0x02 ) nByte += 4; nByte += recoverGetVarint(&a[iOff+nByte], &nPayload); if( eType==0x0D ){ i64 dummy = 0; nByte += recoverGetVarint(&a[iOff+nByte], &dummy); } if( eType!=0x05 ){ int X = (eType==0x0D) ? n-35 : (((n-12)*64/255)-23); int M = ((n-12)*32/255)-23; int K = M+((nPayload-M)%(n-4)); if( nPayload<X ){ nByte += nPayload; }else if( K<=X ){ nByte += K+4; }else{ nByte += M+4; } } if( iOff+nByte>n ){ return 0; } for(iByte=iOff; iByte<(iOff+nByte); iByte++){ if( aUsed[iByte]!=0 ){ return 0; } aUsed[iByte] = 0xFF; } } nActual = 0; for(ii=0; ii<n; ii++){ if( aUsed[ii]==0 ) nActual++; } return (nActual==nFrag); } static int recoverVfsClose(sqlite3_file*); static int recoverVfsRead(sqlite3_file*, void*, int iAmt, sqlite3_int64 iOfst); static int recoverVfsWrite(sqlite3_file*, const void*, int, sqlite3_int64); static int recoverVfsTruncate(sqlite3_file*, sqlite3_int64 size); static int recoverVfsSync(sqlite3_file*, int flags); static int recoverVfsFileSize(sqlite3_file*, sqlite3_int64 *pSize); static int recoverVfsLock(sqlite3_file*, int); static int recoverVfsUnlock(sqlite3_file*, int); static int recoverVfsCheckReservedLock(sqlite3_file*, int *pResOut); static int recoverVfsFileControl(sqlite3_file*, int op, void *pArg); static int recoverVfsSectorSize(sqlite3_file*); static int recoverVfsDeviceCharacteristics(sqlite3_file*); static int recoverVfsShmMap(sqlite3_file*, int, int, int, void volatile**); static int recoverVfsShmLock(sqlite3_file*, int offset, int n, int flags); static void recoverVfsShmBarrier(sqlite3_file*); static int recoverVfsShmUnmap(sqlite3_file*, int deleteFlag); static int recoverVfsFetch(sqlite3_file*, sqlite3_int64, int, void**); static int recoverVfsUnfetch(sqlite3_file *pFd, sqlite3_int64 iOff, void *p); static sqlite3_io_methods recover_methods = { 2, /* iVersion */ recoverVfsClose, recoverVfsRead, recoverVfsWrite, recoverVfsTruncate, recoverVfsSync, recoverVfsFileSize, recoverVfsLock, recoverVfsUnlock, recoverVfsCheckReservedLock, recoverVfsFileControl, recoverVfsSectorSize, recoverVfsDeviceCharacteristics, recoverVfsShmMap, recoverVfsShmLock, recoverVfsShmBarrier, recoverVfsShmUnmap, recoverVfsFetch, recoverVfsUnfetch }; static int recoverVfsClose(sqlite3_file *pFd){ assert( pFd->pMethods!=&recover_methods ); return pFd->pMethods->xClose(pFd); } /* ** Write value v to buffer a[] as a 16-bit big-endian unsigned integer. */ static void recoverPutU16(u8 *a, u32 v){ a[0] = (v>>8) & 0x00FF; a[1] = (v>>0) & 0x00FF; } /* ** Write value v to buffer a[] as a 32-bit big-endian unsigned integer. */ static void recoverPutU32(u8 *a, u32 v){ a[0] = (v>>24) & 0x00FF; a[1] = (v>>16) & 0x00FF; a[2] = (v>>8) & 0x00FF; a[3] = (v>>0) & 0x00FF; } /* ** Detect the page-size of the database opened by file-handle pFd by ** searching the first part of the file for a well-formed SQLite b-tree ** page. If parameter nReserve is non-zero, then as well as searching for ** a b-tree page with zero reserved bytes, this function searches for one ** with nReserve reserved bytes at the end of it. ** ** If successful, set variable p->detected_pgsz to the detected page-size ** in bytes and return SQLITE_OK. Or, if no error occurs but no valid page ** can be found, return SQLITE_OK but leave p->detected_pgsz set to 0. Or, ** if an error occurs (e.g. an IO or OOM error), then an SQLite error code ** is returned. The final value of p->detected_pgsz is undefined in this ** case. */ static int recoverVfsDetectPagesize( sqlite3_recover *p, /* Recover handle */ sqlite3_file *pFd, /* File-handle open on input database */ u32 nReserve, /* Possible nReserve value */ i64 nSz /* Size of database file in bytes */ ){ int rc = SQLITE_OK; const int nMin = 512; const int nMax = 65536; const int nMaxBlk = 4; u32 pgsz = 0; int iBlk = 0; u8 *aPg = 0; u8 *aTmp = 0; int nBlk = 0; aPg = (u8*)sqlite3_malloc(2*nMax); if( aPg==0 ) return SQLITE_NOMEM; aTmp = &aPg[nMax]; nBlk = (nSz+nMax-1)/nMax; if( nBlk>nMaxBlk ) nBlk = nMaxBlk; do { for(iBlk=0; rc==SQLITE_OK && iBlk<nBlk; iBlk++){ int nByte = (nSz>=((iBlk+1)*nMax)) ? nMax : (nSz % nMax); memset(aPg, 0, nMax); rc = pFd->pMethods->xRead(pFd, aPg, nByte, iBlk*nMax); if( rc==SQLITE_OK ){ int pgsz2; for(pgsz2=(pgsz ? pgsz*2 : nMin); pgsz2<=nMax; pgsz2=pgsz2*2){ int iOff; for(iOff=0; iOff<nMax; iOff+=pgsz2){ if( recoverIsValidPage(aTmp, &aPg[iOff], pgsz2-nReserve) ){ pgsz = pgsz2; break; } } } } } if( pgsz>(u32)p->detected_pgsz ){ p->detected_pgsz = pgsz; p->nReserve = nReserve; } if( nReserve==0 ) break; nReserve = 0; }while( 1 ); p->detected_pgsz = pgsz; sqlite3_free(aPg); return rc; } /* ** The xRead() method of the wrapper VFS. This is used to intercept calls ** to read page 1 of the input database. */ static int recoverVfsRead(sqlite3_file *pFd, void *aBuf, int nByte, i64 iOff){ int rc = SQLITE_OK; if( pFd->pMethods==&recover_methods ){ pFd->pMethods = recover_g.pMethods; rc = pFd->pMethods->xRead(pFd, aBuf, nByte, iOff); if( nByte==16 ){ sqlite3_randomness(16, aBuf); }else if( rc==SQLITE_OK && iOff==0 && nByte>=108 ){ /* Ensure that the database has a valid header file. The only fields ** that really matter to recovery are: ** ** + Database page size (16-bits at offset 16) ** + Size of db in pages (32-bits at offset 28) ** + Database encoding (32-bits at offset 56) ** ** Also preserved are: ** ** + first freelist page (32-bits at offset 32) ** + size of freelist (32-bits at offset 36) ** + the wal-mode flags (16-bits at offset 18) ** ** We also try to preserve the auto-vacuum, incr-value, user-version ** and application-id fields - all 32 bit quantities at offsets ** 52, 60, 64 and 68. All other fields are set to known good values. ** ** Byte offset 105 should also contain the page-size as a 16-bit ** integer. */ const int aPreserve[] = {32, 36, 52, 60, 64, 68}; u8 aHdr[108] = { 0x53, 0x51, 0x4c, 0x69, 0x74, 0x65, 0x20, 0x66, 0x6f, 0x72, 0x6d, 0x61, 0x74, 0x20, 0x33, 0x00, 0xFF, 0xFF, 0x01, 0x01, 0x00, 0x40, 0x20, 0x20, 0x00, 0x00, 0x00, 0x00, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x04, 0x00, 0x00, 0x10, 0x00, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x2e, 0x5b, 0x30, 0x0D, 0x00, 0x00, 0x00, 0x00, 0xFF, 0xFF, 0x00 }; u8 *a = (u8*)aBuf; u32 pgsz = recoverGetU16(&a[16]); u32 nReserve = a[20]; u32 enc = recoverGetU32(&a[56]); u32 dbsz = 0; i64 dbFileSize = 0; int ii; sqlite3_recover *p = recover_g.p; if( pgsz==0x01 ) pgsz = 65536; rc = pFd->pMethods->xFileSize(pFd, &dbFileSize); if( rc==SQLITE_OK && p->detected_pgsz==0 ){ rc = recoverVfsDetectPagesize(p, pFd, nReserve, dbFileSize); } if( p->detected_pgsz ){ pgsz = p->detected_pgsz; nReserve = p->nReserve; } if( pgsz ){ dbsz = dbFileSize / pgsz; } if( enc!=SQLITE_UTF8 && enc!=SQLITE_UTF16BE && enc!=SQLITE_UTF16LE ){ enc = SQLITE_UTF8; } sqlite3_free(p->pPage1Cache); p->pPage1Cache = 0; p->pPage1Disk = 0; p->pgsz = nByte; p->pPage1Cache = (u8*)recoverMalloc(p, nByte*2); if( p->pPage1Cache ){ p->pPage1Disk = &p->pPage1Cache[nByte]; memcpy(p->pPage1Disk, aBuf, nByte); aHdr[18] = a[18]; aHdr[19] = a[19]; recoverPutU32(&aHdr[28], dbsz); recoverPutU32(&aHdr[56], enc); recoverPutU16(&aHdr[105], pgsz-nReserve); if( pgsz==65536 ) pgsz = 1; recoverPutU16(&aHdr[16], pgsz); aHdr[20] = nReserve; for(ii=0; ii<(int)(sizeof(aPreserve)/sizeof(aPreserve[0])); ii++){ memcpy(&aHdr[aPreserve[ii]], &a[aPreserve[ii]], 4); } memcpy(aBuf, aHdr, sizeof(aHdr)); memset(&((u8*)aBuf)[sizeof(aHdr)], 0, nByte-sizeof(aHdr)); memcpy(p->pPage1Cache, aBuf, nByte); }else{ rc = p->errCode; } } pFd->pMethods = &recover_methods; }else{ rc = pFd->pMethods->xRead(pFd, aBuf, nByte, iOff); } return rc; } /* ** Used to make sqlite3_io_methods wrapper methods less verbose. */ #define RECOVER_VFS_WRAPPER(code) \ int rc = SQLITE_OK; \ if( pFd->pMethods==&recover_methods ){ \ pFd->pMethods = recover_g.pMethods; \ rc = code; \ pFd->pMethods = &recover_methods; \ }else{ \ rc = code; \ } \ return rc; /* ** Methods of the wrapper VFS. All methods except for xRead() and xClose() ** simply uninstall the sqlite3_io_methods wrapper, invoke the equivalent ** method on the lower level VFS, then reinstall the wrapper before returning. ** Those that return an integer value use the RECOVER_VFS_WRAPPER macro. */ static int recoverVfsWrite( sqlite3_file *pFd, const void *aBuf, int nByte, i64 iOff ){ RECOVER_VFS_WRAPPER ( pFd->pMethods->xWrite(pFd, aBuf, nByte, iOff) ); } static int recoverVfsTruncate(sqlite3_file *pFd, sqlite3_int64 size){ RECOVER_VFS_WRAPPER ( pFd->pMethods->xTruncate(pFd, size) ); } static int recoverVfsSync(sqlite3_file *pFd, int flags){ RECOVER_VFS_WRAPPER ( pFd->pMethods->xSync(pFd, flags) ); } static int recoverVfsFileSize(sqlite3_file *pFd, sqlite3_int64 *pSize){ RECOVER_VFS_WRAPPER ( pFd->pMethods->xFileSize(pFd, pSize) ); } static int recoverVfsLock(sqlite3_file *pFd, int eLock){ RECOVER_VFS_WRAPPER ( pFd->pMethods->xLock(pFd, eLock) ); } static int recoverVfsUnlock(sqlite3_file *pFd, int eLock){ RECOVER_VFS_WRAPPER ( pFd->pMethods->xUnlock(pFd, eLock) ); } static int recoverVfsCheckReservedLock(sqlite3_file *pFd, int *pResOut){ RECOVER_VFS_WRAPPER ( pFd->pMethods->xCheckReservedLock(pFd, pResOut) ); } static int recoverVfsFileControl(sqlite3_file *pFd, int op, void *pArg){ RECOVER_VFS_WRAPPER ( (pFd->pMethods ? pFd->pMethods->xFileControl(pFd, op, pArg) : SQLITE_NOTFOUND) ); } static int recoverVfsSectorSize(sqlite3_file *pFd){ RECOVER_VFS_WRAPPER ( pFd->pMethods->xSectorSize(pFd) ); } static int recoverVfsDeviceCharacteristics(sqlite3_file *pFd){ RECOVER_VFS_WRAPPER ( pFd->pMethods->xDeviceCharacteristics(pFd) ); } static int recoverVfsShmMap( sqlite3_file *pFd, int iPg, int pgsz, int bExtend, void volatile **pp ){ RECOVER_VFS_WRAPPER ( pFd->pMethods->xShmMap(pFd, iPg, pgsz, bExtend, pp) ); } static int recoverVfsShmLock(sqlite3_file *pFd, int offset, int n, int flags){ RECOVER_VFS_WRAPPER ( pFd->pMethods->xShmLock(pFd, offset, n, flags) ); } static void recoverVfsShmBarrier(sqlite3_file *pFd){ if( pFd->pMethods==&recover_methods ){ pFd->pMethods = recover_g.pMethods; pFd->pMethods->xShmBarrier(pFd); pFd->pMethods = &recover_methods; }else{ pFd->pMethods->xShmBarrier(pFd); } } static int recoverVfsShmUnmap(sqlite3_file *pFd, int deleteFlag){ RECOVER_VFS_WRAPPER ( pFd->pMethods->xShmUnmap(pFd, deleteFlag) ); } static int recoverVfsFetch( sqlite3_file *pFd, sqlite3_int64 iOff, int iAmt, void **pp ){ (void)pFd; (void)iOff; (void)iAmt; *pp = 0; return SQLITE_OK; } static int recoverVfsUnfetch(sqlite3_file *pFd, sqlite3_int64 iOff, void *p){ (void)pFd; (void)iOff; (void)p; return SQLITE_OK; } /* ** Install the VFS wrapper around the file-descriptor open on the input ** database for recover handle p. Mutex RECOVER_MUTEX_ID must be held ** when this function is called. */ static void recoverInstallWrapper(sqlite3_recover *p){ sqlite3_file *pFd = 0; assert( recover_g.pMethods==0 ); recoverAssertMutexHeld(); sqlite3_file_control(p->dbIn, p->zDb, SQLITE_FCNTL_FILE_POINTER, (void*)&pFd); assert( pFd==0 || pFd->pMethods!=&recover_methods ); if( pFd && pFd->pMethods ){ int iVersion = 1 + (pFd->pMethods->iVersion>1 && pFd->pMethods->xShmMap!=0); recover_g.pMethods = pFd->pMethods; recover_g.p = p; recover_methods.iVersion = iVersion; pFd->pMethods = &recover_methods; } } /* ** Uninstall the VFS wrapper that was installed around the file-descriptor open ** on the input database for recover handle p. Mutex RECOVER_MUTEX_ID must be ** held when this function is called. */ static void recoverUninstallWrapper(sqlite3_recover *p){ sqlite3_file *pFd = 0; recoverAssertMutexHeld(); sqlite3_file_control(p->dbIn, p->zDb,SQLITE_FCNTL_FILE_POINTER,(void*)&pFd); if( pFd && pFd->pMethods ){ pFd->pMethods = recover_g.pMethods; recover_g.pMethods = 0; recover_g.p = 0; } } /* ** This function does the work of a single sqlite3_recover_step() call. It ** is guaranteed that the handle is not in an error state when this ** function is called. */ static void recoverStep(sqlite3_recover *p){ assert( p && p->errCode==SQLITE_OK ); switch( p->eState ){ case RECOVER_STATE_INIT: /* This is the very first call to sqlite3_recover_step() on this object. */ recoverSqlCallback(p, "BEGIN"); recoverSqlCallback(p, "PRAGMA writable_schema = on"); recoverEnterMutex(); recoverInstallWrapper(p); /* Open the output database. And register required virtual tables and ** user functions with the new handle. */ recoverOpenOutput(p); /* Open transactions on both the input and output databases. */ sqlite3_file_control(p->dbIn, p->zDb, SQLITE_FCNTL_RESET_CACHE, 0); recoverExec(p, p->dbIn, "PRAGMA writable_schema = on"); recoverExec(p, p->dbIn, "BEGIN"); if( p->errCode==SQLITE_OK ) p->bCloseTransaction = 1; recoverExec(p, p->dbIn, "SELECT 1 FROM sqlite_schema"); recoverTransferSettings(p); recoverOpenRecovery(p); recoverCacheSchema(p); recoverUninstallWrapper(p); recoverLeaveMutex(); recoverExec(p, p->dbOut, "BEGIN"); recoverWriteSchema1(p); p->eState = RECOVER_STATE_WRITING; break; case RECOVER_STATE_WRITING: { if( p->w1.pTbls==0 ){ recoverWriteDataInit(p); } if( SQLITE_DONE==recoverWriteDataStep(p) ){ recoverWriteDataCleanup(p); if( p->zLostAndFound ){ p->eState = RECOVER_STATE_LOSTANDFOUND1; }else{ p->eState = RECOVER_STATE_SCHEMA2; } } break; } case RECOVER_STATE_LOSTANDFOUND1: { if( p->laf.pUsed==0 ){ recoverLostAndFound1Init(p); } if( SQLITE_DONE==recoverLostAndFound1Step(p) ){ p->eState = RECOVER_STATE_LOSTANDFOUND2; } break; } case RECOVER_STATE_LOSTANDFOUND2: { if( p->laf.pAllAndParent==0 ){ recoverLostAndFound2Init(p); } if( SQLITE_DONE==recoverLostAndFound2Step(p) ){ p->eState = RECOVER_STATE_LOSTANDFOUND3; } break; } case RECOVER_STATE_LOSTANDFOUND3: { if( p->laf.pInsert==0 ){ recoverLostAndFound3Init(p); } if( SQLITE_DONE==recoverLostAndFound3Step(p) ){ p->eState = RECOVER_STATE_SCHEMA2; } break; } case RECOVER_STATE_SCHEMA2: { int rc = SQLITE_OK; recoverWriteSchema2(p); p->eState = RECOVER_STATE_DONE; /* If no error has occurred, commit the write transaction on the output ** database. Regardless of whether or not an error has occurred, make ** an attempt to end the read transaction on the input database. */ recoverExec(p, p->dbOut, "COMMIT"); rc = sqlite3_exec(p->dbIn, "END", 0, 0, 0); if( p->errCode==SQLITE_OK ) p->errCode = rc; recoverSqlCallback(p, "PRAGMA writable_schema = off"); recoverSqlCallback(p, "COMMIT"); p->eState = RECOVER_STATE_DONE; recoverFinalCleanup(p); break; }; case RECOVER_STATE_DONE: { /* no-op */ break; }; } } /* ** This is a worker function that does the heavy lifting for both init ** functions: ** ** sqlite3_recover_init() ** sqlite3_recover_init_sql() ** ** All this function does is allocate space for the recover handle and ** take copies of the input parameters. All the real work is done within ** sqlite3_recover_run(). */ sqlite3_recover *recoverInit( sqlite3* db, const char *zDb, const char *zUri, /* Output URI for _recover_init() */ int (*xSql)(void*, const char*),/* SQL callback for _recover_init_sql() */ void *pSqlCtx /* Context arg for _recover_init_sql() */ ){ sqlite3_recover *pRet = 0; int nDb = 0; int nUri = 0; int nByte = 0; if( zDb==0 ){ zDb = "main"; } nDb = recoverStrlen(zDb); nUri = recoverStrlen(zUri); nByte = sizeof(sqlite3_recover) + nDb+1 + nUri+1; pRet = (sqlite3_recover*)sqlite3_malloc(nByte); if( pRet ){ memset(pRet, 0, nByte); pRet->dbIn = db; pRet->zDb = (char*)&pRet[1]; pRet->zUri = &pRet->zDb[nDb+1]; memcpy(pRet->zDb, zDb, nDb); if( nUri>0 && zUri ) memcpy(pRet->zUri, zUri, nUri); pRet->xSql = xSql; pRet->pSqlCtx = pSqlCtx; pRet->bRecoverRowid = RECOVER_ROWID_DEFAULT; } return pRet; } /* ** Initialize a recovery handle that creates a new database containing ** the recovered data. */ sqlite3_recover *sqlite3_recover_init( sqlite3* db, const char *zDb, const char *zUri ){ return recoverInit(db, zDb, zUri, 0, 0); } /* ** Initialize a recovery handle that returns recovered data in the ** form of SQL statements via a callback. */ sqlite3_recover *sqlite3_recover_init_sql( sqlite3* db, const char *zDb, int (*xSql)(void*, const char*), void *pSqlCtx ){ return recoverInit(db, zDb, 0, xSql, pSqlCtx); } /* ** Return the handle error message, if any. */ const char *sqlite3_recover_errmsg(sqlite3_recover *p){ return (p && p->errCode!=SQLITE_NOMEM) ? p->zErrMsg : "out of memory"; } /* ** Return the handle error code. */ int sqlite3_recover_errcode(sqlite3_recover *p){ return p ? p->errCode : SQLITE_NOMEM; } /* ** Configure the handle. */ int sqlite3_recover_config(sqlite3_recover *p, int op, void *pArg){ int rc = SQLITE_OK; if( p==0 ){ rc = SQLITE_NOMEM; }else if( p->eState!=RECOVER_STATE_INIT ){ rc = SQLITE_MISUSE; }else{ switch( op ){ case 789: /* This undocumented magic configuration option is used to set the ** name of the auxiliary database that is ATTACH-ed to the database ** connection and used to hold state information during the ** recovery process. This option is for debugging use only and ** is subject to change or removal at any time. */ sqlite3_free(p->zStateDb); p->zStateDb = recoverMPrintf(p, "%s", (char*)pArg); break; case SQLITE_RECOVER_LOST_AND_FOUND: { const char *zArg = (const char*)pArg; sqlite3_free(p->zLostAndFound); if( zArg ){ p->zLostAndFound = recoverMPrintf(p, "%s", zArg); }else{ p->zLostAndFound = 0; } break; } case SQLITE_RECOVER_FREELIST_CORRUPT: p->bFreelistCorrupt = *(int*)pArg; break; case SQLITE_RECOVER_ROWIDS: p->bRecoverRowid = *(int*)pArg; break; case SQLITE_RECOVER_SLOWINDEXES: p->bSlowIndexes = *(int*)pArg; break; default: rc = SQLITE_NOTFOUND; break; } } return rc; } /* ** Do a unit of work towards the recovery job. Return SQLITE_OK if ** no error has occurred but database recovery is not finished, SQLITE_DONE ** if database recovery has been successfully completed, or an SQLite ** error code if an error has occurred. */ int sqlite3_recover_step(sqlite3_recover *p){ if( p==0 ) return SQLITE_NOMEM; if( p->errCode==SQLITE_OK ) recoverStep(p); if( p->eState==RECOVER_STATE_DONE && p->errCode==SQLITE_OK ){ return SQLITE_DONE; } return p->errCode; } /* ** Do the configured recovery operation. Return SQLITE_OK if successful, or ** else an SQLite error code. */ int sqlite3_recover_run(sqlite3_recover *p){ while( SQLITE_OK==sqlite3_recover_step(p) ); return sqlite3_recover_errcode(p); } /* ** Free all resources associated with the recover handle passed as the only ** argument. The results of using a handle with any sqlite3_recover_** ** API function after it has been passed to this function are undefined. ** ** A copy of the value returned by the first call made to sqlite3_recover_run() ** on this handle is returned, or SQLITE_OK if sqlite3_recover_run() has ** not been called on this handle. */ int sqlite3_recover_finish(sqlite3_recover *p){ int rc; if( p==0 ){ rc = SQLITE_NOMEM; }else{ recoverFinalCleanup(p); if( p->bCloseTransaction && sqlite3_get_autocommit(p->dbIn)==0 ){ rc = sqlite3_exec(p->dbIn, "END", 0, 0, 0); if( p->errCode==SQLITE_OK ) p->errCode = rc; } rc = p->errCode; sqlite3_free(p->zErrMsg); sqlite3_free(p->zStateDb); sqlite3_free(p->zLostAndFound); sqlite3_free(p->pPage1Cache); sqlite3_free(p); } return rc; } #endif /* ifndef SQLITE_OMIT_VIRTUALTABLE */ |
Added ext/recover/sqlite3recover.h.
> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 | /* ** 2022-08-27 ** ** The author disclaims copyright to this source code. In place of ** a legal notice, here is a blessing: ** ** May you do good and not evil. ** May you find forgiveness for yourself and forgive others. ** May you share freely, never taking more than you give. ** ************************************************************************* ** ** This file contains the public interface to the "recover" extension - ** an SQLite extension designed to recover data from corrupted database ** files. */ /* ** OVERVIEW: ** ** To use the API to recover data from a corrupted database, an ** application: ** ** 1) Creates an sqlite3_recover handle by calling either ** sqlite3_recover_init() or sqlite3_recover_init_sql(). ** ** 2) Configures the new handle |