jeffrey.j.hallman at frb.gov
2012-Dec-21 20:51 UTC
[R] segfault reading large BLOB from SQL Server
I am getting segfaults when I try to read a large binary object from a SQL Server database via RODBC. I am using the FreeTDS ODBC driver, and it has been working fine when reading from this same database. I have included relevant parts of the session below. Each row of the v_MAFiles view holds a text, html or pdf version of a document. The FileType entry is one of TXT, PDF, or HTML. If it's PDF, the FileString variable is empty and the FileBLOB holds the (binary) contents of the pdf. If FileType is TXT or HTML, then FileBLOB is empty and the file contents are in FileString. Note that the COLUMN_SIZE for FileBLOB and FileString are both 2 GB. However, the actual strings and blobs in there are actually only about 60 KB. I can read the FileString column with no problem. But trying to read a FileBLOB entry segfaults, apparently when R is calling malloc(). I suspect that R is trying and failing to allocate 2 GB of memory to hold something with the reported COLUMN_SIZE. But it doesn't barf when it loads a FileString entry of about the same 60 KB size. Perhaps the RODBC code is somehow discovering that the string in FileString is not really 2 GB, but it's not finding that out for a FileBLOB? Any help with this would be much appreciated.> Sys.info()sysname release version "Linux" "2.6.18-308.24.1.el5" "#1 SMP Wed Nov 21 11:42:14 EST 2012" nodename machine login "mralx1.rsma.frb.gov" "x86_64" "unknown" user effective_user "m1jjh00" "m1jjh00"> sessionInfo()R version 2.15.0 (2012-03-30) Platform: x86_64-redhat-linux-gnu (64-bit) locale: [1] LC_CTYPE=en_US LC_NUMERIC=C LC_TIME=en_US LC_COLLATE=en_US LC_MONETARY=en_US [6] LC_MESSAGES=en_US LC_PAPER=C LC_NAME=C LC_ADDRESS=C LC_TELEPHONE=C [11] LC_MEASUREMENT=en_US LC_IDENTIFICATION=C attached base packages: [1] stats graphics grDevices utils datasets methods base other attached packages: [1] mra_1.0 frb_3.9 fame_2.19 tis_1.23 RODBC_1.3-6 [6] RObjectTables_0.3-1 loaded via a namespace (and not attached): [1] data.table_1.8.2 XML_3.9-4> conn <- h41Connect(devl = T) > odbcGetInfo(conn)DBMS_Name DBMS_Ver Driver_ODBC_Ver Data_Source_Name Driver_Name "Microsoft SQL Server" "10.50.4000" "03.50" "h41_devl" "libtdsodbc.so" Driver_Ver ODBC_Ver Server_Name "0.91" "03.52" "MSSDEV1"> sqlColumns(conn, "v_MAFiles")TABLE_CAT TABLE_SCHEM TABLE_NAME COLUMN_NAME DATA_TYPE TYPE_NAME COLUMN_SIZE BUFFER_LENGTH DECIMAL_DIGITS 1 H41 dbo v_MAFiles ReleaseId 1 char 3 3 NA 2 H41 dbo v_MAFiles AsOfDate -9 date 10 20 NA 3 H41 dbo v_MAFiles GeneratedDate 93 datetime 23 16 3 4 H41 dbo v_MAFiles FileType 12 varchar 4 4 NA 5 H41 dbo v_MAFiles FileBLOB -4 image 2147483647 2147483647 NA 6 H41 dbo v_MAFiles FileString -1 text 2147483647 2147483647 NA 7 H41 dbo v_MAFiles UpdatedBy 12 varchar 60 60 NA 8 H41 dbo v_MAFiles UpdatedDateTime 93 datetime 23 16 3 NUM_PREC_RADIX NULLABLE REMARKS COLUMN_DEF SQL_DATA_TYPE SQL_DATETIME_SUB CHAR_OCTET_LENGTH ORDINAL_POSITION 1 NA 0 <NA> <NA> 1 NA 3 1 2 NA 0 <NA> <NA> -9 NA NA 2 3 NA 0 <NA> <NA> 9 3 NA 3 4 NA 0 <NA> <NA> 12 NA 4 4 5 NA 1 <NA> <NA> -4 NA 2147483647 5 6 NA 1 <NA> <NA> -1 NA 2147483647 6 7 NA 0 <NA> <NA> 12 NA 60 7 8 NA 0 <NA> <NA> 9 3 NA 8 IS_NULLABLE SS_DATA_TYPE 1 NO 47 2 NO 0 3 NO 61 4 NO 39 5 YES 37 6 YES 39 7 NO 39 8 NO 61> z <- odbcQuery(conn, "select FileBLOB from v_MAFiles where ReleaseId 'L61' and AsOfDate = '2012-12-19'") > z[1] 1> sqlGetResults(conn)*** glibc detected *** /opt/R/R-2.15.0/lib64/R/bin/exec/R: free(): invalid next size (fast): 0x00000000110fde60 *** ======= Backtrace: ========/lib64/libc.so.6[0x35efa711df] /lib64/libc.so.6(cfree+0x4b)[0x35efa7163b] /lcl/mra/prod1/files/R/library/RODBC/libs/RODBC.so(RODBCFetchRows +0x401)[0x2ae891623a71] /opt/R/R-2.15.0/lib64/R/lib/libR.so[0x2ae88aeb6d39] /opt/R/R-2.15.0/lib64/R/lib/libR.so(Rf_eval+0x696)[0x2ae88aeecc96] /opt/R/R-2.15.0/lib64/R/lib/libR.so[0x2ae88aeee389] /opt/R/R-2.15.0/lib64/R/lib/libR.so(Rf_eval+0x474)[0x2ae88aeeca74] /opt/R/R-2.15.0/lib64/R/lib/libR.so(Rf_applyClosure+0x308)[0x2ae88aef11e8] /opt/R/R-2.15.0/lib64/R/lib/libR.so(Rf_eval+0x323)[0x2ae88aeec923] /opt/R/R-2.15.0/lib64/R/lib/libR.so[0x2ae88aeef5e7] /opt/R/R-2.15.0/lib64/R/lib/libR.so(Rf_eval+0x474)[0x2ae88aeeca74] /opt/R/R-2.15.0/lib64/R/lib/libR.so[0x2ae88aeee389] /opt/R/R-2.15.0/lib64/R/lib/libR.so(Rf_eval+0x474)[0x2ae88aeeca74] /opt/R/R-2.15.0/lib64/R/lib/libR.so(Rf_applyClosure+0x308)[0x2ae88aef11e8] /opt/R/R-2.15.0/lib64/R/lib/libR.so(Rf_eval+0x323)[0x2ae88aeec923] /opt/R/R-2.15.0/lib64/R/lib/libR.so(Rf_ReplIteration+0x18d)[0x2ae88af2b9cd] /opt/R/R-2.15.0/lib64/R/lib/libR.so(run_Rmainloop+0xc0)[0x2ae88af2bd20] /opt/R/R-2.15.0/lib64/R/bin/exec/R(main+0x1b)[0x40081b] /lib64/libc.so.6(__libc_start_main+0xf4)[0x35efa1d994] /opt/R/R-2.15.0/lib64/R/bin/exec/R[0x400719] ======= Memory map: =======00400000-00401000 r-xp 00000000 00:1b 1305930 /opt/R/R-2.15.0/lib64/R/bin/exec/R 00600000-00601000 rw-p 00000000 00:1b 1305930 /opt/R/R-2.15.0/lib64/R/bin/exec/R 10a5b000-13192000 rw-p 10a5b000 00:00 0 [heap] 35ef600000-35ef61c000 r-xp 00000000 fd:07 131076 /lib64/ld-2.5.so 35ef81c000-35ef81d000 r--p 0001c000 fd:07 131076 /lib64/ld-2.5.so 35ef81d000-35ef81e000 rw-p 0001d000 fd:07 131076 /lib64/ld-2.5.so 35efa00000-35efb4e000 r-xp 00000000 fd:07 131079 /lib64/libc-2.5.so 35efb4e000-35efd4d000 ---p 0014e000 fd:07 131079 /lib64/libc-2.5.so 35efd4d000-35efd51000 r--p 0014d000 fd:07 131079 /lib64/libc-2.5.so 35efd51000-35efd52000 rw-p 00151000 fd:07 131079 /lib64/libc-2.5.so 35efd52000-35efd57000 rw-p 35efd52000 00:00 0 35efe00000-35efe82000 r-xp 00000000 fd:07 131083 /lib64/libm-2.5.so 35efe82000-35f0081000 ---p 00082000 fd:07 131083 /lib64/libm-2.5.so 35f0081000-35f0082000 r--p 00081000 fd:07 131083 /lib64/libm-2.5.so 35f0082000-35f0083000 rw-p 00082000 fd:07 131083 /lib64/libm-2.5.so 35f0200000-35f020d000 r-xp 00000000 fd:0a 817044 /usr/lib64/libgomp.so.1.0.0 35f020d000-35f040c000 ---p 0000d000 fd:0a 817044 /usr/lib64/libgomp.so.1.0.0 35f040c000-35f040d000 rw-p 0000c000 fd:0a 817044 /usr/lib64/libgomp.so.1.0.0 35f0600000-35f0616000 r-xp 00000000 fd:07 131104 /lib64/libpthread-2.5.so 35f0616000-35f0815000 ---p 00016000 fd:07 131104 /lib64/libpthread-2.5.so 35f0815000-35f0816000 r--p 00015000 fd:07 131104 /lib64/libpthread-2.5.so 35f0816000-35f0817000 rw-p 00016000 fd:07 131104 /lib64/libpthread-2.5.so 35f0817000-35f081b000 rw-p 35f0817000 00:00 0 35f0a00000-35f0a5c000 r-xp 00000000 fd:0a 816812 /usr/lib64/libodbc.so.1.0.0 35f0a5c000-35f0c5c000 ---p 0005c000 fd:0a 816812 /usr/lib64/libodbc.so.1.0.0 35f0c5c000-35f0c64000 rw-p 0005c000 fd:0a 816812 /usr/lib64/libodbc.so.1.0.0 35f1a00000-35f1a35000 r-xp 00000000 fd:0a 812139 /usr/lib64/libreadline.so.5.1 35f1a35000-35f1c34000 ---p 00035000 fd:0a 812139 /usr/lib64/libreadline.so.5.1 35f1c34000-35f1c3c000 rw-p 00034000 fd:0a 812139 /usr/lib64/libreadline.so.5.1 35f1c3c000-35f1c3d000 rw-p 35f1c3c000 00:00 0 35f4200000-35f4207000 r-xp 00000000 fd:07 131111 Process R aborted at Fri Dec 21 14:57:23 2012
Reasonably Related Threads
- Example input data with example output using relative pathway in vignette of R package?
- RODBC: Minor problem, I hope.
- PROBLEMAS NOMBRES DE COLUMNAS CON ESPACIOS CONEXION R-SQL
- RODBC: sqlQuery is successful, but a similar sqlFetch returns error
- Help: sqlSave Error
Wisdom of the Ancients
