postgresql - PHP Fatal error: Allowed memory size of xxx bytes exhausted (tried to allocate XX bytes) -


i have php script splits large file , inserts postgresql. import has worked before on php 5.3 , postgresql 8.3 , mac os x 10.5.8. moved on new mac pro. has plenty of ram (16mb), mac os x 10.9.2, php 5.5.8, postgresql 9.3.

the problem when reading large import file. tab-separated file on 181 mb. have tried increase php memory 2gb (!) no more success. guess problem must in code reading text file , splitting it. error:

php fatal error:  allowed memory size of 2097152000 bytes exhausted (tried allocate 72 bytes) in /library/filemaker server/data/scripts/getgbifdata.php on line 20

is there better way this? read file , split lines, again split each line \t (tab). error on line: 

$arr = explode("\t", $line);

here code:

<?php  ## have tried here, memory_limit in php.ini 256m  ini_set("memory_limit","1000m");  $db= pg_connect('host=127.0.0.1 dbname=my_db_name user=username password=pass'); ### sett error_state: pg_set_error_verbosity($db, pgsql_errors_verbose);  ### emtpy db $result = pg_query("truncate table my_db_name");  $fcontents = file ('///library/filemaker\ server/data/documents/export/export_file.tab');    for($i=0; $i<sizeof($fcontents); $i++) {       $line = trim($fcontents[$i]);       $arr = explode("\t", $line);        $query = "insert my_db_name( field1, field2 etc....      )         values ( '{$arr[0]}','{$arr[1]}','{$arr[2]}','{$arr[3]}', etc........         )";       $result = pg_query($query); echo "\n lines:".$i;  pg_send_query($db, $query);  $res1 = pg_get_result($db); } ## update geometry column $sql = pg_query(" update darwincore2 set punkt_geom= st_setsrid(st_makepoint(my_db_name.longitude, darwincore2.latitude),4326); ");

?>

i think problem you're using file() function reads whole file in memory @ once. try reading line line using fopen , fgets.

$fp = fopen(filename, "r"); while (($line = fgets($fp)) !== false) {     ... insert $line db.... } fclose($fp);

you can import file directly copy command (http://www.postgresql.org/docs/9.2/static/sql-copy.html)


Comments

Post a Comment

Popular posts from this blog

apache - Remove .php and add trailing slash in url using htaccess not loading css -

i have add following code htaccess file removing .php extension , add trailing slash urls. worked fine reduced page loading speed , css code not loading @ all. pages ugly. please help rewriteengine on rewritebase / ## hide .php extension snippet # externally redirect /dir/foo.php /dir/foo rewritecond %{the_request} ^[a-z]{3,}\s([^.]+)\.php [nc] rewriterule ^ %1/ [r,l] # add trailing slash rewritecond %{request_filename} !-f rewritecond %{request_uri} !/$ rewriterule . %{request_uri}/ [l,r=301] # internally forward /dir/foo /dir/foo.php rewritecond %{request_filename} !-d rewritecond %{request_filename}.php -f rewriterule ^(.*?)/?$ $1.php [l] we can rewrite urls using .htaccess files. before writing in .htaccess files need verify kind of server using. can check server adding a <?php phpinfo(); ?> in php page. ensure mod_rewrite module enabled in apache server. can identified checking phpinfo included page. common methods used url rewriting follows ...
Read more

inno setup - TLabel or TNewStaticText - change .Font.Style on Focus like Cursor changes with .Cursor -

is possible change .font.style on focus tlabel or tnewstatictext happens cursor when use .cursor ? there no built-in support track mouse hovering in inno setup @ time. however, intercepting window procedure of controls can track yourself. following example need innocallback library: [setup] appname=my program appversion=1.5 defaultdirname={pf}\my program outputdir=userdocs:inno setup examples output [files] source: "innocallback.dll"; destdir: "{tmp}"; flags: dontcopy [code] #ifdef unicode #define aw "w" #else #define aw "a" #endif const gwl_wndproc = -4; wm_mousemove = $0200; type wparam = uint_ptr; lparam = longint; lresult = longint; twindowproc = function(hwnd: hwnd; umsg: uint; wparam: wparam; lparam: lparam): lresult; function setcapture(hwnd: hwnd): hwnd; external 'setcapture@user32.dll stdcall'; function releasecapture: bool; external 'releasecapture@user32.dll stdcall'; func...
Read more