apache pig - Create hive timestamp from pig -


how can create timestamp field in pig string hive accepts timestamp?

i have formatted string in pig match timestamp format in hive, after loading null instead of showing date.

2014-04-10 09:45:56 how format looks in pig, , matching format hive timestamp, cannot load. (only if load string field)

any ideas why?

quick update: no hcatalog available

problem case timestamp fields contains null values , filed become null when using timestamp data type. when putting timestamp column row in above format works fine. real question how null values can handle

i suspect have written data hdfs using pigstorage , want load hive table. problem missing tuple field written pig null treated hive 0.11 null. far good. subsequent fields treated null, can have different values. hive 0.12 doesn't have issue.

depending on serde type, hive can interpret different strings null. in case of lazysimpleserde \n.

you have 2 option:

  • set table's null format property empty string produced pig
  • or store \n in pig null fields

e.g:

given following data in pig 0.11 :

a = load 'data' (txt:chararray, ts:chararray); dump a; (a,2014-04-10 09:45:56) (b,2014-04-11 10:45:56) (,) (e,2014-04-12 11:45:56)

option 1:

store '/user/data';

hive 0.11 :

create external table test (txt string, tms timestamp)  row format delimited fields terminated '\t' location '/user/data';  alter table test set serdeproperties('serialization.null.format' = '');

option 2:

... b = foreach generate txt, (ts null?'\\n':ts); store b '/user/data';

then create table in hive without setting serde property.


Comments

Post a Comment

Popular posts from this blog

apache - Remove .php and add trailing slash in url using htaccess not loading css -

i have add following code htaccess file removing .php extension , add trailing slash urls. worked fine reduced page loading speed , css code not loading @ all. pages ugly. please help rewriteengine on rewritebase / ## hide .php extension snippet # externally redirect /dir/foo.php /dir/foo rewritecond %{the_request} ^[a-z]{3,}\s([^.]+)\.php [nc] rewriterule ^ %1/ [r,l] # add trailing slash rewritecond %{request_filename} !-f rewritecond %{request_uri} !/$ rewriterule . %{request_uri}/ [l,r=301] # internally forward /dir/foo /dir/foo.php rewritecond %{request_filename} !-d rewritecond %{request_filename}.php -f rewriterule ^(.*?)/?$ $1.php [l] we can rewrite urls using .htaccess files. before writing in .htaccess files need verify kind of server using. can check server adding a <?php phpinfo(); ?> in php page. ensure mod_rewrite module enabled in apache server. can identified checking phpinfo included page. common methods used url rewriting follows ...
Read more

javascript - jQuery show full size image on click -

pretty self explanatory not workking . i want display full image when user click on image... yes know there tons of plugins but i want keep simple. any idea ? css : .full { display: none; width: 500px; height: 500px; margin-left: 100px; margin-top: -300px; position: fixed; background-color: #f00; z-index: 6; } .thumbnail { width: 50px; height: 50px; } html: <img class="thumbnail" src="http://4.bp.blogspot.com/-uhbdp2esxng/txneptvi92i/aaaaaaaaaky/upfqis7zto0/s1600/rihanna-wallpaper.jpg"> jquery: $(document).ready(function() { $('.thumbnail').on('click', function() { var img = $('<img />',{src:this.src,'class':'full'}); $('.full').html(img).show(); console.log('done!'); }); }); try looking how jquery works. $(document).ready(function() { $('.thumbnail').on('click', function(...
Read more