hadoop - Questions about Oozie/Sqoop -


i have few questions:

1. why there mapreduce process in sqoop load data hdfs mysql?

e.g.

data in hdfs on directory: /foo/bar

to load data in mysql bar table, why there mapreduce process?

sqoop export --connect jdbc:mysql://localhost/hduser --table foo -m 1 --export-dir /foo/bar

after entering above command, mapreduce process executes.

2. how can enable/disable key in mysql using sqoop/oozie?

since huge data getting loaded mysql, need use enable/disable. how achieve it?

3. how run multiple oozie jobs in parallel?   4. how run oozie jobs in cron?

you can answer 1 or more questions.

thank you.

i'll go through questions 1 one. feel free ask more questions in comments , elaborate on things unclear you.

1. why there mapreduce process in sqoop load data hdfs mysql?

this because sqoop based on mapreduce. if consider how files stored in hdfs, split small chunks , these chunks stored across cluster (some of chunks might on same node). makes perfect sense have mapreduce job map tasks read these chunks of data in parallel , write them mysql.

2. how can enable/disable key in mysql using sqoop/oozie?

i don't know answer one. feel question little ambiguous. please try adding more details & if find i'll on this.

3. how run multiple oozie jobs in parallel?

each oozie job defined workflow.xml , job.properties.

  • if you're talking manual execution of multiple oozie workflows (jobs), can running command start oozie jobs jobs want run in parallel. sample command: oozie job -config job.properties -run

  • if you're talking running multiple actions within oozie workflow in parallel, can have fork trigger off multiple actions in parallel & join point parallel actions meet upon completion. example: 

    <fork name = 'samplefork'>    <path start = 'sampleaction1'/>    <path start = 'sampleaction2'/> </fork>  <action name = 'sampleaction`>   ..   ..   ..   <ok = 'joinactions'/>   <error = 'fail'/> </action>  <join name = 'joinactions' 'seqaction3'/>

4. how run oozie jobs in cron?

if want automate execution of oozie jobs, suggest oozie coordinator. using oozie coordinator, can schedule workflows trigger off after every interval (10 mins, 1 hour, 1 day etc. ).


Comments

Post a Comment

Popular posts from this blog

apache - Remove .php and add trailing slash in url using htaccess not loading css -

i have add following code htaccess file removing .php extension , add trailing slash urls. worked fine reduced page loading speed , css code not loading @ all. pages ugly. please help rewriteengine on rewritebase / ## hide .php extension snippet # externally redirect /dir/foo.php /dir/foo rewritecond %{the_request} ^[a-z]{3,}\s([^.]+)\.php [nc] rewriterule ^ %1/ [r,l] # add trailing slash rewritecond %{request_filename} !-f rewritecond %{request_uri} !/$ rewriterule . %{request_uri}/ [l,r=301] # internally forward /dir/foo /dir/foo.php rewritecond %{request_filename} !-d rewritecond %{request_filename}.php -f rewriterule ^(.*?)/?$ $1.php [l] we can rewrite urls using .htaccess files. before writing in .htaccess files need verify kind of server using. can check server adding a <?php phpinfo(); ?> in php page. ensure mod_rewrite module enabled in apache server. can identified checking phpinfo included page. common methods used url rewriting follows ...
Read more

javascript - jQuery show full size image on click -

pretty self explanatory not workking . i want display full image when user click on image... yes know there tons of plugins but i want keep simple. any idea ? css : .full { display: none; width: 500px; height: 500px; margin-left: 100px; margin-top: -300px; position: fixed; background-color: #f00; z-index: 6; } .thumbnail { width: 50px; height: 50px; } html: <img class="thumbnail" src="http://4.bp.blogspot.com/-uhbdp2esxng/txneptvi92i/aaaaaaaaaky/upfqis7zto0/s1600/rihanna-wallpaper.jpg"> jquery: $(document).ready(function() { $('.thumbnail').on('click', function() { var img = $('<img />',{src:this.src,'class':'full'}); $('.full').html(img).show(); console.log('done!'); }); }); try looking how jquery works. $(document).ready(function() { $('.thumbnail').on('click', function(...
Read more