tag:blogger.com,1999:blog-16243784058909952702024-09-09T20:56:32.734-07:00Ab initio - ParallelismAnonymoushttp://www.blogger.com/profile/13405902819110839597noreply@blogger.comBlogger1125tag:blogger.com,1999:blog-1624378405890995270.post-1899640789868904022014-11-28T23:21:00.003-08:002015-01-18T21:54:20.618-08:00Ab initio Parallelism<div dir="ltr" style="text-align: left;" trbidi="on">
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
<div style="text-align: justify;">
Ab Initio can process data in parallel runtime environment.</div>
</div>
<div class="MsoNormal">
<div style="text-align: justify;">
<br /></div>
</div>
<div class="MsoNormal">
<div style="text-align: justify;">
Ab Initio provides 3 ways of parallelism</div>
</div>
<div class="MsoNormal">
<div style="text-align: justify;">
<br /></div>
</div>
<div class="MsoNormal" style="margin-left: 1.0in; mso-list: l0 level2 lfo1; tab-stops: list 1.0in; text-indent: -.25in;">
<div style="text-align: justify;">
<!--[if !supportLists]--><span style="font-family: "Times New Roman","serif"; mso-fareast-font-family: "Times New Roman";">•<span style="font-family: 'Times New Roman'; font-size: 7pt; font-stretch: normal;">
</span></span><!--[endif]-->Component Parallelism</div>
</div>
<div class="MsoNormal" style="margin-left: 1.0in; mso-list: l0 level2 lfo1; tab-stops: list 1.0in; text-indent: -.25in;">
<div style="text-align: justify;">
<!--[if !supportLists]--><span style="font-family: "Times New Roman","serif"; mso-fareast-font-family: "Times New Roman";">•<span style="font-family: 'Times New Roman'; font-size: 7pt; font-stretch: normal;">
</span></span><!--[endif]-->Pipeline Parallelism</div>
</div>
<div class="MsoNormal" style="margin-left: 1.0in; mso-list: l0 level2 lfo1; tab-stops: list 1.0in; text-indent: -.25in;">
<div style="text-align: justify;">
<!--[if !supportLists]--><span style="font-family: "Times New Roman","serif"; mso-fareast-font-family: "Times New Roman";">•<span style="font-family: 'Times New Roman'; font-size: 7pt; font-stretch: normal;">
</span></span><!--[endif]-->Data Parallelism </div>
</div>
<div class="MsoNormal">
<div style="text-align: justify;">
<br /></div>
</div>
<div class="MsoNormal">
<b><u>Data Parallelism</u></b> </div>
<div class="MsoNormal" style="text-align: justify;">
Data is processed at the
different servers at the same time.</div>
<div class="MsoNormal" style="text-align: justify;">
<span lang="EN-IN">Data parallelism occurs when a graph separates
data into multiple divisions, allowing multiple copies of program components to
operate on the data in all the divisions simultaneously. <o:p></o:p></span></div>
<div class="MsoNormal" style="text-align: justify;">
<span lang="EN-IN">This is the most common parallelism when you
partition your data to be processed fast.This is achieved thru
partitioning. For example you have 1000 records and you divide them to 8
computers to process fast. <o:p></o:p></span></div>
<div class="MsoNormal" style="text-align: justify;">
<br /></div>
<div class="MsoNormal">
<b><u>Pipeline Parallelism</u></b> –</div>
<div class="MsoNormal" style="text-align: justify;">
<div style="text-align: justify;">
<span lang="EN-IN">Pipeline parallelism occurs when several
connected program components on the same branch of a graph execute
simultaneously. </span><span class="apple-converted-space"><span style="background: white; color: #333333; font-family: "Arial","sans-serif"; font-size: 10.0pt; line-height: 115%;"> I</span></span><span lang="EN-IN">f you are using a sort component the pipeline
parallelism does not occur.<o:p></o:p></span></div>
</div>
<div class="MsoNormal" style="text-align: justify;">
<div style="text-align: justify;">
<span lang="EN-IN">A graph with multiple components running
simultaneously on the same data uses pipeline parallelism. Each component in
the pipeline continuously reads from upstream components, processes data, and
writes to downstream components. Since a downstream component can process
records previously written by an upstream component, both components can
operate in parallel.<o:p></o:p></span></div>
</div>
<div class="MsoNormal" style="text-align: justify;">
<div style="text-align: justify;">
<span lang="EN-IN">NOTE: To limit the number of components running
simultaneously, set phases in the graph<o:p></o:p></span></div>
</div>
<div class="MsoNormal" style="text-align: justify;">
<div style="text-align: justify;">
<span lang="EN-IN">For example you can keep on reading the data
from input file(say 10 records) but till now processed only 6
records. This is called pipeline parallelism when one component does not
wait for all the data to come and starts processing parallely in a
pipe. <o:p></o:p></span></div>
</div>
<div class="MsoNormal">
<div style="text-align: justify;">
<br /></div>
</div>
<div class="MsoNormal">
<b><u>Component Parallelism</u></b> <span lang="EN-IN"><o:p></o:p></span></div>
<div class="MsoNormal">
<div style="text-align: justify;">
<span lang="EN-IN">In this two
or more components process the records in parallel.<o:p></o:p></span></div>
</div>
<div class="MsoNormal" style="text-align: justify;">
<div style="text-align: justify;">
<span lang="EN-IN">A graph with multiple processes running
simultaneously on separate data uses component parallelism.<o:p></o:p></span></div>
</div>
<div style="text-align: justify;">
<br /></div>
<div class="MsoNormal" style="text-align: justify;">
<div style="text-align: justify;">
<span lang="EN-IN">This kind of parallelism is specific to your
graph when 2 different components are not interrelated and they process
the data parallely. For example you have 2 input files and you sort the
data of both of them in 2 different flows. Then these 2 components are
under component parallelism.<o:p></o:p></span></div>
<div style="text-align: justify;">
<span lang="EN-IN"><br /></span></div>
<div style="text-align: justify;">
<span lang="EN-IN"><br /></span></div>
<div style="text-align: left;">
<span style="background-color: #cccccc; color: blue; font-family: 'Arial Black', sans-serif;"><b><u>Useful Links</u></b></span></div>
<div style="text-align: left;">
<br /></div>
<span lang="EN-IN"></span><br />
<table border="0" span="" style="font-family: 'Arial Black', sans-serif;"><tbody>
<tr><td><b><span style="color: blue;">1.</span></b></td><td><span style="color: blue;"><a href="http://abinitio-sandbox.blogspot.in/"><span style="color: blue;">Ab Initio Sandbox</span></a></span></td></tr>
<tr><td><b><span style="color: blue;">2.</span></b></td><td><span style="color: blue;"><a href="http://abinitio-components.blogspot.com/"><span style="color: blue;">Ab Initio Components</span></a></span></td></tr>
<tr><td><b><span style="color: blue;">3.</span></b></td><td><span style="color: blue;"><a href="http://abinitio-introduction.blogspot.com/"><span style="color: blue;">Ab initio Intoduction</span></a></span></td></tr>
<tr><td><b><span style="color: blue;">4.</span></b></td><td><span style="color: blue;"><a href="http://abinitio-graph.blogspot.com/"><span style="color: blue;">Ab Initio Basic Graph Development</span> </a> </span></td></tr>
<tr><td><b><span style="color: blue;">5.</span></b></td><td><a href="http://abinitio-multifiles.blogspot.com/"><span style="color: blue;">Ab Initio Multifile</span></a></td></tr>
</tbody></table>
</div>
</div>
Anonymoushttp://www.blogger.com/profile/13405902819110839597noreply@blogger.com0