http://blog.yt-project.org Most recent posts at yt Project Blog posterous.com Mon, 13 Feb 2012 04:10:21 -0800 http://feedproxy.google.com/~r/EnzotoolsBlog/~3/JVVT3NI6OJs/whats-new-with-yt http://blog.yt-project.org/whats-new-with-yt <p> Now that the post-workshop preparations and work have settled down, I thought it might be interesting to share some of the developments going on with yt.  We&#39;re still a long way from a new release, so these interim &quot;development&quot; updates are meant to be a bit of a teaser.  As always, these features are either in the main branch or (if noted) in a public fork on BitBucket.  <b>If they sound interesting, drop us a line on <a href="http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org">yt-dev</a> to see about getting involved!</b><p /><div>Stephen has been pushing lately for more consistency in the code base -- indentation, naming conventions, spaces, and so on.  Specifically, he has been suggesting we follow <a href="http://www.python.org/dev/peps/pep-0008/">PEP-8</a>, which is a standard for Python source code.  This has gotten a lot of support, and so we&#39;re encouraging this in new commits and looking into mechanisms for updating old code.  (Although it can cause some tricky merges, so we&#39;re trying to take it easy for a bit!)</div> <p /><div>JohnZ recently added a particle trajectory mechanism, for correlating particles between outputs and following them.  This lets you see where they go and the character of the gas they pass through.</div> <p /><div>Sam has been looking at improving volume rendering, including adding hard surfaces and a much faster (Cythonized) kD-tree routine.  The initial hard surface stuff looks just great.  (This is all taking place in his fork.)  This code is also threaded, so it should run much faster on multi-core machines.</div> <p /><div>JohnW identified a bug in the ghost zone generation, which has resulted in a big speedup for generating ghost zones!</div><p /><div>Chris has been trying to get the regridding process for ART to be substantially faster, which he&#39;s been having success with.  We&#39;re now trying to together work on changing how &quot;child masking&quot; is thought of; with patch-based codes it only masks those cells where data at finer levels is available.  We&#39;re trying to make it so that it also marks where coarser data is the finest available, which should help out with speed for octree based codes.</div> <p /><div>Finally, I&#39;ve been up to working on geometric selection.  My hope is that by rethinking how we think about geometry in yt and removing a number of intermediate steps, we can avoid creating a whole bunch of temporary arrays and overall speed up the process (and add better support for non-patch based codes!).  Results so far have been pretty good, but it&#39;s a long ways from being ready.  It&#39;s in my refactor fork.</div> <p /><div>There are a lot of exciting things going on, so keep your eyes on this space!  In addition to all of these things, we&#39;ve got web interactors for isolated routines, an all-new hub, improvements to reason, and tons of other stuff.  As always, drop by <a href="http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org">yt-dev</a> or the IRC channel if you&#39;d like to get involved.</div> </p> <p><a href="http://blog.yt-project.org/whats-new-with-yt">Permalink</a> | <a href="http://blog.yt-project.org/whats-new-with-yt#comment">Leave a comment&nbsp;&nbsp;&raquo;</a> </p><img src="http://feeds.feedburner.com/~r/EnzotoolsBlog/~4/JVVT3NI6OJs" height="1" width="1"/> http://files.posterous.com/user_profile_pics/492646/hs_05.jpg http://posterous.com/users/37lnBezefhIJ Matt powersoffour Matt http://blog.yt-project.org/whats-new-with-yt Mon, 30 Jan 2012 10:02:09 -0800 http://feedproxy.google.com/~r/EnzotoolsBlog/~3/22sV74-J0AY/yt-workshop-2012-a-success http://blog.yt-project.org/yt-workshop-2012-a-success <p> The yt workshop last week in Chicago ( <br /><a href="http://yt-project.org/workshop2012/">http://yt-project.org/workshop2012/</a> ) was an enormous success. On <br />behalf of the organizing and technical committees, I'd like to <br />specifically thank the FLASH Center, particularly Don Lamb, Mila <br />Kuntu, Carrie Eder, for their hospitality; the venue was outstanding <br />and their hospitality touching. Additionally, we're very grateful to <br />the Adler Planetarium's Doug Roberts and Mark SubbaRao for hosting us <br />on Wednesday evening -- seeing the planetarium show as well as volume <br />renderings made by yt users up on the dome was so much fun. The yt <br />workshop was supported by NSF Grant 1214147. Thanks to everyone who <br />attended -- your energy and excitement helped make it a success. <p /> Thanks also to the organizing and technical committees: Britton <br />Smith, John ZuHone, Brian O'Shea, Jeff Oishi, Stephen Skory, Sam <br />Skillman, and Cameron Hummels. All talks have been recorded, and you <br />can clone a unified repository of talk slides and worked examples: <p /> hg clone https://bitbucket.org/yt_analysis/workshop2012/ <p /> A few photos have been put up online, too: <p /> <a href="http://goo.gl/g02uP">http://goo.gl/g02uP</a> <p /> As I am able to edit and upload talks, they'll appear on the yt <br />youtube channel as well as on the yt homepage: <p /> <a href="http://www.youtube.com/ytanalysis">http://www.youtube.com/ytanalysis</a> <p /> Thanks again, and wow, what a week! </p> <p><a href="http://blog.yt-project.org/yt-workshop-2012-a-success">Permalink</a> | <a href="http://blog.yt-project.org/yt-workshop-2012-a-success#comment">Leave a comment&nbsp;&nbsp;&raquo;</a> </p><img src="http://feeds.feedburner.com/~r/EnzotoolsBlog/~4/22sV74-J0AY" height="1" width="1"/> http://files.posterous.com/user_profile_pics/492646/hs_05.jpg http://posterous.com/users/37lnBezefhIJ Matt powersoffour Matt http://blog.yt-project.org/yt-workshop-2012-a-success Mon, 16 Jan 2012 19:45:43 -0800 http://feedproxy.google.com/~r/EnzotoolsBlog/~3/aLtalSlg-ek/workshop-in-just-a-week http://blog.yt-project.org/workshop-in-just-a-week <p> The first yt workshop is in just about a week.  We&#39;ve updated the <a href="http://yt-project.org/workshop2012">website</a> with the current list of talks, along with information about getting to and from the workshop from the conference hotel, and information about how to get the sample data.  Keep your eyes on the website in the lead up to the workshop, as we&#39;ll be posting a script for fisheye lens renderings for our viz night at the Adler, information about the talks and example scripts, and other useful info.  Once the workshop is over we&#39;ll update with links to the full videos of the talks, the slides, and scripts. </p> <p><a href="http://blog.yt-project.org/workshop-in-just-a-week">Permalink</a> | <a href="http://blog.yt-project.org/workshop-in-just-a-week#comment">Leave a comment&nbsp;&nbsp;&raquo;</a> </p><img src="http://feeds.feedburner.com/~r/EnzotoolsBlog/~4/aLtalSlg-ek" height="1" width="1"/> http://files.posterous.com/user_profile_pics/492646/hs_05.jpg http://posterous.com/users/37lnBezefhIJ Matt powersoffour Matt http://blog.yt-project.org/workshop-in-just-a-week Thu, 15 Dec 2011 11:44:00 -0800 http://feedproxy.google.com/~r/EnzotoolsBlog/~3/PlhUnwmdYg0/yt-version-23-announcement http://blog.yt-project.org/yt-version-23-announcement <p> <p>Just in time for the New Year, we&rsquo;re happy to announce the release of yt version 2.3! &nbsp;( <a href="http://yt-project.org/">http://yt-project.org/</a> ) &nbsp;The new version includes many new modules and enhancements, and the usual set of bug fixes over the last point release. We encourage all users to upgrade to take advantage of the changes.</p> <p>yt is a community-developed analysis and visualization toolkit for astrophysical simulation data. yt provides full support for Enzo, Orion, Nyx, and FLASH codes, with preliminary support for the RAMSES code (and a handful of others.) &nbsp;It can be used to create many common types of data products, as well as serving as a library for developing your own data reductions and processes.</p> <p>Below is a non-comprehensive list of new features and enhancements:</p> <p>&nbsp;</p> <ul> <li>Improved and expanded documentation located at <a href="http://yt-project.org/doc/">http://yt-project.org/doc/</a>.</li> <li>Boolean logic data containers (joins, intersections and nots) to select arbitrary data regions.</li> <li>Multi-level parallelism for subgroups of MPI tasks.</li> <li>Extensive answer tests.</li> <li>Isocontouring and flux-over-surface calculations, with WebGL interface.</li> <li>A reorganized field system.</li> <li>Adaptive resolution HEALpix-based all-sky volume rendering.</li> <li>Radial column density calculations.</li> <li>Memory usage, performance enhancements and bug fixes throughout the code.</li> </ul> <p>&nbsp;</p> <p>Everything, from installation, to development, to a cookbook, can be found on the homepage: <a href="http://yt-project.org/">http://yt-project.org/</a></p> <p>We have updated the libraries installed with the install script; for more information, see the &ldquo;Dependencies&rdquo; section of the yt docs at <a href="http://yt-project.org/doc/advanced/installing.html">http://yt-project.org/doc/advanced/installing.html</a>.</p> <p>Development has been sponsored by the NSF, DOE, and various University funding. We invite you to get involved with developing and using yt!</p> <p>We&rsquo;re also holding the FIRST YT WORKSHOP from January 24-26 at the FLASH center in Chicago. &nbsp;See the workshop homepage for more information! &nbsp;<a href="http://yt-project.org/workshop2012/">http://yt-project.org/workshop2012/</a></p> <p>Please forward this announcement to interested parties.</p> <p>Sincerely,</p> <p>&nbsp; &nbsp; The yt development team</p> <p>&nbsp;</p> </p> <p><a href="http://blog.yt-project.org/yt-version-23-announcement">Permalink</a> | <a href="http://blog.yt-project.org/yt-version-23-announcement#comment">Leave a comment&nbsp;&nbsp;&raquo;</a> </p><img src="http://feeds.feedburner.com/~r/EnzotoolsBlog/~4/PlhUnwmdYg0" height="1" width="1"/> http://posterous.com/images/profile/missing-user-75.png http://posterous.com/users/heOzHGBSGnjGG Stephen Skory Stephen Stephen Skory http://blog.yt-project.org/yt-version-23-announcement Mon, 28 Nov 2011 14:17:49 -0800 http://feedproxy.google.com/~r/EnzotoolsBlog/~3/qq0N15Mzmg4/yt-workshop-registration-is-open http://blog.yt-project.org/yt-workshop-registration-is-open <p> Registration for the 2012 yt workshop, to be held at the FLASH center <br />in Chicago from January 24-26, is now open. <p /> Please register here: <a href="http://yt-project.org/workshop2012/">http://yt-project.org/workshop2012/</a> </p> <p><a href="http://blog.yt-project.org/yt-workshop-registration-is-open">Permalink</a> | <a href="http://blog.yt-project.org/yt-workshop-registration-is-open#comment">Leave a comment&nbsp;&nbsp;&raquo;</a> </p><img src="http://feeds.feedburner.com/~r/EnzotoolsBlog/~4/qq0N15Mzmg4" height="1" width="1"/> http://files.posterous.com/user_profile_pics/492646/hs_05.jpg http://posterous.com/users/37lnBezefhIJ Matt powersoffour Matt http://blog.yt-project.org/yt-workshop-registration-is-open Wed, 09 Nov 2011 08:03:00 -0800 http://feedproxy.google.com/~r/EnzotoolsBlog/~3/NmQAIRXuKn0/boolean-data-containers http://blog.yt-project.org/boolean-data-containers <p> <p>A useful new addition to yt are <em>boolean</em> data containers. These are hybrid data containers that are built by relating already-defined data containers with each other using boolean operators. Nested boolean logic, using parentheses, is also supported. The boolean data container (or volume) is made by constructing a list of volumes interspersed with operators given as strings. Below are some examples of what can be done with boolean data containers.</p> <p><strong>The &ldquo;OR&rdquo; Operator</strong></p> <p>The &ldquo;OR&rdquo; operator combines volume of the two data containers into one. The two intial volumes may or may not overlap, meaning that the combined volume may constitute several disjoint volumes. Here is an example showing the construction of a boolean volume of two disjoint spheres:</p> <div class="CodeRay"> <div class="code"><pre>sp1 = pf.h.sphere([0.3]*3, .15) sp2 = pf.h.sphere([0.7]*3, .25) bool = pf.h.boolean([sp1, &quot;OR&quot;, sp2])</pre></div> </div> <p>Here is a short video showing the result:</p> <p><iframe src="http://player.vimeo.com/video/31859862?portrait=0" frameborder="0" height="328" width="580"></iframe></p> <p><strong>The &ldquo;AND&rdquo; Operator</strong></p> <p>The &ldquo;AND&rdquo; operator mixes two volumes where both volumes cover the same volume. Put another way, the &ldquo;AND&rdquo; operator produces a new volume that is defined by all cells that lie in both of the initial volumes. Here is an example of the intersection of a sphere and a cube:</p> <div class="CodeRay"> <div class="code"><pre>re1 = pf.h.region([0.5]*3, [0.0]*3, [0.7]*3) sp1 = pf.h.sphere([0.5]*3, 0.5) bool = pf.h.boolean([re1, &quot;AND&quot;, sp1])</pre></div> </div> <p>Here is a short video showing the result:</p> <p><iframe src="http://player.vimeo.com/video/31861314?portrait=0" frameborder="0" height="328" width="580"></iframe></p> <p><strong>The &ldquo;NOT&rdquo; Operator</strong></p> <p>The &ldquo;NOT&rdquo; operator is the only non-transitive operator, and is read from left to right. For example, if there are multiple &ldquo;NOT&rdquo; operators, the first &ldquo;NOT&rdquo; on the left and the two volumes on either side are considered first. The new volume constructed is the volume <em>contained in the first data container</em> that the <em>second data container</em> does <strong>not</strong> cover. This can be thought of as a subtraction from the first volume by the second volume. Here is an example of a cubical region having a corner cut out of it:</p> <div class="CodeRay"> <div class="code"><pre>re1 = pf.h.region([0.5]*3, [0.]*3, [1.]*3) re2 = pf.h.region([0.5]*3, [0.5]*3, [1.]*3) bool = pf.h.boolean([re1, &quot;NOT&quot;, re2])</pre></div> </div> <p>Here is a short video showing the result:</p> <p><iframe src="http://player.vimeo.com/video/31859691?portrait=0" frameborder="0" height="328" width="580"></iframe></p> <p><strong>Nested Logic</strong></p> <p>It is possible to use nested logic using parentheses. When nested logic is used, the order of logical operations begins at the inner-most nested level and proceeds outwards, always respecting the left to right ordering for &ldquo;NOT&rdquo; operations. This may be used to create truly fantastic volumes. Here is an example of a piece of Swiss cheese created from two cubical regions and two spheres. The second sphere <code>sp2</code> wraps around the periodic boundaries and impacts the largest cube in more than one place.</p> <div class="CodeRay"> <div class="code"><pre>re1 = pf.h.region([0.5]*3, [0.]*3, [1.]*3) re2 = pf.h.region([0.5]*3, [0.5]*3, [1.]*3) sp1 = pf.h.sphere([0.5, 0.7, 0.5], .25) sp2 = pf.h.sphere([0.1]*3, .25) bool = pf.h.boolean([re1, &quot;NOT&quot;, &quot;(&quot;, re2, &quot;AND&quot;, sp1, &quot;)&quot;, &quot;NOT&quot;, sp2])</pre></div> </div> <p>Here is a short video showing the result:</p> <p><iframe src="http://player.vimeo.com/video/31863019?portrait=0" frameborder="0" height="328" width="580"></iframe></p> <p>For those wondering how the movies were made, I&rsquo;ve posted <a href="http://paste.enzotools.org/show/1939/" title="enzotools paste">the script here</a>. Note that blocks of comments will need to be turned on/off to get the desired boolean data container.</p> </p> <p><a href="http://blog.yt-project.org/boolean-data-containers">Permalink</a> | <a href="http://blog.yt-project.org/boolean-data-containers#comment">Leave a comment&nbsp;&nbsp;&raquo;</a> </p><img src="http://feeds.feedburner.com/~r/EnzotoolsBlog/~4/NmQAIRXuKn0" height="1" width="1"/> http://posterous.com/images/profile/missing-user-75.png http://posterous.com/users/heOzHGBSGnjGG Stephen Skory Stephen Stephen Skory http://blog.yt-project.org/boolean-data-containers Wed, 09 Nov 2011 04:23:00 -0800 http://feedproxy.google.com/~r/EnzotoolsBlog/~3/Yfo_wHicKx0/google-page-for-yt http://blog.yt-project.org/google-page-for-yt <p> yt now has a <a href="https://plus.google.com/107728486871834552760/">Google Plus page</a>.  Here&#39;s we&#39;ll post smaller, less blog-worthy items, hold video conferencing &quot;hangouts&quot;, and so on.  Encircle away!  And if you post something you&#39;d like to be reshared, just be sure to explicitly share it with &quot;+yt&quot; so we know. </p> <p><a href="http://blog.yt-project.org/google-page-for-yt">Permalink</a> | <a href="http://blog.yt-project.org/google-page-for-yt#comment">Leave a comment&nbsp;&nbsp;&raquo;</a> </p><img src="http://feeds.feedburner.com/~r/EnzotoolsBlog/~4/Yfo_wHicKx0" height="1" width="1"/> http://files.posterous.com/user_profile_pics/492646/hs_05.jpg http://posterous.com/users/37lnBezefhIJ Matt powersoffour Matt http://blog.yt-project.org/google-page-for-yt Mon, 24 Oct 2011 08:14:22 -0700 http://feedproxy.google.com/~r/EnzotoolsBlog/~3/n0VWoqmuYnU/yt-testing-parallelism http://blog.yt-project.org/yt-testing-parallelism <p> <p><span style="font-family: arial; font-size: small;">Hi all, </span></p> <p /> <div>A few of us worked this past week on a couple yt projects and made what we think is&nbsp;significant&nbsp;progress. &nbsp;Two of the items we focused on were testing and parallelism. &nbsp; &nbsp;</div> <p /> <div>For testing, we've broadened the test suite to include many more functions and derived quantities. &nbsp;We now have 548 tests that include (off and on-axis) slices,&nbsp;(off and on-axis)&nbsp;projections, phase distributions, halo finding, volume rendering, and geometrical region cuts such as rectangular solids, spheres, and disks. &nbsp;We use both plain and derived fields for these tests so that it covers as many bases as possible. &nbsp;With this framework, we are now able to keep a gold standard of the test results for any dataset, then test later changes against this standard. &nbsp;These tests can test for bitwise&nbsp;identicality&nbsp;or allow for some tolerance. For a full list of tests, you can run python yt/tests/runall.py -l, and use --help to look at the usage. &nbsp;We will soon be updating the documentation to provide more information on how to set up the testing framework, but I think all of us agree that this will make it much easier to test our changes to make sure bugs have not crept in.</div> <p /> <div>The second big change I'd like to talk about is the way we now handle parallelism in yt. &nbsp;Previously, methods that employed parallelism through MPI calls would first inherit from ParallelAnalysisInterface, which had access to a ton of mpi functions that all work off of MPI.COMM_WORLD. &nbsp;In our revamp we wanted to accomplish two things: 1) merge duplicate mpi calls that were only different by the type of values they work on and do overall cleanup. 2) Allow for nested levels of parallelism where two (or more) separate communicators are able to use barriers and collective operations such as allreduce. &nbsp;To do this, we worked in a two-step process. &nbsp;First we took things like:</div> <div> <p /> <div>&nbsp; &nbsp; def _mpi_allsum(self, data):</div> </div> <div> <div>&nbsp; &nbsp; def _mpi_Allsum_double(self, data):</div> </div> <div> <div>&nbsp; &nbsp; def _mpi_Allsum_long(self, data):</div> </div> <div> <div>&nbsp; &nbsp; def _mpi_allmax(self, data):</div> </div> <div> <div>&nbsp; &nbsp; def _mpi_allmin(self, data):</div> </div> <p /> <div>and packed it into a single function:</div> <p /> <div>&nbsp; &nbsp; def mpi_allreduce(self, data, dtype=None, op='sum'):</div> <p /> <div>When a numpy array is passed to this new mpi_allreduce, dtype is determined from the array properties. &nbsp;If the data is a dictionary, then it is passed to mpi4py's allreduce function that acts on dictionaries. &nbsp;This greatly reduced the number of lines in parallel_analysis_interface (1376 ==&gt; 915), even after adding in additional functionality.</div> <p /> <div>The second step was bundling all of these functions into a new class called Communicator. &nbsp;This Communicator object is initialized with an MPI communicator that no longer is restricted to COMM_WORLD. &nbsp;Using this as the fundamental MPI object, we then built a CommunicationSystem object that manages these communicators. &nbsp;A global communication_system instance is created, that is initialized with COMM_WORLD at the top of the system if the environment is mpi4py-capable. &nbsp;If not, an empty communicator is created that has passthroughs for all the mpi functions.</div> <p /> <div>Using this new framework we are now able to take advantage of multiple communicators. &nbsp;There are two use cases that we have implemented so far:&nbsp;</div> <div>1)&nbsp;parallel_objects</div> <div> <div>parallel_objects is a method un parallel_analysis_interface.py for iterating over a set of objects such that a group of processors work on each object. &nbsp;This could be used, for example, to run N projections each with M processors, allowing for a parallelism of NxM. &nbsp;</div> </div> <p /> <div>2) workgroups</div> <div>workgoups allows users to set up multiple MPI communicators with a non-uniform number of processors to each work on a separate task. &nbsp;<span style="font-family: arial; font-size: small;">This capability lives within the ProcessorPool and Workgroup objects in parallel_analysis_interface.py</span></div> <p /> <div>These are just the first two that we tried out and we are very excited about the new possibilities. &nbsp;</div> <p /> <div>With these changes, there was one implementation change that has already come up once in the mailing list. &nbsp;When you implement a new class that you'd like to have access to the communication objects, you must first inherit ParallelAnalysisInterface, and then make sure that __init__ makes a call to:</div> <div>ParallelAnalysisInterface.__init__()</div> <p /> <div>At that point, your new class will have access to the mpi calls through the self.comm object. &nbsp;For example, to perform a reduction one would do:</div> <div>self.comm.mpi_allreduce(my_data, op='sum')</div> <p /> <div>As I said before, this will be documented soon, but hopefully this will help for now.</div> <p /> <div>Sam, Britton, Cameron, Jeff, and Matt</div> <p>&nbsp;</p> </p> <p><a href="http://blog.yt-project.org/yt-testing-parallelism">Permalink</a> | <a href="http://blog.yt-project.org/yt-testing-parallelism#comment">Leave a comment&nbsp;&nbsp;&raquo;</a> </p><img src="http://feeds.feedburner.com/~r/EnzotoolsBlog/~4/n0VWoqmuYnU" height="1" width="1"/> http://files.posterous.com/user_profile_pics/548841/Sam_Skillman.png http://posterous.com/users/5AvFMjXVEE7v Sam Skillman Sam Sam Skillman http://blog.yt-project.org/yt-testing-parallelism Mon, 03 Oct 2011 18:24:29 -0700 http://feedproxy.google.com/~r/EnzotoolsBlog/~3/AluAedZE-8g/announcing-the-2012-yt-workshop http://blog.yt-project.org/announcing-the-2012-yt-workshop <p> <span style="border-collapse: collapse; color: rgb(34, 34, 34); font-family: arial, sans-serif; font-size: 13px;">I&#39;m pleased to announce the 2012 yt Workshop at the FLASH Center in Chicago, January 24-26.<p /> The workshop will be aimed at both users and developers of yt.  We will begin with intensive user training, moving from basic usage to advanced and parallel usage.  Users are encouraged to bring their ideas and prototypes for new analysis routines as there will be opportunities to work with more experienced developers.  We will then address to how to modify, extend and contribute to yt, and transition to a developers workshop.  In the developers portion of the workshop, we will discuss ideas for improvements to the code and then break into groups to implement new features.  Users are highly encouraged to stay and participate in development.<p /> The FLASH Center has graciously offered to host the workshop.  We have identified a hotel in downtown Chicago (near the river, just off Michigan Avenue) that we are able to book double-occupancy rooms for $99/night, pre-tax.  We are actively pursuing funding opportunities, but as of yet have not secured funding for participant costs; if we are able to do so, it will likely cover hotel stays for a limited number of individuals willing to share rooms for the four nights of the workshop (Jan. 23-26).<p /> As we prepare hotel reservations, funding applications and other technicalities, we need to get a sense of not only how many people are potentially going to attend, but also their current career stage, funding availability and so on. If you are interested in attending the workshop, we would greatly appreciate it if you would visit the following URL and fill out the Google Form: <a href="http://goo.gl/xElrB" target="_blank" style="color: rgb(17, 85, 204);">http://goo.gl/xElrB</a> .  If you have already filled it out, no need to do it again!<p /> Once we have the details of the conference settled, further information will be forthcoming regarding registration, accommodations, and possible financial support.<p />For specific questions regarding the workshop, please email John ZuHone at jzuhone [at] <a href="http://milkyway.gsfc.nasa.gov/" target="_blank" style="color: rgb(17, 85, 204);">milkyway.gsfc.nasa.gov</a>.<br /> </span> </p> <p><a href="http://blog.yt-project.org/announcing-the-2012-yt-workshop">Permalink</a> | <a href="http://blog.yt-project.org/announcing-the-2012-yt-workshop#comment">Leave a comment&nbsp;&nbsp;&raquo;</a> </p><img src="http://feeds.feedburner.com/~r/EnzotoolsBlog/~4/AluAedZE-8g" height="1" width="1"/> http://files.posterous.com/user_profile_pics/492646/hs_05.jpg http://posterous.com/users/37lnBezefhIJ Matt powersoffour Matt http://blog.yt-project.org/announcing-the-2012-yt-workshop Tue, 27 Sep 2011 13:20:56 -0700 http://feedproxy.google.com/~r/EnzotoolsBlog/~3/EUoQKvuan8k/new-screencasts http://blog.yt-project.org/new-screencasts <p> A few of the yt developers have been experimenting with screencasts to show off new features or demonstrate how to do some things.  Sam and I have both prepared screencasts on volume rendering and getting started with developing, respectively.  Check them out below, and please feel free to leave comments and let us know what you think -- not just about the screencasts, but about what they demonstrate, and if you think any of the concepts or routines could be made easier.<p /><div><iframe src="http://player.vimeo.com/video/28952722?portrait=0" frameborder="0" height="328" width="580"></iframe></div><div><iframe src="http://player.vimeo.com/video/29683855?portrait=0" frameborder="0" height="328" width="580"></iframe></div> </p> <p><a href="http://blog.yt-project.org/new-screencasts">Permalink</a> | <a href="http://blog.yt-project.org/new-screencasts#comment">Leave a comment&nbsp;&nbsp;&raquo;</a> </p><img src="http://feeds.feedburner.com/~r/EnzotoolsBlog/~4/EUoQKvuan8k" height="1" width="1"/> http://files.posterous.com/user_profile_pics/492646/hs_05.jpg http://posterous.com/users/37lnBezefhIJ Matt powersoffour Matt http://blog.yt-project.org/new-screencasts Mon, 12 Sep 2011 11:53:00 -0700 http://feedproxy.google.com/~r/EnzotoolsBlog/~3/5Tay14xRW1M/dataset-tracking-with-yt http://blog.yt-project.org/dataset-tracking-with-yt <p> <p>In this post I'd like to discuss a bit of work in progress to highlight some exciting new features that we hope to have working in yt sometime soon.</p> <p>On any machine that runs yt, there is a file created in the users home directory named&nbsp;<em>~/.yt/parameter_files.csv</em>&nbsp;that yt uses internally to keep track of datasets it has seen. This is just a simple text file containing comma-separated entries with a few pieces of information about datasets, like their location on disk and the last date and time they were 'seen' by yt. To keep this file from exploding, it's kept at some maximum number of entries. But, clearly, text is not the ideal way to store this kind of information for anything over a few hundred entries. Recently Matt has been working on updating this system to use a SQLite database, which should have several advantages over the text file in terms of speed and disk usage.</p> <p>This got me thinking about what could be done to extend this local listing of datasets into something more useful, <em>globally</em>. What if there was a way to view any and all datasets ever seen by yt in one convenient place? It could be searchable over a number of attributes, including creation date and when it was last seen by yt, and it would list which machine the dataset is stored on. Finally, this functionality should be transparent to the user once it is set up (with minimal effort) - the global listing of datasets should just be updated automatically in the background as part of the normal workflow.</p> <p>Over a couple days last week I did a quick and dirty implementation of this using <a href="http://aws.amazon.com/simpledb/">Amazon AWS SimpleDB</a>&nbsp;and a simple web-cgi script I wrote in Python. The advantages of SimpleDB are that it is "in the cloud" (sheesh) and very inexpensive. In fact, for small databases with low usage levels, it is free. (As an aside, Amazon is very generous with academic grants, which could be used for this or other yt-related services.) The Python script is very simple and can be <a href="https://bitbucket.org/sskory/mydb/overview">cloned off of BitBucket</a>. The script can be run on any computer with a webserver and Python (which includes Macs and Linux machines), and I envision a website (perhaps mydb.yt-project.org, for example) being created where a user can login from anywhere to view their datasets easily.</p> <p>The entire thing is not finished yet: the updates to SimpleDB are not automatic, nor have we settled on a final list of which attributes to store in the listing.&nbsp;However, in two days I was able to get enough working to show what I think are the key killer features of the system in a screencast which I've linked below. I should note that in the time since I made the screencast, I have made a few improvements. In particular, the numerical columns can now be sorted correctly.</p> <p>I'm excited about the prospects for a simple system like this!</p> <p><iframe src="http://player.vimeo.com/video/28797703?portrait=0" frameborder="0" height="328" width="580"></iframe></p> </p> <p><a href="http://blog.yt-project.org/dataset-tracking-with-yt">Permalink</a> | <a href="http://blog.yt-project.org/dataset-tracking-with-yt#comment">Leave a comment&nbsp;&nbsp;&raquo;</a> </p><img src="http://feeds.feedburner.com/~r/EnzotoolsBlog/~4/5Tay14xRW1M" height="1" width="1"/> http://posterous.com/images/profile/missing-user-75.png http://posterous.com/users/heOzHGBSGnjGG Stephen Skory Stephen Stephen Skory http://blog.yt-project.org/dataset-tracking-with-yt Sat, 10 Sep 2011 09:39:52 -0700 http://feedproxy.google.com/~r/EnzotoolsBlog/~3/kQ3NMNXAa_Q/optimizing-yt http://blog.yt-project.org/optimizing-yt <p> This last week, following the release of version 2.2 of yt, I spent a bit of time looking at speed improvements.  There were several places that the code was unacceptably slow:<div><ul><li>1D profiles (as noted in our method paper, even)</li> <li>Ghost-zone generation</li><li>RAMSES grid data loading</li></ul><div>The first of these was relatively easy to fix.  In the past, 1D profiles (unlike 2D profiles) were calculated using pure-python mechanisms; numpy was used for digitization, then inverse binning was conducted by the numpy &quot;where&quot; command, and these binnings were used to generate the overall histogram.  However, with 2D and 3D profiles, we used specialized C code written expressly for our purposes.  This last week I found myself waiting for profiles for too long, and I wrote a specialized C function that conducted binning in one-dimensions.  This sped up my profiling code by a factor of 3-4, depending on the specific field being profiled.</div> </div><p /><div>The second, ghost zone generation, was harder.  To generated a &#39;smoothed&#39; grid, interpolation is performed cascading down from the root grid to the final grid, allowing for a buffer region.  This helps to avoid dangling nodes.  Ideally, filling ghost zones would be easier and require less interpolation; however, as we do not restrict the characteristics of the mesh in such a way as to ease this, we have to use the most general case.  I spent some time looking over the code, however, and realized that the most general method of interpolation was being used -- which allowed for interpolation from a regular grid onto arbitrary shapes.  After writing a specialized regular-grid to regular-grid interpolator (and ensuring consistency and identicality of results) I saw a speedup of a factor of about 2.5-3 in generating ghost zones; this has applications from volume rendering to finite differencing and so on.</div> <p /><div>Finally, in the past, RAMSES grids following regridding were allowed to cross domains (i.e., processor files.)  By rewriting the regridding process to only allow regrids to exist within a single domain, I was able to speed up the process of loading data, allowing it to preload data for things like projections, as well.  Next this will be used as a load balancer, and it will also ease the process of loading particles from disk.  I am optimistic that this will also enable faster, more specific read times to bring down peak memory usage.</div> <p /><div>Hopefully over the next few months more optimization can be conducted.  If you want to test out how long something takes, particularly if it&#39;s a long-running task, I recommend using &quot;pyprof2html&quot;, which you can install with &quot;pip install pyprof2html&quot;.  Then run a profiling code:</div> <p /><div>$ python2.7 -m cProfile -o my_slow_script.cprof my_slow_script.py</div><div>$ pyprof2html my_slow_script.cprof</div> <p /><div>This will create a directory called &#39;html&#39;, which has a nice presentation of where things are slow.  If you send the .cprof file to the mailing list, we can take a look, too, and see if there are some obvious places to speed things up.</div> </p> <p><a href="http://blog.yt-project.org/optimizing-yt">Permalink</a> | <a href="http://blog.yt-project.org/optimizing-yt#comment">Leave a comment&nbsp;&nbsp;&raquo;</a> </p><img src="http://feeds.feedburner.com/~r/EnzotoolsBlog/~4/kQ3NMNXAa_Q" height="1" width="1"/> http://files.posterous.com/user_profile_pics/492646/hs_05.jpg http://posterous.com/users/37lnBezefhIJ Matt powersoffour Matt http://blog.yt-project.org/optimizing-yt Fri, 02 Sep 2011 07:56:00 -0700 http://feedproxy.google.com/~r/EnzotoolsBlog/~3/mDuzKWzSbyo/yt-22-release-announcement http://blog.yt-project.org/yt-22-release-announcement <p> <div style="background-color: transparent;"><span style=""> </span> <div style="background-color: transparent;"><span style="font-size: 11pt; font-family: Arial; color: #000000; background-color: transparent; font-weight: bold; font-style: normal; font-variant: normal; text-decoration: none; vertical-align: baseline;"><br />Release Announcement</span><p /> <span style="font-size: 11pt; font-family: Arial; color: #000000; background-color: transparent; font-weight: normal; font-style: normal; font-variant: normal; text-decoration: none; vertical-align: baseline;">(Please feel encouraged to forward this message to any other interested parties.)</span></div> </div> <p><span style="font-size: 11pt; font-family: Arial; color: #000000; background-color: transparent; font-weight: normal; font-style: normal; font-variant: normal; text-decoration: none; vertical-align: baseline;"><br /> We are proud to announce the release of yt version 2.2. &nbsp;This release includes several new features, bug fixes, and numerous improvements to the code base and documentation. &nbsp;At the new yt homepage, </span><a href="http://yt-project.org/"><span style="font-size: 11pt; font-family: Arial; color: #000099; background-color: transparent; font-weight: normal; font-style: normal; font-variant: normal; text-decoration: underline; vertical-align: baseline;">http://yt-project.org/</span></a><span style="font-size: 11pt; font-family: Arial; color: #000000; background-color: transparent; font-weight: normal; font-style: normal; font-variant: normal; text-decoration: none; vertical-align: baseline;"> , an installation script, a cookbook, documentation and a guide to getting involved can be found. &nbsp;We are particularly proud of the new GUI, entitled &ldquo;Reason,&rdquo; which allows real-time exploration of datasets, and which can be used (locally or remotely over SSH) with no dependencies other than a web browser. &nbsp;A basic demonstration of its usage can be found at: </span><a href="http://vimeo.com/groups/ytgallery/videos/28506477"><span style="font-size: 11pt; font-family: Arial; color: #000099; background-color: transparent; font-weight: normal; font-style: normal; font-variant: normal; text-decoration: underline; vertical-align: baseline;">http://vimeo.com/groups/ytgallery/videos/28506477</span></a><span style="font-size: 11pt; font-family: Arial; color: #000000; background-color: transparent; font-weight: normal; font-style: normal; font-variant: normal; text-decoration: none; vertical-align: baseline;"> </span>.<p /> <span style="font-size: 11pt; font-family: Arial; color: #000000; background-color: transparent; font-weight: normal; font-style: normal; font-variant: normal; text-decoration: none; vertical-align: baseline;">yt is a community-developed analysis and visualization toolkit for astrophysical simulation data. &nbsp;yt provides full support for Enzo, Orion, Nyx, and FLASH codes, with preliminary support for the RAMSES, ART, and Maestro codes. &nbsp;It can be used to create many common types of data products such as:</span><br /> <span style="font-size: 11pt; font-family: Arial; color: #000000; background-color: transparent; font-weight: normal; font-style: normal; font-variant: normal; text-decoration: none; vertical-align: baseline;"> </span><br /> <span style="font-size: 11pt; font-family: Arial; color: #000000; background-color: transparent; font-weight: normal; font-style: normal; font-variant: normal; text-decoration: none; vertical-align: baseline;">* Slices</span><br /> <span style="font-size: 11pt; font-family: Arial; color: #000000; background-color: transparent; font-weight: normal; font-style: normal; font-variant: normal; text-decoration: none; vertical-align: baseline;">* Projections</span><br /> <span style="font-size: 11pt; font-family: Arial; color: #000000; background-color: transparent; font-weight: normal; font-style: normal; font-variant: normal; text-decoration: none; vertical-align: baseline;">* Profiles</span><br /> <span style="font-size: 11pt; font-family: Arial; color: #000000; background-color: transparent; font-weight: normal; font-style: normal; font-variant: normal; text-decoration: none; vertical-align: baseline;">* Arbitrary Data Selection</span><br /> <span style="font-size: 11pt; font-family: Arial; color: #000000; background-color: transparent; font-weight: normal; font-style: normal; font-variant: normal; text-decoration: none; vertical-align: baseline;">* Cosmological Analysis</span><br /> <span style="font-size: 11pt; font-family: Arial; color: #000000; background-color: transparent; font-weight: normal; font-style: normal; font-variant: normal; text-decoration: none; vertical-align: baseline;">* Halo finding</span><br /> <span style="font-size: 11pt; font-family: Arial; color: #000000; background-color: transparent; font-weight: normal; font-style: normal; font-variant: normal; text-decoration: none; vertical-align: baseline;">* Parallel AMR Volume Rendering</span><br /> <span style="font-size: 11pt; font-family: Arial; color: #000000; background-color: transparent; font-weight: normal; font-style: normal; font-variant: normal; text-decoration: none; vertical-align: baseline;">* Gravitationally Bound Objects Analysis</span><p /> <span style="font-size: 11pt; font-family: Arial; color: #000000; background-color: transparent; font-weight: normal; font-style: normal; font-variant: normal; text-decoration: none; vertical-align: baseline;">There are a few major additions since yt-2.1 (Released April 8, 2011), including:</span><p /> <span style="font-size: 11pt; font-family: Arial; color: #000000; background-color: transparent; font-weight: normal; font-style: normal; font-variant: normal; text-decoration: none; vertical-align: baseline;">* New web GUI &ldquo;Reason,&rdquo; designed for efficient remote usage over SSH tunnels</span><br /> <span style="font-size: 11pt; font-family: Arial; color: #000000; background-color: transparent; font-weight: normal; font-style: normal; font-variant: normal; text-decoration: none; vertical-align: baseline;">* Command-line submission to the yt Hub </span><a><span style="font-size: 11pt; font-family: Arial; color: #000099; background-color: transparent; font-weight: normal; font-style: normal; font-variant: normal; text-decoration: underline; vertical-align: baseline;">(http://hub.yt-project.org/)</span></a><br /> <span style="font-size: 11pt; font-family: Arial; color: #000000; background-color: transparent; font-weight: normal; font-style: normal; font-variant: normal; text-decoration: none; vertical-align: baseline;">* Absorption line spectrum generator for cosmological simulations</span><br /> <span style="font-size: 11pt; font-family: Arial; color: #000000; background-color: transparent; font-weight: normal; font-style: normal; font-variant: normal; text-decoration: none; vertical-align: baseline;">* Support for the Nyx code</span><br /> <span style="font-size: 11pt; font-family: Arial; color: #000000; background-color: transparent; font-weight: normal; font-style: normal; font-variant: normal; text-decoration: none; vertical-align: baseline;">* An order of magnitude speed improvement in the RAMSES support</span><br /> <span style="font-size: 11pt; font-family: Arial; color: #000000; background-color: transparent; font-weight: normal; font-style: normal; font-variant: normal; text-decoration: none; vertical-align: baseline;">* Experimental interoperability with ParaView</span><br /> <span style="font-size: 11pt; font-family: Arial; color: #000000; background-color: transparent; font-weight: normal; font-style: normal; font-variant: normal; text-decoration: none; vertical-align: baseline;">* Quad-tree projections, speeding up the process of projecting by up to an order of magnitude and providing better load balancing</span><br /> <span style="font-size: 11pt; font-family: Arial; color: #000000; background-color: transparent; font-weight: normal; font-style: normal; font-variant: normal; text-decoration: none; vertical-align: baseline;">* &ldquo;mapserver&rdquo; for in-browser, Google Maps-style slice and projection visualization</span><br /> <span style="font-size: 11pt; font-family: Arial; color: #000000; background-color: transparent; font-weight: normal; font-style: normal; font-variant: normal; text-decoration: none; vertical-align: baseline;">* Many bug fixes and performance improvements</span><p /> <span style="font-size: 11pt; font-family: Arial; color: #000000; background-color: transparent; font-weight: normal; font-style: normal; font-variant: normal; text-decoration: none; vertical-align: baseline;">With this release, we also unveil the yt Hub, an astrophysical simulation-specific location for sharing scripts, analysis and visualization tools, documents and repositories used to generated publications. &nbsp;The yt Hub has been designed to allow programmatic access from the command line, and we encourage you to browse the current offerings and contribute your own. &nbsp;The yt Hub can be found at </span><a href="http://hub.yt-project.org/"><span style="font-size: 11pt; font-family: Arial; color: #000099; background-color: transparent; font-weight: normal; font-style: normal; font-variant: normal; text-decoration: underline; vertical-align: baseline;">http://hub.yt-project.org/</span></a><span style="font-size: 11pt; font-family: Arial; color: #000000; background-color: transparent; font-weight: normal; font-style: normal; font-variant: normal; text-decoration: none; vertical-align: baseline;"> .</span><p /> <span style="font-size: 11pt; font-family: Arial; color: #000000; background-color: transparent; font-weight: normal; font-style: normal; font-variant: normal; text-decoration: none; vertical-align: baseline;">Documentation: </span><a href="http://yt-project.org/doc/"><span style="font-size: 11pt; font-family: Arial; color: #000099; background-color: transparent; font-weight: normal; font-style: normal; font-variant: normal; text-decoration: underline; vertical-align: baseline;">http://yt-project.org/doc/</span></a><br /> <span style="font-size: 11pt; font-family: Arial; color: #000000; background-color: transparent; font-weight: normal; font-style: normal; font-variant: normal; text-decoration: none; vertical-align: baseline;">Installation:</span><a href="http://yt-project.org/doc/advanced/installing.html"><span style="font-size: 11pt; font-family: Arial; color: #000099; background-color: transparent; font-weight: normal; font-style: normal; font-variant: normal; text-decoration: underline; vertical-align: baseline;">http://yt-project.org/doc/advanced/installing.html</span></a><br /> <span style="font-size: 11pt; font-family: Arial; color: #000000; background-color: transparent; font-weight: normal; font-style: normal; font-variant: normal; text-decoration: none; vertical-align: baseline;">Cookbook: </span><a href="http://yt-project.org/doc/cookbook/recipes.html"><span style="font-size: 11pt; font-family: Arial; color: #000099; background-color: transparent; font-weight: normal; font-style: normal; font-variant: normal; text-decoration: underline; vertical-align: baseline;">http://yt-project.org/doc/cookbook/recipes.html</span></a><br /> <span style="font-size: 11pt; font-family: Arial; color: #000000; background-color: transparent; font-weight: normal; font-style: normal; font-variant: normal; text-decoration: none; vertical-align: baseline;">Get Involved:</span><a href="http://yt-project.org/doc/advanced/developing.html"><span style="font-size: 11pt; font-family: Arial; color: #000099; background-color: transparent; font-weight: normal; font-style: normal; font-variant: normal; text-decoration: underline; vertical-align: baseline;">http://yt-project.org/doc/advanced/developing.html</span></a><p /> <span style="font-size: 11pt; font-family: Arial; color: #000000; background-color: transparent; font-weight: normal; font-style: normal; font-variant: normal; text-decoration: none; vertical-align: baseline;">If you can&rsquo;t wait to get started, install with: </span><p /> <span style="font-size: 11pt; font-family: Arial; color: #000000; background-color: transparent; font-weight: normal; font-style: normal; font-variant: normal; text-decoration: none; vertical-align: baseline;">$ wget <a href="http://hg.yt-project.org/yt/raw/stable/doc/install_script.sh" class="moz-txt-link-freetext">http://hg.yt-project.org/yt/raw/stable/doc/install_script.sh</a></span><br /> <span style="font-size: 11pt; font-family: Arial; color: #000000; background-color: transparent; font-weight: normal; font-style: normal; font-variant: normal; text-decoration: none; vertical-align: baseline;">$ bash </span><span style="font-size: 11pt; font-family: Arial; color: #000099; background-color: transparent; font-weight: normal; font-style: normal; font-variant: normal; text-decoration: underline; vertical-align: baseline;">install_script.sh</span></p> --> <p><p /> <span style="font-size: 11pt; font-family: Arial; color: #000000; background-color: transparent; font-weight: normal; font-style: normal; font-variant: normal; text-decoration: none; vertical-align: baseline;">Development has been sponsored by the NSF, DOE, and University funding. &nbsp;We invite you to get involved with developing and using yt!</span><p /> <span style="font-size: 11pt; font-family: Arial; color: #000000; background-color: transparent; font-weight: normal; font-style: normal; font-variant: normal; text-decoration: none; vertical-align: baseline;">Please forward this announcement to interested parties.</span><p /> <span style="font-size: 11pt; font-family: Arial; color: #000000; background-color: transparent; font-weight: normal; font-style: normal; font-variant: normal; text-decoration: none; vertical-align: baseline;">Sincerely, </span><p /> <span style="font-size: 11pt; font-family: Arial; color: #000000; background-color: transparent; font-weight: normal; font-style: normal; font-variant: normal; text-decoration: none; vertical-align: baseline;">The yt development team<br /> </span></p> <p>&nbsp;</p> </p> <p><a href="http://blog.yt-project.org/yt-22-release-announcement">Permalink</a> | <a href="http://blog.yt-project.org/yt-22-release-announcement#comment">Leave a comment&nbsp;&nbsp;&raquo;</a> </p><img src="http://feeds.feedburner.com/~r/EnzotoolsBlog/~4/mDuzKWzSbyo" height="1" width="1"/> http://blog.yt-project.org/yt-22-release-announcement Wed, 31 Aug 2011 03:58:50 -0700 http://feedproxy.google.com/~r/EnzotoolsBlog/~3/NjqxHJiAPwc/this-blog-has-moved-but-you-dont-have-to http://blog.yt-project.org/this-blog-has-moved-but-you-dont-have-to <p> In keeping with the <a href="http://blog.yt-project.org/were-now-the-yt-project">project rename</a> we&#39;ve moved this blog from it&#39;s old home at <a href="http://blog.enzotools.org">blog.enzotools.org</a> to its new home at <a href="http://blog.yt-project.org">blog.yt-project.org</a>.  But, we&#39;ve put in a few redirects, and the RSS feed hasn&#39;t moved, so you shouldn&#39;t need to do anything different to get here.  We&#39;ve also enabled anonymous commenting, so feel free to comment below.  (But, of course, letting us know who you are would certainly help with keeping in touch!) </p> <p><a href="http://blog.yt-project.org/this-blog-has-moved-but-you-dont-have-to">Permalink</a> | <a href="http://blog.yt-project.org/this-blog-has-moved-but-you-dont-have-to#comment">Leave a comment&nbsp;&nbsp;&raquo;</a> </p><img src="http://feeds.feedburner.com/~r/EnzotoolsBlog/~4/NjqxHJiAPwc" height="1" width="1"/> http://files.posterous.com/user_profile_pics/492646/hs_05.jpg http://posterous.com/users/37lnBezefhIJ Matt powersoffour Matt http://blog.yt-project.org/this-blog-has-moved-but-you-dont-have-to Wed, 24 Aug 2011 14:28:30 -0700 http://feedproxy.google.com/~r/EnzotoolsBlog/~3/krhxo5hEiNk/introducing-the-yt-hub http://blog.yt-project.org/introducing-the-yt-hub <p> I am very excited to announce the yt Hub: <a href="http://hub.yt-project.org/" target="_blank">http://hub.yt-project.org/</a>.<p />The yt Hub is a gathering place for all things related to running, analyzing, and<br /> presenting astrophysical simulations.  Once upon a time, we had the Enzotools<br /> Barn, which was a place to share Enzo scripts unrelated to yt, including hosting<br />of those scripts.  The Barn was a minor success, but had a very limited scope<br />and a somewhat clunky user experience.<p />Now we have the yt Hub, where users can submit externally-hosted projects in<br /> a variety of categories:<br />- yt scripts<br />- Analysis and visualization tools and utilities<br />- Simulation and data management<br />- Scripts for making figures from published papers<br />- News and announcements<br />- General astrophysical utilities<p /> Users can create their own accounts and share anything they’ve created that might<br />be useful to others.  Currently, the preferred submission method is to create a<br />repository on <a href="http://bitbucket.org/" target="_blank">bitbucket.org</a> and then submit the link to that.  If you would like to host<br /> the repository for your contribution somewhere else, that’s fine, too.  The forthcoming<br />2.2 release of yt will include a utility for automating the creation and uploading of a<br /><a href="http://bitbucket.org/" target="_blank">bitbucket.org</a> repository as well as submitting that repository to the Hub.<p /> In addition, users may also subscribe to email announcements, in which they will<br />receive an email with any new submissions from the previous day.<p />Please, come check it out and contribute.  The site is brand new, so feel free to send<br /> feedback if you encounter any oddities or think of improvements.  The primary goal of<br />yt has been to create a common analysis language for computational astrophysicists<br />coming from different simulation codes.  We hope that the yt Hub will serve as the<br /> meeting place for the conversation. </p> <p><a href="http://blog.yt-project.org/introducing-the-yt-hub">Permalink</a> | <a href="http://blog.yt-project.org/introducing-the-yt-hub#comment">Leave a comment&nbsp;&nbsp;&raquo;</a> </p><img src="http://feeds.feedburner.com/~r/EnzotoolsBlog/~4/krhxo5hEiNk" height="1" width="1"/> http://posterous.com/images/profile/missing-user-75.png http://posterous.com/users/hdoupEkNdEbyO brittonsmith brittonsmith brittonsmith http://blog.yt-project.org/introducing-the-yt-hub Mon, 22 Aug 2011 09:39:56 -0700 http://feedproxy.google.com/~r/EnzotoolsBlog/~3/DQcaYaXuWEQ/were-now-the-yt-project http://blog.yt-project.org/were-now-the-yt-project <p> This is just a brief blog entry -- other news is on the way soon! -- to let everyone know we&#39;ve updated our domains.  In keeping with the change in focus of the yt project, expanding to work with multiple codes and take broader views of analysis and visualization, we have moved from <a href="http://enzotools.org">enzotools.org</a> to <a href="http://yt-project.org">yt-project.org</a>.  Some other exciting changes will be coming from this in the future, too!<p /><div>For more info, see the <a href="http://goo.gl/aHTGv">mailing list announcement</a>.</div> </p> <p><a href="http://blog.yt-project.org/were-now-the-yt-project">Permalink</a> | <a href="http://blog.yt-project.org/were-now-the-yt-project#comment">Leave a comment&nbsp;&nbsp;&raquo;</a> </p><img src="http://feeds.feedburner.com/~r/EnzotoolsBlog/~4/DQcaYaXuWEQ" height="1" width="1"/> http://files.posterous.com/user_profile_pics/492646/hs_05.jpg http://posterous.com/users/37lnBezefhIJ Matt powersoffour Matt http://blog.yt-project.org/were-now-the-yt-project Mon, 08 Aug 2011 21:49:46 -0700 http://feedproxy.google.com/~r/EnzotoolsBlog/~3/fidnMdE6Z30/a-movie-of-yt-in-paraview http://blog.yt-project.org/a-movie-of-yt-in-paraview <p> Jorge posted this video of using yt inside ParaView.  It demonstrated multi-res slices, projections, and even volume rendering with an adjustable transfer function.  I think it&#39;s pretty incredible.  Thanks very much to Jorge and Kitware for their hard work, and it&#39;ll be exciting to see where it goes from here!<p /> <iframe src="http://www.youtube.com/embed/cOv4Ob2q1fM?wmode=transparent" allowfullscreen frameborder="0" height="417" width="500"></iframe> </p> <p><a href="http://blog.yt-project.org/a-movie-of-yt-in-paraview">Permalink</a> | <a href="http://blog.yt-project.org/a-movie-of-yt-in-paraview#comment">Leave a comment&nbsp;&nbsp;&raquo;</a> </p><img src="http://feeds.feedburner.com/~r/EnzotoolsBlog/~4/fidnMdE6Z30" height="1" width="1"/> http://files.posterous.com/user_profile_pics/492646/hs_05.jpg http://posterous.com/users/37lnBezefhIJ Matt powersoffour Matt http://blog.yt-project.org/a-movie-of-yt-in-paraview Wed, 29 Jun 2011 19:28:05 -0700 http://feedproxy.google.com/~r/EnzotoolsBlog/~3/bAUSllT1NHo/a-mission-statement-and-a-new-homepage http://blog.yt-project.org/a-mission-statement-and-a-new-homepage <p> In case you haven&#39;t visited our front page lately, the yt homepage has been <a href="http://yt.enzotools.org/">redesigned</a>!  The old homepage was nice and functional, but it tried to cram too much into too small a place.  The new homepage focuses on the three main questions people have when they go to the yt page: &quot;How do I get it?&quot;, &quot;How do I use it?&quot;, and &quot;How do I participate?&quot;<p /><div>The new page is focused on some concrete examples, some quantitative analysis, and features a heavy dosage of the yt community -- in fact, the redesign came about through a conversation in IRC, where a few of the core developers were speculating about how best to convey how to participate and how strong a community yt has.  The new homepage also features a mission statement.  (<a href="http://goo.gl/RNUzH">Discussion of the mission statement</a>.)  It&#39;s designed to reflect the goals and ambitions -- not where the end of the road is, but which direction it is pointed.  To be honest, as the person who initiated that discussion, I was a bit surprised at how far reaching the discussion became.  The mission statement we converged on is reproduced here, but over time it might evolve and be refined.</div> <p /><i>The yt project aims to produce an integrated science environment for collaboratively asking and answering astrophysical questions. To do so, it will encompass the creation of initial conditions, the execution of simulations, and the detailed exploration and visualization of the resultant data. It will also provide a standard framework based on physical quantities interoperability between codes.<p /> </i><div><div><i>Development of yt is driven by a commitment to Open Science principles as manifested in participatory development, reproducibility, documented and approachable code, a friendly and helpful community of users and developers, and Free and Libre Open Source Software.</i></div> </div><p /><div>Feel free to stop by the IRC channel or the mailing lists if this piques your interest and you&#39;d like to chat more.</div> </p> <p><a href="http://blog.yt-project.org/a-mission-statement-and-a-new-homepage">Permalink</a> | <a href="http://blog.yt-project.org/a-mission-statement-and-a-new-homepage#comment">Leave a comment&nbsp;&nbsp;&raquo;</a> </p><img src="http://feeds.feedburner.com/~r/EnzotoolsBlog/~4/bAUSllT1NHo" height="1" width="1"/> http://files.posterous.com/user_profile_pics/492646/hs_05.jpg http://posterous.com/users/37lnBezefhIJ Matt powersoffour Matt http://blog.yt-project.org/a-mission-statement-and-a-new-homepage Mon, 13 Jun 2011 21:09:34 -0700 http://feedproxy.google.com/~r/EnzotoolsBlog/~3/cjkBmS8_tuo/paraview-and-yt http://blog.yt-project.org/paraview-and-yt <p> <p>Thanks to some awesome work by developers at Kitware, yt can now be called from within ParaView!</p> <p>This relies on a couple things, all of which are pretty exciting moving forward. The idea behind this is to leverage where the two codes have their own strengths and weaknesses, and identify places where they can work productively together. For instance, yt has been designed to provide astrophysical analysis; in this way, it can generate adaptive projections, spectral energy distributions and so on. By feeding data to yt (and not asking yt to read any on its own) these functions can be called from within ParaView, and images and results returned back to ParaView.</p> <p>The strategy taken has, so far, been a light one. Almost all of the yt machinery has been left in place; what happens is a relatively simple process:</p> <ul> <li>Load data into ParaView</li> <li>Feed data into yt <ul> <li>Construct IO handlers that pass the contents of internal VTK structures to yt through a custom <code>IOHandlerBase</code> implementation.</li> <li>Construct a hierarchy from the existing contents of the <code>vtkHierarchicalBoxDataSet</code> object, and its attendant child objects.</li> </ul> </li> <li>Execute analysis</li> <li>Return processed data to paraview</li> </ul> <p>We&rsquo;re currently investigating how to streamline this process; currently, it utilizes a new frontend I wrote that was designed to stream data into yt from arbitrary locations. This can act as a proxy for datasets that are already loaded into yt, for datasets that are constructed <em>ab initio</em> in yt and for accepting data through in-memory transfer from ParaView! One of the fun applications of this will be using ParaView as a means for conducting [CoProcessing][<a href="http://www.kitware.com/blog/home/post/28">http://www.kitware.com/blog/home/post/28</a>] of data.</p> <p>Thanks very much to Jorge, Berk, Charles and George at Kitware for all your hard work to make this happen. This is going to be a really fun way to explore new methods for analysis moving forward.</p> <p>Not only is this exciting because it enables better cross-talk between yt and ParaView, but also because of the technology that is being developed on both sides. The stream handler in yt, in particular, is a great output from this: it can be used in the future to construct arbitrary datasets in memory, as well as to prototype new frontends for codebases. When yt gains the ability to write out datasets for various code types, this will be a valuable tool for constructing datasets from scratch.</p> <p>For more information, see [this post][ <a href="http://public.kitware.com/pipermail/amr/2011-June/000027.html">http://public.kitware.com/pipermail/amr/2011-June/000027.html</a>] on the AMR mailing list at Kitware, which contains an example script. While this functionality will be in the 2.2 release, it will be highlighted in the 2.3 release as we streamline and work out bugs in a proper ParaView frontend.</p> </p> <p><a href="http://blog.yt-project.org/paraview-and-yt">Permalink</a> | <a href="http://blog.yt-project.org/paraview-and-yt#comment">Leave a comment&nbsp;&nbsp;&raquo;</a> </p><img src="http://feeds.feedburner.com/~r/EnzotoolsBlog/~4/cjkBmS8_tuo" height="1" width="1"/> http://files.posterous.com/user_profile_pics/492646/hs_05.jpg http://posterous.com/users/37lnBezefhIJ Matt powersoffour Matt http://blog.yt-project.org/paraview-and-yt Thu, 09 Jun 2011 22:08:48 -0700 http://feedproxy.google.com/~r/EnzotoolsBlog/~3/9OlnXszbpMk/gmaps-ify-your-data http://blog.yt-project.org/gmaps-ify-your-data <p> A couple of us have been working on a comprehensive notebook web GUI for yt.  It&#39;s not ready yet.<p />BUT!  In advance of that, we&#39;ve rolled a portion of that into something called the &quot;mapserver&quot; into the development branch.  This is a small, standalone webapp that implements a rendered-on-the-fly google maps interface in yt.  To run it, just go into a directory that has some data, and run:<p /> yt mapserver DD0054/DD0054<p />(where &quot;DD0054/DD0054&quot; is the same thing you&#39;d feed to &quot;load&quot; in a script.)  You can run with --help to see some options, but what it comes down to is that this will slice, but if you want to project, just do -p like so:<p /> yt mapserver -p DD0054/DD0054<p />This will spawn a webserver on port 8080 which you can then hit in a browser.  You&#39;ll have to forward an SSH tunnel if you&#39;re on a remote machine, but that&#39;s just a matter of logging in with &quot;ssh -L 8080:localhost:8080&quot;.  When you&#39;re done, just hit Ctrl-C and it&#39;ll quit.<p /><div><div class='p_embed p_image_embed'> <a href="http://posterous.com/getfile/files.posterous.com/enzotools/3ck16yRgggwl17mENcIVZHv0jrFsocTOqCsnedRGLRRKoVVkaVARyacdcreP/yt_mapserver.png"><img alt="Yt_mapserver" height="374" src="http://posterous.com/getfile/files.posterous.com/enzotools/3ck16yRgggwl17mENcIVZHv0jrFsocTOqCsnedRGLRRKoVVkaVARyacdcreP/yt_mapserver.png.scaled580.png" width="580" /></a> </div> <p />Anyway, I think this is pretty cool, and hopefully you will too.  If you run into any bugs, report them either with &quot;yt bugreport&quot; or by going to <a href="http://hg.enzotools.org/yt/issues/new">http://hg.enzotools.org/yt/issues/new</a> .<br /> </div> </p> <p><a href="http://blog.yt-project.org/gmaps-ify-your-data">Permalink</a> | <a href="http://blog.yt-project.org/gmaps-ify-your-data#comment">Leave a comment&nbsp;&nbsp;&raquo;</a> </p><img src="http://feeds.feedburner.com/~r/EnzotoolsBlog/~4/9OlnXszbpMk" height="1" width="1"/> http://files.posterous.com/user_profile_pics/492646/hs_05.jpg http://posterous.com/users/37lnBezefhIJ Matt powersoffour Matt http://blog.yt-project.org/gmaps-ify-your-data