<?xml version="1.0" encoding="utf-8"?>
<rss version="2.0" xmlns:dc="http://purl.org/dc/elements/1.1/"><channel><title>Simeon Visser (python)</title><link>http://simeonvisser.com/</link><description></description><language>en</language><lastBuildDate>Thu, 05 Jun 2014 02:47:35 GMT</lastBuildDate><generator>http://getnikola.com/</generator><docs>http://blogs.law.harvard.edu/tech/rss</docs><item><title>On Modularity in Software</title><link>http://simeonvisser.com/posts/on-modularity-in-software.html</link><dc:creator>Simeon Visser</dc:creator><description>&lt;p&gt;In 2009 I started working on a crossword editor written in Python called
&lt;a href="https://bitbucket.org/svisser/palabra"&gt;Palabra&lt;/a&gt; (Spanish for "word"). It
was a desktop application developed for Ubuntu Linux that allowed anyone to
construct a crossword grid, fill in words and enter the clues for each entry.
It also included various operations on the grid, such as shifting cells, decorating
a cell and viewing grid statistics.&lt;/p&gt;
&lt;p&gt;To create a crossword the application needs to provide all those things: a visual editor displaying
the grid, the ability to load and manipulate lists of words and so on.
Implementing everything in the same codebase seemed like a sensible approach
at the time. But looking
back at the code five years later it has become apparent that there is a much better
approach to constructing the same application: by splitting the functionality
into multiple modules that can be released and developed independently.
This approach is useful for very large software projects but it also applies to
seemingly small applications such as a crossword editor.&lt;/p&gt;
&lt;p&gt;For example, the application needs to have the ability to read and write the
crossword to a file format. At the time (and still today) there are various
file formats in use in the crossword community that a crossword editor could
support. This includes the .puz file format, the .jpz file format and the
&lt;a href="http://www.ipuz.org"&gt;.ipuz file format&lt;/a&gt;.&lt;/p&gt;
&lt;p&gt;It's possible to include the read / write functionality for multiple file formats as part of
the application's code. After all it makes little sense to use a
crossword editor if you can't save your work. But this
functionality can be reused in other applications or libraries without
needing the full code of the crossword editor. One can imagine a Python
library for .puz files, a Python library for .jpz files and a Python library
for .ipuz files which can be maintained independently. The
Python library for the .ipuz file format didn't exist yet so I have started
to &lt;a href="https://github.com/svisser/ipuz"&gt;work on that&lt;/a&gt; (&lt;a href="https://pypi.python.org/pypi/ipuz"&gt;entry on PyPI&lt;/a&gt;).
These modules can then be used by the Palabra crossword editor.&lt;/p&gt;
&lt;p&gt;Taking this one step further we can now explore how a monolithic crossword
editor can be split up into multiple modules:&lt;/p&gt;
&lt;ul&gt;&lt;li&gt;A module for reading / writing / validating .puz files.&lt;/li&gt;
&lt;li&gt;A module for reading / writing / validating .jpz files.&lt;/li&gt;
&lt;li&gt;A module for reading / writing / validating .xpf files.&lt;/li&gt;
&lt;li&gt;A module for reading / writing / validating .ipuz files.&lt;/li&gt;
&lt;li&gt;A module with a canonical data structure for a crossword grid plus common operations on the grid.&lt;/li&gt;
&lt;li&gt;A module for filling a crossword grid with words, either manually, semi-assisted or fully-automated.&lt;/li&gt;
&lt;li&gt;A module for loading and processing word lists (including fast searches, such as "give me all five-letter words with an S in the fourth position").&lt;/li&gt;
&lt;/ul&gt;&lt;p&gt;Each of these modules can be useful for other applications or purposes. It also
means that the desktop crossword editor needs to do no more than expose the functionality
of the above modules to the user. For example, it is possible to create a
web-based crossword editor by reusing all the Python modules and by
reimplementing only the logic of the editor.&lt;/p&gt;</description><category>python</category><category>software</category><guid>http://simeonvisser.com/posts/on-modularity-in-software.html</guid><pubDate>Sat, 24 May 2014 19:00:00 GMT</pubDate></item><item><title>How Does 'from __future__ import ...' Work in Python?</title><link>http://simeonvisser.com/posts/how-does-from-future-import-work-in-python.html</link><dc:creator>Simeon Visser</dc:creator><description>&lt;p&gt;A few days ago I learned that &lt;code&gt;from __future__ import barry_as_FLUFL&lt;/code&gt; allows you
to use the &lt;code&gt;&amp;lt;&amp;gt;&lt;/code&gt; operator again for inequality in Python 3.3:&lt;/p&gt;
&lt;pre class="code literal-block"&gt;&lt;span class="o"&gt;&amp;gt;&amp;gt;&amp;gt;&lt;/span&gt; &lt;span class="mi"&gt;3&lt;/span&gt; &lt;span class="o"&gt;&amp;lt;&amp;gt;&lt;/span&gt; &lt;span class="mi"&gt;4&lt;/span&gt;
  &lt;span class="nb"&gt;File&lt;/span&gt; &lt;span class="s2"&gt;"&amp;lt;stdin&amp;gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="nb"&gt;line&lt;/span&gt; &lt;span class="mi"&gt;1&lt;/span&gt;
    &lt;span class="mi"&gt;3&lt;/span&gt; &lt;span class="o"&gt;&amp;lt;&amp;gt;&lt;/span&gt; &lt;span class="mi"&gt;4&lt;/span&gt;
       &lt;span class="p"&gt;^&lt;/span&gt;
&lt;span class="nx"&gt;SyntaxError&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nx"&gt;invalid&lt;/span&gt; &lt;span class="nx"&gt;syntax&lt;/span&gt;
&lt;span class="o"&gt;&amp;gt;&amp;gt;&amp;gt;&lt;/span&gt; &lt;span class="nb"&gt;from&lt;/span&gt; &lt;span class="nx"&gt;__future__&lt;/span&gt; &lt;span class="k"&gt;import&lt;/span&gt; &lt;span class="nx"&gt;barry_as_FLUFL&lt;/span&gt;
&lt;span class="o"&gt;&amp;gt;&amp;gt;&amp;gt;&lt;/span&gt; &lt;span class="mi"&gt;3&lt;/span&gt; &lt;span class="o"&gt;&amp;lt;&amp;gt;&lt;/span&gt; &lt;span class="mi"&gt;4&lt;/span&gt;
&lt;span class="kc"&gt;True&lt;/span&gt;
&lt;/pre&gt;


&lt;p&gt;&lt;a href="http://www.python.org/dev/peps/pep-0401/"&gt;PEP 401&lt;/a&gt;
details the history behind this import.&lt;/p&gt;
&lt;p&gt;Those familiar with
Python will know that the &lt;code&gt;__future__&lt;/code&gt; module is used to make functionality
available in the current version of Python even though it will only be officially
introduced in a future version.&lt;/p&gt;
&lt;p&gt;For example, &lt;code&gt;from __future__ import with_statement&lt;/code&gt; allows you to use the
&lt;a href="http://www.python.org/dev/peps/pep-0343/"&gt;with statement&lt;/a&gt; in Python 2.5 but it
is part of the language as of Python 2.6.&lt;/p&gt;
&lt;p&gt;The syntax &lt;code&gt;from module import function&lt;/code&gt; generally means that the function
from the specified module is made available in the current scope and that it
can be called.&lt;/p&gt;
&lt;p&gt;But the earlier examples demonstrate that importing can also
make new operators or keywords available. This poses the question of
how this module actually works as it is somehow different from a normal
import: Python does not allow anyone to implement new operators or keywords
so how can we import them from a seemingly normal module &lt;code&gt;__future__&lt;/code&gt;?&lt;/p&gt;
&lt;h4&gt;How does it work?&lt;/h4&gt;
&lt;p&gt;Let's have a look at the &lt;a href="http://hg.python.org/cpython/file/3.3/Lib/__future__.py"&gt;source of the future module&lt;/a&gt;.
It turns out that anything you can import from &lt;code&gt;__future__&lt;/code&gt; has been hardcoded
into the language implementation. Each import is specified using a &lt;code&gt;_Feature&lt;/code&gt;
object that records the versions in which the new feature is available (using the
import and officially without the import) and also a special compiler flag.&lt;/p&gt;
&lt;p&gt;Calling &lt;code&gt;repr()&lt;/code&gt; on the imported object also shows this:&lt;/p&gt;
&lt;pre class="code literal-block"&gt;&lt;span class="o"&gt;&amp;gt;&amp;gt;&amp;gt;&lt;/span&gt; &lt;span class="n"&gt;repr&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;barry_as_FLUFL&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;span class="s"&gt;"_Feature((3, 1, 0, 'alpha', 2), (3, 9, 0, 'alpha', 0), 262144)"&lt;/span&gt;
&lt;/pre&gt;


&lt;p&gt;Each of these compiler flags is a constant that is also stored in
&lt;a href="http://hg.python.org/cpython/file/3.3/Include/compile.h"&gt;compile.h&lt;/a&gt;. This won't
tell us much as it merely defines the available imports from &lt;code&gt;__future__&lt;/code&gt;.&lt;/p&gt;
&lt;p&gt;So let's look at the actual code that analyses the code for future language
features, which is in &lt;a href="http://hg.python.org/cpython/file/3.3/Python/future.c"&gt;future.c&lt;/a&gt;.
Most importantly this file defines a function called &lt;code&gt;PyFuture_FromAST&lt;/code&gt; which
analyses the code and builds a &lt;code&gt;PyFutureFeatures&lt;/code&gt; object that records which
imported functionality from &lt;code&gt;__future__&lt;/code&gt; is needed.&lt;/p&gt;
&lt;h4&gt;This is not a normal module&lt;/h4&gt;
&lt;p&gt;We can now see why, although similar in syntax, the &lt;code&gt;__future__&lt;/code&gt; module behaves
differently from normal imports.&lt;/p&gt;
&lt;p&gt;As the new operators and keywords need to be
recognized when parsing the Python source code it is necessary for Python to
be aware of the 'futuristic imports' at a lower level than at the level of
regular imports.&lt;/p&gt;
&lt;p&gt;The abbreviation &lt;em&gt;AST&lt;/em&gt; that we saw in the name
of &lt;code&gt;PyFuture_FromAST&lt;/code&gt; refers to &lt;a href="http://en.wikipedia.org/wiki/Abstract_syntax_tree"&gt;Abstract Syntax Tree&lt;/a&gt;
and this is precisely the level at which Python needs to know which operators
and keywords are available: a source file is analysed, converted into an Abstract
Syntax Tree and this data structure is then converted into bytecode which can
be executed.&lt;/p&gt;
&lt;p&gt;I think this sums up why importing from &lt;code&gt;__future__&lt;/code&gt; is different from other
modules. One can also envision a language where operators and keywords can be
defined in the language itself and then a &lt;code&gt;__future__&lt;/code&gt; module would import those
as any other function or object.&lt;/p&gt;
&lt;p&gt;But Python is not such a language. As a result the new operators or keywords
are baked into the implementation and they can be made available using a
special &lt;code&gt;__future__&lt;/code&gt; module.&lt;/p&gt;</description><category>python</category><category>software</category><guid>http://simeonvisser.com/posts/how-does-from-future-import-work-in-python.html</guid><pubDate>Sun, 26 May 2013 23:18:00 GMT</pubDate></item><item><title>List Comprehensions Are For Lists</title><link>http://simeonvisser.com/posts/list-comprehensions-are-for-lists.html</link><dc:creator>Simeon Visser</dc:creator><description>&lt;p&gt;List comprehensions in Python are a great way of expressing a list but, as
the name suggests, they are for lists and not purely for iterating over an
iterable and calling some method.&lt;/p&gt;
&lt;p&gt;If you find yourself writing:&lt;/p&gt;
&lt;pre&gt;
[obj.some_method() for obj in my_objects]
&lt;/pre&gt;

&lt;p&gt;then you actually intended to write:&lt;/p&gt;
&lt;pre&gt;
for obj in my_objects:
    obj.some_method()
&lt;/pre&gt;

&lt;p&gt;I know, the first is a one-liner which feels exotic compared to an old-school
for loop.&lt;/p&gt;
&lt;p&gt;But the second version expresses what you want to
do: call a method on each object and ignore the return value of that method.
In the first version you're constructing a list, storing the return values and
then forgetting about the list altogether because you're not assigning to
a variable.&lt;/p&gt;
&lt;p&gt;In CPython 3.3 both cases produce similar bytecode but a list is still constructed
unneccessarily. Given that Python is designed for readability the second one expresses
better what you're doing. If you do need to store the return values in a list
then you can rewrite the code later.&lt;/p&gt;
&lt;p&gt;Similarly, the same argument applies to the following construct:&lt;/p&gt;
&lt;pre&gt;
map(lambda obj: obj.some_method(), my_objects)
&lt;/pre&gt;

&lt;p&gt;If you need the constructed list of return values then you can rewrite it as
a list comprehension. If you don't need the list then you can rewrite it as a
for loop.&lt;/p&gt;</description><category>python</category><category>software</category><guid>http://simeonvisser.com/posts/list-comprehensions-are-for-lists.html</guid><pubDate>Wed, 08 May 2013 11:44:00 GMT</pubDate></item><item><title>How to Mock Logging in Python</title><link>http://simeonvisser.com/posts/how-to-mock-logging-in-python.html</link><dc:creator>Simeon Visser</dc:creator><description>&lt;p&gt;As you become familiar with software testing and unit testing it can be 
necessary to assure yourself that information will be logged when
your software platform raises an exception or behaves unexpectedly.&lt;/p&gt;
&lt;p&gt;In other words, you've developed a new feature and you want to make sure that certain
calls to Python's logging module are performed in certain cirumstances.&lt;/p&gt;
&lt;p&gt;That's where the &lt;a href="https://pypi.python.org/pypi/mock"&gt;mock&lt;/a&gt; module comes in:
it allows us to mock the Python logging module and assert that certain calls
are made. If you're using Python 3.3 (or higher) than you can import from the
&lt;code&gt;unittest.mock&lt;/code&gt; (&lt;a href="http://docs.python.org/3/library/unittest.mock.html"&gt;documentation&lt;/a&gt;)
module rather than installing &lt;code&gt;mock&lt;/code&gt; from PyPI.&lt;/p&gt;
&lt;h4&gt;Example of mocking in unit tests&lt;/h4&gt;
&lt;p&gt;Let's say our code looks like this:&lt;/p&gt;
&lt;pre class="code literal-block"&gt;&lt;span class="kn"&gt;import&lt;/span&gt; &lt;span class="nn"&gt;logging&lt;/span&gt;

&lt;span class="k"&gt;def&lt;/span&gt; &lt;span class="nf"&gt;check_value&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;data_dict&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;value&lt;/span&gt;&lt;span class="p"&gt;):&lt;/span&gt;
    &lt;span class="k"&gt;try&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
        &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="n"&gt;data_dict&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="n"&gt;value&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt; &lt;span class="o"&gt;&amp;gt;&lt;/span&gt; &lt;span class="mi"&gt;10&lt;/span&gt;
    &lt;span class="k"&gt;except&lt;/span&gt; &lt;span class="ne"&gt;KeyError&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
        &lt;span class="n"&gt;logging&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;warn&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s"&gt;"Data does not contain '&lt;/span&gt;&lt;span class="si"&gt;%s&lt;/span&gt;&lt;span class="s"&gt;'"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;value&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
    &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="bp"&gt;False&lt;/span&gt;
&lt;/pre&gt;


&lt;p&gt;What we wish to test is that &lt;code&gt;logging.warn&lt;/code&gt; is called when the &lt;code&gt;KeyError&lt;/code&gt;
is raised. So in our unit test we call &lt;code&gt;check_value&lt;/code&gt; with parameters in such
a way that the &lt;code&gt;KeyError&lt;/code&gt; is indeed raised. For example:&lt;/p&gt;
&lt;pre class="code literal-block"&gt;&lt;span class="kn"&gt;import&lt;/span&gt; &lt;span class="nn"&gt;unittest&lt;/span&gt;
&lt;span class="kn"&gt;from&lt;/span&gt; &lt;span class="nn"&gt;my_module&lt;/span&gt; &lt;span class="kn"&gt;import&lt;/span&gt; &lt;span class="n"&gt;check_value&lt;/span&gt;

&lt;span class="k"&gt;class&lt;/span&gt; &lt;span class="nc"&gt;MyUnitTest&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;unittest&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;TestCase&lt;/span&gt;&lt;span class="p"&gt;):&lt;/span&gt;

    &lt;span class="k"&gt;def&lt;/span&gt; &lt;span class="nf"&gt;test_check_value_logs_warning&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="bp"&gt;self&lt;/span&gt;&lt;span class="p"&gt;):&lt;/span&gt;
        &lt;span class="n"&gt;check_value&lt;/span&gt;&lt;span class="p"&gt;({},&lt;/span&gt; &lt;span class="s"&gt;'key'&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;/pre&gt;


&lt;p&gt;The idea behind mocking is as follows: instead of calling the real
&lt;code&gt;warn&lt;/code&gt; function of the &lt;code&gt;logging&lt;/code&gt; module we call a fake mocked version and
that allows us to keep track of how many times the function is
called (if at all).&lt;/p&gt;
&lt;p&gt;We can do that using the &lt;code&gt;patch&lt;/code&gt; function of the &lt;code&gt;mock&lt;/code&gt; library: this
specifies what we wish to mock and what the name of the mocked object should be.&lt;/p&gt;
&lt;p&gt;We can now update our test by writing:&lt;/p&gt;
&lt;pre class="code literal-block"&gt;&lt;span class="kn"&gt;import&lt;/span&gt; &lt;span class="nn"&gt;unittest&lt;/span&gt;
&lt;span class="kn"&gt;from&lt;/span&gt; &lt;span class="nn"&gt;mock&lt;/span&gt; &lt;span class="kn"&gt;import&lt;/span&gt; &lt;span class="n"&gt;patch&lt;/span&gt;
&lt;span class="kn"&gt;from&lt;/span&gt; &lt;span class="nn"&gt;my_module&lt;/span&gt; &lt;span class="kn"&gt;import&lt;/span&gt; &lt;span class="n"&gt;check_value&lt;/span&gt;

&lt;span class="k"&gt;class&lt;/span&gt; &lt;span class="nc"&gt;MyUnitTest&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;unittest&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;TestCase&lt;/span&gt;&lt;span class="p"&gt;):&lt;/span&gt;

    &lt;span class="nd"&gt;@patch&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s"&gt;'my_module.logging'&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
    &lt;span class="k"&gt;def&lt;/span&gt; &lt;span class="nf"&gt;test_check_value_logs_warning&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="bp"&gt;self&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;mock_logging&lt;/span&gt;&lt;span class="p"&gt;):&lt;/span&gt;
        &lt;span class="n"&gt;check_value&lt;/span&gt;&lt;span class="p"&gt;({},&lt;/span&gt; &lt;span class="s"&gt;'key'&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
        &lt;span class="bp"&gt;self&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;assertTrue&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;mock_logging&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;warn&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;called&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;

&lt;span class="k"&gt;if&lt;/span&gt; &lt;span class="n"&gt;__name__&lt;/span&gt; &lt;span class="o"&gt;==&lt;/span&gt; &lt;span class="s"&gt;'__main__'&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
    &lt;span class="n"&gt;unittest&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;main&lt;/span&gt;&lt;span class="p"&gt;()&lt;/span&gt;
&lt;/pre&gt;


&lt;p&gt;This unit test passes as the assertion is indeed true. If you modify the
code in a way that it no longer logs a warning then you'll find that the
assertion in the unit test fails.&lt;/p&gt;</description><category>python</category><category>software</category><guid>http://simeonvisser.com/posts/how-to-mock-logging-in-python.html</guid><pubDate>Sat, 13 Apr 2013 21:08:00 GMT</pubDate></item><item><title>Using 'else' in Python</title><link>http://simeonvisser.com/posts/using-else-in-python.html</link><dc:creator>Simeon Visser</dc:creator><description>&lt;p&gt;Python allows you to use the 'else' keyword in places where most other
programming languages don't. I assume that you're familiar with the
basic &lt;code&gt;if/else&lt;/code&gt; where the block after the 'else' is executed when the
condition in the if-statement isn't true.&lt;/p&gt;
&lt;p&gt;In Python you can also add 'else' to a &lt;code&gt;for&lt;/code&gt; loop, a &lt;code&gt;while&lt;/code&gt; loop as well
as a &lt;code&gt;try/except&lt;/code&gt; section. I was aware of the ability to use 'else' in combination with a for
loop but I only learned of the &lt;code&gt;try/except/else&lt;/code&gt; combination a few days ago
so perhaps it's worth spreading the word about this.&lt;/p&gt;
&lt;h4&gt;Using 'else' in combination with 'if'&lt;/h4&gt;
&lt;p&gt;For completeness it's important to discuss the basics of the &lt;code&gt;if/else&lt;/code&gt; first
as the other uses of 'else' in Python differ slightly differently from this.&lt;/p&gt;
&lt;pre class="code literal-block"&gt;&lt;span class="k"&gt;if&lt;/span&gt; &lt;span class="n"&gt;condition&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
    &lt;span class="c"&gt;# executed when condition == True&lt;/span&gt;
    &lt;span class="o"&gt;...&lt;/span&gt;
&lt;span class="k"&gt;else&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
    &lt;span class="c"&gt;# executed when condition == False&lt;/span&gt;
    &lt;span class="o"&gt;...&lt;/span&gt;
&lt;/pre&gt;


&lt;p&gt;Despite the familiarity of this construct, it's worth noting that the reason
why the &lt;code&gt;else&lt;/code&gt; block is executed is explicitly mentioned in the
code (i.e., the condition in the if-statement).&lt;/p&gt;
&lt;p&gt;For the other uses of 'else'
you'll see that this isn't the case. You'll need to have read a Python guide
(or this article) to know when those 'else' blocks are executed.&lt;/p&gt;
&lt;h4&gt;Using 'else' in combination with 'for'&lt;/h4&gt;
&lt;p&gt;This is where things become interesting as the &lt;code&gt;else&lt;/code&gt; statement can lead
to cleaner code. It frequently happens that we wish to iterate over a collection
of objects and when a certain condition is met we &lt;code&gt;break&lt;/code&gt; from the loop.
For example:&lt;/p&gt;
&lt;pre class="code literal-block"&gt;&lt;span class="k"&gt;for&lt;/span&gt; &lt;span class="n"&gt;car&lt;/span&gt; &lt;span class="ow"&gt;in&lt;/span&gt; &lt;span class="n"&gt;cars&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
    &lt;span class="k"&gt;if&lt;/span&gt; &lt;span class="n"&gt;needs_repair&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;car&lt;/span&gt;&lt;span class="p"&gt;):&lt;/span&gt;
        &lt;span class="n"&gt;send_for_repair&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;car&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
        &lt;span class="k"&gt;break&lt;/span&gt;
&lt;/pre&gt;


&lt;p&gt;In this example we iterate over a collection of cars and we stop when we've
found a car that needs a repair. No further cars will be examined when the
first broken car has been found.&lt;/p&gt;
&lt;p&gt;Now, what if we wish to take a certain action
when we haven't found any car for repair? We could introduce a
variable &lt;code&gt;car_found_for_repair&lt;/code&gt; for this purpose:&lt;/p&gt;
&lt;pre class="code literal-block"&gt;&lt;span class="n"&gt;car_found_for_repair&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="bp"&gt;False&lt;/span&gt;
&lt;span class="k"&gt;for&lt;/span&gt; &lt;span class="n"&gt;car&lt;/span&gt; &lt;span class="ow"&gt;in&lt;/span&gt; &lt;span class="n"&gt;cars&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
    &lt;span class="k"&gt;if&lt;/span&gt; &lt;span class="n"&gt;needs_repair&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;car&lt;/span&gt;&lt;span class="p"&gt;):&lt;/span&gt;
        &lt;span class="n"&gt;send_for_repair&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;car&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
        &lt;span class="n"&gt;car_found_for_repair&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="bp"&gt;True&lt;/span&gt;
        &lt;span class="k"&gt;break&lt;/span&gt;
&lt;span class="k"&gt;if&lt;/span&gt; &lt;span class="ow"&gt;not&lt;/span&gt; &lt;span class="n"&gt;car_found_for_repair&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
    &lt;span class="c"&gt;# do something&lt;/span&gt;
    &lt;span class="o"&gt;...&lt;/span&gt;
&lt;/pre&gt;


&lt;p&gt;Python allows an elegant solution by adding an &lt;code&gt;else&lt;/code&gt;
statement to the for loop. The block of code after the &lt;code&gt;else&lt;/code&gt; is executed when
we have &lt;em&gt;not&lt;/em&gt; performed a &lt;code&gt;break&lt;/code&gt; in the loop. The code now looks as follows
and it behaves the same:&lt;/p&gt;
&lt;pre class="code literal-block"&gt;&lt;span class="k"&gt;for&lt;/span&gt; &lt;span class="n"&gt;car&lt;/span&gt; &lt;span class="ow"&gt;in&lt;/span&gt; &lt;span class="n"&gt;cars&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
    &lt;span class="k"&gt;if&lt;/span&gt; &lt;span class="n"&gt;needs_repair&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;car&lt;/span&gt;&lt;span class="p"&gt;):&lt;/span&gt;
        &lt;span class="n"&gt;send_for_repair&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;car&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
        &lt;span class="k"&gt;break&lt;/span&gt;
&lt;span class="k"&gt;else&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
    &lt;span class="c"&gt;# do something&lt;/span&gt;
    &lt;span class="o"&gt;...&lt;/span&gt;
&lt;/pre&gt;


&lt;p&gt;In other words, when the for loop completes successfully (i.e., without
being exited by a &lt;code&gt;break&lt;/code&gt; statement) the &lt;code&gt;else&lt;/code&gt; section is executed. But
when we break from the loop at some point then that section won't be executed.&lt;/p&gt;
&lt;h4&gt;Using 'else' in combination with 'while'&lt;/h4&gt;
&lt;p&gt;This is feature in the Python language that I only discovered when
reading the &lt;a href="http://docs.python.org/3/reference/grammar.html"&gt;grammar specification&lt;/a&gt;
of the language. This construct may not be that popular or, at least, I have not
seen it in the Python code that I have read over the years.&lt;/p&gt;
&lt;p&gt;Similar to the &lt;code&gt;for&lt;/code&gt; loop you can add an &lt;code&gt;else&lt;/code&gt; statement to a &lt;code&gt;while&lt;/code&gt; loop
and it'll be executed when a &lt;code&gt;break&lt;/code&gt; has &lt;em&gt;not&lt;/em&gt; been performed in the loop.
I'll leave the example out as it would be similar to the above one.&lt;/p&gt;
&lt;h4&gt;Using 'else' in combination with 'try' and 'except'&lt;/h4&gt;
&lt;p&gt;The basics of a &lt;code&gt;try/except&lt;/code&gt; section in Python consist of:&lt;/p&gt;
&lt;pre class="code literal-block"&gt;&lt;span class="k"&gt;try&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
    &lt;span class="c"&gt;# run some code&lt;/span&gt;
&lt;span class="k"&gt;except&lt;/span&gt; &lt;span class="ne"&gt;ValueError&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
    &lt;span class="c"&gt;# catch exception when it is raised&lt;/span&gt;
&lt;/pre&gt;


&lt;p&gt;This means the code within the &lt;code&gt;try&lt;/code&gt; is executed and when a
&lt;code&gt;ValueError&lt;/code&gt; is raised execution continues in the &lt;code&gt;except&lt;/code&gt; block. Some of you
will know that a &lt;code&gt;finally&lt;/code&gt; block can be added to execute code after a
&lt;code&gt;try/except&lt;/code&gt; regardless of whether an exception was raised:&lt;/p&gt;
&lt;pre class="code literal-block"&gt;&lt;span class="k"&gt;try&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
    &lt;span class="c"&gt;# run some code&lt;/span&gt;
&lt;span class="k"&gt;except&lt;/span&gt; &lt;span class="ne"&gt;ValueError&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
    &lt;span class="c"&gt;# run this when ValueError is raised&lt;/span&gt;
&lt;span class="k"&gt;finally&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
    &lt;span class="c"&gt;# run this after the code and any exception handling code&lt;/span&gt;
&lt;/pre&gt;


&lt;p&gt;Similar to the &lt;code&gt;for&lt;/code&gt; loop example above, what if we wish to execute some code
when &lt;em&gt;no&lt;/em&gt; exception has been raised? We could introduce another boolean variable
for that purpose but Python allows you to add an &lt;code&gt;else&lt;/code&gt; block:&lt;/p&gt;
&lt;pre class="code literal-block"&gt;&lt;span class="k"&gt;try&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
    &lt;span class="c"&gt;# run some code&lt;/span&gt;
&lt;span class="k"&gt;except&lt;/span&gt; &lt;span class="ne"&gt;ValueError&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
    &lt;span class="c"&gt;# run this when ValueError is raised&lt;/span&gt;
&lt;span class="k"&gt;else&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
    &lt;span class="c"&gt;# run this only when no exception is raised&lt;/span&gt;
&lt;span class="k"&gt;finally&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
    &lt;span class="c"&gt;# run this after the code and any exception handling code&lt;/span&gt;
&lt;/pre&gt;


&lt;p&gt;The &lt;code&gt;else&lt;/code&gt; section is only executed when no exception at all is raised.&lt;/p&gt;
&lt;p&gt;In this example we are only catching &lt;code&gt;ValueError&lt;/code&gt; exceptions. When, for example,
an &lt;code&gt;IndexError&lt;/code&gt; is raised then Python will run the &lt;code&gt;try&lt;/code&gt; block up until the
exception, the &lt;code&gt;finally&lt;/code&gt; block and the &lt;code&gt;ValueError&lt;/code&gt; won't be caught (yet).&lt;/p&gt;</description><category>python</category><category>software</category><guid>http://simeonvisser.com/posts/using-else-in-python.html</guid><pubDate>Tue, 05 Feb 2013 11:00:00 GMT</pubDate></item><item><title>Python 3: Using "yield from" in Generators - Part 2</title><link>http://simeonvisser.com/posts/python-3-using-yield-from-in-generators-part-2.html</link><dc:creator>Simeon Visser</dc:creator><description>&lt;p&gt;In the previous part of this tutorial,
I have discussed the basics of generators, some differences between functions
and generators. I have also hinted at some benefits of the new &lt;code&gt;yield from&lt;/code&gt; syntax.
I recommend reading that part of the tutorial first before continuing unless
you're well familiar with generators in Python.&lt;/p&gt;
&lt;p&gt;All examples in this part of the tutorial will be in Python 3.3 unless
stated otherwise.&lt;/p&gt;
&lt;h4&gt;Basic binary tree&lt;/h4&gt;
&lt;p&gt;In this second part I'll build upon a basic binary tree example that
demonstrates the uses of the &lt;code&gt;yield from&lt;/code&gt; syntax. For the sake of this example,
I'll let each node in the tree have a list of children rather than, as is often
done, let each node point to its parent node.&lt;/p&gt;
&lt;p&gt;Here is the implementation of this data structure without the new syntax:&lt;/p&gt;
&lt;pre class="code literal-block"&gt;&lt;span class="k"&gt;class&lt;/span&gt; &lt;span class="nc"&gt;Node&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;

    &lt;span class="k"&gt;def&lt;/span&gt; &lt;span class="nf"&gt;__init__&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="bp"&gt;self&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;value&lt;/span&gt;&lt;span class="p"&gt;):&lt;/span&gt;
        &lt;span class="bp"&gt;self&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;left&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="p"&gt;[]&lt;/span&gt;
        &lt;span class="bp"&gt;self&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;value&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;value&lt;/span&gt;
        &lt;span class="bp"&gt;self&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;right&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="p"&gt;[]&lt;/span&gt;

    &lt;span class="k"&gt;def&lt;/span&gt; &lt;span class="nf"&gt;iterate&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="bp"&gt;self&lt;/span&gt;&lt;span class="p"&gt;):&lt;/span&gt;
        &lt;span class="k"&gt;for&lt;/span&gt; &lt;span class="n"&gt;node&lt;/span&gt; &lt;span class="ow"&gt;in&lt;/span&gt; &lt;span class="bp"&gt;self&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;left&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
            &lt;span class="k"&gt;yield&lt;/span&gt; &lt;span class="n"&gt;node&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;value&lt;/span&gt;
        &lt;span class="k"&gt;yield&lt;/span&gt; &lt;span class="bp"&gt;self&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;value&lt;/span&gt;
        &lt;span class="k"&gt;for&lt;/span&gt; &lt;span class="n"&gt;node&lt;/span&gt; &lt;span class="ow"&gt;in&lt;/span&gt; &lt;span class="bp"&gt;self&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;right&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
            &lt;span class="k"&gt;yield&lt;/span&gt; &lt;span class="n"&gt;node&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;value&lt;/span&gt;

&lt;span class="k"&gt;def&lt;/span&gt; &lt;span class="nf"&gt;main&lt;/span&gt;&lt;span class="p"&gt;():&lt;/span&gt;
    &lt;span class="n"&gt;root&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;Node&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="mi"&gt;0&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
    &lt;span class="n"&gt;root&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;left&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="n"&gt;Node&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;i&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="k"&gt;for&lt;/span&gt; &lt;span class="n"&gt;i&lt;/span&gt; &lt;span class="ow"&gt;in&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="mi"&gt;1&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="mi"&gt;2&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="mi"&gt;3&lt;/span&gt;&lt;span class="p"&gt;]]&lt;/span&gt;
    &lt;span class="n"&gt;root&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;right&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="n"&gt;Node&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;i&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="k"&gt;for&lt;/span&gt; &lt;span class="n"&gt;i&lt;/span&gt; &lt;span class="ow"&gt;in&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="mi"&gt;4&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="mi"&gt;5&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="mi"&gt;6&lt;/span&gt;&lt;span class="p"&gt;]]&lt;/span&gt;
    &lt;span class="k"&gt;for&lt;/span&gt; &lt;span class="n"&gt;value&lt;/span&gt; &lt;span class="ow"&gt;in&lt;/span&gt; &lt;span class="n"&gt;root&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;iterate&lt;/span&gt;&lt;span class="p"&gt;():&lt;/span&gt;
        &lt;span class="k"&gt;print&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;value&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;

&lt;span class="k"&gt;if&lt;/span&gt; &lt;span class="n"&gt;__name__&lt;/span&gt; &lt;span class="o"&gt;==&lt;/span&gt; &lt;span class="s"&gt;"__main__"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
    &lt;span class="n"&gt;main&lt;/span&gt;&lt;span class="p"&gt;()&lt;/span&gt;
&lt;/pre&gt;


&lt;p&gt;As expected, this prints the values &lt;code&gt;1&lt;/code&gt;, &lt;code&gt;2&lt;/code&gt;, &lt;code&gt;3&lt;/code&gt; (of the left children),
the value &lt;code&gt;0&lt;/code&gt; (of the root node) and the values &lt;code&gt;4&lt;/code&gt;, &lt;code&gt;5&lt;/code&gt;, and &lt;code&gt;6&lt;/code&gt; (of the
right children).&lt;/p&gt;
&lt;p&gt;However, this example only iterates over the root node
and its children; it won't recursively iterate over the children of the
child nodes (if there happened to be any). Let's modify the &lt;code&gt;iterate()&lt;/code&gt;
method to make that happen:&lt;/p&gt;
&lt;pre class="code literal-block"&gt;&lt;span class="k"&gt;def&lt;/span&gt; &lt;span class="nf"&gt;iterate&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="bp"&gt;self&lt;/span&gt;&lt;span class="p"&gt;):&lt;/span&gt;
    &lt;span class="k"&gt;for&lt;/span&gt; &lt;span class="n"&gt;node&lt;/span&gt; &lt;span class="ow"&gt;in&lt;/span&gt; &lt;span class="bp"&gt;self&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;left&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
        &lt;span class="k"&gt;for&lt;/span&gt; &lt;span class="n"&gt;value&lt;/span&gt; &lt;span class="ow"&gt;in&lt;/span&gt; &lt;span class="n"&gt;node&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;iterate&lt;/span&gt;&lt;span class="p"&gt;():&lt;/span&gt;
            &lt;span class="k"&gt;yield&lt;/span&gt; &lt;span class="n"&gt;value&lt;/span&gt;
    &lt;span class="k"&gt;yield&lt;/span&gt; &lt;span class="bp"&gt;self&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;value&lt;/span&gt;
    &lt;span class="k"&gt;for&lt;/span&gt; &lt;span class="n"&gt;node&lt;/span&gt; &lt;span class="ow"&gt;in&lt;/span&gt; &lt;span class="bp"&gt;self&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;right&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
        &lt;span class="k"&gt;for&lt;/span&gt; &lt;span class="n"&gt;value&lt;/span&gt; &lt;span class="ow"&gt;in&lt;/span&gt; &lt;span class="n"&gt;node&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;iterate&lt;/span&gt;&lt;span class="p"&gt;():&lt;/span&gt;
            &lt;span class="k"&gt;yield&lt;/span&gt; &lt;span class="n"&gt;value&lt;/span&gt;
&lt;/pre&gt;


&lt;p&gt;This version also calls &lt;code&gt;iterate()&lt;/code&gt; on each of the child nodes so this
yields each of the nodes in the tree. This code is rather cumbersome: for
each of the left and right children we yield all the values by using
explicit iteration. This is where &lt;code&gt;yield from&lt;/code&gt;
simplifies the code:&lt;/p&gt;
&lt;pre class="code literal-block"&gt;&lt;span class="k"&gt;def&lt;/span&gt; &lt;span class="nf"&gt;iterate&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="bp"&gt;self&lt;/span&gt;&lt;span class="p"&gt;):&lt;/span&gt;
    &lt;span class="k"&gt;for&lt;/span&gt; &lt;span class="n"&gt;node&lt;/span&gt; &lt;span class="ow"&gt;in&lt;/span&gt; &lt;span class="bp"&gt;self&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;left&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
        &lt;span class="k"&gt;yield from&lt;/span&gt; &lt;span class="n"&gt;node&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;iterate&lt;/span&gt;&lt;span class="p"&gt;()&lt;/span&gt;
    &lt;span class="k"&gt;yield&lt;/span&gt; &lt;span class="bp"&gt;self&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;value&lt;/span&gt;
    &lt;span class="k"&gt;for&lt;/span&gt; &lt;span class="n"&gt;node&lt;/span&gt; &lt;span class="ow"&gt;in&lt;/span&gt; &lt;span class="bp"&gt;self&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;right&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
        &lt;span class="k"&gt;yield from&lt;/span&gt; &lt;span class="n"&gt;node&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;iterate&lt;/span&gt;&lt;span class="p"&gt;()&lt;/span&gt;
&lt;/pre&gt;


&lt;h4&gt;Other aspects of generators&lt;/h4&gt;
&lt;p&gt;Now, the story would be over pretty quickly if that was all there was to it.
To appreciate &lt;code&gt;yield from&lt;/code&gt; I'll need to explain an alternative way of using
a generator.&lt;/p&gt;
&lt;p&gt;A generator can be controlled using methods such as &lt;code&gt;send()&lt;/code&gt; and &lt;code&gt;next()&lt;/code&gt;. These
and related methods allow you to start, stop and continue a generator rather
than having Python handle most of the generator's execution.&lt;/p&gt;
&lt;p&gt;For example, instead of the above basic loop:&lt;/p&gt;
&lt;pre class="code literal-block"&gt;&lt;span class="k"&gt;for&lt;/span&gt; &lt;span class="n"&gt;value&lt;/span&gt; &lt;span class="ow"&gt;in&lt;/span&gt; &lt;span class="n"&gt;root&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;iterate&lt;/span&gt;&lt;span class="p"&gt;():&lt;/span&gt;
    &lt;span class="k"&gt;print&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;value&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;/pre&gt;


&lt;p&gt;we can also write:&lt;/p&gt;
&lt;pre class="code literal-block"&gt;&lt;span class="n"&gt;it&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;root&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;iterate&lt;/span&gt;&lt;span class="p"&gt;()&lt;/span&gt;
&lt;span class="k"&gt;while&lt;/span&gt; &lt;span class="bp"&gt;True&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
    &lt;span class="k"&gt;try&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
        &lt;span class="k"&gt;print&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;it&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;send&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="bp"&gt;None&lt;/span&gt;&lt;span class="p"&gt;))&lt;/span&gt;
    &lt;span class="k"&gt;except&lt;/span&gt; &lt;span class="ne"&gt;StopIteration&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
        &lt;span class="k"&gt;break&lt;/span&gt;
&lt;/pre&gt;


&lt;p&gt;The &lt;code&gt;send()&lt;/code&gt; method allows you to "send" a value into the generator, which means
the &lt;code&gt;yield&lt;/code&gt; expression receives that value. That value can be used by assigning
it to a variable (i.e., &lt;code&gt;v = yield self.value&lt;/code&gt;).&lt;/p&gt;
&lt;p&gt;In this case we repeatedly send
the value &lt;code&gt;None&lt;/code&gt; into the generator and our generator doesn't utilize the
value that it sent into it. Effectively this leads to the same result as
the previous loop that prints the generator's yielded values.&lt;/p&gt;
&lt;h4&gt;Benefits of yield from&lt;/h4&gt;
&lt;p&gt;The primary benefits of &lt;code&gt;yield from&lt;/code&gt; come when you've written a generator that
uses these techniques and when it needs to be refactored. This means you'll have
to subdivide the generator into multiple subgenerators and send / receive values
to and from those subgenerators. Rather than rewriting
the generator to send values to the subgenerator, you can simply use
&lt;code&gt;yield from&lt;/code&gt; and the semantics will remain the same.&lt;/p&gt;
&lt;p&gt;There are some caveats and some situations which aren't handled but that's beyond
the scope of this tutorial (I'll refer
you to &lt;a href="http://www.python.org/dev/peps/pep-0380/"&gt;the official proposal&lt;/a&gt; for details).&lt;/p&gt;
&lt;h4&gt;Another example generator&lt;/h4&gt;
&lt;p&gt;Let's create a small generator that demonstrates the above:&lt;/p&gt;
&lt;pre class="code literal-block"&gt;&lt;span class="k"&gt;def&lt;/span&gt; &lt;span class="nf"&gt;node_iterate&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="bp"&gt;self&lt;/span&gt;&lt;span class="p"&gt;):&lt;/span&gt;
    &lt;span class="k"&gt;for&lt;/span&gt; &lt;span class="n"&gt;node&lt;/span&gt; &lt;span class="ow"&gt;in&lt;/span&gt; &lt;span class="bp"&gt;self&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;left&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
        &lt;span class="n"&gt;input_value&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="k"&gt;yield&lt;/span&gt; &lt;span class="n"&gt;node&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;value&lt;/span&gt;
        &lt;span class="c"&gt;# ...&lt;/span&gt;
    &lt;span class="n"&gt;input_value&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="k"&gt;yield&lt;/span&gt; &lt;span class="bp"&gt;self&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;value&lt;/span&gt;
    &lt;span class="c"&gt;# ...&lt;/span&gt;
    &lt;span class="k"&gt;for&lt;/span&gt; &lt;span class="n"&gt;node&lt;/span&gt; &lt;span class="ow"&gt;in&lt;/span&gt; &lt;span class="bp"&gt;self&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;right&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
        &lt;span class="n"&gt;input_value&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="k"&gt;yield&lt;/span&gt; &lt;span class="n"&gt;node&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;value&lt;/span&gt;
        &lt;span class="c"&gt;# ...&lt;/span&gt;
&lt;/pre&gt;


&lt;p&gt;This generator only iterates over the node and its immediate children. Any
value sent into the generator is stored in the variable &lt;code&gt;input_value&lt;/code&gt;
which is then available for computations (the &lt;code&gt;# ...&lt;/code&gt; sections).&lt;/p&gt;
&lt;p&gt;For example, the code that uses this generator may perform various
computations and it passes intermediate values back into the generator so
that they can be used there. The following shows how the yielded values
are summed and the subtotals are passed back into the generator:&lt;/p&gt;
&lt;pre class="code literal-block"&gt;&lt;span class="n"&gt;total&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="mi"&gt;0&lt;/span&gt;
&lt;span class="n"&gt;it&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;root&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;node_iterate&lt;/span&gt;&lt;span class="p"&gt;()&lt;/span&gt;
&lt;span class="n"&gt;it&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;send&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="bp"&gt;None&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;span class="k"&gt;while&lt;/span&gt; &lt;span class="bp"&gt;True&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
    &lt;span class="k"&gt;try&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
        &lt;span class="n"&gt;value&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;it&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;send&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;total&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
        &lt;span class="n"&gt;total&lt;/span&gt; &lt;span class="o"&gt;+=&lt;/span&gt; &lt;span class="n"&gt;value&lt;/span&gt;
    &lt;span class="k"&gt;except&lt;/span&gt; &lt;span class="ne"&gt;StopIteration&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
        &lt;span class="k"&gt;break&lt;/span&gt;
&lt;/pre&gt;


&lt;h4&gt;Refactoring iteration over children&lt;/h4&gt;
&lt;p&gt;It seems repetitive to have the same code for both the left and right
children so we can refactor that into:&lt;/p&gt;
&lt;pre class="code literal-block"&gt;&lt;span class="k"&gt;def&lt;/span&gt; &lt;span class="nf"&gt;child_iterate&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="bp"&gt;self&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;nodes&lt;/span&gt;&lt;span class="p"&gt;):&lt;/span&gt;
    &lt;span class="k"&gt;for&lt;/span&gt; &lt;span class="n"&gt;node&lt;/span&gt; &lt;span class="ow"&gt;in&lt;/span&gt; &lt;span class="n"&gt;nodes&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
        &lt;span class="n"&gt;input_value&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="k"&gt;yield&lt;/span&gt; &lt;span class="n"&gt;node&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;value&lt;/span&gt;
        &lt;span class="c"&gt;# ...&lt;/span&gt;

&lt;span class="k"&gt;def&lt;/span&gt; &lt;span class="nf"&gt;node_iterate&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="bp"&gt;self&lt;/span&gt;&lt;span class="p"&gt;):&lt;/span&gt;
    &lt;span class="k"&gt;yield from&lt;/span&gt; &lt;span class="bp"&gt;self&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;child_iterate&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="bp"&gt;self&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;left&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
    &lt;span class="n"&gt;input_value&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="k"&gt;yield&lt;/span&gt; &lt;span class="bp"&gt;self&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;value&lt;/span&gt;
    &lt;span class="c"&gt;# ...&lt;/span&gt;
    &lt;span class="k"&gt;yield from&lt;/span&gt; &lt;span class="bp"&gt;self&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;child_iterate&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="bp"&gt;self&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;right&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;/pre&gt;


&lt;p&gt;Note that it is not recommended practice to call a class method
(&lt;code&gt;self.child_iterate&lt;/code&gt;) by passing an instance variable (i.e., the
&lt;code&gt;self.left&lt;/code&gt; and &lt;code&gt;self.right&lt;/code&gt; arguments) but
that is a topic for a different post. The important part is that we can only
refactor in this way as a result of the new &lt;code&gt;yield from&lt;/code&gt; semantics.&lt;/p&gt;
&lt;p&gt;When we send values into &lt;code&gt;node_iterate()&lt;/code&gt; they are automatically
passed into the subgenerator &lt;code&gt;child_iterate()&lt;/code&gt; where &lt;code&gt;input_value&lt;/code&gt;
receives a value.&lt;/p&gt;
&lt;p&gt;Prior to Python 3.3, if we had written:&lt;/p&gt;
&lt;pre class="code literal-block"&gt;&lt;span class="k"&gt;for&lt;/span&gt; &lt;span class="n"&gt;value&lt;/span&gt; &lt;span class="ow"&gt;in&lt;/span&gt; &lt;span class="bp"&gt;self&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;child_iterate&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="bp"&gt;self&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;left&lt;/span&gt;&lt;span class="p"&gt;):&lt;/span&gt;
    &lt;span class="k"&gt;yield&lt;/span&gt; &lt;span class="n"&gt;value&lt;/span&gt;
&lt;/pre&gt;


&lt;p&gt;then this would yield values from &lt;code&gt;child_iterate()&lt;/code&gt; but &lt;code&gt;input_value&lt;/code&gt; would
remain &lt;code&gt;None&lt;/code&gt;. To make it work as expected, we would need to explicitly &lt;code&gt;send()&lt;/code&gt;
the values from &lt;code&gt;node_iterate()&lt;/code&gt; into &lt;code&gt;child_iterate()&lt;/code&gt;.&lt;/p&gt;
&lt;h4&gt;More refactoring&lt;/h4&gt;
&lt;p&gt;Lastly, we can perform one more refactoring step leaving us with only one
section where the &lt;code&gt;input_value&lt;/code&gt; is received and used:&lt;/p&gt;
&lt;pre class="code literal-block"&gt;&lt;span class="k"&gt;def&lt;/span&gt; &lt;span class="nf"&gt;process&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="bp"&gt;self&lt;/span&gt;&lt;span class="p"&gt;):&lt;/span&gt;
    &lt;span class="n"&gt;input_value&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="k"&gt;yield&lt;/span&gt; &lt;span class="bp"&gt;self&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;value&lt;/span&gt;
    &lt;span class="c"&gt;# ...&lt;/span&gt;

&lt;span class="k"&gt;def&lt;/span&gt; &lt;span class="nf"&gt;child_iterate&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="bp"&gt;self&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;nodes&lt;/span&gt;&lt;span class="p"&gt;):&lt;/span&gt;
    &lt;span class="k"&gt;for&lt;/span&gt; &lt;span class="n"&gt;node&lt;/span&gt; &lt;span class="ow"&gt;in&lt;/span&gt; &lt;span class="n"&gt;nodes&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
        &lt;span class="k"&gt;yield from&lt;/span&gt; &lt;span class="n"&gt;node&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;process&lt;/span&gt;&lt;span class="p"&gt;()&lt;/span&gt;

&lt;span class="k"&gt;def&lt;/span&gt; &lt;span class="nf"&gt;node_iterate&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="bp"&gt;self&lt;/span&gt;&lt;span class="p"&gt;):&lt;/span&gt;
    &lt;span class="k"&gt;yield from&lt;/span&gt; &lt;span class="bp"&gt;self&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;child_iterate&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="bp"&gt;self&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;left&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
    &lt;span class="bp"&gt;self&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;process&lt;/span&gt;&lt;span class="p"&gt;()&lt;/span&gt;
    &lt;span class="k"&gt;yield from&lt;/span&gt; &lt;span class="bp"&gt;self&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;child_iterate&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="bp"&gt;self&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;right&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;/pre&gt;


&lt;p&gt;As you become more familiar with using &lt;code&gt;send()&lt;/code&gt; and related functions, you'll
notice that &lt;code&gt;yield from&lt;/code&gt; makes life easier. I suggest reading the exact
semantics in the &lt;a href="http://www.python.org/dev/peps/pep-0380/"&gt;specification&lt;/a&gt;
to know when and how you'll be able to use it.&lt;/p&gt;</description><category>python</category><category>software</category><guid>http://simeonvisser.com/posts/python-3-using-yield-from-in-generators-part-2.html</guid><pubDate>Mon, 04 Feb 2013 12:00:00 GMT</pubDate></item><item><title>Python 3: Using "yield from" in Generators - Part 1</title><link>http://simeonvisser.com/posts/python-3-using-yield-from-in-generators-part-1.html</link><dc:creator>Simeon Visser</dc:creator><description>&lt;p&gt;A new feature in Python 3.3 is the ability to use &lt;code&gt;yield from&lt;/code&gt; when defining
a generator. My current experience with Python 3 has been with Python 3.1 so
I've upgraded my installation in Ubuntu to Python 3.3 to explore "yield from".
In this tutorial, in two parts, I'm going to outline some examples and
potential use cases.&lt;/p&gt;
&lt;p&gt;If you're already familiar with generators then you can skip the first section
of this article and continue with the specifics of "yield from" below it.&lt;/p&gt;
&lt;h4&gt;Brief introduction to generators&lt;/h4&gt;
&lt;p&gt;In Python a generator can be used to let a function return a list of values
without having to store them all at once in memory. This also allows you to
utilize the values immediately without having to wait until all values have
been computed.&lt;/p&gt;
&lt;p&gt;Let's look at the following Python 2 function:&lt;/p&gt;
&lt;pre class="code literal-block"&gt;&lt;span class="k"&gt;def&lt;/span&gt; &lt;span class="nf"&gt;not_a_generator&lt;/span&gt;&lt;span class="p"&gt;():&lt;/span&gt;
    &lt;span class="n"&gt;result&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="p"&gt;[]&lt;/span&gt;
    &lt;span class="k"&gt;for&lt;/span&gt; &lt;span class="n"&gt;i&lt;/span&gt; &lt;span class="ow"&gt;in&lt;/span&gt; &lt;span class="nb"&gt;xrange&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="mi"&gt;2000&lt;/span&gt;&lt;span class="p"&gt;):&lt;/span&gt;
        &lt;span class="n"&gt;result&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;append&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;perform_expensive_computation&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;i&lt;/span&gt;&lt;span class="p"&gt;))&lt;/span&gt;
    &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="n"&gt;result&lt;/span&gt;
&lt;/pre&gt;


&lt;p&gt;When we call &lt;code&gt;not_a_generator()&lt;/code&gt; we have to wait until &lt;code&gt;perform_expensive_computation&lt;/code&gt;
has been performed on all 2000 integers.&lt;/p&gt;
&lt;p&gt;This is inconvenient because we may not actually end up using all the
computed results. For example, we may wish to use the above function
as follows:&lt;/p&gt;
&lt;pre class="code literal-block"&gt;&lt;span class="k"&gt;for&lt;/span&gt; &lt;span class="n"&gt;element&lt;/span&gt; &lt;span class="ow"&gt;in&lt;/span&gt; &lt;span class="n"&gt;not_a_generator&lt;/span&gt;&lt;span class="p"&gt;():&lt;/span&gt;
    &lt;span class="k"&gt;if&lt;/span&gt; &lt;span class="ow"&gt;not&lt;/span&gt; &lt;span class="n"&gt;certain_condition&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;element&lt;/span&gt;&lt;span class="p"&gt;):&lt;/span&gt;
        &lt;span class="k"&gt;break&lt;/span&gt;
    &lt;span class="c"&gt;# ...&lt;/span&gt;
&lt;/pre&gt;


&lt;p&gt;Depending on the behaviour of &lt;code&gt;certain_condition&lt;/code&gt;, it could be that we only
use the first 700 values returned from &lt;code&gt;not_a_generator()&lt;/code&gt; and we would have
wasted time on computing the remaining values.&lt;/p&gt;
&lt;p&gt;We can turn &lt;code&gt;not_a_generator()&lt;/code&gt; into a generator by using the &lt;code&gt;yield&lt;/code&gt; keyword:&lt;/p&gt;
&lt;pre class="code literal-block"&gt;&lt;span class="k"&gt;def&lt;/span&gt; &lt;span class="nf"&gt;my_generator&lt;/span&gt;&lt;span class="p"&gt;():&lt;/span&gt;
    &lt;span class="k"&gt;for&lt;/span&gt; &lt;span class="n"&gt;i&lt;/span&gt; &lt;span class="ow"&gt;in&lt;/span&gt; &lt;span class="nb"&gt;xrange&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="mi"&gt;2000&lt;/span&gt;&lt;span class="p"&gt;):&lt;/span&gt;
        &lt;span class="k"&gt;yield&lt;/span&gt; &lt;span class="n"&gt;perform_expensive_computation&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;i&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;/pre&gt;


&lt;h4&gt;Difference between a function and a generator&lt;/h4&gt;
&lt;p&gt;Calling this generator is no different from our previous function:&lt;/p&gt;
&lt;pre class="code literal-block"&gt;&lt;span class="k"&gt;for&lt;/span&gt; &lt;span class="n"&gt;element&lt;/span&gt; &lt;span class="ow"&gt;in&lt;/span&gt; &lt;span class="n"&gt;my_generator&lt;/span&gt;&lt;span class="p"&gt;():&lt;/span&gt;
    &lt;span class="k"&gt;if&lt;/span&gt; &lt;span class="ow"&gt;not&lt;/span&gt; &lt;span class="n"&gt;certain_condition&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;element&lt;/span&gt;&lt;span class="p"&gt;):&lt;/span&gt;
        &lt;span class="k"&gt;break&lt;/span&gt;
    &lt;span class="c"&gt;# ...&lt;/span&gt;
&lt;/pre&gt;


&lt;p&gt;The difference is that whenever the generator "yields" a value the execution
of the generator is paused and the code continues where the generator was
called. That means the value of the variable &lt;code&gt;element&lt;/code&gt; is known and the execution
of the code can continue.&lt;/p&gt;
&lt;p&gt;When the if-statement involving &lt;code&gt;not certain_condition(element)&lt;/code&gt; is true then
we no longer need the generator to compute the remaining values. On the other
hand, if the for loop finishes for the current value of &lt;code&gt;element&lt;/code&gt; then the
generator is executed again until it yields another value.&lt;/p&gt;
&lt;p&gt;The above will continue until all values of the generator have been yielded or
until we no longer need the generator.&lt;/p&gt;
&lt;p&gt;Note that we can easily get the behaviour of &lt;code&gt;not_a_generator()&lt;/code&gt; back by
storing the results of &lt;code&gt;my_generator&lt;/code&gt; in a list: &lt;code&gt;list(my_generator())&lt;/code&gt;. This
forces the generator to be executed fully and to yield all its values.&lt;/p&gt;
&lt;p&gt;Also note that a generator can't contain &lt;code&gt;return value&lt;/code&gt; statements: using one or
more &lt;code&gt;yield&lt;/code&gt; statements turns a function into a generator and you can only
use &lt;code&gt;yield&lt;/code&gt; to return values to where the generator is called.&lt;/p&gt;
&lt;h4&gt;Why yield from?&lt;/h4&gt;
&lt;p&gt;When a new feature is introduced in a programming language we should ask
ourselves if and why this was necessary. The short explanation is that it
enables you to easily refactor a generator by splitting it up into multiple
generators.&lt;/p&gt;
&lt;p&gt;For basic purposes we can use plain generators to compute values and to
pass those values around. The benefits of &lt;code&gt;yield from&lt;/code&gt; should become clear
when we know what it does and in which situations it can be used.&lt;/p&gt;
&lt;p&gt;Consider a generator that looks like this:&lt;/p&gt;
&lt;pre class="code literal-block"&gt;&lt;span class="k"&gt;def&lt;/span&gt; &lt;span class="nf"&gt;generator&lt;/span&gt;&lt;span class="p"&gt;():&lt;/span&gt;
    &lt;span class="k"&gt;for&lt;/span&gt; &lt;span class="n"&gt;i&lt;/span&gt; &lt;span class="ow"&gt;in&lt;/span&gt; &lt;span class="nb"&gt;range&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="mi"&gt;10&lt;/span&gt;&lt;span class="p"&gt;):&lt;/span&gt;
        &lt;span class="k"&gt;yield&lt;/span&gt; &lt;span class="n"&gt;i&lt;/span&gt;
    &lt;span class="k"&gt;for&lt;/span&gt; &lt;span class="n"&gt;j&lt;/span&gt; &lt;span class="ow"&gt;in&lt;/span&gt; &lt;span class="nb"&gt;range&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="mi"&gt;10&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="mi"&gt;20&lt;/span&gt;&lt;span class="p"&gt;):&lt;/span&gt;
        &lt;span class="k"&gt;yield&lt;/span&gt; &lt;span class="n"&gt;j&lt;/span&gt;
&lt;/pre&gt;


&lt;p&gt;As expected this generator yields the numbers 0 to 19. Let's say we wish
to split this generator into two generators so we reuse them elsewhere.&lt;/p&gt;
&lt;p&gt;We could rewrite the above into:&lt;/p&gt;
&lt;pre class="code literal-block"&gt;&lt;span class="k"&gt;def&lt;/span&gt; &lt;span class="nf"&gt;generator2&lt;/span&gt;&lt;span class="p"&gt;():&lt;/span&gt;
    &lt;span class="k"&gt;for&lt;/span&gt; &lt;span class="n"&gt;i&lt;/span&gt; &lt;span class="ow"&gt;in&lt;/span&gt; &lt;span class="nb"&gt;range&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="mi"&gt;10&lt;/span&gt;&lt;span class="p"&gt;):&lt;/span&gt;
        &lt;span class="k"&gt;yield&lt;/span&gt; &lt;span class="n"&gt;i&lt;/span&gt;

&lt;span class="k"&gt;def&lt;/span&gt; &lt;span class="nf"&gt;generator3&lt;/span&gt;&lt;span class="p"&gt;():&lt;/span&gt;
    &lt;span class="k"&gt;for&lt;/span&gt; &lt;span class="n"&gt;j&lt;/span&gt; &lt;span class="ow"&gt;in&lt;/span&gt; &lt;span class="nb"&gt;range&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="mi"&gt;10&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="mi"&gt;20&lt;/span&gt;&lt;span class="p"&gt;):&lt;/span&gt;
        &lt;span class="k"&gt;yield&lt;/span&gt; &lt;span class="n"&gt;j&lt;/span&gt;

&lt;span class="k"&gt;def&lt;/span&gt; &lt;span class="nf"&gt;generator&lt;/span&gt;&lt;span class="p"&gt;():&lt;/span&gt;
    &lt;span class="k"&gt;for&lt;/span&gt; &lt;span class="n"&gt;i&lt;/span&gt; &lt;span class="ow"&gt;in&lt;/span&gt; &lt;span class="n"&gt;generator2&lt;/span&gt;&lt;span class="p"&gt;():&lt;/span&gt;
        &lt;span class="k"&gt;yield&lt;/span&gt; &lt;span class="n"&gt;i&lt;/span&gt;
    &lt;span class="k"&gt;for&lt;/span&gt; &lt;span class="n"&gt;j&lt;/span&gt; &lt;span class="ow"&gt;in&lt;/span&gt; &lt;span class="n"&gt;generator3&lt;/span&gt;&lt;span class="p"&gt;():&lt;/span&gt;
        &lt;span class="k"&gt;yield&lt;/span&gt; &lt;span class="n"&gt;j&lt;/span&gt;
&lt;/pre&gt;


&lt;p&gt;This version of &lt;code&gt;generator()&lt;/code&gt; also yields the numbers 0 to 19. However, it
feels unnecessary to specify that we wish to iterate over both &lt;code&gt;generator2&lt;/code&gt; and
&lt;code&gt;generator3&lt;/code&gt; and yield their values. This is where &lt;code&gt;yield from&lt;/code&gt; comes in.
Using this new keyword we can rewrite &lt;code&gt;generator&lt;/code&gt; into:&lt;/p&gt;
&lt;pre class="code literal-block"&gt;&lt;span class="k"&gt;def&lt;/span&gt; &lt;span class="nf"&gt;generator&lt;/span&gt;&lt;span class="p"&gt;():&lt;/span&gt;
    &lt;span class="k"&gt;yield from&lt;/span&gt; &lt;span class="n"&gt;generator2&lt;/span&gt;&lt;span class="p"&gt;()&lt;/span&gt;
    &lt;span class="k"&gt;yield from&lt;/span&gt; &lt;span class="n"&gt;generator3&lt;/span&gt;&lt;span class="p"&gt;()&lt;/span&gt;
&lt;/pre&gt;


&lt;p&gt;This gives the same result and it is much cleaner to write and maintain. It is
also quite similar to the way functions are refactored and split up into
multiple functions. For example,  a large function can be split into several
smaller functions, &lt;code&gt;f1()&lt;/code&gt;, &lt;code&gt;f2()&lt;/code&gt; and &lt;code&gt;f3()&lt;/code&gt;, and the original function simply
calls &lt;code&gt;f1()&lt;/code&gt;, &lt;code&gt;f2()&lt;/code&gt; and &lt;code&gt;f3()&lt;/code&gt; in sequence.&lt;/p&gt;
&lt;h4&gt;Useful situations for 'yield from'&lt;/h4&gt;
&lt;p&gt;Those of you familiar with the &lt;code&gt;itertools&lt;/code&gt; module may note that the above example
is rather simple and does not truly justify introducing a new keyword in the language.&lt;/p&gt;
&lt;p&gt;Using the &lt;code&gt;chain&lt;/code&gt; function from the &lt;code&gt;itertools&lt;/code&gt; module we also could have written:&lt;/p&gt;
&lt;pre class="code literal-block"&gt;&lt;span class="kn"&gt;from&lt;/span&gt; &lt;span class="nn"&gt;itertools&lt;/span&gt; &lt;span class="kn"&gt;import&lt;/span&gt; &lt;span class="n"&gt;chain&lt;/span&gt;

&lt;span class="k"&gt;def&lt;/span&gt; &lt;span class="nf"&gt;generator&lt;/span&gt;&lt;span class="p"&gt;():&lt;/span&gt;
    &lt;span class="k"&gt;for&lt;/span&gt; &lt;span class="n"&gt;v&lt;/span&gt; &lt;span class="ow"&gt;in&lt;/span&gt; &lt;span class="n"&gt;chain&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;generator2&lt;/span&gt;&lt;span class="p"&gt;(),&lt;/span&gt; &lt;span class="n"&gt;generator3&lt;/span&gt;&lt;span class="p"&gt;()):&lt;/span&gt;
        &lt;span class="k"&gt;yield&lt;/span&gt; &lt;span class="n"&gt;v&lt;/span&gt;
&lt;/pre&gt;


&lt;p&gt;It can be argued that the &lt;code&gt;yield from&lt;/code&gt; syntax and semantics are slightly cleaner
than importing an additional function from a module but, leaving that aside,
we have not yet seen an example where &lt;code&gt;yield from&lt;/code&gt; has enabled us to do something
new. As we will see later in this tutorial, the main benefit of &lt;code&gt;yield from&lt;/code&gt; is
to allow easy refactoring of generators.&lt;/p&gt;
&lt;p&gt;It should be noted that it is not necessary for new programming language syntax
to also introduce new semantics (i.e, to express something that was not
possible before).&lt;/p&gt;
&lt;p&gt;Many languages introduce syntax, often called
syntactic sugar, to make it easier to write something that would otherwise be
cumbersome to write. For example, Haskell allows you to easily write a string as
&lt;code&gt;"example"&lt;/code&gt; which is shorthand syntax for &lt;code&gt;['e', 'x', 'a', 'm', 'p', 'l', 'e']&lt;/code&gt;
(a list of characters). Cleaner and more maintainable code can suffice to
introduce new syntax into a programming language.&lt;/p&gt;
&lt;h4&gt;Binary tree example&lt;/h4&gt;
&lt;p&gt;&lt;a href="http://www.python.org/dev/peps/pep-0380/"&gt;The proposal&lt;/a&gt; that introduces
the &lt;code&gt;yield from&lt;/code&gt; syntax provides a few examples to
demonstrate the new behaviour. One of them is a basic binary tree and we
can traverse the nodes of the tree using normal for loops as well as the
new &lt;code&gt;yield from&lt;/code&gt; syntax.&lt;/p&gt;
&lt;p&gt;In the next part of this tutorial we will build upon this example to explain
the benefits of &lt;code&gt;yield from&lt;/code&gt;.&lt;/p&gt;</description><category>python</category><category>software</category><guid>http://simeonvisser.com/posts/python-3-using-yield-from-in-generators-part-1.html</guid><pubDate>Sat, 26 Jan 2013 09:12:00 GMT</pubDate></item><item><title>Running Tests in Python with Selenium 2 and WebDriver</title><link>http://simeonvisser.com/posts/running-tests-in-python-with-selenium-2-and-webdriver.html</link><dc:creator>Simeon Visser</dc:creator><description>&lt;h4&gt;Why use Selenium?&lt;/h4&gt;
&lt;p&gt;If your website or web application is growing beyond a certain size, it is no longer feasible to
manually test everything. There are simply too many features and edge case scenarios
to keep clicking through manually. This is where automated testing software comes into view.&lt;/p&gt;
&lt;p&gt;Selenium is a set of tools to automatically test your web project in multiple
browsers. You can let it click on butttons, enter text in forms and browse your website to see
if everything is still behaving as it should.&lt;/p&gt;
&lt;h4&gt;The approach&lt;/h4&gt;
&lt;p&gt;In this tutorial we are going to use &lt;a href="http://seleniumhq.org/"&gt;Selenium WebDriver&lt;/a&gt; and Python to write our
tests.&lt;/p&gt;
&lt;p&gt;The basic idea is:&lt;/p&gt;
&lt;ul&gt;&lt;li&gt;Launch Selenium&lt;/li&gt;
&lt;li&gt;Write some unit tests in Python&lt;/li&gt;
&lt;li&gt;Execute the tests in one or browsers&lt;/li&gt;
&lt;/ul&gt;&lt;p&gt;We'll run the tests on Firefox but the approach below can be scaled to multiple browsers, running
both locally and in virtual machines.&lt;/p&gt;
&lt;p&gt;Before we continue, we'll need to discuss some Selenium terminology:&lt;/p&gt;
&lt;ul&gt;&lt;li&gt;&lt;strong&gt;webdriver&lt;/strong&gt;: a webdriver instance of Selenium that runs the tests for a particular browser configuration.
For example, you can run tests for Firefox or for a particular version of Internet Explorer.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;hub&lt;/strong&gt;: the hub is the Selenium server which handles communication to all the webdrivers.
Each of the webdrivers registers itself with the hub so the hub knows which browser
configurations are available for testing.&lt;/li&gt;
&lt;/ul&gt;&lt;h4&gt;Install Selenium&lt;/h4&gt;
&lt;p&gt;First of all, download the &lt;a href="http://code.google.com/p/selenium/downloads/list"&gt;stand-alone Selenium server&lt;/a&gt;.
You'll need to put this .jar file on all the computers or virtual machines that you're going to
use for automated testing.&lt;/p&gt;
&lt;p&gt;The stand-alone server can run as a hub or as a remote depending on the parameters that you pass
upon launching it. Obviously, you'll only need one hub and at least one webdriver to run your tests.&lt;/p&gt;
&lt;p&gt;For the Python test scripts, install the
&lt;a href="http://pypi.python.org/pypi/selenium"&gt;Selenium binding&lt;/a&gt; from PyPI. To check that
you've installed it correctly, run the python command and type:&lt;/p&gt;
&lt;pre&gt;import selenium&lt;/pre&gt;

&lt;p&gt;If you get no errors then you have installed it correctly.&lt;/p&gt;
&lt;h4&gt;Running Selenium&lt;/h4&gt;
&lt;p&gt;Starting a hub is pretty straightforward:&lt;/p&gt;
&lt;pre&gt;
java -jar selenium-server-standalone-2.18.0.jar -host &amp;lt;IP address&amp;gt; -port 4444 -role hub
&lt;/pre&gt;

&lt;p&gt;Starting a webdriver:&lt;/p&gt;
&lt;pre&gt;
java -jar selenium-server-standalone-2.18.0.jar -host &amp;lt;IP address&amp;gt; -port 5555 -role webdriver -hub http://&amp;lt;IP address of hub&amp;gt;:4444/grid/register -browser browserName=firefox,platform=MAC
&lt;/pre&gt;

&lt;p&gt;In the above commands the &lt;em&gt;IP address&lt;/em&gt; is where the hub or webdriver runs.&lt;/p&gt;
&lt;p&gt;You don't necessarily need to pass all the parameters (for example, Selenium uses port 4444 by default
for the hub) but I think it's better to be explicit.&lt;/p&gt;
&lt;p&gt;The &lt;strong&gt;-role parameter&lt;/strong&gt; specifies whether a hub or a webdriver is being launched.&lt;/p&gt;
&lt;p&gt;The &lt;strong&gt;-hub parameter&lt;/strong&gt; is needed for webdrivers and it specifies the registration URL to the hub.&lt;/p&gt;
&lt;p&gt;The &lt;strong&gt;-browser parameter&lt;/strong&gt; is used to specify the browser configuration. These details will be useful
later on to ask the hub if we can run the tests on a particular browser. This means we
don't need to know where the webdriver instances are running, we can simply ask the hub to run the
tests on, e.g., Firefox and it will handle the rest for us.&lt;/p&gt;
&lt;p&gt;You can visit the following URL to view the status of the hub and the registered webdrivers:&lt;/p&gt;
&lt;pre&gt;
http://&amp;lt;IP address of hub&amp;gt;:4444/grid/console
&lt;/pre&gt;

&lt;h4&gt;Writing Python tests&lt;/h4&gt;
&lt;p&gt;For the Python tests we use the &lt;a href="http://docs.python.org/library/unittest.html"&gt;unittest&lt;/a&gt; module
from the Python Standard Library.&lt;/p&gt;
&lt;pre&gt;
from selenium import webdriver
import unittest
import sys

class ExampleTestCase(unittest.TestCase):

    def setUp(self):
        self.driver = webdriver.Remote(desired_capabilities={
            "browserName": "firefox",
            "platform": "MAC",
        })

    def test_example(self):
        self.driver.get("http://www.google.com")
        self.assertEqual(self.driver.title, "Google")

    def tearDown(self):
        self.driver.quit()

if __name__ == "__main__":
    unittest.main()
&lt;/pre&gt;

&lt;h4&gt;Running Python tests&lt;/h4&gt;
&lt;p&gt;You can now simply run the tests with the following command:&lt;/p&gt;
&lt;pre&gt;python filename.py&lt;/pre&gt;

&lt;p&gt;where filename.py is the the file with the above Python code. It may take
a while for Selenium to open the browser window and execute your commands but
after a while it should print the test results in the console. Selenium loads
the Google homepage and checks whether the title is "Google".&lt;/p&gt;
&lt;h4&gt;Multiple browsers and multiple platforms&lt;/h4&gt;
&lt;p&gt;The above approach is suitable for testing your website in a particular
browser but the real benefit comes from testing it in multiple browsers on multiple
operating systems.&lt;/p&gt;
&lt;p&gt;The browser configuration is specified in the setUp method
of the test case. You could create some functionality (for example, by turning
the above script into a script with some command-line parameters) to run the test case
multiple times with different browser configurations.&lt;/p&gt;
&lt;p&gt;You could specify the capabilities as part of the test case itself:&lt;/p&gt;
&lt;pre&gt;
class ExampleTestCase(unittest.TestCase):
    capabilities = None
&lt;/pre&gt;

&lt;p&gt;which means you can instantiate the driver as follows: &lt;/p&gt;
&lt;pre&gt;
self.driver = webdriver.Remote(desired_capabilities=self.capabilities)
&lt;/pre&gt;

&lt;p&gt;Lastly, you need to pass the command-line parameters to the test case before
running it:&lt;/p&gt;
&lt;pre&gt;
if __name__ == "__main__":
    ExampleTestCase.capabilities = {
        "browserName": sys.argv[1],
        "platform": sys.argv[2],
    }
    unittest.main()
&lt;/pre&gt;

&lt;p&gt;You can now launch multiple Selenium drivers and run the same test code for
multiple browsers and platforms:&lt;/p&gt;
&lt;pre&gt;
python filename.py firefox MAC
python filename.py firefox LINUX
&lt;/pre&gt;</description><category>python</category><category>software</category><guid>http://simeonvisser.com/posts/running-tests-in-python-with-selenium-2-and-webdriver.html</guid><pubDate>Sat, 04 Feb 2012 11:00:00 GMT</pubDate></item></channel></rss>