Noel O'Blog

ANN: Symposium on Visual Analysis of Chemical Data (ACS Spring 2010)

2009-07-15T08:03:00.000-07:00

First Call for Papers:
Visual Analysis of Chemical Data
239th ACS National Meeting
San Francisco, March 21-25, 2010
CINF Division

Dear Colleagues,

We wish to announce an upcoming symposium focusing on innovative methods for visual representation and analysis of chemical data. Just as Edward Tufte has championed maximizing clarity and information content in statistical graphics, there is a need for methods to display chemical information that will maximize understanding, and allow rapid analysis and decision making.

We invite you to submit contributions that address various aspects of visualization of chemical data (such as structures, SAR data, literature, patents) including, but not limited to, the following topics:

With an ever increasing pool of descriptors, along with new and more sophisticated machine learning methods, QSAR models are becoming more difficult to interpret. How can information on model reliability, the presence of activity cliffs, and the range of applicability of a model and other relevant model properties be easily depicted?
Recently, virtual worlds 3D such as Second Life have presented new opportunities and challenges for the representation of chemical data. What is the potential of such a medium in education and communicating with the chemistry community?
Social software allows for rapid and convenient sharing of chemical data. Examples include Google Spreadsheets, ManyEyes, DabbleDB, and wikis, including Wikipedia. What are the implications for chemical research and education?
The visualization of the contents of large chemical datasets presents particular problems. How can an overview of the dataset be visualized so that it presents both the nature of the contents as well as the degree of diversity and similarity within the dataset? How can different datasets be visually compared?
Depicting 3D chemical information in 2D involves a loss of information. However, innovative 2D visualization methods can restore the most relevant information.
Chemical information comprises a diverse array of data types including chemical structures and diagrams (2D and 3D), associated assay results, conformations, QSAR models and their predictions. The visualization and integration of all these data into a single interface that aids interpretation and analysis is a continuing challenge.

We would also like to point out that sponsorship opportunities are available.

The on-line abstract submission system (OASYS) will be open for submissions from 24th August. A second announcement will be made at that time.

Please contact Andrew, Jean-Claude or myself if you have any questions.

Yours sincerely,
Noel O'Boyle

On behalf of the symposium organizers:

Dr. Jean-Claude Bradley,
Drexel University, PA
bradlejc@drexel.edu

Dr. Andrew Lang,
Oral Roberts University, OK
alang@oru.edu

Dr. Noel O’Boyle,
Cambridge Crystallographic Data Centre, U.K.
oboyle@ccdc.cam.ac.uk

Image credit: prehensile

Sledgehammer, meet nut - Using Eclipse for Python

2009-07-07T08:50:00.000-07:00

I usually use gvim or IDLE to edit Python files, but today I thought I'd try something a bit more heavyweight: Eclipse. Eclipse is widely used in the Java world. It's open source and freely available, and most importantly there is a Python plugin for Eclipse called PyDev.

So what does Eclipse have that IDLE doesn't? Well, integration with the Python debugger for a start. Also, this sort of code completion is quite handy (click for a larger image):
It also has nice integration with PyLint (see the bottom pane in the following figure) which catches various errors (e.g. mispelled variables) before you run a script:
Here are some notes:

I followed these installation instructions and then sped through the manual.
Pydev currently supports Eclipse 3.2 to 3.4. It took a while to find an Eclipse download page with version 3.4 but here it is. I installed Eclipse SDK 3.4.2.
Start Eclipse, and click on Help/Software Updates. Add http://pydev.sourceforge.net/updates/ to the list of update sites. Tick the box and click Install to install PyDev.
Following the details at http://www.fabioz.com/pydev/manual_101_interpreter.html, I added a Python interpreter (Name="Python 2.5", Executable="C:\Python25\python.exe").
Installing pylint on Windows is a pain, so I used easy install:
```
C:\Python25\Scripts\easy_install.exe pylint
```
In the PyLint configuration, you need to specify the location of lint.py. Mine was at C:\Python25\Lib\site-packages\pylint-0.18.0-py2.5.egg\pylint\lint.py.

I'll fix the bug...but only if you give me a public domain test file

2009-06-29T06:16:00.000-07:00

Recently, Avogadro/OpenBabel have been increasing their support for computational chemistry log files. I am hoping that they will learn from our experience at GaussSum/cclib.

GaussSum was the first Python program I ever wrote, and still bears the hallmarks. When I first started GaussSum (a program which analyses the results of comp chem calculations), I would use the test cases from users to fix bugs. Then over time, I'd lose the test cases as I moved from computer to computer. I couldn't place the test cases in my version control system as the test cases might have been the results of someone's research, and they mightn't be happy to see them publicly available.

Things came to head when dealing with the parsing of vibrational frequencies in the various versions of GAMESS. It turned out that each version of GAMESS (PC-GAMESS, WinGAMESS and GAMESS US) had slightly different output for vibrational frequencies. I ended up bouncing between code that worked for WinGAMESS but not GAMESS and vice versa, depending on who sent me the last bug report. In other words, I was wasting my time fixing bugs which might reappear later. It was around this time that (a) I realised I needed a test suite, and (b) I needed public domain test files, so I could use them in my test suite.

The parser used by GaussSum is now available as a separate project, cclib, and is developed in collaboration with Adam Tenderholt and Karol Langner. This time I put a lot of thought into the test suite, and I think we've done very well. The parsers are initially developed using a set of calculations which are the same for each comp chem package; our test suite ensures that the same results are found in each case and that the units are consistent. We only fix bugs for which a public domain test file is provided ("I place this file in the public domain" is all we need to hear), and regression tests are easily added to the test suite. Our test suite has the final say on commits; commits are reverted if they cause an existing test to fail. This guarantees that cclib can only improve over time.

The inevitable consequence of this policy is that some reported bugs don't get fixed. Sometimes the reporter simply does not respond to the query to place it in the public domain. On two occasions, the reporter was working in a pharmaceutical company and felt it was more hassle than it was worth to do the necessary paperwork to place it in the public domain. So it goes... On the other hand, we do now have a set of more than 200 comp chem log files which go a long way to ensuring that our parsers can handle anything that is thrown at them. The best way of getting these files is to check the data directory of cclib out of subversion and run wget.sh.

In conclusion, if you are thinking of writing software that handles comp chem files, either try to collaborate with others who are working on the same problem (e.g. cclib or OpenBabel), or at the very least take into account some of the comments here. Otherwise, you are simply building a house of cards.

Using PyActiveResource to access ChemCaster

2009-06-19T08:39:00.000-07:00

ChemCaster, from Rich's Metamolecular, is a platform for developing web-based cheminformatics applications. The advantage of such a system is that the user does not need to install any special software, nor does the application developer need to maintain a server.

Rich invited me to take it for a spin, so I signed up for a trial account and moved quickly on to my first problem, how do I access the API through Python?

It turns out that RESTful APIs tend to have common patterns, a fact which is taken advantage of by Active Resource, a Ruby library for defining classes which directly map onto the objects implied by a RESTful API. Or something like that - I neglected to read any documentation. Instead I just took Rich's example and tried to code it up in Python using PyActiveResource (this is a documentation-free project so using it is quite exciting).

Et voilá

From zero to Zotero - One man's journey out of PDF hell

2009-06-09T01:32:00.000-07:00

Zotero is a reference management software. Sorry, let me correct that - Zotero is THE reference management software. I had tried Zotero before, and it certainly looked good; but frankly I couldn't figure out how to get it to work and so reverted to my usual system, the 'zero' of the title. Hearing the news that Endnote vs. Zotero was just thrown out of court, I decided to try it again.

And it's just amazing.

Let me begin by describing a typical workflow:
(1) Go to the summary page for an ACS paper online
(2) Click on the icon that appears in the address bar (looks like a sheet of paper with writing).

That's it. You've just saved the PDF, the HTML full-text and the paper's metadata.

If you've created an account on zotero.org (free of course!), you can synch your library so that multiple computers can share the same data. And best of all you can also synch the attachments (i.e. PDFs, HTML pages) if you have a WebDAV account (e.g. from your university or in my case, JungleDisk Plus/Amazon S3). If that wasn't enough, it also integrates with Word to make it easy to prepare a publication (~~though I haven't tested this~~ Update: it works just fine, but you first need to install the bibliographic styles you need from Zotero settings/Preferences/Styles/Get additional styles).

In other words, Zotero makes it easy to download papers, back them up, make them accessible from any computer and reference them in papers.

Zotero is open source and freely available from www.zotero.org.

Notes: I'm using Zotero 2.0b5. In the Zotero preferences (click on the gear icon), choose "Automatically attach PDFs and other files when saving items" in the General Tab. JungleDisk and Amazon cost money (we're talking around $1.50 a month), but there may be free alternatives for WebDAV. For any websites that aren't currently supported by Zotero, adding new translators has been made easy. All of the JavaScript files for the translators are stored in a folder on your computer and can easily be extended or added to. That said, I've had no trouble downloading PDFs from Sciencedirect, ACS, RSC, Wiley or BMC.

Image credit: jazzmodeus

The best time to optimise

2009-06-05T03:09:00.000-07:00

As a scientist, I worry more about bugs in software than about speed. Changing correct code to improve speed can introduce errors as well as make it unreadable for others. Sometimes though it's nice to find cases where simple changes can improve the performance.

The 3D structure generation code in OpenBabel uses templates to handle the geometry of rings. There are about 2500 templates, which are represented by SMARTS patterns and associated coordinates (see fragments.txt in the distribution). The SMARTS patterns are ordered from large to small. Now, testing 2500 SMARTS patterns against a molecule takes a wee while so I was interested in seeing whether the process could be speeded up.

To begin with, I timed the code for a test set of 1000 PubChem molecules: it took 60ms per structure. Considering that the easiest way to speed something up is to avoid doing it in the first place, I changed the loop to terminate once all ring atoms had been matched. This brought it down to 38ms per structure. Then I changed it so that it skipped any SMARTS patterns that had more atoms than the number of ring atoms in the molecule: now down to 30ms. This is now within an order of magnitude of greased lightning.

In fact, I could have done slightly better than this; I could have skipped any SMARTS patterns with more atoms than the number of atoms in the largest isolated ring system in the molecule. Calculating this value is a bit of work though and may offset the associated performance gain, and so this has been left as an exercise for the reader.

How else could this code be speeded up? Well, the SMARTS matcher can itself be improved. It currently uses an exhaustive depth-first search algorithm instead of something more optimal like Vflib2. This would improve performance across the board as the SMARTS code is widely used for a variety of tasks. Alternatively, the SMARTS patterns could be fingerprinted based on particular common patterns, e.g. 5-membered rings. If a molecule had no 5-membered rings, such patterns could be skipped.

To begin with, though, the code should be profiled more precisely. It may be that 25 of those 30ms have nothing to do with this loop. In that case, further optimisations may be more work than they are worth.

These are the sorts of small studies that would fit nicely into a summer project for an undergrad computer science or chemistry student. If you want to sponsor OpenBabel development in this way, contact us.

Image credit: jpctalbot

The RSC - Value for money?

2009-05-27T09:13:00.000-07:00

I don't usually advertise for chemical societies, but in these recessionary times I thought the following might be of interest to some readers.

RSC members have:

free access to Wiley, Elsevier, and Springer chemistry journals
free access to 913 chemistry e-books from a variety of sources
20% off Pearson Education Books, 30% off Wiley, 35% off Blackwell
and most importantly, £5 off Pizza Express Club membership

Sure, chemistry societies organise conferences, enable networking, provide travel grants, and lobby politicians; but any society that doesn't look after its most vulnerable members by providing discounted pizza is not a society I want to be a member of.

Have your hamburger and eat it - Edit molecules in PDFs II

2009-05-21T08:22:00.000-07:00

In Part I, I showed how to hack some code together that allowed you to paste images directly from the clipboard (e.g. from a PDF) into Beda's BKChem, a 2D drawing program. The magic conversion from image to chemical was done by Igor's OSRA.

Well, Igor has taken this idea and run with it. The latest version of OSRA now includes plugins for BKChem, Symyx Draw, MolSketch and Pipeline Pilot.

If you use the Windows installer, the Symyx Draw plugin is automatically installed and adds an "Import Structures from OSRA" option to the File menu. The first time you choose it, you will need to change the path to something like "C:\Program Files\osra\osra.exe" under "Settings...". Here's the plugin in action:
Note that the other plugins appear to be only available from the Windows .zip release.

How do enzyme mechanisms evolve?

2009-05-16T03:26:00.000-07:00

Evolution is a fascinating topic. Although the principal mechanism by which evolution occurs is quite simple to understand, namely the introduction of changes (mutations) into the DNA, the consequences that follow are enormous.

The term selective pressure is used to describe an imaginary operator that affects the incidence of particular mutations in a population. What makes evolution difficult for me to get my head around is that selection operates on many levels. In a population, a particular physical characteristic might be more advantageous (think of the famous finches) or more attractive. In your DNA, a particular mutation might preserve the amino acid coded for, or it may change to another amino acid that does not affect the protein's function. On the other hand, if the amino acid is involved in the catalytic action of the protein it's going to be conserved, right? But then how do new mechanisms evolve?

My former postdoc supervisor, Dr. John Mitchell, is currently advertising a PhD position on "Modelling the Evolution of Enzyme Catalysis" at the University of St. Andrews. I'm particularly interested in this project as it builds on earlier work I carried out in the Mitchell Group along with Gemma Holliday and Daniel Almonacid. Here's an excerpt from the project description:

We will create a simulation using a population of model enzyme-catalysed reactions, mimicking a state early in evolutionary history, and allow them to evolve in EC space. The reactions will consist of steps and be represented, in a manner familiar from genetic algorithms, by "chromosomes" describing the chemical properties of each step. Parameters will control the likelihood of different kinds of evolutionary event, such as a change of substrate with the same underlying chemical mechanism, taking place. The simulations will be calibrated, and then compared with the results from a study of real-world convergent and divergent evolution.

Cool. Closing date 31 July.

Image credit: Colin Purrington

Manipulating PDFs with Open Source Tools - Part II

2009-05-05T09:12:00.000-07:00

Part I

There are a couple of journals that have rather generous margins around the text. I prefer to print out two pages per sheet to avoid wasting paper, so it would be nice to be able to remove those margins and increase the size of the text instead.

pdfcrop by Heiko Oberdiek allows you to do just that. It's included with Debian Linux, and "sudo apt-get install texlive-pdfetex" will install it. On other (lesser) Linux distributions you may have to check CPAN or CTAN. Once installed, a straightforward "pdfcrop withmargin.pdf nomargin.pdf" will do the conversion.

Update: A reader points out that Tex Live 2008 includes pdfcrop and is available for both Windows and Linux.

Note that pdfcrop should not be confused with PDFCrop. There's also a patch of pdfcrop called pdfcrop2, but I think that covers the current crop.

And spare a thought for Fermat.

Image credit: B.G. Johnson

ChemPad - Protecting you from chemical structure software

2009-05-05T08:33:00.001-07:00

A colleague has just pointed me to ChemPad, an interesting piece of software that allows chemists to draw chemical diagrams directly into a computer on a Tablet PC. It was designed as a tool to enable undergraduate chemists to quickly enter chemical structures and generate 3D diagrams with which they could gain a deeper understanding of structures. Hash and wedge bonds are understood and are converted faithfully to 3D, for example.

From the point of view of teaching chemistry undergraduates, such software may be preferable to ChemDraw and friends as it allows them to develop the skill of drawing diagrams by hand. Of course, not everyone has a tablet PC but one could imagine similar software for the iPhone or just a regular PC driven by a mouse.

ChemPad is free but not open source, and Windows only. The ChemPad website has tutorials and videos of the software in use.

Broken symmetry - Can SMILES and InChI ensure canonicalisation?

2009-04-24T02:00:00.000-07:00

I've been working on the SMILES code in OpenBabel over the last while. The longer I've spent on it, the more impressed I've been with how it has been handled in the code and also with what a great idea SMILES was in the first place. The same goes for InChI, which has a slightly different goal, but which goes the extra mile and solves normalisation problems which I didn't even know existed.

But do they work? Can their canonicalisation procedures ensure that two identical molecular graphs result in the same canonical SMILES or InChI?

The InChI canonicalisation procedure is summarised in Rich's post. The Daylight algorithm is in Weininger*3, JCICS, 1989, 29, 97. And the review of the field that throws both into question is Ivanciuc's review of Processing Constitutional Information in Gasteiger's Handbook of Cheminformatics.

The key question here is whether the SMILES and InChI algorithms are capable of identifying automorphisms. There is a brute force way to do this, but both SMILES and InChI try to avoid this by identifying symmetry classes using extended connectivity and various graph invariants. An explicit automorphism check is not described as part of either algorithm but yet Ivanciuc argues repeatedly (e.g. at the end of 5.1.4) that any canonicalisation algorithm that does not include an explicit automorphism check "is incomplete, and its use in a chemical database...is unreliable".

The funny thing is that although the SMILES paper came out several years prior to Gasteiger's handbook (1993 vs. 2003), it is not referenced. Furthermore, the InChI developers have followed the same route more recently.

I leave the following question as an exercise for the reader: if a counterexample to the SMILES or InChI algorithms existed, how would one find it?

Image credit: _Blaster_

Are you on my side or not? It's E/Z

2009-04-14T01:18:00.000-07:00

Handling cis/trans stereochemistry with SMILES should be easy, right? You have the canonical examples for trans:

A. I/C=C/Cl
 (I is down, Cl is up)
B. I\C=C\Cl
 (I is up, Cl is down)

and cis:

C. I/C=C\Cl (both are down)
D. I\C=C/Cl (both are up)

The "/" or "\" symbols should be chosen based on whether the substituent occurs before or after the atom attached to the double-bond. Bearing this in mind, the following represents the same trans structure as A:

E. C(=C/Cl)\I

Note that the effect of moving the "I" from one side of the "C" to the other (that is, A vs E) causes the bond symbol to change.

When ring closures occur on the double bond, a further complication arises as the stereobond appears twice, once at each end of the ring closure. The symbol indicating the stereochemistry should only appear at the end on the double bond:

F. I/C=C\1/CCCN1

Of course, where two substituents are shown explicitly at one end of a double bond, it's not necessary to show the stereochemisty for both of the bonds (although it makes things clearer). That is, the following two representations are identical to F:

G. I/C=C1/CCCN1
H. I/C=C\1CCCN1

Image credit: suttonhoo

Some short stories

2009-04-03T06:02:00.000-07:00

I want to flag up Andrew Dalke's course at the end of April on Python and cheminformatics. While I might disagree with Andrew's toolkit of choice, there's no doubt that the skills learnt will be of great benefit to any cheminformatician in their day-to-day work. As well as a cheminformatics portion, the course includes matplotlib (plotting), communicating with Excel, XML processing, subprocess (for calling command-line programs), NumPy, R, SQL and Django.
The first issue of Journal of Cheminformatics has hit the electronic shelves. Point your RSS reader to the feed. Best of luck to Christoph and David.
Is 2009 the year of OChRe on the desktop? After almost a decade of little development in this area, we have in quick succession papers on ChemReader, OSRA and now Clide Pro, an update of the venerable Clide. The techniques used by the new version are described in detail in the paper. Unfortunately, there is little in the way of comparison either to the original Clide or other OChRe software. On the plus side, the dataset of images discussed in the paper has been made available as supporting material with the intention of forming part of a community benchmark for performance comparisons (although it's not clear whether this dataset was also used for training the software).
There seems to be some confusion over the name of this field. Is it OSR (Optical Stucture Recognition, according to OSRA), OCR (Optical Chemical Recognition, ala chemOCR), OCSR (you guessed it, Optical Chemical Structure Recognition, as referred to in the Clide Pro paper), or OChRe (Optical Chemical Recognition again, but spelling out a real word; it also has that InChI up-and-down thing going on)?
Did your experiments fail again? Tell me about it. I mean that literally, because you've got your choice of journals to publish in. There's the All Results Journal ("all results are good results") or (for the more mathematically inclined) Rejecta Mathematica.

The Clockwisdom of SMILES Part II

2009-03-24T02:08:00.000-07:00

As many readers of this blog will be aware, a chiral SMILES is not a lopsided grin. Instead it is a way of describing the relative spatial arrangement of groups around a chiral centre using SMILES notation.

The following examples investigate this notation. The stereotypical examples are the following:

A. C[C@](Br)(Cl)I - ACW(C,Br,Cl,I)
B. C[C@H](Br)I    - ACW(C,H,Br,I)

where ACW(w,x,y,z) indicates anticlockwise in terms of x,y,z when looking from w. However, note that the first group does not necessarily need to appear before the chiral carbon:

C. [C@](C)(Br)(Cl)I - ACW(C,Br,Cl,I) (Same as A)
D. [C@H](C)(Br)I    - ACW(H,C,Br,I)  (Opposite of B)

What about ring closures? These are handled as followed:

E. C[C@H]1CCN1 -
   ACW(C,H,N,C) (the 1 indicates a bond to the chiral C)
F. C[C@]12N(CC2)C1 -
   ACW(C, the C1 carbon, the C2 carbon, N)
G. C[C@@]21N(CC2)C1 -
   CW(C, the C2 carbon, the C1 carbon, N) (the same as F)

Note that ring closures directly before an atom do not indicate a bond to that atom. Try to draw the following and compare your result to that given by Daylight's Depict service:

H. [C@@]123[C@H](C(C=C3)(C)C)CC[C@@](C1)(CCC2)C

If you got that right, consider yourself a SMILES ninja.

Credit: Thanks to Craig James for his patient explanations.
Image credit: sean-b

Time for a test - Any questions?

2009-03-20T15:01:00.000-07:00

I'm a great believer in tests for code quality. In fact, I don't want to contribute code to a project if I can't add a test to the test suite. This is particularly important in collaborative projects where changes by others might impact on bugs I've fixed or features I've added. I've learned my lesson in the past. With a test suite, I can be sure that everything is still working the way I expect it.

I've recently started a new test suite for OpenBabel called obunittest. Although OpenBabel already has a test suite ("make test"), I wanted to have a test suite written in Python where people could easily add new tests.

obunittest is hosted at github and all of the necessary instructions are available at the obunittest website (just scroll down). Git itself isn't required, but you may find it interesting to use - to do so, just create an account on github and fork my project.

So this is your chance to add a test for OpenBabel. Now while this might not be everyone's idea of a fun time, if there's some feature of OpenBabel upon which you rely, write a test for it and send it to me (or "git it" to me). This will ensure that that particular feature will always work in future OpenBabel releases. The same goes if there's something that you know is currently broken - just write a test. Remember that a stitch in time means you won't be saying "darn".

Image credit: Duncan Hull (hi!)

The Clockwisdom of SMILES

2009-03-18T02:36:00.000-07:00

I was recently confronted with a question that many of us face at some point in our lives: how many ways can the groups attached to a chiral C be moved around in a SMILES string while retaining the clockwisdom?

What's all this about clockwisdom? Well, a chiral SMILES string can indicate R or S around a tetrahedral centre using C@ or C@@. The difference is that R or S refer to clockwisdom of groups arranged by CIP priority (with the lowest priority facing away), whereas @ and @@ refer to clockwisdom of groups arranged in order of their appearance in the SMILES string (with the first appearing facing towards) [1]. Whether this was a good design decision by the Daylight gurus, I'm not 100% sure, but that's how it is.

So in short, if you change the order of groups in the SMILES string, you may need to change the clockwisdom to ensure that stereochemistry is preserved. Specifically, if you swap two groups you will get the other enantiomer ("putting the SMILES on the other face"?) unless you flip the clockwisdom; that is, Cl[C@@](Br)(C)I is the same enantiomer as Cl[C@](Br)(I)C. Another swap and we get back a SMILES string with the original clockwisdom.

So I started off by trying to think of a clever program to identify how many swaps were required to convert between two orderings of groups. Next I tried to write a few loops that would simply perform all possible swaps of groups to generate all of rearrangments, but that missed a few. In the end, I just wrote the dumbest program I could think of and got the following results. For an original ordering of groups 0123, the following orderings have the same clockwisdom: 1032, 3021, 2013, 3210, 1320, 3102, 0123, 0231, 0312, 2301, 1203, 2130.

And the point of all this? OpenBabel was not generating the correct stereochemistry around tetrahedral carbons in canonical SMILES. Now fixed.

Update (19/03/09): Tim Vandermeersch pointed out to me a neat way of determining the parity of a particular ordering of groups. Simply count the number of pairs in the ordering where one number is larger than another number to its right. For example, for 1032, there are two pairs (10, 32); for 3021, there are 3 pairs (32, 31, 21). Orderings with even numbers of pairs have one parity while orderings with odd number of pairs have the opposite parity.

[1] The OpenSMILES specification on stereochemistry

Image credit: Swamibu

Review of Hello World - Computer Programming for Kids and Other Beginners

2009-03-02T06:15:00.000-08:00

My generation were among the first children to learn programming. Thanks to the BBC Microcomputer (in my part of the world), kids were provided with a computer and a manual that taught computer programming in both BASIC and Logo. The local library had a complete set of Usborne books that covered everything from arcade games, to fantasy adventures (the old-school text only type, that is), to assembly language programming and sorting algorithms. And these were for children.

So what was it about programming on the BBC (or ZX Spectrum or Commodore 64) that drew kids in? For me it was all about graphics. Drawing circles could only be done dot by dot and led easily to drawing ellipses, and then to hyperboloids of revolution (think cooling towers) whose top you could twist. I read Chaos by James Gleick, couldn't believe the simplicity of generating the Mandelbrot fractal, and lifted my jaw off the floor the first time my BBC drew the little Mandelbeetle (I'm not the only one - see also PMR). I did some astronomy at school and plotted the night sky for different months of the year. And so on.

Since then we've seen the rise of the PC and Windows, which in fairness had QBasic for quite some time (I am a Nibbles master). However, as David Brin pointed out ("Why Johnny can't code", 2006) today there's no easy way for kids to get hooked on programming. Even my favourite language, Python, is lacking here. Out of the box the only usable graphics library for kids is the turtle module, an implementation of LOGO:

C:\Documents and Settings\oboyle> python
Python 2.6.1 (r261:67517, Dec  4 2008, ...
Type "help", "copyright", "credits" ...
>>> from turtle import *
>>> for i in range(10):
...     for j in range(5):
...             forward(100)
...             left(360/5)
...     left(360/10)
...
>>>

It seems to work quite well, although the documentation is aimed at computer science majors rather than teachers (never mind kids). Also, the demo files are only available in the source distribution (you can get them from SVN here).

While Logo might be quite good for introducing the basics of programming languages, its graphics capabilities are limited. pygame is really the way to go. This is one of the big third-party Python extensions that incorporates support for sound, graphics and input devices. As the name implies it has everything necessary to write a decent computer game (see for example, the list of pygame arcade games). The downside is that this library makes no effort to cater for kids.

Enter a recent publication from Manning, "Hello World! - Computer programming for kids and other beginners" by Warren and Carter Sande. Written with 12 year old kids in mind, the preface makes it clear that the authors (one a 12-year old kid himself) know their target audience well:

"For kids especially, one of the most fun parts of using a computer is playing games, with graphics and sound. We’re going to learn how to make our own games and do lots of things with graphics and sound as we go along. Here are pictures of some of the programs we’ll be making:"

(Figure published with permission from Manning Publications)

Lunar lander! Slalom racing! The Sandes have reinvented the Usborne books for the YouTellyTub generation, and then some. Assuming no previous programming knowledge (a reasonable assumption when you're 12), the book teaches Python programming with the goal of writing computer games. The initial chapters cover the basics from variables, through maths, "if" statements and loops. But there's also already the fun stuff like getting input and simple graphical dialogs, and in case attention is waning Chapter 10 (of 24) has the complete listing for a Skiing game. As the book says:"One of the great traditions of learning to program is typing in code you don’t understand. Really!"

After introducing lists, functions, objects and modules, pygame enters the picture in Chapter 16 which covers drawing, images and animation. The following chapters cover sprites and collision detection, events and sound. The final chapters return to useful Python modules such as handling strings, file input and output, and using random numbers. All of the code examples are available for download from the book's website, along with a simple installer that contains all of the examples and modules required, along with Python itself.

As you might have guessed, I think this is a great book that fills a real niche - I don't know of any other programming book on the market that targets kids. What's amazing is that it has set its sights so high, and yet manages to meet its goals. I think it would be great to see this book promoted as a way of teaching programming in primary schools. In the meanwhile if you know any 12+ kids interested in computers, give them an opportunity to develop a fascinating hobby and get them this book.

Repeat after me - for loop in Python

2009-02-20T09:04:00.000-08:00

for loops allow you to repeat the same set of instructions multiple times with a different value of a particular variable.

How do I loop in Python?

The Python for loop has the following general form:

for myvar in myiterator:
   # do something with myvar

How do I loop over a list?

If you coming from other languages such as C, you might be used to:

for i in range(len(mylist)):
   myvar = mylist[i]
   # do something with myvar

In Python, it's much better (that is, the intention is clearer) if you do the following instead:

for myvar in mylist:
   # do something with myvar

If you suddenly realise you want the value of the index i, you can easily adapt the previous loop:

for i, myvar in enumerate(mylist):
   # do something with myvar
   # do something with i

How do I loop over two lists simultaneously?

Let's say you have a list of people's firstnames and another list containing their surnames, and you want to write them on the screen. Don't do the following:

for i in range(len(firstnames)):
    print "%s %s" % (firstnames[i], secondnames[i])

Instead, you should make the intention of the code clear by using zip as follows:

for firstname, secondname in zip(firstnames, secondnames):
    print "%s %s" % (firstname, secondname)

Loop pattern #1 - Building a new list from an old

newlist = []
for mynum in mylist:
   newnum = mynum*2
   newlist.append(newnum)

In the case of this simple example, it should be done as a list comprehension instead:

newlist = [mynum*2 for mynum in mylist]

Here is a slightly more complicated example, involving an "if" statement that acts as a filter for even numbers:

newlist = []
for mynum in mylist:
   if mynum % 2 == 0:
       newnum = mynum*2
       newlist.append(newnum)

Again, this can be done instead as a list comprehension:

newlist = [mynum*2 for mynum in mylist if mynum % 2 == 0]

Loop pattern #2 - Summing things up

Another common pattern is totalling things up using a list:

total = 0
for mynum in mylist:
   total += mynum

Although in this case, it would be easier to just use:

total = sum(mylist)

One thing to be careful of though is that using "+" to add strings is slow. If speed is important, avoid the following:

longstring = ""
for myvar in mylist:
   # Create smallstring here somehow
   longstring += smallstring

Instead use pattern #1 to create a list of strings, and join them at the end:

stringlist = []
for myvar in mylist:
   # Create smallstring here somehow
   stringlist.append(smallstring)
longstring = "".join(stringlist)

Loop pattern #3 - Filling in a dictionary

A common pattern is to update information in a dictionary using a loop. Let's take an example of counting up how many people have a particular firstname given a list of firstnames.

The problem here is that it is necessary to check whether a particular key is in the dictionary before updating it. This can leads to awkward code like the following:

name_freq = {}
for firstname in firstnames:
   if firstname in name_freq:
       name_freq[firstname] += 1
   else:
       name_freq[fistname] = 1

Now while this can be improved somewhat by using name_freq.get(firstname, 0), that's getting a bit complicated (and doesn't extend to dictionaries of lists). Instead you should use a defaultdict, a special dictionary that has a default value, as follows:

from collections import defaultdict
name_freq = defaultdict(int)
for firstname in firstnames:
    name_freq[firstname] += 1

And what about where you wanted to store the corresponding surnames in a dictionary by firstnames? Use a defaultdict(list), of course:

from collections import defaultdict
same_firstname = defaultdict(list)
for firstname, surname in zip(firstnames, surnames):
    same_firstname[firstname].append(surname)

Loop pattern #4 - Don't use a loop

Think dictionary, set, and sort. A lot of tricky algorithms can be implemented in a few lines with one or two of these guys.

A trivial example is finding unique items in a list: set(mylist). Want to check whether that genome sequence only contains 4 letters?: assert len(set(mygenome))==4

The following example uses sort. Given a set of docking scores for 10 poses, find which poses have the top three scores. Here's a solution using the so-called decorate-sort-undecorate paradigm:

# Decorate ("stick your data in a tuple with other stuff")
tmp = [(x,i) for i,x in enumerate(pose_scores)]
# Sort (uses the items in the first position of the tuple)
tmp.sort(reverse=True)
# Undecorate ("get your data back out of the tuple")
top_poses = [z[1] for z in tmp]

print top_poses[:3]

Image credit: PanCa SatRio

Extract of chemical - Paper on OSRA published

2009-02-18T23:55:00.001-08:00

It seems like only last week that another paper was published on an Optical Chemical Recognition software. This week it's the turn of OSRA, the work of Igor Filippov, which has previously been discussed in several posts on this blog:

Igor V. Filippov and Marc C. Nicklaus. Optical Structure Recognition Software To Recover Chemical Information: OSRA, An Open Source Solution. J. Chem. Inf. Model. 2009, In press.

Unlike all existing software, OSRA is open source. You can get it right now from its web site.

One interesting point raised in the paper is how best to assess performance. It appears that a Tanimoto type fingerprint is highly penalised if even a single error is made. This is because an error such as replacement of an O with a C, although visually minor and easily corrected in a drawing package, is highly significant chemically. Perhaps a simple count of (bonds in common)/(union of bonds) would suffice although this would require mapping of bonds from the original structure to the recognised structure (the new SMSD tool by Syed Asad Rahman from Janet Thornton's group might help here).

New software for OChRe - ChemReader

2009-02-06T03:37:00.000-08:00

The good news is that the field of Optical Chemical Recognition (OChRe) has just received a new entry, ChemReader:

J Park, GR Rosania, KA Shedden, M Nguyen, N Lyu, K Saitou. Automated extraction of chemical structure information from digital raster images. Chem. Cent. J. 2009, 3, 4.

According to the test set used (and it's not stated whether the test set was also used to train the software) the results are very good, and not just for ChemReader - OSRA is doing pretty well also:
And the bad news? Rajarshi recently discussed the phenomenon of publishing papers describing software which can neither be purchased nor obtained for free. This is also the case for ChemReader. As the software is not open source, it cannot be made available as it relies on the open source libraries GOCR and Greystoration (also used by OSRA). However, the authors are "planning that ChemReader becomes commercially available in the near future, with removal of open source parts". Doesn't that mean that the final performance of the software, as well as the algorithms used, will be different from that described in the paper?

Of OChRe, OSRA and OASA (but not OSCAR) Part II

2009-02-03T01:07:00.000-08:00

In an earlier post I showed a way of testing the performance of an optical chemical recognition (OChRe) software, OSRA.

Since then, a new version of OSRA has been released (1.1.0). In addition, a new version of OASA has been released which can handle multiple molecules (this caused problems with the earlier test where a disconnected molecule was produced).

The results are shown here.

Notes:
(1) I used Xe to indicate an unknown atom type (only occurs once).
(2) If OSRA detects multiple molecules in the original image, a multi-mol SDF is created where the coordinates do not correspond to the original location in the image. Where this occured here, I have just depicted the first molecule.

TwistyMol is dead - Long live TwirlyMol

2009-01-30T09:21:00.000-08:00

My first attempt at a Javascript molecular viewer culminated in TwistyMol. For TwistyMol, I took as my starting point processing.js and consequently ran into some difficulties on IE, which doesn't support Canvas.

TwirlyMol is my attempt to start over with a browser-independent Javascript vector graphics library. I googled a bit and came across some pointers on Stack Overflow (a Q&A website that does things in a new way). I choose to go ahead with dojox.gfx, which is an experimental component of the Dojo javascript library. On IE, it uses VML, while on Firefox, it uses SVG (Canvas is also supported). I reckoned that this might improve performance on IE. Also, Dojo is likely to be better supported going forward, unlike processing.js which is essentially a one-man job (sure, the one man is the author of jQuery, but still) and its goal is quite different.

So, here's the demo.

It's ready to use for whatever you want. If you think about it, the code behind TwirlyMol can easily be adapted to displaying other types of 3D data, for example principal components graphs. What else would be useful? That's up to your imagination (comments below welcome).

And I think this draws to a close my involvement in Javascript molecular viewers. Hope y'all had fun. I've put the code up on Github just to see what happens. Maybe someone wants to add perspective.

Notes:
(1) Colours of atoms taken from the Blue Obelisk Data Repository. Even if you don't understand what the Blue Obelisk is about, I'm sure you can appreciate the benefit of shared chemical resources like these data.
(2) IE performance is still much slower than Firefox, but is better than with Twistymol.
(3) With Firefox, adding a large number of SVG molecules to a page is much slower than with Canvas. If doing this in practice, you'll need to add them in chunks of 50 or so, with a setTimeout() to add the next 50.
(4) Coding Javascript for IE really is frustrating.
(5) If you use mouse gestures, rotating or zooming with the right mouse button can trigger a gesture event. One way around this would be to implement support for modifier keys so that shift+left button causes right button behaviour, while alt+left would cause middle button behaviour. This would be quite easy as the dojox.gjx library has a function for testing whether shift is held down (I can't remember about the alt).

Turn, turn, turn - TwistyMol ready for action

2009-01-22T11:45:00.000-08:00

A new version of TwistyMol is available. It's now ready for use by others (that means YOU). Here's a simple example and here's another with a few more molecules.

What's changed since last time? Well I've added in an SDF file parser, made it possible to have multiple TwistyMols on the same page, sorted out the rotation (at least for the moment) and gone a bit overboard on the shadows. Performance on IE is still a big problem, and displaying more than a handful of molecules at the same time is out of the question (too slow). I still don't know if this is an intrinsic problem with VML or just the whole Canvas to Excanvas to VML conversion.

If you want to use it yourself, download the Javascript files referenced in the HEAD section of the demo page, and look at the code on the demo page itself. The only tricky part is getting the SDF file into Javascript - you'll either have to place it on a webserver and use an Ajax call or do as I did and convert it to a Javascript string variable (hint: read() it in with Python and convert \n to \\n).

Feel free to modify and distribute the code. You could add a comment below with a link if you use it anywhere.

Molecular viewer now works on Internet Explorer

2009-01-13T03:24:00.000-08:00

I've managed to get my mimimal molecular viewer (see previous post) working on Internet Explorer. Smooth, it is not, but at least it works (I recommend Firefox instead). I've also renamed it to TwistyMol. Minimol sounded a bit boring, like Microsoft. You can try it out here.

So what was the problem with Internet Explorer? Well, for graphics, TwistyMol uses processing.js which draws the molecule using the Canvas tag. Unfortunately, Internet Explorer is the only browser that doesn't support Canvas; instead, it has invented its own system called VML (Vector Markup Language). So, for IE users, I need to import the ExplorerCanvas Javascript library developed by Google; this converts all calls to Canvas to their equivalent in VML. (I also needed to make a few changes to processing.js to enable IE support.)

Next steps - sort out the rotation (Simon suggested a possible solution), turn the Javascript into proper Javascript (encapsulate it a bit), stick in a file format parser (I'm hoping Rich will come through on this one), and deploy to the masses.