Code Penguin

Slipix Version 1.0 for iOS

David Lambert — Fri, 02 Sep 2011 02:21:43 +0000

Introduction

Slipix is a unique new puzzle game where you touch and drag different layers of an image until the real picture is revealed. These are not jigsaw puzzles. Each layer is a different design or pattern that create unique challenges.

There are 40 unique levels to master with four difficulty levels. Quick Play mode can generate an endless amount of levels to satisfy any puzzle solver. There is even a head to head multiplayer mode (iPad only) so you can play with your friends.

More levels are on the way!

Click here to check it out in the App Store!

Video

Screen Shots

Support

Have a comment, need help or have suggestions for new categories and events? I’d love to hear from you. Use the support/contact form or leave a comment in the comments section and I’ll get back to you as soon as I can. Thanks!

Portable Google App Engine For Windows (Enhanced)

David Lambert — Sun, 09 Jan 2011 18:46:32 +0000

5/12/2011: Updated to support Google App Engine Version 1.5.0.

A while back I put together the information and steps to get Google App Engine running on Windows wrapped up in a portable little bundle. Since it has been a while I thought it could use a good refresh with the latest versions of everything and make things work more like the real Google App Engine. Some of the content may seem the same as last time but there definitely will be some new ways of running things and new capabilities that were not available last time. This new way is much easier and should keep up with all the changes that newer versions of Google App Engine may bring.

Installation

Let’s gather all the files that we are going to need. I throw the version numbers on here that this was written for but yours may vary slightly. Download the following:

Portable Python (Download the 2.5.x release. I am using 2.5.4 at the moment.)
Google App Engine SDK (Download the Linux/Other Platforms release. I am using 1.4.1-2011-01-05 at the moment.)

Now that you’ve got all the files, you can grab a USB thumb drive or even just install it to a directory on your local machine. Run the Portable Python installer and specify the location you’d like to store it. I will refer to this location as the “Portable Python folder” from here on.

Extract the “google_appengine” folder from the Google App Engine SDK zip file into the Portable Python folder on your thumb drive. Side Note: In the future if you ever want to update to a newer release, just delete the “google_appengine” folder and extract a new copy from the latest zip file from Google.

Create a new folder in your Portable Python folder called “google_appengine_datastores”. This is where we are going to store all the data for our apps.

Create two new files called “dev_appserver.py.bat” and “appcfg.py.bat” in your Portable Python folder both with the following contents:

@echo off
REM Portable Google App Engine Command Line Rewriter
REM Written by David Lambert (http://www.codepenguin.com)
 
REM Execute the command line rewriting script
app\python.exe _cmdrewrite.py %0 %*
REM Execute the new command line batch file
_cmdrewrite.bat

Create a new file called “_cmdrewrite.py” in your Portable Python folder with the following contents:

#Portable Google App Engine Command Line Rewriter
#Written by David Lambert (http://www.codepenguin.com)
 
#Rewrite the command line with correct paths
import sys, os, re
new_cmd = 'app\python.exe google_appengine/%s' % sys.argv[1]
gae_version = re.search('release: "(.*?)"',open('google_appengine/VERSION','r').read()).group(1)
appname = sys.argv[-1].rstrip('/').split('/')[-1]
if sys.argv[1] == 'dev_appserver.py' and os.path.exists(appname):
  new_cmd += ' --blobstore_path=google_appengine_datastores/%s_blobstore' % (appname)
  new_cmd += ' --datastore_path=google_appengine_datastores/%s_datastore' % (appname)
  new_cmd += ' --history_path=google_appengine_datastores/%s_history' % (appname)
  if gae_version >= "1.5.0":
    new_cmd += ' --skip_sdk_update_check'
    new_cmd += ' --rdbms_sqlite_path=google_appengine_datastores/%s_rdbms' % (appname)
new_cmd += " " + " ".join(sys.argv[2:])
#Write new command to the batch file
f = open('_cmdrewrite.bat','w')
f.write(new_cmd)
f.close()

The above three files make up the command line rewriting system. It basically will take a normal command line that you see on the Google App Engine tutorials and documentation and rewrite them to work with Portable Python. This makes the Portable Google App Engine system much more useful because you can easily follow along with the real documentation and tutorials without having to adjust or rewrite your own command lines. It also makes the datastore and history files portable by storing them in the “google_appengine_datastore” folder created earlier.

Creating your Application

Now that we’ve done all the prep work let’s see how this thing actually works. Creating a new application does take a few steps but they are pretty basic and boring. The quick and dirty steps are as follows:

Create the App folder.
Create a bunch of source files in your App folder.
Run your App!

I’ll walk you through all the above steps to give you an idea of what is going on. We are going to use the example source from Google for the actual example app content parts. It is a great idea to read over their documentation as it will give you more insight into how the Google App Engine works.

All of our applications are going to reside in their own folders inside the Portable Python folder. Create a new folder called “helloworld” in your Portable Python folder. I’ll refer to this as the App folder from now on as all the important files will go inside this folder for our first app. The next few steps come directly from the Hello World tutorial from Google.

Create a new file called “helloworld.py” in your App folder with the following contents:

1
2
3

print 'Content-Type: text/plain'
print ''
print 'Hello, world!'

Create a new file called “app.yaml” in your App folder with the following contents:

application: helloworld
version: 1
runtime: python
api_version: 1
 
handlers:
- url: /.*
  script: helloworld.py

The super important file here is the “app.yaml”. You can think of it as the master control center of your App. It has all the settings and such that tell Google App Engine what to do. Google’s documentation lays out everything for this file. We’ve got enough in ours above to get going but I definitely recommend reading their docs when you get a chance.

Running your Application

Our HelloWorld App is ready to go! Let’s get this thing up and running. We are going to need a new command interpreter window opened to our Portable Python folder. Now type the following:

dev_appserver.py helloworld

After pressing enter you should see something close to the following:

P:\PortablePython_1.1_py2.5.4>dev_appserver.py helloworld
INFO     2011-01-08 15:21:00,736 appengine_rpc.py:153] Server: appengine.google.com
WARNING  2011-01-08 15:21:00,743 datastore_file_stub.py:573] Could not read data
store data from P:\PortablePython_1.1_py2.5.4\google_appengine_datastores\helloworld_datastore
INFO     2011-01-08 15:21:01,075 dev_appserver_main.py:485] Running application
helloworld on port 8080: http://localhost:8080

You can ignore the warning in the middle that says “Could not read data store data”. That just means that the datastore has not been created yet. Now open a web browser and point it to the following URL: http://localhost:8080. You should now see “Hello, world!” in your browser! That is it! Your first Google App Engine application is running. When you want to stop your application press Ctrl+Pause/Break. It will prompt you “Terminate batch job (Y/N)?” so just press ‘y’ and then press enter.

Other available commands

Google App Engine’s dev_appserver.py has a lot of options and functionality to help you manage your application. The new command rewriting system should be compatible with all the options that are available. You should definitely read through the options in their documentation so that you can utilize things like clearing the datastore, changing the webserver port, using SQLite for the datastore and much more. Let’s look at a few of these commands that you will see in the tutorials.

To run the development app server and clear the current datastore use the following command:

dev_appserver.py --clear_datastore helloworld

To run the development app server using port 9999 use the following command:

dev_appserver.py --port=9999 helloworld

If you want to use SQLite for the datastore (increased performance for large amounts of content), use the following command:

dev_appserver.py --use_sqlite helloworld

You also can use the appcfg.py commands to upload your application by using the following command:

appcfg.py update helloworld

Conclusion

You now have a fully functioning Portable Google App Engine development environment! You can follow all the tutorials on the Google App Engine website with ease. If you ever need to update the Google App Engine SDK, just delete the google_appengine folder from your Portable Python folder and extract the new one from the main download site. If you have any questions or ideas on how to improve this system, please let me know!

Growing Pains

David Lambert — Tue, 05 Oct 2010 01:47:34 +0000

I recently moved to a new website host and have been in the process of migrating the main site and tools section over. It seems that in the transition the URL styles were not set correctly causing some of the older links to not work. This has been repaired. Sorry if you got a bunch of “Page not found!” errors over the last week.

When? Version 1.0.3 Just released in the App Store!

David Lambert — Thu, 26 Aug 2010 17:08:15 +0000

I am happy to announce that version 1.0.3 of When? has just been released in the App Store!

Click here to check it out in the App Store!

When Version 1.0.3

This version includes holidays/events for a bunch of new countries:

Austria
Belgium
Denmark
Finland
France
Hong Kong
India
Ireland
Italy
Luxembourg
Mexico
Netherlands
Norway
New Zealand
Russia
Singapore
Spain
Switzerland
Sweden

I’ve got a few Promo Codes available to anyone who would like to try it. First come first serve basis. If you like it (or not) please leave a review on the App Store. I’m always open to new ideas and suggestions.

Promo Codes:

PEYTEJA33AWW
FAXAXXRRJK3K
FM6HJMENEE9Y
YLWPYM4NMLLY
KPPA6LMNRPWM
NAYAJ4XJXYYR
JKW9JAM4L6ER
MMJ9J69TF9KP
NRRLLY3X49RX

When? Holidays and Important Events App for iOS4 Devices

David Lambert — Thu, 29 Jul 2010 01:29:58 +0000

Introduction

Do you want to know when Mother’s Day is this year? Wouldn’t it be easier to have it on your calendar? It gets pretty tedious adding all those important events and holidays by hand. When? is an App for iOS4 Devices that allows you to easily find and add holidays and important events to your built in calendar.

It contains important events for many countries (with states or provinces where applicable) and religions. New categories and events will be continually added. When? makes it simple to add events to your calendar in two easy steps:

Find the event or events you want using categories.
Tap the + button and the event is immediately added to your calendar!

You can even add an entire category with just two taps. Tired of having an event on your calendar? Remove it just as easy by pressing the red check mark. That is all you need to do!

Click here to check it out in the App Store!

WebSolver: An Interactive Cryptogram Solver for the Web!

David Lambert — Sun, 20 Jun 2010 05:48:37 +0000

A great carpenter does not need to know how to make a hammer or saw in order to create a masterpiece.

A while back I wrote a bunch of articles describing how to create your own interactive cryptogram solver using python. I’ve used it a lot in my own personal solving and have had a few friends use it too but it definitely is not a tool for everyone. There are a lot of great tools out there for solving ciphers but many of them are very technical, require a special environment to run, or require programming knowledge and experience. Now, I don’t want to down play the usefulness of these great tools or tell anyone that they shouldn’t learn computer programming (Note: I think all computer users should know the very basics of programming logic as it is so useful for many things). That being said, there are many people out there who love a good challenge but either don’t have the desire or the time to learn to program. Sometimes the greater skill is knowing how to use the tools properly and efficiently. A great carpenter does not need to know how to make a hammer or saw in order to create a masterpiece.

So for all you great carpenters (solvers) out there, I have ported my interactive cryptogram solver to the web! No downloads, no special environments (other than a modern web browser), and no programming skills needed. Just a clean area where you can do what you do best: solving.

Web Solver is available now at http://tools.codepenguin.com/websolver/.

At the moment, the Web Solver only has support for Aristocrats and Patristocrats. I will be adding a lot more as time progresses. I’m still learning so I’ll add my techniques as I learn them. If you have any suggestions or different techniques that could be added, please leave a comment or contact me. I’m always open for suggestions. If you are familiar with my previous interactive solver, then the interface will be very similar. This time there is an integrated help menu available on the right side of the screen that explains all the commands. The interface is mostly keyboard driven as changes can be made more quickly with the keyboard than the mouse. This allows many different types of ciphers to be solved in a similar interface with commands that are specially designed for them.

Quick Overview

Lets run through a quick session of using the Web Solver. I’m going to grab the A-2 cipher from the sample issue of the Cryptogram. When you first open the Web Solver for the first time, it should automatically show you the “Load” Screen. This screen can be accessed at any time by clicking the load button on the right hand side of the screen. The help button is also in that same area providing a handy command reference and usage guide. I’m going to paste/type in the A-2 cipher text into the text area on the loading screen and then click the “Start Solving!” button.

Note: As we go through a few examples of commands, they will be displayed in little terminal boxes in this article for display and usage purposes. Images would be too small and you wouldn’t be able to copy and paste from them. So just be aware that the look will be simulated but pretty close to what the real Web Solver looks like.

For all the cipher types there are three main commands: clear, display, and pattern. Clear just clears the screen in case you want to remove the clutter. Display (shortcut: d) displays the current state of the cipher. This generally will display both the ciphertext and the current state of the plaintext. Since all cipher types are different, the display will vary depending on the type. For example: Aristocrats will have word spacings while Patristocrats will be broken into five letter groups. Lets check out what the display command looks like for our current con.

Aristocrat Solver loaded.
> d
OB ISZDPH *GQG EFBE KZE NZUZPJ SQQO ZE EQ EOFNN AKFA BQT YFP'A EKQTA
-- ------ *--- ---- --- ------ ---- -- -- ----- ---- --- ---'- -----
 
FA AKD YFA VZAKQTA JDAAZPJ F OQTAKITN QI KFZS.
-- --- --- ------- ------- - -------- -- ----.
>

Useful commands

Now that we can see our ciphertext, lets take a look some patterns. The patterns command (shortcut: p) can be used to quickly look up a pattern and return any words that match it. This command is pretty powerful so definitely read the help on it for more information. Lets just look at a quick example. We’ll take at a few examples from the con:

> p ISZDPH
389 matches found.
absurd acting action admire adults advice advise agency agents aliens almost amount
angels answer anyhow around asking author awhile backed backup barely basket beacon
beauty begins behalf behind beings belong betray beyond bishop blocks blonde bodies
boring bother bought brains breaks breast breath bridge bright brings broken budget
burden buried burned busted buster buying campus candle caring carpet casino castle
caught caused chairs change chapel charge chosen cipher claims client closed closer
closet column comedy coming counts county couple course cousin covers coward credit
crimes cruise crying custom dancer danger dating deacon demons denial deputy design
detail direct double doubts
290 words not shown.
 
> p OQTAKITN
5 matches found.
adoption handling partners patients ultimate
 
> p JDAAZPJ
1 match found.
getting

Notice that you can grab the words directly from the ciphertext. You can also search for number based patterns and use filters to narrow down the results. There are four different word lists currently available with varying amounts of words. You can switch to a different word list at any time. We’ll use the set (shortcut: t) command to set the plaintext letters for one of the pattern words that we just found.

> s JDAAZPJ getting
OB ISZDPH *GQG EFBE KZE NZUZPJ SQQO ZE EQ EOFNN AKFA BQT YFP'A EKQTA
-- --ien- *--- ---- -i- -i-ing ---- i- -- ----- t--t --- --n't ----t
 
FA AKD YFA VZAKQTA JDAAZPJ F OQTAKITN QI KFZS.
-t t-e --t -it---t getting - ---t---- -- --i-.

The set command will except full words or single letters. You can even reset a letter by only giving a ciphertext letter. (Example: s X). Now that we have some letters we can narrow down one of our previous pattern searches that had 389 matches.

> p ISZDPH --ien-
3 matches found.
aliens client friend

That looks much better. These filters can be used to quickly narrow down the results. You can even use wild cards if you don’t know exact lengths.

If you want to see the frequency chart, you can use the frequency command (shortcut: f). It can even give you digraphs and trigraphs if you give it a length.

> f
A:12 Q:9 Z:8 F:8 E:7 K:7 T:5 O:4 P:4 N:4 B:3 I:3 S:3 D:3 J:3 G:2 Y:2 H:1 U:1 V:1
 
> f 2
AK:4 QT:4 FA:3 TA:3 ZE:2 ZP:2 PJ:2 KF:2 YF:2 KQ:2
 
> f 3
QTA:3 ZPJ:2 KQT:2

These are only a few of the commands that are available. The help system will give you details for everything with examples. If you have any ideas for new commands, just let me know. They can be easily added and then everyone can have access to new solving techniques.

Save and Continue

Ever been solving a cipher and had to go do something else? Maybe your computer crashed? The Web Solver will automatically save your progress and allow you to come right back where you were as long as you’re using the same computer. All the data is stored locally on your computer.

Command History

Web Solver will keep the history of the last 25 commands that you have used. You can go back through them by pressing up and down. This way you can save some typing if you want to go back to a previously used command.

Conclusion

Web Solver is a solving tool. It won’t do all the work for you but hopefully it will make things easier. It is available now at http://tools.codepenguin.com/websolver/. Please, let me know what you think. Leave a comment or contact me directly. I love to hear your ideas and suggestions. I will continue to improve Web Solver and add new features as time goes on.

Technical Details and Links

For those interested, I’d like to mention the different technologies and techniques that made this possible.

jQuery – A powerful javascript library that makes cross-platform web development a whole lot easier. This is pretty much the core of the entire implementation.
Taffy DB – A JavaScript library that acts as an in memory data layer. This powers the CP List which is a small embedded word list. All the other word lists require an external database connection. CP List can be used without an internet connection.
LazyLoad – A JavaScript library used to load files dynamically only when needed. This is used to load the word list functionality on the fly without taking up a lot of memory.
PersistJS – A JavaScript library that provides client-side persistent storage. This handles all the storage needs and works great in all modern browsers.
How To Create A Sexy Vertical Sliding Panel Using jQuery And CSS3 – This tutorial helped me create the sliding Load and Help panels.

Pattern Words: A Solver's Staple

David Lambert — Sat, 12 Jun 2010 05:08:49 +0000

While working on the Beginner’s Guide to the ACA, there was one technique that consistently showed up in all the research material: Pattern Words! At first I didn’t really understand how great they were. For those who don’t know about them or need a refresher, I’ll discuss a few of the different pattern types that I have come across. Afterwards, I’ll discuss my own Pattern Word Search Tool. Lets get started!

Pattern Types

Fixed Length Patterns

Pattern words have one or more letters that are repeated. These repeated letters provide a clue to the word’s identity. Start by taking a word and assigning numbers to each letter starting at the left with 1 . Use the same number for repeated letters. Now you can look for the number based pattern in a pattern word list to see which words match the pattern. I call these the “Fixed Length Patterns”.

Common pattern words worth memorizing:

1231    123142    12342
that    people    which

122     12343     123314
all     there     little
off     these
see     where
too

Floating Length Patterns

While researching pattern words I came across a few other pattern types that are useful also. Codasaurus.com mentioned a shorted version of the above numeric pattern that only shows the repeated characters. I call these the “Floating Length Patterns”. These are great when you don’t know the full length of the word. First calculate the pattern as above. Then, starting at the front, write down all the positions of the first duplicated letter. Continue this process for all other duplicated letters. This form can be useful when you don’t know the exact length of a word or when multiple word endings are possible. The above examples would be as follows using this method:

14      1426      25
that    people    which

23      35        1534
all     there     little
off     these
see     where
too

High/Low Frequency Patterns

In the comments on my article Social Cryptogram Solving, BOATTAIL mentioned patterns that looked at the frequency of letters. Using a frequency table you can determine which letters are high and low frequency letters. Then replace the letters with H for high and L for low. For English, the high frequency letters are ETAONIRSH. The examples from above would look as follows:

HHHH    LHHLLH    LHHLH
that    people    which

HLL     HHHHH     LHHHLH
all     there     little
off     these
see     where
too

Searching for Pattern Words

You can find pattern word lists on the internet by just searching for them. Most of the time they are a little hard to handle for beginners because they usually come in raw text files and they generally just contain the fixed length patterns. I’ve created a special website that will allow you to search for any of the patterns discussed above in a few different word lists. At the moment I have three different word lists that are searchable. They each have their own strengths and weaknesses. You can access it at http://tools.codepenguin.com/patterns/. The page has instructions for how to use each of the different patterns above and how to apply word filters to weed out the possibilities.

Depending on the word list used, pattern words become less useful simply because they return too many words. That is where the word filters come into play. With word filters, you can shrink the matches for the patterns down to ones that fit your needs. For example: if you look up the pattern 1426 (for people) in the ENABLE2K word list, it returns 130 words. If you know that the first and fourth letters are “p” you can use the filter “p..p*” the matches drop down to 24 words. 24 is much more manageable and realistic.

If you find my pattern search tool useful, please let me know. I’m always open for suggestions on how to improve my tools and for new ideas.

Conclusion

All the different kinds of patterns give you a lot of techniques to aid in solving Aristocrats and Patristocrats. You can get quite far by using them. Pattern word lists are available on the internet and there are some published books that have them too. They are an essential tool for any solver’s tool box.

Beginner's Guide to the ACA

David Lambert — Mon, 07 Jun 2010 03:32:09 +0000

When I first joined the ACA and opened my starter packet, I was eager to jump right in and start solving Aristocrats and Patristocrats. I tried to solve as many as I could. Once I got stuck I would try to learn a different way to solve in hopes that it would get me unstuck. There is so much great information available on solving different cipher types and trying out new techniques. Unfortunately, since there is so much great information, it is difficult to figure out where to start or what resources to use. I found myself frequently getting lost in material too advanced at the early stage and becoming quite frustrated when the information did not penetrate my thick skull.

After one frustrating week, I contacted LIONEL, the ACA Recruitment Committee chairman, in hopes of finding some form of beginner’s guide to help me stay focused. He mentioned the “Young Tyros Newsletter”. It is a wonderful resource and I really recommend all new members and even non-members to read them. They have lots of great insights and tips to get you unstuck. In our discussions, I proposed that we should create a “Beginner’s Guide to the ACA”. I definitely didn’t want to re-hash all the great information out there. My goal was to compile a small list of the most important topics for a beginner to know and then point them in the right direction of all the other great materials out there.

LIONEL put me in contact with a few members who graciously agreed to send me some of their notes on how they solved some of the lower level Aristocrat and Patristocrat cons. Their notes and insights have been a great help in increasing my knowledge and solving abilities. It took quite a long time and many iterations before I was satisfied with it but I finally finished the first version of the “Beginner’s Guide to the ACA!” This would not have been possible without the help of BECASSE, GGMA, LIONEL, MICROPOD, MATANZA, and ZANAC. I am very grateful for their help.

You can download the Beginner’s Guide to the ACA below. Hopefully, it’ll help someone not get frustrated and lose interest in cryptology before they gave it a good chance.

Click here to download!

Quartus II Quick Start

David Lambert — Tue, 09 Feb 2010 03:12:29 +0000

I’m taking a Computer Engineering Course and Lab and have been learning how to use the software Quartus II to generate and test logic circuit schematics. The tutorial given in the class was quite long so it was hard to fully retain all the information and quickly find anything if I forgot a step. I went through the tutorial and wrote down a few helpful steps and thought I would share it.

Quartus II Quick Start

Creating an interactive cryptogram solver (Part IV)

David Lambert — Sun, 01 Nov 2009 03:40:25 +0000

I’ve been working hard on this part of the series because I really wanted the interactive cryptogram solver to make it quick and painless to jump right into solving and still give you plenty of room to expand the functionality and reflect your own style of solving. In this part of the series, we will create solver.py which will become our gateway to solving. It will allow us to quickly select a cipher class that we want to work with. We will also add a self documenting system that will allow us to use the solver without memorizing all the commands or shortcuts that each solver class may use. So, lets just jump right back into the code!

Clearing the screen

While we are using the solver, it is a common place for the screen to get a little crowded. So lets add a new function to our shared.py that will take care of clearing the screen. Clearing the screen is a little complicated because each operating system has it’s own way of handling it. Our clear_screen function will handle all the major players: Windows, Mac, and Linux.

Contents of shared.py (Place at the bottom of the file)

def clear_screen():
  import os
  if os.name == "posix":using the
      # Unix/Linux/MacOS/BSD/etc
      os.system('clear')
  elif os.name in ("nt", "dos", "ce"):
      # DOS/Windows
      os.system('CLS')

All we are really doing here is importing the os and using the os.name to tell which operating system we are in and then executing the appropriate command for clearing the screen.

The Solving Gateway: solver.py

I call solver.py the gateway because this is really the file that you will be executing the most. Everytime we create a new class based on CipherSolver, we’ll want to import it into solver.py so we can access it. I’ll show you how to do that in just a minute. First, lets just show the entire contents of solver.py and then we’ll go through the good parts.

Contents of solver.py

import shared
from cipher import CipherSolver
from aristocrat import AristocratSolver
from patristocrat import PatristocratSolver
 
if __name__ == '__main__':
  solvers = {}
  for name, obj in globals().items():
    if name.endswith("Solver") and issubclass(obj, CipherSolver):
      solvers[name[:-6]] = obj
 
  while True:
    shared.clear_screen()
    print "-" * 37
    print "    Interactive Cryptogram Solver"
    print "-" * 37
    print "Type a cipher type or partial (q to quit) [all]: ",
    cmd = raw_input().lower()
 
    if cmd in ["q","quit"]:
      break
    else:
      list = []
      for name, obj in solvers.items():
        if name.lower().startswith(cmd):
          list.append(name)
      if len(list) > 0:
        list.sort()
        for index in range(1,len(list) + 1):
          print "  {0}. {1}".format(index, list[index-1])
        print "Type number of selection or return to go back: ",
        cmd = raw_input()
        if cmd.isdigit() and (0 < int(cmd) <= len(list)):
          name = list[int(cmd)-1]
          print "\nLoaded {0} Solver.".format(name)
          solvers[name]().solve()
      print ""

The first few lines are essentially the most important part of this file. They import all the CipherSolver classes that we want available. You only need to import the your Solver classes, not the Cipher classes.

2
3
4

from cipher import CipherSolver
from aristocrat import AristocratSolver
from patristocrat import PatristocratSolver

Right now we only have three classes to import. We have to import the CipherSolver class because everything is based on that one. This is where you would include any other Solver classes that you create. The pattern is:

 from FILENAME import SOLVERCLASS

Just add any new ones below the PatristocratSolver line.

if __name__ == '__main__':
  solvers = {}
  for name, obj in globals().items():
    if name.endswith("Solver") and issubclass(obj, CipherSolver):
      solvers[name[:-6]] = obj

Line 6 is an interesting line of code that you may have seen in other Python files. Python files can be executed on their own or imported from other Python files as we have seen above. One problem that arises is that python tries to execute all lines that are in the file as it is imported. But what if we want to execute code only when our file is executed on its own? That is exactly what this line of code does. It essentially says “If this file is being executed on its own and not imported, then do the following:”. Anything indented below the “if statement” will only be executed when we type:

python solver.py

The rest of the lines in this section walks through all the imported classes and functions in the globals dictionary. Whenever we use the import command, the things we import get stored in the globals dictionary. So here we are scanning globals and pulling out any classes that are based off of our CipherSolver class. We’ll store all the classes and their names so we can use them later.

Note: I’m going to warn you right now, this part of the series contains a lot of code that is purely just for display purposes. We’ll be formatting a lot of things things to display on the screen. I won’t go over every line in these cases. Just know that if you see “print” or “output” then those are things that will be displayed or eventually displayed.

    cmd = raw_input().lower()
 
    if cmd in ["q","quit"]:
      break
    else:
      list = []
      for name, obj in solvers.items():
        if name.lower().startswith(cmd):
          list.append(name)
      if len(list) > 0:
        list.sort()
        for index in range(1,len(list) + 1):
          print "  {0}. {1}".format(index, list[index-1])
        print "Type number of selection or return to go back: ",
        cmd = raw_input()
        if cmd.isdigit() and (0 < int(cmd) <= len(list)):
          name = list[int(cmd)-1]
          print "\nLoaded {0} Solver.".format(name)
          solvers[name]().solve()
      print ""

Here we prompt the user to type in the name or part of a name of the class that they would like to use. We then walk through our list of Solver classes and display any that match the inputted name and display them on the screen. The user then gets to type the number that corresponds with the class on the screen. You can also just press enter to get a list of all available classes. Lets see what the looks like when we run it.

python solver.py
-------------------------------------
    Interactive Cryptogram Solver
-------------------------------------
Type a cipher type or partial (q to quit) [all]:

Lets just press enter here to see all the classes that are loaded:

  1. Aristocrat
  2. Cipher
  3. Patristocrat
Type number of selection or return to go back:

If we type “3″ to select Patristocrat and then press enter we get the following:

Type number of selection or return to go back:  3
 
Loaded Patristocrat Solver.
>

Now we are ready to solve a Patristocrat! Our PatristocratSolver class is automatically loaded and executed. If we want to go back to the selection screen we can just press Ctrl+C at any time.

The Documentation System

I don’t know about you but I have a major problem remembering things sometimes. I even have a T-Shirt that says “Insufficient Memory” on it. So if we create a new solver class for the over sixty different cipher types used in the Cryptogram, there is no way I could remember all the different commands and functions that we would need. It would be very beneficial for us to have documentation on all the different classes we use and to be able to display it whenever we need to. I’m not the greatest at writing documentation especially when the code can change so quickly. So our documentation system would have to be easy to update and not super complicated. Alright, here we go. We are going to put the documentation into the CipherSolver class so most of our changes are going to be in the cipher.py file.

Contents of cipher.py (Add at the top on line 4)

4	import inspect,types

Here we are importing two modules. The inspect module is going to be the core of our documentation. The python documentation describes this module as:

inspect – Get useful information from live Python objects.

This module can give you all sorts of information about our classes and objects. We’ll see how its used in a minute.

class CipherSolver(object):
  def __init__(self):
    self.cipher = Cipher()
    self.prompt = ">"
    self.maxlinelen = 70
    self.shortcuts = {"?":"_display_help","d":"display"}

The above code is just slightly different from Part III. We are now telling the CipherSolver that it should inherit from the object class. This is the new style of classes in python and just makes sure our code will work in future version of python and that all the special python modules that use objects will work correctly. We also add a new shortcut on line 35. This shortcut lets us type “?” at any time to activate our help system by calling the _display_help function.

One thing to note here, any function that starts with a “_” will be hidden from the documentation system. They won’t be displayed. This allows you to add helper functions that are not available for calling directly. Lets add a few of these little helper functions. (I’m going to go a little out of order here. If you want to see the cipher.py file in its correct order, view the source at the end of this article.)

  def _inherit_docs(self, BaseClass):
    members = inspect.getmembers(self, inspect.ismethod)
    for name, method in members:
      if method.__doc__ == None and hasattr(BaseClass, name):
        method.im_func.__doc__ = getattr(BaseClass, name).__doc__

The _inherit_docs function will walk through all the different overridden functions in our classes and make sure they have their own documentation or use the documentation from the base class. This may sounds confusing at first but we’ll show actual code examples later. Simply put, since all our classes are going to have a display function that just does things a little different for each class, why should we have to repeat the documentation each time? In our documentation system, the PatristocratSolver will look at the AristocratSolver for its documentation and the AristocratSolver will look at CipherSolver for its documentation. Lets add those calls right now. Add the following to the aristocrat.py and patristocrat.py.

Contents of aristocrat.py (Add to the AristocratSolver.__init__ method)

44	self._inherit_docs(CipherSolver)

Contents of patristocrat.py (Add to the PatristocratSolver.__init__ method)

15	self._inherit_docs(AristocratSolver)

We need to make one major modification to the CipherSolver.solve method. In the previous parts of this series, we made the solve method handle any methods that are part of the solver class and the cipher class. Our modification is going to change this to only go through the solver class. This might sound like a disadvantage at first, but it will make our solver much more powerful in the end. Change your solve method to the following:

  def solve(self):
    while True:
      try:
        raw_line = raw_input(self.prompt + " ")
        line = re.findall('\'[^\']*\'|\"[^\"]*\"|\S+', raw_line)
        line = [item[1:-1] if item[0] in '\'"' else item for item in line]
        print ""
        cmd = line.pop(0)
        attr = None
        if cmd in self.shortcuts:
          cmd = self.shortcuts[cmd]
        if cmd.endswith("?"):
          print self._display_help(cmd[:-1])
        else:
          if hasattr(self, cmd) and cmd != "solve":
            attr = getattr(self, cmd)
          if attr == None:
            print "Unknown command: ", cmd
          else:
            retvalue = None
            if type(attr) == types.MethodType:
                retvalue = attr(*line)
            else:
              if len(line) == 0:
                retvalue = attr
              else:
                setattr(self, cmd, *line)
            if retvalue != None:
              print retvalue
          print ""
      except KeyboardInterrupt:
        print "Exiting Solver"
        break
      except Exception, e:
        print "Error: ", e

The big change here is that we first check to see if the command ends with a question mark. If so, it will pass the command, minus the question mark, to the _display_help function that we’ll discuss in a minute. We also add the support to get and set a properties that are defined in the class.

Now comes the entire documentation system. Its all bundled in a special function called _display_help. Now be prepared, this is a long function.

  def _display_help(self, cmd=None):
    if cmd in self.shortcuts:
      cmd = self.shortcuts[cmd]
 
    def _get_function_def(name,function):
      specs = inspect.getargspec(function)
      defaults = None
      if specs.defaults:
        defaults = dict(zip(specs.args[-len(specs.defaults):],specs.defaults))
      output = name
      for arg in specs.args:
        if arg != 'self':
          output += " " + arg
          if defaults and arg in defaults:.  I
            output += "=" + repr(defaults[arg])
      return output.  I
 
    def _display_doc(obj):
      doc = ""
      if inspect.getdoc(obj):
        for line in inspect.getdoc(obj).split("\n"):
          doc += "   " + line + "\n"
        doc += "\n"
      return doc
 
    cmd_shortcuts = dict([(v,k) for k,v in self.shor.  Itcuts.items()])
    obj = None
    if cmd == None:
      obj = self.__class__
    elif hasattr(self.__class__,cmd):
      obj = getattr(self.__class__,cmd)
    output = ""
    if inspect.isclass(obj):
      output = obj.__name__ + ":\n\n"
      output += _display_doc(obj)
      member_filter = lambda obj: inspect.ismethod(obj) or isinstance(obj,property)
      members = dict(inspect.getmembers(obj, member_filter)).keys()
      members = [name for name in members if not name.startswith("_") and name != "solve"]
      members.sort()
      output += "  Available Commands:\n"
      for index in range(len(members)):
        member = members[index]
        if cmd_shortcuts.get(member):
          member += " [" + cmd_shortcuts.get(member) + "]"
        output += member.rjust(20)
        if (index + 1) % 3 == 0:
          output += "\n"
    elif inspect.ismethod(obj) and cmd != "solve":
      output = cmd
      shortcut = dict([(v,k) for k,v in self.shortcuts.items()])
      if shortcut.get(cmd):
        output += " [" + shortcut.get(cmd) + "]"
      output +=":\n\n" + _display_doc(obj)
      output += "   Usage:\n      " + _get_function_def(cmd, obj)
    elif isinstance(obj,property):
      output = cmd
      if obj.fset == None:
        output += " (Read Only)"
      if cmd_shortcuts.get(cmd):
        output += " [" + cmd_shortcuts.get(cmd) + "]"
      output += ":\n\n" + _display_doc(obj)
      output += "   To display the value:\n      " + cmd + "\n\n"
      if obj.fset != None:
        output += "   To change the value:\n      " + _get_function_def(cmd, obj.fset)
    else:
      output = "Unknown command: " + cmd
    return output.rstrip("\n") + "\n"

It looks daunting at first but is actually quite simple. If you remove all the display code (all the output statements) this is what you’re left with: the function takes a command that you want to see the documentation for and displays it or if you pass in blank, it’ll display all the commands that are available and any shortcuts that are defined. Ok, lets see how this works right now! Fire up sandbox.py for now:

python sandbox.py
> ?
 
PatristocratSolver:
 
  Available Commands:
         display [d]  frequency_list [f]            keys [k]
        print_counts             set [s]
 
>

So if we type “?” It’ll show us all the commands! Notice that the shortcuts are listed in brackets next to the commands. This way you can see them if you have forgotten what they are. Now lets get back to the code. Lets look at a few of the important parts of the _display_help function.

Did you notice that the print_counts function was listed in our command list from above? That isn’t really a command that should be executed in the solver. It’s one of those special helper functions. Lets change its name to _print_counts so that it will be hidden (Note: You will need to change where it is used in aristocrat.py and patristocrat.py too or it will give you an error.).

Lets chop up the _display_help function into the three types of objects that the command passed in can represent. First, a solver class like AristocratSolver. Second, a method of the class like display. Third, a property of the class.

Solver Classes

As we saw before, if no command is passed into the function, then a summary of all available commands is displayed. This will include any shortcuts and properties that are defined for the class. This will also include any commands that are inherited from other classes. Lets look at the code that makes this possible:

95
96

      member_filter = lambda obj: inspect.ismethod(obj) or isinstance(obj,property)
      members = dict(inspect.getmembers(obj, member_filter)).keys()

First we create a quick little lambda function that returns True or false. We want to get a list of all methods and properties of our class, so the lambda function will return True only if the “obj” argument is a method or property. Next, we execute the inspect.getmembers function and pass in the obj (which is our solver class at this moment) and our lambda function from above. In python, the inspect.getmembers function would return the following:

>>> import inspect
>>> from patristocrat import PatristocratSolver
>>> member_filter = lambda obj: inspect.ismethod(obj) or isinstance(obj,property)
>>> inspect.getmembers(PatristocratSolver,member_filter)
[('__init__', ),
('_display_help', ),
('_inherit_docs', ),
('_print_counts', ),
('display', ),
('frequency_list', ),
('keys', ),
('set', ),
('solve', ),
('text', )]
>>>

It returns a big list of name/value pairs for all the members that the member_filter evaluated to True. It would be a much longer list if we had not passed in a filter function. The documentation system will use this list to display a nicely formatted display of commands that we can execute.

Methods

When a method name is passed into the function, the doc string for the function is displayed, followed by the appropriate way to call the function and what parameters to pass in. Doc strings are an important part of Python classes and functions. They help provide documentation automatically. Doc strings are placed on the line directly below the declaration of the method. They are started with three quotes “”" and ended with three more quotes “”" Lets add a doc string to the display method and then see how our documentation system will react to it.

41
42
43

  def display(self):
    """Displays the current data on the screen."""
    print self.cipher.text

Here we have add just one line right below the declaration of the display method. The doc string can contain any information you want displayed. It can really be as long as you want and can span multiple lines. Now lets see how it works. Load up the solver and select Patristocrat.

Loaded Patristocrat Solver.
> display?
 
display [d]:
 
   Displays the current data on the screen.
 
   Usage:
      display
 
>

Thee display isn’t as impressive as if we looked at the frequency_list function. Lets check that one out to better see what is displayed (Note: We haven’t put in a doc string for the frequency_list function yet so you won’t see that):

> frequency_list?
 
frequency_list [f]:
 
   Usage:
      frequency_list length=1 text=''
 
>

We can see the names and default values for the frequency_list. If we don’t pass in anything (by just typing frequency_list and hitting enter), it will automatically pass in a length of 1 and use the internal text value. But we could pass in other lengths if we want to. You don’t have to do anything special to get this to display nicely like this. The code for it is actually quite simple thanks to the inspect module. The inspect code is in the _get_function_def method that we declared above.

113	output += " Usage:\n " + _get_function_def(cmd, obj)

    def _get_function_def(name,function):
      specs = inspect.getargspec(function)
      defaults = None
      if specs.defaults:
        defaults = dict(zip(specs.args[-len(specs.defaults):],specs.defaults))
      output = name
      for arg in specs.args:
        if arg != 'self':
          output += " " + arg
          if defaults and arg in defaults:
            output += "=" + repr(defaults[arg])
      return output

The inspect.getargspec function takes a function as the argument and returns all the information about the function:

>>> specs = inspect.getargspec(PatristocratSolver.frequency_list)
>>> specs
ArgSpec(args=['self', 'length', 'text'], varargs=None, keywords=None, defaults=(1, ''))
>>>

The returned value is called a named tuple. It has all the little chunks of information that we can use. One of the cool things we can do is use the zip function to attach the argument names back to their default values:

>>> defaults = dict(zip(specs.args[-len(specs.defaults):],specs.defaults))
>>> defaults
{'text': '', 'length': 1}
>>>

The rest of the code is just the fluffy display code that makes everything look nice.

Properties

Properties are a great way to get and set different values that our solver will use. We want to be able to get and set the Cipher.text from within our solver class. Lets declare our new property in CipherSolver:

Contents of cipher.py (Add to the CipherSolver class)

  @property
  def text(self):
    """The raw text value used for encoding and decoding."""
    return self.cipher.text
 
  @text.setter
  def text(self,value):
    self.cipher.text = value

The “@property” tells python that we want the method that follows it to be a property. So now whenever we use the “text” command, the cipher.text will be display. The “@text.setter” tells python that if we try to set the text property, it should change the cipher.text. We can create as many of these kinds of properties we want whenever we want to get and set special values. We’ll probably use them a lot when working with out cipher types.

The display for properties is different from the classes and methods from above. Lets see what our new text property looks like in the documentation system and the solver. Lets load up sandbox.py:

> text?
 
text:
 
   The raw text value used for encoding and decoding.
 
   To display the value:
      text
 
   To change the value:
      text value
 
>

So it shows the doc string for the property, how to display it, and also had to change it. Lets display and set our property:

> text
 
GSRHRHZGVH GLUGSVVNVITVMXBYIL ZWXZHGHBHGVN
 
> text "THIS IS A TEST"
 
 
> text
 
THIS IS A TEST
 
>

Now we can easily set the text property whenever we want to start a new con. These properties are really easy to create and are very powerful. Python makes it very easy to enhance our solver with whatever we need to get the job done.

Document Everything!

At this point the implementation of the documentation system is complete. All we have to do now is just add doc strings to all our methods and properties in each of the solver classes.

CipherSolver (cipher.py)

We’ve already added the doc strings to the existing methods and properties in our CipherSolver but lets add one more method that will come in handy. The clear method just clears the screen whenever called. Sometimes when you’d made a lot of changes and there is tons of data on the screen, it is refreshing just to clear the screen before you continue. Add the following code above the display method:

37
38
39

  def clear(self):
    """Clears the screen"""
    shared.clear_screen()

AristocratSolver (aristocrat.py)

Since the AristocratSolver class inherits from CipherSolver, we want to make sure that all our overridden methods like display get their doc strings set correctly so we don’t have to repeat ourselves. We do this by just adding a call to the _inherit_docs method in the AristocratSolver.__init__

class AristocratSolver(CipherSolver):
  def __init__(self):
    CipherSolver.__init__(self)
    self._inherit_docs(CipherSolver)

We need to add a few bits of documentation to our existing methods inside the AristocratSolver class. Add the following doc strings:

60
61

  def frequency_list(self, length = 1, text = ""):
    """Displays counts for frequencies of characters"""

65
66

  def keys(self):
    """Displays the plaintext and ciphertext keys."""

  def set(self, ct, pt = "-"):
    """Sets the plaintext equivalent for each ciphertext character.
    You can enter multiple letters at a time.
    Enter a single dash '-' to set the plaintext characters to blank."""

PatristocratSolver (patristocrat.py)

PatristocratSolver class inherits from AristocratSolver so we need to call the _inherit_docs method in the PatristocratSolver.__init__:

class PatristocratSolver(AristocratSolver):
  def __init__(self):
    AristocratSolver.__init__(self)
    self._inherit_docs(AristocratSolver)

Conclusion

That was a lot! Hopefully, you survived up to this point. This concludes the documentation system for the interactive solver. You can add your own methods to any of the classes and add doc strings to document things. In Part V, the next and last article in this series, we will include loading the cons from the Cryptogram Digital Con files (available at cryptogram.org for members).

Complete Source Code

You can download the complete source code to Part IV at: Creating an interactive cryptogram solver (Part IV – Source).

Creating an interactive cryptogram solver (Introduction)
Creating an interactive cryptogram solver (Part I)
Creating an interactive cryptogram solver (Part II)
Creating an interactive cryptogram solver (Part III)

Code Penguin

Slipix Version 1.0 for iOS

Introduction

Video

Screen Shots

Support

Portable Google App Engine For Windows (Enhanced)

Installation

Creating your Application

Running your Application

Other available commands

Conclusion

Growing Pains

When? Version 1.0.3 Just released in the App Store!

When? Holidays and Important Events App for iOS4 Devices

Introduction

WebSolver: An Interactive Cryptogram Solver for the Web!

Quick Overview

Useful commands

Save and Continue

Command History

Conclusion

Technical Details and Links

Pattern Words: A Solver's Staple

Pattern Types

Fixed Length Patterns

Floating Length Patterns

High/Low Frequency Patterns

Searching for Pattern Words

Conclusion

Beginner's Guide to the ACA

Quartus II Quick Start

Creating an interactive cryptogram solver (Part IV)

Clearing the screen

The Solving Gateway: solver.py

The Documentation System

Solver Classes

Methods

Properties

Document Everything!

CipherSolver (cipher.py)

AristocratSolver (aristocrat.py)

PatristocratSolver (patristocrat.py)

Conclusion

Complete Source Code

Related Posts: