jugad2 - Vasudev Ram on software innovation

Python's inspect module is powerful

2013-05-18T00:43:00.001+05:30

27.13. inspect — Inspect live objects — Python v2.7.5 documentation

The inspect module comes as part of the standard library of Python. It allows you to inspect live objects at run time by using its methods.

Here is an example:

import inspect
class Bar( object):
    def foo( self ):
        print "in Bar.foo 4"
        self .a = 1
        self .di = { 'b' : 2, 'c' : 3 }
        self .li = [ 4, 5, 6 ]

bar = Bar()
bar.foo()
for member in inspect.getmembers(bar):
print member

Running the above code gives this as (partial) output:

in Bar.foo 4

('__dict__', {'a': 1, 'li': [4, 5, 6], 'di': {'c': 3, 'b': 2}})
('__doc__', None)
('a', 1)
('di', {'c': 3, 'b': 2})
('foo', bound method Bar.foo of __main__.Bar object at 0x4035bfac)
('li', [4, 5, 6])

This shows both the bound methods and the member variables of the instance bar.

The inspect module can do a lot of other things too. Check the docs for it, linked at the top of this post.

You can also modify and run the above code snippet at this codepad.org URL:

http://codepad.org/dMLmkhQ6

It may not last there for too long, though, since they probably delete pastes to make room for new ones.

- Vasudev Ram
dancingbison.com

Reading WAV file info with Python

2013-05-16T02:04:00.001+05:30

The Python standard library comes with a module called wave.py. It allows you to read and write WAV format audio files.

Here is an example of reading WAV file info with Python:

import wave

wavf = wave.open('horse.wav', 'r')
print wavf.getnchannels()
print wavf.getsampwidth()
print wavf.getframerate()
print wavf.getnframes()
print wavf.getcompname()

The above code prints the number of channels (mono or stereo), the sampling width, the frame rate, the number of frames, and the name of the compression type (if any) of the WAV file horse.wav.

You can also write WAV files, but you need to know what data to write.

Wikipedia entry for WAV:

http://en.m.wikipedia.org/wiki/WAV

- Vasudev Ram
www.dancingbison.com

A blogging break for jugad2

2013-05-09T23:04:00.001+05:30

Dear readers,

I'm taking a blogging break for some days, due to less free time while I sort out some issues.

Will be back.

Inconvenience is regretted.

- Vasudev
dancingbison.com

PDF in a Bottle - creating PDF using xtopdf, ReportLab, Bottle and Python

2013-05-01T03:46:00.002+05:30

By Vasudev Ram

pdf_bottle.py is a program I wrote that allows you to create a PDF file from text, over the web, by entering your text into a form and submitting it.

Here is the program:

# pdf_bottle.py

# Description: Program to generate PDF from text, over the web,
# using xtopdf, ReportLab and the Bottle web framework.
# It can be used to create short, simple PDF e-books.
# Author: Vasudev Ram - http://dancingbison.com
# Copyright 2013 Vasudev Ram 
# Tested with Python 2.7.

# Version: 0.1

# Dependencies:
# xtopdf - https://bitbucket.org/vasudevram/xtopdf
# bottle - http://bottlepy.org
# ReportLab - http://www.reportlab.com/ftp/reportlab-1.21.zip
# Python - http://python.org

from PDFWriter import PDFWriter

from bottle import route, request, run

@route('/edit_book')
def edit_book():
    return '''
    <form action="/save_book" method="post">
    PDF file name: <input type="text" name="pdf_file_name" />

    Header: <input type="text" name="header" />

    Footer: <input type="text" name="footer" />

    Content:
    <textarea name="content" rows="15"   cols="50"></textarea>

    <input type="submit" value="Submit" />

    </form>
'''

@route('/save_book', method='POST')
def save_book():
    try:
        pdf_file_name = request.forms.get('pdf_file_name')
        header = request.forms.get('header')
        footer = request.forms.get('footer')
        content = request.forms.get('content')

        pw = PDFWriter(pdf_file_name)
        pw.setFont("Courier", 12)
        pw.setHeader(header)
        pw.setFooter(footer)

        lines = content.split('\n')
        for line in lines:
            pw.writeLine(line)

        pw.savePage()
        pw.close()
        return "Done"
    except Exception:
        return "Not done"

def main():
    run(host='localhost', port=9999)

if __name__ == "__main__":
    main()

To run it, you need to have Python, the open source version of ReportLab, my xtopdf toolkit and the Bottle Python web framework installed.

Here is a guide to installing and using xtopdf.

For help with installing the other products, consult their respective sites, linked above.

Then run the program with this command:

python pdf_bottle.py

Then, in a browser window, go to localhost:9999

Enter the details - PDF file name, header, footer and text content - in the form, then click Submit.

The PDF file will be generated in the same directory from where you ran the Python program.

This is the first version, and has been tested only a bit. It you find any issues, please mention them in the comments.

Various improvements are possible, including sending the generated PDF to the user's browser, or providing a link to download it (better), and I'll work on some of them over time.

P.S. Excerpt from the Bottle framework site:

[
Bottle is a fast, simple and lightweight WSGI micro web-framework for Python. It is distributed as a single file module and has no dependencies other than the Python Standard Library.

Routing: Requests to function-call mapping with support for clean and dynamic URLs.
Templates: Fast and pythonic built-in template engine and support for mako, jinja2 and cheetah templates.
Utilities: Convenient access to form data, file uploads, cookies, headers and other HTTP-related metadata.
Server: Built-in HTTP development server and support for paste, fapws3, bjoern, Google App Engine, cherrypy or any other WSGI capable HTTP server.
]

- Vasudev Ram - Dancing Bison Enterprises

Share |

Interesting interview of Gagan Biyani of Udemy

2013-04-30T01:10:00.001+05:30

By Vasudev Ram

Gagan Biyani, Udemy co-founder, interviewed on Mixergy.

And here is the story of Eren Bali, one of the other co-founders of Udemy.

From the Wikipedia page about Udemy:

"In February 2010, the Udemy co-founders decided to raise money to fund their idea.[6] They were rejected 30 times" ...

The Udemy Fundraising Story

- Vasudev Ram - Dancing Bison Enterprises

Share |

Hard-hitting presentation on Indian IT industry

2013-04-29T02:35:00.003+05:30

By Vasudev Ram

Seen via a link posted by @sharads on LinkedIn.

Whats Wrong with Indian IT Industry? from Rajesh "Raj" Varma

Got to say I agree with a lot of it. And I think most people in the know, who are honest, would agree too. Not sure how much is being done about it, on an industry-wide level, though I know of efforts here and there.

- Vasudev Ram - Dancing Bison Enterprises

Share |

PySynth, a pure Python music synthesizer

2013-04-28T03:18:00.002+05:30

By Vasudev Ram

PySynth is a music synthesizer library written in Python. (*)

Excerpt from the site:

[ There are three variants: PySynth A is faster, only needs Python itself, and sounds more like a cross between a flute and organ. PySynth B is more complex in sound and needs NumPy. It's supposed to be a little closer to a piano. (No competition for Pianoteq of course, but a reasonable fit for keyboard music.) Finally, PySynth S is more comparable to a guitar, banjo, or harpsichord, depending on note length and pitch.

The current release of the synthesizer is monophonic, i.e. it can only play one note at a time. (Although successive notes can overlap in PySynth B and S, but not A.) However, two output files can be mixed together as in the case of the stereo files below. ]

(*) Interestingly, the Changes section of the above linked PySynth page seems to indicate that PySynth uses both Pyglet and PyAudio, both of which I had blogged about some time ago:

Playing an MP3 with pyglet and Python is easy

PyAudio and PortAudio - like ODBC for sound

Here is PySynth on Github

PySynth supports ABC notation and can generate WAV audio files.

Here is an example tune:

D D C C B A G

and here is a PySynth program to play that tune as a WAV file:

# pysynth-ddccbag.py

import pysynth

test = ( ('d', 4), ('d', 4), ('c', 4), ('c', 4), ('b', 4), ('a', 4), ('g', 3) )
pysynth.make_wav(test, fn = "ddccbag.wav")

For some reason the last note seems to get partially cut off on my PC. Not sure whether that is a bug or a feature :-) or something to do with my hardware. Maybe the latter, since I'm using my laptop's built-in speakers.

Run that program as:

python pysynth-ddccbag.py

Then play the generated WAV file ddccbag.wav in a music player, such as VLC or some other one.

Here are a few better examples of sound synthesis by PySynth - the links are to MP4 files, which you can download and play without needing Python or PySynth:

The Sailor’s Hornpipe — PySynth S

Bach: Jesu, Joy of Man’s Desiring — PySynth S (treble) and B (bass)

Also check my recent post: Play the piano on your computer with Python.

If you want to try it out, note that the program has a couple of bugs, related to the frequencies of notes, since I am not musically trained; I just wrote it for fun and as an experiment. Though some of the post comments gave some background and suggested corrections, they did not seem to be sure, and corrected themselves, so I have not implemented any of those suggestions yet.

Enjoy.

- Vasudev Ram - Dancing Bison Enterprises

Share |

Copperhead: Data Parallel Python from NVIDIA Research

2013-04-26T02:03:00.000+05:30

By Vasudev Ram

News seen via PythonWeekly.

Copperhead, from NVIDIA Research, is a project to bring data parallelism to Python.

From the site (emphasis mine):

[ We define a small functional, data parallel subset of Python, which we then dynamically compile and execute on parallel platforms. Currently, we target NVIDIA GPUs, as well as multicore CPUs through OpenMP and Threading Building Blocks (TBB). ]

Soumds interesting.

Also see my recent related post:

Python-to-GPU compiler coming from Continuum Analytics and Nvidia

About GPUs and GPGPUs.

- Vasudev Ram - Dancing Bison Enterprises

Share |

Plot me some randomness

2013-04-24T01:34:00.001+05:30

By Vasudev Ram

In this recent post about plot.ly,

plot.ly, fast plotting in the browser,

I mentioned that plotting by pasting data worked, but plotting using Python had an issue, and that they said they were fixing it.

The plot.ly people got back to me saying that the Python issue is fixed.

I tried it out again with a couple of simple plotting examples involving random numbers. They worked.

Here are the examples:

from numpy import *
from random import randint

x = range(1, 10)
y = [ random.randint(1, 10) for i in x ]
plot(x, y)

from numpy import *
from random import randint

x = range(1, 20)
y = [ random.randint(1, 10) for i in x ]
plot(x, y)

(The only difference in the above two scripts is 10 vs 20.)

You can try out that code, or your own, in plot.ly.

Go to https://plot.ly/plot.

Click on the "tab" labeled "+" near the top left of the screen, then click "Script" from the drop-down menu.

Enter your script (no need to Save) and then click Run.

If your code has no errors, your plot will appear in a new tab (not a new browser tab, but a new "tab" within the plot.ly browser tab).

The plot may take 2 to 3 seconds to appear.

plot.ly is beta software, but looks useful. They have many more complex examples in their demos.

- Vasudev Ram - Dancing Bison Enterprises

Share |

pypng, pure Python module to encode/decode PNG

2013-04-22T05:28:00.001+05:30

By Vasudev Ram

pypng is a pure Python library to read/write PNG images.

Here are some code examples of using pypng. Note: the actual library is called png.py, not pypng.py. So import that, not this :-)

I tried out a few of the examples, and they worked.

The image above uses the transparency feature of PNG.

- Vasudev Ram - Dancing Bison Enterprises

Share |

Video: Neil Diamond - They're coming to America

2013-04-21T03:55:00.001+05:30

By Vasudev Ram

- Vasudev Ram - Dancing Bison Enterprises

Share |

Samuru, a new (and smarter?) search engine

2013-04-20T20:10:00.001+05:30

By Vasudev Ram - Dancing Bison Enterprises

Samuru is a new search engine from stremor.com

I tried it out a bit and it gave me some interesting results.

About stremor

About Samuru:

Update in a while, I'm on my mobile at present.

Hacker News thread about Samuru is interesting

- Vasudev Ram

With pleasure develop - JetBrains PyCharm blog

2013-04-20T08:52:00.001+05:30

PyCharm 2.7.2 released | JetBrains PyCharm Blog

you.get(it) ? :-)

sqlite3dbm, an SQLite-backed dbm module

2013-04-19T02:51:00.001+05:30

By Vasudev Ram

Saw this today. It seems to be on the Github account of Yelp.com.

They created it as a tool to help with Hadoop work on Amazon EMR (Elastic Map Reduce).

sqlite3dbm provides a SQLite-backed dictionary conforming to the dbm interface, along with a shelve class that wraps the dict and provides serialization for it.

I tried it out, and it worked as advertised.

How to use sqlite3dbm:

Import the module, use its open() method to create an SQLite database, getting back a handle to it, let's call it "db", then use Python dict syntax on db to store data.

Then, either in the same or another program later, you can again fetch and/or modify that data, with dict syntax.

Interesting idea. dbm modules, which implement key-value stores, are less powerful than relational databases (SQL), and were probably developed earlier (think ISAM, etc.), so it looks a bit backwards to implement a dbm-type store on top of SQLite. But the sqlite3dbm project page gives at least some justification for that:

[ This module was born to provide random-access extra data for Hadoop jobs on Amazon’s Elastic Map Reduce (EMR) cluster. We used to use bsddb for this because of its dead-simple dict interface. Unfortunately, bsddb is deprecated for removal from the standard library and also has inter-version compatability problems that make it not work on EMR. sqlite3 is the obvious alternative for a persistent store, but its powerful SQL interface can be too complex when you just want a dict. Thus, sqlite3dbm was born to provide a simple dictionary API on top of the ubiquitous and easily available sqlite3.

This module requres no setup or configuration once installed. Its goal is a stupid-simple solution whenever a persistent dictionary is desired. ]

- Vasudev Ram - Dancing Bison Enterprises

Share |

Now, Google+ comments on your Blogger blog

2013-04-19T00:45:00.000+05:30

By Vasudev Ram

Just saw the news via an email from Google.

Bringing Google+ Comments to Blogger

Looks like an interesting idea.

Some of the benefits mentioned for this feature:

- View your blog and Google+ comments, all in one place

- Help readers comment and connect with their circles

I enabled the feature for my blog - this one, jugad2.blogspot.com .

Let's see how it works out. There may be a few issues initially, but over time those should get fixed.

- Vasudev Ram - Dancing Bison Enterprises

Share |

Common DB developer mistake? Using natural instead of surrogate keys

2013-04-17T03:13:00.002+05:30

By Vasudev Ram

Just saw this StackOverflow article:

Database development mistakes made by application developers [closed]

via this Twitter conversation between me and @coding_insights.

The article had this as one of the answers:

"3. Using natural rather than surrogate (technical) primary keys"

followed by an explanation of why that person thinks so.

I tend to agree with that comment, based on my own experience working on many database-backed projects.

Though it seems counter-intuitive, using surrogate keys seems to be a better and more robust approach.

The commenter gave many reasons for using surrogate keys instead of natural keys, (and as I said, I've experienced at least some of those reasons), so I won't repeat them here; go take a look at that article.

- Vasudev Ram - Dancing Bison Enterprises

Share |

"Free is only a tactic, though, not a business model." - GigaOm

2013-04-17T01:39:00.000+05:30

By Vasudev Ram

I saw an interesting post by Fred Wilson:

Tenacity And Persistence Pays Off.

Fred's post also had a comment by Philip Sugar, to which I replied, asking for more info, and he replied, with a link in his comment to this post: The Penny Gap. That post had an interesting quote from an article on GigaOm:

[ To be fair to these VCs, they’re not advocating doing everything without pay. They’re suggesting free as a tactic towards getting paid in other ways: through advertising, or by premium services (as in a freemium model), or maybe even through being acquired by a company with a large wallet. Free is only a tactic, though, not a business model.

Conflating the two misleads web application developers into thinking they don’t need to do the hard work of figuring out what’s really of value to users before they build and launch their online service. ]

The GigaOm post is here: Free: a Tactic, not a Business Model

- Vasudev Ram - Dancing Bison Enterprises

Share |

The Python Brochure Project - preview available

2013-04-15T04:17:00.003+05:30

By Vasudev Ram

The Python Brochure Project, an initiative by the Python Software Foundation, has released a preview of the brochure. The purpose of the Python Brochure is to make the Python language and its benefits and uses more widely known, among other things.

The contents of the Python Brochure are pretty interesting, IMO, and show how widely and diversely Python is being used worldwide by companies, governments and other organizations.

Python may even go to Mars in a few years! Not kidding - see the brochure.

Excerpt from the site:

[ The brochure showcases Python projects with great impact, fascinating aspect and interesting twist from all application spaces covering use cases in business, science, industry, education, media, government, public sector and charity. ]

You can get the Python Brochure preview (as a PDF file). Scroll a bit down the page if you need to (depending on your monitor size), to see the download link.

I just downloaded it and scanned through it, and it looks quite good.

A minor nitpick is that I wish that they would have made it a single-column PDF, but that is my personal preference.

Update: Just realized that the two-column format is more suitable for printing (if the columns come on opposing pages), and brochures are printed ... Still wish there was a single-column version for computer viewing, though.

- Vasudev Ram - Dancing Bison Enterprises

Share |

Play the piano on your computer with Python

2013-04-14T03:51:00.001+05:30

By Vasudev Ram

This program is meant for kids of all ages ...

The program, piano.py (shown below), lets you play music on your computer, something like a piano, by pressing the keys C D E F G A B, as in Western classical music.

I've written various versions of such a program in the past, starting from my first home computer, in different languages (BASIC, Turbo Pascal, Turbo C), for fun.

The program works on Windows and uses the Python winsound module. I got to know about the winsound module a while ago, and tried it then, and wrote a program like this.

To run this program, you need Windows and Python, and a working sound system; built-in speakers on a PC or laptop will work, but you will get better sound quality from external speakers.

The program is rudimentary and is not meant to simulate a full piano; that would be much more complex.

Here is the piano program in Python:

# piano.py

# Description: A simple program to make the computer act like 
# a piano. Plays a note for each of the keys C, D, E, F, G, A, B,
# according to the Western music middle octave.

# References:
# http://en.wikipedia.org/wiki/Frequency_of_notes

# Author: Vasudev Ram - http://www.dancingbison.com

import string
from time import sleep
from winsound import Beep
from msvcrt import getch

def play_note(frequency, duration):
    Beep(frequency, duration)

"""
Compute all the frequencies for the notes C D E F G A B,
for the middle C octave, where note A = 400 Hz.

Middle octave note frequencies computed from A = 440 Hz,
using the info at http://en.wikipedia.org/wiki/Frequency_of_notes
(Multiply a note frequency by the square of the twelfth root
of 2 to get the next higher note, and divide it by the same
to get the next lower note).
"""

a = 440
# twelfth_root_2 is the 12th root of 2
twelfth_root_2 = 2 ** (1.0/12.0)
# tr2 is a shorter name for twelfth_root_2
tr2 = twelfth_root_2
# tr2sqrd is tr2 squared
# tr2sqrd is the factor by which to multiply (or divide) the 
# frequency of any note, to get the next higher (or lower) note
# in the same octave.
# E.g. multiplying the frequency for note A in the middle C octave,
# i.e. 440 Hz, by tr2sqrd, gives the frequency for the next higher
# note in the same octave, i.e. note B.
tr2sqrd = tr2 * tr2

b = int(round(a * tr2sqrd))

# C D E F G A B

g = int(round(a / tr2sqrd))
f = int(round(g / tr2sqrd))
e = int(round(f / tr2sqrd))
d = int(round(e / tr2sqrd))
c = int(round(d / tr2sqrd))

"""
print "c =", c, "d =", d, "e =", e, "f =", f
print "g =", g, "a =", a, "b =", b
"""

# Set up the notes dict mapping letter keys to notes to play
notes = {}
notes['c'] = c
notes['d'] = d
notes['e'] = e
notes['f'] = f
notes['g'] = g
notes['a'] = a
notes['b'] = b

print
print "Simple piano program in Python, by Vasudev Ram"
print
print "To play, keep on pressing any of these keys: c d e f g a b"
print "Make sure the Caps Lock light on your keyboard is off"
print
print "Press q to stop"
print
print
print "             The floor is yours, Maestro!"
print chr(7) # bell
print

duration = 300 # milliseconds
note = string.lower(getch())

while True:
    print note,
    if note in notes:
        play_note(notes[note], duration)
    elif note == 'q':
        break
    note = string.lower(getch())

print
print

# And farewell tune ...

sleep(1.5)
for note in ('c', 'c', 'd', 'd', 'g', 'a', 'b'):
    play_note(notes[note], duration)
    sleep(0.1)

sleep(1.5)

for note in ('d', 'd', 'c', 'c', 'b', 'a', 'g'):
    play_note(notes[note], duration)
    sleep(0.1)
print
print "Encore!"
print
#--------------- EOF: piano.py ----------

You can run the piano program with the command:

python piano.py

It gives instructions on how to use it.

It's easy - just keep pressing any of the keys c d e f g a b, or q to quit.

Enjoy.

Note 1: I computed the frequencies for the notes (C D E F G A B) from the Wikipedia link (in the code above), but there are multiple notes with almost the same name (even in the same octave), and I don't have much knowledge of music, so sorry if some of the notes are off key. Feel free to give suggestions in the comments, and I'll try to implement them.

Note 2: I just whipped up the code quickly, it can be improved, I may do that later.

Also, if you are into Python and music, check this post:

Pyknon music library for Python (MIDI)

I also came across a couple of other music-related Python modules, but that is a subject for another post.

- Vasudev Ram - Dancing Bison Enterprises

Share |

The pip search command (Python)

2013-04-13T01:58:00.002+05:30

By Vasudev Ram

I was checking out the help for Python's pip tool (an installation tool for Python packages), and came across its search option.

The search option of the pip command lets you search PyPI, the Python Package Index.

I tried it out with a couple of searches, for the terms bottle (to get info on more PyPI packages related to 1) the Bottle Python web microframework (which I had blogged about a while ago), and 2) PDF, which is one of my areas of interest.

Here are the results of those two pip searches of PyPI:

c:\Python27\Lib\site-packages>pip -v -v -v search bottle
bottle                    - Fast and simple WSGI-framework for small web-
                            applications.
  INSTALLED: 0.11.6 (latest)
bottlenose                - A Python hook into the Amazon.com Product
                            Advertising API
Bottleneck                - Fast NumPy array functions written in Cython
mimerender                - RESTful HTTP Content Negotiation for Flask,
                            Bottle, web.py and webapp2 (Google App Engine)
bottle-cork               - Authentication/Authorization library for Bottle
pasttle                   - Simple pastebin on top of bottle.
bottle-websocket          - WebSockets for bottle
bottle-hotqueue           - FIFO Queue for Bottle built upon HotQueue
Bottle-DebugToolbar       - A port of the Django Debug Toolbar to Bottle
bottle-haml               - UNKNOWN
macaron                   - Simple object-relational mapper for SQLite3,
                            includes plugin for Bottle web framework
bottle-servefiles         - A reusable app that serves static files for bottle
                            apps
bottle-sqlalchemy         - SQLAlchemy integration for Bottle.
bottle-mongodb            - MongoDB integration for Bottle
flaskle                   - bottle-like utility decorators for flask
bottle-redis              - Redis integration for Bottle.
bottle-pystache           - Bottle Pystache template wrappers
bottle-tornado-websocket  - WebSockets for bottle
BottleRack                - BottleRack Markdown HTML Server
Gluino                    - port of web2py libs to bottle, flask, pyramid,
                            tornado (includes copy of modules from the web2py
                            framework)
bottle-request            - Plugin to give bottle a 'stateless' request object
bottle-sqlite             - SQLite3 integration for Bottle.
bottle-renderer           - Renderer plugin for bottle
bottle-memcache           - Memcache integration for Bottle.
bottle-web2pydal          - Web2py Dal integration for Bottle.
Bottle-SSLify             - Force SSL on any Bottle app.
bottle-werkzeug           - Werkzeug integration for Bottle.
bottle-pycassa            - Bottle plugin for Cassandra/Pycassa
bottle_cql                - CQL integration for Bottle.
bottle-extras             - Meta package to install the bottle plugin
                            collection.
bn                        - Lightweight profiling tool to detect performance
                            BottleNecks in Python code.
bottle-agamemnon          - Agamemnon integration for bottle
bottle-flash              - flash plugin for bottle
bottle-tornadosocket      - WebSockets for bottle
bottle-mysql              - MySQL integration for Bottle.
bottle-pgsql              - PgSQL integration for Bottle

c:\Python27\Lib\site-packages>
c:\Python27\Lib\site-packages>pip -v -v -v search pdf
mwlib.rl                  - generate pdfs from mediawiki markup
zopyx.convert             - A Python interface to XSL-FO libraries (Conversion
                            HTML to PDF, RTF, DOCX, WML and ODT)
slc.publications          - A content type to store and parse pdf publications
pdfminer                  - PDF parser and analyzer
zopyx.convert2            - A Python interface for the conversion of HTML to
                            PDF, RTF, DOCX, WML and ODT) - belongs to
                            zopyx.smartprintng.core
pisa                      - PDF generator using HTML and CSS
collective.pdfpeek        - A Plone 4 product that generates image thumbnail
                            previews of PDF files stored on ATFile based
                            objects.
WeasyPrint                - WeasyPrint converts web documents to PDF.
rst2pdf                   - Convert restructured text to PDF via reportlab.
pdfserenitynow            - Create TIFs and JPGs from crappy PDFs
collective.sendaspdf      - An open source product for Plone to download or
                            email a page seen by the user as a PDF file.
Products.SmartPrintNG     - Produce & Publish for Plone - Conversion of Plone
                            content to PDF, RTS, ODT, DOCX and WML
wc.pageturner             - A Plone product that provides the PDF viewer
                            FlexPaper.
PyX                       - Python package for the generation of PostScript
                            and PDF files
PollyReports              - Band-oriented PDF report generation from database
                            query
relatorio                 - A templating library able to output odt and pdf
                            files
pyPdf                     - PDF toolkit
wildcard.pdfpal           - PDF Thumbnail generation, OCR indexing and extra
                            views integrated with plone.app.async
pdfminer3k                - PDF parser and analyzer
pdfparanoia               - pdf watermark remover library for academic papers
template2pdf              - Renders Django/Jinja2 templates into PDF.
ftw.book                  - Produce books with Plone and export them in a high
                            quality PDF.
collective.pdfjs          - pdf.js integration for Plone
ftw.pdfgenerator          - A library for generating PDF representations of
                            Plone objects with LaTeX.
django-wkhtmltopdf        - Converts html to PDF using
                            http://code.google.com/p/wkhtmltopdf/.
TableFactory              - Easily create HTML, spreadsheet, or PDF tables
                            from common Python data sources
paPyro                    - A PDF report generator written in Python
pdfnup                    - Layout multiple pages per sheet of a PDF document.
pdfdocument               - Wrapper for ReportLab which allows easy creation
                            of PDF documents.
xhtml2pdf                 - PDF generator using HTML and CSS
ServPDF                   - ServPDF is a webbased Microsoft Office to PDF
                            Converter
pdfcrowd                  - A client for Pdfcrowd API.
pdfrecycle                - create a PDF file by composing pages from other
                            PDF files.
metapdf                   - A lightweight PDF library optimized for metadata
                            extraction and insertion
tex                       - Convert LaTeX or TeX source to PDF or DVI, and
                            escape strings for LaTeX.
pdfkit                    - Wkhtmltopdf python wrapper to convert html to pdf
                            using the webkit rendering engine and qt
eea.converter             - SVG, PNG, PDF converters using external tools as
                            ImageMagick
pieberry-library-assistant - A program to download pdf documents from public
                            websites, and catalogue them in BibTeX format
InvoiceGenerator          - Library to generate PDF invoice.
pdfserver                 - Pdfserver is a webservice that offers common PDF
                            operations like joining documents, selecting pages
                            or "n pages on one".
rputils                   - An application for dice rolling and reading PDF
                            files.
diffpy.pdfgui             - GUI for PDF simulation and structure refinement.
pyf.components.consumers.rmlpdfwriter - PyF component RML-Based PDF Writer based
 on
                            Z3C.RML and Genshi
django-xhtml2pdf          - A Django app to generate pdfs from templates
aws.pdfbook               - Download Plone content views as PDF
pdfquery                  - Concise and friendly PDF scraper using JQuery or
                            XPath selectors.
OPAF                      - Open PDF Analysis Framework
TecUtils                  - Various utilities for database and config files
                            use. Text to Pdf converter
Products.PDFtoOCR         - PDFtoOCR does OCR processing on PDF documents. The
                            text from OCR is used in the search results.
Flask-WeasyPrint          - Make PDF in your Flask app with WeasyPrint.
pdfmerge                  - Command-line PDF utility.
ftwbook.graphicblock      - Addon for `ftw.book` providing a graphics block
                            for including PDF documents in the book.
pyf.components.consumers.ooowriter - py3o-powered OpenOffice.org ODT Writer for
PyF
                            Framework with support for rendering (pdf, html,
                            doc, docx, etc) on py3o renderservers.
pyjon.reports             - Pyjon.Reports is a module bridging z3c.rml, genshi
                            and pypdf together to provide a simple mean of
                            creating templated pdf documents in python.
django-pdf                - A Django app for managing and processing PDF
                            documents.
gametex-django-print      - Generate PDFs from GameTeX in Django
collective.pdftransform   - A set of portal transform to change pdf into
                            images
pdftools.pdfposter        - Scale and tile PDF images/pages to print on
                            multiple pages.
django-webodt             - ODF template handler and odt to html, pdf, doc,
                            etc converter
slate                     - Extract text from PDF documents easily.
dinbrief                  - PDF renderer for DIN 5008 and DIN 676 compliant
                            letters and invoices
raptus.princexml          - Provides a simple bridge to create PDFs from HTML
                            views using PrinceXML
wallaby-plugin-pdfgenerator - This package provides a PDF generator for wallaby.

z3c.pdftemplate           - PDF Template
wkhtmltopdf               - Simple python wrapper for wkhtmltopdf
diffpy.pdffit2            - PDFfit2 - real space structure refinement program.
pdfid_PL                  - A Python module to analyze and sanitize PDF files,
                            based on Didier Stevens' PDFiD
pypdflib                  - Pango Cairo based Python PDF Library
pdf2zip                   - pdf conversion utility
pyf.components.consumers.xhtmlpdfwriter - XHTML-Based PDF Writer for PyF Framewo
rk based on
                            PISA and Genshi
pdf-link-checker          - Reports broken hyperlinks in PDF documents
scrape-highlighted        - A script to scrape highlighted text from a pdf
                            [Mac only].
pdfgrid                   - Add a grid on top of all pages of a PDF document.
jag-tipdf                 - Combines plain text and images into a single PDF.
HWFormatter               - Format submitted homework assignments from
                            Blackboard into PDFs
collective.pdfLeadImage   - Automatically creates contentleadimage from pdf
                            cover
pdfcat                    - pdf concatenation tool
prynt                     - Generating HTML/PDF directly from your console.
pdfsplit                  - Split a PDF file or rearrange its pages into a new
                            PDF file.
pdfrw                     - Pure Python PDF file reader/writer library
agenda2pdf                - Simple script which generates a book agenda file
                            in PDF format, ready to be printed or to be loaded
                            on a ebook reader
stapler                   - Manipulate PDF documents from the command line
trml2pdf                  - Tiny RML2PDF is a tool to easily create PDF
                            document without programming. It       can be used
                            as a Python library or as a standalone binary. It
                            converts a RML,       an XML dialect that lets you
                            define the precise appearance of a printed
                            document, to a PDF. You can use your existing
                            tools to generate an input file       that exactly
                            describes the layout of a printed document, and
                            RML2PDF converts       it into PDF. RML is a much
                            more powerfull and flexible alternative to XSL:FO.
                            The executable read a RML file to the standard
                            input and output a PDF file to       the standard
                            output.
wsgitrml2pdf              - wsgitrml2pdf is wsgi middleware to convert trml
                            text to pdf.
fpdf.py                   - Simple PDF generation for Python (FPDF PHP port)
DXF-Converter             - Converts Autocad Files v11 and v12 to pdf
JagPDF                    - A library for generating PDF documents.
pdftools.pdfjoin          - Join PDF documents into a single document.
collective.calameo        - Publish your PDF document in Plone with Calameo
PyPDF2                    - PDF toolkit
PDFTron PDFNet SDK for Python - A top notch PDF library for PDF rendering,
                            conversion, content extraction, etc
cubicweb-pdfexport        - export any page as a pdf document
fpdf                      - Simple PDF generation for Python
pywkher                   - wkhtmltopdf for Python on Heroku
PyBooklet                 - Converts PDFs to booklets
pyFPDF                    - generate pdf on python
origapy                   - A Python module to clean PDF files by disabling
                            active content (javascript, launch, etc), using
                            the Ruby Origami PDF parser.
pdftable                  - pdftable: extract tables from PDF files
django-latex              - Django application to generate latex/pdf files.
ttfpdf                    - TrueType support PDF file Generator
pdf2tiff                  - A PDF to TIFF converter for Mac OS X.
cloudooo.handler.pdf      - Python Package to handler PDF documents
pdfbox                    - JCC wrapper for Apache PDFBox
buzzweb2pdf               - An Open Source tool to convert HTML documentation
                            with an index page into a single PDF.
ServPDF and ServPDF-OO    - Web based Office to PDF Converter Server for
                            Microsoft Office and OpenOffice
genenga                   - Generate Nengajo(Japanese new year card) pdf from
                            address list.

As you can see from the output above, a lot of Python packages were found for those two search terms. In the search for the term "bottle", not all of the packages found were actually related to the Bottle framework, but most were.

The search for "PDF" resulted in many results, some of which I've blogged about in the past, such as pisa/xhtmltopdf, PyPDF, pdfdocument, wkhtml2pdf, and others, and many that I didn't know of before.

So it looks like the pip search command is a useful tool.

As for why I used "pip -v -v -v", see "pip --help" :-)

- Vasudev Ram - Dancing Bison Enterprises

Share |

Using xtopdf and pypyodbc to publish MS Access database data to PDF

2013-04-10T17:27:00.003+05:30

By Vasudev Ram

I had blogged about pypyodbc, a pure-Python ODBC library, recently.

Using pypyodbc with my xtopdf toolkit for PDF creation, you can publish your MS Access database data to PDF.

Here is some example code to publish MS Access data to PDF:

First, the program create_ppo_mdb.py, shown below, creates an MS Access database called fruits.mdb, then creates a table called fruits in it, and inserts 3 records into the table:

# create_ppo_mdb.py

import pypyodbc 
             
pypyodbc.win_create_mdb('.\\fruits.mdb')
connection_string = 'Driver={Microsoft Access Driver (*.mdb)};DBQ=.\\fruits.mdb'
connection = pypyodbc.connect(connection_string)

SQL = 'CREATE TABLE fruits (id COUNTER PRIMARY KEY, fruit_name VARCHAR(25));'
connection.cursor().execute(SQL).commit()

SQL = "INSERT INTO fruits values (1, 'apple');"
connection.cursor().execute(SQL).commit()

SQL = "INSERT INTO fruits values (2, 'banana');"
connection.cursor().execute(SQL).commit()

SQL = "INSERT INTO fruits values (3, 'orange');"
connection.cursor().execute(SQL).commit()

# Uncomment the 5 lines below make the program also display the data after creating it.

#SQL = 'SELECT * FROM fruits;'
#cursor = connection.cursor().execute(SQL)
#for row in cursor:
#    for col in row:
#        print col,
#    print

cursor.close()
connection.close()

Next, the program MDBtoPDF.py, shown below, reads the data from the fruits table in the MDB database just created above, and publishes the selected records to PDF:

#-------------------------------------------------------------------

# MDBtoPDF.py
# Description: A program to convert MS Access .MDB data to PDF format.
# Author: Vasudev Ram - http://www.dancingbison.com

#-------------------------------------------------------------------

# imports

import sys 
import os
import time
import string
import pypyodbc 
from PDFWriter import PDFWriter
             
#-------------------------------------------------------------------

# globals

##------------------------ usage ---------------------------------------

def usage():

 sys.stderr.write("Usage: python " + sys.argv[0] + " MDB_DSN table_name pdf_file\n")
 sys.stderr.write("where MDB_DSN is the ODBC DSN (Data Source Name) for the\n")
 sys.stderr.write("MDB file, table_name is the name of the table in that MDB,\n")
 sys.stderr.write("whose data you want to convert to PDF, and pdf_file is the\n")
 sys.stderr.write("output PDF filename.\n")
 sys.stderr.write(sys.argv[0] + " reads the table data from the MDB and\n")
 sys.stderr.write("writes it to pdf_file.\n")

##------------------------ main ------------------------------------------

def main():

 '''Main program to convert MDB data to PDF.
 '''

 # check for right num. of args
 if (len(sys.argv) != 4):
  usage()
  sys.exit(1)

 # extract MDB DSN, table name and pdf filename from args
 mdb_dsn = sys.argv[1]
 table_name = sys.argv[2]
 pdf_fn = sys.argv[3]

 print "mdb_dsn =", mdb_dsn
 print "table_name =", table_name
 print "pdf_fn =", pdf_fn

    # build connection string
 connection_string_prefix = 'Driver={Microsoft Access Driver (*.mdb)};DBQ='
 connection_string = connection_string_prefix + mdb_dsn
 print "connection_string =", connection_string
 connection = pypyodbc.connect(connection_string)
 print "connection =", connection

 # create the PDFWriter instance
 pw = PDFWriter(pdf_fn)

 # and set some of its fields

 # set the font
 pw.setFont("Courier", 10)

 # set the page header
 gen_datetime = time.asctime()
 pw.setHeader("Generated by MDBtoPDF: Input: " + mdb_dsn + \
 " At: " + gen_datetime)

 # set the page footer
 pw.setFooter("Generated by MDBtoPDF: Input: " + mdb_dsn + \
 " At: " + gen_datetime)

 # create the separator for logical grouping of output
 sep = "=" * 60

 # print the data records section title
 pw.writeLine("MDB Data Records from MDB: %s, table: %s" % (mdb_dsn, 
  table_name))

 # print a separator line
 pw.writeLine(sep)

 # read the input MDB data and write it to the PDF file

 SQL = 'SELECT * FROM fruits;'

 cursor = connection.cursor().execute(SQL)
 for row in cursor:
  str_row = ""
  for col in row:
   str_row = str_row + str(col) + " "
  pw.writeLine(str_row)

 # close the cursor and connection
 cursor.close()
 connection.close()

 # print a separator line
 pw.writeLine(sep)

 # save current page
 pw.savePage()

 # close the PDFWriter
 pw.close()

##------------------------ Global code -----------------------------------

# invoke main

if __name__ == '__main__':
 main()

##------------------------ EOF - MDBto_PDF.py ---------------

To make the above programs work, you need to have the Reportlab toolkit v1.21 and the xtopdf toolkit installed, in addition to pypyodbc and Python 2.7. (Click on the "Branches" tab on the xtopdf page linked in the previous sentence to download xtopdf.)

I've had an interest in ODBC ever since I first worked, as team leader, on a middleware software product that used ODBC. The middleware was developed at Infosys Technologies, where I worked at the time.

Though ODBC itself had a good architecture, many driver implementations of the time (this was some years ago) were rather slow, so one of the main goals of the product was to improve the performance of client-server or desktop applications (written in Visual Basic or C) that used ODBC for database access.

I remember learning ODBC as part of the project (and teaching it to the team), and reading most of the book "Inside ODBC" by Kyle Geiger, one of the architects of ODBC - it was a fascinating book, that gave a detailed look inside the architecture of ODBC, the reasons for certain design decisions that were made, and so on.

We succeeded in meeting all the goals of the project, and that middleware product was used in many large client-server applications (using VB and Oracle / Sybase) that were developed by Infosys for its clients. I really had a lot of fun working on that project.

Related links:

ODBC entry on Wikipedia

Inside ODBC - the book, on Amazon

eGenix mxODBC Connect, from eGenix, a German Python products company.

eGenix mxODBC

unixODBC

DataDirect ODBC

iODBC

The Microsoft SQL Server ODBC Driver for Linux - it provides native connectivity from Linux to Microsoft SQL Server. (Seems to be 64-bit only).

- Vasudev Ram - Dancing Bison Enterprises

Share |

It takes only eight hours to learn to code

2013-04-10T03:52:00.000+05:30

By Vasudev Ram

It takes only eight hours to learn to code.

Also see the parent link to the link above :-)

- Vasudev Ram - Dancing Bison Enterprises

Share |

Using sys._current_frames() and the Python traceback module for debugging

2013-04-10T00:46:00.001+05:30

By Vasudev Ram

Python's sys._current_frames() function/method and the traceback module of Python can be useful for debugging your Python code.

In the example below, I've used py, the Python Launcher for Windows. It comes with Python 3.3. If you're on Python 2, you can download the py launcher for Python 2 here. Use either of the versions (32-bit or 64-bit, as appropriate) called launcher*, not launchwin*, for the commands below.

The example below works on both Python 2 and Python 3.

#--------------------------------------------------------
# test_current_frames.py 

import sys, traceback

def foo():

    for thread, frame in sys._current_frames().items():
        print('Thread 0x%x' % thread)
        traceback.print_stack(frame)

def bar():
    foo()

def baz():
    bar()

baz()
#--------------------------------------------------------

Run the program with any of the following 3 commands:

py test_current_frames.py

or

py -2 test_current_frames.py

or

py -3 test_current_frames.py

You should get output similar to this:

Thread 0x17dc
  File "test_current_frames.py", line 17, in 
    baz()
  File "test_current_frames.py", line 15, in baz
    bar()
  File "test_current_frames.py", line 12, in bar
    foo()
  File "test_current_frames.py", line 9, in foo
    traceback.print_stack(frame)

Also read more about the traceback module on Doug Hellmann's Python Module Of The Week (PyMOTW) site, a.k.a. PyMOTW.

- Vasudev Ram - Dancing Bison Enterprises

Share |

py, the Python Launcher for Windows

2013-04-08T04:54:00.000+05:30

By Vasudev Ram

py, or py.exe, is the name of a new(ish) Python Launcher for Windows, which was released as part of Python 3.x.

This is the PEP for py.

I tried out Py and it worked okay, for some basic uses.

It lets you launch a specific version of Python, either 2, or 3, or 2.x or 3.x. I tried using it to launch both Python 2.7.3 and Python 3.3.1.

Examples:

C:\> py -V
Python 2.7.3
C:\> py -3 -V
Python 3.3.1
c:\Python33\Tools\Scripts> py -2 which.py py.exe
C:\Windows\py.exe
c:\Python33\Tools\Scripts>py -2 which.py cmd.exe
C:\Windows\system32\cmd.exe

Here is the partial output of running the py --help command:

c:\Python33\Tools\Scripts>py --help
Python Launcher for Windows Version 3.3.1150.1013

usage: py [ launcher-arguments ] script [ script-arguments ]

Launcher arguments:

-2     : Launch the latest Python 2.x version
-3     : Launch the latest Python 3.x version
-X.Y   : Launch the specified Python version
-X.Y-32: Launch the specified 32bit Python version

- Vasudev Ram - Dancing Bison Enterprises

Share |

EPD Free - Enthought Python Distribution, with scientific and other packages

2013-04-08T01:03:00.001+05:30

By Vasudev Ram

The Enthought Free Python Distribution (EPD Free), is a free Python distribution from Enthought, which includes additional Python packages / modules for scientific computing and graphing / plotting, such as SciPy, matplotlib and more.

I downloaded and installed it, and tried out some of the matplotlib examples in the Examples directory of the installation.

They worked fine.

Only tried it out a little so far, but I've known about Enthought for some time now, and it looks like a useful Python distribution for the stated purposes. They also have higher versions, presumably for pay, with support, etc.

- Vasudev Ram - Dancing Bison Enterprises

Share |