"complexityfeed" via Abigail in Google Reader

Live streaming of workshop on computational social science

2012-05-29T14:36:02Z

Please note that the workshop on computational social science, mentioned in posts below, will be streamed live starting tomorrow, here. (Alternatively, go here.)

Also, the twitter hashtag for the workshop will be: #compsocsci12.

Registration closing: SPRING WORKSHOP ON COMPUTATIONAL SOCIAL SCIENCE

2012-05-22T23:28:03Z

Note, registration is closing tomorrow.
SPRING WORKSHOP ON COMPUTATIONAL SOCIAL SCIENCE
May 30- June 1, 2012
The Institute for Quantitative Social Science, Harvard University
To register go to https://commerce.cashnet.com/SFCCIS2012 (note registration fee $50/day)
Direct questions to M.Lee@neu.edu
Space is limited!

Sponsored by:
The Lazer Lab
The Northeastern Centers for Computational Social Science and Digital Humanities
The Institute for Quantitative Social Science, Harvard
The Human Dynamics Lab, MIT

AGENDA
May 30
8:30am-9am: Registration
9-10am: Opportunities and challenges in the study of digital traces
David Lazer, welcome and introductory remarks
10am-6pm (with breaks)
Workshop 1: From Tweets to Results: How to obtain, mine, and analyze Twitter data
Derek Ruths (McGill University)
Since Twitter's creation in 2006, it has become one of the most popular microblogging platforms in the world. By virtue of its popularity, the relative structural simplicity of Twitter posts, and a tendency towards relaxed privacy settings, Twitter has also become a popular data source for research on a range of topics in sociology, psychology, political science, and anthropology. Nonetheless, despite its widespread use in the research community, there are many pitfalls when working with Twitter data.
In this day-long workshop, we will lead participants through the entire Twitter-based research pipeline: from obtaining Twitter data all the way through performing some of the sophisticated analyses that have been featured in recent high-profile publications. In the morning, we will cover the nuts and bolts of obtaining and working with a Twitter dataset including: using the Twitter API, the firehose, and rate limits; strategies for storing and filtering Twitter data; and how to publish your dataset for other researchers to use. In the afternoon, we will delve into techniques for analyzing Twitter content including constructing retweet, mention, and follower networks; measuring the sentiment of tweets; and inferring the gender of users from their profiles and unstructured text.
We assume that participants will have little to no prior experience with mining Twitter or other social network datasets. As the workshop will be interactive, participants are encouraged to bring a laptop. Code examples and exercises will be given in Python, thus participants should have some familiarity with the language. However, all concepts and techniques covered will be language-independent, so any individual with some background in scripting or programming will benefit from the workshop.

May 31
9am-5pm (with breaks): Workshop 2: Network Visualization
Yu-Ru Lin (Northeastern/Harvard Universities)
The recent availability of new cutting edge datasets such as open government data, cell phone call records and social media communication streams offers unprecedented opportunities to study human behaviors and their relationship to the social system. Relationships between various types of entities arise naturally in the study of social networks as well as many applications such as information retrieval and business intelligence. The interrelated information can be effectively represented as networks, where nodes are various types of entities and edges are relationships. Network visualization serves as a powerful tool to build intuitions, to systematically explore the structures or peculiar patterns of the data, and to communicate findings.
This tutorial aims to provide practical knowledge on network visualization, using the open source tool Gephi. The tutorial will cover three components:
(1) Understand the visual complexity and an effective way of communicating networked data.
(2) Convey network properties and structure through Gephi's functionality.
(3) Use Gephi's advanced features to explore the networks of political contributions, political texts, etc. The tutorial is intended for scholars and researchers who wish to learn how to incorporate network visualization to speed up the data exploration and to communicate the data insights.
Requirements: Familiarity with basic network concepts is preferred but not essential. Participants should come with their own laptop with Gephi installed (The installation instructions will be given to the participants prior to the tutorial).

June 1
10am-12pm: Self-organized discussions
This will be an opportunity for workshop participants to organize into groups to discuss particular opportunities and challenges in specific substantive domains.
1pm-5pm: Workshop 3: Studying the dynamics of human proximity
Human Dynamics Lab, MIT/ Prof. Alex Pentland, Director.
During the last decade we have developed measurement toolkits based on electronic badges, smart phones, and signal processing that allow us to accurately quantify human behavior in everyday situations on a continuous basis over long time periods. In this tutorial we will describe the sociometric badges and Android platform sociometric software that we have developed, covering their function, capability, and typical use. These tools will be made available to interested participants. We will also cover the mathematical toolkit we have developed, describing the theory, capability, and typical use. These tools will also be made available to participants.
Finally, we will illustrate the use of our sociometric measurement tools together with our mathematical analysis tools on a variety of problems, including individual (e.g., passive screening for health problems), small group (e.g., providing a real-time performance meter for groups), organizations (e.g., reengineering communication patterns for greater productivity), and large-scale sociocultural outcomes (e.g., diabetes risk, crime risk). For additional information see http://hd.media.mit.edu

(Note, a light lunch will be served each day.)

Big data and computational social science, take one

2012-05-22T02:45:01Z

I've been remiss in posting entries this last year, and hope to offer some productive provocations this Spring. I would like to begin with a series of postings on "big data"--or at least big data around human behavior (many/most big data are not about human behavior). Big data has been getting big attention of late. Below is a picture of the trend of searches for big data in Google over the last 8 years.

There has been an unmistakable crystallization in interest (and convergence on this specific semantic construction) over the last 2 years. One of my objectives is to deconstruct the hype and the potential of big data. However, before getting into the scientific issues, I'd like to get into the issues of science. That is, how should the science of big data be organized? There has been recent, and rightful, attention to the issue of data sharing and big data, first in a letter to Nature by Bernardo Huberman, and now in a story in the New York Times. This was an issue we also discussed in a piece by a long list of folks, including myself, in Science on "computational social science" from 2009:

Perhaps the thorniest challenges exist on the data side, with respect to access and privacy. Much of these data are proprietary (e.g., mobile phone and financial transactional information). The debacle following AOL's public release of "anonymized" search records of many of its customers highlights the potential risk to individuals and corporations in the sharing of personal data by private companies (14). Robust models of collaboration and data sharing between industry and academia are needed to facilitate research and safeguard consumer privacy and provide liability protection for corporations. More generally, properly managing privacy issues is essential. As the recent U.S. National Research Council's report on geographical information system data highlights, it is often possible to pull individual profiles out of even carefully anonymized data (15). Last year, the U.S. National Institutes of Health and the Wellcome Trust abruptly removed a number of genetic databases from online access (16). These databases were seemingly anonymized, simply reporting the aggregate frequency of particular genetic markers. However, research revealed the potential for de- anonymization, based on the statistical power of the sheer quantity of data collected from each individual in the database (17).

Because a single dramatic incident involving a breach of privacy could produce rules and statutes that stifle the nascent field of computational social science, a self-regulatory regime of procedures, technologies, and rules is needed that reduces this risk but preserves research potential. As a cornerstone of such a self-regulatory regime, U.S. Institutional Review Boards (IRBs) must increase their technical knowledge to understand the potential for intrusion and individual harm because new possibilities do not fit their current paradigms for harm. Many IRBs would be poorly equipped to evaluate the possibility that complex data could be de-anonymized. Further, it may be necessary for IRBs to oversee the creation of a secure, centralized data infrastructure. Currently, existing data sets are scattered among many groups, with uneven skills and understanding of data security and widely varying protocols. Researchers themselves must develop technologies that protect privacy while preserving data essential for research. These systems, in turn, may prove useful for industry in managing customer privacy and data security.

15. National Research Council, Putting People on the Map: Protecting Confidentiality with Linked Social-Spatial Data, M. P. Gutmann, P. Stern, Eds. (National Academy Press, Washington, DC, 2007).
16. J. Felch. "DNA databases blocked from the public," Los Angeles Times, 29 August 2008, p. A31.
17. N. Homer, S. Szelinger, M. Redman, D. Duggan, W. Tembe, PLoS Genet. 4, e1000167 (2008).

What are the stakes involved in sharing big data? And how should our scientific institutions adapt? These are questions that I will tackle in my next post.

Help get Octave developed for Android! (like MATLAB, but free)

2012-05-13T08:58:52Z

The MATLAB language has become ubiquitous in many fields of applied mathematics such as linear algebra, differential equations, control systems and signal processing among many others. MATLAB is a great tool but it also costs a lot! If you are not a student then MATLAB is a very expensive piece of software. For example, my own academic licensed copy with just 4 toolboxes cost more than the rather high powered laptop I use it on. If I left academia then there would be no chance of me owning a copy unless I found an employer willing to stump up the cash for a commercial license. Commercial licenses cost a LOT more than academic licenses.

Octave – The free alternative

The good news is that there is a free alternative to MATLAB in the form of Octave. Octave attempts to be source compatible with MATLAB which means that, in many cases, your MATLAB code will run as-is on Octave. Many of the undergraduate courses taught at my university (The University of Manchester) could be taught using Octave with little or no modification and I imagine that this would be the case elsewhere. One area where Octave falls down is in the provision of toolboxes but this is improving thanks to the Octave-Forge project.

Addi – The beginnings of MATLAB/Octave on Android

As Dylan said The Times They Are a-Changin’ and there is an ever-increasing segment of world-society that are simply skipping over the PC and going straight to mobile devices for their computing needs. It is possible to get your hands on a functional Android mobile phone or tablet for significantly less than the cost of a PC. These cheap mobile devices may be a lot less powerful than even the cheapest of PCs but they are powerful enough for many purposes and are perfectly capable of outgunning Cray supercomputers from the past.

There is, however, no MATLAB for Android devices. The best we have right now is in the form of Addi, a free Android app that makes use of JMathLib to provide a very scaled-back MATLAB-like experience. Addi is the work of Corbin Champion, an android developer from Portland in the US, and he has much bigger plans for the future.

Full Octave/GNUPlot on Android with no caveats

Corbin is working on a full Octave and GNUPlot* port for Android. He has already included a proof of concept in the latest release of Addi which includes an experimental Octave interpreter. To go from this proof of concept to a fully developed Android port, however, is going to take a lot of work. Corbin is up to the task but he would like our help.

[* - GNUPLot is used as the plotting engine for Octave and includes support for advanced 3D graphics]

Donate as little as $1 to help make this project possible

Corbin has launched a Kickstarter project in order to try to obtain funding for this project. He freely admits that he’ll do the work whether or not it gets funded but will be able to devote much more of his time to the project if the funding request is successful. After all, we all need to eat, even great sotware developers.

Although I have never met him, I believe in Corbin and strongly believe that he will deliver on his promise. So much so that I have pledged $100 to the project out of my own pocket.

If, like me, you want to see a well-developed and supported version of Octave on Android then watch the video below and then head over to Corbin’s kickstarter page to get the full details of his proposal. The minimum donation is only $1 and your money will only be taken if the full funding requirement is met.

Click here to go to the Octave on Android kickstarter page

Update (16th May 2012): The project (and this post) made it to Slashdot

Experience and Good Taste in Software/Systems Design

2012-05-10T11:44:00Z

A guest post by Ian Cottam (@iandavidcottam).

I have been a programmer for 40 years this month. I thought I would write a short essay on things I experienced over that time that went into the design of a relatively recent, small program: DropAndCompute. (The purpose and general evolution of that program were described in a blog entry here. Please just watch the video there if you are new to DropAndCompute.)

Once I had the idea for DropAndCompute –inspired equally by the complexity of Condor and the simplicity of Dropbox — I coded and tested it in about two days. (Typically, one bug lasted until it had a real user, but it was no big deal.) My colleague, Mark Whidby, later re-factored/re-coded it to better scale as it grew in popularity here at The University of Manchester. I expect Mark spent about two days on it too. The user interface and basic design did not change. (As the evolution blog discusses, we eventually made the use of Dropbox optional, but that does not detract from this tale.)

Physically dropping programs and their data:
In the early to mid 1970s as well as doing my own programming work I helped scientists in Liverpool to run their code. One approach we used to make them faster was to drop the deck of cards into the input well of a card reader which was remotely connected to the regional supercomputer centre at Manchester. (I never knew what the communication mechanism was – probably a wet string given the technology of the time.) A nearby line printer was similarly connected and our results could be picked up, usually the next day. DropAndCompute is a 21st century version of this activity, without the leg work and humping of large boxes of cards about.

That this approach was worth the effort was made obvious with one of the first card decks I ever submitted. We had been running the code on an ICL 1903A computer in Liverpool; Manchester had a CDC 6600 (hopefully my memory has not let me down – it did become a CDC 7600 at some stage). Running the code locally in Liverpool, with typical data, took around 55 CPU minutes. Dropping it into that card reader so that it automatically ran in Manchester resulted in the jaw dropping time of 4 CPU seconds. (I still had to wait overnight to pick up the results, something that resonates with today’s DropAndCompute users and Manchester’s Condor Pool, which is only large and powerful overnight.)

Capabilities:
Later, but still in the mid 1970s, I worked for Plessey on their System 250 high-reliability, multiprocessor system. It was the first commercial example of a capability architecture. With such there is no supervisor state or privileged code rings or similar. If you held the capability to do something (e.g. read, write or enter another code context) you could do it. If you didn’t hold the appropriate capability, you could not. The only tiny section of code in the System 250 that was special was where capabilities were generated. No one else could make them.

The server side of DropAndCompute generates capabilities to the user client side. They are implemented as zero length files whose information content is just in their names. For job 3159, you get 3159.kill, 3159.vacate and 3159.debug generated*. By dragging and dropping one or more of these zero length files (capabilities) onto the dropbox the remote lower level Condor command code is automatically executed. [* You could try to make your own capability, such as 9513.kill, but it won't work.]

UNIX and Shell Glue Code:
My initial exposure to the UNIX tools philosophy in the late 1970s profoundly influenced me (and still does). In essence, it says that one should build new software by inventing ‘glue’ to stick existing components together. The UNIX Shell is often the language of choice for this, and was for me. DropAndCompute is a good example of where a little bit of glue produced a hopefully impressive synergy.

The Internet not The Web:
DropAndCompute uses the Internet (clearly). It is not a Web application. I only mention this as some younger programmers, who have grown up with the Web always being there, seem to think the best/only architecture to use for a software system design is one implemented through a web browser using web services. I am grateful to be able to remember pre-Web days, as much as I love what Tim Berners-Lee created for us.

Client-Server:
I’m not sure when I first became aware of client-server architecture. I know a famous computer scientist (the late David Wheeler*) once described it as simply the obvious way to implement software systems. For my part, I’m a believer in the less code the client side (user) needs to install the better (less to go wrong on strange environments one has no control over). In the case of DropAndCompute if the user had Dropbox, it was nothing to install, and just downloading Dropbox if they didn’t.
[* As well as being a co-inventer of the subroutine, David Wheeler led the team that designed the first operational capability-based computer: the Cambridge University CAP.]

Rosetta – Software as Magic:
Around a decade ago I worked for Transitive, a University of Manchester spin-out, and the company that produced Rosetta for Apple. With apologies to Arthur C Clarke: all great software appears to be magic to the user. The simpler the user interface, often the more complex the underlying system is to implement the magic. This is true, for example, for Apple iOS and OS X and for Dropbox (simpler, and yet I would bet that it is internally more complex, than its many competitors). One small part of OS X I helped with is Rosetta (or was, as Apple dropped it from the Lion 10.7 release of OS X). Rosetta dynamically (i.e. on-the-fly) translates PowerPC applications into Intel x86 code. There is no noticeable user interface: you double click your desired application, like any other, and, if needed, Rosetta is invoked to do its job in the background.

I have read many interesting web based discussions about Rosetta, several say, or imply, that it is a relatively simple piece of software: nothing could be further from the truth. It’s likely still a commercial secret how it works, but if it were simple, Apple’s transition to Intel would likely have been a disaster. It took a lot of smart people a long time to make the magic appear that simple.

I tried to keep DropAndCompute’s interface to the user as simple as possible, even where it added some complexity to its implementation. The National Grid Service in the UK did their own version of DropAndCompute, but, for my taste, added too many bells and whistles.

In Conclusion:
I hope this brief essay has been of interest and given some insight into how many years of software/system design experience come to be applied, even to small software systems, both consciously and subconsciously, and always, hopefully, with good taste. Hide complexity, keep it simple for the user, make them think it is simply magic!

Spring workshop on computational social science @ IQSS

2012-05-04T12:00:44Z

SPRING WORKSHOP ON COMPUTATIONAL SOCIAL SCIENCE
May 30- June 1, 2012
The Institute for Quantitative Social Science, Harvard University
Contact m.lee@neu.edu to register (note registration fee $50/day)
Space is limited!

Sponsored by:
The Northeastern Centers for Computational Social Science and Digital Humanities
The Institute for Quantitative Social Science, Harvard
The Human Dynamics Lab, MIT

May 30

8:30am-9am: Registration

9-10am: Opportunities and challenges in the study of digital traces
David Lazer, welcome and introductory remarks

10am-6pm (with breaks)
Workshop 1: From Tweets to Results: How to obtain, mine, and analyze Twitter data
Derek Ruths (McGill University)
Since Twitter's creation in 2006, it has become one of the most popular microblogging platforms in the world. By virtue of its popularity, the relative structural simplicity of Twitter posts, and a tendency towards relaxed privacy settings, Twitter has also become a popular data source for research on a range of topics in sociology, psychology, political science, and anthropology. Nonetheless, despite its widespread use in the research community, there are many pitfalls when working with Twitter data.
In this day-long workshop, we will lead participants through the entire Twitter-based research pipeline: from obtaining Twitter data all the way through performing some of the sophisticated analyses that have been featured in recent high-profile publications. In the morning, we will cover the nuts and bolts of obtaining and working with a Twitter dataset including: using the Twitter API, the firehose, and rate limits; strategies for storing and filtering Twitter data; and how to publish your dataset for other researchers to use. In the afternoon, we will delve into techniques for analyzing Twitter content including constructing retweet, mention, and follower networks; measuring the sentiment of tweets; and inferring the gender of users from their profiles and unstructured text.
We assume that participants will have little to no prior experience with mining Twitter or other social network datasets. As the workshop will be interactive, participants are encouraged to bring a laptop. Code examples and exercises will be given in Python, thus participants should have some familiarity with the language. However, all concepts and techniques covered will be language-independent, so any individual with some background in scripting or programming will benefit from the workshop.

May 31

9am-5pm (with breaks): Workshop 2: Network Visualization
Yu-Ru Lin (Northeastern/Harvard Universities)
The recent availability of new cutting edge datasets such as open government data, cell phone call records and social media communication streams offers unprecedented opportunities to study human behaviors and their relationship to the social system. Relationships between various types of entities arise naturally in the study of social networks as well as many applications such as information retrieval and business intelligence. The interrelated information can be effectively represented as networks, where nodes are various types of entities and edges are relationships. Network visualization serves as a powerful tool to build intuitions, to systematically explore the structures or peculiar patterns of the data, and to communicate findings.
This tutorial aims to provide practical knowledge on network visualization, using the open source tool Gephi. The tutorial will cover three components:
(1) Understand the visual complexity and an effective way of communicating networked data.
(2) Convey network properties and structure through Gephi's functionality.
(3) Use Gephi's advanced features to explore the networks of political contributions, political texts, etc. The tutorial is intended for scholars and researchers who wish to learn how to incorporate network visualization to speed up the data exploration and to communicate the data insights.
Requirements: Familiarity with basic network concepts is preferred but not essential. Participants should come with their own laptop with Gephi installed (The installation instructions will be given to the participants prior to the tutorial).

June 1

10am-12pm: Self-organized discussions
This will be an opportunity for workshop participants to organize into groups to discuss particular opportunities and challenges in specific substantive domains.

1pm-5pm: Workshop 3: Studying the dynamics of human proximity
Human Dynamics Lab, MIT/ Prof. Alex Pentland, Director.
During the last decade we have developed measurement toolkits based on electronic badges, smart phones, and signal processing that allow us to accurately quantify human behavior in everyday situations on a continuous basis over long time periods. In this tutorial we will describe the sociometric badges and Android platform sociometric software that we have developed, covering their function, capability, and typical use. These tools will be made available to interested participants. We will also cover the mathematical toolkit we have developed, describing the theory, capability, and typical use. These tools will also be made available to participants.
Finally, we will illustrate the use of our sociometric measurement tools together with our mathematical analysis tools on a variety of problems, including individual (e.g., passive screening for health problems), small group (e.g., providing a real-time performance meter for groups), organizations (e.g., reengineering communication patterns for greater productivity), and large-scale sociocultural outcomes (e.g., diabetes risk, crime risk). For additional information see http://hd.media.mit.edu

A Month of Math Software – April 2012

2012-04-30T20:18:15Z

Welcome to the 16th Month of Math Software where I take a tour around the news and new releases in the world of mathematical software. Thanks so much to this month’s contributors without whom I would really struggle to put this newsletter together. As always, the archives can be found at http://www.walkingrandomly.com/?cat=47 and if you have any news for next month then there are numerous ways in which to contact me.

Google Summer of Code

The world of Open Source mathematical software stands to gain heavily from this year’s Google Summer of Code. Projects include an attempt to port Scilab to Android, improvements to symbolic expressions in Sage, aerospace and signal processing blocksets for Xcos, Just In Time compilation for Octave and many many more. See http://www.google-melange.com/gsoc/projects/list/google/gsoc2012 for the full list of projects.

Mathematics on GP-GPUs (General Purpose – Graphical Processing Units)

An OpenCL implementation of the GPU linear algebra library, MAGMA, has been released. The practical upshot of this is that you can use GPU cards from manufacturers other than NVIDIA now. Version 0.1 Beta includes implementations of LU, QR, and Cholesky matrix factorisations.
A beta version of AMD’s Accelerated Parallel Processing Math Libraries has been released.Version 1.7 beta includes some new BLAS Level 2 and 3 functions and is required for the MAGMA release mentioned above.
Version 2.1 of AccelerEyes superb Jacket Toolbox has been released (last month in fact but I missed it) and includes support for unconstrained optimisation! The full change-log can be found at wiki.accelereyes.com/wiki/index.php/Release_Notes
GPU Systems have released version 2.0 of their Libra SDK. According to the website ‘Libra is a cross processor, cross platform, cross language – standard math library and runtime API – for software application development.’ It supports both OpenCL and CUDA and multiple languages including C, C++, Java, C# and MATLAB.
Benchmarking the new Kepler GTX680 – AccelerEyes compares NVIDIA’s new hardware with its old. The benchmarks are Matrix-Matrix Multiply, Fast Fourier Transform and sorting.

Python

Version 0.7.3 of Pandas has been released. According to the website, Pandas aims to become the most powerful and flexible open source data analysis / manipulation tool available in any language Go to http://pandas.pydata.org/pandas-docs/dev/whatsnew.html to see what’s new.

Version 0.4 of mathics has been released. ‘Mathics is a free, general-purpose online computer algebra system featuring Mathematica-compatible syntax and functions.’ The Mathics project is looking for more developers, contact Jan Pöschko if you’re interested.

Sparse Linear Systems

PaStiX (http://pastix.gforge.inria.fr) is a scientific library that provides a high performance parallel solver for very large sparse linear systems based on direct methods and version 5.2 was released this month.

MATLAB add-ons

Version 2.1 of LAMG (Lean Algebraic Multigrid) was released this month. LAMG is a fast graph Laplacian solver. It can solve Ax=b in O(m) time and storage, where A is the graph Laplacian of a weighted undirected graph with m edges. Free MATLAB code download: http://code.google.com/p/lamg/
Version 4.2 of Chebfun is now available from http://www.maths.ox.ac.uk/chebfun/. Chebfun is an open-source system written in Matlab that overloads Matlab’s operations for vectors and matrices to analogous operations for functions and operators. For an overview of what Chebfun can do, take a look at the collection of more than 100 Chebfun Examples posted at http://www.maths.ox.ac.uk/chebfun/examples/

LAPACK – The standard for Linear Algebra

LAPACK has seen a bug-fix release. See what’s new in version 3.4.1 at http://www.netlib.org/lapack/lapack-3.4.1.html

SPSS In decline?

According to this blog, SPSS is in steep decline with the open source language, R, taking up the slack. This blog, however, isn’t so sure!

How to unblock .mat files in Outlook

2012-04-28T07:43:59Z

A MATLAB user at Manchester recently sent me some code and data as part of a Condor support request but Outlook helpfully blocked access to his .mat files deeming them potentially unsafe! It seems that Microsoft Outlook is afraid of matrices.

I guess that the easiest way to proceed would be to ask him to resend his stuff as a .zip file but that would slow down the support query. I just wanted to stop Outlook being quite so paranoid and let me have access to those scary matrices.

The fix is a registry edit (very user-friendly of Microsoft right?) so please don’t proceed if you are not comfortable editing the registry. The registry controls numerous aspects of the Windows operating system along with many of the applications you have installed. If it had a map then there would be huge areas labelled ‘Here be Dragons.’ Corrupting the registry could lead to an unusable operating system. So, the following is offered without warranty on a ‘it works for me’ basis.

Open up regedit and navigate to the following (I’m using Windows 7 and Outlook 2007, location and procedure may be different for different environments).

HKEY_CURRENT_USER\Software\Microsoft\Office\12.0\Outlook\Security

Once you have the above highlighted, move your mouse to the right hand pane of regedit, and right click to get a context menu. Click on New->Key and create a key-value pair with

Name Level1Remove
Value Data .mat

If you want to unblock more than just .mat then you can separate them with semicolons so .mat;.exe for example. When you’ve finished, this part of the registry should look like the screenshot below

Intel’s new OpenCL SDK gives access to on-die GPU

2012-04-26T11:04:32Z

Intel have just released their OpenCL Software Development Kit (SDK) for Intel processors. The good news is that this version targets the on-die GPU as well as the CPU allowing truly heterogeneous programming. The bad news is that the GPU goodness is for 3rd Generation ‘Ivy Bridge‘ Processors only– us backward Sandy Bridge users have been left in the cold

A quick scan through the release notes reveals the following:-

OpenCL access to the on-die GPU part is currently for Windows only. Linux users only have CPU support at the moment.
No access to the GPU part of Sandy Bridge Processors via this implementation.
The GPU part has single precision only (I guess we’ll see many more mixed-precision algorithms from now on)

I don’t have access to an Ivy Bridge processor and so can’t have a play but I’m looking forward to seeing how much performance OpenCL programmers can squeeze out of this new implementation.

Other WalkingRandomly posts on GPU computing

How to determine the version of MKL being used by MATLAB

2012-04-25T09:39:14Z

MATLAB uses the Intel Math Kernel Library (MKL) behind the scenes to perform many linear algebra operations. I wondered what version of the MKL was being used by MATLAB 2012a on my 64 bit Windows machine. The most straightforward way to determine this is to simply ask MATLAB:

>> version -lapack
ans =
Intel(R) Math Kernel Library Version 10.3.5 Product Build 20110720 for Intel(R) 64 architecture applications
Linear Algebra PACKage Version 3.3.1

>> version -blas
ans =
Intel(R) Math Kernel Library Version 10.3.5 Product Build 20110720 for Intel(R) 64 architecture applications

Thanks to The Mathworks technical support team for this information.

Mathematics in stained glass

2012-04-13T10:33:51Z

I work in a beautiful old building at The University of Manchester called The Sackville Street Building. Yesterday, I took part in a very interesting historical tour of the building and was astonished at the amount of beautiful things that I had never noticed before and yet walk past every day.

For example, I was delighted to discover that one of the stained glass windows in the great hall was themed around mathematics (thanks to Samantha Bradey for the original image which I’ve heavily compressed for this blog post)

While on the tour I posted a quick snap of the above window to twitter using my mobile phone and received some nice feedback along with a few pictures of other mathematical windows. It turns out that Manchester’s Nick Higham has a close up of the above window for example. Also, Kit Yates tweeted about several mathematical windows in Caius College, Cambridge, one of which is below (showing images related to Venn and Fisher).

Do you know of any other examples of mathematics in stained glass? Feel free to contact me and tell me all about it.

Workaround for a bug in the MATLAB 2012a installer

2012-04-11T16:49:15Z

I recently installed MATLAB 2012a on a Windows machine along with a certain set of standard Mathworks toolboxes. In addition, I also installed the excellent NAG Toolbox for MATLAB which is standard practice at my University.

I later realised that I had not installed all of the Mathworks toolboxes I needed so I fired up the MATLAB installer again and asked it to add the missing toolboxes. This extra installation never completed, however, and gave me the error message

The application encountered an unexpected error and needed to close. You may want to try
re-installing your product(s). More infomation can be found at C:\path_to_a_log_file

I took a look at the log file mentioned which revealed a huge java error that began with

 java.util.concurrent.ExecutionException: java.lang.StringIndexOutOfBoundsException:
    String index out of range: -2
 at java.util.concurrent.FutureTask$Sync.innerGet(Unknown Source)
 at java.util.concurrent.FutureTask.get(Unknown Source)
 at javax.swing.SwingWorker.get(Unknown Source)
 at com.mathworks.wizard.worker.WorkerImpl.done(WorkerImpl.java:33)
 at javax.swing.SwingWorker$5.run(Unknown Source)

A little mucking around revealed that the installer was unhappy with the pathdef.m file at C:\Program Files\MATLAB\R2012a\toolbox\local\pathdef.m

The installer for the NAG Toolbox modifies this file by adding the line

'C:\Program Files\MATLAB\R2012a\toolbox\NAG\mex.w64;' ...

near the beginning and the lines

'C:\Program Files\MATLAB\R2012a\help\toolbox\NAG;' ...
'C:\Program Files\MATLAB\R2012a\help\toolbox\NAGToolboxDemos;' ...

Near the end and it seems that the MATLAB installer really doesn’t like this. So, what you do is create a copy of this pathdef.m file (pathdef.m.old for example) and then remove the non-mathworks lines in pathdef.m. Now you can install the extra Mathworks toolboxes you want. Once the installer has finished its work you can re-add the non-mathworks lines back into pathdef.m using your copy as a guide.

I’ll be informing both NAG and The Mathworks about this particular issue but wanted to get this post out there as soon as possible to provide a workaround since at least one other person has hit this problem at my University and I doubt that he will be the last (It’s also going to make SCCM deployment of MATLAB a pain but that’s another story).

Update

The Mathworks technical support have sent me a better workaround than the one detailed above. What you need to do is to change

'C:\Program Files\MATLAB\R2012a\toolbox\NAG\mex.w64;' ...

'C:\Program Files\MATLAB\R2012a\toolbox\NAG\mex.w64;', ...

The Mathworks installer is unhappy about the missing comma.

A Month of Math Software – March 2012

2012-04-02T10:00:10Z

Welcome to this month’s MMS which includes the usual mix of commercial and open source software spanning across multiple disciplines. Last month’s edition is here and the archive of all previous editions is at http://www.walkingrandomly.com/?cat=47. Thanks so much for all of the contributors this month without whom these articles would be significantly more difficult to write. As always, if you have some mathematical software news then feel free to contact me.

General purpose commercial packages

Maple 16 was released in March and according to Maplesoft it has over 4500 additions and improvements compared to the previous version. Maple is very strong in polynomial arithmetic and Maplesoft have released benchmarks showing how Maple 16 is hundreds of times faster than Mathematica 8 in this area. For example, multiplication of two dense polynomials in 3 variables, each of degree 30 takes 110 seconds in Mathematica 8 but only 0.52 seconds in Maple 16 according to these new benchmarks.
MATLAB version 2012a was also released in March with the usual batch of improvements and updates. One of the highlights for me is the fact that The Mathworks now offer the MATLAB Compiler Runtime for free download, significantly simplifying the deployment of compiled applications.
Version 2.18-5 of Magma, the commerical computer algebra system specializing in number theory, has been released. Click here for the v2.18-5 changelog

Mathematical Open Source and Freeware

The popular Mathcad clone, SMath Studio, has been updated to version 0.93. The list of new things is at http://en.smath.info/forum/default.aspx?g=posts&t=1223
Gnuplot, the venerable plotting package for many operating systems, has been updated to version 4.6. The new stuff is at http://www.gnuplot.info/announce.4.6.0
Gnumeric is a free spreadsheet program and is part of the GNOME Office suite. Version 1..11.2 was released in March – see http://projects.gnome.org/gnumeric/announcements/1.11/gnumeric-1.11.2.shtml for the details.
Euler Math Toolbox has moved from its old sourceforge home to http://euler.rene-grothmann.de. Now at version 14.5, this free MATLAB-like application has lots of nice features; read about all the new ones since version 14 here.

An interactive shell for the GNU Scientific Library

The GNU Scientific Library (GSL) is an open source numerical library for C and C++ programmers that’s been around for many years now and most of you have probably heard of it or used it. Perhaps less well known, however, is the GSL Shell Project (at least I hadn’t heard of it until John Coppola emailed me–thanks to him for the info). In short, this project allows you to use the GSL interactively via the Lua programming language. Some links:-

Python takes on R with pandas

The Python Data Analysis library, pandas, is designed to help Python programmers perform in depth data analysis projects without having to resort to R. Version 0.7.2 was released in March and you can see what’s new at http://pandas.pydata.org/pandas-docs/dev/whatsnew.html. Thanks to Tom Brander for the heads up on this one.

Vital statistics

The open-source statistical juggernaut that is R has been updated to version 2.15.0. R-bloggers.com has got a copy of the release notes.

Finite Differences and Elements

“The Chombo software package provides a set of tools for implementing finite difference methods for the solution of partial differential equations on block-structured adaptively refined rectangular grids with embedded boundaries. Both elliptic and time-dependent modules are included. Support for parallel platforms and standardized self-describing file formats are included.” Version 3.1 was released in March…get it at https://commons.lbl.gov/display/chombo/Chombo+Download+Page
The FEniCS book has been published and is available online and in print. The online version is available at http://dx.doi.org/10.1007/978-3-642-23099-8 and hard copies can be ordered from Springer (http://tinyurl.com/3ueq9hk) or book sellers. The FEniCS Project is a collection of free software with an extensive list of features for automated, efficient solution of differential equations.

Eigenvalues everywhere

The following two packages were recently announced on the Numerical Analysis Digest.

“The FEAST solver package is a free high-performance numerical library for solving the standard or generalized eigenvalue problem, and obtaining all the eigenvalues and eigenvectors within a given search interval. The FEAST algorithm takes its inspiration from the density-matrix representation and contour integration technique in quantum mechanics.” Download version 2.0 from http://www.ecs.umass.edu/~polizzi/feast/download.htm
“ELPA” is a new direct eigensolver library that addresses scalability and performance, especially for parallel applications. ELPA builds on the ScaLAPACK type interfaces that are often used in existing implementations, but key parts of the eigenvalue solution are then replaced with ELPA’s own routines in an easy-to-use way. The library is particularly useful when a substantial part of all eigenvalue / eigenvector pairs is needed. Released under an LGPL-like license, ELPA has been successfully tested for matrix sizes up to 680,000 and with up to 294,000 CPU cores on a BlueGene/P system.” A hybrid version (OpemMP and MPI) was released in March. http://elpa.rzg.mpg.de/

From the blogs

New regression capabilities in MATLAB 2012a
Image Processing and computer vision updates in MATLAB 2012a
Adding functionality to Excel using the NAG Library for .NET
Julia, I Love you – A review of the new numerical programming language, Julia

9 ways to program for Android devices using Android devices

2012-03-26T13:47:12Z

These days almost all of us are carrying around seriously capable little computers in the form of our mobile phones. Although these devices have a similar amount of horsepower to supercomputers of old, most of us only use a fraction of their potential– after all, you don’t need a supercompter to send text messages, look at pictures of cats or throw birds at pigs. I believe that the only way to fully unlock the true potential of these devices is to program them yourself.

From fully fledged applications to little snippets of code, I think that there’s something enormously satisfying about writing your own computer programs and it doesn’t have to be difficult to do so. The following 9 apps will allow you to write programs for your Android mobile phone in a variety of languages including C, BASIC, Lisp and MATLAB m-code using only your Android phone. Although you’ll not be able to use them to write the next 3D blockbuster game, you will be able to solve some interesting problems, learn a trick or two and have a lot of fun.

C4droid – £0.95

With c4droid you get the ability to write, compile and run C and C++ programs using only your Android device. That’s a lot of functionality for only 95p!

Out of the box C4droid only handles C programs, making use of a modified version of the Tiny C Compiler to do the compilation work. The standard C library is provided by uClibc which is specially designed for use on embedded systems.

In order to run C++ programs you need to additionally install the free GCC plugin for C4droid — something that I personally haven’t done yet due to its large size. One of the most common user-complaints appears to be ‘this app doens’t allow me to use iostream.h’ which essentially demonstrates that the installation instructions were not followed. Since iostream.h is a C++ library, you’ll need to install and configure the GCC plugin to get access to it and full instructions on how to do this are given on c4droid’s Google Play page.

You only get access to the standard C library with C4droid which means that you can’t generate graphical output or interact with the phone’s hardware in any way (bluetooth, accelerometers, that sort of thing) but that doesn’t stop this from being an impressive piece of work. Also, for an extra 95p you can run pascal programs using the Pascal plugin for C4droid.

C4droid is a superb app that will be invaluable for anyone learning C,C++ or Pascal or for those of us that simply like to fiddle about with these languages on the go.

C4droid on Google Play
GCC plugin for c4droid (needed for C++ access)
Pascal plugin for C4droid – provides the ability to compile and run Pascal programs

Mintoris Basic – £3.77

At the risk of showing my age, I’ll tell you that I first learned how to program in BASIC (Beginners All-Purpose Symbolic Instruction Code) on the Sinclair ZX Spectrum and so I will always have a fondness for the language. Mintoris Basic is a very fully featured implementation of the BASIC programming language and is significantly more powerful than the implementation I cut my teeth on back in the day.

As well as having all of the stuff you’d expect in a BASIC implementation (loops, strings, variables, functions, decisions, graphics etc), Mintoris also allows you to interact with some of your phone’s hardware including Bluetooth, battery level, GPS, and various sensors. Furthermore, you can attach your programs to shortcuts and launch them from your home screens. The level of functionality is so high that you can write some rather nifty apps with relatively little effort.

Frink – Free

Frink is a great language developed by Alan Eliasen that has been around since 2001. Named after Professor Frink from The Simpsons, Frink runs on almost every device you can possibly imagine and has some very interesting features including interval arithmetic, tracking of units of measure throughout calculations, arbitrary precision numbers, regular expressions and graphics.

Get Frink from Google Play
What’s new – Frink is under very active development. See here for the new stuff
Many example programs in Frink
Extensive documentation for Frink

RFO BASIC! + SQL – Free

This implementation of BASIC is completely free and is described as a labour of love by the author, Paul Laughton. Paul is my kind of geek since he is the curator of The Dr. Richard Feynman Observatory and author of Atari Basic and Apple DOS 3.1 among other things.

The feature list of RFO BASIC is impressive and includes Graphics (with Multi-touch), SQL, GPS, Device Sensors, Camera and loads more. There’s a great forum with lots of very engaged developers who are writing some very nice programs.

There are two ways to deploy your programs–either as scripts that require RFO BASIC to be installed or as compiled,standalone programs that can even be added to Google Play (formerly known as the Android Market’).

RFO BASIC on Google Play
RFO BASIC Forum
RFO BASIC website
De Re BASIC! – The .pdf manual for RFO BASIC

Addi and Mathmatiz – Free

These are two MATLAB clones for Android. I’ve mentioned Addi before and they have both been covered over at Alasdair’s Musings so I won’t go into detail here other than to say that they are very cool! Linear algebra, scripting and plotting on your phone!

tiny Lisp ISLisproid

Lisp is a very old programming language which first saw the light of day in 1958! According to wikipedia, the only langauge older than Lisp that is still in common use is Fortran! With this app you can play with the language of the ancients on your super-modern smartphone. This is a no-frills app..essentially little more than a command line shell and list interpreter but that is perhaps as it should be.

tiny Lisp at Google Play

MathStudio – £12.99

I’ve been using MathStudio (formerly SpaceTime Mathematics) for quite a few years now on various operating systems and it’s great to finally have it on Android. MathStudio is a fully featured computer algebra system for your mobile phone– think mini Mathematica or Maple and you are thinking along the right lines. With this app you can write scripts that make use of advanced mathematical features, 2D and 3D graphics, animations and interactive demonstrations.

SigmaScript – Free

SigmaScript is a free implementatuion of the Lua scripting language for Android devices developed by Logimath. You get an editor, scripting engine, small console output and a few simple code examples. No graphics or anything fancy but a very nice way to play with an interesting language.

Carnival of Mathematics – The Next Generation

2012-03-20T16:47:08Z

For the last two years or so I have been doing the administration for The Carnival of Mathematics (CoM) and have had a lot of fun doing so. I first took over for carnival 59 (written by Jason Dyer and hosted over at NumberWarrior) and did the admin right up until number 84 which was published back in December 2011 by Guillermo Bautista (see here and here for some history).

Recently, however, I have struggled to find the time to give the CoM the attention it deserves and so it is time to hand over the baton. Thankfully, some very able hands have taken it from me and I am happy to announce that Peter Rowlett, Katie Steckles and Christian Perfect will be taking care of business from now on. Submissions for Carnival 85 are already open.

I’ll still be around, blogging as usual here at WalkingRandomly and hosting the occasional Carnival myself but the carnival is now being cared for by the next generation. Submit something and give them a great welcome.

A bug in Mathematica’s CUDADot in version 8.0.1

2012-03-14T00:40:52Z

After writing my recent article on GPU accelerated Matrix-Matrix multiplication using Maple, I thought that I’d try the same thing in Mathematica. However, I instantly hit a problem on my 64bit Windows 7 machine running version 8.0.1 of Mathematica.

In[1]:= a = RandomReal[1, {2, 2}]
Out[1]= {{0.363441, 0.528656}, {0.208881, 0.510232}}

In[2]:= b = RandomReal[1, {2, 2}]
Out[2]= {{0.33536, 0.77615}, {0.537533, 0.788522}}

In[3]:= Dot[a, b]
Out[3]= {{0.406054, 0.698942}, {0.344317, 0.564452}}

In[4]:= Needs["CUDALink`"]
CUDADot[a, b]
Out[5]= {{0.741414, 1.47509}, {0.881849, 1.35297}}

In short, CUDADot gives the wrong result for floating point numbers (on my machine at least). An upgrade to version 8.0.4 fixed the problem

A Month of Math Software – February 2012

2012-03-02T19:01:00Z

Welcome to the latest edition of A Month of Math Software which includes information on a new language for scientific computing, statistics software education in New Zealand as well as the usual mix of mathematical and scientific software releases from the worlds of commercial and free software.

If you’d like something added to next month’s edition then contact me (it’s free). If you want to see old editions then take a look at the MMS archive.

A new language for scientific computing

Julia is a “new high-level, high-performance dynamic programming language for technical computing, with syntax that is familiar to users of other technical computing environment.” I haven’t tried it yet but it looks great and includes a Just In Time compiler (JIT) to help boost performance.
Have you test driven Julia? Let me know what you think in the comments.

Learn statistics with the free GenStat for teaching and Learning (GTL)

Genstat is a commercial data analysis tool with an emphasis on the biosciences and they have a free version called GenStat for Teaching and Learning. This is currently being used to teach statistics in many schools in New Zealand (among other places) and a paper has been written about it called Learning Statistics Using Motivational Videos, Real Data and Free Software. Finally, check out these free videos, data sets and lesson plans that use the software.

Mathcad Prime

Mathcad Prime 2.0 has been released and has lots of new improvements. I’ve never been a fan of Mathcad and Prime 1.0 was a big disappointment for me but many people seem to like it. Let me know if you are one of them.

Attack of the clones

A major new release of the free MATLAB clone, Octave, has is now available. Version 3.6.1 has lots of new goodies and you can read all about them in the 3.6.1 NEWS file.
Version 13.7 of the MATLAB-like Euler Math Toolbox is now available. See the change log for the improvements.
Version 0.92 of the popular Mathcad clone, Smath studio, was released in February.

Number theory

A new version of PARI/GP is now available for download. From the software’s website: “PARI/GP is a widely used computer algebra system designed for fast computations in number theory (factorizations, algebraic number theory, elliptic curves…), but also contains a large number of other useful functions to compute with mathematical entities such as matrices, polynomials, power series, algebraic numbers etc., and a lot of transcendental functions.” Head over to the changelog to see what’s new.

Perl and Python

If you like to work with the Perl programming language then you should take a look at the Perl Data Language (PDL). A new version was released in February– version 2.4.10– which includes automatic multi-thread support among other things.
If you prefer Python, you’ll probably like to know that version 0.10.1 of scipy, the scientific library for python, has been released.

Math software on tablets

The Mathematica kernel is now running on iPad!
GeoGebra is a superb piece of free software for mathematics learning and teaching. Thanks to the release of the beta version of GeoGebraWeb, you can now experience some of the GeogGebra goodness on Tablets and Chromebooks. This forum post gives the details.

Finite Elements

February saw the release of two new C++ libraries: ViennaMath and ViennaFEM. The author of the libraries writes “The symbolic math kernel library ViennaMath (http://viennamath.sourceforge.net/) written in C++ allows for both runtime and compiletime evaluation, differentiation, integration, and substitution of simple mathematical expressions. In short, ViennaMath offers some of the advantages of full-fledged computer algebra systems such as Mathematica or Maple directly within C++. The symbolic math kernel is intended to be used for numerical applications and is included in the new finite element library ViennaFEM (http://viennafem.sourceforge.net/), which allows for the specification of either the strong or the weak formulation of the underlying PDE directly in code. Even though ViennaFEM is still in alpha-state, the first release already supports grids in 1d, 2d (triangular,quadrilateral) and 3d (tetrahedral, hexahedral).”

From the blogs

How to solve a NLLS problem using SQP method in Excel? – From The Numerical Algorithms Group (NAG)
Using GPUs in MATLAB – From The Mathworks (click here for my articles on GPUs in MATLAB)
How to Blur an Image with a Fourier Transform in Matlab – Part I – From MATAB Geeks

Gutmann of NSF on computational social science and advanced computing infrastructure

2012-03-01T01:37:37Z

Myron Gutmann, of the National Science Foundation, will be speaking on:

"Computational Social Science and Advanced Computing Infrastructure: Challenges and Opportunities"

11:30am to 1:00pm, March 2, 2012
Tsai Auditorium, 1730 Cambridge St. (IQSS/Harvard)

A light lunch will be served.

Please RSVP to http://www.iq.harvard.edu/events/node/2836

______________________________________

Myron P. Gutmann is Assistant Director of the National Science Foundation, with responsibility for NSF's Social, Behavioral, and Economic Sciences Directorate. He is also Professor of History and Information and Research Professor in the Institute for Social Research at the University of Michigan. Prior to joining NSF, he was Director of the Inter-university Consortium for Political and Social Research (ICPSR). Gutmann has broad interests in interdisciplinary historical research, especially health, population, economy, and the environment. As Director of ICPSR, he was a leader in the archiving and dissemination of electronic research materials related to society, population, and health, with a special interest in the protection of respondent confidentiality. He has written or edited five books and more than eighty articles and chapters. Gutmann has served on a number of national and international advisory committees and editorial
boards.

Everyone is a Programmer and Everyone Should Hone Their Programming Skill

2012-02-26T17:13:31Z

Anyone that has worked with me is tired of me suggested that everyone in business should know how to program. This thought is met with a variety of rebuttals that have only a slight shred of a validity.

Everyone programs. If you get out of bed in the morning and go through any sort of routine (everything is pretty much a routine) you are programming. This is not semantics. Programming is nothing more than organizing things in such a way that they transform into other things. Everyday life is programming, it’s just not the uber-formal (re: very restrictive) programming we think computer programmers do.

When people reject my statement about everyone programs and should get better at what they are actually rejecting is the specific implementations of computer programming – the syntax, the formalities, the tools, the long hours in front of a headache inducing screen.

If you speak, write, draw or communicate at all you have learned a set of rules that you apply to various inputs and produce various outputs. If you work in spreadsheets, at a cash register, with a paint brush, in a lecture haul, in a lab, on a stage, you are programming. If you make yourself a sandwich, eat it and go for a jog, you are programming. Everything you do is taking inputs and transforming it into outputs using various rules of a system. The system is more or less formal, more or less open.

I don’t see there being any room for dispute on this observation or rather this definition or axiom.

With that basic assumption as a starting point let me make the case that honing your more formal, strict and, yes, traditional “computer” programming skill is a must do for anyone participating in modern society. (yes, if you do not participate in modern society and do not wish to do so, you don’t need formal programming skill, but you will always be programming within the universe…)

Without getting too out there – our lives will never have fewer computers, fewer programs, fewer gadgets, fewer controllers monitoring, regulating, data exposing, recommending, and behaving on our behalf. Cell phone penetration is near ubiquitous, every car has computers, trains run on computerized schedules, more than 50% of stocks are algorithmically trade, your money is banked electronically, the government spends your taxes electronically and so on. So in some sense, to not be able to program formally leaves you without any knowledge of how these systems work or miswork. Some will have the argument that “I don’t need to know how my car works to use it/benefit from it.” This is true. But computers and programming are so much more fundamental than your car. To not be able to program is akin, at this point, to not being able to read or write. You are 100% dependent on others in the world. You can function without a working car.

Before you reject my claim outright consider the idea that learning to program is quite natural and dare I say, easy. It requires no special knowledge or skill. It requires only language acquisition skills and concentration which every human i’ve read about or know has these two basic capabilities (before we go on destroying them in college.)

Why do I make this claim of ease?

Programming languages and making programs that work rely on a very small language. Very simple rules. Very simple syntax. Frustratingly simple! The english language (or any spoken language) is so much more ridiculously complicated.

It does not surprise me that people think it’s hard. It’s frustrating. It’s the practice and the simplification of your thoughts into more simple languages and syntax that’s hard. And so is writing a speech others will understand, or painting a masterpiece, or correctly building a financial accounting book, or pretty doing anything you do for a living that requires someone else to understand and use your output.

I firmly believe each persons ability to translate their lives into useful programs is a differentiator of those that have freedom and identity and those that do not. Either you are programming and able to keep watch over the programs you use or you are programmed.

Sure, companies and people are busy at work making easier and easier tools to “program” but that doesn’t change the fundamental problem. The programs you layer on top of other programs (web page builder guis to HTML to browser parsers to web servers…) the more chance of transcription problems (miscommunication), unnoticed malicious use and so forth.

Beyond the issue of freedom it is fun and invigorating to create, to mold your world. This is the part that’s hard for adults. Having spent probably from age 10 to whatever age we all are following rules (others programs) and being rewarded (program feedback loops) we all don’t really do a great job molding our world. Kids are so good at experimenting (playing). And playing is essential to really great programming. Programming that will fill you up and make your life better is the kind that generates wonderfully unexpected but useful results. It’s not always about getting it right or spitting out the answer (though for simple programs that might be the point). It’s about creating, exploring, and finding connections in this world.

I can replace the word programmer (and programming) in this post with Artist, Mathematician, Reader, Writer, Actor, etc and it will be essentially the same piece with the same reasoning. All of these “occupations” and their activities are programming – the only thing that differs are the implementations of language (syntax, medium, tools).

When people are rejecting my argument that everyone should learn to program, they are rejecting the notion of sitting down in front of a blinking cursor on a screen and having a piece of software say “error”. Reject that! I hate that too! For me, correcting grammar in my posts or emails or journals is as painful! (but it doesn’t prevent me from wanting to write better or write at all, i *need* to to survive and be free!)

Don’t reject the notion that you shouldn’t be always trying to communicate or understand better – taking inputs from the world and transforming them into useful outputs. To reject that is essentially rejecting everything. (and that is now the annoying over-reaching philosophical close!)

a world of makers

2012-02-25T14:09:29Z

If you haven’t read Cory Doctorow’s Makers you should.

A couple years after reading it I’m reminded of it daily. The march of technology, culture, business, education towards a future in which large organizations simply can’t withstand the tide of individual creators creating on a small scale and networking upwards.

creative destruction, as it were, little tiny piece by piece. all on the backs and hands of people who probably wont make a fortune on these creations. They will get by enough.

I don’t know if it turns out that everyone gets what they need and this is the new economy capable of supporting 300+ million people. It is the new culture. and maybe we’ll do with less. or we’re have a larger and larger income gap.

artisans, craftmakers, app developers, youtube stars, self employed…

then again, we need infrastructure. roads, info networks, cellular towers. can a world of makers fully exist on top of a large commercial infrastructure? the network is the thing and the network is still owned by huge, controlled, controlling organizations. The pipes and search engines and the social networks, owned by perhaps 10-15 organizations.

Perhaps the rise of 3d printing will make it so that eventually makers can print the necessary network at a scale that removes the requirement of these big infrastructures.

Not sure.

hard to sort out.

i’m too busy making.