TIM GITTOS - graphics and artificial intelligence geek

MIT OCW Linear Algebra - Week 4

2012-03-02T00:00:00-08:00

Computing the Nullspace (Ax = 0)

Consider the matrix $A$:
\[
\begin{bmatrix}
1 & 2 & 2 & 2 \\
2 & 4 & 6 & 8 \\
3 & 6 & 8 & 10
\end{bmatrix}
\]

The means of computing the nullspace is matrix elimination. The difference between this and the row elimination we’ve done previously, is that this will operate on rectangular matrices, not just square ones, and we’ll need to possibly deal with $0$ in the pivot position. Notice that column 2 is not linearly independent, and that row 3 is not linearly independent.

It is important to note that performing matrix elimination doesn’t change the nullspace, although it does change the column space. That is, the solution to the equation doesn’t change.

Following is matrix elimination to solve the equation: $Ax = 0$:
\[
\begin{bmatrix}
1 & 2 & 2 & 2 \\
2 & 4 & 6 & 8 \\
3 & 6 & 8 & 10
\end{bmatrix}
\to
\begin{bmatrix}
1 & 2 & 2 & 2 \\
0 & 0 & 2 & 4 \\
0 & 0 & 2 & 4
\end{bmatrix}
\]

Notice above that in the $2,2$ pivot position there is a zero, and that it’s not possible to perform a row exchange with the row beneath it, because that too has a zero in the pivot position.

This tells us that the second column is not linearly independent, and that it can be formed by a combination of another column.

To continue, we consider $2,3$ to be our pivot position instead:
\[
\begin{bmatrix}
1 & 2 & 2 & 2 \\
0 & 0 & 2 & 4 \\
0 & 0 & 0 & 0
\end{bmatrix}
\]

This gives us the matrix $u$. It’s not upper triangular, but it is in echelon form, which describes the structur of the 0’s in the matrix. We can see that this matrix has 2 pivots – this means that it has a rank of 2. That is, the rank of a matrix is the number of pivots.

The matrix $u$ has 2 pivot columns. A pivot column is a column that holds a pivot. The other columns, that don’t hold a pivot, are called free columns. They are called free columns because we can assign any value to the unknowns that correspond to those columns, and solve for the unknowns in the pivot colums.

For example:
\[
x = \begin{bmatrix}
? \\
1 \\
? \\
0
\end{bmatrix}
\]

We can break out the rows of the matrixs and represent them into their equation form:
\[
\begin{align}
x_1 + 2x_2 + 2x_3 + 2x_4 &= 0 \\
2x_3 + 4x_4 &= 0
\end{align}
\]

From here, using the numbers we picked for the unkowns, we can perform back substitution, and get:
\[
\begin{align}
2x_3 + 4(0) &= 0 \\
x &= 0
\end{align}
\]

To solve for $x_1$:
\[
\begin{align}
x_1 + 2(1) + 2(0) + 2(0) &= 0 \\
x_1 + 2 &= 0 \\
x_1 &= -2
\end{align}
\]

The solution to this is a vector in the nullspace:
\[
x = \begin{bmatrix}
-2 \\ 1 \\ 0 \\ 0
\end{bmatrix}
\]

Any multiple of the vector $x$ will also lie in the nullspace. This describes a line that is in the nullspace.

We can also flip the position of the 0 and the 1 for the free columns:
\[
x = \begin{bmatrix} ? \\ 0 \\ ? \\ 1 \end{bmatrix}
\]

And the result of the back substitution is:
\[
x = \begin{bmatrix} 2 \\ 0 \\ -2 \\ 1 \end{bmatrix}
\]
which also lies in the nullspace.

With two vectors in the nullspace, we can determine the entire nullspace. The nullspace is composed of all the combinations of the two vectors found above, and can be represented as:
\[
c\begin{bmatrix} -2 \\ 1 \\ 0 \\ 0 \end{bmatrix} + d\begin{bmatrix}2 \\ 0 \\ -2 \\ 1\end{bmatrix}
\]

To generalize:
A matrix $m \times n$ with a rank of $r$ will have $n – r$ free variables, and it’s nullspace will be composed of the combinations of $n – r$ vectors. You can find these vectors by assigning the value of $1$ to each free variable while assigning the others a value of $0$, and performing back substitution. You do this $n – r$ times and can derive the nullspace. This is due to the fact that there were only $r$ equations – the free variables are the result of non linearly independent equations.

We can further clean up the matrix $u$, by expressing it in reduced row echelon form. We will call this matrix $R$.
Consider again matrix $u$:
\[
\begin{bmatrix}
1 & 2 & 2 & 2 \\
0 & 0 & 2 & 4 \\
0 & 0 & 0 & 0
\end{bmatrix}
\]

Notice the row of zeros – this comes from the fact that the third row was a combination of the first two rows.

The reduced row echelon form has zeros both above and below the pivots, and the value of 1 in all the pivots:
\[
\begin{bmatrix}
1 & 2 & 0 & -2 \\
0 & 0 & 2 & 4 \\
0 & 0 & 0 & 0
\end{bmatrix}
\]
We subtracted the second row from the first to get a 0 above the second pivot. Now we can divide the second row by 2, which won’t change the solution (as it’s part of the combination of vectors):
\[
\begin{bmatrix}
1 & 2 & 0 & -2 \\
0 & 0 & 1 & 2 \\
0 & 0 & 0 & 0
\end{bmatrix}
\]

This is the matrix $R$. Now, we can break out the linear equations, and solve them like we did above:
\[
\begin{align}
x_1 + 2x_2 – 2x_4 &= 0 \\
x_3 + 2x_4 &= 0
\end{align}
\]

If we then use the procedure to determine the vectors that define the nullspace:
\[
\begin{align}
x &= \begin{bmatrix}
? \\ 1 \\ ? \\ 0
\end{bmatrix} \\
x_3 + 2(0) &= 0 \\
x_3 &0 0 \\
x_1 + 2(1) – 2(0) &= 0 \\
x_1 &= -2
\end{align}
\]
which gives us the solution vector:
\[
\begin{bmatrix}
-2 \\ 1 \\ 0 \\ 0
\end{bmatrix}
\]

and for the other free variable:
\[
\begin{align}
x &= \begin{bmatrix}
? \\ 0 \\ ? \\ 1
\end{bmatrix} \\
x_3 + 2(1) &= 0 \\
x_3 &= -2 \\
x_1 + 2(0) – 2(1) &= 0 \\
x_1 = 2
\end{align}
\]
which gives us the solution vector:
\[
\begin{bmatrix}
2 \\ 0 \\ -2 \\ 1
\end{bmatrix}
\]
which as you can see is the solution we arrived at before, with $Ux = 0$. This equation is called $Rx = 0$.

Looking at the matrix $R$, we can see that the identity matrix I exists in the pivot points. And, that the solution to the pivot unknowns exists in the free variables, but with the signs reversed.

Let us consider a general reduced row echelon form matrix, assuming that the pivot columns are grouped together. It will look like this:
\[
R = \begin{bmatrix}
I & F \\
0 & 0
\end{bmatrix}
\]
where $I$ is the identity matrix, and $F$ are the free variables. There may or may not be a row of zeroes.
This matrix will have $r$ pivot rows, $r$ pivot columns, and $n – r$ free columns.

The nullspace for this general reduced row echelon form matrix can be given by:
\[
\begin{bmatrix}
-F \\ I
\end{bmatrix}
\]
where the columns are the vectors that define the nullspace. These vectors are called the null basis, or the basis of the nullspace.

Remember that the vectors $\begin{bmatrix}1 \\ 0 \\ 0 \end{bmatrix}, \begin{bmatrix}0 \\ 1 \\ 0 \end{bmatrix}, \begin{bmatrix} 0 \\ 0 \\ 1 \end{bmatrix}$ form the basis of the 3 dimensional cartesian space.

Example
Convert the matrix $A$ to reduced row echelon form, and find the solution for the nullspace.
\[
A = \begin{bmatrix}
1 & 2 & 3 \\
2 & 4 & 6 \\
2 & 6 & 8 \\
2 & 8 & 10
\end{bmatrix}
\]
Performing elimination:
\[
\begin{bmatrix}
1 & 2 & 3 \\
0 & 0 & 0 \\
0 & 2 & 2 \\
0 & 4 & 4
\end{bmatrix}
\to
\begin{bmatrix}
1 & 2 & 3 \\
0 & 2 & 2 \\
0 & 0 & 0 \\
0 & 4 & 4
\end{bmatrix}
\to
\begin{bmatrix}
1 & 2 & 3 \\
0 & 2 & 2 \\
0 & 0 & 0 \\
0 & 0 & 0
\end{bmatrix}
\]
The last matrix is the form $U$, the upper triangular or row echelon form. This matrix has a rank of 2, with 2 pivot columns and 1 free column.
To compute the special solutions, we can solve the linear equations by setting the free variable to 1:
\[
\begin{align}
2x_2 + 2(1) &= 0 \\
x_2 &= -1 \\
x_1 + 2(-1) + 3(1) &= 0 \\
x_1 +1 &= 0 \\
x_1 &= -1
\end{align}
\]
This gives us the special solution of:
\[
x = \begin{bmatrix}
-1 \\ -1 \\ 1
\end{bmatrix}
\]

The nullspace is given by the equation: $c\begin{bmatrix}-1 \\ -1 \\ 1 \end{bmatrix}$. This is a line in 3 dimensional space.

We could have continued to reduced row echelon form:
\[
\begin{bmatrix}
1 & 0 & 1 \\
0 & 2 & 2 \\
0 & 0 & 0
\end{bmatrix} \to
\begin{bmatrix}
1 & 0 & 1 \\
0 & 1 & 1 \\
0 & 0 & 0
\end{bmatrix}
\]
Here we can see the identity matrix come out, and the solutions to the pivot variables, $\begin{bmatrix}1 \\ 1 \end{bmatrix}$, with signs inverted.

Solving Ax = b

We will continue using matrix $A$ to demonstrate how to solve a system of linear equations.
The following system of equations is the matrix $A$ in linear equation form:
\[
\begin{align}
x_1 + 2x_2 + 2x_3 + 2x_4 &= b_1 \\
2x_1 + 4x_2 + 6x_3 + 8x_4 &= b_2 \\
3x_1 + 6x_2 + 8x_3 + 10x_4 &= b_3
\end{align}
\]
and remember the 3rd row is the sum of rows 1 & 2 – it is not independent. Therefore, $b_3$ is dependent on $b_1$ and $b_2$. A solution such as $\begin{bmatrix}1 \\ 5 \\ 17\end{bmatrix}$ will not solve this system.

To solve $A$, we need to reduce this matrix:
\[
{
\left[
\begin{matrix}
1 & 2 & 2 & 2 \\
2 & 4 & 6 & 8 \\
3 & 6 & 8 & 10
\end{matrix}
\left|
\begin{matrix}
b_1 \\ b_2 \\ b_3
\end{matrix}
\right.
\right]
}
\rightarrow
{
\left[
\begin{matrix}
1 & 2 & 2 & 2 \\
0 & 0 & 2 & 4 \\
0 & 0 & 2 & 4
\end{matrix}
\left|
\begin{matrix}
b_1 \\ b_2 – 2b_1 \\ b_3 – 3b_1
\end{matrix}
\right.
\right]
}
\rightarrow
{
\left[
\begin{matrix}
1 & 2 & 2 & 2 \\
0 & 0 & 2 & 4 \\
0 & 0 & 0 & 0
\end{matrix}
\left|
\begin{matrix}
b_1 \\ b_2 – 2b_1 \\ b_3 – b_2 – b_1
\end{matrix}
\right.
\right]
}
\]
The last line of the resultant matrix tells us the solvability condition for matrix $A$. That is that $b_3 = b_1 + b_2$. If that doesn’t hold, then $b$ is not a solution. This tells us there is a solution to $A$.

If we suppose that the $b$ before elimination is $\begin{bmatrix} 1 \\ 5 \\ 6 \end{bmatrix} $, then the resultant $b$ after elimination will be:
\[
b = \begin{bmatrix} 1 \\ 3 \\ 0 \end{bmatrix}
\]

To generalize, $A$ is solvable when $b$ is in the column space of $A$, ($C(A)$).

In general, if a combination of rows of $A$ gives a zero row, the same combination of the entries of $b$ must give 0.

To continue finding the complete solution to $Ax = b$, first we find a single solution by setting all free variables to 0 and then solve $Ax = b$ for the pivot variables, using the values for $b$ from above:
\[
\begin{align}
2x_3 + 4(0) &= 3 \\
\therefore x_3 &= \frac{3}{2} \\
x_1 + 2(0) + 2(\frac{3}{2}) + 2(0) &= 1 \\
x_1 + 3 &= 1 \\
\therefore x_1 &= -2
\end{align}
\]
As we can see, one of the solutions to x is:
\[
x = \begin{bmatrix}
-2 \\ 0 \\ \frac{3}{2} \\ 0
\end{bmatrix}
\]

Now that we have one particular solution, the complete solution will be that particular solution added to all vectors in the nullspace.

The complete for solution for $A$ (for the given $b$) is:
\[
x_{complete} = \begin{bmatrix} -2 \\ 0 \\ \frac{3}{2} \\ 0 \end{bmatrix}
+ c \begin{bmatrix} -2 \\ 1 \\ 0 \\ 0 \end{bmatrix} +
d \begin{bmatrix} 2 \\ 0 \\ -2 \\ 1 \end{bmatrix}
\]

This result is a 2 dimensional subspace of $R^4$, that goes through the solution of $x$ that we found, instead of through the origin.

Generally, consider an $m$ by $n$ matrix $A$ of rank $r$. Remember the rank of a matrix is how many pivot columns it has.
We know that $r \le m$ and $r \le n$.
Full column rank means that $r = n$. This means that all the columns independent, the columns are all pivot variables, and that there are no free variables, and that the nullspace will only contain the zero vector. The solution to $Ax = b$ will be unique, if it exists.
Full row rank means that $r = m$. This means that all the rows have pivot variables. This means that we can solve $Ax = b$ for every $b$, and that there are $n – r$ free variables.
A matrix that is full rank has $r = m = n$, and a matrix that is full rank is invertible, it’s reduced row echelon form is the identity matrix, and there is a unique solution.

To summarize:
The rank of a matrix tells you everything about the number of solutions, except what those solutions are.

When $r = m = n$ (full rank)
\[
R = I
\]
and there is 1 solution to $Ax = b$

When $r = n < m$ (full column rank)
\[
R = \begin{bmatrix} I \\ 0 \end{bmatrix}
\]
and there is 0 or 1 solution to $Ax = b$

When $r = m < n$ (full row rank)
\[
R = \begin{bmatrix}I & F \end{bmatrix}
\]
where $I$ and $F$ can be intermingled, and $I$ will be in pivot columns.
There are $\infty$ solutions to $Ax = b$

When $r < m, r < n$
\[
R = \begin{bmatrix} I & F \\ 0 & 0 \end{bmatrix}
\]
where $I$ and $F$ can be intermingled, and $I$ will be in pivot columns.
There are 0 or $\infty$ solutions to $Ax = b$

Udacity AI for Robotics - Week 2

2012-02-29T00:00:00-08:00

Tracking with Kalman Filters

Localization involves orienting a robot in a known environment. When that environment contains moving obstacles, such as other robots, or in the case of a self driving car, pedestrians and other cars, tracking is used to identify those obstacles, and determine how fast they are moving and make predictions about their movements.

Both Kalman filtering and monte carlo methods can be used for both localization and tracking.
Kalman filters are a continuous, uni-modal method, while monte carlo is a discrete, multimodal method.

Later, particle filters will be introduced, which combines both techniques to result in a continuous, multi-modal method.

Gaussian distributions

Monte carlo localization used a histogram to model the probability the robot was in a given location. This histogram is an approximation to a continuous probability:

The Gaussian distribution is another probability distribution. It is characterized by 2 features:
- the mean, $\mu$, or the value of the top of the curve
- the variance, $\sigma^2$, or the width of the curve.

In Kalman filters, the values of $(\mu, \sigma^2)$ will identify the belief state of the location of a given obstacle.
This belief is modelled with the function:
\[
f(x) = \frac{1}{\sqrt{2 \pi \sigma^2 }} e^{-\frac{1}{2}\frac{(x – \mu)^2}{\sigma^2}}
\]
$\frac{1}{\sqrt{2 \pi \sigma^2 }}$ is a constant used to normalize the probability.

Kalman filtering is called a uni-modal method because the Gaussian distribution is uni-modal. This means that it has a single peak in the graph of the distribution. Gaussian distributions are also symmetrical.

$\sigma^2$ is a measure of uncertainty, and in the above equation, it acts to normalize the difference of a value and the mean. The larger the $\sigma^2$, the more uncertain the distribution is.

Stages

Similar to the process of localization, Kalman filters involve repetition over two stages of execution: measurement updates and prediction updates. In localization, and in Kalman filters, the measurement update process involves Bayes rule and calculating products of probabilities, and the prediction updates involve total probability and convolution. The main difference is the probability distributions used, in that Kalman filters use a Gaussian distribution.

Measurement update

As in localization, the robot starts with a prior belief state, which in the case of Kalman filters, is a Gaussian distribution. It is typically very uncertain. If the robot reads it’s sensors, and obtains a new Gaussian distribution that represents what the sensors detected, it needs to multiply those Gaussians together to get the posterier belief state.

This is given by the following equation:
\[
\begin{array}{l}
\mu’ = \frac{r^2\mu + \sigma^2\nu}{r^2 + \sigma^2} \\
{\sigma^2}’ = \frac{1}{\frac{1}{r^2} + \frac{1}{\sigma^2}}
\end{array}
\]

Prediction/Motion update

Due to the uncertain nature of robot movement, as in localization, when the robot moves, it loses information. In the case of Kalman filters, this means that the new calculated Gaussian belief state has a higher variance than it did before it moved. Otherwise, the move works similar to convolution in localization.

The update to the Gaussian belief state is given by:
\[
\begin{array}{l}
\mu’ = \mu + u \\
{\sigma^2}’ = \sigma^2 + r^2
\end{array}
\]

This is saying that the resultant mean is the move distance ($u$) plus the old mean, and the resultant variance is the old variance plus the new variance (uncertainty value in the motion).
This succinctly models the loss of certainty involved in moving.

Multivariate Gaussians

So far, all discussion has been referring to Kalman filtering in 1-D, or one dimension. Real life Kalman filtering will likely take place in 2-D or even 3-D space. To do filtering in these higher dimensions, multivariate Gaussians are used.

Multivariate Gaussians differ in that the mean value becomes an array of means, and the variance value becomes an array of covariance values, of $D \times D $ size, where D is the dimensionality of the matrix.

For more on multivariate Gaussians, see this Stanford handout from the Machine learning class and my Machine Learning notes, particularly the notes on unsupervised learning.

To design a Kalman filter in higher dimensions, the Kalman filter needs a state transition function and a measurement function.

State transition function

When updating the state, as per 1-D Kalman filters, the new location is the old location plus the velocity, and the new velocity is the same as the old velocity. This can be represented by the following equation:
\[
\begin{bmatrix} x’ \\ \dot{x}’ \end{bmatrix} \leftarrow \begin{bmatrix} 1 & 1 \\ 0 & 1 \end{bmatrix} \begin{bmatrix} x \\ \dot{x} \end{bmatrix}
\]

The matrix is known as F, and will be used in the update equations.

Measurement function

Making a measurement involves reading the new location of the object. This can be represented by the following equation:
\[
z \leftarrow \begin{bmatrix} 1 & 0 \end{bmatrix} \begin{bmatrix} x \\ \dot{x} \end{bmatrix}
\]

The row vector is known as H, and will be used in the update equations.

Update equations

The following image outlines the general approach to Kalman filtering with multivariate Gaussians:

The first step is the prediction step. To update the prediction, the location needs to be updated, and the uncertainty covariance needs to be updated.
Updating the location involves multiplying the current belief of the location by the state transition matrix, described above, and adding a motion vector that describes external motion acting on the item.
After updating the location, the uncertainty covariance matrix needs to be updated.

The last step is the measurement update step. To update the measurement, the location needs to be updated, and the uncertainty covariance needs to be updated.
First, calculate the error in the previous prediction to the current measurement. Project the previous uncertainty into the measurement function and add in noise. Calculate the Kalman gain which is a function of the uncertainty. Update the estimate by adding the current value to the Kalman gain influenced error, and update the uncertainty by adding a manipulated Kalman gain value to the existing uncertainty covariance.

More information

An Introduction to the Kalman Filter
Stochastic Models, Estimation and Control – Volume 1, Chapter 1
Kalman Filtering

MIT OCW Single Variable Calculus - Week 4

2012-02-24T00:00:00-08:00

Exponents

To recap from last time, we took the derivative of
\[
a_k = \left(1 – \frac{1}{k} \right)^k
\]
as $k$ tends to infinity:
\[
\lim_{k \to \infty} a_k = e
\]

We came to this result by means of the following (abreviated) process:
\[
\begin{array}{l}
ln\;a_k \to 1 \qquad \text{as k approaches infinity} \\
e^{ln\;a_k} \to e \qquad \text{as k approaches infinity} \\
e^{ln\;a_k} = a_k \quad \text{because} e^{ln\;a} = a
\end{array}
\]

We then used that to come up with a formula for $e$:
\[
e = \lim_{k \to \infty} \left( 1 + \frac{1}{k} \right)^k
\]

We previously covered solving:
\[
\frac{\delta}{\delta x} x^r = r\;x^{r – 1}
\]
for all real r. Now we will cover solving it for all r.

Method 1: use base e.
\[
\begin{align}
x^r &= \left(e^{ln\;x} \right)^r\\
&= e^{r\;ln\;x} \\
\frac{\delta}{\delta x}x^r &= \left( e^{r\;ln\;x} \right)’ \\
&= e^{r\;ln\;x}(r\;ln\;x)’ \\
&= x^r \frac{r}{x} \\
&= r\;x^{r – 1}
\end{align}
\]
which shows that it holds for the above.

Method 2: logarithmic differentiation.
\[
u = x^r, \quad ln\;u = r\;ln\;x
\]
\[
\begin{align}
\frac{u’}{u} &= (ln\;u)’ \\
&= \frac{r}{x} \\
u’ &= u \frac{r}{x} \\
&= x^r \frac{r}{x} \\
&= r\;x^{r – 1}
\end{align}
\]
which shows that it holds for the above.

Review

General formulas

Practice $(u + v)‘, (c\;u)’, (u\;v)‘, (\frac{u}{v})’$, the last two being the product rule and the quotient rule respectively.
Also know the chain rule:
\[
\frac{\delta}{\delta x}f(u) = f’(u)\;u’(x) \qquad \text{where} \quad u = u(x)
\]
and implicit differentiation – inverses, logarithmic differentiation.

Specific functions

$ x^r, sin\;x, cos\;x, tan\;x, sec\;x, e^x, ln\;x, tan^{-1}x, sin{-1}x $

Chain rule

We previously didn’t explain why the chain rule was true. Now we will show it through an example.
Take the function: $y = 10x + b$. $y$ is changing 10 times as fast as x, therefore $\frac{\delta y}{\delta x} = 10$.
Now consider: $x = 5t + a$, then $\frac{\delta x}{\delta t} = 5$.
The chain rule says that if $y$ is increasing 10 times as fast as $x$, and $x$ is increasing 5 times as fast as $t$, then $y$ is increasing 50 times as fast as $t$.

You can use the chain rule to make other rules easier to use.
For example, the quotient rule:
\[
\begin{align}
f &= \left( \frac{1}{v} \right) \\
f’ &= \left( \frac{1}{v} \right)’ \\
&= ( v^{-1} )’ \\
&= ( v^{-1} )’ v’ \qquad \text{by the chain rule} \\
&= -v^{-2}v’
\end{align}
\]
and the full derrivation of the chain rule:
\[
\begin{align}
f &= \left( \frac{u}{v} \right) \\
f’ &= \left( \frac{u}{v} \right)’ \\
&= (u\;v^{-1})’ \\
&= u’v^{-1} + u(-v^{-2}v’) \\
&= \frac{(u’v – uv’)}{v^2}
\end{align}
\]

Examples

Consider: $\frac{\delta}{\delta x} sec\;x$.

\[
\begin{align}
\frac{\delta}{\delta x} sec\;x &= \frac{\delta}{\delta x}(cos\;x)^{-1} \\
&= – (cos\;x)^{-2} cos’x \\
&= – (cos\;x)^{-2}( -sin\;x) \\
&= \frac{sin\;x}{cos^2x} \\
&= \frac{1}{cos\;x} \times \frac{sin\;x}{cos\;x} \\
&= sec\;x\:tan\;x
\end{align}
\]

Generally, when differentiation trig functions, express the answer in terms of the original trig function if at all possible, as we did above.

Consider: \(\frac{\delta}{\delta x} ln(sec\;x)

\[
\begin{align}
\frac{\delta}{\delta x}ln(sec\;x) &= ln(sec\;x)‘(sec\;x)’ \\
&= \frac{1}{sec\;x}(sec\;x)’ \\
&= \frac{1}{sec\;x}(sec\;x\;tan\;x) \\
&= tan\;x
\end{align}
\]

Consider: $\frac{\delta}{\delta x}(x^{10} + 8x)^6$

\[
\frac{\delta}{\delta x}(x^{10} + 8x)^6 = 6(x^{10} + 8x)^5(10x^9 + 8)
\]

We used the chain rule above because expanding the equation out to the 6th power would have taken a lot of space.

Consider: $\frac{\delta}{\delta x}e^{x\;tan^{-1}x}$

\[
\begin{align}
\frac{\delta}{\delta x}e^{x\;tan^{-1}x} &= e^{x\;tan^{-1}x}(x\;tan^{-1}x)’ \\
&= e^{x\;tan^{-1}x}(x(tan^{-1}x)’ + tan^{-1}x) \\
&= e^{x\;tan^{-1}x}\left(tan^{-1}x + \frac{x}{1 + x^2}\right)
\end{align}
\]

Review Continued

The definition of the derivative:
\[
\frac{\delta}{\delta x}f(x) = lim_{\Delta x \to 0} \frac{f(x + \Delta x) – f(x)}{\Delta x}
\]
We discovered the derivative of the following functions using the first principles of differentiation:
$\frac{1}{x}, x^n, sin\;x, cos\;x \text{last two using} \quad x = 0, a^x, (uv), \left(\frac{u}{v}\right)$

We also discussed the meaning of a derivative, that is, the rate of change with respect to some variable.

You can also read the formula for a derivative backwards, and get the function from the limit:
\[
\begin{align}
\lim_{u \to 0}\frac{e^u – 1}{u} &= \frac{e^{u – 0} – e^0}{u} \\
&= \frac{\delta}{\delta x}e^u \bigg|_{u = 0} \\
&= 1
\end{align}
\]

Applications of Differentiation

Linear approximations

The formula for linear approximations:
\[
f(x) \approx (fx_0) + f’(x_0)(x – x_0)
\]

This means that if you have a curve, such as $y = f(x)$, then it is approximately the same as it’s tangent line, $y = f(x_0) + f’(x_0)(x – x_0)$.

For example, consider the function $f(x) = ln\;x$ and it’s derivative, $f’(x) = \frac{1}{x}$, at $x = 1$:
\[
\begin{align}
f(1) &= ln\;1 = 0 \\
f’(1) &= 1 \\
ln\;x &\approx 0 + 1(x – 1) \\
&\approx x – 1
\end{align}
\]

This shows that for the function $ln\;x$, when $x = 1$, it is approximate to the function $y = x – 1$. When $x$ changes significantly, this is no longer true, and $ln\;x$ will approximate some other function:

Remember the definition of a derivative is the limit as the change in $x$ goes to zero, of the ratio of the change in the result of the function and the change in $x$:
\[
f’(x) = \lim_{\Delta x \to 0} \frac{\Delta f}{\Delta x}
\]
You can also look at this backwards, by looking at the limit and determining the derivative based off of that.

In linear approximation, what we’re essentially saying is that as $\Delta x$ gets close to 0:
\[
\frac{\Delta f}{\Delta x} \approx f’(x_0)
\]
That is, the average rate of change ($\frac{\Delta f}{\Delta x}$) is close to the infintesimal rate of change ($f’(x)$) as $x$ approaches some value.

The formula above is the same as the formula for linear approximation:
\[
\begin{align}
\frac{\Delta f}{\Delta x} &\approx f’(x_0) \\
\Delta f &\approx f’(x_0)\Delta x \\
f(x) – f(x_0) &\approx f’(x_0)(x – x_0) \\
f(x) &\approx f(x_0) + f’(x_0)(x – x_0)
\end{align}
\]

Systematic discussion of linear approximation

When performing a systematic discussion, convention is to use $x_0 = 0$. This will result in the formula for linear approximation:
\[
f(x) \approx f(0) + f’(0)x
\]

Below will be an analysis of the linear approximation of a few notable functions:

\[
\begin{array}{ l | l | l | l | l }
f & f’ & f(0) & f’(0) & f(0) + f’(0)x \\ \hline
sin\;x & cos\;x & 0 & 1 & x \\
cos\;x & -sin\;x & 1 & 0 & 1 \\
e^x & e^x & 1 & 1 & 1 + x \\
ln(1 + x) & \frac{1}{1 + x} & 0 & 1 & x \\
(1 + x)^r & r(1 + x)^{r – 1} & 1 & r & 1 + rx \\
\end{array}
\]

And geometrically:

$sin\;x$

$cos\;x$

$e^x$

The last two examples are important. Due to the convention of $x = 0$, we can’t find the tangent of $ln$, because the slope of $ln(0)$ tends $-\infty$. The slope of $x^r$ also tends to $\infty$. To deal with this, we shift the function by starting from a base of one.

Going back to our earlier approximation of $ln$, we can consider the following:
\[
ln\;u \approx u – 1
\]
from above, and just set $u = 1 + x$ to achieve the approximation at $x = 0$.

Applications of linear approximation

Consider the following example:
\[
ln\;1.1 \approx \frac{1}{10}
\]
We achieved this by using the rule from above:
\[
ln(1 + x) \approx x
\]
and just set $x = \frac{1}{10}$.

This only holds because $x$ is sufficiently small. We work with linear approximations of functions because they are easier to calculate than the functions themselves.

You can see this in the following examples:

Example 1
Find the linear approximation “near $x = 0$” ($x \approx 0$) of:
\[
\frac{e^{-3x}}{\sqrt{1 + x}}
\]
We can find this from the formulas and their approximations above:
\[
\begin{align}
\frac{e^{-3x}}{\sqrt{1 + x}} &= e^{-3x}(1 + x)^{-\frac{1}{2}} \\
&\approx (1 – 3x)(1 – \frac{1}{2}x) \\
&\approx 1 – 3x – \frac{1}{2}x + \frac{3}{2}x^2 \\
&\approx 1 – \frac{7}{2}x
\end{align}
\]
Note that we discared the quadratic term $\frac{3}{2}x^2$, because in the transitions above from the functions to their linear approximations, we already discard a lot of quadratic and exponential information, and we’re only concerned with a linear approximation. It stands to reason that we discard all quadratic information. Also note that when $x$ is sufficiently small, any quadratic terms are significantly smaller than the linear terms, and can safely be ignored.

This is kind of the opposite as big O notation in computer science, where we discard smaller terms because we’re only interested in the high value approximations.

Example 2

This example is intended to be relatively real life, and model something you’ll come across in real life.

Consider the surface of the earth, and a satellite in orbit around the earth at velocity $v$. The satellite is a GPS satellite, and requires keeping time, $T$. On the earth, from our frame of reference, we have time $T’$. Special relatively tells us there will be a time dilation between the times $T$ and $T’$, given by:
\[
T’ = \frac{T}{\sqrt{1 – \frac{v^2}{c^2}}}
\]
where $v$ is the velocity of the satellite, and $c$ is the speed of light.
The goal is to determine the difference between times on the satellite and on the earth, or $\Delta T$

We can solve this by using the approximations above, by setting $u = \frac{v^2}{c^2}$:
\[
\begin{align}
T’ &= T(1 – u)^{-\frac{1}{2}} \\
&\approx T(1 + \frac{1}{2}\frac{v^2}{c^2})
\end{align}
\]

To show how this is real life based, when putting the GPS system into orbit, scientists needed to determine whether or not the time dilation would cause errors. The way the satellites are now, $v = 4km/s$ and $c = 3 \times 10^5 km/s$, which means that $\frac{v^2}{c^2} \approx 10^{-10}$. As you can see, this resolves to an error of a few mm of resolution. Therefore it was determined the dilation wouldn’t cause significant error.

Quadratic approximations

The quadratic approximation is an extension of linear approximation. The formula for quadratic approximation is:
\[
f(x) \approx f(x_0) + f’(x_0)(x – x_0) + \frac{f’’(x)}{2}(x – x_0)^2
\]
As you can see, it involves second derrivatives.

Now we can extend the common functions we did for linear approximation, and fill them for quadratic approximation:
\[
\begin{array}{ l | l | l | l | l }
f & f(0) + f’(0)x & f’’ & f’’(0) & f(0) + f’(0)x + \frac{f’’(0)}{2}x^2 \\ \hline
sin\;x & x & -sin\;x & 0 & x \\
cos\;x & 1 & -cos\;x & -1 & 1 – \frac{1}{2}x^2 \\
e^x & 1 + x & e^x & 1 & 1 + x + \frac{1}{2}x^2 \\
\end{array}
\]

Geometrically, the significance of the quadratic term can be shown in the following graph:

As you can see in the graph, the quadratic approximation is a parabola beneath the linear approximation that more closely fits the cosine wave at $x = 0$. This tells us more information about the function, such as what it’s doing on both the positive and negative limits.

Udacity AI for Robotics - Week 1

2012-02-22T00:00:00-08:00

Localization

The problem that localization solves is that of determining the robot’s position to within a 2 – 10cm precision. Using GPS is not sufficient, as it can have a 2 – 10m precision, which can result in the robotic car crashing on the side of the road.

Conceptual Overview

Before getting stuck into the specifics of localization, it will help to get a conceptual overview of what localization does. Localization is a way of modeling where in the environment the robot is.
This is done by way of computing probabilities of possible locations, in a multi step process that involces reading sensor data from the environment, calculating the belief, moving, recalculating belief and then reading from sensor data again, and recalculating belief.

The following diagram will illustrate this process, and then it will be explained step by step:

First, we’re modeling a robot in an environment with 3 doors. The initial belief is called the uniform maximum confusion. This indicates that the robot has absolutely no idea where in the environment it is. It does know what the environment as a whole looks like, however.

First, it takes a sensor reading from the environment, and detects that it is in front of a door. This reading allows the robot to calculate the probability it is at a given position. As can be seen, it believes it is at one of three positions. Note also that there is a margin of error on the sensors, so there is also a low probability that it could be in a position that is not in front of a door. This is called the posterier belief, because it occurs after a measurement.

Next, the robot moves a certain distance, and it updates it’s belief state. This is called a convolution of it’s belief state. Notice that the probabilities have flattened out somewhat, as there is also a margin of error on the movement. This updated belief state is called the prior belief, because it is a belief before any measurement.

Lastly, the robot takes another sensor reading of the environment, and detects it is infront of another door. It calculates the probability that it is at a given point, and multiplies it with the prior probability. This has the effect of reinforcing the fact that it saw a door, moved and saw another door. It now strongly believes it’s currently in front of the second door.

If it moved, convoluted it’s belief, and read the environment again and detected another door, then the strong belief that it was in front of the second door would be weakend, representing the fact that the robot could have had a malfunction in movement.

Uniform maximum confusion

Consider there are 5 discrete locations the robot can be in. The initial uniform maximum confusion will be a uniform probability distributed over the 5 discrete locations, and it will be normalized, meaning it sums to one. Therefore, each location will have a probability of 0.2 of being correct.

Posterier update

After a measurement, the robot updates the probabilities for each location being considered. The probability in a cell is weighted based on the measurement result. An example of this can be seen below:

In the above example, the robot observes it is on a red square. The belief state of all red squares is multiplied by 0.6, which is the probability of measuring red, and the belief state of all green squares is multiplied by 0.2, which is the probability of measuring green.

These probabilities are not valid, however, as they need to be normalized. Here is the result of the normalization:

Robot Motion

When the robot moves, we typically model this action as uncertain. The robot motion may fail for any number of reasons, and so we assign a probability to the success of the motion. In an exact motion model, the probability of motion is 1.

When motion is uncertain, the robot’s position in the world itself because even more uncertain:

Everytime the robot moves, it loses information. This is because the movement is uncertain.

Conclusion

Robot localization is a loop of move and sense. When the robot senses, it gains information. When the robot moves, it loses information.
Here is some python code that demonstrates the concepts discussed above:

p = [0.2, 0.2, 0.2, 0.2, 0.2]
world = [‘green’, ‘red’, ‘red’, ‘green’, ‘green’]

measurements = [‘red’, ‘green’]

motions = [1,1]

pHit = 0.6

pMiss = 0.2

pExact = 0.8

pOvershoot = 0.1

pUndershoot = 0.1
def sense(p, Z):

    q=[]

    for i in range(len(p)):

        hit = (Z == world[i])

        q.append(p[i] * (hit * pHit + (1-hit) * pMiss))

    s = sum(q)

    for i in range(len(q)):

        q[i] = q[i] / s

    return q
def move(p, U):

    q = []

    for i in range(len(p)):

        s = pExact * p[(i-U) % len(p)]

        s = s + pOvershoot * p[(i-U-1) % len(p)]

        s = s + pUndershoot * p[(i-U+1) % len(p)]

        q.append(s)

    return q
for s, m in zip(measurements, motions):

    p = sense(p, s)

    p = move(p, m)
print p

Formal Definition

The heart of the localization algorithm is probability and Bayes rule. For the formal definition, please see the notes on the Stanford online AI class, specifically Week 2.

In the above Python code, we applied Bayes rule as follows:

Assume $X$ is the collection of grid cells, and $Z$ is the measurement made.
\[
P(X_i|Z) = \frac{P(Z|X_i) P(X_i)}{P(Z)}
\]

The above represents the following statement in Bayes Rule:
The probability of being in cell $X_i$ given measurement $Z$ can be found by multiplying the probability of measuring $Z$ while in cell $X_i$ multiplied by the prior belief that the robot is in $X_i$, normalized by the probability of observing $Z$ in any cell.

We represented it programatically as:
\[
\begin{align}
\bar{P}(X_i|Z) &\leftarrow P(Z|X_i) P(X_i) \\
\alpha &\leftarrow \sum \bar{P}(X_i|Z) \\
P(X_i|Z) &\leftarrow \frac{1}{\alpha} \bar{P}(X_i|Z)
\end{align}
\]
where $\alpha$ is the sum of the probabilities of observing $Z$. This was programmed in the sense function of the Python code above.

Motion in localization is covered by the concept of total probability:
\[
P(X_i^t) = \sum_j P(X_j^{t-1}) . P(X_i | X_j)
\]

This says that the probability of being in cell $X_i$ at time $t$ is the sum of the probabilities of being in that cell at $t-1$ times the probability of getting to that cell from any other cells, $X_j$.

You can see this programmed into the move function of the Python code above.

This can be generalized to:
\[
P(A) = \sum_B P(A|B) P(B)
\]

Udacity AI for Robotics

2012-02-21T00:00:00-08:00

THIS IS A PLACEHOLDER SO THAT THE INDEX APPEARS IN THE PROJECT LIST

MIT OCW Linear Algebra - Week 3

2012-02-18T00:00:00-08:00

Permutations

Permutation matrices $P$ execute row exchanges.

The concept of $A = LU$, that is, a matrix $A$ can be factorized into an elimination matrix $U$ and a set of operations performed during the elimination $L$. This model supposes no permutations were required during elimination.

$PA = LU$ is the description of elimination that takes into account row exchanges, or permutations, occur.

Transposes

Transposing a matrix:
\[
\begin{bmatrix}
1 & 3 \\
2 & 3 \\
4 & 1
\end{bmatrix}
^T
=
\begin{bmatrix}
1 & 2 & 4 \\
3 & 3 & 1
\end{bmatrix}
\]

Generally:
\[
A^T_{i,j} = A_{j,i}
\]

Symmetric matrices are matrices that are the transposes of themselves. That is, $A^T = A$. Needless to say, a symmetric matrix must be square.

The product of a matrix and it’s transpose is a symmetric matrix. $A^TA$ is always symmetric:
\[
\begin{bmatrix}
1 & 3 \\
2 & 3 \\
4 & 1
\end{bmatrix}
\begin{bmatrix}
1 & 2 & 4 \\
3 & 3 & 1
\end{bmatrix}
=
\begin{bmatrix}
10 & 11 & 7 \\
11 & 13 & 11 \\
7 & 11 & 17
\end{bmatrix}
\]

Generally:
\[
(A^TA)^T = A^TA^{TT} = A^TA
\]

Vector spaces & subspaces

A vector space is a space, or set of vectors that allow us to take linear combinations of vectors, and multiply vectors by any scalar, and perform vector addition.

Examples:
$R^2$ is a space of all 2-dimension real vectors. Such as $ \begin{bmatrix}3 \\ 2 \end{bmatrix}, \begin{bmatrix}0\\0\end{bmatrix}, \begin{bmatrix}\pi\\e\end{bmatrix}$. This is also known as the x-y plane.

$R^3$ is a space of all 3-dimensional real vectors, similar to above, that is, 3 real compents.

Generally, $R^n$ is the space of all vectors with $n$ real numbers.

Let us imagine the vector space $R^2$, only lets define it only for positive components. This reduces the vector space to $\frac{1}{4}$ of it’s normal size. Consider the task of multiplying a vector by the scalar $-1$. This would produce a result that is undefined for this restriction. Therefore, this restriction is not a vector space, as it doesn’t satisfy the rules of a space. That is, it’s not closed under multiplication of real numbers.

Imagining the same vector space $r^2$, and inside this space we have a vector. Lets say we restrict our space to only linear combinations of our vector. That is, we can make our vector as long as as short as we want, and can add and subtract vectors that lie on the same line as the original vector. By that definition, we have defined a vector space inside $R^2$, called a subspace of $R^2$. Note, to be a valid subspace, it must contain 0. Therefore the line must pass through the origin in our $R^2$ space.
Note, that the subspace of $R^2$ is not $R^1$ – the vectors still contain 2 components.

Subspaces of $R^2$:
1. all of $R^2$. A space is a subspace of itself.
2. any line through the point $\begin{bmatrix}0\\0\end{bmatrix}$.
3. the zero vector only – $Z$

Lets apply this concept of subspaces to matrices.

Consider the matrix A:
\[
\begin{bmatrix}
1 & 3 \\
2 & 3 \\
4 & 1
\end{bmatrix}
\]

The columns of this matrix and all their combinations form a subspace. This is called the column space, $C(A)$.
Note this is a subspace of $R^3$, because each column is $R^3$, and they form a plane.

The union of two subspaces does not make a subspace in itself. If you add a vector from one subspace to another vector from a different subspace may result in a vector that lies outside the union of subspaces.

However, the intersection of two subspaces is a subspace. Consider two subspaces, $S$ and $T$, and two vectors in the intersection of those subspaces, $v$ and $u$. If $v$ and $u$ are both in $S$, and $S$ is a subspace, then $v+u$ will also be in $S$. Likewise, $v$ and $u$ are both in $T$, and $T$ is a subspace, therefore $v+u$ will be in $T$. This shows that the intersection of $S$ and $T$ is a subspace.

Column Space

Column space of matrix $A$:
\[
\begin{bmatrix}
1 & 1 & 2 \\
2 & 1 & 3 \\
3 & 1 & 4\\
4 & 1 & 5
\end{bmatrix}
\]

The column space of $A$ is a subspace of $R^4$, because $A$ is a $4 \times 3$ matrix. It contains:
- The columns of $A$
- All linear combinations of the columns of $A$
- The zero vector
The column subspace is itself a subspace of $R^4$.

To answer how we know that, we need to reframe our purpose in studying linear algebra.
The purpose of studying linear algebra is to understand the equation:
\[
Ax = b
\]
and to know if, there is a solution to this equation for ever $b$.

There is not a solution for every $b$, because if you look at A, you can see there are 4 equations with only 3 unknowns:
\[
Ax =
\begin{bmatrix}
1 & 1 & 2 \\
2 & 1 & 3 \\
3 & 1 & 4 \\
4 & 1 & 5
\end{bmatrix}
\begin{bmatrix}
x_1 \\
x_2 \\
x_3
\end{bmatrix}
=
\begin{bmatrix}
b_1 \\
b_2 \\
b_3 \\
b_4
\end{bmatrix}
\]
This is because the linear combinations of the columns of $A$ won’t fill all of $R^4$.

However, there are some solutions, and all of these solutions lie in the column space, that is, they are linear combinations of the columns of $A$.

Note that all the columns of $A$ are not linearly independent – column 3 is linearly dependent on columns 1 & 2, and column 1 is linearly dependent on 2 & 3. We can get rid of either column 1 or column 3 (but not both) and not lose information about the column space.
Therefore, the column space is actually a subspace in $R^2$ of $R^4$.

Nullspace

Lets consider again the matrix $A$, above.

The null space of $A$ contains all solutions $x$, as above, to the equation:
\[
Ax = 0
\]
That is, all vectors that nulify the matrix $A$.

The nullspace is a subspace of $R^3$, because there are only 3 unknowns, as you can see by looking at $x$.

\[
Ax =
\begin{bmatrix}
1 & 1 & 2 \\
2 & 1 & 3 \\
3 & 1 & 4 \\
4 & 1 & 5
\end{bmatrix}
\begin{bmatrix}
x_1 \\
x_2 \\
x_3
\end{bmatrix}
=
\begin{bmatrix}
0 \\
0 \\
0 \\
0
\end{bmatrix}
\]

One solution to the above equation is:
\[
\begin{bmatrix}
1 \\ 1 \\ -1
\end{bmatrix}
\]
and in fact, the definition of the nullspace of $A$ is any combination of the above vector, or:
\[
c
\begin{bmatrix}
1 \\ 1 \\ -1
\end{bmatrix}
\]

The nullspace of $A$ is a subspace of $R^3$ and forms a line.

We can prove that the solutions to $Ax = 0$ always give a subspace algebraically:
if $Av = 0$ and $Aw = 0$, then $A(v + w) = 0$. We can restart $A(v + w) = 0$ as $Av + Aw = 0$. $0 + 0 = 0$ will always hold, therefore the solutions to $Ax = 0$ will always give a subspace.

MIT OCW Single Variable Calculus - Week 3

2012-02-17T00:00:00-08:00

Implicit differentiation

Implicit differentiation is a technique that allows you to differentiate functions that you may not have even been able to see.

\[
\frac{\delta}{\delta x}x^2 = ax^{a-1}
\]

So far we’ve covered the case where $a$ is an integer: $a = 0, \pm 1, \pm 2, \ldots $. Now we’re going to consider the case where $a$ is a rational number: $ a = \frac{m}{n}$, where m & n are integers.

Example 1

\[
\begin{align}
y &= x^{\frac{m}{n}} \qquad \text{(1)} \\
y^n &= x^m \qquad \text{(2)}
\end{align}
\]

Now, we apply $\frac{\delta}{\delta x}$ to equation (2). We don’t apply to equation (1) because right now, we don’t know how to differentiate it. We do know how to differentiate (2).

\[
\begin{align}
\frac{\delta}{\delta x} y^n &= \frac{\delta}{\delta x}x^m \\
\left( \frac{\delta}{\delta y}y^n \right) \frac{\delta y}{\delta x} &= mx^{m-1} \qquad \text{(by the chain rule)} \\
ny^{n-1}\frac{\delta y}{\delta x} &= mx^{m-1} \\
\frac{\delta y}{\delta x} &= \frac{m x^{m – 1}}{n y^{n – 1}} \\
&= \frac{m}{n} \frac{x^{m-1}}{(x^\frac{m}{n})^{n-1}} \\
&= a x^{m-1-(n-1)\frac{m}{n}} \\
&= a x^{m-1-m+\frac{m}{n}} \\
&= a x^{-1 + \frac{m}{n}} \\
&= a x^{a-1}
\end{align}
\]

NOTE: THE CHAIN RULE WILL CAUSE YOU ISSUES

Example 2

\[
x^2 + y^2 = 1
\]

This is defining $y$ as a function of $x$ implictily. That means it can be arranged to express $y$ in terms of $x$.

Solving for y:

\[
\begin{align}
y^2 &= 1 – x^2 \\
y &= \pm \sqrt{1 – x^2} \\
y &= (1-x^2)^{\frac{1}{2}} \qquad \text{considering positive branch only} \\
y’ &= \frac{1}{2}(1 – x^2)^{-\frac{1}{2}}(-2x) \\
y’ &= (1 – x^2)^{-\frac{1}{2}}(-x) \\
y’ &= \frac{-x}{\sqrt{1 – x^2}}
\end{align}
\]

The above is the explicit differentiation.
Following is the implicit method:

\[
\begin{align}
\frac{\delta}{\delta x} (x^2 + y^2 &= 1) \\
2x + 2yy’ &= 0 \qquad \text{(by the chain rule)} \\
y’ &= -\frac{2x}{2y} \\
y’ &= -\frac{x}{y} \\
\end{align}
\]

The result of the explicit and implicit methods are the same, and the implicit is better than the explicit because it doesn’t differentiate between positive and negative values for $y$ and $x$.

Example 3

Consider the equation $y^4 + xy^2 – 2 = 0$.

We can solve it explicitly: (using the quadratic forumula )

\[
\begin{align}
y^2 &= \frac{-x \pm \sqrt{x^2 – 4(-2)}}{2} \\
y &= \pm \sqrt{\frac{-x \pm \sqrt{x^2+8}}{2}} \\
\end{align}
\]

Notice the 4 different roots of the equation (because it’s to the 4th power).
It’s very complex and time consuming.

Compare this to the implicit method:

\[
\begin{align}
\frac{\delta}{\delta x}(y^4 + xy^2 -2 = 0) \\
4y^3y’ + y^2 + x(2yy’) – 0 &= 0 \\
(4y^3 + 2xy)y’ + y^2 &= 0 \\
(4y^3 + 2xy)y’ &= -y^2 \\
y’ &= \frac{-y^2}{4y^3 + 2xy}
\end{align}
\]

To solve $y’$ in terms of x, we’d need to put the result of the explicit method (so far) into the implicit method. The two go hand in hand.
Although the implicit method hides the conplexity of solving a quartic equation, it still exists, and at some point, we’re going to have to deal with it. That means going through all possible 4 roots to find the answer.

Inverse functions

The inverse of a function is the function that gets us back to our original arguments. For example:
\[
y = \sqrt{x} \text{for x \gt 0}, y^2 = x
\]
if we define these in terms of functions:
\[
f(x) = \sqrt{x}, g(y) = x, g(y) = y^2
\]

In general:
\[
\begin{array}{2,1}
y = f(x), g(y) = x \\
g(f(x)) = x, g = f^{-1}, f = g^{-1}
\end{array}
\]

Consider the case of graphing a function and it’s inverse. The inverse can be obtained by swapping the $x$ and $y$ values, which in effect is mirroring the function along the line $x = y$.

Implicit differentiation allows us to find the derivative of any inverse function, provided we know the derivative of the function.

Example 1:

Consider the function:

\[
y = tan^{-1}x
\]

which can be rearranged/simplified by taking $tan$ of each side:

\[
tan\;y = x
\]

This can be graphed as follows:

Recall that

\[
\begin{align}
\frac{\delta}{\delta y} tan\;y &= \frac{\delta}{\delta y} \frac{sin\;y}{cos\;y} \\
&= \frac{1}{cos^2y}
\end{align}
\]

\[
\begin{align}
\frac{\delta}{\delta x} (tan\;y &= x) \\
(\frac{\delta}{\delta y} tan\;y)\frac{\delta y}{\delta x} &= 1 \\
\frac{1}{cos^2y} y’ &= 1 \\
y’ &= cos^2y
\end{align}
\]

Where we apply $\frac{\delta}{\delta x}$ to both sides (thats why the x turns into a 1). However, we wanted to find $\frac{\delta}{\delta x}tan^{-1}x$. We can simplify this by expressing $cos\;y$ in terms of it’s defining ratio:

which shows us the hypotenuse is $\sqrt{1 + x^2}$. This gives us:

\[
\begin{align}
cos\;y &= \frac{1}{\sqrt{1 + x^2}} \\
cos^2y &= \frac{1}{1 + x^2}
\end{align}
\]

and then we substitute in the above:

\[
\begin{align}
\frac{\delta}{\delta x}tan^{-1}x &= cos^2(tan^{-1}x) \\
&= \frac{1}{1 + x^2}
\end{align}
\]

Example 2:

\[
\begin{align}
y &= sin^{-1}x \\
sin y &= x \\
(cos\;y)y’ &= 1 \\
y’ &= \frac{1}{cos\;y} \\
y’ &= \frac{1}{\sqrt{1 – sin^2y}} \\
y’ &= \frac{1}{\sqrt{1 – x^2}} \\
\therefore \frac{\delta}{\delta x}sin^{-1}x &= \frac{1}{\sqrt{1 – x^2}}
\end{align}
\]

Exponentials & Logarithms

First, lets review exponents.
Consider some base, $a$, such that $a > 0$. We know that $a^0 = 0$, $a^1 = a$, $a^2 = a \times a$. Generally:
\[
a^{x_1 + x_2} = a^{x_1}a^{x_2}
\]

This is known as the law of exponents. From this we can derive:
\[
(a^{x_1})^{x_2} = a^{x_1x_2}
\]

Likewise:
\[
a^{\frac{m}{n}} = \sqrt[n]{a^m}
\]

$a^x$ is defined for all x by continuity. We can calculate things like $a^\pi$ and $a^{\sqrt{2}}$ using the rules above.

Example graph of $y = 2^x$:

WThe ultimate goal is to be able to solve
\[
\frac{\delta}{\delta x}a^x
\]

We can begin by going back to first principles:
\[
\begin{align}
\frac{\delta}{\delta x}a^x &= \lim_{\Delta x \to 0} \frac{a^{x + \Delta x} – a^x}{\Delta x} \\
&= \lim_{\Delta x \to 0} \frac{a^x a^{\Delta x} – a^x}{\Delta x} \\
&= \lim_{\Delta x \to 0} a^x \frac{a^{\Delta x} – 1}{\Delta x} \\
&= a^x \lim_{\Delta x \to 0} \frac{a^{\Delta x} – 1}{\Delta x}
\end{align}
\]

We move the $a^x$ out of the limit, because as the limit tends to 0, $a^x$ is constant, and can be moved out of the limit.
Next, lets define a new variable, and express the above equation in a new way:
\[
M(a) = \lim_{\Delta x \to 0} \frac{a^{\Delta x} – 1}{\Delta x}
\]
so that
\[
\frac{\delta}{\delta x}a^x = M(a)a^x
\]

If we recall back to the start of the course, we can recognise that as the equation of a line, the tangent line. From here, we can work out the slope of $a^x$. First, plug in $a = 0$ to the above:
\[
\begin{align}
\left. \frac{\delta}{\delta x}a^x \right|_{x = 0} &= M(a)a^0 \\
&= M(a)
\end{align}
\]

So we can see that the slope of $a^x$ at $x = 0$ is $M(a)$. What is $M(a)$?
Solving $M(a)$ is difficult. Lets beg the question, and instead, define $e$ as the number such that $M(e) = 1$.
The consequences of having a number such as $e$ are as follows:
\[
\begin{align}
\frac{\delta}{\delta x}e^x &= M(e)e^x \\
&= e^x
\end{align}
\]
which is based on the above derivation.
Also note that $M(e)$ at $x = 0$ is 1. That is, the slope of the tangent line at $x = 0$ is 1.

How do we know that a number such as $e$ exists?

Without defining $e$, we can show graphically that it must exist.

Consider the graph of the function $y = 2^x$, and compare the tangent line at $x = 0$ ($M(2)$) to the secant line through the points $(0,1), (1,2)$:

Notice the slope of the line $M(2)$ is less than the secant.

Now compare the graph of the function $y = 4^x$ and compare $M(4)$ against the same secant:

Notice the slope of the line is greater than the secant.

This shows that somewhere between the bases of 2 and 4, there is a number where $M(a) = 1$.

To fit this concept together with where we left the derivation off, we need to use the natural log.
If $y = e^x$, then $ln\;y = x$. That is, the natural log is the inverse of $e^x$.

A refresher on logarithms:
- $ln(x_1x_2) = ln\;x_1 + ln\;x_2$
- $ln\;1 = 0$
- $ln\;e = 1$

Next, we need to cover computing the derivative of a logarithm. Remember from implicit differentiation, we can compute the derivative of a function if we know the derivative of it’s inverse. That’s the rational behind finding the derivative.

First, lets define $w = ln\;x$. Using the above rules of $ln$ and $e$ we can show that:
\[
\begin{align}
e^w &= x \\
\frac{\delta}{\delta x}e^w &= \frac{\delta}{\delta x} x \\
&= 1 \\
(\frac{\delta}{\delta w}e^w)(\frac{\delta w}{\delta x}) &= 1 \\
e^w \frac{\delta w}{\delta x} &= 1 \\
\frac{\delta w}{\delta x} &= \frac{1}{e^w} \\
&= \frac{1}{x}
\end{align}
\]

See that $\frac{\delta}{\delta w}e^w = e^w$ based on the assumption that $M(e) = 1$.
For further info, check the lecture notes for this lecture.

Therefore we get:
\[
\frac{\delta}{\delta x}ln\;x = \frac{1}{x}
\]

To summarize, we have these two rates of change:
\[
\frac{\delta}{\delta x} e^x = e^x
\]
and
\[
\frac{\delta}{\delta x} ln\;x = \frac{1}{x}
\]

With these two derivatives in hand, we can return to the task of solving $M(a)$, and so differentiate $a^x$. There are two methods we can use to solve any exponential.

Method 1:

Convert the exponential to base $e$:
\[
\begin{align}
a^x &= (e^{ln\;a})^x \\
&= e^{x\;ln\;a}
\end{align}
\]
and then differentiate:
\[
\begin{align}
\frac{\delta}{\delta x}a^x &= \frac{\delta}{\delta x}e^{x\;ln\;a} \\
&= (ln\;a)e^{x\;ln\;a}
&= (ln\;a)a^x
\end{align}
\]
and we can see therefore that $M(a) = ln\;a$.

The above works because of 2 things, and can be illustrated as follows:
\[
\frac{\delta}{\delta x}e^{3x} = 3e^{3x}
\]
The 3 is the derivative of $3x$, and is multiplied by the derivative of $e^{3x}$, which is $e^{3x}$.
The same is occuring above. $ln\;a$ is a constant, and so we differentiate out the $x$ and multiply it by the derivative of $e$, which is itself.

Method 2:

The second method involves logarithmic differentiation.

Sometimes it’s easier to differentiate the logarithm of a function instead of the function itself.
Lets assume some function $u$, and solve:
\[
\begin{align}
\frac{\delta}{\delta u} ln\;u &= \left(\frac{\delta ln\;u}{\delta u}\right)\left(\frac{\delta u}{\delta x}\right) \\
&= \frac{1}{u} \frac{\delta u}{\delta x}
&= \frac{u’}{u}
\end{align}
\]
where we switch notation in the last step.

It’s easy to see where to go now from here to solve for $a^x$:
\[
\begin{align}
\frac{\delta}{\delta x} a^x \bigg| u = a^x \\
ln\;u &= x\;ln\;a \\
\frac{\delta}{\delta x} x\;ln\;a &= ln\;a \\
(a^x)’ &= (ln\;a)a^x
\end{align}
\]

Example 2 of method 2
An example of having moving exponents and a moving base.

Let’s consider the function:
\[
v = x^x
\]

Solve $\frac{\delta}{\delta x} v$ using logarithmic differentiation:
\[
\begin{align}
ln\;v &= x\;ln\;x \\
(ln\;v)’ &= ln\;x(x)’ + x(ln\;x)’ \\
&= ln\;x + x . \frac{1}{x} \\
&= ln\;x + 1 \\
\frac{v’}{v} &= ln\;x + 1 \\
v’ &= v(ln\;x + 1) \\
&= x^x(1 + ln\;x)
\end{align}
\]
Where on the second line, we apply the product rule.

Example 3

We’re going to evaluate:
\[
\lim_{n \to \infty}\left(1 + \frac{1}{n}\right)^n
\]
\[
ln \left( \left(1 + \frac{1}{n} \right)^n \right) = n\;ln\left(1 + \frac{1}{n}\right)
\]
here, we’re going to assign $\Delta x = \frac{1}{n}$, which makes $n = \frac{1}{\Delta x}$:
\[
n\;ln \left( 1 + \frac{1}{n} \right) = \frac{1}{\Delta x} (ln(1 + \Delta x) – ln\;1)
\]
here, we’re adding 0, in the form of $ln\;1$.
This results in an equation that matches the form of another equation we know:
\[
\begin{align}
\frac{ln(1 + \Delta x) – ln\;1}{\Delta x} &= \frac{\delta}{\delta x}ln\;x \quad \text{at x = 1} \\
&= \frac{1}{x} \quad \text{at x = 1} \\
&= 1
\end{align}
\]
Now we can just work back:
\[
\begin{align}
\lim_{n \to \infty}\left( 1 + \frac{1}{n} \right)^n &= e^{\left[ ln \left[ lim_{n \to \infty} \left( 1 + \frac{1}{n} \right)^n \right] \right]} \\
&= e^1 \\
&= e
\end{align}
\]

MIT OCW Linear Algebra - Week 2

2012-02-03T00:00:00-08:00

Matrix Multiplication

Consider the problem of multiplying two matrices:

\[
A \times B = C
\]

We can view this problem in 4 different ways.

The first way is as a row x columns problem.
Consider the element $C_{i,j}$. It is given by:

\[
\begin{align}
C_{i,j} &= A_{i,1} B_{1,j} + A_{i,2} B_{2,j} + \ldots \\
&= \sum_{k=1}^n A_{i,k} B_{k,j}
\end{align}
\]

This illustrates the matrix multiplication rule that:

\[
m \times n \;. n \times p = m \times p
\]

Namely, you can only multiply matrices if the first matrix has the same number of columns as the second does rows. The result of the multiplication is a matrix with the same amount of rows as the first matrix and the same amount of columns as the second matrix.

We can also think of multiplying matrices in terms of multiplying columns, or vectors. In the case of $AB = C$, we can think of $B$ as a group of column vectors: $A \times [\vec{b_1}, \vec{b_2}, \vec{b_3}] = [A \vec{b_1}, A \vec{b_2}, A \vec{b_3}] $. Each column of C is a combination of the columns of A.

Another way we can think of multiplying matrices is in terms of multiplying rows. In the case of $AB = C$, we can think of $A$ as a group of row vectors: $ \begin{bmatrix} \vec{a_1} \\ \vec{a_2} \\ \vec{a_3} \end{bmatrix} \times B = \begin{bmatrix} \vec{a_1}B \\ \vec{a_2}B \\ \vec{a_3}B \end{bmatrix} $. That is, each row of C is a combination of the rows of B.

Next, we can think of multiplying matrices in terms of columns x rows.
Remember a column is a m x 1 matrix, and a row is a 1 x n matrix. Multiplying these together gives:

\[
m \times 1 \;. 1 \times p = m \times p
\]

Consider $AB = C $ generally:

\[
\begin{bmatrix}
\vec{a_1} & \vec{a_2} & \vec{a_3}
\end{bmatrix}
\times
\begin{bmatrix}
\vec{b_1} \\
\vec{b_2} \\
\vec{b_3}
\end{bmatrix}
=
\vec{a_1} \vec{b_1} + \vec{a_2} \vec{b_2} + \vec{a_3} \vec{b_3}
\]

where $\vec{a}$ are column vectors of dimension $m \times 1$ and $\vec{b}$ are row vectors of dimension $1 \times n$, and $\vec{a}\vec{b}$ is a matrix of dimension $m \times n$.

Lastly, it should be noted that you can segment matrices and multiply the segments and still obtain the correct answer, so long as the segments are compatible.

For example:

\[
\begin{bmatrix}
a_1 & a_2 \\
a_3 & a_4
\end{bmatrix}
\times
\begin{bmatrix}
b_1 & b_2 \\
b_3 & b_4
\end{bmatrix}
=
\begin{bmatrix}
a_1 b_1 + a_2 b_3 & a_1 b_2 + a_2 b_4 \\
a_3 b_1 + a_4 b_3 & a_3 b_2 + a_b b_4
\end{bmatrix}
\]

where $a$ and $b$ are matrices, segments of a larger matrix.

Inverses

Inverses don’t always exist. Not every matrix is invertable. There are two kinds of inverses. Left inverse ($A^{-1}A = I$) and right inverse ($AA^{-1} = I$).

For a square matrix, $A^{-1}A = I = AA^{-1}$ is true, if A has an inverse.

A matrix that has an inverse is called invertible or nonsingular.

Singular Matrices

The following matrix is singular, and thus has no inverse:

\[
\begin{bmatrix}
1 & 2 \\
3 & 6
\end{bmatrix}
\]

The above matrix is singular because it’s determinate is 0. Determinates are covered later. It is also singular because each column is a multiple of the other, that is, they lie on the same line. When this matrix is multiplied with another matrix, the resultant matrix’s columns will be a combination of the above matrix’s columns – and they too will lie on the same line. There is no way to achieve $\begin{bmatrix}1 & 0 \\ 0 & 1 \end{bmatrix}$ because neither column of $I$ lies on that line.

Another reason a square matrix won’t have an inverse is that it is possible to find a non-zero vector $x$ such that $Ax = 0$:

\[
\begin{bmatrix}
1 & 3 \\
2 & 6
\end{bmatrix}
\begin{bmatrix}
3 \\ -1
\end{bmatrix}
= 0
\]

A matrix can’t have an inverse if some combination of it’s columns gives nothing.
If we assume A does have an inverse, then:

\[
\begin{align}
(Ax)A^{-1} &= 0A^{-1} \\
x &=0
\end{align}
\]

which we can see is clearly not true ($x = \begin{bmatrix} 3 \\ -1 \end{bmatrix} $), therefore this matrix does not have an inverse.

Non-singular matrices

Consider the following matrix:

\[
\begin{bmatrix}
1 & 3 \\
2 & 7
\end{bmatrix}
\]

This matrix is invertible and non-singular.

\[
\begin{bmatrix}
1 & 3 \\
2 & 7
\end{bmatrix}
\begin{bmatrix}
a & c \\
b & d
\end{bmatrix} =
\begin{bmatrix}
1 & 0 \\
0 & 1
\end{bmatrix}
\]

We need to find out the value of $A^{-1}$, $\begin{bmatrix} a & c \\ b & d \end{bmatrix}$.
Going back to matrix multiplication, we can see that we can decompose this into two problems:

\[
\begin{bmatrix}
1 & 3 \\
2 & 7
\end{bmatrix}
\begin{bmatrix}
a \\ b
\end{bmatrix}
=
\begin{bmatrix}
1 \\ 0
\end{bmatrix}
\]

and

\[
\begin{bmatrix}
1 & 3 \\
2 & 7
\end{bmatrix}
\begin{bmatrix}
c \\ d
\end{bmatrix}
=
\begin{bmatrix}
0 \\ 1
\end{bmatrix}
\]

These two equations can be solved at once using Gauss-Jordan. First, create an augmented matrix, with the left hand side as $A$ and the right hand side as $I$:

\[
\left[
\begin{matrix}
1 & 3 \\
2 & 7
\end{matrix}
\quad
\left|
\quad
\begin{matrix}
1 & 0 \\
0 & 1
\end{matrix}
\right.
\right]
\]

We can now perform Gaussian elimination on this augmented matrix:

\[
\left[
\begin{matrix}
1 & 3 \\
0 & 1
\end{matrix}
\quad
\left|
\quad
\begin{matrix}
1 & 0 \\
-2 & 1
\end{matrix}
\right.
\right]
\]

The left hand side is now in upper triangular form, and under normal Gaussian substitution, we’d finish. However in Gauss-Jordan, we then do a upwards elimination to achieve $I$ on the left side:

\[
\left[
\begin{matrix}
1 & 0 \\
0 & 1
\end{matrix}
\quad
\left|
\quad
\begin{matrix}
7 & -3 \\
-2 & 1
\end{matrix}
\right.
\right]
\]

The right hand side is now $A^{-1}$.

We can show how Gauss-Jordan works by the following sum:

\[
\begin{align}
E[A \quad I] &= [EA \quad EI] \\
&= [I \quad E]
\end{align}
\]

where E is the elimination matrix that performs the above calculation (augmented matrices). You can see that if $E \times A = I$, then $E$ must be $A^{-1}$, and therefore $E \times I = E$, which gives the result we observed above.
$E \times A = I$ was true because we made it true, by eliminating to $I$.

Inverse of a product

We know $AA^{-1} = I = A^{-1}A$.

For products, $AB B^{-1}A^{-1} = I$ This is due to the associative nature of matrix multiplication.
Performing the calculation:

\[
\begin{align}
AB ( B^{-1}A^{-1} ) &= I \\
A ( B B^{-1} ) A &= I \\
AI A^{-1} &= I \\
A A^{-1} &= I \\
I &= I
\end{align}
\]

The same holds for $B^{-1}A^{-1}AB = I$, which is placing the inverse group on the left of the equation.

A note about transposing. Two things about transposing a matrix:

When you transpose a matrix, you have to swap arguments:
\[
A A^{-1} = I = (A^{-1})^T A^T
\]
The inverse of a transposed matrix is the inverse of the matrix transposed:
\[
(A^{-1})^T = (A^T)^{-1}
\]

Factorization into A = LU

Elimination with no row matrices

Consider the following:

\[
E_{2,1}
\begin{bmatrix}
2 & 1 \\
8 & 7
\end{bmatrix}
=
\begin{bmatrix}
2 & 1 \\
0 & 3
\end{bmatrix}
\]

This is the first step in Gaussian elimination, where the first matrix is matrix $A$, and the last matrix is matrix $U$. The $E_{2,1}$ matrix elminates $A_{2,1}$ to make $U$ an upper-triangular matrix.

\[
\begin{bmatrix}
1 & 0 \\
-4 & 1
\end{bmatrix}
\begin{bmatrix}
2 & 1 \\
8 & 7
\end{bmatrix}
=
\begin{bmatrix}
2 & 1 \\
0 & 3
\end{bmatrix}
\]

is the solution for $E_{2,1}$.

To put this in the form of $A = LU$, we need to recognise that $L = E_{2,1}^{-1}$, and multiply both sides:

\[
\begin{array}{rcl}
( E_{2,1} E_{2,1}^{-1} )
\begin{bmatrix}
2 & 1 \\
8 & 7
\end{bmatrix}
=&
E_{2,1}^{-1}
\begin{bmatrix}
2 & 1 \\
0 & 3
\end{bmatrix}
\\
\begin{bmatrix}
2 & 1 \\
8 & 7
\end{bmatrix}
=&
\begin{bmatrix}
1 & 0 \\
4 & 1
\end{bmatrix}
\begin{bmatrix}
2 & 1 \\
0 & 3
\end{bmatrix}
\end{array}
\]

Notice that $U$ is an upper-triangle and $L$ is a lower-triangle. $U$ has the pivots in the matrix, and these can be pulled out:

\[
\begin{bmatrix}
2 & 1 \\
0 & 3
\end{bmatrix}
=
\begin{bmatrix}
2 & 0 \\
0 & 3
\end{bmatrix}
\begin{bmatrix}
1 & \frac{1}{2} \\
0 & 1
\end{bmatrix}
\]

where the first matrix on the right hand side is the matrix $P$ for pivot, and is a diagonal matrix, and the second matrix is $U$, the upper triangular matrix.

Now lets consider the same operations a larger matrix, assuming again we don’t perform a row exchange:

\[
E_{3,2}E_{3,1}E_{2,1}A = U
\]

Where we eliminate $A_{2,1}$ first, then from the resultant matrix we elminate $A_{3,1}$ and then from that matrix we eliminate $A_{3,2}$. To put this into the form of $A = LU$, we need to invert the matrices and move them to the other side of the equation:

\[
\begin{align}
A &= E_{2,1}^{-1}E_{3,1}^{-1}E_{3,2}^{-1}U \\
&= LU
\end{align}
\]

We perform $E_{2,1}^{-1}E_{3,1}^{-1}E_{3,2}^{-1}$ instead of $(E_{3,2}E_{3,1}E_{2,1})^{-1}$ because it’s easier to compute.

If we perform no row exchanges, the multipliers in $E$ used to eliminate elements of A go directly into $L$. For example:

\[
\begin{array}{rl}
E_{2,1}^{-1}E_{3,1}^{-1}E_{3,2}^{-1}U =&
\begin{bmatrix}
1 & 0 & 0 \\
0 & 1 & 0 \\
0 & 5 & 1
\end{bmatrix}
I
\begin{bmatrix}
1 & 0 & 0 \\
2 & 1 & 0 \\
0 & 0 & 1
\end{bmatrix}
U \\
=&
\begin{bmatrix}
1 & 0 & 0 \\
0 & 1 & 0 \\
0 & 5 & 1
\end{bmatrix}
\begin{bmatrix}
1 & 0 & 0 \\
2 & 1 & 0 \\
0 & 0 & 1
\end{bmatrix}
U \\
=&
\begin{bmatrix}
1 & 0 & 0 \\
2 & 1 & 1 \\
0 & 5 & 1
\end{bmatrix}
U \\
=&
LU
\end{array}
\]

In terms of computation, performing elimination on an $n \times n$ matrix is $O(\frac{1}{3}n^3)$ in the worst case. It’s no coincidence that this looks like the integral of $x^2$, as it represents a small change in $x^2$ over time.

There are 6 permutation matricies for $3 \times 3$ matricies:

For permutations matrices, $P^{-1} = P^T$. In general, there are $n!$ for an $n \times n$ matrix.

MIT OCW Single Variable Calculus - Week 2

2012-01-25T00:00:00-08:00

Week 2

Derivative Formulas

To work towards the goal of “differentiating everything”, need to talk about derivative formulas.

Specfic – $f’(x)$, where $f(x) = x^n, \frac{1}{x}$ as a few examples.
General – $(u + v)’ = u’ + v’$ and $(cu)’ = cu’$ where c is constant, as a few examples.

Start with specific trigonometric derivative formulas.

$\frac{\delta}{\delta x} sin \; x$

\[
\frac{\delta}{\delta x} sin \; x = \lim_{x \to 0} \frac{sin(x + \Delta) \; – sin(x)}{\Delta x}
\]

To solve this, we need to revist the trigonometric sum and difference formulas, namely:

\[
sin(A + B) = sin A \; cos B + cos A \; sin B
\]

This is a proof of the identity, using Eulers formula

Going back to the derivative:

\[
\begin{align}
\frac{\delta}{\delta x} sin \; x &= \lim_{x \to 0} \frac{sin(x + \Delta) \; – sin(x)}{\Delta x} \\
&= \lim_{x \to 0} \frac{sin\:x \; cos\:\Delta x + cos\;x \; sin\:\Delta x – sin\:x}{\Delta x} \\
&= \lim_{x \to 0} \quad sin\:x(\frac{cos\:\Delta x – 1}{\Delta x}) + cos\:x(\frac{sin\:\Delta x}{\Delta x}) \\
&= cos\:x
\end{align}
\]

due to the fact that $\frac{cos\:\Delta x – 1}{\Delta x}$ tends to 0 as $\Delta x$ tends to 0, and $\frac{sin\:\Delta x}{\Delta x}$ tends to 1 as $\Delta x$ tends to 0. Label these as A and B respectively, as we will prove them later.

$\frac{\delta}{\delta x} cos x$

Same as above, only this time using the identity:

\[
cos(A + B) = cos A \; cos B – sin A \; sin B
\]

\[
\begin{align}
\frac{\delta}{\delta x} cos \; x &= \lim_{x \to 0} \frac{cos(x + \Delta) \; – cos(x)}{\Delta x} \\
&= \lim_{x \to 0} \frac{cos\;x \: cos\;\Delta x – sin\;x \: sin\;\Delta x – cos\;x}{\Delta x} \\
&= \lim_{x \to 0} \quad cos\;x(\frac{cos\;\Delta x – 1}{\Delta x}) – sin\;x(\frac{\sin\;\Delta x}{\Delta x}) \\
&= \quad – sin\;x
\end{align}
\]

again, by A and B as stated above.

Proofs of A and B

First, some remarks:

\[
\frac{\delta}{\delta x} cos\;x \quad \Bigg|^{x=0}
\quad \lim_{\Delta x \to 0} \frac{cos(\Delta x) – 1}{\Delta x}
\]

This shows that the rate of change of $cos x$ at $x = 0$ is a limit as $\Delta x$ tends to 0, which has been simplified:

\[
\begin{align}
&\frac{cos(0 + \Delta x) – cos(0)}{\Delta x} \\
= &\frac{cos(\Delta x) – 1}{\Delta x}
\end{align}
\]

and likewise

\[
\frac{\delta}{\delta x} sin\;x \quad \Bigg|^{x=0}
\quad \lim_{\Delta x \to 0} \frac{sin(\Delta x)}{\Delta x}
\]

because of the simplification:

\[
\begin{align}
&\frac{sin(0 + \Delta x) – sin(0)}{\Delta x} \\
= &\frac{sin(\Delta x)}{\Delta x}
\end{align}
\]

We should note that the derivatives of sine and cosine at $x = 0$ give all values of $\frac{\delta}{\delta x} sin x$, $\frac{\delta}{\delta x} cos x$

Back to the proofs:

Property B

Geometric proof:

Lets rename $\Delta x$ as $\theta$.

Consider the ratio of the arc length to the line $sin \theta$. Then double it for the two $\theta$ that were drawn:

\[
\begin{align}
&\frac{2 sin \theta}{2 \theta} \\
= &\frac{sin \theta}{\theta}
\end{align}
\]

Now, as $\theta$ approaches 0, the lines $sin \theta$ and $arclength \theta$ get close and close to the same line, so the result of the ratio is 1, intuitively because short pieces of curves are nearly straight.

Property A

The basis for the proof for cosine is the same as the proof for sine – that of short pieces of curves being nearly straight.

First, $cos \theta$ is going to be negative, so we need to invert the statement:

\[
\frac{1 – cos \theta}{\theta}
\]

This is a similar image as before, only magnified.
The distance from the centre of the circle to the circumference is 1 (because this is a unit circle, and the gap between the $sin \theta$ line and the circumference is $1 – cos \theta$. As $\theta$ tends to 0, this gap gets smaller and smaller, and as such, $\frac{1 – cos \theta}{\theta}$ tends to 0.

NOTE TO SELF – RECORD GEOMETRIC PROOF

General Derivative Rules

Product Rule

\[
(uv)’ = u’v + uv’
\]

can be remembered as changing one variable at a time. Will be proven as such.

Example:

\[
\frac{\delta}{\delta x} (x^n sin\;x) = nx^{n-1}sin\;x + x^n cos\;x
\]

Proof:

Going back to first principles of calculus, we can rewrite the sum in terms of the rate of change:

\[
\begin{align}
\frac{\delta}{\delta x} uv &= \lim_{\Delta x \to 0} \frac{u(x + \Delta x)v(x + \Delta x) – u(x)v(x)}{\Delta x} \\
&= \lim_{\Delta x \to 0} \frac{u(x + \Delta x)(v(x + \Delta x) – v(x)) + v(x)(u(x + \Delta x) – u(x))}{\Delta x} \qquad \text{(1)} \\
&= \lim_{\Delta x \to 0} u(x + \Delta x)\frac{\delta v}{\delta x} + v(x)\frac{\delta u}{\delta x} \\
&= uv’ + vu’
\end{align}
\]

We converted the sum in (1) to a new form to aid in the proof. You can multiply out (1) to show that it is indeed equal:

\[
\begin{align}
u(x + \Delta x)(v(x + \Delta x) – v(x)) + v(x)(u(x + \Delta x) – u(x)) &= u(x + \Delta x)v(x + \Delta x) – v(x)u(x + \Delta x) + v(x)u(x + \Delta x) – v(x)u(x) \\
&= u(x + \Delta x)v(x + \Delta x) – v(x)u(x)
\end{align}
\]

As $\Delta x$ tends to 0, then $u(x + \Delta x)$ approaches $u(x)$, which will complete the proof.

Quotient Rule

\[
\left(\frac{u}{v}\right)’ = \frac{u’v – uv’}{v^2}
\]

only works when $v \ne 0$.

Proof:

First, recall that $\frac{\delta}{\delta x}$ is equal to $\frac{\Delta y}{\Delta x}$ and we can compute $\Delta y$ in a separate step, and then just divide by $\Delta x$.

\[
\begin{align}
\Delta \left( \frac{u}{v} \right) &= \frac{u + \Delta u}{v + \Delta v} – \frac{u}{v} \\
&= \frac{uv + v(\Delta u) – uv – u(\Delta v)}{(v + \Delta v)v} \\
&= \frac{v(\Delta u) – u(\Delta v)}{(v + \Delta v)v} \\
\frac{\Delta \left( \frac{u}{v} \right) }{\Delta x} &= \frac{ \frac{\Delta u}{\Delta x}v – u\frac{\Delta v}{\Delta x} }{(v + \Delta v)v} \\
\frac{\delta}{\delta x} \left( \frac{u}{v} \right) &= \lim_{\Delta x \to 0} \frac{ \frac{\delta u}{\delta x}v – u\frac{\delta v}{\delta x} }{v^2} \\
&= \frac{u’v – uv’}{v^2}
\end{align}
\]

Let us reconsider the above, setting $u = 1$:

\[
\begin{align}
\frac{\delta}{\delta x} \left( \frac{1}{v} \right) &= \frac{-1v’}{v^2} \\
&= -v^{-2}v’
\end{align}
\]

and now, lets consider the above example generally, with $\frac{1}{x^n}$ represented as $x^{-n}$:

\[
\begin{align}
\frac{\delta}{\delta x} x^{-n} &= \frac{\delta}{\delta x} \left( \frac{1}{x^n} \right) \\
&= -x^{-2n}nx^{n-1} \\
&= -nx^{-n-1}
\end{align}
\]

You’ll notice that this follows the same pattern that applies to $x^n$.

Chain rule

We’ll start by derriving the rule, then writing it down.

Proof:

\[
\begin{align}
\frac{\Delta y}{\Delta t} &= \frac{\Delta y}{\Delta x} \frac{\Delta x}{\Delta t} \\
\frac{\delta y}{\delta t} &= \frac{\delta y}{\delta x} \frac{\delta x}{\delta t}
\end{align}
\]

TODO: FIND A BETTER PROOF OF THIS, OR SHOW A MORE EXPLICIT EXAMPLE *

Example:

\[
\begin{align}
\frac{\delta}{\delta t} (sin\;t)^{10} &= x^{10} & \text{Substituting x = sin t} \\
&= 10x^9 cos\;t \\
&= 10(sin\;t)^9 cos\;t & \text{Substituting the value of x back in}
\end{align}
\]

Example 2:

\[
\begin{align}
\frac{\delta}{\delta t} sin(10t) &= sin(x) & \text{Substituting x = 10t} \\
&= cos(x) \times 10 \\
&= cos(10t) \times 10 & \text{Substituting 10t = x back in}
\end{align}
\]

Higher derrivatives *

Refers to multiple steps of differentiation.

Consider the function $u = u(x)$, then $u’$ is a new function in and of itself. We can differentiate this function: $u’’$. This is the second derivative, and it too is a new function, and can be differentiated.

You can continue this to any number of higher derrivatives, second, third, fourth, etc.

Notation:
\[
\begin{align}
u’ &= \frac{\delta u}{\delta x} = \frac{\delta}{\delta x} u \\
u’’ &= \frac{\delta}{\delta x}\frac{\delta u}{\delta x} = \frac{\delta}{\delta x}\frac{\delta}{\delta x} u \\
&= \left(\frac{\delta}{\delta x}\right)^2 u \\
&= \frac{\delta^2}{(\delta x)^2} u \\
&= \frac{\delta^2}{\delta x^2} u
\end{align}
\]

Example:

\[
\begin{align}
D^nx^n &= ? \\
Dx^n &= nx^{n-1} \\
D^2x^n &= n(n-1)x^{n-2} \\
D^3x^n &= n(n-1)(n-2)x^{n-3} \\
D^{n-1}x^n &= (n(n-1)(n-2) \ldots 2)x^1 \\
D^nx^n &= (n(n-1)(n-2) \ldots 2 \times 1) \times 1 \\
&= n!
\end{align}
\]

MIT OCW Single Variable Calculus - Week 1

2012-01-18T00:00:00-08:00

Differentiation

What is a derivative?

Geometric interpretation

Consider the problem of finding the tangent line to some function $y=f(x)$ at $P=(x_0,y_0)$:

The tangent line has an equation of the form $y – y_0 = m(x – x_0)$, where m is the slope or gradient of the line.
Finding $y_0$ is easy – $y_0 = f(x_0)$.
Finding m involves calculus – $m = f’(x_0)$.
That is, $f’(x_0)$, the derivative of $f$ at $x_0$ is the slope of the tangent line to $y = f(x)$ at P.

Consider the following graph:

How do we know which of these two lines is the tangent, given that both pass through the same point?
Lets define the point where the secant and the tangent intersect as P, and the other point on the secant as Q.
As the point Q gets closer to the point P, the slope of the secant line starts to match the slope of the tangent line.
Therefore:
The tangent is the limit of the secant line $PQ$ as the point $Q \rightarrow P$, assuming P is fixed.

Again consider the secant line $PQ$.
Calculating the gradient of $PQ$:
The difference between Q’s x value and P’s x value is known as $\Delta x$ (delta x), or the change in x.
The change in y is called $\Delta f$ (in this case?).
The gradient between $P$ and $Q$ is the ratio of $\frac{\Delta f}{\Delta x}$.
As the point Q gets closer to P, $\Delta x$ gets smaller.
Thus $m = \lim_{\Delta x \to 0} \frac{\Delta f}{\Delta x}$

If P is the point $(x_0, f(x_0))$, then we can represent Q as $(x_0 + \Delta x, f(x_0 + \Delta x))$.
Therefore $m = \lim_{\Delta x \to 0} \frac{f(x_0 + \Delta x) – f(x_0)}{\Delta x}$.
This is the heart of the derivative.

Example:
Consider the graph for $\frac{1}{x}$

Find the tanget at point $x_0$:
$$m = \lim_{\Delta x \to 0} \frac{ \frac{1}{x_0 + \Delta x} – \frac{1}{x_0} }{\Delta x}$$
$$m = \lim_{\Delta x \to 0} \frac{1}{\Delta x}\left( \frac{ x_0 – (x_0 + \Delta x) }{ (x_0 + \Delta x)x_0 } \right)$$
$$m = \lim_{\Delta x \to 0} \frac{1}{\Delta x}\left( \frac{ -\Delta x }{ (x_0 + \Delta x)x_0 } \right)$$
$$m = \lim_{\Delta x \to 0} -\frac{1}{ (x_0 + \Delta x)x_0 }$$
$$m \rightarrow -\frac{1}{ x_0^2 }$$

Example 2:
Find the area of the triangle formed by the axes and the curve described by $y = \frac{1}{x}$.
This is a word problem with only one part of calculus – the rest is regular math.
Graph the problem:

The only part of the problem that involves calculus is finding the tangent. We found the gradient $m$ above, so lets use that.
$y – y_0 = -\frac{1}{x_0^2}(x – x_0)$
To calculate the area of the triangle, we need the base length (along the $x$ axis) and the base height (along the $y$ axis).
We can find those by calculating the $x$ intercept and $y$ intercepts.
$x$ intercept is at $y = 0$, and we can express $y_0$ in terms of $f(x)$:
$$ – \frac{1}{x_0} = -\frac{1}{x_0^2}(x – x_0) $$
$$ – \frac{1}{x_0} = -\frac{x}{x_0^2} + \frac{1}{x_0} $$
$$ \implies \frac{x}{x_0^2} = \frac{2}{x_0} $$
$$ \implies x = 2x_0 $$
Using symmetry, we can find the $y$ intercept quickly.
Firsly, a note about symmetry:
$$ y = \frac{1}{x} \implies xy = 1 \implies x = \frac{1}{y} $$
$$ \therefore y = 2y_0 $$
The area of a triangle is $\frac{1}{2}$ base x perpendicular height.
$\therefore area = \frac{1}{2}(2_x0)(2y_0) = 2x_0y_0$ = 2
The area is 2 because $y = \frac{1}{x}$, and $xy$ will always equal 1.

Notation:
$y = f(x), \Delta y = \Delta f$
$f’ = \frac{\delta f}{\delta x} = \frac{\delta y}{\delta x} = \frac{\delta}{\delta x}f = \frac{\delta}{\delta x}y$

Example 3:
$y = \frac{\delta}{\delta x}x^n$
$$ \frac{\Delta f}{\Delta x} = \lim_{\Delta x \to 0} \frac{(x + \Delta x)^n – x^n}{\Delta x} $$
In order to solve this, we need to revist the binomial theorem (algebra).
Binomial theorem states that:
$ (x + \Delta x)^n = (x + \Delta x) … (x + \Delta x) $
$ = x^n + nx^{n-1} + O((\Delta x)^n)$
continuing:
$$ \frac{\Delta f}{\Delta x} = \frac{1}{\Delta x}\big((x + \Delta x)^n – x^n \big) $$
$$ \frac{\Delta f}{\Delta x} = \frac{1}{\Delta x}\big(x^n + nx^{n-1}\Delta x + O((\Delta x)^n) – x^n \big) $$
$$ \frac{\Delta f}{\Delta x} = \frac{1}{\Delta x}\big(nx^{n-1}\Delta x + O((\Delta x)^n) \big) $$
$$ \frac{\Delta f}{\Delta x} = nx^{n-1} + O((\Delta x)^n) $$
$$ \frac{\Delta f}{\Delta x} = \lim_{\Delta x \to 0} nx^{n-1} + O((\Delta x)^n) = nx^{n-1} $$
$$ \therefore \frac{\delta}{\delta x}x^n = nx^{n-1} $$

Rate of Change

Geometrically, a derivative can be thought of as the slope of a tangent line for a curve.
It can also be considered from the point of view of relative rate of change. That is, the change in variables $x$ and $y$ for a given function $f(x)$.
$\frac{\Delta x}{\Delta y}$ is the average rate of change – that is, the entire change over the entire interval.
$\frac{\delta x}{\delta y}$ is the average rate of change limited, and is the instantaneous rate of change.

Examples:
1. q = charge, $\frac{\delta q}{\delta t}$ = current. The rate of change in charge over time is current.
2. s = distance, $\frac{\delta s}{\delta t}$ = speed. The rate of change in distance over time is speed.

Consider the MIT pumpkin drop:
From the top of one of the buildings in MIT, on Halloween, students drop pumpkins from the top of building to the ground.
Assume the building is 80m high, and assume the rate of acceleration due to gravity is 5m² (not 9.8m²).
$ h = 80 – 5t^2 $
at t = 0, h = 80
at t = 4, h = 0

The average speed is:
$\frac{\Delta h}{\Delta t} = \frac{0 – 80}{4 – 0} = -20m/s $

The instantaneous speed as it hits the ground is:
$\frac{\delta}{\delta t}h = 0 – 10t$ (differentiated the function for height)
at t = 4, instantaneous speed is -40m/s.

Examples that don’t involve time:
3. T = temperature, $\frac{\delta T}{\delta x}$ = temperature gradient (used in weather forecasting & airflow)
4. Sensitivity of measurements

Limits & Continuity

Left & right hand limits.
$\lim_{x \to x_0^+}$ – right hand limit. Says that x is bigger than $x_0$ and we’re approaching $x_0$ with a negative $\Delta$
$\lim_{x \to x_0^-}$ – left hand limit. Says that x is smaller than $x_0$ and we’re approaching $x_0$ with a positive $\Delta$

Example:
$$
f(x) = \left\{
\begin{array}{l l}
x + 1 & \quad x > 0\\
-x + 2 & \quad x < 0\\
\end{array} \right.
$$

$$\lim_{x \to x_0^+} f(x) = \lim_{x \to x_0} x + 1 = 1$$

$$\lim_{x \to x_0^-} f(x) = \lim_{x \to x_0^-} -x + 2 = 2$$

Note: we didn’t need x = 0 value.

Continuity:
$f(x)$ is continuous at $x_0$ when $\lim_{x \to x_0} = f(x_0$
Requirements:
- $\lim_{x \to x_0} f(x)$ exists (from left & right, and left & right must be the same)
- $f(x_0)$ is defined
- Both of the above are equal to each other

Continuous functions are easier to calculate limits for because you can just insert the $x_0$ term into the function and determine the limit.

Discontinuity:
If $f(x)$ is not continuous, then it is discontinuous.
Jump discontinuity – Limit from left and right exist but are not equal
Removable discontinuity – Limit from left and right are equal. Discontinuity can be removed by redefining the function for the point of discontinuity.
Example:
$$
\begin{array}{1 1}
g(x) = \frac{sin x}{x} \\
h(x) = \frac{1 – cosx}{x}
\end{array}
$$
g(0) is undefined, but:
$$
\begin{array}{1 1}
\lim_{x \to 0} \frac{sin x}{x} = 1 \\
\lim_{x \to 0} \frac{1 – cos x}{x} = 0
\end{array}
$$

These functions have a removable discontinuity at x = 0

Infinite discontinuity
Consider $\frac{1}{x}$:
$$
\begin{array}{1 1}
\lim_{x \to x_0^+} \frac{1}{x} = \infty \\
\lim_{x \to x_0^-} \frac{1}{x} = -\infty
\end{array}
$$

Other (ugly) discontinuities
$y = sin \frac{1}{x} \text{as} \; x \to 0$
No left or right limit

Differential implies continuous theorem

If f is differentiable at $x_0$, then x is continuous at $x_0$
Proof:
$$
\lim_{x \to x_0} f(x) – f(x_0) = 0
$$

$$
\lim_{x \to x_0} \frac{f(x) – f(x_0)}{x – x_0} (x – x_0) = f’(x_0) . 0 = 0
$$

MIT OCW Single Variable Calculus

2012-01-18T00:00:00-08:00

THIS IS A PLACEHOLDER SO THAT THE INDEX APPEARS IN THE PROJECT LIST

D&D Monster Tokens

2012-01-09T00:00:00-08:00

Everyone knowns how much easier combat is to understand when you have some spatial aids. Wizards of the Coast and other gaming companies are pushing hard on minis, but they’re fairly expensive and take up a lot of storage room. Did I also mention they happen to be expensive to collect a lot?

As an alternative, you can use tokens, and make them at home. I used to think home-made tokens looked cheap, because the tokens I saw people use had hodge-podge artwork on them, in different styles and mediums, and there was no sense of cohesion.

Which is why I decided to make my own tokens. I like to think I’ve come up with a pretty streamline way to make tokens fast, and I’m quickly expanding my repetoire.

Here are the things you need:

- One of these bad boys:

which is a circle punch used in scrapbooking, available at any good craft store, including Michaels here in the USA. This will do for most medium sized creatures, but if you need to go larger, you can use one of these:

which is a circle cutter, and should do 1" – 5" circles, which should be good for any monster you can come up with.
- Some contact paper, available at any stationary store.
- Some card stock
- A computer and a printer.

Using these supplies, you can make your own tokens fairly quickly and easily. Now, I like to draw, and I’m fairly good at it, so I draw my own tokens. For those of you who either don’t have the time, inclination or skill to draw your own, I’m making mine available for free.

As you can see, the art is consistent, and the templates are pretty easy to use. Simply print the template onto card stock, contact both sides so you can write on them with dry erase markers, line up the circle punch and punch them out. Instant tokens.

I’m slowly adding more monsters as my adventuring party encounter them, so periodically check back for updated monsters.
Here are the monsters that I have currently:
- Zombie
- Skeleton
- Hobgoblin
- Spider
- Ghoul
- Homunculus
- Evil cleric
- Human berzerker
- Gelatinous cube (coming soon)
- Goblin (coming soon)
- Kobold (coming soon)

(current as of 01-09-2012, and we play once a week, with new monsters twice a month or so)

MIT OCW Linear Algebra - Week 1

2012-01-06T00:00:00-08:00

Linear equations

n equations, n unknowns – easy to solve.

Example:
$$
\begin{array}{1 1}
2x – y = 0 \\
-x + 2y = 3
\end{array}
$$
can be solved with matrices as follows:
$$
\begin{bmatrix}
2 & -1 \\
-1 & 2
\end{bmatrix}
\begin{bmatrix}
x \\
y
\end{bmatrix}
=
\begin{bmatrix}
0 \\
3
\end{bmatrix}
$$

The matrix of co-efficients will be called $A$, the vector of unknowns will be called $X$ and the resultant matrix will be called $b$.

Row picture

The point of intersection is the solution to this pair of linear equations.

Column picture

We can reinterpret these two equations into a single matrix equation:
$$
x \begin{bmatrix}
2 \\
-1
\end{bmatrix} + y \begin{bmatrix}
-1 \\
2
\end{bmatrix} = \begin{bmatrix}
0 \\
3
\end{bmatrix}
$$

Now the equation is asking us to find the correct linear combination of the columns, or vectors, that solves the equation.
The linear combination is the proportion of each vector that solves the equation.

Lets plug in the correct results from above, and take 1 of $\begin{bmatrix}2\\-1\end{bmatrix}$ and 2 of $\begin{bmatrix}-1\\2\end{bmatrix}$:

As you can see, we get the answer $b$, which is $\begin{bmatrix}0\\3\end{bmatrix}$

Example 2
Now lets consider 3 equations with 3 unknowns:

$$
\begin{array}{1 1}
2x – y = 0 \\
-x + 2y – z = -1 \\
-3y + 4z = 4
\end{array}
$$

Matrix form:

$$
A =
\begin{bmatrix}
2 & -1 & 0 \\
-1 & 2 & -1 \\
0 & -3 & 4
\end{bmatrix}
\quad
b =
\begin{bmatrix}
0 \\
-1 \\
4
\end{bmatrix}
$$

Row picture:

A 3D cartesian axis with 3 planes that meet in a point. Omitting the plot due to the complexity of the problem.
This points to a weakness in this interpretation of the problem in higher dimensions.

Column picture:

$$
x
\begin{bmatrix}
2 \\
-1 \\
0
\end{bmatrix} + y
\begin{bmatrix}
-1 \\
2 \\
-3
\end{bmatrix} + z
\begin{bmatrix}
0 \\
-1 \\
4
\end{bmatrix} =
\begin{bmatrix}
0 \\
-1 \\
4
\end{bmatrix}
$$

The solution is $x = 0, y = 0, z = 1$, which is clearly seen from the matrices.

Consider the case where we have a different RHS:

$$
x
\begin{bmatrix}
2 \\
-1 \\
0
\end{bmatrix} + y
\begin{bmatrix}
-1 \\
2 \\
-3
\end{bmatrix} + z
\begin{bmatrix}
0 \\
-1 \\
4
\end{bmatrix} =
\begin{bmatrix}
1 \\
1 \\
-3
\end{bmatrix}
$$

You can see the solution is $x = 1, y = 1, z = 0$.

Consider all RHS sides? Can we solve $Ax = b$ for every b? To restate, do the linear combinations of the columns fill 3D space?
The answer for the above matrix A, the answer is yes. This means the matrix is non-singular and invertable.
If the answer is no, then the matrix is singular and non-invertable.

For example, if the three vectors lie in the same plane, then the combinations will lie in the same plane – but not all of 3D space.

A matrix is singular and non-invertable if the columns are linearly independent from each other.

Multiplying a matrix by a vector:
Column way, as opposed to the row way:

$$
\begin{bmatrix} 2 & 5 \\ 1 & 3 \end{bmatrix}
\begin{bmatrix} 1 \\ 2 \end{bmatrix}
$$
$$
= 1 \begin{bmatrix} 2 \\ 1 \end{bmatrix} + 2 \begin{bmatrix} 5 \\ 3 \end{bmatrix}
$$
$$
= \begin{bmatrix} 12 \\ 7 \end{bmatrix}
$$

So you can see that $Ax$ is a combination of the columns of $A$

Matrix Elimination

Discovered by Gauss.

We will be working from the following example equations:

$$
\begin{array}{1 3}
x + 2y + z = 2 \\
3x + 8y + z = 12 \\
4y + z = 2
\end{array}
$$

the coefficients of which can be represented by the following matrix:

$$
A = \begin{bmatrix}
1 & 2 & 1 \\
3 & 8 & 1 \\
0 & 4 & 1
\end{bmatrix}
$$

the answer with:

$$
b = \begin{bmatrix}
2 \\
12 \\
2
\end{bmatrix}
$$

and the full equation represented by:

$$
Ax = b
$$

To begin, the purpose of elimination is to get rid of as many unknowns as we can and then solve for the remaining unknown.
The first step in this case would be to subtract $x + 2y + z = 2$ from $3x + 8y + z$ with the goal of eliminating the $x$ variable.

Firstly, declare the $x$ coefficient in the first row of matrix $A$ to be the pivot. Then find the multiple of the pivot required to eliminate it. In this case, it’s 3. Then subtract the first row multiplied by the multiple from the second row. This will result in:

\[
\begin{matrix}
1 & 2 & 1 \\
0 & 2 & -2 \\
0 & 4 & 1
\end{matrix}
\]

This has eliminated $x$ from the second row, the [2, 1] position. The next step would be to eliminate $x$ from the [3, 1] position, however there is no need to do this, as it is already 0.

Next, eliminate $y$ from the third row. Set the pivot to be the $y$ coefficient in $A$ and find the multiple required to eliminate it. The multiple is 2. Then subtract the second row multiplied by the multiple from the third row:

\[
\begin{matrix}
1 & 2 & 1 \\
0 & 2 & -2 \\
0 & 0 & 5
\end{matrix}
\]

This results in a type of matrix called an upper-triangular matrix. This is the form of matrix we need to complete this form of elimination. We can call this matrix $u$.

Note: pivots can’t be 0.

This process will fail if there’s a 0 in a pivot position. This will require reorganizing the matrix if possible, so that there is a non-zero number in the pivot position. If you can’t find 3 pivots, the matrix is non-invertable and the solution cannot be found.

Back substitution

After finding the multiples, we can apply the same algorithm to the right hand side of the equation:

\[
b =
\begin{matrix}
2 \\ 12 \\ 2
\end{matrix}
\quad
\rightarrow
\quad
b =
\begin{matrix}
2 \\ 6 \\ 2
\end{matrix}
\quad
\rightarrow
\quad
b =
\begin{matrix}
2 \\ 6 \\ -10
\end{matrix}
\]

This leaves us with the augmented matrix:

\[
\begin{matrix}
1 & 2 & 1 \\
0 & 2 & -2 \\
0 & 0 & 5
\end{matrix}
\left|
\begin{matrix}
2 \\ 6 \\ -10
\end{matrix}
\right.
\]

and we can see that we have the equation $5z = -10$ and that $z = -2$.

We can then substitute that back in to the above equation for $2y – 2(-2) = 6$ to get $y = 1$, and then substitute that back into the first equation for $x + 2(1) + 1(-2) = 2$ and get $x = 2$.

Thus using elimination and substitution we have solved the equation with $x = 2$, $y = 1$ and $z = -2$

Matrices

Now we’re going to represent the above calculation in matrix form. We want to perform elimination on the following matrix:

\[
\begin{bmatrix}
1 & 2 & 1 \\
3 & 8 & 1 \\
0 & 4 & 1 \\
\end{bmatrix}
\]

First, we want to find the matrix that will subtract 3 times row 1 from row 2, and leaves the other rows. So we want to find the matrix $X$ that solves the following:

\[
X \begin{bmatrix}
1 & 2 & 1 \\
3 & 8 & 1 \\
0 & 4 & 1
\end{bmatrix}
\quad = \quad
\begin{bmatrix}
1 & 2 & 1 \\
0 & 2 & -2 \\
0 & 4 & 1
\end{bmatrix}
\]

Lets go back to the concept of multiplying matrices first, in order to solve this intuitively.

Consider this problem:

\[
\begin{bmatrix}
x_{1,1} & y_{1,2} & z_{1,3} \\
x_{2,1} & y_{2,2} & z_{2,3} \\
x_{3,1} & y_{3,2} & z_{3,3}
\end{bmatrix}
\begin{bmatrix}
4 \\ 5 \\ 6
\end{bmatrix}
\]

This is saying take 4 of the first column, 5 of the second column and 6 of the third column:

\[
\begin{bmatrix}
4 (x_{1,1} + x_{2,1} + x_{3,1}) \\
5 (y_{1,2} + y_{2,2} + y_{3,2}) \\
6 (z_{1,3} + z_{2,3} + z_{3,3})
\end{bmatrix}
\]

\[
= \begin{bmatrix}
12x \\
15y \\
18z
\end{bmatrix}
\]

which is a combination of the columns of the matrix. This will result in a column vector.

Where as this problem:

\[
\begin{bmatrix}
4 & 5 & 6
\end{bmatrix}
\begin{bmatrix}
x_{1,1} & y_{1,2} & z_{1,3} \\
x_{2,1} & y_{2,2} & z_{2,3} \\
x_{3,1} & y_{3,2} & z_{3,3}
\end{bmatrix}
\]

is saying "take 4 of the first row, 5 of the second row and 6 of the third row:

\[
\left[
\begin{array}{3 1}
4 (x_{1,1} + y_{1,2} + z_{1,3}) + \\
5 (x_{2,1} + y_{2,2} + z_{2,3}) + \\
6 (x_{3,1} + y_{3,2} + z_{3,3})
\end{array}
\right]
\]

\[
= \begin{bmatrix}
15x & 15y & 15z
\end{bmatrix}
\]

which is a combination of the rows of the matrix. This will result in a row vector.

Now going back to the problem at hand:

\[
X \begin{bmatrix}
1 & 2 & 1 \\
3 & 8 & 1 \\
0 & 4 & 1
\end{bmatrix}
\quad = \quad
\begin{bmatrix}
1 & 2 & 1 \\
0 & 2 & -2 \\
0 & 4 & 1
\end{bmatrix}
\]

You can see that we need 1 of the first row, and 1 of the last row to solve this equation. Thus, we can see that:

\[
X =
\begin{bmatrix}
1 & 0 & 0 \\
x_{2,1} & x_{2,2} & x_{2,3} \\
0 & 0 & 1
\end{bmatrix}
\]

because if you consider $\begin{bmatrix} 1 & 0 & 0 \end{bmatrix}$ as a row vector, that will take 1 of the first row of the matrix and none of the second and third rows of the matrix, which is what the first row of the answer is. Likewise for the last row.

Now we need the row that will subtract 3 of the first row from the second row. We can see the answer is:

\[
X =
\begin{bmatrix}
1 & 0 & 0 \\
-3 & 1 & 0 \\
0 & 0 & 1
\end{bmatrix}
\]

We can also call this matrix $E_{2,1}$, the elimination matrix to eliminate the 2,1 position of the co-efficient matrix.

Now we need to represent step 2, which will eliminate the position 3,2, matrix $E_{3,2}$:

\[
E_{3,2}
\begin{bmatrix}
1 & 2 & 1 \\
0 & 2 & -2 \\
0 & 4 & 1
\end{bmatrix}
\quad = \quad
\begin{bmatrix}
1 & 2 & 1 \\
0 & 2 & -2 \\
0 & 0 & 5
\end{bmatrix}
\]

Using the above steps, we know that:

\[
E_{3,2} =
\begin{bmatrix}
1 & 0 & 0 \\
0 & 1 & 0 \\
0 & -2 & 1
\end{bmatrix}
\]

We can now roll both steps into a single step, a single matrix $E$ that does all the steps at once.
Firstly, lets represet what we did in an equation:

\[
E_{3,2}(E_{2,1}A) = u
\]

where u is the final result of the elimination, before we started the back substitution.

Given that matrix multiplication is associative, we can represent that as:

\[
(E_{3,2}E_{2,1})A = u
\]

which will give E:

\[
\begin{bmatrix}
1 & 0 & 0 \\
0 & 1 & 0 \\
0 & -2 & 1
\end{bmatrix}
\begin{bmatrix}
1 & 0 & 0 \\
-3 & 1 & 0 \\
0 & 0 & 1
\end{bmatrix}
\quad = \quad
\begin{bmatrix}
1 & 0 & 0 \\
-3 & 1 & 0 \\
6 & -2 & 1
\end{bmatrix}
\]

However, that’s not the optimal way. Lets consider the equation again:

\[
(E_{3,2}E_{2,1})A = u
\]

Lets consider, instead of how we’d get from A to u, but from u to A, using inverses. That’s a topic for next time.

Note: If we needed to exchange rows, we would have done so using a permutation matrix. This is easy to derrive by thinking about it in terms of functions of rows in terms of row exchanges, and columns in terms of column exchanges, as per the above examples.
For example, exchanging rows in a 2×2 matrix:

\[
\begin{bmatrix}
0 & 1 \\
1 & 0 \\
\end{bmatrix}
\begin{bmatrix}
a & b \\
c & d
\end{bmatrix}
\quad = \quad
\begin{bmatrix}
c & d \\
a & b
\end{bmatrix}
\]

and exchanging columns in a 2×2 matrix:

\[
\begin{bmatrix}
a & b \\
c & d
\end{bmatrix}
\begin{bmatrix}
0 & 1 \\
1 & 0
\end{bmatrix}
\quad = \quad
\begin{bmatrix}
b & a \\
d & c
\end{bmatrix}
\]

MIT OCW Linear Algebra

2012-01-06T00:00:00-08:00

THIS IS A PLACEHOLDER SO THAT THE INDEX APPEARS IN THE PROJECT LIST

2011 In Review, And a Look Forward to 2012

2012-01-02T00:00:00-08:00

Although I didn’t start out with a plan for 2011, I’m going to perform a brief review of the last year and outline some of the remarkable things that happened to me. This will hopefully help me feel like I’ve achieved something significant.

Got married (technically that happened at the end of 2010, but I’m going to include it here)
Moved to the US permanently
Started a new job at an Austin based startup
Made some new friends, and started playing D&D again (this is something I didn’t think I could ever do again, so it’s a good thing for me)
Visited India for the first time
Completed 2 courses in Stanfords experimental online university courses – Introduction to Artificial Intelligence, and Machine Learning. This lead me to watch some lectures and learn material I’ve been meaning to get around to for a few years
Realized the importance of working on only one thing at a time.

The last one, I think, is the most important to my future growth.

Up until now, I’ve been trying to do too many things at once, and thus got nothing done. I’ve always been saying I’ll concentrate on one thing at a time, but I never follow through.

The Stanford online classes helped me finally follow through with that goal, and for 2 months, I focused only on completing those classes. It opened my eyes to just how important it is to work on only one thing at a time.

Due to this, I am devoting the entirety of 2012 to my artwork, and I’m setting some specific goals to measure my progress. After all, if you’re not measuring it, you can’t learn from it.

These goals are as follows:

Finish the 4′×5′ airbrush portrait of myself and my wife for my mother-in-law.
Sell a piece of artwork
Earn enough money through art on the side to buy a Cintiq
Submit art for Spectrum 20 (being accepted would be fantastic, but submitting is good enough)

This isn’t the full list of goals, but it is a list of landmarks to mark my progress. I’m hoping that I’ll get the first 2 done before June.
At the end of 2012, I’ll review how I did with these goals.

Things that Suck

2011-12-16T00:00:00-08:00

I do a lot of reading about startups and entrepreneurship. A common theme on the subject of getting started is finding problems to solve. “Easier said than done”, I’ve always thought. People in the tech startup world love to claim that ideas are cheap and easy, execution is where it counts. Ideas weren’t cheap and easy to me, until I learned a trick.

Startup literature writes that you should find meaningful problems to solve in order to start your startup. That seems kind of abstract to me. If I had a meaningful problem, usually I’ve already solved it myself.
Getting more specific, people bring up “pain points.” Find something in your life that is a “pain point” and fix that, and chances are, other people have the same “pain point” and you can help them solve it. Again, that’s a little abstract. What causes me pain in my life? Not much.

While dwelling on the meaning of “pain point”, I came up with a rephrasing and a quick acid test. A pain point is something that makes you say “Man, this sucks.”

“Driving home in traffic sucks”
“Planning a wedding sucks”
“Changing travel plans sucks”
“Lost luggage sucks”

Something is a “pain point” if doing it, or experiencing it, or fixing it makes you think “Man, this sucks.”

If something sucks, and you can fix it, then you start to put the idea through the whole “will people pay for this” wringer.

Stanford Online Artificial Intelligence Class

2011-12-15T00:00:00-08:00

What is this?

Stanford is running an artificial intelligence class during the summer of 2011. I am participating in this class, and this page will track my progress. I will be posting things like lecture notes and additional reading/research.

I hope to both provide help to other people who are running the class, information to people who may have missed the class, and accountability for myself to follow this class through to completion.

Update: December 15, 2011

With the class come to a close, I can publish all of my individual lecture notes in a single document:
Entire course notes
and the solution to the last week’s programming exercise:
aiclassnlp1.rb – solution for the first question
aiclassnlp2.rb – solution for the second question
corpus.txt – training corpus for the second question (Flatland by Edwin A. Abbot, from Project Gutenberg
input.txt – input for the second question (from the notes on the video)

Week 1

Lecture notes for the first week’s lessons.

Week 2

Lecture notes for the second week’s lessons.
Also, the homework for Week 2 really kicked my butt. I had some issues getting my head around some of the probability, so here are some resources I’m finding useful:
- http://www.reddit.com/r/aiclass/comments/lrojo/understanding_bayes_rule_intuitively_without/
- http://www.reddit.com/r/aiclass/comments/lm7g7/dseparation_and_the_bayes_ball_algorithm/
- http://postimage.org/image/udh7nv217/

Week 3

Lecture notes for the third week’s lessons.

Week 4

Lecture notes for the fourth week’s lessons.

Week 5

Lecture notes for the fifth week’s lessons.

Week 6

Lecture notes for the sixth week’s lessons.

Week 7

Lecture notes for the seventh’s week’s lessons.

Week 8

Lecture notes for the eighth week’s lessons.

This week, we also (kind of) learn how the Kinect does it’s thing. Kind of awesome.

Week 9

Lecture notes for the ninth week’s lessons.

These notes are in txt format because I had to take the notes on a laptop and not my iPad as I normally do.

Week 10

Lecture notes for the tenth week’s lessons.

This week we also got to do optional programming assignments. Here are my solutions in Ruby (1.91):
aiclassnlp1.rb – solution for the first question
aiclassnlp2.rb – solution for the second question
corpus.txt – training corpus for the second question (Flatland by Edwin A. Abbot, from Project Gutenberg
input.txt – input for the second question (from the notes on the video)

Stanford Online Machine Learning Class

2011-12-11T00:00:00-08:00

What is this?

Stanford is running a machine learning class during the summer of 2011. I am participating in this class, and this page will track my progress. I will be posting things like lecture notes and additional reading/research.

I hope to both provide help to other people who are running the class, information to people who may have missed the class, and accountability for myself to follow this class through to completion.

Week 1

Lecture notes for the first week’s lessons.

Week 2

Lecture notes for the second week’s lessons.
Programming exercise posted in the second week. (I’ll put these up at the end of the course, considering you can submit late)

Week 3

Lecture notes for the third week’s lessons.
Programming exercise posted in the third week. (I’ll put these up at the end of the course, considering you can submit late)
Practical classifying with logistic regression – an interesting write up about how Khan Academy uses logistic regression to assess student mastery through a course.

Week 4 & 5

Both weeks 4 & 5 are about Neural Networks, and due to a change in the structure of week 4 a little into week 5, I’ve rolled these two weeks into one.

Lecture notes for the fourth and fifth week’s lessons.
Programming exercise posted in the fourth week and the programming exercise for the fifth week. (I’ll put these up at the end of the course, considering you can submit late)

Also, I want to say that these two weeks’ programming exercises were to me the most interesting. Not only did we achieve a real, practical goal (recognizing hand written digits), but we got some insight about how to represent data in machine vision problems.
Week 4 was to implement forward propagation, and week 5 was backward propagation. In the end you end up with a neural network that can classify hand written digets that you built and trained from scratch yourself. Very cool.

Week 6

Lecture notes for the sixth week’s lessons.
Programming exercise posted in the sixth week. (I’ll put these up at the end of the course, considering you can submit late)

Week 7

Lecture notes for the seventh week’s lessons.
Programming exercise posted in the seventh week. (I’ll put these up at the end of the course, considering you can submit late)

Week 8

Lecture notes for the eighth week’s lessons.
Programming exercise posted in the eighth week. (I’ll put these up at the end of the course, considering you can submit late)

Week 9

Lecture notes for the nineth week’s lessons.
Programming exercise posted in the nineth week. (I’ll put these up at the end of the course, considering you can submit late)

Time

2011-11-08T00:00:00-08:00

The Knife Maker

Firstly, this film resonates with me strongly. It’s how I feel about my work, and the hobbies I choose to invest my time in.

One of my favorite insights from the HN thread I found this in: “Time must be used up. You don’t get to just stop using your time until you have something in particular you want to use it for. If making something by hand is rewarding to you, and someone else will pay you for it then it’s probably a good use of your time.”

Rails 3.1 Model Property Name Gotcha

2011-10-28T00:00:00-07:00

If you’re trying to instantiate a Rails model and you keep getting nil back instead of an instance of that model, chances are you have done something dumb like try to create a property on that model called “class”, or something equally ill-advised.

I haven’t tested this for all Ruby keywords, but chances are good that if the build in Object class has an instance property by the same name as one of your model properties, you’re going to run into trouble.

It’s obvious when you’re not dealing with the consequences, but when you’re sitting there in a console and your .new instantiation is returning nil, this get pretty confusing.

Games as Platforms

2011-10-18T00:00:00-07:00

I just read an article about Minecraft’s direction away from a platform towards a computer game , and couldn’t help thinking of one of my recent posts about businesses doing the opposite . If anything, it reinforces the concept of a platform for inverting the responsibility of creating value.

I know I’d much rather sell a game that makes money by itself, and enable others to create value for it by modding it. Hell, you could even let content creators earn money and take a commission.
I guess the only thing you need to be aware of is becoming a lame middleware vendor instead of a consumer product vendor.

Platform as product first, then extendable by the community.

WebGL Book

2011-10-13T00:00:00-07:00

Status

The book is still very much in the planning stages. There are a few fundamental topics that I’m still a little fuzzy about (deferred shading for lighting/bump mapping, etc mostly), and I’m still researching these topics. Once I am a little more well informed, I will begin outlining the book and writing it.

Updates will be pushed through my RSS feed, so if you want to keep track of the progress of this book, simply subscribe to the feed.

Platforms and Sandboxes

2011-10-12T00:00:00-07:00

I’m in the middle of reading this HackerNews posting and I’m starting to see some parallels between building a tech business on a platform, and creating a sandbox RPG game¹.

Both are based on inverting the work load. Provide a means for other people to do your work for you and they will. The tech business (RPG) will move in a direction your customers (players) want, and you can focus on more important parts of your business (game).

Interesting.

_{1. For those unfamiliar with sandbox RPG games, the person running the game basically presents a complete world with an overall plot, and lets the characters choose their own actions and interact how they will with this world, as opposed to creating individual, fixed story adventures for their players to run through.}

A Case for Brevity

2011-10-10T00:00:00-07:00

I have a somewhat keen interest in speed reading, because the world is fascinating and I only have so many years to learn it’s secrets. One speed reading technique involves reading only the first few sentences of a paragraph, because that’s where the author makes their point. The rest is just to convince you of it’s validity. I have seen this true in most places, especially on the internet in blog posts.

Which is why I find myself wondering why I even write posts, every time I write a post. What am I trying to say? If I have something to say, is it worth an entire post? Do I need to convince people it’s true?

I just read a post about perseverence being the key to success, and it was quite long. At least 600 words. I read the first few sentences, then the first sentence of each paragraph and gained no new insights. The author spent their entire post trying to convince me that perseverence is essential for success in anything. Is that something that people need to be convinced of? Are there really people who don’t already know this? If so, would they likely be reading this article? I don’t think so.
The entire article could have been replaced with “Natural talent is mistakenly assumed to be largely responsible for success; perseverence has more effect on success”.

That’s what’s been holding me back from blogging a lot. I find it difficult to justify padding a simple point out to a “respectable” sized post. I think it’s time I embraced brevity.

Literati - Ruby based literate programming library

2011-08-15T00:00:00-07:00

Description

One of the big issues with development is how fast code
documentation becomes out of date, when it is written at all.

While programs are executed by computers, we write them mostly
for the benefit of other programmers, hence the proliferation of
high level languages such as Ruby and Python.

Literate programming is an attempt to solve this problem by
tightly coupling the code to the documentation, and to help
programmers write clearer programs for each other first, and for
computers second.

Literate programming was pioneered by Donald Knuth, and while there
are a lot of software packages to tangle/weave literate programming
files, there’s a wide variety of syntaxes for writing documents.

Literati attempts to keep the literate programming syntax as
close to regular text as possible, so you don’t have to think
about how to format your programs and documentation.

Literati keeps out of your way so you can do what you do best: code.

Status

This is still very much a rough proof of concept.

At the moment, the library is contained in one file: literati.rb.
There are 2 sample programs that you can run through Literati and it
will produce a Ruby and a Python based calculator.

I’m in the middle of a rewrite of Literati, in Literati format.
This can be found in the lib directory, however it’s not functional.

Source

http://github.com/tgittos/literati

Syntax

I’ve tried to keep the literati syntax as simple as possible. You should be able
to pick it up by just reading through the two sample programs, however there
may be some finer points that aren’t obvious.

Each code block must have a header, in the form of
== Identifier ==
The identifier will uniquely identify that code block, and allow you to reference
it later in the document.

You can optionally put more documentation after the header. This documentation
can be any format you wish.
After your documentation, you can close the documentation section with
@api, doc
This tag tells literati which documentation types this documentation block belongs to.

Then your source code comes next. You can terminate your source code by starting a new block.

At the end, you need a program description block, which looks like
== @filename.rb ==
The @ in the header marks the name of the file, and below the header block are code section
references, as defined in the unique identifiers above.

I know none of this documentation makes any sense, but considering this is still in dev,
that’s to be expected.

I’ll rewrite the documentation when I finish refactoring the code.

Modeler - 3D modeling in the browser

2011-06-08T00:00:00-07:00

Goals

3D modeling in the browser. No plugins, no programs to install. Just fire up a browser and off you go.

Status

[07-24-2011] Added in a directional and ambient light to the shader. Next up is point lighting.

[06-15-2011] Optimizations have been finished and merged into the master branch.
Next up is implementing a simple lighting model, and then work on setting up the interface to allow geometry to be
put into a scene.

[06-08-2011] I’m putting this demo page up because somebody on Github has asked to see a demo. This project is far
from nearing a state where you can call it a 3D modeling application. Right now, I have a basic renderer that can take
a scene made of some primitives and render it to screen. I’m working on optimizations because there’s currently a large
amount of overhead associated with getting objects to the GPU right now.
My goal is that once I’ve finished optimizing the core of the renderer, I can start designing a basic interface and
method of inputing geometry. Then I will perhaps release a stable version of the source.

Demo

Code

For the latest, bleeding edge code, you can go to Github and clone it locally.
There’s a terse readme that will describe how to get it up and running on your local computer.

There are no stable releases right now, as the project is very far from where it needs to be to live up to it’s goal.

Now On Jekyll

2011-05-27T00:00:00-07:00

I first started my blog in order to help myself learn. I figured that I would be motivated to write, and in order to fully explain things, I would force myself to do research and increase my knowledge on a given topic. So I set up a Wordpress blog, created a very simple theme and got started with it.

This was all good in theory, but in practice it didn’t quite work. This really bugged me, because I didn’t know why it didn’t work. I thought about it for a long time, and I think I’ve figured it out.

The reason it wasn’t working is because the way I think and do things is very different from the workflow of Wordpress. Wordpress is a very linear blogging system. Posts are displayed in the order they’re created, and if you want to do a series of detailed posts in multiple posts, you either need to do them one after the other, or you need to keep them as drafts until they’re all done.

I tend to work a little different. I like to do a lot of things at once, and nibble away at posts. I’ll write a little here and a little there, and generally take my time to produce content. This wasn’t working for me in Wordpress, and it was actually starting to make me feel bad.

So, I decided to change to a platform that’s a little more freestyle. I chose Jekyll because I decided to integrate my site with Github. Now I don’t have to worry about logging in to a web based control panel, because everything is handled with public/private key encryptions on Github, and it’s very easy to deploy to my sever with a post-commit hook that tells my server to do a git pull.

I’m also writing in textile now, which means that should I produce any content that’s worth publishing, converting it to TeX then to a PDF or something similar should be equally easy.

This is probably time for the obligatory “this is a work in progress” comment, and to say that things will be in flux for a while, at least until I get settled on a format/workflow that really works for me. I’ve been working with this for a while now offline, and it seems to be doing the trick, but there are still a few bugs to iron out.

You can also find all of my old posts still on the website, in my archives , however I couldn’t bring across all the comments, so to those of you who commented on my writing, thank you and I’m sorry.

Pivoting

2011-04-20T00:00:00-07:00

In start-up parlance, to pivot is to change the direction of your attack. Which makes it the perfect description for what I’m doing.

When I started this blog, I just wanted to build a name for myself out on the internet. I was aware that potential employers would probably be searching for my name, and I wanted to control the content that they found. I had no overall goals for the blog itself, other than what it would do for me.

Over time, it evolved into a place for me to share snippets of code and techniques that I’d found along the way. Each time I did something kind of cool, I’d write it up and post it online. While I hope that these snippets have helped people, and I know some of them have, I’ve decided that’s not really what I want to do with my blog anymore.

I’ve long held the belief that information channels in the human mind are serial. You’re either consuming information or you’re producing, but it’s very hard to do both at the same time. I’m a consumer, I always have been and I always will be. I’m fascinated by a whole bunch of different but loosely related things. I don’t have the time to write regularly while I’m trying to do all this cool stuff.

But there’s the rub: I like to write about what I learn. It helps order my thoughts and creates a reference document for me to refresh my memory from. It’s secondary to doing cool stuff, and a blog demands to be first choice when it comes to building audiences and producing quality content regularly.

I’m not well suited to the blog format of writing. I’d prefer to do something awesome, write it up and publish it, and disappear for another 6 months while I do something else awesome. And that’s exactly what I’m going to do.

Lately I’ve been working deep inside computer graphics, reading and learning and tinkering around in WebGL. I will be doing that for a while, and won’t be posting here much. But when I do, I’ll post all about the things I learn.

Between here and there, I will be migrating off Wordpress and onto Jekyll. These days I carry around a computing device of some kind that I can store files on, so web based access to content is no longer important to me. Jekyll will make the site faster, more secure and easier for me to change and deal with. I have no timeframe for this, and will publish none.

The Social Media Revolution

2011-02-15T00:00:00-08:00

I can’t remember exactly when I first started hearing about the so-called “Social Media revolution.” It was right when I was starting to really get into web development. It was easily at least 3 years ago, before Twitter had hit the mainstream like it has now. All I remember is that I didn’t buy into it.

I had been a member of a social network called deviantArt since 2001, and joined Facebook quickly after it opened it’s doors to people who weren’t in school. I had graduated when it reached Australia, and wasn’t able to create an account. I had interacted with people through an online media for a long time before I heard the phrases “social network” and “social media.” To me, all this talk from marketers and web designers about leveraging social media, claiming that social media was the way forward, was just that – talk, nothing but hot air.

I think today I can say that I understand the social media revolution. All it took was to get involved in creating a social network.

I have recently changed jobs to an Austin company that is busy building it’s business social network. Working as close as I am with their social network and learning about how they plan to leverage the community, I can see that I took the wrong end of the stick with regards to social media.

I’ve read a lot of criticism lately about Facebook, with critics saying that Facebook is a hollow way to interact with your friends. They say that it cheapens relationships, and waters down the meaning of ‘friend.’ This is the view I’d had of all social media in the past, and while it may or may not be a valid criticism of Facebook, it’s certainly not a valid criticism of social media.

I had a discussion today with a friend about the future of publishing. She’s in the publishing business, and is violently opposed to e-books and the effects they’re having on brick and mortar book stores. It was while I was explaining to her my vision of the future that I realised the value of social media. We were musing on the role of publishers in a primarily e-book formatted world where there was almost no hand selling done. She was worried about the authors that weren’t super stars, and how they would get eyes on their books in a world where you didn’t have covers to browse, and browsing had a greater friction.

I explained that it would be social media and social networks that would step in. Websites like Goodreads are a good start, where people share their opinions on books and list books they’ve read and want to read. You’d build a network of readers that you know, and know the tastes of, and that would provide the data you’d need to make decisions about new books.

It was then that I realised the true value of social networks. It’s a new way for applications to gather real, meaningful data in the most efficient way. By crowd sourcing data, you build a knowledge base in a relevant domain in a very hands off way. By leveraging this data, you can provide very personalised and useful functionality in applications. Rather than make assumptions about your visitors and users, you can obtain hard data and act upon it.

Safari and Firing Keypress Events with Backspace

2010-10-05T00:00:00-07:00

Whilst working on a project for a client that uses a lot of jQuery and ASP.NET AJAX, I found that the ASP.NET AjaxControlToolkit control MaskedEditExtender doesn’t quite work in Safari. The symptoms of the problem are the backspace and delete keys not working to clear the text in the mask.

A little too late for my project, there was a fix posted on the project’s Codeplex issue tracker (which I haven’t verified personally). It involves patching AjaxControlToolkit javascript in source to sniff specifically for Webkit browsers and compiling a new AjaxControlToolkit.dll.

Instead, I took the extenders out and replaced them with a custom jQuery date mask. I researched a few of the more popular jQuery plugins that do that, but found they didn’t quite work the way we wanted them to, so I had to roll my own.

Stupidly, I tested mostly in Firefox, and after a while got it working pretty well. I went to test it in Safari, and it didn’t work. I feared all the work I had done was for naught. I took a quick look in Safari’s javascript console and found that Safari wasn’t firing keypress events for the backspace key, which is what my plugin was listening for.

As it turns out, this was a known issue with Webkit browsers. I tested around and figured out that it was just the keypress event that wasn’t being fired. The keydown event was being fired fine. So I switched my plugin to listen to the keydown event instead of the keypress event, and that resolved the issue.

So if you want to support Webkit browsers and you’re doing key press detection for ‘special’ keys, use the keydown event instead of the keypress event. It will save you some frustration.

Evaluating JSON in Safari

2010-08-13T00:00:00-07:00

Just a quickie on parsing JSON strings in Safari.

I’m in the middle of a project developing a timesheet/rostering application, which uses a lot of Javascript. I have a series of web services that interact with jQuery. For reasons beyond the scope of this post, the web service returns a JSON string, which needs to be parsed into a JSON object, and then further processing is done on the object.

I had a bug where Safari was choking on the JSON string returned by my webservice, yet Firefox and IE were happy parsing it. As it turns out, my JSON string had a Javascript keyword in it; ‘break’.

Here is my JSON string (well, an example):

{employee: 'Joe', worked: true, periods: [{start: '09:00 AM', finish: '17:00 PM'}], break: {paid: true, hours: '1'}}

Safari was choking on the break in the above JSON string.

When I wrapped the attribute in single quotes, like so:

{employee: 'Joe', worked: true, periods: [{start: '09:00 AM', finish: '17:00 PM'}], 'break': {paid: true, hours: '1'}}

it was happy.

You can replicate this by using the below code snippet to parse out your JSON string quickly in Safari’s debug window:

var json = &quot;{employee: 'Joe', worked: true, periods: [{start: '09:00 AM', finish: '17:00 PM'}], break: {paid: true, hours: '1'}}&quot;;
var obj = (new Function("return " + json))();

which should fail. If you add the single quotes around break though, it should work just fine.

This also fails for other Javascript keywords:

(new Function(&quot;return &quot; + &quot;{if: true}&quot;))();

fails. If you wrap if in single quotes, it works.

I think the take away lesson is to always wrap your object attributes in a JSON string in single quotes, otherwise Safari will choke on it.

Autostart a VPN in Windows XP/Vista

2010-03-10T00:00:00-08:00

Autostart a VPN in Windows XP/Vista

Just a quick one today, and it’s exactly what it says on the box.

I was unreasonably inconvenienced by the need to manually dial into my works VPN everytime I start my computer up. For some reason it really rubbed me the wrong way.

A few days ago, I finally decided to solve this niggling problem once and for all. Here are the steps you’ll need, assuming you already have the Windows VPN client configured for a VPN:

Save your authentication details in the connection
Set the VPN connection to not prompt for user details (Start → Connect To → right click on your VPN → Properties → Options → uncheck “Prompt for name and password”)
Create a .bat file somewhere safe, and put in it the following: rasdial "Name of your VPN Connection"
Create a shortcut to this .bar file
Put this shortcut into your startup folder

Now, the next time reboot, your VPN should start as soon as you get an internet connection.

ProxyBotLib

2010-02-19T00:00:00-08:00

A library for connecting a .NET based bot to the Starcraft Brood War API http://code.google.com/p/bwapi/. Contains source for the BWAPI socket communication dll and the .NET proxy bot.

Available from Github.

Created for use with the EIS Starcraft AI competition http://eis.ucsc.edu/StarCraftAICompetition

Based on http://breakablec.redirectme.net/svn/repos/trunk/StarProxyBot_net/ which is based on http://eis.ucsc.edu/StarCraftRemote

Getting Started

This library is intended to aid an external, .NET based AI bot to connect to the BWAPI socket. To get started, create a new console application which will be the base for your AI agent. Add the ProxyBotLib project to the solution, and add to the console application a reference to ProxyBotLib. There is a sample agent in ProxyBotLib.Tests that you should move to the console application. Create an instance of the agent, and create an instance of the ProxyBot, passing the agent to the constructor of the ProxyBot.

Example


[STAThread]
public static void Main(System.String[] args)
{
    StarCraftAgent agent = new StarCraftAgent();
    ProxyBot proxyBot = new ProxyBot(agent);

    try
    {
        proxyBot.start();
    }
    catch (SocketException e)
    {
        Console.Error.WriteLine(e.StackTrace);
        System.Environment.Exit(0);
    }
    catch (System.Exception e)
    {
        Console.Error.WriteLine(e.StackTrace)
    }
}

OpenGLBase

2010-02-17T00:00:00-08:00

Please see the project on Github for more details:
On Github

ASP.NET GridView with an ObjectDataSource

2009-10-26T00:00:00-07:00

Just a quick tip:

I got the following exception today:

If a data source does not return ICollection and cannot return the total row count, 
it cannot be used by the GridView to implement server-side paging.

If you’re trying to return an IEnumerable from your select method, try changing it to a List or an array. That should clear the problem right up.

Going Dark

2009-10-13T00:00:00-07:00

Nearly 12 months on from the start of my blog (the first post went up at the end of November 2008), I’m finding less and less time to do the quality kind of writing I have set out to do. I feel that, as I change, so too does my website need to change.

As much as this site is currently a programming blog, it’s also my electronic representation. The website shares my name, and is representative of me. There’s more to me than just programming. For a while I’ve tried blogging or writing about non-programming related topics on another domain, however I don’t feel this is fair to myself.

I guess the best way to describe myself is an aspiring polymath. I have an interest in a broad range of topics, notably programming/computer science, mathematics and art. This website doesn’t really represent that. I can’t maintain multiple websites to separate various areas of my life – I’m a single person with multiple facets, and I should have a single website, with multiple facets.

Additionally, I find myself with less and less time to devote to this kind of writing, as it slowly makes it’s way down to the bottom of my priority list. Currently I’m trying to rekindle my math education by going back to basics, learn Punjabi so I can impress the fiance’s parents, do several programming projects at once and get back into doing art regularly, and writing is a very far last.

So, all that lead up to say that I’m going dark, and will be overhauling the site sometime in the future. I’m going to change the format of the site so that it more accurately represents me as a whole, is better organised and has a nice format that allows me to post quick, smaller posts that can be filtered out when subscribing to RSS feed. I may or may not stick with Wordpress – I think I’ll probably start hacking away at Enki and do some funky stuff with wildcard subdomains or something.

All older posts will remain, as a few of my posts are rather popular (by more than 50% compared to the rest of the site) and I suspect they’re really helping people out.

Mario AI Competition 2009 - Getting Started

2009-08-25T00:00:00-07:00

Programming is fun. That’s why I decided it would be a good career for me. I love solving puzzles, and I love the concept of making computers, these mysterious boxes of my childhood, do things. However, when you finally get to working as a professional programmer, the types of projects you get to work on are often not as interesting as you’d like. This is why I am interested in AI. AI presents new and unique challenges, and techniques of approaching these challenges. I said in a previous post that AI was more commonplace than you’d think, but didn’t really follow it up with suggestions of where you can work on AI related programs.

I’ve recently stumbled on an interesting AI competition (before it got big with the video – I’ve been writing this post for about a month), based off of an open source Java project called Inifine Mario, called Mario AI Competition 2009. The competition is to create an AI agent that can navigate randomly generated Mario levels, pitting each agent against each other. The first round of this competition has already closed, however there is another round that closes September 3.

I won’t be attending the conferences nor will I be submitting agents to the competition. I will however work on a few agents, as this is probably the most fun application of AI that I’ve seen in a while, and it’s a small enough environment to enable me to get very familiar with it very quickly, and hence get into making solutions very quickly.

On the surface, this seems a trivial, niche problem. An agent that can navigate a video game level doesn’t really have a lot of diverse applications. However, it’s a great way to begin practicing some fundamental concepts of AI. The parameters of the competition don’t limit the techniques you can use to create the agent. In the video below, Robin Baumgarten is using A* path finding techniques.

However, A* is just the start, and is a fairly basic approach (though it works pretty well by the demos), and there are many more options, including machine learning and neural networks. Unfortunately, I won’t be covering any of that yet. The goal of this post is to get myself, and anyone else interested, up to speed on the environment, and the basics of creating an agent. I will be creating a very simple reactive agent that has no planning, and is quite flawed. However, you should be able to walk away with a firm grasp of how the environment works and what tools you have at your disposal.

Before I go on, a disclaimer is necessary: I haven’t worked with Java since my university days, so about 6 years or so. I will post some code, and that code will probably suck. You have been warned.

The Game Environment

The details on the code available from the competition website are fairly vague, so I’m posting this to perhaps help some people who are interested in getting into it, and also for my reference to help while working on it. Firstly, obviously, you’re going to have to download the code available on the competition website. The code is relatively stable at this point, however it was constantly being updated with fixes for bugs posted on the Google Group for the competition. These changes would be posted on the group, so I’d check it regularly, or grab it from Google Code with svn, and just update your repo regularly. Once you’ve downloaded it, open it in your IDE/editor of choice, and roll up your sleeves.

Firstly, I’m going to reference classes by their fully qualified class names; ie: ch.idsia.scenarios.Play, which references Play.java located in the src/ch/idsia/scenarios folder. This way you can play along at home.

According to the online documentation, you can fire up a run by typing

java ch.idsia.scenarios.Play

from within the “classes” directory. This will start a copy of the Infinite Mario game with a human player. To start with a pre written agent, type

java ch.idsia.scenarios.Play ch.idsia.ai.agents.ai.ForwardJumpingAgent

to start with an agent that will run forward and jump as soon as it can.

The ch.idsia.scenarios.Play class is a simple class that sets up the agent, whether it’s human controlled or automated, and then it sets some play environment variables and runs the application. If an argument is supplied, it will attempt to instantiate an instance of the class and pass it through to the play environment.

I won’t be covering the the details of the simulation code, as it’s all pretty logically separated and you don’t really need to know how to do that. Information is passed through to the agent through an implementation of the interface ch.idsia.mario.environments.Environment. This argument to the getAction function provides the agent with a serialised view of the terrain and the location of enemies, along with information about Mario. Basically, any information you can get by looking at the scene.

This information is provided in the form of a byte matrix, through several functions. observation.getLevelSceneObservation() will return a matrix with details of the terrain. It will also tell you what the type of the terrain is, such as a pipe or a one way platform. This detail is found in ch.idsia.mario.engine.LevelScene. You can see the number codes for the different types of terrain. It’s broken down into gaps, hard borders, half borders and flower pots.

The observation.getEnemiesObservation() is very similar to observation.getLevelSceneObservation(), except it addresses enemy locations. Again, you’ll also get the type of the enemy from the matrix in the form of an integer code, which you can find in ch.idsia.mario.engine.sprites.Sprite.

I’ve written a debugging function that will output both the enemy and terrain matricies to give you a code view of what’s going on. It’s interesting and enlightening to watch the code view of the simulation next to the simulation itself, and it will help us write code to fit our design. The function is as follows:

byte[][] levelState = observation.getLevelSceneObservation();
byte[][] enemyState = observation.getEnemiesObservation();
//Debug information

for (int y = 0; y < 22; y++)

{

    for (int x = 0; x < 22; x++)

    {

        //Check for enemies first

        if (enemyState[y][x] > 0)

        {

            String enemyValue = Integer.toString(enemyState[y][x]);

            int padding = 3 – enemyValue.length();

            if (enemyState[y][x] == 1)

                System.out.print("mmm");

            else if (padding == 2)

                System.out.print(" " + enemyValue + " ");

            else if (padding == 1)

                System.out.print(" " + enemyValue);

            else

                System.out.print(enemyValue);

        }

        else

        {

            //Check for terrain next

            String terrainValue = Integer.toString(levelState[y][x]);

            int padding = 3 – terrainValue.length();

            if (padding == 2)

                System.out.print(" " + terrainValue + " ");

            else if (padding == 1)

                System.out.print(" " + terrainValue);

            else

                System.out.print(terrainValue);

        }

        System.out.print("|");

    }

    System.out.print("\n");

}

System.out.print("———————-\n");

I put the above code into a function in the agent itself, and called the function from the getAction function of the agent, purely because I know this is going to get called once per tick. You can put it wherever you want to put it.

Creating a Dumb Agent

As I mentioned at the start of this post, I’m just going to start with creating a very simple, dumb agent that will attempt to traverse the level, and even then not get it right all the time for reasons I will cover. On the surface, Mario’s tasks for navigating a stage without enemies is pretty straightforward. All he needs to do is to get himself around the level and jump over pits.

However, sometimes Mario can find himself falling into a gap, or over-jumping his mark and jumping into a hole. So basically, we need to check if there’s something in Mario’s path that needs to be jumped over, or if there’s a gap that we need to jump over. But, we can’t just jump – we also need to check if jumping is going to put is into a gap and kill Mario, and if Mario runs off of where ever he is, he might fall into a hole.

This would be fairly straight forward if we had a robust method of predicting where Mario’s jumps will take him, but unfortunately we don’t officially get this information.
There is a movement algorithm in ch.idsia.mario.engine.sprites.Mario, which starts at the move method, however, not only is this not in the spirit of the competition, but more importantly, it’s very hard to figure out, for me at least. I’ve not been able to reliably modify the movement code to act as a predictor of Mario’s path if you plug-in how long to hold each key for.

Hence, at this stage, we’ll just be making Mario go right and try and jump over obstacles. Seems fairly simple.

You’ll see in the provided agents that the function getAction is the heart of the agent, and an instance of the Environment interface discussed before. This function is the work horse of your agent, as it will tell the simulation what the agent is going to do.

Although the observation.getLevelSceneObservation() and observation.getEnemiesObservation() methods return the full view of the level, all the features are jumbled together. Later on, we’ll need to know what’s a cannon and what’s a pipe, and what’s raised ground, and maybe even what’s a block we can punch. We’ll need to know this because cannons shoot Bullet Bills and enemy flowers live in pipes, and these can appear at any time, and if Mario is near by, he’s going to get hurt.

I’ve created the following function to categorize the environment into pipes, cannons and pits:

private Boolean[] pits;
private Vector cannons;
private Vector pipes;
private void processTerrain(byte[][] state)
{
    //Init features
    pits = new boolean[22];
    cannons = new Vector();
    pipes = new Vector();
//Init pits
for (int i = 0; i < 22; i++)
{
pits[i] = true;
}
for (int y = 21; y >= 0; y—)
{
for (int x = 0; x < 22; x++)
{
//Look for gaps
pits[x] = pits[x] && state[y][x] == 0;
//Look for cannons and pipes
if (state[y][x] == 20)
{
boolean pipeFound = false;
//Check if x,y is in pipes
int pipeLength = pipes.size();
for (int pi = 0; pi < pipeLength; pi++)
{
int[] coords = (int[])pipes.elementAt(pi);
if (coords != null && coords⁰ == x && coords¹ == y)
{
//Pipe found, skip it
pipeFound = true;
}
}
if (!pipeFound)
{
//Peek ahead to determine if it’s a pipe or a cannon
if(x < 21 && state[y][x + 1] == 20)
{
//It’s a pipe, add it and the next one to the pipe array
pipes.add(new int[]{x + 1, y});
}
else
{
//It’s a cannon
cannons.add(new int[]{x, y});
}
}
}
}
}
}

The three variables are defined as private members of the agent, and calling this function will populate them with tile locations in the case of pipes and cannons, and tile columns in the case of pits. You don’t have to do this with such a simple agent – you can simply use similar techniques to parse over the whole level state.

I use this information to determine how far away obstacles are. At first I was going to make the distance between obstacles affect the jumping distance and height, however without an accurate way to determine where jumps are going to land, this can actually cause more harm than good. So I ignore it, and just use a fairly blunt approach.

Firstly, another helper function:

private int setDeltaForObstacles(Vector tiles)
{
    int length = tiles.size();
    int buffer = 0;
    int localDeltaX = 0;
    //Loop through every tile
    for (int counter = 0; counter &lt; length; counter++)
    {
        //Get coords of tile
        int[] coords = (int[])tiles.elementAt(counter);
        //Calculate the delta, then check to see if it's good enough
        localDeltaX = coords[0] - 11;
        if (action[Mario.KEY_RIGHT] &amp;&amp; coords[0] &gt; 11)
        {
            //If the tile is after mario, and we're heading right, delta is good
            break;
        }
        else if (action[Mario.KEY_LEFT] &amp;&amp; coords[0] &lt; 11)
        {
            //If the tile is before mario and we're heading left, add to buffer
            if (coords[0] &gt; buffer)
                buffer = coords[0];
        }
        else if (action[Mario.KEY_LEFT])
        {
            //If the tile is after mario, but we're heading left, get the last tile loc from the buffer
            localDeltaX = buffer - 11;
            break;
        }
    }
    return localDeltaX;
}

This function takes a Vector and will return the closes tile to Mario, depending on his direction. It’s a very coarse way to determine the closes obstacle to Mario. Bear in mind that it doesn’t recognise when obstacles are put close together, thus doesn’t optimise Mario’s jumping.

The goal of the agent is to jump over the closest obstacle. That’s it. Considering our obstacles are now in three different datastructures (cannons in one, pipes in another, and pits and raised terrain in the original byte array), we need to store the delta (distance between Mario and the closes obstacle) and update it only when we’ve found a smaller delta. Also, I want to keep the delta of any pits separate, as I’m going to have different rules for pits than other obstacles (because pits are the only thing that can kill Mario at this stage).

To this end, I create two integer variables at the top of the getAction method. They are intitialized to 11 because this is where Mario is:

int deltaX = 11;
int deltaJump = 11;

Here is the code to extract the delta of the nearest obstacle, and the delta of the nearest pit. Note that levelState is the results from a call to observation.getLevelSceneObservation:

//Analyse pipes       
int pipeDelta = setDeltaForObstacles(pipes);  
if (pipeDelta &lt; deltaX &amp;&amp; pipeDelta &gt; 0)
    deltaX = pipeDelta;
//Analyse cannons
int cannonDelta = setDeltaForObstacles(cannons);
if (cannonDelta &lt; deltaX &amp;&amp; cannonDelta &gt; 0)
    deltaX = cannonDelta;
//Analyse the gaps

boolean gap = false;

for(int y = 0; y < 22; y++)

{

    if (pits[y])

    {

        if (action[Mario.KEY_RIGHT] && y > 11)

        {

            int localDelta = y – 11;

            if (localDelta < deltaJump)

                deltaJump = localDelta;

        }

        else if (action[Mario.KEY_LEFT] && y < 11)

        {

            int localDelta = 11 – y;

            if (localDelta < deltaJump)

                deltaJump = localDelta;

        }

        gap = true;

    }

}
//Analyse ground above Mario

boolean raisedGround = false;

if (action[Mario.KEY_RIGHT])

{

    for (int x = 12; x < 22; x++)

    {

        for (int y = 12; y >= 0; y—)

        {

            if (levelState[y][x] == -10)

            {

                int localDelta = x – 11;

                if (localDelta < deltaX)

                    deltaX = localDelta;

                raisedGround = true;

                break;

            }

        }

        if (raisedGround)

            break;

    }

}

Once we have the deltas, we can use this information to decide what to do. I mentioned above treating obstacle deltas different from jump deltas. This is an attempt to make Mario take smaller jumps when he can see a pit is after an obstacle. This is controlled by two instance members that keep track of jump height and jump length, called jumpHeight and jumpLength respectively. This is very hit and miss. Notice that Mario does not respond to anything more than 4 tiles away, and tiles that are behind him. This is to prevent him from going jump crazy:

jumpHeight = 7;
jumpLength = 14;// * (deltaX / 11);

if (((pipes.size() > 0 || cannons.size() > 0 || raisedGround) && deltaX >= 0 && deltaX < 5))

{

    if (observation.mayMarioJump() && !action[Mario.KEY_JUMP])

        action[Mario.KEY_JUMP] = true;

}
//If mario is jumping, there must be an obstacle in the way

if (action[Mario.KEY_JUMP])

{

    if (gap && deltaJump >= 3)

    {

        jumpHeight = 7;

        jumpLength = 1;

    }

}

//No obstacles, but there is a gap

else if (gap && deltaJump >= 0 && deltaJump <= 3)

{

    if (observation.mayMarioJump() && !action[Mario.KEY_JUMP])

        action[Mario.KEY_JUMP] = true;

}

maintainJump();

Finally, the maintainJump method is to ensure we’re holding down the jump key for the optimal amount of time. It is also coded to provide rudimentary control over jump height and length, however this isn’t being taken advantage of.

private void maintainJump()
{

    if(action[Mario.KEY_JUMP] && (jumpCount > jumpHeight))

    {

        action[Mario.KEY_JUMP] = false;

        jumpCount = 0;

    }

    // otherwise you’re in the middle of jump, increment jumpCount

    else if (action[Mario.KEY_JUMP])

    {

        jumpCount++;

    }

    if (jumping)

    {

        if (action[Mario.KEY_RIGHT] && (rightCount > jumpLength))

        {

            action[Mario.KEY_RIGHT] = false;

            rightCount = 0;

        }

        else if (action[Mario.KEY_RIGHT])

        }

    }

}

The complete agent is at the bottom of this post for you to download and include in your project. Simply put it in the path src/timgittos, configure the ch.idsia.scenarios.Play class to have no enemies, with options.setPauseWorld(true), and run any level you want (passable result with difficulty 3). Then build the project and run it with java ch.idsia.scenarios.Play timgittos.BlogAgent.

As you will observe, the agent will make Mario go towards the right as fast as possible, jumping over obstacles as it meets them. For the most part, it works well enough. However, there are quite a few fatal flaws, and I will explain why it fails in this way.

Most obviously, the agent frequently falls down gaps. It doesn’t fall down all gaps – which would indicate that it’s not a failure in the gap detection, but rather something else. In fact, what’s going on is that the gaps that Mario tends to fall down are those that enter his field of consideration (4 tiles ahead of him) while he’s in the air. The agent makes no effort to check whether it’s jump will land Mario in a pit (since it can’t predict where Mario will land), and my previous efforts at trying to make the agent force Mario to the left when it detects he’s going to go down a pit have failed.

Mario will also fall down pits if there are multiple pits in a row. Again, this is because the agent doesn’t exert much control over the size and length of Mario’s jumps ,and so can’t pin point that it wants to land in the middle of the two pits, so it tends to overshoot the jump and fall down the second pit.

Most annoyingly, the agent will sometimes get stuck. If it reaches an obstacle, such as a cannon or a bit of wall or a pipe and jumps at the wrong angle, it will enter a weird wall jump loop, where it jumps, gets stuck on the obstacle, jumps off, jumps again, and gets stuck again. This will loop until there is no time. This is a side effect of the environment allowing wall jumps, which is cool when you can get the agent to make Mario jump out of a pit he’s falling into, but annoying when he gets stuck.

In any case, this agent is far from complete, and just serves to demonstrate how the simulation environment works, and how to interact with it. Once you’re clear how to work with the simulation, you can concentrate on building a useful agent, either by starting from scratch (like you would if you were going to use a machine learning approach), or by building on top of this base (for a determinate approach like A*).

Download: Mario AI Blog agent (.java)

Configuring webHttpBinding When Using WCF with JQuery

2009-06-24T00:00:00-07:00

Firstly, a disclaimer: I don’t have a lot of experience working with AJAX enabled WCF services, and from reading some of Rick Strahl’s posts on using JQuery with ASP.NET, I’m doing things in a really hackish and terrible manner. Hopefully my mistakes won’t impact the usefulness of this short tip.

When working with JQuery and AJAX enabled WCF services, I recently encountered the following error, through Firebug and Fiddler:



The maximum string content length quota (8192) has been exceeded while reading XML data. 

This quota may be increased by changing the 

MaxStringContentLength property on the XmlDictionaryReaderQuotas 

object used when creating the XML reader.

This is caused by WCF webHttpBinding default having fairly draconion limitations on XML message length and depth. This thread on MSDN helped diagnose the problem, however it didn’t do much to help me solve my particular problem. Ultimately, I stumbled upon the solution quite accidentally.

Firstly, as per the MSDN thread, you need to add the custom binding configuration underneath the configuration section:
[xml]

[/xml]

At first I thought this was a custom binding, and tried to replace the service endpoint binding to the custom binding defined above, however this was causing an “System.ServiceModel.ServiceActivationException” exception.

The above binding declaration is actually a configuration for the webHttpBinding. As such, you need to add the BindingConfiguration property onto your service endpoint, along side your binding declaration. This is as follows:
[xml]

[/xml]

This will allow you to send larger messages to your WCF services via AJAX, and get on with creating applications.

Going Camping with CouchDB On OS X Tiger

2009-06-12T00:00:00-07:00

With Snow Leopard being released at the end of September this year, Tiger is definitely starting to show it’s age. I’ve been putting off upgrading to Leopard for quite some time, not being able to justify the down time associated with upgrading an OS. This has had an unpleasant side effect of not being able to do things quite as easily as I want, and finding information is hard.

Due to the recommendations in the comments of my Ruby CMS article, I have decided to start playing with the Camping Ruby microframework backed by CouchDB. Getting Camping up and running was easy enough, and I managed to get some trivial functionality going. Camping uses SQLite for it’s database, which is fine if I wanted to massage my data into a relational form. However, I have models with optional attributes, and rather than making nullable database types, I want to leverage CouchDB’s schema-less, RESTful document storage. Each persisted object in my application will be available on a unique url, so CouchDB is a nice fit for what I want to do.

The Great Installation

I found this great article about getting Camping talking to CouchDB, and it seemed easy enough, so I decided to try it. Since I already had Camping working, all I needed was to get CouchDB installed on my Macbook. As much as I’m not scared of compiling applications from source, I’ve had issues in the past with OS X and compiling, so I decide to find a simple package or disk image to install. I found CouchDBX, however checking the requirements showed that it supports Leopard only.

With a little more research, I found out I had to use Macports to install CouchDB. Macports relies on XCode Tools which come available on the DVDs that came with OS X. If you haven’t previously installed XCode Tools, then you should install them before you install Macports. Something that is buried in a support ticket, however, is that Macports targets XCode tools 2.5. My XCode tools was something like 2.4.8. While I did manage to get Macports installed, the installation for CouchDB fails when it tries to build tk. If you, like me, have an older XCode Tools than 2.5, you’ll need to download 2.5, which you can find deep inside the Apple website. This does require an Apple Developer Connection account to get, so if you don’t have one, you’ll need to create one.

Installing CouchDB is as simple as issuing
[bash]
sudo port install couchdb
[/bash]
and Macports will take care of the rest. CouchDB lists it’s dependencies as Spidermonkey, Erlang and a few others, however each of those has dependencies, and each of those dependencies has dependencies, and it ends up taking quite a long time and downloading a fair few packages. The upside is that if you didn’t have Erlang previously installed, you do now. Erlang is something I’ve been wanting to take a look at.

Start The Server Before You Leap

While the article I linked to above about getting Camping and CouchDB talking is good, it’s not exactly the most verbose explanation especially if you haven’t read the documentation for CouchDB and prefer to just dive in. For those who are still a little unsure like I was about how exactly to do this, I’ll outline my approach below.

Before you get started writing any code at all, you need to start your CouchDB server. This is something that escaped me for the longest time, and when I finally twigged, it gave me a bit of grief. To start CouchDB, issue the following command:
[bash]
sudo couchdb
[/bash]
If you try this without escalating privilege, it will return an error "{"init terminating in do_boot",,[{couch_server_sup,start_server,1},{erl_eval,do_apply,5},{erl_eval,exprs,5}, {init,start_it,1},{init,start_em,1}]}}", which searching for will bring you to the CouchDB wiki page for error messages. This page will tell you the problem is an unavailable port. For me, at least, this is not true. CouchDB runs on port 5984 which as far as I know is not used for any system services or other software. The reason I was getting this error was that I wasn’t running it with enough privilege.

Once you get CouchDB started, you’ll notice it blocks the terminal. I don’t like to have too many windows open at once, so I’d prefer to have it run as a background process. Fortunately CouchDB will let you do that with a command line switch option, along with changing the location of the pid file. I’ll leave figuring out how you want to start it up to you.

Once you have CouchDB started, you’ll have access to a web-based administration panel called Futon. Futon is available to you at http://localhost:5984/_utils/, assuming you’re running CouchDB on your local host. The Futon utility will allow you to create databases and insert documents in preparation to test your connectivity with Camping.

Camping Time

There were two Ruby gems I had considered when deciding how to connect Ruby to CouchDB. I should point out that due to the RESTful nature of CouchDB, you strictly don’t need any Ruby gems and could roll your own fairly easily, if you wanted to. I didn’t want that level of control, personally. The two gems I considered were CouchRest and RelaxDB. CouchRest is a simple Ruby wrapper around the CouchDB REST API, whereas RelaxDB is more abstracted, bringing ActiveRecord-like functionality to CouchDB. While before I would have chosen RelaxDB, I’m intentionally trying to work a little closer to the raw APIs here, so I chose CouchRest.

You can install CouchRest through RubyGems:
[bash]
sudo gem install couchrest
[/bash]
however I found that the gem version wouldn’t work and kept throwing errors when I started my Camping application. To overcome this, I just cloned it from the GitHun repository and placed it in my Camping app directory and referenced it that way.

The first thing to do would be to create a normal Camping sample application, if you haven’t done already. For the purposes of this post, I’m going to use the “”http://camping.rubyforge.org/files/README.html">skeletal Camping blog" application used in the documentation, and edit that to work with CouchDB. I figure this will give everyone a relatively common base from which to start. Import CouchRest just below where the application imports Camping:
[ruby]
require ‘camping’
require ‘couchrest’ # or ‘couchrest/couchrest’ or similar if you’ve cloned from Github
[/ruby]

Given that the purpose of this article is to demonstrate connecting to CouchDB, and not the design of a framework around it, we can safely ignore defining a model for now, and rely on CouchDB as our model. So if you’re following along from the example application, you can safely delete the classes from Blog::Models module. We’re not going to delete the whole module, because of the create method.

The Camping create method that is run when the server starts up your Camping app. From the documentation, “This is a good place to check for database tables and create those tables to save users of your application from needing to manually set them up.” Instead, we’re going to use this method to set up CouchRest in our application:
[ruby]
module Blog::Models
def Blog.create
db_url = ‘http://localhost:5984/’
storage = CouchRest.database(“#{db_url}blog”)
Blog::Controllers::Index.set_storage(storage)
end
end
[/ruby]

Next, because we’re now calling a new method on the controller class, we need to modify that. Change the Blog::Controllers::Index class to the following:
[ruby]
module Blog::Controllers
class Index < R ‘/(\w+)’
def Index.set_storage(storage)
@storage = storage end def get(id) @post = @storage.get(‘posts/’ + id)
render :index
end
end
end
[/ruby]
What I’ve done here is given the controller class a static reference to my CouchDB database (from the create method). Then the controller was changed to respond to a regex, which is passed into the get method. We use the id passed in to retrieve a document from CouchDB, which we assign as an instance variable.

Lastly, we need to modify the Index function view to pull data from the CouchDB database:
[ruby]
def index
h1 @post[:title]
end
[/ruby]
This will output the title of the test document created at the start of the article.

If you haven’t already, start up Futon and create a database for the purposes of this demo. Call the database “blog”, and fill it with a test document with the id of “posts/test”, and at least a “title” property with the value “Test Post”, and any other properties you wish.

Start the Camping application with camping blog.rb, and point a browser to http://localhost:3301/test, and you should see “Test Post” in a header tag, rendered out to the browser, which means that Camping has successfully communicated with CouchDB.

I didn’t cover putting data into CouchDB, however you would do it in a similar vein, writing a post method inside your routes to create data and save it to CouchDB. From here, personally, I’m going to create some wrapper classes for requests and models and build up something which is a little DRYer, however as far as a demo, this is good enough.

My Best Guesses About Wolfram Alpha

2009-05-19T00:00:00-07:00

Following the “soft launch” of Wolfram Alpha a few days ago, I’ve been wondering exactly how Wolfram Alpha works. From the description of all the hype leading up to it, it seemed like a question answering application that is specialized in answering questions, as opposed to say Google, which is more into finding and presenting data, leaving the interpretation to humans. Now that it’s here, we can have a play around and see what it can do.

Firstly, I tried to ask Wolfram Alpha how it worked, and it just gave me a puzzled look. So I can only guess, and guess I shall.

The Wolfram Alpha website lays it all out on the table, with regards to what it’s all about. Wolfram Alpha’s goal “is to accept completely free-form input, and to serve as a knowledge engine” to answer search queries entered. From the very limited playing I’ve done, it seems pretty cool.

Following my move from Perth to Texas, I typed in “”http://www.wolframalpha.com/input/?i=austin+to+perth">Austin to Perth" into Wolfram Alpha just to see what it told me. I got back map, with a line drawn between the cities. I got the distance in miles between the two and the flight time. I got a side by side comparison of the local times, populations and approximate elevations.

Google however, delivered me what I was actually looking for, sort of. Maybe that’s just a difference in expectations and thinking.¹

It’s clear that Wolfram Alpha is pretty cool, and will do some awesome things with the data before it returns it back to you, but it doesn’t really help me figure out how it works. My thoughts are that Wolfram Alpha combines a large database of indexed knowledge in the form of a knowledge base and a powerful logical language, the language being Stephen Wolfram’s own Mathematica, naturally. As he describes it, Wolfram Alpha is the “killer-app” for Mathematica.

First, lets discuss the vast store of data available to Wolfram Alpha, most likely in the form of a knowledge base. A knowledge base is an information representation scheme designed to allow information to be operated on using logic. Information is stored in “sentences” in a knowledge representation language. The knowledge base allows logical agents, in this case the Wolfram Alpha application, to receive queries, and then search and analyse the knowledge base, performing logical inferences and information compounding, then collating it and returning the results. The logical agent does all the heavy lifting, but the very structure of the knowledge base helps the agent.

The heart of the logical agent is it’s knowledge representation language. It allows the agent to “reason” through sentences and make inferences in order to derive new facts. Proposition logic and First-order logic are both knowledge representation languages. Knowledge representation languages use syntax and semantics to define knowledge, where syntax outlines the structure of a language, and the semantics are the meaning.

Logic is a very mathematical concept, and mathematics can be seen as a kind of logic that operates on numbers. Mathematics has a syntax, that defines the infix notation of logical operators on operands, and the semantics are what those operators do. 1 + 1 = 2 is an example of a complex logical language, where the syntax defines that the + operator acts on each 1, and the = operator defines the result of the previous operation. The semantics define that the value of 1 + 1 is 2.

Mathematica, the heart of Wolfram Alpha, is a computational language, according to it’s Wikipedia page. It’s built on top of the Symbolic Manipulation Program, designed by both Chris A. Cole and Stephen Wolfram. The Symbolic Manipulation Program is a “computer algebra system”, which facilitates symbolic computation, allowing computers to operate on symbols and manipulate expressions instead of operating on their values, which sounds a heck of a lot like something that could be used to create a logic that can operate on a knowledge base.

Going back to logical agents and knowledge bases, there are a number of ways logical agents can extract and infer information from knowledge bases. The topic of knowledge based logical agents is a pretty vast one, and I couldn’t possibly explain it completely in a single post, even if I knew more than just a fraction of it, which I don’t. But I can summarize what I do understand, and how it applies to Wolfram Alpha.

As referred to often in the post so far, inference or entailment plays a large part of how a logical agent can extract “new” information from a knowledge base. Inference is deeply rooted in formal logic, and techniques have been developed to allow automated software processes to apply it to a knowledge base.

Logical entailment is the idea that a sentence “logically follows” another sentence. If a given sentence a is true, and the truth of sentence b depends on the truth of a, then b must be true, and also that if b is true, then logically a must also be true. This is useful if the knowledge base doesn’t contain the fact that one of a or b is true, but it does contain definition that the truth of b depends on the truth of a.

The knowledge base is designed to be a model of reality, and holds sentences about the world intended to describe it to a level of detail. Sentences can be as atomic as facts, such as “the sky is blue”, or can be logical sentences such as “if the sun is in the sky, it is not night”, which of course has exceptions (such as in certain seasons in the polar regions), which also need to be modeled in the knowledge base.

This is relevant because, with a finite model represented by the knowledge base, there are a finite number of entailment and inferences that can be made on the model, which means that a logical agent such as Wolfram Alpha can reduce the world to a finite search space, and can run a search over this space looking for information that follows on from the set of facts in the knowledge base.

Search techniques for searching the knowledge base are similar as search techniques in other AI based problems, such as constraint satisfaction problems and touring problems. These searching algorithms include backtracking and local search methods, such as hill climbing.

When searching this knowledge base, the logical agent needs to abide by a series of rules, known as inference rules, in order to guarantee the the knowledge it entails and infers is valid.

Firstly, the logical agent needs to be aware of the equivalence of two logical sentences. If two sentences are both true under the same model, such as the Wolfram Alpha knowledge base, then they are logically equivalent, and can be used to format facts in a different way to aid inference.

Secondly, the logical agent needs to be aware of the concept of validity. A sentence is valid if it is true in every model, which essentially means it’s a tautology and is always true. These tautologies are useful because they aid in validating inferences, where a infers b if a implies b is a tautology, where implication is such that b is true if and only if a is true. This can get a little confusing, and if you can, I’d suggest you read Chapter 7 in AI:AMA
.

Thirdly, it needs to ensure the sentence is satisfiable, which means it’s true in at least one model, which is basically just searching for inferences and determining that, for a given search space state, the inference being suggested is true.

The logical agent and inference/entailment from a knowledge base can be observed in Wolfram Alpha with a fairly simple query: “”http://www.wolframalpha.com/input/?i=red+%2B+yellow">red + yellow". In this query, the logical agent would probably hit the knowledge base and tell it that both red and yellow are true, meaning that we want to find information about those. It would then search for all sentences that are entailed or logical structures that mention the fact red or yellow, such as a sentence “orange is a mix of red and yellow”. It would then tell the knowledge base that this fact had been entailed, and tell it that orange is true. Then, when it finds the sentence “blue is complementary to orange”, it can add that to the record set. Obviously there will be other sentences in the knowledge base, and the agent will have to use a filtering algorithm to determine which sentences and facts are relevant to the query, otherwise it might return results that the user is not interested in.

To successfully parse the search query, it would also need a measure of natural language processing, so that it can convert a human-readable query into a machine readable query. This, combined with a large, extensive knowledge base, a fast logical language such as Mathematica and AI techniques for inference and entailment of knowledge mean that Wolfram Alpha can answer a wide range of questions that a simple indexed and categorised search engine couldn’t.

However that’s just my best guess. And keep in mind who I am: I’m just a young web developer with an interest in AI. I don’t actively participate in AI research, and I have no formal training. My guesses and musings are probably widely off the mark, and if anyone has any other ideas or corrections, clarifications or discussions, I’d love to hear them in the comments.

If you mistype a search query into Wolfram Alpha, curiously it won’t suggest a correction. Considering how relatively easy this task is (just use it’s Levenshtein Distance), I find it odd that it doesn’t support this.

Now Appearing In Austin, Texas

2009-04-26T00:00:00-07:00

This blog has gone dark over the last two or so weeks, but for a very good reason. On the 21^st of April, I flew from Perth, Western Australia to Austin, Texas, where I will be staying for the next 6 months. For those not acquainted with the trip, it’s half a world away at over 13,000km, for a total flying time of 29 hours not including layovers (for my trip, anyway).

I’ve been spending the last week settling down here in Austin, and the week before that preparing for the trip. I’m still suffering from some jet lag, and catching up on the last 8 months with my girlfriend, who I hadn’t seen since August 2008. I haven’t had the time nor the concentration required to write a blog post, and I haven’t touched my Google Reader account, which I can see has over 1,600 unread articles.

However, I do have some interesting posts coming up. I’m planning a response to an AI article that was linked to me in a comment, a more in depth look at the importance of searching by solving the 8-Queens problems in Python using some different search techniques, and a look at creating some graceful degradation control adapters for ASP.NET server controls that don’t work without Javascript.

So, don’t unsubscribe from my feed in your readers, the blog isn’t abandoned, life has just taken priority over it. When things settle down, I’ll start writing articles again.

Where Do We Draw The Line On Complexity?

2009-04-15T00:00:00-07:00

I’ve got a few posts planned about AI and HCI, however these require a lot of time to research and write code for, time that I don’t have given I’m flying to the US on the 21^st of April. So I apologise to people who saw my last AI article on Hacker News and Reddit and subscribed to my feed expecting more of the same. There will be more, I promise. I just need more time. For now, just some musings.

I’ve been thinking a lot lately about solving problems. I read a number of programming aggregators and startup blogs, and the term “solve a problem” keeps nagging at me. To be successful, at programming and business, you need to solve a problem. I’d love to eventually run a development company, but I don’t know what problem to solve. So, I consider the problem my current employer is solving.

I currently work for a web development company that is doing fairly well. We have a suite of products that we sell to our clients, and sell services related to those products. They’re doing well, so obviously, they must be solving a problem. So, from “client” point of view, the problem we solve is that of letting people who don’t know anything about websites build and run complex websites.

We have a CMS product that allows non web geeks manage websites. This includes writing new pages and editing content on existing pages, publishing news items, distributing these to an email list, managing uploaded files and inserting modules of dynamic content to their hearts desire. The modules we offer include contact forms, random image rotators, resume upload controls, auto-generating bread crumbs and menus, and so on. These modules aim to reduce the complexity of managing dynamic sites, so that ordinary people can do it.

On top of that, we do custom applications built on top of that CMS site. Clients have a site they run themselves with the CMS, but some also have dedicated applications with a separate interface. These are separate because they’re custom jobs. They provide functionality that is not available in the CMS (yet), or functionality that would be too complex to maintain using the CMS. So again, we’re reducing the complexity of running a custom application.

So, on first looks, we build websites and applications that non-geeks can use. But looking further, we manage complexity. We reduce the complexity for the client, so that the client can do difficult things with ease, and don’t have to know how it’s done, only that they can do it.

On the other hand, good programming is about managing complexity. That is, trying to keep your code as free from complexity as you can get away with while still doing the job. Complex code is buggy and unmaintainable.

Complexity exists, and it is immutable. Once you’ve simplified a task to the point that the only steps involved are those that accomplish that task, and that task cannot be accomplished without those steps, you have your complexity, and it isn’t going anywhere. The task cannot be made more simple, because to do that we’d have to leave out steps, which would not accomplish the task.

What we do from then on is delegate complexity, shift it around and make it other people’s problem. Take for example, the task of creating and publishing a blog post. Our software has tagging, categories, post titles and post content. Creating a post involves writing a post, giving it a title, adding tags, selecting categories and publishing it.

Breaking it down, we need to create a body, create a title, create all categories, create all tags, link the tags to the post, link the post to the categories and publish it. Each link has a display name and a URL slug, and each category has a display name and a URL slug. URL slugs cannot contain certain characters, however their display names can.

Now, we can delegate this complexity to the user, which will result in simple code. The user will be responsible for creating tags and categories, and giving them URL slugs that fit our rules, while the application will just take the data and store it, and retrieve it when asked. The user will be responsible for assigning tags to the post and the categories, and the application just stores it. This is obviously an unacceptable level of complexity for the user.

We could also delegate all the complexity to the application, making the user experience simple. The user just types in the tags they want, picks the categories, and the application will link the tags if they exist, create them if they don’t, automatically strip unwanted characters from URL slugs and hide a lot of the complexity. In more complicated cases, this will result in very complex code that will be bug prone.

So the question is, where do we draw the line? What balance of code complexity and user interface complexity is successful? I guess you could say “make it 50/50” and think that’s good enough, but I don’t agree. I think the balance is very contextual, and often leans in favour of code complexity. In simple examples like above, taking on all the complexity of the task in the code is a trivial thing and easily done. For nuclear reactor control software, things might not be straight forward.

This is the key to a successful software product/business, I think. Finding the right balance between complexity. If you put too much complexity on the user, user’s won’t be able to use it, and your software won’t sell, and the whole thing will bomb. If you put too much complexity on the software, it will be buggy and difficult to maintain, and will eat up development time and money, and possibly even end up costing more than it earns, and then it will bomb. That’s not even including the more personal aspects of programmer job satisfaction and sanity.

Success is finding the sweet spot when balancing complexity.

jQuery Context Menu Unbind Click Fix

2009-04-09T00:00:00-07:00

Updated on May 28, 2009, due to partial fix. The fix should now be a complete fix.
—
Apologies for the somewhat confusing title.
I’ve been doing a lot of jQuery work, replacing all the AJAX.NET crap in my project at work to speed the interface up. I’ve started using the awesome jQuery Context Menu plugin. However, I found it was also messing around with jsTree which I am using on the same page.

The problem is that the jQuery Context menu overzealously unbinds all click events from the document. This means that it’s going to break all your other Javascript that relies on clicking. Checking the documentation for jQuery’s unbind function, you can see that it accepts a function as a second parameter. This is the function that will be unbound if you pass that in as a parameter.
So, I fixed the jQuery Context Menu by changing the following:

In the main contextMenu function, near the end, just above the return statement, I added the following:
[javascript]
// External click event for document
function onDocumentClick(e) {
var menu = $(‘#’ + o.menu);
$(document).unbind(‘click’, onDocumentClick).unbind(‘keypress’);
$(menu).fadeOut(o.outSpeed);
return false;
}
[/javascript]
Next, I change the function that assigns the click listener to the document to assign the defined function, instead of an anonymous function. So, change:
[javascript]
// Hide bindings
setTimeout( function() { // Delay for Mozilla
$(document).click( function() {
$(document).unbind(‘click’).unbind(‘keypress’);
$(menu).fadeOut(o.outSpeed);
return false;
});
}, 0);
[/javascript]
to
[javascript]
// Hide bindings
setTimeout(function() { // Delay for Mozilla
$(document).click(onDocumentClick);
}, 0);
[/javascript]
Lastly, I remove all instances of:
[javascript]
$(document).unbind(‘click’);
[/javascript]

That’s it. That fixed the problem for me. You’ll notice I don’t bother with the keypress bindings. That’s because I’m not using them, so I’m not bothered. I’m assuming the fix for that will be a similar strategy, for those who are bothered.

I’ve uploaded my version for those who can’t get it working using the tips in this post, but remember the copyright remains with Cory S.N. LaViska over at A Beautiful Site.
jquery.contextMenu.js

Correcting 5 Misconceptions About AI

2009-04-07T00:00:00-07:00

For the longest time, I have been interested in artificial intelligence. The idea of computers that could make decisions and think independently fascinated me. Of course, I knew that AI wasn’t quite that advanced, but the idea still captivated me. However, the barrier to learning AI was too high. It was too complex and too academic. Just looking at the mathematical notation involved made my head spin.

I eventually took the plunge and convinced my family to buy me Artificial Intelligence: A Modern Approach for Christmas, and 12 months later, I actually started reading it. I’m still reading it, and will be the first to admit I don’t understand everything that is discussed, however there are a few things that I’m realising about AI.

It’s All About Searching
Searching is a constant in AI techniques. Problems are defined and solution spaces are created. Problems can be represented in a number of ways (graphs, trees, logic knowledge bases), however in the end, it always comes back to searching.

This is important because searching is easy. Searching is something that a lot of programmers already know, even if it’s only at a most basic, brute force level. We do searching all the time. We loop through arrays searching for values, we use regular expressions to match string patterns, we retrieve records from databases. We search.

At it’s simplest, you can search using brute force searching, iterating over all combinations and permutations of solutions looking for one that satisfies the problem, but beyond that, you can involve tricky heuristics to make optimal decisions about how to search. You can have local searches, which will pick a solution space and search it for local maximums, such as hill climbing, and searches that will find global minimums, such as simulated annealing. But it’s still search. It’s still something you can do.
It’s More Common Than You Think
AI is a hell of a lot more prevalent than most people realise. I didn’t think AI had much commercial application before I started learning about it, but luckily that didn’t dim my interest. For those who are interested, but holding back because they don’t see how it would benefit them, here’s some good news. AI techniques are used everywhere.

That international 4 city flight you booked used AI techniques. A constraint satisfaction problem solver took all the constraints about needing to be at this city by that time, flying on this airline for that much, and creating a plan for you. When Amazon recommends products you might be interested in, it calculates this with Bayesian networks and classifiers, a method of probabilistically linking a set of variables. Circuit design, product manufacturing, supply chain optimisation, all of these things use techniques that AI use. They’re not the sole domain of AI, but learning AI will cause you to learn these too.
You Can Use It Today
Whatever you’re working on, you can probably use AI techniques in it. Even some of the more exotic sounding techniques like neural networks can be of use to you. Self healing databases? Hell yeah. Even Bayesian classifiers to catalog and categorise products, heuristic searches to mine data in databases, hierarchical task planners to plan that holiday or manage that Gantt chart.

Can you use it anywhere? No. Your simple CRUD app probably won’t benefit from a wizz-bang heuristic search. But if you’re doing anything that involves large amounts of data, interacting with people, predicting trends and recognising patterns, you can use it.
It’s In Demand
You may have heard of the Netflix Prize. Guess what? That’s AI. Google is the biggest search engine around, and index billions upon billions of pages on the internet, and can get you relevant results to a question in a matter of seconds. That’s one hell of a big knowledge base, and one smart search algorithm. Amazon sell products all over the world, and aggressively upsell and cross sell. I get emails about related products I might like based on my wishlist and purchase history, and they’re actually pretty accurate.

Also, computer games. Enough said.

AI skills are in demand. Not huge demand, but probably more that you would have guessed. These skills are hugely profitable in the right hands, and big companies want to extract every single little morsel of useful information about your browsing, shopping, eating, travelling, viewing and reading habits in order to market to you more effectively. Now that sounds a little creepy to me, but if that doesn’t bug you, more power.
There’s a Lot of Information Out There
AI isn’t some weirdo niche science topic. There’s actually quite a lot of information out there, once you start going down the rabbit hole. “AI: A Modern Approach” cites hundreds of papers and books. There’s thousands of websites out there on the subject. There are many academic papers that are made available for free online. There are communities, like AIGameDev, dedicated to spreading that delicious knowledge.

I think the hardest part about finding information is getting the terminology. It’s pretty dense when you first get into it, especially when talking about acyclic directed graphs, and your idea of a graph was like mine was about a year ago, namely a few bars on a 2D Cartesian axis. But once you’ve got a foot in the door of the lingo, it can become pretty accessible, and information starts becoming more bountiful. That foot in the door can be either a good, basic website, or in my case, a university level textbook designed to introduce people to AI.

AI is a big field, full of fascinating and interesting concepts and techniques, and it’s a young field that’s still full of potential. It’s not as complex or confusing as film and television would have you think. That’s not to say it’s a walk in the park, as I stated earlier, I’m probably running a 70% rate of understanding what I’m reading, but I’m managing. And if I can manage, so can you. So if you’re interested, there’s no better time to start than now.

A CMS in Ruby on Rails, and Why I Stopped

2009-04-06T00:00:00-07:00

For the last couple of months I have been writing a series of posts describing a personal Ruby on Rails based CMS. I have been writing tutorial style posts outlining what I was doing, and why I was doing it. Don’t bother trying to look for those posts, because I’ve archived them.

This decision was pretty easy after I slowed down and reviewed the situation objectively for a few minutes. That clarity, coupled with a few truths that hit home while reading the source of a few other Rails applications pretty much sealed the deal.

The next post in the series was going to be an overview of another Ruby CMS project, BrowserCMS. I saw a video from RailsConf about this project, and recognised a lot of their goals as coinciding with mine. So I was going to poke through their code, see how they’ve done things, compare it to how I did the same things, or was planning to do them.

While I was reading through their source, it occurred to me just how much I don’t get Rails. I have a bit of a poke around with scaffolding and get familiar enough with the generators and I decide that I’m ready to tell people how do create a CMS in Ruby, because I’ve nearly finished creating a CMS in ASP.NET.

It’s clear to me, now, how silly that was.

Rails isn’t just a framework for Ruby, it’s a whole change in paradigm. Intellectually, I know Rails is “opinionated software”, however I don’t think I understood what that really means. I tried to make my code as flexible and configurable as possible, and I was struggling with all but the most basic CRUD tasks.

So, first I realised I wasn’t ready to tell anyone how to do anything in Rails.

But I have a series of about 5 posts doing just that. What do I do with those? Do I continue on, plugging away at learning while trying to instruct? I decided the answer to that depended on what I had achieved with my posts. What was the value of them?

Then I realised that I had covered very little other than simple CRUD. Sure, I had some unusual object relations, like trees and self referential comments, and I implemented some business logic like compiling pages into templates, however the total sum of the non-CRUD related content could have fit into 1 post. So I spent 4 posts blathering away at how to achieve the same task as running script/scaffold.

What’s worse is that I realised I was back in the rut of creating a CMS. How on earth did I do that again, after stating loudly and proudly that I don’t even like programming websites.? Well, I told myself, you are planning on using this on your own sites, your many, many online business ideas. Sometimes you have to do the same old thing to earn money. Which is true, if I had done anything of worth, but I hadn’t, I was a long winded scaffold script. I got carried away in the joy of learning a new language/framework (which isn’t a bad thing), and fell into the familiar territory of doing what I always do (which is a bad thing).

So, now I realise that I’m not only trying to teach people something I don’t understand, I’m trying to teach them how to do something I don’t even like doing.

That’s stupid.

I’ve archived the posts. I may revisit, as I still have grand ideas of what a CMS should be, but for now I’m shelving the whole thing. Luckily I have such a small readership (read: none) at this stage, nobody will be affected. For that I’m grateful.

And yet, through all this stupidity, I have learnt a few things of value.

I’ve learnt that Ruby on Rails is more complex than I thought, and will hit the books again to pick up some more advanced techniques.

I’ve learnt that I’m scared to push myself in programming. This revelation is a pointy one. I love programming, it’s my job, it’s my hobby and it’s my passion. The fact that I’m scared to push myself to innovate, hiding behind the excuse that I’m not smart enough or I’m not creative enough is double edged. On one hand, it’s sad that I’m not as ambitious as I thought. On the other, it’s great that I know now, so that I can get stuck into remedying that.

I’ve also learnt a lot about blogging. Those posts of mine weren’t really providing much value. I don’t have many readers because I’m not saying anything new, and I’m not showing anybody cool things they can’t find in a hundred other blogs. I also didn’t have much of a personal voice, and was writing them like I would a textbook, which is missing the point of a blog. It’s supposed to be informal, and I’m supposed to show my personality. So, I’ve found my voice, while realising that I’m not providing value. Another pointy idea, but ultimately good, because that too is something I can work on.

So, inevitably, I have to question the value of this post. What could this post provide to someone who stumbles across this website? Well, a few things.

Firstly, self awareness is a great thing to possess as a programmer. If you know where your weaknesses are, you can train in them, and get better. But that’s not enough, you also need to know what you’re afraid of, so you can recognise that you don’t even know that it’s a weakness. That fear hides the existence of the things you’re not good of, and that’s a major roadblock to improving.

And lastly, if you don’t have anything to add to the conversation, don’t say anything. The whole internet is a conversation, and I’ve just been babbling to myself in a corner the whole time. If you want to contribute, add your own thoughts, your own interpretation on subjects that are well known. Go into new levels of detail on old ideas and technologies. Introduce new ideas or technology, or modify existing ones. Contribute, don’t just talk for the sake of talking.

Visual Studio 2008 Deletes LINQ DBML Designer File

2009-04-01T00:00:00-07:00

Do you spend hours wrangling with your “Interactive” Development Environment, trying to prevent it from deleting your boiler plate .designer file for your LINQ DBML? Do you get baffled watching Visual Studio delete said file from your software version control system, and then struggle to get the two back into sync?

Then you might be encountering this bug right here. The reason for this post is because this bug seems very hard to find information on in Google. So hopefully I can raise some awareness, and at least have it pop up on a Google search.

Typical symptoms are the designer file for the DBML being deleted when the DBML is updated. If you’re also using source control, Visual Studio will attempt to delete it from your source control at the same time. Renaming the DBML file will cause it to regenerate, however it can cause major headaches when combined with source control, removing it, adding it, pending delete actions hidden in a project you thought you had unchecked out.

The solution is the suitably moronic action of moving all your using statements inside your namespace declaration. A co-worker found this solution in this MSDN forum thread. Once you move your usings, Visual Studio happily stops deleting your designer file, and you can finally get back to work programming instead of fighting your tools.

In Defense of the Jerk

2009-03-29T00:00:00-07:00

Not just any jerk, but the jerk who knows his stuff. The technically strong, socially weak programmer who does not play well with others.

During a recent employee review, I was told that I need to work on a few areas, namely the way I communicate with my co-workers, and my tendencies to shoot down ideas without supplying alternatives. I was also told that my technical proficiency was more than sufficient.

The technically capable but socially awkward programmer is a cliche in every day society, and the rogue programmer is a cliche in development teams. Seth Godin’s post, and another one that cropped up in Hacker News reminded me of this, and made me think. I realised that, to some extend, I am that jerk.

The point of this isn’t to toot my own horn, rather to try to offer an explanation on behalf of all well-meaning jerks in software development teams. Now, I’m not talking about those jerk jerks, who code maze-like code to secure themselves a job, those who are needlessly rude and abrasive, and those that ignore the other members of the team to the detriment of the whole project.

I’m talking about the jerks who are otherwise good employees. We’re not trying to sabotage the project, to exert our will over the project, to make anyone feel bad or insufficient.

We just care about the code.

At least, I do. I care about what I’m doing. I care about the project. I want my code to be the best code I’ve ever written, I want this project to be the most successful project the company has. I own my code, and take pride in it.

When somebody suggests something that I know won’t work, that will negatively affect the quality of the code, I freak out, somewhat. The first thing on my mind is to shoot down this idea, because I don’t want it reducing the quality of the project.

If my manager is planning on putting another programmer on the project with me, and I don’t believe they’re quite up to standard, I’m going to try to limit the damage they do to the application. I’m going to try and push them to work in a relatively isolated corner of the application. Unfortunately, I know this is a bad team work based attitude, however I can’t help it.

Now, I’m not defending jerks. Being a jerk is a bad thing, I know this. I agree with what my boss said, and agree that I need to work on these areas. However, instantly firing the jerk, if they’re a well meaning jerk, is not a good idea, because you’ll lose someone who is passionate about what they do, and constantly striving to improve them.

Instead, just try to work things through, and soften the edges of the jerk.

Parsing ASP.NET pages with SGMLParser

2009-03-09T00:00:00-07:00

I’m going to take a short break in my Ruby CMS series to post something I encountered at work.

During my development of the CMS at work, I’ve had to deal with parsing HTML content in order to compile page content into tags. This involves being able to replace certain elements of a page with other elements. At first I tried to do this with regular expressions, however this didn’t turn out too well when it came to dealing with the inconsistencies in legacy sites running on the current, older CMS. Next, I tried parsing the pages as XDocuments, which worked ok as long as I was parsing well formed pages. As soon as it hit a malformed page, however, it died in a fiery burst of exceptions.

Then I looked at SGMLParser, and it seemed like it was my savior. It parsed SGML, of which HTML is a subset, and it auto corrected malformed content, and would handle all the inconsistencies of life, in use HTML, and it would allow me to parse it into an XDocument so I can manipulate nodes. However, even SGMLParser had it’s own problems: it wouldn’t handle ASP.NET server tags very well, which was a problem.

We provide “plug in” functionality to CMS sites by the way of ASP.NET server controls. You build your functionality, stick the server control files into your site, drop the binary into the bin folder and insert the server control tags into your page. Fire it up, and it will appear and function. However, when running pages with server control tags through the SGMLParser, it wouldn’t recognise the namespaces, and would strip them, completely breaking them.

This was unsatisfactory, as we were planning on using a similar sort of set up to perform the same thing in the new CMS. So, I hit Google, looking for ways to get around this. There’s not a lot of information out there about it, other than “SGML parses SGML, not ASP.NET, which isn’t valid SGML” which was exceedingly unhelpful, because I already know.

I peeked into the source of SGMLReader to see if maybe I could remove the functionality that stripped out unknown namespaces. Sure, this would probably defeat the purpose of using SGMLParser, but it would help me out in this specific project. Unfortunately, the codebase is rather complicated and I really didn’t want to dig in and spend hours on something I wasn’t sure was even going to work.

I saw that the project has a custom HTML dtd, so I considered briefly writing a dtd for ASP.NET. However, while I was thinking of all the different permutations of server tags, especially when you can specify your own namespaces and tag names and attributes, I quickly decided that wasn’t suitable either.

In the end, and I believe this is honestly the only way to do it, I ended up pre-processing my content, wrapping ASP.NET server tags within <![CDATA[ ]]> tags. I noticed that SGMLParser was wrapping my older <% %> ASP style tags in this fashion, and leaving them unmolested. After I parse it into the XDocument, do my manipulations and pull it back out as the desired content, I run a post-process through it, removing the <![CDATA[ ]]> tags that were added in the pre-process. After, I’m left with my content as it went in, with the modifications done through the compiling process.

Of course, I do the pre-processing and post-processing with a regular expression:
[code language=“html”]
</?[\w]+:[^>]*/?>
[/code]
which I think is kind of funny, in a way. I can never seem to escape regular expressions.

ASP.NET Ajax, FCKEditor and Firebug

2009-02-12T00:00:00-08:00

Today at work I had the pleasure of dealing with a functionality bug in our new CMS that is written in ASP.NET. Our CMS uses FCKEditor for the rich text editor, and we’re using a whole gang of ASP.NET Ajax stuff for the interface. Specifically, we’re using a TabContainer with user controls inside each tab.

The CMS is designed in the way that most CMS’s are – we have 2 major components. Page objects contain the content data of a page, so text, HTML meta data, etc. Template objects contain templates – that is, HTML files with areas where you insert the page content, similar to how Dreamweaver does it’s templates for you.

When you publish a page/site, the CMS compiles the content into the template, then saves that as a file. When you’re editing or creating a page in the admin interface, the application reads the content areas available in the current template you have selected for that page, and creates an FCKEditor dialog for it.

Because you might not know how the content you’re writing fits into the template you’re going to use, we provide a preview function. This preview function will read in the content of the FCKEditor dialogs, create a temporary page object, set the src of an iframe to our preview page, which takes the page objects and feed it to a preview function which will compile the page and return the string, which we dump to the output stream of the preview page. We then fire up the iframe inside an ASP.NET popup extender, for a nice modal preview.

This all worked fine, until we got the tab container involved.

In order to streamline the work flow and reduce the page refreshes on postbacks, we decided to put the content in a tab container, with different parts of the page in different tabs. When we did this, we noticed that when the preview button did a partial postback, it reloaded the FCKEditor dialogs from the database, obliterating the changes we made to the page (if any), or just displaying nothing for the content if it was a new page.

This was just on my computer.

On a co-workers computer, it worked perfectly as intended, despite the fact we were both working from the same source, in the same IDE, running it in the same browser. What gives?

After poking around on both computers and trying the application in different browsers, we concluded that it doesn’t work only in my Firefox – IE was fine on my computer, and all browsers were fine on his. Acting on a hunch, I disabled Firebug for my Firefox, because I noticed my co-worker didn’t have Firebug installed.

The application started working again.

I don’t know why this happens, and why Firebug should be interfering with the Javascript in an obtrusive way, or even which Javascript it’s interfering with. I don’t know if Firebug was screwing with FCKEditor or ASP.NET Ajax. So, this post is partially to put this info out there, in the hopes that someone else can answer for me.

It’s also partially to say, if you’re seeing odd behavior that seems to be cropping up inconsistently across machines running Firefox, try disabling Firebug. Since being done by Firebug, I’ve decided to leave it off during all development, and thinking back, I wonder how many other Javascript bugs I’ve fought with that were caused by Firebug, and not the code.

Consistent Designs Across Browsers

2009-02-08T00:00:00-08:00

Firstly, a caveat: I am not a designer. I am a server side programmer who dabbles in web design and development. I do not know CSS as well as someone who designs websites for a living would. So take this advice with a grain of salt.

Today @RealtorPaul asked why his site wasn’t displaying right in Firefox, when it was displaying fine in IE. This is a common enough occurrence in web design, and it’s due to the fact that different browser vendors use different render engines, and each render engine has it’s own share of bugs.

Bugs are qualified by measuring against a set of standards laid down by the W3C as a target for render engines. Among the various browsers, IE typically has the longest history of the worst standards support around, with IE 6 being the #1 criminal, and admittedly IE 7 is much better, but still flawed.

When users develop a HTML interface that they check primarily against IE, testing how it looks in IE, etc, they’ll often get a nasty shock when they check in a Gecko (Firefox, etc) browser, as their designs fall apart and generally look horrible.

While the ideal way to trouble shoot these problems is to go over your CSS code with a fine tooth comb and understand how all the various rules are working, and understand how all the browsers render those rules, that’s not very practical. Luckily, the 80/20 rule tends to come into play here, meaning the largest problems between browsers are caused by a small selection of things. Here is a list of things you can check against:

Do you have a valid doctype in your HTML code? A valid doctype will look like this:
```
<!DOCTYPE html PUBLIC “-//W3C//DTD HTML 4.01//EN” 
```
```
“http://www.w3.org/TR/html4/strict.dtd”>
```
You can find a list of available doctypes at the W3C website. I personally recommend the HTML 4.01 doctype over the XHTML doctypes, for reasons that deserve a topic of their own. To use a doctype, put it as the first thing in every HTML page in your website.

Are you using a lot of padding and margins? One of the differences between Firefox and IE (just to name two) that most people hit is the difference in the box model. IE renders it’s box model incorrectly, and as a result, can cause a lot of grief when using a lot of padding and margins.

Are you using properties that do not have 100% support in all browsers? Some CSS features are not implemented completely in all browsers, and their support is patchy at best. You can find a list of these CSS features at quirksmode.org.

Are you floating a lot of elements? Floating elements removes them from the regular flow of the page, and as a result, can have some funky results on your design. Floating elements can be awesome when you know how to use it, but can be confusing as hell when you don’t. Make sure you know what you’re doing when you float stuff.

Are you changing the flow of elements with the “display” property? Certain elements are block level elements (they sit on their own line, and push elements beside them down) and some are inline elements (they happily co-exist in the middle of other elements). If you’re changing how some elements display, be aware that other properties (I’m looking at you, “width” and “height”) don’t work on inline elements. Know what happens when you change how things are displayed.

Those are the CSS concepts that get me the most. Many problems are solved just by adding a valid doctype to your page, however if you’re still finding you’re having problems, give the other items on the list a look.

This list is far from exhaustive, nor is it even very in depth. However, it’s a good place to start looking, and it might point you in the right direction. With the state of the browser market as it is, debugging CSS is as much an art as creating the design for a site is.

When debugging CSS, you might want to start with just bare bones HTML, and slowly add rules as you go, and isolate that way which rules are the ones that are messing everything up. Also, keep in mind that CSS is cascading – which means you need to put your more specific rules at the bottom of your stylesheet, general rules at the top, otherwise your general rules will override your specific rules.

The best way I’ve found to debug CSS is to incrementally apply styles, and use a tool like Firebug to turn rules on and off dynamically, so you can see what happens. The most important thing to remember is that even experienced developers have to deal with browser incompatibility. If you find you’re struggling, keep at it, and don’t lose heart. It’s all part of designing a website.

Apps Hungarian is Just Good Naming

2009-02-06T00:00:00-08:00

Application Hungarian is just good variable naming convention, but worse, because it’s needlessly concise, and hard to read at a glance. It also comes from an environment that is growing more and more obsolete. More on that in a bit.

Firstly, clarification is needed as to the definition of Application Hungarian notation, as opposed to Systems Hungarian. All of this is relatively useless, however, if you are in the same situation I was in a few years back and don’t know what Hungarian notation even is.

Hungarian notation is a variable and function naming convention where you add either prefixes or suffixes to names. What these prefixes/suffixes mean depends on what type of Hungarian notation you’re using. Systems Hungarian is the method of adding information about the type of the value of a variable, or a return from a function. For example, a string that represents a name would be called “strName” using Systems Hungarian. Applications Hungarian is where you add information about the semantic meaning of the value of a variable.If this is a little foreign, that’s because most people think Systems Hungarian when they think about Hungarian notation.

In Making Wrong Code Look Wrong, which is what inspired this post, Joel Spoelsky outlined exactly what Applications Hungarian notation is, and why it’s so useful. He outlines that in Applications Hungarian, you add information about the semantic value of a variable/method return. To illustrate, he demonstrates that you can use Applications Hungarian to differentiate safe and unsafe strings in a web application environment. He uses the prefix ‘s’ to define a “safe” string, that is safe against XSS attacks (ie: A html encoded string). He uses the ‘us’ prefix to define an “unsafe” string, that is a string read directly from user input that may contain potentially dangerous code.

Spolsky also demonstrates using the notation to make wrong code long wrong. That is, by combining prefixes on methods, based on their return types, you can use the names as a sort of litmus test. So if the return prefix of a function specifies it returns a ‘us’ or unsafe string, and you assign it to a ‘s’ or supposedly safe function, this is clearly a logic bug, and looks wrong.

I think this is great, and it’s something I’ve never thought of, but my question is, how is it different from properly descriptive variable naming, aside from it’s somewhat obscure brevity?

The virtues of clear, descriptive names in programming have been sung again and again. If you make your variables and method names descriptive enough, your code becomes somewhat self documenting. You capture the intention and meaning in your code when you use variables like “customerFirstName” and “formatAddress”. You look at those names and you can immediately see the intention behind the code.

I see a huge overlap between the best practice of descriptive variable naming and Spolsky’s intention behind his use of Apps Hungarian, so much so that I don’t see a difference between the two, with relation to their purported goals. Apps Hungarian is more terse, and more obscure, and has somewhat of a learning curve if you’re not familiar with the nomenclature of the system. This, to me, seems like a needless barrier to understanding.

Am I missing something? Why is Apps Hungarian better than regular, descriptive and intelligent variable naming? I can’t see any benefit to using Apps Hungarian explicitly, and to my understanding, Apps Hungarian is just a more terse method of good variable naming.

Pixel Perfection is Impossible

2009-02-03T00:00:00-08:00

This post is an elaboration of a comment that I left on a post about CSS frameworks. I agree wholeheartedly with Joren’s position on CSS frameworks after having used 960gs for the layout of this site (at the time of posting this).. As I mentioned in my reply to Joren’s post, I used the framework to try to cut down the stress I feel when I try to make things line up properly and that most certainly didn’t work as intended.

Specifically, I have a little widget that shows my most recent tweet on the front page, and I’m using a cute Twitter bird vector from DarkMotion.com as the background for this element. If you are viewing this post in IE, Chrome on a small resolution, and probably Opera and Safari, the bird is probably sitting obnoxiously below the recent post bar, getting in the way of content.

This little bird has given me so much grief in the design of this site, that I’ve thrown my hands up in despair and given up. The reason it’s plagued me so is the way 960gs automatically floats everything for it’s grid system. If you want to work outside the grid system, even a little bit, the framework makes you pay with blood.

I love the bird, I love showing my most recent tweet. I will not remove it. However, I can’t claim to be a serious developer, whilst still allowing obvious display bugs on the site, and I can’t bring myself to use a third-party theme to represent my personal blog. It just seems wrong.

In Joren’s post, there is an excellent comment defending frameworks, which makes mention that Joren’s blog has a 3 pixel difference in margins of the content, between the header and the footer. While I acknowledge that this wasn’t the author’s main point, it brings up something that I’ve never understood in web design, and the mentality of many web designers, both new to design and coming from print design.

A web page is a fluid medium. Web designers advocate abandoning the pixel for the em, citing greater flexibility and fluidity. So, at some level, they recognise this. However, people still insist on making designs that rely on pixel perfect precision positioning across all major browser vendors, which is where the need for CSS frameworks and reset stylesheets come from.

Web browsers are products competing in a market, and they have different ways of rendering CSS, and they have their own unique quirks. We know this, as well. Beyond glaring CSS interpretation bugs and discrepancies in box models, they have unique margins and padding on elements, and sometimes these only differ by a few pixels.

Why is this a problem?

Why do these people feel the need to contort CSS, torture it into burying these differences, into displaying the same across each browser. There are far too many permutations of resolution and browser, not to mention whether or not someone has their browser full screened (I don’t) to try to enforce this pixel perfect design.

Embrace the differences. Embrace the fact that your margins may not look identical. Recognise that at the design stage, and design your sites to compensate for this. Don’t split images into different elements if they’re intended to fit together seamlessly. Find another way. Don’t stress if the margin between your content and your header is 3 pixels bigger on IE than it is in Firefox. As long as the general communication goals of your design work across all browsers, and it doesn’t look like a train wreck, be happy.

I know all of this is easier said than done. I’m a server side programmer, not a CSS guru or a designer. However, I think it’s a worthy to keep in mind while designing. Don’t fight the diversity, embrace it.

C# Interface Method Gotcha When Inheriting

2009-02-02T00:00:00-08:00

I noticed some interesting behavior from C# yesterday at work. Truth be told, I’ve noticed this in the past, but I’ve only figured out what’s causing it today.

The behavior is this: when you inherit from a class that implements an interface, calling a hidden/overriden method from an object of the inheriting class that has been upcast to the interface will result in a call the the inherited function.

Ok, so that probably made no sense at all, which is why I will explain it, but not before a warning. I’m not sure if this is intended behavior, and I’m not sure if this is something I’m doing wrong. If someone can point me in the direction of an explanation or a correction, I would be very grateful.

On to the explanation.

Take a sample, trivial application that has widgets. These widgets just print messages out to the console. We have an interface, IWidget, which forms the contract for all widgets:

interface IWidget
{
    void DoSomething();
    void DoSomethingElse();
}

Next, we have a generic widget class, which we use for most widgets. This is called GenericWidget:

class GenericWidget : IWidget
{

    public void DoSomething()

    {

        Console.WriteLine("Doing something in GenericWidget");

    }
    public void DoSomethingElse()

    {

        Console.WriteLine("Doing something else in GenericWidget");

    }

}

And finally, we have a specific type of widget that does something different, called SpecificWidget:

class SpecificWidget : GenericWidget
{

    public new void  DoSomething()

    {

        Console.WriteLine("Doing something in SpecificWidget");

    }

    public new void DoSomethingElse()

    {

    Console.WriteLine("Doing something else in SpecificWidget");

    }

}

Now, we want to run some tests on our widgets, to see what’s going on. First, we’ll test the widgets themselves. Running:

GenericWidget genericWidget = new GenericWidget();
genericWidget.DoSomething();

genericWidget.DoSomethingElse();
SpecificWidget specificWidget = new SpecificWidget();

specificWidget.DoSomething();

specificWidget.DoSomethingElse();

will output:

Doing something in GenericWidget
Doing something else in GenericWidget

Doing something in SpecificWidget
Doing something else in SpecificWidget

Which is exactly what we expected to see. Test has passed.

Next, lets say we have a function that we want to pass widgets to, but we don’t know what kind of widgets we will be passing, only that they are widgets. So we pass them as widgets, but upcast them to IWidget, and call the methods on them from the interface. For example:

IWidget genericUpcast = new GenericWidget();
genericUpcast.DoSomething();
genericUpcast.DoSomethingElse();
IWidget specificUpcast = new SpecificWidget();

specificUpcast.DoSomething();

specificUpcast.DoSomethingElse();

which will give the following output:

Doing something in GenericWidget
Doing something else in GenericWidget

Doing something in GenericWidget
Doing something else in GenericWidget

Wait, that’s not right. The specificUpcast should call the method on SpecificWidget, shouldn’t it? At least, that’s my expected behavior. Just to prove there’s not something wrong with our specificUpcast:

(specificUpcast as SpecificWidget).DoSomething();
(specificUpcast as SpecificWidget).DoSomethingElse();

will yield:

Doing something in SpecificWidget
Doing something else in SpecificWidget

which is our expected output all along.

If someone can explain this to me, or point me to a better method to upcast, then please do so. I’m finding a lot of my code needs to check for specific types in order to call the proper method, and it seems clunky and awkward. I’m sure I’m doing something wrong, I’m just not sure what.

As usual, you can find the source code here: Interface Upcasting Inherited Classes Gotcha

Using Twitter to Gather Information

2009-01-30T00:00:00-08:00

I’ve mentioned briefly the virtues I see in Twitter when I discussed how to implement a project in 3 hours. Now I want to expand on how to use Twitter to gather information. I mentioned in that the key to utilising Twitter to gather information was to follow the right kinds of people. However, you’ll find that when you start following more than 100 people, things get a little hectic.

Every programmer is more effective with a solid selection of tools, and Twitter is no different. TweetDeck is the best Twitter desktop client I’ve used, and it’s an AIR app, so it runs on all major operating systems. Most Twitter clients have a single stream that Tweets come in through, and everything gets lumped together. TweetDeck’s strongest feature, for the purposes of leveraging Twitter to gather information, is the option of splitting your stream into groups.

TweetDeck allows you to put people you follow into groups, and have their tweets appear in a different window. The notification alert will also show this segregation. In addition to splitting your followers, you can specify windows that will feed you the results of searches, using the Twitter search operators outlined here.

The search operators and TweetDeck are the engine of our information gathering machine. Using the search operators you can follow the public stream and cherry pick Tweets that contain information you want. You can look for specific phrases, specific hashtags, tweets after a certain date and tweets with links.

For example, I use use a search for “artificial intelligence” with links to find people who tweet interesting links concerning artificial intelligence, which is a subject I’m highly interested in. I can run several of these filters in different windows at the same time, and grok the whole public stream. No longer am I restricted to listening only to those that I follow. Of course, that’s not to say I no longer follow people. If someone pops up multiple times in my filters, I’ll follow them to be sure not to miss a single tweet from them.

This method, however, isn’t flawless. If you look too narrowly, you won’t find any tweets. Also, when you finally do find tweets, even by trying to pick out key phrases/keywords and links, there will be a significant signal to noise ratio. My search for AI topics often lead to students linking university class websites and time tables, which is something I’m obviously not interested in. Often people who pop up in searches all the time turn out to have a high noise to signal ratio.

Having said that, while there are drawbacks, the ability to monitor the public stream as well as the people you follow, and sift out tweets that have a higher likelihood of being relevant is a powerful thing, and will often connect you to bits of information you would have never found otherwise. That’s the power of Twitter, and that’s why I love Twitter.

3 Hours from Concept To Implementation With Ruby

2009-01-27T00:00:00-08:00

Although I’m still relatively new to it, I’m a big fan of Twitter. However, not for the social media aspects of it. I view Twitter as an excellent method of gathering information, as long as you follow the right kinds of people. By the right kinds of people, I mean people who tweet interesting links they find online. Simply follow people who are interested or involved in the same subjects that you’re interested in, and they will feed you information over time, in a nice steady flow.

I’m very much interested in AI, and AI as it applies to games. I am a fee paying member of Alex J. Champandard’s AIGameDev and I follow him on Twitter. While this is great, his flow of information isn’t fast enough for me. So I set to Twitter to search for people who are interested in AI and game development, when I found the @playwitter account, and the associated website. As you can see from the @playwitter tweet, they’re not yet fully functional. I didn’t really want to wait, and I didn’t really want to follow all those people on my own. So I decided to come up with an intermediary solution.

The concept is a Twitter bot that will read in an RSS feed and tweet the entries. I decided to use Ruby, as I haven’t done any significant programming in Ruby before. All I knew was the basic syntax that I read from the Pickaxe book.

First, I hit Google looking for a Ruby RSS parser library. Oh! It turns out Ruby has one built into the language. So I read a few websites on how to do that, and it seems pretty straight forward. So I grab the feed over at Playwitter and start pulling it down with my Ruby script. The quick script I wrote in 10 minutes worked like a charm.

The next part was to tweet the entries. I looked for a Twitter library and settled on the Twitter gem. I installed the gem, and this took me a while to get it working as a library, but get it working I did. I was able to send tweets from an automated script.

It took me about half an hour to wire these together and before I knew it, I was done. However, it was fairly crude, and I found out when I tried to deploy it to my server that it wouldn’t make the grade. I set about refactoring and upgrading the code and trying to get it run. 3 hours later, it was complete, and it was working.

In the end, I implemented the idea in 3 hours, with the following features:

YAML config file for defining Twitter account details and RSS feeds to tweet from.

Timestamp based polling of the RSS feeds, storing only updates new since the last poll.

Tweeting with added hashtag, defined in the YAML with the feed

Deployment as a Ruby Gem

Details of the project are the subject of another post, and when that project is a little mature, I will create that post.

From concept to deployment on a live site, with a basic knowledge of Ruby only, in 3 hours. That’s a fantastic result that I’ve never experienced in another language, compiled or not. It was fun, it was fast, and I felt the freedom of PHP with the benefit of a well implemented OO system.

Man, I love Ruby.

Frameworks Make Reading New Code Hard

2009-01-24T00:00:00-08:00

In an effort to increase my knowledge of both Ruby and Rails, I have been keeping my eyes open for interesting Ruby/Rails apps to peek at the source code of. Whilst looking at @j_stirk’s website, I noticed it was running on Radiant CMS, a Ruby CMS. So, I grabbed a copy of the source code.

Whilst I was having a peek at the source, I began to think that I was in a little over my head. I managed to muddle my way through the config/routes.rb file and figure out (with judicious use of Google) what requests were being directed to what controllers. The problems started happening when I hit the controller, and started seeing filters and symbols and statements that I didn’t know where to go next to find out about.

At first I thought it was to do with the fact that I don’t really know Ruby. I know enough to recognise it, and muddle through basic syntax, but I don’t know any tricks, any coding conventions and the nuances of Ruby programming. However, upon reflecting on my dilemma, I realised this isn’t something that was restricted to Ruby, Rails and Radiant CMS.

I remember last year looking into Symfony, before deciding that PHP was too god-awful ugly to waste my time on. I recall that too had a fairly high learning curve, and looking through an existing Symfony app, it was very hard to figure out what exactly was going on. There was much API browsing.

A few years ago when I first learnt ASP.NET, I was in a similar situation. Server tags and user controls and page events and assemblies were all very new and very confusing to this ex-PHP & ASP 3.0 developer. I stormed through a book on ASP.NET and was thrown head first into a contracting gig before catching my stride again.

I love frameworks. Ever since I heard about this thing called “Rails” a few years back, and realised there were a few similar frameworks written in PHP, I’ve been in love with frameworks. And I’ve always learnt them from the ground up, creating my own application. Last night was the first time I’ve ever tried to learn a framework from an application.

And it’s hard.

While frameworks increase programmer productivity and make maintenance easier for programmers who already are in the know, they make things really hard for a new maintainer who is not familiar with the code to dive in and figure out what’s going on. Not only do they have to learn the individual nuances of the programmer who wrote the application, they also have to figure out how the framework works.

Reconciling YAGNI With Reversibility

2009-01-21T00:00:00-08:00

Yesterday I started reading Andrew Hunt’s The Pragmatic Programmer, which has been sitting around waiting to be read for at least several months. Every programmer I talk to says that it’s a book that will really stimulate growth as a programmer, and I certainly need to grow.

Already several of the things the author has said have rung true, and I can see many of the scenarios he’s described as having happened, or nearly happened, in a few of the projects I myself has been involved with. Which is exactly what has piqued this post.

Hunt outlines the need for what he calls reversibility. He states that there are critical decisions to be made in each project, and each of these involves a commitment that often isn’t easily changed, such as committing to a type of database, or committing to a design pattern.

He goes on to describe the need for a Schrödinger’s Cat approach to software design. The basis is the Schrödinger’s Cat theory which illustrates quantum superposition – where at the point of a decision, multiple dimensions spawn off and you don’t know which dimension you’re in until the decision is observed.

Hunt argues that we should design code to be flexible enough to allow as many of the “possible dimensions” as possible. What he means by this is to allow for these critical decisions to be changed, with minimal impact on the code, hence the term reversibility.

To do this, we should code for possibilities. We should always keep in mind what might happen, and design our software so that we can accommodate the change. This flies directly into the face of another principle of programming, You Ain’t Gonna Need It (YAGNI), and the general belief that the more code you write, the more opportunity there is to introduce bugs in code (something Hunt himself mentions).

YAGNI outlines that you shouldn’t write code you don’t need, and cites several reasons why, and coupled along with the idea of writing less code is better, this is something I believe. However Hunt’s Reversibility idea is in almost direct opposition – he advocates writing code “just in case.”

Can these two ideas be reconciled?

I believe they can, and it’s an approach I’ve used in the past. I’ve found that it’s possible to quickly and safely recognise that there is the possibility of a change by writing interfaces and method stubs for new functionality, and have them throw not-implemented exceptions.

This way, you acknowledge that a decision might be changed and have already got the application hooks to hook into it, but you haven’t wasted time implementing something that may not come to pass. Your software will build, and at the existing functionality won’t be broken while you wait to implement these new changes.

This seems like a trivial thing to do, however it can really streamline the process in my experience.

AI Application Programming - Book Review

2009-01-18T00:00:00-08:00

I’m spending a lot of my time lately reading about AI, both game related, business application related and academia related. Having recently completed “AI Application Programming” by M. Tim Jones (Charles River Media, 2003, First Edition), I have decided to review it in the context of my current situation.

I am new to the field of AI, and my previous reading has been limited to sporadic Wikipedia research on concepts as I come across them. This book covers a lot of material that I haven’t read about before, but it also covers some material I have. This allows me to evaluate the book as both someone with prior knowledge, and someone with no knowledge.

Firstly, the book was a fairly broad overview on integrating aspects of artificial intelligence into application development. Therefore it was no surprise that a majority of the AI concepts discussed were optimization oriented. The book provided an overview of the history of AI, then it covered simulated annealing, adaptive resonance theory, ant algorithms/ant colony optimization, neural networks, genetic algorithms, a chapter on using some of the techniques described to run artificial life simulations, rules based systems, fuzzy logic, bigrams/Markov chains, agent based software and a wrap up the future of AI.

Each chapter is fairly formulaic. It starts with an overview of the theory of the subject for that chapter, and the presentation of any relevant formula. Then a basic example is covered, without code, in order to show how the theory being covered can be applied. Then a specific example with C source is covered, with a detailed explanation of what’s going on. I largely glossed over the source, as I don’t really have much familiarity with C, so cannot comment on the quality of the source.

Overall I have mixed feelings about the book. A lot of the topics have domain specific language, such as referring to parts of algorithms as genes and candidate algorithms as chromosomes in genetic algorithms, and I feel the author sticks to these to the detriment of comprehension of the topic. For example, the author was discussing “energy” and “temperature” with relation to simulated annealing, without outlining that it’s goal was to narrow the search space for a solution to a given problem.

Another detracting factor was the numerous errors in diagrams, equations and the text. These errors can be attributed to a variety of sources, from poor copying jobs on some of the diagrams to bad editing. One particularly annoying error was the diagram showing how a NOT logic gate could be constructed using a single neuron neural network – the diagram showed an input bias of 1, with a weight of 1 on the input, claiming that the output will always be the inverse of the input, by the equation output = input * input weight + bias.
An input of 0 into this equation will indeed output a 1, however an input 1 will yield 2, which is incorrect. The correct diagram should have a input weight of -1, which will yield a correct answer with both the 0 and 1 inputs. These errors hindered the understanding of the topic.

However, it’s not all bad news. From my personal reading, I had always struggled with how the back propagation algorithm in neural networks worked, and this book managed to successfully explain it. A lot of the other topics, such as fuzzy logic and the rules based system were also well presented and relatively straight forward.

From reading this book, I feel I have a solid enough understanding of the basics of the topics covered that I could successfully go out and perform my own, more in depth research and succeed fairly well, as well as integrate a few of the more rudimentary features of these approaches into an application.

How To Run Your Own Webserver

2009-01-14T00:00:00-08:00

For some time, I’ve run my own web server from the comfort of my own home. I made this decision due to the fact that I wanted more control over my server, and I wanted to run as many web development languages side by side as I could, and mainly because I wanted to run SVN. Before I started hosting my own websites, I had a sketchy knowledge of how the DNS works, and how to go about hosting your own websites on a Linux operating system. I also had minimal Linux administration skills.

There are a few good reasons for hosting your own website:

No artificial caps on bandwidth and hard disk usage. You pay for what you use, and if you need more, you buy more.

Total control over your hosting environment. If you want a language installed, install it. If you want to run a certain version of Rails, run it. It’s your server.

Experience. Hosting your own websites on your own server can give you a lot of experience with systems administration on a server operating system.

Cost. You have complete control over how much everything costs, and you know exactly where your money is going. You can spend as much or as little as you wanted.

How Do You Do It?

First, it helps to get an idea of what you really want. Do you want to run a huge corporate intranet with load balancing and proxy servers, or do you just want a small server to host personal sites and tinker? Is your budget limitless, or are you already living on ramen for dinner 5 nights a week? Do you want to host proprietary languages like Cold Fusion or ASP.NET, or will a Linux/Apache install satisfy you?

Personally, I was only interested in hosting personal sites, and doing it cheaply was my utmost concern. This is what I will focus on, as this is what I have experience in. So, what I was looking for was a relatively easy solution that was above all cheap.

1. Get Your Hardware

Before you can start, you need to have hardware to put your server on, and if you already don’t have it, appropriate networking gear to support multiple computers accessing the same internet connection (if you want to use a personal computer as well).

As your end goal is a web server, this server preferably needs to be up 24/7, which means whatever hardware you choose to run it on will essentially cease to exist with regards to usage. Most of the time, hopefully, you’ll never need to do much work on the server itself, only transferring data to and from it.

I already had my networking gear, and I commandeered an old desktop computer for my server. It was a Pentium IV 3.0GHz with 2GB of RAM. I found in the year or so I ran the server that my memory usage didn’t ever go above ~400mb. I also had a 120GB hard disk in there. This computer used to serve as my old gaming workstation, however I don’t play games much these days, and I bought a laptop that was more powerful.

Cost: Free

2. Get Your Software

I chose to run CentOS 5 as my Linux distribution. My choice was admittedly relatively arbitrary, however I was relatively happy with the choice. I installed the OS and included only the packages that I knew I needed – that meant no xserver, and no GUI. I was going to administrate this server via ssh on my laptop. I then updated the OS and installed all the required software.

I immediately recognised that it would be hard to individually install and hook up everything that I wanted to run, and decided to run a web-server software package to help. I eventually settled on Webmin after a few false starts, and Webmin proved to be very user friendly and relatively powerful. If I was doing the same thing again, I would be relatively comfortable doing it all individually without Webmin, but it was Webmin that provided me with a safety net while learning.

Cost: Free

3. Get Connected

I was already running the biggest Internet providing plan that my ISP offered. As I live in Australia, we have to live with usage caps on our Internet plans. Mine was 20GB download during peak traffic, 40GB download during off-peak, and unmetered upload. This was perfect for hosting a server, as most of my traffic was upload, once I got everything installed.

However, you do need more than just an Internet connection to get a server online, you also need a static IP address. Technically you can do it with a dynamic IP and a dynamic DNS service, however that was a level of complexity I didn’t have the patience to deal with. So I put a static IP on my account as well.

Cost: $10/month for the static IP, Internet was free because I was using the connection anyway

4. Get Hosted

This is the final step to getting your server ready to host websites. So far, you have a server with the software installed, and you have an Internet connection and a static IP. You can type your static IP in a browser and get your Webmin’s default landing page (if your router supports NAT loopback, which mine didn’t). To get your web server attached to a domain name, you first must register the domain name, then point it to nameservers that run a DNS server.

You could run the DNS server on the same machine, or a different machine, but that was all too complicated for me. I instead chose to get a hosted DNS solution with Nettica, and have Nettica manage my name servers. Then it was simply a matter of pointing my domains to the Nettica nameservers, pointing my Nettica entries to my server IP, and add the virtual servers with Webmin, letting it take care of configuring Apache for named virtual hosting.

Cost: $50/year

So that’s it. For about $170/year, I could host my own server and serve as many websites as I had room for on the Internet. I had complete control over the server, and I could install what I wanted, when I wanted.

I don’t run the server now, as I have recently moved to a location where I cannot justify it, but overall I found the experience very rewarding. Sure, you can get cheaper shared hosting, but there’s nothing like deciding you want to install, say, SVN and just installing it, and having it up and ready in 10 minutes for no charge.

PHP Has An Identity Crisis

2009-01-11T00:00:00-08:00

I have been using PHP for quie a while. I first learnt PHP 4 when I was 18-19, building a dynamic art gallery type site for my drawing attempts. I thought it was wonderful that I could programmatically display new images without having to update the HTML file. I also used PHP for my final year project in university, and by the time I graduated, I was quite proficient in it.

However, as I slowly got more and more interested in programming, and the principles of programming, I learn more and more languages, and learnt OOP practices that I never applied to PHP, simply because I was self taught, and at the time PHP wasn’t pushing itself as an OOP language.

However, times change, and the PHP community tends to push the fact that PHP is an OOP language. Firstly, this isn’t entirely accurate. PHP is a language that supports OOP, however it isn’t an OOP language. There is no string object – I cannot take a substring of a string by calling a method on the string variable. Instead I must use a global function substr() to do it. That’s not OOP, it’s procedural wrapped in OOP.

However, PHP does support OOP. You can define custom classes, you can create objects and you can call methods on that object. In a hostile world full of more OOP languages like Ruby and Python that are gaining traction in the web development world, PHP must push more and more it’s OOP support. If you talk with serious developers in the world of PHP, they will insist that although PHP is not inherently an OOP language, it’s close enough.

Today I found out that from PHP has introduced in PHP 5.3 the GOTO statement.

For those of you who are unaware of the specifics of the GOTO function, this post, also the post that I learnt this fact from, outlines it. While experienced programmers know enough to avoid the GOTO statement, new programmers may not, and PHP can be abused enough already without the GOTO statement.

So, what is it PHP? Are you an OOP language, or are you procedural? The PHP community pushes it’s support for OOP, and then it introduces a control statement that characterises the worst of procedural code. PHP needs to clarify it’s future as a language, and decide whether to push itself as an OOP language or as a procedural language.

Encapsulation in Python

2009-01-03T00:00:00-08:00

Or, “I see London, I see France, I see Python’s underpants”

I come from a primarily C# background, where I have fairly tight control over how visible my properties and methods are. I can choose to make them private to the class, protected to the class and it’s children, internal to the assembly or public for everything.

In my course of learning Python, I’ve discovered awesome things like docstrings, multiple inheritance and sequence unpacking. However, I’ve also been shocked and made more than a little uncomfortable by the realisation that Python doesn’t implement encapsulation. I don’t trust other people not to abuse access to methods that I’ve intended to be private – I’ve seen too much bad code for that. While I know there are ways to get around this, I don’t feel I should have to perform code gymnastics to implement what I feel is an essential part of an OO language.

Another thing that makes me cringe a little about Python is the need to pass in the self parameter for all object methods. I realise that this is required to differentiate between an instance method and a class/static method, however I just can’t shake the idea that it seems a little clunky, even though I know it’s no different from writing a static at the start of my function definition.

However, to wrap up the last two weeks, I feel confident in saying that from what I’ve seen I like Ruby more than I like Python. That’s not to say these are bad languages; I first learnt web based programming with PHP, and I’m sure that I’m going to leave PHP in the dust for one of these two languages. I will also disclose that I did not learn Python 3.0, and haven’t really looked into it whatsoever, so if 3.0 changes some of the stuff here, forgive me.

Having said that, I will investigate both languages more, and definitely work in both more. I’ve found PHP doesn’t do it for me anymore, I’ve seen a better side of programming where I don’t have to live with ugly warts in my languages, and PHP is ugly. Long live Python and Ruby.

Don't Get the Wrong Degree

2009-01-03T00:00:00-08:00

I recently read an interesting post that popped up in my feed reeder on Saturday night, on a subject that strikes very close to home. Obviously I am not in the exact same situation; I have a degree.

But I have the wrong degree.

The Software Engineering degree I studied was very heavy on the management side of software engineering, and light on the actual implementation side of programming. I don’t know if this is common to SE degrees, however I found this was the case with mine.

I share Daniel’s love and thirst for information, and have a very strong drive to learn as much about everything as I can. So, on the surface, the dilemma of not having the right degree seems fairly harmless. Daniel describes the phenomena of “perception is reality” – people perceive that having a degree is better than not having a degree. So I guess I have a head start in that aspect. I have the piece of paper, I don’t have to deal with these negative perceptions.

Instead, I had to deal with the mistaken belief that I was more competent than I was.

I have never worked in a large company. The company I’m with now is the largest company I’ve ever been with, and we only have 4 developers, including myself. I’ve always worked with smaller companies, that either only dabble in programming, or were start-ups. It wasn’t until I looked at a job posting at a larger company that I realised that I may have made a huge, 3 year long mistake.

The job posting was your standard outline of positional requirements and expected experience and education, but it also had a puzzle attached. You had to complete the puzzle, in your language of choice, and submit the code along with your resume for consideration. This was in late 2006. I can say now that it was a simple graph problem, finding the minimal spanning tree of any given graph. Back then, however, I thought it was just an odd puzzle.

I didn’t know this was a class of problem. I didn’t know this was rooted in mathematical theory, I didn’t know there were algorithms for solving this. I charged head first into solving the puzzle blindly. I ended up with a 90% complete solution that came very close to Prim’s algorithm. I submitted my crippled solution, and naturally didn’t get the position. Later, someone pointed out what class of problem it was, and I was off searching for it.

That’s when it hit me.

That’s when I found out how little I knew about my craft.

Since that day, I’ve regretted every year in that software engineering degree, every year I learnt things that I wasn’t interested in (management) that I could have spent doing computer science. If I could afford it, and if I could afford the time off, I would go back, start fresh and do a computer science degree. However, the more I learn, the more this desire fades. But there is not a day that I do not strive to fill the gaps in my knowledge myself.

I’m not bashing Software Engineering. I know it was my fault for not properly researching the course material offered, I know that I could have gotten what I missed from another institution. The only reason I know this, however, is because I learnt enough about the industry to know what I was missing, and subsequently, know what I needed. And that’s not something every potential high school graduate looking to get into programming will know.

It’s not enough to get a degree – you have to make sure you get the right degree, from the right place. Unfortunately, that generally takes more experience to judge than you have right out of high-school.

Passing Parameters in Actionscript

2008-12-28T00:00:00-08:00

While in the process of trying to learn Python, I decided it would be a good idea to attempt to learn some AI and AI related concepts at the same time using the PyGame framework, as advised by AIGameDev.

For a while, I’ve had this really nice steering demo/white paper in my Delicious account, something that I’ve meant to try implementing myself. Because this was entirely new territory, and I know my physics/math isn’t up to scratch, I decided to try to implement some of the behaviors outlined in the previous paper in Actionscript 3.0 first, then when I got the hang of it, port it to Python.

After about 8 hours or so, I had finally puzzled out the seek/flee behavior, and tried to put together a demo as in the above page, when I hit a strange bug. If I put the seek behavior vehicle on the stage on it’s own, it performed correctly. If I placed the flee behavior vehicle on the stage on it’s own, it performed correctly. If I had both on the stage at once, they freaked out and opposed one another, and the two vehicles often got caught in a state of equilibrium, their velocities canceling each other out.

It seemed as if my little vehicle views were sharing the same vehicle model, and the two different behaviors were acting on the same vehicle model. Which didn’t make any sense, considering they were two separate instances of the same class.

After a lot of bug testing and fiddling around, I finally realise I had been caught by the way Actionscript passes parameters into functions/constructors: unless you’re passing a primitive data type, Actionscript passes by reference.

I was instantiating my vehicles with the same velocity and starting position, and I was using arrays to pass in this information into the constructors, little knowing that the actual array each class was manipulating was the same array.
You can easily bypass this by importing

mx.utils.ObjectUtil

and using the

ObjectUtil.copy

method (remembering to cast it back to your desired data type, in my case an array).
So, if you find your instances seem to be sharing private variables they shouldn’t be, check your parameters and make sure you’re not passing things by reference to multiple instances and manipulating them.

Ruby Wrap Up

2008-12-25T00:00:00-08:00

Today marks the last day of my Ruby portion of my Ruby/Python Bootcamp. Unfortunately with the Christmas and related Christmas stresses, and an impending move, I didn’t get to spend a lot of time with the language, beyond the fundamentals.

I did a dumb thing, and started to try to get a website up and going on Rails 2.2. Rails is a great framework, although I’ve found the learning curve beyond your basic scaffolding and MVC set up to be fairly high. I made extensive use of this Rails API guide and it certainly helped a lot, as did the Ruby on Rails guides portal. The reason this was as dumb idea had nothing to do with Rails or Ruby, and everything to do with the fact I hate making websites.

Seriously, I’m on holiday. I should be staying away from websites for a while.

So, I didn’t end up doing much on the website I promised I’d show, and there’s not really anything to show, yet. I did manage to set up an excellent workflow with SVN and my web-host, which will eliminate the need for an FTP client, and Ruby did help me complete my partial understanding of RESTful design.

So, I walk away after a week of Ruby with a rudimentary knowledge of Ruby syntax, which is enough to let me read Ruby code and fill in the gaps myself, a stronger knowledge of RESTful design and a much smarter development workflow.

Thanks, Ruby.

So, tomorrow, or tonight, I’ll kick off Python, and I’ll also see if I can’t use it while investigating a few other programming related interests, like OpenGL programming or some AI stuff, if I can figure it out. Hopefully I’ll be able to get as much out of Python as I got out of Ruby.

First Impressions Of Ruby

2008-12-20T00:00:00-08:00

I like it.

A lot.

But I always knew I would. I’m so far impressed with how much effort Ruby goes to at making itself understood at a casual glance, and just how easy it is to write self documenting code. Writing code that’s free of braces and parenthesis is refreshing, and it’s nice to know the safety blanket is there (with regards to parenthesis for functions, conditional statements, etc) is there if I need it.

I started last night by flicking through Why’s (Poignant) Guide, and found it pretty good for the basics. I quickly picked up the general concepts and basic syntax fast, and Why’s Guide was pretty whimsical and kept things interesting. However I quickly bored of the absurdity in chapter 5 where the majority of its content is fluff.

I tried switching to SaphireSteel’s The Book of Ruby, however right at the start when the author stated:

if (subtotal < 0.0) then subtotal = 0.0 end

Putting everything on one line like this adds nothing to the clarity of the code, which is why I prefer to avoid it

I knew that this book would not help me. I already know many languages that require simple statements like that to be spread over multiple lines. I want a language to do this:

[code language=“ruby”]subtotal = 0.0 if subtotal < 0.0[/code]

Hi, Ruby.

Then I browsed over to Ruby in 20 Minutes and quickly flicked through the pages, and it concreted my knowledge. I like iterators, I like respond_to?, I like how everything seems to make sense. I also am pleased to see how many concepts map onto similar things in C#. Modules are like namespaces, but better. attr_accessor is like { get; set;}. attr_reader is like {get; private set;}.

I feel I’m ready to go find some code in the wild and attempt to read it. I’ll also see if I can come up with a post that maps Ruby concepts to C# concepts for other C# programmers interested in Ruby (and with IronRuby, there’s no reason why you shouldn’t be).

Ruby & Python Bootcamp

2008-12-19T00:00:00-08:00

It’s common advice in the programming industry to learn one new language every year to stay on top of things. It’s something I honestly believe in as well. Unfortunately I can’t say I’ve learnt a new language this year.

Last year, I learnt Actionscript 3.0, which is different enough from Actionscript 2.0 to be called a new language. I did this because of work, I did it in December and I had to do it fast. That worked pretty well for me. I got thrown in the deep end and had to learn or drown.

This year, with this being my last day at work for 2 weeks, I’m going to do the same thing, but I’m going to make it a little more interesting. I consider myself a reasonably competent programmer, and fairly solid in my knowledge of OOP paradigms. I think I can learn a language pretty quick. So, during my holidays, I’m going to:

See how fast I really can learn a language at least to the point where I know what to search for online and can find my own way around reasonably confidently.

See just how easily various concepts from different languages transfer across syntax

Build some stuff I’ve been meaning to build for a while but kept putting off

With that in mind, this year I’m going to learn both Ruby and Python in 2 weeks, dedicating a week to both languages. In the process of learning these languages, I will also build a base for 2 websites for 2 of my domains that I have been meaning to build. I won’t spend any time designing them, just building the code.

So, over the next 2 weeks, the majority of posts on here will be reflecting my thoughts on Ruby and Python, and linking to resources that I plan to use in my learning, with a final post being the unveiling of the websites I manage to build.

Wish me luck!

Wireframe Your Interfaces First

2008-12-17T00:00:00-08:00

As I’ve mentioned in a past post, I’m doing a rebuild of the CMS here at work and I’ve reached the point where the internals are done, and now I have to create a user interface for the application.

Now, in the past I have worked a variety of jobs and in those jobs a variety of roles. One of my jobs saw me as a web developer, like now, but with a difference: I was the only web developer.

I was responsible for quoting, requirements analysis, planning, design, development, testing, debugging, deployment and maintenance. If there’s only one thing I learned there, it was that web development is more than HTML and Javascript and a server side language, and a freelance web developer needs to know more than just programming and design.

What I really found hard was the design part.

That is, until I read somewhere that design is visual communication. Just like any other form of communication, you need to have a point, something to say. However, that point can very easily get lost in the minutiae of designing, of making sure something looks good.

I’ve found that wireframing is essential. You use the wireframe to establish the point of the page, the information that a page needs to contain, free of it’s appearance. You use the wireframe to tweak the structure, to experiment with different work flows. When you’re done, you go over the top and start designing.

Wireframing has enabled me to quickly build a general structure for a user interface, concentrating on the information and tasks that interface needs to perform.

I wireframe with Balsamiq, which I’ve found to be an excellent tool. You can try it out indefinitely on their site, fully functional, and all it costs is pop-up every 5 minutes asking you to buy it. I’m recommending we buy at least one copy for work, and I recommend you check it out if you want to make effective applications.

Learning Mathematics

2008-12-14T00:00:00-08:00

Although I’m proud of my education and work experience thus far, the most important thing I have come to realise is where the vast gaps in my education are. A little too late did I realise that the degree I really wanted was Computer Science, and instead I studied Software Engineering.

If you need someone to analyse a complex problem and break it down into it’s components, I can do that. If you need someone to manage a project and make sure your resources are optimised, I’m your man.

However, during my degree, we only breifly touched on mathematics and algorithms. If I recall correctly, we covered some basic boolean logic and very basically touched on sets. We didn’t really get into the nitty gritty stuff that I’ve seen CompSci graduates talk about.

However, I’m not the kind of person who would shrug “Oh well” and go about my business – I’m taking it upon myself to learn in my own time.

I don’t know how other people learn. Although I’ve done a little reading into it, I’m still trying to come to terms with how I learn. While trying to study mathematics, I’ve realised I have a tendency to explore various elements of a topic in depth – I tend to dive into the intricacies of a concept, then go back to the shallow level and move on to a new concept.

However, I’ve found that this is very difficult to do with mathematics. All of the higher level concepts rely on lower level concepts, and there is much cross referencing. I might be looking into Euler’s Totient function, which will lead me to the concept of coprimes, which will lead me to rings, which are contrasted to groups, which were referenced back in the totient function! What’s worse is all of these topics, especially the more abstractable rings and groups, extend past the real number systems, which I haven’t even considered yet.

I’ve found the best way to learn mathematics is to go broad and shallow, pick up the basic concept of all related concepts, then slowly work your way down. It results in a kind of net of understanding that isn’t complete, but allows a person to easily recognise holes, and more intuitively link concepts together.

Learning methods and methods of thinking are both subjects that fascinate me, and I’ll probably have a deeper look into it when I get some time. I think these topics have a vast impact one what kind of knowledge people can learn easily, and can account for people having a “knack” for something.

Compile a User Control as a DLL

2008-12-07T00:00:00-08:00

At work, I’m redeveloping the old, ASP 3.0 CMS application and replacing it with a shiny new .NET 3.5 implementation. The software is designed to be run as a virtual directory from all websites, and thus shares the same from installation to installation.

The old CMS is a mutant. It started as an ASP 3.0 application, and then had extra bits made of .NET 1.0 & .NET 2.0 tacked on to it, much in the same manner that Frankenstein’s monster was created. The core ASP 3.0 application handled the page/news creation and publishing, and served as the interface.

The additions were mostly in the form of user controls (.ascx files) that were placed in the content for a page that needed one. These user controls took care of things like random image rotators, contact forms, bread crumb menus, etc. Various sites used a various sub-section of a pool of controls.

The modus operandi with the current system is that each installation has it’s own collection of controls which are inserted into pages by us on behalf of the client, because it involves putting ASP.NET markup into the content areas.

I’ve decided this isn’t such a great idea, and as such, I’m centralising the user controls as a library, seperate from the CMS, so that each site accesses the single pool of controls, and if a client leaves, it will be relatively simple to recompile a new library with the controls they use.

Anyway, that’s the back story. Essentially what I need to do is compile a whole load of .ascx files into a dll, and load them into pages. I’ve spent all morning looking on Google for tips on how to do this. And frankly the results were less than helpful, however I have a history of not reading documentation properly.

What I found out you need to do, is to add a class library project to your solution, and add the .ascx file as per normal. However, you need to do several steps to get this working properly:

Go to the properties of your .ascx file and change the build action to Embedded Resource.

From your control declaration on the .ascx file, delete AutoEventWireup. We need to wire these things up ourselves. Personally I cleared it of everything except the language attribute.

In the old code behind (now unattached class file) for the .ascx, you need to override the OnLoad event handler, and put the following code:
[sourcecode language=“csharp”]
string content = String.Empty;
Stream stream = Assembly.GetExecutingAssembly().GetManifestResourceStream(GetType(), “example.ascx”);
using (StreamReader reader = new StreamReader(stream))
{
content = reader.ReadToEnd();
}
Control userControl = Page.ParseControl(content);
this.Controls.Add(control);
[/sourcecode]
This source will use reflection to open the currently executing dll (in this case, our .ascx library) and find the content for the .ascx, parse it as a new control and add it to the currently executing context.

For every control on the .ascx, in our now orphaned .ascx.cs you need to declare a private/protected variable of the same time, with the same variable name as the ID on the control (for sanity’s sake). You need to find this control in our newly loaded .ascx and assign a reference:
[sourcecode language=“csharp”]
labelInAscx = (Label)control.FindControl(“labelInAscx”);
[/sourcecode]

The above two steps is exactly what the AutoWireUp attribute on the control declaration does for you. We do it manually because we have to use reflection to extract the content of the ascx file, which is compiled into our dll.

After taking these steps, compile your project, and in another project, on a regular aspx page, register the assembly and use as you would any normal .ascx and hopefully the content will render out fine.

Compile .ascx as DLL solution for Visual Studio 2008

0.999... = 1 Is Important

2008-12-04T00:00:00-08:00

Before I get into this, I am not a mathematician. I’m not even an amateur. I’m just a guy who barely passed high school math 7 years ago, and who is interested in learning more.

It’s a generally accepted (though not completely) fact that 0.999… = 1. I’m not going to go into proofs, however if you need to see for yourself, you can find some at the Wikipedia article for 0.999… And I’m not going to debate or even acknowledge that people disagree, because that’s not the point of this post.

The point is, that 0.999… = 1 is an important concept that demonstrates the abstraction of the numeral from the number. The people who see 0.999… = 1 and can’t believe it, I believe, haven’t adequately realised there is a difference between a number and a numeral.

A numeral is a symbol that represents a number – that’s all it is. It is, of itself, inherently useless. The numeral doesn’t mean anything without the number. However, a number itself has meaning, no matter what numeral you use to represent it. This is what’s going on with 0.999… = 1.

The reason this distinction is important is one of advancement in mathematics, specifically in algebra and all other relatively abstract fields. When you realise that 1, the numeral, represents the number 1, and 0.999… the numeral, can also represent the number 1, you realise that 4/4 is also a valid numeral for the number 1. Then, it’s just a very small step to understanding that x can represent the number 1, and that in the equation 2x + y = 5, y also represents a number, and we need to find out that number.

Although the example I gave is trivial, I believe it’s an important concept to grasp, and it’s a concept many people don’t grasp.

Adobe AIR 1.5 Update Framework

2008-12-03T00:00:00-08:00

Lately I’ve been working on a small Adobe AIR application, and one of the last things I had to do for this application was build in auto-update functionality. So, naturally, I hit the all knowing sage with my question, and found many blog posts referencing the official Adobe AIR Update Framework, an additional swc library you can download and include in your project.

Last night I found out that this framework has been rolled into the latest AIR 1.5 SDK, so I checked my version and downloaded the update. My experience with the Flex SDK told me I’d have to find the SDK path in Flex Builder and point it to my new SDK to get the auto-update goodness.

This, however, was not the case – there was no variable. So I searched more online for the answers to my question, and finally found some official Adobe documentation. After a quick browse through, I deemed this useless. I still could not find how to get access to the ApplicationUpdaterUI class.

Eventually I re-read that documentation, and a section that I thought was incorrectly copied from the old update framework download actually held the key.

You still need to copy the swc library to your project.

Only, the swc is in the AIR SDK package, not available as a separate download. Having copied the library to my project, I then had access to the auto-update goodness, but no thanks to Adobe’s confusing documentation.

Interconnectivity of Media

2008-11-27T00:00:00-08:00

On the 27th of November, 2008, terrorists attacked Mumbai, India. They hit several places and killed 80 people (at the time of this post), and wounding many more.

I first heard about this through MSN via my girlfriend, who sent me a news link. Twitter lit up, several tweets about people’s reactions to the news, and with one of those tweets, a link to live CNN footage streaming online direct from Mumbai. A quick look on Google trends showed that the news of the events in India have circled the globe in less than 24 hours.

I saw this as well during the US Presidential elections, and it was even more intense. Twitter was non-stop tweeting about the various speeches and states, every major international news site had a streaming video and up to date Flash based maps for voting, and Google trends was all about the election and Prop 8. I was watching live footage of Obama’s and McCain’s speeches, and footage from Kenya.

I felt it then, and I feel it now: overwhelming awe.

The growth of the internet since 1983, away from a purely academic/military network with text only media, to today with streaming video and interactive content, has truly made our world a global community.

Working day to day on the internet, creating websites to further the individual causes of various people, it’s easy to lose site of how awesome the internet really is, and how interconnected everything is these days.

It’s nice to be able to take a step back.

Wordpress Or Roll Your Own

2008-11-26T00:00:00-08:00

Laziness is a good quality for a programmer to have. Code reuse is smart, it’s fast and efficient, and leads to less bugs (depending on how popular the code you use is).

I know this.

I’ve known this since university. Do I do it? I’m starting to, I’m slowly learning it. This is relevant, because prior to setting up Wordpress on my server, I developed roughly 4 version of my own PHP blogging platform. Two of them were relatively usable though still buggy, and 2 of them never made it past the full execution stack stage.

Now, when I phrase it like this, it doesn’t seem so bad, but I was constantly referencing Wordpress admin screen shots and had installs of Wordpress, Joomla and Drupal to pick and choose interface elements and functionality that I liked from these software applications.

I had planned to start from scratch and slowly build up the featureset of my application to match those of the more popular open source applications. And to get experience I planned to use these applications… but only in a sandbox, which was not publicly accessible.

At the same time as trying to develop, in my own time, my own blogging/CMS solution, I’m upgrading the CMS we use at work. This upgrade is a complete rewrite, migrating the funcitonality from ASP Classic to ASP.NET. This involves designing and developing, on my own, a fully functioning CMS system with support for page editing and a news section.

So I was spending all time at work during the day creating an ASP.NET CMS, then coming home at the end of the day to fire up Zend Studio and do the exact same thing in PHP. I was reluctant to program, and the application was stalling, and I wasn’t blogging.

And the worst, I wasn’t aware of why I was stalling, and why I didn’t want to work on it. I would spend my time at home goofing off, watching movies and chatting to friends. At least I had the intelligence to sit down and design a theme for my blog, once the code was finished. But I still wasn’t developing the blog engine, and the whole process became this big elephant in the room that I was trying to ignore, blocking the doorway to me finally starting to do something I’ve been wanting to do for a while.

I didn’t realise the lunacy of what I was doing until I stepped back and started programming something else entirely unrelated to both PHP and blogs. Once I had done so I realised a shocking truth: I don’t even like programming websites.

At least, not in my own time.

Programming blog engines and CMSes isn’t something I want to spend all my time doing. I want to work on other projects, push the boundaries of my own knowledge – but the blog/CMS thing is a rut that I’ve fallen into. Been there, done that, I know the problem well and I know the solution well.

So, something to take away from my experience is this:

Unless your business/hobby/passion is writing blog engines, just use an off the shelf one. Your time is better spent actually blogging than writing a blog engine. You learn more, it’s generally more interesting, and you’re creating content sooner.

And isn’t content what it’s all about in the end?

Why Blog?

2008-11-25T00:00:00-08:00

Or to put it another way, why you should care about this blog.

Just in setting this blog up, I’ve learnt many things, many things that I can share with people. Is this why I’m choosing to blog? Partially. But the really compelling reason is learning.

I’ve often heard it said that if you want to learn more about something, teach it. Through experiences on online forums, helping people with code, I find that this is a true statement. However, teaching in a classroom is beyond me – both in terms of time and in terms of relative skill levels. So I’m turning to the internet, a blog, to write about and share my experiences. Not because I think I’m qualified to teach anyone, but because I’m not.

So, we come back to the question, “who is this guy, and why should I listen to him?”

I’m a 24 year old developer currently living in Perth, Western Australia. I have a Bachelor of Science (Software Engineering) that I obtained from the Edith Cowan University. I have been working in the field of programming since late 2005. I have worked as a debugger/maintenance programmer for a desktop application for a now gone Perth startup, I have worked as a one-stop web development shop for a small company that also doesn’t exist, and am currently a backend developer for a web development company.

But none of that is important.

I have worked in Actionscript 2.0, Director Lingo, ASP 3.0, PHP, ASP.NET (C#) and Actionscript 3.0, all in major projects. I have worked extensively with MySQL, PostgreSQL and SQL Server. I have developed both desktop applications and web applications, as well as web services and desktop Windows services. I am familiar with Windows OSes, OS X and Linux based systems. I can adequately administrate both IIS and Apache servers.

None of that is important either.

What is important is that I have made mistakes. A lot of mistakes. About 10 years worth of mistakes. From those mistakes, I have learnt a lot, and I have clear goals in life.
I have made lifestyle mistakes, scholastic mistakes, career mistakes and mistakes in programming. I have tanked a couple of large projects, and have sabotaged my own efforts at starting and maintaining a small business.

From these mistakes comes a passion for correcting what I can – and to do that, I have to learn. I have to upgrade my skills, I have to grow as a person. I intend to do that by writing about them. Both to sort my own thoughts out, and to recieve feedback and learn from that feedback.

That’s why you should listen to me. And tell me when I’m wrong, and tell me when I’m being stupid and tell me when I am getting it right.

I want to learn everything I can, and while learning, hopefully give something back.