Paramiko is a python library that allows you to make SSH connections. Useful when you want to automate things as you can easily ssh into remote server from your local computer.

The code down below will let you ssh to a remote server via Paramiko and then SCP files from that server to some other remote server ie. copying/transferring files from one remote server to another remote server.

Here’s what it does:

• SSH to remote-server-1
• Execute a psql command to export entries as a CSV file on the server
• SCP that generated file to remote-server-2
• Remove the generated file from remote-server-1 once its successfully copied.

import paramiko
from scp import SCPClient
from paramiko_expect import SSHClientInteraction
remote_server_1 = "remote-server-1.com"
remote_server_1_username = "ssh_username"
remote_server_1_password = "ssh_password"
remote_server_2 = "remote-server-2.com"
remote_server_2_username = "ssh_username2"
remote_server_2_password = "ssh_password2"
p = paramiko.SSHClient()
p.set_missing_host_key_policy(paramiko.AutoAddPolicy())
filename = 'filename_whatever'
export = "psql -U username database -c 'COPY (SELECT * FROM some_table WHERE id=1) TO " + filename + ".csv" +" With CSV HEADER;'"
p.connect(remote_server_1, port=22, username=remote_server_1_username, password=remote_server_1_password)
stdin, stdout, stderr = p.exec_command(export) # executes psql command once connected
stdin.write("psql_passwordn") # password for psql user
print("Entering")
stdin.flush()
status = stdout.channel.recv_exit_status() # wait till file has been exported
if (status==0): # if successful in generating file scp it to remote server
   print("Copying..")
   stdin_, stdout_, stderr_ = p.exec_command("expect -c 'spawn scp ./" + filename + ".csv " + remote_server_2_username + "@" + remote_server_2 + ":~;expect password;send \"" + remote_server_2_password + "\r\";interact'", get_pty=True)
   status_ = stdout_.channel.recv_exit_status()
   stdin_, stdout_, stderr_ = p.exec_command('rm ' + str(filename) + '.csv') #remove file after copying
   status_ = stdout_.channel.recv_exit_status()
   print("Done: ", str(filename))
else:
   print("Error")
   print(stderr.readlines())
p.close()

Here we are copying files from remote_server_1 to remote_server_2. We ssh into remote_server_1 and then with the help of paramiko’s exec_command method, execute the terminal command to log into psql and generate a file (ie. whatever’s in ‘export’ variable). Once we execute the psql command, it will ask for password of the user. We provide the password by writing it to the stdin: stdin.write(“psql_passwordn”).

We then wait until the command has finished executing: stdout.channel.recv_exit_status(). If the execution was successful (ie status == 0), we execute another command to copy the file to remote_server_2. This command may look complex, but all we are doing here is executing scp with the help of expect. Then we finally delete the file from remote_server_1: p.exec_command(‘rm ‘ + str(ids) + ‘.csv’) .

Hope this was helpful!

The post SCP between two remote servers using Paramiko in Python appeared first on The Lacuna Blog.

By default, scp doesn’t let you pass password as a parameter. For instance, for mysql, I can do something like: mysql -uusername -ppassword and login without having to give username and password input separately. That is very helpful while scripting or automating things because that’s one less case to handle.

So how do you pass password as parameter in scp? You cannot. At least not using scp on its own. The best method I could find is by using expect.

expect is a scripting language used to deal with scripts/programs that require user interaction. Here we’re going to use expect to start scp (with spawn) and send password as our response without any interaction from our end — all in a single line.

expect -c 'spawn scp ./path/to/source/filename user@server.com:~/path/to/destination;expect password;send "serverpass\r";interact'

That’s it. Obviously, ./path/to/source/filename.csv is the path to your file that you are trying to copy (source file), /path/to/destination is the path where you are trying to copy it to on a remote host (destination directory), destination remote host here is user@server.com, and serverpass is the password of the destination host.

You may need to install expect if its not already installed:

• apt-get install expect
• yum install expect

The post Passing Password to SCP with expect (single line) appeared first on The Lacuna Blog.

What do you do when you want to track whether a particular application has been launched on Android?

METHOD 1: POLLING TASK LIST REGULARLY

Unless the application that you want to monitor gives specific broadcast intents when it launches, the only way is to regularly poll the running task list and get the topmost activity off of that list — that would be the foreground app. So, you could, in theory, have a background service running every 5 seconds, polling the running task list in order to achieve this. (Remember 5 seconds is too low and Alarm Manager is probably not the right thing for it as 1: its not exact timing based and 2: system will move things around for timers less than 1 minute — so your alarm won’t be launched that often all the time.)

In your activity, you could create the following method to create a 1 minute timer with AlarmManager. This would create a broadcast intent. Don’t forget to call the method!

  private void setAlarm(){
  private static final long timeInMillis = TimeUnit.MINUTES.toMillis(1);
        AlarmManager alarmManager = (AlarmManager) getSystemService(Context.ALARM_SERVICE);
        Intent intent = new Intent(this, MyAlarm.class);
        PendingIntent pendingIntent = PendingIntent.getBroadcast(this, 0, intent, 0);
        alarmManager.setRepeating(AlarmManager.ELAPSED_REALTIME, SystemClock.elapsedRealtime(), timeInMillis, pendingIntent);
    }

Now, you would create a MyAlarm broadcast receiver such that when the AlarmManager fires the broadcast intent every 1 minute, you will catch it on your receiver and process it. Note that you will need to declare the broadcast receiver on the manifest file.

public class MyAlarm extends BroadcastReceiver {
    @Override
    public void onReceive(Context context, Intent intent) {
      // code to check the task list.
        }
    }
}

We have the code for the repeated 1 minute call setup! Next, we put together the code to check the task list.

String topPackageName = null;
        if (android.os.Build.VERSION.SDK_INT < android.os.Build.VERSION_CODES.LOLLIPOP) {
            ActivityManager am = (ActivityManager) this.getSystemService(Context.ACTIVITY_SERVICE);
            ActivityManager.RunningTaskInfo foregroundTaskInfo = am.getRunningTasks(1).get(0);
            topPackageName = foregroundTaskInfo.topActivity.getPackageName();
        }else{
            UsageStatsManager usage = (UsageStatsManager) this.getSystemService(Context.USAGE_STATS_SERVICE);
            long time = System.currentTimeMillis();
            List<UsageStats> stats = usage.queryUsageStats(UsageStatsManager.INTERVAL_DAILY, time - 1000*1000, time);
            if (stats != null) {
                SortedMap<Long, UsageStats> runningTask = new TreeMap<Long,UsageStats>();
                for (UsageStats usageStats : stats) {
                    runningTask.put(usageStats.getLastTimeUsed(), usageStats);
                }
                if (runningTask.isEmpty()) {
                    topPackageName="None";
                }
                topPackageName =  runningTask.get(runningTask.lastKey()).getPackageName();
            }
        }
        Log.e("Task List", "Current App in foreground is: " + topPackageName);

Put the code above in the onReceive method of the MyAlarm broadcast receiver that we created earlier and that’s all setup.

You will need to add the following permissions to the manifest file.

<uses-permission android:name="android.permission.GET_TASKS" />
<uses-permission android:name="android.permission.PACKAGE_USAGE_STATS" tools:ignore="ProtectedPermissions"/>

Downside to this method is that it does not seem to work 100% of the time. I noticed that sometimes it would return launcher as the foreground app even though there’s Chrome running on the top. So, use it with caution. The next method although very reliable, cannot be used if you’re going to put your app on PlayStore.  :)

METHOD 2: USING ACCESSIBILITY SERVICE

Like I mentioned above, you won’t be able to go very far with this method if you’re trying to put your app on Play Store. This is because this method makes use of AccessibilityService, a service that’s meant to be used by Accessibility related applications only. So we are kind of misusing it. But, depending on your use case, this might still be useful to you!

What are we doing? We are using AccessiblityService to listen to a particular event i.e. WINDOW_STATE_CHANGED event. So whenever a window state change occurs on the phone, our Service will get a callback or sort of a broadcast. We can then narrow down to the application’s package name, and see which application is on the foreground. That means we won’t need to poll the task list regularly like in Method 1.

The method is explained in detail over here on StackOverflow.

Thank you!

PS. None of the codes above were originally written by me. They have been put together by copying off of StackOverflow and other websites.

The post How To Detect Foreground Application in Android? appeared first on The Lacuna Blog.

Have you thought about why most of the online compilers do not allow you to give input? I mean, there are online compilers that allow you to type your code and run it, but they don’t accept input thereafter. Suppose you made a C program that accepts input from user. You can compile & run it online. But, you won’t be able to pass it input when it asks for one because the online compilers restrict you to do so.

Why? Because they don’t use websocket. If you don’t use websocket, the only way to achieve something like this is to regularly poll the program to see if its waiting for the standard input. If it is, then you’ll send a response to the web user asking for the input that the program is waiting for. But that’s easier said than done, because polling is not at all efficient, and secondly, its complex to write it. The other reason online compilers do not allow input is related with security.

Websocket is meant for interactive duplex communication. And there are already libraries that help you  emulate the whole terminal over on the web via websocket. That means you just show the whole terminal to the user and then the python script, or c code or whatever the user wanted to compile, would be compiled and ran on the same terminal. User would be able to interact with it and enter the input as well — problem solved!

What about security? Wouldn’t users have access of a terminal session that could possibly bring down the VPS/Virtual machine and wreck havoc (Remember we’re just exposing the terminal on the web, but it IS RUNNING on the VPS/Virtual Machine actually — that means the users have access to a terminal session of the virtual machine) ? Yes, that is what happens if you don’t get smart about it.

A simple way to avoid the problem is to use a sandbox. Meaning, you don’t expose the actual terminal. You expose a terminal that is run inside a sandbox. So whatever the user does on the exposed terminal will have no impact whatsoever on the system that is outside the sandbox, and better yet, the sandbox programs already restrict the user from doing a lot of actions on terminal. For example, the sandbox program I have used called Firejail not only has a lot of restrictions in place, but also allows users to add their own restrictions.

So here’s how I’ve structured the project:

• Using Ace editor, the user can type the codes.
• When he clicks on RUN, the code gets saved on the virtual machine. I’ve used REST API for this purpose, but using websocket for this would be better — websocket is comparatively faster than REST (in this case).
• Then, a terminal session is initiated inside a sandbox (firejail). It is exposed to the web via terminado.
• The initiated terminal session is not bash, but restricted bash aka rbash. Rbash has already many built in restrictions in place. For example, it does not allow you to type ‘/’ or change directory (‘cd’). I did this just to make it extra secure — a restricted bash inside a sandbox that is already restricted.
• There’s a unique terminal for every user. So, when a user quits, the sandbox also gets shut down.
• As it is with nginx, the user is automatically disconnected / connection is halted if there’s no activity for ~30 seconds i.e. Terminal exits in case of no activity for 30 seconds.
• With the help of cron job, I’ve set in further two restrictions: terminal session is closed after 10 minutes and any program running longer than 1 minute in the terminal is killed automatically (to handle programs taking too much memory because of infinite loops etc.).
• I’ve used monit, a system monitoring utility to monitor resource/memory usage of the application and restart the API, Nginx, and the major application in case the usage exceeds a sudden threshold. This is required in case the resources are hogged when there are several users online compiling at the same time.

I am thinking of writing a more detailed step-by-step guide on how to make an online code compiler/code judge — I may eventually write an eBook for it since it is a lot to be covered in a blog post. I’ll of course share the news here if I ever get around to it. :)

For now, you can find the code for the program here: https://github.com/sknepal/webcompiler.

PS. Wrote the post on a single sitting; so please try to ignore the shortcomings you see. :P

The post How To Make An Online Code Compiler? appeared first on The Lacuna Blog.

I have been lurking around /r/FractalPorn so much that I finally decided to put my love for Fractals down on words. These structures are hauntingly beautiful and on this post, I want to share their beauty with you. Although not as technical as previous posts, it should be interesting nonetheless.

A fractal is a never-ending pattern. Fractals are infinitely complex patterns that are self-similar across different scales. They are created by repeating a simple process over and over in an ongoing feedback loop. Driven by recursion, fractals are images of dynamic systems – the pictures of Chaos. – fractalfoundation.org

We will go about making our own Mandelbrot Set, which is one of the easiest and most famous fractals. Here’s a beautiful example of a realtime Mandelbrot Fractal zoomer: http://jblang.github.io/XaoSjs/. Left click on an area to zoom-in and right click to zoom-out. You can see that the fractal is infinitely detailed.  Our implementation would not be as advanced; it would just display a static image.

Mandelbrot Set is a set of complex numbers c for which the function z=z² + c does not diverge. The idea is simple: we just need to perform iteration over a function. In the formula  z=z² + c, both z and c are complex numbers. We begin the iteration with z set to 0 i.e. 0+0i. c is a constant and remains the same throughout the process, whereas z‘s value is updated with every iteration.

We run the iteration for, say, a 100 times. Then, the value of z changes throughout the 100 iterations. If the value goes beyond a set limit, then we know it has diverged, whereas if it does not leave the set limit even after performing those 100 iterations, then it is said to have converged (for that 100 iterations).

Let,

z = (a +bi) and c = (d+ei)

Then,

z² = (a²-b²) + (2ab)i

z² + c =  (a²-b²) + (2ab)i + (d+e)i = (a²-b²+d) + (2ab + e)i

We now need to do the same thing that we did above for a range of complex values c. We start with a xy plane that represents the real components on the x axis and the imaginary components on the y axis. Then, we take a particular point on the plane, say (1+1i), test it for convergence, just like we did above. In order to do that, first of all it is assumed to be constant for the iteration period and then perform a iteration, say 100 times, starting with z set to 0. Then we compare the absolute value with our threshold each time and if its beyond the threshold then we say it has diverged. We then choose another point in the plane, say, (1+2i) and do the same for it . In this way we test for convergence of complex number c that is within the xy plane.

Now you should be beginning to get an idea as to how the fractal is visualized. In each of those xy values that we tested, if c converged, then we could give it one value, and if it diverged we could give it another value (read: 1 if converges, 0 if diverges or vice versa). How do we make it colorful, though?

In order to make the fractal colorful, we will store in an array the exact iteration count it took for the absolute value to grow beyond our chosen threshold. Then finally we will plot that array. Remember that any point in the plotted array is a complex number which has its real component on the x-axis, and the imaginary component on the y-axis. We’re plotting a complex number on the plane.

close all;
clear all;
clc;
xvals = linspace(-0.10, -0.90, 1000); %x axis plane size
yvals = linspace(-0.03, -0.69, 1000); %y axis plane size
xlength = length(xvals); % length of the real component
ylength = length(yvals); % length of the imaginary component
fractal = zeros(xlength, ylength); % the final image array
                        % to store the iteration count
                        % that results in color

                        
for i=1:xlength 
    for j=1:ylength % iterate through the components
        x = xvals(i); % pick a real value
        y = yvals(j); % pick an imaginary value
        complex_no = complex(x, y); % create a complex number
        fractal(i,j)=mbrot(complex_no,100);
                    % and check it for convergence/divergence
    end
end


image(fractal.') % show the fractal. the ' transposes the array
    % without it the image will display sideways
    % so its just for convenience
colormap(winter)

function itercount = mbrot(complex_no, max)
    z = complex(0,0); % initialize z = 0+0i
    for itercount=1:max % iterate till max no of iterations
        z = (z*z)+complex_no; % z squared + c
        if abs(z) > 4 % if absolute value is greater than 4
            break % exit loop else continue till you reach 
                  % the max no. of iterations
        end       % then return the iterations reached
                   % in case the z diverges, then iteration
                  % count will be less than max
                  % whereas if it does not diverage, iteration
                  % count will be equal to max.
    end
end

I have commented the code to make it as descriptive as possible. Here are some of the fractals produced from the above code. You can try it out yourself by changing the xvals, yvals and max number of iterations. For a more detailed study into Mandelbrots, you can read the book Make your own Mandelbrot by Tariq Rashid. I have shamelessly borrowed several concepts from the book. ;)

Maybe next time, we will look into how Julia sets (another fractal) relate to Mandelbrots.

PS. Don’t forget to go through this article: How Mandelbrot’s fractals changed the world

The post For The Love of Fractals appeared first on The Lacuna Blog.

I’m pursuing a Bachelors in Science, Computer Science and Information Technology (BSc. CSIT) in Nepal. About fourth semester in, I realized I could not just rely on what I was taught in the class. For one, I didn’t understand what the teachers said. They made things complex than it was absolutely necessary and obviously not all of them were qualified in the subject matter. I prefer learning from videos than from texts, so I started looking for MOOCs.

This is a list of MOOCs that I followed during my four year study of BSc. CSIT (although I started following MOOCs since fourth semester only). As of the moment of writing this, I am in seventh semester so this list is in no way comprehensive or complete, for that matter. It’s just something I hope would help you along the way just like it helped me.

Fourth Semester

• Theory of Computation: Jeffrey Ullman, the author of the textbook recommended for the course, hosted an excellent MOOC on Coursera in 2012 that covered most of the topics mentioned on CSIT syllabus. It is not available on Coursera now, but you can watch it on YouTube. Another one you might find worth watching (especially for the latter parts in the syllabus) is by Prof. Harry Porter (no kidding): Theory of Computation.

• Database Mangement System: Available via NPTEL Open Online Course (NOC), this series on Fundamentals of Database Systems, taught by Prof. Arnab Bhattacharya of IIT Kanpur, closely relates to our syllabus.

• Computer Graphics: I followed Prof. Sukhendu Das’ (IIT Madras) lectures. It covers most of the things, but you’ll also need to refer to books/notes (I’ll mention a neat trick for it down below).

Fifth Semester

• Computer Networks: I followed the Computer Networks MOOC which was hosted by Prof. David Wetherall (University of Washington). It was an awesome course where the professor even shared the advances in the current networking technologies, which even my college’s professor didn’t know of, and was not mentioned on the textbooks either. One of the reasons why I love MOOCs. Unfortunately, the course is no longer available on Coursera and I couldn’t find it on YouTube.  I was able to find it preserved by the Archive Team. But, you’ll need to download a huge archive (44G to be exact :O ) to view it. Here’s the gist file that contains the download link and the related slug name: https://gist.github.com/mihaitodor/b0d8c8dd824ab936c057508edec377ad. Search for “Computer Networks” and download the archive that is linked just above it.

• Design and Analysis of Algorithms: One of my friends suggested me the NPTEL MOOC hosted by Madhavan Mukund (Chennai Mathematical Institute). It’s very easy to understand and very thorough. I’d recommend it to anyone. It was available on YouTube as a Playlist. But seems like they’ve changed the videos to unlisted status, so now you need to go through their site’s course listing.

• Neural Networks: This course is very vast. I mean, it covers a lot of grounds that are not at all introductory. Our teacher claimed that it is better suited for graduates. You can find several resources to learn Neural Network on the web. Nonetheless, the best ones include the well renowned course on Machine Learning taught by Prof. Andrew Ng. If you like, you can follow it up with Prof. Geoffrey Hinton’s course. For a practical introduction, I highly recommend you to go through the book Make Your Own Neural Network by Tariq Rashid.

• Cryptography: There are a few courses on Cryptography on Coursera: Course 1, Course 2. I followed these selectively, because I found many of the videos difficult to understand.

Sixth Semester

• Real Time System: It is one of those subjects that you just can’ t get enough of. Just kidding. Real Time System was hated by most of my friends, and I did too. Until I came across an MOOC, that is. It made life easier. Although it doesn’t cover everything on our syllabus, it certainly makes most of the things understandable. Do not completely depend on it, as it does not cover things in as detail as the book does, and the later chapters are not even mentioned. But, it does make it easy to understand the basic concepts so that you can follow what’s being said on the book. The course is available on Coursera.

• Compiler Design and Construction: Prof. Alex Aiken’s MOOC is very thorough and understandable. But, the semantic analysis part differs from the one in our syllabus. So, I had to follow it up with lectures by Sarfaraz Masood. However, these videos are quite fast and may not be suitable for someone who hasn’t gone through the text already.

How & Where to get Notes?

Sure there are several sites where you can get notes. But, what I usually do is, I google. As you know, most of the part of our course is lifted off of Indian courses. So, I usually look up for Indian notes as well. Faadooengineers is a site with study materials for Indian students, and the forum is where users post their notes and study materials. So, what you can do to search for notes is, go to Google and search for: “site:Faadooengineers.com subject notes“. Obviously, you will need to replace subject with the name of the subject whose notes you’re searching. You might need to register before you are able to download the attached files. I have found some useful notes this way. I even saw one of my teachers downloading lecture contents from the site.

Bonus 1: Notes of every semester.

Bonus 2: List of final year projects.

The post MOOCs for BSc CSIT appeared first on The Lacuna Blog.

It took me about a week of my Dashain vacation to make this simple captcha breaker (5 days to be exact). I had been reading Image Processing in my CS undergrad for some time then, and wanted to put my skills to use, although, I must admit, its not as overly complicated or full of image processing bits as I’d have likely enjoyed. I had only learned the basics when I started doing this. If I were to do it now, I would have definitely done it differently.

I decoded a simple text based CAPTCHA with an acceptable level of accuracy (acceptable to me, that is). I did not generate any CAPTCHA for the purpose, but instead used a live website to grab the CAPTCHAs from. A now-defunct WordPress plugin called Captcha on Login generated these CAPTCHAs, and the plugin is still used by several websites.

With the help of the selenium module, I put together a Python script to take screenshot of the website whenever the CAPTCHA was shown, and then simply crop the image to fit to the CAPTCHA and save it to my local disk. In this way, I prepared a training and testing set for my Neural Network which I was going to use for recognizing the letters.

The reason I called it a simple text based CAPTCHA is because, all the characters have a uniform color, they are not connected, and the characters have a constant font. The background color is also uniform although completely different from the color of the characters, and there are no distortions, rotations etc. That means, a lot of the hardwork that goes into identifying text locations, and extracting characters from the background was unnecessary. I only needed to focus on segmentation, and its not difficult either, given that the characters are not connected, and the background and text color is clearly separable.

The first image is what the CAPTCHA looks like. It’s easier to work with grayscale image than it is with a colored image. So, I converted it into a grayscaled version, which you can see on the second image. Most of the work has been accomplished by using the pillow imaging library in Python.

How did I do it? First, I converted it to a GIF image so that the available colors are reduced to 256 (GIF supports 8 bits per pixel, that means, the maximum number of colors a frame can use is 256). Then, as you can see, the characters have clearly a distinct color from the background. So, I made a color histogram of the now converted image, and then chose a pixel that is moderately common. In the above CAPTCHA, orange color (i.e. the background) is the most common. Using the color histogram, I tried to take a guess at the character’s pixel (i.e. Red colored pixel) since all the characters in every CAPTCHA had red color. Then, I made a new image the same size as the first image with a white background, and then went through the first image, looking for the pixel of the color that I had chosen earlier, then when I found that pixel, I marked the pixel on the same position on the second image as black. I continued this until I had correctly chosen the pixels for red color. Turns out, anything below 52 would give me just the characters from the original image. And so, I could successfully convert the original CAPTCHA with background to a binarized image.

The next and major step is segmentation. The neural network would only be able to recognize one character at a time. That means, I need to feed one character’s image at a time. The grayscale image that we now have, needs to be divided into separate images for each characters that it contains, and that is done with the help of vertical slicing.

This is actually pretty easy in this case because the characters are not connected at all. If they were, I’d have to put more effort into untangling them. But, in this case, I can keep track of where each character starts and ends and just extract that part.

The way I have done that is iterate through the column of the image first and then through each of the rows in that column. I then check for the value of the pixel at that row and column, and if it is 255, I know that it is background. If it is not 255, however, I know that I’m inside a character, and I mark the value of the current column as the starting location. I keep on iterating until I find out the immediate next column where all of the rows have a pixel value of 255, which means that the column only contains the white background. And then I mark that column as the end location for that character.

In this way, I build an array with starting and ending point for each of the characters. The problem with this however, is that, sometimes there are gaps in the pixels i.e. some characters might miss a pixel or two because earlier when we converted the image to grayscale, we only chose to blacken the pixels that were under 52, so some of the pixels will be missed. So, I added a check to make sure that the new character is indeed a new character, and not the gaps between the pixels of the same character. If the difference between the end location and start location is greater than 5 then I took it as a new character otherwise, I assumed it to belong to the same character. I then cropped the image using the starting and ending point of each of the characters.

This is what I would have done differently had I done it today. Instead of such a naive assumption of the pixel distance, I would have just used gaussian blur, morphological processing such as erosion and dilation or line thickening to tackle this issue. Unfortunately, when I did this, I was yet to learn about all these things. So, one thing you can do to improve this approach, and maybe increase accuracy, is to try these out rather than my naive assumption way.

Since the Neural Network expected a uniform input, I created images of 60×60 dimension with white background, and placed the cropped image of each of the characters in the middle of that image. So, each cropped character was saved with a dimension of 60×60 in the middle of a white background. And thus, all of the characters were successfully segmented.

Now the only part left is to train the Neural Network. Each of the segmented characters were saved into their respective directories identified by their number in serial order (starting with 0, 1, 2, …, 9 for digits, 10, 11, 12, …, 35 for alphabets, and 36, 37, 38, …, 61 for capital alphabets). Some of the digits and alphabets were not used in the CAPTCHA, so some of the folders turned out to be empty.

The neural network is implemented the same way as it has been before. Using 3600 input nodes, 2700 hidden nodes, 62 output nodes, and a learning rate of 0.005 gave me the best result. Just a single run did not give me satisfactory result, so I trained the neural network by epoch for about 30 runs. With a training set of 1350 samples, and a test set of 50 samples, it was able to recognize characters with a 90% accuracy. In the end, the CAPTCHA breaker was tested with 200 CAPTCHAs and it recognized almost 80% of it — and that includes the process of reading the image, converting it to grayscale, segmenting each characters, feeding the extracted characters to the neural network in order to recognize it, and then concatenating the outputs of the neural networks together. I also tried using Tesseract directly on the CAPTCHA, and it gave me a 25% accuracy.

As I’ve mentioned before, it was a simple CAPTCHA, and I broke it with such minimal effort that it should be a lesson not to roll your own CAPTCHA. Everyone should be using reCAPTCHA. Now go ahead, look for sites that have weak CAPTCHAs and try breaking them. Here’s one for you to try out your skills.

All the codes and the dataset are available on my GitHub Repository. For your convenience:

• Code to grab CAPTCHA from the website
• Code to prepare the CSV dataset for the characters
• Code to train the Neural Network
• Code for the main program (i.e. read, segment, resize, feed into Neural Network, and concatenate output)
• Training set for the segmented characters: Actual Images | CSV File
• Testing set for the segmented characters  : Actual Images | CSV File
• Testing set for the whole CAPTCHA           : Actual Images

Next challenge:

Hidden Markov Models.

The post Breaking a simple text based CAPTCHA appeared first on The Lacuna Blog.

Neural Network was one of the electives available during my 5th semester, and knowing that it was available, I was certain that I wanted to study it over any other electives. But, unfortunately (or fortunately), my college, as it is with many other colleges of Nepal, made it a compulsion to take Cryptography as the elective subject. I, obviously, wanted to study Neural Net and could not convince the college on teaching ANN instead of Cryptography. I was left with no choice but to study both, which I did. A few of us studied both the subjects as an elective. Since the teacher was not available during the weekdays, we took classes on every weekend for 3 hours each. So, there were no holidays for us, for 4 months, I think.

When it came time to study, I enjoyed Cryptography more than I enjoyed Neural Network. I’d say, this is in part due to the course content, as well as the teacher’s ability to make the class interesting. Anyway, as the teacher put it, we had to implement the “hello world” problem of Neural Network as a class project. Meaning, handwritten digit recognition using MNIST dataset. I followed a great practical introductory  book that helped me understand and implement it, which I will share just in a moment.

Having tried out the MNIST dataset, I decided I should give Nepali handwritten digit recognition a shot too. Looking around, I found a dataset provided by Computer Vision Research Group, Nepal. It contains both the devanagari letters and digits. I chose to work on just the digits. Since reading the images one by one and doing the computation would have been restrictive, I extracted the pixels from all the images and stored them in a CSV file. That way, its easier to feed the pixels into the neural net, and also to share the dataset among enthusiastic readers who would want to try it out themselves.

All the images of the digits in the original dataset were used. In total, 1700 examples of each digit was used to create a training set and 300 examples of each digit was used to create a testing set.

In the CSV file, the first value in the row is the actual value of the image, ie. the digit it represents, and the other values are its pixels. All the images in the dataset are of 32×32 dimension, so in total there are 1025 values (or columns) for each images including the first value which is the digit the image represents. The new dataset was then shuffled in order to avoid the occurrence of same kinds of examples one after another, which could lead to an inefficient learning.

Dataset

You can learn more about the dataset and download it from here.

Neural Network

Now that we have pre-processed the existing raw images in order to create a dataset that is easy to work with, we have a training set and a testing set.

Our Neural network will consist of 3 layers: an input, hidden and output layer. The number of these are governed by our data. Since we have 1024 pixel values for each images, there will be 1024 input nodes. Similarly, since our output needs to be one of the 10 digits (0 to 9), that means we will have 10 output nodes. Whichever node has the highest value will be the output of the network. However, the science of hidden layer is not quite clear, and what is done here is rather a simple guess at choosing the number of hidden layers. It is expected to be less than the input layer, and a value of 300 has been chosen. The learning rate is also chosen without any scientific reason whatsoever, at 0.3.

By Offnfopt [CC BY-SA 3.0]

Signals are transferred from input layer to next layer by multiplying the matrix of weight values and inputs. The weights are chosen randomly between -0.5 and +0.5. The output of this multiplication is the combined moderated signal into the hidden layer.

We have used sigmoid function as an activation function. The combined moderated signal is passed to the sigmoid function, 1/ (1 + e^-x ), which squashes the input and gives an output for the layer. The same process is repeated between hidden and output layer.

The final output of the network is compared with the expected output in order to calculate the error. We can compute the error in the output layer by simply subtracting its output to the expected output. But, to learn the error in the hidden layer, we have to propagate the error backwards i.e. we need to split the errors according to the weight of the contributing links. This can be obtained by multiplying the matrix of weights of hidden layer to output layer and the error of the output layer.

Once we have the error, we need to update the link weights according to the error. We use gradient descent algorithm to refine the weights so that we can minimize our cost function.

Output

An accuracy of about 95%.

Code

Available on my GitHub: https://github.com/sknepal/devanagari_digit_recognition

If all these seemed foreign to you, I strongly recommend you to read this book called Make Your Own Neural Network by Tariq Rashid. The python implementation of the Neural network on GitHub is based on the book.

Next Challenge

Breaking a simple text captcha. 

The post Handwritten Devanagari Digit Recognition using Neural Network appeared first on The Lacuna Blog.