CS.Techruto

Creating a Ruby DSL: A Metaprogramming Tutorial

2017-10-07T23:09:00.001-07:00

This article is written by an author from toptal.com ( Origin )

Domain specific languages (DSL) are an incredibly powerful tool for making it easier to program or configure complex systems. They are also everywhere—as a software engineer you are most likely using several different DSLs on a daily basis.

In this article, you will learn what domain specific languages are, when they should be used, and finally how you can make your very own DSL in Ruby using advanced metaprogramming techniques.

This article builds upon Nikola Todorovic’s introduction to Ruby metaprogramming, also published on the Toptal Blog. So if you are new to metaprogramming, make sure you read that first.

What Is a Domain Specific Language?

The general definition of DSLs is that they are languages specialized to a particular application domain or use case. This means that you can only use them for specific things—they are not suitable for general-purpose software development. If that sounds broad, that’s because it is—DSLs come in many different shapes and sizes. Here are a few important categories:

Markup languages such as HTML and CSS are designed for describing specific things like the structure, content, and styles of web pages. It is not possible to write arbitrary algorithms with them, so they fit the description of a DSL.
Macro and query languages (e.g., SQL) sit on top of a particular system or another programming language and are usually limited in what they can do. Therefore they obviously qualify as domain specific languages.
Many DSLs do not have their own syntax—instead, they use the syntax of an established programming language in a clever way that feels like using a separate mini-language.

This last category is called an internal DSL, and it is one of these that we are going to create as an example very soon. But before we get into that, let’s take a look at a few well-known examples of internal DSLs. The route definition syntax in Rails is one of them:

Rails.application.routes.draw do
  root to: "pages#main"

  resources :posts do
    get :preview

    resources :comments, only: [:new, :create, :destroy]
  end
end

This is Ruby code, yet it feels more like a custom route definition language, thanks to the various metaprogramming techniques that make such a clean, easy-to-use interface possible. Notice that the structure of the DSL is implemented using Ruby blocks, and method calls such as get and resources are used for defining the keywords of this mini-language.

Metaprogramming is used even more heavily in the RSpec testing library:

describe UsersController, type: :controller do
  before do
    allow(controller).to receive(:current_user).and_return(nil)
  end

  describe "GET #new" do
    subject { get :new }

    it "returns success" do
      expect(subject).to be_success
    end
  end
end

This piece of code also contains examples for fluent interfaces, which allow declarations to be read out loud as plain English sentences, making it a lot easier to understand what the code is doing:

allow(controller).to receive(:current_user).and_return(nil)

expect(subject).to be_success

Another example of a fluent interface is the query interface of ActiveRecord and Arel, which uses an abstract syntax tree internally for building complex SQL queries:

Post.                               
  select([                          
    Post[Arel.star],                
    Comment[:id].count.             
      as("num_comments"),           
  ]).                               
  joins(:comments).                 
                                    
  where.not(status: :draft).        
  where(                            
    Post[:created_at].lte(Time.now) 
  ).                                
  group(Post[:id])

Although the clean and expressive syntax of Ruby along with its metaprogramming capabilities makes it uniquely suited for building domain specific languages, DSLs exist in other languages as well. Here is an example of a JavaScript test using the Jasmine framework:

describe("Helper functions", function() {
  beforeEach(function() {
    this.helpers = window.helpers;
  });

  describe("log error", function() {
    it("logs error message to console", function() {
      spyOn(console, "log").and.returnValue(true);
      this.helpers.log_error("oops!");
      expect(console.log).toHaveBeenCalledWith("ERROR: oops!");
    });
  });
});

This syntax is perhaps not as clean as that of the Ruby examples, but it shows that with clever naming and creative use of the syntax, internal DSLs can be created using almost any language.

The benefit of internal DSLs is that they don’t require a separate parser, which can be notoriously difficult to implement properly. And because they use the syntax of the language they are implemented in, they also integrate seamlessly with the rest of the codebase.

What we have to give up in return is syntactic freedom—internal DSLs have to be syntactically valid in their implementation language. How much you have to compromise in this regard depends largely on the selected language, with verbose, statically typed languages such as Java and VB.NET being on one end of the spectrum, and dynamic languages with extensive metaprogramming capabilities such as Ruby on the other end.

Building Our Own—A Ruby DSL for Class Configuration

The example DSL we are going to build in Ruby is a reusable configuration engine for specifying the configuration attributes of a Ruby class using a very simple syntax. Adding configuration capabilities to a class is a very common requirement in the Ruby world, especially when it comes to configuring external gems and API clients. The usual solution is an interface like this:

MyApp.configure do |config|
  config.app_id = "my_app"
  config.title = "My App"
  config.cookie_name = "my_app_session"
end

Let’s implement this interface first—and then, using it as the starting point, we can improve it step by step by adding more features, cleaning up the syntax, and making our work reusable.

What do we need to make this interface work? The MyApp class should have a configure class method that takes a block and then executes that block by yielding to it, passing in a configuration object that has accessor methods for reading and writing the configuration values:

class MyApp
  class << self
    def config
      @config ||= Configuration.new
    end

    def configure
      yield config
    end
  end

  class Configuration
    attr_accessor :app_id, :title, :cookie_name
  end
end

Once the configuration block has run, we can easily access and modify the values:

MyApp.config
=> 

MyApp.config.title
=> "My App"

MyApp.config.app_id = "not_my_app"
=> "not_my_app"

So far, this implementation does not feel like a custom language enough to be considered a DSL. But let’s take things one step at a time. Next, we will decouple the configuration functionality from the MyApp class and make it generic enough to be usable in many different use cases.

Making It Reusable

Right now, if we wanted to add similar configuration capabilities to a different class, we would have to copy both the Configuration class and its related setup methods into that other class, as well as edit the attr_accessor list to change the accepted configuration attributes. To avoid having to do this, let’s move the configuration features into a separate module called Configurable. With that, our MyApp class will look like this:

class MyApp
  include Configurable
  
end

Everything related to configuration has been moved to the Configurable module:

module Configurable
  def self.included(host_class)
    host_class.extend ClassMethods
  end

  module ClassMethods
    def config
      @config ||= Configuration.new
    end

    def configure
      yield config
    end
  end

  class Configuration
    attr_accessor :app_id, :title, :cookie_name
  end
end

Not much has changed here, except for the new self.included method. We need this method because including a module only mixes in its instance methods, so our config and configure class methods will not be added to the host class by default. However, if we define a special method called included on a module, Ruby will call it whenever that module is included in a class. There we can manually extend the host class with the methods in

ClassMethods:

def self.included(host_class)     
  host_class.extend ClassMethods  
end

We are not done yet—our next step is to make it possible to specify the supported attributes in the host class that includes the Configurable module. A solution like this would look nice:

class MyApp
  include Configurable.with(:app_id, :title, :cookie_name)

end

Perhaps somewhat surprisingly, the code above is syntactically correct—include is not a keyword but simply a regular method that expects a Module object as its parameter. As long as we pass it an expression that returns a Module, it will happily include it. So, instead of including Configurable directly, we need a method with the name with on it that generates a new module that is customized with the specified attributes:

module Configurable
  def self.with(*attrs)
    
    config_class = Class.new do
      attr_accessor *attrs
    end

    class_methods = Module.new do
      define_method :config do
        @config ||= config_class.new
      end

      def configure
        yield config
      end
    end

    Module.new do
      singleton_class.send :define_method, :included do |host_class|
        host_class.extend class_methods
      end
    end
  end
end

There is a lot to unpack here. The entire Configurable module now consists of just a single with method, with everything happening within that method. First, we create a new anonymous class with Class.new to hold our attribute accessor methods. Because Class.new takes the class definition as a block and blocks have access to outside variables, we are able to pass the attrs variable to attr_accessor without problems.

def self.with(*attrs)            
   
  config_class = Class.new do    
    attr_accessor *attrs         
  end

The fact that blocks in Ruby have access to outside variables is also the reason why they are sometimes called closures, as they include, or “close over” the outside environment that they were defined in. Note that I used the phrase “defined in” and not “executed in”. That’s correct – regardless of when and where our define_method blocks will eventually be executed, they will always be able to access the variables config_class and class_methods, even after the with method has finished running and returned. The following example demonstrates this behavior:

def create_block 
  foo = "hello"            
   return Proc.new { foo }   
end

  block = create_block       

  block.call                 
 => "hello"

Now that we know about this neat behavior of blocks, we can go ahead and define an anonymous module in class_methods for the class methods that will be added to the host class when our generated module is included. Here we have to use define_method to define the config method, because we need access to the outside config_class variable from within the method. Defining the method using the def keyword would not give us that access because regular method definitions with def are not closures – however, define_method takes a block, so this will work:

config_class =  

 class_methods = Module.new do      
   define_method :config do          
    @config ||= config_class.new   
   end

Finally, we call Module.new to create the module that we are going to return. Here we need to define our self.included method, but unfortunately we cannot do that with the def keyword, as the method needs access to the outside class_methods variable. Therefore, we have to use define_method with a block again, but this time on the singleton class of the module, as we are defining a method on the module instance itself. Oh, and since define_method is a private method of the singleton class, we have to use send to invoke it instead of calling it directly:

class_methods =  
 
Module.new do 
  singleton_class.send :define_method, :included do |host_class| 
    host_class.extend class_methods   
  end 
end

Phew, that was some pretty hardcore metaprogramming already. But was the added complexity worth it? Take a look at how easy it is to use and decide for yourself:

class SomeClass
  include Configurable.with(:foo, :bar)
end

SomeClass.configure do |config|
  config.foo = "wat"
  config.bar = "huh"
end

SomeClass.config.foo
=> "wat"

But we can do even better. In the next step we will clean up the syntax of the configure block a little bit to make our module even more convenient to use.

Cleaning Up the Syntax

There is one last thing that is still bothering me with our current implementation—we have to repeat config on every single line in the configuration block. A proper DSL would know that everything within the configure block should be executed in the context of our configuration object and enable us to achieve the same thing with just this:

MyApp.configure do
  app_id "my_app"
  title "My App"
  cookie_name "my_app_session"
end

Let’s implement it, shall we? From the looks of it, we will need two things. First, we need a way to execute the block passed to configure in the context of the configuration object so that method calls within the block go to that object. Second, we have to change the accessor methods so that they write the value if an argument is provided to them and read it back when called without an argument. A possible implementation looks like this:

module Configurable
  def self.with(*attrs)
    not_provided = Object.new
  
    config_class = Class.new do
      attrs.each do |attr|
        define_method attr do |value = not_provided|
          if value === not_provided
            instance_variable_get("@#{attr}")
          else
            instance_variable_set("@#{attr}", value)
          end
        end
      end

      attr_writer *attrs
    end

    class_methods = Module.new do
      

      def configure(&block)
        config.instance_eval(&block)
      end
    end   
    
  end
end

The simpler change here is running the configure block in the context of the configuration object. Calling Ruby’s instance_eval method on an object lets you execute an arbitrary block of code as if it was running within that object, which means that when the configuration block calls the app_id method on the first line, that call will go to our configuration class instance.
The change to the attribute accessor methods in config_class is a bit more complicated. To understand it, we need to first understand what exactly attr_accessor was doing behind the scenes. Take the following attr_accessor call for example:

class SomeClass
  attr_accessor :foo, :bar
end

This is equivalent to defining a reader and writer method for each specified attribute:

class SomeClass
  def foo
    @foo
  end

  def foo=(value)
    @foo = value
  end
  
end

So when we wrote attr_accessor *attrs in the original code, Ruby defined the attribute reader and writer methods for us for every attribute in attrs—that is, we got the following standard accessor methods: app_id, app_id=, title, title= and so on. In our new version, we want to keep the standard writer methods so that assignments like this still work properly:

MyApp.config.app_id = "not_my_app"
=> "not_my_app"

We can keep auto-generating the writer methods by calling attr_writer *attrs. However, we can no longer use the standard reader methods, as they also have to be capable of writing the attribute to support this new syntax:

MyApp.configure do
  app_id "my_app" 
  app_id          
end

To generate the reader methods ourselves, we loop over the attrs array and define a method for each attribute that returns the current value of the matching instance variable if no new value is provided and writes the new value if it is specified:

not_provided = Object.new

attrs.each do |attr|
  define_method attr do |value = not_provided|
    if value === not_provided
      instance_variable_get("@#{attr}")
    else
      instance_variable_set("@#{attr}", value)
    end
  end
end

Here we use Ruby’s instance_variable_get method to read an instance variable with an arbitrary name, and instance_variable_set to assign a new value to it. Unfortunately the variable name must be prefixed with an “@” sign in both cases—hence the string interpolation.
You might be wondering why we have to use a blank object as the default value for “not provided” and why we can’t simply use nil for that purpose. The reason is simple—nil is a valid value that someone might want to set for a configuration attribute. If we tested for nil, we would not be able to tell these two scenarios apart:

MyApp.configure do
  app_id nil 
  app_id     
end

That blank object stored in not_provided is only ever going to be equal to itself, so this way we can be certain that nobody is going to pass it into our method and cause an unintended read instead of a write.

Adding Support for References

There is one more feature that we could add to make our module even more versatile—the ability to reference a configuration attribute from another one:

MyApp.configure do
  app_id "my_app"
  title "My App"
  cookie_name { "#{app_id}_session" }
End

MyApp.config.cookie_name
=> "my_app_session"

Here we added a reference from cookie_name to the app_id attribute. Note that the expression containing the reference is passed in as a block—this is necessary in order to support the delayed evaluation of the attribute value. The idea is to only evaluate the block later when the attribute is read and not when it is defined—otherwise funny things would happen if we defined the attributes in the “wrong” order:

SomeClass.configure do
  foo "#{bar}_baz"     
  bar "hello"
end

SomeClass.config.foo
=> "_baz"

If the expression is wrapped in a block, that will prevent it from being evaluated right away. Instead, we can save the block to be executed later when the attribute value is retrieved:

SomeClass.configure do
  foo { "#{bar}_baz" }  
  bar "hello"
end

SomeClass.config.foo    
=> "hello_baz"

We do not have to make big changes to the Configurable module to add support for delayed evaluation using blocks. In fact, we only have to change the attribute method definition:

define_method attr do |value = not_provided, &block|
  if value === not_provided && block.nil?
    result = instance_variable_get("@#{attr}")
    result.is_a?(Proc) ? instance_eval(&result) : result
  else
    instance_variable_set("@#{attr}", block || value)
  end
end

When setting an attribute, the block || value expression saves the block if one was passed in, or otherwise it saves the value. Then, when the attribute is later read, we check if it is a block and evaluate it using instance_eval if it is, or if it is not a block, we return it like we did before.
Supporting references comes with its own caveats and edge cases, of course. For example, you can probably figure out what happens if you read any of the attributes in this configuration:

SomeClass.configure do
  foo { bar }
  bar { foo }
end

The Finished Module

In the end, we have got ourselves a pretty neat module for making an arbitrary class configurable and then specifying those configuration values using a clean and simple DSL that also lets us reference one configuration attribute from another:

class MyApp
  include Configurable.with(:app_id, :title, :cookie_name)
end

SomeClass.configure do
  app_id "my_app"
  title "My App"
  cookie_name { "#{app_id}_session" }
end

Here is the final version of the module that implements our DSL—a total of 36 lines of code:

module Configurable
  def self.with(*attrs)
    not_provided = Object.new

    config_class = Class.new do
      attrs.each do |attr|
        define_method attr do |value = not_provided, &block|
          if value === not_provided && block.nil?
            result = instance_variable_get("@#{attr}")
            result.is_a?(Proc) ? instance_eval(&result) : result
          else
            instance_variable_set("@#{attr}", block || value)
          end
        end
      end

      attr_writer *attrs
    end

    class_methods = Module.new do
      define_method :config do
        @config ||= config_class.new
      end

      def configure(&block)
        config.instance_eval(&block)
      end
    end

    Module.new do
      singleton_class.send :define_method, :included do |host_class|
        host_class.extend class_methods
      end
    end
  end
end

Looking at all this Ruby magic in a piece of code that is nearly unreadable and therefore very hard to maintain, you might wonder if all this effort was worth it just to make our domain specific language a little bit nicer. The short answer is that it depends—which brings us to the final topic of this article.

Ruby DSLs—When to Use and When Not to Use Them

You have probably noticed while reading the implementation steps of our DSL that, as we made the external facing syntax of the language cleaner and easier to use, we had to use an ever increasing number of meta-programming tricks under the hood to make it happen. This resulted in an implementation that will be incredibly hard to understand and modify in the future. Like so many other things in software development, this is also a tradeoff that must be carefully examined.

For a domain specific language to be worth its implementation and maintenance cost, it must bring an even greater sum of benefits to the table. This is usually achieved by making the language reusable in as many different scenarios as possible, thereby amortizing the total cost between many different use cases. Frameworks and libraries are more likely to contain their own DSLs exactly because they are used by lots of developers, each of whom can enjoy the productivity benefits of those embedded languages.

So, as a general principle, only build DSLs if you, other developers, or the end users of your application will be getting a lot of use out of them. If you do create a DSL, make sure to include a comprehensive test suite with it, as well as properly document its syntax as it can be very hard to figure out from the implementation alone. Future you and your fellow developers will thank you for it.

Visit & subscribe toptal.com for such insightful articles. It's a #1 blog for Engineers.

About the Author of this post :

Máté is a full-stack software engineer with 7+ years of experience working on web applications of all sizes, from small greenfield projects to complex legacy systems. He is extremely organized, communicates very well, and prides himself in writing clean, future-proof code with a low tolerance for technical debt. Máté's formal education is in accounting, banking, and finance; therefore, he has extensive domain knowledge in these fields as well.

Logical operator Vs Bitwise operator in Java : && vs &, || vs |

2017-06-30T06:44:00.000-07:00

&& operator is a Logical operator in Java. In Java it's alias names are short-circuit AND operator, and conditional AND operator.

AND Logic Gate

It operates on Boolean variables only (true, false).

In an AND operation involving multiple conditions, if one of the condition is false the rest is not checked, as the result will always be false - This short circuit behavior of the Logical operator is the reason why it is also known as short-circuit operator.

It's called Bitwise operator in Java, as it performs Logical AND operation on bit level.

it is a.k.a Bitwise AND, Unconditional Operator in Java.

It performs AND logical operation on each bit.

Let's observe some important points about && and &.

Compilation error because Operator && is defined only for Boolean values

Note #1: && is defined only for boolean values. It's operands can only be true or false. Else, it produce compilation error.

Compilation error because Operator & is defined only for Integer and Boolean values

Note #2: & is defined only for boolean and Integer (i.e., byte, short, char, int and long) values. Else it produce compilation error.

  

package LogicalVsBitwise;



public class LogicalVsBitwise {

public static void main(String[] args) {

// Logical && Vs Bitwise &

int a = 2;

int b = 9;

System.out.println((a > b) && (++a < b)); //false && -skipped- = false

System.out.println(a); //-skipped- so a did't get incremented. So, a=2
System.out.println((a > b) & (++a < b)); //false & true = false
System.out.println(a); // didn't skip so ++a = 3
}
}

Output:-

false

Note #3: && operator skips rest of the conditions if it finds any false condition. Because, the result will be false.

Meanwhile, & operator doesn't skip any condition even if it finds one false condition during the calculation.



package ObesrveIt 

public class ObesrveIt  {

    public static void main(String[] args) {

        // & operator works only on boolean or int operands

        // Default size of int is 32 bit

        int a = 2;          // 00000000 00000000 00000000 00000010

        int b = 9;          // 00000000 00000000 00000000 00001001

        int result = a & b; // 00000000 00000000 00000000 00000000

        System.out.println(result); 

    } 

}

Output: 0

Note #4 : AND operation is performed on every bit. Hence, & operator is also known as bit-wise operator in Java.

here

Difference between && and & operator

Note #5:

If a String variable stringName is null, executing stringName.length() command throws a java.lang.NullPointerException.
&& ignores rest of the condition as soon as it finds a false condition from LEFT-> RIGHT.
& checks for every condition even if it finds a false condition. So, here name.length() command got executed and java.lang.NullPointerException got thrown.

Difference between || and |

OR Logic Gate

Similar to Logical && operator, Logical || operator is known as:

Logical Operator
Conditional Operator and
Logical OR Operator

If a Logical OR operation is performed on multiple conditions, || stops executing rest of the conditions as soon as it finds a true condition. Because, the result will be true anyways. - (Opposite to && operator)

Just like && operator, || operator is defined for Boolean values only.

Similar to Bit-wise & operator, | operator is known as:

Bit-wise Operator
Unconditional Operator and
Bit-wise OR Operator

If a Bit-wise OR operation is performed on multiple conditions, unlike || operator, | operator executes every condition even if it finds a true condition in between.

Just like & operator, | operator is defined for boolean and Integer values only. And it performs a Logical OR operation on every bit.

Conclusion:

(Note: There are many other Logical as well as bit operators in Java. But, here we discussed only about &&, &, || and | operator. To read further, check the notes on Bit-Shift operators in Java.)

LOGICAL && and || OPERATOR

Both && and || operator stops executing remaining expression as soon as it finds the main result. i.e.,&& stops execution as soon as it finds a false condition. Whereas, || stops the operation as soon as it find a true condition - This short-circuit like nature gives it the name Short-circuit AND and OR operand.
Both are defined only for Boolean operands.
Both gives out Boolean output only- i.e., true, or false.
It's alias name includes:- Conditional operator, Logical operator, And Short-circuit operator.

BIT-WISE & and | OPERATOR

Unlike Logical operators, Both '&' and '|' operator executes every expression.
These are defined only for boolean and Integer (i.e., byte, short, char, int and long) operands.
& operator performs the AND operation and | operator performs the OR operation on every bit of the input boolean or Integer data types - Hence the name Bit-wise operator.
It's alias name includes:- Unconditional operator, and Bitwise operator.

Bit shift operators in Java

2017-06-09T10:14:00.000-07:00

Above and beyond bitwise operators, JLS provides 3 bit shift operators for bit manipulation in Java.
Namely:

Left Shift Operator (<<)
Signed Right Shift Operator (>>)
Unsigned Right Shift Operator (>>>)

Syntax: Left-hand operand SHIFT OPERATOR Right-hand operand
Here,
Left-hand Operand is the value to be shifted, i.e., shifting of it's binary value.
Right-hand Operand specifies the shift distance.

i.e., Value to be shifted SHIFT OPERATOR the shift distance.

1. Left Shift Operator "<<":

It shifts the binary value of the left operand with sign to left side and fills the empty place with 0.

Let's solve some question to understand the concept fo Left shift operator:

Q.1 Solve 20 << 2 shift 20 by 2 bits in the left.

Sol. The binary representation of 20 is: 00010100

Fig. 1.1 - Logic behind left shift operation 20 << 2

Therefore, 20 << 2 = 80. So, Answer = 80.

Note: If you closely observe:
System.out.println(10<<2); // output = = 40
System.out.println(10<<3); // output = = 80
System.out.println(10<<4); // output = = 160

Left Shift Operator also act's as . But, remember the default size of the data type.

Q. 2. what's the result of -20 << 2 ?

Sol. Negative integers are stored as it two's complement in Java.

Revision: Two's complement
Two's complement of a negative integer can be found by Inverting and adding '1' to the binary representation of the integer with -ve sign.

i.e., to reverse the sign you simply invert the bits (0 1, and 1 0) and add '1' to the resulting number.


Fig. 1.2 - Basic rules of Binary addition

Even subtraction (A - B) is considered as (A + (-B)) i.e., addition of a binary value 'A' with the two's complement of another binary value 'B'.

Fig. 1.3 - Basic Addition in Binary

Fig. 1.4 - Basic Subtraction in Binary

Suppose, we're working with 8-bit binary values (for simple calculation) and we want to find how -20 would be expressed in two's complement notation.

Step 1: Write the integer value (here 20) in it's binary form:

Fig. 1.5 : Representation of 20 in it's binary form i.e., =

i.e., =

Step 2: Then we invert the digits. 0 becomes 1, and 1 becomes 0:

1 1 1 0 1 0 1 1

Step 3: Then we add '1':

So, we represent as . So, Answer = 11101100.

That is how one would write -20 in 8-bit binary.

Note: MSB tells us whether the binary representation is of a -ve or +ve Integer.

if MSB = 0 it represents a +ve integer, and
if MSB = 1 it represents a -ve integer.

Conversion from Two's Complement

Let's understand this with the help of above example. Let's convert the two's complement back to it's original form.

Step 1: Here, MSB = 1. So, we can conclude that it is a binary representation of a -ve integer.

Rule #1: to reverse the sign you simply invert the bits (0 goes to 1, and 1 to 0) and add one to the resulting number.

Step 2: Invert the digits. 0 becomes 1, and 1 becomes 0:

0 0 0 1 0 0 1 1

Step 3: Add '1':

So, from results found by step 1 and step 3. We found that = Answer = -20.

So, let's convert -20 to it's binary equivalent:

Now, let's shift it to left with shift distance = 2, as mentioned in Q.2.

Since, the given integer was represented in 8 bits, we will only consider 8 bits for now. So, we got 1 0 1 1 0 0 0 0

since MSB = 1. It implies, this binary value is a representation of a -ve integer.

After inverting the digits 0 1 0 0 1 1 1 1

After adding 1 0 1 0 1 0 0 0 0

128*0+ 64*1 + 32*0 + 16*1 + 8*0 + 4*0 + 2*0 + 1*0 = 64 + 16 = 80

So, since we found that it's a -ve integer => Answer = -80.

Q. 3. what's the result of 20 << -2 ?

Sol. Here the shift distance is "-2". So, this expression is considered undefined in C. But, in Java using Netbeans, we got an output for this expression. And Let's find it out how:

2 is represented in binary form as => 00000010

Inverting it => 11111101

After adding '1' => 11111110

So, -2 is represented as 11111110 in binary form.

Note: According to JSL:

If the promoted type of the left-hand operand is int, only the five lowest-order bits of the right-hand operand are used as the shift distance. It is as if the right-hand operand were subjected to a bitwise logical AND operator with the mask value 0x1f (0b11111). The shift distance actually used is therefore always in the range 0 to 31, inclusive.
If the promoted type of the left-hand operand is long, then only the six lowest-order bits of the right-hand operand are used as the shift distance. It is as if the right-hand operand were subjected to a bitwise logical AND operator with the mask value 0x3f (0b111111). The shift distance actually used is therefore always in the range 0 to 63, inclusive.

So, the last 5 bits of 11111110 is 11110

After bitwise AND operation we get 11110. In integer form it's represented as 30.

Shift distance = 30.
If we will shift , 30 times to left. We will get 32 0's. As 20 is a positive integer. And the size of int in java is 32 bit. Answer = 0.

Q. 4. what's the result of `-20 << -2` ?

Sol. Two's complement of -20 =.
Two's complement of -2 = .
After subjecting the last 5 bits of 11111110 to bitwise AND operation with the mask 0b11111 we get 11110 it is the binary representation of 30.
shift distance = 30.
If we will shift , 30 times to left. We will get 32 0's. As -20 is an integer. And the size of int in java is 32 bit. Answer = 0.

TEST CODE FOR ANALYSING LEFT SHIFT OPERATOR

public class ShiftOpptn {

    public static void main(String[] args) {
         /* Left shift operator */
        System.out.println(20 << 2);   // output = 80
        System.out.println(Integer.toBinaryString(20 << 2));   // 1010000
        System.out.println(-20 << 2); // -80
        System.out.println(Integer.toBinaryString(-20 << 2));
        // output =11111111111111111111111110110000
        System.out.println(20 << -2); // 0
        System.out.println(Integer.toBinaryString(20 << -2));   // 0
        System.out.println(-20 << -2); // 0
        System.out.println(Integer.toBinaryString(-20 << -2)); // 0
    }
}

Output:

Fig. 1.5 - Output produced for above code in Netbeans IDE

2. Signed Right shift operator ">>"

It shifts the binary value of the left operand with sign to RIGHT side and fills the empty place with the value of SIGN BIT.

For +ve (positive) number it fills with 0. And,
for -ve (negative) number it fills with 1.

Example: 20 >> 2

Fig. 2.1 - Logic behind signed right shift operation 20 >> 2

Therefore, 20 >> 2 = 5 . And it is a +ve integer. So, rest of the empty space will be filled with the sign value i.e., 0.

5 will be represented as 00000000 00000000 00000000 00000101 in binary.

Q.1. What's the result of -20 >> 2 ?

Sol. Two's complement of -20 = .
Since, the LHS operand is a -ve integer. The blank space will be filled by the -ve sign value '1'.

So, after shifting 2 bits right 11111011 two's complement of -5.

-20 >> 2 = -5, which is represented as 11111111 11111111 11111111 11111011 in binary.

Q.2. What's the result of 20 >> -2 ?

Sol. Binary form of 20 =
Two's complement of -2 = .
Since, the LHS operand is a +ve integer. The blank space will be filled by the +ve sign value i.e., '0'.

So, by JLS's specified rule for shift operator. least 5 digits of the RHS operand is passed through bitwise AND operation with the MASK value 0b11111.

So, 11110 AND 0b11111 = 11110 , which is the binary representation of 30.

After shifting the binary of LHS operand to 30 bits right 00000000 0.

20 >> -2 = 0, which is represented as 00000000 00000000 00000000 00000000 in binary.

Q.3. What's the result of -20 >> -2 ?

Sol. Two's complement of -20 =
Two's complement of -2 = .
Since, the LHS operand is a -ve integer. The blank space will be filled by the -ve sign value i.e., '1'.

So, by JLS's specified rule for shift operator. least 5 digits of the RHS operand is passed through bitwise AND operation with the MASK value 0b11111.

So, 11110 AND 0b11111 = 11110 , which is the binary representation of 30.

After shifting the binary of LHS operand to 30 bits right 11111111 Two's complement of -1.

-20 >> -2 = -1, which is represented as 11111111 11111111 11111111 11111111 in binary.

Note: If you closely observe:
System.out.println(10<<2); // output = => 2
System.out.println(10<<3); // output = => 1
System.out.println(10<<4); // output = => 0

Right Shift Operator also act's as . But, remember the default size of the data type.

TEST CODE FOR ANALYSING RIGHT SHIFT OPERATOR

public class ShiftOpptn {

    public static void main(String[] args) {
         /* Signed Right shift operator */
        System.out.println("1. 20 in binary = "+Integer.toBinaryString(20));
        System.out.println("2. 20 >> 2 = "+ (20 >> 2));
        System.out.println("Binary = "+ Integer.toBinaryString(20 >> 2));
        System.out.println("3. -20 >> 2 = "+ (-20 >> 2));
        System.out.println("Binary = "+ Integer.toBinaryString(-20 >> 2));
        System.out.println("4. 20 >> -2 = "+ (20 >> -2));
        System.out.println("Binary = "+ Integer.toBinaryString(20 >> -2));
        System.out.println("5. -20 >> -2 = "+ (-20 >> -2));
        System.out.println("Binary = "+ Integer.toBinaryString(-20 >> -2));
    }
}

Output:

Fig 2.2 Output produced for above code in Netbeans IDE

3. Unsigned right shift operator ">>>"

It shifts the binary value of the left operand without preserving SIGN to RIGHT side and fills the empty place with 0.

So, It is also known as unsigned right shift operator, and right shift with zero fill.

Q.1. What's the result of 20 >> 2 ?

Sol. Two's complement of 20 = .
>>> is an unsigned right shift operator. So, the blank space will be filled by '0'.

after shifting 2 bits right (00000000 00000000 00000000 00000101)₂ = (5)_10.

20 >> 2 = 5, which is represented as 00000000 00000000 00000000 00000101 in binary.

Q.2. What's the result of -20 >> 2 ?

Sol. Two's complement of -20 = .
>>> is an unsigned right shift operator. So, the blank space will be filled by '0'.

after shifting 2 bits right (00111111111111111111111111111011)₂ = (1073741819)_10.

-20 >> 2 = 1073741819, which is represented as 00111111 11111111 11111111 11111011 in binary.

Q.3. What's the result of 20 >> -2 ?

Sol. Binary form of 20 =
Two's complement of -2 = .
>>> is an unsigned right shift operator. So, the blank space will be filled by '0'.

So, by JLS's specified rule for shift operator. least 5 digits of the RHS operand is passed through bitwise AND operation with the MASK value 0b11111.

So, 11110 AND 0b11111 = 11110 , which is the binary representation of 30.

After shifting the binary of LHS operand to 30 bits right 00000000 0.

20 >> -2 = 0, which is represented as 00000000 00000000 00000000 00000000 in binary.

Q.4. What's the result of -20 >> -2 ?

Sol. Two's complement of -20 =
Two's complement of -2 = .
>>> is an unsigned right shift operator. So, the blank space will be filled by '0'.

So, by JLS's specified rule for shift operator. least 5 digits of the RHS operand is passed through bitwise AND operation with the MASK value 0b11111.

So, 11110 AND 0b11111 = 11110 , which is the binary representation of 30.

After shifting the binary of LHS operand to 30 bits right (00000000000000000000000000000011)₂ = (3)₁₀

-20 >> -2 = 3, which is represented as 00000000 00000000 00000000 00000011 in binary.

TEST CODE FOR ANALYSING UNSIGNED RIGHT SHIFT OPERATOR

public class ShiftOpptn {

    public static void main(String[] args) {
         /* Unsigned Right shift operator ">>>" */
        System.out.println("1. 20 in binary = "+Integer.toBinaryString(20));
        System.out.println("2. 20 >>> 2 = "+ (20 >>> 2));
        System.out.println("Binary = "+ Integer.toBinaryString(20 >>> 2));
        System.out.println("3. -20 >>> 2 = "+ (-20 >>> 2));
        System.out.println("Binary = "+ Integer.toBinaryString(-20 >>> 2));
        System.out.println("4. 20 >>> -2 = "+ (20 >>> -2));
        System.out.println("Binary = "+ Integer.toBinaryString(20 >>> -2));
        System.out.println("5. -20 >>> -2 = "+ (-20 >>> -2));
        System.out.println("Binary = "+ Integer.toBinaryString(-20 >>> -2));
    }
}

Output:

Fig 3.1 The output produced by above code in Netbeans IDE

More on bit-shift operators

The shift operators are left-associative.

Did you know?ln programming language, associativity^[1] (a.k.a fixity) of an operatorproperty that determines how operators of the same precedence are grouped in the absence of parentheses.

If there are two operators with same precedence before and after an operand then the choice of which operations to apply the operand to, is determined by the "associativity" of the operators.

Operators may be associative (meaning the operations can be grouped arbitrarily), left-associative (meaning the operations are grouped from the left), right-associative (meaning the operations are grouped from the right) or non-associative (meaning operations cannot be chained, often because the output type is incompatible with the input types).

different programming languages may have different associativity and precedence for the same type of operator.

Consider the expression a ~ b ~ c. If the operator ~ has left associativity, this expression would be interpreted as (a ~ b) ~ c. If the operator has right associativity, the expression would be interpreted as a ~ (b ~ c). If the operator is non-associative, the expression might be a syntax error, or it might have some special meaning.

Some mathematical operators have inherent associativity. For example, subtraction and division, as used in conventional math notation, are inherently left-associative. Addition and multiplication, by contrast, are both left and right associative. (e.g. (a * b) * c = a * (b * c)).

The concept of notational associativity described here is related to, but different from the mathematical associativity. An operation that is mathematically associative, by definition requires no notational associativity. (For example, addition has the associative property, therefore it does not have to be either left associative or right associative.) An operation that is not mathematically associative, however, must be notationally left-, right-, or non-associative. (For example, subtraction does not have the associative property, therefore it must have notational associativity)[...]

After Unary numeric promotion, if the type of each of the operands of a shift operator is not a primitive integral type, then that will cause a compile time error.

FYI : Variables and Data types in Java
Variable:- is any name used to refer a memory location.
Data type:- represents the values that can be stored in the variable.

There are 3 types of variable in Java:
1. Local ( variable declared inside the method ).
2. Instance ( variable declared inside the class but outside the method ).
3. Static ( Non-local variable that is declared as static ).

There are 2 types of compile-time data types in Java:
1. Primitive, and
2. Non-primitive.

Fig.3.1 - Data types in Java

Therefore, by primitive integral type we mean:-
(a.) Integer : byte, short, int, long. And
(b.) Floating-point : float, double.

In Java,
Data Type Default size    Default value
boolean       1 bit                 false
byte       1 byte               0
char    2 byte              '\u0000'
short       2 byte                0
int               4 byte                0
long             8 byte              0L
float             4 byte                0.0f
double         8 byte                 0.0d

Java follows UNICODE system instead of ASCII code system. Well,
Lowest value: '\u0000'
Highest value: '\uFFFF'.
So, char in Java takes 2 byte.

UNICODE = Universal Interntionl Standard Character Encoding.

Different country were following different code system. So, to make it common for all, UNICODE System was developed.

FYI : Numeric promotions
Numeric promotions: - are used to convert the operands of a numeric operator to a common type so that the operation can be performed.

There are two kinds of numeric promotion:
1. Unary numeric promotion. And,
2. Binary numeric promotion.

1. Unary numeric promotion: - During an operation, some operators apply unary numeric promotion on the single operand which is to produce a numeric type value.

The operand of compile-time type byte, short, char, or int is subjected to unboxing conversion, and it is then promoted to a value of type int by a widening primitive conversion or an identity conversion.
Else, if the operand is of compile-time type long, float, or double, it is subjected to unboxing conversion.
Else, it remains as it is, without converting.

Unary numeric promotion is performed on expressions in the following situations:

Each dimension expression in an array creation expression. Ex: inte ar[] = new int[2];
The index expression in an array access expression. Ex: ar['\u0001'] = 1;
The operand of a unary +, -, ~ complimentary, or <<, >>, >>> shift operators. Ex: ar[0] = -'\u0001';

Note: A long shift distance (RHS operand of the SHIFT OPERATOR <<, >> or >>>) doesn't promote the value being shifted ( LHS operand of the shift operator) to long.

Example:

 byte a = 17;   // 1. assigning byte value

byte b = +a; // 2. Compiler error

2nd statement creates compile error because of the following rule mentioned by JLS: "if the operand is of compile-time type byte, short or char unary numeric promotion promotes it to a value of type int by a widening conversion."

Fig. 3.2 - Example to illustrate compiler error produced during unary numeric promotion

To remove the error, we can explicitely caste it to byte byte c = (byte) +a;

2. Binary numeric promotion: When an operator applies binary numeric promotion to a pair of operands, the following rules apply (in order):

Reference type operand is subjected to unboxing conversion.
Widening primitive conversion is applied to convert either or both operands as specified by the following rules:
- If either operand is of type double, the other is converted to double.
- Otherwise, if either operand is of type float, the other is converted to float.
- Otherwise, if either operand is of type long, the other is converted to long.
- Otherwise, both operands are converted to type int.

It implies, during this operation the priority of data type in descending order is double, float, long, int.

After the conversion(s), if any, value set conversion is then applied to each operand.

Binary numeric promotion is performed on the operands of certain operators:

*, /, and % (multiplicative operator)
+ and - (additive operator)
<, <=, >, and >= (comparison operator)
== and != (equality operator)
&, ^, and | (integer bitwise operator)
? : (conditional operator)

Fig. 3.3 - Example to illustrate the compile error caused on breaking the rules of binary numeric promotion

Capsule Notes

The shift operators include left shift <<, signed right shift >>, and unsigned right shift >>>.

The value of n<<s is n left-shifted s bit positions with zero-extension.
The value of n>>s is n right-shifted s bit positions with sign-extension.
The value of n>>>s is n right-shifted s bit positions with zero-extension.

Negative integers are stores as it's two's complement in Java.

Note: According to JSL:

If the promoted type of the left-hand operand is int, only the five lowest-order bits of the right-hand operand are used as the shift distance after getting subjected to a bitwise logical AND operator with the mask value 0x1f (0b11111). The shift distance actually used is therefore always in the range 0 to 31, inclusive.

If the promoted type of the left-hand operand is long, then only the six lowest-order bits of the right-hand operand are used as the shift distance. It is as if the right-hand operand were subjected to a bitwise logical AND operator with the mask value 0x3f (0b111111). The shift distance actually used is therefore always in the range 0 to 63, inclusive.

The shift operators are left-associative.

After Unary numeric promotion, if the type of each of the operands of a shift operator is not a primitive integral type, then that will cause a compile time error.

Interview Questions:

Q. 1. Why is there no <<< operator in Java 8?
Ans. To avoid redundancy. In java the sign values are stored in MSB. In left shift operation, the right side bit gets empty. So, in left shift operation there is no point in talking about signed and unsigned variable. The available left shift operator does the work.

FYI, there is no unsigned right shift operator ">>>" in C, but it's there is Java. So, we can say that it's a java specific operator.

p.s. this post content is concisely based on the data provided by the JLS ( Java Language Specification) on Java 8 release.

* ( Most of the questions have already been answered in the explanation. Yet if you find any question left unanswere. Please comment it below... )

Bibliography:

^[1]This article uses material from the Wikipedia article Operator associativity, which is released under the Creative Commons Attribution-Share-Alike License 3.0 (view authors).

CS.Techruto

Creating a Ruby DSL: A Metaprogramming Tutorial

What Is a Domain Specific Language?

Building Our Own—A Ruby DSL for Class Configuration

Making It Reusable

Cleaning Up the Syntax

Adding Support for References

The Finished Module

Ruby DSLs—When to Use and When Not to Use Them

Logical operator Vs Bitwise operator in Java : && vs &, || vs |

Difference between || and |

Conclusion:

Bit shift operators in Java

Q. 4. what's the result of -20 << -2 ?

Q. 4. what's the result of `-20 << -2` ?