Tech Spot

Consumerization of IT – what it means for the Architect

noreply@blogger.com (Munish K Gupta) — Tue, 24 Jan 2012 07:30:00 +0000

Consumerization is described as the trend for IT to first emerge in the Consumer space and subsequently makes its way into the enterprise world. But what exactly in the consumer world, that is making the users, demand the similar things from the enterprise IT. To understand the underlying need, we need to first examine the basic requirements of the user.

Kathy Sierra, co-creator of the Head First series of books and founder of javaranch.com, describes the hierarchy of needs from the user(s) perspective. The needs are stacked in the order of increasing engagement from the user. Starting with the basic needs of a defined functionality and its correctness, moving on to the learnability, efficiency & usability and finally culminating in intuitiveness and enchantment. Merely provision of correctly working functionality is not guarantee of the success of the application(s). The idea is to hook the user; the application needs to do something extra.

Using the hierarchy of needs as the background, we will see how the applications in the consumer and enterprise world stack up.
Now, in the consumer world, the advent of Consumerization started with the proliferation of the mobile devices. In the connected world, the device (smartphone or tablet) imposed certain constraints

Screen size is small and limited
Processing power of the device is limited
Interface is touch enabled
Connectivity is not constant (read patchy) and speed/data is limited (no unlimited data plans)
User audience demography not known

Now, the Apps needed to be designed keeping in mind the constraints, which meant that the Apps are

Focused on providing only one functionality
Have simple and intuitive messages/steps in the absence of any help/guides
Providing simple interfaces (cater to worldwide audience)
Having uncluttered screens as real estate is limited
Using UI Controls that are big and usable (touch interface)
Having intuitive workflows
Engaging to stand apart among the millions of other Apps

This meant that the functionality does not require the user to be a geek. The average individual user can understand and use the App functionality without any external help. Majority of the Consumer Apps are heavily focused on the Efficiency and Usability part. Few of the Apps have been able to enchant and engage the user (e.g. Angry Birds).

In the enterprise world, the device is most likely to be powerful desktop/laptop with a large screen. The connectivity is via LAN or broadband. As a result the Enterprise Applications are more industrial with high focus on providing long list of functionality and ensuring the correctness of the same. Consumers facing business applications usually focused on usability, Specialized enterprise applications (e.g. Call Center Applications) focus on the overall workflow efficiency, but majority of the applications are resource intensive, lack efficiency and have high learnability curves.

The enterprise user experiences the simple, intuitive consumer apps on their mobile devices. With the advent of BYOD (Bring Your Own Device), enterprise user’s started bringing their smart devices into the enterprise. Soon the users are comparing the experience of consumer apps with the enterprise business apps; they wonder why the enterprise apps can’t provide a similar experience. Why are the enterprise apps so low on learnability, efficiency, intuitiveness factors?

The whole idea behind the Consumerization of IT is not just measures like BYOD but to bring the missing intuitiveness, usability and efficiency into the enterprise applications.

So, what does Consumerization means for the Architect?

When designing and architecting enterprise business applications, following considerations need to be kept in mind

Connected users means the applications need to be available 24 X 7 ( Always On, Always Available), 24 X 7 availability requires application solution(s) to be elastic – expand or shrink based on the load
Intuitiveness and Usability have to be high on the agenda when designing the interface and workflows
Solution needs offer APIs for additional applications to be build
Integration with systems and other applications (including SaaS) need to be simple, straight forward and well documented
User experience is the key to successful business application (it was always the key) but need to be (re)designed for the connected & mobile devices
Product evaluation(s) will now include SaaS applications that can provide the functionality
Saas adoption will increase leading to newer challenges in integration, data security and privacy measures
Applications need to be designed/tested keeping in mind various devices/OS combinations (days of designing apps that worked on IE6 only are over)

The impact of consumerization on the Enterprise IT will be felt in the years to comes, BYOD is just the harbinger of things to come.

This post originally appeared at www.techspot.co.in

Bigdata Trends in 2012

noreply@blogger.com (Munish K Gupta) — Tue, 17 Jan 2012 07:30:00 +0000

Bigdata has caught the fancy of each and everyone. Consumer and enterprise world is looking to adopt the Bigdata. Analysts are talking about it, proclaiming it to the next big thing!

This is my take on how the Bigdata market space will evolve in 2012 and years to come.

Data Analytics Service – We will start seeing SaaS vendors that will start offering data analytics as a service (e.g. 1010data). These vendors will be able to pull in your unstructured data and provide you standardized reports and data services. This is very similar to the Web analytics (like Omniture) but access lot more data to provide patterns. 70-80% of the data analysis pattern will be similar across industries. These vendors will be able to provide these reports as off the shelf capabilities. For the other 20%, the web interface will allow the user to create patterns to analyze the data

Data Visualization Service – Analysis is only one part of the data, the other big thing will be able to visualize the data (more like infographics) that help make sense of the scale and data points. Business will need help to make sense of all this analysis and again 3rd party vendors (via products or Saas) will be fulfilling this space

Crowdsourcing Model – enterprise will start using the crowdsourcing model to help analyze and find patterns in the data. Goldcorp was one the original pioneers that used the crowdsourcing model to analyze the geological data. This model will be seen more when dealing large geospatial data in the areas of mining, drilling, oil ad gas

RDBMS vendors will make a comeback – Bigdata has been scoring high on account of their ability to analyze and process unstructured data. The NoSQL vendors and products are all claiming to be the holy grail of data storage, fast processing and analytics. I believe, the Big Vendors –Oracle, IBM and to some extent Microsoft will release their version of relational databases that can store (using automated data sharding) and process large amount of data (with a SQL MapReduce layer). The vendors will incorporate techniques from the NoSQL world coupled with in-memory storage (think Coherence, eXtreme Scale) to provide sub second read write response times. The market is too big for the existing vendors to not claim supremacy

NoSQL market will shake up – The current crop of NoSQL solutions has emerged as need to fulfill certain use cases. Their base features and functionality have been decided by the patronizing user client/project co-sponsors (e.g. Cassandra – Facebook, Hadoop-Yahoo, membase-Zynga). These NoSQL solutions work very well for the specific use case. Attempts to broad base the use case, mean a lot of effort needs to put in. Along the way, some products will merge, some will lose community support and other will get acquired by product vendor companies

Data Analysis Libraries - To help the developers analyze data, the common patterns and designs will be available as off the shelf libraries. This means, analyzing data for most common patterns will be easier and will become no-brainer. The projected need for Data scientists will not materialize to the same extent with the proliferation of these libraries. E.g. MS Cloud Numerics

Bigdata will follow the Consumerization model, where the core infrastructure either comes as a service or an appliance. The data analytics, data visualization will be provided as a standard set of services on top of the raw data with an ability to create your own model.

Do let me know, how do you see the Bigdata market moving forward and do you agree or disagree with my predictions?

Application Performance and Antipatterns

noreply@blogger.com (Munish K Gupta) — Wed, 04 Jan 2012 13:35:00 +0000

Any application you pick up, there are some issues – big or small. There will be copy-paste code, mistakes, algorithms which could have better thought through. But what distinguishes an antipattern from these normal errors is that like patterns these antipatterns are recurring throughout the code base. In my recent experience in dealing with performance issues, I had observed certain recurrent themes that are undermining the overall application performance. Most of these antipatterns are well documented but it seems we do not learn from others mistakes. We need to make our own mistakes. I am recounting some of the common patterns that I observed in the recent months.

Excessive Layering - Most of the underlying performance starts with the excessive layering antipattern. The application design has grown over the usage of controllers, commands and facades. In order to decouple each layer, the designers are adding facades at each of the tiers. Now, for every request at the web tier, the request call goes through multiple layers just to fetch the results. Imagine doing this for thousands of requests coming in and the load the JVM need to handle to process these requests. The number of objects that get created and destroyed when making these calls add to the memory overhead. This further limits the amount of requests that can be handled by each server node.

Based on the size of the application, deployment model, the number of user’s, appropriate decision need to be taken to reduce the number of layers. E.g. if the entire application gets deployed in the same container, there is no need to create multiple layers of process beans, service beans(business beans), data access objects etc. Similarly, when developing an internet scale application, large number of layers start adding overheads to the request processing.

Remember, large number of layers means large number of classes which effectively start impacting the overall application maintainability.
Round Tripping- With the advent of ORM mappings, Session/DAO objects, the programmer starts making calls to beans for every data. This leading to excessive calls between the layers. Another side issue is the number of method calls each layer start having to support this model. Worse case is, when the beans are web service based. Client tier making multiple web service calls within a single user request have a direct impact on the application performance.

To reduce the round tripping, the application needs to handle or combine multiple requests at the business tier.
Overstuffed Session- Session object is a feature provided by the JEE container to track user session during the web site visit. The application start with the promise of putting very minimal information in the session but over a period of time, the session object keeps on growing. Too much of data or wrong kind of data is stuffed into the session object. Large data objects will mean that the objects placed in the session will linger on till the session object is destroyed. This impacts the number of user’s that can be served by the application server node. Further, I have seen, application using session clustering to support availability requirements but adding significant overheads to the network traffic and ability of application to handle higher number of users.

To unstuff the session object, take an inventory of what all goes there, see what is necessary, what objects can be defaulted to request scope. For others, remove the objects from session when their usage is over.
Golden Hammer (Everything is a Service) – With the advent of SOA, there is tendency to expose the business services, which can be orchestrated into process services. In the older applications, one can observe similar pattern being implemented with EJBs. This pattern coupled with the bottom up design approach at times, means exposing each and every data entity as a business service. This kind of design might be working correctly functionally, but from the performance and maintenance point of view, it soon becomes a night mare. Every web service call adds overhead in terms of data serialization and deserialization. At times, the data(XML) being passed with web service calls is also huge leading to performance issues.

The usage of services or ejb’s should to be evaluated from application usage perspective. Attention needs to be paid on the contract design.
Chatty Services – Another pattern observed is the way the service is implemented via multiple web service calls each of which is communicating a small piece of data. This results in explosion of web services and which leads to degradation of performance and unmaintainable code. Also, from the deployment perspective, the application starts running into problems. I have come across projects which have hundred plus services all getting crammed into a single deployment unit. When the application comes up, the base heap requirement is already in 2Gb range leaving not much space for application to run.

If the application is having too many fine grained services, then it an indication towards the application of this antipattern.

The above mentioned antipatterns are frequent causes of application performance issues. The teams usually start with the right intentions but over a period of time, things will start slipping. Some of the common reasons

Lack of design standards and reviews processes – even if these exists, the delivery pressure is leading to skipping these processes
Team members inexperience or narrow view leads to every programmer only looking at their module and nobody is looking at the overall application performance
Continuous Integration(CI) tools not integrated with compliance check tools like PMD, Checkstyle, FindBugs etc
No focus on profiling of the application on regular basis during the code construction phase
Not evaluating the results from the Load tests to decipher and fix the underlying issue (blaming the poor infrastructure setup)

What are the other antipatterns you have observed that have contributed to the degradation in the application performance. Do share!

Deciphering an Outage

noreply@blogger.com (Munish K Gupta) — Mon, 19 Dec 2011 13:40:00 +0000

Recently, there was a major fire outage reported. The building maintenance team informed the fire brigade department. The fire brigade came in, helped douse the fire and tried to salvage whatever they could. The owner of the building demanded an independent assessment of the reasons behind the fire, what triggered it and how to make sure this does not happen again. The building maintenance team called in an independent assessment team and asked them to comb the carnage and find out what might have caused this!

The independent assessment team went through the motions of accumulating the evidence, looking for telltale signs of what might have gone wrong and then eventually did an event reconstruction on how events would have unfolded on that day. Subsequently, the team came out with the recommendations that need to be implemented to make sure that such an outage does not happen again.

Typical reasons for most of fire outages are –not conducting regular fire safety drills, violation of the building codes, not designing the fire exit paths, not keeping common area’s clean, storing hazards materials in the basement and building unauthorized extensions. This was the same case here and the recommendations were mostly in same line. Do regular fire safety drills, design fire exit paths, adhere to the building codes, demolish unauthorized extensions, stop storing hazards materials in the basement.

Can you guess the reasons for?

why the building was built in violation of so many norms
how did all the inspectors certified the building
why did the client not got an independent assessment of the building done before possession
why did the maintenance team accepted the building without looking into the cost of maintenance in the absence of basic safety
why should the people using the building not lose confidence in the safety of the building

The above scenario is analogous to what I encountered recently. Only difference is that it was a customer facing web application that faced the outage.

Managing Enterprise Group Mail Content

noreply@blogger.com (Munish K Gupta) — Mon, 07 Nov 2011 05:56:00 +0000

How many times, we have seen that one of the easiest way of cleaning up your mail box is to delete all the enterprise announcements and group e-mails? Are we getting bombarded with too many policy change mails, group emails and announcements? After a few days of the email bombardment, are our brain gets conditioned to just ignore them ?

All this leads to a point where we start becoming clueless about

Happenings in the Enterprise
Policy/Process Changes that might be affecting us

which means we might be working harder and not smarter.

Imagine, the enterprise announces some template for making presentations. The same is announced with enterprise wide email. We ignore the email contents and continuing using the older templates. One fine day, the boss looks at the presentation and says why are you using the older template and there is no body else to blame except self.

Imagine, you did saw the email and know about the new template. But you deleted the email. When a fort night later, there was a requirement to make a presentation, you are searching high and dry for the link from where you can download the template. There is no direct link available on your intranet that points to the template. There is no way to retrieve the email also.

Why do enterprise not make available all those emails available on the intranet? Is there an easier way to manage this flood of important information? How can the individual employee choose to subscribe/unsubscribe to such emails? Is there a way the employee can refer to these emails later on - where the content is available tagged and ready to be searched?

The answer to these question is simple tool - Blogs.

Enterprise can start publishing all the announcements as blog entries. This simple tool alleviates all the pains and provide a fantastic way of organizing all the content that is properly tagged and available to be searched, subscribed and even allow debates on some of those emails. How many times we wanted to comment on some policy change but could not do so because it is a group email and there is no way to provide feedback.

Having a blog does not mean, the enterprise stops sending out emails but the blog can become the one stop content repository for all such announcements. Today, publishing an entry is as easy as sending an email.

Modern Web Social Collaboration tools provide blog as a feature out of box. Enterprises can make use of the Blog as a means of storing enterprise group email content.

Most of the internet based business already make use of the blogs to connect and disseminate information about their products and services. It is just a matter of picking up the same best practice within the enterprise.

Building Scalable Enterprise Systems

noreply@blogger.com (Munish K Gupta) — Mon, 17 Oct 2011 11:04:00 +0000

Successful enterprise are all about business agility and able to introduce new products and services in the market. All this business agility coupled with reduced IT overheads means, the enterprise need to find better ways to improve and transform their enterprise systems.

The advent of Cloud, Social, Mobile and Consumerization of IT means enterprises applications need to be adapt to the changing environment. Today, every enterprise is looking to fulfill the following demands

Adopt cloud for their enterprise applications (whether private / public / hybrid is a matter of enterprise priorities)
Replace or retire in-house enterprise applications ( where applicable) in favor of the equivalent SaaS applications
Expose enterprise data for third party consumption
Make the enterprise functionality available over variety of channels (web, mobile) - Consumerization
Make the systems available 24X7 to meet the ever growing business demands

The Enterprise application patterns for creating applications – Portals, CMS, SOA, Centralized DB’s are not functionally capable or geared to meet the new business demands. The enterprise needs to adopt the newer application patterns that are coming out of the consumer web world. Some of the patterns emerging out of new generation consumer web applications that can be applied for enterprise applications

Partition by Function – Divide the functionality by usage patterns or resource consumption. The functionality can be exposed using RESTful or Web services. This functional grouping will help in achieving optimization at the resource level. This helps to scale individual functional groups based on their resource characteristics (High CPU, More RAM, High I/O). This pattern organizes the enterprise platform around services which in turn provides agility. Things can be done in parallel because the output is a service. This allows fast time to market. Helps create an infrastructure that allows services to be built very fast.
Segregate the providers and Consumers – Instead of building large monolithic applications, segregate the functionality providers versus consumers. The enterprise should have separate teams – one that deal with the functionality provider and other that consume the functionality. The functionality is exposed via simple API’s having standard versioned interfaces
Public/private cloud integration – Enterprises need to have standardized integration strategy as the need on board multiple applications from multiple vendors start becoming common.
- Enterprise need to start integrating with SaaS vendors that can provide the required apps for the consumption of the enterprise users. At times, the integration might need to be handled at the API levels specifically for authentication/authorization.
- Enterprise applications specially that are consumer facing can also be hosted in public clouds.
Enterprise need to integrate services that handle data coming in or going out from public cloud / SaaS vendors to their data centers.

Why focus on Performance Engineering ?

noreply@blogger.com (Munish K Gupta) — Fri, 14 Oct 2011 10:55:00 +0000

Time and again, we witness, when a program goes into acceptance testing phase, the client and teams suddenly realize that the application is not meeting the Non-Functional requirements. Usually the application is very slow, or it is frequently going down or not scaling up as expected. I am not even talking of requirements mismatch here.

When client suddenly raises all these questions, all hell breaks loose within the development team. The team tries to find the hot spots and try to fix them. Invariably, I have seen the teams end only fixing the symptoms of the problem but not able to correct the problem. E.g if the application is making too many calls to the database. The quick fix at times is to cache the data. But invariably, this leads to problem in the future of cache invalidations and cache synchronization if the application is clustered. Remember, when we are fixing the symptom of the problem, we are only postponing the problem to a future stage (Technical Debt). All this means, with gating criteria's not getting met, cost and schedule over run becomes a norm. This becomes the most crucial phase of the program as the chances of program getting abandoned starts increasing exponentially. All this forms the long tail, as the program just keeps dragging on.

Focus on the Non-functional requirements right from day one is very important. Getting the architecture right at the beginning is very important, as the cost of fixing the problems keep on increasing exponentially as the application develops further and in the later stages it takes a heavy toll on every one. Techniques like Performance Driven Development is a good step in the direction. Other way to analyse each aspect of the requirement and design using the lens of non-functional requirement. At times, we as architects will be able to push requirements that do not meet the NFRs.

Why are Enterprises looking at BigData?

noreply@blogger.com (Munish K Gupta) — Mon, 10 Oct 2011 07:01:00 +0000

The advent of the Social Collaboration, Online Selling, Digital Goods, Mobile means every enterprise wants to process the transactional and analytical data, that is being collected at multiple customer touch points. All this data need to be processed so that the enterprise can better understand the customer, his social network, his buying patterns and other things.

This has led to ever increasing amount of data, which is leading to the following issues within the enterprise

Business do not have access to the real time data feeds
Queries are running in hrs and minutes and not in seconds
The batch processes, ETL, data loads are taking too much of time
Ability to construct new models based on the data is time consuming
Scalability of systems with ever increasing data is becoming a problem
Data Storage/Redundancy is another problem
Increasing License cost of software/hardware is another issue

When Business looks and reads about how Facebook, Yahoo, Google etc are managing large amounts of data (BigData) and are able to process the same at real time, they want to adopt some of their systems and techniques.

The new systems/technologies that have been open sourced and are getting adopted rapidly by the enterprises are Hadoop and its various commercial versions (Cloudera, Hortonworks, GreenPlum). In addition, other commercial vendors have also jumped in with their specific BigData offerings - IBM has BigInsights, Oracle has Exalytics In-memory machine. The commercial vendors are trying to sell big machines - with more RAM and more CPU to be able to process more data.

But, the question is - are enterprises looking at to buy more hardware, software, acquire more licenses to process data or they want to solve the issues.

I believe, the fundamental problem is speed of access to the data (FastData) which is a paramount requirement for the enterprise. BigData only promises to help solve the problem of large amounts of data but it still has a long way to go before it can fulfill rest of the enterprise needs.

Grappling with Requirement Complexities

noreply@blogger.com (Munish K Gupta) — Fri, 30 Sep 2011 05:01:00 +0000

For an Architect, as if deciphering the requirements between Functional and Non-Functional requirements was not enough, DevOps and Technical Debt has opened another head for bunching requirements.

For the uninitiated, DevOps is a movement that is meant to break down the silo approach between the Dev, QA and IT teams, where the code moved in a batch mode from Dev->QA->IT Operations teams. This silo'ed approach meant delay, disruption and duplication of efforts, which ultimately leads to delays in pushing out business functionality.
DevOps promote having a integrated team of application development, QA and operations people that shortens the delivery cycles and work within the 2-3 weeks sprint cycle. This means people with multiple skills (read polyglot programming) - either coders with sysadmin skills or vice-verse. Advent of cloud has accelerated this methodology as the team developing the feature is also responsible for its deployment.

With clients insisting on shorted delivery cycles, multiple releases in a day at times means teams need be agile and staffed with appropriate skills to be able to deliver the business functionality.

Damon Edwards explained DevOps very beautifully in a blog post last year:

The most fundamental business process in any company is getting an idea from inception to where it is making you money … The whole point of DevOps is to enable your business to react to market forces as quickly, efficiently, and reliably as possible.

Traditional models of SDLC like waterfall model can not cope with the newer demands of the business. Business wants to try out new features / functionality to guage the client reaction means more releases, more agility expected from the teams.

All this agility, means the code going out is not optimal and can be messy at times. This means maintaining code or adding new features become harder and harder as we progress. All this slows down the sprint cycles.

Technical Debt is a metaphor developed by Ward Cunningham, who drew the comparison between technical complexity and debt and said

Shipping first time code is like going into debt. A little debt speeds development so long as it is paid back promptly with a rewrite... The danger occurs when the debt is not repaid. Every minute spent on not-quite-right code counts as interest on that debt.

If the business insists on reduction of technical debt along with the new features being developed, it adds another layer of complexity to the whole engagements.

Imagine, with every sprint, the team is adding new features , refactoring / rewriting the old code (to reduce/payback the Technical Debt) , ensure and account for any new Technical Debt getting added, the job of the architect to co-ordinate and co-relate all these varied requirements become lot more harder.

PaaS - is it right for me ?

noreply@blogger.com (Munish K Gupta) — Thu, 29 Sep 2011 12:31:00 +0000

An organisation embarking on a cloud journey invariably ends up looking for PaaS solutions. PaaS sounds like something that will help me take away all my pains. Creating your application, using drag and drop controls in a browser and everything else taken care (from deployment to running to scalability to back up of data and what not) sounds something exciting and very hard to believe.

Anyway, who wants to deal with the IT folks and deal with all the tantrums they throw, what is possible and what is not.

Bernard Golden, in his article "What You Need to Know About PaaS" talks about the various issues that need to be looked at, when selecting your PaaS vendor.

Lock-in - The productivity gains you get by using vendor API's also mean you get locked on to the platform and application migration becomes all the more difficult.
Complexity - Writing PaaS application is not the same as writing regular applications. Depending on the vendor, the complexity of writing application increases as you try to add more functionality
Cloud Service Provider Differentiation - The whole premise of PaaS platform is to make the writing of applications easy. With all the abstraction the platform provide, you end up getting locked at operation level
New Skills - PaaS platforms require new set of skills which means either re-train your existing employee or hire new people
Mapping existing practices to new frameworks - Every enterprise will have standard processes, tools and methodologies of working and developing applications. All these need to be re-looked based on the PaaS platform

All in all, PaaS platforms need to be evaluated based on your requirements and fitment as per your organisation needs.

Read the complete article here.

Practical approach to Capacity Planning

noreply@blogger.com (Munish K Gupta) — Wed, 28 Sep 2011 06:03:00 +0000

Capacity Planning is all about managing you resources better. Resources are finite, resources need to be procured, resources come at a cost, resources get consumed, as a result you need to do some capacity planning.

Capacity planning is an exercise undertaken in all the industries. There are plenty of models on how to perform capacity planning. But somehow application of these models in the software industry is too cumbersome, tedious and at times completely useless. These models work best when you have a standardized product and process. In software, every release changes the dynamics of the software product. The code base changes, the code performance changes, the usage pattern might change leading to the failure of the previous capacity planning exercises.

Amazon.com released a one day promotion- Lady Gaga album for 99 cents and the site went down. Lots of customers could not buy the offered album songs. Leaving lot of customers angry and loosing potential business

Looking at the problem, following question comes to mind, when to do capacity planning :

Should I wait for disaster to stuck?
Should I start capacity planning from day one?

Next question comes

How to go about doing capacity planning?

There is a simple process for doing capacity planning. It is not different from any other planning process. There are primarily four steps in the process.

Determine your capacity planning goals - For any capacity planning exercise, it is very important to determine the goals in mind. e.g. for an internet application, managing the ever increasing number of users might be a goal. Similarly, for an intranet application, the number of users might be constant, availability and response time might be the drivers for the capacity planning. Once we understand the nature of the application, and what is the end objective, we can plan the goals. These goals will determine what the measuring points that are required from application. More the measuring points, better will the forecasts.
Collect data and metrics that map to those goals - Once the goals and the measuring points are determined, we need to collect data that maps to the state goals. e.g. if 99.9% availability of the application is the goal, we need to be able to track the application up-time. We need to track and measure all the links in the applications (be it load balancers, application servers, web servers, data base servers, network, storage, routers, switches, power supply). We need to keep track of the various application usage patterns that can affect the application availability.
Plot that data and make forecasts - Once we collect the data, we need to plot that data and create trend curves. We will need to extrapolate and do curve fitting on the time series data. This will give us data in terms of application usage changes and help us in product planning. This is an on-going iterative exercise. Every new data brings in more perspective and helps make the forecast better.
Use those forecasts to enhance the capacity - Based on the forecasts, we need to enhance the capacity to tackle the new growth and achieve our goals.

Any changes in the application (bug fixing or functionality enhancement) will render the old data invalid. We will need to account for these new changes and map into the measuring points. This will help in accounting for the changes in the forecasts.

Remember, capacity planning is something that starts, when you conceive the application and it continues till the application is sun set.

Capacity planning is not an end in itself but a journey.

Enterprise Mobile Web Strategy

noreply@blogger.com (Munish K Gupta) — Wed, 03 Aug 2011 06:11:00 +0000

With the proliferation of devices in various screen sizes and running different OS, having a coherent mobile web strategy has become somewhat of a nightmare for the enterprise. Gone are the days, when the enterprise would just optimize the existing web site for mobile and let the content be served. Today’s consumer with the ever powerful device and increasing bandwidth is looking for experience that is equal to or at times better than the web.

When trying to create an mobile strategy, enterprise need to ponder over this

Should I build a native application for the mobile device
Should I build a mobile optimized web site

To be able to answer these questions, you need to study the consumer usage patterns and ask

What is the value the information provides to the consumer?
What is the frequency of access of information by the consumer?
What is the experience I want to provide to the consumer?

What is my business all about? E.g. If the enterprise is in the business of movie screenings. The consumer is interested in the information with regards to the nearest theatre location, what movies are playing, their timings and how can I buy the tickets. The user will be accessing the application multiple times during the month. In this kind of scenario, having a dedicated application is must.

But there will other consumers who have not yet installed the application. For such user’s your web site should be optimized for mobile access and provide the same information and features. In the end, you can ask him to install the application to make the future search and transaction a lot easier.

Having an application also allows you to tap into the features of the mobile device to provide an enriching experience to the consumer. E.g usage of accelerometer, camera, GPS, compass can greatly enhance the consumer experience.

But, if your enterprise is not direct consumer market, having a mobile optimized web site might will do and you need not create a native mobile application.

So, overall your mobile strategy needs to keep in mind these

You definitely need a mobile web strategy, meaning a mobile optimized web site. Whether you need a dedicated mobile app or not depends upon your answers in the earlier section.
Make sure you optimize the web site across all major mobile platforms (Android, iOS, Windows Phone 7, Blackberry) and across the common device screen sizes
Make sure you try to leverage mobile device features - accelerometer, camera, GPS, compass etc to enhance the experience
Make use of the technologies that allows to build and deploy a common code base with little variations across devices. HTML5, CSS3, webfonts should be used when building your mobile applications
Keep the mobile application design simple. Single column layouts work the best with vertical hierarchy menus. Go easy on the data consumption, today’s devices are faster but consumers do not like application that just hog their bandwidth. Try to keep images to the bare minimum
When building mobile applications, keep the usability aspects of mobile devices in mind. Taps vs clicks – means larger buttons are required, instant feedback (e.g. change the button colors to indicate that the action has been registered), typing is usually difficult – having a tap based options help.

Next step comes to the tools that can be used to build the native mobile applications

Adobe Flash Builder 4.5 offers developers one tool, one development framework and one codebase to build mobile applications on Android, BlackBerry Tablet OS and iOS. It is one integrated development platform that allows you to create mobile application for multiple platforms. You can build very rich UI that make use of device features like accelerometer, camera, GPS, compass etc. You can make use of your enterprise assets - java libraries when building solutions
PhoneGap is an open source development framework for building cross-platform mobile apps using HTML, CSS and JavaScript. The tool enables developers to utilize the core features of the iPhone, Android, Palm, Symbian and Blackberry smartphones, including geolocation, accelerometer, contacts, sound and vibration to build applications
RhoMobile offers Rhodes, an open source, Ruby-based framework that allows for development of native apps for a wide range of smartphone devices and operating systems(iPhone, Android, Windows Mobile, RIM and Symbian)
Appcelerator provides the Titanium Development Platform which allows for the development of native mobile, tablet and desktop applications through typical web dev languages such as JavaScript, PHP, Python, Ruby and HTML

In addition, if you are looking for just the mobile optimization of your web sites and functionality then look for offerings from - netbiscuits, mBlox, iLoop Mobile, Crisp Media, Quattro Wireless, 3Cinteractive.

MapReduce for dummies

noreply@blogger.com (Munish K Gupta) — Mon, 11 Jul 2011 13:28:00 +0000

Continuing the coverage on Hadoop component, we will go through the MapReduce component. MapReduce is a concept that has been programming model of LISP. But before we jump into MapReduce, lets start with an example to understand how MapReduce works.

Given a couple of sentences, write a program that counts the number of words.

Now, the traditional thinking when solving this problem is read a word, check whether the word is one of the stop words, if not , add the word in a HashMap with key as the word and set the value to number of occurrences. If the word is not found in HashMap, then add the word and set the value to 1. If the word is found, then increment the value and word the same in HashMap.

Now, in this scenario, the program is processing the sentence in a serial fashion. Now, imagine if instead of a sentence, we need to count the number of words in encylopedia. Serial processing of this amount of data is time consuming. So, question is is there another algorithm we can use to speed up the processing.

Lets take the same problem and divide the same into 2 steps. In the first step, we take each sentence each and map the number of words in that sentence.

Once, the words have been mapped, lets move to the next step. In this step, we combine (reduce) the maps from two sentences into a single map.

That’s it, we have just seen how, individual sentences can be mapped individually and then once mapped, can be reduced to a single resulting map. Advantage of the MapReduce approach is

The whole process got distributed in small tasks that will help in faster completion of the job
Both the steps can be broken down into tasks. In the first, instance, run multiple map tasks, once the mapping is done, run multiple reduce tasks to combine the results and finally aggregate the results

Now, imagine this MapReduce paradigm working on the HDFS. HDFS has data nodes that splits and store the files in blocks. Now, if map the tasks on each of the data nodes, then we can easily leverage the compute power of those data node machines.

So, each of the data nodes, can run tasks (map or reduce) which are the essence of the MapReduce. As each data nodes stores data for multiple files, multiple tasks might be running at the same time for different data blocks.

To control the MapReduce tasks, there are 2 processes that need to be understood

JobTracker - The JobTracker is the service within Hadoop that farms out MapReduce tasks to specific nodes in the cluster, ideally the nodes that have the data, or at least are in the same rack.
TaskTracker - TaskTracker is a process that starts and tracks MapReduce Tasks in a cluster. It contacts the JobTracker for Task assignments and reporting results.

These Trackers are part of the Hadoop itself and can be tracked easily via

http://<host-name>:50030/ - web UI for MapReduce job tracker(s)
http://<host-name>:50060/ - web UI for task tracker(s)

HDFS for dummies

noreply@blogger.com (Munish K Gupta) — Fri, 08 Jul 2011 04:30:00 +0000

Whenever a newbie wants to start learning the Hadoop, the number of elements in a Hadoop stack are mind bogling and at times difficult to comprehend. I am trying to de-crypt the whole stack and help explain the basic pieces in my own way. Before we start talking about the Hadoop Stack, let us take a step back and try to understand what led to the origins to the Hadoop.

Problem – With the prolification of the internet, the amount of data stored growing up. Lets take an example of a search engine (like Google), that needs to index the large of amount of data that is being generated. The search engine crawls and indexes the data. The index data is stored and retrieval from a single storage device. As the data generated grows, the search index data will keep on increasing.

As the number of queries to access data increase, the current file system I/O becomes inadequate to retrieve large amounts of data simuntaneously. Further, the model of one large single storage starts becoming a bottleneck. To overcome the problem, we move the file system from a single disk storage to a clustered file system. But as the amount of data keeps growing the underlying data that can go one one machine starts to become a bottleneck.

As data reaches TB’s, existing file system based solutions starts faltering. Data access, multiple writers, large file sizes soon become a problem in scaling up the system.

Solution – To overcome the problems, an distributed file system was concieved that provided solution to the above problems. The solution tackled the problem as

When dealing with large files, I/O becomes a big bottleneck. So, we divide the files into small blocks and store in multiple machines. [Block Storage]
When we need to read the file, the client sends a request to multiple machines, each machine sends a block of file which is then combined together to pierce the whole file.
With the advent of block storage, the data access becomes distributed and leads to a faster retrieval/write
As the data blocks are stored on multiple machines, it helps in removing single point of failure by having the same block on multiple machines. Meaning, if one machine goes, the client can request the block from another machine.

Now, any solution that implements file storage as blocks needs to have the following characteristics

Manage the meta data information – Since the file gets broken into multiple blocks, somebody needs to keep track of no of blocks and storage of these blocks on different machines [NameNode]
Manage the stored blocks of data and fulfill the read/write requests [DataNodes]

So, in the context of Hadoop –The NameNode is the arbitrator and repository for all metadata. The NameNode executes file system namespace operations like opening, closing, and renaming files and directories. It also determines the mapping of blocks to DataNodes. DataNodes are responsible for serving read and write requests from the file system’s clients. The DataNodes also perform block creation, deletion, and replication upon instruction from the NameNode. All these component together form the Distributed File System called as HDFS (Hadoop Distributed File System).

Reference - http://hadoop.apache.org/common/docs/current/hdfs_design.html

HDFS has an inbuild redundancy and replication feature that makes sure that any failure of the machine can be dealt without any loss of data. The HDFS balances itself whenever a new data node is added to the cluster or any of the existing datanode fails.

In addition to the distributed file system called HDFS (Hadoop Distributed File System), there are 2 other core components

Hadoop Common – are set of utilities that support the Hadoop subprojects. Hadoop Common includes FileSystem, RPC, and serialization libraries.
Hadoop MapReduce - is a programming model and software framework for writing applications that rapidly process vast amounts of data in parallel on large clusters of compute nodes

So, effectively, when you start working with Hadoop, HDFS and Hadoop MapReduce are the first 2 things you encounter. I will cover MapReduce in subsequent posts.

Building High Performance Applications

noreply@blogger.com (Munish K Gupta) — Sun, 12 Jun 2011 15:24:00 +0000

Performance is one word which is used to describe multiple scenarios’ when talking about the application performance. When someone says, I need a High Performance Application; it might mean any/all of the following

Low Web latency Application ( meaning low Page Loading times)
Application that can serve ever increasing number of users (scalability)
Application that does not go down (either highly available or continuously available)

For each of the above, as an Architect you need to dig deeper to find out what the user is asking for. With the advent of cloud, every CIO is looking to build applications that meet all of the above scenarios. With the advent of elastic compute, one tends to think that by throwing hardware to the application, we may be able to achieve all of the above objectives.

The techniques employed to achieve the above scenarios at times are different and it is important to find the right approach to the solution that meets the above objectives. We will examine some of the common techniques that can help us achieve the objectives

Latency Contributors

Application Tier ing – One of the biggest contributors to the latency is the application tier ing. The hops from WebServer -> Application Server -> Database and back, data serialization/deserialization are some of the biggest contributor to the overall latency. Having Web and Application tier within the same box or even within same JVM can help reduce the network latency factor. One can have logical separation in the application code between Web Tier and Application Tier but need not have physical separation. Using Spring Container that has Web/App tier can help achieve the same. If the application is making use of SOA and making multiple web services or JMS message calls, network latency and serialization of data once again adds to the latency. Solutions like IBM Datapower XML Accelerators can be used to reduce the XML overheads. Similarly, the application can use Solace Message Router’s to speed up the messaging.
Bring Data closer to Application – Data needs to be close to the application so that making all those Database connection calls and getting data from DB can be reduced. Application can cache data to reduce the calls to DB. One can also use cache servers like memcached / ehCache to cache data at the Web/application Tier. Web Tier can cache data like static HTML fragments/images/javascript/CSS files. Application Tier can cache non-transactional data (like lookup maps). OR Mapping tools like Hibernate also support data caching. If it is an Internet Web Application, one can also make use of CDN (content delivery networks)/ Edge Networks (like Akamai) to speed up the delivery of static content.
Disk I/O – Another weak link in the application performance chain is Disk I/O. One way to overcome the limitations with regards to the Disk I/O is too keep data in memory. In Memory databases (like Volt DB or Solid DB or Oracle TimesTen), XTP solutions (like Oracle coherence, IBM eXtreme Scale, GigaSpaces eXtreme Application Platform) can used to speed up the application performance.
Optimized Hardware – The hardware on which application is hosted can also be tuned to reduce latency. Optimization s like 10G/20G network, fiber channels, low latency switches, SSD (Solid State Drives), not using virtualization can make sure the application latency is reduced.
Transport Mechanism – At times, the transport mechanism can also add to the application latency. E.g. secure communication (like https) can add to the latency with the additional overhead of deciphering the data at the receiving end. One way is to offload the SSL at the Load Balancer/Firewall.

Application Scalability – Scalability means ability of an application to handle growing amount of data and concurrency in an efficient manner without impacting performance. Important thing to notice is scalability should not be at the cost of application performance. Some of the techniques that can help scale the application

Stateless Application/Service – The application should store its state in some centralized repository, but the application itself should be stateless. It means no storing of data or state on local file systems. Stateless application allows one to add any number of application instances to accommodate the increasing growth. But soon, the centralized repository starts becoming the bottleneck. With ever increasing data, repositories like (RDBMS) may start buckling down. One approach to this issue is to minimize mutable state in the database. To handle such scenarios, techniques like data sharding need to be applied. Another approach to managing write contention in the database is to look at the possibility of using NoSQL data stores for some or all of the application data.
Load Balancing – As the traffic starts going up, the application can handle the additional load by adding additional server instances to service the requests. The load balancer will make sure none of the servers are working beyond their stated load and new instance should be automatically added as and when the load goes up (auto scaling). One can also add load balance to database with techniques like Master-Master topology or Master-Slave(with partitioning read and write data) to handle the additional load. But if the data is going in Petabytes ranges, data sharding with data replication techniques need to be used. The in-memory data grid architecture can also be utilized to scale the data.
Fault Tolerance / Dynamic Discoverable Elements – When dealing with application that is running in large clusters, it is very important to avoid manual interventions. E.g. when the application load reaches a defined load, the application monitoring should be able to add a new instance and load balancer should be able to recognize the same to utilize it. Similarly, when data gets shard, the applications should be able to recognize and look up the new IP to connect. Similarly, if the application is not able to connect to particular resource, the application should be intelligent enough to recognize the fault and try accessing the alternate resource availability. The application will need to have a central meta data repository for all such fault tolerance scenarios that can be tapped by the application.

Application availability – Availability of an application is very much a function of scalability. Following factors have an impact on the application availability

Redundancy – The application needs to be scalable to be able to compensate for the loss of any instance (whether hardware or software). The redundancy needs to be build at all layers, Software, Hardware, Power and even at data center levels. Even if the data center goes, the user should be able to access the application. Many at times, the level of redundancy and down time is a factor of how money is being thrown at the solution. Remember some problems have no solutions within the context of today’s technology. E.g. real time data mirroring or data sync across data centers that are located geographically apart.
Fault Tolerance – The application needs to be fault tolerant (e.g. retry mechanism) to make sure it can take advantage of dynamically allocated resources to keep functioning.
Monitoring/Testing – Another overlooked factor of application availability is application monitoring. If application is not properly monitored, outages can go undetected leading to application unavailability. Ability to monitor the entire application stack and take corrective actions is very important. This capability is build over a period of time. Once the application has monitoring, auto-scaling features, testing to make sure they work is also important. Something like Chaos Monkey used by Netflix is very helpful.
Configuration Data – Any application that needs to be continuously available needs to be able to run using configuration. E.g. if the application introduces the new service interface, the application should have the ability to either make use of the new interface or keep using the old one. This factor becomes very important when rolling out new features/services and all of them cannot be rolled out at once.

Let me know, if you have used some other techniques to achieve the 3 factors of availability, scalability and low latency.

Maladies of Large scale Software Design

noreply@blogger.com (Munish K Gupta) — Tue, 26 Apr 2011 11:38:00 +0000

When designing and maintaining large scale software architecture, certain rules need to be adhered religiously. In the absence of which the software starts degrading, maintenance becomes difficult, adding features becomes a nightmare and soon everything comes crashing down as a house of cards.

Tiering/Layering your logical architecture – a clear logical separation between different layers lead to simplified code navigation and comprehension. The layers should also follow a strict and clear naming convention for all packages and types. One can make of tools to make sure the code follows the logical architecture and all dependencies are in order.
Cyclical dependency – Adhere to the principle of well defined and cycle free application. Cyclic dependencies can soon lead to bloating of code. Even package should be validated for any cyclical dependencies
NCCD (Normalized cumulative component dependency) is another factor that needs to be adhered. NCCD of compilation units must not be bigger than 7. If this value grows over the threshold, one should isolate layers and subsystem by only letting them have interfaces as entry points. Breaking cyclic dependencies can also shrink this metric considerably.

In addition to the above rules, one can also gauge the health of the software using additional metrics like

Code Coverage– describes the degree to which the source code of a program has been (unit) tested. Popular tools to measure code coverage in Java are Emma.
Code Complexity - is another measure that can used to identify and quantify the code maintainability. Factors like Cylometric Complexity, Halstead's software science metrics and Lines of code metrics are some of the factors. In fact, there is a measure called Maintainability Index that can be calculated by using the factors.
Documentation - is another factor that be used to quantify code maintainability. Documentation can be your normal JavaDocs or the inline source code commenting. You have tools like JavaNCSS where NCSS ( non commenting source statements) that can be applied at class, method/function level
Code Format - is not really a measure but is an indicator of the readability of the code. Tools like CheckStyle can help identify the code formatting problems (from non standard class headers, indentation issues to spacing, variable naming convention and so on). Any issues reported by the tool can be used an indicator of the maintainability of the code

From a programmer’s perspective, the following rules should be followed, that will help in building and maintaining a robust and lively system

Use a consistent formatting and naming scheme
Limit the access to types and methods
Number of types in a package must not exceed 50
Lines of code (compilation unit) must not exceed 700
Number of method parameters must not exceed 7
Cyclomatic Complexity must not exceed 20
Use assertions to ensure pre-conditions and post conditions

Do share your best practices that you followed or came across when build or maintaining large scale software architectures.

Is Google the new Knowledge Manager?

noreply@blogger.com (Munish K Gupta) — Wed, 20 Apr 2011 08:42:00 +0000

In today’s world, whenever you are facing a problem, the first impulse is to open Google and see if other people had already faced the similar problem and how they did they resolved the same. The good part is you are most likely to find the solution to the problem also. Does that mean, Google is the new Knowledge Manager for your enterprise?

Why do the employees tend to search on Google/Bing for solutions and not look at the KM systems?

Some of the factors that might be pushing your employees to search on external search engines might be

Disparate Systems – In a typical enterprise, the traditional KM systems have evolved over a period of time. The sum total of KM is usually a combination of SharePoint repositories, Discussion groups, Emails groups, Blogs or even file system dumps. Now when is looking for information, he needs to search through multiple repositories to mine the information. Instead of trying to search through this myriad of information pile, he gives up. After a few tries, he is most likely to form an opinion that these systems are not helping, so either they do not have information or are too difficult to mine.
Usability – Another issue with enterprise KM systems is lack of usability. Since the disparate systems have been built over a period of time, they have multiple interfaces and at times multiple logins to access. So, if the user has to log into multiple systems to search or access information, he is most likely to get pissed off. Another usability factor is the speed of access. Since, KM systems are not really on high end servers with Gbs of bandwidth, these systems tend to be slow, which when compared to Google proves to be another sore point for users.
Education – When the user’s of your KM systems were studying, the colleges they studied at, are not really very great proponents of KM. Most colleges hardly have KM systems with latest information and some of them which do have, suffer from the same problems of disparate systems and usability. The students in the end start relying on Google for solutions to all their problems. The same experience continues when they join the industry.

Today, the challenge of Knowledge managers is not really building a knowledge repository but better the kind of access, speed and accuracy of helping provide solutions to the employee problems.

.

Business ideas waiting in the Healthcare Sector

noreply@blogger.com (Munish K Gupta) — Wed, 06 Apr 2011 08:21:00 +0000

Healthcare Sector in India is poised to grow to $280 Billion USD by 2020. The investment required in the next 10 years is to the tune of $86 Billion USD, majority of which will be contributed by the private sector. All this investment will need require lot of people that need to be integrated in this sector and IT backbone will be needed to be able to service the ever-growing Indian population.

Some of the opportunities I see that are up for grabs in the healthcare sector

Training – There will be requirement to train people to handle emergencies, health care, education services. What better way than to have 3D virtual areas to train people across regions. Players like http://play2train.us/ are already available in the US market. We will need similar business models in India, that can take into account multiple languages and varied cultures
Communities – Another big aspect is building communities (online/mobile) that can form a symbiotic relationship with other stakeholders and make sure that the remote corners of India can avail healthcare. These communities can be around following
- Patients – people with similar problems being able to talk and discuss their problems and potential treatments with each other. We make use of multiple medicine streams – Allopathic, Ayurveda, Homeopathy, Naturopathy, Yoga and what not. At times, when we discuss our medical problems, cure in one stream might be better than the other. These patient communities can help people connect with each other. The community can create wiki’s (user generated content) around their problem that can become important sources of knowledge and information for others.
- Physicians – Another community can be around physicians and some of their experiences when treating different problems. The physician community can also contribute towards a knowledge wiki (from physician perspective) that can be useful for physician in far flung area’s and even disseminate knowledge among tier 2 physicians. Physicians can blog/podcast to discuss case studies.
- Donors communities – Unlike US, we do not have donor list or communities which can be accessed and available online. There are privacy concerns but I do not think those are insurmountable.
- Pharma Companies – The Pharma companies can come together to create communities with their patients (their clients) to discuss the usage of the various drugs and their administration mechanism. People can share their experiences. E.g. Biocon can lead a community around Diabetes, where you promote healthy lifestyle. The Pharma companies can also tap into the various medical insurance firms, because it is in their interest to have a healthy community.
- Clinical Trial Companies – Another great community can be around clinical trial patients, where the potential patients or existing patients can tap into each other, bringing a lot more transparency in the working of these companies.
So, overall I believe a great number of ideas/sub ideas can arise from building these communities. The whole user generated content from these communities can be precursor to a great knowledge base that can be tapped from all over the world.
Location based services – currently, the existing location based services help you find friends or restaurants or places of interest. Another use of such service can be in case of emergencies. E.g. a physician can opt to register on a location based service. In case of emergency, the people raise an alert online. The online service automatically scans the vicinity for a physician who has opted for such thing and he can be notified immediately. The physician can then tell the service, whether he can attend the emergency or not, leading the online service scanning for next available physician. With the kind of traffic we have, such a service can be of immense potential. Further, such a service can SMS based.

We are seeing more and more adoption of mobiles phones in India, any service that allows access to such communities and knowledge base over SMS will be of immense help. The cloud computing will enable the implementation and scalability of such idea's.

Let me know, how do you find these idea’s and what else you think might be useful in the context of the Indian healthcare sector.

Image: jscreationzs / FreeDigitalPhotos.net

Challenges of Developing Applications for hosting in cloud

noreply@blogger.com (Munish K Gupta) — Tue, 15 Mar 2011 05:49:00 +0000

Recently, I was asked this simple question; do I need to do anything differently when developing applications that will get hosted in cloud?

Before answering the question, my though process went something like this –

Well, cloud uses lots of commodity boxes; and failure is almost given. The application needs to take care of resource failures, meaning build fault tolerance. Besides, you may want to make use of additional cloud features (like if you are using AWS, then besides the EC2, S3, RDS, CloudFront, CloudWatch, Load Balancing, auto scaling etc) but if you are migrating an existing application then you need not do so in the first phase.

Besides this one reason, I could not fathom another good solid reason. Anything you will do additional in the application, that you would not had done when deploying to given hardware spec. I could think of many things as an application developer that you need to take care, but then you should had done this in any cases.

Application State – Any application that needs to be scaled up, cannot store application state in the local machine. The state either needs to be stored in a centralized DB or file system.
Resource Independence – When developing applications, the application should the agnostic to the resource location. The resources needs to be looked up at run time and should not be hard coded into application code. Common application programming guideline. Patterns like ResourceLocator are meant for the same. Application servers also provide mechanism to the similar effect.

If one does not adhere to these common programming practices, development and deployment will become difficult in any case.

So, have you faced or come across patterns that need to be handled specifically when developing applications that get deployed in cloud (private or public), do share them.

.

Creating Web 2.0 sites with Enterprise Portal Software

noreply@blogger.com (Munish K Gupta) — Sun, 27 Feb 2011 06:24:00 +0000

The whole premise behind the Enterprise Portals was to consolidate and bring together the various silo’ webs sites in an organization. The Enterprise Portal provides tools to the same effect like User Management, Credential Vault, Personalization and Collaboration to name a few. But in recent times, the Enterprise Portal is being used outside of organization (Intranets) to create corporate/transactional sites for your end customers. The rigors of Internet sites demanded are very different from Intranet sites and at times, Enterprise Portal are not able to meet their demands. We will some of the functionality expected from Web2.0 internet sites that need to be factored when making that all important decision of using an Enterprise Portal Software.

Web Analytics – For an internet site, web analytics is very important. The simple tracking of user can be easily achieved by adding the required code in the theme part of the portal software. If you need to track the user activity at more granular level like button clicked, banners served/clicked, time spend on page etc, the required tracking code needs to be added to each Portlet. When these Portlets are added to the page, their behavior might not be what you expected. Reasons may be tracking code contradicting each other, methods with same name, and similar names to the tracking events and so on. Only way out is to have a very elaborate strategy that needs to be followed diligently.
SEO – Another important aspect is the SEO for the internet site. The Portal page need not correspond to a single page on the internet. Based on the input, the content shown might be different. As part of the SEO, things like Title, page metadata like keywords, description is very important. Changing these to conform to the SEO can be nightmarish and not something achieved without a deployment cycle.
Advertising/Cross Sell/Up Sell Banners – Any internet site, you may want to show advertising banners to cross sell/up sell your products. Now, you might want to integrate the banner serving through a 3rd party site or software. Whether the banner should be a Portlet or JavaScript driven starts impacting your design. E.g. if you decide to use portlets to serve banners, and want to show 2-3 banners on a page, the page load starts suffering because as the number of portlets starts going beyond 5, the page performance started suffering. If you intend to use JavaScript, then the same needs to be added to the theme. Theme means deployment cycle and also you may not loose flexibility in changing the banner positions on the fly.
A/B and Multivariate Testing - Another tool used by the business managers to fine tune their web pages is A/B and Multivariate testing. Now, using multiple versions of the same page within a Portal is not something easy to accomplish. If you are using an external authentication provider, running multivariate’s for the landing page becomes even more challenging.
3rd party Integrations (like Facebook, Twitter)- At times, you want to integrate the customer experience with 3rd party sites like Facebook and Twitter. Without an effective SEO, the FB/Twitter can be big failure. The copy paste approach promoted by FB/Twitter is not something very applicable in the Portal world. You will require a deployment cycle just to add a Like button.

So, if you are planning to use an Enterprise Portal Software to build your internet side, ask you vendor to demonstrate how effectively and easily the above points are achieved.

IBM Tuesday – Packt Publishes Four New IBM Books

noreply@blogger.com (Munish K Gupta) — Mon, 14 Feb 2011 07:11:00 +0000

On Tuesday 22^nd January 2011 Packt Publishing released four brand new IBM books on a range of different subject matters:

· IBM Lotus Quickr 8.5 for Domino Administration

· Getting Started with IBM FileNet P8 Content Manager

· IBM WebSphere Application Server v7.0 Security

· IBM Rational Team Concert 2 Essentials

This follows the success of Packt’s 2010 IBM range, which includes bestselling titles the IBM Lotus Notes 8.5 User Guide and IBM Cognos 8 Report Studio Cookbook.

James Lumsden, the Packt Enterprise publisher, states “IBM books represent a vitally important and significant part of Packt’s publishing strategy. 2010 was a good year, whilst 2011 and beyond will see more books across more technologies being published. IBM Tuesday represents just the beginning.” IBM Tuesday also emphasizes Packt’s commitment to providing top quality books for a range of IBM software users.

To mark this special occasion Packt is offering a 20% discount off all Packt IBM books throughout February. There is also a chance to win a PacktLib subscription, worth $220, every Tuesday in February with the IBM Tuesday Competition.

For further information on this, and other books published by Packt Publishing, please visit www.PacktPub.com

PS : This is a paid post.

Cloud computing fueling new generation of companies

noreply@blogger.com (Munish K Gupta) — Thu, 10 Feb 2011 14:36:00 +0000

Cloud computing has become a kind of buzz word. The availability of cheap computing power allows organizations to start with out a heavy capital cost in the infrastructure. But the fundamental change it has bought is terms of the business models of the new organization starting up.

Massive Computing - Gaming companies require massive computing and generate lot of data.Cheap computing power means, companies starting can scale overnight when the demand starts going up. Firms like Zynga ( remember Farmville) are products of this cheap computing power. Companies moving to Facebook encountered massive traffic overnight. It was the availability of cheap computing power from vendors like AWS that allowed them to scale overnight. And today, the valuations these facebook app companies are commanding is mind boggling.
Cloud Infrastructure Management - Another area spawned by cloud providers is service providers for the cloud infrastructure. Companies like RightScale, Elastra, Scalr and othere are there to help manage your infrastructure in the cloud. Numerous companies in the area of server monitoring have come up to manage the increasing demand of cloud resources
Business Intelligence/Data Mining Services - Another business area that has flourished are the companies providing data mining facilities. Some of the allied services on twitter are possible because of cheap computing. Ability to index and maintain huge data allows companies to start providing Business Intelligence and Data mining functionality to their customers.
Data Storage/Back up Apps - Another area is the number of data storage application that have come up in the recent past. Apps like DropBox, JungleDisk are offering free storage courtesy of cheap storage available.

Do you think, any other business that have been fueled by advent of cheap computing/storage? Do leave your comment!

Create Facebook (FB) Applications in a ziffy

noreply@blogger.com (Munish K Gupta) — Wed, 09 Feb 2011 13:02:00 +0000

Social Media presence has become the de-facto standard for all the companies. Companies trying to create FB applications have multiple choices available to create and integrate applications in FB. Facebook provides a rich set of API's to allow developer's to integrate FB functionality.

Recently, I came across from embarcadro RadPHP XE that allows you to create PHP based application including Facebook apps in a easy and intuitive style. One need not break head on the FB developer site to understand the API's, as the RadPHP XE provides facebook libraries that wrap the whole functionality.

Check out the tutorials

Some useful links on IBM products

noreply@blogger.com (Munish K Gupta) — Wed, 24 Nov 2010 07:10:00 +0000

IBM Lotus Sametime 8 Essentials: A User's Guide

iNotes and Sametime—Chatting from the Web

Installing and Using Sametime Connect

IBM Cognos 8 Report Studio Cookbook

Tips and Tricks: Report Page in IBM Cognos 8 Report Studio

Use of macros in IBM Cognos 8 Report Studio

IBM Lotus Notes and Domino 8.5.1

Integrating Lotus QuickPlace/Quickr with IBM Lotus Notes and Domino 8.5.1

IBM WebSphere eXtreme Scale 6

Data Modeling Naming Standards with IBM InfoSphere Data Architect

Application Development for IBM WebSphere Process Server 7 and Enterprise Service Bus 7

Building Your Hello Mediation Project using IBM WebSphere Process Server 7 and Enterprise Service Bus 7

IBM Cognos 8 Planning

Reporting Planning Data in IBM Cognos 8: Publish and BI Integration

Understanding the Model Development Process in IBM Cognos 8

IBM InfoSphere Replication Server and Data Event Publisher

IBM WebSphere MQ commands

Q Replication Components in IBM Replication Server

MQ Listener, Channel and Queue Management

WebSphere MQ Sample Programs

IBM Lotus Notes 8.5

Feeds in IBM Lotus Notes 8.5

Mail Basics in IBM Lotus Notes 8.5

Enterprise Instant Messaging using Sametime in IBM Lotus Notes 8.5

These links are courtesy of PacktPub, received as part of IBM Month.

Why India has a long way to go before producing a Google or Microsoft?

noreply@blogger.com (Munish K Gupta) — Wed, 10 Nov 2010 14:31:00 +0000

There have been talks about how CXO’s from American companies see India producing a la Google or Microsoft. The huge IT capabilities lead them to believe that a world class product company is just around the corner. But, in my opinion, a product company in the league of Google/Microsoft/Apple is a long way to go and my reasons for the same

• Lack of Local Market – the biggest problem is lack of big enough local market in India. Any product company cannot become big unless they have a large enough local market to prove and refine themselves. Look at Google, Microsoft, Facebook, Twitter and one big reason for their early success is the large enough local market for them to grab. They ventured out of US only after they have established themselves in their local market. In India, barring a few domains like online gaming, no other domain provides a big enough market for any product company.

• Purchasing power of Indian consumer – The purchasing power of your customer decides how fast you can become big. If you sell a product for Rs2 to 100,000 customers, you make 200000 rupees, if you sell the similar product in US - $2 product to 100,000 customers you make 200,000 USD (~90 lakh rupees). The difference is to the ratio of the exchange rate (~45). In terms of infrastructure cost of building and marketing a product is not in the same ratio (~45). As a result, Indian companies either need to sell to more customers or charge more. Both cases not so easy. All said and done, 200,000 USD will lot more compared to 200,000 rupees.

• Lack of Internet/PC penetration - Lack of penetration of internet/PC is another grouse for establishing any large product company in the IT product space. Mobile is the market where Indian companies can score. 600 million+ mobiles phones give you a readymade market for mobile applications. And we already see lots of product companies in this space selling doing everything to cater to the vast Indian consumer.

• Risk averse nature of Indian Companies – Indian companies are by and large risk averse. They are not willing to put their neck on line and go with a startup. They will still prefer to buy from a large known name instead of relying on a newbie. Like they say, nobody gets fired for choosing a IBM/Microsoft product.

• Lack of original ideas - most of the product companies getting started are replica’s of their successful US counterparts. But then, the Indian VC community is also responsible for this, as they tend to invest in those copycats. Copycat idea cannot create great companies although they can be good acquisition targets.

Given all the challenges, saying that Indian product companies have no future, will be wrong. It is just that they need look out of India to market their products. iPhone/iPad applications are being made in hundreds in India and these apps have led to some successful companies. Online gaming, we see already some successful companies.