The Object Oriented Life

Coding Is Solved. Context Engineering Is Your New Job

2026-03-14T23:49:00.000-07:00

In this blog, I want to talk about something I have been experiencing first-hand over the last several months and hearing echoed in almost every conversation I have with developers, both inside and outside the company. AI can write code. That part is done. The question now is, what do we do about everything else?

Coding is solved

I use AI coding assistants every day. GitHub Copilot at work and Claude Code and other tools for personal projects - they all produce working code that I would have spent real time writing myself. And when I talk to other developers, the story is the same. The code that comes out is not perfect, but it is good enough that the bottleneck has clearly shifted. We are no longer waiting on the model to get better at writing functions. It is already there.

But here is what I have also noticed. Writing code was never the hard part of delivery. It is maybe 25% of the work. The rest is understanding what to build, making architecture decisions, writing tests that actually cover the right things, deploying safely, keeping documentation alive. And in all of those areas, AI is mostly disconnected. Your architecture docs sit in Confluence getting stale while the model hallucinates a database schema that contradicts your actual system. This wouldn't be a surprise for those building software for long time, but somehow it is :).

So the question that I keep coming back to is not "can AI code?" but rather "why does the same model give me brilliant output on one project and garbage on another?" And the answer, every single time, comes down to one thing.

Context Engineering is the real challenge

I started noticing a pattern. When I give the model my architecture doc, my domain rules, my coding conventions — the output is almost ready to commit. When I give it nothing, it writes generic tutorial code that does not fit the project at all. Same model. Same settings. Completely different result. The only variable is the context I provide.

This is what people are now calling Context Engineering. It is the practice of structuring and curating the information you feed to an AI so that it produces output that actually fits your team's codebase, your domain, and your way of working.

And here is the hard truth that I have learned from trying to shortcut this. No vendor does it for you. Your architecture is unique. Your domain knowledge is proprietary. Your coding standards are local decisions that your team made for specific reasons. Your workflow is not the same as any other team's workflow. I have tried using other people's instruction files and prompt setups. They do not help. Their context is not my context.

This is also why the developer community is so split on AI right now. On one end you have the vibe coders who ship impressive demos that fall apart in production. On the other end you have skeptics who see AI-generated bugs and decide the whole thing is not worth the risk. I think both are wrong. The vibe coders lack discipline. The skeptics lack structure. The developers in the middle, the ones who apply AI with curated context and human review at every stage, those are the ones actually shipping faster without creating new problems.

The other thing I have realised is that Context Engineering is not a solo activity. It is a team-level investment. You need your architecture documented where the model can read it. You need your requirements kept current after every release. You need your domain knowledge written down, not locked in someone's head. You need your workflow defined so that everyone on the team uses AI the same way. When all four of those things are maintained, AI output aligns with your codebase. When any one of them goes stale, the output drifts back to generic territory. The model did not get worse. Your context did.

There is also a practical sweet spot worth knowing about. From what I have calculated - a codebase in the range of 20k to 40k lines of code, with context docs totalling around 1k to 2k lines, fits comfortably within the context windows available today, which range from 1M to 4M tokens depending on the model. That means for a well-scoped project with good documentation, you can feed the model nearly everything it needs to understand your system in a single session. That is a significant advantage and one more reason to invest in keeping your context docs concise and current. Well, you can argue that this is not straightforward translation to LoC to tokens, you can use the Context Window UI in the new version of VS code to get a sense of how it working for your project.

And the best part is that it compounds. Every feature you ship with structured context makes the next feature faster. The architecture doc gets richer. The domain rules get sharper. The prompts get tighter. It is compound interest for engineering velocity. But the inverse is also true. If you wait for a turnkey solution, you will still be waiting while other teams are two cycles ahead.

Context Engineering across different system architectures

One thing I have noticed is that Context Engineering does not look the same for every type of system. The architecture you are working with fundamentally changes what context you need to provide and how easy it is to manage.

If you are working on a modular monolith with full stack code in a single repository, you are in the best position. The model can see your routes, your business logic, your templates, your database layer, all in one place. Context Engineering here is relatively straightforward. The main thing to be deliberate about is documenting the interfaces between your modules clearly. If you have stored procedures, for instance, write down how they connect to the application layer, what calls them, what data they expect, and what they return. The model will not infer those connections from code alone, especially when the logic crosses the boundary between your application and your database. But once those interfaces are documented, the model can reason about your entire system coherently because everything lives together.

Micro-services are a different story. When your system is spread across multiple repositories, each service only sees its own code. The model has no visibility into the services your code depends on or the services that depend on yours. This is where Context Engineering becomes critical and also harder. You need to provide clear documentation about inter-service connections: what APIs your service calls, what contracts it expects, what events it publishes or consumes, and how authentication flows between services. Without that, the model treats each service as if it exists in isolation and produces code that breaks at integration boundaries. I have also noticed that teams with very thin micro services, services that do almost nothing on their own, struggle more with AI-assisted development. The model has so little code to work with in each repo that it lacks the context to make meaningful contributions. There is a practical minimum size below which AI assistance loses its leverage.

Three ways to start learning and applying Context Engineering

So here are three things I would recommend, based on what has worked for me and the teams I have spoken with.

First, write down what the model cannot find on its own. Every team has knowledge that lives in Slack threads, in senior engineers' heads, in tribal lore that "you just have to know." That is exactly the knowledge that makes AI output go from generic to useful. Start with an architecture doc, a domain doc, and a standards doc. Keep them in a docs folder in the repo. Keep them short. A page or two each is fine. And update them when they drift because stale context is worse than no context. It teaches the model the wrong patterns.

Second, build a repeatable workflow with human review gates. The failure mode of AI adoption is not that the model is bad. It is that nobody on the team agreed on how to use it. One developer prompts from scratch every time. Another copy-pastes from ChatGPT. A third refuses to use it at all. No consistency, no compounding. What has worked for me is a staged approach: generate a structured user story from context docs, review it, then run code and test generation in parallel from that approved story, review both, then generate automation tests, review those, and finally ship and update the context docs so the next feature benefits from everything you just learned. That last step, updating the docs after shipping, is the one everyone skips. It is also the one that makes the whole flywheel turn.

Third, treat your prompt templates like shared code. Stop writing prompts from scratch every time. Create reusable templates for the tasks you do repeatedly — story writing, code implementation, test generation, code review. Store them in the repo, version them, improve them when the output quality dips. When a new team member joins, they should not have to reinvent your prompts. They use the same templates, get the same quality, from day one. That is how you scale AI-assisted delivery beyond a few power users.

In conclusion

Coding is solved. The teams still debating whether AI can write code are solving last year's problem. The real challenge, and the real opportunity, is Context Engineering: curating the architecture, domain, standards, and workflow knowledge that turns a generic model into something that actually fits your team.

No one sells this off the shelf. No plugin auto-generates it. It is built by the team, for the team, one feature cycle at a time. And it compounds.

I would strongly recommend starting this week. Write the docs, define the workflow, template the prompts. The teams that structure their AI use deliberately now will be unreachable in a year. The ones that wait will still be vibe coding demos.

Lessons Learned : Setting up an MCP registry for Github Copilot to enable enterprise governance

2026-01-14T02:07:00.000-08:00

While GitHub Copilot significantly boosts developer productivity, its ability to integrate with external tools through the Model Context Protocol (MCP) necessitates a robust governance framework to prevent unvetted data access. As per the official documentation, an internal MCP registry that acts as a centralized gateway, replacing fragmented local configurations with a single source of truth for approved servers. This allows us to curate a catalog of vetted tools, ensuring the security standards across the organization.

We selected the Playwright MCP server as our flagship test case because it addressed a critical gap in our toolchain and was the most requested integration among our engineering teams. From a developer experience standpoint, this setup allows engineers to author, execute, and troubleshoot end-to-end tests directly within the GitHub Copilot chat, effectively transforming the IDE into a live automation environment. By allowing the AI to interact with a real browser instance through the governed registry, we have significantly minimized context-switching and accelerated the overall testing lifecycle.

So, we built the registry using community version . Its essentially 3 APIs as Endpoint and specification requirements.

A valid registry must support URL routing and follow the v0.1 MCP registry specification, including the following endpoints:

GET /v0.1/servers: Returns a list of all included MCP servers
GET /v0.1/servers/{serverName}/versions/latest: Returns the latest version of a specific server
GET /v0.1/servers/{serverName}/versions/{version}: Returns the details for a specific version of a server

After successfully deploying our internal registry and modifying the GitHub enterprise policies settings, we encountered a baffling roadblock: VS Code was picking up the registry configurations perfectly, yet it was hard-blocking every server in the toolchain. It was a classic 'dark hole' scenario that forced us to look deeper into the handshake between the IDE's security layer and our governance APIs.

As we were debugging the issue, I got curios about the error message in the IDE. I remembered that VS code itself is opensource and why don't I try to understand the inner workings of the validation. There we go, and we have some new information. So, While the Github documentation speaks about the matching of the name of the server name, the actual validation covers name, description and version. Also the schema of the response should be of appropriate version. AI was of much help to get this information easily out.

Also, I had to create a input schema to add the server in appropriate way. After much try including with Microsoft Copilot and Google Gemini, I got the working version as below,

{

  "$schema": "https://static.modelcontextprotocol.io/schemas/2025-10-17/server.schema.json",

  "name": "com.microsoft/playwright-mcp",
  "version": "0.0.54",

  "title": "Playwright MCP",
  "description": "Browser automation using Playwright with structured accessibility snapshots.",

  "websiteUrl": "https://github.com/microsoft/playwright-mcp",
  "repository": {
    "url": "https://github.com/microsoft/playwright-mcp",
    "source": "github"
  },

  "icons": [
    {
      "src": "https://raw.githubusercontent.com/microsoft/playwright-mcp/main/assets/icon.png",
      "type": "image/png",
      "size": 128
    }
  ],

  "packages": [
    {
      "registryType": "npm",
      "identifier": "@playwright/mcp",
      "version": "0.0.54",

      "transport": { "type": "stdio" },

      "packageArguments": [],
      "environmentVariables": []
    }
  ]
}

Most important thing is, we should install our MCP Server to be from the IDE using the "packages" tag so that all the validation parameters are correct. With this all the issues were resolved and we could use Playwright MCP Server in VS Code. Also it blocked others servers demonstrating the validity of registry settings. I have also added a pull request to improve the documentation on Github docs.

When AI learns from what you wrote - An example in the context of AI Web Scraping debate

2025-08-09T22:18:00.000-07:00

The use of content created by others to train AI models has been a significant debate, on topics such as intellectual property rights, fair compensation, and the very nature of creativity. Proponents of AI training argue that it is a "transformative" fair use, akin to how a human learns by consuming and being inspired by a vast array of existing works. They contend that AI systems are not simply copying and pasting content, but rather analyzing patterns and relationships within massive datasets to generate new, original outputs.

On the other hand, critics, particularly content creators, artists, and writers, argue that training AI on their work without permission or compensation constitutes a form of theft. They believe that AI companies are profiting from their labor, and that AI-generated content can directly compete with and devalue the original human-made creations, threatening their livelihoods.

Recently, there is a online battle when Cloudflare accused Perplexity of systematically ignoring website blocks and masking its identity to scrape data from sites, as reported in this article. So, I decided to do a little experiment on some content I created, to see how AI reads and uses it.

I have picked my top answer to a stackoverflow questions, and asked the same question to Perplexity.

As expected, it gives the answer and provides a reference to my answer in SO. If it were google, user would have visited the site to know but here answer is available right there, with even better explanations.

The question is whether the assistance AI provides, even for a simple task like writing this blog post, justifies the lack of credit given to the original creators whose content trained the AI.

An experiment with Vibe coding on replit - where it works and where it fails

2025-04-27T04:11:00.000-07:00

AI Coding Agents

AI coding agents are advanced tools designed to assist developers in writing, debugging, and optimizing code. These agents leverage artificial intelligence and machine learning models to understand natural language commands, generate code snippets, and even handle full-stack development tasks.

Vibe Coding
Vibe coding is a programming technique where developers leverage AI tools to generate code based on natural language prompts, focusing on the desired outcome rather than the technical implementation. Instead of writing code, you describe what you want your app to do, and AI tools handle the technical aspects, including coding.

Replit

Replit is an online Integrated Development Environment (IDE) that allows users to write, run, and deploy code directly from their browser. It supports over 50 programming languages, including Python, JavaScript, and C++

Replit is designed for accessibility and ease of use, making it ideal for beginners and experienced developers alike.

Key features of Replit include:

Real-time collaboration: Multiple users can work on the same project simultaneously.
Built-in hosting and deployment: Users can deploy their applications directly from the platform.
AI-powered assistance: The Ghostwriter feature provides code suggestions, autocompletion, and explanations
No setup required: Everything runs in the cloud, eliminating the need for local installations
Replit is particularly popular for rapid prototyping, educational purposes, and collaborative coding projects.

Attempt to build an application on Replit

My objectives were, to create a fairly complex application, front-end, API Integrations, and Backend with DB.

So, this was the initial prompt,

A web-based Engineering Management Dashboard that aggregates team activities from GitHub and JIRA, providing consolidated insights for engineering managers to track their team's progress.
Core Features:
Team management with ability to create teams and add/remove developers
Integration with GitHub API to track code contributions (commits, pull requests, reviews)
Integration with JIRA API to monitor work item status and updates
Chronological activity feed for each developer combining both GitHub and JIRA updates with date filtering
Team dashboard showing consolidated metrics and individual developer summary screens
Visual References:
Inspired by GitHub's activity feed and JIRA's project dashboards, combining clean data presentation with actionable insights.

My Observations:

Excellent ability to create an initial prototype. It knows so much about the domain, to understood the context and did lot of customization by itself. For ex. Created widgets, used native Github and JIRA terminologies, . I liked the communication back to the user. You will be blown away by the output of the first version!

An application build plan was discussed and asked for approval. Code fairly well organized and easy to understand.

The choice of frameworks, components and node modules looks very new and modern.

Replit created the DB schema as below, It's very decent for the purpose.

I was very impressed by the creation of persistence layer, together with insertion of the seed data.

It would have taken around 20 hours for a developer to build this app, replit did it in 20 mins!.

Adding the first feature

I asked for this feature,

Ability to add team members in the team view with below attributes - Name, Email, GitId, Jira Id

Agent performed below task, and displayed below message. Again impressed by the communication.

I'm analyzing our current progress on implementing the "Add Team Member" feature.
✓ Created add-team-member-dialog component

✓ Created add-team-member-form component
✓ Created a dedicated add-developer page
✓ Updated the team page with an "Add Developer" button
✓ Added route for the add-developer page

But I could see that all the files were updated, but the feature was not working. I looked at the code and realized that it was not getting triggered. So, I manually triggered by custom url, the page appeared.

Next, I wanted to update the GitHub and JIRA API keys to test if they are working. I found it very difficult to navigate the flow and test the changes. As I was trying multiple attempt to debug the issues, I realized that I need to move to the "Developer" leading mode. Agent sounded confused for many questions.

At this stage, I could not continue on the replit agent, I took out the code to my local to continue with developer lead way (AI as assistant).

Conclusions

The ease with which non-developers can customize applications generated by AI coding tools varies depending on the complexity of the desired modifications and the specific platform being used. Non-developers should approach AI coding for prototyping with realistic expectations, recognizing that while these tools are powerful, they are not a substitute for software development expertise, especially when dealing with complex applications or aiming for production-ready code.

Agentic AI tools (Vibe Coding) are useful for non-developers to to create quick prototypes. It works best with UI heavy applications built with HTML, CSS, Java Script and Node. How ever further customization of the Agent build application will not be smooth sailing. There are rapid development in capability, but AI Agents are not reached a stage were one can create a production grade full-stack application without the involvement of expert developers.

Building Evolutionary Architectures - Book review

2020-03-21T08:31:00.000-07:00

In this blog, I want to speak about the book Building Evolutionary Architectures by Neal Ford, Rebecca Parsons, and Patrick Kua. I have attended Neal's conference talk on this topic and heard from many other speakers about the fitness functions. That’s the reason I wanted to read and understand the concepts mentioned in the book.

As the title implies, the book talks about building evolutionary architecture. The question that the book trying to solve is, how do we make sure, our software architecture stays intact with the changing requirements? How do we build the system which can adapt to future needs or how do we know the decision that we are taking is not impacting the architecture of the system?

The book speaks about fitness functions, to solve this concern. An architectural fitness function provides an objective integrity assessment of some architectural characteristics. So, in a system there may be many characteristics that we want to measure, so you would write separate fitness functions for each of them. In the book, a fitness function is not defined in a concrete way but rather in an abstract form of graphs, tests or any other methods by which we know that our system is doing good with the change. This means you would still need to use your intellect not only to write the fitness function but also to make sense of them.

For me, the best thing about the book is, it provides software architects with a rich vocabulary to communicate their intentions behind their choices to a variety of audiences, including business stakeholders, teams, and engineering leadership. The book also gives you a survey of various architectural approaches. It also talks about some of the practical ideas on how do we implement evolutionary architectures I particularly like the focus on the organizational factors and how does it apply to the software architectures.

In conclusion, I would recommend this book to any software architect. Use it as your communication guide, use it to improve your vocabulary, use it to get a sense of what is happening across the industry, so that you could choose what best for your situation.

4 ways to contribute to the community for a software developer

2019-12-22T03:30:00.000-08:00

If you are a software professional and looking for something new to start here are the 4 things to try for!

1. Attend a community event or user group gettogether or local meetup

2. Answer questions at the stackoverflow or contribute to support forums

3. Share your experience via blog or twitter or other forums with the community.

4. Contribute to opensource

When to stay with modular monoliths over microservices

2019-03-15T08:23:00.004-07:00

We have seen the developments in the microservices architecture maturing, where by more and more people are trying to evaluate the benefits before jumping on to the unknown trajectory.

In the talk titled When to stay with modular monoliths over microservices at Oracle Code, Bangalore, I tried to discuss these points. You can view the slides below.

When to stay with modular monoliths over microservices from Manu Pk

According to me, Over simplified version of decision tree come down to two criteria's, Business Context & Relative Scaling. I tried to explore the same in my presentation. As Martin Fowler puts it, you shouldn't start a new project with microservices, even if you're sure your application will be big enough to make it worthwhile.

Here is a link to the YouTube Recording of the session. Let me know what you think about these topics.

Practical communication strategies for software architects

2018-11-03T09:45:00.000-07:00

Here is a video recording of my session titled Practical communication strategies for software architects on Bangalore software architect meetup.

The session covers communication ideas for various stages and to different stakeholders in a project scenario.

Practical communication strategies for software architects from Manu Pk

Have a look at the video recording of the session

An approach to help developers write meaningful tests

2016-08-01T03:32:00.000-07:00

Over the last few years we have been adding unit tests to our existing product to improve its internal quality. During this period we always had the challenge of choosing unit-vs-Integration tests. I would like to mention some of the approaches we have applied to improve the quality of existing system.

At its core, unit testing is about testing a single component at a time by isolating its dependencies. The classical Unit tests have these properties "Fast, Independent, Repeatable, Self-Validating,Timely". Typically in java a method is considered as a unit. So traditional (and most common) approach is to test the single method of a class separated from all its dependencies.

Interestingly there is no hard-core definition of "what makes a unit". Many times a combination of methods which spread across multiple classes can form a single behavior. So in this context the behavior should be considered as a unit. I have seen people breaking these units and writing multiple tests for the sake of testing a single method. If the intermediate results are not significant this will only increase the complexity of the system. The best way to test a system is to test with its dependencies where ever we can accommodate them. Hence we should try to use the actual implementation and not mocks. Uncle bob puts this point very well, "Mock across architecturally significant boundaries, but not within those boundaries." in his article.

If the software is build with TDD approach it might not be a challenge to isolate dependencies or adding a test for your next feature. But not all software's built like these. Unfortunately we have systems where there are only few or none of the test are written. When working with these systems we can make use of the above principle and use tests at different levels. Terry Yin provides an excellent graphics (which is show below) in his presentation titled Misconceptions of unit testing. This shows how different tests can add values and what are its drawbacks.

Many of our projects uses Java and Spring framework. We have used springs @RunWith and SpringJUnit4ClassRunner to create AppLevel Tests which gives you the objects with all its dependencies initiated. You could selectively mock certain dependencies if you would like to isolate them. This sets a nice platform to write unit tests with multiple collaborating objects. we call them App level tests. These are still fast running test with no external dependencies. A different term was chosen to differentiate itself from the classical unit test. We also had Integration test which would connect with external systems. So, the overall picture of developer tests can be summarized as below,

Tests	Naming convention	Runs at	When to use	Exec Time
Unit Test	Ends with Test	Every build	Rule based implementations where the logic can be tested in isolation	Few Milliseconds
App Level Tests	Ends with TestApp	Every build / Nightly builds (Teams choice)	Tests the service layers in connection with others. Frees you from creation of mock objects. Application context is loaded in the tests.	Few Seconds
Integration Test	Ends with TestIntg	Runs on demand when a special profile is used in build.	All the above + Use when you need to connect to external points like DB, web services etc..	Depends on the integration points.
Manually Running Tests	Ends withTestIntgManual	Manually running tests, Used debugging a specific problem locally	All the above - Can't be automated.	Depends on the integration points.

This approach gives the developers choose the right level of abstractions to test and helps in optimizing their time. Nowadays my default choice is App Level tests and I go to unit tests if I have a complicated logic to implement.

Do you need microservices architecture?

2016-04-18T01:55:00.002-07:00

Over the last few years there has been lot of attention on microservices. After the initial "hype" we saw that what problems it solves and what it can not. I have tried to cover what are microservices and where it can be useful and where it is not. I want to share the guidelines which can be used to choose between a monolith and microservices.

I feel that one must answer the below questions before they choose a microservices architecture and it will be beneficial to you if the answer to these questions are "YES".

1. Does your services represents different business cases/domains..?

2. Does the services needs to be deployed and managed independently..?

3. Does different parts of the application has different scaling/Technology needs..?

A modular monolith can be transformed to a set of microservices in case the need arises. So, we should start with monolith when we are not sure about the future.

Do you need microservices architecture? from Manu Pk

I have spoken couple of times about the microdervices. In April 2016, I spoke at the Bangalore Software Architects Meetup on the topic "Do you need microservices architecture?". Later in 2017 April there was a brief introductory talk titled, Introduction to Microservices at the Microservices and Serverless event. I hope this post helps to complement the slides.

Introduction to Microservices from Manu Pk

Application Security for Java Developers

2016-01-24T22:23:00.000-08:00

Security is a top priority item on everyone's checklist nowadays. In this post, I will introduce you to useful reference material that can help you get started with securing applications. I want to focus more on web applications built with Java related technologies.

1. Authentication and Authorization

When it comes to security the most fundamental concepts are Authentication and Authorization. Unless you have a strong reason you should be following a widely accepted framework for this purpose. We have Java EE Authentication and Spring Security to help us out in this context. I have worked with spring security in the past and it can be customized to suite your specific needs.

2. Security in the Web Layer

In our application stack the web layer is most vulnarable to attacks. We have may established standard practices and detection mechanisms to minimize these risks. OWASP Top 10 list is a must have check point for security checks. The Open Web Application Security Project (OWASP) mission is to make software security visible, so that individuals and organizations worldwide can make informed decisions about true software security risks.

3. API Security

With the rise of mobile applications and stronger browsers expressing functionalities using the API is more popular day by day. We need to follow the same security practices for the web layer. All the API requests should be authenticated and we should use the principle of least privilege. I found the presentation from Greg Patton in the AppSec EU15 titled The API Assessment Primer is a great start for API security validations. Two major points focused in his talk are,
Do not expose any operations that are not needed
Do not expose any data that is not required
Which is in line with the basic security principle of giving least privilege by default.

To authenticate the services, we can create simple token-based API authentication mechanism based OAuth2 standards. If the services expose any sensitive data, it is better to use https so that man-in-the-middle attacks can be avoided.

4. Validating the User Input

Be aware that any JavaScript input validation performed on the client can be bypassed by an attacker that disables JavaScript or uses a Web Proxy. Ensure that any input validation performed on the client is also performed on the server. Go through the OWASP and WASC checklist to identify the potential validations you need to do in your application.

Other Useful Reference Materials

1. A Security Checklist for Web Application Design

2. Java EE Security

Identifying the skills gap for a Software Developer

2014-07-18T01:27:00.000-07:00

This April I had to create a Individual Development Plan (IDP) for me as part of the regular official procedures. One of step was to identify the gaps in you compared to the ideal position you want to reach. Thinking more in this line I have created the below table which contains ways to identify the specific areas of development for a developer.

Guide to reading the table:-

Ask yourself the questions in the column (D). If your answer is "Yes" to any of the questions then you needs to consider the action plans listed in column (E).

A. Sl No	B. Section	C	D. These things happens with You	E. Your Action Plans
1	Understanding what to do	What to do? (40%)	1. You have missed some of the requirements. 2. You hear others say "This feature was not supposed to work like this" 3. Your completed work gets re-opened during QA or User Testing.	-Improve your domain knowledge. -Ask more questions to your PO so that you can impove your understanding of the requirements. -Push for improved requirements documentation. -Spend more time in testing your features. -Listen to sprint demos to get the overview of all the new features added.
2	Knowledge of Frameworks, Design patterns, practices and principles	How to do it - Your skills to do it (20 - 30 %)	1. You don't know where to start with when you have to implement a new feature 2. You don't know if a similar functionality already exists in the application or not 3. You don't completely understand the frameworks in the application and how they are used	-Pair program with an experienced developer to learn how he approaches a problem. -Learn more about the frameworks used in your app. -Try creating sample applications using them. -Identify the patterns and principles used in your app and try to use them.
3	Problem solving, Analytical, Debugging skills	How to do it - Your Ability to do it (10 - 20 %)	1. You face difficulties when it comes to writing algorithms 2. You are weak in debugging and finding issues in the code	-See if you can apply some known patterns to solve the problem.
4	Communicating with your code	How well you did it? How easily somebody can understand how ? (15%)	1. Your code is not up to the standards or frequently ignores code quality. 2. You don't have enough code coverage 3. You can't write a quality documentation	-Use tools like sonar to asses the quality of your code. -Spend more time in refactoring and improving the code quality.
5	Communicating about your work	How well can you communicate about your work (5%)	1. Your don't follow the process in the team. 2. Your check-in comments are not useful. 3. Your team don't know what you are working on.	-Understand and adhere to the team policies. If you feel that there is somethings wrong, communicate and get it clarified.

This is the first draft of the version I have created. Try to apply this to you or your team and let me know your feedback. I hope I can expand each area by writing more in the future.

EXIN Cloud Computing Foundation Exam Review

2014-05-04T00:43:00.001-07:00

Recently I have attended a workshop on EXIN Cloud Computing Foundation course and cleared the certification exam. This post is bit about topics covered in the exam and my experience learning those topics.

The principles of Cloud Computing. This chapter deals with definitions, types of clouds (Public, Private and Hybrid) and cloud services (IAAS, PAAS, SAAS).
Most contents in the section are from the The NIST Definition of Cloud Computing paper. Other topics include The Evolution Toward Cloud Computing, Cloud Computing Architectures and Benefits and Limitations of Cloud Computing. The part about Virtualization and its role in the raise of Could computing was quite interesting for me.

Using the Cloud. This part is about accessing the cloud and mobility in the cloud.
This module covers the topics Overview of Accessing the Cloud, How Cloud Computing Can Support Business Processes and Service Providers Using the Cloud.

Security and Compliance. Is about the risks of cloud computing and the measures you can take
This module covers the paper Top Threats to Cloud Computing prepared by the Cloud Security Alliance under the Security Risks and Mitigating Measures title. Managing Identity and Privacy section deals with Triple-A authentication and various aspects of identity management.

Implementing and managing Cloud Computing. You learn about local cloud networks and how to support the use of cloud computing
This module includes the topics Building a Local Cloud Environment and Managing Cloud Services. There is a lot of focus on managing cloud services and related governance frameworks.

Evaluation of Cloud Computing. Examples of the subjects here are cost aspects, (dis)advantages and SLA’s.
This module speaks about the business case for cloud computing. For example cost implications to an organization evaluating the cloud services in terms of capex and opex. Forming the Service level requirements and agreements.

The text in italics are taken from the official exam page.
Written with StackEdit.

Integration Testing for Spring Applications with JNDI Connection Pools

2014-03-24T06:49:00.002-07:00

We all know we need to use connection pools where ever we connect to a database. All of the modern drivers using JDBC type 4 supports it. In this post we will have look at an overview of connection pooling in spring applications and how to deal with same context in a non JEE enviorements (like tests).

Most examples of connecting to database in spring is done using DriverManagerDataSource. If you don't read the documentation properly then you are going to miss a very important point.

NOTE: This class is not an actual connection pool; it does not actually pool Connections. It just serves as simple replacement for a full-blown connection pool, implementing the same standard interface, but creating new Connections on every call.

Useful for test or standalone environments outside of a J2EE container, either as a DataSource bean in a corresponding ApplicationContext or in conjunction with a simple JNDI environment. Pool-assuming Connection.close() calls will simply close the Connection, so any DataSource-aware persistence code should work.

Yes, by default the spring applications does not use pooled connections. There are two ways to implement the connection pooling. Depending on who is managing the pool. If you are running in a JEE environment, then it is prefered use the container for it. In a non-JEE setup there are libraries which will help the application to manage the connection pools. Lets discuss them in bit detail below.

1. Server (Container) managed connection pool (Using JNDI)

When the application connects to the database server, establishing the physical actual connection takes much more than the execution of the scripts. Connection pooling is a technique that was pioneered by database vendors to allow multiple clients to share a cached set of connection objects that provide access to a database resource. The JavaWorld article gives a good overview about this.

In a J2EE container, it is recommended to use a JNDI DataSource provided by the container. Such a DataSource can be exposed as a DataSource bean in a Spring ApplicationContext via JndiObjectFactoryBean, for seamless switching to and from a local DataSource bean like this class.

The below articles helped me in setting up the data source in JBoss AS.

1. DebaJava Post

2. JBoss Installation Guide

3. JBoss Wiki

Next step is to use these connections created by the server from the application. As mentioned in the documentation you can use the JndiObjectFactoryBean for this. It is as simple as below

If you want to write any tests using springs "SpringJUnit4ClassRunner" it can't load the context becuase the JNDI resource will not be available.

For tests, you can then either set up a mock JNDI environment through Spring's SimpleNamingContextBuilder, or switch the bean definition to a local DataSource (which is simpler and thus recommended).

As I was looking for a good solutions to this problem (I did not want a separate context for tests) this SO answer helped me. It sort of uses the various tips given in the Javadoc to good effect. The issue with the above solution is the repetition of code to create the JNDI connections. I have solved it using a customized runner SpringWithJNDIRunner. This class adds the JNDI capabilities to the SpringJUnit4ClassRunner. It reads the data source from "test-datasource.xml" file in the class path and binds it to the JNDI resource with name "java:/my-ds". After the execution of this code the JNDI resource is available for the spring container to consume.

To use this runner you just need to use the annotation @RunWith(SpringWithJNDIRunner.class) in your test. This class extends SpringJUnit4ClassRunner beacuse a there can only be one class in the @RunWith annotation. The JNDI is created only once is a test cycle. This class provides a clean solution to the problem.

2. Application managed connection pool

If you need a "real" connection pool outside of a J2EE container, consider Apache's Jakarta Commons DBCP or C3P0. Commons DBCP's BasicDataSource and C3P0's ComboPooledDataSource are full connection pool beans, supporting the same basic properties as this class plus specific settings (such as minimal/maximal pool size etc).

Below user guides can help you configure this.

1. Spring Docs

2. C3P0 Userguide

3. DBCP Userguide

The below articles speaks about the general guidelines and best practices in configuring the connection pools.

1. SO question on Spring JDBC Connection pools

2. Connection pool max size in MS SQL Server 2008
3. How to decide the max number of connections
4. Monitoring the number of active connections in SQL Server 2008

Note:- All the text in italics are copied from the spring documentation of the DriverManagerDataSource.

TalkNotes - The story of SonarQube told to a DevOps Engineer

2014-03-01T08:13:00.003-08:00

This week I spoke at Bangalore DevOps meetup on the topic "The story of SonarQube told to a DevOps Engineer". I have started writing TalkNotes inspired from Martin Fowler.Unlike his detailed article my posts aims to help the audience better understand my slides. SonarQube is a open source code quality management platform. It was a 30 mins talk focused at the need, setup, CI Infrastructure and administration of the SonarQube to the DevOps community.

The story of SonarQube told to a DevOps Engineer from Manu Pk

I have started the talk with one of my favorite subject Technical Debt. We have also looked at some of the parameters which determines the quality are coding standards breach, duplication, lack of unit tests, bad distribution of complexity, Spaghetti Design etc... I have spoke about this in more detail at previous post. There are various existing tools that helps reduce the technical debt by improve the code quality. What was missing was a easier way of tracking these code rule violations. For examples I need to know how much debt was introduced or was cleaned up? As a developer how do you quantify improvement the which a particular code refactoring has brought to the team etc..

This is where Sonar come to your help. Sonar's rich feature set allows you to do these and more. Currently it can run the quality analysis on more than 20 languages including Java, C#, C/C++, PL/SQL, Javascript, PHP, Web, XML etc.... It stores the analysis results and the data is displayed through various dashboards. Further slides discusses the sonar platform overview and installation.

The below diagram shows the CI environment including SonarQube. The Hudson plugin for SonarQube can be configured by following the wiki.

Image Idea from this blog.

The best part of sonar is its documentation. This was the most comprehensive documentation I have read about any open source product. You just need their wiki page to get 99% of answers. Now that I have it configured hope to write more about it in the coming moths.

4 simple steps to migrate legacy projects from Ant to Maven

2014-02-23T00:47:00.000-08:00

For some time we were thinking about migrating our build to maven from ant. It happened last month and was actually simpler than what we have anticipated. From my experience, here is a brief about the steps we have followed. Our application is a is a enterprise web application build with multiple frameworks and technologies and is deployed as a single WAR.

1. Create maven project directory structure.

As told in the Maven user guide create the below directory structure. We have done it under a new folder for the project.

2. Move the files/folders keeping the SCM logs.

Even though the folder structure is new the source files will be old ones! We want to keep the SCM logs while moving them to new locations. Remember to commit the folders created in step 1 before you start moving your files. If you use SVN, see this user guide or SO question on how to do this. Move the java source, unit/integration test and configuration resources to appropriate folders.

3. Create the POM and add dependencies

Most critical part in the migration is adding the dependencies in the POM. Start by adding the dependencies for the frameworks used in your application. Make sure you are adding the right version of the jars. You can find the version of the jar by reading the MANIFEST.MF file inside the META-INF folder of the jar. This will help if the versions are missing from the file name.

Any third party jars can be added to the maven repository as told here. If you are using very old versions of jar files some of them may not be available in maven repository.Here you can either try upgrading to newer versions or prepare local install as told before. Once you have added all the dependencies try building the application. Watch out for any major issues.

4. Make sure you haven't changed much in the WAR

Maven is a build tool. This means your WAR should not change. So, in the last step we compare both versions and make sure they are the same. Make sure you are on top of all the differences. Also, compare the jar files generated by maven and your existing files, Sync them by

- Adding <exclusions> to remove the unwanted jars

- Add the dependencies for the missing jars

This can be a tiring tasks depending on the number of jars you have in your lib. But make sure that you have each one covered and know that why they exists in your app.

May be this is a late post, most applications might have already been migrated by now. Anyways, better late than never! According to many experts Gradle is also a good choice as a build tool for your new project.

Why is tomcat a Webserver and not an Application Server

2014-01-02T05:17:00.000-08:00

Many application developers do not focus much on the infrastructure on which their code runs. When it comes to web applications there are common confusions like what is the difference between webserver and applications server or when to go for a EAR vs WAR file deployment etc...

There are many good answers that differentiate between web servers and applications servers like this one. Most of the times the terms Web Server and Application server are used interchangeably. This article explains the working of a typical web server. Typically we get confused with the example of Tomcat Server (an example for a web server) having the capability to run the enterprise applications. So, tomcat is a web server or an application server? Let me tell you how I convinced my self regarding this.

Some time back I was struck with the question What's the difference between JPA and Hibernate on stack overflow. I did answer it, but one of the comment lead me to a more detailed understanding of the JavaEE spec and certified servers. If you can understand this then differentiating between the web server and application server is easy. During my investigations I got this article, which discusses the advantages of both.

A more detailed look in to the meaning JavaEE specification will throw some light in to our discussions. As we know specifications are set of rules. Simply put they contain the interface. Any JavaEE servers which needs to comply to spec needs to have the implementation of these interfaces. You can find the certified JavaEE servers list here. If you are deploying your enterprise applications (means you have JPA, EJB or some technology which is part of Java EE) to the a server which comply to JavaEE then the lib need not contain the API implementation jars. But these are needed if you are using a web server like tomcat for deployment.

For example, if you use JPA in your applications and deploying it to the Jboss AS 7, then you need any additional jars in the lib. But the same application you want to deploy to the tomcat server then you need to have additional jars to lib that implements the JPA spec may be eclipselink or Hibernate. This is what makes JBoss AS 7 an application server and tomcat a web server. Another key difference is that we can not deploy an EAR file to tomcat, it could only handle WAR files.

Does the View in database reduce the query performance?

2013-12-18T04:05:00.000-08:00

Not really!! Here is why...

Few days back, I was speaking about one of our newly created Filter API with server side pagination to my friend. In the implementation we were querying the view to get the data. Since the pagination was at the server side, at a time the query will return only 10 records. His question was since views internally going to use the tables for querying, will it end up being less performative? Will it create the result set of all data and then pick the top 10 items?

My initial reaction was to agree with question and confess we have an issue. But I was sure that this can not be the case. If this is so then we could have had severe issues. I am not much of a DB guru myself so went and did some research with others to figure out the story behind the scenes. By the way we use MS SQL Server 2008 and I am not sure if this descriptions applies to other database or not.

The question to ask was. We know that the views are used to simplify other queries or standardize access to data. but do we compromise on speed while doing so?

I got a related answer from SO. When an SQL statement references a nonindexed view, the parser and query optimizer analyze the source of both the SQL statement and the view and then resolve them into a single execution plan. There is not one plan for the SQL statement and a separate plan for the view. So it is fine to use the views for my above use case.

Further, I got to know about Indexed views from this discussions, which can further improve the performance. Microsoft has a nice documentation on Improving Performance with SQL Server 2005 Indexed Views.

A MindMap for Java Developer Interviews

2013-10-22T08:42:00.000-07:00

Over the years I have been a panelist in many of the interviews for Java Developers. I have previously written a post titled Top 7 tips for succeeding in a technical interview for software engineers which covers few of the general guidelines. In this post I will share a mind map containing general topics covered in a Java developer interview. I have prepared this as a general reference for myself to remember the pointers and to keep a common standard across the multiple interviews.

XMind gives a nice listing of the map. You can find the map here. Here is Image which you can download and use.

Finally here is a old fashioned tabbed content list which is easier to copy paste.

Java-Topics
OOPs
Encapsulation
Abstraction
Inheritance
Interface - Abstract Class
Casting
IS-A vs HAS-A Relationships
Aggregation vs Composition
Plymorphism
Method overloading vs Method Overloading
Compile time vs Runtime
Threads
Creating threads
Multitasking
Synchronization
Thread Transitions
Marker Interface
Serialization
Clonnable
Shallow copy vs Deep Copy
Collections
Map, List and Set
Equals - Hashcode
Legacy - Synchronized Classes
JVM
Stack vs Heap Memory
Garbage Collection
JRE, JVM, JDK
Class loaders
Exception
Checked Vs Unchecked Exceptions
Exception handling best practices
try, catch, finally, throw, throws
APIs
Files
String - StringBuffer - String Builder
Java IO
XML
SAX Based & DOM Based
JAXB - Java API for XML Binding
Access specifier
Access modifier
public
protected
deafult
private
final
static
synchronized
abstract
transient
volatile
Inner/Nested Classes
JavaEE Basics
Packaging the Applications
WAR
EAR
Basics
MVC
Servlets
Listeners
Lifecycle
JSPs
APIs
JPA
JAX-WS
SOAP, WSDL Webservices basics
Contract first vs
JAX-RS
RESTful and its advantages

JSF

This is a work in progress and I hope to refine it further. Let me know if you have any comments.

On the way to a blogger - My Journey with this blog so far

2013-10-09T07:54:00.001-07:00

TalkNotes - How can Agile help you in clearing the technical debt?

2013-09-21T22:25:00.001-07:00

This post is a write up to my talk titled "How can Agile help you in clearing the technical debt?" presented at the Scrum Bangalore Meetup 2013. It was a short talk for 30 Minutes, So thought of a write up to elaborate the ideas on the slides.

How can agile help you in clearing the technical debt from Manu Pk

Defining Technical Debt

As per Wikipedia, "Technical debt metaphor referring to the eventual consequences of poor or evolving software architecture and software development within a code base". Technical debt is most understood when it is compared with Cost of Change of software. More Debt means the cost of introducing a change in your system is more.

The term Technical Debt was coined by Ward Cunningham in the year 1992. He used the term to explain to his manager about a particular refactoring which he wanted to do. Since he was working in the financial industry it was easy to communicate in terms of debt. Ward explains more about the metaphor here. The term is immensely popular in communication of the importance of code refactoring to the business and stake holders. Like financial debt technical debt is some times unavoidable but we should take steps to minimize it.

As you can see in the above image when you have technical debt in your system you could either pay the interest or pay back the principle. When we do regular stories we do bit of "extra" work than what ideally needed due to the technical debt. This is an example of the interest payment. It is important that we take some time and effort to pay back the principle by refactoring and cleaning up of the code.

Technical Debt in Agile Context

The below principle from the list of agile principles was the best one I could find which refers to clearing technical debt "Continuous attention to technical excellence and good design enhances agility". In agile projects working software has a lot of emphasis but this in no way implies that the intrinsic value of software can be compromised. As we push for more and more features it is important that we take time to look back at the health of the application. So, make make sure that your agile projects do not become Fragile as it grows.

There are lots of ways to deal with technical debt. I want to talk about 3 simple steps that we have tried in our projects.

1. Tools - Your must have defense

One of the major improvement with the agile development practices is the widespread usage of continuous integration (CI). Your CI environment should be configured to measure the health of the application. It should include the report from static code analysis and test coverage applications. SonarQube is a god platform where you can create a dashboard and see results from the tools mentioned above. It also has a plugin to view the technical debt. Apart from the tools you should be using good programming practices and guidelines.

2. Negotiating with Product Owner

The second techniques that we have tried to minimize technical debt is to negotiate with the product owner and take him on board in the technical improvements that we wanted to do. You have to Use the opportunity of Change requests to clean up the functionality. We had a module in our application with lots of bugs. There was a major change request which was planned on that module. After much discussions we identified that it would be much easier to rebuild the module than fixing the issues. Once we rebuild the change request was 75% easier to implement compared to the old code.

This means that we have to evaluate the possibility of refactoring with every opportunity and go for it when the time is right. It is the professional responsibility of a developer to communicate dept of technical debt in the application and we should take all steps possible to minimize it.

We can also have technical Stories in Release planning. Contrary to a user story, a senior developer should describe the success criteria of the story. This is another case where you need to get your product owner on board.

3. Trying out new Stuff

In our projects most things can be done multiple ways and we have to choose one of them. There are also situations where we are not sure about the complexity of the task performed and story point estimation is difficult. We can use spike to deal with such situations. A spike is an experiment that allows developers to learn just enough about something unknown in a user story, e.g. a new technology, to be able to estimate that user story. A spike must be time-boxed. This defines the maximum time that will be spent learning and fixes the estimate for the spike.

A daemon thread is a low priority thread that is running on the operating systems. We have used the same strategy to do a major change in our application. It was upgrading our frameworks (Spring, Hibernate and Tapestry) to their latest versions. Since it is a time consuming activity, we do not want to stop everything else and work on it. So, we choose to keep doing it as a low priority tasks and it took us 10 sprints to complete. Since the team continuously delivered features during this period it was very easy to get the buy in from product owner.

Useful Links:-

1. Martin Fowler's Talk

2. Martin Fowler on Technical Debt

3. Agile Architecture Emerges or Does It

4. Using Spikes

5. Steve McConnell’s Talk

6. http://www.ontechnicaldebt.com

7. Ward Explains Debt Metaphor

Note:- There are many ways to define and minimize technical debt. I have discussed few ways that we have successfully tried. You can go through the above links to find more information and choose the best methods that suits you.

GOF Design Patterns Notes on Trello!

2013-05-23T07:00:00.000-07:00

Recently I was reviewing Gang of Four Design Patterns and created a Trello board for the same. Well, Here is how it look like!

Hope it could be useful for others as well.

NoSQL Distilled : Book Review

2013-04-29T07:24:00.001-07:00

I have finished reading the book titled "NoSQL Distilled: A Brief Guide to the Emerging World of Polyglot Persistence" written by Martin Fowler and Pramod J. Sadalage recently and thought of a writing short review. First of all it short book and you can run through the pages fast. As the title says, the book is really an introduction of No SQL databases.

It was..

The number one reason for the use of NoSQL databases is not performance and should be our use case. This means if our data model does not fit well in to the relational model, rather than twisting the data we can choose alternate data stores. This is absolutely critical if you want NoSQL to be used in most of the mid sized enterprise applications. Also companies will start to believe that they should consider NoSQL it even if they are not Google or Amazon.

Normally the technology books written by its creators of it will only focus on the advantages of using it. It can be quite an issue if you do not do proper research before choosing your application architecture. Since both Martin and Pramod do not belong to this category, they does a fair evaluation of various choices available to us. The section which says where to use it and where not to use is particularly impressive on that front.

Even though the book is selling you a new idea of NoSQL databases, it does not plays down the significance and importance of existing RDBMS. The book also brings light to some of the fundamental trade offs in the world of data stores like consistency, availability and performance.

and is also..

When I was reading the book I thought that Polygot persistence is about using multiple data stores in same application depending on the use cases. But authors also seems to suggest that applications should wrap around the data stores with services ( like web services or REST). If we need to do this ideally we could only have one data store accessed by one service. Which would conflict the first statement. I am not sure about the reverence this question but I would love to have read more about using the different data store in the same application and its pros and cons.

I mean to say,

This is a very great book if you are looking get in to wild jungle of NoSQL databases. The authors does a great job in giving the brief introduction and making sure the basics are covered.

Package your classes by Feature and not by Layers

2013-04-21T00:43:00.000-07:00

Most of the enterprise Java applications share some similarities in their design. Mostly the packaging of these applications are driven by the framework used by them like Spring, EJBs or Hibernate etc. Alternatively you can group you packages by features. Like any other item regarding modeling this is not free from any issues. Lets discuss some trade-off and how get around them. In this post we will discuss the pros and cons of both approaches against common usage scenarios.

Package By Layer (PBL)

This is a the first thing that developers do when the create an enterprise application in to split it to number of layers like DAO, SERVICE, VIEW etc.. This gives nice separation of code when we use different frameworks at different layers. For example if I were to use Hibernate, Spring and JSF, then I will have all my Hibernate dependent code in the DAO layer and JSF related code in the VIEW Layer. This is sort of good in case I need to migrate to a new framework only at the view layer or DAO Layer.

Package By Feature (PBF)

Package-by-feature uses packages to reflect the feature set. It places all items related to a single feature (and only that feature) into a single directory/package. This results in packages with high cohesion and high modularity, and with minimal coupling between packages. Items that work closely together are placed next to each other. They aren't spread out all over the application.

This also increases coherence as a large percentage of a the dependencies of a class are located close to that class.

Comparing the approaches

Let me compare both the approaches in below dimensions.

1. Adding a new Feature.

In case of PBL code has to be added to VIEW, SERVICE and DAO Layers and it can be tedious. PBF solves this problem by grouping all the code related to same feature in to a single directory.

2. Changing a framework used.

As discussed already PBL makes it easier to change a framework as all the related code are kept at same place. Here we know exactly the scope of the change and its impact. In case of PBF we need to dig into all the feature set to see the framework related classes.

If you choose to migrate the framework module by module, then it could be argued that PBF is better than PBL.

3. Code Navigation.

As developers needs to work on the features most of the time using PBF is easier for code navigation. When you know exactly know what has to be done its not much advantage.

4. Keeping the Common Code.

Every application will have some components which will be reused across the features ie, the features are not always exclusive. In such case if we package all the features separately the interactions between them can be quite messy. As a general principle we need to reduce such interactions and increase the cohesion inside the package. We can get around the situation be adding such entities to a common package. This approach is used in many projects including Hibernate.

Conclusion

Most application will have 4 types of classes. ie,

1. Domain Objects

2. Business Services

3. Data Retrieval Logic

4. Data Representation Logic

If we use PBF it gives us a good structural representation but does not give any functional representation. We need our architecture to resemble the problem domain. So its better to use Package By Feature style. We could internally classify the classes by using naming conventions like MyFeatureDAO or MyFeatureService . This way we could communicate the intend of the classes.

I have found some discussions on this topic and hope that might also help you choose.

1. Java Practices Article

2. Lessons to Learn from the Hibernate Core Implementation

3. How Changing Java Package Names Transformed my System Architecture

4. Presentation By Kostis Kapelonis

5. The Principles of OOD By Uncle Bob

6. Article says its Better to Use Package By Layer

7. Question on SO Is package by feature approach good?

A 3 Step Guide to Getting Started with NoSQL

2013-03-20T00:28:00.000-07:00

I have been looking in to NoSQL databases for few months and would like to share my experience with it. This is a post might help you if you indent to start learning about the NoSQL Databases. I would try to link the resources which I found useful here.

Step 1: What is NoSQL?

NoSQL DEFINITION: Next Generation Databases mostly addressing some of the points: being non-relational, distributed, open-source and horizontally scalable. The original intention has been modern web-scale databases. The movement began early 2009 and is growing rapidly. Often more characteristics apply such as: schema-free, easy replication support, simple API, eventually consistent / BASE (not ACID), a huge amount of data and more. So the misleading term "nosql" (the community now translates it mostly with "not only sql").

As seen on NoSQL-Database.org.

Martin Flower's NoSQL page is a good starting point. His below talk on Goto Conference explains the need and structure of NoSQL data stores. Martin and Pramod has written a book titled "NoSQL Distilled: A Brief Guide to the Emerging World of Polyglot Persistence" and is a good read. It summarizes his talks and other blog post into a book. Martin has been an influential speaker on this topic and has written number of articles on this. I have read and seen many introductions but his work helped me to get things in to my head.

If you likes to view the slides, then the below presentation by Tobias Lindaaker on slideshare might inspire you. He gives similar ideas.

NOSQL for Dummies from Tobias Lindaaker

MongoDB has an online course MongoDB for Java Developers which is really useful if you are interested in trying out things.

Step 2: How and for what are NoSQL used in Real world?

Once you have some idea, try to find the usage patterns. The above presentations will give lot of information on how these systems are used. You could go through the below links, which explains how specific business problems are solved using NoSQL. This is important because we could easily relate the case studies and get more insights into the capabilities of these systems.

1. MongoDB Customers page.
2. Powerd By Haddop
3. Neo4J Customers Page

Step 3 : Find Usage Patterns that you could work on!

Once you have reached this point, you should try and implement the concepts. Look back at the application that you are working on and see if there is a need for an alternative data store. Do you store Product recommendations? Do you have issues with heterogeneous data? Can your application compromise ACID model for scalability? Do you store XML files or Images on you relational DB? These are some of the questions that you could ask. This way you could determine if there is a serious need of a investigation for a alternative persistence mechanisms. This is in no way means removing the RDBMS completely but moving to a polygot structure of data stores.

If there is no opportunity to try out these concepts in your work, you could create your own test projects and implement them. This way you would encounter problems and will learn from them.