Spreading Knol

Give Your LLM Hands: Turn Browser Automation Into MCP Tools

2026-07-01T01:54:43.484-07:00

LLMs are brilliant at deciding what to do and useless at actually doing it. Automation tools are the opposite — they execute flawlessly but can't reason about anything. The obvious move is to bolt the two together: let the model plan, and let deterministic automation carry out each step. The question is how you connect them cleanly.

The answer that's become a standard is MCP — the Model Context Protocol. It's a common language for exposing "tools" to a model. Wrap your automation actions as MCP tools, point a reasoning model at them, and the model can drive real software. This is the exact pattern under MeBot's agentic RPA. Let's build a small version of it.

Step 1 — Install the pieces

pip install "mcp[cli]" playwright
playwright install chromium

Step 2 — Stand up an MCP server

FastMCP (bundled with the MCP SDK) turns plain Python functions into tools. Any function you decorate becomes something a model can call — the docstring and type hints are the tool's description, so the model knows when to use it.

from mcp.server.fastmcp import FastMCP
from playwright.sync_api import sync_playwright

mcp = FastMCP("mebot-browser")

_pw = sync_playwright().start()
browser = _pw.chromium.launch(headless=False)
page = browser.new_page()

Step 3 — Expose automation actions as tools

Here's the key design decision: the tools themselves stay strictly deterministic. Each one does exactly one concrete thing — no cleverness, no improvising. You never want an LLM "creatively interpreting" a click on a banking screen. The intelligence lives in the model's choice of which tool to call; the execution is boringly reliable.

@mcp.tool()
def open_url(url: str) -> str:
    """Navigate the browser to a URL."""
    page.goto(url)
    return f"Opened {url}"

@mcp.tool()
def click(text: str) -> str:
    """Click the visible element matching the given text."""
    page.get_by_text(text, exact=False).first.click()
    return f"Clicked '{text}'"

@mcp.tool()
def type_text(label: str, value: str) -> str:
    """Type text into the field with the given label."""
    page.get_by_label(label).fill(value)
    return f"Typed into '{label}'"

@mcp.tool()
def read_screen() -> str:
    """Return the visible text on the current page."""
    return page.inner_text("body")

if __name__ == "__main__":
    mcp.run()

Step 4 — Let a reasoning model drive

Now connect a model as an MCP client. It requests the tool list, then works toward a goal by calling tools and reading results — a loop of observe → decide → act. Give it an objective like:

Goal: "Log into the portal and download this month's invoice."

The model plans and calls tools in sequence:
  open_url("https://portal.example.com")
  read_screen()                      -> sees a login form
  type_text("Username", "...")
  type_text("Password", "...")
  click("Sign in")
  read_screen()                      -> sees the dashboard
  click("Invoices")
  ...

Crucially, the model reads the screen between steps. When a button moves or a field is renamed, it adapts — because it's reasoning about what's actually there, not replaying recorded coordinates. That's the difference between a bot that breaks on the first UI change and one that copes.

Step 5 — Guardrails

Autonomy without brakes is a liability, so real deployments add two things: an approvals gate for irreversible or high-risk actions (payments, deletions, submissions) that pauses for a human, and vision — OCR and UI detection — for legacy screens where there's no clean text or label to target. Together they make the agent safe to point at production systems, including the old Windows apps that classic RPA struggles with most.

Deterministic execution, reasoning on top

That's the whole philosophy in one line: keep the doing precise, put the thinking in a layer above it. Wrapping actions as MCP tools is what lets a model orchestrate real work without you hard-coding every branch — and it's exactly how MeBot turns brittle scripts into automation that adapts.

If you'd rather deploy this than assemble it — with the vision layer, approvals inbox, and legacy-app support already built — that's what we ship → takemebot.com

Embed Analytics Into Your Product in an Afternoon (Without Building a BI Team)

2026-07-01T01:53:12.323-07:00

Sooner or later, every SaaS product hears the same request from customers: "Can we get reporting on our own data, inside the app?" And every engineering team gives the same sigh, because building analytics from scratch — a query layer, a charting engine, per-tenant security, a dashboard builder — is months of work that has nothing to do with your actual product.

That's the whole reason Datalytics ships as an embeddable SDK. Instead of building BI, you drop a component into your app and pass it a secure token. Here's the full integration, end to end.

Step 1 — Drop in the component

Datalytics dashboards are web components (built on Angular Elements), which means they run in any frontend — React, Vue, Angular, or plain HTML. Add the script and place the element:

<script src="https://cdn.getdatalytics.com/embed.js"></script>

<datalytics-dashboard
    dashboard-id="sales-overview"
    theme="light">
</datalytics-dashboard>

At this point the component exists but shows nothing — because it doesn't yet know who's asking or what they're allowed to see. That's the important part.

Step 2 — Mint a signed token on your backend

Embedded analytics lives or dies on security: customer A must never see customer B's data. The clean way to enforce this is a two-token handshake. Token one is an identity assertion your own backend signs — you already know who the logged-in user is and which tenant they belong to, so you vouch for them.

Your server signs a short-lived RS256 JWT with your private key. Datalytics verifies it with your public key — so the browser never holds any long-lived secret:

// Node / NestJS backend
import jwt from 'jsonwebtoken';

function mintDatalyticsToken(user) {
  return jwt.sign(
    {
      sub: user.id,
      tenant: user.tenantId,          // scopes all queries to this tenant
      scope: ['dashboard:sales-overview'],
    },
    PRIVATE_KEY,                       // your RSA private key
    {
      algorithm: 'RS256',
      expiresIn: '10m',                // short-lived on purpose
      audience: 'datalytics',
      issuer: 'your-app',
    },
  );
}

Step 3 — The two-token exchange

Your frontend fetches that identity token and hands it to the component. Datalytics verifies the signature, reads the tenant and scope, and exchanges it for its own short-lived session token that governs the live dashboard connection. Two tokens, two jobs: yours proves identity, theirs runs the session.

const el = document.querySelector('datalytics-dashboard');

// fetch the signed identity token from YOUR backend
const res = await fetch('/api/analytics-token');
const { token } = await res.json();

// hand it over — the component does the exchange internally
el.token = token;

The dashboard renders, and every query it runs is automatically filtered to that tenant. Row-level security is enforced by the signed tenant claim — not by anything editable in the browser.

Step 4 — Make it look like your app

An embedded dashboard should feel native, not bolted-on. Theme it with CSS custom properties so it inherits your product's look:

datalytics-dashboard {
  --dl-accent: #0e9384;
  --dl-font: 'Inter', sans-serif;
  --dl-radius: 10px;
}

What you just avoided

No query engine, no charting library, no dashboard builder, no per-tenant security model — and no BI team to maintain any of it. Your users get live, self-serve analytics inside your product; you shipped it in an afternoon and moved on with your roadmap.

If you've been putting off "in-app reporting" because building it is a project of its own, this is the shortcut → getdatalytics.com

How We Run Our Own LLMs on Our Own Hardware (and Why We Don't Just Call the Big APIs)

2026-07-01T01:49:55.028-07:00

When you need an LLM, the default move in 2026 is to grab an API key from one of the big providers and start calling. For a lot of what we build at Megam, that's the wrong default. Our automation reads client back-office data. Our telehealth platform touches patient records. Our conferencing tool handles private calls. None of that can be shipped off to someone else's cloud — and honestly, even where it could, the economics and the loss of control often don't make sense.

So we run our own inference. Here's how we stand it up, and where the line actually falls between self-hosting and calling an API.

Why self-host at all

Data residency & privacy. The data never leaves infrastructure we control. For regulated clients — healthcare, finance, GCC data-residency rules — this isn't a preference, it's the requirement.
Cost at volume. Per-token pricing is cheap for a demo and brutal at scale. A fixed box you've already paid for doesn't meter you by the request.
Control. The model doesn't change under you overnight, doesn't deprecate, doesn't rate-limit you mid-batch. You pin the version and it stays put.
Offline capability. Air-gapped and on-prem deployments simply can't reach a public API. Local is the only option.

Step 1 — Pick the box, size the model to the VRAM

The single constraint that matters is GPU memory. Rough working guide, using 4-bit quantized models:

~7-8B  model  ->  ~6 GB VRAM   (fits almost anything)
~14B   model  ->  ~10-12 GB
~27-32B model  ->  ~20-24 GB    (a 32 GB card handles this comfortably)
~70B   model  ->  ~40 GB+       (big card or multi-GPU)

A modern 24-32 GB card covers the vast majority of real workloads. You don't need frontier-scale hardware to get genuinely useful results.

Step 2 — Install Ollama and pull your models

Ollama is the simplest way to serve open models. One server can host several — chat, vision, and embeddings — and load them on demand.

# Linux install
curl -fsSL https://ollama.com/install.sh | sh

# Pull a few models for different jobs
ollama pull qwen2.5          # general chat / reasoning
ollama pull qwen2.5vl        # vision (images, screenshots, documents)
ollama pull nomic-embed-text # embeddings for RAG / search

Step 3 — Serve it on the network with an OpenAI-compatible API

By default Ollama listens only on localhost. To let your apps reach it, bind it to all interfaces:

# Expose Ollama on the LAN
export OLLAMA_HOST=0.0.0.0:11434
ollama serve

Important: Ollama has no built-in authentication, so never expose port 11434 to the open internet directly. Put Nginx in front of it for TLS and an API key, so only your apps get through:

server {
    listen 443 ssl http2;
    server_name ai.yourdomain.com;

    # ssl_certificate ... ssl_certificate_key ...;

    location /v1/ {
        # simple shared-secret gate
        if ($http_authorization != "Bearer YOUR_SECRET_KEY") {
            return 401;
        }
        proxy_pass http://127.0.0.1:11434;
        proxy_set_header Host $host;
        proxy_read_timeout 300s;   # long generations
    }
}

Step 4 — Call it from any app (it's just the OpenAI SDK)

Because Ollama speaks the OpenAI API format, your application code doesn't care that it's talking to your own box. You change the base_url and nothing else. Migrating from a public API — or back — is a one-line change.

from openai import OpenAI

client = OpenAI(
    base_url="https://ai.yourdomain.com/v1",
    api_key="YOUR_SECRET_KEY",
)

resp = client.chat.completions.create(
    model="qwen2.5",
    messages=[{"role": "user", "content": "Summarize this ticket in one line: ..."}],
)
print(resp.choices[0].message.content)

Step 5 — One server, many jobs

The same endpoint routes different tasks to different models — send chat to qwen2.5, screenshots and documents to qwen2.5vl, and use nomic-embed-text to build the vectors behind a RAG search. Ollama swaps models in and out of VRAM as calls come in, so a single machine covers a surprising amount of ground.

Where this powers our products

This isn't a lab experiment — it's the shared engine under everything we ship. The reasoning layer in MeBot (our agentic RPA) calls it to decide next actions. Datalytics uses it for natural-language analytics. Olivasal runs meeting summaries on it. Saffron, our telehealth platform, runs clinical models on it. One on-prem inference tier, many products — and no client data ever leaves the building.

The honest tradeoffs

Self-hosting isn't free. You own the ops — uptime, drivers, model updates. Open models are excellent now but a very hardest reasoning task may still favour a frontier API. And you'll think about concurrency and batching that a managed API hides from you. Our rule of thumb: self-host anything touching sensitive data or running at volume; reach for a frontier API only for the rare task that genuinely needs it. For most real work, your own hardware wins on cost, privacy, and control at the same time.

If you need AI running inside your own walls — because your data can't leave, or the API bill doesn't scale — that's precisely what we build for clients → megamtech.com

Build a Self-Hosted Meeting-Summary Pipeline (No Cloud, No Data Leaving Your Box)

2026-07-01T01:45:15.037-07:00

Every AI notetaker on the market has the same catch: to summarize your meeting, it uploads your audio to its cloud. For a regulated team — healthcare, finance, anyone under data-residency rules — that's a non-starter. Your private calls quietly become someone else's data.

This is exactly why we built the summary engine inside Olivasal to run entirely on infrastructure you control. In this post I'll show you the core of it: take a recorded meeting, produce a clean transcript, and generate a structured summary with action items — all on your own box, nothing leaving the network.

What you'll build

A three-stage pipeline: recording → transcript → AI summary. Stage 1 uses your meeting recording (from LiveKit egress, or any file). Stage 2 transcribes with faster-whisper. Stage 3 summarizes with a local Qwen model served by Ollama. No external API is ever called.

Prerequisites

# System: ffmpeg for audio, plus Python 3.10+
# A GPU makes transcription much faster, but CPU works too

pip install faster-whisper openai

# Ollama for the local LLM (https://ollama.com)
ollama pull qwen2.5

Step 1 — Get the audio

If you're running a conference stack, LiveKit's egress writes a recording when the call ends. Composite egress gives you one mixed file; per-track egress gives you one file per participant (we'll use that later for speaker labels). Either way, extract a clean 16 kHz mono WAV — the format Whisper likes best:

ffmpeg -i meeting.mp4 -ar 16000 -ac 1 -c:a pcm_s16le audio.wav

Step 2 — Transcribe locally with faster-whisper

faster-whisper is a reimplementation of Whisper that's several times quicker and lighter on memory. The vad_filter flag trims silence so you don't waste compute on dead air.

from faster_whisper import WhisperModel

# use device="cpu", compute_type="int8" if you have no GPU
model = WhisperModel("large-v3", device="cuda", compute_type="float16")

segments, info = model.transcribe(
    "audio.wav",
    beam_size=5,
    vad_filter=True,
)

transcript = " ".join(seg.text.strip() for seg in segments)
print(f"Detected language: {info.language}")
print(transcript)

That's your full transcript, produced without a single byte leaving the machine.

Step 3 — Summarize with a local LLM

Ollama exposes an OpenAI-compatible endpoint, so you can use the standard openai client and just point it at localhost. We ask for structured JSON so the output is easy to store and render.

from openai import OpenAI
import json

# api_key is required by the client but ignored by Ollama
client = OpenAI(base_url="http://localhost:11434/v1", api_key="ollama")

prompt = f"""You are a meeting assistant. From the transcript below, return JSON with:
- "summary": a 3-sentence overview
- "decisions": a list of key decisions made
- "action_items": a list of objects with "task" and "owner"

Transcript:
{transcript}"""

resp = client.chat.completions.create(
    model="qwen2.5",
    messages=[{"role": "user", "content": prompt}],
    response_format={"type": "json_object"},
    temperature=0.2,
)

notes = json.loads(resp.choices[0].message.content)
print(json.dumps(notes, indent=2))

You now have clean, structured meeting notes — summary, decisions, and owner-tagged action items — generated entirely on your own hardware.

Step 4 — Add speaker labels (optional)

This is where per-track egress earns its keep. Instead of one mixed file, you get a separate audio track per participant. Transcribe each track on its own, tag every segment with that participant's identity, then merge all segments back together in timestamp order. The result is a speaker-attributed transcript — "Priya: … / Arun: …" — which makes the LLM's action-item ownership far more accurate, because it can see who actually committed to what.

Why self-hosted matters here

The convenience of AI summaries usually comes at the cost of handing your conversations to a vendor. This pipeline gives you the convenience and keeps every stage — media, transcript, and model — inside your own walls. For regulated teams, that's not a nice-to-have; it's the whole requirement.

This is the engine behind Olivasal, our self-hosted video conferencing with built-in AI summaries — and the same core powers the clinical scribe in our telehealth platform, Saffron. If you'd rather have this running out of the box than wire it up yourself, that's exactly what we built → olivasal.com

Catching Insurance Claim Denials Before You Submit

2026-07-01T01:40:11.351-07:00

In revenue-cycle management, the denial is the most expensive message you'll ever receive. Every denied claim is money you already earned — care delivered, work done — now frozen behind rework, appeals, and a filing clock that's ticking down. Some of it comes back after weeks of effort. A painful share never comes back at all, and quietly gets written off.

What makes it worse is when most teams deal with denials: after they happen. The claim goes out, the payer sends it back, and only then does someone open a worklist and start the reactive scramble. It's the most expensive possible point to intervene — you're now paying staff to recover revenue you'd already booked.

Here's the thing we kept noticing across RCM work: denials aren't random. They cluster into a handful of predictable patterns — a missing or expired prior authorization, an eligibility lapse, a diagnosis-to-service code mismatch, a modifier issue, a timely-filing miss, a duplicate. And if a pattern is predictable, it's catchable — before the claim leaves the building, when the fix costs a few minutes instead of a multi-week appeal.

That single shift — from working denials to preventing them — is where the money is. So we've been building it.

Why a generic scrubber doesn't cut it

The hard part is that "what causes a denial" depends entirely on where you're billing.

In the US, claims run on X12, and the reasons come back as CARC/RARC codes buried in the remittance.
In KSA, it's NPHIES — a FHIR R4 world with its own adjudication outcomes and error structures.
In the UAE, it's eClaimLink / DHPO — an XML format with DHA and DOH code sets of their own.

A rule that catches a denial in one market is meaningless in another. And on top of the market differences, every individual payer has its own quirks — the undocumented reasons this insurer rejects that service. A one-size-fits-all checker misses most of what actually matters.

What we're building

The approach we're rolling into our RCM stack has two layers, on purpose:

An explainable rule engine. Market-aware, payer-aware rules that check a claim before submission — eligibility, authorization, coding logic, completeness, timely-filing windows. Rules are the right first layer because they're editable and they tell you exactly why a claim is risky. When a client says "this payer always denies X," that becomes a rule in minutes.
A denial-probability model on top. Trained on the client's own historical claims and remittances, it scores each new claim and surfaces the likely denial reason plus a suggested fix — learning the payer quirks the rules haven't caught yet.

Everything lands where the client can see it: a live dashboard (built on Datalytics) showing first-pass denial rate, the top denial reasons by payer and service, and — the number that actually matters — how that rate drops over time.

The whole philosophy fits on a sticky note: the cheapest denial is the one that never happens. Prevention beats appeal every time, because prevention costs minutes and appeals cost weeks.

We're putting this to work in RCM across the UAE, KSA, and US markets. If denials are eating into your margin and your team is stuck working them after the fact, that's exactly the problem we're solving — let's talk → megamtech.com

Jenkins JSON API usage

2018-12-11T04:41:00.003-08:00

<?php

class Jenkins

{

    private $crumb_field;

    private $crumb_value;

/**

 * Undocumented check the jenkins url is active

 *

 * @return boolean

 */

    function isActive()

    {

        $response = \Httpful\Request::get(JENKINS_URL . 'api/json/')

            ->expectsJson()

            ->authenticateWith(JENKINS_USER, JENKINS_PASS)

            ->send();

        return isset($response->body->assignedLabels[0]->name);

    }

    function getVersion()

    {

        $response = \Httpful\Request::get(JENKINS_URL . 'api/json/')

            ->expectsJson()

            ->authenticateWith(JENKINS_USER, JENKINS_PASS)

            ->send();

    }

/**

 * 

 * getLastBuild information for the $jobName sent

 * @param mixed $jobName - Valid job name

 */

    function getLastBuildInfo($jobName)

    {

        $response = \Httpful\Request::get(JENKINS_URL . '/job/' . $jobName . '/lastBuild/api/json')

            ->expectsJson()

            ->authenticateWith(JENKINS_USER, JENKINS_PASS)

            ->send();

        return $response->body;

    }

    /**

     *  get all available jobs from the Jenkins with basic information

     *

     * @return Object

     */

    function getJobs()

    {

        $response = \Httpful\Request::get(JENKINS_URL . '/api/json?tree=jobs[name,color,url,description,buildable,inQueue,healthReport,lastStatbleBuild,lastUnstableBuild,lastSuccessfulBuild,lastFailedBuild]')

            ->expectsJson()

            ->authenticateWith(JENKINS_USER, JENKINS_PASS)

            ->send();

        return $response->body;

    }

    /**

     * get config.xml for the jobname passed from Jenkins URL 

     *

     * @param [string] $jobName

     * @return Object

     */

    function getJobConfig($jobName)

    {

        $response = \Httpful\Request::get(JENKINS_URL . 'job/' . $jobName . '/config.xml')

            ->expectsXml()

            ->authenticateWith(JENKINS_USER, JENKINS_PASS)

            ->send();

        return $response;

    }

    /**

     * trigger a job 

     *

     * @param [string] $jobName

     * @return Object

     */

    function buildJob($jobName)

    {

        if ($this->crumb_field == '' || $this->crumb_value == '') {

            $this->getCrumb();

        }

        $response = \Httpful\Request::post(JENKINS_URL .'job/'.$jobName .'/build')

            ->addHeader($this->crumb_field, $this->crumb_value)

            ->authenticateWith(JENKINS_USER, JENKINS_PASS)

            ->send();

        return $response;

    }

    /**

     * Get crumb information 

     *

     * @return void

     */

    function getCrumb()

    {

        $response = \Httpful\Request::get(JENKINS_URL . 'crumbIssuer/api/json')

            ->authenticateWith(JENKINS_USER, JENKINS_PASS)

            ->expectsJson()

            ->send();

        $this->crumb_field = $response->body->crumbRequestField;

        $this->crumb_value = $response->body->crumb;

    }

    /**

     * Copy job to a new name

     *

     * @param [type] $from - Jobname

     * @param [type] $to - Jobname

     * @return Object

     */

    function createJob($from, $to)

    {

        if ($this->crumb_field == '' || $this->crumb_value == '') {

            $this->getCrumb();

        }

        $response = \Httpful\Request::post(JENKINS_URL . 'createItem?name=' . $to . '&from=' . $from . '&mode=copy')

            ->addHeader($this->crumb_field, $this->crumb_value)

            ->authenticateWith(JENKINS_USER, JENKINS_PASS)

            ->send();

        return $response;

    }

}

Cloudflare 522 ERROR - Nginx - Fail2Ban

2016-08-10T04:14:00.003-07:00

In my case I was using Ubuntu 16.04 with Fail2ban installed as if fail2ban takes the ips from nginx access log, Which has repeated cloudflare ips listed on the file (As if Cloudflare used as reverse proxy). Fail2Ban started block Cloudflare ips

The solution would be logging the original ips of the request on the access file instead of Cloudflare ip

First check your Nginx has "ngx_http_realip_module"

nginx -V

If Enabled add the ips below to a file under conf.d

set_real_ip_from 103.21.244.0/22;
set_real_ip_from 103.22.200.0/22;
set_real_ip_from 103.31.4.0/22;
set_real_ip_from 104.16.0.0/12;
set_real_ip_from 108.162.192.0/18;
set_real_ip_from 131.0.72.0/22;
set_real_ip_from 141.101.64.0/18;
set_real_ip_from 162.158.0.0/15;
set_real_ip_from 172.64.0.0/13;
set_real_ip_from 173.245.48.0/20;
set_real_ip_from 188.114.96.0/20;
set_real_ip_from 190.93.240.0/20;
set_real_ip_from 197.234.240.0/22;
set_real_ip_from 198.41.128.0/17;
set_real_ip_from 199.27.128.0/21;
set_real_ip_from 2400:cb00::/32;
set_real_ip_from 2606:4700::/32;
set_real_ip_from 2803:f800::/32;
set_real_ip_from 2405:b500::/32;
set_real_ip_from 2405:8100::/32;
set_real_ip_from 2c0f:f248::/32;
set_real_ip_from 2a06:98c0::/29;

# use any of the following two
real_ip_header CF-Connecting-IP;
#real_ip_header X-Forwarded-For;

CloudFlare ip might change so please check this link

nginx error - ERR_SPDY_INADEQUATE_TRANSPORT_SECURITY

2016-08-03T22:56:00.000-07:00

If you get the following error

This site can’t be reached
The webpage at https://yourdomain.com/ might be temporarily down or it may have moved permanently to a new web address.
ERR_SPDY_INADEQUATE_TRANSPORT_SECURITY

Change your nginx conf as follows

server
{
    listen 443 ssl http2;
    server_name yourdomain.com;
    ...
    ssl_ciphers 'ECDHE-RSA-AES128-GCM-SHA256:AES256+EECDH:AES256+EDH';
    ...
}

Reference : CloudFlare

Remove Skype ads

2016-06-12T23:18:00.001-07:00

Exit Skype
Open Internet Explorer
Open Internet Options
Go to Security and select Restricted sites
Click Sites
Add https://apps.skype.com/, click OK
Go to General, click Delete, and click Delete again
Open Skype, the blank space should be gone and there will be no ads

I don't know what the implications are and whether there any issues with other skype apps, but so far this is the only possible solution that I can think of."

Find malicious or hacked file in linux

2016-06-04T00:23:00.005-07:00

First find the outgoing connections with the following command

netstat -nputwN

Check the connections and find the connection which is trying to attack the other systems. For example PID 11009 in this scenario.

Use the following command to identify the list of files involved in the process execution

lsof -p 11009

why mcrypt_create_iv is slow

2016-04-05T00:11:00.002-07:00

If you don't specify argument for mcrypt_create_iv(), it will use /dev/random(on Linux) by default as a random number generator. The problem of /dev/random is that it's random pool depends on the system interrupts, when there is not enough system interrupts, it cannot generate enough random numbers, then the process tries to get the random numbers will wait and hang.

So instead of

mcrypt_create_iv($size)

use

mcrypt_create_iv($size, MCRYPT_DEV_URANDOM);

See the difference then

Block traffic to your server from a particular Country

2016-03-17T00:26:00.006-07:00

Create a file where we can declare some rules to use:

sudo nano /etc/iptables.firewall.rules

Inside there you'll want to paste the following:

*filter
# Allow all loopback (lo0) traffic and drop all traffic to 127/8 that doesn't use lo0

-A INPUT -i lo -j ACCEPT

-A INPUT -d 127.0.0.0/8 -j REJECT

# Accept all established inbound connections

-A INPUT -m state --state ESTABLISHED,RELATED -j ACCEPT

# Allow all outbound traffic - you can modify this to only allow certain traffic

-A OUTPUT -j ACCEPT

# Allow HTTP and HTTPS connections from anywhere (the normal ports for websites and SSL).

-A INPUT -p tcp --dport 80 -j ACCEPT

-A INPUT -p tcp --dport 443 -j ACCEPT

# Allow SSH connections

#

# The -dport number should be the same port number you set in sshd_config

-A INPUT -p tcp -m state --state NEW --dport 22 -j ACCEPT

# Allow ping

-A INPUT -p icmp --icmp-type echo-request -j ACCEPT

# Log iptables denied calls

-A INPUT -m limit --limit 5/min -j LOG --log-prefix "iptables denied: " --log-level 7

# Drop all other inbound - default deny unless explicitly allowed policy

-A INPUT -j DROP

-A FORWARD -j DROP

COMMIT

Save that. Next, we need to apply those rules – this is just a text file, and we need to instruct iptables to actually use it.

sudo iptables-restore < /etc/iptables.firewall.rules

That should have loaded the rules and applied them; you can check by

iptables -L

The output of that command ought to look like

Chain INPUT (policy ACCEPT)

target prot opt source destination

ACCEPT all -- anywhere anywhere

REJECT all -- anywhere 127.0.0.0/8 reject-with icmp-port-unreachable

ACCEPT all -- anywhere anywhere state RELATED,ESTABLISHED

ACCEPT tcp -- anywhere anywhere tcp dpt:http

ACCEPT tcp -- anywhere anywhere tcp dpt:https

ACCEPT tcp -- anywhere anywhere state NEW tcp dpt:ssh

ACCEPT icmp -- anywhere anywhere

LOG all -- anywhere anywhere limit: avg 5/min burst 5 LOG level debug prefix "iptables denied: "

DROP all -- anywhere anywhere

Chain FORWARD (policy ACCEPT)

target prot opt source destination

DROP all -- anywhere anywhere

Chain OUTPUT (policy ACCEPT)

target prot opt source destination

ACCEPT all -- anywhere anywhere

Great, it's working! But if you reboot the server it won't be. So lets fix that by creating a file which will run at boot.

sudo nano /etc/network/if-pre-up.d/firewall

Inside that file paste:

#!/bin/sh

/sbin/iptables-restore < /etc/iptables.firewall.rules

Save it. Now we must make sure it's allowed to execute:

sudo chmod +x /etc/network/if-pre-up.d/firewall

Done. The firewall is now running with those rules applied and those rules will be re-applied every time the server reboots. But it's not blocking China yet; it's only blocking anything not on port 80 or 443 (http and https).
Using ipset to block China

You can't manually add a few thousand IP addresses to your iptables, and even doing it automatically is a bad idea because it can cause a lot of CPU load (or so I've read). Instead we can use ipset which is designed for this sort of thing. ipset handles big lists of ip addresses; you just create a list and then tell iptables to use that list in a rule.

Note; I assume that the entirety of the following is done as root. Adjust accordingly if your system is based on sudo.

apt-get install ipset

Next, I wrote a small Bash script to do all the work, which you should be able to understand from the comments in it. Create a file:

nano /etc/block-china.sh

Here's what you want to paste into it:

# Create the ipset list

ipset -N china hash:net

# remove any old list that might exist from previous runs of this script

rm cn.zone

# Pull the latest IP set for China

wget -P . http://www.ipdeny.com/ipblocks/data/countries/cn.zone

# Add each IP address from the downloaded list into the ipset 'china'

for i in $(cat /etc/cn.zone ); do ipset -A china $i; done

# Restore iptables

/sbin/iptables-restore < /etc/iptables.firewall.rules

Save the file. Make it executable:

chmod +x /etc/block-china.sh

This hasn't done anything yet, but it will in a minute when we run the script. First, we need to add a rule into iptables that refers to this new ipset list the script above defines:

nano /etc/iptables.firewall.rules

Add the following line:

-A INPUT -p tcp -m set --match-set china src -j DROP

Save the file. To be clear, my full iptables.firewall.rules now looks like this:

*filter

# Allow all loopback (lo0) traffic and drop all traffic to 127/8 that doesn't use lo0

-A INPUT -i lo -j ACCEPT

-A INPUT -d 127.0.0.0/8 -j REJECT

# Accept all established inbound connections

-A INPUT -m state --state ESTABLISHED,RELATED -j ACCEPT

# Block anything from China

# These rules are pulled from ipset's china list

# The source file is at /etc/cn.zone (which in turn is generated by a shell script at /etc/block-china.sh )

-A INPUT -p tcp -m set --match-set china src -j DROP

# Allow all outbound traffic - you can modify this to only allow certain traffic

-A OUTPUT -j ACCEPT

# Allow HTTP and HTTPS connections from anywhere (the normal ports for websites and SSL).

-A INPUT -p tcp --dport 80 -j ACCEPT

-A INPUT -p tcp --dport 443 -j ACCEPT

# Allow SSH connections

#

# The -dport number should be the same port number you set in sshd_config

#

-A INPUT -p tcp -m state --state NEW --dport 22 -j ACCEPT

# Allow ping

-A INPUT -p icmp -j ACCEPT

# Log iptables denied calls

-A INPUT -m limit --limit 5/min -j LOG --log-prefix "iptables denied: " --log-level 7

# Drop all other inbound - default deny unless explicitly allowed policy

-A INPUT -j DROP

-A FORWARD -j DROP

COMMIT

Right now, nothing has changed with the server because no new rules have been applied; to do so, run the block-china.sh script:

/etc/block-china.sh

This should show some output as it pulls a fresh list of Chinese based IPs and then, after a few seconds or so, it will complete and drop you back to a command prompt.

To test if it worked, run:

iptables -L

You should now see a new rule blocking China – the output ought to look like this:

Chain INPUT (policy ACCEPT)

target prot opt source destination

ACCEPT all -- anywhere anywhere

REJECT all -- anywhere loopback/8 reject-with icmp-port-unreachable

ACCEPT all -- anywhere anywhere state RELATED,ESTABLISHED

DROP tcp -- anywhere anywhere match-set china src

ACCEPT tcp -- anywhere anywhere tcp dpt:http

ACCEPT tcp -- anywhere anywhere tcp dpt:https

ACCEPT tcp -- anywhere anywhere state NEW tcp dpt:ssh

ACCEPT icmp -- anywhere anywhere

LOG all -- anywhere anywhere limit: avg 5/min burst 5 LOG level debug prefix "iptables denied: "

DROP all -- anywhere anywhere

Chain FORWARD (policy ACCEPT)

target prot opt source destination

DROP all -- anywhere anywhere

Chain OUTPUT (policy ACCEPT)

target prot opt source destination

ACCEPT all -- anywhere anywhere

Almost done! This works, and will continue to work on re-boots. But, IP addresses change and that list will grow stale over time. If you want to pull and apply an updated list of IPs you can just run the block-china.sh script again.

Configure your websever:

We use Ngnix, So steps to block traffic from China do as follow :

Check ip modules are enabled

nginx -V

If you see --with-http_geoip_module in the output, you are ready to use the GeoIP database with nginx:

root@server1:~# nginx -V
nginx version: nginx/1.2.1
TLS SNI support enabled
configure arguments: --prefix=/etc/nginx --conf-path=/etc/nginx/nginx.conf --error-log-path=/var/log/nginx/error.log --http-client-body-temp-path=/var/lib/nginx/body --http-fastcgi-temp-path=/var/lib/nginx/fastcgi --http-log-path=/var/log/nginx/access.log --http-proxy-temp-path=/var/lib/nginx/proxy --http-scgi-temp-path=/var/lib/nginx/scgi --http-uwsgi-temp-path=/var/lib/nginx/uwsgi --lock-path=/var/lock/nginx.lock --pid-path=/var/run/nginx.pid --with-pcre-jit --with-debug --with-http_addition_module --with-http_dav_module --with-http_geoip_module --with-http_gzip_static_module --with-http_image_filter_module --with-http_realip_module --with-http_stub_status_module --with-http_ssl_module --with-http_sub_module --with-http_xslt_module --with-ipv6 --with-sha1=/usr/include/openssl --with-md5=/usr/include/openssl --with-mail --with-mail_ssl_module --add-module=/build/buildd-nginx_1.2.1-2.1-amd64-fMGfEu/nginx-1.2.1/debian/modules/nginx-auth-pam --add-module=/build/buildd-nginx_1.2.1-2.1-amd64-fMGfEu/nginx-1.2.1/debian/modules/nginx-echo --add-module=/build/buildd-nginx_1.2.1-2.1-amd64-fMGfEu/nginx-1.2.1/debian/modules/nginx-upstream-fair --add-module=/build/buildd-nginx_1.2.1-2.1-amd64-fMGfEu/nginx-1.2.1/debian/modules/nginx-dav-ext-module
root@server1:~#

Installing The GeoIP Database

On Debian/Ubuntu, the GeoIP database can be installed as follows:

apt-get install geoip-database libgeoip1

This places the GeoIP database in /usr/share/GeoIP/GeoIP.dat.

It is possible that it is a bit outdated. Therefore we can optionally download a fresh copy from the GeoIP web site:

mv /usr/share/GeoIP/GeoIP.dat /usr/share/GeoIP/GeoIP.dat_bak

cd /usr/share/GeoIP/
wget http://geolite.maxmind.com/download/geoip/database/GeoLiteCountry/GeoIP.dat.gz
gunzip GeoIP.dat.gz

Configuring nginx

Open /etc/nginx/nginx.conf...

vi /etc/nginx/nginx.conf

... and place this in the http {} block, before any include lines:

[...]
geoip_country /usr/share/GeoIP/GeoIP.dat;
map $geoip_country_code $allowed_country {
default yes;
FK no;
FM no;
EH no;
}
[...]

This allows all countries, except the three countries set to no (you can find a list of country codes here). To do it the other way round, i.e. block all countries and allow only a few, you'd do it this way:

[...]
geoip_country /usr/share/GeoIP/GeoIP.dat;
map $geoip_country_code $allowed_country {
default no;
FK yes;
FM yes;
EH yes;
}
[...]

Now, this actually doesn't block any country, it just sets the $allowed_country variable. To actually block countries, you must open your vhost configuration and place the following code in the server {} container (this can go inside and also outside any location {} block):

[...]
if ($allowed_country = no) {
return 444;
}
[...]

This returns the 444 error code to any visitor from a blocked country. What this does is it closes the connection without sending any headers. You can also use another error code like 403 ("Forbidden") if you like.

Reload nginx afterwards:

/etc/init.d/nginx reload

4 Links

· nginx: http://nginx.org/

· nginx Wiki: http://wiki.nginx.org/

· HttpGeoipModule: http://wiki.nginx.org/HttpGeoipModule

Multiple JDK versions in Centos

2015-08-17T00:53:00.000-07:00

Downloading Latest Java Archive

Java latest archive is available on its official site. We recommend to download latest version of Java from Oracle official website. After completing download also extract archive with given commands.

For 64 Bit:-

# cd /opt/
# wget --no-cookies --no-check-certificate --header "Cookie: gpw_e24=http%3A%2F%2Fwww.oracle.com%2F; oraclelicense=accept-securebackup-cookie" "http://download.oracle.com/otn-pub/java/jdk/7u79-b15/jdk-7u79-linux-x64.tar.gz"

# tar xzf jdk-7u79-linux-x64.tar.gz

For 32 Bit:-

# cd /opt/
# wget --no-cookies --no-check-certificate --header "Cookie: gpw_e24=http%3A%2F%2Fwww.oracle.com%2F; oraclelicense=accept-securebackup-cookie" "http://download.oracle.com/otn-pub/java/jdk/7u79-b15/jdk-7u79-linux-i586.tar.gz"

# tar xzf jdk-7u79-linux-i586.tar.gz

Note: If Above wget command doesn’t not work for you watch this example video to download java source archive using terminal.

Use archive file as per your system configuration. For this example we are using CentOS 7.0 (64 bit) system.

Install Java with Alternatives

After extracting Java archive file, we just need to set up to use newer version of Java using alternatives. Use the following commands to do it.

# cd /opt/jdk1.7.0_79/
# alternatives --install /usr/bin/java java /opt/jdk1.7.0_79/bin/java 2
# alternatives --config java

There are 3 programs which provide 'java'.

  Selection    Command
-----------------------------------------------
*  1           /opt/jdk1.7.0_60/bin/java
 + 2           /opt/jdk1.7.0_72/bin/java
   3           /opt/jdk1.7.0_79/bin/java

Enter to keep the current selection[+], or type selection number: 3 [Press Enter]

Now you may also required to set up javac and jar commands path using alternatives command.

# alternatives --install /usr/bin/jar jar /opt/jdk1.7.0_79/bin/jar 2
# alternatives --install /usr/bin/javac javac /opt/jdk1.7.0_79/bin/javac 2
# alternatives --set jar /opt/jdk1.7.0_79/bin/jar
# alternatives --set javac /opt/jdk1.7.0_79/bin/javac

Check Installed Java Version

Use following command to check which version of Java is currently being used by system.

# java -version

java version "1.7.0_79"
Java(TM) SE Runtime Environment (build 1.7.0_79-b15)
Java HotSpot(TM) 64-Bit Server VM (build 24.79-b02, mixed mode)

New Features in Mongo DB 3.0

2015-05-21T05:24:00.002-07:00

Enhanced Query Language and Tools

Key MongoDB tools mongoimport, mongoexport, mongodump, mongorestore, mongostat, mongotop and mongooplog have been re-written as multi-threaded processes in Go, allowing faster operations and smaller binaries.

mongodump and mongorestore now execute parallelized backup and recovery for small MongoDB instances. Dumps created by earlier releases can be restored to instances running MongoDB 3.0.

mongoimport can parallelize loads across multiple collections with multi-threaded bulk inserts allowing for significantly faster imports of CSV, TSV and JSON data exported from other databases or applications. Ensuring data quality, mongoimport now also supports input validation of field names during the import process.

Improved DBA Productivity: Enhanced Query Engine Introspection The MongoDB "explain() method is an invaluable tool for DBAs in optimizing performance. Using explain() output, DBAs can review query plans, ensuring common queries are serviced by well-defined indexes, as well as eliminating any unnecessary indexes that can increase write latency and add overhead during query planning and optimization.

In the latest MongoDB 3.0 release explain() has been significantly enhanced:

The query plan can now be calculated and returned without first having to run the query. This enables DBAs to review which plan will be used to execute the query, without having to wait for the query to run to completion.
DBAs can run explain() to generate detailed statistics on all query plans considered by the optimizer. Execution statistics are available for every evaluated plan, down to the granularity of execution stage. Now, for example, it is possible for the DBA to distinguish the amount of time a query plan spent sorting the result set from the amount of time spent reading index keys.
The explain() method exposes query introspection to a wider range of operations, including find, count, update, remove, group, and aggregate, enabling DBAs to optimize for a wider range of query types.

Richer Geospatial Apps: Big Polygon Support for Multi-Hemisphere Queries

MongoDB’s geospatial indexes and queries are widely used by developers building modern location-aware applications across industries as diverse as high technology, retail, telecommunications and government. MongoDB 3.0 adds big polygon geospatial support with$intersects and $within operators, allowing execution of queries over geographic areas extending across multiple hemispheres and areas that exceed 50% of the earth’s surface. As an example, an airline can now run queries to identify all its aircraft that have traveled across multiple hemispheres in the past 24 hours.

Enhanced Data Type Support: Easier Time-Series Analytics & Reporting
The MongoDB 3.0 aggregation pipeline offers a new $dateToString operator that simplifies report generation and grouping data by time interval. The operator formats the ISO Date type as a string with a user-supplied format, allowing developers to construct rich queries with less code.

Faster Issue Resolution: Enhanced Logging
Log analysis is a critical part of identifying issues and determining root cause. Now in MongoDB 3.0 developers, QA and operations staff have much greater control over the granularity of log messages and specific functional areas of the server to more precisely investigate issues.

Deploying Geo-Distributed, Datacenter-Aware Applications
Delivering a low latency experience to customers wherever they are located is a key design consideration for distributed systems. Using MongoDB’s native replica sets, copies (replicas) of the database can be deployed to sites physically closer to users, thereby reducing the effects of network latency. Reads can be issued with the nearestread preference, ensuring the query is served from the replica closest to the user, based on ping distance.

Pluggable Storage Engines: Extending MongoDB to New Applications

Multiple storage engines can co-exist within a single MongoDB replica set, making it easy to evaluate and migrate engines. Running multiple storage engines within a replica set can also simplify the process of managing the data lifecycle. For example as different storage engines for MongoDB are developed, it would be possible to create a mixed replica set configured in such a way that:

Operational data requiring low latency and high throughput performance is managed by replica set members using the WiredTiger or in-memory storage engine (currently experimental).

Replica set members configured with an HDFS storage engine expose the operational data to analytical processes running in a Hadoop cluster, which is executing interactive or batch operations rather than real time queries.

MongoDB replication automatically migrates data between primary and secondary replica set members, independent of their underlying storage format. This eliminates complex ETL tools that have traditionally been used to manage data movement.

MongoDB 3.0 ships with two supported storage engines:

The default MMAPv1 engine, an improved version of the engine used in prior MongoDB releases, now enhanced with collection level concurrency control.

The new WiredTiger storage engine. For many applications, WiredTiger's more granular concurrency control and native compression will provide significant benefits in the areas of lower storage costs, greater hardware utilization, higher throughput, and more predictable performance.

Higher Performance & Efficiency

Between 7x and 10x Greater Write Performance
MongoDB 3.0 provides more granular document-level concurrency control, delivering between 7x and 10x greater throughput for most write-intensive applications, while maintaining predictable low latency.

Compression: Up to 80% Reduction in Storage Costs
Despite data storage costs declining 30% to 40% per annum, overall storage expenses continue to escalate as data volumes double every 12 to 18 months. To make matters worse, improvements to storage bandwidth and latency are not keeping pace with data growth, making disk I/O a common bottleneck to scaling overall database performance.

Administrators have the flexibility to configure specific compression algorithms for collections, indexes and the journal, choosing between:

Snappy (the default library for documents and the journal), providing a good balance between high compression ratio – typically around 70%, depending on document data types – and low CPU overhead.

zlib, providing higher document and journal compression ratios for storage-intensive applications, at the expense of extra CPU overhead.

Prefix compression for indexes reducing the in-memory footprint of index storage by around 50% (workload dependent), freeing up more of the working set for frequently accessed documents.

Administrators can modify the default compression settings for all collections and indexes. Compression is also configurable on a per-collection and per-index basis during collection and index creation.

By introducing compression, operations teams get higher performance per node and reduced storage costs.

Creating Multi-Temperature Storage
Combining compression with MongoDB’s location-aware sharding, administrators can build highly efficient tiered storage models to support the data lifecycle. Administrators can balance query latency with storage density and cost by assigning data sets to specific storage devices. For example, consider an application where recent data needs to be accessed quickly, while for older data, latency is less of a priority than storage costs:

Recent, frequently accessed data can be assigned to high performance SSDs with Snappy compression enabled.

Older, less frequently accessed data is tagged to higher capacity, lower-throughput hard disk drives where it is compressed with zlib to attain maximum storage density and lower cost-per-bit.

MongoDB will automatically migrate data between storage tiers based on user-defined policies without administrators having to build tools or ETL processes to manage data movement.

Source : Mongo DB

Wordpress image upload http error on Nginx

2015-01-29T02:43:00.002-08:00

This was due to the following param in Nginx.conf under http section, Just change it to maximum size of you want to upload

example

client_max_body_size 5M;

Indian Rupee Symbol in OpenCart

2014-10-09T03:46:00.001-07:00

Execute these Queries in you database

Alter table oc_currency change column symbol_left symbol_left varchar(50);

Update oc_currency set symbol_left='<i class="fa fa-inr"> </i>' where currency_id=4;

Change the stylesheet.css in "catalog/view/theme/default/stylesheet/stylesheet.css" To remove the font size to make it even across the application

Note by default Font Awesome is included in 2.0 Opencart if your theme doesn't have added it to make this solution work

Jquery 1.10 Cheetsheet

2014-09-22T23:02:00.001-07:00

Selectors

BASICS

HIERARCHY

BASIC FILTERS

CONTENT FILTERS

VISIBILITY FILTERS

ATTRIBUTE

CHILD FILTERS

FORMS

Attributes / CSS

ATTRIBUTES

CSS

DIMENSIONS

OFFSET

DATA

Manipulation

COPYING

.clone()

DOM INSERTION, AROUND

DOM INSERTION, INSIDE

DOM INSERTION, OUTSIDE

DOM REMOVAL

DOM REPLACEMENT

Traversing

FILTERING

MISCELLANEOUS TRAVERSING

TREE TRAVERSAL

Events

BROWSER EVENTS

DOCUMENT LOADING

EVENT HANDLER ATTACHMENT

FORM EVENTS

KEYBOARD EVENTS

MOUSE EVENTS

EVENT OBJECT

Effects

BASICS

CUSTOM

FADING

SLIDING

Ajax

GLOBAL AJAX EVENT HANDLERS

HELPER FUNCTIONS

LOW-LEVEL INTERFACE

SHORTHAND METHODS

Core

JQUERY OBJECT

UTILITIES

DOM ELEMENT METHODS

INTERNALS

DEFERRED OBJECT

CALLBACKS OBJECT

Note : RED colored functions/items are frequently used

install Nginx PHP mongodb ubuntu

2014-07-24T22:36:00.003-07:00

Install prerequisites

apt-get update
apt-get install php5-cli php5-common php5-cgi php5-curl php5-fpm php5-json php5-mcrypt php5-dev php-pear php-apc

Enable Mongo

pecl install mongo
echo "extension=mongo.so" >> /etc/php5/fpm/php.ini

Install MongoDB

apt-get install mongodb mongodb-server

Restart Services

service nginx restart
service php5-fpm restart

done

Installing Apache Solr with Jetty in CentOS

2014-04-09T23:19:00.002-07:00

Apache Solr comes with the Jetty web server to be run as an example server. For single-instance installations, this built-in configuration is robust enough. Enabling Solr as a typical startup service makes management much easier. All of the following directions should be run as the root user and are loosely based on the Solr wiki.

As a prerequisite, newer versions of Solr require an upgraded version of Java -

yum -y install java-1.7.0-openjdk.x86_64

To download Solr and drop it into our installation directory (/opt/solr) -

curl -O http://www.us.apache.org/dist/lucene/solr/4.4.0/solr-4.4.0.tgz
tar xzf solr-4.4.0.tgz
mv solr-4.4.0/example /opt/solr

Now to enable Solr as a standard service (since Red Hat typically uses /etc/sysconfig for storing environment variables, this location is changed as well) -

# Download the Jetty default startup script
curl -o /etc/init.d/solr http://dev.eclipse.org/svnroot/rt/org.eclipse.jetty/jetty/trunk/jetty-distribution/src/main/resources/bin/jetty.sh

# Change file to be executable
chmod +x /etc/init.d/solr

# Change references from Jetty configuration to Solr
perl -pi -e 's/\/default\/jetty/\/sysconfig\/solr/g' /etc/init.d/solr

# Enable the Solr service
chkconfig solr on

Creating a dedicated Solr user allows permissions to be explicitly defined and reduce confusion caused by shared user accounts -

useradd -r -d /opt/solr -M -c "Apache Solr" solr

Change permissions of Solr installation to this user

chown -R solr:solr /opt/solr/

The startup script looks for environment variables in /etc/sysconfig/solr. That configuration file should look like -

JAVA_HOME=/usr/java/default
JAVA_OPTIONS="-Dsolr.solr.home=/opt/solr/solr $JAVA_OPTIONS"
JETTY_HOME=/opt/solr
JETTY_USER=solr
JETTY_LOGS=/opt/solr/logs

How To Install and Configure Varnish with Apache on Ubuntu 12.04

2014-03-05T01:21:00.002-08:00

About Varnish

Varnish is an HTTP accelerator and a useful tool for speeding up a server, especially during a times when there is high traffic to a site. It works by redirecting visitors to static pages whenever possible and only drawing on the virtual private server itself if there is a need for an active process.

Setup

To perform the steps in this tutorial, you will need to both have a user with sudo privileges and apache installed on your virtual private server.

To create a user with sudo privileges, go through the third and fourth steps of the initial ubuntu server setup tutorial
Apache can be installed on your VPS with a single command from the apt-get repository.

sudo apt-get install apache2

Step One—Install Varnish

The varnish site recommends installing the varnish package through their repository.

You can start that process by grabbing the repository:

sudo curl http://repo.varnish-cache.org/debian/GPG-key.txt | sudo apt-key add -

The next step is to add the repository to the list of apt sources. Go ahead and open up that file.

sudo nano /etc/apt/sources.list

Once inside the file, add the varnish repository to the list of sources.

deb http://repo.varnish-cache.org/ubuntu/ lucid varnish-3.0

Save and exit.

Finally, update apt-get and install varnish.

sudo apt-get update
sudo apt-get install varnish

Step Two—Configure Varnish

Once you have both apache and varnish installed, you can start to configure them to ease the load on your server from future visitors.

Varnish will serve the content on port 80, while fetching it from apache which will run on port 8080.

Let’s go ahead and start setting that up by opening the /etc/default/varnish file:

sudo nano /etc/default/varnish

Uncomment all of the lines under “DAEMON_OPTS”—under Alternative 2, and make the configuration match the following code:

 DAEMON_OPTS="-a :80 \
             -T localhost:6082 \
             -f /etc/varnish/default.vcl \
             -S /etc/varnish/secret \
             -s malloc,256m"

Once you save and exit out of that file, open up the default.vcl file:

sudo nano /etc/varnish/default.vcl

This file tells varnish where to look for the webserver content. Although Apache listens on port 80 by default, we will change the settings for it later. Within this file, we will tell varnish to look for the content on port 8080. The configuration should like this:

backend default {
    .host = "127.0.0.1";
    .port = "8080";
}

Step Three—Configure Apache

So far we have told varnish that apache ports will be running on 8080. However the default settings for apache are still on port 80. We will correct the discrepancy now.

Open up the apache ports file:

sudo nano /etc/apache2/ports.conf

Change the port number for both the NameVirtualHost and the Listen line to port 8080, and the virtual host should only be accessible from the localhost. The configuration should look like this:

NameVirtualHost 127.0.0.1:8080
Listen 127.0.0.1:8080

Change these settings in the default virtual host file as well:

sudo nano /etc/apache2/sites-available/default

The Virtual Host should also be set to port 8080, and updated line looks like this:

 <VirtualHost 127.0.0.1:8080>

Save and exit the file and proceed to restart Apache and Varnish to make the changes effective.

sudo service apache2 restart
sudo service varnish restart

Accessing your domain should instantly take you to the varnish cached version, and you can see the details of varnish’s workings with this command:

varnishstat

MySQL Storage Engines Comparison

2013-09-29T23:29:00.000-07:00

MyISAM Storage Engine Features(default storage engine)

Storage limits	256TB	Transactions	No	Locking granularity	Table
MVCC	No	Geospatial datatype support	Yes	Geospatial indexing support	Yes
B-tree indexes	Yes	Hash indexes	No	Full-text search indexes	Yes
Clustered indexes	No	Data caches	No	Index caches	Yes
Compressed data	Yes[a]	Encrypted data[b]	Yes	Cluster database support	No
Replication support[c]	Yes	Foreign key support	No	Backup / point-in-time recovery[d]	Yes
Query cache support	Yes	Update statistics for data dictionary	Yes

InnoDB Storage Engine Features

Storage limits	64TB	Transactions	Yes	Locking granularity	Row
MVCC	Yes	Geospatial datatype support	Yes	Geospatial indexing support	No
B-tree indexes	Yes	Hash indexes	No	Full-text search indexes	No
Clustered indexes	Yes	Data caches	Yes	Index caches	Yes
Compressed data	Yes[a]	Encrypted data[b]	Yes	Cluster database support	No
Replication support[c]	Yes	Foreign key support	Yes	Backup / point-in-time recovery[d]	Yes
Query cache support	Yes	Update statistics for data dictionary	Yes

MEMORY Storage Engine Features

The MEMORY storage engine creates tables with contents that are stored in memory. Formerly, these were known as HEAP tables. MEMORY is the preferred term, although HEAP remains supported for backward compatibility.

Storage limits	RAM	Transactions	No	Locking granularity	Table
MVCC	No	Geospatial datatype support	No	Geospatial indexing support	No
B-tree indexes	Yes	Hash indexes	Yes	Full-text search indexes	No
Clustered indexes	No	Data caches	N/A	Index caches	N/A
Compressed data	No	Encrypted data[a]	Yes	Cluster database support	No
Replication support[b]	Yes	Foreign key support	No	Backup / point-in-time recovery[c]	Yes
Query cache support	Yes	Update statistics for data dictionary	Yes

ARCHIVE Storage Engine Features

The ARCHIVE storage engine is used for storing large amounts of data without indexes in a very small footprint.

Storage limits	None	Transactions	No	Locking granularity	Row
MVCC	No	Geospatial datatype support	Yes	Geospatial indexing support	No
B-tree indexes	No	Hash indexes	No	Full-text search indexes	No
Clustered indexes	No	Data caches	No	Index caches	No
Compressed data	Yes	Encrypted data[a]	Yes	Cluster database support	No
Replication support[b]	Yes	Foreign key support	No	Backup / point-in-time recovery[c]	Yes
Query cache support	Yes	Update statistics for data dictionary	Yes

Others

The BLACKHOLE storage engine acts as a “black hole” that accepts data but throws it away and does not store it.

The CSV storage engine stores data in text files using comma-separated values format.

The FEDERATED storage engine enables data to be accessed from a remote MySQL database on a local server without using replication or cluster technology. When using a FEDERATED table, queries on the local server are automatically executed on the remote (federated) tables. No data is stored on the local tables.

The EXAMPLE storage engine is a stub engine that does nothing. Its purpose is to serve as an example in the MySQL source code that illustrates how to begin writing new storage engines. As such, it is primarily of interest to developers.

The MERGE storage engine, also known as the MRG_MyISAM engine, is a collection of identical MyISAM tables that can be used as one. “Identical” means that all tables have identical column and index information. You cannot merge MyISAM tables in which the columns are listed in a different order, do not have exactly the same columns, or have the indexes in different order.

The IBMDB2I storage engine is designed as a fully featured transaction-capable storage engine that enables MySQL to store its data in DB2 tables running on IBM i. With the IBMDB2I storage engine, data can be shared between MySQL applications and applications coded for native DB2 for i interfaces.

2013-09-29T23:28:00.000-07:00

Apache Solr Installation and Configuration

Installation

1. Download Apache Solr Package (http://www.apache.org/dyn/closer.cgi/lucene/solr/)

2. In shell enter “tar –xvzf apache-solr-x.x.x.tgz” this step will create a folder apache-solr-x.x.x

3. Rename the folder for ease of use “mv apache-solr-x.x.x apache-solr”

4. Install JDK (requires > 1.6.0) can download from sun downloads(http://java.sun.com/javase/downloads/widget/jdk6.jsp)

5. “cd apache-solr/example/”

6. Start the servlet “Java –jar start.jar”, Long list of messages will appear at the end you can see message as follows : INFO: Started SocketConnector @ 0.0.0.0:8983

7. Check for errors in java if so please correct it and restart the server. To Stop the server use “CTRL+C”

8. If everything goes fine you should see a screen as below in your browser

Configuring with MySQL Database

1. Download jdbc driver for mysql (mysql-connector-java-5.1.12-bin.jar) put it in the “lib” folder , this is the one which is responsible for communication between MySQL db and apache Solr

2. Open data-config.xml in your conf folder under example directory (example->conf->data-config.xml) in any text editor

3. Add XML as below

4. Data source is the tag which holds the data about the connecter (which database ,user, password, url, driver etc.,), we can have many data source in a single file and use the name of data source to connect

5. Change the “Query” in the “entity” tag as you want to create index it can also use joins and db functions in it

6. There is another set of tags called “Field” which used to specify what are the column which are returned by the query is going to be used (take care in giving values to name attribute),”Column” attribute is used to map the column name to the name attribute

7. Open “schema.xml” in the same folder search for “fields” tag and remove the content of it and add your columns names as picture below

Interface Vs Abstract Class

2013-09-29T23:24:00.000-07:00

Feature	Interface	Abstract class
Multiple inheritance	A class may inherit several interfaces.	A class may inherit only one abstract class.
Default implementation	An interface cannot provide any code, just the signature.	An abstract class can provide complete, default code and/or just the details that have to be overridden.
Access Modfiers	An interface cannot have access modifiers for the subs, functions, properties etc everything is assumed as public	An abstract class can contain access modifiers for the subs, functions, properties
Core VS Peripheral	Interfaces are used to define the peripheral abilities of a class. In other words both Human and Vehicle can inherit from a IMovable interface.	An abstract class defines the core identity of a class and there it is used for objects of the same type.
Homogeneity	If various implementations only share method signatures then it is better to use Interfaces.	If various implementations are of the same kind and use common behaviour or status then abstract class is better to use.
Speed	Requires more time to find the actual method in the corresponding classes.	Fast
Adding functionality (Versioning)	If we add a new method to an Interface then we have to track down all the implementations of the interface and define implementation for the new method.	If we add a new method to an abstract class then we have the option of providing default implementation and therefore all the existing code might work properly.
Fields and Constants	No fields can be defined in interfaces	An abstract class can have fields and constrants defined

Hudson in Tomcat

2013-09-29T23:22:00.000-07:00

Installation

To install Hudson on Tomcat, simply copy hudson.war to $TOMCAT_HOME/webapps, then access http://yourhost/hudson.

If you are running Tomcat just to host Hudson, then remove everything from $TOMCAT_HOME/webapps, and place hudson.war as ROOT.war. Tomcat should expand this and create the ROOT directory, and you should see Hudson in http://yourhost (if you accepted the Tomcat defaults - http://yourhost:8080) without any additional path. This also works nicely when you set up a virtual host, as it allows a single Tomcat instance to run multiple applications, yet users can still access your hudson with URLs like http://hudson.acme.org/ without any additional path. See the Tomcat documentation for more about how to set up a virtual host.

Upgrade

Simply overwrite your hudson.war with the new version and delete the hudson folder in webapps (exploded war). Tomcat should automatically redeploy the application.

Setting HUDSON_HOME

Before starting Tomcat, set CATALINA_OPTS like this. This can be also used to specify JVM options to increase the heap size:

$ export CATALINA_OPTS="-DHUDSON_HOME=/path/to/hudson_home/ -Xmx512m"
$ catalina.sh start

Or on Tomcat 1.6+ CATALINA_OPTS has been replaced by JAVA_OPTS (Thanks to Ronoaldo Pereira):

$ export JAVA_OPTS="-DHUDSON_HOME=/path/to/hudson_home/ -Xmx512m"
$ catalina.sh start

Or if that fails for some reason, you should still be able to use the environment variable:

$ export HUDSON_HOME=/path/to/hudson_home/
$ catalina.sh start

Increasing Tomcat PermGen space

If you see permgen errors when running Tomcat, edit the catalina.sh file on *NIX systems to assign the following options

JAVA_OPTS="-Xmx1024m -XX:NewSize=256m -XX:MaxNewSize=256m -XX:PermSize=256m -XX:MaxPermSize=256m"

Or on windows use the Configure Tomcat GUI Java panel to set the same additional options and restart Tomcat.

Securing Hudson on Tomcat

Tomcat by default stores the user database in an XML file called $TOMCAT_HOME/conf/tomcat-users.xml. You can add the following lines to this file to create an admin user.

<role rolename="admin"/>
<user username="hudson-admin" password="secret" roles="admin"/>

i18n

Some versions of Tomcat (such as 5.0.28) uses iso-8859-1 to decode URLs, which is in a clear violation of the relevant RFCs. To fix this problem, add the following URIEncoding attribute to the connector definition in $TOMCAT_HOME/conf/server.xml.

<Connector port="8080" URIEncoding="UTF-8"/>

Other people reported that a presence of RequestDumperValve would also cause a problem.

Relevant Issues

See the list of issues filed that are specifically marked relevant to Tomcat.

There is also a known issue in Tomcat 6.0.21 to 6.0.26 that may cause some requests to hang.

Tomcat from XAMPP

If you are using XAMPP's tomcat installation, and you have Java > 1.5, then you need to remove the following jars from the common\lib directory, otherwise you will get FileNotFound exceptions from the changelog.xml generator:

xalan.jar
xercesImpl.jar
xercesSamples.jar

Tomcat from Ubuntu

If you get Tomcat from Ubuntu via apt-get, Hudson will report an error citing the security permission issue. This is because Tomcat in Ubuntu comes with the security manager on by default. This can be disabled by modifying /etc/default/tomcat5 (version number will be different depending on which version of Tomcat you install.) See issue #719 for more details.

Tomcat from Debian

Fix security issues adding the next lines at /etc/tomcat5.5/policy.d/04webapps.policy:

grant codeBase "file:/var/lib/tomcat5.5/webapps/hudson/-" {
    permission java.security.AllPermission;
};

Tomcat from Windows

GUI Testing in Windows

Most Windows services -- including those run with the option "Allow service to interact with desktop" in Windows XP and Vista -- do not have access to many of the computer's resources, including the console display. This may cause Automated GUI Tests to fail if you are running Apache Tomcat as a Windows Service and are doing any GUI testing. This is true at least for AWT and Abbot frameworks. A typical error might look similar to this:

[junit] \# An unexpected error has been detected by HotSpot Virtual Machine:
[junit] \#
[junit] \# EXCEPTION_ACCESS_VIOLATION (0xc0000005) at pc=0x6d07baf4, pid=3260, tid=288
[junit] \#
[junit] \# Java VM: Java HotSpot(TM) Client VM (1.5.0_09-b03 mixed mode, sharing)
[junit] \# Problematic frame:
[junit] \# C [awt.dll+0xbaf4|awt.dll+0xbaf4]
[junit] \#

This limitation can be resolved by not running Tomcat as a Windows Service, but instead through a "Scheduled Task" as an Application that runs at logon. There are several options for doing this, an example would be to run "$TOMCAT_HOME\bin\tomcat5.exe". When setting up the scheduled task in Windows Vista consider choosing the check-box for "Run with highest privileges" from the general tab, as this removes the need to always provide administrator privileges and may resolve other issues as well.

Add service Auto start centos

2013-09-29T23:21:00.004-07:00

To auto start services in Centos or Redhat OS, you can use builtin chkconfig utility. It is located in /sbin directory. In the example below using a service called noip2d.

To auto start a noip2d service:

Add Service to auto start
sudo /sbin/chkconfig --add noip2d
Check is it addded
sudo /sbin/chkconfig --list noip2d
noip2d 0:off 1:off 2:off 3:on 4:off 5:off 6:off
Make it auto start on run different levels
sudo /sbin/chkconfig noip2d on
Check it on run levels
sudo /sbin/chkconfig --list noip2d
noip2d 0:off 1:off 2:on 3:on 4:on 5:on 6:off