After several iterations, the ITIL standard is now on version 4, published in February 2019. The key components of ITIL 4 are the four dimensions model of service agreement, and the ITIL Service Value System (SVS), illustrated in the diagram below:

The four dimensions model are:

1. Organizations and People
2. Information and Technology
3. Partners and Suppliers
4. Value Streams and Processes

More detailed explanation of the model and components are in the YASM Wiki site.

For IT System Administrators and Managers, ITIL 4 framework has a focus on overseeing the infrastructure and platforms used by organization that enables monitoring of available technology solutions. It includes a model to manage vendors providing Software as a Service (SaaS) and cloud computing environments, as a way of flexible and on-demand expansion. It outlines requirements for configurable computing resources and network access.

The framework of ITIL 4 for IT infrastructure and platform management, stem from the following:

1. IT Infrastructure components such as physical (ie. Dell, HP, Sparc servers), or virtual (ie. VMWare, Citrix, AWS), including the technologies used behind-the-scene, such as storage, network, Middleware applications (ie. JBOSS, Elasticsearch), and Operating Systems (ie. Linux, Windows).
   
2. Develop an implementation and administration strategy for infrastructure or platform that is unique for the organization, to fulfill the business and technical requirements.
   
3. Design communication methods between the organization’s own systems (cloud or on-premise), as well as to vendors in a secure and efficient way.

Of course, the above key concepts are an over simplification of the actual implementation. Running a data center is difficult and can be costly. Software is not perfect and fails without proper maintenance. Cybersecurity is complex, both in social and technological contexts. Technology keeps changing that will require constant training of the IT staff to keep up to date.

These are some of the reason why guidelines like ITIL 4 is valuable for IT Managers and Directors to be familiar with. It serves as a starting point to build IT infrastructure and platforms. They need to apply the practice on their own organization. With proper investment, going through many deployment iterations and lessons learned, an organization will be able to achieve the desired stability and security required.

The post Understanding ITIL v4 for Infrastructure and Platform Management appeared first on Building IT.

How prepared are we during an outage or data loss?

The typical follow up questions would be:

• What are the root causes?
• How do we recover?
• How do we prevent it from happening again?
• What is the cost of the damage?

The Ponemon Institute study (2016) showed the most common cause of outages are UPS power failure, cyber crime, and human error. If the research is conducted now (2019), cyber crime will probably be on top – as seen from today’s headlines with many banks and corporations (including manufacturing sector) hacked and data breached. The report also indicates this, as the trend jumped from 2% in 2010 to 22% in 2016!

Preparation for inevitable disasters will certainly involve more investment in cyber security training and update outdated software and hardware. It also helps to keep things simple, and not introduce unproven technology just for the sake of being trendy, or on the “bleeding edge”.

It is easier said than done. However, it’s not impossible. Management needs to be more aware that complicated business process introduce more human errors. Introducing many systems can also expose many weaknesses when IT teams tries to connect them together, to share data. Having multiple sites outside a traditional Enterprise data center also exposes data to be breached either by external hacker, or internal leak.

Prevention is certainly the priority for many concerned IT experts. Knowing the common points of failures, additional checks and balances in data recovery services and stressing security concerns for the employees are important first steps. One can’t simply wait for the storm to come. Instead, prepare for the storm and budget accordingly.

The post Be Prepared For An Outage appeared first on Building IT.

Can any company make money from Open Source?  The idea of open source work is like charity – it’s a great service for the community, but it won’t make anyone rich like Bill Gates, Steve Jobs, or Larry Ellison.  That thought may be right and wrong.

One example was MySQL. It was not capable of beating, or even competing, with Oracle Database.  However, it was the cheaper (free) solution to run web sites for bloggers (like this one) or SMBs. Since then, Oracle decided to buy MySQL’s Innobase engine because of the large install base. The same with Java which was once touted by Sun Microsystems as the ideal platform for Enterprise open-source language, was acquired by default when the Oracle bought Sun. No doubt, Larry Ellison had a thought that with this many users, there was a potential revenue to be made.

A decade ago, there was a speculation that an open source operating system like Linux is a possible money maker.  Back then, enterprise customers were still mostly invested in Solaris (Sparc) and Windows (x86) OS.  Red Hat was the biggest name in Linux distribution, and they were making money from providing support for it.  Now, IBM saw the Linux adoption kept going up, so it was only logical for IBM to acquire Red Hat, and the growing customer base along with it.

Linux adoption became bigger when Microsoft decided to include Linux as part of Windows 10 distribution, and contributed a large chunk of their code as open source.  The thinking is that contributing to vibrant and open community brings a sort of likeability to giants like Microsoft.  It’s no surprise Microsoft is touted to be a better technology innovator than Apple, Samsung, IBM, or even Google.

Speaking of likeability, or “coolness” factor, another example is Elastic offering a solid product based on Lucene open source search engine. With customers like Uber and SpaceX adopting their (based-on) open source search engine, Elastic is poised to make plenty of revenue.  So much so, they’re gaining competition from Amazon Web Services offering the same solution based on Elasticsearch open source software. The potential revenue is definitely available for the taking.

Can anyone get rich from Open Source?  Absolutely.  As long as there are mass adoptions, rich use cases, growing libraries, and plenty of community experts, open source is now becoming the standard for technology adoption in Enterprise environments.  The most successful companies will succeed in the open source game, only if they can make a compelling product that works really well and be able to support it. The customers are there – just make them happy!

[EDIT 8/1/2019]: Wired has a nice write-up on how companies should take the “moral” ground and mutual benefits when it comes to licensing open source software. My thought this can be tricky because of the old saying: “It’s just business, nothing personal.” Although it’s nice that we expect people to play nice, making money is a dirty business.

The post Can Anyone Get Rich from Open Source? appeared first on Building IT.

The advantages of using screen are:

1. When my SSH connection is broken, the command line sessions are still working. Useful when running shell scripts that take a long time to complete.
2. Having a shell with command line history, I could review the previous executions, in case I forgot to document something.
3. Instead of using the mouse to click on a different window, I use the keyboard shortcut Ctrl-A and the number keys, to switch between screens. Way quicker.

With the introduction of Red Hat Enterprise Linux 8, I was introduced (read: forced) to use a new screen replacement called TMUX. Apparently, it’s not a new util but it’s way more powerful – and useful. After using it for a while, I saw these advantages:

1. Having a vendor managed Firewall, I didn’t have a choice for connection keep-alives. My SSH connections will drop after inactivity. With TMUX, there’s a clock display that forces the connection to send data once a minute. Thus keeping the connection alive – indefinitely. No more dropped connections and reconnecting effort.
2. Being able to run screen within TMUX window is pretty nifty. I have another layer of switchable window, which is really handy when I have multiple servers representing the different layers for a site (ie. web, JBOSS, database, etc.) This is possible because TMUX’s key bindings for switching window is configurable and by default it’s different than screen’s.
3. TMUX has window panes, for dashboard like monitoring. Plus, it looks awesome!

Most of the Red Hat Enterprise Linux I’m working with is version 6.x, TMUX is not included as part of RHN repository. Thus, I had to build it from source. These are the steps to do it:

1. Download, compile, and install the latest libevent and ncurses.
2. Download TMUX and compile using the following configure flags (note, I installed on local home directory):
   CFLAGS="-I$HOME/local/include -I$HOME/local/include/ncurses" LDFLAGS="-L$HOME/local/lib -L$HOME/local/include/ncurses -L$HOME/local/include" CPPFLAGS="-I$HOME/local/include -I$HOME/local/include/ncurses"

If there’s a doubt that command line is important to a sysadmin’s daily work, Microsoft Developers are proud to present an expanded version of Windows OS command prompt. The video below has the full highlights and it looks great!

There’s even a trailer that rivals an iPhone launch commercial!

I’m excited that Operating System vendors are now providing more robust terminal tools, making command line a much better experience for all of IT folks!

The post TMUX: A Command Line Must! appeared first on Building IT.

When dealing with indices, it’s inevitable there will be a need to change the mapped fields. For example, in a firewall log, due to default mappings, a field like “RepeatCount” was stored as text instead of integer. To fix this, first write an ingest pipeline (using Kibana) to convert the field from text to integer:

PUT _ingest/pipeline/string-to-long
{
  "description": "convert RepeatCount field from string into long",
    "processors": [      
      {
        "convert": {
          "field": "RepeatCount",
          "type": "long",
          "ignore_missing": true
        }
      }
    ]
}

Next, run the POST command reindex the old index into the new one, while running the pipeline for conversion:

POST _reindex 
{ 
   "source": { 
      "index": "fwlogs-2019.02.01" 
   }, 
   "dest": { 
      "index": "fwlogs-2019.02.01-v2", 
      "pipeline": "string-to-long"
   } 
}

If there are multiple indices, it’s recommended to use a shell script to deal with the individual index systematically, such as “fwlogs-2019.02.01”, “fwlogs-2019.02.02”, etc.

#!/bin/sh
# The list of index names in rlist.txt file
LIST=`cat rlist.txt`
for index in $LIST; do
  curl -HContent-Type:application/json --user elastic:password -XPOST https://mysearch.domain.net:9200/_reindex?pretty -d'{
    "source": {
      "index": "'$index'"
    },
    "dest": {
      "index": "'$index'-v2",
      "pipeline": "string-to-long"
    }
  }'
done

Finally, clean up the old indices by deleting them. It’s a temptation to use Kibana to DELETE fwlogs-2019.02*, but beware the new indices have the suffix “-v2” and it will be deleted if the wildcard argument is used. Instead use the shell script to delete based on the names specifically listed in the txt file.

#!/bin/sh
# The list of index names in rlist.txt file
LIST=`cat rlist.txt`
for index in $LIST; do
  curl --user elastic:password -XDELETE "https://mysearch.domain.net:9200/$index"
done

The post Re-Index With Elasticsearch appeared first on Building IT.

Technology changes quickly. This is especially true in web development. With companies such as Google, Facebook, Amazon, or Netflix leading the way, there will always be the “next best thing” every IT professional has to pay attention to. Depending on the size and budget, not all companies can invest in the latest trend of technology. The question always asked: “What can we invest in?” As a guideline, annually Gartner publishes their infamous Hype Cycle, that charts the popularity (or decline) of technology. For those who are on the cutting edge will try to follow anything towards the “Peak of Inflated Expectations”, where the technology is hot. However, the most interesting set are the ones sliding into the “Trough of Disillusionment”. In 2018, those web applications were:

• Point-of-Decision HTAP
• Cloud-Native Application Architecture
• Reactive Programming
• Microservices
• Mesh App and Service Architecture
• Public Web APIs
• Miniservices

Enterprise has already started to invest in those declining trendy ideas.  However, in order to get to full adoption, IT Professionals have to familiarize with (and embrace) the new technology. It’ll be a difficult journey, but may be worth the investment. At this point, a great deal of material will be available since the concept has been around for a few years already. This is known as the “Slope of Enlightenment”. In order to get started, here are some suggestions on which presentation to listen to:

• NDC
• GOTO
• AWS re:Invent
• Devoxx
• Traversy Media
• A Cloud Guru

After listening to the presentations, one can determine the trend and make decisions on where/how to go to get Enterprise environments to the next level. It’ll take more time to get to the “Plateau of Productivity” where value can be realized by streamlining their execution for the long term production use.

Enterprise sure has plenty of work to do!

The post Hype Cycle 2018 For Web Applications appeared first on Building IT.

It’s imperative any exposed ports are being denied on the firewall side to prevent any successful hack. In a real world example, in the past 7 days, the hackers were scanning for popular vulnerable applications such as telnet, RDP (Windows Remote Desktop), Microsoft SQL, or SMTP.

Thankfully, those ports are being blocked on the firewall. Unfortunately, this does not deter them from trying again and again. Network and system admins must put in the due diligence in controlling access and patching applications. No matter the business requirements, security must take precedence and IT Professionals must have the tools to detect, analyze, and protect.

The post Are the Russian (Hackers) Still Coming? appeared first on Building IT.

With the recent federal government shutdown, it’s quite apparent their IT administrators still renew SSL certificates manually since many government websites went offline after the certs expired. Politics aside, since having secured connection and valid certificates are important these days, it should be a point for administrators to start automating the process. At the very least, have a project or plan in place to anticipate the shutdown and go through all of the important websites for possible cert renewals, 1-2 months in advance. As an Enterprise administrator, it’s also essential to have alerts or calendar reminders to renew an expiring cert. However, the best solution is to setup an automated job.

This is where tools out there like getssl and certbot can help. For this website, getssl is used to automate the SSL renewal process. The key processes are as follows:

Ensure Apache web server is setup. Since getssl relies on obtaining the proper “ACME” code from the target website to confirm the correct URL host, a regular port 80 HTTP connection must be made available first.

Per getssl documentation, run the inital setup to create the proper folders and files in $HOME/.getssl

getssl -c yourdomain.com

Edit the getssl.cfg in $HOME/.getssl/yourdomain.com folder with the correct directory for Apache web server’s doc-root and configuration files. Note, package installed Apache HTTPD uses /etc/apache2 as the default config directory.

When getssl is all setup, create a crontab to run getssl twice every month, for timely renewal (within 30 days). Be sure to restart Apache HTTPD to make sure the web server reloads the latest cert files.

0 9 1,15 * * $HOME/getssl/getssl -u -a > $HOME/getssl/getssl.out.txt 2>&1

The post Automatically Renew SSL Cert with LetsEncrypt and getssl appeared first on Building IT.

First, install the PHP Curl support for Apache on Linux:

apt-get -y install php-curl

Setup the PHP Composer in the doc-root folder, as outlined from elasticsearch-php github. Setup the php libraries via Composer:

php composer.phar init
curl -s http://getcomposer.org/installer | php
php composer.phar install --no-dev

Be sure to get the dependency package “elasticsearch/elasticsearch” and use the latest version as default. Note, skip the development package as it’s not really necessary.

Then, edit the composer.json file to include the directive:

   "require": {
            "elasticsearch/elasticsearch": "~6.0"
   }

Finally, create a test page to see if it can connect to the Elasticsearch server:

<?php

require 'vendor/autoload.php';

use Elasticsearch\ClientBuilder;

$hosts = [
   'http://myelasticsearchhost:9200'
];

$client = ClientBuilder::create()
   ->setHosts($hosts)
   ->build();

$params = [
    'index' => 'myindexname',
    'body' => [
        'query' => [
            'match' => [
                'post_title' => 'elasticsearch'
            ]
        ]
    ]
];

$response = $client->search($params);

$totalhits = $response['hits']['total'];
echo "We have $totalhits total hits\n";

echo "<P>The hits are the following:</P>";
$result = null;
$i=0;
while ($i <= $totalhits)
{
        $result[$i] = $response['hits']['hits'][$i]['_source'];
        $i++;
}

foreach ($result as $key => $value)
{
        echo $value['post_title'], "<br>";
}

?>

Output will look something like this:

We have 2 total hits

The hits are the following:

Using Elasticsearch for JBOSS Logs
Deleting Entries in Elasticsearch Based On Timestamp

Update Nov/2019: Since Elasticsearch updated their basic license to include basic username/password security, it’s advisable to set them up. It’s a straight-forward addition:

$hosts = [
   [
      'host' => 'myelasticsearchhost',
      'port' => '9200',
      'scheme' => 'http',
      'user' => 'myElasticUser',
      'pass' => 'myPassword'
   ]
];

Edit November 6, 2020: If there’s an upgrade or re-install of the OS into the latest version (such as from Ubuntu 16.x to 18.x), it is possible the version of cURL installed for PHP is a different one. For example, running php -m reveals:

PHP 7.2.34-8+ubuntu18.04.1+deb.sury.org+1 (cli) (built: Oct 31 2020 16:57:15) ( NTS )
Copyright (c) 1997-2018 The PHP Group
Zend Engine v3.2.0, Copyright (c) 1998-2018 Zend Technologies
with Zend OPcache v7.2.34-8+ubuntu18.04.1+deb.sury.org+1, Copyright (c) 1999-2018, by Zend Technologies

Since it is version 7.2 of PHP, install the cURL PHP library: apt-get install php7.2-curl

The post Installing Elasticsearch Client on PHP appeared first on Building IT.

Elastic is doing rapid development with Elasticsearch. As of this writing, they’re now on version 6.5.3 – when 6.5.2 was released less than 2 weeks ago!  Luckily, with a package install from repo (such as RPM on CentOS/RHEL), the upgrade process to minor versions is less painful.  However, it’s not without its pitfall. For example, an  upgrade from version 6.4.x to the latest 6.5.x could lead to Kibana not able to start due to incompatible indices.

In order to alleviate this, shutdown the Kibana service, and instruct Elasticsearch to perform a recovery on the .kibana index:

curl --user elasticuser:userpassword -s https://search.mydomain.net:9200/.kibana/_recovery?pretty

If it’s connected to a big cluster with a lot of shards, speed up the recovery process without using replicas:

curl --user elasticuser:userpassword -H 'Content-Type: application/json' -XPUT 'https://search.mydomain.net:9200/.kibana/_settings' -d '{ "index" : { "number_of_replicas" : 0 } }'

Give it a few minutes (depending how much data is there) and then start up Kibana service.  If, for some reason, it still takes a long time, there may be a problem with the migration process.  The kibana.log may indicate something like this:

{“type”:”log”,”@timestamp”:”2018-12-12T17:17:40Z”,”tags”:[“warning”,”stats-collection”],”pid”:15141,”message”:”Unable to fetch data from kibana_settings collector”}
{“type”:”log”,”@timestamp”:”2018-12-12T17:17:42Z”,”tags”:[“reporting”,”warning”],”pid”:15141,”message”:”Enabling the Chromium sandbox provides an additional layer of protection.”}
{“type”:”log”,”@timestamp”:”2018-12-12T17:17:42Z”,”tags”:[“info”,”migrations”],”pid”:15141,”message”:”Creating index .kibana_2.”}
{“type”:”log”,”@timestamp”:”2018-12-12T17:17:44Z”,”tags”:[“warning”,”migrations”],”pid”:15141,”message”:”Another Kibana instance appears to be migrating the index. Waiting for that migration to complete. If no other Kibana instance is attempting migrations, you can get past this message by deleting index .kibana_2 and restarting Kibana.”}

Shutdown Kibana again, and delete the .kibana_2 index:

curl --user elasticuser:userpassword -XDELETE https://search.mydomain.net:9200/.kibana_2

Start the Kibana service again and give it a few more minutes to perform house-keeping.  Kibana should be up and running now.

The post Recovering Kibana After Upgrade appeared first on Building IT.