Security, networking and system integration

noreply@blogger.com (Zeljko Milinovic, MSc) — Tue, 9 Aug 2022 03:02:00 -0700

Oracle Exadata Smart Scan conditions

Exadata Smart Scan gives a possiblity to the Oracle Database to offload SQL processing to the Exadata Storage Servers. In order to make usage of Smart Scan features of Exadata, the initialization parameter cell_offload_processing must be set to TRUE.

SQL> show parameter cell_offload_pr

cell_offload_processing boolean TRUE

The Smart Scan magic SQL processing happens at the storage tier, instead of the database tier, to improve query performance. It reduces the volume of data sent to the database tier thereby reducing the CPU usage on database nodes.

source: oracle.com

We have 4 basic conditions for Smart Scan to be triggered

- There must be a full scan of an object.

- Oracle’s Direct Path Read mechanis must be used, and path reads are generally used when we read

into PGA memory (not the buffer cache).

- Each segment being scanned must be on a Cell disk group (usually with ASSM it is).

- We need the CELL_OFFLOAD_PROCESSING initialization parameter to be enabled.

If we get the STORAGE keyword in the explanation plan, does not guarantee that the offload will occur. Sometimes Exadata Storage Server could decide that Smart Scan will not produce any performance.

Detect slow PostgreSQL queries, pg_stat vs auto logging overhead.

noreply@blogger.com (Zeljko Milinovic, MSc) — Mon, 18 Oct 2021 00:33:00 -0700

Detect slow PostgreSQL queries, pg_stat vs auto logging overhead

To be able to find and detect slow queries on a RDBMS PostgreSQL is an important thing as the database tends to grow. Optimizing performance and expensive SQL queries is one of the major task of the database system maintainers. These problems can be approached with many ways and external systems, but I have tried to obtain the fast and “keep it simple” way, which I will try to explain in this blog post.

source: stackify

One of the cool PostgreSQL features is the auto_explain module , which provides a means for logging execution plans of slow statements automatically, without having to run EXPLAIN by hand (stated by the documentation). It is also stated in the documentation that there is a price in overhead for that. I would suggest not using the excessive logging from auto_explain on production systems.

The alternative that I am using often is the pg_stat_statements module which can be easily configured on a production or test system.

After that we need to run CREATE EXTENSION pg_stat_statements in our databases, which will allow PostgreSQL to create a view for us.

SELECT * FROM pg_stat_statement

The created view is able to tell us, which query has been executed how often and show us insights about the total runtime of this type of query as well as about the distribution of runtimes for those particular queries.

This allows me to get a quick overview of the CPU percentage being used and a quick overview of the I/O behavior of many types of queries, which could be a great deal of the reason that causes high loads on your production systems.

Oracle Blockchain contributions

noreply@blogger.com (Zeljko Milinovic, MSc) — Tue, 1 Oct 2019 02:30:00 -0700

Oracle Blockchain Contributions

Oracle has added developer-oriented productivity enhancements, enhanced privacy, confidentiality, and identity management features that are critical to diverse organizations conducting business transactions. New DevOps capabilities make the platform easier to integrate with existing business and IT systems. Additionally, as blockchain becomes an important data store in the enterprise, the platform enables Oracle Autonomous Data Warehouse customers to transparently capture blockchain transaction history and current state data for analytics and to integrate it with other data sources


source: Oracle Blog

I found one of the new cool features to be much welcome in the portfolio:

- Rich history database shadows transaction history into a relational database schema in the Autonomous Data Warehouse or other Oracle databases, which transparently enables analytics integration for interactive dashboards and reports. Here is an example of the use of rich history stored on Oracle's Autonomous Datawarehouse and visualized using Oracle Analytics Cloud.

One more important thing is the introduction of the Third-party certificate support for registering client organizations on the blockchain network to enable them to use existing certificates issued by trusted third parties. The security was a major topic in the Enterprise world, so did the Oracle what other big developing companies do, introduce the standard methods of security.

To avoid the Lock-In seen in the previous licensing models, the big companies need to contribute to the development of Blockchain services with the same speed as other smaller communities do. This is the only way to hold the sustainabillity of the Blockchain Ecosystem.

Server side request forgery (SSRF) attack

noreply@blogger.com (Zeljko Milinovic, MSc) — Tue, 23 Apr 2019 22:09:00 -0700

Server side request forgery (SSRF) attack

In a Server-Side Request Forgery (SSRF) attack, the attacker can abuse functionality on the server to read or update internal resources. The attacker can supply or a modify a URL which the code running on the server will read or submit data to, and by carefully selecting the URLs, the attacker may be able to read server configuration such as AWS metadata, connect to internal services like http enabled databases or perform post requests towards internal services which are not intended to be exposed.

source: Acunetix

You can no longer request information from internal systems, but can still make internal API-calls. Imagine the following PHP code (source Detectify):

//getimage.php

$content = file_get_contents($_GET['url']);

file_put_contents(‘image.jpg’, $content);

The above code will fetch data from a URL using PHP’s file_get_contents() function and then save it to the disk. A legitime request would then look like:

GET /getimage.php?url=https://website.com/images/cat.jpg

And the web application would make a request to https://website.com/images/cat.jpg. This code could be exploited with SSRF. Such an attack could look something like:

GET /getimage.php?url=http://127.0.0.1/api/v1/getuser/id/1

In this case the vulnerable web application would make a GET request to the internal REST API service, trying to access the /api/v1/getuser/id/1 endpoint. This REST API service is only accessible on the local network, but due to a SSRF vulnerability it was possible for the attacker to make such an internal request and read that response. Sometimes you can make a request to an external server, and the request itself may contain sensitive headers. One of many examples would be HTTP basic passwords, if a proxy has been used. SSRF can therefore be carried out to both internal and external services.

Due to microservices and serverless platforms, SSRF will probably be a bigger thing in the future. Making internal requests now means that you can interact with other parts of the service, pretending to be the actual service.

Mitigation

The most robust way to avoid Server Side Request Forgery (SSRF) is to whitelist the DNS name or IP address that your application needs to access. If a whitelist approach does not suit you and you must rely on a blacklist, it’s important to validate user input properly. For example, do not allow requests to private (non-routable) IP addresses.

To prevent response data leaking to the attacker, you must ensure that the received response is as expected. Under no circumstances should the raw response body from the request sent by the server be delivered to the client.

If your application only uses HTTP or HTTPS to make requests, allow only these URL schemas. If you disable unused URL schemas, the attacker will be unable to use the web application to make requests using potentially dangerous schemas such as file:///, dict://, ftp:// and gopher://

Uncomplicated Firewall with Python

noreply@blogger.com (Zeljko Milinovic, MSc) — Tue, 23 Apr 2019 00:53:00 -0700

Uncomplicated Firewall with Python

Uncomplicated Firewall, is an interface to iptables that simplifyies the process of configuring a firewall. Iptables is flexible, it can be difficult for beginners to learn how to use it to properly configure a firewall.

In the example below is a python program that makes it easy allowing and blocking various services by IP address.

## Description :

##    Generate ip-host binding list for a list of nodes, when internal DNS is missing.

## 1. For existing nodes, allow traffic from new nodes

## 2. For new nodes, allow traffic from all nodes

## Sample:

##    python ./ufw_allow_ip.py --old_ip_list_file /tmp/old_ip_list --new_ip_list_file /tmp/new_ip_list \

## --ssh_username root --ssh_port 22 --ssh_key_file ~/.ssh/id_rsa

##-------------------------------------------------------------------

import os, sys

import paramiko

import argparse

# multiple threading for a list of ssh servers

import Queue

import threading

import logging

log_folder = "%s/log" % (os.path.expanduser('~'))

if os.path.exists(log_folder) is False:

os.makedirs(log_folder)

log_file = "%s/%s.log" % (log_folder, os.path.basename(__file__).rstrip('\.py'))

logging.basicConfig(filename=log_file, level=logging.DEBUG, format='%(asctime)s %(message)s')

logging.getLogger().addHandler(logging.StreamHandler())

def get_list_from_file(fname):

l = []

with open(fname,'r') as f:

for row in f:

row = row.strip()

if row.startswith('#') or row == '':

continue

l.append(row)

return l

def ufw_allow_ip_list(server_ip, ip_list, ssh_connect_args):

if len(ip_list) == 0:

print("Skip run ufw update in %s, since ip_list is empty" % (server_ip))

return("OK", "")

[ssh_username, ssh_port, ssh_key_file, key_passphrase] = ssh_connect_args

ssh_command = ""

# TODO: improve this command, by using a library

for ip in ip_list:

ssh_command = "%s && ufw allow from %s" % (ssh_command, ip)

if ssh_command.startswith(" && "):

ssh_command = ssh_command[len(" && "):]

print("Update ufw in %s. ssh_command: %s" % (server_ip, ssh_command))

output = ""

try:

ssh = paramiko.SSHClient()

ssh.set_missing_host_key_policy(paramiko.AutoAddPolicy())

        key = paramiko.RSAKey.from_private_key_file(ssh_key_file, password=key_passphrase)

ssh.connect(server_ip, username=ssh_username, port=ssh_port, pkey=key)

stdin, stdout, stderr = ssh.exec_command(ssh_command)

output = "\n".join(stdout.readlines())

output = output.rstrip("\n")

print("Command output in %s: %s" % (server_ip, output))

ssh.close()

except:

        return ("ERROR", "Unexpected on server: %s error: %s\n" % (server_ip, sys.exc_info()[0]))

return ("OK", output)

###############################################################

if __name__ == '__main__':

# get parameters from users

parser = argparse.ArgumentParser()

parser.add_argument('--old_ip_list_file', required=True, \

help="IP list of current cluster", type=str)

parser.add_argument('--new_ip_list_file', required=True, \

help="IP list of new nodes", type=str)

parser.add_argument('--ssh_username', required=False, default="root", \

help="Which OS user to ssh", type=str)

parser.add_argument('--ssh_port', required=False, default="22", \

help="Which port to connect sshd", type=int)

    parser.add_argument('--ssh_key_file', required=False, default="%s/.ssh/id_rsa" % os.path.expanduser('~'), \

help="ssh key file to connect", type=str)

parser.add_argument('--key_passphrase', required=False, default="", \

help="Which OS user to ssh", type=str)

l = parser.parse_args()

    ssh_connect_args = [l.ssh_username, l.ssh_port, l.ssh_key_file, l.key_passphrase]

old_ip_list = get_list_from_file(l.old_ip_list_file)

new_ip_list = get_list_from_file(l.new_ip_list_file)

has_error = False

# TODO: speed up this process by multiple threading

for old_ip in old_ip_list:

(status, output) = ufw_allow_ip_list(old_ip, new_ip_list, ssh_connect_args)

if status != "OK":

has_error = True

print("Error in %s. errmsg: %s" % (old_ip, output))

for new_ip in new_ip_list:

        (status, output) = ufw_allow_ip_list(new_ip, new_ip_list + old_ip_list, ssh_connect_args)

if status != "OK":

has_error = True

print("Error in %s. errmsg: %s" % (new_ip, output))

if has_error is True:

sys.exit(1)

#!/usr/bin/python

Benefits of Blockchain

noreply@blogger.com (Zeljko Milinovic, MSc) — Thu, 20 Sep 2018 06:39:00 -0700

Benefits of Blockchain

Blockchain’s promise is “the decentralization of trust, enabling value flow without intermediaries”. It allows financial transactions to be verified and cleared without the need for a trusted third party sitting between market participants. Removing intermediaries reduces costs and complexity.

The blockchain’s security and privacy protocols are based on the use of a “cryptographic hash function” — each block (of transactions) in the chain is identified by its own “hash” key. This approach was developed to prevent the “double spending” of Bitcoins. The complexity of the crypto hash function reduces the blockchain’s susceptibility to fraud.

The distributed ledger approach means that all the members of a financial market (the network) share an identical system of record, rather than each maintaining their own proprietary view of it. This replicated, shared ledger provides consensus, provenance, immutability and finality for the transactions concerned — payments, asset transfers, etc. This shared approach removes the need for reconciliations.

New transactions are only accepted for posting to the distributed ledger (through the creation of new blocks for the chain) once all the computers in the network achieve consensus as to their validity. The

verification of transactions by all network users reduces error rates and queries.

At the heart of blockchain is a new type of distributed database. This provides for the exchange of information in a synchronous and even manner, as well as allows it to be updated constantly, providing near-instant clearing and settlement. The provision of faster settlement means less risk in the financial system and so reduces the capital requirements of market participants.

The new distributed database functionality also allows code to run with the blockchain to modify data (both on and off the chain) automatically. This enables the blockchain to support self-enforcing or “smart” contracts, allowing the automation of a variety of business functions.

Supply chain and trade finance

• Plays directly to many of the blockchain’s principal strengths

• Mathematics is used to achieve trust between the parties to a transaction

- The current role of banks in trade transactionsis principally to accept risk, thereby allowing the

parties to trust each other.

• Allows shared access for all parties within the supply chain (buyers, suppliers, banks, logistics companies, insurance companies, customs and health authorities, etc.) to the single view of the truth

• A private, permissioned blockchain can restrict access only to those parties involved in the supply

chain concerned • The delivery and use of rich information — i.e., transactions on the chain would include invoice numbers, certificates of origin, bills of lading, bills of exchange, insurance documents, customs documents, health certificates, etc.

• “Smart contract” capabilities enable automated decision making and information handling.

In summary, the focus on blockchain has shifted and is now on understanding the technology and the value it offers, as opposed to some months ago when it was more of a collective huddle and a focus on collaboration and shared intent. However, there is still no clear emergence of where blockchain its in terms of technology, business benefits and application. “We don’t know who will own it and who will benefit so maybe the wrong group within a bank is looking at the benefit?” said one interviewee.

So where next? The banks did not plan for blockchain, but the direction is clear. Few people fully understand blockchain within banks, but everyone is asking powerful questions as to what it is and what it offers. It is not seen as a lethal threat to banking, but it is seen as disruptive new technology they have to understand more fully, and for which they must develop a strategic approach to deployment. As one interviewee said: “I actually don’t understand how we have reached such a stage of involvement in the bank without ownership or control. We need a blockchain czar.”

What is absolutely clear is the recognition that there is a need to separate the hype from the reality and from an executive and strategic perspective, to take a fresh look at what blockchain offers banks, and what banks must do to adopt it.

Source: ACI Worldwide

Application Centric Infrastructure with Python

noreply@blogger.com (Zeljko Milinovic, MSc) — Mon, 20 Feb 2017 04:59:00 -0800

In application-centric networking, troubleshooting does not mean logging into discrete devices and examining networking state information. If a web application is not performing, you start with the web application. The fact that relevant state information might exist within a router, switch, or even a web server is secondary. You want to gather intelligence on the application itself. That data needs to be collected and correlated. And you aren’t done until the application is up and running as it should.

There are several questions you need to ask: Can you see how applications are talking across your network? For distributed applications, do you have a view of which components are where? Can you see how data is flowing between them?

Here I would like to present a small Python programm that I adapted from the Cisco code repository.

# Copyright (c) 2015 Cisco Systems                                             #

"""

Simple application that logs on to the APIC and displays all

of the Interfaces.

"""

import sys

import re

import json

import acitoolkit.acitoolkit as aci

def main():

"""

Main execution routine

:return: None

"""

# Take login credentials from the command line if provided

# Otherwise, take them from your environment variables file ~/.profile

description = 'Simple application that logs on to the APIC and displays all of the Interfaces.'

creds = aci.Credentials('apic', description)

args = creds.get()

# Login to APIC

session = aci.Session(args.url, args.login, args.password)

resp = session.login()

if not resp.ok:

print('%% Could not login to APIC')

sys.exit(0)

resp = session.get('/api/class/ipv4Addr.json')

intfs = json.loads(resp.text)['imdata']

data = {}

for i in intfs:

ip = i['ipv4Addr']['attributes']['addr']

op = i['ipv4Addr']['attributes']['operSt']

cfg = i['ipv4Addr']['attributes']['operStQual']

dn = i['ipv4Addr']['attributes']['dn']

node = dn.split('/')[2]

intf = re.split(r'\[|\]', dn)[1]

vrf = re.split(r'/|dom-', dn)[7]

if vrf not in data.keys():

data[vrf] = []

else:

data[vrf].append((node, intf, ip, cfg, op))

for k in data.keys():

header = 'IP Interface Status for VRF "{}"'.format(k)

print header

template = "{0:15} {1:10} {2:20} {3:8} {4:10}"

print(template.format("Node", "Interface", "IP Address ", "Admin Status", "Status"))

for rec in sorted(data[k]):

print(template.format(*rec))

if __name__ == '__main__':

main()

With the help of the ACI Toolkit library from Cisco we could define and create many functions and procedures needed to manipulate and programm the ACI Fabric. As an universal data format JSON is used to retrieve data from the APIC controller. The data interested to us are VRFs, IPs, VLANs and so on. In the final function we define a loop that that will search for all of the the values stored inside the dictionary data structure from Python and sort them using a template.

For those wanted to know more about the ACI please checkout the source text from the SDX Central.

Cisco ACI is a tightly coupled policy-driven solution that integrates software and hardware. The hardware for Cisco ACI is based on the Cisco Nexus 9000 family of switches. The software and integration points for ACI include a few components, including Additional Data Center Pod, Data Center Policy Engine, and Non-Directly Attached Virtual and Physical Leaf Switches. While there isn’t an explicit reliance on any specific virtual switch, at this point, policies can only be pushed down to the virtual switches if Cisco’s Application Virtual Switch (AVS) is used, though there has been talk about extending this to Open vSwitch in the near future.

In a leaf-spine ACI fabric, Cisco is provisioning a native Layer 3 IP fabric that supports equal-cost multi-path (ECMP) routing between any two endpoints in the network, but uses overlay protocols, such as virtual extensible local area network (VXLAN) under the covers to allow any workload to exist anywhere in the network. Supporting overlay protocols is what will give the fabric the ability to have machines, either physical or virtual, in the same logical network (Layer 2 domain), even while running Layer 3 routing down to the top of each rack. Cisco ACI supports VLAN, VXLAN, and network virtualization using generic routing encapsulation (NV-GRE), which can be combined and bridged together to create a logical network/domain as needed.

From a management perspective, the central SDN Controller of the ACI solution, the Application Policy Infrastructure Controller (APIC) manages and configures the policy on each of the switches in the ACI fabric. Hardware becomes stateless with Cisco ACI, much like it is with Cisco’s UCS Computing Platform. This means no configuration is tied to the device. The APIC acts as a central repository for all policies and has the ability to rapidly deploy and re-deploy hardware, as needed, by using this stateless computing model.

Feel free to comment.

Quick peek inside Kubernetes Containers.

noreply@blogger.com (Zeljko Milinovic, MSc) — Sun, 2 Oct 2016 09:14:00 -0700

Quick peek inside Kubernetes Containers

To get a fresh start inside the Container world I have tested a couple of technologies. The new and interresting container managament project that I tested is called Kubernetes. The Wiki definition follows: Kubernetes (commonly referred to as "k8s") is an open source container cluster manager originally designed by Google and donated to the Cloud Native Computing Foundation. It aims to provide a "platform for automating deployment, scaling, and operations of application containers across clusters of hosts".

On my simple home Lab I will install and quickly deploy Kubernetes on a fresh Centos 7 server with minimal desktop features. So let us get started.

First what we need is to disable firewall services.

systemctl disable firewalld.service

systemctl stop firewalld.service

After this what we need to install is the network daemon service:

yum -y install ntp

systemctl start ntpd

systemctl enable ntpd

To download the packages need for the Kubernetes cluster manager we need to add a new repository to the Centos defaults. We do this creating a file inside the /etc/yum.repos.d folder called virt7-docker-common-release.repo

This file should contain the following Urls and content for us to be able to download the Kubernetes packages need for the setup of the container manager:

[root@Cent01 yum.repos.d]# cat virt7-docker-common-release.repo

[virt7-docker-common-release]

name=virt7-docker-common-release

baseurl=http://cbs.centos.org/repos/virt7-docker-common-release/x86_64/os/

gpgcheck=0

Now are ready to download the basic Kubernetes managament packages and install them:

yum install docker etcd kubernetes

As the packages are downloaded from the repository they will be automatically installed using the Yellow dog manager for Centos packages. If you need more info or want to see the verbose informatons add the --v switch after the install command.

To restart and enable the services we would need to define a FOR loop:

for SERVICE in docker etcd kube-apiserver kube-controller-manager kube-scheduler kube-proxy kubelet; do

systemctl restart $SERVICE

systemctl enable $SERVICE

done

To make the managament available over the Web GUI (which I find very handy from the beginning) we would need to install the Cockpit manager.

yum install cockpit cockpit-kubernetes

systemctl enable cockpit.socket

systemctl start cockpit.socket

That was for now on installing the packages. The next step would be to start the web GUI using the port 9090. You just need to call this url: https://server_ip:9090

The base image of the Cockpit Kubernetes manager will show us the options needed to create and monitor the container Also the cluster managament is prepared and deployment of Micro services.

Under Tools we can still maintain the Bare metal Centos 7 server using the command line.

And as simpel as that we can find many Docker container images from the repositories and download them. From simple web servers to complex MySQL redundant scenarios. As simple as download-click-run image system is a performance booster.

What I also find interresant is the possibility to build the apps from Manifest files and deploy them as Micro Services. OpenShift supported from RedHat could be also used as a great tool to build the applications on Docker Container technology.

And in a matter of minutes we can have a running cluster based container image managament orchestra. The deploy of manifests will be explained in the next blog where I will research more on Openshift and app buiding.

Stay tuned and follow me and please send your comments.

Automate Mysql with Puppet

noreply@blogger.com (Zeljko Milinovic, MSc) — Sun, 30 Nov 2014 02:49:00 -0800

Automate Mysql with Puppet

Puppet, based on my personal experience is a great technology that allows new ways in DevOps environments and automation. With this technology people can code and manage the complete IT infrastructure from a central location, or a within a cloud infrastructure. It has completely changed the way on providing IT services and managing the same. Some cool features like role based access control and activity login allow people to define stable management strategy.

To get started, I created a simple Client / Server based puppet scenario in which we will automate Mysql server implementation in a linux environment. Puppet master server is configured to provide modules, manifests and classes to the agent servers in the complete infrastructure.

After installing the puppet master server the users can use the following folder etc/puppet/manifests to develop and write some code for deploying the services to the agent servers.

First, on the puppet master server we should download and install the package module for the Mysql server using the following command:

sudo puppet module search mysql

After the successful installation you should verify that the folder mysql exists in the /etc/puppet/modules folder with the source installation of mysql server.

After the validation of folder existence , the source file are located there. Now we have to define the site.pp file inside the /etc/puppet/manifests as a manifest file that will send the code execution of installing the mysql server on an agent server. The content of the file should look like the following:

node 'agent_node01' { include mysql::server }

This line of code defines the agent_node01 will look for the installation of mysql server inside the mysql server module installation on the puppet master server. The next step is to connect to the agent node and start the puppet agent service.

sudo /etc/init.d/puppet start

After restarting the service I will initiate the agent test command to force the pulling the package and the module from the puppet master.

As we can see the catalog is finished and the Mysql server service is up and running. The allow_virtual parameter can be ignored as that is a deprecation warning by default.

Feel free to test and comment.

Make linux process invisible with new Centos kernel

noreply@blogger.com (Zeljko Milinovic, MSc) — Sat, 27 Sep 2014 11:34:00 -0700

Make linux process invisible with new Centos kernel

Processes carry out tasks within the operating system. A program is a set of machine code instructions and data stored in an executable image on disk and is, as such, a passive entity; a process can be thought of as a computer program in action.

After I have compiled the new version of Centos 3.2 kernel I have decided to test some security features that this version offers. How to check which kernel version you have installed, well easy:

[root@centos01 ~]# uname -r

3.2.48

[root@centos01 ~]#

As many other Linux servers, they run in a multi-user enviroments. That means that every user are using shared hardware and software resources of the server. From a security stand point of view, informations of user/usage processes ownership is not relevant for every user to see it. To prevent these informations to every shared resource we are going to tamper a little bit with the /proc filesystem. So if you have the Centos 3.2+ kernel compiled and installed on your test or production machine you can develope this situation further.

The task is simple, all we have to configure is the /proc file system mount with new security options, so that reading of every process can be delegated only to the owner of the process. The new option that we are going to introduce is hidepid.

We have three options available:

hidepid=0 - anyone can read the /proc/pid files

hidepid=1 - this option prevents users to access /proc directories , except of their own. Important

background tasks of the server are now prevented to be shown.

hidepid=2 - this option is an addition to the option 1 , with more security, denying everybody the information about the running processes. Now an intruder is not able to list sensitive data.

Before setting this security options we had a normal situation where a local user could read all of the root and system processes informations.

To continue setting this to prevent users information leakage we have to type further commands:

mount -o remount,rw,hidepid=2 /proc

To have the configuration over a rebooted server we have to update the FSTAB file.

vi /etc/fstab

And we have to add the following info to the file:

proc /proc proc defaults,hidepid=2 0 0

Save and update the file.

This is all to it. Log into a stanard user and use the following command to list the processes:

ps -ef

As a standard user you should not be able to see the processes from other users and applications.

Feel free to test and comment.

Using tcpdump with Linux

noreply@blogger.com (Zeljko Milinovic, MSc) — Sat, 16 Aug 2014 11:36:00 -0700

Using tcpdump with Linux

I find tcpdump as a very powerfull and useful tool to sniff network traffic from a Linux box. It is independent on the distro you are working and very easy to learn. It`s simplicity is inside the command line shell and can be very useful for remote troubleshooting of server and desktop systems.

It is a built in package that exists in various distros and can be used to capture received and transfered packets over a complete network or only from a host. There are a set of options and switch flags thah can be used with this command.

I will try to demonstrate a couple of them with explanation, the ones that I find useful:

-i any listen on any interface that is available on the system
-n do not resolve hostnames
-c get number of certain packets and then stop (usefull in not getting to much informations)
-e get the Ethernet header along with the capture
-E decrypt IPSEC traffic with providing a password key

Simple usage of tcpdump for viewing packets can be done with a couple of command line options. Whether you would like to go into the details of the packets or only the basic view it can be displayed on the command line.

tcpdump -nS simple communication of packets inside the network

tcpdump -nnvvS more advanced packet view with more verbosity

tcpdump -nnvvXS a more deeper look into the package with the content details (derrived from the GUI)

We could do now an example of a displaying only two packages TCP with and inside deeper view of the content. This simple command will show us a two packets with their content and headers.

tcpdump -nnvXSs 0 -c2 tcp

A more specific network goal is not to have too many traffic displayed on the shell. This leaves the options that are not needed out of the picture, and makes troubleshooting much easier.

To see the traffic derrived from a particular host we can use the following command:

tcpdump host 192.168.1.100

Another useful command is writting a certain type of traffic inside a text file for later troubleshooting:

tcpdump -s 1514 port 21 -w output.txt

And for the final packet show command in this blog, you can use a simple switch only to filter IPV6 traffic:

tcpdump ip6

This is only a small demonstration of this powerfull tool. More can be read through the MAN pages, or from similar sysadmin books. Feel free to share and comment.

Very Secure FTP on Centos Server

noreply@blogger.com (Zeljko Milinovic, MSc) — Fri, 23 May 2014 02:53:00 -0700

Very Secure FTP on Centos Server

For this blog post I will be using a VSFTPD as a fast, realible and very secure SFTP server for transferring data between client and server sites. One notice FTP is inherently insecure. If you must use FTP, consider securing your FTP connection with SSL/TLS. Otherwise, it is best to use SFTP, a secure alternative to FTP.

I have a basic built of Centos 6.5 server that is updated with the latest kernel and important security packages for this example. Once we have configured and installed the SFTP server, you will need a SFTP client application to test the connection. I often use Filezilla or WinScp as a alternative application.

Login via ssh to your Linux server and use the su command to become root and start the package installation. At the picture below this is a simple first step.

After this issues a simple command to install the VSFTPD package on your linux server:

yum install vsftpd

After this command issued you should have the package installed and as an example on the next picture I have shown here.

Also for a public server, should be a FTP connection availlable if the client has no possibility to use the SFTP protocol with this software. Then we can install also these packages.

yum install ftp

Now a ftp server should be installed and default configured as an service on your Centos machine.

BASIC VSFTPD CONFIGURATION

The default file for the configuration of this service is located under the /etc/vsftpd folder. I will use Nano editor to change the settings here.

nano /etc/vsftpd/vsftpd.conf

The first item you should configure , is the option to disable anoymous login:

anonymous_enable = NO

Next one is to enable local user logins with the command below:

locale_enable = yes

Next very important configuration to uncomment and set is the chroot option. This option will make a possibility for users, only to use their dedicated home folders on the server, and not able to traverse over different folders. This is a good security practice.

chroot_local_user=YES

So now bassically these are the most command and important settings to get the server up and running. You can fine tune the other settings , like change the default port, or certificates and etc. But this is not needed in this blog demontstration of basic secure server setttings.

We should now restart the service and make it a startup one on boot time of the Centos server. We achieve this with two simple commands:

service vsftpd restart

chkconfig vsftpd on

That is all to it. Now we can test our connection with a SFTP or FTP client.

I have used the Filezilla with a SFTP protocol and I have succesfully connected to my server via a secure channel.

Up and ready for receiving traffic. Now the users can enjoy security, performance and stability in your network.

Feel free to comment and suggest more topics.

Flushing infected mail traffic from Postfix server

noreply@blogger.com (Zeljko Milinovic, MSc) — Sun, 2 Mar 2014 09:36:00 -0800

Flushing infected mail traffic from Postfix server

Postfix consists of a combination of server programs that run in the background, and client programs that are invoked by user programs or by system administrators.

The Postfix core consists of several dozen server programs that run in the background, each handling one specific aspect of email delivery. Examples are the SMTP server, the scheduler, the address rewriter, and the local delivery server. For damage-control purposes, most server programs run with fixed reduced privileges, and terminate voluntarily after processing a limited number of requests. To conserve system resources, most server programs terminate when they become idle.

Client programs run outside the Postfix core. They interact with Postfix server programs through mail delivery instructions in the user's ~/.forward file, and through small "gate" programs to submit mail or to request queue status information.

Other programs provide administrative support to start or stop Postfix, query status information, manipulate the queue, or to examine or update its configuration files.

If Postfix cannot deliver a message to a recipient it is placed in the deferred queue. The queue manager will scan the deferred queue to see it if can place mail back into the active queue. How often this scan occurs is determined by the queue_run_delay. Postfix will scan the incoming queue at the same time as the deferred queue just to make sure that one does not take all the resources and so each can continue to move messages.

The real question is, What is causing messages to be deferred? One of the major reasons that messages are deferred is that your server is going to place mail to “unknown recipients” into the deferred queue if they do not have a legitimate user to go to.

First thing that should be done to analyze the mails that are stuck in the queue is typing the mailq command. If you see a lot of mails in the queue shown in the output, than something fishy is going on on you server. Just looking on the mails, IT people should recognize the domain that has the most mails in the queue. When you find out the domain example.com than the next step to do is to run a bash script that will delete only those mails that are from the infected domain.

#!/bin/bash

match="$1"

find /var/spool/postfix/deferred/*/ -type f -exec grep -l $match '{}' \; | xargs -n1 basename | xargs -n1 postsuper -d

find /var/spool/postfix/active/ -type f -exec grep -l $match '{}' \; | xargs -n1 basename | xargs -n1 postsuper -d

This simple scripts are using bash language to find the deferred and active mails from the user keyboard input on the CLI. After that the script is executing the postsuper -d command that is flushing the queue with that specific domain.

match="$1" is a simple regex that matched text by the first capturing group, in our case a user inputed domain.

After this the mail queue should be emptied with the infected mails and the server will have some freed up resources. Another faster or simple solution, if the mails are not important at the moment, is to flush the complete queue in the deferred folder.

For this we have a simple command: postsuper -d ALL deferred

Feel free to comment..

Server 2008 R2 as RADIUS for CISCO ASA VPN Clients

noreply@blogger.com (Zeljko Milinovic, MSc) — Sat, 1 Feb 2014 13:44:00 -0800

Server 2008 R2 as RADIUS for CISCO ASA VPN Clients

As in every Enterprise or a private Data Centar network one must use various of IT systems to insure the security of via meshed systems. The other day I implemented a Cisco 5520 Failover scenario and the main problem I had with the users, is how will they manage so many passwords for VPN, AD, Mail and etc. So I thought why not use Kerberos for VPN and simplify the tasks.

This easy done task I will explain as short and much I can. The main goal is to make Cisco ASA failover to use the Active Directory for authenticating the users against VPN policy.

Easiest way to configure ASA quick is using the ASDM utility. I use CLI only for initial interface and http commands , after that all is downstream.

First we need to configure an object:

Using the Firewall section we expand Objects and select IP names. Then click ADD and describe the Radius server. After that we enter the IP address of the Intranet located Domain controller.

Next step is to define a AAA Radius group:

Click the Remote Access VPN section.

Expand AAA Setup and select AAA Server Groups.

Click the Add button to the right of the AAA Server Groups section.

Give the server group a name, like TEST-AD, and make sure the RADIUS protocol is selected.

Accept the default for the other settings.

And click OK.

Next step is to add our RADIUS server to this created group:

Select the server group created in the step above.

Click the Add button to the right of Servers in the Select Group.

Under the Interface Name select the interface on the ASA that will have access to the RADIUS server, most likely inside.

Under Server Name or IP Address enter the IP Name you created for the RADIUS server above.

Skip to the Server Secret Key field and create a complex password. Make sure you document this as it is required when configuring the RADIUS server. Re-enter the secret in the Common Password field.

Leave the rest of the settings at the defaults and click Ok.

To enable RADIUS on Server 2008 we must add a role:

Connect to the Windows Server 2008 server and launch Server Manager.

Click the Roles object and then click the Add Roles link on the right.

Click Next on the Before You Begin page.

Select the Network Policy and Access Services role and click Next.

Under Role Service select only the Network Policy Server service and click Next.

Click Install.

After launching the NPS tool right-click on the entry NPS(Local) and click the Register Server in Active Directory. Follow the default prompts.

We need to define a Radius CLIENT on Server 2008 for our ASA Cluster:

Right-click on RADIUS Clients and select New RADIUS Client.

Create a Friendly Name for the ASA device. I used “CiscoASA” but if you had more than one you might want to make it more unique and identifiable. Make sure you document the Friendly Name used as it will be used later in some of the policies created.

Enter the Server Secret Key specified on during the ASA configuration in the Shared secret and Confirm shared secret field.

Leave the default values for the other settings and click OK. See Figure 1 for all the complete RADIUS Client properties.

Connection Request Policy

Expand the Policies folder.

Right-click on the Connection Request Policies and click New.

Set the Policy Nameto something meaningful. I used CiscoASA because this policy is geared specifically for that RADIUS client. Leave the Type of network access server as Unspecified and click Next.

Under Conditions click Add. Scroll down and select the Client Friendly Name condition and click Add…

Specify the friendly name that you used when creating the RADIUS Client above. Click OK and Next.

On the next two pages leave the default settings and click Next.

Under the Specify a Realm Name select the Attribute option on the left. From the drop down menu next to Attribute: on the right select User-Name. Click Next again.

Review the settings on the next page and click Finish.

Create a Network Policy

Right-click the Network Policy folder and click New.

Set the Policy Name to something meaningful. Leave the Type of network access server as Unspecified and click Next.

Under Conditions click Add.

Add a UsersGroup condition to limit access to a specific AD user group. You can use a generic group like Domain Users or create a group specifically to restrict access.

Add a Client Friendly Name condition and again specify the Friendly Name you used for your RADIUS client.

Click Next. Leave Access granted selected and click Next again.

(Important Step) On the authentication methods leave the default selection and add Unencrypted authentication (PAP, SPAP).

Accept the default Constraints and click Next.

Accept the default Radius Settings and click Next. Review the settings and click Finish.

Restart the Network Policy Server service.

The last thing left is to Test and Save the config.

If necessary re-launch the ASDM utility.

Return to Configuration -> Remote Access VPN -> AAA Setup -> AAA Server Groups.

Select the new Server Group you created.

From the Servers in the Selected Group section highlight the server you created. Click the Test button on the right.

Select the Authentication radio button. Enter the Username and Password of a user that meets the conditions specified in the Network Policy created above then click OK.

Feel free to comment.

Security ethical hacking checklist

noreply@blogger.com (Zeljko Milinovic, MSc) — Mon, 6 Jan 2014 13:28:00 -0800

Security ethical hacking checklist

We can define an Ethical Hacking Expert as a person who has many skills in many segments of Information Technology. On behalf of the owners of the Information Technology Systems an expert in Network and Systems attacks the complete organization. The main goal is to find vulnerabilities that would a malicious person use to exploit to gain important informations.

These tasks include Penetration testing, risk assessment and intrusion testing. Many companies also involve some code reviewers to scan the web application code. Complete set of tasks that are involved in, for example a penetration test, are useful to find weaknesses in open source code that is used often in application development. Developers often are too busy in creating the applications and contributing to the opensource community, so the security concerns are not always highlighted.

These situations require an expertise from people that can objectively look at the code and to verify the completion of the cycles needed for application implementation.

I will explain some basic steps in my White Hat general checklist that every IT security concerned people should know and use.

RECONNAISSANCE

This is a military term that was used to seek out the intentions and plans of the enemy, using various methods to find out capabilities and composition of the enemy. In ethical hacking world this word is used for information gathering of the target. This is useful to find the weakest spot in the target Information Technology system to exploit and use it for the final goal.

There is also another side of footprinting that is used for protecting the system instead of attacking it. First of the basic methods of information gathering is:

PING the remote target system to gather basic IP info.

For example Start-Run-CMD> ping www.google.com

Or some other range of public IP addresses to see if there are some hosts that are alive on the other side. This is a good starting point for every information gathering.

PORT scanning of the remote services running on the target system. These TCP scans can be individual or we can scan a range of ports to identify different services. We can use a great tool found on www.nmap.org

An example of a command line scan: NMAP -T4 -A -v scanme.nmap.org

Target public information such as company info, telephone numbers, email addresses and many other are very useful to create a big picture. This can be done via whois lookups of the company domain and gathering info from DNS protocol. There are some great online sites for this http://www.uwhois.com/

EMAIL tracking is a good way to analyze the email header which will provide us the informations on the IP stack of the mail servers and other gateway functionalities. A good application that can be used for this is the EmailTracker that can be found on http://www.emailtrackerpro.com/

Network Connections from your computer or from the target systems can be useful to find out incoming and outgoing connections that are persistent and important for the target users. A free command utility called netstat is the best and fastest way to achieve this.

An example command line: netstat -ano

Explore the internet libraries to find out the history of the web page is sometimes important. Some URLs are useful in this information gathering.

https://archive.org/

http://www.thememoryhole.org/

Company location can be found using Google Earth. This is important if the company has IT storage rooms in many countries and from a network standpoint, a clear picture on GeoIP locations.

Network nodes displaying will show us the information on various path we can get to the target system. This is useful to find out the most optimized way to enumerate services in the target systems. A cool application for this is called NeoTrace and can be found on this link: http://neotrace-pro.en.softonic.com/

DNS Enumeration

By Enumerating DNS it is possible to get some important public (May be sometime Private information too) information such as Server name, Server IP address, Sub-domain etc.

Useful PERL script called dnsenum.pl can be found on this URL

SCANNING

In general we have three types of scanning:

Port scanning
Network scanning
Vulnerability scanning

Active information gathering produces more details about your network and helps you see your systems from an attacker’s perspective. We can see which server systems are alive and what services they are providing for the target users. The important fact is the system operating systems and the architecture that they are using.

I can number some types of PORT scanning methods:

- Vanilla: the scanner attempts to connect to all 65,535 ports

- Fragmented packets: the scanner sends packet fragments that get through simple packet filters in a firewall

- UDP: the scanner looks for open UDP ports

- Sweep: the scanner connects to the same port on more than one machine

- Stealth scan: the scanner blocks the scanned computer from recording the port scan activities

Network Scanning is the process of examining the activity on a network, which can include monitoring data flow as well as monitoring the functioning of network devices. Network Scanning serves to promote both the security and performance of a network. Network Scanning may also be employed from outside a network in order to identify potential network vulnerabilities.

Vulnerability scanning employs software that seeks out security flaws based on a database of known flaws, testing systems for the occurrence of these flaws and generating a report of the findings that an individual or an enterprise can use to tighten the network's security.

SUBNET Information whether public or private is very important for time consuming security testing methods. A very useful tool for gathering the subnet information is the AngryIP application. This application is available on this location http://angryip.org/w/Home

There are some useful tools used for target system scannings:

Mcafee superscan tool

http://www.mcafee.com/us/downloads/free-tools/superscan.aspx

Network port scanning

Scan network ports with NetScanTools Pro or Nmap.

UDP ports scanner, very fast and powerfull WUPS

Unicornscan is an attempt at a User-land Distributed TCP/IP stack for information gathering and correlation

The app can be found on http://www.unicornscan.org/

ENUMERATION

Using the previous gathered information the attacker usually start scanning against the victim such as Port scanning, Banner Grabbing, Vulnerability Scanning, Finding Username/Emails address. This is usually active attack(May get detected by IDs or may get blocked by Firewalls.

Enumeration is the first attack on target network; Enumeration is a process to gather the information about user names, machine names, network resources, shares and services ; Enumeration makes a fixed active connection to a system

Null session - exploitation of Windows SMB communications network protocols.

We can exploit a remote machine without any credentials using: net use \\ip address\\IPC$ ""/u:""

If we have only one authenticated user credentials we can use this exploit for many machines in the domain. Also a good tool for enumeration these weak spots is enum4linux.pl found on this url

NetBios/over/TCP/IP can be used with a integrated tool nbtstat that will display protocol statistics and current TCP/IP connections. We can also provide our information database with the MAC address.

Usage: nbtstat -A 192.168.1.1

FTP Enumeration - a crafty tool on NIX is useful for enumerating the TCP port 21 with useful information like server version and the list of users on the target system.

# perl -MCPAN -e shell

cpan> install Getopt::Std

This is used for the installation. And the usage on the target system

Usage: ftp-user-enum.pl [options] (-u username|-U file-of-usernames) (-t host|-T file-of-targets)

TELNET to a service on different number of ports to see if a service is running on the remote server.

Usage: telnet <IP or FQDN> <port>

List of ports for services can be found on this URL.

A list of some useful tools used for enumeration:

IP Tools

http://www.ks-soft.net/ip-tools.eng/downpage.htm

SoftPerfect Network Scanner is a free multi-threaded IP, NetBIOS and SNMP scanner with a modern interface and many advanced features. It is intended for both system administrators and general users interested in computer security. The program pings computers, scans for listening TCP/UDP ports and displays which types of resources are shared on the network, including system and hidden ones.

http://www.softperfect.com/products/networkscanner/

SomarSoft's DumpSec is a (free) security auditing program for Microsoft Windows NT/2000. It dumps the permissions (DACLs) and audit settings (SACLs) for the file system, registry, printers and shares in a concise, readable format, so that holes in system security are readily apparent. DumpSec also dumps user, group and replication information. DumpSec is a must-have product for Windows NT systems administrators and computer security auditors.

https://www.enisa.europa.eu/activities/cert/support/chiht/tools/dumpsec-dump-windows-acl-and-audit-settings

Enumerate some devices like routers, printers, servers, backup devices and similar with default passwords. Many useful passwords can be found on google searches, and one of the list can be found on this URL.

Netcat is a simple networking utility which reads and writes data across network connections using the TCP/IP protocol. It's a wonderful tool for debugging all kinds of network problems. It allows you to read and write data over a network socket just as simply as you can read data from stdin or write to stdout. I have put together a few examples of what this can be used to accomplish.

Establishing a connection and getting some data over HTTP:#

nc example.com 80

GET / HTTP/1.0

<HTML>

</HTML>

HACKING

When above steps has be done the attacker start exploiting the all found vulnerability which may lead to compromise the System or an website.

There are of four types of password attack:

1. Passive online attack - man in the middle, sniffing and similar

2. Active online attack - password guessing

3. Offline attack - brute force attack, directory attack and hybrid attacks

4. Non technical attack - social engineering

A rootkit is a stealthy type of software, typically malicious, designed to hide the existence of certain processes or programs from normal methods of detection and enable continued privileged access to a computer.[1] The term rootkit is a concatenation of "root" (the traditional name of the privileged account on Unix operating systems) and the word "kit" (which refers to the software components that implement the tool.

Steganography (Listen) is the art and science of encoding hidden messages in such a way that no one, apart from the sender and intended recipient, suspects the existence of the message. It is a form of security through obscurity.

An ethical hacker should equip himself with a database and dictionaries of default password. Some useful URLs can be a good starting point.

http://www.defaultpassword.com/?char=&action=dpl
http://www.cirt.net/passwords
http://www.virus.org/default-password%20

LOPTHCRACK can be used as a useful tool to recover passwords
Can be found on this URL

HACKING Windows Server administrator password is a powerful method of gaining access of target systems. Windows servers use the SAM database to encrypt and store passwords. There are many tools to exploit these passwords. One of them is offline NT password recovery tool that can be found on this URL

Keyloggers are useful software tools that log every keystroke that a user generates on the keyboard. These stealth tools are useful to capture credentials on target systems.
Free versions can be found on this URL More stealthy keyloggers are USB ones that hold the keylogger software on the USB stick, that can be manipulated inside the organization and can send credentials outside the corporate networks. Useful software can be downloaded from this URL

Scapy is a powerful interactive packet manipulation program. It is able to forge or decode packets of a wide number of protocols, send them on the wire, capture them, match requests and replies, and much more.

Openpuff is a opensource stganography tool that can be used to create hidden scripts and apps inside cool extensions like PDF and JPG. Can be found on this URL

Lynis is an auditing tool for Unix/Linux. It performs a security scan and determines the hardening state of the machine. Any detected security issues will be provided in the form of a suggestion or warning. Beside security related information it will also scan for general system information, installed packages and possible configuration errors.

Yersinia is a network hacking tool designed to take advantage of the weaknesses in some network protocols. It pretends to be a framework for analyzing deployed networks and systems. It implements a number of attacks for the following protocols: STP, CDP, DTP, DHCP, VTP, ISL and etc.

The Metasploit Framework is an open-source development platform for creating security tools and exploits. The framework is used to test systems, verify patch installations, and perform regression testing. The framework allows users to configure exploit modules and test systems against attack.

The PsEXEC tool allows white hat people to remote execute applications and processes on target systems. It can launch interactive command prompts on remote computers.
Syntax: psexec \\computer[,computer[,..] [options] command [arguments]

Core Impact is a penetration-testing tool for testing security threats. It allows systems administrators to test
security patches, network infrastructure, and system upgrades before an attacker does. It is frequently updated,so it is likely to stay ahead of new exploits.

Ratproxy is a semiautomated and largely passive Web application security audit tool. It detects and annotates potential problems and security-relevant design patterns based on the observation of existing and user-initiated traffic. It does not generate a high volume of traffic, taking very little bandwidth.

In this blog I tried to create a small checklist of tools I use , and some of the I have skipped (to be continued). Also the checklist methodology I think will be a good starting point for enthusiastic people that are concerned for the security of their IT systems.

To be continued ...

Check status of IMAP server

noreply@blogger.com (Zeljko Milinovic, MSc) — Sun, 29 Dec 2013 05:22:00 -0800

Check status of IMAP server

The Internet Message Access Protocol (commonly known as IMAP) is an Application Layer Internet protocol that allows an e-mail client to access e-mail on a remote mail server. The current version, IMAP version 4 revision 1 (IMAP4rev1), is defined by RFC 3501. An IMAP server typically listens on well-known port 143.

I had configured a Dovecot server with IMAP status, for many users , so I needed a mechanism to check if the server is responding on client requests during the high traffic. I wanted to do this using a Cron job and a simple script. This script will telnet to the IMAP port on the Linux server and check the status every 60 seconds. This is how often I configured the Cron job, it can be configured on every 5 minutes or so.

Now let us take a look on this simple code I wrote:

#!/bin/bash

#http://itstuffallaround.blogspot.com/

#program to check if connection is possible with Dovecot and log errors and success full connections

if telnet localhost 143 </dev/null 2>&1 | grep -q Escape; then

echo "Connected Dovecot on $(date)" >> DOVSTATUS.txt

else

echo "No connection to Dovecot on $(date)" >> DOVSTATUS.txt

The simple BASH language script is constructed of a single loop, that telnets to port 143 and returns the status of the service to a dovstatus.txt file.

The parameter /dev/null 2>&1 was very useful to me, because it will disable returning the on screen prompt for action on telnet, and Escape the login sequence because it is not neccessary, it will redirect both the output and the error streams. Even if your program writes to stderr, that output will not be shown. After this rename the file as .sh and add the execute perrmissions on it. Configure it as a cron job and wait for the results in the dovstatus.txt file.

Feel free to code more!

Improved Linux DDOS detection program

noreply@blogger.com (Zeljko Milinovic, MSc) — Wed, 4 Dec 2013 08:02:00 -0800

Improved Linux DDOS detection program

With a lot of help with some friend on the Linux comunity, I have improved the DDOS detection program on Linux systems. This BASH code gives the IT people possibility to fine list what is currently going on at their servers. And what is more important where from.

The code presented in the following blog is not to be used in loops of any sort becuase it would deny the admin resources to log on.

#Zeljko Milinovic - http://itstuffallaround.blogspot.com/

#Cached lookup of ddos whois IP sockets

#v1.1

#!/bin/bash

cachefile="$HOME/ddostestercache"

# return 0 if address is to be filtered from the processed

filter()

{

case "$1" in

0.* | 127.* | 10.* | 172.1[6-9].* | 172.2[0-9].* | 172.3[0-1].* | 192.168.* | 169.254.*)

return 0

;;

esac

return 1

}

remote_ips()

{

# print only IPv4 addresses

netstat -tun4 | awk '/:/ {gsub(/:.*/,"",$5);print $5}' | sort -n | uniq -c

}

get_country()

{

local country=$(sed -nr "s/^$1 (.*)/\1/p" $cachefile 2>/dev/null)

if [ -z "$country" ];then

# some queries produce multiple lines so for now use only the first line..

country=$(whois "$1" | sed -nr 's/^country:[[:space:]]+(.*)/\1/ip' | head -1)

country=${country:-unknown}

# cache search result for future use

echo "$1 $country" >> $cachefile

# let's not print the text "unknown" to screen

[ "$country" = "unknown" ] && unset country

echo "$country"

}

remote_ips | while read count ip;do

if ! filter $ip;then

echo "$count $ip $(get_country $ip)"

done

And finally the output for the script:

1 50.31.xxx.xxx US

9 98.28.xxx.xxx DK

1 109.12.xxx.xxx BA

As we can see, in the output we have 3 IP addresses, numbered, listed with concurrent connections , and their country origin. This is very useful to detect where from is the attack, and to mitigate fast.

Feel free to code more and comment.

Linux Security script to determine DDOS origin location

noreply@blogger.com (Zeljko Milinovic, MSc) — Sat, 30 Nov 2013 13:21:00 -0800

Linux Security script to determine DDOS origin location

In computing, a denial-of-service attack (DoS attack) or distributed denial-of-service attack (DDoS attack) is an attempt to make a machine or network resource unavailable to its intended users. Although the means to carry out, motives for, and targets of a DoS attack may vary, it generally consists of efforts to temporarily or indefinitely interrupt or suspend services of a host connected to the Internet.

On various Nix server setups we are always exposed to the DDOS attacks from various other similar setups or intended use. Often in some cases our server is used as a Botnet machine to exploit resources on other systems.

These attacks can be verified from the shell in a form of many open sockets from one or more IP addresses. Often these open sockets are more than 150 , which is not normal. Many IT people are using a DDOS prevention scripts to ban those IP addresses. I stumbled upon a request from a friend to write a script that will tell us the Country of attack origin. This was always missing in our troubleshooting.

So I have written a small and useful script, that is a combination of often used Netstat and Whois commands that can be found online. Also similar script code can be found on the internet, and people can adjust the code to their needs. I needed a script that will associate and display the origin of country and the IP socket combination.

Code

#!/bin/bash

{

cat=$( netstat -ntu | grep ':' | awk '{print $5}' | sed 's/::ffff://' | cut -f1 -d ':' | sort | sort -nr | less);

for i in $cat; do

Country=$( whois $i | grep -i Country | awk '{print $2}' );

echo "Land+IP= $Country $i ";

done;

}

end of code.

To elaborate more on code I will explain the details. I am using the Bash shell scripting, which is very common. This code is using a Netstat command from the classic and tuned ddos Deflate script that is common for fighting DDOS attacks.

netstat -ntu | grep ':' | awk '{print $5}' | sed 's/::ffff://' | cut -f1 -d ':' | sort | sort -nr | less

This command gives us the output of IP sockets and we print them out using the AWK for text processing. I have attached the sed switch to replace the empty addresses and space with null ffff value.

The sort and less switches are helpful for sorting and properly displaying the addresses. I have put this into on CAT function and defined this concencated output with a $ sign as a variable.

The variable is further used for a loop that is needed for the WHOIS command which will tel use the Country of origin. Classic for loop is using a i for the increment value.

Country=$( whois $i | grep -i Country | awk '{print $2}' );

If we use the whois command with the grep function for the Contry it will only display us the Country of origin. So I have used this command with the incremented concencated display in the loop.

Simple enough we get a display of current IP sockets with Country of origin:

Land+IP= BA 71.222.xxx.xxx

Land+IP= IT 88.138.xxx.xxx

Land+IP= BA 61.38.xxx.xxx

So this output will generate all the Sockets, and if we see many exact same sockets from one Country we can pinpoint the location and origin from the attack. Script can be more fine tuned so everyone is welcome.

Feel free to code.

Configure Corefig as a free managament tool for Hyper-V 2012

noreply@blogger.com (Zeljko Milinovic, MSc) — Mon, 18 Nov 2013 06:26:00 -0800

Configure Corefig as a free managament tool for Hyper-V 2012

Reading a lot of articles online I have found that Hyper-V 2012 has some cool new features that are free to use. Some of them are in battle with VMware, like HA and the new SMB 3.0 protocol. I have installed a nested Hyper-V 2012 under the VMware setup to test the management tools.

One tool I have found as a useful collection of PowerShell scripts is the Corefig. Here are some steps to use with this management tool and how to configure it.

On a fresh install of the Hyper-V Hypervisor one should enable the Remote Managament under the initial powershell options. To copy the files from the downloaded Corefig site firewall should be disabled on the Hyper-V 2012. This can be done via a simple command using netsh.

The next step is to download the Corefig.zip file and copy it to the Hyper-V hypervisor. One can copy the files using the Netbios protocol, and simply typing the \\hyper-vsrv location and creating a folder under the root of the Hypervisor called Corefig.

The link for the Corefig installation can be found here.

After extracting the files , we should start the Powershell script to initialize the Corefig installation process and to make it as a startup service. This can be done via a simple command:

CD C:\COREFIG

POWERSHELL .\COREFIG.PS1

Soon after that we have an instance of the Corefig started and can use all of the managament functions it offers us. A simple screenshot will show the GUI of the tool.

We can easily change the network settings, a small Control Panel utilities, and general Hyper-V settings. One great security tool is to manage the firewall via the GUI is easy for creating the first initial rules. I have disabled the firewall for testing purposes.

Acording to Microsoft this tools is verified to work with these setups:

Verified: Microsoft Windows Server 2012 (Core Installation)
Verified: Microsoft Windows Server 2012 (Complete GUI Installation)
Verified: Microsoft Hyper-V Server 2012

Feel free to use the tool and comment on it.

Windows Server 2008 PKI Single Tier CDP

noreply@blogger.com (Zeljko Milinovic, MSc) — Mon, 18 Nov 2013 00:06:00 -0800

Windows Server 2008 PKI Single Tier CDP

In cryptography, a PKI is an arrangement that binds public keys with respective user identities by means of a certificate authority (CA). The user identity must be unique within each CA domain. The third-party validation authority (VA) can provide this information on behalf of CA. The binding is established through the registration and issuance process, which, depending on the assurance level of the binding, may be carried out by software at a CA or under human supervision.

Active Directory Certificate Services (AD CS) is an Identity and Access Control security technology that provides customizable services for creating and managing public key certificates used in software security systems that employ public key technologies.

A system or systems where the CRL (Certificate Revocation List) is placed for retrieval by Relying Parties or others throughout the PKI environment. A CDP should be referenced in each Certificate so that Relying Parties can readily check the CRL before relying on the Certificate. Most CDPs are accessible via HTTP or LDAP.

In this small setup we have a Windows Server 2008 R2 with following rolles installed :

Active Directory Certificate Services
Active Directory Domain Services
DNS Server
Web Server (IIS)

We have a client Windows 7 desktop machine that is joined to the domain. We want to test if the machine has got a certificate for negotiating the authentification and other domain procedures. And also we want to ensure that the AutoEnrollement is turned so that every other machine in the domain will do this automatically.

After installing the roles we should create a Certificate Authority policy file as a template for all the other certificates and save it under the c:\windows folder as a CaPolicy.inf.

[Version]

Signature="$Windows NT$"

[PolicyStatementExtension]

Policies=InternalPolicy

[InternalPolicy]

OID= 1.2.3.4.1455.67.89.5

Notice="Legal Policy Statement"

URL=http://pki.corp.local/cps.txt

[Certsrv_Server]

RenewalKeyLength=2048

RenewalValidityPeriod=Years

RenewalValidityPeriodUnits=10

LoadDefaultTemplates=0

AlternateSignatureAlgorithm=1

To see the location of the CDP we point ourselves to the Start>AdministrativeTools>Certification Authority

No to ensure that all the PCs in the Active Directory domain called corp.local enroll these certificates we should modify the default domain Group Policy. This can be done via the gpmc.msc policy command.

To review the Cetificate Enrollment we should checkout the local GPO settings on the client machine. This can be done using the MMC console on the Client Windows 7 machine. The Snapint is the Certificate Authority to manage all the local certificates.

We can see that we have enrolled the certificate from the DC1 that is our Domain Controller. And the last thing to see is the purpose of the certificate.

We can see that we have got the All Issuance Policies certificate installed. This also means that the Windows 7 recognized the OID numbers from the CaPolicy.inf file. To research further one can use the Microsoft Technet for other CA purposes.

Feel free to comment.

Compile Source of Apache/MySQL/PHP on a Linux VPS

noreply@blogger.com (Zeljko Milinovic, MSc) — Wed, 13 Nov 2013 12:47:00 -0800

Compile Source of Apache/MySQL/PHP on a Linux VPS

Linux IT Engineers often use the Debina APT, or RedHat YUM repositories for an quick and easy install of the services on their servers. But , in some cases we often need to test the latest packages. For example the latest version of MySQL is 5.7 and we cannot get it via the apt, we have to manually download and install in on our Virtual Private Server. Then we can configure it to our production enviroment.

I have configured the VPS server with the 12.04 LTS versions. I prefer the LTS version because of the support for the security and long term update.

First we start with creating the sources folder and downloading the Apache packages that are needed for our web server:

sudo mkdir /usr/src/sources

wget http://httpd.apache.org/dev/dist/httpd-2.4.2.tar.gz

tar xvfz httpd-2.4.2.tar.gz

After downloading the httpd packages and extracting them we can move on further in our process. We now need the APR utilities and the APR package itself. The APR stands for Apache Portable Runtime.

wget http://apache.spinellicreations.com//apr/apr-1.4.8.tar.gz

tar -xzf apr-1.4.8.tar.gz

rm apr-1.4.8.tar.gz

cd apr-1.4.8/

sudo apt-get install make

sudo ./configure

sudo make

sudo make install

Then we need the APR Utils to be configured.

wget http://mirrors.axint.net/apache//apr/apr-util-1.4.1.tar.gz

tar -xvzf apr-util-1.4.1.tar.gz

cd apr-util-1.4.1

./configure --with-apr=/usr/local/apr

make

make install

cd ..

Now we can return to the HTTPD folder to compile and install the Apache:

cd /usr/local/sources/httpd-2.4.2

./configure --enable-file-cache --enable-cache --enable-disk-cache --enable-mem-cache --enable-deflate --enable-expires --enable-headers --enable-usertrack --enable-ssl --enable-cgi --enable-vhost-alias --enable-rewrite --enable-so --with-apr=/usr/local/apr/

make

make install

cd ..

To startup the Web server we will create a soft link to a startup script and copy the startup script to the init.d folder for startup options.

ln -s /usr/local/apache2/bin/apachectl /usr/bin/apachectl

cp /usr/local/apache2/bin/apachectl /etc/init.d

update-rc.d apachectl defaults

Now we can reboot the server and check if it is running. And we can see that the daemon is running.

root@ubsrv1:~# ps aux | grep httpd

root 1063 0.0 0.2 0:00 /usr/local/apache2/bin/httpd -k start

daemon 1065 0.0 0.2 0:00 /usr/local/apache2/bin/httpd -k start

daemon 1066 0.0 0.2 3 0:00 /usr/local/apache2/bin/httpd -k start

Next what we should do is to continue with the PHP support and installation.

cd /usr/src/sources

wget http://us2.php.net/get/php-5.5.5.tar.gz/from/ar2.php.net/mirror

tar xfvz php-5.5.5.tar.gz

cd php-5.5.5

./configure --prefix=/var/www/ --with-apxs2=/var/apache2/bin/apxs --with-config-file- path=/var/www/php --with-mysql

make

make install

The --prefix folder depends on the folder where you installed the apache, you can change this to your needs. And the final step is to install the MySQL server.

groupadd mysql

useradd -r -g mysql mysql

cd /usr/src/sources

wget http://dev.mysql.com/get/Downloads/MySQL-5.6/mysql-5.6.14.tar.gz/from/http://cdn.mysql.com/

tar zxvf mysql-5.6.14.tar.gz

ln -s /usr/src/sources/mysql-5.6.14 /usr/local/mysql

cd /usr/local/mysql

chown -R mysql .

chgrp -R mysql .

scripts/mysql_install_db --user=mysql

chown -R root .

chown -R mysql data

Here we have a simply longer procedure. First we create a user and group. Then download the source files, extract them, create a soft link and run the mysql_install_db script. Add the permissions to the folders and that is all.

Now we can restart the server and everything should work fine. If not check out some cool tutorials on Ubuntu community.

Source http://community.ubuntu.com/

Linux SWAP Partition as twice the RAM size - why ?

noreply@blogger.com (Zeljko Milinovic, MSc) — Mon, 11 Nov 2013 02:29:00 -0800

Linux SWAP Partition as twice the RAM size - why ?

Linux divides its physical RAM (random access memory) into chucks of memory called pages. Swapping is the process whereby a page of memory is copied to the preconfigured space on the hard disk, called swap space, to free up that page of memory. The combined sizes of the physical memory and the swap space is the amount of virtual memory available.

Swapping is necessary for two important reasons. First, when the system requires more memory than is physically available, the kernel swaps out less used pages and gives memory to the current application (process) that needs the memory immediately. Second, a significant number of the pages used by an application during its startup phase may only be used for initialization and then never used again. The system can swap out those pages and free the memory for other applications or even for the disk cache.

To see a VPS machine with 512 MB of physical RAM and the ratio of SWAP space we can use free -m

The picture shows us that we have total of 490 MB of physical ram and twice the size of the SWAP memory on the hdd (which is not yet used) of 991 MB. As this server has small amount of free memory (only 76MB) I had to investigate further. I have used the HTOP utility to see the real memory consumer.

The process with the ID of 1310 is using some memory resources dynamically. We cannot see the real proccess name, because it is assigned to multiple instances of one application. To investigate further we should use the PID number to see which service is killing the VPS machine.

This can be done with the PMP command : pmap -x 1310

The output shows us that the SAMBA libraries are attached to the PID. Simply stopping the SAMBA service I will free up some memory.

To get back on the inital question , I will try to have a short explanation of the SWAP size. The memory hirearchy presented to application by the Linux system is arranged in few levels:

Processor/CPU registers - bits in size
L1 Cache - kbits in size
L2 Cache - MBs in size
L3 cache - 100 of MBs in size
RAM - GBs in size

The data that is used in application loading is moved to RAM, and some of that data need the L3 cache also. The used application data is often moved to the faster L2 and L1 cache memory. As we can see the data is moved from the RAM up to the last CPU register table. In this order we can get that the SWAP data should be between 1.5 and 2 times the Actual RAM is.

This is the main reason why we should create the SWAP twice the RAM. And applications should not allocate that data to other HDDs, especially large ones, because of the slow I/O operations. If your RAM is free then there is no use of swap partition.

Feel free to comment.

Linux service security - Deny Hosts

noreply@blogger.com (Zeljko Milinovic, MSc) — Sun, 10 Nov 2013 01:28:00 -0800

Linux service security - Deny Hosts

DenyHosts is a log-based intrusion prevention security tool for SSH servers written in Python. It is intended to prevent brute-force attacks on SSH servers by monitoring invalid login attempts in the authentication log and blocking the originating IP addresses.

DenyHosts checks the end of the authentication log for recent failed login attempts. It records information about their originating IP addresses and compares the number of invalid attempts to a user-specified threshold. If there have been too many invalid attempts it assumes a dictionary attack is occurring and prevents the IP address from making any further attempts by adding it to /etc/hosts.deny on the server.

To install and configure the DenyHosts we should use the EPEL repository. A simple BASH command:

yum --enablerepo=epel install denyhosts

After a successfull installation we should take a first look at the configuration file to allow certain secure IP addresses to log into the server console.

nano /etc/hosts.allow

I have added a Local Area Network IP address to have access to the SSH service. All other IP addresses are blocked by default to log into the ssh server.

Optionally and IT admin can use the /etc/denyhosts.conf file to create email alerts if a user tries to log on to the server from a different IP address.

To comply to the setting we should now restart the denyhosts service and add it as a startup script.

chkconfig denyhosts on

service denyhosts start

To see the logs on tried and failled logins , or a simulated attack we should tail a log file:

tail -f /var/log/secure

We see that we have an Accepted password from our IP address that we allowed.

If you’ve list of static IP address that you want to whitelist permanently. Open the file /var/lib/denyhosts/allowed-hosts file. Whatever IP address included in this file will not be banned by default.

Feel free to comment.

Soft File links in Linux

noreply@blogger.com (Zeljko Milinovic, MSc) — Thu, 7 Nov 2013 11:18:00 -0800

Soft File links in Linux

A symbolic or “soft” link points to a file by name. When the kernel comes upon a symbolic link in the course of looking up a pathname, it redirects its attention to the pathname stored as the contents of the link. The difference between hard links and symbolic links is that a hard link is a direct reference, whereas a symbolic link is a reference by name. Symbolic links are distinct from the files they point to.

Symbolic links operate transparently for most operations: programs that read or write to files named by a symbolic link will behave as if operating directly on the target file. However, programs that need to handle symbolic links specially (e.g., backup utilities) may identify and manipulate them directly.

To create a symbolic link in Unix or Linux, at the shell prompt, enter the following command:

ln -s {target-filename} {symbolic-filename}

So let us make a simple example of a index.php file in the web server directory.

The file /home/guru/index.php could then be moved elsewhere without causing the symbolic link to stop working (not that moving this directory is advisable).

It is a common mistake to think that the first argument to ln -s is interpreted relative to your current working directory. However, it is not resolved as a filename by ln; it’s simply a literal string that becomes the target of the symbolic link.

After creating the symbolic link, it may generally be treated as an alias for the target. Any file system management commands (e.g., cp, rm) may be used on the symbolic link. Commands which read or write file contents will access the contents of the target file. The rm (delete file) command, however, removes the link itself, not the target file.

HTOP Interactive process viewer on Linux

noreply@blogger.com (Zeljko Milinovic, MSc) — Thu, 7 Nov 2013 04:11:00 -0800

HTOP Interactive process viewer on Linux

Htop is an interactive system-monitor process-viewer written for Linux. It is designed to replace the Unix program top. It shows a frequently updated list of the processes running on a computer, normally ordered by the amount of CPU usage. Unlike top, htop provides a full list of processes running, instead of the top resource-consuming processes. Htop uses color and gives visual information about processor, swap and memory status.

Users often deploy htop in cases where Unix top does not provide enough information about the systems processes, for example when trying to find minor memory leaks in applications. Compared to top, it provides a more convenient, cursor-controlled interface for killing processes.

It is a very simple forward package installation on Centos server.

yum install htop

And that is all to it. You start the the application just by typing the htop command in shell. Let us take a look at the interface.

You can use a cool features for filtering and killing the processes you think that are using up the resources on the server. I often use the SortBy command to sort the processes by either the CPU or the MEMORY usage.

Have fun!!!

Source HTOP