micharg

Patched Nagios scripts to monitor top 5 processes using CPU / memory

2019-05-11T13:40:00.002-07:00

Overview

Unfortunately the Nagios scripts (plugins) provided in https://exchange.nagios.org/directory/Plugins/System-Metrics/CPU-Usage-and-Load/CPU-Stats-2FMemory-Used-Plugin--2D-Top-5-CPU-2FMemory-Consuming-Processes-for-Unix-Servers/details are not maintained since 2012 and do not work in Linux (tried with CentOS 7.x). I provide later on this page their patched versions for Linux (tried with CentOS 7.x but it should be OK for any Linux version - please confirm / or not).

Prerequisites

Install the following:
# yum install bc ksh sysstat

Patched Nagios scripts

Patched files working on CentOS 7.x:

JTable: overriding default behaviours

2015-11-26T01:56:00.003-08:00

JTable and Tab/Shift-Tab

By default when the focus in in a JTable then Tab changes focus to the next cell. If the cell is the last one then the 1st cell is focused. Shift-Tab does the opposite.

disable Tab/Shift-Tab

myJTable.getInputMap(JComponent.WHEN_ANCESTOR_OF_FOCUSED_COMPONENT).put(
    KeyStroke.getKeyStroke(KeyEvent.VK_TAB, 0), "none");
myJTable.getInputMap(JComponent.WHEN_ANCESTOR_OF_FOCUSED_COMPONENT).put(
    KeyStroke.getKeyStroke(KeyEvent.VK_TAB, InputEvent.SHIFT_DOWN_MASK), "none");

moving to next component

Assuming we want to move out of the JTable to the next/prev UI component (instead of the next/prev cell) we can do the following to modify the default behaviour:

Set<AWTKeyStroke> forward = new HashSet<AWTKeyStroke>(
        myJTable.getFocusTraversalKeys(KeyboardFocusManager.FORWARD_TRAVERSAL_KEYS));
forward.add(KeyStroke.getKeyStroke("TAB"));
myJTable.setFocusTraversalKeys(KeyboardFocusManager.FORWARD_TRAVERSAL_KEYS, forward);
Set<AWTKeyStroke> backward = new HashSet<AWTKeyStroke>(
        myJTable.getFocusTraversalKeys(KeyboardFocusManager.BACKWARD_TRAVERSAL_KEYS));
backward.add(KeyStroke.getKeyStroke("shift TAB"));
myJTable.setFocusTraversalKeys(KeyboardFocusManager.BACKWARD_TRAVERSAL_KEYS, backward);

// create the policy which determines what will be the next/prev UI component on Tab/Shift-Tab
class TabFocusTraversalPolicy extends DefaultFocusTraversalPolicy {

    private static final long serialVersionUID = 1L;

     @Override
    public Component getComponentAfter(Container container, Component component) {
        Component nextComponent = null;

        if(...) {
            nextComponent = ...;
        } else if(component == ...) {
            nextComponent = ...;
        } else {
            nextComponent = super.getComponentAfter(container, component);
        }

        return nextComponent;
    }

     @Override
    public Component getComponentBefore(Container container, Component component) {
        Component nextComponent = null;

        if(...) {
            nextComponent = ...;
        } else if(...) {
            nextComponent = ...;
         } else {
            nextComponent = super.getComponentBefore(container, component);
        }

        return nextComponent;
    }

        @Override
        public Component getDefaultComponent(Container container) {
            return super.getDefaultComponent(container);
        }

        @Override
        public Component getFirstComponent(Container container) {
            return super.getFirstComponent(container);
        }

        @Override
        public Component getLastComponent(Container container) {
            return super.getLastComponent(container);
        }

}

// install the policy to the JTable
myTable.setFocusCycleRoot(true); // if not true then the policy is not triggered
myTable..setFocusTraversalPolicy(newPolicy);

references:

StackOverflow

JTable and Enter

Enter: selects next row

By default when we click Enter in a JTable where a row is selected then the next row gets the focus.

In order to disable this do the following:
myJTable.getActionMap().put("Enter", new AbstractAction() {
        private static final long serialVersionUID = 1L;

        @Override
        public void actionPerformed(ActionEvent ae) {
            // override default behaviour which selects the next row
        }
});

How to sort an Array of Strings

2015-09-12T10:48:00.001-07:00

This is a silly mistake I did..

Don't use
Arrays.sort(strs.toArray(new String[0]));

Instead use the following:
Collections.sort(strs);

MongoDB: The ugly things

2015-09-09T00:20:00.003-07:00

Here is a list of all the ugly things that I have found so far regarding MongoDB:

Writes that return errors to the client due to wtimeout may in fact succeed (link)

Regarding the option wtimeout:

This option specifies a time limit, in milliseconds, for the write concern. wtimeout is only applicable for w values greater than 1.
wtimeout causes write operations to return with an error after the specified limit, even if the required write concern will eventually succeed. When these write operations return, MongoDB does not undo successful data modifications performed before the write concern exceeded the wtimeout time limit.
If you do not specify the wtimeout option and the level of write concern is unachievable, the write operation will block indefinitely. Specifying a wtimeout value of 0 is equivalent to a write concern without the wtimeout option.

WTF?

Maximum document size is 16MB (link)

The maximum BSON document size is 16 megabytes.

A client may think that a write was successful but it wasn't!

Even with (w=1 or w='majority') and (j=1), it is possible to wind up rolling back a committed write to the primary on failover (link) and the client may think that the write was successful but it wasn't!

Examples

The network TCP connection between the application and the server was reset after the server received a write but before a response could be sent.
The MongoDB server terminates between receiving the write and responding to it.
The network fails between the time of the write and the time the client receives a response to the write.

w='majority' doesn't resolve the problem because the parameter j doesn't affect the replicas (link):

Requiring journaled write concern in a replica set only requires a journal commit of the write operation to the primary of the set regardless of the level of replica acknowledged write concern.

You can find more information on the official MongoDB docs and a relative StackOverflow post.

A client doesn't get a response after a write and thinks that the write didn't happen but happened!

Usually the MongoDB client (e.g. Java driver) tackles the following problems transparently to the developer:

insert isn't acknowledged --> re-execute it (provided the _id was set by the client and is the same on both executions)
update isn't acknowledged --> treat it as deletion and insertion
deletion isn't acknowledged --> idempotent; re-execute it
find isn't acknowledged --> idempotent; re-execute it

During failover writes cannot occur (i.e. they fail)

When the primary node fails, another is elected from its replica set. During the election the client cannot write. This is because in MongoDB only the primary node can accept writes.

Therefore you may have to guard your code and try to re-execute the failed write.
By the way can the MongoDB driver do anything about it?

How to hibernate Gnome Shell

2014-12-20T06:55:00.003-08:00

Install the relative plugin found on https://extensions.gnome.org/extension/755/hibernate-status-button/

Oracle Database error: ORA-01109: database not open

2014-11-07T00:18:00.001-08:00

Oracle db installed on VM / my host computer powered off => VM halted
When trying to run a SQL script I got "ORA-01109: database not open"

More info.. starting SQLPLUS:
SQL> shutdown normal;
ORA-01109: database not open

SQL> startup;
ORACLE instance started.
Database mounted.
ORA-00600: internal error code, arguments: [kcratr_scan_lastbwr], [], [], [],
[], [], [], [], [], [], [], []

SQL> recover database;
Media recovery complete.

SQL> shutdown normal;
ORA-01109: database not open
Database dismounted.
ORACLE instance shut down.

SQL> startup;
Database opened.

Virtualbox: How to mount shared folder

2014-11-06T04:44:00.000-08:00

create shared folder on host:
Settings / Shared Folders / Adds a new shared folder definition. /
Folder Path: C:/shared
Folder Name: shared
mark: "Auto-mount" and "Make Permanent"

Manually mount shared folder:
mount shared folder on guest os (linux):

# mkir /mnt/shared
# usermod -G vboxsf -a oracle
# mount.vboxsf shared /mnt/shared

Auto-mount shared folder on start-up:
# mkir /mnt/shared
/etc/fstab:
shared /mnt/shared vboxsf defaults 0 0

add user xxx to sudo

2014-11-06T03:38:00.002-08:00

Add user micharg to sudo:
$ sudo -i
# visudo
micharg ALL=(ALL) ALL

Virtualbox: copy/paste between host-guest doesn't work (solved)

2014-11-06T03:36:00.002-08:00

Solution (doesn't work):

Virtualbox copy/paste between host-guest doesn't work:
Settings-->Storage-->SATA Controller and 2. 'Solid-state drive' is checked

DIDN'T WORK

Solution #2:

Settings / General / Advanced:
Shared Clipboard: Bidirectional
Drag'n'Drop: Bidirectional

Java 64bit not supported by LibreOffice nor OpenOffice

2014-11-05T07:07:00.000-08:00

Install Java 32bit

Spring Data: How to call a stored procedure

2014-10-11T07:28:00.002-07:00

Spring data reference supports calling stored procedure (from JPA 2.1) but didn't work for me. Therefore I called the stored procedure with a native query:

@Repository
public interface MyEntityRepository extends PagingAndSortingRepository<MyEntity, Long> {

    @Query(nativeQuery=true, value="{call GetFoo()}")
    Object[] callMyStoredProcedure();
}

Details:

1) We are obliged to write the call in an entity repository. Therefore choose an entity in random (e.g. I used MyEntity) and place there your stored procedure call.

2) The stored procedure with name GetFoo() joins some tables, etc, and returns a row. Its columns are stored in the returned Object[].

PS: For Spring Data and Stored Procedures check the following sites:
- http://blogs.isostech.com/database/jpa-spring-data-awesomeness/

Review of book "Apache Camel Developer's Cookbook"

2014-10-05T23:56:00.000-07:00

Spoiler: I have been assigned to review this "Apache Camel Developer's Cookbook" since I won a free copy in a contest on LinkedIn.

This book should be read by developers already familiar with Camel existence (e.g. read first "Camel in Action" - you will learn how Camel works). It references the most Valuable EIPs with a good discussion and examples (and code maven managed!) about each one. It presents EIPs like Content Based Router, Filter, Wire Tap, Multicast, Content Enricher, Recipient List, Throttler, Load Balancer, Routing Slip, Enricher, Splitter, etc. Its follows the common PACKT's presentation format "Getting ready" / "How to do it..." / "How it works..." / "There's more..." / "See also". It felt like reading Camel's official Documentation but from a user guide's approach and not a detailed reference approach.

What I liked most was a whole a chapter on testing. Integration with Spring (a common usage scenario) is also presented throughout the book. I really liked "Chapter 8: Transactions and Idempotency" where more advanced issues are presented about transactions on a single-resource and on how to implement distributed transactions without XA (hint: using Idempotent EIP).

To sum up:

- Don't read it if this is your first contact with Camel.

- Read it if you want good examples on how to use Camel's patterns and official documentation is not enough for you. Source code provides hands-on experience on all the EIPs presented on the book.

SVN: delete all .svn (recursively)

2014-09-23T07:19:00.000-07:00

Windows:
FOR /F "tokens=*" %G IN ('DIR /B /AD /S *.svn*') DO RMDIR /S /Q "%G"

Linux:
find . -type d -name '.svn' -print -exec rm -rf {} \;

Spring Security + @Async problem: (SecurityContextHolder is empty)

2014-09-18T05:23:00.000-07:00

Problem: I annotated a method with @Async and @PreAuthorize so that is executed asynchronously and be secured.
But it seems the security context (SecurityContextHolder) is not populated (althouth user has authenticated).
This happens only if the method is annotated with @Async.

Solution: From SecurityContextHolder we can get e.g. the logged in username (e.g. check this). SecurityContextHolder is saved in
current thread [1]. When we spawn a new thread the current thread's SecurityContextHolder (which is a ThreadLocals)
is not copied/inherited.

In order to inherit it do the following:

    <bean class="org.springframework.beans.factory.config.MethodInvokingFactoryBean">
        <property name="targetClass" value="org.springframework.security.core.context.SecurityContextHolder" />
        <property name="targetMethod" value="setStrategyName" />
        <property name="arguments">
            <list>
                <value>MODE_INHERITABLETHREADLOCAL</value>
            </list>
        </property>
    </bean>

All possible modes:

MODE_THREADLOCAL (default strategy)
MODE_INHERITABLETHREADLOCAL: spawned threads inherit SecurityContext of the parent thread
MODE_THREADLOCAL
SYSTEM_PROPERTY

[1] From the Spring Reference: "By default the SecurityContextHolder uses a ThreadLocal to store these details, which means that the security context is always available to methods in the same thread of execution ..."

Apache CXF+Spring: How to add authentication (dynamic credentials: from db, AD, etc)

2014-09-18T04:58:00.000-07:00

Server exposing Web Service

@WebService
public interface MyWebService {
...

}

@WebService(endpointInterface = "com.micharg.MyWebService")
public class MyWebServiceImpl implements MyWebService {
...
}

public interface WebServiceAuthenticationService {

}

@Service("webServiceAuthenticationService")
public class WebServiceAuthenticationServiceImpl implements Validator, WebServiceAuthenticationService {

    @Autowired
    private AuthenticationService authenticationService;


    @Override
    public Credential validate(Credential credential, RequestData data) throws WSSecurityException {

        String candidateUsername = credential.getUsernametoken().getName();
        String candidatePassword = credential.getUsernametoken().getPassword();

        boolean isUserAuthenticated = authenticationService.authenticate(candidateUsername, candidatePassword);

        if(!isUserAuthenticated ) {
            throw new WSSecurityException("Wrong credentials (" + candidateUsername + ", " + candidatePassword + "); can not execute web service");
        }

        return credential;
    }

}

<bean id="inbound-security" class="org.apache.cxf.ws.security.wss4j.WSS4JInInterceptor">
        <constructor-arg>
            <map>
                <entry key="action" value="UsernameToken" />
                <entry key="passwordType" value="PasswordText" />
            </map>
        </constructor-arg>
</bean>


<jaxws:endpoint id="myWebService" implementor="#myWebServiceImpl" address="/myendpoint">

        <jaxws:properties>
            <entry key="ws-security.ut.validator" value-ref="webServiceAuthenticationService" />
        </jaxws:properties>

        <jaxws:inInterceptors>
            <ref bean="inbound-security" />
        </jaxws:inInterceptors>
</jaxws:endpoint>

Client consuming Web Service

<!-- consume web service located on http://foo.com/myendpoint

<jaxws:client id="myWebServicesClient"
            serviceClass="com.micharg.MyWebService"
            address="http://foo.com/myendpoint">

            <jaxws:outInterceptors>
                <ref bean="outbound-security" />
            </jaxws:outInterceptors>
    </jaxws:client>

<bean class="eu.europa.europarl.apapeople.people.web.ws.WebServicesCallbackHandler" id="webServicesCallbackHandler" />

<bean class="org.apache.cxf.ws.security.wss4j.WSS4JOutInterceptor" id="outbound-security">
        <constructor-arg>
            <map>
                <entry key="action" value="UsernameToken"/>
                <entry key="user" value="dummy password - passwordCallbackClass overrides it"/>
                <entry key="passwordType" value="PasswordText"/>
                <entry key="passwordCallbackRef" value-ref="webServicesCallbackHandler"/>
            </map>
        </constructor-arg>
    </bean>

public class WebServicesCallbackHandler implements CallbackHandler {

    private String username = "...";
    private String password = "...";

    @Override
    public void handle(Callback[] callbacks) throws IOException, UnsupportedCallbackException {
        WSPasswordCallback c = (WSPasswordCallback) callbacks[0];
        c.setIdentifier(username);
        c.setPassword(password);
    }

}

Spring+Apache CXF: How to disable XML payload limits

2014-09-18T04:48:00.000-07:00

Error: "org.apache.cxf.staxutils.DepthExceededStaxException: reach the innerElementCountThreshold:50000"
Solution: Disable XML payload limits

   <!-- disable XML payload limits -->
    <bean class="org.springframework.beans.factory.config.MethodInvokingFactoryBean">
        <property name="targetClass" value="java.lang.System" />
        <property name="targetMethod" value="setProperty" />
        <property name="arguments">
            <list>
                <value>org.apache.cxf.stax.maxChildElements</value>
                <value>1000000</value>
            </list>
        </property>
    </bean>

PS: org.apache.cxf.interceptor.security.DepthRestrictingStreamInterceptor seem not to work

Maven: How to execute main-class

2014-09-12T10:00:00.002-07:00

mvn compile exec:java -Dexec.mainClass=micharg.Main

Excel doesn't render correctly UTF-8 characters when it opens CSVs

2014-08-25T02:34:00.000-07:00

Scenario #1: I create a CSV in UTF-8 encoding with Java, save the character 'è'
but Excel doesn't render it well.

Scenario #2: I create a CSV in UTF-8 encoding with Notepad, save the character 'è'
and Excel renders it well.

Explanation (from Wikipedia)

"BOM use is optional, and, if used, should appear at the start of the text stream.
... the BOM character may also indicate which of the several Unicode representations
the text is encoded in."

"In UTF-16, a BOM (U+FEFF) may be placed as the first character of a file ... " (Java is UTF-16)

"The UTF-8 representation of the BOM is the byte sequence 0xEF,0xBB,0xBF." (Windows is UTF-8)

"many pieces of software on Microsoft Windows such as Notepad will not correctly
read UTF-8 text unless it has only ASCII characters or it starts with the BOM,
and will add a BOM to the start when saving text as UTF-8"

Therefore:
=> Scenario #1: notepad saves a CSV UTF-8 with the BOM U+EFBBBF therefore Excel opens it correctly
=> Scenario #2: but when the CSV UTF-8 is created with Java the Excel doesn't open it correctly

Demo on how to solve the problem on Scenario #1:
Adding the prefix BOM U+EFBBBF on the CSV created programmatically

and now opening the CSV with excel and it is correct!

Code solution for scenario #1 (creating CSV programmatically): add to generated CSV the
BOM suffix U+EFBBBF (for UTF-8 Operating Systems like Windows)

How to do it in Java
Java is UTF-16; therefore you have to write the UTF-16 BOM U+FEFF:

// BOM prefix U+FEFF for exported CSVs so that if the csv is openned with
// excel in UTF-8 OS (e.g. Windows) encoding is ok (e.g. char è)
char CSV_BOM = '\uFEFF';
writer.write(CSV_BOM); // writer is e.g. a BufferedWriter

Note: if you open the exported file with a hex editor like I did on the last image you will see the BOM U+EFBBBF.

Android: How to fake GPS

2014-08-01T02:32:00.003-07:00

How to fake GPS on Android; e.g. when you use the emulator

String mockLocationProvider = LocationManager.GPS_PROVIDER;
Location mockLocation = new Location(mocLocationProvider);
mockLocation.setLatitude(1d);
mockLocation.setLongitude(1d);

MySQL: How to insert blob in table

2014-07-30T13:32:00.002-07:00

In order to insert a blob (image, txt, etc) in table FOO in MySQL you should execute the LOAD_FILE [1] statement:

mysql> insert into FOO(data) values(LOAD_FILE('/path/to/file.txt'));

But in my case NULLs were inserted in column data.

Troubleshooting [2]:
mysql> select load_file('/path/to/file.txt'); -- returns NULL

Solution: file.txt should be owned by mysql user/group (although its permissions are -rw-rw-r--; i.e. granted read access to others)

$ sudo chown mysql:mysql /path/to/file.txt

References

[1] From the manual:

    LOAD_FILE(file_name)

    Reads the file and returns the file contents as a string. To use this function, the file must be located on the server host, you must specify the full path name to the file, and you must have the FILE privilege. The file must be readable by all and its size less than max_allowed_packet bytes. If the secure_file_priv system variable is set to a nonempty directory name, the file to be loaded must be located in that directory.

    If the file does not exist or cannot be read because one of the preceding conditions is not satisfied, the function returns NULL.

    As of MySQL 5.0.19, the character_set_filesystem system variable controls interpretation of file names that are given as literal strings.

[2] Thanks StackOverflow

MySQL error: Cannot delete or update a parent row: a foreign key constraint fails"

2014-07-29T13:34:00.000-07:00

When trying to delete table foo(DROP TABLE foo1;) I got the following error:

Cannot delete or update a parent row: a foreign key constraint fails

As root execute the following to see what in what state is the db:

SHOW ENGINE INNODB STATUS;
------------------------
LATEST FOREIGN KEY ERROR
------------------------
140729 23:16:08 Cannot drop table `mydb`.`foo1`
because it is referenced by `mydb`.`foo2`

This tells us that table foo2 has fk to foo1. So remove the fk in foo2 (set it to null) or drop foo2 (this was the case for me since foo2 was empty -- db was in an inconsistent state).

Squirrel: doesn't support DELIMITER

2014-07-29T12:50:00.001-07:00

Note to myself: don't use DELIMITER in Squirrel; just remove it

Install jar with maven in local repository

2014-07-29T11:42:00.005-07:00

$ ls
foo.jar
$ mvn install:install-file -Dfile=foo.jar -DgroupId=foo-group -DartifactId=foo-artifact -Dversion=1 -Dpackaging=jar

pom.xml:
<dependency>
   <groupId>foo-group</groupId>
   <artifactId>foo-artifact</artifactId>
   <version>1</version>
</dependency>

Aspose.Words problem with fonts (encoding)

2014-07-29T01:26:00.001-07:00

Summary
When using Aspose.Words in order to produce a pdf I had problem with some characters not renderer properly (e.g. ȗ) -- see the attached image where the created PDF does not render properly the character ȗ and it has embedded the Gentium fonts. The problem was that fonts were missing from my Operating System and the default fronts that Aspose.Words uses in this case don't support all the characters.

More Info
Aspose.Words when creates PDF uses True Type fonts. If it doesn't find True Type fonts it uses Gentium font; actually it embeds the fonts in the PDF (see the attached image). Gentium font doesn't support ȗ.

Therefore make sure that you have intalled on your Operating system True Type fonts. The algorithm that Aspose looks for fonts is the following (from here):

Aspose.Words tries to find a font on the file system with an exact font name match.
Next, Aspose.Words tries to find the required font among the fonts embedded in the original document. Some document formats such as DOCX can contain embedded fonts
If Aspose.Words cannot find a font with the exact name match, it will attempt to to use the default font specified under FontSettings.DefaultFontName. If the user has not chosen their own default font then "Times New Roman" is the selected default font that is used. See the How to Set the Default Font used when Rendering topic for further information on setting default font.
If Aspose.Words is unable to locate the font defined under FontSettings.DefaultFontName, it attempts to select the most suitable font from all of the available fonts.
Finally, if Aspose.Words cannot find any fonts on the file system, it renders the document using the free Gentium font that is embedded into the Aspose.Words assembly.

Programmatically setting the fonts path has worked for me:

import com.aspose.words.FolderFontSource;

final String pathToFonts = "PATH TO TRUE TYPE FONTS";

FolderFontSource folderFontSource = new FolderFontSource(pathToFonts, true);
fontSources.add(folderFontSource);
FontSourceBase[] fontSourceBase = (FontSourceBase[])fontSources.toArray(new FontSourceBase[fontSources.size()]);
FontSettings.setFontsSources(fontSourceBase);

Default font locations for various Operating Systems

Unix (Linux, Solaris)
/usr/share/fonts
/usr/local/share/fonts
/usr/X11R6/lib/X11/fonts

Windows
Start | Run --> Fonts

Java: How to remove accents

2014-07-10T07:45:00.000-07:00

How to remove accents in strings [1]

attempt 1 (remove accents) -- with problems:
Normalizer.normalize(str, Form.NFD).replaceAll("\\p{InCombiningDiacriticalMarks}+", "")
ç gives c
' gives '
ê gives e
Ö gives O
ü gives u.
´ gives ´
but Ø gives Ø

Accents are removed! But letters (non-ascii) are not removed.

attempt 2 (Remove non-ascii chars) -- successfully!
Normalizer.normalize(str, Form.NFD).replaceAll("[^\\p{ASCII}]", "")
ç gives c
' gives '
ê gives e
Ö gives O
ü gives u.
´ gives nothing
Ø gives nothing

[1] This may be useful e.g. if you want to give filenames in non-unicode filesystem.