• Slow down of web page loading
• Images use a lot more hard disk space
• Bandwidth consumption grows
• Bad user experience when visiting your site

One blocker to compress images is that you can have lots of them in your site and going one by one to check their properties is simply not an option. In order to find a way around, we can use a bash script and a couple of nice linux libraries, imagemagick and jpegoptim. The first library is used to scale the image to a specific width and height, the second one is used to add compression to the images and reduce the size

In order to install this libraries in the form of linux packages we can use this command in a debian like distribution

sudo apt-get install imagemagick jpegoptim

Next, we show the bash script with the image optimization function that was created

function optimize_web_images() {   
    declare -a search_dirs=(
        "/var/www/mypage.com/images" 
        "/var/www/anotherpage.com/images"
    )

    counter=1
    ## now loop through the dirs array
    for dir in "${search_dirs[@]}"
    do
        echo "Processing folder: ${dir} ..";
        find "$dir" -type f -exec file --mime-type {}  \; | awk '{if ($NF == "image/jpeg") print $0 }' | while read f
        do
            local image_file="${f::-12}";
            local image_width=$(identify -format '%w' "${image_file}");
            local max_width="1024";
            local max_optim="80";

            if [[ "${image_width}" -gt "${max_width}" ]]; then
                echo "Optimizing image ${image_file} .. old width: ${image_width}px, new width: ${max_width}px";
                convert "${image_file}" -resize "${max_width}"\> "${image_file}";
                jpegoptim --strip-all -q --max="${max_optim}" "${image_file}";
            fi
            
            # if processed images are 500, print progress
            if ! (( $counter % 500 )) ; then
                echo "Processed $counter images .."
            fi

            # count processed images
            ((counter++))
        done
    done
}

Basically what this function does is to search for all image files recursively in a certain directory of a list that is passed as a parameter. To determine the file is actually an image we query for the file mime-type, this in order to avoid problems with the file extension in case of using approaches like using globs (‘*.jpg’)

Once the images are found we proceed to determine if the image is a candidate of optimization by taking a look at the image dimension, for this we use the identify command of the imagemagick package. In case the image is actually a candidate for optimization we first use the convert command of the same library in order to scale the image to the desired size and then we use the jpegoptim command to compress the resulting image

In this way we optimize a list of images in a web server in an automated way that can be executed periodically in order to have reduced images and faster web page loads. Of course this script is just an example of the things we can do and it fits well with our use case, feel free to modify it to your needs and we hope is helpful to you.

Cheers!
-Yohan

http://www.nvidia.com.mx/Download/index.aspx?lang=en-us

The latest version of the driver available in the case of my video cards is 390.25 and is available in the following link http://us.download.nvidia.com/XFree86/Linux-x86_64/390.25/NVIDIA-Linux-x86_64-390.25.run, the most desirable thing is to make sure to download the latest version of the driver available for your video card.

Also, it is necessary to download the CUDA Toolkit 9.1 from the Nvidia page located in the following link https://developer.nvidia.com/cuda-downloads?target_os=Linux&target_arch=x86_64&target_distro=Ubuntu&target_version=1704&target_type=runfilelocal. The toolkit installation file that is needed is the RUN version for Ubuntu, the file name is: cuda_9.1.85_387.26_linux.run.

Now we proceed to verify the requirements to install CUDA. First of all you need to confirm that you have a device that supports GPUs.

cuda_9_1_install.sh

# verify we have a cuda capable gpu
lspci | grep -i nvidia

The previous instruction should yield results, otherwise, you should check the status of the video card. Then, we proceed to check our version of Linux

cuda_9_1_install.sh

# Linux version
uname -m && cat /etc/*release

What tells us that we have Debian Stretch and is the version that is being used and supported for this post

x86_64
PRETTY_NAME="Debian GNU/Linux 9 (stretch)"
NAME="Debian GNU/Linux"
VERSION_ID="9"
VERSION="9 (stretch)"
ID=debian
HOME_URL="https://www.debian.org/"
SUPPORT_URL="https://www.debian.org/support"
BUG_REPORT_URL="https://bugs.debian.org/"

Now you need to check that you have a valid GCC compiler

cuda_9_1_install.sh

# verify gcc version 
gcc --version

If you do not have a valid GCC compiler you can install it with the following command

cuda_9_1_install.sh

# gcc install
sudo apt-get install build-essential

Before proceeding with the installation process you need to uninstall any version of CUDA that has been previously installed, if this is the first time you install CUDA you can skip this step. To uninstall previous versions of CUDA the following command is used

cuda_9_1_install.sh

# cuda uninstall
sudo /usr/local/cuda-X.Y/bin/uninstall_cuda_X.Y.pl

Where X and Y is the CUDA version. If you are updating the Nvidia driver you do not need to do anything since the installer removes the previous drivers automatically. If you want to uninstall the CUDA driver, you can do it with the following command.

cuda_9_1_install.sh

# cuda driver uninstall
sudo /usr/bin/nvidia-uninstall

It is likely that you will need to uninstall nvidia packages if they have been installed through the repository. For this it could be sufficient to execute the following command

sudo apt-get remove 'nvidia*'

Once the previous steps have been completed and since the necessary files for the installation are available, we proceed to disable the nouveau driver, which is the default driver that comes with the Debian versions. In order to accomplish this a new file needs to be created

cuda_9_1_install.sh

# edit debian driver configuration file
vim /etc/modprobe.d/disable-nouveau.conf 

And we add the following lines

cuda_9_1_install.sh

# blacklist defualt driver
blacklist nouveau
options nouveau modeset=0

Note: to verify the nouveau driver is running or not, the following command needs to be executed and in the case the command shows results that means that you have the nouveau driver running in the system

cuda_9_1_install.sh

# verify driver
lsmod | grep nouveau

After having created the file to block the default driver, we proceed to restart the system in default mode. Once restarted we will notice that the resolution has dropped quality, this means that the default nouveau driver was not loaded, now we will proceed to install the linux headers from our repository in order to compile the Nvidia driver, we do this through the next command.

cuda_9_1_install.sh

# dependencies
sudo apt-get install linux-headers-$(uname -r)

After installing these packages we proceed to remove the nouveau driver completely from our system with the following command

cuda_9_1_install.sh

# remove
apt-get remove --purge xserver-xorg-video-nouveau

Once the driver is removed, the system is restarted in recovery mode. Once rebooted and in the console, we will proceed with the driver installation process. First, you need to know with what version of the compiler the kernel was compiled, this is because the system needs to compile the Nvidia driver. If you do not know the version of the compiler or the kernel, you can know that information by using the following command

cuda_9_1_install.sh

# find curr gcc version
cat /proc/version

Once you know the correct version of the compiler, you need to set it up by means of an export in the following way

cuda_9_1_install.sh

# set gcc version
export CC=gcc-4.8

Now you can proceed with the driver installation, navigate to the driver download directory and run

cuda_9_1_install.sh

# execute driver install
sudo sh NVIDIA-Linux-x86_64-390.25.run

Once the installation is finished, we proceed to restart our system. Note that the resolution went up in quality which means that our Nvidia drivers were installed correctly. Now, we proceed with the installation of the Cuda toolkit in the following way

cuda_9_1_install.sh

# execute toolkit install
sudo sh cuda_9.1.85_387.26_linux.run

When asked, you need to select the option of not install the Nvidia driver since it has already been installed previously, and select the default installation directory for the examples. The last thing you have to do is to compile these examples and execute them, this is done by navigating to the installation directory of the examples using the make command. Usually the default directory of the examples lies in our home directory so the following command should do the trick

cd ~/NVIDIA_CUDA-9.1_Samples
make

And that is pretty much it!

Reference Links

Cuda Installation Guide
https://developer.download.nvidia.com/compute/cuda/9.1/Prod/docs/sidebar/CUDA_Installation_Guide_Linux.pdf
CUDA Quick Start Guide
http://developer.download.nvidia.com/compute/cuda/9.1/Prod/docs/sidebar/CUDA_Quick_Start_Guide.pdf
CUDA Toolkit Official Documentation
http://docs.nvidia.com/cuda/index.html
CUDA Download Page
https://developer.nvidia.com/cuda-downloads
Nvidia Driver Download Page
http://www.nvidia.com.mx/Download/index.aspx?lang=en-us
CUDA 9.1 Performance Presentation
http://on-demand.gputechconf.com/gtc/2017/video/s7495-jain-optimizing-application-performance-cuda-profiling.mp4
CUDA Toolkit Release Notes
https://docs.nvidia.com/cuda/cuda-toolkit-release-notes/index.html

Enjoy!
-Yohan

Practices

Probably these tips are not for all people. Remember that my passion and main occupation is to program. And as a programmer I have in mind some concepts that I can not easily get rid of. When using you are practically programming, the difference is that when you compile, instead of watching a program, page, component, app, robot or whatever you are creating, you will see a beatiful document in pdf format. Although in reality it can be another format, but at the end of the day in this article I refer to a document and specifically, a pdf.

When programming a system, I always like and as much as possible have organization in the code. I like to separate the pieces of the program into easily manageable logical entities. I applied this practices to the writing of my doctoral document. And, similar to any programming language, allows you to modulate files of multiple formats in the document itself, without the need to have everything embedded in a monolithic file. When talking about files I mean multiple formats, in my thesis, I have separated text content files in multiple levels, image files, code files, bibliography files, among which I can think for the moment.

Thesis Structure

I have always been used to reading books and a large part of the decisions made to write a thesis is based precisely on the format of the books I have read, taking into account the ease of reading, the quality of the content, the ease of finding conclusions and references about something quickly, and in general, the experience of the end user. Probably the suggested structure is more like a book than a thesis document.

• Cover
  For the cover there is not much to say, there is no standard way to do it. Therefore, it becomes somewhat an art because of the design side. The only thing to emphasize is the fulfillment of including all the information required by the university.
  
  
• Dedication
  From my point of view, this is a brief but indispensable part of the document. Styles vary but more than seeking a recommendation is to find the inspiration that dictates the right words that reflect these feelings.
  
  
• Abstract
  Although this section is not strictly necessary as a rule, I have always liked to know what a document is about, especially when it is of a considerable size. In any case, some authors caution against extending this section too much because of the possibility of duplicating content or discouraging the reader. In my opinion and as it is done in science articles, it is always good to know what a document is about and its objectives before immersing yourself in its reading.
  
  
• Preface
  It should be mentioned that this section is not strictly necessary but it is my pleasure to include it in the structure.
  
  
• Acknowledgment
  Space for acknowledgments.
  
  
• Table of Contents
  The typical table that shows the content of the thesis.
  
  
• List of figures
  The common list of figures.
  
  
• Code Listings
  I have decided to include code listings for readability and ease.
  
  
• Chapters
  One of the important decisions of the structure of a thesis is the design in the writing of chapters. In recent dates, I have seen some theses and books that produce chapters as independent units in terms of bibliography, this has always been to my liking since it is easier to find the references that belong to the chapter that is being read and not have to go to the end of the book to locate a reference. In my case, I have gone further and have decided to include conclusions for each chapter since I consider that this content should be included in the same section of the content.
  
  
• Conclusions, Results and Future Work
  It was decided that both the conclusions and references be global, unlike the conclusions and references of each chapter. With this we have the opportunity to express in a global context the results and conclusions reached.
  
  
• Appendices
  List of Appendices.
  
  

Directories

Since it is intended to have all the content of the thesis separated, it has been decided to make a logical separation according to the defined structure, that is, each section will have its separate directory and each directory will contain a child directory for the figures and another one for the code files. Virtually all files will be separate and independent and only a document with this embedded information will be generated when the solution is compiled.

LaTeX

Once having the desired structure available. We proceed to capture the code to build our document. The main code file is listed below.

thesis.tex

\documentclass[12pt,letterpaper]{report}
 
% the structure file contains all the packages that needs to be included to generate
% the main structure of the document
\include{structure/structure}
 
% Define a new glossary types
\newglossary[glg]{glossary}{gls}{glo}{Glosario}
\newglossary[slg]{symbol}{sym}{sbl}{Lista de Símbolos}
\newglossary[alg]{acronym}{acr}{acn}{Lista de Abreviaturas}
\makeglossaries
 
\include{acronyms/acronyms}  % include acronyms
\include{symbols/symbols}    % include symbols
\include{glossary/glossary}  % include glossary
 
\begin{document}
 
% custom background image command
\AddToShipoutPicture*{\BackgroundLogo}
 
\showboxdepth=-1
 
\include{front/front}
 
\thispagestyle{empty}
\hbox{}\vfill\hbox{}\pagebreak
\setcounter{page}{1}
 
% with this we prevent the paragraph alignment issue
\raggedbottom
 
% first use roman numerals for page numbers
\pagenumbering{roman}
 
% adding custom headers/footers defined above
\noneheads
\nonechapterheads
 
% including sections
\include{dedication/dedication}
\include{abstract/abstract}
\include{preface/preface}
\include{acknowledgments/acknowledgments}
 
% table of contents
\tableofcontents
\newpage % Begins on a new page instead of on the same page as the table of contents
 
% figure listings
\renewcommand\listfigurename{Lista de Figuras}
\listoffigures
\newpage
 
% code listings
\addcontentsline{toc}{chapter}{\lstlistlistingname}
\lstlistoflistings
\newpage
 
% set one blank space between paragraphs in empty line
%\setlength{\parskip}{\baselineskip}
 
% return page numbering to arabic
\pagenumbering{arabic}
 
\chapterheads
\commonheads
 
% including chapters
\include{chapters/1/introduction}
\include{chapters/2/theory}
 
% including appendixes
\begin{appendices}
  \include{appendixes/a/computer_programs}
\end{appendices}
 
% include glossaries
\printglossaries
 
\end{document}

As you can see in the code, the include latex command is used to include the other parts of the document from the main code file. The same technique is used for the chapters where the directory of the code files is added as well as the directory of the figures. Also, the bibliography files are added for each chapter including the directories. In this way constructs the document in parts according to the definitions in the code. Below is the example of a chapter with the definition of images, code and bibliography.

introduction.tex

\chapter{Introducción}
 
% code and figures paths
\graphicspath{{chapters/1/figures/}}
\lstset{inputpath=chapters/1/code/}
 
\lipsum. \cite{0071548297}
 
\section{Sección}
\lipsum
 
% abreviatura
\gls{fdtd}
 
% glosario
\glspl{debian}
 
\subsection{Subsección}
\lipsum
 
\section{Conclusiones}
\lipsum 
 
\bibliographystyle{unsrt}
\bibliography{chapters/1/introduction}

Compiling

Has multiple commands to compile the code and generate a beautiful document. Some of these commands are pdflatex, makeindex, makeglossaries, bibtex, among others. Each of these commands has a specific mechanism to create a pdf and may require several passes to complete the construction of the entire document. For simple documents there are several popular IDEs that automatically build the document by selecting configuration options graphically. It is also possible to define the compilation flow graphically. A simple example of an IDE that I personally use and recommend is Kile, of course, open source and cross platform.

makefile

shell := /bin/bash
curr_date := $(shell /bin/date '+%Y-%m-%d_%H.%M.%S')
 
main_file = "./thesis"
output_file = "./thesis"
 
papers = "../papers"
output_file_papers = "../build/thesis_papers_"$(curr_date)
paper1_name = "ACES_Journal_20150117_v3.pdf"
paper2_name = "APL_101_261902_2012.pdf"
 
#chapters = "chapters/1/introduction" "chapters/2/conclusions"
 
# main entry point
all: aux glossary acronyms symbols bibliography pdf add_papers clean
 
# PDFLaTeX - first pass to generate aux files
aux:
    echo ">>>>> AUX"
    pdflatex -interaction=nonstopmode $(main_file).tex
     
# MakeIndex - generate glossary files
glossary:
    echo ">>>>> GLOSSARY"
    makeindex $(main_file).glo -s $(main_file).ist -t $(main_file).glg -o $(main_file).gls
 
# MakeIndex - generate acronyms files
acronyms:
    echo ">>>>> ACRONYMS"
    makeindex $(main_file).acn -s $(main_file).ist -t $(main_file).alg -o $(main_file).acr
 
# MakeIndex - generate symbols files
symbols: 
    echo ">>>>> SYMBOLS"
    makeindex $(main_file).sbl -s $(main_file).ist -t $(main_file).slg -o $(main_file).sym
 
# BibTex - generate chapters bibliography 
bibliography:
    echo ">>>>> BIBLIOGRAPHY"
    # find name of chapters .tex files, remove extension and execute bibtex on the result
    find ./chapters -type f -name '*.tex' | while read f; do bibtex "$${f%.*}"; done
    #for chapter in $(chapters); do \
    #   bibtex $$chapter; \
    #done
 
# PDFLaTeX - last pass to generate final pdf files
pdf:
    echo ">>>>> PDF"
    pdflatex -interaction=nonstopmode $(main_file).tex
    makeglossaries $(main_file).glo
    makeglossaries $(main_file).acn
    makeglossaries $(main_file).sbl
    pdflatex -interaction=nonstopmode $(main_file).tex
 
# Append papers pdf to generated thesis pdf
add_papers:
    echo ">>>>> ADD_PAPERS"
    pdftk $(output_file).pdf $(papers)/$(paper1_name) $(papers)/$(paper2_name) cat output $(output_file_papers).pdf
 
# cd -; returns to previous dir
clean:
    echo ">>>>> CLEAN";
    # find and remove unneded extension files
    find . -name "*.acn" -o -name "*.acr" -o -name "*.alg" -o -name "*.aux" -o -name "*.glg" -o -name "*.glo" -o -name "*.gls" -o -name "*.idx" -o -name "*.ist" -o -name "*.slg" -o -name "*.sym" -o -name "*.lof" -o -name "*.blg" -o -name "*.bbl" -o -name "*.toc" -o -name "*.sbl" -o -name "*.lol" -type f | xargs rm;

There are many other considerations to take into account when structuring a doctoral thesis document but the truth is that it is not my intention to go into detail about each problem that had to be solved in order to get to the desired document. Instead, I have prepared an example template type that you can easily explore and adapt to your needs and hopefully you can save a good percentage of the hours that I had to invest to reach this result. Next I attach the file of the example and a pdf as the resulting document. Of course these files are designed to work in Linux.

Sample template files

Pdf of the sample template

¡Enjoy!
-Yohan

This GMail filtering system is very useful and powerful. The problem raise when you have to mark all email in some of the labels as read. And I say problem because the real problem comes up when trying to mark all email in a label and all the corresponding sub labels at any level as read automatically. And this is simply not supported by GMail.

Google Apps Scripts to the Rescue
It turns out that Google has a variety of APIs for all its products that can be used to interact with them from a defined language, in this case Javascript. The APIs are quite friendly and easy to use. Also, the documentation is excellent and highly understandable. And not to do the long story, I created a script to do just that, mark as read all emails under a defined tag including all child tags.

function markLabelThreadsAsRead(label) {
  var labels = GmailApp.getUserLabels();
  for (var i = 0; i < labels.length; i++) {
    var index = labels[i].getName().indexOf(label, 0);
    if (index === 0) { // if label name starts with parent label name
      Logger.log("processing label:" + labels[i].getName());
      // batch processing ref: https://gist.github.com/gene1wood/0f455239490e5342fa49
      var batchSize = 100 // Process up to 100 threads at once
      var threads = GmailApp.search("label:" + labels[i].getName() + " is:unread");
      for (j = 0; j < threads.length; j+=batchSize) {
        Logger.log("mark thread as read:" + threads[j].getFirstMessageSubject());
        GmailApp.markThreadsRead(threads.slice(j, j+batchSize));
      }     
    }
  } 
}

markLabelThreadsAsRead("ParentLabel");

To run the script simply visit the Google online editor ( https://script.google.com/ ), paste the script there and then you can press the run or debug option. The APIs documentation can be found here https://developers.google.com/apps-script/overview.

Enjoy! =)
-Yohan

I’m the kind of guy that puts more attention to email when it comes to data or system alerts. So I decided that, to send my site statistics to my email instead of having to use the password protected url to see them. I started to search a plugin or a built-in functionality to do this in the simplest way possible in order to not spend time getting this to work. Maybe I did not search enough because I didn’t found a way to do it automatically.

So I started to look at some options to accomplish sending my site stats to the email. Here is the approach that I came up with. Maybe it can be useful to someone.

function send_awstats() {
    # Download awstats for all sites. Send them as html email 
    for ((i = 0; i < ${#__server_sites[@]}; i++)); do
        wget --tries=3 --output-document=/tmp/${__server_sites[$i]}.html --no-check-certificate --user user --password pass https://awstats/url/?config=${__server_sites[$i]}
        mail -a "MIME-Version: 1.0" -a "Content-Type: text/html" -s "Daily Awstats [${__now}]: ${__server_sites_domains[$i]}" root@localhost < /tmp/${__server_sites[$i]}.html
        rm -f /tmp/${__server_sites[$i]}.html
    done
}

Basically what this script does is that it makes a request to the Awstats url and saves the content as an html file in a temporal directory. After that it sends the saved file as an html email. In order for this to work you have to have Awstats installed and also postfix for sending emails.

This script can be placed in a .sh file and setup a cron job to periodically and according to your needs send the email with the latest site statics data.

Enjoy!
-Yohan

So I decided to make a small script to automate this simple task that was becoming tedious to do just because of the fact that one needed to remember to add an entry to the global header file each time a new header class was added. With this script it is not needed anymore to remember to add any entry to the global module file cause in the build target we added a reference to the new bash script that generates automatically all the header reference that are contained in the actual module.

Here is the script code for the automatic generation of C++ header code. I hope you find it useful.

module_name="${1}"
module_name_upper=${module_name^^}  # Module name in uppercase
 
# Run as user, -e allows to interpret new line \n
echo -e "\n \
********************************************************** \n \
** Automatic header generation for module ${module_name} \n \
**********************************************************"
 
inc_dir="${2}"
out_file="${3}"
 
# "ls -l $inc_dir"        = get a directory listing
# --time-style="long-iso" = makes sure that the same format for the date-time string is the same in all environments.
# "| egrep '^d'"          = pipe to egrep and select only the directories
# "awk '{print $8}'"      = pipe the result from egrep to awk and print only the 8th field
inc_dirs=`ls -l --time-style="long-iso" $inc_dir | egrep '^d' | awk '{print $8}'`
 
cat <<EOF >"${out_file}"
/*
 * ${module_name}.hpp
 *
 * Auto-Generated module header file, do not modify directly
 * to prevent changes loss, to modify please change GenHeaders.sh
 * 
 *  Created on: 2014
 *      Author: nano
 */
 
#ifndef ${module_name_upper}_HPP_
#define ${module_name_upper}_HPP_
 
// Include all headers that are part of the ${module_name} module
EOF
 
# Loop through the directories
for dir in ${inc_dirs}; do
    echo -e "\n${dir}";  
    curr_dir=${inc_dir}/${dir}
    file_cnt=$( find ${curr_dir} -name "*.h*" | wc -l )
    curr_files=$( find ${curr_dir} -name "*.h*" )
    if [ ${file_cnt} -gt 0 ]; then  # check  if we have headers in dir
        for file in ${curr_files}; do # Loop through files and add includes
            echo "#include \"${file}\"" >> ${out_file};
            echo "${file}";
        done       
    fi
done
echo -e "\n";
 
# Add closing ifdef    
echo "" >> ${out_file}
echo "#endif /* ${module_name_upper}_HPP_ */" >> ${out_file}

And here are some autogenerated entries in the global header module file.

/*
 * common.hpp
 *
 * Auto-Generated module header file, do not modify directly
 * to prevent changes loss, to modify please change GenHeaders.sh
 * 
 *  Created on: 2014
 *      Author: nano
 */

#ifndef COMMON_HPP_
#define COMMON_HPP_

// Include all headers that are part of the common module
#include "./include/common/CommonBase.hpp"
#include "./include/data/Config.hpp"
#include "./include/data/Programs.hpp"
#include "./include/data/VersionInfo.hpp"
#include "./include/exception/CustomException.hpp"
#include "./include/hdf5/Hdf5Grid2D.hpp"
#include "./include/hdf5/Hdf5Grid2DSlice.hpp"
#include "./include/hdf5/Hdf5Grid1DSlice.hpp"
#include "./include/hdf5/Hdf5Grid1D.hpp"
#include "./include/hdf5/Hdf5Base.hpp"
#include "./include/io/Folder.hpp"
#include "./include/io/File.hpp"
#include "./include/utils/DateTime.hpp"
#include "./include/utils/Logger.hpp"
#include "./include/utils/Formatter.hpp"
#include "./include/utils/ArrSizeHelper.hpp"
#include "./include/utils/Matrix.hpp"
#include "./include/utils/Utilities.hpp"
#include "./include/utils/Strings.hpp"
#include "./include/utils/Command.hpp"
#include "./include/utils/SingleLoggerConfig.hpp"

#endif /* COMMON_HPP_ */

Enjoy!
-Yohan

The first function presented below receives the current function name as first parameter and a parameter list that is used to create a nice formated flow function string to print. The second function is an example on how to use the first function to print the flow.

local function debug_print_trac(func, ...)
    local res = func .. "( ";
    for i,v in ipairs(arg) do
        res = res .. tostring(v) .. " ";
    end
    res = res .. " )";
    debug_print(res);
end
 
local function set_monitor(monitor)
    debug_print_trac(debug.getinfo(1, "n").name, monitor);
    if (monitor == "principal") then
        set_window_position(0,0);
    else
        set_window_position(2561,0);
    end
end

And here are some printed results of function calls. We have our current flow functions and its parameters values printed.

set_top( false  )
Window Name: The Linux Programming Interface
Application name: Document Viewer
max_in( principal 4 false  )
set_monitor( principal  )

¡Cheers!
-Yohan

Git uses a distributed repositories paradigm which means that every client has a complete copy of the whole code repository which makes working with the code be possible even without a network connection, this is the main difference between the centralized systems like subversion which requires a network connection in order to checkout files for editing and/or deleting.

This guide does not pretend to be a complete reference of git commands and descriptions, that guide already exists, instead, this guide pretends to help people that wants to start learning how to use git as a source code control in a fast manner and want to start to be productive using this tool as soon as possible.

Configuration

Basically, the first thing to do after installing git is to configure its global options in order to use git effectively. First we need to increment the post buffer size to be able to post large files into our repository. We do that in the following way

# increment git post buffer to 500MB
git config --global http.postBuffer 524288000

Next, we need to establish a credential helper cache of 10 hours to not have git to be asking for credentials every pull and/or push we make to our repository

# cache credentials for 10 hours
git config --global credential.helper cache
git config --global credential.helper "cache --timeout=36000"

After that we setup some console output color just to make the shell more attractive

# and give some love to the git output
git config --global color.ui auto
git config --global color.branch auto
git config --global color.diff auto
git config --global color.status auto

After that, we need to set some identity information like the user name and email, this data is going to appear in each commit we do to our repositories

# add commit global information
git config --global user.name "Yohan Jasdid"
git config --global user.email yohan.jasdid@gmail.com
git config --global push.default simple

Then, we have the mergetool configuration, the merge tool is the tool used to resolve any issue at the moment of making any merge between branches, this tool usually shows the differences between local and remote files and allow us to resolve any conflicts in a visual and easy way, in this case I use kdiff but of course there are many good merge tools that you can use

# configure merge tool global setting
git config --global --add merge.tool kdiff3
git config --global --add mergetool.kdiff3.path "/usr/bin/kdiff3"
git config --global --add mergetool.kdiff3.trustExitCode false

After the mergetool we setup the diff tool, this is the tool that will be used when showing differences between revisions, here again I’m using kdiff

# configure diff tool global setting
git config --global --add diff.guitool kdiff3
git config --global --add difftool.kdiff3.path "/usr/bin/kdiff3"
git config --global --add difftool.kdiff3.trustExitCode false

Lastly, I added a setting to disable the SSL certificate. This was just because I’m using a self signed certificate in my source control server and git is unable to verify the authenticity of the certificate, I should get rid of this setting once I use a letsencrypt certificate

# disable ssl checking (not recommended for global)
git config --global http.sslVerify false

Cloning Repos

Cloning a repository in git means that we will obtain a full copy of the remote repository in the local machine, to do this we use the following command

# clone a repository
git clone https://my_repo.git

Branching

Git is based on the concept of branching, all branches basically have a parent which they’re based on, a branch can be seen as a snapshot of the current repository at the moment of creating the branch. The new created branch is intended to contain a set of code that can be integrated on the main branch or any other branch.

# create branch
git branch <branch_name>
 
# create and move to branch
git checkout -d <branch_name>

There are 2 types of branches, local and remote branches. Once a local branch has been created it need to be pushed in order to create the remote branch mapped to the local branch. To push the local branch we use the following command

# send local branch to origin in case the branch is just local and start tracking remote branch with local branch
git push origin <branch_name>

And to push the local branch and map it to a remote branch we use

# send local branch to origin and map with remote branch
git push --set-upstream origin <branch_name>

Once we have created branches in our repository, we may need to delete some of this branches, to do that we can use the following commands, to delete a branch safely or forcedly

# delete local branch (safely)
git branch -d <branch_name>
 
# delete local branch (forced)
git branch -D <branch_name>
 
# delete remote branch
git push origin --delete <branch_name>

If we want to see a list of local or remote branches we can do this by executing the following commands

# show local branches
git branch
 
# show all branches (local and remote)
git branch -a

After using or deleting branches is necessary to do a clean up and update the references to the current branches. We can do this by using the following commands

# Update references to remote branches, if remote branch is deleted then with this command the branch list gets updated
git remote prune origin

Once we have work with branches, we probably have the need to make changes in branches that are available remotely and that are not owned by us. To do that we first need to retrieve the branch locally so we can make the desired changes. There are some known ways to do that , the two most used ways are to retrieve a remote branch and map it to a local branch that has a different name and to retrieve a remote branch and map it to a local branch with the same name respectively, to do that we use the following commands

# Checkout a remote branch and track it locally (it is possible to use different names for local and remote branches)
git checkout -b <local_branch> origin/<remote_branch>       # Will create <local_branch> and track origin/<remote_branch>
 
# Checkout a remote branch and track it locally (it is NOT possible to use different names for local and remote branches)
git checkout --track origin/<remote_branch>                # Will only create '<remote_branch>', not a branch with a different name

There are times when something goes wrong in a local branch and we just want to simply regenerate the local branch based on the remote branch, to do that we use the reset command in the following way

# Force replace local branch with remote branch
git reset --hard origin/<remote_branch>

Also, there are times when is useful to simply delete the local branch and create it again using the remote branch, to do that we use this commands

# Recreate branch locally
git checkout master
git branch -D <local_branch>
git checkout --track origin/<remote_branch>

Add

Once we have understand and mastered branches and is time to do a code change or simply develop a new feature in the system, we proceed to make our changes, git is smart, at the moment we do our changes in a branch git automatically recognizes them and it display them when we type the following command

# show status of modified files detected by git
git status

In git there are different states of the files, there is a state called untracked, that means that git detected a fie that had never been tracked by git, there is also the tracked state, that means that git already tracked changes for that file but is in a modified state at the moment. Wherever the state of the file change is, if we want to integrate the change into our local branch and then into the remote branch, we first need to add the change to staging, the staging area is the area that identifies all the file changes we want to add to the repository, to add a file change to staging we use the following command

# adds a file to staging
git add <file_name>

There are probably many changes that we want to add to staging at the same time at once, and to avoid being adding changes one by one, we can use pattern matching and add a bunch of changes to staging at once like this

# add all files with pattern match to staging
git add *
 
# add all files with pattern match to staging
git add .

And lastly, we can add only the untracked file changes to staging with the following command

# add all files that are untracked to staging
git add -u

Commits

Once we have added a change or group of changes to staging, we can proceed to make commits, a commit is basically a change or a group or changes encapsulated in a sort of bag with an ID that is a hash that git generates, to generate a commit or “bag of changes” we can use the following commands

# Standard commit with simple message
git commit -m "Message of commit"
 
# Commit with message and description - this commit is intended for when having a title and a long description
git commit -m "Title of commit" -m "Detail message of commit"

Pushing

Once we have our commit of changes or group of commits of changes in place in our local branch, what we want to do now is to send these changes to the remote branch to make them publicly available, this is what the push command does for us, is worth mentioning that the push command can push a branch with changes to the remote repository or just a sets of changes in a branch that already exists remotely, we do that by using the following commands

# Send all changes that are already in commits to remote repository that is tracked
git push
 
# Send local branch to origin in case the branch is just local and start tracking remote branch with local branch
git push origin <branch_name>

Revert

There are cases were for whatever reason we decide to undo all the local changes we already did to a file or group of files or just undo all the changes we made to all files, to this effect we use the following commands

# revert a file to a commit state
git checkout <commit_hash> path/to/file
 
# discard file changes 
git checkout path/to/file
 
# discard file changes with pattern matching
git checkout .

Stash and Clean

There are also some times when we start making changes and developing new functionality in the system and we realize that we were not doing this changes in the correct branch, what can I do then?, we use git stash!. The stash commands allow us to save a change or group of changes in some kind of temporary location which allow us to switch to another branch and restore all changes, nice!, we do this by using the following command

# when having changes in different branch and want them to move to another branch use git stash
# git stash saves changes to a temporary location and applies them with a command
# sample workflow to move changes from one branch to another
git checkout <branch_name>       # move to a branch
<make some changes>
git stash                        # stash the changes in temp location
git checkout <another_branch>    # move to a different branch
git stash pop                    # apply the last stash changes to <another_branch> and drops the stash from the stack

Git stash also allows to save one set or many set of changes, to see all sets of changes available by git stash we use the following command

# shows a list of stashes. there can be more than one stash in form of a list
git stash list

There are times that the local and remote branches diverge, in this case what we want to do is regenerate the local branch, and we can do that with the reset commands in the following way

# when remote and local branches had diverged you can use reset to reset your local branch
# and sync with the remote branch
git fetch
git reset --hard origin/<branch_name>

There are other cases that we added a bunch of new files to the branch that we realize that were not needed to be added and we want to remove them, in that case we can use the clean command like this

# delete untracked files - show untracked files to delete using the -n option:
git clean -f -n
 
# delete untracked files - beware: this will delete files
git clean -f

We can also discard all added changes with the following command

# remove all changes in current branch and checkout branch changes
git checkout .

Log and Show

Once we have branches, commits, pushed commits and a change history, we may regularly need to query our repository for the list of latest commits done. This is useful to find a specific change or just to see who made some change. To do this, we use the log command which is very useful, we can show a list of changes with descriptions like this

# show commits history with descriptions
git log

And if we want to look for all the commits done by a certain author we can do this

# show commits history with descriptions of a certain author
git log --author=bob

To show the commit history in a one line per change and/or with decoration format we can do the following

# to see a very compressed log where each commit is one line
git log --pretty=oneline
 
# to see an ASCII art tree of all the branches, decorated with the names of tags and branches
git log --graph --oneline --decorate --all

To show only the modifies files, we do

# see only which files have changed
git log --name-status

And to display the command help we use

# show help for log
git log --help

To show the differences between commits between branches we do

# show commit difference between branches
git log master..<branch_name>

And to show the details of a commit, we do

# show commit diff details
git show <commit_hash>

Diff

There is another useful command called diff that allow us to view all differences between branches or commits. We use this command in the following way

# show changes between source and target branches
git diff <source_branch> <target_branch>
 
# show changes between commit's ancestor and commit
git diff <commit_hash>~ <commit_hash>
 
# show modified files between branches
git diff --name-only <source_branch> <target_branch>
 
# show file differences between branches
git diff --name-status <branch1>..<branch2>
 
# show file content differences between branches
git diff <branch1> <branch2> -- <file_path_name>

Tags

# it's recommended to create tags for software releases. this is a known concept, create a new tag named 1.0.0 with this command
# the 1b2e1d63ff stands for the first 10 characters of the commit id you want to reference with your tag.
git tag 1.0.0 1b2e1d63ff
 
# Show tag list
git tag
 
# Create a breanch based on tag
git checkout -b <branch_name> <tag_name>

Escenarios

# Usually at home in my server I'll do the following
git checkout development   # Move to dev branch
Do some programming.
git status                              # to see what files I changed.
git diff [file]                           # to see exactly what I modified.
git add .                               # to stage all changes
git commit -m [message]     # to commit.
git push                               # push changes to branch
 
# Steps to fast forward merge from development branch into master and return to development to continue working
git checkout master && git pull && git merge development && git push && git checkout development

References

• Git – Home: https://git-scm.com/
• Git – Reference: https://git-scm.com/docs
• Git – The Simple Guide: http://rogerdudler.github.io/git-guide/
• Git – Documentation: https://git.wiki.kernel.org/index.php/GitDocumentation
• Git – Understanding Git Conceptually: https://www.sbf5.com/~cduan/technical/git/

Cheers!
-Yohan

http://www.nvidia.com.mx/Download/index.aspx?lang=en-us

The latest driver version available for my cards is 352.79 and is available at http://us.download.nvidia.com/XFree86/Linux-x86_64/352.79/NVIDIA-Linux-x86_64-352.79.run, please make sure to download the latest version of your card driver.

Also we need to download the CUDA toolkit 7.5 from located at http://developer.download.nvidia.com/compute/cuda/7.5/Prod/local_installers/cuda_7.5.18_linux.run. The installer file of the toolkit needed is the Linux RUN file for Ubuntu, the file name is: cuda_7.5.18_linux.run.

Now we need to verify the requirements to install CUDA. First we need a capable GPU device.

cuda7_5_install.sh

# verify we have a cuda capable gpu
lspci | grep -i nvidia

We need to check our Linux version

cuda7_5_install.sh

# Linux version
uname -m && cat /etc/*release

Now we need to verify that we have a valid GCC compiler

cuda7_5_install.sh

# verify gcc version 
gcc --version

If you don’t have a valid GCC compiler installed please install it by running the following command

cuda7_5_install.sh

# gcc install
sudo apt-get install build-essential

Before we proceed with the actual installation process we need to uninstall any version of CUDA we had installed previously, if this is the first time you install CUDA please skip this step, we uninstall previous installations by running the following command

cuda7_5_install.sh

# cuda uninstall
sudo /usr/local/cuda-X.Y/bin/uninstall_cuda_X.Y.pl

Where X and Y is the cuda version. If we are upgrading the Nvidia driver we don’t have to do anything just because the installer itself removes old driver versions. If we want to uninstall the CUDA driver just run the following command.

cuda7_5_install.sh

# cuda driver uninstall
sudo /usr/bin/nvidia-uninstall

Once we have completed the steps above and have downloaded the files needed for the installation we proceed to disable the nouveau driver which is the default driver for Debian, we do that by creating a new file:

cuda7_5_install.sh

# edit debian driver configuration file
vim /etc/modprobe.d/disable-nouveau.conf 

And add the following content to it

cuda7_5_install.sh

# blacklist defualt driver
blacklist nouveau
options nouveau modeset=0

Note: to verify that we have the nouveau driver running, just run the following command, if it outputs something it means that we actually have a nouveau driver running

cuda7_5_install.sh

# verify driver
lsmod | grep nouveau

After having the file for blocking the default driver in place, we proceed to restart the machine in default mode. Once restarted we are going to note that the screen resolution got at a lower level and that means that the default driver was not loaded, then we need to get some header files from our repository to build the Nvidia driver, we do this by executing the following command.

cuda7_5_install.sh

# dependencies
sudo apt-get install linux-headers-$(uname -r)

After installing this files we proceed to remove the nouveau driver completely from our system by executing the following command

cuda7_5_install.sh

# remove
apt-get remove --purge xserver-xorg-video-nouveau

After the removal of the driver has happened we need to restart our system in recovery mode, once restarted in recovery mode and having the nice console interface we proceed with the driver installation process, first we need to know which version of the compiler was used to compile the current system kernel, this because the system needs to compile the Nvidia driver, if we don’t know which version was used and which kernel we have, we can find that out by running the following command

cuda7_5_install.sh

# find curr gcc version
cat /proc/version

Once we know the correct compiler version, we need to set it up with an export like this

cuda7_5_install.sh

# set gcc version
export CC=gcc-4.8

Now we can proceed with the driver installation, we navigate to our download path and execute the driver

cuda7_5_install.sh

# execute driver install
sudo sh NVIDIA-Linux-x86_64-346.47.run

Once the installation is finished we proceed to reboot the machine and we’re going to note that we have a nice screen resolution and our new Nvidia drivers installed, next we proceed with the toolkit installation like this.

cuda7_5_install.sh

# execute toolkit install
sudo sh cuda_7.5.18_linux.run

When prompted we need to select not to install the Nvidia driver since we already installed it, and choose the default path for the samples (recommended).
The last thing to do is to compile the samples and run them, to do this we navigate to the samples directory that we choose and just execute a make command

And that’s it!

Here are some useful links related to CUDA 7.5

The CUDA getting started manual which covers in deep installation instructions
http://developer.download.nvidia.com/compute/cuda/7_0/Prod/doc/CUDA_Getting_Started_Linux.pdf

The CUDA downloads page
https://developer.nvidia.com/cuda-downloads

The Nvidia Driver downloads page
http://www.nvidia.com.mx/Download/index.aspx?lang=en-us

A blog post of the CUDA 7.5 features
https://devblogs.nvidia.com/parallelforall/new-features-cuda-7-5/

The Nvidia CUDA toolkit 7.5
http://developer.download.nvidia.com/compute/cuda/7.5/Prod/docs/sidebar/CUDA_Toolkit_Release_Notes.pdf

Have Fun!
-Yohan

http://www.nvidia.com.mx/Download/index.aspx?lang=en-us

The required driver version that supports the new CUDA 7.0 features is 346 or above, please make sure to download the latest version of your card driver

Also we need to download the CUDA toolkit 7.0 from the following link

https://developer.nvidia.com/cuda-toolkit

The installer file of the toolkit we need to download is the Linux RUN file for Ubuntu, the file name is: cuda_7.0.28_linux.run and the direct link is the following

http://developer.download.nvidia.com/compute/cuda/7_0/Prod/local_installers/cuda_7.0.28_linux.run

Now we need to verify that we have a CUDA capable GPU by running the following command

cuda7_install.sh

# verify we have a cuda capable gpu
lspci | grep -i nvidia

In my case the output is something like this

cuda7_install.sh

# lscpi output
03:00.0 VGA compatible controller: NVIDIA Corporation Device 11c0 (rev a1)
03:00.1 Audio device: NVIDIA Corporation Device 0e0b (rev a1)
04:00.0 VGA compatible controller: NVIDIA Corporation Device 0fc6 (rev a1)
04:00.1 Audio device: NVIDIA Corporation Device 0e1b (rev a1)

If you don’t get any output you should verify that your hardware is correctly installed, next we proceed to verify our Linux version by running the following command

cuda7_install.sh

# Linux version
uname -m && cat /etc/*release

For this post I’m using a Debian Wheezy installation so you should see something like this

cuda7_install.sh

# version output
x86_64
PRETTY_NAME="Debian GNU/Linux 7 (wheezy)"
NAME="Debian GNU/Linux"
VERSION_ID="7"
VERSION="7 (wheezy)"
ID=debian
ANSI_COLOR="1;31"
HOME_URL="http://www.debian.org/"
SUPPORT_URL="http://www.debian.org/support/"
BUG_REPORT_URL="http://bugs.debian.org/"

Now we need to verify that we have a valid GCC compiler, this is required in order to install the Nvidia driver, we check it by running the following command

cuda7_install.sh

# verify gcc version 
gcc --version

And you should see an output like this

cuda7_install.sh

# gcc output
gcc (Debian 4.7.2-5) 4.7.2
Copyright (C) 2012 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

If you don’t have a valid GCC compiler installed please install it by running the following command

cuda7_install.sh

# gcc install
sudo apt-get install build-essential

Before we proceed with the actual installation process we need to uninstall any version of CUDA we had installed previously, if this is the first time you install CUDA please skip this step, we uninstall previous installations by running the following command

cuda7_install.sh

# cuda uninstall
sudo /usr/local/cuda-X.Y/bin/uninstall_cuda_X.Y.pl

Where X and Y is the cuda version

If we are upgrading the Nvidia driver we don’t have to do anything just because the installer itself removes old driver versions

If we want to uninstall the CUDA driver just run the following command

cuda7_install.sh

# cuda driver uninstall
sudo /usr/bin/nvidia-uninstall

Once we have completed the steps above and have downloaded the files needed for the installation we proceed to disable the nouveau driver which is the default driver for Debian, we do that by creating a new file:

cuda7_install.sh

# edit debian driver configuration file
vim /etc/modprobe.d/disable-nouveau.conf 

And add the following content to it

cuda7_install.sh

# blacklist defualt driver
blacklist nouveau
options nouveau modeset=0

Note: to verify that we have the nouveau driver running, just run the following command, if it outputs something it means that we actually have a nouveau driver running

cuda7_install.sh

# verify noveau
lsmod | grep nouveau

After having the file for blocking the default driver in place, we proceed to restart the machine in default mode,

Once restarted we are going to note that the screen resolution got at a lower level and that means that the default driver was not loaded, then we need to get some header files from our repository to build the Nvidia driver, we do this by executing the following command

cuda7_install.sh

# dependencies
sudo apt-get install linux-headers-$(uname -r)

After installing this files we proceed to remove the nouveau driver completely from our system by executing the following command

cuda7_install.sh

# remove the nouveau driver completly
apt-get remove --purge xserver-xorg-video-nouveau

After the removal of the driver has happened we need to restart our system in recovery mode, once restarted in recovery mode and having the nice console interface we proceed with the driver installation process, first we need to know which version of the compiler was used to compile the current system kernel, this because the system needs to compile the Nvidia driver, if we don’t know which version was used and which kernel we have, we can find that out by running the following command

cuda7_install.sh

# find curr gcc version
cat /proc/version

And we get something like this

cuda7_install.sh

# version output
Linux version 3.2.0-4-amd64 (debian-kernel@lists.debian.org) (gcc version 4.6.3 (Debian 4.6.3-14) ) #1 SMP Debian 3.2.54-2

Once we know the correct compiler version, we need to set it up with an export like this

cuda7_install.sh

# set gcc version
export CC=gcc-4.6

Now we can proceed with the driver installation, we navigate to our download path and execute the driver

cuda7_install.sh

# execute driver install
sudo sh NVIDIA-Linux-x86_64-346.47.run

Once the installation is finished we proceed to reboot the machine and we’re going to note that we have a nice screen resolution and our new Nvidia drivers installed, next we proceed with the toolkit installation like this.

cuda7_install.sh

# execute toolkit install
sudo sh cuda_7.0.28_linux.run

When prompted we need to select not to install the Nvidia driver since we already installed it, and choose the default path for the samples (recommended)

The last thing to do is to compile the samples and run them, to do this we navigate to the samples directory that we choose and just execute a make command

And that’s it!

Here are some useful links related to CUDA 7.0

The CUDA getting started manual which covers in deep installation instructions
http://developer.download.nvidia.com/compute/cuda/7_0/Prod/doc/CUDA_Getting_Started_Linux.pdf

The CUDA downloads page
https://developer.nvidia.com/cuda-downloads

The Nvidia Driver downloads page
http://www.nvidia.com.mx/Download/index.aspx?lang=en-us

A couple of nice blog posts of the CUDA 7.0 features
http://devblogs.nvidia.com/parallelforall/power-cpp11-cuda-7/
http://devblogs.nvidia.com/parallelforall/cuda-7-release-candidate-feature-overview/

Have Fun!
-Yohan