Aditya Basu

Setup kgdboc for kernel debugging

Wed, 25 Mar 2020 00:00:00 -0400

The goal is to setup a Linux based VirtualBox VM which can be debugged from the host, or another VM. To debug using another physical machine, see Debug from another (physical) machine section.

Introduction

kgdboc is the debugger that helps debug the kernel over a remote gdb session.

kdb gives a debug console to run commands directly against the kernel. It can dump physical memory, list running processes etc.

At runtime one can dynamically switch between both the debugger. The latest official kernel doc is here.

Setup the VirtualBox VM

We will be debugging the VM’s kernel. Before we can debug the VM’s kernel, we need to prep the VM.

To begin, go to the VM’s settings and open “Serial Ports”. Enable COM1, select port mode to be “Host Pipe”. Give the path as “/tmp/vm-serial-socket”. Make sure to have “Connect to existing pipe/socket” unchecked.

Alternatively if you are using vagrant, then add the following to your Vagrantfile and then run vagrant reload.

config.vm.provider "virtualbox" do |vb|
  vb.customize ["modifyvm", :id,
    "--uart1", "0x3f8", "4",
    "--uartmode1", "server", "  /tmp/vm-serial-socket"]
end

Now we need to make sure that the kernel is compiled with debugging support. In kernel config, look under “Kernel hacking” ‣ “Kernel debugging” and select KGDB: kernel debugger.

To be able to add software breakpoints to the running kernel, we need to disable kernel’s read-only data protection. In kernel config, look under “Kernel hacking” ‣ “Write protect kernel read-only data structures” and unselect this option.

The above will modify the .config file as below -

# These options need to enabled in .config
CONFIG_FRAME_POINTER=y
CONFIG_KGDB=y
CONFIG_KGDB_SERIAL_CONSOLE=y

# To disable RO protection on older kernels (ex. v3.2)
CONFIG_DEBUG_RODATA=n

# To disable RO protection on newer kernels (ex. v5.6.0)
CONFIG_STRICT_KERNEL_RWX=n

Recompile and install the newly compiled kernel.

Now to enable debugging support during boot, we need to add kgdboc=ttyS0,115200 to the kernel’s command line options. Also to do meaningful debugging, we disable KASLR using nokaslr. For Ubuntu based distro, this can be done via the /etc/default/grub file.

vm1$ echo 'GRUB_CMDLINE_LINUX_DEFAULT="console=tty0 nokaslr kgdboc=kbd,ttyS0,115200"' | sudo tee -a /etc/default/grub

vm1$ sudo update-grub
Generating grub.cfg ...
Found linux image: /boot/vmlinuz-3.2.0-pfwall-fedora-patches+
Found initrd image: /boot/initrd.img-3.2.0-pfwall-fedora-patches+
Found linux image: /boot/vmlinuz-3.2.0-126-virtual
Found initrd image: /boot/initrd.img-3.2.0-126-virtual
Found memtest86+ image: /boot/memtest86+.bin
done

vm1$ sudo reboot

After rebooting the virtual machine, verify that the serial port configured -

# Verify serial port
vm1$ dmesg | grep serial
[    2.149591] serial8250: ttyS0 at I/O 0x3f8 (irq = 4) is a 16550A

# Verify kernel debugging support
vm1$ dmesg | grep kgdboc
[    0.000000] Command line: BOOT_IMAGE=/boot/vmlinuz-3.2.0-pfwall-fedora-patches+ root=UUID=7d69eb56-ea8c-4243-8e7d-e2714fced818 ro console=tty0 kgdboc=kbd,ttyS0,115200
[    0.000000] Kernel command line: BOOT_IMAGE=/boot/vmlinuz-3.2.0-pfwall-fedora-patches+ root=UUID=7d69eb56-ea8c-4243-8e7d-e2714fced818 ro console=tty0 kgdboc=kbd,ttyS0,115200
[    3.458898] kgdb: Registered I/O driver kgdboc.

# To dynamically change the kgdboc parameters, use -
vm1$ echo ttyS0,115200 | sudo tee /sys/module/kgdboc/parameters/kgdboc

To manually trigger a kernel trap and enter into the debug prompt, use the following command. Note that the ssh session or VM’s console will hang after running the below command.

# Trigger a trap
vm1$ echo g | sudo tee /proc/sysrq-trigger

Option 1: Debug via Host

/tmp/vm-serial-socket gets created by VirtualBox when the VM is started.

host$ file -i /tmp/vm-serial-socket
/tmp/vm-serial-socket: inode/socket; charset=binary

To enable gdb to access the socket, we need to expose a pseudoterminal device. For this we use socat.

host$ socat /tmp/vm-serial-socket PTY,link=/tmp/vm-serial-pty &

# For debugging
host$ socat -d -d -d -d /tmp/vm-serial-socket PTY,link=/tmp/vm-serial-pty

Looking at the pty file that socat creates -

host$ file /tmp/vm-serial-pty
/tmp/vm-serial-pty: symbolic link to /dev/pts/8

host$ file /dev/pts/8
/dev/pts/8: character special (136/8)

Using gdbserver

Run the below commands after trigerring a kernel trap in the VM (via sysrq), otherwise gdb will timeout with the error.

host$ gdb kernel/vmlinux -ex "target remote /tmp/vm-serial-pty"
GNU gdb (Ubuntu 8.1-0ubuntu3.2) 8.1.0.20180409-git
Copyright (C) 2018 Free Software Foundation, Inc.
...
Reading symbols from kernel/vmlinux...
done.
Remote debugging using /tmp/vm-serial-pty
kgdb_breakpoint () at kernel/debug/debug_core.c:960
960     kernel/debug/debug_core.c: No such file or directory.
(gdb) 
(gdb) bt
#0  kgdb_breakpoint () at kernel/debug/debug_core.c:960
#1  0xffffffff810bbef0 in sysrq_handle_dbg (key=<optimized out>) at kernel/debug/debug_core.c:750
#2  0xffffffff81386181 in __handle_sysrq (key=103, check_mask=false) at drivers/tty/sysrq.c:522
#3  0xffffffff8138627b in write_sysrq_trigger (file=<optimized out>, buf=<optimized out>, count=2, ppos=<optimized out>) at drivers/tty/sysrq.c:870
#4  0xffffffff811b20c1 in proc_reg_write (file=0xffff88026ca7db00, buf=0x7fffffffc650 "g\n", count=2, ppos=0xffff88026c503f48) at fs/proc/inode.c:218
#5  0xffffffff811596a5 in vfs_write (file=0xffff88026ca7db00, buf=0x7fffffffc650 "g\n", count=2, pos=0xffff88026c503f48) at fs/read_write.c:438
#6  0xffffffff811599bb in sys_write (fd=<optimized out>, buf=0x7fffffffc650 "g\n", count=2) at fs/read_write.c:495
#7  <signal handler called>
#8  0x00007ffff7b01f10 in ?? ()
#9  0x0000000000000000 in ?? ()
(gdb) continue

# To reboot (refer kernel/debug/gdbstub.c:gdb_cmd_reboot())
maintenance packet R0

# To switch to the kdb shell
maintenance packet 3
# and exit gdb with CTRL-C

The following is the timeout error if the kernel being debugged does not trap before gdb tries to connect. If this happens, then quit gdb, run echo g | sudo tee /proc/sysrq-trigger in the vm (kernel being debugged) and then run the above gdb command.

# Error on timeout
host$ gdb kernel/vmlinux -ex "target remote /tmp/vm-serial-pty"
GNU gdb (Ubuntu 8.1-0ubuntu3.2) 8.1.0.20180409-git
Copyright (C) 2018 Free Software Foundation, Inc.
...
Remote debugging using /tmp/vm-serial-pty
Ignoring packet error, continuing...
warning: unrecognized item "timeout" in "qSupported" response
Ignoring packet error, continuing...
Remote replied unexpectedly to 'vMustReplyEmpty': timeout
(gdb)

# Optionally, to view gdb's protocol packets
host$ gdb kernel/vmlinux -ex "set debug remote 1" -ex "target remote /tmp/vm-serial-pty"

Using kbd

After issuing maintenance packet 3 in gdb shell, you would have switched to the kbd shell. Now you can connect to the serial port (pty) directly using screen.

Again note that you can see a blank screen (after invoking screen) unless you issue a kernel trap in the kernel being debugged. If this happens then run echo g | sudo tee /proc/sysrq-trigger in the vm (kernel being debugged).

host$ screen /tmp/vm-serial-pty
[1]kdb> ps
3 idle processes (state I) and 
51 sleeping system daemon (state M) processes suppressed,
use 'ps A' to see all.
Task Addr               Pid   Parent [*] cpu State Thread             Command
0xffff88026c51dc70     2033     2032  1    1   R  0xffff88026c51e0e0 *tee

0xffff88028e4c0000        1        0  0    1   S  0xffff88028e4c0470  init
0xffff88026d7bdc70      280        1  0    2   S  0xffff88026d7be0e0  python
0xffff88026d7fdc70      367        1  0    1   S  0xffff88026d7fe0e0  upstart-udev-br
0xffff88026efb5c70      369        1  0    1   S  0xffff88026efb60e0  udevd
...

# For help
[1]kdb> ?
Command         Usage                Description
----------------------------------------------------------
md              <vaddr>              Display Memory Contents, also mdWcN, e.g. md8c1
mdr             <vaddr> <bytes>      Display Raw Memory
mdp             <paddr> <bytes>      Display Physical Memory
mds             <vaddr>              Display Memory Symbolically
mm              <vaddr> <contents>   Modify Memory Contents
go              [<vaddr>]            Continue Execution
...

# To continue kernel execution
[1]kdb> go

# To revert to kgdboc
[1]kdb> kgdboc
Entering please attach debugger or use $D#44+ or $3#33

# Use Ctrl-A + k to kill the screen session.
# Connect over kgdboc using gdb.

Option 2: Debug via another VM

We can also spawn another virtual machine (say VM2) and debug the the first virtual machine (VM1).

For this we need to set up VM2 to be the client for the socket created by VM1. Go to the VM2’s settings and open “Serial Ports”. Enable COM1, select port mode to be “Host Pipe”. Give the path as “/tmp/vm-serial-socket”. Check the option “Connect to existing pipe/socket”.

Alternatively if you are using vagrant, then add the following to your Vagrantfile and then run vagrant reload.

  config.vm.provider "virtualbox" do |vb|
    vb.customize ["modifyvm", :id,
      "--uart1", "0x3f8", "4",
      "--uartmode1", "client", "  /tmp/vm-serial-socket"]
  end

Now start VM2 and verify the serial connection using -

# Verify serial port
vm2$ dmesg | grep serial
[    2.149591] serial8250: ttyS0 at I/O 0x3f8 (irq = 4) is a 16550A

Now VM2 sees a physical serial port instead of a socket. Hence we no longer need to use socat to create a pseudoterminal device. The following is a summary of updated commands -

# Debug via kgdboc
vm2$ sudo gdb /vagrant/kernel/vmlinux -ex "target remote /dev/ttyS0"

# Debug via kdb
vm2$ sudo screen /dev/ttyS0

Gotchas

Remote debugging (i.e. press CTRL-C to drop into gdb prompt) cannot be triggered over serial or via gdb. You need to break in from the guest kernel (using sysrq).

Extras

Kernel’s command line parameters

vm1$ cat /proc/cmdline 
BOOT_IMAGE=/boot/vmlinuz-3.2.0-pfwall-fedora-patches+ root=UUID=7d69eb56-ea8c-4243-8e7d-e2714fced818 ro console=tty0 nokaslr kgdboc=kbd,ttyS0,115200

Get kernel symbols at runtime

# Get runtime symbol table
vm1$ sudo head -5 /proc/kallsyms 
0000000000000000 D irq_stack_union
0000000000000000 D __per_cpu_start
0000000000004000 D gdt_page
0000000000005000 d exception_stacks
000000000000b000 d tlb_vector_offset

Note that only superuser (i.e. root) can read the kernel addresses of the symbols.

vm1$ head -5 /proc/kallsyms 
0000000000000000 D irq_stack_union
0000000000000000 D __per_cpu_start
0000000000000000 D gdt_page
0000000000000000 d exception_stacks
0000000000000000 d tlb_vector_offset

Print variables from the live kernel

The file /proc/kcore is the kernel’s core dump file at a specific time. gdb can be used to interpret it.

vm1$ gdb /vagrant/kernel/vmlinux /proc/kcore
GNU gdb (Ubuntu/Linaro 7.4-2012.04-0ubuntu2.1) 7.4-2012.04
Copyright (C) 2012 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-linux-gnu".
For bug reporting instructions, please see:
<http://bugs.launchpad.net/gdb-linaro/>...
Reading symbols from /home/vagrant/linux-pfwall/vmlinux...done.
[New process 1]
Core was generated by `BOOT_IMAGE=/boot/vmlinuz-3.2.0-pfwall-fedora-patches+ root=UUID=7d69eb56-ea8c-42'.
#0  0x0000000000000000 in ?? ()

(gdb) p vfs_read
$4 = {ssize_t (struct file *, char *, size_t, loff_t *)} 0xffffffff811597a1 <vfs_read>

(gdb) print irq_err_count
$11 = {counter = 0}

Reloading the core dump -

(gdb) core-file /proc/kcore
[New process 1]
Core was generated by `BOOT_IMAGE=/boot/vmlinuz-3.2.0-pfwall-fedora-patches+ root=UUID=7d69eb56-ea8c-42'.
#0  0x0000000000000000 in ?? ()

View printk messages in gdb

This feature is available on newer kernels such as v5.6.0. Use any of the following methods to enable printks’ in the gdb console -

Method #1: Add kgdbcon to kernel’s command line option.

Method #2: At runtime, use sysfs before configuring an I/O driver (i.e. kgdb)

echo 1 | sudo tee /sys/module/kgdb/parameters/kgdb_use_con

Debug from another (physical) machine

When using a physical serial connection, i.e. debugging one physical machine (instead of a VM) using another physical machine, it may be necessary to specify the baud rate. Also root permissions are needed to access the serial port (hence the sudo).

gdbserver

host$ file /dev/ttyUSB0
/dev/ttyUSB0: character special (188/0)

# Debug via physical serial
host$ sudo gdb vmlinux -ex "target remote /dev/ttyUSB0" -b 115200

kbd

# -L records the entire session in screenlog.0
host$ sudo screen -L /dev/ttyUSB0 115200

# To exit use: Ctrl+a k y

Updates

October 1, 2020:
- Guidelines to debug a physical machine
- Instructions to reboot kernel via gdb
April 7, 2020:
- Add kernel config option to disable read-only kernel protection
- Method to view printk messages in gdb
March 31, 2020:
- Add – Print variables from the live kernel
- Add – Kernel’s command line parameters
- Add – Get kernel symbols at runtime

Disable Hyper-V

Wed, 18 Sep 2019 00:00:00 -0400

Running 64-bit VirtualBox 6.0 VMs with Hyper-V does not work yet. So one possible solution is to turn off Hyper-V. However turning off Hyper-V from “Turn Windows Features On or Off” does not completely turn off Hyper-V.

VirtualBox Logs

The following logs represent the problem -

...
00:02.401079 NEM: Adjusting APIC configuration from X2APIC to APIC max mode.  X2APIC is not supported by the WinHvPlatform API!
00:02.401084 NEM: Disable Hyper-V if you need X2APIC for your guests!
00:02.401180 NEM: NEMR3Init: Active.
00:02.401218 MM: cbHyperHeap=0x140000 (1310720)
00:02.401801 CPUM: No hardware-virtualization capability detected
...
00:02.405504 VMSetError: F:\tinderbox\win-6.0\src\VBox\VMM\VMMR3\NEMR3Native-win.cpp(1463) int __cdecl nemR3NativeInitAfterCPUM(struct VM *); rc=VERR_NEM_VM_CREATE_FAILED
00:02.405534 VMSetError: Call to WHvSetupPartition failed: ERROR_SUCCESS (Last=0xc000000d/87)
00:02.405558 NEM: Destroying partition 0000000001763a60 with its 0 VCpus...
00:02.533155 ERROR [COM]: aRC=E_FAIL (0x80004005) aIID={872da645-4a9b-1727-bee2-5585105b9eed} aComponent={ConsoleWrap} aText={Call to WHvSetupPartition failed: ERROR_SUCCESS (Last=0xc000000d/87) (VERR_NEM_VM_CREATE_FAILED)}, preserve=false aResultDetail=-6805
00:02.533347 Console: Machine state changed to 'PoweredOff'
00:02.540651 Power up failed (vrc=VERR_NEM_VM_CREATE_FAILED, rc=E_FAIL (0X80004005))
...

Fix

To disable Hyper-V during boot, run the following in cmd.exe (Administrator) -

bcdedit /set hypervisorlaunchtype off

Note that WSL 2.0 relies on Hyper-V. So it will likely not work.

Setting up SSH Agent

Thu, 11 Jul 2019 00:00:00 -0400

Stuff about ssh agents.

Manaully add ssh keys to the agent

Add identities -

# On Linux
ssh-add -k

# On macOS
ssh-add -K

List loaded identities

ssh-add -l

Automatically load keys

To auto-load to the agent while logging in -

echo "AddKeysToAgent yes" >>~/.ssh/config

Versioning Resume (& LaTeX docs) using Git SHA

Fri, 05 Jul 2019 00:00:00 -0400

tl;dr Tagging resume using version numbers derived from git commit SHA. This helps in quickly retrieving any specific version of the resume. Also storing old PDF copies of the resume is not necessary.

Workflow

My resume is typeset using LaTeX and revision controlled under git. There is a Makefile in the root directory to take care of compiling the resume and producing the final PDF. My old resume editing workflow is roughly -

Update my resume (comprises of a couple of commits).
Tag the latest commit with an incrementing semantic version number. I used to maintain a major and minor release number, ex. v1.4.
Push the updates to GitHub.
Make a release on GitHub for the relevant tag and upload the a PDF version for that release.

So why maintain a version number? Simply put it helps me retrieve the exact copy of my resume that someone else has. And there is no need to store every PDF version of my resume.

However semantic versioning has issues! They are pretty useful in general, but they introduce quite a bit of overhead. My main dislike for them stems from the manual version increments that are necessary, esp. when doing just some minor updates.

The alternative. A rolling release with version numbers that correspond to the abbrev (or shortened) git commit SHA. The extra thing I add on top is the commit date. This helps my human brain quickly identify the latest version. The build date can also be used but it changes every time the resume gets recompiled. So my preference is the commit date.

Benefit. Now every time someone has a question regarding my resume, I simply get the abbrev SHA version. Now both of us are looking at the exact same copy of the resume!

My Setup

Step 1 - Update Makefile

I have the following recipe in my Makefile.

# $ man git-log
#	%h:  abbreviated commit hash
#	%ad: author date (format respects --date= option)

.PHONY: version
version:
	@git log --date=short --format="dt. %ad || ver. %h" -n 1

Example output:

$ make version
dt. 2019-07-02 || ver. 53267a4

Step 2 - Update LaTeX source

Inside my LaTeX document, the following snippet embeds the output of the above command.

\input|"make  --silent version"

Step 3 - Tweak LaTeX command-line flags

Now for the above command to work, the --shell-escape flag needs to be passed to pdflatex. The compilation command looks like -

# pdflatex
pdflatex --shell-escape resume.tex

# xelatex
xelatex --shell-escape resume.tex

Wrapping up

My entire Makefile:

SRC=resume.tex
TARGET=resume.pdf

$TARGET: *.tex *.cls
	xelatex --shell-escape ${SRC}
	open ${TARGET}

.PHONY: version
version:
	@git log --date=short --format="dt. %ad || ver. %h" -n 1

.PHONY: clean
clean:
	rm -rf *.aux *.log *.out *.synctex.gz ${TARGET}

These are all the steps needed to get a rolling release. The final document looks like -

One important point is that any changes made to the resume need to be committed first. Otherwise the resume will get marked with an older (and incorrect) version number.

Bonus

Alternatively, a TeX only approach looks like -

% Store the command output in a temporary file
\immediate\write18{git log --date=short --format="dt. %ad || ver. %h" -n 1 >gitcommit.tex}

% Use the output
\input{gitcommit.tex}

% Remove the temporary file
\immediate\write18{rm gitcommit.tex}

Quirks about PCI

Fri, 28 Jun 2019 00:00:00 -0400

tl;dr Notes while working with PCI.

The 64-bit PCI registers are broken up as in to two 32-bit registers. So,

REGISTER is represented as
	REGISTER_1
	REGISTER_1

Generating favicon for personal blog

Wed, 21 Nov 2018 00:00:00 -0500

We will go through creating a favicon (an avatar), converting it to necessary image format(s), and finally configuring webpages to use it.

First we need to create a favicon to add it to our website. For a personal blog I prefer using a personalized avatar. I used avatarmaker.com to create my avatar and downloaded the svg version to have a high quality image. Now the only bowsers that support svg favicons are Firefox and Safari. Hence it is a good idea to create a png version as well for say Chrome users.

Note: You might also want to create an .ico version.

Generating the png version

For this I used svgexport. This utility can convert .svg images to .png, .jpg and .jpeg versions.

First you will need to install npm. Go to this link to download and install npm.

To install svgexport on mac type the following in the terminal:

sudo npm install --global --unsafe-perm=true --allow-root svgexport

Now to generate the png version of the favicon, use

svgexport images/favicon.svg images/favicon.png 64x

For me, the file sizes were:

favicon.svg = 51 KB
favicon.png = 1.1 MB

Note: I tried using ImageMagick to convert the .svg file to .png file. However it gave me errors and failed converting the svg file downloaded from avatarmaker.com.

Embedding the favicons in html page

Next we need to add the following html tags to the <head>...</head> section of all webpages:

<link rel="shortcut icon" type="image/svg" href="/images/favicon.svg" />
<link rel="shortcut icon" type="image/png" href="/images/favicon.png" />

Now the favicon should be picked up by web browsers whenever your website gets loaded.

References

https://avatarmaker.com/
https://github.com/shakiba/svgexport
https://stackoverflow.com/questions/9853325/how-to-convert-a-svg-to-a-png-with-image-magick
https://www.w3.org/2005/10/howto-favicon
https://codex.wordpress.org/Creating_a_Favicon

Managing multiple repositories using mr

Thu, 02 Apr 2015 00:00:00 -0400

tl;dr Using the mr tool, you can issue commands to multiple repositories in parallel. You can pull new changes, do simultaneous commits or just get the last commit log.

My typical day starts with updating a few code repositories. When dealing with a small number of repos, it’s not a problem to do a git pull for each repo. But once you handle more than 4-5 repos, it’s worth automating the process.

Also, at times when you forget to commit some work, it can be handy to run git status on all your repos and see which repos need attention.

I recently came across a tool made by Joey: myrepos (or mr). The tool supports many types of repositories - git, mercurial, cvs, svn, etc.

Installation

# Debian / Ubuntu (recent versions)
sudo apt-get install mr

# Mac
brew install mr

# Others:
# - Just put the `mr` perl script in your `PATH` and it should work!

Core Usage

The tool looks for a file called .mrconfig in the current directory. This is an ini file which tracks all the repositories using relative paths, for example -

# file: ~/.mrconfig

[.homesick/repos/dotfiles]
checkout = git clone 'git@github.com:mitthu/dotfiles.git' 'dotfiles'

To start managing an existing repository (ex. handbook), run -

# handbook repo is located in ~/handbook
$ mr register handbook

The above updates the .mrconfig to -

# file: ~/.mrconfig

[.homesick/repos/dotfiles]
checkout = git clone 'git@github.com:mitthu/dotfiles.git' 'dotfiles'

[handbook]
checkout = git clone 'git@github.com:mitthu/handbook.git' 'handbook'

Action: git checkout

To checkout/clone the repositories in .mrconfig (skips already cloned repositories), run -

$ mr checkout
mr checkout: /Users/mitthu/.homesick/repos/dotfiles
Cloning into 'dotfiles'...
remote: Enumerating objects: 1406, done.
remote: Total 1406 (delta 0), reused 0 (delta 0), pack-reused 1406
Receiving objects: 100% (1406/1406), 455.60 KiB | 3.21 MiB/s, done.
Resolving deltas: 100% (823/823), done.

mr checkout: /Users/mitthu/handbook
Cloning into 'handbook'...
remote: Enumerating objects: 32, done.
remote: Total 32 (delta 0), reused 0 (delta 0), pack-reused 32
Receiving objects: 100% (32/32), 4.75 KiB | 4.75 MiB/s, done.
Resolving deltas: 100% (14/14), done.

mr checkout: finished (2 ok)

Action: git pull

Now to update (i.e. do a git pull on) all managed repos, run -

# Update all managed repositories (sequential)
$ mr update
mr update: /Users/mitthu/.homesick/repos/dotfiles
Already up to date.

mr update: /Users/mitthu/handbook
Already up to date.

mr update: finished (2 ok)

# Run `git pull` simultaneously, say on 8 repos at a time
$ mr update -j8

Action: git status

To find out any uncommited/unpushed changes, run -

$ mr status
mr status: /home/mitthu/.homesick/repos/dotfiles
8c6f508 (HEAD, master) Add X automation script.

mr status: /home/mitthu/handbook

mr status: finished (2 ok)

My Workflow

I have mutiple .mrconfig’s, but I do make sure not to have too many configs. Using too many configs, would defeat the very purpose of automation, as you will be doing mr update in multiple directories.

I have a directory called repos in my home directory -

# Directory structure of ~/repos
~/repos
|-- following
|   |-- ansible-django
|   .... (all repos I follow) ....
|   `-- .mrconfig
|-- ops
|   |-- dotfiles_deploy
|   .... (my personal automation stuff) ....
|   `-- .mrconfig
`-- work
    |-- gitosis-admin
    .... (work related stuff) ...
    `-- .mrconfig

Apart from the above, I also have an .mrconfig in my home directory. In this config, I add all the repos which I use daily and all my current projects. So everyday, I just have to do an mr update in my home folder.

There are a few repos to which I do not have push permissions (~/repos/following/*). mr will fail to push to such repos. It does not look very encouraging to see commands failing. To fix this, I override the mr push command for those specific repos. So for example, if I don’t want to push to the handbook repo, I will update my .mrconfig as follows -

# file: ~/.mrconfig

[handbook]
checkout = git clone 'git@github.com:mitthu/handbook.git' 'handbook'
push = echo "DO NOT PUSH"

The above would just run echo out “DO NOT PUSH” on an mr push run. For example -

$ mr push
mr push: /home/mitthu/handbook
DO NOT PUSH

mr push: finished (1 ok)

Tips

Teach new commands to mr (or overrride existing commands) via the ‘DEFAULT’ section in .mrconfig, for example -

[DEFAULT]
# Teach mr how to "mr gc" in git repos.
git_gc = git gc "$@"
# Always fetch and rebase changes
git_update = git pull --rebase

Use cron and mr to keep forked repositories up to date with upstream.
Use cron, mr and pushover to send notifications about uncommitted/unpushed changes at 5:00 pm.
Doc: man page of mr (man mr)
Doc: Homepage - mr

Last updated on: June 18, 2019