井民全觀點 (Jing's Perspective)

[vm] Pause all VM instances if a running instances do not active for a long time

noreply@blogger.com (Jing(井民全)) — Mon, 13 Jan 2025 09:25:00 +0000

Pause all VM instances if a running instances do not active for a long time.

使用 VirtualBox 常常遇到 VM XDM does not respones any user input when the Windows host backed from sleep or hibernate. If I got this, a comon solution is ssh to the VM and restart the gdm by folloing command.

sudo systemctl restart gdm

Follow the procedure that will pause all VM instances if you do not touch the VM for a long time. It will prevent the XDM issue when the host backed from the sleep.

1. Concept

目前無法讓 VM 在系統即將進入 sleep 模式時, 收到 Windows 事件通知. 所以改用 AutoHotKey (download)

2. Pause all VM (script)

Windows Batch file

@ECHO OFF

REM 列出所有正在運行的虛擬機

FOR /F "tokens=1 delims= " %%i IN ('VBoxManage list runningvms') DO (

ECHO 暫停虛擬機: %%i

VBoxManage controlvm "%%i" pause

)

ECHO 所有虛擬機已暫停。

PAUSE

E.g.

File: pause_all_vm.bat

@ECHO OFF

REM List all running VM instances

FOR /F "tokens=1 delims= " %%i IN ('"C:\Program Files\Oracle\VirtualBox\VBoxManage" list runningvms') DO (

ECHO Pause the instance: %%i

"C:\Program Files\Oracle\VirtualBox\VBoxManage" controlvm "%%i" pause

)

ECHO All instances are paused.

PAUSE

3. AutoHotkey script

File: pause_al_vm.ahk

#Requires AutoHotkey v2.0

SetTitleMatchMode("RegEx") ;

windowTitle := ".*Running.*"

inactiveTime := 60*1000 ; Time in milliseconds (e.g., 60000 ms = 1 minute)

checkInterval := 1000 ; Interval to check the window status in milliseconds

idleDuration := 0

MsgBox ("AuotHotKey script Begin")

batFilePath := "pause_al_vm.bat"

SetTimer(CheckWindow, checkInterval)

CheckWindow() {

global windowTitle, inactiveTime, idleDuration

if WinExist(windowTitle) && !WinActive(windowTitle) { ; Check if the window exists and is not active

idleDuration += checkInterval ; Increment idle duration

if (idleDuration >= inactiveTime) {

MsgBox("AuotHotKey: Run pause all VM instances batch.")

Run(batFilePath)

idleDuration := 0 ; Reset idle duration after action

}

} else {

idleDuration := 0 ; Reset if the window is active or does not exist

}

Esc::

{

; Display a message box with Yes and No options

if (MsgBox("Are you sure you want to exit the script?", "Exit Confirmation", 4) == "Yes")

{

ExitApp ; Exit if the user clicks Yes

}

[docker, gui] How to start a container that can run an GUI application

noreply@blogger.com (Jing(井民全)) — Tue, 22 Oct 2024 16:58:00 +0000

xhost +local:* # allow host X server to be connected

docker run -it --name my_container -v /home/jing:/jing -v /tmp/.X11-unix/:/tmp/.X11-unix/ -e DISPLAY=$DISPLAY -d ubuntu:20.04

docker exec -it my_container /bin/bash

==================

=================

# stop and remove the container

docker stop my_container

docker rm my_container

# disallow X server to be connected

xhost -local:*

[mapbox, android] How to write an Android Mapbox APP

noreply@blogger.com (Jing(井民全)) — Wed, 13 Sep 2023 15:52:00 +0000

How to write an Android Mapbox APP

井民全, Jing, mqjing@gmail.com

Google doc: (view)

This document is following the guide of MapBox MAP SDK for Android. To make clear, I added some snapshots for each step.

Table of Contents

1. Install Android Studio 2

2. Create New Project 2

2.1. Configure Credentials 3

2.1.1. Setup Secure Access Token 3

2.1.2. Setup Public Token 4

2.2. Configure Permissions 5

2.3. Add the Dependency 6

5. Result 10

1. Install Android Studio

Official site: https://developer.android.com/studio

2. Create New Project

Step 1: Choose a template

Step 2: Setup the project

Choose [Build configuration language]: Groovy DSL (build.grandle)

2.1. Configure Credentials

Get the mapbox (1) public access token and (2)secret access token (Download: Read)

A public access token: From your account's tokens page, you can either copy your default public token or click the Create a token button to create a new public token.
A secret access token with the Downloads:Read scope.

From your account's tokens page, click the Create a token button.
From the token creation page, give your token a name and make sure the box next to the Downloads:Read scope is checked.
Click the Create token button at the bottom of the page to create your token.
The token you've created is a secret token, which means you will only have one opportunity to copy it somewhere secure.

2.1.1. Setup Secure Access Token

Step 1:Create a file named as gradle.properties

Location: User home/.grandle/

Filename: gradle.propertie

Step 2: Add the token

See this to create the token.

MAPBOX_DOWNLOADS_TOKEN=TOKEN-STRING

E.g.

2.1.2. Setup Public Token

2.2. Configure Permissions

If you plan to display user's location, we need to add the following permissions setting.

File: manifests

<uses-permission android:name="android.permission.ACCESS_COARSE_LOCATION" />

<uses-permission android:name="android.permission.ACCESS_FINE_LOCATION" />

E.g.

2.3. Add the Dependency

Make sure the [Build configuration language] is Groovy DSL (build.grandle).

Step 1: Open [setting.gradle]

Step 2: Setup the configuration as following in blue.

File: setting.gradle

dependencyResolutionManagement {

repositoriesMode.set(RepositoriesMode.FAIL_ON_PROJECT_REPOS)

repositories {

google()

mavenCentral()

maven {

url 'https://api.mapbox.com/downloads/v2/releases/maven'

authentication {

basic(BasicAuthentication)

}

credentials {

// Do not change the username below.

// This should always be `mapbox` (not your username).

username = "mapbox"

// Use the secret token you stored in gradle.properties as the password

password = MAPBOX_DOWNLOADS_TOKEN

}

Step 3: Add Mapbox SDK dependency

Open [build.gradle] and add the SDK dependency.

File: build.gradle

dependencies {

implementation 'com.mapbox.maps:android:10.16.0'

}

E.g.

3. Coding

3.1. Add a map

Step 1: Overwrite the MainActivity

File: MainActivity.kt

package com.example.myapplication2

import android.os.Bundle

import androidx.appcompat.app.AppCompatActivity

import com.mapbox.maps.MapView

import com.mapbox.maps.Style

var mapView: MapView? = null

class MainActivity : AppCompatActivity() {

override fun onCreate(savedInstanceState: Bundle?) {

super.onCreate(savedInstanceState)

setContentView(R.layout.activity_main)

mapView = findViewById(R.id.mapView)

mapView?.getMapboxMap()?.loadStyleUri(Style.MAPBOX_STREETS)

}

E.g.

Step 2: Write the layout

File: activity_main.xml

<com.mapbox.maps.MapView xmlns:android="http://schemas.android.com/apk/res/android"

xmlns:tools="http://schemas.android.com/tools"

xmlns:mapbox="http://schemas.android.com/apk/res-auto"

android:id="@+id/mapView"

android:layout_width="match_parent"

android:layout_height="match_parent"

mapbox:mapbox_cameraTargetLat="25.05"

mapbox:mapbox_cameraTargetLng="121.50"

mapbox:mapbox_cameraZoom="9.0"

E.g.

4. Run

Step 1: Make sure the [Run/Debug configuration] and [Running device]

Step 2: [Run] -> [Run app]

Result

Enjoy!

By Jing.

6. Reference

https://docs.mapbox.com/android/maps/guides/install/

[v4l2, camera, capture] How to use v4l2 API to capture the V4L2_PIX_FMT_MJPEG stream and then decode it to show the image using OpenCV

noreply@blogger.com (Jing(井民全)) — Sun, 13 Aug 2023 14:47:00 +0000

How to use v4l2 API to capture the V4L2_PIX_FMT_MJPEG stream and then decode it to show the image using OpenCV

井民全, Jing, mqjing@gmail.com

github

main.cpp

#include <iostream>

#include <fcntl.h>

#include <unistd.h>

#include <sys/ioctl.h>

#include <linux/videodev2.h>

#include <sys/mman.h>

#include <opencv2/opencv.hpp>

// #include <opencv2/imgcodecs.hpp>

#define DEVICE_PATH "/dev/video0"

#define IMAGE_WIDTH 640

#define IMAGE_HEIGHT 480

int main() {

int fd = open(DEVICE_PATH, O_RDWR);

if (fd == -1) {

perror("Failed to open device");

return 1;

}

struct v4l2_capability caps;

if (ioctl(fd, VIDIOC_QUERYCAP, &caps) == -1) {

perror("Failed to query capabilities");

close(fd);

return 1;

}

if (!(caps.capabilities & V4L2_CAP_VIDEO_CAPTURE)) {

std::cerr << "Device does not support video capture" << std::endl;

close(fd);

return 1;

}

struct v4l2_format format;

format.type = V4L2_BUF_TYPE_VIDEO_CAPTURE;

format.fmt.pix.width = IMAGE_WIDTH;

format.fmt.pix.height = IMAGE_HEIGHT;

format.fmt.pix.pixelformat = V4L2_PIX_FMT_MJPEG; // V4L2_PIX_FMT_YUYV; // V4L2_PIX_FMT_RGB24; //V4L2_PIX_FMT_MJPEG; // MJPEG pixel format

format.fmt.pix.field = V4L2_FIELD_NONE;

if (ioctl(fd, VIDIOC_S_FMT, &format) == -1) {

perror("Failed to set format");

close(fd);

return 1;

}

struct v4l2_requestbuffers req;

req.count = 1;

req.type = V4L2_BUF_TYPE_VIDEO_CAPTURE;

req.memory = V4L2_MEMORY_MMAP;

if (ioctl(fd, VIDIOC_REQBUFS, &req) == -1) {

perror("Failed to request buffer");

close(fd);

return 1;

}

struct v4l2_buffer buffer;

memset(&buffer, 0, sizeof(buffer));

buffer.type = V4L2_BUF_TYPE_VIDEO_CAPTURE;

buffer.memory = V4L2_MEMORY_MMAP;

buffer.index = 0;

if (ioctl(fd, VIDIOC_QUERYBUF, &buffer) == -1) {

perror("Failed to query buffer");

close(fd);

return 1;

}

void *buffer_start = mmap(nullptr, buffer.length, PROT_READ | PROT_WRITE, MAP_SHARED, fd, buffer.m.offset);

if (buffer_start == MAP_FAILED) {

perror("Failed to map buffer");

close(fd);

return 1;

}

if (ioctl(fd, VIDIOC_QBUF, &buffer) == -1) {

perror("Failed to queue buffer");

close(fd);

return 1;

}

enum v4l2_buf_type type = V4L2_BUF_TYPE_VIDEO_CAPTURE;

if (ioctl(fd, VIDIOC_STREAMON, &type) == -1) {

perror("Failed to start streaming");

close(fd);

return 1;

}

if (ioctl(fd, VIDIOC_DQBUF, &buffer) == -1) {

perror("Failed to dequeue buffer");

close(fd);

return 1;

}

// SAVE to JPEG file for test

FILE* fout;

fout = fopen("test.jpg", "wb");

if (!fout) {

perror("Cannot open image");

exit(EXIT_FAILURE);

}

printf("(buffer).bytesused 2= %d\n", (buffer).bytesused);

fwrite(buffer_start, (buffer).bytesused, 1, fout);

fclose(fout);

cv::Mat image2 = cv::imread("./test.jpg");

if (image2.empty() == true) {

std::cout << "image2 empty"<< std::endl;

return 0;

}

cv::imshow("opencv_logo", image2);

printf("image2 cols = %d\n", image2.cols);

printf("image2 rows = %d\n", image2.rows);

printf("image2 depth = %d\n", image2.depth());

printf("image2 type() = %d\n", image2.type());

// OPENCV Part

unsigned char* newbuffer = (unsigned char*) malloc((buffer).bytesused);

memcpy(newbuffer, buffer_start, (buffer).bytesused);

// TEST

FILE* fk;

fk = fopen("test.jpg", "rb");

fread(newbuffer, (buffer).bytesused, 1, fk);

fclose(fk);

cv::Mat image = cv::imdecode(cv::Mat(1, (buffer).bytesused, CV_8UC1, (void*) newbuffer), cv::IMREAD_COLOR); // work

// work

cv::Mat raw(1, (buffer).bytesused, CV_8UC1, buffer_start);

cv::Mat image = cv::imdecode(raw, cv::IMREAD_COLOR);

// work

cv::Mat raw(format.fmt.pix.height, format.fmt.pix.width, CV_8UC3 , buffer_start);

cv::Mat image = cv::imdecode(raw, cv::IMREAD_COLOR);

// work

cv::Mat raw(format.fmt.pix.height, format.fmt.pix.width, CV_8UC3 );

raw.data = (unsigned char*) buffer_start;

cv::Mat image = cv::imdecode(raw, cv::IMREAD_COLOR);

// cv::Mat image = cv::imdecode(cv::Mat(1, (buffer).bytesused, CV_8UC1, buffer_start), cv::IMREAD_COLOR); // work

cv::Mat image = cv::imdecode(cv::Mat(1, (buffer).bytesused, CV_8UC1, (void*) buffer_start), cv::IMREAD_COLOR); // work

printf("cols = %d\n", image.cols);

printf("rows = %d\n", image.rows);

printf("depth = %d\n", image.depth());

printf("type() = %d\n", image.type());

if (image.empty() == true) {

std::cout << "image empty"<< std::endl;

return 0;

}

cv::imshow("Captured Image", image);

cv::waitKey(0);

if (ioctl(fd, VIDIOC_STREAMOFF, &type) == -1) {

perror("Failed to stop streaming");

close(fd);

return 1;

}

munmap(buffer_start, buffer.length);

close(fd);

return 0;

}

Result

References

[opencv, decode, jpeg] How to decode and show the jpeg stream

noreply@blogger.com (Jing(井民全)) — Sun, 13 Aug 2023 14:21:00 +0000

How to decode and show the jpeg stream

井民全, Jing, mqjing@mail.com

#include <opencv2/opencv.hpp>

#include <iostream>

int main() {

// Define the JPEG data as a char array

unsigned char jpegData[] = {

// Your JPEG data here

};

int jpegDataSize = sizeof(jpegData);

// Decode the JPEG data using imdecode

cv::Mat decodedImage = cv::imdecode(cv::Mat(1, jpegDataSize, CV_8UC1, jpegData), cv::IMREAD_COLOR);

// Check if the image was successfully decoded

if (decodedImage.empty()) {

std::cout << "Failed to decode the image." << std::endl;

return -1;

}

// Display the decoded image

cv::imshow("Decoded Image", decodedImage);

cv::waitKey(0);

return 0;

}

[btrfs, raid] How to setup a RAID single filesystem from 3 disks + local loop device associated file

noreply@blogger.com (Jing(井民全)) — Fri, 06 Jan 2023 20:19:00 +0000

How to setup a RAID single filesystem from 3 disks + local loop file

We can use btrfs filesystem easily to setup a software RAID-0. In this document, I'll show you the procedure that not only setup the RAID-0 which combine all external storage devices but also expand it with a big local loop file into an one logical storage.

Table of content

1. Quick 2

2. Procedure 4

2.1. Setup BTRFS file system 4

2.2. Mount and create subvolume 4

2.2.1. Step 1: create subvolume 4

2.2.2. Step 2: Get the UUID 5

2.3. Expand the capacity 6

2.3.1. Create a loop device with 100GB 7

2.3.1.1. Step 1: Create a regular file 7

2.3.1.2. Step 2: Make btrfs filesystem 7

2.4. Associate a loop device 7

2.4.1. Associate the device (manually, Optional) 7

2.4.2. Permanent loop device 7

2.4.2.1. Step 1: Create the associate file 7

2.4.2.2. Step 2: Set executable permission 8

2.4.2.3. Step 3: Create a new systemd service 8

2.4.2.4. Verification 9

2.4.3. Expand the original BTRFS system 9

3. References 11

4. Appendix 11

5. Debug 11

1. Quick

Following quick instructions were used for my nextcloud server.

# == (A) Setup a btrfs RAID-0 logic device==

# Data: Single, M: RAID1

sudo apt-get install btrfs-progs

sudo mkfs.btrfs -L mybtrfs-pool -d single -f /dev/sda /dev/sdb /dev/sdc

sudo mkdir /media/nextcloud/data

sudo mount /dev/sda /media/nextcloud/data/

# verify

df -h # <-- check the /dev/sda capacity

sudo btrfs filesystem show # <-- check the v-dev structure

# == (B) Setup a loop device: 100GB ==

# create a regular file

for i in {1..1}; do truncate -s 100G vdev_$i.img; done

mkfs.btrfs vdev_1.img

# associate a loop device

sudo losetup /dev/loop99 /home/pi/vdev_1.img # 使用 99 的原因是怕別的軟體占用,

# 可用 sudo losetup -f 找出可用的 device

# link: ref

# permanent loop device

vi /home/pi/mybtrfs-loop.sh

File: /home/pi/mybtrfs-loop.sh

#!/bin/bash

sudo losetup /dev/loop99 /home/pi/vdev_1.img

# Important!!

# Mount the system. Don't mount the filesystem in fstab file

# replace fstab, use this line to mout btrfs system

sudo mount -o loop /home/pi/vdev_1.img /media/nextcloud/data/

# undo

# sudo losetup --detach /dev/loop99 vdev_1.img

chmod u+x /home/pi/mybtrfs-loop.sh # important

# create a new systemd service

sudo vi /etc/systemd/system/mybtrfs-loop.service

File: /etc/systemd/system/mybtrfs-loop.service

[Unit]

Description= mybtrfs loop

[Timer]

OnBootSec=60

[Service]

Type=simple

RemainAfterExit=yes

ExecStart=/home/pi/mybtrfs-loop.sh

TimeoutStartSec=0

[Install]

WantedBy=default.target

# activate the chage and make sure this is called autlmatically every reboot

sudo systemctl daemon-reload

sudo systemctl enable mybtrfs-loop.service

sudo systemctl | grep mybtrfs-loop

# verification

sudo shutdown -r now #<---------- reboot test

sudo losetup -a # check if the loop6 was mapped

# == Expand the original BTRFS system ==

sudo btrfs filesystem show # show original

df -h # the original capacity

# (C) == Expand the original BTRFS system ==

sudo btrfs device add /dev/loop99 /media/nextcloud/data/ -f # change the capacity

df -h # check the new capacity

# verification

sudo shutdown -r now

df -h

2. Procedure

2.1. Setup BTRFS file system

Data: Single, M: RAID1

sudo apt-get install btrfs-progs

sudo mkfs.btrfs -L mybtrfs-pool -d single -f /dev/sda /dev/sdb /dev/sdc

sudo mkdir /mnt/mybtrfs-test-1

sudo mount /dev/sda /mnt/mybtrfs-test-1/

Verification

df -h

2.2. Mount and create subvolume

2.2.1. Step 1: create subvolume

sudo mount /dev/sda /mnt/mybtrfs-test-1/

sudo btrfs subvolume create /mnt/mybtrfs-test-1/sub1

E. g.

2.2.2. Step 2: Get the UUID

sudo btrfs filesystem show

E.g.

Get the UUID: 3d087427-ec67-4d28-b593-a8d51dc5c28e

1.2.3. Set up fstab

注意: 如果你後面要用 loop device, 新增 btrfs loop 空間, 就不要在 fstab 自動 mout. 你要用 systemd service 的方式來 mount. 否則會因為 filesystem 還沒起來, 導致開不了機

sudo vi /etc/fstab

File: /etc/fstab

UUID=3d087427-ec67-4d28-b593-a8d51dc5c28e /mnt/mybtrfs-test-1 btrfs defaults 0 0

1.2.4. Verification

sudo shutdown -r now

df -h

E.g.

sudo btrfs filesystem show

E.g.

2.3. Expand the capacity

2.3.1. Create a loop device with 100GB

2.3.1.1. Step 1: Create a regular file

for i in {1..1}; do truncate -s 100G vdev_$i.img; done

2.3.1.2. Step 2: Make btrfs filesystem

mkfs.btrfs vdev_1.img

2.4. Associate a loop device

2.4.1. Associate the device (manually, Optional)

sudo losetup -a

sudo losetup /dev/loop6 vdev_1.img # 使用 6 的原因是怕別的軟體占用,

# 可用 sudo losetup -f 找出可用的 device

# link: ref

# sudo losetup /dev/loop6 /home/pi/vdev_1.img

# undo

# sudo losetup --detach /dev/loop6 vdev_1.img

2.4.2. Permanent loop device

2.4.2.1. Step 1: Create the associate file

vi /home/pi/mybtrfs-loop.sh

File: /home/pi/mybtrfs-loop.sh

#!/bin/bash

sudo losetup /dev/loop99 /home/pi/vdev_1.img

# Important!!

# Mount the system. Don't mount the filesystem in fstab file

# replace fstab, use this line to mout btrfs system

sudo mount -o loop /home/pi/vdev_1.img /mnt/mybtrfs-test-1/

# undo

# sudo losetup --detach /dev/loop6 vdev_1.img

2.4.2.2. Step 2: Set executable permission

chmod u+x /home/pi/mybtrfs-loop.sh # important

2.4.2.3. Step 3: Create a new systemd service

sudo vi /etc/systemd/system/mybtrfs-loop.service

File: /etc/systemd/system/mybtrfs-loop.service

[Unit]

Description= mybtrfs loop

[Timer]

OnBootSec=60

[Service]

Type=simple

RemainAfterExit=yes

ExecStart=/home/pi/mybtrfs-loop.sh

TimeoutStartSec=0

[Install]

WantedBy=default.target

Step 4: Activate the chage and make sure this is called autlmatically every reboot

sudo systemctl daemon-reload

sudo systemctl enable mybtrfs-loop.service

sudo systemctl | grep mybtrfs-loop

E.g.

2.4.2.4. Verification

sudo shutdown -r now #<---------- reboot test

sudo losetup -a # check if the loop6 was mapped

E.g.

2.4.3. Expand the original BTRFS system

sudo btrfs filesystem show # show original

df -h # the original capacity

sudo btrfs device add /dev/loop6 /mnt/mybtrfs-test-1/ -f # change the capacity

df -h # check the new capacity

E. g.

Check the original capacity

Fig. This is an example that use /dev/loop1.

After resize

reboot

3. References

4. Appendix

5. Debug

How to see the Systemd log (ref)

journalctl -u service-name.service

E. g.

sudo journalctl -u mybtrfs-loop.service

For 64bit OS, following instructions will failure in loop.service.

sudo mount /dev/sda /mnt/mybtrfs-test-1/

The error message

Dec 20 00:48:16 raspberrypi systemd[1]: Started mybtrfs loop.

Dec 20 00:48:17 raspberrypi sudo[425]: root : PWD=/ ; USER=root ; COMMAND=/usr/sbin/losetup /dev/loop6 /home/pi/vdev_1.img

Dec 20 00:48:17 raspberrypi sudo[425]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=0)

Dec 20 00:48:17 raspberrypi sudo[425]: pam_unix(sudo:session): session closed for user root

Dec 20 00:48:17 raspberrypi sudo[537]: root : PWD=/ ; USER=root ; COMMAND=/usr/bin/mount -L mybtrfs-pool /mnt/mybtrfs-test-1/

Dec 20 00:48:17 raspberrypi sudo[537]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=0)

Dec 20 00:48:18 raspberrypi mybtrfs-loop.sh[540]: mount: /mnt/mybtrfs-test-1: wrong fs type, bad option, bad superblock on /dev/sdc, missing codepage or helper program, or other error.

Dec 20 00:48:18 raspberrypi sudo[537]: pam_unix(sudo:session): session closed for user root

Dec 20 00:48:18 raspberrypi systemd[1]: mybtrfs-loop.service: Main process exited, code=exited, status=32/n/a

Dec 20 00:48:18 raspberrypi systemd[1]: mybtrfs-loop.service: Failed with result 'exit-code'.

Work-around

Postpone to mount the filesystem after system boot in 10sec.

File: /etc/systemd/system/mybtrfs-loop.service

...

[Timer]

OnBootSec=60

...

2. MOVE the mount command in .bashrc

vi .bashrc

sudo mount -L mybtrfs-pool /mnt/mybtrfs-test-1/

Fstab mount? 直接 reboot 會出錯!

不知道為甚麼> 只要加入下面的 loop devie

sudo btrfs device add /dev/loop6 /mnt/mybtrfs-test-1/ -f # change the capacity

fstab 動態 mount sudo mount /dev/sda /mnt/mybtrfs-test-1/

就會出錯!

Q: 能不能直接用 systemd script 取代 fstab?

Note for RAID5 configuration

If the /mnt/mybtrfs is btrfs RAID5, available capacity 170GB even you add a single vdev.ce

RAid5

sudo mkfs.btrfs -L mybtrfs-pool -d raid5 -m raid5 -f /dev/sda /dev/sdb /dev/sdc

sudo mount /dev/sda /mnt/mybtrfs-test-1/

See following.

[nextcloud, external, local] How to enable external storage (Local)

noreply@blogger.com (Jing(井民全)) — Fri, 06 Jan 2023 19:34:00 +0000

How to enable external storage (Local)

井民全, Jing, mqjing@gmail.com

You can use nextcloud external storage app to expand your storage capacity. Here, I'll show you how to do that.

1. Installation

Step 1: Add [External storage support] App

Command

[Apps]: External storage support

2. Setup

Step 1: Check the Disk information

Command

[Administration settings] -> [System]

https://192.168.1.200/index.php/settings/admin/serverinfo

E.g.

https://192.168.1.200/index.php/settings/admin/serverinfo

Now, we know the local path is the '/media/nextcloud/data'.

Step 2: Setup the local folder

Command

[Administrator setting] -> Administration: External Storage

Folder name: Local

Authentication: None

Configuration: /media/nextcloud/data

3. Test

Folder

Test 1: create a folder: test

Test 2: Create md5 file

Test 3: upload a photo and video

4. Reference

https://docs.nextcloud.com/server/latest/admin_manual/configuration_server/index.html

[memory, zram] How to enable zram

noreply@blogger.com (Jing(井民全)) — Thu, 05 Jan 2023 17:25:00 +0000

How to enable zram

This document is following the procedures metioned by [1].

When memory was low, system would unresponsive and even freeze. ZRAM creates RAM based block storage named /dev/zram0 (or 1, 2, 3, etc). Pages written there are compressed and stored in memory. This allows for very fast I/O and the compression savings provide additional memory. See the document in detail.

1. Quick

sudo apt install zram-tools

sudo vi /etc/default/zramswap

File: /etc/default/zramswap

...

PERCENT=50 # 50% ram use zram

....

sudo zramswap stop

sudo zramswap start

sudo zramswap status

# Kernel

sudo vi /etc/sysctl.conf

File: /etc/sysctl.conf

vm.vfs_cache_pressure=500

vm.swappiness=100

vm.dirty_background_ratio=1

vm.dirty_ratio=50

sudo sysctl --system # enable the parameters

2. Procedure

2.1. Before

Verification

free -m

sudo cat /proc/swaps

2.2. Install zram-tool

sudo apt install zram-tools

E.g.

Check

sudo cat /proc/swaps

zramswap status

2.3. Config

sudo vi /etc/default/zramswap

File: /etc/default/zramswap

...

PERCENT=50 # 50% ram use zram

....

sudo zramswap stop

sudo zramswap start

sudo zramswap status

E.g.

2.4. Verification

sudo zramswap status

E.g.

3. Setup Kernel parameters

3.1. Before

sudo cat /proc/swaps

3.2. Setup

sudo vi /etc/sysctl.conf

File: /etc/sysctl.conf

vm.vfs_cache_pressure=500

vm.swappiness=100

vm.dirty_background_ratio=1

vm.dirty_ratio=50

sudo sysctl --system

E.g.

3.3. Verification

sudo cat /proc/swaps

4. References

[pi, memory, swap] How to increase your swap file size for your Raspberry Pi

noreply@blogger.com (Jing(井民全)) — Mon, 26 Dec 2022 15:55:00 +0000

How to increase your swap file size for your Raspberry Pi?

井民全, Jing, mqjing@gmail.com

1. Quick

top # check the original swap file size

# turn off the swap system

sudo dphys-swapfile swapoff

sudo vi /etc/dphys-swapfile # setup new swap size

File: /etc/dphys-swapfile

...

CONF_SWAPSIZE=1024

# recreate the swap file

sudo dphys-swapfile setup

# turn on the swap file system

sudo dphys-swapfile swapon

# Check

top

2. Procedure

Check current swap size

top

E.g.

2.1. Step 1: Turn off the swap file function

sudo dphys-swapfile swapoff

E.g.

Check

top

2.2. Step 2: Setup the configuration file

vi sudo vi /etc/dphys-swapfile

File: /etc/dphys-swapfile

...

CONF_SWAPSIZE=1024

E.g.

2.3. Step 3: Re-initialize the system

Force system to re-create a new swap file.

sudo dphys-swapfile setup

E.g.

2.4. Step 4: Turn the swap system back on

sudo dphys-swapfile swapon

E.g.

3. Reference

https://pimylifeup.com/raspberry-pi-swap-file/

[btrfs, raid5, lxd] How to setup a RAID5 btrfs pool from 3 disks as LXD storage pool

noreply@blogger.com (Jing(井民全)) — Fri, 16 Dec 2022 19:00:00 +0000

How to setup a RAID5 btrfs pool from 3 disks
井民全, Jing, mqjing@gmall.com

1. Quick1.1. Clean all disclsblk #check all your block storage devices

# umount all partitions
declare -a  arr=("/dev/sda" "/dev/sdb" "/dev/sdc")
for i in "${arr[@]}"; do echo "${i}";sudo umount "${i}"?; sudo wipefs --all --force "${i}"?; sudo wipefs --all --force "${i}"; done


1.2. Install the system# install btrfs
sudo apt-get install btrfs-progs  
sudo mkfs.btrfs -L mybtrfs-pool -d raid5 -m raid5 -f /dev/sda /dev/sdb /dev/sdc

# mount and create subvolume
mkdir /home/pi/data
sudo mount /dev/sdb /home/pi/data 
sudo btrfs subvolume create /home/pi/data/sub1

# create the lxd storage pool as the btrfs volume
lxd init

Interactive
Name of hte new storage pool: mylxd-btrfspool
Name of the existing BTRFS pool or dataset: /home/pi/data/sub1


# fstab
sudo btrfs filesystem show
vi /etc/fstab
UUID=56bd1605-9358-466d-b45c-a988957d7945 /home/pi/data           btrfs   defaults      0  0



2. Procedure2.1. Step 1: Install BTRFSsudo apt-get install btrfs-progs
E.g.

2.2. Step 2: Create BTRFS pool with RAID-5 using three diskssudo mkfs.btrfs -L mybtrfs-pool -d raid5 -m raid5 -f /dev/sda /dev/sdb /dev/sdc

E.g. 


2.3. Step 3: Mount and create btrfs subvolume on itTo mount the RAID-5 pool, you can reference either device when your file system contains multiple devices. Here, the /dev/sdb was chosen.
mkdir /home/pi/data
sudo mount /dev/sdb /home/pi/data 
sudo btrfs subvolume create /home/pi/data/sub1

 
2.4. Step 4 Create a new lxd storage pool
lxd init

Interactive
Name of hte new storage pool: mylxd-btrfspool
Name of the existing BTRFS pool or dataset: /home/pi/data/sub1




2.4.1. Check the storagelxc storage list

2.4.2. Check the storage sizelxc  storage info mylxd-btrfspool

3. Mounting the btrfs filesystem3.1. Get btrfs filesystem UUIDsudo btrfs filesystem show
E.g.

3.2. Setup fstabWe mount the mybtrfs-pool (56bd1605-9358-466d-b45c-a988957d7945) on /home/pi/data.
vi /etc/fstab


UUID=56bd1605-9358-466d-b45c-a988957d7945 /home/pi/data           btrfs   defaults      0  0


4. References
Mount & resize the btrfs system, https://www.thegeekdiary.com/centos-rhel-how-to-create-and-mount-btrfs-file-system-explained-with-examples/
UUID=e7e5c123-fg76-5gxx-a87d-gt5fed9r768e /data           btrfs   defaults      0  0

[linux, swipe] How to wipe all data from block devices

noreply@blogger.com (Jing(井民全)) — Fri, 16 Dec 2022 15:29:00 +0000

Simple form

lsblk #check all your block storage devices # umount all partitions sudo umount /dev/sda?; sudo wipefs --all --force /dev/sda?; sudo wipefs --all --force /dev/sda sudo umount /dev/sdb?; sudo wipefs --all --force /dev/sdb?; sudo wipefs --all --force /dev/sdb sudo umount /dev/sdc?; sudo wipefs --all --force /dev/sdc?; sudo wipefs --all --force /dev/sdc lsblk # check no mount points

Array form

lsblk #check all your block storage devices

# umount all partitions
declare -a arr=("/dev/sda" "/dev/sdb" "/dev/sdc")
for i in "${arr[@]}"; do echo "${i}";sudo umount "${i}"?; sudo wipefs --all --force "${i}"?; sudo wipefs --all --force "${i}"; done

lsblk # check no mount points

# install zfs on your raspberry
sudo apt install raspberrypi-kernel-headers
sudo apt install -y zfs-dkms zfsutils-linux # it takes time: 30 mins
dmesg | grep ZFS # check zfs module
zfs --version # check the zfs version
sudo reboot

[zfs, create pool] How to create a zfs pool from file vdevs

noreply@blogger.com (Jing(井民全)) — Thu, 15 Dec 2022 13:03:00 +0000

How to create a zfs pool from file vdevs

井民全, Jing, mqjing@gmail.com

1. Procedure

1.1. Step 1: Create 3 virtaul devices for test

Here, I use 3 files as the vdev to service the pool.

for i in {1..3}; do truncate -s 2G vdev_$i.img; done

E.g.

1.2. Step 2: Create a raid pool

Assemble to the raidz1 pool.

sudo zpool create myzpool raidz1 /home/jing/test/tank/vdev_1.img /home/jing/test/tank/vdev_2.img /home/jing/test/tank/vdev_3.img

E.g.

1.3. Verify

zpool list

Now, we got a 5.5G zpool.

zpool status myzpool

E.g.

2. Destroy

To destroy a zpool, you can do this.

zpool destory myzpool

3. Reference

https://wiki.archlinux.org/title/ZFS/Virtual_disks

[lxd, zfs, pool, expand] How to expand the lxd default zfs pool capacity

noreply@blogger.com (Jing(井民全)) — Wed, 14 Dec 2022 12:49:00 +0000

How to expand the LXD default zfs pool capacity

井民全, Jing, mqjing@gmail.com

FAQ: The Docker, KVM, Vagrant, VirtualBox, VMWare, Hyper-V

You can follow the instructions to expand the LXD default zfs pool capacity if it is not enough.

Table of contents

1. Quick 1

2. Procedure 2

2.1. Step 1: Stop all running containers 2

2.2. Step 2: Resize the storage device 3

2.2.1. Get the virtual disk location 3

2.2.2. Resize the device 3

2.3. Step 3: Expand the pool size 4

2.4. Step 4: Restart all containers 5

3. References 5

1. Quick

# stop all container

lxd list # get the container name

lxd stop ubuntu-container

# resize the virtual disk

zpool status # get the virtual disk location

sudo ls -la /var/snap/lxd/common/lxd/disks/default.img # check the original size

sudo truncate -s +20G /var/snap/lxd/common/lxd/disks/default.img

sudo ls -la /var/snap/lxd/common/lxd/disks/default.img # check the expended size

# expend the pool

zpool list # check size

sudo zpool status -vg default # get the device id for the pool

sudo zpool online -e default 14277165744894305174 # aware the zfs

# check the pool capacity

zpool list

# restart containers

lxd start ubuntu-container

2. Procedure

2.1. Step 1: Stop all running containers

E.g.

lxd list

lxd stop ubuntu-container

2.2. Step 2: Resize the storage device

Here, this is a virtual disk.

2.2.1. Get the virtual disk location

zpool status

2.2.2. Resize the device

sudo ls -la /var/snap/lxd/common/lxd/disks/default.img # check the original size

sudo truncate -s +20G /var/snap/lxd/common/lxd/disks/default.img

sudo ls -la /var/snap/lxd/common/lxd/disks/default.img # check the expended size

E.g.

Fig.Before expending the file size.

Fig. After expanding the file size.

2.3. Step 3: Expand the pool size

zpool list # check size

sudo zpool status -vg default # get the device id for the pool

sudo zpool online -e default 14277165744894305174 # aware the zfs

Fig. Chck the default pool capacit.

Fig. Get the pool device ID.

Fig. Run the pool expansion.

Verification

zpool list

Fig. Check the capacity after

2.4. Step 4: Restart all containers

lxc start ubuntu-container

3. References

[nextcloud, raspberry, lxd, zfs] How to setup Nextcloud server in Raspberry LXD Container with ZFS system backend

noreply@blogger.com (Jing(井民全)) — Sun, 11 Dec 2022 07:17:00 +0000

How to setup Nextcloud server in Raspberry LXD Container with ZFS system backend

井民全, Jing, mqjing@gmail.com

Slow but worked. Maybe It's time to replace 3B+ to 4.

下面這篇文章把樹梅派, Nextcloud, ZFS, software RAID, LXD container, Proxy network 全部合在一起, 按圖施工照表操課.

系統正常運作, 但是樹梅派已經吃不消了!
不就是簡單的 WebDav protocol 都顯著變慢.

相較之下，之前的設定用 loop device 搭配簡單 btrfs, 效能好很多. 也是我現在正在用的系統設定, 只是沒有使用到 RAID，
如果你 btrfs 版本有興趣, 可以參考下面連結.

https://docs.google.com/document/d/14gJyfRz6ARCvxZnsUP8Ov0NY5XReC4fvB8pHA1UMkZY/edit?usp=sharing

如果你對 zfs +raidz 有興趣，請繼續閱讀。

I like ZFS fiesystem, so, yesterday, I decided set up experiment Nextcloud server in my Raspberry LXD container with ZFS system. Here is the detail procedure, you can find at blew.

However, the performance running on Raspberry Pi is poor. Maybe RAIDZ1 with compress and deduct take too much resource to handle simple WebDev protocol for RasperryPi 3B+?

The performance issue gots me interesting on whether the configuration that btrfs with software RAID has the same issue.

[nextcloud, rasp, lxd, btrfs] How to install Nextcloud server in Raspberry LXD Container with BTRFS storage backend (view), (blog)

Anyway, I'll test it for days.

1. Procedure 2

1.1. ZFS 2

1.1.1. Step 1: Install & setup zfs 2

1.1.2. Step 2: Create a zfs pool & mount the pool 2

1.1.3. Step 3: Create a dataset with compression and dedup function 2

1.1.4. Verification 3

1.2. LXD 3

1.2.1. Step 1: Install & setup lxd 3

1.2.2. Step 2: Set up the exist zfs dataset for container 3

1.2.3. Verification 4

1.2.3.1. zfs list 4

1.2.3.2. zpool list 4

1.3. Ubuntu Intance 5

1.3.1. Install & setup Ubuntu instance 5

1.3.2. Setup network 5

1.3.2.1. Router Mode 5

1.3.2.2. NAT Mode 6

1.3.2.2.1. Setup Network & update 6

1.3.2.2.2. Setup proxy 7

1.4. Nextcloud 8

1.4.1. Step 1: Install & setup 8

1.4.2. Step 2: Setup self-sign (for https) 8

2. Reference 9

3. Further Info 9

1. Procedure

1.1. ZFS

1.1.1. Step 1: Install & setup zfs

lsblk #check all your block storage devices

# umount all partitions

sudo umount /dev/sda?; sudo wipefs --all --force /dev/sda?; sudo wipefs --all --force /dev/sda

sudo umount /dev/sdb?; sudo wipefs --all --force /dev/sdb?; sudo wipefs --all --force /dev/sdb

sudo umount /dev/sdc?; sudo wipefs --all --force /dev/sdc?; sudo wipefs --all --force /dev/sdc

lsblk # check no mount points

# install zfs on your raspberry

sudo apt install raspberrypi-kernel-headers

sudo apt install zfs-dkms zfsutils-linux # it takes time: 30 mins

dmesg | grep ZFS # check zfs module

zfs --version # check the zfs version

1.1.2. Step 2: Create a zfs pool & mount the pool

# create a RAIDZ1 zpool

sudo zpool create myzfspool raidz1 sda sdb sdc -f

lsblk # check the partitions for each devices

# verify

zpool status # check the pool status

# setup properties

sudo zfs set mountpoint=/mnt/myzfspool myzfspool # setup mount point

1.1.3. Step 3: Create a dataset with compression and dedup function

sudo zfs create myzfspool/mydataset # create dataset on the pool

sudo zfs set compression=on myzfspool/mydataset # enable compression

sudo zfs set dedup=on myzfspool/mydataset # enable deduption

# check

df -h #make sure linux can see it

ls /mnt/myzfspool/mydataset

# undo

# sudo zfs destroy -r myzfspool/mydataset

1.1.4. Verification

zpool status # check the pool status

1.2. LXD

1.2.1. Step 1: Install & setup lxd

sudo snap install lxd --channel=latest/stable

sudo usermod -a -G lxd ${USER}

su - ${USER} # apply the new group membership

1.2.2. Step 2: Set up the exist zfs dataset for container

lxd init

1.2.3. Verification

1.2.3.1. zfs list

1.2.3.2. zpool list

1.3. Ubuntu Intance

1.3.1. Install & setup Ubuntu instance

lxc launch images:ubuntu/20.04 ubuntu-container

lxc exec ubuntu-container -- ip a # 10.84.19.200

lxc exec ubuntu-container -- apt-get update

E.g.

Check po

1.3.2. Setup network

1.3.2.1. Router Mode

# Step 1: Create a template profile for your next containers

lxc profile create my-routed

EDITOR=vim lxc profile edit my-routed

config:

user.network-config: |

version: 2

ethernets:

eth0:

addresses:

- 192.168.1.201/32

nameservers:

addresses:

- 8.8.8.8

search: []

routes:

- to: 0.0.0.0/0

via: 169.254.0.1

on-link: true

description: Default LXD profile

devices:

eth0:

ipv4.address: 192.168.1.201

nictype: routed

parent: wlan0

type: nic

used_by:

# Step 2: Setup your container with the profile and run

lxc profile copy my-routed my-routed_192.168.1.201

EDITOR=vim lxc profile edit my-routed_192.168.1.201

lxc launch ubuntu:20.04 c2 --profile default --profile my-routed_192.168.1.201

# Verification

lxc exec c1 -- /bin/bash

ping www.kimo.com

ip a

E.g.

1.3.2.2. NAT Mode

1.3.2.2.1. Setup Network & update

lxc launch images:ubuntu/20.04 ubuntu-container

lxc exec ubuntu-container -- passwd ubuntu

lxc exec ubuntu-container -- apt-get update

lxc exec ubuntu-container -- ip a # 10.84.19.200

E.g.

1.3.2.2.2. Setup proxy

lxc config device add ubuntu-container myport443 proxy listen=tcp:0.0.0.0:443 connect=tcp:127.0.0.1:443

Test

# lxc exec ubuntu-container -- /bin/bash

lxc exec ubuntu-container -- su --login ubuntu

E.g.

Check pool

1.4. Nextcloud

1.4.1. Step 1: Install & setup

lxc exec ubuntu-container -- su --login ubuntu

# reinstall the packages in container for nextcloud

sudo apt install -y snapd

sudo snap install core

sudo reboot # reboot the container

lxc exec ubuntu-container -- su --login ubuntu

# stable version

sudo snap install nextcloud # 24.0.6 --> issue: cannot create subfolder.

# latest version (26.0.0 dev) Nextcloud Hub 3 (26.0.0 dev Build:2022-11-06T23:34:07

# sudo snap remove nextcloud # remove previous nextcloud

# sudo snap install nextcloud --channel=latest/edge

verify

snap changes nextcloud

snap info nextcloud

sudo systemctl | grep nextcloud

1.4.2. Step 2: Setup self-sign (for https)

sudo /snap/bin/nextcloud.enable-https self-signed

E.g.

Check

https://192.168.1.200

2. Reference

https://docs.google.com/spreadsheets/d/1yOlHZjv9NlYr-PwiiCUjcxa9euxTlzKO5YWOHms0Qb4/edit?usp=sharing

3. Further Info

[nextcloud, rasp, lxd, btrfs] How to install Nextcloud server in Raspberry LXD Container with BTRFS storage backend (view), (blog)

[lxc, network, bridge] How to setup IP addresses from your LAN using a bridge to your LXD Containers (DHCP)

noreply@blogger.com (Jing(井民全)) — Sat, 10 Dec 2022 11:57:00 +0000

How to setup IP addresses from your LAN using a bridge to your LXD Containers (DHCP)

井民全, Jing, mqjing@gmail.com

This document shows you how to assign your LXD containers IP address from your host LAN using a bridge. Whole knowledge was from the Simos Xenitellis blog page[2].

Table of contents

1. Quick 1

2. Procedure 2

2.1. Create a new bridge and Interface to the host 3

2.2. Obtain the IP 3

2.2.1. Setup the brigade IP from DHCP 3

2.2.2. Setup a static IP to your bridge 4

2.3. Use the Bridge 5

2.3.1. Create a new profile for bridge networking 5

2.3.2. List profile 6

2.3.3. Launch a container and use the bridge profile 7

3. References 8

1. Quick

# create a new bridge

sudo nmcli con add ifname br0 type bridge con-name br0 # create a bridge br0

# add a interface to ens33 nic

sudo nmcli con add type bridge-slave ifname ens33 master br0 # add a interface to nic

nmcli connection show

# enable the bridge

sudo nmcli con up br0

ip a s br0 # verification

# use the bridge

# Step 1: create a profile, called bridge-profile

lxc profile create bridge-profile

cat <<EOF | lxc profile edit bridge-profile

description: Bridged networking LXD profile

devices:

eth0:

nictype: bridged

parent: br0

type: nic

EOF

#Check

lxc profile list

# Step 2: Launch the container

lxc launch ubuntu:20.04 c3 --profile default --profile bridge-profile

#Verify

lxc exec c3 -- /bin/bash

ping www.google.com

2. Procedure

2.1. Create a new bridge and Interface to the host

Step 1: Create a bridge called br0

sudo nmcli con add ifname br0 type bridge con-name br0 # create a bridge br0

Step 2: Add an interface to the host nic ens33

sudo nmcli con add type bridge-slave ifname ens33 master br0 # add a br0 interface to nic

e. g.

nmcli connection show

2.2. Obtain the IP

2.2.1. Setup the brigade IP from DHCP

Setup a DHCP based IP for your bridge

sudo nmcli con up br0

ip a s br0

If you do not see the ip that assigned to the br0, try to renew the ip

sudo dhclient -r br0; sudo dhclient br0

systemctl restart networking.service

Reboot the VM.

2.2.2. Setup a static IP to your bridge

sudo nmcli connection modify br0 ipv4.addresses '192.168.1.202/24'

sudo nmcli connection modify br0 ipv4.gateway '192.168.1.1'

sudo nmcli connection modify br0 ipv4.dns '8.8.8.8'

sudo nmcli connection modify br0 ipv4.method manual

sudo nmcli con up br0

nmcli con show

e.g.

2.3. Use the Bridge

2.3.1. Create a new profile for bridge networking

lxc profile create bridge-profile

Here, the eth0 is the LXD container's nic name, and setup the traffic to the br0 interface via parent parameter.

cat <<EOF | lxc profile edit bridge-profile

description: Bridged networking LXD profile

devices:

eth0:

nictype: bridged

parent: br0

type: nic

EOF

e.g.

2.3.2. List profile

lxc profile list

2.3.3. Launch a container and use the bridge profile

lxc launch ubuntu:20.04 c3 --profile default --profile bridge-profile

Verify

lxc exec c3 -- /bin/bash

test : connect from Host

ping 192.168.1.105

3. References

[lxc, network, router] How to setup network for your LXD containers between hosts (Static IP)

noreply@blogger.com (Jing(井民全)) — Sat, 10 Dec 2022 08:19:00 +0000

How to setup network for your LXD containers between hosts (Static IP)

井民全, Jing, mqjing@gmail.com

FAQ: Docker, LXD, KVM, Vagrant, VirtualBox, VMWare, Hyper-V

如果你想讓你的 LXD container 掛上 host 的 subnetwork, 並且能夠讓上面的各式 host 自由存取containers 上

執行的為服務, 就像在 VirtualBox 或 VMWare 設定你的 VM 網路使用 bridge 模式一樣. 請服用這篇文章.

This document shows you how to get your containers to be accessed between hosts. You can set your containers on the same subnet with host and communicate each other just like as an other machine on the net.

The key idea is to create a LXD profile in which we setup the router and redirect the traffic to the parent of host nic, for my case, this is the ens33. The knowledge used in this document were fully follow to the doc[1], if you are interested in detail, check it to get more.

Table of contents

1. Quick 2

2. Procedure 3

2.1. Install LXD 3

2.2. Get your host network informaiton 4

2.3. Create a templated lxd profile 5

2.4. Copy and Use the profile for your containers 5

3. Verification 6

3.1. Test 1: Access www.kimo.com 6

3.2. Test 2: Access from Host 7

4. Appendix 8

4.1. Create another container with the router 8

4.2. Verification 9

4.2.1. Test 1: Test traffic to Internet 9

4.2.2. Test 2: Traffic from Host 9

5. References 9

1. Quick

# Step 1: Install LXD and initial it

sudo snap install lxd --channel=latest/stable

sudo apt-get install bridge-util # install bridge util

sudo usermod -a -G lxd ${USER}

su - ${USER} # apply the new group membership

lxd init

# Step 2: Get your host nic name (here is ens33) and the host subnet

ip a # Get to known the sub-net and the host nic name

lxc network list

# Step 3: Create a template profile for your next containers

lxc profile create my-routed

lxc profile edit my-routed

config:

user.network-config: |

version: 2

ethernets:

eth0:

addresses:

- 192.168.1.200/32

nameservers:

addresses:

- 8.8.8.8

search: []

routes:

- to: 0.0.0.0/0

via: 169.254.0.1

on-link: true

description: Default LXD profile

devices:

eth0:

ipv4.address: 192.168.1.200

nictype: routed

parent: ens33

type: nic

used_by:

# Step 4: Setup your container with the profile and run

lxc profile copy my-routed my-routed_192.168.1.200

EDITOR=vim lxc profile edit my-routed_192.168.1.200

lxc launch ubuntu:20.04 c1 --profile default --profile my-routed_192.168.1.200

# Verification

lxc exec c1 -- /bin/bash

ping www.kimo.com

ip a

2. Procedure

2.1. Install LXD

sudo snap install lxd --channel=latest/stable

sudo usermod -a -G lxd ${USER}

su - ${USER} # apply the new group membership

lxd init

==> Size in GiB of the new loop device (1GiB minimum) [default=23GiB]: 115GiB

# if you want to remove the container

# lxc delete ubuntu-container --force

# install bridge util

sudo apt-get install bridge-util

# list network

lxc network list

# delete default router

# sudo apt-get install bridge-utils

# sudo apt-get install

# sudo ifconfig lxdbr0 down

# sudo brctl delbr lxdbr0

# lxc network delete lxdbr0

# remove lxd

sudo snap remove lxd --purge

2.2. Get your host network informaiton

The nic name and the subnet are the points that will be used for the traffic parent the ip address for your container. Here, the parent nic will be ens33 and the subnet will be 192.168.1.x.

e.g.

2.3. Create a templated lxd profile

lxc profile create my-routed

lxc profile edit my-routed

Fill the following setup. Nerermind, the statement "169.254.0.1". It will get assigned by lxd for the routed inteface.

config:

user.network-config: |

version: 2

ethernets:

eth0:

addresses:

- 192.168.1.200/32

nameservers:

addresses:

- 8.8.8.8

search: []

routes:

- to: 0.0.0.0/0

via: 169.254.0.1

on-link: true

description: Default LXD profile

devices:

eth0:

ipv4.address: 192.168.1.200

nictype: routed

parent: ens33

type: nic

used_by:

2.4. Copy and Use the profile for your containers

lxc profile copy my-routed my-routed_192.168.1.200

EDITOR=vim lxc profile edit my-routed_192.168.1.200

lxc launch ubuntu:20.04 c1 --profile default --profile my-routed_192.168.1.200

lxc profile copy my-routed my-routed_192.168.1.201

EDITOR=vim lxc profile edit my-routed_192.168.1.201

check

lxc profile list

e.g.

Check the container router.

The traffic will via 169.254.0.1.

3. Verification

3.1. Test 1: Access www.kimo.com

lxc exec c1 -- /bin/bash

ping www.kimo.com

ip a

3.2. Test 2: Access from Host

ping 192.168.1.200

4. Appendix

4.1. Create another container with the router

lxc launch ubuntu:20.04 c2 --profile default --profile my-routed_192.168.1.201

lxc list

e.g.

4.2. Verification

4.2.1. Test 1: Test traffic to Internet

4.2.2. Test 2: Traffic from Host

5. References

[nextcloud, rasp, lxd] Install NextCloud in LXD container on Raspberry Pi

noreply@blogger.com (Jing(井民全)) — Sat, 05 Nov 2022 18:26:00 +0000

Install NextCloud in LXD container on Raspberry Pi

井民全, Jing, mqjing@gmail.com

This document shows you how to install the NextCloud in a LXD container on Raspberry Pi.

1. Quick

# Raspberry Pi setup

# Setup network

sudo arp-scan -I ens33 --localnet # find your raspberry pi

sudo apt-get -y update --allow-releaseinfo-change

# Setup static IP

sudo vi /etc/dhcpcd.conf

-----------------------------

interface wlan0

static routers=192.168.1.1

static domain_name_servers=8.8.8.8

static ip_address=192.168.1.198/24

-----------------------------

sudo ip link set wlan0 down && sudo ip link set wlan0 up

# install snap

sudo apt install -y snapd

sudo snap install core

sudo reboot

# LXD

# install LXD

sudo snap install lxd --channel=latest/stable

sudo /snap/bin/lxd init

==> Size in GiB of the new loop device (1GiB minimum) [default=23GiB]: 115GiB

sudo usermod -a -G lxd ${USER}

su - ${USER} # apply the new group membership

# create LXD container for nextcloud

lxc image list images: architecture=armhf os=Ubuntu # check ubuntu images for Raspberry Pi

lxc launch images:ubuntu/20.04 ubuntu-container

lxc exec ubuntu-container -- passwd ubuntu # setup the default passwd for normal user

lxc exec ubuntu-container -- apt-get update

# add proxy device

lxc config device add ubuntu-container myport443 proxy listen=tcp:0.0.0.0:443 connect=tcp:127.0.0.1:443

# Nextcloud

# install packages

lxc exec ubuntu-container -- su --login ubuntu # enter the container

sudo apt install -y snapd

sudo snap install core

sudo reboot

# install nextcloud

lxc exec ubuntu-container -- su --login ubuntu # enter the container

sudo snap install nextcloud

verify

snap changes nextcloud

snap info nextcloud

sudo systemctl | grep nextcloud

# enable https

sudo /snap/bin/nextcloud.enable-https self-signed

# verify

https://192.168.1.198/

2. System Requirement

https://docs.nextcloud.com/server/latest/admin_manual/installation/system_requirements.html#server

3. Detail Procedure

3.1. Raspberry Pi

3.1.1. Step 1:Setup network

sudo arp-scan -I ens33 --localnet # find your raspberry pi

sudo apt-get -y update --allow-releaseinfo-change

# Setup static IP

sudo vi /etc/dhcpcd.conf

-----------------------------

interface wlan0

static routers=192.168.1.1

static domain_name_servers=8.8.8.8

static ip_address=192.168.1.198/24

-----------------------------

sudo ip link set wlan0 down && sudo ip link set wlan0 up

3.1.2. Step 2: Install packages

# install snap

sudo apt install -y snapd

sudo snap install core

sudo reboot

3.2. LXD

3.2.1. Setp 1: Install & Initialized

sudo snap install lxd --channel=latest/stable

sudo /snap/bin/lxd init

==> Size in GiB of the new loop device (1GiB minimum) [default=23GiB]: 115GiB

sudo usermod -a -G lxd ${USER}

su - ${USER} # apply the new group membership

# if you want to remove the container

# lxc delete ubuntu-container --force

3.2.2. Step 2: Create a container

lxc image list images: architecture=armhf os=Ubuntu # check ubuntu images for Raspberry Pi

lxc launch images:ubuntu/20.04 ubuntu-container

lxc exec ubuntu-container -- passwd ubuntu # setup the default passwd for normal user

lxc exec ubuntu-container -- apt-get update

E.g.

3.2.3. Step 3: Add proxy device to container

Redirect https traffic to the container that nextcloud service located.

lxc config device add ubuntu-container myport443 proxy listen=tcp:0.0.0.0:443 connect=tcp:127.0.0.1:443

E.g.

3.3. Install NextCloud in LXD

3.3.1. Step 1: Intall packages

lxc exec ubuntu-container -- su --login ubuntu # enter the container

sudo apt install -y snapd

sudo snap install core

sudo reboot

3.3.2. Step 2: Install NextCloud

lxc exec ubuntu-container -- su --login ubuntu # enter the container

sudo snap install nextcloud

verify

snap changes nextcloud

snap info nextcloud

sudo systemctl | grep nextcloud

# Other Options

# latest version (26.0.0 dev)

# sudo snap remove nextcloud # remove previous nextcloud

# sudo snap install nextcloud --channel=latest/edge

E.g.

3.3.3. Step 3: Enable https

sudo /snap/bin/nextcloud.enable-https self-signed

E.g.

4. Result

4.1. Browser

https://192.168.1.198/

4.2. iPhone

Windows NextCloud Client

5. Appendix

5.1. Check Temperture

vcgencmd measure_temp

E.g

[lxc, proxy] How to setup a proxy device to your container for your service

noreply@blogger.com (Jing(井民全)) — Sat, 05 Nov 2022 04:14:00 +0000

How to setup a proxy device to your container for your service

井民全, Jing, mqjing@gmail.com

This document shows you how to create a web service (here I use ngix) behind in a LXC container and setup a proxy device to redirect all TCP:8080 traffics to the ngix service on the container TCP localhost:80.

The following command means: The proxy device listen on any network interfaces on host to TCP:8080, and map to the container on TCP:80

lxc config device add ubuntu-container myport80 proxy listen=tcp:0.0.0.0:8080 connect=tcp:127.0.0.1:80

Quick

# setup lxc container

sudo snap install lxd --channel=5.0/stable

sudo lxd init

sudo lxc launch images:ubuntu/20.04 ubuntu-container # launch a container

lxc exec ubuntu-container -- passwd ubuntu # change the user ubuntu passwd

# verify

sudo lxd list

# Install a test service

lxc exec ubuntu-container -- sudo apt install -y nginx

# add proxy device to the container

lxc config device add ubuntu-container myport80 proxy listen=tcp:0.0.0.0:8080 connect=tcp:127.0.0.1:80

# verify

[on host]

curl 192.168.1.100:8080

[on the other host]

gio open http://192.168.1.100:8080

Procedure

Step 1: Setup a LXC Container

sudo snap install lxd --channel=5.0/stable

sudo lxd init

sudo lxc launch images:ubuntu/20.04 ubuntu-container # launch a container

lxc exec ubuntu-container -- passwd ubuntu # change the user ubuntu passwd

# verify

sudo lxd list

Step 2: Install ngix on the container

lxc exec ubuntu-container -- sudo apt install -y nginx

Verify

curl 10.199.76.115

E.g.

Step 3: Add a proxy device

Create a proxy device named as myport80 to the container, ubuntu-container. The proxy listen on the host to all network interfaces on TCP:8080 and then redirect the traffic to the container to the port 80 on 127.0.0.1 (localhost), which your service listen on.

lxc config device add ubuntu-container myport80 proxy listen=tcp:0.0.0.0:8080 connect=tcp:127.0.0.1:80

E.g.

Verification

Test 1: Host test:

Try to send traffic to the host ip:8080 port, see the service on container responses.

[on host]

curl 192.168.1.100:8080

Test 2: Another machine test

Try to send traffic from another machine to the host ip:8080, see the service on container responses.

[on the other host]

gio open http://192.168.1.100:8080

Reference

https://blog.simos.info/how-to-use-the-lxd-proxy-device-to-map-ports-between-the-host-and-the-containers/

[lxc, ssh] How to connect the lxd container using ssh?

noreply@blogger.com (Jing(井民全)) — Sat, 05 Nov 2022 01:26:00 +0000

How to connect the lxd container using ssh

井民全, Jing, mqjing@gmail.com

Quick

lxc list # get your container list

lxc exec ubuntu-container -- apt-get install openssh-server

# verify

ssh ubuntu@10.199.76.115

lxc exec ubuntu-container -- passwd ubuntu # in case you forgot the passswd

Procedure

Step 1: Get the container list to know your container's name, ip

lxc list # get your container list

E.g.

Step 2: Install openssh server

lxc exec ubuntu-container -- apt-get install openssh-server

E.g.

Verification

ssh ubuntu@10.199.76.115

Note: If you forget the container user ubuntu's passwd, follow the instruction to change it.

lxc exec ubuntu-container -- passwd ubuntu

[lxc, user] How to setup the password for the LXC Ubuntu 5.0/stable container default user ubuntu

noreply@blogger.com (Jing(井民全)) — Sat, 05 Nov 2022 00:54:00 +0000

How to setup the password for the LXC Ubuntu 5.0/stable container default user ubuntu

井民全, Jing, mqjing@gmail.com

Command

lxc exec YourContainer-Name -- passwd ubuntu

e.g.

lxc exec ubuntu-container -- passwd ubuntu

Result

[gsheet, formula, value] How to change color of a range of cells if the value of a cell is greater than the above one

noreply@blogger.com (Jing(井民全)) — Fri, 21 Oct 2022 12:52:00 +0000

How to change color of a range of cells if the value of a cell is greater than the above one

井民全, Jing, mqjing@gmail.com

Method 1: Just point to the address of the above cell without dollar sign, $.

(demo)

Step 1: Formula

If your cell's address is C3, set the formula as

Greater than =C2

Step 2: Apply the formula range to whole column.

Method 2: Indirect + Address + Row + Col Commands

(demo)

Step 1: Peak any cell and input the

= INDIRECT(ADDRESS(ROW(),COLUMN()))> INDIRECT(ADDRESS(ROW()-1,COLUMN()))

Step 2: Apply to whole range of cells in column

[gsheet, formula, address] How to get the address of a cell

noreply@blogger.com (Jing(井民全)) — Thu, 20 Oct 2022 01:59:00 +0000

[gsheet, formula, address] How to get the address of a cell

Command

=ADDRESS(ROW(), COLUMN())

E.g.

Example

(view)

[linux, ip, nic] How to reset your network interface controller (nic)

noreply@blogger.com (Jing(井民全)) — Sat, 15 Oct 2022 06:18:00 +0000

ifconfig # check your nic name

sudo ip link set ens33 down && sudo ip link set ens33 up

[rapi, zfs] How to install ZFS on Raspberry Pi

noreply@blogger.com (Jing(井民全)) — Sun, 09 Oct 2022 19:32:00 +0000

How to install ZFS on Raspberry Pi

井民全, Jing, mqjing@gmail.com

1. Quick

lsblk #check all your block storage devices

# umount all partitions

sudo umount /dev/sda?; sudo wipefs --all --force /dev/sda?; sudo wipefs --all --force /dev/sda

sudo umount /dev/sdb?; sudo wipefs --all --force /dev/sdb?; sudo wipefs --all --force /dev/sdb

sudo umount /dev/sdc?; sudo wipefs --all --force /dev/sdc?; sudo wipefs --all --force /dev/sdc

sudo umount /dev/sdd?; sudo wipefs --all --force /dev/sdd?; sudo wipefs --all --force /dev/sdd

lsblk # check no mount points

# install zfs on your raspberry

sudo apt install raspberrypi-kernel-headers

sudo apt install zfs-dkms zfsutils-linux # it takes time

dmesg | grep ZFS # check zfs module

zfs --version # check the zfs version

# create a RAIDZ1 zpool

sudo zpool create myzfspool raidz1 sda sdb sdc sdd -f

lsblk # check the partitions for each devices

zpool status # check the pool status

# adding compression to the pool

sudo zfs set compression=on myzfspool

# setup mount point

sudo zfs set mountpoint=/mnt/myzfspool myzfspool

df -h #make sure linux can see it

# create dataset on the pool

sudo zfs create myzfspool/mydataset

2. Prepare the disks

At least 3 disks for setup ZFS. Here, I prepared 4 devices.

(Edit)

2.1. Check the disks

Command

lsblk

e.g.

2.2. Unmount and erase filesystem

sudo umount /dev/sda?; sudo wipefs --all --force /dev/sda?; sudo wipefs --all --force /dev/sda

sudo umount /dev/sdb?; sudo wipefs --all --force /dev/sdb?; sudo wipefs --all --force /dev/sdb

sudo umount /dev/sdc?; sudo wipefs --all --force /dev/sdc?; sudo wipefs --all --force /dev/sdc

sudo umount /dev/sdd?; sudo wipefs --all --force /dev/sdd?; sudo wipefs --all --force /dev/sdd

Verification

Check, no disks were mounted.

3. Install ZFS on your Raspberry PI

3.1. Procedure

Command

sudo apt install raspberrypi-kernel-headers

sudo apt install zfs-dkms zfsutils-linux # it takes time. (30 mins)

3.2. Verification

Test module

dmesg | grep ZFS

Check zfs version

zfs --version

4. Create a RAIDZ1 zpool

4.1. Procedure

Command

sudo zpool create myzfspool raidz1 sda sdb sdc sdd -f

Result

Check the partition. Zfs makes two partitions foreach on your devices, one is the ZFS pool which takes almost the entire space, in our case sda1, sdb1, sdc1 and sdd1. The second partition is for ZFS maintenance.

4.2. Check status

zpool status

5. Adding compression to the pool

sudo zfs set compression=on myzfspool

6. Setup mount point

6.1. Procedure

Comand

sudo zfs set mountpoint=/mnt/myzfspool myzfspool

e.g.

The dataset will be mounted on /mnt/myzfspool/mydataset.

6.2. Make sure Linux can see the pool.

df -h

7. Create dataset

Command

sudo zfs create myzfspool/mydataset

e.g.

8. Destroy pool

Command

sudo zpool destroy myzfspool

9. References

[rapi, ip] How to setup a static ip for the Raspberry Pi

noreply@blogger.com (Jing(井民全)) — Sat, 08 Oct 2022 17:24:00 +0000

How to setup a static ip for the Raspberry Pi

井民全, jing, mqjing@gmail.com

1. Quick

vi /etc/dhcpcd.conf

interface wlan0

static routers=192.168.1.1

static domain_name_servers=8.8.8.8

static ip_address=192.168.1.200/24

ifdown wlan0 && ifup wlan0

2. Procedure

Step 1: Check your router ip and the device

Command

ip r

Step 2: Setup static ip

If you want to setup a static ip as 192.168.1.200, edit the dhcpcd.conf.

File: /etc/dhcpcd.conf

interface wlan0

static routers=192.168.1.1

static domain_name_servers=8.8.8.8

static ip_address=192.168.1.200/24

ifdown wlan0 && ifup wlan0

3. Verification

Scan the network.

Command

sudo arp-scan -I ens33 --localnet

Result

You can find the raspberry Pi ip was assigned to 192.168.1.200. Where 192.168.1.108 is my another Pi.

4. Reference

https://www.tomshardware.com/how-to/static-ip-raspberry-pi

5. Further Information

https://www.makeuseof.com/raspberry-pi-set-static-ip/