themacteppieces

How to obtain an embedded PDF file

themactep@gmail.com (Paul Philippov) — Sun, 26 Nov 2023 11:26:25 GMT

If a website allows you to view a PDF file in the browser, but does not allow you to save the file locally, you can still have a local copy of the file using this simple hack.

Open Web Developer Tools in your web browser, switch to the Network tab, check the Disable Cache box and make sure that All types of content is enabled, then refresh the PDF document preview page. You will see all the requests to each resource used to build this page, supposedly including the PDF document.

Look under the largest resources (clicking the Size header to sort the list by size helps) and see if you have a resource with a stream of characters sent as a payload. This is usually returned by a request to a server script with a document ID sent as a request parameter. If you find something that looks like a possibly encoded document, copy the content to the clipboard and paste it into a text file. Save the file and decode it with base64.

1Gbit/s Ethernel port negotiates to 100Mb/s

themactep@gmail.com (Paul Philippov) — Wed, 05 Jul 2023 19:58:58 GMT

My Supermicro server has been acting weird lately. I’ve done a fresh Debian 12 installation on it and the network was barely usable.

lspci confirmed that the server has two Gigabit Ethernet ports

$ lspci | grep Ethernet
02:00.0 Ethernet controller: Intel Corporation 82574L Gigabit Network Connection
03:00.0 Ethernet controller: Intel Corporation 82574L Gigabit Network Connection

lshw showed that both were connected at 100 megabits per second

$ sudo lshw -class network | grep speed
       configuration: autonegotiation=on broadcast=yes driver=e1000e driverversion=6.1.0-9-amd64 duplex=full firmware=1.9-0 ip=192.168.1.15 latency=0 link=yes multicast=yes port=twisted pair speed=100Mb/s
       configuration: autonegotiation=on broadcast=yes driver=e1000e driverversion=6.1.0-9-amd64 duplex=full firmware=1.9-0 ip=192.168.1.16 latency=0 link=yes multicast=yes port=twisted pair speed=100Mb/s

So I forced both interfaces into 1Gbit mode disabling negotiation

$ sudo ethtool -s enp2s0 speed 1000 duplex full autoneg off
$ sudo ethtool -s enp3s0 speed 1000 duplex full autoneg off

This sped things up a lot

$ sudo lshw -class network | grep speed
       configuration: autonegotiation=on broadcast=yes driver=e1000e driverversion=6.1.0-9-amd64 duplex=full firmware=1.9-0 ip=192.168.1.15 latency=0 link=yes multicast=yes port=twisted pair speed=1Gbit/s
       configuration: autonegotiation=on broadcast=yes driver=e1000e driverversion=6.1.0-9-amd64 duplex=full firmware=1.9-0 ip=192.168.1.16 latency=0 link=yes multicast=yes port=twisted pair speed=1Gbit/s

Flashing IP Camera with full OpenIPC binary firmware from SD card

themactep@gmail.com (Paul Philippov) — Mon, 07 Nov 2022 05:56:40 GMT

This method is useful when your camera already has OpenIPC bootloader installed, and especially when the camera does not have access to internet from bootloader console (e.g. it’s a WiFi-only camera) but you have access to the bootloader console via UART connection.

Go to https://openipc.org/supported-hardware/ and download a monolitic binary file with full version of OpenIPC firmware for your camera.

Create a sole 64MB partition formatted into FAT32 on the SD card.
Save the downloaded file onto that partitions.
Unmount the card and install it into the camera.
Open the console and run these commands:

For Lite version:

mw.b ${baseaddr} 0xff 0x800000; fatload mmc 0:1 ${baseaddr} openipc-${soc}-lite-8mb.bin
sf probe 0; sf erase 0x0 0x800000; sf write ${baseaddr} 0x0 ${filesize}

For Ultimate version:

mw.b ${baseaddr} 0xff 0x1000000; fatload mmc 0:1 ${baseaddr} openipc-${soc}-ultimate-16mb.bin
sf probe 0; sf erase 0x0 0x1000000; sf write ${baseaddr} 0x0 ${filesize}

Type reset to restart the camera and start pressing Enter to interrupt normal booting sequence and boot into the newly flashed bootloader.

Now, you need to set up bootloader environment to match your flash memory size.

For 8MB NOR flash:

run setnor8m

For 16MB NOR flash:

run setnor16m

Flashing XM530 R80X30-PQ to OpenIPC and back

themactep@gmail.com (Paul Philippov) — Wed, 07 Sep 2022 14:28:31 GMT

Flashing OpenIPC

Download OpenIPC firmware and extract it into root of your TFTP server.

cd /srv/tftp
wget https://github.com/OpenIPC/firmware/releases/download/latest/openipc.xm530-br.tgz
tar xvf openipc.xm530-br.tgz

Get access to bootloader console. Connect UART adapter to camera board, power up camera, and hit Ctrl-C to stop loading.

Set TFTP server IP address.

setenv serverip 192.168.1.254
saveenv

Set boot parameters.

setenv bootargs 'mem=36M console=ttyAMA0,115200 root=/dev/mtdblock3 rootfstype=squashfs init=/init mtdparts=xm_sfc:256k(boot),64k(env),2048k(kernel),5120k(rootfs),-(rootfs_data)'
setenv bootcmd 'sf probe 0; sf read 0x80007fc0 0x50000 0x200000; bootm 0x80007fc0'
setenv soc xm530
saveenv

Flash OpenIPC kernel and rootfs.

mw.b 0x81000000 ff 800000
tftp 0x81000000 uImage.${soc}
sf probe 0
sf erase 0x50000 0x200000
sf write 0x81000000 0x50000 ${filesize}

mw.b 0x81000000 ff 800000
tftp 0x81000000 rootfs.squashfs.${soc}
sf probe 0
sf erase 0x250000 0x500000
sf write 0x81000000 0x250000 ${filesize}

Reboot the camera.

reset

Rolling back to original firmware

Download vendor’s firmware binary file for your camera and extract it into root of your TFT server.

cd /srv/tftp
wget https://obs-xm-customer.obs.cn-east-2.myhuaweicloud.com/00030751.1IPC_XM530_R80X30-PQ_WIFIXM713G_TB.713g.Nat.dss.OnvifS_V5.00.R02.zip
unzip 00030751.1IPC_XM530_R80X30-PQ_WIFIXM713G_TB.713g.Nat.dss.OnvifS_V5.00.R02.zip

Boot into bootloader console and run printenv. Make sure that your environment includes following lines:

da=mw.b 0x81000000 ff 800000;tftp 0x81000000 u-boot.bin.img;sf probe 0;flwrite
dc=mw.b 0x81000000 ff 800000;tftp 0x81000000 custom-x.cramfs.img;sf probe 0;flwrite
dd=mw.b 0x81000000 ff 800000;tftp 0x81000000 mtd-x.jffs2.img;sf probe 0;flwrite
dr=mw.b 0x81000000 ff 800000;tftp 0x81000000 romfs-x.cramfs.img;sf probe 0;flwrite
du=mw.b 0x81000000 ff 800000;tftp 0x81000000 user-x.cramfs.img;sf probe 0;flwrite
dw=mw.b 0x81000000 ff 800000;tftp 0x81000000 web-x.cramfs.img;sf probe 0;flwrite
tk=mw.b 0x81000000 ff 800000;tftp 0x81000000 uImage; bootm 0x81000000
ua=mw.b 0x81000000 ff 800000;tftp 0x81000000 upall_verify.img;sf probe 0;flwrite
up=mw.b 0x81000000 ff 800000;tftp 0x81000000 update.img;sf probe 0;flwrite

Restore orginal boot parameters.

setenv bootargs 'mem=36M console=ttyAMA0,115200 root=/dev/mtdblock2 rootfstype=cramfs mtdparts=xm_sfc:256K(boot),1536K(kernel),1280K(romfs),4544K(user),256K(custom),320K(mtd)'
setenv bootcmd 'sf probe 0; sf read 0x80007fc0 0x40000 0x180000; bootm 0x80007fc0'
saveenv

Flash files from TFTP server.

run da; run dc; run dd; run dr; run du; run dw

There is no shortcut for flashing kernel, so you need to flash it with a full command.

mw.b 0x81000000 ff 800000
tftp 0x81000000 uImage.img
sf probe 0
flwrite

Reboot the camera.

reset

Some thoughts on IP cameras firmware.

themactep@gmail.com (Paul Philippov) — Mon, 05 Sep 2022 08:23:41 GMT

Getting and setting configuration via Onvif.
Independent credentials for different services. Common credentials for every service is an isolated case of independent credentials where every set matches the others. Let that sink in.
Video streams with minimum delay (UDP).
Snapshot conveyor is independent and rock solid. No empty buffers. No stuck requests.
EXIF for JPEG images.

How to link to your Patreon page without using JavaScript

themactep@gmail.com (Paul Philippov) — Tue, 23 Mar 2021 02:12:43 GMT

HTML

<a href="https://www.patreon.com/properrussian" class="patreon">Become a Patron!</a>

CSS

a.patreon {
    background: #ff424d;
    border: solid 1px #ff424d;
    border-radius: 0.5rem;
    box-shadow: 2px 2px 3px #888888;
    color: #ffffff !important;
    display: block;
    font-weight: 400;
    font-size: 1rem;
    margin: 1rem 0;
    padding: 0.75rem 1.25rem;
    text-decoration: none !important;
    width: 14rem;
}
a.patreon::before {
    border-left: solid 4px #ffffff;
    color: #ffffff !important;
    content: "⬤";
    font-size: 1.2rem;
    font-weight: 700;
    line-height: 1;
    padding: 0 0.5rem 0.5rem 0.1rem;
}

Live Preview

Check out https://properrussian.com/.

Why JavaScript email obfuscation does not work

themactep@gmail.com (Paul Philippov) — Wed, 10 Mar 2021 19:53:34 GMT

#!/usr/bin/env python

from requests_html import HTMLSession
import re

session = HTMLSession()
resp = session.get(put_your_url_here)
resp.html.render()
email = re.findall(r'[\w\.-]+@[\w\.-]+', resp.html.html)
print(email)

Here’s a page with different obfuscation techniques. Feel free to use it for scrapping.

How to install Syncthing on a Debian server to serve multiple users

themactep@gmail.com (Paul Philippov) — Wed, 03 Mar 2021 16:44:09 GMT

First, Install Syncthing:

sudo -s
curl -s https://syncthing.net/release-key.txt | apt-key add -
echo "deb https://apt.syncthing.net/ syncthing stable" | tee /etc/apt/sources.list.d/syncthing.list
printf "Package: *\nPin: origin apt.syncthing.net\nPin-Priority: 990\n" | tee /etc/apt/preferences.d/syncthing
apt update
apt install syncthing
exit

By default, Syncthing runs on port 8384. In order to serve multiple users, each user has to have a dedicated instance of Syncthing listening on a unique port. I found it easier to assign each user a port in 8390+ range re-using the last digit of user’s UID. The following sequense of commands would make it easier for you to automate the process of creating new istances. Note, that you might need to tweak it if your setup uses multiple IP addresses, or your users’ UIDs are not in the 1000-1009 range.

sudo -s -u <username>
cd
user=$(whoami)
sudo systemctl start syncthing@$user.service
sudo systemctl stop syncthing@$user.service
ip_address=$(hostname -I | xargs)
port=$(echo $UID | sed s/^100/839/)
sed -i "s/127\.0\.0\.1:8384/$ip_address:$port/" ~/.config/syncthing/config.xml
sudo systemctl enable syncthing@$user.service
sudo systemctl start syncthing@$user.service
exit

Open web panel at http://$ip_address:$port and protect future access with login and password.

Also, you might want to mount your server sync directory on your desktop.

Self-hosting is easy

themactep@gmail.com (Paul Philippov) — Mon, 15 Feb 2021 22:31:04 GMT

With many geeks trying to self-host their own servers lately, I see some confusion about how all that stuff work, and what should you do to host a web blog and a gemini capsule on the same server on the same domain name.

Here’s my humble attempt to explain, as simple as possible, what’s going on when someone makes a request to an internet resource.

Each internet resource (web server, mail server, file server, chat server, etc) has a unique reference called URL. That’s how we locate different resources on the net. URL includes, among others, protocol, host name, and port of the resource.

Client machine queries its DNS server where the target domain name is located, in internet terms. DNS server returns an IP address which is associated with the requested domain name.

Client machine builds path to the IP address through multiple nodes using network routing. As the desired IP address is reached, client software tries to establish a connection on a specific port of the target machine.

Every internet protocol has a designated port, the one used by default if no particular port specified in the URL. Say, designated port for FTP is 21, for SSH – 22, for SMTP – 25, for HTTP – 80, for HTTPS – 443, and so on. Gemini protocol has a designated port 1965 (by the year of the first manned Gemini mission flight), thus a request to URI gemini://themactep.com will try to open a connection on port 1965 of a server with the IP address 192.186.103.98.

But what if there are multiple servers with the same protocol hosted on the same machine? Or if the server is set up to listen on a non-standard port? How do we reach it then? In that case you need to add the specific port to your URL, like this http://themactep.com:9999 which is equal to http://192.186.103.98:9999 in this particular case.

Why the particular case? Because some servers can host not only multiple services for the same domain name, but multiple domain names on the same internet server. E.g. web servers like Apache and NGINX can determine what exact virtual host was contacted on the same IP address and the same port by the domain name used in the request. That makes requests to two different URLs, http://themactep.com/ and http://paulphilippov.com/, connecting the same port 80 on the same IP address 192.186.103.98, but resulting in different contents.

I hope this short article makes all that internet sorcery a little bit less magical. If you still have questions, don’t hesitate to ask in comments.

Shorter is better and smaller, too

themactep@gmail.com (Paul Philippov) — Wed, 23 Mar 2016 08:05:25 GMT

Below are two variants of the same program for Arduino micro-controller.

First program, the longer one, was found among examples somewhere on the Net.

#define RELAY_ON  0
#define RELAY_OFF 1

#define Relay_1  3
#define Relay_2  2
#define Relay_3  20
#define Relay_4  19
#define Relay_5  18
#define Relay_6  11
#define Relay_7  6
#define Relay_8  5

int waittime;

void setup()
{
  waittime = 1000;
  digitalWrite(Relay_1, RELAY_OFF);
  digitalWrite(Relay_2, RELAY_OFF);
  digitalWrite(Relay_3, RELAY_OFF);
  digitalWrite(Relay_4, RELAY_OFF);
  digitalWrite(Relay_5, RELAY_OFF);
  digitalWrite(Relay_6, RELAY_OFF);
  digitalWrite(Relay_7, RELAY_OFF);
  digitalWrite(Relay_8, RELAY_OFF);

  pinMode(Relay_1, OUTPUT);
  pinMode(Relay_2, OUTPUT);
  pinMode(Relay_3, OUTPUT);
  pinMode(Relay_4, OUTPUT);
  pinMode(Relay_5, OUTPUT);
  pinMode(Relay_6, OUTPUT);
  pinMode(Relay_7, OUTPUT);
  pinMode(Relay_8, OUTPUT);

  delay(4000);
}

void loop()
{
  digitalWrite(Relay_1, RELAY_ON);
  delay(waittime);
  digitalWrite(Relay_2, RELAY_ON);
  delay(waittime);
  digitalWrite(Relay_3, RELAY_ON);
  delay(waittime);
  digitalWrite(Relay_4, RELAY_ON);
  delay(waittime);
  digitalWrite(Relay_5, RELAY_ON);
  delay(waittime);
  digitalWrite(Relay_6, RELAY_ON);
  delay(waittime);
  digitalWrite(Relay_7, RELAY_ON);
  delay(waittime);
  digitalWrite(Relay_8, RELAY_ON);
  delay(waittime);

  digitalWrite(Relay_1, RELAY_OFF);
  delay(waittime);
  digitalWrite(Relay_2, RELAY_OFF);
  delay(waittime);
  digitalWrite(Relay_3, RELAY_OFF);
  delay(waittime);
  digitalWrite(Relay_4, RELAY_OFF);
  delay(waittime);
  digitalWrite(Relay_5, RELAY_OFF);
  delay(waittime);
  digitalWrite(Relay_6, RELAY_OFF);
  delay(waittime);
  digitalWrite(Relay_7, RELAY_OFF);
  delay(waittime);
  digitalWrite(Relay_8, RELAY_OFF);
  delay(waittime);
}

Sketch uses 5,336 bytes (18%) of program storage space. Maximum is 28,672 bytes.
Global variables use 153 bytes (5%) of dynamic memory, leaving 2,407 bytes for local variables. Maximum is 2,560 bytes.

The other, shorter one, is my reworked variant of the first program. In my opinion it is easier to read and is more flexible, happens that I need to make any changes.

#define RELAY_ON  LOW
#define RELAY_OFF HIGH
#define WAIT_TIME 1000

uint8_t pins[] = {3, 2, 20, 19, 18, 11, 6, 5};

void setup() {
  for (uint8_t i = 0; i < sizeof(pins); i++) {
    uint8_t pin = pins[i];
    digitalWrite(pin, RELAY_OFF);
    pinMode(pin, OUTPUT);
  }
  delay(4000);
}

void loop() {
  for (uint8_t i = 0; i < sizeof(pins); i++) {
    uint8_t pin = pins[i];
    digitalWrite(pin, !digitalRead(pin));
    delay(WAIT_TIME);
  }
}

Sketch uses 4,706 bytes (16%) of program storage space. Maximum is 28,672 bytes.
Global variables use 156 bytes (6%) of dynamic memory, leaving 2,404 bytes for local variables. Maximum is 2,560 bytes.

Surprisingly, the shorter variant requires significantly less memory which is very beneficial when writing for micro-controllers.