Posts tagged "linux":

Backup private keys

<2021-05-26>

A few days ago, a good friend asked me: "How do you restore your encrypted offsite backup when you've lost all your other machines".

If all my devices are down at the same time, I might actually have a problem, as I would need at least a PGP key to decrypt my password manager.

So after some research I decided to copy my private ssh and gpg keys to an encrypted veracrypt volume. This is basically a small encrypted folder (~10MB) you can borrow to some of your friends.

Create encrypted volume

# Create a new volume through a text user interface
veracrypt --text --create

# Volume type
1  # normal, default

# Enter volume path:
/private/unencrypted/nextcloud/keys

# Enter volume size:
10M  #  du -sh ~/.gnupg/ ~/.ssh/ # will give you a good idea

# Encryption Algorithm:
1  # For AES, default

# Hash algorithm:
1  # For SHA-512, default

# Filesystem
2  # For FAT, default

# Enter passphrase
MAGA2020!:D

# Enter PIM
# For more information: https://documentation.help/VeraCrypt/Personal%20Iterations%20Multiplier%20(PIM).html
# I leave this empty because I think my passphrase is strong enough

# Enter keyfile
none # default

Decrypt volume and mount it

mkdir /tmp/keys
veracrypt --text /private/unencrypted/nextcloud/keys /tmp/keys

Copy keys

scp -rp ~/.ssh/ /tmp/keys/
scp -rp ~/.gnupg /tmp/keys/
# ...
touch /tmp/keys/FILES_ARE_HIDDEN  # just a hint for the future

Dismount volume

veracrypt --text --dismount /tmp/keys/
# make sure you are not in /tmp/keys, otherwise you will see:
# Error: umount: /tmp/keys: target is busy.
rm -rf /tmp/keys

Setup Headless RaspberryPi With SSH and WLAN

<2020-04-16>

From time to time I have to setup a Raspberry Pi including ssh and wlan. But it's always super annoying to connect keyboard and screen to setup ssh and wifi. And there are other ways:

Use dd or etcher to flash your (raspbian) image to the Pi
Enable SSH
- Mount sd card on your Computer (not on your raspberry pi)
- Create an empty file named ssh and store it on the sd card on /boot

Enable Wifi

create /boot/wpa_supplicant.conf

ctrl_interface=DIR=/var/run/wpa_supplicant GROUP=netdev
network={
    ssid="YOUR_SSID"
    psk="YOUR_WIFI_PASSWORD"
    key_mgmt=WPA-PSK
}

Pi will get an IP adress from your DHCP-Server
unmount

Start Rarspberry Pi
- put in sd card
- power on
Get IP Adress

On your computer:
```
nmap -sP 192.168.XXX.1/24  # replace XXX
```
- most likely the hostname you are locking for is rasberrypi ;)

Connect to Raspberry Pi:

ssh pi@IpFromPrevCommand
# pw: raspberry

Configure system

sudo raspi-config
# Change default password!
# Update System
# Change settings

HomeAssistant: How I backup and restore my data!

<2019-12-17>

Update [2021-01-24 Sun]

Fixed command to restore corrupt backup. It should be sqlite3 for both dump and restore! Thank you Philip for the hint!

Assuming you're using hass.io on a raspberry pi:

Backup

There are multiple option of backing up your home assistant data:

1. Build in option

Login to WebUI -> in the sidebar go to Hass.io -> snapshots
Here your can create a new snapshot
after clicking the reload button (upper right corner) you can download your snapshot

2. An automation to create snapshots on a regular base

Put the following in your automation.yaml

- alias: Create hourly backup
  trigger:
    - platform: time_pattern
      minutes: '/60'
  action:
    - service: notify.mobile_app_iphone_your_name
      data:
        message: Hourly Backup 💾 is created!
        title: Backup
    - service: hassio.snapshot_full
      data_template:
        name: >
          Backup_{{ now().strftime('%d-%m-%YT%H-%M-%S') }}

3. Automatic offside backups

Create new backup user on your server
Add the user to the allowed group of ssh users in /etc/ssh/sshd_config
sudo systemctl restart sshd
Create new ssh key for hass.io
Add new ssh pub key to ~/authorized_keys
Add this repo to your plugins https://github.com/carstenschroeder/hassio-addons

Addon config:

{
  "ssh_enabled": true,
  "ssh_host": "192.168.178.27",
  "ssh_port": 22,
  "ssh_user": "ms-backup",
  "ssh_key": [
    "-----BEGIN OPENSSH PRIVATE KEY-----",
    "AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA",
    cat ~/.ssh/hassio-key
    "ZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZ",
    "-----END OPENSSH PRIVATE KEY-----"
  ],
  "remote_directory": "/hpdata/homes/ms-backup/backup_hassio",
  "zip_password": "",
  "keep_local_backup": "48",
  "rsync_enabled": false,
  "rsync_host": "",
  "rsync_rootfolder": "hassio-sync",
  "rsync_user": "",
  "rsync_password": ""
}

Add your new private key to the configuration above
create a new automation in automations.yaml

- alias: Offside backup
  trigger:
    platform: time_pattern
    minutes: '/60'
  action:
  - service: notify.mobile_app_iphone_your_name
    data:
      message: Offside backup 💾 is created!
      title: Backup
  - service: hassio.addon_start
    data:
      addon: 36883ed7_remote_backup
      #      ^- to get the propper ID go to hass.io -> Addons -> Remote Backup 
      #         I couldn't figure out where this ID comes from

This will trigger the remote backup addon. After the backup is done the state of remote backup goes back to stopped till it is triggered the next time. While testing this did not worked for very short intervals < 3 Minutes. For longer intervals it works like expected.

Restore

No matter how you created the backup to restore it, proceed as follows

Install latest hass.io to sd card.
Boot raspberry pi
It will take up to 20 minutes to initialize hass.io
Create a tmp account
power off
copy backup to sd card (/hassos-data/supervisor/backup)
Restart raspberry pi
In the sidebar go to Hass.io -> Snapshots, choose your snapshot -> wipe and restore, You're done!

I had some problems, with a corrupt database. (HomeAssistant could not add any more new data. I think there was something like (sqlite3.DatabaseError) database disk image is malformed in the logs).

To fix/repair your database:

Copy your (corrupt) database (home-assistant_v2.db) to your local linux system

sqlite3 home-assistant_v2.db ".dump" | sqlite3 home-assistant_v2.db_fixed

Now delete your old (corrupt database) and replace it with home-assistant_v2.db_fixed. To do that delete the corrupt one and rename home-assistant_v2.db_fixed to home-assistant_v2.db. Now, power on your raspberry pi and everything should be fine again.

Remote jupyter notebook

<2018-08-04>

1.) Access the remote server via ssh

2.) Start jupyter server on the remote machine

jupyter notebook --no-browser --port=8889

3.) Forward port 8889 from your remote machine to port 8888 of your local machine

ssh -N -f -L localhost:8888:localhost:8889 remote_user@remote_host

4.) Access the remote notebook from your local browser visiting localhost:8888

I find this very usefull, especially when working with plots and audio.

Comments

If you have comments, questions or opinions please drop me a line at 2018-08-04–remote-jupyter-notebook AT zngguvnf dot org. Please tell me whether it's ok to publish your comment here or not.

Matplotlib on a remote machine

<2018-07-21>

Make sure X forwarding is enabled on your remote machine

cat /etc/ssh/sshd_config | grep X11Forwarding

If the result it not X11Forwarding yes adjust the file.

Access remote machine with X forwarding from your local machine.

ssh -X user@remote-host

On the remote machine: Activate X-compatible backend

import matplotlib
matplotlib.use('tkagg')

On the remote machine: Plot as normal

import matplotlib.pyplot as plt
plt.plot([1,2,3])
plt.show()

See plot on your local machine.

Comments

If you have comments, questions or opinions please drop me a line at 2018-07-21–matplotlib-on-remote-machine AT zngguvnf dot org. Please tell me whether it's ok to publish your comment here or not.

Create space on your hard disk

<2018-03-21>

Today was the day again, my hard drive was full and I had the choice:

Buy a new hard disk and rest for the next few years
Delete some old files and save a lot of money

I finally decided to delete some files and it was easier than I thought.

Summarize disk usage and sort the results:

du -ah | sort -h

With this output in mind, you can concentrate on the large files and folders. I could quickly find and delete many large duplicate folders and large files that I no longer needed.

To check your free disk space:

df -h

Remember to make backups before you delete files

Comments

If you have comments, questions or opinions please drop me a line at blog AT zngguvnf dot org. Please tell me whether it's ok to publish your comment here or not.

Building Signal-Desktop from source

<2018-02-25>

Update [2018-06-11 Mon]

use yarn instead of npm

In the following lines I will explain how I managed to build signal-desktop from source.

Since there aren't any official packages for rpm based distros like fedora I decided to build Signal from source.

Install dependencies:

sudo dnf install npm
sudo dnf install libXScrnSaver
sudo dnf install gcc-c++

Clone the repository:

git clone https://github.com/signalapp/Signal-Desktop.git ~/.gitware/Signal-Desktop

Checkout the branch you want to build: (I like to build the latest tag-release. Right now ~~v1.3.0~~ v1.12.0 is the latest version)

cd Signal-Desktop
git checkout tags/v1.3.0

Build the electron app:

# npm install  # outdated
# npm run dist-prod # outdated
npm install yarn
yarn install
yarn generate
yarn build-release

When the build process is finished you can start signal-desktop:

# ./dist/linux-unpacked/signal-desktop # outdated
./release/linux-unpacked/signal-desktop

To create an app launcher: Create a symbolic link:

# sudo ln -s ~/.gitware/Signal-Desktop/dist/linux-unpacked/signal-desktop /usr/local/bin/signal-desktop
sudo ln -s ~/.gitware/Signal-Desktop/release/linux-unpacked/signal-desktop /usr/local/bin/signal-desktop

Copy the icon:

sudo cp ~/.gitware/Signal-Desktop/images/icon_250.png /usr/local/share/icons/signal.png

(If the directory /usr/local/share/icons does not exsist yet, create it sudo mkdir /usr/local/share/icons/)

With the editor of your choice (you might want to use emacs or nano over vim) create a signal.desktop file:

sudo vi /usr/local/share/applications/signal.desktop

Enter the the following content:

[Desktop Entry]
Name=Signal
Comment=Private messaging from your desktop
Exec="signal-desktop" %U
Terminal=false
Type=Application
Icon=signal.png
StartupWMClass=Signal

That's it. Close the file and open Signal from your launcher.

Please remember that you are responsible for updating the app yourself. To update the app:

git checkout tags/vNewVersion
# npm install
# npm run dist-prod
yarn generate
yarn build-release

To verify your version open the Signal -> Help -> About Signal Desktop

Comments

If you have comments, questions or opinions please drop me a line at blog AT zngguvnf dot org. Please tell me whether it's ok to publish your comment here or not.

Ubuntu 17.10 and CiscoAnyconnect

<2017-12-04>

tl;dr

If you are running Ubuntu 17.10 and fail to install/start Cisco Anyconnect (version 4.5.0.2033):

sudo apt install libpangox-1.0-0

fixed it for me.

The whole story

To login into the virtual private network (VPN) of my university you need to install CiscoAnyconnect.

So I downloaded the installer from the university page and installed it

./anyconnect-linux64-4.5.02033-core-vpn-webdeploy-k9.sh

(Actually this is not the latest version. On the Cisco page you can find version 4.5.02036 but I don't know if this changes anything.)

After the installation an icon "Cisco Anyconnect" appeared in my starter but nothing happend when I clicked the icon. No new window, no error message, nothing.

So I had a closer look:

There was a log file in /tmp/ which told me that the everything was installed to /opt/cisco/anyconnect/.

If you cd to /opt/cisco/anyconnect/bin and run

./vpn

you can configure your VPN using the command line.

So it looks like there is something broken with the ui. Nevertheless, for me this workaround fixes all my problems.

But maybe we can fix the ui, too.

In /opt/cisco/anyconnect/bin there is a file called vpnui. Let's give it a try:

./vpnui

./vpnui: error while loading shared libraries: libpangox-1.0.so.0: cannot open shared object file: No such file or directory

So now we have an error message!

Let's see how to fix this:

apt search libpangox

Sorting... Done
Full Text Search... Done
libpangox-1.0-0/artful 0.0.2-5 amd64
  pango library X backend

libpangox-1.0-dev/artful 0.0.2-5 amd64
  pango library X backend - development files

libpangoxft-1.0-0/artful,now 1.40.12-1 amd64 [installed]
  Layout and rendering of internationalized text

Looks like libpangox-1.0-0 is what we're looking for.

sudo apt install libpangox-1.0-0

And … it works! ./vpnui as well as the "Cisco Anyconnect" starter will let you configure your VPN.

Comments

If you have comments, questions or opinions please drop me a line at blog AT zngguvnf dot org. Please tell me whether it's ok to publish your comment here or not.

Processing multiple PDFs using the command line

<2017-12-01>

Update [2018-04-17 Tue]

Remove password from pdf
Remove string from pdf

Update [2018-03-30 Fri]:

qpdf commands as alternativ to pdftk which is no longer available for fedora.
Quote Stackexchange answer to 'How can I reduce the file size of a scanned PDF file?'

From time to time I need to process lots of .pdf files.

Here are a few commandline calls that help me a lot:

Split pdf in single pages

Split one .pdf with multiple pages in multiple .pdf files with just one page.

pdftk PdfWithMultiplePages.pdf burst

qpdf --split-pages input.pdf output.pdf

Merge pages to single pdf

Merge multiple .pdf with one or more pages into one single .pdf.

To merge all .pdf in currenct directory to one single file:

pdftk ./*.pdf cat output PdfWithMultiplePages.pdf

Alternatively to can type pdftk, mark all files you want to combine in your file manager, drag and drop them to your terminal and finish the command with cat output PDFWithMultiplePages.pdf

pdfjam ./*pdf -o PdfWithMultiplePages.pdf

Convert from DIN A3 (landscape) to DIN A4 portrait

Sometimes .pdf are in DIN A3 (landscape) and it looks like two DIN A4 pages side by side.

Use the following command to split those documents:

mutool poster -y 2 input.pdf output.pdf

(use -y to preform a vertical split or -x for a horizontal split.) mutool comes as part of mupdf (sudo apt install mupdf-tools)

Convert to DIN A4

pdfjam --outfile filename.pdf --paper a4paper filename.pdf

Batch processing

To convert all .pdf files including those in subfolders to a4

for f in ./**/*.pdf ; do
  pdfjam --outfile "$f" --paper a4paper "$f"
done

Reduce file size of scanned PDF file

There is a question for this on stackexchange and a fantastic answer, which I would like to insert here for reference:

Use the following ghostscript command:

gs -sDEVICE=pdfwrite -dCompatibilityLevel=1.4 -dPDFSETTINGS=/screen -dNOPAUSE -dQUIET -dBATCH -sOutputFile=output.pdf input.pdf

-dPDFSETTINGS=/screen lower quality, smaller size.
-dPDFSETTINGS=/ebook for better quality, but slightly larger pdfs.
-dPDFSETTINGS=/prepress output similar to Acrobat Distiller "Prepress Optimized" setting
-dPDFSETTINGS=/printer selects output similar to the Acrobat Distiller "Print Optimized" setting
-dPDFSETTINGS=/default selects output intended to be useful across a wide variety of uses, possibly at the expense of a larger output file

Remove password from pdf

qpdf -password=YourTopSecretPassword -decrypt password-protected-file.pdf file-without-password.pdf

Remove string from pdf

(works to remove text that you can mark in the pdf)

qpdf --stream-data=uncompress YourFile.pdf uncompressed.pdf

Replace 'Some Text' with whitespace

sed 's/Some Text/ /g' < uncompressed.pdf > uncompressed_without_string.pdf

If you want to replace things other than letters (such as brackets), the sed manual will help you. Sometimes it is helpful to remove the desired expression in individual steps (but watch out that you only delete it where you want it to be deleted).

qpdf --stream-data=compress uncompressed_without_string.pdf YourFile_free.pdf

Comments

If you have comments, questions or opinions please drop me a line at blog AT zngguvnf dot org. Please tell me whether it's ok to publish your comment here or not.