Sep 212018
 

Sooo, my TP-Link Archer C8 AC1750 Gigabit router decided to not play nice after a short power outage. Didn’t do much anyway except keep a couple of machines in a their own little subnet in which the pi-hole was located.

Replaced it with… NETGEAR GS108Tv2 8-Port Gigabit managed switch and flatten the network a little. Everything was peachy with that set up, but the pihole was still on 172.16.0.0/24… reconfigured that with pihole -r and all looked well except for the pi-hole could not ping outside of the LAN. Grrr! Much hair pulling until…

pi@pihole:~ $ ping 1.1.1.1
PING 1.1.1.1 (1.1.1.1) 56(84) bytes of data.
From 192.168.2.101 icmp_seq=1 Destination Host Unreachable
^Z
[11]+  Stopped                 ping 1.1.1.1
pi@pihole:~ $ clear

pi@pihole:~ $ route
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
default         172.16.0.1      0.0.0.0         UG    202    0        0 eth0
default         192.168.2.1     0.0.0.0         UG    303    0        0 wlan0
default         192.168.2.1     0.0.0.0         UG    304    0        0 wlan1
192.168.2.0     0.0.0.0         255.255.255.0   U     202    0        0 eth0
192.168.2.0     0.0.0.0         255.255.255.0   U     303    0        0 wlan0
192.168.2.0     0.0.0.0         255.255.255.0   U     304    0        0 wlan1

.
.
.
pi@pihole:~ $ sudo route del -net 0.0.0.0 gw 172.16.0.1 metric 202 dev eth0
.
.
.
pi@pihole:~ $ route
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
default         192.168.2.1     0.0.0.0         UG    303    0        0 wlan0
default         192.168.2.1     0.0.0.0         UG    304    0        0 wlan1
192.168.2.0     0.0.0.0         255.255.255.0   U     202    0        0 eth0
192.168.2.0     0.0.0.0         255.255.255.0   U     303    0        0 wlan0
192.168.2.0     0.0.0.0         255.255.255.0   U     304    0        0 wlan1
pi@pihole:~ $ ping 1.1.1.1
PING 1.1.1.1 (1.1.1.1) 56(84) bytes of data.
64 bytes from 1.1.1.1: icmp_seq=1 ttl=59 time=20.9 ms
64 bytes from 1.1.1.1: icmp_seq=2 ttl=59 time=21.3 ms

DICE!

Lord knows when I purchased the Archer, but it was a good few years ago, and I sure as eggs didn’t spent that much on it. What gives?

Jun 162018
 

Getting pretty annoyed with keeping track of which browser on which machine has which add blockers installed, so it’s time for a more holistic approach. Enter, stage left, the Pi-hole. In short it:

…acts as a forwarding DNS server, which means if it doesn’t know where a domain is, it has to forward your query to another server that does. When you install Pi-hole, it knows where the ad-serving domains are (because you tell it), so it doesn’t forward those requests.

With a Rapsberry Pi 3 model b and a 64GB micro SD card, a set-up that is beefier than it needs to be, but who knows what the Pi will be used for in the future…

#1 Get Raspbian, and format micro SD card— after much jiggling with the unlock tab on the adapter and delicately repeatedly inserting-half-inserting into 2012 Macbook Pro’s gunked-up card reader port…

$ wget http://director.downloads.raspberrypi.org/raspbian/images/raspbian-2018-04-19/2018-04-18-raspbian-stretch.zip

$ unzip -a 2018-04-18-raspbian-stretch.zip

$ diskutil list
...
/dev/disk3 (internal, physical):
   #:                       TYPE NAME                    SIZE       IDENTIFIER
   0:     FDisk_partition_scheme                        *62.0 GB    disk3
   1:               Windows_NTFS                         62.0 GB    disk3s1

$ sudo diskutil eraseDisk FAT32 RASPBIAN MBRFormat /dev/disk3
...
Finished erase on disk3

$ df -h
/dev/disk3s1    62Gi  1.5Mi   62Gi     1%       0                   0  100%   /Volumes/RASPBIAN

$ sudo diskutil unmount /dev/disk3s1
Volume RASPBIAN on disk3s1 unmounted

$ sudo dd if=/Users/yearluk/Downloads/2018-04-18-raspbian-stretch.img of=/dev/disk3 bs=4m

#2 First boot on the Pi, change hostname, and run the installer…

$ sudo apt update
$ sudo apt upgrade
$ sudo nano /etc/hostname
pihole
...
$ curl -sSL https://install.pi-hole.net | bash

#3 Select eth0 as the interface, and 1.1.1.1 (Cloudflare) and 8.8.8.8 (Google) as the upstream DNS providers.

IP: 192.168.2.23/24
Default Gateway: 192.168.2.1

Log queries and select web-gui option.

admin password xxxxxxx

install log at /etc/pihole/install.log

web gui 192.168.2.23/admin

As easy as Pi 🙂

Set up SSH and VNC access (just a couple of checkboxes in Raspberian’s GUI), enable wi-fi (DCHP– 192.168.0.104)

Expand blacklists…

curl -s https://tspprs.com/dl/fraud | pihole -g
curl -s https://tspprs.com/dl/ads | pihole -g
curl -s https://tspprs.com/dl/spam | pihole -g
curl -s https://tspprs.com/dl/scam | pihole -g
curl -s https://tspprs.com/dl/ransomware | pihole -g
curl -s https://tspprs.com/dl/phishing | pihole -g
curl -s https://tspprs.com/dl/tracking | pihole -g

Unbound and setting up resolving/recursive DNS (rather than merely forwarding)

What’s the difference? With forwarding, if a name has not been previously associated with an IP (ie. ached on the pi-hole), the request is sent upstream and the result cahed.

Aaand in recursive… request is sent to ROOT servers for resolving say, “.us”, and thence to TLD name servers. Domain lookup will go to AUTHORATATIVE servers handling “yearl” (and subdomains), and all will be relayed and chached locally. So, a few more steps? Why do this? Neither Cloudflare nor Google (or whatever my upstream DNS would have been) will know where I am going. So, err privacy. Why not do this? It will take longer for the first resolution of a name.

Install the recursive DNS resolver:

sudo apt install unbound

Update list of primary root servers:

wget -O root.hints https://www.internic.net/domain/named.root
sudo mv root.hints /var/lib/unbound/

Configure unbound:

sudo nano /etc/unbound/unbound.conf.d/pi-hole.conf
...
server:
    verbosity: 1
    port: 5353
    do-ip4: yes
    do-udp: yes
    do-tcp: yes

<h1>May be set to yes if you have IPv6 connectivity</h1>

<pre><code>do-ip6: no

# Use this only when you downloaded the list of primary root servers!
root-hints: "/var/lib/unbound/root.hints"

# Trust glue only if it is within the servers authority
harden-glue: yes

# Require DNSSEC data for trust-anchored zones, if such data is absent, the zone becomes BOGUS
harden-dnssec-stripped: yes

# Don't use Capitalization randomization as it known to cause DNSSEC issues sometimes
# see https://discourse.pi-hole.net/t/unbound-stubby-or-dnscrypt-proxy/9378 for further details
use-caps-for-id: no

# Reduce EDNS reassembly buffer size.
# Suggested by the unbound man page to reduce fragmentation reassembly problems
edns-buffer-size: 1472

# TTL bounds for cache
cache-min-ttl: 3600
cache-max-ttl: 86400

# Perform prefetching of close to expired message cache entries
# This only applies to domains that have been frequently queried
prefetch: yes

# One thread should be sufficient, can be increased on beefy machines
num-threads: 1

# Ensure kernel buffer is large enough to not loose messages in traffic spikes
so-rcvbuf: 1m

# Ensure privacy of local IP ranges
private-address: 192.168.0.0/16
private-address: 172.16.0.0/12
private-address: 10.0.0.0/8
</code></pre>

And start unbound and validate:

sudo service unbound start
dig yearl.us @127.0.0.1 -p 5353
...
; &lt;&lt;>> DiG 9.10.3-P4-Raspbian &lt;&lt;>> yearl.us @127.0.0.1 -p 5353
;; global options: +cmd
;; Got answer:
;; ->>HEADER&lt;&lt;- opcode: QUERY, status: NOERROR, id: 26331
;; flags: qr rd ra; QUERY: 1, ANSWER: 2, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 1472
;; QUESTION SECTION:
;yearl.us.          IN  A

;; ANSWER SECTION:
yearl.us.       3585    IN  A   104.28.19.121
yearl.us.       3585    IN  A   104.28.18.121

dig sigok.verteiltesysteme.net @127.0.0.1 -p 5353
...
NOERR
...

And then set custom upstram DNS in the pi-hole webgui to “127.0.0.1#5353”

Some basic Pi (Debian Stretch base) stuff…

Basic Pi config:

$ sudo raspi-config

Move over, ifconfig!

$ hostname -I
192.168.2.23 192.168.0.104

$ ip -4 addr show | grep global
    inet 192.168.2.23/24 brd 192.168.2.255 scope global eth0
    inet 192.168.0.104/24 brd 192.168.0.255 scope global wlan0

$ cat /etc/resolv.conf
# Generated by resolvconf
nameserver 127.0.0.1

What’re my interfaces default gateways?

$ ip route | grep default | awk '{print $3}'
192.168.2.1
192.168.0.1

Can configure a static IP via /etc/network/interfaces or /etc/dhcpcd.conf Might do this when moving the Pi from the 192.168.2.0 subnet to the router “guarding” 192.168.0.0

$ sudo route add default gw 192.168.0.1 eth0
$ sudo /etc/init.d/networking restart

EDIT (2018-06-19): Pi-hole was running quite nicely over wi-fi (assigned to 192.168.0.0), went to put it on the LAN and after removing SD card to placement of the Pi board inso some case the SD card decided to go fuck up. Anyhoo repeated above steps with a new (32GB) SD card, and all appears to be peachy-creamy.



Jun 012018
 

My local radio station has a quiz every once in a while. The questions are hardly “University Challenge”, but there is one question that always is fiendishly tricksy and slippery: to identify a place in the county from an anagram of it.

Constantly getting all but one question correct, it is, of course time to cheat use lateral thinking.

#1 get a list of place names. A gazetteer should do.

#2 Build an index (hash). First thoughts were to md5 the names, but there’s an easier way: equalise case, remove spaces and punctuation, order the string…

places = Hash.new([])

File.open("./places.txt", "r") do |file|
  while line = file.gets
    place = line.chomp
    # kill possesives!!
    places_hash = line.chomp.downcase.delete(' ').delete("'")
    places[places_hash.chars.sort.join]+= [place]
  end
end

File.open("places_hash", "w") do |file|
  Marshal.dump(places, file)
end

puts places.inspect

#3 Simply pull out the entry that matches the key…

places = nil

File.open("places_hash", "r") do |file|
  places = Marshal.load(file)
end

wrangler = "REPLACE_ME"

rewrangler = wrangler.downcase.delete(' ').delete("'")
sorted_wrangler = rewrangler.chars.sort.join
answer = places[sorted_wrangler]

answer = answer[0] ||= "nowhere to be found in Shropshire!"
puts "wrangler: #{wrangler} \n\nIt's probably... #{answer}"

And that’s all she wrote… a basic anagram solver.

program output

program output

May 252018
 

Happy GDPR Day!

With the slew of new cookie warnings and privacy notifications and such, it’s time to do something with those annoying domains from content farms that so often pepper search engine results.

Step 1: install Tampermonkey extension

Step 2: set to writing a bunch of JS. But wait, surely someone else has already done this… they have:
https://greasyfork.org/en/scripts/1682-google-hit-hider-by-domain-search-filter-block-sites So a big, BIG thank you to Jefferson Scher.

EDIT: Another script to restore “old” youtube. The non-sucky version (the one with ‘up next’ right below the video being played)… https://cable.ayra.ch/tampermonkey/data/youtube_old_design.user.js

May 182018
 

In all the years I’ve been running OS X (or MacOS as it is now called) I’ve never had the need to su,sudo was always good enough. And this is as it should be.

Until now. Obvs, “root” is disabled in High Sierra. So…

$ cd /System/Library/CoreServices/Applications/
$ ls

Some cool apps here…

drwxr-xr-x  3 root  wheel    96B Mar 30 12:48 About This Mac.app
drwxr-xr-x  3 root  wheel    96B Mar 30 12:48 Archive Utility.app
drwxr-xr-x  3 root  wheel    96B Mar 30 12:48 Directory Utility.app
drwxr-xr-x  3 root  wheel    96B Mar 30 12:48 Feedback Assistant.app
drwxr-xr-x  3 root  wheel    96B Mar 30 12:48 Folder Actions Setup.app
drwxr-xr-x  3 root  wheel    96B Mar 30 12:48 Network Utility.app
drwxr-xr-x  3 root  wheel    96B May  2 10:03 RAID Utility.app
drwxr-xr-x  3 root  wheel    96B Mar 30 12:48 Screen Sharing.app
drwxr-xr-x  3 root  wheel    96B Mar 30 12:48 Storage Management.app
drwxr-xr-x  3 root  wheel    96B May  2 10:03 System Image Utility.app
drwxr-xr-x  3 root  wheel    96B Mar 30 12:48 Wireless Diagnostics.app

Open Directory Utility is the chappie we are interested in

$ open Directory\ Utility.app

@ menu bar: Edit >> Enable Root User
Set the password… y’all know the rules here.

Oh, look! What’s this? Past Self left me a little note. Thanks, Past Self! http://stephen.yearl.us/ssh-key-pair-authentication/

$ su
Password:
sh-3.2#
sh-3.2# cd ~
sh-3.2# pwd
/var/root
sh-3.2# ssh-keygen -t rsa -b 4096 -f ~/.ssh/id_rsa_4096
Generating public/private rsa key pair.
...
Your identification has been saved in /var/root/.ssh/id_rsa_4096.
Your public key has been saved in /var/root/.ssh/id_rsa_4096.pub.
...
sh-3.2# cp id_rsa_4096.pub /Users/yearluk/Desktop/root@xolotl-id_rsa_4096.pub
sh-3.2# exit

Job’s a good un!

Apr 192018
 

#1 plug laptop into eth0

#2 Edgerouter X defaults to 192.168.1.1 as did the TG589vac, but that doesn’t matter as it is now just a modem

#3 get on same net segment… statically configure laptop to
IP 192.168.1.42 (’cause Douglas Adams, obvs)
NM 255.255.255.0
DG 192.168.1.1

#4 Browser to 192.168.1.1, default login usr and pswd: ‘ubnt’ / ‘ubnt’

Dice!

OOH WHAT A PRETTY INTERFACE. Kinda overwhelming actually. Enable SSH, enable DNS forwarding on all interfaces. Should be good to go right? Wrong!

$ ping 1.1.1.1
PING 1.1.1.1 (1.1.1.1): 56 data bytes
64 bytes from 1.1.1.1: icmp_seq=0 ttl=59 time=15.007 ms
64 bytes from 1.1.1.1: icmp_seq=1 ttl=59 time=15.275 ms

$ ping google.com
ping: cannot resolve google.com: Unknown host

Oh noes.

$ ssh 192.168.1.1 -p 2222 -l ubnt

[email protected]:~$ show dns forwarding nameservers
&#45;----------------------------------------------
 Nameservers configured for DNS forwarding
&#45;----------------------------------------------
1.1.1.1 available via 'system'
8.8.8.8 available via 'system'
89.145.254.78 available via 'ppp pppoe0'
94.30.127.100 available via 'ppp pppoe0'

[email protected]:~$ ping 1.1.1.1
PING 1.1.1.1 (1.1.1.1) 56(84) bytes of data.
64 bytes from 1.1.1.1: icmp_req=1 ttl=60 time=18.4 ms
64 bytes from 1.1.1.1: icmp_req=2 ttl=60 time=14.6 ms

[email protected]:~$ ping google.com
PING google.com (216.58.213.78) 56(84) bytes of data.
64 bytes from lhr25s01-in-f14.1e100.net (216.58.213.78): icmp_req=1 ttl=57 time=15.3 ms
64 bytes from lhr25s01-in-f14.1e100.net (216.58.213.78): icmp_req=2 ttl=57 time=15.3 ms

[email protected]:~$configure
[email protected]#set service dns forwarding system
[email protected]#commit
[email protected]# commit
[edit]
[email protected]# save
Saving configuration to '/config/config.boot'...
Done
[edit]
[email protected]# exit
exit
[email protected]:~$

DICE!

Now for the rest of the network… but it’s sooo nice outside. There’s this orange thing in the sky, apparently

Apr 192018
 

Like >90% of folk, I’ve typically used whatever POS modem/router/switch/WAP combo that my ISP supplied me with. These all-in-ones are pretty convenient, but there is something just not right about bundling so many features into a single unit… it basically means that whilst is can do all these things it does not do any one thing particularly well.

Wireless range, for example, is particularly bad (even on the 2.4MHz band). Time for a change. I’ve finally had enough.

#1 get your router’s loin credentials to the service. My ISP had these hard-coded and were not supplied to me. It was a royal waste of 40 minutes trying to explain that I was not trying to login to the TG589vac itself, but instead to use that which allows login to the ISPs edge router… need this to pass on to the replacement router (Ubiquiti’s Edgerouter X) otherwise… no Internetz!

#2 logging in to the TG589vac as “admin” gives one rather limited options, and certainly none for resetting the router. It’s not in the consumer documentation, but log in as “engineer” and use the value next to “Access Key” on the bottom of the device.

#3 Gateway >> Setup Your Gateway >> change “routed type” to bridge

#4 if VSDL retain VLAN 101; if ADSL set ATM VP to 0 and ATM VC to 38

#5 Cross fingers and reboot

#6 Congratulations you now have no Internet access; the TG589vac is now just a modem!

#7 need to get into the device again… that’s an ethernet cable into port #4, so obvs do not connect your new router to this port.

For giggles as “engineer” you get to see that your ISP has probably enabled TR-069 / CWMP. The what now? That’s the L7 protocol that kept your device up-to-date with firmware and such… or is/was a means for them to have get another means to sniff your packets. If you disabled it as soon as you initially got your device, TURN IT BACK ON BEFORE YOU CALL your ISP’s tech support (see #1). Mine got real pissy when I called on an unrelated issue and they could not get in and poke around.

Mar 092018
 

Having just dropped a couple of 4TB Western Digital NAS drives in the openmediavault box, I can now get around to adding more media. My Moby Dick is split over two DVD disks… that’s a pain. Let’s join them…

The obvious, and lazy, and probably-will-not-work way:

~/MyRips/Moby Dick (1998)
$ cat "Moby Dick (1998) - CD 1.avi" "Moby Dick (1998) - CD 2.avi" > "Moby Dick (1998).avi"

Video will only play the length of the first file because this is where the avi header information is. Have to rebuild header information.

http://www.mactricksandtips.com/2011/01/join-avi-or-other-movie-files-together.html suggests mencoder, but another tool from half-a-decade ago seems like hassle.

So let’s do this with ffmpeg (making temporary list first for flexibility)…

~/MyRips/Moby Dick (1998)
$ printf "file '%s'\n" ./*.avi > manifest.txt
$ ffmpeg -f concat -safe 0 -i manifest.txt -c copy "Moby Dick (1998)"

DICE!

UNTRIED, BUT MAYBE NEATER:

$ find STREAM -type f -name '*' -printf "file '$PWD/%p'\n"
Mar 082018
 

Again with the Openmediavault NAS.

A fair few movies I encoded to result in less than 1080P files in order to preserve disk space. That is not really an issue (for now). So, what movies have what resolution?

(I feel like this should all been in shell script, but hey; whatever works right?!)

require 'find'

basedir = '/Volumes/Movies'

# get filetypes
# find . -type f | egrep -i -E -o "\.{1}\w*$" | sort -su

files = %x[find #{basedir} -type f | egrep -i -E  "\.mp4|\.avi|\.mkv|\.m4v|\.mpg" | sort -d]
videos = files.split(/\n+/)
videos = videos.reject! {|item| item =~ /sample.*|\._.*/i }

# puts "#{videos.size} videos found\n"
# videos.each { |x|
#   puts "#{x}\n"
# }

File.open('low-res-video.txt', 'w') do |fo|
  videos.each do |x|
    size = %x[/usr/local/bin/ffprobe -v error -select_streams v:0 -show_entries stream=width -of default=noprint_wrappers=1:nokey=1 '#{x}']
    if size.to_i < 1080
      fo.puts "#{x}...#{size} less than 1080P... reencode?"
    end
  end
end

Output something like:

/Volumes/Movies/_German language movies/Nackt unter Wolfen (2015)/Nackt unter Wolfen (2015).mkv...720
 less than 1080P... reencode?
/Volumes/Movies/_German language movies/The Edukators (2004)/The Edukators (2004)-CD1.avi...592
 less than 1080P... reencode?
/Volumes/Movies/_German language movies/The Edukators (2004)/The Edukators (2004)-CD2.avi...592
 less than 1080P... reencode?
May 112017
 

Part 2 of installing Solus Linux on a 2012 Macbook Pro.

Part 1- simply booting the liveCD to even allow installation is here:
https://stephen.yearl.us/installing-solus-linux-on-a-macbook-pro-9

Rebooting after install lead to, you guessed it, the Black Screen of Death again. Solus was installed, accepting all defaults (and therefore one humongous / paritiion), to /dev/sdb2, /dev/sda1 is the EFI partition on which rEFInd was installed when running OSX MacOS.

❯ lsblk
NAME   MAJ:MIN RM   SIZE RO TYPE MOUNTPOINT
sdb      8:16   0   477G  0 disk
├─sdb2   8:18   0 473.2G  0 part /
└─sdb1   8:17   0   3.7G  0 part [SWAP]
sdc      8:32   1   3.9G  0 disk
sda      8:0    0 465.8G  0 disk
├─sda2   8:2    0   465G  0 part
├─sda3   8:3    0 619.9M  0 part
└─sda1   8:1    0   200M  0 part

#1 boot back into MacOS, mount EFI paritition:

sudo /Users/yearluk/Downloads/refind-bin-0.10.7/mountesp
❯  cd /Volumes/ESP/loader/entries
nano Solus-lts-4.9.22-17.conf

#2 CHANGE:

title Solus 2017.04.18.0
linux /EFI/com.solus-project/kernel-com.solus-project.lts.4.9.22-17
initrd /EFI/com.solus-project/initrd-com.solus-project.lts.4.9.22-17
options root=PARTUUID=c96bc351-b364-4c61-9fe6-8489f0ceec8f quiet ro splash resume=UUID=8f7d1509-fe95-4e47-8017-41611ad0a14c

TO:

title Solus 2017.04.18.0
linux /EFI/com.solus-project/kernel-com.solus-project.lts.4.9.22-17
initrd /EFI/com.solus-project/initrd-com.solus-project.lts.4.9.22-17
options root=PARTUUID=c96bc351-b364-4c61-9fe6-8489f0ceec8f ro nomodeset nouveau.blacklist=1  resume=UUID=8f7d1509-fe95-4e47-8017-41611ad0a14c text 3

#3 Repeat steps #3 through #8 from Part 1 (#7 this time was modprobe nvidia)

#4 update entire system

sudo eopkg up

#5 using the Apple bootloader [hold opt key whilst booting] to boot… boots graphically now.