A collection of awesome lists, manuals, blogs, hacks, one-liners and tools for Awesome Ninja Admins.
Who is Ninja Admins?
- race of pure evil who rule the network through a monarchist feudal system
- they never opened the door for strangers (or anyone at all)
- they know very nasty piece of code like a fork bombs
- they can make dd is not a destroyer of disks
- they know that
#!/usr/bin/env bashsuperior to#!/bin/bash - they know that
su -logs in completely as root - they miss and cry for Slackware on production
- they love the old admin nix-world
☑️ Todo
- Add useful shell functions
- Add one-liners for collection tools (eg. CLI Tools)
- Add Ninja Admins T-Shirt stickers
- Generate Awesome Ninja Admins book (eg. pdf format)
Ninja Admins Collection
CLI Tools
▪️ Shells
▪️ Managers
▪️ Network
▪️ SSL
▪️ Auditing Tools
▪️ System Diagnostics/Debuggers
▪️ Log Analyzers
▪️ Databases
Web Tools
▪️ SSL
▪️ HTTP Headers
▪️ DNS
▪️ Mail
▪️ Mass scanners (search engines)
▪️ Net-tools
▪️ Code parsers/playgrounds
▪️ Performance
▪️ Passwords
Manuals/Howtos/Tutorials
▪️ Bash
▪️ Unix tutorials
▪️ Security
▪️ Web Apps
▪️ Secret Knowledge
Blogs
Systems/Services
▪️ Systems
▪️ HTTP(s) Services
▪️ Security/hardening
Lists
Hacking/Penetration testing
▪️ Bounty programs
▪️ Web Training Apps (local installation)
▪️ Labs
One-liners
Table of Contents
Tool: terminal
Reload shell without exit
Close shell keeping all subprocess running
Exit without saving shell history
kill -9 $$ unset HISTFILE && exit
Perform a branching conditional
true && { echo success;} || { echo failed; }
Pipe stdout and stderr to separate commands
some_command > >(/bin/cmd_for_stdout) 2> >(/bin/cmd_for_stderr)
Redirect stdout and stderr each to separate files and print both to the screen
(some_command 2>&1 1>&3 | tee errorlog ) 3>&1 1>&2 | tee stdoutlog
List of commands you use most often
history | \ awk '{CMD[$2]++;count++;}END { for (a in CMD)print CMD[a] " " CMD[a]/count*100 "% " a;}' | \ grep -v "./" | \ column -c3 -s " " -t | \ sort -nr | nl | head -n 20
Empty a file (truncate to 0 size)
Quickly backup a file
Delete all files in a folder that don't match a certain file extension
Edit a file on a remote host using vim
vim scp://user@host//etc/fstab
Create a directory and change into it at the same time
mkd () { mkdir -p "$@" && cd "$@"; }
Convert uppercase files to lowercase files
Print a row of characters across the terminal
printf "%`tput cols`s" | tr ' ' '#'
Show shell history without line numbers
history | cut -c 8- fc -l -n 1 | sed 's/^\s*//'
Run command(s) after exit session
cat > /etc/profile << __EOF__ _after_logout() { username=$(whoami) for _pid in $(ps afx | grep sshd | grep "$username" | awk '{print $1}') ; do kill -9 $_pid done } trap _after_logout EXIT __EOF__
Generate a sequence of numbers
for ((i=1; i<=10; i+=2)) ; do echo $i ; done # alternative: seq 1 2 10 for ((i=5; i<=10; ++i)) ; do printf '%02d\n' $i ; done # alternative: seq -w 5 10
Tool: mount
Mount a temporary ram partition
mount -t tmpfs tmpfs /mnt -o size=64M
-t- filesystem type-o- mount options
Remount a filesystem as read/write
Tool: fuser
Kills a process that is locking a file
Show what PID is listening on specific port
Tool: ps
Show a 4-way scrollable process tree with full details
Processes per user counter
ps hax -o user | sort | uniq -c | sort -r
Tool: find
Find files that have been modified on your system in the past 60 minutes
Find all files larger than 20M
find / -type f -size +20M
Find duplicate files (based on MD5 hash)
find -type f -exec md5sum '{}' ';' | sort | uniq --all-repeated=separate -w 33
Change permission only for files
cd /var/www/site && find . -type f -exec chmod 766 {} \; cd /var/www/site && find . -type f -exec chmod 664 {} +
Change permission only for directories
cd /var/www/site && find . -type d -exec chmod g+x {} \; cd /var/www/site && find . -type d -exec chmod g+rwx {} +
Find files and directories for specific user
find . -user <username> -print
Find files and directories for all without specific user
find . \!-user <username> -print
Delete older files than 60 days
find . -type f -mtime +60 -deleteRecursively remove all empty sub-directories from a directory
find . -depth -type d -empty -exec rmdir {} \;
How to find all hard links to a file
find </path/to/dir> -xdev -samefile filename
Tool: top
Use top to monitor only all processes with the specific string
top -p $(pgrep -d , <str>)<str>- process containing str (eg. nginx, worker)
Tool: strace
Track the open request of a network port
strace -f -e trace=bind nc -l 80
Track the open request of a network port (show TCP/UDP)
strace -f -e trace=network nc -lu 80
Tool: kill
Kill a process running on port
kill -9 $(lsof -i :<port> | awk '{l=$2} END {print l}')
Tool: diff
Compare two directory trees
diff <(cd directory1 && find | sort) <(cd directory2 && find | sort)
Tool: tail
Annotate tail -f with timestamps
tail -f file | while read ; do echo "$(date +%T.%N) $REPLY" ; done
Analyse an Apache access log for the most common IP addresses
tail -10000 access_log | awk '{print $1}' | sort | uniq -c | sort -n | tail
Analyse web server log and show only 5xx http codes
tail -n 100 -f /path/to/logfile | grep "HTTP/[1-2].[0-1]\" [5]"
Tool: tar
System backup with exclude specific directories
cd / tar -czvpf /mnt/system$(date +%d%m%Y%s).tgz --directory=/ \ --exclude=proc/* --exclude=sys/* --exclude=dev/* --exclude=mnt/* .
System backup with exclude specific directories (pigz)
cd / tar cvpf /backup/snapshot-$(date +%d%m%Y%s).tgz --directory=/ \ --exclude=proc/* --exclude=sys/* --exclude=dev/* \ --exclude=mnt/* --exclude=tmp/* --use-compress-program=pigz .
Tool: dump
System backup to file
dump -y -u -f /backup/system$(date +%d%m%Y%s).lzo /Restore system from lzo file
cd / restore -rf /backup/system$(date +%d%m%Y%s).lzo
Tool: cpulimit
Limit the cpu usage of a process
Tool: pwdx
Show current working directory of a process
Tool: taskset
Start a command on only one CPU core
Tool: tr
Show directories in the PATH, one per line
Tool: chmod
Remove executable bit from all files in the current directory
Restore permission for /bin/chmod
# 1: cp /bin/ls chmod.01 cp /bin/chmod chmod.01 ./chmod.01 700 file # 2: /bin/busybox chmod 0700 /bin/chmod # 3: setfacl --set u::rwx,g::---,o::--- /bin/chmod
Tool: who
Find last reboot time
Tool: screen
Start screen in detached mode
Tool: du
Show 20 biggest directories with 'K M G'
du | \ sort -r -n | \ awk '{split("K M G",v); s=1; while($1>1024){$1/=1024; s++} print int($1)" "v[s]"\t"$2}' | \ head -n 20
Tool: inotifywait
Init tool everytime a file in a directory is modified
while true ; do inotifywait -r -e MODIFY dir/ && ls dir/ ; done;
Tool: openssl
Testing connection to remote host
echo | openssl s_client -connect google.com:443 -showcerts
Testing connection to remote host (with SNI support)
echo | openssl s_client -showcerts -servername google.com -connect google.com:443
Testing connection to remote host with specific ssl version
openssl s_client -tls1_2 -connect google.com:443
Testing connection to remote host with specific ssl cipher
openssl s_client -cipher 'AES128-SHA' -connect google.com:443Generate private key
# _ciph: des3, aes ( _ciph="des3" ; _fd="private.key" ; _len="2048" ; \ openssl genrsa -${_ciph} -out ${_fd} ${_len} )
Remove password from private key
( _fd="private.key" ; _fd_unp="private_unp.key" ; \ openssl rsa -in ${_fd} -out ${_fd_unp} )
Get public key from private key
( _fd="private.key" ; _fd_pub="public.key" ; \ openssl rsa -pubout -in ${_fd} -out ${_fd_pub} )
Generate private key + csr
( _fd="private.key" ; _fd_csr="request.csr" ; _len="2048" ; \ openssl req -out ${_fd_csr} -new -newkey rsa:${_len} -nodes -keyout ${_fd} )
Generate csr
( _fd="private.key" ; _fd_csr="request.csr" ; \ openssl req -out ${_fd_csr} -new -key ${_fd} )
Generate csr (metadata from exist certificate)
( _fd="private.key" ; _fd_csr="request.csr" ; _fd_crt="cert.crt" ; \ openssl x509 -x509toreq -in ${_fd_crt} -out ${_fd_csr} -signkey ${_fd} )
Generate csr with -config param
( _fd="private.key" ; _fd_csr="request.csr" ; \ openssl req -new -sha256 -key ${_fd} -out ${_fd_csr} \ -config <( cat <<-EOF [req] default_bits = 2048 prompt = no default_md = sha256 req_extensions = req_ext distinguished_name = dn [ dn ] C=<two-letter ISO abbreviation for your country> ST=<state or province where your organization is legally located> L=<city where your organization is legally located> O=<legal name of your organization> OU=<section of the organization> CN=<fully qualified domain name> [ req_ext ] subjectAltName = @alt_names [ alt_names ] DNS.1 = <fully qualified domain name> DNS.2 = <next domain> DNS.3 = <next domain> EOF ))
Convert DER to PEM
( _fd_der="cert.crt" ; _fd_pem="cert.pem" ; \ openssl x509 -in ${_fd_der} -inform der -outform pem -out ${_fd_pem} )
Convert PEM to DER
( _fd_der="cert.crt" ; _fd_pem="cert.pem" ; \ openssl x509 -in ${_fd_pem} -outform der -out ${_fd_der} )
Checking whether the private key and the certificate match
(openssl rsa -noout -modulus -in private.key | openssl md5 ; openssl x509 -noout -modulus -in certificate.crt | openssl md5) | uniq
Tool: gnutls-cli
Testing connection to remote host (with sni)
gnutls-cli -p 443 google.com
Testing connection to remote host (without sni)
gnutls-cli --disable-sni -p 443 google.com
Tool: secure-delete
Secure delete with shred
shred -vfuz -n 10 file shred --verbose --random-source=/dev/urandom -n 1 /dev/sda
Secure delete with scrub
scrub -p dod /dev/sda scrub -p dod -r file
Secure delete with badblocks
badblocks -s -w -t random -v /dev/sda badblocks -c 10240 -s -w -t random -v /dev/sda
Secure delete with secure-delete
srm -vz /tmp/file
sfill -vz /local
sdmem -v
swapoff /dev/sda5 && sswap -vz /dev/sda5Tool: dd
Show dd status every so often
dd <dd_params> status=progress watch --interval 5 killall -USR1 dd
Tool: curl
curl -Iks https://www.google.com
-I- show response headers only-k- insecure connection when using ssl-s- silent mode (not display body)
curl -Iks --location -X GET -A "x-agent" https://www.google.com--location- follow redirects-X- set method-A- set user-agent
curl -Iks --location -X GET -A "x-agent" --proxy http://127.0.0.1:16379 https://www.google.com--proxy [socks5://|http://]- set proxy server
Check DNS and HTTP trace with headers for specific domains
### Set domains and external dns servers. _domain_list=(google.com) ; _dns_list=("8.8.8.8" "1.1.1.1") for _domain in "${_domain_list[@]}" ; do printf '=%.0s' {1..48} echo printf "[\\e[1;32m+\\e[m] resolve: %s\\n" "$_domain" for _dns in "${_dns_list[@]}" ; do # Resolve domain. host "${_domain}" "${_dns}" echo done for _proto in http https ; do printf "[\\e[1;32m+\\e[m] trace + headers: %s://%s\\n" "$_proto" "$_domain" # Get trace and http headers. curl -Iks -A "x-agent" --location "${_proto}://${_domain}" echo done done unset _domain_list _dns_list
Tool: httpie
http -p Hh https://www.google.com
-p- print request and response headersH- request headersB- request bodyh- response headersb- response body
http -p Hh --follow --max-redirects 5 --verify no https://www.google.com
-F, --follow- follow redirects--max-redirects N- maximum for--follow--verify no- skip SSL verification
http -p Hh --follow --max-redirects 5 --verify no --proxy http:http://127.0.0.1:16379 https://www.google.com
--proxy [http:]- set proxy server
Tool: ssh
Compare a remote file with a local file
ssh user@host cat /path/to/remotefile | diff /path/to/localfile -SSH connection through host in the middle
ssh -t reachable_host ssh unreachable_host
Run command over ssh on remote host
cat > cmd.txt << __EOF__ cat /etc/hosts __EOF__ ssh host -l user $(<cmd.txt)
Get public key from private key
ssh-keygen -y -f ~/.ssh/id_rsaGet all fingerprints
ssh-keygen -l -f .ssh/known_hosts
Ssh authentication with user password
ssh -o PreferredAuthentications=password -o PubkeyAuthentication=no user@remote_host
Ssh authentication with publickey
ssh -o PreferredAuthentications=publickey -o PubkeyAuthentication=yes -i id_rsa user@remote_host
Simple recording SSH session
function _ssh_sesslog() { _sesdir="<path/to/session/logs>" mkdir -p "${_sesdir}" && \ ssh $@ 2>&1 | tee -a "${_sesdir}/$(date +%Y%m%d).log" } # Alias: alias ssh='_ssh_sesslog'
Using Keychain for SSH logins
### Delete all of ssh-agent's keys. function _scl() { /usr/bin/keychain --clear } ### Add key to keychain. function _scg() { /usr/bin/keychain /path/to/private-key source "$HOME/.keychain/$HOSTNAME-sh" }
Tool: linux-dev
Testing remote connection to port
timeout 1 bash -c "</dev/<proto>/<host>/<port>" >/dev/null 2>&1 ; echo $?
<proto- set protocol (tcp/udp)<host>- set remote host<port>- set destination port
Read and write to TCP or UDP sockets with common bash tools
exec 5<>/dev/tcp/<host>/<port>; cat <&5 & cat >&5; exec 5>&-
Tool: tcpdump
tcpdump -ne -i eth0 -Q in host 192.168.252.1 and port 443-n- don't convert addresses-e- print the link-level headers-i [iface|any]- set interface-Q|-D [in|out|inout]- choose send/receive direction (-D- for old tcpdump versions)host [ip|hostname]- set host, also[host not][and|or]- set logicport [1-65535]- set port number, also[port not]
tcpdump -ne -i eth0 -Q in host 192.168.252.1 and port 443 -c 5 -w tcpdump.pcap-c [num]- capture only num number of packets-w [filename]- write packets to file,-r [filename]- reading from file
Tool: tcpick
Analyse packets in real-time
while true ; do tcpick -a -C -r dump.pcap ; sleep 2 ; clear ; done
Tool: ngrep
ngrep -d eth0 "www.google.com" port 443-d [iface|any]- set interface[domain]- set hostnameport [1-65535]- set port number
ngrep -d eth0 "www.google.com" (host 10.240.20.2) and (port 443)(host [ip|hostname])- filter by ip or hostname(port [1-65535])- filter by port number
ngrep -d eth0 -qt -O ngrep.pcap "www.google.com" port 443-q- quiet mode (only payloads)-t- added timestamps-O [filename]- save output to file,-I [filename]- reading from file
ngrep -d eth0 -qt 'HTTP' 'tcp'
HTTP- show http headerstcp|udp- set protocol[src|dst] host [ip|hostname]- set direction for specific node
ngrep -l -q -d eth0 -i "User-Agent: curl*"-l- stdout line buffered-i- case-insensitive search
Tool: hping3
hping3 -V -p 80 -s 5050 <scan_type> www.google.com
-V|--verbose- verbose mode-p|--destport- set destination port-s|--baseport- set source port<scan_type>- set scan type-F|--fin- set FIN flag, port open if no reply-S|--syn- set SYN flag-P|--push- set PUSH flag-A|--ack- set ACK flag (use when ping is blocked, RST response back if the port is open)-U|--urg- set URG flag-Y|--ymas- set Y unused flag (0x80 - nullscan), port open if no reply-M 0 -UPF- set TCP sequence number and scan type (URG+PUSH+FIN), port open if no reply
hping3 -V -c 1 -1 -C 8 www.google.com
-c [num]- packet count-1- set ICMP mode-C|--icmptype [icmp-num]- set icmp type (default icmp-echo = 8)
hping3 -V -c 1000000 -d 120 -S -w 64 -p 80 --flood --rand-source <remote_host>
--flood- sent packets as fast as possible (don't show replies)--rand-source- random source address mode-d --data- data size-w|--win- winsize (default 64)
Tool: netcat
-l- listen for an incoming connection-k- listening after client has disconnected>filename.out- save receive data to file (optional)
nc 192.168.0.1 5051 < filename.in< filename.in- send data to remote host
-v- verbose output-z- scan for listening daemons
nc -vzu 10.240.30.3 1-65535
-u- scan only udp ports
Transfer data file (archive)
server> nc -l 5000 | tar xzvfp - client> tar czvfp - /path/to/dir | nc 10.240.30.3 5000
Launch remote shell
server> nc -l 5000 -e /bin/bash client> nc 10.240.30.3 5000
Simple file server
while true ; do nc -l 5000 | tar -xvf - ; done
Simple minimal HTTP Server
while true ; do nc -l -p 1500 -c 'echo -e "HTTP/1.1 200 OK\n\n $(date)"' ; done
Simple HTTP Server
Restarts web server after each request - remove
whilecondition for only single connection.
cat > index.html << __EOF__ <!doctype html> <head> <meta charset="utf-8"> <meta http-equiv="X-UA-Compatible" content="IE=edge,chrome=1"> <title></title> <meta name="description" content=""> <meta name="viewport" content="width=device-width, initial-scale=1"> </head> <body> <p> Hello! It's a site. </p> </body> </html> __EOF__
server> while : ; do \ (echo -ne "HTTP/1.1 200 OK\r\nContent-Length: $(wc -c <index.html)\r\n\r\n" ; cat index.html;) | \ nc -l -p 5000 \ ; done
-p- port number
Simple HTTP Proxy (single connection)
#!/usr/bin/env bash if [[ $# != 2 ]] ; then printf "%s\\n" \ "usage: ./nc-proxy listen-port bk_host:bk_port" fi _listen_port="$1" _bk_host=$(echo "$2" | cut -d ":" -f1) _bk_port=$(echo "$2" | cut -d ":" -f2) printf " lport: %s\\nbk_host: %s\\nbk_port: %s\\n\\n" \ "$_listen_port" "$_bk_host" "$_bk_port" _tmp=$(mktemp -d) _back="$_tmp/pipe.back" _sent="$_tmp/pipe.sent" _recv="$_tmp/pipe.recv" trap 'rm -rf "$_tmp"' EXIT mkfifo -m 0600 "$_back" "$_sent" "$_recv" sed "s/^/=> /" <"$_sent" & sed "s/^/<= /" <"$_recv" & nc -l -p "$_listen_port" <"$_back" | \ tee "$_sent" | \ nc "$_bk_host" "$_bk_port" | \ tee "$_recv" >"$_back"
server> chmod +x nc-proxy && ./nc-proxy 8080 192.168.252.10:8000 lport: 8080 bk_host: 192.168.252.10 bk_port: 8000 client> http -p h 10.240.30.3:8080 HTTP/1.1 200 OK Accept-Ranges: bytes Cache-Control: max-age=31536000 Content-Length: 2748 Content-Type: text/html; charset=utf-8 Date: Sun, 01 Jul 2018 20:12:08 GMT Last-Modified: Sun, 01 Apr 2018 21:53:37 GMT
Create a single-use TCP or UDP proxy
### TCP -> TCP nc -l -p 2000 -c "nc [ip|hostname] 3000" ### TCP -> UDP nc -l -p 2000 -c "nc -u [ip|hostname] 3000" ### UDP -> UDP nc -l -u -p 2000 -c "nc -u [ip|hostname] 3000" ### UDP -> TCP nc -l -u -p 2000 -c "nc [ip|hostname] 3000"
Tool: socat
Testing remote connection to port
socat - TCP4:10.240.30.3:22
-- standard input (STDIO)TCP4:<params>- set tcp4 connection with specific params[hostname|ip]- set hostname/ip[1-65535]- set port number
Redirecting TCP-traffic to a UNIX domain socket under Linux
socat TCP-LISTEN:1234,bind=127.0.0.1,reuseaddr,fork,su=nobody,range=127.0.0.0/8 UNIX-CLIENT:/tmp/foo
TCP-LISTEN:<params>- set tcp listen with specific params[1-65535]- set port numberbind=[hostname|ip]- set bind hostname/ipreuseaddr- allows other sockets to bind to an addressfork- keeps the parent process attempting to produce more connectionssu=nobody- set userrange=[ip-range]- ip range
UNIX-CLIENT:<params>- communicates with the specified peer socketfilename- define socket
Tool: lsof
Show process that use internet connection at the moment
Show process that use specific port number
Lists all listening ports together with the PID of the associated process
List all open ports and their owning executables
lsof -i -P | grep -i "listen"
Show all open ports
Show open ports (LISTEN)
lsof -Pni4 | grep LISTEN | column -t
List all files opened by a particular command
View user activity per directory
lsof -u username -a +D /etc
Show 10 Largest Open Files
lsof / | \ awk '{ if($7 > 1048576) print $7/1048576 "MB" " " $9 " " $1 }' | \ sort -n -u | tail | column -t
Tool: netstat
Graph # of connections for each hosts
netstat -an | \ grep ESTABLISHED | \ awk '{print $5}' | \ awk -F: '{print $1}' | \ grep -v -e '^[[:space:]]*$' | \ sort | uniq -c | \ awk '{ printf("%s\t%s\t",$2,$1) ; for (i = 0; i < $1; i++) {printf("*")}; print "" }'
Monitor open connections for specific port including listen, count and sort it per IP
watch "netstat -plan | grep :443 | awk {'print \$5'} | cut -d: -f 1 | sort | uniq -c | sort -nk 1"Tool: rsync
Rsync remote data as root using sudo
rsync --rsync-path 'sudo rsync' username@hostname:/path/to/dir/ /local/Tool: host
Resolves the domain name (using external dns server)
Checks the domain administrator (SOA record)
host -t soa google.com 9.9.9.9
Tool: dig
Resolves the domain name (short output)
Lookup NS record for specific domain
dig @9.9.9.9 google.com NS
Query only answer section
dig google.com +nocomments +noquestion +noauthority +noadditional +nostats
Query ALL DNS Records
dig google.com ANY +noall +answer
DNS Reverse Look-up
dig -x 172.217.16.14 +short
Tool: network-other
Get all subnets for specific AS (Autonomous system)
AS="AS32934" whois -h whois.radb.net -- "-i origin ${AS}" | \ grep "^route:" | \ cut -d ":" -f2 | \ sed -e 's/^[ \t]//' | \ sort -n -t . -k 1,1 -k 2,2 -k 3,3 -k 4,4 | \ cut -d ":" -f2 | \ sed -e 's/^[ \t]/allow /' | \ sed 's/$/;/' | \ sed 's/allow */subnet -> /g'
Resolves domain name from dns.google.com with curl and jq
_dname="google.com" ; curl -s "https://dns.google.com/resolve?name=${_dname}&type=A" | jq .
Tool: awk
Remove duplicate entries in a file without sorting
Exclude multiple columns using AWK
awk '{$1=$3=""}1' filenameTool: sed
To print a specific line from a file
Remove a specific line from a file
sed -i 10d /path/to/file
# alternative (BSD): sed -i'' 10d /path/to/fileRemove a range of lines from a file
sed -i <file> -re '<start>,<end>d'
Tool: grep
Search for a "pattern" inside all files in the current directory
grep -rn "pattern" grep -RnisI "pattern" * fgrep "pattern" * -R
Remove blank lines from a file and save output to new file
grep . filename > newfilename
Except multiple patterns
grep -vE '(error|critical|warning)' filenameShow data from file without comments
grep -v ^[[:space:]]*# filename