- Joye’s “moreutils” collection – http://joeyh.name/code/moreutils/. Contains following:
- chronic: runs a command quietly unless it fails
- combine: combine the lines in two files using boolean operations
- ifdata: get network interface info without parsing ifconfig output
- ifne: run a program if the standard input is not empty
- isutf8: check if a file or standard input is utf-8
- lckdo: execute a program with a lock held
- mispipe: pipe two commands, returning the exit status of the first
- parallel: run multiple jobs at once
- pee: tee standard input to pipes
- sponge: soak up standard input and write to a file
- ts: timestamp standard input
- vidir: edit a directory in your text editor
- vipe: insert a text editor into a pipe
- zrun: automatically uncompress arguments to command
- num-utils – http://suso.suso.org/programs/num-utils/. Contains:
- average: A program for calculating the average of numbers.
- bound: Finds the boundary numbers (min and max) of input.
- interval: Shows the numeric intervals between each number in a sequence.
- normalize: Normalizes a set of numbers between 0 and 1 by default.
- numgrep: Like normal grep, but for sets of numbers.
- numprocess: Do mathmatical operations on numbers.
- numsum: Add up all the numbers.
- random: Generate a random number from a given expression.
- range: Generate a set of numbers in a range expression.
- round: Round each number according to it’s value.
- Scylla and Charybdis, Tools – http://www.scylla-charybdis.com/tool.php. Contains:
- checkrun: Program watchdog to terminate a program with starving output)
- cmpfast: Fast compare two files binary)
- count: Copy lines, shows progress)
- dbm: A little tool to access gdbm files from shell.)
- dirlist: Primitive directory lister, quicker than ls, find and echo *)
- diskus: Disk geometry checking and repair tool)
- getrealpath: Print realpath to stdout)
- histogram: Count bytes in file)
- kdmktone: Make the console beep)
- keypressed: Nonblocking, nondestructible test for waiting data on TTYs, sockets and probably pipes)
- killmem: Protect some memory against paging until you need free memory)
- lockdir: Create a directory for locking purpose)
- lockrun: Exclusively run something by placing a file lock)
- md5chk: Create md5sums for easy shell usage)
- minicron: This is a program which starts other programs after some time)
- mvatom: Move files by atomic rename instead of copy.)
- printansi: Like /bin/echo but ANSI-escapes the output)
- printargs: Like “hello world” but dumps the argc array)
- ptybuffer: daemonize interactive tty line driven programs with output history)
- runningfor: Return true until the given time periode is reached.)
- slowdown: Slowdown processes or pipes.)
- socklinger: Execute quick hack shell scripts connected to a socket.)
- sq: SQLITE3 query tool for shell usage)
- timeout: Execute a command or pipe only for a given duration.)
- timestart: Start a program N-M times in parallel by running it it each A-B seconds)
- tinohtmlparse: Simple HTML parser to extract information from HTML files by shell)
- tinoseq: An integer seq implementation)
- udevraw: Dump udev events for bash usage)
- unbuffered: Copy stdin to stdout and stderr, unbuffered)
- watcher: A Python2.6 script to watch files, pipes or Unix domain sockets)
2012.06.25
More unix tools
2011.09.30
awk
- AWK Compatibility List – http://www.shelldorado.com/articles/awkcompat.html
- comp.lang.awk FAQ – http://www.faqs.org/faqs/computer-lang/awk/faq/
- Bruce Barnett’s Awk tutorial at “The Grymoire – home for UNIX wizards” – http://www.grymoire.com/Unix/Awk.html
- The GNU Awk User’s Guide – http://www.gnu.org/s/gawk/manual/gawk.html
- AWK info at WikiPedia – http://en.wikipedia.org/wiki/AWK
Passing shell variables to AWK
Thing that works well for me:
awk '{print "'"$VARIABLE"'"}' 1 > 2 |
- “Four Ways to Pass Shell Variables in AWK” by Chi Hung Chan – http://chihungchan.blogspot.com/2009/03/four-ways-to-pass-shell-variables-in.html
- Setting a BASH environment variable directly in AWK (in an AWK one-liner) – http://stackoverflow.com/questions/3107727/setting-a-bash-environment-variable-directly-in-awk-in-an-awk-one-liner
- Invoking AWK programs – http://www.shelldorado.com/goodcoding/awkinvoke.html
Related here: Scripting languages – https://eikonal.wordpress.com/2010/06/15/awk-sed/ | Unix tricks – https://eikonal.wordpress.com/2011/02/15/unix-tricks/ | SED tricks – https://eikonal.wordpress.com/2010/10/05/sed-tricks/ | Memory of things disappearing > nmap stuff > getports.awk – https://eikonal.wordpress.com/2010/06/23/memory-of-things-disappearing-nmap-stuff-getports-awk/
2011.07.08
Auditing Unix Security
- Lynis – http://www.rootkit.nl/
- CIS Scoring tools – http://www.cisecurity.org/
- OpenSCAP – http://www.open-scap.org/
Misc
- “UNIX/Linux local audit tool” by Seán Boran (author of “IT Security Cookbook” – http://www.boran.com/security/index.html) – http://www.boran.com/security/sp/solaris/audit_tool.html – scripts made for Solaris
- Several unix auditing scrips by Marc Heuse (circa 2001) – http://web.archive.org/web/20050426143820/http://www.suse.de/~marc/audit/: audit_aix.sh, audit_hpux10.sh, audit_hpux11.sh, audit_nokia.sh, audit_oracle.sql, audit_solaris.sh, audit_suse.sh
- Also:
- expire_users.sh – a script to run as a cron job to disable user accounts which are unused for a specified time – http://web.archive.org/web/20050426075316/http://www.suse.de/~marc/expire_users.sh
- checkneverlogin.sh – A script which checks if some user accounts on the system were never used. (Note that is an updated version which skips some stuff if you are not root, the old one is now part of the seccheck package on SuSE.) – http://web.archive.org/web/20050426075316/http://www.suse.de/~marc/checkneverlogin.sh
- aix2shadow.pl – This script converts the AIX /etc/security/passwd format to a “regular” /etc/shadow format – so you can use an unshadow software and then crack passwords. – http://web.archive.org/web/20050426075316/http://www.suse.de/~marc/aix2shadow.pl
- seccheck – http://freshmeat.net/projects/seccheck/ – Seccheck is a feature rich, modular, host-level security checker for Solaris 10. Easily expandable with customised modules, Seccheck produces highly detailed reports based around known and published security best-practices and guidelines. It also produces recommendations on how to fix flagged security issues.
2011.05.03
Password expiration script for Unix
Sources:
- http://www.linuxmail.info/files/checkpwexpire
- “Unix vs freeBSD” – http://kindakecil.wordpress.com/unix-vs-freebsd/
- “Check password age” (The UNIX and Linux Forums; 2006.12.18) – http://www.unix.com/shell-programming-scripting/33854-check-password-age.html
Code
#!/bin/sh # # Goran Cvetanoski - 19/12/2006 # # pwage # # This script works out the time left before a password expires # # It will send a reminder email 10 days and 3 days before the password # will expire. The email will go to unix.admin@mydomain.com.au unless an # alternate email address is specified. An email will also be sent if a # password has expired. # # The following command will send results to unix.admin@mydomain.com.au # pwage oracle # # Specify an alternate email address if you would like the results to be # sent to a different email address. # ie: # pwage oracle oracledba@mydomain.com.au # # # CHANGE LOG # ========================================================================= # 19/12/2006 - Goran Base script created # 05/08/2009 - Ricky Smith added code to check each user # LOG=/tmp/pwage.log DASHES="-----------------------------" show() { echo "$DASHES $1 $DASHES" >> $LOG shift eval "$@" >> $LOG echo "" >> $LOG } SendMail() { cat $LOG | mailx -s "$1" "$2" } reminder () { echo "Date: `date`" echo "" echo "Please change your password within the next $EXPIRE days" } expired () { echo "Date: `date`" echo "" echo "The password for $USER has expired" echo "$USER last changed their password on $LSTCNG" echo "The maximum age for the password is $MAX days" echo "and it has expired $EXPIRE days ago" } CheckUser() { USER=$1 EMAIL=$2 CURRENT_EPOCH=`grep $USER /etc/shadow | cut -d: -f3` if [ "$CURRENT_EPOCH" = "" ]; then return fi # Find the epoch time since the user's password was last changed EPOCH=`perl -e 'print int(time/(60*60*24))'` # Compute the age of the user's password AGE=`echo $EPOCH - $CURRENT_EPOCH | bc` # Compute and display the number of days until password expiration MAX=`grep $USER /etc/shadow | cut -d: -f5` if [ "$MAX" = "" ]; then return fi EXPIRE=`echo $MAX - $AGE | bc` CHANGE=`echo $CURRENT_EPOCH + 1 | bc` LSTCNG="`perl -e 'print scalar localtime('$CHANGE' * 24 *3600);'`" WARN=`grep $USER /etc/shadow | cut -d: -f6` if [ "$WARN" = "" ]; then WARN=0 fi if [ "$EXPIRE" -le "$WARN" ]; then show "R E M I N D E R" reminder SendMail "$USER Password Info On `uname -n`" "$EMAIL" elif [ "$EXPIRE" -lt 0 ]; then show "E X P I R E D" expired SendMail "WARNING: $USER Password Expired On `uname -n`" "$EMAIL" fi } # Main Code domain=$1 if [ "$domain" = "" ]; then domain=$(dnsdomainname) fi minuid=$2 if [ "$minuid" = "" ]; then minuid=500 fi IFS=':' while read user pass uid gid full home shell do if [ $uid -ge $minuid ]; then cat /dev/null > $LOG CheckUser $user "\"$full\" " fi done </etc/passwd |
2011.05.02
RPM (Red Hat Package Manager)
- “Maximum RPM: Taking the Red Hat Package Manager to the Limit” – http://www.rpm.org/max-rpm/index.html
Misc
- Interogating packages:
- rpm -qa = lists installed packages
- rpm -redhatrequires cron
- rpm -redhatprovides cron
- rpm -qpR cron *
- Installing packages: -i = –install
- rpm -i package1 package2 … = rpm –install package1 package2 …
- rpm -i -nodeps package1 package2 … = install packages without their dependencies
- rpm -U package1 package2 … = installs the latest versions of packages = updates the packages
- Remove packages: -e
- rpm -e package1 package2 …
- rpm -e –force package1 … = forces uninstallation without regards for remaining dependency constraints
- rpm -e –nodeps package1 …
Sites
Antimalware for Unix
- “Tasmanian Dept. of Education Wants Anti-Virus for Linux, OS X” 9SlashDot; 2011.05.020 – http://apple.slashdot.org/story/11/05/02/0539228/Tasmanian-Dept-of-Education-Wants-Anti-Virus-for-Linux-OS-X
- “One of Australia’s largest government technology buyers, the Tasmanian Department of Education, has gone to market for a security vendor to supply anti-virus software for its 40,000-odd desktop PCs and laptops, as well as servers. But the department’s not just running Windows — it runs Mac OS X and Linux as well, and has requested that whatever solution it buys must be able to run on those platforms as well. But have we reached the stage were Mac OS X and Linux even need third-party security software? It seems like most Mac and Linux users don’t run it.”
- “Tassie education dept wants Mac, Linux anti-virus” by Renai LeMay (2011.05.02) – http://delimiter.com.au/2011/05/02/tassie-education-dept-wants-mac-linux-anti-virus/
2011.04.04
lastlog
- freq – lastlog analyzer – http://www.bangmoney.org/projects/freq/
- Formatting and Printing Lastlog – http://www.hcidata.info/lastlog.htm
Unix pages (at this blog)
- Unix system administration – https://eikonal.wordpress.com/2011/03/02/unix-system-administration/
- Unix tricks – https://eikonal.wordpress.com/2011/02/15/unix-tricks/
- Unix hardening – https://eikonal.wordpress.com/2010/04/20/unix-hardening/
- Logging – https://eikonal.wordpress.com/2010/04/13/logging/
- PAM – https://eikonal.wordpress.com/2011/03/11/pam/
- SSH, OpenSSH – https://eikonal.wordpress.com/2010/12/16/ssh-openssh/
- Unix “script” command – https://eikonal.wordpress.com/2010/12/02/unix-script-command/
- vi editor – https://eikonal.wordpress.com/2010/11/15/vi-editor/
- sed tricks – https://eikonal.wordpress.com/2010/10/05/sed-tricks/
- Nmap options, switches and uses – https://eikonal.wordpress.com/2010/09/20/nmap-options-swtiches-and-uses/
- netcat – https://eikonal.wordpress.com/2010/07/12/netcat/
- Command line based text replace – https://eikonal.wordpress.com/2010/07/13/command-line-based-text-replace/
- Archiving and compression tools – https://eikonal.wordpress.com/2010/08/03/archiving-and-compression-tools/
- Cygwin – https://eikonal.wordpress.com/2010/07/12/cygwin-stuff/
- googlecl – https://eikonal.wordpress.com/2010/07/06/googlecl/
- John the Ripper – https://eikonal.wordpress.com/2010/05/25/john-the-ripper/
2011.03.11
PAM (Pluggable Authentication Modules)
Articles
- “Introduction to PAM” by Jennifer Vesperman (O’Reilly Linux DevCenter; 2001.09.27) – http://linuxdevcenter.com/lpt/a/1120
- “PAM Modules” by Jennifer Vesperman (O’Reilly Linux DevCenter; 2001.10.05) – http://linuxdevcenter.com/pub/a/linux/2001/10/05/PamModules.html
- The primary distribution site for the Linux-PAM (Pluggable Authentication Modules for Linux) project – http://www.kernel.org/pub/linux/libs/pam/
- “Pluggable Authentication Modules” by Dag-Erling Smørgrav (2003) – http://www.freebsd.org/doc/en_US.ISO8859-1/articles/pam/
- “PAM Documentation” – http://uw714doc.sco.com/en/SEC_pam/pamintro.html
- “Chapter 18. Pluggable Authentication Modules (PAM)” (NetBSD documentation) – http://www.netbsd.org/docs/guide/en/chap-pam.html
- PAM at WikiPedia – http://en.wikipedia.org/wiki/Pluggable_Authentication_Modules
- How to login as ROOT (Fedora forums) – http://forums.fedoraforum.org/showthread.php?t=209638
Roles of PAM files
- /etc/pam.conf – all-in-one configuration file for early versions of PAM. It may still be used in some modern versions.
- /etc/pam.d/ – directory containing configurations files for each of separately configured program
- /etc/pam.d/other – the default config file regulating all files that do not have their own separate PAM config file
- /etc/pam.d/login
- /etc/pam.d/system-auth
- /etc/pam.d/sshd
- “16.6. Configuration of the /etc/pam.d/ssh file” (Securing and Optimizing Linux: RedHat Edition -A Hands on Guide; Chapter 16. Software -Securties(commercial)) – http://www.linuxtopia.org/online_books/linux_system_administration/securing_and_optimizing_linux/chap16sec132.html | also at http://www.faqs.org/docs/securing/chap16sec132.html
- Example (from the above link):
#%PAM-1.0 auth required /lib/security/pam_pwdb.so shadow auth required /lib/security/pam_nologin.so account required /lib/security/pam_pwdb.so password required /lib/security/pam_cracklib.so password required /lib/security/pam_pwdb.so use_authtok nullok md5 shadow session required /lib/security/pam_pwdb.so
- Example (from http://quark.humbug.org.au/publications/ldap/system_auth/sage-au/system_auth.html):
#%PAM-1.0 auth sufficient /lib/security/pam_ldap.so auth required /lib/security/pam_pwdb.so shadow auth required /lib/security/pam_nologin.so account sufficient /lib/security/pam_ldap.so account required /lib/security/pam_pwdb.so password required /lib/security/pam_cracklib.so password sufficient /lib/security/pam_ldap.so password required /lib/security/pam_pwdb.so shadow nullok use_authtok session sufficient /lib/security/pam_ldap.so session required /lib/security/pam_pwdb.so
- “Linux PAM configuration that allows or deny login via the sshd server” by LinuxTitli (UnixCraft; 2006.07.30) – http://www.cyberciti.biz/tips/linux-pam-configuration-that-allows-or-deny-login-via-the-sshd-server.html
- /etc/pam.d/su
- /etc/pam.d/gdm – the GNOME Display Manager PAM file.
- Example (from http://ubuntuforums.org/showthread.php?t=1506759):
#%PAM-1.0 auth requisite pam_nologin.so auth required pam_env.so readenv=1 auth required pam_env.so readenv=1 envfile=/etc/default/locale auth sufficient pam_succeed_if.so user ingroup nopasswdlogin @include common-auth auth optional pam_gnome_keyring.so @include common-account session [success=ok ignore=ignore module_unknown=ignore default=bad] pam_selinux.so close session required pam_limits.so @include common-session session [success=ok ignore=ignore module_unknown=ignore default=bad] pam_selinux.so open session optional pam_gnome_keyring.so auto_start @include common-password
- Example (from http://ubuntuforums.org/showthread.php?t=1506759):
Syntax of config files
Each line has format:
module-type control-flag module-path arguments
PAM modules
- pam_deny.so module –
- pam_permit.so module –
- pam_warn.so module – used to interface to syslog
2011.03.02
Unix system administration
- “The Decline and Fall of System Administration” (SlashDot; 2011.03.02) – http://tech.slashdot.org/story/11/03/02/132215/The-Decline-and-Fall-of-System-Administration
- “Deep End’s Paul Venezia questions whether server virtualization technologies are contributing to the decline of real server administration skills (http://infoworld.com/d/data-center/the-decline-and-fall-system-administration-375), as more and more sysadmins argue in favor of re-imaging as a solution to Unix server woes (http://tech.slashdot.org/story/11/02/21/1734259/Why-You-Shouldnt-Reboot-Unix-Servers). ‘This has always been the (many times undeserved) joke about clueless Windows admins: They have a small arsenal of possible fixes, and once they’ve exhausted the supply, they punt and rebuild the server from scratch rather than dig deeper. On the Unix side of the house, that concept has been met with derision since the dawn of time, but as Linux has moved into the mainstream — and the number of marginal Linux admins has grown — those ideas are suddenly somehow rational.'”
- Paul Venezia (InfoWorld) sysadmin articles:
- “The decline and fall of system administration” (2011.02.28) – http://infoworld.com/d/data-center/the-decline-and-fall-system-administration-375
- Virtualization makes it all too easy to spawn new instances rather than figuring out what went wrong. Is this the end of Unix best practices?
- “When in doubt, reboot? Not Unix boxes” (2011.02.21) – http://infoworld.com/t/unix/when-in-doubt-reboot-not-unix-boxes-061
- Rebooting Windows boxes is a way of life, but rebooting by default can you get you nowhere fast when running Unix.
- “Nine traits of the veteran Unix admin” (I2011.02.21) – http://infoworld.com/t/unix/nine-traits-the-veteran-unix-admin-276
- Follow this field guide if you want to understand the rare and elusive hard-core Unix geek.
- “The decline and fall of system administration” (2011.02.28) – http://infoworld.com/d/data-center/the-decline-and-fall-system-administration-375
AIX
- AIX 5L Version 5.3: Commands Reference, Volume 5, s – u (SC23-4892-07; 2010.04) – http://publib.boulder.ibm.com/infocenter/pseries/v5r3/topic/com.ibm.aix.cmds/doc/aixcmds5/aixcmds5.htm
- AIX Tips – http://www.ee.pw.edu.pl/~pileckip/aix/
- “AIX commands you should not leave home without” by Shiv Dutta (IBM developerWorks; 2006.10.24) – http://www.ibm.com/developerworks/aix/library/au-dutta_cmds.html
- “AIX: Commands” by John Flecther – http://www.unixguide.net/ibm/aixcommands.shtml
- “AIX Command Crib Sheet” (2001.01.29) – http://www.unixguide.net/ibm/ibmcribsheet.shtml | http://www.mort.level5.net/johnr/howto/aix.txt
- AIX COMMAND GUIDE – http://bigcalm.tripod.com/aix/handycommands.htm
- Useful AIX commands – http://stromberg.dnsalias.org/~strombrg/Useful-AIX-commands.html
- IBM’s developerWorks collection of articles on AIX and Unix – http://www.ibm.com/developerworks/aix/
- IBM’s AIX and UNIX Forums – http://www.ibm.com/developerworks/forums/dw_auforums.jspa
Specific tasks:
- Getting OS version:
- uname command – http://publib.boulder.ibm.com/infocenter/pseries/v5r3/topic/com.ibm.aix.cmds/doc/aixcmds5/uname.htm
- “Displaying the operating system name (uname command)” – http://publib.boulder.ibm.com/infocenter/pseries/v5r3/topic/com.ibm.aix.baseadmn/doc/baseadmndita/displaying_os_name.htm
- “AIX Tip of the Week: Using uname to Identify RS/6000 Model” (AIX Tips; 1999.02) – http://www.ee.pw.edu.pl/~pileckip/aix/lsconfig.htm
- “AIX: How To Determine Its Detail Version” (SysAdmin’s Diary; 2008.02.24) – http://blog.irwan.name/?p=438
- Use uname -a and oslevel
From article “AIX commands you should not leave home without” (fully referenced above):
- uname -a Displays the system name, nodename, version, machine ID.
- uname -p Displays the chip type of the system. For example, PowerPC./li>
- uname -r Displays the release number of the operating system./li>
- uname -s Displays the system name. For example, AIX./li>
- uname -n Displays the name of the node./li>
- uname -M Displays the system model name. For example, IBM, 9114-275./li>
- uname -v Displays the operating system version./li>
- uname -m Displays the machine ID number of the hardware running the system./li>
- uname -u Displays the system ID number./li>
- Finding open files:
- “Finding open files with lsof” by Sean A. Walberg (IBM developerWorks; 2006.07.25) – http://www.ibm.com/developerworks/aix/library/au-lsof.html
- lsof information:
- lsof source – ftp://lsof.itap.purdue.edu/pub/tools/unix/lsof/
- lsof man page – http://www.netadmintools.com/html/lsof.man.html
Related here: Unix tricks – https://eikonal.wordpress.com/2011/02/15/unix-tricks/
2011.03.01
MS Windows Registry transversal by Cygwin
This may be an old thing, but it is new to me. Today I accidentally discovered that from inside Cygwin (i.e.e the Bash shell) one can transverse the Windows Registry. For example, one can list the content of HKEY_CLASSES_ROOT hive by simply typing:
ls -la /proc/registry/HKEY_CLASSES_ROOT/ |
with partial output:
total 0 dr-xr-x--- 5 Administrators SYSTEM 0 Apr 26 2010 * dr-xr-x--- 7342 Administrators SYSTEM 0 Feb 18 10:32 . dr-xr-xr-x 8 thisuser mkgroup-l-d 0 Mar 1 12:02 .. dr-xr-x--- 3 Administrators SYSTEM 0 Feb 16 2010 .123 dr-xr-x--- 2 Administrators SYSTEM 0 Feb 3 2009 .323 dr-xr-x--- 3 Administrators SYSTEM 0 Feb 3 2009 .386 dr-xr-x--- 2 Administrators SYSTEM 0 Nov 17 2009 .3g2 dr-xr-x--- 2 Administrators SYSTEM 0 Nov 17 2009 .3gp dr-x------ 2 thisuser ???????? 0 Jun 7 2010 .5vw dr-x------ 4 thisuser ???????? 0 Nov 17 2009 .A1wish dr-x------ 2 ???????? None 0 Aug 5 2009 .ARC dr-x------ 2 ???????? None 0 Aug 5 2009 .ARJ dr-xr-x--- 3 Administrators SYSTEM 0 Feb 4 2009 .AddIn dr-x------ 3 ???????? ???????? 0 Aug 7 2009 .AudioCD dr-x------ 2 ???????? None 0 Aug 5 2009 .B64 dr-x------ 2 ???????? None 0 Aug 5 2009 .BHX dr-x------ 2 ???????? ???????? 0 Aug 7 2009 .CTT dr-xr-x--- 3 Administrators SYSTEM 0 Feb 3 2009 .DeskLink dr-x------ 3 ???????? ???????? 0 Aug 7 2009 .Folder dr-xr-x--- 2 Administrators SYSTEM 0 Feb 3 2009 .JSE dr-x------ 2 ???????? None 0 Aug 5 2009 .LZH dr-xr-x--- 3 Administrators SYSTEM 0 Feb 3 2009 .MAPIMail dr-x------ 2 ???????? None 0 Aug 5 2009 .MIM ... |
Such feature is available in Microsoft’s PowerShell for several years now, but I did not know that this exists in the Cygwin’s Bash. When was this feature incorporated in the Cygwin? Was it inspired by PowerShell?
Also, the same feature works for /proc/registry32/ and /proc/registry64/
Related: Cygwin stuff – https://eikonal.wordpress.com/2010/07/12/cygwin-stuff//a> | MS Windows Registry transversal by Cygwin – https://eikonal.wordpress.com/2011/03/01/registry-transversal-by-cygwin | Unix tricks – https://eikonal.wordpress.com/2011/02/15/unix-tricks/
2011.02.15
2011.02.10
SUDO
- “Quick HOWTO : Ch09 : Linux Users and Sudo” (LHN = Linux Home Networking)- http://www.linuxhomenetworking.com/wiki/index.php/Quick_HOWTO_:_Ch09_:_Linux_Users_and_Sudo
- Sudo home(s): http://www.sudo.ws/sudo/ | http://www.gratisoft.us/sudo/sudo.html:
- Sudo man – http://www.sudo.ws/sudo/sudo.man.html | http://www.gratisoft.us/sudo/sudo.man.html
- Sudoers Manual – http://www.sudo.ws/sudo/sudoers.man.html | http://www.gratisoft.us/sudo/sudoers.man.html
- Sudoers samples – http://www.sudo.ws/sudo/sample.sudoers
- Tools For Use With Sudo – http://www.gratisoft.us/sudo/tools.html:
- sudolog-usage – http://www.komar.org/pres/sudo-tools/ – summarizes the sudo log file (from syslog).
- sudoers-lint – http://www.komar.org/pres/sudo-tools/ – is a sudoers file sanity checker that finds orphaned and unused entries to help keep your sudoers file squeaky clean.
- sudoscript – http://egbok.com/sudoscript – uses the script(1) command to log a sudo session. This can be useful for people who insist on running shells under sudo.
- sudosh – http://sourceforge.net/projects/sudosh/ – is a filter that allows the use of a root shell while logging the commands typed via syslog.
- Sudo Alternatives – http://www.gratisoft.us/sudo/other.html
- WordPress blogs on sudo: http://en.wordpress.com/tag/sudoers/ | http://en.wordpress.com/tag/sudo/
- “Super User Access in Linux” (Mahesh’s programming rants; 2011.01.03) – http://knmaheshy2k.wordpress.com/2011/01/03/super-user-access-in-linux/
- “when to su instead of sudo” by Albert Decatur (history | less; 2010.12.12) – http://zosim26.wordpress.com/2010/12/12/when-to-su-instead-of-sudo/
- “Things Every New Linux User Should Know” – http://geekum.wordpress.com/2011/01/15/things-every-new-linux-user-should-know/
- “Sudo -g privilege escalation (CVE-2011-0010)” (Cатсн²² (in)sесuяitу; 2011.01.13) – http://blog.c22.cc/2011/01/13/sudo-g-privilege-escalation-cve-2011-0010/
2011.01.13
Jaibreaking
- GeoHot jailbreaking news – http://www.geohot.us/
- George Hotz – http://en.wikipedia.org/wiki/George_Hotz
News:
- “Sony Asks Court to Remove PlayStation 3 Jailbreak From Net” – by David Kravets (Wired; 2010.07.27) – http://www.wired.com/threatlevel/2011/01/playstation3-hack-lawsuit/
- “Jailbreaking iPhone Legal, U.S. Government Says” – by David Kravets (ABC News; 2010.07.27) – http://abcnews.go.com/Technology/us-government-jailbreaking-iphone-legal/story?id=11254253
- “Geohot Jailbreaks PS3 with Custom Firmware 3.55 (Video)” (GeoHot; 2011.01.08) – http://www.geohot.us/2011/01/geohot-jailbreaks-ps3-with-custom.html
2010.12.16
SSH, OpenSSH
- “Top 20 OpenSSH Server Best Security Practices” at UnixCraft – http://www.cyberciti.biz/tips/linux-unix-bsd-openssh-server-best-practices.html
- “How do I permit specific users SSH access?” (SoftLayer) – http://knowledgelayer.softlayer.com/questions/295/How+do+I+permit+specific+users+SSH+access%3F
- “Six things I wish Mom told me (about ssh)” – http://blog.ksplice.com/2010/08/six-things-i-wish-mom-told-me-about-ssh/
- “ProxyCommand – SSH Key on Proxing Machine” (comp.security.ssh; 2009) – http://groups.google.com/group/comp.security.ssh/browse_thread/thread/1e5ae560420b9d12
- “Quick HOWTO: Ch17 : Secure Remote Logins and File Copying” (LHN) – http://www.linuxhomenetworking.com/wiki/index.php/Quick_HOWTO_:_Ch17_:_Secure_Remote_Logins_and_File_Copying
- “Securing your ssh server” (Racker Hacker blog) – http://rackerhacker.com/2010/10/12/securing-your-ssh-server/
- “How to connect to a non standard SSH port in the Mac terminal” – http://chimac.net/2011/01/08/how-to-connect-to-a-non-standard-ssh-port-in-the-mac-terminal/
- ssh -p 12345 remoteUsername@address.of.remote.system
- “Best Practices to secure a OPENSSH/SSH Server” – http://teknoteknik.wordpress.com/2010/07/06/best-practices-to-secure-a-opensshssh-server/
- “Quick and dirty manual compile of OpenSSH on CentOS 5” (#!/bin/blog; 2008.04.06) – http://binblog.info/2008/04/06/quick-and-dirty-manual-compile-of-openssh-on-centos-5/
- “Using the SSH agent from daemon processes” (#!/bin/blog; 2008.12.31) – http://binblog.info/2008/12/31/using-the-ssh-agent-from-daemon-processes/
- “OpenSSH: Going flexible with forced commands” (#!/bin/blog; 2008.10.20) – http://binblog.info/2008/10/20/openssh-going-flexible-with-forced-commands/
- Secure Copy (SCP) (WikiPedia) – http://en.wikipedia.org/wiki/Secure_copy
- SSH tricks at SuperUser – http://superuser.com/questions/tagged/telnet+ssh
- related:
- Telnet tricks at SuperUser: Telnet tricks – http://superuser.com/questions/tagged/telnet | PuTTY tricks – http://superuser.com/questions/tagged/putty | Tunnels – http://superuser.com/questions/tagged/putty+tunnel
SSHFS (SSH FileSystem)
- “SSH Filesystem” – http://fuse.sourceforge.net/sshfs.html
- http://en.wikipedia.org/wiki/SSHFS
- https://help.ubuntu.com/community/SSHFS
- SHSFS FAQ – http://sourceforge.net/apps/mediawiki/fuse/index.php?title=SshfsFaq
Related:
- FISH (Files transferred over shell) protocol (WikiPedia) – http://en.wikipedia.org/wiki/Files_transferred_over_shell_protocol
- FTPFS – http://en.wikipedia.org/wiki/FTPFS
- WebDrive – http://en.wikipedia.org/wiki/WebDrive | http://www.webdrive.com/products/webdrive/
- FTPDrive – http://en.wikipedia.org/wiki/FTPDrive | http://www.killprog.com/fdrve.html
Authentication via public keys
- ‘OpenSSH key management” by Daniel Robbins (IBM):
- OpenSSH key management, Part 1 (2001.07.01) – http://www.ibm.com/developerworks/library/l-keyc.html
- OpenSSH key management, Part 2 (2001.09.01) – http://www.ibm.com/developerworks/library/l-keyc2/
- OpenSSH key management, Part 3 (2002.02.01) – http://www.ibm.com/developerworks/library/l-keyc3/
- “SSH with authentication key instead of password” – http://www.debian-administration.org/articles/530
- “Public key authentication with ssh” – http://www.linuxquestions.org/linux/answers/Networking/Public_key_authentication_with_ssh
- OpenSSH Public Key Authentication – http://web.archive.org/web/20070418231823/http://sial.org/howto/openssh/publickey-auth/
- “SSH Public Key (/w RSA) Authentication and SSH Tunneling – Part 1” (ipsure; 2010.02.03) – http://www.ipsure.com/blog/2010/ssh-public-key-w-rsa-authentication-and-ssh-tunneling-part-1/
SFTP
- “Chrooted SFTP with Public Key Authentication” (IPSURE; 2010.12.10) – http://www.ipsure.com/blog/2010/chrooted-sftp-with-public-key-authentication/
- How to mount SFTP accesses – http://wiki.gilug.org/index.php/How_to_mount_SFTP_accesses
- “OpenSSH chrooted SFTP (e.g. for Webhosting)” (#!/bin/blog; 2008.04.06) – http://binblog.info/2008/04/06/openssh-chrooted-sftp-eg-for-webhosting/
FTPS vs SFTP
- “FTPS vs. SFTP: What to Choose” by Eugene Mayevski (CodeGuru; 2007.10.11) – http://www.codeguru.com/csharp/.net/net_general/internet/article.php/c14329
- “FTPS vs. SFTP, once and for all” (#!/bin/blog; 2010.10.12) – http://binblog.info/2010/10/12/ftps-vs-sftp-once-and-for-all/
- FTPS (WikiPedia) – http://en.wikipedia.org/wiki/FTPS
- SSH File Transfer Protocol (Wikipedia) – http://en.wikipedia.org/wiki/SSH_file_transfer_protocol
- “Setup groups and users in FileZilla Server and connect with ftpes” (Banbika’s Blog; 2010.08.28) – http://banbika.wordpress.com/2010/08/28/setup-groups-and-users-in-filezilla-server-and-connect-with-ftpes/
- “Install and Configure FTP Secure (FTPS) or FTP-SSL using FileZilla” (Banbika’s Blog; 2010.08.24) – http://banbika.wordpress.com/2010/08/24/install-and-configure-ftp-secure-ftps-or-ftp-ssl-using-filezilla/
Using SCP
- Example syntax for Secure Copy (scp) – http://www.hypexr.org/linux_scp_help.php
- Copy the file “foobar.txt” from a remote host to the local host: $ scp your_username@remotehost.edu:foobar.txt /some/local/directory
- Copy the file “foobar.txt” from the local host to a remote host: $ scp foobar.txt your_username@remotehost.edu:/some/remote/directory
- Copy the directory “foo” from the local host to a remote host’s directory “bar”: $ scp -r foo your_username@remotehost.edu:/some/remote/directory/bar
- Copy the file “foobar.txt” from remote host “rh1.edu” to remote host “rh2.edu”: $ scp your_username@rh1.edu:/some/remote/directory/foobar.txt \
your_username@rh2.edu:/some/remote/directory/ - Copying the files “foo.txt” and “bar.txt” from the local host to your home directory on the remote host: $ scp foo.txt bar.txt your_username@remotehost.edu:~
- Copy multiple files from the remote host to your current directory on the local host: $ scp your_username@remotehost.edu:/some/remote/directory/\{a,b,c\}. Also:: $ scp your_username@remotehost.edu:~/\{foo.txt,bar.txt\} .
Use of Expect with SSH suite applications
Password-less SFTP
Establish the SFTP connection to the system AAAA where the user account BBBB has password CCCC, and go to the directory DDDD, all without being prompted to enter the password:
sftpToAAAA.expect |
#!/bin/expect # sftpToAAAA.expect spawn sftp BBBB@AAAA expect "password" { sleep 1 send "CCCC\n" } send "cd DDDD\n" interact |
All normal warning on the danger of hard-wiring the password into scripts are in place here.
Password-less SCP
Use the SCP to upload connect system AAAA with user account BBBB (that has password CCCC), and upload the file EEEE to the directory DDDD, all without being prompted to enter the password:
UploadEEEEtoAAAA.expect |
#!/bin/expect spawn scp EEEE BBBB@AAAA:DDDD/EEEE expect "password" { send "CCCC\n" } |
More
- Expect and SSH – http://rootprompt.org/article.php3?article=9187
- expect and ssh – http://www.unix.com/unix-advanced-expert-users/50467-expect-ssh.html
- Expect and SSH in Cygwin – http://forums.devshed.com/unix-help-35/expect-and-ssh-in-cygwin-176556.html
2010.12.02
Unix “script” command
- Unix script command – http://www.softpanorama.org/Utilities/script.shtml
- Sample of What You’ll See Using the Unix script Command – http://ultra.pr.erau.edu/~jaffem/classes/cs125/script.htm
- The Unix script command – http://www-users.cs.umn.edu/~skim/cs1901/script.html
- Help in Creating a Script File on a Unix System – http://myweb.stedwards.edu/laurab/help/scripthelp.html
Implementation of the same functionality in Expect
Cygwin does not come with the “script” commands, so following implementation in Expect can do the same thing:
script.expect |
#!/bin/expect spawn /bin/bash if {[lindex $argv 0]==""} { log_file SESSION.txt } else { log_file [lindex $argv 0] } interact |
2010.11.15
vi editor
- “vi tips and tricks: Ten cool commands sure to impress your friends” by Martin Wicks (IBM DeveloperWorks, 2010.07.27) – http://www.ibm.com/developerworks/aix/library/au-vitips.html?ca=dgr-lnxw06VITips10dth-AIX
- “UNIX tips and tricks for a new user, Part 2: The vi text editor” by Tim McIntire (IBM DeveloperWorks, 2006.11.07) – http://www.ibm.com/developerworks/aix/tutorials/au-unixtips2/
- “vi intro — the cheat sheet method” by Daniel Robbins (IBM DeveloperWorks, 2006.11.15) – http://www.ibm.com/developerworks/linux/tutorials/l-vi/
- “Learn Linux, 101: File editing with vi” by Ian Shields (IBM DeveloperWorks, 2010.02.10) – http://www.ibm.com/developerworks/linux/library/l-lpic1-v3-103-8/
- “The vi Lovers Home Page” – http://thomer.com//vi/vi.html
- The vim home page – http://www.vim.org/
- “VI Text Editor Reference Sheet” by Roger Murray (1994.10.19) – http://www.its.caltech.edu/info/Applications/vi.shtml
- “An Introduction to Display Editing with Vi” by William Joy and Mark Horton – http://docs.freebsd.org/44doc/usd/12.vi/paper.html
- Vim – Vi IMproved – http://www.moolenaar.net/vim.html
2010.08.07
Samba/SMB/CIFS security links
- Samba project: http://us1.samba.org/samba/
- Samba Security Releases – http://us1.samba.org/samba/history/security.html
- CIFS VFS – Advanced Common Internet File System for Linux – http://linux-cifs.samba.org/
- Using SAMBA/CIFS to access Windows Shares – http://webscript.princeton.edu/~pug/faqwiki/index.php?title=Using_SAMBA/CIFS_to_access_Windows_Shares
2010.07.13
Command line based text replace
sed
-
sed 's/Mark Monre/Marc Monroe/' 1.txt > 2.txt
-
find ./* -type f -exec sed -i 's///g' {} \;
The “replace” command
- Syntax:
replace OLD-STRING NEW-STRING OUTPUT-FILE
- Example:
$ replace UNIX Linux newfile
- Example:
$ cat /etc/passwd | replace : '|'
- Partial support for regular expressions: \^ – matches start of line, and $ matches end of line.
- Example: replace all IP address 192.168.1.2 start of line:
$ replace \^192.168.1.2 192.168.5.10 newfile
- a bash script, ‘fixer.sh’
#!/bin/bash replace CHANGEFROM CHANGETO $1.tmp rm $1 mv $1.tmp $1
now run this command line:
$ grep CHANGEFROM |cut -d':' -f1 |xargs -n 1 fixer.sh
the results is that all files in the directory (or whatever you grep for) will be changed automagically.
just make sure the grep doesn’t include the fixer script itself, or it will die half-way through changing when execute permissions are reset!
Perl
- Perl Pie:
perl -p -i -e ’s/hello/goodbye/g’ textfile.txt
- http://www.debian-administration.org/articles/298 has a fine article and discussion on Perl Pie.
-
perl -p -i -e ’s/|00000000.00|/||/g’ myfile.txt
Sources:
- How do I replace text string in many files at once? – http://www.cyberciti.biz/tips/how-do-i-replace-text-string-in-many-files-at-once.html
Related: Regular expressions – https://eikonal.wordpress.com/2010/04/02/regular-expressions/ | Perl online – https://eikonal.wordpress.com/2010/02/15/perl-online/
2010.07.12
Cygwin stuff
Various Cygwin links
- Home – http://www.cygwin.com/ | http://cygwin.org/
- FAQ – http://www.cygwin.com/faq/
- Mirrors – http://www.cygwin.com/mirrors.html
- Cygwin/X – http://x.cygwin.com/
- Cygwin info at RedHat – http://www.redhat.com/services/custom/cygwin/
- project pages at Freshmeat – http://freshmeat.net/projects/cygwin/
- Cygwin infor at Softpanorama – http://www.softpanorama.org/Unixification/cygwin.shtml
- Windows Unix compatibility – http://techdir.rutgers.edu/winunix.html
- at WikiPedia – http://en.wikipedia.org/wiki/Cygwin
Portable Cygwin
- “Cygwin on portable storage devices” by Ali Devin Sezer (2005.02.05) – http://sites.google.com/site/devinsezer/Home/software/portable-cygwin
- “A Guide to Building a Portable Cygwin” (at Fuzzix blog; not clear when)- http://blogs.linux.ie/fuzzbucket/2008/02/23/cygwin/
- (2008): http://www.symbiosoft.net/cygwinportable | http://sourceforge.net/projects/cygwinportable/files/ | http://portableapps.com/node/14482
- “Cygwin Portable – (some) Linux power on a USB for Windows” (Bits of Wizardry; 2010.01.29) – http://bitsofwizardry.wordpress.com/2010/01/29/cygwin-portable-some-linux-power-on-a-usb-for-windows/
- “How to make your own portable Cygwin on CD” by fergus bonhard (2003.07.19) – http://www.cygwin.com/ml/cygwin/2003-07/msg01117.html
- “A CD based on Cygwin for X Windows forwarding via ssh” – http://xlivecd.indiana.edu/
Tricks
Setting effective HOME to your Windows “My Documents” folder.
- Start a Cygwin shell
- You should be located in your default directory which inside Cygwin looks like /home/thisuser, but from the Windows standpoint could be something like c:\cygwin\home\thisuser.
- Type ln -s /cygdrive/c/Documents and Settings/thisuser/My Documents/ ./docs
- Edit .profile and .bashrc to put at their ends following lines:
export HOME=./docs cd $HOME
- that is it. Restart Cygwin. You should now be in the /home/thisuser/docs/ which is the same place as Windows’ c:\Documents and Settings\thisuser\My Documents/
Related: MS Windows Registry transversal by Cygwin – https://eikonal.wordpress.com/2011/03/01/registry-transversal-by-cygwin | Unix tricks – https://eikonal.wordpress.com/2011/02/15/unix-tricks/