More unix tools

• Joye’s “moreutils” collection – http://joeyh.name/code/moreutils/. Contains following:

  • chronic: runs a command quietly unless it fails
  • combine: combine the lines in two files using boolean operations
  • ifdata: get network interface info without parsing ifconfig output
  • ifne: run a program if the standard input is not empty
  • isutf8: check if a file or standard input is utf-8
  • lckdo: execute a program with a lock held
  • mispipe: pipe two commands, returning the exit status of the first
  • parallel: run multiple jobs at once
  • pee: tee standard input to pipes
  • sponge: soak up standard input and write to a file
  • ts: timestamp standard input
  • vidir: edit a directory in your text editor
  • vipe: insert a text editor into a pipe
  • zrun: automatically uncompress arguments to command

• num-utils – http://suso.suso.org/programs/num-utils/. Contains:

  • average: A program for calculating the average of numbers.
  • bound: Finds the boundary numbers (min and max) of input.
  • interval: Shows the numeric intervals between each number in a sequence.
  • normalize: Normalizes a set of numbers between 0 and 1 by default.
  • numgrep: Like normal grep, but for sets of numbers.
  • numprocess: Do mathmatical operations on numbers.
  • numsum: Add up all the numbers.
  • random: Generate a random number from a given expression.
  • range: Generate a set of numbers in a range expression.
  • round: Round each number according to it’s value.

• Scylla and Charybdis, Tools – http://www.scylla-charybdis.com/tool.php. Contains:

  • checkrun: Program watchdog to terminate a program with starving output)
  • cmpfast: Fast compare two files binary)
  • count: Copy lines, shows progress)
  • dbm: A little tool to access gdbm files from shell.)
  • dirlist: Primitive directory lister, quicker than ls, find and echo *)
  • diskus: Disk geometry checking and repair tool)
  • getrealpath: Print realpath to stdout)
  • histogram: Count bytes in file)
  • kdmktone: Make the console beep)
  • keypressed: Nonblocking, nondestructible test for waiting data on TTYs, sockets and probably pipes)
  • killmem: Protect some memory against paging until you need free memory)
  • lockdir: Create a directory for locking purpose)
  • lockrun: Exclusively run something by placing a file lock)
  • md5chk: Create md5sums for easy shell usage)
  • minicron: This is a program which starts other programs after some time)
  • mvatom: Move files by atomic rename instead of copy.)
  • printansi: Like /bin/echo but ANSI-escapes the output)
  • printargs: Like “hello world” but dumps the argc array)
  • ptybuffer: daemonize interactive tty line driven programs with output history)
  • runningfor: Return true until the given time periode is reached.)
  • slowdown: Slowdown processes or pipes.)
  • socklinger: Execute quick hack shell scripts connected to a socket.)
  • sq: SQLITE3 query tool for shell usage)
  • timeout: Execute a command or pipe only for a given duration.)
  • timestart: Start a program N-M times in parallel by running it it each A-B seconds)
  • tinohtmlparse: Simple HTML parser to extract information from HTML files by shell)
  • tinoseq: An integer seq implementation)
  • udevraw: Dump udev events for bash usage)
  • unbuffered: Copy stdin to stdout and stderr, unbuffered)
  • watcher: A Python2.6 script to watch files, pipes or Unix domain sockets)

Unix pages (at this blog)

• Unix system administration – https://eikonal.wordpress.com/2011/03/02/unix-system-administration/
• Unix tricks – https://eikonal.wordpress.com/2011/02/15/unix-tricks/
• Unix hardening – https://eikonal.wordpress.com/2010/04/20/unix-hardening/
• Logging – https://eikonal.wordpress.com/2010/04/13/logging/
• PAM – https://eikonal.wordpress.com/2011/03/11/pam/
• SSH, OpenSSH – https://eikonal.wordpress.com/2010/12/16/ssh-openssh/
• Unix “script” command – https://eikonal.wordpress.com/2010/12/02/unix-script-command/
• vi editor – https://eikonal.wordpress.com/2010/11/15/vi-editor/
• sed tricks – https://eikonal.wordpress.com/2010/10/05/sed-tricks/
• Nmap options, switches and uses – https://eikonal.wordpress.com/2010/09/20/nmap-options-swtiches-and-uses/
• netcat – https://eikonal.wordpress.com/2010/07/12/netcat/
• Command line based text replace – https://eikonal.wordpress.com/2010/07/13/command-line-based-text-replace/
• Archiving and compression tools – https://eikonal.wordpress.com/2010/08/03/archiving-and-compression-tools/
• Cygwin – https://eikonal.wordpress.com/2010/07/12/cygwin-stuff/
• googlecl – https://eikonal.wordpress.com/2010/07/06/googlecl/
• John the Ripper – https://eikonal.wordpress.com/2010/05/25/john-the-ripper/

Unix system administration

• “The Decline and Fall of System Administration” (SlashDot; 2011.03.02) – http://tech.slashdot.org/story/11/03/02/132215/The-Decline-and-Fall-of-System-Administration
  “Deep End’s Paul Venezia questions whether server virtualization technologies are contributing to the decline of real server administration skills (http://infoworld.com/d/data-center/the-decline-and-fall-system-administration-375), as more and more sysadmins argue in favor of re-imaging as a solution to Unix server woes (http://tech.slashdot.org/story/11/02/21/1734259/Why-You-Shouldnt-Reboot-Unix-Servers). ‘This has always been the (many times undeserved) joke about clueless Windows admins: They have a small arsenal of possible fixes, and once they’ve exhausted the supply, they punt and rebuild the server from scratch rather than dig deeper. On the Unix side of the house, that concept has been met with derision since the dawn of time, but as Linux has moved into the mainstream — and the number of marginal Linux admins has grown — those ideas are suddenly somehow rational.'”
• Paul Venezia (InfoWorld) sysadmin articles:
  • “The decline and fall of system administration” (2011.02.28) – http://infoworld.com/d/data-center/the-decline-and-fall-system-administration-375
    Virtualization makes it all too easy to spawn new instances rather than figuring out what went wrong. Is this the end of Unix best practices?
  • “When in doubt, reboot? Not Unix boxes” (2011.02.21) – http://infoworld.com/t/unix/when-in-doubt-reboot-not-unix-boxes-061
    Rebooting Windows boxes is a way of life, but rebooting by default can you get you nowhere fast when running Unix.
  • “Nine traits of the veteran Unix admin” (I2011.02.21) – http://infoworld.com/t/unix/nine-traits-the-veteran-unix-admin-276
    Follow this field guide if you want to understand the rare and elusive hard-core Unix geek.

AIX

• AIX 5L Version 5.3: Commands Reference, Volume 5, s – u (SC23-4892-07; 2010.04) – http://publib.boulder.ibm.com/infocenter/pseries/v5r3/topic/com.ibm.aix.cmds/doc/aixcmds5/aixcmds5.htm
• AIX Tips – http://www.ee.pw.edu.pl/~pileckip/aix/
• “AIX commands you should not leave home without” by Shiv Dutta (IBM developerWorks; 2006.10.24) – http://www.ibm.com/developerworks/aix/library/au-dutta_cmds.html
• “AIX: Commands” by John Flecther – http://www.unixguide.net/ibm/aixcommands.shtml
• “AIX Command Crib Sheet” (2001.01.29) – http://www.unixguide.net/ibm/ibmcribsheet.shtml | http://www.mort.level5.net/johnr/howto/aix.txt
• AIX COMMAND GUIDE – http://bigcalm.tripod.com/aix/handycommands.htm
• Useful AIX commands – http://stromberg.dnsalias.org/~strombrg/Useful-AIX-commands.html
• IBM’s developerWorks collection of articles on AIX and Unix – http://www.ibm.com/developerworks/aix/
• IBM’s AIX and UNIX Forums – http://www.ibm.com/developerworks/forums/dw_auforums.jspa

Specific tasks:

• Getting OS version:
  • uname command – http://publib.boulder.ibm.com/infocenter/pseries/v5r3/topic/com.ibm.aix.cmds/doc/aixcmds5/uname.htm
  • “Displaying the operating system name (uname command)” – http://publib.boulder.ibm.com/infocenter/pseries/v5r3/topic/com.ibm.aix.baseadmn/doc/baseadmndita/displaying_os_name.htm
  • “AIX Tip of the Week: Using uname to Identify RS/6000 Model” (AIX Tips; 1999.02) – http://www.ee.pw.edu.pl/~pileckip/aix/lsconfig.htm
  • “AIX: How To Determine Its Detail Version” (SysAdmin’s Diary; 2008.02.24) – http://blog.irwan.name/?p=438
    • Use uname -a and oslevel

    From article “AIX commands you should not leave home without” (fully referenced above):

    • uname -a Displays the system name, nodename, version, machine ID.
    • uname -p Displays the chip type of the system. For example, PowerPC./li>
    • uname -r Displays the release number of the operating system./li>
    • uname -s Displays the system name. For example, AIX./li>
    • uname -n Displays the name of the node./li>
    • uname -M Displays the system model name. For example, IBM, 9114-275./li>
    • uname -v Displays the operating system version./li>
    • uname -m Displays the machine ID number of the hardware running the system./li>
    • uname -u Displays the system ID number./li>
• Finding open files:
  • “Finding open files with lsof” by Sean A. Walberg (IBM developerWorks; 2006.07.25) – http://www.ibm.com/developerworks/aix/library/au-lsof.html
  • lsof information:
    • lsof source – ftp://lsof.itap.purdue.edu/pub/tools/unix/lsof/
    • lsof man page – http://www.netadmintools.com/html/lsof.man.html

---

Related here: Unix tricks – https://eikonal.wordpress.com/2011/02/15/unix-tricks/

Cygwin stuff

Various Cygwin links

• Home – http://www.cygwin.com/ | http://cygwin.org/
  • FAQ – http://www.cygwin.com/faq/
  • Mirrors – http://www.cygwin.com/mirrors.html
  • Cygwin/X – http://x.cygwin.com/
• Cygwin info at RedHat – http://www.redhat.com/services/custom/cygwin/
• project pages at Freshmeat – http://freshmeat.net/projects/cygwin/
• Cygwin infor at Softpanorama – http://www.softpanorama.org/Unixification/cygwin.shtml
• Windows Unix compatibility – http://techdir.rutgers.edu/winunix.html
• at WikiPedia – http://en.wikipedia.org/wiki/Cygwin

Portable Cygwin

• “Cygwin on portable storage devices” by Ali Devin Sezer (2005.02.05) – http://sites.google.com/site/devinsezer/Home/software/portable-cygwin
• “A Guide to Building a Portable Cygwin” (at Fuzzix blog; not clear when)- http://blogs.linux.ie/fuzzbucket/2008/02/23/cygwin/
• (2008): http://www.symbiosoft.net/cygwinportable | http://sourceforge.net/projects/cygwinportable/files/ | http://portableapps.com/node/14482
• “Cygwin Portable – (some) Linux power on a USB for Windows” (Bits of Wizardry; 2010.01.29) – http://bitsofwizardry.wordpress.com/2010/01/29/cygwin-portable-some-linux-power-on-a-usb-for-windows/
• “How to make your own portable Cygwin on CD” by fergus bonhard (2003.07.19) – http://www.cygwin.com/ml/cygwin/2003-07/msg01117.html
• “A CD based on Cygwin for X Windows forwarding via ssh” – http://xlivecd.indiana.edu/

Tricks

Setting effective HOME to your Windows “My Documents” folder.

1. Start a Cygwin shell
2. You should be located in your default directory which inside Cygwin looks like /home/thisuser, but from the Windows standpoint could be something like c:\cygwin\home\thisuser.
3. Type ln -s /cygdrive/c/Documents and Settings/thisuser/My Documents/ ./docs
4. Edit .profile and .bashrc to put at their ends following lines:

   export HOME=./docs
   cd $HOME
     

5. that is it. Restart Cygwin. You should now be in the /home/thisuser/docs/ which is the same place as Windows’ c:\Documents and Settings\thisuser\My Documents/

---

Related: MS Windows Registry transversal by Cygwin – https://eikonal.wordpress.com/2011/03/01/registry-transversal-by-cygwin | Unix tricks – https://eikonal.wordpress.com/2011/02/15/unix-tricks/

John the Ripper

Places

• Home – http://www.openwall.com/john/
• Custom binary builds, by various users – http://openwall.info/wiki/john/custom-builds

Simple dictionary-based cracking

For Linux systems, the hashed passwords are contained in the /etc/shadow file. To use John the ripper, one needs both that file and /etc/passwd.

• Unshadowing:
  ./unshadow.exe passwd.txt shadow.txt > passwd-unshadowed.txt
• To run John against the unshadowed password file passwdFile-unshadowed.txt using the predefined word-list mywords.lst, run
  following:
  ./john.exe –wordlist=mywords.lst passwd-unshadowed.txt
• To see the cracked passwords run:
  ./john.exe –show passwdFile-unshadowed.txt
• and to save that file:
  ./john.exe –show passwdFile-unshadowed.txt > passwdFile-cracked.txt

Articles

• “Linux Password Cracking: Explain unshadow and john commands ( john the ripper tool )” by Vivek Gite (UnixCraft) – http://www.cyberciti.biz/faq/unix-linux-password-cracking-john-the-ripper/
• “Cracking OpenVMS passwords with John the Ripper” by Jean-loup Gailly – http://gailly.net/security/john-VMS-readme.html
  • John the Ripper v1.3.6 source package patched to include OpenVMS SYSUAF.DAT files parsing – http://gailly.net/security/john-VMS-readme.html. It compiles well on Cygwin.
• So far, John does not work for SHA hashes. A patch allowing one to do this is presented at:
  • “Re: “No password hashes loaded” on Ubuntu 9.04″ by Solar Designer (2009.09.02) – http://www.openwall.com/lists/john-users/2009/09/02/3
  • “Crack Password with John the Ripper on Ubuntu 9.10” by Junjun Mao (2010.02.01) – http://pka.engr.ccny.cuny.edu/~jmao/node/26

  This patch allows use of John against SSH type hashes, but requires running John on the same type of system (i.e. unix system that supports the same hashes in logon authentication module).

---

Related here: Default passwords, wordlist and Rainbow tables – https://eikonal.wordpress.com/2010/03/29/default-passwords/ | Passwords cracking – https://eikonal.wordpress.com/2010/01/06/password-crackers/

Gathering information on a Unix system

Test	Linux	AIX	HP-UX
Hardware			ioscan -v
Kernel parameter information			kmtune -l
Network Configuration parameters			ndd -h supported
Network and routing tables.			netstat -in netstat -rn
General machine information			uname -a
Raid Configuration			/sbin/irdiag -v
System Resources			sar -b <interval> <count>

Similar:

• Rosetta Stone for Unix can be found in several versions at many locations (Google is your friend ;-). Some are:
  • http://www.pafumi.net/Unix_Command_Translator.html (other resources at http://www.pafumi.net/ are also worth visting).
  • http://www.talisman.org/unix/rosetta.shtml (see also list of unix books http://www.talisman.org/unix/books.shtml and other files at that directory http://www.talisman.org/unix/
  • http://bhami.com/rosetta.html
  • http://www.scribd.com/doc/31467371/Rosetta-Stone-For-Unix
• UnixGuide has a similar table: http://www.unixguide.net/unixguide.shtml

Tools

• Config to HTML: http://come.to/cfg2html
• kmtune.pl – a Perl script wrapping kmtune: http://forums2.itrc.hp.com/service/forums/getattachment.do?attachmentId=4902&ext=.txt. (Local copy: https://eikonal.wordpress.com/2010/03/23/kmtune-pl.)

---

Sources:

• Scripts to collect unix configuration parameters (for HP-UX)

Unix links

• “The UNIX Programmer’s Manual” online: http://man.freetechsecrets.com/
• “The Unix-Haters Handbook”: http://www.cs.washington.edu/homes/weise/uhh-download.html
• “10 Tools To Add Some Spice To Your UNIX Shell Scripts” by Vivek Gite: http://www.cyberciti.biz/tips/spice-up-your-unix-linux-shell-scripts.html
• Rosetta Stone for Unix:
  • http://www.pafumi.net/Unix_Command_Translator.html
  • http://www.talisman.org/unix/rosetta.shtml
  • http://bhami.com/rosetta.html
  • http://www.scribd.com/doc/31467371/Rosetta-Stone-For-Unix
• http://www.talisman.org/unix/:
  • Rosetta stone: http://www.talisman.org/unix/rosetta.shtml
  • A list of unix books: http://www.talisman.org/unix/books.shtml
• UnixGuide’s table of commands across various unix flavors: http://www.unixguide.net/unixguide.shtml
• http://www.pafumi.net/
• UNIX Shell Scripting Articles (at SHELLdorado): http://www.shelldorado.com/articles/
• http://www.zazzybob.com/
• Hal Pomeranz – http://www.deer-run.com/~hal/
• LXer – http://lxer.com/ (Linux News: The world is talking about GNU/Linux and Free/Open Source Software)
• RootPrompt.org – an unix-themed wiki – http://rootprompt.org/
• Linux.org.uk by Alan Cox – http://www.linux.org.uk/
• LWN.net – http://lwn.net/
• Freshmeat – http://freshmeat.net/
• LinuxDevices – http://linuxdevices.com/
• DesktopLinux.com – http://www.desktoplinux.com/</li
• Linux Gazette – http://www.linuxgazette.net/
• Free Software Magazine – http://www.freesoftwaremagazine.com/
• OSDir.com @ O’Reilly Network – http://www.osdir.com/
• LinuxQuestions.org – http://www.linuxquestions.org/
• LinuxForums.org – http://www.linuxforums.org/
• LinuxHomepage.com – http://linuxhomepage.com/
• MobileTechNews.com – http://www.mobiletechnews.com/
• USENIX – http://www.usenix.org/
• “My SysAd” Blog – http://swik.net/Unix/My+SysAd+Blog
• *nixShell blog – http://nixshell.wordpress.com/

Unix humor

• “Nine traits of the veteran Unix admin” by Paul Venezia (InfoWorld; 2011.02.14) – http://www.infoworld.com/t/unix/nine-traits-the-veteran-unix-admin-276
  • “Common Traits of the Veteran Unix Admin” by Soulskill (SlashDot; 2011.02.14) – http://tech.slashdot.org/story/11/02/14/2357242/Common-Traits-of-the-Veteran-Unix-Admin
• Dilbert & Unix 2002-04-22 09:01am – http://www.perturb.org/display/entry/462/