Eikonal Blog

2011.02.15

Unix tricks

Filed under: unix — Tags: , , , , , , , , — sandokan65 @ 15:38

Sources:

System information

Check cpu info:

    cat /proc/cpuinfo
    

Usernames

To get the list of usernames with its user ID in formatted way:

     awk -F":" '{ print "username: " $1 "\t\tuid:" $3 }' /etc/passwd
    

Filenames

Find the particular string from the list of files in current directory:

    cd /etc
    for i in $(find -type f); do grep -iH nfsnobody $i; done
    

Or

    grep -iH nfsnobody *
    

Counting words

Get the no of occurrences of particular word in file:

    awk '/ServerName/ {i=i+1} END {print i}' /etc/httpd/conf/httpd.conf
    grep ServerName /etc/httpd/conf/httpd.conf
    

Semaphores

To delete resources of semaphore arrays from memory:

    ipcs -s | 
    grep apache | 
    perl -e 'while () { @a=split(/\s+/); print`ipcrm sem $a[1]`}'
    

Convering Unix timestamp to human readable format

All one needs here is neatly summarized by Anton Olson in his blog posting “BASH: Convert Unix Timestamp to a Date” – http://www.antonolsen.com/2006/04/06/bash-convert-unix-timestamp-to-a-date/:

  • 1) perl -e “require ‘ctime.pl’; print &ctime($EPOCH);” – where ctime is a Perl module (available where?)
  • 2) perl -e “print scalar(localtime($EPOCH))”
  • 3) echo $EPOCH|awk ‘{print strftime(“%c”,$1)}’
  • 4) # date -d @1000000042 on Linux (and Cygwin)
  • 5) $ date -d ’1970-01-01 sec’. It is important to use the GMT tag:
    • #date -d ’1970-01-01 1000000000 sec GMT’ with output: Sat Sep 8 20:46:40 CDT 2001
    • #date -d ’1970-01-01 1000000000 sec’ gives output: Sun Sep 9 02:46:40 CDT 2001
  • 6) $ date -r 1229519950 – on FreeBSD

The third method works fine if you have available only shell (e.g. working in Cygwin) and does not require Perl interpreter. For example:

    In: echo 101000070 | awk '{print strftime("%c",$1)}'
    Out: Wed Mar 14 19:34:30 1973
    

Perl

To check whether perl module is installed correctly or not: if all is correct then output of this command nothing

    perl -e 'require Mail::SPF::Query'
    

To install CPAN module:

    cpan
    cpan> install Mail::SPF::Query
    CPAN: Storable loaded ok
    Going to read /root/.cpan/Metadata
    Database was generated on Thu, 24 Nov 2005 14:54:20 GMT
    Mail::SPF::Query is up to date.
    

IP Addresses

To get the list of IP addresses in the server:

    ifconfig | 
    grep -vw inet6 | 
    grep -w inet | 
    cut -d : -f 2 | 
    cut -d \ -f 1
    

Find list of IP address along with eth device and network mask:

    ifconfig | 
    cut -d " " -f1,12,16 | 
    grep -A 1 eth | 
    tr -d - | 
    tr -s "\n" |sed -e :a -e N -e 's/\n/ /'
    

hard disk

Know the performance of your HardDisk: change the device address as per your servers configuration

    hdparm -Tt /dev/sda
    

Logging

Get the customized output of raw accesslog of httpd: Navigate the folder where your http access log reside, then execute following:

    tail -f access_log | 
    awk '{if ($11 ~"\"-\"") print $1, $7, $12; else print $1, $10, $11, $12}'
    

Checking SSH failure log (in GNU/Linux Debian)

    /bin/cat /var/log/auth.log  |
    grep sshd:auth |
    grep failure |
    awk '{print $1"-" $2 "-" $3 "-->" $12 "->" $14 "->" $15}'
    

Open connections

The details of the present http connections can be found by using:

    netstat -plan | 
    grep ":80 " | 
    awk {'print $5'} |
    awk -F: {'print $1'}|
    sort
    
    cat /proc/net/ip_conntrack | 
    grep "port=80" | 
    wc -l
    

Number of connection from the particular IP addfess:

    netstat -ntu | awk '{print $5}'| cut -d: -f1 | sort | uniq -c | sort -nr | more
    

No of conections:

    netstat -alntp
    /sbin/ldconfig /usr/local/lib - Update the system linker cache
    

Real Time Network Activity Examples:

    root# watch -d "netstat -nalp |grep -v DGRAM |grep -v STREAM |grep -v LISTEN"
    root# watch "netstat -nalp"|grep ":TCP PORT Number"
    root# watch "netstat -nalp"|grep ":22"
    

Port scanning

Port scanning using nmap:
You can customized it to get more informative output

    nmap -sS localhost -
    

instead host localhost, it could be IP address of another server which is in question

Bash loops

You can execute bash command a certain number of times by using something similar to the following:

    n=0;while test -$n -gt -10; do echo n=$n; n=$[$n+1]; done
    

that code will print “n=0”, “n=1”, and so on 10 times.

Directory content

Only get the listing of directories:

    ls -F $1 | 
    grep \/ | 
    sed -e 's/\/$/4/g'
    

Killing processes

Kill program one time click base keyword:

    for a in $( ps aux | 
    grep  "some-thing-text" | 
    awk '{ print $2 }'); do kill $a; done;
    

Kill crontab processes:

    for a in $( ps aux | grep  "/USR/SBIN/CRON" | awk '{ print $2 }'); do kill $a; done;
    

Environment variables

CDPATH env variable, if it is set for instance

    [bash ~]$ export CDPATH=/usr/local/apache
    

when you enter from any directory you’re in

    [bash /var/log]$ cd htdocs
    

it will take you to /usr/local/apache/htdocs.

Bash completion, command repetition and command history

  • !! at the command shell executes the last executed command
  • !m – executes the last command in history that starts with m for instance mail
  • !m:p – will print what was the last command that starts with m

vi

  • hitting twice z while holding a shift key in vi in command mode saves the edited file and quits vi

File dates

Updating atime on file foo (e.g. to 21:00 of Jan 1 1970):

    touch -t 197001012100 foo

help, man, apropos

Redirect a rather big man page to a text file in easy readable format:

    man manpage | col -b > manpage.txt

Conversion DOS to UNIX and vice versa

Getting rid off annoying ^M (CTRL-M) in DOS-like files:

    tr -d "15"  /unixfile

In vi editor use search-and-replace command:

    :%s/^M//g

To get ^M (in vi)you press CTRL+V+M


Related here: Unix system administration – https://eikonal.wordpress.com/2011/03/02/unix-system-administration/ | Cygwin stuff – https://eikonal.wordpress.com/2010/07/12/cygwin-stuff | MS Windows Registry transversal by Cygwin – https://eikonal.wordpress.com/2011/03/01/registry-transversal-by-cygwin

Related here: Scripting languages – https://eikonal.wordpress.com/2010/06/15/awk-sed/ | Unix tricks – https://eikonal.wordpress.com/2011/02/15/unix-tricks/ | SED tricks – https://eikonal.wordpress.com/2010/10/05/sed-tricks/ | Memory of things disappearing > nmap stuff > getports.awk – https://eikonal.wordpress.com/2010/06/23/memory-of-things-disappearing-nmap-stuff-getports-awk/ | AWK – https://eikonal.wordpress.com/2011/09/30/awk/

2010.07.13

Command line based text replace

sed

  • sed 's/Mark Monre/Marc Monroe/' 1.txt > 2.txt
  • find ./* -type f -exec sed -i 's///g' {} \;

The “replace” command

  • Syntax:
    replace OLD-STRING NEW-STRING OUTPUT-FILE
  • Example:
    $ replace UNIX Linux  newfile
  • Example:
    $ cat /etc/passwd | replace : '|'
  • Partial support for regular expressions: \^ – matches start of line, and $ matches end of line.
  • Example: replace all IP address 192.168.1.2 start of line:
    $ replace \^192.168.1.2 192.168.5.10  newfile
  • a bash script, ‘fixer.sh’
    #!/bin/bash
    replace CHANGEFROM CHANGETO $1.tmp
    rm $1
    mv $1.tmp $1
    

    now run this command line:

    $ grep CHANGEFROM |cut -d':' -f1 |xargs -n 1 fixer.sh

    the results is that all files in the directory (or whatever you grep for) will be changed automagically.
    just make sure the grep doesn’t include the fixer script itself, or it will die half-way through changing when execute permissions are reset!


Perl


Sources:


Related: Regular expressions – https://eikonal.wordpress.com/2010/04/02/regular-expressions/ | Perl online – https://eikonal.wordpress.com/2010/02/15/perl-online/

2010.04.06

FAQ makers/creators

Filed under: it, javascript — Tags: , , , — sandokan65 @ 15:30

2010.04.02

Regular expressions

Sites

Tools

Standalone tools:

Online testers:

Books

Tidbits

Sources: The above links.

  • [abc] – A single character: a, b or c
  • [^abc] – Any single character but a, b, or c
  • [a-z] – Any single character in the range a-z
  • [a-zA-Z] – Any single character in the range a-z or A-Z
  • ^ – Start of line
  • $ – End of line
  • \A – Start of string
  • \z – End of string
  • . – Any single character
  • \s – Any whitespace character
  • \S – Any non-whitespace character
  • \d – Any digit
  • \D – Any non-digit
  • \w – Any word character (letter, number, underscore)
  • \W – Any non-word character
  • \b – Any word boundary character
  • (…) – Capture everything enclosed
  • (a|b) – a or b
  • a? – Zero or one of a
  • a* – Zero or more of a
  • a+ – One or more of a
  • a{3} – Exactly 3 of a
  • a{3,} – 3 or more of a
  • a{3,6} – Between 3 and 6 of a
  • ^\s[ \t]*$ – Match a blank line
  • \d{2}-\d{5} – Validate an ID number consisting of 2 digits, a hyphen, and another 5 digits

Special common strings:

  • Personal Name: ^[\w\.\’]{2,}([\s][\w\.\’]{2,})+$
  • Username: ^[\w\d\_\.]{4,}$
  • Password at least 6 symbols: ^.{6,}$
  • Password or empty input: ^.{6,}$|^$
  • email: ^[\_]*([a-z0-9]+(\.|\_*)?)+@([a-z][a-z0-9\-]+(\.|\-*\.))+[a-z]{2,6}$
  • Email address: \b\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}\b[A-z0-9_.%+-]+@[A-z0-9_.%+-]+\.[A-z]{2,4}
  • US phone: \W?\d{3}\W?\d{3}\W?\d{4}
  • US Phone number: ^\+?[\d\s]{3,}$
  • US Phone with code: ^\+?[\d\s]+\(?[\d\s]{10,}$
  • URL: \W?\d{3}\W?\d{3}\W?\d{4}\b\w+://(\w|-|\.|/)+(/|\b)
  • US Social Security Number (SSN): \d{3}-\d{2}-\d{4}
  • US ZIP: \d{5}(-\d{4})?
  • IP (v4) address: \b\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}\b
  • IP (v4) address: \b(?:(?:25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\.){3}(?:25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\b
  • IP (v4) address: ^(\d|[1-9]\d|1\d\d|2[0-4]\d|25[0-5])\.(\d|[1-9]\d|1\d\d|2[0-4]\d|25[0-5]){3}$
  • IP (v4) address: \b(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\.(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\.(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\.(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\b
  • IP (v4) address: \b(?:(?:25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\.){3}(?:25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\b
  • IP (v6) address:
  • MAC address: ^([0-9a-fA-F][0-9a-fA-F]:){5}([0-9a-fA-F][0-9a-fA-F])$
  • Positive Integers: ^\d+$
  • Negative Integers: ^-\d+$
  • Integer: ^-{0,1}\d+$
  • Positive Number: ^\d*\.{0,1}\d+$
  • Negative Number: ^-\d*\.{0,1}\d+$
  • Positive Number or Negative Number: ^-{0,1}\d*\.{0,1}\d+$
  • Floating point number: [-+]?([0-9]*\.[0-9]+|[0-9]+)
  • Floating point number: [-+]?(?:\b[0-9]+(?:\.[0-9]*)?|\.[0-9]+\b)(?:[eE][-+]?[0-9]+\b)?
  • Roman number: ^(?i:(?=[MDCLXVI])((M{0,3})((C[DM])|(D?C{0,3}))?((X[LC])|(L?XX{0,2})|L)?((I[VX])|(V?(II{0,2}))|V)?))$
  • Domain Name: ^([a-zA-Z0-9]([a-zA-Z0-9\-]{0,61}[a-zA-Z0-9])?\.)+[a-zA-Z]{2,6}$
  • Domain Name: ^([a-z][a-z0-9\-]+(\.|\-*\.))+[a-z]{2,6}$
  • Windows File Name: (?i)^(?!^(PRN|AUX|CLOCK\$|NUL|CON|COM\d|LPT\d|\..*)(\..+)?$)[^\\\./:\*\?\”\|][^\\/:\*\?\”\|]{0,254}$
  • Date in format yyyy-MM-dd: (19|20)\d\d([- /.])(0[1-9]|1[012])\2(0[1-9]|[12][0-9]|3[01])
  • Date (dd mm yyyy, d/m/yyyy, etc.): ^([1-9]|0[1-9]|[12][0-9]|3[01])\D([1-9]|0[1-9]|1[012])\D(19[0-9][0-9]|20[0-9][0-9])$
  • Year 1900-2099: ^(19|20)[\d]{2,2}$

Related (here at this blog):
Command line based text replace – https://eikonal.wordpress.com/2010/07/13/command-line-based-text-replace/ |
Perl online – https://eikonal.wordpress.com/2010/02/15/perl-online/

2010.03.23

kmtune.pl

Filed under: unix, VA (Vulnerability Assessment) — Tags: , — sandokan65 @ 14:30

kmtune.pl – a Perl script wrapping kmtune: http://forums2.itrc.hp.com/service/forums/getattachment.do?attachmentId=4902&ext=.txt. Author: H.Merijn Brand. (Source: http://forums13.itrc.hp.com/service/forums/questionanswer.do?admit=109447627+1269354030577+28353475&threadId=939626).

Local copy:

#!/pro/bin/perl -w

use strict;
use integer;

if (@ARGV) {
    local $" = '/i || m/';
    eval "sub pat { local \$_ = shift; m/@ARGV/i }";
    }
else {
    eval "sub pat { 1 }"
    }

my (%tune, %parm, $PARM, $parm, %ref);

open my $list, "kmtune -l |";
while () {
    s/\s+$//;
    my ($p, $v) = split m/:\s+/, $_, 2 or next;
    $v =~ s/\b0X([\dA-Fa-f]+)\b/0x\L$1/g;
    $p eq "Parameter" and $parm = $v, next;

    $tune{$parm}{$p} = $v;

    $p eq "Value" or next;
    if ($v =~ m/^-?(0x[\da-f]+|\d+)$/) {
	$parm{uc $parm} = 0 + $v =~ m/^-?0x/ ? hex $v : $v;
	}
    else {
	#printf STDERR "%-20s: '%s'\n", $p, $v;
	$ref{$parm} = $v;
	}
    }
close $list;

while (keys %ref) {
    foreach my $p (keys %ref) {
	my $up = uc $p;
	my $v  = $tune{$p}{Value};
	#my @r = (m/\b([A-Za-z]\w*)\b/g);
	my $x = 0;
	eval q(
	    $v =~ s/\b([A-Za-z]\w*)\b/exists$parm{uc $1}?$parm{uc $1}:do{$x++,$1}/ge;
	    );
	$x and next;
	eval "\$v = $v";
	$parm{$up} = $v;
	delete $ref{$p};
	}
    }

$= = 64;
foreach $parm (sort keys %tune) {
    $tune{$parm}{Default} eq $tune{$parm}{Value} and $tune{$parm}{Default} = "";
    $PARM = uc $parm;
    pat ("$parm $parm{$PARM} $tune{$parm}{Value} $tune{$parm}{Default}\n") and
	write;
    }

format STDOUT_TOP =
Parameter            Value hex    Value dec   Function                    Default
-------------------- ------------ ----------- --------------------------- --------------------
.
format STDOUT =
@<<<<<<<<<<<<<<<<<<>>>>>>>>>> @>>>>>>>>>> ^<<<<<<<<<<<<<<<<<<<<<<<<<< ^<<<<<<<<<<<<<<<<<<<
$parm,sprintf("0x%010x",$parm{$PARM}),$parm{$PARM},$tune{$parm}{Value},$tune{$parm}{Default}
~~                                            ^<<<<<<<<<<<<<<<<<<<<<<<<<< ^<<<<<<<<<<<<<<<<<<<
					      $tune{$parm}{Value},        $tune{$parm}{Default}
.

2010.03.17

Cryptography resources

Sites

Historic cyphers

Hash algorithms

  • Passphrase Hashes – http://www.users.zetnet.co.uk/hopwood/crypto/scan/ph.html

    • Authenticators: When a passphrase is verified, the first few characters of the authenticator [= “magic”] determine which mechanism is used:
      • If the first three characters are “$1$”, MD5-crypt is used.
      • If the first four characters are “$2a$”, bcrypt is used.
      • If the first character is not “$” or “_”, Traditional-crypt3 is used.
  • The HashClash website – http://www.win.tue.nl/hashclash/ – hash algorithms collisions

RSA

  • export-a-crypto-system sig – http://www.cypherspace.org/rsa/, http://www.cypherspace.org/rsa/rsa-details.html – a Perl 3-line implementation of RSA encryptor and decryptor.
      #!/bin/perl -sp0777i<X+d*lMLa^*lN%0]dsXx++lMlN/dsM0<j]dsj 
      $/=unpack('H*',$_);$_=`echo 16dio\U$k"SK$/SM$n\EsN0p[lN*1 
      lK[d2%Sa2/d0$^Ixp"|dc`;s/\W//g;$_=pack('H*',/((..)*)$/)
        

    A 2-line version:

      print pack"C*",split/\D+/,`echo "16iII*o\U@{$/=$z;[(pop,pop,unpack"H*", 
      )]}\EsMsKsN0[lN*1lK[d2%Sa2/d0<X+d*lMLa^*lN%0]dsXx++lMlN/dsM0<J]dsJxp"|dc`
        
    • Use:
      • Encryption: echo “squeamish ossifrage” | rsa -k=10001 -n=1967cb529 > msg.rsa
      • Decryption: % rsa -d -k=ac363601 -n=1967cb529 < msg.rsa
    • requires GNU dc (http://www.cypherspace.org/rsa/dc.html)
    • .

2010.02.15

Perl online

Hashes

Files

Chomp()

Control structures

Tidbits

Rename files

Alex Batko says (at http://www.cs.mcgill.ca/~abatko/computers/programming/perl/):

Here is a brilliant program for renaming one or more files according to a specified Perl expression. I found it on page 706 of Programming Perl (3rd edition).

#!/usr/bin/perl
$op = shift;
for( @ARGV ) {
    $was = $_;
    eval $op;
    die if $@;
    rename( $was, $_ ) unless $was eq $_;
}

In the code above, the second last line calls the built-in function “rename”, not the program itself (which is named “rename.pl”). Below are a few examples of use.

% rename.pl 's/\.htm/\.html/' *.htm         # append an 'l'
% rename.pl '$_ .= ".old"' *.html           # append '.old'
% rename.pl 'tr/A-Z/a-z/' *.HTML            # lowercase
% rename.pl 'y/A-Z/a-z/ unless /^Make/' *   # lowercase

Printing hashes

Starting with an input file with data in two columns separated by coma (,):

#/bin/perl -t

my %TempHash = ();
my $InputFile = shift;
print "Input file = ",$InputFile,"\n";

my ($line,$column1,$column2,);

#reading input file to generate hash
open (INPUTSTREAM, '<',  $InputFile) || die ("Could not open $InputFile");
while ( $line =  ) {
	chomp;
        #print $line;
	($column1, $column2) = split ',', $line;
        $TempHash{$column1}=$column2;
        #print $column1," ==> ",$TempHash{$column1};
}
close (INPUTSTREAM);

## printing hash - way #1
print "The following are in the DB: ",join(', ',values %TempHash),"\n";

## printing hash - way #2
while (($key, $value) = each %TempHash)
{
     print "$key ==> $value";
}

## printing hash - way #3
foreach $key (sort keys %TempHash){
   print "$key ==> $TempHash{$key}";
}

Removing white spaces

Sources:

# Declare the subroutines
sub trim($);
sub ltrim($);
sub rtrim($);

# Perl trim function to remove whitespace from the start and end of the string
sub trim($)
{
	my $string = shift;
	$string =~ s/^\s+//;
	$string =~ s/\s+$//;
	return $string;
}
# Left trim function to remove leading whitespace
sub ltrim($)
{
	my $string = shift;
	$string =~ s/^\s+//;
	return $string;
}
# Right trim function to remove trailing whitespace
sub rtrim($)
{
	my $string = shift;
	$string =~ s/\s+$//;
	return $string;
}

# Here is how to output the trimmed text "Hello world!"
print trim($string)."\n";
print ltrim($string)."\n";
print rtrim($string)."\n";


Related: Regular Expressions – https://eikonal.wordpress.com/2010/04/02/regular-expressions/ | Command line based text replace – https://eikonal.wordpress.com/2010/07/13/command-line-based-text-replace/

2010.01.28

Cisco “password 7” decryption – Perl code

Filed under: infosec — Tags: , , , — sandokan65 @ 17:19

Source: somewhere from the web.

#!/usr/bin/perl -w
# $Id: ios7decrypt.pl,v 1.1 1998/01/11 21:31:12 mesrik Exp $
#
# Credits for orginal code and description hobbit@avian.org,
# SPHiXe, .mudge et al. and for John Bashinski 
# for Cisco IOS password encryption facts.
#
# Use for any malice or illegal purposes strictly prohibited!
#

@xlat = ( 0x64, 0x73, 0x66, 0x64, 0x3b, 0x6b, 0x66, 0x6f, 0x41,
          0x2c, 0x2e, 0x69, 0x79, 0x65, 0x77, 0x72, 0x6b, 0x6c,
          0x64, 0x4a, 0x4b, 0x44, 0x48, 0x53 , 0x55, 0x42 );

while () {
        if (/(password|md5)\s+7\s+([\da-f]+)/io) {
            if (!(length($2) & 1)) {
                $ep = $2; $dp = "";
                ($s, $e) = ($2 =~ /^(..)(.+)/o);
                for ($i = 0; $i < length($e); $i+=2) {
                    $dp .= sprintf "%c",hex(substr($e,$i,2))^$xlat[$s++];
                }
                s/7\s+$ep/$dp/;
            }
        }
        print;
}


Related: https://eikonal.wordpress.com/2010/05/21/cisco-%e2%80%9cpassword-7%e2%80%b3/

Blog at WordPress.com.